2023-10-08T00:37:30.443 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-08T00:37:30.443 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --verbose --name yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151 -- /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/orig.config.yaml 2023-10-08T00:37:30.488 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T00:37:30.572 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151 branch: wip-yuri-testing-2023-10-06-0949-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7418151' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-10-06-0949-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-07-1508-quincy sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 owner: scheduled_yuriw@teuthology priority: 74 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8174 sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-07-1508-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: abf19fc7a04fe4fceab4e98d4013ae1da7c06006 targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB0cPAV/RG2BkvrxzZHnNdQyZ/3qQ2le4eTUMsncY32AFChDBT4VBnZ39y8jKIMU159V4Mr8ZGHz43X4cptSBhw= smithi184.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHKxQN3H142f/b14dRMLiTxs+qYVY/Qjhf0KIIvtk8nU49zt4ybtEXjzkUV3+nkQpiaJ0GXj9y8cgd6QJbyB3X8= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-07_22:15:20 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-08T00:37:30.573 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa; will attempt to use it 2023-10-08T00:37:30.574 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa/tasks 2023-10-08T00:37:30.574 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-08T00:37:30.575 INFO:teuthology.task.internal:Checking packages... 2023-10-08T00:37:30.599 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '08cac0187919fc3a247efdf445f652e2fb30a647' 2023-10-08T00:37:30.599 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-08T00:37:30.599 INFO:teuthology.packaging:ref: None 2023-10-08T00:37:30.600 INFO:teuthology.packaging:tag: None 2023-10-08T00:37:30.600 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-06-0949-quincy 2023-10-08T00:37:30.600 INFO:teuthology.packaging:sha1: 08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:37:30.600 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-06-0949-quincy 2023-10-08T00:37:30.830 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1320.g08cac018 2023-10-08T00:37:30.830 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-08T00:37:30.837 INFO:teuthology.task.internal:no buildpackages task found 2023-10-08T00:37:30.838 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-08T00:37:30.861 INFO:teuthology.task.internal:Saving configuration 2023-10-08T00:37:30.878 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-08T00:37:30.885 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T00:37:30.913 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 00:32:16.316855', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB0cPAV/RG2BkvrxzZHnNdQyZ/3qQ2le4eTUMsncY32AFChDBT4VBnZ39y8jKIMU159V4Mr8ZGHz43X4cptSBhw='} 2023-10-08T00:37:30.935 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 00:32:16.318299', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHKxQN3H142f/b14dRMLiTxs+qYVY/Qjhf0KIIvtk8nU49zt4ybtEXjzkUV3+nkQpiaJ0GXj9y8cgd6QJbyB3X8='} 2023-10-08T00:37:30.935 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-08T00:37:31.068 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-08T00:37:31.134 INFO:teuthology.task.internal:roles: ubuntu@smithi184.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-08T00:37:31.134 INFO:teuthology.run_tasks:Running task console_log... 2023-10-08T00:37:31.207 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f94ae875b80>, signals=[15]) 2023-10-08T00:37:31.209 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-08T00:37:31.216 INFO:teuthology.task.internal:Opening connections... 2023-10-08T00:37:31.216 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T00:37:31.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:37:31.286 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T00:37:31.287 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:37:31.354 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-08T00:37:31.361 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2023-10-08T00:37:31.378 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2023-10-08T00:37:31.378 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-10-08T00:37:31.433 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="8.4 (Ootpa)" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:ID="rhel" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="fedora" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="8.4" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el8" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T00:37:31.434 INFO:teuthology.orchestra.run.smithi119.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T00:37:31.435 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T00:37:31.435 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:37:31.435 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T00:37:31.435 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-08T00:37:31.435 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T00:37:31.435 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-08T00:37:31.436 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-10-08T00:37:31.455 DEBUG:teuthology.orchestra.run.smithi184:> uname -m 2023-10-08T00:37:31.471 INFO:teuthology.orchestra.run.smithi184.stdout:x86_64 2023-10-08T00:37:31.472 DEBUG:teuthology.orchestra.run.smithi184:> cat /etc/os-release 2023-10-08T00:37:31.527 INFO:teuthology.orchestra.run.smithi184.stdout:NAME="Red Hat Enterprise Linux" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION="8.4 (Ootpa)" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:ID="rhel" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:ID_LIKE="fedora" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:VERSION_ID="8.4" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:PLATFORM_ID="platform:el8" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:ANSI_COLOR="0;31" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:HOME_URL="https://www.redhat.com/" 2023-10-08T00:37:31.528 INFO:teuthology.orchestra.run.smithi184.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-10-08T00:37:31.529 INFO:teuthology.orchestra.run.smithi184.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-08T00:37:31.529 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:37:31.529 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-08T00:37:31.529 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-10-08T00:37:31.529 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-10-08T00:37:31.529 INFO:teuthology.orchestra.run.smithi184.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-10-08T00:37:31.529 INFO:teuthology.lock.ops:Updating smithi184.front.sepia.ceph.com on lock server 2023-10-08T00:37:31.554 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-08T00:37:31.565 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-08T00:37:31.573 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-08T00:37:31.573 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2023-10-08T00:37:31.576 DEBUG:teuthology.orchestra.run.smithi184:> test '!' -e /home/ubuntu/cephtest 2023-10-08T00:37:31.591 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-08T00:37:31.598 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-08T00:37:31.598 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2023-10-08T00:37:31.632 DEBUG:teuthology.orchestra.run.smithi184:> test -z $(ls -A /var/lib/ceph) 2023-10-08T00:37:31.646 INFO:teuthology.orchestra.run.smithi119.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-08T00:37:31.648 INFO:teuthology.orchestra.run.smithi184.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-10-08T00:37:31.649 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-08T00:37:31.702 INFO:teuthology.run_tasks:Running task kernel... 2023-10-08T00:37:31.718 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-10-08T00:37:31.719 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-10-08T00:37:31.719 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-10-08T00:37:31.719 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T00:37:31.720 DEBUG:teuthology.orchestra.run.smithi184:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-08T00:37:31.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:37:31.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-08T00:37:31.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:37:31.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2023-10-08T00:37:31.911 DEBUG:teuthology.task.kernel:sha1 for client.0 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T00:37:31.911 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-08T00:37:31.911 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-10-08T00:37:31.928 INFO:teuthology.orchestra.run.smithi119.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-08T00:37:31.928 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T00:37:31.928 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-08T00:37:31.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2023-10-08T00:37:31.965 DEBUG:teuthology.task.kernel:sha1 for client.1 is fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T00:37:31.965 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c"... 2023-10-08T00:37:31.966 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2023-10-08T00:37:31.981 INFO:teuthology.orchestra.run.smithi184.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-10-08T00:37:31.982 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.25.1.el8_4.x86_64 vs fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T00:37:31.982 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-10-08T00:37:31.982 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum upgrade -y linux-firmware 2023-10-08T00:37:33.299 INFO:teuthology.orchestra.run.smithi119.stdout:Updating Subscription Management repositories. 2023-10-08T00:37:33.299 INFO:teuthology.orchestra.run.smithi119.stdout:Unable to read consumer identity 2023-10-08T00:37:33.407 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-08T00:37:33.409 INFO:teuthology.orchestra.run.smithi184.stdout:Unable to read consumer identity 2023-10-08T00:37:34.651 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2023-10-08T00:37:34.796 INFO:teuthology.orchestra.run.smithi184.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-10-08T00:37:35.154 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 34 MB/s | 16 MB 00:00 2023-10-08T00:37:36.173 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-10-08T00:37:38.378 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2023-10-08T00:37:39.273 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-10-08T00:37:39.274 INFO:teuthology.orchestra.run.smithi119.stdout:Nothing to do. 2023-10-08T00:37:39.274 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-10-08T00:37:39.385 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-08T00:37:39.386 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T00:37:39.462 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 4.7 MB/s | 24 kB 00:00 2023-10-08T00:37:39.524 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-08T00:37:39.524 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-08T00:37:39.527 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2023-10-08T00:37:40.397 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-10-08T00:37:40.398 INFO:teuthology.orchestra.run.smithi184.stdout:Nothing to do. 2023-10-08T00:37:40.398 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-10-08T00:37:40.531 INFO:teuthology.task.kernel:Downloading kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-08T00:37:40.531 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c 2023-10-08T00:37:40.747 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ 2023-10-08T00:37:40.747 DEBUG:teuthology.orchestra.run.smithi184:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/ --input-file=- 2023-10-08T00:37:40.750 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi184.front.sepia.ceph.com to complete... 2023-10-08T00:37:56.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08 00:37:56 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-08T00:37:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2023-10-08 00:37:56-- 2023-10-08T00:37:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 17s 2023-10-08T00:37:56.869 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 363M in 17s (21.3 MB/s) 2023-10-08T00:37:56.870 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.0... 2023-10-08T00:37:56.871 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-08T00:37:58.609 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2023-10-08T00:37:59.063 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2023-10-08T00:37:59.189 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2023-10-08T00:38:04.860 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08 00:38:04 URL:https://3.chacra.ceph.com/r/kernel/testing/fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c/centos/8/flavors/default/x86_64/kernel-6.5.0_rc7_gfef64f93da2b-1.x86_64.rpm [380759012/380759012] -> "/tmp/kernel.x86_64.rpm" [1] 2023-10-08T00:38:04.860 INFO:teuthology.orchestra.run.smithi184.stderr:FINISHED --2023-10-08 00:38:04-- 2023-10-08T00:38:04.860 INFO:teuthology.orchestra.run.smithi184.stderr:Total wall clock time: 24s 2023-10-08T00:38:04.860 INFO:teuthology.orchestra.run.smithi184.stderr:Downloaded: 1 files, 363M in 24s (15.4 MB/s) 2023-10-08T00:38:04.861 INFO:teuthology.task.kernel:Installing kernel fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c on client.1... 2023-10-08T00:38:04.862 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-10-08T00:38:06.619 INFO:teuthology.orchestra.run.smithi184.stdout:Verifying... ######################################## 2023-10-08T00:38:07.044 INFO:teuthology.orchestra.run.smithi184.stdout:Preparing... ######################################## 2023-10-08T00:38:07.176 INFO:teuthology.orchestra.run.smithi184.stdout:Updating / installing... 2023-10-08T00:38:37.349 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-08T00:38:45.200 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-6.5.0_rc7_gfef64f93da2b-1 ######################################## 2023-10-08T00:39:00.359 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-08T00:39:00.359 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-08T00:39:00.422 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-08T00:39:00.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-08T00:39:00.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-08T00:39:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-08T00:39:00.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-08T00:39:00.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-08T00:39:00.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-08T00:39:00.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-08T00:39:00.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-08T00:39:00.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-08T00:39:00.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-08T00:39:00.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-08T00:39:00.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-08T00:39:00.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-08T00:39:00.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-08T00:39:00.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-08T00:39:00.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-08T00:39:00.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-08T00:39:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-08T00:39:00.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-08T00:39:00.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-08T00:39:00.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-08T00:39:00.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-08T00:39:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-08T00:39:00.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-08T00:39:00.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-08T00:39:00.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-08T00:39:00.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-08T00:39:00.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-08T00:39:00.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-08T00:39:00.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-08T00:39:00.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-08T00:39:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-08T00:39:00.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-08T00:39:00.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-08T00:39:00.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-08T00:39:00.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-08T00:39:00.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-08T00:39:00.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-08T00:39:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-08T00:39:00.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-08T00:39:00.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-08T00:39:00.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-08T00:39:00.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-08T00:39:00.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-08T00:39:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-08T00:39:00.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-08T00:39:00.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-08T00:39:00.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-08T00:39:00.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-08T00:39:00.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-08T00:39:00.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-08T00:39:00.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-08T00:39:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-08T00:39:00.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-08T00:39:00.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-08T00:39:00.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-08T00:39:00.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-08T00:39:00.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-08T00:39:00.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-08T00:39:00.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-08T00:39:00.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-08T00:39:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-08T00:39:00.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-08T00:39:00.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-08T00:39:00.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-08T00:39:00.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-08T00:39:00.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-08T00:39:00.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-08T00:39:00.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-08T00:39:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-08T00:39:00.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-08T00:39:00.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-08T00:39:00.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-08T00:39:00.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-08T00:39:00.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-08T00:39:00.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-08T00:39:00.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-08T00:39:00.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-08T00:39:00.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-08T00:39:00.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-08T00:39:00.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-08T00:39:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-08T00:39:00.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-08T00:39:00.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-08T00:39:00.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-08T00:39:00.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-08T00:39:00.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-08T00:39:00.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-08T00:39:00.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-08T00:39:00.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-08T00:39:00.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-08T00:39:00.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-08T00:39:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-08T00:39:00.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-08T00:39:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-08T00:39:00.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-08T00:39:00.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-08T00:39:00.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-08T00:39:00.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-08T00:39:00.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-08T00:39:00.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-08T00:39:00.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-08T00:39:00.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-08T00:39:00.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-08T00:39:00.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-08T00:39:00.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-08T00:39:00.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-08T00:39:00.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-08T00:39:00.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-08T00:39:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-08T00:39:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-08T00:39:00.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-08T00:39:00.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-08T00:39:00.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-08T00:39:00.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-08T00:39:00.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-08T00:39:00.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-08T00:39:00.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-08T00:39:00.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-08T00:39:00.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-08T00:39:00.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-08T00:39:00.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-08T00:39:00.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-08T00:39:00.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-08T00:39:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-08T00:39:00.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-08T00:39:00.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-08T00:39:00.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-08T00:39:00.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-08T00:39:00.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-08T00:39:00.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-08T00:39:00.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-08T00:39:00.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-08T00:39:00.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-08T00:39:00.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-08T00:39:00.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-08T00:39:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-08T00:39:00.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-08T00:39:00.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-08T00:39:00.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-08T00:39:00.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-08T00:39:00.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-08T00:39:00.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-08T00:39:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-08T00:39:00.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-08T00:39:00.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-08T00:39:00.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-08T00:39:00.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-08T00:39:00.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-08T00:39:00.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-08T00:39:00.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-08T00:39:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-08T00:39:00.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-08T00:39:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-08T00:39:00.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-08T00:39:00.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-08T00:39:00.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-08T00:39:00.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-08T00:39:00.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-08T00:39:00.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-08T00:39:00.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-08T00:39:00.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-08T00:39:00.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-08T00:39:00.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-08T00:39:00.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-08T00:39:00.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-08T00:39:00.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-08T00:39:00.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-08T00:39:00.584 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.584 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-08T00:39:00.606 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2023-10-08T00:39:00.606 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-08T00:39:00.606 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T00:39:00.606 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-10-08T00:39:00.607 INFO:teuthology.orchestra.run.smithi119.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-08T00:39:00.608 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-08T00:39:00.608 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.608 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-10-08T00:39:00.609 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2023-10-08T00:39:00.681 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2023-10-08T00:39:00.681 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2023-10-08T00:39:00.681 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.02 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 99.el8_4.1 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Group : System Environment/Base 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 9264631 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-08T00:39:00.682 INFO:teuthology.orchestra.run.smithi119.stdout:Relocations : (not relocatable) 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : Red Hat, Inc. 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : Red Hat, Inc. 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:39:00.683 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2023-10-08T00:39:00.684 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-08T00:39:00.684 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:00.685 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-08T00:39:04.780 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2023-10-08T00:39:06.934 INFO:teuthology.orchestra.run.smithi119.stderr:done 2023-10-08T00:39:06.936 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-10-08T00:39:06.950 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.72ooG49tV3 2023-10-08T00:39:06.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.72ooG49tV3 2023-10-08T00:39:07.028 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.72ooG49tV3 2023-10-08T00:39:07.196 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.72ooG49tV3 is 6KB 2023-10-08T00:39:07.244 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.72ooG49tV3 2023-10-08T00:39:07.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2023-10-08T00:39:07.328 INFO:teuthology.orchestra.run.smithi119.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-08T00:39:07.328 INFO:teuthology.orchestra.run.smithi119.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-08T00:39:07.328 INFO:teuthology.orchestra.run.smithi119.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-08T00:39:07.328 INFO:teuthology.orchestra.run.smithi119.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-08T00:39:07.328 INFO:teuthology.orchestra.run.smithi119.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-08T00:39:07.329 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:07.494 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2023-10-08T00:39:08.462 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi184.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-10-08T00:39:08.463 DEBUG:teuthology.orchestra.run.smithi184:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/config-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/boot/vmlinuz-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto 2023-10-08T00:39:08.517 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crc32c-intel.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-10-08T00:39:08.518 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-avx2.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce 2023-10-08T00:39:08.519 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-amd.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm-intel.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/arch/x86/kvm/kvm.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ansi_cprng.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/anubis.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/arc4.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_memcpy.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_pq.ko 2023-10-08T00:39:08.520 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_raid6_recov.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_tx.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/async_xor.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/async_tx/raid6test.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authenc.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/authencesn.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blake2b_generic.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_common.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/blowfish_generic.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/camellia_generic.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast5_generic.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast6_generic.ko 2023-10-08T00:39:08.521 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cast_common.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/ccm.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cmac.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crc32_generic.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_engine.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/crypto_user.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/cts.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/des_generic.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/dh_generic.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/echainiv.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/essiv.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/fcrypt.ko 2023-10-08T00:39:08.522 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/khazad.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/lz4hc.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/md4.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/michael_mic.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcbc.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/pcrypt.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/rmd160.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/seed.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/serpent_generic.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tcrypt.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/tea.ko 2023-10-08T00:39:08.523 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_common.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/twofish_generic.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/vmac.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/wp512.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xcbc.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xor.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/crypto/xxhash_generic.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_ipmi.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/acpi_pad.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/custom_method.ko 2023-10-08T00:39:08.524 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/ec_sys.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/platform_profile.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbs.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/sbshc.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/acpi/video.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/acard-ahci.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ahci_platform.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/ata_generic.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/libahci_platform.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_acpi.ko 2023-10-08T00:39:08.525 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ali.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_amd.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_artop.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atiixp.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_atp867x.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd640.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cmd64x.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_cypress.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_efar.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt366.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt37x.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x2n.ko 2023-10-08T00:39:08.526 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_hpt3x3.ko 2023-10-08T00:39:08.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it8213.ko 2023-10-08T00:39:08.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_it821x.ko 2023-10-08T00:39:08.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_jmicron.ko 2023-10-08T00:39:08.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_marvell.ko 2023-10-08T00:39:08.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_mpiix.ko 2023-10-08T00:39:08.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_netcell.ko 2023-10-08T00:39:08.527 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ninja32.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87410.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_ns87415.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_oldpiix.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_opti.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_optidma.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pcmcia.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc2027x.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_piccolo.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_rdc.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sch.ko 2023-10-08T00:39:08.528 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_serverworks.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sil680.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sis.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_sl82c105.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_triflex.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pata_via.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/pdc_adma.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_inic162x.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_mv.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_nv.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_promise.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_qstor.ko 2023-10-08T00:39:08.529 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sil24.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sis.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_svw.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_sx4.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_uli.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_via.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ata/sata_vsc.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/auxdisplay/ks0108.ko 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap 2023-10-08T00:39:08.530 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-i2c.ko 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/base/regmap/regmap-mmio.ko 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/bcma/bcma.ko 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/aoe/aoe.ko 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/brd.ko 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/drbd/drbd.ko 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/floppy.ko 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/loop.ko 2023-10-08T00:39:08.531 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/nbd.ko 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/null_blk/null_blk.ko 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/pktcdvd.ko 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/rbd.ko 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/virtio_blk.ko 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/xen-blkfront.ko 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram 2023-10-08T00:39:08.532 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/block/zram/zram.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hangcheck-timer.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/amd-rng.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/intel-rng.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/via-rng.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/hw_random/virtio-rng.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-10-08T00:39:08.533 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_si.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/lp.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/mwave/mwave.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/ppdev.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tlclk.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_atmel.ko 2023-10-08T00:39:08.534 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_infineon.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_nsc.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/uv_mmtimer.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/char/virtio_console.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/p4-clockmod.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/powernow-k8.ko 2023-10-08T00:39:08.535 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/cpufreq/speedstep-lib.ko 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/ccp/ccp.ko 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-aes.ko 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/padlock-sha.ko 2023-10-08T00:39:08.536 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dca/dca.ko 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/dw/dw_dmac.ko 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/dma/ioat/ioatdma.ko 2023-10-08T00:39:08.537 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/amd64_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/e752x_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/edac_mce_amd.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3000_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i3200_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5100_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i5400_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7300_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i7core_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/i82975x_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/ie31200_edac.ko 2023-10-08T00:39:08.538 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/sb_edac.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/edac/x38_edac.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/edd.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/firmware/iscsi_ibft.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-ich.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/gpio/gpio-viperboard.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-appleir.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-aureal.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-axff.ko 2023-10-08T00:39:08.539 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-dr.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elecom.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-elo.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-emsff.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gaff.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gt683r.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-gyration.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-kbd.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtek-mouse.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-holtekff.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-hyperv.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-icade.ko 2023-10-08T00:39:08.540 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-keytouch.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-kye.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lcpower.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-led.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-lenovo.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-dj.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-multitouch.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-ortek.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-penmount.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-petalynx.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-picolcd.ko 2023-10-08T00:39:08.541 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-pl.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-plantronics.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-primax.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-rmi.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-arvo.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-common.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-isku.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kone.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-konepure.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-lua.ko 2023-10-08T00:39:08.542 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-pyra.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-ryos.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat-savu.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-roccat.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-saitek.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-samsung.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sensor-hub.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sjoy.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sony.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-speedlink.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-steelseries.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-sunplus.ko 2023-10-08T00:39:08.543 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-thrustmaster.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tivo.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-tmff.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-topseed.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-twinhan.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-uclogic.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-waltop.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-wiimote.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-xinmo.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zpff.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/hid-zydacron.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/uhid.ko 2023-10-08T00:39:08.544 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hid/wacom.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_balloon.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_utils.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hv/hv_vmbus.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/abituguru3.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/acpi_power_meter.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7414.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ad7418.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adc128d818.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1025.ko 2023-10-08T00:39:08.545 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1026.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1029.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm1031.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adm9240.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ads7828.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7410.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7411.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7462.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7470.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7475.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/adt7x10.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/amc6821.ko 2023-10-08T00:39:08.546 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/applesmc.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asb100.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asc7621.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/asus_atk0110.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/atxp1.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/coretemp.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/dme1737.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds1621.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ds620.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc1403.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/emc6w201.ko 2023-10-08T00:39:08.547 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71805f.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f71882fg.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/f75375s.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fam15h_power.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/fschmd.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g760a.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/g762.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl518sm.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/gl520sm.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/hwmon-vid.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5500_temp.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/i5k_amb.ko 2023-10-08T00:39:08.548 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmaem.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ibmpex.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina209.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ina2xx.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/it87.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k10temp.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/k8temp.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lineage-pem.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm63.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm73.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm75.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm77.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm78.ko 2023-10-08T00:39:08.549 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm80.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm83.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm85.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm87.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm90.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm92.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm93.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95234.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95241.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/lm95245.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc2945.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4151.ko 2023-10-08T00:39:08.550 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4215.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4222.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4245.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4260.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/ltc4261.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max16065.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1619.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max1668.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max197.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6639.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6650.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/max6697.ko 2023-10-08T00:39:08.551 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/mcp3021.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6683.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775-core.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct6775.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/nct7802.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87360.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pc87427.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pcf8591.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max16064.ko 2023-10-08T00:39:08.552 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max34440.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/max8688.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/powr1220.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5627.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch5636.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sch56xx-common.ko 2023-10-08T00:39:08.553 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht15.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sht21.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/shtc1.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/sis5595.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47b397.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m1.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/smsc47m192.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/thmc50.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp102.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp103.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp401.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/tmp421.ko 2023-10-08T00:39:08.554 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via-cputemp.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/via686a.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt1211.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/vt8231.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627ehf.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83627hf.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83781d.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83791d.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83792d.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83793.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83795.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l785ts.ko 2023-10-08T00:39:08.555 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/hwmon/w83l786ng.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-10-08T00:39:08.556 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-i801.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-isch.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-parport.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-10-08T00:39:08.557 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-via.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-dev.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-mux.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-smbus.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/i2c/i2c-stub.ko 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core 2023-10-08T00:39:08.558 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_cm.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_core.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_umad.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/iw_cm.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_cm.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-10-08T00:39:08.559 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-10-08T00:39:08.560 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/fm801-gp.ko 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/gameport.ko 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/lightning.ko 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/gameport/ns558.ko 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/joydev.ko 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/keyboard/gpio_keys.ko 2023-10-08T00:39:08.561 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/appletouch.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/bcm5974.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/cyapatp.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/elan_i2c.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/sermouse.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/synaptics_usb.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/mouse/vsxxxaa.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/rmi4/rmi_core.ko 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio 2023-10-08T00:39:08.562 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/altera_ps2.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/arc_ps2.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/serio/serio_raw.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/input/sparse-keymap.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/iommu/amd/iommu_v2.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-blinkm.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lm3530.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-lp3944.ko 2023-10-08T00:39:08.563 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/leds-ss4200.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache 2023-10-08T00:39:08.564 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/bcache/bcache.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-bio-prison.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache-smq.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-cache.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-crypt.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-delay.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-flakey.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-log-userspace.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-multipath.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-queue-length.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-raid.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-round-robin.ko 2023-10-08T00:39:08.565 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-service-time.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-switch.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-thin-pool.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/dm-verity.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/faulty.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/linear.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/multipath.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid0.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid1.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid10.ko 2023-10-08T00:39:08.566 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/md/raid456.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptbase.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptctl.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptfc.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptlan.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptsas.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptscsih.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/message/fusion/mptspi.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_ich.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/lpc_sch.ko 2023-10-08T00:39:08.567 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/mfd-core.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/sm501.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/viperboard.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/vx855.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/mfd/wl1273-core.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds9802als.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/apds990x.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/bh1770glc.ko 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710 2023-10-08T00:39:08.568 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/cb710/cb710.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/at24.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/eeprom/max6875.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/enclosure.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/hpilo.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29003.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/isl29020.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-10-08T00:39:08.569 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-me.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei-txe.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/mei/mei.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-gru/gru.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xp.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpc.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_7xx1.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tifm_core.ko 2023-10-08T00:39:08.570 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/tsl2550.ko 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_balloon.ko 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/bonding/bonding.ko 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/b53/b53_common.ko 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/bcm-sf2.ko 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6060.ko 2023-10-08T00:39:08.571 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/dummy.ko 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/eql.ko 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/8390.ko 2023-10-08T00:39:08.572 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/agere/et131x.ko 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-10-08T00:39:08.573 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-10-08T00:39:08.574 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-10-08T00:39:08.575 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco 2023-10-08T00:39:08.576 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-10-08T00:39:08.577 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/dnet.ko 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ethoc.ko 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/fealnx.ko 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e100.ko 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e 2023-10-08T00:39:08.578 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe 2023-10-08T00:39:08.579 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/jme.ko 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/skge.ko 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-10-08T00:39:08.580 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-10-08T00:39:08.581 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-10-08T00:39:08.582 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/atp.ko 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-10-08T00:39:08.583 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis190.ko 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sis/sis900.ko 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac 2023-10-08T00:39:08.584 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/cassini.ko 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/niu.ko 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sungem.ko 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/ti/tlan.ko 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via 2023-10-08T00:39:08.585 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/geneve.ko 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ieee802154/fakelb.ko 2023-10-08T00:39:08.586 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ifb.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ipvlan/ipvlan.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvlan.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/macvtap.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mdio/mdio-regmap.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/mii.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/net_failover.ko 2023-10-08T00:39:08.587 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/netconsole.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/nlmon.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs-lynx.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/amd.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm7xxx.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/bcm87xx.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/broadcom.ko 2023-10-08T00:39:08.588 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/cicada.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/davicom.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/dp83640.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/et1011c.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/icplus.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/lxt.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/marvell.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/micrel.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/national.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/phylink.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/qsemi.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/realtek.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/smsc.ko 2023-10-08T00:39:08.589 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/ste10Xp.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/phy/vitesse.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/bsd_comp.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_async.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_deflate.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_generic.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_mppe.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/ppp_synctty.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppoe.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pppox.ko 2023-10-08T00:39:08.590 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/ppp/pptp.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slhc.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/slip/slip.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/sungem_phy.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tap.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_activebackup.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_broadcast.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_random.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-10-08T00:39:08.591 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/tun.ko 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/veth.ko 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/virtio_net.ko 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/vxlan/vxlan.ko 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netback/xen-netback.ko 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/net/xen-netfront.ko 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ntb/ntb.ko 2023-10-08T00:39:08.592 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme-core.ko 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/nvme/host/nvme.ko 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport.ko 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_cs.ko 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_pc.ko 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/parport/parport_serial.ko 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-10-08T00:39:08.593 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/pcie/aer_inject.ko 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pci/xen-pcifront.ko 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/i82092.ko 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/pd6729.ko 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pcmcia/yenta_socket.ko 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86 2023-10-08T00:39:08.594 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acer-wmi.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/acerhdf.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/apple-gmux.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-laptop.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/asus-wmi.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/classmate-laptop.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/hdaps.ko 2023-10-08T00:39:08.595 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/intel_ips.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/msi-wmi.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/mxm-wmi.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-laptop.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/samsung-q10.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/topstar-laptop.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/toshiba_haps.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi-bmof.ko 2023-10-08T00:39:08.596 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/platform/x86/wmi.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_common.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/powercap/intel_rapl_msr.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-gpio.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps-ldisc.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/clients/pps_parport.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/pps/pps_core.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ptp/ptp_kvm.ko 2023-10-08T00:39:08.597 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq32k.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-bq4802.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1286.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1307.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1374.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1511.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1553.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1672.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds1742.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds2404.ko 2023-10-08T00:39:08.598 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-ds3232.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-em3027.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-fm3130.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl12022.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-isl1208.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m41t80.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t35.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-m48t59.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-max6900.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-msm6242.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf2127.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf85063.ko 2023-10-08T00:39:08.599 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8523.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8563.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-pcf8583.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rp5c01.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rs5c372.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8025.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-rx8581.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/rtc/rtc-x1205.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-9xxx.ko 2023-10-08T00:39:08.600 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-sas.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/3w-xxxx.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/BusLogic.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/a100u2w.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aacraid/aacraid.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/advansys.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/am53c974.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-10-08T00:39:08.601 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/atp870u.ko 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bfa/bfa.ko 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ch.ko 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor 2023-10-08T00:39:08.602 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/csiostor/csiostor.ko 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dc395x.ko 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-10-08T00:39:08.603 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/dmx3191d.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esas2r/esas2r.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/esp_scsi.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/fcoe.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/fnic/fnic.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hpsa.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hptiop.ko 2023-10-08T00:39:08.604 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/hv_storvsc.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/initio.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ipr.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ips.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/isci/isci.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/iscsi_tcp.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libfc/libfc.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libiscsi_tcp.ko 2023-10-08T00:39:08.605 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/libsas/libsas.ko 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/lpfc/lpfc.ko 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid.ko 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-10-08T00:39:08.606 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvsas/mvsas.ko 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/mvumi.ko 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/pmcraid.ko 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla1280.ko 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-10-08T00:39:08.607 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/raid_class.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_debug.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_fc.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_sas.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_spi.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/scsi_transport_srp.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/ses.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/st.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/stex.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/virtio_scsi.ko 2023-10-08T00:39:08.608 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/vmw_pvscsi.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/scsi/wd719x.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ssb/ssb.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/loopback/tcm_loop.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_file.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_iblock.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_mod.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_pscsi.ko 2023-10-08T00:39:08.609 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/target_core_user.ko 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-10-08T00:39:08.610 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_gsm.ko 2023-10-08T00:39:08.611 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/n_hdlc.ko 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/nozomi.ko 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/arc_uart.ko 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/serial/jsm/jsm.ko 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/tty/synclink_gt.ko 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/core/ufshcd-core.ko 2023-10-08T00:39:08.612 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio.ko 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_aec.ko 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_cif.ko 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_pci_generic.ko 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/uio/uio_sercos3.ko 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-acm.ko 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/cdc-wdm.ko 2023-10-08T00:39:08.613 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usblp.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/class/usbtmc.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/host/sl811-hcd.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/mdc800.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/image/microtek.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/adutux.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/appledisplay.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi26.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/emi62.ko 2023-10-08T00:39:08.614 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ezusb.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/idmouse.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/iowarrior.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/isight_firmware.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/ldusb.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/legousbtower.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/trancevibrator.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usb3503.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usblcd.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/usbsevseg.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/uss720.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/misc/yurex.ko 2023-10-08T00:39:08.615 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/phy/phy-generic.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/aircable.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ark3116.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/belkin_sa.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ch341.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cp210x.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cyberjack.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/cypress_m8.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/digi_acceleport.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/empeg.ko 2023-10-08T00:39:08.616 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ftdi_sio.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/garmin_gps.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_edgeport.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/io_ti.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipaq.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ipw.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ir-usb.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/keyspan_pda.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kl5kusb105.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/kobil_sct.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mct_u232.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7720.ko 2023-10-08T00:39:08.617 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/mos7840.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/navman.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/omninet.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/opticon.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/option.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/oti6858.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/pl2303.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcaux.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/qcserial.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/quatech2.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/safe_serial.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/sierra.ko 2023-10-08T00:39:08.618 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/spcp8x5.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ssu100.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/symbolserial.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_debug.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/usb_wwan.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/visor.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/whiteheat.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/serial/xsens_mt.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/uas.ko 2023-10-08T00:39:08.619 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-alauda.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-cypress.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-datafab.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-freecom.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-isd200.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-karma.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-onetouch.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-realtek.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr09.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-sddr55.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/ums-usbat.ko 2023-10-08T00:39:08.620 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/storage/usb-storage.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-core.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/usbip-host.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/pci/vfio-pci.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost.ko 2023-10-08T00:39:08.621 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_iotlb.ko 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_net.ko 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/vhost/vhost_scsi.ko 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/apple_bl.ko 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/lcd.ko 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/video/backlight/platform_lcd.ko 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio.ko 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_balloon.ko 2023-10-08T00:39:08.622 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_mmio.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/virtio/virtio_ring.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2482.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/masters/ds2490.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-10-08T00:39:08.623 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_smem.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/slaves/w1_therm.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/w1/wire.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim1535_wdt.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/alim7101_wdt.ko 2023-10-08T00:39:08.624 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/f71808e_wdt.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/hpwdt.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/i6300esb.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/iTCO_wdt.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ib700wdt.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ibmasr.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it8712f_wdt.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/it87_wdt.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/machzwd.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/nv_tco.ko 2023-10-08T00:39:08.625 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_pci.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/pcwd_usb.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sch311x_wdt.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/softdog.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/sp5100_tco.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/via_wdt.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83877f_wdt.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/w83977f_wdt.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/wdt_pci.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/watchdog/xen_wdt.ko 2023-10-08T00:39:08.626 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-acpi-processor.ko 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-evtchn.ko 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntalloc.ko 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-gntdev.ko 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xen-privcmd.ko 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/drivers/xen/xenfs/xenfs.ko 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p 2023-10-08T00:39:08.627 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/9p/9p.ko 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/affs/affs.ko 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/befs/befs.ko 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/binfmt_misc.ko 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/btrfs/btrfs.ko 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cachefiles/cachefiles.ko 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ceph/ceph.ko 2023-10-08T00:39:08.628 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/coda/coda.ko 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/cramfs/cramfs.ko 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/dlm/dlm.ko 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ecryptfs/ecryptfs.ko 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/f2fs/f2fs.ko 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/fat.ko 2023-10-08T00:39:08.629 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/msdos.ko 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fat/vfat.ko 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fscache/fscache.ko 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/cuse.ko 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/fuse/fuse.ko 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/gfs2/gfs2.ko 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfs/hfs.ko 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus 2023-10-08T00:39:08.630 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/hfsplus/hfsplus.ko 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/isofs/isofs.ko 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/jfs/jfs.ko 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/lockd/lockd.ko 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/minix/minix.ko 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/netfs/netfs.ko 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout 2023-10-08T00:39:08.631 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfs.ko 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv3.ko 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs/nfsv4.ko 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/grace.ko 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfs_common/nfs_acl.ko 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd 2023-10-08T00:39:08.632 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nfsd/nfsd.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nilfs2/nilfs2.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-celtic.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-centeuro.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-croatian.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-cyrillic.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-gaelic.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-greek.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-iceland.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-inuit.ko 2023-10-08T00:39:08.633 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-roman.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-romanian.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/mac-turkish.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1250.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1251.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp1255.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp737.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp775.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp850.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp852.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp855.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp857.ko 2023-10-08T00:39:08.634 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp860.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp861.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp862.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp863.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp864.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp865.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp866.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp869.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp874.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp932.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp936.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp949.ko 2023-10-08T00:39:08.635 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_cp950.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_euc-jp.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-1.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-13.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-14.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-15.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-2.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-3.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-4.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-5.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-6.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-7.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_iso8859-9.ko 2023-10-08T00:39:08.636 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-r.ko 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-ru.ko 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_koi8-u.ko 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/nls/nls_utf8.ko 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-10-08T00:39:08.637 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2.ko 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/overlayfs/overlay.ko 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/pstore/ramoops.ko 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/reiserfs/reiserfs.ko 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/romfs/romfs.ko 2023-10-08T00:39:08.638 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/client/cifs.ko 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_arc4.ko 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/smb/common/cifs_md4.ko 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/squashfs/squashfs.ko 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/sysv/sysv.ko 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/udf/udf.ko 2023-10-08T00:39:08.639 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/ufs/ufs.ko 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/fs/xfs/xfs.ko 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/rcu/rcutorture.ko 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/torture.ko 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/asn1_encoder.ko 2023-10-08T00:39:08.640 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc-itu-t.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crc8.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libarc4.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/crypto/libdes.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/libcrc32c.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lru_cache.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4_compress.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/lz4/lz4hc_compress.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/math/cordic.ko 2023-10-08T00:39:08.641 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/raid6/raid6_pq.ko 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/reed_solomon/reed_solomon.ko 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_bm.ko 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_fsm.ko 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/ts_kmp.ko 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/lib/zstd/zstd_compress.ko 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/mm/hwpoison-inject.ko 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net 2023-10-08T00:39:08.642 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/6lowpan.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_dest.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_fragment.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_hop.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_ipv6.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_mobility.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_routing.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/6lowpan/nhc_udp.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/garp.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/mrp.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/p8022.ko 2023-10-08T00:39:08.643 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/psnap.ko 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/802/stp.ko 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/8021q/8021q.ko 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet.ko 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_fd.ko 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_rdma.ko 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/9p/9pnet_virtio.ko 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/batman-adv/batman-adv.ko 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge 2023-10-08T00:39:08.644 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/br_netfilter.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/bridge.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_among.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arp.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_limit.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_log.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark.ko 2023-10-08T00:39:08.645 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_snat.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_stp.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/ebtables.ko 2023-10-08T00:39:08.646 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ceph/libceph.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/failover.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/core/pktgen.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_diag.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv4.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dccp/dccp_ipv6.ko 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver 2023-10-08T00:39:08.647 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dns_resolver/dns_resolver.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/dsa_core.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_brcm.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_dsa.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_none.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/dsa/tag_trailer.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ieee802154/ieee802154_socket.ko 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4 2023-10-08T00:39:08.648 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ah4.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/esp4.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/fou.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/gre.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/inet_diag.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_gre.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_tunnel.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ip_vti.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipcomp.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/ipip.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arp_tables.ko 2023-10-08T00:39:08.649 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/iptable_security.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-10-08T00:39:08.650 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_bic.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_dctcp.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_diag.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_highspeed.ko 2023-10-08T00:39:08.651 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_htcp.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_hybla.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_illinois.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_lp.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_scalable.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_vegas.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_veno.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_westwood.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tcp_yeah.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/tunnel4.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_diag.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/udp_tunnel.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv4/xfrm4_tunnel.ko 2023-10-08T00:39:08.652 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ah6.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/esp6.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/fou6.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_tunnel.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ip6_vti.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/ipcomp6.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-10-08T00:39:08.653 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-10-08T00:39:08.654 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/sit.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/tunnel6.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/ipv6/xfrm6_tunnel.ko 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key 2023-10-08T00:39:08.655 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/key/af_key.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_core.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_debugfs.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_eth.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ip6.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_netlink.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/l2tp/l2tp_ppp.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/llc/llc.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mac802154/mac802154.ko 2023-10-08T00:39:08.656 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/mpls/mpls_gso.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-10-08T00:39:08.657 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-10-08T00:39:08.658 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conncount.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack.ko 2023-10-08T00:39:08.659 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_h323.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_irc.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sane.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_sip.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-10-08T00:39:08.660 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_dup_netdev.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_flow_table_inet.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_log_syslog.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_amanda.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_ftp.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_irc.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_sip.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_nat_tftp.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_synproxy_core.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nf_tables.ko 2023-10-08T00:39:08.661 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_acct.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_log.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_osf.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nfnetlink_queue.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_chain_nat.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_compat.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_ct.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_dup_netdev.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fib.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_fwd_netdev.ko 2023-10-08T00:39:08.662 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_hash.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_limit.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_log.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_masq.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_nat.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_numgen.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_queue.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_quota.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_redir.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_reject_inet.ko 2023-10-08T00:39:08.663 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_socket.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/nft_tunnel.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_AUDIT.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CHECKSUM.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CLASSIFY.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_CT.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_DSCP.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HL.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_HMARK.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_IDLETIMER.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LED.ko 2023-10-08T00:39:08.664 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_LOG.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_MASQUERADE.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NETMAP.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFLOG.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_NFQUEUE.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_RATEEST.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_REDIRECT.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_SECMARK.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPMSS.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TEE.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TPROXY.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_TRACE.ko 2023-10-08T00:39:08.665 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_addrtype.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_bpf.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cgroup.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cluster.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_comment.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connbytes.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlabel.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connlimit.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_connmark.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_conntrack.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_cpu.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dccp.ko 2023-10-08T00:39:08.666 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_devgroup.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_dscp.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ecn.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_esp.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hashlimit.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_helper.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_hl.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipcomp.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_iprange.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_ipvs.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_l2tp.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_length.ko 2023-10-08T00:39:08.667 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_limit.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mac.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_mark.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_multiport.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nat.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_nfacct.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_osf.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_owner.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_physdev.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_pkttype.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_policy.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_quota.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_rateest.ko 2023-10-08T00:39:08.668 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_realm.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_recent.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_sctp.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_set.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_socket.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_state.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_statistic.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_string.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_tcpmss.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_time.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netfilter/xt_u32.ko 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink 2023-10-08T00:39:08.669 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/netlink/netlink_diag.ko 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/nsh/nsh.ko 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/openvswitch.ko 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-geneve.ko 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-gre.ko 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/openvswitch/vport-vxlan.ko 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/packet/af_packet_diag.ko 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds.ko 2023-10-08T00:39:08.670 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_rdma.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/rds/rds_tcp.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_csum.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_gact.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_ipt.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_mirred.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_nat.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_pedit.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_police.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_simple.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_skbedit.ko 2023-10-08T00:39:08.671 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/act_vlan.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_basic.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_bpf.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_flow.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_fw.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_route.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/cls_u32.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_cmp.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_ipset.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_meta.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_nbyte.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_text.ko 2023-10-08T00:39:08.672 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/em_u32.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_choke.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_codel.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_drr.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_fq.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_gred.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hfsc.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_hhf.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_htb.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_ingress.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_mqprio_lib.ko 2023-10-08T00:39:08.673 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_multiq.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_netem.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_pie.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_plug.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_prio.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_qfq.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_red.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfb.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_sfq.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_tbf.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sched/sch_teql.ko 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp 2023-10-08T00:39:08.674 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp.ko 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sctp/sctp_diag.ko 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/sunrpc.ko 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/diag.ko 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/tipc/tipc.ko 2023-10-08T00:39:08.675 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/unix/unix_diag.ko 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock.ko 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_diag.ko 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/vmw_vsock/vsock_loopback.ko 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/net/xfrm/xfrm_ipcomp.ko 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys 2023-10-08T00:39:08.676 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/security/keys/trusted-keys/trusted.ko 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/kernel/virt/lib/irqbypass.ko 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.alias.bin 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.bin 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.builtin.modinfo 2023-10-08T00:39:08.677 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep 2023-10-08T00:39:08.678 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.dep.bin 2023-10-08T00:39:08.678 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.devname 2023-10-08T00:39:08.678 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.order 2023-10-08T00:39:08.678 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.softdep 2023-10-08T00:39:08.678 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols 2023-10-08T00:39:08.678 INFO:teuthology.orchestra.run.smithi184.stdout:/lib/modules/6.5.0-rc7-gfef64f93da2b/modules.symbols.bin 2023-10-08T00:39:08.678 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.679 DEBUG:teuthology.orchestra.run.smithi184:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:postinstall scriptlet (using /bin/sh): 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b -a -r /boot/System.map-6.5.0-rc7-gfef64f93da2b ]; then 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:cp /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:cp /boot/System.map-6.5.0-rc7-gfef64f93da2b /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:rm -f /boot/vmlinuz-6.5.0-rc7-gfef64f93da2b /boot/System.map-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:/sbin/installkernel 6.5.0-rc7-gfef64f93da2b /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:rm -f /boot/.vmlinuz-6.5.0-rc7-gfef64f93da2b-rpm /boot/.System.map-6.5.0-rc7-gfef64f93da2b-rpm 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2023-10-08T00:39:08.700 INFO:teuthology.orchestra.run.smithi184.stdout:preuninstall scriptlet (using /bin/sh): 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:new-kernel-pkg --remove 6.5.0-rc7-gfef64f93da2b --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc7-gfef64f93da2b.img 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:kernel-install remove 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:postuninstall scriptlet (using /bin/sh): 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:if [ -x /sbin/update-bootloader ]; then 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:/sbin/update-bootloader --remove 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.701 INFO:teuthology.orchestra.run.smithi184.stdout:fi 2023-10-08T00:39:08.702 DEBUG:teuthology.orchestra.run.smithi184:> sudo rpm -qi grub2-tools 2023-10-08T00:39:08.774 INFO:teuthology.orchestra.run.smithi184.stdout:Name : grub2-tools 2023-10-08T00:39:08.774 INFO:teuthology.orchestra.run.smithi184.stdout:Epoch : 1 2023-10-08T00:39:08.774 INFO:teuthology.orchestra.run.smithi184.stdout:Version : 2.02 2023-10-08T00:39:08.774 INFO:teuthology.orchestra.run.smithi184.stdout:Release : 99.el8_4.1 2023-10-08T00:39:08.774 INFO:teuthology.orchestra.run.smithi184.stdout:Architecture: x86_64 2023-10-08T00:39:08.774 INFO:teuthology.orchestra.run.smithi184.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Group : System Environment/Base 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Size : 9264631 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:License : GPLv3+ 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Relocations : (not relocatable) 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Packager : Red Hat, Inc. 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Vendor : Red Hat, Inc. 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-08T00:39:08.775 INFO:teuthology.orchestra.run.smithi184.stdout:Summary : Support tools for GRUB. 2023-10-08T00:39:08.776 INFO:teuthology.orchestra.run.smithi184.stdout:Description : 2023-10-08T00:39:08.776 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:39:08.776 INFO:teuthology.orchestra.run.smithi184.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-08T00:39:08.776 INFO:teuthology.orchestra.run.smithi184.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-08T00:39:08.776 INFO:teuthology.orchestra.run.smithi184.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-08T00:39:08.776 INFO:teuthology.orchestra.run.smithi184.stdout:hardware devices. 2023-10-08T00:39:08.776 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:39:08.776 INFO:teuthology.orchestra.run.smithi184.stdout:This subpackage provides tools for support of all platforms. 2023-10-08T00:39:08.777 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-08T00:39:08.777 INFO:teuthology.task.kernel:Updating grub on smithi184 to boot 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:08.777 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-08T00:39:13.220 INFO:teuthology.orchestra.run.smithi184.stderr:Generating grub configuration file ... 2023-10-08T00:39:15.465 INFO:teuthology.orchestra.run.smithi184.stderr:done 2023-10-08T00:39:15.467 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-10-08T00:39:15.481 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.fPOLlJb0NF 2023-10-08T00:39:15.481 DEBUG:teuthology.orchestra.run.smithi184:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fPOLlJb0NF 2023-10-08T00:39:15.550 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.fPOLlJb0NF 2023-10-08T00:39:15.710 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.fPOLlJb0NF is 6KB 2023-10-08T00:39:15.757 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.fPOLlJb0NF 2023-10-08T00:39:15.771 DEBUG:teuthology.orchestra.run.smithi184:> sudo /bin/ls /boot/loader/entries || true 2023-10-08T00:39:15.832 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-10-08T00:39:15.832 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-10-08T00:39:15.833 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-10-08T00:39:15.833 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-10-08T00:39:15.833 INFO:teuthology.orchestra.run.smithi184.stdout:298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b.conf 2023-10-08T00:39:15.834 DEBUG:teuthology.orchestra.run.smithi184:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-6.5.0-rc7-gfef64f93da2b 2023-10-08T00:39:15.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo shutdown -r now 2023-10-08T00:39:37.515 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-10-08T00:39:37.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-10-08T00:39:37.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:39:46.010 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-10-08T00:39:46.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-08T00:39:46.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:39:56.026 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-10-08T00:40:04.474 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-10-08T00:40:05.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-10-08T00:40:05.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:40:08.094 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-10-08T00:40:13.478 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-08T00:40:13.479 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:40:16.539 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.184 2023-10-08T00:40:20.098 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-10-08T00:40:20.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:40:28.546 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-08T00:40:28.547 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:40:38.522 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-10-08T00:40:53.538 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-10-08T00:40:53.539 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:40:53.927 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-10-08T00:40:54.381 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-10-08T00:40:54.382 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-08T00:40:54.382 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-10-08T00:40:54.417 INFO:teuthology.orchestra.run.smithi119.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-08T00:40:54.417 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:40:54.417 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-08T00:40:54.417 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-08T00:40:55.419 DEBUG:teuthology.parallel:result is None 2023-10-08T00:41:00.429 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-10-08T00:41:00.862 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-08T00:41:00.862 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.5.0-rc7-gfef64f93da2b"... 2023-10-08T00:41:00.862 DEBUG:teuthology.orchestra.run.smithi184:> uname -r 2023-10-08T00:41:00.898 INFO:teuthology.orchestra.run.smithi184.stdout:6.5.0-rc7-gfef64f93da2b 2023-10-08T00:41:00.898 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc7-gfef64f93da2b vs 6.5.0-rc7-gfef64f93da2b 2023-10-08T00:41:00.898 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-08T00:41:00.899 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-10-08T00:41:01.900 DEBUG:teuthology.parallel:result is None 2023-10-08T00:41:01.900 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-08T00:41:01.916 INFO:teuthology.task.internal:Creating test directory... 2023-10-08T00:41:01.917 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T00:41:01.921 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-08T00:41:01.962 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-08T00:41:01.969 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-08T00:41:01.976 INFO:teuthology.task.internal:Creating archive directory... 2023-10-08T00:41:01.976 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T00:41:02.003 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-08T00:41:02.055 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-08T00:41:02.062 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-08T00:41:02.062 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T00:41:02.067 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-08T00:41:02.213 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T00:41:02.250 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T00:41:02.290 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T00:41:02.374 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-08T00:41:02.378 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-08T00:41:02.389 INFO:teuthology.task.internal:Configuring sudo... 2023-10-08T00:41:02.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T00:41:02.393 DEBUG:teuthology.orchestra.run.smithi184:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-08T00:41:02.457 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-08T00:41:02.467 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-08T00:41:02.467 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T00:41:02.471 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-08T00:41:02.507 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T00:41:02.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T00:41:02.706 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T00:41:02.771 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T00:41:02.845 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:41:02.845 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T00:41:02.912 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T00:41:02.979 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-08T00:41:03.072 DEBUG:teuthology.orchestra.run.smithi184:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T00:41:03.139 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-08T00:41:03.214 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:41:03.214 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-08T00:41:03.278 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2023-10-08T00:41:03.283 DEBUG:teuthology.orchestra.run.smithi184:> sudo service rsyslog restart 2023-10-08T00:41:03.395 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T00:41:03.494 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T00:41:03.789 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-08T00:41:03.809 INFO:teuthology.task.internal:Starting timer... 2023-10-08T00:41:03.809 INFO:teuthology.run_tasks:Running task pcp... 2023-10-08T00:41:03.832 INFO:teuthology.run_tasks:Running task selinux... 2023-10-08T00:41:03.845 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-08T00:41:03.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2023-10-08T00:41:03.943 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: [ OK ] 2023-10-08T00:41:03.947 DEBUG:teuthology.orchestra.run.smithi184:> sudo service auditd rotate 2023-10-08T00:41:04.072 INFO:teuthology.orchestra.run.smithi184.stdout:Rotating logs: [ OK ] 2023-10-08T00:41:04.076 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-08T00:41:04.076 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2023-10-08T00:41:04.123 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2023-10-08T00:41:04.124 DEBUG:teuthology.orchestra.run.smithi184:> /usr/sbin/getenforce 2023-10-08T00:41:04.171 INFO:teuthology.orchestra.run.smithi184.stdout:Permissive 2023-10-08T00:41:04.171 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi184.front.sepia.ceph.com': 'permissive'} 2023-10-08T00:41:04.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T00:41:04.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:41:04.233 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T00:41:04.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:41:04.298 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-08T00:41:04.298 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2023-10-08T00:41:04.359 DEBUG:teuthology.orchestra.run.smithi184:> sudo /usr/sbin/setenforce permissive 2023-10-08T00:41:04.420 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-08T00:41:04.429 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-08T00:41:04.429 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-08T00:41:04.446 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-08T00:41:04.447 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi184.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-08T00:52:21.694 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi184.front.sepia.ceph.com')] 2023-10-08T00:52:21.716 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-10-08T00:52:21.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:52:21.806 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-10-08T00:52:21.909 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-10-08T00:52:21.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-08T00:52:21.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi184.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-08T00:52:21.993 DEBUG:teuthology.orchestra.run.smithi184:> true 2023-10-08T00:52:22.097 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi184.front.sepia.ceph.com' 2023-10-08T00:52:22.097 INFO:teuthology.run_tasks:Running task clock... 2023-10-08T00:52:22.107 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-08T00:52:22.107 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T00:52:22.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T00:52:22.112 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-08T00:52:22.113 DEBUG:teuthology.orchestra.run.smithi184:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T00:52:22.174 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T00:52:22.203 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-08T00:52:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T00:52:22.237 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-08T00:52:22.267 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2023-10-08T00:52:22.294 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2023-10-08T00:52:22.298 INFO:teuthology.orchestra.run.smithi184.stderr:sudo: ntpd: command not found 2023-10-08T00:52:22.324 INFO:teuthology.orchestra.run.smithi184.stdout:506 Cannot talk to daemon 2023-10-08T00:52:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T00:52:22.358 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-08T00:52:22.359 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T00:52:22.392 INFO:teuthology.orchestra.run.smithi184.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-08T00:52:22.445 INFO:teuthology.orchestra.run.smithi119.stderr:bash: ntpq: command not found 2023-10-08T00:52:22.449 INFO:teuthology.orchestra.run.smithi119.stdout:210 Number of sources = 4 2023-10-08T00:52:22.450 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T00:52:22.450 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-10-08T00:52:22.450 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:52:22.450 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:52:22.450 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:52:22.450 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:52:22.477 INFO:teuthology.orchestra.run.smithi184.stderr:bash: ntpq: command not found 2023-10-08T00:52:22.482 INFO:teuthology.orchestra.run.smithi184.stdout:210 Number of sources = 4 2023-10-08T00:52:22.482 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T00:52:22.482 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2023-10-08T00:52:22.482 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:52:22.482 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:52:22.482 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:52:22.483 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T00:52:22.483 INFO:teuthology.run_tasks:Running task install... 2023-10-08T00:52:22.492 DEBUG:teuthology.task.install:project ceph 2023-10-08T00:52:22.492 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}} 2023-10-08T00:52:22.492 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'} 2023-10-08T00:52:22.492 INFO:teuthology.task.install:Using flavor: default 2023-10-08T00:52:22.498 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-08T00:52:22.498 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-08T00:52:22.499 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-08T00:52:22.500 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:52:22.502 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'tag': None, 'wait_for_package': False} 2023-10-08T00:52:22.502 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:52:22.646 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-08T00:52:22.646 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-08T00:52:22.650 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/ 2023-10-08T00:52:22.651 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1320.g08cac018 2023-10-08T00:52:22.757 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T00:52:22.758 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:52:22.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T00:52:22.820 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-06-0949-quincy/08cac0187919fc3a247efdf445f652e2fb30a647/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-08T00:52:22.820 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:52:22.820 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-08T00:52:22.821 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-08T00:52:22.821 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T00:52:22.883 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-08T00:52:22.883 DEBUG:teuthology.orchestra.run.smithi184:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/08cac0187919fc3a247efdf445f652e2fb30a647/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-08T00:52:22.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T00:52:22.986 DEBUG:teuthology.orchestra.run.smithi184:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-08T00:52:23.011 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T00:52:23.070 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2023-10-08T00:52:23.073 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2023-10-08T00:52:23.074 DEBUG:teuthology.orchestra.run.smithi184:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-08T00:52:23.134 INFO:teuthology.orchestra.run.smithi184.stdout:check_obsoletes = 1 2023-10-08T00:52:23.137 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum clean all 2023-10-08T00:52:23.529 INFO:teuthology.orchestra.run.smithi119.stdout:Updating Subscription Management repositories. 2023-10-08T00:52:23.596 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-08T00:52:23.855 INFO:teuthology.orchestra.run.smithi119.stdout:44 files removed 2023-10-08T00:52:23.913 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-08T00:52:23.929 INFO:teuthology.orchestra.run.smithi184.stdout:44 files removed 2023-10-08T00:52:23.995 DEBUG:teuthology.orchestra.run.smithi184:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-08T00:52:24.374 INFO:teuthology.orchestra.run.smithi119.stdout:Updating Subscription Management repositories. 2023-10-08T00:52:24.452 INFO:teuthology.orchestra.run.smithi184.stdout:Updating Subscription Management repositories. 2023-10-08T00:52:25.130 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 351 kB/s | 78 kB 00:00 2023-10-08T00:52:25.309 INFO:teuthology.orchestra.run.smithi184.stdout:ceph packages for x86_64 247 kB/s | 78 kB 00:00 2023-10-08T00:52:25.417 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 38 kB/s | 9.2 kB 00:00 2023-10-08T00:52:25.654 INFO:teuthology.orchestra.run.smithi184.stdout:ceph noarch packages 30 kB/s | 9.2 kB 00:00 2023-10-08T00:52:25.753 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 4.8 kB/s | 1.5 kB 00:00 2023-10-08T00:52:25.863 INFO:teuthology.orchestra.run.smithi119.stdout:Copr repo for python3-asyncssh owned by ceph 43 kB/s | 3.5 kB 00:00 2023-10-08T00:52:25.992 INFO:teuthology.orchestra.run.smithi184.stdout:ceph source packages 4.8 kB/s | 1.5 kB 00:00 2023-10-08T00:52:26.106 INFO:teuthology.orchestra.run.smithi184.stdout:Copr repo for python3-asyncssh owned by ceph 41 kB/s | 3.5 kB 00:00 2023-10-08T00:52:26.717 INFO:teuthology.orchestra.run.smithi184.stdout:Extra Packages for Enterprise Linux 27 MB/s | 16 MB 00:00 2023-10-08T00:52:27.199 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 12 MB/s | 16 MB 00:01 2023-10-08T00:52:30.200 INFO:teuthology.orchestra.run.smithi184.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-10-08T00:52:30.679 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 3.6 MB/s | 24 kB 00:00 2023-10-08T00:52:30.696 INFO:teuthology.orchestra.run.smithi184.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 80 MB/s | 33 MB 00:00 2023-10-08T00:52:31.213 INFO:teuthology.orchestra.run.smithi119.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 85 MB/s | 38 MB 00:00 2023-10-08T00:52:36.342 INFO:teuthology.orchestra.run.smithi184.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 83 MB/s | 38 MB 00:00 2023-10-08T00:52:37.109 INFO:teuthology.orchestra.run.smithi119.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 83 MB/s | 33 MB 00:00 2023-10-08T00:52:44.918 INFO:teuthology.orchestra.run.smithi184.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T00:52:44.918 INFO:teuthology.orchestra.run.smithi184.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T00:52:44.929 INFO:teuthology.orchestra.run.smithi184.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-08T00:52:44.930 INFO:teuthology.orchestra.run.smithi184.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-08T00:52:44.931 INFO:teuthology.orchestra.run.smithi184.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-08T00:52:44.932 INFO:teuthology.orchestra.run.smithi184.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T00:52:44.934 INFO:teuthology.orchestra.run.smithi184.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T00:52:44.936 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-08T00:52:44.937 INFO:teuthology.orchestra.run.smithi184.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-08T00:52:45.422 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T00:52:45.423 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-08T00:52:45.434 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-10-08T00:52:45.435 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-10-08T00:52:45.436 INFO:teuthology.orchestra.run.smithi184.stdout:Dependencies resolved. 2023-10-08T00:52:45.436 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-10-08T00:52:45.437 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-08T00:52:45.438 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-08T00:52:45.441 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-10-08T00:52:45.442 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-10-08T00:52:45.447 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================================================== 2023-10-08T00:52:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: Package Arch Version Repository Size 2023-10-08T00:52:45.448 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================================================== 2023-10-08T00:52:45.448 INFO:teuthology.orchestra.run.smithi184.stdout:Installing: 2023-10-08T00:52:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T00:52:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-08T00:52:45.448 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-08T00:52:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-08T00:52:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-08T00:52:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-08T00:52:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-08T00:52:45.449 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-08T00:52:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-08T00:52:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-08T00:52:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T00:52:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-08T00:52:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-08T00:52:45.450 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-08T00:52:45.451 INFO:teuthology.orchestra.run.smithi184.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T00:52:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-08T00:52:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T00:52:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-08T00:52:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-08T00:52:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-08T00:52:45.452 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-08T00:52:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-08T00:52:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T00:52:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-08T00:52:45.453 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-08T00:52:45.453 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrading: 2023-10-08T00:52:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T00:52:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-08T00:52:45.454 INFO:teuthology.orchestra.run.smithi184.stdout:Installing dependencies: 2023-10-08T00:52:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T00:52:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-08T00:52:45.454 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-08T00:52:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-08T00:52:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-08T00:52:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-08T00:52:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T00:52:45.455 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-08T00:52:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-08T00:52:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T00:52:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T00:52:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-08T00:52:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T00:52:45.456 INFO:teuthology.orchestra.run.smithi184.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T00:52:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-08T00:52:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-08T00:52:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-08T00:52:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T00:52:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-08T00:52:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T00:52:45.457 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-08T00:52:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-08T00:52:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-08T00:52:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T00:52:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-08T00:52:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T00:52:45.458 INFO:teuthology.orchestra.run.smithi184.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T00:52:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T00:52:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T00:52:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-08T00:52:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T00:52:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T00:52:45.459 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T00:52:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T00:52:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T00:52:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T00:52:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T00:52:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T00:52:45.460 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T00:52:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-08T00:52:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-08T00:52:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T00:52:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T00:52:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T00:52:45.461 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T00:52:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T00:52:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T00:52:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T00:52:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T00:52:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T00:52:45.462 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T00:52:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T00:52:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T00:52:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T00:52:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T00:52:45.463 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T00:52:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T00:52:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T00:52:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T00:52:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T00:52:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T00:52:45.464 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T00:52:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T00:52:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T00:52:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T00:52:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T00:52:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T00:52:45.465 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T00:52:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T00:52:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T00:52:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-08T00:52:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T00:52:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T00:52:45.466 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T00:52:45.467 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T00:52:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T00:52:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:52:45.468 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction Summary 2023-10-08T00:52:45.468 INFO:teuthology.orchestra.run.smithi184.stdout:====================================================================================================================== 2023-10-08T00:52:45.468 INFO:teuthology.orchestra.run.smithi184.stdout:Install 118 Packages 2023-10-08T00:52:45.468 INFO:teuthology.orchestra.run.smithi184.stdout:Upgrade 2 Packages 2023-10-08T00:52:45.468 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:52:45.469 INFO:teuthology.orchestra.run.smithi184.stdout:Total download size: 202 M 2023-10-08T00:52:45.469 INFO:teuthology.orchestra.run.smithi184.stdout:Downloading Packages: 2023-10-08T00:52:45.589 INFO:teuthology.orchestra.run.smithi184.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 69 kB/s | 6.4 kB 00:00 2023-10-08T00:52:45.778 INFO:teuthology.orchestra.run.smithi184.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 4.5 MB/s | 876 kB 00:00 2023-10-08T00:52:45.801 INFO:teuthology.orchestra.run.smithi184.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 7.0 MB/s | 162 kB 00:00 2023-10-08T00:52:45.942 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-10-08T00:52:45.952 INFO:teuthology.orchestra.run.smithi119.stdout:====================================================================================================================== 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout:====================================================================================================================== 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:17.2.6-1320.g08cac018.el8 ceph 6.4 k 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.2 M 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 876 k 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1320.g08cac018.el8 ceph 162 k 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:17.2.6-1320.g08cac018.el8 ceph 1.6 M 2023-10-08T00:52:45.953 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 121 k 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 1.7 M 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 7.4 M 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 47 k 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:17.2.6-1320.g08cac018.el8 ceph 52 M 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 267 k 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 80 k 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 255 k 2023-10-08T00:52:45.954 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 18 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 25 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 797 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:17.2.6-1320.g08cac018.el8 ceph 130 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:17.2.6-1320.g08cac018.el8 ceph 190 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:17.2.6-1320.g08cac018.el8 ceph 371 k 2023-10-08T00:52:45.955 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 358 k 2023-10-08T00:52:45.956 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:17.2.6-1320.g08cac018.el8 ceph 107 k 2023-10-08T00:52:45.956 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 90 k 2023-10-08T00:52:45.956 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T00:52:45.956 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 173 k 2023-10-08T00:52:45.956 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-10-08T00:52:45.956 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-10-08T00:52:45.956 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.5 M 2023-10-08T00:52:45.956 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.6 M 2023-10-08T00:52:45.957 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 23 M 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 23 k 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:17.2.6-1320.g08cac018.el8 ceph 2.3 M 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 238 k 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:17.2.6-1320.g08cac018.el8 ceph 3.9 M 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:17.2.6-1320.g08cac018.el8 ceph 17 M 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1320.g08cac018.el8 ceph-noarch 14 k 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:17.2.6-1320.g08cac018.el8 ceph 24 k 2023-10-08T00:52:45.958 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:17.2.6-1320.g08cac018.el8 ceph 174 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-10-08T00:52:45.959 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 507 k 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:17.2.6-1320.g08cac018.el8 ceph 5.6 M 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-10-08T00:52:45.960 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:17.2.6-1320.g08cac018.el8 ceph 45 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:17.2.6-1320.g08cac018.el8 ceph 115 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-08T00:52:45.961 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-10-08T00:52:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-10-08T00:52:45.963 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-08T00:52:45.964 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout:====================================================================================================================== 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout:Install 118 Packages 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 2 Packages 2023-10-08T00:52:45.965 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:52:45.966 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 202 M 2023-10-08T00:52:45.966 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-10-08T00:52:46.034 INFO:teuthology.orchestra.run.smithi184.stdout:(4/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 9.9 MB/s | 2.3 MB 00:00 2023-10-08T00:52:46.142 INFO:teuthology.orchestra.run.smithi184.stdout:(5/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 14 MB/s | 1.6 MB 00:00 2023-10-08T00:52:46.344 INFO:teuthology.orchestra.run.smithi184.stdout:(6/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 19 MB/s | 3.9 MB 00:00 2023-10-08T00:52:46.543 INFO:teuthology.orchestra.run.smithi119.stdout:(1/120): ceph-17.2.6-1320.g08cac018.el8.x86_64. 52 kB/s | 6.4 kB 00:00 2023-10-08T00:52:47.014 INFO:teuthology.orchestra.run.smithi119.stdout:(2/120): ceph-fuse-17.2.6-1320.g08cac018.el8.x8 1.8 MB/s | 876 kB 00:00 2023-10-08T00:52:47.084 INFO:teuthology.orchestra.run.smithi119.stdout:(3/120): ceph-immutable-object-cache-17.2.6-132 2.2 MB/s | 162 kB 00:00 2023-10-08T00:52:47.114 INFO:teuthology.orchestra.run.smithi184.stdout:(7/120): ceph-osd-17.2.6-1320.g08cac018.el8.x86 22 MB/s | 17 MB 00:00 2023-10-08T00:52:47.894 INFO:teuthology.orchestra.run.smithi119.stdout:(4/120): ceph-mds-17.2.6-1320.g08cac018.el8.x86 2.8 MB/s | 2.3 MB 00:00 2023-10-08T00:52:47.934 INFO:teuthology.orchestra.run.smithi184.stdout:(8/120): ceph-radosgw-17.2.6-1320.g08cac018.el8 20 MB/s | 17 MB 00:00 2023-10-08T00:52:47.956 INFO:teuthology.orchestra.run.smithi184.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 1.1 MB/s | 24 kB 00:00 2023-10-08T00:52:48.161 INFO:teuthology.orchestra.run.smithi119.stdout:(5/120): ceph-mgr-17.2.6-1320.g08cac018.el8.x86 5.9 MB/s | 1.6 MB 00:00 2023-10-08T00:52:48.968 INFO:teuthology.orchestra.run.smithi119.stdout:(6/120): ceph-mon-17.2.6-1320.g08cac018.el8.x86 4.9 MB/s | 3.9 MB 00:00 2023-10-08T00:52:50.072 INFO:teuthology.orchestra.run.smithi184.stdout:(10/120): ceph-base-17.2.6-1320.g08cac018.el8.x 1.1 MB/s | 5.2 MB 00:04 2023-10-08T00:52:50.175 INFO:teuthology.orchestra.run.smithi119.stdout:(7/120): ceph-base-17.2.6-1320.g08cac018.el8.x8 1.4 MB/s | 5.2 MB 00:03 2023-10-08T00:52:50.393 INFO:teuthology.orchestra.run.smithi184.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 797 kB/s | 255 kB 00:00 2023-10-08T00:52:50.500 INFO:teuthology.orchestra.run.smithi184.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 231 kB/s | 25 kB 00:00 2023-10-08T00:52:50.531 INFO:teuthology.orchestra.run.smithi119.stdout:(8/120): ceph-osd-17.2.6-1320.g08cac018.el8.x86 11 MB/s | 17 MB 00:01 2023-10-08T00:52:50.570 INFO:teuthology.orchestra.run.smithi119.stdout:(9/120): ceph-selinux-17.2.6-1320.g08cac018.el8 637 kB/s | 24 kB 00:00 2023-10-08T00:52:51.464 INFO:teuthology.orchestra.run.smithi184.stdout:(13/120): ceph-test-17.2.6-1320.g08cac018.el8.x 15 MB/s | 52 MB 00:03 2023-10-08T00:52:51.468 INFO:teuthology.orchestra.run.smithi184.stdout:(14/120): libcephfs2-17.2.6-1320.g08cac018.el8. 824 kB/s | 797 kB 00:00 2023-10-08T00:52:51.515 INFO:teuthology.orchestra.run.smithi184.stdout:(15/120): libcephsqlite-17.2.6-1320.g08cac018.e 3.4 MB/s | 174 kB 00:00 2023-10-08T00:52:51.574 INFO:teuthology.orchestra.run.smithi184.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 8.4 MB/s | 507 kB 00:00 2023-10-08T00:52:51.947 INFO:teuthology.orchestra.run.smithi184.stdout:(17/120): librados-devel-17.2.6-1320.g08cac018. 271 kB/s | 130 kB 00:00 2023-10-08T00:52:52.045 INFO:teuthology.orchestra.run.smithi184.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 454 kB/s | 45 kB 00:00 2023-10-08T00:52:52.083 INFO:teuthology.orchestra.run.smithi184.stdout:(19/120): librgw2-17.2.6-1320.g08cac018.el8.x86 11 MB/s | 5.6 MB 00:00 2023-10-08T00:52:52.142 INFO:teuthology.orchestra.run.smithi184.stdout:(20/120): python3-cephfs-17.2.6-1320.g08cac018. 3.2 MB/s | 190 kB 00:00 2023-10-08T00:52:52.205 INFO:teuthology.orchestra.run.smithi184.stdout:(21/120): python3-rados-17.2.6-1320.g08cac018.e 5.7 MB/s | 371 kB 00:00 2023-10-08T00:52:52.206 INFO:teuthology.orchestra.run.smithi184.stdout:(22/120): python3-ceph-common-17.2.6-1320.g08ca 718 kB/s | 115 kB 00:00 2023-10-08T00:52:52.266 INFO:teuthology.orchestra.run.smithi184.stdout:(23/120): python3-rbd-17.2.6-1320.g08cac018.el8 5.8 MB/s | 358 kB 00:00 2023-10-08T00:52:52.322 INFO:teuthology.orchestra.run.smithi184.stdout:(24/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 1.6 MB/s | 90 kB 00:00 2023-10-08T00:52:52.371 INFO:teuthology.orchestra.run.smithi184.stdout:(25/120): python3-rgw-17.2.6-1320.g08cac018.el8 649 kB/s | 107 kB 00:00 2023-10-08T00:52:52.720 INFO:teuthology.orchestra.run.smithi184.stdout:(26/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 8.7 MB/s | 3.5 MB 00:00 2023-10-08T00:52:52.732 INFO:teuthology.orchestra.run.smithi184.stdout:(27/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 481 kB/s | 173 kB 00:00 2023-10-08T00:52:52.760 INFO:teuthology.orchestra.run.smithi184.stdout:(28/120): ceph-grafana-dashboards-17.2.6-1320.g 585 kB/s | 23 kB 00:00 2023-10-08T00:52:52.927 INFO:teuthology.orchestra.run.smithi184.stdout:(29/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 622 kB/s | 121 kB 00:00 2023-10-08T00:52:52.933 INFO:teuthology.orchestra.run.smithi184.stdout:(30/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 9.8 MB/s | 1.7 MB 00:00 2023-10-08T00:52:52.981 INFO:teuthology.orchestra.run.smithi184.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 4.9 MB/s | 238 kB 00:00 2023-10-08T00:52:53.019 INFO:teuthology.orchestra.run.smithi184.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 1.2 MB/s | 47 kB 00:00 2023-10-08T00:52:53.051 INFO:teuthology.orchestra.run.smithi184.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 444 kB/s | 14 kB 00:00 2023-10-08T00:52:53.098 INFO:teuthology.orchestra.run.smithi184.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 5.6 MB/s | 267 kB 00:00 2023-10-08T00:52:53.141 INFO:teuthology.orchestra.run.smithi184.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 1.8 MB/s | 80 kB 00:00 2023-10-08T00:52:53.176 INFO:teuthology.orchestra.run.smithi184.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 522 kB/s | 18 kB 00:00 2023-10-08T00:52:53.322 INFO:teuthology.orchestra.run.smithi184.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 2.1 MB/s | 306 kB 00:00 2023-10-08T00:52:53.343 INFO:teuthology.orchestra.run.smithi184.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 2.7 MB/s | 59 kB 00:00 2023-10-08T00:52:53.366 INFO:teuthology.orchestra.run.smithi184.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 3.3 MB/s | 75 kB 00:00 2023-10-08T00:52:53.425 INFO:teuthology.orchestra.run.smithi184.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 7.9 MB/s | 476 kB 00:00 2023-10-08T00:52:53.453 INFO:teuthology.orchestra.run.smithi184.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.5 MB/s | 44 kB 00:00 2023-10-08T00:52:53.476 INFO:teuthology.orchestra.run.smithi184.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 7.8 MB/s | 185 kB 00:00 2023-10-08T00:52:53.507 INFO:teuthology.orchestra.run.smithi184.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.1 MB/s | 33 kB 00:00 2023-10-08T00:52:53.530 INFO:teuthology.orchestra.run.smithi184.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 715 kB/s | 16 kB 00:00 2023-10-08T00:52:53.554 INFO:teuthology.orchestra.run.smithi184.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 7.1 MB/s | 173 kB 00:00 2023-10-08T00:52:53.583 INFO:teuthology.orchestra.run.smithi184.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 13 MB/s | 384 kB 00:00 2023-10-08T00:52:53.615 INFO:teuthology.orchestra.run.smithi184.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 2.5 MB/s | 81 kB 00:00 2023-10-08T00:52:53.639 INFO:teuthology.orchestra.run.smithi184.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 458 kB/s | 11 kB 00:00 2023-10-08T00:52:53.661 INFO:teuthology.orchestra.run.smithi184.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 810 kB/s | 18 kB 00:00 2023-10-08T00:52:53.721 INFO:teuthology.orchestra.run.smithi184.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 19 MB/s | 1.2 MB 00:00 2023-10-08T00:52:53.742 INFO:teuthology.orchestra.run.smithi184.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 2.4 MB/s | 49 kB 00:00 2023-10-08T00:52:53.764 INFO:teuthology.orchestra.run.smithi184.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 2.6 MB/s | 59 kB 00:00 2023-10-08T00:52:53.784 INFO:teuthology.orchestra.run.smithi184.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 2.9 MB/s | 60 kB 00:00 2023-10-08T00:52:53.811 INFO:teuthology.orchestra.run.smithi184.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 11 MB/s | 283 kB 00:00 2023-10-08T00:52:53.834 INFO:teuthology.orchestra.run.smithi184.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 682 kB/s | 16 kB 00:00 2023-10-08T00:52:53.863 INFO:teuthology.orchestra.run.smithi184.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.1 MB/s | 34 kB 00:00 2023-10-08T00:52:53.894 INFO:teuthology.orchestra.run.smithi184.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 6.2 MB/s | 196 kB 00:00 2023-10-08T00:52:53.918 INFO:teuthology.orchestra.run.smithi184.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 2.7 MB/s | 64 kB 00:00 2023-10-08T00:52:53.929 INFO:teuthology.orchestra.run.smithi119.stdout:(10/120): ceph-test-17.2.6-1320.g08cac018.el8.x 15 MB/s | 52 MB 00:03 2023-10-08T00:52:53.941 INFO:teuthology.orchestra.run.smithi184.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 802 kB/s | 19 kB 00:00 2023-10-08T00:52:53.959 INFO:teuthology.orchestra.run.smithi119.stdout:(11/120): cephfs-mirror-17.2.6-1320.g08cac018.e 8.5 MB/s | 255 kB 00:00 2023-10-08T00:52:53.962 INFO:teuthology.orchestra.run.smithi184.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 1.2 MB/s | 24 kB 00:00 2023-10-08T00:52:53.980 INFO:teuthology.orchestra.run.smithi119.stdout:(12/120): libcephfs-devel-17.2.6-1320.g08cac018 1.1 MB/s | 25 kB 00:00 2023-10-08T00:52:53.990 INFO:teuthology.orchestra.run.smithi184.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 1.0 MB/s | 29 kB 00:00 2023-10-08T00:52:54.010 INFO:teuthology.orchestra.run.smithi184.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.4 MB/s | 27 kB 00:00 2023-10-08T00:52:54.021 INFO:teuthology.orchestra.run.smithi119.stdout:(13/120): libcephfs2-17.2.6-1320.g08cac018.el8. 19 MB/s | 797 kB 00:00 2023-10-08T00:52:54.040 INFO:teuthology.orchestra.run.smithi184.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 8.5 MB/s | 255 kB 00:00 2023-10-08T00:52:54.051 INFO:teuthology.orchestra.run.smithi119.stdout:(14/120): libcephsqlite-17.2.6-1320.g08cac018.e 5.8 MB/s | 174 kB 00:00 2023-10-08T00:52:54.069 INFO:teuthology.orchestra.run.smithi184.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 8.4 MB/s | 251 kB 00:00 2023-10-08T00:52:54.074 INFO:teuthology.orchestra.run.smithi119.stdout:(15/120): librados-devel-17.2.6-1320.g08cac018. 5.7 MB/s | 130 kB 00:00 2023-10-08T00:52:54.098 INFO:teuthology.orchestra.run.smithi184.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 2.1 MB/s | 61 kB 00:00 2023-10-08T00:52:54.111 INFO:teuthology.orchestra.run.smithi119.stdout:(16/120): libradosstriper1-17.2.6-1320.g08cac01 13 MB/s | 507 kB 00:00 2023-10-08T00:52:54.119 INFO:teuthology.orchestra.run.smithi184.stdout:(66/120): python3-webtest-2.0.33-1.el8.noarch.r 4.0 MB/s | 85 kB 00:00 2023-10-08T00:52:54.145 INFO:teuthology.orchestra.run.smithi184.stdout:(67/120): python3-zc-lockfile-2.0-2.el8.noarch. 883 kB/s | 23 kB 00:00 2023-10-08T00:52:54.194 INFO:teuthology.orchestra.run.smithi184.stdout:(68/120): thrift-0.13.0-2.el8.x86_64.rpm 36 MB/s | 1.7 MB 00:00 2023-10-08T00:52:54.199 INFO:teuthology.orchestra.run.smithi184.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 17 MB/s | 70 kB 00:00 2023-10-08T00:52:54.445 INFO:teuthology.orchestra.run.smithi184.stdout:(70/120): python3-scipy-1.0.0-20.module+el8.1.0 59 MB/s | 15 MB 00:00 2023-10-08T00:52:54.462 INFO:teuthology.orchestra.run.smithi184.stdout:(71/120): python3-prettytable-0.7.2-14.el8.noar 2.6 MB/s | 44 kB 00:00 2023-10-08T00:52:54.477 INFO:teuthology.orchestra.run.smithi184.stdout:(72/120): python3-jsonpatch-1.21-2.el8.noarch.r 1.8 MB/s | 27 kB 00:00 2023-10-08T00:52:54.493 INFO:teuthology.orchestra.run.smithi184.stdout:(73/120): jq-1.5-12.el8.x86_64.rpm 10 MB/s | 161 kB 00:00 2023-10-08T00:52:54.509 INFO:teuthology.orchestra.run.smithi119.stdout:(17/120): librgw2-17.2.6-1320.g08cac018.el8.x86 14 MB/s | 5.6 MB 00:00 2023-10-08T00:52:54.511 INFO:teuthology.orchestra.run.smithi184.stdout:(74/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-10-08T00:52:54.531 INFO:teuthology.orchestra.run.smithi184.stdout:(75/120): python3-werkzeug-0.12.2-4.el8.noarch. 23 MB/s | 457 kB 00:00 2023-10-08T00:52:54.544 INFO:teuthology.orchestra.run.smithi119.stdout:(18/120): python3-ceph-argparse-17.2.6-1320.g08 1.3 MB/s | 45 kB 00:00 2023-10-08T00:52:54.586 INFO:teuthology.orchestra.run.smithi119.stdout:(19/120): python3-ceph-common-17.2.6-1320.g08ca 2.7 MB/s | 115 kB 00:00 2023-10-08T00:52:54.616 INFO:teuthology.orchestra.run.smithi184.stdout:(76/120): python3-babel-2.5.1-5.el8.noarch.rpm 57 MB/s | 4.8 MB 00:00 2023-10-08T00:52:54.630 INFO:teuthology.orchestra.run.smithi184.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.8 MB/s | 39 kB 00:00 2023-10-08T00:52:54.631 INFO:teuthology.orchestra.run.smithi119.stdout:(20/120): python3-cephfs-17.2.6-1320.g08cac018. 4.1 MB/s | 190 kB 00:00 2023-10-08T00:52:54.652 INFO:teuthology.orchestra.run.smithi184.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 31 MB/s | 688 kB 00:00 2023-10-08T00:52:54.669 INFO:teuthology.orchestra.run.smithi184.stdout:(79/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-08T00:52:54.684 INFO:teuthology.orchestra.run.smithi184.stdout:(80/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 15 MB/s | 225 kB 00:00 2023-10-08T00:52:54.684 INFO:teuthology.orchestra.run.smithi119.stdout:(21/120): python3-rados-17.2.6-1320.g08cac018.e 6.8 MB/s | 371 kB 00:00 2023-10-08T00:52:54.697 INFO:teuthology.orchestra.run.smithi184.stdout:(81/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 9.4 MB/s | 126 kB 00:00 2023-10-08T00:52:54.740 INFO:teuthology.orchestra.run.smithi119.stdout:(22/120): python3-rbd-17.2.6-1320.g08cac018.el8 6.3 MB/s | 358 kB 00:00 2023-10-08T00:52:54.765 INFO:teuthology.orchestra.run.smithi184.stdout:(82/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 56 MB/s | 3.7 MB 00:00 2023-10-08T00:52:54.780 INFO:teuthology.orchestra.run.smithi184.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-10-08T00:52:54.904 INFO:teuthology.orchestra.run.smithi119.stdout:(23/120): python3-rgw-17.2.6-1320.g08cac018.el8 2.1 MB/s | 107 kB 00:00 2023-10-08T00:52:54.904 INFO:teuthology.orchestra.run.smithi119.stdout:(24/120): rbd-fuse-17.2.6-1320.g08cac018.el8.x8 2.1 MB/s | 90 kB 00:00 2023-10-08T00:52:54.905 INFO:teuthology.orchestra.run.smithi184.stdout:(84/120): python3-jinja2-2.10.1-2.el8_0.noarch. 27 MB/s | 538 kB 00:00 2023-10-08T00:52:54.905 INFO:teuthology.orchestra.run.smithi184.stdout:(85/120): socat-1.7.3.3-2.el8.x86_64.rpm 19 MB/s | 302 kB 00:00 2023-10-08T00:52:54.905 INFO:teuthology.orchestra.run.smithi184.stdout:(86/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-10-08T00:52:54.905 INFO:teuthology.orchestra.run.smithi184.stdout:(87/120): python3-pyasn1-modules-0.3.7-6.el8.no 8.4 MB/s | 110 kB 00:00 2023-10-08T00:52:54.905 INFO:teuthology.orchestra.run.smithi184.stdout:(88/120): librdkafka-0.11.4-1.el8.x86_64.rpm 21 MB/s | 353 kB 00:00 2023-10-08T00:52:54.906 INFO:teuthology.orchestra.run.smithi184.stdout:(89/120): python3-jsonpointer-1.10-11.el8.noarc 1.7 MB/s | 20 kB 00:00 2023-10-08T00:52:54.906 INFO:teuthology.orchestra.run.smithi184.stdout:(90/120): libtool-2.4.6-25.el8.x86_64.rpm 33 MB/s | 709 kB 00:00 2023-10-08T00:52:54.906 INFO:teuthology.orchestra.run.smithi184.stdout:(91/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.1 MB/s | 54 kB 00:00 2023-10-08T00:52:54.989 INFO:teuthology.orchestra.run.smithi184.stdout:(92/120): openblas-threads-0.3.12-1.el8.x86_64. 56 MB/s | 4.7 MB 00:00 2023-10-08T00:52:55.070 INFO:teuthology.orchestra.run.smithi184.stdout:(93/120): openblas-0.3.12-1.el8.x86_64.rpm 57 MB/s | 4.6 MB 00:00 2023-10-08T00:52:55.093 INFO:teuthology.orchestra.run.smithi184.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 31 MB/s | 713 kB 00:00 2023-10-08T00:52:55.105 INFO:teuthology.orchestra.run.smithi184.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.6 MB/s | 19 kB 00:00 2023-10-08T00:52:55.117 INFO:teuthology.orchestra.run.smithi184.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.6 MB/s | 43 kB 00:00 2023-10-08T00:52:55.130 INFO:teuthology.orchestra.run.smithi184.stdout:(97/120): python3-oauthlib-2.1.0-1.el8.noarch.r 12 MB/s | 155 kB 00:00 2023-10-08T00:52:55.145 INFO:teuthology.orchestra.run.smithi184.stdout:(98/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 15 MB/s | 220 kB 00:00 2023-10-08T00:52:55.158 INFO:teuthology.orchestra.run.smithi184.stdout:(99/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.9 MB/s | 48 kB 00:00 2023-10-08T00:52:55.172 INFO:teuthology.orchestra.run.smithi184.stdout:(100/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 16 MB/s | 229 kB 00:00 2023-10-08T00:52:55.178 INFO:teuthology.orchestra.run.smithi119.stdout:(25/120): rbd-mirror-17.2.6-1320.g08cac018.el8. 10 MB/s | 3.5 MB 00:00 2023-10-08T00:52:55.185 INFO:teuthology.orchestra.run.smithi184.stdout:(101/120): python3-requests-oauthlib-1.0.0-1.el 3.1 MB/s | 43 kB 00:00 2023-10-08T00:52:55.231 INFO:teuthology.orchestra.run.smithi119.stdout:(26/120): rbd-nbd-17.2.6-1320.g08cac018.el8.x86 3.2 MB/s | 173 kB 00:00 2023-10-08T00:52:55.232 INFO:teuthology.orchestra.run.smithi184.stdout:(102/120): openssl-devel-1.1.1g-15.el8_3.x86_64 50 MB/s | 2.3 MB 00:00 2023-10-08T00:52:55.245 INFO:teuthology.orchestra.run.smithi184.stdout:(103/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.3 MB/s | 18 kB 00:00 2023-10-08T00:52:55.271 INFO:teuthology.orchestra.run.smithi184.stdout:(104/120): pcre2-devel-10.32-2.el8.x86_64.rpm 29 MB/s | 605 kB 00:00 2023-10-08T00:52:55.279 INFO:teuthology.orchestra.run.smithi184.stdout:(105/120): fuse-2.9.7-12.el8.x86_64.rpm 6.1 MB/s | 83 kB 00:00 2023-10-08T00:52:55.280 INFO:teuthology.orchestra.run.smithi119.stdout:(27/120): ceph-grafana-dashboards-17.2.6-1320.g 467 kB/s | 23 kB 00:00 2023-10-08T00:52:55.291 INFO:teuthology.orchestra.run.smithi184.stdout:(106/120): elfutils-libelf-devel-0.182-3.el8.x8 4.8 MB/s | 59 kB 00:00 2023-10-08T00:52:55.305 INFO:teuthology.orchestra.run.smithi184.stdout:(107/120): libquadmath-8.4.1-1.el8.x86_64.rpm 12 MB/s | 169 kB 00:00 2023-10-08T00:52:55.317 INFO:teuthology.orchestra.run.smithi184.stdout:(108/120): libcom_err-devel-1.45.6-1.el8.x86_64 3.2 MB/s | 38 kB 00:00 2023-10-08T00:52:55.329 INFO:teuthology.orchestra.run.smithi184.stdout:(109/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.9 MB/s | 86 kB 00:00 2023-10-08T00:52:55.336 INFO:teuthology.orchestra.run.smithi119.stdout:(28/120): ceph-mgr-cephadm-17.2.6-1320.g08cac01 2.1 MB/s | 121 kB 00:00 2023-10-08T00:52:55.341 INFO:teuthology.orchestra.run.smithi184.stdout:(110/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 4.7 MB/s | 58 kB 00:00 2023-10-08T00:52:55.354 INFO:teuthology.orchestra.run.smithi184.stdout:(111/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.9 MB/s | 47 kB 00:00 2023-10-08T00:52:55.373 INFO:teuthology.orchestra.run.smithi184.stdout:(112/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 28 MB/s | 559 kB 00:00 2023-10-08T00:52:55.394 INFO:teuthology.orchestra.run.smithi184.stdout:(113/120): libgfortran-8.4.1-1.el8.x86_64.rpm 30 MB/s | 639 kB 00:00 2023-10-08T00:52:55.407 INFO:teuthology.orchestra.run.smithi184.stdout:(114/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 5.5 MB/s | 70 kB 00:00 2023-10-08T00:52:55.421 INFO:teuthology.orchestra.run.smithi184.stdout:(115/120): libselinux-devel-2.9-5.el8.x86_64.rp 14 MB/s | 200 kB 00:00 2023-10-08T00:52:55.434 INFO:teuthology.orchestra.run.smithi184.stdout:(116/120): libuuid-devel-2.32.1-27.el8.x86_64.r 7.5 MB/s | 97 kB 00:00 2023-10-08T00:52:55.563 INFO:teuthology.orchestra.run.smithi119.stdout:(29/120): ceph-mgr-dashboard-17.2.6-1320.g08cac 7.5 MB/s | 1.7 MB 00:00 2023-10-08T00:52:56.543 INFO:teuthology.orchestra.run.smithi184.stdout:(117/120): librados2-17.2.6-1320.g08cac018.el8. 3.1 MB/s | 3.5 MB 00:01 2023-10-08T00:52:56.549 INFO:teuthology.orchestra.run.smithi119.stdout:(30/120): ceph-mgr-diskprediction-local-17.2.6- 7.5 MB/s | 7.4 MB 00:00 2023-10-08T00:52:56.620 INFO:teuthology.orchestra.run.smithi119.stdout:(31/120): ceph-mgr-modules-core-17.2.6-1320.g08 3.3 MB/s | 238 kB 00:00 2023-10-08T00:52:56.681 INFO:teuthology.orchestra.run.smithi119.stdout:(32/120): ceph-mgr-rook-17.2.6-1320.g08cac018.e 775 kB/s | 47 kB 00:00 2023-10-08T00:52:56.741 INFO:teuthology.orchestra.run.smithi119.stdout:(33/120): ceph-prometheus-alerts-17.2.6-1320.g0 242 kB/s | 14 kB 00:00 2023-10-08T00:52:56.812 INFO:teuthology.orchestra.run.smithi119.stdout:(34/120): ceph-volume-17.2.6-1320.g08cac018.el8 3.7 MB/s | 267 kB 00:00 2023-10-08T00:52:56.878 INFO:teuthology.orchestra.run.smithi119.stdout:(35/120): cephadm-17.2.6-1320.g08cac018.el8.noa 1.2 MB/s | 80 kB 00:00 2023-10-08T00:52:56.940 INFO:teuthology.orchestra.run.smithi119.stdout:(36/120): cephfs-top-17.2.6-1320.g08cac018.el8. 291 kB/s | 18 kB 00:00 2023-10-08T00:52:57.172 INFO:teuthology.orchestra.run.smithi184.stdout:(118/120): librbd1-17.2.6-1320.g08cac018.el8.x8 5.7 MB/s | 3.6 MB 00:00 2023-10-08T00:52:57.511 INFO:teuthology.orchestra.run.smithi119.stdout:(37/120): gperftools-libs-2.7-9.el8.x86_64.rpm 536 kB/s | 306 kB 00:00 2023-10-08T00:52:57.597 INFO:teuthology.orchestra.run.smithi119.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 683 kB/s | 59 kB 00:00 2023-10-08T00:52:57.687 INFO:teuthology.orchestra.run.smithi119.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 840 kB/s | 75 kB 00:00 2023-10-08T00:52:57.849 INFO:teuthology.orchestra.run.smithi119.stdout:(40/120): python3-asyncssh-2.7.0-2.el8.noarch.r 2.9 MB/s | 476 kB 00:00 2023-10-08T00:52:57.928 INFO:teuthology.orchestra.run.smithi119.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 555 kB/s | 44 kB 00:00 2023-10-08T00:52:58.017 INFO:teuthology.orchestra.run.smithi119.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-10-08T00:52:58.095 INFO:teuthology.orchestra.run.smithi119.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 422 kB/s | 33 kB 00:00 2023-10-08T00:52:58.174 INFO:teuthology.orchestra.run.smithi119.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 211 kB/s | 16 kB 00:00 2023-10-08T00:52:58.260 INFO:teuthology.orchestra.run.smithi119.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.9 MB/s | 173 kB 00:00 2023-10-08T00:52:58.362 INFO:teuthology.orchestra.run.smithi119.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 3.7 MB/s | 384 kB 00:00 2023-10-08T00:52:58.445 INFO:teuthology.orchestra.run.smithi119.stdout:(47/120): python3-google-auth-1.1.1-10.el8.noar 989 kB/s | 81 kB 00:00 2023-10-08T00:52:58.522 INFO:teuthology.orchestra.run.smithi119.stdout:(48/120): python3-jaraco-6.2-6.el8.noarch.rpm 144 kB/s | 11 kB 00:00 2023-10-08T00:52:58.600 INFO:teuthology.orchestra.run.smithi119.stdout:(49/120): python3-jaraco-functools-2.0-4.el8.no 226 kB/s | 18 kB 00:00 2023-10-08T00:52:58.754 INFO:teuthology.orchestra.run.smithi119.stdout:(50/120): python3-kubernetes-11.0.0-6.el8.noarc 7.6 MB/s | 1.2 MB 00:00 2023-10-08T00:52:58.841 INFO:teuthology.orchestra.run.smithi119.stdout:(51/120): python3-logutils-0.3.5-11.el8.noarch. 560 kB/s | 49 kB 00:00 2023-10-08T00:52:58.921 INFO:teuthology.orchestra.run.smithi119.stdout:(52/120): python3-more-itertools-7.2.0-3.el8.no 742 kB/s | 59 kB 00:00 2023-10-08T00:52:59.000 INFO:teuthology.orchestra.run.smithi119.stdout:(53/120): python3-natsort-7.1.1-2.el8.noarch.rp 762 kB/s | 60 kB 00:00 2023-10-08T00:52:59.101 INFO:teuthology.orchestra.run.smithi119.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2023-10-08T00:52:59.179 INFO:teuthology.orchestra.run.smithi119.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 202 kB/s | 16 kB 00:00 2023-10-08T00:52:59.257 INFO:teuthology.orchestra.run.smithi119.stdout:(56/120): python3-repoze-lru-0.7-6.el8.noarch.r 431 kB/s | 34 kB 00:00 2023-10-08T00:52:59.343 INFO:teuthology.orchestra.run.smithi119.stdout:(57/120): python3-routes-2.4.1-12.el8.noarch.rp 2.2 MB/s | 196 kB 00:00 2023-10-08T00:52:59.422 INFO:teuthology.orchestra.run.smithi119.stdout:(58/120): python3-rsa-4.9-2.el8.noarch.rpm 814 kB/s | 64 kB 00:00 2023-10-08T00:52:59.499 INFO:teuthology.orchestra.run.smithi119.stdout:(59/120): python3-simplegeneric-0.8.1-17.el8.no 243 kB/s | 19 kB 00:00 2023-10-08T00:52:59.577 INFO:teuthology.orchestra.run.smithi119.stdout:(60/120): python3-singledispatch-3.4.0.3-18.el8 310 kB/s | 24 kB 00:00 2023-10-08T00:52:59.654 INFO:teuthology.orchestra.run.smithi119.stdout:(61/120): python3-tempora-1.14.1-5.el8.noarch.r 378 kB/s | 29 kB 00:00 2023-10-08T00:52:59.732 INFO:teuthology.orchestra.run.smithi119.stdout:(62/120): python3-trustme-0.6.0-4.el8.noarch.rp 356 kB/s | 27 kB 00:00 2023-10-08T00:52:59.820 INFO:teuthology.orchestra.run.smithi119.stdout:(63/120): python3-waitress-1.4.3-1.el8.noarch.r 2.8 MB/s | 255 kB 00:00 2023-10-08T00:52:59.908 INFO:teuthology.orchestra.run.smithi119.stdout:(64/120): python3-webob-1.8.5-1.el8.1.noarch.rp 2.8 MB/s | 251 kB 00:00 2023-10-08T00:52:59.964 INFO:teuthology.orchestra.run.smithi184.stdout:(119/120): ceph-mgr-diskprediction-local-17.2.6 1.1 MB/s | 7.4 MB 00:07 2023-10-08T00:52:59.987 INFO:teuthology.orchestra.run.smithi119.stdout:(65/120): python3-websocket-client-0.56.0-5.el8 778 kB/s | 61 kB 00:00 2023-10-08T00:53:00.075 INFO:teuthology.orchestra.run.smithi119.stdout:(66/120): ceph-radosgw-17.2.6-1320.g08cac018.el 1.7 MB/s | 17 MB 00:09 2023-10-08T00:53:00.076 INFO:teuthology.orchestra.run.smithi119.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 959 kB/s | 85 kB 00:00 2023-10-08T00:53:00.235 INFO:teuthology.orchestra.run.smithi119.stdout:(68/120): thrift-0.13.0-2.el8.x86_64.rpm 11 MB/s | 1.7 MB 00:00 2023-10-08T00:53:00.241 INFO:teuthology.orchestra.run.smithi119.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 17 MB/s | 70 kB 00:00 2023-10-08T00:53:00.273 INFO:teuthology.orchestra.run.smithi119.stdout:(70/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 597 kB/s | 19 kB 00:00 2023-10-08T00:53:00.288 INFO:teuthology.orchestra.run.smithi119.stdout:(71/120): python3-jwt-1.6.1-2.el8.noarch.rpm 2.9 MB/s | 43 kB 00:00 2023-10-08T00:53:00.303 INFO:teuthology.orchestra.run.smithi119.stdout:(72/120): python3-oauthlib-2.1.0-1.el8.noarch.r 10 MB/s | 155 kB 00:00 2023-10-08T00:53:00.321 INFO:teuthology.orchestra.run.smithi119.stdout:(73/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-10-08T00:53:00.336 INFO:teuthology.orchestra.run.smithi119.stdout:(74/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.2 MB/s | 48 kB 00:00 2023-10-08T00:53:00.352 INFO:teuthology.orchestra.run.smithi119.stdout:(75/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 14 MB/s | 229 kB 00:00 2023-10-08T00:53:00.366 INFO:teuthology.orchestra.run.smithi119.stdout:(76/120): python3-requests-oauthlib-1.0.0-1.el8 3.2 MB/s | 43 kB 00:00 2023-10-08T00:53:00.398 INFO:teuthology.orchestra.run.smithi119.stdout:(77/120): python3-zc-lockfile-2.0-2.el8.noarch. 71 kB/s | 23 kB 00:00 2023-10-08T00:53:00.420 INFO:teuthology.orchestra.run.smithi119.stdout:(78/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 43 MB/s | 2.3 MB 00:00 2023-10-08T00:53:00.425 INFO:teuthology.orchestra.run.smithi119.stdout:(79/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 666 kB/s | 18 kB 00:00 2023-10-08T00:53:00.440 INFO:teuthology.orchestra.run.smithi119.stdout:(80/120): fuse-2.9.7-12.el8.x86_64.rpm 5.7 MB/s | 83 kB 00:00 2023-10-08T00:53:00.445 INFO:teuthology.orchestra.run.smithi119.stdout:(81/120): pcre2-devel-10.32-2.el8.x86_64.rpm 24 MB/s | 605 kB 00:00 2023-10-08T00:53:00.454 INFO:teuthology.orchestra.run.smithi119.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 4.2 MB/s | 59 kB 00:00 2023-10-08T00:53:00.461 INFO:teuthology.orchestra.run.smithi119.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2023-10-08T00:53:00.467 INFO:teuthology.orchestra.run.smithi119.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.9 MB/s | 38 kB 00:00 2023-10-08T00:53:00.475 INFO:teuthology.orchestra.run.smithi119.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.9 MB/s | 86 kB 00:00 2023-10-08T00:53:00.484 INFO:teuthology.orchestra.run.smithi119.stdout:(86/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.6 MB/s | 58 kB 00:00 2023-10-08T00:53:00.492 INFO:teuthology.orchestra.run.smithi119.stdout:(87/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.9 MB/s | 47 kB 00:00 2023-10-08T00:53:00.504 INFO:teuthology.orchestra.run.smithi119.stdout:(88/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 26 MB/s | 559 kB 00:00 2023-10-08T00:53:00.515 INFO:teuthology.orchestra.run.smithi119.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 28 MB/s | 639 kB 00:00 2023-10-08T00:53:00.517 INFO:teuthology.orchestra.run.smithi119.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 5.4 MB/s | 70 kB 00:00 2023-10-08T00:53:00.532 INFO:teuthology.orchestra.run.smithi119.stdout:(91/120): libselinux-devel-2.9-5.el8.x86_64.rpm 12 MB/s | 200 kB 00:00 2023-10-08T00:53:00.533 INFO:teuthology.orchestra.run.smithi119.stdout:(92/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.3 MB/s | 97 kB 00:00 2023-10-08T00:53:00.547 INFO:teuthology.orchestra.run.smithi119.stdout:(93/120): python3-prettytable-0.7.2-14.el8.noar 3.0 MB/s | 44 kB 00:00 2023-10-08T00:53:00.559 INFO:teuthology.orchestra.run.smithi119.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.1 MB/s | 27 kB 00:00 2023-10-08T00:53:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:(95/120): jq-1.5-12.el8.x86_64.rpm 10 MB/s | 161 kB 00:00 2023-10-08T00:53:00.593 INFO:teuthology.orchestra.run.smithi119.stdout:(96/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-10-08T00:53:00.615 INFO:teuthology.orchestra.run.smithi119.stdout:(97/120): python3-werkzeug-0.12.2-4.el8.noarch. 21 MB/s | 457 kB 00:00 2023-10-08T00:53:00.725 INFO:teuthology.orchestra.run.smithi119.stdout:(98/120): python3-babel-2.5.1-5.el8.noarch.rpm 44 MB/s | 4.8 MB 00:00 2023-10-08T00:53:00.739 INFO:teuthology.orchestra.run.smithi119.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.8 MB/s | 39 kB 00:00 2023-10-08T00:53:00.768 INFO:teuthology.orchestra.run.smithi119.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 26 MB/s | 688 kB 00:00 2023-10-08T00:53:00.781 INFO:teuthology.orchestra.run.smithi119.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 15 MB/s | 259 kB 00:00 2023-10-08T00:53:00.828 INFO:teuthology.orchestra.run.smithi119.stdout:(102/120): python3-scipy-1.0.0-20.module+el8.1. 49 MB/s | 15 MB 00:00 2023-10-08T00:53:00.832 INFO:teuthology.orchestra.run.smithi119.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 4.4 MB/s | 225 kB 00:00 2023-10-08T00:53:00.842 INFO:teuthology.orchestra.run.smithi119.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 8.9 MB/s | 126 kB 00:00 2023-10-08T00:53:00.859 INFO:teuthology.orchestra.run.smithi119.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 11 MB/s | 187 kB 00:00 2023-10-08T00:53:00.882 INFO:teuthology.orchestra.run.smithi119.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 23 MB/s | 538 kB 00:00 2023-10-08T00:53:00.910 INFO:teuthology.orchestra.run.smithi119.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 48 MB/s | 3.7 MB 00:00 2023-10-08T00:53:00.912 INFO:teuthology.orchestra.run.smithi119.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 10 MB/s | 302 kB 00:00 2023-10-08T00:53:00.925 INFO:teuthology.orchestra.run.smithi119.stdout:(109/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-10-08T00:53:00.927 INFO:teuthology.orchestra.run.smithi119.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 7.4 MB/s | 110 kB 00:00 2023-10-08T00:53:00.939 INFO:teuthology.orchestra.run.smithi119.stdout:(111/120): python3-jsonpointer-1.10-11.el8.noar 1.6 MB/s | 20 kB 00:00 2023-10-08T00:53:00.943 INFO:teuthology.orchestra.run.smithi119.stdout:(112/120): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-10-08T00:53:00.957 INFO:teuthology.orchestra.run.smithi119.stdout:(113/120): python3-pytz-2017.2-9.el8.noarch.rpm 4.0 MB/s | 54 kB 00:00 2023-10-08T00:53:00.963 INFO:teuthology.orchestra.run.smithi119.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 30 MB/s | 709 kB 00:00 2023-10-08T00:53:01.065 INFO:teuthology.orchestra.run.smithi119.stdout:(115/120): openblas-threads-0.3.12-1.el8.x86_64 44 MB/s | 4.7 MB 00:00 2023-10-08T00:53:01.087 INFO:teuthology.orchestra.run.smithi119.stdout:(116/120): openblas-0.3.12-1.el8.x86_64.rpm 37 MB/s | 4.6 MB 00:00 2023-10-08T00:53:01.098 INFO:teuthology.orchestra.run.smithi119.stdout:(117/120): automake-1.16.1-7.el8.noarch.rpm 22 MB/s | 713 kB 00:00 2023-10-08T00:53:01.831 INFO:teuthology.orchestra.run.smithi119.stdout:(118/120): librados2-17.2.6-1320.g08cac018.el8. 4.7 MB/s | 3.5 MB 00:00 2023-10-08T00:53:02.757 INFO:teuthology.orchestra.run.smithi119.stdout:(119/120): ceph-common-17.2.6-1320.g08cac018.el 1.4 MB/s | 23 MB 00:16 2023-10-08T00:53:03.231 INFO:teuthology.orchestra.run.smithi119.stdout:(120/120): librbd1-17.2.6-1320.g08cac018.el8.x8 1.7 MB/s | 3.6 MB 00:02 2023-10-08T00:53:03.236 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-10-08T00:53:03.237 INFO:teuthology.orchestra.run.smithi119.stdout:Total 12 MB/s | 202 MB 00:17 2023-10-08T00:53:03.308 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-10-08T00:53:03.432 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-10-08T00:53:03.433 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-10-08T00:53:03.457 INFO:teuthology.orchestra.run.smithi184.stdout:(120/120): ceph-common-17.2.6-1320.g08cac018.el 1.3 MB/s | 23 MB 00:17 2023-10-08T00:53:03.461 INFO:teuthology.orchestra.run.smithi184.stdout:-------------------------------------------------------------------------------- 2023-10-08T00:53:03.462 INFO:teuthology.orchestra.run.smithi184.stdout:Total 11 MB/s | 202 MB 00:18 2023-10-08T00:53:03.532 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction check 2023-10-08T00:53:03.657 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction check succeeded. 2023-10-08T00:53:03.657 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction test 2023-10-08T00:53:05.219 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-10-08T00:53:05.224 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-10-08T00:53:05.614 INFO:teuthology.orchestra.run.smithi184.stdout:Transaction test succeeded. 2023-10-08T00:53:05.618 INFO:teuthology.orchestra.run.smithi184.stdout:Running transaction 2023-10-08T00:53:10.253 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-10-08T00:53:10.361 INFO:teuthology.orchestra.run.smithi184.stdout: Preparing : 1/1 2023-10-08T00:53:10.457 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-08T00:53:10.677 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-08T00:53:10.840 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 2/122 2023-10-08T00:53:10.913 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-10-08T00:53:11.035 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-08T00:53:11.158 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 2/122 2023-10-08T00:53:11.356 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-08T00:53:11.478 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/122 2023-10-08T00:53:11.539 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-10-08T00:53:11.710 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T00:53:11.764 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-08T00:53:12.165 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T00:53:12.232 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-10-08T00:53:12.422 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T00:53:12.585 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T00:53:12.741 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T00:53:12.877 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 5/122 2023-10-08T00:53:12.926 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:53:13.050 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:53:13.130 INFO:teuthology.orchestra.run.smithi184.stdout: Upgrading : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:53:13.249 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:53:13.294 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:53:13.513 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:53:13.602 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/122 2023-10-08T00:53:13.636 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:53:14.045 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T00:53:14.320 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-08T00:53:14.476 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:53:14.618 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:53:14.775 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/122 2023-10-08T00:53:14.875 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-08T00:53:14.988 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:53:15.094 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-08T00:53:15.128 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:53:15.285 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:53:15.378 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-10-08T00:53:15.399 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:53:15.585 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T00:53:15.678 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T00:53:15.728 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-10-08T00:53:15.850 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-10-08T00:53:16.150 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-10-08T00:53:16.188 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:53:16.307 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:53:16.355 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-10-08T00:53:16.480 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T00:53:16.543 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-10-08T00:53:16.581 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 14/122 2023-10-08T00:53:16.676 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-08T00:53:16.752 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-10-08T00:53:16.837 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-10-08T00:53:16.954 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-08T00:53:17.026 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-08T00:53:17.112 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-10-08T00:53:17.313 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-08T00:53:17.681 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-10-08T00:53:17.884 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-08T00:53:17.922 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-10-08T00:53:18.138 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-08T00:53:18.480 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-10-08T00:53:18.692 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-10-08T00:53:18.721 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-10-08T00:53:18.937 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-10-08T00:53:19.559 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-10-08T00:53:19.778 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-10-08T00:53:19.818 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-10-08T00:53:20.080 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-10-08T00:53:20.131 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-10-08T00:53:20.257 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-10-08T00:53:20.398 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-08T00:53:20.436 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-10-08T00:53:20.592 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-08T00:53:20.702 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-10-08T00:53:20.730 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-08T00:53:20.994 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-10-08T00:53:21.036 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-08T00:53:21.234 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-10-08T00:53:21.279 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-10-08T00:53:21.396 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 26/122 2023-10-08T00:53:21.541 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 27/122 2023-10-08T00:53:21.591 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-10-08T00:53:21.816 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 28/122 2023-10-08T00:53:21.834 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-08T00:53:22.945 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-10-08T00:53:23.283 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-10-08T00:53:23.611 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-10-08T00:53:23.891 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-10-08T00:53:24.152 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-10-08T00:53:24.257 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-10-08T00:53:24.361 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-08T00:53:24.663 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-10-08T00:53:25.816 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-10-08T00:53:26.172 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-10-08T00:53:26.453 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-10-08T00:53:26.681 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-08T00:53:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-10-08T00:53:27.164 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-10-08T00:53:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-08T00:53:27.164 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-08T00:53:27.298 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-08T00:53:28.507 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-08T00:53:28.507 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-10-08T00:53:28.507 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-10-08T00:53:28.507 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-10-08T00:53:28.507 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-10-08T00:53:28.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-10-08T00:53:28.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-10-08T00:53:28.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 38/122 2023-10-08T00:53:28.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 39/122 2023-10-08T00:53:28.511 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 40/122 2023-10-08T00:53:28.512 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-10-08T00:53:28.599 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-08T00:53:29.441 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-10-08T00:53:29.441 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-10-08T00:53:29.442 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-10-08T00:53:29.442 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-10-08T00:53:29.442 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-10-08T00:53:29.442 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-10-08T00:53:29.442 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-10-08T00:53:29.442 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-10-08T00:53:29.442 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-10-08T00:53:29.447 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-10-08T00:53:30.573 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-10-08T00:53:30.731 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-10-08T00:53:30.731 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-10-08T00:53:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-10-08T00:53:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-10-08T00:53:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-10-08T00:53:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-10-08T00:53:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-10-08T00:53:30.732 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-10-08T00:53:30.732 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-10-08T00:53:30.932 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-10-08T00:53:31.051 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-10-08T00:53:31.153 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-10-08T00:53:31.303 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-10-08T00:53:31.353 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-10-08T00:53:31.538 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-10-08T00:53:31.634 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-10-08T00:53:31.743 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-10-08T00:53:31.837 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-08T00:53:31.936 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-08T00:53:31.989 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-08T00:53:32.036 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-10-08T00:53:32.455 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-08T00:53:32.473 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-10-08T00:53:32.474 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-10-08T00:53:32.534 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-08T00:53:32.665 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-10-08T00:53:32.758 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-08T00:53:32.835 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-10-08T00:53:32.922 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-08T00:53:33.258 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-10-08T00:53:33.258 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-08T00:53:33.294 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-08T00:53:33.387 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-10-08T00:53:33.486 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-10-08T00:53:33.536 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-08T00:53:33.650 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-10-08T00:53:33.729 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-08T00:53:33.792 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-10-08T00:53:33.945 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-08T00:53:33.988 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-10-08T00:53:34.108 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-08T00:53:34.188 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-10-08T00:53:34.374 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-08T00:53:34.464 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 68/122 2023-10-08T00:53:34.626 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-08T00:53:34.631 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 69/122 2023-10-08T00:53:34.835 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 70/122 2023-10-08T00:53:34.913 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-08T00:53:35.067 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 71/122 2023-10-08T00:53:35.185 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-08T00:53:35.319 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 72/122 2023-10-08T00:53:35.433 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-08T00:53:35.543 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 73/122 2023-10-08T00:53:35.670 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-08T00:53:35.844 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-portend-2.6-1.el8.noarch 74/122 2023-10-08T00:53:35.930 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-08T00:53:36.120 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 75/122 2023-10-08T00:53:36.350 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 76/122 2023-10-08T00:53:36.652 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 77/122 2023-10-08T00:53:37.364 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-08T00:53:37.723 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-08T00:53:37.982 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-08T00:53:38.062 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 78/122 2023-10-08T00:53:38.307 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-08T00:53:38.424 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 79/122 2023-10-08T00:53:38.556 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-08T00:53:38.684 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 80/122 2023-10-08T00:53:38.966 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-08T00:53:39.009 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 81/122 2023-10-08T00:53:39.220 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-08T00:53:39.280 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 82/122 2023-10-08T00:53:39.403 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-08T00:53:39.648 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-08T00:53:39.668 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/122 2023-10-08T00:53:39.894 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 84/122 2023-10-08T00:53:40.076 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 85/122 2023-10-08T00:53:40.293 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 86/122 2023-10-08T00:53:46.873 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:53:47.072 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:53:47.727 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:53:47.741 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:53:47.967 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:53:48.055 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T00:53:48.196 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T00:53:48.511 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T00:53:48.642 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 87/122 2023-10-08T00:53:48.977 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T00:53:49.123 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 88/122 2023-10-08T00:53:49.308 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T00:54:07.848 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T00:54:07.848 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /dev 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /run 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /dev 2023-10-08T00:54:07.849 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:08.285 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T00:54:08.935 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 89/122 2023-10-08T00:54:08.935 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /sys 2023-10-08T00:54:08.935 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /proc 2023-10-08T00:54:08.935 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /dev 2023-10-08T00:54:08.935 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /run 2023-10-08T00:54:08.935 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /mnt 2023-10-08T00:54:08.936 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /var/tmp 2023-10-08T00:54:08.936 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /home 2023-10-08T00:54:08.936 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /tmp 2023-10-08T00:54:08.936 INFO:teuthology.orchestra.run.smithi184.stdout:skipping the directory /dev 2023-10-08T00:54:08.936 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:09.371 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T00:54:09.905 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T00:54:10.101 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T00:54:10.374 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T00:54:10.568 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T00:54:10.997 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 90/122 2023-10-08T00:54:11.022 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T00:54:11.159 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T00:54:11.214 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T00:54:11.215 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T00:54:11.487 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 91/122 2023-10-08T00:54:11.722 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T00:54:12.175 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 92/122 2023-10-08T00:54:12.354 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T00:54:12.394 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 93/122 2023-10-08T00:54:15.563 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T00:54:15.771 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T00:54:15.967 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-08T00:54:16.145 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-08T00:54:16.357 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T00:54:16.624 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T00:54:16.763 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T00:54:16.825 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T00:54:16.960 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 94/122 2023-10-08T00:54:17.144 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 95/122 2023-10-08T00:54:17.347 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 96/122 2023-10-08T00:54:17.519 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T00:54:17.783 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 97/122 2023-10-08T00:54:17.935 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T00:54:18.022 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T00:54:18.141 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T00:54:18.282 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T00:54:18.685 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-08T00:54:18.859 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T00:54:18.928 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T00:54:19.066 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T00:54:19.093 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T00:54:19.140 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 98/122 2023-10-08T00:54:19.310 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T00:54:19.450 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 99/122 2023-10-08T00:54:19.849 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 100/122 2023-10-08T00:54:20.056 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T00:54:20.125 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 101/122 2023-10-08T00:54:20.260 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T00:54:20.288 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 102/122 2023-10-08T00:54:20.808 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-08T00:54:21.041 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T00:54:21.210 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T00:54:21.349 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T00:54:21.422 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T00:54:21.608 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T00:54:22.006 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 103/122 2023-10-08T00:54:22.018 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T00:54:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T00:54:22.272 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T00:54:22.398 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 104/122 2023-10-08T00:54:22.577 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T00:54:22.650 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1320.g08cac 105/122 2023-10-08T00:54:22.811 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T00:54:23.219 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 106/122 2023-10-08T00:54:23.427 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T00:54:27.541 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T00:54:28.791 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 107/122 2023-10-08T00:54:28.839 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-08T00:54:29.146 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 108/122 2023-10-08T00:54:29.213 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-10-08T00:54:29.300 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-10-08T00:54:29.397 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 110/122 2023-10-08T00:54:29.648 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-10-08T00:54:29.806 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-10-08T00:54:29.833 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-10-08T00:54:30.028 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-08T00:54:30.178 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-08T00:54:30.340 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-08T00:54:30.371 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-10-08T00:54:30.566 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-08T00:54:30.749 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-10-08T00:54:30.817 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-08T00:54:30.986 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 111/122 2023-10-08T00:54:31.039 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-10-08T00:54:31.118 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-10-08T00:54:31.163 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-10-08T00:54:31.314 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-10-08T00:54:31.338 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-10-08T00:54:31.348 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 113/122 2023-10-08T00:54:31.499 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-10-08T00:54:31.517 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-10-08T00:54:31.518 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T00:54:31.685 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 115/122 2023-10-08T00:54:31.728 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T00:54:31.729 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:54:31.886 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 116/122 2023-10-08T00:54:32.052 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 117/122 2023-10-08T00:54:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:54:32.100 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-08T00:54:32.100 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:32.288 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-10-08T00:54:32.516 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-10-08T00:54:32.601 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-10-08T00:54:32.850 INFO:teuthology.orchestra.run.smithi184.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-10-08T00:54:32.875 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-10-08T00:54:32.875 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T00:54:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-08T00:54:32.968 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-08T00:54:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-08T00:54:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T00:54:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-08T00:54:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:54:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:54:32.969 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T00:54:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-08T00:54:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:54:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-08T00:54:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-08T00:54:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:54:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-08T00:54:32.970 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-08T00:54:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-08T00:54:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-08T00:54:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-08T00:54:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-08T00:54:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-08T00:54:32.971 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-08T00:54:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-08T00:54:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-08T00:54:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-08T00:54:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-08T00:54:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-08T00:54:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-08T00:54:32.972 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-08T00:54:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-08T00:54:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-08T00:54:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-08T00:54:32.973 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-08T00:54:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-08T00:54:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-08T00:54:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-08T00:54:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-08T00:54:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-08T00:54:32.985 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-08T00:54:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-08T00:54:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-08T00:54:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-08T00:54:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-08T00:54:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-08T00:54:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-08T00:54:32.986 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-08T00:54:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-08T00:54:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-08T00:54:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-08T00:54:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-08T00:54:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-08T00:54:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-08T00:54:32.987 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-08T00:54:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-08T00:54:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-08T00:54:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-08T00:54:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-08T00:54:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-08T00:54:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-08T00:54:32.988 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-08T00:54:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-08T00:54:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-08T00:54:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-08T00:54:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-08T00:54:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-08T00:54:32.989 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-08T00:54:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-08T00:54:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-08T00:54:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-08T00:54:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-08T00:54:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-08T00:54:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-08T00:54:32.990 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-10-08T00:54:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-10-08T00:54:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-10-08T00:54:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-10-08T00:54:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-10-08T00:54:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-10-08T00:54:32.991 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-10-08T00:54:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-10-08T00:54:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-10-08T00:54:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-10-08T00:54:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-10-08T00:54:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-10-08T00:54:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-10-08T00:54:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-10-08T00:54:32.992 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-10-08T00:54:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-10-08T00:54:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-10-08T00:54:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-10-08T00:54:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-10-08T00:54:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-10-08T00:54:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-10-08T00:54:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-10-08T00:54:32.993 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-10-08T00:54:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-10-08T00:54:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-10-08T00:54:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-10-08T00:54:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-10-08T00:54:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-10-08T00:54:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-10-08T00:54:32.994 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-10-08T00:54:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-10-08T00:54:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-10-08T00:54:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-10-08T00:54:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-10-08T00:54:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-10-08T00:54:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-10-08T00:54:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-10-08T00:54:32.995 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-10-08T00:54:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-10-08T00:54:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-10-08T00:54:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-10-08T00:54:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-10-08T00:54:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-10-08T00:54:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-10-08T00:54:32.996 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-10-08T00:54:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-10-08T00:54:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-10-08T00:54:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-08T00:54:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-08T00:54:32.997 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-08T00:54:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-10-08T00:54:33.115 INFO:teuthology.orchestra.run.smithi184.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:54:33.504 INFO:teuthology.orchestra.run.smithi184.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:54:33.504 INFO:teuthology.orchestra.run.smithi184.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-10-08T00:54:33.504 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:34.382 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-2:17.2.6-1320.g08cac018.el8.x86_64 1/122 2023-10-08T00:54:34.382 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2/122 2023-10-08T00:54:34.382 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 3/122 2023-10-08T00:54:34.382 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 4/122 2023-10-08T00:54:34.382 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1320.g08cac 5/122 2023-10-08T00:54:34.382 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 6/122 2023-10-08T00:54:34.382 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 7/122 2023-10-08T00:54:34.383 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 8/122 2023-10-08T00:54:34.383 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 9/122 2023-10-08T00:54:34.383 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 10/122 2023-10-08T00:54:34.383 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 11/122 2023-10-08T00:54:34.383 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 12/122 2023-10-08T00:54:34.383 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 13/122 2023-10-08T00:54:34.383 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_ 14/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 15/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 16/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados-devel-2:17.2.6-1320.g08cac018.el8.x86_6 17/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86 18/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 19/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1320.g08cac018.el 20/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-ceph-common-2:17.2.6-1320.g08cac018.el8. 21/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_6 22/122 2023-10-08T00:54:34.384 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 23/122 2023-10-08T00:54:34.385 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 24/122 2023-10-08T00:54:34.385 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 25/122 2023-10-08T00:54:34.385 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 26/122 2023-10-08T00:54:34.385 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 27/122 2023-10-08T00:54:34.395 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 28/122 2023-10-08T00:54:34.395 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1320.g08cac018. 29/122 2023-10-08T00:54:34.395 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noa 30/122 2023-10-08T00:54:34.395 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.n 31/122 2023-10-08T00:54:34.395 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1320.g08c 32/122 2023-10-08T00:54:34.395 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el 33/122 2023-10-08T00:54:34.395 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 34/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.e 35/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 36/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephadm-2:17.2.6-1320.g08cac018.el8.noarch 37/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 38/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-10-08T00:54:34.396 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/122 2023-10-08T00:54:34.397 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/122 2023-10-08T00:54:34.398 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-10-08T00:54:34.399 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-10-08T00:54:34.400 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-10-08T00:54:34.401 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-10-08T00:54:34.402 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-10-08T00:54:34.403 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-10-08T00:54:34.404 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-10-08T00:54:34.404 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-10-08T00:54:34.404 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-10-08T00:54:34.404 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-10-08T00:54:34.404 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-10-08T00:54:34.404 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-2:17.2.6-1320.g08cac018.el8.x86_64 119/122 2023-10-08T00:54:34.404 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-10-08T00:54:34.405 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 121/122 2023-10-08T00:54:35.345 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:54:35.345 INFO:teuthology.orchestra.run.smithi119.stdout:Last metadata expiration check: 0:01:59 ago on Sun 08 Oct 2023 12:52:36 AM UTC. 2023-10-08T00:54:36.832 INFO:teuthology.orchestra.run.smithi184.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-10-08T00:54:36.832 INFO:teuthology.orchestra.run.smithi184.stdout:Last metadata expiration check: 0:02:00 ago on Sun 08 Oct 2023 12:52:36 AM UTC. 2023-10-08T00:54:37.728 INFO:teuthology.orchestra.run.smithi119.stdout:Installed products updated. 2023-10-08T00:54:37.859 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:37.859 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-10-08T00:54:37.859 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.859 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.859 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-10-08T00:54:37.859 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T00:54:37.859 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.860 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.861 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.5-12.el8.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.862 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T00:54:37.863 INFO:teuthology.orchestra.run.smithi119.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T00:54:37.864 INFO:teuthology.orchestra.run.smithi119.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T00:54:37.865 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T00:54:37.866 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T00:54:37.866 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T00:54:37.866 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-08T00:54:37.866 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T00:54:37.866 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T00:54:37.866 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T00:54:37.867 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-08T00:54:37.868 INFO:teuthology.orchestra.run.smithi119.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:37.869 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-08T00:54:37.870 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T00:54:37.870 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-08T00:54:37.870 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T00:54:37.871 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-08T00:54:37.871 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:37.872 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-10-08T00:54:38.259 DEBUG:teuthology.parallel:result is None 2023-10-08T00:54:39.182 INFO:teuthology.orchestra.run.smithi184.stdout:Installed products updated. 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout:Upgraded: 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: librados2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: librbd1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout:Installed: 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: automake-1.16.1-7.el8.noarch 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: bison-3.0.4-10.el8.x86_64 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-base-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.313 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-grafana-dashboards-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-immutable-object-cache-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mds-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-dashboard-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-modules-core-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mgr-rook-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-mon-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.314 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-osd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-prometheus-alerts-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-radosgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-selinux-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-test-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: ceph-volume-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: cephadm-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: cephfs-top-2:17.2.6-1320.g08cac018.el8.noarch 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: flex-2.6.1-9.el8.x86_64 2023-10-08T00:54:39.315 INFO:teuthology.orchestra.run.smithi184.stdout: fuse-2.9.7-12.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: jq-1.5-12.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: libcephfs2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: libcephsqlite-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-08T00:54:39.316 INFO:teuthology.orchestra.run.smithi184.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: librados-devel-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: libradosstriper1-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: librgw2-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-10-08T00:54:39.317 INFO:teuthology.orchestra.run.smithi184.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-0.3.12-1.el8.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-babel-2.5.1-5.el8.noarch 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-08T00:54:39.318 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-argparse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-ceph-common-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cephfs-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-08T00:54:39.319 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-mako-1.0.6-13.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-08T00:54:39.320 INFO:teuthology.orchestra.run.smithi184.stdout: python3-portend-2.6-1.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-pytz-2017.2-9.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rados-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rgw-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-08T00:54:39.321 INFO:teuthology.orchestra.run.smithi184.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-10-08T00:54:39.322 INFO:teuthology.orchestra.run.smithi184.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-08T00:54:39.322 INFO:teuthology.orchestra.run.smithi184.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-08T00:54:39.322 INFO:teuthology.orchestra.run.smithi184.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-08T00:54:39.322 INFO:teuthology.orchestra.run.smithi184.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-08T00:54:39.322 INFO:teuthology.orchestra.run.smithi184.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-08T00:54:39.322 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-08T00:54:39.322 INFO:teuthology.orchestra.run.smithi184.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-fuse-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-mirror-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: rbd-nbd-2:17.2.6-1320.g08cac018.el8.x86_64 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: socat-1.7.3.3-2.el8.x86_64 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-10-08T00:54:39.323 INFO:teuthology.orchestra.run.smithi184.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-08T00:54:39.324 INFO:teuthology.orchestra.run.smithi184.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-10-08T00:54:39.324 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:39.324 INFO:teuthology.orchestra.run.smithi184.stdout:Complete! 2023-10-08T00:54:39.622 DEBUG:teuthology.parallel:result is None 2023-10-08T00:54:39.623 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:54:39.852 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T00:54:39.902 INFO:teuthology.orchestra.run.smithi119.stdout:17.2.6-1320.g08cac018.el8 2023-10-08T00:54:39.902 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-08T00:54:39.902 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-08T00:54:39.903 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=08cac0187919fc3a247efdf445f652e2fb30a647 2023-10-08T00:54:40.129 DEBUG:teuthology.orchestra.run.smithi184:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-08T00:54:40.178 INFO:teuthology.orchestra.run.smithi184.stdout:17.2.6-1320.g08cac018.el8 2023-10-08T00:54:40.178 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1320.g08cac018.el8 2023-10-08T00:54:40.178 INFO:teuthology.task.install:The correct ceph version 17.2.6-1320.g08cac018 is installed. 2023-10-08T00:54:40.179 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-08T00:54:40.179 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:40.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T00:54:40.260 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:54:40.260 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-08T00:54:40.343 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-08T00:54:40.343 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:40.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T00:54:40.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T00:54:40.504 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:54:40.505 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/daemon-helper 2023-10-08T00:54:40.569 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-08T00:54:40.672 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-08T00:54:40.673 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:40.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T00:54:40.732 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T00:54:40.831 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:54:40.831 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-08T00:54:40.894 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-08T00:54:40.995 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-08T00:54:40.995 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:40.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T00:54:41.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T00:54:41.156 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:54:41.156 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/usr/bin/stdin-killer 2023-10-08T00:54:41.217 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-08T00:54:41.318 INFO:teuthology.run_tasks:Running task ceph... 2023-10-08T00:54:41.384 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-08T00:54:41.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2023-10-08T00:54:41.389 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 777 /var/log/ceph 2023-10-08T00:54:41.450 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-08T00:54:41.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T00:54:41.454 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-08T00:54:41.514 INFO:tasks.ceph:Creating extra log directories... 2023-10-08T00:54:41.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T00:54:41.519 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-08T00:54:41.586 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-08T00:54:41.586 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-08T00:54:41.586 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-08T00:54:41.586 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:41.587 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T00:54:41.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T00:54:42.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-08T00:54:42.496 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:54:42.496 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-08T00:54:42.535 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-08T00:54:42.976 DEBUG:teuthology.orchestra.run.smithi184:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-08T00:54:43.045 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-08T00:54:43.045 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-08T00:54:43.045 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151', 'branch': 'wip-yuri-testing-2023-10-06-0949-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7418151', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-06-0949-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-07-1508-quincy', 'sha1': 'abf19fc7a04fe4fceab4e98d4013ae1da7c06006'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 74, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8174, 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-07-1508-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_abf19fc7a04fe4fceab4e98d4013ae1da7c06006/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'abf19fc7a04fe4fceab4e98d4013ae1da7c06006', 'targets': {'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB0cPAV/RG2BkvrxzZHnNdQyZ/3qQ2le4eTUMsncY32AFChDBT4VBnZ39y8jKIMU159V4Mr8ZGHz43X4cptSBhw=', 'smithi184.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHKxQN3H142f/b14dRMLiTxs+qYVY/Qjhf0KIIvtk8nU49zt4ybtEXjzkUV3+nkQpiaJ0GXj9y8cgd6QJbyB3X8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '08cac0187919fc3a247efdf445f652e2fb30a647', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-07_22:15:20', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-08T00:54:43.046 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T00:54:43.051 DEBUG:teuthology.orchestra.run.smithi184:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-08T00:54:43.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T00:54:43.131 DEBUG:teuthology.orchestra.run.smithi184:> sudo install -d -m0777 -- /var/run/ceph 2023-10-08T00:54:43.193 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:43.193 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T00:54:43.229 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T00:54:43.230 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2023-10-08T00:54:43.307 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T00:54:43.308 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:54:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-08T00:54:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:54:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:54:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-10-08 00:54:32.414628340 +0000 2023-10-08T00:54:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-10-08 00:51:40.277998731 +0000 2023-10-08T00:54:43.308 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-10-08 00:51:40.277998731 +0000 2023-10-08T00:54:43.308 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2023-10-08T00:54:43.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T00:54:43.405 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-10-08T00:54:43.405 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-10-08T00:54:43.405 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000389702 s, 1.3 MB/s 2023-10-08T00:54:43.408 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T00:54:43.446 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2023-10-08T00:54:43.522 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T00:54:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:54:43.523 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-08T00:54:43.523 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:54:43.523 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:54:43.523 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-10-08 00:54:32.415628316 +0000 2023-10-08T00:54:43.523 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-10-08 00:51:41.058980456 +0000 2023-10-08T00:54:43.523 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-10-08 00:51:41.058980456 +0000 2023-10-08T00:54:43.523 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2023-10-08T00:54:43.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T00:54:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-10-08T00:54:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-10-08T00:54:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000470753 s, 1.1 MB/s 2023-10-08T00:54:43.623 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T00:54:43.660 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2023-10-08T00:54:43.737 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T00:54:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:54:43.738 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-08T00:54:43.738 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:54:43.738 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:54:43.738 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-10-08 00:54:32.415628316 +0000 2023-10-08T00:54:43.738 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-10-08 00:51:41.828962438 +0000 2023-10-08T00:54:43.738 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-10-08 00:51:41.828962438 +0000 2023-10-08T00:54:43.738 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2023-10-08T00:54:43.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T00:54:43.834 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-10-08T00:54:43.834 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-10-08T00:54:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000317428 s, 1.6 MB/s 2023-10-08T00:54:43.837 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T00:54:43.875 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2023-10-08T00:54:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T00:54:43.952 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:54:43.952 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-08T00:54:43.952 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:54:43.952 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:54:43.953 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-10-08 00:54:32.415628316 +0000 2023-10-08T00:54:43.953 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-10-08 00:51:42.585944725 +0000 2023-10-08T00:54:43.953 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-10-08 00:51:42.585944725 +0000 2023-10-08T00:54:43.953 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: - 2023-10-08T00:54:43.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T00:54:44.048 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-10-08T00:54:44.049 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-10-08T00:54:44.049 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000327853 s, 1.6 MB/s 2023-10-08T00:54:44.051 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T00:54:44.089 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:54:44.090 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:54:44.090 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/scratch_devs of=/dev/stdout 2023-10-08T00:54:44.127 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-08T00:54:44.127 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_1 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 452 Links: 1 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-08 00:54:33.834555704 +0000 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-08 00:51:42.406674457 +0000 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-08 00:51:42.406674457 +0000 2023-10-08T00:54:44.204 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-08T00:54:44.205 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-08T00:54:44.301 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-08T00:54:44.301 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-08T00:54:44.301 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000380303 s, 1.3 MB/s 2023-10-08T00:54:44.304 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-08T00:54:44.341 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_2 2023-10-08T00:54:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-08T00:54:44.421 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:54:44.422 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 458 Links: 1 2023-10-08T00:54:44.422 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:54:44.422 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:54:44.422 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-08 00:54:33.835555685 +0000 2023-10-08T00:54:44.422 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-08 00:51:43.171659629 +0000 2023-10-08T00:54:44.422 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-08 00:51:43.171659629 +0000 2023-10-08T00:54:44.422 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-08T00:54:44.423 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-08T00:54:44.519 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-08T00:54:44.520 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-08T00:54:44.520 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.00049896 s, 1.0 MB/s 2023-10-08T00:54:44.521 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-08T00:54:44.559 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_3 2023-10-08T00:54:44.636 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-08T00:54:44.636 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:54:44.636 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 464 Links: 1 2023-10-08T00:54:44.637 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:54:44.637 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:54:44.637 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-08 00:54:33.835555685 +0000 2023-10-08T00:54:44.637 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-08 00:51:43.937644783 +0000 2023-10-08T00:54:44.637 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-08 00:51:43.937644783 +0000 2023-10-08T00:54:44.637 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-08T00:54:44.637 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-08T00:54:44.737 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-08T00:54:44.738 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-08T00:54:44.738 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000338648 s, 1.5 MB/s 2023-10-08T00:54:44.741 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-08T00:54:44.783 DEBUG:teuthology.orchestra.run.smithi184:> stat /dev/vg_nvme/lv_4 2023-10-08T00:54:44.866 INFO:teuthology.orchestra.run.smithi184.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-08T00:54:44.866 INFO:teuthology.orchestra.run.smithi184.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-08T00:54:44.867 INFO:teuthology.orchestra.run.smithi184.stdout:Device: 5h/5d Inode: 470 Links: 1 2023-10-08T00:54:44.867 INFO:teuthology.orchestra.run.smithi184.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-08T00:54:44.867 INFO:teuthology.orchestra.run.smithi184.stdout:Context: system_u:object_r:device_t:s0 2023-10-08T00:54:44.867 INFO:teuthology.orchestra.run.smithi184.stdout:Access: 2023-10-08 00:54:33.835555685 +0000 2023-10-08T00:54:44.867 INFO:teuthology.orchestra.run.smithi184.stdout:Modify: 2023-10-08 00:51:44.708629841 +0000 2023-10-08T00:54:44.867 INFO:teuthology.orchestra.run.smithi184.stdout:Change: 2023-10-08 00:51:44.708629841 +0000 2023-10-08T00:54:44.867 INFO:teuthology.orchestra.run.smithi184.stdout: Birth: - 2023-10-08T00:54:44.867 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-08T00:54:44.970 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records in 2023-10-08T00:54:44.971 INFO:teuthology.orchestra.run.smithi184.stderr:1+0 records out 2023-10-08T00:54:44.971 INFO:teuthology.orchestra.run.smithi184.stderr:512 bytes copied, 0.000516352 s, 992 kB/s 2023-10-08T00:54:44.973 DEBUG:teuthology.orchestra.run.smithi184:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-08T00:54:45.019 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:54:45.019 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T00:54:45.020 INFO:tasks.ceph:Generating config... 2023-10-08T00:54:45.021 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-08T00:54:45.021 INFO:tasks.ceph:[client] debug client = 20 2023-10-08T00:54:45.021 INFO:tasks.ceph:[client] debug ms = 1 2023-10-08T00:54:45.021 INFO:tasks.ceph:[client] pid file = 2023-10-08T00:54:45.021 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-08T00:54:45.022 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-08T00:54:45.022 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-10-08T00:54:45.022 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-08T00:54:45.022 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-08T00:54:45.022 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-08T00:54:45.022 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-08T00:54:45.022 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-08T00:54:45.022 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-08T00:54:45.023 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-08T00:54:45.024 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-08T00:54:45.024 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-08T00:54:45.024 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-08T00:54:45.024 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-08T00:54:45.024 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-08T00:54:45.024 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-08T00:54:45.024 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-08T00:54:45.024 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-08T00:54:45.024 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-08T00:54:45.025 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-08T00:54:45.025 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-08T00:54:45.025 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-08T00:54:45.025 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-08T00:54:45.025 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-08T00:54:45.025 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-08T00:54:45.025 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-08T00:54:45.025 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-08T00:54:45.025 INFO:tasks.ceph:Setting up mon.a... 2023-10-08T00:54:45.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-08T00:54:45.124 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.keyring 2023-10-08T00:54:45.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-08T00:54:45.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T00:54:45.275 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.119'), ('mon.b', '172.21.15.184'), ('mon.c', '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]')] 2023-10-08T00:54:45.276 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.119,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-08T00:54:45.277 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:45.277 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T00:54:45.314 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.119 --add b 172.21.15.184 --addv c '[v2:172.21.15.184:3301,v1:172.21.15.184:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:54:45.429 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:54:45.430 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: generated fsid 260cffbb-ac40-4cab-945b-d7e76ccdf950 2023-10-08T00:54:45.430 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = octopus 2023-10-08T00:54:45.430 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 0 2023-10-08T00:54:45.431 INFO:teuthology.orchestra.run.smithi119.stdout:fsid 260cffbb-ac40-4cab-945b-d7e76ccdf950 2023-10-08T00:54:45.431 INFO:teuthology.orchestra.run.smithi119.stdout:last_changed 2023-10-08T00:54:45.429887+0000 2023-10-08T00:54:45.431 INFO:teuthology.orchestra.run.smithi119.stdout:created 2023-10-08T00:54:45.429887+0000 2023-10-08T00:54:45.431 INFO:teuthology.orchestra.run.smithi119.stdout:min_mon_release 15 (octopus) 2023-10-08T00:54:45.432 INFO:teuthology.orchestra.run.smithi119.stdout:election_strategy: 1 2023-10-08T00:54:45.432 INFO:teuthology.orchestra.run.smithi119.stdout:0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.a 2023-10-08T00:54:45.432 INFO:teuthology.orchestra.run.smithi119.stdout:1: [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] mon.b 2023-10-08T00:54:45.432 INFO:teuthology.orchestra.run.smithi119.stdout:2: [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] mon.c 2023-10-08T00:54:45.432 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-08T00:54:45.435 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-08T00:54:45.470 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 260cffbb-ac40-4cab-945b-d7e76ccdf950... 2023-10-08T00:54:45.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T00:54:45.517 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-08T00:54:45.622 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2023-10-08T00:54:45.623 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2023-10-08T00:54:45.623 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T00:54:45.623 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2023-10-08T00:54:45.623 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.623 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2023-10-08T00:54:45.623 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.623 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2023-10-08T00:54:45.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.624 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2023-10-08T00:54:45.624 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2023-10-08T00:54:45.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.624 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2023-10-08T00:54:45.624 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2023-10-08T00:54:45.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.624 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2023-10-08T00:54:45.625 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T00:54:45.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2023-10-08T00:54:45.627 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = 260cffbb-ac40-4cab-945b-d7e76ccdf950 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2023-10-08T00:54:45.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2023-10-08T00:54:45.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2023-10-08T00:54:45.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2023-10-08T00:54:45.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2023-10-08T00:54:45.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2023-10-08T00:54:45.629 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2023-10-08T00:54:45.630 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2023-10-08T00:54:45.631 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.631 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2023-10-08T00:54:45.631 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2023-10-08T00:54:45.632 INFO:teuthology.orchestra.run.smithi119.stdout: bdev async discard = True 2023-10-08T00:54:45.632 INFO:teuthology.orchestra.run.smithi119.stdout: bdev enable discard = True 2023-10-08T00:54:45.632 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore allocator = bitmap 2023-10-08T00:54:45.632 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2023-10-08T00:54:45.633 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2023-10-08T00:54:45.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 1/20 2023-10-08T00:54:45.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 1/20 2023-10-08T00:54:45.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-10-08T00:54:45.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2023-10-08T00:54:45.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 4/10 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-10-08T00:54:45.634 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout: debug finisher = 20 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.635 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2023-10-08T00:54:45.636 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T00:54:45.637 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T00:54:45.638 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.638 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2023-10-08T00:54:45.638 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2023-10-08T00:54:45.638 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.638 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2023-10-08T00:54:45.638 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2023-10-08T00:54:45.638 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:54:45.638 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-10-08T00:54:45.639 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2023-10-08T00:54:45.640 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-10-08T00:54:45.641 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2023-10-08T00:54:45.641 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2023-10-08T00:54:45.641 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2023-10-08T00:54:45.641 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2023-10-08T00:54:45.641 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-10-08T00:54:45.641 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-10-08T00:54:45.641 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-10-08T00:54:45.642 INFO:teuthology.orchestra.run.smithi184.stdout:[global] 2023-10-08T00:54:45.642 INFO:teuthology.orchestra.run.smithi184.stdout: chdir = "" 2023-10-08T00:54:45.642 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-08T00:54:45.642 INFO:teuthology.orchestra.run.smithi184.stdout: auth supported = cephx 2023-10-08T00:54:45.642 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.642 INFO:teuthology.orchestra.run.smithi184.stdout: filestore xattr use omap = true 2023-10-08T00:54:45.642 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.642 INFO:teuthology.orchestra.run.smithi184.stdout: mon clock drift allowed = 1.000 2023-10-08T00:54:45.643 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.643 INFO:teuthology.orchestra.run.smithi184.stdout: osd crush chooseleaf type = 0 2023-10-08T00:54:45.643 INFO:teuthology.orchestra.run.smithi184.stdout: auth debug = true 2023-10-08T00:54:45.643 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.643 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on old message = true 2023-10-08T00:54:45.643 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on bug = true 2023-10-08T00:54:45.643 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.643 INFO:teuthology.orchestra.run.smithi184.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: mon pg warn max object skew = 0 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: # disable pg_autoscaler by default for new pools 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default size = 2 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow primary affinity = true 2023-10-08T00:54:45.644 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd allow pg remap = true 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on legacy crush tunables = false 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on crush straw calc version zero = false 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on no sortbitwise = false 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on osd down out interval zero = false 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: mon warn on too few osds = false 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: mon_allow_pool_size_one = true 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.645 INFO:teuthology.orchestra.run.smithi184.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-08T00:54:45.646 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.646 INFO:teuthology.orchestra.run.smithi184.stdout: osd default data pool replay window = 5 2023-10-08T00:54:45.646 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.646 INFO:teuthology.orchestra.run.smithi184.stdout: mon allow pool delete = true 2023-10-08T00:54:45.646 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.646 INFO:teuthology.orchestra.run.smithi184.stdout: mon cluster log file level = debug 2023-10-08T00:54:45.646 INFO:teuthology.orchestra.run.smithi184.stdout: debug asserts on shutdown = true 2023-10-08T00:54:45.647 INFO:teuthology.orchestra.run.smithi184.stdout: mon health detail to clog = false 2023-10-08T00:54:45.647 INFO:teuthology.orchestra.run.smithi184.stdout: mon host = "172.21.15.119,172.21.15.184,[v2:172.21.15.184:3301,v1:172.21.15.184:6790]" 2023-10-08T00:54:45.647 INFO:teuthology.orchestra.run.smithi184.stdout: ms die on skipped message = False 2023-10-08T00:54:45.647 INFO:teuthology.orchestra.run.smithi184.stdout: fsid = 260cffbb-ac40-4cab-945b-d7e76ccdf950 2023-10-08T00:54:45.647 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.648 INFO:teuthology.orchestra.run.smithi184.stdout:[osd] 2023-10-08T00:54:45.648 INFO:teuthology.orchestra.run.smithi184.stdout: osd journal size = 100 2023-10-08T00:54:45.648 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.648 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub load threshold = 5.0 2023-10-08T00:54:45.648 INFO:teuthology.orchestra.run.smithi184.stdout: osd scrub max interval = 600 2023-10-08T00:54:45.649 INFO:teuthology.orchestra.run.smithi184.stdout: osd mclock profile = high_recovery_ops 2023-10-08T00:54:45.649 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.649 INFO:teuthology.orchestra.run.smithi184.stdout: osd recover clone overlap = true 2023-10-08T00:54:45.649 INFO:teuthology.orchestra.run.smithi184.stdout: osd recovery max chunk = 1048576 2023-10-08T00:54:45.649 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.650 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug shutdown = true 2023-10-08T00:54:45.650 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug op order = true 2023-10-08T00:54:45.650 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug verify stray on activate = true 2023-10-08T00:54:45.650 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.650 INFO:teuthology.orchestra.run.smithi184.stdout: osd open classes on start = true 2023-10-08T00:54:45.651 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug pg log writeout = true 2023-10-08T00:54:45.651 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.651 INFO:teuthology.orchestra.run.smithi184.stdout: osd deep scrub update digest min age = 30 2023-10-08T00:54:45.651 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.651 INFO:teuthology.orchestra.run.smithi184.stdout: osd map max advance = 10 2023-10-08T00:54:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: journal zero on create = true 2023-10-08T00:54:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: filestore ondisk finisher threads = 3 2023-10-08T00:54:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: filestore apply finisher threads = 3 2023-10-08T00:54:45.652 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.653 INFO:teuthology.orchestra.run.smithi184.stdout: bdev debug aio = true 2023-10-08T00:54:45.653 INFO:teuthology.orchestra.run.smithi184.stdout: osd debug misdirected ops = true 2023-10-08T00:54:45.653 INFO:teuthology.orchestra.run.smithi184.stdout: bdev async discard = True 2023-10-08T00:54:45.653 INFO:teuthology.orchestra.run.smithi184.stdout: bdev enable discard = True 2023-10-08T00:54:45.653 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore allocator = bitmap 2023-10-08T00:54:45.653 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore block size = 96636764160 2023-10-08T00:54:45.654 INFO:teuthology.orchestra.run.smithi184.stdout: bluestore fsck on mount = True 2023-10-08T00:54:45.654 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluefs = 1/20 2023-10-08T00:54:45.654 INFO:teuthology.orchestra.run.smithi184.stdout: debug bluestore = 1/20 2023-10-08T00:54:45.654 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-08T00:54:45.654 INFO:teuthology.orchestra.run.smithi184.stdout: debug osd = 20 2023-10-08T00:54:45.654 INFO:teuthology.orchestra.run.smithi184.stdout: debug rocksdb = 4/10 2023-10-08T00:54:45.655 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd backfillfull_ratio = 0.85 2023-10-08T00:54:45.655 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd full ratio = 0.9 2023-10-08T00:54:45.655 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd nearfull ratio = 0.8 2023-10-08T00:54:45.655 INFO:teuthology.orchestra.run.smithi184.stdout: osd failsafe full ratio = 0.95 2023-10-08T00:54:45.655 INFO:teuthology.orchestra.run.smithi184.stdout: osd objectstore = bluestore 2023-10-08T00:54:45.655 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-10-08T00:54:45.656 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.656 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr] 2023-10-08T00:54:45.656 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-08T00:54:45.656 INFO:teuthology.orchestra.run.smithi184.stdout: debug mgr = 20 2023-10-08T00:54:45.656 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-10-08T00:54:45.656 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-10-08T00:54:45.657 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-10-08T00:54:45.657 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-10-08T00:54:45.657 INFO:teuthology.orchestra.run.smithi184.stdout: mgr/telemetry/nag = false 2023-10-08T00:54:45.657 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2023-10-08T00:54:45.657 INFO:teuthology.orchestra.run.smithi184.stdout: debug finisher = 20 2023-10-08T00:54:45.658 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.658 INFO:teuthology.orchestra.run.smithi184.stdout:[mon] 2023-10-08T00:54:45.658 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-08T00:54:45.658 INFO:teuthology.orchestra.run.smithi184.stdout: debug mon = 20 2023-10-08T00:54:45.658 INFO:teuthology.orchestra.run.smithi184.stdout: debug paxos = 20 2023-10-08T00:54:45.658 INFO:teuthology.orchestra.run.smithi184.stdout: debug auth = 20 2023-10-08T00:54:45.658 INFO:teuthology.orchestra.run.smithi184.stdout: mon data avail warn = 5 2023-10-08T00:54:45.659 INFO:teuthology.orchestra.run.smithi184.stdout: mon mgr mkfs grace = 240 2023-10-08T00:54:45.659 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min pgs per osd = 4 2023-10-08T00:54:45.659 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd reporter subtree level = osd 2023-10-08T00:54:45.659 INFO:teuthology.orchestra.run.smithi184.stdout: mon osd prime pg temp = true 2023-10-08T00:54:45.659 INFO:teuthology.orchestra.run.smithi184.stdout: mon reweight min bytes per osd = 10 2023-10-08T00:54:45.660 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.660 INFO:teuthology.orchestra.run.smithi184.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-08T00:54:45.660 INFO:teuthology.orchestra.run.smithi184.stdout: auth mon ticket ttl = 660 # 11m 2023-10-08T00:54:45.660 INFO:teuthology.orchestra.run.smithi184.stdout: auth service ticket ttl = 240 # 4m 2023-10-08T00:54:45.660 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.660 INFO:teuthology.orchestra.run.smithi184.stdout: # don't complain about insecure global_id in the test suite 2023-10-08T00:54:45.661 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-08T00:54:45.661 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-08T00:54:45.661 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.661 INFO:teuthology.orchestra.run.smithi184.stdout: # 1m isn't quite enough 2023-10-08T00:54:45.662 INFO:teuthology.orchestra.run.smithi184.stdout: mon_down_mkfs_grace = 2m 2023-10-08T00:54:45.662 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.662 INFO:teuthology.orchestra.run.smithi184.stdout: mon_warn_on_filestore_osds = false 2023-10-08T00:54:45.662 INFO:teuthology.orchestra.run.smithi184.stdout: mon op complaint time = 120 2023-10-08T00:54:45.663 INFO:teuthology.orchestra.run.smithi184.stdout: 2023-10-08T00:54:45.663 INFO:teuthology.orchestra.run.smithi184.stdout:[client] 2023-10-08T00:54:45.663 INFO:teuthology.orchestra.run.smithi184.stdout: rgw cache enabled = true 2023-10-08T00:54:45.663 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable ops log = true 2023-10-08T00:54:45.663 INFO:teuthology.orchestra.run.smithi184.stdout: rgw enable usage log = true 2023-10-08T00:54:45.664 INFO:teuthology.orchestra.run.smithi184.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-08T00:54:45.664 INFO:teuthology.orchestra.run.smithi184.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-08T00:54:45.664 INFO:teuthology.orchestra.run.smithi184.stdout: client mount timeout = 600 2023-10-08T00:54:45.664 INFO:teuthology.orchestra.run.smithi184.stdout: debug client = 20 2023-10-08T00:54:45.665 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-08T00:54:45.665 INFO:teuthology.orchestra.run.smithi184.stdout: pid file = "" 2023-10-08T00:54:45.666 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-10-08T00:54:45.666 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-10-08T00:54:45.666 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.a] 2023-10-08T00:54:45.666 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.b] 2023-10-08T00:54:45.667 INFO:teuthology.orchestra.run.smithi184.stdout:[mon.c] 2023-10-08T00:54:45.667 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2023-10-08T00:54:45.667 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2023-10-08T00:54:45.667 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2023-10-08T00:54:45.667 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2023-10-08T00:54:45.667 INFO:teuthology.orchestra.run.smithi184.stdout:[mds] 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: debug mds = 20 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: debug ms = 1 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug frag = True 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: mds debug scatterstat = True 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: mds op complaint time = 180 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: mds verify scatter = True 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: osd op complaint time = 180 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: rados mon op timeout = 900 2023-10-08T00:54:45.668 INFO:teuthology.orchestra.run.smithi184.stdout: rados osd op timeout = 900 2023-10-08T00:54:45.670 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-08T00:54:45.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-08T00:54:45.769 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-08T00:54:45.769 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:45.769 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-08T00:54:45.805 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:45.805 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-08T00:54:45.882 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T00:54:45.883 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:45.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T00:54:45.883 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T00:54:46.011 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:54:46.011 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:54:46.089 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T00:54:46.090 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:54:46.090 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-08T00:54:46.090 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-08T00:54:46.180 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:54:46.181 DEBUG:teuthology.orchestra.run.smithi184:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:54:46.255 INFO:tasks.ceph:Setting up mon nodes... 2023-10-08T00:54:46.256 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-08T00:54:46.256 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T00:54:46.364 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-08T00:54:46.369 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T00:54:46.492 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-08T00:54:46.499 INFO:tasks.ceph:Setting up mds nodes... 2023-10-08T00:54:46.499 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T00:54:46.607 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-08T00:54:46.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-08T00:54:46.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T00:54:46.781 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-08T00:54:46.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-08T00:54:46.846 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T00:54:46.956 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-08T00:54:46.961 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-08T00:54:47.022 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T00:54:47.129 INFO:teuthology.orchestra.run.smithi184.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-08T00:54:47.134 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-08T00:54:47.195 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-08T00:54:47.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-08T00:54:47.278 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-08T00:54:47.308 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-08T00:54:47.389 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-08T00:54:47.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-08T00:54:47.505 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-08T00:54:47.536 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-08T00:54:47.619 INFO:teuthology.orchestra.run.smithi184.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-08T00:54:47.649 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-08T00:54:47.649 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi184.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-08T00:54:47.650 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-08T00:54:47.710 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:54:47.711 INFO:tasks.ceph:role: osd.0 2023-10-08T00:54:47.711 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T00:54:47.711 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T00:54:47.779 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:54:47.779 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:54:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:54:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 2023-10-08T00:54:47.780 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:54:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:54:47.780 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:54:47.780 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:54:47.780 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:54:47.780 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:54:48.209 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-10-08T00:54:48.213 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-10-08T00:54:48.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-08T00:54:48.403 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-08T00:54:48.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-08T00:54:48.534 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:54:48.534 INFO:tasks.ceph:role: osd.1 2023-10-08T00:54:48.534 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T00:54:48.535 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T00:54:48.601 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:54:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:54:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:54:48.601 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 2023-10-08T00:54:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:54:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:54:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:54:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:54:48.602 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:54:48.602 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:54:49.008 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-10-08T00:54:49.012 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-10-08T00:54:49.013 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-08T00:54:49.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-08T00:54:49.214 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-08T00:54:49.274 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:54:49.274 INFO:tasks.ceph:role: osd.2 2023-10-08T00:54:49.275 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T00:54:49.275 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T00:54:49.342 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:54:49.343 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:54:49.791 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-10-08T00:54:49.795 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-10-08T00:54:49.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-08T00:54:49.959 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-08T00:54:50.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-08T00:54:50.088 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-08T00:54:50.088 INFO:tasks.ceph:role: osd.3 2023-10-08T00:54:50.089 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T00:54:50.089 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T00:54:50.156 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:54:50.156 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:54:50.157 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:54:50.157 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 2023-10-08T00:54:50.157 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:54:50.157 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:54:50.157 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:54:50.157 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:54:50.157 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:54:50.157 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:54:50.588 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-10-08T00:54:50.592 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-10-08T00:54:50.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-08T00:54:50.776 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-08T00:54:50.843 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:54:50.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:50.959+0000 7f0c2a857540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-08T00:54:50.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:50.959+0000 7f0c2a857540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-08T00:54:50.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:50.959+0000 7f0c2a857540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T00:54:50.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:50.959+0000 7f0c2a857540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T00:54:50.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:50.959+0000 7f0c2a857540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-08T00:54:50.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:50.961+0000 7f0c2a857540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-08T00:54:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:51.498+0000 7f0c2a857540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:52.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:52.258+0000 7f0c2a857540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:53.026+0000 7f0c2a857540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:53.841 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-08T00:54:53.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:54:53.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:53.998+0000 7f5f727c5540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-08T00:54:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:53.998+0000 7f5f727c5540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-08T00:54:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:53.998+0000 7f5f727c5540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T00:54:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:53.998+0000 7f5f727c5540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T00:54:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:53.998+0000 7f5f727c5540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-08T00:54:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:54.000+0000 7f5f727c5540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-08T00:54:54.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:54.535+0000 7f5f727c5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:55.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:55.297+0000 7f5f727c5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:56.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:56.064+0000 7f5f727c5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:56.881 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-08T00:54:56.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:54:57.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:57.033+0000 7f907bfed540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-08T00:54:57.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:57.034+0000 7f907bfed540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-08T00:54:57.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:57.034+0000 7f907bfed540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T00:54:57.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:57.034+0000 7f907bfed540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T00:54:57.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:57.034+0000 7f907bfed540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-08T00:54:57.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:57.035+0000 7f907bfed540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-08T00:54:57.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:57.573+0000 7f907bfed540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:58.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:58.337+0000 7f907bfed540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:59.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:54:59.117+0000 7f907bfed540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:54:59.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-08T00:55:00.001 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:55:00.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:00.091+0000 7f8040da5540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-08T00:55:00.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:00.091+0000 7f8040da5540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-08T00:55:00.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:00.091+0000 7f8040da5540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T00:55:00.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:00.091+0000 7f8040da5540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T00:55:00.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:00.091+0000 7f8040da5540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-08T00:55:00.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:00.093+0000 7f8040da5540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-08T00:55:00.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:00.635+0000 7f8040da5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:01.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:01.400+0000 7f8040da5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:02.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:02.163+0000 7f8040da5540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:02.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-08T00:55:03.042 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-08T00:55:03.114 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:55:03.114 INFO:tasks.ceph:role: osd.4 2023-10-08T00:55:03.115 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T00:55:03.115 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-08T00:55:03.190 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:55:03.191 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:55:03.621 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-10-08T00:55:03.627 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-10-08T00:55:03.627 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-08T00:55:03.805 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-08T00:55:03.876 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-08T00:55:03.937 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:55:03.937 INFO:tasks.ceph:role: osd.5 2023-10-08T00:55:03.937 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T00:55:03.937 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-08T00:55:04.005 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:55:04.005 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:55:04.005 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:55:04.005 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 2023-10-08T00:55:04.006 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:55:04.006 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:55:04.006 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:55:04.006 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:55:04.006 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:55:04.006 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:55:04.412 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-10-08T00:55:04.416 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-10-08T00:55:04.416 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-08T00:55:04.546 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-08T00:55:04.614 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-08T00:55:04.674 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:55:04.674 INFO:tasks.ceph:role: osd.6 2023-10-08T00:55:04.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T00:55:04.674 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-08T00:55:04.742 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:55:04.743 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:55:04.743 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:55:04.743 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 2023-10-08T00:55:04.743 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:55:04.744 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:55:04.744 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:55:04.744 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:55:04.744 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:55:04.744 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:55:05.186 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-10-08T00:55:05.191 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-10-08T00:55:05.191 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-08T00:55:05.373 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-08T00:55:05.442 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-08T00:55:05.502 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-08T00:55:05.502 INFO:tasks.ceph:role: osd.7 2023-10-08T00:55:05.502 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T00:55:05.502 DEBUG:teuthology.orchestra.run.smithi184:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-08T00:55:05.570 INFO:teuthology.orchestra.run.smithi184.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout: = reflink=1 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout: = sunit=0 swidth=0 blks 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-08T00:55:05.571 INFO:teuthology.orchestra.run.smithi184.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-08T00:55:05.996 INFO:teuthology.orchestra.run.smithi184.stdout:Discarding blocks...Done. 2023-10-08T00:55:06.000 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi184.front.sepia.ceph.com -o noatime 2023-10-08T00:55:06.000 DEBUG:teuthology.orchestra.run.smithi184:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-08T00:55:06.172 DEBUG:teuthology.orchestra.run.smithi184:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-08T00:55:06.239 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:55:06.343 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:06.342+0000 7fcfca81c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-08T00:55:06.344 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:06.342+0000 7fcfca81c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-08T00:55:06.344 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:06.342+0000 7fcfca81c540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T00:55:06.344 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:06.342+0000 7fcfca81c540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T00:55:06.344 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:06.342+0000 7fcfca81c540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-08T00:55:06.345 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:06.344+0000 7fcfca81c540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-08T00:55:06.882 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:06.880+0000 7fcfca81c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:07.646 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:07.645+0000 7fcfca81c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:08.411 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:08.410+0000 7fcfca81c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:09.227 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-08T00:55:09.290 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:55:09.382 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:09.379+0000 7fa09d415540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-08T00:55:09.382 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:09.380+0000 7fa09d415540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-08T00:55:09.382 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:09.380+0000 7fa09d415540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T00:55:09.383 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:09.380+0000 7fa09d415540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T00:55:09.383 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:09.380+0000 7fa09d415540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-08T00:55:09.383 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:09.381+0000 7fa09d415540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-08T00:55:09.919 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:09.918+0000 7fa09d415540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:10.681 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:10.679+0000 7fa09d415540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:11.444 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:11.443+0000 7fa09d415540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:12.259 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-08T00:55:12.324 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:55:12.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:12.414+0000 7f3e2e1f2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-08T00:55:12.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:12.414+0000 7f3e2e1f2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-08T00:55:12.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:12.414+0000 7f3e2e1f2540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T00:55:12.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:12.414+0000 7f3e2e1f2540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T00:55:12.416 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:12.414+0000 7f3e2e1f2540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-08T00:55:12.417 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:12.416+0000 7f3e2e1f2540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-08T00:55:12.958 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:12.957+0000 7f3e2e1f2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:13.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:55:13.053 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:55:13.722 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:13.721+0000 7f3e2e1f2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:14.489 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:14.488+0000 7f3e2e1f2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:15.305 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-08T00:55:15.373 DEBUG:teuthology.orchestra.run.smithi184:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:55:15.471 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:15.468+0000 7f69fb9ae540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-08T00:55:15.472 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:15.468+0000 7f69fb9ae540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-08T00:55:15.472 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:15.468+0000 7f69fb9ae540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T00:55:15.472 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:15.468+0000 7f69fb9ae540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T00:55:15.472 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:15.468+0000 7f69fb9ae540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-08T00:55:15.472 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:15.470+0000 7f69fb9ae540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-08T00:55:16.013 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:16.011+0000 7f69fb9ae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:16.777 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:16.776+0000 7f69fb9ae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:17.544 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:17.543+0000 7f69fb9ae540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-08T00:55:18.356 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-08T00:55:18.420 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-08T00:55:18.421 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:18.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-08T00:55:18.487 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:18.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-08T00:55:18.591 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:18.591 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-08T00:55:18.693 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:18.693 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-08T00:55:18.794 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:18.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-08T00:55:18.894 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:18.894 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-08T00:55:18.993 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:18.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-08T00:55:19.092 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.092 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-08T00:55:19.161 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.161 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-08T00:55:19.266 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.267 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-08T00:55:19.368 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.368 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-08T00:55:19.470 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.470 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-08T00:55:19.570 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.570 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-08T00:55:19.670 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.670 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-08T00:55:19.770 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:19.770 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-08T00:55:19.807 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:19.807 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-08T00:55:19.882 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.883 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-08T00:55:19.918 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:19.918 DEBUG:teuthology.orchestra.run.smithi184:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-08T00:55:19.994 INFO:tasks.ceph:Adding keys to all mons... 2023-10-08T00:55:19.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T00:55:19.999 DEBUG:teuthology.orchestra.run.smithi184:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-08T00:55:20.053 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2023-10-08T00:55:20.053 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg== 2023-10-08T00:55:20.053 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-10-08T00:55:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA== 2023-10-08T00:55:20.054 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-10-08T00:55:20.054 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw== 2023-10-08T00:55:20.054 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2023-10-08T00:55:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg== 2023-10-08T00:55:20.055 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2023-10-08T00:55:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw== 2023-10-08T00:55:20.055 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2023-10-08T00:55:20.055 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw== 2023-10-08T00:55:20.055 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA== 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw== 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A== 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ== 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw== 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw== 2023-10-08T00:55:20.056 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA== 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA== 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ== 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout:[client.2] 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow== 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw== 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout:[client.3] 2023-10-08T00:55:20.057 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw== 2023-10-08T00:55:20.092 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.y] 2023-10-08T00:55:20.092 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg== 2023-10-08T00:55:20.092 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.a] 2023-10-08T00:55:20.092 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA== 2023-10-08T00:55:20.092 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.c] 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw== 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.0] 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg== 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.1] 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw== 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.2] 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw== 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.3] 2023-10-08T00:55:20.093 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA== 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout:[mgr.x] 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw== 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.b] 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A== 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout:[mds.d] 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ== 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.4] 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw== 2023-10-08T00:55:20.094 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.5] 2023-10-08T00:55:20.095 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw== 2023-10-08T00:55:20.095 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.6] 2023-10-08T00:55:20.095 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA== 2023-10-08T00:55:20.095 INFO:teuthology.orchestra.run.smithi184.stdout:[osd.7] 2023-10-08T00:55:20.095 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA== 2023-10-08T00:55:20.095 INFO:teuthology.orchestra.run.smithi184.stdout:[client.0] 2023-10-08T00:55:20.095 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ== 2023-10-08T00:55:20.095 INFO:teuthology.orchestra.run.smithi184.stdout:[client.2] 2023-10-08T00:55:20.096 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow== 2023-10-08T00:55:20.096 INFO:teuthology.orchestra.run.smithi184.stdout:[client.1] 2023-10-08T00:55:20.096 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw== 2023-10-08T00:55:20.096 INFO:teuthology.orchestra.run.smithi184.stdout:[client.3] 2023-10-08T00:55:20.096 INFO:teuthology.orchestra.run.smithi184.stdout: key = AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw== 2023-10-08T00:55:20.097 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T00:55:20.104 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T00:55:20.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:55:20.236 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:55:20.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:55:20.336 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:55:20.434 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:20.439 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:20.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:20.537 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:20.631 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:20.636 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:20.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:20.736 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:20.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T00:55:20.835 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-08T00:55:20.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:55:20.951 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:55:21.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:55:21.053 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-08T00:55:21.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:21.156 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:21.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:21.256 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:21.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:21.365 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:21.468 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:21.473 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-08T00:55:21.568 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:55:21.572 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:55:21.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:55:21.674 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:55:21.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:55:21.780 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:55:21.879 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:55:21.886 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-08T00:55:21.985 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-08T00:55:21.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-08T00:55:22.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T00:55:22.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-08T00:55:22.218 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-08T00:55:22.279 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T00:55:22.388 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-08T00:55:22.448 DEBUG:teuthology.orchestra.run.smithi184:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-08T00:55:22.508 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-08T00:55:22.615 DEBUG:teuthology.orchestra.run.smithi184:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-08T00:55:22.676 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:55:22.680 DEBUG:teuthology.orchestra.run.smithi184:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-08T00:55:22.716 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-08T00:55:22.716 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-08T00:55:22.716 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-08T00:55:22.759 INFO:tasks.ceph.mon.a:Started 2023-10-08T00:55:22.759 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-08T00:55:22.759 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-08T00:55:22.764 INFO:tasks.ceph.mon.b:Started 2023-10-08T00:55:22.764 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-08T00:55:22.764 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-08T00:55:22.768 INFO:tasks.ceph.mon.c:Started 2023-10-08T00:55:22.768 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-08T00:55:22.768 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-08T00:55:22.769 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-08T00:55:22.773 INFO:tasks.ceph.mgr.y:Started 2023-10-08T00:55:22.773 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-08T00:55:22.774 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-08T00:55:22.779 INFO:tasks.ceph.mgr.x:Started 2023-10-08T00:55:22.780 INFO:tasks.ceph:Setting crush tunables to default 2023-10-08T00:55:22.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd crush tunables default 2023-10-08T00:55:22.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.959+0000 7f054e19d700 1 Processor -- start 2023-10-08T00:55:22.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.959+0000 7f054e19d700 1 -- start start 2023-10-08T00:55:22.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.960+0000 7f054e19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f05480b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:22.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.960+0000 7f054e19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 0x7f05480b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:22.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.960+0000 7f054e19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05480b2080 con 0x7f05480b4e70 2023-10-08T00:55:22.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.960+0000 7f054e19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05480cfb00 con 0x7f05480b33b0 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.960+0000 7f054e19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480b5c40 con 0x7f05480b16d0 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.960+0000 7f054d99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 0x7f05480b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.960+0000 7f054d99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 0x7f05480b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42622/0 (socket says 172.21.15.119:42622) 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.960+0000 7f054d99c700 1 -- 172.21.15.119:0/211555248 learned_addr learned my addr 172.21.15.119:0/211555248 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054c99a700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f05480b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054d99c700 1 -- 172.21.15.119:0/211555248 >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 msgr2=0x7f05480b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054d99c700 1 -- 172.21.15.119:0/211555248 >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 msgr2=0x7f05480b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054d99c700 1 --2- 172.21.15.119:0/211555248 >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 0x7f05480b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054d99c700 1 --2- 172.21.15.119:0/211555248 >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 0x7f05480b3830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054c99a700 1 -- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 msgr2=0x7f05480b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-08T00:55:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054c99a700 1 -- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 msgr2=0x7f05480b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-08T00:55:22.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054c99a700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f05480b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-08T00:55:22.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:22.961+0000 7f054c99a700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f05480b1b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T00:55:23.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.154+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068342467 0 0) 0x7f05480b2080 con 0x7f05480b4e70 2023-10-08T00:55:23.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.154+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0534003570 con 0x7f05480b4e70 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3087986341 0 0) 0x7f0534003570 con 0x7f05480b4e70 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 msgr2=0x7f05480b1b40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f05480b1b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 msgr2=0x7f05480b3830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 --2- 172.21.15.119:0/211555248 >> v2:172.21.15.119:3300/0 conn(0x7f05480b33b0 0x7f05480b3830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480b5f20 con 0x7f05480b4e70 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f0544001470 con 0x7f05480b4e70 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 >> v1:172.21.15.184:6789/0 conn(0x7f05480b4e70 legacy=0x7f05480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f0534003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:23.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0534004160 0x7f0534004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480cfb00 con 0x7f0534004d50 2023-10-08T00:55:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480b5c40 con 0x7f0534004160 2023-10-08T00:55:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.158+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053400bdc0 con 0x7f05480b16d0 2023-10-08T00:55:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054d19b700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f0534003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054c99a700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0534004160 0x7f0534004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:23.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054d99c700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054d99c700 1 -- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 msgr2=0x7f0534003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054d99c700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f0534003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054d99c700 1 -- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0534004160 msgr2=0x7f0534004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054d99c700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0534004160 0x7f0534004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054d99c700 1 -- 172.21.15.119:0/211555248 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0534003570 con 0x7f0534004d50 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.159+0000 7f054d19b700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480b16d0 0x7f0534003ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.160+0000 7f054d99c700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05340051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0540009410 tx=0x7f05400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1387584fe9223d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.160+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05400097a0 con 0x7f0534004d50 2023-10-08T00:55:23.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.160+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054000f040 con 0x7f0534004d50 2023-10-08T00:55:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.160+0000 7f053ffff700 1 -- 172.21.15.119:0/211555248 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0540007d70 con 0x7f0534004d50 2023-10-08T00:55:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.161+0000 7f054e19d700 1 -- 172.21.15.119:0/211555248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 msgr2=0x7f05340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.161+0000 7f054e19d700 1 --2- 172.21.15.119:0/211555248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05340051c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0540009410 tx=0x7f05400091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.161+0000 7f054d99c700 1 -- 172.21.15.119:0/211555248 reap_dead start 2023-10-08T00:55:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.161+0000 7f054e19d700 1 -- 172.21.15.119:0/211555248 shutdown_connections 2023-10-08T00:55:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.161+0000 7f054e19d700 1 -- 172.21.15.119:0/211555248 >> 172.21.15.119:0/211555248 conn(0x7f05480d10c0 msgr2=0x7f05480affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.161+0000 7f054e19d700 1 -- 172.21.15.119:0/211555248 shutdown_connections 2023-10-08T00:55:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.161+0000 7f054e19d700 1 -- 172.21.15.119:0/211555248 wait complete. 2023-10-08T00:55:23.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.163+0000 7f054e19d700 1 Processor -- start 2023-10-08T00:55:23.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.163+0000 7f054e19d700 1 -- start start 2023-10-08T00:55:23.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054e19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05480cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:23.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054e19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480c6360 0x7f054816c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:23.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054e19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0548068390 0x7f05480687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:23.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054e19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05480c8050 con 0x7f0534004d50 2023-10-08T00:55:23.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054e19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0548068d20 con 0x7f0548068390 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054e19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05480c6ad0 con 0x7f05480c6360 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05480cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05480cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42668/0 (socket says 172.21.15.119:42668) 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d19b700 1 -- 172.21.15.119:0/2508374153 learned_addr learned my addr 172.21.15.119:0/2508374153 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d99c700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0548068390 0x7f05480687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054c99a700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480c6360 0x7f054816c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d19b700 1 -- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480c6360 msgr2=0x7f054816c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d19b700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480c6360 0x7f054816c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d19b700 1 -- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0548068390 msgr2=0x7f05480687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:23.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d19b700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0548068390 0x7f05480687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d19b700 1 -- 172.21.15.119:0/2508374153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05480c6db0 con 0x7f0534004d50 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054c99a700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480c6360 0x7f054816c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.164+0000 7f054d99c700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0548068390 0x7f05480687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.165+0000 7f054d19b700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05480cb0a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0544000df0 tx=0x7f0544002e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.165+0000 7f053d7fa700 1 -- 172.21.15.119:0/2508374153 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544007e70 con 0x7f0534004d50 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.165+0000 7f053d7fa700 1 -- 172.21.15.119:0/2508374153 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054400db30 con 0x7f0534004d50 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.165+0000 7f053d7fa700 1 -- 172.21.15.119:0/2508374153 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544014470 con 0x7f0534004d50 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.165+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05480c7090 con 0x7f0534004d50 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.165+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05480c67d0 con 0x7f0534004d50 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.165+0000 7f053d7fa700 1 -- 172.21.15.119:0/2508374153 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0544014aa0 con 0x7f0534004d50 2023-10-08T00:55:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.165+0000 7f053d7fa700 1 -- 172.21.15.119:0/2508374153 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f054401db00 con 0x7f0534004d50 2023-10-08T00:55:23.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.166+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0548088990 con 0x7f0534004d50 2023-10-08T00:55:23.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.171+0000 7f053d7fa700 1 -- 172.21.15.119:0/2508374153 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0544018020 con 0x7f0534004d50 2023-10-08T00:55:23.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.248+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f054804bb80 con 0x7f0534004d50 2023-10-08T00:55:23.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.255+0000 7f053d7fa700 1 -- 172.21.15.119:0/2508374153 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0544026430 con 0x7f0534004d50 2023-10-08T00:55:23.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 msgr2=0x7f05480cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:23.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05480cb0a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0544000df0 tx=0x7f0544002e20 comp rx=0 tx=0).stop 2023-10-08T00:55:23.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 shutdown_connections 2023-10-08T00:55:23.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0548068390 0x7f05480687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0534004d50 0x7f05480cb0a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 --2- 172.21.15.119:0/2508374153 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05480c6360 0x7f054816c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:23.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 >> 172.21.15.119:0/2508374153 conn(0x7f05480d10c0 msgr2=0x7f0548157be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:23.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 shutdown_connections 2023-10-08T00:55:23.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:23.256+0000 7f054e19d700 1 -- 172.21.15.119:0/2508374153 wait complete. 2023-10-08T00:55:23.258 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2023-10-08T00:55:23.274 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-08T00:55:23.274 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:23.274 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-08T00:55:23.334 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:23.334 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-08T00:55:23.436 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:23.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-08T00:55:23.538 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-10-08T00:55:23.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-08T00:55:23.639 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:23.639 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-08T00:55:23.700 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:23.700 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-08T00:55:23.801 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:23.801 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-08T00:55:23.905 DEBUG:teuthology.orchestra.run.smithi184:> set -ex 2023-10-08T00:55:23.905 DEBUG:teuthology.orchestra.run.smithi184:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-08T00:55:24.007 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new a5097618-d43f-4d26-b009-6a2b9340a46c 0 2023-10-08T00:55:24.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.224+0000 7f75f2d9d700 1 Processor -- start 2023-10-08T00:55:24.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.224+0000 7f75f2d9d700 1 -- start start 2023-10-08T00:55:24.227 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.225+0000 7f75f2d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f75ec0b4e70 0x7f75ec0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.227 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.225+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b16d0 0x7f75ec0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.227 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.225+0000 7f75f2d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec0b2080 con 0x7f75ec15d240 2023-10-08T00:55:24.227 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.225+0000 7f75f2d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ec0cfb00 con 0x7f75ec0b4e70 2023-10-08T00:55:24.227 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.225+0000 7f75f2d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0b5c40 con 0x7f75ec0b16d0 2023-10-08T00:55:24.227 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.225+0000 7f75f159a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b16d0 0x7f75ec0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f159a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b16d0 0x7f75ec0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:58838/0 (socket says 172.21.15.184:58838) 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f159a700 1 -- 172.21.15.184:0/2118750548 learned_addr learned my addr 172.21.15.184:0/2118750548 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f1d9b700 1 --2- 172.21.15.184:0/2118750548 >> v2:172.21.15.119:3300/0 conn(0x7f75ec0b4e70 0x7f75ec0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f159a700 1 -- 172.21.15.184:0/2118750548 >> v2:172.21.15.119:3300/0 conn(0x7f75ec0b4e70 msgr2=0x7f75ec0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f159a700 1 --2- 172.21.15.184:0/2118750548 >> v2:172.21.15.119:3300/0 conn(0x7f75ec0b4e70 0x7f75ec0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f159a700 1 -- 172.21.15.184:0/2118750548 >> v1:172.21.15.119:6789/0 conn(0x7f75ec15d240 legacy=0x7f75ec15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f159a700 1 -- 172.21.15.184:0/2118750548 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec0b5f20 con 0x7f75ec0b16d0 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f159a700 1 --2- 172.21.15.184:0/2118750548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b16d0 0x7f75ec0b1b40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0099b0 tx=0x7f75dc009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=921e4232b8a19d2b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:24.228 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f0d99700 1 -- 172.21.15.184:0/2118750548 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc009ea0 con 0x7f75ec0b16d0 2023-10-08T00:55:24.229 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f0d99700 1 -- 172.21.15.184:0/2118750548 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc00b860 con 0x7f75ec0b16d0 2023-10-08T00:55:24.229 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.226+0000 7f75f0d99700 1 -- 172.21.15.184:0/2118750548 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00bc00 con 0x7f75ec0b16d0 2023-10-08T00:55:24.232 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.227+0000 7f75f2d9d700 1 -- 172.21.15.184:0/2118750548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b16d0 msgr2=0x7f75ec0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.227+0000 7f75f2d9d700 1 --2- 172.21.15.184:0/2118750548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b16d0 0x7f75ec0b1b40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f75dc0099b0 tx=0x7f75dc009250 comp rx=0 tx=0).stop 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.227+0000 7f75f2d9d700 1 -- 172.21.15.184:0/2118750548 shutdown_connections 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.227+0000 7f75f2d9d700 1 --2- 172.21.15.184:0/2118750548 >> v2:172.21.15.119:3300/0 conn(0x7f75ec0b4e70 0x7f75ec0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.227+0000 7f75f2d9d700 1 --2- 172.21.15.184:0/2118750548 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b16d0 0x7f75ec0b1b40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.227+0000 7f75f2d9d700 1 -- 172.21.15.184:0/2118750548 >> 172.21.15.184:0/2118750548 conn(0x7f75ec0d10c0 msgr2=0x7f75ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.227+0000 7f75f2d9d700 1 -- 172.21.15.184:0/2118750548 shutdown_connections 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.227+0000 7f75f2d9d700 1 -- 172.21.15.184:0/2118750548 wait complete. 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.229+0000 7f75f2d9d700 1 Processor -- start 2023-10-08T00:55:24.233 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.229+0000 7f75f2d9d700 1 -- start start 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec0b16d0 0x7f75ec0cb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b4e70 0x7f75ec0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec15d240 0x7f75ec15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f2d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0c6c20 con 0x7f75ec0b16d0 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f2d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0cbf10 con 0x7f75ec15d240 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f2d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0cc1f0 con 0x7f75ec0b4e70 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f159a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b4e70 0x7f75ec0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec15d240 0x7f75ec15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec0b16d0 0x7f75ec0cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f159a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b4e70 0x7f75ec0c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:58850/0 (socket says 172.21.15.184:58850) 2023-10-08T00:55:24.234 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f159a700 1 -- 172.21.15.184:0/4067266823 learned_addr learned my addr 172.21.15.184:0/4067266823 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f1d9b700 1 -- 172.21.15.184:0/4067266823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b4e70 msgr2=0x7f75ec0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f1d9b700 1 --2- 172.21.15.184:0/4067266823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b4e70 0x7f75ec0c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f1d9b700 1 -- 172.21.15.184:0/4067266823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec15d240 msgr2=0x7f75ec15ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f1d9b700 1 --2- 172.21.15.184:0/4067266823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec15d240 0x7f75ec15ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f1d9b700 1 -- 172.21.15.184:0/4067266823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0094f0 con 0x7f75ec0b16d0 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.230+0000 7f75f159a700 1 --2- 172.21.15.184:0/4067266823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b4e70 0x7f75ec0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.231+0000 7f75f1d9b700 1 --2- 172.21.15.184:0/4067266823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec0b16d0 0x7f75ec0cb850 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75e800ea50 tx=0x7f75e80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.231+0000 7f75e1ffb700 1 -- 172.21.15.184:0/4067266823 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800ee90 con 0x7f75ec0b16d0 2023-10-08T00:55:24.235 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.231+0000 7f75e1ffb700 1 -- 172.21.15.184:0/4067266823 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e800c7f0 con 0x7f75ec0b16d0 2023-10-08T00:55:24.236 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.231+0000 7f75e1ffb700 1 -- 172.21.15.184:0/4067266823 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800cb90 con 0x7f75ec0b16d0 2023-10-08T00:55:24.236 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.231+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec0c65d0 con 0x7f75ec0b16d0 2023-10-08T00:55:24.236 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.231+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec0c98a0 con 0x7f75ec0b16d0 2023-10-08T00:55:24.236 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.231+0000 7f75e1ffb700 1 -- 172.21.15.184:0/4067266823 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f75e8005c80 con 0x7f75ec0b16d0 2023-10-08T00:55:24.236 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.231+0000 7f75e1ffb700 1 -- 172.21.15.184:0/4067266823 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f75e8013470 con 0x7f75ec0b16d0 2023-10-08T00:55:24.236 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.232+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec088990 con 0x7f75ec0b16d0 2023-10-08T00:55:24.236 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.235+0000 7f75e1ffb700 1 -- 172.21.15.184:0/4067266823 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f75e8021060 con 0x7f75ec0b16d0 2023-10-08T00:55:24.320 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.319+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a5097618-d43f-4d26-b009-6a2b9340a46c", "id": 0} v 0) v1 -- 0x7f75ec04bb80 con 0x7f75ec0b16d0 2023-10-08T00:55:24.327 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.327+0000 7f75e1ffb700 1 -- 172.21.15.184:0/4067266823 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a5097618-d43f-4d26-b009-6a2b9340a46c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f75e8017070 con 0x7f75ec0b16d0 2023-10-08T00:55:24.328 INFO:teuthology.orchestra.run.smithi184.stdout:0 2023-10-08T00:55:24.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec0b16d0 msgr2=0x7f75ec0cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 --2- 172.21.15.184:0/4067266823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec0b16d0 0x7f75ec0cb850 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f75e800ea50 tx=0x7f75e80091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 shutdown_connections 2023-10-08T00:55:24.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 --2- 172.21.15.184:0/4067266823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f75ec15d240 0x7f75ec15ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 --2- 172.21.15.184:0/4067266823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f75ec0b16d0 0x7f75ec0cb850 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.329 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 --2- 172.21.15.184:0/4067266823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f75ec0b4e70 0x7f75ec0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.330 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 >> 172.21.15.184:0/4067266823 conn(0x7f75ec0d10c0 msgr2=0x7f75ec158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:24.330 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 shutdown_connections 2023-10-08T00:55:24.330 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.328+0000 7f75f2d9d700 1 -- 172.21.15.184:0/4067266823 wait complete. 2023-10-08T00:55:24.345 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 9d357ab1-5e58-4e2f-8c87-f5049b795080 1 2023-10-08T00:55:24.524 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.523+0000 7f361a19d700 1 Processor -- start 2023-10-08T00:55:24.524 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.523+0000 7f361a19d700 1 -- start start 2023-10-08T00:55:24.525 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.524+0000 7f361a19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36140b1b40 0x7f36140b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.524+0000 7f361a19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36140cfb00 con 0x7f36140b16d0 2023-10-08T00:55:24.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.524+0000 7f361a19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36140b5c40 con 0x7f36140b4e70 2023-10-08T00:55:24.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.524+0000 7f361a19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36140b5f20 con 0x7f36140b1b40 2023-10-08T00:55:24.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.525+0000 7f361999c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36140b1b40 0x7f36140b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.525+0000 7f361919b700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f36140b4e70 0x7f36140ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:35840/0 (socket says 172.21.15.184:35840) 2023-10-08T00:55:24.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.525+0000 7f361919b700 1 -- 172.21.15.184:0/915495243 learned_addr learned my addr 172.21.15.184:0/915495243 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-10-08T00:55:24.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.525+0000 7f361999c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36140b1b40 0x7f36140b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:52808/0 (socket says 172.21.15.184:52808) 2023-10-08T00:55:24.526 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.525+0000 7f361999c700 1 -- 172.21.15.184:0/915495243 >> v1:172.21.15.184:6789/0 conn(0x7f36140b4e70 legacy=0x7f36140ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.527 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.525+0000 7f361999c700 1 -- 172.21.15.184:0/915495243 >> v1:172.21.15.119:6789/0 conn(0x7f36140b16d0 legacy=0x7f36140b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.527 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.525+0000 7f361999c700 1 -- 172.21.15.184:0/915495243 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36140b6200 con 0x7f36140b1b40 2023-10-08T00:55:24.527 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.525+0000 7f361999c700 1 --2- 172.21.15.184:0/915495243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b1b40 0x7f36140b3ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f360c009e60 tx=0x7f360c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0ef1b0ae02e054 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:24.528 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f360bfff700 1 -- 172.21.15.184:0/915495243 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c00e070 con 0x7f36140b1b40 2023-10-08T00:55:24.528 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f360bfff700 1 -- 172.21.15.184:0/915495243 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c00bc30 con 0x7f36140b1b40 2023-10-08T00:55:24.528 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f360bfff700 1 -- 172.21.15.184:0/915495243 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c012430 con 0x7f36140b1b40 2023-10-08T00:55:24.528 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f361a19d700 1 -- 172.21.15.184:0/915495243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b1b40 msgr2=0x7f36140b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.528 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f361a19d700 1 --2- 172.21.15.184:0/915495243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b1b40 0x7f36140b3ac0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f360c009e60 tx=0x7f360c009780 comp rx=0 tx=0).stop 2023-10-08T00:55:24.528 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f361a19d700 1 -- 172.21.15.184:0/915495243 shutdown_connections 2023-10-08T00:55:24.529 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f361a19d700 1 --2- 172.21.15.184:0/915495243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b1b40 0x7f36140b3ac0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.529 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f361a19d700 1 -- 172.21.15.184:0/915495243 >> 172.21.15.184:0/915495243 conn(0x7f36140d10c0 msgr2=0x7f36140affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:24.529 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f361a19d700 1 -- 172.21.15.184:0/915495243 shutdown_connections 2023-10-08T00:55:24.529 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.526+0000 7f361a19d700 1 -- 172.21.15.184:0/915495243 wait complete. 2023-10-08T00:55:24.529 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.528+0000 7f361a19d700 1 Processor -- start 2023-10-08T00:55:24.529 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.528+0000 7f361a19d700 1 -- start start 2023-10-08T00:55:24.532 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361a19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36140b16d0 0x7f36140a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.532 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361a19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36140b1b40 0x7f36140a7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.532 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361a19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b4e70 0x7f36140c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.532 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361a19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140ca800 con 0x7f36140b16d0 2023-10-08T00:55:24.532 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361a19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36140c9910 con 0x7f36140b4e70 2023-10-08T00:55:24.532 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361a19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36140c9bf0 con 0x7f36140b1b40 2023-10-08T00:55:24.532 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361999c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b4e70 0x7f36140c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.532 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361999c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b4e70 0x7f36140c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:52812/0 (socket says 172.21.15.184:52812) 2023-10-08T00:55:24.533 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361999c700 1 -- 172.21.15.184:0/1023406370 learned_addr learned my addr 172.21.15.184:0/1023406370 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:24.533 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361919b700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36140b16d0 0x7f36140a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.533 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361899a700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36140b1b40 0x7f36140a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.533 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361999c700 1 -- 172.21.15.184:0/1023406370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36140b1b40 msgr2=0x7f36140a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.534 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361999c700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36140b1b40 0x7f36140a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.534 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361999c700 1 -- 172.21.15.184:0/1023406370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36140b16d0 msgr2=0x7f36140a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.534 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361999c700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36140b16d0 0x7f36140a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.534 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361999c700 1 -- 172.21.15.184:0/1023406370 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c009a20 con 0x7f36140b4e70 2023-10-08T00:55:24.534 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.529+0000 7f361919b700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36140b16d0 0x7f36140a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:24.534 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.530+0000 7f361999c700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b4e70 0x7f36140c74e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f360c00b6c0 tx=0x7f360c019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:24.534 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.530+0000 7f36097fa700 1 -- 172.21.15.184:0/1023406370 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c012e40 con 0x7f36140b4e70 2023-10-08T00:55:24.534 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.530+0000 7f36097fa700 1 -- 172.21.15.184:0/1023406370 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c007000 con 0x7f36140b4e70 2023-10-08T00:55:24.535 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.530+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36140c9ed0 con 0x7f36140b4e70 2023-10-08T00:55:24.535 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.530+0000 7f36097fa700 1 -- 172.21.15.184:0/1023406370 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c01b850 con 0x7f36140b4e70 2023-10-08T00:55:24.535 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.530+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36140ca490 con 0x7f36140b4e70 2023-10-08T00:55:24.535 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.530+0000 7f36097fa700 1 -- 172.21.15.184:0/1023406370 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f360c01bc80 con 0x7f36140b4e70 2023-10-08T00:55:24.535 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.530+0000 7f36097fa700 1 -- 172.21.15.184:0/1023406370 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f360c02fd70 con 0x7f36140b4e70 2023-10-08T00:55:24.535 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.531+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36140ca970 con 0x7f36140b4e70 2023-10-08T00:55:24.536 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.536+0000 7f36097fa700 1 -- 172.21.15.184:0/1023406370 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f360c02f3f0 con 0x7f36140b4e70 2023-10-08T00:55:24.539 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:24.538+0000 7f5fc3832180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T00:55:24.557 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:24.557+0000 7f92d0645180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T00:55:24.619 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.618+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9d357ab1-5e58-4e2f-8c87-f5049b795080", "id": 1} v 0) v1 -- 0x7f361404bb80 con 0x7f36140b4e70 2023-10-08T00:55:24.627 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.626+0000 7f36097fa700 1 -- 172.21.15.184:0/1023406370 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9d357ab1-5e58-4e2f-8c87-f5049b795080", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f360c0373f0 con 0x7f36140b4e70 2023-10-08T00:55:24.627 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-10-08T00:55:24.628 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.627+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b4e70 msgr2=0x7f36140c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.628 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.627+0000 7f361a19d700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b4e70 0x7f36140c74e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f360c00b6c0 tx=0x7f360c019000 comp rx=0 tx=0).stop 2023-10-08T00:55:24.628 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.627+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 shutdown_connections 2023-10-08T00:55:24.628 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.627+0000 7f361a19d700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36140b4e70 0x7f36140c74e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.629 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.627+0000 7f361a19d700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36140b16d0 0x7f36140a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.629 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.627+0000 7f361a19d700 1 --2- 172.21.15.184:0/1023406370 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36140b1b40 0x7f36140a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.629 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.627+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 >> 172.21.15.184:0/1023406370 conn(0x7f36140d10c0 msgr2=0x7f3614159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:24.629 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.627+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 shutdown_connections 2023-10-08T00:55:24.629 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.628+0000 7f361a19d700 1 -- 172.21.15.184:0/1023406370 wait complete. 2023-10-08T00:55:24.644 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new e7ae1773-1b32-4fa9-9ccc-a984771510e5 2 2023-10-08T00:55:24.705 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:24.704+0000 7f5fc3832180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T00:55:24.728 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:24.728+0000 7f92d0645180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T00:55:24.800 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:24.799+0000 7f5fc3832180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T00:55:24.824 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.820+0000 7f815a79d700 1 Processor -- start 2023-10-08T00:55:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.820+0000 7f815a79d700 1 -- start start 2023-10-08T00:55:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f815a79d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81540b1b40 0x7f81540b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f815a79d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81540cfb00 con 0x7f81540b16d0 2023-10-08T00:55:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f815a79d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81540b5c40 con 0x7f81540b4e70 2023-10-08T00:55:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f815a79d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81540b5f20 con 0x7f81540b1b40 2023-10-08T00:55:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f8159f9c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81540b1b40 0x7f81540b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f815979b700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f81540b4e70 0x7f81540ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:35852/0 (socket says 172.21.15.184:35852) 2023-10-08T00:55:24.825 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f8159f9c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81540b1b40 0x7f81540b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:52828/0 (socket says 172.21.15.184:52828) 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f815979b700 1 -- 172.21.15.184:0/3026119763 learned_addr learned my addr 172.21.15.184:0/3026119763 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f814bfff700 1 -- 172.21.15.184:0/3026119763 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957157883 0 0) 0x7f81540b5c40 con 0x7f81540b4e70 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f814bfff700 1 -- 172.21.15.184:0/3026119763 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8140003570 con 0x7f81540b4e70 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f814bfff700 1 -- 172.21.15.184:0/3026119763 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410825700 0 0) 0x7f81540cfb00 con 0x7f81540b16d0 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f8159f9c700 1 -- 172.21.15.184:0/3026119763 >> v1:172.21.15.184:6789/0 conn(0x7f81540b4e70 legacy=0x7f81540ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f8159f9c700 1 -- 172.21.15.184:0/3026119763 >> v1:172.21.15.119:6789/0 conn(0x7f81540b16d0 legacy=0x7f81540b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.822+0000 7f8159f9c700 1 -- 172.21.15.184:0/3026119763 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540b6200 con 0x7f81540b1b40 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.823+0000 7f8159f9c700 1 --2- 172.21.15.184:0/3026119763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b1b40 0x7f81540b3ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f814c009e60 tx=0x7f814c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62b168254f94b72c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:24.826 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.823+0000 7f814bfff700 1 -- 172.21.15.184:0/3026119763 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c00e070 con 0x7f81540b1b40 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.823+0000 7f814bfff700 1 -- 172.21.15.184:0/3026119763 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c00bb60 con 0x7f81540b1b40 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.823+0000 7f814bfff700 1 -- 172.21.15.184:0/3026119763 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c012430 con 0x7f81540b1b40 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.824+0000 7f815a79d700 1 -- 172.21.15.184:0/3026119763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b1b40 msgr2=0x7f81540b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.824+0000 7f815a79d700 1 --2- 172.21.15.184:0/3026119763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b1b40 0x7f81540b3ac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f814c009e60 tx=0x7f814c009780 comp rx=0 tx=0).stop 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.824+0000 7f815a79d700 1 -- 172.21.15.184:0/3026119763 shutdown_connections 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.824+0000 7f815a79d700 1 --2- 172.21.15.184:0/3026119763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b1b40 0x7f81540b3ac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.824+0000 7f815a79d700 1 -- 172.21.15.184:0/3026119763 >> 172.21.15.184:0/3026119763 conn(0x7f81540d10c0 msgr2=0x7f81540affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.824+0000 7f815a79d700 1 -- 172.21.15.184:0/3026119763 shutdown_connections 2023-10-08T00:55:24.827 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.824+0000 7f815a79d700 1 -- 172.21.15.184:0/3026119763 wait complete. 2023-10-08T00:55:24.828 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.825+0000 7f815a79d700 1 Processor -- start 2023-10-08T00:55:24.828 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.825+0000 7f815a79d700 1 -- start start 2023-10-08T00:55:24.828 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.826+0000 7f815a79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b16d0 0x7f81540c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.829 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.826+0000 7f815a79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81540b1b40 0x7f81540b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.829 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.826+0000 7f815a79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81540b4e70 0x7f8154156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:24.829 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.826+0000 7f815a79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81540b4a10 con 0x7f81540b1b40 2023-10-08T00:55:24.829 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.826+0000 7f815a79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81540c6660 con 0x7f81540b16d0 2023-10-08T00:55:24.829 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.826+0000 7f815a79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81540c5a50 con 0x7f81540b4e70 2023-10-08T00:55:24.829 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:24.826+0000 7f92d0645180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T00:55:24.831 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.826+0000 7f815979b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b16d0 0x7f81540c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.831 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f815979b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b16d0 0x7f81540c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:52832/0 (socket says 172.21.15.184:52832) 2023-10-08T00:55:24.831 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f8159f9c700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81540b4e70 0x7f8154156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.831 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f815979b700 1 -- 172.21.15.184:0/2321020320 learned_addr learned my addr 172.21.15.184:0/2321020320 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:24.831 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f8158f9a700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81540b1b40 0x7f81540b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:24.831 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f8159f9c700 1 -- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b16d0 msgr2=0x7f81540c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f8159f9c700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b16d0 0x7f81540c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f8159f9c700 1 -- 172.21.15.184:0/2321020320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81540b1b40 msgr2=0x7f81540b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f8159f9c700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81540b1b40 0x7f81540b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f8159f9c700 1 -- 172.21.15.184:0/2321020320 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c009a20 con 0x7f81540b4e70 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f815979b700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b16d0 0x7f81540c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f8159f9c700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81540b4e70 0x7f8154156310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f814c016000 tx=0x7f814c019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.827+0000 7f81497fa700 1 -- 172.21.15.184:0/2321020320 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0263f0 con 0x7f81540b4e70 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.828+0000 7f81497fa700 1 -- 172.21.15.184:0/2321020320 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c026550 con 0x7f81540b4e70 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.828+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81540c5d30 con 0x7f81540b4e70 2023-10-08T00:55:24.832 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.828+0000 7f81497fa700 1 -- 172.21.15.184:0/2321020320 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0268f0 con 0x7f81540b4e70 2023-10-08T00:55:24.833 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.828+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81540c62f0 con 0x7f81540b4e70 2023-10-08T00:55:24.833 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.828+0000 7f81497fa700 1 -- 172.21.15.184:0/2321020320 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f814c02f400 con 0x7f81540b4e70 2023-10-08T00:55:24.833 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.828+0000 7f81497fa700 1 -- 172.21.15.184:0/2321020320 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f814c02fae0 con 0x7f81540b4e70 2023-10-08T00:55:24.833 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.829+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8154064630 con 0x7f81540b4e70 2023-10-08T00:55:24.833 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.831+0000 7f81497fa700 1 -- 172.21.15.184:0/2321020320 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f814c02f6b0 con 0x7f81540b4e70 2023-10-08T00:55:24.916 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.916+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e7ae1773-1b32-4fa9-9ccc-a984771510e5", "id": 2} v 0) v1 -- 0x7f815404bb80 con 0x7f81540b4e70 2023-10-08T00:55:24.922 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.922+0000 7f81497fa700 1 -- 172.21.15.184:0/2321020320 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e7ae1773-1b32-4fa9-9ccc-a984771510e5", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f814c037430 con 0x7f81540b4e70 2023-10-08T00:55:24.922 INFO:teuthology.orchestra.run.smithi184.stdout:2 2023-10-08T00:55:24.923 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81540b4e70 msgr2=0x7f8154156310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:24.924 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81540b4e70 0x7f8154156310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f814c016000 tx=0x7f814c019000 comp rx=0 tx=0).stop 2023-10-08T00:55:24.924 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 shutdown_connections 2023-10-08T00:55:24.924 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81540b4e70 0x7f8154156310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.924 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81540b16d0 0x7f81540c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.924 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 --2- 172.21.15.184:0/2321020320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81540b1b40 0x7f81540b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:24.924 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 >> 172.21.15.184:0/2321020320 conn(0x7f81540d10c0 msgr2=0x7f8154159210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:24.924 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 shutdown_connections 2023-10-08T00:55:24.924 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:24.923+0000 7f815a79d700 1 -- 172.21.15.184:0/2321020320 wait complete. 2023-10-08T00:55:24.941 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 08734d45-7b33-4bbc-8931-0e74277a0b6d 3 2023-10-08T00:55:25.024 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:25.023+0000 7f5fc3832180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:55:25.060 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:25.060+0000 7f92d0645180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:55:25.119 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.119+0000 7f05dffff700 1 Processor -- start 2023-10-08T00:55:25.120 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.119+0000 7f05dffff700 1 -- start start 2023-10-08T00:55:25.121 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05dffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b4e70 0x7f05e00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.121 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05dffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05e00b16d0 0x7f05e00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05dffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e00b2080 con 0x7f05e015d240 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05dffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e00cfb00 con 0x7f05e00b16d0 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05dffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00b5c40 con 0x7f05e00b4e70 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05deffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b4e70 0x7f05e00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05deffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b4e70 0x7f05e00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:58886/0 (socket says 172.21.15.184:58886) 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05deffd700 1 -- 172.21.15.184:0/2609386429 learned_addr learned my addr 172.21.15.184:0/2609386429 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.120+0000 7f05de7fc700 1 --2- 172.21.15.184:0/2609386429 >> v2:172.21.15.119:3300/0 conn(0x7f05e00b16d0 0x7f05e00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05de7fc700 1 -- 172.21.15.184:0/2609386429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b4e70 msgr2=0x7f05e00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:25.122 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05de7fc700 1 --2- 172.21.15.184:0/2609386429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b4e70 0x7f05e00ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.123 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05de7fc700 1 -- 172.21.15.184:0/2609386429 >> v1:172.21.15.119:6789/0 conn(0x7f05e015d240 legacy=0x7f05e015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.123 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05de7fc700 1 -- 172.21.15.184:0/2609386429 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00b5f20 con 0x7f05e00b16d0 2023-10-08T00:55:25.123 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05deffd700 1 --2- 172.21.15.184:0/2609386429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b4e70 0x7f05e00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:25.123 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05de7fc700 1 --2- 172.21.15.184:0/2609386429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e00b16d0 0x7f05e00b1b40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05d40098e0 tx=0x7f05d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e606eaaa081df3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:25.124 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05ddffb700 1 -- 172.21.15.184:0/2609386429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400b640 con 0x7f05e00b16d0 2023-10-08T00:55:25.125 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05ddffb700 1 -- 172.21.15.184:0/2609386429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d400b7a0 con 0x7f05e00b16d0 2023-10-08T00:55:25.125 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.121+0000 7f05ddffb700 1 -- 172.21.15.184:0/2609386429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d400bb40 con 0x7f05e00b16d0 2023-10-08T00:55:25.125 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.122+0000 7f05dffff700 1 -- 172.21.15.184:0/2609386429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e00b16d0 msgr2=0x7f05e00b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.125 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.122+0000 7f05dffff700 1 --2- 172.21.15.184:0/2609386429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e00b16d0 0x7f05e00b1b40 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f05d40098e0 tx=0x7f05d40091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.125 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.122+0000 7f05dffff700 1 -- 172.21.15.184:0/2609386429 shutdown_connections 2023-10-08T00:55:25.125 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.122+0000 7f05dffff700 1 --2- 172.21.15.184:0/2609386429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b4e70 0x7f05e00ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.125 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.122+0000 7f05dffff700 1 --2- 172.21.15.184:0/2609386429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e00b16d0 0x7f05e00b1b40 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.125 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.122+0000 7f05dffff700 1 -- 172.21.15.184:0/2609386429 >> 172.21.15.184:0/2609386429 conn(0x7f05e00d10c0 msgr2=0x7f05e00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:25.126 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.122+0000 7f05dffff700 1 -- 172.21.15.184:0/2609386429 shutdown_connections 2023-10-08T00:55:25.126 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.122+0000 7f05dffff700 1 -- 172.21.15.184:0/2609386429 wait complete. 2023-10-08T00:55:25.126 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.124+0000 7f05dffff700 1 Processor -- start 2023-10-08T00:55:25.126 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.124+0000 7f05dffff700 1 -- start start 2023-10-08T00:55:25.129 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05dffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b16d0 0x7f05e00c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.129 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05dffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05e00b4e70 0x7f05e01459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.129 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05dffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e015d240 0x7f05e0158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.129 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05dffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00cb520 con 0x7f05e015d240 2023-10-08T00:55:25.129 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05dffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00c6360 con 0x7f05e00b4e70 2023-10-08T00:55:25.129 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05dffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00cac50 con 0x7f05e00b16d0 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05deffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b16d0 0x7f05e00c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05deffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b16d0 0x7f05e00c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:58894/0 (socket says 172.21.15.184:58894) 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05deffd700 1 -- 172.21.15.184:0/223840065 learned_addr learned my addr 172.21.15.184:0/223840065 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05df7fe700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e015d240 0x7f05e0158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05de7fc700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05e00b4e70 0x7f05e01459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05deffd700 1 -- 172.21.15.184:0/223840065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05e00b4e70 msgr2=0x7f05e01459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05deffd700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05e00b4e70 0x7f05e01459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.125+0000 7f05deffd700 1 -- 172.21.15.184:0/223840065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e015d240 msgr2=0x7f05e0158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05deffd700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e015d240 0x7f05e0158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.130 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05deffd700 1 -- 172.21.15.184:0/223840065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d00094f0 con 0x7f05e00b16d0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05df7fe700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e015d240 0x7f05e0158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05deffd700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b16d0 0x7f05e00c7050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05d000ec50 tx=0x7f05d0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05c6ffd700 1 -- 172.21.15.184:0/223840065 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d000c850 con 0x7f05e00b16d0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05c6ffd700 1 -- 172.21.15.184:0/223840065 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d000c9b0 con 0x7f05e00b16d0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d4009420 con 0x7f05e00b16d0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05c6ffd700 1 -- 172.21.15.184:0/223840065 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d000cd50 con 0x7f05e00b16d0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e00c6570 con 0x7f05e00b16d0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05c6ffd700 1 -- 172.21.15.184:0/223840065 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f05d0005a00 con 0x7f05e00b16d0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.126+0000 7f05c6ffd700 1 -- 172.21.15.184:0/223840065 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f05d0013da0 con 0x7f05e00b16d0 2023-10-08T00:55:25.131 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.127+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e00c6ea0 con 0x7f05e00b16d0 2023-10-08T00:55:25.132 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.130+0000 7f05c6ffd700 1 -- 172.21.15.184:0/223840065 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f05d0021060 con 0x7f05e00b16d0 2023-10-08T00:55:25.216 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.215+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "08734d45-7b33-4bbc-8931-0e74277a0b6d", "id": 3} v 0) v1 -- 0x7f05e004bb80 con 0x7f05e00b16d0 2023-10-08T00:55:25.222 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.221+0000 7f05c6ffd700 1 -- 172.21.15.184:0/223840065 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "08734d45-7b33-4bbc-8931-0e74277a0b6d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f05d0017070 con 0x7f05e00b16d0 2023-10-08T00:55:25.222 INFO:teuthology.orchestra.run.smithi184.stdout:3 2023-10-08T00:55:25.223 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.222+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b16d0 msgr2=0x7f05e00c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.222+0000 7f05dffff700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b16d0 0x7f05e00c7050 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f05d000ec50 tx=0x7f05d0009250 comp rx=0 tx=0).stop 2023-10-08T00:55:25.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.222+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 shutdown_connections 2023-10-08T00:55:25.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.222+0000 7f05dffff700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05e00b16d0 0x7f05e00c7050 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.222+0000 7f05dffff700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05e015d240 0x7f05e0158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.222+0000 7f05dffff700 1 --2- 172.21.15.184:0/223840065 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05e00b4e70 0x7f05e01459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.222+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 >> 172.21.15.184:0/223840065 conn(0x7f05e00d10c0 msgr2=0x7f05e0069910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:25.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.223+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 shutdown_connections 2023-10-08T00:55:25.224 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.223+0000 7f05dffff700 1 -- 172.21.15.184:0/223840065 wait complete. 2023-10-08T00:55:25.242 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:25.239+0000 7f5fc3832180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T00:55:25.243 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 96b1be81-a6ef-46fc-b9db-7b666d515607 4 2023-10-08T00:55:25.283 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:25.282+0000 7f92d0645180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T00:55:25.335 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:25.335+0000 7f5fc3832180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T00:55:25.382 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:25.381+0000 7f92d0645180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T00:55:25.420 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.420+0000 7fc81359d700 1 Processor -- start 2023-10-08T00:55:25.420 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.420+0000 7fc81359d700 1 -- start start 2023-10-08T00:55:25.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.421+0000 7fc81359d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b16d0 0x7fc80c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.421+0000 7fc81359d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc80c0b33b0 0x7fc80c0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.421+0000 7fc81359d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc80c0b2080 con 0x7fc80c0b4e70 2023-10-08T00:55:25.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.421+0000 7fc81359d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80c0cfb00 con 0x7fc80c0b33b0 2023-10-08T00:55:25.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.421+0000 7fc81359d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0b5c40 con 0x7fc80c0b16d0 2023-10-08T00:55:25.422 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc81259b700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fc80c0b4e70 0x7fc80c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:35854/0 (socket says 172.21.15.184:35854) 2023-10-08T00:55:25.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc81259b700 1 -- 172.21.15.184:0/4180867821 learned_addr learned my addr 172.21.15.184:0/4180867821 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-10-08T00:55:25.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc811d9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b16d0 0x7fc80c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.423 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc812d9c700 1 --2- 172.21.15.184:0/4180867821 >> v2:172.21.15.119:3300/0 conn(0x7fc80c0b33b0 0x7fc80c0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc811599700 1 -- 172.21.15.184:0/4180867821 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539566966 0 0) 0x7fc80c0b2080 con 0x7fc80c0b4e70 2023-10-08T00:55:25.424 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc811d9a700 1 -- 172.21.15.184:0/4180867821 >> v2:172.21.15.119:3300/0 conn(0x7fc80c0b33b0 msgr2=0x7fc80c0b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc811d9a700 1 --2- 172.21.15.184:0/4180867821 >> v2:172.21.15.119:3300/0 conn(0x7fc80c0b33b0 0x7fc80c0b3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc811d9a700 1 -- 172.21.15.184:0/4180867821 >> v1:172.21.15.184:6789/0 conn(0x7fc80c0b4e70 legacy=0x7fc80c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc811d9a700 1 -- 172.21.15.184:0/4180867821 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc80c0b5f20 con 0x7fc80c0b16d0 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.422+0000 7fc811d9a700 1 --2- 172.21.15.184:0/4180867821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b16d0 0x7fc80c0b1b40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098e0 tx=0x7fc7fc0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddd2dd458631fa9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.423+0000 7fc811599700 1 -- 172.21.15.184:0/4180867821 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00b640 con 0x7fc80c0b16d0 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.423+0000 7fc811599700 1 -- 172.21.15.184:0/4180867821 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7fc00b7a0 con 0x7fc80c0b16d0 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.423+0000 7fc811599700 1 -- 172.21.15.184:0/4180867821 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00bb40 con 0x7fc80c0b16d0 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.423+0000 7fc81359d700 1 -- 172.21.15.184:0/4180867821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b16d0 msgr2=0x7fc80c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.425 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.423+0000 7fc81359d700 1 --2- 172.21.15.184:0/4180867821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b16d0 0x7fc80c0b1b40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098e0 tx=0x7fc7fc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.424+0000 7fc81359d700 1 -- 172.21.15.184:0/4180867821 shutdown_connections 2023-10-08T00:55:25.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.424+0000 7fc81359d700 1 --2- 172.21.15.184:0/4180867821 >> v2:172.21.15.119:3300/0 conn(0x7fc80c0b33b0 0x7fc80c0b3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.424+0000 7fc81359d700 1 --2- 172.21.15.184:0/4180867821 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b16d0 0x7fc80c0b1b40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.424+0000 7fc81359d700 1 -- 172.21.15.184:0/4180867821 >> 172.21.15.184:0/4180867821 conn(0x7fc80c0d10c0 msgr2=0x7fc80c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:25.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.424+0000 7fc81359d700 1 -- 172.21.15.184:0/4180867821 shutdown_connections 2023-10-08T00:55:25.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.424+0000 7fc81359d700 1 -- 172.21.15.184:0/4180867821 wait complete. 2023-10-08T00:55:25.426 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.426+0000 7fc81359d700 1 Processor -- start 2023-10-08T00:55:25.427 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.426+0000 7fc81359d700 1 -- start start 2023-10-08T00:55:25.429 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.427+0000 7fc81359d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c0b16d0 0x7fc80c157e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.429 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.427+0000 7fc81359d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b33b0 0x7fc80c0aa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.429 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.427+0000 7fc81359d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc80c0b4e70 0x7fc80c0ac630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.429 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.427+0000 7fc81359d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0ab100 con 0x7fc80c0b4e70 2023-10-08T00:55:25.429 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.427+0000 7fc81359d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0cb6e0 con 0x7fc80c0b16d0 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.427+0000 7fc81359d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80c0cadb0 con 0x7fc80c0b33b0 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.427+0000 7fc81259b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c0b16d0 0x7fc80c157e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc811d9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b33b0 0x7fc80c0aa7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc81259b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c0b16d0 0x7fc80c157e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:52848/0 (socket says 172.21.15.184:52848) 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc81259b700 1 -- 172.21.15.184:0/3548317846 learned_addr learned my addr 172.21.15.184:0/3548317846 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc812d9c700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc80c0b4e70 0x7fc80c0ac630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc812d9c700 1 -- 172.21.15.184:0/3548317846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b33b0 msgr2=0x7fc80c0aa7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc812d9c700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b33b0 0x7fc80c0aa7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc812d9c700 1 -- 172.21.15.184:0/3548317846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c0b16d0 msgr2=0x7fc80c157e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.430 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc812d9c700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c0b16d0 0x7fc80c157e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc812d9c700 1 -- 172.21.15.184:0/3548317846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc009420 con 0x7fc80c0b4e70 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc81259b700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c0b16d0 0x7fc80c157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc812d9c700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc80c0b4e70 0x7fc80c0ac630 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc804009e80 tx=0x7fc80400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.428+0000 7fc8027fc700 1 -- 172.21.15.184:0/3548317846 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80400caa0 con 0x7fc80c0b4e70 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.429+0000 7fc8027fc700 1 -- 172.21.15.184:0/3548317846 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80400f040 con 0x7fc80c0b4e70 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.429+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80c0cb090 con 0x7fc80c0b4e70 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.429+0000 7fc8027fc700 1 -- 172.21.15.184:0/3548317846 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80400ce70 con 0x7fc80c0b4e70 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.429+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c1583a0 con 0x7fc80c0b4e70 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.429+0000 7fc8027fc700 1 -- 172.21.15.184:0/3548317846 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc804014690 con 0x7fc80c0b4e70 2023-10-08T00:55:25.431 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.429+0000 7fc8027fc700 1 -- 172.21.15.184:0/3548317846 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fc80401d4f0 con 0x7fc80c0b4e70 2023-10-08T00:55:25.432 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.430+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80c088990 con 0x7fc80c0b4e70 2023-10-08T00:55:25.432 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.432+0000 7fc8027fc700 1 -- 172.21.15.184:0/3548317846 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc80403b6d0 con 0x7fc80c0b4e70 2023-10-08T00:55:25.496 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:25.496+0000 7f5fc3832180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T00:55:25.516 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.516+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd new", "uuid": "96b1be81-a6ef-46fc-b9db-7b666d515607", "id": 4} v 0) v1 -- 0x7fc80c04bb80 con 0x7fc80c0b4e70 2023-10-08T00:55:25.521 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.520+0000 7fc8027fc700 1 -- 172.21.15.184:0/3548317846 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "96b1be81-a6ef-46fc-b9db-7b666d515607", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc80402a070 con 0x7fc80c0b4e70 2023-10-08T00:55:25.521 INFO:teuthology.orchestra.run.smithi184.stdout:4 2023-10-08T00:55:25.522 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.521+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc80c0b4e70 msgr2=0x7fc80c0ac630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.522 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.521+0000 7fc81359d700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc80c0b4e70 0x7fc80c0ac630 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc804009e80 tx=0x7fc80400b040 comp rx=0 tx=0).stop 2023-10-08T00:55:25.522 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.522+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 shutdown_connections 2023-10-08T00:55:25.522 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.522+0000 7fc81359d700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc80c0b4e70 0x7fc80c0ac630 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.523 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.522+0000 7fc81359d700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc80c0b16d0 0x7fc80c157e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.523 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.522+0000 7fc81359d700 1 --2- 172.21.15.184:0/3548317846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80c0b33b0 0x7fc80c0aa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.523 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.522+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 >> 172.21.15.184:0/3548317846 conn(0x7fc80c0d10c0 msgr2=0x7fc80c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:25.523 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.522+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 shutdown_connections 2023-10-08T00:55:25.523 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.522+0000 7fc81359d700 1 -- 172.21.15.184:0/3548317846 wait complete. 2023-10-08T00:55:25.539 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 44d73378-7e35-4961-8f78-c27cd700d909 5 2023-10-08T00:55:25.552 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:25.552+0000 7f92d0645180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T00:55:25.615 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:25.615+0000 7f5fc3832180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T00:55:25.675 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:25.674+0000 7f92d0645180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T00:55:25.727 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.727+0000 7f44de79d700 1 Processor -- start 2023-10-08T00:55:25.728 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.727+0000 7f44de79d700 1 -- start start 2023-10-08T00:55:25.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.728+0000 7f44de79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.728+0000 7f44de79d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d80b2080 con 0x7f44d80b33b0 2023-10-08T00:55:25.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.728+0000 7f44de79d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d80cfb00 con 0x7f44d80b4e70 2023-10-08T00:55:25.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.728+0000 7f44de79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d80b5c40 con 0x7f44d80b16d0 2023-10-08T00:55:25.729 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.729+0000 7f44dcf9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.730 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.729+0000 7f44dd79b700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f44d80b4e70 0x7f44d80ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:35866/0 (socket says 172.21.15.184:35866) 2023-10-08T00:55:25.730 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.729+0000 7f44dd79b700 1 -- 172.21.15.184:0/336760356 learned_addr learned my addr 172.21.15.184:0/336760356 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.729+0000 7f44dcf9a700 1 -- 172.21.15.184:0/336760356 >> v1:172.21.15.184:6789/0 conn(0x7f44d80b4e70 legacy=0x7f44d80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.729+0000 7f44dcf9a700 1 -- 172.21.15.184:0/336760356 >> v1:172.21.15.119:6789/0 conn(0x7f44d80b33b0 legacy=0x7f44d80b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.729+0000 7f44dcf9a700 1 -- 172.21.15.184:0/336760356 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d80b5f20 con 0x7f44d80b16d0 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.730+0000 7f44dcf9a700 1 --2- 172.21.15.184:0/336760356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80b1b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009910 tx=0x7f44c80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19c2e30a2c845c9d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.730+0000 7f44cffff700 1 -- 172.21.15.184:0/336760356 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800b860 con 0x7f44d80b16d0 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.730+0000 7f44cffff700 1 -- 172.21.15.184:0/336760356 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c800b9c0 con 0x7f44d80b16d0 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.730+0000 7f44cffff700 1 -- 172.21.15.184:0/336760356 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800bd60 con 0x7f44d80b16d0 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.730+0000 7f44de79d700 1 -- 172.21.15.184:0/336760356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 msgr2=0x7f44d80b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.730+0000 7f44de79d700 1 --2- 172.21.15.184:0/336760356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80b1b40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009910 tx=0x7f44c80091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.731+0000 7f44de79d700 1 -- 172.21.15.184:0/336760356 shutdown_connections 2023-10-08T00:55:25.732 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.731+0000 7f44de79d700 1 --2- 172.21.15.184:0/336760356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80b1b40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.731+0000 7f44de79d700 1 -- 172.21.15.184:0/336760356 >> 172.21.15.184:0/336760356 conn(0x7f44d80d10c0 msgr2=0x7f44d80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:25.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.731+0000 7f44de79d700 1 -- 172.21.15.184:0/336760356 shutdown_connections 2023-10-08T00:55:25.733 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.731+0000 7f44de79d700 1 -- 172.21.15.184:0/336760356 wait complete. 2023-10-08T00:55:25.735 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.732+0000 7f44de79d700 1 Processor -- start 2023-10-08T00:55:25.735 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.732+0000 7f44de79d700 1 -- start start 2023-10-08T00:55:25.735 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44de79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.735 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44de79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d80b33b0 0x7f44d80aee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.736 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44de79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d80b4e70 0x7f44d816d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:25.736 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44de79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d816b130 con 0x7f44d80b33b0 2023-10-08T00:55:25.736 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44de79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d80c86b0 con 0x7f44d80b4e70 2023-10-08T00:55:25.736 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44de79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d80c8990 con 0x7f44d80b16d0 2023-10-08T00:55:25.736 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44dd79b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44dd79b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80c7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:58944/0 (socket says 172.21.15.184:58944) 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44dd79b700 1 -- 172.21.15.184:0/3312039920 learned_addr learned my addr 172.21.15.184:0/3312039920 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44ddf9c700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d80b4e70 0x7f44d816d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.733+0000 7f44dcf9a700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d80b33b0 0x7f44d80aee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44dd79b700 1 -- 172.21.15.184:0/3312039920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d80b4e70 msgr2=0x7f44d816d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44dd79b700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d80b4e70 0x7f44d816d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44dd79b700 1 -- 172.21.15.184:0/3312039920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d80b33b0 msgr2=0x7f44d80aee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44dd79b700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d80b33b0 0x7f44d80aee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.737 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44dd79b700 1 -- 172.21.15.184:0/3312039920 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c8009420 con 0x7f44d80b16d0 2023-10-08T00:55:25.738 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44dd79b700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80c7db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c560 tx=0x7f44d4001020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:25.738 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44cd7fa700 1 -- 172.21.15.184:0/3312039920 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4005b30 con 0x7f44d80b16d0 2023-10-08T00:55:25.738 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44cd7fa700 1 -- 172.21.15.184:0/3312039920 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d4005c90 con 0x7f44d80b16d0 2023-10-08T00:55:25.738 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44cd7fa700 1 -- 172.21.15.184:0/3312039920 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d4007320 con 0x7f44d80b16d0 2023-10-08T00:55:25.738 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d80c8c70 con 0x7f44d80b16d0 2023-10-08T00:55:25.738 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.734+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d80c82f0 con 0x7f44d80b16d0 2023-10-08T00:55:25.738 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.735+0000 7f44cd7fa700 1 -- 172.21.15.184:0/3312039920 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f44d401c6c0 con 0x7f44d80b16d0 2023-10-08T00:55:25.738 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.735+0000 7f44cd7fa700 1 -- 172.21.15.184:0/3312039920 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f44d4005e00 con 0x7f44d80b16d0 2023-10-08T00:55:25.739 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.736+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d8088990 con 0x7f44d80b16d0 2023-10-08T00:55:25.739 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.738+0000 7f44cd7fa700 1 -- 172.21.15.184:0/3312039920 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f44d4017070 con 0x7f44d80b16d0 2023-10-08T00:55:25.824 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.824+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "44d73378-7e35-4961-8f78-c27cd700d909", "id": 5} v 0) v1 -- 0x7f44d804bb80 con 0x7f44d80b16d0 2023-10-08T00:55:25.833 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.833+0000 7f44cd7fa700 1 -- 172.21.15.184:0/3312039920 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "44d73378-7e35-4961-8f78-c27cd700d909", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f44d40076d0 con 0x7f44d80b16d0 2023-10-08T00:55:25.833 INFO:teuthology.orchestra.run.smithi184.stdout:5 2023-10-08T00:55:25.834 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 msgr2=0x7f44d80c7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:25.835 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80c7db0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f44d400c560 tx=0x7f44d4001020 comp rx=0 tx=0).stop 2023-10-08T00:55:25.835 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 shutdown_connections 2023-10-08T00:55:25.835 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d80b16d0 0x7f44d80c7db0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.835 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d80b4e70 0x7f44d816d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.835 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 --2- 172.21.15.184:0/3312039920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d80b33b0 0x7f44d80aee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:25.835 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 >> 172.21.15.184:0/3312039920 conn(0x7f44d80d10c0 msgr2=0x7f44d80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:25.835 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 shutdown_connections 2023-10-08T00:55:25.835 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:25.834+0000 7f44de79d700 1 -- 172.21.15.184:0/3312039920 wait complete. 2023-10-08T00:55:25.852 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 138ed202-bc1d-4bd7-934a-784b5cb23b3e 6 2023-10-08T00:55:26.032 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.031+0000 7f6e61b9d700 1 Processor -- start 2023-10-08T00:55:26.032 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.031+0000 7f6e61b9d700 1 -- start start 2023-10-08T00:55:26.034 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.032+0000 7f6e61b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b4e70 0x7f6e5c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.034 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.032+0000 7f6e61b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e5c0b16d0 0x7f6e5c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.034 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.032+0000 7f6e61b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e5c15d240 0x7f6e5c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.034 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.032+0000 7f6e61b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5c0b2080 con 0x7f6e5c15d240 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.032+0000 7f6e61b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e5c0cfb00 con 0x7f6e5c0b16d0 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.032+0000 7f6e61b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0b5c40 con 0x7f6e5c0b4e70 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e5bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e5c0b16d0 0x7f6e5c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e60b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b4e70 0x7f6e5c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e60b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b4e70 0x7f6e5c0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:58958/0 (socket says 172.21.15.184:58958) 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e5bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e5c0b16d0 0x7f6e5c0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:52858/0 (socket says 172.21.15.184:52858) 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e60b9b700 1 -- 172.21.15.184:0/3843644978 learned_addr learned my addr 172.21.15.184:0/3843644978 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e6139c700 1 --2- 172.21.15.184:0/3843644978 >> v2:172.21.15.119:3300/0 conn(0x7f6e5c15d240 0x7f6e5c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e60b9b700 1 -- 172.21.15.184:0/3843644978 >> v2:172.21.15.184:3300/0 conn(0x7f6e5c0b16d0 msgr2=0x7f6e5c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e60b9b700 1 --2- 172.21.15.184:0/3843644978 >> v2:172.21.15.184:3300/0 conn(0x7f6e5c0b16d0 0x7f6e5c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.035 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e60b9b700 1 -- 172.21.15.184:0/3843644978 >> v2:172.21.15.119:3300/0 conn(0x7f6e5c15d240 msgr2=0x7f6e5c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:26.036 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e60b9b700 1 --2- 172.21.15.184:0/3843644978 >> v2:172.21.15.119:3300/0 conn(0x7f6e5c15d240 0x7f6e5c15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.036 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e60b9b700 1 -- 172.21.15.184:0/3843644978 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e5c0b5f20 con 0x7f6e5c0b4e70 2023-10-08T00:55:26.036 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.033+0000 7f6e5bfff700 1 --2- 172.21.15.184:0/3843644978 >> v2:172.21.15.184:3300/0 conn(0x7f6e5c0b16d0 0x7f6e5c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:26.036 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.034+0000 7f6e60b9b700 1 --2- 172.21.15.184:0/3843644978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b4e70 0x7f6e5c0ccff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c089440 tx=0x7f6e50009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56cba7874dd4dcd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:26.036 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.034+0000 7f6e5b7fe700 1 -- 172.21.15.184:0/3843644978 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000b640 con 0x7f6e5c0b4e70 2023-10-08T00:55:26.036 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.034+0000 7f6e5b7fe700 1 -- 172.21.15.184:0/3843644978 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5000b7a0 con 0x7f6e5c0b4e70 2023-10-08T00:55:26.036 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.034+0000 7f6e5b7fe700 1 -- 172.21.15.184:0/3843644978 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5000bb60 con 0x7f6e5c0b4e70 2023-10-08T00:55:26.039 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.034+0000 7f6e61b9d700 1 -- 172.21.15.184:0/3843644978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b4e70 msgr2=0x7f6e5c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.039 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.035+0000 7f6e61b9d700 1 --2- 172.21.15.184:0/3843644978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b4e70 0x7f6e5c0ccff0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c089440 tx=0x7f6e50009250 comp rx=0 tx=0).stop 2023-10-08T00:55:26.039 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.035+0000 7f6e61b9d700 1 -- 172.21.15.184:0/3843644978 shutdown_connections 2023-10-08T00:55:26.039 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.035+0000 7f6e61b9d700 1 --2- 172.21.15.184:0/3843644978 >> v2:172.21.15.119:3300/0 conn(0x7f6e5c15d240 0x7f6e5c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.035+0000 7f6e61b9d700 1 --2- 172.21.15.184:0/3843644978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b4e70 0x7f6e5c0ccff0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.035+0000 7f6e61b9d700 1 --2- 172.21.15.184:0/3843644978 >> v2:172.21.15.184:3300/0 conn(0x7f6e5c0b16d0 0x7f6e5c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.035+0000 7f6e61b9d700 1 -- 172.21.15.184:0/3843644978 >> 172.21.15.184:0/3843644978 conn(0x7f6e5c0d10c0 msgr2=0x7f6e5c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.035+0000 7f6e61b9d700 1 -- 172.21.15.184:0/3843644978 shutdown_connections 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.035+0000 7f6e61b9d700 1 -- 172.21.15.184:0/3843644978 wait complete. 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.036+0000 7f6e61b9d700 1 Processor -- start 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.036+0000 7f6e61b9d700 1 -- start start 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.037+0000 7f6e61b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b16d0 0x7f6e5c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.037+0000 7f6e61b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e5c0b4e70 0x7f6e5c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.037+0000 7f6e61b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5c15d240 0x7f6e5c15c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.040 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.037+0000 7f6e61b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0c79b0 con 0x7f6e5c0b4e70 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.037+0000 7f6e61b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0ca370 con 0x7f6e5c15d240 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.037+0000 7f6e61b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5c0ca650 con 0x7f6e5c0b16d0 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5c15d240 0x7f6e5c15c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e60b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b16d0 0x7f6e5c158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5c15d240 0x7f6e5c15c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.184:52862/0 (socket says 172.21.15.184:52862) 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 -- 172.21.15.184:0/2469946907 learned_addr learned my addr 172.21.15.184:0/2469946907 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e5bfff700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e5c0b4e70 0x7f6e5c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 -- 172.21.15.184:0/2469946907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b16d0 msgr2=0x7f6e5c158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b16d0 0x7f6e5c158330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 -- 172.21.15.184:0/2469946907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e5c0b4e70 msgr2=0x7f6e5c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.041 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e5c0b4e70 0x7f6e5c0c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 -- 172.21.15.184:0/2469946907 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48009420 con 0x7f6e5c15d240 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e60b9b700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b16d0 0x7f6e5c158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e6139c700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5c15d240 0x7f6e5c15c4f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009df0 tx=0x7f6e5400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.038+0000 7f6e58ff9700 1 -- 172.21.15.184:0/2469946907 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400caa0 con 0x7f6e5c15d240 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.039+0000 7f6e58ff9700 1 -- 172.21.15.184:0/2469946907 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5400f040 con 0x7f6e5c15d240 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.039+0000 7f6e58ff9700 1 -- 172.21.15.184:0/2469946907 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5400ce70 con 0x7f6e5c15d240 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.039+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e500094f0 con 0x7f6e5c15d240 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.039+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5c0cac10 con 0x7f6e5c15d240 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.039+0000 7f6e58ff9700 1 -- 172.21.15.184:0/2469946907 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f6e54014690 con 0x7f6e5c15d240 2023-10-08T00:55:26.042 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.039+0000 7f6e58ff9700 1 -- 172.21.15.184:0/2469946907 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f6e5401daa0 con 0x7f6e5c15d240 2023-10-08T00:55:26.043 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.040+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5c088990 con 0x7f6e5c15d240 2023-10-08T00:55:26.044 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.042+0000 7f6e58ff9700 1 -- 172.21.15.184:0/2469946907 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f6e54018020 con 0x7f6e5c15d240 2023-10-08T00:55:26.143 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.143+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd new", "uuid": "138ed202-bc1d-4bd7-934a-784b5cb23b3e", "id": 6} v 0) v1 -- 0x7f6e5c0c7af0 con 0x7f6e5c15d240 2023-10-08T00:55:26.150 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.149+0000 7f6e58ff9700 1 -- 172.21.15.184:0/2469946907 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "138ed202-bc1d-4bd7-934a-784b5cb23b3e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6e54014940 con 0x7f6e5c15d240 2023-10-08T00:55:26.150 INFO:teuthology.orchestra.run.smithi184.stdout:6 2023-10-08T00:55:26.151 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.150+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5c15d240 msgr2=0x7f6e5c15c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.151 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.150+0000 7f6e61b9d700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5c15d240 0x7f6e5c15c4f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e54009df0 tx=0x7f6e5400b040 comp rx=0 tx=0).stop 2023-10-08T00:55:26.151 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.150+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 shutdown_connections 2023-10-08T00:55:26.151 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.150+0000 7f6e61b9d700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e5c0b16d0 0x7f6e5c158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.151 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.150+0000 7f6e61b9d700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e5c15d240 0x7f6e5c15c4f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.152 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.150+0000 7f6e61b9d700 1 --2- 172.21.15.184:0/2469946907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e5c0b4e70 0x7f6e5c0c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.152 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.150+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 >> 172.21.15.184:0/2469946907 conn(0x7f6e5c0d10c0 msgr2=0x7f6e5c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:26.152 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.151+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 shutdown_connections 2023-10-08T00:55:26.152 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.151+0000 7f6e61b9d700 1 -- 172.21.15.184:0/2469946907 wait complete. 2023-10-08T00:55:26.168 DEBUG:teuthology.orchestra.run.smithi184:> sudo ceph --cluster ceph osd new 4b904447-2c07-4602-b89e-f3d50025a783 7 2023-10-08T00:55:26.215 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:26.215+0000 7f5fc3832180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T00:55:26.299 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:26.298+0000 7f92d0645180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T00:55:26.310 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:26.310+0000 7f5fc3832180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T00:55:26.345 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.344+0000 7f0ca09fe700 1 Processor -- start 2023-10-08T00:55:26.345 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.344+0000 7f0ca09fe700 1 -- start start 2023-10-08T00:55:26.347 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0ca09fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b4e70 0x7f0c9c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.347 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0ca09fe700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c0cfb00 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.348 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0ca09fe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c9c15e060 con 0x7f0c9c0b1b40 2023-10-08T00:55:26.348 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0ca09fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c0b5c40 con 0x7f0c9c0b4e70 2023-10-08T00:55:26.348 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0c9ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b4e70 0x7f0c9c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.348 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0c9b59e700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f0c9c0b1b40 0x7f0c9c15d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.184:35876/0 (socket says 172.21.15.184:35876) 2023-10-08T00:55:26.348 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0c9b59e700 1 -- 172.21.15.184:0/554500126 learned_addr learned my addr 172.21.15.184:0/554500126 (peer_addr_for_me v1:172.21.15.184:0/0) 2023-10-08T00:55:26.348 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0c9ad9d700 1 -- 172.21.15.184:0/554500126 >> v1:172.21.15.184:6789/0 conn(0x7f0c9c0b1b40 legacy=0x7f0c9c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.348 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0c9ad9d700 1 -- 172.21.15.184:0/554500126 >> v1:172.21.15.119:6789/0 conn(0x7f0c9c0b16d0 legacy=0x7f0c9c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.349 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.346+0000 7f0c9ad9d700 1 -- 172.21.15.184:0/554500126 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9c0b5f20 con 0x7f0c9c0b4e70 2023-10-08T00:55:26.349 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.347+0000 7f0c9ad9d700 1 --2- 172.21.15.184:0/554500126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b4e70 0x7f0c9c0ccff0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009910 tx=0x7f0c8c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33d6b371d9e3ed6d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:26.349 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.347+0000 7f0c99d9b700 1 -- 172.21.15.184:0/554500126 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00b860 con 0x7f0c9c0b4e70 2023-10-08T00:55:26.349 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.347+0000 7f0c99d9b700 1 -- 172.21.15.184:0/554500126 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c00b9c0 con 0x7f0c9c0b4e70 2023-10-08T00:55:26.349 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.347+0000 7f0c99d9b700 1 -- 172.21.15.184:0/554500126 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00bd60 con 0x7f0c9c0b4e70 2023-10-08T00:55:26.352 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.347+0000 7f0ca09fe700 1 -- 172.21.15.184:0/554500126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b4e70 msgr2=0x7f0c9c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.352 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.347+0000 7f0ca09fe700 1 --2- 172.21.15.184:0/554500126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b4e70 0x7f0c9c0ccff0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c009910 tx=0x7f0c8c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.352 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.348+0000 7f0ca09fe700 1 -- 172.21.15.184:0/554500126 shutdown_connections 2023-10-08T00:55:26.353 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.348+0000 7f0ca09fe700 1 --2- 172.21.15.184:0/554500126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b4e70 0x7f0c9c0ccff0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.353 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.348+0000 7f0ca09fe700 1 -- 172.21.15.184:0/554500126 >> 172.21.15.184:0/554500126 conn(0x7f0c9c0d10c0 msgr2=0x7f0c9c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:26.353 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.348+0000 7f0ca09fe700 1 -- 172.21.15.184:0/554500126 shutdown_connections 2023-10-08T00:55:26.353 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.348+0000 7f0ca09fe700 1 -- 172.21.15.184:0/554500126 wait complete. 2023-10-08T00:55:26.353 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.349+0000 7f0ca09fe700 1 Processor -- start 2023-10-08T00:55:26.353 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.349+0000 7f0ca09fe700 1 -- start start 2023-10-08T00:55:26.353 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0ca09fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b16d0 0x7f0c9c0c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.353 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0ca09fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c9c0b1b40 0x7f0c9c16d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0ca09fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0b4e70 0x7f0c9c169780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0ca09fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c16deb0 con 0x7f0c9c0b4e70 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0ca09fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c16e190 con 0x7f0c9c0b1b40 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0ca09fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9c0c72f0 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0c9ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b16d0 0x7f0c9c0c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0c9a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c9c0b1b40 0x7f0c9c16d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0c9ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b16d0 0x7f0c9c0c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.184:59002/0 (socket says 172.21.15.184:59002) 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0c9ad9d700 1 -- 172.21.15.184:0/722169766 learned_addr learned my addr 172.21.15.184:0/722169766 (peer_addr_for_me v2:172.21.15.184:0/0) 2023-10-08T00:55:26.354 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.350+0000 7f0c9b59e700 1 --2- 172.21.15.184:0/722169766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0b4e70 0x7f0c9c169780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.355 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c9ad9d700 1 -- 172.21.15.184:0/722169766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c9c0b1b40 msgr2=0x7f0c9c16d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.355 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c9ad9d700 1 --2- 172.21.15.184:0/722169766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c9c0b1b40 0x7f0c9c16d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.355 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c9ad9d700 1 -- 172.21.15.184:0/722169766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0b4e70 msgr2=0x7f0c9c169780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.355 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c9ad9d700 1 --2- 172.21.15.184:0/722169766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0b4e70 0x7f0c9c169780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.355 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c9ad9d700 1 -- 172.21.15.184:0/722169766 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c009420 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.355 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c9ad9d700 1 --2- 172.21.15.184:0/722169766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b16d0 0x7f0c9c0c7af0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c010 tx=0x7f0c8c017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:26.355 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c82ffd700 1 -- 172.21.15.184:0/722169766 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c01f070 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.356 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c82ffd700 1 -- 172.21.15.184:0/722169766 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8c00bd40 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.356 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0c82ffd700 1 -- 172.21.15.184:0/722169766 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c01a670 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.356 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.351+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9c0c75d0 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.356 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.352+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9c16daf0 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.356 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.352+0000 7f0c82ffd700 1 -- 172.21.15.184:0/722169766 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f0c8c01ace0 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.356 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.352+0000 7f0c82ffd700 1 -- 172.21.15.184:0/722169766 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f0c8c002f30 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.357 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.353+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c9c0c77f0 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.357 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.355+0000 7f0c82ffd700 1 -- 172.21.15.184:0/722169766 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f0c8c028070 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.399 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:26.398+0000 7f92d0645180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T00:55:26.406 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:26.406+0000 7f5fc3832180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T00:55:26.440 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.440+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4b904447-2c07-4602-b89e-f3d50025a783", "id": 7} v 0) v1 -- 0x7f0c9c04bb80 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.447 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.447+0000 7f0c82ffd700 1 -- 172.21.15.184:0/722169766 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4b904447-2c07-4602-b89e-f3d50025a783", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0c8c015020 con 0x7f0c9c0b16d0 2023-10-08T00:55:26.447 INFO:teuthology.orchestra.run.smithi184.stdout:7 2023-10-08T00:55:26.449 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b16d0 msgr2=0x7f0c9c0c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.449 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 --2- 172.21.15.184:0/722169766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b16d0 0x7f0c9c0c7af0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00c010 tx=0x7f0c8c017000 comp rx=0 tx=0).stop 2023-10-08T00:55:26.449 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 shutdown_connections 2023-10-08T00:55:26.449 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 --2- 172.21.15.184:0/722169766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c9c0b16d0 0x7f0c9c0c7af0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.450 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 --2- 172.21.15.184:0/722169766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c9c0b4e70 0x7f0c9c169780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.450 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 --2- 172.21.15.184:0/722169766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c9c0b1b40 0x7f0c9c16d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.450 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 >> 172.21.15.184:0/722169766 conn(0x7f0c9c0d10c0 msgr2=0x7f0c9c159210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:26.450 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 shutdown_connections 2023-10-08T00:55:26.450 INFO:teuthology.orchestra.run.smithi184.stderr:2023-10-08T00:55:26.448+0000 7f0ca09fe700 1 -- 172.21.15.184:0/722169766 wait complete. 2023-10-08T00:55:26.465 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-08T00:55:26.466 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-08T00:55:26.470 INFO:tasks.ceph.osd.0:Started 2023-10-08T00:55:26.471 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-08T00:55:26.471 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-08T00:55:26.475 INFO:tasks.ceph.osd.1:Started 2023-10-08T00:55:26.475 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-08T00:55:26.476 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-08T00:55:26.480 INFO:tasks.ceph.osd.2:Started 2023-10-08T00:55:26.481 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-08T00:55:26.481 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-08T00:55:26.486 INFO:tasks.ceph.osd.3:Started 2023-10-08T00:55:26.486 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-08T00:55:26.487 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-08T00:55:26.490 INFO:tasks.ceph.osd.4:Started 2023-10-08T00:55:26.491 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-08T00:55:26.491 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-08T00:55:26.495 INFO:tasks.ceph.osd.5:Started 2023-10-08T00:55:26.495 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-08T00:55:26.495 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-08T00:55:26.500 INFO:tasks.ceph.osd.6:Started 2023-10-08T00:55:26.500 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-08T00:55:26.500 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-08T00:55:26.501 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:26.499+0000 7f92d0645180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T00:55:26.507 INFO:tasks.ceph.osd.7:Started 2023-10-08T00:55:26.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:26.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.742+0000 7f503119d700 1 Processor -- start 2023-10-08T00:55:26.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.743+0000 7f503119d700 1 -- start start 2023-10-08T00:55:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f503119d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f502c142ea0 0x7f502c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f503119d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0ad870 0x7f502c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f503119d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502c0b40f0 con 0x7f502c0b3b70 2023-10-08T00:55:26.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f503119d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502c069640 con 0x7f502c142ea0 2023-10-08T00:55:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f503119d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c069920 con 0x7f502c0ad870 2023-10-08T00:55:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f502bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f502c142ea0 0x7f502c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f503099c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f502c0b3b70 0x7f502c06cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41540/0 (socket says 172.21.15.119:41540) 2023-10-08T00:55:26.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f503099c700 1 -- 172.21.15.119:0/765570630 learned_addr learned my addr 172.21.15.119:0/765570630 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f502b7fe700 1 --2- 172.21.15.119:0/765570630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0ad870 0x7f502c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.744+0000 7f502affd700 1 -- 172.21.15.119:0/765570630 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000289120 0 0) 0x7f502c0b40f0 con 0x7f502c0b3b70 2023-10-08T00:55:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.745+0000 7f502bfff700 1 -- 172.21.15.119:0/765570630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0ad870 msgr2=0x7f502c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:26.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.745+0000 7f502bfff700 1 --2- 172.21.15.119:0/765570630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0ad870 0x7f502c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.745+0000 7f502bfff700 1 -- 172.21.15.119:0/765570630 >> v1:172.21.15.119:6789/0 conn(0x7f502c0b3b70 legacy=0x7f502c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.745+0000 7f502bfff700 1 -- 172.21.15.119:0/765570630 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c069c00 con 0x7f502c142ea0 2023-10-08T00:55:26.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.745+0000 7f502bfff700 1 --2- 172.21.15.119:0/765570630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c142ea0 0x7f502c0b3630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f501c0098e0 tx=0x7f501c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d1f24011abdf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:26.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.745+0000 7f502affd700 1 -- 172.21.15.119:0/765570630 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00b640 con 0x7f502c142ea0 2023-10-08T00:55:26.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.745+0000 7f502affd700 1 -- 172.21.15.119:0/765570630 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c00b7a0 con 0x7f502c142ea0 2023-10-08T00:55:26.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.745+0000 7f502affd700 1 -- 172.21.15.119:0/765570630 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00bb40 con 0x7f502c142ea0 2023-10-08T00:55:26.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.746+0000 7f503119d700 1 -- 172.21.15.119:0/765570630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c142ea0 msgr2=0x7f502c0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.746+0000 7f503119d700 1 --2- 172.21.15.119:0/765570630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c142ea0 0x7f502c0b3630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f501c0098e0 tx=0x7f501c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.746+0000 7f503119d700 1 -- 172.21.15.119:0/765570630 shutdown_connections 2023-10-08T00:55:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.746+0000 7f503119d700 1 --2- 172.21.15.119:0/765570630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c142ea0 0x7f502c0b3630 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.746+0000 7f503119d700 1 --2- 172.21.15.119:0/765570630 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c0ad870 0x7f502c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.746+0000 7f503119d700 1 -- 172.21.15.119:0/765570630 >> 172.21.15.119:0/765570630 conn(0x7f502c0ce430 msgr2=0x7f502c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:26.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.746+0000 7f503119d700 1 -- 172.21.15.119:0/765570630 shutdown_connections 2023-10-08T00:55:26.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.746+0000 7f503119d700 1 -- 172.21.15.119:0/765570630 wait complete. 2023-10-08T00:55:26.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.747+0000 7f503119d700 1 Processor -- start 2023-10-08T00:55:26.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.747+0000 7f503119d700 1 -- start start 2023-10-08T00:55:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f503119d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c0b3b70 0x7f502c169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f503119d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c142ea0 0x7f502c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f503119d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c159100 0x7f502c15cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f503119d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c16ab40 con 0x7f502c0b3b70 2023-10-08T00:55:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f503119d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c0ca4f0 con 0x7f502c142ea0 2023-10-08T00:55:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f503119d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c16d3d0 con 0x7f502c159100 2023-10-08T00:55:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f502bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c0b3b70 0x7f502c169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f502bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c0b3b70 0x7f502c169ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58124/0 (socket says 172.21.15.119:58124) 2023-10-08T00:55:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.748+0000 7f502bfff700 1 -- 172.21.15.119:0/561183838 learned_addr learned my addr 172.21.15.119:0/561183838 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f502b7fe700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c142ea0 0x7f502c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f503099c700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c159100 0x7f502c15cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f502bfff700 1 -- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c159100 msgr2=0x7f502c15cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f502bfff700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c159100 0x7f502c15cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f502bfff700 1 -- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c142ea0 msgr2=0x7f502c0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f502bfff700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c142ea0 0x7f502c0c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f502bfff700 1 -- 172.21.15.119:0/561183838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c009420 con 0x7f502c0b3b70 2023-10-08T00:55:26.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f503099c700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c159100 0x7f502c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:26.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f502bfff700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c0b3b70 0x7f502c169ba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f501c009f80 tx=0x7f501c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:26.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f5013fff700 1 -- 172.21.15.119:0/561183838 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c00bb20 con 0x7f502c0b3b70 2023-10-08T00:55:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f5013fff700 1 -- 172.21.15.119:0/561183838 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501c000c90 con 0x7f502c0b3b70 2023-10-08T00:55:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502c16a2a0 con 0x7f502c0b3b70 2023-10-08T00:55:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.749+0000 7f5013fff700 1 -- 172.21.15.119:0/561183838 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501c011650 con 0x7f502c0b3b70 2023-10-08T00:55:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.750+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502c16a860 con 0x7f502c0b3b70 2023-10-08T00:55:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.750+0000 7f5013fff700 1 -- 172.21.15.119:0/561183838 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f501c011c80 con 0x7f502c0b3b70 2023-10-08T00:55:26.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.750+0000 7f5013fff700 1 -- 172.21.15.119:0/561183838 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f501c02dad0 con 0x7f502c0b3b70 2023-10-08T00:55:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.750+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502c04c3c0 con 0x7f502c0b3b70 2023-10-08T00:55:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.754+0000 7f5013fff700 1 -- 172.21.15.119:0/561183838 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f501c029070 con 0x7f502c0b3b70 2023-10-08T00:55:26.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.825+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f502c16d1a0 con 0x7f502c0b3b70 2023-10-08T00:55:26.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.826+0000 7f5013fff700 1 -- 172.21.15.119:0/561183838 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f501c015020 con 0x7f502c0b3b70 2023-10-08T00:55:26.827 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:26.827 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":10,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:26.441992+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c0b3b70 msgr2=0x7f502c169ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c0b3b70 0x7f502c169ba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f501c009f80 tx=0x7f501c017000 comp rx=0 tx=0).stop 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 shutdown_connections 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f502c159100 0x7f502c15cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f502c0b3b70 0x7f502c169ba0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 --2- 172.21.15.119:0/561183838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f502c142ea0 0x7f502c0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 >> 172.21.15.119:0/561183838 conn(0x7f502c0ce430 msgr2=0x7f502c0cc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 shutdown_connections 2023-10-08T00:55:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:26.827+0000 7f503119d700 1 -- 172.21.15.119:0/561183838 wait complete. 2023-10-08T00:55:26.846 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-08T00:55:26.846 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-08T00:55:27.162 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-10-08T00:55:27.162+0000 7fe07cfba540 -1 Falling back to public interface 2023-10-08T00:55:27.163 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-08T00:55:27.163+0000 7fad75e80540 -1 Falling back to public interface 2023-10-08T00:55:27.173 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-08T00:55:27.173+0000 7f3144fe4540 -1 Falling back to public interface 2023-10-08T00:55:27.175 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-10-08T00:55:27.175+0000 7f10e9eb3540 -1 Falling back to public interface 2023-10-08T00:55:27.176 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-10-08T00:55:27.175+0000 7f9236201540 -1 Falling back to public interface 2023-10-08T00:55:27.176 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-10-08T00:55:27.176+0000 7f922d3fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6809/0: (98) Address already in use 2023-10-08T00:55:27.187 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-08T00:55:27.187+0000 7f78f4d27540 -1 Falling back to public interface 2023-10-08T00:55:27.191 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-10-08T00:55:27.191+0000 7fbf0b153540 -1 Falling back to public interface 2023-10-08T00:55:27.197 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:27.197+0000 7f5fc3832180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T00:55:27.206 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-08T00:55:27.205+0000 7fb4549c0540 -1 Falling back to public interface 2023-10-08T00:55:27.278 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:27.277+0000 7f92d0645180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T00:55:27.369 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:27.369+0000 7f5fc3832180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:55:27.447 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:27.446+0000 7f92d0645180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:55:27.503 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:27.503+0000 7f5fc3832180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T00:55:27.560 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:27.559+0000 7f92d0645180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T00:55:27.655 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:27.806 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:27.806+0000 7f5fc3832180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T00:55:27.818 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.818+0000 7f00a09fe700 1 Processor -- start 2023-10-08T00:55:27.818 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.818+0000 7f00a09fe700 1 -- start start 2023-10-08T00:55:27.821 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.819+0000 7f00a09fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15dcc0 0x7f009c0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:27.821 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f00a09fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f009c15b140 0x7f009c0b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:27.821 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f00a09fe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f009c159ef0 con 0x7f009c15ce50 2023-10-08T00:55:27.821 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f00a09fe700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f009c0b6090 con 0x7f009c15b140 2023-10-08T00:55:27.821 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f00a09fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c0b51b0 con 0x7f009c15dcc0 2023-10-08T00:55:27.821 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f009c15b140 0x7f009c0b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f009c15b140 0x7f009c0b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58136/0 (socket says 172.21.15.119:58136) 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009b59e700 1 -- 172.21.15.119:0/782963540 learned_addr learned my addr 172.21.15.119:0/782963540 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009a59c700 1 --2- 172.21.15.119:0/782963540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15dcc0 0x7f009c0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009b59e700 1 -- 172.21.15.119:0/782963540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15dcc0 msgr2=0x7f009c0b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009b59e700 1 --2- 172.21.15.119:0/782963540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15dcc0 0x7f009c0b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009b59e700 1 -- 172.21.15.119:0/782963540 >> v1:172.21.15.184:6789/0 conn(0x7f009c15ce50 legacy=0x7f009c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009b59e700 1 -- 172.21.15.119:0/782963540 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0b5480 con 0x7f009c15b140 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.820+0000 7f009b59e700 1 --2- 172.21.15.119:0/782963540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15b140 0x7f009c0b3c60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0094009f20 tx=0x7f0094009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba0eef9b6d014cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f0099d9b700 1 -- 172.21.15.119:0/782963540 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400e070 con 0x7f009c15b140 2023-10-08T00:55:27.822 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f0099d9b700 1 -- 172.21.15.119:0/782963540 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009400ba60 con 0x7f009c15b140 2023-10-08T00:55:27.823 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f0099d9b700 1 -- 172.21.15.119:0/782963540 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00940092e0 con 0x7f009c15b140 2023-10-08T00:55:27.823 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f00a09fe700 1 -- 172.21.15.119:0/782963540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15b140 msgr2=0x7f009c0b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f00a09fe700 1 --2- 172.21.15.119:0/782963540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15b140 0x7f009c0b3c60 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0094009f20 tx=0x7f0094009840 comp rx=0 tx=0).stop 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f00a09fe700 1 -- 172.21.15.119:0/782963540 shutdown_connections 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f00a09fe700 1 --2- 172.21.15.119:0/782963540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15b140 0x7f009c0b3c60 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f00a09fe700 1 --2- 172.21.15.119:0/782963540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15dcc0 0x7f009c0b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f00a09fe700 1 -- 172.21.15.119:0/782963540 >> 172.21.15.119:0/782963540 conn(0x7f009c087ff0 msgr2=0x7f009c06d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f00a09fe700 1 -- 172.21.15.119:0/782963540 shutdown_connections 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.821+0000 7f00a09fe700 1 -- 172.21.15.119:0/782963540 wait complete. 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.823+0000 7f00a09fe700 1 Processor -- start 2023-10-08T00:55:27.824 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.823+0000 7f00a09fe700 1 -- start start 2023-10-08T00:55:27.826 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f00a09fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15b140 0x7f009c146680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:27.826 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f00a09fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15ce50 0x7f009c157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:27.826 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f00a09fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f009c15dcc0 0x7f009c158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f00a09fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c09ad80 con 0x7f009c15ce50 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f00a09fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f009c09a170 con 0x7f009c15dcc0 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f00a09fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009c09a450 con 0x7f009c15b140 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15ce50 0x7f009c157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15b140 0x7f009c146680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f009c15dcc0 0x7f009c158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15b140 0x7f009c146680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:59012/0 (socket says 172.21.15.119:59012) 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15ce50 0x7f009c157fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58140/0 (socket says 172.21.15.119:58140) 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009ad9d700 1 -- 172.21.15.119:0/2904709074 learned_addr learned my addr 172.21.15.119:0/2904709074 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009ad9d700 1 -- 172.21.15.119:0/2904709074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f009c15dcc0 msgr2=0x7f009c158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:27.827 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009ad9d700 1 --2- 172.21.15.119:0/2904709074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f009c15dcc0 0x7f009c158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.824+0000 7f009ad9d700 1 -- 172.21.15.119:0/2904709074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15ce50 msgr2=0x7f009c157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f009ad9d700 1 --2- 172.21.15.119:0/2904709074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15ce50 0x7f009c157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f009ad9d700 1 -- 172.21.15.119:0/2904709074 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0094009ae0 con 0x7f009c15b140 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f009ad9d700 1 --2- 172.21.15.119:0/2904709074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15b140 0x7f009c146680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f008c00c7c0 tx=0x7f008c007f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f0082ffd700 1 -- 172.21.15.119:0/2904709074 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c011590 con 0x7f009c15b140 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f0082ffd700 1 -- 172.21.15.119:0/2904709074 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008c0116f0 con 0x7f009c15b140 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f0082ffd700 1 -- 172.21.15.119:0/2904709074 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008c011ab0 con 0x7f009c15b140 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f009c09a730 con 0x7f009c15b140 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f009c146bc0 con 0x7f009c15b140 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f0082ffd700 1 -- 172.21.15.119:0/2904709074 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f008c01c6c0 con 0x7f009c15b140 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.825+0000 7f0082ffd700 1 -- 172.21.15.119:0/2904709074 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f008c012b00 con 0x7f009c15b140 2023-10-08T00:55:27.828 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.826+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009c064b50 con 0x7f009c15b140 2023-10-08T00:55:27.829 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.829+0000 7f0082ffd700 1 -- 172.21.15.119:0/2904709074 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f008c017070 con 0x7f009c15b140 2023-10-08T00:55:27.862 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:27.862+0000 7f92d0645180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T00:55:27.901 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.901+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f009c0b1d20 con 0x7f009c15b140 2023-10-08T00:55:27.902 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.902+0000 7f0082ffd700 1 -- 172.21.15.119:0/2904709074 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f008c01c970 con 0x7f009c15b140 2023-10-08T00:55:27.902 INFO:teuthology.misc.health.smithi119.stdout: 2023-10-08T00:55:27.902 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":10,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:26.441992+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:27.903 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15b140 msgr2=0x7f009c146680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:27.903 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 --2- 172.21.15.119:0/2904709074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15b140 0x7f009c146680 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f008c00c7c0 tx=0x7f008c007f70 comp rx=0 tx=0).stop 2023-10-08T00:55:27.903 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 shutdown_connections 2023-10-08T00:55:27.903 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 --2- 172.21.15.119:0/2904709074 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f009c15b140 0x7f009c146680 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:27.903 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 --2- 172.21.15.119:0/2904709074 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f009c15dcc0 0x7f009c158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:27.903 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 --2- 172.21.15.119:0/2904709074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f009c15ce50 0x7f009c157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:27.904 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 >> 172.21.15.119:0/2904709074 conn(0x7f009c087ff0 msgr2=0x7f009c0b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:27.904 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 shutdown_connections 2023-10-08T00:55:27.904 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:27.903+0000 7f00a09fe700 1 -- 172.21.15.119:0/2904709074 wait complete. 2023-10-08T00:55:27.906 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:27.906+0000 7f5fc3832180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T00:55:27.917 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-08T00:55:27.959 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:27.959+0000 7f92d0645180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T00:55:28.905 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:28.905+0000 7f5fc3832180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T00:55:28.970 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:28.970+0000 7f92d0645180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T00:55:28.996 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:28.996+0000 7f5fc3832180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T00:55:29.062 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:29.061+0000 7f92d0645180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T00:55:29.089 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:29.089+0000 7f5fc3832180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T00:55:29.157 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:29.157+0000 7f92d0645180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T00:55:29.253 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:29.253+0000 7f5fc3832180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T00:55:29.318 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:29.317+0000 7f92d0645180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T00:55:29.357 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:29.357+0000 7f5fc3832180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T00:55:29.424 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:29.424+0000 7f92d0645180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T00:55:29.547 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:29.547+0000 7f5fc3832180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T00:55:29.613 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:29.613+0000 7f92d0645180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T00:55:29.649 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:29.649+0000 7f5fc3832180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T00:55:29.730 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:29.730+0000 7f92d0645180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T00:55:29.753 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:29.753+0000 7f5fc3832180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T00:55:29.823 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:29.823+0000 7f92d0645180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T00:55:29.845 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:29.845+0000 7f5fc3832180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T00:55:29.940 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:55:29.940+0000 7f92d0645180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T00:55:30.686 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-10-08T00:55:30.686+0000 7f10e9eb3540 -1 osd.3 0 log_to_monitors true 2023-10-08T00:55:30.702 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-08T00:55:30.701+0000 7f78f4d27540 -1 osd.5 0 log_to_monitors true 2023-10-08T00:55:30.728 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-10-08T00:55:30.728+0000 7f9236201540 -1 osd.0 0 log_to_monitors true 2023-10-08T00:55:30.740 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-10-08T00:55:30.740+0000 7fbf0b153540 -1 osd.2 0 log_to_monitors true 2023-10-08T00:55:30.746 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-08T00:55:30.746+0000 7f3144fe4540 -1 osd.6 0 log_to_monitors true 2023-10-08T00:55:30.749 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-10-08T00:55:30.749+0000 7fe07cfba540 -1 osd.1 0 log_to_monitors true 2023-10-08T00:55:30.756 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-08T00:55:30.755+0000 7fad75e80540 -1 osd.7 0 log_to_monitors true 2023-10-08T00:55:30.778 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-08T00:55:30.777+0000 7fb4549c0540 -1 osd.4 0 log_to_monitors true 2023-10-08T00:55:30.863 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:30.863+0000 7f5f7bbfa700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-08T00:55:30.863 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:55:30.863+0000 7f5f7bbfa700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-08T00:55:34.215 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-08T00:55:34.215+0000 7fad699f6700 -1 osd.7 0 waiting for initial osdmap 2023-10-08T00:55:34.216 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-10-08T00:55:34.215+0000 7fbefebf6700 -1 osd.2 0 waiting for initial osdmap 2023-10-08T00:55:34.223 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-10-08T00:55:34.222+0000 7fe0721f9700 -1 osd.1 0 waiting for initial osdmap 2023-10-08T00:55:34.224 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-10-08T00:55:34.224+0000 7fbef7be8700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:55:34.227 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-08T00:55:34.226+0000 7fad629e8700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:55:34.228 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-08T00:55:34.228+0000 7fb449bf9700 -1 osd.4 0 waiting for initial osdmap 2023-10-08T00:55:34.233 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-10-08T00:55:34.232+0000 7fe0699e8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:55:34.237 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-08T00:55:34.236+0000 7fb4413e8700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:55:34.244 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-10-08T00:55:34.244+0000 7f10df1f9700 -1 osd.3 0 waiting for initial osdmap 2023-10-08T00:55:34.252 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-10-08T00:55:34.252+0000 7f10d69e8700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:55:34.258 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-08T00:55:34.258+0000 7f78e9ff9700 -1 osd.5 0 waiting for initial osdmap 2023-10-08T00:55:34.259 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-10-08T00:55:34.259+0000 7f9229bf6700 -1 osd.0 0 waiting for initial osdmap 2023-10-08T00:55:34.260 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-08T00:55:34.260+0000 7f313a1f9700 -1 osd.6 0 waiting for initial osdmap 2023-10-08T00:55:34.263 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-08T00:55:34.263+0000 7f78e17e8700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:55:34.266 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-10-08T00:55:34.266+0000 7f9222be8700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:55:34.268 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-08T00:55:34.268+0000 7f31319e8700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-08T00:55:34.723 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:34.880 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.880+0000 7f4e17dff700 1 Processor -- start 2023-10-08T00:55:34.881 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.880+0000 7f4e17dff700 1 -- start start 2023-10-08T00:55:34.882 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.881+0000 7f4e17dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4e101562f0 0x7f4e100c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:34.882 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.881+0000 7f4e17dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4e1006b5b0 0x7f4e100c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:34.882 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.881+0000 7f4e17dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e100c8150 con 0x7f4e101566c0 2023-10-08T00:55:34.882 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.881+0000 7f4e17dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e10156e70 con 0x7f4e101562f0 2023-10-08T00:55:34.882 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.881+0000 7f4e17dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e100c7540 con 0x7f4e1006b5b0 2023-10-08T00:55:34.882 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e15b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4e101562f0 0x7f4e100c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:34.882 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e15b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4e101562f0 0x7f4e100c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50186/0 (socket says 172.21.15.119:50186) 2023-10-08T00:55:34.882 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e15b9b700 1 -- 172.21.15.119:0/4080259746 learned_addr learned my addr 172.21.15.119:0/4080259746 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:34.884 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e1639c700 1 --2- 172.21.15.119:0/4080259746 >> v2:172.21.15.184:3300/0 conn(0x7f4e1006b5b0 0x7f4e100c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:34.884 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e15b9b700 1 -- 172.21.15.119:0/4080259746 >> v2:172.21.15.184:3300/0 conn(0x7f4e1006b5b0 msgr2=0x7f4e100c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:34.884 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e15b9b700 1 --2- 172.21.15.119:0/4080259746 >> v2:172.21.15.184:3300/0 conn(0x7f4e1006b5b0 0x7f4e100c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e15b9b700 1 -- 172.21.15.119:0/4080259746 >> v1:172.21.15.184:6789/0 conn(0x7f4e101566c0 legacy=0x7f4e1006aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e15b9b700 1 -- 172.21.15.119:0/4080259746 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e100c7820 con 0x7f4e101562f0 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e1639c700 1 --2- 172.21.15.119:0/4080259746 >> v2:172.21.15.184:3300/0 conn(0x7f4e1006b5b0 0x7f4e100c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e15b9b700 1 --2- 172.21.15.119:0/4080259746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100c69d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e10069c10 tx=0x7f4e0c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e60af94e1ccb2f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.882+0000 7f4e14b99700 1 -- 172.21.15.119:0/4080259746 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c005c10 con 0x7f4e101562f0 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e14b99700 1 -- 172.21.15.119:0/4080259746 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c005d70 con 0x7f4e101562f0 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e14b99700 1 -- 172.21.15.119:0/4080259746 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00b840 con 0x7f4e101562f0 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e17dff700 1 -- 172.21.15.119:0/4080259746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 msgr2=0x7f4e100c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e17dff700 1 --2- 172.21.15.119:0/4080259746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100c69d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4e10069c10 tx=0x7f4e0c009220 comp rx=0 tx=0).stop 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e17dff700 1 -- 172.21.15.119:0/4080259746 shutdown_connections 2023-10-08T00:55:34.885 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e17dff700 1 --2- 172.21.15.119:0/4080259746 >> v2:172.21.15.184:3300/0 conn(0x7f4e1006b5b0 0x7f4e100c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:34.886 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e17dff700 1 --2- 172.21.15.119:0/4080259746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100c69d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:34.886 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e17dff700 1 -- 172.21.15.119:0/4080259746 >> 172.21.15.119:0/4080259746 conn(0x7f4e1006d350 msgr2=0x7f4e100cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:34.886 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e17dff700 1 -- 172.21.15.119:0/4080259746 shutdown_connections 2023-10-08T00:55:34.886 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.883+0000 7f4e17dff700 1 -- 172.21.15.119:0/4080259746 wait complete. 2023-10-08T00:55:34.888 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.885+0000 7f4e17dff700 1 Processor -- start 2023-10-08T00:55:34.888 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.885+0000 7f4e17dff700 1 -- start start 2023-10-08T00:55:34.888 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e17dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e1006b5b0 0x7f4e100c3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e17dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e17dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e101566c0 0x7f4e100ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e17dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e100c66e0 con 0x7f4e101562f0 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e17dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e10169a30 con 0x7f4e101566c0 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e17dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e100c1a10 con 0x7f4e1006b5b0 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e1539a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e1539a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50200/0 (socket says 172.21.15.119:50200) 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e1539a700 1 -- 172.21.15.119:0/2085017357 learned_addr learned my addr 172.21.15.119:0/2085017357 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:34.889 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e1639c700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e101566c0 0x7f4e100ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.886+0000 7f4e15b9b700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e1006b5b0 0x7f4e100c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e1539a700 1 -- 172.21.15.119:0/2085017357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e1006b5b0 msgr2=0x7f4e100c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e1539a700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e1006b5b0 0x7f4e100c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e1539a700 1 -- 172.21.15.119:0/2085017357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e101566c0 msgr2=0x7f4e100ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e1539a700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e101566c0 0x7f4e100ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e1539a700 1 -- 172.21.15.119:0/2085017357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009480 con 0x7f4e101562f0 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e1539a700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100ae3b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b5e0 tx=0x7f4e00009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e05ffb700 1 -- 172.21.15.119:0/2085017357 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000f510 con 0x7f4e101562f0 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e05ffb700 1 -- 172.21.15.119:0/2085017357 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0000f670 con 0x7f4e101562f0 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e100c1520 con 0x7f4e101562f0 2023-10-08T00:55:34.890 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e05ffb700 1 -- 172.21.15.119:0/2085017357 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000fa30 con 0x7f4e101562f0 2023-10-08T00:55:34.891 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.887+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e101693e0 con 0x7f4e101562f0 2023-10-08T00:55:34.892 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.888+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e10169ba0 con 0x7f4e101562f0 2023-10-08T00:55:34.892 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.889+0000 7f4e05ffb700 1 -- 172.21.15.119:0/2085017357 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7f4e0001a400 con 0x7f4e101562f0 2023-10-08T00:55:34.892 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.891+0000 7f4e05ffb700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4df4066ea0 0x7f4df4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:34.892 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.891+0000 7f4e05ffb700 1 -- 172.21.15.119:0/2085017357 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7f4e000830d0 con 0x7f4e101562f0 2023-10-08T00:55:34.892 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.891+0000 7f4e15b9b700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4df4066ea0 0x7f4df4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:34.892 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.891+0000 7f4e15b9b700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4df4066ea0 0x7f4df4069350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c003080 tx=0x7f4e0c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:34.895 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:34.895+0000 7f4e05ffb700 1 -- 172.21.15.119:0/2085017357 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e00051190 con 0x7f4e101562f0 2023-10-08T00:55:35.090 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.090+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e100c7b00 con 0x7f4e101562f0 2023-10-08T00:55:35.090 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.090+0000 7f4e05ffb700 1 -- 172.21.15.119:0/2085017357 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f4e000547b0 con 0x7f4e101562f0 2023-10-08T00:55:35.091 INFO:teuthology.misc.health.smithi119.stdout: 2023-10-08T00:55:35.091 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":12,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:31.902711+0000","last_up_change":"0.000000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:35.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4df4066ea0 msgr2=0x7f4df4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:35.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4df4066ea0 0x7f4df4069350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c003080 tx=0x7f4e0c015000 comp rx=0 tx=0).stop 2023-10-08T00:55:35.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 msgr2=0x7f4e100ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100ae3b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4e0000b5e0 tx=0x7f4e00009120 comp rx=0 tx=0).stop 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 shutdown_connections 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e1006b5b0 0x7f4e100c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4df4066ea0 0x7f4df4069350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e101566c0 0x7f4e100ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 --2- 172.21.15.119:0/2085017357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e101562f0 0x7f4e100ae3b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 >> 172.21.15.119:0/2085017357 conn(0x7f4e1006d350 msgr2=0x7f4e10157040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 shutdown_connections 2023-10-08T00:55:35.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:35.093+0000 7f4e17dff700 1 -- 172.21.15.119:0/2085017357 wait complete. 2023-10-08T00:55:35.108 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-08T00:55:41.918 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:42.084 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.083+0000 7fe9c639d700 1 Processor -- start 2023-10-08T00:55:42.084 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.083+0000 7fe9c639d700 1 -- start start 2023-10-08T00:55:42.086 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.084+0000 7fe9c639d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9c00ac020 0x7fe9c00ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.087 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.084+0000 7fe9c639d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c006f390 con 0x7fe9c00aff00 2023-10-08T00:55:42.087 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.084+0000 7fe9c639d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c006e780 con 0x7fe9c00ad7a0 2023-10-08T00:55:42.087 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.084+0000 7fe9c639d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c006ea60 con 0x7fe9c00ac020 2023-10-08T00:55:42.087 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9c5b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe9c00aff00 0x7fe9c0163340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42132/0 (socket says 172.21.15.119:42132) 2023-10-08T00:55:42.087 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9c539b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9c00ac020 0x7fe9c00ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.087 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9c5b9c700 1 -- 172.21.15.119:0/2779621336 learned_addr learned my addr 172.21.15.119:0/2779621336 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:42.087 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9b7fff700 1 -- 172.21.15.119:0/2779621336 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944883644 0 0) 0x7fe9c006f390 con 0x7fe9c00aff00 2023-10-08T00:55:42.088 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9b7fff700 1 -- 172.21.15.119:0/2779621336 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ac003570 con 0x7fe9c00aff00 2023-10-08T00:55:42.088 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9c539b700 1 -- 172.21.15.119:0/2779621336 >> v1:172.21.15.184:6789/0 conn(0x7fe9c00ad7a0 legacy=0x7fe9c00ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.088 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9c539b700 1 -- 172.21.15.119:0/2779621336 >> v1:172.21.15.119:6789/0 conn(0x7fe9c00aff00 legacy=0x7fe9c0163340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.088 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9c539b700 1 -- 172.21.15.119:0/2779621336 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c006ed40 con 0x7fe9c00ac020 2023-10-08T00:55:42.088 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9c539b700 1 --2- 172.21.15.119:0/2779621336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00ad260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0098e0 tx=0x7fe9bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac9323cf2e19003 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.088 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.085+0000 7fe9b7fff700 1 -- 172.21.15.119:0/2779621336 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00b640 con 0x7fe9c00ac020 2023-10-08T00:55:42.088 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9b7fff700 1 -- 172.21.15.119:0/2779621336 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc00b7a0 con 0x7fe9c00ac020 2023-10-08T00:55:42.088 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9b7fff700 1 -- 172.21.15.119:0/2779621336 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00bb40 con 0x7fe9c00ac020 2023-10-08T00:55:42.089 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9c639d700 1 -- 172.21.15.119:0/2779621336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 msgr2=0x7fe9c00ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.089 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9c639d700 1 --2- 172.21.15.119:0/2779621336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00ad260 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0098e0 tx=0x7fe9bc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.089 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9c639d700 1 -- 172.21.15.119:0/2779621336 shutdown_connections 2023-10-08T00:55:42.089 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9c639d700 1 --2- 172.21.15.119:0/2779621336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00ad260 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.089 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9c639d700 1 -- 172.21.15.119:0/2779621336 >> 172.21.15.119:0/2779621336 conn(0x7fe9c0087ff0 msgr2=0x7fe9c00ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:42.090 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9c639d700 1 -- 172.21.15.119:0/2779621336 shutdown_connections 2023-10-08T00:55:42.090 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.086+0000 7fe9c639d700 1 -- 172.21.15.119:0/2779621336 wait complete. 2023-10-08T00:55:42.090 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.088+0000 7fe9c639d700 1 Processor -- start 2023-10-08T00:55:42.090 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.088+0000 7fe9c639d700 1 -- start start 2023-10-08T00:55:42.092 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c639d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.092 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c639d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9c00ad7a0 0x7fe9c0152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.092 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c639d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9c00aff00 0x7fe9c01546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.092 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c639d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00cd9f0 con 0x7fe9c00ac020 2023-10-08T00:55:42.092 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c639d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00cbee0 con 0x7fe9c00aff00 2023-10-08T00:55:42.092 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c639d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00cb5c0 con 0x7fe9c00ad7a0 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50246/0 (socket says 172.21.15.119:50246) 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 -- 172.21.15.119:0/1839985832 learned_addr learned my addr 172.21.15.119:0/1839985832 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c5b9c700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9c00aff00 0x7fe9c01546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c4b9a700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9c00ad7a0 0x7fe9c0152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 -- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9c00ad7a0 msgr2=0x7fe9c0152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9c00ad7a0 0x7fe9c0152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 -- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9c00aff00 msgr2=0x7fe9c01546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9c00aff00 0x7fe9c01546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.093 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 -- 172.21.15.119:0/1839985832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc009420 con 0x7fe9c00ac020 2023-10-08T00:55:42.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c4b9a700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9c00ad7a0 0x7fe9c0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:42.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.089+0000 7fe9c539b700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00cd4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0065920 tx=0x7fe9bc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.090+0000 7fe9b57fa700 1 -- 172.21.15.119:0/1839985832 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00bb20 con 0x7fe9c00ac020 2023-10-08T00:55:42.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.090+0000 7fe9b57fa700 1 -- 172.21.15.119:0/1839985832 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc011420 con 0x7fe9c00ac020 2023-10-08T00:55:42.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.090+0000 7fe9b57fa700 1 -- 172.21.15.119:0/1839985832 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0117c0 con 0x7fe9c00ac020 2023-10-08T00:55:42.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.090+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c01594e0 con 0x7fe9c00ac020 2023-10-08T00:55:42.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.090+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c00cbb70 con 0x7fe9c00ac020 2023-10-08T00:55:42.094 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.091+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c0088890 con 0x7fe9c00ac020 2023-10-08T00:55:42.095 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.092+0000 7fe9b57fa700 1 -- 172.21.15.119:0/1839985832 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9bc011b70 con 0x7fe9c00ac020 2023-10-08T00:55:42.095 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.093+0000 7fe9b57fa700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe9a4066f30 0x7fe9a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.095 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.093+0000 7fe9b57fa700 1 -- 172.21.15.119:0/1839985832 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe9bc085ec0 con 0x7fe9c00ac020 2023-10-08T00:55:42.095 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.093+0000 7fe9c4b9a700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe9a4066f30 0x7fe9a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.095 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.094+0000 7fe9c4b9a700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe9a4066f30 0x7fe9a40693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0156210 tx=0x7fe9b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.098 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.097+0000 7fe9b57fa700 1 -- 172.21.15.119:0/1839985832 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9bc053460 con 0x7fe9c00ac020 2023-10-08T00:55:42.292 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.291+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9c0064b50 con 0x7fe9c00ac020 2023-10-08T00:55:42.292 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.291+0000 7fe9b57fa700 1 -- 172.21.15.119:0/1839985832 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7fe9bc056a80 con 0x7fe9c00ac020 2023-10-08T00:55:42.293 INFO:teuthology.misc.health.smithi119.stdout: 2023-10-08T00:55:42.293 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":16,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:38.240549+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:42.296 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe9a4066f30 msgr2=0x7fe9a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.296 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe9a4066f30 0x7fe9a40693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0156210 tx=0x7fe9b0009040 comp rx=0 tx=0).stop 2023-10-08T00:55:42.296 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 msgr2=0x7fe9c00cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.296 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00cd4b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0065920 tx=0x7fe9bc017000 comp rx=0 tx=0).stop 2023-10-08T00:55:42.296 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 shutdown_connections 2023-10-08T00:55:42.296 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe9a4066f30 0x7fe9a40693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.296 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9c00aff00 0x7fe9c01546b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.296 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9c00ac020 0x7fe9c00cd4b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.297 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 --2- 172.21.15.119:0/1839985832 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9c00ad7a0 0x7fe9c0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.297 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 >> 172.21.15.119:0/1839985832 conn(0x7fe9c0087ff0 msgr2=0x7fe9c00690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:42.297 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 shutdown_connections 2023-10-08T00:55:42.297 INFO:teuthology.misc.health.smithi119.stderr:2023-10-08T00:55:42.294+0000 7fe9c639d700 1 -- 172.21.15.119:0/1839985832 wait complete. 2023-10-08T00:55:42.310 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-08T00:55:42.311 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-08T00:55:42.311 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:55:42.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:55:42.315 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:55:42.316 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:55:42.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:55:42.320 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:55:42.320 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:55:42.320 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:55:42.325 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:55:42.325 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:55:42.325 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:55:42.330 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:55:42.330 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-08T00:55:42.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:42.454 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T00:55:42.499 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T00:55:42.500 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T00:55:42.500 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T00:55:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.523+0000 7f17d139d700 1 Processor -- start 2023-10-08T00:55:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.523+0000 7f17d139d700 1 -- start start 2023-10-08T00:55:42.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.524+0000 7f17d139d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f17cc159420 0x7f17cc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.524+0000 7f17d139d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc1597f0 0x7f17cc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.524+0000 7f17d139d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc0b11d0 con 0x7f17cc15d230 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.524+0000 7f17d139d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0b14b0 con 0x7f17cc1597f0 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.524+0000 7f17d139d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0b1790 con 0x7f17cc159420 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc1597f0 0x7f17cc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc1597f0 0x7f17cc0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50296/0 (socket says 172.21.15.119:50296) 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cb7fe700 1 -- 172.21.15.119:0/3009605920 learned_addr learned my addr 172.21.15.119:0/3009605920 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cbfff700 1 --2- 172.21.15.119:0/3009605920 >> v2:172.21.15.184:3300/0 conn(0x7f17cc159420 0x7f17cc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cb7fe700 1 -- 172.21.15.119:0/3009605920 >> v2:172.21.15.184:3300/0 conn(0x7f17cc159420 msgr2=0x7f17cc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cb7fe700 1 --2- 172.21.15.119:0/3009605920 >> v2:172.21.15.184:3300/0 conn(0x7f17cc159420 0x7f17cc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cb7fe700 1 -- 172.21.15.119:0/3009605920 >> v1:172.21.15.184:6789/0 conn(0x7f17cc15d230 legacy=0x7f17cc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cb7fe700 1 -- 172.21.15.119:0/3009605920 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0b1a70 con 0x7f17cc1597f0 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.525+0000 7f17cb7fe700 1 --2- 172.21.15.119:0/3009605920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc1597f0 0x7f17cc0b2270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0b2f20 tx=0x7f17c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b9da25765be1a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17caffd700 1 -- 172.21.15.119:0/3009605920 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0009e70 con 0x7f17cc1597f0 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17caffd700 1 -- 172.21.15.119:0/3009605920 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c000b860 con 0x7f17cc1597f0 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17caffd700 1 -- 172.21.15.119:0/3009605920 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000bc00 con 0x7f17cc1597f0 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17d139d700 1 -- 172.21.15.119:0/3009605920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc1597f0 msgr2=0x7f17cc0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17d139d700 1 --2- 172.21.15.119:0/3009605920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc1597f0 0x7f17cc0b2270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0b2f20 tx=0x7f17c0009250 comp rx=0 tx=0).stop 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17d139d700 1 -- 172.21.15.119:0/3009605920 shutdown_connections 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17d139d700 1 --2- 172.21.15.119:0/3009605920 >> v2:172.21.15.184:3300/0 conn(0x7f17cc159420 0x7f17cc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17d139d700 1 --2- 172.21.15.119:0/3009605920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc1597f0 0x7f17cc0b2270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17d139d700 1 -- 172.21.15.119:0/3009605920 >> 172.21.15.119:0/3009605920 conn(0x7f17cc0d0660 msgr2=0x7f17cc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:42.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17d139d700 1 -- 172.21.15.119:0/3009605920 shutdown_connections 2023-10-08T00:55:42.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.526+0000 7f17d139d700 1 -- 172.21.15.119:0/3009605920 wait complete. 2023-10-08T00:55:42.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.528+0000 7f17d139d700 1 Processor -- start 2023-10-08T00:55:42.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.528+0000 7f17d139d700 1 -- start start 2023-10-08T00:55:42.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17d139d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc159420 0x7f17cc14e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17d139d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc1597f0 0x7f17cc068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17d139d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc15d230 0x7f17cc15c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17d139d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0b4a60 con 0x7f17cc159420 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17d139d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc14e9d0 con 0x7f17cc1597f0 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17d139d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc15ce00 con 0x7f17cc15d230 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc159420 0x7f17cc14e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc159420 0x7f17cc14e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50308/0 (socket says 172.21.15.119:50308) 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cbfff700 1 -- 172.21.15.119:0/3280495086 learned_addr learned my addr 172.21.15.119:0/3280495086 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cb7fe700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc1597f0 0x7f17cc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17d0b9c700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc15d230 0x7f17cc15c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cbfff700 1 -- 172.21.15.119:0/3280495086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc15d230 msgr2=0x7f17cc15c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cbfff700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc15d230 0x7f17cc15c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cbfff700 1 -- 172.21.15.119:0/3280495086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc1597f0 msgr2=0x7f17cc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cbfff700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc1597f0 0x7f17cc068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.529+0000 7f17cbfff700 1 -- 172.21.15.119:0/3280495086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c00094f0 con 0x7f17cc159420 2023-10-08T00:55:42.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.530+0000 7f17cbfff700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc159420 0x7f17cc14e490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00ec80 tx=0x7f17bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.530+0000 7f17b3fff700 1 -- 172.21.15.119:0/3280495086 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00c850 con 0x7f17cc159420 2023-10-08T00:55:42.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.530+0000 7f17b3fff700 1 -- 172.21.15.119:0/3280495086 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17bc00c9b0 con 0x7f17cc159420 2023-10-08T00:55:42.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.530+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc0b4cb0 con 0x7f17cc159420 2023-10-08T00:55:42.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.530+0000 7f17b3fff700 1 -- 172.21.15.119:0/3280495086 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17bc00cd50 con 0x7f17cc159420 2023-10-08T00:55:42.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.530+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc0b5270 con 0x7f17cc159420 2023-10-08T00:55:42.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.531+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc0b4360 con 0x7f17cc159420 2023-10-08T00:55:42.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.532+0000 7f17b3fff700 1 -- 172.21.15.119:0/3280495086 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17bc013680 con 0x7f17cc159420 2023-10-08T00:55:42.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.532+0000 7f17b3fff700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b8066f30 0x7f17b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.533+0000 7f17b3fff700 1 -- 172.21.15.119:0/3280495086 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f17bc084540 con 0x7f17cc159420 2023-10-08T00:55:42.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.533+0000 7f17cb7fe700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b8066f30 0x7f17b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.533+0000 7f17cb7fe700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b8066f30 0x7f17b80693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0b2f20 tx=0x7f17c0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.537+0000 7f17b3fff700 1 -- 172.21.15.119:0/3280495086 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17bc051a60 con 0x7f17cc159420 2023-10-08T00:55:42.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.728+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17cc04c3c0 con 0x7f17cc159420 2023-10-08T00:55:42.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.729+0000 7f17b3fff700 1 -- 172.21.15.119:0/3280495086 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f17bc055080 con 0x7f17cc159420 2023-10-08T00:55:42.730 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:42.730 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:38.240549+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:42.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b8066f30 msgr2=0x7f17b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b8066f30 0x7f17b80693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0b2f20 tx=0x7f17c0017000 comp rx=0 tx=0).stop 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc159420 msgr2=0x7f17cc14e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc159420 0x7f17cc14e490 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f17bc00ec80 tx=0x7f17bc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 shutdown_connections 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc15d230 0x7f17cc15c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b8066f30 0x7f17b80693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc159420 0x7f17cc14e490 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 --2- 172.21.15.119:0/3280495086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc1597f0 0x7f17cc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 >> 172.21.15.119:0/3280495086 conn(0x7f17cc0d0660 msgr2=0x7f17cc156090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 shutdown_connections 2023-10-08T00:55:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.731+0000 7f17d139d700 1 -- 172.21.15.119:0/3280495086 wait complete. 2023-10-08T00:55:42.750 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:55:42.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:55:42.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.932+0000 7f36c31ff700 1 Processor -- start 2023-10-08T00:55:42.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.932+0000 7f36c31ff700 1 -- start start 2023-10-08T00:55:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.933+0000 7f36c31ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36bc15ac80 0x7f36bc069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.933+0000 7f36c31ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15b050 0x7f36bc069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.933+0000 7f36c31ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36bc0af2f0 con 0x7f36bc0afa60 2023-10-08T00:55:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.933+0000 7f36c31ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36bc0af5d0 con 0x7f36bc15ac80 2023-10-08T00:55:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.933+0000 7f36c31ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc0c8660 con 0x7f36bc15b050 2023-10-08T00:55:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36c0f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36bc15ac80 0x7f36bc069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36c0f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36bc15ac80 0x7f36bc069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50312/0 (socket says 172.21.15.119:50312) 2023-10-08T00:55:42.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36bbfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15b050 0x7f36bc069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36c0f9b700 1 -- 172.21.15.119:0/3877429237 learned_addr learned my addr 172.21.15.119:0/3877429237 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:42.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36c0f9b700 1 -- 172.21.15.119:0/3877429237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15b050 msgr2=0x7f36bc069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:42.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36c0f9b700 1 --2- 172.21.15.119:0/3877429237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15b050 0x7f36bc069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36c0f9b700 1 -- 172.21.15.119:0/3877429237 >> v1:172.21.15.184:6789/0 conn(0x7f36bc0afa60 legacy=0x7f36bc0aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36c0f9b700 1 -- 172.21.15.119:0/3877429237 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc0c8940 con 0x7f36bc15ac80 2023-10-08T00:55:42.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36bb7fe700 1 -- 172.21.15.119:0/3877429237 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643846573 0 0) 0x7f36bc0af2f0 con 0x7f36bc0afa60 2023-10-08T00:55:42.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36bbfff700 1 --2- 172.21.15.119:0/3877429237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15b050 0x7f36bc069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:42.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.934+0000 7f36c0f9b700 1 --2- 172.21.15.119:0/3877429237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15ac80 0x7f36bc069770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36b00099b0 tx=0x7f36b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489ef4cc5538780e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36bb7fe700 1 -- 172.21.15.119:0/3877429237 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0009ea0 con 0x7f36bc15ac80 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36bb7fe700 1 -- 172.21.15.119:0/3877429237 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b000b860 con 0x7f36bc15ac80 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36bb7fe700 1 -- 172.21.15.119:0/3877429237 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b000bc00 con 0x7f36bc15ac80 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36c31ff700 1 -- 172.21.15.119:0/3877429237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15ac80 msgr2=0x7f36bc069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3877429237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15ac80 0x7f36bc069770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f36b00099b0 tx=0x7f36b0009250 comp rx=0 tx=0).stop 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36c31ff700 1 -- 172.21.15.119:0/3877429237 shutdown_connections 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3877429237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15ac80 0x7f36bc069770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3877429237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15b050 0x7f36bc069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36c31ff700 1 -- 172.21.15.119:0/3877429237 >> 172.21.15.119:0/3877429237 conn(0x7f36bc068e10 msgr2=0x7f36bc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36c31ff700 1 -- 172.21.15.119:0/3877429237 shutdown_connections 2023-10-08T00:55:42.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.935+0000 7f36c31ff700 1 -- 172.21.15.119:0/3877429237 wait complete. 2023-10-08T00:55:42.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.937+0000 7f36c31ff700 1 Processor -- start 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.937+0000 7f36c31ff700 1 -- start start 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c31ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36bc0afa60 0x7f36bc0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c31ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15ac80 0x7f36bc156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c31ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15b050 0x7f36bc15f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c31ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15cec0 con 0x7f36bc15b050 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c31ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15d1a0 con 0x7f36bc0afa60 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c31ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc15fcc0 con 0x7f36bc15ac80 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c179c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15b050 0x7f36bc15f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c0f9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36bc0afa60 0x7f36bc0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c179c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15b050 0x7f36bc15f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50328/0 (socket says 172.21.15.119:50328) 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c179c700 1 -- 172.21.15.119:0/3913019506 learned_addr learned my addr 172.21.15.119:0/3913019506 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36bbfff700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15ac80 0x7f36bc156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c179c700 1 -- 172.21.15.119:0/3913019506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15ac80 msgr2=0x7f36bc156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c179c700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15ac80 0x7f36bc156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c179c700 1 -- 172.21.15.119:0/3913019506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36bc0afa60 msgr2=0x7f36bc0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c179c700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36bc0afa60 0x7f36bc0ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.938+0000 7f36c179c700 1 -- 172.21.15.119:0/3913019506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a8009420 con 0x7f36bc15b050 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.939+0000 7f36c179c700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15b050 0x7f36bc15f420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36b4001560 tx=0x7f36b4001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.939+0000 7f36b8ff9700 1 -- 172.21.15.119:0/3913019506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b40040d0 con 0x7f36bc15b050 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.939+0000 7f36b8ff9700 1 -- 172.21.15.119:0/3913019506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b400a7e0 con 0x7f36bc15b050 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.939+0000 7f36b8ff9700 1 -- 172.21.15.119:0/3913019506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b400aba0 con 0x7f36bc15b050 2023-10-08T00:55:42.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.939+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b00094f0 con 0x7f36bc15b050 2023-10-08T00:55:42.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.939+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc160280 con 0x7f36bc15b050 2023-10-08T00:55:42.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.940+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc15d310 con 0x7f36bc15b050 2023-10-08T00:55:42.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.942+0000 7f36b8ff9700 1 -- 172.21.15.119:0/3913019506 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f36b4025430 con 0x7f36bc15b050 2023-10-08T00:55:42.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.943+0000 7f36b8ff9700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f36a0066f30 0x7f36a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:42.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.943+0000 7f36b8ff9700 1 -- 172.21.15.119:0/3913019506 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f36b4084a40 con 0x7f36bc15b050 2023-10-08T00:55:42.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.943+0000 7f36c0f9b700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f36a0066f30 0x7f36a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:42.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.944+0000 7f36c0f9b700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f36a0066f30 0x7f36a00693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f36b00099b0 tx=0x7f36b0017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:42.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:42.946+0000 7f36b8ff9700 1 -- 172.21.15.119:0/3913019506 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36b4052090 con 0x7f36bc15b050 2023-10-08T00:55:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.138+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36bc0c8c20 con 0x7f36bc15b050 2023-10-08T00:55:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.139+0000 7f36b8ff9700 1 -- 172.21.15.119:0/3913019506 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36b40556b0 con 0x7f36bc15b050 2023-10-08T00:55:43.140 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:55:43.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f36a0066f30 msgr2=0x7f36a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f36a0066f30 0x7f36a00693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f36b00099b0 tx=0x7f36b0017000 comp rx=0 tx=0).stop 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15b050 msgr2=0x7f36bc15f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15b050 0x7f36bc15f420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f36b4001560 tx=0x7f36b4001770 comp rx=0 tx=0).stop 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 shutdown_connections 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f36a0066f30 0x7f36a00693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36bc15b050 0x7f36bc15f420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36bc0afa60 0x7f36bc0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 --2- 172.21.15.119:0/3913019506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36bc15ac80 0x7f36bc156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 >> 172.21.15.119:0/3913019506 conn(0x7f36bc068e10 msgr2=0x7f36bc06daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 shutdown_connections 2023-10-08T00:55:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.141+0000 7f36c31ff700 1 -- 172.21.15.119:0/3913019506 wait complete. 2023-10-08T00:55:43.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:55:43.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:55:43.209 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:55:43.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.343+0000 7ff8639ff700 1 Processor -- start 2023-10-08T00:55:43.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.343+0000 7ff8639ff700 1 -- start start 2023-10-08T00:55:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.344+0000 7ff8639ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff85c0ccc40 0x7ff85c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.344+0000 7ff8639ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.344+0000 7ff8639ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85c0ad930 con 0x7ff85c156150 2023-10-08T00:55:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.344+0000 7ff8639ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85c0acd20 con 0x7ff85c0ccc40 2023-10-08T00:55:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.344+0000 7ff8639ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0ad000 con 0x7ff85c0cd010 2023-10-08T00:55:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff861f9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff86179b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff85c0ccc40 0x7ff85c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff860f9a700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff85c156150 0x7ff85c0ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:57100/0 (socket says 172.21.15.119:57100) 2023-10-08T00:55:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff860f9a700 1 -- 172.21.15.119:0/3126605895 learned_addr learned my addr 172.21.15.119:0/3126605895 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff853fff700 1 -- 172.21.15.119:0/3126605895 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888419211 0 0) 0x7ff85c0ad930 con 0x7ff85c156150 2023-10-08T00:55:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff853fff700 1 -- 172.21.15.119:0/3126605895 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff848003570 con 0x7ff85c156150 2023-10-08T00:55:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff861f9c700 1 -- 172.21.15.119:0/3126605895 >> v2:172.21.15.184:3300/0 conn(0x7ff85c0ccc40 msgr2=0x7ff85c155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff861f9c700 1 --2- 172.21.15.119:0/3126605895 >> v2:172.21.15.184:3300/0 conn(0x7ff85c0ccc40 0x7ff85c155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff861f9c700 1 -- 172.21.15.119:0/3126605895 >> v1:172.21.15.184:6789/0 conn(0x7ff85c156150 legacy=0x7ff85c0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff861f9c700 1 -- 172.21.15.119:0/3126605895 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c0ad2e0 con 0x7ff85c0cd010 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.345+0000 7ff86179b700 1 --2- 172.21.15.119:0/3126605895 >> v2:172.21.15.184:3300/0 conn(0x7ff85c0ccc40 0x7ff85c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.346+0000 7ff861f9c700 1 --2- 172.21.15.119:0/3126605895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0b0300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff854009e60 tx=0x7ff854009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa2440d25baa4711 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.346+0000 7ff853fff700 1 -- 172.21.15.119:0/3126605895 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85400e070 con 0x7ff85c0cd010 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.346+0000 7ff853fff700 1 -- 172.21.15.119:0/3126605895 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85400bab0 con 0x7ff85c0cd010 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.346+0000 7ff853fff700 1 -- 172.21.15.119:0/3126605895 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85400be70 con 0x7ff85c0cd010 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.346+0000 7ff8639ff700 1 -- 172.21.15.119:0/3126605895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 msgr2=0x7ff85c0b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.346+0000 7ff8639ff700 1 --2- 172.21.15.119:0/3126605895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0b0300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff854009e60 tx=0x7ff854009780 comp rx=0 tx=0).stop 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.347+0000 7ff8639ff700 1 -- 172.21.15.119:0/3126605895 shutdown_connections 2023-10-08T00:55:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.347+0000 7ff8639ff700 1 --2- 172.21.15.119:0/3126605895 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0b0300 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.347+0000 7ff8639ff700 1 --2- 172.21.15.119:0/3126605895 >> v2:172.21.15.184:3300/0 conn(0x7ff85c0ccc40 0x7ff85c155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.347+0000 7ff8639ff700 1 -- 172.21.15.119:0/3126605895 >> 172.21.15.119:0/3126605895 conn(0x7ff85c06a320 msgr2=0x7ff85c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.347+0000 7ff8639ff700 1 -- 172.21.15.119:0/3126605895 shutdown_connections 2023-10-08T00:55:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.347+0000 7ff8639ff700 1 -- 172.21.15.119:0/3126605895 wait complete. 2023-10-08T00:55:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.348+0000 7ff8639ff700 1 Processor -- start 2023-10-08T00:55:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.348+0000 7ff8639ff700 1 -- start start 2023-10-08T00:55:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff8639ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c0ccc40 0x7ff85c0cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff8639ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0ca530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff8639ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff85c156150 0x7ff85c151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff8639ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c0cca20 con 0x7ff85c0ccc40 2023-10-08T00:55:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff8639ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c169da0 con 0x7ff85c156150 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff8639ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c16a080 con 0x7ff85c0cd010 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff86179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c0ccc40 0x7ff85c0cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff86179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c0ccc40 0x7ff85c0cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50340/0 (socket says 172.21.15.119:50340) 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.349+0000 7ff86179b700 1 -- 172.21.15.119:0/766021779 learned_addr learned my addr 172.21.15.119:0/766021779 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff860f9a700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0ca530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff861f9c700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff85c156150 0x7ff85c151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff86179b700 1 -- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 msgr2=0x7ff85c0ca530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff86179b700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0ca530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff86179b700 1 -- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff85c156150 msgr2=0x7ff85c151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff86179b700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff85c156150 0x7ff85c151c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff86179b700 1 -- 172.21.15.119:0/766021779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff858009420 con 0x7ff85c0ccc40 2023-10-08T00:55:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff860f9a700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff86179b700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c0ccc40 0x7ff85c0cc4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff85800ec80 tx=0x7ff8580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff8517fa700 1 -- 172.21.15.119:0/766021779 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85800c9d0 con 0x7ff85c0ccc40 2023-10-08T00:55:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff8517fa700 1 -- 172.21.15.119:0/766021779 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff85800cb30 con 0x7ff85c0ccc40 2023-10-08T00:55:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff8517fa700 1 -- 172.21.15.119:0/766021779 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85800ceb0 con 0x7ff85c0ccc40 2023-10-08T00:55:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff854009a20 con 0x7ff85c0ccc40 2023-10-08T00:55:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.350+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c16a640 con 0x7ff85c0ccc40 2023-10-08T00:55:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.352+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85c0658d0 con 0x7ff85c0ccc40 2023-10-08T00:55:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.352+0000 7ff8517fa700 1 -- 172.21.15.119:0/766021779 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff858013680 con 0x7ff85c0ccc40 2023-10-08T00:55:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.354+0000 7ff8517fa700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff840066f30 0x7ff8400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.354+0000 7ff8517fa700 1 -- 172.21.15.119:0/766021779 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7ff858084620 con 0x7ff85c0ccc40 2023-10-08T00:55:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.354+0000 7ff860f9a700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff840066f30 0x7ff8400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.354+0000 7ff860f9a700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff840066f30 0x7ff8400693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0c8610 tx=0x7ff84c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.358+0000 7ff8517fa700 1 -- 172.21.15.119:0/766021779 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff858051be0 con 0x7ff85c0ccc40 2023-10-08T00:55:43.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.594+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff85c16cf70 con 0x7ff85c0ccc40 2023-10-08T00:55:43.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.594+0000 7ff8517fa700 1 -- 172.21.15.119:0/766021779 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3536 (secure 0 0 0) 0x7ff858017de0 con 0x7ff85c0ccc40 2023-10-08T00:55:43.595 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:43.596 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-10-08T00:55:43.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff840066f30 msgr2=0x7ff8400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff840066f30 0x7ff8400693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff85c0c8610 tx=0x7ff84c009040 comp rx=0 tx=0).stop 2023-10-08T00:55:43.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c0ccc40 msgr2=0x7ff85c0cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c0ccc40 0x7ff85c0cc4e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff85800ec80 tx=0x7ff8580091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 shutdown_connections 2023-10-08T00:55:43.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff840066f30 0x7ff8400693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff85c156150 0x7ff85c151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff85c0ccc40 0x7ff85c0cc4e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 --2- 172.21.15.119:0/766021779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff85c0cd010 0x7ff85c0ca530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 >> 172.21.15.119:0/766021779 conn(0x7ff85c06a320 msgr2=0x7ff85c0af5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:43.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 shutdown_connections 2023-10-08T00:55:43.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.596+0000 7ff8639ff700 1 -- 172.21.15.119:0/766021779 wait complete. 2023-10-08T00:55:43.599 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 3 2023-10-08T00:55:43.616 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:43.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.795+0000 7f22b85ff700 1 Processor -- start 2023-10-08T00:55:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.795+0000 7f22b85ff700 1 -- start start 2023-10-08T00:55:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.796+0000 7f22b85ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f22b00ccc40 0x7f22b0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.796+0000 7f22b85ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00cd010 0x7f22b00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.796+0000 7f22b85ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b00ad930 con 0x7f22b0156150 2023-10-08T00:55:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b85ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b00acd20 con 0x7f22b00ccc40 2023-10-08T00:55:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b85ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00ad000 con 0x7f22b00cd010 2023-10-08T00:55:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b639b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f22b00ccc40 0x7f22b0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b6b9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00cd010 0x7f22b00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b639b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f22b00ccc40 0x7f22b0155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54894/0 (socket says 172.21.15.119:54894) 2023-10-08T00:55:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b639b700 1 -- 172.21.15.119:0/1293419581 learned_addr learned my addr 172.21.15.119:0/1293419581 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b639b700 1 -- 172.21.15.119:0/1293419581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00cd010 msgr2=0x7f22b00b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b639b700 1 --2- 172.21.15.119:0/1293419581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00cd010 0x7f22b00b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b639b700 1 -- 172.21.15.119:0/1293419581 >> v1:172.21.15.184:6789/0 conn(0x7f22b0156150 legacy=0x7f22b00ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b639b700 1 -- 172.21.15.119:0/1293419581 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b00ad2e0 con 0x7f22b00ccc40 2023-10-08T00:55:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.797+0000 7f22b6b9c700 1 --2- 172.21.15.119:0/1293419581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00cd010 0x7f22b00b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.798+0000 7f22b639b700 1 --2- 172.21.15.119:0/1293419581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b00ccc40 0x7f22b0155c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0099b0 tx=0x7f22ac009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4c01631cb120a0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.798+0000 7f22b5399700 1 -- 172.21.15.119:0/1293419581 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac009ea0 con 0x7f22b00ccc40 2023-10-08T00:55:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.798+0000 7f22b5399700 1 -- 172.21.15.119:0/1293419581 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22ac00b860 con 0x7f22b00ccc40 2023-10-08T00:55:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.798+0000 7f22b5399700 1 -- 172.21.15.119:0/1293419581 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00bc00 con 0x7f22b00ccc40 2023-10-08T00:55:43.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.799+0000 7f22b85ff700 1 -- 172.21.15.119:0/1293419581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b00ccc40 msgr2=0x7f22b0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.799+0000 7f22b85ff700 1 --2- 172.21.15.119:0/1293419581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b00ccc40 0x7f22b0155c10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0099b0 tx=0x7f22ac009250 comp rx=0 tx=0).stop 2023-10-08T00:55:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.799+0000 7f22b85ff700 1 -- 172.21.15.119:0/1293419581 shutdown_connections 2023-10-08T00:55:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.799+0000 7f22b85ff700 1 --2- 172.21.15.119:0/1293419581 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00cd010 0x7f22b00b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.799+0000 7f22b85ff700 1 --2- 172.21.15.119:0/1293419581 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b00ccc40 0x7f22b0155c10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.799+0000 7f22b85ff700 1 -- 172.21.15.119:0/1293419581 >> 172.21.15.119:0/1293419581 conn(0x7f22b006a320 msgr2=0x7f22b00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.799+0000 7f22b85ff700 1 -- 172.21.15.119:0/1293419581 shutdown_connections 2023-10-08T00:55:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.799+0000 7f22b85ff700 1 -- 172.21.15.119:0/1293419581 wait complete. 2023-10-08T00:55:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.800+0000 7f22b85ff700 1 Processor -- start 2023-10-08T00:55:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.800+0000 7f22b85ff700 1 -- start start 2023-10-08T00:55:43.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.801+0000 7f22b85ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00ccc40 0x7f22b00aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.801+0000 7f22b85ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b00cd010 0x7f22b00a7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.801+0000 7f22b85ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b0156150 0x7f22b00a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.801+0000 7f22b85ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00c1450 con 0x7f22b00cd010 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.801+0000 7f22b85ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00c1710 con 0x7f22b0156150 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.801+0000 7f22b85ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b00c19f0 con 0x7f22b00ccc40 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.801+0000 7f22b5b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b00cd010 0x7f22b00a7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b5b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b00cd010 0x7f22b00a7610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50356/0 (socket says 172.21.15.119:50356) 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b5b9a700 1 -- 172.21.15.119:0/3557081977 learned_addr learned my addr 172.21.15.119:0/3557081977 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b6b9c700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b0156150 0x7f22b00a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b639b700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00ccc40 0x7f22b00aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b5b9a700 1 -- 172.21.15.119:0/3557081977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00ccc40 msgr2=0x7f22b00aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b5b9a700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00ccc40 0x7f22b00aa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b5b9a700 1 -- 172.21.15.119:0/3557081977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b0156150 msgr2=0x7f22b00a7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b5b9a700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b0156150 0x7f22b00a7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b5b9a700 1 -- 172.21.15.119:0/3557081977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a8009a20 con 0x7f22b00cd010 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b5b9a700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b00cd010 0x7f22b00a7610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c590 tx=0x7f22a0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22a67fc700 1 -- 172.21.15.119:0/3557081977 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011590 con 0x7f22b00cd010 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22a67fc700 1 -- 172.21.15.119:0/3557081977 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a0011710 con 0x7f22b00cd010 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac0094f0 con 0x7f22b00cd010 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.802+0000 7f22a67fc700 1 -- 172.21.15.119:0/3557081977 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a0011ad0 con 0x7f22b00cd010 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.803+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b00c1fb0 con 0x7f22b00cd010 2023-10-08T00:55:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.803+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b004c3c0 con 0x7f22b00cd010 2023-10-08T00:55:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.804+0000 7f22a67fc700 1 -- 172.21.15.119:0/3557081977 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f22a001b3b0 con 0x7f22b00cd010 2023-10-08T00:55:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.805+0000 7f22a67fc700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2294066f30 0x7f22940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.805+0000 7f22a67fc700 1 -- 172.21.15.119:0/3557081977 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f22a00836d0 con 0x7f22b00cd010 2023-10-08T00:55:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.806+0000 7f22b639b700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2294066f30 0x7f22940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.806+0000 7f22b639b700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2294066f30 0x7f22940693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0099b0 tx=0x7f22ac017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:43.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:43.809+0000 7f22a67fc700 1 -- 172.21.15.119:0/3557081977 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f22a0050bf0 con 0x7f22b00cd010 2023-10-08T00:55:44.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.002+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22b00ad5c0 con 0x7f22b00cd010 2023-10-08T00:55:44.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.003+0000 7f22a67fc700 1 -- 172.21.15.119:0/3557081977 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12175 (secure 0 0 0) 0x7f22a0017020 con 0x7f22b00cd010 2023-10-08T00:55:44.004 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:44.004 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:38.240549+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2294066f30 msgr2=0x7f22940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2294066f30 0x7f22940693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f22ac0099b0 tx=0x7f22ac017000 comp rx=0 tx=0).stop 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b00cd010 msgr2=0x7f22b00a7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b00cd010 0x7f22b00a7610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f22a000c590 tx=0x7f22a0007f50 comp rx=0 tx=0).stop 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 shutdown_connections 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22b00ccc40 0x7f22b00aa650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2294066f30 0x7f22940693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22b0156150 0x7f22b00a7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 --2- 172.21.15.119:0/3557081977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22b00cd010 0x7f22b00a7610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 >> 172.21.15.119:0/3557081977 conn(0x7f22b006a320 msgr2=0x7f22b00ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:44.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.005+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 shutdown_connections 2023-10-08T00:55:44.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.006+0000 7f22b85ff700 1 -- 172.21.15.119:0/3557081977 wait complete. 2023-10-08T00:55:44.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:55:44.025 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:55:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.207+0000 7fe8fd5ff700 1 Processor -- start 2023-10-08T00:55:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.207+0000 7fe8fd5ff700 1 -- start start 2023-10-08T00:55:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.208+0000 7fe8fd5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe8f80c83f0 0x7fe8f80c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.209+0000 7fe8fd5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f815b0d0 con 0x7fe8f80ccee0 2023-10-08T00:55:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.209+0000 7fe8fd5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f80c7a60 con 0x7fe8f81568e0 2023-10-08T00:55:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.209+0000 7fe8fd5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f815a4c0 con 0x7fe8f80c83f0 2023-10-08T00:55:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.209+0000 7fe8f67fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe8f80ccee0 0x7fe8f81561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42154/0 (socket says 172.21.15.119:42154) 2023-10-08T00:55:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.209+0000 7fe8f67fc700 1 -- 172.21.15.119:0/1913186833 learned_addr learned my addr 172.21.15.119:0/1913186833 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.209+0000 7fe8f6ffd700 1 --2- 172.21.15.119:0/1913186833 >> v2:172.21.15.184:3300/0 conn(0x7fe8f80c83f0 0x7fe8f80c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.209+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256611282 0 0) 0x7fe8f815b0d0 con 0x7fe8f80ccee0 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.209+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e0003570 con 0x7fe8f80ccee0 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4138089309 0 0) 0x7fe8e0003570 con 0x7fe8f80ccee0 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 >> v2:172.21.15.184:3300/0 conn(0x7fe8f80c83f0 msgr2=0x7fe8f80c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 --2- 172.21.15.119:0/1913186833 >> v2:172.21.15.184:3300/0 conn(0x7fe8f80c83f0 0x7fe8f80c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 >> v1:172.21.15.184:6789/0 conn(0x7fe8f81568e0 legacy=0x7fe8f80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f815a7a0 con 0x7fe8f80ccee0 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fe8ec001470 con 0x7fe8f80ccee0 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 >> v1:172.21.15.119:6789/0 conn(0x7fe8f80ccee0 legacy=0x7fe8f81561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e0003a60 0x7fe8e0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0004810 0x7fe8e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e00051c0 0x7fe8e000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f815a4c0 con 0x7fe8e0004810 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e000c050 con 0x7fe8e0003a60 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e000c210 con 0x7fe8e00051c0 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f67fc700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0004810 0x7fe8e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f6ffd700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e0003a60 0x7fe8e0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f77fe700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e00051c0 0x7fe8e000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f67fc700 1 -- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e00051c0 msgr2=0x7fe8e000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f67fc700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e00051c0 0x7fe8e000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f67fc700 1 -- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e0003a60 msgr2=0x7fe8e0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f67fc700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8e0003a60 0x7fe8e0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f67fc700 1 -- 172.21.15.119:0/1913186833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0003570 con 0x7fe8e0004810 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f77fe700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e00051c0 0x7fe8e000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.210+0000 7fe8f77fe700 1 -- 172.21.15.119:0/1913186833 reap_dead start 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8f67fc700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0004810 0x7fe8e0004c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00c3c0 tx=0x7fe8ec001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83966a3701d72b6b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec00cc90 con 0x7fe8e0004810 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec00cdf0 con 0x7fe8e0004810 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8f5ffb700 1 -- 172.21.15.119:0/1913186833 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec005b70 con 0x7fe8e0004810 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/1913186833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0004810 msgr2=0x7fe8e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8fd5ff700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0004810 0x7fe8e0004c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00c3c0 tx=0x7fe8ec001e70 comp rx=0 tx=0).stop 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/1913186833 shutdown_connections 2023-10-08T00:55:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8fd5ff700 1 --2- 172.21.15.119:0/1913186833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8e0004810 0x7fe8e0004c80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/1913186833 >> 172.21.15.119:0/1913186833 conn(0x7fe8f8068e10 msgr2=0x7fe8f80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:44.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/1913186833 shutdown_connections 2023-10-08T00:55:44.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.211+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/1913186833 wait complete. 2023-10-08T00:55:44.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.213+0000 7fe8fd5ff700 1 Processor -- start 2023-10-08T00:55:44.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.213+0000 7fe8fd5ff700 1 -- start start 2023-10-08T00:55:44.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8fd5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e0004810 0x7fe8f8148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8fd5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f8145e90 0x7fe8f80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8fd5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f80c4260 0x7fe8f80c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8fd5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8157850 con 0x7fe8f8145e90 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8fd5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f81491b0 con 0x7fe8f80c4260 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8fd5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8156eb0 con 0x7fe8e0004810 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f8145e90 0x7fe8f80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f8145e90 0x7fe8f80c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50370/0 (socket says 172.21.15.119:50370) 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 -- 172.21.15.119:0/3318701990 learned_addr learned my addr 172.21.15.119:0/3318701990 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f77fe700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f80c4260 0x7fe8f80c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 -- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e0004810 msgr2=0x7fe8f8148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f6ffd700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e0004810 0x7fe8f8148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e0004810 0x7fe8f8148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 -- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f80c4260 msgr2=0x7fe8f80c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f80c4260 0x7fe8f80c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 -- 172.21.15.119:0/3318701990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8157190 con 0x7fe8f8145e90 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f77fe700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f80c4260 0x7fe8f80c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f6ffd700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e0004810 0x7fe8f8148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.214+0000 7fe8f67fc700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f8145e90 0x7fe8f80c50a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec000e40 tx=0x7fe8ec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.215+0000 7fe8deffd700 1 -- 172.21.15.119:0/3318701990 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec022070 con 0x7fe8f8145e90 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.215+0000 7fe8deffd700 1 -- 172.21.15.119:0/3318701990 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ec001790 con 0x7fe8f8145e90 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.215+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f8157470 con 0x7fe8f8145e90 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.215+0000 7fe8deffd700 1 -- 172.21.15.119:0/3318701990 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec007320 con 0x7fe8f8145e90 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.215+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f80c8d00 con 0x7fe8f8145e90 2023-10-08T00:55:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.216+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f8157990 con 0x7fe8f8145e90 2023-10-08T00:55:44.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.217+0000 7fe8deffd700 1 -- 172.21.15.119:0/3318701990 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8ec007690 con 0x7fe8f8145e90 2023-10-08T00:55:44.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.217+0000 7fe8deffd700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8e4066f30 0x7fe8e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.218+0000 7fe8deffd700 1 -- 172.21.15.119:0/3318701990 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fe8ec0852d0 con 0x7fe8f8145e90 2023-10-08T00:55:44.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.218+0000 7fe8f6ffd700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8e4066f30 0x7fe8e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.218+0000 7fe8f6ffd700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8e4066f30 0x7fe8e40693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80093f0 tx=0x7fe8e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:44.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.221+0000 7fe8deffd700 1 -- 172.21.15.119:0/3318701990 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8f8157990 con 0x7fe8f8145e90 2023-10-08T00:55:44.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.410+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8f8157990 con 0x7fe8f8145e90 2023-10-08T00:55:44.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.411+0000 7fe8deffd700 1 -- 172.21.15.119:0/3318701990 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8ec017020 con 0x7fe8f8145e90 2023-10-08T00:55:44.412 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:55:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8e4066f30 msgr2=0x7fe8e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8e4066f30 0x7fe8e40693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80093f0 tx=0x7fe8e8008040 comp rx=0 tx=0).stop 2023-10-08T00:55:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f8145e90 msgr2=0x7fe8f80c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f8145e90 0x7fe8f80c50a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec000e40 tx=0x7fe8ec015000 comp rx=0 tx=0).stop 2023-10-08T00:55:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 shutdown_connections 2023-10-08T00:55:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8e0004810 0x7fe8f8148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8e4066f30 0x7fe8e40693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f80c4260 0x7fe8f80c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 --2- 172.21.15.119:0/3318701990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f8145e90 0x7fe8f80c50a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.413+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 >> 172.21.15.119:0/3318701990 conn(0x7fe8f8068e10 msgr2=0x7fe8f80c2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.414+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 shutdown_connections 2023-10-08T00:55:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.414+0000 7fe8fd5ff700 1 -- 172.21.15.119:0/3318701990 wait complete. 2023-10-08T00:55:44.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:55:44.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.612+0000 7f65fffff700 1 Processor -- start 2023-10-08T00:55:44.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.612+0000 7f65fffff700 1 -- start start 2023-10-08T00:55:44.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65fffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6600159420 0x7f66000b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65fffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 0x7f66000b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65fffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66000b11d0 con 0x7f660015d230 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65fffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66000b14b0 con 0x7f6600159420 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65fffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000b1790 con 0x7f66001597f0 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65feffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6600159420 0x7f66000b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65ff7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f660015d230 0x7f66000b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42162/0 (socket says 172.21.15.119:42162) 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65ff7fe700 1 -- 172.21.15.119:0/690383948 learned_addr learned my addr 172.21.15.119:0/690383948 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.613+0000 7f65fe7fc700 1 --2- 172.21.15.119:0/690383948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 0x7f66000b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65feffd700 1 -- 172.21.15.119:0/690383948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 msgr2=0x7f66000b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65feffd700 1 --2- 172.21.15.119:0/690383948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 0x7f66000b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65feffd700 1 -- 172.21.15.119:0/690383948 >> v1:172.21.15.119:6789/0 conn(0x7f660015d230 legacy=0x7f66000b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65feffd700 1 -- 172.21.15.119:0/690383948 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000b1a70 con 0x7f6600159420 2023-10-08T00:55:44.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65fdffb700 1 -- 172.21.15.119:0/690383948 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 178681378 0 0) 0x7f66000b11d0 con 0x7f660015d230 2023-10-08T00:55:44.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65feffd700 1 --2- 172.21.15.119:0/690383948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000b5f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f65f00098e0 tx=0x7f65f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aa0e51f27e6f61 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:44.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65fdffb700 1 -- 172.21.15.119:0/690383948 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0009e80 con 0x7f6600159420 2023-10-08T00:55:44.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65fdffb700 1 -- 172.21.15.119:0/690383948 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f000b860 con 0x7f6600159420 2023-10-08T00:55:44.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.614+0000 7f65fdffb700 1 -- 172.21.15.119:0/690383948 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f000bbe0 con 0x7f6600159420 2023-10-08T00:55:44.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.615+0000 7f65fffff700 1 -- 172.21.15.119:0/690383948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 msgr2=0x7f66000b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.615+0000 7f65fffff700 1 --2- 172.21.15.119:0/690383948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000b5f40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f65f00098e0 tx=0x7f65f00091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.615+0000 7f65fffff700 1 -- 172.21.15.119:0/690383948 shutdown_connections 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.615+0000 7f65fffff700 1 --2- 172.21.15.119:0/690383948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000b5f40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.615+0000 7f65fffff700 1 --2- 172.21.15.119:0/690383948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 0x7f66000b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.615+0000 7f65fffff700 1 -- 172.21.15.119:0/690383948 >> 172.21.15.119:0/690383948 conn(0x7f66000d0660 msgr2=0x7f66000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.615+0000 7f65fffff700 1 -- 172.21.15.119:0/690383948 shutdown_connections 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.615+0000 7f65fffff700 1 -- 172.21.15.119:0/690383948 wait complete. 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.616+0000 7f65fffff700 1 Processor -- start 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.616+0000 7f65fffff700 1 -- start start 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65fffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65fffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 0x7f66000cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65fffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f660015d230 0x7f66000c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65fffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000c8090 con 0x7f6600159420 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65fffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000c7480 con 0x7f660015d230 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65fffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000c7760 con 0x7f66001597f0 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65feffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65feffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50384/0 (socket says 172.21.15.119:50384) 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.617+0000 7f65feffd700 1 -- 172.21.15.119:0/4151487757 learned_addr learned my addr 172.21.15.119:0/4151487757 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65ff7fe700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f660015d230 0x7f66000c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65fe7fc700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 0x7f66000cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65feffd700 1 -- 172.21.15.119:0/4151487757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 msgr2=0x7f66000cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65feffd700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 0x7f66000cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65feffd700 1 -- 172.21.15.119:0/4151487757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f660015d230 msgr2=0x7f66000c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65feffd700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f660015d230 0x7f66000c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65feffd700 1 -- 172.21.15.119:0/4151487757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f0009420 con 0x7f6600159420 2023-10-08T00:55:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65feffd700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000c57c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6600089340 tx=0x7f65f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65e6ffd700 1 -- 172.21.15.119:0/4151487757 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f000bbe0 con 0x7f6600159420 2023-10-08T00:55:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65e6ffd700 1 -- 172.21.15.119:0/4151487757 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f0011420 con 0x7f6600159420 2023-10-08T00:55:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65e6ffd700 1 -- 172.21.15.119:0/4151487757 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f00117e0 con 0x7f6600159420 2023-10-08T00:55:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66000c7a40 con 0x7f6600159420 2023-10-08T00:55:44.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.618+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66000c5d00 con 0x7f6600159420 2023-10-08T00:55:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.619+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660004c3c0 con 0x7f6600159420 2023-10-08T00:55:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.620+0000 7f65e6ffd700 1 -- 172.21.15.119:0/4151487757 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f65f0011bd0 con 0x7f6600159420 2023-10-08T00:55:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.621+0000 7f65e6ffd700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f65ec066f30 0x7f65ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.621+0000 7f65e6ffd700 1 -- 172.21.15.119:0/4151487757 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f65f0085cf0 con 0x7f6600159420 2023-10-08T00:55:44.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.622+0000 7f65fe7fc700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f65ec066f30 0x7f65ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:44.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.622+0000 7f65fe7fc700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f65ec066f30 0x7f65ec0693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66000cedd0 tx=0x7f65f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:44.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.625+0000 7f65e6ffd700 1 -- 172.21.15.119:0/4151487757 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65f0053210 con 0x7f6600159420 2023-10-08T00:55:44.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.817+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f66000b1d50 con 0x7f6600159420 2023-10-08T00:55:44.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.817+0000 7f65e6ffd700 1 -- 172.21.15.119:0/4151487757 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f65f0015060 con 0x7f6600159420 2023-10-08T00:55:44.819 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:44.819 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T00:55:44.819 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:55:44.819 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T00:55:44.819 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T00:55:44.819 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:55:44.819 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T00:55:44.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f65ec066f30 msgr2=0x7f65ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f65ec066f30 0x7f65ec0693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f66000cedd0 tx=0x7f65f4008040 comp rx=0 tx=0).stop 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 msgr2=0x7f66000c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000c57c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6600089340 tx=0x7f65f0017000 comp rx=0 tx=0).stop 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 shutdown_connections 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f65ec066f30 0x7f65ec0693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f660015d230 0x7f66000c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6600159420 0x7f66000c57c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 --2- 172.21.15.119:0/4151487757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66001597f0 0x7f66000cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 >> 172.21.15.119:0/4151487757 conn(0x7f66000d0660 msgr2=0x7f66000b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 shutdown_connections 2023-10-08T00:55:44.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:44.820+0000 7f65fffff700 1 -- 172.21.15.119:0/4151487757 wait complete. 2023-10-08T00:55:44.839 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:55:44.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:55:45.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.026+0000 7f3d57fff700 1 Processor -- start 2023-10-08T00:55:45.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.026+0000 7f3d57fff700 1 -- start start 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.027+0000 7f3d57fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d58156350 0x7f3d58157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.027+0000 7f3d57fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d5815a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.027+0000 7f3d57fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d580b0570 con 0x7f3d580b0760 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.027+0000 7f3d57fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d581578b0 con 0x7f3d58156350 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d57fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580af4d0 con 0x7f3d580b0c00 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d56ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d58156350 0x7f3d58157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d56ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3d58156350 0x7f3d58157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41612/0 (socket says 172.21.15.119:41612) 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d56ffd700 1 -- 172.21.15.119:0/376418359 learned_addr learned my addr 172.21.15.119:0/376418359 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:45.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d567fc700 1 --1- 172.21.15.119:0/376418359 >> v1:172.21.15.119:6789/0 conn(0x7f3d580b0760 0x7f3d58159f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35256/0 (socket says 172.21.15.119:35256) 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d577fe700 1 --2- 172.21.15.119:0/376418359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d5815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d56ffd700 1 -- 172.21.15.119:0/376418359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 msgr2=0x7f3d5815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d56ffd700 1 --2- 172.21.15.119:0/376418359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d5815a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d56ffd700 1 -- 172.21.15.119:0/376418359 >> v1:172.21.15.119:6789/0 conn(0x7f3d580b0760 legacy=0x7f3d58159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d55ffb700 1 -- 172.21.15.119:0/376418359 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310360975 0 0) 0x7f3d580b0570 con 0x7f3d580b0760 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d56ffd700 1 -- 172.21.15.119:0/376418359 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d580af7b0 con 0x7f3d58156350 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d577fe700 1 --2- 172.21.15.119:0/376418359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d5815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.028+0000 7f3d56ffd700 1 --2- 172.21.15.119:0/376418359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d58156350 0x7f3d58157370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d480099b0 tx=0x7f3d48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4641fb2c133237 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d55ffb700 1 -- 172.21.15.119:0/376418359 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800b860 con 0x7f3d58156350 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d55ffb700 1 -- 172.21.15.119:0/376418359 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4800b9c0 con 0x7f3d58156350 2023-10-08T00:55:45.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d55ffb700 1 -- 172.21.15.119:0/376418359 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4800bd60 con 0x7f3d58156350 2023-10-08T00:55:45.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d57fff700 1 -- 172.21.15.119:0/376418359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d58156350 msgr2=0x7f3d58157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d57fff700 1 --2- 172.21.15.119:0/376418359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d58156350 0x7f3d58157370 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3d480099b0 tx=0x7f3d48009250 comp rx=0 tx=0).stop 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d57fff700 1 -- 172.21.15.119:0/376418359 shutdown_connections 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d57fff700 1 --2- 172.21.15.119:0/376418359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d5815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d57fff700 1 --2- 172.21.15.119:0/376418359 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d58156350 0x7f3d58157370 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d57fff700 1 -- 172.21.15.119:0/376418359 >> 172.21.15.119:0/376418359 conn(0x7f3d58068e20 msgr2=0x7f3d580cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d57fff700 1 -- 172.21.15.119:0/376418359 shutdown_connections 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.029+0000 7f3d57fff700 1 -- 172.21.15.119:0/376418359 wait complete. 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.031+0000 7f3d57fff700 1 Processor -- start 2023-10-08T00:55:45.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.031+0000 7f3d57fff700 1 -- start start 2023-10-08T00:55:45.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d580b0760 0x7f3d580c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d57fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d58146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d57fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d58156350 0x7f3d581527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d57fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d580cbde0 con 0x7f3d580b0760 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d57fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58152350 con 0x7f3d58156350 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d57fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58147350 con 0x7f3d580b0c00 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d56ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d580b0760 0x7f3d580c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d56ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d580b0760 0x7f3d580c9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41622/0 (socket says 172.21.15.119:41622) 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d56ffd700 1 -- 172.21.15.119:0/4077403057 learned_addr learned my addr 172.21.15.119:0/4077403057 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d567fc700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d58146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.032+0000 7f3d577fe700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d58156350 0x7f3d581527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d56ffd700 1 -- 172.21.15.119:0/4077403057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 msgr2=0x7f3d58146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d56ffd700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d58146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d56ffd700 1 -- 172.21.15.119:0/4077403057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d58156350 msgr2=0x7f3d581527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d56ffd700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d58156350 0x7f3d581527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d56ffd700 1 -- 172.21.15.119:0/4077403057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d480094f0 con 0x7f3d580b0760 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d56ffd700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d580b0760 0x7f3d580c9e20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3d580714a0 tx=0x7f3d48017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d3effd700 1 -- 172.21.15.119:0/4077403057 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4801a420 con 0x7f3d580b0760 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d3effd700 1 -- 172.21.15.119:0/4077403057 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d4801a580 con 0x7f3d580b0760 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d58149d10 con 0x7f3d580b0760 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d3effd700 1 -- 172.21.15.119:0/4077403057 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d4801a940 con 0x7f3d580b0760 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.033+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d58151d00 con 0x7f3d580b0760 2023-10-08T00:55:45.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.034+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d58151b60 con 0x7f3d580b0760 2023-10-08T00:55:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.035+0000 7f3d3effd700 1 -- 172.21.15.119:0/4077403057 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d4801acb0 con 0x7f3d580b0760 2023-10-08T00:55:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.036+0000 7f3d3effd700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d44066f30 0x7f3d440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.037+0000 7f3d3effd700 1 -- 172.21.15.119:0/4077403057 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f3d48085d10 con 0x7f3d580b0760 2023-10-08T00:55:45.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.037+0000 7f3d567fc700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d44066f30 0x7f3d440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.037+0000 7f3d567fc700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d44066f30 0x7f3d440693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c007fd0 tx=0x7f3d4c007e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:45.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.041+0000 7f3d3effd700 1 -- 172.21.15.119:0/4077403057 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d58151b60 con 0x7f3d580b0760 2023-10-08T00:55:45.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.232+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3d58151b60 con 0x7f3d580b0760 2023-10-08T00:55:45.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.477+0000 7f3d3effd700 1 -- 172.21.15.119:0/4077403057 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3d58151b60 con 0x7f3d580b0760 2023-10-08T00:55:45.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.480+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d44066f30 msgr2=0x7f3d440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.480+0000 7f3d57fff700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d44066f30 0x7f3d440693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c007fd0 tx=0x7f3d4c007e10 comp rx=0 tx=0).stop 2023-10-08T00:55:45.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.480+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d580b0760 msgr2=0x7f3d580c9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.480+0000 7f3d57fff700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d580b0760 0x7f3d580c9e20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3d580714a0 tx=0x7f3d48017000 comp rx=0 tx=0).stop 2023-10-08T00:55:45.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.481+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 shutdown_connections 2023-10-08T00:55:45.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.481+0000 7f3d57fff700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d44066f30 0x7f3d440693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.481+0000 7f3d57fff700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d58156350 0x7f3d581527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.481+0000 7f3d57fff700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d580b0760 0x7f3d580c9e20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.481+0000 7f3d57fff700 1 --2- 172.21.15.119:0/4077403057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d580b0c00 0x7f3d58146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.481+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 >> 172.21.15.119:0/4077403057 conn(0x7f3d58068e20 msgr2=0x7f3d580add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:45.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.481+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 shutdown_connections 2023-10-08T00:55:45.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.481+0000 7f3d57fff700 1 -- 172.21.15.119:0/4077403057 wait complete. 2023-10-08T00:55:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T00:55:45.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:55:45.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.691+0000 7fb36efff700 1 Processor -- start 2023-10-08T00:55:45.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.692+0000 7fb36efff700 1 -- start start 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36efff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3681569f0 0x7fb3680c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36efff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368068390 0x7fb368068830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36efff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36815a6f0 con 0x7fb368156620 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36efff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3680c8d30 con 0x7fb3681569f0 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36efff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb368068d70 con 0x7fb368068390 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36d59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368068390 0x7fb368068830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb35ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3681569f0 0x7fb3680c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36d59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368068390 0x7fb368068830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34084/0 (socket says 172.21.15.119:34084) 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36cd9b700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb368156620 0x7fb3680ace70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:55870/0 (socket says 172.21.15.119:55870) 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb35ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3681569f0 0x7fb3680c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54348/0 (socket says 172.21.15.119:54348) 2023-10-08T00:55:45.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.693+0000 7fb36d59c700 1 -- 172.21.15.119:0/2353771587 learned_addr learned my addr 172.21.15.119:0/2353771587 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:45.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb36d59c700 1 -- 172.21.15.119:0/2353771587 >> v2:172.21.15.184:3300/0 conn(0x7fb3681569f0 msgr2=0x7fb3680c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:45.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb36d59c700 1 --2- 172.21.15.119:0/2353771587 >> v2:172.21.15.184:3300/0 conn(0x7fb3681569f0 0x7fb3680c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb36d59c700 1 -- 172.21.15.119:0/2353771587 >> v1:172.21.15.184:6789/0 conn(0x7fb368156620 legacy=0x7fb3680ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb36d59c700 1 -- 172.21.15.119:0/2353771587 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36815a9d0 con 0x7fb368068390 2023-10-08T00:55:45.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb35ffff700 1 --2- 172.21.15.119:0/2353771587 >> v2:172.21.15.184:3300/0 conn(0x7fb3681569f0 0x7fb3680c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:45.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb35f7fe700 1 -- 172.21.15.119:0/2353771587 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208524158 0 0) 0x7fb36815a6f0 con 0x7fb368156620 2023-10-08T00:55:45.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb36d59c700 1 --2- 172.21.15.119:0/2353771587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368068390 0x7fb368068830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb3600099b0 tx=0x7fb360009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db400bad26ae82 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb35f7fe700 1 -- 172.21.15.119:0/2353771587 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360009ea0 con 0x7fb368068390 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb35f7fe700 1 -- 172.21.15.119:0/2353771587 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36000b860 con 0x7fb368068390 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.694+0000 7fb35f7fe700 1 -- 172.21.15.119:0/2353771587 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36000bc00 con 0x7fb368068390 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.695+0000 7fb36efff700 1 -- 172.21.15.119:0/2353771587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368068390 msgr2=0x7fb368068830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.695+0000 7fb36efff700 1 --2- 172.21.15.119:0/2353771587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368068390 0x7fb368068830 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb3600099b0 tx=0x7fb360009250 comp rx=0 tx=0).stop 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.695+0000 7fb36efff700 1 -- 172.21.15.119:0/2353771587 shutdown_connections 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.695+0000 7fb36efff700 1 --2- 172.21.15.119:0/2353771587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368068390 0x7fb368068830 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.695+0000 7fb36efff700 1 --2- 172.21.15.119:0/2353771587 >> v2:172.21.15.184:3300/0 conn(0x7fb3681569f0 0x7fb3680c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.695+0000 7fb36efff700 1 -- 172.21.15.119:0/2353771587 >> 172.21.15.119:0/2353771587 conn(0x7fb3680cc540 msgr2=0x7fb3680cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:45.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.695+0000 7fb36efff700 1 -- 172.21.15.119:0/2353771587 shutdown_connections 2023-10-08T00:55:45.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.695+0000 7fb36efff700 1 -- 172.21.15.119:0/2353771587 wait complete. 2023-10-08T00:55:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.696+0000 7fb36efff700 1 Processor -- start 2023-10-08T00:55:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.697+0000 7fb36efff700 1 -- start start 2023-10-08T00:55:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.697+0000 7fb36efff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb368068390 0x7fb3680c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.697+0000 7fb36efff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368156620 0x7fb368153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.697+0000 7fb36efff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3681569f0 0x7fb3680b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.697+0000 7fb36efff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3680c4540 con 0x7fb3681569f0 2023-10-08T00:55:45.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.697+0000 7fb36efff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3680c4820 con 0x7fb368068390 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.697+0000 7fb36efff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3680c4b00 con 0x7fb368156620 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3681569f0 0x7fb3680b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3681569f0 0x7fb3680b0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41636/0 (socket says 172.21.15.119:41636) 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 -- 172.21.15.119:0/96223830 learned_addr learned my addr 172.21.15.119:0/96223830 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36cd9b700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb368068390 0x7fb3680c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb35ffff700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368156620 0x7fb368153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 -- 172.21.15.119:0/96223830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368156620 msgr2=0x7fb368153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368156620 0x7fb368153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 -- 172.21.15.119:0/96223830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb368068390 msgr2=0x7fb3680c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb368068390 0x7fb3680c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 -- 172.21.15.119:0/96223830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb358009420 con 0x7fb3681569f0 2023-10-08T00:55:45.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb36d59c700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3681569f0 0x7fb3680b0e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb360006fd0 tx=0x7fb360017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.698+0000 7fb35cff9700 1 -- 172.21.15.119:0/96223830 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360011420 con 0x7fb3681569f0 2023-10-08T00:55:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.699+0000 7fb35cff9700 1 -- 172.21.15.119:0/96223830 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb360011580 con 0x7fb3681569f0 2023-10-08T00:55:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.699+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3600094f0 con 0x7fb3681569f0 2023-10-08T00:55:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.699+0000 7fb35cff9700 1 -- 172.21.15.119:0/96223830 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb360011940 con 0x7fb3681569f0 2023-10-08T00:55:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.699+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3680c50c0 con 0x7fb3681569f0 2023-10-08T00:55:45.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.700+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3680718b0 con 0x7fb3681569f0 2023-10-08T00:55:45.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.701+0000 7fb35cff9700 1 -- 172.21.15.119:0/96223830 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb36002e430 con 0x7fb3681569f0 2023-10-08T00:55:45.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.701+0000 7fb35cff9700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb354066f30 0x7fb3540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:45.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.702+0000 7fb35cff9700 1 -- 172.21.15.119:0/96223830 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5223+0+0 (secure 0 0 0) 0x7fb360085e30 con 0x7fb3681569f0 2023-10-08T00:55:45.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.702+0000 7fb36cd9b700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb354066f30 0x7fb3540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:45.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.702+0000 7fb36cd9b700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb354066f30 0x7fb3540693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3680c3500 tx=0x7fb364001b00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:45.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.706+0000 7fb35cff9700 1 -- 172.21.15.119:0/96223830 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb360053290 con 0x7fb3681569f0 2023-10-08T00:55:45.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:45.898+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb3680678d0 con 0x7fb3681569f0 2023-10-08T00:55:46.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.490+0000 7fb35cff9700 1 -- 172.21.15.119:0/96223830 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb3600568b0 con 0x7fb3681569f0 2023-10-08T00:55:46.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.507+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb354066f30 msgr2=0x7fb3540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:46.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.507+0000 7fb36efff700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb354066f30 0x7fb3540693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3680c3500 tx=0x7fb364001b00 comp rx=0 tx=0).stop 2023-10-08T00:55:46.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.507+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3681569f0 msgr2=0x7fb3680b0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:46.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.507+0000 7fb36efff700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3681569f0 0x7fb3680b0e40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb360006fd0 tx=0x7fb360017000 comp rx=0 tx=0).stop 2023-10-08T00:55:46.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.507+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 shutdown_connections 2023-10-08T00:55:46.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.507+0000 7fb36efff700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb354066f30 0x7fb3540693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:46.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.507+0000 7fb36efff700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3681569f0 0x7fb3680b0e40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:46.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.508+0000 7fb36efff700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb368068390 0x7fb3680c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:46.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.508+0000 7fb36efff700 1 --2- 172.21.15.119:0/96223830 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb368156620 0x7fb368153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:46.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.508+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 >> 172.21.15.119:0/96223830 conn(0x7fb3680cc540 msgr2=0x7fb368155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:46.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.509+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 shutdown_connections 2023-10-08T00:55:46.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.509+0000 7fb36efff700 1 -- 172.21.15.119:0/96223830 wait complete. 2023-10-08T00:55:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T00:55:46.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:55:46.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.714+0000 7f663c9fe700 1 Processor -- start 2023-10-08T00:55:46.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.714+0000 7f663c9fe700 1 -- start start 2023-10-08T00:55:46.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.715+0000 7f663c9fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f663806a710 0x7f6638160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:46.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.715+0000 7f663c9fe700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663806e150 con 0x7f6638153900 2023-10-08T00:55:46.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.715+0000 7f663c9fe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66380c7800 con 0x7f66380c9db0 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.715+0000 7f663c9fe700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66380c7610 con 0x7f663806a710 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.715+0000 7f663759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f663806a710 0x7f6638160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.715+0000 7f663759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f663806a710 0x7f6638160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41638/0 (socket says 172.21.15.119:41638) 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.715+0000 7f663759e700 1 -- 172.21.15.119:0/2798961871 learned_addr learned my addr 172.21.15.119:0/2798961871 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.715+0000 7f6636d9d700 1 --1- 172.21.15.119:0/2798961871 >> v1:172.21.15.119:6789/0 conn(0x7f6638153900 0x7f66380c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35268/0 (socket says 172.21.15.119:35268) 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.716+0000 7f663759e700 1 -- 172.21.15.119:0/2798961871 >> v1:172.21.15.184:6789/0 conn(0x7f66380c9db0 legacy=0x7f663806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.716+0000 7f663759e700 1 -- 172.21.15.119:0/2798961871 >> v1:172.21.15.119:6789/0 conn(0x7f6638153900 legacy=0x7f66380c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.716+0000 7f663759e700 1 -- 172.21.15.119:0/2798961871 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66380c7ae0 con 0x7f663806a710 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.716+0000 7f663759e700 1 --2- 172.21.15.119:0/2798961871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663806a710 0x7f6638160e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6630009f20 tx=0x7f6630009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92fd119ffd69141 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:46.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.716+0000 7f6635d9b700 1 -- 172.21.15.119:0/2798961871 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f663000e070 con 0x7f663806a710 2023-10-08T00:55:46.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.716+0000 7f6635d9b700 1 -- 172.21.15.119:0/2798961871 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663000ba80 con 0x7f663806a710 2023-10-08T00:55:46.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.716+0000 7f6635d9b700 1 -- 172.21.15.119:0/2798961871 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300092e0 con 0x7f663806a710 2023-10-08T00:55:46.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.717+0000 7f663c9fe700 1 -- 172.21.15.119:0/2798961871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663806a710 msgr2=0x7f6638160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:46.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.717+0000 7f663c9fe700 1 --2- 172.21.15.119:0/2798961871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663806a710 0x7f6638160e70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6630009f20 tx=0x7f6630009840 comp rx=0 tx=0).stop 2023-10-08T00:55:46.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.717+0000 7f663c9fe700 1 -- 172.21.15.119:0/2798961871 shutdown_connections 2023-10-08T00:55:46.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.717+0000 7f663c9fe700 1 --2- 172.21.15.119:0/2798961871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663806a710 0x7f6638160e70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:46.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.717+0000 7f663c9fe700 1 -- 172.21.15.119:0/2798961871 >> 172.21.15.119:0/2798961871 conn(0x7f66380cc1e0 msgr2=0x7f66380cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:46.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.717+0000 7f663c9fe700 1 -- 172.21.15.119:0/2798961871 shutdown_connections 2023-10-08T00:55:46.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.717+0000 7f663c9fe700 1 -- 172.21.15.119:0/2798961871 wait complete. 2023-10-08T00:55:46.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.718+0000 7f663c9fe700 1 Processor -- start 2023-10-08T00:55:46.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.719+0000 7f663c9fe700 1 -- start start 2023-10-08T00:55:46.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.719+0000 7f663c9fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f663806a710 0x7f663816d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:46.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.719+0000 7f663c9fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66380c9db0 0x7f6638152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:46.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.719+0000 7f663c9fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6638153900 0x7f66381555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:46.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.719+0000 7f663c9fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663816e340 con 0x7f6638153900 2023-10-08T00:55:46.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.719+0000 7f663c9fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6638155140 con 0x7f663806a710 2023-10-08T00:55:46.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.719+0000 7f663c9fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663816dd80 con 0x7f66380c9db0 2023-10-08T00:55:46.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6638153900 0x7f66381555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:46.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6638153900 0x7f66381555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41648/0 (socket says 172.21.15.119:41648) 2023-10-08T00:55:46.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 -- 172.21.15.119:0/127411870 learned_addr learned my addr 172.21.15.119:0/127411870 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:46.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f6636d9d700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f663806a710 0x7f663816d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:46.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663659c700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66380c9db0 0x7f6638152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:46.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 -- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66380c9db0 msgr2=0x7f6638152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:46.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66380c9db0 0x7f6638152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:46.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 -- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f663806a710 msgr2=0x7f663816d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:46.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f663806a710 0x7f663816d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:46.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 -- 172.21.15.119:0/127411870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6630009ae0 con 0x7f6638153900 2023-10-08T00:55:46.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f6636d9d700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f663806a710 0x7f663816d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:46.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663659c700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66380c9db0 0x7f6638152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:46.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f663759e700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6638153900 0x7f66381555d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6630012000 tx=0x7f6630018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:46.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.720+0000 7f661effd700 1 -- 172.21.15.119:0/127411870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6630008e80 con 0x7f6638153900 2023-10-08T00:55:46.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.721+0000 7f661effd700 1 -- 172.21.15.119:0/127411870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f663000be90 con 0x7f6638153900 2023-10-08T00:55:46.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.721+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663816e060 con 0x7f6638153900 2023-10-08T00:55:46.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.721+0000 7f661effd700 1 -- 172.21.15.119:0/127411870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66300266b0 con 0x7f6638153900 2023-10-08T00:55:46.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.721+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663816d950 con 0x7f6638153900 2023-10-08T00:55:46.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.722+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663816db40 con 0x7f6638153900 2023-10-08T00:55:46.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.723+0000 7f661effd700 1 -- 172.21.15.119:0/127411870 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6630003040 con 0x7f6638153900 2023-10-08T00:55:46.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.724+0000 7f661effd700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6624066f30 0x7f66240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:46.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.724+0000 7f661effd700 1 -- 172.21.15.119:0/127411870 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f6630086680 con 0x7f6638153900 2023-10-08T00:55:46.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.724+0000 7f6636d9d700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6624066f30 0x7f66240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:46.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.724+0000 7f6636d9d700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6624066f30 0x7f66240693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6628007fd0 tx=0x7f6628009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:46.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.728+0000 7f661effd700 1 -- 172.21.15.119:0/127411870 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f663816db40 con 0x7f6638153900 2023-10-08T00:55:46.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:46.963+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f663816db40 con 0x7f6638153900 2023-10-08T00:55:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.494+0000 7f661effd700 1 -- 172.21.15.119:0/127411870 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f663816db40 con 0x7f6638153900 2023-10-08T00:55:47.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6624066f30 msgr2=0x7f66240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:47.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6624066f30 0x7f66240693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6628007fd0 tx=0x7f6628009040 comp rx=0 tx=0).stop 2023-10-08T00:55:47.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6638153900 msgr2=0x7f66381555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:47.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6638153900 0x7f66381555d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6630012000 tx=0x7f6630018000 comp rx=0 tx=0).stop 2023-10-08T00:55:47.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 shutdown_connections 2023-10-08T00:55:47.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6624066f30 0x7f66240693e0 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6628007fd0 tx=0x7f6628009040 comp rx=0 tx=0).stop 2023-10-08T00:55:47.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6638153900 0x7f66381555d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f663806a710 0x7f663816d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 --2- 172.21.15.119:0/127411870 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66380c9db0 0x7f6638152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.501+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 >> 172.21.15.119:0/127411870 conn(0x7f66380cc1e0 msgr2=0x7f66380c16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:47.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.511+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 shutdown_connections 2023-10-08T00:55:47.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.515+0000 7f663c9fe700 1 -- 172.21.15.119:0/127411870 wait complete. 2023-10-08T00:55:47.521 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 2 and data pool 3 2023-10-08T00:55:47.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:47.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.720+0000 7f50a33ff700 1 Processor -- start 2023-10-08T00:55:47.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.720+0000 7f50a33ff700 1 -- start start 2023-10-08T00:55:47.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.721+0000 7f50a33ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c155c10 0x7f509c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.721+0000 7f50a33ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f509c0ac230 0x7f509c0ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.721+0000 7f50a33ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f509c0ad930 con 0x7f509c1568a0 2023-10-08T00:55:47.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.721+0000 7f50a33ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509c0acd20 con 0x7f509c0ac230 2023-10-08T00:55:47.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.721+0000 7f50a33ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0ad000 con 0x7f509c155c10 2023-10-08T00:55:47.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.721+0000 7f50a199c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f509c1568a0 0x7f509c156cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35280/0 (socket says 172.21.15.119:35280) 2023-10-08T00:55:47.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.721+0000 7f50a199c700 1 -- 172.21.15.119:0/1407301658 learned_addr learned my addr 172.21.15.119:0/1407301658 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:47.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f50a099a700 1 --2- 172.21.15.119:0/1407301658 >> v2:172.21.15.184:3300/0 conn(0x7f509c0ac230 0x7f509c0ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f50a119b700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c155c10 0x7f509c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808671169 0 0) 0x7f509c0ad930 con 0x7f509c1568a0 2023-10-08T00:55:47.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5088003570 con 0x7f509c1568a0 2023-10-08T00:55:47.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3193980443 0 0) 0x7f5088003570 con 0x7f509c1568a0 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c155c10 msgr2=0x7f509c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c155c10 0x7f509c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 >> v2:172.21.15.184:3300/0 conn(0x7f509c0ac230 msgr2=0x7f509c0ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 --2- 172.21.15.119:0/1407301658 >> v2:172.21.15.184:3300/0 conn(0x7f509c0ac230 0x7f509c0ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0ad2e0 con 0x7f509c1568a0 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f50940017f0 con 0x7f509c1568a0 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 >> v1:172.21.15.119:6789/0 conn(0x7f509c1568a0 legacy=0x7f509c156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 0x7f5088003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5088004810 0x7f5088004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c155c10 0x7f50880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0acd20 con 0x7f5088004810 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c0ad000 con 0x7f5088003a60 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508800ba60 con 0x7f509c155c10 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.722+0000 7f50a099a700 1 --2- 172.21.15.119:0/1407301658 >> v2:172.21.15.184:3300/0 conn(0x7f509c0ac230 0x7f509c0ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a119b700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 0x7f5088003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a199c700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c155c10 0x7f50880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a099a700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5088004810 0x7f5088004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a119b700 1 -- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c155c10 msgr2=0x7f50880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a119b700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c155c10 0x7f50880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a119b700 1 -- 172.21.15.119:0/1407301658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5088004810 msgr2=0x7f5088004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a119b700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5088004810 0x7f5088004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a119b700 1 -- 172.21.15.119:0/1407301658 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088003570 con 0x7f5088003a60 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.723+0000 7f50a119b700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 0x7f5088003e70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f509800acf0 tx=0x7f509800a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dac1064e02092d6c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800e7b0 con 0x7f5088003a60 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509800e910 con 0x7f5088003a60 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f5093fff700 1 -- 172.21.15.119:0/1407301658 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509800eaa0 con 0x7f5088003a60 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f50a33ff700 1 -- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 msgr2=0x7f5088003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f50a33ff700 1 --2- 172.21.15.119:0/1407301658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 0x7f5088003e70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f509800acf0 tx=0x7f509800a330 comp rx=0 tx=0).stop 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f50a199c700 1 -- 172.21.15.119:0/1407301658 reap_dead start 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f50a33ff700 1 -- 172.21.15.119:0/1407301658 shutdown_connections 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f50a33ff700 1 -- 172.21.15.119:0/1407301658 >> 172.21.15.119:0/1407301658 conn(0x7f509c06a320 msgr2=0x7f509c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f50a33ff700 1 -- 172.21.15.119:0/1407301658 shutdown_connections 2023-10-08T00:55:47.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.724+0000 7f50a33ff700 1 -- 172.21.15.119:0/1407301658 wait complete. 2023-10-08T00:55:47.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.726+0000 7f50a33ff700 1 Processor -- start 2023-10-08T00:55:47.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.726+0000 7f50a33ff700 1 -- start start 2023-10-08T00:55:47.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a33ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 0x7f509c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a33ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c14f6b0 0x7f509c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a33ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c14fb20 0x7f509c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a33ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c1518b0 con 0x7f509c14fb20 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a33ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509c1509f0 con 0x7f5088003a60 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a33ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509c150ca0 con 0x7f509c14f6b0 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c14fb20 0x7f509c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c14fb20 0x7f509c06ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41664/0 (socket says 172.21.15.119:41664) 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 -- 172.21.15.119:0/3069735974 learned_addr learned my addr 172.21.15.119:0/3069735974 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a119b700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 0x7f509c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a099a700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c14f6b0 0x7f509c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 -- 172.21.15.119:0/3069735974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c14f6b0 msgr2=0x7f509c06d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:47.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c14f6b0 0x7f509c06d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 -- 172.21.15.119:0/3069735974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 msgr2=0x7f509c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 0x7f509c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 -- 172.21.15.119:0/3069735974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c150f80 con 0x7f509c14fb20 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.727+0000 7f50a199c700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c14fb20 0x7f509c06ddd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5094000f80 tx=0x7f509c0ad8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.728+0000 7f50917fa700 1 -- 172.21.15.119:0/3069735974 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509400acd0 con 0x7f509c14fb20 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.728+0000 7f50917fa700 1 -- 172.21.15.119:0/3069735974 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509400ae50 con 0x7f509c14fb20 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.728+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509c151260 con 0x7f509c14fb20 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.728+0000 7f50917fa700 1 -- 172.21.15.119:0/3069735974 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094013640 con 0x7f509c14fb20 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.728+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509c0c9230 con 0x7f509c14fb20 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.729+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f509c151a60 con 0x7f509c14fb20 2023-10-08T00:55:47.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.730+0000 7f50917fa700 1 -- 172.21.15.119:0/3069735974 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f50940139b0 con 0x7f509c14fb20 2023-10-08T00:55:47.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.731+0000 7f50917fa700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5080066f60 0x7f5080069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:47.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.731+0000 7f50917fa700 1 -- 172.21.15.119:0/3069735974 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5094084e90 con 0x7f509c14fb20 2023-10-08T00:55:47.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.731+0000 7f50a119b700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5080066f60 0x7f5080069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:47.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.731+0000 7f50a119b700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5080066f60 0x7f5080069410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f509800ac10 tx=0x7f5098019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:47.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.734+0000 7f50917fa700 1 -- 172.21.15.119:0/3069735974 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f509c151a60 con 0x7f509c14fb20 2023-10-08T00:55:47.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.926+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f509c04c3c0 con 0x7f509c14fb20 2023-10-08T00:55:47.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.927+0000 7f50917fa700 1 -- 172.21.15.119:0/3069735974 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f5094052060 con 0x7f509c14fb20 2023-10-08T00:55:47.928 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:47.928 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":19,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:46.965586+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5080066f60 msgr2=0x7f5080069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:47.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5080066f60 0x7f5080069410 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f509800ac10 tx=0x7f5098019000 comp rx=0 tx=0).stop 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c14fb20 msgr2=0x7f509c06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c14fb20 0x7f509c06ddd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5094000f80 tx=0x7f509c0ad8a0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 shutdown_connections 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5080066f60 0x7f5080069410 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f509c14fb20 0x7f509c06ddd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5088003a60 0x7f509c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 --2- 172.21.15.119:0/3069735974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509c14f6b0 0x7f509c06d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.929+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 >> 172.21.15.119:0/3069735974 conn(0x7f509c06a320 msgr2=0x7f509c154750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:47.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.930+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 shutdown_connections 2023-10-08T00:55:47.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:47.930+0000 7f50a33ff700 1 -- 172.21.15.119:0/3069735974 wait complete. 2023-10-08T00:55:47.949 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:48.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.129+0000 7f1832dff700 1 Processor -- start 2023-10-08T00:55:48.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.130+0000 7f1832dff700 1 -- start start 2023-10-08T00:55:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1832dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f182c1599c0 0x7f182c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1832dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f182c069aa0 0x7f182c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1832dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15ab90 0x7f182c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1832dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c06a220 con 0x7f182c1599c0 2023-10-08T00:55:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1832dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f182c069f40 con 0x7f182c069aa0 2023-10-08T00:55:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1832dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c15a0b0 con 0x7f182c15ab90 2023-10-08T00:55:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1830b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f182c1599c0 0x7f182c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1830b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f182c1599c0 0x7f182c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41672/0 (socket says 172.21.15.119:41672) 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1830b9b700 1 -- 172.21.15.119:0/3140565218 learned_addr learned my addr 172.21.15.119:0/3140565218 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1830b9b700 1 -- 172.21.15.119:0/3140565218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15ab90 msgr2=0x7f182c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1830b9b700 1 --2- 172.21.15.119:0/3140565218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15ab90 0x7f182c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f1830b9b700 1 -- 172.21.15.119:0/3140565218 >> v2:172.21.15.184:3300/0 conn(0x7f182c069aa0 msgr2=0x7f182c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.131+0000 7f182bfff700 1 --2- 172.21.15.119:0/3140565218 >> v2:172.21.15.184:3300/0 conn(0x7f182c069aa0 0x7f182c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f1830b9b700 1 --2- 172.21.15.119:0/3140565218 >> v2:172.21.15.184:3300/0 conn(0x7f182c069aa0 0x7f182c0cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f1830b9b700 1 -- 172.21.15.119:0/3140565218 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c15a390 con 0x7f182c1599c0 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f182bfff700 1 --2- 172.21.15.119:0/3140565218 >> v2:172.21.15.184:3300/0 conn(0x7f182c069aa0 0x7f182c0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f1830b9b700 1 --2- 172.21.15.119:0/3140565218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1599c0 0x7f182c069560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1820009f20 tx=0x7f1820009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df89257fcbf0e7c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f182b7fe700 1 -- 172.21.15.119:0/3140565218 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182000e070 con 0x7f182c1599c0 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f182b7fe700 1 -- 172.21.15.119:0/3140565218 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182000ba40 con 0x7f182c1599c0 2023-10-08T00:55:48.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f182b7fe700 1 -- 172.21.15.119:0/3140565218 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18200092e0 con 0x7f182c1599c0 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f1832dff700 1 -- 172.21.15.119:0/3140565218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1599c0 msgr2=0x7f182c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.132+0000 7f1832dff700 1 --2- 172.21.15.119:0/3140565218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1599c0 0x7f182c069560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1820009f20 tx=0x7f1820009840 comp rx=0 tx=0).stop 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.133+0000 7f1832dff700 1 -- 172.21.15.119:0/3140565218 shutdown_connections 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.133+0000 7f1832dff700 1 --2- 172.21.15.119:0/3140565218 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15ab90 0x7f182c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.133+0000 7f1832dff700 1 --2- 172.21.15.119:0/3140565218 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1599c0 0x7f182c069560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.133+0000 7f1832dff700 1 --2- 172.21.15.119:0/3140565218 >> v2:172.21.15.184:3300/0 conn(0x7f182c069aa0 0x7f182c0cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.133+0000 7f1832dff700 1 -- 172.21.15.119:0/3140565218 >> 172.21.15.119:0/3140565218 conn(0x7f182c068e10 msgr2=0x7f182c0d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.133+0000 7f1832dff700 1 -- 172.21.15.119:0/3140565218 shutdown_connections 2023-10-08T00:55:48.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.133+0000 7f1832dff700 1 -- 172.21.15.119:0/3140565218 wait complete. 2023-10-08T00:55:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.134+0000 7f1832dff700 1 Processor -- start 2023-10-08T00:55:48.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.134+0000 7f1832dff700 1 -- start start 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1832dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c069aa0 0x7f182c140ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1832dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c1599c0 0x7f182c0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1832dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c15ab90 0x7f182c0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1832dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c140ff0 con 0x7f182c069aa0 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1832dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b5dd0 con 0x7f182c15ab90 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1832dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0ca750 con 0x7f182c1599c0 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1830b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c069aa0 0x7f182c140ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1830b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c069aa0 0x7f182c140ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41686/0 (socket says 172.21.15.119:41686) 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.135+0000 7f1830b9b700 1 -- 172.21.15.119:0/2431311222 learned_addr learned my addr 172.21.15.119:0/2431311222 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f182bfff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c1599c0 0x7f182c0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f183139c700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c15ab90 0x7f182c0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1830b9b700 1 -- 172.21.15.119:0/2431311222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c1599c0 msgr2=0x7f182c0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1830b9b700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c1599c0 0x7f182c0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1830b9b700 1 -- 172.21.15.119:0/2431311222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c15ab90 msgr2=0x7f182c0c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1830b9b700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c15ab90 0x7f182c0c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1830b9b700 1 -- 172.21.15.119:0/2431311222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820009ae0 con 0x7f182c069aa0 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1830b9b700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c069aa0 0x7f182c140ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f182000b6c0 tx=0x7f1820018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1828ff9700 1 -- 172.21.15.119:0/2431311222 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820008be0 con 0x7f182c069aa0 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1828ff9700 1 -- 172.21.15.119:0/2431311222 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1820008d60 con 0x7f182c069aa0 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1828ff9700 1 -- 172.21.15.119:0/2431311222 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18200266b0 con 0x7f182c069aa0 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c0b5400 con 0x7f182c069aa0 2023-10-08T00:55:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.136+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c0b4ad0 con 0x7f182c069aa0 2023-10-08T00:55:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.137+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c064550 con 0x7f182c069aa0 2023-10-08T00:55:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.138+0000 7f1828ff9700 1 -- 172.21.15.119:0/2431311222 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1820003040 con 0x7f182c069aa0 2023-10-08T00:55:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.139+0000 7f1828ff9700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1810066f30 0x7f18100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.139+0000 7f1828ff9700 1 -- 172.21.15.119:0/2431311222 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1820086530 con 0x7f182c069aa0 2023-10-08T00:55:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.139+0000 7f182bfff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1810066f30 0x7f18100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.140+0000 7f182bfff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1810066f30 0x7f18100693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f182c0c6af0 tx=0x7f1818008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:48.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.144+0000 7f1828ff9700 1 -- 172.21.15.119:0/2431311222 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1820053700 con 0x7f182c069aa0 2023-10-08T00:55:48.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.335+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f182c04c3c0 con 0x7f182c069aa0 2023-10-08T00:55:48.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.337+0000 7f1828ff9700 1 -- 172.21.15.119:0/2431311222 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f1820056d20 con 0x7f182c069aa0 2023-10-08T00:55:48.338 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:48.338 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":19,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:46.965586+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:48.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1810066f30 msgr2=0x7f18100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1810066f30 0x7f18100693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f182c0c6af0 tx=0x7f1818008040 comp rx=0 tx=0).stop 2023-10-08T00:55:48.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c069aa0 msgr2=0x7f182c140ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c069aa0 0x7f182c140ab0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f182000b6c0 tx=0x7f1820018000 comp rx=0 tx=0).stop 2023-10-08T00:55:48.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 shutdown_connections 2023-10-08T00:55:48.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1810066f30 0x7f18100693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c15ab90 0x7f182c0c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c069aa0 0x7f182c140ab0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 --2- 172.21.15.119:0/2431311222 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c1599c0 0x7f182c0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 >> 172.21.15.119:0/2431311222 conn(0x7f182c068e10 msgr2=0x7f182c155460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:48.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 shutdown_connections 2023-10-08T00:55:48.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.339+0000 7f1832dff700 1 -- 172.21.15.119:0/2431311222 wait complete. 2023-10-08T00:55:48.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:55:48.474 INFO:tasks.ceph.mds.b.smithi184.stderr:2023-10-08T00:55:48.473+0000 7f5aec9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:55:48.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.544+0000 7f848099d700 1 Processor -- start 2023-10-08T00:55:48.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.545+0000 7f848099d700 1 -- start start 2023-10-08T00:55:48.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.546+0000 7f848099d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f060 0x7f847c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.546+0000 7f848099d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f847c06f4b0 0x7f847c0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.546+0000 7f848099d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c06ebd0 con 0x7f847c0cb1a0 2023-10-08T00:55:48.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.546+0000 7f848099d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847c15b2e0 con 0x7f847c06f4b0 2023-10-08T00:55:48.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.546+0000 7f848099d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c15aa70 con 0x7f847c06f060 2023-10-08T00:55:48.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f060 0x7f847c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f847c06f4b0 0x7f847c0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f060 0x7f847c0ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34166/0 (socket says 172.21.15.119:34166) 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847affd700 1 -- 172.21.15.119:0/1339243865 learned_addr learned my addr 172.21.15.119:0/1339243865 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847affd700 1 -- 172.21.15.119:0/1339243865 >> v2:172.21.15.184:3300/0 conn(0x7f847c06f4b0 msgr2=0x7f847c0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847affd700 1 --2- 172.21.15.119:0/1339243865 >> v2:172.21.15.184:3300/0 conn(0x7f847c06f4b0 0x7f847c0ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847affd700 1 -- 172.21.15.119:0/1339243865 >> v1:172.21.15.184:6789/0 conn(0x7f847c0cb1a0 legacy=0x7f847c15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847affd700 1 -- 172.21.15.119:0/1339243865 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c15ad20 con 0x7f847c06f060 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847affd700 1 --2- 172.21.15.119:0/1339243865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f060 0x7f847c0ae0d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f84700099b0 tx=0x7f8470009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d834780758b35a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.547+0000 7f847a7fc700 1 -- 172.21.15.119:0/1339243865 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470009ea0 con 0x7f847c06f060 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f847a7fc700 1 -- 172.21.15.119:0/1339243865 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847000b8f0 con 0x7f847c06f060 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f847a7fc700 1 -- 172.21.15.119:0/1339243865 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847000bc90 con 0x7f847c06f060 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f848099d700 1 -- 172.21.15.119:0/1339243865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f060 msgr2=0x7f847c0ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f848099d700 1 --2- 172.21.15.119:0/1339243865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f060 0x7f847c0ae0d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f84700099b0 tx=0x7f8470009250 comp rx=0 tx=0).stop 2023-10-08T00:55:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f848099d700 1 -- 172.21.15.119:0/1339243865 shutdown_connections 2023-10-08T00:55:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f848099d700 1 --2- 172.21.15.119:0/1339243865 >> v2:172.21.15.184:3300/0 conn(0x7f847c06f4b0 0x7f847c0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f848099d700 1 --2- 172.21.15.119:0/1339243865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f060 0x7f847c0ae0d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f848099d700 1 -- 172.21.15.119:0/1339243865 >> 172.21.15.119:0/1339243865 conn(0x7f847c0cc1e0 msgr2=0x7f847c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f848099d700 1 -- 172.21.15.119:0/1339243865 shutdown_connections 2023-10-08T00:55:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.548+0000 7f848099d700 1 -- 172.21.15.119:0/1339243865 wait complete. 2023-10-08T00:55:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.550+0000 7f848099d700 1 Processor -- start 2023-10-08T00:55:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.550+0000 7f848099d700 1 -- start start 2023-10-08T00:55:48.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f848099d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f847c06f060 0x7f847c153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f848099d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f4b0 0x7f847c152420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f848099d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f847c0cb1a0 0x7f847c150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f848099d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c152960 con 0x7f847c0cb1a0 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f848099d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c155270 con 0x7f847c06f060 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f848099d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c154660 con 0x7f847c06f4b0 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f847c0cb1a0 0x7f847c150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f4b0 0x7f847c152420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f847c06f060 0x7f847c153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f847c0cb1a0 0x7f847c150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41688/0 (socket says 172.21.15.119:41688) 2023-10-08T00:55:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847bfff700 1 -- 172.21.15.119:0/3864156399 learned_addr learned my addr 172.21.15.119:0/3864156399 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847bfff700 1 -- 172.21.15.119:0/3864156399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f4b0 msgr2=0x7f847c152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847bfff700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f4b0 0x7f847c152420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847bfff700 1 -- 172.21.15.119:0/3864156399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f847c06f060 msgr2=0x7f847c153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847bfff700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f847c06f060 0x7f847c153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.551+0000 7f847bfff700 1 -- 172.21.15.119:0/3864156399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700094f0 con 0x7f847c0cb1a0 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.552+0000 7f847bfff700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f847c0cb1a0 0x7f847c150e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8474009e90 tx=0x7f847400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.552+0000 7f84637fe700 1 -- 172.21.15.119:0/3864156399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847400cb70 con 0x7f847c0cb1a0 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.552+0000 7f84637fe700 1 -- 172.21.15.119:0/3864156399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847400f040 con 0x7f847c0cb1a0 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.552+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c154940 con 0x7f847c0cb1a0 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.552+0000 7f84637fe700 1 -- 172.21.15.119:0/3864156399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474014480 con 0x7f847c0cb1a0 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.552+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c154f00 con 0x7f847c0cb1a0 2023-10-08T00:55:48.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.553+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c06c450 con 0x7f847c0cb1a0 2023-10-08T00:55:48.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.554+0000 7f84637fe700 1 -- 172.21.15.119:0/3864156399 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8474014830 con 0x7f847c0cb1a0 2023-10-08T00:55:48.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.555+0000 7f84637fe700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8468066f30 0x7f84680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:48.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.555+0000 7f84637fe700 1 -- 172.21.15.119:0/3864156399 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8474084fb0 con 0x7f847c0cb1a0 2023-10-08T00:55:48.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.555+0000 7f847b7fe700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8468066f30 0x7f84680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:48.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.555+0000 7f847b7fe700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8468066f30 0x7f84680693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f846c003a80 tx=0x7f846c007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:48.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.559+0000 7f84637fe700 1 -- 172.21.15.119:0/3864156399 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84740b65a0 con 0x7f847c0cb1a0 2023-10-08T00:55:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:48.793+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f847c04bb80 con 0x7f847c0cb1a0 2023-10-08T00:55:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.512+0000 7f84637fe700 1 -- 172.21.15.119:0/3864156399 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8474052220 con 0x7f847c0cb1a0 2023-10-08T00:55:49.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8468066f30 msgr2=0x7f84680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8468066f30 0x7f84680693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f846c003a80 tx=0x7f846c007ec0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f847c0cb1a0 msgr2=0x7f847c150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f847c0cb1a0 0x7f847c150e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8474009e90 tx=0x7f847400b040 comp rx=0 tx=0).stop 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 shutdown_connections 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8468066f30 0x7f84680693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f847c0cb1a0 0x7f847c150e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f847c06f060 0x7f847c153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 --2- 172.21.15.119:0/3864156399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f847c06f4b0 0x7f847c152420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 >> 172.21.15.119:0/3864156399 conn(0x7f847c0cc1e0 msgr2=0x7f847c0c2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:49.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 shutdown_connections 2023-10-08T00:55:49.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.515+0000 7f848099d700 1 -- 172.21.15.119:0/3864156399 wait complete. 2023-10-08T00:55:49.536 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:55:49.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.723+0000 7fe092d9d700 1 Processor -- start 2023-10-08T00:55:49.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.724+0000 7fe092d9d700 1 -- start start 2023-10-08T00:55:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe092d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c0c9db0 0x7fe08c153900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe092d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe08c157130 0x7fe08c0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe092d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe08c159ad0 0x7fe08c159f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe092d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c06e150 con 0x7fe08c157130 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe092d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c153e40 con 0x7fe08c159ad0 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe092d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c0c7800 con 0x7fe08c0c9db0 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe09159a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe08c157130 0x7fe08c0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe09159a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe08c157130 0x7fe08c0c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41704/0 (socket says 172.21.15.119:41704) 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe09159a700 1 -- 172.21.15.119:0/2727685964 learned_addr learned my addr 172.21.15.119:0/2727685964 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe091d9b700 1 --2- 172.21.15.119:0/2727685964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c0c9db0 0x7fe08c153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.725+0000 7fe09259c700 1 --2- 172.21.15.119:0/2727685964 >> v2:172.21.15.184:3300/0 conn(0x7fe08c159ad0 0x7fe08c159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe09159a700 1 -- 172.21.15.119:0/2727685964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c0c9db0 msgr2=0x7fe08c153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe09159a700 1 --2- 172.21.15.119:0/2727685964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c0c9db0 0x7fe08c153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe09159a700 1 -- 172.21.15.119:0/2727685964 >> v2:172.21.15.184:3300/0 conn(0x7fe08c159ad0 msgr2=0x7fe08c159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe09159a700 1 --2- 172.21.15.119:0/2727685964 >> v2:172.21.15.184:3300/0 conn(0x7fe08c159ad0 0x7fe08c159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe09159a700 1 -- 172.21.15.119:0/2727685964 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0c7ae0 con 0x7fe08c157130 2023-10-08T00:55:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe09159a700 1 --2- 172.21.15.119:0/2727685964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c157130 0x7fe08c0c6f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe08c06d1e0 tx=0x7fe07c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc403b6168f770af server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe090d99700 1 -- 172.21.15.119:0/2727685964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00b640 con 0x7fe08c157130 2023-10-08T00:55:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe090d99700 1 -- 172.21.15.119:0/2727685964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00b7a0 con 0x7fe08c157130 2023-10-08T00:55:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.726+0000 7fe090d99700 1 -- 172.21.15.119:0/2727685964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00bb60 con 0x7fe08c157130 2023-10-08T00:55:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 -- 172.21.15.119:0/2727685964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c157130 msgr2=0x7fe08c0c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2727685964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c157130 0x7fe08c0c6f00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe08c06d1e0 tx=0x7fe07c009250 comp rx=0 tx=0).stop 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 -- 172.21.15.119:0/2727685964 shutdown_connections 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2727685964 >> v2:172.21.15.184:3300/0 conn(0x7fe08c159ad0 0x7fe08c159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2727685964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c0c9db0 0x7fe08c153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2727685964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c157130 0x7fe08c0c6f00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 -- 172.21.15.119:0/2727685964 >> 172.21.15.119:0/2727685964 conn(0x7fe08c0cc1e0 msgr2=0x7fe08c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 -- 172.21.15.119:0/2727685964 shutdown_connections 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.727+0000 7fe092d9d700 1 -- 172.21.15.119:0/2727685964 wait complete. 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.728+0000 7fe092d9d700 1 Processor -- start 2023-10-08T00:55:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.728+0000 7fe092d9d700 1 -- start start 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe092d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c0c9db0 0x7fe08c1476d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe092d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c157130 0x7fe08c14b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe092d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe08c159ad0 0x7fe08c148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe092d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c147c10 con 0x7fe08c0c9db0 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe092d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c148a50 con 0x7fe08c159ad0 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe092d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c14be50 con 0x7fe08c157130 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe091d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c0c9db0 0x7fe08c1476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe091d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c0c9db0 0x7fe08c1476d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41710/0 (socket says 172.21.15.119:41710) 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.729+0000 7fe091d9b700 1 -- 172.21.15.119:0/2403112771 learned_addr learned my addr 172.21.15.119:0/2403112771 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe09259c700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe08c159ad0 0x7fe08c148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:49.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe09159a700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c157130 0x7fe08c14b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe091d9b700 1 -- 172.21.15.119:0/2403112771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c157130 msgr2=0x7fe08c14b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe091d9b700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c157130 0x7fe08c14b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe091d9b700 1 -- 172.21.15.119:0/2403112771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe08c159ad0 msgr2=0x7fe08c148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe091d9b700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe08c159ad0 0x7fe08c148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe091d9b700 1 -- 172.21.15.119:0/2403112771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c0094f0 con 0x7fe08c0c9db0 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe091d9b700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c0c9db0 0x7fe08c1476d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe08800e950 tx=0x7fe0880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe081ffb700 1 -- 172.21.15.119:0/2403112771 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800c850 con 0x7fe08c0c9db0 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe081ffb700 1 -- 172.21.15.119:0/2403112771 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08800c9b0 con 0x7fe08c0c9db0 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c147e40 con 0x7fe08c0c9db0 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe081ffb700 1 -- 172.21.15.119:0/2403112771 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800cdb0 con 0x7fe08c0c9db0 2023-10-08T00:55:49.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.730+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c1486e0 con 0x7fe08c0c9db0 2023-10-08T00:55:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.731+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c06c450 con 0x7fe08c0c9db0 2023-10-08T00:55:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.732+0000 7fe081ffb700 1 -- 172.21.15.119:0/2403112771 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe088013680 con 0x7fe08c0c9db0 2023-10-08T00:55:49.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.733+0000 7fe081ffb700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe070066f30 0x7fe0700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.733+0000 7fe081ffb700 1 -- 172.21.15.119:0/2403112771 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe088084860 con 0x7fe08c0c9db0 2023-10-08T00:55:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.733+0000 7fe09159a700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe070066f30 0x7fe0700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:49.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.734+0000 7fe09159a700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe070066f30 0x7fe0700693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe08c06d1e0 tx=0x7fe07c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:49.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.737+0000 7fe081ffb700 1 -- 172.21.15.119:0/2403112771 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe088051a30 con 0x7fe08c0c9db0 2023-10-08T00:55:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:49.972+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe08c0c7dc0 con 0x7fe08c0c9db0 2023-10-08T00:55:50.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.514+0000 7fe081ffb700 1 -- 172.21.15.119:0/2403112771 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe088055050 con 0x7fe08c0c9db0 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe070066f30 msgr2=0x7fe0700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe070066f30 0x7fe0700693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe08c06d1e0 tx=0x7fe07c017000 comp rx=0 tx=0).stop 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c0c9db0 msgr2=0x7fe08c1476d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c0c9db0 0x7fe08c1476d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe08800e950 tx=0x7fe0880091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 shutdown_connections 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe070066f30 0x7fe0700693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe08c159ad0 0x7fe08c148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe08c0c9db0 0x7fe08c1476d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 --2- 172.21.15.119:0/2403112771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe08c157130 0x7fe08c14b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 >> 172.21.15.119:0/2403112771 conn(0x7fe08c0cc1e0 msgr2=0x7fe08c0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:50.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.517+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 shutdown_connections 2023-10-08T00:55:50.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.518+0000 7fe092d9d700 1 -- 172.21.15.119:0/2403112771 wait complete. 2023-10-08T00:55:50.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:55:50.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.722+0000 7f25831ff700 1 Processor -- start 2023-10-08T00:55:50.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.722+0000 7f25831ff700 1 -- start start 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f25831ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f257c0b1850 0x7f257c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f25831ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f257c0ae680 0x7f257c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f25831ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0aeb00 0x7f257c157820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f25831ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c06fc30 con 0x7f257c0ae680 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f25831ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f257c0cc260 con 0x7f257c0b1850 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f25831ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c0cb300 con 0x7f257c0aeb00 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f257bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f257c0ae680 0x7f257c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f257bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f257c0ae680 0x7f257c1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41722/0 (socket says 172.21.15.119:41722) 2023-10-08T00:55:50.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f257bfff700 1 -- 172.21.15.119:0/2472267176 learned_addr learned my addr 172.21.15.119:0/2472267176 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f257bfff700 1 -- 172.21.15.119:0/2472267176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0aeb00 msgr2=0x7f257c157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f258179c700 1 --2- 172.21.15.119:0/2472267176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0aeb00 0x7f257c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f2580f9b700 1 --2- 172.21.15.119:0/2472267176 >> v2:172.21.15.184:3300/0 conn(0x7f257c0b1850 0x7f257c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f257bfff700 1 --2- 172.21.15.119:0/2472267176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0aeb00 0x7f257c157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f257bfff700 1 -- 172.21.15.119:0/2472267176 >> v2:172.21.15.184:3300/0 conn(0x7f257c0b1850 msgr2=0x7f257c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f257bfff700 1 --2- 172.21.15.119:0/2472267176 >> v2:172.21.15.184:3300/0 conn(0x7f257c0b1850 0x7f257c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f257bfff700 1 -- 172.21.15.119:0/2472267176 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c0cb5e0 con 0x7f257c0ae680 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f2580f9b700 1 --2- 172.21.15.119:0/2472267176 >> v2:172.21.15.184:3300/0 conn(0x7f257c0b1850 0x7f257c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.724+0000 7f258179c700 1 --2- 172.21.15.119:0/2472267176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0aeb00 0x7f257c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:50.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f257bfff700 1 --2- 172.21.15.119:0/2472267176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c1572e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f257c0641e0 tx=0x7f2568009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c519d8b4b3605c57 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:50.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f257b7fe700 1 -- 172.21.15.119:0/2472267176 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256800b820 con 0x7f257c0ae680 2023-10-08T00:55:50.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f257b7fe700 1 -- 172.21.15.119:0/2472267176 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256800b980 con 0x7f257c0ae680 2023-10-08T00:55:50.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f257b7fe700 1 -- 172.21.15.119:0/2472267176 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256800bd20 con 0x7f257c0ae680 2023-10-08T00:55:50.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f25831ff700 1 -- 172.21.15.119:0/2472267176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 msgr2=0x7f257c1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f25831ff700 1 --2- 172.21.15.119:0/2472267176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c1572e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f257c0641e0 tx=0x7f2568009250 comp rx=0 tx=0).stop 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f25831ff700 1 -- 172.21.15.119:0/2472267176 shutdown_connections 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f25831ff700 1 --2- 172.21.15.119:0/2472267176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0aeb00 0x7f257c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f25831ff700 1 --2- 172.21.15.119:0/2472267176 >> v2:172.21.15.184:3300/0 conn(0x7f257c0b1850 0x7f257c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f25831ff700 1 --2- 172.21.15.119:0/2472267176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c1572e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.725+0000 7f25831ff700 1 -- 172.21.15.119:0/2472267176 >> 172.21.15.119:0/2472267176 conn(0x7f257c06a320 msgr2=0x7f257c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.726+0000 7f25831ff700 1 -- 172.21.15.119:0/2472267176 shutdown_connections 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.726+0000 7f25831ff700 1 -- 172.21.15.119:0/2472267176 wait complete. 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.727+0000 7f25831ff700 1 Processor -- start 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.727+0000 7f25831ff700 1 -- start start 2023-10-08T00:55:50.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f25831ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f25831ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c0aeb00 0x7f257c155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f25831ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0b1850 0x7f257c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f25831ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c15d060 con 0x7f257c0ae680 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f25831ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c15d2b0 con 0x7f257c0aeb00 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f25831ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f257c15d590 con 0x7f257c0b1850 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f2580f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f2580f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c0b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41736/0 (socket says 172.21.15.119:41736) 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f2580f9b700 1 -- 172.21.15.119:0/797223678 learned_addr learned my addr 172.21.15.119:0/797223678 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f257bfff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c0aeb00 0x7f257c155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.728+0000 7f258179c700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0b1850 0x7f257c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:50.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2580f9b700 1 -- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0b1850 msgr2=0x7f257c155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2580f9b700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0b1850 0x7f257c155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2580f9b700 1 -- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c0aeb00 msgr2=0x7f257c155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2580f9b700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c0aeb00 0x7f257c155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2580f9b700 1 -- 172.21.15.119:0/797223678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25680094f0 con 0x7f257c0ae680 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f258179c700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0b1850 0x7f257c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2580f9b700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c0b0980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f257000c830 tx=0x7f2570008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2578ff9700 1 -- 172.21.15.119:0/797223678 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570011600 con 0x7f257c0ae680 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2578ff9700 1 -- 172.21.15.119:0/797223678 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2570011760 con 0x7f257c0ae680 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c15d870 con 0x7f257c0ae680 2023-10-08T00:55:50.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f2578ff9700 1 -- 172.21.15.119:0/797223678 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570011b20 con 0x7f257c0ae680 2023-10-08T00:55:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.729+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c15de30 con 0x7f257c0ae680 2023-10-08T00:55:50.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.730+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c169be0 con 0x7f257c0ae680 2023-10-08T00:55:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.731+0000 7f2578ff9700 1 -- 172.21.15.119:0/797223678 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25700193b0 con 0x7f257c0ae680 2023-10-08T00:55:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.732+0000 7f2578ff9700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2560066f30 0x7f25600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.732+0000 7f2578ff9700 1 -- 172.21.15.119:0/797223678 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2570084530 con 0x7f257c0ae680 2023-10-08T00:55:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.732+0000 7f257bfff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2560066f30 0x7f25600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:50.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.732+0000 7f257bfff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2560066f30 0x7f25600693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f257c0641e0 tx=0x7f2568017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:50.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.735+0000 7f2578ff9700 1 -- 172.21.15.119:0/797223678 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2570051780 con 0x7f257c0ae680 2023-10-08T00:55:50.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.971+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f257c04c3c0 con 0x7f257c0ae680 2023-10-08T00:55:50.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.972+0000 7f2578ff9700 1 -- 172.21.15.119:0/797223678 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f2570054da0 con 0x7f257c0ae680 2023-10-08T00:55:50.973 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:50.973 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:55:46.965967+0000","modified":"2023-10-08T00:55:50.512168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:55:50.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2560066f30 msgr2=0x7f25600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:50.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2560066f30 0x7f25600693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f257c0641e0 tx=0x7f2568017000 comp rx=0 tx=0).stop 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 msgr2=0x7f257c0b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c0b0980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f257000c830 tx=0x7f2570008040 comp rx=0 tx=0).stop 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 shutdown_connections 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f257c0b1850 0x7f257c155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2560066f30 0x7f25600693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f257c0ae680 0x7f257c0b0980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 --2- 172.21.15.119:0/797223678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f257c0aeb00 0x7f257c155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 >> 172.21.15.119:0/797223678 conn(0x7f257c06a320 msgr2=0x7f257c0ab650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:50.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 shutdown_connections 2023-10-08T00:55:50.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:50.974+0000 7f25831ff700 1 -- 172.21.15.119:0/797223678 wait complete. 2023-10-08T00:55:50.977 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-10-08T00:55:50.992 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:51.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.172+0000 7f2045fff700 1 Processor -- start 2023-10-08T00:55:51.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.172+0000 7f2045fff700 1 -- start start 2023-10-08T00:55:51.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.173+0000 7f2045fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f20401578b0 0x7f2040155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.173+0000 7f2045fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20400af2f0 0x7f20400af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.173+0000 7f2045fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20400ad930 con 0x7f2040155c10 2023-10-08T00:55:51.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.173+0000 7f2045fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20400b0c80 con 0x7f20401578b0 2023-10-08T00:55:51.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.173+0000 7f2045fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20400acd20 con 0x7f20400af2f0 2023-10-08T00:55:51.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203f7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2040155c10 0x7f20400ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35296/0 (socket says 172.21.15.119:35296) 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203f7fe700 1 -- 172.21.15.119:0/3850889847 learned_addr learned my addr 172.21.15.119:0/3850889847 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525655066 0 0) 0x7f20400ad930 con 0x7f2040155c10 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2028004680 con 0x7f2040155c10 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203effd700 1 --2- 172.21.15.119:0/3850889847 >> v2:172.21.15.184:3300/0 conn(0x7f20401578b0 0x7f2040155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203ffff700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20400af2f0 0x7f20400af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3189067521 0 0) 0x7f2028004680 con 0x7f2040155c10 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20400af2f0 msgr2=0x7f20400af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20400af2f0 0x7f20400af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 >> v2:172.21.15.184:3300/0 conn(0x7f20401578b0 msgr2=0x7f2040155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 --2- 172.21.15.119:0/3850889847 >> v2:172.21.15.184:3300/0 conn(0x7f20401578b0 0x7f2040155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20400ad000 con 0x7f2040155c10 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f2030001470 con 0x7f2040155c10 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203effd700 1 --2- 172.21.15.119:0/3850889847 >> v2:172.21.15.184:3300/0 conn(0x7f20401578b0 0x7f2040155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 >> v1:172.21.15.119:6789/0 conn(0x7f2040155c10 legacy=0x7f20400ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2028004bf0 0x7f2028005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.174+0000 7f203e7fc700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20400af2f0 0x7f2028005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203e7fc700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2028005e60 0x7f20280062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400b0c80 con 0x7f2028005e60 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400acd20 con 0x7f2028004bf0 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2028003770 con 0x7f20400af2f0 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203f7fe700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2028004bf0 0x7f2028005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203ffff700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2028005e60 0x7f20280062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203effd700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20400af2f0 0x7f2028005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203f7fe700 1 -- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20400af2f0 msgr2=0x7f2028005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203f7fe700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f20400af2f0 0x7f2028005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203f7fe700 1 -- 172.21.15.119:0/3850889847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2028005e60 msgr2=0x7f20280062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203f7fe700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2028005e60 0x7f20280062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203f7fe700 1 -- 172.21.15.119:0/3850889847 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2028004680 con 0x7f2028004bf0 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203ffff700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2028005e60 0x7f20280062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.175+0000 7f203f7fe700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2028004bf0 0x7f2028005000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f203000c2d0 tx=0x7f2030001dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8ffc8acfda6fb8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203000cba0 con 0x7f2028004bf0 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203000cd20 con 0x7f2028004bf0 2023-10-08T00:55:51.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f203e7fc700 1 -- 172.21.15.119:0/3850889847 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2030005b70 con 0x7f2028004bf0 2023-10-08T00:55:51.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f2045fff700 1 -- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2028004bf0 msgr2=0x7f2028005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f2045fff700 1 --2- 172.21.15.119:0/3850889847 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2028004bf0 0x7f2028005000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f203000c2d0 tx=0x7f2030001dc0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f203ffff700 1 -- 172.21.15.119:0/3850889847 reap_dead start 2023-10-08T00:55:51.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f2045fff700 1 -- 172.21.15.119:0/3850889847 shutdown_connections 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f2045fff700 1 -- 172.21.15.119:0/3850889847 >> 172.21.15.119:0/3850889847 conn(0x7f204006a320 msgr2=0x7f20400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f2045fff700 1 -- 172.21.15.119:0/3850889847 shutdown_connections 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.176+0000 7f2045fff700 1 -- 172.21.15.119:0/3850889847 wait complete. 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.178+0000 7f2045fff700 1 Processor -- start 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.178+0000 7f2045fff700 1 -- start start 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f2045fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2028004bf0 0x7f20400c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f2045fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20400c9460 0x7f20400ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f2045fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400aaa10 0x7f20400aae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f2045fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400aa580 con 0x7f20400aaa10 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f2045fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20400a9970 con 0x7f20400c9460 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f2045fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20400a9c50 con 0x7f2028004bf0 2023-10-08T00:55:51.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f203ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400aaa10 0x7f20400aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f203ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400aaa10 0x7f20400aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41768/0 (socket says 172.21.15.119:41768) 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f203ffff700 1 -- 172.21.15.119:0/576295853 learned_addr learned my addr 172.21.15.119:0/576295853 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f203effd700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20400c9460 0x7f20400ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.179+0000 7f203f7fe700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2028004bf0 0x7f20400c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f203ffff700 1 -- 172.21.15.119:0/576295853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2028004bf0 msgr2=0x7f20400c85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f203ffff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2028004bf0 0x7f20400c85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f203ffff700 1 -- 172.21.15.119:0/576295853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20400c9460 msgr2=0x7f20400ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f203ffff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20400c9460 0x7f20400ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f203ffff700 1 -- 172.21.15.119:0/576295853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20400a9f30 con 0x7f20400aaa10 2023-10-08T00:55:51.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f203ffff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400aaa10 0x7f20400aae50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2040069550 tx=0x7f2038009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:51.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f20277fe700 1 -- 172.21.15.119:0/576295853 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038009e00 con 0x7f20400aaa10 2023-10-08T00:55:51.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f20277fe700 1 -- 172.21.15.119:0/576295853 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2038015420 con 0x7f20400aaa10 2023-10-08T00:55:51.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20400aa210 con 0x7f20400aaa10 2023-10-08T00:55:51.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f20277fe700 1 -- 172.21.15.119:0/576295853 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2038015820 con 0x7f20400aaa10 2023-10-08T00:55:51.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.180+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20400caa10 con 0x7f20400aaa10 2023-10-08T00:55:51.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.181+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20400aa6f0 con 0x7f20400aaa10 2023-10-08T00:55:51.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.182+0000 7f20277fe700 1 -- 172.21.15.119:0/576295853 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2038003040 con 0x7f20400aaa10 2023-10-08T00:55:51.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.183+0000 7f20277fe700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f202c066f30 0x7f202c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.183+0000 7f20277fe700 1 -- 172.21.15.119:0/576295853 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2038084d80 con 0x7f20400aaa10 2023-10-08T00:55:51.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.183+0000 7f203f7fe700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f202c066f30 0x7f202c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.183+0000 7f203f7fe700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f202c066f30 0x7f202c0693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2030015040 tx=0x7f2030012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:51.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.187+0000 7f20277fe700 1 -- 172.21.15.119:0/576295853 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2038051f50 con 0x7f20400aaa10 2023-10-08T00:55:51.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.377+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f204004c3c0 con 0x7f20400aaa10 2023-10-08T00:55:51.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.378+0000 7f20277fe700 1 -- 172.21.15.119:0/576295853 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2038055570 con 0x7f20400aaa10 2023-10-08T00:55:51.379 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:51.380 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:51.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f202c066f30 msgr2=0x7f202c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f202c066f30 0x7f202c0693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2030015040 tx=0x7f2030012000 comp rx=0 tx=0).stop 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400aaa10 msgr2=0x7f20400aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400aaa10 0x7f20400aae50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2040069550 tx=0x7f2038009710 comp rx=0 tx=0).stop 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 shutdown_connections 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2028004bf0 0x7f20400c85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f202c066f30 0x7f202c0693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f20400aaa10 0x7f20400aae50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 --2- 172.21.15.119:0/576295853 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f20400c9460 0x7f20400ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 >> 172.21.15.119:0/576295853 conn(0x7f204006a320 msgr2=0x7f204006d920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:51.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 shutdown_connections 2023-10-08T00:55:51.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.380+0000 7f2045fff700 1 -- 172.21.15.119:0/576295853 wait complete. 2023-10-08T00:55:51.399 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:55:51.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:55:51.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.581+0000 7f30887ff700 1 Processor -- start 2023-10-08T00:55:51.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.581+0000 7f30887ff700 1 -- start start 2023-10-08T00:55:51.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.582+0000 7f30887ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3080159c60 0x7f308015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.582+0000 7f30887ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f308015db10 0x7f3080069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.582+0000 7f30887ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308015a160 con 0x7f3080159890 2023-10-08T00:55:51.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.582+0000 7f30887ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308015df50 con 0x7f3080159c60 2023-10-08T00:55:51.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.582+0000 7f30887ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30800d13a0 con 0x7f308015db10 2023-10-08T00:55:51.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.582+0000 7f3085d9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3080159c60 0x7f308015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.582+0000 7f308659b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3080159890 0x7f308015abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35312/0 (socket says 172.21.15.119:35312) 2023-10-08T00:55:51.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.582+0000 7f308659b700 1 -- 172.21.15.119:0/2340341096 learned_addr learned my addr 172.21.15.119:0/2340341096 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:51.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3086d9c700 1 --2- 172.21.15.119:0/2340341096 >> v2:172.21.15.184:3300/0 conn(0x7f308015db10 0x7f3080069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085599700 1 -- 172.21.15.119:0/2340341096 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675462232 0 0) 0x7f308015a160 con 0x7f3080159890 2023-10-08T00:55:51.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085d9a700 1 -- 172.21.15.119:0/2340341096 >> v2:172.21.15.184:3300/0 conn(0x7f308015db10 msgr2=0x7f3080069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:51.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085d9a700 1 --2- 172.21.15.119:0/2340341096 >> v2:172.21.15.184:3300/0 conn(0x7f308015db10 0x7f3080069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085d9a700 1 -- 172.21.15.119:0/2340341096 >> v1:172.21.15.119:6789/0 conn(0x7f3080159890 legacy=0x7f308015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085d9a700 1 -- 172.21.15.119:0/2340341096 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308015e140 con 0x7f3080159c60 2023-10-08T00:55:51.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3086d9c700 1 --2- 172.21.15.119:0/2340341096 >> v2:172.21.15.184:3300/0 conn(0x7f308015db10 0x7f3080069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085d9a700 1 --2- 172.21.15.119:0/2340341096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080159c60 0x7f308015d5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f308006a890 tx=0x7f30700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d04ff3dd23bf31 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085599700 1 -- 172.21.15.119:0/2340341096 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000b6b0 con 0x7f3080159c60 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085599700 1 -- 172.21.15.119:0/2340341096 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307000b810 con 0x7f3080159c60 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.583+0000 7f3085599700 1 -- 172.21.15.119:0/2340341096 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307000bbb0 con 0x7f3080159c60 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.584+0000 7f30887ff700 1 -- 172.21.15.119:0/2340341096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080159c60 msgr2=0x7f308015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.584+0000 7f30887ff700 1 --2- 172.21.15.119:0/2340341096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080159c60 0x7f308015d5d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f308006a890 tx=0x7f30700091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.584+0000 7f30887ff700 1 -- 172.21.15.119:0/2340341096 shutdown_connections 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.584+0000 7f30887ff700 1 --2- 172.21.15.119:0/2340341096 >> v2:172.21.15.184:3300/0 conn(0x7f308015db10 0x7f3080069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.584+0000 7f30887ff700 1 --2- 172.21.15.119:0/2340341096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080159c60 0x7f308015d5d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.584+0000 7f30887ff700 1 -- 172.21.15.119:0/2340341096 >> 172.21.15.119:0/2340341096 conn(0x7f3080068e20 msgr2=0x7f30800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:51.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.584+0000 7f30887ff700 1 -- 172.21.15.119:0/2340341096 shutdown_connections 2023-10-08T00:55:51.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.584+0000 7f30887ff700 1 -- 172.21.15.119:0/2340341096 wait complete. 2023-10-08T00:55:51.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.585+0000 7f30887ff700 1 Processor -- start 2023-10-08T00:55:51.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.586+0000 7f30887ff700 1 -- start start 2023-10-08T00:55:51.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.586+0000 7f30887ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080159890 0x7f308014a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.586+0000 7f30887ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3080159c60 0x7f30800c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f30887ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f308015db10 0x7f308014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f30887ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308014aa10 con 0x7f308015db10 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f30887ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308014c000 con 0x7f3080159890 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f30887ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308014bb70 con 0x7f3080159c60 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f308015db10 0x7f308014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f308015db10 0x7f308014c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41788/0 (socket says 172.21.15.119:41788) 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 -- 172.21.15.119:0/3388082043 learned_addr learned my addr 172.21.15.119:0/3388082043 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 -- 172.21.15.119:0/3388082043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3080159c60 msgr2=0x7f30800c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3080159c60 0x7f30800c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 -- 172.21.15.119:0/3388082043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080159890 msgr2=0x7f308014a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080159890 0x7f308014a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 -- 172.21.15.119:0/3388082043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3070009420 con 0x7f308015db10 2023-10-08T00:55:51.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f3086d9c700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f308015db10 0x7f308014c410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f307800cf20 tx=0x7f3078007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:51.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.587+0000 7f30767fc700 1 -- 172.21.15.119:0/3388082043 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307800adf0 con 0x7f308015db10 2023-10-08T00:55:51.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.588+0000 7f30767fc700 1 -- 172.21.15.119:0/3388082043 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3078016420 con 0x7f308015db10 2023-10-08T00:55:51.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.588+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308014af60 con 0x7f308015db10 2023-10-08T00:55:51.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.588+0000 7f30767fc700 1 -- 172.21.15.119:0/3388082043 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30780167e0 con 0x7f308015db10 2023-10-08T00:55:51.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.588+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308014b520 con 0x7f308015db10 2023-10-08T00:55:51.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.589+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308004c3c0 con 0x7f308015db10 2023-10-08T00:55:51.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.589+0000 7f30767fc700 1 -- 172.21.15.119:0/3388082043 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3078016bd0 con 0x7f308015db10 2023-10-08T00:55:51.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.590+0000 7f30767fc700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3064066f30 0x7f30640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:51.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.590+0000 7f30767fc700 1 -- 172.21.15.119:0/3388082043 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3078012070 con 0x7f308015db10 2023-10-08T00:55:51.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.590+0000 7f308659b700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3064066f30 0x7f30640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:51.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.591+0000 7f308659b700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3064066f30 0x7f30640693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f307c0011e0 tx=0x7f307c0079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:51.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.594+0000 7f30767fc700 1 -- 172.21.15.119:0/3388082043 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f307804e5a0 con 0x7f308015db10 2023-10-08T00:55:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.781+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f308014bce0 con 0x7f3064066f30 2023-10-08T00:55:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.790+0000 7f30767fc700 1 -- 172.21.15.119:0/3388082043 <== mgr.4101 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+253982 (secure 0 0 0) 0x7f308014bce0 con 0x7f3064066f30 2023-10-08T00:55:51.795 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:51.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3064066f30 msgr2=0x7f30640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3064066f30 0x7f30640693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f307c0011e0 tx=0x7f307c0079d0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f308015db10 msgr2=0x7f308014c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:51.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f308015db10 0x7f308014c410 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f307800cf20 tx=0x7f3078007860 comp rx=0 tx=0).stop 2023-10-08T00:55:51.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 shutdown_connections 2023-10-08T00:55:51.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3064066f30 0x7f30640693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f308015db10 0x7f308014c410 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080159890 0x7f308014a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 --2- 172.21.15.119:0/3388082043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3080159c60 0x7f30800c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:51.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 >> 172.21.15.119:0/3388082043 conn(0x7f3080068e20 msgr2=0x7f3080155560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:51.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 shutdown_connections 2023-10-08T00:55:51.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:51.794+0000 7f30887ff700 1 -- 172.21.15.119:0/3388082043 wait complete. 2023-10-08T00:55:51.798 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T00:55:51.812 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-10-08T00:55:49.858491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380776,"kb_used_data":2472,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593944,"statfs":{"total":773094113280,"available":770656198656,"internally_reserved":0,"allocated":2531328,"data_stored":1591456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.381160"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.483742+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.484076+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.482800+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.481585+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:45.097939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.483970+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:37.847976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.480840+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:37.847976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.484359+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:45.097939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.486920+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.481834+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.487435+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:45.097939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.482351+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.481109+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:45.097939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.483542+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:45.097939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.488479+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.481575+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.491092+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:45.097939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.485952+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:37.847976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.480686+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.486843+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:37.847976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.491125+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.487867+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":160,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-10-08T00:55:39.754356+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T00:55:39.754356+0000","last_peered":"2023-10-08T00:55:39.754356+0000","last_clean":"2023-10-08T00:55:39.754356+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T00:55:39.754356+0000","last_undegraded":"2023-10-08T00:55:39.754356+0000","last_fullsized":"2023-10-08T00:55:39.754356+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.482983+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.482167+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.480721+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:07:45.097939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.480840+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.483485+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.480756+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.480768+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.482116+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.491071+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.480594+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:37.847976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T00:55:45.471023+0000","last_change":"2023-10-08T00:55:45.490933+0000","last_active":"2023-10-08T00:55:45.471023+0000","last_peered":"2023-10-08T00:55:45.471023+0000","last_clean":"2023-10-08T00:55:45.471023+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:45.471023+0000","last_undegraded":"2023-10-08T00:55:45.471023+0000","last_fullsized":"2023-10-08T00:55:45.471023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T00:55:46.477666+0000","last_change":"2023-10-08T00:55:46.477666+0000","last_active":"2023-10-08T00:55:46.477666+0000","last_peered":"2023-10-08T00:55:46.477666+0000","last_clean":"2023-10-08T00:55:46.477666+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T00:55:46.477666+0000","last_undegraded":"2023-10-08T00:55:46.477666+0000","last_fullsized":"2023-10-08T00:55:46.477666+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":641708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0249999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297452,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074388,"statfs":{"total":96636764160,"available":96332173312,"internally_reserved":0,"allocated":167936,"data_stored":51340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.87}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297452,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074388,"statfs":{"total":96636764160,"available":96332173312,"internally_reserved":0,"allocated":167936,"data_stored":51340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298032,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073808,"statfs":{"total":96636764160,"available":96331579392,"internally_reserved":0,"allocated":761856,"data_stored":641708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297452,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074388,"statfs":{"total":96636764160,"available":96332173312,"internally_reserved":0,"allocated":167936,"data_stored":51340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297452,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074388,"statfs":{"total":96636764160,"available":96332173312,"internally_reserved":0,"allocated":167936,"data_stored":51340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0189999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.04}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297452,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074388,"statfs":{"total":96636764160,"available":96332173312,"internally_reserved":0,"allocated":167936,"data_stored":51340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0980000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297452,"kb_used_data":164,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074388,"statfs":{"total":96636764160,"available":96332173312,"internally_reserved":0,"allocated":167936,"data_stored":51340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T00:55:51.820 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:55:52.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.033+0000 7ff5d1dff700 1 Processor -- start 2023-10-08T00:55:52.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.033+0000 7ff5d1dff700 1 -- start start 2023-10-08T00:55:52.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.034+0000 7ff5d1dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5cc06bcd0 0x7ff5cc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.034+0000 7ff5d1dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5cc15df50 0x7ff5cc15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.034+0000 7ff5d1dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5cc15d7e0 con 0x7ff5cc06c0a0 2023-10-08T00:55:52.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.034+0000 7ff5d1dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc15dac0 con 0x7ff5cc15df50 2023-10-08T00:55:52.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.034+0000 7ff5d1dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5cc15aa30 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5cc15df50 0x7ff5cc15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5cc15df50 0x7ff5cc15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41796/0 (socket says 172.21.15.119:41796) 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cbfff700 1 -- 172.21.15.119:0/2953617518 learned_addr learned my addr 172.21.15.119:0/2953617518 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cb7fe700 1 --2- 172.21.15.119:0/2953617518 >> v2:172.21.15.184:3300/0 conn(0x7ff5cc06bcd0 0x7ff5cc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cbfff700 1 -- 172.21.15.119:0/2953617518 >> v2:172.21.15.184:3300/0 conn(0x7ff5cc06bcd0 msgr2=0x7ff5cc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cbfff700 1 --2- 172.21.15.119:0/2953617518 >> v2:172.21.15.184:3300/0 conn(0x7ff5cc06bcd0 0x7ff5cc0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cbfff700 1 -- 172.21.15.119:0/2953617518 >> v1:172.21.15.184:6789/0 conn(0x7ff5cc06c0a0 legacy=0x7ff5cc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cbfff700 1 -- 172.21.15.119:0/2953617518 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5cc15ad10 con 0x7ff5cc15df50 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.035+0000 7ff5cbfff700 1 --2- 172.21.15.119:0/2953617518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc15df50 0x7ff5cc15e3b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0691a0 tx=0x7ff5c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcf038a2caa69f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5ca7fc700 1 -- 172.21.15.119:0/2953617518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400e070 con 0x7ff5cc15df50 2023-10-08T00:55:52.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5ca7fc700 1 -- 172.21.15.119:0/2953617518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c400ba50 con 0x7ff5cc15df50 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5ca7fc700 1 -- 172.21.15.119:0/2953617518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c40092e0 con 0x7ff5cc15df50 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2953617518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc15df50 msgr2=0x7ff5cc15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2953617518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc15df50 0x7ff5cc15e3b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc0691a0 tx=0x7ff5c4009840 comp rx=0 tx=0).stop 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2953617518 shutdown_connections 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2953617518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc15df50 0x7ff5cc15e3b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2953617518 >> v2:172.21.15.184:3300/0 conn(0x7ff5cc06bcd0 0x7ff5cc0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2953617518 >> 172.21.15.119:0/2953617518 conn(0x7ff5cc0ce600 msgr2=0x7ff5cc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2953617518 shutdown_connections 2023-10-08T00:55:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.036+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2953617518 wait complete. 2023-10-08T00:55:52.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.038+0000 7ff5d1dff700 1 Processor -- start 2023-10-08T00:55:52.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.038+0000 7ff5d1dff700 1 -- start start 2023-10-08T00:55:52.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5d1dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc06bcd0 0x7ff5cc1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5d1dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5cc06c0a0 0x7ff5cc16de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5d1dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5cc15df50 0x7ff5cc153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5d1dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc169e60 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5d1dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc16a6c0 con 0x7ff5cc15df50 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5d1dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5cc154e90 con 0x7ff5cc06c0a0 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc06bcd0 0x7ff5cc1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc06bcd0 0x7ff5cc1548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41808/0 (socket says 172.21.15.119:41808) 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cb7fe700 1 -- 172.21.15.119:0/2185334792 learned_addr learned my addr 172.21.15.119:0/2185334792 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cb7fe700 1 -- 172.21.15.119:0/2185334792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5cc06c0a0 msgr2=0x7ff5cc16de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cb7fe700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5cc06c0a0 0x7ff5cc16de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cbfff700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5cc15df50 0x7ff5cc153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cb7fe700 1 -- 172.21.15.119:0/2185334792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5cc15df50 msgr2=0x7ff5cc153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cb7fe700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5cc15df50 0x7ff5cc153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cb7fe700 1 -- 172.21.15.119:0/2185334792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c4009ae0 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.039+0000 7ff5cbfff700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5cc15df50 0x7ff5cc153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:52.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.040+0000 7ff5cb7fe700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc06bcd0 0x7ff5cc1548b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00d860 tx=0x7ff5bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.040+0000 7ff5b37fe700 1 -- 172.21.15.119:0/2185334792 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc009610 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.040+0000 7ff5b37fe700 1 -- 172.21.15.119:0/2185334792 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc009770 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.040+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5cc16a070 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.040+0000 7ff5b37fe700 1 -- 172.21.15.119:0/2185334792 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc009b30 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.040+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5cc153690 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.041+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5cc04c3c0 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.042+0000 7ff5b37fe700 1 -- 172.21.15.119:0/2185334792 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff5bc01e460 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.042+0000 7ff5b37fe700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff5b8066f30 0x7ff5b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.043+0000 7ff5b37fe700 1 -- 172.21.15.119:0/2185334792 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff5bc084bf0 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.043+0000 7ff5caffd700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff5b8066f30 0x7ff5b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.043+0000 7ff5caffd700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff5b8066f30 0x7ff5b80693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc153f40 tx=0x7ff5c0007b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.047+0000 7ff5b37fe700 1 -- 172.21.15.119:0/2185334792 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5bc051dc0 con 0x7ff5cc06bcd0 2023-10-08T00:55:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.237+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5cc16a800 con 0x7ff5b8066f30 2023-10-08T00:55:52.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.246+0000 7ff5b37fe700 1 -- 172.21.15.119:0/2185334792 <== mgr.4101 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264631 (secure 0 0 0) 0x7ff5c0014410 con 0x7ff5b8066f30 2023-10-08T00:55:52.251 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff5b8066f30 msgr2=0x7ff5b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff5b8066f30 0x7ff5b80693e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc153f40 tx=0x7ff5c0007b40 comp rx=0 tx=0).stop 2023-10-08T00:55:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc06bcd0 msgr2=0x7ff5cc1548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc06bcd0 0x7ff5cc1548b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00d860 tx=0x7ff5bc008040 comp rx=0 tx=0).stop 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 shutdown_connections 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff5b8066f30 0x7ff5b80693e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5cc15df50 0x7ff5cc153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5cc06bcd0 0x7ff5cc1548b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 --2- 172.21.15.119:0/2185334792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5cc06c0a0 0x7ff5cc16de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 >> 172.21.15.119:0/2185334792 conn(0x7ff5cc0ce600 msgr2=0x7ff5cc0c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 shutdown_connections 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.250+0000 7ff5d1dff700 1 -- 172.21.15.119:0/2185334792 wait complete. 2023-10-08T00:55:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T00:55:52.269 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-08T00:55:51.858811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381128,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593592,"statfs":{"total":773094113280,"available":770655838208,"internally_reserved":0,"allocated":2891776,"data_stored":1750580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000906"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524928+0000","last_change":"2023-10-08T00:55:47.519961+0000","last_active":"2023-10-08T00:55:47.524928+0000","last_peered":"2023-10-08T00:55:47.524928+0000","last_clean":"2023-10-08T00:55:47.524928+0000","last_became_active":"2023-10-08T00:55:47.519667+0000","last_became_peered":"2023-10-08T00:55:47.519667+0000","last_unstale":"2023-10-08T00:55:47.524928+0000","last_undegraded":"2023-10-08T00:55:47.524928+0000","last_fullsized":"2023-10-08T00:55:47.524928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:20.644250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520418+0000","last_change":"2023-10-08T00:55:47.516100+0000","last_active":"2023-10-08T00:55:47.520418+0000","last_peered":"2023-10-08T00:55:47.520418+0000","last_clean":"2023-10-08T00:55:47.520418+0000","last_became_active":"2023-10-08T00:55:47.515088+0000","last_became_peered":"2023-10-08T00:55:47.515088+0000","last_unstale":"2023-10-08T00:55:47.520418+0000","last_undegraded":"2023-10-08T00:55:47.520418+0000","last_fullsized":"2023-10-08T00:55:47.520418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523625+0000","last_change":"2023-10-08T00:55:47.514661+0000","last_active":"2023-10-08T00:55:47.523625+0000","last_peered":"2023-10-08T00:55:47.523625+0000","last_clean":"2023-10-08T00:55:47.523625+0000","last_became_active":"2023-10-08T00:55:47.512991+0000","last_became_peered":"2023-10-08T00:55:47.512991+0000","last_unstale":"2023-10-08T00:55:47.523625+0000","last_undegraded":"2023-10-08T00:55:47.523625+0000","last_fullsized":"2023-10-08T00:55:47.523625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:25.085209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525503+0000","last_change":"2023-10-08T00:55:47.520755+0000","last_active":"2023-10-08T00:55:47.525503+0000","last_peered":"2023-10-08T00:55:47.525503+0000","last_clean":"2023-10-08T00:55:47.525503+0000","last_became_active":"2023-10-08T00:55:47.519925+0000","last_became_peered":"2023-10-08T00:55:47.519925+0000","last_unstale":"2023-10-08T00:55:47.525503+0000","last_undegraded":"2023-10-08T00:55:47.525503+0000","last_fullsized":"2023-10-08T00:55:47.525503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.513000+0000","last_change":"2023-10-08T00:55:47.510024+0000","last_active":"2023-10-08T00:55:47.513000+0000","last_peered":"2023-10-08T00:55:47.513000+0000","last_clean":"2023-10-08T00:55:47.513000+0000","last_became_active":"2023-10-08T00:55:47.509649+0000","last_became_peered":"2023-10-08T00:55:47.509649+0000","last_unstale":"2023-10-08T00:55:47.513000+0000","last_undegraded":"2023-10-08T00:55:47.513000+0000","last_fullsized":"2023-10-08T00:55:47.513000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525212+0000","last_change":"2023-10-08T00:55:47.514675+0000","last_active":"2023-10-08T00:55:47.525212+0000","last_peered":"2023-10-08T00:55:47.525212+0000","last_clean":"2023-10-08T00:55:47.525212+0000","last_became_active":"2023-10-08T00:55:47.514414+0000","last_became_peered":"2023-10-08T00:55:47.514414+0000","last_unstale":"2023-10-08T00:55:47.525212+0000","last_undegraded":"2023-10-08T00:55:47.525212+0000","last_fullsized":"2023-10-08T00:55:47.525212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:07.821288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524753+0000","last_change":"2023-10-08T00:55:47.516013+0000","last_active":"2023-10-08T00:55:47.524753+0000","last_peered":"2023-10-08T00:55:47.524753+0000","last_clean":"2023-10-08T00:55:47.524753+0000","last_became_active":"2023-10-08T00:55:47.515695+0000","last_became_peered":"2023-10-08T00:55:47.515695+0000","last_unstale":"2023-10-08T00:55:47.524753+0000","last_undegraded":"2023-10-08T00:55:47.524753+0000","last_fullsized":"2023-10-08T00:55:47.524753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:44.607367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.527135+0000","last_change":"2023-10-08T00:55:47.525615+0000","last_active":"2023-10-08T00:55:47.527135+0000","last_peered":"2023-10-08T00:55:47.527135+0000","last_clean":"2023-10-08T00:55:47.527135+0000","last_became_active":"2023-10-08T00:55:47.525281+0000","last_became_peered":"2023-10-08T00:55:47.525281+0000","last_unstale":"2023-10-08T00:55:47.527135+0000","last_undegraded":"2023-10-08T00:55:47.527135+0000","last_fullsized":"2023-10-08T00:55:47.527135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522070+0000","last_change":"2023-10-08T00:55:47.514405+0000","last_active":"2023-10-08T00:55:47.522070+0000","last_peered":"2023-10-08T00:55:47.522070+0000","last_clean":"2023-10-08T00:55:47.522070+0000","last_became_active":"2023-10-08T00:55:47.513815+0000","last_became_peered":"2023-10-08T00:55:47.513815+0000","last_unstale":"2023-10-08T00:55:47.522070+0000","last_undegraded":"2023-10-08T00:55:47.522070+0000","last_fullsized":"2023-10-08T00:55:47.522070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:34.078412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524336+0000","last_change":"2023-10-08T00:55:47.518070+0000","last_active":"2023-10-08T00:55:47.524336+0000","last_peered":"2023-10-08T00:55:47.524336+0000","last_clean":"2023-10-08T00:55:47.524336+0000","last_became_active":"2023-10-08T00:55:47.517802+0000","last_became_peered":"2023-10-08T00:55:47.517802+0000","last_unstale":"2023-10-08T00:55:47.524336+0000","last_undegraded":"2023-10-08T00:55:47.524336+0000","last_fullsized":"2023-10-08T00:55:47.524336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518986+0000","last_change":"2023-10-08T00:55:47.517157+0000","last_active":"2023-10-08T00:55:47.518986+0000","last_peered":"2023-10-08T00:55:47.518986+0000","last_clean":"2023-10-08T00:55:47.518986+0000","last_became_active":"2023-10-08T00:55:47.516876+0000","last_became_peered":"2023-10-08T00:55:47.516876+0000","last_unstale":"2023-10-08T00:55:47.518986+0000","last_undegraded":"2023-10-08T00:55:47.518986+0000","last_fullsized":"2023-10-08T00:55:47.518986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533808+0000","last_change":"2023-10-08T00:55:47.520192+0000","last_active":"2023-10-08T00:55:47.533808+0000","last_peered":"2023-10-08T00:55:47.533808+0000","last_clean":"2023-10-08T00:55:47.533808+0000","last_became_active":"2023-10-08T00:55:47.519911+0000","last_became_peered":"2023-10-08T00:55:47.519911+0000","last_unstale":"2023-10-08T00:55:47.533808+0000","last_undegraded":"2023-10-08T00:55:47.533808+0000","last_fullsized":"2023-10-08T00:55:47.533808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:44.817925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526111+0000","last_change":"2023-10-08T00:55:47.522124+0000","last_active":"2023-10-08T00:55:47.526111+0000","last_peered":"2023-10-08T00:55:47.526111+0000","last_clean":"2023-10-08T00:55:47.526111+0000","last_became_active":"2023-10-08T00:55:47.521242+0000","last_became_peered":"2023-10-08T00:55:47.521242+0000","last_unstale":"2023-10-08T00:55:47.526111+0000","last_undegraded":"2023-10-08T00:55:47.526111+0000","last_fullsized":"2023-10-08T00:55:47.526111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520534+0000","last_change":"2023-10-08T00:55:47.508213+0000","last_active":"2023-10-08T00:55:47.520534+0000","last_peered":"2023-10-08T00:55:47.520534+0000","last_clean":"2023-10-08T00:55:47.520534+0000","last_became_active":"2023-10-08T00:55:47.507914+0000","last_became_peered":"2023-10-08T00:55:47.507914+0000","last_unstale":"2023-10-08T00:55:47.520534+0000","last_undegraded":"2023-10-08T00:55:47.520534+0000","last_fullsized":"2023-10-08T00:55:47.520534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.510943+0000","last_change":"2023-10-08T00:55:47.504306+0000","last_active":"2023-10-08T00:55:47.510943+0000","last_peered":"2023-10-08T00:55:47.510943+0000","last_clean":"2023-10-08T00:55:47.510943+0000","last_became_active":"2023-10-08T00:55:47.504037+0000","last_became_peered":"2023-10-08T00:55:47.504037+0000","last_unstale":"2023-10-08T00:55:47.510943+0000","last_undegraded":"2023-10-08T00:55:47.510943+0000","last_fullsized":"2023-10-08T00:55:47.510943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530847+0000","last_change":"2023-10-08T00:55:47.524003+0000","last_active":"2023-10-08T00:55:47.530847+0000","last_peered":"2023-10-08T00:55:47.530847+0000","last_clean":"2023-10-08T00:55:47.530847+0000","last_became_active":"2023-10-08T00:55:47.521541+0000","last_became_peered":"2023-10-08T00:55:47.521541+0000","last_unstale":"2023-10-08T00:55:47.530847+0000","last_undegraded":"2023-10-08T00:55:47.530847+0000","last_fullsized":"2023-10-08T00:55:47.530847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:44.607367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.511787+0000","last_change":"2023-10-08T00:55:47.509224+0000","last_active":"2023-10-08T00:55:47.511787+0000","last_peered":"2023-10-08T00:55:47.511787+0000","last_clean":"2023-10-08T00:55:47.511787+0000","last_became_active":"2023-10-08T00:55:47.508714+0000","last_became_peered":"2023-10-08T00:55:47.508714+0000","last_unstale":"2023-10-08T00:55:47.511787+0000","last_undegraded":"2023-10-08T00:55:47.511787+0000","last_fullsized":"2023-10-08T00:55:47.511787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525712+0000","last_change":"2023-10-08T00:55:47.519330+0000","last_active":"2023-10-08T00:55:47.525712+0000","last_peered":"2023-10-08T00:55:47.525712+0000","last_clean":"2023-10-08T00:55:47.525712+0000","last_became_active":"2023-10-08T00:55:47.519070+0000","last_became_peered":"2023-10-08T00:55:47.519070+0000","last_unstale":"2023-10-08T00:55:47.525712+0000","last_undegraded":"2023-10-08T00:55:47.525712+0000","last_fullsized":"2023-10-08T00:55:47.525712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:03.776133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.514578+0000","last_change":"2023-10-08T00:55:47.509945+0000","last_active":"2023-10-08T00:55:47.514578+0000","last_peered":"2023-10-08T00:55:47.514578+0000","last_clean":"2023-10-08T00:55:47.514578+0000","last_became_active":"2023-10-08T00:55:47.509596+0000","last_became_peered":"2023-10-08T00:55:47.509596+0000","last_unstale":"2023-10-08T00:55:47.514578+0000","last_undegraded":"2023-10-08T00:55:47.514578+0000","last_fullsized":"2023-10-08T00:55:47.514578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:07.510837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525580+0000","last_change":"2023-10-08T00:55:46.514258+0000","last_active":"2023-10-08T00:55:47.525580+0000","last_peered":"2023-10-08T00:55:47.525580+0000","last_clean":"2023-10-08T00:55:47.525580+0000","last_became_active":"2023-10-08T00:55:46.514002+0000","last_became_peered":"2023-10-08T00:55:46.514002+0000","last_unstale":"2023-10-08T00:55:47.525580+0000","last_undegraded":"2023-10-08T00:55:47.525580+0000","last_fullsized":"2023-10-08T00:55:47.525580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520841+0000","last_change":"2023-10-08T00:55:47.519479+0000","last_active":"2023-10-08T00:55:47.520841+0000","last_peered":"2023-10-08T00:55:47.520841+0000","last_clean":"2023-10-08T00:55:47.520841+0000","last_became_active":"2023-10-08T00:55:47.519179+0000","last_became_peered":"2023-10-08T00:55:47.519179+0000","last_unstale":"2023-10-08T00:55:47.520841+0000","last_undegraded":"2023-10-08T00:55:47.520841+0000","last_fullsized":"2023-10-08T00:55:47.520841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523749+0000","last_change":"2023-10-08T00:55:46.507226+0000","last_active":"2023-10-08T00:55:47.523749+0000","last_peered":"2023-10-08T00:55:47.523749+0000","last_clean":"2023-10-08T00:55:47.523749+0000","last_became_active":"2023-10-08T00:55:46.506878+0000","last_became_peered":"2023-10-08T00:55:46.506878+0000","last_unstale":"2023-10-08T00:55:47.523749+0000","last_undegraded":"2023-10-08T00:55:47.523749+0000","last_fullsized":"2023-10-08T00:55:47.523749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523694+0000","last_change":"2023-10-08T00:55:47.514717+0000","last_active":"2023-10-08T00:55:47.523694+0000","last_peered":"2023-10-08T00:55:47.523694+0000","last_clean":"2023-10-08T00:55:47.523694+0000","last_became_active":"2023-10-08T00:55:47.514339+0000","last_became_peered":"2023-10-08T00:55:47.514339+0000","last_unstale":"2023-10-08T00:55:47.523694+0000","last_undegraded":"2023-10-08T00:55:47.523694+0000","last_fullsized":"2023-10-08T00:55:47.523694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522336+0000","last_change":"2023-10-08T00:55:46.517327+0000","last_active":"2023-10-08T00:55:47.522336+0000","last_peered":"2023-10-08T00:55:47.522336+0000","last_clean":"2023-10-08T00:55:47.522336+0000","last_became_active":"2023-10-08T00:55:46.516908+0000","last_became_peered":"2023-10-08T00:55:46.516908+0000","last_unstale":"2023-10-08T00:55:47.522336+0000","last_undegraded":"2023-10-08T00:55:47.522336+0000","last_fullsized":"2023-10-08T00:55:47.522336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525482+0000","last_change":"2023-10-08T00:55:47.520563+0000","last_active":"2023-10-08T00:55:47.525482+0000","last_peered":"2023-10-08T00:55:47.525482+0000","last_clean":"2023-10-08T00:55:47.525482+0000","last_became_active":"2023-10-08T00:55:47.519545+0000","last_became_peered":"2023-10-08T00:55:47.519545+0000","last_unstale":"2023-10-08T00:55:47.525482+0000","last_undegraded":"2023-10-08T00:55:47.525482+0000","last_fullsized":"2023-10-08T00:55:47.525482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.528020+0000","last_change":"2023-10-08T00:55:46.521160+0000","last_active":"2023-10-08T00:55:47.528020+0000","last_peered":"2023-10-08T00:55:47.528020+0000","last_clean":"2023-10-08T00:55:47.528020+0000","last_became_active":"2023-10-08T00:55:46.520820+0000","last_became_peered":"2023-10-08T00:55:46.520820+0000","last_unstale":"2023-10-08T00:55:47.528020+0000","last_undegraded":"2023-10-08T00:55:47.528020+0000","last_fullsized":"2023-10-08T00:55:47.528020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:24.078567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533581+0000","last_change":"2023-10-08T00:55:47.526348+0000","last_active":"2023-10-08T00:55:47.533581+0000","last_peered":"2023-10-08T00:55:47.533581+0000","last_clean":"2023-10-08T00:55:47.533581+0000","last_became_active":"2023-10-08T00:55:47.526167+0000","last_became_peered":"2023-10-08T00:55:47.526167+0000","last_unstale":"2023-10-08T00:55:47.533581+0000","last_undegraded":"2023-10-08T00:55:47.533581+0000","last_fullsized":"2023-10-08T00:55:47.533581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522448+0000","last_change":"2023-10-08T00:55:46.505995+0000","last_active":"2023-10-08T00:55:47.522448+0000","last_peered":"2023-10-08T00:55:47.522448+0000","last_clean":"2023-10-08T00:55:47.522448+0000","last_became_active":"2023-10-08T00:55:46.505673+0000","last_became_peered":"2023-10-08T00:55:46.505673+0000","last_unstale":"2023-10-08T00:55:47.522448+0000","last_undegraded":"2023-10-08T00:55:47.522448+0000","last_fullsized":"2023-10-08T00:55:47.522448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533980+0000","last_change":"2023-10-08T00:55:47.525941+0000","last_active":"2023-10-08T00:55:47.533980+0000","last_peered":"2023-10-08T00:55:47.533980+0000","last_clean":"2023-10-08T00:55:47.533980+0000","last_became_active":"2023-10-08T00:55:47.524635+0000","last_became_peered":"2023-10-08T00:55:47.524635+0000","last_unstale":"2023-10-08T00:55:47.533980+0000","last_undegraded":"2023-10-08T00:55:47.533980+0000","last_fullsized":"2023-10-08T00:55:47.533980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:09.261793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522850+0000","last_change":"2023-10-08T00:55:46.506810+0000","last_active":"2023-10-08T00:55:47.522850+0000","last_peered":"2023-10-08T00:55:47.522850+0000","last_clean":"2023-10-08T00:55:47.522850+0000","last_became_active":"2023-10-08T00:55:46.506504+0000","last_became_peered":"2023-10-08T00:55:46.506504+0000","last_unstale":"2023-10-08T00:55:47.522850+0000","last_undegraded":"2023-10-08T00:55:47.522850+0000","last_fullsized":"2023-10-08T00:55:47.522850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524500+0000","last_change":"2023-10-08T00:55:47.516584+0000","last_active":"2023-10-08T00:55:47.524500+0000","last_peered":"2023-10-08T00:55:47.524500+0000","last_clean":"2023-10-08T00:55:47.524500+0000","last_became_active":"2023-10-08T00:55:47.516222+0000","last_became_peered":"2023-10-08T00:55:47.516222+0000","last_unstale":"2023-10-08T00:55:47.524500+0000","last_undegraded":"2023-10-08T00:55:47.524500+0000","last_fullsized":"2023-10-08T00:55:47.524500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.506266+0000","last_change":"2023-10-08T00:55:46.499483+0000","last_active":"2023-10-08T00:55:47.506266+0000","last_peered":"2023-10-08T00:55:47.506266+0000","last_clean":"2023-10-08T00:55:47.506266+0000","last_became_active":"2023-10-08T00:55:46.499153+0000","last_became_peered":"2023-10-08T00:55:46.499153+0000","last_unstale":"2023-10-08T00:55:47.506266+0000","last_undegraded":"2023-10-08T00:55:47.506266+0000","last_fullsized":"2023-10-08T00:55:47.506266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525986+0000","last_change":"2023-10-08T00:55:47.521897+0000","last_active":"2023-10-08T00:55:47.525986+0000","last_peered":"2023-10-08T00:55:47.525986+0000","last_clean":"2023-10-08T00:55:47.525986+0000","last_became_active":"2023-10-08T00:55:47.521630+0000","last_became_peered":"2023-10-08T00:55:47.521630+0000","last_unstale":"2023-10-08T00:55:47.525986+0000","last_undegraded":"2023-10-08T00:55:47.525986+0000","last_fullsized":"2023-10-08T00:55:47.525986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.502490+0000","last_change":"2023-10-08T00:55:46.520040+0000","last_active":"2023-10-08T00:55:47.502490+0000","last_peered":"2023-10-08T00:55:47.502490+0000","last_clean":"2023-10-08T00:55:47.502490+0000","last_became_active":"2023-10-08T00:55:46.519780+0000","last_became_peered":"2023-10-08T00:55:46.519780+0000","last_unstale":"2023-10-08T00:55:47.502490+0000","last_undegraded":"2023-10-08T00:55:47.502490+0000","last_fullsized":"2023-10-08T00:55:47.502490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523199+0000","last_change":"2023-10-08T00:55:47.518074+0000","last_active":"2023-10-08T00:55:47.523199+0000","last_peered":"2023-10-08T00:55:47.523199+0000","last_clean":"2023-10-08T00:55:47.523199+0000","last_became_active":"2023-10-08T00:55:47.517822+0000","last_became_peered":"2023-10-08T00:55:47.517822+0000","last_unstale":"2023-10-08T00:55:47.523199+0000","last_undegraded":"2023-10-08T00:55:47.523199+0000","last_fullsized":"2023-10-08T00:55:47.523199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507109+0000","last_change":"2023-10-08T00:55:46.520607+0000","last_active":"2023-10-08T00:55:47.507109+0000","last_peered":"2023-10-08T00:55:47.507109+0000","last_clean":"2023-10-08T00:55:47.507109+0000","last_became_active":"2023-10-08T00:55:46.520340+0000","last_became_peered":"2023-10-08T00:55:46.520340+0000","last_unstale":"2023-10-08T00:55:47.507109+0000","last_undegraded":"2023-10-08T00:55:47.507109+0000","last_fullsized":"2023-10-08T00:55:47.507109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.531004+0000","last_change":"2023-10-08T00:55:47.524779+0000","last_active":"2023-10-08T00:55:47.531004+0000","last_peered":"2023-10-08T00:55:47.531004+0000","last_clean":"2023-10-08T00:55:47.531004+0000","last_became_active":"2023-10-08T00:55:47.524520+0000","last_became_peered":"2023-10-08T00:55:47.524520+0000","last_unstale":"2023-10-08T00:55:47.531004+0000","last_undegraded":"2023-10-08T00:55:47.531004+0000","last_fullsized":"2023-10-08T00:55:47.531004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.505478+0000","last_change":"2023-10-08T00:55:46.520234+0000","last_active":"2023-10-08T00:55:47.505478+0000","last_peered":"2023-10-08T00:55:47.505478+0000","last_clean":"2023-10-08T00:55:47.505478+0000","last_became_active":"2023-10-08T00:55:46.519769+0000","last_became_peered":"2023-10-08T00:55:46.519769+0000","last_unstale":"2023-10-08T00:55:47.505478+0000","last_undegraded":"2023-10-08T00:55:47.505478+0000","last_fullsized":"2023-10-08T00:55:47.505478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523717+0000","last_change":"2023-10-08T00:55:47.514712+0000","last_active":"2023-10-08T00:55:47.523717+0000","last_peered":"2023-10-08T00:55:47.523717+0000","last_clean":"2023-10-08T00:55:47.523717+0000","last_became_active":"2023-10-08T00:55:47.514326+0000","last_became_peered":"2023-10-08T00:55:47.514326+0000","last_unstale":"2023-10-08T00:55:47.523717+0000","last_undegraded":"2023-10-08T00:55:47.523717+0000","last_fullsized":"2023-10-08T00:55:47.523717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.513999+0000","last_change":"2023-10-08T00:55:46.520938+0000","last_active":"2023-10-08T00:55:47.513999+0000","last_peered":"2023-10-08T00:55:47.513999+0000","last_clean":"2023-10-08T00:55:47.513999+0000","last_became_active":"2023-10-08T00:55:46.520707+0000","last_became_peered":"2023-10-08T00:55:46.520707+0000","last_unstale":"2023-10-08T00:55:47.513999+0000","last_undegraded":"2023-10-08T00:55:47.513999+0000","last_fullsized":"2023-10-08T00:55:47.513999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:43.811282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.534039+0000","last_change":"2023-10-08T00:55:47.527280+0000","last_active":"2023-10-08T00:55:47.534039+0000","last_peered":"2023-10-08T00:55:47.534039+0000","last_clean":"2023-10-08T00:55:47.534039+0000","last_became_active":"2023-10-08T00:55:47.526962+0000","last_became_peered":"2023-10-08T00:55:47.526962+0000","last_unstale":"2023-10-08T00:55:47.534039+0000","last_undegraded":"2023-10-08T00:55:47.534039+0000","last_fullsized":"2023-10-08T00:55:47.534039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:03.776133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.529949+0000","last_change":"2023-10-08T00:55:46.496099+0000","last_active":"2023-10-08T00:55:47.529949+0000","last_peered":"2023-10-08T00:55:47.529949+0000","last_clean":"2023-10-08T00:55:47.529949+0000","last_became_active":"2023-10-08T00:55:46.495752+0000","last_became_peered":"2023-10-08T00:55:46.495752+0000","last_unstale":"2023-10-08T00:55:47.529949+0000","last_undegraded":"2023-10-08T00:55:47.529949+0000","last_fullsized":"2023-10-08T00:55:47.529949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524959+0000","last_change":"2023-10-08T00:55:47.521795+0000","last_active":"2023-10-08T00:55:47.524959+0000","last_peered":"2023-10-08T00:55:47.524959+0000","last_clean":"2023-10-08T00:55:47.524959+0000","last_became_active":"2023-10-08T00:55:47.521477+0000","last_became_peered":"2023-10-08T00:55:47.521477+0000","last_unstale":"2023-10-08T00:55:47.524959+0000","last_undegraded":"2023-10-08T00:55:47.524959+0000","last_fullsized":"2023-10-08T00:55:47.524959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:02.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520109+0000","last_change":"2023-10-08T00:55:46.509869+0000","last_active":"2023-10-08T00:55:47.520109+0000","last_peered":"2023-10-08T00:55:47.520109+0000","last_clean":"2023-10-08T00:55:47.520109+0000","last_became_active":"2023-10-08T00:55:46.509581+0000","last_became_peered":"2023-10-08T00:55:46.509581+0000","last_unstale":"2023-10-08T00:55:47.520109+0000","last_undegraded":"2023-10-08T00:55:47.520109+0000","last_fullsized":"2023-10-08T00:55:47.520109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.521324+0000","last_change":"2023-10-08T00:55:47.512931+0000","last_active":"2023-10-08T00:55:47.521324+0000","last_peered":"2023-10-08T00:55:47.521324+0000","last_clean":"2023-10-08T00:55:47.521324+0000","last_became_active":"2023-10-08T00:55:47.512674+0000","last_became_peered":"2023-10-08T00:55:47.512674+0000","last_unstale":"2023-10-08T00:55:47.521324+0000","last_undegraded":"2023-10-08T00:55:47.521324+0000","last_fullsized":"2023-10-08T00:55:47.521324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.501686+0000","last_change":"2023-10-08T00:55:46.519403+0000","last_active":"2023-10-08T00:55:47.501686+0000","last_peered":"2023-10-08T00:55:47.501686+0000","last_clean":"2023-10-08T00:55:47.501686+0000","last_became_active":"2023-10-08T00:55:46.516356+0000","last_became_peered":"2023-10-08T00:55:46.516356+0000","last_unstale":"2023-10-08T00:55:47.501686+0000","last_undegraded":"2023-10-08T00:55:47.501686+0000","last_fullsized":"2023-10-08T00:55:47.501686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507923+0000","last_change":"2023-10-08T00:55:47.504128+0000","last_active":"2023-10-08T00:55:47.507923+0000","last_peered":"2023-10-08T00:55:47.507923+0000","last_clean":"2023-10-08T00:55:47.507923+0000","last_became_active":"2023-10-08T00:55:47.503735+0000","last_became_peered":"2023-10-08T00:55:47.503735+0000","last_unstale":"2023-10-08T00:55:47.507923+0000","last_undegraded":"2023-10-08T00:55:47.507923+0000","last_fullsized":"2023-10-08T00:55:47.507923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.509141+0000","last_change":"2023-10-08T00:55:46.506608+0000","last_active":"2023-10-08T00:55:47.509141+0000","last_peered":"2023-10-08T00:55:47.509141+0000","last_clean":"2023-10-08T00:55:47.509141+0000","last_became_active":"2023-10-08T00:55:46.506233+0000","last_became_peered":"2023-10-08T00:55:46.506233+0000","last_unstale":"2023-10-08T00:55:47.509141+0000","last_undegraded":"2023-10-08T00:55:47.509141+0000","last_fullsized":"2023-10-08T00:55:47.509141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526384+0000","last_change":"2023-10-08T00:55:47.522727+0000","last_active":"2023-10-08T00:55:47.526384+0000","last_peered":"2023-10-08T00:55:47.526384+0000","last_clean":"2023-10-08T00:55:47.526384+0000","last_became_active":"2023-10-08T00:55:47.522019+0000","last_became_peered":"2023-10-08T00:55:47.522019+0000","last_unstale":"2023-10-08T00:55:47.526384+0000","last_undegraded":"2023-10-08T00:55:47.526384+0000","last_fullsized":"2023-10-08T00:55:47.526384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520506+0000","last_change":"2023-10-08T00:55:46.506201+0000","last_active":"2023-10-08T00:55:47.520506+0000","last_peered":"2023-10-08T00:55:47.520506+0000","last_clean":"2023-10-08T00:55:47.520506+0000","last_became_active":"2023-10-08T00:55:46.505555+0000","last_became_peered":"2023-10-08T00:55:46.505555+0000","last_unstale":"2023-10-08T00:55:47.520506+0000","last_undegraded":"2023-10-08T00:55:47.520506+0000","last_fullsized":"2023-10-08T00:55:47.520506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533925+0000","last_change":"2023-10-08T00:55:47.528839+0000","last_active":"2023-10-08T00:55:47.533925+0000","last_peered":"2023-10-08T00:55:47.533925+0000","last_clean":"2023-10-08T00:55:47.533925+0000","last_became_active":"2023-10-08T00:55:47.528573+0000","last_became_peered":"2023-10-08T00:55:47.528573+0000","last_unstale":"2023-10-08T00:55:47.533925+0000","last_undegraded":"2023-10-08T00:55:47.533925+0000","last_fullsized":"2023-10-08T00:55:47.533925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:57:57.391083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.517881+0000","last_change":"2023-10-08T00:55:46.506125+0000","last_active":"2023-10-08T00:55:47.517881+0000","last_peered":"2023-10-08T00:55:47.517881+0000","last_clean":"2023-10-08T00:55:47.517881+0000","last_became_active":"2023-10-08T00:55:46.505815+0000","last_became_peered":"2023-10-08T00:55:46.505815+0000","last_unstale":"2023-10-08T00:55:47.517881+0000","last_undegraded":"2023-10-08T00:55:47.517881+0000","last_fullsized":"2023-10-08T00:55:47.517881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533963+0000","last_change":"2023-10-08T00:55:47.529148+0000","last_active":"2023-10-08T00:55:47.533963+0000","last_peered":"2023-10-08T00:55:47.533963+0000","last_clean":"2023-10-08T00:55:47.533963+0000","last_became_active":"2023-10-08T00:55:47.528897+0000","last_became_peered":"2023-10-08T00:55:47.528897+0000","last_unstale":"2023-10-08T00:55:47.533963+0000","last_undegraded":"2023-10-08T00:55:47.533963+0000","last_fullsized":"2023-10-08T00:55:47.533963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492782+0000","last_change":"2023-10-08T00:55:46.507942+0000","last_active":"2023-10-08T00:55:47.492782+0000","last_peered":"2023-10-08T00:55:47.492782+0000","last_clean":"2023-10-08T00:55:47.492782+0000","last_became_active":"2023-10-08T00:55:46.507685+0000","last_became_peered":"2023-10-08T00:55:46.507685+0000","last_unstale":"2023-10-08T00:55:47.492782+0000","last_undegraded":"2023-10-08T00:55:47.492782+0000","last_fullsized":"2023-10-08T00:55:47.492782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530557+0000","last_change":"2023-10-08T00:55:47.521486+0000","last_active":"2023-10-08T00:55:47.530557+0000","last_peered":"2023-10-08T00:55:47.530557+0000","last_clean":"2023-10-08T00:55:47.530557+0000","last_became_active":"2023-10-08T00:55:47.521230+0000","last_became_peered":"2023-10-08T00:55:47.521230+0000","last_unstale":"2023-10-08T00:55:47.530557+0000","last_undegraded":"2023-10-08T00:55:47.530557+0000","last_fullsized":"2023-10-08T00:55:47.530557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:34.078412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.501573+0000","last_change":"2023-10-08T00:55:46.519867+0000","last_active":"2023-10-08T00:55:47.501573+0000","last_peered":"2023-10-08T00:55:47.501573+0000","last_clean":"2023-10-08T00:55:47.501573+0000","last_became_active":"2023-10-08T00:55:46.519268+0000","last_became_peered":"2023-10-08T00:55:46.519268+0000","last_unstale":"2023-10-08T00:55:47.501573+0000","last_undegraded":"2023-10-08T00:55:47.501573+0000","last_fullsized":"2023-10-08T00:55:47.501573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530667+0000","last_change":"2023-10-08T00:55:47.517018+0000","last_active":"2023-10-08T00:55:47.530667+0000","last_peered":"2023-10-08T00:55:47.530667+0000","last_clean":"2023-10-08T00:55:47.530667+0000","last_became_active":"2023-10-08T00:55:47.516713+0000","last_became_peered":"2023-10-08T00:55:47.516713+0000","last_unstale":"2023-10-08T00:55:47.530667+0000","last_undegraded":"2023-10-08T00:55:47.530667+0000","last_fullsized":"2023-10-08T00:55:47.530667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:40.798392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507081+0000","last_change":"2023-10-08T00:55:47.500984+0000","last_active":"2023-10-08T00:55:47.507081+0000","last_peered":"2023-10-08T00:55:47.507081+0000","last_clean":"2023-10-08T00:55:47.507081+0000","last_became_active":"2023-10-08T00:55:47.499299+0000","last_became_peered":"2023-10-08T00:55:47.499299+0000","last_unstale":"2023-10-08T00:55:47.507081+0000","last_undegraded":"2023-10-08T00:55:47.507081+0000","last_fullsized":"2023-10-08T00:55:47.507081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.494532+0000","last_change":"2023-10-08T00:55:46.503917+0000","last_active":"2023-10-08T00:55:47.494532+0000","last_peered":"2023-10-08T00:55:47.494532+0000","last_clean":"2023-10-08T00:55:47.494532+0000","last_became_active":"2023-10-08T00:55:46.503620+0000","last_became_peered":"2023-10-08T00:55:46.503620+0000","last_unstale":"2023-10-08T00:55:47.494532+0000","last_undegraded":"2023-10-08T00:55:47.494532+0000","last_fullsized":"2023-10-08T00:55:47.494532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.527104+0000","last_change":"2023-10-08T00:55:47.525523+0000","last_active":"2023-10-08T00:55:47.527104+0000","last_peered":"2023-10-08T00:55:47.527104+0000","last_clean":"2023-10-08T00:55:47.527104+0000","last_became_active":"2023-10-08T00:55:47.525238+0000","last_became_peered":"2023-10-08T00:55:47.525238+0000","last_unstale":"2023-10-08T00:55:47.527104+0000","last_undegraded":"2023-10-08T00:55:47.527104+0000","last_fullsized":"2023-10-08T00:55:47.527104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.498288+0000","last_change":"2023-10-08T00:55:46.516320+0000","last_active":"2023-10-08T00:55:47.498288+0000","last_peered":"2023-10-08T00:55:47.498288+0000","last_clean":"2023-10-08T00:55:47.498288+0000","last_became_active":"2023-10-08T00:55:46.516011+0000","last_became_peered":"2023-10-08T00:55:46.516011+0000","last_unstale":"2023-10-08T00:55:47.498288+0000","last_undegraded":"2023-10-08T00:55:47.498288+0000","last_fullsized":"2023-10-08T00:55:47.498288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493566+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T00:55:47.493566+0000","last_peered":"2023-10-08T00:55:47.493566+0000","last_clean":"2023-10-08T00:55:47.493566+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T00:55:47.493566+0000","last_undegraded":"2023-10-08T00:55:47.493566+0000","last_fullsized":"2023-10-08T00:55:47.493566+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524595+0000","last_change":"2023-10-08T00:55:47.518197+0000","last_active":"2023-10-08T00:55:47.524595+0000","last_peered":"2023-10-08T00:55:47.524595+0000","last_clean":"2023-10-08T00:55:47.524595+0000","last_became_active":"2023-10-08T00:55:47.517929+0000","last_became_peered":"2023-10-08T00:55:47.517929+0000","last_unstale":"2023-10-08T00:55:47.524595+0000","last_undegraded":"2023-10-08T00:55:47.524595+0000","last_fullsized":"2023-10-08T00:55:47.524595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:55.386688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493737+0000","last_change":"2023-10-08T00:55:46.513277+0000","last_active":"2023-10-08T00:55:47.493737+0000","last_peered":"2023-10-08T00:55:47.493737+0000","last_clean":"2023-10-08T00:55:47.493737+0000","last_became_active":"2023-10-08T00:55:46.513027+0000","last_became_peered":"2023-10-08T00:55:46.513027+0000","last_unstale":"2023-10-08T00:55:47.493737+0000","last_undegraded":"2023-10-08T00:55:47.493737+0000","last_fullsized":"2023-10-08T00:55:47.493737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:43.811282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.506236+0000","last_change":"2023-10-08T00:55:46.518643+0000","last_active":"2023-10-08T00:55:47.506236+0000","last_peered":"2023-10-08T00:55:47.506236+0000","last_clean":"2023-10-08T00:55:47.506236+0000","last_became_active":"2023-10-08T00:55:46.518305+0000","last_became_peered":"2023-10-08T00:55:46.518305+0000","last_unstale":"2023-10-08T00:55:47.506236+0000","last_undegraded":"2023-10-08T00:55:47.506236+0000","last_fullsized":"2023-10-08T00:55:47.506236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.499384+0000","last_change":"2023-10-08T00:55:47.498680+0000","last_active":"2023-10-08T00:55:47.499384+0000","last_peered":"2023-10-08T00:55:47.499384+0000","last_clean":"2023-10-08T00:55:47.499384+0000","last_became_active":"2023-10-08T00:55:47.498370+0000","last_became_peered":"2023-10-08T00:55:47.498370+0000","last_unstale":"2023-10-08T00:55:47.499384+0000","last_undegraded":"2023-10-08T00:55:47.499384+0000","last_fullsized":"2023-10-08T00:55:47.499384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492862+0000","last_change":"2023-10-08T00:55:46.508309+0000","last_active":"2023-10-08T00:55:47.492862+0000","last_peered":"2023-10-08T00:55:47.492862+0000","last_clean":"2023-10-08T00:55:47.492862+0000","last_became_active":"2023-10-08T00:55:46.507338+0000","last_became_peered":"2023-10-08T00:55:46.507338+0000","last_unstale":"2023-10-08T00:55:47.492862+0000","last_undegraded":"2023-10-08T00:55:47.492862+0000","last_fullsized":"2023-10-08T00:55:47.492862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518720+0000","last_change":"2023-10-08T00:55:47.516628+0000","last_active":"2023-10-08T00:55:47.518720+0000","last_peered":"2023-10-08T00:55:47.518720+0000","last_clean":"2023-10-08T00:55:47.518720+0000","last_became_active":"2023-10-08T00:55:47.516317+0000","last_became_peered":"2023-10-08T00:55:47.516317+0000","last_unstale":"2023-10-08T00:55:47.518720+0000","last_undegraded":"2023-10-08T00:55:47.518720+0000","last_fullsized":"2023-10-08T00:55:47.518720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:07.510837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526183+0000","last_change":"2023-10-08T00:55:46.511031+0000","last_active":"2023-10-08T00:55:47.526183+0000","last_peered":"2023-10-08T00:55:47.526183+0000","last_clean":"2023-10-08T00:55:47.526183+0000","last_became_active":"2023-10-08T00:55:46.510736+0000","last_became_peered":"2023-10-08T00:55:46.510736+0000","last_unstale":"2023-10-08T00:55:47.526183+0000","last_undegraded":"2023-10-08T00:55:47.526183+0000","last_fullsized":"2023-10-08T00:55:47.526183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520686+0000","last_change":"2023-10-08T00:55:47.513619+0000","last_active":"2023-10-08T00:55:47.520686+0000","last_peered":"2023-10-08T00:55:47.520686+0000","last_clean":"2023-10-08T00:55:47.520686+0000","last_became_active":"2023-10-08T00:55:47.513202+0000","last_became_peered":"2023-10-08T00:55:47.513202+0000","last_unstale":"2023-10-08T00:55:47.520686+0000","last_undegraded":"2023-10-08T00:55:47.520686+0000","last_fullsized":"2023-10-08T00:55:47.520686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493466+0000","last_change":"2023-10-08T00:55:46.517881+0000","last_active":"2023-10-08T00:55:47.493466+0000","last_peered":"2023-10-08T00:55:47.493466+0000","last_clean":"2023-10-08T00:55:47.493466+0000","last_became_active":"2023-10-08T00:55:46.517717+0000","last_became_peered":"2023-10-08T00:55:46.517717+0000","last_unstale":"2023-10-08T00:55:47.493466+0000","last_undegraded":"2023-10-08T00:55:47.493466+0000","last_fullsized":"2023-10-08T00:55:47.493466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:15.507854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518670+0000","last_change":"2023-10-08T00:55:47.516448+0000","last_active":"2023-10-08T00:55:47.518670+0000","last_peered":"2023-10-08T00:55:47.518670+0000","last_clean":"2023-10-08T00:55:47.518670+0000","last_became_active":"2023-10-08T00:55:47.516204+0000","last_became_peered":"2023-10-08T00:55:47.516204+0000","last_unstale":"2023-10-08T00:55:47.518670+0000","last_undegraded":"2023-10-08T00:55:47.518670+0000","last_fullsized":"2023-10-08T00:55:47.518670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:16.514497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492576+0000","last_change":"2023-10-08T00:55:46.513481+0000","last_active":"2023-10-08T00:55:47.492576+0000","last_peered":"2023-10-08T00:55:47.492576+0000","last_clean":"2023-10-08T00:55:47.492576+0000","last_became_active":"2023-10-08T00:55:46.513158+0000","last_became_peered":"2023-10-08T00:55:46.513158+0000","last_unstale":"2023-10-08T00:55:47.492576+0000","last_undegraded":"2023-10-08T00:55:47.492576+0000","last_fullsized":"2023-10-08T00:55:47.492576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524756+0000","last_change":"2023-10-08T00:55:47.520913+0000","last_active":"2023-10-08T00:55:47.524756+0000","last_peered":"2023-10-08T00:55:47.524756+0000","last_clean":"2023-10-08T00:55:47.524756+0000","last_became_active":"2023-10-08T00:55:47.520660+0000","last_became_peered":"2023-10-08T00:55:47.520660+0000","last_unstale":"2023-10-08T00:55:47.524756+0000","last_undegraded":"2023-10-08T00:55:47.524756+0000","last_fullsized":"2023-10-08T00:55:47.524756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492624+0000","last_change":"2023-10-08T00:55:46.508048+0000","last_active":"2023-10-08T00:55:47.492624+0000","last_peered":"2023-10-08T00:55:47.492624+0000","last_clean":"2023-10-08T00:55:47.492624+0000","last_became_active":"2023-10-08T00:55:46.507778+0000","last_became_peered":"2023-10-08T00:55:46.507778+0000","last_unstale":"2023-10-08T00:55:47.492624+0000","last_undegraded":"2023-10-08T00:55:47.492624+0000","last_fullsized":"2023-10-08T00:55:47.492624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526130+0000","last_change":"2023-10-08T00:55:47.524176+0000","last_active":"2023-10-08T00:55:47.526130+0000","last_peered":"2023-10-08T00:55:47.526130+0000","last_clean":"2023-10-08T00:55:47.526130+0000","last_became_active":"2023-10-08T00:55:47.523813+0000","last_became_peered":"2023-10-08T00:55:47.523813+0000","last_unstale":"2023-10-08T00:55:47.526130+0000","last_undegraded":"2023-10-08T00:55:47.526130+0000","last_fullsized":"2023-10-08T00:55:47.526130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.534398+0000","last_change":"2023-10-08T00:55:46.514845+0000","last_active":"2023-10-08T00:55:47.534398+0000","last_peered":"2023-10-08T00:55:47.534398+0000","last_clean":"2023-10-08T00:55:47.534398+0000","last_became_active":"2023-10-08T00:55:46.514581+0000","last_became_peered":"2023-10-08T00:55:46.514581+0000","last_unstale":"2023-10-08T00:55:47.534398+0000","last_undegraded":"2023-10-08T00:55:47.534398+0000","last_fullsized":"2023-10-08T00:55:47.534398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:02.769491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526325+0000","last_change":"2023-10-08T00:55:47.522621+0000","last_active":"2023-10-08T00:55:47.526325+0000","last_peered":"2023-10-08T00:55:47.526325+0000","last_clean":"2023-10-08T00:55:47.526325+0000","last_became_active":"2023-10-08T00:55:47.522352+0000","last_became_peered":"2023-10-08T00:55:47.522352+0000","last_unstale":"2023-10-08T00:55:47.526325+0000","last_undegraded":"2023-10-08T00:55:47.526325+0000","last_fullsized":"2023-10-08T00:55:47.526325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.494485+0000","last_change":"2023-10-08T00:55:46.520580+0000","last_active":"2023-10-08T00:55:47.494485+0000","last_peered":"2023-10-08T00:55:47.494485+0000","last_clean":"2023-10-08T00:55:47.494485+0000","last_became_active":"2023-10-08T00:55:46.520260+0000","last_became_peered":"2023-10-08T00:55:46.520260+0000","last_unstale":"2023-10-08T00:55:47.494485+0000","last_undegraded":"2023-10-08T00:55:47.494485+0000","last_fullsized":"2023-10-08T00:55:47.494485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524951+0000","last_change":"2023-10-08T00:55:47.519077+0000","last_active":"2023-10-08T00:55:47.524951+0000","last_peered":"2023-10-08T00:55:47.524951+0000","last_clean":"2023-10-08T00:55:47.524951+0000","last_became_active":"2023-10-08T00:55:47.518580+0000","last_became_peered":"2023-10-08T00:55:47.518580+0000","last_unstale":"2023-10-08T00:55:47.524951+0000","last_undegraded":"2023-10-08T00:55:47.524951+0000","last_fullsized":"2023-10-08T00:55:47.524951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:14.159608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526450+0000","last_change":"2023-10-08T00:55:46.514711+0000","last_active":"2023-10-08T00:55:47.526450+0000","last_peered":"2023-10-08T00:55:47.526450+0000","last_clean":"2023-10-08T00:55:47.526450+0000","last_became_active":"2023-10-08T00:55:46.514420+0000","last_became_peered":"2023-10-08T00:55:46.514420+0000","last_unstale":"2023-10-08T00:55:47.526450+0000","last_undegraded":"2023-10-08T00:55:47.526450+0000","last_fullsized":"2023-10-08T00:55:47.526450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:15.507854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525520+0000","last_change":"2023-10-08T00:55:47.520974+0000","last_active":"2023-10-08T00:55:47.525520+0000","last_peered":"2023-10-08T00:55:47.525520+0000","last_clean":"2023-10-08T00:55:47.525520+0000","last_became_active":"2023-10-08T00:55:47.520259+0000","last_became_peered":"2023-10-08T00:55:47.520259+0000","last_unstale":"2023-10-08T00:55:47.525520+0000","last_undegraded":"2023-10-08T00:55:47.525520+0000","last_fullsized":"2023-10-08T00:55:47.525520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:57:57.391083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.510639+0000","last_change":"2023-10-08T00:55:46.500260+0000","last_active":"2023-10-08T00:55:47.510639+0000","last_peered":"2023-10-08T00:55:47.510639+0000","last_clean":"2023-10-08T00:55:47.510639+0000","last_became_active":"2023-10-08T00:55:46.499965+0000","last_became_peered":"2023-10-08T00:55:46.499965+0000","last_unstale":"2023-10-08T00:55:47.510639+0000","last_undegraded":"2023-10-08T00:55:47.510639+0000","last_fullsized":"2023-10-08T00:55:47.510639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533942+0000","last_change":"2023-10-08T00:55:47.529127+0000","last_active":"2023-10-08T00:55:47.533942+0000","last_peered":"2023-10-08T00:55:47.533942+0000","last_clean":"2023-10-08T00:55:47.533942+0000","last_became_active":"2023-10-08T00:55:47.528809+0000","last_became_peered":"2023-10-08T00:55:47.528809+0000","last_unstale":"2023-10-08T00:55:47.533942+0000","last_undegraded":"2023-10-08T00:55:47.533942+0000","last_fullsized":"2023-10-08T00:55:47.533942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:07.821288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526404+0000","last_change":"2023-10-08T00:55:47.522934+0000","last_active":"2023-10-08T00:55:47.526404+0000","last_peered":"2023-10-08T00:55:47.526404+0000","last_clean":"2023-10-08T00:55:47.526404+0000","last_became_active":"2023-10-08T00:55:47.522385+0000","last_became_peered":"2023-10-08T00:55:47.522385+0000","last_unstale":"2023-10-08T00:55:47.526404+0000","last_undegraded":"2023-10-08T00:55:47.526404+0000","last_fullsized":"2023-10-08T00:55:47.526404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.515109+0000","last_change":"2023-10-08T00:55:47.508997+0000","last_active":"2023-10-08T00:55:47.515109+0000","last_peered":"2023-10-08T00:55:47.515109+0000","last_clean":"2023-10-08T00:55:47.515109+0000","last_became_active":"2023-10-08T00:55:47.508483+0000","last_became_peered":"2023-10-08T00:55:47.508483+0000","last_unstale":"2023-10-08T00:55:47.515109+0000","last_undegraded":"2023-10-08T00:55:47.515109+0000","last_fullsized":"2023-10-08T00:55:47.515109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:25.085209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525515+0000","last_change":"2023-10-08T00:55:47.520643+0000","last_active":"2023-10-08T00:55:47.525515+0000","last_peered":"2023-10-08T00:55:47.525515+0000","last_clean":"2023-10-08T00:55:47.525515+0000","last_became_active":"2023-10-08T00:55:47.520385+0000","last_became_peered":"2023-10-08T00:55:47.520385+0000","last_unstale":"2023-10-08T00:55:47.525515+0000","last_undegraded":"2023-10-08T00:55:47.525515+0000","last_fullsized":"2023-10-08T00:55:47.525515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:02.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524523+0000","last_change":"2023-10-08T00:55:47.517588+0000","last_active":"2023-10-08T00:55:47.524523+0000","last_peered":"2023-10-08T00:55:47.524523+0000","last_clean":"2023-10-08T00:55:47.524523+0000","last_became_active":"2023-10-08T00:55:47.517257+0000","last_became_peered":"2023-10-08T00:55:47.517257+0000","last_unstale":"2023-10-08T00:55:47.524523+0000","last_undegraded":"2023-10-08T00:55:47.524523+0000","last_fullsized":"2023-10-08T00:55:47.524523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536372+0000","last_change":"2023-10-08T00:55:47.534943+0000","last_active":"2023-10-08T00:55:47.536372+0000","last_peered":"2023-10-08T00:55:47.536372+0000","last_clean":"2023-10-08T00:55:47.536372+0000","last_became_active":"2023-10-08T00:55:47.534593+0000","last_became_peered":"2023-10-08T00:55:47.534593+0000","last_unstale":"2023-10-08T00:55:47.536372+0000","last_undegraded":"2023-10-08T00:55:47.536372+0000","last_fullsized":"2023-10-08T00:55:47.536372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523043+0000","last_change":"2023-10-08T00:55:47.519521+0000","last_active":"2023-10-08T00:55:47.523043+0000","last_peered":"2023-10-08T00:55:47.523043+0000","last_clean":"2023-10-08T00:55:47.523043+0000","last_became_active":"2023-10-08T00:55:47.519205+0000","last_became_peered":"2023-10-08T00:55:47.519205+0000","last_unstale":"2023-10-08T00:55:47.523043+0000","last_undegraded":"2023-10-08T00:55:47.523043+0000","last_fullsized":"2023-10-08T00:55:47.523043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.528710+0000","last_change":"2023-10-08T00:55:47.525909+0000","last_active":"2023-10-08T00:55:47.528710+0000","last_peered":"2023-10-08T00:55:47.528710+0000","last_clean":"2023-10-08T00:55:47.528710+0000","last_became_active":"2023-10-08T00:55:47.525638+0000","last_became_peered":"2023-10-08T00:55:47.525638+0000","last_unstale":"2023-10-08T00:55:47.528710+0000","last_undegraded":"2023-10-08T00:55:47.528710+0000","last_fullsized":"2023-10-08T00:55:47.528710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536365+0000","last_change":"2023-10-08T00:55:47.534909+0000","last_active":"2023-10-08T00:55:47.536365+0000","last_peered":"2023-10-08T00:55:47.536365+0000","last_clean":"2023-10-08T00:55:47.536365+0000","last_became_active":"2023-10-08T00:55:47.534654+0000","last_became_peered":"2023-10-08T00:55:47.534654+0000","last_unstale":"2023-10-08T00:55:47.536365+0000","last_undegraded":"2023-10-08T00:55:47.536365+0000","last_fullsized":"2023-10-08T00:55:47.536365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525191+0000","last_change":"2023-10-08T00:55:47.520067+0000","last_active":"2023-10-08T00:55:47.525191+0000","last_peered":"2023-10-08T00:55:47.525191+0000","last_clean":"2023-10-08T00:55:47.525191+0000","last_became_active":"2023-10-08T00:55:47.519793+0000","last_became_peered":"2023-10-08T00:55:47.519793+0000","last_unstale":"2023-10-08T00:55:47.525191+0000","last_undegraded":"2023-10-08T00:55:47.525191+0000","last_fullsized":"2023-10-08T00:55:47.525191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:34.451497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536355+0000","last_change":"2023-10-08T00:55:47.534891+0000","last_active":"2023-10-08T00:55:47.536355+0000","last_peered":"2023-10-08T00:55:47.536355+0000","last_clean":"2023-10-08T00:55:47.536355+0000","last_became_active":"2023-10-08T00:55:47.534589+0000","last_became_peered":"2023-10-08T00:55:47.534589+0000","last_unstale":"2023-10-08T00:55:47.536355+0000","last_undegraded":"2023-10-08T00:55:47.536355+0000","last_fullsized":"2023-10-08T00:55:47.536355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530986+0000","last_change":"2023-10-08T00:55:47.524426+0000","last_active":"2023-10-08T00:55:47.530986+0000","last_peered":"2023-10-08T00:55:47.530986+0000","last_clean":"2023-10-08T00:55:47.530986+0000","last_became_active":"2023-10-08T00:55:47.524142+0000","last_became_peered":"2023-10-08T00:55:47.524142+0000","last_unstale":"2023-10-08T00:55:47.530986+0000","last_undegraded":"2023-10-08T00:55:47.530986+0000","last_fullsized":"2023-10-08T00:55:47.530986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523999+0000","last_change":"2023-10-08T00:55:47.515215+0000","last_active":"2023-10-08T00:55:47.523999+0000","last_peered":"2023-10-08T00:55:47.523999+0000","last_clean":"2023-10-08T00:55:47.523999+0000","last_became_active":"2023-10-08T00:55:47.514978+0000","last_became_peered":"2023-10-08T00:55:47.514978+0000","last_unstale":"2023-10-08T00:55:47.523999+0000","last_undegraded":"2023-10-08T00:55:47.523999+0000","last_fullsized":"2023-10-08T00:55:47.523999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526430+0000","last_change":"2023-10-08T00:55:47.523078+0000","last_active":"2023-10-08T00:55:47.526430+0000","last_peered":"2023-10-08T00:55:47.526430+0000","last_clean":"2023-10-08T00:55:47.526430+0000","last_became_active":"2023-10-08T00:55:47.522818+0000","last_became_peered":"2023-10-08T00:55:47.522818+0000","last_unstale":"2023-10-08T00:55:47.526430+0000","last_undegraded":"2023-10-08T00:55:47.526430+0000","last_fullsized":"2023-10-08T00:55:47.526430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1259999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83199999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98599999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67300000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":661632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93999999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T00:55:52.274 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:55:52.274 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-08T00:55:52.275 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-08T00:55:52.275 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-08T00:55:52.275 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-08T00:55:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.488+0000 7f8f57dff700 1 Processor -- start 2023-10-08T00:55:52.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.488+0000 7f8f57dff700 1 -- start start 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f57dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f5006bcd0 0x7f8f500caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f57dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5006c0a0 0x7f8f5015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f57dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8f5015df50 0x7f8f5015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f57dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5015d7e0 con 0x7f8f5006bcd0 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f57dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5015dac0 con 0x7f8f5015df50 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f57dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5015aa30 con 0x7f8f5006c0a0 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f55b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f5006bcd0 0x7f8f500caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f55b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f5006bcd0 0x7f8f500caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41816/0 (socket says 172.21.15.119:41816) 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f55b9b700 1 -- 172.21.15.119:0/3744745888 learned_addr learned my addr 172.21.15.119:0/3744745888 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f5539a700 1 --2- 172.21.15.119:0/3744745888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5006c0a0 0x7f8f5015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f5639c700 1 --2- 172.21.15.119:0/3744745888 >> v2:172.21.15.184:3300/0 conn(0x7f8f5015df50 0x7f8f5015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f55b9b700 1 -- 172.21.15.119:0/3744745888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5006c0a0 msgr2=0x7f8f5015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f55b9b700 1 --2- 172.21.15.119:0/3744745888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5006c0a0 0x7f8f5015a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f55b9b700 1 -- 172.21.15.119:0/3744745888 >> v2:172.21.15.184:3300/0 conn(0x7f8f5015df50 msgr2=0x7f8f5015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f55b9b700 1 --2- 172.21.15.119:0/3744745888 >> v2:172.21.15.184:3300/0 conn(0x7f8f5015df50 0x7f8f5015e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f55b9b700 1 -- 172.21.15.119:0/3744745888 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5015ad10 con 0x7f8f5006bcd0 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.490+0000 7f8f5639c700 1 --2- 172.21.15.119:0/3744745888 >> v2:172.21.15.184:3300/0 conn(0x7f8f5015df50 0x7f8f5015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f55b9b700 1 --2- 172.21.15.119:0/3744745888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f500caa90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f500b4600 tx=0x7f8f4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a8dcf929e4fe93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f54b99700 1 -- 172.21.15.119:0/3744745888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c009e70 con 0x7f8f5006bcd0 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f54b99700 1 -- 172.21.15.119:0/3744745888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c00b860 con 0x7f8f5006bcd0 2023-10-08T00:55:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f54b99700 1 -- 172.21.15.119:0/3744745888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c00bc20 con 0x7f8f5006bcd0 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 -- 172.21.15.119:0/3744745888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 msgr2=0x7f8f500caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 --2- 172.21.15.119:0/3744745888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f500caa90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f500b4600 tx=0x7f8f4c009250 comp rx=0 tx=0).stop 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 -- 172.21.15.119:0/3744745888 shutdown_connections 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 --2- 172.21.15.119:0/3744745888 >> v2:172.21.15.184:3300/0 conn(0x7f8f5015df50 0x7f8f5015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 --2- 172.21.15.119:0/3744745888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f500caa90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 --2- 172.21.15.119:0/3744745888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5006c0a0 0x7f8f5015a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 -- 172.21.15.119:0/3744745888 >> 172.21.15.119:0/3744745888 conn(0x7f8f500ce600 msgr2=0x7f8f500cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 -- 172.21.15.119:0/3744745888 shutdown_connections 2023-10-08T00:55:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.491+0000 7f8f57dff700 1 -- 172.21.15.119:0/3744745888 wait complete. 2023-10-08T00:55:52.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.493+0000 7f8f57dff700 1 Processor -- start 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.493+0000 7f8f57dff700 1 -- start start 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f57dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f501555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f57dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5006c0a0 0x7f8f500b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f57dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5015df50 0x7f8f5015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f57dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500b3c60 con 0x7f8f5006bcd0 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f57dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500b40f0 con 0x7f8f5006c0a0 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f57dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f500b3050 con 0x7f8f5015df50 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f55b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f501555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f55b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f501555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41832/0 (socket says 172.21.15.119:41832) 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f55b9b700 1 -- 172.21.15.119:0/1947093314 learned_addr learned my addr 172.21.15.119:0/1947093314 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f5539a700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5006c0a0 0x7f8f500b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f5639c700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5015df50 0x7f8f5015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f55b9b700 1 -- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5015df50 msgr2=0x7f8f5015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f55b9b700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5015df50 0x7f8f5015c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f55b9b700 1 -- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5006c0a0 msgr2=0x7f8f500b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f55b9b700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5006c0a0 0x7f8f500b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f55b9b700 1 -- 172.21.15.119:0/1947093314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0094f0 con 0x7f8f5006bcd0 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f5539a700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5006c0a0 0x7f8f500b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.494+0000 7f8f5639c700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5015df50 0x7f8f5015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.495+0000 7f8f55b9b700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f501555d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0094c0 tx=0x7f8f4c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.495+0000 7f8f45ffb700 1 -- 172.21.15.119:0/1947093314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c000bc0 con 0x7f8f5006bcd0 2023-10-08T00:55:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.495+0000 7f8f45ffb700 1 -- 172.21.15.119:0/1947093314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4c00b9d0 con 0x7f8f5006bcd0 2023-10-08T00:55:52.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.495+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f500b3330 con 0x7f8f5006bcd0 2023-10-08T00:55:52.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.495+0000 7f8f45ffb700 1 -- 172.21.15.119:0/1947093314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4c025420 con 0x7f8f5006bcd0 2023-10-08T00:55:52.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.495+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f500b38f0 con 0x7f8f5006bcd0 2023-10-08T00:55:52.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.496+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5006b190 con 0x7f8f5006bcd0 2023-10-08T00:55:52.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.497+0000 7f8f45ffb700 1 -- 172.21.15.119:0/1947093314 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f4c023070 con 0x7f8f5006bcd0 2023-10-08T00:55:52.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.498+0000 7f8f45ffb700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8f34066f30 0x7f8f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.498+0000 7f8f45ffb700 1 -- 172.21.15.119:0/1947093314 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8f4c085710 con 0x7f8f5006bcd0 2023-10-08T00:55:52.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.498+0000 7f8f5539a700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8f34066f30 0x7f8f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.498+0000 7f8f5539a700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8f34066f30 0x7f8f340693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f40006f00 tx=0x7f8f40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.502+0000 7f8f45ffb700 1 -- 172.21.15.119:0/1947093314 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f4c0528e0 con 0x7f8f5006bcd0 2023-10-08T00:55:52.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.746+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f8f500cf510 con 0x7f8f5006bcd0 2023-10-08T00:55:52.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.750+0000 7f8f45ffb700 1 -- 172.21.15.119:0/1947093314 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163705 (secure 0 0 0) 0x7f8f4c009300 con 0x7f8f5006bcd0 2023-10-08T00:55:52.753 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8f34066f30 msgr2=0x7f8f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8f34066f30 0x7f8f340693e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8f40006f00 tx=0x7f8f40008040 comp rx=0 tx=0).stop 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 msgr2=0x7f8f501555d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f501555d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c0094c0 tx=0x7f8f4c017000 comp rx=0 tx=0).stop 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 shutdown_connections 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8f5015df50 0x7f8f5015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8f34066f30 0x7f8f340693e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f5006bcd0 0x7f8f501555d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 --2- 172.21.15.119:0/1947093314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8f5006c0a0 0x7f8f500b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 >> 172.21.15.119:0/1947093314 conn(0x7f8f500ce600 msgr2=0x7f8f500c3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:52.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 shutdown_connections 2023-10-08T00:55:52.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.753+0000 7f8f57dff700 1 -- 172.21.15.119:0/1947093314 wait complete. 2023-10-08T00:55:52.771 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":85511},{"type":"v1","addr":"172.21.15.119:6833","nonce":85511}]},"active_addr":"172.21.15.119:6833/85511","active_change":"2023-10-08T00:55:29.847113+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2510601793}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":4274210250}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":538579956}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1792355627}]}]} 2023-10-08T00:55:52.776 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-08T00:55:52.776 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-08T00:55:52.776 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.955+0000 7f90edb9d700 1 Processor -- start 2023-10-08T00:55:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.955+0000 7f90edb9d700 1 -- start start 2023-10-08T00:55:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.956+0000 7f90edb9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f90e81597f0 0x7f90e80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.956+0000 7f90edb9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90e815d230 0x7f90e815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.956+0000 7f90edb9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e815db70 con 0x7f90e8159420 2023-10-08T00:55:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.956+0000 7f90edb9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80b11d0 con 0x7f90e815d230 2023-10-08T00:55:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.956+0000 7f90edb9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e80b14b0 con 0x7f90e81597f0 2023-10-08T00:55:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90ed39c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90e815d230 0x7f90e815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90ed39c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90e815d230 0x7f90e815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41846/0 (socket says 172.21.15.119:41846) 2023-10-08T00:55:52.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90ed39c700 1 -- 172.21.15.119:0/588299087 learned_addr learned my addr 172.21.15.119:0/588299087 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90e7fff700 1 --2- 172.21.15.119:0/588299087 >> v2:172.21.15.184:3300/0 conn(0x7f90e81597f0 0x7f90e80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90ed39c700 1 -- 172.21.15.119:0/588299087 >> v2:172.21.15.184:3300/0 conn(0x7f90e81597f0 msgr2=0x7f90e80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90ed39c700 1 --2- 172.21.15.119:0/588299087 >> v2:172.21.15.184:3300/0 conn(0x7f90e81597f0 0x7f90e80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90ed39c700 1 -- 172.21.15.119:0/588299087 >> v1:172.21.15.184:6789/0 conn(0x7f90e8159420 legacy=0x7f90e80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90ed39c700 1 -- 172.21.15.119:0/588299087 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e80b1790 con 0x7f90e815d230 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90e77fe700 1 -- 172.21.15.119:0/588299087 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309645469 0 0) 0x7f90e815db70 con 0x7f90e8159420 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90ed39c700 1 --2- 172.21.15.119:0/588299087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815d630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f90e8089750 tx=0x7f90e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fab3d4ae7ffe9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.957+0000 7f90e77fe700 1 -- 172.21.15.119:0/588299087 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e000e070 con 0x7f90e815d230 2023-10-08T00:55:52.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90e77fe700 1 -- 172.21.15.119:0/588299087 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e000ba20 con 0x7f90e815d230 2023-10-08T00:55:52.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90e77fe700 1 -- 172.21.15.119:0/588299087 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e00092e0 con 0x7f90e815d230 2023-10-08T00:55:52.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90edb9d700 1 -- 172.21.15.119:0/588299087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 msgr2=0x7f90e815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90edb9d700 1 --2- 172.21.15.119:0/588299087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815d630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f90e8089750 tx=0x7f90e0009840 comp rx=0 tx=0).stop 2023-10-08T00:55:52.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90edb9d700 1 -- 172.21.15.119:0/588299087 shutdown_connections 2023-10-08T00:55:52.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90edb9d700 1 --2- 172.21.15.119:0/588299087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815d630 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90edb9d700 1 --2- 172.21.15.119:0/588299087 >> v2:172.21.15.184:3300/0 conn(0x7f90e81597f0 0x7f90e80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90edb9d700 1 -- 172.21.15.119:0/588299087 >> 172.21.15.119:0/588299087 conn(0x7f90e80d0660 msgr2=0x7f90e80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:52.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90edb9d700 1 -- 172.21.15.119:0/588299087 shutdown_connections 2023-10-08T00:55:52.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.958+0000 7f90edb9d700 1 -- 172.21.15.119:0/588299087 wait complete. 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.959+0000 7f90edb9d700 1 Processor -- start 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.960+0000 7f90edb9d700 1 -- start start 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.960+0000 7f90edb9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8159420 0x7f90e8168ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.960+0000 7f90edb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e81597f0 0x7f90e815f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90edb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90edb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e80c6260 con 0x7f90e815d230 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90edb9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8145690 con 0x7f90e8159420 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90edb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e8145970 con 0x7f90e81597f0 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41850/0 (socket says 172.21.15.119:41850) 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 -- 172.21.15.119:0/3438560221 learned_addr learned my addr 172.21.15.119:0/3438560221 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ecb9b700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8159420 0x7f90e8168ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90e7fff700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e81597f0 0x7f90e815f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 -- 172.21.15.119:0/3438560221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e81597f0 msgr2=0x7f90e815f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e81597f0 0x7f90e815f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 -- 172.21.15.119:0/3438560221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8159420 msgr2=0x7f90e8168ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8159420 0x7f90e8168ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 -- 172.21.15.119:0/3438560221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0009ae0 con 0x7f90e815d230 2023-10-08T00:55:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.961+0000 7f90ed39c700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815f880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009ab0 tx=0x7f90e0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.962+0000 7f90e4ff9700 1 -- 172.21.15.119:0/3438560221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0008bb0 con 0x7f90e815d230 2023-10-08T00:55:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.962+0000 7f90e4ff9700 1 -- 172.21.15.119:0/3438560221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e0008d10 con 0x7f90e815d230 2023-10-08T00:55:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.962+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e8145c50 con 0x7f90e815d230 2023-10-08T00:55:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.962+0000 7f90e4ff9700 1 -- 172.21.15.119:0/3438560221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e00256b0 con 0x7f90e815d230 2023-10-08T00:55:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.962+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e8146210 con 0x7f90e815d230 2023-10-08T00:55:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.963+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90e8146430 con 0x7f90e815d230 2023-10-08T00:55:52.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.964+0000 7f90e4ff9700 1 -- 172.21.15.119:0/3438560221 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f90e0003040 con 0x7f90e815d230 2023-10-08T00:55:52.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.964+0000 7f90e4ff9700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f90cc066f30 0x7f90cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:52.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.965+0000 7f90e4ff9700 1 -- 172.21.15.119:0/3438560221 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f90e00861f0 con 0x7f90e815d230 2023-10-08T00:55:52.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.965+0000 7f90ecb9b700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f90cc066f30 0x7f90cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:52.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.965+0000 7f90ecb9b700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f90cc066f30 0x7f90cc0693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90dc007fd0 tx=0x7f90dc007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:52.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:52.969+0000 7f90e4ff9700 1 -- 172.21.15.119:0/3438560221 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90e00533c0 con 0x7f90e815d230 2023-10-08T00:55:53.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.163+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90e804c3c0 con 0x7f90e815d230 2023-10-08T00:55:53.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.164+0000 7f90e4ff9700 1 -- 172.21.15.119:0/3438560221 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f90e00569e0 con 0x7f90e815d230 2023-10-08T00:55:53.166 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:53.166 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f90cc066f30 msgr2=0x7f90cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f90cc066f30 0x7f90cc0693e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90dc007fd0 tx=0x7f90dc007d30 comp rx=0 tx=0).stop 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 msgr2=0x7f90e815f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815f880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f90e0009ab0 tx=0x7f90e0018000 comp rx=0 tx=0).stop 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 shutdown_connections 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f90cc066f30 0x7f90cc0693e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e815d230 0x7f90e815f880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e8159420 0x7f90e8168ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 --2- 172.21.15.119:0/3438560221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e81597f0 0x7f90e815f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 >> 172.21.15.119:0/3438560221 conn(0x7f90e80d0660 msgr2=0x7f90e8068ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 shutdown_connections 2023-10-08T00:55:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.167+0000 7f90edb9d700 1 -- 172.21.15.119:0/3438560221 wait complete. 2023-10-08T00:55:53.186 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-08T00:55:53.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:53.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.364+0000 7f2154b9d700 1 Processor -- start 2023-10-08T00:55:53.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.364+0000 7f2154b9d700 1 -- start start 2023-10-08T00:55:53.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f2154b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2150159420 0x7f21500b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f2154b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21501597f0 0x7f21500b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f2154b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21500b11d0 con 0x7f215015d230 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f2154b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21500b14b0 con 0x7f21501597f0 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f2154b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21500b1790 con 0x7f2150159420 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f214effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21501597f0 0x7f21500b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f214effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21501597f0 0x7f21500b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41858/0 (socket says 172.21.15.119:41858) 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f214effd700 1 -- 172.21.15.119:0/619236519 learned_addr learned my addr 172.21.15.119:0/619236519 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.365+0000 7f214f7fe700 1 --2- 172.21.15.119:0/619236519 >> v2:172.21.15.184:3300/0 conn(0x7f2150159420 0x7f21500b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.366+0000 7f214effd700 1 -- 172.21.15.119:0/619236519 >> v2:172.21.15.184:3300/0 conn(0x7f2150159420 msgr2=0x7f21500b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.366+0000 7f214effd700 1 --2- 172.21.15.119:0/619236519 >> v2:172.21.15.184:3300/0 conn(0x7f2150159420 0x7f21500b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.366+0000 7f214effd700 1 -- 172.21.15.119:0/619236519 >> v1:172.21.15.184:6789/0 conn(0x7f215015d230 legacy=0x7f215015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.366+0000 7f214effd700 1 -- 172.21.15.119:0/619236519 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21500b1a70 con 0x7f21501597f0 2023-10-08T00:55:53.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.366+0000 7f214effd700 1 --2- 172.21.15.119:0/619236519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21501597f0 0x7f21500b2270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f21500b2f20 tx=0x7f2144009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20efc8d43916b85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.366+0000 7f214e7fc700 1 -- 172.21.15.119:0/619236519 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144009e70 con 0x7f21501597f0 2023-10-08T00:55:53.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.366+0000 7f214e7fc700 1 -- 172.21.15.119:0/619236519 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214400b860 con 0x7f21501597f0 2023-10-08T00:55:53.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.366+0000 7f214e7fc700 1 -- 172.21.15.119:0/619236519 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214400bc00 con 0x7f21501597f0 2023-10-08T00:55:53.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.367+0000 7f2154b9d700 1 -- 172.21.15.119:0/619236519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21501597f0 msgr2=0x7f21500b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.367+0000 7f2154b9d700 1 --2- 172.21.15.119:0/619236519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21501597f0 0x7f21500b2270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f21500b2f20 tx=0x7f2144009250 comp rx=0 tx=0).stop 2023-10-08T00:55:53.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.367+0000 7f2154b9d700 1 -- 172.21.15.119:0/619236519 shutdown_connections 2023-10-08T00:55:53.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.367+0000 7f2154b9d700 1 --2- 172.21.15.119:0/619236519 >> v2:172.21.15.184:3300/0 conn(0x7f2150159420 0x7f21500b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.367+0000 7f2154b9d700 1 --2- 172.21.15.119:0/619236519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21501597f0 0x7f21500b2270 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.367+0000 7f2154b9d700 1 -- 172.21.15.119:0/619236519 >> 172.21.15.119:0/619236519 conn(0x7f21500d0660 msgr2=0x7f21500d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.367+0000 7f2154b9d700 1 -- 172.21.15.119:0/619236519 shutdown_connections 2023-10-08T00:55:53.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.367+0000 7f2154b9d700 1 -- 172.21.15.119:0/619236519 wait complete. 2023-10-08T00:55:53.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.368+0000 7f2154b9d700 1 Processor -- start 2023-10-08T00:55:53.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.368+0000 7f2154b9d700 1 -- start start 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.369+0000 7f2154b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2150159420 0x7f215014e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.369+0000 7f2154b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21501597f0 0x7f2150068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.369+0000 7f2154b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215015d230 0x7f215015c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.369+0000 7f2154b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21500b4a60 con 0x7f2150159420 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.369+0000 7f2154b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215014e9d0 con 0x7f21501597f0 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.369+0000 7f2154b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215015ce00 con 0x7f215015d230 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.369+0000 7f214f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2150159420 0x7f215014e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2150159420 0x7f215014e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41870/0 (socket says 172.21.15.119:41870) 2023-10-08T00:55:53.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214f7fe700 1 -- 172.21.15.119:0/1499631844 learned_addr learned my addr 172.21.15.119:0/1499631844 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214f7fe700 1 -- 172.21.15.119:0/1499631844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215015d230 msgr2=0x7f215015c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214effd700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21501597f0 0x7f2150068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214f7fe700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215015d230 0x7f215015c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214f7fe700 1 -- 172.21.15.119:0/1499631844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21501597f0 msgr2=0x7f2150068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214f7fe700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21501597f0 0x7f2150068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214f7fe700 1 -- 172.21.15.119:0/1499631844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21440094f0 con 0x7f2150159420 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214effd700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21501597f0 0x7f2150068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f214f7fe700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2150159420 0x7f215014e490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f214000ec80 tx=0x7f21400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f21377fe700 1 -- 172.21.15.119:0/1499631844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214000c9d0 con 0x7f2150159420 2023-10-08T00:55:53.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f21377fe700 1 -- 172.21.15.119:0/1499631844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214000cb30 con 0x7f2150159420 2023-10-08T00:55:53.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21500b4cb0 con 0x7f2150159420 2023-10-08T00:55:53.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f21377fe700 1 -- 172.21.15.119:0/1499631844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2140013470 con 0x7f2150159420 2023-10-08T00:55:53.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.370+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21500b5270 con 0x7f2150159420 2023-10-08T00:55:53.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.371+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215004c3c0 con 0x7f2150159420 2023-10-08T00:55:53.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.372+0000 7f21377fe700 1 -- 172.21.15.119:0/1499631844 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2140013820 con 0x7f2150159420 2023-10-08T00:55:53.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.373+0000 7f21377fe700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f213c066f30 0x7f213c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.374+0000 7f21377fe700 1 -- 172.21.15.119:0/1499631844 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2140084960 con 0x7f2150159420 2023-10-08T00:55:53.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.374+0000 7f214effd700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f213c066f30 0x7f213c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.374+0000 7f214effd700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f213c066f30 0x7f213c0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21500b2f20 tx=0x7f2144017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.377+0000 7f21377fe700 1 -- 172.21.15.119:0/1499631844 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2140051b30 con 0x7f2150159420 2023-10-08T00:55:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.573+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21500af710 con 0x7f2150159420 2023-10-08T00:55:53.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.574+0000 7f21377fe700 1 -- 172.21.15.119:0/1499631844 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2140055150 con 0x7f2150159420 2023-10-08T00:55:53.575 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:53.575 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:53.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f213c066f30 msgr2=0x7f213c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f213c066f30 0x7f213c0693e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f21500b2f20 tx=0x7f2144017000 comp rx=0 tx=0).stop 2023-10-08T00:55:53.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2150159420 msgr2=0x7f215014e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2150159420 0x7f215014e490 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f214000ec80 tx=0x7f21400091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 shutdown_connections 2023-10-08T00:55:53.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f215015d230 0x7f215015c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f213c066f30 0x7f213c0693e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2150159420 0x7f215014e490 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 --2- 172.21.15.119:0/1499631844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21501597f0 0x7f2150068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 >> 172.21.15.119:0/1499631844 conn(0x7f21500d0660 msgr2=0x7f2150156090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 shutdown_connections 2023-10-08T00:55:53.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.576+0000 7f2154b9d700 1 -- 172.21.15.119:0/1499631844 wait complete. 2023-10-08T00:55:53.595 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-08T00:55:53.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-08T00:55:53.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-08T00:55:53.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-08T00:55:53.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-08T00:55:53.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-08T00:55:53.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-08T00:55:53.599 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-08T00:55:53.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.924+0000 7f29689fe700 1 Processor -- start 2023-10-08T00:55:53.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.924+0000 7f29689fe700 1 -- start start 2023-10-08T00:55:53.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.926+0000 7f29689fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f295c052320 0x7f295c053450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.926+0000 7f29689fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c0ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.926+0000 7f29689fe700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295c006030 con 0x7f295c033b80 2023-10-08T00:55:53.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.926+0000 7f29689fe700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295c0e52d0 con 0x7f295c052320 2023-10-08T00:55:53.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.926+0000 7f29689fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c053cc0 con 0x7f295c04f450 2023-10-08T00:55:53.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.927+0000 7faee5b9d700 1 Processor -- start 2023-10-08T00:55:53.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.927+0000 7f296359e700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f295c033b80 0x7f295c0e1f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35324/0 (socket says 172.21.15.119:35324) 2023-10-08T00:55:53.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.927+0000 7f296359e700 1 -- 172.21.15.119:0/3647986605 learned_addr learned my addr 172.21.15.119:0/3647986605 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:53.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.927+0000 7f2962d9d700 1 --2- 172.21.15.119:0/3647986605 >> v2:172.21.15.119:3300/0 conn(0x7f295c052320 0x7f295c053450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.928+0000 7f2961d9b700 1 -- 172.21.15.119:0/3647986605 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560634908 0 0) 0x7f295c006030 con 0x7f295c033b80 2023-10-08T00:55:53.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.928+0000 7f2961d9b700 1 -- 172.21.15.119:0/3647986605 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c003570 con 0x7f295c033b80 2023-10-08T00:55:53.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.928+0000 7f296259c700 1 --2- 172.21.15.119:0/3647986605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c0ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.928+0000 7fe9005ff700 1 Processor -- start 2023-10-08T00:55:53.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.928+0000 7fe9005ff700 1 -- start start 2023-10-08T00:55:53.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7f2962d9d700 1 -- 172.21.15.119:0/3647986605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 msgr2=0x7f295c0ddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7f44fc7ff700 1 Processor -- start 2023-10-08T00:55:53.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7f2962d9d700 1 --2- 172.21.15.119:0/3647986605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c0ddf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7f2962d9d700 1 -- 172.21.15.119:0/3647986605 >> v1:172.21.15.119:6789/0 conn(0x7f295c033b80 legacy=0x7f295c0e1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.927+0000 7faee5b9d700 1 -- start start 2023-10-08T00:55:53.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7f2962d9d700 1 -- 172.21.15.119:0/3647986605 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c053fa0 con 0x7f295c052320 2023-10-08T00:55:53.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7faee5b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faed80e3f80 0x7faed80e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.930+0000 7f296259c700 1 --2- 172.21.15.119:0/3647986605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c0ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:53.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7faee5b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faed80ea2b0 0x7faed80ea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7faee5b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faed80eac30 con 0x7faed8006a40 2023-10-08T00:55:53.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7faee5b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed8006fb0 con 0x7faed80ea2b0 2023-10-08T00:55:53.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7faee5b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faed8007290 con 0x7faed80e3f80 2023-10-08T00:55:53.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7faedffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7faed8006a40 0x7faed804ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35328/0 (socket says 172.21.15.119:35328) 2023-10-08T00:55:53.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7faedffff700 1 -- 172.21.15.119:0/4074596695 learned_addr learned my addr 172.21.15.119:0/4074596695 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:53.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.929+0000 7f44fc7ff700 1 -- start start 2023-10-08T00:55:53.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7fe9005ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe8f806ba60 0x7fe8f806abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7fe9005ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f806bf30 con 0x7fe8f815de10 2023-10-08T00:55:53.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7fe9005ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8f806a740 con 0x7fe8f806de70 2023-10-08T00:55:53.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7fe9005ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8f80c9a30 con 0x7fe8f806ba60 2023-10-08T00:55:53.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7faee539c700 1 --2- 172.21.15.119:0/4074596695 >> v2:172.21.15.119:3300/0 conn(0x7faed80ea2b0 0x7faed80ea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7fe8fdb9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe8f806ba60 0x7fe8f806abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7fe8fdb9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe8f806ba60 0x7fe8f806abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54612/0 (socket says 172.21.15.119:54612) 2023-10-08T00:55:53.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7fe8fdb9a700 1 -- 172.21.15.119:0/3143191693 learned_addr learned my addr 172.21.15.119:0/3143191693 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7f2962d9d700 1 --2- 172.21.15.119:0/3647986605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c052320 0x7f295c053450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f29540098d0 tx=0x7f2954009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be33a7c469cadc97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7f44fc7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44ec0e3e70 0x7f44ec0e4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7f44fc7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f44ec006930 0x7f44ec04fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7f44fc7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44ec006ea0 con 0x7f44ec0ea1a0 2023-10-08T00:55:53.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7f44fc7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec007180 con 0x7f44ec0e3e70 2023-10-08T00:55:53.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.931+0000 7f44fc7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44ec007460 con 0x7f44ec006930 2023-10-08T00:55:53.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7fe8fdb9a700 1 -- 172.21.15.119:0/3143191693 >> v1:172.21.15.184:6789/0 conn(0x7fe8f806de70 legacy=0x7fe8f806e2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:55:53.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7fe8fdb9a700 1 -- 172.21.15.119:0/3143191693 >> v1:172.21.15.119:6789/0 conn(0x7fe8f815de10 legacy=0x7fe8f806b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7fe8fdb9a700 1 -- 172.21.15.119:0/3143191693 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8069b70 con 0x7fe8f806ba60 2023-10-08T00:55:53.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7faee4b9b700 1 --2- 172.21.15.119:0/4074596695 >> v2:172.21.15.184:3300/0 conn(0x7faed80e3f80 0x7faed80e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7faee539c700 1 -- 172.21.15.119:0/4074596695 >> v2:172.21.15.184:3300/0 conn(0x7faed80e3f80 msgr2=0x7faed80e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7faee539c700 1 --2- 172.21.15.119:0/4074596695 >> v2:172.21.15.184:3300/0 conn(0x7faed80e3f80 0x7faed80e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7faee539c700 1 -- 172.21.15.119:0/4074596695 >> v1:172.21.15.119:6789/0 conn(0x7faed8006a40 legacy=0x7faed804ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7faedf7fe700 1 -- 172.21.15.119:0/4074596695 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321593861 0 0) 0x7faed80eac30 con 0x7faed8006a40 2023-10-08T00:55:53.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7faee539c700 1 -- 172.21.15.119:0/4074596695 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8007570 con 0x7faed80ea2b0 2023-10-08T00:55:53.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7fe8fdb9a700 1 --2- 172.21.15.119:0/3143191693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806ba60 0x7fe8f806abd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009f20 tx=0x7fe8f0009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d406fb585ca65b0f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7fe8fd399700 1 -- 172.21.15.119:0/3143191693 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f000e070 con 0x7fe8f806ba60 2023-10-08T00:55:53.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7fe8fd399700 1 -- 172.21.15.119:0/3143191693 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f000ba40 con 0x7fe8f806ba60 2023-10-08T00:55:53.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7fe8fd399700 1 -- 172.21.15.119:0/3143191693 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00092e0 con 0x7fe8f806ba60 2023-10-08T00:55:53.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7f2961d9b700 1 -- 172.21.15.119:0/3647986605 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954005c10 con 0x7f295c052320 2023-10-08T00:55:53.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7f44f9d9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f44ec006930 0x7f44ec04fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7f44fa59b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44ec0e3e70 0x7f44ec0e4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7f44f9d9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f44ec006930 0x7f44ec04fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54624/0 (socket says 172.21.15.119:54624) 2023-10-08T00:55:53.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7f44f9d9a700 1 -- 172.21.15.119:0/3309434836 learned_addr learned my addr 172.21.15.119:0/3309434836 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7f44fa59b700 1 -- 172.21.15.119:0/3309434836 >> v2:172.21.15.184:3300/0 conn(0x7f44ec006930 msgr2=0x7f44ec04fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7f44fa59b700 1 --2- 172.21.15.119:0/3309434836 >> v2:172.21.15.184:3300/0 conn(0x7f44ec006930 0x7f44ec04fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7f44fa59b700 1 -- 172.21.15.119:0/3309434836 >> v1:172.21.15.119:6789/0 conn(0x7f44ec0ea1a0 legacy=0x7f44ec0e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7f44fa59b700 1 -- 172.21.15.119:0/3309434836 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec007740 con 0x7f44ec0e3e70 2023-10-08T00:55:53.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7f2961d9b700 1 -- 172.21.15.119:0/3647986605 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2954005d70 con 0x7f295c052320 2023-10-08T00:55:53.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.933+0000 7f2961d9b700 1 -- 172.21.15.119:0/3647986605 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954006280 con 0x7f295c052320 2023-10-08T00:55:53.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7f238b9ff700 1 Processor -- start 2023-10-08T00:55:53.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.932+0000 7f238b9ff700 1 -- start start 2023-10-08T00:55:53.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7faee539c700 1 --2- 172.21.15.119:0/4074596695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed80ea2b0 0x7faed80ea6f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faed800d890 tx=0x7faecc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f10b8f7a9b32f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7f44fa59b700 1 --2- 172.21.15.119:0/3309434836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44ec0e3e70 0x7f44ec0e4240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00dfe0 tx=0x7f44e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=592a4e8a6c6cf598 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7fe9005ff700 1 -- 172.21.15.119:0/3143191693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806ba60 msgr2=0x7fe8f806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7fe9005ff700 1 --2- 172.21.15.119:0/3143191693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806ba60 0x7fe8f806abd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009f20 tx=0x7fe8f0009840 comp rx=0 tx=0).stop 2023-10-08T00:55:53.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7f29689fe700 1 -- 172.21.15.119:0/3647986605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c052320 msgr2=0x7f295c053450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7fe9005ff700 1 -- 172.21.15.119:0/3143191693 shutdown_connections 2023-10-08T00:55:53.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7fe9005ff700 1 --2- 172.21.15.119:0/3143191693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806ba60 0x7fe8f806abd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7fe9005ff700 1 -- 172.21.15.119:0/3143191693 >> 172.21.15.119:0/3143191693 conn(0x7fe8f80cccb0 msgr2=0x7fe8f80c8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7f29689fe700 1 --2- 172.21.15.119:0/3647986605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c052320 0x7f295c053450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f29540098d0 tx=0x7f2954009120 comp rx=0 tx=0).stop 2023-10-08T00:55:53.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7faedf7fe700 1 -- 172.21.15.119:0/4074596695 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00b6b0 con 0x7faed80ea2b0 2023-10-08T00:55:53.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7faedf7fe700 1 -- 172.21.15.119:0/4074596695 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faecc00b830 con 0x7faed80ea2b0 2023-10-08T00:55:53.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7faedf7fe700 1 -- 172.21.15.119:0/4074596695 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faecc00bbf0 con 0x7faed80ea2b0 2023-10-08T00:55:53.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f29689fe700 1 -- 172.21.15.119:0/3647986605 shutdown_connections 2023-10-08T00:55:53.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7f238b9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f238415e8a0 0x7f23840b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7f238b9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f23840b1a50 0x7f238415f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7f238b9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23840ccf20 con 0x7f238415e4d0 2023-10-08T00:55:53.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7f238b9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f238415f2f0 con 0x7f23840b1a50 2023-10-08T00:55:53.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.934+0000 7f238b9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f238415ecf0 con 0x7f238415e8a0 2023-10-08T00:55:53.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f29689fe700 1 --2- 172.21.15.119:0/3647986605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c052320 0x7f295c053450 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f2389f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f23840b1a50 0x7f238415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f2389f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f23840b1a50 0x7f238415f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41918/0 (socket says 172.21.15.119:41918) 2023-10-08T00:55:53.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f2389f9c700 1 -- 172.21.15.119:0/267054717 learned_addr learned my addr 172.21.15.119:0/267054717 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7fe9005ff700 1 -- 172.21.15.119:0/3143191693 shutdown_connections 2023-10-08T00:55:53.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f29689fe700 1 --2- 172.21.15.119:0/3647986605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c0ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f2388f9a700 1 --2- 172.21.15.119:0/267054717 >> v2:172.21.15.184:3300/0 conn(0x7f238415e8a0 0x7f23840b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7fe9005ff700 1 -- 172.21.15.119:0/3143191693 wait complete. 2023-10-08T00:55:53.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f29689fe700 1 -- 172.21.15.119:0/3647986605 >> 172.21.15.119:0/3647986605 conn(0x7f295c051810 msgr2=0x7f295c051c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7faee5b9d700 1 -- 172.21.15.119:0/4074596695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed80ea2b0 msgr2=0x7faed80ea6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7faee5b9d700 1 --2- 172.21.15.119:0/4074596695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed80ea2b0 0x7faed80ea6f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7faed800d890 tx=0x7faecc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f29689fe700 1 -- 172.21.15.119:0/3647986605 shutdown_connections 2023-10-08T00:55:53.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f29689fe700 1 -- 172.21.15.119:0/3647986605 wait complete. 2023-10-08T00:55:53.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f237bfff700 1 -- 172.21.15.119:0/267054717 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076952779 0 0) 0x7f23840ccf20 con 0x7f238415e4d0 2023-10-08T00:55:53.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.935+0000 7f237bfff700 1 -- 172.21.15.119:0/267054717 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2370003570 con 0x7f238415e4d0 2023-10-08T00:55:53.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7faee5b9d700 1 -- 172.21.15.119:0/4074596695 shutdown_connections 2023-10-08T00:55:53.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7faee5b9d700 1 --2- 172.21.15.119:0/4074596695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed80ea2b0 0x7faed80ea6f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7faee5b9d700 1 --2- 172.21.15.119:0/4074596695 >> v2:172.21.15.184:3300/0 conn(0x7faed80e3f80 0x7faed80e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7faee5b9d700 1 -- 172.21.15.119:0/4074596695 >> 172.21.15.119:0/4074596695 conn(0x7faed8054da0 msgr2=0x7faed80551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7f2389f9c700 1 -- 172.21.15.119:0/267054717 >> v2:172.21.15.184:3300/0 conn(0x7f238415e8a0 msgr2=0x7f23840b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7f2389f9c700 1 --2- 172.21.15.119:0/267054717 >> v2:172.21.15.184:3300/0 conn(0x7f238415e8a0 0x7f23840b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7f2389f9c700 1 -- 172.21.15.119:0/267054717 >> v1:172.21.15.184:6789/0 conn(0x7f238415e4d0 legacy=0x7f23840af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7f2389f9c700 1 -- 172.21.15.119:0/267054717 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840b0070 con 0x7f23840b1a50 2023-10-08T00:55:53.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.936+0000 7f2389f9c700 1 --2- 172.21.15.119:0/267054717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23840b1a50 0x7f238415f780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f238406d3c0 tx=0x7f237c0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe519567430cfd6c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.937+0000 7faee5b9d700 1 -- 172.21.15.119:0/4074596695 shutdown_connections 2023-10-08T00:55:53.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.937+0000 7f44f9599700 1 -- 172.21.15.119:0/3309434836 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400b6c0 con 0x7f44ec0e3e70 2023-10-08T00:55:53.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.937+0000 7f44f9599700 1 -- 172.21.15.119:0/3309434836 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e400b820 con 0x7f44ec0e3e70 2023-10-08T00:55:53.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.937+0000 7f44f9599700 1 -- 172.21.15.119:0/3309434836 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400bbc0 con 0x7f44ec0e3e70 2023-10-08T00:55:53.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.937+0000 7f237bfff700 1 -- 172.21.15.119:0/267054717 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c016070 con 0x7f23840b1a50 2023-10-08T00:55:53.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.937+0000 7f237bfff700 1 -- 172.21.15.119:0/267054717 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c00b920 con 0x7f23840b1a50 2023-10-08T00:55:53.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.937+0000 7f237bfff700 1 -- 172.21.15.119:0/267054717 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c00e070 con 0x7f23840b1a50 2023-10-08T00:55:53.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f29689fe700 1 Processor -- start 2023-10-08T00:55:53.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f44fc7ff700 1 -- 172.21.15.119:0/3309434836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44ec0e3e70 msgr2=0x7f44ec0e4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f44fc7ff700 1 --2- 172.21.15.119:0/3309434836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44ec0e3e70 0x7f44ec0e4240 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00dfe0 tx=0x7f44e40091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f29689fe700 1 -- start start 2023-10-08T00:55:53.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f238b9ff700 1 -- 172.21.15.119:0/267054717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23840b1a50 msgr2=0x7f238415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f238b9ff700 1 --2- 172.21.15.119:0/267054717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23840b1a50 0x7f238415f780 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f238406d3c0 tx=0x7f237c0097e0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f238b9ff700 1 -- 172.21.15.119:0/267054717 shutdown_connections 2023-10-08T00:55:53.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f238b9ff700 1 --2- 172.21.15.119:0/267054717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f23840b1a50 0x7f238415f780 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f44fc7ff700 1 -- 172.21.15.119:0/3309434836 shutdown_connections 2023-10-08T00:55:53.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f44fc7ff700 1 --2- 172.21.15.119:0/3309434836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44ec0e3e70 0x7f44ec0e4240 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f44fc7ff700 1 --2- 172.21.15.119:0/3309434836 >> v2:172.21.15.184:3300/0 conn(0x7f44ec006930 0x7f44ec04fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f44fc7ff700 1 -- 172.21.15.119:0/3309434836 >> 172.21.15.119:0/3309434836 conn(0x7f44ec054c90 msgr2=0x7f44ec055090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f238b9ff700 1 --2- 172.21.15.119:0/267054717 >> v2:172.21.15.184:3300/0 conn(0x7f238415e8a0 0x7f23840b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f238b9ff700 1 -- 172.21.15.119:0/267054717 >> 172.21.15.119:0/267054717 conn(0x7f2384068e20 msgr2=0x7f23840cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.938+0000 7f238b9ff700 1 -- 172.21.15.119:0/267054717 shutdown_connections 2023-10-08T00:55:53.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.939+0000 7f44fc7ff700 1 -- 172.21.15.119:0/3309434836 shutdown_connections 2023-10-08T00:55:53.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.939+0000 7f44fc7ff700 1 -- 172.21.15.119:0/3309434836 wait complete. 2023-10-08T00:55:53.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.939+0000 7f098a59d700 1 Processor -- start 2023-10-08T00:55:53.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.937+0000 7faee5b9d700 1 -- 172.21.15.119:0/4074596695 wait complete. 2023-10-08T00:55:53.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.939+0000 7fe9005ff700 1 Processor -- start 2023-10-08T00:55:53.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.939+0000 7faee5b9d700 1 Processor -- start 2023-10-08T00:55:53.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.939+0000 7f098a59d700 1 -- start start 2023-10-08T00:55:53.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.939+0000 7faee5b9d700 1 -- start start 2023-10-08T00:55:53.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.939+0000 7f238b9ff700 1 -- 172.21.15.119:0/267054717 wait complete. 2023-10-08T00:55:53.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.940+0000 7f29689fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f295c033b80 0x7f295c0dd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.940+0000 7f29689fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c04d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7f29689fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c052320 0x7f295c0cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7f2962d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f295c033b80 0x7f295c0dd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7f2962d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f295c033b80 0x7f295c0dd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54654/0 (socket says 172.21.15.119:54654) 2023-10-08T00:55:53.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7f296259c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c04d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7f44fc7ff700 1 Processor -- start 2023-10-08T00:55:53.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.940+0000 7fe9005ff700 1 -- start start 2023-10-08T00:55:53.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7fe9005ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f806ba60 0x7fe8f815eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7fe9005ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806de70 0x7fe8f815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7fe9005ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f815de10 0x7fe8f815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7fe9005ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f815bcb0 con 0x7fe8f806ba60 2023-10-08T00:55:53.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7fe9005ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f8158d00 con 0x7fe8f806de70 2023-10-08T00:55:53.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7fe9005ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f815f6b0 con 0x7fe8f815de10 2023-10-08T00:55:53.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7f098a59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09841597f0 0x7f09840b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f238b9ff700 1 Processor -- start 2023-10-08T00:55:53.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f44fc7ff700 1 -- start start 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7faee5b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed8006a40 0x7faed8036e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7faee5b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed80e3f80 0x7faed80e29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7faee5b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faed80ea2b0 0x7faed80d2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7faee5b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed8036980 con 0x7faed8006a40 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7faee5b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed8037350 con 0x7faed80e3f80 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7faee5b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faed8036370 con 0x7faed80ea2b0 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7faee4b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed8006a40 0x7faed8036e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7faee4b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed8006a40 0x7faed8036e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41958/0 (socket says 172.21.15.119:41958) 2023-10-08T00:55:53.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7faee4b9b700 1 -- 172.21.15.119:0/1015821335 learned_addr learned my addr 172.21.15.119:0/1015821335 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.941+0000 7f296259c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c04d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34356/0 (socket says 172.21.15.119:34356) 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f2962d9d700 1 -- 172.21.15.119:0/1599774723 learned_addr learned my addr 172.21.15.119:0/1599774723 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f296359e700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c052320 0x7f295c0cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c04e040 con 0x7f295c052320 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c04c5d0 con 0x7f295c033b80 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c04dd60 con 0x7f295c04f450 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f2962d9d700 1 -- 172.21.15.119:0/1599774723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 msgr2=0x7f295c04d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f2962d9d700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f295c04f450 0x7f295c04d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f2962d9d700 1 -- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c052320 msgr2=0x7f295c0cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f2962d9d700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f295c052320 0x7f295c0cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f2962d9d700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29540093c0 con 0x7f295c033b80 2023-10-08T00:55:53.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f238b9ff700 1 -- start start 2023-10-08T00:55:53.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7fe8feb9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f815de10 0x7fe8f815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f44fc7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44ec006930 0x7f44ec04cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f44fc7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44ec0e3e70 0x7f44ec0ed170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f098a59d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09840b11d0 con 0x7f098415d230 2023-10-08T00:55:53.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f098a59d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09840b14b0 con 0x7f0984159420 2023-10-08T00:55:53.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f098a59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09840b1790 con 0x7f09841597f0 2023-10-08T00:55:53.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f0989d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f098415d230 0x7f098415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35348/0 (socket says 172.21.15.119:35348) 2023-10-08T00:55:53.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7f0989d9c700 1 -- 172.21.15.119:0/1401236181 learned_addr learned my addr 172.21.15.119:0/1401236181 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:53.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7fea399ff700 1 Processor -- start 2023-10-08T00:55:53.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.942+0000 7fea399ff700 1 -- start start 2023-10-08T00:55:53.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f44fc7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44ec0ea1a0 0x7f44ec03a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f2962d9d700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f295c033b80 0x7f295c0dd290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f295c007c30 tx=0x7f2954016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295401a630 con 0x7f295c033b80 2023-10-08T00:55:53.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7fe8fdb9a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806de70 0x7fe8f815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fe8fdb9a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806de70 0x7fe8f815b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54656/0 (socket says 172.21.15.119:54656) 2023-10-08T00:55:53.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fe8fdb9a700 1 -- 172.21.15.119:0/1906097208 learned_addr learned my addr 172.21.15.119:0/1906097208 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fe8fe39b700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f806ba60 0x7fe8f815eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f44fc7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec04e520 con 0x7f44ec0e3e70 2023-10-08T00:55:53.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.943+0000 7f44fc7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec04e9b0 con 0x7f44ec006930 2023-10-08T00:55:53.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f44fc7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44ec04d910 con 0x7f44ec0ea1a0 2023-10-08T00:55:53.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295401a7b0 con 0x7f295c033b80 2023-10-08T00:55:53.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7faee4b9b700 1 -- 172.21.15.119:0/1015821335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faed80ea2b0 msgr2=0x7faed80d2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:55:53.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fe8fdb9a700 1 -- 172.21.15.119:0/1906097208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f815de10 msgr2=0x7fe8f815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fe8fdb9a700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8f815de10 0x7fe8f815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7faee4b9b700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faed80ea2b0 0x7faed80d2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fe8fdb9a700 1 -- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f806ba60 msgr2=0x7fe8f815eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fe8fdb9a700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8f806ba60 0x7fe8f815eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fe8fdb9a700 1 -- 172.21.15.119:0/1906097208 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0009ae0 con 0x7fe8f806de70 2023-10-08T00:55:53.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f44fad9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44ec0ea1a0 0x7f44ec03a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7faee4b9b700 1 -- 172.21.15.119:0/1015821335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed80e3f80 msgr2=0x7faed80e29a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:55:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7faee4b9b700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faed80e3f80 0x7faed80e29a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f44fad9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44ec0ea1a0 0x7f44ec03a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34376/0 (socket says 172.21.15.119:34376) 2023-10-08T00:55:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f44fad9c700 1 -- 172.21.15.119:0/2159541803 learned_addr learned my addr 172.21.15.119:0/2159541803 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7faee4b9b700 1 -- 172.21.15.119:0/1015821335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc009420 con 0x7faed8006a40 2023-10-08T00:55:53.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f238b9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23840b1a50 0x7f238412b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fea399ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340691b0 0x7fea340ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fea399ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea340b2490 con 0x7fea340cba20 2023-10-08T00:55:53.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fea399ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea340ac4e0 con 0x7fea34156050 2023-10-08T00:55:53.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7fea399ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea340aceb0 con 0x7fea340691b0 2023-10-08T00:55:53.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f238b9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f238415e4d0 0x7f2384125f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f238b9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f238415e8a0 0x7f2384175cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7f44fa59b700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44ec006930 0x7f44ec04cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f238b9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238412bed0 con 0x7f238415e4d0 2023-10-08T00:55:53.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f238b9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2384126480 con 0x7f23840b1a50 2023-10-08T00:55:53.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7faee4b9b700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed8006a40 0x7faed8036e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faed400d890 tx=0x7faed4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f238b9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238412ccf0 con 0x7f238415e8a0 2023-10-08T00:55:53.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7fea32ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fea340cba20 0x7fea34068aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35356/0 (socket says 172.21.15.119:35356) 2023-10-08T00:55:53.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7fea32ffd700 1 -- 172.21.15.119:0/30297655 learned_addr learned my addr 172.21.15.119:0/30297655 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:53.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7fea327fc700 1 --2- 172.21.15.119:0/30297655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340691b0 0x7fea340ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7f2389f9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f238415e8a0 0x7f2384175cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7f2389f9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f238415e8a0 0x7f2384175cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34404/0 (socket says 172.21.15.119:34404) 2023-10-08T00:55:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7f2389f9c700 1 -- 172.21.15.119:0/2037653797 learned_addr learned my addr 172.21.15.119:0/2037653797 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f2388f9a700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f238415e4d0 0x7f2384125f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f2389f9c700 1 -- 172.21.15.119:0/2037653797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23840b1a50 msgr2=0x7f238412b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f238979b700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23840b1a50 0x7f238412b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f2389f9c700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23840b1a50 0x7f238412b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f44fa59b700 1 -- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44ec0ea1a0 msgr2=0x7f44ec03a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f44fa59b700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44ec0ea1a0 0x7f44ec03a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f44fa59b700 1 -- 172.21.15.119:0/2159541803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44ec0e3e70 msgr2=0x7f44ec0ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f44fa59b700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44ec0e3e70 0x7f44ec0ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f44fa59b700 1 -- 172.21.15.119:0/2159541803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e4009420 con 0x7f44ec006930 2023-10-08T00:55:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f2389f9c700 1 -- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f238415e4d0 msgr2=0x7f2384125f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f2389f9c700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f238415e4d0 0x7f2384125f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400b690 con 0x7faed8006a40 2023-10-08T00:55:53.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f2389f9c700 1 -- 172.21.15.119:0/2037653797 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c009a80 con 0x7f238415e8a0 2023-10-08T00:55:53.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed400b7f0 con 0x7faed8006a40 2023-10-08T00:55:53.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400b690 con 0x7faed8006a40 2023-10-08T00:55:53.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7faee5b9d700 1 -- 172.21.15.119:0/1015821335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed80d3570 con 0x7faed8006a40 2023-10-08T00:55:53.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f238979b700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f23840b1a50 0x7f238412b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:53.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f2388f9a700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f238415e4d0 0x7f2384125f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:53.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f44fa59b700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44ec006930 0x7f44ec04cfa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44e4015000 tx=0x7f44e4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1286666343 0 0) 0x7f09840b11d0 con 0x7f098415d230 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.944+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0970003570 con 0x7f098415d230 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713621420 0 0) 0x7f09840b14b0 con 0x7f0984159420 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09840b11d0 con 0x7f0984159420 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7f0988d9a700 1 --2- 172.21.15.119:0/1401236181 >> v2:172.21.15.119:3300/0 conn(0x7f09841597f0 0x7f09840b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 20874486 0 0) 0x7f0970003570 con 0x7f098415d230 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 >> v2:172.21.15.119:3300/0 conn(0x7f09841597f0 msgr2=0x7f09840b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 --2- 172.21.15.119:0/1401236181 >> v2:172.21.15.119:3300/0 conn(0x7f09841597f0 0x7f09840b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 >> v1:172.21.15.184:6789/0 conn(0x7f0984159420 legacy=0x7f09840b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840b1a70 con 0x7f098415d230 2023-10-08T00:55:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f097c0017f0 con 0x7f098415d230 2023-10-08T00:55:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 >> v1:172.21.15.119:6789/0 conn(0x7f098415d230 legacy=0x7f098415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0970003c20 0x7f0970003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f0970004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0970005210 0x7f0970009a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0974001210 con 0x7f0970005210 2023-10-08T00:55:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09840b1790 con 0x7f0970004810 2023-10-08T00:55:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097000a220 con 0x7f0970003c20 2023-10-08T00:55:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7f2389f9c700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f238415e8a0 0x7f2384175cf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f237c009820 tx=0x7f237c018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7faee5b9d700 1 -- 172.21.15.119:0/1015821335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed80d3850 con 0x7faed8006a40 2023-10-08T00:55:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7fea31ffb700 1 -- 172.21.15.119:0/30297655 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351239045 0 0) 0x7fea340ac4e0 con 0x7fea34156050 2023-10-08T00:55:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7fea31ffb700 1 -- 172.21.15.119:0/30297655 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea1c003570 con 0x7fea34156050 2023-10-08T00:55:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4011cb0 con 0x7f44ec006930 2023-10-08T00:55:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7fea31ffb700 1 -- 172.21.15.119:0/30297655 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715010964 0 0) 0x7fea24004b00 con 0x7fea340cba20 2023-10-08T00:55:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7fea31ffb700 1 -- 172.21.15.119:0/30297655 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea340ac4e0 con 0x7fea340cba20 2023-10-08T00:55:53.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44ec04dbf0 con 0x7f44ec006930 2023-10-08T00:55:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7fea327fc700 1 -- 172.21.15.119:0/30297655 >> v1:172.21.15.184:6789/0 conn(0x7fea34156050 legacy=0x7fea341564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7fea327fc700 1 -- 172.21.15.119:0/30297655 >> v1:172.21.15.119:6789/0 conn(0x7fea340cba20 legacy=0x7fea34068aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44ec04e1b0 con 0x7f44ec006930 2023-10-08T00:55:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7f0988d9a700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f0970004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7fea327fc700 1 -- 172.21.15.119:0/30297655 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea34156cd0 con 0x7fea340691b0 2023-10-08T00:55:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.947+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4020070 con 0x7f44ec006930 2023-10-08T00:55:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.948+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e401b6a0 con 0x7f44ec006930 2023-10-08T00:55:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c04bf80 con 0x7f295c033b80 2023-10-08T00:55:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c04da70 con 0x7f295c033b80 2023-10-08T00:55:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.948+0000 7f098959b700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0970003c20 0x7f0970003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7fe8fdb9a700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806de70 0x7fe8f815b030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8070e20 tx=0x7fe8f0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0012db0 con 0x7fe8f806de70 2023-10-08T00:55:53.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.948+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295401ab70 con 0x7f295c033b80 2023-10-08T00:55:53.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.948+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374001470 con 0x7f238415e8a0 2023-10-08T00:55:53.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.948+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237c012da0 con 0x7f238415e8a0 2023-10-08T00:55:53.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.948+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237c01c670 con 0x7f238415e8a0 2023-10-08T00:55:53.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7fea327fc700 1 --2- 172.21.15.119:0/30297655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340691b0 0x7fea340ac970 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fea2800aa20 tx=0x7fea2800a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13a5c36a00ea1072 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f001d420 con 0x7fe8f806de70 2023-10-08T00:55:53.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f001d7e0 con 0x7fe8f806de70 2023-10-08T00:55:53.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.945+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f815f990 con 0x7fe8f806de70 2023-10-08T00:55:53.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.946+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f8159230 con 0x7fe8f806de70 2023-10-08T00:55:53.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7faee5b9d700 1 -- 172.21.15.119:0/1015821335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faed80054e0 con 0x7faed8006a40 2023-10-08T00:55:53.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8f001dc10 con 0x7fe8f806de70 2023-10-08T00:55:53.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7f0989d9c700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0970005210 0x7f0970009a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f44dc000fd0 con 0x7f44ec006930 2023-10-08T00:55:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2384176230 con 0x7f238415e8a0 2023-10-08T00:55:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7fea31ffb700 1 -- 172.21.15.119:0/30297655 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800ae60 con 0x7fea340691b0 2023-10-08T00:55:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7fea31ffb700 1 -- 172.21.15.119:0/30297655 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2800c820 con 0x7fea340691b0 2023-10-08T00:55:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.949+0000 7fea31ffb700 1 -- 172.21.15.119:0/30297655 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800cbc0 con 0x7fea340691b0 2023-10-08T00:55:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.950+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238412c3c0 con 0x7f238415e8a0 2023-10-08T00:55:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.950+0000 7f18d51ff700 1 Processor -- start 2023-10-08T00:55:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.950+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8e0000fd0 con 0x7fe8f806de70 2023-10-08T00:55:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.950+0000 7fe8ee7fc700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8dc067080 0x7fe8dc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe8f0085c80 con 0x7fe8f806de70 2023-10-08T00:55:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f238406c650 con 0x7f238415e8a0 2023-10-08T00:55:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7fe8ee7fc700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7fe8dc06f290 0x7fe8dc0716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 --> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8dc071d50 con 0x7fe8dc06f290 2023-10-08T00:55:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7fe8fe39b700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8dc067080 0x7fe8dc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f008a200 con 0x7fe8f806de70 2023-10-08T00:55:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.950+0000 7fea399ff700 1 -- 172.21.15.119:0/30297655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340691b0 msgr2=0x7fea340ac970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.950+0000 7fea399ff700 1 --2- 172.21.15.119:0/30297655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340691b0 0x7fea340ac970 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fea2800aa20 tx=0x7fea2800a290 comp rx=0 tx=0).stop 2023-10-08T00:55:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7fea399ff700 1 -- 172.21.15.119:0/30297655 shutdown_connections 2023-10-08T00:55:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7fea399ff700 1 --2- 172.21.15.119:0/30297655 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340691b0 0x7fea340ac970 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7fea399ff700 1 -- 172.21.15.119:0/30297655 >> 172.21.15.119:0/30297655 conn(0x7fea340cd000 msgr2=0x7fea34068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.951+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f295c0054e0 con 0x7f295c033b80 2023-10-08T00:55:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.950+0000 7f18d51ff700 1 -- start start 2023-10-08T00:55:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f0988d9a700 1 -- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0970003c20 msgr2=0x7f0970003ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f0988d9a700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0970003c20 0x7f0970003ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f0988d9a700 1 -- 172.21.15.119:0/1401236181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0970005210 msgr2=0x7f0970009a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2954011420 con 0x7f295c033b80 2023-10-08T00:55:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f0988d9a700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0970005210 0x7f0970009a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f0988d9a700 1 -- 172.21.15.119:0/1401236181 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840b1a70 con 0x7f0970004810 2023-10-08T00:55:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f0989d9c700 1 -- 172.21.15.119:0/1401236181 reap_dead start 2023-10-08T00:55:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7fe8fe39b700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8dc067080 0x7fe8dc069530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ab0 tx=0x7fe8e8009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faed400b960 con 0x7faed8006a40 2023-10-08T00:55:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f0988d9a700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f0970004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f098000bb60 tx=0x7f0980008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77f4beeaea733438 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7fe8feb9c700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7fe8dc06f290 0x7fe8dc0716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:53.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f237c01ca60 con 0x7f238415e8a0 2023-10-08T00:55:53.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098000d5d0 con 0x7f0970004810 2023-10-08T00:55:53.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098000d730 con 0x7f0970004810 2023-10-08T00:55:53.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7f097bfff700 1 -- 172.21.15.119:0/1401236181 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098000daf0 con 0x7f0970004810 2023-10-08T00:55:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7f294affd700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2950067000 0x7f29500694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7fe8feb9c700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7fe8dc06f290 0x7fe8dc0716a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7faedcff9700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7faec4066ff0 0x7faec40694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7faed40846f0 con 0x7faed8006a40 2023-10-08T00:55:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2954085ab0 con 0x7f295c033b80 2023-10-08T00:55:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f18d51ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f18d00afc40 0x7f18d00b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f18d51ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d00b2140 0x7f18d015d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f18d51ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d00b0750 con 0x7f18d00b0010 2023-10-08T00:55:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7f18ce59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d00b2140 0x7f18d015d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7f18ce59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d00b2140 0x7f18d015d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34430/0 (socket says 172.21.15.119:34430) 2023-10-08T00:55:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.953+0000 7f18ce59c700 1 -- 172.21.15.119:0/1634757485 learned_addr learned my addr 172.21.15.119:0/1634757485 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.952+0000 7f18d51ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00aca70 con 0x7f18d00afc40 2023-10-08T00:55:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651899067 0 0) 0x7f18d00b0750 con 0x7f18d00b0010 2023-10-08T00:55:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7faedcff9700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7faec406f180 0x7faec4071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 --> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faec4071c40 con 0x7faec406f180 2023-10-08T00:55:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7faed400b690 con 0x7faed8006a40 2023-10-08T00:55:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44e4022030 con 0x7f44ec006930 2023-10-08T00:55:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7fea399ff700 1 -- 172.21.15.119:0/30297655 shutdown_connections 2023-10-08T00:55:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7faedffff700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7faec4066ff0 0x7faec40694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f296259c700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2950067000 0x7f29500694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f098a59d700 1 -- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 msgr2=0x7f0970004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f098a59d700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f0970004c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f098000bb60 tx=0x7f0980008040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f294affd700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f295006f0a0 0x7f29500714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2950071b60 con 0x7f295006f0a0 2023-10-08T00:55:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f295408a220 con 0x7f295c033b80 2023-10-08T00:55:54.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.954+0000 7f23797fa700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2368067030 0x7f23680694e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7faee539c700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7faec406f180 0x7faec4071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f296359e700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f295006f0a0 0x7f29500714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f098a59d700 1 -- 172.21.15.119:0/1401236181 shutdown_connections 2023-10-08T00:55:54.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f098a59d700 1 --2- 172.21.15.119:0/1401236181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f0970004c80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f098a59d700 1 -- 172.21.15.119:0/1401236181 >> 172.21.15.119:0/1401236181 conn(0x7f09840d0660 msgr2=0x7f09840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f237c085fc0 con 0x7f238415e8a0 2023-10-08T00:55:54.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f23797fa700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f236806f240 0x7f2368071650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f098a59d700 1 -- 172.21.15.119:0/1401236181 shutdown_connections 2023-10-08T00:55:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f296359e700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f295006f0a0 0x7f29500714b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 --> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2368071d00 con 0x7f236806f240 2023-10-08T00:55:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f238979b700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2368067030 0x7f23680694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f237c088520 con 0x7f238415e8a0 2023-10-08T00:55:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7faee539c700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7faec406f180 0x7faec4071590 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.956+0000 7f296259c700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2950067000 0x7f29500694b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f295c03ab40 tx=0x7f2958006f90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.956+0000 7faedffff700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7faec4066ff0 0x7faec40694a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faee0000eb0 tx=0x7faee0067410 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.956+0000 7f238979b700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2368067030 0x7f23680694e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2374008fd0 tx=0x7f23740084a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.955+0000 7f098a59d700 1 -- 172.21.15.119:0/1401236181 wait complete. 2023-10-08T00:55:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.956+0000 7f18d51ff700 1 -- 172.21.15.119:0/1634757485 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00acd50 con 0x7f18d00b2140 2023-10-08T00:55:54.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.956+0000 7f2388f9a700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f236806f240 0x7f2368071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.956+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 <== osd.3 v2:172.21.15.119:6808/85788 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe8dc071d50 con 0x7fe8dc06f290 2023-10-08T00:55:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b8003570 con 0x7f18d00b0010 2023-10-08T00:55:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18ced9d700 1 --2- 172.21.15.119:0/1634757485 >> v2:172.21.15.119:3300/0 conn(0x7f18d00afc40 0x7f18d00b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 <== osd.0 v2:172.21.15.119:6810/85787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7faec4071c40 con 0x7faec406f180 2023-10-08T00:55:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 <== osd.1 v2:172.21.15.119:6800/85785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2964091dd0 con 0x7f295006f0a0 2023-10-08T00:55:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3952651976 0 0) 0x7f18b8003570 con 0x7f18d00b0010 2023-10-08T00:55:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d00b2140 msgr2=0x7f18d015d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d00b2140 0x7f18d015d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 >> v2:172.21.15.119:3300/0 conn(0x7f18d00afc40 msgr2=0x7f18d00b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 --2- 172.21.15.119:0/1634757485 >> v2:172.21.15.119:3300/0 conn(0x7f18d00afc40 0x7f18d00b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00ad030 con 0x7f18d00b0010 2023-10-08T00:55:54.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f2388f9a700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f236806f240 0x7f2368071650 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f44ea7fc700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f44d8066fb0 0x7f44d8069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f44e4086690 con 0x7f44ec006930 2023-10-08T00:55:54.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f44ea7fc700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7f44d806f1c0 0x7f44d80715d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 --> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44d8071c80 con 0x7f44d806f1c0 2023-10-08T00:55:54.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44e40869c0 con 0x7f44ec006930 2023-10-08T00:55:54.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f098a59d700 1 Processor -- start 2023-10-08T00:55:54.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f098a59d700 1 -- start start 2023-10-08T00:55:54.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f18c4001470 con 0x7f18d00b0010 2023-10-08T00:55:54.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 >> v1:172.21.15.184:6789/0 conn(0x7f18d00b0010 legacy=0x7f18d015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18b8003a80 0x7f18b8003eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d00b2140 0x7f18b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8004d50 0x7f18b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00aca70 con 0x7f18b8003a80 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00acd50 con 0x7f18b8004d50 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.957+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b800bb10 con 0x7f18d00b2140 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f18ced9d700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18b8003a80 0x7f18b8003eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f18cf59e700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8004d50 0x7f18b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f18ced9d700 1 -- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d00b2140 msgr2=0x7f18b8004810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f18ced9d700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d00b2140 0x7f18b8004810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f18ced9d700 1 -- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8004d50 msgr2=0x7f18b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f18ced9d700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18b8004d50 0x7f18b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.958+0000 7f18ced9d700 1 -- 172.21.15.119:0/1634757485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00ad030 con 0x7f18b8003a80 2023-10-08T00:55:54.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.959+0000 7f44f9d9a700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f44d8066fb0 0x7f44d8069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.959+0000 7f18ced9d700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18b8003a80 0x7f18b8003eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18c8007440 tx=0x7f18c8009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6ac7b3910ec920b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.959+0000 7f098a59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f098415c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.959+0000 7f44f9d9a700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f44d8066fb0 0x7f44d8069460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44f404d2d0 tx=0x7f44f4066260 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 <== osd.2 v2:172.21.15.119:6824/85796 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f2368071d00 con 0x7f236806f240 2023-10-08T00:55:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f098a59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f098406ba40 0x7f0984149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f098959b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f098415c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f098959b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f098415c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54712/0 (socket says 172.21.15.119:54712) 2023-10-08T00:55:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f098959b700 1 -- 172.21.15.119:0/3756006746 learned_addr learned my addr 172.21.15.119:0/3756006746 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f098a59d700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f098406beb0 0x7f0984154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098415bea0 con 0x7f098406beb0 2023-10-08T00:55:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09841519a0 con 0x7f0970004810 2023-10-08T00:55:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.960+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0984151c80 con 0x7f098406ba40 2023-10-08T00:55:54.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f44fad9c700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7f44d806f1c0 0x7f44d80715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f098959b700 1 -- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f098406ba40 msgr2=0x7f0984149420 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:55:54.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f098959b700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f098406ba40 0x7f0984149420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f098959b700 1 -- 172.21.15.119:0/3756006746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f098406beb0 msgr2=0x7f0984154e40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:55:54.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f098959b700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f098406beb0 0x7f0984154e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f098959b700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0984151f60 con 0x7f0970004810 2023-10-08T00:55:54.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8009a40 con 0x7f18b8003a80 2023-10-08T00:55:54.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8007cf0 con 0x7f18b8003a80 2023-10-08T00:55:54.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f098959b700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f098415c290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f097400fec0 tx=0x7f0974001580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7f18cdd9b700 1 -- 172.21.15.119:0/1634757485 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8009a40 con 0x7f18b8003a80 2023-10-08T00:55:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.959+0000 7fea399ff700 1 -- 172.21.15.119:0/30297655 wait complete. 2023-10-08T00:55:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7fea399ff700 1 Processor -- start 2023-10-08T00:55:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.961+0000 7fea399ff700 1 -- start start 2023-10-08T00:55:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097400de90 con 0x7f0970004810 2023-10-08T00:55:54.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0984152240 con 0x7f0970004810 2023-10-08T00:55:54.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f44fad9c700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7f44d806f1c0 0x7f44d80715d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f18d51ff700 1 -- 172.21.15.119:0/1634757485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18b8003a80 msgr2=0x7f18b8003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f18d51ff700 1 --2- 172.21.15.119:0/1634757485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18b8003a80 0x7f18b8003eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f18c8007440 tx=0x7f18c8009710 comp rx=0 tx=0).stop 2023-10-08T00:55:54.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09841515c0 con 0x7f0970004810 2023-10-08T00:55:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f18cf59e700 1 -- 172.21.15.119:0/1634757485 reap_dead start 2023-10-08T00:55:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f18d51ff700 1 -- 172.21.15.119:0/1634757485 shutdown_connections 2023-10-08T00:55:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.962+0000 7f18d51ff700 1 -- 172.21.15.119:0/1634757485 >> 172.21.15.119:0/1634757485 conn(0x7f18d006a320 msgr2=0x7f18d00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.963+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 <== osd.4 v2:172.21.15.184:6824/86003 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f44d8071c80 con 0x7f44d806f1c0 2023-10-08T00:55:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.963+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974014070 con 0x7f0970004810 2023-10-08T00:55:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.963+0000 7f18d51ff700 1 -- 172.21.15.119:0/1634757485 shutdown_connections 2023-10-08T00:55:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.963+0000 7f18d51ff700 1 -- 172.21.15.119:0/1634757485 wait complete. 2023-10-08T00:55:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.964+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974001b10 con 0x7f0970004810 2023-10-08T00:55:54.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.966+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0974003110 con 0x7f0970004810 2023-10-08T00:55:54.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.966+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0964000fd0 con 0x7f0970004810 2023-10-08T00:55:54.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.963+0000 7fea399ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340cba20 0x7fea3414cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.963+0000 7fea399ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea34156050 0x7fea34179cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.963+0000 7fea399ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea3406e250 0x7fea3414e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.964+0000 7fea337fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea3406e250 0x7fea3414e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.964+0000 7fea337fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea3406e250 0x7fea3414e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42012/0 (socket says 172.21.15.119:42012) 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.964+0000 7fea337fe700 1 -- 172.21.15.119:0/1840003330 learned_addr learned my addr 172.21.15.119:0/1840003330 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.964+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3414ddd0 con 0x7fea3406e250 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.964+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea34087e40 con 0x7fea34156050 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.964+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3414f580 con 0x7fea340cba20 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.965+0000 7fea32ffd700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340cba20 0x7fea3414cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.965+0000 7fea337fe700 1 -- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340cba20 msgr2=0x7fea3414cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.965+0000 7fea337fe700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea340cba20 0x7fea3414cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.965+0000 7fea337fe700 1 -- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea34156050 msgr2=0x7fea34179cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.965+0000 7fea337fe700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea34156050 0x7fea34179cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.965+0000 7fea337fe700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2800a530 con 0x7fea3406e250 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.966+0000 7fea337fe700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea3406e250 0x7fea3414e5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00cbc0 tx=0x7fea2c001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.966+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00a730 con 0x7fea3406e250 2023-10-08T00:55:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.966+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c00a890 con 0x7fea3406e250 2023-10-08T00:55:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.967+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00ac30 con 0x7fea3406e250 2023-10-08T00:55:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.965+0000 7f18d51ff700 1 Processor -- start 2023-10-08T00:55:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.966+0000 7f18d51ff700 1 -- start start 2023-10-08T00:55:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.967+0000 7f18d51ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18d00aafd0 0x7f18d0161cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.967+0000 7f18d51ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d0150520 0x7f18d015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.967+0000 7f18d51ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18d015fc40 0x7f18d00a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.967+0000 7f18d51ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00ab3a0 con 0x7f18d015fc40 2023-10-08T00:55:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f09797fa700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0968066f80 0x7f0968069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.967+0000 7f18d51ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d015e660 con 0x7f18d00aafd0 2023-10-08T00:55:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.967+0000 7f18d51ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d015e940 con 0x7f18d0150520 2023-10-08T00:55:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f18ce59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d0150520 0x7f18d015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f18ce59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d0150520 0x7f18d015f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34464/0 (socket says 172.21.15.119:34464) 2023-10-08T00:55:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f18ce59c700 1 -- 172.21.15.119:0/3978232051 learned_addr learned my addr 172.21.15.119:0/3978232051 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0974085410 con 0x7f0970004810 2023-10-08T00:55:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f18ced9d700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18d00aafd0 0x7f18d0161cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f09797fa700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f096806efa0 0x7f09680713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f0988d9a700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0968066f80 0x7f0968069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7f18cf59e700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18d015fc40 0x7f18d00a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.968+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3414f860 con 0x7fea3406e250 2023-10-08T00:55:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f18ced9d700 1 -- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d0150520 msgr2=0x7f18d015f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f18ced9d700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d0150520 0x7f18d015f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f18ced9d700 1 -- 172.21.15.119:0/3978232051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18d015fc40 msgr2=0x7f18d00a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f18ced9d700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18d015fc40 0x7f18d00a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f18ced9d700 1 -- 172.21.15.119:0/3978232051 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d015ec20 con 0x7f18d00aafd0 2023-10-08T00:55:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f18ce59c700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18d0150520 0x7f18d015f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:54.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f0988d9a700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0968066f80 0x7f0968069430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f098000bb60 tx=0x7f098001b000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0968071a60 con 0x7f096806efa0 2023-10-08T00:55:54.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0974052660 con 0x7f0970004810 2023-10-08T00:55:54.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7f18ced9d700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18d00aafd0 0x7f18d0161cd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18c8017000 tx=0x7f18c8018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.969+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3406eab0 con 0x7fea3406e250 2023-10-08T00:55:54.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.971+0000 7f0989d9c700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f096806efa0 0x7f09680713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.970+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8009a40 con 0x7f18d00aafd0 2023-10-08T00:55:54.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.970+0000 7f18d51ff700 1 -- 172.21.15.119:0/3978232051 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d015ef00 con 0x7f18d00aafd0 2023-10-08T00:55:54.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.970+0000 7f18d51ff700 1 -- 172.21.15.119:0/3978232051 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d0162210 con 0x7f18d00aafd0 2023-10-08T00:55:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.971+0000 7f0989d9c700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f096806efa0 0x7f09680713b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.971+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f18b0000fd0 con 0x7f18d00aafd0 2023-10-08T00:55:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.972+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8022070 con 0x7f18d00aafd0 2023-10-08T00:55:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.972+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8009a40 con 0x7f18d00aafd0 2023-10-08T00:55:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.972+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 <== osd.5 v2:172.21.15.184:6816/86004 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0968071a60 con 0x7f096806efa0 2023-10-08T00:55:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.973+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea3404c3c0 con 0x7fea3406e250 2023-10-08T00:55:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.975+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea2c003030 con 0x7fea3406e250 2023-10-08T00:55:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.975+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f18c8003030 con 0x7f18d00aafd0 2023-10-08T00:55:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.977+0000 7fea1affd700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066f80 0x7fea20069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.977+0000 7fea32ffd700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066f80 0x7fea20069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fea2c084660 con 0x7fea3406e250 2023-10-08T00:55:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7fea1affd700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7fea2006f020 0x7fea20071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea20071ae0 con 0x7fea2006f020 2023-10-08T00:55:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea2c086bc0 con 0x7fea3406e250 2023-10-08T00:55:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7fea327fc700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7fea2006f020 0x7fea20071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.979+0000 7fea32ffd700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066f80 0x7fea20069430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea34151700 tx=0x7fea2401b350 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.977+0000 7f18b6ffd700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18bc066ff0 0x7f18bc0694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f18c8024070 con 0x7f18d00aafd0 2023-10-08T00:55:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7f18b6ffd700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7f18bc06fd90 0x7f18bc072220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 --> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f18bc0728f0 con 0x7f18bc06fd90 2023-10-08T00:55:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.978+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18c8009a40 con 0x7f18d00aafd0 2023-10-08T00:55:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.979+0000 7f18cf59e700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7f18bc06fd90 0x7f18bc072220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.979+0000 7f18cf59e700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7f18bc06fd90 0x7f18bc072220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.980+0000 7fea327fc700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7fea2006f020 0x7fea20071430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.980+0000 7faee5b9d700 1 -- 172.21.15.119:0/1015821335 --> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faed8007850 con 0x7faec406f180 2023-10-08T00:55:54.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.980+0000 7f18ce59c700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18bc066ff0 0x7f18bc0694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.981+0000 7faedcff9700 1 -- 172.21.15.119:0/1015821335 <== osd.0 v2:172.21.15.119:6810/85787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faed8007850 con 0x7faec406f180 2023-10-08T00:55:54.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.980+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 <== osd.7 v2:172.21.15.184:6800/86001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f18bc0728f0 con 0x7f18bc06fd90 2023-10-08T00:55:54.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.981+0000 7f18ce59c700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18bc066ff0 0x7f18bc0694a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009fd0 tx=0x7f18c0009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.982+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 <== osd.6 v2:172.21.15.184:6808/86002 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fea20071ae0 con 0x7fea2006f020 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.982+0000 7faeca7fc700 1 -- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7faec406f180 msgr2=0x7faec4071590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.982+0000 7faeca7fc700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7faec406f180 0x7faec4071590 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.983+0000 7faeca7fc700 1 -- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7faec4066ff0 msgr2=0x7faec40694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.983+0000 7faeca7fc700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7faec4066ff0 0x7faec40694a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faee0000eb0 tx=0x7faee0067410 comp rx=0 tx=0).stop 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.983+0000 7faeca7fc700 1 -- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed8006a40 msgr2=0x7faed8036e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.983+0000 7faeca7fc700 1 --2- 172.21.15.119:0/1015821335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faed8006a40 0x7faed8036e10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7faed400d890 tx=0x7faed4008040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.983+0000 7faee539c700 1 -- 172.21.15.119:0/1015821335 reap_dead start 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.983+0000 7faeca7fc700 1 -- 172.21.15.119:0/1015821335 shutdown_connections 2023-10-08T00:55:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.983+0000 7faeca7fc700 1 -- 172.21.15.119:0/1015821335 >> 172.21.15.119:0/1015821335 conn(0x7faed8054da0 msgr2=0x7faed80d3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.984+0000 7faeca7fc700 1 -- 172.21.15.119:0/1015821335 shutdown_connections 2023-10-08T00:55:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.984+0000 7faeca7fc700 1 -- 172.21.15.119:0/1015821335 wait complete. 2023-10-08T00:55:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.987+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 --> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8e0002d30 con 0x7fe8dc06f290 2023-10-08T00:55:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.990+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 --> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f295c054280 con 0x7f295006f0a0 2023-10-08T00:55:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.991+0000 7fe8ee7fc700 1 -- 172.21.15.119:0/1906097208 <== osd.3 v2:172.21.15.119:6808/85788 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe8e0002d30 con 0x7fe8dc06f290 2023-10-08T00:55:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.992+0000 7f294affd700 1 -- 172.21.15.119:0/1599774723 <== osd.1 v2:172.21.15.119:6800/85785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f295c054280 con 0x7f295006f0a0 2023-10-08T00:55:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.992+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7fe8dc06f290 msgr2=0x7fe8dc0716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.992+0000 7fe9005ff700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7fe8dc06f290 0x7fe8dc0716a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.992+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8dc067080 msgr2=0x7fe8dc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.992+0000 7fe9005ff700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe8dc067080 0x7fe8dc069530 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009ab0 tx=0x7fe8e8009060 comp rx=0 tx=0).stop 2023-10-08T00:55:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.992+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806de70 msgr2=0x7fe8f815b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.992+0000 7fe9005ff700 1 --2- 172.21.15.119:0/1906097208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8f806de70 0x7fe8f815b030 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8070e20 tx=0x7fe8f0017000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.994+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f295006f0a0 msgr2=0x7f29500714b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.994+0000 7f29689fe700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f295006f0a0 0x7f29500714b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.994+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2950067000 msgr2=0x7f29500694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.994+0000 7f29689fe700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2950067000 0x7f29500694b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f295c03ab40 tx=0x7f2958006f90 comp rx=0 tx=0).stop 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.994+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f295c033b80 msgr2=0x7f295c0dd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.994+0000 7f29689fe700 1 --2- 172.21.15.119:0/1599774723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f295c033b80 0x7f295c0dd290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f295c007c30 tx=0x7f2954016000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.995+0000 7f296359e700 1 -- 172.21.15.119:0/1599774723 reap_dead start 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.995+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 shutdown_connections 2023-10-08T00:55:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.995+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 >> 172.21.15.119:0/1599774723 conn(0x7f295c051810 msgr2=0x7f295c04ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.995+0000 7fe8feb9c700 1 -- 172.21.15.119:0/1906097208 reap_dead start 2023-10-08T00:55:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.997+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 --> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f238404bb80 con 0x7f236806f240 2023-10-08T00:55:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.997+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 shutdown_connections 2023-10-08T00:55:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.997+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 >> 172.21.15.119:0/1906097208 conn(0x7fe8f80cccb0 msgr2=0x7fe8f80c1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.999+0000 7f23797fa700 1 -- 172.21.15.119:0/2037653797 <== osd.2 v2:172.21.15.119:6824/85796 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f238404bb80 con 0x7f236806f240 2023-10-08T00:55:54.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.999+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 shutdown_connections 2023-10-08T00:55:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.999+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 shutdown_connections 2023-10-08T00:55:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.999+0000 7fe9005ff700 1 -- 172.21.15.119:0/1906097208 wait complete. 2023-10-08T00:55:54.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.999+0000 7f29689fe700 1 -- 172.21.15.119:0/1599774723 wait complete. 2023-10-08T00:55:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.999+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f236806f240 msgr2=0x7f2368071650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:53.999+0000 7f238b9ff700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f236806f240 0x7f2368071650 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.000+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2368067030 msgr2=0x7f23680694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.000+0000 7f238b9ff700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2368067030 0x7f23680694e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2374008fd0 tx=0x7f23740084a0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.000+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f238415e8a0 msgr2=0x7f2384175cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.000+0000 7f238b9ff700 1 --2- 172.21.15.119:0/2037653797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f238415e8a0 0x7f2384175cf0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f237c009820 tx=0x7f237c018000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.000+0000 7f2389f9c700 1 -- 172.21.15.119:0/2037653797 reap_dead start 2023-10-08T00:55:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.000+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 shutdown_connections 2023-10-08T00:55:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.000+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 >> 172.21.15.119:0/2037653797 conn(0x7f2384068e20 msgr2=0x7f238415f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.001+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 shutdown_connections 2023-10-08T00:55:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.001+0000 7f238b9ff700 1 -- 172.21.15.119:0/2037653797 wait complete. 2023-10-08T00:55:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.002+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 --> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f44dc002d30 con 0x7f44d806f1c0 2023-10-08T00:55:54.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.004+0000 7f44ea7fc700 1 -- 172.21.15.119:0/2159541803 <== osd.4 v2:172.21.15.184:6824/86003 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f44dc002d30 con 0x7f44d806f1c0 2023-10-08T00:55:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.005+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7f44d806f1c0 msgr2=0x7f44d80715d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.006+0000 7f44fc7ff700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7f44d806f1c0 0x7f44d80715d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.006+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f44d8066fb0 msgr2=0x7f44d8069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.006+0000 7f44fc7ff700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f44d8066fb0 0x7f44d8069460 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f44f404d2d0 tx=0x7f44f4066260 comp rx=0 tx=0).stop 2023-10-08T00:55:54.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.006+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44ec006930 msgr2=0x7f44ec04cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.007+0000 7f44fc7ff700 1 --2- 172.21.15.119:0/2159541803 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44ec006930 0x7f44ec04cfa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44e4015000 tx=0x7f44e4016000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.007+0000 7f44fad9c700 1 -- 172.21.15.119:0/2159541803 reap_dead start 2023-10-08T00:55:54.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.007+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 --> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0964002c90 con 0x7f096806efa0 2023-10-08T00:55:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.007+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 shutdown_connections 2023-10-08T00:55:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.007+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 >> 172.21.15.119:0/2159541803 conn(0x7f44ec054c90 msgr2=0x7f44ec0049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.007+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 shutdown_connections 2023-10-08T00:55:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.008+0000 7f44fc7ff700 1 -- 172.21.15.119:0/2159541803 wait complete. 2023-10-08T00:55:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.008+0000 7f09797fa700 1 -- 172.21.15.119:0/3756006746 <== osd.5 v2:172.21.15.184:6816/86004 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0964002c90 con 0x7f096806efa0 2023-10-08T00:55:54.051 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:54.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T00:55:54.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.010+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 --> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f18b0002d30 con 0x7f18bc06fd90 2023-10-08T00:55:54.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.011+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f096806efa0 msgr2=0x7f09680713b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.011+0000 7f098a59d700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f096806efa0 0x7f09680713b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.011+0000 7f18b6ffd700 1 -- 172.21.15.119:0/3978232051 <== osd.7 v2:172.21.15.184:6800/86001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f18b0002d30 con 0x7f18bc06fd90 2023-10-08T00:55:54.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.011+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0968066f80 msgr2=0x7f0968069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.011+0000 7f098a59d700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0968066f80 0x7f0968069430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f098000bb60 tx=0x7f098001b000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.011+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 msgr2=0x7f098415c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.011+0000 7f098a59d700 1 --2- 172.21.15.119:0/3756006746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0970004810 0x7f098415c290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f097400fec0 tx=0x7f0974001580 comp rx=0 tx=0).stop 2023-10-08T00:55:54.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.011+0000 7f0989d9c700 1 -- 172.21.15.119:0/3756006746 reap_dead start 2023-10-08T00:55:54.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 shutdown_connections 2023-10-08T00:55:54.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 >> 172.21.15.119:0/3756006746 conn(0x7f09840d0660 msgr2=0x7f0984069c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7f18bc06fd90 msgr2=0x7f18bc072220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18b4ff9700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7f18bc06fd90 0x7f18bc072220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18bc066ff0 msgr2=0x7f18bc0694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18b4ff9700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18bc066ff0 0x7f18bc0694a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009fd0 tx=0x7f18c0009250 comp rx=0 tx=0).stop 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18d00aafd0 msgr2=0x7f18d0161cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18b4ff9700 1 --2- 172.21.15.119:0/3978232051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18d00aafd0 0x7f18d0161cd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f18c8017000 tx=0x7f18c8018000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18cf59e700 1 -- 172.21.15.119:0/3978232051 reap_dead start 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 shutdown_connections 2023-10-08T00:55:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.012+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 >> 172.21.15.119:0/3978232051 conn(0x7f18d006a320 msgr2=0x7f18d006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.013+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 shutdown_connections 2023-10-08T00:55:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.013+0000 7f098a59d700 1 -- 172.21.15.119:0/3756006746 wait complete. 2023-10-08T00:55:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.014+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 shutdown_connections 2023-10-08T00:55:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.014+0000 7f18b4ff9700 1 -- 172.21.15.119:0/3978232051 wait complete. 2023-10-08T00:55:54.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.016+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 --> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea340ad410 con 0x7fea2006f020 2023-10-08T00:55:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.017+0000 7fea1affd700 1 -- 172.21.15.119:0/1840003330 <== osd.6 v2:172.21.15.184:6808/86002 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea340ad410 con 0x7fea2006f020 2023-10-08T00:55:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.018+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7fea2006f020 msgr2=0x7fea20071430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.018+0000 7fea399ff700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7fea2006f020 0x7fea20071430 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.020+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066f80 msgr2=0x7fea20069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.020+0000 7fea399ff700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066f80 0x7fea20069430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fea34151700 tx=0x7fea2401b350 comp rx=0 tx=0).stop 2023-10-08T00:55:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.020+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea3406e250 msgr2=0x7fea3414e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.020+0000 7fea399ff700 1 --2- 172.21.15.119:0/1840003330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea3406e250 0x7fea3414e5b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00cbc0 tx=0x7fea2c001770 comp rx=0 tx=0).stop 2023-10-08T00:55:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.020+0000 7fea337fe700 1 -- 172.21.15.119:0/1840003330 reap_dead start 2023-10-08T00:55:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.021+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 shutdown_connections 2023-10-08T00:55:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.021+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 >> 172.21.15.119:0/1840003330 conn(0x7fea340cd000 msgr2=0x7fea340b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.021+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 shutdown_connections 2023-10-08T00:55:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.021+0000 7fea399ff700 1 -- 172.21.15.119:0/1840003330 wait complete. 2023-10-08T00:55:54.059 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:54.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T00:55:54.061 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:54.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T00:55:54.062 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2023-10-08T00:55:54.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T00:55:54.065 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:54.066 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T00:55:54.066 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:54.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T00:55:54.067 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2023-10-08T00:55:54.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T00:55:54.068 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2023-10-08T00:55:54.069 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T00:55:54.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.418+0000 7f92613ff700 1 Processor -- start 2023-10-08T00:55:54.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.418+0000 7f92613ff700 1 -- start start 2023-10-08T00:55:54.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.420+0000 7f92613ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f925c159840 0x7f925c0b3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.420+0000 7f92613ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c0cd8b0 0x7f925c0cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.420+0000 7f92613ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c15a680 con 0x7f925c159470 2023-10-08T00:55:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.420+0000 7f92613ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c15ae90 con 0x7f925c159840 2023-10-08T00:55:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.420+0000 7f92613ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0b5070 con 0x7f925c0cd8b0 2023-10-08T00:55:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.421+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c0cd8b0 0x7f925c0cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.421+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c0cd8b0 0x7f925c0cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34474/0 (socket says 172.21.15.119:34474) 2023-10-08T00:55:54.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.421+0000 7f925b7fe700 1 -- 172.21.15.119:0/2775097508 learned_addr learned my addr 172.21.15.119:0/2775097508 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.422+0000 7f925b7fe700 1 -- 172.21.15.119:0/2775097508 >> v2:172.21.15.119:3300/0 conn(0x7f925c159840 msgr2=0x7f925c0b3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.422+0000 7f925b7fe700 1 --2- 172.21.15.119:0/2775097508 >> v2:172.21.15.119:3300/0 conn(0x7f925c159840 0x7f925c0b3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.422+0000 7f925b7fe700 1 -- 172.21.15.119:0/2775097508 >> v1:172.21.15.184:6789/0 conn(0x7f925c159470 legacy=0x7f925c15b100 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T00:55:54.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.423+0000 7f925b7fe700 1 -- 172.21.15.119:0/2775097508 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c0b4d40 con 0x7f925c0cd8b0 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.419+0000 7f2c5ef9d700 1 Processor -- start 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.420+0000 7f2c5ef9d700 1 -- start start 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.422+0000 7f2c5ef9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2c500506c0 0x7f2c500e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.422+0000 7f2c5d79a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2c500de400 0x7f2c5004ffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35372/0 (socket says 172.21.15.119:35372) 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.422+0000 7f2c5df9b700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f2c500de030 0x7f2c50053330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:55978/0 (socket says 172.21.15.119:55978) 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.423+0000 7f2c5d79a700 1 -- 172.21.15.119:0/1067561397 learned_addr learned my addr 172.21.15.119:0/1067561397 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.423+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/1067561397 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c50054690 con 0x7f2c500de400 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.423+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/1067561397 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c500061f0 con 0x7f2c500de030 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.423+0000 7f2c5e79c700 1 --2- 172.21.15.119:0/1067561397 >> v2:172.21.15.184:3300/0 conn(0x7f2c500506c0 0x7f2c500e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.423+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/1067561397 --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c50053df0 con 0x7f2c500506c0 2023-10-08T00:55:54.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.423+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661842643 0 0) 0x7f2c50054690 con 0x7f2c500de400 2023-10-08T00:55:54.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.423+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c44003570 con 0x7f2c500de400 2023-10-08T00:55:54.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361577200 0 0) 0x7f2c500061f0 con 0x7f2c500de030 2023-10-08T00:55:54.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c50054690 con 0x7f2c500de030 2023-10-08T00:55:54.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f925b7fe700 1 --2- 172.21.15.119:0/2775097508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c0cd8b0 0x7f925c0cf100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9254009ec0 tx=0x7f92540097e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99af73e9b50aa8d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f9259ffb700 1 -- 172.21.15.119:0/2775097508 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400e070 con 0x7f925c0cd8b0 2023-10-08T00:55:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f9259ffb700 1 -- 172.21.15.119:0/2775097508 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925400bbb0 con 0x7f925c0cd8b0 2023-10-08T00:55:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1998330784 0 0) 0x7f2c44003570 con 0x7f2c500de400 2023-10-08T00:55:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 >> v2:172.21.15.184:3300/0 conn(0x7f2c500506c0 msgr2=0x7f2c500e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f2c5cf99700 1 --2- 172.21.15.119:0/1067561397 >> v2:172.21.15.184:3300/0 conn(0x7f2c500506c0 0x7f2c500e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 >> v1:172.21.15.184:6789/0 conn(0x7f2c500de030 legacy=0x7f2c50053330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500540d0 con 0x7f2c500de400 2023-10-08T00:55:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.424+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f2c54001470 con 0x7f2c500de400 2023-10-08T00:55:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 >> v1:172.21.15.119:6789/0 conn(0x7f2c500de400 legacy=0x7f2c5004ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f2c5cf99700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c44003f50 0x7f2c44004320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f2c5cf99700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c44004990 0x7f2c44004e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f92613ff700 1 -- 172.21.15.119:0/2775097508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c0cd8b0 msgr2=0x7f925c0cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f92613ff700 1 --2- 172.21.15.119:0/2775097508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c0cd8b0 0x7f925c0cf100 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9254009ec0 tx=0x7f92540097e0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.426+0000 7f92613ff700 1 -- 172.21.15.119:0/2775097508 shutdown_connections 2023-10-08T00:55:54.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.426+0000 7f92613ff700 1 --2- 172.21.15.119:0/2775097508 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c0cd8b0 0x7f925c0cf100 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.426+0000 7f92613ff700 1 --2- 172.21.15.119:0/2775097508 >> v2:172.21.15.119:3300/0 conn(0x7f925c159840 0x7f925c0b3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.426+0000 7f92613ff700 1 -- 172.21.15.119:0/2775097508 >> 172.21.15.119:0/2775097508 conn(0x7f925c0d0ff0 msgr2=0x7f925c0ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.426+0000 7f92613ff700 1 -- 172.21.15.119:0/2775097508 shutdown_connections 2023-10-08T00:55:54.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f2c5cf99700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c44005550 0x7f2c4400e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c480010e0 con 0x7f2c44004990 2023-10-08T00:55:54.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50053df0 con 0x7f2c44005550 2023-10-08T00:55:54.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4400e5c0 con 0x7f2c44003f50 2023-10-08T00:55:54.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.425+0000 7f2c5df9b700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c44003f50 0x7f2c44004320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.426+0000 7f2c5d79a700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c44004990 0x7f2c44004e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.427+0000 7f2c5d79a700 1 -- 172.21.15.119:0/1067561397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c44003f50 msgr2=0x7f2c44004320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.427+0000 7f2c5d79a700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c44003f50 0x7f2c44004320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.427+0000 7f2c5d79a700 1 -- 172.21.15.119:0/1067561397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c44005550 msgr2=0x7f2c4400e080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T00:55:54.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.427+0000 7f2c5d79a700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c44005550 0x7f2c4400e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.427+0000 7f2c5d79a700 1 -- 172.21.15.119:0/1067561397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44003570 con 0x7f2c44004990 2023-10-08T00:55:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.427+0000 7f2c5d79a700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c44004990 0x7f2c44004e00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c1e0 tx=0x7f2c54001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c7b71eac734305 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.427+0000 7f2c5e79c700 1 -- 172.21.15.119:0/1067561397 reap_dead start 2023-10-08T00:55:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.428+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400cb10 con 0x7f2c44004990 2023-10-08T00:55:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.428+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5400cc70 con 0x7f2c44004990 2023-10-08T00:55:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.428+0000 7f2c5cf99700 1 -- 172.21.15.119:0/1067561397 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54005920 con 0x7f2c44004990 2023-10-08T00:55:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.426+0000 7f92613ff700 1 -- 172.21.15.119:0/2775097508 wait complete. 2023-10-08T00:55:54.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.428+0000 7f92613ff700 1 Processor -- start 2023-10-08T00:55:54.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.428+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/1067561397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c44004990 msgr2=0x7f2c44004e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.429+0000 7f92613ff700 1 -- start start 2023-10-08T00:55:54.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.428+0000 7f2c5ef9d700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c44004990 0x7f2c44004e00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400c1e0 tx=0x7f2c54001d40 comp rx=0 tx=0).stop 2023-10-08T00:55:54.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.429+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/1067561397 shutdown_connections 2023-10-08T00:55:54.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.429+0000 7f2c5ef9d700 1 --2- 172.21.15.119:0/1067561397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c44004990 0x7f2c44004e00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.429+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/1067561397 >> 172.21.15.119:0/1067561397 conn(0x7f2c50054da0 msgr2=0x7f2c500551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.430+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/1067561397 shutdown_connections 2023-10-08T00:55:54.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.430+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/1067561397 wait complete. 2023-10-08T00:55:54.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.431+0000 7f92613ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c159470 0x7f925c06c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.431+0000 7f92613ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c159840 0x7f925c157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.431+0000 7f92613ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f925c16a9f0 0x7f925c06dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.430+0000 7f1696dff700 1 Processor -- start 2023-10-08T00:55:54.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.431+0000 7f1696dff700 1 -- start start 2023-10-08T00:55:54.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.431+0000 7f925affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c159470 0x7f925c06c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.432+0000 7f925affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c159470 0x7f925c06c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34492/0 (socket says 172.21.15.119:34492) 2023-10-08T00:55:54.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.432+0000 7f925b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f925c16a9f0 0x7f925c06dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.432+0000 7f925affd700 1 -- 172.21.15.119:0/308729036 learned_addr learned my addr 172.21.15.119:0/308729036 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.432+0000 7f92613ff700 1 -- 172.21.15.119:0/308729036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c06f0b0 con 0x7f925c16a9f0 2023-10-08T00:55:54.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.432+0000 7f925a7fc700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c159840 0x7f925c157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.432+0000 7f92613ff700 1 -- 172.21.15.119:0/308729036 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0c5a60 con 0x7f925c159840 2023-10-08T00:55:54.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.432+0000 7f92613ff700 1 -- 172.21.15.119:0/308729036 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c158250 con 0x7f925c159470 2023-10-08T00:55:54.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f1696dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16880e1e50 0x7f16880078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f1696dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1688007450 con 0x7f16880063b0 2023-10-08T00:55:54.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f1696dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1688006840 con 0x7f1688039340 2023-10-08T00:55:54.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f1696dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1688006b20 con 0x7f16880e1e50 2023-10-08T00:55:54.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f169539c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f1688039340 0x7f168803b060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:55998/0 (socket says 172.21.15.119:55998) 2023-10-08T00:55:54.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f169539c700 1 -- 172.21.15.119:0/610419052 learned_addr learned my addr 172.21.15.119:0/610419052 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:54.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f925b7fe700 1 -- 172.21.15.119:0/308729036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c159470 msgr2=0x7f925c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f925b7fe700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c159470 0x7f925c06c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f925b7fe700 1 -- 172.21.15.119:0/308729036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c159840 msgr2=0x7f925c157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2812282015 0 0) 0x7f1688006840 con 0x7f1688039340 2023-10-08T00:55:54.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1680003570 con 0x7f1688039340 2023-10-08T00:55:54.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f925b7fe700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c159840 0x7f925c157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f925b7fe700 1 -- 172.21.15.119:0/308729036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9254009a80 con 0x7f925c16a9f0 2023-10-08T00:55:54.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f1694b9b700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16880e1e50 0x7f16880078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051842267 0 0) 0x7f1680003570 con 0x7f1688039340 2023-10-08T00:55:54.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16880e1e50 msgr2=0x7f16880078e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f2c5ef9d700 1 Processor -- start 2023-10-08T00:55:54.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f2c5ef9d700 1 -- start start 2023-10-08T00:55:54.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f2c5ef9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c44004990 0x7f2c500d6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.434+0000 7f2c5ef9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5004cb10 0x7f2c500de940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f2c5ef9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c5004cf80 0x7f2c500dee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f2c5ef9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50038510 con 0x7f2c5004cb10 2023-10-08T00:55:54.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16880e1e50 0x7f16880078e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 >> v1:172.21.15.119:6789/0 conn(0x7f16880063b0 legacy=0x7f16880e2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688006e00 con 0x7f1688039340 2023-10-08T00:55:54.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f1684001470 con 0x7f1688039340 2023-10-08T00:55:54.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 >> v1:172.21.15.184:6789/0 conn(0x7f1688039340 legacy=0x7f168803b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680003ae0 0x7f1680003ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1680004810 0x7f1680004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1680005210 0x7f168000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1688006b20 con 0x7f1680003ae0 2023-10-08T00:55:54.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168000c100 con 0x7f1680005210 2023-10-08T00:55:54.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168000c2c0 con 0x7f1680004810 2023-10-08T00:55:54.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f2c5ef9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c500d7150 con 0x7f2c44004990 2023-10-08T00:55:54.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f2c5ef9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c50037be0 con 0x7f2c5004cf80 2023-10-08T00:55:54.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f2c5d79a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5004cb10 0x7f2c500de940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f169539c700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1680005210 0x7f168000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f628edff700 1 Processor -- start 2023-10-08T00:55:54.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f2c5d79a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5004cb10 0x7f2c500de940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42060/0 (socket says 172.21.15.119:42060) 2023-10-08T00:55:54.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f2c5d79a700 1 -- 172.21.15.119:0/3789643507 learned_addr learned my addr 172.21.15.119:0/3789643507 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f925b7fe700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f925c16a9f0 0x7f925c06dcf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f925c065b70 tx=0x7f9254019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f169539c700 1 -- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1680004810 msgr2=0x7f1680004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:55:54.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f169539c700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1680004810 0x7f1680004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f169539c700 1 -- 172.21.15.119:0/610419052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680003ae0 msgr2=0x7f1680003ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f169539c700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680003ae0 0x7f1680003ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f169539c700 1 -- 172.21.15.119:0/610419052 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688006e00 con 0x7f1680005210 2023-10-08T00:55:54.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f2c5d79a700 1 -- 172.21.15.119:0/3789643507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c5004cf80 msgr2=0x7f2c500dee80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:55:54.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f2c5d79a700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c5004cf80 0x7f2c500dee80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f2c5d79a700 1 -- 172.21.15.119:0/3789643507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c44004990 msgr2=0x7f2c500d6c10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:55:54.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f2c5d79a700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c44004990 0x7f2c500d6c10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f2c5d79a700 1 -- 172.21.15.119:0/3789643507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c50037ec0 con 0x7f2c5004cb10 2023-10-08T00:55:54.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f628edff700 1 -- start start 2023-10-08T00:55:54.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f9242ffd700 1 -- 172.21.15.119:0/308729036 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254026420 con 0x7f925c16a9f0 2023-10-08T00:55:54.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f169539c700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1680005210 0x7f168000ba60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f168400c320 tx=0x7f168400c630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57d1f169d13c749 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f92613ff700 1 -- 172.21.15.119:0/308729036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c06c890 con 0x7f925c16a9f0 2023-10-08T00:55:54.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168400ccc0 con 0x7f1680005210 2023-10-08T00:55:54.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168400ce20 con 0x7f1680005210 2023-10-08T00:55:54.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f168f7fe700 1 -- 172.21.15.119:0/610419052 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1684005b50 con 0x7f1680005210 2023-10-08T00:55:54.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f92613ff700 1 -- 172.21.15.119:0/308729036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c06ce50 con 0x7f925c16a9f0 2023-10-08T00:55:54.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f9242ffd700 1 -- 172.21.15.119:0/308729036 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9254020070 con 0x7f925c16a9f0 2023-10-08T00:55:54.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f9242ffd700 1 -- 172.21.15.119:0/308729036 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540267d0 con 0x7f925c16a9f0 2023-10-08T00:55:54.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f2c5d79a700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5004cb10 0x7f2c500de940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005d60 tx=0x7f2c54015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f2c4dffb700 1 -- 172.21.15.119:0/3789643507 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54023070 con 0x7f2c5004cb10 2023-10-08T00:55:54.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f2c4dffb700 1 -- 172.21.15.119:0/3789643507 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c54004830 con 0x7f2c5004cb10 2023-10-08T00:55:54.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f2c4dffb700 1 -- 172.21.15.119:0/3789643507 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54004bb0 con 0x7f2c5004cb10 2023-10-08T00:55:54.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/3789643507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c500381a0 con 0x7f2c5004cb10 2023-10-08T00:55:54.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/3789643507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c500389a0 con 0x7f2c5004cb10 2023-10-08T00:55:54.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f1696dff700 1 -- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1680005210 msgr2=0x7f168000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f1696dff700 1 --2- 172.21.15.119:0/610419052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1680005210 0x7f168000ba60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f168400c320 tx=0x7f168400c630 comp rx=0 tx=0).stop 2023-10-08T00:55:54.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f169539c700 1 -- 172.21.15.119:0/610419052 reap_dead start 2023-10-08T00:55:54.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f1696dff700 1 -- 172.21.15.119:0/610419052 shutdown_connections 2023-10-08T00:55:54.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f1696dff700 1 -- 172.21.15.119:0/610419052 >> 172.21.15.119:0/610419052 conn(0x7f1688059cd0 msgr2=0x7f1688055c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f137299d700 1 Processor -- start 2023-10-08T00:55:54.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.433+0000 7f137299d700 1 -- start start 2023-10-08T00:55:54.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f137299d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1364007140 0x7f1364033d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f137299d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1364054870 con 0x7f13640503e0 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f137299d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1364053c60 con 0x7f1364033960 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.435+0000 7f137299d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1364053a60 con 0x7f1364007140 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f137119a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f13640503e0 0x7f13640e6190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35384/0 (socket says 172.21.15.119:35384) 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.436+0000 7f137119a700 1 -- 172.21.15.119:0/1836477186 learned_addr learned my addr 172.21.15.119:0/1836477186 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f1370999700 1 -- 172.21.15.119:0/1836477186 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453485988 0 0) 0x7f1364054870 con 0x7f13640503e0 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f1370999700 1 -- 172.21.15.119:0/1836477186 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1358003570 con 0x7f13640503e0 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.439+0000 7f137219c700 1 --2- 172.21.15.119:0/1836477186 >> v2:172.21.15.184:3300/0 conn(0x7f1364007140 0x7f1364033d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.440+0000 7f137219c700 1 -- 172.21.15.119:0/1836477186 >> v1:172.21.15.184:6789/0 conn(0x7f1364033960 legacy=0x7f136404fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.440+0000 7f137219c700 1 -- 172.21.15.119:0/1836477186 >> v1:172.21.15.119:6789/0 conn(0x7f13640503e0 legacy=0x7f13640e6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.440+0000 7f137219c700 1 -- 172.21.15.119:0/1836477186 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1364053f40 con 0x7f1364007140 2023-10-08T00:55:54.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.440+0000 7f137219c700 1 --2- 172.21.15.119:0/1836477186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1364007140 0x7f1364033d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f136c000eb0 tx=0x7f136c066260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a15b5758621e076 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f628edff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6288068aa0 0x7f6288068e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f628edff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6288166960 0x7f62880ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f628edff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62880ad400 con 0x7f62880abe00 2023-10-08T00:55:54.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f628edff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6288087e70 con 0x7f6288068aa0 2023-10-08T00:55:54.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.437+0000 7f628edff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62880693b0 con 0x7f6288166960 2023-10-08T00:55:54.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f628d39c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f62880abe00 0x7f628816a270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35394/0 (socket says 172.21.15.119:35394) 2023-10-08T00:55:54.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f628d39c700 1 -- 172.21.15.119:0/4143312221 learned_addr learned my addr 172.21.15.119:0/4143312221 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:54.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.440+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/3789643507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c500387f0 con 0x7f2c5004cb10 2023-10-08T00:55:54.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.440+0000 7f1696dff700 1 -- 172.21.15.119:0/610419052 shutdown_connections 2023-10-08T00:55:54.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.441+0000 7f1696dff700 1 -- 172.21.15.119:0/610419052 wait complete. 2023-10-08T00:55:54.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.441+0000 7f92613ff700 1 -- 172.21.15.119:0/308729036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c0052b0 con 0x7f925c16a9f0 2023-10-08T00:55:54.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.442+0000 7f1370999700 1 -- 172.21.15.119:0/1836477186 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c067610 con 0x7f1364007140 2023-10-08T00:55:54.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.442+0000 7f1370999700 1 -- 172.21.15.119:0/1836477186 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c067790 con 0x7f1364007140 2023-10-08T00:55:54.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.442+0000 7f1370999700 1 -- 172.21.15.119:0/1836477186 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0647a0 con 0x7f1364007140 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f628cb9b700 1 --2- 172.21.15.119:0/4143312221 >> v2:172.21.15.119:3300/0 conn(0x7f6288068aa0 0x7f6288068e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.438+0000 7f6287fff700 1 --2- 172.21.15.119:0/4143312221 >> v2:172.21.15.184:3300/0 conn(0x7f6288166960 0x7f62880ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.439+0000 7f628cb9b700 1 -- 172.21.15.119:0/4143312221 >> v2:172.21.15.184:3300/0 conn(0x7f6288166960 msgr2=0x7f62880ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.439+0000 7f628cb9b700 1 --2- 172.21.15.119:0/4143312221 >> v2:172.21.15.184:3300/0 conn(0x7f6288166960 0x7f62880ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.439+0000 7f628cb9b700 1 -- 172.21.15.119:0/4143312221 >> v1:172.21.15.119:6789/0 conn(0x7f62880abe00 legacy=0x7f628816a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.439+0000 7f628cb9b700 1 -- 172.21.15.119:0/4143312221 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62880ac610 con 0x7f6288068aa0 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.440+0000 7f628cb9b700 1 --2- 172.21.15.119:0/4143312221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288068aa0 0x7f6288068e70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f627c00a9f0 tx=0x7f627c00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae700335be0b639 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.442+0000 7f62877fe700 1 -- 172.21.15.119:0/4143312221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c00ad60 con 0x7f6288068aa0 2023-10-08T00:55:54.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.442+0000 7f62877fe700 1 -- 172.21.15.119:0/4143312221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f627c0158c0 con 0x7f6288068aa0 2023-10-08T00:55:54.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.442+0000 7f2c4dffb700 1 -- 172.21.15.119:0/3789643507 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c54019070 con 0x7f2c5004cb10 2023-10-08T00:55:54.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.442+0000 7f62877fe700 1 -- 172.21.15.119:0/4143312221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627c015c80 con 0x7f6288068aa0 2023-10-08T00:55:54.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.442+0000 7f9242ffd700 1 -- 172.21.15.119:0/308729036 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9254026b80 con 0x7f925c16a9f0 2023-10-08T00:55:54.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f628edff700 1 -- 172.21.15.119:0/4143312221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288068aa0 msgr2=0x7f6288068e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f628edff700 1 --2- 172.21.15.119:0/4143312221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288068aa0 0x7f6288068e70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f627c00a9f0 tx=0x7f627c00d040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f54873ff700 1 Processor -- start 2023-10-08T00:55:54.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f1696dff700 1 Processor -- start 2023-10-08T00:55:54.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f1696dff700 1 -- start start 2023-10-08T00:55:54.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f137299d700 1 -- 172.21.15.119:0/1836477186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1364007140 msgr2=0x7f1364033d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f137299d700 1 --2- 172.21.15.119:0/1836477186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1364007140 0x7f1364033d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f136c000eb0 tx=0x7f136c066260 comp rx=0 tx=0).stop 2023-10-08T00:55:54.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f54873ff700 1 -- start start 2023-10-08T00:55:54.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f5f987ff700 1 Processor -- start 2023-10-08T00:55:54.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.443+0000 7f2c4dffb700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2c3c067080 0x7f2c3c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.444+0000 7f2c4dffb700 1 -- 172.21.15.119:0/3789643507 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2c54085400 con 0x7f2c5004cb10 2023-10-08T00:55:54.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.444+0000 7f5f987ff700 1 -- start start 2023-10-08T00:55:54.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.444+0000 7f9242ffd700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9248067000 0x7f92480694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.444+0000 7f2c5df9b700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2c3c067080 0x7f2c3c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.444+0000 7f925affd700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9248067000 0x7f92480694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f137299d700 1 -- 172.21.15.119:0/1836477186 shutdown_connections 2023-10-08T00:55:54.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f137299d700 1 --2- 172.21.15.119:0/1836477186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1364007140 0x7f1364033d30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f137299d700 1 -- 172.21.15.119:0/1836477186 >> 172.21.15.119:0/1836477186 conn(0x7f13640516f0 msgr2=0x7f1364051af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f1696dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680005210 0x7f168804c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f628edff700 1 -- 172.21.15.119:0/4143312221 shutdown_connections 2023-10-08T00:55:54.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f628edff700 1 --2- 172.21.15.119:0/4143312221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288068aa0 0x7f6288068e70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f628edff700 1 --2- 172.21.15.119:0/4143312221 >> v2:172.21.15.184:3300/0 conn(0x7f6288166960 0x7f62880ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f628edff700 1 -- 172.21.15.119:0/4143312221 >> 172.21.15.119:0/4143312221 conn(0x7f62880cd000 msgr2=0x7f6288068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f2c5df9b700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2c3c067080 0x7f2c3c069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c48001010 tx=0x7f2c48001b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f628edff700 1 -- 172.21.15.119:0/4143312221 shutdown_connections 2023-10-08T00:55:54.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f628edff700 1 -- 172.21.15.119:0/4143312221 wait complete. 2023-10-08T00:55:54.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f1696dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1688050bd0 0x7f16880d44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f1696dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f168804eec0 0x7f168804f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f1696dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168804f850 con 0x7f1680005210 2023-10-08T00:55:54.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f1696dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168804e1c0 con 0x7f1688050bd0 2023-10-08T00:55:54.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.445+0000 7f1696dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1688051190 con 0x7f168804eec0 2023-10-08T00:55:54.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f54873ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480159890 0x7f548015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f1694b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680005210 0x7f168804c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f1694b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680005210 0x7f168804c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42086/0 (socket says 172.21.15.119:42086) 2023-10-08T00:55:54.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f1694b9b700 1 -- 172.21.15.119:0/3564820369 learned_addr learned my addr 172.21.15.119:0/3564820369 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f5f987ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f90159750 0x7f5f900b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f54873ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5480069560 0x7f54800699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f5f987ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5f900b5ab0 0x7f5f900b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f5f987ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5f90159b20 0x7f5f900b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f5f987ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900ad730 con 0x7f5f90159b20 2023-10-08T00:55:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f548599c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5480069560 0x7f54800699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f548599c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5480069560 0x7f54800699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54808/0 (socket says 172.21.15.119:54808) 2023-10-08T00:55:54.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f5f987ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900cb5d0 con 0x7f5f900b5ab0 2023-10-08T00:55:54.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f548499a700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f5480159c60 0x7f548015d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:56012/0 (socket says 172.21.15.119:56012) 2023-10-08T00:55:54.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f5f987ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900cb880 con 0x7f5f90159750 2023-10-08T00:55:54.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f548599c700 1 -- 172.21.15.119:0/1047784651 learned_addr learned my addr 172.21.15.119:0/1047784651 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f5f9759d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5f90159b20 0x7f5f900b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f1694b9b700 1 -- 172.21.15.119:0/3564820369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f168804eec0 msgr2=0x7f168804f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f1694b9b700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f168804eec0 0x7f168804f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f1694b9b700 1 -- 172.21.15.119:0/3564820369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1688050bd0 msgr2=0x7f16880d44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f5f9759d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5f90159b20 0x7f5f900b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42100/0 (socket says 172.21.15.119:42100) 2023-10-08T00:55:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f1694b9b700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1688050bd0 0x7f16880d44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f1694b9b700 1 -- 172.21.15.119:0/3564820369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688051470 con 0x7f1680005210 2023-10-08T00:55:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f5f9759d700 1 -- 172.21.15.119:0/2447381427 learned_addr learned my addr 172.21.15.119:0/2447381427 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f54873ff700 1 -- 172.21.15.119:0/1047784651 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548015a160 con 0x7f5480159c60 2023-10-08T00:55:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f54873ff700 1 -- 172.21.15.119:0/1047784651 --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5480069f20 con 0x7f5480069560 2023-10-08T00:55:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f54873ff700 1 -- 172.21.15.119:0/1047784651 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54800d13a0 con 0x7f5480159890 2023-10-08T00:55:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f1694b9b700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680005210 0x7f168804c8c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f167c00d810 tx=0x7f167c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.447+0000 7f628edff700 1 Processor -- start 2023-10-08T00:55:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5f9759d700 1 -- 172.21.15.119:0/2447381427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f90159750 msgr2=0x7f5f900b3c60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T00:55:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5f9759d700 1 --2- 172.21.15.119:0/2447381427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f90159750 0x7f5f900b3c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5f9759d700 1 -- 172.21.15.119:0/2447381427 >> v2:172.21.15.184:3300/0 conn(0x7f5f900b5ab0 msgr2=0x7f5f900b4db0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-08T00:55:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5f9759d700 1 --2- 172.21.15.119:0/2447381427 >> v2:172.21.15.184:3300/0 conn(0x7f5f900b5ab0 0x7f5f900b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5f9759d700 1 -- 172.21.15.119:0/2447381427 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900cbb60 con 0x7f5f90159b20 2023-10-08T00:55:54.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5477fff700 1 -- 172.21.15.119:0/1047784651 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1403149456 0 0) 0x7f548015a160 con 0x7f5480159c60 2023-10-08T00:55:54.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5477fff700 1 -- 172.21.15.119:0/1047784651 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f546c003570 con 0x7f5480159c60 2023-10-08T00:55:54.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f548519b700 1 --2- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480159890 0x7f548015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5f9759d700 1 --2- 172.21.15.119:0/2447381427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159b20 0x7f5f900b52f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5f90089520 tx=0x7f5f80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450418398b95ba83 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f548599c700 1 -- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480159890 msgr2=0x7f548015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f5477fff700 1 -- 172.21.15.119:0/1047784651 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1562304893 0 0) 0x7f546c003570 con 0x7f5480159c60 2023-10-08T00:55:54.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f548599c700 1 --2- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480159890 0x7f548015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f548599c700 1 -- 172.21.15.119:0/1047784651 >> v1:172.21.15.184:6789/0 conn(0x7f5480159c60 legacy=0x7f548015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f548599c700 1 -- 172.21.15.119:0/1047784651 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548015e140 con 0x7f5480069560 2023-10-08T00:55:54.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f628edff700 1 -- start start 2023-10-08T00:55:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7f548519b700 1 --2- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480159890 0x7f548015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f548599c700 1 --2- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480069560 0x7f54800699e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5478009f20 tx=0x7f5478009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aff3e890ac8de01 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.448+0000 7efd4ed9d700 1 Processor -- start 2023-10-08T00:55:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7efd4ed9d700 1 -- start start 2023-10-08T00:55:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f5477fff700 1 -- 172.21.15.119:0/1047784651 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547800e070 con 0x7f5480069560 2023-10-08T00:55:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f5477fff700 1 -- 172.21.15.119:0/1047784651 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54780090d0 con 0x7f5480069560 2023-10-08T00:55:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f5477fff700 1 -- 172.21.15.119:0/1047784651 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470001450 con 0x7f5480069560 2023-10-08T00:55:54.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f925affd700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9248067000 0x7f92480694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9250009aa0 tx=0x7f9250008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628edff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288068aa0 0x7f628806aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628edff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62880abe00 0x7f628806b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628edff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288166960 0x7f628817a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628edff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288154960 con 0x7f6288166960 2023-10-08T00:55:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628edff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6288153d50 con 0x7f6288068aa0 2023-10-08T00:55:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628edff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6288154030 con 0x7f62880abe00 2023-10-08T00:55:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f9242ffd700 1 -- 172.21.15.119:0/308729036 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f9254022070 con 0x7f925c16a9f0 2023-10-08T00:55:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628d39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288166960 0x7f628817a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628d39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288166960 0x7f628817a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42110/0 (socket says 172.21.15.119:42110) 2023-10-08T00:55:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f628d39c700 1 -- 172.21.15.119:0/2940428857 learned_addr learned my addr 172.21.15.119:0/2940428857 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f137299d700 1 -- 172.21.15.119:0/1836477186 shutdown_connections 2023-10-08T00:55:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.446+0000 7f137299d700 1 -- 172.21.15.119:0/1836477186 wait complete. 2023-10-08T00:55:54.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f137299d700 1 Processor -- start 2023-10-08T00:55:54.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.449+0000 7f137299d700 1 -- start start 2023-10-08T00:55:54.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f628cb9b700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288068aa0 0x7f628806aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f6287fff700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62880abe00 0x7f628806b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f628d39c700 1 -- 172.21.15.119:0/2940428857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62880abe00 msgr2=0x7f628806b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f628d39c700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62880abe00 0x7f628806b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f54873ff700 1 -- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480069560 msgr2=0x7f54800699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f54873ff700 1 --2- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480069560 0x7f54800699e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5478009f20 tx=0x7f5478009840 comp rx=0 tx=0).stop 2023-10-08T00:55:54.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f628d39c700 1 -- 172.21.15.119:0/2940428857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288068aa0 msgr2=0x7f628806aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f628d39c700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288068aa0 0x7f628806aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f628d39c700 1 -- 172.21.15.119:0/2940428857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627c00a4d0 con 0x7f6288166960 2023-10-08T00:55:54.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f54873ff700 1 -- 172.21.15.119:0/1047784651 shutdown_connections 2023-10-08T00:55:54.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f54873ff700 1 --2- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480069560 0x7f54800699e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f54873ff700 1 --2- 172.21.15.119:0/1047784651 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480159890 0x7f548015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f54873ff700 1 -- 172.21.15.119:0/1047784651 >> 172.21.15.119:0/1047784651 conn(0x7f5480068e20 msgr2=0x7f54800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f137299d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1364007140 0x7f1364037ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f137299d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364033960 0x7f13640cf940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f54873ff700 1 -- 172.21.15.119:0/1047784651 shutdown_connections 2023-10-08T00:55:54.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f628d39c700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288166960 0x7f628817a200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f627800c840 tx=0x7f6278007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.450+0000 7f54873ff700 1 -- 172.21.15.119:0/1047784651 wait complete. 2023-10-08T00:55:54.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f137299d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13640503e0 0x7f13640c1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f137299d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1364038410 con 0x7f1364007140 2023-10-08T00:55:54.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f137299d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13640381e0 con 0x7f13640503e0 2023-10-08T00:55:54.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f137299d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13640cfe80 con 0x7f1364033960 2023-10-08T00:55:54.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd48147550 0x7efd480696c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f137119a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364033960 0x7f13640cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4ed9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd480ca290 0x7efd480c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4ed9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd48069f10 con 0x7efd480c6f80 2023-10-08T00:55:54.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4ed9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd480c7400 con 0x7efd480ca290 2023-10-08T00:55:54.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4ed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4806a1f0 con 0x7efd48147550 2023-10-08T00:55:54.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f137119a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364033960 0x7f13640cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34544/0 (socket says 172.21.15.119:34544) 2023-10-08T00:55:54.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f137119a700 1 -- 172.21.15.119:0/2738696081 learned_addr learned my addr 172.21.15.119:0/2738696081 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4dd9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd48147550 0x7efd480696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f137219c700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13640503e0 0x7f13640c1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4dd9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd48147550 0x7efd480696c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34546/0 (socket says 172.21.15.119:34546) 2023-10-08T00:55:54.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4dd9b700 1 -- 172.21.15.119:0/1112261780 learned_addr learned my addr 172.21.15.119:0/1112261780 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7efd4d59a700 1 --2- 172.21.15.119:0/1112261780 >> v2:172.21.15.119:3300/0 conn(0x7efd480ca290 0x7efd480c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7efd4dd9b700 1 -- 172.21.15.119:0/1112261780 >> v2:172.21.15.119:3300/0 conn(0x7efd480ca290 msgr2=0x7efd480c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:54.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.451+0000 7f168cff9700 1 -- 172.21.15.119:0/3564820369 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c0125f0 con 0x7f1680005210 2023-10-08T00:55:54.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f6284ff9700 1 -- 172.21.15.119:0/2940428857 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278005900 con 0x7f6288166960 2023-10-08T00:55:54.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1688051750 con 0x7f1680005210 2023-10-08T00:55:54.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7efd4dd9b700 1 --2- 172.21.15.119:0/1112261780 >> v2:172.21.15.119:3300/0 conn(0x7efd480ca290 0x7efd480c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7efd4dd9b700 1 -- 172.21.15.119:0/1112261780 >> v1:172.21.15.119:6789/0 conn(0x7efd480c6f80 legacy=0x7efd4806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7efd4dd9b700 1 -- 172.21.15.119:0/1112261780 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4806a4d0 con 0x7efd48147550 2023-10-08T00:55:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16880e2f80 con 0x7f1680005210 2023-10-08T00:55:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f2c4dffb700 1 -- 172.21.15.119:0/3789643507 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c54052700 con 0x7f2c5004cb10 2023-10-08T00:55:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f137219c700 1 -- 172.21.15.119:0/2738696081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364033960 msgr2=0x7f13640cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f137219c700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364033960 0x7f13640cf940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f137219c700 1 -- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1364007140 msgr2=0x7f1364037ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f137219c700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1364007140 0x7f1364037ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f137219c700 1 -- 172.21.15.119:0/2738696081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c04c9a0 con 0x7f13640503e0 2023-10-08T00:55:54.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7efd4dd9b700 1 --2- 172.21.15.119:0/1112261780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd48147550 0x7efd480696c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efd440099b0 tx=0x7efd44009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68c5731cd2fc8f73 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7efd4cd99700 1 -- 172.21.15.119:0/1112261780 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4400b860 con 0x7efd48147550 2023-10-08T00:55:54.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7efd4cd99700 1 -- 172.21.15.119:0/1112261780 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4400b9c0 con 0x7efd48147550 2023-10-08T00:55:54.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.453+0000 7efd4cd99700 1 -- 172.21.15.119:0/1112261780 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4400bd40 con 0x7efd48147550 2023-10-08T00:55:54.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.453+0000 7f5f95599700 1 -- 172.21.15.119:0/2447381427 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000b860 con 0x7f5f90159b20 2023-10-08T00:55:54.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.453+0000 7f54873ff700 1 Processor -- start 2023-10-08T00:55:54.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.453+0000 7f137219c700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13640503e0 0x7f13640c1220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f136c04c860 tx=0x7f136c08b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.453+0000 7f54873ff700 1 -- start start 2023-10-08T00:55:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6288154310 con 0x7f6288166960 2023-10-08T00:55:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.452+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6288154df0 con 0x7f6288166960 2023-10-08T00:55:54.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.453+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1688005530 con 0x7f1680005210 2023-10-08T00:55:54.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.453+0000 7efd4ed9d700 1 -- 172.21.15.119:0/1112261780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd48147550 msgr2=0x7efd480696c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.453+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/1112261780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd48147550 0x7efd480696c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efd440099b0 tx=0x7efd44009250 comp rx=0 tx=0).stop 2023-10-08T00:55:54.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7f1361ffb700 1 -- 172.21.15.119:0/2738696081 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c096070 con 0x7f13640503e0 2023-10-08T00:55:54.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7f1361ffb700 1 -- 172.21.15.119:0/2738696081 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136c067610 con 0x7f13640503e0 2023-10-08T00:55:54.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7f1361ffb700 1 -- 172.21.15.119:0/2738696081 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136c0908b0 con 0x7f13640503e0 2023-10-08T00:55:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7efd4ed9d700 1 -- 172.21.15.119:0/1112261780 shutdown_connections 2023-10-08T00:55:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/1112261780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd48147550 0x7efd480696c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/1112261780 >> v2:172.21.15.119:3300/0 conn(0x7efd480ca290 0x7efd480c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7efd4ed9d700 1 -- 172.21.15.119:0/1112261780 >> 172.21.15.119:0/1112261780 conn(0x7efd480cc1f0 msgr2=0x7efd480cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f628804c3c0 con 0x7f6288166960 2023-10-08T00:55:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7efd4ed9d700 1 -- 172.21.15.119:0/1112261780 shutdown_connections 2023-10-08T00:55:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7f137299d700 1 -- 172.21.15.119:0/2738696081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1364039100 con 0x7f13640503e0 2023-10-08T00:55:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7f137299d700 1 -- 172.21.15.119:0/2738696081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13640387d0 con 0x7f13640503e0 2023-10-08T00:55:54.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7efd4ed9d700 1 -- 172.21.15.119:0/1112261780 wait complete. 2023-10-08T00:55:54.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.454+0000 7f54873ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480069560 0x7f548015b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f54873ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480159890 0x7f5480171e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f548519b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480069560 0x7f548015b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f548519b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480069560 0x7f548015b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34556/0 (socket says 172.21.15.119:34556) 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f54873ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5480159c60 0x7f54800b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f548499a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480159890 0x7f5480171e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f548499a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480159890 0x7f5480171e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54824/0 (socket says 172.21.15.119:54824) 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f548519b700 1 -- 172.21.15.119:0/2970778708 learned_addr learned my addr 172.21.15.119:0/2970778708 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f548599c700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5480159c60 0x7f54800b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f54873ff700 1 -- 172.21.15.119:0/2970778708 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480171980 con 0x7f5480159c60 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f54873ff700 1 -- 172.21.15.119:0/2970778708 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548015bd00 con 0x7f5480159890 2023-10-08T00:55:54.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.455+0000 7f54873ff700 1 -- 172.21.15.119:0/2970778708 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548015bfe0 con 0x7f5480069560 2023-10-08T00:55:54.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.456+0000 7f137299d700 1 -- 172.21.15.119:0/2738696081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1364051ac0 con 0x7f13640503e0 2023-10-08T00:55:54.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.456+0000 7f548519b700 1 -- 172.21.15.119:0/2970778708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480159890 msgr2=0x7f5480171e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.456+0000 7f548519b700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480159890 0x7f5480171e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.456+0000 7f548519b700 1 -- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5480159c60 msgr2=0x7f54800b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.456+0000 7f548519b700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5480159c60 0x7f54800b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.456+0000 7f548519b700 1 -- 172.21.15.119:0/2970778708 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5478009ae0 con 0x7f5480069560 2023-10-08T00:55:54.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.456+0000 7efd4ed9d700 1 Processor -- start 2023-10-08T00:55:54.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.457+0000 7efd4ed9d700 1 -- start start 2023-10-08T00:55:54.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.457+0000 7f1361ffb700 1 -- 172.21.15.119:0/2738696081 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f136c090ca0 con 0x7f13640503e0 2023-10-08T00:55:54.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.457+0000 7f548519b700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480069560 0x7f548015b4d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f547c00e8d0 tx=0x7f547c008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.457+0000 7f54757fa700 1 -- 172.21.15.119:0/2970778708 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c013410 con 0x7f5480069560 2023-10-08T00:55:54.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7f54757fa700 1 -- 172.21.15.119:0/2970778708 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c013570 con 0x7f5480069560 2023-10-08T00:55:54.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd480c6f80 0x7efd480b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7f54873ff700 1 -- 172.21.15.119:0/2970778708 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548015c2c0 con 0x7f5480069560 2023-10-08T00:55:54.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7f54757fa700 1 -- 172.21.15.119:0/2970778708 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0138f0 con 0x7f5480069560 2023-10-08T00:55:54.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd480ca290 0x7efd480c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd48147550 0x7efd4815d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7f54873ff700 1 -- 172.21.15.119:0/2970778708 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548015ba10 con 0x7f5480069560 2023-10-08T00:55:54.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4dd9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd480c6f80 0x7efd480b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4dd9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd480c6f80 0x7efd480b1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34560/0 (socket says 172.21.15.119:34560) 2023-10-08T00:55:54.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4dd9b700 1 -- 172.21.15.119:0/2905296086 learned_addr learned my addr 172.21.15.119:0/2905296086 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7f1361ffb700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1350067000 0x7f13500694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7f1361ffb700 1 -- 172.21.15.119:0/2738696081 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f136c0fc0d0 con 0x7f13640503e0 2023-10-08T00:55:54.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4d59a700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd480ca290 0x7efd480c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd480c4c50 con 0x7efd48147550 2023-10-08T00:55:54.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd480c4e60 con 0x7efd480ca290 2023-10-08T00:55:54.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.458+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd480c5140 con 0x7efd480c6f80 2023-10-08T00:55:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.459+0000 7f5f95599700 1 -- 172.21.15.119:0/2447381427 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8000b9c0 con 0x7f5f90159b20 2023-10-08T00:55:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.459+0000 7f5f95599700 1 -- 172.21.15.119:0/2447381427 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8000bd80 con 0x7f5f90159b20 2023-10-08T00:55:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.459+0000 7efd4e59c700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd48147550 0x7efd4815d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.459+0000 7f168cff9700 1 -- 172.21.15.119:0/3564820369 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167c012960 con 0x7f1680005210 2023-10-08T00:55:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.459+0000 7efd4d59a700 1 -- 172.21.15.119:0/2905296086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd480c6f80 msgr2=0x7efd480b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.459+0000 7f168cff9700 1 -- 172.21.15.119:0/3564820369 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167c012ad0 con 0x7f1680005210 2023-10-08T00:55:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7efd4d59a700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd480c6f80 0x7efd480b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7efd4d59a700 1 -- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd48147550 msgr2=0x7efd4815d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7efd4d59a700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd48147550 0x7efd4815d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7efd4d59a700 1 -- 172.21.15.119:0/2905296086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd440094f0 con 0x7efd480ca290 2023-10-08T00:55:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7f5f987ff700 1 -- 172.21.15.119:0/2447381427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159b20 msgr2=0x7f5f900b52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.459+0000 7f137199b700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1350067000 0x7f13500694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7f137199b700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1350067000 0x7f13500694b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13680017d0 tx=0x7f13680079f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7f5f987ff700 1 --2- 172.21.15.119:0/2447381427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159b20 0x7f5f900b52f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5f90089520 tx=0x7f5f80009250 comp rx=0 tx=0).stop 2023-10-08T00:55:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7f6284ff9700 1 -- 172.21.15.119:0/2940428857 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6278005c70 con 0x7f6288166960 2023-10-08T00:55:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.460+0000 7f6284ff9700 1 -- 172.21.15.119:0/2940428857 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6278005e00 con 0x7f6288166960 2023-10-08T00:55:54.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7efd4d59a700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd480ca290 0x7efd480c4710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd3800d890 tx=0x7efd38008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f54873ff700 1 -- 172.21.15.119:0/2970778708 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54600052b0 con 0x7f5480069560 2023-10-08T00:55:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f5f987ff700 1 -- 172.21.15.119:0/2447381427 shutdown_connections 2023-10-08T00:55:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f5f987ff700 1 --2- 172.21.15.119:0/2447381427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159b20 0x7f5f900b52f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f5f987ff700 1 --2- 172.21.15.119:0/2447381427 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f90159750 0x7f5f900b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f5f987ff700 1 --2- 172.21.15.119:0/2447381427 >> v2:172.21.15.184:3300/0 conn(0x7f5f900b5ab0 0x7f5f900b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f5f987ff700 1 -- 172.21.15.119:0/2447381427 >> 172.21.15.119:0/2447381427 conn(0x7f5f90099970 msgr2=0x7f5f900b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f5f987ff700 1 -- 172.21.15.119:0/2447381427 shutdown_connections 2023-10-08T00:55:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f5f987ff700 1 -- 172.21.15.119:0/2447381427 wait complete. 2023-10-08T00:55:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f54757fa700 1 -- 172.21.15.119:0/2970778708 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f547c013ce0 con 0x7f5480069560 2023-10-08T00:55:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.461+0000 7f9242ffd700 1 -- 172.21.15.119:0/308729036 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9254053460 con 0x7f925c16a9f0 2023-10-08T00:55:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.462+0000 7f54757fa700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5464066f60 0x7f5464069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.462+0000 7f54757fa700 1 -- 172.21.15.119:0/2970778708 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f547c084210 con 0x7f5480069560 2023-10-08T00:55:54.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.462+0000 7f168cff9700 1 -- 172.21.15.119:0/3564820369 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f167c012c30 con 0x7f1680005210 2023-10-08T00:55:54.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.463+0000 7f168cff9700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1674066f90 0x7f1674069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.463+0000 7f168cff9700 1 -- 172.21.15.119:0/3564820369 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f167c0569d0 con 0x7f1680005210 2023-10-08T00:55:54.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7f168ffff700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1674066f90 0x7f1674069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7f1361ffb700 1 -- 172.21.15.119:0/2738696081 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f136c0c93a0 con 0x7f13640503e0 2023-10-08T00:55:54.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7f5f987ff700 1 Processor -- start 2023-10-08T00:55:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7efd3dffb700 1 -- 172.21.15.119:0/2905296086 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd380125f0 con 0x7efd480ca290 2023-10-08T00:55:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7efd3dffb700 1 -- 172.21.15.119:0/2905296086 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd38012750 con 0x7efd480ca290 2023-10-08T00:55:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7efd3dffb700 1 -- 172.21.15.119:0/2905296086 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38012b10 con 0x7efd480ca290 2023-10-08T00:55:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd480c5420 con 0x7efd480ca290 2023-10-08T00:55:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd480c59e0 con 0x7efd480ca290 2023-10-08T00:55:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.465+0000 7f5f987ff700 1 -- start start 2023-10-08T00:55:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.465+0000 7f168ffff700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1674066f90 0x7f1674069440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f169004d2d0 tx=0x7f1690067410 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.464+0000 7f548499a700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5464066f60 0x7f5464069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.465+0000 7f548499a700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5464066f60 0x7f5464069410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5470001600 tx=0x7f5470000bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.465+0000 7f6284ff9700 1 -- 172.21.15.119:0/2940428857 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62780070f0 con 0x7f6288166960 2023-10-08T00:55:54.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.466+0000 7f6284ff9700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6274066fb0 0x7f6274069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.467+0000 7f5f987ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f900b5ab0 0x7f5f900c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.467+0000 7f6284ff9700 1 -- 172.21.15.119:0/2940428857 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6278084930 con 0x7f6288166960 2023-10-08T00:55:54.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.467+0000 7f5f987ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159750 0x7f5f9015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.467+0000 7f628cb9b700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6274066fb0 0x7f6274069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.467+0000 7f5f9659b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f900b5ab0 0x7f5f900c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.467+0000 7f5f9659b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f900b5ab0 0x7f5f900c9d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34570/0 (socket says 172.21.15.119:34570) 2023-10-08T00:55:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.468+0000 7f5f9659b700 1 -- 172.21.15.119:0/271489123 learned_addr learned my addr 172.21.15.119:0/271489123 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.468+0000 7f5f987ff700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90159b20 0x7f5f9015c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.468+0000 7f54757fa700 1 -- 172.21.15.119:0/2970778708 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f547c054f10 con 0x7f5480069560 2023-10-08T00:55:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.468+0000 7f5f95d9a700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159750 0x7f5f9015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.468+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900c8da0 con 0x7f5f90159750 2023-10-08T00:55:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.468+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9015ce70 con 0x7f5f90159b20 2023-10-08T00:55:54.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.468+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9015a800 con 0x7f5f900b5ab0 2023-10-08T00:55:54.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.468+0000 7f5f9759d700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90159b20 0x7f5f9015c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.469+0000 7f628cb9b700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6274066fb0 0x7f6274069460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f627c00a4a0 tx=0x7f627c01c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.469+0000 7f5f9659b700 1 -- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90159b20 msgr2=0x7f5f9015c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.469+0000 7f5f9659b700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90159b20 0x7f5f9015c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.469+0000 7f5f9659b700 1 -- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159750 msgr2=0x7f5f9015b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.469+0000 7f5f9659b700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159750 0x7f5f9015b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.469+0000 7f5f9659b700 1 -- 172.21.15.119:0/271489123 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f800094f0 con 0x7f5f900b5ab0 2023-10-08T00:55:54.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.470+0000 7f168cff9700 1 -- 172.21.15.119:0/3564820369 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f167c052da0 con 0x7f1680005210 2023-10-08T00:55:54.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.471+0000 7efd3dffb700 1 -- 172.21.15.119:0/2905296086 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd3801e460 con 0x7efd480ca290 2023-10-08T00:55:54.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.471+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4804c3c0 con 0x7efd480ca290 2023-10-08T00:55:54.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.471+0000 7f5f9659b700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f900b5ab0 0x7f5f900c9d40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800bf10 tx=0x7f5f88009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.472+0000 7f5f867fc700 1 -- 172.21.15.119:0/271489123 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8800cb50 con 0x7f5f900b5ab0 2023-10-08T00:55:54.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.472+0000 7f5f867fc700 1 -- 172.21.15.119:0/271489123 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8800ccb0 con 0x7f5f900b5ab0 2023-10-08T00:55:54.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.472+0000 7f5f867fc700 1 -- 172.21.15.119:0/271489123 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88014690 con 0x7f5f900b5ab0 2023-10-08T00:55:54.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.472+0000 7efd3dffb700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efd2c067000 0x7efd2c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.473+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9015aae0 con 0x7f5f900b5ab0 2023-10-08T00:55:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.472+0000 7efd3dffb700 1 -- 172.21.15.119:0/2905296086 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7efd38084a00 con 0x7efd480ca290 2023-10-08T00:55:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.473+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9015b0a0 con 0x7f5f900b5ab0 2023-10-08T00:55:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.473+0000 7efd4dd9b700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efd2c067000 0x7efd2c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.476+0000 7f5f867fc700 1 -- 172.21.15.119:0/271489123 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f8801d3b0 con 0x7f5f900b5ab0 2023-10-08T00:55:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.477+0000 7efd4dd9b700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efd2c067000 0x7efd2c0694b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd440094c0 tx=0x7efd44017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.477+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9004c3c0 con 0x7f5f900b5ab0 2023-10-08T00:55:54.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.477+0000 7efd3dffb700 1 -- 172.21.15.119:0/2905296086 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd38051c70 con 0x7efd480ca290 2023-10-08T00:55:54.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.478+0000 7f6284ff9700 1 -- 172.21.15.119:0/2940428857 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6278051be0 con 0x7f6288166960 2023-10-08T00:55:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.479+0000 7f5f867fc700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5f74067080 0x7f5f74069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.479+0000 7f5f867fc700 1 -- 172.21.15.119:0/271489123 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5f88084610 con 0x7f5f900b5ab0 2023-10-08T00:55:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.481+0000 7f5f95d9a700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5f74067080 0x7f5f74069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.482+0000 7f5f95d9a700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5f74067080 0x7f5f74069530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c006fd0 tx=0x7f5f8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.487+0000 7f5f867fc700 1 -- 172.21.15.119:0/271489123 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f88051860 con 0x7f5f900b5ab0 2023-10-08T00:55:54.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.888+0000 7f2c5ef9d700 1 -- 172.21.15.119:0/3789643507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2c50038330 con 0x7f2c5004cb10 2023-10-08T00:55:54.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.889+0000 7f2c4dffb700 1 -- 172.21.15.119:0/3789643507 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2c54055d20 con 0x7f2c5004cb10 2023-10-08T00:55:54.890 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2023-10-08T00:55:54.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.893+0000 7f2c3b7fe700 1 -- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2c3c067080 msgr2=0x7f2c3c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.893+0000 7f2c3b7fe700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2c3c067080 0x7f2c3c069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2c48001010 tx=0x7f2c48001b70 comp rx=0 tx=0).stop 2023-10-08T00:55:54.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.893+0000 7f2c3b7fe700 1 -- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5004cb10 msgr2=0x7f2c500de940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.893+0000 7f2c3b7fe700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5004cb10 0x7f2c500de940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2c44005d60 tx=0x7f2c54015000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.894+0000 7f2c3b7fe700 1 -- 172.21.15.119:0/3789643507 shutdown_connections 2023-10-08T00:55:54.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.894+0000 7f2c3b7fe700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c5004cf80 0x7f2c500dee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.894+0000 7f2c3b7fe700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2c3c067080 0x7f2c3c069530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.894+0000 7f2c3b7fe700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c44004990 0x7f2c500d6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.894+0000 7f2c3b7fe700 1 --2- 172.21.15.119:0/3789643507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5004cb10 0x7f2c500de940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.894+0000 7f2c3b7fe700 1 -- 172.21.15.119:0/3789643507 >> 172.21.15.119:0/3789643507 conn(0x7f2c50054da0 msgr2=0x7f2c5004a550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.895+0000 7f2c3b7fe700 1 -- 172.21.15.119:0/3789643507 shutdown_connections 2023-10-08T00:55:54.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.895+0000 7f2c3b7fe700 1 -- 172.21.15.119:0/3789643507 wait complete. 2023-10-08T00:55:54.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.895+0000 7f137299d700 1 -- 172.21.15.119:0/2738696081 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1364005380 con 0x7f13640503e0 2023-10-08T00:55:54.897 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:54.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.896+0000 7f1361ffb700 1 -- 172.21.15.119:0/2738696081 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f136c06e090 con 0x7f13640503e0 2023-10-08T00:55:54.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.900+0000 7f134f7fe700 1 -- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1350067000 msgr2=0x7f13500694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.900+0000 7f134f7fe700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1350067000 0x7f13500694b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f13680017d0 tx=0x7f13680079f0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.900+0000 7f92613ff700 1 -- 172.21.15.119:0/308729036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f923c005560 con 0x7f925c16a9f0 2023-10-08T00:55:54.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.900+0000 7f134f7fe700 1 -- 172.21.15.119:0/2738696081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13640503e0 msgr2=0x7f13640c1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.900+0000 7f134f7fe700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13640503e0 0x7f13640c1220 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f136c04c860 tx=0x7f136c08b000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.901+0000 7f9242ffd700 1 -- 172.21.15.119:0/308729036 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9254055780 con 0x7f925c16a9f0 2023-10-08T00:55:54.903 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2023-10-08T00:55:54.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f134f7fe700 1 -- 172.21.15.119:0/2738696081 shutdown_connections 2023-10-08T00:55:54.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f9240ff9700 1 -- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9248067000 msgr2=0x7f92480694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f134f7fe700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1350067000 0x7f13500694b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f134f7fe700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13640503e0 0x7f13640c1220 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f134f7fe700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1364007140 0x7f1364037ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f134f7fe700 1 --2- 172.21.15.119:0/2738696081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1364033960 0x7f13640cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f134f7fe700 1 -- 172.21.15.119:0/2738696081 >> 172.21.15.119:0/2738696081 conn(0x7f13640516f0 msgr2=0x7f136404a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f9240ff9700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9248067000 0x7f92480694b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9250009aa0 tx=0x7f9250008040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f134f7fe700 1 -- 172.21.15.119:0/2738696081 shutdown_connections 2023-10-08T00:55:54.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.905+0000 7f9240ff9700 1 -- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f925c16a9f0 msgr2=0x7f925c06dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.906+0000 7f9240ff9700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f925c16a9f0 0x7f925c06dcf0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f925c065b70 tx=0x7f9254019000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.906+0000 7f134f7fe700 1 -- 172.21.15.119:0/2738696081 wait complete. 2023-10-08T00:55:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.906+0000 7f9240ff9700 1 -- 172.21.15.119:0/308729036 shutdown_connections 2023-10-08T00:55:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.906+0000 7f9240ff9700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f925c159470 0x7f925c06c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.906+0000 7f9240ff9700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9248067000 0x7f92480694b0 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9250009aa0 tx=0x7f9250008040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.906+0000 7f9240ff9700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f925c16a9f0 0x7f925c06dcf0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.906+0000 7f9240ff9700 1 --2- 172.21.15.119:0/308729036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f925c159840 0x7f925c157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.906+0000 7f9240ff9700 1 -- 172.21.15.119:0/308729036 >> 172.21.15.119:0/308729036 conn(0x7f925c0d0ff0 msgr2=0x7f925c15b100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.907+0000 7f9240ff9700 1 -- 172.21.15.119:0/308729036 shutdown_connections 2023-10-08T00:55:54.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.907+0000 7f9240ff9700 1 -- 172.21.15.119:0/308729036 wait complete. 2023-10-08T00:55:54.911 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:54.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.909+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7efd4806a7b0 con 0x7efd480ca290 2023-10-08T00:55:54.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.910+0000 7efd3dffb700 1 -- 172.21.15.119:0/2905296086 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efd38055290 con 0x7efd480ca290 2023-10-08T00:55:54.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.911+0000 7f54873ff700 1 -- 172.21.15.119:0/2970778708 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5460005830 con 0x7f5480069560 2023-10-08T00:55:54.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.912+0000 7f54757fa700 1 -- 172.21.15.119:0/2970778708 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f547c054aa0 con 0x7f5480069560 2023-10-08T00:55:54.913 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2023-10-08T00:55:54.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.914+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efd2c067000 msgr2=0x7efd2c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.914+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efd2c067000 0x7efd2c0694b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd440094c0 tx=0x7efd44017000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.914+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd480ca290 msgr2=0x7efd480c4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.914+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd480ca290 0x7efd480c4710 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7efd3800d890 tx=0x7efd38008040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.915+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 shutdown_connections 2023-10-08T00:55:54.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.915+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd480c6f80 0x7efd480b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.915+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efd2c067000 0x7efd2c0694b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.915+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd48147550 0x7efd4815d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.915+0000 7efd4ed9d700 1 --2- 172.21.15.119:0/2905296086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd480ca290 0x7efd480c4710 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.915+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 >> 172.21.15.119:0/2905296086 conn(0x7efd480cc1f0 msgr2=0x7efd4806dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.916+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 shutdown_connections 2023-10-08T00:55:54.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.916+0000 7efd4ed9d700 1 -- 172.21.15.119:0/2905296086 wait complete. 2023-10-08T00:55:54.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.916+0000 7f546affd700 1 -- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5464066f60 msgr2=0x7f5464069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.916+0000 7f546affd700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5464066f60 0x7f5464069410 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5470001600 tx=0x7f5470000bc0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.916+0000 7f546affd700 1 -- 172.21.15.119:0/2970778708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480069560 msgr2=0x7f548015b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.916+0000 7f546affd700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480069560 0x7f548015b4d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f547c00e8d0 tx=0x7f547c008040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.917+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6288154aa0 con 0x7f6288166960 2023-10-08T00:55:54.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.918+0000 7f6284ff9700 1 -- 172.21.15.119:0/2940428857 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6278055200 con 0x7f6288166960 2023-10-08T00:55:54.919 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2023-10-08T00:55:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.919+0000 7f546affd700 1 -- 172.21.15.119:0/2970778708 shutdown_connections 2023-10-08T00:55:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.919+0000 7f546affd700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5480069560 0x7f548015b4d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.919+0000 7f546affd700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5464066f60 0x7f5464069410 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.919+0000 7f546affd700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5480159c60 0x7f54800b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.919+0000 7f546affd700 1 --2- 172.21.15.119:0/2970778708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5480159890 0x7f5480171e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.919+0000 7f546affd700 1 -- 172.21.15.119:0/2970778708 >> 172.21.15.119:0/2970778708 conn(0x7f5480068e20 msgr2=0x7f5480156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.919+0000 7f546affd700 1 -- 172.21.15.119:0/2970778708 shutdown_connections 2023-10-08T00:55:54.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.919+0000 7f546affd700 1 -- 172.21.15.119:0/2970778708 wait complete. 2023-10-08T00:55:54.922 INFO:teuthology.orchestra.run.smithi119.stdout:55834574853 2023-10-08T00:55:54.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6274066fb0 msgr2=0x7f6274069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6274066fb0 0x7f6274069460 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f627c00a4a0 tx=0x7f627c01c000 comp rx=0 tx=0).stop 2023-10-08T00:55:54.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288166960 msgr2=0x7f628817a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288166960 0x7f628817a200 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f627800c840 tx=0x7f6278007eb0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 shutdown_connections 2023-10-08T00:55:54.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6274066fb0 0x7f6274069460 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6288166960 0x7f628817a200 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.920+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f16880070e0 con 0x7f1680005210 2023-10-08T00:55:54.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.921+0000 7f168cff9700 1 -- 172.21.15.119:0/3564820369 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f167c052f40 con 0x7f1680005210 2023-10-08T00:55:54.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6288068aa0 0x7f628806aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 --2- 172.21.15.119:0/2940428857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62880abe00 0x7f628806b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.922+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 >> 172.21.15.119:0/2940428857 conn(0x7f62880cd000 msgr2=0x7f628806d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.923+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 shutdown_connections 2023-10-08T00:55:54.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.923+0000 7f628edff700 1 -- 172.21.15.119:0/2940428857 wait complete. 2023-10-08T00:55:54.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.925+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1674066f90 msgr2=0x7f1674069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.925+0000 7f1696dff700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1674066f90 0x7f1674069440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f169004d2d0 tx=0x7f1690067410 comp rx=0 tx=0).stop 2023-10-08T00:55:54.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.925+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680005210 msgr2=0x7f168804c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.925+0000 7f1696dff700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680005210 0x7f168804c8c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f167c00d810 tx=0x7f167c008040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.929+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5f9015a940 con 0x7f5f900b5ab0 2023-10-08T00:55:54.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-10-08T00:55:54.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.926+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 shutdown_connections 2023-10-08T00:55:54.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.926+0000 7f1696dff700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f168804eec0 0x7f168804f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.926+0000 7f1696dff700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1674066f90 0x7f1674069440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.926+0000 7f1696dff700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1680005210 0x7f168804c8c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.926+0000 7f1696dff700 1 --2- 172.21.15.119:0/3564820369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1688050bd0 0x7f16880d44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.926+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 >> 172.21.15.119:0/3564820369 conn(0x7f1688059cd0 msgr2=0x7f1688056d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.926+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 shutdown_connections 2023-10-08T00:55:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.926+0000 7f1696dff700 1 -- 172.21.15.119:0/3564820369 wait complete. 2023-10-08T00:55:54.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.930+0000 7f5f867fc700 1 -- 172.21.15.119:0/271489123 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f9015a940 con 0x7f5f900b5ab0 2023-10-08T00:55:54.933 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:54.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.933+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5f74067080 msgr2=0x7f5f74069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.933+0000 7f5f987ff700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5f74067080 0x7f5f74069530 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c006fd0 tx=0x7f5f8c008040 comp rx=0 tx=0).stop 2023-10-08T00:55:54.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.933+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f900b5ab0 msgr2=0x7f5f900c9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:54.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.933+0000 7f5f987ff700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f900b5ab0 0x7f5f900c9d40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800bf10 tx=0x7f5f88009710 comp rx=0 tx=0).stop 2023-10-08T00:55:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.934+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 shutdown_connections 2023-10-08T00:55:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.934+0000 7f5f987ff700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f900b5ab0 0x7f5f900c9d40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.934+0000 7f5f987ff700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5f74067080 0x7f5f74069530 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.934+0000 7f5f987ff700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90159b20 0x7f5f9015c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.934+0000 7f5f987ff700 1 --2- 172.21.15.119:0/271489123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90159750 0x7f5f9015b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.934+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 >> 172.21.15.119:0/271489123 conn(0x7f5f90099970 msgr2=0x7f5f90158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.934+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 shutdown_connections 2023-10-08T00:55:54.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:54.934+0000 7f5f987ff700 1 -- 172.21.15.119:0/271489123 wait complete. 2023-10-08T00:55:54.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-10-08T00:55:54.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-08T00:55:54.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-10-08T00:55:54.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-10-08T00:55:54.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-10-08T00:55:54.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-10-08T00:55:54.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.7 2023-10-08T00:55:55.933 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-08T00:55:55.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-08T00:55:55.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-08T00:55:55.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-08T00:55:55.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-08T00:55:55.956 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-08T00:55:55.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-08T00:55:55.963 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-08T00:55:56.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.218+0000 7f3c8a1ff700 1 Processor -- start 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.218+0000 7f3c8a1ff700 1 -- start start 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.221+0000 7f3c8a1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c840b1c50 0x7f3c840879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.221+0000 7f3c8a1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c84068f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.221+0000 7f3c8a1ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c840acdf0 con 0x7f3c840875d0 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.221+0000 7f3c8a1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c84069480 con 0x7f3c840b1c50 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.221+0000 7f3c8a1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c840abfd0 con 0x7f3c84068aa0 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.222+0000 7f3c83fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c84068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.222+0000 7f3c83fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c84068f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52026/0 (socket says 172.21.15.119:52026) 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.222+0000 7f3c83fff700 1 -- 172.21.15.119:0/2653201017 learned_addr learned my addr 172.21.15.119:0/2653201017 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.223+0000 7f3c827fc700 1 -- 172.21.15.119:0/2653201017 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636839364 0 0) 0x7f3c840acdf0 con 0x7f3c840875d0 2023-10-08T00:55:56.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.223+0000 7f3c827fc700 1 -- 172.21.15.119:0/2653201017 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c6c003550 con 0x7f3c840875d0 2023-10-08T00:55:56.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.221+0000 7f3c82ffd700 1 --2- 172.21.15.119:0/2653201017 >> v2:172.21.15.119:3300/0 conn(0x7f3c840b1c50 0x7f3c840879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.225+0000 7f3c82ffd700 1 -- 172.21.15.119:0/2653201017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 msgr2=0x7f3c84068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.225+0000 7f3c82ffd700 1 --2- 172.21.15.119:0/2653201017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c84068f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.225+0000 7f3c82ffd700 1 -- 172.21.15.119:0/2653201017 >> v1:172.21.15.119:6789/0 conn(0x7f3c840875d0 legacy=0x7f3c840ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.225+0000 7f3c82ffd700 1 -- 172.21.15.119:0/2653201017 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c840ac1e0 con 0x7f3c840b1c50 2023-10-08T00:55:56.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.226+0000 7f3c82ffd700 1 --2- 172.21.15.119:0/2653201017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c840b1c50 0x7f3c840879a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400dd10 tx=0x7f3c7400d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b9685a2000eb18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.226+0000 7f3c827fc700 1 -- 172.21.15.119:0/2653201017 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74012070 con 0x7f3c840b1c50 2023-10-08T00:55:56.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.226+0000 7f3c827fc700 1 -- 172.21.15.119:0/2653201017 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7400de70 con 0x7f3c840b1c50 2023-10-08T00:55:56.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.226+0000 7f3c827fc700 1 -- 172.21.15.119:0/2653201017 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c74012070 con 0x7f3c840b1c50 2023-10-08T00:55:56.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.228+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2653201017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c840b1c50 msgr2=0x7f3c840879a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.228+0000 7f3c8a1ff700 1 --2- 172.21.15.119:0/2653201017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c840b1c50 0x7f3c840879a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400dd10 tx=0x7f3c7400d630 comp rx=0 tx=0).stop 2023-10-08T00:55:56.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.228+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2653201017 shutdown_connections 2023-10-08T00:55:56.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.228+0000 7f3c8a1ff700 1 --2- 172.21.15.119:0/2653201017 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c84068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.228+0000 7f3c8a1ff700 1 --2- 172.21.15.119:0/2653201017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c840b1c50 0x7f3c840879a0 secure :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3c7400dd10 tx=0x7f3c7400d630 comp rx=0 tx=0).stop 2023-10-08T00:55:56.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.228+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2653201017 >> 172.21.15.119:0/2653201017 conn(0x7f3c840cd000 msgr2=0x7f3c84068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.229+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2653201017 shutdown_connections 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.229+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2653201017 wait complete. 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.231+0000 7f3c8a1ff700 1 Processor -- start 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.231+0000 7f3c8a1ff700 1 -- start start 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.232+0000 7f3c8a1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c841773c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.233+0000 7f3c8a1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c840875d0 0x7f3c8406e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.233+0000 7f3c8a1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c84160900 0x7f3c841685e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.233+0000 7f3c837fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c841773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.233+0000 7f3c837fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c841773c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52032/0 (socket says 172.21.15.119:52032) 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.234+0000 7f3c83fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c84160900 0x7f3c841685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.234+0000 7f3c83fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c84160900 0x7f3c841685e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32996/0 (socket says 172.21.15.119:32996) 2023-10-08T00:55:56.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.234+0000 7f3c82ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c840875d0 0x7f3c8406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.234+0000 7f3c837fe700 1 -- 172.21.15.119:0/2071022494 learned_addr learned my addr 172.21.15.119:0/2071022494 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.234+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2071022494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8415e5b0 con 0x7f3c84160900 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.234+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2071022494 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c84177900 con 0x7f3c840875d0 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.234+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2071022494 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c84160e00 con 0x7f3c84068aa0 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.235+0000 7f3c837fe700 1 -- 172.21.15.119:0/2071022494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c840875d0 msgr2=0x7f3c8406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.235+0000 7f3c837fe700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c840875d0 0x7f3c8406e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.235+0000 7f3c837fe700 1 -- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c84160900 msgr2=0x7f3c841685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.235+0000 7f3c837fe700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c84160900 0x7f3c841685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.235+0000 7f3c837fe700 1 -- 172.21.15.119:0/2071022494 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7400d8d0 con 0x7f3c84068aa0 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.235+0000 7f3c83fff700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c84160900 0x7f3c841685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:56.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.235+0000 7f3c837fe700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c841773c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3c78002e60 tx=0x7f3c78007dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.238+0000 7f3c6b7fe700 1 -- 172.21.15.119:0/2071022494 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78015c40 con 0x7f3c84068aa0 2023-10-08T00:55:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.238+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2071022494 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c841633d0 con 0x7f3c84068aa0 2023-10-08T00:55:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.238+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2071022494 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c84163990 con 0x7f3c84068aa0 2023-10-08T00:55:56.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.239+0000 7f3c6b7fe700 1 -- 172.21.15.119:0/2071022494 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c78015dc0 con 0x7f3c84068aa0 2023-10-08T00:55:56.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.239+0000 7f3c6b7fe700 1 -- 172.21.15.119:0/2071022494 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7801e6c0 con 0x7f3c84068aa0 2023-10-08T00:55:56.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.242+0000 7f3c6b7fe700 1 -- 172.21.15.119:0/2071022494 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c7801b070 con 0x7f3c84068aa0 2023-10-08T00:55:56.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.244+0000 7f3c6b7fe700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c700670b0 0x7f3c70069560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.245+0000 7f3c82ffd700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c700670b0 0x7f3c70069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.246+0000 7f3c82ffd700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c700670b0 0x7f3c70069560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c840b2090 tx=0x7f3c7401c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.247+0000 7f3c6b7fe700 1 -- 172.21.15.119:0/2071022494 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3c7808c460 con 0x7f3c84068aa0 2023-10-08T00:55:56.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.247+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2071022494 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c640052b0 con 0x7f3c84068aa0 2023-10-08T00:55:56.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.254+0000 7f3c6b7fe700 1 -- 172.21.15.119:0/2071022494 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c780596b0 con 0x7f3c84068aa0 2023-10-08T00:55:56.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.302+0000 7f7736dff700 1 Processor -- start 2023-10-08T00:55:56.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.303+0000 7f7736dff700 1 -- start start 2023-10-08T00:55:56.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.305+0000 7f7736dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7730159c10 0x7f773015b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.305+0000 7f772ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f77300b5ea0 0x7f7730068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54276/0 (socket says 172.21.15.119:54276) 2023-10-08T00:55:56.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.306+0000 7f7734b9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7730159c10 0x7f773015b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.306+0000 7f7734b9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7730159c10 0x7f773015b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44670/0 (socket says 172.21.15.119:44670) 2023-10-08T00:55:56.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.306+0000 7f772ffff700 1 -- 172.21.15.119:0/3114646054 learned_addr learned my addr 172.21.15.119:0/3114646054 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:56.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.307+0000 7f7736dff700 1 -- 172.21.15.119:0/3114646054 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773015c480 con 0x7f77300b5ea0 2023-10-08T00:55:56.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.308+0000 7f7736dff700 1 -- 172.21.15.119:0/3114646054 --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f773015b5d0 con 0x7f773015c910 2023-10-08T00:55:56.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.308+0000 7f7736dff700 1 -- 172.21.15.119:0/3114646054 --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77300b2a40 con 0x7f7730159c10 2023-10-08T00:55:56.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.308+0000 7f772f7fe700 1 -- 172.21.15.119:0/3114646054 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121209053 0 0) 0x7f773015c480 con 0x7f77300b5ea0 2023-10-08T00:55:56.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.309+0000 7f772f7fe700 1 -- 172.21.15.119:0/3114646054 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7718003570 con 0x7f77300b5ea0 2023-10-08T00:55:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.309+0000 7f7734b9b700 1 -- 172.21.15.119:0/3114646054 >> v1:172.21.15.184:6789/0 conn(0x7f773015c910 legacy=0x7f7730159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.310+0000 7f7734b9b700 1 -- 172.21.15.119:0/3114646054 >> v1:172.21.15.119:6789/0 conn(0x7f77300b5ea0 legacy=0x7f7730068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.310+0000 7f7734b9b700 1 -- 172.21.15.119:0/3114646054 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f773015b870 con 0x7f7730159c10 2023-10-08T00:55:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.310+0000 7f7734b9b700 1 --2- 172.21.15.119:0/3114646054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 0x7f773015b090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77240098e0 tx=0x7f77240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4649534dd6d40b98 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.309+0000 7f772f7fe700 1 -- 172.21.15.119:0/3114646054 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362471789 0 0) 0x7f773015b5d0 con 0x7f773015c910 2023-10-08T00:55:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7f772f7fe700 1 -- 172.21.15.119:0/3114646054 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77200013f0 con 0x7f7730159c10 2023-10-08T00:55:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7f772f7fe700 1 -- 172.21.15.119:0/3114646054 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7724011420 con 0x7f7730159c10 2023-10-08T00:55:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.306+0000 7fbf22b9d700 1 Processor -- start 2023-10-08T00:55:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7f772f7fe700 1 -- 172.21.15.119:0/3114646054 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77240117c0 con 0x7f7730159c10 2023-10-08T00:55:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7f7736dff700 1 -- 172.21.15.119:0/3114646054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 msgr2=0x7f773015b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7f7736dff700 1 --2- 172.21.15.119:0/3114646054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 0x7f773015b090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f77240098e0 tx=0x7f77240091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.306+0000 7fbf22b9d700 1 -- start start 2023-10-08T00:55:56.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.308+0000 7fbf22b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf1c15dc70 0x7fbf1c0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.308+0000 7fbf22b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0ad0a0 con 0x7fbf1c0af370 2023-10-08T00:55:56.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.308+0000 7fbf22b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0ac770 con 0x7fbf1c15d8a0 2023-10-08T00:55:56.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.308+0000 7fbf22b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1c0ac580 con 0x7fbf1c15dc70 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.309+0000 7fbf2239c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbf1c0af370 0x7fbf1c0ad530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54284/0 (socket says 172.21.15.119:54284) 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.309+0000 7fbf2239c700 1 -- 172.21.15.119:0/1654744558 learned_addr learned my addr 172.21.15.119:0/1654744558 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.310+0000 7fbf2139a700 1 --2- 172.21.15.119:0/1654744558 >> v2:172.21.15.119:3300/0 conn(0x7fbf1c15dc70 0x7fbf1c0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7fbf20b99700 1 -- 172.21.15.119:0/1654744558 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487124411 0 0) 0x7fbf1c0ad0a0 con 0x7fbf1c0af370 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7fbf20b99700 1 -- 172.21.15.119:0/1654744558 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf08003570 con 0x7fbf1c0af370 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7fbf20b99700 1 -- 172.21.15.119:0/1654744558 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1444111886 0 0) 0x7fbf1c0ac770 con 0x7fbf1c15d8a0 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7fbf20b99700 1 -- 172.21.15.119:0/1654744558 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf1c0ad0a0 con 0x7fbf1c15d8a0 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7fbf2139a700 1 -- 172.21.15.119:0/1654744558 >> v1:172.21.15.184:6789/0 conn(0x7fbf1c15d8a0 legacy=0x7fbf1c0af970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7fbf2139a700 1 -- 172.21.15.119:0/1654744558 >> v1:172.21.15.119:6789/0 conn(0x7fbf1c0af370 legacy=0x7fbf1c0ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.311+0000 7fbf2139a700 1 -- 172.21.15.119:0/1654744558 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0aca50 con 0x7fbf1c15dc70 2023-10-08T00:55:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7fbf2139a700 1 --2- 172.21.15.119:0/1654744558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf1c15dc70 0x7fbf1c0b15d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009b40 tx=0x7fbf180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c592b8f133c9a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7f7736dff700 1 -- 172.21.15.119:0/3114646054 shutdown_connections 2023-10-08T00:55:56.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7f7736dff700 1 --2- 172.21.15.119:0/3114646054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 0x7f773015b090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7f7736dff700 1 -- 172.21.15.119:0/3114646054 >> 172.21.15.119:0/3114646054 conn(0x7f7730099a20 msgr2=0x7f77300af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7f7736dff700 1 -- 172.21.15.119:0/3114646054 shutdown_connections 2023-10-08T00:55:56.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7f7736dff700 1 -- 172.21.15.119:0/3114646054 wait complete. 2023-10-08T00:55:56.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7fbf20b99700 1 -- 172.21.15.119:0/1654744558 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf140017f0 con 0x7fbf1c15dc70 2023-10-08T00:55:56.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7fbf20b99700 1 -- 172.21.15.119:0/1654744558 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1800b550 con 0x7fbf1c15dc70 2023-10-08T00:55:56.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.312+0000 7fbf20b99700 1 -- 172.21.15.119:0/1654744558 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800b6c0 con 0x7fbf1c15dc70 2023-10-08T00:55:56.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.313+0000 7fbf22b9d700 1 -- 172.21.15.119:0/1654744558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf1c15dc70 msgr2=0x7fbf1c0b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.313+0000 7fbf22b9d700 1 --2- 172.21.15.119:0/1654744558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf1c15dc70 0x7fbf1c0b15d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009b40 tx=0x7fbf180091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.314+0000 7fbf22b9d700 1 -- 172.21.15.119:0/1654744558 shutdown_connections 2023-10-08T00:55:56.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.314+0000 7fbf22b9d700 1 --2- 172.21.15.119:0/1654744558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf1c15dc70 0x7fbf1c0b15d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.314+0000 7fbf22b9d700 1 -- 172.21.15.119:0/1654744558 >> 172.21.15.119:0/1654744558 conn(0x7fbf1c0ccc80 msgr2=0x7fbf1c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.314+0000 7f7736dff700 1 Processor -- start 2023-10-08T00:55:56.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.313+0000 7fee7d79d700 1 Processor -- start 2023-10-08T00:55:56.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.313+0000 7fee7d79d700 1 -- start start 2023-10-08T00:55:56.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.315+0000 7fed3b59e700 1 Processor -- start 2023-10-08T00:55:56.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.315+0000 7f7736dff700 1 -- start start 2023-10-08T00:55:56.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.315+0000 7fed3b59e700 1 -- start start 2023-10-08T00:55:56.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.315+0000 7fee7d79d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee7815de70 0x7fee7806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fee7d79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee7806ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fbf22b9d700 1 -- 172.21.15.119:0/1654744558 shutdown_connections 2023-10-08T00:55:56.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fee7d79d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7806e8c0 con 0x7fee7815daa0 2023-10-08T00:55:56.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fee7d79d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780ca740 con 0x7fee7815de70 2023-10-08T00:55:56.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7f7736dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77300b5ea0 0x7f773006c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fee777fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee7815de70 0x7fee7806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fee7d79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7806e5e0 con 0x7fee780696c0 2023-10-08T00:55:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fee777fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee7815de70 0x7fee7806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33012/0 (socket says 172.21.15.119:33012) 2023-10-08T00:55:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fee777fe700 1 -- 172.21.15.119:0/335609755 learned_addr learned my addr 172.21.15.119:0/335609755 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.316+0000 7fee7cf9c700 1 --2- 172.21.15.119:0/335609755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee7806ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7f7734b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77300b5ea0 0x7f773006c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fed3b59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fed3c06dc00 0x7fed3c15dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fed3b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c06a720 0x7fed3c06ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fed3b59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed3c0c7120 con 0x7fed3c15d8f0 2023-10-08T00:55:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fed3b59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed3c0c6860 con 0x7fed3c06dc00 2023-10-08T00:55:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fed3b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0c85c0 con 0x7fed3c06a720 2023-10-08T00:55:56.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7f7734b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77300b5ea0 0x7f773006c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33024/0 (socket says 172.21.15.119:33024) 2023-10-08T00:55:56.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7f7734b9b700 1 -- 172.21.15.119:0/4245410618 learned_addr learned my addr 172.21.15.119:0/4245410618 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7f7736dff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 0x7f773006c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7f7736dff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f773015c910 0x7f773016d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773006d030 con 0x7f77300b5ea0 2023-10-08T00:55:56.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f773006c6b0 con 0x7f7730159c10 2023-10-08T00:55:56.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f773006dd60 con 0x7f773015c910 2023-10-08T00:55:56.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7fed3a59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fed3c15d8f0 0x7fed3c06d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54292/0 (socket says 172.21.15.119:54292) 2023-10-08T00:55:56.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7fed3a59c700 1 -- 172.21.15.119:0/1999650880 learned_addr learned my addr 172.21.15.119:0/1999650880 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:56.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7fed3ad9d700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c06a720 0x7fed3c06ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7f773539c700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f773015c910 0x7f773016d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7f772ffff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 0x7f773006c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed39d9b700 1 --2- 172.21.15.119:0/1999650880 >> v2:172.21.15.184:3300/0 conn(0x7fed3c06dc00 0x7fed3c15dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085234558 0 0) 0x7fed3c0c7120 con 0x7fed3c15d8f0 2023-10-08T00:55:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed24003570 con 0x7fed3c15d8f0 2023-10-08T00:55:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fbf22b9d700 1 -- 172.21.15.119:0/1654744558 wait complete. 2023-10-08T00:55:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2020129974 0 0) 0x7fed24003570 con 0x7fed3c15d8f0 2023-10-08T00:55:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c06a720 msgr2=0x7fed3c06ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c06a720 0x7fed3c06ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 >> v2:172.21.15.184:3300/0 conn(0x7fed3c06dc00 msgr2=0x7fed3c15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 --2- 172.21.15.119:0/1999650880 >> v2:172.21.15.184:3300/0 conn(0x7fed3c06dc00 0x7fed3c15dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c0c6ad0 con 0x7fed3c15d8f0 2023-10-08T00:55:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fed30001450 con 0x7fed3c15d8f0 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 >> v1:172.21.15.119:6789/0 conn(0x7fed3c15d8f0 legacy=0x7fed3c06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed3959a700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed24003a60 0x7fed24003e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed3959a700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed24004810 0x7fed24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed3959a700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed240051c0 0x7fed2400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0c6860 con 0x7fed240051c0 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c0c85c0 con 0x7fed24004810 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2400bfa0 con 0x7fed24003a60 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fee777fe700 1 -- 172.21.15.119:0/335609755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 msgr2=0x7fee7806ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fee777fe700 1 --2- 172.21.15.119:0/335609755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee7806ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fee777fe700 1 -- 172.21.15.119:0/335609755 >> v1:172.21.15.184:6789/0 conn(0x7fee7815daa0 legacy=0x7fee780c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.317+0000 7fee777fe700 1 -- 172.21.15.119:0/335609755 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780ca0f0 con 0x7fee7815de70 2023-10-08T00:55:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7fee777fe700 1 --2- 172.21.15.119:0/335609755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 0x7fee7806e000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fee680099b0 tx=0x7fee68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe640a42ba6639e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7fee76ffd700 1 -- 172.21.15.119:0/335609755 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee68009ea0 con 0x7fee7815de70 2023-10-08T00:55:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7fee76ffd700 1 -- 172.21.15.119:0/335609755 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6800b860 con 0x7fee7815de70 2023-10-08T00:55:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.318+0000 7fee76ffd700 1 -- 172.21.15.119:0/335609755 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6800bc20 con 0x7fee7815de70 2023-10-08T00:55:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fee7d79d700 1 -- 172.21.15.119:0/335609755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 msgr2=0x7fee7806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fee7d79d700 1 --2- 172.21.15.119:0/335609755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 0x7fee7806e000 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fee680099b0 tx=0x7fee68009250 comp rx=0 tx=0).stop 2023-10-08T00:55:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7febb739d700 1 Processor -- start 2023-10-08T00:55:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed6679d700 1 Processor -- start 2023-10-08T00:55:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed3a59c700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed24003a60 0x7fed24003e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fed6679d700 1 -- start start 2023-10-08T00:55:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7f773539c700 1 -- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 msgr2=0x7f773006c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7f773539c700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 0x7f773006c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7f773539c700 1 -- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77300b5ea0 msgr2=0x7f773006c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7f773539c700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77300b5ea0 0x7f773006c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7f773539c700 1 -- 172.21.15.119:0/4245410618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724009420 con 0x7f773015c910 2023-10-08T00:55:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7f773539c700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f773015c910 0x7f773016d3d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f772800c810 tx=0x7f7728001520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.319+0000 7fbf22b9d700 1 Processor -- start 2023-10-08T00:55:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7fbf22b9d700 1 -- start start 2023-10-08T00:55:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fed3ad9d700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed240051c0 0x7fed2400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fed3a59c700 1 -- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed24004810 msgr2=0x7fed24004c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:55:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fed3a59c700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed24004810 0x7fed24004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fed3a59c700 1 -- 172.21.15.119:0/1999650880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed240051c0 msgr2=0x7fed2400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fed3a59c700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed240051c0 0x7fed2400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fed3a59c700 1 -- 172.21.15.119:0/1999650880 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c0c6ad0 con 0x7fed24003a60 2023-10-08T00:55:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fbf22b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c0af370 0x7fbf1c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fbf22b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf1c15d8a0 0x7fbf1c0adcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fbf22b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf1c15dc70 0x7fbf1c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fbf22b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0c3370 con 0x7fbf1c15dc70 2023-10-08T00:55:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fbf22b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0c3650 con 0x7fbf1c15d8a0 2023-10-08T00:55:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fbf22b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c0c3930 con 0x7fbf1c0af370 2023-10-08T00:55:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fbf21b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c0af370 0x7fbf1c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fbf21b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c0af370 0x7fbf1c0c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52070/0 (socket says 172.21.15.119:52070) 2023-10-08T00:55:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fbf21b9b700 1 -- 172.21.15.119:0/2855594257 learned_addr learned my addr 172.21.15.119:0/2855594257 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.320+0000 7febb739d700 1 -- start start 2023-10-08T00:55:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7febb739d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feba8006a40 0x7feba80e4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7febb739d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 0x7feba80ea730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7febb739d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba8006fb0 con 0x7feba80e3f80 2023-10-08T00:55:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7febb739d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba80eac70 con 0x7feba8006a40 2023-10-08T00:55:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7febb739d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba8007290 con 0x7feba80ea2b0 2023-10-08T00:55:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7f772cff9700 1 -- 172.21.15.119:0/4245410618 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280115b0 con 0x7f773015c910 2023-10-08T00:55:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7f772cff9700 1 -- 172.21.15.119:0/4245410618 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7728011730 con 0x7f773015c910 2023-10-08T00:55:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773006d430 con 0x7f773015c910 2023-10-08T00:55:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7f7734b9b700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77300b5ea0 0x7f773006c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773006d9f0 con 0x7f773015c910 2023-10-08T00:55:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7f772cff9700 1 -- 172.21.15.119:0/4245410618 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728011af0 con 0x7f773015c910 2023-10-08T00:55:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fbf21b9b700 1 -- 172.21.15.119:0/2855594257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf1c15d8a0 msgr2=0x7fbf1c0adcd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:55:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7febb639b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7feba80e3f80 0x7feba804ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54298/0 (socket says 172.21.15.119:54298) 2023-10-08T00:55:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7febb639b700 1 -- 172.21.15.119:0/4102749663 learned_addr learned my addr 172.21.15.119:0/4102749663 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fed3a59c700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed24003a60 0x7fed24003e90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fed3000c410 tx=0x7fed30007920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9feed70773156e43 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fbf21b9b700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf1c15d8a0 0x7fbf1c0adcd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fee7d79d700 1 -- 172.21.15.119:0/335609755 shutdown_connections 2023-10-08T00:55:56.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fee7d79d700 1 --2- 172.21.15.119:0/335609755 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee7806ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fee7d79d700 1 --2- 172.21.15.119:0/335609755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 0x7fee7806e000 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.321+0000 7fee7d79d700 1 -- 172.21.15.119:0/335609755 >> 172.21.15.119:0/335609755 conn(0x7fee780cc1f0 msgr2=0x7fee780cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fee7d79d700 1 -- 172.21.15.119:0/335609755 shutdown_connections 2023-10-08T00:55:56.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fee7d79d700 1 -- 172.21.15.119:0/335609755 wait complete. 2023-10-08T00:55:56.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fbf21b9b700 1 -- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf1c15dc70 msgr2=0x7fbf1c0c5be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:55:56.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7febb6b9c700 1 --2- 172.21.15.119:0/4102749663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 0x7feba80ea730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fbf21b9b700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf1c15dc70 0x7fbf1c0c5be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7febb5b9a700 1 --2- 172.21.15.119:0/4102749663 >> v2:172.21.15.119:3300/0 conn(0x7feba8006a40 0x7feba80e4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7fbf21b9b700 1 -- 172.21.15.119:0/2855594257 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf18009420 con 0x7fbf1c0af370 2023-10-08T00:55:56.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fbf21b9b700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c0af370 0x7fbf1c0c4410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00c790 tx=0x7fbf0c007d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30007c50 con 0x7fed24003a60 2023-10-08T00:55:56.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3000cd00 con 0x7fed24003a60 2023-10-08T00:55:56.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fed3959a700 1 -- 172.21.15.119:0/1999650880 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30007c50 con 0x7fed24003a60 2023-10-08T00:55:56.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f773006def0 con 0x7f773015c910 2023-10-08T00:55:56.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fbf11ffb700 1 -- 172.21.15.119:0/2855594257 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c011590 con 0x7fbf1c0af370 2023-10-08T00:55:56.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fbf11ffb700 1 -- 172.21.15.119:0/2855594257 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c011710 con 0x7fbf1c0af370 2023-10-08T00:55:56.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fbf11ffb700 1 -- 172.21.15.119:0/2855594257 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c011b10 con 0x7fbf1c0af370 2023-10-08T00:55:56.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7febb5399700 1 -- 172.21.15.119:0/4102749663 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111089377 0 0) 0x7feba8006fb0 con 0x7feba80e3f80 2023-10-08T00:55:56.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c0c3c10 con 0x7fbf1c0af370 2023-10-08T00:55:56.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c0ae210 con 0x7fbf1c0af370 2023-10-08T00:55:56.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7fed6679d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fed6006f370 0x7fed6006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fed6679d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed600ca410 con 0x7fed6015de50 2023-10-08T00:55:56.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fed6679d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed600c7220 con 0x7fed6015da80 2023-10-08T00:55:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fed6679d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed600c7500 con 0x7fed6006f370 2023-10-08T00:55:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fee7d79d700 1 Processor -- start 2023-10-08T00:55:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fee7d79d700 1 -- start start 2023-10-08T00:55:56.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7febb5399700 1 -- 172.21.15.119:0/4102749663 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb9c003570 con 0x7feba80e3f80 2023-10-08T00:55:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fed64f9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fed6015de50 0x7fed6006ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54302/0 (socket says 172.21.15.119:54302) 2023-10-08T00:55:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7febb6b9c700 1 -- 172.21.15.119:0/4102749663 >> v2:172.21.15.119:3300/0 conn(0x7feba8006a40 msgr2=0x7feba80e4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7febb6b9c700 1 --2- 172.21.15.119:0/4102749663 >> v2:172.21.15.119:3300/0 conn(0x7feba8006a40 0x7feba80e4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.322+0000 7f0a6a3ff700 1 Processor -- start 2023-10-08T00:55:56.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.323+0000 7f0a6a3ff700 1 -- start start 2023-10-08T00:55:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7febb6b9c700 1 -- 172.21.15.119:0/4102749663 >> v1:172.21.15.119:6789/0 conn(0x7feba80e3f80 legacy=0x7feba804ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fed3b59e700 1 -- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed24003a60 msgr2=0x7fed24003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fed3b59e700 1 --2- 172.21.15.119:0/1999650880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed24003a60 0x7fed24003e90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fed3000c410 tx=0x7fed30007920 comp rx=0 tx=0).stop 2023-10-08T00:55:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fed64f9a700 1 -- 172.21.15.119:0/3195857298 learned_addr learned my addr 172.21.15.119:0/3195857298 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7febb6b9c700 1 -- 172.21.15.119:0/4102749663 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8007570 con 0x7feba80ea2b0 2023-10-08T00:55:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.324+0000 7fed65f9c700 1 --2- 172.21.15.119:0/3195857298 >> v2:172.21.15.184:3300/0 conn(0x7fed6006f370 0x7fed6006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7febb5b9a700 1 --2- 172.21.15.119:0/4102749663 >> v2:172.21.15.119:3300/0 conn(0x7feba8006a40 0x7feba80e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:55:56.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7f772cff9700 1 -- 172.21.15.119:0/4245410618 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7728003050 con 0x7f773015c910 2023-10-08T00:55:56.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed3ad9d700 1 -- 172.21.15.119:0/1999650880 reap_dead start 2023-10-08T00:55:56.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed3b59e700 1 -- 172.21.15.119:0/1999650880 shutdown_connections 2023-10-08T00:55:56.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed3b59e700 1 -- 172.21.15.119:0/1999650880 >> 172.21.15.119:0/1999650880 conn(0x7fed3c0c8bf0 msgr2=0x7fed3c0c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7febb6b9c700 1 --2- 172.21.15.119:0/4102749663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 0x7feba80ea730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7febac0098e0 tx=0x7febac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cf6bb5a2132dd9e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197762972 0 0) 0x7fed600ca410 con 0x7fed6015de50 2023-10-08T00:55:56.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4c003570 con 0x7fed6015de50 2023-10-08T00:55:56.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fee7d79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee78157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7f0a6a3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0a6415dc70 0x7f0a640b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7f0a6a3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a64069f90 0x7f0a640cae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7f0a6a3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a640cebb0 con 0x7f0a640cb370 2023-10-08T00:55:56.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7f0a6a3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a640b31f0 con 0x7f0a6415dc70 2023-10-08T00:55:56.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7f0a6a3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640cb7f0 con 0x7f0a64069f90 2023-10-08T00:55:56.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7f0a63fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0a6415dc70 0x7f0a640b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fee7d79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee7815daa0 0x7fee7815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4118699102 0 0) 0x7fed4c003570 con 0x7fed6015de50 2023-10-08T00:55:56.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7f0a5b7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a64069f90 0x7f0a640cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7f0a5b7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a64069f90 0x7f0a640cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52090/0 (socket says 172.21.15.119:52090) 2023-10-08T00:55:56.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7f0a5b7fe700 1 -- 172.21.15.119:0/964631155 learned_addr learned my addr 172.21.15.119:0/964631155 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fee77fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee78157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fee77fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee78157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52106/0 (socket says 172.21.15.119:52106) 2023-10-08T00:55:56.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fee7d79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 0x7fee78158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 >> v2:172.21.15.184:3300/0 conn(0x7fed6006f370 msgr2=0x7fed6006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7f772cff9700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f771c067080 0x7f771c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed57fff700 1 --2- 172.21.15.119:0/3195857298 >> v2:172.21.15.184:3300/0 conn(0x7fed6006f370 0x7fed6006e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 >> v1:172.21.15.184:6789/0 conn(0x7fed6015da80 legacy=0x7fed600c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fee777fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee7815daa0 0x7fee7815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fee777fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee7815daa0 0x7fee7815bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44722/0 (socket says 172.21.15.119:44722) 2023-10-08T00:55:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fee7d79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7816ecd0 con 0x7fee7815de70 2023-10-08T00:55:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fee7d79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7815e430 con 0x7fee7815daa0 2023-10-08T00:55:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed600c9dc0 con 0x7fed6015de50 2023-10-08T00:55:56.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.325+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fed5c001470 con 0x7fed6015de50 2023-10-08T00:55:56.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7f7734b9b700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f771c067080 0x7f771c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7f0a5b7fe700 1 -- 172.21.15.119:0/964631155 >> v2:172.21.15.119:3300/0 conn(0x7f0a6415dc70 msgr2=0x7f0a640b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7f0a5b7fe700 1 --2- 172.21.15.119:0/964631155 >> v2:172.21.15.119:3300/0 conn(0x7f0a6415dc70 0x7f0a640b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7f0a5b7fe700 1 -- 172.21.15.119:0/964631155 >> v1:172.21.15.119:6789/0 conn(0x7f0a640cb370 legacy=0x7f0a641618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7f772cff9700 1 -- 172.21.15.119:0/4245410618 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7728084530 con 0x7f773015c910 2023-10-08T00:55:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7f0a5b7fe700 1 -- 172.21.15.119:0/964631155 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a640b2f10 con 0x7f0a64069f90 2023-10-08T00:55:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 >> v1:172.21.15.119:6789/0 conn(0x7fed6015de50 legacy=0x7fed6006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7f0a5b7fe700 1 --2- 172.21.15.119:0/964631155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a64069f90 0x7f0a640cae30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a540099b0 tx=0x7f0a54009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74525985439ef0a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fed57fff700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed4c003b30 0x7fed4c003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fbf11ffb700 1 -- 172.21.15.119:0/2855594257 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf0c012420 con 0x7fbf1c0af370 2023-10-08T00:55:56.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fed3b59e700 1 -- 172.21.15.119:0/1999650880 shutdown_connections 2023-10-08T00:55:56.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fed57fff700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c004810 0x7fed4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fed57fff700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c0053a0 0x7fed4c00ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600c7500 con 0x7fed4c004810 2023-10-08T00:55:56.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00c360 con 0x7fed4c003b30 2023-10-08T00:55:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.326+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00c520 con 0x7fed4c0053a0 2023-10-08T00:55:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fed64f9a700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c004810 0x7fed4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fed65f9c700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c0053a0 0x7fed4c00ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fed6579b700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed4c003b30 0x7fed4c003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fed3b59e700 1 -- 172.21.15.119:0/1999650880 wait complete. 2023-10-08T00:55:56.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fed64f9a700 1 -- 172.21.15.119:0/3195857298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c0053a0 msgr2=0x7fed4c00ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fed64f9a700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c0053a0 0x7fed4c00ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fed64f9a700 1 -- 172.21.15.119:0/3195857298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed4c003b30 msgr2=0x7fed4c003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:56.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fed64f9a700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed4c003b30 0x7fed4c003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fed64f9a700 1 -- 172.21.15.119:0/3195857298 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c003570 con 0x7fed4c004810 2023-10-08T00:55:56.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fed65f9c700 1 -- 172.21.15.119:0/3195857298 reap_dead start 2023-10-08T00:55:56.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fed64f9a700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c004810 0x7fed4c004c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00c2d0 tx=0x7fed5c001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77a054cd9ff6d2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c0c4130 con 0x7fbf1c0af370 2023-10-08T00:55:56.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fee7d79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7816fd20 con 0x7fee780696c0 2023-10-08T00:55:56.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fee77fff700 1 -- 172.21.15.119:0/554315498 learned_addr learned my addr 172.21.15.119:0/554315498 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.327+0000 7fee7cf9c700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 0x7fee78158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fee7cf9c700 1 -- 172.21.15.119:0/554315498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 msgr2=0x7fee78157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fee7cf9c700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee78157090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fee7cf9c700 1 -- 172.21.15.119:0/554315498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee7815daa0 msgr2=0x7fee7815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fee7cf9c700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee7815daa0 0x7fee7815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fee7cf9c700 1 -- 172.21.15.119:0/554315498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee680094f0 con 0x7fee7815de70 2023-10-08T00:55:56.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.328+0000 7fee7cf9c700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 0x7fee78158270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee7000ecf0 tx=0x7fee7000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fee77fff700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee78157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:56.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fbf11ffb700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf00067000 0x7fbf000694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fee777fe700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee7815daa0 0x7fee7815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:56.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fbf11ffb700 1 -- 172.21.15.119:0/2855594257 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbf0c0842c0 con 0x7fbf1c0af370 2023-10-08T00:55:56.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7f7734b9b700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f771c067080 0x7f771c069530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f772400a010 tx=0x7f7724017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c00cba0 con 0x7fed4c004810 2023-10-08T00:55:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c00cd00 con 0x7fed4c004810 2023-10-08T00:55:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fed57fff700 1 -- 172.21.15.119:0/3195857298 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c005b50 con 0x7fed4c004810 2023-10-08T00:55:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fbf2139a700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf00067000 0x7fbf000694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed3b59e700 1 Processor -- start 2023-10-08T00:55:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fee5ffff700 1 -- 172.21.15.119:0/554315498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7000ca30 con 0x7fee7815de70 2023-10-08T00:55:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fee5ffff700 1 -- 172.21.15.119:0/554315498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7000f040 con 0x7fee7815de70 2023-10-08T00:55:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.329+0000 7fee5ffff700 1 -- 172.21.15.119:0/554315498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7000ce00 con 0x7fee7815de70 2023-10-08T00:55:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7816f110 con 0x7fee7815de70 2023-10-08T00:55:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed3b59e700 1 -- start start 2023-10-08T00:55:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed6679d700 1 -- 172.21.15.119:0/3195857298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c004810 msgr2=0x7fed4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed6679d700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c004810 0x7fed4c004c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fed5c00c2d0 tx=0x7fed5c001da0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7816f6d0 con 0x7fee7815de70 2023-10-08T00:55:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed6679d700 1 -- 172.21.15.119:0/3195857298 shutdown_connections 2023-10-08T00:55:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed6679d700 1 --2- 172.21.15.119:0/3195857298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed4c004810 0x7fed4c004c80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed6679d700 1 -- 172.21.15.119:0/3195857298 >> 172.21.15.119:0/3195857298 conn(0x7fed600cc1f0 msgr2=0x7fed600cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fbf2139a700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf00067000 0x7fbf000694b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0c2bc0 tx=0x7fbf18017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed6679d700 1 -- 172.21.15.119:0/3195857298 shutdown_connections 2023-10-08T00:55:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.330+0000 7fed6679d700 1 -- 172.21.15.119:0/3195857298 wait complete. 2023-10-08T00:55:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.331+0000 7f772cff9700 1 -- 172.21.15.119:0/4245410618 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7728051700 con 0x7f773015c910 2023-10-08T00:55:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.331+0000 7f0a637fe700 1 -- 172.21.15.119:0/964631155 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5400b820 con 0x7f0a64069f90 2023-10-08T00:55:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.331+0000 7febb5399700 1 -- 172.21.15.119:0/4102749663 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac011630 con 0x7feba80ea2b0 2023-10-08T00:55:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.331+0000 7fed3b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c15ca70 0x7fed3c15e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.331+0000 7fed3b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed3c15eb20 0x7fed3c16e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.331+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7816ee90 con 0x7fee7815de70 2023-10-08T00:55:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.332+0000 7fed3a59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c15ca70 0x7fed3c15e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.332+0000 7fed3b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed3c16e590 0x7fed3c16e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.332+0000 7fed3a59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c15ca70 0x7fed3c15e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52116/0 (socket says 172.21.15.119:52116) 2023-10-08T00:55:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.332+0000 7fed39d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed3c15eb20 0x7fed3c16e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.332+0000 7fed39d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed3c15eb20 0x7fed3c16e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33060/0 (socket says 172.21.15.119:33060) 2023-10-08T00:55:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.332+0000 7fed3a59c700 1 -- 172.21.15.119:0/318521618 learned_addr learned my addr 172.21.15.119:0/318521618 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.332+0000 7fed3ad9d700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed3c16e590 0x7fed3c16e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.333+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c15fbb0 con 0x7fed3c15eb20 2023-10-08T00:55:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.333+0000 7fee5ffff700 1 -- 172.21.15.119:0/554315498 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee70014690 con 0x7fee7815de70 2023-10-08T00:55:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.333+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c16ef20 con 0x7fed3c16e590 2023-10-08T00:55:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.333+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed3c15f310 con 0x7fed3c15ca70 2023-10-08T00:55:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.333+0000 7fed6679d700 1 Processor -- start 2023-10-08T00:55:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.333+0000 7fed6679d700 1 -- start start 2023-10-08T00:55:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.334+0000 7fed3a59c700 1 -- 172.21.15.119:0/318521618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed3c16e590 msgr2=0x7fed3c16e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.334+0000 7fed3a59c700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed3c16e590 0x7fed3c16e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.334+0000 7fed3a59c700 1 -- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed3c15eb20 msgr2=0x7fed3c16e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.334+0000 7fee5ffff700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee64067080 0x7fee64069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.334+0000 7fed3a59c700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed3c15eb20 0x7fed3c16e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.334+0000 7fee5ffff700 1 -- 172.21.15.119:0/554315498 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee70085070 con 0x7fee7815de70 2023-10-08T00:55:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.334+0000 7fed3a59c700 1 -- 172.21.15.119:0/318521618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c15f5f0 con 0x7fed3c15ca70 2023-10-08T00:55:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.334+0000 7fee77fff700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee64067080 0x7fee64069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.335+0000 7fed3a59c700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c15ca70 0x7fed3c15e5e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed3000ca00 tx=0x7fed30015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.335+0000 7fee77fff700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee64067080 0x7fee64069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fee78158fb0 tx=0x7fee6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.335+0000 7fbf11ffb700 1 -- 172.21.15.119:0/2855594257 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf1c0c4130 con 0x7fbf1c0af370 2023-10-08T00:55:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7fed6679d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c004810 0x7fed6015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7fed6679d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed60153940 0x7fed60160480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7fed6679d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed600c51e0 0x7fed600c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7fed6679d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600b01b0 con 0x7fed600c51e0 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7fed6679d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6015e2d0 con 0x7fed60153940 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7fed6679d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6015e890 con 0x7fed4c004810 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.335+0000 7f0a637fe700 1 -- 172.21.15.119:0/964631155 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5400b980 con 0x7f0a64069f90 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.335+0000 7f0a637fe700 1 -- 172.21.15.119:0/964631155 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5400bd00 con 0x7f0a64069f90 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/964631155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a64069f90 msgr2=0x7f0a640cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/964631155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a64069f90 0x7f0a640cae30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a540099b0 tx=0x7f0a54009250 comp rx=0 tx=0).stop 2023-10-08T00:55:56.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.335+0000 7febb5399700 1 -- 172.21.15.119:0/4102749663 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac011790 con 0x7feba80ea2b0 2023-10-08T00:55:56.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.335+0000 7febb5399700 1 -- 172.21.15.119:0/4102749663 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac011b10 con 0x7feba80ea2b0 2023-10-08T00:55:56.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7febb739d700 1 -- 172.21.15.119:0/4102749663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 msgr2=0x7feba80ea730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.336+0000 7febb739d700 1 --2- 172.21.15.119:0/4102749663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 0x7feba80ea730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7febac0098e0 tx=0x7febac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7febb739d700 1 -- 172.21.15.119:0/4102749663 shutdown_connections 2023-10-08T00:55:56.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7febb739d700 1 --2- 172.21.15.119:0/4102749663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 0x7feba80ea730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7febb739d700 1 --2- 172.21.15.119:0/4102749663 >> v2:172.21.15.119:3300/0 conn(0x7feba8006a40 0x7feba80e4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7febb739d700 1 -- 172.21.15.119:0/4102749663 >> 172.21.15.119:0/4102749663 conn(0x7feba8054da0 msgr2=0x7feba80551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/964631155 shutdown_connections 2023-10-08T00:55:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/964631155 >> v2:172.21.15.119:3300/0 conn(0x7f0a6415dc70 0x7f0a640b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/964631155 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a64069f90 0x7f0a640cae30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/964631155 >> 172.21.15.119:0/964631155 conn(0x7f0a64068e20 msgr2=0x7f0a640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed65f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed600c51e0 0x7fed600c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed65f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed600c51e0 0x7fed600c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33074/0 (socket says 172.21.15.119:33074) 2023-10-08T00:55:56.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed65f9c700 1 -- 172.21.15.119:0/595657102 learned_addr learned my addr 172.21.15.119:0/595657102 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed2a7fc700 1 -- 172.21.15.119:0/318521618 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30007c50 con 0x7fed3c15ca70 2023-10-08T00:55:56.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/964631155 shutdown_connections 2023-10-08T00:55:56.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed3c15f8d0 con 0x7fed3c15ca70 2023-10-08T00:55:56.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3c15ef90 con 0x7fed3c15ca70 2023-10-08T00:55:56.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed65f9c700 1 -- 172.21.15.119:0/595657102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c004810 msgr2=0x7fed6015ed20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:55:56.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed65f9c700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c004810 0x7fed6015ed20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed65f9c700 1 -- 172.21.15.119:0/595657102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed60153940 msgr2=0x7fed60160480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:55:56.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed65f9c700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed60153940 0x7fed60160480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7fed65f9c700 1 -- 172.21.15.119:0/595657102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed600c5b90 con 0x7fed600c51e0 2023-10-08T00:55:56.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.338+0000 7fed2a7fc700 1 -- 172.21.15.119:0/318521618 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed30007460 con 0x7fed3c15ca70 2023-10-08T00:55:56.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.338+0000 7fed65f9c700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed600c51e0 0x7fed600c5650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed600ca380 tx=0x7fed58009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.339+0000 7fed2a7fc700 1 -- 172.21.15.119:0/318521618 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30007800 con 0x7fed3c15ca70 2023-10-08T00:55:56.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.338+0000 7febb739d700 1 -- 172.21.15.119:0/4102749663 shutdown_connections 2023-10-08T00:55:56.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.340+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed200052b0 con 0x7fed3c15ca70 2023-10-08T00:55:56.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.338+0000 7febb739d700 1 -- 172.21.15.119:0/4102749663 wait complete. 2023-10-08T00:55:56.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.341+0000 7febb739d700 1 Processor -- start 2023-10-08T00:55:56.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.341+0000 7febb739d700 1 -- start start 2023-10-08T00:55:56.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.337+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/964631155 wait complete. 2023-10-08T00:55:56.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7fee5ffff700 1 -- 172.21.15.119:0/554315498 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee700522c0 con 0x7fee7815de70 2023-10-08T00:55:56.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.340+0000 7f0a6a3ff700 1 Processor -- start 2023-10-08T00:55:56.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.340+0000 7f0a6a3ff700 1 -- start start 2023-10-08T00:55:56.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.341+0000 7f0a6a3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a64069f90 0x7f0a64154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.341+0000 7f0a6a3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a640cb370 0x7f0a640b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a6a3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a6415dc70 0x7f0a640b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a6a3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640b6060 con 0x7f0a64069f90 2023-10-08T00:55:56.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7fed557fa700 1 -- 172.21.15.119:0/595657102 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58009b10 con 0x7fed600c51e0 2023-10-08T00:55:56.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7fed557fa700 1 -- 172.21.15.119:0/595657102 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed58002d50 con 0x7fed600c51e0 2023-10-08T00:55:56.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7fed557fa700 1 -- 172.21.15.119:0/595657102 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58015670 con 0x7fed600c51e0 2023-10-08T00:55:56.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a6a3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a640b48b0 con 0x7f0a640cb370 2023-10-08T00:55:56.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed601609c0 con 0x7fed600c51e0 2023-10-08T00:55:56.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a6a3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a64155cf0 con 0x7f0a6415dc70 2023-10-08T00:55:56.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7febb739d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8006a40 0x7feba80d3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a5b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a640cb370 0x7f0a640b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed600b06e0 con 0x7fed600c51e0 2023-10-08T00:55:56.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a5b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a640cb370 0x7f0a640b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44752/0 (socket says 172.21.15.119:44752) 2023-10-08T00:55:56.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a5b7fe700 1 -- 172.21.15.119:0/251446322 learned_addr learned my addr 172.21.15.119:0/251446322 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7febb739d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feba80e3f80 0x7feba803a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a6899c700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a6415dc70 0x7f0a640b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.342+0000 7f0a63fff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a64069f90 0x7f0a64154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7febb739d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 0x7feba80e2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7f0a5b7fe700 1 -- 172.21.15.119:0/251446322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a6415dc70 msgr2=0x7f0a640b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7f0a5b7fe700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a6415dc70 0x7f0a640b59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7febb739d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba80d4320 con 0x7feba8006a40 2023-10-08T00:55:56.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7febb739d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba80d4600 con 0x7feba80e3f80 2023-10-08T00:55:56.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7febb639b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8006a40 0x7feba80d3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7febb739d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba80d48e0 con 0x7feba80ea2b0 2023-10-08T00:55:56.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7febb639b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8006a40 0x7feba80d3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33100/0 (socket says 172.21.15.119:33100) 2023-10-08T00:55:56.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7febb639b700 1 -- 172.21.15.119:0/4191536799 learned_addr learned my addr 172.21.15.119:0/4191536799 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:56.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7f0a5b7fe700 1 -- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a64069f90 msgr2=0x7f0a64154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7f0a5b7fe700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a64069f90 0x7f0a64154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7f0a5b7fe700 1 -- 172.21.15.119:0/251446322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a540094f0 con 0x7f0a640cb370 2023-10-08T00:55:56.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.343+0000 7f0a5b7fe700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a640cb370 0x7f0a640b5480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009290 tx=0x7f0a54016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7fed2a7fc700 1 -- 172.21.15.119:0/318521618 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed30014630 con 0x7fed3c15ca70 2023-10-08T00:55:56.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7f0a60ff9700 1 -- 172.21.15.119:0/251446322 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a54011ce0 con 0x7f0a640cb370 2023-10-08T00:55:56.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7f0a60ff9700 1 -- 172.21.15.119:0/251446322 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a54020070 con 0x7f0a640cb370 2023-10-08T00:55:56.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.344+0000 7f0a60ff9700 1 -- 172.21.15.119:0/251446322 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5401c6a0 con 0x7f0a640cb370 2023-10-08T00:55:56.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.345+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a641553c0 con 0x7f0a640cb370 2023-10-08T00:55:56.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.345+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a64155980 con 0x7f0a640cb370 2023-10-08T00:55:56.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.345+0000 7febb639b700 1 -- 172.21.15.119:0/4191536799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 msgr2=0x7feba80e2780 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:55:56.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.345+0000 7febb639b700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 0x7feba80e2780 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.345+0000 7febb639b700 1 -- 172.21.15.119:0/4191536799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feba80e3f80 msgr2=0x7feba803a470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:55:56.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.346+0000 7febb639b700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feba80e3f80 0x7feba803a470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.346+0000 7febb639b700 1 -- 172.21.15.119:0/4191536799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0009420 con 0x7feba8006a40 2023-10-08T00:55:56.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.346+0000 7febb639b700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8006a40 0x7feba80d3d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7febb004d2d0 tx=0x7febb0067410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.346+0000 7fed2a7fc700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed1c066f80 0x7fed1c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.346+0000 7fed2a7fc700 1 -- 172.21.15.119:0/318521618 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fed300578f0 con 0x7fed3c15ca70 2023-10-08T00:55:56.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.347+0000 7fed557fa700 1 -- 172.21.15.119:0/595657102 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed58003040 con 0x7fed600c51e0 2023-10-08T00:55:56.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.347+0000 7feba67fc700 1 -- 172.21.15.119:0/4191536799 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0067ee0 con 0x7feba8006a40 2023-10-08T00:55:56.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.347+0000 7feba67fc700 1 -- 172.21.15.119:0/4191536799 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb0068040 con 0x7feba8006a40 2023-10-08T00:55:56.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.348+0000 7fed557fa700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed44067080 0x7fed44069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.347+0000 7feba67fc700 1 -- 172.21.15.119:0/4191536799 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb006a620 con 0x7feba8006a40 2023-10-08T00:55:56.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.348+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac009420 con 0x7feba8006a40 2023-10-08T00:55:56.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.348+0000 7fed557fa700 1 -- 172.21.15.119:0/595657102 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fed58084ef0 con 0x7fed600c51e0 2023-10-08T00:55:56.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.347+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6404c3c0 con 0x7f0a640cb370 2023-10-08T00:55:56.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.348+0000 7f0a60ff9700 1 -- 172.21.15.119:0/251446322 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a54022030 con 0x7f0a640cb370 2023-10-08T00:55:56.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.348+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba803a9b0 con 0x7feba8006a40 2023-10-08T00:55:56.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.349+0000 7f0a60ff9700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0a44067000 0x7f0a440694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.349+0000 7f0a60ff9700 1 -- 172.21.15.119:0/251446322 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0a540863e0 con 0x7f0a640cb370 2023-10-08T00:55:56.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.349+0000 7fed6579b700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed44067080 0x7fed44069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.349+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed600b08d0 con 0x7fed600c51e0 2023-10-08T00:55:56.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.350+0000 7f0a63fff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0a44067000 0x7f0a440694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.350+0000 7f0a63fff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0a44067000 0x7f0a440694b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a640cabe0 tx=0x7f0a4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.350+0000 7fed6579b700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed44067080 0x7fed44069530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed500039a0 tx=0x7fed50007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.351+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba80d4e00 con 0x7feba8006a40 2023-10-08T00:55:56.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.352+0000 7fed2a7fc700 1 -- 172.21.15.119:0/318521618 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed30053cc0 con 0x7fed3c15ca70 2023-10-08T00:55:56.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.352+0000 7fed39d9b700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed1c066f80 0x7fed1c069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.353+0000 7fed39d9b700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed1c066f80 0x7fed1c069430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0c6720 tx=0x7fed2c0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.352+0000 7feba67fc700 1 -- 172.21.15.119:0/4191536799 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7febb006aa50 con 0x7feba8006a40 2023-10-08T00:55:56.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.354+0000 7feba67fc700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7feb94067000 0x7feb940694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:56.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.354+0000 7feba67fc700 1 -- 172.21.15.119:0/4191536799 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7febb00cd1d0 con 0x7feba8006a40 2023-10-08T00:55:56.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.354+0000 7f0a60ff9700 1 -- 172.21.15.119:0/251446322 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a54053630 con 0x7f0a640cb370 2023-10-08T00:55:56.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.355+0000 7febb5b9a700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7feb94067000 0x7feb940694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:56.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.356+0000 7febb5b9a700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7feb94067000 0x7feb940694b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feba804af80 tx=0x7feba00091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.360+0000 7fed557fa700 1 -- 172.21.15.119:0/595657102 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed58052140 con 0x7fed600c51e0 2023-10-08T00:55:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.369+0000 7feba67fc700 1 -- 172.21.15.119:0/4191536799 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7febb00c95a0 con 0x7feba8006a40 2023-10-08T00:55:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.693+0000 7f3c8a1ff700 1 -- 172.21.15.119:0/2071022494 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3c64005560 con 0x7f3c84068aa0 2023-10-08T00:55:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.694+0000 7f3c6b7fe700 1 -- 172.21.15.119:0/2071022494 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3c7805ccd0 con 0x7f3c84068aa0 2023-10-08T00:55:56.695 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.698+0000 7f3c697fa700 1 -- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c700670b0 msgr2=0x7f3c70069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.698+0000 7f3c697fa700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c700670b0 0x7f3c70069560 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3c840b2090 tx=0x7f3c7401c000 comp rx=0 tx=0).stop 2023-10-08T00:55:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.698+0000 7f3c697fa700 1 -- 172.21.15.119:0/2071022494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 msgr2=0x7f3c841773c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.698+0000 7f3c697fa700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c841773c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3c78002e60 tx=0x7f3c78007dd0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.699+0000 7f3c697fa700 1 -- 172.21.15.119:0/2071022494 shutdown_connections 2023-10-08T00:55:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.699+0000 7f3c697fa700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c84068aa0 0x7f3c841773c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.699+0000 7f3c697fa700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c700670b0 0x7f3c70069560 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.699+0000 7f3c697fa700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c84160900 0x7f3c841685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.699+0000 7f3c697fa700 1 --2- 172.21.15.119:0/2071022494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c840875d0 0x7f3c8406e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.699+0000 7f3c697fa700 1 -- 172.21.15.119:0/2071022494 >> 172.21.15.119:0/2071022494 conn(0x7f3c840cd000 msgr2=0x7f3c840ade20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.700+0000 7f3c697fa700 1 -- 172.21.15.119:0/2071022494 shutdown_connections 2023-10-08T00:55:56.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.700+0000 7f3c697fa700 1 -- 172.21.15.119:0/2071022494 wait complete. 2023-10-08T00:55:56.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-10-08T00:55:56.731 DEBUG:teuthology.parallel:result is None 2023-10-08T00:55:56.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.759+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0a6404c3c0 con 0x7f0a640cb370 2023-10-08T00:55:56.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.760+0000 7f0a60ff9700 1 -- 172.21.15.119:0/251446322 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a54011e60 con 0x7f0a640cb370 2023-10-08T00:55:56.761 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:56.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.762+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0a44067000 msgr2=0x7f0a440694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.762+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0a44067000 0x7f0a440694b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0a640cabe0 tx=0x7f0a4c008040 comp rx=0 tx=0).stop 2023-10-08T00:55:56.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.762+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a640cb370 msgr2=0x7f0a640b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.762+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a640cb370 0x7f0a640b5480 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0a54009290 tx=0x7f0a54016000 comp rx=0 tx=0).stop 2023-10-08T00:55:56.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.763+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 shutdown_connections 2023-10-08T00:55:56.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.763+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a6415dc70 0x7f0a640b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.763+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0a44067000 0x7f0a440694b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.763+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a64069f90 0x7f0a64154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.763+0000 7f0a6a3ff700 1 --2- 172.21.15.119:0/251446322 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a640cb370 0x7f0a640b5480 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.763+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 >> 172.21.15.119:0/251446322 conn(0x7f0a64068e20 msgr2=0x7f0a640cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.764+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 shutdown_connections 2023-10-08T00:55:56.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.764+0000 7f0a6a3ff700 1 -- 172.21.15.119:0/251446322 wait complete. 2023-10-08T00:55:56.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.765+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f773004c3c0 con 0x7f773015c910 2023-10-08T00:55:56.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.766+0000 7f772cff9700 1 -- 172.21.15.119:0/4245410618 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7728054d20 con 0x7f773015c910 2023-10-08T00:55:56.767 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2023-10-08T00:55:56.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.769+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f771c067080 msgr2=0x7f771c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.770+0000 7f7736dff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f771c067080 0x7f771c069530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f772400a010 tx=0x7f7724017000 comp rx=0 tx=0).stop 2023-10-08T00:55:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.770+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f773015c910 msgr2=0x7f773016d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.770+0000 7f7736dff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f773015c910 0x7f773016d3d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f772800c810 tx=0x7f7728001520 comp rx=0 tx=0).stop 2023-10-08T00:55:56.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.770+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 shutdown_connections 2023-10-08T00:55:56.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.771+0000 7f7736dff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f773015c910 0x7f773016d3d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.771+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbf1c0c4130 con 0x7fbf1c0af370 2023-10-08T00:55:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.771+0000 7f7736dff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f771c067080 0x7f771c069530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.771+0000 7f7736dff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77300b5ea0 0x7f773006c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.771+0000 7f7736dff700 1 --2- 172.21.15.119:0/4245410618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7730159c10 0x7f773006c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.771+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 >> 172.21.15.119:0/4245410618 conn(0x7f7730099a20 msgr2=0x7f77300b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.771+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 shutdown_connections 2023-10-08T00:55:56.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.771+0000 7f7736dff700 1 -- 172.21.15.119:0/4245410618 wait complete. 2023-10-08T00:55:56.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.772+0000 7fbf11ffb700 1 -- 172.21.15.119:0/2855594257 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbf0c051510 con 0x7fbf1c0af370 2023-10-08T00:55:56.775 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:56.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.776+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf00067000 msgr2=0x7fbf000694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.776+0000 7fbf22b9d700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf00067000 0x7fbf000694b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0c2bc0 tx=0x7fbf18017000 comp rx=0 tx=0).stop 2023-10-08T00:55:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c0af370 msgr2=0x7fbf1c0c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c0af370 0x7fbf1c0c4410 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00c790 tx=0x7fbf0c007d70 comp rx=0 tx=0).stop 2023-10-08T00:55:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 shutdown_connections 2023-10-08T00:55:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf1c0af370 0x7fbf1c0c4410 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf00067000 0x7fbf000694b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf1c15dc70 0x7fbf1c0c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 --2- 172.21.15.119:0/2855594257 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf1c15d8a0 0x7fbf1c0adcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 >> 172.21.15.119:0/2855594257 conn(0x7fbf1c0ccc80 msgr2=0x7fbf1c0b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 shutdown_connections 2023-10-08T00:55:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.777+0000 7fbf22b9d700 1 -- 172.21.15.119:0/2855594257 wait complete. 2023-10-08T00:55:56.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.779+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fee7804c3c0 con 0x7fee7815de70 2023-10-08T00:55:56.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.780+0000 7fee5ffff700 1 -- 172.21.15.119:0/554315498 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fee700558e0 con 0x7fee7815de70 2023-10-08T00:55:56.782 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:56.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee64067080 msgr2=0x7fee64069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee64067080 0x7fee64069530 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fee78158fb0 tx=0x7fee6c009040 comp rx=0 tx=0).stop 2023-10-08T00:55:56.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 msgr2=0x7fee78158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 0x7fee78158270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fee7000ecf0 tx=0x7fee7000b040 comp rx=0 tx=0).stop 2023-10-08T00:55:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 shutdown_connections 2023-10-08T00:55:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee780696c0 0x7fee78157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee64067080 0x7fee64069530 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee7815de70 0x7fee78158270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 --2- 172.21.15.119:0/554315498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee7815daa0 0x7fee7815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.784+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 >> 172.21.15.119:0/554315498 conn(0x7fee780cc1f0 msgr2=0x7fee781593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.785+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 shutdown_connections 2023-10-08T00:55:56.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.785+0000 7fee7d79d700 1 -- 172.21.15.119:0/554315498 wait complete. 2023-10-08T00:55:56.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.789+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fed6004c3c0 con 0x7fed600c51e0 2023-10-08T00:55:56.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.789+0000 7fed557fa700 1 -- 172.21.15.119:0/595657102 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed58055760 con 0x7fed600c51e0 2023-10-08T00:55:56.791 INFO:teuthology.orchestra.run.smithi119.stdout:55834574854 2023-10-08T00:55:56.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-10-08T00:55:56.792 DEBUG:teuthology.parallel:result is None 2023-10-08T00:55:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.792+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed44067080 msgr2=0x7fed44069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.792+0000 7fed6679d700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed44067080 0x7fed44069530 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fed500039a0 tx=0x7fed50007d30 comp rx=0 tx=0).stop 2023-10-08T00:55:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.792+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed600c51e0 msgr2=0x7fed600c5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.792+0000 7fed6679d700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed600c51e0 0x7fed600c5650 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fed600ca380 tx=0x7fed58009710 comp rx=0 tx=0).stop 2023-10-08T00:55:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.793+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 shutdown_connections 2023-10-08T00:55:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.793+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fed20005260 con 0x7fed3c15ca70 2023-10-08T00:55:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.793+0000 7fed6679d700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed4c004810 0x7fed6015ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.794+0000 7fed6679d700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed44067080 0x7fed44069530 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.794+0000 7fed6679d700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed600c51e0 0x7fed600c5650 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.794+0000 7fed6679d700 1 --2- 172.21.15.119:0/595657102 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed60153940 0x7fed60160480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.794+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 >> 172.21.15.119:0/595657102 conn(0x7fed600cc1f0 msgr2=0x7fed600c1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.794+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 shutdown_connections 2023-10-08T00:55:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.794+0000 7fed6679d700 1 -- 172.21.15.119:0/595657102 wait complete. 2023-10-08T00:55:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.794+0000 7fed2a7fc700 1 -- 172.21.15.119:0/318521618 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fed30053e60 con 0x7fed3c15ca70 2023-10-08T00:55:56.797 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2023-10-08T00:55:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed1c066f80 msgr2=0x7fed1c069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed1c066f80 0x7fed1c069430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fed3c0c6720 tx=0x7fed2c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c15ca70 msgr2=0x7fed3c15e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c15ca70 0x7fed3c15e5e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fed3000ca00 tx=0x7fed30015000 comp rx=0 tx=0).stop 2023-10-08T00:55:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 shutdown_connections 2023-10-08T00:55:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed3c15ca70 0x7fed3c15e5e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fed1c066f80 0x7fed1c069430 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed3c16e590 0x7fed3c16e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 --2- 172.21.15.119:0/318521618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed3c15eb20 0x7fed3c16e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 >> 172.21.15.119:0/318521618 conn(0x7fed3c0c8bf0 msgr2=0x7fed3c069ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 shutdown_connections 2023-10-08T00:55:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.797+0000 7fed3b59e700 1 -- 172.21.15.119:0/318521618 wait complete. 2023-10-08T00:55:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.799+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7feba80054e0 con 0x7feba8006a40 2023-10-08T00:55:56.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-10-08T00:55:56.802 DEBUG:teuthology.parallel:result is None 2023-10-08T00:55:56.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.801+0000 7feba67fc700 1 -- 172.21.15.119:0/4191536799 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7febb00c95a0 con 0x7feba8006a40 2023-10-08T00:55:56.803 INFO:teuthology.orchestra.run.smithi119.stdout:55834574855 2023-10-08T00:55:56.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.804+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7feb94067000 msgr2=0x7feb940694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.804+0000 7febb739d700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7feb94067000 0x7feb940694b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7feba804af80 tx=0x7feba00091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.804+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8006a40 msgr2=0x7feba80d3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.804+0000 7febb739d700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8006a40 0x7feba80d3d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7febb004d2d0 tx=0x7febb0067410 comp rx=0 tx=0).stop 2023-10-08T00:55:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.806+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 shutdown_connections 2023-10-08T00:55:56.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.806+0000 7febb739d700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feba80ea2b0 0x7feba80e2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.806+0000 7febb739d700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7feb94067000 0x7feb940694b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.806+0000 7febb739d700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feba8006a40 0x7feba80d3d40 secure :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7febb004d2d0 tx=0x7febb0067410 comp rx=0 tx=0).stop 2023-10-08T00:55:56.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.806+0000 7febb739d700 1 --2- 172.21.15.119:0/4191536799 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feba80e3f80 0x7feba803a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:56.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.806+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 >> 172.21.15.119:0/4191536799 conn(0x7feba8054da0 msgr2=0x7feba8049b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:56.809 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-08T00:55:56.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.808+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 shutdown_connections 2023-10-08T00:55:56.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:56.808+0000 7febb739d700 1 -- 172.21.15.119:0/4191536799 wait complete. 2023-10-08T00:55:56.810 DEBUG:teuthology.parallel:result is None 2023-10-08T00:55:56.814 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-10-08T00:55:56.815 DEBUG:teuthology.parallel:result is None 2023-10-08T00:55:56.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-10-08T00:55:56.828 DEBUG:teuthology.parallel:result is None 2023-10-08T00:55:56.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.7 2023-10-08T00:55:56.829 DEBUG:teuthology.parallel:result is None 2023-10-08T00:55:56.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-08T00:55:56.830 DEBUG:teuthology.parallel:result is None 2023-10-08T00:55:56.831 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T00:55:56.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:55:57.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.047+0000 7f43563ff700 1 Processor -- start 2023-10-08T00:55:57.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.048+0000 7f43563ff700 1 -- start start 2023-10-08T00:55:57.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f43563ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43501599e0 0x7f4350069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f43563ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4350159db0 0x7f4350069080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f43563ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43500c96a0 con 0x7f435006a2b0 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f43563ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43500cb9e0 con 0x7f43501599e0 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f43563ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43500cab60 con 0x7f4350159db0 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f434ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43501599e0 0x7f4350069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f434ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43501599e0 0x7f4350069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33112/0 (socket says 172.21.15.119:33112) 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f434ffff700 1 -- 172.21.15.119:0/2436737643 learned_addr learned my addr 172.21.15.119:0/2436737643 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.049+0000 7f4347fff700 1 --2- 172.21.15.119:0/2436737643 >> v2:172.21.15.184:3300/0 conn(0x7f4350159db0 0x7f4350069080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.050+0000 7f434ffff700 1 -- 172.21.15.119:0/2436737643 >> v2:172.21.15.184:3300/0 conn(0x7f4350159db0 msgr2=0x7f4350069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.050+0000 7f434ffff700 1 --2- 172.21.15.119:0/2436737643 >> v2:172.21.15.184:3300/0 conn(0x7f4350159db0 0x7f4350069080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.050+0000 7f434ffff700 1 -- 172.21.15.119:0/2436737643 >> v1:172.21.15.184:6789/0 conn(0x7f435006a2b0 legacy=0x7f435006a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.050+0000 7f434ffff700 1 -- 172.21.15.119:0/2436737643 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500cb420 con 0x7f43501599e0 2023-10-08T00:55:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.050+0000 7f434ffff700 1 --2- 172.21.15.119:0/2436737643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f4350069ac0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43500c73c0 tx=0x7f4340009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b65b93c7931debe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.050+0000 7f434f7fe700 1 -- 172.21.15.119:0/2436737643 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4340009e70 con 0x7f43501599e0 2023-10-08T00:55:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.050+0000 7f434f7fe700 1 -- 172.21.15.119:0/2436737643 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434000b860 con 0x7f43501599e0 2023-10-08T00:55:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.050+0000 7f434f7fe700 1 -- 172.21.15.119:0/2436737643 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000bc00 con 0x7f43501599e0 2023-10-08T00:55:57.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.051+0000 7f43563ff700 1 -- 172.21.15.119:0/2436737643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 msgr2=0x7f4350069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.051+0000 7f43563ff700 1 --2- 172.21.15.119:0/2436737643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f4350069ac0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f43500c73c0 tx=0x7f4340009250 comp rx=0 tx=0).stop 2023-10-08T00:55:57.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.051+0000 7f43563ff700 1 -- 172.21.15.119:0/2436737643 shutdown_connections 2023-10-08T00:55:57.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.051+0000 7f43563ff700 1 --2- 172.21.15.119:0/2436737643 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f4350069ac0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.051+0000 7f43563ff700 1 --2- 172.21.15.119:0/2436737643 >> v2:172.21.15.184:3300/0 conn(0x7f4350159db0 0x7f4350069080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.051+0000 7f43563ff700 1 -- 172.21.15.119:0/2436737643 >> 172.21.15.119:0/2436737643 conn(0x7f43500c9860 msgr2=0x7f43500c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:57.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.051+0000 7f43563ff700 1 -- 172.21.15.119:0/2436737643 shutdown_connections 2023-10-08T00:55:57.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.051+0000 7f43563ff700 1 -- 172.21.15.119:0/2436737643 wait complete. 2023-10-08T00:55:57.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.052+0000 7f43563ff700 1 Processor -- start 2023-10-08T00:55:57.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.053+0000 7f43563ff700 1 -- start start 2023-10-08T00:55:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.053+0000 7f43563ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f435006a2b0 0x7f43500c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.053+0000 7f43563ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f435014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.053+0000 7f43563ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4350159db0 0x7f43500c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.053+0000 7f43563ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500c6d90 con 0x7f43501599e0 2023-10-08T00:55:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.053+0000 7f43563ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43500c5120 con 0x7f4350159db0 2023-10-08T00:55:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.053+0000 7f43563ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43500c6ab0 con 0x7f435006a2b0 2023-10-08T00:55:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f435014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f435014cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33126/0 (socket says 172.21.15.119:33126) 2023-10-08T00:55:57.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 -- 172.21.15.119:0/3242058263 learned_addr learned my addr 172.21.15.119:0/3242058263 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:57.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f435499c700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4350159db0 0x7f43500c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f434ffff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f435006a2b0 0x7f43500c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 -- 172.21.15.119:0/3242058263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f435006a2b0 msgr2=0x7f43500c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f435006a2b0 0x7f43500c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 -- 172.21.15.119:0/3242058263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4350159db0 msgr2=0x7f43500c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4350159db0 0x7f43500c5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 -- 172.21.15.119:0/3242058263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400094f0 con 0x7f43501599e0 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.054+0000 7f4347fff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f435014cd40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f433800e900 tx=0x7f43380091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.055+0000 7f434cff9700 1 -- 172.21.15.119:0/3242058263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338013680 con 0x7f43501599e0 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.055+0000 7f434cff9700 1 -- 172.21.15.119:0/3242058263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43380137e0 con 0x7f43501599e0 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.055+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43500c4ad0 con 0x7f43501599e0 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.055+0000 7f434cff9700 1 -- 172.21.15.119:0/3242058263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4338013ba0 con 0x7f43501599e0 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.055+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f435015afb0 con 0x7f43501599e0 2023-10-08T00:55:57.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.056+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43500c4930 con 0x7f43501599e0 2023-10-08T00:55:57.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.057+0000 7f434cff9700 1 -- 172.21.15.119:0/3242058263 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f433800c460 con 0x7f43501599e0 2023-10-08T00:55:57.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.058+0000 7f434cff9700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.058+0000 7f434cff9700 1 -- 172.21.15.119:0/3242058263 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4338084750 con 0x7f43501599e0 2023-10-08T00:55:57.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.058+0000 7f434ffff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.059+0000 7f434ffff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4330066f30 0x7f43300693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f43500c73c0 tx=0x7f4340017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.061+0000 7f434cff9700 1 -- 172.21.15.119:0/3242058263 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f43500c4930 con 0x7f43501599e0 2023-10-08T00:55:57.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.247+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43500c4930 con 0x7f4330066f30 2023-10-08T00:55:57.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.255+0000 7f434cff9700 1 -- 172.21.15.119:0/3242058263 <== mgr.4101 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264483 (secure 0 0 0) 0x7f43500c4930 con 0x7f4330066f30 2023-10-08T00:55:57.259 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:57.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.258+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4330066f30 msgr2=0x7f43300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4330066f30 0x7f43300693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f43500c73c0 tx=0x7f4340017000 comp rx=0 tx=0).stop 2023-10-08T00:55:57.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 msgr2=0x7f435014cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f435014cd40 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f433800e900 tx=0x7f43380091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 shutdown_connections 2023-10-08T00:55:57.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f435006a2b0 0x7f43500c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4330066f30 0x7f43300693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4350159db0 0x7f43500c5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 --2- 172.21.15.119:0/3242058263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43501599e0 0x7f435014cd40 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 >> 172.21.15.119:0/3242058263 conn(0x7f43500c9860 msgr2=0x7f43500c81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:57.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 shutdown_connections 2023-10-08T00:55:57.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.259+0000 7f43563ff700 1 -- 172.21.15.119:0/3242058263 wait complete. 2023-10-08T00:55:57.263 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T00:55:57.277 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T00:55:55.859674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381128,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593592,"statfs":{"total":773094113280,"available":770655838208,"internally_reserved":0,"allocated":2891776,"data_stored":1750580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.368133"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524928+0000","last_change":"2023-10-08T00:55:47.519961+0000","last_active":"2023-10-08T00:55:47.524928+0000","last_peered":"2023-10-08T00:55:47.524928+0000","last_clean":"2023-10-08T00:55:47.524928+0000","last_became_active":"2023-10-08T00:55:47.519667+0000","last_became_peered":"2023-10-08T00:55:47.519667+0000","last_unstale":"2023-10-08T00:55:47.524928+0000","last_undegraded":"2023-10-08T00:55:47.524928+0000","last_fullsized":"2023-10-08T00:55:47.524928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:20.644250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520418+0000","last_change":"2023-10-08T00:55:47.516100+0000","last_active":"2023-10-08T00:55:47.520418+0000","last_peered":"2023-10-08T00:55:47.520418+0000","last_clean":"2023-10-08T00:55:47.520418+0000","last_became_active":"2023-10-08T00:55:47.515088+0000","last_became_peered":"2023-10-08T00:55:47.515088+0000","last_unstale":"2023-10-08T00:55:47.520418+0000","last_undegraded":"2023-10-08T00:55:47.520418+0000","last_fullsized":"2023-10-08T00:55:47.520418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523625+0000","last_change":"2023-10-08T00:55:47.514661+0000","last_active":"2023-10-08T00:55:47.523625+0000","last_peered":"2023-10-08T00:55:47.523625+0000","last_clean":"2023-10-08T00:55:47.523625+0000","last_became_active":"2023-10-08T00:55:47.512991+0000","last_became_peered":"2023-10-08T00:55:47.512991+0000","last_unstale":"2023-10-08T00:55:47.523625+0000","last_undegraded":"2023-10-08T00:55:47.523625+0000","last_fullsized":"2023-10-08T00:55:47.523625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:25.085209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525503+0000","last_change":"2023-10-08T00:55:47.520755+0000","last_active":"2023-10-08T00:55:47.525503+0000","last_peered":"2023-10-08T00:55:47.525503+0000","last_clean":"2023-10-08T00:55:47.525503+0000","last_became_active":"2023-10-08T00:55:47.519925+0000","last_became_peered":"2023-10-08T00:55:47.519925+0000","last_unstale":"2023-10-08T00:55:47.525503+0000","last_undegraded":"2023-10-08T00:55:47.525503+0000","last_fullsized":"2023-10-08T00:55:47.525503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.513000+0000","last_change":"2023-10-08T00:55:47.510024+0000","last_active":"2023-10-08T00:55:47.513000+0000","last_peered":"2023-10-08T00:55:47.513000+0000","last_clean":"2023-10-08T00:55:47.513000+0000","last_became_active":"2023-10-08T00:55:47.509649+0000","last_became_peered":"2023-10-08T00:55:47.509649+0000","last_unstale":"2023-10-08T00:55:47.513000+0000","last_undegraded":"2023-10-08T00:55:47.513000+0000","last_fullsized":"2023-10-08T00:55:47.513000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525212+0000","last_change":"2023-10-08T00:55:47.514675+0000","last_active":"2023-10-08T00:55:47.525212+0000","last_peered":"2023-10-08T00:55:47.525212+0000","last_clean":"2023-10-08T00:55:47.525212+0000","last_became_active":"2023-10-08T00:55:47.514414+0000","last_became_peered":"2023-10-08T00:55:47.514414+0000","last_unstale":"2023-10-08T00:55:47.525212+0000","last_undegraded":"2023-10-08T00:55:47.525212+0000","last_fullsized":"2023-10-08T00:55:47.525212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:07.821288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524753+0000","last_change":"2023-10-08T00:55:47.516013+0000","last_active":"2023-10-08T00:55:47.524753+0000","last_peered":"2023-10-08T00:55:47.524753+0000","last_clean":"2023-10-08T00:55:47.524753+0000","last_became_active":"2023-10-08T00:55:47.515695+0000","last_became_peered":"2023-10-08T00:55:47.515695+0000","last_unstale":"2023-10-08T00:55:47.524753+0000","last_undegraded":"2023-10-08T00:55:47.524753+0000","last_fullsized":"2023-10-08T00:55:47.524753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:44.607367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.527135+0000","last_change":"2023-10-08T00:55:47.525615+0000","last_active":"2023-10-08T00:55:47.527135+0000","last_peered":"2023-10-08T00:55:47.527135+0000","last_clean":"2023-10-08T00:55:47.527135+0000","last_became_active":"2023-10-08T00:55:47.525281+0000","last_became_peered":"2023-10-08T00:55:47.525281+0000","last_unstale":"2023-10-08T00:55:47.527135+0000","last_undegraded":"2023-10-08T00:55:47.527135+0000","last_fullsized":"2023-10-08T00:55:47.527135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522070+0000","last_change":"2023-10-08T00:55:47.514405+0000","last_active":"2023-10-08T00:55:47.522070+0000","last_peered":"2023-10-08T00:55:47.522070+0000","last_clean":"2023-10-08T00:55:47.522070+0000","last_became_active":"2023-10-08T00:55:47.513815+0000","last_became_peered":"2023-10-08T00:55:47.513815+0000","last_unstale":"2023-10-08T00:55:47.522070+0000","last_undegraded":"2023-10-08T00:55:47.522070+0000","last_fullsized":"2023-10-08T00:55:47.522070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:34.078412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524336+0000","last_change":"2023-10-08T00:55:47.518070+0000","last_active":"2023-10-08T00:55:47.524336+0000","last_peered":"2023-10-08T00:55:47.524336+0000","last_clean":"2023-10-08T00:55:47.524336+0000","last_became_active":"2023-10-08T00:55:47.517802+0000","last_became_peered":"2023-10-08T00:55:47.517802+0000","last_unstale":"2023-10-08T00:55:47.524336+0000","last_undegraded":"2023-10-08T00:55:47.524336+0000","last_fullsized":"2023-10-08T00:55:47.524336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518986+0000","last_change":"2023-10-08T00:55:47.517157+0000","last_active":"2023-10-08T00:55:47.518986+0000","last_peered":"2023-10-08T00:55:47.518986+0000","last_clean":"2023-10-08T00:55:47.518986+0000","last_became_active":"2023-10-08T00:55:47.516876+0000","last_became_peered":"2023-10-08T00:55:47.516876+0000","last_unstale":"2023-10-08T00:55:47.518986+0000","last_undegraded":"2023-10-08T00:55:47.518986+0000","last_fullsized":"2023-10-08T00:55:47.518986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533808+0000","last_change":"2023-10-08T00:55:47.520192+0000","last_active":"2023-10-08T00:55:47.533808+0000","last_peered":"2023-10-08T00:55:47.533808+0000","last_clean":"2023-10-08T00:55:47.533808+0000","last_became_active":"2023-10-08T00:55:47.519911+0000","last_became_peered":"2023-10-08T00:55:47.519911+0000","last_unstale":"2023-10-08T00:55:47.533808+0000","last_undegraded":"2023-10-08T00:55:47.533808+0000","last_fullsized":"2023-10-08T00:55:47.533808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:44.817925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526111+0000","last_change":"2023-10-08T00:55:47.522124+0000","last_active":"2023-10-08T00:55:47.526111+0000","last_peered":"2023-10-08T00:55:47.526111+0000","last_clean":"2023-10-08T00:55:47.526111+0000","last_became_active":"2023-10-08T00:55:47.521242+0000","last_became_peered":"2023-10-08T00:55:47.521242+0000","last_unstale":"2023-10-08T00:55:47.526111+0000","last_undegraded":"2023-10-08T00:55:47.526111+0000","last_fullsized":"2023-10-08T00:55:47.526111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520534+0000","last_change":"2023-10-08T00:55:47.508213+0000","last_active":"2023-10-08T00:55:47.520534+0000","last_peered":"2023-10-08T00:55:47.520534+0000","last_clean":"2023-10-08T00:55:47.520534+0000","last_became_active":"2023-10-08T00:55:47.507914+0000","last_became_peered":"2023-10-08T00:55:47.507914+0000","last_unstale":"2023-10-08T00:55:47.520534+0000","last_undegraded":"2023-10-08T00:55:47.520534+0000","last_fullsized":"2023-10-08T00:55:47.520534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.510943+0000","last_change":"2023-10-08T00:55:47.504306+0000","last_active":"2023-10-08T00:55:47.510943+0000","last_peered":"2023-10-08T00:55:47.510943+0000","last_clean":"2023-10-08T00:55:47.510943+0000","last_became_active":"2023-10-08T00:55:47.504037+0000","last_became_peered":"2023-10-08T00:55:47.504037+0000","last_unstale":"2023-10-08T00:55:47.510943+0000","last_undegraded":"2023-10-08T00:55:47.510943+0000","last_fullsized":"2023-10-08T00:55:47.510943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530847+0000","last_change":"2023-10-08T00:55:47.524003+0000","last_active":"2023-10-08T00:55:47.530847+0000","last_peered":"2023-10-08T00:55:47.530847+0000","last_clean":"2023-10-08T00:55:47.530847+0000","last_became_active":"2023-10-08T00:55:47.521541+0000","last_became_peered":"2023-10-08T00:55:47.521541+0000","last_unstale":"2023-10-08T00:55:47.530847+0000","last_undegraded":"2023-10-08T00:55:47.530847+0000","last_fullsized":"2023-10-08T00:55:47.530847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:44.607367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.511787+0000","last_change":"2023-10-08T00:55:47.509224+0000","last_active":"2023-10-08T00:55:47.511787+0000","last_peered":"2023-10-08T00:55:47.511787+0000","last_clean":"2023-10-08T00:55:47.511787+0000","last_became_active":"2023-10-08T00:55:47.508714+0000","last_became_peered":"2023-10-08T00:55:47.508714+0000","last_unstale":"2023-10-08T00:55:47.511787+0000","last_undegraded":"2023-10-08T00:55:47.511787+0000","last_fullsized":"2023-10-08T00:55:47.511787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525712+0000","last_change":"2023-10-08T00:55:47.519330+0000","last_active":"2023-10-08T00:55:47.525712+0000","last_peered":"2023-10-08T00:55:47.525712+0000","last_clean":"2023-10-08T00:55:47.525712+0000","last_became_active":"2023-10-08T00:55:47.519070+0000","last_became_peered":"2023-10-08T00:55:47.519070+0000","last_unstale":"2023-10-08T00:55:47.525712+0000","last_undegraded":"2023-10-08T00:55:47.525712+0000","last_fullsized":"2023-10-08T00:55:47.525712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:03.776133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.514578+0000","last_change":"2023-10-08T00:55:47.509945+0000","last_active":"2023-10-08T00:55:47.514578+0000","last_peered":"2023-10-08T00:55:47.514578+0000","last_clean":"2023-10-08T00:55:47.514578+0000","last_became_active":"2023-10-08T00:55:47.509596+0000","last_became_peered":"2023-10-08T00:55:47.509596+0000","last_unstale":"2023-10-08T00:55:47.514578+0000","last_undegraded":"2023-10-08T00:55:47.514578+0000","last_fullsized":"2023-10-08T00:55:47.514578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:07.510837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525580+0000","last_change":"2023-10-08T00:55:46.514258+0000","last_active":"2023-10-08T00:55:47.525580+0000","last_peered":"2023-10-08T00:55:47.525580+0000","last_clean":"2023-10-08T00:55:47.525580+0000","last_became_active":"2023-10-08T00:55:46.514002+0000","last_became_peered":"2023-10-08T00:55:46.514002+0000","last_unstale":"2023-10-08T00:55:47.525580+0000","last_undegraded":"2023-10-08T00:55:47.525580+0000","last_fullsized":"2023-10-08T00:55:47.525580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520841+0000","last_change":"2023-10-08T00:55:47.519479+0000","last_active":"2023-10-08T00:55:47.520841+0000","last_peered":"2023-10-08T00:55:47.520841+0000","last_clean":"2023-10-08T00:55:47.520841+0000","last_became_active":"2023-10-08T00:55:47.519179+0000","last_became_peered":"2023-10-08T00:55:47.519179+0000","last_unstale":"2023-10-08T00:55:47.520841+0000","last_undegraded":"2023-10-08T00:55:47.520841+0000","last_fullsized":"2023-10-08T00:55:47.520841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523749+0000","last_change":"2023-10-08T00:55:46.507226+0000","last_active":"2023-10-08T00:55:47.523749+0000","last_peered":"2023-10-08T00:55:47.523749+0000","last_clean":"2023-10-08T00:55:47.523749+0000","last_became_active":"2023-10-08T00:55:46.506878+0000","last_became_peered":"2023-10-08T00:55:46.506878+0000","last_unstale":"2023-10-08T00:55:47.523749+0000","last_undegraded":"2023-10-08T00:55:47.523749+0000","last_fullsized":"2023-10-08T00:55:47.523749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523694+0000","last_change":"2023-10-08T00:55:47.514717+0000","last_active":"2023-10-08T00:55:47.523694+0000","last_peered":"2023-10-08T00:55:47.523694+0000","last_clean":"2023-10-08T00:55:47.523694+0000","last_became_active":"2023-10-08T00:55:47.514339+0000","last_became_peered":"2023-10-08T00:55:47.514339+0000","last_unstale":"2023-10-08T00:55:47.523694+0000","last_undegraded":"2023-10-08T00:55:47.523694+0000","last_fullsized":"2023-10-08T00:55:47.523694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522336+0000","last_change":"2023-10-08T00:55:46.517327+0000","last_active":"2023-10-08T00:55:47.522336+0000","last_peered":"2023-10-08T00:55:47.522336+0000","last_clean":"2023-10-08T00:55:47.522336+0000","last_became_active":"2023-10-08T00:55:46.516908+0000","last_became_peered":"2023-10-08T00:55:46.516908+0000","last_unstale":"2023-10-08T00:55:47.522336+0000","last_undegraded":"2023-10-08T00:55:47.522336+0000","last_fullsized":"2023-10-08T00:55:47.522336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525482+0000","last_change":"2023-10-08T00:55:47.520563+0000","last_active":"2023-10-08T00:55:47.525482+0000","last_peered":"2023-10-08T00:55:47.525482+0000","last_clean":"2023-10-08T00:55:47.525482+0000","last_became_active":"2023-10-08T00:55:47.519545+0000","last_became_peered":"2023-10-08T00:55:47.519545+0000","last_unstale":"2023-10-08T00:55:47.525482+0000","last_undegraded":"2023-10-08T00:55:47.525482+0000","last_fullsized":"2023-10-08T00:55:47.525482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.528020+0000","last_change":"2023-10-08T00:55:46.521160+0000","last_active":"2023-10-08T00:55:47.528020+0000","last_peered":"2023-10-08T00:55:47.528020+0000","last_clean":"2023-10-08T00:55:47.528020+0000","last_became_active":"2023-10-08T00:55:46.520820+0000","last_became_peered":"2023-10-08T00:55:46.520820+0000","last_unstale":"2023-10-08T00:55:47.528020+0000","last_undegraded":"2023-10-08T00:55:47.528020+0000","last_fullsized":"2023-10-08T00:55:47.528020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:24.078567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533581+0000","last_change":"2023-10-08T00:55:47.526348+0000","last_active":"2023-10-08T00:55:47.533581+0000","last_peered":"2023-10-08T00:55:47.533581+0000","last_clean":"2023-10-08T00:55:47.533581+0000","last_became_active":"2023-10-08T00:55:47.526167+0000","last_became_peered":"2023-10-08T00:55:47.526167+0000","last_unstale":"2023-10-08T00:55:47.533581+0000","last_undegraded":"2023-10-08T00:55:47.533581+0000","last_fullsized":"2023-10-08T00:55:47.533581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522448+0000","last_change":"2023-10-08T00:55:46.505995+0000","last_active":"2023-10-08T00:55:47.522448+0000","last_peered":"2023-10-08T00:55:47.522448+0000","last_clean":"2023-10-08T00:55:47.522448+0000","last_became_active":"2023-10-08T00:55:46.505673+0000","last_became_peered":"2023-10-08T00:55:46.505673+0000","last_unstale":"2023-10-08T00:55:47.522448+0000","last_undegraded":"2023-10-08T00:55:47.522448+0000","last_fullsized":"2023-10-08T00:55:47.522448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533980+0000","last_change":"2023-10-08T00:55:47.525941+0000","last_active":"2023-10-08T00:55:47.533980+0000","last_peered":"2023-10-08T00:55:47.533980+0000","last_clean":"2023-10-08T00:55:47.533980+0000","last_became_active":"2023-10-08T00:55:47.524635+0000","last_became_peered":"2023-10-08T00:55:47.524635+0000","last_unstale":"2023-10-08T00:55:47.533980+0000","last_undegraded":"2023-10-08T00:55:47.533980+0000","last_fullsized":"2023-10-08T00:55:47.533980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:09.261793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522850+0000","last_change":"2023-10-08T00:55:46.506810+0000","last_active":"2023-10-08T00:55:47.522850+0000","last_peered":"2023-10-08T00:55:47.522850+0000","last_clean":"2023-10-08T00:55:47.522850+0000","last_became_active":"2023-10-08T00:55:46.506504+0000","last_became_peered":"2023-10-08T00:55:46.506504+0000","last_unstale":"2023-10-08T00:55:47.522850+0000","last_undegraded":"2023-10-08T00:55:47.522850+0000","last_fullsized":"2023-10-08T00:55:47.522850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524500+0000","last_change":"2023-10-08T00:55:47.516584+0000","last_active":"2023-10-08T00:55:47.524500+0000","last_peered":"2023-10-08T00:55:47.524500+0000","last_clean":"2023-10-08T00:55:47.524500+0000","last_became_active":"2023-10-08T00:55:47.516222+0000","last_became_peered":"2023-10-08T00:55:47.516222+0000","last_unstale":"2023-10-08T00:55:47.524500+0000","last_undegraded":"2023-10-08T00:55:47.524500+0000","last_fullsized":"2023-10-08T00:55:47.524500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.506266+0000","last_change":"2023-10-08T00:55:46.499483+0000","last_active":"2023-10-08T00:55:47.506266+0000","last_peered":"2023-10-08T00:55:47.506266+0000","last_clean":"2023-10-08T00:55:47.506266+0000","last_became_active":"2023-10-08T00:55:46.499153+0000","last_became_peered":"2023-10-08T00:55:46.499153+0000","last_unstale":"2023-10-08T00:55:47.506266+0000","last_undegraded":"2023-10-08T00:55:47.506266+0000","last_fullsized":"2023-10-08T00:55:47.506266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525986+0000","last_change":"2023-10-08T00:55:47.521897+0000","last_active":"2023-10-08T00:55:47.525986+0000","last_peered":"2023-10-08T00:55:47.525986+0000","last_clean":"2023-10-08T00:55:47.525986+0000","last_became_active":"2023-10-08T00:55:47.521630+0000","last_became_peered":"2023-10-08T00:55:47.521630+0000","last_unstale":"2023-10-08T00:55:47.525986+0000","last_undegraded":"2023-10-08T00:55:47.525986+0000","last_fullsized":"2023-10-08T00:55:47.525986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.502490+0000","last_change":"2023-10-08T00:55:46.520040+0000","last_active":"2023-10-08T00:55:47.502490+0000","last_peered":"2023-10-08T00:55:47.502490+0000","last_clean":"2023-10-08T00:55:47.502490+0000","last_became_active":"2023-10-08T00:55:46.519780+0000","last_became_peered":"2023-10-08T00:55:46.519780+0000","last_unstale":"2023-10-08T00:55:47.502490+0000","last_undegraded":"2023-10-08T00:55:47.502490+0000","last_fullsized":"2023-10-08T00:55:47.502490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523199+0000","last_change":"2023-10-08T00:55:47.518074+0000","last_active":"2023-10-08T00:55:47.523199+0000","last_peered":"2023-10-08T00:55:47.523199+0000","last_clean":"2023-10-08T00:55:47.523199+0000","last_became_active":"2023-10-08T00:55:47.517822+0000","last_became_peered":"2023-10-08T00:55:47.517822+0000","last_unstale":"2023-10-08T00:55:47.523199+0000","last_undegraded":"2023-10-08T00:55:47.523199+0000","last_fullsized":"2023-10-08T00:55:47.523199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507109+0000","last_change":"2023-10-08T00:55:46.520607+0000","last_active":"2023-10-08T00:55:47.507109+0000","last_peered":"2023-10-08T00:55:47.507109+0000","last_clean":"2023-10-08T00:55:47.507109+0000","last_became_active":"2023-10-08T00:55:46.520340+0000","last_became_peered":"2023-10-08T00:55:46.520340+0000","last_unstale":"2023-10-08T00:55:47.507109+0000","last_undegraded":"2023-10-08T00:55:47.507109+0000","last_fullsized":"2023-10-08T00:55:47.507109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.531004+0000","last_change":"2023-10-08T00:55:47.524779+0000","last_active":"2023-10-08T00:55:47.531004+0000","last_peered":"2023-10-08T00:55:47.531004+0000","last_clean":"2023-10-08T00:55:47.531004+0000","last_became_active":"2023-10-08T00:55:47.524520+0000","last_became_peered":"2023-10-08T00:55:47.524520+0000","last_unstale":"2023-10-08T00:55:47.531004+0000","last_undegraded":"2023-10-08T00:55:47.531004+0000","last_fullsized":"2023-10-08T00:55:47.531004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.505478+0000","last_change":"2023-10-08T00:55:46.520234+0000","last_active":"2023-10-08T00:55:47.505478+0000","last_peered":"2023-10-08T00:55:47.505478+0000","last_clean":"2023-10-08T00:55:47.505478+0000","last_became_active":"2023-10-08T00:55:46.519769+0000","last_became_peered":"2023-10-08T00:55:46.519769+0000","last_unstale":"2023-10-08T00:55:47.505478+0000","last_undegraded":"2023-10-08T00:55:47.505478+0000","last_fullsized":"2023-10-08T00:55:47.505478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523717+0000","last_change":"2023-10-08T00:55:47.514712+0000","last_active":"2023-10-08T00:55:47.523717+0000","last_peered":"2023-10-08T00:55:47.523717+0000","last_clean":"2023-10-08T00:55:47.523717+0000","last_became_active":"2023-10-08T00:55:47.514326+0000","last_became_peered":"2023-10-08T00:55:47.514326+0000","last_unstale":"2023-10-08T00:55:47.523717+0000","last_undegraded":"2023-10-08T00:55:47.523717+0000","last_fullsized":"2023-10-08T00:55:47.523717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.513999+0000","last_change":"2023-10-08T00:55:46.520938+0000","last_active":"2023-10-08T00:55:47.513999+0000","last_peered":"2023-10-08T00:55:47.513999+0000","last_clean":"2023-10-08T00:55:47.513999+0000","last_became_active":"2023-10-08T00:55:46.520707+0000","last_became_peered":"2023-10-08T00:55:46.520707+0000","last_unstale":"2023-10-08T00:55:47.513999+0000","last_undegraded":"2023-10-08T00:55:47.513999+0000","last_fullsized":"2023-10-08T00:55:47.513999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:43.811282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.534039+0000","last_change":"2023-10-08T00:55:47.527280+0000","last_active":"2023-10-08T00:55:47.534039+0000","last_peered":"2023-10-08T00:55:47.534039+0000","last_clean":"2023-10-08T00:55:47.534039+0000","last_became_active":"2023-10-08T00:55:47.526962+0000","last_became_peered":"2023-10-08T00:55:47.526962+0000","last_unstale":"2023-10-08T00:55:47.534039+0000","last_undegraded":"2023-10-08T00:55:47.534039+0000","last_fullsized":"2023-10-08T00:55:47.534039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:03.776133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.529949+0000","last_change":"2023-10-08T00:55:46.496099+0000","last_active":"2023-10-08T00:55:47.529949+0000","last_peered":"2023-10-08T00:55:47.529949+0000","last_clean":"2023-10-08T00:55:47.529949+0000","last_became_active":"2023-10-08T00:55:46.495752+0000","last_became_peered":"2023-10-08T00:55:46.495752+0000","last_unstale":"2023-10-08T00:55:47.529949+0000","last_undegraded":"2023-10-08T00:55:47.529949+0000","last_fullsized":"2023-10-08T00:55:47.529949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524959+0000","last_change":"2023-10-08T00:55:47.521795+0000","last_active":"2023-10-08T00:55:47.524959+0000","last_peered":"2023-10-08T00:55:47.524959+0000","last_clean":"2023-10-08T00:55:47.524959+0000","last_became_active":"2023-10-08T00:55:47.521477+0000","last_became_peered":"2023-10-08T00:55:47.521477+0000","last_unstale":"2023-10-08T00:55:47.524959+0000","last_undegraded":"2023-10-08T00:55:47.524959+0000","last_fullsized":"2023-10-08T00:55:47.524959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:02.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520109+0000","last_change":"2023-10-08T00:55:46.509869+0000","last_active":"2023-10-08T00:55:47.520109+0000","last_peered":"2023-10-08T00:55:47.520109+0000","last_clean":"2023-10-08T00:55:47.520109+0000","last_became_active":"2023-10-08T00:55:46.509581+0000","last_became_peered":"2023-10-08T00:55:46.509581+0000","last_unstale":"2023-10-08T00:55:47.520109+0000","last_undegraded":"2023-10-08T00:55:47.520109+0000","last_fullsized":"2023-10-08T00:55:47.520109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.521324+0000","last_change":"2023-10-08T00:55:47.512931+0000","last_active":"2023-10-08T00:55:47.521324+0000","last_peered":"2023-10-08T00:55:47.521324+0000","last_clean":"2023-10-08T00:55:47.521324+0000","last_became_active":"2023-10-08T00:55:47.512674+0000","last_became_peered":"2023-10-08T00:55:47.512674+0000","last_unstale":"2023-10-08T00:55:47.521324+0000","last_undegraded":"2023-10-08T00:55:47.521324+0000","last_fullsized":"2023-10-08T00:55:47.521324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.501686+0000","last_change":"2023-10-08T00:55:46.519403+0000","last_active":"2023-10-08T00:55:47.501686+0000","last_peered":"2023-10-08T00:55:47.501686+0000","last_clean":"2023-10-08T00:55:47.501686+0000","last_became_active":"2023-10-08T00:55:46.516356+0000","last_became_peered":"2023-10-08T00:55:46.516356+0000","last_unstale":"2023-10-08T00:55:47.501686+0000","last_undegraded":"2023-10-08T00:55:47.501686+0000","last_fullsized":"2023-10-08T00:55:47.501686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507923+0000","last_change":"2023-10-08T00:55:47.504128+0000","last_active":"2023-10-08T00:55:47.507923+0000","last_peered":"2023-10-08T00:55:47.507923+0000","last_clean":"2023-10-08T00:55:47.507923+0000","last_became_active":"2023-10-08T00:55:47.503735+0000","last_became_peered":"2023-10-08T00:55:47.503735+0000","last_unstale":"2023-10-08T00:55:47.507923+0000","last_undegraded":"2023-10-08T00:55:47.507923+0000","last_fullsized":"2023-10-08T00:55:47.507923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.509141+0000","last_change":"2023-10-08T00:55:46.506608+0000","last_active":"2023-10-08T00:55:47.509141+0000","last_peered":"2023-10-08T00:55:47.509141+0000","last_clean":"2023-10-08T00:55:47.509141+0000","last_became_active":"2023-10-08T00:55:46.506233+0000","last_became_peered":"2023-10-08T00:55:46.506233+0000","last_unstale":"2023-10-08T00:55:47.509141+0000","last_undegraded":"2023-10-08T00:55:47.509141+0000","last_fullsized":"2023-10-08T00:55:47.509141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526384+0000","last_change":"2023-10-08T00:55:47.522727+0000","last_active":"2023-10-08T00:55:47.526384+0000","last_peered":"2023-10-08T00:55:47.526384+0000","last_clean":"2023-10-08T00:55:47.526384+0000","last_became_active":"2023-10-08T00:55:47.522019+0000","last_became_peered":"2023-10-08T00:55:47.522019+0000","last_unstale":"2023-10-08T00:55:47.526384+0000","last_undegraded":"2023-10-08T00:55:47.526384+0000","last_fullsized":"2023-10-08T00:55:47.526384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520506+0000","last_change":"2023-10-08T00:55:46.506201+0000","last_active":"2023-10-08T00:55:47.520506+0000","last_peered":"2023-10-08T00:55:47.520506+0000","last_clean":"2023-10-08T00:55:47.520506+0000","last_became_active":"2023-10-08T00:55:46.505555+0000","last_became_peered":"2023-10-08T00:55:46.505555+0000","last_unstale":"2023-10-08T00:55:47.520506+0000","last_undegraded":"2023-10-08T00:55:47.520506+0000","last_fullsized":"2023-10-08T00:55:47.520506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533925+0000","last_change":"2023-10-08T00:55:47.528839+0000","last_active":"2023-10-08T00:55:47.533925+0000","last_peered":"2023-10-08T00:55:47.533925+0000","last_clean":"2023-10-08T00:55:47.533925+0000","last_became_active":"2023-10-08T00:55:47.528573+0000","last_became_peered":"2023-10-08T00:55:47.528573+0000","last_unstale":"2023-10-08T00:55:47.533925+0000","last_undegraded":"2023-10-08T00:55:47.533925+0000","last_fullsized":"2023-10-08T00:55:47.533925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:57:57.391083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.517881+0000","last_change":"2023-10-08T00:55:46.506125+0000","last_active":"2023-10-08T00:55:47.517881+0000","last_peered":"2023-10-08T00:55:47.517881+0000","last_clean":"2023-10-08T00:55:47.517881+0000","last_became_active":"2023-10-08T00:55:46.505815+0000","last_became_peered":"2023-10-08T00:55:46.505815+0000","last_unstale":"2023-10-08T00:55:47.517881+0000","last_undegraded":"2023-10-08T00:55:47.517881+0000","last_fullsized":"2023-10-08T00:55:47.517881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533963+0000","last_change":"2023-10-08T00:55:47.529148+0000","last_active":"2023-10-08T00:55:47.533963+0000","last_peered":"2023-10-08T00:55:47.533963+0000","last_clean":"2023-10-08T00:55:47.533963+0000","last_became_active":"2023-10-08T00:55:47.528897+0000","last_became_peered":"2023-10-08T00:55:47.528897+0000","last_unstale":"2023-10-08T00:55:47.533963+0000","last_undegraded":"2023-10-08T00:55:47.533963+0000","last_fullsized":"2023-10-08T00:55:47.533963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492782+0000","last_change":"2023-10-08T00:55:46.507942+0000","last_active":"2023-10-08T00:55:47.492782+0000","last_peered":"2023-10-08T00:55:47.492782+0000","last_clean":"2023-10-08T00:55:47.492782+0000","last_became_active":"2023-10-08T00:55:46.507685+0000","last_became_peered":"2023-10-08T00:55:46.507685+0000","last_unstale":"2023-10-08T00:55:47.492782+0000","last_undegraded":"2023-10-08T00:55:47.492782+0000","last_fullsized":"2023-10-08T00:55:47.492782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530557+0000","last_change":"2023-10-08T00:55:47.521486+0000","last_active":"2023-10-08T00:55:47.530557+0000","last_peered":"2023-10-08T00:55:47.530557+0000","last_clean":"2023-10-08T00:55:47.530557+0000","last_became_active":"2023-10-08T00:55:47.521230+0000","last_became_peered":"2023-10-08T00:55:47.521230+0000","last_unstale":"2023-10-08T00:55:47.530557+0000","last_undegraded":"2023-10-08T00:55:47.530557+0000","last_fullsized":"2023-10-08T00:55:47.530557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:34.078412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.501573+0000","last_change":"2023-10-08T00:55:46.519867+0000","last_active":"2023-10-08T00:55:47.501573+0000","last_peered":"2023-10-08T00:55:47.501573+0000","last_clean":"2023-10-08T00:55:47.501573+0000","last_became_active":"2023-10-08T00:55:46.519268+0000","last_became_peered":"2023-10-08T00:55:46.519268+0000","last_unstale":"2023-10-08T00:55:47.501573+0000","last_undegraded":"2023-10-08T00:55:47.501573+0000","last_fullsized":"2023-10-08T00:55:47.501573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530667+0000","last_change":"2023-10-08T00:55:47.517018+0000","last_active":"2023-10-08T00:55:47.530667+0000","last_peered":"2023-10-08T00:55:47.530667+0000","last_clean":"2023-10-08T00:55:47.530667+0000","last_became_active":"2023-10-08T00:55:47.516713+0000","last_became_peered":"2023-10-08T00:55:47.516713+0000","last_unstale":"2023-10-08T00:55:47.530667+0000","last_undegraded":"2023-10-08T00:55:47.530667+0000","last_fullsized":"2023-10-08T00:55:47.530667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:40.798392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507081+0000","last_change":"2023-10-08T00:55:47.500984+0000","last_active":"2023-10-08T00:55:47.507081+0000","last_peered":"2023-10-08T00:55:47.507081+0000","last_clean":"2023-10-08T00:55:47.507081+0000","last_became_active":"2023-10-08T00:55:47.499299+0000","last_became_peered":"2023-10-08T00:55:47.499299+0000","last_unstale":"2023-10-08T00:55:47.507081+0000","last_undegraded":"2023-10-08T00:55:47.507081+0000","last_fullsized":"2023-10-08T00:55:47.507081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.494532+0000","last_change":"2023-10-08T00:55:46.503917+0000","last_active":"2023-10-08T00:55:47.494532+0000","last_peered":"2023-10-08T00:55:47.494532+0000","last_clean":"2023-10-08T00:55:47.494532+0000","last_became_active":"2023-10-08T00:55:46.503620+0000","last_became_peered":"2023-10-08T00:55:46.503620+0000","last_unstale":"2023-10-08T00:55:47.494532+0000","last_undegraded":"2023-10-08T00:55:47.494532+0000","last_fullsized":"2023-10-08T00:55:47.494532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.527104+0000","last_change":"2023-10-08T00:55:47.525523+0000","last_active":"2023-10-08T00:55:47.527104+0000","last_peered":"2023-10-08T00:55:47.527104+0000","last_clean":"2023-10-08T00:55:47.527104+0000","last_became_active":"2023-10-08T00:55:47.525238+0000","last_became_peered":"2023-10-08T00:55:47.525238+0000","last_unstale":"2023-10-08T00:55:47.527104+0000","last_undegraded":"2023-10-08T00:55:47.527104+0000","last_fullsized":"2023-10-08T00:55:47.527104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.498288+0000","last_change":"2023-10-08T00:55:46.516320+0000","last_active":"2023-10-08T00:55:47.498288+0000","last_peered":"2023-10-08T00:55:47.498288+0000","last_clean":"2023-10-08T00:55:47.498288+0000","last_became_active":"2023-10-08T00:55:46.516011+0000","last_became_peered":"2023-10-08T00:55:46.516011+0000","last_unstale":"2023-10-08T00:55:47.498288+0000","last_undegraded":"2023-10-08T00:55:47.498288+0000","last_fullsized":"2023-10-08T00:55:47.498288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493566+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T00:55:47.493566+0000","last_peered":"2023-10-08T00:55:47.493566+0000","last_clean":"2023-10-08T00:55:47.493566+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T00:55:47.493566+0000","last_undegraded":"2023-10-08T00:55:47.493566+0000","last_fullsized":"2023-10-08T00:55:47.493566+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524595+0000","last_change":"2023-10-08T00:55:47.518197+0000","last_active":"2023-10-08T00:55:47.524595+0000","last_peered":"2023-10-08T00:55:47.524595+0000","last_clean":"2023-10-08T00:55:47.524595+0000","last_became_active":"2023-10-08T00:55:47.517929+0000","last_became_peered":"2023-10-08T00:55:47.517929+0000","last_unstale":"2023-10-08T00:55:47.524595+0000","last_undegraded":"2023-10-08T00:55:47.524595+0000","last_fullsized":"2023-10-08T00:55:47.524595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:55.386688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493737+0000","last_change":"2023-10-08T00:55:46.513277+0000","last_active":"2023-10-08T00:55:47.493737+0000","last_peered":"2023-10-08T00:55:47.493737+0000","last_clean":"2023-10-08T00:55:47.493737+0000","last_became_active":"2023-10-08T00:55:46.513027+0000","last_became_peered":"2023-10-08T00:55:46.513027+0000","last_unstale":"2023-10-08T00:55:47.493737+0000","last_undegraded":"2023-10-08T00:55:47.493737+0000","last_fullsized":"2023-10-08T00:55:47.493737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:43.811282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.506236+0000","last_change":"2023-10-08T00:55:46.518643+0000","last_active":"2023-10-08T00:55:47.506236+0000","last_peered":"2023-10-08T00:55:47.506236+0000","last_clean":"2023-10-08T00:55:47.506236+0000","last_became_active":"2023-10-08T00:55:46.518305+0000","last_became_peered":"2023-10-08T00:55:46.518305+0000","last_unstale":"2023-10-08T00:55:47.506236+0000","last_undegraded":"2023-10-08T00:55:47.506236+0000","last_fullsized":"2023-10-08T00:55:47.506236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.499384+0000","last_change":"2023-10-08T00:55:47.498680+0000","last_active":"2023-10-08T00:55:47.499384+0000","last_peered":"2023-10-08T00:55:47.499384+0000","last_clean":"2023-10-08T00:55:47.499384+0000","last_became_active":"2023-10-08T00:55:47.498370+0000","last_became_peered":"2023-10-08T00:55:47.498370+0000","last_unstale":"2023-10-08T00:55:47.499384+0000","last_undegraded":"2023-10-08T00:55:47.499384+0000","last_fullsized":"2023-10-08T00:55:47.499384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T00:55:49.395102+0000","last_change":"2023-10-08T00:55:46.508309+0000","last_active":"2023-10-08T00:55:49.395102+0000","last_peered":"2023-10-08T00:55:49.395102+0000","last_clean":"2023-10-08T00:55:49.395102+0000","last_became_active":"2023-10-08T00:55:46.507338+0000","last_became_peered":"2023-10-08T00:55:46.507338+0000","last_unstale":"2023-10-08T00:55:49.395102+0000","last_undegraded":"2023-10-08T00:55:49.395102+0000","last_fullsized":"2023-10-08T00:55:49.395102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518720+0000","last_change":"2023-10-08T00:55:47.516628+0000","last_active":"2023-10-08T00:55:47.518720+0000","last_peered":"2023-10-08T00:55:47.518720+0000","last_clean":"2023-10-08T00:55:47.518720+0000","last_became_active":"2023-10-08T00:55:47.516317+0000","last_became_peered":"2023-10-08T00:55:47.516317+0000","last_unstale":"2023-10-08T00:55:47.518720+0000","last_undegraded":"2023-10-08T00:55:47.518720+0000","last_fullsized":"2023-10-08T00:55:47.518720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:07.510837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526183+0000","last_change":"2023-10-08T00:55:46.511031+0000","last_active":"2023-10-08T00:55:47.526183+0000","last_peered":"2023-10-08T00:55:47.526183+0000","last_clean":"2023-10-08T00:55:47.526183+0000","last_became_active":"2023-10-08T00:55:46.510736+0000","last_became_peered":"2023-10-08T00:55:46.510736+0000","last_unstale":"2023-10-08T00:55:47.526183+0000","last_undegraded":"2023-10-08T00:55:47.526183+0000","last_fullsized":"2023-10-08T00:55:47.526183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520686+0000","last_change":"2023-10-08T00:55:47.513619+0000","last_active":"2023-10-08T00:55:47.520686+0000","last_peered":"2023-10-08T00:55:47.520686+0000","last_clean":"2023-10-08T00:55:47.520686+0000","last_became_active":"2023-10-08T00:55:47.513202+0000","last_became_peered":"2023-10-08T00:55:47.513202+0000","last_unstale":"2023-10-08T00:55:47.520686+0000","last_undegraded":"2023-10-08T00:55:47.520686+0000","last_fullsized":"2023-10-08T00:55:47.520686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493466+0000","last_change":"2023-10-08T00:55:46.517881+0000","last_active":"2023-10-08T00:55:47.493466+0000","last_peered":"2023-10-08T00:55:47.493466+0000","last_clean":"2023-10-08T00:55:47.493466+0000","last_became_active":"2023-10-08T00:55:46.517717+0000","last_became_peered":"2023-10-08T00:55:46.517717+0000","last_unstale":"2023-10-08T00:55:47.493466+0000","last_undegraded":"2023-10-08T00:55:47.493466+0000","last_fullsized":"2023-10-08T00:55:47.493466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:15.507854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518670+0000","last_change":"2023-10-08T00:55:47.516448+0000","last_active":"2023-10-08T00:55:47.518670+0000","last_peered":"2023-10-08T00:55:47.518670+0000","last_clean":"2023-10-08T00:55:47.518670+0000","last_became_active":"2023-10-08T00:55:47.516204+0000","last_became_peered":"2023-10-08T00:55:47.516204+0000","last_unstale":"2023-10-08T00:55:47.518670+0000","last_undegraded":"2023-10-08T00:55:47.518670+0000","last_fullsized":"2023-10-08T00:55:47.518670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:16.514497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492576+0000","last_change":"2023-10-08T00:55:46.513481+0000","last_active":"2023-10-08T00:55:47.492576+0000","last_peered":"2023-10-08T00:55:47.492576+0000","last_clean":"2023-10-08T00:55:47.492576+0000","last_became_active":"2023-10-08T00:55:46.513158+0000","last_became_peered":"2023-10-08T00:55:46.513158+0000","last_unstale":"2023-10-08T00:55:47.492576+0000","last_undegraded":"2023-10-08T00:55:47.492576+0000","last_fullsized":"2023-10-08T00:55:47.492576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524756+0000","last_change":"2023-10-08T00:55:47.520913+0000","last_active":"2023-10-08T00:55:47.524756+0000","last_peered":"2023-10-08T00:55:47.524756+0000","last_clean":"2023-10-08T00:55:47.524756+0000","last_became_active":"2023-10-08T00:55:47.520660+0000","last_became_peered":"2023-10-08T00:55:47.520660+0000","last_unstale":"2023-10-08T00:55:47.524756+0000","last_undegraded":"2023-10-08T00:55:47.524756+0000","last_fullsized":"2023-10-08T00:55:47.524756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492624+0000","last_change":"2023-10-08T00:55:46.508048+0000","last_active":"2023-10-08T00:55:47.492624+0000","last_peered":"2023-10-08T00:55:47.492624+0000","last_clean":"2023-10-08T00:55:47.492624+0000","last_became_active":"2023-10-08T00:55:46.507778+0000","last_became_peered":"2023-10-08T00:55:46.507778+0000","last_unstale":"2023-10-08T00:55:47.492624+0000","last_undegraded":"2023-10-08T00:55:47.492624+0000","last_fullsized":"2023-10-08T00:55:47.492624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526130+0000","last_change":"2023-10-08T00:55:47.524176+0000","last_active":"2023-10-08T00:55:47.526130+0000","last_peered":"2023-10-08T00:55:47.526130+0000","last_clean":"2023-10-08T00:55:47.526130+0000","last_became_active":"2023-10-08T00:55:47.523813+0000","last_became_peered":"2023-10-08T00:55:47.523813+0000","last_unstale":"2023-10-08T00:55:47.526130+0000","last_undegraded":"2023-10-08T00:55:47.526130+0000","last_fullsized":"2023-10-08T00:55:47.526130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.534398+0000","last_change":"2023-10-08T00:55:46.514845+0000","last_active":"2023-10-08T00:55:47.534398+0000","last_peered":"2023-10-08T00:55:47.534398+0000","last_clean":"2023-10-08T00:55:47.534398+0000","last_became_active":"2023-10-08T00:55:46.514581+0000","last_became_peered":"2023-10-08T00:55:46.514581+0000","last_unstale":"2023-10-08T00:55:47.534398+0000","last_undegraded":"2023-10-08T00:55:47.534398+0000","last_fullsized":"2023-10-08T00:55:47.534398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:02.769491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526325+0000","last_change":"2023-10-08T00:55:47.522621+0000","last_active":"2023-10-08T00:55:47.526325+0000","last_peered":"2023-10-08T00:55:47.526325+0000","last_clean":"2023-10-08T00:55:47.526325+0000","last_became_active":"2023-10-08T00:55:47.522352+0000","last_became_peered":"2023-10-08T00:55:47.522352+0000","last_unstale":"2023-10-08T00:55:47.526325+0000","last_undegraded":"2023-10-08T00:55:47.526325+0000","last_fullsized":"2023-10-08T00:55:47.526325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.494485+0000","last_change":"2023-10-08T00:55:46.520580+0000","last_active":"2023-10-08T00:55:47.494485+0000","last_peered":"2023-10-08T00:55:47.494485+0000","last_clean":"2023-10-08T00:55:47.494485+0000","last_became_active":"2023-10-08T00:55:46.520260+0000","last_became_peered":"2023-10-08T00:55:46.520260+0000","last_unstale":"2023-10-08T00:55:47.494485+0000","last_undegraded":"2023-10-08T00:55:47.494485+0000","last_fullsized":"2023-10-08T00:55:47.494485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524951+0000","last_change":"2023-10-08T00:55:47.519077+0000","last_active":"2023-10-08T00:55:47.524951+0000","last_peered":"2023-10-08T00:55:47.524951+0000","last_clean":"2023-10-08T00:55:47.524951+0000","last_became_active":"2023-10-08T00:55:47.518580+0000","last_became_peered":"2023-10-08T00:55:47.518580+0000","last_unstale":"2023-10-08T00:55:47.524951+0000","last_undegraded":"2023-10-08T00:55:47.524951+0000","last_fullsized":"2023-10-08T00:55:47.524951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:14.159608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526450+0000","last_change":"2023-10-08T00:55:46.514711+0000","last_active":"2023-10-08T00:55:47.526450+0000","last_peered":"2023-10-08T00:55:47.526450+0000","last_clean":"2023-10-08T00:55:47.526450+0000","last_became_active":"2023-10-08T00:55:46.514420+0000","last_became_peered":"2023-10-08T00:55:46.514420+0000","last_unstale":"2023-10-08T00:55:47.526450+0000","last_undegraded":"2023-10-08T00:55:47.526450+0000","last_fullsized":"2023-10-08T00:55:47.526450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:15.507854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525520+0000","last_change":"2023-10-08T00:55:47.520974+0000","last_active":"2023-10-08T00:55:47.525520+0000","last_peered":"2023-10-08T00:55:47.525520+0000","last_clean":"2023-10-08T00:55:47.525520+0000","last_became_active":"2023-10-08T00:55:47.520259+0000","last_became_peered":"2023-10-08T00:55:47.520259+0000","last_unstale":"2023-10-08T00:55:47.525520+0000","last_undegraded":"2023-10-08T00:55:47.525520+0000","last_fullsized":"2023-10-08T00:55:47.525520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:57:57.391083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.510639+0000","last_change":"2023-10-08T00:55:46.500260+0000","last_active":"2023-10-08T00:55:47.510639+0000","last_peered":"2023-10-08T00:55:47.510639+0000","last_clean":"2023-10-08T00:55:47.510639+0000","last_became_active":"2023-10-08T00:55:46.499965+0000","last_became_peered":"2023-10-08T00:55:46.499965+0000","last_unstale":"2023-10-08T00:55:47.510639+0000","last_undegraded":"2023-10-08T00:55:47.510639+0000","last_fullsized":"2023-10-08T00:55:47.510639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533942+0000","last_change":"2023-10-08T00:55:47.529127+0000","last_active":"2023-10-08T00:55:47.533942+0000","last_peered":"2023-10-08T00:55:47.533942+0000","last_clean":"2023-10-08T00:55:47.533942+0000","last_became_active":"2023-10-08T00:55:47.528809+0000","last_became_peered":"2023-10-08T00:55:47.528809+0000","last_unstale":"2023-10-08T00:55:47.533942+0000","last_undegraded":"2023-10-08T00:55:47.533942+0000","last_fullsized":"2023-10-08T00:55:47.533942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:07.821288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526404+0000","last_change":"2023-10-08T00:55:47.522934+0000","last_active":"2023-10-08T00:55:47.526404+0000","last_peered":"2023-10-08T00:55:47.526404+0000","last_clean":"2023-10-08T00:55:47.526404+0000","last_became_active":"2023-10-08T00:55:47.522385+0000","last_became_peered":"2023-10-08T00:55:47.522385+0000","last_unstale":"2023-10-08T00:55:47.526404+0000","last_undegraded":"2023-10-08T00:55:47.526404+0000","last_fullsized":"2023-10-08T00:55:47.526404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.515109+0000","last_change":"2023-10-08T00:55:47.508997+0000","last_active":"2023-10-08T00:55:47.515109+0000","last_peered":"2023-10-08T00:55:47.515109+0000","last_clean":"2023-10-08T00:55:47.515109+0000","last_became_active":"2023-10-08T00:55:47.508483+0000","last_became_peered":"2023-10-08T00:55:47.508483+0000","last_unstale":"2023-10-08T00:55:47.515109+0000","last_undegraded":"2023-10-08T00:55:47.515109+0000","last_fullsized":"2023-10-08T00:55:47.515109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:25.085209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525515+0000","last_change":"2023-10-08T00:55:47.520643+0000","last_active":"2023-10-08T00:55:47.525515+0000","last_peered":"2023-10-08T00:55:47.525515+0000","last_clean":"2023-10-08T00:55:47.525515+0000","last_became_active":"2023-10-08T00:55:47.520385+0000","last_became_peered":"2023-10-08T00:55:47.520385+0000","last_unstale":"2023-10-08T00:55:47.525515+0000","last_undegraded":"2023-10-08T00:55:47.525515+0000","last_fullsized":"2023-10-08T00:55:47.525515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:02.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524523+0000","last_change":"2023-10-08T00:55:47.517588+0000","last_active":"2023-10-08T00:55:47.524523+0000","last_peered":"2023-10-08T00:55:47.524523+0000","last_clean":"2023-10-08T00:55:47.524523+0000","last_became_active":"2023-10-08T00:55:47.517257+0000","last_became_peered":"2023-10-08T00:55:47.517257+0000","last_unstale":"2023-10-08T00:55:47.524523+0000","last_undegraded":"2023-10-08T00:55:47.524523+0000","last_fullsized":"2023-10-08T00:55:47.524523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536372+0000","last_change":"2023-10-08T00:55:47.534943+0000","last_active":"2023-10-08T00:55:47.536372+0000","last_peered":"2023-10-08T00:55:47.536372+0000","last_clean":"2023-10-08T00:55:47.536372+0000","last_became_active":"2023-10-08T00:55:47.534593+0000","last_became_peered":"2023-10-08T00:55:47.534593+0000","last_unstale":"2023-10-08T00:55:47.536372+0000","last_undegraded":"2023-10-08T00:55:47.536372+0000","last_fullsized":"2023-10-08T00:55:47.536372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523043+0000","last_change":"2023-10-08T00:55:47.519521+0000","last_active":"2023-10-08T00:55:47.523043+0000","last_peered":"2023-10-08T00:55:47.523043+0000","last_clean":"2023-10-08T00:55:47.523043+0000","last_became_active":"2023-10-08T00:55:47.519205+0000","last_became_peered":"2023-10-08T00:55:47.519205+0000","last_unstale":"2023-10-08T00:55:47.523043+0000","last_undegraded":"2023-10-08T00:55:47.523043+0000","last_fullsized":"2023-10-08T00:55:47.523043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.528710+0000","last_change":"2023-10-08T00:55:47.525909+0000","last_active":"2023-10-08T00:55:47.528710+0000","last_peered":"2023-10-08T00:55:47.528710+0000","last_clean":"2023-10-08T00:55:47.528710+0000","last_became_active":"2023-10-08T00:55:47.525638+0000","last_became_peered":"2023-10-08T00:55:47.525638+0000","last_unstale":"2023-10-08T00:55:47.528710+0000","last_undegraded":"2023-10-08T00:55:47.528710+0000","last_fullsized":"2023-10-08T00:55:47.528710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536365+0000","last_change":"2023-10-08T00:55:47.534909+0000","last_active":"2023-10-08T00:55:47.536365+0000","last_peered":"2023-10-08T00:55:47.536365+0000","last_clean":"2023-10-08T00:55:47.536365+0000","last_became_active":"2023-10-08T00:55:47.534654+0000","last_became_peered":"2023-10-08T00:55:47.534654+0000","last_unstale":"2023-10-08T00:55:47.536365+0000","last_undegraded":"2023-10-08T00:55:47.536365+0000","last_fullsized":"2023-10-08T00:55:47.536365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525191+0000","last_change":"2023-10-08T00:55:47.520067+0000","last_active":"2023-10-08T00:55:47.525191+0000","last_peered":"2023-10-08T00:55:47.525191+0000","last_clean":"2023-10-08T00:55:47.525191+0000","last_became_active":"2023-10-08T00:55:47.519793+0000","last_became_peered":"2023-10-08T00:55:47.519793+0000","last_unstale":"2023-10-08T00:55:47.525191+0000","last_undegraded":"2023-10-08T00:55:47.525191+0000","last_fullsized":"2023-10-08T00:55:47.525191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:34.451497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536355+0000","last_change":"2023-10-08T00:55:47.534891+0000","last_active":"2023-10-08T00:55:47.536355+0000","last_peered":"2023-10-08T00:55:47.536355+0000","last_clean":"2023-10-08T00:55:47.536355+0000","last_became_active":"2023-10-08T00:55:47.534589+0000","last_became_peered":"2023-10-08T00:55:47.534589+0000","last_unstale":"2023-10-08T00:55:47.536355+0000","last_undegraded":"2023-10-08T00:55:47.536355+0000","last_fullsized":"2023-10-08T00:55:47.536355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530986+0000","last_change":"2023-10-08T00:55:47.524426+0000","last_active":"2023-10-08T00:55:47.530986+0000","last_peered":"2023-10-08T00:55:47.530986+0000","last_clean":"2023-10-08T00:55:47.530986+0000","last_became_active":"2023-10-08T00:55:47.524142+0000","last_became_peered":"2023-10-08T00:55:47.524142+0000","last_unstale":"2023-10-08T00:55:47.530986+0000","last_undegraded":"2023-10-08T00:55:47.530986+0000","last_fullsized":"2023-10-08T00:55:47.530986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523999+0000","last_change":"2023-10-08T00:55:47.515215+0000","last_active":"2023-10-08T00:55:47.523999+0000","last_peered":"2023-10-08T00:55:47.523999+0000","last_clean":"2023-10-08T00:55:47.523999+0000","last_became_active":"2023-10-08T00:55:47.514978+0000","last_became_peered":"2023-10-08T00:55:47.514978+0000","last_unstale":"2023-10-08T00:55:47.523999+0000","last_undegraded":"2023-10-08T00:55:47.523999+0000","last_fullsized":"2023-10-08T00:55:47.523999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526430+0000","last_change":"2023-10-08T00:55:47.523078+0000","last_active":"2023-10-08T00:55:47.526430+0000","last_peered":"2023-10-08T00:55:47.526430+0000","last_clean":"2023-10-08T00:55:47.526430+0000","last_became_active":"2023-10-08T00:55:47.522818+0000","last_became_peered":"2023-10-08T00:55:47.522818+0000","last_unstale":"2023-10-08T00:55:47.526430+0000","last_undegraded":"2023-10-08T00:55:47.526430+0000","last_fullsized":"2023-10-08T00:55:47.526430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":661632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T00:55:57.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:55:57.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.496+0000 7fd746bff700 1 Processor -- start 2023-10-08T00:55:57.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.497+0000 7fd746bff700 1 -- start start 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd746bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd740159890 0x7fd74015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd746bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd740069560 0x7fd7400699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd746bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd740069ee0 con 0x7fd740159c60 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd746bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74015a160 con 0x7fd740159890 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd746bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7400d13a0 con 0x7fd740069560 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd74499b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd740159890 0x7fd74015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd74499b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd740159890 0x7fd74015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33128/0 (socket says 172.21.15.119:33128) 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd74499b700 1 -- 172.21.15.119:0/3869904478 learned_addr learned my addr 172.21.15.119:0/3869904478 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:57.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.498+0000 7fd74519c700 1 --2- 172.21.15.119:0/3869904478 >> v2:172.21.15.184:3300/0 conn(0x7fd740069560 0x7fd7400699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.499+0000 7fd74499b700 1 -- 172.21.15.119:0/3869904478 >> v2:172.21.15.184:3300/0 conn(0x7fd740069560 msgr2=0x7fd7400699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:57.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.499+0000 7fd74499b700 1 --2- 172.21.15.119:0/3869904478 >> v2:172.21.15.184:3300/0 conn(0x7fd740069560 0x7fd7400699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.499+0000 7fd74499b700 1 -- 172.21.15.119:0/3869904478 >> v1:172.21.15.184:6789/0 conn(0x7fd740159c60 legacy=0x7fd74015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.499+0000 7fd74499b700 1 -- 172.21.15.119:0/3869904478 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74015e140 con 0x7fd740159890 2023-10-08T00:55:57.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.499+0000 7fd74499b700 1 --2- 172.21.15.119:0/3869904478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159890 0x7fd74015abd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd74006a890 tx=0x7fd734009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132d885a57f00efc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.499+0000 7fd73f7fe700 1 -- 172.21.15.119:0/3869904478 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73400b820 con 0x7fd740159890 2023-10-08T00:55:57.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.499+0000 7fd73f7fe700 1 -- 172.21.15.119:0/3869904478 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73400b980 con 0x7fd740159890 2023-10-08T00:55:57.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.499+0000 7fd73f7fe700 1 -- 172.21.15.119:0/3869904478 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73400bd20 con 0x7fd740159890 2023-10-08T00:55:57.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.500+0000 7fd746bff700 1 -- 172.21.15.119:0/3869904478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159890 msgr2=0x7fd74015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.500+0000 7fd746bff700 1 --2- 172.21.15.119:0/3869904478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159890 0x7fd74015abd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd74006a890 tx=0x7fd734009250 comp rx=0 tx=0).stop 2023-10-08T00:55:57.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.500+0000 7fd746bff700 1 -- 172.21.15.119:0/3869904478 shutdown_connections 2023-10-08T00:55:57.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.500+0000 7fd746bff700 1 --2- 172.21.15.119:0/3869904478 >> v2:172.21.15.184:3300/0 conn(0x7fd740069560 0x7fd7400699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.500+0000 7fd746bff700 1 --2- 172.21.15.119:0/3869904478 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159890 0x7fd74015abd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.500+0000 7fd746bff700 1 -- 172.21.15.119:0/3869904478 >> 172.21.15.119:0/3869904478 conn(0x7fd740068e20 msgr2=0x7fd7400d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:57.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.500+0000 7fd746bff700 1 -- 172.21.15.119:0/3869904478 shutdown_connections 2023-10-08T00:55:57.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.500+0000 7fd746bff700 1 -- 172.21.15.119:0/3869904478 wait complete. 2023-10-08T00:55:57.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.501+0000 7fd746bff700 1 Processor -- start 2023-10-08T00:55:57.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.501+0000 7fd746bff700 1 -- start start 2023-10-08T00:55:57.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.502+0000 7fd746bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd740069560 0x7fd740152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.502+0000 7fd746bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd740159890 0x7fd7400c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.502+0000 7fd746bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159c60 0x7fd7400c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.502+0000 7fd746bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400b36c0 con 0x7fd740159c60 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.502+0000 7fd746bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400c86f0 con 0x7fd740159890 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.502+0000 7fd746bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400c89d0 con 0x7fd740069560 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.502+0000 7fd74519c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159c60 0x7fd7400c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74519c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159c60 0x7fd7400c9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33144/0 (socket says 172.21.15.119:33144) 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74519c700 1 -- 172.21.15.119:0/1421578018 learned_addr learned my addr 172.21.15.119:0/1421578018 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74499b700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd740069560 0x7fd740152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd73ffff700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd740159890 0x7fd7400c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74519c700 1 -- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd740069560 msgr2=0x7fd740152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74519c700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd740069560 0x7fd740152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74519c700 1 -- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd740159890 msgr2=0x7fd7400c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74519c700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd740159890 0x7fd7400c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74519c700 1 -- 172.21.15.119:0/1421578018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7340094f0 con 0x7fd740159c60 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74499b700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd740069560 0x7fd740152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd74519c700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159c60 0x7fd7400c9d70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd738009fa0 tx=0x7fd738009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd73cff9700 1 -- 172.21.15.119:0/1421578018 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73800ed20 con 0x7fd740159c60 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd73cff9700 1 -- 172.21.15.119:0/1421578018 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73800ee80 con 0x7fd740159c60 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7400c8cb0 con 0x7fd740159c60 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.503+0000 7fd73cff9700 1 -- 172.21.15.119:0/1421578018 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd738014670 con 0x7fd740159c60 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.504+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd740152d40 con 0x7fd740159c60 2023-10-08T00:55:57.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.504+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74004c3c0 con 0x7fd740159c60 2023-10-08T00:55:57.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.505+0000 7fd73cff9700 1 -- 172.21.15.119:0/1421578018 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7380149e0 con 0x7fd740159c60 2023-10-08T00:55:57.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.506+0000 7fd73cff9700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd724066f30 0x7fd7240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.506+0000 7fd73cff9700 1 -- 172.21.15.119:0/1421578018 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd738084d50 con 0x7fd740159c60 2023-10-08T00:55:57.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.506+0000 7fd74499b700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd724066f30 0x7fd7240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.507+0000 7fd74499b700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd724066f30 0x7fd7240693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd74006a890 tx=0x7fd734016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.510+0000 7fd73cff9700 1 -- 172.21.15.119:0/1421578018 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd738051f20 con 0x7fd740159c60 2023-10-08T00:55:57.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.700+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7400b3800 con 0x7fd724066f30 2023-10-08T00:55:57.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.707+0000 7fd73cff9700 1 -- 172.21.15.119:0/1421578018 <== mgr.4101 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264483 (secure 0 0 0) 0x7fd7400b3800 con 0x7fd724066f30 2023-10-08T00:55:57.712 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd724066f30 msgr2=0x7fd7240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd724066f30 0x7fd7240693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd74006a890 tx=0x7fd734016000 comp rx=0 tx=0).stop 2023-10-08T00:55:57.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159c60 msgr2=0x7fd7400c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159c60 0x7fd7400c9d70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd738009fa0 tx=0x7fd738009900 comp rx=0 tx=0).stop 2023-10-08T00:55:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 shutdown_connections 2023-10-08T00:55:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd740069560 0x7fd740152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd724066f30 0x7fd7240693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd740159c60 0x7fd7400c9d70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 --2- 172.21.15.119:0/1421578018 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd740159890 0x7fd7400c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 >> 172.21.15.119:0/1421578018 conn(0x7fd740068e20 msgr2=0x7fd7400d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:57.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 shutdown_connections 2023-10-08T00:55:57.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.712+0000 7fd746bff700 1 -- 172.21.15.119:0/1421578018 wait complete. 2023-10-08T00:55:57.716 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T00:55:57.730 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T00:55:55.859674+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381128,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593592,"statfs":{"total":773094113280,"available":770655838208,"internally_reserved":0,"allocated":2891776,"data_stored":1750580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.368133"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524928+0000","last_change":"2023-10-08T00:55:47.519961+0000","last_active":"2023-10-08T00:55:47.524928+0000","last_peered":"2023-10-08T00:55:47.524928+0000","last_clean":"2023-10-08T00:55:47.524928+0000","last_became_active":"2023-10-08T00:55:47.519667+0000","last_became_peered":"2023-10-08T00:55:47.519667+0000","last_unstale":"2023-10-08T00:55:47.524928+0000","last_undegraded":"2023-10-08T00:55:47.524928+0000","last_fullsized":"2023-10-08T00:55:47.524928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:20:20.644250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520418+0000","last_change":"2023-10-08T00:55:47.516100+0000","last_active":"2023-10-08T00:55:47.520418+0000","last_peered":"2023-10-08T00:55:47.520418+0000","last_clean":"2023-10-08T00:55:47.520418+0000","last_became_active":"2023-10-08T00:55:47.515088+0000","last_became_peered":"2023-10-08T00:55:47.515088+0000","last_unstale":"2023-10-08T00:55:47.520418+0000","last_undegraded":"2023-10-08T00:55:47.520418+0000","last_fullsized":"2023-10-08T00:55:47.520418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523625+0000","last_change":"2023-10-08T00:55:47.514661+0000","last_active":"2023-10-08T00:55:47.523625+0000","last_peered":"2023-10-08T00:55:47.523625+0000","last_clean":"2023-10-08T00:55:47.523625+0000","last_became_active":"2023-10-08T00:55:47.512991+0000","last_became_peered":"2023-10-08T00:55:47.512991+0000","last_unstale":"2023-10-08T00:55:47.523625+0000","last_undegraded":"2023-10-08T00:55:47.523625+0000","last_fullsized":"2023-10-08T00:55:47.523625+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:25.085209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525503+0000","last_change":"2023-10-08T00:55:47.520755+0000","last_active":"2023-10-08T00:55:47.525503+0000","last_peered":"2023-10-08T00:55:47.525503+0000","last_clean":"2023-10-08T00:55:47.525503+0000","last_became_active":"2023-10-08T00:55:47.519925+0000","last_became_peered":"2023-10-08T00:55:47.519925+0000","last_unstale":"2023-10-08T00:55:47.525503+0000","last_undegraded":"2023-10-08T00:55:47.525503+0000","last_fullsized":"2023-10-08T00:55:47.525503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.513000+0000","last_change":"2023-10-08T00:55:47.510024+0000","last_active":"2023-10-08T00:55:47.513000+0000","last_peered":"2023-10-08T00:55:47.513000+0000","last_clean":"2023-10-08T00:55:47.513000+0000","last_became_active":"2023-10-08T00:55:47.509649+0000","last_became_peered":"2023-10-08T00:55:47.509649+0000","last_unstale":"2023-10-08T00:55:47.513000+0000","last_undegraded":"2023-10-08T00:55:47.513000+0000","last_fullsized":"2023-10-08T00:55:47.513000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525212+0000","last_change":"2023-10-08T00:55:47.514675+0000","last_active":"2023-10-08T00:55:47.525212+0000","last_peered":"2023-10-08T00:55:47.525212+0000","last_clean":"2023-10-08T00:55:47.525212+0000","last_became_active":"2023-10-08T00:55:47.514414+0000","last_became_peered":"2023-10-08T00:55:47.514414+0000","last_unstale":"2023-10-08T00:55:47.525212+0000","last_undegraded":"2023-10-08T00:55:47.525212+0000","last_fullsized":"2023-10-08T00:55:47.525212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:07.821288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524753+0000","last_change":"2023-10-08T00:55:47.516013+0000","last_active":"2023-10-08T00:55:47.524753+0000","last_peered":"2023-10-08T00:55:47.524753+0000","last_clean":"2023-10-08T00:55:47.524753+0000","last_became_active":"2023-10-08T00:55:47.515695+0000","last_became_peered":"2023-10-08T00:55:47.515695+0000","last_unstale":"2023-10-08T00:55:47.524753+0000","last_undegraded":"2023-10-08T00:55:47.524753+0000","last_fullsized":"2023-10-08T00:55:47.524753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:44.607367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.527135+0000","last_change":"2023-10-08T00:55:47.525615+0000","last_active":"2023-10-08T00:55:47.527135+0000","last_peered":"2023-10-08T00:55:47.527135+0000","last_clean":"2023-10-08T00:55:47.527135+0000","last_became_active":"2023-10-08T00:55:47.525281+0000","last_became_peered":"2023-10-08T00:55:47.525281+0000","last_unstale":"2023-10-08T00:55:47.527135+0000","last_undegraded":"2023-10-08T00:55:47.527135+0000","last_fullsized":"2023-10-08T00:55:47.527135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522070+0000","last_change":"2023-10-08T00:55:47.514405+0000","last_active":"2023-10-08T00:55:47.522070+0000","last_peered":"2023-10-08T00:55:47.522070+0000","last_clean":"2023-10-08T00:55:47.522070+0000","last_became_active":"2023-10-08T00:55:47.513815+0000","last_became_peered":"2023-10-08T00:55:47.513815+0000","last_unstale":"2023-10-08T00:55:47.522070+0000","last_undegraded":"2023-10-08T00:55:47.522070+0000","last_fullsized":"2023-10-08T00:55:47.522070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:34.078412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524336+0000","last_change":"2023-10-08T00:55:47.518070+0000","last_active":"2023-10-08T00:55:47.524336+0000","last_peered":"2023-10-08T00:55:47.524336+0000","last_clean":"2023-10-08T00:55:47.524336+0000","last_became_active":"2023-10-08T00:55:47.517802+0000","last_became_peered":"2023-10-08T00:55:47.517802+0000","last_unstale":"2023-10-08T00:55:47.524336+0000","last_undegraded":"2023-10-08T00:55:47.524336+0000","last_fullsized":"2023-10-08T00:55:47.524336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518986+0000","last_change":"2023-10-08T00:55:47.517157+0000","last_active":"2023-10-08T00:55:47.518986+0000","last_peered":"2023-10-08T00:55:47.518986+0000","last_clean":"2023-10-08T00:55:47.518986+0000","last_became_active":"2023-10-08T00:55:47.516876+0000","last_became_peered":"2023-10-08T00:55:47.516876+0000","last_unstale":"2023-10-08T00:55:47.518986+0000","last_undegraded":"2023-10-08T00:55:47.518986+0000","last_fullsized":"2023-10-08T00:55:47.518986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533808+0000","last_change":"2023-10-08T00:55:47.520192+0000","last_active":"2023-10-08T00:55:47.533808+0000","last_peered":"2023-10-08T00:55:47.533808+0000","last_clean":"2023-10-08T00:55:47.533808+0000","last_became_active":"2023-10-08T00:55:47.519911+0000","last_became_peered":"2023-10-08T00:55:47.519911+0000","last_unstale":"2023-10-08T00:55:47.533808+0000","last_undegraded":"2023-10-08T00:55:47.533808+0000","last_fullsized":"2023-10-08T00:55:47.533808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:44.817925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526111+0000","last_change":"2023-10-08T00:55:47.522124+0000","last_active":"2023-10-08T00:55:47.526111+0000","last_peered":"2023-10-08T00:55:47.526111+0000","last_clean":"2023-10-08T00:55:47.526111+0000","last_became_active":"2023-10-08T00:55:47.521242+0000","last_became_peered":"2023-10-08T00:55:47.521242+0000","last_unstale":"2023-10-08T00:55:47.526111+0000","last_undegraded":"2023-10-08T00:55:47.526111+0000","last_fullsized":"2023-10-08T00:55:47.526111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520534+0000","last_change":"2023-10-08T00:55:47.508213+0000","last_active":"2023-10-08T00:55:47.520534+0000","last_peered":"2023-10-08T00:55:47.520534+0000","last_clean":"2023-10-08T00:55:47.520534+0000","last_became_active":"2023-10-08T00:55:47.507914+0000","last_became_peered":"2023-10-08T00:55:47.507914+0000","last_unstale":"2023-10-08T00:55:47.520534+0000","last_undegraded":"2023-10-08T00:55:47.520534+0000","last_fullsized":"2023-10-08T00:55:47.520534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.510943+0000","last_change":"2023-10-08T00:55:47.504306+0000","last_active":"2023-10-08T00:55:47.510943+0000","last_peered":"2023-10-08T00:55:47.510943+0000","last_clean":"2023-10-08T00:55:47.510943+0000","last_became_active":"2023-10-08T00:55:47.504037+0000","last_became_peered":"2023-10-08T00:55:47.504037+0000","last_unstale":"2023-10-08T00:55:47.510943+0000","last_undegraded":"2023-10-08T00:55:47.510943+0000","last_fullsized":"2023-10-08T00:55:47.510943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530847+0000","last_change":"2023-10-08T00:55:47.524003+0000","last_active":"2023-10-08T00:55:47.530847+0000","last_peered":"2023-10-08T00:55:47.530847+0000","last_clean":"2023-10-08T00:55:47.530847+0000","last_became_active":"2023-10-08T00:55:47.521541+0000","last_became_peered":"2023-10-08T00:55:47.521541+0000","last_unstale":"2023-10-08T00:55:47.530847+0000","last_undegraded":"2023-10-08T00:55:47.530847+0000","last_fullsized":"2023-10-08T00:55:47.530847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:44.607367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.511787+0000","last_change":"2023-10-08T00:55:47.509224+0000","last_active":"2023-10-08T00:55:47.511787+0000","last_peered":"2023-10-08T00:55:47.511787+0000","last_clean":"2023-10-08T00:55:47.511787+0000","last_became_active":"2023-10-08T00:55:47.508714+0000","last_became_peered":"2023-10-08T00:55:47.508714+0000","last_unstale":"2023-10-08T00:55:47.511787+0000","last_undegraded":"2023-10-08T00:55:47.511787+0000","last_fullsized":"2023-10-08T00:55:47.511787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525712+0000","last_change":"2023-10-08T00:55:47.519330+0000","last_active":"2023-10-08T00:55:47.525712+0000","last_peered":"2023-10-08T00:55:47.525712+0000","last_clean":"2023-10-08T00:55:47.525712+0000","last_became_active":"2023-10-08T00:55:47.519070+0000","last_became_peered":"2023-10-08T00:55:47.519070+0000","last_unstale":"2023-10-08T00:55:47.525712+0000","last_undegraded":"2023-10-08T00:55:47.525712+0000","last_fullsized":"2023-10-08T00:55:47.525712+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:03.776133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.514578+0000","last_change":"2023-10-08T00:55:47.509945+0000","last_active":"2023-10-08T00:55:47.514578+0000","last_peered":"2023-10-08T00:55:47.514578+0000","last_clean":"2023-10-08T00:55:47.514578+0000","last_became_active":"2023-10-08T00:55:47.509596+0000","last_became_peered":"2023-10-08T00:55:47.509596+0000","last_unstale":"2023-10-08T00:55:47.514578+0000","last_undegraded":"2023-10-08T00:55:47.514578+0000","last_fullsized":"2023-10-08T00:55:47.514578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:07.510837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525580+0000","last_change":"2023-10-08T00:55:46.514258+0000","last_active":"2023-10-08T00:55:47.525580+0000","last_peered":"2023-10-08T00:55:47.525580+0000","last_clean":"2023-10-08T00:55:47.525580+0000","last_became_active":"2023-10-08T00:55:46.514002+0000","last_became_peered":"2023-10-08T00:55:46.514002+0000","last_unstale":"2023-10-08T00:55:47.525580+0000","last_undegraded":"2023-10-08T00:55:47.525580+0000","last_fullsized":"2023-10-08T00:55:47.525580+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520841+0000","last_change":"2023-10-08T00:55:47.519479+0000","last_active":"2023-10-08T00:55:47.520841+0000","last_peered":"2023-10-08T00:55:47.520841+0000","last_clean":"2023-10-08T00:55:47.520841+0000","last_became_active":"2023-10-08T00:55:47.519179+0000","last_became_peered":"2023-10-08T00:55:47.519179+0000","last_unstale":"2023-10-08T00:55:47.520841+0000","last_undegraded":"2023-10-08T00:55:47.520841+0000","last_fullsized":"2023-10-08T00:55:47.520841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523749+0000","last_change":"2023-10-08T00:55:46.507226+0000","last_active":"2023-10-08T00:55:47.523749+0000","last_peered":"2023-10-08T00:55:47.523749+0000","last_clean":"2023-10-08T00:55:47.523749+0000","last_became_active":"2023-10-08T00:55:46.506878+0000","last_became_peered":"2023-10-08T00:55:46.506878+0000","last_unstale":"2023-10-08T00:55:47.523749+0000","last_undegraded":"2023-10-08T00:55:47.523749+0000","last_fullsized":"2023-10-08T00:55:47.523749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523694+0000","last_change":"2023-10-08T00:55:47.514717+0000","last_active":"2023-10-08T00:55:47.523694+0000","last_peered":"2023-10-08T00:55:47.523694+0000","last_clean":"2023-10-08T00:55:47.523694+0000","last_became_active":"2023-10-08T00:55:47.514339+0000","last_became_peered":"2023-10-08T00:55:47.514339+0000","last_unstale":"2023-10-08T00:55:47.523694+0000","last_undegraded":"2023-10-08T00:55:47.523694+0000","last_fullsized":"2023-10-08T00:55:47.523694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522336+0000","last_change":"2023-10-08T00:55:46.517327+0000","last_active":"2023-10-08T00:55:47.522336+0000","last_peered":"2023-10-08T00:55:47.522336+0000","last_clean":"2023-10-08T00:55:47.522336+0000","last_became_active":"2023-10-08T00:55:46.516908+0000","last_became_peered":"2023-10-08T00:55:46.516908+0000","last_unstale":"2023-10-08T00:55:47.522336+0000","last_undegraded":"2023-10-08T00:55:47.522336+0000","last_fullsized":"2023-10-08T00:55:47.522336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525482+0000","last_change":"2023-10-08T00:55:47.520563+0000","last_active":"2023-10-08T00:55:47.525482+0000","last_peered":"2023-10-08T00:55:47.525482+0000","last_clean":"2023-10-08T00:55:47.525482+0000","last_became_active":"2023-10-08T00:55:47.519545+0000","last_became_peered":"2023-10-08T00:55:47.519545+0000","last_unstale":"2023-10-08T00:55:47.525482+0000","last_undegraded":"2023-10-08T00:55:47.525482+0000","last_fullsized":"2023-10-08T00:55:47.525482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.528020+0000","last_change":"2023-10-08T00:55:46.521160+0000","last_active":"2023-10-08T00:55:47.528020+0000","last_peered":"2023-10-08T00:55:47.528020+0000","last_clean":"2023-10-08T00:55:47.528020+0000","last_became_active":"2023-10-08T00:55:46.520820+0000","last_became_peered":"2023-10-08T00:55:46.520820+0000","last_unstale":"2023-10-08T00:55:47.528020+0000","last_undegraded":"2023-10-08T00:55:47.528020+0000","last_fullsized":"2023-10-08T00:55:47.528020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:24.078567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533581+0000","last_change":"2023-10-08T00:55:47.526348+0000","last_active":"2023-10-08T00:55:47.533581+0000","last_peered":"2023-10-08T00:55:47.533581+0000","last_clean":"2023-10-08T00:55:47.533581+0000","last_became_active":"2023-10-08T00:55:47.526167+0000","last_became_peered":"2023-10-08T00:55:47.526167+0000","last_unstale":"2023-10-08T00:55:47.533581+0000","last_undegraded":"2023-10-08T00:55:47.533581+0000","last_fullsized":"2023-10-08T00:55:47.533581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522448+0000","last_change":"2023-10-08T00:55:46.505995+0000","last_active":"2023-10-08T00:55:47.522448+0000","last_peered":"2023-10-08T00:55:47.522448+0000","last_clean":"2023-10-08T00:55:47.522448+0000","last_became_active":"2023-10-08T00:55:46.505673+0000","last_became_peered":"2023-10-08T00:55:46.505673+0000","last_unstale":"2023-10-08T00:55:47.522448+0000","last_undegraded":"2023-10-08T00:55:47.522448+0000","last_fullsized":"2023-10-08T00:55:47.522448+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:27.624629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533980+0000","last_change":"2023-10-08T00:55:47.525941+0000","last_active":"2023-10-08T00:55:47.533980+0000","last_peered":"2023-10-08T00:55:47.533980+0000","last_clean":"2023-10-08T00:55:47.533980+0000","last_became_active":"2023-10-08T00:55:47.524635+0000","last_became_peered":"2023-10-08T00:55:47.524635+0000","last_unstale":"2023-10-08T00:55:47.533980+0000","last_undegraded":"2023-10-08T00:55:47.533980+0000","last_fullsized":"2023-10-08T00:55:47.533980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:09.261793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.522850+0000","last_change":"2023-10-08T00:55:46.506810+0000","last_active":"2023-10-08T00:55:47.522850+0000","last_peered":"2023-10-08T00:55:47.522850+0000","last_clean":"2023-10-08T00:55:47.522850+0000","last_became_active":"2023-10-08T00:55:46.506504+0000","last_became_peered":"2023-10-08T00:55:46.506504+0000","last_unstale":"2023-10-08T00:55:47.522850+0000","last_undegraded":"2023-10-08T00:55:47.522850+0000","last_fullsized":"2023-10-08T00:55:47.522850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524500+0000","last_change":"2023-10-08T00:55:47.516584+0000","last_active":"2023-10-08T00:55:47.524500+0000","last_peered":"2023-10-08T00:55:47.524500+0000","last_clean":"2023-10-08T00:55:47.524500+0000","last_became_active":"2023-10-08T00:55:47.516222+0000","last_became_peered":"2023-10-08T00:55:47.516222+0000","last_unstale":"2023-10-08T00:55:47.524500+0000","last_undegraded":"2023-10-08T00:55:47.524500+0000","last_fullsized":"2023-10-08T00:55:47.524500+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.506266+0000","last_change":"2023-10-08T00:55:46.499483+0000","last_active":"2023-10-08T00:55:47.506266+0000","last_peered":"2023-10-08T00:55:47.506266+0000","last_clean":"2023-10-08T00:55:47.506266+0000","last_became_active":"2023-10-08T00:55:46.499153+0000","last_became_peered":"2023-10-08T00:55:46.499153+0000","last_unstale":"2023-10-08T00:55:47.506266+0000","last_undegraded":"2023-10-08T00:55:47.506266+0000","last_fullsized":"2023-10-08T00:55:47.506266+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525986+0000","last_change":"2023-10-08T00:55:47.521897+0000","last_active":"2023-10-08T00:55:47.525986+0000","last_peered":"2023-10-08T00:55:47.525986+0000","last_clean":"2023-10-08T00:55:47.525986+0000","last_became_active":"2023-10-08T00:55:47.521630+0000","last_became_peered":"2023-10-08T00:55:47.521630+0000","last_unstale":"2023-10-08T00:55:47.525986+0000","last_undegraded":"2023-10-08T00:55:47.525986+0000","last_fullsized":"2023-10-08T00:55:47.525986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.502490+0000","last_change":"2023-10-08T00:55:46.520040+0000","last_active":"2023-10-08T00:55:47.502490+0000","last_peered":"2023-10-08T00:55:47.502490+0000","last_clean":"2023-10-08T00:55:47.502490+0000","last_became_active":"2023-10-08T00:55:46.519780+0000","last_became_peered":"2023-10-08T00:55:46.519780+0000","last_unstale":"2023-10-08T00:55:47.502490+0000","last_undegraded":"2023-10-08T00:55:47.502490+0000","last_fullsized":"2023-10-08T00:55:47.502490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523199+0000","last_change":"2023-10-08T00:55:47.518074+0000","last_active":"2023-10-08T00:55:47.523199+0000","last_peered":"2023-10-08T00:55:47.523199+0000","last_clean":"2023-10-08T00:55:47.523199+0000","last_became_active":"2023-10-08T00:55:47.517822+0000","last_became_peered":"2023-10-08T00:55:47.517822+0000","last_unstale":"2023-10-08T00:55:47.523199+0000","last_undegraded":"2023-10-08T00:55:47.523199+0000","last_fullsized":"2023-10-08T00:55:47.523199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507109+0000","last_change":"2023-10-08T00:55:46.520607+0000","last_active":"2023-10-08T00:55:47.507109+0000","last_peered":"2023-10-08T00:55:47.507109+0000","last_clean":"2023-10-08T00:55:47.507109+0000","last_became_active":"2023-10-08T00:55:46.520340+0000","last_became_peered":"2023-10-08T00:55:46.520340+0000","last_unstale":"2023-10-08T00:55:47.507109+0000","last_undegraded":"2023-10-08T00:55:47.507109+0000","last_fullsized":"2023-10-08T00:55:47.507109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.531004+0000","last_change":"2023-10-08T00:55:47.524779+0000","last_active":"2023-10-08T00:55:47.531004+0000","last_peered":"2023-10-08T00:55:47.531004+0000","last_clean":"2023-10-08T00:55:47.531004+0000","last_became_active":"2023-10-08T00:55:47.524520+0000","last_became_peered":"2023-10-08T00:55:47.524520+0000","last_unstale":"2023-10-08T00:55:47.531004+0000","last_undegraded":"2023-10-08T00:55:47.531004+0000","last_fullsized":"2023-10-08T00:55:47.531004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.505478+0000","last_change":"2023-10-08T00:55:46.520234+0000","last_active":"2023-10-08T00:55:47.505478+0000","last_peered":"2023-10-08T00:55:47.505478+0000","last_clean":"2023-10-08T00:55:47.505478+0000","last_became_active":"2023-10-08T00:55:46.519769+0000","last_became_peered":"2023-10-08T00:55:46.519769+0000","last_unstale":"2023-10-08T00:55:47.505478+0000","last_undegraded":"2023-10-08T00:55:47.505478+0000","last_fullsized":"2023-10-08T00:55:47.505478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523717+0000","last_change":"2023-10-08T00:55:47.514712+0000","last_active":"2023-10-08T00:55:47.523717+0000","last_peered":"2023-10-08T00:55:47.523717+0000","last_clean":"2023-10-08T00:55:47.523717+0000","last_became_active":"2023-10-08T00:55:47.514326+0000","last_became_peered":"2023-10-08T00:55:47.514326+0000","last_unstale":"2023-10-08T00:55:47.523717+0000","last_undegraded":"2023-10-08T00:55:47.523717+0000","last_fullsized":"2023-10-08T00:55:47.523717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.513999+0000","last_change":"2023-10-08T00:55:46.520938+0000","last_active":"2023-10-08T00:55:47.513999+0000","last_peered":"2023-10-08T00:55:47.513999+0000","last_clean":"2023-10-08T00:55:47.513999+0000","last_became_active":"2023-10-08T00:55:46.520707+0000","last_became_peered":"2023-10-08T00:55:46.520707+0000","last_unstale":"2023-10-08T00:55:47.513999+0000","last_undegraded":"2023-10-08T00:55:47.513999+0000","last_fullsized":"2023-10-08T00:55:47.513999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:43.811282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.534039+0000","last_change":"2023-10-08T00:55:47.527280+0000","last_active":"2023-10-08T00:55:47.534039+0000","last_peered":"2023-10-08T00:55:47.534039+0000","last_clean":"2023-10-08T00:55:47.534039+0000","last_became_active":"2023-10-08T00:55:47.526962+0000","last_became_peered":"2023-10-08T00:55:47.526962+0000","last_unstale":"2023-10-08T00:55:47.534039+0000","last_undegraded":"2023-10-08T00:55:47.534039+0000","last_fullsized":"2023-10-08T00:55:47.534039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:03.776133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.529949+0000","last_change":"2023-10-08T00:55:46.496099+0000","last_active":"2023-10-08T00:55:47.529949+0000","last_peered":"2023-10-08T00:55:47.529949+0000","last_clean":"2023-10-08T00:55:47.529949+0000","last_became_active":"2023-10-08T00:55:46.495752+0000","last_became_peered":"2023-10-08T00:55:46.495752+0000","last_unstale":"2023-10-08T00:55:47.529949+0000","last_undegraded":"2023-10-08T00:55:47.529949+0000","last_fullsized":"2023-10-08T00:55:47.529949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524959+0000","last_change":"2023-10-08T00:55:47.521795+0000","last_active":"2023-10-08T00:55:47.524959+0000","last_peered":"2023-10-08T00:55:47.524959+0000","last_clean":"2023-10-08T00:55:47.524959+0000","last_became_active":"2023-10-08T00:55:47.521477+0000","last_became_peered":"2023-10-08T00:55:47.521477+0000","last_unstale":"2023-10-08T00:55:47.524959+0000","last_undegraded":"2023-10-08T00:55:47.524959+0000","last_fullsized":"2023-10-08T00:55:47.524959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:02.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520109+0000","last_change":"2023-10-08T00:55:46.509869+0000","last_active":"2023-10-08T00:55:47.520109+0000","last_peered":"2023-10-08T00:55:47.520109+0000","last_clean":"2023-10-08T00:55:47.520109+0000","last_became_active":"2023-10-08T00:55:46.509581+0000","last_became_peered":"2023-10-08T00:55:46.509581+0000","last_unstale":"2023-10-08T00:55:47.520109+0000","last_undegraded":"2023-10-08T00:55:47.520109+0000","last_fullsized":"2023-10-08T00:55:47.520109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:27:51.809343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.521324+0000","last_change":"2023-10-08T00:55:47.512931+0000","last_active":"2023-10-08T00:55:47.521324+0000","last_peered":"2023-10-08T00:55:47.521324+0000","last_clean":"2023-10-08T00:55:47.521324+0000","last_became_active":"2023-10-08T00:55:47.512674+0000","last_became_peered":"2023-10-08T00:55:47.512674+0000","last_unstale":"2023-10-08T00:55:47.521324+0000","last_undegraded":"2023-10-08T00:55:47.521324+0000","last_fullsized":"2023-10-08T00:55:47.521324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.501686+0000","last_change":"2023-10-08T00:55:46.519403+0000","last_active":"2023-10-08T00:55:47.501686+0000","last_peered":"2023-10-08T00:55:47.501686+0000","last_clean":"2023-10-08T00:55:47.501686+0000","last_became_active":"2023-10-08T00:55:46.516356+0000","last_became_peered":"2023-10-08T00:55:46.516356+0000","last_unstale":"2023-10-08T00:55:47.501686+0000","last_undegraded":"2023-10-08T00:55:47.501686+0000","last_fullsized":"2023-10-08T00:55:47.501686+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507923+0000","last_change":"2023-10-08T00:55:47.504128+0000","last_active":"2023-10-08T00:55:47.507923+0000","last_peered":"2023-10-08T00:55:47.507923+0000","last_clean":"2023-10-08T00:55:47.507923+0000","last_became_active":"2023-10-08T00:55:47.503735+0000","last_became_peered":"2023-10-08T00:55:47.503735+0000","last_unstale":"2023-10-08T00:55:47.507923+0000","last_undegraded":"2023-10-08T00:55:47.507923+0000","last_fullsized":"2023-10-08T00:55:47.507923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.509141+0000","last_change":"2023-10-08T00:55:46.506608+0000","last_active":"2023-10-08T00:55:47.509141+0000","last_peered":"2023-10-08T00:55:47.509141+0000","last_clean":"2023-10-08T00:55:47.509141+0000","last_became_active":"2023-10-08T00:55:46.506233+0000","last_became_peered":"2023-10-08T00:55:46.506233+0000","last_unstale":"2023-10-08T00:55:47.509141+0000","last_undegraded":"2023-10-08T00:55:47.509141+0000","last_fullsized":"2023-10-08T00:55:47.509141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526384+0000","last_change":"2023-10-08T00:55:47.522727+0000","last_active":"2023-10-08T00:55:47.526384+0000","last_peered":"2023-10-08T00:55:47.526384+0000","last_clean":"2023-10-08T00:55:47.526384+0000","last_became_active":"2023-10-08T00:55:47.522019+0000","last_became_peered":"2023-10-08T00:55:47.522019+0000","last_unstale":"2023-10-08T00:55:47.526384+0000","last_undegraded":"2023-10-08T00:55:47.526384+0000","last_fullsized":"2023-10-08T00:55:47.526384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:23.427683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520506+0000","last_change":"2023-10-08T00:55:46.506201+0000","last_active":"2023-10-08T00:55:47.520506+0000","last_peered":"2023-10-08T00:55:47.520506+0000","last_clean":"2023-10-08T00:55:47.520506+0000","last_became_active":"2023-10-08T00:55:46.505555+0000","last_became_peered":"2023-10-08T00:55:46.505555+0000","last_unstale":"2023-10-08T00:55:47.520506+0000","last_undegraded":"2023-10-08T00:55:47.520506+0000","last_fullsized":"2023-10-08T00:55:47.520506+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533925+0000","last_change":"2023-10-08T00:55:47.528839+0000","last_active":"2023-10-08T00:55:47.533925+0000","last_peered":"2023-10-08T00:55:47.533925+0000","last_clean":"2023-10-08T00:55:47.533925+0000","last_became_active":"2023-10-08T00:55:47.528573+0000","last_became_peered":"2023-10-08T00:55:47.528573+0000","last_unstale":"2023-10-08T00:55:47.533925+0000","last_undegraded":"2023-10-08T00:55:47.533925+0000","last_fullsized":"2023-10-08T00:55:47.533925+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:57:57.391083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.517881+0000","last_change":"2023-10-08T00:55:46.506125+0000","last_active":"2023-10-08T00:55:47.517881+0000","last_peered":"2023-10-08T00:55:47.517881+0000","last_clean":"2023-10-08T00:55:47.517881+0000","last_became_active":"2023-10-08T00:55:46.505815+0000","last_became_peered":"2023-10-08T00:55:46.505815+0000","last_unstale":"2023-10-08T00:55:47.517881+0000","last_undegraded":"2023-10-08T00:55:47.517881+0000","last_fullsized":"2023-10-08T00:55:47.517881+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:55.901248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533963+0000","last_change":"2023-10-08T00:55:47.529148+0000","last_active":"2023-10-08T00:55:47.533963+0000","last_peered":"2023-10-08T00:55:47.533963+0000","last_clean":"2023-10-08T00:55:47.533963+0000","last_became_active":"2023-10-08T00:55:47.528897+0000","last_became_peered":"2023-10-08T00:55:47.528897+0000","last_unstale":"2023-10-08T00:55:47.533963+0000","last_undegraded":"2023-10-08T00:55:47.533963+0000","last_fullsized":"2023-10-08T00:55:47.533963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:10.076808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492782+0000","last_change":"2023-10-08T00:55:46.507942+0000","last_active":"2023-10-08T00:55:47.492782+0000","last_peered":"2023-10-08T00:55:47.492782+0000","last_clean":"2023-10-08T00:55:47.492782+0000","last_became_active":"2023-10-08T00:55:46.507685+0000","last_became_peered":"2023-10-08T00:55:46.507685+0000","last_unstale":"2023-10-08T00:55:47.492782+0000","last_undegraded":"2023-10-08T00:55:47.492782+0000","last_fullsized":"2023-10-08T00:55:47.492782+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530557+0000","last_change":"2023-10-08T00:55:47.521486+0000","last_active":"2023-10-08T00:55:47.530557+0000","last_peered":"2023-10-08T00:55:47.530557+0000","last_clean":"2023-10-08T00:55:47.530557+0000","last_became_active":"2023-10-08T00:55:47.521230+0000","last_became_peered":"2023-10-08T00:55:47.521230+0000","last_unstale":"2023-10-08T00:55:47.530557+0000","last_undegraded":"2023-10-08T00:55:47.530557+0000","last_fullsized":"2023-10-08T00:55:47.530557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:34.078412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.501573+0000","last_change":"2023-10-08T00:55:46.519867+0000","last_active":"2023-10-08T00:55:47.501573+0000","last_peered":"2023-10-08T00:55:47.501573+0000","last_clean":"2023-10-08T00:55:47.501573+0000","last_became_active":"2023-10-08T00:55:46.519268+0000","last_became_peered":"2023-10-08T00:55:46.519268+0000","last_unstale":"2023-10-08T00:55:47.501573+0000","last_undegraded":"2023-10-08T00:55:47.501573+0000","last_fullsized":"2023-10-08T00:55:47.501573+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530667+0000","last_change":"2023-10-08T00:55:47.517018+0000","last_active":"2023-10-08T00:55:47.530667+0000","last_peered":"2023-10-08T00:55:47.530667+0000","last_clean":"2023-10-08T00:55:47.530667+0000","last_became_active":"2023-10-08T00:55:47.516713+0000","last_became_peered":"2023-10-08T00:55:47.516713+0000","last_unstale":"2023-10-08T00:55:47.530667+0000","last_undegraded":"2023-10-08T00:55:47.530667+0000","last_fullsized":"2023-10-08T00:55:47.530667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:40.798392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.507081+0000","last_change":"2023-10-08T00:55:47.500984+0000","last_active":"2023-10-08T00:55:47.507081+0000","last_peered":"2023-10-08T00:55:47.507081+0000","last_clean":"2023-10-08T00:55:47.507081+0000","last_became_active":"2023-10-08T00:55:47.499299+0000","last_became_peered":"2023-10-08T00:55:47.499299+0000","last_unstale":"2023-10-08T00:55:47.507081+0000","last_undegraded":"2023-10-08T00:55:47.507081+0000","last_fullsized":"2023-10-08T00:55:47.507081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.494532+0000","last_change":"2023-10-08T00:55:46.503917+0000","last_active":"2023-10-08T00:55:47.494532+0000","last_peered":"2023-10-08T00:55:47.494532+0000","last_clean":"2023-10-08T00:55:47.494532+0000","last_became_active":"2023-10-08T00:55:46.503620+0000","last_became_peered":"2023-10-08T00:55:46.503620+0000","last_unstale":"2023-10-08T00:55:47.494532+0000","last_undegraded":"2023-10-08T00:55:47.494532+0000","last_fullsized":"2023-10-08T00:55:47.494532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.527104+0000","last_change":"2023-10-08T00:55:47.525523+0000","last_active":"2023-10-08T00:55:47.527104+0000","last_peered":"2023-10-08T00:55:47.527104+0000","last_clean":"2023-10-08T00:55:47.527104+0000","last_became_active":"2023-10-08T00:55:47.525238+0000","last_became_peered":"2023-10-08T00:55:47.525238+0000","last_unstale":"2023-10-08T00:55:47.527104+0000","last_undegraded":"2023-10-08T00:55:47.527104+0000","last_fullsized":"2023-10-08T00:55:47.527104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.498288+0000","last_change":"2023-10-08T00:55:46.516320+0000","last_active":"2023-10-08T00:55:47.498288+0000","last_peered":"2023-10-08T00:55:47.498288+0000","last_clean":"2023-10-08T00:55:47.498288+0000","last_became_active":"2023-10-08T00:55:46.516011+0000","last_became_peered":"2023-10-08T00:55:46.516011+0000","last_unstale":"2023-10-08T00:55:47.498288+0000","last_undegraded":"2023-10-08T00:55:47.498288+0000","last_fullsized":"2023-10-08T00:55:47.498288+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493566+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T00:55:47.493566+0000","last_peered":"2023-10-08T00:55:47.493566+0000","last_clean":"2023-10-08T00:55:47.493566+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T00:55:47.493566+0000","last_undegraded":"2023-10-08T00:55:47.493566+0000","last_fullsized":"2023-10-08T00:55:47.493566+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524595+0000","last_change":"2023-10-08T00:55:47.518197+0000","last_active":"2023-10-08T00:55:47.524595+0000","last_peered":"2023-10-08T00:55:47.524595+0000","last_clean":"2023-10-08T00:55:47.524595+0000","last_became_active":"2023-10-08T00:55:47.517929+0000","last_became_peered":"2023-10-08T00:55:47.517929+0000","last_unstale":"2023-10-08T00:55:47.524595+0000","last_undegraded":"2023-10-08T00:55:47.524595+0000","last_fullsized":"2023-10-08T00:55:47.524595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:55.386688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493737+0000","last_change":"2023-10-08T00:55:46.513277+0000","last_active":"2023-10-08T00:55:47.493737+0000","last_peered":"2023-10-08T00:55:47.493737+0000","last_clean":"2023-10-08T00:55:47.493737+0000","last_became_active":"2023-10-08T00:55:46.513027+0000","last_became_peered":"2023-10-08T00:55:46.513027+0000","last_unstale":"2023-10-08T00:55:47.493737+0000","last_undegraded":"2023-10-08T00:55:47.493737+0000","last_fullsized":"2023-10-08T00:55:47.493737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:15:43.811282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.506236+0000","last_change":"2023-10-08T00:55:46.518643+0000","last_active":"2023-10-08T00:55:47.506236+0000","last_peered":"2023-10-08T00:55:47.506236+0000","last_clean":"2023-10-08T00:55:47.506236+0000","last_became_active":"2023-10-08T00:55:46.518305+0000","last_became_peered":"2023-10-08T00:55:46.518305+0000","last_unstale":"2023-10-08T00:55:47.506236+0000","last_undegraded":"2023-10-08T00:55:47.506236+0000","last_fullsized":"2023-10-08T00:55:47.506236+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.499384+0000","last_change":"2023-10-08T00:55:47.498680+0000","last_active":"2023-10-08T00:55:47.499384+0000","last_peered":"2023-10-08T00:55:47.499384+0000","last_clean":"2023-10-08T00:55:47.499384+0000","last_became_active":"2023-10-08T00:55:47.498370+0000","last_became_peered":"2023-10-08T00:55:47.498370+0000","last_unstale":"2023-10-08T00:55:47.499384+0000","last_undegraded":"2023-10-08T00:55:47.499384+0000","last_fullsized":"2023-10-08T00:55:47.499384+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-08T00:55:49.395102+0000","last_change":"2023-10-08T00:55:46.508309+0000","last_active":"2023-10-08T00:55:49.395102+0000","last_peered":"2023-10-08T00:55:49.395102+0000","last_clean":"2023-10-08T00:55:49.395102+0000","last_became_active":"2023-10-08T00:55:46.507338+0000","last_became_peered":"2023-10-08T00:55:46.507338+0000","last_unstale":"2023-10-08T00:55:49.395102+0000","last_undegraded":"2023-10-08T00:55:49.395102+0000","last_fullsized":"2023-10-08T00:55:49.395102+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:39.791749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518720+0000","last_change":"2023-10-08T00:55:47.516628+0000","last_active":"2023-10-08T00:55:47.518720+0000","last_peered":"2023-10-08T00:55:47.518720+0000","last_clean":"2023-10-08T00:55:47.518720+0000","last_became_active":"2023-10-08T00:55:47.516317+0000","last_became_peered":"2023-10-08T00:55:47.516317+0000","last_unstale":"2023-10-08T00:55:47.518720+0000","last_undegraded":"2023-10-08T00:55:47.518720+0000","last_fullsized":"2023-10-08T00:55:47.518720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:07.510837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526183+0000","last_change":"2023-10-08T00:55:46.511031+0000","last_active":"2023-10-08T00:55:47.526183+0000","last_peered":"2023-10-08T00:55:47.526183+0000","last_clean":"2023-10-08T00:55:47.526183+0000","last_became_active":"2023-10-08T00:55:46.510736+0000","last_became_peered":"2023-10-08T00:55:46.510736+0000","last_unstale":"2023-10-08T00:55:47.526183+0000","last_undegraded":"2023-10-08T00:55:47.526183+0000","last_fullsized":"2023-10-08T00:55:47.526183+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:55:32.036861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.520686+0000","last_change":"2023-10-08T00:55:47.513619+0000","last_active":"2023-10-08T00:55:47.520686+0000","last_peered":"2023-10-08T00:55:47.520686+0000","last_clean":"2023-10-08T00:55:47.520686+0000","last_became_active":"2023-10-08T00:55:47.513202+0000","last_became_peered":"2023-10-08T00:55:47.513202+0000","last_unstale":"2023-10-08T00:55:47.520686+0000","last_undegraded":"2023-10-08T00:55:47.520686+0000","last_fullsized":"2023-10-08T00:55:47.520686+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.493466+0000","last_change":"2023-10-08T00:55:46.517881+0000","last_active":"2023-10-08T00:55:47.493466+0000","last_peered":"2023-10-08T00:55:47.493466+0000","last_clean":"2023-10-08T00:55:47.493466+0000","last_became_active":"2023-10-08T00:55:46.517717+0000","last_became_peered":"2023-10-08T00:55:46.517717+0000","last_unstale":"2023-10-08T00:55:47.493466+0000","last_undegraded":"2023-10-08T00:55:47.493466+0000","last_fullsized":"2023-10-08T00:55:47.493466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:15.507854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.518670+0000","last_change":"2023-10-08T00:55:47.516448+0000","last_active":"2023-10-08T00:55:47.518670+0000","last_peered":"2023-10-08T00:55:47.518670+0000","last_clean":"2023-10-08T00:55:47.518670+0000","last_became_active":"2023-10-08T00:55:47.516204+0000","last_became_peered":"2023-10-08T00:55:47.516204+0000","last_unstale":"2023-10-08T00:55:47.518670+0000","last_undegraded":"2023-10-08T00:55:47.518670+0000","last_fullsized":"2023-10-08T00:55:47.518670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:16.514497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492576+0000","last_change":"2023-10-08T00:55:46.513481+0000","last_active":"2023-10-08T00:55:47.492576+0000","last_peered":"2023-10-08T00:55:47.492576+0000","last_clean":"2023-10-08T00:55:47.492576+0000","last_became_active":"2023-10-08T00:55:46.513158+0000","last_became_peered":"2023-10-08T00:55:46.513158+0000","last_unstale":"2023-10-08T00:55:47.492576+0000","last_undegraded":"2023-10-08T00:55:47.492576+0000","last_fullsized":"2023-10-08T00:55:47.492576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524756+0000","last_change":"2023-10-08T00:55:47.520913+0000","last_active":"2023-10-08T00:55:47.524756+0000","last_peered":"2023-10-08T00:55:47.524756+0000","last_clean":"2023-10-08T00:55:47.524756+0000","last_became_active":"2023-10-08T00:55:47.520660+0000","last_became_peered":"2023-10-08T00:55:47.520660+0000","last_unstale":"2023-10-08T00:55:47.524756+0000","last_undegraded":"2023-10-08T00:55:47.524756+0000","last_fullsized":"2023-10-08T00:55:47.524756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.492624+0000","last_change":"2023-10-08T00:55:46.508048+0000","last_active":"2023-10-08T00:55:47.492624+0000","last_peered":"2023-10-08T00:55:47.492624+0000","last_clean":"2023-10-08T00:55:47.492624+0000","last_became_active":"2023-10-08T00:55:46.507778+0000","last_became_peered":"2023-10-08T00:55:46.507778+0000","last_unstale":"2023-10-08T00:55:47.492624+0000","last_undegraded":"2023-10-08T00:55:47.492624+0000","last_fullsized":"2023-10-08T00:55:47.492624+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:08.255151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526130+0000","last_change":"2023-10-08T00:55:47.524176+0000","last_active":"2023-10-08T00:55:47.526130+0000","last_peered":"2023-10-08T00:55:47.526130+0000","last_clean":"2023-10-08T00:55:47.526130+0000","last_became_active":"2023-10-08T00:55:47.523813+0000","last_became_peered":"2023-10-08T00:55:47.523813+0000","last_unstale":"2023-10-08T00:55:47.526130+0000","last_undegraded":"2023-10-08T00:55:47.526130+0000","last_fullsized":"2023-10-08T00:55:47.526130+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.534398+0000","last_change":"2023-10-08T00:55:46.514845+0000","last_active":"2023-10-08T00:55:47.534398+0000","last_peered":"2023-10-08T00:55:47.534398+0000","last_clean":"2023-10-08T00:55:47.534398+0000","last_became_active":"2023-10-08T00:55:46.514581+0000","last_became_peered":"2023-10-08T00:55:46.514581+0000","last_unstale":"2023-10-08T00:55:47.534398+0000","last_undegraded":"2023-10-08T00:55:47.534398+0000","last_fullsized":"2023-10-08T00:55:47.534398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T00:59:02.769491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526325+0000","last_change":"2023-10-08T00:55:47.522621+0000","last_active":"2023-10-08T00:55:47.526325+0000","last_peered":"2023-10-08T00:55:47.526325+0000","last_clean":"2023-10-08T00:55:47.526325+0000","last_became_active":"2023-10-08T00:55:47.522352+0000","last_became_peered":"2023-10-08T00:55:47.522352+0000","last_unstale":"2023-10-08T00:55:47.526325+0000","last_undegraded":"2023-10-08T00:55:47.526325+0000","last_fullsized":"2023-10-08T00:55:47.526325+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:33.582238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.494485+0000","last_change":"2023-10-08T00:55:46.520580+0000","last_active":"2023-10-08T00:55:47.494485+0000","last_peered":"2023-10-08T00:55:47.494485+0000","last_clean":"2023-10-08T00:55:47.494485+0000","last_became_active":"2023-10-08T00:55:46.520260+0000","last_became_peered":"2023-10-08T00:55:46.520260+0000","last_unstale":"2023-10-08T00:55:47.494485+0000","last_undegraded":"2023-10-08T00:55:47.494485+0000","last_fullsized":"2023-10-08T00:55:47.494485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:05:35.102973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524951+0000","last_change":"2023-10-08T00:55:47.519077+0000","last_active":"2023-10-08T00:55:47.524951+0000","last_peered":"2023-10-08T00:55:47.524951+0000","last_clean":"2023-10-08T00:55:47.524951+0000","last_became_active":"2023-10-08T00:55:47.518580+0000","last_became_peered":"2023-10-08T00:55:47.518580+0000","last_unstale":"2023-10-08T00:55:47.524951+0000","last_undegraded":"2023-10-08T00:55:47.524951+0000","last_fullsized":"2023-10-08T00:55:47.524951+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:14.159608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526450+0000","last_change":"2023-10-08T00:55:46.514711+0000","last_active":"2023-10-08T00:55:47.526450+0000","last_peered":"2023-10-08T00:55:47.526450+0000","last_clean":"2023-10-08T00:55:47.526450+0000","last_became_active":"2023-10-08T00:55:46.514420+0000","last_became_peered":"2023-10-08T00:55:46.514420+0000","last_unstale":"2023-10-08T00:55:47.526450+0000","last_undegraded":"2023-10-08T00:55:47.526450+0000","last_fullsized":"2023-10-08T00:55:47.526450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:15.507854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525520+0000","last_change":"2023-10-08T00:55:47.520974+0000","last_active":"2023-10-08T00:55:47.525520+0000","last_peered":"2023-10-08T00:55:47.525520+0000","last_clean":"2023-10-08T00:55:47.525520+0000","last_became_active":"2023-10-08T00:55:47.520259+0000","last_became_peered":"2023-10-08T00:55:47.520259+0000","last_unstale":"2023-10-08T00:55:47.525520+0000","last_undegraded":"2023-10-08T00:55:47.525520+0000","last_fullsized":"2023-10-08T00:55:47.525520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:57:57.391083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.510639+0000","last_change":"2023-10-08T00:55:46.500260+0000","last_active":"2023-10-08T00:55:47.510639+0000","last_peered":"2023-10-08T00:55:47.510639+0000","last_clean":"2023-10-08T00:55:47.510639+0000","last_became_active":"2023-10-08T00:55:46.499965+0000","last_became_peered":"2023-10-08T00:55:46.499965+0000","last_unstale":"2023-10-08T00:55:47.510639+0000","last_undegraded":"2023-10-08T00:55:47.510639+0000","last_fullsized":"2023-10-08T00:55:47.510639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:45.471023+0000","last_clean_scrub_stamp":"2023-10-08T00:55:45.471023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:27.668192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.533942+0000","last_change":"2023-10-08T00:55:47.529127+0000","last_active":"2023-10-08T00:55:47.533942+0000","last_peered":"2023-10-08T00:55:47.533942+0000","last_clean":"2023-10-08T00:55:47.533942+0000","last_became_active":"2023-10-08T00:55:47.528809+0000","last_became_peered":"2023-10-08T00:55:47.528809+0000","last_unstale":"2023-10-08T00:55:47.533942+0000","last_undegraded":"2023-10-08T00:55:47.533942+0000","last_fullsized":"2023-10-08T00:55:47.533942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:54:07.821288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526404+0000","last_change":"2023-10-08T00:55:47.522934+0000","last_active":"2023-10-08T00:55:47.526404+0000","last_peered":"2023-10-08T00:55:47.526404+0000","last_clean":"2023-10-08T00:55:47.526404+0000","last_became_active":"2023-10-08T00:55:47.522385+0000","last_became_peered":"2023-10-08T00:55:47.522385+0000","last_unstale":"2023-10-08T00:55:47.526404+0000","last_undegraded":"2023-10-08T00:55:47.526404+0000","last_fullsized":"2023-10-08T00:55:47.526404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:53:58.740974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.515109+0000","last_change":"2023-10-08T00:55:47.508997+0000","last_active":"2023-10-08T00:55:47.515109+0000","last_peered":"2023-10-08T00:55:47.515109+0000","last_clean":"2023-10-08T00:55:47.515109+0000","last_became_active":"2023-10-08T00:55:47.508483+0000","last_became_peered":"2023-10-08T00:55:47.508483+0000","last_unstale":"2023-10-08T00:55:47.515109+0000","last_undegraded":"2023-10-08T00:55:47.515109+0000","last_fullsized":"2023-10-08T00:55:47.515109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:25.085209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525515+0000","last_change":"2023-10-08T00:55:47.520643+0000","last_active":"2023-10-08T00:55:47.525515+0000","last_peered":"2023-10-08T00:55:47.525515+0000","last_clean":"2023-10-08T00:55:47.525515+0000","last_became_active":"2023-10-08T00:55:47.520385+0000","last_became_peered":"2023-10-08T00:55:47.520385+0000","last_unstale":"2023-10-08T00:55:47.525515+0000","last_undegraded":"2023-10-08T00:55:47.525515+0000","last_fullsized":"2023-10-08T00:55:47.525515+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:04:02.453741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.524523+0000","last_change":"2023-10-08T00:55:47.517588+0000","last_active":"2023-10-08T00:55:47.524523+0000","last_peered":"2023-10-08T00:55:47.524523+0000","last_clean":"2023-10-08T00:55:47.524523+0000","last_became_active":"2023-10-08T00:55:47.517257+0000","last_became_peered":"2023-10-08T00:55:47.517257+0000","last_unstale":"2023-10-08T00:55:47.524523+0000","last_undegraded":"2023-10-08T00:55:47.524523+0000","last_fullsized":"2023-10-08T00:55:47.524523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536372+0000","last_change":"2023-10-08T00:55:47.534943+0000","last_active":"2023-10-08T00:55:47.536372+0000","last_peered":"2023-10-08T00:55:47.536372+0000","last_clean":"2023-10-08T00:55:47.536372+0000","last_became_active":"2023-10-08T00:55:47.534593+0000","last_became_peered":"2023-10-08T00:55:47.534593+0000","last_unstale":"2023-10-08T00:55:47.536372+0000","last_undegraded":"2023-10-08T00:55:47.536372+0000","last_fullsized":"2023-10-08T00:55:47.536372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523043+0000","last_change":"2023-10-08T00:55:47.519521+0000","last_active":"2023-10-08T00:55:47.523043+0000","last_peered":"2023-10-08T00:55:47.523043+0000","last_clean":"2023-10-08T00:55:47.523043+0000","last_became_active":"2023-10-08T00:55:47.519205+0000","last_became_peered":"2023-10-08T00:55:47.519205+0000","last_unstale":"2023-10-08T00:55:47.523043+0000","last_undegraded":"2023-10-08T00:55:47.523043+0000","last_fullsized":"2023-10-08T00:55:47.523043+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:59.370630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.528710+0000","last_change":"2023-10-08T00:55:47.525909+0000","last_active":"2023-10-08T00:55:47.528710+0000","last_peered":"2023-10-08T00:55:47.528710+0000","last_clean":"2023-10-08T00:55:47.528710+0000","last_became_active":"2023-10-08T00:55:47.525638+0000","last_became_peered":"2023-10-08T00:55:47.525638+0000","last_unstale":"2023-10-08T00:55:47.528710+0000","last_undegraded":"2023-10-08T00:55:47.528710+0000","last_fullsized":"2023-10-08T00:55:47.528710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:22.912453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536365+0000","last_change":"2023-10-08T00:55:47.534909+0000","last_active":"2023-10-08T00:55:47.536365+0000","last_peered":"2023-10-08T00:55:47.536365+0000","last_clean":"2023-10-08T00:55:47.536365+0000","last_became_active":"2023-10-08T00:55:47.534654+0000","last_became_peered":"2023-10-08T00:55:47.534654+0000","last_unstale":"2023-10-08T00:55:47.536365+0000","last_undegraded":"2023-10-08T00:55:47.536365+0000","last_fullsized":"2023-10-08T00:55:47.536365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:45:39.215552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.525191+0000","last_change":"2023-10-08T00:55:47.520067+0000","last_active":"2023-10-08T00:55:47.525191+0000","last_peered":"2023-10-08T00:55:47.525191+0000","last_clean":"2023-10-08T00:55:47.525191+0000","last_became_active":"2023-10-08T00:55:47.519793+0000","last_became_peered":"2023-10-08T00:55:47.519793+0000","last_unstale":"2023-10-08T00:55:47.525191+0000","last_undegraded":"2023-10-08T00:55:47.525191+0000","last_fullsized":"2023-10-08T00:55:47.525191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:08:34.451497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.536355+0000","last_change":"2023-10-08T00:55:47.534891+0000","last_active":"2023-10-08T00:55:47.536355+0000","last_peered":"2023-10-08T00:55:47.536355+0000","last_clean":"2023-10-08T00:55:47.536355+0000","last_became_active":"2023-10-08T00:55:47.534589+0000","last_became_peered":"2023-10-08T00:55:47.534589+0000","last_unstale":"2023-10-08T00:55:47.536355+0000","last_undegraded":"2023-10-08T00:55:47.536355+0000","last_fullsized":"2023-10-08T00:55:47.536355+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:35:38.560131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.530986+0000","last_change":"2023-10-08T00:55:47.524426+0000","last_active":"2023-10-08T00:55:47.530986+0000","last_peered":"2023-10-08T00:55:47.530986+0000","last_clean":"2023-10-08T00:55:47.530986+0000","last_became_active":"2023-10-08T00:55:47.524142+0000","last_became_peered":"2023-10-08T00:55:47.524142+0000","last_unstale":"2023-10-08T00:55:47.530986+0000","last_undegraded":"2023-10-08T00:55:47.530986+0000","last_fullsized":"2023-10-08T00:55:47.530986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:13:19.461561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.523999+0000","last_change":"2023-10-08T00:55:47.515215+0000","last_active":"2023-10-08T00:55:47.523999+0000","last_peered":"2023-10-08T00:55:47.523999+0000","last_clean":"2023-10-08T00:55:47.523999+0000","last_became_active":"2023-10-08T00:55:47.514978+0000","last_became_peered":"2023-10-08T00:55:47.514978+0000","last_unstale":"2023-10-08T00:55:47.523999+0000","last_undegraded":"2023-10-08T00:55:47.523999+0000","last_fullsized":"2023-10-08T00:55:47.523999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-08T00:55:47.526430+0000","last_change":"2023-10-08T00:55:47.523078+0000","last_active":"2023-10-08T00:55:47.526430+0000","last_peered":"2023-10-08T00:55:47.526430+0000","last_clean":"2023-10-08T00:55:47.526430+0000","last_became_active":"2023-10-08T00:55:47.522818+0000","last_became_peered":"2023-10-08T00:55:47.522818+0000","last_unstale":"2023-10-08T00:55:47.526430+0000","last_undegraded":"2023-10-08T00:55:47.526430+0000","last_fullsized":"2023-10-08T00:55:47.526430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:46.477666+0000","last_clean_scrub_stamp":"2023-10-08T00:55:46.477666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:40:31.755599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298072,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073768,"statfs":{"total":96636764160,"available":96331538432,"internally_reserved":0,"allocated":802816,"data_stored":661576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89400000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92500000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.069}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297496,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074344,"statfs":{"total":96636764160,"available":96332128256,"internally_reserved":0,"allocated":212992,"data_stored":70814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298080,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073760,"statfs":{"total":96636764160,"available":96331530240,"internally_reserved":0,"allocated":811008,"data_stored":661632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70099999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":204800,"data_stored":70654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":72190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.621}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297500,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074340,"statfs":{"total":96636764160,"available":96332124160,"internally_reserved":0,"allocated":217088,"data_stored":70780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":208896,"data_stored":70744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78100000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T00:55:57.736 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T00:55:57.736 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-08T00:55:57.736 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-08T00:55:57.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-08T00:55:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.949+0000 7fa1e759d700 1 Processor -- start 2023-10-08T00:55:57.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.950+0000 7fa1e759d700 1 -- start start 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e759d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1e01594d0 0x7fa1e006f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e759d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1e00cd950 0x7fa1e00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e759d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e00ce1f0 con 0x7fa1e01598a0 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e759d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e00cdf10 con 0x7fa1e00cd950 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e759d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e006e6a0 con 0x7fa1e01594d0 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e5d9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa1e01598a0 0x7fa1e00b2c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54314/0 (socket says 172.21.15.119:54314) 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e5d9a700 1 -- 172.21.15.119:0/1629134650 learned_addr learned my addr 172.21.15.119:0/1629134650 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e6d9c700 1 --2- 172.21.15.119:0/1629134650 >> v2:172.21.15.119:3300/0 conn(0x7fa1e00cd950 0x7fa1e00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e659b700 1 --2- 172.21.15.119:0/1629134650 >> v2:172.21.15.184:3300/0 conn(0x7fa1e01594d0 0x7fa1e006f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e5599700 1 -- 172.21.15.119:0/1629134650 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1222138905 0 0) 0x7fa1e00ce1f0 con 0x7fa1e01598a0 2023-10-08T00:55:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.951+0000 7fa1e5599700 1 -- 172.21.15.119:0/1629134650 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1cc003570 con 0x7fa1e01598a0 2023-10-08T00:55:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.952+0000 7fa1e6d9c700 1 -- 172.21.15.119:0/1629134650 >> v2:172.21.15.184:3300/0 conn(0x7fa1e01594d0 msgr2=0x7fa1e006f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.952+0000 7fa1e6d9c700 1 --2- 172.21.15.119:0/1629134650 >> v2:172.21.15.184:3300/0 conn(0x7fa1e01594d0 0x7fa1e006f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.952+0000 7fa1e6d9c700 1 -- 172.21.15.119:0/1629134650 >> v1:172.21.15.119:6789/0 conn(0x7fa1e01598a0 legacy=0x7fa1e00b2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.952+0000 7fa1e6d9c700 1 -- 172.21.15.119:0/1629134650 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e006e980 con 0x7fa1e00cd950 2023-10-08T00:55:57.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.952+0000 7fa1e6d9c700 1 --2- 172.21.15.119:0/1629134650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e00cd950 0x7fa1e00aede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00afeb0 tx=0x7fa1d8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea543448d343fa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.952+0000 7fa1e5599700 1 -- 172.21.15.119:0/1629134650 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800e070 con 0x7fa1e00cd950 2023-10-08T00:55:57.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.952+0000 7fa1e5599700 1 -- 172.21.15.119:0/1629134650 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d800bab0 con 0x7fa1e00cd950 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.952+0000 7fa1e5599700 1 -- 172.21.15.119:0/1629134650 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d800be70 con 0x7fa1e00cd950 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.953+0000 7fa1e759d700 1 -- 172.21.15.119:0/1629134650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e00cd950 msgr2=0x7fa1e00aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.953+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1629134650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e00cd950 0x7fa1e00aede0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00afeb0 tx=0x7fa1d8009780 comp rx=0 tx=0).stop 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.953+0000 7fa1e759d700 1 -- 172.21.15.119:0/1629134650 shutdown_connections 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.953+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1629134650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e00cd950 0x7fa1e00aede0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.953+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1629134650 >> v2:172.21.15.184:3300/0 conn(0x7fa1e01594d0 0x7fa1e006f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.953+0000 7fa1e759d700 1 -- 172.21.15.119:0/1629134650 >> 172.21.15.119:0/1629134650 conn(0x7fa1e00694a0 msgr2=0x7fa1e00698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.953+0000 7fa1e759d700 1 -- 172.21.15.119:0/1629134650 shutdown_connections 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.953+0000 7fa1e759d700 1 -- 172.21.15.119:0/1629134650 wait complete. 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.954+0000 7fa1e759d700 1 Processor -- start 2023-10-08T00:55:57.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.954+0000 7fa1e759d700 1 -- start start 2023-10-08T00:55:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.955+0000 7fa1e759d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00cd950 0x7fa1e0151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.955+0000 7fa1e759d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e01594d0 0x7fa1e0155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.955+0000 7fa1e759d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e01598a0 0x7fa1e015a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.955+0000 7fa1e759d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0156270 con 0x7fa1e01594d0 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.955+0000 7fa1e759d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0156550 con 0x7fa1e00cd950 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.955+0000 7fa1e759d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0156830 con 0x7fa1e01598a0 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e01594d0 0x7fa1e0155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e01594d0 0x7fa1e0155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33150/0 (socket says 172.21.15.119:33150) 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 -- 172.21.15.119:0/1194622546 learned_addr learned my addr 172.21.15.119:0/1194622546 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e6d9c700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e01598a0 0x7fa1e015a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e659b700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00cd950 0x7fa1e0151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 -- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e01598a0 msgr2=0x7fa1e015a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e01598a0 0x7fa1e015a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 -- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00cd950 msgr2=0x7fa1e0151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00cd950 0x7fa1e0151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:57.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 -- 172.21.15.119:0/1194622546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8009a20 con 0x7fa1e01594d0 2023-10-08T00:55:57.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e6d9c700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e01598a0 0x7fa1e015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:57.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e659b700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00cd950 0x7fa1e0151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:57.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e5d9a700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e01594d0 0x7fa1e0155900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c7c0 tx=0x7fa1d0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1d67fc700 1 -- 172.21.15.119:0/1194622546 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0005b30 con 0x7fa1e01594d0 2023-10-08T00:55:57.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1d67fc700 1 -- 172.21.15.119:0/1194622546 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d0005c90 con 0x7fa1e01594d0 2023-10-08T00:55:57.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1d67fc700 1 -- 172.21.15.119:0/1194622546 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0007340 con 0x7fa1e01594d0 2023-10-08T00:55:57.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.956+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e016a390 con 0x7fa1e01594d0 2023-10-08T00:55:57.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.957+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e016a080 con 0x7fa1e01594d0 2023-10-08T00:55:57.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.958+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e006d560 con 0x7fa1e01594d0 2023-10-08T00:55:57.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.958+0000 7fa1d67fc700 1 -- 172.21.15.119:0/1194622546 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa1d001b400 con 0x7fa1e01594d0 2023-10-08T00:55:57.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.959+0000 7fa1d67fc700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:57.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.959+0000 7fa1d67fc700 1 -- 172.21.15.119:0/1194622546 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1d00840e0 con 0x7fa1e01594d0 2023-10-08T00:55:57.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.959+0000 7fa1e659b700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:57.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.960+0000 7fa1e659b700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1c4066f30 0x7fa1c40693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc006f00 tx=0x7fa1dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:57.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:57.963+0000 7fa1d67fc700 1 -- 172.21.15.119:0/1194622546 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1d0051330 con 0x7fa1e01594d0 2023-10-08T00:55:58.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.197+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fa1e0069bc0 con 0x7fa1e01594d0 2023-10-08T00:55:58.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.198+0000 7fa1d67fc700 1 -- 172.21.15.119:0/1194622546 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fa1d0054950 con 0x7fa1e01594d0 2023-10-08T00:55:58.199 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:58.199 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-08T00:55:58.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1c4066f30 msgr2=0x7fa1c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1c4066f30 0x7fa1c40693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc006f00 tx=0x7fa1dc008040 comp rx=0 tx=0).stop 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e01594d0 msgr2=0x7fa1e0155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e01594d0 0x7fa1e0155900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa1d000c7c0 tx=0x7fa1d0007e00 comp rx=0 tx=0).stop 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 shutdown_connections 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1e01598a0 0x7fa1e015a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1c4066f30 0x7fa1c40693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1e00cd950 0x7fa1e0151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 --2- 172.21.15.119:0/1194622546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1e01594d0 0x7fa1e0155900 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 >> 172.21.15.119:0/1194622546 conn(0x7fa1e00694a0 msgr2=0x7fa1e00ade40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 shutdown_connections 2023-10-08T00:55:58.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.200+0000 7fa1e759d700 1 -- 172.21.15.119:0/1194622546 wait complete. 2023-10-08T00:55:58.219 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-08T00:55:58.219 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:55:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.397+0000 7f538a3ff700 1 Processor -- start 2023-10-08T00:55:58.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.397+0000 7f538a3ff700 1 -- start start 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f538a3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53841578b0 0x7f5384155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f538a3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f53840af2f0 0x7f53840af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f538a3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53840ad930 con 0x7f5384155c10 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f538a3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53840b0c80 con 0x7f53841578b0 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f538a3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53840acd20 con 0x7f53840af2f0 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f53837fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53841578b0 0x7f5384155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f53837fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f53841578b0 0x7f5384155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33158/0 (socket says 172.21.15.119:33158) 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f53837fe700 1 -- 172.21.15.119:0/3931620664 learned_addr learned my addr 172.21.15.119:0/3931620664 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:58.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f538899c700 1 --2- 172.21.15.119:0/3931620664 >> v2:172.21.15.184:3300/0 conn(0x7f53840af2f0 0x7f53840af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f53837fe700 1 -- 172.21.15.119:0/3931620664 >> v2:172.21.15.184:3300/0 conn(0x7f53840af2f0 msgr2=0x7f53840af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f53837fe700 1 --2- 172.21.15.119:0/3931620664 >> v2:172.21.15.184:3300/0 conn(0x7f53840af2f0 0x7f53840af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f53837fe700 1 -- 172.21.15.119:0/3931620664 >> v1:172.21.15.184:6789/0 conn(0x7f5384155c10 legacy=0x7f53840ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:55:58.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.399+0000 7f53837fe700 1 -- 172.21.15.119:0/3931620664 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53840ad000 con 0x7f53841578b0 2023-10-08T00:55:58.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.400+0000 7f53837fe700 1 --2- 172.21.15.119:0/3931620664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53841578b0 0x7f5384155fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5384065be0 tx=0x7f537800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919ade0956656c25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.400+0000 7f5382ffd700 1 -- 172.21.15.119:0/3931620664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537800dc50 con 0x7f53841578b0 2023-10-08T00:55:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.400+0000 7f5382ffd700 1 -- 172.21.15.119:0/3931620664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537800ddb0 con 0x7f53841578b0 2023-10-08T00:55:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.400+0000 7f5382ffd700 1 -- 172.21.15.119:0/3931620664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780196d0 con 0x7f53841578b0 2023-10-08T00:55:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.400+0000 7f538a3ff700 1 -- 172.21.15.119:0/3931620664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53841578b0 msgr2=0x7f5384155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.400+0000 7f538a3ff700 1 --2- 172.21.15.119:0/3931620664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53841578b0 0x7f5384155fe0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5384065be0 tx=0x7f537800b4f0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.401+0000 7f538a3ff700 1 -- 172.21.15.119:0/3931620664 shutdown_connections 2023-10-08T00:55:58.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.401+0000 7f538a3ff700 1 --2- 172.21.15.119:0/3931620664 >> v2:172.21.15.184:3300/0 conn(0x7f53840af2f0 0x7f53840af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.401+0000 7f538a3ff700 1 --2- 172.21.15.119:0/3931620664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f53841578b0 0x7f5384155fe0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.401+0000 7f538a3ff700 1 -- 172.21.15.119:0/3931620664 >> 172.21.15.119:0/3931620664 conn(0x7f538406a320 msgr2=0x7f53840cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.401+0000 7f538a3ff700 1 -- 172.21.15.119:0/3931620664 shutdown_connections 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.401+0000 7f538a3ff700 1 -- 172.21.15.119:0/3931620664 wait complete. 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.402+0000 7f538a3ff700 1 Processor -- start 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.402+0000 7f538a3ff700 1 -- start start 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f538a3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53840af2f0 0x7f5384144040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f538a3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5384155c10 0x7f53840a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f538a3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53841578b0 0x7f53840a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f538a3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5384144580 con 0x7f5384155c10 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f538a3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538415d700 con 0x7f53840af2f0 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f538a3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538415d9a0 con 0x7f53841578b0 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f53837fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5384155c10 0x7f53840a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f53837fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5384155c10 0x7f53840a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33164/0 (socket says 172.21.15.119:33164) 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.403+0000 7f53837fe700 1 -- 172.21.15.119:0/555579646 learned_addr learned my addr 172.21.15.119:0/555579646 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f538899c700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53841578b0 0x7f53840a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f5383fff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53840af2f0 0x7f5384144040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f53837fe700 1 -- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53841578b0 msgr2=0x7f53840a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f53837fe700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53841578b0 0x7f53840a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f53837fe700 1 -- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53840af2f0 msgr2=0x7f5384144040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f53837fe700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53840af2f0 0x7f5384144040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f53837fe700 1 -- 172.21.15.119:0/555579646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537800b790 con 0x7f5384155c10 2023-10-08T00:55:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f5383fff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53840af2f0 0x7f5384144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:55:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f53837fe700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5384155c10 0x7f53840a5ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5384064290 tx=0x7f537801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f536bfff700 1 -- 172.21.15.119:0/555579646 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378010750 con 0x7f5384155c10 2023-10-08T00:55:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f536bfff700 1 -- 172.21.15.119:0/555579646 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53780108b0 con 0x7f5384155c10 2023-10-08T00:55:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f536bfff700 1 -- 172.21.15.119:0/555579646 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378010c70 con 0x7f5384155c10 2023-10-08T00:55:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538415dc80 con 0x7f5384155c10 2023-10-08T00:55:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.404+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53840a67d0 con 0x7f5384155c10 2023-10-08T00:55:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.405+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538415e1a0 con 0x7f5384155c10 2023-10-08T00:55:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.406+0000 7f536bfff700 1 -- 172.21.15.119:0/555579646 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f537802c430 con 0x7f5384155c10 2023-10-08T00:55:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.407+0000 7f536bfff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5370066f30 0x7f53700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.407+0000 7f536bfff700 1 -- 172.21.15.119:0/555579646 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5378014070 con 0x7f5384155c10 2023-10-08T00:55:58.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.407+0000 7f5383fff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5370066f30 0x7f53700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.408+0000 7f5383fff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5370066f30 0x7f53700693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5374001840 tx=0x7f5374009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:58.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.411+0000 7f536bfff700 1 -- 172.21.15.119:0/555579646 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f538415e1a0 con 0x7f5384155c10 2023-10-08T00:55:58.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.603+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f538415e1a0 con 0x7f5384155c10 2023-10-08T00:55:58.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.604+0000 7f536bfff700 1 -- 172.21.15.119:0/555579646 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f538415e1a0 con 0x7f5384155c10 2023-10-08T00:55:58.605 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:55:58.606 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:55:58.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5370066f30 msgr2=0x7f53700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5370066f30 0x7f53700693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5374001840 tx=0x7f5374009040 comp rx=0 tx=0).stop 2023-10-08T00:55:58.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5384155c10 msgr2=0x7f53840a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5384155c10 0x7f53840a5ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5384064290 tx=0x7f537801b000 comp rx=0 tx=0).stop 2023-10-08T00:55:58.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 shutdown_connections 2023-10-08T00:55:58.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53841578b0 0x7f53840a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5370066f30 0x7f53700693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f53840af2f0 0x7f5384144040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 --2- 172.21.15.119:0/555579646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5384155c10 0x7f53840a5ed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.606+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 >> 172.21.15.119:0/555579646 conn(0x7f538406a320 msgr2=0x7f5384154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:58.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.607+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 shutdown_connections 2023-10-08T00:55:58.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.607+0000 7f538a3ff700 1 -- 172.21.15.119:0/555579646 wait complete. 2023-10-08T00:55:58.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:55:45.270364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:55:45.937180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:55:58.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:55:58.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.808+0000 7fd61919d700 1 Processor -- start 2023-10-08T00:55:58.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.808+0000 7fd61919d700 1 -- start start 2023-10-08T00:55:58.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.809+0000 7fd61919d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd614159a60 0x7fd6140c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.809+0000 7fd61919d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6140c9cb0 con 0x7fd6140cb230 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.809+0000 7fd61919d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6140c9f30 con 0x7fd6140ccc00 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.809+0000 7fd61919d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61406dfc0 con 0x7fd614159a60 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.809+0000 7fd613fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd614159a60 0x7fd6140c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.809+0000 7fd61899c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd6140cb230 0x7fd61406d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54322/0 (socket says 172.21.15.119:54322) 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.809+0000 7fd61899c700 1 -- 172.21.15.119:0/271930349 learned_addr learned my addr 172.21.15.119:0/271930349 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.809+0000 7fd613fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd614159a60 0x7fd6140c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33178/0 (socket says 172.21.15.119:33178) 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.810+0000 7fd613fff700 1 -- 172.21.15.119:0/271930349 >> v1:172.21.15.184:6789/0 conn(0x7fd6140ccc00 legacy=0x7fd614069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.810+0000 7fd613fff700 1 -- 172.21.15.119:0/271930349 >> v1:172.21.15.119:6789/0 conn(0x7fd6140cb230 legacy=0x7fd61406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.810+0000 7fd613fff700 1 -- 172.21.15.119:0/271930349 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6140ca470 con 0x7fd614159a60 2023-10-08T00:55:58.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.810+0000 7fd613fff700 1 --2- 172.21.15.119:0/271930349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd614159a60 0x7fd6140c9770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd604009e60 tx=0x7fd604009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116cbee52f1eb0b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:58.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.810+0000 7fd612ffd700 1 -- 172.21.15.119:0/271930349 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60400e070 con 0x7fd614159a60 2023-10-08T00:55:58.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.810+0000 7fd612ffd700 1 -- 172.21.15.119:0/271930349 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60400bcc0 con 0x7fd614159a60 2023-10-08T00:55:58.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.810+0000 7fd612ffd700 1 -- 172.21.15.119:0/271930349 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604012680 con 0x7fd614159a60 2023-10-08T00:55:58.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.811+0000 7fd61919d700 1 -- 172.21.15.119:0/271930349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd614159a60 msgr2=0x7fd6140c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.811+0000 7fd61919d700 1 --2- 172.21.15.119:0/271930349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd614159a60 0x7fd6140c9770 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd604009e60 tx=0x7fd604009780 comp rx=0 tx=0).stop 2023-10-08T00:55:58.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.811+0000 7fd61919d700 1 -- 172.21.15.119:0/271930349 shutdown_connections 2023-10-08T00:55:58.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.811+0000 7fd61919d700 1 --2- 172.21.15.119:0/271930349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd614159a60 0x7fd6140c9770 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.811+0000 7fd61919d700 1 -- 172.21.15.119:0/271930349 >> 172.21.15.119:0/271930349 conn(0x7fd61406baa0 msgr2=0x7fd61406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:58.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.811+0000 7fd61919d700 1 -- 172.21.15.119:0/271930349 shutdown_connections 2023-10-08T00:55:58.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.811+0000 7fd61919d700 1 -- 172.21.15.119:0/271930349 wait complete. 2023-10-08T00:55:58.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.812+0000 7fd61919d700 1 Processor -- start 2023-10-08T00:55:58.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.813+0000 7fd61919d700 1 -- start start 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.813+0000 7fd61919d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140cb230 0x7fd614153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.813+0000 7fd61919d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6140ccc00 0x7fd6140b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.813+0000 7fd61919d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd614159a60 0x7fd61416a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.813+0000 7fd61919d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b1430 con 0x7fd6140cb230 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.813+0000 7fd61919d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b1680 con 0x7fd6140ccc00 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.813+0000 7fd61919d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6140b1960 con 0x7fd614159a60 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140cb230 0x7fd614153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140cb230 0x7fd614153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33192/0 (socket says 172.21.15.119:33192) 2023-10-08T00:55:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 -- 172.21.15.119:0/2561360056 learned_addr learned my addr 172.21.15.119:0/2561360056 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd61899c700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd614159a60 0x7fd61416a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd6137fe700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6140ccc00 0x7fd6140b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 -- 172.21.15.119:0/2561360056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd614159a60 msgr2=0x7fd61416a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd614159a60 0x7fd61416a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 -- 172.21.15.119:0/2561360056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6140ccc00 msgr2=0x7fd6140b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6140ccc00 0x7fd6140b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 -- 172.21.15.119:0/2561360056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd604009a20 con 0x7fd6140cb230 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.814+0000 7fd613fff700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140cb230 0x7fd614153290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6040097c0 tx=0x7fd604019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.815+0000 7fd5fbfff700 1 -- 172.21.15.119:0/2561360056 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604027470 con 0x7fd6140cb230 2023-10-08T00:55:58.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.815+0000 7fd5fbfff700 1 -- 172.21.15.119:0/2561360056 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6040275d0 con 0x7fd6140cb230 2023-10-08T00:55:58.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.815+0000 7fd5fbfff700 1 -- 172.21.15.119:0/2561360056 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd604027990 con 0x7fd6140cb230 2023-10-08T00:55:58.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.815+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6140b1c40 con 0x7fd6140cb230 2023-10-08T00:55:58.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.815+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6141537d0 con 0x7fd6140cb230 2023-10-08T00:55:58.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.816+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6140b2140 con 0x7fd6140cb230 2023-10-08T00:55:58.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.816+0000 7fd5fbfff700 1 -- 172.21.15.119:0/2561360056 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd604024070 con 0x7fd6140cb230 2023-10-08T00:55:58.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.817+0000 7fd5fbfff700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd600066f30 0x7fd6000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:58.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.817+0000 7fd5fbfff700 1 -- 172.21.15.119:0/2561360056 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd604087070 con 0x7fd6140cb230 2023-10-08T00:55:58.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.818+0000 7fd6137fe700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd600066f30 0x7fd6000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:58.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.818+0000 7fd6137fe700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd600066f30 0x7fd6000693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd61415ad60 tx=0x7fd608009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:58.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:58.821+0000 7fd5fbfff700 1 -- 172.21.15.119:0/2561360056 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6040b85a0 con 0x7fd6140cb230 2023-10-08T00:55:59.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.012+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6140ca750 con 0x7fd6140cb230 2023-10-08T00:55:59.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.013+0000 7fd5fbfff700 1 -- 172.21.15.119:0/2561360056 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd604054240 con 0x7fd6140cb230 2023-10-08T00:55:59.014 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd600066f30 msgr2=0x7fd6000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd600066f30 0x7fd6000693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd61415ad60 tx=0x7fd608009040 comp rx=0 tx=0).stop 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140cb230 msgr2=0x7fd614153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140cb230 0x7fd614153290 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd6040097c0 tx=0x7fd604019000 comp rx=0 tx=0).stop 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 shutdown_connections 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd614159a60 0x7fd61416a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd600066f30 0x7fd6000693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6140cb230 0x7fd614153290 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 --2- 172.21.15.119:0/2561360056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6140ccc00 0x7fd6140b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 >> 172.21.15.119:0/2561360056 conn(0x7fd61406baa0 msgr2=0x7fd6140ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:59.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 shutdown_connections 2023-10-08T00:55:59.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.015+0000 7fd61919d700 1 -- 172.21.15.119:0/2561360056 wait complete. 2023-10-08T00:55:59.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:55:59.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.217+0000 7fd599bff700 1 Processor -- start 2023-10-08T00:55:59.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.217+0000 7fd599bff700 1 -- start start 2023-10-08T00:55:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd599bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5941568e0 0x7fd5941616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd599bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59415b0d0 con 0x7fd5940ccee0 2023-10-08T00:55:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd599bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59415a4c0 con 0x7fd5940c83f0 2023-10-08T00:55:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd599bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd59415a7a0 con 0x7fd5941568e0 2023-10-08T00:55:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd593fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5941568e0 0x7fd5941616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd593fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5941568e0 0x7fd5941616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33204/0 (socket says 172.21.15.119:33204) 2023-10-08T00:55:59.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd593fff700 1 -- 172.21.15.119:0/896584892 learned_addr learned my addr 172.21.15.119:0/896584892 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:59.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd593fff700 1 -- 172.21.15.119:0/896584892 >> v1:172.21.15.184:6789/0 conn(0x7fd5940c83f0 legacy=0x7fd5940c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd593fff700 1 -- 172.21.15.119:0/896584892 >> v1:172.21.15.119:6789/0 conn(0x7fd5940ccee0 legacy=0x7fd5941561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd593fff700 1 -- 172.21.15.119:0/896584892 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd59415aa80 con 0x7fd5941568e0 2023-10-08T00:55:59.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd5927fc700 1 -- 172.21.15.119:0/896584892 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 498874448 0 0) 0x7fd59415b0d0 con 0x7fd5940ccee0 2023-10-08T00:55:59.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.219+0000 7fd593fff700 1 --2- 172.21.15.119:0/896584892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5941568e0 0x7fd5941616c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009f20 tx=0x7fd58c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795c40a849c2aef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:59.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd5927fc700 1 -- 172.21.15.119:0/896584892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c00e070 con 0x7fd5941568e0 2023-10-08T00:55:59.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd5927fc700 1 -- 172.21.15.119:0/896584892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd58c00ba80 con 0x7fd5941568e0 2023-10-08T00:55:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd5927fc700 1 -- 172.21.15.119:0/896584892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0092e0 con 0x7fd5941568e0 2023-10-08T00:55:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd599bff700 1 -- 172.21.15.119:0/896584892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5941568e0 msgr2=0x7fd5941616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd599bff700 1 --2- 172.21.15.119:0/896584892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5941568e0 0x7fd5941616c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd58c009f20 tx=0x7fd58c009840 comp rx=0 tx=0).stop 2023-10-08T00:55:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd599bff700 1 -- 172.21.15.119:0/896584892 shutdown_connections 2023-10-08T00:55:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd599bff700 1 --2- 172.21.15.119:0/896584892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5941568e0 0x7fd5941616c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd599bff700 1 -- 172.21.15.119:0/896584892 >> 172.21.15.119:0/896584892 conn(0x7fd594068e10 msgr2=0x7fd5940cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd599bff700 1 -- 172.21.15.119:0/896584892 shutdown_connections 2023-10-08T00:55:59.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.220+0000 7fd599bff700 1 -- 172.21.15.119:0/896584892 wait complete. 2023-10-08T00:55:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.222+0000 7fd599bff700 1 Processor -- start 2023-10-08T00:55:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.222+0000 7fd599bff700 1 -- start start 2023-10-08T00:55:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd599bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5940c83f0 0x7fd59414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd599bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5940ccee0 0x7fd5940b01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd599bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5941568e0 0x7fd5940b1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd599bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5940b1f00 con 0x7fd5940c83f0 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd599bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd594158c20 con 0x7fd5940ccee0 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd599bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd594157d70 con 0x7fd5941568e0 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd5937fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5940c83f0 0x7fd59414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd5937fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5940c83f0 0x7fd59414ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33212/0 (socket says 172.21.15.119:33212) 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd5937fe700 1 -- 172.21.15.119:0/3471159418 learned_addr learned my addr 172.21.15.119:0/3471159418 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd592ffd700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5940ccee0 0x7fd5940b01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd593fff700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5941568e0 0x7fd5940b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd5937fe700 1 -- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5941568e0 msgr2=0x7fd5940b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd5937fe700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5941568e0 0x7fd5940b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd5937fe700 1 -- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5940ccee0 msgr2=0x7fd5940b01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd5937fe700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5940ccee0 0x7fd5940b01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd5937fe700 1 -- 172.21.15.119:0/3471159418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c009ae0 con 0x7fd5940c83f0 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd592ffd700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5940ccee0 0x7fd5940b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.223+0000 7fd593fff700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5941568e0 0x7fd5940b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.224+0000 7fd5937fe700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5940c83f0 0x7fd59414ff10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c7c0 tx=0x7fd584007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.224+0000 7fd57b7fe700 1 -- 172.21.15.119:0/3471159418 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584011590 con 0x7fd5940c83f0 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.224+0000 7fd57b7fe700 1 -- 172.21.15.119:0/3471159418 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd584011710 con 0x7fd5940c83f0 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.224+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd594158010 con 0x7fd5940c83f0 2023-10-08T00:55:59.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.224+0000 7fd57b7fe700 1 -- 172.21.15.119:0/3471159418 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd584011ad0 con 0x7fd5940c83f0 2023-10-08T00:55:59.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.224+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5941585d0 con 0x7fd5940c83f0 2023-10-08T00:55:59.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.225+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd59404c3c0 con 0x7fd5940c83f0 2023-10-08T00:55:59.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.226+0000 7fd57b7fe700 1 -- 172.21.15.119:0/3471159418 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd58401b3b0 con 0x7fd5940c83f0 2023-10-08T00:55:59.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.227+0000 7fd57b7fe700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd580066f30 0x7fd5800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.227+0000 7fd57b7fe700 1 -- 172.21.15.119:0/3471159418 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd584083b90 con 0x7fd5940c83f0 2023-10-08T00:55:59.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.227+0000 7fd592ffd700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd580066f30 0x7fd5800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.228+0000 7fd592ffd700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd580066f30 0x7fd5800693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5940cb240 tx=0x7fd588009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:59.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.231+0000 7fd57b7fe700 1 -- 172.21.15.119:0/3471159418 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd584050d60 con 0x7fd5940c83f0 2023-10-08T00:55:59.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.422+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd59415ad60 con 0x7fd5940c83f0 2023-10-08T00:55:59.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.422+0000 7fd57b7fe700 1 -- 172.21.15.119:0/3471159418 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd584017020 con 0x7fd5940c83f0 2023-10-08T00:55:59.424 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T00:55:59.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.424+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd580066f30 msgr2=0x7fd5800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd580066f30 0x7fd5800693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd5940cb240 tx=0x7fd588009040 comp rx=0 tx=0).stop 2023-10-08T00:55:59.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5940c83f0 msgr2=0x7fd59414ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5940c83f0 0x7fd59414ff10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c7c0 tx=0x7fd584007f60 comp rx=0 tx=0).stop 2023-10-08T00:55:59.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 shutdown_connections 2023-10-08T00:55:59.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5941568e0 0x7fd5940b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd580066f30 0x7fd5800693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5940c83f0 0x7fd59414ff10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 --2- 172.21.15.119:0/3471159418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5940ccee0 0x7fd5940b01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 >> 172.21.15.119:0/3471159418 conn(0x7fd594068e10 msgr2=0x7fd5940cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:59.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 shutdown_connections 2023-10-08T00:55:59.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.425+0000 7fd599bff700 1 -- 172.21.15.119:0/3471159418 wait complete. 2023-10-08T00:55:59.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:55:59.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.626+0000 7f488359e700 1 Processor -- start 2023-10-08T00:55:59.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.626+0000 7f488359e700 1 -- start start 2023-10-08T00:55:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.627+0000 7f488359e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f48840b16d0 0x7f48840b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.627+0000 7f488359e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48840b2080 con 0x7f48840b33b0 2023-10-08T00:55:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.627+0000 7f488359e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48840cfb00 con 0x7f48840b4e70 2023-10-08T00:55:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.627+0000 7f488359e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48840b5c40 con 0x7f48840b16d0 2023-10-08T00:55:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.627+0000 7f4882d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f48840b33b0 0x7f48840b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54352/0 (socket says 172.21.15.119:54352) 2023-10-08T00:55:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.627+0000 7f4882d9d700 1 -- 172.21.15.119:0/2878056227 learned_addr learned my addr 172.21.15.119:0/2878056227 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:55:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.627+0000 7f487bfff700 1 --2- 172.21.15.119:0/2878056227 >> v2:172.21.15.184:3300/0 conn(0x7f48840b16d0 0x7f48840b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f4881d9b700 1 -- 172.21.15.119:0/2878056227 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506147103 0 0) 0x7f48840b2080 con 0x7f48840b33b0 2023-10-08T00:55:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f4881d9b700 1 -- 172.21.15.119:0/2878056227 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4870003570 con 0x7f48840b33b0 2023-10-08T00:55:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f4881d9b700 1 -- 172.21.15.119:0/2878056227 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2872438048 0 0) 0x7f48840cfb00 con 0x7f48840b4e70 2023-10-08T00:55:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f4881d9b700 1 -- 172.21.15.119:0/2878056227 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48840b2080 con 0x7f48840b4e70 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f487bfff700 1 -- 172.21.15.119:0/2878056227 >> v1:172.21.15.184:6789/0 conn(0x7f48840b4e70 legacy=0x7f48840ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f487bfff700 1 -- 172.21.15.119:0/2878056227 >> v1:172.21.15.119:6789/0 conn(0x7f48840b33b0 legacy=0x7f48840b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f487bfff700 1 -- 172.21.15.119:0/2878056227 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840b5f20 con 0x7f48840b16d0 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f487bfff700 1 --2- 172.21.15.119:0/2878056227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b16d0 0x7f48840b1b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f486c0098e0 tx=0x7f486c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d169a1c81655831b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.628+0000 7f4881d9b700 1 -- 172.21.15.119:0/2878056227 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c011630 con 0x7f48840b16d0 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f4881d9b700 1 -- 172.21.15.119:0/2878056227 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486c011790 con 0x7f48840b16d0 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f4881d9b700 1 -- 172.21.15.119:0/2878056227 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486c011b30 con 0x7f48840b16d0 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f488359e700 1 -- 172.21.15.119:0/2878056227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b16d0 msgr2=0x7f48840b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f488359e700 1 --2- 172.21.15.119:0/2878056227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b16d0 0x7f48840b1b40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f486c0098e0 tx=0x7f486c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f488359e700 1 -- 172.21.15.119:0/2878056227 shutdown_connections 2023-10-08T00:55:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f488359e700 1 --2- 172.21.15.119:0/2878056227 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b16d0 0x7f48840b1b40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f488359e700 1 -- 172.21.15.119:0/2878056227 >> 172.21.15.119:0/2878056227 conn(0x7f48840d10c0 msgr2=0x7f48840affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f488359e700 1 -- 172.21.15.119:0/2878056227 shutdown_connections 2023-10-08T00:55:59.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.629+0000 7f488359e700 1 -- 172.21.15.119:0/2878056227 wait complete. 2023-10-08T00:55:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.631+0000 7f488359e700 1 Processor -- start 2023-10-08T00:55:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.631+0000 7f488359e700 1 -- start start 2023-10-08T00:55:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.631+0000 7f488359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48840b16d0 0x7f48840cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48840b33b0 0x7f488415f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b4e70 0x7f488415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48841583c0 con 0x7f48840b16d0 2023-10-08T00:55:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48840cc070 con 0x7f48840b4e70 2023-10-08T00:55:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48840cc350 con 0x7f48840b33b0 2023-10-08T00:55:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48840b16d0 0x7f48840cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48840b16d0 0x7f48840cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33218/0 (socket says 172.21.15.119:33218) 2023-10-08T00:55:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 -- 172.21.15.119:0/3453307323 learned_addr learned my addr 172.21.15.119:0/3453307323 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:55:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f487bfff700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48840b33b0 0x7f488415f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f4882d9d700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b4e70 0x7f488415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 -- 172.21.15.119:0/3453307323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48840b33b0 msgr2=0x7f488415f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48840b33b0 0x7f488415f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 -- 172.21.15.119:0/3453307323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b4e70 msgr2=0x7f488415fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b4e70 0x7f488415fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 -- 172.21.15.119:0/3453307323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c009420 con 0x7f48840b16d0 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.632+0000 7f488259c700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48840b16d0 0x7f48840cb9b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f487400c580 tx=0x7f4874007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.633+0000 7f487a7fc700 1 -- 172.21.15.119:0/3453307323 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874010510 con 0x7f48840b16d0 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.633+0000 7f487a7fc700 1 -- 172.21.15.119:0/3453307323 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874010670 con 0x7f48840b16d0 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.633+0000 7f487a7fc700 1 -- 172.21.15.119:0/3453307323 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874010a30 con 0x7f48840b16d0 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.633+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48840cc630 con 0x7f48840b16d0 2023-10-08T00:55:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.633+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488415add0 con 0x7f48840b16d0 2023-10-08T00:55:59.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.634+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488415aff0 con 0x7f48840b16d0 2023-10-08T00:55:59.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.635+0000 7f487a7fc700 1 -- 172.21.15.119:0/3453307323 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f487401b400 con 0x7f48840b16d0 2023-10-08T00:55:59.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.635+0000 7f487a7fc700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4864066f30 0x7f48640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:55:59.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.636+0000 7f487a7fc700 1 -- 172.21.15.119:0/3453307323 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f48740841d0 con 0x7f48840b16d0 2023-10-08T00:55:59.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.636+0000 7f487bfff700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4864066f30 0x7f48640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:55:59.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.636+0000 7f487bfff700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4864066f30 0x7f48640693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f486c0093f0 tx=0x7f486c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:55:59.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.639+0000 7f487a7fc700 1 -- 172.21.15.119:0/3453307323 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48740513a0 con 0x7f48840b16d0 2023-10-08T00:55:59.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.828+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f488404bb80 con 0x7f48840b16d0 2023-10-08T00:55:59.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.829+0000 7f487a7fc700 1 -- 172.21.15.119:0/3453307323 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48740549c0 con 0x7f48840b16d0 2023-10-08T00:55:59.831 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4864066f30 msgr2=0x7f48640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4864066f30 0x7f48640693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f486c0093f0 tx=0x7f486c017000 comp rx=0 tx=0).stop 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48840b16d0 msgr2=0x7f48840cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48840b16d0 0x7f48840cb9b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f487400c580 tx=0x7f4874007e10 comp rx=0 tx=0).stop 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 shutdown_connections 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4864066f30 0x7f48640693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48840b4e70 0x7f488415fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48840b16d0 0x7f48840cb9b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 --2- 172.21.15.119:0/3453307323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48840b33b0 0x7f488415f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:55:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 >> 172.21.15.119:0/3453307323 conn(0x7f48840d10c0 msgr2=0x7f48840c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:55:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 shutdown_connections 2023-10-08T00:55:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:55:59.831+0000 7f488359e700 1 -- 172.21.15.119:0/3453307323 wait complete. 2023-10-08T00:55:59.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:00.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.031+0000 7fe1dadff700 1 Processor -- start 2023-10-08T00:56:00.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.031+0000 7fe1dadff700 1 -- start start 2023-10-08T00:56:00.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1dadff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40cbf30 0x7fe1d40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1dadff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe1d40b1420 0x7fe1d40c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1dadff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d40cc300 con 0x7fe1d40ccf90 2023-10-08T00:56:00.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1dadff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d4156bf0 con 0x7fe1d40b1420 2023-10-08T00:56:00.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1dadff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40ae950 con 0x7fe1d40cbf30 2023-10-08T00:56:00.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1d3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe1d40ccf90 0x7fe1d40b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54358/0 (socket says 172.21.15.119:54358) 2023-10-08T00:56:00.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1d3fff700 1 -- 172.21.15.119:0/537542921 learned_addr learned my addr 172.21.15.119:0/537542921 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1d8b9b700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40cbf30 0x7fe1d40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1d939c700 1 --2- 172.21.15.119:0/537542921 >> v2:172.21.15.184:3300/0 conn(0x7fe1d40b1420 0x7fe1d40c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329505380 0 0) 0x7fe1d40cc300 con 0x7fe1d40ccf90 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.033+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1c4003570 con 0x7fe1d40ccf90 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1066462934 0 0) 0x7fe1c4003570 con 0x7fe1d40ccf90 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40cbf30 msgr2=0x7fe1d40ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40cbf30 0x7fe1d40ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 >> v2:172.21.15.184:3300/0 conn(0x7fe1d40b1420 msgr2=0x7fe1d40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 --2- 172.21.15.119:0/537542921 >> v2:172.21.15.184:3300/0 conn(0x7fe1d40b1420 0x7fe1d40c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40ad370 con 0x7fe1d40ccf90 2023-10-08T00:56:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fe1c0001470 con 0x7fe1d40ccf90 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 >> v1:172.21.15.119:6789/0 conn(0x7fe1d40ccf90 legacy=0x7fe1d40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40cbf30 0x7fe1c4003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004010 0x7fe1c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1c4004d50 0x7fe1c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4156bf0 con 0x7fe1c4004d50 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40ae950 con 0x7fe1c4004010 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c400bab0 con 0x7fe1d40cbf30 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d939c700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1c4004d50 0x7fe1c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d8b9b700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40cbf30 0x7fe1c4003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d3fff700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004010 0x7fe1c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d939c700 1 -- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40cbf30 msgr2=0x7fe1c4003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d939c700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40cbf30 0x7fe1c4003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d939c700 1 -- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004010 msgr2=0x7fe1c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.034+0000 7fe1d939c700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004010 0x7fe1c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.035+0000 7fe1d939c700 1 -- 172.21.15.119:0/537542921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c4003570 con 0x7fe1c4004d50 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.035+0000 7fe1d939c700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1c4004d50 0x7fe1c40051c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40b1820 tx=0x7fe1cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa422fcd959ab5bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.035+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00ea90 con 0x7fe1c4004d50 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.035+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc00f040 con 0x7fe1c4004d50 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.035+0000 7fe1d37fe700 1 -- 172.21.15.119:0/537542921 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00ee40 con 0x7fe1c4004d50 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.035+0000 7fe1dadff700 1 -- 172.21.15.119:0/537542921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1c4004d50 msgr2=0x7fe1c40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.035+0000 7fe1dadff700 1 --2- 172.21.15.119:0/537542921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1c4004d50 0x7fe1c40051c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe1d40b1820 tx=0x7fe1cc00b040 comp rx=0 tx=0).stop 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.035+0000 7fe1d939c700 1 -- 172.21.15.119:0/537542921 reap_dead start 2023-10-08T00:56:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.036+0000 7fe1dadff700 1 -- 172.21.15.119:0/537542921 shutdown_connections 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.036+0000 7fe1dadff700 1 -- 172.21.15.119:0/537542921 >> 172.21.15.119:0/537542921 conn(0x7fe1d406a320 msgr2=0x7fe1d40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.036+0000 7fe1dadff700 1 -- 172.21.15.119:0/537542921 shutdown_connections 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.036+0000 7fe1dadff700 1 -- 172.21.15.119:0/537542921 wait complete. 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.037+0000 7fe1dadff700 1 Processor -- start 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.037+0000 7fe1dadff700 1 -- start start 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1dadff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004d50 0x7fe1d4169d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1dadff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1d416a8a0 0x7fe1d4157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1dadff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40abae0 0x7fe1d40abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1dadff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40ac4c0 con 0x7fe1d416a8a0 2023-10-08T00:56:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1dadff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d41591a0 con 0x7fe1c4004d50 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1dadff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d4158590 con 0x7fe1d40abae0 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1d416a8a0 0x7fe1d4157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1d416a8a0 0x7fe1d4157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33238/0 (socket says 172.21.15.119:33238) 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1d3fff700 1 -- 172.21.15.119:0/2561937484 learned_addr learned my addr 172.21.15.119:0/2561937484 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1d939c700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40abae0 0x7fe1d40abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.038+0000 7fe1d8b9b700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004d50 0x7fe1d4169d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d3fff700 1 -- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40abae0 msgr2=0x7fe1d40abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d3fff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40abae0 0x7fe1d40abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d3fff700 1 -- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004d50 msgr2=0x7fe1d4169d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d3fff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004d50 0x7fe1d4169d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d3fff700 1 -- 172.21.15.119:0/2561937484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4158870 con 0x7fe1d416a8a0 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d8b9b700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004d50 0x7fe1d4169d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d3fff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1d416a8a0 0x7fe1d4157d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00073a0 tx=0x7fe1c0001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d0ff9700 1 -- 172.21.15.119:0/2561937484 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000f870 con 0x7fe1d416a8a0 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d0ff9700 1 -- 172.21.15.119:0/2561937484 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000f9d0 con 0x7fe1d416a8a0 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d4158b50 con 0x7fe1d416a8a0 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1d0ff9700 1 -- 172.21.15.119:0/2561937484 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000fd90 con 0x7fe1d416a8a0 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.039+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d416ad10 con 0x7fe1d416a8a0 2023-10-08T00:56:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.040+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d404c3c0 con 0x7fe1d416a8a0 2023-10-08T00:56:00.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.041+0000 7fe1d0ff9700 1 -- 172.21.15.119:0/2561937484 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1c00135c0 con 0x7fe1d416a8a0 2023-10-08T00:56:00.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.042+0000 7fe1d0ff9700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe1b8066f30 0x7fe1b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.042+0000 7fe1d0ff9700 1 -- 172.21.15.119:0/2561937484 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe1c00843a0 con 0x7fe1d416a8a0 2023-10-08T00:56:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.042+0000 7fe1d8b9b700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe1b8066f30 0x7fe1b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.042+0000 7fe1d8b9b700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe1b8066f30 0x7fe1b80693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8006f00 tx=0x7fe1c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.045+0000 7fe1d0ff9700 1 -- 172.21.15.119:0/2561937484 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1c0051570 con 0x7fe1d416a8a0 2023-10-08T00:56:00.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.277+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1d40ad650 con 0x7fe1d416a8a0 2023-10-08T00:56:00.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.278+0000 7fe1d0ff9700 1 -- 172.21.15.119:0/2561937484 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fe1c001a090 con 0x7fe1d416a8a0 2023-10-08T00:56:00.279 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:00.279 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:55:46.965967+0000","modified":"2023-10-08T00:55:50.512168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:56:00.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe1b8066f30 msgr2=0x7fe1b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe1b8066f30 0x7fe1b80693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8006f00 tx=0x7fe1c8008040 comp rx=0 tx=0).stop 2023-10-08T00:56:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1d416a8a0 msgr2=0x7fe1d4157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1d416a8a0 0x7fe1d4157d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00073a0 tx=0x7fe1c0001e50 comp rx=0 tx=0).stop 2023-10-08T00:56:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 shutdown_connections 2023-10-08T00:56:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1d40abae0 0x7fe1d40abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe1b8066f30 0x7fe1b80693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1c4004d50 0x7fe1d4169d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 --2- 172.21.15.119:0/2561937484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1d416a8a0 0x7fe1d4157d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 >> 172.21.15.119:0/2561937484 conn(0x7fe1d406a320 msgr2=0x7fe1d4156160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.280+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 shutdown_connections 2023-10-08T00:56:00.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.281+0000 7fe1dadff700 1 -- 172.21.15.119:0/2561937484 wait complete. 2023-10-08T00:56:00.285 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-10-08T00:56:00.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:00.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.478+0000 7f802b59e700 1 Processor -- start 2023-10-08T00:56:00.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.479+0000 7f802b59e700 1 -- start start 2023-10-08T00:56:00.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.480+0000 7f802b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f802c150180 0x7f802c157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.480+0000 7f802b59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c06e2a0 con 0x7f802c0c7410 2023-10-08T00:56:00.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.480+0000 7f802b59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c06b2e0 con 0x7f802c150550 2023-10-08T00:56:00.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.480+0000 7f802b59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c0c78e0 con 0x7f802c150180 2023-10-08T00:56:00.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.480+0000 7f802a59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f802c150180 0x7f802c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.480+0000 7f802a59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f802c150180 0x7f802c157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33248/0 (socket says 172.21.15.119:33248) 2023-10-08T00:56:00.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.480+0000 7f802a59c700 1 -- 172.21.15.119:0/1098603082 learned_addr learned my addr 172.21.15.119:0/1098603082 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:00.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f802a59c700 1 -- 172.21.15.119:0/1098603082 >> v1:172.21.15.184:6789/0 conn(0x7f802c150550 legacy=0x7f802c0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f802a59c700 1 -- 172.21.15.119:0/1098603082 >> v1:172.21.15.119:6789/0 conn(0x7f802c0c7410 legacy=0x7f802c069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f802a59c700 1 -- 172.21.15.119:0/1098603082 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0c7bc0 con 0x7f802c150180 2023-10-08T00:56:00.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f802a59c700 1 --2- 172.21.15.119:0/1098603082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150180 0x7f802c157770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f801c0099b0 tx=0x7f801c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4fb73523ce7768 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f8029d9b700 1 -- 172.21.15.119:0/1098603082 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00b860 con 0x7f802c150180 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f8029d9b700 1 -- 172.21.15.119:0/1098603082 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c00b9c0 con 0x7f802c150180 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f8029d9b700 1 -- 172.21.15.119:0/1098603082 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00bd60 con 0x7f802c150180 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f802b59e700 1 -- 172.21.15.119:0/1098603082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150180 msgr2=0x7f802c157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.481+0000 7f802b59e700 1 --2- 172.21.15.119:0/1098603082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150180 0x7f802c157770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f801c0099b0 tx=0x7f801c009250 comp rx=0 tx=0).stop 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.482+0000 7f802b59e700 1 -- 172.21.15.119:0/1098603082 shutdown_connections 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.482+0000 7f802b59e700 1 --2- 172.21.15.119:0/1098603082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150180 0x7f802c157770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.482+0000 7f802b59e700 1 -- 172.21.15.119:0/1098603082 >> 172.21.15.119:0/1098603082 conn(0x7f802c0cc1f0 msgr2=0x7f802c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.482+0000 7f802b59e700 1 -- 172.21.15.119:0/1098603082 shutdown_connections 2023-10-08T00:56:00.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.482+0000 7f802b59e700 1 -- 172.21.15.119:0/1098603082 wait complete. 2023-10-08T00:56:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.483+0000 7f802b59e700 1 Processor -- start 2023-10-08T00:56:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.483+0000 7f802b59e700 1 -- start start 2023-10-08T00:56:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0c7410 0x7f802c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c150180 0x7f802c16a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150550 0x7f802c0c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802b59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0c3340 con 0x7f802c150550 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802b59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0c3620 con 0x7f802c0c7410 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0c3900 con 0x7f802c150180 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150550 0x7f802c0c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150550 0x7f802c0c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33252/0 (socket says 172.21.15.119:33252) 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802ad9d700 1 -- 172.21.15.119:0/2890080550 learned_addr learned my addr 172.21.15.119:0/2890080550 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.484+0000 7f802a59c700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0c7410 0x7f802c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f8023fff700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c150180 0x7f802c16a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f802ad9d700 1 -- 172.21.15.119:0/2890080550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c150180 msgr2=0x7f802c16a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f802ad9d700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c150180 0x7f802c16a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f802ad9d700 1 -- 172.21.15.119:0/2890080550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0c7410 msgr2=0x7f802c0ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f802ad9d700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0c7410 0x7f802c0ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f802ad9d700 1 -- 172.21.15.119:0/2890080550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c0094f0 con 0x7f802c150550 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f802ad9d700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150550 0x7f802c0c43e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f802400c910 tx=0x7f80240016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f80227fc700 1 -- 172.21.15.119:0/2890080550 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024012630 con 0x7f802c150550 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f80227fc700 1 -- 172.21.15.119:0/2890080550 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80240127b0 con 0x7f802c150550 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f80227fc700 1 -- 172.21.15.119:0/2890080550 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024012b70 con 0x7f802c150550 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c0c3be0 con 0x7f802c150550 2023-10-08T00:56:00.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.485+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c0ae6c0 con 0x7f802c150550 2023-10-08T00:56:00.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.486+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c0c4100 con 0x7f802c150550 2023-10-08T00:56:00.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.487+0000 7f80227fc700 1 -- 172.21.15.119:0/2890080550 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f802400a3b0 con 0x7f802c150550 2023-10-08T00:56:00.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.488+0000 7f80227fc700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f800c066f30 0x7f800c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.488+0000 7f80227fc700 1 -- 172.21.15.119:0/2890080550 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f8024084820 con 0x7f802c150550 2023-10-08T00:56:00.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.488+0000 7f802a59c700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f800c066f30 0x7f800c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.489+0000 7f802a59c700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f800c066f30 0x7f800c0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f801c0099b0 tx=0x7f801c018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.492+0000 7f80227fc700 1 -- 172.21.15.119:0/2890080550 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f802c0c4100 con 0x7f802c150550 2023-10-08T00:56:00.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.683+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f802c0c4100 con 0x7f802c150550 2023-10-08T00:56:00.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.684+0000 7f80227fc700 1 -- 172.21.15.119:0/2890080550 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f802c0c4100 con 0x7f802c150550 2023-10-08T00:56:00.686 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:00.686 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:00.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.686+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f800c066f30 msgr2=0x7f800c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.686+0000 7f802b59e700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f800c066f30 0x7f800c0693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f801c0099b0 tx=0x7f801c018000 comp rx=0 tx=0).stop 2023-10-08T00:56:00.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.686+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150550 msgr2=0x7f802c0c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.686+0000 7f802b59e700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150550 0x7f802c0c43e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f802400c910 tx=0x7f80240016f0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.687+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 shutdown_connections 2023-10-08T00:56:00.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.687+0000 7f802b59e700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f800c066f30 0x7f800c0693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.687+0000 7f802b59e700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c150550 0x7f802c0c43e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.687+0000 7f802b59e700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0c7410 0x7f802c0ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.687+0000 7f802b59e700 1 --2- 172.21.15.119:0/2890080550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c150180 0x7f802c16a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.687+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 >> 172.21.15.119:0/2890080550 conn(0x7f802c0cc1f0 msgr2=0x7f802c0c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.687+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 shutdown_connections 2023-10-08T00:56:00.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.687+0000 7f802b59e700 1 -- 172.21.15.119:0/2890080550 wait complete. 2023-10-08T00:56:00.705 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:55:45.270364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:55:45.937180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:56:00.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:56:00.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.888+0000 7f682999d700 1 Processor -- start 2023-10-08T00:56:00.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.888+0000 7f682999d700 1 -- start start 2023-10-08T00:56:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.889+0000 7f682999d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68240cb1a0 0x7f682415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.889+0000 7f682999d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682415a8a0 0x7f682406f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682999d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f682406ebd0 con 0x7f682406f5a0 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682999d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f682415b2e0 con 0x7f68240cb1a0 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682999d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682415ad20 con 0x7f682415a8a0 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682899b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68240cb1a0 0x7f682415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682899b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68240cb1a0 0x7f682415a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33262/0 (socket says 172.21.15.119:33262) 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682919c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f682406f5a0 0x7f68240ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54362/0 (socket says 172.21.15.119:54362) 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682899b700 1 -- 172.21.15.119:0/3088314707 learned_addr learned my addr 172.21.15.119:0/3088314707 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682899b700 1 -- 172.21.15.119:0/3088314707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682415a8a0 msgr2=0x7f682406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f6823fff700 1 --2- 172.21.15.119:0/3088314707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682415a8a0 0x7f682406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682899b700 1 --2- 172.21.15.119:0/3088314707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682415a8a0 0x7f682406f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682899b700 1 -- 172.21.15.119:0/3088314707 >> v1:172.21.15.119:6789/0 conn(0x7f682406f5a0 legacy=0x7f68240ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f68237fe700 1 -- 172.21.15.119:0/3088314707 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982837680 0 0) 0x7f682406ebd0 con 0x7f682406f5a0 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682899b700 1 -- 172.21.15.119:0/3088314707 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682415b000 con 0x7f68240cb1a0 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f6823fff700 1 --2- 172.21.15.119:0/3088314707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682415a8a0 0x7f682406f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.890+0000 7f682899b700 1 --2- 172.21.15.119:0/3088314707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68240cb1a0 0x7f682415a360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f68180099b0 tx=0x7f6818009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f908e33d88ddf9dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f68237fe700 1 -- 172.21.15.119:0/3088314707 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800b8f0 con 0x7f68240cb1a0 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f68237fe700 1 -- 172.21.15.119:0/3088314707 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681800ba50 con 0x7f68240cb1a0 2023-10-08T00:56:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f68237fe700 1 -- 172.21.15.119:0/3088314707 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681800bdf0 con 0x7f68240cb1a0 2023-10-08T00:56:00.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f682999d700 1 -- 172.21.15.119:0/3088314707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68240cb1a0 msgr2=0x7f682415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f682999d700 1 --2- 172.21.15.119:0/3088314707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68240cb1a0 0x7f682415a360 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f68180099b0 tx=0x7f6818009250 comp rx=0 tx=0).stop 2023-10-08T00:56:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f682999d700 1 -- 172.21.15.119:0/3088314707 shutdown_connections 2023-10-08T00:56:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f682999d700 1 --2- 172.21.15.119:0/3088314707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68240cb1a0 0x7f682415a360 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f682999d700 1 --2- 172.21.15.119:0/3088314707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682415a8a0 0x7f682406f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f682999d700 1 -- 172.21.15.119:0/3088314707 >> 172.21.15.119:0/3088314707 conn(0x7f68240cc1e0 msgr2=0x7f68240cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f682999d700 1 -- 172.21.15.119:0/3088314707 shutdown_connections 2023-10-08T00:56:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.891+0000 7f682999d700 1 -- 172.21.15.119:0/3088314707 wait complete. 2023-10-08T00:56:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.893+0000 7f682999d700 1 Processor -- start 2023-10-08T00:56:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.893+0000 7f682999d700 1 -- start start 2023-10-08T00:56:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682999d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682406f5a0 0x7f68241578d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682999d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68240cb1a0 0x7f6824157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682999d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824169b70 0x7f6824169fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682999d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824158ea0 con 0x7f6824169b70 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682999d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6824158350 con 0x7f68240cb1a0 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682999d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6824158600 con 0x7f682406f5a0 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682919c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824169b70 0x7f6824169fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682919c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824169b70 0x7f6824169fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33270/0 (socket says 172.21.15.119:33270) 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682919c700 1 -- 172.21.15.119:0/1430957301 learned_addr learned my addr 172.21.15.119:0/1430957301 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f6823fff700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68240cb1a0 0x7f6824157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682899b700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682406f5a0 0x7f68241578d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682919c700 1 -- 172.21.15.119:0/1430957301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682406f5a0 msgr2=0x7f68241578d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682919c700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682406f5a0 0x7f68241578d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682919c700 1 -- 172.21.15.119:0/1430957301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68240cb1a0 msgr2=0x7f6824157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682919c700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68240cb1a0 0x7f6824157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.894+0000 7f682919c700 1 -- 172.21.15.119:0/1430957301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68180094f0 con 0x7f6824169b70 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.895+0000 7f682919c700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824169b70 0x7f6824169fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f681c00cd50 tx=0x7f681c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.895+0000 7f6820ff9700 1 -- 172.21.15.119:0/1430957301 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c005df0 con 0x7f6824169b70 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.895+0000 7f6820ff9700 1 -- 172.21.15.119:0/1430957301 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f681c00a900 con 0x7f6824169b70 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.895+0000 7f6820ff9700 1 -- 172.21.15.119:0/1430957301 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f681c00acc0 con 0x7f6824169b70 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.895+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68241588e0 con 0x7f6824169b70 2023-10-08T00:56:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.895+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f682414e840 con 0x7f6824169b70 2023-10-08T00:56:00.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.896+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f682414e6a0 con 0x7f6824169b70 2023-10-08T00:56:00.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.897+0000 7f6820ff9700 1 -- 172.21.15.119:0/1430957301 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f681c007300 con 0x7f6824169b70 2023-10-08T00:56:00.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.898+0000 7f6820ff9700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6808066f30 0x7f68080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:00.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.898+0000 7f6820ff9700 1 -- 172.21.15.119:0/1430957301 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f681c084720 con 0x7f6824169b70 2023-10-08T00:56:00.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.898+0000 7f682899b700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6808066f30 0x7f68080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:00.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.899+0000 7f682899b700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6808066f30 0x7f68080693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68180099b0 tx=0x7f6818017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:00.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:00.902+0000 7f6820ff9700 1 -- 172.21.15.119:0/1430957301 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f682414e6a0 con 0x7f6824169b70 2023-10-08T00:56:01.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.096+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f682414e6a0 con 0x7f6824169b70 2023-10-08T00:56:01.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.097+0000 7f6820ff9700 1 -- 172.21.15.119:0/1430957301 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f682414e6a0 con 0x7f6824169b70 2023-10-08T00:56:01.099 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:56:01.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6808066f30 msgr2=0x7f68080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6808066f30 0x7f68080693e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f68180099b0 tx=0x7f6818017000 comp rx=0 tx=0).stop 2023-10-08T00:56:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824169b70 msgr2=0x7f6824169fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824169b70 0x7f6824169fe0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f681c00cd50 tx=0x7f681c001ae0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 shutdown_connections 2023-10-08T00:56:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f682406f5a0 0x7f68241578d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6808066f30 0x7f68080693e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6824169b70 0x7f6824169fe0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 --2- 172.21.15.119:0/1430957301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68240cb1a0 0x7f6824157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 >> 172.21.15.119:0/1430957301 conn(0x7f68240cc1e0 msgr2=0x7f68240aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:01.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 shutdown_connections 2023-10-08T00:56:01.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.099+0000 7f682999d700 1 -- 172.21.15.119:0/1430957301 wait complete. 2023-10-08T00:56:01.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:56:01.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.300+0000 7f7ba91ff700 1 Processor -- start 2023-10-08T00:56:01.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.300+0000 7f7ba91ff700 1 -- start start 2023-10-08T00:56:01.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba91ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7ba415ac80 0x7f7ba4069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba91ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba4069cb0 0x7f7ba40c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba91ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ba40af2f0 con 0x7f7ba415b050 2023-10-08T00:56:01.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba91ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ba406a130 con 0x7f7ba415ac80 2023-10-08T00:56:01.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba91ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40af5d0 con 0x7f7ba4069cb0 2023-10-08T00:56:01.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7b9bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7ba415b050 0x7f7ba40afa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54366/0 (socket says 172.21.15.119:54366) 2023-10-08T00:56:01.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7b9bfff700 1 -- 172.21.15.119:0/1959190775 learned_addr learned my addr 172.21.15.119:0/1959190775 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:01.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba359e700 1 --2- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba4069cb0 0x7f7ba40c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba2d9d700 1 --2- 172.21.15.119:0/1959190775 >> v2:172.21.15.184:3300/0 conn(0x7f7ba415ac80 0x7f7ba4069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba259c700 1 -- 172.21.15.119:0/1959190775 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150851861 0 0) 0x7f7ba40af2f0 con 0x7f7ba415b050 2023-10-08T00:56:01.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.302+0000 7f7ba259c700 1 -- 172.21.15.119:0/1959190775 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b90003570 con 0x7f7ba415b050 2023-10-08T00:56:01.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba2d9d700 1 -- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba4069cb0 msgr2=0x7f7ba40c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:01.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba2d9d700 1 --2- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba4069cb0 0x7f7ba40c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba2d9d700 1 -- 172.21.15.119:0/1959190775 >> v1:172.21.15.119:6789/0 conn(0x7f7ba415b050 legacy=0x7f7ba40afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba2d9d700 1 -- 172.21.15.119:0/1959190775 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40c8660 con 0x7f7ba415ac80 2023-10-08T00:56:01.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba359e700 1 --2- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba4069cb0 0x7f7ba40c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:01.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba2d9d700 1 --2- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ba415ac80 0x7f7ba4069770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b940098e0 tx=0x7f7b940091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dab688c9a0c6593a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:01.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba259c700 1 -- 172.21.15.119:0/1959190775 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94011630 con 0x7f7ba415ac80 2023-10-08T00:56:01.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba259c700 1 -- 172.21.15.119:0/1959190775 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94011790 con 0x7f7ba415ac80 2023-10-08T00:56:01.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.303+0000 7f7ba259c700 1 -- 172.21.15.119:0/1959190775 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94011b10 con 0x7f7ba415ac80 2023-10-08T00:56:01.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.304+0000 7f7ba91ff700 1 -- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ba415ac80 msgr2=0x7f7ba4069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.304+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ba415ac80 0x7f7ba4069770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7b940098e0 tx=0x7f7b940091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.304+0000 7f7ba91ff700 1 -- 172.21.15.119:0/1959190775 shutdown_connections 2023-10-08T00:56:01.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.304+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba4069cb0 0x7f7ba40c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.304+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/1959190775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ba415ac80 0x7f7ba4069770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.304+0000 7f7ba91ff700 1 -- 172.21.15.119:0/1959190775 >> 172.21.15.119:0/1959190775 conn(0x7f7ba4068e10 msgr2=0x7f7ba40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:01.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.304+0000 7f7ba91ff700 1 -- 172.21.15.119:0/1959190775 shutdown_connections 2023-10-08T00:56:01.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.304+0000 7f7ba91ff700 1 -- 172.21.15.119:0/1959190775 wait complete. 2023-10-08T00:56:01.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.305+0000 7f7ba91ff700 1 Processor -- start 2023-10-08T00:56:01.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.306+0000 7f7ba91ff700 1 -- start start 2023-10-08T00:56:01.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.306+0000 7f7ba91ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba4069cb0 0x7f7ba4155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba91ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba415ac80 0x7f7ba4167ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba91ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ba415b050 0x7f7ba40c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba91ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40c35e0 con 0x7f7ba4069cb0 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba91ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40c5d60 con 0x7f7ba415b050 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba91ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ba40c6040 con 0x7f7ba415ac80 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba4069cb0 0x7f7ba4155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba4069cb0 0x7f7ba4155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33282/0 (socket says 172.21.15.119:33282) 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 -- 172.21.15.119:0/3213010489 learned_addr learned my addr 172.21.15.119:0/3213010489 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 -- 172.21.15.119:0/3213010489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba415ac80 msgr2=0x7f7ba4167ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba415ac80 0x7f7ba4167ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 -- 172.21.15.119:0/3213010489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ba415b050 msgr2=0x7f7ba40c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ba415b050 0x7f7ba40c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 -- 172.21.15.119:0/3213010489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c009a20 con 0x7f7ba4069cb0 2023-10-08T00:56:01.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.307+0000 7f7ba2d9d700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba4069cb0 0x7f7ba4155970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4063d40 tx=0x7f7b94017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:01.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.308+0000 7f7b9affd700 1 -- 172.21.15.119:0/3213010489 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94011b10 con 0x7f7ba4069cb0 2023-10-08T00:56:01.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.308+0000 7f7b9affd700 1 -- 172.21.15.119:0/3213010489 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94000cc0 con 0x7f7ba4069cb0 2023-10-08T00:56:01.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.308+0000 7f7b9affd700 1 -- 172.21.15.119:0/3213010489 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9401a680 con 0x7f7ba4069cb0 2023-10-08T00:56:01.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.308+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b94009420 con 0x7f7ba4069cb0 2023-10-08T00:56:01.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.308+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ba40c37f0 con 0x7f7ba4069cb0 2023-10-08T00:56:01.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.309+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba40c3a10 con 0x7f7ba4069cb0 2023-10-08T00:56:01.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.309+0000 7f7b9affd700 1 -- 172.21.15.119:0/3213010489 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b9401aa70 con 0x7f7ba4069cb0 2023-10-08T00:56:01.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.310+0000 7f7b9affd700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.310+0000 7f7b9affd700 1 -- 172.21.15.119:0/3213010489 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7b940861e0 con 0x7f7ba4069cb0 2023-10-08T00:56:01.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.310+0000 7f7b9bfff700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.311+0000 7f7b9bfff700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7b84066f30 0x7f7b840693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0058d0 tx=0x7f7b8c0079d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:01.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.314+0000 7f7b9affd700 1 -- 172.21.15.119:0/3213010489 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b94053450 con 0x7f7ba4069cb0 2023-10-08T00:56:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.502+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7ba404c3c0 con 0x7f7ba4069cb0 2023-10-08T00:56:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.503+0000 7f7b9affd700 1 -- 172.21.15.119:0/3213010489 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7b94056a70 con 0x7f7ba4069cb0 2023-10-08T00:56:01.504 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T00:56:01.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7b84066f30 msgr2=0x7f7b840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7b84066f30 0x7f7b840693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c0058d0 tx=0x7f7b8c0079d0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba4069cb0 msgr2=0x7f7ba4155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba4069cb0 0x7f7ba4155970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4063d40 tx=0x7f7b94017000 comp rx=0 tx=0).stop 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 shutdown_connections 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7b84066f30 0x7f7b840693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ba415b050 0x7f7ba40c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ba4069cb0 0x7f7ba4155970 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 --2- 172.21.15.119:0/3213010489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ba415ac80 0x7f7ba4167ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 >> 172.21.15.119:0/3213010489 conn(0x7f7ba4068e10 msgr2=0x7f7ba40b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 shutdown_connections 2023-10-08T00:56:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.505+0000 7f7ba91ff700 1 -- 172.21.15.119:0/3213010489 wait complete. 2023-10-08T00:56:01.523 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:56:01.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.705+0000 7fc232bff700 1 Processor -- start 2023-10-08T00:56:01.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.705+0000 7fc232bff700 1 -- start start 2023-10-08T00:56:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc232bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc232bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c06e950 con 0x7fc22c155eb0 2023-10-08T00:56:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc232bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22c06eb90 con 0x7fc22c0c70b0 2023-10-08T00:56:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc232bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c06ee70 con 0x7fc22c069e80 2023-10-08T00:56:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc23099b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc22c155eb0 0x7fc22c06e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54370/0 (socket says 172.21.15.119:54370) 2023-10-08T00:56:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc23099b700 1 -- 172.21.15.119:0/3210382687 learned_addr learned my addr 172.21.15.119:0/3210382687 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc23119c700 1 --2- 172.21.15.119:0/3210382687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc22b7fe700 1 -- 172.21.15.119:0/3210382687 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011540311 0 0) 0x7fc220001110 con 0x7fc22c155eb0 2023-10-08T00:56:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc22b7fe700 1 -- 172.21.15.119:0/3210382687 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc21c003570 con 0x7fc22c155eb0 2023-10-08T00:56:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.707+0000 7fc22b7fe700 1 -- 172.21.15.119:0/3210382687 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783767248 0 0) 0x7fc22c06eb90 con 0x7fc22c0c70b0 2023-10-08T00:56:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.708+0000 7fc22b7fe700 1 -- 172.21.15.119:0/3210382687 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc220001110 con 0x7fc22c0c70b0 2023-10-08T00:56:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.708+0000 7fc23119c700 1 -- 172.21.15.119:0/3210382687 >> v1:172.21.15.184:6789/0 conn(0x7fc22c0c70b0 legacy=0x7fc22c069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.708+0000 7fc23119c700 1 -- 172.21.15.119:0/3210382687 >> v1:172.21.15.119:6789/0 conn(0x7fc22c155eb0 legacy=0x7fc22c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.708+0000 7fc23119c700 1 -- 172.21.15.119:0/3210382687 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22c06f150 con 0x7fc22c069e80 2023-10-08T00:56:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.708+0000 7fc23119c700 1 --2- 172.21.15.119:0/3210382687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c160bf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc224009e90 tx=0x7fc224009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1468b5337b2719c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:01.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.708+0000 7fc22b7fe700 1 -- 172.21.15.119:0/3210382687 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22400e070 con 0x7fc22c069e80 2023-10-08T00:56:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.708+0000 7fc22b7fe700 1 -- 172.21.15.119:0/3210382687 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22400bb80 con 0x7fc22c069e80 2023-10-08T00:56:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.708+0000 7fc22b7fe700 1 -- 172.21.15.119:0/3210382687 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224012430 con 0x7fc22c069e80 2023-10-08T00:56:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.709+0000 7fc232bff700 1 -- 172.21.15.119:0/3210382687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 msgr2=0x7fc22c160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.709+0000 7fc232bff700 1 --2- 172.21.15.119:0/3210382687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c160bf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc224009e90 tx=0x7fc224009780 comp rx=0 tx=0).stop 2023-10-08T00:56:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.709+0000 7fc232bff700 1 -- 172.21.15.119:0/3210382687 shutdown_connections 2023-10-08T00:56:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.709+0000 7fc232bff700 1 --2- 172.21.15.119:0/3210382687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c160bf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.709+0000 7fc232bff700 1 -- 172.21.15.119:0/3210382687 >> 172.21.15.119:0/3210382687 conn(0x7fc22c068e10 msgr2=0x7fc22c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.709+0000 7fc232bff700 1 -- 172.21.15.119:0/3210382687 shutdown_connections 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.709+0000 7fc232bff700 1 -- 172.21.15.119:0/3210382687 wait complete. 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.710+0000 7fc232bff700 1 Processor -- start 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.710+0000 7fc232bff700 1 -- start start 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.711+0000 7fc232bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c0ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.711+0000 7fc232bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0c70b0 0x7fc22c0b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.711+0000 7fc232bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc22c155eb0 0x7fc22c0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.711+0000 7fc232bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0ae4d0 con 0x7fc22c155eb0 2023-10-08T00:56:01.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.711+0000 7fc232bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0adc30 con 0x7fc22c0c70b0 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.711+0000 7fc232bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c0adf10 con 0x7fc22c069e80 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.711+0000 7fc23119c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc22c155eb0 0x7fc22c0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23119c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc22c155eb0 0x7fc22c0af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33288/0 (socket says 172.21.15.119:33288) 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23119c700 1 -- 172.21.15.119:0/576240658 learned_addr learned my addr 172.21.15.119:0/576240658 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23099b700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c0ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc22bfff700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0c70b0 0x7fc22c0b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23119c700 1 -- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 msgr2=0x7fc22c0ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23119c700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c0ad5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23119c700 1 -- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0c70b0 msgr2=0x7fc22c0b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23119c700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0c70b0 0x7fc22c0b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23119c700 1 -- 172.21.15.119:0/576240658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc224009a20 con 0x7fc22c155eb0 2023-10-08T00:56:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23099b700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc23119c700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc22c155eb0 0x7fc22c0af7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc224013000 tx=0x7fc224019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc228ff9700 1 -- 172.21.15.119:0/576240658 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224012e00 con 0x7fc22c155eb0 2023-10-08T00:56:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc228ff9700 1 -- 172.21.15.119:0/576240658 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22401b630 con 0x7fc22c155eb0 2023-10-08T00:56:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc228ff9700 1 -- 172.21.15.119:0/576240658 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22401b9f0 con 0x7fc22c155eb0 2023-10-08T00:56:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.712+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c0ae1f0 con 0x7fc22c155eb0 2023-10-08T00:56:01.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.713+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c0afce0 con 0x7fc22c155eb0 2023-10-08T00:56:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.714+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c04c3c0 con 0x7fc22c155eb0 2023-10-08T00:56:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.714+0000 7fc228ff9700 1 -- 172.21.15.119:0/576240658 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2240253f0 con 0x7fc22c155eb0 2023-10-08T00:56:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.715+0000 7fc228ff9700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc210066f30 0x7fc2100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:01.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.715+0000 7fc228ff9700 1 -- 172.21.15.119:0/576240658 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc224085fb0 con 0x7fc22c155eb0 2023-10-08T00:56:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.715+0000 7fc23099b700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc210066f30 0x7fc2100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.716+0000 7fc23099b700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc210066f30 0x7fc2100693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc2200058d0 tx=0x7fc220009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:01.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.719+0000 7fc228ff9700 1 -- 172.21.15.119:0/576240658 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc224056cc0 con 0x7fc22c155eb0 2023-10-08T00:56:01.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.911+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc22c06f430 con 0x7fc22c155eb0 2023-10-08T00:56:01.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.911+0000 7fc228ff9700 1 -- 172.21.15.119:0/576240658 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc224056850 con 0x7fc22c155eb0 2023-10-08T00:56:01.913 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T00:56:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.913+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc210066f30 msgr2=0x7fc2100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.913+0000 7fc232bff700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc210066f30 0x7fc2100693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc2200058d0 tx=0x7fc220009040 comp rx=0 tx=0).stop 2023-10-08T00:56:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.913+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc22c155eb0 msgr2=0x7fc22c0af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.913+0000 7fc232bff700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc22c155eb0 0x7fc22c0af7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc224013000 tx=0x7fc224019000 comp rx=0 tx=0).stop 2023-10-08T00:56:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.914+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 shutdown_connections 2023-10-08T00:56:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.914+0000 7fc232bff700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc22c069e80 0x7fc22c0ad5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.914+0000 7fc232bff700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc210066f30 0x7fc2100693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.914+0000 7fc232bff700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc22c155eb0 0x7fc22c0af7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.914+0000 7fc232bff700 1 --2- 172.21.15.119:0/576240658 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc22c0c70b0 0x7fc22c0b1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.914+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 >> 172.21.15.119:0/576240658 conn(0x7fc22c068e10 msgr2=0x7fc22c0c1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.914+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 shutdown_connections 2023-10-08T00:56:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:01.914+0000 7fc232bff700 1 -- 172.21.15.119:0/576240658 wait complete. 2023-10-08T00:56:01.933 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:02.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.112+0000 7f15cc1ff700 1 Processor -- start 2023-10-08T00:56:02.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.112+0000 7f15cc1ff700 1 -- start start 2023-10-08T00:56:02.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.113+0000 7f15cc1ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f15c4159890 0x7f15c415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.113+0000 7f15cc1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15c4159c60 0x7f15c415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.113+0000 7f15cc1ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15c415a160 con 0x7f15c415db10 2023-10-08T00:56:02.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.113+0000 7f15cc1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c415df50 con 0x7f15c4159c60 2023-10-08T00:56:02.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.113+0000 7f15cc1ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15c40d13a0 con 0x7f15c4159890 2023-10-08T00:56:02.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c979a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15c4159c60 0x7f15c415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c979a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f15c4159c60 0x7f15c415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33296/0 (socket says 172.21.15.119:33296) 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c979a700 1 -- 172.21.15.119:0/2986280898 learned_addr learned my addr 172.21.15.119:0/2986280898 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c9f9b700 1 --2- 172.21.15.119:0/2986280898 >> v2:172.21.15.184:3300/0 conn(0x7f15c4159890 0x7f15c415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c979a700 1 -- 172.21.15.119:0/2986280898 >> v2:172.21.15.184:3300/0 conn(0x7f15c4159890 msgr2=0x7f15c415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c979a700 1 --2- 172.21.15.119:0/2986280898 >> v2:172.21.15.184:3300/0 conn(0x7f15c4159890 0x7f15c415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c979a700 1 -- 172.21.15.119:0/2986280898 >> v1:172.21.15.184:6789/0 conn(0x7f15c415db10 legacy=0x7f15c4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c979a700 1 -- 172.21.15.119:0/2986280898 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c415e140 con 0x7f15c4159c60 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c9f9b700 1 --2- 172.21.15.119:0/2986280898 >> v2:172.21.15.184:3300/0 conn(0x7f15c4159890 0x7f15c415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c979a700 1 --2- 172.21.15.119:0/2986280898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c415d5d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15c40d1090 tx=0x7f15b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c69d80b8067a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.114+0000 7f15c8f99700 1 -- 172.21.15.119:0/2986280898 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b400b860 con 0x7f15c4159c60 2023-10-08T00:56:02.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15c8f99700 1 -- 172.21.15.119:0/2986280898 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b400b9c0 con 0x7f15c4159c60 2023-10-08T00:56:02.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15c8f99700 1 -- 172.21.15.119:0/2986280898 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b400bd60 con 0x7f15c4159c60 2023-10-08T00:56:02.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15cc1ff700 1 -- 172.21.15.119:0/2986280898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 msgr2=0x7f15c415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/2986280898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c415d5d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15c40d1090 tx=0x7f15b4009250 comp rx=0 tx=0).stop 2023-10-08T00:56:02.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15cc1ff700 1 -- 172.21.15.119:0/2986280898 shutdown_connections 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/2986280898 >> v2:172.21.15.184:3300/0 conn(0x7f15c4159890 0x7f15c415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/2986280898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c415d5d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15cc1ff700 1 -- 172.21.15.119:0/2986280898 >> 172.21.15.119:0/2986280898 conn(0x7f15c4068e20 msgr2=0x7f15c40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15cc1ff700 1 -- 172.21.15.119:0/2986280898 shutdown_connections 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.115+0000 7f15cc1ff700 1 -- 172.21.15.119:0/2986280898 wait complete. 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.117+0000 7f15cc1ff700 1 Processor -- start 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.117+0000 7f15cc1ff700 1 -- start start 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15cc1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c4159890 0x7f15c4140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15cc1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c40b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15cc1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15c415db10 0x7f15c40b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15cc1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40b3fc0 con 0x7f15c4159c60 2023-10-08T00:56:02.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15cc1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15c41528a0 con 0x7f15c4159890 2023-10-08T00:56:02.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15cc1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c40b1130 con 0x7f15c415db10 2023-10-08T00:56:02.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c40b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c40b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33304/0 (socket says 172.21.15.119:33304) 2023-10-08T00:56:02.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 -- 172.21.15.119:0/1640856450 learned_addr learned my addr 172.21.15.119:0/1640856450 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c9f9b700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c4159890 0x7f15c4140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15ca79c700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15c415db10 0x7f15c40b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 -- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15c415db10 msgr2=0x7f15c40b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15c415db10 0x7f15c40b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 -- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c4159890 msgr2=0x7f15c4140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c4159890 0x7f15c4140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 -- 172.21.15.119:0/1640856450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b40094f0 con 0x7f15c4159c60 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15ca79c700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15c415db10 0x7f15c40b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.118+0000 7f15c979a700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c40b05c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009290 tx=0x7f15b4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.119+0000 7f15b9ffb700 1 -- 172.21.15.119:0/1640856450 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4026480 con 0x7f15c4159c60 2023-10-08T00:56:02.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.119+0000 7f15b9ffb700 1 -- 172.21.15.119:0/1640856450 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b401f070 con 0x7f15c4159c60 2023-10-08T00:56:02.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.119+0000 7f15b9ffb700 1 -- 172.21.15.119:0/1640856450 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4026850 con 0x7f15c4159c60 2023-10-08T00:56:02.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.119+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15c40b1410 con 0x7f15c4159c60 2023-10-08T00:56:02.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.119+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15c40b19d0 con 0x7f15c4159c60 2023-10-08T00:56:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.120+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15c404c3c0 con 0x7f15c4159c60 2023-10-08T00:56:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.121+0000 7f15b9ffb700 1 -- 172.21.15.119:0/1640856450 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f15b401a430 con 0x7f15c4159c60 2023-10-08T00:56:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.122+0000 7f15b9ffb700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f15a8066f30 0x7f15a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.122+0000 7f15b9ffb700 1 -- 172.21.15.119:0/1640856450 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f15b4021070 con 0x7f15c4159c60 2023-10-08T00:56:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.122+0000 7f15c9f9b700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f15a8066f30 0x7f15a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.122+0000 7f15c9f9b700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f15a8066f30 0x7f15a80693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006f00 tx=0x7f15c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.126+0000 7f15b9ffb700 1 -- 172.21.15.119:0/1640856450 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15b4053710 con 0x7f15c4159c60 2023-10-08T00:56:02.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.354+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15c406a5b0 con 0x7f15c4159c60 2023-10-08T00:56:02.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.355+0000 7f15b9ffb700 1 -- 172.21.15.119:0/1640856450 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f15b4056d30 con 0x7f15c4159c60 2023-10-08T00:56:02.357 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:02.357 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:55:46.965967+0000","modified":"2023-10-08T00:55:50.512168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:56:02.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f15a8066f30 msgr2=0x7f15a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f15a8066f30 0x7f15a80693e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006f00 tx=0x7f15c0008040 comp rx=0 tx=0).stop 2023-10-08T00:56:02.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 msgr2=0x7f15c40b05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c40b05c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009290 tx=0x7f15b4017000 comp rx=0 tx=0).stop 2023-10-08T00:56:02.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 shutdown_connections 2023-10-08T00:56:02.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15c415db10 0x7f15c40b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f15a8066f30 0x7f15a80693e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15c4159890 0x7f15c4140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 --2- 172.21.15.119:0/1640856450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15c4159c60 0x7f15c40b05c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 >> 172.21.15.119:0/1640856450 conn(0x7f15c4068e20 msgr2=0x7f15c4153190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:02.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 shutdown_connections 2023-10-08T00:56:02.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.358+0000 7f15cc1ff700 1 -- 172.21.15.119:0/1640856450 wait complete. 2023-10-08T00:56:02.364 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-10-08T00:56:02.376 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:02.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.561+0000 7fc6a339d700 1 Processor -- start 2023-10-08T00:56:02.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.561+0000 7fc6a339d700 1 -- start start 2023-10-08T00:56:02.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.562+0000 7fc6a339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c1597f0 0x7fc69c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.562+0000 7fc6a339d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc69c15d230 0x7fc69c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.562+0000 7fc6a339d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc69c15dbd0 con 0x7fc69c159420 2023-10-08T00:56:02.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.562+0000 7fc6a339d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c0b11d0 con 0x7fc69c15d230 2023-10-08T00:56:02.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.562+0000 7fc6a339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0b14b0 con 0x7fc69c1597f0 2023-10-08T00:56:02.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.562+0000 7fc6a239b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc69c159420 0x7fc69c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54374/0 (socket says 172.21.15.119:54374) 2023-10-08T00:56:02.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a239b700 1 -- 172.21.15.119:0/942333250 learned_addr learned my addr 172.21.15.119:0/942333250 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:02.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1b9a700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c1597f0 0x7fc69c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a2b9c700 1 --2- 172.21.15.119:0/942333250 >> v2:172.21.15.184:3300/0 conn(0x7fc69c15d230 0x7fc69c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864951151 0 0) 0x7fc698001110 con 0x7fc69c159420 2023-10-08T00:56:02.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc688003570 con 0x7fc69c159420 2023-10-08T00:56:02.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3757374368 0 0) 0x7fc688003570 con 0x7fc69c159420 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c1597f0 msgr2=0x7fc69c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c1597f0 0x7fc69c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 >> v2:172.21.15.184:3300/0 conn(0x7fc69c15d230 msgr2=0x7fc69c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 --2- 172.21.15.119:0/942333250 >> v2:172.21.15.184:3300/0 conn(0x7fc69c15d230 0x7fc69c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c0b1790 con 0x7fc69c159420 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fc698001600 con 0x7fc69c159420 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 >> v1:172.21.15.119:6789/0 conn(0x7fc69c159420 legacy=0x7fc69c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688003a60 0x7fc688003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c1597f0 0x7fc688004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc688004d50 0x7fc6880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0b11d0 con 0x7fc688004d50 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0b14b0 con 0x7fc688003a60 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.563+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68800ba60 con 0x7fc69c1597f0 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a2b9c700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc688004d50 0x7fc6880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a239b700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688003a60 0x7fc688003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a1b9a700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c1597f0 0x7fc688004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a2b9c700 1 -- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c1597f0 msgr2=0x7fc688004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a2b9c700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c1597f0 0x7fc688004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a2b9c700 1 -- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688003a60 msgr2=0x7fc688003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a2b9c700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688003a60 0x7fc688003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a2b9c700 1 -- 172.21.15.119:0/942333250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc688003570 con 0x7fc688004d50 2023-10-08T00:56:02.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a2b9c700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc688004d50 0x7fc6880051c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc69400e210 tx=0x7fc69400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb2dd3392e75530 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400ea60 con 0x7fc688004d50 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69400f040 con 0x7fc688004d50 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.564+0000 7fc6a1399700 1 -- 172.21.15.119:0/942333250 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400ee10 con 0x7fc688004d50 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.565+0000 7fc6a339d700 1 -- 172.21.15.119:0/942333250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc688004d50 msgr2=0x7fc6880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.565+0000 7fc6a339d700 1 --2- 172.21.15.119:0/942333250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc688004d50 0x7fc6880051c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc69400e210 tx=0x7fc69400b040 comp rx=0 tx=0).stop 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.565+0000 7fc6a2b9c700 1 -- 172.21.15.119:0/942333250 reap_dead start 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.565+0000 7fc6a339d700 1 -- 172.21.15.119:0/942333250 shutdown_connections 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.565+0000 7fc6a339d700 1 -- 172.21.15.119:0/942333250 >> 172.21.15.119:0/942333250 conn(0x7fc69c0d0660 msgr2=0x7fc69c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:02.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.565+0000 7fc6a339d700 1 -- 172.21.15.119:0/942333250 shutdown_connections 2023-10-08T00:56:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.565+0000 7fc6a339d700 1 -- 172.21.15.119:0/942333250 wait complete. 2023-10-08T00:56:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.566+0000 7fc6a339d700 1 Processor -- start 2023-10-08T00:56:02.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.567+0000 7fc6a339d700 1 -- start start 2023-10-08T00:56:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.567+0000 7fc6a339d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688004d50 0x7fc69c143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.567+0000 7fc6a339d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c0cc130 0x7fc69c068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c0688d0 0x7fc69c0ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a339d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c068d40 con 0x7fc69c0cc130 2023-10-08T00:56:02.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a339d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0cbca0 con 0x7fc688004d50 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0cb090 con 0x7fc69c0688d0 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c0cc130 0x7fc69c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c0cc130 0x7fc69c068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33324/0 (socket says 172.21.15.119:33324) 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 -- 172.21.15.119:0/2421347069 learned_addr learned my addr 172.21.15.119:0/2421347069 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a239b700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688004d50 0x7fc69c143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a2b9c700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c0688d0 0x7fc69c0ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 -- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c0688d0 msgr2=0x7fc69c0ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c0688d0 0x7fc69c0ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 -- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688004d50 msgr2=0x7fc69c143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688004d50 0x7fc69c143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 -- 172.21.15.119:0/2421347069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c0cb370 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a239b700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688004d50 0x7fc69c143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.568+0000 7fc6a1b9a700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c0cc130 0x7fc69c068390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc68c006ac0 tx=0x7fc68c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.569+0000 7fc6927fc700 1 -- 172.21.15.119:0/2421347069 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c00fa10 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.569+0000 7fc6927fc700 1 -- 172.21.15.119:0/2421347069 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c00fb70 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.569+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c0cb650 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.569+0000 7fc6927fc700 1 -- 172.21.15.119:0/2421347069 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c014440 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.569+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c144240 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.570+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c0cbe50 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.570+0000 7fc6927fc700 1 -- 172.21.15.119:0/2421347069 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc68c014830 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.571+0000 7fc6927fc700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.571+0000 7fc6927fc700 1 -- 172.21.15.119:0/2421347069 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc68c018030 con 0x7fc69c0cc130 2023-10-08T00:56:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.572+0000 7fc6a239b700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.572+0000 7fc6a239b700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc680066f30 0x7fc6800693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc698000e10 tx=0x7fc698009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.575+0000 7fc6927fc700 1 -- 172.21.15.119:0/2421347069 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc69c0cbe50 con 0x7fc69c0cc130 2023-10-08T00:56:02.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.768+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc69c0cbe10 con 0x7fc69c0cc130 2023-10-08T00:56:02.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.769+0000 7fc6927fc700 1 -- 172.21.15.119:0/2421347069 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fc68c022090 con 0x7fc69c0cc130 2023-10-08T00:56:02.771 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:02.771 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:02.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc680066f30 msgr2=0x7fc6800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc680066f30 0x7fc6800693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fc698000e10 tx=0x7fc698009040 comp rx=0 tx=0).stop 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c0cc130 msgr2=0x7fc69c068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c0cc130 0x7fc69c068390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc68c006ac0 tx=0x7fc68c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 shutdown_connections 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc69c0688d0 0x7fc69c0ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc680066f30 0x7fc6800693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc688004d50 0x7fc69c143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 --2- 172.21.15.119:0/2421347069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc69c0cc130 0x7fc69c068390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 >> 172.21.15.119:0/2421347069 conn(0x7fc69c0d0660 msgr2=0x7fc69c158410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 shutdown_connections 2023-10-08T00:56:02.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.772+0000 7fc6a339d700 1 -- 172.21.15.119:0/2421347069 wait complete. 2023-10-08T00:56:02.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:02.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.970+0000 7f1996dff700 1 Processor -- start 2023-10-08T00:56:02.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.971+0000 7f1996dff700 1 -- start start 2023-10-08T00:56:02.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f1996dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1990159890 0x7f199015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f1996dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 0x7f199015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f1996dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199015a160 con 0x7f199015db10 2023-10-08T00:56:02.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f1996dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19900d13a0 con 0x7f1990159890 2023-10-08T00:56:02.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f1996dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199015e140 con 0x7f1990159c60 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f199539c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f199015db10 0x7f1990069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54380/0 (socket says 172.21.15.119:54380) 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f199539c700 1 -- 172.21.15.119:0/429665560 learned_addr learned my addr 172.21.15.119:0/429665560 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f1994b9b700 1 --2- 172.21.15.119:0/429665560 >> v2:172.21.15.184:3300/0 conn(0x7f1990159890 0x7f199015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.972+0000 7f198ffff700 1 --2- 172.21.15.119:0/429665560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 0x7f199015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198f7fe700 1 -- 172.21.15.119:0/429665560 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2650081327 0 0) 0x7f199015a160 con 0x7f199015db10 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198f7fe700 1 -- 172.21.15.119:0/429665560 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1980003570 con 0x7f199015db10 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198ffff700 1 -- 172.21.15.119:0/429665560 >> v2:172.21.15.184:3300/0 conn(0x7f1990159890 msgr2=0x7f199015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198ffff700 1 --2- 172.21.15.119:0/429665560 >> v2:172.21.15.184:3300/0 conn(0x7f1990159890 0x7f199015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198ffff700 1 -- 172.21.15.119:0/429665560 >> v1:172.21.15.119:6789/0 conn(0x7f199015db10 legacy=0x7f1990069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198f7fe700 1 -- 172.21.15.119:0/429665560 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612936033 0 0) 0x7f1980003570 con 0x7f199015db10 2023-10-08T00:56:02.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198ffff700 1 -- 172.21.15.119:0/429665560 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199015a3d0 con 0x7f1990159c60 2023-10-08T00:56:02.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f1994b9b700 1 --2- 172.21.15.119:0/429665560 >> v2:172.21.15.184:3300/0 conn(0x7f1990159890 0x7f199015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:02.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198ffff700 1 --2- 172.21.15.119:0/429665560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 0x7f199015d5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f197c0098e0 tx=0x7f197c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8573c186cfab2dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.973+0000 7f198f7fe700 1 -- 172.21.15.119:0/429665560 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19880017f0 con 0x7f1990159c60 2023-10-08T00:56:02.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f198f7fe700 1 -- 172.21.15.119:0/429665560 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c011420 con 0x7f1990159c60 2023-10-08T00:56:02.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f198f7fe700 1 -- 172.21.15.119:0/429665560 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c0117c0 con 0x7f1990159c60 2023-10-08T00:56:02.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f1996dff700 1 -- 172.21.15.119:0/429665560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 msgr2=0x7f199015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f1996dff700 1 --2- 172.21.15.119:0/429665560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 0x7f199015d5d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f197c0098e0 tx=0x7f197c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f1996dff700 1 -- 172.21.15.119:0/429665560 shutdown_connections 2023-10-08T00:56:02.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f1996dff700 1 --2- 172.21.15.119:0/429665560 >> v2:172.21.15.184:3300/0 conn(0x7f1990159890 0x7f199015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f1996dff700 1 --2- 172.21.15.119:0/429665560 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 0x7f199015d5d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f1996dff700 1 -- 172.21.15.119:0/429665560 >> 172.21.15.119:0/429665560 conn(0x7f1990068e20 msgr2=0x7f19900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:02.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f1996dff700 1 -- 172.21.15.119:0/429665560 shutdown_connections 2023-10-08T00:56:02.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.974+0000 7f1996dff700 1 -- 172.21.15.119:0/429665560 wait complete. 2023-10-08T00:56:02.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.975+0000 7f1996dff700 1 Processor -- start 2023-10-08T00:56:02.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.976+0000 7f1996dff700 1 -- start start 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.976+0000 7f1996dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1990159890 0x7f199016afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.976+0000 7f1996dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 0x7f199016ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.976+0000 7f1996dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199015db10 0x7f1990140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.976+0000 7f1996dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19900b53e0 con 0x7f1990159890 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.976+0000 7f1996dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199016b500 con 0x7f199015db10 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.976+0000 7f1996dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199016e2f0 con 0x7f1990159c60 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1990159890 0x7f199016afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1990159890 0x7f199016afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33332/0 (socket says 172.21.15.119:33332) 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 -- 172.21.15.119:0/2312926214 learned_addr learned my addr 172.21.15.119:0/2312926214 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:02.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 -- 172.21.15.119:0/2312926214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 msgr2=0x7f199016ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 0x7f199016ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 -- 172.21.15.119:0/2312926214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199015db10 msgr2=0x7f1990140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199015db10 0x7f1990140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 -- 172.21.15.119:0/2312926214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1984009420 con 0x7f1990159890 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1994b9b700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1990159890 0x7f199016afc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f198400ec80 tx=0x7f19840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f198cff9700 1 -- 172.21.15.119:0/2312926214 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198400c9d0 con 0x7f1990159890 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f198cff9700 1 -- 172.21.15.119:0/2312926214 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198400cb30 con 0x7f1990159890 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f198cff9700 1 -- 172.21.15.119:0/2312926214 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198400ceb0 con 0x7f1990159890 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.977+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c009420 con 0x7f1990159890 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.978+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199016a500 con 0x7f1990159890 2023-10-08T00:56:02.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.979+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19900b56a0 con 0x7f1990159890 2023-10-08T00:56:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.979+0000 7f198cff9700 1 -- 172.21.15.119:0/2312926214 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1984013680 con 0x7f1990159890 2023-10-08T00:56:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.980+0000 7f198cff9700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1974066f30 0x7f19740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.980+0000 7f198cff9700 1 -- 172.21.15.119:0/2312926214 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f1984084990 con 0x7f1990159890 2023-10-08T00:56:02.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.981+0000 7f198ffff700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1974066f30 0x7f19740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:02.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.981+0000 7f198ffff700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1974066f30 0x7f19740693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f199015a090 tx=0x7f197c016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:02.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:02.985+0000 7f198cff9700 1 -- 172.21.15.119:0/2312926214 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1984051c00 con 0x7f1990159890 2023-10-08T00:56:03.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.215+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f199004c3c0 con 0x7f1990159890 2023-10-08T00:56:03.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.215+0000 7f198cff9700 1 -- 172.21.15.119:0/2312926214 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f1984055220 con 0x7f1990159890 2023-10-08T00:56:03.217 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:03.217 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:55:46.965967+0000","modified":"2023-10-08T00:55:50.512168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:56:03.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1974066f30 msgr2=0x7f19740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1974066f30 0x7f19740693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f199015a090 tx=0x7f197c016000 comp rx=0 tx=0).stop 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1990159890 msgr2=0x7f199016afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1990159890 0x7f199016afc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f198400ec80 tx=0x7f19840091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 shutdown_connections 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1974066f30 0x7f19740693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f199015db10 0x7f1990140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1990159890 0x7f199016afc0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 --2- 172.21.15.119:0/2312926214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1990159c60 0x7f199016ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 >> 172.21.15.119:0/2312926214 conn(0x7f1990068e20 msgr2=0x7f1990156e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 shutdown_connections 2023-10-08T00:56:03.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.218+0000 7f1996dff700 1 -- 172.21.15.119:0/2312926214 wait complete. 2023-10-08T00:56:03.221 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-10-08T00:56:03.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:55:46.965967+0000', 'modified': '2023-10-08T00:55:50.512168+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4282}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4282': {'gid': 4282, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6834/1701092387', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6832', 'nonce': 1701092387}, {'type': 'v1', 'addr': '172.21.15.184:6834', 'nonce': 1701092387}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:56:03.639 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:56:03.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T00:56:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.820+0000 7f7e68f9d700 1 Processor -- start 2023-10-08T00:56:03.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.820+0000 7f7e68f9d700 1 -- start start 2023-10-08T00:56:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.821+0000 7f7e68f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e64157770 0x7f7e64150180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.821+0000 7f7e68f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6406e2a0 con 0x7f7e640c7150 2023-10-08T00:56:03.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.821+0000 7f7e68f9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e64157b40 con 0x7f7e640c6d00 2023-10-08T00:56:03.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.821+0000 7f7e68f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6406b2e0 con 0x7f7e64157770 2023-10-08T00:56:03.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.821+0000 7f7e637fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e64157770 0x7f7e64150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.821+0000 7f7e63fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7e640c7150 0x7f7e6406ee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54382/0 (socket says 172.21.15.119:54382) 2023-10-08T00:56:03.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.821+0000 7f7e63fff700 1 -- 172.21.15.119:0/1778125808 learned_addr learned my addr 172.21.15.119:0/1778125808 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:03.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e627fc700 1 -- 172.21.15.119:0/1778125808 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672052983 0 0) 0x7f7e6406e2a0 con 0x7f7e640c7150 2023-10-08T00:56:03.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e627fc700 1 -- 172.21.15.119:0/1778125808 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e4c003570 con 0x7f7e640c7150 2023-10-08T00:56:03.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e637fe700 1 -- 172.21.15.119:0/1778125808 >> v1:172.21.15.184:6789/0 conn(0x7f7e640c6d00 legacy=0x7f7e6406e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e637fe700 1 -- 172.21.15.119:0/1778125808 >> v1:172.21.15.119:6789/0 conn(0x7f7e640c7150 legacy=0x7f7e6406ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e637fe700 1 -- 172.21.15.119:0/1778125808 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e640c78e0 con 0x7f7e64157770 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e637fe700 1 --2- 172.21.15.119:0/1778125808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e64157770 0x7f7e64150180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e540098e0 tx=0x7f7e540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8f05508511225c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e627fc700 1 -- 172.21.15.119:0/1778125808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400b640 con 0x7f7e64157770 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e627fc700 1 -- 172.21.15.119:0/1778125808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5400b7a0 con 0x7f7e64157770 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.822+0000 7f7e627fc700 1 -- 172.21.15.119:0/1778125808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5400bb40 con 0x7f7e64157770 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.823+0000 7f7e68f9d700 1 -- 172.21.15.119:0/1778125808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e64157770 msgr2=0x7f7e64150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.823+0000 7f7e68f9d700 1 --2- 172.21.15.119:0/1778125808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e64157770 0x7f7e64150180 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7e540098e0 tx=0x7f7e540091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.823+0000 7f7e68f9d700 1 -- 172.21.15.119:0/1778125808 shutdown_connections 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.823+0000 7f7e68f9d700 1 --2- 172.21.15.119:0/1778125808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e64157770 0x7f7e64150180 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.823+0000 7f7e68f9d700 1 -- 172.21.15.119:0/1778125808 >> 172.21.15.119:0/1778125808 conn(0x7f7e640cc1f0 msgr2=0x7f7e640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:03.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.823+0000 7f7e68f9d700 1 -- 172.21.15.119:0/1778125808 shutdown_connections 2023-10-08T00:56:03.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.823+0000 7f7e68f9d700 1 -- 172.21.15.119:0/1778125808 wait complete. 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.824+0000 7f7e68f9d700 1 Processor -- start 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.824+0000 7f7e68f9d700 1 -- start start 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.825+0000 7f7e68f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e640c6d00 0x7f7e6414b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.825+0000 7f7e68f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e640c7150 0x7f7e640c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.825+0000 7f7e68f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e64157770 0x7f7e640c1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.825+0000 7f7e68f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6414c980 con 0x7f7e640c7150 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.825+0000 7f7e68f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e641546c0 con 0x7f7e64157770 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.825+0000 7f7e68f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e641699b0 con 0x7f7e640c6d00 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.825+0000 7f7e62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e640c7150 0x7f7e640c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e640c7150 0x7f7e640c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33344/0 (socket says 172.21.15.119:33344) 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e62ffd700 1 -- 172.21.15.119:0/3796550389 learned_addr learned my addr 172.21.15.119:0/3796550389 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e63fff700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e64157770 0x7f7e640c1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e637fe700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e640c6d00 0x7f7e6414b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e62ffd700 1 -- 172.21.15.119:0/3796550389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e640c6d00 msgr2=0x7f7e6414b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e640c6d00 0x7f7e6414b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e62ffd700 1 -- 172.21.15.119:0/3796550389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e64157770 msgr2=0x7f7e640c1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e64157770 0x7f7e640c1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e62ffd700 1 -- 172.21.15.119:0/3796550389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e54009420 con 0x7f7e640c7150 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e640c7150 0x7f7e640c1ab0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c510 tx=0x7f7e5800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e4b7fe700 1 -- 172.21.15.119:0/3796550389 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58011590 con 0x7f7e640c7150 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e4b7fe700 1 -- 172.21.15.119:0/3796550389 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58011710 con 0x7f7e640c7150 2023-10-08T00:56:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e4b7fe700 1 -- 172.21.15.119:0/3796550389 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58011ad0 con 0x7f7e640c7150 2023-10-08T00:56:03.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.826+0000 7f7e68f9d700 1 -- 172.21.15.119:0/3796550389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e640c26c0 con 0x7f7e640c7150 2023-10-08T00:56:03.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.827+0000 7f7e68f9d700 1 -- 172.21.15.119:0/3796550389 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e640c2c80 con 0x7f7e640c7150 2023-10-08T00:56:03.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.828+0000 7f7e4b7fe700 1 -- 172.21.15.119:0/3796550389 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e5801b3b0 con 0x7f7e640c7150 2023-10-08T00:56:03.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.829+0000 7f7e4b7fe700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7e50066f30 0x7f7e500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.829+0000 7f7e4b7fe700 1 -- 172.21.15.119:0/3796550389 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e58083960 con 0x7f7e640c7150 2023-10-08T00:56:03.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.830+0000 7f7e637fe700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7e50066f30 0x7f7e500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.830+0000 7f7e637fe700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7e50066f30 0x7f7e500693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e540093f0 tx=0x7f7e54017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.831+0000 7f7e78908b80 1 Processor -- start 2023-10-08T00:56:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.831+0000 7f7e78908b80 1 -- start start 2023-10-08T00:56:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 0x561d9e6427d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e663850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e618b80 0x561d9e618ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e78908b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9e56fa70 con 0x561d9e6165f0 2023-10-08T00:56:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e78908b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9e56d270 con 0x561d9e618b80 2023-10-08T00:56:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e78908b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d9e569060 con 0x561d9e663480 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e663850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e663850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33346/0 (socket says 172.21.15.119:33346) 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.832+0000 7f7e62ffd700 1 -- 172.21.15.119:0/2062393506 learned_addr learned my addr 172.21.15.119:0/2062393506 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e637fe700 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 0x561d9e6427d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e63fff700 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e618b80 0x561d9e618ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e62ffd700 1 -- 172.21.15.119:0/2062393506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 msgr2=0x561d9e6427d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 0x561d9e6427d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e62ffd700 1 -- 172.21.15.119:0/2062393506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e618b80 msgr2=0x561d9e618ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e618b80 0x561d9e618ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e62ffd700 1 -- 172.21.15.119:0/2062393506 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d9e403270 con 0x561d9e6165f0 2023-10-08T00:56:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e663850 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7e58054780 tx=0x7f7e58081d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9269ba39ecdde47e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e48ff9700 1 -- 172.21.15.119:0/2062393506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e580547f0 con 0x561d9e6165f0 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e48ff9700 1 -- 172.21.15.119:0/2062393506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58057a40 con 0x561d9e6165f0 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.833+0000 7f7e48ff9700 1 -- 172.21.15.119:0/2062393506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5805b790 con 0x561d9e6165f0 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 -- 172.21.15.119:0/2062393506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 msgr2=0x561d9e663850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e663850 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7e58054780 tx=0x7f7e58081d40 comp rx=0 tx=0).stop 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 -- 172.21.15.119:0/2062393506 shutdown_connections 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 0x561d9e6427d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e618b80 0x561d9e618ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 --2- 172.21.15.119:0/2062393506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e663850 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 -- 172.21.15.119:0/2062393506 >> 172.21.15.119:0/2062393506 conn(0x561d9e61a0c0 msgr2=0x561d9e61a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 -- 172.21.15.119:0/2062393506 shutdown_connections 2023-10-08T00:56:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.834+0000 7f7e78908b80 1 -- 172.21.15.119:0/2062393506 wait complete. 2023-10-08T00:56:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.835+0000 7f7e78908b80 1 -- start start 2023-10-08T00:56:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.835+0000 7f7e78908b80 1 Processor -- start 2023-10-08T00:56:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e672710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e618b80 0x561d9e643b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 0x561d9e647a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e78908b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9e514890 con 0x561d9e6165f0 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e78908b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9e52fea0 con 0x561d9e663480 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e78908b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d9e3ebe30 con 0x561d9e618b80 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e78908b80 10 client.0 _ll_register_callbacks cb 0x561d9e5dc2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e637fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e672710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e637fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e672710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33352/0 (socket says 172.21.15.119:33352) 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e637fe700 1 -- 172.21.15.119:0/33500072 learned_addr learned my addr 172.21.15.119:0/33500072 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e63fff700 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 0x561d9e647a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e618b80 0x561d9e643b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e637fe700 1 -- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e618b80 msgr2=0x561d9e643b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e637fe700 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e618b80 0x561d9e643b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e637fe700 1 -- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 msgr2=0x561d9e647a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e637fe700 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 0x561d9e647a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e637fe700 1 -- 172.21.15.119:0/33500072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e640c2c80 con 0x561d9e6165f0 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.836+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e618b80 0x561d9e643b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e637fe700 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e672710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009f80 tx=0x7f7e54021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f3c816f7dcea00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e41ffb700 1 -- 172.21.15.119:0/33500072 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5402f070 con 0x561d9e6165f0 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e41ffb700 1 -- 172.21.15.119:0/33500072 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e5401a630 con 0x561d9e6165f0 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 1 -- 172.21.15.119:0/33500072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d9e60dcd0 con 0x561d9e6165f0 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e41ffb700 1 -- 172.21.15.119:0/33500072 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e54021070 con 0x561d9e6165f0 2023-10-08T00:56:03.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 10 client.4659 fetch_fsmap learned FSMap version 7 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 1 -- 172.21.15.119:0/33500072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x561d9e3a5fe0 con 0x561d9e6165f0 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e41ffb700 1 -- 172.21.15.119:0/33500072 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5403c070 con 0x561d9e6165f0 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e41ffb700 1 -- 172.21.15.119:0/33500072 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7e5401aa00 con 0x561d9e6165f0 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 10 client.4659 fetch_fsmap finished waiting for FSMap version 7 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 10 client.4659 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 20 client.4659 populate_metadata read hostname 'smithi119' 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] conn(0x561d9e642d00 0x561d9e6430d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 4 client.4659 mds_command: new command op to 4282 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.837+0000 7f7e78908b80 1 -- 172.21.15.119:0/33500072 --> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d9e5eaaa0 con 0x561d9e642d00 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.838+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] conn(0x561d9e642d00 0x561d9e6430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.838+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] conn(0x561d9e642d00 0x561d9e6430d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855814aa9e0b8d1b server_cookie=9f450a455e38f781 in_seq=0 out_seq=0 2023-10-08T00:56:03.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.838+0000 7f7e41ffb700 10 client.4659 ms_handle_connect on v2:172.21.15.184:6832/1701092387 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.839+0000 7f7e41ffb700 1 -- 172.21.15.119:0/33500072 <== mds.0 v2:172.21.15.184:6832/1701092387 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x561d9e5eaaa0 con 0x561d9e642d00 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.839+0000 7f7e41ffb700 10 client.4659 handle_command_reply: tid=0 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.839+0000 7f7e78908b80 1 client.4659 shutdown 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 -- 172.21.15.119:0/33500072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 msgr2=0x561d9e672710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e672710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7e54009f80 tx=0x7f7e54021000 comp rx=0 tx=0).stop 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 -- 172.21.15.119:0/33500072 shutdown_connections 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] conn(0x561d9e642d00 0x561d9e6430d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 0x561d9e647a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e6165f0 0x561d9e672710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e41ffb700 0 client.4659 ms_handle_reset on v2:172.21.15.184:6832/1701092387 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 --2- 172.21.15.119:0/33500072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e618b80 0x561d9e643b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 -- 172.21.15.119:0/33500072 >> 172.21.15.119:0/33500072 conn(0x561d9e61a0c0 msgr2=0x561d9e61c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 -- 172.21.15.119:0/33500072 shutdown_connections 2023-10-08T00:56:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 1 -- 172.21.15.119:0/33500072 wait complete. 2023-10-08T00:56:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.840+0000 7f7e78908b80 20 client.4659 trim_cache size 0 max 16384 2023-10-08T00:56:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.857+0000 7f7e78908b80 1 Processor -- start 2023-10-08T00:56:03.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.857+0000 7f7e78908b80 1 -- start start 2023-10-08T00:56:03.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e6165f0 0x561d9e615220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e679410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 0x561d9e615c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e78908b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9e646c20 con 0x561d9e618b80 2023-10-08T00:56:03.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e78908b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9e6119e0 con 0x561d9e6165f0 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e78908b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d9e679950 con 0x561d9e663480 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e679410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e679410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33364/0 (socket says 172.21.15.119:33364) 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.858+0000 7f7e62ffd700 1 -- 172.21.15.119:0/4073065481 learned_addr learned my addr 172.21.15.119:0/4073065481 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e62ffd700 1 -- 172.21.15.119:0/4073065481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 msgr2=0x561d9e615c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/4073065481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 0x561d9e615c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e62ffd700 1 -- 172.21.15.119:0/4073065481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e6165f0 msgr2=0x561d9e615220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/4073065481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e6165f0 0x561d9e615220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e62ffd700 1 -- 172.21.15.119:0/4073065481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d9e537780 con 0x561d9e618b80 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/4073065481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e679410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e58054780 tx=0x7f7e58057000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44a6a81b20615b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:03.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e497fa700 1 -- 172.21.15.119:0/4073065481 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58052930 con 0x561d9e618b80 2023-10-08T00:56:03.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e497fa700 1 -- 172.21.15.119:0/4073065481 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58011c50 con 0x561d9e618b80 2023-10-08T00:56:03.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.859+0000 7f7e497fa700 1 -- 172.21.15.119:0/4073065481 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58011dc0 con 0x561d9e618b80 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 -- 172.21.15.119:0/4073065481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 msgr2=0x561d9e679410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 --2- 172.21.15.119:0/4073065481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e679410 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7e58054780 tx=0x7f7e58057000 comp rx=0 tx=0).stop 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 -- 172.21.15.119:0/4073065481 shutdown_connections 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 --2- 172.21.15.119:0/4073065481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e663480 0x561d9e615c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 --2- 172.21.15.119:0/4073065481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e6165f0 0x561d9e615220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 --2- 172.21.15.119:0/4073065481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e679410 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 -- 172.21.15.119:0/4073065481 >> 172.21.15.119:0/4073065481 conn(0x561d9e61a0c0 msgr2=0x561d9e61a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 -- 172.21.15.119:0/4073065481 shutdown_connections 2023-10-08T00:56:03.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 -- 172.21.15.119:0/4073065481 wait complete. 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.860+0000 7f7e78908b80 1 -- start start 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.861+0000 7f7e78908b80 1 Processor -- start 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e6165f0 0x561d9e6c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e6c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e78908b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 0x561d9e661f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e78908b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9e6c6220 con 0x561d9e618b80 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e78908b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d9e6bf230 con 0x561d9e663480 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e78908b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d9e641f50 con 0x561d9e6165f0 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e78908b80 10 client.0 _ll_register_callbacks cb 0x561d9e5dc2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e6c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e6c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33380/0 (socket says 172.21.15.119:33380) 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 -- 172.21.15.119:0/1151952680 learned_addr learned my addr 172.21.15.119:0/1151952680 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 -- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e6165f0 msgr2=0x561d9e6c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e6165f0 0x561d9e6c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 -- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 msgr2=0x561d9e661f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 0x561d9e661f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 -- 172.21.15.119:0/1151952680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d9e403270 con 0x561d9e618b80 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e62ffd700 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e6c55f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7e58001050 tx=0x7f7e5807d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8777439eb015fdd0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.862+0000 7f7e41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e41ffb700 1 -- 172.21.15.119:0/1151952680 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58052a00 con 0x561d9e618b80 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e41ffb700 1 -- 172.21.15.119:0/1151952680 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7e58011dc0 con 0x561d9e618b80 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 1 -- 172.21.15.119:0/1151952680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e58057a40 con 0x561d9e618b80 2023-10-08T00:56:03.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e41ffb700 1 -- 172.21.15.119:0/1151952680 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e5805b510 con 0x561d9e618b80 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 10 client.4665 fetch_fsmap learned FSMap version 7 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 1 -- 172.21.15.119:0/1151952680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x561d9e51e470 con 0x561d9e618b80 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e41ffb700 1 -- 172.21.15.119:0/1151952680 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58074080 con 0x561d9e618b80 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e41ffb700 1 -- 172.21.15.119:0/1151952680 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 7) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7e58070180 con 0x561d9e618b80 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 10 client.4665 fetch_fsmap finished waiting for FSMap version 7 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 10 client.4665 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 20 client.4665 populate_metadata read hostname 'smithi119' 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] conn(0x561d9e63df30 0x561d9e6624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 4 client.4665 mds_command: new command op to 4282 tid=0[{"prefix": "status"}] 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e78908b80 1 -- 172.21.15.119:0/1151952680 --> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561d9e50be00 con 0x561d9e63df30 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.863+0000 7f7e637fe700 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] conn(0x561d9e63df30 0x561d9e6624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:03.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e41ffb700 10 client.4665 ms_handle_connect on v2:172.21.15.184:6832/1701092387 2023-10-08T00:56:03.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e637fe700 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] conn(0x561d9e63df30 0x561d9e6624b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfe32994f5f4ba36 server_cookie=56f6175d9118f953 in_seq=0 out_seq=0 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e41ffb700 1 -- 172.21.15.119:0/1151952680 <== mds.0 v2:172.21.15.184:6832/1701092387 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x561d9e50be00 con 0x561d9e63df30 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e41ffb700 10 client.4665 handle_command_reply: tid=0 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 client.4665 shutdown 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 -- 172.21.15.119:0/1151952680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 msgr2=0x561d9e6c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e6c55f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7e58001050 tx=0x7f7e5807d000 comp rx=0 tx=0).stop 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 -- 172.21.15.119:0/1151952680 shutdown_connections 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:6832/1701092387,v1:172.21.15.184:6834/1701092387] conn(0x561d9e63df30 0x561d9e6624b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x561d9e6165f0 0x561d9e6c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x561d9e663480 0x561d9e661f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e41ffb700 0 client.4665 ms_handle_reset on v2:172.21.15.184:6832/1701092387 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 --2- 172.21.15.119:0/1151952680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561d9e618b80 0x561d9e6c55f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 -- 172.21.15.119:0/1151952680 >> 172.21.15.119:0/1151952680 conn(0x561d9e61a0c0 msgr2=0x561d9e6489b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.864+0000 7f7e78908b80 1 -- 172.21.15.119:0/1151952680 shutdown_connections 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e78908b80 1 -- 172.21.15.119:0/1151952680 wait complete. 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e78908b80 20 client.4665 trim_cache size 0 max 16384 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T00:56:03.871 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4282, 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 16.375446720999999, 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 7, 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 20, 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 20, 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 21.390232563000001 2023-10-08T00:56:03.872 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T00:56:03.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 -- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7e50066f30 msgr2=0x7f7e500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7e50066f30 0x7f7e500693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7e540093f0 tx=0x7f7e54017000 comp rx=0 tx=0).stop 2023-10-08T00:56:03.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 -- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e640c7150 msgr2=0x7f7e640c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:03.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e640c7150 0x7f7e640c1ab0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800c510 tx=0x7f7e5800c820 comp rx=0 tx=0).stop 2023-10-08T00:56:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 -- 172.21.15.119:0/3796550389 shutdown_connections 2023-10-08T00:56:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e640c6d00 0x7f7e6414b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7e50066f30 0x7f7e500693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e64157770 0x7f7e640c1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 --2- 172.21.15.119:0/3796550389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e640c7150 0x7f7e640c1ab0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 -- 172.21.15.119:0/3796550389 >> 172.21.15.119:0/3796550389 conn(0x7f7e640cc1f0 msgr2=0x7f7e640b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 -- 172.21.15.119:0/3796550389 shutdown_connections 2023-10-08T00:56:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:03.865+0000 7f7e68f9d700 1 -- 172.21.15.119:0/3796550389 wait complete. 2023-10-08T00:56:03.883 INFO:teuthology.run_tasks:Running task kclient... 2023-10-08T00:56:03.904 INFO:tasks.kclient:Mounting kernel clients... 2023-10-08T00:56:03.904 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-08T00:56:03.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:03.904 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T00:56:03.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:03.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-10-08T00:56:04.045 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2023-10-08T00:56:04.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:56:04.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:04.048 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T00:56:04.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:04.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-10-08T00:56:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2023-10-08T00:56:04.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:56:04.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:04.191 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-08T00:56:04.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:04.232 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-10-08T00:56:04.338 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2023-10-08T00:56:04.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:56:04.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:04.341 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-08T00:56:04.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:04.378 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-10-08T00:56:04.480 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2023-10-08T00:56:04.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T00:56:04.483 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-10-08T00:56:04.483 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:56:04.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.663+0000 7fbf961ff700 1 Processor -- start 2023-10-08T00:56:04.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.663+0000 7fbf961ff700 1 -- start start 2023-10-08T00:56:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.664+0000 7fbf961ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf90159890 0x7fbf9015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:04.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.664+0000 7fbf961ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbf90159c60 0x7fbf9015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.664+0000 7fbf961ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf9015a160 con 0x7fbf9015db10 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.664+0000 7fbf961ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf9015df50 con 0x7fbf90159890 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.664+0000 7fbf961ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf900d13a0 con 0x7fbf90159c60 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.664+0000 7fbf8f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf90159890 0x7fbf9015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.664+0000 7fbf8f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf90159890 0x7fbf9015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33382/0 (socket says 172.21.15.119:33382) 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.664+0000 7fbf8f7fe700 1 -- 172.21.15.119:0/92292743 learned_addr learned my addr 172.21.15.119:0/92292743 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8effd700 1 --2- 172.21.15.119:0/92292743 >> v2:172.21.15.184:3300/0 conn(0x7fbf90159c60 0x7fbf9015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8f7fe700 1 -- 172.21.15.119:0/92292743 >> v2:172.21.15.184:3300/0 conn(0x7fbf90159c60 msgr2=0x7fbf9015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8f7fe700 1 --2- 172.21.15.119:0/92292743 >> v2:172.21.15.184:3300/0 conn(0x7fbf90159c60 0x7fbf9015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8f7fe700 1 -- 172.21.15.119:0/92292743 >> v1:172.21.15.184:6789/0 conn(0x7fbf9015db10 legacy=0x7fbf90069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:04.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8f7fe700 1 -- 172.21.15.119:0/92292743 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9015e140 con 0x7fbf90159890 2023-10-08T00:56:04.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8f7fe700 1 --2- 172.21.15.119:0/92292743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf90159890 0x7fbf9015abd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbf900b67f0 tx=0x7fbf80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ccdc2289de080f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:04.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8e7fc700 1 -- 172.21.15.119:0/92292743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8000b860 con 0x7fbf90159890 2023-10-08T00:56:04.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8e7fc700 1 -- 172.21.15.119:0/92292743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf8000b9c0 con 0x7fbf90159890 2023-10-08T00:56:04.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.665+0000 7fbf8e7fc700 1 -- 172.21.15.119:0/92292743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8000bd60 con 0x7fbf90159890 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.666+0000 7fbf961ff700 1 -- 172.21.15.119:0/92292743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf90159890 msgr2=0x7fbf9015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.666+0000 7fbf961ff700 1 --2- 172.21.15.119:0/92292743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf90159890 0x7fbf9015abd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbf900b67f0 tx=0x7fbf80009250 comp rx=0 tx=0).stop 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.666+0000 7fbf961ff700 1 -- 172.21.15.119:0/92292743 shutdown_connections 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.666+0000 7fbf961ff700 1 --2- 172.21.15.119:0/92292743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf90159890 0x7fbf9015abd0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.666+0000 7fbf961ff700 1 --2- 172.21.15.119:0/92292743 >> v2:172.21.15.184:3300/0 conn(0x7fbf90159c60 0x7fbf9015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.666+0000 7fbf961ff700 1 -- 172.21.15.119:0/92292743 >> 172.21.15.119:0/92292743 conn(0x7fbf90068e20 msgr2=0x7fbf900d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.666+0000 7fbf961ff700 1 -- 172.21.15.119:0/92292743 shutdown_connections 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.666+0000 7fbf961ff700 1 -- 172.21.15.119:0/92292743 wait complete. 2023-10-08T00:56:04.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.667+0000 7fbf961ff700 1 Processor -- start 2023-10-08T00:56:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.667+0000 7fbf961ff700 1 -- start start 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.668+0000 7fbf961ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf90159890 0x7fbf900ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.668+0000 7fbf961ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf90159c60 0x7fbf900cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.668+0000 7fbf961ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9015db10 0x7fbf901587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.668+0000 7fbf961ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900cfeb0 con 0x7fbf9015db10 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.668+0000 7fbf961ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900c7bf0 con 0x7fbf90159890 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.668+0000 7fbf961ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf900cf3b0 con 0x7fbf90159c60 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9015db10 0x7fbf901587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9015db10 0x7fbf901587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33388/0 (socket says 172.21.15.119:33388) 2023-10-08T00:56:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 -- 172.21.15.119:0/2694501810 learned_addr learned my addr 172.21.15.119:0/2694501810 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8f7fe700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf90159890 0x7fbf900ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8effd700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf90159c60 0x7fbf900cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 -- 172.21.15.119:0/2694501810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf90159c60 msgr2=0x7fbf900cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf90159c60 0x7fbf900cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 -- 172.21.15.119:0/2694501810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf90159890 msgr2=0x7fbf900ca1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf90159890 0x7fbf900ca1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 -- 172.21.15.119:0/2694501810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf800094f0 con 0x7fbf9015db10 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf8ffff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9015db10 0x7fbf901587d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c840 tx=0x7fbf88001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf777fe700 1 -- 172.21.15.119:0/2694501810 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88012630 con 0x7fbf9015db10 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf777fe700 1 -- 172.21.15.119:0/2694501810 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf880127b0 con 0x7fbf9015db10 2023-10-08T00:56:04.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf777fe700 1 -- 172.21.15.119:0/2694501810 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88012b70 con 0x7fbf9015db10 2023-10-08T00:56:04.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.669+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf900cf610 con 0x7fbf9015db10 2023-10-08T00:56:04.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.670+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf900cfc50 con 0x7fbf9015db10 2023-10-08T00:56:04.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.671+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf900b4550 con 0x7fbf9015db10 2023-10-08T00:56:04.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.671+0000 7fbf777fe700 1 -- 172.21.15.119:0/2694501810 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf8801a3a0 con 0x7fbf9015db10 2023-10-08T00:56:04.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.672+0000 7fbf777fe700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:04.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.672+0000 7fbf777fe700 1 -- 172.21.15.119:0/2694501810 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbf88084700 con 0x7fbf9015db10 2023-10-08T00:56:04.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.672+0000 7fbf8f7fe700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:04.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.673+0000 7fbf8f7fe700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf7c066f30 0x7fbf7c0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf900b67f0 tx=0x7fbf80017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:04.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.676+0000 7fbf777fe700 1 -- 172.21.15.119:0/2694501810 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf900b4550 con 0x7fbf9015db10 2023-10-08T00:56:04.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.910+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbf900b4550 con 0x7fbf9015db10 2023-10-08T00:56:04.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.911+0000 7fbf777fe700 1 -- 172.21.15.119:0/2694501810 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbf900b4550 con 0x7fbf9015db10 2023-10-08T00:56:04.912 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf7c066f30 msgr2=0x7fbf7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf7c066f30 0x7fbf7c0693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf900b67f0 tx=0x7fbf80017000 comp rx=0 tx=0).stop 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9015db10 msgr2=0x7fbf901587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9015db10 0x7fbf901587d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800c840 tx=0x7fbf88001670 comp rx=0 tx=0).stop 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 shutdown_connections 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbf7c066f30 0x7fbf7c0693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9015db10 0x7fbf901587d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf90159890 0x7fbf900ca1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 --2- 172.21.15.119:0/2694501810 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf90159c60 0x7fbf900cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 >> 172.21.15.119:0/2694501810 conn(0x7fbf90068e20 msgr2=0x7fbf90153160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 shutdown_connections 2023-10-08T00:56:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:04.913+0000 7fbf961ff700 1 -- 172.21.15.119:0/2694501810 wait complete. 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:56:04.932 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:56:04.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:04.933 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T00:56:04.968 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:04.969 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42287sec preferred_lft 42287sec 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T00:56:04.970 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:04.971 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T00:56:04.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:04.971 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:04.971 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:04.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2023-10-08T00:56:04.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2023-10-08T00:56:04.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2023-10-08T00:56:04.971 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T00:56:04.971 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:05.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:05.410 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:56:05.472 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T00:56:05.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:05.475 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T00:56:05.524 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T00:56:05.524 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:56:05.524 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:56:05.524 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:56:05.525 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:56:05.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:05.525 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:05.526 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:05.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:56:05.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:56:05.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:56:05.526 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:06.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:06.230 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T00:56:06.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:06.267 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-10-08T00:56:06.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:06.342 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:06.343 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:06.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:56:06.343 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-08T00:56:06.343 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:06.506 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-08T00:56:06.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:06.506 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:06.507 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:06.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-08T00:56:06.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T00:56:06.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-08T00:56:06.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-08T00:56:06.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-08T00:56:06.507 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:06.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:06.760 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:06.760 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:06.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T00:56:06.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T00:56:06.760 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:06.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:06.917 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:56:06.953 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:56:06.953 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:06.954 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:56:07.029 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:56:07.029 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:56:07.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:07.183 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:07.183 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:07.615 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:07.615 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:07.615 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T00:56:07.615 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T00:56:07.615 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T00:56:07.622 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:56:07.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:07.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:56:07.684 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-10-08T00:56:07.684 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:56:07.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.865+0000 7fe878d9d700 1 Processor -- start 2023-10-08T00:56:07.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.865+0000 7fe878d9d700 1 -- start start 2023-10-08T00:56:07.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.866+0000 7fe878d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe874159420 0x7fe8740b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:07.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.866+0000 7fe878d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740b2980 con 0x7fe87415d230 2023-10-08T00:56:07.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.866+0000 7fe878d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740b11d0 con 0x7fe8741597f0 2023-10-08T00:56:07.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.866+0000 7fe878d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8740b14b0 con 0x7fe874159420 2023-10-08T00:56:07.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe873fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe87415d230 0x7fe87415d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39452/0 (socket says 172.21.15.119:39452) 2023-10-08T00:56:07.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe873fff700 1 -- 172.21.15.119:0/1921356490 learned_addr learned my addr 172.21.15.119:0/1921356490 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:07.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe8737fe700 1 --2- 172.21.15.119:0/1921356490 >> v2:172.21.15.184:3300/0 conn(0x7fe874159420 0x7fe8740b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:07.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe8727fc700 1 -- 172.21.15.119:0/1921356490 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554901251 0 0) 0x7fe8740b2980 con 0x7fe87415d230 2023-10-08T00:56:07.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe8727fc700 1 -- 172.21.15.119:0/1921356490 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85c003570 con 0x7fe87415d230 2023-10-08T00:56:07.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe8727fc700 1 -- 172.21.15.119:0/1921356490 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 168873225 0 0) 0x7fe85c003570 con 0x7fe87415d230 2023-10-08T00:56:07.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe8737fe700 1 -- 172.21.15.119:0/1921356490 >> v1:172.21.15.184:6789/0 conn(0x7fe8741597f0 legacy=0x7fe8740b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:07.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe8737fe700 1 -- 172.21.15.119:0/1921356490 >> v1:172.21.15.119:6789/0 conn(0x7fe87415d230 legacy=0x7fe87415d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:07.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.868+0000 7fe8737fe700 1 -- 172.21.15.119:0/1921356490 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740b1790 con 0x7fe874159420 2023-10-08T00:56:07.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe8737fe700 1 --2- 172.21.15.119:0/1921356490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 0x7fe8740b5f40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe8640098e0 tx=0x7fe8640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbfcf562e8b3f959 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe8727fc700 1 -- 172.21.15.119:0/1921356490 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0017f0 con 0x7fe874159420 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe8727fc700 1 -- 172.21.15.119:0/1921356490 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864011420 con 0x7fe874159420 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe8727fc700 1 -- 172.21.15.119:0/1921356490 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8640117c0 con 0x7fe874159420 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe878d9d700 1 -- 172.21.15.119:0/1921356490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 msgr2=0x7fe8740b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe878d9d700 1 --2- 172.21.15.119:0/1921356490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 0x7fe8740b5f40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe8640098e0 tx=0x7fe8640091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe878d9d700 1 -- 172.21.15.119:0/1921356490 shutdown_connections 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe878d9d700 1 --2- 172.21.15.119:0/1921356490 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 0x7fe8740b5f40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe878d9d700 1 -- 172.21.15.119:0/1921356490 >> 172.21.15.119:0/1921356490 conn(0x7fe8740d0660 msgr2=0x7fe8740d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe878d9d700 1 -- 172.21.15.119:0/1921356490 shutdown_connections 2023-10-08T00:56:07.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.869+0000 7fe878d9d700 1 -- 172.21.15.119:0/1921356490 wait complete. 2023-10-08T00:56:07.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.870+0000 7fe878d9d700 1 Processor -- start 2023-10-08T00:56:07.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.870+0000 7fe878d9d700 1 -- start start 2023-10-08T00:56:07.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe878d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 0x7fe8740b5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:07.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe878d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8741597f0 0x7fe874156260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:07.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe878d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe87415d230 0x7fe8741567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:07.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe878d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740c55a0 con 0x7fe87415d230 2023-10-08T00:56:07.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe878d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740c5840 con 0x7fe874159420 2023-10-08T00:56:07.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe878d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740c5b20 con 0x7fe8741597f0 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe873fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe87415d230 0x7fe8741567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe873fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe87415d230 0x7fe8741567a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54828/0 (socket says 172.21.15.119:54828) 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe873fff700 1 -- 172.21.15.119:0/2457443299 learned_addr learned my addr 172.21.15.119:0/2457443299 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe8737fe700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 0x7fe8740b5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.871+0000 7fe872ffd700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8741597f0 0x7fe874156260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe873fff700 1 -- 172.21.15.119:0/2457443299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8741597f0 msgr2=0x7fe874156260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe873fff700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8741597f0 0x7fe874156260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe873fff700 1 -- 172.21.15.119:0/2457443299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 msgr2=0x7fe8740b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe873fff700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 0x7fe8740b5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:07.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe873fff700 1 -- 172.21.15.119:0/2457443299 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe864009420 con 0x7fe87415d230 2023-10-08T00:56:07.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe873fff700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe87415d230 0x7fe8741567a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00cb90 tx=0x7fe86c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:07.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe85b7fe700 1 -- 172.21.15.119:0/2457443299 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c004dc0 con 0x7fe87415d230 2023-10-08T00:56:07.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe85b7fe700 1 -- 172.21.15.119:0/2457443299 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c00a5d0 con 0x7fe87415d230 2023-10-08T00:56:07.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe85b7fe700 1 -- 172.21.15.119:0/2457443299 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00a990 con 0x7fe87415d230 2023-10-08T00:56:07.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.872+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8740c5e00 con 0x7fe87415d230 2023-10-08T00:56:07.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.874+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8740b6000 con 0x7fe87415d230 2023-10-08T00:56:07.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.874+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8740c6320 con 0x7fe87415d230 2023-10-08T00:56:07.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.874+0000 7fe85b7fe700 1 -- 172.21.15.119:0/2457443299 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe86c007150 con 0x7fe87415d230 2023-10-08T00:56:07.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.875+0000 7fe85b7fe700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe860066f30 0x7fe8600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:07.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.875+0000 7fe85b7fe700 1 -- 172.21.15.119:0/2457443299 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe86c083bd0 con 0x7fe87415d230 2023-10-08T00:56:07.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.875+0000 7fe8737fe700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe860066f30 0x7fe8600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:07.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.877+0000 7fe8737fe700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe860066f30 0x7fe8600693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8640091e0 tx=0x7fe864016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:07.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:07.880+0000 7fe85b7fe700 1 -- 172.21.15.119:0/2457443299 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8740c6320 con 0x7fe87415d230 2023-10-08T00:56:08.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.120+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe8740c6320 con 0x7fe87415d230 2023-10-08T00:56:08.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.120+0000 7fe85b7fe700 1 -- 172.21.15.119:0/2457443299 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe8740c6320 con 0x7fe87415d230 2023-10-08T00:56:08.121 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:56:08.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe860066f30 msgr2=0x7fe8600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:08.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe860066f30 0x7fe8600693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe8640091e0 tx=0x7fe864016000 comp rx=0 tx=0).stop 2023-10-08T00:56:08.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe87415d230 msgr2=0x7fe8741567a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:08.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe87415d230 0x7fe8741567a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe86c00cb90 tx=0x7fe86c001640 comp rx=0 tx=0).stop 2023-10-08T00:56:08.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 shutdown_connections 2023-10-08T00:56:08.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe860066f30 0x7fe8600693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:08.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe87415d230 0x7fe8741567a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:08.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe874159420 0x7fe8740b5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:08.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 --2- 172.21.15.119:0/2457443299 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8741597f0 0x7fe874156260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:08.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 >> 172.21.15.119:0/2457443299 conn(0x7fe8740d0660 msgr2=0x7fe874156fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:08.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 shutdown_connections 2023-10-08T00:56:08.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:08.123+0000 7fe878d9d700 1 -- 172.21.15.119:0/2457443299 wait complete. 2023-10-08T00:56:08.140 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T00:56:08.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:56:08.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T00:56:08.141 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-08T00:56:08.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-08T00:56:08.141 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T00:56:08.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T00:56:08.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:56:08.141 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-08T00:56:08.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.141 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T00:56:08.178 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:08.178 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:08.178 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:08.179 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:08.180 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:56:08.180 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:08.180 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:56:08.180 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42283sec preferred_lft 42283sec 2023-10-08T00:56:08.180 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T00:56:08.180 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link tentative 2023-10-08T00:56:08.181 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.182 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:56:08.277 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T00:56:08.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.280 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T00:56:08.318 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T00:56:08.318 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:56:08.318 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:56:08.318 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:56:08.318 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:56:08.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.319 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:08.319 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:08.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:56:08.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:56:08.320 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:56:08.320 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:08.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.514 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T00:56:08.550 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:56:08.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.550 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-10-08T00:56:08.626 INFO:teuthology.orchestra.run.smithi119.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-08T00:56:08.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.626 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:08.626 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:08.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:56:08.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-08T00:56:08.627 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:08.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T00:56:08.851 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:08.851 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:08.852 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:08.852 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.852 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:08.852 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.852 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:56:08.852 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 62:86:2b:cd:93:c7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T00:56:08.852 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T00:56:08.853 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.853 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::6086:2bff:fecd:93c7/64 scope link 2023-10-08T00:56:08.853 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-08T00:56:08.915 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:08.915 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:08.915 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:08.916 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.916 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:08.916 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.916 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:56:08.916 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 62:86:2b:cd:93:c7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T00:56:08.917 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T00:56:08.917 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.917 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::6086:2bff:fecd:93c7/64 scope link 2023-10-08T00:56:08.917 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:08.918 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-08T00:56:08.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:08.919 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:08.919 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:08.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-08T00:56:08.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T00:56:08.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-08T00:56:08.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-08T00:56:08.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-08T00:56:08.919 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:09.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:09.148 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:09.148 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:09.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.1 up 2023-10-08T00:56:09.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T00:56:09.149 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:09.305 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:09.305 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-08T00:56:09.341 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-08T00:56:09.341 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:09.341 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-08T00:56:09.416 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-10-08T00:56:09.416 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:56:09.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:09.545 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:09.546 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:09.574 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:09.575 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:09.575 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T00:56:09.575 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 2@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T00:56:09.575 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T00:56:09.579 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:56:09.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:09.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-08T00:56:09.639 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-10-08T00:56:09.640 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:56:09.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.820+0000 7fc4edfff700 1 Processor -- start 2023-10-08T00:56:09.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.821+0000 7fc4edfff700 1 -- start start 2023-10-08T00:56:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4edfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc4e80cbf30 0x7fc4e80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4edfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4e80ccf90 0x7fc4e80b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4edfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e80cc300 con 0x7fc4e815a270 2023-10-08T00:56:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4edfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e8156bf0 con 0x7fc4e80ccf90 2023-10-08T00:56:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4edfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e80ae950 con 0x7fc4e80cbf30 2023-10-08T00:56:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4e6ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4e80ccf90 0x7fc4e80b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4e6ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc4e80ccf90 0x7fc4e80b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54832/0 (socket says 172.21.15.119:54832) 2023-10-08T00:56:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4e6ffd700 1 -- 172.21.15.119:0/193526373 learned_addr learned my addr 172.21.15.119:0/193526373 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.822+0000 7fc4e77fe700 1 --2- 172.21.15.119:0/193526373 >> v2:172.21.15.184:3300/0 conn(0x7fc4e80cbf30 0x7fc4e80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.823+0000 7fc4e6ffd700 1 -- 172.21.15.119:0/193526373 >> v2:172.21.15.184:3300/0 conn(0x7fc4e80cbf30 msgr2=0x7fc4e80ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.823+0000 7fc4e6ffd700 1 --2- 172.21.15.119:0/193526373 >> v2:172.21.15.184:3300/0 conn(0x7fc4e80cbf30 0x7fc4e80ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.823+0000 7fc4e6ffd700 1 -- 172.21.15.119:0/193526373 >> v1:172.21.15.184:6789/0 conn(0x7fc4e815a270 legacy=0x7fc4e80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.823+0000 7fc4e6ffd700 1 -- 172.21.15.119:0/193526373 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80ad370 con 0x7fc4e80ccf90 2023-10-08T00:56:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.823+0000 7fc4e6ffd700 1 --2- 172.21.15.119:0/193526373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80ccf90 0x7fc4e80b0d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8063d40 tx=0x7fc4dc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11780d547b866b7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.823+0000 7fc4e67fc700 1 -- 172.21.15.119:0/193526373 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00be60 con 0x7fc4e80ccf90 2023-10-08T00:56:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.823+0000 7fc4e67fc700 1 -- 172.21.15.119:0/193526373 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc00dd80 con 0x7fc4e80ccf90 2023-10-08T00:56:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.823+0000 7fc4e67fc700 1 -- 172.21.15.119:0/193526373 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0106a0 con 0x7fc4e80ccf90 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.824+0000 7fc4edfff700 1 -- 172.21.15.119:0/193526373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80ccf90 msgr2=0x7fc4e80b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.824+0000 7fc4edfff700 1 --2- 172.21.15.119:0/193526373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80ccf90 0x7fc4e80b0d10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8063d40 tx=0x7fc4dc00b4f0 comp rx=0 tx=0).stop 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.824+0000 7fc4edfff700 1 -- 172.21.15.119:0/193526373 shutdown_connections 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.824+0000 7fc4edfff700 1 --2- 172.21.15.119:0/193526373 >> v2:172.21.15.184:3300/0 conn(0x7fc4e80cbf30 0x7fc4e80ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.824+0000 7fc4edfff700 1 --2- 172.21.15.119:0/193526373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80ccf90 0x7fc4e80b0d10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.824+0000 7fc4edfff700 1 -- 172.21.15.119:0/193526373 >> 172.21.15.119:0/193526373 conn(0x7fc4e806a320 msgr2=0x7fc4e80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.824+0000 7fc4edfff700 1 -- 172.21.15.119:0/193526373 shutdown_connections 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.824+0000 7fc4edfff700 1 -- 172.21.15.119:0/193526373 wait complete. 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.825+0000 7fc4edfff700 1 Processor -- start 2023-10-08T00:56:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.825+0000 7fc4edfff700 1 -- start start 2023-10-08T00:56:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4edfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80cbf30 0x7fc4e816aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4edfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e80ccf90 0x7fc4e80c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4edfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e815a270 0x7fc4e80a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4edfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80c7a60 con 0x7fc4e80cbf30 2023-10-08T00:56:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4edfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80c6bf0 con 0x7fc4e815a270 2023-10-08T00:56:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4edfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80c6e50 con 0x7fc4e80ccf90 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4e77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80cbf30 0x7fc4e816aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4e77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80cbf30 0x7fc4e816aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54848/0 (socket says 172.21.15.119:54848) 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.826+0000 7fc4e77fe700 1 -- 172.21.15.119:0/1120481699 learned_addr learned my addr 172.21.15.119:0/1120481699 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4e77fe700 1 -- 172.21.15.119:0/1120481699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e80ccf90 msgr2=0x7fc4e80c66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4e77fe700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e80ccf90 0x7fc4e80c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4e77fe700 1 -- 172.21.15.119:0/1120481699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e815a270 msgr2=0x7fc4e80a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4e77fe700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e815a270 0x7fc4e80a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4e77fe700 1 -- 172.21.15.119:0/1120481699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc00b790 con 0x7fc4e80cbf30 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4e77fe700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80cbf30 0x7fc4e816aa60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009fa0 tx=0x7fc4d8009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4cf7fe700 1 -- 172.21.15.119:0/1120481699 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800ed20 con 0x7fc4e80cbf30 2023-10-08T00:56:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4cf7fe700 1 -- 172.21.15.119:0/1120481699 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800ee80 con 0x7fc4e80cbf30 2023-10-08T00:56:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e80c7130 con 0x7fc4e80cbf30 2023-10-08T00:56:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4cf7fe700 1 -- 172.21.15.119:0/1120481699 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8014670 con 0x7fc4e80cbf30 2023-10-08T00:56:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.827+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e80c76f0 con 0x7fc4e80cbf30 2023-10-08T00:56:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.829+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e804c3c0 con 0x7fc4e80cbf30 2023-10-08T00:56:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.830+0000 7fc4cf7fe700 1 -- 172.21.15.119:0/1120481699 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc4d80149e0 con 0x7fc4e80cbf30 2023-10-08T00:56:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.830+0000 7fc4cf7fe700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc4d4066f30 0x7fc4d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.830+0000 7fc4cf7fe700 1 -- 172.21.15.119:0/1120481699 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc4d8084d50 con 0x7fc4e80cbf30 2023-10-08T00:56:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.831+0000 7fc4e6ffd700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc4d4066f30 0x7fc4d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:09.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.831+0000 7fc4e6ffd700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc4d4066f30 0x7fc4d40693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8063d40 tx=0x7fc4dc01a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:09.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:09.835+0000 7fc4cf7fe700 1 -- 172.21.15.119:0/1120481699 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc4d8051f20 con 0x7fc4e80cbf30 2023-10-08T00:56:10.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.070+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc4e806f8a0 con 0x7fc4e80cbf30 2023-10-08T00:56:10.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.070+0000 7fc4cf7fe700 1 -- 172.21.15.119:0/1120481699 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc4d80099c0 con 0x7fc4e80cbf30 2023-10-08T00:56:10.071 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:56:10.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc4d4066f30 msgr2=0x7fc4d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:10.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc4d4066f30 0x7fc4d40693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8063d40 tx=0x7fc4dc01a000 comp rx=0 tx=0).stop 2023-10-08T00:56:10.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80cbf30 msgr2=0x7fc4e816aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:10.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80cbf30 0x7fc4e816aa60 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009fa0 tx=0x7fc4d8009880 comp rx=0 tx=0).stop 2023-10-08T00:56:10.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 shutdown_connections 2023-10-08T00:56:10.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc4d4066f30 0x7fc4d40693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:10.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc4e815a270 0x7fc4e80a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:10.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4e80cbf30 0x7fc4e816aa60 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:10.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 --2- 172.21.15.119:0/1120481699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc4e80ccf90 0x7fc4e80c66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:10.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.072+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 >> 172.21.15.119:0/1120481699 conn(0x7fc4e806a320 msgr2=0x7fc4e80b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:10.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.073+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 shutdown_connections 2023-10-08T00:56:10.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:10.073+0000 7fc4edfff700 1 -- 172.21.15.119:0/1120481699 wait complete. 2023-10-08T00:56:10.090 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T00:56:10.090 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:56:10.091 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2023-10-08T00:56:10.091 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-08T00:56:10.091 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-08T00:56:10.091 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T00:56:10.091 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T00:56:10.091 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:56:10.091 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:56:10.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:10.091 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2023-10-08T00:56:10.132 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:10.132 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:10.132 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:10.132 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:10.132 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:10.132 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:10.132 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:10.132 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42290sec preferred_lft 42290sec 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link 2023-10-08T00:56:10.133 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:10.134 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-08T00:56:10.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:10.134 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:10.134 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:10.134 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add name ceph-brx type bridge 2023-10-08T00:56:10.134 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr flush dev ceph-brx 2023-10-08T00:56:10.134 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set ceph-brx up 2023-10-08T00:56:10.134 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-08T00:56:10.134 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:10.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:10.584 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:56:10.645 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-10-08T00:56:10.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:10.648 DEBUG:teuthology.orchestra.run.smithi184:> route 2023-10-08T00:56:10.697 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2023-10-08T00:56:10.697 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:56:10.697 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:56:10.697 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:56:10.698 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:56:10.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:10.699 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:10.699 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:10.699 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:56:10.699 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:56:10.699 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:56:10.699 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:11.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:11.467 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-08T00:56:11.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:11.504 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2023-10-08T00:56:11.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:11.581 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:11.581 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:11.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:56:11.581 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-08T00:56:11.581 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:11.746 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-08T00:56:11.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:11.746 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:11.746 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:11.746 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-08T00:56:11.746 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-08T00:56:11.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-08T00:56:11.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-08T00:56:11.747 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-08T00:56:11.747 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:11.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:11.991 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:11.991 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:11.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.0 up 2023-10-08T00:56:11.991 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.0 master ceph-brx 2023-10-08T00:56:11.991 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:12.147 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:12.147 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-08T00:56:12.183 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-08T00:56:12.184 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:12.184 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:56:12.258 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-10-08T00:56:12.258 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:56:12.258 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:12.405 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:12.405 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:12.857 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:12.857 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:12.857 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T00:56:12.857 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with new device syntax: 1@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T00:56:12.857 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T00:56:12.862 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:56:12.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:12.863 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-08T00:56:12.924 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-10-08T00:56:12.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:56:13.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.117+0000 7f4a2c5ff700 1 Processor -- start 2023-10-08T00:56:13.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.117+0000 7f4a2c5ff700 1 -- start start 2023-10-08T00:56:13.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.118+0000 7f4a2c5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4a24159890 0x7f4a2415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:13.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.118+0000 7f4a2c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a2415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.118+0000 7f4a2c5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a2415db10 0x7f4a24069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.118+0000 7f4a2c5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a2415a160 con 0x7f4a2415db10 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.118+0000 7f4a2c5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a240d13a0 con 0x7f4a24159890 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.118+0000 7f4a2c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2415e140 con 0x7f4a24159c60 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a2415db10 0x7f4a24069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a2415db10 0x7f4a24069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54850/0 (socket says 172.21.15.119:54850) 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 -- 172.21.15.119:0/2767123842 learned_addr learned my addr 172.21.15.119:0/2767123842 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a29b9a700 1 --2- 172.21.15.119:0/2767123842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a2415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2a39b700 1 --2- 172.21.15.119:0/2767123842 >> v2:172.21.15.184:3300/0 conn(0x7f4a24159890 0x7f4a2415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:13.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 -- 172.21.15.119:0/2767123842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 msgr2=0x7f4a2415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 --2- 172.21.15.119:0/2767123842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a2415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 -- 172.21.15.119:0/2767123842 >> v2:172.21.15.184:3300/0 conn(0x7f4a24159890 msgr2=0x7f4a2415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 --2- 172.21.15.119:0/2767123842 >> v2:172.21.15.184:3300/0 conn(0x7f4a24159890 0x7f4a2415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 -- 172.21.15.119:0/2767123842 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2415a3d0 con 0x7f4a2415db10 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a29b9a700 1 --2- 172.21.15.119:0/2767123842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a2415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2a39b700 1 --2- 172.21.15.119:0/2767123842 >> v2:172.21.15.184:3300/0 conn(0x7f4a24159890 0x7f4a2415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.119+0000 7f4a2ab9c700 1 --2- 172.21.15.119:0/2767123842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2415db10 0x7f4a24069560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a240d1090 tx=0x7f4a1c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c99769ccc110c8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a29399700 1 -- 172.21.15.119:0/2767123842 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c00e070 con 0x7f4a2415db10 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a29399700 1 -- 172.21.15.119:0/2767123842 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c00ba50 con 0x7f4a2415db10 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a29399700 1 -- 172.21.15.119:0/2767123842 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0092e0 con 0x7f4a2415db10 2023-10-08T00:56:13.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/2767123842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2415db10 msgr2=0x7f4a24069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:13.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/2767123842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2415db10 0x7f4a24069560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a240d1090 tx=0x7f4a1c009840 comp rx=0 tx=0).stop 2023-10-08T00:56:13.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/2767123842 shutdown_connections 2023-10-08T00:56:13.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/2767123842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a2415db10 0x7f4a24069560 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/2767123842 >> v2:172.21.15.184:3300/0 conn(0x7f4a24159890 0x7f4a2415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/2767123842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a2415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/2767123842 >> 172.21.15.119:0/2767123842 conn(0x7f4a24068e20 msgr2=0x7f4a240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:13.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/2767123842 shutdown_connections 2023-10-08T00:56:13.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.120+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/2767123842 wait complete. 2023-10-08T00:56:13.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.122+0000 7f4a2c5ff700 1 Processor -- start 2023-10-08T00:56:13.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.122+0000 7f4a2c5ff700 1 -- start start 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2c5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a24159890 0x7f4a240b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a24149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2c5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2415db10 0x7f4a240ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2c5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240ce650 con 0x7f4a24159890 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2c5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240ce370 con 0x7f4a2415db10 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240b1aa0 con 0x7f4a24159c60 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a24159890 0x7f4a240b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a24159890 0x7f4a240b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54852/0 (socket says 172.21.15.119:54852) 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2a39b700 1 -- 172.21.15.119:0/259210632 learned_addr learned my addr 172.21.15.119:0/259210632 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:13.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a29b9a700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a24149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2ab9c700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2415db10 0x7f4a240ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2a39b700 1 -- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 msgr2=0x7f4a24149210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2a39b700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a24149210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2a39b700 1 -- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2415db10 msgr2=0x7f4a240ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2a39b700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2415db10 0x7f4a240ce900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2a39b700 1 -- 172.21.15.119:0/259210632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c009ae0 con 0x7f4a24159890 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a29b9a700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a24149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.123+0000 7f4a2ab9c700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2415db10 0x7f4a240ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.124+0000 7f4a2a39b700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a24159890 0x7f4a240b2580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000e8d0 tx=0x7f4a20008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:13.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.124+0000 7f4a1a7fc700 1 -- 172.21.15.119:0/259210632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000ee70 con 0x7f4a24159890 2023-10-08T00:56:13.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.124+0000 7f4a1a7fc700 1 -- 172.21.15.119:0/259210632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2000c5e0 con 0x7f4a24159890 2023-10-08T00:56:13.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.124+0000 7f4a1a7fc700 1 -- 172.21.15.119:0/259210632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000c980 con 0x7f4a24159890 2023-10-08T00:56:13.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.124+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a240b1d80 con 0x7f4a24159890 2023-10-08T00:56:13.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.124+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a240ce000 con 0x7f4a24159890 2023-10-08T00:56:13.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.125+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a240b2220 con 0x7f4a24159890 2023-10-08T00:56:13.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.126+0000 7f4a1a7fc700 1 -- 172.21.15.119:0/259210632 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a20013410 con 0x7f4a24159890 2023-10-08T00:56:13.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.127+0000 7f4a1a7fc700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:13.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.127+0000 7f4a1a7fc700 1 -- 172.21.15.119:0/259210632 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f4a200843e0 con 0x7f4a24159890 2023-10-08T00:56:13.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.127+0000 7f4a29b9a700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:13.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.128+0000 7f4a29b9a700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4a08066f30 0x7f4a080693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a240c5600 tx=0x7f4a14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:13.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.131+0000 7f4a1a7fc700 1 -- 172.21.15.119:0/259210632 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a200515b0 con 0x7f4a24159890 2023-10-08T00:56:13.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:56:13.336 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:56:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.366+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4a240b23d0 con 0x7f4a24159890 2023-10-08T00:56:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.367+0000 7f4a1a7fc700 1 -- 172.21.15.119:0/259210632 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4a20054bd0 con 0x7f4a24159890 2023-10-08T00:56:13.367 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:56:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4a08066f30 msgr2=0x7f4a080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4a08066f30 0x7f4a080693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4a240c5600 tx=0x7f4a14008040 comp rx=0 tx=0).stop 2023-10-08T00:56:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a24159890 msgr2=0x7f4a240b2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a24159890 0x7f4a240b2580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4a2000e8d0 tx=0x7f4a20008040 comp rx=0 tx=0).stop 2023-10-08T00:56:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 shutdown_connections 2023-10-08T00:56:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4a08066f30 0x7f4a080693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a2415db10 0x7f4a240ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a24159890 0x7f4a240b2580 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 --2- 172.21.15.119:0/259210632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a24159c60 0x7f4a24149210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.369+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 >> 172.21.15.119:0/259210632 conn(0x7f4a24068e20 msgr2=0x7f4a24149a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.370+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 shutdown_connections 2023-10-08T00:56:13.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:13.370+0000 7f4a2c5ff700 1 -- 172.21.15.119:0/259210632 wait complete. 2023-10-08T00:56:13.389 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-08T00:56:13.389 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:56:13.389 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi184.front.sepia.ceph.com 2023-10-08T00:56:13.389 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-08T00:56:13.389 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-08T00:56:13.389 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-08T00:56:13.389 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-08T00:56:13.389 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:56:13.390 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-08T00:56:13.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:13.390 DEBUG:teuthology.orchestra.run.smithi184:> ip addr 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:16 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:13.426 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f0 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:d9:37:17 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: altname enp7s0f1 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8e brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether 0c:c4:7a:88:7b:8f brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: inet 172.21.15.184/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft 42286sec preferred_lft 42286sec 2023-10-08T00:56:13.427 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::ec4:7aff:fe88:7b8f/64 scope link 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether d2:9a:09:74:6f:92 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::308d:e0ff:febb:ed8b/64 scope link 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether d2:9a:09:74:6f:92 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::d09a:9ff:fe74:6f92/64 scope link 2023-10-08T00:56:13.428 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:13.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:13.429 DEBUG:teuthology.orchestra.run.smithi184:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:56:13.525 INFO:teuthology.orchestra.run.smithi184.stdout:1 2023-10-08T00:56:13.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:13.528 DEBUG:teuthology.orchestra.run.smithi184:> route 2023-10-08T00:56:13.565 INFO:teuthology.orchestra.run.smithi184.stdout:Kernel IP routing table 2023-10-08T00:56:13.566 INFO:teuthology.orchestra.run.smithi184.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:56:13.566 INFO:teuthology.orchestra.run.smithi184.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:56:13.566 INFO:teuthology.orchestra.run.smithi184.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:56:13.566 INFO:teuthology.orchestra.run.smithi184.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:56:13.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:13.567 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:13.567 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:13.567 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:56:13.567 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:56:13.567 DEBUG:teuthology.orchestra.run.smithi184:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:56:13.568 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:13.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:13.761 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-08T00:56:13.797 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T00:56:13.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:13.797 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list-id 2023-10-08T00:56:13.873 INFO:teuthology.orchestra.run.smithi184.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-08T00:56:13.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:13.873 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:13.873 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:13.873 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T00:56:13.874 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-08T00:56:13.874 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:14.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:14.042 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T00:56:14.103 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:14.103 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:14.103 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:14.103 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:14.103 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:14.103 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:14.103 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:56:14.104 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether fa:6f:4f:ed:a6:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T00:56:14.104 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T00:56:14.104 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:14.104 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::f86f:4fff:feed:a601/64 scope link 2023-10-08T00:56:14.104 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:14.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:14.106 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-08T00:56:14.166 INFO:teuthology.orchestra.run.smithi184.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:14.166 INFO:teuthology.orchestra.run.smithi184.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:14.166 INFO:teuthology.orchestra.run.smithi184.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:14.166 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:14.166 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:14.167 INFO:teuthology.orchestra.run.smithi184.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:56:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: link/ether fa:6f:4f:ed:a6:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-08T00:56:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-08T00:56:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:14.167 INFO:teuthology.orchestra.run.smithi184.stdout: inet6 fe80::f86f:4fff:feed:a601/64 scope link 2023-10-08T00:56:14.168 INFO:teuthology.orchestra.run.smithi184.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:14.169 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-08T00:56:14.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:14.170 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:14.170 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:14.170 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-08T00:56:14.170 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-08T00:56:14.171 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-08T00:56:14.171 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-08T00:56:14.171 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-08T00:56:14.171 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:14.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:14.408 DEBUG:teuthology.orchestra.run.smithi184:> (cd / && exec bash -c ' 2023-10-08T00:56:14.408 DEBUG:teuthology.orchestra.run.smithi184:> set -e 2023-10-08T00:56:14.408 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set brx.1 up 2023-10-08T00:56:14.408 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link set dev brx.1 master ceph-brx 2023-10-08T00:56:14.409 DEBUG:teuthology.orchestra.run.smithi184:> ') 2023-10-08T00:56:14.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:14.564 DEBUG:teuthology.orchestra.run.smithi184:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-08T00:56:14.600 INFO:teuthology.orchestra.run.smithi184.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-08T00:56:14.601 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:14.601 DEBUG:teuthology.orchestra.run.smithi184:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-08T00:56:14.676 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-10-08T00:56:14.676 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:56:14.676 DEBUG:teuthology.orchestra.run.smithi184:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:14.806 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:14.806 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:14.835 INFO:teuthology.orchestra.run.smithi184.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:14.835 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:14.835 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T00:56:14.836 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: trying mount with new device syntax: 3@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T00:56:14.836 INFO:teuthology.orchestra.run.smithi184.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T00:56:14.840 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:56:14.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:14.841 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-08T00:56:14.901 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-08T00:56:14.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.110+0000 7f7dc65ff700 1 Processor -- start 2023-10-08T00:56:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.110+0000 7f7dc65ff700 1 -- start start 2023-10-08T00:56:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.111+0000 7f7dc65ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159890 0x7f7dc015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.111+0000 7f7dc65ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7dc0159c60 0x7f7dc015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.111+0000 7f7dc65ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc015a160 con 0x7f7dc015db10 2023-10-08T00:56:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.111+0000 7f7dc65ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc00d13a0 con 0x7f7dc0159c60 2023-10-08T00:56:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.111+0000 7f7dc65ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc015e140 con 0x7f7dc0159890 2023-10-08T00:56:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbf7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7dc0159c60 0x7f7dc015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbf7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7dc0159c60 0x7f7dc015d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34908/0 (socket says 172.21.15.119:34908) 2023-10-08T00:56:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbf7fe700 1 -- 172.21.15.119:0/972925224 learned_addr learned my addr 172.21.15.119:0/972925224 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbffff700 1 --2- 172.21.15.119:0/972925224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159890 0x7f7dc015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbf7fe700 1 -- 172.21.15.119:0/972925224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159890 msgr2=0x7f7dc015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbf7fe700 1 --2- 172.21.15.119:0/972925224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159890 0x7f7dc015abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbf7fe700 1 -- 172.21.15.119:0/972925224 >> v1:172.21.15.184:6789/0 conn(0x7f7dc015db10 legacy=0x7f7dc0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbf7fe700 1 -- 172.21.15.119:0/972925224 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc015a3d0 con 0x7f7dc0159c60 2023-10-08T00:56:15.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.112+0000 7f7dbeffd700 1 -- 172.21.15.119:0/972925224 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050212737 0 0) 0x7f7dc015a160 con 0x7f7dc015db10 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dbf7fe700 1 --2- 172.21.15.119:0/972925224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc0159c60 0x7f7dc015d5d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7db40099b0 tx=0x7f7db4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a1e999338c5c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dbeffd700 1 -- 172.21.15.119:0/972925224 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db4009ea0 con 0x7f7dc0159c60 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dbeffd700 1 -- 172.21.15.119:0/972925224 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db400b860 con 0x7f7dc0159c60 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dbeffd700 1 -- 172.21.15.119:0/972925224 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db400bc00 con 0x7f7dc0159c60 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dc65ff700 1 -- 172.21.15.119:0/972925224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc0159c60 msgr2=0x7f7dc015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/972925224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc0159c60 0x7f7dc015d5d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7db40099b0 tx=0x7f7db4009250 comp rx=0 tx=0).stop 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dc65ff700 1 -- 172.21.15.119:0/972925224 shutdown_connections 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/972925224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159890 0x7f7dc015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/972925224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc0159c60 0x7f7dc015d5d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dc65ff700 1 -- 172.21.15.119:0/972925224 >> 172.21.15.119:0/972925224 conn(0x7f7dc0068e20 msgr2=0x7f7dc00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.113+0000 7f7dc65ff700 1 -- 172.21.15.119:0/972925224 shutdown_connections 2023-10-08T00:56:15.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.114+0000 7f7dc65ff700 1 -- 172.21.15.119:0/972925224 wait complete. 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.115+0000 7f7dc65ff700 1 Processor -- start 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.115+0000 7f7dc65ff700 1 -- start start 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc65ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc0159890 0x7f7dc00c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc65ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159c60 0x7f7dc01492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc65ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015db10 0x7f7dc01516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc65ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00c8ae0 con 0x7f7dc015db10 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc65ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc0151260 con 0x7f7dc0159890 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc65ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc00ca780 con 0x7f7dc0159c60 2023-10-08T00:56:15.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc4b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015db10 0x7f7dc01516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc4b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015db10 0x7f7dc01516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34912/0 (socket says 172.21.15.119:34912) 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dc4b9c700 1 -- 172.21.15.119:0/2156483286 learned_addr learned my addr 172.21.15.119:0/2156483286 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dbffff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc0159890 0x7f7dc00c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.116+0000 7f7dbf7fe700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159c60 0x7f7dc01492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7dc4b9c700 1 -- 172.21.15.119:0/2156483286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159c60 msgr2=0x7f7dc01492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7dc4b9c700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159c60 0x7f7dc01492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7dc4b9c700 1 -- 172.21.15.119:0/2156483286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc0159890 msgr2=0x7f7dc00c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7dc4b9c700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc0159890 0x7f7dc00c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7dc4b9c700 1 -- 172.21.15.119:0/2156483286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db40094f0 con 0x7f7dc015db10 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7dc4b9c700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015db10 0x7f7dc01516f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c8c0 tx=0x7f7db8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7da7fff700 1 -- 172.21.15.119:0/2156483286 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8012630 con 0x7f7dc015db10 2023-10-08T00:56:15.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7da7fff700 1 -- 172.21.15.119:0/2156483286 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7db8012790 con 0x7f7dc015db10 2023-10-08T00:56:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc00ca1c0 con 0x7f7dc015db10 2023-10-08T00:56:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7da7fff700 1 -- 172.21.15.119:0/2156483286 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7db8012b50 con 0x7f7dc015db10 2023-10-08T00:56:15.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.117+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc00c87f0 con 0x7f7dc015db10 2023-10-08T00:56:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.118+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc01513f0 con 0x7f7dc015db10 2023-10-08T00:56:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.119+0000 7f7da7fff700 1 -- 172.21.15.119:0/2156483286 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7db8003030 con 0x7f7dc015db10 2023-10-08T00:56:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.120+0000 7f7da7fff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7dac066f30 0x7f7dac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.120+0000 7f7da7fff700 1 -- 172.21.15.119:0/2156483286 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7db8084440 con 0x7f7dc015db10 2023-10-08T00:56:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.120+0000 7f7dbffff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7dac066f30 0x7f7dac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.121+0000 7f7dbffff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7dac066f30 0x7f7dac0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009db0 tx=0x7f7db00091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.124+0000 7f7da7fff700 1 -- 172.21.15.119:0/2156483286 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7db80516b0 con 0x7f7dc015db10 2023-10-08T00:56:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.316+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dc004c3c0 con 0x7f7dc015db10 2023-10-08T00:56:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.317+0000 7f7da7fff700 1 -- 172.21.15.119:0/2156483286 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7db8054cd0 con 0x7f7dc015db10 2023-10-08T00:56:15.318 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:15.318 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7dac066f30 msgr2=0x7f7dac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7dac066f30 0x7f7dac0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7db0009db0 tx=0x7f7db00091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015db10 msgr2=0x7f7dc01516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015db10 0x7f7dc01516f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f7db800c8c0 tx=0x7f7db8001630 comp rx=0 tx=0).stop 2023-10-08T00:56:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 shutdown_connections 2023-10-08T00:56:15.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7dac066f30 0x7f7dac0693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7dc015db10 0x7f7dc01516f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7dc0159890 0x7f7dc00c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 --2- 172.21.15.119:0/2156483286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7dc0159c60 0x7f7dc01492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 >> 172.21.15.119:0/2156483286 conn(0x7f7dc0068e20 msgr2=0x7f7dc0158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:15.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 shutdown_connections 2023-10-08T00:56:15.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.320+0000 7f7dc65ff700 1 -- 172.21.15.119:0/2156483286 wait complete. 2023-10-08T00:56:15.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:55:45.270364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:55:45.937180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:56:15.339 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:56:15.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.521+0000 7fcb321ff700 1 Processor -- start 2023-10-08T00:56:15.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.521+0000 7fcb321ff700 1 -- start start 2023-10-08T00:56:15.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb321ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcb2c069d90 0x7fcb2c1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb321ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c15a090 con 0x7fcb2c0699c0 2023-10-08T00:56:15.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb321ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb2c06de10 con 0x7fcb2c159c10 2023-10-08T00:56:15.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb321ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb2c06e0f0 con 0x7fcb2c069d90 2023-10-08T00:56:15.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb2b7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcb2c0699c0 0x7fcb2c06f210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37498/0 (socket says 172.21.15.119:37498) 2023-10-08T00:56:15.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb2b7fe700 1 -- 172.21.15.119:0/4266479244 learned_addr learned my addr 172.21.15.119:0/4266479244 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:15.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb2affd700 1 --2- 172.21.15.119:0/4266479244 >> v2:172.21.15.184:3300/0 conn(0x7fcb2c069d90 0x7fcb2c1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108888951 0 0) 0x7fcb2c15a090 con 0x7fcb2c0699c0 2023-10-08T00:56:15.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.523+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb14003570 con 0x7fcb2c0699c0 2023-10-08T00:56:15.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1083720123 0 0) 0x7fcb14003570 con 0x7fcb2c0699c0 2023-10-08T00:56:15.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 >> v2:172.21.15.184:3300/0 conn(0x7fcb2c069d90 msgr2=0x7fcb2c1596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 --2- 172.21.15.119:0/4266479244 >> v2:172.21.15.184:3300/0 conn(0x7fcb2c069d90 0x7fcb2c1596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 >> v1:172.21.15.184:6789/0 conn(0x7fcb2c159c10 legacy=0x7fcb2c0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c06e3d0 con 0x7fcb2c0699c0 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fcb1c0017f0 con 0x7fcb2c0699c0 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 >> v1:172.21.15.119:6789/0 conn(0x7fcb2c0699c0 legacy=0x7fcb2c06f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb14003a60 0x7fcb14003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14004810 0x7fcb14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb140051c0 0x7fcb1400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c06e0f0 con 0x7fcb14003a60 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1400c0d0 con 0x7fcb140051c0 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb1400c290 con 0x7fcb14004810 2023-10-08T00:56:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2b7fe700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb14003a60 0x7fcb14003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2bfff700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb140051c0 0x7fcb1400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2affd700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14004810 0x7fcb14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2b7fe700 1 -- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14004810 msgr2=0x7fcb14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2b7fe700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14004810 0x7fcb14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2b7fe700 1 -- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb140051c0 msgr2=0x7fcb1400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2b7fe700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb140051c0 0x7fcb1400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2b7fe700 1 -- 172.21.15.119:0/4266479244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14003570 con 0x7fcb14003a60 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2bfff700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb140051c0 0x7fcb1400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2affd700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14004810 0x7fcb14004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.524+0000 7fcb2bfff700 1 -- 172.21.15.119:0/4266479244 reap_dead start 2023-10-08T00:56:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb2b7fe700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb14003a60 0x7fcb14003e70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00a900 tx=0x7fcb2c0cd600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a426e4735a2797cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0070d0 con 0x7fcb14003a60 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1c007230 con 0x7fcb14003a60 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb2a7fc700 1 -- 172.21.15.119:0/4266479244 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c0075f0 con 0x7fcb14003a60 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb321ff700 1 -- 172.21.15.119:0/4266479244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb14003a60 msgr2=0x7fcb14003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb321ff700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb14003a60 0x7fcb14003e70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00a900 tx=0x7fcb2c0cd600 comp rx=0 tx=0).stop 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb321ff700 1 -- 172.21.15.119:0/4266479244 shutdown_connections 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb321ff700 1 --2- 172.21.15.119:0/4266479244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb14003a60 0x7fcb14003e70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb321ff700 1 -- 172.21.15.119:0/4266479244 >> 172.21.15.119:0/4266479244 conn(0x7fcb2c068e10 msgr2=0x7fcb2c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.525+0000 7fcb321ff700 1 -- 172.21.15.119:0/4266479244 shutdown_connections 2023-10-08T00:56:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.526+0000 7fcb321ff700 1 -- 172.21.15.119:0/4266479244 wait complete. 2023-10-08T00:56:15.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.527+0000 7fcb321ff700 1 Processor -- start 2023-10-08T00:56:15.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.527+0000 7fcb321ff700 1 -- start start 2023-10-08T00:56:15.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb321ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14003a60 0x7fcb2c147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb321ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb2c0b1880 0x7fcb2c0c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb321ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb2c0b1cf0 0x7fcb2c0c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb321ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c147660 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb321ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0b1050 con 0x7fcb2c0b1880 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb321ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0b0440 con 0x7fcb14003a60 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb2c0b1cf0 0x7fcb2c0c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb2c0b1cf0 0x7fcb2c0c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34922/0 (socket says 172.21.15.119:34922) 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2bfff700 1 -- 172.21.15.119:0/1002793162 learned_addr learned my addr 172.21.15.119:0/1002793162 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2affd700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb2c0b1880 0x7fcb2c0c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2b7fe700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14003a60 0x7fcb2c147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2bfff700 1 -- 172.21.15.119:0/1002793162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14003a60 msgr2=0x7fcb2c147120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2bfff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14003a60 0x7fcb2c147120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2bfff700 1 -- 172.21.15.119:0/1002793162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb2c0b1880 msgr2=0x7fcb2c0c2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2bfff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb2c0b1880 0x7fcb2c0c2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.528+0000 7fcb2bfff700 1 -- 172.21.15.119:0/1002793162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c0b0720 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.529+0000 7fcb2bfff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb2c0b1cf0 0x7fcb2c0c2990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0b2530 tx=0x7fcb24009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.529+0000 7fcb137fe700 1 -- 172.21.15.119:0/1002793162 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24014480 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.529+0000 7fcb137fe700 1 -- 172.21.15.119:0/1002793162 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb240145e0 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.529+0000 7fcb137fe700 1 -- 172.21.15.119:0/1002793162 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24014980 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.529+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c0b0a00 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.529+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c1523f0 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.530+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c06ead0 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.531+0000 7fcb137fe700 1 -- 172.21.15.119:0/1002793162 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb240133e0 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.532+0000 7fcb137fe700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcb18066f30 0x7fcb180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.532+0000 7fcb137fe700 1 -- 172.21.15.119:0/1002793162 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcb24083fb0 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.532+0000 7fcb2b7fe700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcb18066f30 0x7fcb180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.533+0000 7fcb2b7fe700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcb18066f30 0x7fcb180693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00a3f0 tx=0x7fcb1c015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.536+0000 7fcb137fe700 1 -- 172.21.15.119:0/1002793162 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb24051180 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.726+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb2c0b11e0 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.726+0000 7fcb137fe700 1 -- 172.21.15.119:0/1002793162 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb24019020 con 0x7fcb2c0b1cf0 2023-10-08T00:56:15.727 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:56:15.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcb18066f30 msgr2=0x7fcb180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcb18066f30 0x7fcb180693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00a3f0 tx=0x7fcb1c015000 comp rx=0 tx=0).stop 2023-10-08T00:56:15.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb2c0b1cf0 msgr2=0x7fcb2c0c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb2c0b1cf0 0x7fcb2c0c2990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c0b2530 tx=0x7fcb24009040 comp rx=0 tx=0).stop 2023-10-08T00:56:15.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 shutdown_connections 2023-10-08T00:56:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcb14003a60 0x7fcb2c147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcb18066f30 0x7fcb180693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb2c0b1cf0 0x7fcb2c0c2990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 --2- 172.21.15.119:0/1002793162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcb2c0b1880 0x7fcb2c0c2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 >> 172.21.15.119:0/1002793162 conn(0x7fcb2c068e10 msgr2=0x7fcb2c153110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 shutdown_connections 2023-10-08T00:56:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.729+0000 7fcb321ff700 1 -- 172.21.15.119:0/1002793162 wait complete. 2023-10-08T00:56:15.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:56:15.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.931+0000 7f93667ff700 1 Processor -- start 2023-10-08T00:56:15.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.931+0000 7f93667ff700 1 -- start start 2023-10-08T00:56:15.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f93667ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 0x7f936006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f93667ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93600c70b0 0x7f9360069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f93667ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936006eb90 con 0x7f9360069cb0 2023-10-08T00:56:15.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f93667ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936006a130 con 0x7f93600c70b0 2023-10-08T00:56:15.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f93667ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936006ee70 con 0x7f9360155eb0 2023-10-08T00:56:15.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f935f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93600c70b0 0x7f9360069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f935f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93600c70b0 0x7f9360069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34928/0 (socket says 172.21.15.119:34928) 2023-10-08T00:56:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f935f7fe700 1 -- 172.21.15.119:0/1580153539 learned_addr learned my addr 172.21.15.119:0/1580153539 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f935ffff700 1 --2- 172.21.15.119:0/1580153539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 0x7f936006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f935f7fe700 1 -- 172.21.15.119:0/1580153539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 msgr2=0x7f936006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f935f7fe700 1 --2- 172.21.15.119:0/1580153539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 0x7f936006e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f935f7fe700 1 -- 172.21.15.119:0/1580153539 >> v1:172.21.15.184:6789/0 conn(0x7f9360069cb0 legacy=0x7f9360159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.933+0000 7f935f7fe700 1 -- 172.21.15.119:0/1580153539 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936006f150 con 0x7f93600c70b0 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.934+0000 7f935f7fe700 1 --2- 172.21.15.119:0/1580153539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360069770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93540099b0 tx=0x7f9354009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cbafe989e8f00 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.934+0000 7f935effd700 1 -- 172.21.15.119:0/1580153539 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935400b860 con 0x7f93600c70b0 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.934+0000 7f935effd700 1 -- 172.21.15.119:0/1580153539 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935400b9c0 con 0x7f93600c70b0 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.934+0000 7f935effd700 1 -- 172.21.15.119:0/1580153539 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935400bd60 con 0x7f93600c70b0 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.934+0000 7f93667ff700 1 -- 172.21.15.119:0/1580153539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 msgr2=0x7f9360069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.934+0000 7f93667ff700 1 --2- 172.21.15.119:0/1580153539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360069770 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93540099b0 tx=0x7f9354009250 comp rx=0 tx=0).stop 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.934+0000 7f93667ff700 1 -- 172.21.15.119:0/1580153539 shutdown_connections 2023-10-08T00:56:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.935+0000 7f93667ff700 1 --2- 172.21.15.119:0/1580153539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 0x7f936006e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.935+0000 7f93667ff700 1 --2- 172.21.15.119:0/1580153539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360069770 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.935+0000 7f93667ff700 1 -- 172.21.15.119:0/1580153539 >> 172.21.15.119:0/1580153539 conn(0x7f9360068e10 msgr2=0x7f93600cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.935+0000 7f93667ff700 1 -- 172.21.15.119:0/1580153539 shutdown_connections 2023-10-08T00:56:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.935+0000 7f93667ff700 1 -- 172.21.15.119:0/1580153539 wait complete. 2023-10-08T00:56:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.936+0000 7f93667ff700 1 Processor -- start 2023-10-08T00:56:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.936+0000 7f93667ff700 1 -- start start 2023-10-08T00:56:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f93667ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9360069cb0 0x7f93600afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f93667ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f93667ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 0x7f93600cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f93667ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600aca80 con 0x7f93600c70b0 2023-10-08T00:56:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f93667ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600abb20 con 0x7f9360069cb0 2023-10-08T00:56:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f93667ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600abe00 con 0x7f9360155eb0 2023-10-08T00:56:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f935f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f935f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360140ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34938/0 (socket says 172.21.15.119:34938) 2023-10-08T00:56:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.937+0000 7f935f7fe700 1 -- 172.21.15.119:0/853976479 learned_addr learned my addr 172.21.15.119:0/853976479 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f935f7fe700 1 -- 172.21.15.119:0/853976479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 msgr2=0x7f93600cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f935f7fe700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 0x7f93600cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f935ffff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9360069cb0 0x7f93600afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f935f7fe700 1 -- 172.21.15.119:0/853976479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9360069cb0 msgr2=0x7f93600afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f935f7fe700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9360069cb0 0x7f93600afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f935f7fe700 1 -- 172.21.15.119:0/853976479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93540094f0 con 0x7f93600c70b0 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f935ffff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9360069cb0 0x7f93600afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f935f7fe700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360140ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f936006fe40 tx=0x7f9354017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f9347fff700 1 -- 172.21.15.119:0/853976479 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935401a420 con 0x7f93600c70b0 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f9347fff700 1 -- 172.21.15.119:0/853976479 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935401a5a0 con 0x7f93600c70b0 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93600ac0e0 con 0x7f93600c70b0 2023-10-08T00:56:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f9347fff700 1 -- 172.21.15.119:0/853976479 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935401a960 con 0x7f93600c70b0 2023-10-08T00:56:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.938+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93600ac6a0 con 0x7f93600c70b0 2023-10-08T00:56:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.939+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93600acbc0 con 0x7f93600c70b0 2023-10-08T00:56:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.940+0000 7f9347fff700 1 -- 172.21.15.119:0/853976479 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f935401ad10 con 0x7f93600c70b0 2023-10-08T00:56:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.941+0000 7f9347fff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f934c066f30 0x7f934c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.941+0000 7f9347fff700 1 -- 172.21.15.119:0/853976479 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f93540860d0 con 0x7f93600c70b0 2023-10-08T00:56:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.941+0000 7f935ffff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f934c066f30 0x7f934c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.942+0000 7f935ffff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f934c066f30 0x7f934c0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9350009990 tx=0x7f9350008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:15.945+0000 7f9347fff700 1 -- 172.21.15.119:0/853976479 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93600acbc0 con 0x7f93600c70b0 2023-10-08T00:56:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.139+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93600acbc0 con 0x7f93600c70b0 2023-10-08T00:56:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.140+0000 7f9347fff700 1 -- 172.21.15.119:0/853976479 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93600acbc0 con 0x7f93600c70b0 2023-10-08T00:56:16.140 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T00:56:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f934c066f30 msgr2=0x7f934c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f934c066f30 0x7f934c0693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9350009990 tx=0x7f9350008040 comp rx=0 tx=0).stop 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 msgr2=0x7f9360140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360140ea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f936006fe40 tx=0x7f9354017000 comp rx=0 tx=0).stop 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 shutdown_connections 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9360155eb0 0x7f93600cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f934c066f30 0x7f934c0693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9360069cb0 0x7f93600afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 --2- 172.21.15.119:0/853976479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93600c70b0 0x7f9360140ea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 >> 172.21.15.119:0/853976479 conn(0x7f9360068e10 msgr2=0x7f93600c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 shutdown_connections 2023-10-08T00:56:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.142+0000 7f93667ff700 1 -- 172.21.15.119:0/853976479 wait complete. 2023-10-08T00:56:16.160 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:56:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.344+0000 7fcddd7ff700 1 Processor -- start 2023-10-08T00:56:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.345+0000 7fcddd7ff700 1 -- start start 2023-10-08T00:56:16.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.346+0000 7fcddd7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c83f0 0x7fcdd80c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.346+0000 7fcddd7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcdd80ccee0 0x7fcdd81561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.346+0000 7fcddd7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd815b0d0 con 0x7fcdd80c87c0 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.346+0000 7fcddd7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd80c7a60 con 0x7fcdd80ccee0 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.346+0000 7fcddd7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd815a4c0 con 0x7fcdd80c83f0 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.346+0000 7fcdcffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcdd80ccee0 0x7fcdd81561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.346+0000 7fcdcffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcdd80ccee0 0x7fcdd81561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34946/0 (socket says 172.21.15.119:34946) 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.346+0000 7fcdcffff700 1 -- 172.21.15.119:0/2342726055 learned_addr learned my addr 172.21.15.119:0/2342726055 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdcffff700 1 -- 172.21.15.119:0/2342726055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c83f0 msgr2=0x7fcdd80c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdd6ffd700 1 --2- 172.21.15.119:0/2342726055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c83f0 0x7fcdd80c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdcffff700 1 --2- 172.21.15.119:0/2342726055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c83f0 0x7fcdd80c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdcffff700 1 -- 172.21.15.119:0/2342726055 >> v1:172.21.15.184:6789/0 conn(0x7fcdd80c87c0 legacy=0x7fcdd806eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:16.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdcffff700 1 -- 172.21.15.119:0/2342726055 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd815a7a0 con 0x7fcdd80ccee0 2023-10-08T00:56:16.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdd6ffd700 1 --2- 172.21.15.119:0/2342726055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c83f0 0x7fcdd80c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:16.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdcffff700 1 --2- 172.21.15.119:0/2342726055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd81561d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00099b0 tx=0x7fcdc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c253ae53013f78 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:16.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdd67fc700 1 -- 172.21.15.119:0/2342726055 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc000b820 con 0x7fcdd80ccee0 2023-10-08T00:56:16.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdd67fc700 1 -- 172.21.15.119:0/2342726055 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc000b980 con 0x7fcdd80ccee0 2023-10-08T00:56:16.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.347+0000 7fcdd67fc700 1 -- 172.21.15.119:0/2342726055 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc000bd20 con 0x7fcdd80ccee0 2023-10-08T00:56:16.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.348+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2342726055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 msgr2=0x7fcdd81561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.348+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2342726055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd81561d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00099b0 tx=0x7fcdc0009250 comp rx=0 tx=0).stop 2023-10-08T00:56:16.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.348+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2342726055 shutdown_connections 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.348+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2342726055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c83f0 0x7fcdd80c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.348+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2342726055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd81561d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.348+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2342726055 >> 172.21.15.119:0/2342726055 conn(0x7fcdd8068e10 msgr2=0x7fcdd80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.348+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2342726055 shutdown_connections 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.348+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2342726055 wait complete. 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.349+0000 7fcddd7ff700 1 Processor -- start 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.350+0000 7fcddd7ff700 1 -- start start 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.350+0000 7fcddd7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd80c83f0 0x7fcdd8144700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.350+0000 7fcddd7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c87c0 0x7fcdd80c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.350+0000 7fcddd7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd80c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcddd7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8157990 con 0x7fcdd80ccee0 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcddd7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8156d80 con 0x7fcdd80c83f0 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcddd7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8157060 con 0x7fcdd80c87c0 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd80c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd80c27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34948/0 (socket says 172.21.15.119:34948) 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 -- 172.21.15.119:0/2724929765 learned_addr learned my addr 172.21.15.119:0/2724929765 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdcffff700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c87c0 0x7fcdd80c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd6ffd700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd80c83f0 0x7fcdd8144700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 -- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c87c0 msgr2=0x7fcdd80c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c87c0 0x7fcdd80c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 -- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd80c83f0 msgr2=0x7fcdd8144700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd80c83f0 0x7fcdd8144700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 -- 172.21.15.119:0/2724929765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00094f0 con 0x7fcdd80ccee0 2023-10-08T00:56:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd6ffd700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd80c83f0 0x7fcdd8144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.351+0000 7fcdd77fe700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd80c27e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000c930 tx=0x7fcdd00026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.352+0000 7fcdceffd700 1 -- 172.21.15.119:0/2724929765 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000ce70 con 0x7fcdd80ccee0 2023-10-08T00:56:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.352+0000 7fcdceffd700 1 -- 172.21.15.119:0/2724929765 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd000a7e0 con 0x7fcdd80ccee0 2023-10-08T00:56:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.352+0000 7fcdceffd700 1 -- 172.21.15.119:0/2724929765 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd000aba0 con 0x7fcdd80ccee0 2023-10-08T00:56:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.352+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd8157340 con 0x7fcdd80ccee0 2023-10-08T00:56:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.352+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd8144c40 con 0x7fcdd80ccee0 2023-10-08T00:56:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.353+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd804c3c0 con 0x7fcdd80ccee0 2023-10-08T00:56:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.354+0000 7fcdceffd700 1 -- 172.21.15.119:0/2724929765 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdd0012420 con 0x7fcdd80ccee0 2023-10-08T00:56:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.355+0000 7fcdceffd700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcdb8066f30 0x7fcdb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.355+0000 7fcdceffd700 1 -- 172.21.15.119:0/2724929765 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcdd00849a0 con 0x7fcdd80ccee0 2023-10-08T00:56:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.355+0000 7fcdd6ffd700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcdb8066f30 0x7fcdb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.355+0000 7fcdd6ffd700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcdb8066f30 0x7fcdb80693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8008b20 tx=0x7fcdc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:16.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.358+0000 7fcdceffd700 1 -- 172.21.15.119:0/2724929765 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdd0051c20 con 0x7fcdd80ccee0 2023-10-08T00:56:16.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.548+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcdd8157ad0 con 0x7fcdd80ccee0 2023-10-08T00:56:16.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.548+0000 7fcdceffd700 1 -- 172.21.15.119:0/2724929765 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcdd0055240 con 0x7fcdd80ccee0 2023-10-08T00:56:16.549 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T00:56:16.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcdb8066f30 msgr2=0x7fcdb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcdb8066f30 0x7fcdb80693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8008b20 tx=0x7fcdc8008040 comp rx=0 tx=0).stop 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 msgr2=0x7fcdd80c27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd80c27e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000c930 tx=0x7fcdd00026e0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 shutdown_connections 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcdb8066f30 0x7fcdb80693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd80ccee0 0x7fcdd80c27e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdd80c83f0 0x7fcdd8144700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 --2- 172.21.15.119:0/2724929765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c87c0 0x7fcdd80c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 >> 172.21.15.119:0/2724929765 conn(0x7fcdd8068e10 msgr2=0x7fcdd8155250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 shutdown_connections 2023-10-08T00:56:16.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.551+0000 7fcddd7ff700 1 -- 172.21.15.119:0/2724929765 wait complete. 2023-10-08T00:56:16.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:16.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.749+0000 7f571499d700 1 Processor -- start 2023-10-08T00:56:16.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.749+0000 7f571499d700 1 -- start start 2023-10-08T00:56:16.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.750+0000 7f571499d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57101597f0 0x7f57100b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.750+0000 7f571499d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100b11d0 con 0x7f571015d230 2023-10-08T00:56:16.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.750+0000 7f571499d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57100b14b0 con 0x7f5710159420 2023-10-08T00:56:16.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.750+0000 7f571499d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57100b1790 con 0x7f57101597f0 2023-10-08T00:56:16.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.750+0000 7f570effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57101597f0 0x7f57100b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.750+0000 7f570ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f571015d230 0x7f571015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37504/0 (socket says 172.21.15.119:37504) 2023-10-08T00:56:16.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.750+0000 7f570ffff700 1 -- 172.21.15.119:0/4240713942 learned_addr learned my addr 172.21.15.119:0/4240713942 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570e7fc700 1 -- 172.21.15.119:0/4240713942 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250122237 0 0) 0x7f57100b11d0 con 0x7f571015d230 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570e7fc700 1 -- 172.21.15.119:0/4240713942 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f8003570 con 0x7f571015d230 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570effd700 1 -- 172.21.15.119:0/4240713942 >> v1:172.21.15.184:6789/0 conn(0x7f5710159420 legacy=0x7f57100b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570effd700 1 -- 172.21.15.119:0/4240713942 >> v1:172.21.15.119:6789/0 conn(0x7f571015d230 legacy=0x7f571015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570effd700 1 -- 172.21.15.119:0/4240713942 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57100b1a70 con 0x7f57101597f0 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570effd700 1 --2- 172.21.15.119:0/4240713942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57101597f0 0x7f57100b2270 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57040098e0 tx=0x7f57040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=614b48d32608b209 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570e7fc700 1 -- 172.21.15.119:0/4240713942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570400b640 con 0x7f57101597f0 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570e7fc700 1 -- 172.21.15.119:0/4240713942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570400b7a0 con 0x7f57101597f0 2023-10-08T00:56:16.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.751+0000 7f570e7fc700 1 -- 172.21.15.119:0/4240713942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570400bb40 con 0x7f57101597f0 2023-10-08T00:56:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.752+0000 7f571499d700 1 -- 172.21.15.119:0/4240713942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57101597f0 msgr2=0x7f57100b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.752+0000 7f571499d700 1 --2- 172.21.15.119:0/4240713942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57101597f0 0x7f57100b2270 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f57040098e0 tx=0x7f57040091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.752+0000 7f571499d700 1 -- 172.21.15.119:0/4240713942 shutdown_connections 2023-10-08T00:56:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.752+0000 7f571499d700 1 --2- 172.21.15.119:0/4240713942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57101597f0 0x7f57100b2270 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.752+0000 7f571499d700 1 -- 172.21.15.119:0/4240713942 >> 172.21.15.119:0/4240713942 conn(0x7f57100d0660 msgr2=0x7f57100d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.752+0000 7f571499d700 1 -- 172.21.15.119:0/4240713942 shutdown_connections 2023-10-08T00:56:16.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.752+0000 7f571499d700 1 -- 172.21.15.119:0/4240713942 wait complete. 2023-10-08T00:56:16.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.754+0000 7f571499d700 1 Processor -- start 2023-10-08T00:56:16.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.754+0000 7f571499d700 1 -- start start 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f571499d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5710159420 0x7f5710154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f571499d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57101597f0 0x7f57100c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f571499d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571015d230 0x7f57100c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f571499d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571015bf80 con 0x7f571015d230 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f571499d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57101695d0 con 0x7f5710159420 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f571499d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57101698b0 con 0x7f57101597f0 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571015d230 0x7f57100c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571015d230 0x7f57100c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34966/0 (socket says 172.21.15.119:34966) 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570ffff700 1 -- 172.21.15.119:0/2609328971 learned_addr learned my addr 172.21.15.119:0/2609328971 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570effd700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57101597f0 0x7f57100c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570f7fe700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5710159420 0x7f5710154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570ffff700 1 -- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57101597f0 msgr2=0x7f57100c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570ffff700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57101597f0 0x7f57100c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570ffff700 1 -- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5710159420 msgr2=0x7f5710154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570ffff700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5710159420 0x7f5710154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570ffff700 1 -- 172.21.15.119:0/2609328971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5704009420 con 0x7f571015d230 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.755+0000 7f570f7fe700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5710159420 0x7f5710154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.756+0000 7f570ffff700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571015d230 0x7f57100c6300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f570800cc70 tx=0x7f5708001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.756+0000 7f56f77fe700 1 -- 172.21.15.119:0/2609328971 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570800ce20 con 0x7f571015d230 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.756+0000 7f56f77fe700 1 -- 172.21.15.119:0/2609328971 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570800ab30 con 0x7f571015d230 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.756+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57100c7220 con 0x7f571015d230 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.756+0000 7f56f77fe700 1 -- 172.21.15.119:0/2609328971 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57080070f0 con 0x7f571015d230 2023-10-08T00:56:16.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.756+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57100c6c60 con 0x7f571015d230 2023-10-08T00:56:16.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.757+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571004c3c0 con 0x7f571015d230 2023-10-08T00:56:16.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.758+0000 7f56f77fe700 1 -- 172.21.15.119:0/2609328971 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f57080074e0 con 0x7f571015d230 2023-10-08T00:56:16.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.759+0000 7f56f77fe700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:16.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.759+0000 7f56f77fe700 1 -- 172.21.15.119:0/2609328971 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5708084a40 con 0x7f571015d230 2023-10-08T00:56:16.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.759+0000 7f570f7fe700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:16.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.760+0000 7f570f7fe700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f56fc066f30 0x7f56fc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5700001840 tx=0x7f5700009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:16.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.763+0000 7f56f77fe700 1 -- 172.21.15.119:0/2609328971 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5708051c00 con 0x7f571015d230 2023-10-08T00:56:16.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.957+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57101699f0 con 0x7f571015d230 2023-10-08T00:56:16.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.958+0000 7f56f77fe700 1 -- 172.21.15.119:0/2609328971 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f5708055220 con 0x7f571015d230 2023-10-08T00:56:16.959 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:16.959 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:16.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.960+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f56fc066f30 msgr2=0x7f56fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.960+0000 7f571499d700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f56fc066f30 0x7f56fc0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5700001840 tx=0x7f5700009040 comp rx=0 tx=0).stop 2023-10-08T00:56:16.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.960+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571015d230 msgr2=0x7f57100c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:16.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.960+0000 7f571499d700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571015d230 0x7f57100c6300 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f570800cc70 tx=0x7f5708001a00 comp rx=0 tx=0).stop 2023-10-08T00:56:16.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.960+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 shutdown_connections 2023-10-08T00:56:16.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.960+0000 7f571499d700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f56fc066f30 0x7f56fc0693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.961+0000 7f571499d700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f571015d230 0x7f57100c6300 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.961+0000 7f571499d700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5710159420 0x7f5710154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.961+0000 7f571499d700 1 --2- 172.21.15.119:0/2609328971 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57101597f0 0x7f57100c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:16.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.961+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 >> 172.21.15.119:0/2609328971 conn(0x7f57100d0660 msgr2=0x7f5710155240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:16.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.961+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 shutdown_connections 2023-10-08T00:56:16.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:16.961+0000 7f571499d700 1 -- 172.21.15.119:0/2609328971 wait complete. 2023-10-08T00:56:16.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:55:45.270364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:55:45.937180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:56:16.979 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:56:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.161+0000 7f5096d9d700 1 Processor -- start 2023-10-08T00:56:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.162+0000 7f5096d9d700 1 -- start start 2023-10-08T00:56:17.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5096d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 0x7f5090068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5096d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f50900688d0 0x7f50900caec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5096d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50900ca920 con 0x7f5090159a70 2023-10-08T00:56:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5096d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5090068d50 con 0x7f50900688d0 2023-10-08T00:56:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5096d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509006af00 con 0x7f509006bfa0 2023-10-08T00:56:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5095d9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5090159a70 0x7f50900c82f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37510/0 (socket says 172.21.15.119:37510) 2023-10-08T00:56:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5095d9b700 1 -- 172.21.15.119:0/3834686569 learned_addr learned my addr 172.21.15.119:0/3834686569 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f509659c700 1 --2- 172.21.15.119:0/3834686569 >> v2:172.21.15.119:3300/0 conn(0x7f50900688d0 0x7f50900caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5094d99700 1 -- 172.21.15.119:0/3834686569 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200477039 0 0) 0x7f50900ca920 con 0x7f5090159a70 2023-10-08T00:56:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f5094d99700 1 -- 172.21.15.119:0/3834686569 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f507c003570 con 0x7f5090159a70 2023-10-08T00:56:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.163+0000 7f509559a700 1 --2- 172.21.15.119:0/3834686569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 0x7f5090068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.164+0000 7f509659c700 1 -- 172.21.15.119:0/3834686569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 msgr2=0x7f5090068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.164+0000 7f509659c700 1 --2- 172.21.15.119:0/3834686569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 0x7f5090068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.164+0000 7f509659c700 1 -- 172.21.15.119:0/3834686569 >> v1:172.21.15.119:6789/0 conn(0x7f5090159a70 legacy=0x7f50900c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.164+0000 7f509659c700 1 -- 172.21.15.119:0/3834686569 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509006b1e0 con 0x7f50900688d0 2023-10-08T00:56:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.164+0000 7f509659c700 1 --2- 172.21.15.119:0/3834686569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50900688d0 0x7f50900caec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5088009e60 tx=0x7f5088009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5181d130fdc5a6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.164+0000 7f5094d99700 1 -- 172.21.15.119:0/3834686569 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508800e070 con 0x7f50900688d0 2023-10-08T00:56:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.164+0000 7f5094d99700 1 -- 172.21.15.119:0/3834686569 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508800bb60 con 0x7f50900688d0 2023-10-08T00:56:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.164+0000 7f5094d99700 1 -- 172.21.15.119:0/3834686569 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088012430 con 0x7f50900688d0 2023-10-08T00:56:17.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.165+0000 7f5096d9d700 1 -- 172.21.15.119:0/3834686569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50900688d0 msgr2=0x7f50900caec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.165+0000 7f5096d9d700 1 --2- 172.21.15.119:0/3834686569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50900688d0 0x7f50900caec0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5088009e60 tx=0x7f5088009780 comp rx=0 tx=0).stop 2023-10-08T00:56:17.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.165+0000 7f5096d9d700 1 -- 172.21.15.119:0/3834686569 shutdown_connections 2023-10-08T00:56:17.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.165+0000 7f5096d9d700 1 --2- 172.21.15.119:0/3834686569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50900688d0 0x7f50900caec0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.165+0000 7f5096d9d700 1 --2- 172.21.15.119:0/3834686569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 0x7f5090068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.165+0000 7f5096d9d700 1 -- 172.21.15.119:0/3834686569 >> 172.21.15.119:0/3834686569 conn(0x7f50900cddc0 msgr2=0x7f50900c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:17.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.165+0000 7f5096d9d700 1 -- 172.21.15.119:0/3834686569 shutdown_connections 2023-10-08T00:56:17.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.165+0000 7f5096d9d700 1 -- 172.21.15.119:0/3834686569 wait complete. 2023-10-08T00:56:17.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.166+0000 7f5096d9d700 1 Processor -- start 2023-10-08T00:56:17.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.166+0000 7f5096d9d700 1 -- start start 2023-10-08T00:56:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.167+0000 7f5096d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50900688d0 0x7f5090155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.167+0000 7f5096d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 0x7f5090152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.167+0000 7f5096d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5090159a70 0x7f5090157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.167+0000 7f5096d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50900c6630 con 0x7f5090159a70 2023-10-08T00:56:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.167+0000 7f5096d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50900b1290 con 0x7f50900688d0 2023-10-08T00:56:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.167+0000 7f5096d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50900c6070 con 0x7f509006bfa0 2023-10-08T00:56:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.167+0000 7f509659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5090159a70 0x7f5090157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5090159a70 0x7f5090157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34982/0 (socket says 172.21.15.119:34982) 2023-10-08T00:56:17.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509659c700 1 -- 172.21.15.119:0/2474183952 learned_addr learned my addr 172.21.15.119:0/2474183952 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:17.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509559a700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 0x7f5090152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f5095d9b700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50900688d0 0x7f5090155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509659c700 1 -- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 msgr2=0x7f5090152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509659c700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 0x7f5090152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509659c700 1 -- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50900688d0 msgr2=0x7f5090155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509659c700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50900688d0 0x7f5090155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509659c700 1 -- 172.21.15.119:0/2474183952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5088009a20 con 0x7f5090159a70 2023-10-08T00:56:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f5095d9b700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50900688d0 0x7f5090155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f509659c700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5090159a70 0x7f5090157a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f509006a6a0 tx=0x7f5088019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f5085ffb700 1 -- 172.21.15.119:0/2474183952 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088012e50 con 0x7f5090159a70 2023-10-08T00:56:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f5085ffb700 1 -- 172.21.15.119:0/2474183952 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508801d630 con 0x7f5090159a70 2023-10-08T00:56:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f5085ffb700 1 -- 172.21.15.119:0/2474183952 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508801d9f0 con 0x7f5090159a70 2023-10-08T00:56:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.168+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50900c6350 con 0x7f5090159a70 2023-10-08T00:56:17.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.169+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5090153430 con 0x7f5090159a70 2023-10-08T00:56:17.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.170+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50900b1060 con 0x7f5090159a70 2023-10-08T00:56:17.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.170+0000 7f5085ffb700 1 -- 172.21.15.119:0/2474183952 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5088024070 con 0x7f5090159a70 2023-10-08T00:56:17.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.171+0000 7f5085ffb700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5074066f30 0x7f50740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.171+0000 7f5085ffb700 1 -- 172.21.15.119:0/2474183952 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5088086cf0 con 0x7f5090159a70 2023-10-08T00:56:17.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.171+0000 7f5095d9b700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5074066f30 0x7f50740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.172+0000 7f5095d9b700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5074066f30 0x7f50740693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f508c0058d0 tx=0x7f508c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.175+0000 7f5085ffb700 1 -- 172.21.15.119:0/2474183952 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f50900b1060 con 0x7f5090159a70 2023-10-08T00:56:17.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.362+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50900b1060 con 0x7f5090159a70 2023-10-08T00:56:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.362+0000 7f5085ffb700 1 -- 172.21.15.119:0/2474183952 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f50900b1060 con 0x7f5090159a70 2023-10-08T00:56:17.363 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:56:17.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.364+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5074066f30 msgr2=0x7f50740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.364+0000 7f5096d9d700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5074066f30 0x7f50740693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f508c0058d0 tx=0x7f508c009040 comp rx=0 tx=0).stop 2023-10-08T00:56:17.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.364+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5090159a70 msgr2=0x7f5090157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.364+0000 7f5096d9d700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5090159a70 0x7f5090157a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f509006a6a0 tx=0x7f5088019000 comp rx=0 tx=0).stop 2023-10-08T00:56:17.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.365+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 shutdown_connections 2023-10-08T00:56:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.365+0000 7f5096d9d700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5074066f30 0x7f50740693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.365+0000 7f5096d9d700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5090159a70 0x7f5090157a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.365+0000 7f5096d9d700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f50900688d0 0x7f5090155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.365+0000 7f5096d9d700 1 --2- 172.21.15.119:0/2474183952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f509006bfa0 0x7f5090152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.365+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 >> 172.21.15.119:0/2474183952 conn(0x7f50900cddc0 msgr2=0x7f50900c2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:17.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.365+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 shutdown_connections 2023-10-08T00:56:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.365+0000 7f5096d9d700 1 -- 172.21.15.119:0/2474183952 wait complete. 2023-10-08T00:56:17.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:56:17.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.565+0000 7f5e01dff700 1 Processor -- start 2023-10-08T00:56:17.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.565+0000 7f5e01dff700 1 -- start start 2023-10-08T00:56:17.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5e01dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc155eb0 0x7f5dfc06e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5e01dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dfc069e80 0x7f5dfc0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5e01dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dfc06eb90 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5e01dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dfc06ee70 con 0x7f5dfc069e80 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5e01dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc06f150 con 0x7f5dfc155eb0 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5dfbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dfc069e80 0x7f5dfc0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5df3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5dfc0c70b0 0x7f5dfc069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37524/0 (socket says 172.21.15.119:37524) 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5df3fff700 1 -- 172.21.15.119:0/972460093 learned_addr learned my addr 172.21.15.119:0/972460093 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5dfb7fe700 1 --2- 172.21.15.119:0/972460093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc155eb0 0x7f5dfc06e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5dfbfff700 1 -- 172.21.15.119:0/972460093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc155eb0 msgr2=0x7f5dfc06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5dfbfff700 1 --2- 172.21.15.119:0/972460093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc155eb0 0x7f5dfc06e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5dfbfff700 1 -- 172.21.15.119:0/972460093 >> v1:172.21.15.119:6789/0 conn(0x7f5dfc0c70b0 legacy=0x7f5dfc069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5dfbfff700 1 -- 172.21.15.119:0/972460093 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc06f430 con 0x7f5dfc069e80 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5dfb7fe700 1 --2- 172.21.15.119:0/972460093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc155eb0 0x7f5dfc06e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:17.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.567+0000 7f5dfaffd700 1 -- 172.21.15.119:0/972460093 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911998161 0 0) 0x7f5dfc06eb90 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5dfbfff700 1 --2- 172.21.15.119:0/972460093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc069e80 0x7f5dfc0ca4c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009e60 tx=0x7f5df4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8798c87a60f53cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5dfaffd700 1 -- 172.21.15.119:0/972460093 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400e070 con 0x7f5dfc069e80 2023-10-08T00:56:17.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5dfaffd700 1 -- 172.21.15.119:0/972460093 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df400bc30 con 0x7f5dfc069e80 2023-10-08T00:56:17.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5dfaffd700 1 -- 172.21.15.119:0/972460093 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4012430 con 0x7f5dfc069e80 2023-10-08T00:56:17.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5e01dff700 1 -- 172.21.15.119:0/972460093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc069e80 msgr2=0x7f5dfc0ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5e01dff700 1 --2- 172.21.15.119:0/972460093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc069e80 0x7f5dfc0ca4c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009e60 tx=0x7f5df4009780 comp rx=0 tx=0).stop 2023-10-08T00:56:17.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5e01dff700 1 -- 172.21.15.119:0/972460093 shutdown_connections 2023-10-08T00:56:17.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5e01dff700 1 --2- 172.21.15.119:0/972460093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc069e80 0x7f5dfc0ca4c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.568+0000 7f5e01dff700 1 --2- 172.21.15.119:0/972460093 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc155eb0 0x7f5dfc06e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.569+0000 7f5e01dff700 1 -- 172.21.15.119:0/972460093 >> 172.21.15.119:0/972460093 conn(0x7f5dfc068e10 msgr2=0x7f5dfc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:17.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.569+0000 7f5e01dff700 1 -- 172.21.15.119:0/972460093 shutdown_connections 2023-10-08T00:56:17.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.569+0000 7f5e01dff700 1 -- 172.21.15.119:0/972460093 wait complete. 2023-10-08T00:56:17.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.570+0000 7f5e01dff700 1 Processor -- start 2023-10-08T00:56:17.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.570+0000 7f5e01dff700 1 -- start start 2023-10-08T00:56:17.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5e01dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc069e80 0x7f5dfc16a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5e01dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc0c70b0 0x7f5dfc0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5e01dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dfc155eb0 0x7f5dfc16cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5e01dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc16ded0 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5e01dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc169930 con 0x7f5dfc155eb0 2023-10-08T00:56:17.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5e01dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc169eb0 con 0x7f5dfc069e80 2023-10-08T00:56:17.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5df3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc0c70b0 0x7f5dfc0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5df3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc0c70b0 0x7f5dfc0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34998/0 (socket says 172.21.15.119:34998) 2023-10-08T00:56:17.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.571+0000 7f5df3fff700 1 -- 172.21.15.119:0/150770082 learned_addr learned my addr 172.21.15.119:0/150770082 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:17.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5dfb7fe700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc069e80 0x7f5dfc16a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df3fff700 1 -- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc069e80 msgr2=0x7f5dfc16a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df3fff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc069e80 0x7f5dfc16a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5dfbfff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dfc155eb0 0x7f5dfc16cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df3fff700 1 -- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dfc155eb0 msgr2=0x7f5dfc16cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df3fff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dfc155eb0 0x7f5dfc16cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df3fff700 1 -- 172.21.15.119:0/150770082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df4009a20 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5dfb7fe700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc069e80 0x7f5dfc16a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df3fff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc0c70b0 0x7f5dfc0add40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5de400c7e0 tx=0x7f5de4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df37fe700 1 -- 172.21.15.119:0/150770082 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4005b30 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df37fe700 1 -- 172.21.15.119:0/150770082 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de4005c90 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5df37fe700 1 -- 172.21.15.119:0/150770082 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de4007300 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc16d910 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.572+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc169bc0 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.573+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc16a940 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.574+0000 7f5df37fe700 1 -- 172.21.15.119:0/150770082 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5de401b400 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.575+0000 7f5df37fe700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ddc066f30 0x7f5ddc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.575+0000 7f5df37fe700 1 -- 172.21.15.119:0/150770082 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5de40840d0 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.575+0000 7f5dfb7fe700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ddc066f30 0x7f5ddc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.576+0000 7f5dfb7fe700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ddc066f30 0x7f5ddc0693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5dec006fd0 tx=0x7f5dec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.579+0000 7f5df37fe700 1 -- 172.21.15.119:0/150770082 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dfc16a940 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.770+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5dfc16a940 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.770+0000 7f5df37fe700 1 -- 172.21.15.119:0/150770082 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5dfc16a940 con 0x7f5dfc0c70b0 2023-10-08T00:56:17.771 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T00:56:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.772+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ddc066f30 msgr2=0x7f5ddc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ddc066f30 0x7f5ddc0693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5dec006fd0 tx=0x7f5dec008040 comp rx=0 tx=0).stop 2023-10-08T00:56:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc0c70b0 msgr2=0x7f5dfc0add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc0c70b0 0x7f5dfc0add40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5de400c7e0 tx=0x7f5de4007df0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 shutdown_connections 2023-10-08T00:56:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dfc069e80 0x7f5dfc16a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ddc066f30 0x7f5ddc0693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dfc155eb0 0x7f5dfc16cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 --2- 172.21.15.119:0/150770082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dfc0c70b0 0x7f5dfc0add40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 >> 172.21.15.119:0/150770082 conn(0x7f5dfc068e10 msgr2=0x7f5dfc0c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 shutdown_connections 2023-10-08T00:56:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.773+0000 7f5e01dff700 1 -- 172.21.15.119:0/150770082 wait complete. 2023-10-08T00:56:17.790 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:56:17.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.974+0000 7f031f19d700 1 Processor -- start 2023-10-08T00:56:17.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.975+0000 7f031f19d700 1 -- start start 2023-10-08T00:56:17.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.976+0000 7f031f19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0318153900 0x7f03180c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.976+0000 7f031f19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031806e150 con 0x7f03180c9db0 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.976+0000 7f031f19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03180c7800 con 0x7f031806a710 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.976+0000 7f031f19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03180c7ae0 con 0x7f0318153900 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.976+0000 7f031e19b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0318153900 0x7f03180c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.976+0000 7f031d99a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f03180c9db0 0x7f031806a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37528/0 (socket says 172.21.15.119:37528) 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.976+0000 7f031d99a700 1 -- 172.21.15.119:0/3340893565 learned_addr learned my addr 172.21.15.119:0/3340893565 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.977+0000 7f031e19b700 1 -- 172.21.15.119:0/3340893565 >> v1:172.21.15.184:6789/0 conn(0x7f031806a710 legacy=0x7f031806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.977+0000 7f031e19b700 1 -- 172.21.15.119:0/3340893565 >> v1:172.21.15.119:6789/0 conn(0x7f03180c9db0 legacy=0x7f031806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.977+0000 7f031e19b700 1 -- 172.21.15.119:0/3340893565 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180c7dc0 con 0x7f0318153900 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.977+0000 7f031e19b700 1 --2- 172.21.15.119:0/3340893565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0318153900 0x7f03180c6f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03140098e0 tx=0x7f03140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3634abd39f27a089 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.977+0000 7f031d199700 1 -- 172.21.15.119:0/3340893565 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314009e80 con 0x7f0318153900 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.977+0000 7f031d199700 1 -- 172.21.15.119:0/3340893565 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031400b820 con 0x7f0318153900 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.977+0000 7f031d199700 1 -- 172.21.15.119:0/3340893565 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031400bba0 con 0x7f0318153900 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.978+0000 7f031f19d700 1 -- 172.21.15.119:0/3340893565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0318153900 msgr2=0x7f03180c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.978+0000 7f031f19d700 1 --2- 172.21.15.119:0/3340893565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0318153900 0x7f03180c6f00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f03140098e0 tx=0x7f03140091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.978+0000 7f031f19d700 1 -- 172.21.15.119:0/3340893565 shutdown_connections 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.978+0000 7f031f19d700 1 --2- 172.21.15.119:0/3340893565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0318153900 0x7f03180c6f00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.978+0000 7f031f19d700 1 -- 172.21.15.119:0/3340893565 >> 172.21.15.119:0/3340893565 conn(0x7f03180cc1e0 msgr2=0x7f03180cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:17.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.978+0000 7f031f19d700 1 -- 172.21.15.119:0/3340893565 shutdown_connections 2023-10-08T00:56:17.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.978+0000 7f031f19d700 1 -- 172.21.15.119:0/3340893565 wait complete. 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.979+0000 7f031f19d700 1 Processor -- start 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.979+0000 7f031f19d700 1 -- start start 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031f19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f031806a710 0x7f0318163380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031f19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03180c9db0 0x7f03181559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031f19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0318153900 0x7f0318157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031f19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03181638c0 con 0x7f031806a710 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031f19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03180ac4b0 con 0x7f03180c9db0 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031f19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0318156440 con 0x7f0318153900 2023-10-08T00:56:17.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031e19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f031806a710 0x7f0318163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031e19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f031806a710 0x7f0318163380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35004/0 (socket says 172.21.15.119:35004) 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.980+0000 7f031e19b700 1 -- 172.21.15.119:0/1098337026 learned_addr learned my addr 172.21.15.119:0/1098337026 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031d99a700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03180c9db0 0x7f03181559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031e99c700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0318153900 0x7f0318157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031e19b700 1 -- 172.21.15.119:0/1098337026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0318153900 msgr2=0x7f0318157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031e19b700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0318153900 0x7f0318157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031e19b700 1 -- 172.21.15.119:0/1098337026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03180c9db0 msgr2=0x7f03181559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031e19b700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03180c9db0 0x7f03181559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031e19b700 1 -- 172.21.15.119:0/1098337026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0314009420 con 0x7f031806a710 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031e19b700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f031806a710 0x7f0318163380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f03140091e0 tx=0x7f0314018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f030e7fc700 1 -- 172.21.15.119:0/1098337026 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031400bba0 con 0x7f031806a710 2023-10-08T00:56:17.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f030e7fc700 1 -- 172.21.15.119:0/1098337026 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031401e040 con 0x7f031806a710 2023-10-08T00:56:17.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03180abb80 con 0x7f031806a710 2023-10-08T00:56:17.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.981+0000 7f030e7fc700 1 -- 172.21.15.119:0/1098337026 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0314018670 con 0x7f031806a710 2023-10-08T00:56:17.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.982+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03180ac140 con 0x7f031806a710 2023-10-08T00:56:17.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.982+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031806c450 con 0x7f031806a710 2023-10-08T00:56:17.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.983+0000 7f030e7fc700 1 -- 172.21.15.119:0/1098337026 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0314018aa0 con 0x7f031806a710 2023-10-08T00:56:17.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.984+0000 7f030e7fc700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f02fc066f30 0x7f02fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:17.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.984+0000 7f030e7fc700 1 -- 172.21.15.119:0/1098337026 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f0314014070 con 0x7f031806a710 2023-10-08T00:56:17.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.984+0000 7f031d99a700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f02fc066f30 0x7f02fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:17.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.985+0000 7f031d99a700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f02fc066f30 0x7f02fc0693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03180aea10 tx=0x7f0308009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:17.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:17.988+0000 7f030e7fc700 1 -- 172.21.15.119:0/1098337026 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03140554b0 con 0x7f031806a710 2023-10-08T00:56:18.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.180+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f031806dd20 con 0x7f031806a710 2023-10-08T00:56:18.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.180+0000 7f030e7fc700 1 -- 172.21.15.119:0/1098337026 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0314058ad0 con 0x7f031806a710 2023-10-08T00:56:18.181 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T00:56:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.182+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f02fc066f30 msgr2=0x7f02fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.182+0000 7f031f19d700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f02fc066f30 0x7f02fc0693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03180aea10 tx=0x7f0308009040 comp rx=0 tx=0).stop 2023-10-08T00:56:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f031806a710 msgr2=0x7f0318163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f031806a710 0x7f0318163380 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f03140091e0 tx=0x7f0314018000 comp rx=0 tx=0).stop 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 shutdown_connections 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0318153900 0x7f0318157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f02fc066f30 0x7f02fc0693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f031806a710 0x7f0318163380 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 --2- 172.21.15.119:0/1098337026 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03180c9db0 0x7f03181559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 >> 172.21.15.119:0/1098337026 conn(0x7f03180cc1e0 msgr2=0x7f031814f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 shutdown_connections 2023-10-08T00:56:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.183+0000 7f031f19d700 1 -- 172.21.15.119:0/1098337026 wait complete. 2023-10-08T00:56:18.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.381+0000 7fb06d9ff700 1 Processor -- start 2023-10-08T00:56:18.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.381+0000 7fb06d9ff700 1 -- start start 2023-10-08T00:56:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.382+0000 7fb06d9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb068155c10 0x7fb068155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.382+0000 7fb06d9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb0680ac6a0 0x7fb0680b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.382+0000 7fb06d9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0680acac0 con 0x7fb0680ac230 2023-10-08T00:56:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.382+0000 7fb06d9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0680ad930 con 0x7fb0680ac6a0 2023-10-08T00:56:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.382+0000 7fb06d9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0680acd20 con 0x7fb068155c10 2023-10-08T00:56:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb05ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb0680ac230 0x7fb0681568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37538/0 (socket says 172.21.15.119:37538) 2023-10-08T00:56:18.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb05ffff700 1 -- 172.21.15.119:0/1803113144 learned_addr learned my addr 172.21.15.119:0/1803113144 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb066ffd700 1 --2- 172.21.15.119:0/1803113144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb068155c10 0x7fb068155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb0677fe700 1 --2- 172.21.15.119:0/1803113144 >> v2:172.21.15.184:3300/0 conn(0x7fb0680ac6a0 0x7fb0680b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb0667fc700 1 -- 172.21.15.119:0/1803113144 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051362271 0 0) 0x7fb0680acac0 con 0x7fb0680ac230 2023-10-08T00:56:18.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb0667fc700 1 -- 172.21.15.119:0/1803113144 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb054003570 con 0x7fb0680ac230 2023-10-08T00:56:18.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb066ffd700 1 -- 172.21.15.119:0/1803113144 >> v2:172.21.15.184:3300/0 conn(0x7fb0680ac6a0 msgr2=0x7fb0680b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:18.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb066ffd700 1 --2- 172.21.15.119:0/1803113144 >> v2:172.21.15.184:3300/0 conn(0x7fb0680ac6a0 0x7fb0680b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb066ffd700 1 -- 172.21.15.119:0/1803113144 >> v1:172.21.15.119:6789/0 conn(0x7fb0680ac230 legacy=0x7fb0681568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.383+0000 7fb066ffd700 1 -- 172.21.15.119:0/1803113144 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0680ad000 con 0x7fb068155c10 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.384+0000 7fb066ffd700 1 --2- 172.21.15.119:0/1803113144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb068155c10 0x7fb068155fe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb05800bb90 tx=0x7fb05800b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6add14fe85a802ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.384+0000 7fb0667fc700 1 -- 172.21.15.119:0/1803113144 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800d960 con 0x7fb068155c10 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.384+0000 7fb0667fc700 1 -- 172.21.15.119:0/1803113144 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb05800dae0 con 0x7fb068155c10 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.384+0000 7fb0667fc700 1 -- 172.21.15.119:0/1803113144 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800dea0 con 0x7fb068155c10 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.385+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1803113144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb068155c10 msgr2=0x7fb068155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.385+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1803113144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb068155c10 0x7fb068155fe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb05800bb90 tx=0x7fb05800b430 comp rx=0 tx=0).stop 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.385+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1803113144 shutdown_connections 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.385+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1803113144 >> v2:172.21.15.184:3300/0 conn(0x7fb0680ac6a0 0x7fb0680b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.385+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1803113144 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb068155c10 0x7fb068155fe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.385+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1803113144 >> 172.21.15.119:0/1803113144 conn(0x7fb06806a320 msgr2=0x7fb0680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:18.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.385+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1803113144 shutdown_connections 2023-10-08T00:56:18.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.385+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1803113144 wait complete. 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.386+0000 7fb06d9ff700 1 Processor -- start 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.386+0000 7fb06d9ff700 1 -- start start 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.387+0000 7fb06d9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0680ac230 0x7fb068150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.387+0000 7fb06d9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0680ac6a0 0x7fb068159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.387+0000 7fb06d9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb068155c10 0x7fb068152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.387+0000 7fb06d9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06815b970 con 0x7fb068155c10 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.387+0000 7fb06d9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06815a8d0 con 0x7fb0680ac6a0 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.387+0000 7fb06d9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb06815abb0 con 0x7fb0680ac230 2023-10-08T00:56:18.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb068155c10 0x7fb068152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb068155c10 0x7fb068152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35018/0 (socket says 172.21.15.119:35018) 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 -- 172.21.15.119:0/1272571779 learned_addr learned my addr 172.21.15.119:0/1272571779 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb066ffd700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0680ac230 0x7fb068150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb05ffff700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0680ac6a0 0x7fb068159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 -- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0680ac230 msgr2=0x7fb068150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0680ac230 0x7fb068150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 -- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0680ac6a0 msgr2=0x7fb068159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0680ac6a0 0x7fb068159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 -- 172.21.15.119:0/1272571779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05800b6d0 con 0x7fb068155c10 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb05ffff700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0680ac6a0 0x7fb068159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:18.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb0677fe700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb068155c10 0x7fb068152810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb060009e90 tx=0x7fb06000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb05effd700 1 -- 172.21.15.119:0/1272571779 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06000cc20 con 0x7fb068155c10 2023-10-08T00:56:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb05effd700 1 -- 172.21.15.119:0/1272571779 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb06000f040 con 0x7fb068155c10 2023-10-08T00:56:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.388+0000 7fb05effd700 1 -- 172.21.15.119:0/1272571779 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb060014480 con 0x7fb068155c10 2023-10-08T00:56:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.389+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06815ae90 con 0x7fb068155c10 2023-10-08T00:56:18.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.389+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06815b450 con 0x7fb068155c10 2023-10-08T00:56:18.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.390+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb068150d60 con 0x7fb068155c10 2023-10-08T00:56:18.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.390+0000 7fb05effd700 1 -- 172.21.15.119:0/1272571779 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb060014830 con 0x7fb068155c10 2023-10-08T00:56:18.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.391+0000 7fb05effd700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb048066f30 0x7fb0480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.391+0000 7fb05effd700 1 -- 172.21.15.119:0/1272571779 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb060085260 con 0x7fb068155c10 2023-10-08T00:56:18.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.392+0000 7fb066ffd700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb048066f30 0x7fb0480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.392+0000 7fb066ffd700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb048066f30 0x7fb0480693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb05800bb90 tx=0x7fb058019000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:18.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.395+0000 7fb05effd700 1 -- 172.21.15.119:0/1272571779 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb060055f60 con 0x7fb068155c10 2023-10-08T00:56:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.592+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb06804c3c0 con 0x7fb068155c10 2023-10-08T00:56:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.593+0000 7fb05effd700 1 -- 172.21.15.119:0/1272571779 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fb060055af0 con 0x7fb068155c10 2023-10-08T00:56:18.593 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:18.594 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:18.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb048066f30 msgr2=0x7fb0480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb048066f30 0x7fb0480693e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb05800bb90 tx=0x7fb058019000 comp rx=0 tx=0).stop 2023-10-08T00:56:18.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb068155c10 msgr2=0x7fb068152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb068155c10 0x7fb068152810 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb060009e90 tx=0x7fb06000b040 comp rx=0 tx=0).stop 2023-10-08T00:56:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 shutdown_connections 2023-10-08T00:56:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb0680ac230 0x7fb068150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb048066f30 0x7fb0480693e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb068155c10 0x7fb068152810 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 --2- 172.21.15.119:0/1272571779 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb0680ac6a0 0x7fb068159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 >> 172.21.15.119:0/1272571779 conn(0x7fb06806a320 msgr2=0x7fb068153690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 shutdown_connections 2023-10-08T00:56:18.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.596+0000 7fb06d9ff700 1 -- 172.21.15.119:0/1272571779 wait complete. 2023-10-08T00:56:18.614 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:55:45.270364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:55:45.937180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:56:18.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:56:18.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.798+0000 7fe2223ff700 1 Processor -- start 2023-10-08T00:56:18.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.798+0000 7fe2223ff700 1 -- start start 2023-10-08T00:56:18.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.799+0000 7fe2223ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.799+0000 7fe2223ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21c06e950 con 0x7fe21c0cd0b0 2023-10-08T00:56:18.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.799+0000 7fe2223ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21c06eb90 con 0x7fe21c0ccc40 2023-10-08T00:56:18.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.799+0000 7fe2223ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c1563f0 con 0x7fe21c0c70b0 2023-10-08T00:56:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe22099c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe21c0cd0b0 0x7fe21c069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37542/0 (socket says 172.21.15.119:37542) 2023-10-08T00:56:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe22099c700 1 -- 172.21.15.119:0/711228402 learned_addr learned my addr 172.21.15.119:0/711228402 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe21bfff700 1 --2- 172.21.15.119:0/711228402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe21affd700 1 -- 172.21.15.119:0/711228402 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938250769 0 0) 0x7fe214001490 con 0x7fe21c0cd0b0 2023-10-08T00:56:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe21affd700 1 -- 172.21.15.119:0/711228402 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe204003570 con 0x7fe21c0cd0b0 2023-10-08T00:56:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe21bfff700 1 -- 172.21.15.119:0/711228402 >> v1:172.21.15.184:6789/0 conn(0x7fe21c0ccc40 legacy=0x7fe21c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe21affd700 1 -- 172.21.15.119:0/711228402 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303840718 0 0) 0x7fe21c06eb90 con 0x7fe21c0ccc40 2023-10-08T00:56:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe21bfff700 1 -- 172.21.15.119:0/711228402 >> v1:172.21.15.119:6789/0 conn(0x7fe21c0cd0b0 legacy=0x7fe21c069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe21bfff700 1 -- 172.21.15.119:0/711228402 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c06ee70 con 0x7fe21c0c70b0 2023-10-08T00:56:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.800+0000 7fe21bfff700 1 --2- 172.21.15.119:0/711228402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c155eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009910 tx=0x7fe20c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290fc518706b3000 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe21affd700 1 -- 172.21.15.119:0/711228402 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00b680 con 0x7fe21c0c70b0 2023-10-08T00:56:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe21affd700 1 -- 172.21.15.119:0/711228402 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c00b7e0 con 0x7fe21c0c70b0 2023-10-08T00:56:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe21affd700 1 -- 172.21.15.119:0/711228402 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00bba0 con 0x7fe21c0c70b0 2023-10-08T00:56:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe2223ff700 1 -- 172.21.15.119:0/711228402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 msgr2=0x7fe21c155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe2223ff700 1 --2- 172.21.15.119:0/711228402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c155eb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe20c009910 tx=0x7fe20c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe2223ff700 1 -- 172.21.15.119:0/711228402 shutdown_connections 2023-10-08T00:56:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe2223ff700 1 --2- 172.21.15.119:0/711228402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c155eb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe2223ff700 1 -- 172.21.15.119:0/711228402 >> 172.21.15.119:0/711228402 conn(0x7fe21c068e10 msgr2=0x7fe21c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe2223ff700 1 -- 172.21.15.119:0/711228402 shutdown_connections 2023-10-08T00:56:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.801+0000 7fe2223ff700 1 -- 172.21.15.119:0/711228402 wait complete. 2023-10-08T00:56:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.803+0000 7fe2223ff700 1 Processor -- start 2023-10-08T00:56:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.803+0000 7fe2223ff700 1 -- start start 2023-10-08T00:56:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe2223ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c0aa730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe2223ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c0ccc40 0x7fe21c14d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe2223ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe21c0cd0b0 0x7fe21c0cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe2223ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0b04a0 con 0x7fe21c0ccc40 2023-10-08T00:56:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe2223ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0aac70 con 0x7fe21c0cd0b0 2023-10-08T00:56:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe2223ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c14d610 con 0x7fe21c0c70b0 2023-10-08T00:56:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c0ccc40 0x7fe21c14d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c0ccc40 0x7fe21c14d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35034/0 (socket says 172.21.15.119:35034) 2023-10-08T00:56:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21b7fe700 1 -- 172.21.15.119:0/52826878 learned_addr learned my addr 172.21.15.119:0/52826878 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:18.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21bfff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c0aa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe22099c700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe21c0cd0b0 0x7fe21c0cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21b7fe700 1 -- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 msgr2=0x7fe21c0aa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21b7fe700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c0aa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21b7fe700 1 -- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe21c0cd0b0 msgr2=0x7fe21c0cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21b7fe700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe21c0cd0b0 0x7fe21c0cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe21b7fe700 1 -- 172.21.15.119:0/52826878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c009420 con 0x7fe21c0ccc40 2023-10-08T00:56:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.804+0000 7fe22099c700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe21c0cd0b0 0x7fe21c0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:18.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.805+0000 7fe21b7fe700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c0ccc40 0x7fe21c14d0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe21000c7e0 tx=0x7fe210007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.805+0000 7fe203fff700 1 -- 172.21.15.119:0/52826878 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210011590 con 0x7fe21c0ccc40 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.805+0000 7fe203fff700 1 -- 172.21.15.119:0/52826878 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe210011710 con 0x7fe21c0ccc40 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.805+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c0c1ee0 con 0x7fe21c0ccc40 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.805+0000 7fe203fff700 1 -- 172.21.15.119:0/52826878 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe210011ad0 con 0x7fe21c0ccc40 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.805+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c0a9f30 con 0x7fe21c0ccc40 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.806+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21c0a9a80 con 0x7fe21c0ccc40 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.807+0000 7fe203fff700 1 -- 172.21.15.119:0/52826878 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe21001b3b0 con 0x7fe21c0ccc40 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.808+0000 7fe203fff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe208066f30 0x7fe2080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.808+0000 7fe203fff700 1 -- 172.21.15.119:0/52826878 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe210083a80 con 0x7fe21c0ccc40 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.808+0000 7fe21bfff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe208066f30 0x7fe2080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:18.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.809+0000 7fe21bfff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe208066f30 0x7fe2080693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0093f0 tx=0x7fe20c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:18.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:18.812+0000 7fe203fff700 1 -- 172.21.15.119:0/52826878 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe210050cf0 con 0x7fe21c0ccc40 2023-10-08T00:56:19.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.003+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe21c04c3c0 con 0x7fe21c0ccc40 2023-10-08T00:56:19.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.004+0000 7fe203fff700 1 -- 172.21.15.119:0/52826878 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe210017070 con 0x7fe21c0ccc40 2023-10-08T00:56:19.004 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:56:19.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe208066f30 msgr2=0x7fe2080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe208066f30 0x7fe2080693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe20c0093f0 tx=0x7fe20c017000 comp rx=0 tx=0).stop 2023-10-08T00:56:19.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c0ccc40 msgr2=0x7fe21c14d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c0ccc40 0x7fe21c14d0d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe21000c7e0 tx=0x7fe210007df0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 shutdown_connections 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe21c0c70b0 0x7fe21c0aa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe208066f30 0x7fe2080693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe21c0cd0b0 0x7fe21c0cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 --2- 172.21.15.119:0/52826878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe21c0ccc40 0x7fe21c14d0d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 >> 172.21.15.119:0/52826878 conn(0x7fe21c068e10 msgr2=0x7fe21c0ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 shutdown_connections 2023-10-08T00:56:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.006+0000 7fe2223ff700 1 -- 172.21.15.119:0/52826878 wait complete. 2023-10-08T00:56:19.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:56:19.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.206+0000 7f5b23fff700 1 Processor -- start 2023-10-08T00:56:19.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.206+0000 7f5b23fff700 1 -- start start 2023-10-08T00:56:19.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.207+0000 7f5b23fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240b16d0 0x7f5b240b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.207+0000 7f5b23fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b240b2080 con 0x7f5b240b4e70 2023-10-08T00:56:19.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.207+0000 7f5b23fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b240cfb00 con 0x7f5b240b33b0 2023-10-08T00:56:19.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.207+0000 7f5b23fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b5c40 con 0x7f5b240b16d0 2023-10-08T00:56:19.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.207+0000 7f5b22ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5b240b4e70 0x7f5b240ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37546/0 (socket says 172.21.15.119:37546) 2023-10-08T00:56:19.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.207+0000 7f5b22ffd700 1 -- 172.21.15.119:0/1538706540 learned_addr learned my addr 172.21.15.119:0/1538706540 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:19.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b227fc700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240b16d0 0x7f5b240b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856938928 0 0) 0x7f5b240b2080 con 0x7f5b240b4e70 2023-10-08T00:56:19.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b0c003570 con 0x7f5b240b4e70 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1085566555 0 0) 0x7f5b0c003570 con 0x7f5b240b4e70 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240b16d0 msgr2=0x7f5b240b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240b16d0 0x7f5b240b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 >> v1:172.21.15.184:6789/0 conn(0x7f5b240b33b0 legacy=0x7f5b240b3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240b5f20 con 0x7f5b240b4e70 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f5b14001470 con 0x7f5b240b4e70 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 >> v1:172.21.15.119:6789/0 conn(0x7f5b240b4e70 legacy=0x7f5b240ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240b16d0 0x7f5b0c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c003fd0 0x7f5b0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0c004d50 0x7f5b0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240b5c40 con 0x7f5b0c004d50 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c00bbf0 con 0x7f5b0c003fd0 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.208+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c00bdb0 con 0x7f5b240b16d0 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b237fe700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0c004d50 0x7f5b0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b227fc700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c003fd0 0x7f5b0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b22ffd700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240b16d0 0x7f5b0c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b237fe700 1 -- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240b16d0 msgr2=0x7f5b0c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b237fe700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b240b16d0 0x7f5b0c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b237fe700 1 -- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c003fd0 msgr2=0x7f5b0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b237fe700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b0c003fd0 0x7f5b0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b237fe700 1 -- 172.21.15.119:0/1538706540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c003570 con 0x7f5b0c004d50 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b237fe700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0c004d50 0x7f5b0c0051c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0020a0 tx=0x7f5b1c0016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95135054f9b31c07 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:19.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c0019e0 con 0x7f5b0c004d50 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c00c8b0 con 0x7f5b0c004d50 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.209+0000 7f5b21ffb700 1 -- 172.21.15.119:0/1538706540 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00cc50 con 0x7f5b0c004d50 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.210+0000 7f5b23fff700 1 -- 172.21.15.119:0/1538706540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0c004d50 msgr2=0x7f5b0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.210+0000 7f5b23fff700 1 --2- 172.21.15.119:0/1538706540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b0c004d50 0x7f5b0c0051c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0020a0 tx=0x7f5b1c0016d0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.210+0000 7f5b237fe700 1 -- 172.21.15.119:0/1538706540 reap_dead start 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.210+0000 7f5b23fff700 1 -- 172.21.15.119:0/1538706540 shutdown_connections 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.210+0000 7f5b23fff700 1 -- 172.21.15.119:0/1538706540 >> 172.21.15.119:0/1538706540 conn(0x7f5b240d10c0 msgr2=0x7f5b240affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.210+0000 7f5b23fff700 1 -- 172.21.15.119:0/1538706540 shutdown_connections 2023-10-08T00:56:19.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.210+0000 7f5b23fff700 1 -- 172.21.15.119:0/1538706540 wait complete. 2023-10-08T00:56:19.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.211+0000 7f5b23fff700 1 Processor -- start 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.212+0000 7f5b23fff700 1 -- start start 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.212+0000 7f5b23fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b0c004d50 0x7f5b2414bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.212+0000 7f5b23fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b240c8240 0x7f5b240c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.212+0000 7f5b23fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2415e330 0x7f5b2415e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.212+0000 7f5b23fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2415ece0 con 0x7f5b2415e330 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.212+0000 7f5b23fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b241695e0 con 0x7f5b240c8240 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.212+0000 7f5b23fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240c8d00 con 0x7f5b0c004d50 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2415e330 0x7f5b2415e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2415e330 0x7f5b2415e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35060/0 (socket says 172.21.15.119:35060) 2023-10-08T00:56:19.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 -- 172.21.15.119:0/3810638425 learned_addr learned my addr 172.21.15.119:0/3810638425 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b22ffd700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b0c004d50 0x7f5b2414bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b227fc700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b240c8240 0x7f5b240c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 -- 172.21.15.119:0/3810638425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b0c004d50 msgr2=0x7f5b2414bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b0c004d50 0x7f5b2414bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 -- 172.21.15.119:0/3810638425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b240c8240 msgr2=0x7f5b240c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b240c8240 0x7f5b240c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 -- 172.21.15.119:0/3810638425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b240c8fe0 con 0x7f5b2415e330 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.213+0000 7f5b237fe700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2415e330 0x7f5b2415e7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c012000 tx=0x7f5b1c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.214+0000 7f5b0affd700 1 -- 172.21.15.119:0/3810638425 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00c6d0 con 0x7f5b2415e330 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.214+0000 7f5b0affd700 1 -- 172.21.15.119:0/3810638425 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1c022070 con 0x7f5b2415e330 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.214+0000 7f5b0affd700 1 -- 172.21.15.119:0/3810638425 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c00cc30 con 0x7f5b2415e330 2023-10-08T00:56:19.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.214+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b240c92c0 con 0x7f5b2415e330 2023-10-08T00:56:19.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.214+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b240c7bf0 con 0x7f5b2415e330 2023-10-08T00:56:19.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.215+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b24169790 con 0x7f5b2415e330 2023-10-08T00:56:19.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.216+0000 7f5b0affd700 1 -- 172.21.15.119:0/3810638425 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b1c01e430 con 0x7f5b2415e330 2023-10-08T00:56:19.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.217+0000 7f5b0affd700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5b10066f30 0x7f5b100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.218+0000 7f5b0affd700 1 -- 172.21.15.119:0/3810638425 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5b1c086350 con 0x7f5b2415e330 2023-10-08T00:56:19.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.218+0000 7f5b22ffd700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5b10066f30 0x7f5b100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.218+0000 7f5b22ffd700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5b10066f30 0x7f5b100693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5b140010e0 tx=0x7f5b14009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:19.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.221+0000 7f5b0affd700 1 -- 172.21.15.119:0/3810638425 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b24169790 con 0x7f5b2415e330 2023-10-08T00:56:19.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.413+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b24169750 con 0x7f5b2415e330 2023-10-08T00:56:19.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.413+0000 7f5b0affd700 1 -- 172.21.15.119:0/3810638425 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b1c0535c0 con 0x7f5b2415e330 2023-10-08T00:56:19.413 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T00:56:19.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.415+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5b10066f30 msgr2=0x7f5b100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.415+0000 7f5b23fff700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5b10066f30 0x7f5b100693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5b140010e0 tx=0x7f5b14009040 comp rx=0 tx=0).stop 2023-10-08T00:56:19.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2415e330 msgr2=0x7f5b2415e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2415e330 0x7f5b2415e7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c012000 tx=0x7f5b1c016000 comp rx=0 tx=0).stop 2023-10-08T00:56:19.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 shutdown_connections 2023-10-08T00:56:19.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b0c004d50 0x7f5b2414bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5b10066f30 0x7f5b100693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b2415e330 0x7f5b2415e7a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 --2- 172.21.15.119:0/3810638425 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b240c8240 0x7f5b240c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 >> 172.21.15.119:0/3810638425 conn(0x7f5b240d10c0 msgr2=0x7f5b24158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:19.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 shutdown_connections 2023-10-08T00:56:19.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.416+0000 7f5b23fff700 1 -- 172.21.15.119:0/3810638425 wait complete. 2023-10-08T00:56:19.433 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:56:19.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.620+0000 7f7eecdff700 1 Processor -- start 2023-10-08T00:56:19.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.620+0000 7f7eecdff700 1 -- start start 2023-10-08T00:56:19.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.621+0000 7f7eecdff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7ee80c83f0 0x7f7ee80c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.621+0000 7f7eecdff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ee80ccee0 0x7f7ee81561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.621+0000 7f7eecdff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee815b0d0 con 0x7f7ee80c87c0 2023-10-08T00:56:19.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.621+0000 7f7eecdff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee80c7a60 con 0x7f7ee80ccee0 2023-10-08T00:56:19.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.621+0000 7f7eecdff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee815a4c0 con 0x7f7ee80c83f0 2023-10-08T00:56:19.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee5d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ee80ccee0 0x7f7ee81561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee6d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7ee80c87c0 0x7f7ee806eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37548/0 (socket says 172.21.15.119:37548) 2023-10-08T00:56:19.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee6d9d700 1 -- 172.21.15.119:0/638872303 learned_addr learned my addr 172.21.15.119:0/638872303 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:19.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee659c700 1 --2- 172.21.15.119:0/638872303 >> v2:172.21.15.184:3300/0 conn(0x7f7ee80c83f0 0x7f7ee80c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee559a700 1 -- 172.21.15.119:0/638872303 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561092321 0 0) 0x7f7ee815b0d0 con 0x7f7ee80c87c0 2023-10-08T00:56:19.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee5d9b700 1 -- 172.21.15.119:0/638872303 >> v2:172.21.15.184:3300/0 conn(0x7f7ee80c83f0 msgr2=0x7f7ee80c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:19.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee5d9b700 1 --2- 172.21.15.119:0/638872303 >> v2:172.21.15.184:3300/0 conn(0x7f7ee80c83f0 0x7f7ee80c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee5d9b700 1 -- 172.21.15.119:0/638872303 >> v1:172.21.15.119:6789/0 conn(0x7f7ee80c87c0 legacy=0x7f7ee806eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee5d9b700 1 -- 172.21.15.119:0/638872303 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee815a7a0 con 0x7f7ee80ccee0 2023-10-08T00:56:19.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.622+0000 7f7ee5d9b700 1 --2- 172.21.15.119:0/638872303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ccee0 0x7f7ee81561d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80c9fd0 tx=0x7f7edc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e872d4f5f7a7c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:19.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7ee559a700 1 -- 172.21.15.119:0/638872303 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00b6b0 con 0x7f7ee80ccee0 2023-10-08T00:56:19.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7ee559a700 1 -- 172.21.15.119:0/638872303 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7edc00b810 con 0x7f7ee80ccee0 2023-10-08T00:56:19.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7ee559a700 1 -- 172.21.15.119:0/638872303 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7edc00bbb0 con 0x7f7ee80ccee0 2023-10-08T00:56:19.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7eecdff700 1 -- 172.21.15.119:0/638872303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ccee0 msgr2=0x7f7ee81561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7eecdff700 1 --2- 172.21.15.119:0/638872303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ccee0 0x7f7ee81561d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80c9fd0 tx=0x7f7edc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7eecdff700 1 -- 172.21.15.119:0/638872303 shutdown_connections 2023-10-08T00:56:19.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7eecdff700 1 --2- 172.21.15.119:0/638872303 >> v2:172.21.15.184:3300/0 conn(0x7f7ee80c83f0 0x7f7ee80c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7eecdff700 1 --2- 172.21.15.119:0/638872303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ccee0 0x7f7ee81561d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7eecdff700 1 -- 172.21.15.119:0/638872303 >> 172.21.15.119:0/638872303 conn(0x7f7ee8068e10 msgr2=0x7f7ee80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:19.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7eecdff700 1 -- 172.21.15.119:0/638872303 shutdown_connections 2023-10-08T00:56:19.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.623+0000 7f7eecdff700 1 -- 172.21.15.119:0/638872303 wait complete. 2023-10-08T00:56:19.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.625+0000 7f7eecdff700 1 Processor -- start 2023-10-08T00:56:19.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.625+0000 7f7eecdff700 1 -- start start 2023-10-08T00:56:19.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7eecdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80c83f0 0x7f7ee80c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7eecdff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80c87c0 0x7f7ee8154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7eecdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ccee0 0x7f7ee8146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7eecdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee816a030 con 0x7f7ee80c83f0 2023-10-08T00:56:19.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7eecdff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee816db60 con 0x7f7ee80c87c0 2023-10-08T00:56:19.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7eecdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee816de40 con 0x7f7ee80ccee0 2023-10-08T00:56:19.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80c83f0 0x7f7ee80c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80c83f0 0x7f7ee80c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35076/0 (socket says 172.21.15.119:35076) 2023-10-08T00:56:19.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee659c700 1 -- 172.21.15.119:0/3423502492 learned_addr learned my addr 172.21.15.119:0/3423502492 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:19.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee5d9b700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80c87c0 0x7f7ee8154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee6d9d700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ccee0 0x7f7ee8146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee659c700 1 -- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ccee0 msgr2=0x7f7ee8146bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee659c700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ccee0 0x7f7ee8146bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee659c700 1 -- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80c87c0 msgr2=0x7f7ee8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee659c700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80c87c0 0x7f7ee8154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee659c700 1 -- 172.21.15.119:0/3423502492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc009420 con 0x7f7ee80c83f0 2023-10-08T00:56:19.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.626+0000 7f7ee6d9d700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ccee0 0x7f7ee8146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:19.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.627+0000 7f7ee659c700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80c83f0 0x7f7ee80c4fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800d860 tx=0x7f7ed8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:19.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.627+0000 7f7ed67fc700 1 -- 172.21.15.119:0/3423502492 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed80125f0 con 0x7f7ee80c83f0 2023-10-08T00:56:19.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.627+0000 7f7ed67fc700 1 -- 172.21.15.119:0/3423502492 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed8012750 con 0x7f7ee80c83f0 2023-10-08T00:56:19.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.627+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee816e120 con 0x7f7ee80c83f0 2023-10-08T00:56:19.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.627+0000 7f7ed67fc700 1 -- 172.21.15.119:0/3423502492 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8012b10 con 0x7f7ee80c83f0 2023-10-08T00:56:19.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.627+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee80c5520 con 0x7f7ee80c83f0 2023-10-08T00:56:19.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.628+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee816a380 con 0x7f7ee80c83f0 2023-10-08T00:56:19.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.629+0000 7f7ed67fc700 1 -- 172.21.15.119:0/3423502492 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ed801e460 con 0x7f7ee80c83f0 2023-10-08T00:56:19.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.630+0000 7f7ed67fc700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:19.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.630+0000 7f7ed67fc700 1 -- 172.21.15.119:0/3423502492 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ed8084ad0 con 0x7f7ee80c83f0 2023-10-08T00:56:19.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.630+0000 7f7ee5d9b700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:19.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.631+0000 7f7ee5d9b700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0093f0 tx=0x7f7edc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:19.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.634+0000 7f7ed67fc700 1 -- 172.21.15.119:0/3423502492 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ed8051d40 con 0x7f7ee80c83f0 2023-10-08T00:56:19.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.821+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7ee804c3c0 con 0x7f7ee80c83f0 2023-10-08T00:56:19.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.821+0000 7f7ed67fc700 1 -- 172.21.15.119:0/3423502492 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ed8055360 con 0x7f7ee80c83f0 2023-10-08T00:56:19.822 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T00:56:19.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec8066f30 msgr2=0x7f7ec80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0093f0 tx=0x7f7edc017000 comp rx=0 tx=0).stop 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80c83f0 msgr2=0x7f7ee80c4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80c83f0 0x7f7ee80c4fe0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ed800d860 tx=0x7f7ed8008040 comp rx=0 tx=0).stop 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 shutdown_connections 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ccee0 0x7f7ee8146bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80c83f0 0x7f7ee80c4fe0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 --2- 172.21.15.119:0/3423502492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80c87c0 0x7f7ee8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 >> 172.21.15.119:0/3423502492 conn(0x7f7ee8068e10 msgr2=0x7f7ee80b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:19.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 shutdown_connections 2023-10-08T00:56:19.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:19.824+0000 7f7eecdff700 1 -- 172.21.15.119:0/3423502492 wait complete. 2023-10-08T00:56:19.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:20.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.022+0000 7feccd39d700 1 Processor -- start 2023-10-08T00:56:20.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.023+0000 7feccd39d700 1 -- start start 2023-10-08T00:56:20.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7feccd39d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fecc8157270 0x7fecc8068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7feccd39d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fecc80b0bd0 0x7fecc815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7feccd39d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc815a980 con 0x7fecc80688d0 2023-10-08T00:56:20.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7feccd39d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc8068d50 con 0x7fecc80b0bd0 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7feccd39d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc815a050 con 0x7fecc8157270 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7fecccb9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fecc80b0bd0 0x7fecc815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7fecccb9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fecc80b0bd0 0x7fecc815d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35086/0 (socket says 172.21.15.119:35086) 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7fecc77fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fecc80688d0 0x7fecc80b04c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37556/0 (socket says 172.21.15.119:37556) 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7fecccb9c700 1 -- 172.21.15.119:0/4236050781 learned_addr learned my addr 172.21.15.119:0/4236050781 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.024+0000 7fecc7fff700 1 --2- 172.21.15.119:0/4236050781 >> v2:172.21.15.184:3300/0 conn(0x7fecc8157270 0x7fecc8068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecccb9c700 1 -- 172.21.15.119:0/4236050781 >> v2:172.21.15.184:3300/0 conn(0x7fecc8157270 msgr2=0x7fecc8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecccb9c700 1 --2- 172.21.15.119:0/4236050781 >> v2:172.21.15.184:3300/0 conn(0x7fecc8157270 0x7fecc8068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecccb9c700 1 -- 172.21.15.119:0/4236050781 >> v1:172.21.15.119:6789/0 conn(0x7fecc80688d0 legacy=0x7fecc80b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecccb9c700 1 -- 172.21.15.119:0/4236050781 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc815a330 con 0x7fecc80b0bd0 2023-10-08T00:56:20.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecc6ffd700 1 -- 172.21.15.119:0/4236050781 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722477368 0 0) 0x7fecc815a980 con 0x7fecc80688d0 2023-10-08T00:56:20.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecccb9c700 1 --2- 172.21.15.119:0/4236050781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80b0bd0 0x7fecc815d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fecc80b3360 tx=0x7fecc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7604c89e909bf3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecc6ffd700 1 -- 172.21.15.119:0/4236050781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc0009e70 con 0x7fecc80b0bd0 2023-10-08T00:56:20.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecc6ffd700 1 -- 172.21.15.119:0/4236050781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc000b860 con 0x7fecc80b0bd0 2023-10-08T00:56:20.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.025+0000 7fecc6ffd700 1 -- 172.21.15.119:0/4236050781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc000bc00 con 0x7fecc80b0bd0 2023-10-08T00:56:20.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.026+0000 7feccd39d700 1 -- 172.21.15.119:0/4236050781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80b0bd0 msgr2=0x7fecc815d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.026+0000 7feccd39d700 1 --2- 172.21.15.119:0/4236050781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80b0bd0 0x7fecc815d300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fecc80b3360 tx=0x7fecc0009250 comp rx=0 tx=0).stop 2023-10-08T00:56:20.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.026+0000 7feccd39d700 1 -- 172.21.15.119:0/4236050781 shutdown_connections 2023-10-08T00:56:20.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.026+0000 7feccd39d700 1 --2- 172.21.15.119:0/4236050781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80b0bd0 0x7fecc815d300 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.026+0000 7feccd39d700 1 --2- 172.21.15.119:0/4236050781 >> v2:172.21.15.184:3300/0 conn(0x7fecc8157270 0x7fecc8068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.026+0000 7feccd39d700 1 -- 172.21.15.119:0/4236050781 >> 172.21.15.119:0/4236050781 conn(0x7fecc80cd310 msgr2=0x7fecc80cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:20.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.026+0000 7feccd39d700 1 -- 172.21.15.119:0/4236050781 shutdown_connections 2023-10-08T00:56:20.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.026+0000 7feccd39d700 1 -- 172.21.15.119:0/4236050781 wait complete. 2023-10-08T00:56:20.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.027+0000 7feccd39d700 1 Processor -- start 2023-10-08T00:56:20.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.027+0000 7feccd39d700 1 -- start start 2023-10-08T00:56:20.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.028+0000 7feccd39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80688d0 0x7fecc81564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.028+0000 7feccd39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc80b0bd0 0x7fecc81542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.028+0000 7feccd39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc8157270 0x7fecc815dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.028+0000 7feccd39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815d7c0 con 0x7fecc80688d0 2023-10-08T00:56:20.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.028+0000 7feccd39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc816d8e0 con 0x7fecc80b0bd0 2023-10-08T00:56:20.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.028+0000 7feccd39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc815d170 con 0x7fecc8157270 2023-10-08T00:56:20.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80688d0 0x7fecc81564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80688d0 0x7fecc81564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35092/0 (socket says 172.21.15.119:35092) 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 -- 172.21.15.119:0/4081010482 learned_addr learned my addr 172.21.15.119:0/4081010482 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc77fe700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc80b0bd0 0x7fecc81542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecccb9c700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc8157270 0x7fecc815dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 -- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc8157270 msgr2=0x7fecc815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc8157270 0x7fecc815dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 -- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc80b0bd0 msgr2=0x7fecc81542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc80b0bd0 0x7fecc81542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 -- 172.21.15.119:0/4081010482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc00094f0 con 0x7fecc80688d0 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecccb9c700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc8157270 0x7fecc815dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecc7fff700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80688d0 0x7fecc81564c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009e50 tx=0x7fecb800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecaffff700 1 -- 172.21.15.119:0/4081010482 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb800cc20 con 0x7fecc80688d0 2023-10-08T00:56:20.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecaffff700 1 -- 172.21.15.119:0/4081010482 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb800f040 con 0x7fecc80688d0 2023-10-08T00:56:20.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7fecaffff700 1 -- 172.21.15.119:0/4081010482 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8014480 con 0x7fecc80688d0 2023-10-08T00:56:20.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.029+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc815d450 con 0x7fecc80688d0 2023-10-08T00:56:20.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.030+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc8156a00 con 0x7fecc80688d0 2023-10-08T00:56:20.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.030+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc804c3c0 con 0x7fecc80688d0 2023-10-08T00:56:20.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.031+0000 7fecaffff700 1 -- 172.21.15.119:0/4081010482 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecb80147f0 con 0x7fecc80688d0 2023-10-08T00:56:20.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.032+0000 7fecaffff700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.032+0000 7fecaffff700 1 -- 172.21.15.119:0/4081010482 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fecb8084ff0 con 0x7fecc80688d0 2023-10-08T00:56:20.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.032+0000 7fecc77fe700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.033+0000 7fecc77fe700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fecb4066f30 0x7fecb40693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fecc8158ff0 tx=0x7fecbc007bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.036+0000 7fecaffff700 1 -- 172.21.15.119:0/4081010482 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecb80521e0 con 0x7fecc80688d0 2023-10-08T00:56:20.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.225+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecc815a610 con 0x7fecc80688d0 2023-10-08T00:56:20.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.226+0000 7fecaffff700 1 -- 172.21.15.119:0/4081010482 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fecb8055800 con 0x7fecc80688d0 2023-10-08T00:56:20.227 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:20.227 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:20.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fecb4066f30 msgr2=0x7fecb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fecb4066f30 0x7fecb40693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fecc8158ff0 tx=0x7fecbc007bf0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80688d0 msgr2=0x7fecc81564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80688d0 0x7fecc81564c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009e50 tx=0x7fecb800b040 comp rx=0 tx=0).stop 2023-10-08T00:56:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 shutdown_connections 2023-10-08T00:56:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc8157270 0x7fecc815dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fecb4066f30 0x7fecb40693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc80688d0 0x7fecc81564c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 --2- 172.21.15.119:0/4081010482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc80b0bd0 0x7fecc81542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 >> 172.21.15.119:0/4081010482 conn(0x7fecc80cd310 msgr2=0x7fecc806c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 shutdown_connections 2023-10-08T00:56:20.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.229+0000 7feccd39d700 1 -- 172.21.15.119:0/4081010482 wait complete. 2023-10-08T00:56:20.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:55:45.270364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:55:45.937180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:56:20.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:56:20.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.429+0000 7fd95759e700 1 Processor -- start 2023-10-08T00:56:20.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.430+0000 7fd95759e700 1 -- start start 2023-10-08T00:56:20.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.431+0000 7fd95759e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd9580c9db0 0x7fd95806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.431+0000 7fd95759e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd95806e150 con 0x7fd958153900 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.431+0000 7fd95759e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9580c7800 con 0x7fd958157130 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.431+0000 7fd95759e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9580c7610 con 0x7fd9580c9db0 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.431+0000 7fd95659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd958153900 0x7fd9580c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37572/0 (socket says 172.21.15.119:37572) 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.431+0000 7fd95659c700 1 -- 172.21.15.119:0/4097974879 learned_addr learned my addr 172.21.15.119:0/4097974879 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.431+0000 7fd94ffff700 1 --2- 172.21.15.119:0/4097974879 >> v2:172.21.15.184:3300/0 conn(0x7fd9580c9db0 0x7fd95806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.431+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874409287 0 0) 0x7fd95806e150 con 0x7fd958153900 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd944004680 con 0x7fd958153900 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361372469 0 0) 0x7fd9580c7800 con 0x7fd958157130 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd95806e150 con 0x7fd958157130 2023-10-08T00:56:20.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1423780978 0 0) 0x7fd944004680 con 0x7fd958153900 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 >> v2:172.21.15.184:3300/0 conn(0x7fd9580c9db0 msgr2=0x7fd95806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 --2- 172.21.15.119:0/4097974879 >> v2:172.21.15.184:3300/0 conn(0x7fd9580c9db0 0x7fd95806a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 >> v1:172.21.15.184:6789/0 conn(0x7fd958157130 legacy=0x7fd95806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9580c7ae0 con 0x7fd958153900 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fd948001470 con 0x7fd958153900 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 >> v1:172.21.15.119:6789/0 conn(0x7fd958153900 legacy=0x7fd9580c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944004d50 0x7fd944005140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd944005920 0x7fd944005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944006320 0x7fd94400cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94400d290 con 0x7fd944006320 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94400d400 con 0x7fd944004d50 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94400d5c0 con 0x7fd944005920 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd956d9d700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944006320 0x7fd94400cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd95659c700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944004d50 0x7fd944005140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.432+0000 7fd94ffff700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd944005920 0x7fd944005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd956d9d700 1 -- 172.21.15.119:0/4097974879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd944005920 msgr2=0x7fd944005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd956d9d700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd944005920 0x7fd944005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd956d9d700 1 -- 172.21.15.119:0/4097974879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944004d50 msgr2=0x7fd944005140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:20.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd956d9d700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944004d50 0x7fd944005140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd956d9d700 1 -- 172.21.15.119:0/4097974879 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944004680 con 0x7fd944006320 2023-10-08T00:56:20.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd956d9d700 1 -- 172.21.15.119:0/4097974879 reap_dead start 2023-10-08T00:56:20.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd956d9d700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944006320 0x7fd94400cb70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd9500020f0 tx=0x7fd9500017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b0a5cdb1ea6eae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950001ab0 con 0x7fd944006320 2023-10-08T00:56:20.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd95000c8b0 con 0x7fd944006320 2023-10-08T00:56:20.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.433+0000 7fd955d9b700 1 -- 172.21.15.119:0/4097974879 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95000cc50 con 0x7fd944006320 2023-10-08T00:56:20.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.434+0000 7fd95759e700 1 -- 172.21.15.119:0/4097974879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944006320 msgr2=0x7fd94400cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.434+0000 7fd95759e700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944006320 0x7fd94400cb70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd9500020f0 tx=0x7fd9500017a0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.434+0000 7fd95759e700 1 -- 172.21.15.119:0/4097974879 shutdown_connections 2023-10-08T00:56:20.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.434+0000 7fd95759e700 1 --2- 172.21.15.119:0/4097974879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944006320 0x7fd94400cb70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.434+0000 7fd95759e700 1 -- 172.21.15.119:0/4097974879 >> 172.21.15.119:0/4097974879 conn(0x7fd9580cc1e0 msgr2=0x7fd9580cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:20.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.434+0000 7fd95759e700 1 -- 172.21.15.119:0/4097974879 shutdown_connections 2023-10-08T00:56:20.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.434+0000 7fd95759e700 1 -- 172.21.15.119:0/4097974879 wait complete. 2023-10-08T00:56:20.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.435+0000 7fd95759e700 1 Processor -- start 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.435+0000 7fd95759e700 1 -- start start 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd95759e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944006320 0x7fd958165960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd95759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9580ae720 0x7fd958159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd95759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95814ee60 0x7fd958150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd95759e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95814ff00 con 0x7fd95814ee60 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd95759e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd95814f2f0 con 0x7fd944006320 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd95759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd95814f5d0 con 0x7fd9580ae720 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd956d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95814ee60 0x7fd958150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd956d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95814ee60 0x7fd958150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35116/0 (socket says 172.21.15.119:35116) 2023-10-08T00:56:20.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.436+0000 7fd956d9d700 1 -- 172.21.15.119:0/3756173812 learned_addr learned my addr 172.21.15.119:0/3756173812 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd94ffff700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9580ae720 0x7fd958159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd95659c700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944006320 0x7fd958165960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd956d9d700 1 -- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9580ae720 msgr2=0x7fd958159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd956d9d700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9580ae720 0x7fd958159030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd956d9d700 1 -- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944006320 msgr2=0x7fd958165960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd956d9d700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944006320 0x7fd958165960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd956d9d700 1 -- 172.21.15.119:0/3756173812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95814f8b0 con 0x7fd95814ee60 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd95659c700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944006320 0x7fd958165960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd956d9d700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95814ee60 0x7fd958150390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd95806d2f0 tx=0x7fd950016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd94e7fc700 1 -- 172.21.15.119:0/3756173812 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd95000cc30 con 0x7fd95814ee60 2023-10-08T00:56:20.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd94e7fc700 1 -- 172.21.15.119:0/3756173812 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd950027470 con 0x7fd95814ee60 2023-10-08T00:56:20.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd94e7fc700 1 -- 172.21.15.119:0/3756173812 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd950027810 con 0x7fd95814ee60 2023-10-08T00:56:20.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.437+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd95814fb90 con 0x7fd95814ee60 2023-10-08T00:56:20.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.438+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9580ac720 con 0x7fd95814ee60 2023-10-08T00:56:20.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.439+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9581500b0 con 0x7fd95814ee60 2023-10-08T00:56:20.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.439+0000 7fd94e7fc700 1 -- 172.21.15.119:0/3756173812 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd950024070 con 0x7fd95814ee60 2023-10-08T00:56:20.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.440+0000 7fd94e7fc700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd938066f30 0x7fd9380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.440+0000 7fd94e7fc700 1 -- 172.21.15.119:0/3756173812 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd950086490 con 0x7fd95814ee60 2023-10-08T00:56:20.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.440+0000 7fd95659c700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd938066f30 0x7fd9380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.441+0000 7fd95659c700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd938066f30 0x7fd9380693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd948001bb0 tx=0x7fd948009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.444+0000 7fd94e7fc700 1 -- 172.21.15.119:0/3756173812 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9581500b0 con 0x7fd95814ee60 2023-10-08T00:56:20.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.634+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd95814fd20 con 0x7fd95814ee60 2023-10-08T00:56:20.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.635+0000 7fd94e7fc700 1 -- 172.21.15.119:0/3756173812 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd950053700 con 0x7fd95814ee60 2023-10-08T00:56:20.635 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:56:20.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd938066f30 msgr2=0x7fd9380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd938066f30 0x7fd9380693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd948001bb0 tx=0x7fd948009040 comp rx=0 tx=0).stop 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95814ee60 msgr2=0x7fd958150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95814ee60 0x7fd958150390 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd95806d2f0 tx=0x7fd950016000 comp rx=0 tx=0).stop 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 shutdown_connections 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd938066f30 0x7fd9380693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd95814ee60 0x7fd958150390 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944006320 0x7fd958165960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 --2- 172.21.15.119:0/3756173812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9580ae720 0x7fd958159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 >> 172.21.15.119:0/3756173812 conn(0x7fd9580cc1e0 msgr2=0x7fd9580c1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 shutdown_connections 2023-10-08T00:56:20.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.637+0000 7fd95759e700 1 -- 172.21.15.119:0/3756173812 wait complete. 2023-10-08T00:56:20.655 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:56:20.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.838+0000 7fe4aabff700 1 Processor -- start 2023-10-08T00:56:20.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.838+0000 7fe4aabff700 1 -- start start 2023-10-08T00:56:20.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.839+0000 7fe4aabff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4a40c70b0 0x7fe4a4069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.839+0000 7fe4aabff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe4a4069cb0 0x7fe4a40ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.839+0000 7fe4aabff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a406e950 con 0x7fe4a4155eb0 2023-10-08T00:56:20.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.839+0000 7fe4aabff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a406a130 con 0x7fe4a40c70b0 2023-10-08T00:56:20.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.839+0000 7fe4aabff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a406eb90 con 0x7fe4a4069cb0 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a899b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe4a4155eb0 0x7fe4a406e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37582/0 (socket says 172.21.15.119:37582) 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a899b700 1 -- 172.21.15.119:0/3407406392 learned_addr learned my addr 172.21.15.119:0/3407406392 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a3fff700 1 --2- 172.21.15.119:0/3407406392 >> v2:172.21.15.119:3300/0 conn(0x7fe4a40c70b0 0x7fe4a4069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a919c700 1 --2- 172.21.15.119:0/3407406392 >> v2:172.21.15.184:3300/0 conn(0x7fe4a4069cb0 0x7fe4a40ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a37fe700 1 -- 172.21.15.119:0/3407406392 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 349055585 0 0) 0x7fe498001110 con 0x7fe4a4155eb0 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a37fe700 1 -- 172.21.15.119:0/3407406392 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe494003570 con 0x7fe4a4155eb0 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a3fff700 1 -- 172.21.15.119:0/3407406392 >> v2:172.21.15.184:3300/0 conn(0x7fe4a4069cb0 msgr2=0x7fe4a40ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a3fff700 1 --2- 172.21.15.119:0/3407406392 >> v2:172.21.15.184:3300/0 conn(0x7fe4a4069cb0 0x7fe4a40ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a3fff700 1 -- 172.21.15.119:0/3407406392 >> v1:172.21.15.119:6789/0 conn(0x7fe4a4155eb0 legacy=0x7fe4a406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a3fff700 1 -- 172.21.15.119:0/3407406392 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a406ee70 con 0x7fe4a40c70b0 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.840+0000 7fe4a3fff700 1 --2- 172.21.15.119:0/3407406392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a40c70b0 0x7fe4a4069770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40cc450 tx=0x7fe49000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6bd8663ce96f52 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4a37fe700 1 -- 172.21.15.119:0/3407406392 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49000c970 con 0x7fe4a40c70b0 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4a37fe700 1 -- 172.21.15.119:0/3407406392 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49000cad0 con 0x7fe4a40c70b0 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4a37fe700 1 -- 172.21.15.119:0/3407406392 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49000ce70 con 0x7fe4a40c70b0 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4aabff700 1 -- 172.21.15.119:0/3407406392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a40c70b0 msgr2=0x7fe4a4069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3407406392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a40c70b0 0x7fe4a4069770 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40cc450 tx=0x7fe49000a330 comp rx=0 tx=0).stop 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4aabff700 1 -- 172.21.15.119:0/3407406392 shutdown_connections 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3407406392 >> v2:172.21.15.184:3300/0 conn(0x7fe4a4069cb0 0x7fe4a40ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3407406392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a40c70b0 0x7fe4a4069770 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4aabff700 1 -- 172.21.15.119:0/3407406392 >> 172.21.15.119:0/3407406392 conn(0x7fe4a4068e10 msgr2=0x7fe4a40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:20.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4aabff700 1 -- 172.21.15.119:0/3407406392 shutdown_connections 2023-10-08T00:56:20.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.841+0000 7fe4aabff700 1 -- 172.21.15.119:0/3407406392 wait complete. 2023-10-08T00:56:20.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.843+0000 7fe4aabff700 1 Processor -- start 2023-10-08T00:56:20.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.843+0000 7fe4aabff700 1 -- start start 2023-10-08T00:56:20.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4aabff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a4069cb0 0x7fe4a415dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4aabff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4a40c70b0 0x7fe4a4156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4aabff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4155eb0 0x7fe4a4143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4aabff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41567c0 con 0x7fe4a4069cb0 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4aabff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40c3b60 con 0x7fe4a4155eb0 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4aabff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40c3e40 con 0x7fe4a40c70b0 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4a899b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a4069cb0 0x7fe4a415dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4a899b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a4069cb0 0x7fe4a415dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35136/0 (socket says 172.21.15.119:35136) 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4a899b700 1 -- 172.21.15.119:0/3171911270 learned_addr learned my addr 172.21.15.119:0/3171911270 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4a899b700 1 -- 172.21.15.119:0/3171911270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4a40c70b0 msgr2=0x7fe4a4156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4a899b700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4a40c70b0 0x7fe4a4156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4a919c700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4155eb0 0x7fe4a4143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.844+0000 7fe4a899b700 1 -- 172.21.15.119:0/3171911270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4155eb0 msgr2=0x7fe4a4143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.845+0000 7fe4a899b700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4155eb0 0x7fe4a4143ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:20.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.845+0000 7fe4a899b700 1 -- 172.21.15.119:0/3171911270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49000a5d0 con 0x7fe4a4069cb0 2023-10-08T00:56:20.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.845+0000 7fe4a899b700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a4069cb0 0x7fe4a415dbb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe49800c590 tx=0x7fe498007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.845+0000 7fe4a0ff9700 1 -- 172.21.15.119:0/3171911270 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498005b30 con 0x7fe4a4069cb0 2023-10-08T00:56:20.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.845+0000 7fe4a0ff9700 1 -- 172.21.15.119:0/3171911270 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe498005cb0 con 0x7fe4a4069cb0 2023-10-08T00:56:20.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.845+0000 7fe4a0ff9700 1 -- 172.21.15.119:0/3171911270 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe498007320 con 0x7fe4a4069cb0 2023-10-08T00:56:20.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.845+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a40c4120 con 0x7fe4a4069cb0 2023-10-08T00:56:20.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.845+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a415d3b0 con 0x7fe4a4069cb0 2023-10-08T00:56:20.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.846+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a404c3c0 con 0x7fe4a4069cb0 2023-10-08T00:56:20.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.847+0000 7fe4a0ff9700 1 -- 172.21.15.119:0/3171911270 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe49801b3b0 con 0x7fe4a4069cb0 2023-10-08T00:56:20.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.848+0000 7fe4a0ff9700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe488066f30 0x7fe4880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:20.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.848+0000 7fe4a0ff9700 1 -- 172.21.15.119:0/3171911270 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe4980839d0 con 0x7fe4a4069cb0 2023-10-08T00:56:20.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.848+0000 7fe4a3fff700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe488066f30 0x7fe4880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:20.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.848+0000 7fe4a3fff700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe488066f30 0x7fe4880693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40c6550 tx=0x7fe49001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:20.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:20.852+0000 7fe4a0ff9700 1 -- 172.21.15.119:0/3171911270 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe498050ba0 con 0x7fe4a4069cb0 2023-10-08T00:56:21.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.044+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4a406e950 con 0x7fe4a4069cb0 2023-10-08T00:56:21.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.044+0000 7fe4a0ff9700 1 -- 172.21.15.119:0/3171911270 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4a406e950 con 0x7fe4a4069cb0 2023-10-08T00:56:21.045 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T00:56:21.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe488066f30 msgr2=0x7fe4880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe488066f30 0x7fe4880693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40c6550 tx=0x7fe49001a000 comp rx=0 tx=0).stop 2023-10-08T00:56:21.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a4069cb0 msgr2=0x7fe4a415dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a4069cb0 0x7fe4a415dbb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe49800c590 tx=0x7fe498007c70 comp rx=0 tx=0).stop 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 shutdown_connections 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe488066f30 0x7fe4880693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4a4155eb0 0x7fe4a4143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a4069cb0 0x7fe4a415dbb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 --2- 172.21.15.119:0/3171911270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4a40c70b0 0x7fe4a4156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 >> 172.21.15.119:0/3171911270 conn(0x7fe4a4068e10 msgr2=0x7fe4a40c23d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 shutdown_connections 2023-10-08T00:56:21.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.047+0000 7fe4aabff700 1 -- 172.21.15.119:0/3171911270 wait complete. 2023-10-08T00:56:21.064 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:56:21.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.248+0000 7fcbeb59e700 1 Processor -- start 2023-10-08T00:56:21.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.248+0000 7fcbeb59e700 1 -- start start 2023-10-08T00:56:21.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.249+0000 7fcbeb59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcbec153900 0x7fcbec0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbeb59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbec06a710 0x7fcbec06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbeb59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbec06e150 con 0x7fcbec0c9db0 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbeb59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbec0c7800 con 0x7fcbec06a710 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbeb59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbec0c7ae0 con 0x7fcbec153900 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbead9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbec06a710 0x7fcbec06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbead9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbec06a710 0x7fcbec06e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35148/0 (socket says 172.21.15.119:35148) 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbead9d700 1 -- 172.21.15.119:0/1418272360 learned_addr learned my addr 172.21.15.119:0/1418272360 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbea59c700 1 --2- 172.21.15.119:0/1418272360 >> v2:172.21.15.184:3300/0 conn(0x7fcbec153900 0x7fcbec0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbead9d700 1 -- 172.21.15.119:0/1418272360 >> v2:172.21.15.184:3300/0 conn(0x7fcbec153900 msgr2=0x7fcbec0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbead9d700 1 --2- 172.21.15.119:0/1418272360 >> v2:172.21.15.184:3300/0 conn(0x7fcbec153900 0x7fcbec0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbead9d700 1 -- 172.21.15.119:0/1418272360 >> v1:172.21.15.184:6789/0 conn(0x7fcbec0c9db0 legacy=0x7fcbec06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.250+0000 7fcbead9d700 1 -- 172.21.15.119:0/1418272360 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec0c7dc0 con 0x7fcbec06a710 2023-10-08T00:56:21.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbead9d700 1 --2- 172.21.15.119:0/1418272360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec06e5e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcbec06aed0 tx=0x7fcbe4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749d91ee573afa4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbe9d9b700 1 -- 172.21.15.119:0/1418272360 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400e070 con 0x7fcbec06a710 2023-10-08T00:56:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbe9d9b700 1 -- 172.21.15.119:0/1418272360 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe400ba10 con 0x7fcbec06a710 2023-10-08T00:56:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbe9d9b700 1 -- 172.21.15.119:0/1418272360 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40092e0 con 0x7fcbec06a710 2023-10-08T00:56:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbeb59e700 1 -- 172.21.15.119:0/1418272360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 msgr2=0x7fcbec06e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/1418272360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec06e5e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcbec06aed0 tx=0x7fcbe4009840 comp rx=0 tx=0).stop 2023-10-08T00:56:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbeb59e700 1 -- 172.21.15.119:0/1418272360 shutdown_connections 2023-10-08T00:56:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/1418272360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec06e5e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/1418272360 >> v2:172.21.15.184:3300/0 conn(0x7fcbec153900 0x7fcbec0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.251+0000 7fcbeb59e700 1 -- 172.21.15.119:0/1418272360 >> 172.21.15.119:0/1418272360 conn(0x7fcbec0cc1e0 msgr2=0x7fcbec0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.252+0000 7fcbeb59e700 1 -- 172.21.15.119:0/1418272360 shutdown_connections 2023-10-08T00:56:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.252+0000 7fcbeb59e700 1 -- 172.21.15.119:0/1418272360 wait complete. 2023-10-08T00:56:21.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.253+0000 7fcbeb59e700 1 Processor -- start 2023-10-08T00:56:21.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.253+0000 7fcbeb59e700 1 -- start start 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbeb59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec0ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbeb59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbec0c9db0 0x7fcbec144f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbeb59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbec153900 0x7fcbec153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbeb59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec145470 con 0x7fcbec06a710 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbeb59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec157560 con 0x7fcbec153900 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbeb59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec0aca90 con 0x7fcbec0c9db0 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbea59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec0ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbea59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec0ac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35160/0 (socket says 172.21.15.119:35160) 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.254+0000 7fcbea59c700 1 -- 172.21.15.119:0/4208285517 learned_addr learned my addr 172.21.15.119:0/4208285517 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:21.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbead9d700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbec153900 0x7fcbec153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbea59c700 1 -- 172.21.15.119:0/4208285517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbec0c9db0 msgr2=0x7fcbec144f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbea59c700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbec0c9db0 0x7fcbec144f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbea59c700 1 -- 172.21.15.119:0/4208285517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbec153900 msgr2=0x7fcbec153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbea59c700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbec153900 0x7fcbec153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbea59c700 1 -- 172.21.15.119:0/4208285517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe4009ae0 con 0x7fcbec06a710 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbead9d700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbec153900 0x7fcbec153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbea59c700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec0ac4b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00d860 tx=0x7fcbdc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbe27fc700 1 -- 172.21.15.119:0/4208285517 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0125f0 con 0x7fcbec06a710 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbe27fc700 1 -- 172.21.15.119:0/4208285517 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc012750 con 0x7fcbec06a710 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec0ae1d0 con 0x7fcbec06a710 2023-10-08T00:56:21.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbe27fc700 1 -- 172.21.15.119:0/4208285517 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc012b10 con 0x7fcbec06a710 2023-10-08T00:56:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.255+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec0ae790 con 0x7fcbec06a710 2023-10-08T00:56:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.256+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbec1576f0 con 0x7fcbec06a710 2023-10-08T00:56:21.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.257+0000 7fcbe27fc700 1 -- 172.21.15.119:0/4208285517 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcbdc01e460 con 0x7fcbec06a710 2023-10-08T00:56:21.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.258+0000 7fcbe27fc700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcbcc066f30 0x7fcbcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.258+0000 7fcbe27fc700 1 -- 172.21.15.119:0/4208285517 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcbdc084bf0 con 0x7fcbec06a710 2023-10-08T00:56:21.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.258+0000 7fcbe3fff700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcbcc066f30 0x7fcbcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.259+0000 7fcbe3fff700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcbcc066f30 0x7fcbcc0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4007fd0 tx=0x7fcbd4007b20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:21.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.262+0000 7fcbe27fc700 1 -- 172.21.15.119:0/4208285517 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbdc051dc0 con 0x7fcbec06a710 2023-10-08T00:56:21.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.454+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcbec06dd20 con 0x7fcbec06a710 2023-10-08T00:56:21.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.455+0000 7fcbe27fc700 1 -- 172.21.15.119:0/4208285517 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcbdc0553e0 con 0x7fcbec06a710 2023-10-08T00:56:21.455 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T00:56:21.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcbcc066f30 msgr2=0x7fcbcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcbcc066f30 0x7fcbcc0693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4007fd0 tx=0x7fcbd4007b20 comp rx=0 tx=0).stop 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 msgr2=0x7fcbec0ac4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec0ac4b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00d860 tx=0x7fcbdc008040 comp rx=0 tx=0).stop 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 shutdown_connections 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcbcc066f30 0x7fcbcc0693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbec153900 0x7fcbec153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbec06a710 0x7fcbec0ac4b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 --2- 172.21.15.119:0/4208285517 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbec0c9db0 0x7fcbec144f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 >> 172.21.15.119:0/4208285517 conn(0x7fcbec0cc1e0 msgr2=0x7fcbec06e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:21.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 shutdown_connections 2023-10-08T00:56:21.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.457+0000 7fcbeb59e700 1 -- 172.21.15.119:0/4208285517 wait complete. 2023-10-08T00:56:21.483 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-10-08T00:56:21.483 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T00:56:21.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-08T00:56:21.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.664+0000 7fee2f59d700 1 Processor -- start 2023-10-08T00:56:21.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.664+0000 7fee2f59d700 1 -- start start 2023-10-08T00:56:21.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.665+0000 7fee2f59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee280b1e50 0x7fee28159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.665+0000 7fee2f59d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee280cfe20 con 0x7fee281593b0 2023-10-08T00:56:21.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.665+0000 7fee2f59d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee280cf210 con 0x7fee280ceaa0 2023-10-08T00:56:21.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.665+0000 7fee2f59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee280cf4f0 con 0x7fee280b1e50 2023-10-08T00:56:21.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.665+0000 7fee2e59b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fee281593b0 0x7fee280b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37594/0 (socket says 172.21.15.119:37594) 2023-10-08T00:56:21.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2e59b700 1 -- 172.21.15.119:0/2101565726 learned_addr learned my addr 172.21.15.119:0/2101565726 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:21.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2dd9a700 1 --2- 172.21.15.119:0/2101565726 >> v2:172.21.15.119:3300/0 conn(0x7fee280b1e50 0x7fee28159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2dd9a700 1 -- 172.21.15.119:0/2101565726 >> v1:172.21.15.184:6789/0 conn(0x7fee280ceaa0 legacy=0x7fee280b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2dd9a700 1 -- 172.21.15.119:0/2101565726 >> v1:172.21.15.119:6789/0 conn(0x7fee281593b0 legacy=0x7fee280b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2dd9a700 1 -- 172.21.15.119:0/2101565726 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee280cf7d0 con 0x7fee280b1e50 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2d599700 1 -- 172.21.15.119:0/2101565726 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385530010 0 0) 0x7fee280cfe20 con 0x7fee281593b0 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2dd9a700 1 --2- 172.21.15.119:0/2101565726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28159780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee180098e0 tx=0x7fee180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83833c312bbecb9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2d599700 1 -- 172.21.15.119:0/2101565726 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18009e80 con 0x7fee280b1e50 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.666+0000 7fee2d599700 1 -- 172.21.15.119:0/2101565726 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1800b820 con 0x7fee280b1e50 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.667+0000 7fee2d599700 1 -- 172.21.15.119:0/2101565726 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1800bba0 con 0x7fee280b1e50 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.667+0000 7fee2f59d700 1 -- 172.21.15.119:0/2101565726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 msgr2=0x7fee28159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.667+0000 7fee2f59d700 1 --2- 172.21.15.119:0/2101565726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28159780 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fee180098e0 tx=0x7fee180091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.667+0000 7fee2f59d700 1 -- 172.21.15.119:0/2101565726 shutdown_connections 2023-10-08T00:56:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.667+0000 7fee2f59d700 1 --2- 172.21.15.119:0/2101565726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28159780 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.667+0000 7fee2f59d700 1 -- 172.21.15.119:0/2101565726 >> 172.21.15.119:0/2101565726 conn(0x7fee280d1130 msgr2=0x7fee280b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.667+0000 7fee2f59d700 1 -- 172.21.15.119:0/2101565726 shutdown_connections 2023-10-08T00:56:21.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.667+0000 7fee2f59d700 1 -- 172.21.15.119:0/2101565726 wait complete. 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.668+0000 7fee2f59d700 1 Processor -- start 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.669+0000 7fee2f59d700 1 -- start start 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.669+0000 7fee2f59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.669+0000 7fee2f59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee280ceaa0 0x7fee2814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.669+0000 7fee2f59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee281593b0 0x7fee2815f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.669+0000 7fee2f59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2815f330 con 0x7fee280b1e50 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.669+0000 7fee2f59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2815e720 con 0x7fee280ceaa0 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.669+0000 7fee2f59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2815ea00 con 0x7fee281593b0 2023-10-08T00:56:21.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35174/0 (socket says 172.21.15.119:35174) 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 -- 172.21.15.119:0/2093017000 learned_addr learned my addr 172.21.15.119:0/2093017000 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2dd9a700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee280ceaa0 0x7fee2814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2ed9c700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee281593b0 0x7fee2815f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 -- 172.21.15.119:0/2093017000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee281593b0 msgr2=0x7fee2815f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee281593b0 0x7fee2815f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 -- 172.21.15.119:0/2093017000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee280ceaa0 msgr2=0x7fee2814e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee280ceaa0 0x7fee2814e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 -- 172.21.15.119:0/2093017000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18009420 con 0x7fee280b1e50 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.670+0000 7fee2e59b700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28154b60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c5b0 tx=0x7fee24001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:21.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.671+0000 7fee1e7fc700 1 -- 172.21.15.119:0/2093017000 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24005b30 con 0x7fee280b1e50 2023-10-08T00:56:21.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.671+0000 7fee1e7fc700 1 -- 172.21.15.119:0/2093017000 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee24005c90 con 0x7fee280b1e50 2023-10-08T00:56:21.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.671+0000 7fee1e7fc700 1 -- 172.21.15.119:0/2093017000 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee24007300 con 0x7fee280b1e50 2023-10-08T00:56:21.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.671+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee2815ece0 con 0x7fee280b1e50 2023-10-08T00:56:21.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.671+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee281550a0 con 0x7fee280b1e50 2023-10-08T00:56:21.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.672+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2815f610 con 0x7fee280b1e50 2023-10-08T00:56:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.673+0000 7fee1e7fc700 1 -- 172.21.15.119:0/2093017000 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee2401b3b0 con 0x7fee280b1e50 2023-10-08T00:56:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.673+0000 7fee1e7fc700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee0c066f30 0x7fee0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.674+0000 7fee1e7fc700 1 -- 172.21.15.119:0/2093017000 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fee24083b00 con 0x7fee280b1e50 2023-10-08T00:56:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.674+0000 7fee2dd9a700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee0c066f30 0x7fee0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:21.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.674+0000 7fee2dd9a700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee0c066f30 0x7fee0c0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fee180030c0 tx=0x7fee18016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:21.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.677+0000 7fee1e7fc700 1 -- 172.21.15.119:0/2093017000 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee24085e60 con 0x7fee280b1e50 2023-10-08T00:56:21.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:21.941+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fee2804c3c0 con 0x7fee280b1e50 2023-10-08T00:56:22.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.029+0000 7fee1e7fc700 1 -- 172.21.15.119:0/2093017000 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7fee240170e0 con 0x7fee280b1e50 2023-10-08T00:56:22.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee0c066f30 msgr2=0x7fee0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:22.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee0c066f30 0x7fee0c0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fee180030c0 tx=0x7fee18016000 comp rx=0 tx=0).stop 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 msgr2=0x7fee28154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28154b60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c5b0 tx=0x7fee24001070 comp rx=0 tx=0).stop 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 shutdown_connections 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee281593b0 0x7fee2815f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee0c066f30 0x7fee0c0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee280b1e50 0x7fee28154b60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 --2- 172.21.15.119:0/2093017000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee280ceaa0 0x7fee2814e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 >> 172.21.15.119:0/2093017000 conn(0x7fee280d1130 msgr2=0x7fee280b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 shutdown_connections 2023-10-08T00:56:22.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.032+0000 7fee2f59d700 1 -- 172.21.15.119:0/2093017000 wait complete. 2023-10-08T00:56:22.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T00:56:22.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.233+0000 7f3c40dff700 1 Processor -- start 2023-10-08T00:56:22.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.233+0000 7f3c40dff700 1 -- start start 2023-10-08T00:56:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.234+0000 7f3c40dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3c3c159890 0x7f3c3c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.234+0000 7f3c40dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c069560 0x7f3c3c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.234+0000 7f3c40dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c3c069ee0 con 0x7f3c3c159c60 2023-10-08T00:56:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.234+0000 7f3c40dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c3c15a160 con 0x7f3c3c159890 2023-10-08T00:56:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.234+0000 7f3c40dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0d13a0 con 0x7f3c3c069560 2023-10-08T00:56:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.234+0000 7f3c39d9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3c3c159c60 0x7f3c3c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37596/0 (socket says 172.21.15.119:37596) 2023-10-08T00:56:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.234+0000 7f3c39d9b700 1 -- 172.21.15.119:0/2750604454 learned_addr learned my addr 172.21.15.119:0/2750604454 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3ad9d700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c069560 0x7f3c3c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3a59c700 1 --2- 172.21.15.119:0/2750604454 >> v2:172.21.15.184:3300/0 conn(0x7f3c3c159890 0x7f3c3c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4242437838 0 0) 0x7f3c3c069ee0 con 0x7f3c3c159c60 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c24003570 con 0x7f3c3c159c60 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1539798570 0 0) 0x7f3c24003570 con 0x7f3c3c159c60 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c069560 msgr2=0x7f3c3c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c069560 0x7f3c3c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 >> v2:172.21.15.184:3300/0 conn(0x7f3c3c159890 msgr2=0x7f3c3c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 --2- 172.21.15.119:0/2750604454 >> v2:172.21.15.184:3300/0 conn(0x7f3c3c159890 0x7f3c3c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c15e140 con 0x7f3c3c159c60 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f3c30001470 con 0x7f3c3c159c60 2023-10-08T00:56:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 >> v1:172.21.15.119:6789/0 conn(0x7f3c3c159c60 legacy=0x7f3c3c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c069560 0x7f3c24003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c24004d50 0x7f3c240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c15a160 con 0x7f3c240040b0 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c0d13a0 con 0x7f3c24004d50 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.235+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2400bb60 con 0x7f3c3c069560 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c39d9b700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c3a59c700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c069560 0x7f3c24003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c3ad9d700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c24004d50 0x7f3c240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c39d9b700 1 -- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c069560 msgr2=0x7f3c24003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c39d9b700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c069560 0x7f3c24003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c39d9b700 1 -- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c24004d50 msgr2=0x7f3c240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c39d9b700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c24004d50 0x7f3c240051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c39d9b700 1 -- 172.21.15.119:0/2750604454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24003570 con 0x7f3c240040b0 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c39d9b700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c24004810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000c2a0 tx=0x7f3c30001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faff512b348833 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000cb70 con 0x7f3c240040b0 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3000ccd0 con 0x7f3c240040b0 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.236+0000 7f3c3959a700 1 -- 172.21.15.119:0/2750604454 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30005b70 con 0x7f3c240040b0 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.237+0000 7f3c40dff700 1 -- 172.21.15.119:0/2750604454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 msgr2=0x7f3c24004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.237+0000 7f3c40dff700 1 --2- 172.21.15.119:0/2750604454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c24004810 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000c2a0 tx=0x7f3c30001dc0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.237+0000 7f3c3ad9d700 1 -- 172.21.15.119:0/2750604454 reap_dead start 2023-10-08T00:56:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.237+0000 7f3c40dff700 1 -- 172.21.15.119:0/2750604454 shutdown_connections 2023-10-08T00:56:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.237+0000 7f3c40dff700 1 -- 172.21.15.119:0/2750604454 >> 172.21.15.119:0/2750604454 conn(0x7f3c3c068e20 msgr2=0x7f3c3c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.237+0000 7f3c40dff700 1 -- 172.21.15.119:0/2750604454 shutdown_connections 2023-10-08T00:56:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.237+0000 7f3c40dff700 1 -- 172.21.15.119:0/2750604454 wait complete. 2023-10-08T00:56:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.238+0000 7f3c40dff700 1 Processor -- start 2023-10-08T00:56:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.239+0000 7f3c40dff700 1 -- start start 2023-10-08T00:56:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.239+0000 7f3c40dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c3c14c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.239+0000 7f3c40dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c3c0b1a40 0x7f3c3c149d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c40dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c0b1eb0 0x7f3c3c14a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c40dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c14a7c0 con 0x7f3c240040b0 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c40dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c14a960 con 0x7f3c3c0b1a40 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c40dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c169fb0 con 0x7f3c3c0b1eb0 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c3c14c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c3c14c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35194/0 (socket says 172.21.15.119:35194) 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 -- 172.21.15.119:0/3020492020 learned_addr learned my addr 172.21.15.119:0/3020492020 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 -- 172.21.15.119:0/3020492020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c0b1eb0 msgr2=0x7f3c3c14a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c0b1eb0 0x7f3c3c14a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 -- 172.21.15.119:0/3020492020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c3c0b1a40 msgr2=0x7f3c3c149d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c3c0b1a40 0x7f3c3c149d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 -- 172.21.15.119:0/3020492020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c16a230 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.240+0000 7f3c3a59c700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c3c14c940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c011d40 tx=0x7f3c2c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.241+0000 7f3c2a7fc700 1 -- 172.21.15.119:0/3020492020 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00fac0 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.241+0000 7f3c2a7fc700 1 -- 172.21.15.119:0/3020492020 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2c00fc20 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.241+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c16a510 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.241+0000 7f3c2a7fc700 1 -- 172.21.15.119:0/3020492020 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c014440 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.241+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c16aad0 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.242+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c04c3c0 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.243+0000 7f3c2a7fc700 1 -- 172.21.15.119:0/3020492020 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3c2c014a80 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.244+0000 7f3c2a7fc700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c1c066f30 0x7f3c1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.244+0000 7f3c2a7fc700 1 -- 172.21.15.119:0/3020492020 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f3c2c018030 con 0x7f3c240040b0 2023-10-08T00:56:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.244+0000 7f3c39d9b700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c1c066f30 0x7f3c1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:22.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.244+0000 7f3c39d9b700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c1c066f30 0x7f3c1c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0b2c40 tx=0x7f3c30012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:22.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.247+0000 7f3c2a7fc700 1 -- 172.21.15.119:0/3020492020 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3c2c052520 con 0x7f3c240040b0 2023-10-08T00:56:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.438+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f3c3c0caba0 con 0x7f3c240040b0 2023-10-08T00:56:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.443+0000 7f3c2a7fc700 1 -- 172.21.15.119:0/3020492020 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3c2c05b030 con 0x7f3c240040b0 2023-10-08T00:56:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c1c066f30 msgr2=0x7f3c1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c1c066f30 0x7f3c1c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0b2c40 tx=0x7f3c30012000 comp rx=0 tx=0).stop 2023-10-08T00:56:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 msgr2=0x7f3c3c14c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c3c14c940 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c011d40 tx=0x7f3c2c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 shutdown_connections 2023-10-08T00:56:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3c3c0b1eb0 0x7f3c3c14a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3c1c066f30 0x7f3c1c0693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c240040b0 0x7f3c3c14c940 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 --2- 172.21.15.119:0/3020492020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3c3c0b1a40 0x7f3c3c149d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:22.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.445+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 >> 172.21.15.119:0/3020492020 conn(0x7f3c3c068e20 msgr2=0x7f3c3c0c6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:22.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.446+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 shutdown_connections 2023-10-08T00:56:22.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:22.446+0000 7f3c40dff700 1 -- 172.21.15.119:0/3020492020 wait complete. 2023-10-08T00:56:22.463 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T00:56:22.463 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:56:22.541 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T00:56:22.632 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T00:56:22.632 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T00:56:22.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:22.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-10-08T00:56:22.718 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T00:56:22.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:22.719 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T00:56:22.763 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-10-08T00:56:22.763 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:22.763 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-10-08T00:56:22.865 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T00:56:22.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:22.866 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-08T00:56:22.913 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-10-08T00:56:22.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:22.913 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-10-08T00:56:22.995 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T00:56:22.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:22.995 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-08T00:56:23.057 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-10-08T00:56:23.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:23.057 DEBUG:teuthology.orchestra.run.smithi184:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-10-08T00:56:23.140 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T00:56:23.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:23.141 DEBUG:teuthology.orchestra.run.smithi184:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-08T00:56:23.182 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T00:56:23.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:23.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.367+0000 7f7fbaf9d700 1 Processor -- start 2023-10-08T00:56:23.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.368+0000 7f7fbaf9d700 1 -- start start 2023-10-08T00:56:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fbaf9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7fb4150180 0x7f7fb4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fbaf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb4150550 0x7f7fb40c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fbaf9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb406e2a0 con 0x7f7fb40c7240 2023-10-08T00:56:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fbaf9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb40c76c0 con 0x7f7fb4150180 2023-10-08T00:56:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fbaf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb406b2e0 con 0x7f7fb4150550 2023-10-08T00:56:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fb9f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7fb4150180 0x7f7fb4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fb9f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7fb4150180 0x7f7fb4157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35210/0 (socket says 172.21.15.119:35210) 2023-10-08T00:56:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fb9f9b700 1 -- 172.21.15.119:0/3043463718 learned_addr learned my addr 172.21.15.119:0/3043463718 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.369+0000 7f7fb979a700 1 --2- 172.21.15.119:0/3043463718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb4150550 0x7f7fb40c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb9f9b700 1 -- 172.21.15.119:0/3043463718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb4150550 msgr2=0x7f7fb40c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb9f9b700 1 --2- 172.21.15.119:0/3043463718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb4150550 0x7f7fb40c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb9f9b700 1 -- 172.21.15.119:0/3043463718 >> v1:172.21.15.184:6789/0 conn(0x7f7fb40c7240 legacy=0x7f7fb4069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb9f9b700 1 -- 172.21.15.119:0/3043463718 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40c78e0 con 0x7f7fb4150180 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb8f99700 1 -- 172.21.15.119:0/3043463718 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144927523 0 0) 0x7f7fb406e2a0 con 0x7f7fb40c7240 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb9f9b700 1 --2- 172.21.15.119:0/3043463718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150180 0x7f7fb4157770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000ab80 tx=0x7f7fb000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd046acca5f13975 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb8f99700 1 -- 172.21.15.119:0/3043463718 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb000ca00 con 0x7f7fb4150180 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb8f99700 1 -- 172.21.15.119:0/3043463718 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb000cb60 con 0x7f7fb4150180 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.370+0000 7f7fb8f99700 1 -- 172.21.15.119:0/3043463718 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb0018400 con 0x7f7fb4150180 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.371+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/3043463718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150180 msgr2=0x7f7fb4157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.371+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/3043463718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150180 0x7f7fb4157770 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000ab80 tx=0x7f7fb000a3f0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.371+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/3043463718 shutdown_connections 2023-10-08T00:56:23.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.371+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/3043463718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150180 0x7f7fb4157770 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.371+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/3043463718 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb4150550 0x7f7fb40c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.371+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/3043463718 >> 172.21.15.119:0/3043463718 conn(0x7f7fb40cc1f0 msgr2=0x7f7fb40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:23.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.371+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/3043463718 shutdown_connections 2023-10-08T00:56:23.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.371+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/3043463718 wait complete. 2023-10-08T00:56:23.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.372+0000 7f7fbaf9d700 1 Processor -- start 2023-10-08T00:56:23.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.372+0000 7f7fbaf9d700 1 -- start start 2023-10-08T00:56:23.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.373+0000 7f7fbaf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40c7240 0x7f7fb40ae400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.373+0000 7f7fbaf9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb4150180 0x7f7fb40c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.373+0000 7f7fbaf9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150550 0x7f7fb40c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.373+0000 7f7fbaf9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40ad440 con 0x7f7fb4150550 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.373+0000 7f7fbaf9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40ae940 con 0x7f7fb4150180 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.373+0000 7f7fbaf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40ac830 con 0x7f7fb40c7240 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150550 0x7f7fb40c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150550 0x7f7fb40c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35222/0 (socket says 172.21.15.119:35222) 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 -- 172.21.15.119:0/393936913 learned_addr learned my addr 172.21.15.119:0/393936913 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fb979a700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb4150180 0x7f7fb40c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fb9f9b700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40c7240 0x7f7fb40ae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 -- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40c7240 msgr2=0x7f7fb40ae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40c7240 0x7f7fb40ae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 -- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb4150180 msgr2=0x7f7fb40c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb4150180 0x7f7fb40c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 -- 172.21.15.119:0/393936913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb000a690 con 0x7f7fb4150550 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fb9f9b700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40c7240 0x7f7fb40ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fb979a700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb4150180 0x7f7fb40c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fba79c700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150550 0x7f7fb40c4640 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001560 tx=0x7f7fac001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.374+0000 7f7fa9ffb700 1 -- 172.21.15.119:0/393936913 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00a780 con 0x7f7fb4150550 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.375+0000 7f7fa9ffb700 1 -- 172.21.15.119:0/393936913 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac00a8e0 con 0x7f7fb4150550 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.375+0000 7f7fa9ffb700 1 -- 172.21.15.119:0/393936913 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac00aca0 con 0x7f7fb4150550 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.375+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb40acb10 con 0x7f7fb4150550 2023-10-08T00:56:23.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.375+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb40ad0d0 con 0x7f7fb4150550 2023-10-08T00:56:23.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.376+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb40aef90 con 0x7f7fb4150550 2023-10-08T00:56:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.376+0000 7f7fa9ffb700 1 -- 172.21.15.119:0/393936913 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fac012630 con 0x7f7fb4150550 2023-10-08T00:56:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.377+0000 7f7fa9ffb700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7f98066f30 0x7f7f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.377+0000 7f7fa9ffb700 1 -- 172.21.15.119:0/393936913 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7fac084850 con 0x7f7fb4150550 2023-10-08T00:56:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.377+0000 7f7fb9f9b700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7f98066f30 0x7f7f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.378+0000 7f7fb9f9b700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7f98066f30 0x7f7f980693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000ab80 tx=0x7f7fb001a000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:23.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.381+0000 7f7fa9ffb700 1 -- 172.21.15.119:0/393936913 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fac051ac0 con 0x7f7fb4150550 2023-10-08T00:56:23.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.614+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fb404c3c0 con 0x7f7fb4150550 2023-10-08T00:56:23.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.614+0000 7f7fa9ffb700 1 -- 172.21.15.119:0/393936913 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f7fac0550e0 con 0x7f7fb4150550 2023-10-08T00:56:23.615 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:23.615 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:55:46.965967+0000","modified":"2023-10-08T00:55:50.512168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:56:23.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7f98066f30 msgr2=0x7f7f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7f98066f30 0x7f7f980693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7fb000ab80 tx=0x7f7fb001a000 comp rx=0 tx=0).stop 2023-10-08T00:56:23.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150550 msgr2=0x7f7fb40c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150550 0x7f7fb40c4640 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7fac001560 tx=0x7f7fac001770 comp rx=0 tx=0).stop 2023-10-08T00:56:23.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 shutdown_connections 2023-10-08T00:56:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fb40c7240 0x7f7fb40ae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7f98066f30 0x7f7f980693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fb4150550 0x7f7fb40c4640 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 --2- 172.21.15.119:0/393936913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fb4150180 0x7f7fb40c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 >> 172.21.15.119:0/393936913 conn(0x7f7fb40cc1f0 msgr2=0x7f7fb40c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 shutdown_connections 2023-10-08T00:56:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.617+0000 7f7fbaf9d700 1 -- 172.21.15.119:0/393936913 wait complete. 2023-10-08T00:56:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-10-08T00:56:23.635 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.817+0000 7f252b7ff700 1 Processor -- start 2023-10-08T00:56:23.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.817+0000 7f252b7ff700 1 -- start start 2023-10-08T00:56:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.818+0000 7f252b7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2524159c60 0x7f252415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.818+0000 7f252b7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f252415db10 0x7f2524069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.818+0000 7f252b7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252415a160 con 0x7f2524159890 2023-10-08T00:56:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.818+0000 7f252b7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252415df50 con 0x7f2524159c60 2023-10-08T00:56:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.818+0000 7f252b7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240d13a0 con 0x7f252415db10 2023-10-08T00:56:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2528d9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2524159c60 0x7f252415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2529d9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f252415db10 0x7f2524069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2528d9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2524159c60 0x7f252415d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:55724/0 (socket says 172.21.15.119:55724) 2023-10-08T00:56:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2528d9a700 1 -- 172.21.15.119:0/2215670425 learned_addr learned my addr 172.21.15.119:0/2215670425 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2529d9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f252415db10 0x7f2524069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:41662/0 (socket says 172.21.15.119:41662) 2023-10-08T00:56:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f252959b700 1 --1- 172.21.15.119:0/2215670425 >> v1:172.21.15.184:6789/0 conn(0x7f2524159890 0x7f252415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:50166/0 (socket says 172.21.15.119:50166) 2023-10-08T00:56:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f251bfff700 1 -- 172.21.15.119:0/2215670425 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782312554 0 0) 0x7f252415a160 con 0x7f2524159890 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f251bfff700 1 -- 172.21.15.119:0/2215670425 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2510003570 con 0x7f2524159890 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2529d9c700 1 -- 172.21.15.119:0/2215670425 >> v2:172.21.15.184:3300/0 conn(0x7f2524159c60 msgr2=0x7f252415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2529d9c700 1 --2- 172.21.15.119:0/2215670425 >> v2:172.21.15.184:3300/0 conn(0x7f2524159c60 0x7f252415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2529d9c700 1 -- 172.21.15.119:0/2215670425 >> v1:172.21.15.184:6789/0 conn(0x7f2524159890 legacy=0x7f252415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2529d9c700 1 -- 172.21.15.119:0/2215670425 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252415e140 con 0x7f252415db10 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.819+0000 7f2528d9a700 1 --2- 172.21.15.119:0/2215670425 >> v2:172.21.15.184:3300/0 conn(0x7f2524159c60 0x7f252415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f2529d9c700 1 --2- 172.21.15.119:0/2215670425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f252415db10 0x7f2524069560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f251c009e60 tx=0x7f251c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c34f30bdfad8a9c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f251bfff700 1 -- 172.21.15.119:0/2215670425 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c00e070 con 0x7f252415db10 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f251bfff700 1 -- 172.21.15.119:0/2215670425 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c00bab0 con 0x7f252415db10 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f251bfff700 1 -- 172.21.15.119:0/2215670425 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c00be30 con 0x7f252415db10 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f252b7ff700 1 -- 172.21.15.119:0/2215670425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f252415db10 msgr2=0x7f2524069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2215670425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f252415db10 0x7f2524069560 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f251c009e60 tx=0x7f251c009780 comp rx=0 tx=0).stop 2023-10-08T00:56:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f252b7ff700 1 -- 172.21.15.119:0/2215670425 shutdown_connections 2023-10-08T00:56:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2215670425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f252415db10 0x7f2524069560 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2215670425 >> v2:172.21.15.184:3300/0 conn(0x7f2524159c60 0x7f252415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f252b7ff700 1 -- 172.21.15.119:0/2215670425 >> 172.21.15.119:0/2215670425 conn(0x7f2524068e20 msgr2=0x7f25240d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f252b7ff700 1 -- 172.21.15.119:0/2215670425 shutdown_connections 2023-10-08T00:56:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.820+0000 7f252b7ff700 1 -- 172.21.15.119:0/2215670425 wait complete. 2023-10-08T00:56:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.822+0000 7f252b7ff700 1 Processor -- start 2023-10-08T00:56:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.822+0000 7f252b7ff700 1 -- start start 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252b7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524159890 0x7f2524156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252b7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2524159c60 0x7f25240ce520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252b7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f252415db10 0x7f25240cea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252b7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25241581b0 con 0x7f2524159890 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252b7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25241575a0 con 0x7f252415db10 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252b7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2524157880 con 0x7f2524159c60 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524159890 0x7f2524156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524159890 0x7f2524156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35236/0 (socket says 172.21.15.119:35236) 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252959b700 1 -- 172.21.15.119:0/2347876603 learned_addr learned my addr 172.21.15.119:0/2347876603 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f2528d9a700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2524159c60 0x7f25240ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f2529d9c700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f252415db10 0x7f25240cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252959b700 1 -- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2524159c60 msgr2=0x7f25240ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252959b700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2524159c60 0x7f25240ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252959b700 1 -- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f252415db10 msgr2=0x7f25240cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.823+0000 7f252959b700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f252415db10 0x7f25240cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.824+0000 7f252959b700 1 -- 172.21.15.119:0/2347876603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140094f0 con 0x7f2524159890 2023-10-08T00:56:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.824+0000 7f2528d9a700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2524159c60 0x7f25240ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:23.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.824+0000 7f252959b700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524159890 0x7f2524156b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f252000c7c0 tx=0x7f2520007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.824+0000 7f25197fa700 1 -- 172.21.15.119:0/2347876603 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520005b40 con 0x7f2524159890 2023-10-08T00:56:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.824+0000 7f25197fa700 1 -- 172.21.15.119:0/2347876603 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2520005ca0 con 0x7f2524159890 2023-10-08T00:56:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.824+0000 7f25197fa700 1 -- 172.21.15.119:0/2347876603 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2520007320 con 0x7f2524159890 2023-10-08T00:56:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.824+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251c009a20 con 0x7f2524159890 2023-10-08T00:56:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.824+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2524157e40 con 0x7f2524159890 2023-10-08T00:56:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.825+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f252404c3c0 con 0x7f2524159890 2023-10-08T00:56:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.826+0000 7f25197fa700 1 -- 172.21.15.119:0/2347876603 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f252001b3b0 con 0x7f2524159890 2023-10-08T00:56:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.827+0000 7f25197fa700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2508066f30 0x7f25080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.827+0000 7f25197fa700 1 -- 172.21.15.119:0/2347876603 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2520083a20 con 0x7f2524159890 2023-10-08T00:56:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.827+0000 7f2528d9a700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2508066f30 0x7f25080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:23.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.828+0000 7f2528d9a700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2508066f30 0x7f25080693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25240cf6b0 tx=0x7f2514009250 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:23.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:23.831+0000 7f25197fa700 1 -- 172.21.15.119:0/2347876603 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2520050bf0 con 0x7f2524159890 2023-10-08T00:56:24.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.027+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f252406a5b0 con 0x7f2524159890 2023-10-08T00:56:24.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.028+0000 7f25197fa700 1 -- 172.21.15.119:0/2347876603 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f2520017020 con 0x7f2524159890 2023-10-08T00:56:24.029 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:24.029 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:24.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2508066f30 msgr2=0x7f25080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2508066f30 0x7f25080693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f25240cf6b0 tx=0x7f2514009250 comp rx=0 tx=0).stop 2023-10-08T00:56:24.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524159890 msgr2=0x7f2524156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524159890 0x7f2524156b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f252000c7c0 tx=0x7f2520007dc0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 shutdown_connections 2023-10-08T00:56:24.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2508066f30 0x7f25080693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f252415db10 0x7f25240cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524159890 0x7f2524156b50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 --2- 172.21.15.119:0/2347876603 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2524159c60 0x7f25240ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 >> 172.21.15.119:0/2347876603 conn(0x7f2524068e20 msgr2=0x7f25240b0540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:24.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 shutdown_connections 2023-10-08T00:56:24.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.031+0000 7f252b7ff700 1 -- 172.21.15.119:0/2347876603 wait complete. 2023-10-08T00:56:24.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:55:45.270364+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:55:45.937180+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T00:56:24.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:56:24.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.233+0000 7f55f3fff700 1 Processor -- start 2023-10-08T00:56:24.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.233+0000 7f55f3fff700 1 -- start start 2023-10-08T00:56:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.234+0000 7f55f3fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f55f40aef00 0x7f55f415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.234+0000 7f55f3fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40aea70 con 0x7f55f406e470 2023-10-08T00:56:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.234+0000 7f55f3fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f40ae480 con 0x7f55f415a3b0 2023-10-08T00:56:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.234+0000 7f55f3fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f415dd30 con 0x7f55f40aef00 2023-10-08T00:56:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f37fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f55f406e470 0x7f55f40c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37612/0 (socket says 172.21.15.119:37612) 2023-10-08T00:56:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f37fe700 1 -- 172.21.15.119:0/619930279 learned_addr learned my addr 172.21.15.119:0/619930279 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:24.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f27fc700 1 --2- 172.21.15.119:0/619930279 >> v2:172.21.15.184:3300/0 conn(0x7f55f40aef00 0x7f55f415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f1ffb700 1 -- 172.21.15.119:0/619930279 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099124458 0 0) 0x7f55f40aea70 con 0x7f55f406e470 2023-10-08T00:56:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f1ffb700 1 -- 172.21.15.119:0/619930279 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55dc003570 con 0x7f55f406e470 2023-10-08T00:56:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f1ffb700 1 -- 172.21.15.119:0/619930279 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719161399 0 0) 0x7f55f40ae480 con 0x7f55f415a3b0 2023-10-08T00:56:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f1ffb700 1 -- 172.21.15.119:0/619930279 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55f40aea70 con 0x7f55f415a3b0 2023-10-08T00:56:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f27fc700 1 -- 172.21.15.119:0/619930279 >> v1:172.21.15.184:6789/0 conn(0x7f55f415a3b0 legacy=0x7f55f415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f27fc700 1 -- 172.21.15.119:0/619930279 >> v1:172.21.15.119:6789/0 conn(0x7f55f406e470 legacy=0x7f55f40c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.235+0000 7f55f27fc700 1 -- 172.21.15.119:0/619930279 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f415e010 con 0x7f55f40aef00 2023-10-08T00:56:24.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f27fc700 1 --2- 172.21.15.119:0/619930279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f40aef00 0x7f55f415d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f55e80098e0 tx=0x7f55e80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6757220a759ce8c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f1ffb700 1 -- 172.21.15.119:0/619930279 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011630 con 0x7f55f40aef00 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f1ffb700 1 -- 172.21.15.119:0/619930279 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e8011790 con 0x7f55f40aef00 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f1ffb700 1 -- 172.21.15.119:0/619930279 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011b10 con 0x7f55f40aef00 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f3fff700 1 -- 172.21.15.119:0/619930279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f40aef00 msgr2=0x7f55f415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f3fff700 1 --2- 172.21.15.119:0/619930279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f40aef00 0x7f55f415d3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f55e80098e0 tx=0x7f55e80091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f3fff700 1 -- 172.21.15.119:0/619930279 shutdown_connections 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f3fff700 1 --2- 172.21.15.119:0/619930279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f40aef00 0x7f55f415d3c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.236+0000 7f55f3fff700 1 -- 172.21.15.119:0/619930279 >> 172.21.15.119:0/619930279 conn(0x7f55f40c9f70 msgr2=0x7f55f406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.237+0000 7f55f3fff700 1 -- 172.21.15.119:0/619930279 shutdown_connections 2023-10-08T00:56:24.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.237+0000 7f55f3fff700 1 -- 172.21.15.119:0/619930279 wait complete. 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.238+0000 7f55f3fff700 1 Processor -- start 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.238+0000 7f55f3fff700 1 -- start start 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f406e470 0x7f55f416d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f40aef00 0x7f55f416da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55f415a3b0 0x7f55f40b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4167b90 con 0x7f55f40aef00 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f3fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f4168150 con 0x7f55f406e470 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f40c5220 con 0x7f55f415a3b0 2023-10-08T00:56:24.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f27fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f40aef00 0x7f55f416da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f27fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f40aef00 0x7f55f416da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35246/0 (socket says 172.21.15.119:35246) 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f27fc700 1 -- 172.21.15.119:0/862867398 learned_addr learned my addr 172.21.15.119:0/862867398 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f2ffd700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f406e470 0x7f55f416d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.239+0000 7f55f37fe700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55f415a3b0 0x7f55f40b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55f27fc700 1 -- 172.21.15.119:0/862867398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55f415a3b0 msgr2=0x7f55f40b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55f27fc700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55f415a3b0 0x7f55f40b1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55f27fc700 1 -- 172.21.15.119:0/862867398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f406e470 msgr2=0x7f55f416d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55f27fc700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f406e470 0x7f55f416d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55f27fc700 1 -- 172.21.15.119:0/862867398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e8009420 con 0x7f55f40aef00 2023-10-08T00:56:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55f27fc700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f40aef00 0x7f55f416da90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f55e80116b0 tx=0x7f55e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55daffd700 1 -- 172.21.15.119:0/862867398 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e8011b10 con 0x7f55f40aef00 2023-10-08T00:56:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55daffd700 1 -- 172.21.15.119:0/862867398 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e8000d10 con 0x7f55f40aef00 2023-10-08T00:56:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f40c5500 con 0x7f55f40aef00 2023-10-08T00:56:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55daffd700 1 -- 172.21.15.119:0/862867398 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e801a660 con 0x7f55f40aef00 2023-10-08T00:56:24.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.240+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f40b1dd0 con 0x7f55f40aef00 2023-10-08T00:56:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.241+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f4167fa0 con 0x7f55f40aef00 2023-10-08T00:56:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.242+0000 7f55daffd700 1 -- 172.21.15.119:0/862867398 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f55e801aa10 con 0x7f55f40aef00 2023-10-08T00:56:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.243+0000 7f55daffd700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f55e0066f30 0x7f55e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.243+0000 7f55daffd700 1 -- 172.21.15.119:0/862867398 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f55e80862c0 con 0x7f55f40aef00 2023-10-08T00:56:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.243+0000 7f55f2ffd700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f55e0066f30 0x7f55e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.244+0000 7f55f2ffd700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f55e0066f30 0x7f55e00693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f55e4007fd0 tx=0x7f55e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:24.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.247+0000 7f55daffd700 1 -- 172.21.15.119:0/862867398 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f55e8053490 con 0x7f55f40aef00 2023-10-08T00:56:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.436+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55f40c8110 con 0x7f55f40aef00 2023-10-08T00:56:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.437+0000 7f55daffd700 1 -- 172.21.15.119:0/862867398 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55e8056ab0 con 0x7f55f40aef00 2023-10-08T00:56:24.438 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f55e0066f30 msgr2=0x7f55e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f55e0066f30 0x7f55e00693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f55e4007fd0 tx=0x7f55e4009040 comp rx=0 tx=0).stop 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f40aef00 msgr2=0x7f55f416da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f40aef00 0x7f55f416da90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f55e80116b0 tx=0x7f55e8017000 comp rx=0 tx=0).stop 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 shutdown_connections 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55f415a3b0 0x7f55f40b1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f55e0066f30 0x7f55e00693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55f406e470 0x7f55f416d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 --2- 172.21.15.119:0/862867398 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55f40aef00 0x7f55f416da90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 >> 172.21.15.119:0/862867398 conn(0x7f55f40c9f70 msgr2=0x7f55f415aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 shutdown_connections 2023-10-08T00:56:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.439+0000 7f55f3fff700 1 -- 172.21.15.119:0/862867398 wait complete. 2023-10-08T00:56:24.458 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T00:56:24.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.641+0000 7fba9f5ff700 1 Processor -- start 2023-10-08T00:56:24.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.641+0000 7fba9f5ff700 1 -- start start 2023-10-08T00:56:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.642+0000 7fba9f5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba98159c60 0x7fba9815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.642+0000 7fba9f5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fba9815db10 0x7fba98069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.642+0000 7fba9f5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9815a160 con 0x7fba98159890 2023-10-08T00:56:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.642+0000 7fba9f5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba980d13a0 con 0x7fba9815db10 2023-10-08T00:56:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.642+0000 7fba9f5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9815e140 con 0x7fba98159c60 2023-10-08T00:56:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9db9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fba9815db10 0x7fba98069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9d39b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fba98159890 0x7fba9815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37622/0 (socket says 172.21.15.119:37622) 2023-10-08T00:56:24.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9d39b700 1 -- 172.21.15.119:0/438977290 learned_addr learned my addr 172.21.15.119:0/438977290 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:24.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9cb9a700 1 --2- 172.21.15.119:0/438977290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba98159c60 0x7fba9815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9db9c700 1 -- 172.21.15.119:0/438977290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba98159c60 msgr2=0x7fba9815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:24.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9db9c700 1 --2- 172.21.15.119:0/438977290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba98159c60 0x7fba9815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9db9c700 1 -- 172.21.15.119:0/438977290 >> v1:172.21.15.119:6789/0 conn(0x7fba98159890 legacy=0x7fba9815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9db9c700 1 -- 172.21.15.119:0/438977290 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9815a3d0 con 0x7fba9815db10 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9cb9a700 1 --2- 172.21.15.119:0/438977290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba98159c60 0x7fba9815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba8ffff700 1 -- 172.21.15.119:0/438977290 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636152442 0 0) 0x7fba9815a160 con 0x7fba98159890 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.643+0000 7fba9db9c700 1 --2- 172.21.15.119:0/438977290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba9815db10 0x7fba98069560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba90009e60 tx=0x7fba90009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17676bfa3026bf92 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba8ffff700 1 -- 172.21.15.119:0/438977290 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9000e070 con 0x7fba9815db10 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba8ffff700 1 -- 172.21.15.119:0/438977290 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba9000bc30 con 0x7fba9815db10 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba8ffff700 1 -- 172.21.15.119:0/438977290 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba90012430 con 0x7fba9815db10 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba9f5ff700 1 -- 172.21.15.119:0/438977290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba9815db10 msgr2=0x7fba98069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/438977290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba9815db10 0x7fba98069560 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fba90009e60 tx=0x7fba90009780 comp rx=0 tx=0).stop 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba9f5ff700 1 -- 172.21.15.119:0/438977290 shutdown_connections 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/438977290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba9815db10 0x7fba98069560 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/438977290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba98159c60 0x7fba9815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba9f5ff700 1 -- 172.21.15.119:0/438977290 >> 172.21.15.119:0/438977290 conn(0x7fba98068e20 msgr2=0x7fba980d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:24.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba9f5ff700 1 -- 172.21.15.119:0/438977290 shutdown_connections 2023-10-08T00:56:24.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.644+0000 7fba9f5ff700 1 -- 172.21.15.119:0/438977290 wait complete. 2023-10-08T00:56:24.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.646+0000 7fba9f5ff700 1 Processor -- start 2023-10-08T00:56:24.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.646+0000 7fba9f5ff700 1 -- start start 2023-10-08T00:56:24.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9f5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba98159890 0x7fba980cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9f5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba98159c60 0x7fba980b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9f5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba9815db10 0x7fba981583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9f5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980b0990 con 0x7fba98159c60 2023-10-08T00:56:24.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9f5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba980cf630 con 0x7fba98159890 2023-10-08T00:56:24.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9f5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba98151c90 con 0x7fba9815db10 2023-10-08T00:56:24.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9cb9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba98159c60 0x7fba980b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9cb9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba98159c60 0x7fba980b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35262/0 (socket says 172.21.15.119:35262) 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9cb9a700 1 -- 172.21.15.119:0/2438557656 learned_addr learned my addr 172.21.15.119:0/2438557656 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9d39b700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba98159890 0x7fba980cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9db9c700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba9815db10 0x7fba981583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9cb9a700 1 -- 172.21.15.119:0/2438557656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba9815db10 msgr2=0x7fba981583b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9cb9a700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba9815db10 0x7fba981583b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9cb9a700 1 -- 172.21.15.119:0/2438557656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba98159890 msgr2=0x7fba980cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9cb9a700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba98159890 0x7fba980cf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.647+0000 7fba9cb9a700 1 -- 172.21.15.119:0/2438557656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba90009a20 con 0x7fba98159c60 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.648+0000 7fba9cb9a700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba98159c60 0x7fba980b5cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba8800c8b0 tx=0x7fba88008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.648+0000 7fba8d7fa700 1 -- 172.21.15.119:0/2438557656 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88007670 con 0x7fba98159c60 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.648+0000 7fba8d7fa700 1 -- 172.21.15.119:0/2438557656 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba880077d0 con 0x7fba98159c60 2023-10-08T00:56:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.648+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba980b0660 con 0x7fba98159c60 2023-10-08T00:56:24.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.648+0000 7fba8d7fa700 1 -- 172.21.15.119:0/2438557656 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88007b70 con 0x7fba98159c60 2023-10-08T00:56:24.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.648+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba980cf8b0 con 0x7fba98159c60 2023-10-08T00:56:24.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.649+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba980b0b40 con 0x7fba98159c60 2023-10-08T00:56:24.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.650+0000 7fba8d7fa700 1 -- 172.21.15.119:0/2438557656 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fba880113f0 con 0x7fba98159c60 2023-10-08T00:56:24.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.651+0000 7fba8d7fa700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fba7c066f30 0x7fba7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:24.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.651+0000 7fba8d7fa700 1 -- 172.21.15.119:0/2438557656 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fba880846b0 con 0x7fba98159c60 2023-10-08T00:56:24.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.651+0000 7fba9d39b700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fba7c066f30 0x7fba7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:24.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.652+0000 7fba9d39b700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fba7c066f30 0x7fba7c0693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba940011e0 tx=0x7fba94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:24.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.655+0000 7fba8d7fa700 1 -- 172.21.15.119:0/2438557656 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fba88051920 con 0x7fba98159c60 2023-10-08T00:56:24.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.843+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba9804c3c0 con 0x7fba98159c60 2023-10-08T00:56:24.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.843+0000 7fba8d7fa700 1 -- 172.21.15.119:0/2438557656 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba88054f40 con 0x7fba98159c60 2023-10-08T00:56:24.845 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T00:56:24.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.845+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fba7c066f30 msgr2=0x7fba7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.845+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fba7c066f30 0x7fba7c0693e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fba940011e0 tx=0x7fba94009040 comp rx=0 tx=0).stop 2023-10-08T00:56:24.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba98159c60 msgr2=0x7fba980b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:24.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba98159c60 0x7fba980b5cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fba8800c8b0 tx=0x7fba88008040 comp rx=0 tx=0).stop 2023-10-08T00:56:24.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 shutdown_connections 2023-10-08T00:56:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba9815db10 0x7fba981583b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fba7c066f30 0x7fba7c0693e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba98159890 0x7fba980cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 --2- 172.21.15.119:0/2438557656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba98159c60 0x7fba980b5cf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 >> 172.21.15.119:0/2438557656 conn(0x7fba98068e20 msgr2=0x7fba9815d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 shutdown_connections 2023-10-08T00:56:24.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:24.846+0000 7fba9f5ff700 1 -- 172.21.15.119:0/2438557656 wait complete. 2023-10-08T00:56:24.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T00:56:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.047+0000 7f584dd9d700 1 Processor -- start 2023-10-08T00:56:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.047+0000 7f584dd9d700 1 -- start start 2023-10-08T00:56:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.048+0000 7f584dd9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f584806f060 0x7f58480ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.048+0000 7f584dd9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f584806f4b0 0x7f58480ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584dd9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584806ebd0 con 0x7f58480cb1a0 2023-10-08T00:56:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584dd9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584815b2e0 con 0x7f584806f4b0 2023-10-08T00:56:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584dd9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584815aa70 con 0x7f584806f060 2023-10-08T00:56:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584d59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f584806f4b0 0x7f58480ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584d59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f584806f4b0 0x7f58480ae610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46248/0 (socket says 172.21.15.119:46248) 2023-10-08T00:56:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584d59c700 1 -- 172.21.15.119:0/3532391860 learned_addr learned my addr 172.21.15.119:0/3532391860 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f5847fff700 1 --2- 172.21.15.119:0/3532391860 >> v2:172.21.15.184:3300/0 conn(0x7f584806f060 0x7f58480ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584d59c700 1 -- 172.21.15.119:0/3532391860 >> v2:172.21.15.184:3300/0 conn(0x7f584806f060 msgr2=0x7f58480ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584d59c700 1 --2- 172.21.15.119:0/3532391860 >> v2:172.21.15.184:3300/0 conn(0x7f584806f060 0x7f58480ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584d59c700 1 -- 172.21.15.119:0/3532391860 >> v1:172.21.15.184:6789/0 conn(0x7f58480cb1a0 legacy=0x7f584815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.049+0000 7f584d59c700 1 -- 172.21.15.119:0/3532391860 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584815ad20 con 0x7f584806f4b0 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f584d59c700 1 --2- 172.21.15.119:0/3532391860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58480ae610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5848070d30 tx=0x7f5840009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f4bbbaccd10441 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f58477fe700 1 -- 172.21.15.119:0/3532391860 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584000e070 con 0x7f584806f4b0 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f58477fe700 1 -- 172.21.15.119:0/3532391860 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584000bae0 con 0x7f584806f4b0 2023-10-08T00:56:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f58477fe700 1 -- 172.21.15.119:0/3532391860 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58400092e0 con 0x7f584806f4b0 2023-10-08T00:56:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f584dd9d700 1 -- 172.21.15.119:0/3532391860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 msgr2=0x7f58480ae610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f584dd9d700 1 --2- 172.21.15.119:0/3532391860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58480ae610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5848070d30 tx=0x7f5840009840 comp rx=0 tx=0).stop 2023-10-08T00:56:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f584dd9d700 1 -- 172.21.15.119:0/3532391860 shutdown_connections 2023-10-08T00:56:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f584dd9d700 1 --2- 172.21.15.119:0/3532391860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58480ae610 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f584dd9d700 1 --2- 172.21.15.119:0/3532391860 >> v2:172.21.15.184:3300/0 conn(0x7f584806f060 0x7f58480ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f584dd9d700 1 -- 172.21.15.119:0/3532391860 >> 172.21.15.119:0/3532391860 conn(0x7f58480cc1e0 msgr2=0x7f58480cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.050+0000 7f584dd9d700 1 -- 172.21.15.119:0/3532391860 shutdown_connections 2023-10-08T00:56:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.051+0000 7f584dd9d700 1 -- 172.21.15.119:0/3532391860 wait complete. 2023-10-08T00:56:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.052+0000 7f584dd9d700 1 Processor -- start 2023-10-08T00:56:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.052+0000 7f584dd9d700 1 -- start start 2023-10-08T00:56:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f584dd9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f584806f060 0x7f5848142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f584dd9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58481591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f584dd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58480cb1a0 0x7f5848159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f584dd9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58481479d0 con 0x7f584806f4b0 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f584dd9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5848156d60 con 0x7f584806f060 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f584dd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5848157040 con 0x7f58480cb1a0 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f5847fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58481591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f5847fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58481591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46260/0 (socket says 172.21.15.119:46260) 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f5847fff700 1 -- 172.21.15.119:0/2104670044 learned_addr learned my addr 172.21.15.119:0/2104670044 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f584cd9b700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f584806f060 0x7f5848142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f584d59c700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58480cb1a0 0x7f5848159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f5847fff700 1 -- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58480cb1a0 msgr2=0x7f5848159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f5847fff700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58480cb1a0 0x7f5848159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f5847fff700 1 -- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f584806f060 msgr2=0x7f5848142e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.053+0000 7f5847fff700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f584806f060 0x7f5848142e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.054+0000 7f5847fff700 1 -- 172.21.15.119:0/2104670044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5840009ae0 con 0x7f584806f4b0 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.054+0000 7f584cd9b700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f584806f060 0x7f5848142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.054+0000 7f5847fff700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58481591f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f583400d890 tx=0x7f5834008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.054+0000 7f5844ff9700 1 -- 172.21.15.119:0/2104670044 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340125f0 con 0x7f584806f4b0 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.054+0000 7f5844ff9700 1 -- 172.21.15.119:0/2104670044 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5834012770 con 0x7f584806f4b0 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.054+0000 7f5844ff9700 1 -- 172.21.15.119:0/2104670044 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834012b30 con 0x7f584806f4b0 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.054+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58480b1140 con 0x7f584806f4b0 2023-10-08T00:56:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.054+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58480b1700 con 0x7f584806f4b0 2023-10-08T00:56:25.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.055+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58480cbfa0 con 0x7f584806f4b0 2023-10-08T00:56:25.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.056+0000 7f5844ff9700 1 -- 172.21.15.119:0/2104670044 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f583401e460 con 0x7f584806f4b0 2023-10-08T00:56:25.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.057+0000 7f5844ff9700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f582c066f30 0x7f582c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.057+0000 7f5844ff9700 1 -- 172.21.15.119:0/2104670044 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f5834084cd0 con 0x7f584806f4b0 2023-10-08T00:56:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.057+0000 7f584cd9b700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f582c066f30 0x7f582c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.057+0000 7f584cd9b700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f582c066f30 0x7f582c0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f583c007fd0 tx=0x7f583c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.061+0000 7f5844ff9700 1 -- 172.21.15.119:0/2104670044 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5834051ea0 con 0x7f584806f4b0 2023-10-08T00:56:25.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.251+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58481571b0 con 0x7f584806f4b0 2023-10-08T00:56:25.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.252+0000 7f5844ff9700 1 -- 172.21.15.119:0/2104670044 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f58340554c0 con 0x7f584806f4b0 2023-10-08T00:56:25.253 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f582c066f30 msgr2=0x7f582c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f582c066f30 0x7f582c0693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f583c007fd0 tx=0x7f583c009040 comp rx=0 tx=0).stop 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 msgr2=0x7f58481591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58481591f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f583400d890 tx=0x7f5834008040 comp rx=0 tx=0).stop 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 shutdown_connections 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58480cb1a0 0x7f5848159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f582c066f30 0x7f582c0693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f584806f060 0x7f5848142e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 --2- 172.21.15.119:0/2104670044 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f584806f4b0 0x7f58481591f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 >> 172.21.15.119:0/2104670044 conn(0x7f58480cc1e0 msgr2=0x7f584815a040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:25.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 shutdown_connections 2023-10-08T00:56:25.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.254+0000 7f584dd9d700 1 -- 172.21.15.119:0/2104670044 wait complete. 2023-10-08T00:56:25.273 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.451+0000 7fbc215ff700 1 Processor -- start 2023-10-08T00:56:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.451+0000 7fbc215ff700 1 -- start start 2023-10-08T00:56:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc215ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc1c0cbf30 0x7fbc1c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc215ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0ccf90 0x7fbc1c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc215ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c0cc300 con 0x7fbc1c15a270 2023-10-08T00:56:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc215ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c156bf0 con 0x7fbc1c0cbf30 2023-10-08T00:56:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc215ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0ae950 con 0x7fbc1c0ccf90 2023-10-08T00:56:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc1affd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc1c0cbf30 0x7fbc1c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc1a7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0ccf90 0x7fbc1c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc1a7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0ccf90 0x7fbc1c0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:40986/0 (socket says 172.21.15.119:40986) 2023-10-08T00:56:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc1affd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc1c0cbf30 0x7fbc1c0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:38488/0 (socket says 172.21.15.119:38488) 2023-10-08T00:56:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc1a7fc700 1 -- 172.21.15.119:0/320696120 learned_addr learned my addr 172.21.15.119:0/320696120 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.453+0000 7fbc1b7fe700 1 --1- 172.21.15.119:0/320696120 >> v1:172.21.15.184:6789/0 conn(0x7fbc1c15a270 0x7fbc1c0c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:41050/0 (socket says 172.21.15.119:41050) 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc1a7fc700 1 -- 172.21.15.119:0/320696120 >> v2:172.21.15.184:3300/0 conn(0x7fbc1c0cbf30 msgr2=0x7fbc1c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc1a7fc700 1 --2- 172.21.15.119:0/320696120 >> v2:172.21.15.184:3300/0 conn(0x7fbc1c0cbf30 0x7fbc1c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc1a7fc700 1 -- 172.21.15.119:0/320696120 >> v1:172.21.15.184:6789/0 conn(0x7fbc1c15a270 legacy=0x7fbc1c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc19ffb700 1 -- 172.21.15.119:0/320696120 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479442914 0 0) 0x7fbc1c0cc300 con 0x7fbc1c15a270 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc1a7fc700 1 -- 172.21.15.119:0/320696120 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c0ad370 con 0x7fbc1c0ccf90 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc1a7fc700 1 --2- 172.21.15.119:0/320696120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0ccf90 0x7fbc1c0b0d10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc100099b0 tx=0x7fbc10009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a64788c6aedd0e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc19ffb700 1 -- 172.21.15.119:0/320696120 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10009ea0 con 0x7fbc1c0ccf90 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc19ffb700 1 -- 172.21.15.119:0/320696120 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1000b860 con 0x7fbc1c0ccf90 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.454+0000 7fbc19ffb700 1 -- 172.21.15.119:0/320696120 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1000bc00 con 0x7fbc1c0ccf90 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.455+0000 7fbc215ff700 1 -- 172.21.15.119:0/320696120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0ccf90 msgr2=0x7fbc1c0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.455+0000 7fbc215ff700 1 --2- 172.21.15.119:0/320696120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0ccf90 0x7fbc1c0b0d10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc100099b0 tx=0x7fbc10009250 comp rx=0 tx=0).stop 2023-10-08T00:56:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.455+0000 7fbc215ff700 1 -- 172.21.15.119:0/320696120 shutdown_connections 2023-10-08T00:56:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.455+0000 7fbc215ff700 1 --2- 172.21.15.119:0/320696120 >> v2:172.21.15.184:3300/0 conn(0x7fbc1c0cbf30 0x7fbc1c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.455+0000 7fbc215ff700 1 --2- 172.21.15.119:0/320696120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0ccf90 0x7fbc1c0b0d10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.455+0000 7fbc215ff700 1 -- 172.21.15.119:0/320696120 >> 172.21.15.119:0/320696120 conn(0x7fbc1c06a320 msgr2=0x7fbc1c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.455+0000 7fbc215ff700 1 -- 172.21.15.119:0/320696120 shutdown_connections 2023-10-08T00:56:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.455+0000 7fbc215ff700 1 -- 172.21.15.119:0/320696120 wait complete. 2023-10-08T00:56:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.456+0000 7fbc215ff700 1 Processor -- start 2023-10-08T00:56:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.456+0000 7fbc215ff700 1 -- start start 2023-10-08T00:56:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.457+0000 7fbc215ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0cbf30 0x7fbc1c0ac7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.457+0000 7fbc215ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c0ccf90 0x7fbc1c14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.457+0000 7fbc215ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c15a270 0x7fbc1c154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.457+0000 7fbc215ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c153a00 con 0x7fbc1c15a270 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.457+0000 7fbc215ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c153ce0 con 0x7fbc1c0ccf90 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.457+0000 7fbc215ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c153fc0 con 0x7fbc1c0cbf30 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c15a270 0x7fbc1c154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c15a270 0x7fbc1c154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46262/0 (socket says 172.21.15.119:46262) 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 -- 172.21.15.119:0/2516487594 learned_addr learned my addr 172.21.15.119:0/2516487594 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1a7fc700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c0ccf90 0x7fbc1c14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1affd700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0cbf30 0x7fbc1c0ac7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 -- 172.21.15.119:0/2516487594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0cbf30 msgr2=0x7fbc1c0ac7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0cbf30 0x7fbc1c0ac7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 -- 172.21.15.119:0/2516487594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c0ccf90 msgr2=0x7fbc1c14f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c0ccf90 0x7fbc1c14f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 -- 172.21.15.119:0/2516487594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc100094f0 con 0x7fbc1c15a270 2023-10-08T00:56:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc1b7fe700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c15a270 0x7fbc1c154aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c530 tx=0x7fbc14007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.458+0000 7fbc02ffd700 1 -- 172.21.15.119:0/2516487594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14005a70 con 0x7fbc1c15a270 2023-10-08T00:56:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.459+0000 7fbc02ffd700 1 -- 172.21.15.119:0/2516487594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc14005bd0 con 0x7fbc1c15a270 2023-10-08T00:56:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.459+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c1542a0 con 0x7fbc1c15a270 2023-10-08T00:56:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.459+0000 7fbc02ffd700 1 -- 172.21.15.119:0/2516487594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc140070d0 con 0x7fbc1c15a270 2023-10-08T00:56:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.459+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c0acd20 con 0x7fbc1c15a270 2023-10-08T00:56:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.460+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c1547c0 con 0x7fbc1c15a270 2023-10-08T00:56:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.460+0000 7fbc02ffd700 1 -- 172.21.15.119:0/2516487594 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc14007480 con 0x7fbc1c15a270 2023-10-08T00:56:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.461+0000 7fbc02ffd700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.461+0000 7fbc02ffd700 1 -- 172.21.15.119:0/2516487594 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fbc14083af0 con 0x7fbc1c15a270 2023-10-08T00:56:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.462+0000 7fbc1affd700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.462+0000 7fbc1affd700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc08066f30 0x7fbc080693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c15b670 tx=0x7fbc0c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.466+0000 7fbc02ffd700 1 -- 172.21.15.119:0/2516487594 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc1c1547c0 con 0x7fbc1c15a270 2023-10-08T00:56:25.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.700+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc1c1547c0 con 0x7fbc1c15a270 2023-10-08T00:56:25.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.701+0000 7fbc02ffd700 1 -- 172.21.15.119:0/2516487594 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fbc1c1547c0 con 0x7fbc1c15a270 2023-10-08T00:56:25.702 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:25.702 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:55:46.965967+0000","modified":"2023-10-08T00:55:50.512168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:56:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc08066f30 msgr2=0x7fbc080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc08066f30 0x7fbc080693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c15b670 tx=0x7fbc0c00b040 comp rx=0 tx=0).stop 2023-10-08T00:56:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c15a270 msgr2=0x7fbc1c154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c15a270 0x7fbc1c154aa0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400c530 tx=0x7fbc14007f90 comp rx=0 tx=0).stop 2023-10-08T00:56:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 shutdown_connections 2023-10-08T00:56:25.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0cbf30 0x7fbc1c0ac7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c15a270 0x7fbc1c154aa0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 --2- 172.21.15.119:0/2516487594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c0ccf90 0x7fbc1c14f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 >> 172.21.15.119:0/2516487594 conn(0x7fbc1c06a320 msgr2=0x7fbc1c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 shutdown_connections 2023-10-08T00:56:25.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.703+0000 7fbc215ff700 1 -- 172.21.15.119:0/2516487594 wait complete. 2023-10-08T00:56:25.707 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-10-08T00:56:25.722 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.903+0000 7fde8df9d700 1 Processor -- start 2023-10-08T00:56:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.903+0000 7fde8df9d700 1 -- start start 2023-10-08T00:56:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde8df9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fde88159420 0x7fde880b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde8df9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde881597f0 0x7fde880b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde8df9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde8815d230 0x7fde8815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde8df9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8815dbb0 con 0x7fde881597f0 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde8df9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde880b11d0 con 0x7fde88159420 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde8df9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde880b14b0 con 0x7fde8815d230 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde87fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde881597f0 0x7fde880b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde87fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde881597f0 0x7fde880b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46274/0 (socket says 172.21.15.119:46274) 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.904+0000 7fde87fff700 1 -- 172.21.15.119:0/168505460 learned_addr learned my addr 172.21.15.119:0/168505460 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde8cf9b700 1 --2- 172.21.15.119:0/168505460 >> v2:172.21.15.184:3300/0 conn(0x7fde88159420 0x7fde880b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde8d79c700 1 --2- 172.21.15.119:0/168505460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde8815d230 0x7fde8815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde87fff700 1 -- 172.21.15.119:0/168505460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde8815d230 msgr2=0x7fde8815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde87fff700 1 --2- 172.21.15.119:0/168505460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde8815d230 0x7fde8815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde87fff700 1 -- 172.21.15.119:0/168505460 >> v2:172.21.15.184:3300/0 conn(0x7fde88159420 msgr2=0x7fde880b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde87fff700 1 --2- 172.21.15.119:0/168505460 >> v2:172.21.15.184:3300/0 conn(0x7fde88159420 0x7fde880b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde87fff700 1 -- 172.21.15.119:0/168505460 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde880b1790 con 0x7fde881597f0 2023-10-08T00:56:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde8cf9b700 1 --2- 172.21.15.119:0/168505460 >> v2:172.21.15.184:3300/0 conn(0x7fde88159420 0x7fde880b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde8d79c700 1 --2- 172.21.15.119:0/168505460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde8815d230 0x7fde8815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde87fff700 1 --2- 172.21.15.119:0/168505460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde881597f0 0x7fde880b2270 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde880b2f20 tx=0x7fde74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c58b0df94272613 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde877fe700 1 -- 172.21.15.119:0/168505460 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7400b860 con 0x7fde881597f0 2023-10-08T00:56:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde877fe700 1 -- 172.21.15.119:0/168505460 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7400b9c0 con 0x7fde881597f0 2023-10-08T00:56:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.905+0000 7fde877fe700 1 -- 172.21.15.119:0/168505460 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7400bd60 con 0x7fde881597f0 2023-10-08T00:56:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 -- 172.21.15.119:0/168505460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde881597f0 msgr2=0x7fde880b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 --2- 172.21.15.119:0/168505460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde881597f0 0x7fde880b2270 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde880b2f20 tx=0x7fde74009250 comp rx=0 tx=0).stop 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 -- 172.21.15.119:0/168505460 shutdown_connections 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 --2- 172.21.15.119:0/168505460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde8815d230 0x7fde8815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 --2- 172.21.15.119:0/168505460 >> v2:172.21.15.184:3300/0 conn(0x7fde88159420 0x7fde880b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 --2- 172.21.15.119:0/168505460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde881597f0 0x7fde880b2270 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 -- 172.21.15.119:0/168505460 >> 172.21.15.119:0/168505460 conn(0x7fde880d0660 msgr2=0x7fde880d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 -- 172.21.15.119:0/168505460 shutdown_connections 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.906+0000 7fde8df9d700 1 -- 172.21.15.119:0/168505460 wait complete. 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.907+0000 7fde8df9d700 1 Processor -- start 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.907+0000 7fde8df9d700 1 -- start start 2023-10-08T00:56:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.908+0000 7fde8df9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde88159420 0x7fde88159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.908+0000 7fde8df9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde881597f0 0x7fde88153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.908+0000 7fde8df9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde8815d230 0x7fde880cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.908+0000 7fde8df9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8815c290 con 0x7fde8815d230 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.908+0000 7fde8df9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8815c720 con 0x7fde881597f0 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.908+0000 7fde8df9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8815b680 con 0x7fde88159420 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde8815d230 0x7fde880cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde8815d230 0x7fde880cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46284/0 (socket says 172.21.15.119:46284) 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 -- 172.21.15.119:0/3375958325 learned_addr learned my addr 172.21.15.119:0/3375958325 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde87fff700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde881597f0 0x7fde88153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8cf9b700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde88159420 0x7fde88159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 -- 172.21.15.119:0/3375958325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde88159420 msgr2=0x7fde88159e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde88159420 0x7fde88159e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 -- 172.21.15.119:0/3375958325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde881597f0 msgr2=0x7fde88153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde881597f0 0x7fde88153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 -- 172.21.15.119:0/3375958325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde740094f0 con 0x7fde8815d230 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde8d79c700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde8815d230 0x7fde880cc2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fde8000cef0 tx=0x7fde80007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.909+0000 7fde84ff9700 1 -- 172.21.15.119:0/3375958325 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000ad20 con 0x7fde8815d230 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.910+0000 7fde84ff9700 1 -- 172.21.15.119:0/3375958325 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde80016420 con 0x7fde8815d230 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.910+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8815b960 con 0x7fde8815d230 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.910+0000 7fde84ff9700 1 -- 172.21.15.119:0/3375958325 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8000aea0 con 0x7fde8815d230 2023-10-08T00:56:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.910+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8815bf20 con 0x7fde8815d230 2023-10-08T00:56:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.911+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde8804c3c0 con 0x7fde8815d230 2023-10-08T00:56:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.911+0000 7fde84ff9700 1 -- 172.21.15.119:0/3375958325 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde80016a70 con 0x7fde8815d230 2023-10-08T00:56:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.912+0000 7fde84ff9700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fde6c066f30 0x7fde6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.912+0000 7fde84ff9700 1 -- 172.21.15.119:0/3375958325 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fde80012070 con 0x7fde8815d230 2023-10-08T00:56:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.913+0000 7fde8cf9b700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fde6c066f30 0x7fde6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.913+0000 7fde8cf9b700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fde6c066f30 0x7fde6c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00acc0 tx=0x7fde7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:25.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:25.916+0000 7fde84ff9700 1 -- 172.21.15.119:0/3375958325 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde8004e5a0 con 0x7fde8815d230 2023-10-08T00:56:26.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.112+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde880b1a70 con 0x7fde8815d230 2023-10-08T00:56:26.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.113+0000 7fde84ff9700 1 -- 172.21.15.119:0/3375958325 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fde80051fc0 con 0x7fde8815d230 2023-10-08T00:56:26.114 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:26.114 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:26.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fde6c066f30 msgr2=0x7fde6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fde6c066f30 0x7fde6c0693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00acc0 tx=0x7fde7c008040 comp rx=0 tx=0).stop 2023-10-08T00:56:26.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde8815d230 msgr2=0x7fde880cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde8815d230 0x7fde880cc2f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fde8000cef0 tx=0x7fde80007860 comp rx=0 tx=0).stop 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 shutdown_connections 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde88159420 0x7fde88159e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fde6c066f30 0x7fde6c0693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde8815d230 0x7fde880cc2f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 --2- 172.21.15.119:0/3375958325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde881597f0 0x7fde88153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 >> 172.21.15.119:0/3375958325 conn(0x7fde880d0660 msgr2=0x7fde88157210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 shutdown_connections 2023-10-08T00:56:26.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.115+0000 7fde8df9d700 1 -- 172.21.15.119:0/3375958325 wait complete. 2023-10-08T00:56:26.134 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T00:56:26.134 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.312+0000 7f3617bff700 1 Processor -- start 2023-10-08T00:56:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.312+0000 7f3617bff700 1 -- start start 2023-10-08T00:56:26.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f3617bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f3617bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36100b1420 0x7f36100c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f3617bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36100cc300 con 0x7f36100ccf90 2023-10-08T00:56:26.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f3617bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3610156bf0 con 0x7f36100b1420 2023-10-08T00:56:26.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f3617bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36100ae950 con 0x7f36100cbf30 2023-10-08T00:56:26.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f361599b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f361619c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36100b1420 0x7f36100c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f361519a700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f36100ccf90 0x7f36100b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:41056/0 (socket says 172.21.15.119:41056) 2023-10-08T00:56:26.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f361619c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36100b1420 0x7f36100c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:38528/0 (socket says 172.21.15.119:38528) 2023-10-08T00:56:26.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f361599b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:41018/0 (socket says 172.21.15.119:41018) 2023-10-08T00:56:26.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.314+0000 7f361519a700 1 -- 172.21.15.119:0/4246583856 learned_addr learned my addr 172.21.15.119:0/4246583856 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:26.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f361619c700 1 -- 172.21.15.119:0/4246583856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 msgr2=0x7f36100ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:26.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f361619c700 1 --2- 172.21.15.119:0/4246583856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f361619c700 1 -- 172.21.15.119:0/4246583856 >> v1:172.21.15.184:6789/0 conn(0x7f36100ccf90 legacy=0x7f36100b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f361619c700 1 -- 172.21.15.119:0/4246583856 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36100ad370 con 0x7f36100b1420 2023-10-08T00:56:26.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f3614999700 1 -- 172.21.15.119:0/4246583856 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795312449 0 0) 0x7f36100cc300 con 0x7f36100ccf90 2023-10-08T00:56:26.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f361599b700 1 --2- 172.21.15.119:0/4246583856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:26.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f361619c700 1 --2- 172.21.15.119:0/4246583856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100b1420 0x7f36100c9740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3608009e90 tx=0x7f3608009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849bdb58164dc301 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:26.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f3614999700 1 -- 172.21.15.119:0/4246583856 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360800e070 con 0x7f36100b1420 2023-10-08T00:56:26.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f3614999700 1 -- 172.21.15.119:0/4246583856 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360800bbf0 con 0x7f36100b1420 2023-10-08T00:56:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.315+0000 7f3614999700 1 -- 172.21.15.119:0/4246583856 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608012430 con 0x7f36100b1420 2023-10-08T00:56:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.316+0000 7f3617bff700 1 -- 172.21.15.119:0/4246583856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100b1420 msgr2=0x7f36100c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.316+0000 7f3617bff700 1 --2- 172.21.15.119:0/4246583856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100b1420 0x7f36100c9740 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3608009e90 tx=0x7f3608009780 comp rx=0 tx=0).stop 2023-10-08T00:56:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.316+0000 7f3617bff700 1 -- 172.21.15.119:0/4246583856 shutdown_connections 2023-10-08T00:56:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.316+0000 7f3617bff700 1 --2- 172.21.15.119:0/4246583856 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100b1420 0x7f36100c9740 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.316+0000 7f3617bff700 1 --2- 172.21.15.119:0/4246583856 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.316+0000 7f3617bff700 1 -- 172.21.15.119:0/4246583856 >> 172.21.15.119:0/4246583856 conn(0x7f361006a320 msgr2=0x7f36100cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.316+0000 7f3617bff700 1 -- 172.21.15.119:0/4246583856 shutdown_connections 2023-10-08T00:56:26.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.316+0000 7f3617bff700 1 -- 172.21.15.119:0/4246583856 wait complete. 2023-10-08T00:56:26.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.317+0000 7f3617bff700 1 Processor -- start 2023-10-08T00:56:26.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.317+0000 7f3617bff700 1 -- start start 2023-10-08T00:56:26.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.318+0000 7f3617bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36100b1420 0x7f3610154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.318+0000 7f3617bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.318+0000 7f3617bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100ccf90 0x7f36100abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.318+0000 7f3617bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36100c1400 con 0x7f36100b1420 2023-10-08T00:56:26.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.318+0000 7f3617bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36100a6920 con 0x7f36100ccf90 2023-10-08T00:56:26.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.318+0000 7f3617bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36100a6c00 con 0x7f36100cbf30 2023-10-08T00:56:26.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36100b1420 0x7f3610154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36100b1420 0x7f3610154470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46296/0 (socket says 172.21.15.119:46296) 2023-10-08T00:56:26.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 -- 172.21.15.119:0/1018531351 learned_addr learned my addr 172.21.15.119:0/1018531351 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:26.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361619c700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100ccf90 0x7f36100abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361519a700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 -- 172.21.15.119:0/1018531351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 msgr2=0x7f36100a5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100a5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 -- 172.21.15.119:0/1018531351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100ccf90 msgr2=0x7f36100abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100ccf90 0x7f36100abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 -- 172.21.15.119:0/1018531351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c009420 con 0x7f36100b1420 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.319+0000 7f361599b700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36100b1420 0x7f3610154470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f360c00ec80 tx=0x7f360c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.320+0000 7f3605ffb700 1 -- 172.21.15.119:0/1018531351 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c00c920 con 0x7f36100b1420 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.320+0000 7f3605ffb700 1 -- 172.21.15.119:0/1018531351 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c00ca80 con 0x7f36100b1420 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.320+0000 7f3605ffb700 1 -- 172.21.15.119:0/1018531351 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c00ce00 con 0x7f36100b1420 2023-10-08T00:56:26.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.320+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3608009a20 con 0x7f36100b1420 2023-10-08T00:56:26.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.320+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36100a71c0 con 0x7f36100b1420 2023-10-08T00:56:26.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.321+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36100c16e0 con 0x7f36100b1420 2023-10-08T00:56:26.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.322+0000 7f3605ffb700 1 -- 172.21.15.119:0/1018531351 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f360c013680 con 0x7f36100b1420 2023-10-08T00:56:26.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.322+0000 7f3605ffb700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f35f4066f30 0x7f35f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.322+0000 7f3605ffb700 1 -- 172.21.15.119:0/1018531351 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f360c084cc0 con 0x7f36100b1420 2023-10-08T00:56:26.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.323+0000 7f361519a700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f35f4066f30 0x7f35f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.323+0000 7f361519a700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f35f4066f30 0x7f35f40693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36100cc910 tx=0x7f3600007f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:26.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.326+0000 7f3605ffb700 1 -- 172.21.15.119:0/1018531351 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f360c08b080 con 0x7f36100b1420 2023-10-08T00:56:26.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.564+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3610064760 con 0x7f36100b1420 2023-10-08T00:56:26.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.565+0000 7f3605ffb700 1 -- 172.21.15.119:0/1018531351 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7f360c017090 con 0x7f36100b1420 2023-10-08T00:56:26.566 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:26.566 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:55:46.965967+0000","modified":"2023-10-08T00:55:50.512168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:56:26.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f35f4066f30 msgr2=0x7f35f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f35f4066f30 0x7f35f40693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f36100cc910 tx=0x7f3600007f50 comp rx=0 tx=0).stop 2023-10-08T00:56:26.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36100b1420 msgr2=0x7f3610154470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36100b1420 0x7f3610154470 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f360c00ec80 tx=0x7f360c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 shutdown_connections 2023-10-08T00:56:26.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f35f4066f30 0x7f35f40693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36100ccf90 0x7f36100abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36100b1420 0x7f3610154470 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 --2- 172.21.15.119:0/1018531351 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36100cbf30 0x7f36100a5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 >> 172.21.15.119:0/1018531351 conn(0x7f361006a320 msgr2=0x7f36101552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:26.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 shutdown_connections 2023-10-08T00:56:26.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.567+0000 7f3617bff700 1 -- 172.21.15.119:0/1018531351 wait complete. 2023-10-08T00:56:26.570 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-10-08T00:56:26.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.764+0000 7fc8095ff700 1 Processor -- start 2023-10-08T00:56:26.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.764+0000 7fc8095ff700 1 -- start start 2023-10-08T00:56:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.765+0000 7fc8095ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc8040b0a10 0x7fc804155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.765+0000 7fc8095ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8040ad930 con 0x7fc804155c10 2023-10-08T00:56:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.766+0000 7fc8095ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8040acd20 con 0x7fc8041578b0 2023-10-08T00:56:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.766+0000 7fc8095ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8040ad000 con 0x7fc8040b0a10 2023-10-08T00:56:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.766+0000 7fc802ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc804155c10 0x7fc8040ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45106/0 (socket says 172.21.15.119:45106) 2023-10-08T00:56:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.766+0000 7fc802ffd700 1 -- 172.21.15.119:0/586167239 learned_addr learned my addr 172.21.15.119:0/586167239 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.766+0000 7fc8037fe700 1 --2- 172.21.15.119:0/586167239 >> v2:172.21.15.184:3300/0 conn(0x7fc8040b0a10 0x7fc804155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.766+0000 7fc801ffb700 1 -- 172.21.15.119:0/586167239 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663834850 0 0) 0x7fc8040ad930 con 0x7fc804155c10 2023-10-08T00:56:26.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.766+0000 7fc801ffb700 1 -- 172.21.15.119:0/586167239 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7ec004680 con 0x7fc804155c10 2023-10-08T00:56:26.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.767+0000 7fc801ffb700 1 -- 172.21.15.119:0/586167239 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2181296879 0 0) 0x7fc8040acd20 con 0x7fc8041578b0 2023-10-08T00:56:26.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.767+0000 7fc8037fe700 1 -- 172.21.15.119:0/586167239 >> v1:172.21.15.184:6789/0 conn(0x7fc8041578b0 legacy=0x7fc8040af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.767+0000 7fc8037fe700 1 -- 172.21.15.119:0/586167239 >> v1:172.21.15.119:6789/0 conn(0x7fc804155c10 legacy=0x7fc8040ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.767+0000 7fc8037fe700 1 -- 172.21.15.119:0/586167239 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8040ad2e0 con 0x7fc8040b0a10 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.767+0000 7fc8037fe700 1 --2- 172.21.15.119:0/586167239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8040b0a10 0x7fc804155fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009e60 tx=0x7fc7fc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d89012e9d356a445 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.767+0000 7fc801ffb700 1 -- 172.21.15.119:0/586167239 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00e070 con 0x7fc8040b0a10 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.767+0000 7fc801ffb700 1 -- 172.21.15.119:0/586167239 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7fc00b850 con 0x7fc8040b0a10 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.767+0000 7fc801ffb700 1 -- 172.21.15.119:0/586167239 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4001470 con 0x7fc8040b0a10 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.768+0000 7fc8095ff700 1 -- 172.21.15.119:0/586167239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8040b0a10 msgr2=0x7fc804155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.768+0000 7fc8095ff700 1 --2- 172.21.15.119:0/586167239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8040b0a10 0x7fc804155fe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009e60 tx=0x7fc7fc009780 comp rx=0 tx=0).stop 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.768+0000 7fc8095ff700 1 -- 172.21.15.119:0/586167239 shutdown_connections 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.768+0000 7fc8095ff700 1 --2- 172.21.15.119:0/586167239 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8040b0a10 0x7fc804155fe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.768+0000 7fc8095ff700 1 -- 172.21.15.119:0/586167239 >> 172.21.15.119:0/586167239 conn(0x7fc80406a320 msgr2=0x7fc8040cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:26.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.768+0000 7fc8095ff700 1 -- 172.21.15.119:0/586167239 shutdown_connections 2023-10-08T00:56:26.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.768+0000 7fc8095ff700 1 -- 172.21.15.119:0/586167239 wait complete. 2023-10-08T00:56:26.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.769+0000 7fc8095ff700 1 Processor -- start 2023-10-08T00:56:26.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.769+0000 7fc8095ff700 1 -- start start 2023-10-08T00:56:26.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc8095ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8040b0a10 0x7fc80415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc8095ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc804155c10 0x7fc8040a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc8095ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8041578b0 0x7fc804144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc8095ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8040a5a00 con 0x7fc8040b0a10 2023-10-08T00:56:26.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc8095ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80415b200 con 0x7fc8041578b0 2023-10-08T00:56:26.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc8095ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80415b4e0 con 0x7fc804155c10 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc802ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8040b0a10 0x7fc80415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc802ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8040b0a10 0x7fc80415ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46300/0 (socket says 172.21.15.119:46300) 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.770+0000 7fc802ffd700 1 -- 172.21.15.119:0/1421416978 learned_addr learned my addr 172.21.15.119:0/1421416978 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc8037fe700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8041578b0 0x7fc804144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc8027fc700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc804155c10 0x7fc8040a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc802ffd700 1 -- 172.21.15.119:0/1421416978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc804155c10 msgr2=0x7fc8040a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc802ffd700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc804155c10 0x7fc8040a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc802ffd700 1 -- 172.21.15.119:0/1421416978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8041578b0 msgr2=0x7fc804144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc802ffd700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8041578b0 0x7fc804144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc802ffd700 1 -- 172.21.15.119:0/1421416978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc009a20 con 0x7fc8040b0a10 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc802ffd700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8040b0a10 0x7fc80415ab00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40015a0 tx=0x7fc7f4001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:26.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc7eaffd700 1 -- 172.21.15.119:0/1421416978 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400ce40 con 0x7fc8040b0a10 2023-10-08T00:56:26.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc7eaffd700 1 -- 172.21.15.119:0/1421416978 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f4007360 con 0x7fc8040b0a10 2023-10-08T00:56:26.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80415b7c0 con 0x7fc8040b0a10 2023-10-08T00:56:26.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.771+0000 7fc7eaffd700 1 -- 172.21.15.119:0/1421416978 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4004830 con 0x7fc8040b0a10 2023-10-08T00:56:26.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.772+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80415bd80 con 0x7fc8040b0a10 2023-10-08T00:56:26.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.773+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80404c3c0 con 0x7fc8040b0a10 2023-10-08T00:56:26.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.773+0000 7fc7eaffd700 1 -- 172.21.15.119:0/1421416978 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7f4004ba0 con 0x7fc8040b0a10 2023-10-08T00:56:26.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.774+0000 7fc7eaffd700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7f0066f30 0x7fc7f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:26.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.774+0000 7fc7eaffd700 1 -- 172.21.15.119:0/1421416978 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc7f4083970 con 0x7fc8040b0a10 2023-10-08T00:56:26.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.775+0000 7fc8027fc700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7f0066f30 0x7fc7f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:26.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.775+0000 7fc8027fc700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7f0066f30 0x7fc7f00693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc80416acb0 tx=0x7fc7f8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:26.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:26.778+0000 7fc7eaffd700 1 -- 172.21.15.119:0/1421416978 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7f4050c60 con 0x7fc8040b0a10 2023-10-08T00:56:27.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.010+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8040ad5c0 con 0x7fc8040b0a10 2023-10-08T00:56:27.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.010+0000 7fc7eaffd700 1 -- 172.21.15.119:0/1421416978 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4603 (secure 0 0 0) 0x7fc7f4018020 con 0x7fc8040b0a10 2023-10-08T00:56:27.012 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:27.012 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4272,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/3006073490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3006073490},{"type":"v1","addr":"172.21.15.119:6835","nonce":3006073490}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4275,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2018229845","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":2018229845},{"type":"v1","addr":"172.21.15.119:6837","nonce":2018229845}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4279,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2233434766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2233434766},{"type":"v1","addr":"172.21.15.184:6835","nonce":2233434766}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:55:46.965967+0000","modified":"2023-10-08T00:55:50.512168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4282},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4282":{"gid":4282,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6834/1701092387","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1701092387},{"type":"v1","addr":"172.21.15.184:6834","nonce":1701092387}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-08T00:56:27.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7f0066f30 msgr2=0x7fc7f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7f0066f30 0x7fc7f00693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc80416acb0 tx=0x7fc7f8009040 comp rx=0 tx=0).stop 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8040b0a10 msgr2=0x7fc80415ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8040b0a10 0x7fc80415ab00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc7f40015a0 tx=0x7fc7f4001bf0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 shutdown_connections 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7f0066f30 0x7fc7f00693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc8041578b0 0x7fc804144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8040b0a10 0x7fc80415ab00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 --2- 172.21.15.119:0/1421416978 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc804155c10 0x7fc8040a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 >> 172.21.15.119:0/1421416978 conn(0x7fc80406a320 msgr2=0x7fc8040afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:27.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 shutdown_connections 2023-10-08T00:56:27.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.013+0000 7fc8095ff700 1 -- 172.21.15.119:0/1421416978 wait complete. 2023-10-08T00:56:27.016 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-10-08T00:56:27.031 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.212+0000 7f7ed959d700 1 Processor -- start 2023-10-08T00:56:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.212+0000 7f7ed959d700 1 -- start start 2023-10-08T00:56:27.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.213+0000 7f7ed959d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 0x7f7ed40c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.213+0000 7f7ed959d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ed406e730 0x7f7ed406ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.213+0000 7f7ed959d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed406e2a0 con 0x7f7ed4157770 2023-10-08T00:56:27.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.213+0000 7f7ed959d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed4157b40 con 0x7f7ed406e730 2023-10-08T00:56:27.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.213+0000 7f7ed959d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40c7690 con 0x7f7ed40c6d00 2023-10-08T00:56:27.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.213+0000 7f7ed8d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ed406e730 0x7f7ed406ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.213+0000 7f7ed3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7ed4157770 0x7f7ed4150180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45122/0 (socket says 172.21.15.119:45122) 2023-10-08T00:56:27.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.213+0000 7f7ed3fff700 1 -- 172.21.15.119:0/3260142839 learned_addr learned my addr 172.21.15.119:0/3260142839 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:27.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed37fe700 1 --2- 172.21.15.119:0/3260142839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 0x7f7ed40c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed8d9c700 1 -- 172.21.15.119:0/3260142839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 msgr2=0x7f7ed40c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed8d9c700 1 --2- 172.21.15.119:0/3260142839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 0x7f7ed40c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed8d9c700 1 -- 172.21.15.119:0/3260142839 >> v1:172.21.15.119:6789/0 conn(0x7f7ed4157770 legacy=0x7f7ed4150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed8d9c700 1 -- 172.21.15.119:0/3260142839 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed406b2e0 con 0x7f7ed406e730 2023-10-08T00:56:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed2ffd700 1 -- 172.21.15.119:0/3260142839 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1338393449 0 0) 0x7f7ed406e2a0 con 0x7f7ed4157770 2023-10-08T00:56:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed8d9c700 1 --2- 172.21.15.119:0/3260142839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed406ebb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009e90 tx=0x7f7ecc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d287048af25dd99e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed2ffd700 1 -- 172.21.15.119:0/3260142839 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00e070 con 0x7f7ed406e730 2023-10-08T00:56:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed2ffd700 1 -- 172.21.15.119:0/3260142839 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ecc00bbf0 con 0x7f7ed406e730 2023-10-08T00:56:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.214+0000 7f7ed2ffd700 1 -- 172.21.15.119:0/3260142839 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc012430 con 0x7f7ed406e730 2023-10-08T00:56:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.215+0000 7f7ed959d700 1 -- 172.21.15.119:0/3260142839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 msgr2=0x7f7ed406ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.215+0000 7f7ed959d700 1 --2- 172.21.15.119:0/3260142839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed406ebb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc009e90 tx=0x7f7ecc009780 comp rx=0 tx=0).stop 2023-10-08T00:56:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.215+0000 7f7ed959d700 1 -- 172.21.15.119:0/3260142839 shutdown_connections 2023-10-08T00:56:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.215+0000 7f7ed959d700 1 --2- 172.21.15.119:0/3260142839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed406ebb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.215+0000 7f7ed959d700 1 --2- 172.21.15.119:0/3260142839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 0x7f7ed40c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.215+0000 7f7ed959d700 1 -- 172.21.15.119:0/3260142839 >> 172.21.15.119:0/3260142839 conn(0x7f7ed40cc1f0 msgr2=0x7f7ed40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.215+0000 7f7ed959d700 1 -- 172.21.15.119:0/3260142839 shutdown_connections 2023-10-08T00:56:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.215+0000 7f7ed959d700 1 -- 172.21.15.119:0/3260142839 wait complete. 2023-10-08T00:56:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.217+0000 7f7ed959d700 1 Processor -- start 2023-10-08T00:56:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.217+0000 7f7ed959d700 1 -- start start 2023-10-08T00:56:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed959d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed414beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed959d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 0x7f7ed4154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed959d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed4157770 0x7f7ed40c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed959d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed4155ab0 con 0x7f7ed406e730 2023-10-08T00:56:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed959d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed41557d0 con 0x7f7ed4157770 2023-10-08T00:56:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed959d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed40c1130 con 0x7f7ed40c6d00 2023-10-08T00:56:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed414beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed414beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46326/0 (socket says 172.21.15.119:46326) 2023-10-08T00:56:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed3fff700 1 -- 172.21.15.119:0/1521217539 learned_addr learned my addr 172.21.15.119:0/1521217539 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed37fe700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 0x7f7ed4154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed8d9c700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed4157770 0x7f7ed40c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed3fff700 1 -- 172.21.15.119:0/1521217539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 msgr2=0x7f7ed4154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed3fff700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 0x7f7ed4154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed3fff700 1 -- 172.21.15.119:0/1521217539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed4157770 msgr2=0x7f7ed40c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed3fff700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed4157770 0x7f7ed40c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.218+0000 7f7ed3fff700 1 -- 172.21.15.119:0/1521217539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc009a20 con 0x7f7ed406e730 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.219+0000 7f7ed3fff700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed414beb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c5b0 tx=0x7f7ec4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.219+0000 7f7ebbfff700 1 -- 172.21.15.119:0/1521217539 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4005b30 con 0x7f7ed406e730 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.219+0000 7f7ebbfff700 1 -- 172.21.15.119:0/1521217539 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec4005c90 con 0x7f7ed406e730 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.219+0000 7f7ebbfff700 1 -- 172.21.15.119:0/1521217539 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4007300 con 0x7f7ed406e730 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.219+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed40c1410 con 0x7f7ed406e730 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.219+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed414c3f0 con 0x7f7ed406e730 2023-10-08T00:56:27.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.220+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed40c0f90 con 0x7f7ed406e730 2023-10-08T00:56:27.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.221+0000 7f7ebbfff700 1 -- 172.21.15.119:0/1521217539 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ec40076b0 con 0x7f7ed406e730 2023-10-08T00:56:27.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.222+0000 7f7ebbfff700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec0066f30 0x7f7ec00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.222+0000 7f7ebbfff700 1 -- 172.21.15.119:0/1521217539 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f7ec40839d0 con 0x7f7ed406e730 2023-10-08T00:56:27.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.222+0000 7f7ed37fe700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec0066f30 0x7f7ec00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.223+0000 7f7ed37fe700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec0066f30 0x7f7ec00693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40c2250 tx=0x7f7ec8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:27.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.226+0000 7f7ebbfff700 1 -- 172.21.15.119:0/1521217539 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ed40c0f90 con 0x7f7ed406e730 2023-10-08T00:56:27.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.417+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ed40c0f90 con 0x7f7ed406e730 2023-10-08T00:56:27.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.418+0000 7f7ebbfff700 1 -- 172.21.15.119:0/1521217539 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+15398 (secure 0 0 0) 0x7f7ed40c0f90 con 0x7f7ed406e730 2023-10-08T00:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:27.420 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:55:48.498333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:55:45.270364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-08T00:55:45.937180+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:27.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec0066f30 msgr2=0x7f7ec00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec0066f30 0x7f7ec00693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7ed40c2250 tx=0x7f7ec8008040 comp rx=0 tx=0).stop 2023-10-08T00:56:27.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 msgr2=0x7f7ed414beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed414beb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7ec400c5b0 tx=0x7f7ec4001070 comp rx=0 tx=0).stop 2023-10-08T00:56:27.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 shutdown_connections 2023-10-08T00:56:27.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7ec0066f30 0x7f7ec00693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ed4157770 0x7f7ed40c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ed406e730 0x7f7ed414beb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 --2- 172.21.15.119:0/1521217539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ed40c6d00 0x7f7ed4154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 >> 172.21.15.119:0/1521217539 conn(0x7f7ed40cc1f0 msgr2=0x7f7ed414ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:27.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 shutdown_connections 2023-10-08T00:56:27.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.421+0000 7f7ed959d700 1 -- 172.21.15.119:0/1521217539 wait complete. 2023-10-08T00:56:27.440 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T00:56:27.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.624+0000 7fcd13fff700 1 Processor -- start 2023-10-08T00:56:27.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.624+0000 7fcd13fff700 1 -- start start 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.625+0000 7fcd13fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcd14150550 0x7fcd14157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.625+0000 7fcd13fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd140c7410 0x7fcd14069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.625+0000 7fcd13fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1406e2a0 con 0x7fcd14150180 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.625+0000 7fcd13fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1406b2e0 con 0x7fcd14150550 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.625+0000 7fcd13fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140c78e0 con 0x7fcd140c7410 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.625+0000 7fcd12ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcd14150180 0x7fcd140c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45136/0 (socket says 172.21.15.119:45136) 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.625+0000 7fcd12ffd700 1 -- 172.21.15.119:0/685028143 learned_addr learned my addr 172.21.15.119:0/685028143 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd137fe700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd140c7410 0x7fcd14069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd127fc700 1 --2- 172.21.15.119:0/685028143 >> v2:172.21.15.184:3300/0 conn(0x7fcd14150550 0x7fcd14157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3608813226 0 0) 0x7fcd1406e2a0 con 0x7fcd14150180 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccfc003570 con 0x7fcd14150180 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4100146279 0 0) 0x7fccfc003570 con 0x7fcd14150180 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd140c7410 msgr2=0x7fcd14069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd140c7410 0x7fcd14069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 >> v2:172.21.15.184:3300/0 conn(0x7fcd14150550 msgr2=0x7fcd14157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 --2- 172.21.15.119:0/685028143 >> v2:172.21.15.184:3300/0 conn(0x7fcd14150550 0x7fcd14157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd140c7bc0 con 0x7fcd14150180 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fcd04001470 con 0x7fcd14150180 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 >> v1:172.21.15.119:6789/0 conn(0x7fcd14150180 legacy=0x7fcd140c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd140c7410 0x7fccfc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc003fe0 0x7fccfc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc004d50 0x7fccfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1406b2e0 con 0x7fccfc003fe0 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140c78e0 con 0x7fccfc004d50 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.626+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccfc00dca0 con 0x7fcd140c7410 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd127fc700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc003fe0 0x7fccfc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd12ffd700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd140c7410 0x7fccfc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd137fe700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc004d50 0x7fccfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd127fc700 1 -- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd140c7410 msgr2=0x7fccfc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd127fc700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd140c7410 0x7fccfc003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd127fc700 1 -- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc004d50 msgr2=0x7fccfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd127fc700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccfc004d50 0x7fccfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd127fc700 1 -- 172.21.15.119:0/685028143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc003570 con 0x7fccfc003fe0 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd127fc700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc003fe0 0x7fccfc004810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800acc0 tx=0x7fcd0800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f9434a557d9aa2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800e720 con 0x7fccfc003fe0 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0800e880 con 0x7fccfc003fe0 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.627+0000 7fcd11ffb700 1 -- 172.21.15.119:0/685028143 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0800ea10 con 0x7fccfc003fe0 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.628+0000 7fcd13fff700 1 -- 172.21.15.119:0/685028143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc003fe0 msgr2=0x7fccfc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.628+0000 7fcd13fff700 1 --2- 172.21.15.119:0/685028143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccfc003fe0 0x7fccfc004810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800acc0 tx=0x7fcd0800a330 comp rx=0 tx=0).stop 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.628+0000 7fcd137fe700 1 -- 172.21.15.119:0/685028143 reap_dead start 2023-10-08T00:56:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.628+0000 7fcd13fff700 1 -- 172.21.15.119:0/685028143 shutdown_connections 2023-10-08T00:56:27.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.628+0000 7fcd13fff700 1 -- 172.21.15.119:0/685028143 >> 172.21.15.119:0/685028143 conn(0x7fcd140cc1f0 msgr2=0x7fcd140cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:27.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.628+0000 7fcd13fff700 1 -- 172.21.15.119:0/685028143 shutdown_connections 2023-10-08T00:56:27.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.628+0000 7fcd13fff700 1 -- 172.21.15.119:0/685028143 wait complete. 2023-10-08T00:56:27.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.629+0000 7fcd13fff700 1 Processor -- start 2023-10-08T00:56:27.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.630+0000 7fcd13fff700 1 -- start start 2023-10-08T00:56:27.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.630+0000 7fcd13fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc003fe0 0x7fcd1414bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.630+0000 7fcd13fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd140ad440 0x7fcd140b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.630+0000 7fcd13fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd1416d320 0x7fcd1416d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd13fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1414c1e0 con 0x7fcd140ad440 2023-10-08T00:56:27.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd13fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd140b1e20 con 0x7fcd1416d320 2023-10-08T00:56:27.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd13fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1416dcd0 con 0x7fccfc003fe0 2023-10-08T00:56:27.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd140ad440 0x7fcd140b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd140ad440 0x7fcd140b18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46338/0 (socket says 172.21.15.119:46338) 2023-10-08T00:56:27.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 -- 172.21.15.119:0/2417434766 learned_addr learned my addr 172.21.15.119:0/2417434766 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:27.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd137fe700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd1416d320 0x7fcd1416d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd12ffd700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc003fe0 0x7fcd1414bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 -- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc003fe0 msgr2=0x7fcd1414bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc003fe0 0x7fcd1414bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 -- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd1416d320 msgr2=0x7fcd1416d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd1416d320 0x7fcd1416d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 -- 172.21.15.119:0/2417434766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1414cf60 con 0x7fcd140ad440 2023-10-08T00:56:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd137fe700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd1416d320 0x7fcd1416d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.631+0000 7fcd127fc700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd140ad440 0x7fcd140b18e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800afd0 tx=0x7fcd08019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.632+0000 7fccfaffd700 1 -- 172.21.15.119:0/2417434766 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08021070 con 0x7fcd140ad440 2023-10-08T00:56:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.632+0000 7fccfaffd700 1 -- 172.21.15.119:0/2417434766 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0800ed80 con 0x7fcd140ad440 2023-10-08T00:56:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.632+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd140ad940 con 0x7fcd140ad440 2023-10-08T00:56:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.632+0000 7fccfaffd700 1 -- 172.21.15.119:0/2417434766 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0801c260 con 0x7fcd140ad440 2023-10-08T00:56:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.632+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd140adf00 con 0x7fcd140ad440 2023-10-08T00:56:27.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.633+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd140c87a0 con 0x7fcd140ad440 2023-10-08T00:56:27.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.634+0000 7fccfaffd700 1 -- 172.21.15.119:0/2417434766 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcd0801a640 con 0x7fcd140ad440 2023-10-08T00:56:27.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.634+0000 7fccfaffd700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcd00066f30 0x7fcd000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:27.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.634+0000 7fccfaffd700 1 -- 172.21.15.119:0/2417434766 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fcd08015030 con 0x7fcd140ad440 2023-10-08T00:56:27.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.635+0000 7fcd12ffd700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcd00066f30 0x7fcd000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:27.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.635+0000 7fcd12ffd700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcd00066f30 0x7fcd000693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd04001120 tx=0x7fcd04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:27.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.639+0000 7fccfaffd700 1 -- 172.21.15.119:0/2417434766 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd08052d60 con 0x7fcd140ad440 2023-10-08T00:56:27.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.870+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcd140cd160 con 0x7fcd140ad440 2023-10-08T00:56:27.871 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T00:56:27.870+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T00:56:27.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.877+0000 7fccfaffd700 1 -- 172.21.15.119:0/2417434766 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcd08056380 con 0x7fcd140ad440 2023-10-08T00:56:27.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcd00066f30 msgr2=0x7fcd000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcd00066f30 0x7fcd000693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fcd04001120 tx=0x7fcd04009040 comp rx=0 tx=0).stop 2023-10-08T00:56:27.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd140ad440 msgr2=0x7fcd140b18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:27.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd140ad440 0x7fcd140b18e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800afd0 tx=0x7fcd08019000 comp rx=0 tx=0).stop 2023-10-08T00:56:27.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 shutdown_connections 2023-10-08T00:56:27.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccfc003fe0 0x7fcd1414bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcd00066f30 0x7fcd000693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd1416d320 0x7fcd1416d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 --2- 172.21.15.119:0/2417434766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd140ad440 0x7fcd140b18e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:27.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 >> 172.21.15.119:0/2417434766 conn(0x7fcd140cc1f0 msgr2=0x7fcd1414eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:27.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 shutdown_connections 2023-10-08T00:56:27.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:27.879+0000 7fcd13fff700 1 -- 172.21.15.119:0/2417434766 wait complete. 2023-10-08T00:56:27.882 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T00:56:27.883 INFO:tasks.ceph.mds.b.smithi184.stderr: -2023> 2023-10-08T00:55:48.473+0000 7f5aec9f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:56:27.898 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T00:56:27.928 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T00:56:28.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.082+0000 7f90385ff700 1 Processor -- start 2023-10-08T00:56:28.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.082+0000 7f90385ff700 1 -- start start 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.083+0000 7f90385ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f90300c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.083+0000 7f90385ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f90301568e0 0x7f90300ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.083+0000 7f90385ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903015b0d0 con 0x7f90300ccee0 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.083+0000 7f90385ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90300c7a60 con 0x7f90301568e0 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.083+0000 7f90385ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903015a4c0 con 0x7f90300c83f0 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f9035b9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f90300ccee0 0x7f90301561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45148/0 (socket says 172.21.15.119:45148) 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f9035b9a700 1 -- 172.21.15.119:0/298520515 learned_addr learned my addr 172.21.15.119:0/298520515 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f903639b700 1 --2- 172.21.15.119:0/298520515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f90300c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f9036b9c700 1 --2- 172.21.15.119:0/298520515 >> v2:172.21.15.184:3300/0 conn(0x7f90301568e0 0x7f90300ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f9035399700 1 -- 172.21.15.119:0/298520515 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563068242 0 0) 0x7f903015b0d0 con 0x7f90300ccee0 2023-10-08T00:56:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f9035399700 1 -- 172.21.15.119:0/298520515 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f901c003570 con 0x7f90300ccee0 2023-10-08T00:56:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f903639b700 1 -- 172.21.15.119:0/298520515 >> v2:172.21.15.184:3300/0 conn(0x7f90301568e0 msgr2=0x7f90300ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f903639b700 1 --2- 172.21.15.119:0/298520515 >> v2:172.21.15.184:3300/0 conn(0x7f90301568e0 0x7f90300ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f903639b700 1 -- 172.21.15.119:0/298520515 >> v1:172.21.15.119:6789/0 conn(0x7f90300ccee0 legacy=0x7f90301561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.084+0000 7f903639b700 1 -- 172.21.15.119:0/298520515 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903015a7a0 con 0x7f90300c83f0 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.085+0000 7f903639b700 1 --2- 172.21.15.119:0/298520515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f90300c7520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f902c0098e0 tx=0x7f902c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=427f8038777944a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.085+0000 7f9035399700 1 -- 172.21.15.119:0/298520515 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011630 con 0x7f90300c83f0 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.085+0000 7f9035399700 1 -- 172.21.15.119:0/298520515 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c011790 con 0x7f90300c83f0 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.085+0000 7f9035399700 1 -- 172.21.15.119:0/298520515 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011b30 con 0x7f90300c83f0 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.085+0000 7f90385ff700 1 -- 172.21.15.119:0/298520515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 msgr2=0x7f90300c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.085+0000 7f90385ff700 1 --2- 172.21.15.119:0/298520515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f90300c7520 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f902c0098e0 tx=0x7f902c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.086+0000 7f90385ff700 1 -- 172.21.15.119:0/298520515 shutdown_connections 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.086+0000 7f90385ff700 1 --2- 172.21.15.119:0/298520515 >> v2:172.21.15.184:3300/0 conn(0x7f90301568e0 0x7f90300ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.086+0000 7f90385ff700 1 --2- 172.21.15.119:0/298520515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f90300c7520 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.086+0000 7f90385ff700 1 -- 172.21.15.119:0/298520515 >> 172.21.15.119:0/298520515 conn(0x7f9030068e10 msgr2=0x7f90300cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.086+0000 7f90385ff700 1 -- 172.21.15.119:0/298520515 shutdown_connections 2023-10-08T00:56:28.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.086+0000 7f90385ff700 1 -- 172.21.15.119:0/298520515 wait complete. 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.087+0000 7f90385ff700 1 Processor -- start 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.087+0000 7f90385ff700 1 -- start start 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f90385ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f903014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f90385ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90300ccee0 0x7f9030150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f90385ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90301568e0 0x7f90301554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f90385ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903014eb10 con 0x7f90300ccee0 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f90385ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9030169330 con 0x7f90301568e0 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f90385ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9030166230 con 0x7f90300c83f0 2023-10-08T00:56:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f9035b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90300ccee0 0x7f9030150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f9035b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90300ccee0 0x7f9030150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46354/0 (socket says 172.21.15.119:46354) 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f9035b9a700 1 -- 172.21.15.119:0/2497087401 learned_addr learned my addr 172.21.15.119:0/2497087401 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f903639b700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f903014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.088+0000 7f9036b9c700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90301568e0 0x7f90301554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f9035b9a700 1 -- 172.21.15.119:0/2497087401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 msgr2=0x7f903014e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f9035b9a700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f903014e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f9035b9a700 1 -- 172.21.15.119:0/2497087401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90301568e0 msgr2=0x7f90301554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f9035b9a700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90301568e0 0x7f90301554d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f9035b9a700 1 -- 172.21.15.119:0/2497087401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c009420 con 0x7f90300ccee0 2023-10-08T00:56:28.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f9035b9a700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90300ccee0 0x7f9030150930 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f902000c7c0 tx=0x7f9020007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:28.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f90267fc700 1 -- 172.21.15.119:0/2497087401 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020005cd0 con 0x7f90300ccee0 2023-10-08T00:56:28.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f90267fc700 1 -- 172.21.15.119:0/2497087401 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9020005e30 con 0x7f90300ccee0 2023-10-08T00:56:28.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9030166510 con 0x7f90300ccee0 2023-10-08T00:56:28.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.089+0000 7f90267fc700 1 -- 172.21.15.119:0/2497087401 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9020007320 con 0x7f90300ccee0 2023-10-08T00:56:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.090+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9030166ad0 con 0x7f90300ccee0 2023-10-08T00:56:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.090+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90301694c0 con 0x7f90300ccee0 2023-10-08T00:56:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.091+0000 7f90267fc700 1 -- 172.21.15.119:0/2497087401 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f902001b3b0 con 0x7f90300ccee0 2023-10-08T00:56:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.092+0000 7f90267fc700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9014066f30 0x7f90140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.092+0000 7f90267fc700 1 -- 172.21.15.119:0/2497087401 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f9020083cc0 con 0x7f90300ccee0 2023-10-08T00:56:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.092+0000 7f903639b700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9014066f30 0x7f90140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.093+0000 7f903639b700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9014066f30 0x7f90140693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f902c0091e0 tx=0x7f902c017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.096+0000 7f90267fc700 1 -- 172.21.15.119:0/2497087401 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f902008a080 con 0x7f90300ccee0 2023-10-08T00:56:28.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.329+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f903004c3c0 con 0x7f90300ccee0 2023-10-08T00:56:28.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.877+0000 7f90267fc700 1 -- 172.21.15.119:0/2497087401 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f90200170a0 con 0x7f90300ccee0 2023-10-08T00:56:28.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.879+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9014066f30 msgr2=0x7f90140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:28.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.879+0000 7f90385ff700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9014066f30 0x7f90140693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f902c0091e0 tx=0x7f902c017000 comp rx=0 tx=0).stop 2023-10-08T00:56:28.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.879+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90300ccee0 msgr2=0x7f9030150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:28.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.879+0000 7f90385ff700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90300ccee0 0x7f9030150930 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f902000c7c0 tx=0x7f9020007e00 comp rx=0 tx=0).stop 2023-10-08T00:56:28.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.879+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 shutdown_connections 2023-10-08T00:56:28.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.879+0000 7f90385ff700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90300c83f0 0x7f903014e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.879+0000 7f90385ff700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9014066f30 0x7f90140693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.880+0000 7f90385ff700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90301568e0 0x7f90301554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.880+0000 7f90385ff700 1 --2- 172.21.15.119:0/2497087401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90300ccee0 0x7f9030150930 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:28.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.880+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 >> 172.21.15.119:0/2497087401 conn(0x7f9030068e10 msgr2=0x7f90300cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:28.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.880+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 shutdown_connections 2023-10-08T00:56:28.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:28.880+0000 7f90385ff700 1 -- 172.21.15.119:0/2497087401 wait complete. 2023-10-08T00:56:28.900 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T00:56:29.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.085+0000 7f5dd9b9d700 1 Processor -- start 2023-10-08T00:56:29.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.085+0000 7f5dd9b9d700 1 -- start start 2023-10-08T00:56:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.086+0000 7f5dd9b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 0x7f5dd40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.086+0000 7f5dd9b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dd40b0790 0x7f5dd40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.086+0000 7f5dd9b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5dd41591e0 0x7f5dd4159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:29.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.086+0000 7f5dd9b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd40cfa00 con 0x7f5dd40b0790 2023-10-08T00:56:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.086+0000 7f5dd9b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd40cb580 con 0x7f5dd41591e0 2023-10-08T00:56:29.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.086+0000 7f5dd9b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40b2de0 con 0x7f5dd40b03c0 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.086+0000 7f5dd3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dd40b0790 0x7f5dd40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5dd40b0790 0x7f5dd40b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46360/0 (socket says 172.21.15.119:46360) 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd3fff700 1 -- 172.21.15.119:0/3740475377 learned_addr learned my addr 172.21.15.119:0/3740475377 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd939c700 1 --2- 172.21.15.119:0/3740475377 >> v2:172.21.15.184:3300/0 conn(0x7f5dd41591e0 0x7f5dd4159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd3fff700 1 -- 172.21.15.119:0/3740475377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 msgr2=0x7f5dd40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd8b9b700 1 --2- 172.21.15.119:0/3740475377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 0x7f5dd40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd3fff700 1 --2- 172.21.15.119:0/3740475377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 0x7f5dd40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd3fff700 1 -- 172.21.15.119:0/3740475377 >> v2:172.21.15.184:3300/0 conn(0x7f5dd41591e0 msgr2=0x7f5dd4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd3fff700 1 --2- 172.21.15.119:0/3740475377 >> v2:172.21.15.184:3300/0 conn(0x7f5dd41591e0 0x7f5dd4159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:29.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd3fff700 1 -- 172.21.15.119:0/3740475377 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40cf160 con 0x7f5dd40b0790 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd939c700 1 --2- 172.21.15.119:0/3740475377 >> v2:172.21.15.184:3300/0 conn(0x7f5dd41591e0 0x7f5dd4159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd3fff700 1 --2- 172.21.15.119:0/3740475377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40b28a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40cb4f0 tx=0x7f5dc0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165a767540c4b70c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd37fe700 1 -- 172.21.15.119:0/3740475377 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000b860 con 0x7f5dd40b0790 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd37fe700 1 -- 172.21.15.119:0/3740475377 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc000b9c0 con 0x7f5dd40b0790 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.087+0000 7f5dd37fe700 1 -- 172.21.15.119:0/3740475377 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc000bd60 con 0x7f5dd40b0790 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/3740475377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 msgr2=0x7f5dd40b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/3740475377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40b28a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40cb4f0 tx=0x7f5dc0009250 comp rx=0 tx=0).stop 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/3740475377 shutdown_connections 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/3740475377 >> v2:172.21.15.184:3300/0 conn(0x7f5dd41591e0 0x7f5dd4159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/3740475377 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 0x7f5dd40cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/3740475377 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40b28a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:29.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/3740475377 >> 172.21.15.119:0/3740475377 conn(0x7f5dd4063230 msgr2=0x7f5dd4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:29.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/3740475377 shutdown_connections 2023-10-08T00:56:29.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.088+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/3740475377 wait complete. 2023-10-08T00:56:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.089+0000 7f5dd9b9d700 1 Processor -- start 2023-10-08T00:56:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.090+0000 7f5dd9b9d700 1 -- start start 2023-10-08T00:56:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.090+0000 7f5dd9b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 0x7f5dd40c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.090+0000 7f5dd9b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd9b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dd41591e0 0x7f5dd41574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd9b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40c5250 con 0x7f5dd40b0790 2023-10-08T00:56:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd9b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40c4640 con 0x7f5dd41591e0 2023-10-08T00:56:29.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd9b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40c4920 con 0x7f5dd40b03c0 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40c6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46366/0 (socket says 172.21.15.119:46366) 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd3fff700 1 -- 172.21.15.119:0/1319049866 learned_addr learned my addr 172.21.15.119:0/1319049866 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd8b9b700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 0x7f5dd40c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd939c700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dd41591e0 0x7f5dd41574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd3fff700 1 -- 172.21.15.119:0/1319049866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 msgr2=0x7f5dd40c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd3fff700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 0x7f5dd40c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd3fff700 1 -- 172.21.15.119:0/1319049866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dd41591e0 msgr2=0x7f5dd41574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd3fff700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dd41591e0 0x7f5dd41574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.091+0000 7f5dd3fff700 1 -- 172.21.15.119:0/1319049866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc00094f0 con 0x7f5dd40b0790 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.092+0000 7f5dd3fff700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40c6a50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40cfe40 tx=0x7f5dc0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:29.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.092+0000 7f5dd0ff9700 1 -- 172.21.15.119:0/1319049866 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0011e70 con 0x7f5dd40b0790 2023-10-08T00:56:29.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.092+0000 7f5dd0ff9700 1 -- 172.21.15.119:0/1319049866 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0024440 con 0x7f5dd40b0790 2023-10-08T00:56:29.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.092+0000 7f5dd0ff9700 1 -- 172.21.15.119:0/1319049866 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0024800 con 0x7f5dd40b0790 2023-10-08T00:56:29.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.092+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd40c4c00 con 0x7f5dd40b0790 2023-10-08T00:56:29.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.092+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd40c6f90 con 0x7f5dd40b0790 2023-10-08T00:56:29.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.093+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd404c3c0 con 0x7f5dd40b0790 2023-10-08T00:56:29.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.094+0000 7f5dd0ff9700 1 -- 172.21.15.119:0/1319049866 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5dc001a420 con 0x7f5dd40b0790 2023-10-08T00:56:29.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.095+0000 7f5dd0ff9700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5db8066f30 0x7f5db80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:29.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.095+0000 7f5dd0ff9700 1 -- 172.21.15.119:0/1319049866 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5791+0+0 (secure 0 0 0) 0x7f5dc0086490 con 0x7f5dd40b0790 2023-10-08T00:56:29.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.095+0000 7f5dd8b9b700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5db8066f30 0x7f5db80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:29.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.096+0000 7f5dd8b9b700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5db8066f30 0x7f5db80693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:29.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.100+0000 7f5dd0ff9700 1 -- 172.21.15.119:0/1319049866 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dc0053730 con 0x7f5dd40b0790 2023-10-08T00:56:29.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:29.290+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5dd40cad50 con 0x7f5dd40b0790 2023-10-08T00:56:30.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.068+0000 7f5dd0ff9700 1 -- 172.21.15.119:0/1319049866 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v22) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5dc0056d50 con 0x7f5dd40b0790 2023-10-08T00:56:30.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.075+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5db8066f30 msgr2=0x7f5db80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:30.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.075+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5db8066f30 0x7f5db80693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8008040 comp rx=0 tx=0).stop 2023-10-08T00:56:30.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.075+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 msgr2=0x7f5dd40c6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:30.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.075+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40c6a50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5dd40cfe40 tx=0x7f5dc0017000 comp rx=0 tx=0).stop 2023-10-08T00:56:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.076+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 shutdown_connections 2023-10-08T00:56:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.076+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dd40b03c0 0x7f5dd40c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.076+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5db8066f30 0x7f5db80693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.076+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dd41591e0 0x7f5dd41574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.076+0000 7f5dd9b9d700 1 --2- 172.21.15.119:0/1319049866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dd40b0790 0x7f5dd40c6a50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.076+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 >> 172.21.15.119:0/1319049866 conn(0x7f5dd4063230 msgr2=0x7f5dd40b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.076+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 shutdown_connections 2023-10-08T00:56:30.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.076+0000 7f5dd9b9d700 1 -- 172.21.15.119:0/1319049866 wait complete. 2023-10-08T00:56:30.082 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T00:56:30.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T00:56:30.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.291+0000 7fe5fe79d700 1 Processor -- start 2023-10-08T00:56:30.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.291+0000 7fe5fe79d700 1 -- start start 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.292+0000 7fe5fe79d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5f80b03c0 0x7fe5f80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.292+0000 7fe5fe79d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5f81591e0 0x7fe5f8159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.292+0000 7fe5fe79d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f80cfa00 con 0x7fe5f80b0790 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.292+0000 7fe5fe79d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f80cb580 con 0x7fe5f81591e0 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.292+0000 7fe5fe79d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f80cf160 con 0x7fe5f80b03c0 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.292+0000 7fe5fdf9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5f81591e0 0x7fe5f8159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fdf9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5f81591e0 0x7fe5f8159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46374/0 (socket says 172.21.15.119:46374) 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fcf9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe5f80b0790 0x7fe5f80b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45154/0 (socket says 172.21.15.119:45154) 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fdf9c700 1 -- 172.21.15.119:0/3858765899 learned_addr learned my addr 172.21.15.119:0/3858765899 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:30.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fd79b700 1 --2- 172.21.15.119:0/3858765899 >> v2:172.21.15.184:3300/0 conn(0x7fe5f80b03c0 0x7fe5f80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fdf9c700 1 -- 172.21.15.119:0/3858765899 >> v2:172.21.15.184:3300/0 conn(0x7fe5f80b03c0 msgr2=0x7fe5f80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fdf9c700 1 --2- 172.21.15.119:0/3858765899 >> v2:172.21.15.184:3300/0 conn(0x7fe5f80b03c0 0x7fe5f80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fdf9c700 1 -- 172.21.15.119:0/3858765899 >> v1:172.21.15.119:6789/0 conn(0x7fe5f80b0790 legacy=0x7fe5f80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5effff700 1 -- 172.21.15.119:0/3858765899 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2053968521 0 0) 0x7fe5f80cfa00 con 0x7fe5f80b0790 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fdf9c700 1 -- 172.21.15.119:0/3858765899 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80cf440 con 0x7fe5f81591e0 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.293+0000 7fe5fdf9c700 1 --2- 172.21.15.119:0/3858765899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f81591e0 0x7fe5f8159600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009f20 tx=0x7fe5f0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eb507ba0348caea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5effff700 1 -- 172.21.15.119:0/3858765899 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000e070 con 0x7fe5f81591e0 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5effff700 1 -- 172.21.15.119:0/3858765899 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f000ba40 con 0x7fe5f81591e0 2023-10-08T00:56:30.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5effff700 1 -- 172.21.15.119:0/3858765899 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00092e0 con 0x7fe5f81591e0 2023-10-08T00:56:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3858765899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f81591e0 msgr2=0x7fe5f8159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3858765899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f81591e0 0x7fe5f8159600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0009f20 tx=0x7fe5f0009840 comp rx=0 tx=0).stop 2023-10-08T00:56:30.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3858765899 shutdown_connections 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3858765899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f81591e0 0x7fe5f8159600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3858765899 >> v2:172.21.15.184:3300/0 conn(0x7fe5f80b03c0 0x7fe5f80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3858765899 >> 172.21.15.119:0/3858765899 conn(0x7fe5f8063230 msgr2=0x7fe5f8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3858765899 shutdown_connections 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.294+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3858765899 wait complete. 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.296+0000 7fe5fe79d700 1 Processor -- start 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.296+0000 7fe5fe79d700 1 -- start start 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fe79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5f80b03c0 0x7fe5f8158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:30.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fe79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f80b0790 0x7fe5f80c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:30.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fe79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5f81591e0 0x7fe5f80c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:30.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fe79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f8158f70 con 0x7fe5f80b0790 2023-10-08T00:56:30.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fe79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815d0e0 con 0x7fe5f81591e0 2023-10-08T00:56:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fe79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815d3c0 con 0x7fe5f80b03c0 2023-10-08T00:56:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fcf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f80b0790 0x7fe5f80c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fcf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f80b0790 0x7fe5f80c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46382/0 (socket says 172.21.15.119:46382) 2023-10-08T00:56:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fcf9a700 1 -- 172.21.15.119:0/3790268110 learned_addr learned my addr 172.21.15.119:0/3790268110 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fdf9c700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5f81591e0 0x7fe5f80c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fd79b700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5f80b03c0 0x7fe5f8158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fcf9a700 1 -- 172.21.15.119:0/3790268110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5f80b03c0 msgr2=0x7fe5f8158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:30.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fcf9a700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5f80b03c0 0x7fe5f8158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fcf9a700 1 -- 172.21.15.119:0/3790268110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5f81591e0 msgr2=0x7fe5f80c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fcf9a700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5f81591e0 0x7fe5f80c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.297+0000 7fe5fcf9a700 1 -- 172.21.15.119:0/3790268110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f0009ae0 con 0x7fe5f80b0790 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.298+0000 7fe5fcf9a700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f80b0790 0x7fe5f80c8830 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80011e0 tx=0x7fe5e8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.298+0000 7fe5ed7fa700 1 -- 172.21.15.119:0/3790268110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8005a70 con 0x7fe5f80b0790 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.298+0000 7fe5ed7fa700 1 -- 172.21.15.119:0/3790268110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e8005bd0 con 0x7fe5f80b0790 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.298+0000 7fe5ed7fa700 1 -- 172.21.15.119:0/3790268110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e80070d0 con 0x7fe5f80b0790 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.298+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f815d6a0 con 0x7fe5f80b0790 2023-10-08T00:56:30.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.298+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f80c5660 con 0x7fe5f80b0790 2023-10-08T00:56:30.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.299+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f80cdce0 con 0x7fe5f80b0790 2023-10-08T00:56:30.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.300+0000 7fe5ed7fa700 1 -- 172.21.15.119:0/3790268110 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5e8007480 con 0x7fe5f80b0790 2023-10-08T00:56:30.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.301+0000 7fe5ed7fa700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe5dc066f30 0x7fe5dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:30.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.301+0000 7fe5ed7fa700 1 -- 172.21.15.119:0/3790268110 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5469+0+0 (secure 0 0 0) 0x7fe5e8083930 con 0x7fe5f80b0790 2023-10-08T00:56:30.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.301+0000 7fe5fd79b700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe5dc066f30 0x7fe5dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:30.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.301+0000 7fe5fd79b700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe5dc066f30 0x7fe5dc0693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4006fd0 tx=0x7fe5f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:30.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.304+0000 7fe5ed7fa700 1 -- 172.21.15.119:0/3790268110 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5e804b5a0 con 0x7fe5f80b0790 2023-10-08T00:56:30.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:30.496+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe5f804c3c0 con 0x7fe5f80b0790 2023-10-08T00:56:31.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.070+0000 7fe5ed7fa700 1 -- 172.21.15.119:0/3790268110 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe5e8017070 con 0x7fe5f80b0790 2023-10-08T00:56:31.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.075+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe5dc066f30 msgr2=0x7fe5dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:31.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe5dc066f30 0x7fe5dc0693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4006fd0 tx=0x7fe5f4008040 comp rx=0 tx=0).stop 2023-10-08T00:56:31.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f80b0790 msgr2=0x7fe5f80c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:31.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f80b0790 0x7fe5f80c8830 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe5e80011e0 tx=0x7fe5e8007e10 comp rx=0 tx=0).stop 2023-10-08T00:56:31.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 shutdown_connections 2023-10-08T00:56:31.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5f80b03c0 0x7fe5f8158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe5dc066f30 0x7fe5dc0693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5f81591e0 0x7fe5f80c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 --2- 172.21.15.119:0/3790268110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5f80b0790 0x7fe5f80c8830 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.076+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 >> 172.21.15.119:0/3790268110 conn(0x7fe5f8063230 msgr2=0x7fe5f80694c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:31.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.082+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 shutdown_connections 2023-10-08T00:56:31.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.082+0000 7fe5fe79d700 1 -- 172.21.15.119:0/3790268110 wait complete. 2023-10-08T00:56:31.086 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T00:56:31.105 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T00:56:31.105 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T00:56:31.105 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T00:56:31.106 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:56:31.106 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T00:56:31.106 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T00:56:31.107 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T00:56:31.107 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:56:31.107 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T00:56:31.107 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T00:56:31.108 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T00:56:31.108 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:56:31.108 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T00:56:31.109 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T00:56:31.109 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T00:56:31.109 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T00:56:31.120 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T00:56:31.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T00:56:31.122 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T00:56:31.122 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T00:56:31.125 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T00:56:31.125 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T00:56:31.126 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T00:56:31.126 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T00:56:31.165 INFO:tasks.ceph.mds.a:Started 2023-10-08T00:56:31.166 DEBUG:teuthology.parallel:result is None 2023-10-08T00:56:31.168 INFO:tasks.ceph.mds.c:Started 2023-10-08T00:56:31.168 DEBUG:teuthology.parallel:result is None 2023-10-08T00:56:31.170 INFO:tasks.ceph.mds.b:Started 2023-10-08T00:56:31.170 DEBUG:teuthology.parallel:result is None 2023-10-08T00:56:31.175 INFO:tasks.ceph.mds.d:Started 2023-10-08T00:56:31.175 DEBUG:teuthology.parallel:result is None 2023-10-08T00:56:31.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T00:56:31.301 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T00:56:31.301 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T00:56:31.306 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T00:56:31.306 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T00:56:31.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.368+0000 7ff3ad9ff700 1 Processor -- start 2023-10-08T00:56:31.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.368+0000 7ff3ad9ff700 1 -- start start 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.369+0000 7ff3ad9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.369+0000 7ff3ad9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff3a80ccf90 0x7ff3a80b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.369+0000 7ff3ad9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3a815a270 0x7ff3a80b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.369+0000 7ff3ad9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a8156bf0 con 0x7ff3a815a270 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.369+0000 7ff3ad9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3a80ae950 con 0x7ff3a80ccf90 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.369+0000 7ff3ad9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a80cc300 con 0x7ff3a80cbf30 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3a815a270 0x7ff3a80b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3a815a270 0x7ff3a80b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46432/0 (socket says 172.21.15.119:46432) 2023-10-08T00:56:31.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 -- 172.21.15.119:0/732854762 learned_addr learned my addr 172.21.15.119:0/732854762 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:31.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a6ffd700 1 --2- 172.21.15.119:0/732854762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:31.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a67fc700 1 --2- 172.21.15.119:0/732854762 >> v2:172.21.15.184:3300/0 conn(0x7ff3a80ccf90 0x7ff3a80b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:31.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 -- 172.21.15.119:0/732854762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 msgr2=0x7ff3a80ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:31.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 --2- 172.21.15.119:0/732854762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a80ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 -- 172.21.15.119:0/732854762 >> v2:172.21.15.184:3300/0 conn(0x7ff3a80ccf90 msgr2=0x7ff3a80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:31.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 --2- 172.21.15.119:0/732854762 >> v2:172.21.15.184:3300/0 conn(0x7ff3a80ccf90 0x7ff3a80b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 -- 172.21.15.119:0/732854762 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a80ad370 con 0x7ff3a815a270 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a67fc700 1 --2- 172.21.15.119:0/732854762 >> v2:172.21.15.184:3300/0 conn(0x7ff3a80ccf90 0x7ff3a80b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a77fe700 1 --2- 172.21.15.119:0/732854762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a80b1250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8064210 tx=0x7ff3a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab164313d8e10e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.370+0000 7ff3a5ffb700 1 -- 172.21.15.119:0/732854762 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a000e070 con 0x7ff3a815a270 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3a5ffb700 1 -- 172.21.15.119:0/732854762 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a000b930 con 0x7ff3a815a270 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3a5ffb700 1 -- 172.21.15.119:0/732854762 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00092e0 con 0x7ff3a815a270 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/732854762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 msgr2=0x7ff3a80b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/732854762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a80b1250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8064210 tx=0x7ff3a0009840 comp rx=0 tx=0).stop 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/732854762 shutdown_connections 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/732854762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a80b1250 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/732854762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a80ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/732854762 >> v2:172.21.15.184:3300/0 conn(0x7ff3a80ccf90 0x7ff3a80b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/732854762 >> 172.21.15.119:0/732854762 conn(0x7ff3a806a320 msgr2=0x7ff3a80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/732854762 shutdown_connections 2023-10-08T00:56:31.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.371+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/732854762 wait complete. 2023-10-08T00:56:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.373+0000 7ff3ad9ff700 1 Processor -- start 2023-10-08T00:56:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.373+0000 7ff3ad9ff700 1 -- start start 2023-10-08T00:56:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3ad9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a81529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3ad9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a80ccf90 0x7ff3a816a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3ad9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a8153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:31.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3ad9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a816d990 con 0x7ff3a815a270 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3ad9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a816a9c0 con 0x7ff3a80ccf90 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3ad9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a8153880 con 0x7ff3a80cbf30 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a8153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a8153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46448/0 (socket says 172.21.15.119:46448) 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 -- 172.21.15.119:0/2210925600 learned_addr learned my addr 172.21.15.119:0/2210925600 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a6ffd700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a81529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a67fc700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a80ccf90 0x7ff3a816a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 -- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 msgr2=0x7ff3a81529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a81529a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 -- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a80ccf90 msgr2=0x7ff3a816a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a80ccf90 0x7ff3a816a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:31.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 -- 172.21.15.119:0/2210925600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c009420 con 0x7ff3a815a270 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a6ffd700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a81529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a67fc700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a80ccf90 0x7ff3a816a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.374+0000 7ff3a77fe700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a8153b20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8064290 tx=0x7ff3a0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.375+0000 7ff38effd700 1 -- 172.21.15.119:0/2210925600 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0008c30 con 0x7ff3a815a270 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.375+0000 7ff38effd700 1 -- 172.21.15.119:0/2210925600 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0008d90 con 0x7ff3a815a270 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.375+0000 7ff38effd700 1 -- 172.21.15.119:0/2210925600 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a00266b0 con 0x7ff3a815a270 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.375+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a0009ae0 con 0x7ff3a815a270 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.375+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a816d060 con 0x7ff3a815a270 2023-10-08T00:56:31.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.376+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a804c3c0 con 0x7ff3a815a270 2023-10-08T00:56:31.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.377+0000 7ff38effd700 1 -- 172.21.15.119:0/2210925600 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3a0003040 con 0x7ff3a815a270 2023-10-08T00:56:31.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.378+0000 7ff38effd700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:31.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.378+0000 7ff38effd700 1 -- 172.21.15.119:0/2210925600 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4939+0+0 (secure 0 0 0) 0x7ff3a0085c00 con 0x7ff3a815a270 2023-10-08T00:56:31.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.378+0000 7ff3a6ffd700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:31.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.378+0000 7ff3a6ffd700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff394066f30 0x7ff3940693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff398006f00 tx=0x7ff398008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:31.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.381+0000 7ff38effd700 1 -- 172.21.15.119:0/2210925600 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3a0053180 con 0x7ff3a815a270 2023-10-08T00:56:31.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:31.584+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff3a806f8a0 con 0x7ff3a815a270 2023-10-08T00:56:32.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.091+0000 7ff38effd700 1 -- 172.21.15.119:0/2210925600 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v25) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3a0009940 con 0x7ff3a815a270 2023-10-08T00:56:32.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff394066f30 msgr2=0x7ff3940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff394066f30 0x7ff3940693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff398006f00 tx=0x7ff398008040 comp rx=0 tx=0).stop 2023-10-08T00:56:32.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 msgr2=0x7ff3a8153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a8153b20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8064290 tx=0x7ff3a0018000 comp rx=0 tx=0).stop 2023-10-08T00:56:32.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 shutdown_connections 2023-10-08T00:56:32.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3a80cbf30 0x7ff3a81529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3a815a270 0x7ff3a8153b20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 --2- 172.21.15.119:0/2210925600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3a80ccf90 0x7ff3a816a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 >> 172.21.15.119:0/2210925600 conn(0x7ff3a806a320 msgr2=0x7ff3a8155230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:32.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 shutdown_connections 2023-10-08T00:56:32.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.094+0000 7ff3ad9ff700 1 -- 172.21.15.119:0/2210925600 wait complete. 2023-10-08T00:56:32.097 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T00:56:32.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T00:56:32.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.298+0000 7fd15e19d700 1 Processor -- start 2023-10-08T00:56:32.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.299+0000 7fd15e19d700 1 -- start start 2023-10-08T00:56:32.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.300+0000 7fd15e19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1580ca660 0x7fd1581598e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.300+0000 7fd15e19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1580a9c20 con 0x7fd158159510 2023-10-08T00:56:32.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.300+0000 7fd15e19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1580696c0 con 0x7fd15806a3f0 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.300+0000 7fd15e19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15815e400 con 0x7fd1580ca660 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.300+0000 7fd15d19b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd158159510 0x7fd15806d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45174/0 (socket says 172.21.15.119:45174) 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.300+0000 7fd15d19b700 1 -- 172.21.15.119:0/1354751538 learned_addr learned my addr 172.21.15.119:0/1354751538 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.300+0000 7fd15c99a700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1580ca660 0x7fd1581598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2275182193 0 0) 0x7fd1580a9c20 con 0x7fd158159510 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd144003570 con 0x7fd158159510 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3765994906 0 0) 0x7fd1580696c0 con 0x7fd15806a3f0 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1580a9c20 con 0x7fd15806a3f0 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635198044 0 0) 0x7fd144003570 con 0x7fd158159510 2023-10-08T00:56:32.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1580ca660 msgr2=0x7fd1581598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1580ca660 0x7fd1581598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 >> v1:172.21.15.184:6789/0 conn(0x7fd15806a3f0 legacy=0x7fd15806a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158069f60 con 0x7fd158159510 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fd154001470 con 0x7fd158159510 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 >> v1:172.21.15.119:6789/0 conn(0x7fd158159510 legacy=0x7fd15806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd144003c20 0x7fd144004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd144004810 0x7fd144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1440051c0 0x7fd14400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15815e400 con 0x7fd144004810 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14400c0b0 con 0x7fd144003c20 2023-10-08T00:56:32.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14400c270 con 0x7fd1440051c0 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd15c99a700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1580ca660 0x7fd1581598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd15d19b700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd144003c20 0x7fd144004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd15c99a700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd144004810 0x7fd144004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.301+0000 7fd15d99c700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1440051c0 0x7fd14400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd15d19b700 1 -- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1440051c0 msgr2=0x7fd14400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd15d19b700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1440051c0 0x7fd14400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd15d19b700 1 -- 172.21.15.119:0/1354751538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd144004810 msgr2=0x7fd144004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd15d19b700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd144004810 0x7fd144004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd15d19b700 1 -- 172.21.15.119:0/1354751538 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd144003570 con 0x7fd144003c20 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd15c99a700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd144004810 0x7fd144004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:32.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd15d19b700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd144003c20 0x7fd144004010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c210 tx=0x7fd15806be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a7aa0be67ca5d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15400cc10 con 0x7fd144003c20 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15400cd70 con 0x7fd144003c20 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.302+0000 7fd14ffff700 1 -- 172.21.15.119:0/1354751538 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd154005b70 con 0x7fd144003c20 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.303+0000 7fd15e19d700 1 -- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd144003c20 msgr2=0x7fd144004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.303+0000 7fd15e19d700 1 --2- 172.21.15.119:0/1354751538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd144003c20 0x7fd144004010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c210 tx=0x7fd15806be30 comp rx=0 tx=0).stop 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.303+0000 7fd15d99c700 1 -- 172.21.15.119:0/1354751538 reap_dead start 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.303+0000 7fd15e19d700 1 -- 172.21.15.119:0/1354751538 shutdown_connections 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.303+0000 7fd15e19d700 1 -- 172.21.15.119:0/1354751538 >> 172.21.15.119:0/1354751538 conn(0x7fd1580cc1f0 msgr2=0x7fd1580cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.303+0000 7fd15e19d700 1 -- 172.21.15.119:0/1354751538 shutdown_connections 2023-10-08T00:56:32.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.303+0000 7fd15e19d700 1 -- 172.21.15.119:0/1354751538 wait complete. 2023-10-08T00:56:32.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.304+0000 7fd15e19d700 1 Processor -- start 2023-10-08T00:56:32.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.305+0000 7fd15e19d700 1 -- start start 2023-10-08T00:56:32.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.305+0000 7fd15e19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd144003c20 0x7fd158151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.305+0000 7fd15e19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1580b1af0 0x7fd1581525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.305+0000 7fd15e19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1580c68c0 0x7fd1580c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.305+0000 7fd15e19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1580c7270 con 0x7fd1580b1af0 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.305+0000 7fd15e19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd158151880 con 0x7fd1580c68c0 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.305+0000 7fd15e19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd158153980 con 0x7fd144003c20 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1580b1af0 0x7fd1581525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1580b1af0 0x7fd1581525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46478/0 (socket says 172.21.15.119:46478) 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 -- 172.21.15.119:0/3835904639 learned_addr learned my addr 172.21.15.119:0/3835904639 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15d99c700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1580c68c0 0x7fd1580c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15d19b700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd144003c20 0x7fd158151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 -- 172.21.15.119:0/3835904639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd144003c20 msgr2=0x7fd158151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd144003c20 0x7fd158151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 -- 172.21.15.119:0/3835904639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1580c68c0 msgr2=0x7fd1580c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1580c68c0 0x7fd1580c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 -- 172.21.15.119:0/3835904639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd158152b10 con 0x7fd1580b1af0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.306+0000 7fd15c99a700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1580b1af0 0x7fd1581525d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd148011e00 tx=0x7fd1480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.307+0000 7fd14d7fa700 1 -- 172.21.15.119:0/3835904639 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14800fc20 con 0x7fd1580b1af0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.307+0000 7fd14d7fa700 1 -- 172.21.15.119:0/3835904639 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14800fd80 con 0x7fd1580b1af0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.307+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd158152d70 con 0x7fd1580b1af0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.307+0000 7fd14d7fa700 1 -- 172.21.15.119:0/3835904639 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14801d6d0 con 0x7fd1580b1af0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.307+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd158153330 con 0x7fd1580b1af0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.308+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15804c3c0 con 0x7fd1580b1af0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.309+0000 7fd14d7fa700 1 -- 172.21.15.119:0/3835904639 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd14801da80 con 0x7fd1580b1af0 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.310+0000 7fd14d7fa700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd13c066f30 0x7fd13c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.310+0000 7fd14d7fa700 1 -- 172.21.15.119:0/3835904639 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd148018070 con 0x7fd1580b1af0 2023-10-08T00:56:32.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.310+0000 7fd15d19b700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd13c066f30 0x7fd13c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.311+0000 7fd15d19b700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd13c066f30 0x7fd13c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c210 tx=0x7fd154015000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:32.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.314+0000 7fd14d7fa700 1 -- 172.21.15.119:0/3835904639 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd148052700 con 0x7fd1580b1af0 2023-10-08T00:56:32.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.576+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd158153ac0 con 0x7fd1580b1af0 2023-10-08T00:56:32.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.577+0000 7fd14d7fa700 1 -- 172.21.15.119:0/3835904639 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd148022090 con 0x7fd1580b1af0 2023-10-08T00:56:32.578 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:32.579 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T00:56:32.579 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T00:56:32.579 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.579 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T00:56:32.579 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T00:56:32.579 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.579 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T00:56:32.579 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.580 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T00:56:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T00:56:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T00:56:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.581 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.582 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T00:56:32.582 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T00:56:32.582 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.582 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T00:56:32.582 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T00:56:32.582 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T00:56:32.583 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.583 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.583 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.583 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.583 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T00:56:32.583 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T00:56:32.583 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.584 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T00:56:32.584 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T00:56:32.584 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T00:56:32.584 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.584 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.585 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.585 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.585 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T00:56:32.585 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T00:56:32.585 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.585 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T00:56:32.585 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T00:56:32.586 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.587 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T00:56:32.588 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T00:56:32.589 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T00:56:32.590 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.591 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.592 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T00:56:32.593 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.594 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T00:56:32.595 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-mds", 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQD7/SFlbZ3YBxAAb4TQUJ7eHJZrL7ZuWq5VwA==", 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-mds" 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.596 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-mgr", 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQD7/SFlh6jYBxAAk6c4rYuLI9SYmn0fFdJQNg==", 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-osd", 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQD7/SFlxLHYBxAArhriiotalSztCRJxUkMTsA==", 2023-10-08T00:56:32.597 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-osd" 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rbd", 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQD7/SFlcP/YBxAAY2OSqkasOv5TEg1JNugOtg==", 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.598 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQD7/SFlRQjZBxAAwpFcM6L4KK8XuCb/yKDgTA==", 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rgw", 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQD7/SFlMBHZBxAASVqJEjlxd5fhq9zOzVJEgA==", 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.599 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:32.600 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T00:56:32.601 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T00:56:32.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd13c066f30 msgr2=0x7fd13c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd13c066f30 0x7fd13c0693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd15400c210 tx=0x7fd154015000 comp rx=0 tx=0).stop 2023-10-08T00:56:32.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1580b1af0 msgr2=0x7fd1581525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1580b1af0 0x7fd1581525d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd148011e00 tx=0x7fd1480091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 shutdown_connections 2023-10-08T00:56:32.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd144003c20 0x7fd158151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd13c066f30 0x7fd13c0693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1580c68c0 0x7fd1580c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 --2- 172.21.15.119:0/3835904639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1580b1af0 0x7fd1581525d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 >> 172.21.15.119:0/3835904639 conn(0x7fd1580cc1f0 msgr2=0x7fd158158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 shutdown_connections 2023-10-08T00:56:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.579+0000 7fd15e19d700 1 -- 172.21.15.119:0/3835904639 wait complete. 2023-10-08T00:56:32.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-08T00:56:32.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.784+0000 7f2aff5ff700 1 Processor -- start 2023-10-08T00:56:32.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.784+0000 7f2aff5ff700 1 -- start start 2023-10-08T00:56:32.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.785+0000 7f2aff5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af80ccf90 0x7f2af80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.785+0000 7f2aff5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af80cc300 con 0x7f2af80b1420 2023-10-08T00:56:32.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.785+0000 7f2aff5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af80ad370 con 0x7f2af80cbf30 2023-10-08T00:56:32.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.785+0000 7f2aff5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8156bf0 con 0x7f2af80ccf90 2023-10-08T00:56:32.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2afcb9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2af80b1420 0x7f2af80c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45176/0 (socket says 172.21.15.119:45176) 2023-10-08T00:56:32.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2afcb9a700 1 -- 172.21.15.119:0/3575624157 learned_addr learned my addr 172.21.15.119:0/3575624157 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:32.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2afdb9c700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af80ccf90 0x7f2af80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022175325 0 0) 0x7f2af80cc300 con 0x7f2af80b1420 2023-10-08T00:56:32.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae4003570 con 0x7f2af80b1420 2023-10-08T00:56:32.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043281606 0 0) 0x7f2af80ad370 con 0x7f2af80cbf30 2023-10-08T00:56:32.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af80cc300 con 0x7f2af80cbf30 2023-10-08T00:56:32.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 87908063 0 0) 0x7f2ae4003570 con 0x7f2af80b1420 2023-10-08T00:56:32.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af80ccf90 msgr2=0x7f2af80ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af80ccf90 0x7f2af80ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 >> v1:172.21.15.184:6789/0 conn(0x7f2af80cbf30 legacy=0x7f2af80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80ad650 con 0x7f2af80b1420 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f2ae8001470 con 0x7f2af80b1420 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.786+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 >> v1:172.21.15.119:6789/0 conn(0x7f2af80b1420 legacy=0x7f2af80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2aeffff700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae4003c20 0x7f2ae4004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2aeffff700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af80ccf90 0x7f2ae4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2aeffff700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2ae40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8156bf0 con 0x7f2ae4004d50 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae400dd50 con 0x7f2ae4003c20 2023-10-08T00:56:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae400df10 con 0x7f2af80ccf90 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afdb9c700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2ae40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afd39b700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae4003c20 0x7f2ae4004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afcb9a700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af80ccf90 0x7f2ae4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afdb9c700 1 -- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af80ccf90 msgr2=0x7f2ae4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afdb9c700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af80ccf90 0x7f2ae4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afdb9c700 1 -- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae4003c20 msgr2=0x7f2ae4004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afdb9c700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae4003c20 0x7f2ae4004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afdb9c700 1 -- 172.21.15.119:0/3575624157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae4003570 con 0x7f2ae4004d50 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.787+0000 7f2afd39b700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ae4003c20 0x7f2ae4004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2afdb9c700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2ae40051c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e210 tx=0x7f2af000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a12107ca7aad2a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000eac0 con 0x7f2ae4004d50 2023-10-08T00:56:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000f040 con 0x7f2ae4004d50 2023-10-08T00:56:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aeffff700 1 -- 172.21.15.119:0/3575624157 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000ee70 con 0x7f2ae4004d50 2023-10-08T00:56:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aff5ff700 1 -- 172.21.15.119:0/3575624157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 msgr2=0x7f2ae40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aff5ff700 1 --2- 172.21.15.119:0/3575624157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2ae40051c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e210 tx=0x7f2af000b040 comp rx=0 tx=0).stop 2023-10-08T00:56:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2afdb9c700 1 -- 172.21.15.119:0/3575624157 reap_dead start 2023-10-08T00:56:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aff5ff700 1 -- 172.21.15.119:0/3575624157 shutdown_connections 2023-10-08T00:56:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aff5ff700 1 -- 172.21.15.119:0/3575624157 >> 172.21.15.119:0/3575624157 conn(0x7f2af806a320 msgr2=0x7f2af80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aff5ff700 1 -- 172.21.15.119:0/3575624157 shutdown_connections 2023-10-08T00:56:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.788+0000 7f2aff5ff700 1 -- 172.21.15.119:0/3575624157 wait complete. 2023-10-08T00:56:32.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.790+0000 7f2aff5ff700 1 Processor -- start 2023-10-08T00:56:32.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.790+0000 7f2aff5ff700 1 -- start start 2023-10-08T00:56:32.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2aff5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2af8154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2aff5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af81578b0 0x7f2af80c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2aff5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af80c85a0 0x7f2af815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2aff5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8155810 con 0x7f2ae4004d50 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2aff5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80c8a10 con 0x7f2af80c85a0 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2aff5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af8155260 con 0x7f2af81578b0 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afd39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2af8154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afd39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2af8154d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46496/0 (socket says 172.21.15.119:46496) 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afd39b700 1 -- 172.21.15.119:0/4052234076 learned_addr learned my addr 172.21.15.119:0/4052234076 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afdb9c700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af80c85a0 0x7f2af815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afcb9a700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af81578b0 0x7f2af80c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afd39b700 1 -- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af81578b0 msgr2=0x7f2af80c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afd39b700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af81578b0 0x7f2af80c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afd39b700 1 -- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af80c85a0 msgr2=0x7f2af815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.791+0000 7f2afd39b700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af80c85a0 0x7f2af815ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.792+0000 7f2afd39b700 1 -- 172.21.15.119:0/4052234076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af815bb60 con 0x7f2ae4004d50 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.792+0000 7f2afdb9c700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af80c85a0 0x7f2af815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.792+0000 7f2afd39b700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2af8154d20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2af400f930 tx=0x7f2af4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.792+0000 7f2aed7fa700 1 -- 172.21.15.119:0/4052234076 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af4002240 con 0x7f2ae4004d50 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.792+0000 7f2aed7fa700 1 -- 172.21.15.119:0/4052234076 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af40023a0 con 0x7f2ae4004d50 2023-10-08T00:56:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.792+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af815b880 con 0x7f2ae4004d50 2023-10-08T00:56:32.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.792+0000 7f2aed7fa700 1 -- 172.21.15.119:0/4052234076 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40016b0 con 0x7f2ae4004d50 2023-10-08T00:56:32.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.792+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af81570b0 con 0x7f2ae4004d50 2023-10-08T00:56:32.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.793+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af804c3c0 con 0x7f2ae4004d50 2023-10-08T00:56:32.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.794+0000 7f2aed7fa700 1 -- 172.21.15.119:0/4052234076 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2af4001ae0 con 0x7f2ae4004d50 2023-10-08T00:56:32.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.795+0000 7f2aed7fa700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:32.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.795+0000 7f2aed7fa700 1 -- 172.21.15.119:0/4052234076 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f2af4084430 con 0x7f2ae4004d50 2023-10-08T00:56:32.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.795+0000 7f2afcb9a700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:32.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.796+0000 7f2afcb9a700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2adc066f30 0x7f2adc0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8001200 tx=0x7f2ae8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:32.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:32.799+0000 7f2aed7fa700 1 -- 172.21.15.119:0/4052234076 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2af40519f0 con 0x7f2ae4004d50 2023-10-08T00:56:33.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.066+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f2af80ad930 con 0x7f2ae4004d50 2023-10-08T00:56:33.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.070+0000 7f2aed7fa700 1 -- 172.21.15.119:0/4052234076 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2af4055010 con 0x7f2ae4004d50 2023-10-08T00:56:33.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.072+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2adc066f30 msgr2=0x7f2adc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.072+0000 7f2aff5ff700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2adc066f30 0x7f2adc0693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8001200 tx=0x7f2ae8009040 comp rx=0 tx=0).stop 2023-10-08T00:56:33.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.072+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 msgr2=0x7f2af8154d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.072+0000 7f2aff5ff700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2af8154d20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f2af400f930 tx=0x7f2af4007d30 comp rx=0 tx=0).stop 2023-10-08T00:56:33.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.073+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 shutdown_connections 2023-10-08T00:56:33.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.073+0000 7f2aff5ff700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2adc066f30 0x7f2adc0693e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.073+0000 7f2aff5ff700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2af80c85a0 0x7f2af815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.073+0000 7f2aff5ff700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4004d50 0x7f2af8154d20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.073+0000 7f2aff5ff700 1 --2- 172.21.15.119:0/4052234076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2af81578b0 0x7f2af80c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.073+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 >> 172.21.15.119:0/4052234076 conn(0x7f2af806a320 msgr2=0x7f2af8155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:33.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.073+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 shutdown_connections 2023-10-08T00:56:33.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.073+0000 7f2aff5ff700 1 -- 172.21.15.119:0/4052234076 wait complete. 2023-10-08T00:56:33.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-08T00:56:33.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.273+0000 7fd632f9d700 1 Processor -- start 2023-10-08T00:56:33.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.273+0000 7fd632f9d700 1 -- start start 2023-10-08T00:56:33.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.274+0000 7fd632f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd62c06d950 0x7fd62c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.274+0000 7fd632f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd62c06dd20 0x7fd62c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.274+0000 7fd632f9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62c06ae70 con 0x7fd62c06a0a0 2023-10-08T00:56:33.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.274+0000 7fd632f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c06a540 con 0x7fd62c06d950 2023-10-08T00:56:33.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.274+0000 7fd632f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c06a820 con 0x7fd62c06dd20 2023-10-08T00:56:33.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.274+0000 7fd631f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd62c06d950 0x7fd62c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.274+0000 7fd631f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd62c06d950 0x7fd62c0c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46500/0 (socket says 172.21.15.119:46500) 2023-10-08T00:56:33.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.274+0000 7fd631f9b700 1 -- 172.21.15.119:0/1283198935 learned_addr learned my addr 172.21.15.119:0/1283198935 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:33.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.275+0000 7fd63279c700 1 --2- 172.21.15.119:0/1283198935 >> v2:172.21.15.184:3300/0 conn(0x7fd62c06dd20 0x7fd62c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.275+0000 7fd631f9b700 1 -- 172.21.15.119:0/1283198935 >> v2:172.21.15.184:3300/0 conn(0x7fd62c06dd20 msgr2=0x7fd62c0c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:33.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.275+0000 7fd631f9b700 1 --2- 172.21.15.119:0/1283198935 >> v2:172.21.15.184:3300/0 conn(0x7fd62c06dd20 0x7fd62c0c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.275+0000 7fd631f9b700 1 -- 172.21.15.119:0/1283198935 >> v1:172.21.15.184:6789/0 conn(0x7fd62c06a0a0 legacy=0x7fd62c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:33.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.275+0000 7fd631f9b700 1 -- 172.21.15.119:0/1283198935 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c06ab00 con 0x7fd62c06d950 2023-10-08T00:56:33.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.275+0000 7fd631f9b700 1 --2- 172.21.15.119:0/1283198935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06d950 0x7fd62c0c8350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd62c06e420 tx=0x7fd628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=945c7cd4b69f784c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:33.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.275+0000 7fd630f99700 1 -- 172.21.15.119:0/1283198935 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62800b820 con 0x7fd62c06d950 2023-10-08T00:56:33.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd630f99700 1 -- 172.21.15.119:0/1283198935 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62800b980 con 0x7fd62c06d950 2023-10-08T00:56:33.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd630f99700 1 -- 172.21.15.119:0/1283198935 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62800bd20 con 0x7fd62c06d950 2023-10-08T00:56:33.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd632f9d700 1 -- 172.21.15.119:0/1283198935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06d950 msgr2=0x7fd62c0c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd632f9d700 1 --2- 172.21.15.119:0/1283198935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06d950 0x7fd62c0c8350 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd62c06e420 tx=0x7fd628009250 comp rx=0 tx=0).stop 2023-10-08T00:56:33.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd632f9d700 1 -- 172.21.15.119:0/1283198935 shutdown_connections 2023-10-08T00:56:33.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd632f9d700 1 --2- 172.21.15.119:0/1283198935 >> v2:172.21.15.184:3300/0 conn(0x7fd62c06dd20 0x7fd62c0c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd632f9d700 1 --2- 172.21.15.119:0/1283198935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06d950 0x7fd62c0c8350 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd632f9d700 1 -- 172.21.15.119:0/1283198935 >> 172.21.15.119:0/1283198935 conn(0x7fd62c0cc1f0 msgr2=0x7fd62c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:33.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd632f9d700 1 -- 172.21.15.119:0/1283198935 shutdown_connections 2023-10-08T00:56:33.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.276+0000 7fd632f9d700 1 -- 172.21.15.119:0/1283198935 wait complete. 2023-10-08T00:56:33.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.278+0000 7fd632f9d700 1 Processor -- start 2023-10-08T00:56:33.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.278+0000 7fd632f9d700 1 -- start start 2023-10-08T00:56:33.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd632f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62c06a0a0 0x7fd62c16b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd632f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd62c06d950 0x7fd62c0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd632f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06dd20 0x7fd62c1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd632f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c150fc0 con 0x7fd62c06dd20 2023-10-08T00:56:33.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd632f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c151f70 con 0x7fd62c06d950 2023-10-08T00:56:33.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd632f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c1511d0 con 0x7fd62c06a0a0 2023-10-08T00:56:33.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06dd20 0x7fd62c1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06dd20 0x7fd62c1528b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46516/0 (socket says 172.21.15.119:46516) 2023-10-08T00:56:33.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63279c700 1 -- 172.21.15.119:0/2840080046 learned_addr learned my addr 172.21.15.119:0/2840080046 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:33.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63179a700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd62c06d950 0x7fd62c0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd631f9b700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62c06a0a0 0x7fd62c16b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63279c700 1 -- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62c06a0a0 msgr2=0x7fd62c16b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63279c700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62c06a0a0 0x7fd62c16b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63279c700 1 -- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd62c06d950 msgr2=0x7fd62c0cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63279c700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd62c06d950 0x7fd62c0cada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63279c700 1 -- 172.21.15.119:0/2840080046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280094f0 con 0x7fd62c06dd20 2023-10-08T00:56:33.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.279+0000 7fd63179a700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd62c06d950 0x7fd62c0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:33.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.280+0000 7fd63279c700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06dd20 0x7fd62c1528b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd62400cdb0 tx=0x7fd6240077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:33.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.280+0000 7fd621ffb700 1 -- 172.21.15.119:0/2840080046 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62400aa20 con 0x7fd62c06dd20 2023-10-08T00:56:33.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.280+0000 7fd621ffb700 1 -- 172.21.15.119:0/2840080046 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62400ab80 con 0x7fd62c06dd20 2023-10-08T00:56:33.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.280+0000 7fd621ffb700 1 -- 172.21.15.119:0/2840080046 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624017430 con 0x7fd62c06dd20 2023-10-08T00:56:33.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.280+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c1514b0 con 0x7fd62c06dd20 2023-10-08T00:56:33.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.280+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c151840 con 0x7fd62c06dd20 2023-10-08T00:56:33.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.281+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c04c3c0 con 0x7fd62c06dd20 2023-10-08T00:56:33.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.282+0000 7fd621ffb700 1 -- 172.21.15.119:0/2840080046 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd624003020 con 0x7fd62c06dd20 2023-10-08T00:56:33.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.283+0000 7fd621ffb700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd610066f30 0x7fd6100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.283+0000 7fd621ffb700 1 -- 172.21.15.119:0/2840080046 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fd624013070 con 0x7fd62c06dd20 2023-10-08T00:56:33.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.283+0000 7fd631f9b700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd610066f30 0x7fd6100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.284+0000 7fd631f9b700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd610066f30 0x7fd6100693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd62c06e420 tx=0x7fd628016000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:33.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.287+0000 7fd621ffb700 1 -- 172.21.15.119:0/2840080046 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd624051b90 con 0x7fd62c06dd20 2023-10-08T00:56:33.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.550+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fd62c0c19d0 con 0x7fd62c06dd20 2023-10-08T00:56:33.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.554+0000 7fd621ffb700 1 -- 172.21.15.119:0/2840080046 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd6240551b0 con 0x7fd62c06dd20 2023-10-08T00:56:33.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd610066f30 msgr2=0x7fd6100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd610066f30 0x7fd6100693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd62c06e420 tx=0x7fd628016000 comp rx=0 tx=0).stop 2023-10-08T00:56:33.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06dd20 msgr2=0x7fd62c1528b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06dd20 0x7fd62c1528b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd62400cdb0 tx=0x7fd6240077d0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 shutdown_connections 2023-10-08T00:56:33.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd62c06a0a0 0x7fd62c16b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd610066f30 0x7fd6100693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd62c06dd20 0x7fd62c1528b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 --2- 172.21.15.119:0/2840080046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd62c06d950 0x7fd62c0cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 >> 172.21.15.119:0/2840080046 conn(0x7fd62c0cc1f0 msgr2=0x7fd62c0c1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:33.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 shutdown_connections 2023-10-08T00:56:33.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.556+0000 7fd632f9d700 1 -- 172.21.15.119:0/2840080046 wait complete. 2023-10-08T00:56:33.575 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-08T00:56:33.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.755+0000 7fbc3f19d700 1 Processor -- start 2023-10-08T00:56:33.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.755+0000 7fbc3f19d700 1 -- start start 2023-10-08T00:56:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3f19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 0x7fbc38157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3f19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc38150550 0x7fbc380c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3f19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3806e2a0 con 0x7fbc380c7240 2023-10-08T00:56:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3f19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc380c76e0 con 0x7fbc38150550 2023-10-08T00:56:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3f19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3806b2e0 con 0x7fbc38150180 2023-10-08T00:56:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3d99a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc38150550 0x7fbc380c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3d99a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc38150550 0x7fbc380c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46520/0 (socket says 172.21.15.119:46520) 2023-10-08T00:56:33.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3d99a700 1 -- 172.21.15.119:0/984542469 learned_addr learned my addr 172.21.15.119:0/984542469 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:33.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.756+0000 7fbc3e19b700 1 --2- 172.21.15.119:0/984542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 0x7fbc38157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3d99a700 1 -- 172.21.15.119:0/984542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 msgr2=0x7fbc38157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:33.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3d99a700 1 --2- 172.21.15.119:0/984542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 0x7fbc38157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3d99a700 1 -- 172.21.15.119:0/984542469 >> v1:172.21.15.184:6789/0 conn(0x7fbc380c7240 legacy=0x7fbc38069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3d99a700 1 -- 172.21.15.119:0/984542469 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc380c78e0 con 0x7fbc38150550 2023-10-08T00:56:33.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3e19b700 1 --2- 172.21.15.119:0/984542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 0x7fbc38157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:33.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3d99a700 1 --2- 172.21.15.119:0/984542469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc38150550 0x7fbc380c6d00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbc280099b0 tx=0x7fbc28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4ab983c8888de8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3d199700 1 -- 172.21.15.119:0/984542469 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800c970 con 0x7fbc38150550 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3d199700 1 -- 172.21.15.119:0/984542469 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2800cad0 con 0x7fbc38150550 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.757+0000 7fbc3d199700 1 -- 172.21.15.119:0/984542469 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2800ce70 con 0x7fbc38150550 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.758+0000 7fbc3f19d700 1 -- 172.21.15.119:0/984542469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc38150550 msgr2=0x7fbc380c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.758+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/984542469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc38150550 0x7fbc380c6d00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbc280099b0 tx=0x7fbc28009250 comp rx=0 tx=0).stop 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.758+0000 7fbc3f19d700 1 -- 172.21.15.119:0/984542469 shutdown_connections 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.758+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/984542469 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 0x7fbc38157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.758+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/984542469 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc38150550 0x7fbc380c6d00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.758+0000 7fbc3f19d700 1 -- 172.21.15.119:0/984542469 >> 172.21.15.119:0/984542469 conn(0x7fbc380cc1f0 msgr2=0x7fbc380cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:33.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.758+0000 7fbc3f19d700 1 -- 172.21.15.119:0/984542469 shutdown_connections 2023-10-08T00:56:33.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.758+0000 7fbc3f19d700 1 -- 172.21.15.119:0/984542469 wait complete. 2023-10-08T00:56:33.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.759+0000 7fbc3f19d700 1 Processor -- start 2023-10-08T00:56:33.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.759+0000 7fbc3f19d700 1 -- start start 2023-10-08T00:56:33.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3f19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c7240 0x7fbc380af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3f19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 0x7fbc380b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3f19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38150550 0x7fbc380c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3f19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380c3fe0 con 0x7fbc380c7240 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3f19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380c33d0 con 0x7fbc38150550 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3f19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380c36b0 con 0x7fbc38150180 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3e19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c7240 0x7fbc380af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3e19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c7240 0x7fbc380af950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46526/0 (socket says 172.21.15.119:46526) 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.760+0000 7fbc3e19b700 1 -- 172.21.15.119:0/1461511509 learned_addr learned my addr 172.21.15.119:0/1461511509 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc3e19b700 1 -- 172.21.15.119:0/1461511509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 msgr2=0x7fbc380b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc3e19b700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 0x7fbc380b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc3e19b700 1 -- 172.21.15.119:0/1461511509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38150550 msgr2=0x7fbc380c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc3e99c700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38150550 0x7fbc380c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc3e19b700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38150550 0x7fbc380c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc3e19b700 1 -- 172.21.15.119:0/1461511509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280094f0 con 0x7fbc380c7240 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc3e19b700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c7240 0x7fbc380af950 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e8d0 tx=0x7fbc34008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc2e7fc700 1 -- 172.21.15.119:0/1461511509 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400ee70 con 0x7fbc380c7240 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc2e7fc700 1 -- 172.21.15.119:0/1461511509 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3400c5e0 con 0x7fbc380c7240 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc2e7fc700 1 -- 172.21.15.119:0/1461511509 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3400c980 con 0x7fbc380c7240 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.761+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc380c3990 con 0x7fbc380c7240 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.762+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc380c5730 con 0x7fbc380c7240 2023-10-08T00:56:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.762+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3804c3c0 con 0x7fbc380c7240 2023-10-08T00:56:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.763+0000 7fbc2e7fc700 1 -- 172.21.15.119:0/1461511509 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc34013410 con 0x7fbc380c7240 2023-10-08T00:56:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.764+0000 7fbc2e7fc700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc1c066f30 0x7fbc1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.764+0000 7fbc2e7fc700 1 -- 172.21.15.119:0/1461511509 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fbc34084310 con 0x7fbc380c7240 2023-10-08T00:56:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.764+0000 7fbc3d99a700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc1c066f30 0x7fbc1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:33.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.765+0000 7fbc3d99a700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc1c066f30 0x7fbc1c0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbc280099b0 tx=0x7fbc28018000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:33.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:33.768+0000 7fbc2e7fc700 1 -- 172.21.15.119:0/1461511509 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc34051830 con 0x7fbc380c7240 2023-10-08T00:56:34.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.040+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fbc3806d650 con 0x7fbc380c7240 2023-10-08T00:56:34.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.044+0000 7fbc2e7fc700 1 -- 172.21.15.119:0/1461511509 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbc34054e50 con 0x7fbc380c7240 2023-10-08T00:56:34.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc1c066f30 msgr2=0x7fbc1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc1c066f30 0x7fbc1c0693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbc280099b0 tx=0x7fbc28018000 comp rx=0 tx=0).stop 2023-10-08T00:56:34.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c7240 msgr2=0x7fbc380af950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c7240 0x7fbc380af950 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400e8d0 tx=0x7fbc34008040 comp rx=0 tx=0).stop 2023-10-08T00:56:34.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 shutdown_connections 2023-10-08T00:56:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fbc1c066f30 0x7fbc1c0693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc38150550 0x7fbc380c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c7240 0x7fbc380af950 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 --2- 172.21.15.119:0/1461511509 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc38150180 0x7fbc380b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.046+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 >> 172.21.15.119:0/1461511509 conn(0x7fbc380cc1f0 msgr2=0x7fbc380abab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.047+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 shutdown_connections 2023-10-08T00:56:34.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.047+0000 7fbc3f19d700 1 -- 172.21.15.119:0/1461511509 wait complete. 2023-10-08T00:56:34.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-08T00:56:34.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.246+0000 7f8c6bbff700 1 Processor -- start 2023-10-08T00:56:34.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.246+0000 7f8c6bbff700 1 -- start start 2023-10-08T00:56:34.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.247+0000 7f8c6bbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c64159760 0x7f8c640c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.247+0000 7f8c6bbff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8c640c84c0 0x7f8c6406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.247+0000 7f8c6bbff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c640c7af0 con 0x7f8c64159b30 2023-10-08T00:56:34.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.247+0000 7f8c6bbff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c640c74e0 con 0x7f8c640c84c0 2023-10-08T00:56:34.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.247+0000 7f8c6bbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640cc870 con 0x7f8c64159760 2023-10-08T00:56:34.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6a19c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8c640c84c0 0x7f8c6406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6999b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c64159760 0x7f8c640c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6999b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c64159760 0x7f8c640c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:41240/0 (socket says 172.21.15.119:41240) 2023-10-08T00:56:34.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6999b700 1 -- 172.21.15.119:0/1530765304 learned_addr learned my addr 172.21.15.119:0/1530765304 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c68999700 1 -- 172.21.15.119:0/1530765304 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497994332 0 0) 0x7f8c640c7af0 con 0x7f8c64159b30 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c68999700 1 -- 172.21.15.119:0/1530765304 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c50003570 con 0x7f8c64159b30 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6a19c700 1 -- 172.21.15.119:0/1530765304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c64159760 msgr2=0x7f8c640c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6a19c700 1 --2- 172.21.15.119:0/1530765304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c64159760 0x7f8c640c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6a19c700 1 -- 172.21.15.119:0/1530765304 >> v1:172.21.15.184:6789/0 conn(0x7f8c64159b30 legacy=0x7f8c640af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6a19c700 1 -- 172.21.15.119:0/1530765304 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c640cc590 con 0x7f8c640c84c0 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.248+0000 7f8c6999b700 1 --2- 172.21.15.119:0/1530765304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c64159760 0x7f8c640c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.249+0000 7f8c6a19c700 1 --2- 172.21.15.119:0/1530765304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c640c84c0 0x7f8c6406e520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009e60 tx=0x7f8c5c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ab7724f9886582f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:34.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.249+0000 7f8c68999700 1 -- 172.21.15.119:0/1530765304 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00e070 con 0x7f8c640c84c0 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.249+0000 7f8c68999700 1 -- 172.21.15.119:0/1530765304 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c00bab0 con 0x7f8c640c84c0 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.249+0000 7f8c68999700 1 -- 172.21.15.119:0/1530765304 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c00be70 con 0x7f8c640c84c0 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.250+0000 7f8c6bbff700 1 -- 172.21.15.119:0/1530765304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c640c84c0 msgr2=0x7f8c6406e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.250+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/1530765304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c640c84c0 0x7f8c6406e520 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009e60 tx=0x7f8c5c009780 comp rx=0 tx=0).stop 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.250+0000 7f8c6bbff700 1 -- 172.21.15.119:0/1530765304 shutdown_connections 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.250+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/1530765304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c640c84c0 0x7f8c6406e520 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.250+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/1530765304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c64159760 0x7f8c640c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.250+0000 7f8c6bbff700 1 -- 172.21.15.119:0/1530765304 >> 172.21.15.119:0/1530765304 conn(0x7f8c640ca060 msgr2=0x7f8c6406de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.250+0000 7f8c6bbff700 1 -- 172.21.15.119:0/1530765304 shutdown_connections 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.250+0000 7f8c6bbff700 1 -- 172.21.15.119:0/1530765304 wait complete. 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.251+0000 7f8c6bbff700 1 Processor -- start 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.251+0000 7f8c6bbff700 1 -- start start 2023-10-08T00:56:34.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6bbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640c84c0 0x7f8c64151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6bbff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c64159760 0x7f8c6415e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6bbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64159b30 0x7f8c640b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6bbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640c5710 con 0x7f8c64159b30 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6bbff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640c4b00 con 0x7f8c64159760 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6bbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c640c4de0 con 0x7f8c640c84c0 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6a19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64159b30 0x7f8c640b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6a19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64159b30 0x7f8c640b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46536/0 (socket says 172.21.15.119:46536) 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.252+0000 7f8c6a19c700 1 -- 172.21.15.119:0/4152052449 learned_addr learned my addr 172.21.15.119:0/4152052449 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6999b700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640c84c0 0x7f8c64151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6a19c700 1 -- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640c84c0 msgr2=0x7f8c64151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6a19c700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640c84c0 0x7f8c64151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6a19c700 1 -- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c64159760 msgr2=0x7f8c6415e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6919a700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c64159760 0x7f8c6415e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6a19c700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c64159760 0x7f8c6415e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6a19c700 1 -- 172.21.15.119:0/4152052449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c600094f0 con 0x7f8c64159b30 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6999b700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640c84c0 0x7f8c64151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6919a700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c64159760 0x7f8c6415e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c6a19c700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64159b30 0x7f8c640b1510 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8c6406da80 tx=0x7f8c5c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.253+0000 7f8c59ffb700 1 -- 172.21.15.119:0/4152052449 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c027470 con 0x7f8c64159b30 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.254+0000 7f8c59ffb700 1 -- 172.21.15.119:0/4152052449 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c0275f0 con 0x7f8c64159b30 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.254+0000 7f8c59ffb700 1 -- 172.21.15.119:0/4152052449 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c0279b0 con 0x7f8c64159b30 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.254+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c009a20 con 0x7f8c64159b30 2023-10-08T00:56:34.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.254+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c640c53a0 con 0x7f8c64159b30 2023-10-08T00:56:34.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.255+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c640c58c0 con 0x7f8c64159b30 2023-10-08T00:56:34.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.255+0000 7f8c59ffb700 1 -- 172.21.15.119:0/4152052449 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c5c024070 con 0x7f8c64159b30 2023-10-08T00:56:34.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.257+0000 7f8c59ffb700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8c48066f30 0x7f8c480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.257+0000 7f8c59ffb700 1 -- 172.21.15.119:0/4152052449 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f8c5c086b30 con 0x7f8c64159b30 2023-10-08T00:56:34.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.257+0000 7f8c6999b700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8c48066f30 0x7f8c480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.258+0000 7f8c6999b700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8c48066f30 0x7f8c480693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c600094c0 tx=0x7f8c60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:34.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.261+0000 7f8c59ffb700 1 -- 172.21.15.119:0/4152052449 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c640c58c0 con 0x7f8c64159b30 2023-10-08T00:56:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.529+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8c640c77b0 con 0x7f8c64159b30 2023-10-08T00:56:34.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.533+0000 7f8c59ffb700 1 -- 172.21.15.119:0/4152052449 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8c5c02c0a0 con 0x7f8c64159b30 2023-10-08T00:56:34.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8c48066f30 msgr2=0x7f8c480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8c48066f30 0x7f8c480693e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8c600094c0 tx=0x7f8c60008040 comp rx=0 tx=0).stop 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64159b30 msgr2=0x7f8c640b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64159b30 0x7f8c640b1510 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8c6406da80 tx=0x7f8c5c019000 comp rx=0 tx=0).stop 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 shutdown_connections 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c640c84c0 0x7f8c64151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8c48066f30 0x7f8c480693e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c64159b30 0x7f8c640b1510 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 --2- 172.21.15.119:0/4152052449 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c64159760 0x7f8c6415e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 >> 172.21.15.119:0/4152052449 conn(0x7f8c640ca060 msgr2=0x7f8c6415a380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 shutdown_connections 2023-10-08T00:56:34.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.536+0000 7f8c6bbff700 1 -- 172.21.15.119:0/4152052449 wait complete. 2023-10-08T00:56:34.554 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-08T00:56:34.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.734+0000 7fdb8ad9d700 1 Processor -- start 2023-10-08T00:56:34.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.734+0000 7fdb8ad9d700 1 -- start start 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.735+0000 7fdb8ad9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdb841597f0 0x7fdb840b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.735+0000 7fdb8ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb8415d230 0x7fdb8415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.735+0000 7fdb8ad9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8415db70 con 0x7fdb84159420 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.735+0000 7fdb8ad9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb840b11d0 con 0x7fdb8415d230 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.735+0000 7fdb8ad9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb840b14b0 con 0x7fdb841597f0 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8a59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb8415d230 0x7fdb8415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8a59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb8415d230 0x7fdb8415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46546/0 (socket says 172.21.15.119:46546) 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8a59c700 1 -- 172.21.15.119:0/1855598291 learned_addr learned my addr 172.21.15.119:0/1855598291 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:34.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8959a700 1 --2- 172.21.15.119:0/1855598291 >> v2:172.21.15.184:3300/0 conn(0x7fdb841597f0 0x7fdb840b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8a59c700 1 -- 172.21.15.119:0/1855598291 >> v2:172.21.15.184:3300/0 conn(0x7fdb841597f0 msgr2=0x7fdb840b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:34.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8a59c700 1 --2- 172.21.15.119:0/1855598291 >> v2:172.21.15.184:3300/0 conn(0x7fdb841597f0 0x7fdb840b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8a59c700 1 -- 172.21.15.119:0/1855598291 >> v1:172.21.15.184:6789/0 conn(0x7fdb84159420 legacy=0x7fdb840b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:34.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8a59c700 1 -- 172.21.15.119:0/1855598291 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb840b1790 con 0x7fdb8415d230 2023-10-08T00:56:34.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb8a59c700 1 --2- 172.21.15.119:0/1855598291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb8415d630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdb84089750 tx=0x7fdb7c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdabeaf5cd041a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:34.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb88d99700 1 -- 172.21.15.119:0/1855598291 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00e070 con 0x7fdb8415d230 2023-10-08T00:56:34.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb88d99700 1 -- 172.21.15.119:0/1855598291 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c00ba10 con 0x7fdb8415d230 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.736+0000 7fdb88d99700 1 -- 172.21.15.119:0/1855598291 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0092e0 con 0x7fdb8415d230 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.737+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/1855598291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 msgr2=0x7fdb8415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.737+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/1855598291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb8415d630 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdb84089750 tx=0x7fdb7c009840 comp rx=0 tx=0).stop 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.737+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/1855598291 shutdown_connections 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.737+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/1855598291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb8415d630 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.737+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/1855598291 >> v2:172.21.15.184:3300/0 conn(0x7fdb841597f0 0x7fdb840b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.737+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/1855598291 >> 172.21.15.119:0/1855598291 conn(0x7fdb840d0660 msgr2=0x7fdb840d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.737+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/1855598291 shutdown_connections 2023-10-08T00:56:34.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.737+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/1855598291 wait complete. 2023-10-08T00:56:34.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.738+0000 7fdb8ad9d700 1 Processor -- start 2023-10-08T00:56:34.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.739+0000 7fdb8ad9d700 1 -- start start 2023-10-08T00:56:34.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.739+0000 7fdb8ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb84159420 0x7fdb84169be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.739+0000 7fdb8ad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb841597f0 0x7fdb840c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.739+0000 7fdb8ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb840b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.739+0000 7fdb8ad9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb84166f60 con 0x7fdb8415d230 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.739+0000 7fdb8ad9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8416a820 con 0x7fdb841597f0 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.739+0000 7fdb8ad9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8416b0c0 con 0x7fdb84159420 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb840b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb840b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46552/0 (socket says 172.21.15.119:46552) 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 -- 172.21.15.119:0/2246716828 learned_addr learned my addr 172.21.15.119:0/2246716828 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:34.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb89d9b700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb84159420 0x7fdb84169be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8959a700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb841597f0 0x7fdb840c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 -- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb84159420 msgr2=0x7fdb84169be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb84159420 0x7fdb84169be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 -- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb841597f0 msgr2=0x7fdb840c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:34.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb841597f0 0x7fdb840c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:34.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 -- 172.21.15.119:0/2246716828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c009ae0 con 0x7fdb8415d230 2023-10-08T00:56:34.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb89d9b700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb84159420 0x7fdb84169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:34.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8959a700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb841597f0 0x7fdb840c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:34.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb8a59c700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb840b4e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb84089340 tx=0x7fdb7c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:34.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.740+0000 7fdb79ffb700 1 -- 172.21.15.119:0/2246716828 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c012e60 con 0x7fdb8415d230 2023-10-08T00:56:34.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.741+0000 7fdb79ffb700 1 -- 172.21.15.119:0/2246716828 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c0092e0 con 0x7fdb8415d230 2023-10-08T00:56:34.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.741+0000 7fdb79ffb700 1 -- 172.21.15.119:0/2246716828 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c01d670 con 0x7fdb8415d230 2023-10-08T00:56:34.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.741+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb840c5e50 con 0x7fdb8415d230 2023-10-08T00:56:34.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.741+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8416ad50 con 0x7fdb8415d230 2023-10-08T00:56:34.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.742+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8404c3c0 con 0x7fdb8415d230 2023-10-08T00:56:34.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.743+0000 7fdb79ffb700 1 -- 172.21.15.119:0/2246716828 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb7c01da60 con 0x7fdb8415d230 2023-10-08T00:56:34.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.744+0000 7fdb79ffb700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdb68066f30 0x7fdb680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:34.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.744+0000 7fdb79ffb700 1 -- 172.21.15.119:0/2246716828 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7fdb7c085ad0 con 0x7fdb8415d230 2023-10-08T00:56:34.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.744+0000 7fdb89d9b700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdb68066f30 0x7fdb680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:34.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.745+0000 7fdb89d9b700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdb68066f30 0x7fdb680693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdb80002880 tx=0x7fdb80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:34.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:34.748+0000 7fdb79ffb700 1 -- 172.21.15.119:0/2246716828 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb7c052ff0 con 0x7fdb8415d230 2023-10-08T00:56:35.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.014+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fdb8416b200 con 0x7fdb8415d230 2023-10-08T00:56:35.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.018+0000 7fdb79ffb700 1 -- 172.21.15.119:0/2246716828 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdb7c056610 con 0x7fdb8415d230 2023-10-08T00:56:35.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdb68066f30 msgr2=0x7fdb680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdb68066f30 0x7fdb680693e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdb80002880 tx=0x7fdb80009040 comp rx=0 tx=0).stop 2023-10-08T00:56:35.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 msgr2=0x7fdb840b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb840b4e20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb84089340 tx=0x7fdb7c017000 comp rx=0 tx=0).stop 2023-10-08T00:56:35.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 shutdown_connections 2023-10-08T00:56:35.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb84159420 0x7fdb84169be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdb68066f30 0x7fdb680693e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb8415d230 0x7fdb840b4e20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 --2- 172.21.15.119:0/2246716828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb841597f0 0x7fdb840c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 >> 172.21.15.119:0/2246716828 conn(0x7fdb840d0660 msgr2=0x7fdb841572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:35.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 shutdown_connections 2023-10-08T00:56:35.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.021+0000 7fdb8ad9d700 1 -- 172.21.15.119:0/2246716828 wait complete. 2023-10-08T00:56:35.040 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-08T00:56:35.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.223+0000 7f6c7159d700 1 Processor -- start 2023-10-08T00:56:35.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.223+0000 7f6c7159d700 1 -- start start 2023-10-08T00:56:35.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.224+0000 7f6c7159d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 0x7f6c6c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.224+0000 7f6c7159d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0ca860 con 0x7f6c6c156310 2023-10-08T00:56:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.224+0000 7f6c7159d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c6c0cab40 con 0x7f6c6c1566e0 2023-10-08T00:56:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.224+0000 7f6c7159d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0cae20 con 0x7f6c6c0ae710 2023-10-08T00:56:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.224+0000 7f6c6bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6c6c156310 0x7f6c6c0c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45018/0 (socket says 172.21.15.119:45018) 2023-10-08T00:56:35.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.224+0000 7f6c6bfff700 1 -- 172.21.15.119:0/3672542149 learned_addr learned my addr 172.21.15.119:0/3672542149 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c70d9c700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 0x7f6c6c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466818359 0 0) 0x7f6c6c0ca860 con 0x7f6c6c156310 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c58003570 con 0x7f6c6c156310 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 897166847 0 0) 0x7f6c58003570 con 0x7f6c6c156310 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 msgr2=0x7f6c6c0c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 0x7f6c6c0c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 >> v1:172.21.15.184:6789/0 conn(0x7f6c6c1566e0 legacy=0x7f6c6c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c0cb100 con 0x7f6c6c156310 2023-10-08T00:56:35.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f6c5c001470 con 0x7f6c6c156310 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 >> v1:172.21.15.119:6789/0 conn(0x7f6c6c156310 legacy=0x7f6c6c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 0x7f6c58003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c58004010 0x7f6c58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c58004d50 0x7f6c580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0cae20 con 0x7f6c58004010 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5800bbf0 con 0x7f6c58004d50 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.225+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5800bdb0 con 0x7f6c6c0ae710 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c63fff700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c58004010 0x7f6c58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c6bfff700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 0x7f6c58003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c70d9c700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c58004d50 0x7f6c580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c63fff700 1 -- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 msgr2=0x7f6c58003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c63fff700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 0x7f6c58003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c63fff700 1 -- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c58004d50 msgr2=0x7f6c580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c63fff700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c58004d50 0x7f6c580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c63fff700 1 -- 172.21.15.119:0/3672542149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58003570 con 0x7f6c58004010 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c70d9c700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c58004d50 0x7f6c580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c6bfff700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c6c0ae710 0x7f6c58003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c63fff700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c58004010 0x7f6c58004810 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c156b50 tx=0x7f6c54007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fc3d1d5c31d13b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54012420 con 0x7f6c58004010 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c54012580 con 0x7f6c58004010 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.226+0000 7f6c6b7fe700 1 -- 172.21.15.119:0/3672542149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54012920 con 0x7f6c58004010 2023-10-08T00:56:35.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.227+0000 7f6c7159d700 1 -- 172.21.15.119:0/3672542149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c58004010 msgr2=0x7f6c58004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.227+0000 7f6c7159d700 1 --2- 172.21.15.119:0/3672542149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c58004010 0x7f6c58004810 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c156b50 tx=0x7f6c54007d30 comp rx=0 tx=0).stop 2023-10-08T00:56:35.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.227+0000 7f6c70d9c700 1 -- 172.21.15.119:0/3672542149 reap_dead start 2023-10-08T00:56:35.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.227+0000 7f6c7159d700 1 -- 172.21.15.119:0/3672542149 shutdown_connections 2023-10-08T00:56:35.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.227+0000 7f6c7159d700 1 -- 172.21.15.119:0/3672542149 >> 172.21.15.119:0/3672542149 conn(0x7f6c6c0cc1f0 msgr2=0x7f6c6c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:35.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.227+0000 7f6c7159d700 1 -- 172.21.15.119:0/3672542149 shutdown_connections 2023-10-08T00:56:35.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.227+0000 7f6c7159d700 1 -- 172.21.15.119:0/3672542149 wait complete. 2023-10-08T00:56:35.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.228+0000 7f6c7159d700 1 Processor -- start 2023-10-08T00:56:35.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.229+0000 7f6c7159d700 1 -- start start 2023-10-08T00:56:35.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.229+0000 7f6c7159d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c58004010 0x7f6c6c14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.229+0000 7f6c7159d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c6c14f310 0x7f6c6c14f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.229+0000 7f6c7159d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c6c0c9ca0 0x7f6c6c0ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.229+0000 7f6c7159d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c0ca650 con 0x7f6c6c14f310 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.229+0000 7f6c7159d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c1509e0 con 0x7f6c6c0c9ca0 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.229+0000 7f6c7159d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c14fdd0 con 0x7f6c58004010 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c6c14f310 0x7f6c6c14f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c6c14f310 0x7f6c6c14f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53732/0 (socket says 172.21.15.119:53732) 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 -- 172.21.15.119:0/3131731632 learned_addr learned my addr 172.21.15.119:0/3131731632 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c70d9c700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c6c0c9ca0 0x7f6c6c0ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c6bfff700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c58004010 0x7f6c6c14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 -- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c58004010 msgr2=0x7f6c6c14c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c58004010 0x7f6c6c14c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 -- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c6c0c9ca0 msgr2=0x7f6c6c0ca110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c6c0c9ca0 0x7f6c6c0ca110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 -- 172.21.15.119:0/3131731632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c1500b0 con 0x7f6c6c14f310 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c70d9c700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c6c0c9ca0 0x7f6c6c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.230+0000 7f6c63fff700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c6c14f310 0x7f6c6c14f780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c580057d0 tx=0x7f6c54015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.231+0000 7f6c68ff9700 1 -- 172.21.15.119:0/3131731632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54012420 con 0x7f6c6c14f310 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.231+0000 7f6c68ff9700 1 -- 172.21.15.119:0/3131731632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c540126f0 con 0x7f6c6c14f310 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.231+0000 7f6c68ff9700 1 -- 172.21.15.119:0/3131731632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54012ea0 con 0x7f6c6c14f310 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.231+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c150390 con 0x7f6c6c14f310 2023-10-08T00:56:35.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.231+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c151610 con 0x7f6c6c14f310 2023-10-08T00:56:35.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.232+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c150b20 con 0x7f6c6c14f310 2023-10-08T00:56:35.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.233+0000 7f6c68ff9700 1 -- 172.21.15.119:0/3131731632 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6c54019030 con 0x7f6c6c14f310 2023-10-08T00:56:35.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.233+0000 7f6c68ff9700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6c4c066f30 0x7f6c4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.233+0000 7f6c68ff9700 1 -- 172.21.15.119:0/3131731632 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6c54085810 con 0x7f6c6c14f310 2023-10-08T00:56:35.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.234+0000 7f6c6bfff700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6c4c066f30 0x7f6c4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.234+0000 7f6c6bfff700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6c4c066f30 0x7f6c4c0693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0010e0 tx=0x7f6c5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:35.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.237+0000 7f6c68ff9700 1 -- 172.21.15.119:0/3131731632 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6c6c150b20 con 0x7f6c6c14f310 2023-10-08T00:56:35.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.506+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f6c6c0ca650 con 0x7f6c6c14f310 2023-10-08T00:56:35.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.509+0000 7f6c68ff9700 1 -- 172.21.15.119:0/3131731632 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6c54052d30 con 0x7f6c6c14f310 2023-10-08T00:56:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.511+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6c4c066f30 msgr2=0x7f6c4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.511+0000 7f6c7159d700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6c4c066f30 0x7f6c4c0693e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c0010e0 tx=0x7f6c5c009040 comp rx=0 tx=0).stop 2023-10-08T00:56:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c6c14f310 msgr2=0x7f6c6c14f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c6c14f310 0x7f6c6c14f780 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6c580057d0 tx=0x7f6c54015000 comp rx=0 tx=0).stop 2023-10-08T00:56:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 shutdown_connections 2023-10-08T00:56:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6c58004010 0x7f6c6c14c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6c4c066f30 0x7f6c4c0693e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6c6c0c9ca0 0x7f6c6c0ca110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 --2- 172.21.15.119:0/3131731632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c6c14f310 0x7f6c6c14f780 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 >> 172.21.15.119:0/3131731632 conn(0x7f6c6c0cc1f0 msgr2=0x7f6c6c0c11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 shutdown_connections 2023-10-08T00:56:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.512+0000 7f6c7159d700 1 -- 172.21.15.119:0/3131731632 wait complete. 2023-10-08T00:56:35.531 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:35.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.710+0000 7f33b899d700 1 Processor -- start 2023-10-08T00:56:35.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.710+0000 7f33b899d700 1 -- start start 2023-10-08T00:56:35.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.711+0000 7f33b899d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b4150180 0x7f33b4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.711+0000 7f33b899d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f33b40c7410 0x7f33b4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.711+0000 7f33b899d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b406e2a0 con 0x7f33b4150550 2023-10-08T00:56:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.711+0000 7f33b899d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b406b2e0 con 0x7f33b40c7410 2023-10-08T00:56:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.711+0000 7f33b899d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40c78e0 con 0x7f33b4150180 2023-10-08T00:56:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.711+0000 7f33abfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f33b4150550 0x7f33b40c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45022/0 (socket says 172.21.15.119:45022) 2023-10-08T00:56:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.711+0000 7f33abfff700 1 -- 172.21.15.119:0/71426491 learned_addr learned my addr 172.21.15.119:0/71426491 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:35.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573462557 0 0) 0x7f33b406e2a0 con 0x7f33b4150550 2023-10-08T00:56:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33a0004680 con 0x7f33b4150550 2023-10-08T00:56:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4008628857 0 0) 0x7f33a0004680 con 0x7f33b4150550 2023-10-08T00:56:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b4150180 msgr2=0x7f33b4157770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:56:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b4150180 0x7f33b4157770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 >> v2:172.21.15.184:3300/0 conn(0x7f33b40c7410 msgr2=0x7f33b4069f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:56:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 --2- 172.21.15.119:0/71426491 >> v2:172.21.15.184:3300/0 conn(0x7f33b40c7410 0x7f33b4069f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40c7bc0 con 0x7f33b4150550 2023-10-08T00:56:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f339c001470 con 0x7f33b4150550 2023-10-08T00:56:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 >> v1:172.21.15.119:6789/0 conn(0x7f33b4150550 legacy=0x7f33b40c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33a0004b70 0x7f33a0004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b4150180 0x7f33a0005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33a0005f60 0x7f33a00063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b406b2e0 con 0x7f33a0004b70 2023-10-08T00:56:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40c78e0 con 0x7f33a0005f60 2023-10-08T00:56:35.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.712+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0003510 con 0x7f33b4150180 2023-10-08T00:56:35.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b37fe700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33a0004b70 0x7f33a0004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b37fe700 1 -- 172.21.15.119:0/71426491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b4150180 msgr2=0x7f33a0005920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:56:35.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b37fe700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b4150180 0x7f33a0005920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b37fe700 1 -- 172.21.15.119:0/71426491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33a0005f60 msgr2=0x7f33a00063d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T00:56:35.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b37fe700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33a0005f60 0x7f33a00063d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b37fe700 1 -- 172.21.15.119:0/71426491 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a0004680 con 0x7f33a0004b70 2023-10-08T00:56:35.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b37fe700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33a0004b70 0x7f33a0004f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009920 tx=0x7f33a4006ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95f502f3bbfb6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:35.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4011400 con 0x7f33a0004b70 2023-10-08T00:56:35.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a4011580 con 0x7f33a0004b70 2023-10-08T00:56:35.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.713+0000 7f33b2ffd700 1 -- 172.21.15.119:0/71426491 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4011940 con 0x7f33a0004b70 2023-10-08T00:56:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.714+0000 7f33b899d700 1 -- 172.21.15.119:0/71426491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33a0004b70 msgr2=0x7f33a0004f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.714+0000 7f33b899d700 1 --2- 172.21.15.119:0/71426491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33a0004b70 0x7f33a0004f80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f33a4009920 tx=0x7f33a4006ef0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.714+0000 7f33b3fff700 1 -- 172.21.15.119:0/71426491 reap_dead start 2023-10-08T00:56:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.714+0000 7f33b899d700 1 -- 172.21.15.119:0/71426491 shutdown_connections 2023-10-08T00:56:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.714+0000 7f33b899d700 1 -- 172.21.15.119:0/71426491 >> 172.21.15.119:0/71426491 conn(0x7f33b40cc1f0 msgr2=0x7f33b40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.714+0000 7f33b899d700 1 -- 172.21.15.119:0/71426491 shutdown_connections 2023-10-08T00:56:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.714+0000 7f33b899d700 1 -- 172.21.15.119:0/71426491 wait complete. 2023-10-08T00:56:35.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.715+0000 7f33b899d700 1 Processor -- start 2023-10-08T00:56:35.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.715+0000 7f33b899d700 1 -- start start 2023-10-08T00:56:35.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.716+0000 7f33b899d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b41491b0 0x7f33b40ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.716+0000 7f33b899d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33b40add60 0x7f33b40b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.716+0000 7f33b899d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b414ac00 0x7f33b414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.716+0000 7f33b899d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b414b5e0 con 0x7f33b40add60 2023-10-08T00:56:35.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.716+0000 7f33b899d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40acaa0 con 0x7f33b41491b0 2023-10-08T00:56:35.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.716+0000 7f33b899d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b4149580 con 0x7f33b414ac00 2023-10-08T00:56:35.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33b40add60 0x7f33b40b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33b40add60 0x7f33b40b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53754/0 (socket says 172.21.15.119:53754) 2023-10-08T00:56:35.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 -- 172.21.15.119:0/1340628913 learned_addr learned my addr 172.21.15.119:0/1340628913 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:35.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33b3fff700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b414ac00 0x7f33b414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 -- 172.21.15.119:0/1340628913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b414ac00 msgr2=0x7f33b414b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b414ac00 0x7f33b414b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 -- 172.21.15.119:0/1340628913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b41491b0 msgr2=0x7f33b40ac460 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T00:56:35.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b41491b0 0x7f33b40ac460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 -- 172.21.15.119:0/1340628913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40acfa0 con 0x7f33b40add60 2023-10-08T00:56:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33b3fff700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b414ac00 0x7f33b414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33abfff700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33b40add60 0x7f33b40b1680 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f339c00ca90 tx=0x7f33b406d620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33ab7fe700 1 -- 172.21.15.119:0/1340628913 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c004b00 con 0x7f33b40add60 2023-10-08T00:56:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33ab7fe700 1 -- 172.21.15.119:0/1340628913 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339c004ca0 con 0x7f33b40add60 2023-10-08T00:56:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b40ad280 con 0x7f33b40add60 2023-10-08T00:56:35.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.717+0000 7f33ab7fe700 1 -- 172.21.15.119:0/1340628913 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339c0073a0 con 0x7f33b40add60 2023-10-08T00:56:35.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.718+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b40ad840 con 0x7f33b40add60 2023-10-08T00:56:35.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.719+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b404bbe0 con 0x7f33b40add60 2023-10-08T00:56:35.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.719+0000 7f33ab7fe700 1 -- 172.21.15.119:0/1340628913 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f339c00a5d0 con 0x7f33b40add60 2023-10-08T00:56:35.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.720+0000 7f33ab7fe700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3394066f30 0x7f33940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:35.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.720+0000 7f33ab7fe700 1 -- 172.21.15.119:0/1340628913 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f339c083940 con 0x7f33b40add60 2023-10-08T00:56:35.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.720+0000 7f33b37fe700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3394066f30 0x7f33940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:35.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.721+0000 7f33b37fe700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3394066f30 0x7f33940693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f33b406d5f0 tx=0x7f33a4012000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:35.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.724+0000 7f33ab7fe700 1 -- 172.21.15.119:0/1340628913 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f339c04c5a0 con 0x7f33b40add60 2023-10-08T00:56:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.916+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33b404c3c0 con 0x7f33b40add60 2023-10-08T00:56:35.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.917+0000 7f33ab7fe700 1 -- 172.21.15.119:0/1340628913 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+12182 (secure 0 0 0) 0x7f339c018090 con 0x7f33b40add60 2023-10-08T00:56:35.919 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:35.919 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:56:32.101173+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.919+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3394066f30 msgr2=0x7f33940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3394066f30 0x7f33940693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f33b406d5f0 tx=0x7f33a4012000 comp rx=0 tx=0).stop 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33b40add60 msgr2=0x7f33b40b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33b40add60 0x7f33b40b1680 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f339c00ca90 tx=0x7f33b406d620 comp rx=0 tx=0).stop 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 shutdown_connections 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33b414ac00 0x7f33b414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3394066f30 0x7f33940693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33b41491b0 0x7f33b40ac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 --2- 172.21.15.119:0/1340628913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33b40add60 0x7f33b40b1680 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 >> 172.21.15.119:0/1340628913 conn(0x7f33b40cc1f0 msgr2=0x7f33b415a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 shutdown_connections 2023-10-08T00:56:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:35.920+0000 7f33b899d700 1 -- 172.21.15.119:0/1340628913 wait complete. 2023-10-08T00:56:35.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T00:56:35.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T00:56:36.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.122+0000 7f137d1ff700 1 Processor -- start 2023-10-08T00:56:36.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.122+0000 7f137d1ff700 1 -- start start 2023-10-08T00:56:36.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.123+0000 7f137d1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13780c83f0 0x7f13780c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f137d1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 0x7f13780ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f137d1ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137815b0d0 con 0x7f13780ccee0 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f137d1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13780c7a60 con 0x7f13780c83f0 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f137d1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137815a4c0 con 0x7f13781568e0 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f1376d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13780c83f0 0x7f13780c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f1376d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13780c83f0 0x7f13780c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53770/0 (socket says 172.21.15.119:53770) 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f1376d9d700 1 -- 172.21.15.119:0/2161173972 learned_addr learned my addr 172.21.15.119:0/2161173972 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f137759e700 1 --2- 172.21.15.119:0/2161173972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 0x7f13780ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f1376d9d700 1 -- 172.21.15.119:0/2161173972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 msgr2=0x7f13780ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f1376d9d700 1 --2- 172.21.15.119:0/2161173972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 0x7f13780ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f1376d9d700 1 -- 172.21.15.119:0/2161173972 >> v1:172.21.15.184:6789/0 conn(0x7f13780ccee0 legacy=0x7f13781561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.124+0000 7f1376d9d700 1 -- 172.21.15.119:0/2161173972 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137815a7a0 con 0x7f13780c83f0 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.125+0000 7f1376d9d700 1 --2- 172.21.15.119:0/2161173972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f13780c7520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f13680099b0 tx=0x7f1368009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54302358f82baac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.125+0000 7f1375d9b700 1 -- 172.21.15.119:0/2161173972 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136800b860 con 0x7f13780c83f0 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.125+0000 7f1375d9b700 1 -- 172.21.15.119:0/2161173972 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136800b9c0 con 0x7f13780c83f0 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.125+0000 7f1375d9b700 1 -- 172.21.15.119:0/2161173972 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136800bd60 con 0x7f13780c83f0 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.125+0000 7f137d1ff700 1 -- 172.21.15.119:0/2161173972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 msgr2=0x7f13780c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.125+0000 7f137d1ff700 1 --2- 172.21.15.119:0/2161173972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f13780c7520 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f13680099b0 tx=0x7f1368009250 comp rx=0 tx=0).stop 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.126+0000 7f137d1ff700 1 -- 172.21.15.119:0/2161173972 shutdown_connections 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.126+0000 7f137d1ff700 1 --2- 172.21.15.119:0/2161173972 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 0x7f13780ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.126+0000 7f137d1ff700 1 --2- 172.21.15.119:0/2161173972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f13780c7520 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.126+0000 7f137d1ff700 1 -- 172.21.15.119:0/2161173972 >> 172.21.15.119:0/2161173972 conn(0x7f1378068e10 msgr2=0x7f13780cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:36.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.126+0000 7f137d1ff700 1 -- 172.21.15.119:0/2161173972 shutdown_connections 2023-10-08T00:56:36.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.126+0000 7f137d1ff700 1 -- 172.21.15.119:0/2161173972 wait complete. 2023-10-08T00:56:36.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.127+0000 7f137d1ff700 1 Processor -- start 2023-10-08T00:56:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.127+0000 7f137d1ff700 1 -- start start 2023-10-08T00:56:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f137d1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f1378159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f137d1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13780ccee0 0x7f13780b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f137d1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 0x7f13780b0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f137d1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137814d200 con 0x7f13780c83f0 2023-10-08T00:56:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f137d1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13780c3670 con 0x7f13780ccee0 2023-10-08T00:56:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f137d1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1378159ac0 con 0x7f13781568e0 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f1376d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f1378159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f1376d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f1378159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53776/0 (socket says 172.21.15.119:53776) 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.128+0000 7f1376d9d700 1 -- 172.21.15.119:0/307825601 learned_addr learned my addr 172.21.15.119:0/307825601 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f1376d9d700 1 -- 172.21.15.119:0/307825601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 msgr2=0x7f13780b0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f1376d9d700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 0x7f13780b0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f1376d9d700 1 -- 172.21.15.119:0/307825601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13780ccee0 msgr2=0x7f13780b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f1376d9d700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13780ccee0 0x7f13780b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f1376d9d700 1 -- 172.21.15.119:0/307825601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13680094f0 con 0x7f13780c83f0 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f1376d9d700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f1378159580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1378063d40 tx=0x7f1368017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f135effd700 1 -- 172.21.15.119:0/307825601 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368024440 con 0x7f13780c83f0 2023-10-08T00:56:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f135effd700 1 -- 172.21.15.119:0/307825601 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13680245c0 con 0x7f13780c83f0 2023-10-08T00:56:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1378157ca0 con 0x7f13780c83f0 2023-10-08T00:56:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f135effd700 1 -- 172.21.15.119:0/307825601 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368024980 con 0x7f13780c83f0 2023-10-08T00:56:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.129+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1378158260 con 0x7f13780c83f0 2023-10-08T00:56:36.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.130+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13780c7fe0 con 0x7f13780c83f0 2023-10-08T00:56:36.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.131+0000 7f135effd700 1 -- 172.21.15.119:0/307825601 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f136801a420 con 0x7f13780c83f0 2023-10-08T00:56:36.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.132+0000 7f135effd700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1364066f30 0x7f13640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.132+0000 7f135effd700 1 -- 172.21.15.119:0/307825601 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1368085ff0 con 0x7f13780c83f0 2023-10-08T00:56:36.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.132+0000 7f137659c700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1364066f30 0x7f13640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.133+0000 7f137659c700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1364066f30 0x7f13640693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f13780cb1c0 tx=0x7f136c007b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.136+0000 7f135effd700 1 -- 172.21.15.119:0/307825601 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1368053570 con 0x7f13780c83f0 2023-10-08T00:56:36.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.325+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f137804c3c0 con 0x7f13780c83f0 2023-10-08T00:56:36.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.326+0000 7f135effd700 1 -- 172.21.15.119:0/307825601 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1368056b90 con 0x7f13780c83f0 2023-10-08T00:56:36.327 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T00:56:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1364066f30 msgr2=0x7f13640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1364066f30 0x7f13640693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f13780cb1c0 tx=0x7f136c007b40 comp rx=0 tx=0).stop 2023-10-08T00:56:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 msgr2=0x7f1378159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f1378159580 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1378063d40 tx=0x7f1368017000 comp rx=0 tx=0).stop 2023-10-08T00:56:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 shutdown_connections 2023-10-08T00:56:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13781568e0 0x7f13780b0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1364066f30 0x7f13640693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13780c83f0 0x7f1378159580 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 --2- 172.21.15.119:0/307825601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13780ccee0 0x7f13780b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 >> 172.21.15.119:0/307825601 conn(0x7f1378068e10 msgr2=0x7f137814e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 shutdown_connections 2023-10-08T00:56:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.328+0000 7f137d1ff700 1 -- 172.21.15.119:0/307825601 wait complete. 2023-10-08T00:56:36.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T00:56:36.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.527+0000 7f1b12dff700 1 Processor -- start 2023-10-08T00:56:36.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.527+0000 7f1b12dff700 1 -- start start 2023-10-08T00:56:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.528+0000 7f1b12dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1b0c1578b0 0x7f1b0c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.528+0000 7f1b12dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b0c0af2f0 0x7f1b0c0af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.528+0000 7f1b12dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c0ad930 con 0x7f1b0c155c10 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.528+0000 7f1b12dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c0b0c80 con 0x7f1b0c0af2f0 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.528+0000 7f1b12dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c0acd20 con 0x7f1b0c1578b0 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b1139c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b0c0af2f0 0x7f1b0c0af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b1139c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b0c0af2f0 0x7f1b0c0af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53788/0 (socket says 172.21.15.119:53788) 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b1139c700 1 -- 172.21.15.119:0/1401761164 learned_addr learned my addr 172.21.15.119:0/1401761164 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b0bfff700 1 --2- 172.21.15.119:0/1401761164 >> v2:172.21.15.184:3300/0 conn(0x7f1b0c1578b0 0x7f1b0c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b1139c700 1 -- 172.21.15.119:0/1401761164 >> v2:172.21.15.184:3300/0 conn(0x7f1b0c1578b0 msgr2=0x7f1b0c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b1139c700 1 --2- 172.21.15.119:0/1401761164 >> v2:172.21.15.184:3300/0 conn(0x7f1b0c1578b0 0x7f1b0c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b1139c700 1 -- 172.21.15.119:0/1401761164 >> v1:172.21.15.184:6789/0 conn(0x7f1b0c155c10 legacy=0x7f1b0c0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b1139c700 1 -- 172.21.15.119:0/1401761164 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0ad000 con 0x7f1b0c0af2f0 2023-10-08T00:56:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.529+0000 7f1b1139c700 1 --2- 172.21.15.119:0/1401761164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c0af2f0 0x7f1b0c0af730 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0c8c60 tx=0x7f1b0400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee938e2f786d2f16 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b0b7fe700 1 -- 172.21.15.119:0/1401761164 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b04010070 con 0x7f1b0c0af2f0 2023-10-08T00:56:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b0b7fe700 1 -- 172.21.15.119:0/1401761164 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b0400de90 con 0x7f1b0c0af2f0 2023-10-08T00:56:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b0b7fe700 1 -- 172.21.15.119:0/1401761164 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0400b320 con 0x7f1b0c0af2f0 2023-10-08T00:56:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b12dff700 1 -- 172.21.15.119:0/1401761164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c0af2f0 msgr2=0x7f1b0c0af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b12dff700 1 --2- 172.21.15.119:0/1401761164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c0af2f0 0x7f1b0c0af730 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c0c8c60 tx=0x7f1b0400ba50 comp rx=0 tx=0).stop 2023-10-08T00:56:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b12dff700 1 -- 172.21.15.119:0/1401761164 shutdown_connections 2023-10-08T00:56:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b12dff700 1 --2- 172.21.15.119:0/1401761164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c0af2f0 0x7f1b0c0af730 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b12dff700 1 --2- 172.21.15.119:0/1401761164 >> v2:172.21.15.184:3300/0 conn(0x7f1b0c1578b0 0x7f1b0c155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b12dff700 1 -- 172.21.15.119:0/1401761164 >> 172.21.15.119:0/1401761164 conn(0x7f1b0c06a320 msgr2=0x7f1b0c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b12dff700 1 -- 172.21.15.119:0/1401761164 shutdown_connections 2023-10-08T00:56:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.530+0000 7f1b12dff700 1 -- 172.21.15.119:0/1401761164 wait complete. 2023-10-08T00:56:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.532+0000 7f1b12dff700 1 Processor -- start 2023-10-08T00:56:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.532+0000 7f1b12dff700 1 -- start start 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b12dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0c0af2f0 0x7f1b0c1453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b12dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c155c10 0x7f1b0c0cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b12dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b0c1578b0 0x7f1b0c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b12dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cbf00 con 0x7f1b0c155c10 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b12dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cb330 con 0x7f1b0c1578b0 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b12dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cb5d0 con 0x7f1b0c0af2f0 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c155c10 0x7f1b0c0cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c155c10 0x7f1b0c0cc970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53790/0 (socket says 172.21.15.119:53790) 2023-10-08T00:56:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 -- 172.21.15.119:0/3549889831 learned_addr learned my addr 172.21.15.119:0/3549889831 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b1139c700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b0c1578b0 0x7f1b0c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b10b9b700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0c0af2f0 0x7f1b0c1453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 -- 172.21.15.119:0/3549889831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0c0af2f0 msgr2=0x7f1b0c1453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0c0af2f0 0x7f1b0c1453e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 -- 172.21.15.119:0/3549889831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b0c1578b0 msgr2=0x7f1b0c0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b0c1578b0 0x7f1b0c0c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 -- 172.21.15.119:0/3549889831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0400bcf0 con 0x7f1b0c155c10 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.533+0000 7f1b0bfff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c155c10 0x7f1b0c0cc970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1af800d860 tx=0x7f1af8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.534+0000 7f1b08ff9700 1 -- 172.21.15.119:0/3549889831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8009610 con 0x7f1b0c155c10 2023-10-08T00:56:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.534+0000 7f1b08ff9700 1 -- 172.21.15.119:0/3549889831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af8009770 con 0x7f1b0c155c10 2023-10-08T00:56:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.534+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c0cb8b0 con 0x7f1b0c155c10 2023-10-08T00:56:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.534+0000 7f1b08ff9700 1 -- 172.21.15.119:0/3549889831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8009b30 con 0x7f1b0c155c10 2023-10-08T00:56:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.534+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c0cceb0 con 0x7f1b0c155c10 2023-10-08T00:56:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.535+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c04c3c0 con 0x7f1b0c155c10 2023-10-08T00:56:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.536+0000 7f1b08ff9700 1 -- 172.21.15.119:0/3549889831 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1af801e460 con 0x7f1b0c155c10 2023-10-08T00:56:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.536+0000 7f1b08ff9700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1af0066f30 0x7f1af00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.536+0000 7f1b08ff9700 1 -- 172.21.15.119:0/3549889831 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f1af80848e0 con 0x7f1b0c155c10 2023-10-08T00:56:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.537+0000 7f1b10b9b700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1af0066f30 0x7f1af00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.537+0000 7f1b10b9b700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1af0066f30 0x7f1af00693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c06fc90 tx=0x7f1b00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.540+0000 7f1b08ff9700 1 -- 172.21.15.119:0/3549889831 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1af8051e00 con 0x7f1b0c155c10 2023-10-08T00:56:36.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.733+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1b0c0ad2e0 con 0x7f1b0c155c10 2023-10-08T00:56:36.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.734+0000 7f1b08ff9700 1 -- 172.21.15.119:0/3549889831 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v27) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1af8055420 con 0x7f1b0c155c10 2023-10-08T00:56:36.735 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:36.735 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T00:56:36.736 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:36.736 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T00:56:36.736 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T00:56:36.736 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:36.736 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T00:56:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1af0066f30 msgr2=0x7f1af00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1af0066f30 0x7f1af00693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c06fc90 tx=0x7f1b00009040 comp rx=0 tx=0).stop 2023-10-08T00:56:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c155c10 msgr2=0x7f1b0c0cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c155c10 0x7f1b0c0cc970 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1af800d860 tx=0x7f1af8008040 comp rx=0 tx=0).stop 2023-10-08T00:56:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 shutdown_connections 2023-10-08T00:56:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b0c0af2f0 0x7f1b0c1453e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1af0066f30 0x7f1af00693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b0c1578b0 0x7f1b0c0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 --2- 172.21.15.119:0/3549889831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b0c155c10 0x7f1b0c0cc970 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 >> 172.21.15.119:0/3549889831 conn(0x7f1b0c06a320 msgr2=0x7f1b0c153540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 shutdown_connections 2023-10-08T00:56:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.736+0000 7f1b12dff700 1 -- 172.21.15.119:0/3549889831 wait complete. 2023-10-08T00:56:36.754 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T00:56:36.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T00:56:36.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.938+0000 7f7a75f9d700 1 Processor -- start 2023-10-08T00:56:36.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.938+0000 7f7a75f9d700 1 -- start start 2023-10-08T00:56:36.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.939+0000 7f7a75f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 0x7f7a700cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.939+0000 7f7a75f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a700cfa00 con 0x7f7a700b03c0 2023-10-08T00:56:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.939+0000 7f7a75f9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a700cb580 con 0x7f7a701591e0 2023-10-08T00:56:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.939+0000 7f7a75f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700cf160 con 0x7f7a700b0790 2023-10-08T00:56:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.939+0000 7f7a74f9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7a700b03c0 0x7f7a700b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45028/0 (socket says 172.21.15.119:45028) 2023-10-08T00:56:36.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a74f9b700 1 -- 172.21.15.119:0/1672222703 learned_addr learned my addr 172.21.15.119:0/1672222703 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:36.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6ffff700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 0x7f7a700cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2556807856 0 0) 0x7f7a700cfa00 con 0x7f7a700b03c0 2023-10-08T00:56:36.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a60003570 con 0x7f7a700b03c0 2023-10-08T00:56:36.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198202293 0 0) 0x7f7a700cb580 con 0x7f7a701591e0 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a700cfa00 con 0x7f7a701591e0 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 624550112 0 0) 0x7f7a60003570 con 0x7f7a700b03c0 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 msgr2=0x7f7a700cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 0x7f7a700cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 >> v1:172.21.15.184:6789/0 conn(0x7f7a701591e0 legacy=0x7f7a70159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a700cf440 con 0x7f7a700b03c0 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f7a64001470 con 0x7f7a700b03c0 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6ffff700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 0x7f7a700cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 >> v1:172.21.15.119:6789/0 conn(0x7f7a700b03c0 legacy=0x7f7a700b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60003c20 0x7f7a60004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60004810 0x7f7a60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.940+0000 7f7a6f7fe700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 0x7f7a600051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700cf160 con 0x7f7a60004810 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6000bbe0 con 0x7f7a60003c20 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6000bda0 con 0x7f7a700b0790 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6ffff700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60004810 0x7f7a60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a74f9b700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60003c20 0x7f7a60004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a7579c700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 0x7f7a600051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6ffff700 1 -- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 msgr2=0x7f7a600051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6ffff700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 0x7f7a600051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6ffff700 1 -- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60003c20 msgr2=0x7f7a60004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6ffff700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60003c20 0x7f7a60004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6ffff700 1 -- 172.21.15.119:0/1672222703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c009420 con 0x7f7a60004810 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a7579c700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a700b0790 0x7f7a600051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.941+0000 7f7a6ffff700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60004810 0x7f7a60004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009df0 tx=0x7f7a5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92142ae68bf4c9e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c009740 con 0x7f7a60004810 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c00e920 con 0x7f7a60004810 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a6f7fe700 1 -- 172.21.15.119:0/1672222703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c00ece0 con 0x7f7a60004810 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a75f9d700 1 -- 172.21.15.119:0/1672222703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60004810 msgr2=0x7f7a60004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a75f9d700 1 --2- 172.21.15.119:0/1672222703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a60004810 0x7f7a60004c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c009df0 tx=0x7f7a5c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a7579c700 1 -- 172.21.15.119:0/1672222703 reap_dead start 2023-10-08T00:56:36.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a75f9d700 1 -- 172.21.15.119:0/1672222703 shutdown_connections 2023-10-08T00:56:36.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a75f9d700 1 -- 172.21.15.119:0/1672222703 >> 172.21.15.119:0/1672222703 conn(0x7f7a70063230 msgr2=0x7f7a70063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:36.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a75f9d700 1 -- 172.21.15.119:0/1672222703 shutdown_connections 2023-10-08T00:56:36.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.942+0000 7f7a75f9d700 1 -- 172.21.15.119:0/1672222703 wait complete. 2023-10-08T00:56:36.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.944+0000 7f7a75f9d700 1 Processor -- start 2023-10-08T00:56:36.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.944+0000 7f7a75f9d700 1 -- start start 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a75f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60004810 0x7f7a701499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a75f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a7015a9d0 0x7f7a700ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a75f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a700ce600 0x7f7a70169b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a75f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a70154f00 con 0x7f7a700ce600 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a75f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a700cea80 con 0x7f7a60004810 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a75f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7016aae0 con 0x7f7a7015a9d0 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a7579c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a700ce600 0x7f7a70169b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a7579c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a700ce600 0x7f7a70169b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53814/0 (socket says 172.21.15.119:53814) 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a7579c700 1 -- 172.21.15.119:0/986328868 learned_addr learned my addr 172.21.15.119:0/986328868 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:36.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a74f9b700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60004810 0x7f7a701499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a6ffff700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a7015a9d0 0x7f7a700ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a7579c700 1 -- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a7015a9d0 msgr2=0x7f7a700ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a7579c700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a7015a9d0 0x7f7a700ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a7579c700 1 -- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60004810 msgr2=0x7f7a701499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:36.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a7579c700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60004810 0x7f7a701499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:36.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a7579c700 1 -- 172.21.15.119:0/986328868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a70149f00 con 0x7f7a700ce600 2023-10-08T00:56:36.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.945+0000 7f7a74f9b700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60004810 0x7f7a701499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:36.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.946+0000 7f7a7579c700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a700ce600 0x7f7a70169b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7a700b1cf0 tx=0x7f7a68009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.946+0000 7f7a6cff9700 1 -- 172.21.15.119:0/986328868 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68017070 con 0x7f7a700ce600 2023-10-08T00:56:36.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.946+0000 7f7a6cff9700 1 -- 172.21.15.119:0/986328868 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a680032e0 con 0x7f7a700ce600 2023-10-08T00:56:36.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.946+0000 7f7a6cff9700 1 -- 172.21.15.119:0/986328868 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a68006ba0 con 0x7f7a700ce600 2023-10-08T00:56:36.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.946+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7016a800 con 0x7f7a700ce600 2023-10-08T00:56:36.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.946+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7015a1d0 con 0x7f7a700ce600 2023-10-08T00:56:36.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.947+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7004c3c0 con 0x7f7a700ce600 2023-10-08T00:56:36.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.948+0000 7f7a6cff9700 1 -- 172.21.15.119:0/986328868 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a68001680 con 0x7f7a700ce600 2023-10-08T00:56:36.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.948+0000 7f7a6cff9700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7a54066f30 0x7f7a540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:36.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.949+0000 7f7a6cff9700 1 -- 172.21.15.119:0/986328868 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f7a68083cc0 con 0x7f7a700ce600 2023-10-08T00:56:36.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.949+0000 7f7a74f9b700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7a54066f30 0x7f7a540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:36.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.949+0000 7f7a74f9b700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7a54066f30 0x7f7a540693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a640015a0 tx=0x7f7a64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:36.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:36.952+0000 7f7a6cff9700 1 -- 172.21.15.119:0/986328868 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a6808a080 con 0x7f7a700ce600 2023-10-08T00:56:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:37.144+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7a700cf720 con 0x7f7a700ce600 2023-10-08T00:56:38.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.102+0000 7f7a6cff9700 1 -- 172.21.15.119:0/986328868 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v28) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7a680511e0 con 0x7f7a700ce600 2023-10-08T00:56:38.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.105+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7a54066f30 msgr2=0x7f7a540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:38.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.105+0000 7f7a75f9d700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7a54066f30 0x7f7a540693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7a640015a0 tx=0x7f7a64009040 comp rx=0 tx=0).stop 2023-10-08T00:56:38.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.105+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a700ce600 msgr2=0x7f7a70169b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:38.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.105+0000 7f7a75f9d700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a700ce600 0x7f7a70169b50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7a700b1cf0 tx=0x7f7a68009040 comp rx=0 tx=0).stop 2023-10-08T00:56:38.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.106+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 shutdown_connections 2023-10-08T00:56:38.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.106+0000 7f7a75f9d700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7a54066f30 0x7f7a540693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:38.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.106+0000 7f7a75f9d700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a700ce600 0x7f7a70169b50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:38.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.106+0000 7f7a75f9d700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a60004810 0x7f7a701499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:38.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.106+0000 7f7a75f9d700 1 --2- 172.21.15.119:0/986328868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a7015a9d0 0x7f7a700ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:38.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.106+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 >> 172.21.15.119:0/986328868 conn(0x7f7a70063230 msgr2=0x7f7a70063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:38.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.106+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 shutdown_connections 2023-10-08T00:56:38.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.106+0000 7f7a75f9d700 1 -- 172.21.15.119:0/986328868 wait complete. 2023-10-08T00:56:38.109 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T00:56:38.129 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T00:56:38.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.323+0000 7ffbcc3ff700 1 Processor -- start 2023-10-08T00:56:38.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.323+0000 7ffbcc3ff700 1 -- start start 2023-10-08T00:56:38.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.324+0000 7ffbcc3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbc406d6c0 0x7ffbc4156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:38.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.324+0000 7ffbcc3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc4157420 con 0x7ffbc4069fa0 2023-10-08T00:56:38.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.324+0000 7ffbcc3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc4156340 con 0x7ffbc40c85c0 2023-10-08T00:56:38.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.324+0000 7ffbcc3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc40c8990 con 0x7ffbc406d6c0 2023-10-08T00:56:38.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.325+0000 7ffbc999a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffbc4069fa0 0x7ffbc40c6ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45042/0 (socket says 172.21.15.119:45042) 2023-10-08T00:56:38.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.325+0000 7ffbc999a700 1 -- 172.21.15.119:0/4158943488 learned_addr learned my addr 172.21.15.119:0/4158943488 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:38.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.325+0000 7ffbca99c700 1 --2- 172.21.15.119:0/4158943488 >> v2:172.21.15.119:3300/0 conn(0x7ffbc406d6c0 0x7ffbc4156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:38.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.325+0000 7ffbc9199700 1 -- 172.21.15.119:0/4158943488 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710534168 0 0) 0x7ffbc4157420 con 0x7ffbc4069fa0 2023-10-08T00:56:38.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.325+0000 7ffbc9199700 1 -- 172.21.15.119:0/4158943488 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbb0004680 con 0x7ffbc4069fa0 2023-10-08T00:56:38.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.325+0000 7ffbca99c700 1 -- 172.21.15.119:0/4158943488 >> v1:172.21.15.184:6789/0 conn(0x7ffbc40c85c0 legacy=0x7ffbc4069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:38.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.325+0000 7ffbca99c700 1 -- 172.21.15.119:0/4158943488 >> v1:172.21.15.119:6789/0 conn(0x7ffbc4069fa0 legacy=0x7ffbc40c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:38.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.325+0000 7ffbca99c700 1 -- 172.21.15.119:0/4158943488 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc406ee20 con 0x7ffbc406d6c0 2023-10-08T00:56:38.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbca99c700 1 --2- 172.21.15.119:0/4158943488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc4156ee0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009e60 tx=0x7ffbbc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec9ff4c640bb8b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbc9199700 1 -- 172.21.15.119:0/4158943488 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc00e070 con 0x7ffbc406d6c0 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbc9199700 1 -- 172.21.15.119:0/4158943488 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbbc00bb60 con 0x7ffbc406d6c0 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbc9199700 1 -- 172.21.15.119:0/4158943488 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc012430 con 0x7ffbc406d6c0 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/4158943488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 msgr2=0x7ffbc4156ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbcc3ff700 1 --2- 172.21.15.119:0/4158943488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc4156ee0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc009e60 tx=0x7ffbbc009780 comp rx=0 tx=0).stop 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/4158943488 shutdown_connections 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbcc3ff700 1 --2- 172.21.15.119:0/4158943488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc4156ee0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/4158943488 >> 172.21.15.119:0/4158943488 conn(0x7ffbc40cc540 msgr2=0x7ffbc40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/4158943488 shutdown_connections 2023-10-08T00:56:38.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.326+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/4158943488 wait complete. 2023-10-08T00:56:38.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.328+0000 7ffbcc3ff700 1 Processor -- start 2023-10-08T00:56:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.328+0000 7ffbcc3ff700 1 -- start start 2023-10-08T00:56:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbcc3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc4069fa0 0x7ffbc40a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbcc3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc41553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbcc3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc40c85c0 0x7ffbc40bfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbcc3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40c1030 con 0x7ffbc406d6c0 2023-10-08T00:56:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbcc3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40c14c0 con 0x7ffbc4069fa0 2023-10-08T00:56:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbcc3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40c0420 con 0x7ffbc40c85c0 2023-10-08T00:56:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbc999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc41553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbca19b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc4069fa0 0x7ffbc40a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbc999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc41553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53838/0 (socket says 172.21.15.119:53838) 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbc999a700 1 -- 172.21.15.119:0/2428407481 learned_addr learned my addr 172.21.15.119:0/2428407481 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.329+0000 7ffbca99c700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc40c85c0 0x7ffbc40bfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbc999a700 1 -- 172.21.15.119:0/2428407481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc40c85c0 msgr2=0x7ffbc40bfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbc999a700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc40c85c0 0x7ffbc40bfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbc999a700 1 -- 172.21.15.119:0/2428407481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc4069fa0 msgr2=0x7ffbc40a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbc999a700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc4069fa0 0x7ffbc40a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:38.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbc999a700 1 -- 172.21.15.119:0/2428407481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc009a20 con 0x7ffbc406d6c0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbc999a700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc41553f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c7c0 tx=0x7ffbb4007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbba7fc700 1 -- 172.21.15.119:0/2428407481 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4005c00 con 0x7ffbc406d6c0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbba7fc700 1 -- 172.21.15.119:0/2428407481 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4005d60 con 0x7ffbc406d6c0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbba7fc700 1 -- 172.21.15.119:0/2428407481 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4007320 con 0x7ffbc406d6c0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc40c0700 con 0x7ffbc406d6c0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.330+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc40c0cc0 con 0x7ffbc406d6c0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.331+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc40c11a0 con 0x7ffbc406d6c0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.332+0000 7ffbba7fc700 1 -- 172.21.15.119:0/2428407481 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbb401b3b0 con 0x7ffbc406d6c0 2023-10-08T00:56:38.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.333+0000 7ffbba7fc700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffba8066f30 0x7ffba80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:38.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.334+0000 7ffbba7fc700 1 -- 172.21.15.119:0/2428407481 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5223+0+0 (secure 0 0 0) 0x7ffbb40838d0 con 0x7ffbc406d6c0 2023-10-08T00:56:38.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.334+0000 7ffbca19b700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffba8066f30 0x7ffba80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:38.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.334+0000 7ffbca19b700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffba8066f30 0x7ffba80693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40aeb50 tx=0x7ffbc0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:38.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.338+0000 7ffbba7fc700 1 -- 172.21.15.119:0/2428407481 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbb4050c80 con 0x7ffbc406d6c0 2023-10-08T00:56:38.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:38.531+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ffbc40718b0 con 0x7ffbc406d6c0 2023-10-08T00:56:39.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.107+0000 7ffbba7fc700 1 -- 172.21.15.119:0/2428407481 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v29) v1 ==== 170+0+0 (secure 0 0 0) 0x7ffbb4017070 con 0x7ffbc406d6c0 2023-10-08T00:56:39.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.115+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffba8066f30 msgr2=0x7ffba80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.115+0000 7ffbcc3ff700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffba8066f30 0x7ffba80693e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40aeb50 tx=0x7ffbc0009040 comp rx=0 tx=0).stop 2023-10-08T00:56:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.115+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 msgr2=0x7ffbc41553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:39.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.115+0000 7ffbcc3ff700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc41553f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c7c0 tx=0x7ffbb4007dd0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.117+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 shutdown_connections 2023-10-08T00:56:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.117+0000 7ffbcc3ff700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc40c85c0 0x7ffbc40bfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.117+0000 7ffbcc3ff700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffba8066f30 0x7ffba80693e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.117+0000 7ffbcc3ff700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc4069fa0 0x7ffbc40a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.118+0000 7ffbcc3ff700 1 --2- 172.21.15.119:0/2428407481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc406d6c0 0x7ffbc41553f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.118+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 >> 172.21.15.119:0/2428407481 conn(0x7ffbc40cc540 msgr2=0x7ffbc40c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:39.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.122+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 shutdown_connections 2023-10-08T00:56:39.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.123+0000 7ffbcc3ff700 1 -- 172.21.15.119:0/2428407481 wait complete. 2023-10-08T00:56:39.128 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T00:56:39.148 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T00:56:39.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.332+0000 7f731b59e700 1 Processor -- start 2023-10-08T00:56:39.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.332+0000 7f731b59e700 1 -- start start 2023-10-08T00:56:39.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.333+0000 7f731b59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f731c0b4e70 0x7f731c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:39.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.333+0000 7f731b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b1b40 0x7f731c15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:39.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.333+0000 7f731b59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f731c0cfb00 con 0x7f731c0b16d0 2023-10-08T00:56:39.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.333+0000 7f731b59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f731c0b5c40 con 0x7f731c0b4e70 2023-10-08T00:56:39.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.333+0000 7f731b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c0b5f20 con 0x7f731c0b1b40 2023-10-08T00:56:39.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.334+0000 7f731ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b1b40 0x7f731c15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:39.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.334+0000 7f731ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b1b40 0x7f731c15d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:53062/0 (socket says 172.21.15.119:53062) 2023-10-08T00:56:39.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.334+0000 7f731ad9d700 1 -- 172.21.15.119:0/801819312 learned_addr learned my addr 172.21.15.119:0/801819312 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:39.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.334+0000 7f731a59c700 1 --2- 172.21.15.119:0/801819312 >> v2:172.21.15.184:3300/0 conn(0x7f731c0b4e70 0x7f731c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:39.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.334+0000 7f731ad9d700 1 -- 172.21.15.119:0/801819312 >> v2:172.21.15.184:3300/0 conn(0x7f731c0b4e70 msgr2=0x7f731c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:39.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.334+0000 7f731ad9d700 1 --2- 172.21.15.119:0/801819312 >> v2:172.21.15.184:3300/0 conn(0x7f731c0b4e70 0x7f731c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.334+0000 7f731ad9d700 1 -- 172.21.15.119:0/801819312 >> v1:172.21.15.184:6789/0 conn(0x7f731c0b16d0 legacy=0x7f731c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:39.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.334+0000 7f731ad9d700 1 -- 172.21.15.119:0/801819312 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c0b6200 con 0x7f731c0b1b40 2023-10-08T00:56:39.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.335+0000 7f731ad9d700 1 --2- 172.21.15.119:0/801819312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b1b40 0x7f731c15d950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7314009f20 tx=0x7f7314009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6979efca62f5a479 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:39.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.335+0000 7f731959a700 1 -- 172.21.15.119:0/801819312 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731400e070 con 0x7f731c0b1b40 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.335+0000 7f731959a700 1 -- 172.21.15.119:0/801819312 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731400ba80 con 0x7f731c0b1b40 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.335+0000 7f731959a700 1 -- 172.21.15.119:0/801819312 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140092e0 con 0x7f731c0b1b40 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.336+0000 7f731b59e700 1 -- 172.21.15.119:0/801819312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b1b40 msgr2=0x7f731c15d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.336+0000 7f731b59e700 1 --2- 172.21.15.119:0/801819312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b1b40 0x7f731c15d950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7314009f20 tx=0x7f7314009840 comp rx=0 tx=0).stop 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.336+0000 7f731b59e700 1 -- 172.21.15.119:0/801819312 shutdown_connections 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.336+0000 7f731b59e700 1 --2- 172.21.15.119:0/801819312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b1b40 0x7f731c15d950 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.336+0000 7f731b59e700 1 --2- 172.21.15.119:0/801819312 >> v2:172.21.15.184:3300/0 conn(0x7f731c0b4e70 0x7f731c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.336+0000 7f731b59e700 1 -- 172.21.15.119:0/801819312 >> 172.21.15.119:0/801819312 conn(0x7f731c0d10c0 msgr2=0x7f731c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.336+0000 7f731b59e700 1 -- 172.21.15.119:0/801819312 shutdown_connections 2023-10-08T00:56:39.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.336+0000 7f731b59e700 1 -- 172.21.15.119:0/801819312 wait complete. 2023-10-08T00:56:39.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.337+0000 7f731b59e700 1 Processor -- start 2023-10-08T00:56:39.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.338+0000 7f731b59e700 1 -- start start 2023-10-08T00:56:39.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b16d0 0x7f731c16a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:39.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f731c0b1b40 0x7f731c16d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:39.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731c0b4e70 0x7f731c1459c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:39.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731b59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c16e470 con 0x7f731c0b4e70 2023-10-08T00:56:39.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731b59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f731c169ea0 con 0x7f731c0b1b40 2023-10-08T00:56:39.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f731c16a420 con 0x7f731c0b16d0 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731c0b4e70 0x7f731c1459c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731c0b4e70 0x7f731c1459c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53846/0 (socket says 172.21.15.119:53846) 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731ad9d700 1 -- 172.21.15.119:0/1783172820 learned_addr learned my addr 172.21.15.119:0/1783172820 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731a59c700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b16d0 0x7f731c16a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f7319d9b700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f731c0b1b40 0x7f731c16d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731ad9d700 1 -- 172.21.15.119:0/1783172820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b16d0 msgr2=0x7f731c16a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731ad9d700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b16d0 0x7f731c16a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731ad9d700 1 -- 172.21.15.119:0/1783172820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f731c0b1b40 msgr2=0x7f731c16d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731ad9d700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f731c0b1b40 0x7f731c16d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.339+0000 7f731ad9d700 1 -- 172.21.15.119:0/1783172820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314009ae0 con 0x7f731c0b4e70 2023-10-08T00:56:39.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.340+0000 7f731ad9d700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731c0b4e70 0x7f731c1459c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f731400b6c0 tx=0x7f7314018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:39.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.340+0000 7f730a7fc700 1 -- 172.21.15.119:0/1783172820 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140094e0 con 0x7f731c0b4e70 2023-10-08T00:56:39.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.340+0000 7f730a7fc700 1 -- 172.21.15.119:0/1783172820 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73140032f0 con 0x7f731c0b4e70 2023-10-08T00:56:39.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.340+0000 7f730a7fc700 1 -- 172.21.15.119:0/1783172820 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314026690 con 0x7f731c0b4e70 2023-10-08T00:56:39.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.340+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c16deb0 con 0x7f731c0b4e70 2023-10-08T00:56:39.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.340+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c16a130 con 0x7f731c0b4e70 2023-10-08T00:56:39.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.341+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731c16aea0 con 0x7f731c0b4e70 2023-10-08T00:56:39.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.342+0000 7f730a7fc700 1 -- 172.21.15.119:0/1783172820 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7314003040 con 0x7f731c0b4e70 2023-10-08T00:56:39.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.343+0000 7f730a7fc700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f72fc066f30 0x7f72fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:39.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.343+0000 7f730a7fc700 1 -- 172.21.15.119:0/1783172820 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f7314086710 con 0x7f731c0b4e70 2023-10-08T00:56:39.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.343+0000 7f731a59c700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f72fc066f30 0x7f72fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:39.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.344+0000 7f731a59c700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f72fc066f30 0x7f72fc0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f731c0cbd10 tx=0x7f730c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:39.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.347+0000 7f730a7fc700 1 -- 172.21.15.119:0/1783172820 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73140539c0 con 0x7f731c0b4e70 2023-10-08T00:56:39.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:39.580+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f731c16a560 con 0x7f731c0b4e70 2023-10-08T00:56:40.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.137+0000 7f730a7fc700 1 -- 172.21.15.119:0/1783172820 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7314056fe0 con 0x7f731c0b4e70 2023-10-08T00:56:40.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.144+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f72fc066f30 msgr2=0x7f72fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.144+0000 7f731b59e700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f72fc066f30 0x7f72fc0693e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f731c0cbd10 tx=0x7f730c008040 comp rx=0 tx=0).stop 2023-10-08T00:56:40.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.144+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731c0b4e70 msgr2=0x7f731c1459c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.144+0000 7f731b59e700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731c0b4e70 0x7f731c1459c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f731400b6c0 tx=0x7f7314018000 comp rx=0 tx=0).stop 2023-10-08T00:56:40.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.146+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 shutdown_connections 2023-10-08T00:56:40.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.146+0000 7f731b59e700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f731c0b16d0 0x7f731c16a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.146+0000 7f731b59e700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f72fc066f30 0x7f72fc0693e0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f731c0cbd10 tx=0x7f730c008040 comp rx=0 tx=0).stop 2023-10-08T00:56:40.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.146+0000 7f731b59e700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731c0b4e70 0x7f731c1459c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.146+0000 7f731b59e700 1 --2- 172.21.15.119:0/1783172820 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f731c0b1b40 0x7f731c16d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.146+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 >> 172.21.15.119:0/1783172820 conn(0x7f731c0d10c0 msgr2=0x7f731c159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:40.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.152+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 shutdown_connections 2023-10-08T00:56:40.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.154+0000 7f731b59e700 1 -- 172.21.15.119:0/1783172820 wait complete. 2023-10-08T00:56:40.158 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 4 and data pool 5 2023-10-08T00:56:40.181 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:40.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.360+0000 7fe268d9d700 1 Processor -- start 2023-10-08T00:56:40.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.360+0000 7fe268d9d700 1 -- start start 2023-10-08T00:56:40.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.361+0000 7fe268d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2640ae710 0x7fe2640c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.361+0000 7fe268d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2640ca860 con 0x7fe264156310 2023-10-08T00:56:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.361+0000 7fe268d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2640cab40 con 0x7fe2641566e0 2023-10-08T00:56:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.361+0000 7fe268d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640cae20 con 0x7fe2640ae710 2023-10-08T00:56:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.361+0000 7fe2637fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe264156310 0x7fe2640c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45044/0 (socket says 172.21.15.119:45044) 2023-10-08T00:56:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.361+0000 7fe2637fe700 1 -- 172.21.15.119:0/3161421523 learned_addr learned my addr 172.21.15.119:0/3161421523 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe263fff700 1 --2- 172.21.15.119:0/3161421523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2640ae710 0x7fe2640c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe2627fc700 1 -- 172.21.15.119:0/3161421523 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1174960894 0 0) 0x7fe2640ca860 con 0x7fe264156310 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe2627fc700 1 -- 172.21.15.119:0/3161421523 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe24c003570 con 0x7fe264156310 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe2627fc700 1 -- 172.21.15.119:0/3161421523 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227618801 0 0) 0x7fe2640cab40 con 0x7fe2641566e0 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe2627fc700 1 -- 172.21.15.119:0/3161421523 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2640ca860 con 0x7fe2641566e0 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe2627fc700 1 -- 172.21.15.119:0/3161421523 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3154006159 0 0) 0x7fe24c003570 con 0x7fe264156310 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe263fff700 1 -- 172.21.15.119:0/3161421523 >> v1:172.21.15.184:6789/0 conn(0x7fe2641566e0 legacy=0x7fe2640ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe263fff700 1 -- 172.21.15.119:0/3161421523 >> v1:172.21.15.119:6789/0 conn(0x7fe264156310 legacy=0x7fe2640c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe263fff700 1 -- 172.21.15.119:0/3161421523 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640cb100 con 0x7fe2640ae710 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.362+0000 7fe263fff700 1 --2- 172.21.15.119:0/3161421523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2640ae710 0x7fe2640c6830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009e90 tx=0x7fe25c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb23715bdb4efeb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe2627fc700 1 -- 172.21.15.119:0/3161421523 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c00e070 con 0x7fe2640ae710 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe2627fc700 1 -- 172.21.15.119:0/3161421523 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c00b870 con 0x7fe2640ae710 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe2627fc700 1 -- 172.21.15.119:0/3161421523 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe254001470 con 0x7fe2640ae710 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe268d9d700 1 -- 172.21.15.119:0/3161421523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2640ae710 msgr2=0x7fe2640c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe268d9d700 1 --2- 172.21.15.119:0/3161421523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2640ae710 0x7fe2640c6830 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009e90 tx=0x7fe25c009780 comp rx=0 tx=0).stop 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe268d9d700 1 -- 172.21.15.119:0/3161421523 shutdown_connections 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe268d9d700 1 --2- 172.21.15.119:0/3161421523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2640ae710 0x7fe2640c6830 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe268d9d700 1 -- 172.21.15.119:0/3161421523 >> 172.21.15.119:0/3161421523 conn(0x7fe2640cc1f0 msgr2=0x7fe2640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe268d9d700 1 -- 172.21.15.119:0/3161421523 shutdown_connections 2023-10-08T00:56:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.363+0000 7fe268d9d700 1 -- 172.21.15.119:0/3161421523 wait complete. 2023-10-08T00:56:40.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.365+0000 7fe268d9d700 1 Processor -- start 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.365+0000 7fe268d9d700 1 -- start start 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe268d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2640ae710 0x7fe264140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe268d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe264156310 0x7fe264153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe268d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2641566e0 0x7fe264153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe268d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26416c090 con 0x7fe264156310 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe268d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26416c370 con 0x7fe2640ae710 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe268d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26416c650 con 0x7fe2641566e0 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe262ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe264156310 0x7fe264153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe262ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe264156310 0x7fe264153910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53854/0 (socket says 172.21.15.119:53854) 2023-10-08T00:56:40.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe262ffd700 1 -- 172.21.15.119:0/2510556793 learned_addr learned my addr 172.21.15.119:0/2510556793 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:40.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe262ffd700 1 -- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2641566e0 msgr2=0x7fe264153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe263fff700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2641566e0 0x7fe264153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe2637fe700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2640ae710 0x7fe264140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe262ffd700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2641566e0 0x7fe264153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe262ffd700 1 -- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2640ae710 msgr2=0x7fe264140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe262ffd700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2640ae710 0x7fe264140d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe262ffd700 1 -- 172.21.15.119:0/2510556793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c009a20 con 0x7fe264156310 2023-10-08T00:56:40.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe263fff700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2641566e0 0x7fe264153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:40.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.366+0000 7fe2637fe700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2640ae710 0x7fe264140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:40.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.367+0000 7fe262ffd700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe264156310 0x7fe264153910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe25800c7c0 tx=0x7fe258007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:40.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.367+0000 7fe24b7fe700 1 -- 172.21.15.119:0/2510556793 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258010510 con 0x7fe264156310 2023-10-08T00:56:40.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.367+0000 7fe24b7fe700 1 -- 172.21.15.119:0/2510556793 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe258010690 con 0x7fe264156310 2023-10-08T00:56:40.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.367+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26416c930 con 0x7fe264156310 2023-10-08T00:56:40.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.367+0000 7fe24b7fe700 1 -- 172.21.15.119:0/2510556793 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258010a50 con 0x7fe264156310 2023-10-08T00:56:40.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.367+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe264141280 con 0x7fe264156310 2023-10-08T00:56:40.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.368+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26416ce50 con 0x7fe264156310 2023-10-08T00:56:40.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.369+0000 7fe24b7fe700 1 -- 172.21.15.119:0/2510556793 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe25801b400 con 0x7fe264156310 2023-10-08T00:56:40.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.370+0000 7fe24b7fe700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe250066f30 0x7fe2500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.370+0000 7fe24b7fe700 1 -- 172.21.15.119:0/2510556793 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe258084380 con 0x7fe264156310 2023-10-08T00:56:40.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.370+0000 7fe2637fe700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe250066f30 0x7fe2500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.370+0000 7fe2637fe700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe250066f30 0x7fe2500693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe254007bc0 tx=0x7fe254009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:40.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.373+0000 7fe24b7fe700 1 -- 172.21.15.119:0/2510556793 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe26416ce50 con 0x7fe264156310 2023-10-08T00:56:40.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.563+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe26416ce50 con 0x7fe264156310 2023-10-08T00:56:40.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.564+0000 7fe24b7fe700 1 -- 172.21.15.119:0/2510556793 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15411 (secure 0 0 0) 0x7fe26416ce50 con 0x7fe264156310 2023-10-08T00:56:40.566 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:40.566 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":30,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:56:39.582546+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:56:37.182342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:56:38.569786+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:40.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe250066f30 msgr2=0x7fe2500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe250066f30 0x7fe2500693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe254007bc0 tx=0x7fe254009040 comp rx=0 tx=0).stop 2023-10-08T00:56:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe264156310 msgr2=0x7fe264153910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe264156310 0x7fe264153910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe25800c7c0 tx=0x7fe258007de0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 shutdown_connections 2023-10-08T00:56:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2641566e0 0x7fe264153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe250066f30 0x7fe2500693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2640ae710 0x7fe264140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 --2- 172.21.15.119:0/2510556793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe264156310 0x7fe264153910 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 >> 172.21.15.119:0/2510556793 conn(0x7fe2640cc1f0 msgr2=0x7fe26406a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:40.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 shutdown_connections 2023-10-08T00:56:40.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.567+0000 7fe268d9d700 1 -- 172.21.15.119:0/2510556793 wait complete. 2023-10-08T00:56:40.586 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:40.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.764+0000 7f6f70dff700 1 Processor -- start 2023-10-08T00:56:40.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.764+0000 7f6f70dff700 1 -- start start 2023-10-08T00:56:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f70dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6f6c0b1420 0x7f6f6c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f70dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6f6c0ccf90 0x7f6f6c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f70dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6c0cc300 con 0x7f6f6c0cbf30 2023-10-08T00:56:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f70dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6c156bf0 con 0x7f6f6c0ccf90 2023-10-08T00:56:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f70dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6c0ae950 con 0x7f6f6c0b1420 2023-10-08T00:56:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6f6c0ccf90 0x7f6f6c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6a59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6f6c0cbf30 0x7f6f6c0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45048/0 (socket says 172.21.15.119:45048) 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6a59c700 1 -- 172.21.15.119:0/1547310205 learned_addr learned my addr 172.21.15.119:0/1547310205 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f69d9b700 1 --2- 172.21.15.119:0/1547310205 >> v2:172.21.15.184:3300/0 conn(0x7f6f6c0b1420 0x7f6f6c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6959a700 1 -- 172.21.15.119:0/1547310205 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200703913 0 0) 0x7f6f6c0cc300 con 0x7f6f6c0cbf30 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6959a700 1 -- 172.21.15.119:0/1547310205 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f54003570 con 0x7f6f6c0cbf30 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6ad9d700 1 -- 172.21.15.119:0/1547310205 >> v2:172.21.15.184:3300/0 conn(0x7f6f6c0b1420 msgr2=0x7f6f6c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6ad9d700 1 --2- 172.21.15.119:0/1547310205 >> v2:172.21.15.184:3300/0 conn(0x7f6f6c0b1420 0x7f6f6c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6ad9d700 1 -- 172.21.15.119:0/1547310205 >> v1:172.21.15.119:6789/0 conn(0x7f6f6c0cbf30 legacy=0x7f6f6c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.766+0000 7f6f6ad9d700 1 -- 172.21.15.119:0/1547310205 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c0ad370 con 0x7f6f6c0ccf90 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.767+0000 7f6f6ad9d700 1 --2- 172.21.15.119:0/1547310205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0ccf90 0x7f6f6c0c9740 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0c9280 tx=0x7f6f64009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520d81e4216092ab server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.767+0000 7f6f6959a700 1 -- 172.21.15.119:0/1547310205 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400e070 con 0x7f6f6c0ccf90 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.767+0000 7f6f6959a700 1 -- 172.21.15.119:0/1547310205 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400ba70 con 0x7f6f6c0ccf90 2023-10-08T00:56:40.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.767+0000 7f6f6959a700 1 -- 172.21.15.119:0/1547310205 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400be30 con 0x7f6f6c0ccf90 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.767+0000 7f6f70dff700 1 -- 172.21.15.119:0/1547310205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0ccf90 msgr2=0x7f6f6c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.767+0000 7f6f70dff700 1 --2- 172.21.15.119:0/1547310205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0ccf90 0x7f6f6c0c9740 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0c9280 tx=0x7f6f64009780 comp rx=0 tx=0).stop 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.768+0000 7f6f70dff700 1 -- 172.21.15.119:0/1547310205 shutdown_connections 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.768+0000 7f6f70dff700 1 --2- 172.21.15.119:0/1547310205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0ccf90 0x7f6f6c0c9740 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.768+0000 7f6f70dff700 1 --2- 172.21.15.119:0/1547310205 >> v2:172.21.15.184:3300/0 conn(0x7f6f6c0b1420 0x7f6f6c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.768+0000 7f6f70dff700 1 -- 172.21.15.119:0/1547310205 >> 172.21.15.119:0/1547310205 conn(0x7f6f6c06a320 msgr2=0x7f6f6c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.768+0000 7f6f70dff700 1 -- 172.21.15.119:0/1547310205 shutdown_connections 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.768+0000 7f6f70dff700 1 -- 172.21.15.119:0/1547310205 wait complete. 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.769+0000 7f6f70dff700 1 Processor -- start 2023-10-08T00:56:40.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.769+0000 7f6f70dff700 1 -- start start 2023-10-08T00:56:40.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f70dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0b1420 0x7f6f6c156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f70dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6c0cbf30 0x7f6f6c0ac950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f70dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6c0ccf90 0x7f6f6c0c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f70dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c16a3a0 con 0x7f6f6c0b1420 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f70dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c169790 con 0x7f6f6c0cbf30 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f70dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c169a70 con 0x7f6f6c0ccf90 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f6a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0b1420 0x7f6f6c156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f6a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0b1420 0x7f6f6c156080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53868/0 (socket says 172.21.15.119:53868) 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.770+0000 7f6f6a59c700 1 -- 172.21.15.119:0/3592803221 learned_addr learned my addr 172.21.15.119:0/3592803221 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f69d9b700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6c0cbf30 0x7f6f6c0ac950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f6ad9d700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6c0ccf90 0x7f6f6c0c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f6a59c700 1 -- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6c0ccf90 msgr2=0x7f6f6c0c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f6a59c700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6c0ccf90 0x7f6f6c0c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f6a59c700 1 -- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6c0cbf30 msgr2=0x7f6f6c0ac950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f6a59c700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6c0cbf30 0x7f6f6c0ac950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f6a59c700 1 -- 172.21.15.119:0/3592803221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64009a20 con 0x7f6f6c0b1420 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f69d9b700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6c0cbf30 0x7f6f6c0ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f6ad9d700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6c0ccf90 0x7f6f6c0c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f6a59c700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0b1420 0x7f6f6c156080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c7c0 tx=0x7f6f5c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f5a7fc700 1 -- 172.21.15.119:0/3592803221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c005b30 con 0x7f6f6c0b1420 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f5a7fc700 1 -- 172.21.15.119:0/3592803221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c005c90 con 0x7f6f6c0b1420 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6c169d50 con 0x7f6f6c0b1420 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f5a7fc700 1 -- 172.21.15.119:0/3592803221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c007340 con 0x7f6f6c0b1420 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.771+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c1565c0 con 0x7f6f6c0b1420 2023-10-08T00:56:40.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.772+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6c04c3c0 con 0x7f6f6c0b1420 2023-10-08T00:56:40.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.773+0000 7f6f5a7fc700 1 -- 172.21.15.119:0/3592803221 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f5c01b400 con 0x7f6f6c0b1420 2023-10-08T00:56:40.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.774+0000 7f6f5a7fc700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6f4c066f30 0x7f6f4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:40.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.774+0000 7f6f5a7fc700 1 -- 172.21.15.119:0/3592803221 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f6f5c084070 con 0x7f6f6c0b1420 2023-10-08T00:56:40.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.774+0000 7f6f69d9b700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6f4c066f30 0x7f6f4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:40.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.775+0000 7f6f69d9b700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6f4c066f30 0x7f6f4c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006f00 tx=0x7f6f60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:40.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.778+0000 7f6f5a7fc700 1 -- 172.21.15.119:0/3592803221 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f5c0512c0 con 0x7f6f6c0b1420 2023-10-08T00:56:40.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.977+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f6c0ad650 con 0x7f6f6c0b1420 2023-10-08T00:56:40.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.978+0000 7f6f5a7fc700 1 -- 172.21.15.119:0/3592803221 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+15411 (secure 0 0 0) 0x7f6f5c0548e0 con 0x7f6f6c0b1420 2023-10-08T00:56:40.979 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:40.979 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":30,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:56:39.582546+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:56:37.182342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:56:38.569786+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6f4c066f30 msgr2=0x7f6f4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6f4c066f30 0x7f6f4c0693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006f00 tx=0x7f6f60008040 comp rx=0 tx=0).stop 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0b1420 msgr2=0x7f6f6c156080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0b1420 0x7f6f6c156080 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00c7c0 tx=0x7f6f5c007e00 comp rx=0 tx=0).stop 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 shutdown_connections 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6c0ccf90 0x7f6f6c0c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6f4c066f30 0x7f6f4c0693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6c0b1420 0x7f6f6c156080 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 --2- 172.21.15.119:0/3592803221 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6c0cbf30 0x7f6f6c0ac950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 >> 172.21.15.119:0/3592803221 conn(0x7f6f6c06a320 msgr2=0x7f6f6c0c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 shutdown_connections 2023-10-08T00:56:40.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:40.980+0000 7f6f70dff700 1 -- 172.21.15.119:0/3592803221 wait complete. 2023-10-08T00:56:40.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T00:56:41.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.186+0000 7ff0463ff700 1 Processor -- start 2023-10-08T00:56:41.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.187+0000 7ff0463ff700 1 -- start start 2023-10-08T00:56:41.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.188+0000 7ff0463ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff0401568e0 0x7ff0401616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:41.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.188+0000 7ff0463ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04015b0d0 con 0x7ff0400c83f0 2023-10-08T00:56:41.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.188+0000 7ff0463ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04015a4c0 con 0x7ff0400ccee0 2023-10-08T00:56:41.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.188+0000 7ff0463ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04015a7a0 con 0x7ff0401568e0 2023-10-08T00:56:41.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.188+0000 7ff03ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff0400c83f0 0x7ff0400c7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45058/0 (socket says 172.21.15.119:45058) 2023-10-08T00:56:41.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.188+0000 7ff03ffff700 1 -- 172.21.15.119:0/263752826 learned_addr learned my addr 172.21.15.119:0/263752826 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:41.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.188+0000 7ff04499c700 1 --2- 172.21.15.119:0/263752826 >> v2:172.21.15.184:3300/0 conn(0x7ff0401568e0 0x7ff0401616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:41.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff03effd700 1 -- 172.21.15.119:0/263752826 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4008960131 0 0) 0x7ff04015b0d0 con 0x7ff0400c83f0 2023-10-08T00:56:41.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff03effd700 1 -- 172.21.15.119:0/263752826 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff028003570 con 0x7ff0400c83f0 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff04499c700 1 -- 172.21.15.119:0/263752826 >> v1:172.21.15.184:6789/0 conn(0x7ff0400ccee0 legacy=0x7ff0401561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff04499c700 1 -- 172.21.15.119:0/263752826 >> v1:172.21.15.119:6789/0 conn(0x7ff0400c83f0 legacy=0x7ff0400c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff04499c700 1 -- 172.21.15.119:0/263752826 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04015aa80 con 0x7ff0401568e0 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff04499c700 1 --2- 172.21.15.119:0/263752826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0401568e0 0x7ff0401616c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff038009e60 tx=0x7ff038009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8917c75acbe547d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff03effd700 1 -- 172.21.15.119:0/263752826 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800e070 con 0x7ff0401568e0 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff03effd700 1 -- 172.21.15.119:0/263752826 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03800bb60 con 0x7ff0401568e0 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.189+0000 7ff03effd700 1 -- 172.21.15.119:0/263752826 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038012430 con 0x7ff0401568e0 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.190+0000 7ff0463ff700 1 -- 172.21.15.119:0/263752826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0401568e0 msgr2=0x7ff0401616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.190+0000 7ff0463ff700 1 --2- 172.21.15.119:0/263752826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0401568e0 0x7ff0401616c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff038009e60 tx=0x7ff038009780 comp rx=0 tx=0).stop 2023-10-08T00:56:41.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.190+0000 7ff0463ff700 1 -- 172.21.15.119:0/263752826 shutdown_connections 2023-10-08T00:56:41.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.190+0000 7ff0463ff700 1 --2- 172.21.15.119:0/263752826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0401568e0 0x7ff0401616c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:41.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.190+0000 7ff0463ff700 1 -- 172.21.15.119:0/263752826 >> 172.21.15.119:0/263752826 conn(0x7ff040068e10 msgr2=0x7ff0400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:41.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.190+0000 7ff0463ff700 1 -- 172.21.15.119:0/263752826 shutdown_connections 2023-10-08T00:56:41.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.190+0000 7ff0463ff700 1 -- 172.21.15.119:0/263752826 wait complete. 2023-10-08T00:56:41.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.191+0000 7ff0463ff700 1 Processor -- start 2023-10-08T00:56:41.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.192+0000 7ff0463ff700 1 -- start start 2023-10-08T00:56:41.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.192+0000 7ff0463ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0400c83f0 0x7ff040159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:41.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff0463ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0400ccee0 0x7ff04014f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:41.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff0463ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0401568e0 0x7ff040169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff0463ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400af630 con 0x7ff0400ccee0 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff0463ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400af910 con 0x7ff0400c83f0 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff0463ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04016a700 con 0x7ff0401568e0 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0400ccee0 0x7ff04014f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0400ccee0 0x7ff04014f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53878/0 (socket says 172.21.15.119:53878) 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 -- 172.21.15.119:0/863584962 learned_addr learned my addr 172.21.15.119:0/863584962 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03ffff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0400c83f0 0x7ff040159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff04499c700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0401568e0 0x7ff040169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 -- 172.21.15.119:0/863584962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0401568e0 msgr2=0x7ff040169e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0401568e0 0x7ff040169e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:41.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 -- 172.21.15.119:0/863584962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0400c83f0 msgr2=0x7ff040159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0400c83f0 0x7ff040159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 -- 172.21.15.119:0/863584962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff038009a20 con 0x7ff0400ccee0 2023-10-08T00:56:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.193+0000 7ff03f7fe700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0400ccee0 0x7ff04014f010 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff03400c7c0 tx=0x7ff034007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.194+0000 7ff027fff700 1 -- 172.21.15.119:0/863584962 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034011590 con 0x7ff0400ccee0 2023-10-08T00:56:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.194+0000 7ff027fff700 1 -- 172.21.15.119:0/863584962 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff034011710 con 0x7ff0400ccee0 2023-10-08T00:56:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.194+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04016a9e0 con 0x7ff0400ccee0 2023-10-08T00:56:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.194+0000 7ff027fff700 1 -- 172.21.15.119:0/863584962 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034011ad0 con 0x7ff0400ccee0 2023-10-08T00:56:41.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.194+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04014f550 con 0x7ff0400ccee0 2023-10-08T00:56:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.195+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0400afaa0 con 0x7ff0400ccee0 2023-10-08T00:56:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.196+0000 7ff027fff700 1 -- 172.21.15.119:0/863584962 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff03401b3b0 con 0x7ff0400ccee0 2023-10-08T00:56:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.196+0000 7ff027fff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff02c066f30 0x7ff02c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.197+0000 7ff027fff700 1 -- 172.21.15.119:0/863584962 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7ff034083b60 con 0x7ff0400ccee0 2023-10-08T00:56:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.197+0000 7ff03ffff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff02c066f30 0x7ff02c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:41.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.197+0000 7ff03ffff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff02c066f30 0x7ff02c0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff030007fd0 tx=0x7ff030009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:41.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.200+0000 7ff027fff700 1 -- 172.21.15.119:0/863584962 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff034050d30 con 0x7ff0400ccee0 2023-10-08T00:56:41.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:41.438+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff04004c3c0 con 0x7ff0400ccee0 2023-10-08T00:56:42.089 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-10-08T00:56:42.088+0000 7f3bf07f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T00:56:42.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.160+0000 7ff027fff700 1 -- 172.21.15.119:0/863584962 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff034017020 con 0x7ff0400ccee0 2023-10-08T00:56:42.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff02c066f30 msgr2=0x7ff02c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:42.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff02c066f30 0x7ff02c0693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff030007fd0 tx=0x7ff030009040 comp rx=0 tx=0).stop 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0400ccee0 msgr2=0x7ff04014f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0400ccee0 0x7ff04014f010 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff03400c7c0 tx=0x7ff034007f60 comp rx=0 tx=0).stop 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 shutdown_connections 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0401568e0 0x7ff040169e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff02c066f30 0x7ff02c0693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0400c83f0 0x7ff040159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 --2- 172.21.15.119:0/863584962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0400ccee0 0x7ff04014f010 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 >> 172.21.15.119:0/863584962 conn(0x7ff040068e10 msgr2=0x7ff040069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 shutdown_connections 2023-10-08T00:56:42.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.163+0000 7ff0463ff700 1 -- 172.21.15.119:0/863584962 wait complete. 2023-10-08T00:56:42.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T00:56:42.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.365+0000 7fb3f29ff700 1 Processor -- start 2023-10-08T00:56:42.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.366+0000 7fb3f29ff700 1 -- start start 2023-10-08T00:56:42.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.367+0000 7fb3f29ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb3ec0c70b0 0x7fb3ec155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:42.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.367+0000 7fb3f29ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb3ec0cd0b0 0x7fb3ec069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:42.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.367+0000 7fb3f29ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3ec06e950 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.367+0000 7fb3f29ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ec1563f0 con 0x7fb3ec0c70b0 2023-10-08T00:56:42.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.367+0000 7fb3f29ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3ec06eb90 con 0x7fb3ec0cd0b0 2023-10-08T00:56:42.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.367+0000 7fb3e3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb3ec0ccc40 0x7fb3ec06e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45064/0 (socket says 172.21.15.119:45064) 2023-10-08T00:56:42.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.367+0000 7fb3e3fff700 1 -- 172.21.15.119:0/3329125986 learned_addr learned my addr 172.21.15.119:0/3329125986 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:42.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.367+0000 7fb3ebfff700 1 --2- 172.21.15.119:0/3329125986 >> v2:172.21.15.119:3300/0 conn(0x7fb3ec0c70b0 0x7fb3ec155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:42.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3eb7fe700 1 -- 172.21.15.119:0/3329125986 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918197394 0 0) 0x7fb3d4001110 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3eb7fe700 1 -- 172.21.15.119:0/3329125986 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3d8003570 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3ebfff700 1 -- 172.21.15.119:0/3329125986 >> v2:172.21.15.184:3300/0 conn(0x7fb3ec0cd0b0 msgr2=0x7fb3ec069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:42.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3f0f9c700 1 --2- 172.21.15.119:0/3329125986 >> v2:172.21.15.184:3300/0 conn(0x7fb3ec0cd0b0 0x7fb3ec069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:42.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3ebfff700 1 --2- 172.21.15.119:0/3329125986 >> v2:172.21.15.184:3300/0 conn(0x7fb3ec0cd0b0 0x7fb3ec069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3ebfff700 1 -- 172.21.15.119:0/3329125986 >> v1:172.21.15.119:6789/0 conn(0x7fb3ec0ccc40 legacy=0x7fb3ec06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3ebfff700 1 -- 172.21.15.119:0/3329125986 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec06ee70 con 0x7fb3ec0c70b0 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3f0f9c700 1 --2- 172.21.15.119:0/3329125986 >> v2:172.21.15.184:3300/0 conn(0x7fb3ec0cd0b0 0x7fb3ec069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3ebfff700 1 --2- 172.21.15.119:0/3329125986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0c70b0 0x7fb3ec155eb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec06d270 tx=0x7fb3dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c22997f36a89d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3eb7fe700 1 -- 172.21.15.119:0/3329125986 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc011630 con 0x7fb3ec0c70b0 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.368+0000 7fb3eb7fe700 1 -- 172.21.15.119:0/3329125986 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc011790 con 0x7fb3ec0c70b0 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3eb7fe700 1 -- 172.21.15.119:0/3329125986 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc011b30 con 0x7fb3ec0c70b0 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3f29ff700 1 -- 172.21.15.119:0/3329125986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0c70b0 msgr2=0x7fb3ec155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/3329125986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0c70b0 0x7fb3ec155eb0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec06d270 tx=0x7fb3dc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3f29ff700 1 -- 172.21.15.119:0/3329125986 shutdown_connections 2023-10-08T00:56:42.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/3329125986 >> v2:172.21.15.184:3300/0 conn(0x7fb3ec0cd0b0 0x7fb3ec069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/3329125986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0c70b0 0x7fb3ec155eb0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3f29ff700 1 -- 172.21.15.119:0/3329125986 >> 172.21.15.119:0/3329125986 conn(0x7fb3ec068e10 msgr2=0x7fb3ec0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3f29ff700 1 -- 172.21.15.119:0/3329125986 shutdown_connections 2023-10-08T00:56:42.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.369+0000 7fb3f29ff700 1 -- 172.21.15.119:0/3329125986 wait complete. 2023-10-08T00:56:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.371+0000 7fb3f29ff700 1 Processor -- start 2023-10-08T00:56:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.371+0000 7fb3f29ff700 1 -- start start 2023-10-08T00:56:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3f29ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3ec0c70b0 0x7fb3ec0c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3f29ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0ccc40 0x7fb3ec152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:42.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3f29ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3ec0cd0b0 0x7fb3ec0c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3f29ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec0c54d0 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3f29ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec0c4b00 con 0x7fb3ec0cd0b0 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3f29ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec0c3ef0 con 0x7fb3ec0c70b0 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3e3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0ccc40 0x7fb3ec152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3e3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0ccc40 0x7fb3ec152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53882/0 (socket says 172.21.15.119:53882) 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3e3fff700 1 -- 172.21.15.119:0/2030033139 learned_addr learned my addr 172.21.15.119:0/2030033139 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3f0f9c700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3ec0cd0b0 0x7fb3ec0c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3ebfff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3ec0c70b0 0x7fb3ec0c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3e3fff700 1 -- 172.21.15.119:0/2030033139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3ec0c70b0 msgr2=0x7fb3ec0c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:42.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3e3fff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3ec0c70b0 0x7fb3ec0c4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3e3fff700 1 -- 172.21.15.119:0/2030033139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3ec0cd0b0 msgr2=0x7fb3ec0c35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3e3fff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3ec0cd0b0 0x7fb3ec0c35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.372+0000 7fb3e3fff700 1 -- 172.21.15.119:0/2030033139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc009420 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.373+0000 7fb3e3fff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0ccc40 0x7fb3ec152680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400c7c0 tx=0x7fb3d4007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.373+0000 7fb3e8ff9700 1 -- 172.21.15.119:0/2030033139 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4005b30 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.373+0000 7fb3e8ff9700 1 -- 172.21.15.119:0/2030033139 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d4005c90 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.373+0000 7fb3e8ff9700 1 -- 172.21.15.119:0/2030033139 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4007340 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.373+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3ec0c41d0 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.373+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3ec0c4790 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.374+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3ec0c4cb0 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.375+0000 7fb3e8ff9700 1 -- 172.21.15.119:0/2030033139 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb3d40076b0 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.376+0000 7fb3e8ff9700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb3cc066f30 0x7fb3cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:42.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.376+0000 7fb3e8ff9700 1 -- 172.21.15.119:0/2030033139 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fb3d4083f10 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.376+0000 7fb3ebfff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb3cc066f30 0x7fb3cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:42.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.376+0000 7fb3ebfff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb3cc066f30 0x7fb3cc0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0093f0 tx=0x7fb3dc017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:42.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.379+0000 7fb3e8ff9700 1 -- 172.21.15.119:0/2030033139 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb3ec0c4cb0 con 0x7fb3ec0ccc40 2023-10-08T00:56:42.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:42.614+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb3ec06e950 con 0x7fb3ec0ccc40 2023-10-08T00:56:43.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.162+0000 7fb3e8ff9700 1 -- 172.21.15.119:0/2030033139 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb3ec06e950 con 0x7fb3ec0ccc40 2023-10-08T00:56:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb3cc066f30 msgr2=0x7fb3cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb3cc066f30 0x7fb3cc0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc0093f0 tx=0x7fb3dc017000 comp rx=0 tx=0).stop 2023-10-08T00:56:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0ccc40 msgr2=0x7fb3ec152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0ccc40 0x7fb3ec152680 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400c7c0 tx=0x7fb3d4007c70 comp rx=0 tx=0).stop 2023-10-08T00:56:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 shutdown_connections 2023-10-08T00:56:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb3ec0c70b0 0x7fb3ec0c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb3cc066f30 0x7fb3cc0693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb3ec0cd0b0 0x7fb3ec0c35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 --2- 172.21.15.119:0/2030033139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0ccc40 0x7fb3ec152680 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 >> 172.21.15.119:0/2030033139 conn(0x7fb3ec068e10 msgr2=0x7fb3ec0cb5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 shutdown_connections 2023-10-08T00:56:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.165+0000 7fb3f29ff700 1 -- 172.21.15.119:0/2030033139 wait complete. 2023-10-08T00:56:43.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:43.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.365+0000 7f38695ff700 1 Processor -- start 2023-10-08T00:56:43.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.365+0000 7f38695ff700 1 -- start start 2023-10-08T00:56:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.366+0000 7f38695ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38641578b0 0x7f3864155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.366+0000 7f38695ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38640ad930 con 0x7f38640af2f0 2023-10-08T00:56:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.366+0000 7f38695ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38640acd20 con 0x7f3864155c10 2023-10-08T00:56:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.366+0000 7f38695ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38640ad000 con 0x7f38641578b0 2023-10-08T00:56:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f38637fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f38640af2f0 0x7f38640af710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45072/0 (socket says 172.21.15.119:45072) 2023-10-08T00:56:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f38637fe700 1 -- 172.21.15.119:0/2499544985 learned_addr learned my addr 172.21.15.119:0/2499544985 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893349027 0 0) 0x7f38640ad930 con 0x7f38640af2f0 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f38627fc700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38641578b0 0x7f3864155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f384c003570 con 0x7f38640af2f0 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 357359647 0 0) 0x7f384c003570 con 0x7f38640af2f0 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38641578b0 msgr2=0x7f3864155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38641578b0 0x7f3864155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 >> v1:172.21.15.184:6789/0 conn(0x7f3864155c10 legacy=0x7f38640ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38640ad2e0 con 0x7f38640af2f0 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f385c0017f0 con 0x7f38640af2f0 2023-10-08T00:56:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 >> v1:172.21.15.119:6789/0 conn(0x7f38640af2f0 legacy=0x7f38640af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c003a90 0x7f384c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.367+0000 7f3861ffb700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c004810 0x7f384c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f3861ffb700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38641578b0 0x7f384c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38640ad000 con 0x7f384c004810 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f384c00bb90 con 0x7f384c003a90 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c00bd50 con 0x7f38641578b0 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f38627fc700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c004810 0x7f384c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f38637fe700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38641578b0 0x7f384c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f3862ffd700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c003a90 0x7f384c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f38627fc700 1 -- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38641578b0 msgr2=0x7f384c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f38627fc700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f38641578b0 0x7f384c0051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f38627fc700 1 -- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c003a90 msgr2=0x7f384c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f38627fc700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f384c003a90 0x7f384c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f38627fc700 1 -- 172.21.15.119:0/2499544985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c003570 con 0x7f384c004810 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.368+0000 7f38627fc700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c004810 0x7f384c004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3858009c00 tx=0x7f38580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c71128bd555815 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800e710 con 0x7f384c004810 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385800e870 con 0x7f384c004810 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f3861ffb700 1 -- 172.21.15.119:0/2499544985 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385800ec30 con 0x7f384c004810 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f38695ff700 1 -- 172.21.15.119:0/2499544985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c004810 msgr2=0x7f384c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f38695ff700 1 --2- 172.21.15.119:0/2499544985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f384c004810 0x7f384c004c80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3858009c00 tx=0x7f38580091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f38637fe700 1 -- 172.21.15.119:0/2499544985 reap_dead start 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f38695ff700 1 -- 172.21.15.119:0/2499544985 shutdown_connections 2023-10-08T00:56:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f38695ff700 1 -- 172.21.15.119:0/2499544985 >> 172.21.15.119:0/2499544985 conn(0x7f386406a320 msgr2=0x7f38640cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f38695ff700 1 -- 172.21.15.119:0/2499544985 shutdown_connections 2023-10-08T00:56:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.369+0000 7f38695ff700 1 -- 172.21.15.119:0/2499544985 wait complete. 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.371+0000 7f38695ff700 1 Processor -- start 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.371+0000 7f38695ff700 1 -- start start 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38695ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c004810 0x7f38641451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38695ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3864142ec0 0x7f386415a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38695ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3864143330 0x7f386406e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38695ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3864147b40 con 0x7f3864143330 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38695ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3864146ec0 con 0x7f3864142ec0 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38695ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38641471a0 con 0x7f384c004810 2023-10-08T00:56:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38637fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3864143330 0x7f386406e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38637fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3864143330 0x7f386406e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53896/0 (socket says 172.21.15.119:53896) 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38637fe700 1 -- 172.21.15.119:0/3404732992 learned_addr learned my addr 172.21.15.119:0/3404732992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f3862ffd700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c004810 0x7f38641451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38627fc700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3864142ec0 0x7f386415a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38637fe700 1 -- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c004810 msgr2=0x7f38641451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38637fe700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c004810 0x7f38641451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38637fe700 1 -- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3864142ec0 msgr2=0x7f386415a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38637fe700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3864142ec0 0x7f386415a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38637fe700 1 -- 172.21.15.119:0/3404732992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3864147480 con 0x7f3864143330 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.372+0000 7f38627fc700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3864142ec0 0x7f386415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.373+0000 7f38637fe700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3864143330 0x7f386406e410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f386406d2a0 tx=0x7f38640ad8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.373+0000 7f384affd700 1 -- 172.21.15.119:0/3404732992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00ae10 con 0x7f3864143330 2023-10-08T00:56:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.373+0000 7f384affd700 1 -- 172.21.15.119:0/3404732992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c0133f0 con 0x7f3864143330 2023-10-08T00:56:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.373+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3864147760 con 0x7f3864143330 2023-10-08T00:56:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.373+0000 7f384affd700 1 -- 172.21.15.119:0/3404732992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c0137b0 con 0x7f3864143330 2023-10-08T00:56:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.373+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f386415ac70 con 0x7f3864143330 2023-10-08T00:56:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.374+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3864147c80 con 0x7f3864143330 2023-10-08T00:56:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.375+0000 7f384affd700 1 -- 172.21.15.119:0/3404732992 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f385c013b60 con 0x7f3864143330 2023-10-08T00:56:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.376+0000 7f384affd700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3850066f30 0x7f38500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.376+0000 7f384affd700 1 -- 172.21.15.119:0/3404732992 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f385c0850b0 con 0x7f3864143330 2023-10-08T00:56:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.376+0000 7f3862ffd700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3850066f30 0x7f38500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.376+0000 7f3862ffd700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3850066f30 0x7f38500693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3854003960 tx=0x7f3854009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:43.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.380+0000 7f384affd700 1 -- 172.21.15.119:0/3404732992 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f385c052280 con 0x7f3864143330 2023-10-08T00:56:43.480 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:56:43.485 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:56:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.618+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38640b0830 con 0x7f3864143330 2023-10-08T00:56:43.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.619+0000 7f384affd700 1 -- 172.21.15.119:0/3404732992 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f385c0558a0 con 0x7f3864143330 2023-10-08T00:56:43.620 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:43.620 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/768996961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":768996961},{"type":"v1","addr":"172.21.15.184:6834","nonce":768996961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1876644893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1876644893},{"type":"v1","addr":"172.21.15.119:6837","nonce":1876644893}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1187803776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1187803776},{"type":"v1","addr":"172.21.15.184:6835","nonce":1187803776}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:56:39.582864+0000","modified":"2023-10-08T00:56:43.160226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3785845701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3785845701},{"type":"v1","addr":"172.21.15.119:6836","nonce":3785845701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:56:43.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3850066f30 msgr2=0x7f38500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3850066f30 0x7f38500693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f3854003960 tx=0x7f3854009040 comp rx=0 tx=0).stop 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3864143330 msgr2=0x7f386406e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3864143330 0x7f386406e410 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f386406d2a0 tx=0x7f38640ad8a0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 shutdown_connections 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f384c004810 0x7f38641451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3850066f30 0x7f38500693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3864143330 0x7f386406e410 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 --2- 172.21.15.119:0/3404732992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3864142ec0 0x7f386415a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.621+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 >> 172.21.15.119:0/3404732992 conn(0x7f386406a320 msgr2=0x7f386414a810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.622+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 shutdown_connections 2023-10-08T00:56:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.622+0000 7f38695ff700 1 -- 172.21.15.119:0/3404732992 wait complete. 2023-10-08T00:56:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2023-10-08T00:56:43.641 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T00:56:43.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.819+0000 7fd7e73ff700 1 Processor -- start 2023-10-08T00:56:43.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.819+0000 7fd7e73ff700 1 -- start start 2023-10-08T00:56:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.820+0000 7fd7e73ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd7e006bcd0 0x7fd7e00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.820+0000 7fd7e73ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7e015df50 0x7fd7e015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.820+0000 7fd7e73ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e015d7e0 con 0x7fd7e006c0a0 2023-10-08T00:56:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.820+0000 7fd7e73ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e015dac0 con 0x7fd7e015df50 2023-10-08T00:56:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.820+0000 7fd7e73ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e015aa30 con 0x7fd7e006bcd0 2023-10-08T00:56:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e599c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7e015df50 0x7fd7e015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e599c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7e015df50 0x7fd7e015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53906/0 (socket says 172.21.15.119:53906) 2023-10-08T00:56:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e599c700 1 -- 172.21.15.119:0/895192271 learned_addr learned my addr 172.21.15.119:0/895192271 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e519b700 1 --2- 172.21.15.119:0/895192271 >> v2:172.21.15.184:3300/0 conn(0x7fd7e006bcd0 0x7fd7e00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e599c700 1 -- 172.21.15.119:0/895192271 >> v2:172.21.15.184:3300/0 conn(0x7fd7e006bcd0 msgr2=0x7fd7e00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e599c700 1 --2- 172.21.15.119:0/895192271 >> v2:172.21.15.184:3300/0 conn(0x7fd7e006bcd0 0x7fd7e00caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e599c700 1 -- 172.21.15.119:0/895192271 >> v1:172.21.15.184:6789/0 conn(0x7fd7e006c0a0 legacy=0x7fd7e015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e599c700 1 -- 172.21.15.119:0/895192271 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e015ad10 con 0x7fd7e015df50 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.821+0000 7fd7e599c700 1 --2- 172.21.15.119:0/895192271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e015df50 0x7fd7e015e3b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00691a0 tx=0x7fd7d8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d86e6cef141ad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7d7fff700 1 -- 172.21.15.119:0/895192271 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800e070 con 0x7fd7e015df50 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7d7fff700 1 -- 172.21.15.119:0/895192271 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d800ba20 con 0x7fd7e015df50 2023-10-08T00:56:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7d7fff700 1 -- 172.21.15.119:0/895192271 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d80092e0 con 0x7fd7e015df50 2023-10-08T00:56:43.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7e73ff700 1 -- 172.21.15.119:0/895192271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e015df50 msgr2=0x7fd7e015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/895192271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e015df50 0x7fd7e015e3b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00691a0 tx=0x7fd7d8009840 comp rx=0 tx=0).stop 2023-10-08T00:56:43.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7e73ff700 1 -- 172.21.15.119:0/895192271 shutdown_connections 2023-10-08T00:56:43.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/895192271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e015df50 0x7fd7e015e3b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/895192271 >> v2:172.21.15.184:3300/0 conn(0x7fd7e006bcd0 0x7fd7e00caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7e73ff700 1 -- 172.21.15.119:0/895192271 >> 172.21.15.119:0/895192271 conn(0x7fd7e00ce600 msgr2=0x7fd7e00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:43.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7e73ff700 1 -- 172.21.15.119:0/895192271 shutdown_connections 2023-10-08T00:56:43.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.822+0000 7fd7e73ff700 1 -- 172.21.15.119:0/895192271 wait complete. 2023-10-08T00:56:43.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.824+0000 7fd7e73ff700 1 Processor -- start 2023-10-08T00:56:43.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.824+0000 7fd7e73ff700 1 -- start start 2023-10-08T00:56:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e73ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7e006bcd0 0x7fd7e01548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e73ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e006c0a0 0x7fd7e016de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e73ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7e015df50 0x7fd7e0153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e73ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0169e60 con 0x7fd7e006c0a0 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e73ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e016a6c0 con 0x7fd7e006bcd0 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e73ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e0154e90 con 0x7fd7e015df50 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e499a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e006c0a0 0x7fd7e016de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e499a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e006c0a0 0x7fd7e016de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53920/0 (socket says 172.21.15.119:53920) 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e499a700 1 -- 172.21.15.119:0/1716709775 learned_addr learned my addr 172.21.15.119:0/1716709775 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e519b700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7e006bcd0 0x7fd7e01548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.825+0000 7fd7e599c700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7e015df50 0x7fd7e0153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7e499a700 1 -- 172.21.15.119:0/1716709775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7e015df50 msgr2=0x7fd7e0153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7e499a700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7e015df50 0x7fd7e0153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7e499a700 1 -- 172.21.15.119:0/1716709775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7e006bcd0 msgr2=0x7fd7e01548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7e499a700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7e006bcd0 0x7fd7e01548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:43.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7e499a700 1 -- 172.21.15.119:0/1716709775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d8009ae0 con 0x7fd7e006c0a0 2023-10-08T00:56:43.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7e499a700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e006c0a0 0x7fd7e016de70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000c4e0 tx=0x7fd7d000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:43.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7d57fa700 1 -- 172.21.15.119:0/1716709775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0011590 con 0x7fd7e006c0a0 2023-10-08T00:56:43.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7d57fa700 1 -- 172.21.15.119:0/1716709775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d0011710 con 0x7fd7e006c0a0 2023-10-08T00:56:43.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e016a070 con 0x7fd7e006c0a0 2023-10-08T00:56:43.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7d57fa700 1 -- 172.21.15.119:0/1716709775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0011ad0 con 0x7fd7e006c0a0 2023-10-08T00:56:43.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.826+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e0153690 con 0x7fd7e006c0a0 2023-10-08T00:56:43.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.827+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e016a510 con 0x7fd7e006c0a0 2023-10-08T00:56:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.828+0000 7fd7d57fa700 1 -- 172.21.15.119:0/1716709775 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7d001b3b0 con 0x7fd7e006c0a0 2023-10-08T00:56:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.829+0000 7fd7d57fa700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7c4066f30 0x7fd7c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.830+0000 7fd7d57fa700 1 -- 172.21.15.119:0/1716709775 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd7d0083960 con 0x7fd7e006c0a0 2023-10-08T00:56:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.830+0000 7fd7e519b700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7c4066f30 0x7fd7c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:43.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.830+0000 7fd7e519b700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7c4066f30 0x7fd7c40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009d10 tx=0x7fd7dc0091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:43.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:43.834+0000 7fd7d57fa700 1 -- 172.21.15.119:0/1716709775 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd7d0050b30 con 0x7fd7e006c0a0 2023-10-08T00:56:44.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.027+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7e004c3c0 con 0x7fd7e006c0a0 2023-10-08T00:56:44.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.027+0000 7fd7d57fa700 1 -- 172.21.15.119:0/1716709775 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+15398 (secure 0 0 0) 0x7fd7d0017070 con 0x7fd7e006c0a0 2023-10-08T00:56:44.029 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:44.029 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":31,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T00:56:41.123024+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:56:37.182342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:56:38.569786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T00:56:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7c4066f30 msgr2=0x7fd7c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7c4066f30 0x7fd7c40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009d10 tx=0x7fd7dc0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e006c0a0 msgr2=0x7fd7e016de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e006c0a0 0x7fd7e016de70 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000c4e0 tx=0x7fd7d000c7f0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 shutdown_connections 2023-10-08T00:56:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7e015df50 0x7fd7e0153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7c4066f30 0x7fd7c40693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7e006bcd0 0x7fd7e01548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 --2- 172.21.15.119:0/1716709775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e006c0a0 0x7fd7e016de70 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 >> 172.21.15.119:0/1716709775 conn(0x7fd7e00ce600 msgr2=0x7fd7e00c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 shutdown_connections 2023-10-08T00:56:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.030+0000 7fd7e73ff700 1 -- 172.21.15.119:0/1716709775 wait complete. 2023-10-08T00:56:44.048 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T00:56:44.049 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:56:44.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.230+0000 7fa813fff700 1 Processor -- start 2023-10-08T00:56:44.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.230+0000 7fa813fff700 1 -- start start 2023-10-08T00:56:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.231+0000 7fa813fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150180 0x7fa814157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.231+0000 7fa813fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa8140c7410 0x7fa814069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.231+0000 7fa813fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa81406e2a0 con 0x7fa814150550 2023-10-08T00:56:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.231+0000 7fa813fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa81406b2e0 con 0x7fa8140c7410 2023-10-08T00:56:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.231+0000 7fa813fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140c78e0 con 0x7fa814150180 2023-10-08T00:56:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa812ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150180 0x7fa814157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa8137fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa8140c7410 0x7fa814069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa812ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150180 0x7fa814157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:53154/0 (socket says 172.21.15.119:53154) 2023-10-08T00:56:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa8137fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa8140c7410 0x7fa814069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58050/0 (socket says 172.21.15.119:58050) 2023-10-08T00:56:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa812ffd700 1 -- 172.21.15.119:0/2205337380 learned_addr learned my addr 172.21.15.119:0/2205337380 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa812ffd700 1 -- 172.21.15.119:0/2205337380 >> v2:172.21.15.184:3300/0 conn(0x7fa8140c7410 msgr2=0x7fa814069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa812ffd700 1 --2- 172.21.15.119:0/2205337380 >> v2:172.21.15.184:3300/0 conn(0x7fa8140c7410 0x7fa814069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa812ffd700 1 -- 172.21.15.119:0/2205337380 >> v1:172.21.15.184:6789/0 conn(0x7fa814150550 legacy=0x7fa8140c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa812ffd700 1 -- 172.21.15.119:0/2205337380 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8140c7bc0 con 0x7fa814150180 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.232+0000 7fa811ffb700 1 -- 172.21.15.119:0/2205337380 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822731390 0 0) 0x7fa81406e2a0 con 0x7fa814150550 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa812ffd700 1 --2- 172.21.15.119:0/2205337380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150180 0x7fa814157770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa80400ab50 tx=0x7fa80400a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1797ac576f08f1a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa811ffb700 1 -- 172.21.15.119:0/2205337380 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa80400ca40 con 0x7fa814150180 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa811ffb700 1 -- 172.21.15.119:0/2205337380 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa80400cba0 con 0x7fa814150180 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa811ffb700 1 -- 172.21.15.119:0/2205337380 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804018400 con 0x7fa814150180 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa813fff700 1 -- 172.21.15.119:0/2205337380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150180 msgr2=0x7fa814157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa813fff700 1 --2- 172.21.15.119:0/2205337380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150180 0x7fa814157770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa80400ab50 tx=0x7fa80400a3f0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa813fff700 1 -- 172.21.15.119:0/2205337380 shutdown_connections 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa813fff700 1 --2- 172.21.15.119:0/2205337380 >> v2:172.21.15.184:3300/0 conn(0x7fa8140c7410 0x7fa814069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa813fff700 1 --2- 172.21.15.119:0/2205337380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150180 0x7fa814157770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa813fff700 1 -- 172.21.15.119:0/2205337380 >> 172.21.15.119:0/2205337380 conn(0x7fa8140cc1f0 msgr2=0x7fa8140cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.233+0000 7fa813fff700 1 -- 172.21.15.119:0/2205337380 shutdown_connections 2023-10-08T00:56:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.234+0000 7fa813fff700 1 -- 172.21.15.119:0/2205337380 wait complete. 2023-10-08T00:56:44.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.235+0000 7fa813fff700 1 Processor -- start 2023-10-08T00:56:44.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.235+0000 7fa813fff700 1 -- start start 2023-10-08T00:56:44.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa813fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8140c7410 0x7fa81414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa813fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa814150180 0x7fa8140ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa813fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150550 0x7fa8140cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa813fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b06c0 con 0x7fa8140c7410 2023-10-08T00:56:44.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa813fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b0100 con 0x7fa814150180 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa813fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8140b03e0 con 0x7fa814150550 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa812ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8140c7410 0x7fa81414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa812ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8140c7410 0x7fa81414ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53926/0 (socket says 172.21.15.119:53926) 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa812ffd700 1 -- 172.21.15.119:0/2211947872 learned_addr learned my addr 172.21.15.119:0/2211947872 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa8127fc700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa814150180 0x7fa8140ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.236+0000 7fa8137fe700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150550 0x7fa8140cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa812ffd700 1 -- 172.21.15.119:0/2211947872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150550 msgr2=0x7fa8140cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa812ffd700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150550 0x7fa8140cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa812ffd700 1 -- 172.21.15.119:0/2211947872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa814150180 msgr2=0x7fa8140ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa812ffd700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa814150180 0x7fa8140ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa812ffd700 1 -- 172.21.15.119:0/2211947872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80400a690 con 0x7fa8140c7410 2023-10-08T00:56:44.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa812ffd700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8140c7410 0x7fa81414ac70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa80400a660 tx=0x7fa80401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:44.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa7faffd700 1 -- 172.21.15.119:0/2211947872 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804023440 con 0x7fa8140c7410 2023-10-08T00:56:44.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa7faffd700 1 -- 172.21.15.119:0/2211947872 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8040235a0 con 0x7fa8140c7410 2023-10-08T00:56:44.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa7faffd700 1 -- 172.21.15.119:0/2211947872 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa804023920 con 0x7fa8140c7410 2023-10-08T00:56:44.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8140acdf0 con 0x7fa8140c7410 2023-10-08T00:56:44.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.237+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81414b1b0 con 0x7fa8140c7410 2023-10-08T00:56:44.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.238+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8140acc50 con 0x7fa8140c7410 2023-10-08T00:56:44.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.239+0000 7fa7faffd700 1 -- 172.21.15.119:0/2211947872 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8040103f0 con 0x7fa8140c7410 2023-10-08T00:56:44.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.241+0000 7fa7faffd700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa800066f30 0x7fa8000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.241+0000 7fa7faffd700 1 -- 172.21.15.119:0/2211947872 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa804085c00 con 0x7fa8140c7410 2023-10-08T00:56:44.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.241+0000 7fa8127fc700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa800066f30 0x7fa8000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.242+0000 7fa8127fc700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa800066f30 0x7fa8000693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa81414fba0 tx=0x7fa808007ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:44.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.245+0000 7fa7faffd700 1 -- 172.21.15.119:0/2211947872 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa8140acc50 con 0x7fa8140c7410 2023-10-08T00:56:44.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.434+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa81414b340 con 0x7fa800066f30 2023-10-08T00:56:44.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.443+0000 7fa7faffd700 1 -- 172.21.15.119:0/2211947872 <== mgr.4101 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289140 (secure 0 0 0) 0x7fa81414b340 con 0x7fa800066f30 2023-10-08T00:56:44.448 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:44.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa800066f30 msgr2=0x7fa8000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa800066f30 0x7fa8000693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa81414fba0 tx=0x7fa808007ec0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8140c7410 msgr2=0x7fa81414ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8140c7410 0x7fa81414ac70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa80400a660 tx=0x7fa80401a000 comp rx=0 tx=0).stop 2023-10-08T00:56:44.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 shutdown_connections 2023-10-08T00:56:44.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa814150550 0x7fa8140cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa800066f30 0x7fa8000693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8140c7410 0x7fa81414ac70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 --2- 172.21.15.119:0/2211947872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa814150180 0x7fa8140ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 >> 172.21.15.119:0/2211947872 conn(0x7fa8140cc1f0 msgr2=0x7fa8141597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:44.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 shutdown_connections 2023-10-08T00:56:44.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.447+0000 7fa813fff700 1 -- 172.21.15.119:0/2211947872 wait complete. 2023-10-08T00:56:44.452 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T00:56:44.466 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2023-10-08T00:56:43.868915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382152,"kb_used_data":3848,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592568,"statfs":{"total":773094113280,"available":770654789632,"internally_reserved":0,"allocated":3940352,"data_stored":2207348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.767352"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143111+0000","last_change":"2023-10-08T00:56:40.143111+0000","last_active":"2023-10-08T00:56:40.143111+0000","last_peered":"2023-10-08T00:56:40.143111+0000","last_clean":"2023-10-08T00:56:40.143111+0000","last_became_active":"2023-10-08T00:56:40.142814+0000","last_became_peered":"2023-10-08T00:56:40.142814+0000","last_unstale":"2023-10-08T00:56:40.143111+0000","last_undegraded":"2023-10-08T00:56:40.143111+0000","last_fullsized":"2023-10-08T00:56:40.143111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:16.075601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.154161+0000","last_change":"2023-10-08T00:56:40.154161+0000","last_active":"2023-10-08T00:56:40.154161+0000","last_peered":"2023-10-08T00:56:40.154161+0000","last_clean":"2023-10-08T00:56:40.154161+0000","last_became_active":"2023-10-08T00:56:40.153948+0000","last_became_peered":"2023-10-08T00:56:40.153948+0000","last_unstale":"2023-10-08T00:56:40.154161+0000","last_undegraded":"2023-10-08T00:56:40.154161+0000","last_fullsized":"2023-10-08T00:56:40.154161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:47.647727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166906+0000","last_change":"2023-10-08T00:56:40.166906+0000","last_active":"2023-10-08T00:56:40.166906+0000","last_peered":"2023-10-08T00:56:40.166906+0000","last_clean":"2023-10-08T00:56:40.166906+0000","last_became_active":"2023-10-08T00:56:40.166592+0000","last_became_peered":"2023-10-08T00:56:40.166592+0000","last_unstale":"2023-10-08T00:56:40.166906+0000","last_undegraded":"2023-10-08T00:56:40.166906+0000","last_fullsized":"2023-10-08T00:56:40.166906+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:49.816730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.159033+0000","last_change":"2023-10-08T00:56:40.159033+0000","last_active":"2023-10-08T00:56:40.159033+0000","last_peered":"2023-10-08T00:56:40.159033+0000","last_clean":"2023-10-08T00:56:40.159033+0000","last_became_active":"2023-10-08T00:56:40.158659+0000","last_became_peered":"2023-10-08T00:56:40.158659+0000","last_unstale":"2023-10-08T00:56:40.159033+0000","last_undegraded":"2023-10-08T00:56:40.159033+0000","last_fullsized":"2023-10-08T00:56:40.159033+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:49.816730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.144767+0000","last_change":"2023-10-08T00:56:40.144767+0000","last_active":"2023-10-08T00:56:40.144767+0000","last_peered":"2023-10-08T00:56:40.144767+0000","last_clean":"2023-10-08T00:56:40.144767+0000","last_became_active":"2023-10-08T00:56:40.144499+0000","last_became_peered":"2023-10-08T00:56:40.144499+0000","last_unstale":"2023-10-08T00:56:40.144767+0000","last_undegraded":"2023-10-08T00:56:40.144767+0000","last_fullsized":"2023-10-08T00:56:40.144767+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:41.258863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.147652+0000","last_change":"2023-10-08T00:56:40.147652+0000","last_active":"2023-10-08T00:56:40.147652+0000","last_peered":"2023-10-08T00:56:40.147652+0000","last_clean":"2023-10-08T00:56:40.147652+0000","last_became_active":"2023-10-08T00:56:40.147351+0000","last_became_peered":"2023-10-08T00:56:40.147351+0000","last_unstale":"2023-10-08T00:56:40.147652+0000","last_undegraded":"2023-10-08T00:56:40.147652+0000","last_fullsized":"2023-10-08T00:56:40.147652+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:03:03.376251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140536+0000","last_change":"2023-10-08T00:56:40.140536+0000","last_active":"2023-10-08T00:56:40.140536+0000","last_peered":"2023-10-08T00:56:40.140536+0000","last_clean":"2023-10-08T00:56:40.140536+0000","last_became_active":"2023-10-08T00:56:40.140211+0000","last_became_peered":"2023-10-08T00:56:40.140211+0000","last_unstale":"2023-10-08T00:56:40.140536+0000","last_undegraded":"2023-10-08T00:56:40.140536+0000","last_fullsized":"2023-10-08T00:56:40.140536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:30.610379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140578+0000","last_change":"2023-10-08T00:56:40.140578+0000","last_active":"2023-10-08T00:56:40.140578+0000","last_peered":"2023-10-08T00:56:40.140578+0000","last_clean":"2023-10-08T00:56:40.140578+0000","last_became_active":"2023-10-08T00:56:40.140236+0000","last_became_peered":"2023-10-08T00:56:40.140236+0000","last_unstale":"2023-10-08T00:56:40.140578+0000","last_undegraded":"2023-10-08T00:56:40.140578+0000","last_fullsized":"2023-10-08T00:56:40.140578+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:33.219535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.163385+0000","last_change":"2023-10-08T00:56:40.163385+0000","last_active":"2023-10-08T00:56:40.163385+0000","last_peered":"2023-10-08T00:56:40.163385+0000","last_clean":"2023-10-08T00:56:40.163385+0000","last_became_active":"2023-10-08T00:56:40.163071+0000","last_became_peered":"2023-10-08T00:56:40.163071+0000","last_unstale":"2023-10-08T00:56:40.163385+0000","last_undegraded":"2023-10-08T00:56:40.163385+0000","last_fullsized":"2023-10-08T00:56:40.163385+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:41.313783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162314+0000","last_change":"2023-10-08T00:56:40.162314+0000","last_active":"2023-10-08T00:56:40.162314+0000","last_peered":"2023-10-08T00:56:40.162314+0000","last_clean":"2023-10-08T00:56:40.162314+0000","last_became_active":"2023-10-08T00:56:40.162047+0000","last_became_peered":"2023-10-08T00:56:40.162047+0000","last_unstale":"2023-10-08T00:56:40.162314+0000","last_undegraded":"2023-10-08T00:56:40.162314+0000","last_fullsized":"2023-10-08T00:56:40.162314+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:24.551023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.164708+0000","last_change":"2023-10-08T00:56:40.164708+0000","last_active":"2023-10-08T00:56:40.164708+0000","last_peered":"2023-10-08T00:56:40.164708+0000","last_clean":"2023-10-08T00:56:40.164708+0000","last_became_active":"2023-10-08T00:56:40.164375+0000","last_became_peered":"2023-10-08T00:56:40.164375+0000","last_unstale":"2023-10-08T00:56:40.164708+0000","last_undegraded":"2023-10-08T00:56:40.164708+0000","last_fullsized":"2023-10-08T00:56:40.164708+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:47.647727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.147892+0000","last_change":"2023-10-08T00:56:40.147892+0000","last_active":"2023-10-08T00:56:40.147892+0000","last_peered":"2023-10-08T00:56:40.147892+0000","last_clean":"2023-10-08T00:56:40.147892+0000","last_became_active":"2023-10-08T00:56:40.147630+0000","last_became_peered":"2023-10-08T00:56:40.147630+0000","last_unstale":"2023-10-08T00:56:40.147892+0000","last_undegraded":"2023-10-08T00:56:40.147892+0000","last_fullsized":"2023-10-08T00:56:40.147892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:58.921683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.161236+0000","last_change":"2023-10-08T00:56:40.161236+0000","last_active":"2023-10-08T00:56:40.161236+0000","last_peered":"2023-10-08T00:56:40.161236+0000","last_clean":"2023-10-08T00:56:40.161236+0000","last_became_active":"2023-10-08T00:56:40.160964+0000","last_became_peered":"2023-10-08T00:56:40.160964+0000","last_unstale":"2023-10-08T00:56:40.161236+0000","last_undegraded":"2023-10-08T00:56:40.161236+0000","last_fullsized":"2023-10-08T00:56:40.161236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:18.390267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.137588+0000","last_change":"2023-10-08T00:56:40.137588+0000","last_active":"2023-10-08T00:56:40.137588+0000","last_peered":"2023-10-08T00:56:40.137588+0000","last_clean":"2023-10-08T00:56:40.137588+0000","last_became_active":"2023-10-08T00:56:40.137260+0000","last_became_peered":"2023-10-08T00:56:40.137260+0000","last_unstale":"2023-10-08T00:56:40.137588+0000","last_undegraded":"2023-10-08T00:56:40.137588+0000","last_fullsized":"2023-10-08T00:56:40.137588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:48.913792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.145486+0000","last_change":"2023-10-08T00:56:40.145486+0000","last_active":"2023-10-08T00:56:40.145486+0000","last_peered":"2023-10-08T00:56:40.145486+0000","last_clean":"2023-10-08T00:56:40.145486+0000","last_became_active":"2023-10-08T00:56:40.142311+0000","last_became_peered":"2023-10-08T00:56:40.142311+0000","last_unstale":"2023-10-08T00:56:40.145486+0000","last_undegraded":"2023-10-08T00:56:40.145486+0000","last_fullsized":"2023-10-08T00:56:40.145486+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:47.647727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.141179+0000","last_change":"2023-10-08T00:56:40.141179+0000","last_active":"2023-10-08T00:56:40.141179+0000","last_peered":"2023-10-08T00:56:40.141179+0000","last_clean":"2023-10-08T00:56:40.141179+0000","last_became_active":"2023-10-08T00:56:40.140910+0000","last_became_peered":"2023-10-08T00:56:40.140910+0000","last_unstale":"2023-10-08T00:56:40.141179+0000","last_undegraded":"2023-10-08T00:56:40.141179+0000","last_fullsized":"2023-10-08T00:56:40.141179+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:48.913792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143165+0000","last_change":"2023-10-08T00:56:40.143165+0000","last_active":"2023-10-08T00:56:40.143165+0000","last_peered":"2023-10-08T00:56:40.143165+0000","last_clean":"2023-10-08T00:56:40.143165+0000","last_became_active":"2023-10-08T00:56:40.142770+0000","last_became_peered":"2023-10-08T00:56:40.142770+0000","last_unstale":"2023-10-08T00:56:40.143165+0000","last_undegraded":"2023-10-08T00:56:40.143165+0000","last_fullsized":"2023-10-08T00:56:40.143165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:27.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.165894+0000","last_change":"2023-10-08T00:56:40.165894+0000","last_active":"2023-10-08T00:56:40.165894+0000","last_peered":"2023-10-08T00:56:40.165894+0000","last_clean":"2023-10-08T00:56:40.165894+0000","last_became_active":"2023-10-08T00:56:40.165644+0000","last_became_peered":"2023-10-08T00:56:40.165644+0000","last_unstale":"2023-10-08T00:56:40.165894+0000","last_undegraded":"2023-10-08T00:56:40.165894+0000","last_fullsized":"2023-10-08T00:56:40.165894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:50.990273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166648+0000","last_change":"2023-10-08T00:56:40.166648+0000","last_active":"2023-10-08T00:56:40.166648+0000","last_peered":"2023-10-08T00:56:40.166648+0000","last_clean":"2023-10-08T00:56:40.166648+0000","last_became_active":"2023-10-08T00:56:40.166365+0000","last_became_peered":"2023-10-08T00:56:40.166365+0000","last_unstale":"2023-10-08T00:56:40.166648+0000","last_undegraded":"2023-10-08T00:56:40.166648+0000","last_fullsized":"2023-10-08T00:56:40.166648+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:44:25.602012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.145126+0000","last_change":"2023-10-08T00:56:40.145126+0000","last_active":"2023-10-08T00:56:40.145126+0000","last_peered":"2023-10-08T00:56:40.145126+0000","last_clean":"2023-10-08T00:56:40.145126+0000","last_became_active":"2023-10-08T00:56:40.144768+0000","last_became_peered":"2023-10-08T00:56:40.144768+0000","last_unstale":"2023-10-08T00:56:40.145126+0000","last_undegraded":"2023-10-08T00:56:40.145126+0000","last_fullsized":"2023-10-08T00:56:40.145126+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:24.005622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.156444+0000","last_change":"2023-10-08T00:56:40.156444+0000","last_active":"2023-10-08T00:56:40.156444+0000","last_peered":"2023-10-08T00:56:40.156444+0000","last_clean":"2023-10-08T00:56:40.156444+0000","last_became_active":"2023-10-08T00:56:40.156202+0000","last_became_peered":"2023-10-08T00:56:40.156202+0000","last_unstale":"2023-10-08T00:56:40.156444+0000","last_undegraded":"2023-10-08T00:56:40.156444+0000","last_fullsized":"2023-10-08T00:56:40.156444+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:47.749054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.160143+0000","last_change":"2023-10-08T00:56:40.160143+0000","last_active":"2023-10-08T00:56:40.160143+0000","last_peered":"2023-10-08T00:56:40.160143+0000","last_clean":"2023-10-08T00:56:40.160143+0000","last_became_active":"2023-10-08T00:56:40.159873+0000","last_became_peered":"2023-10-08T00:56:40.159873+0000","last_unstale":"2023-10-08T00:56:40.160143+0000","last_undegraded":"2023-10-08T00:56:40.160143+0000","last_fullsized":"2023-10-08T00:56:40.160143+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:47.023393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140903+0000","last_change":"2023-10-08T00:56:40.140903+0000","last_active":"2023-10-08T00:56:40.140903+0000","last_peered":"2023-10-08T00:56:40.140903+0000","last_clean":"2023-10-08T00:56:40.140903+0000","last_became_active":"2023-10-08T00:56:40.140560+0000","last_became_peered":"2023-10-08T00:56:40.140560+0000","last_unstale":"2023-10-08T00:56:40.140903+0000","last_undegraded":"2023-10-08T00:56:40.140903+0000","last_fullsized":"2023-10-08T00:56:40.140903+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:59.756389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.139421+0000","last_change":"2023-10-08T00:56:40.139421+0000","last_active":"2023-10-08T00:56:40.139421+0000","last_peered":"2023-10-08T00:56:40.139421+0000","last_clean":"2023-10-08T00:56:40.139421+0000","last_became_active":"2023-10-08T00:56:40.139090+0000","last_became_peered":"2023-10-08T00:56:40.139090+0000","last_unstale":"2023-10-08T00:56:40.139421+0000","last_undegraded":"2023-10-08T00:56:40.139421+0000","last_fullsized":"2023-10-08T00:56:40.139421+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:33.219535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162377+0000","last_change":"2023-10-08T00:56:40.162377+0000","last_active":"2023-10-08T00:56:40.162377+0000","last_peered":"2023-10-08T00:56:40.162377+0000","last_clean":"2023-10-08T00:56:40.162377+0000","last_became_active":"2023-10-08T00:56:40.162116+0000","last_became_peered":"2023-10-08T00:56:40.162116+0000","last_unstale":"2023-10-08T00:56:40.162377+0000","last_undegraded":"2023-10-08T00:56:40.162377+0000","last_fullsized":"2023-10-08T00:56:40.162377+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:37.050655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143998+0000","last_change":"2023-10-08T00:56:40.143998+0000","last_active":"2023-10-08T00:56:40.143998+0000","last_peered":"2023-10-08T00:56:40.143998+0000","last_clean":"2023-10-08T00:56:40.143998+0000","last_became_active":"2023-10-08T00:56:40.143684+0000","last_became_peered":"2023-10-08T00:56:40.143684+0000","last_unstale":"2023-10-08T00:56:40.143998+0000","last_undegraded":"2023-10-08T00:56:40.143998+0000","last_fullsized":"2023-10-08T00:56:40.143998+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:02.299184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.163818+0000","last_change":"2023-10-08T00:56:40.163818+0000","last_active":"2023-10-08T00:56:40.163818+0000","last_peered":"2023-10-08T00:56:40.163818+0000","last_clean":"2023-10-08T00:56:40.163818+0000","last_became_active":"2023-10-08T00:56:40.163260+0000","last_became_peered":"2023-10-08T00:56:40.163260+0000","last_unstale":"2023-10-08T00:56:40.163818+0000","last_undegraded":"2023-10-08T00:56:40.163818+0000","last_fullsized":"2023-10-08T00:56:40.163818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:32:10.599348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.153895+0000","last_change":"2023-10-08T00:56:40.153895+0000","last_active":"2023-10-08T00:56:40.153895+0000","last_peered":"2023-10-08T00:56:40.153895+0000","last_clean":"2023-10-08T00:56:40.153895+0000","last_became_active":"2023-10-08T00:56:40.153554+0000","last_became_peered":"2023-10-08T00:56:40.153554+0000","last_unstale":"2023-10-08T00:56:40.153895+0000","last_undegraded":"2023-10-08T00:56:40.153895+0000","last_fullsized":"2023-10-08T00:56:40.153895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:20:27.686995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.164491+0000","last_change":"2023-10-08T00:56:40.164491+0000","last_active":"2023-10-08T00:56:40.164491+0000","last_peered":"2023-10-08T00:56:40.164491+0000","last_clean":"2023-10-08T00:56:40.164491+0000","last_became_active":"2023-10-08T00:56:40.164231+0000","last_became_peered":"2023-10-08T00:56:40.164231+0000","last_unstale":"2023-10-08T00:56:40.164491+0000","last_undegraded":"2023-10-08T00:56:40.164491+0000","last_fullsized":"2023-10-08T00:56:40.164491+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:16.075601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.142758+0000","last_change":"2023-10-08T00:56:40.142758+0000","last_active":"2023-10-08T00:56:40.142758+0000","last_peered":"2023-10-08T00:56:40.142758+0000","last_clean":"2023-10-08T00:56:40.142758+0000","last_became_active":"2023-10-08T00:56:40.142449+0000","last_became_peered":"2023-10-08T00:56:40.142449+0000","last_unstale":"2023-10-08T00:56:40.142758+0000","last_undegraded":"2023-10-08T00:56:40.142758+0000","last_fullsized":"2023-10-08T00:56:40.142758+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:32:10.599348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.136840+0000","last_change":"2023-10-08T00:56:40.136840+0000","last_active":"2023-10-08T00:56:40.136840+0000","last_peered":"2023-10-08T00:56:40.136840+0000","last_clean":"2023-10-08T00:56:40.136840+0000","last_became_active":"2023-10-08T00:56:40.136494+0000","last_became_peered":"2023-10-08T00:56:40.136494+0000","last_unstale":"2023-10-08T00:56:40.136840+0000","last_undegraded":"2023-10-08T00:56:40.136840+0000","last_fullsized":"2023-10-08T00:56:40.136840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:47.749054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.146968+0000","last_change":"2023-10-08T00:56:40.146968+0000","last_active":"2023-10-08T00:56:40.146968+0000","last_peered":"2023-10-08T00:56:40.146968+0000","last_clean":"2023-10-08T00:56:40.146968+0000","last_became_active":"2023-10-08T00:56:40.146718+0000","last_became_peered":"2023-10-08T00:56:40.146718+0000","last_unstale":"2023-10-08T00:56:40.146968+0000","last_undegraded":"2023-10-08T00:56:40.146968+0000","last_fullsized":"2023-10-08T00:56:40.146968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:58.921683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140895+0000","last_change":"2023-10-08T00:56:40.140895+0000","last_active":"2023-10-08T00:56:40.140895+0000","last_peered":"2023-10-08T00:56:40.140895+0000","last_clean":"2023-10-08T00:56:40.140895+0000","last_became_active":"2023-10-08T00:56:40.140589+0000","last_became_peered":"2023-10-08T00:56:40.140589+0000","last_unstale":"2023-10-08T00:56:40.140895+0000","last_undegraded":"2023-10-08T00:56:40.140895+0000","last_fullsized":"2023-10-08T00:56:40.140895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:24.800653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162623+0000","last_change":"2023-10-08T00:56:39.121635+0000","last_active":"2023-10-08T00:56:40.162623+0000","last_peered":"2023-10-08T00:56:40.162623+0000","last_clean":"2023-10-08T00:56:40.162623+0000","last_became_active":"2023-10-08T00:56:39.121330+0000","last_became_peered":"2023-10-08T00:56:39.121330+0000","last_unstale":"2023-10-08T00:56:40.162623+0000","last_undegraded":"2023-10-08T00:56:40.162623+0000","last_fullsized":"2023-10-08T00:56:40.162623+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:51:30.476911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143613+0000","last_change":"2023-10-08T00:56:40.143613+0000","last_active":"2023-10-08T00:56:40.143613+0000","last_peered":"2023-10-08T00:56:40.143613+0000","last_clean":"2023-10-08T00:56:40.143613+0000","last_became_active":"2023-10-08T00:56:40.143359+0000","last_became_peered":"2023-10-08T00:56:40.143359+0000","last_unstale":"2023-10-08T00:56:40.143613+0000","last_undegraded":"2023-10-08T00:56:40.143613+0000","last_fullsized":"2023-10-08T00:56:40.143613+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:15.309253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.129233+0000","last_change":"2023-10-08T00:56:39.134938+0000","last_active":"2023-10-08T00:56:40.129233+0000","last_peered":"2023-10-08T00:56:40.129233+0000","last_clean":"2023-10-08T00:56:40.129233+0000","last_became_active":"2023-10-08T00:56:39.134622+0000","last_became_peered":"2023-10-08T00:56:39.134622+0000","last_unstale":"2023-10-08T00:56:40.129233+0000","last_undegraded":"2023-10-08T00:56:40.129233+0000","last_fullsized":"2023-10-08T00:56:40.129233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:22.998880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.160449+0000","last_change":"2023-10-08T00:56:40.160449+0000","last_active":"2023-10-08T00:56:40.160449+0000","last_peered":"2023-10-08T00:56:40.160449+0000","last_clean":"2023-10-08T00:56:40.160449+0000","last_became_active":"2023-10-08T00:56:40.160221+0000","last_became_peered":"2023-10-08T00:56:40.160221+0000","last_unstale":"2023-10-08T00:56:40.160449+0000","last_undegraded":"2023-10-08T00:56:40.160449+0000","last_fullsized":"2023-10-08T00:56:40.160449+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:53.246518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.167367+0000","last_change":"2023-10-08T00:56:39.126463+0000","last_active":"2023-10-08T00:56:40.167367+0000","last_peered":"2023-10-08T00:56:40.167367+0000","last_clean":"2023-10-08T00:56:40.167367+0000","last_became_active":"2023-10-08T00:56:39.126177+0000","last_became_peered":"2023-10-08T00:56:39.126177+0000","last_unstale":"2023-10-08T00:56:40.167367+0000","last_undegraded":"2023-10-08T00:56:40.167367+0000","last_fullsized":"2023-10-08T00:56:40.167367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:46.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.163648+0000","last_change":"2023-10-08T00:56:40.163648+0000","last_active":"2023-10-08T00:56:40.163648+0000","last_peered":"2023-10-08T00:56:40.163648+0000","last_clean":"2023-10-08T00:56:40.163648+0000","last_became_active":"2023-10-08T00:56:40.162993+0000","last_became_peered":"2023-10-08T00:56:40.162993+0000","last_unstale":"2023-10-08T00:56:40.163648+0000","last_undegraded":"2023-10-08T00:56:40.163648+0000","last_fullsized":"2023-10-08T00:56:40.163648+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:30.610379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.164906+0000","last_change":"2023-10-08T00:56:39.118290+0000","last_active":"2023-10-08T00:56:40.164906+0000","last_peered":"2023-10-08T00:56:40.164906+0000","last_clean":"2023-10-08T00:56:40.164906+0000","last_became_active":"2023-10-08T00:56:39.117915+0000","last_became_peered":"2023-10-08T00:56:39.117915+0000","last_unstale":"2023-10-08T00:56:40.164906+0000","last_undegraded":"2023-10-08T00:56:40.164906+0000","last_fullsized":"2023-10-08T00:56:40.164906+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:24.142738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.147736+0000","last_change":"2023-10-08T00:56:40.147736+0000","last_active":"2023-10-08T00:56:40.147736+0000","last_peered":"2023-10-08T00:56:40.147736+0000","last_clean":"2023-10-08T00:56:40.147736+0000","last_became_active":"2023-10-08T00:56:40.147400+0000","last_became_peered":"2023-10-08T00:56:40.147400+0000","last_unstale":"2023-10-08T00:56:40.147736+0000","last_undegraded":"2023-10-08T00:56:40.147736+0000","last_fullsized":"2023-10-08T00:56:40.147736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:37.050655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.136561+0000","last_change":"2023-10-08T00:56:39.128607+0000","last_active":"2023-10-08T00:56:40.136561+0000","last_peered":"2023-10-08T00:56:40.136561+0000","last_clean":"2023-10-08T00:56:40.136561+0000","last_became_active":"2023-10-08T00:56:39.127151+0000","last_became_peered":"2023-10-08T00:56:39.127151+0000","last_unstale":"2023-10-08T00:56:40.136561+0000","last_undegraded":"2023-10-08T00:56:40.136561+0000","last_fullsized":"2023-10-08T00:56:40.136561+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:15:15.964678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.155045+0000","last_change":"2023-10-08T00:56:40.155045+0000","last_active":"2023-10-08T00:56:40.155045+0000","last_peered":"2023-10-08T00:56:40.155045+0000","last_clean":"2023-10-08T00:56:40.155045+0000","last_became_active":"2023-10-08T00:56:40.154768+0000","last_became_peered":"2023-10-08T00:56:40.154768+0000","last_unstale":"2023-10-08T00:56:40.155045+0000","last_undegraded":"2023-10-08T00:56:40.155045+0000","last_fullsized":"2023-10-08T00:56:40.155045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.135308+0000","last_change":"2023-10-08T00:56:39.128735+0000","last_active":"2023-10-08T00:56:40.135308+0000","last_peered":"2023-10-08T00:56:40.135308+0000","last_clean":"2023-10-08T00:56:40.135308+0000","last_became_active":"2023-10-08T00:56:39.128392+0000","last_became_peered":"2023-10-08T00:56:39.128392+0000","last_unstale":"2023-10-08T00:56:40.135308+0000","last_undegraded":"2023-10-08T00:56:40.135308+0000","last_fullsized":"2023-10-08T00:56:40.135308+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:05.959100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140482+0000","last_change":"2023-10-08T00:56:40.140482+0000","last_active":"2023-10-08T00:56:40.140482+0000","last_peered":"2023-10-08T00:56:40.140482+0000","last_clean":"2023-10-08T00:56:40.140482+0000","last_became_active":"2023-10-08T00:56:40.138728+0000","last_became_peered":"2023-10-08T00:56:40.138728+0000","last_unstale":"2023-10-08T00:56:40.140482+0000","last_undegraded":"2023-10-08T00:56:40.140482+0000","last_fullsized":"2023-10-08T00:56:40.140482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:59:08.290249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.127626+0000","last_change":"2023-10-08T00:56:39.126328+0000","last_active":"2023-10-08T00:56:39.127626+0000","last_peered":"2023-10-08T00:56:39.127626+0000","last_clean":"2023-10-08T00:56:39.127626+0000","last_became_active":"2023-10-08T00:56:39.126020+0000","last_became_peered":"2023-10-08T00:56:39.126020+0000","last_unstale":"2023-10-08T00:56:39.127626+0000","last_undegraded":"2023-10-08T00:56:39.127626+0000","last_fullsized":"2023-10-08T00:56:39.127626+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:46.742312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.155211+0000","last_change":"2023-10-08T00:56:40.155211+0000","last_active":"2023-10-08T00:56:40.155211+0000","last_peered":"2023-10-08T00:56:40.155211+0000","last_clean":"2023-10-08T00:56:40.155211+0000","last_became_active":"2023-10-08T00:56:40.154923+0000","last_became_peered":"2023-10-08T00:56:40.154923+0000","last_unstale":"2023-10-08T00:56:40.155211+0000","last_undegraded":"2023-10-08T00:56:40.155211+0000","last_fullsized":"2023-10-08T00:56:40.155211+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:59.609984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.125132+0000","last_change":"2023-10-08T00:56:39.120650+0000","last_active":"2023-10-08T00:56:39.125132+0000","last_peered":"2023-10-08T00:56:39.125132+0000","last_clean":"2023-10-08T00:56:39.125132+0000","last_became_active":"2023-10-08T00:56:39.120364+0000","last_became_peered":"2023-10-08T00:56:39.120364+0000","last_unstale":"2023-10-08T00:56:39.125132+0000","last_undegraded":"2023-10-08T00:56:39.125132+0000","last_fullsized":"2023-10-08T00:56:39.125132+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:00.290088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.151821+0000","last_change":"2023-10-08T00:56:40.151821+0000","last_active":"2023-10-08T00:56:40.151821+0000","last_peered":"2023-10-08T00:56:40.151821+0000","last_clean":"2023-10-08T00:56:40.151821+0000","last_became_active":"2023-10-08T00:56:40.151557+0000","last_became_peered":"2023-10-08T00:56:40.151557+0000","last_unstale":"2023-10-08T00:56:40.151821+0000","last_undegraded":"2023-10-08T00:56:40.151821+0000","last_fullsized":"2023-10-08T00:56:40.151821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:30.610379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.134134+0000","last_change":"2023-10-08T00:56:39.133218+0000","last_active":"2023-10-08T00:56:39.134134+0000","last_peered":"2023-10-08T00:56:39.134134+0000","last_clean":"2023-10-08T00:56:39.134134+0000","last_became_active":"2023-10-08T00:56:39.132889+0000","last_became_peered":"2023-10-08T00:56:39.132889+0000","last_unstale":"2023-10-08T00:56:39.134134+0000","last_undegraded":"2023-10-08T00:56:39.134134+0000","last_fullsized":"2023-10-08T00:56:39.134134+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:15.068859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.164743+0000","last_change":"2023-10-08T00:56:40.164743+0000","last_active":"2023-10-08T00:56:40.164743+0000","last_peered":"2023-10-08T00:56:40.164743+0000","last_clean":"2023-10-08T00:56:40.164743+0000","last_became_active":"2023-10-08T00:56:40.164154+0000","last_became_peered":"2023-10-08T00:56:40.164154+0000","last_unstale":"2023-10-08T00:56:40.164743+0000","last_undegraded":"2023-10-08T00:56:40.164743+0000","last_fullsized":"2023-10-08T00:56:40.164743+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:00.141477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.123050+0000","last_change":"2023-10-08T00:56:39.119761+0000","last_active":"2023-10-08T00:56:39.123050+0000","last_peered":"2023-10-08T00:56:39.123050+0000","last_clean":"2023-10-08T00:56:39.123050+0000","last_became_active":"2023-10-08T00:56:39.119464+0000","last_became_peered":"2023-10-08T00:56:39.119464+0000","last_unstale":"2023-10-08T00:56:39.123050+0000","last_undegraded":"2023-10-08T00:56:39.123050+0000","last_fullsized":"2023-10-08T00:56:39.123050+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:17.383525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143570+0000","last_change":"2023-10-08T00:56:40.143570+0000","last_active":"2023-10-08T00:56:40.143570+0000","last_peered":"2023-10-08T00:56:40.143570+0000","last_clean":"2023-10-08T00:56:40.143570+0000","last_became_active":"2023-10-08T00:56:40.143276+0000","last_became_peered":"2023-10-08T00:56:40.143276+0000","last_unstale":"2023-10-08T00:56:40.143570+0000","last_undegraded":"2023-10-08T00:56:40.143570+0000","last_fullsized":"2023-10-08T00:56:40.143570+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:10.506479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.153174+0000","last_change":"2023-10-08T00:56:39.133022+0000","last_active":"2023-10-08T00:56:40.153174+0000","last_peered":"2023-10-08T00:56:40.153174+0000","last_clean":"2023-10-08T00:56:40.153174+0000","last_became_active":"2023-10-08T00:56:39.132745+0000","last_became_peered":"2023-10-08T00:56:39.132745+0000","last_unstale":"2023-10-08T00:56:40.153174+0000","last_undegraded":"2023-10-08T00:56:40.153174+0000","last_fullsized":"2023-10-08T00:56:40.153174+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:38.708733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166944+0000","last_change":"2023-10-08T00:56:40.166944+0000","last_active":"2023-10-08T00:56:40.166944+0000","last_peered":"2023-10-08T00:56:40.166944+0000","last_clean":"2023-10-08T00:56:40.166944+0000","last_became_active":"2023-10-08T00:56:40.166570+0000","last_became_peered":"2023-10-08T00:56:40.166570+0000","last_unstale":"2023-10-08T00:56:40.166944+0000","last_undegraded":"2023-10-08T00:56:40.166944+0000","last_fullsized":"2023-10-08T00:56:40.166944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:32:10.599348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.170014+0000","last_change":"2023-10-08T00:56:39.123529+0000","last_active":"2023-10-08T00:56:40.170014+0000","last_peered":"2023-10-08T00:56:40.170014+0000","last_clean":"2023-10-08T00:56:40.170014+0000","last_became_active":"2023-10-08T00:56:39.123280+0000","last_became_peered":"2023-10-08T00:56:39.123280+0000","last_unstale":"2023-10-08T00:56:40.170014+0000","last_undegraded":"2023-10-08T00:56:40.170014+0000","last_fullsized":"2023-10-08T00:56:40.170014+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.307041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.158852+0000","last_change":"2023-10-08T00:56:40.158852+0000","last_active":"2023-10-08T00:56:40.158852+0000","last_peered":"2023-10-08T00:56:40.158852+0000","last_clean":"2023-10-08T00:56:40.158852+0000","last_became_active":"2023-10-08T00:56:40.158516+0000","last_became_peered":"2023-10-08T00:56:40.158516+0000","last_unstale":"2023-10-08T00:56:40.158852+0000","last_undegraded":"2023-10-08T00:56:40.158852+0000","last_fullsized":"2023-10-08T00:56:40.158852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:59.756389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162546+0000","last_change":"2023-10-08T00:56:39.129980+0000","last_active":"2023-10-08T00:56:40.162546+0000","last_peered":"2023-10-08T00:56:40.162546+0000","last_clean":"2023-10-08T00:56:40.162546+0000","last_became_active":"2023-10-08T00:56:39.126062+0000","last_became_peered":"2023-10-08T00:56:39.126062+0000","last_unstale":"2023-10-08T00:56:40.162546+0000","last_undegraded":"2023-10-08T00:56:40.162546+0000","last_fullsized":"2023-10-08T00:56:40.162546+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:59.321210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.154251+0000","last_change":"2023-10-08T00:56:40.154251+0000","last_active":"2023-10-08T00:56:40.154251+0000","last_peered":"2023-10-08T00:56:40.154251+0000","last_clean":"2023-10-08T00:56:40.154251+0000","last_became_active":"2023-10-08T00:56:40.153878+0000","last_became_peered":"2023-10-08T00:56:40.153878+0000","last_unstale":"2023-10-08T00:56:40.154251+0000","last_undegraded":"2023-10-08T00:56:40.154251+0000","last_fullsized":"2023-10-08T00:56:40.154251+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:33.219535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.122173+0000","last_change":"2023-10-08T00:56:39.116641+0000","last_active":"2023-10-08T00:56:39.122173+0000","last_peered":"2023-10-08T00:56:39.122173+0000","last_clean":"2023-10-08T00:56:39.122173+0000","last_became_active":"2023-10-08T00:56:39.115728+0000","last_became_peered":"2023-10-08T00:56:39.115728+0000","last_unstale":"2023-10-08T00:56:39.122173+0000","last_undegraded":"2023-10-08T00:56:39.122173+0000","last_fullsized":"2023-10-08T00:56:39.122173+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:13.696837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.158206+0000","last_change":"2023-10-08T00:56:40.158206+0000","last_active":"2023-10-08T00:56:40.158206+0000","last_peered":"2023-10-08T00:56:40.158206+0000","last_clean":"2023-10-08T00:56:40.158206+0000","last_became_active":"2023-10-08T00:56:40.157885+0000","last_became_peered":"2023-10-08T00:56:40.157885+0000","last_unstale":"2023-10-08T00:56:40.158206+0000","last_undegraded":"2023-10-08T00:56:40.158206+0000","last_fullsized":"2023-10-08T00:56:40.158206+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.133588+0000","last_change":"2023-10-08T00:56:39.132658+0000","last_active":"2023-10-08T00:56:39.133588+0000","last_peered":"2023-10-08T00:56:39.133588+0000","last_clean":"2023-10-08T00:56:39.133588+0000","last_became_active":"2023-10-08T00:56:39.132251+0000","last_became_peered":"2023-10-08T00:56:39.132251+0000","last_unstale":"2023-10-08T00:56:39.133588+0000","last_undegraded":"2023-10-08T00:56:39.133588+0000","last_fullsized":"2023-10-08T00:56:39.133588+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:23.793911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.151078+0000","last_change":"2023-10-08T00:56:40.151078+0000","last_active":"2023-10-08T00:56:40.151078+0000","last_peered":"2023-10-08T00:56:40.151078+0000","last_clean":"2023-10-08T00:56:40.151078+0000","last_became_active":"2023-10-08T00:56:40.150782+0000","last_became_peered":"2023-10-08T00:56:40.150782+0000","last_unstale":"2023-10-08T00:56:40.151078+0000","last_undegraded":"2023-10-08T00:56:40.151078+0000","last_fullsized":"2023-10-08T00:56:40.151078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:47.023393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.169913+0000","last_change":"2023-10-08T00:56:39.119286+0000","last_active":"2023-10-08T00:56:40.169913+0000","last_peered":"2023-10-08T00:56:40.169913+0000","last_clean":"2023-10-08T00:56:40.169913+0000","last_became_active":"2023-10-08T00:56:39.118993+0000","last_became_peered":"2023-10-08T00:56:39.118993+0000","last_unstale":"2023-10-08T00:56:40.169913+0000","last_undegraded":"2023-10-08T00:56:40.169913+0000","last_fullsized":"2023-10-08T00:56:40.169913+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:52.239776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.147586+0000","last_change":"2023-10-08T00:56:40.147586+0000","last_active":"2023-10-08T00:56:40.147586+0000","last_peered":"2023-10-08T00:56:40.147586+0000","last_clean":"2023-10-08T00:56:40.147586+0000","last_became_active":"2023-10-08T00:56:40.147259+0000","last_became_peered":"2023-10-08T00:56:40.147259+0000","last_unstale":"2023-10-08T00:56:40.147586+0000","last_undegraded":"2023-10-08T00:56:40.147586+0000","last_fullsized":"2023-10-08T00:56:40.147586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:39.715475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.145986+0000","last_change":"2023-10-08T00:56:39.127361+0000","last_active":"2023-10-08T00:56:40.145986+0000","last_peered":"2023-10-08T00:56:40.145986+0000","last_clean":"2023-10-08T00:56:40.145986+0000","last_became_active":"2023-10-08T00:56:39.127106+0000","last_became_peered":"2023-10-08T00:56:39.127106+0000","last_unstale":"2023-10-08T00:56:40.145986+0000","last_undegraded":"2023-10-08T00:56:40.145986+0000","last_fullsized":"2023-10-08T00:56:40.145986+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:40.252121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.151703+0000","last_change":"2023-10-08T00:56:40.151703+0000","last_active":"2023-10-08T00:56:40.151703+0000","last_peered":"2023-10-08T00:56:40.151703+0000","last_clean":"2023-10-08T00:56:40.151703+0000","last_became_active":"2023-10-08T00:56:40.151415+0000","last_became_peered":"2023-10-08T00:56:40.151415+0000","last_unstale":"2023-10-08T00:56:40.151703+0000","last_undegraded":"2023-10-08T00:56:40.151703+0000","last_fullsized":"2023-10-08T00:56:40.151703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:48.913792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162813+0000","last_change":"2023-10-08T00:56:39.125489+0000","last_active":"2023-10-08T00:56:40.162813+0000","last_peered":"2023-10-08T00:56:40.162813+0000","last_clean":"2023-10-08T00:56:40.162813+0000","last_became_active":"2023-10-08T00:56:39.125245+0000","last_became_peered":"2023-10-08T00:56:39.125245+0000","last_unstale":"2023-10-08T00:56:40.162813+0000","last_undegraded":"2023-10-08T00:56:40.162813+0000","last_fullsized":"2023-10-08T00:56:40.162813+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:44.534690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.159049+0000","last_change":"2023-10-08T00:56:40.159049+0000","last_active":"2023-10-08T00:56:40.159049+0000","last_peered":"2023-10-08T00:56:40.159049+0000","last_clean":"2023-10-08T00:56:40.159049+0000","last_became_active":"2023-10-08T00:56:40.158632+0000","last_became_peered":"2023-10-08T00:56:40.158632+0000","last_unstale":"2023-10-08T00:56:40.159049+0000","last_undegraded":"2023-10-08T00:56:40.159049+0000","last_fullsized":"2023-10-08T00:56:40.159049+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:27.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.141025+0000","last_change":"2023-10-08T00:56:39.129189+0000","last_active":"2023-10-08T00:56:40.141025+0000","last_peered":"2023-10-08T00:56:40.141025+0000","last_clean":"2023-10-08T00:56:40.141025+0000","last_became_active":"2023-10-08T00:56:39.128873+0000","last_became_peered":"2023-10-08T00:56:39.128873+0000","last_unstale":"2023-10-08T00:56:40.141025+0000","last_undegraded":"2023-10-08T00:56:40.141025+0000","last_fullsized":"2023-10-08T00:56:40.141025+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:32:26.820438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.142116+0000","last_change":"2023-10-08T00:56:40.142116+0000","last_active":"2023-10-08T00:56:40.142116+0000","last_peered":"2023-10-08T00:56:40.142116+0000","last_clean":"2023-10-08T00:56:40.142116+0000","last_became_active":"2023-10-08T00:56:40.141828+0000","last_became_peered":"2023-10-08T00:56:40.141828+0000","last_unstale":"2023-10-08T00:56:40.142116+0000","last_undegraded":"2023-10-08T00:56:40.142116+0000","last_fullsized":"2023-10-08T00:56:40.142116+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:39.715475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.134036+0000","last_change":"2023-10-08T00:56:39.128608+0000","last_active":"2023-10-08T00:56:39.134036+0000","last_peered":"2023-10-08T00:56:39.134036+0000","last_clean":"2023-10-08T00:56:39.134036+0000","last_became_active":"2023-10-08T00:56:39.127152+0000","last_became_peered":"2023-10-08T00:56:39.127152+0000","last_unstale":"2023-10-08T00:56:39.134036+0000","last_undegraded":"2023-10-08T00:56:39.134036+0000","last_fullsized":"2023-10-08T00:56:39.134036+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:07.575298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.146094+0000","last_change":"2023-10-08T00:56:40.146094+0000","last_active":"2023-10-08T00:56:40.146094+0000","last_peered":"2023-10-08T00:56:40.146094+0000","last_clean":"2023-10-08T00:56:40.146094+0000","last_became_active":"2023-10-08T00:56:40.145805+0000","last_became_peered":"2023-10-08T00:56:40.145805+0000","last_unstale":"2023-10-08T00:56:40.146094+0000","last_undegraded":"2023-10-08T00:56:40.146094+0000","last_fullsized":"2023-10-08T00:56:40.146094+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:14.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.165193+0000","last_change":"2023-10-08T00:56:39.124087+0000","last_active":"2023-10-08T00:56:40.165193+0000","last_peered":"2023-10-08T00:56:40.165193+0000","last_clean":"2023-10-08T00:56:40.165193+0000","last_became_active":"2023-10-08T00:56:39.123837+0000","last_became_peered":"2023-10-08T00:56:39.123837+0000","last_unstale":"2023-10-08T00:56:40.165193+0000","last_undegraded":"2023-10-08T00:56:40.165193+0000","last_fullsized":"2023-10-08T00:56:40.165193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:13.696837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.131072+0000","last_change":"2023-10-08T00:56:40.131072+0000","last_active":"2023-10-08T00:56:40.131072+0000","last_peered":"2023-10-08T00:56:40.131072+0000","last_clean":"2023-10-08T00:56:40.131072+0000","last_became_active":"2023-10-08T00:56:40.130777+0000","last_became_peered":"2023-10-08T00:56:40.130777+0000","last_unstale":"2023-10-08T00:56:40.131072+0000","last_undegraded":"2023-10-08T00:56:40.131072+0000","last_fullsized":"2023-10-08T00:56:40.131072+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:59:08.290249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.120044+0000","last_change":"2023-10-08T00:56:39.117963+0000","last_active":"2023-10-08T00:56:39.120044+0000","last_peered":"2023-10-08T00:56:39.120044+0000","last_clean":"2023-10-08T00:56:39.120044+0000","last_became_active":"2023-10-08T00:56:39.117518+0000","last_became_peered":"2023-10-08T00:56:39.117518+0000","last_unstale":"2023-10-08T00:56:39.120044+0000","last_undegraded":"2023-10-08T00:56:39.120044+0000","last_fullsized":"2023-10-08T00:56:39.120044+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:32:52.556872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.131110+0000","last_change":"2023-10-08T00:56:40.131110+0000","last_active":"2023-10-08T00:56:40.131110+0000","last_peered":"2023-10-08T00:56:40.131110+0000","last_clean":"2023-10-08T00:56:40.131110+0000","last_became_active":"2023-10-08T00:56:40.130817+0000","last_became_peered":"2023-10-08T00:56:40.130817+0000","last_unstale":"2023-10-08T00:56:40.131110+0000","last_undegraded":"2023-10-08T00:56:40.131110+0000","last_fullsized":"2023-10-08T00:56:40.131110+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:51:31.483653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.137557+0000","last_change":"2023-10-08T00:56:39.131192+0000","last_active":"2023-10-08T00:56:39.137557+0000","last_peered":"2023-10-08T00:56:39.137557+0000","last_clean":"2023-10-08T00:56:39.137557+0000","last_became_active":"2023-10-08T00:56:39.130881+0000","last_became_peered":"2023-10-08T00:56:39.130881+0000","last_unstale":"2023-10-08T00:56:39.137557+0000","last_undegraded":"2023-10-08T00:56:39.137557+0000","last_fullsized":"2023-10-08T00:56:39.137557+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:24.142738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140733+0000","last_change":"2023-10-08T00:56:40.140733+0000","last_active":"2023-10-08T00:56:40.140733+0000","last_peered":"2023-10-08T00:56:40.140733+0000","last_clean":"2023-10-08T00:56:40.140733+0000","last_became_active":"2023-10-08T00:56:40.139984+0000","last_became_peered":"2023-10-08T00:56:40.139984+0000","last_unstale":"2023-10-08T00:56:40.140733+0000","last_undegraded":"2023-10-08T00:56:40.140733+0000","last_fullsized":"2023-10-08T00:56:40.140733+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:24.551023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.137872+0000","last_change":"2023-10-08T00:56:39.134838+0000","last_active":"2023-10-08T00:56:39.137872+0000","last_peered":"2023-10-08T00:56:39.137872+0000","last_clean":"2023-10-08T00:56:39.137872+0000","last_became_active":"2023-10-08T00:56:39.134523+0000","last_became_peered":"2023-10-08T00:56:39.134523+0000","last_unstale":"2023-10-08T00:56:39.137872+0000","last_undegraded":"2023-10-08T00:56:39.137872+0000","last_fullsized":"2023-10-08T00:56:39.137872+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:59:07.283507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.145421+0000","last_change":"2023-10-08T00:56:40.145421+0000","last_active":"2023-10-08T00:56:40.145421+0000","last_peered":"2023-10-08T00:56:40.145421+0000","last_clean":"2023-10-08T00:56:40.145421+0000","last_became_active":"2023-10-08T00:56:40.144856+0000","last_became_peered":"2023-10-08T00:56:40.144856+0000","last_unstale":"2023-10-08T00:56:40.145421+0000","last_undegraded":"2023-10-08T00:56:40.145421+0000","last_fullsized":"2023-10-08T00:56:40.145421+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:26.715198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166495+0000","last_change":"2023-10-08T00:56:39.119709+0000","last_active":"2023-10-08T00:56:40.166495+0000","last_peered":"2023-10-08T00:56:40.166495+0000","last_clean":"2023-10-08T00:56:40.166495+0000","last_became_active":"2023-10-08T00:56:39.119470+0000","last_became_peered":"2023-10-08T00:56:39.119470+0000","last_unstale":"2023-10-08T00:56:40.166495+0000","last_undegraded":"2023-10-08T00:56:40.166495+0000","last_fullsized":"2023-10-08T00:56:40.166495+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:22.998880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166939+0000","last_change":"2023-10-08T00:56:40.166939+0000","last_active":"2023-10-08T00:56:40.166939+0000","last_peered":"2023-10-08T00:56:40.166939+0000","last_clean":"2023-10-08T00:56:40.166939+0000","last_became_active":"2023-10-08T00:56:40.166616+0000","last_became_peered":"2023-10-08T00:56:40.166616+0000","last_unstale":"2023-10-08T00:56:40.166939+0000","last_undegraded":"2023-10-08T00:56:40.166939+0000","last_fullsized":"2023-10-08T00:56:40.166939+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:37.050655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.125246+0000","last_change":"2023-10-08T00:56:39.124211+0000","last_active":"2023-10-08T00:56:39.125246+0000","last_peered":"2023-10-08T00:56:39.125246+0000","last_clean":"2023-10-08T00:56:39.125246+0000","last_became_active":"2023-10-08T00:56:39.123936+0000","last_became_peered":"2023-10-08T00:56:39.123936+0000","last_unstale":"2023-10-08T00:56:39.125246+0000","last_undegraded":"2023-10-08T00:56:39.125246+0000","last_fullsized":"2023-10-08T00:56:39.125246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:40.252121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.165576+0000","last_change":"2023-10-08T00:56:40.165576+0000","last_active":"2023-10-08T00:56:40.165576+0000","last_peered":"2023-10-08T00:56:40.165576+0000","last_clean":"2023-10-08T00:56:40.165576+0000","last_became_active":"2023-10-08T00:56:40.165293+0000","last_became_peered":"2023-10-08T00:56:40.165293+0000","last_unstale":"2023-10-08T00:56:40.165576+0000","last_undegraded":"2023-10-08T00:56:40.165576+0000","last_fullsized":"2023-10-08T00:56:40.165576+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:16.075601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.128304+0000","last_change":"2023-10-08T00:56:39.125519+0000","last_active":"2023-10-08T00:56:39.128304+0000","last_peered":"2023-10-08T00:56:39.128304+0000","last_clean":"2023-10-08T00:56:39.128304+0000","last_became_active":"2023-10-08T00:56:39.125300+0000","last_became_peered":"2023-10-08T00:56:39.125300+0000","last_unstale":"2023-10-08T00:56:39.128304+0000","last_undegraded":"2023-10-08T00:56:39.128304+0000","last_fullsized":"2023-10-08T00:56:39.128304+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:17.383525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.153773+0000","last_change":"2023-10-08T00:56:40.153773+0000","last_active":"2023-10-08T00:56:40.153773+0000","last_peered":"2023-10-08T00:56:40.153773+0000","last_clean":"2023-10-08T00:56:40.153773+0000","last_became_active":"2023-10-08T00:56:40.153431+0000","last_became_peered":"2023-10-08T00:56:40.153431+0000","last_unstale":"2023-10-08T00:56:40.153773+0000","last_undegraded":"2023-10-08T00:56:40.153773+0000","last_fullsized":"2023-10-08T00:56:40.153773+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:53.246518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.134949+0000","last_change":"2023-10-08T00:56:39.131851+0000","last_active":"2023-10-08T00:56:39.134949+0000","last_peered":"2023-10-08T00:56:39.134949+0000","last_clean":"2023-10-08T00:56:39.134949+0000","last_became_active":"2023-10-08T00:56:39.131574+0000","last_became_peered":"2023-10-08T00:56:39.131574+0000","last_unstale":"2023-10-08T00:56:39.134949+0000","last_undegraded":"2023-10-08T00:56:39.134949+0000","last_fullsized":"2023-10-08T00:56:39.134949+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:59:07.283507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.133915+0000","last_change":"2023-10-08T00:56:40.133915+0000","last_active":"2023-10-08T00:56:40.133915+0000","last_peered":"2023-10-08T00:56:40.133915+0000","last_clean":"2023-10-08T00:56:40.133915+0000","last_became_active":"2023-10-08T00:56:40.133648+0000","last_became_peered":"2023-10-08T00:56:40.133648+0000","last_unstale":"2023-10-08T00:56:40.133915+0000","last_undegraded":"2023-10-08T00:56:40.133915+0000","last_fullsized":"2023-10-08T00:56:40.133915+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:47.749054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.124280+0000","last_change":"2023-10-08T00:56:39.121401+0000","last_active":"2023-10-08T00:56:39.124280+0000","last_peered":"2023-10-08T00:56:39.124280+0000","last_clean":"2023-10-08T00:56:39.124280+0000","last_became_active":"2023-10-08T00:56:39.121104+0000","last_became_peered":"2023-10-08T00:56:39.121104+0000","last_unstale":"2023-10-08T00:56:39.124280+0000","last_undegraded":"2023-10-08T00:56:39.124280+0000","last_fullsized":"2023-10-08T00:56:39.124280+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:25.708456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.160230+0000","last_change":"2023-10-08T00:56:40.160230+0000","last_active":"2023-10-08T00:56:40.160230+0000","last_peered":"2023-10-08T00:56:40.160230+0000","last_clean":"2023-10-08T00:56:40.160230+0000","last_became_active":"2023-10-08T00:56:40.159987+0000","last_became_peered":"2023-10-08T00:56:40.159987+0000","last_unstale":"2023-10-08T00:56:40.160230+0000","last_undegraded":"2023-10-08T00:56:40.160230+0000","last_fullsized":"2023-10-08T00:56:40.160230+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:24.539053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.137571+0000","last_change":"2023-10-08T00:56:39.131798+0000","last_active":"2023-10-08T00:56:39.137571+0000","last_peered":"2023-10-08T00:56:39.137571+0000","last_clean":"2023-10-08T00:56:39.137571+0000","last_became_active":"2023-10-08T00:56:39.131483+0000","last_became_peered":"2023-10-08T00:56:39.131483+0000","last_unstale":"2023-10-08T00:56:39.137571+0000","last_undegraded":"2023-10-08T00:56:39.137571+0000","last_fullsized":"2023-10-08T00:56:39.137571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:26.416748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.160128+0000","last_change":"2023-10-08T00:56:40.160128+0000","last_active":"2023-10-08T00:56:40.160128+0000","last_peered":"2023-10-08T00:56:40.160128+0000","last_clean":"2023-10-08T00:56:40.160128+0000","last_became_active":"2023-10-08T00:56:40.159843+0000","last_became_peered":"2023-10-08T00:56:40.159843+0000","last_unstale":"2023-10-08T00:56:40.160128+0000","last_undegraded":"2023-10-08T00:56:40.160128+0000","last_fullsized":"2023-10-08T00:56:40.160128+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:15.309253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.134160+0000","last_change":"2023-10-08T00:56:39.133222+0000","last_active":"2023-10-08T00:56:39.134160+0000","last_peered":"2023-10-08T00:56:39.134160+0000","last_clean":"2023-10-08T00:56:39.134160+0000","last_became_active":"2023-10-08T00:56:39.132908+0000","last_became_peered":"2023-10-08T00:56:39.132908+0000","last_unstale":"2023-10-08T00:56:39.134160+0000","last_undegraded":"2023-10-08T00:56:39.134160+0000","last_fullsized":"2023-10-08T00:56:39.134160+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:14.302511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140308+0000","last_change":"2023-10-08T00:56:40.140308+0000","last_active":"2023-10-08T00:56:40.140308+0000","last_peered":"2023-10-08T00:56:40.140308+0000","last_clean":"2023-10-08T00:56:40.140308+0000","last_became_active":"2023-10-08T00:56:40.139963+0000","last_became_peered":"2023-10-08T00:56:40.139963+0000","last_unstale":"2023-10-08T00:56:40.140308+0000","last_undegraded":"2023-10-08T00:56:40.140308+0000","last_fullsized":"2023-10-08T00:56:40.140308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:20:27.686995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.126650+0000","last_change":"2023-10-08T00:56:39.121784+0000","last_active":"2023-10-08T00:56:39.126650+0000","last_peered":"2023-10-08T00:56:39.126650+0000","last_clean":"2023-10-08T00:56:39.126650+0000","last_became_active":"2023-10-08T00:56:39.121413+0000","last_became_peered":"2023-10-08T00:56:39.121413+0000","last_unstale":"2023-10-08T00:56:39.126650+0000","last_undegraded":"2023-10-08T00:56:39.126650+0000","last_fullsized":"2023-10-08T00:56:39.126650+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:13.696837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.127317+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T00:56:39.127317+0000","last_peered":"2023-10-08T00:56:39.127317+0000","last_clean":"2023-10-08T00:56:39.127317+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T00:56:39.127317+0000","last_undegraded":"2023-10-08T00:56:39.127317+0000","last_fullsized":"2023-10-08T00:56:39.127317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":718208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.85999999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66800000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80600000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.752},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.83699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":344064,"data_stored":128338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.621},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.51200000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58399999999999996}]},{"osd":2,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.44800000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":352256,"data_stored":129308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.32800000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48299999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.876},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.73699999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.84399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":929792,"data_stored":718118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.46500000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.505},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67700000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.94399999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.91200000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.008}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":339968,"data_stored":127820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45600000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.443}]},{"osd":4,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":352256,"data_stored":129308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.442},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.32000000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.48399999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.68300000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.67200000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.54200000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.55500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":344064,"data_stored":127854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60999999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45400000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.628},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59399999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.70199999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.73599999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.71499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":344064,"data_stored":128394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51900000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.499}]},{"osd":2,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69199999999999995}]},{"osd":4,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.625},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.746}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T00:56:44.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T00:56:44.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.687+0000 7fe96e7ff700 1 Processor -- start 2023-10-08T00:56:44.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.687+0000 7fe96e7ff700 1 -- start start 2023-10-08T00:56:44.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe96e7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9680b1420 0x7fe9680ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe96e7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9680cc300 con 0x7fe9680cbf30 2023-10-08T00:56:44.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe96e7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9680ad370 con 0x7fe9680ccf90 2023-10-08T00:56:44.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe96e7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe968156bf0 con 0x7fe9680b1420 2023-10-08T00:56:44.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe967fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe9680cbf30 0x7fe9680b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45074/0 (socket says 172.21.15.119:45074) 2023-10-08T00:56:44.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe967fff700 1 -- 172.21.15.119:0/3437288631 learned_addr learned my addr 172.21.15.119:0/3437288631 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe95f7fe700 1 --2- 172.21.15.119:0/3437288631 >> v2:172.21.15.119:3300/0 conn(0x7fe9680b1420 0x7fe9680ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe9677fe700 1 -- 172.21.15.119:0/3437288631 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250245915 0 0) 0x7fe9680cc300 con 0x7fe9680cbf30 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe95f7fe700 1 -- 172.21.15.119:0/3437288631 >> v1:172.21.15.184:6789/0 conn(0x7fe9680ccf90 legacy=0x7fe9680c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe95f7fe700 1 -- 172.21.15.119:0/3437288631 >> v1:172.21.15.119:6789/0 conn(0x7fe9680cbf30 legacy=0x7fe9680b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.689+0000 7fe95f7fe700 1 -- 172.21.15.119:0/3437288631 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9680ad650 con 0x7fe9680b1420 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe95f7fe700 1 --2- 172.21.15.119:0/3437288631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680b1420 0x7fe9680ae410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe950009e60 tx=0x7fe950009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451506afba810ab0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe9677fe700 1 -- 172.21.15.119:0/3437288631 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95000e070 con 0x7fe9680b1420 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe9677fe700 1 -- 172.21.15.119:0/3437288631 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95000bc30 con 0x7fe9680b1420 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe9677fe700 1 -- 172.21.15.119:0/3437288631 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950012430 con 0x7fe9680b1420 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe96e7ff700 1 -- 172.21.15.119:0/3437288631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680b1420 msgr2=0x7fe9680ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe96e7ff700 1 --2- 172.21.15.119:0/3437288631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680b1420 0x7fe9680ae410 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe950009e60 tx=0x7fe950009780 comp rx=0 tx=0).stop 2023-10-08T00:56:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe96e7ff700 1 -- 172.21.15.119:0/3437288631 shutdown_connections 2023-10-08T00:56:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe96e7ff700 1 --2- 172.21.15.119:0/3437288631 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680b1420 0x7fe9680ae410 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.690+0000 7fe96e7ff700 1 -- 172.21.15.119:0/3437288631 >> 172.21.15.119:0/3437288631 conn(0x7fe96806a320 msgr2=0x7fe9680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.691+0000 7fe96e7ff700 1 -- 172.21.15.119:0/3437288631 shutdown_connections 2023-10-08T00:56:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.691+0000 7fe96e7ff700 1 -- 172.21.15.119:0/3437288631 wait complete. 2023-10-08T00:56:44.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.692+0000 7fe96e7ff700 1 Processor -- start 2023-10-08T00:56:44.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.692+0000 7fe96e7ff700 1 -- start start 2023-10-08T00:56:44.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96e7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9680b1420 0x7fe9681698f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96e7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9680cbf30 0x7fe9680c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96e7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680ccf90 0x7fe9680c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96e7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680a6bf0 con 0x7fe9680ccf90 2023-10-08T00:56:44.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96e7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680aa4a0 con 0x7fe9680cbf30 2023-10-08T00:56:44.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96e7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9680c8340 con 0x7fe9680b1420 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96cd9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680ccf90 0x7fe9680c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96cd9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680ccf90 0x7fe9680c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53934/0 (socket says 172.21.15.119:53934) 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe96cd9c700 1 -- 172.21.15.119:0/461370588 learned_addr learned my addr 172.21.15.119:0/461370588 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe967fff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9680b1420 0x7fe9681698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.693+0000 7fe95f7fe700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9680cbf30 0x7fe9680c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe96cd9c700 1 -- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9680b1420 msgr2=0x7fe9681698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe96cd9c700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9680b1420 0x7fe9681698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe96cd9c700 1 -- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9680cbf30 msgr2=0x7fe9680c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe96cd9c700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9680cbf30 0x7fe9680c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe96cd9c700 1 -- 172.21.15.119:0/461370588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009a20 con 0x7fe9680ccf90 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe967fff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9680b1420 0x7fe9681698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe95f7fe700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9680cbf30 0x7fe9680c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe96cd9c700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680ccf90 0x7fe9680c7b20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe96000c4c0 tx=0x7fe960007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe964ff9700 1 -- 172.21.15.119:0/461370588 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960011590 con 0x7fe9680ccf90 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe964ff9700 1 -- 172.21.15.119:0/461370588 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe960011710 con 0x7fe9680ccf90 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9680a65a0 con 0x7fe9680ccf90 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe964ff9700 1 -- 172.21.15.119:0/461370588 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe960011ad0 con 0x7fe9680ccf90 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.694+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9680c8060 con 0x7fe9680ccf90 2023-10-08T00:56:44.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.695+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9680a6da0 con 0x7fe9680ccf90 2023-10-08T00:56:44.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.696+0000 7fe964ff9700 1 -- 172.21.15.119:0/461370588 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe96001b3b0 con 0x7fe9680ccf90 2023-10-08T00:56:44.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.697+0000 7fe964ff9700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe948066f30 0x7fe9480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:44.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.697+0000 7fe964ff9700 1 -- 172.21.15.119:0/461370588 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fe960083a30 con 0x7fe9680ccf90 2023-10-08T00:56:44.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.697+0000 7fe967fff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe948066f30 0x7fe9480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:44.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.698+0000 7fe967fff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe948066f30 0x7fe9480693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9580011e0 tx=0x7fe958009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:44.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.701+0000 7fe964ff9700 1 -- 172.21.15.119:0/461370588 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9680a6da0 con 0x7fe9680ccf90 2023-10-08T00:56:44.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.886+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe96804c3c0 con 0x7fe948066f30 2023-10-08T00:56:44.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.895+0000 7fe964ff9700 1 -- 172.21.15.119:0/461370588 <== mgr.4101 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289140 (secure 0 0 0) 0x7fe96804c3c0 con 0x7fe948066f30 2023-10-08T00:56:44.899 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe948066f30 msgr2=0x7fe9480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe948066f30 0x7fe9480693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9580011e0 tx=0x7fe958009040 comp rx=0 tx=0).stop 2023-10-08T00:56:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680ccf90 msgr2=0x7fe9680c7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:44.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680ccf90 0x7fe9680c7b20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe96000c4c0 tx=0x7fe960007f50 comp rx=0 tx=0).stop 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 shutdown_connections 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9680b1420 0x7fe9681698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe948066f30 0x7fe9480693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9680ccf90 0x7fe9680c7b20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 --2- 172.21.15.119:0/461370588 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9680cbf30 0x7fe9680c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 >> 172.21.15.119:0/461370588 conn(0x7fe96806a320 msgr2=0x7fe9680cb000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 shutdown_connections 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:44.899+0000 7fe96e7ff700 1 -- 172.21.15.119:0/461370588 wait complete. 2023-10-08T00:56:44.903 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T00:56:44.917 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":58,"stamp":"2023-10-08T00:56:43.868915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":91,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382152,"kb_used_data":3848,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592568,"statfs":{"total":773094113280,"available":770654789632,"internally_reserved":0,"allocated":3940352,"data_stored":2207348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.767352"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143111+0000","last_change":"2023-10-08T00:56:40.143111+0000","last_active":"2023-10-08T00:56:40.143111+0000","last_peered":"2023-10-08T00:56:40.143111+0000","last_clean":"2023-10-08T00:56:40.143111+0000","last_became_active":"2023-10-08T00:56:40.142814+0000","last_became_peered":"2023-10-08T00:56:40.142814+0000","last_unstale":"2023-10-08T00:56:40.143111+0000","last_undegraded":"2023-10-08T00:56:40.143111+0000","last_fullsized":"2023-10-08T00:56:40.143111+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:16.075601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.154161+0000","last_change":"2023-10-08T00:56:40.154161+0000","last_active":"2023-10-08T00:56:40.154161+0000","last_peered":"2023-10-08T00:56:40.154161+0000","last_clean":"2023-10-08T00:56:40.154161+0000","last_became_active":"2023-10-08T00:56:40.153948+0000","last_became_peered":"2023-10-08T00:56:40.153948+0000","last_unstale":"2023-10-08T00:56:40.154161+0000","last_undegraded":"2023-10-08T00:56:40.154161+0000","last_fullsized":"2023-10-08T00:56:40.154161+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:47.647727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166906+0000","last_change":"2023-10-08T00:56:40.166906+0000","last_active":"2023-10-08T00:56:40.166906+0000","last_peered":"2023-10-08T00:56:40.166906+0000","last_clean":"2023-10-08T00:56:40.166906+0000","last_became_active":"2023-10-08T00:56:40.166592+0000","last_became_peered":"2023-10-08T00:56:40.166592+0000","last_unstale":"2023-10-08T00:56:40.166906+0000","last_undegraded":"2023-10-08T00:56:40.166906+0000","last_fullsized":"2023-10-08T00:56:40.166906+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:49.816730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.159033+0000","last_change":"2023-10-08T00:56:40.159033+0000","last_active":"2023-10-08T00:56:40.159033+0000","last_peered":"2023-10-08T00:56:40.159033+0000","last_clean":"2023-10-08T00:56:40.159033+0000","last_became_active":"2023-10-08T00:56:40.158659+0000","last_became_peered":"2023-10-08T00:56:40.158659+0000","last_unstale":"2023-10-08T00:56:40.159033+0000","last_undegraded":"2023-10-08T00:56:40.159033+0000","last_fullsized":"2023-10-08T00:56:40.159033+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:01:49.816730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.144767+0000","last_change":"2023-10-08T00:56:40.144767+0000","last_active":"2023-10-08T00:56:40.144767+0000","last_peered":"2023-10-08T00:56:40.144767+0000","last_clean":"2023-10-08T00:56:40.144767+0000","last_became_active":"2023-10-08T00:56:40.144499+0000","last_became_peered":"2023-10-08T00:56:40.144499+0000","last_unstale":"2023-10-08T00:56:40.144767+0000","last_undegraded":"2023-10-08T00:56:40.144767+0000","last_fullsized":"2023-10-08T00:56:40.144767+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:41.258863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.147652+0000","last_change":"2023-10-08T00:56:40.147652+0000","last_active":"2023-10-08T00:56:40.147652+0000","last_peered":"2023-10-08T00:56:40.147652+0000","last_clean":"2023-10-08T00:56:40.147652+0000","last_became_active":"2023-10-08T00:56:40.147351+0000","last_became_peered":"2023-10-08T00:56:40.147351+0000","last_unstale":"2023-10-08T00:56:40.147652+0000","last_undegraded":"2023-10-08T00:56:40.147652+0000","last_fullsized":"2023-10-08T00:56:40.147652+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:03:03.376251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140536+0000","last_change":"2023-10-08T00:56:40.140536+0000","last_active":"2023-10-08T00:56:40.140536+0000","last_peered":"2023-10-08T00:56:40.140536+0000","last_clean":"2023-10-08T00:56:40.140536+0000","last_became_active":"2023-10-08T00:56:40.140211+0000","last_became_peered":"2023-10-08T00:56:40.140211+0000","last_unstale":"2023-10-08T00:56:40.140536+0000","last_undegraded":"2023-10-08T00:56:40.140536+0000","last_fullsized":"2023-10-08T00:56:40.140536+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:30.610379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140578+0000","last_change":"2023-10-08T00:56:40.140578+0000","last_active":"2023-10-08T00:56:40.140578+0000","last_peered":"2023-10-08T00:56:40.140578+0000","last_clean":"2023-10-08T00:56:40.140578+0000","last_became_active":"2023-10-08T00:56:40.140236+0000","last_became_peered":"2023-10-08T00:56:40.140236+0000","last_unstale":"2023-10-08T00:56:40.140578+0000","last_undegraded":"2023-10-08T00:56:40.140578+0000","last_fullsized":"2023-10-08T00:56:40.140578+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:33.219535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.163385+0000","last_change":"2023-10-08T00:56:40.163385+0000","last_active":"2023-10-08T00:56:40.163385+0000","last_peered":"2023-10-08T00:56:40.163385+0000","last_clean":"2023-10-08T00:56:40.163385+0000","last_became_active":"2023-10-08T00:56:40.163071+0000","last_became_peered":"2023-10-08T00:56:40.163071+0000","last_unstale":"2023-10-08T00:56:40.163385+0000","last_undegraded":"2023-10-08T00:56:40.163385+0000","last_fullsized":"2023-10-08T00:56:40.163385+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:41.313783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162314+0000","last_change":"2023-10-08T00:56:40.162314+0000","last_active":"2023-10-08T00:56:40.162314+0000","last_peered":"2023-10-08T00:56:40.162314+0000","last_clean":"2023-10-08T00:56:40.162314+0000","last_became_active":"2023-10-08T00:56:40.162047+0000","last_became_peered":"2023-10-08T00:56:40.162047+0000","last_unstale":"2023-10-08T00:56:40.162314+0000","last_undegraded":"2023-10-08T00:56:40.162314+0000","last_fullsized":"2023-10-08T00:56:40.162314+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:24.551023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.164708+0000","last_change":"2023-10-08T00:56:40.164708+0000","last_active":"2023-10-08T00:56:40.164708+0000","last_peered":"2023-10-08T00:56:40.164708+0000","last_clean":"2023-10-08T00:56:40.164708+0000","last_became_active":"2023-10-08T00:56:40.164375+0000","last_became_peered":"2023-10-08T00:56:40.164375+0000","last_unstale":"2023-10-08T00:56:40.164708+0000","last_undegraded":"2023-10-08T00:56:40.164708+0000","last_fullsized":"2023-10-08T00:56:40.164708+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:47.647727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.147892+0000","last_change":"2023-10-08T00:56:40.147892+0000","last_active":"2023-10-08T00:56:40.147892+0000","last_peered":"2023-10-08T00:56:40.147892+0000","last_clean":"2023-10-08T00:56:40.147892+0000","last_became_active":"2023-10-08T00:56:40.147630+0000","last_became_peered":"2023-10-08T00:56:40.147630+0000","last_unstale":"2023-10-08T00:56:40.147892+0000","last_undegraded":"2023-10-08T00:56:40.147892+0000","last_fullsized":"2023-10-08T00:56:40.147892+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:58.921683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.161236+0000","last_change":"2023-10-08T00:56:40.161236+0000","last_active":"2023-10-08T00:56:40.161236+0000","last_peered":"2023-10-08T00:56:40.161236+0000","last_clean":"2023-10-08T00:56:40.161236+0000","last_became_active":"2023-10-08T00:56:40.160964+0000","last_became_peered":"2023-10-08T00:56:40.160964+0000","last_unstale":"2023-10-08T00:56:40.161236+0000","last_undegraded":"2023-10-08T00:56:40.161236+0000","last_fullsized":"2023-10-08T00:56:40.161236+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:18.390267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.137588+0000","last_change":"2023-10-08T00:56:40.137588+0000","last_active":"2023-10-08T00:56:40.137588+0000","last_peered":"2023-10-08T00:56:40.137588+0000","last_clean":"2023-10-08T00:56:40.137588+0000","last_became_active":"2023-10-08T00:56:40.137260+0000","last_became_peered":"2023-10-08T00:56:40.137260+0000","last_unstale":"2023-10-08T00:56:40.137588+0000","last_undegraded":"2023-10-08T00:56:40.137588+0000","last_fullsized":"2023-10-08T00:56:40.137588+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:48.913792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.145486+0000","last_change":"2023-10-08T00:56:40.145486+0000","last_active":"2023-10-08T00:56:40.145486+0000","last_peered":"2023-10-08T00:56:40.145486+0000","last_clean":"2023-10-08T00:56:40.145486+0000","last_became_active":"2023-10-08T00:56:40.142311+0000","last_became_peered":"2023-10-08T00:56:40.142311+0000","last_unstale":"2023-10-08T00:56:40.145486+0000","last_undegraded":"2023-10-08T00:56:40.145486+0000","last_fullsized":"2023-10-08T00:56:40.145486+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:47.647727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.141179+0000","last_change":"2023-10-08T00:56:40.141179+0000","last_active":"2023-10-08T00:56:40.141179+0000","last_peered":"2023-10-08T00:56:40.141179+0000","last_clean":"2023-10-08T00:56:40.141179+0000","last_became_active":"2023-10-08T00:56:40.140910+0000","last_became_peered":"2023-10-08T00:56:40.140910+0000","last_unstale":"2023-10-08T00:56:40.141179+0000","last_undegraded":"2023-10-08T00:56:40.141179+0000","last_fullsized":"2023-10-08T00:56:40.141179+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:48.913792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143165+0000","last_change":"2023-10-08T00:56:40.143165+0000","last_active":"2023-10-08T00:56:40.143165+0000","last_peered":"2023-10-08T00:56:40.143165+0000","last_clean":"2023-10-08T00:56:40.143165+0000","last_became_active":"2023-10-08T00:56:40.142770+0000","last_became_peered":"2023-10-08T00:56:40.142770+0000","last_unstale":"2023-10-08T00:56:40.143165+0000","last_undegraded":"2023-10-08T00:56:40.143165+0000","last_fullsized":"2023-10-08T00:56:40.143165+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:27.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.165894+0000","last_change":"2023-10-08T00:56:40.165894+0000","last_active":"2023-10-08T00:56:40.165894+0000","last_peered":"2023-10-08T00:56:40.165894+0000","last_clean":"2023-10-08T00:56:40.165894+0000","last_became_active":"2023-10-08T00:56:40.165644+0000","last_became_peered":"2023-10-08T00:56:40.165644+0000","last_unstale":"2023-10-08T00:56:40.165894+0000","last_undegraded":"2023-10-08T00:56:40.165894+0000","last_fullsized":"2023-10-08T00:56:40.165894+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:50.990273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166648+0000","last_change":"2023-10-08T00:56:40.166648+0000","last_active":"2023-10-08T00:56:40.166648+0000","last_peered":"2023-10-08T00:56:40.166648+0000","last_clean":"2023-10-08T00:56:40.166648+0000","last_became_active":"2023-10-08T00:56:40.166365+0000","last_became_peered":"2023-10-08T00:56:40.166365+0000","last_unstale":"2023-10-08T00:56:40.166648+0000","last_undegraded":"2023-10-08T00:56:40.166648+0000","last_fullsized":"2023-10-08T00:56:40.166648+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:44:25.602012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.145126+0000","last_change":"2023-10-08T00:56:40.145126+0000","last_active":"2023-10-08T00:56:40.145126+0000","last_peered":"2023-10-08T00:56:40.145126+0000","last_clean":"2023-10-08T00:56:40.145126+0000","last_became_active":"2023-10-08T00:56:40.144768+0000","last_became_peered":"2023-10-08T00:56:40.144768+0000","last_unstale":"2023-10-08T00:56:40.145126+0000","last_undegraded":"2023-10-08T00:56:40.145126+0000","last_fullsized":"2023-10-08T00:56:40.145126+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:24.005622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.156444+0000","last_change":"2023-10-08T00:56:40.156444+0000","last_active":"2023-10-08T00:56:40.156444+0000","last_peered":"2023-10-08T00:56:40.156444+0000","last_clean":"2023-10-08T00:56:40.156444+0000","last_became_active":"2023-10-08T00:56:40.156202+0000","last_became_peered":"2023-10-08T00:56:40.156202+0000","last_unstale":"2023-10-08T00:56:40.156444+0000","last_undegraded":"2023-10-08T00:56:40.156444+0000","last_fullsized":"2023-10-08T00:56:40.156444+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:47.749054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.160143+0000","last_change":"2023-10-08T00:56:40.160143+0000","last_active":"2023-10-08T00:56:40.160143+0000","last_peered":"2023-10-08T00:56:40.160143+0000","last_clean":"2023-10-08T00:56:40.160143+0000","last_became_active":"2023-10-08T00:56:40.159873+0000","last_became_peered":"2023-10-08T00:56:40.159873+0000","last_unstale":"2023-10-08T00:56:40.160143+0000","last_undegraded":"2023-10-08T00:56:40.160143+0000","last_fullsized":"2023-10-08T00:56:40.160143+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:47.023393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140903+0000","last_change":"2023-10-08T00:56:40.140903+0000","last_active":"2023-10-08T00:56:40.140903+0000","last_peered":"2023-10-08T00:56:40.140903+0000","last_clean":"2023-10-08T00:56:40.140903+0000","last_became_active":"2023-10-08T00:56:40.140560+0000","last_became_peered":"2023-10-08T00:56:40.140560+0000","last_unstale":"2023-10-08T00:56:40.140903+0000","last_undegraded":"2023-10-08T00:56:40.140903+0000","last_fullsized":"2023-10-08T00:56:40.140903+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:59.756389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.139421+0000","last_change":"2023-10-08T00:56:40.139421+0000","last_active":"2023-10-08T00:56:40.139421+0000","last_peered":"2023-10-08T00:56:40.139421+0000","last_clean":"2023-10-08T00:56:40.139421+0000","last_became_active":"2023-10-08T00:56:40.139090+0000","last_became_peered":"2023-10-08T00:56:40.139090+0000","last_unstale":"2023-10-08T00:56:40.139421+0000","last_undegraded":"2023-10-08T00:56:40.139421+0000","last_fullsized":"2023-10-08T00:56:40.139421+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:33.219535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162377+0000","last_change":"2023-10-08T00:56:40.162377+0000","last_active":"2023-10-08T00:56:40.162377+0000","last_peered":"2023-10-08T00:56:40.162377+0000","last_clean":"2023-10-08T00:56:40.162377+0000","last_became_active":"2023-10-08T00:56:40.162116+0000","last_became_peered":"2023-10-08T00:56:40.162116+0000","last_unstale":"2023-10-08T00:56:40.162377+0000","last_undegraded":"2023-10-08T00:56:40.162377+0000","last_fullsized":"2023-10-08T00:56:40.162377+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:37.050655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143998+0000","last_change":"2023-10-08T00:56:40.143998+0000","last_active":"2023-10-08T00:56:40.143998+0000","last_peered":"2023-10-08T00:56:40.143998+0000","last_clean":"2023-10-08T00:56:40.143998+0000","last_became_active":"2023-10-08T00:56:40.143684+0000","last_became_peered":"2023-10-08T00:56:40.143684+0000","last_unstale":"2023-10-08T00:56:40.143998+0000","last_undegraded":"2023-10-08T00:56:40.143998+0000","last_fullsized":"2023-10-08T00:56:40.143998+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:02.299184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.163818+0000","last_change":"2023-10-08T00:56:40.163818+0000","last_active":"2023-10-08T00:56:40.163818+0000","last_peered":"2023-10-08T00:56:40.163818+0000","last_clean":"2023-10-08T00:56:40.163818+0000","last_became_active":"2023-10-08T00:56:40.163260+0000","last_became_peered":"2023-10-08T00:56:40.163260+0000","last_unstale":"2023-10-08T00:56:40.163818+0000","last_undegraded":"2023-10-08T00:56:40.163818+0000","last_fullsized":"2023-10-08T00:56:40.163818+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:32:10.599348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.153895+0000","last_change":"2023-10-08T00:56:40.153895+0000","last_active":"2023-10-08T00:56:40.153895+0000","last_peered":"2023-10-08T00:56:40.153895+0000","last_clean":"2023-10-08T00:56:40.153895+0000","last_became_active":"2023-10-08T00:56:40.153554+0000","last_became_peered":"2023-10-08T00:56:40.153554+0000","last_unstale":"2023-10-08T00:56:40.153895+0000","last_undegraded":"2023-10-08T00:56:40.153895+0000","last_fullsized":"2023-10-08T00:56:40.153895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:20:27.686995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.164491+0000","last_change":"2023-10-08T00:56:40.164491+0000","last_active":"2023-10-08T00:56:40.164491+0000","last_peered":"2023-10-08T00:56:40.164491+0000","last_clean":"2023-10-08T00:56:40.164491+0000","last_became_active":"2023-10-08T00:56:40.164231+0000","last_became_peered":"2023-10-08T00:56:40.164231+0000","last_unstale":"2023-10-08T00:56:40.164491+0000","last_undegraded":"2023-10-08T00:56:40.164491+0000","last_fullsized":"2023-10-08T00:56:40.164491+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:16.075601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.142758+0000","last_change":"2023-10-08T00:56:40.142758+0000","last_active":"2023-10-08T00:56:40.142758+0000","last_peered":"2023-10-08T00:56:40.142758+0000","last_clean":"2023-10-08T00:56:40.142758+0000","last_became_active":"2023-10-08T00:56:40.142449+0000","last_became_peered":"2023-10-08T00:56:40.142449+0000","last_unstale":"2023-10-08T00:56:40.142758+0000","last_undegraded":"2023-10-08T00:56:40.142758+0000","last_fullsized":"2023-10-08T00:56:40.142758+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:32:10.599348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.136840+0000","last_change":"2023-10-08T00:56:40.136840+0000","last_active":"2023-10-08T00:56:40.136840+0000","last_peered":"2023-10-08T00:56:40.136840+0000","last_clean":"2023-10-08T00:56:40.136840+0000","last_became_active":"2023-10-08T00:56:40.136494+0000","last_became_peered":"2023-10-08T00:56:40.136494+0000","last_unstale":"2023-10-08T00:56:40.136840+0000","last_undegraded":"2023-10-08T00:56:40.136840+0000","last_fullsized":"2023-10-08T00:56:40.136840+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:47.749054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.146968+0000","last_change":"2023-10-08T00:56:40.146968+0000","last_active":"2023-10-08T00:56:40.146968+0000","last_peered":"2023-10-08T00:56:40.146968+0000","last_clean":"2023-10-08T00:56:40.146968+0000","last_became_active":"2023-10-08T00:56:40.146718+0000","last_became_peered":"2023-10-08T00:56:40.146718+0000","last_unstale":"2023-10-08T00:56:40.146968+0000","last_undegraded":"2023-10-08T00:56:40.146968+0000","last_fullsized":"2023-10-08T00:56:40.146968+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:22:58.921683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140895+0000","last_change":"2023-10-08T00:56:40.140895+0000","last_active":"2023-10-08T00:56:40.140895+0000","last_peered":"2023-10-08T00:56:40.140895+0000","last_clean":"2023-10-08T00:56:40.140895+0000","last_became_active":"2023-10-08T00:56:40.140589+0000","last_became_peered":"2023-10-08T00:56:40.140589+0000","last_unstale":"2023-10-08T00:56:40.140895+0000","last_undegraded":"2023-10-08T00:56:40.140895+0000","last_fullsized":"2023-10-08T00:56:40.140895+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:24.800653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'4","reported_seq":13,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162623+0000","last_change":"2023-10-08T00:56:39.121635+0000","last_active":"2023-10-08T00:56:40.162623+0000","last_peered":"2023-10-08T00:56:40.162623+0000","last_clean":"2023-10-08T00:56:40.162623+0000","last_became_active":"2023-10-08T00:56:39.121330+0000","last_became_peered":"2023-10-08T00:56:39.121330+0000","last_unstale":"2023-10-08T00:56:40.162623+0000","last_undegraded":"2023-10-08T00:56:40.162623+0000","last_fullsized":"2023-10-08T00:56:40.162623+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:51:30.476911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143613+0000","last_change":"2023-10-08T00:56:40.143613+0000","last_active":"2023-10-08T00:56:40.143613+0000","last_peered":"2023-10-08T00:56:40.143613+0000","last_clean":"2023-10-08T00:56:40.143613+0000","last_became_active":"2023-10-08T00:56:40.143359+0000","last_became_peered":"2023-10-08T00:56:40.143359+0000","last_unstale":"2023-10-08T00:56:40.143613+0000","last_undegraded":"2023-10-08T00:56:40.143613+0000","last_fullsized":"2023-10-08T00:56:40.143613+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:15.309253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.129233+0000","last_change":"2023-10-08T00:56:39.134938+0000","last_active":"2023-10-08T00:56:40.129233+0000","last_peered":"2023-10-08T00:56:40.129233+0000","last_clean":"2023-10-08T00:56:40.129233+0000","last_became_active":"2023-10-08T00:56:39.134622+0000","last_became_peered":"2023-10-08T00:56:39.134622+0000","last_unstale":"2023-10-08T00:56:40.129233+0000","last_undegraded":"2023-10-08T00:56:40.129233+0000","last_fullsized":"2023-10-08T00:56:40.129233+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:22.998880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.160449+0000","last_change":"2023-10-08T00:56:40.160449+0000","last_active":"2023-10-08T00:56:40.160449+0000","last_peered":"2023-10-08T00:56:40.160449+0000","last_clean":"2023-10-08T00:56:40.160449+0000","last_became_active":"2023-10-08T00:56:40.160221+0000","last_became_peered":"2023-10-08T00:56:40.160221+0000","last_unstale":"2023-10-08T00:56:40.160449+0000","last_undegraded":"2023-10-08T00:56:40.160449+0000","last_fullsized":"2023-10-08T00:56:40.160449+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:53.246518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.167367+0000","last_change":"2023-10-08T00:56:39.126463+0000","last_active":"2023-10-08T00:56:40.167367+0000","last_peered":"2023-10-08T00:56:40.167367+0000","last_clean":"2023-10-08T00:56:40.167367+0000","last_became_active":"2023-10-08T00:56:39.126177+0000","last_became_peered":"2023-10-08T00:56:39.126177+0000","last_unstale":"2023-10-08T00:56:40.167367+0000","last_undegraded":"2023-10-08T00:56:40.167367+0000","last_fullsized":"2023-10-08T00:56:40.167367+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:46.016651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.163648+0000","last_change":"2023-10-08T00:56:40.163648+0000","last_active":"2023-10-08T00:56:40.163648+0000","last_peered":"2023-10-08T00:56:40.163648+0000","last_clean":"2023-10-08T00:56:40.163648+0000","last_became_active":"2023-10-08T00:56:40.162993+0000","last_became_peered":"2023-10-08T00:56:40.162993+0000","last_unstale":"2023-10-08T00:56:40.163648+0000","last_undegraded":"2023-10-08T00:56:40.163648+0000","last_fullsized":"2023-10-08T00:56:40.163648+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:30.610379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.164906+0000","last_change":"2023-10-08T00:56:39.118290+0000","last_active":"2023-10-08T00:56:40.164906+0000","last_peered":"2023-10-08T00:56:40.164906+0000","last_clean":"2023-10-08T00:56:40.164906+0000","last_became_active":"2023-10-08T00:56:39.117915+0000","last_became_peered":"2023-10-08T00:56:39.117915+0000","last_unstale":"2023-10-08T00:56:40.164906+0000","last_undegraded":"2023-10-08T00:56:40.164906+0000","last_fullsized":"2023-10-08T00:56:40.164906+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:24.142738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.147736+0000","last_change":"2023-10-08T00:56:40.147736+0000","last_active":"2023-10-08T00:56:40.147736+0000","last_peered":"2023-10-08T00:56:40.147736+0000","last_clean":"2023-10-08T00:56:40.147736+0000","last_became_active":"2023-10-08T00:56:40.147400+0000","last_became_peered":"2023-10-08T00:56:40.147400+0000","last_unstale":"2023-10-08T00:56:40.147736+0000","last_undegraded":"2023-10-08T00:56:40.147736+0000","last_fullsized":"2023-10-08T00:56:40.147736+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:37.050655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.136561+0000","last_change":"2023-10-08T00:56:39.128607+0000","last_active":"2023-10-08T00:56:40.136561+0000","last_peered":"2023-10-08T00:56:40.136561+0000","last_clean":"2023-10-08T00:56:40.136561+0000","last_became_active":"2023-10-08T00:56:39.127151+0000","last_became_peered":"2023-10-08T00:56:39.127151+0000","last_unstale":"2023-10-08T00:56:40.136561+0000","last_undegraded":"2023-10-08T00:56:40.136561+0000","last_fullsized":"2023-10-08T00:56:40.136561+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:15:15.964678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.155045+0000","last_change":"2023-10-08T00:56:40.155045+0000","last_active":"2023-10-08T00:56:40.155045+0000","last_peered":"2023-10-08T00:56:40.155045+0000","last_clean":"2023-10-08T00:56:40.155045+0000","last_became_active":"2023-10-08T00:56:40.154768+0000","last_became_peered":"2023-10-08T00:56:40.154768+0000","last_unstale":"2023-10-08T00:56:40.155045+0000","last_undegraded":"2023-10-08T00:56:40.155045+0000","last_fullsized":"2023-10-08T00:56:40.155045+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.135308+0000","last_change":"2023-10-08T00:56:39.128735+0000","last_active":"2023-10-08T00:56:40.135308+0000","last_peered":"2023-10-08T00:56:40.135308+0000","last_clean":"2023-10-08T00:56:40.135308+0000","last_became_active":"2023-10-08T00:56:39.128392+0000","last_became_peered":"2023-10-08T00:56:39.128392+0000","last_unstale":"2023-10-08T00:56:40.135308+0000","last_undegraded":"2023-10-08T00:56:40.135308+0000","last_fullsized":"2023-10-08T00:56:40.135308+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:05.959100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140482+0000","last_change":"2023-10-08T00:56:40.140482+0000","last_active":"2023-10-08T00:56:40.140482+0000","last_peered":"2023-10-08T00:56:40.140482+0000","last_clean":"2023-10-08T00:56:40.140482+0000","last_became_active":"2023-10-08T00:56:40.138728+0000","last_became_peered":"2023-10-08T00:56:40.138728+0000","last_unstale":"2023-10-08T00:56:40.140482+0000","last_undegraded":"2023-10-08T00:56:40.140482+0000","last_fullsized":"2023-10-08T00:56:40.140482+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:59:08.290249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.127626+0000","last_change":"2023-10-08T00:56:39.126328+0000","last_active":"2023-10-08T00:56:39.127626+0000","last_peered":"2023-10-08T00:56:39.127626+0000","last_clean":"2023-10-08T00:56:39.127626+0000","last_became_active":"2023-10-08T00:56:39.126020+0000","last_became_peered":"2023-10-08T00:56:39.126020+0000","last_unstale":"2023-10-08T00:56:39.127626+0000","last_undegraded":"2023-10-08T00:56:39.127626+0000","last_fullsized":"2023-10-08T00:56:39.127626+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:46.742312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.155211+0000","last_change":"2023-10-08T00:56:40.155211+0000","last_active":"2023-10-08T00:56:40.155211+0000","last_peered":"2023-10-08T00:56:40.155211+0000","last_clean":"2023-10-08T00:56:40.155211+0000","last_became_active":"2023-10-08T00:56:40.154923+0000","last_became_peered":"2023-10-08T00:56:40.154923+0000","last_unstale":"2023-10-08T00:56:40.155211+0000","last_undegraded":"2023-10-08T00:56:40.155211+0000","last_fullsized":"2023-10-08T00:56:40.155211+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:59.609984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.125132+0000","last_change":"2023-10-08T00:56:39.120650+0000","last_active":"2023-10-08T00:56:39.125132+0000","last_peered":"2023-10-08T00:56:39.125132+0000","last_clean":"2023-10-08T00:56:39.125132+0000","last_became_active":"2023-10-08T00:56:39.120364+0000","last_became_peered":"2023-10-08T00:56:39.120364+0000","last_unstale":"2023-10-08T00:56:39.125132+0000","last_undegraded":"2023-10-08T00:56:39.125132+0000","last_fullsized":"2023-10-08T00:56:39.125132+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:58:00.290088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.151821+0000","last_change":"2023-10-08T00:56:40.151821+0000","last_active":"2023-10-08T00:56:40.151821+0000","last_peered":"2023-10-08T00:56:40.151821+0000","last_clean":"2023-10-08T00:56:40.151821+0000","last_became_active":"2023-10-08T00:56:40.151557+0000","last_became_peered":"2023-10-08T00:56:40.151557+0000","last_unstale":"2023-10-08T00:56:40.151821+0000","last_undegraded":"2023-10-08T00:56:40.151821+0000","last_fullsized":"2023-10-08T00:56:40.151821+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:30.610379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.134134+0000","last_change":"2023-10-08T00:56:39.133218+0000","last_active":"2023-10-08T00:56:39.134134+0000","last_peered":"2023-10-08T00:56:39.134134+0000","last_clean":"2023-10-08T00:56:39.134134+0000","last_became_active":"2023-10-08T00:56:39.132889+0000","last_became_peered":"2023-10-08T00:56:39.132889+0000","last_unstale":"2023-10-08T00:56:39.134134+0000","last_undegraded":"2023-10-08T00:56:39.134134+0000","last_fullsized":"2023-10-08T00:56:39.134134+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:15.068859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.164743+0000","last_change":"2023-10-08T00:56:40.164743+0000","last_active":"2023-10-08T00:56:40.164743+0000","last_peered":"2023-10-08T00:56:40.164743+0000","last_clean":"2023-10-08T00:56:40.164743+0000","last_became_active":"2023-10-08T00:56:40.164154+0000","last_became_peered":"2023-10-08T00:56:40.164154+0000","last_unstale":"2023-10-08T00:56:40.164743+0000","last_undegraded":"2023-10-08T00:56:40.164743+0000","last_fullsized":"2023-10-08T00:56:40.164743+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:26:00.141477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.123050+0000","last_change":"2023-10-08T00:56:39.119761+0000","last_active":"2023-10-08T00:56:39.123050+0000","last_peered":"2023-10-08T00:56:39.123050+0000","last_clean":"2023-10-08T00:56:39.123050+0000","last_became_active":"2023-10-08T00:56:39.119464+0000","last_became_peered":"2023-10-08T00:56:39.119464+0000","last_unstale":"2023-10-08T00:56:39.123050+0000","last_undegraded":"2023-10-08T00:56:39.123050+0000","last_fullsized":"2023-10-08T00:56:39.123050+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:17.383525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.143570+0000","last_change":"2023-10-08T00:56:40.143570+0000","last_active":"2023-10-08T00:56:40.143570+0000","last_peered":"2023-10-08T00:56:40.143570+0000","last_clean":"2023-10-08T00:56:40.143570+0000","last_became_active":"2023-10-08T00:56:40.143276+0000","last_became_peered":"2023-10-08T00:56:40.143276+0000","last_unstale":"2023-10-08T00:56:40.143570+0000","last_undegraded":"2023-10-08T00:56:40.143570+0000","last_fullsized":"2023-10-08T00:56:40.143570+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:10.506479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"30'3","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.153174+0000","last_change":"2023-10-08T00:56:39.133022+0000","last_active":"2023-10-08T00:56:40.153174+0000","last_peered":"2023-10-08T00:56:40.153174+0000","last_clean":"2023-10-08T00:56:40.153174+0000","last_became_active":"2023-10-08T00:56:39.132745+0000","last_became_peered":"2023-10-08T00:56:39.132745+0000","last_unstale":"2023-10-08T00:56:40.153174+0000","last_undegraded":"2023-10-08T00:56:40.153174+0000","last_fullsized":"2023-10-08T00:56:40.153174+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:38.708733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166944+0000","last_change":"2023-10-08T00:56:40.166944+0000","last_active":"2023-10-08T00:56:40.166944+0000","last_peered":"2023-10-08T00:56:40.166944+0000","last_clean":"2023-10-08T00:56:40.166944+0000","last_became_active":"2023-10-08T00:56:40.166570+0000","last_became_peered":"2023-10-08T00:56:40.166570+0000","last_unstale":"2023-10-08T00:56:40.166944+0000","last_undegraded":"2023-10-08T00:56:40.166944+0000","last_fullsized":"2023-10-08T00:56:40.166944+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:32:10.599348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"30'6","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.170014+0000","last_change":"2023-10-08T00:56:39.123529+0000","last_active":"2023-10-08T00:56:40.170014+0000","last_peered":"2023-10-08T00:56:40.170014+0000","last_clean":"2023-10-08T00:56:40.170014+0000","last_became_active":"2023-10-08T00:56:39.123280+0000","last_became_peered":"2023-10-08T00:56:39.123280+0000","last_unstale":"2023-10-08T00:56:40.170014+0000","last_undegraded":"2023-10-08T00:56:40.170014+0000","last_fullsized":"2023-10-08T00:56:40.170014+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:40.307041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.158852+0000","last_change":"2023-10-08T00:56:40.158852+0000","last_active":"2023-10-08T00:56:40.158852+0000","last_peered":"2023-10-08T00:56:40.158852+0000","last_clean":"2023-10-08T00:56:40.158852+0000","last_became_active":"2023-10-08T00:56:40.158516+0000","last_became_peered":"2023-10-08T00:56:40.158516+0000","last_unstale":"2023-10-08T00:56:40.158852+0000","last_undegraded":"2023-10-08T00:56:40.158852+0000","last_fullsized":"2023-10-08T00:56:40.158852+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:59.756389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162546+0000","last_change":"2023-10-08T00:56:39.129980+0000","last_active":"2023-10-08T00:56:40.162546+0000","last_peered":"2023-10-08T00:56:40.162546+0000","last_clean":"2023-10-08T00:56:40.162546+0000","last_became_active":"2023-10-08T00:56:39.126062+0000","last_became_peered":"2023-10-08T00:56:39.126062+0000","last_unstale":"2023-10-08T00:56:40.162546+0000","last_undegraded":"2023-10-08T00:56:40.162546+0000","last_fullsized":"2023-10-08T00:56:40.162546+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:02:59.321210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.154251+0000","last_change":"2023-10-08T00:56:40.154251+0000","last_active":"2023-10-08T00:56:40.154251+0000","last_peered":"2023-10-08T00:56:40.154251+0000","last_clean":"2023-10-08T00:56:40.154251+0000","last_became_active":"2023-10-08T00:56:40.153878+0000","last_became_peered":"2023-10-08T00:56:40.153878+0000","last_unstale":"2023-10-08T00:56:40.154251+0000","last_undegraded":"2023-10-08T00:56:40.154251+0000","last_fullsized":"2023-10-08T00:56:40.154251+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:33.219535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.122173+0000","last_change":"2023-10-08T00:56:39.116641+0000","last_active":"2023-10-08T00:56:39.122173+0000","last_peered":"2023-10-08T00:56:39.122173+0000","last_clean":"2023-10-08T00:56:39.122173+0000","last_became_active":"2023-10-08T00:56:39.115728+0000","last_became_peered":"2023-10-08T00:56:39.115728+0000","last_unstale":"2023-10-08T00:56:39.122173+0000","last_undegraded":"2023-10-08T00:56:39.122173+0000","last_fullsized":"2023-10-08T00:56:39.122173+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:13.696837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.158206+0000","last_change":"2023-10-08T00:56:40.158206+0000","last_active":"2023-10-08T00:56:40.158206+0000","last_peered":"2023-10-08T00:56:40.158206+0000","last_clean":"2023-10-08T00:56:40.158206+0000","last_became_active":"2023-10-08T00:56:40.157885+0000","last_became_peered":"2023-10-08T00:56:40.157885+0000","last_unstale":"2023-10-08T00:56:40.158206+0000","last_undegraded":"2023-10-08T00:56:40.158206+0000","last_fullsized":"2023-10-08T00:56:40.158206+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:06.531519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.133588+0000","last_change":"2023-10-08T00:56:39.132658+0000","last_active":"2023-10-08T00:56:39.133588+0000","last_peered":"2023-10-08T00:56:39.133588+0000","last_clean":"2023-10-08T00:56:39.133588+0000","last_became_active":"2023-10-08T00:56:39.132251+0000","last_became_peered":"2023-10-08T00:56:39.132251+0000","last_unstale":"2023-10-08T00:56:39.133588+0000","last_undegraded":"2023-10-08T00:56:39.133588+0000","last_fullsized":"2023-10-08T00:56:39.133588+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:30:23.793911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.151078+0000","last_change":"2023-10-08T00:56:40.151078+0000","last_active":"2023-10-08T00:56:40.151078+0000","last_peered":"2023-10-08T00:56:40.151078+0000","last_clean":"2023-10-08T00:56:40.151078+0000","last_became_active":"2023-10-08T00:56:40.150782+0000","last_became_peered":"2023-10-08T00:56:40.150782+0000","last_unstale":"2023-10-08T00:56:40.151078+0000","last_undegraded":"2023-10-08T00:56:40.151078+0000","last_fullsized":"2023-10-08T00:56:40.151078+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:47.023393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"30'8","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.169913+0000","last_change":"2023-10-08T00:56:39.119286+0000","last_active":"2023-10-08T00:56:40.169913+0000","last_peered":"2023-10-08T00:56:40.169913+0000","last_clean":"2023-10-08T00:56:40.169913+0000","last_became_active":"2023-10-08T00:56:39.118993+0000","last_became_peered":"2023-10-08T00:56:39.118993+0000","last_unstale":"2023-10-08T00:56:40.169913+0000","last_undegraded":"2023-10-08T00:56:40.169913+0000","last_fullsized":"2023-10-08T00:56:40.169913+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:52.239776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.147586+0000","last_change":"2023-10-08T00:56:40.147586+0000","last_active":"2023-10-08T00:56:40.147586+0000","last_peered":"2023-10-08T00:56:40.147586+0000","last_clean":"2023-10-08T00:56:40.147586+0000","last_became_active":"2023-10-08T00:56:40.147259+0000","last_became_peered":"2023-10-08T00:56:40.147259+0000","last_unstale":"2023-10-08T00:56:40.147586+0000","last_undegraded":"2023-10-08T00:56:40.147586+0000","last_fullsized":"2023-10-08T00:56:40.147586+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:39.715475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.145986+0000","last_change":"2023-10-08T00:56:39.127361+0000","last_active":"2023-10-08T00:56:40.145986+0000","last_peered":"2023-10-08T00:56:40.145986+0000","last_clean":"2023-10-08T00:56:40.145986+0000","last_became_active":"2023-10-08T00:56:39.127106+0000","last_became_peered":"2023-10-08T00:56:39.127106+0000","last_unstale":"2023-10-08T00:56:40.145986+0000","last_undegraded":"2023-10-08T00:56:40.145986+0000","last_fullsized":"2023-10-08T00:56:40.145986+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:40.252121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.151703+0000","last_change":"2023-10-08T00:56:40.151703+0000","last_active":"2023-10-08T00:56:40.151703+0000","last_peered":"2023-10-08T00:56:40.151703+0000","last_clean":"2023-10-08T00:56:40.151703+0000","last_became_active":"2023-10-08T00:56:40.151415+0000","last_became_peered":"2023-10-08T00:56:40.151415+0000","last_unstale":"2023-10-08T00:56:40.151703+0000","last_undegraded":"2023-10-08T00:56:40.151703+0000","last_fullsized":"2023-10-08T00:56:40.151703+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:48.913792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.162813+0000","last_change":"2023-10-08T00:56:39.125489+0000","last_active":"2023-10-08T00:56:40.162813+0000","last_peered":"2023-10-08T00:56:40.162813+0000","last_clean":"2023-10-08T00:56:40.162813+0000","last_became_active":"2023-10-08T00:56:39.125245+0000","last_became_peered":"2023-10-08T00:56:39.125245+0000","last_unstale":"2023-10-08T00:56:40.162813+0000","last_undegraded":"2023-10-08T00:56:40.162813+0000","last_fullsized":"2023-10-08T00:56:40.162813+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:52:44.534690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.159049+0000","last_change":"2023-10-08T00:56:40.159049+0000","last_active":"2023-10-08T00:56:40.159049+0000","last_peered":"2023-10-08T00:56:40.159049+0000","last_clean":"2023-10-08T00:56:40.159049+0000","last_became_active":"2023-10-08T00:56:40.158632+0000","last_became_peered":"2023-10-08T00:56:40.158632+0000","last_unstale":"2023-10-08T00:56:40.159049+0000","last_undegraded":"2023-10-08T00:56:40.159049+0000","last_fullsized":"2023-10-08T00:56:40.159049+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:27.621075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"30'1","reported_seq":10,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.141025+0000","last_change":"2023-10-08T00:56:39.129189+0000","last_active":"2023-10-08T00:56:40.141025+0000","last_peered":"2023-10-08T00:56:40.141025+0000","last_clean":"2023-10-08T00:56:40.141025+0000","last_became_active":"2023-10-08T00:56:39.128873+0000","last_became_peered":"2023-10-08T00:56:39.128873+0000","last_unstale":"2023-10-08T00:56:40.141025+0000","last_undegraded":"2023-10-08T00:56:40.141025+0000","last_fullsized":"2023-10-08T00:56:40.141025+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:32:26.820438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.142116+0000","last_change":"2023-10-08T00:56:40.142116+0000","last_active":"2023-10-08T00:56:40.142116+0000","last_peered":"2023-10-08T00:56:40.142116+0000","last_clean":"2023-10-08T00:56:40.142116+0000","last_became_active":"2023-10-08T00:56:40.141828+0000","last_became_peered":"2023-10-08T00:56:40.141828+0000","last_unstale":"2023-10-08T00:56:40.142116+0000","last_undegraded":"2023-10-08T00:56:40.142116+0000","last_fullsized":"2023-10-08T00:56:40.142116+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:48:39.715475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.134036+0000","last_change":"2023-10-08T00:56:39.128608+0000","last_active":"2023-10-08T00:56:39.134036+0000","last_peered":"2023-10-08T00:56:39.134036+0000","last_clean":"2023-10-08T00:56:39.134036+0000","last_became_active":"2023-10-08T00:56:39.127152+0000","last_became_peered":"2023-10-08T00:56:39.127152+0000","last_unstale":"2023-10-08T00:56:39.134036+0000","last_undegraded":"2023-10-08T00:56:39.134036+0000","last_fullsized":"2023-10-08T00:56:39.134036+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:07.575298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.146094+0000","last_change":"2023-10-08T00:56:40.146094+0000","last_active":"2023-10-08T00:56:40.146094+0000","last_peered":"2023-10-08T00:56:40.146094+0000","last_clean":"2023-10-08T00:56:40.146094+0000","last_became_active":"2023-10-08T00:56:40.145805+0000","last_became_peered":"2023-10-08T00:56:40.145805+0000","last_unstale":"2023-10-08T00:56:40.146094+0000","last_undegraded":"2023-10-08T00:56:40.146094+0000","last_fullsized":"2023-10-08T00:56:40.146094+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:14.703579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.165193+0000","last_change":"2023-10-08T00:56:39.124087+0000","last_active":"2023-10-08T00:56:40.165193+0000","last_peered":"2023-10-08T00:56:40.165193+0000","last_clean":"2023-10-08T00:56:40.165193+0000","last_became_active":"2023-10-08T00:56:39.123837+0000","last_became_peered":"2023-10-08T00:56:39.123837+0000","last_unstale":"2023-10-08T00:56:40.165193+0000","last_undegraded":"2023-10-08T00:56:40.165193+0000","last_fullsized":"2023-10-08T00:56:40.165193+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:13.696837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.131072+0000","last_change":"2023-10-08T00:56:40.131072+0000","last_active":"2023-10-08T00:56:40.131072+0000","last_peered":"2023-10-08T00:56:40.131072+0000","last_clean":"2023-10-08T00:56:40.131072+0000","last_became_active":"2023-10-08T00:56:40.130777+0000","last_became_peered":"2023-10-08T00:56:40.130777+0000","last_unstale":"2023-10-08T00:56:40.131072+0000","last_undegraded":"2023-10-08T00:56:40.131072+0000","last_fullsized":"2023-10-08T00:56:40.131072+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:59:08.290249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.120044+0000","last_change":"2023-10-08T00:56:39.117963+0000","last_active":"2023-10-08T00:56:39.120044+0000","last_peered":"2023-10-08T00:56:39.120044+0000","last_clean":"2023-10-08T00:56:39.120044+0000","last_became_active":"2023-10-08T00:56:39.117518+0000","last_became_peered":"2023-10-08T00:56:39.117518+0000","last_unstale":"2023-10-08T00:56:39.120044+0000","last_undegraded":"2023-10-08T00:56:39.120044+0000","last_fullsized":"2023-10-08T00:56:39.120044+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:32:52.556872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.131110+0000","last_change":"2023-10-08T00:56:40.131110+0000","last_active":"2023-10-08T00:56:40.131110+0000","last_peered":"2023-10-08T00:56:40.131110+0000","last_clean":"2023-10-08T00:56:40.131110+0000","last_became_active":"2023-10-08T00:56:40.130817+0000","last_became_peered":"2023-10-08T00:56:40.130817+0000","last_unstale":"2023-10-08T00:56:40.131110+0000","last_undegraded":"2023-10-08T00:56:40.131110+0000","last_fullsized":"2023-10-08T00:56:40.131110+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:51:31.483653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.137557+0000","last_change":"2023-10-08T00:56:39.131192+0000","last_active":"2023-10-08T00:56:39.137557+0000","last_peered":"2023-10-08T00:56:39.137557+0000","last_clean":"2023-10-08T00:56:39.137557+0000","last_became_active":"2023-10-08T00:56:39.130881+0000","last_became_peered":"2023-10-08T00:56:39.130881+0000","last_unstale":"2023-10-08T00:56:39.137557+0000","last_undegraded":"2023-10-08T00:56:39.137557+0000","last_fullsized":"2023-10-08T00:56:39.137557+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:24.142738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140733+0000","last_change":"2023-10-08T00:56:40.140733+0000","last_active":"2023-10-08T00:56:40.140733+0000","last_peered":"2023-10-08T00:56:40.140733+0000","last_clean":"2023-10-08T00:56:40.140733+0000","last_became_active":"2023-10-08T00:56:40.139984+0000","last_became_peered":"2023-10-08T00:56:40.139984+0000","last_unstale":"2023-10-08T00:56:40.140733+0000","last_undegraded":"2023-10-08T00:56:40.140733+0000","last_fullsized":"2023-10-08T00:56:40.140733+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:37:24.551023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.137872+0000","last_change":"2023-10-08T00:56:39.134838+0000","last_active":"2023-10-08T00:56:39.137872+0000","last_peered":"2023-10-08T00:56:39.137872+0000","last_clean":"2023-10-08T00:56:39.137872+0000","last_became_active":"2023-10-08T00:56:39.134523+0000","last_became_peered":"2023-10-08T00:56:39.134523+0000","last_unstale":"2023-10-08T00:56:39.137872+0000","last_undegraded":"2023-10-08T00:56:39.137872+0000","last_fullsized":"2023-10-08T00:56:39.137872+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:59:07.283507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.145421+0000","last_change":"2023-10-08T00:56:40.145421+0000","last_active":"2023-10-08T00:56:40.145421+0000","last_peered":"2023-10-08T00:56:40.145421+0000","last_clean":"2023-10-08T00:56:40.145421+0000","last_became_active":"2023-10-08T00:56:40.144856+0000","last_became_peered":"2023-10-08T00:56:40.144856+0000","last_unstale":"2023-10-08T00:56:40.145421+0000","last_undegraded":"2023-10-08T00:56:40.145421+0000","last_fullsized":"2023-10-08T00:56:40.145421+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:26.715198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"30'2","reported_seq":11,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166495+0000","last_change":"2023-10-08T00:56:39.119709+0000","last_active":"2023-10-08T00:56:40.166495+0000","last_peered":"2023-10-08T00:56:40.166495+0000","last_clean":"2023-10-08T00:56:40.166495+0000","last_became_active":"2023-10-08T00:56:39.119470+0000","last_became_peered":"2023-10-08T00:56:39.119470+0000","last_unstale":"2023-10-08T00:56:40.166495+0000","last_undegraded":"2023-10-08T00:56:40.166495+0000","last_fullsized":"2023-10-08T00:56:40.166495+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:22.998880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.166939+0000","last_change":"2023-10-08T00:56:40.166939+0000","last_active":"2023-10-08T00:56:40.166939+0000","last_peered":"2023-10-08T00:56:40.166939+0000","last_clean":"2023-10-08T00:56:40.166939+0000","last_became_active":"2023-10-08T00:56:40.166616+0000","last_became_peered":"2023-10-08T00:56:40.166616+0000","last_unstale":"2023-10-08T00:56:40.166939+0000","last_undegraded":"2023-10-08T00:56:40.166939+0000","last_fullsized":"2023-10-08T00:56:40.166939+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:37.050655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.125246+0000","last_change":"2023-10-08T00:56:39.124211+0000","last_active":"2023-10-08T00:56:39.125246+0000","last_peered":"2023-10-08T00:56:39.125246+0000","last_clean":"2023-10-08T00:56:39.125246+0000","last_became_active":"2023-10-08T00:56:39.123936+0000","last_became_peered":"2023-10-08T00:56:39.123936+0000","last_unstale":"2023-10-08T00:56:39.125246+0000","last_undegraded":"2023-10-08T00:56:39.125246+0000","last_fullsized":"2023-10-08T00:56:39.125246+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:40.252121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.165576+0000","last_change":"2023-10-08T00:56:40.165576+0000","last_active":"2023-10-08T00:56:40.165576+0000","last_peered":"2023-10-08T00:56:40.165576+0000","last_clean":"2023-10-08T00:56:40.165576+0000","last_became_active":"2023-10-08T00:56:40.165293+0000","last_became_peered":"2023-10-08T00:56:40.165293+0000","last_unstale":"2023-10-08T00:56:40.165576+0000","last_undegraded":"2023-10-08T00:56:40.165576+0000","last_fullsized":"2023-10-08T00:56:40.165576+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:17:16.075601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.128304+0000","last_change":"2023-10-08T00:56:39.125519+0000","last_active":"2023-10-08T00:56:39.128304+0000","last_peered":"2023-10-08T00:56:39.128304+0000","last_clean":"2023-10-08T00:56:39.128304+0000","last_became_active":"2023-10-08T00:56:39.125300+0000","last_became_peered":"2023-10-08T00:56:39.125300+0000","last_unstale":"2023-10-08T00:56:39.128304+0000","last_undegraded":"2023-10-08T00:56:39.128304+0000","last_fullsized":"2023-10-08T00:56:39.128304+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:17.383525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.153773+0000","last_change":"2023-10-08T00:56:40.153773+0000","last_active":"2023-10-08T00:56:40.153773+0000","last_peered":"2023-10-08T00:56:40.153773+0000","last_clean":"2023-10-08T00:56:40.153773+0000","last_became_active":"2023-10-08T00:56:40.153431+0000","last_became_peered":"2023-10-08T00:56:40.153431+0000","last_unstale":"2023-10-08T00:56:40.153773+0000","last_undegraded":"2023-10-08T00:56:40.153773+0000","last_fullsized":"2023-10-08T00:56:40.153773+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:53.246518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.134949+0000","last_change":"2023-10-08T00:56:39.131851+0000","last_active":"2023-10-08T00:56:39.134949+0000","last_peered":"2023-10-08T00:56:39.134949+0000","last_clean":"2023-10-08T00:56:39.134949+0000","last_became_active":"2023-10-08T00:56:39.131574+0000","last_became_peered":"2023-10-08T00:56:39.131574+0000","last_unstale":"2023-10-08T00:56:39.134949+0000","last_undegraded":"2023-10-08T00:56:39.134949+0000","last_fullsized":"2023-10-08T00:56:39.134949+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:59:07.283507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.133915+0000","last_change":"2023-10-08T00:56:40.133915+0000","last_active":"2023-10-08T00:56:40.133915+0000","last_peered":"2023-10-08T00:56:40.133915+0000","last_clean":"2023-10-08T00:56:40.133915+0000","last_became_active":"2023-10-08T00:56:40.133648+0000","last_became_peered":"2023-10-08T00:56:40.133648+0000","last_unstale":"2023-10-08T00:56:40.133915+0000","last_undegraded":"2023-10-08T00:56:40.133915+0000","last_fullsized":"2023-10-08T00:56:40.133915+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:16:47.749054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.124280+0000","last_change":"2023-10-08T00:56:39.121401+0000","last_active":"2023-10-08T00:56:39.124280+0000","last_peered":"2023-10-08T00:56:39.124280+0000","last_clean":"2023-10-08T00:56:39.124280+0000","last_became_active":"2023-10-08T00:56:39.121104+0000","last_became_peered":"2023-10-08T00:56:39.121104+0000","last_unstale":"2023-10-08T00:56:39.124280+0000","last_undegraded":"2023-10-08T00:56:39.124280+0000","last_fullsized":"2023-10-08T00:56:39.124280+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:50:25.708456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.160230+0000","last_change":"2023-10-08T00:56:40.160230+0000","last_active":"2023-10-08T00:56:40.160230+0000","last_peered":"2023-10-08T00:56:40.160230+0000","last_clean":"2023-10-08T00:56:40.160230+0000","last_became_active":"2023-10-08T00:56:40.159987+0000","last_became_peered":"2023-10-08T00:56:40.159987+0000","last_unstale":"2023-10-08T00:56:40.160230+0000","last_undegraded":"2023-10-08T00:56:40.160230+0000","last_fullsized":"2023-10-08T00:56:40.160230+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:18:24.539053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.137571+0000","last_change":"2023-10-08T00:56:39.131798+0000","last_active":"2023-10-08T00:56:39.137571+0000","last_peered":"2023-10-08T00:56:39.137571+0000","last_clean":"2023-10-08T00:56:39.137571+0000","last_became_active":"2023-10-08T00:56:39.131483+0000","last_became_peered":"2023-10-08T00:56:39.131483+0000","last_unstale":"2023-10-08T00:56:39.137571+0000","last_undegraded":"2023-10-08T00:56:39.137571+0000","last_fullsized":"2023-10-08T00:56:39.137571+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:26.416748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.160128+0000","last_change":"2023-10-08T00:56:40.160128+0000","last_active":"2023-10-08T00:56:40.160128+0000","last_peered":"2023-10-08T00:56:40.160128+0000","last_clean":"2023-10-08T00:56:40.160128+0000","last_became_active":"2023-10-08T00:56:40.159843+0000","last_became_peered":"2023-10-08T00:56:40.159843+0000","last_unstale":"2023-10-08T00:56:40.160128+0000","last_undegraded":"2023-10-08T00:56:40.160128+0000","last_fullsized":"2023-10-08T00:56:40.160128+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:15.309253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.134160+0000","last_change":"2023-10-08T00:56:39.133222+0000","last_active":"2023-10-08T00:56:39.134160+0000","last_peered":"2023-10-08T00:56:39.134160+0000","last_clean":"2023-10-08T00:56:39.134160+0000","last_became_active":"2023-10-08T00:56:39.132908+0000","last_became_peered":"2023-10-08T00:56:39.132908+0000","last_unstale":"2023-10-08T00:56:39.134160+0000","last_undegraded":"2023-10-08T00:56:39.134160+0000","last_fullsized":"2023-10-08T00:56:39.134160+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:52:14.302511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":8,"reported_epoch":30,"state":"active+clean","last_fresh":"2023-10-08T00:56:40.140308+0000","last_change":"2023-10-08T00:56:40.140308+0000","last_active":"2023-10-08T00:56:40.140308+0000","last_peered":"2023-10-08T00:56:40.140308+0000","last_clean":"2023-10-08T00:56:40.140308+0000","last_became_active":"2023-10-08T00:56:40.139963+0000","last_became_peered":"2023-10-08T00:56:40.139963+0000","last_unstale":"2023-10-08T00:56:40.140308+0000","last_undegraded":"2023-10-08T00:56:40.140308+0000","last_fullsized":"2023-10-08T00:56:40.140308+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:39.101155+0000","last_clean_scrub_stamp":"2023-10-08T00:56:39.101155+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:20:27.686995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.126650+0000","last_change":"2023-10-08T00:56:39.121784+0000","last_active":"2023-10-08T00:56:39.126650+0000","last_peered":"2023-10-08T00:56:39.126650+0000","last_clean":"2023-10-08T00:56:39.126650+0000","last_became_active":"2023-10-08T00:56:39.121413+0000","last_became_peered":"2023-10-08T00:56:39.121413+0000","last_unstale":"2023-10-08T00:56:39.126650+0000","last_undegraded":"2023-10-08T00:56:39.126650+0000","last_fullsized":"2023-10-08T00:56:39.126650+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:56:38.094413+0000","last_clean_scrub_stamp":"2023-10-08T00:56:38.094413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:13.696837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":29,"state":"active+clean","last_fresh":"2023-10-08T00:56:39.127317+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T00:56:39.127317+0000","last_peered":"2023-10-08T00:56:39.127317+0000","last_clean":"2023-10-08T00:56:39.127317+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T00:56:39.127317+0000","last_undegraded":"2023-10-08T00:56:39.127317+0000","last_fullsized":"2023-10-08T00:56:39.127317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298200,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073640,"statfs":{"total":96636764160,"available":96331407360,"internally_reserved":0,"allocated":933888,"data_stored":718208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.85999999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.66800000000000004}]},{"osd":3,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.80600000000000005}]},{"osd":4,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.752},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.83699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":344064,"data_stored":128338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.621},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.51200000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.58399999999999996}]},{"osd":2,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.44800000000000001}]},{"osd":5,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.7569999999999999,"5min":1.7569999999999999,"15min":1.7569999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.71599999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":352256,"data_stored":129308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.32800000000000001}]},{"osd":2,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48299999999999998}]},{"osd":3,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.876},{"interface":"front","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.73699999999999999}]},{"osd":6,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.617,"5min":1.617,"15min":1.617},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.82999999999999996}]},{"osd":7,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":0.84399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298196,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073644,"statfs":{"total":96636764160,"available":96331411456,"internally_reserved":0,"allocated":929792,"data_stored":718118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.46500000000000002}]},{"osd":2,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.505},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.53300000000000003}]},{"osd":3,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.67700000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.94399999999999995}]},{"osd":6,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.91200000000000003}]},{"osd":7,"last update":"Sun Oct 8 00:56:37 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.997},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.008}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297620,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074220,"statfs":{"total":96636764160,"available":96332001280,"internally_reserved":0,"allocated":339968,"data_stored":127820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.45600000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.42499999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.443}]},{"osd":4,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64800000000000002}]},{"osd":5,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.63300000000000001}]},{"osd":6,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:56:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.58599999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297632,"kb_used_data":344,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074208,"statfs":{"total":96636764160,"available":96331988992,"internally_reserved":0,"allocated":352256,"data_stored":129308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.442},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.32000000000000001}]},{"osd":1,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.5629999999999999,"5min":1.5629999999999999,"15min":1.5629999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"last":0.48399999999999999}]},{"osd":4,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.68300000000000005}]},{"osd":5,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.431,"5min":1.431,"15min":1.431},"last":0.67200000000000004}]},{"osd":6,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.458,"5min":1.458,"15min":1.458},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.54200000000000004}]},{"osd":7,"last update":"Sun Oct 8 00:56:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.55500000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":344064,"data_stored":127854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.49399999999999999}]},{"osd":1,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.60999999999999999}]},{"osd":2,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.45400000000000001}]},{"osd":3,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.628},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.59399999999999997}]},{"osd":5,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.70199999999999996}]},{"osd":6,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.73599999999999999}]},{"osd":7,"last update":"Sun Oct 8 00:56:40 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.71499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297624,"kb_used_data":336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074216,"statfs":{"total":96636764160,"available":96331997184,"internally_reserved":0,"allocated":344064,"data_stored":128394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51900000000000002}]},{"osd":1,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.077,"5min":1.077,"15min":1.077},"last":0.499}]},{"osd":2,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.48499999999999999}]},{"osd":3,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.69199999999999995}]},{"osd":4,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.65700000000000003}]},{"osd":6,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.625},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.59699999999999998}]},{"osd":7,"last update":"Sun Oct 8 00:56:36 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3879999999999999,"5min":1.3879999999999999,"15min":1.3879999999999999},"last":0.746}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T00:56:44.925 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T00:56:44.925 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T00:56:45.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.146+0000 7fc14179d700 1 Processor -- start 2023-10-08T00:56:45.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.146+0000 7fc14179d700 1 -- start start 2023-10-08T00:56:45.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.147+0000 7fc14179d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c1595d0 0x7fc13c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.147+0000 7fc14179d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13c0b10f0 con 0x7fc13c15d930 2023-10-08T00:56:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.147+0000 7fc14179d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13c0cd530 con 0x7fc13c0cf480 2023-10-08T00:56:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.147+0000 7fc14179d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c15e000 con 0x7fc13c1595d0 2023-10-08T00:56:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc140f9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc13c15d930 0x7fc13c0b1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37418/0 (socket says 172.21.15.119:37418) 2023-10-08T00:56:45.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc140f9c700 1 -- 172.21.15.119:0/1636468397 learned_addr learned my addr 172.21.15.119:0/1636468397 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:45.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13bfff700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c1595d0 0x7fc13c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156845696 0 0) 0x7fc13c0b10f0 con 0x7fc13c15d930 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc124003570 con 0x7fc13c15d930 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3362648225 0 0) 0x7fc124003570 con 0x7fc13c15d930 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c1595d0 msgr2=0x7fc13c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c1595d0 0x7fc13c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 >> v1:172.21.15.184:6789/0 conn(0x7fc13c0cf480 legacy=0x7fc13c15d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c0cf990 con 0x7fc13c15d930 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fc1340017f0 con 0x7fc13c15d930 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 >> v1:172.21.15.119:6789/0 conn(0x7fc13c15d930 legacy=0x7fc13c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124003a90 0x7fc124003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.148+0000 7fc13affd700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc124004810 0x7fc124004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13affd700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c1595d0 0x7fc124005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c15e000 con 0x7fc124004810 2023-10-08T00:56:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12400bbf0 con 0x7fc124003a90 2023-10-08T00:56:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12400bd80 con 0x7fc13c1595d0 2023-10-08T00:56:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13b7fe700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc124004810 0x7fc124004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13bfff700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124003a90 0x7fc124003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc140f9c700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c1595d0 0x7fc124005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13b7fe700 1 -- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c1595d0 msgr2=0x7fc124005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13b7fe700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c1595d0 0x7fc124005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13b7fe700 1 -- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124003a90 msgr2=0x7fc124003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13b7fe700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124003a90 0x7fc124003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.149+0000 7fc13b7fe700 1 -- 172.21.15.119:0/1636468397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc124003570 con 0x7fc124004810 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc13b7fe700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc124004810 0x7fc124004c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc1300018e0 tx=0x7fc130007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76079b3b82a41303 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130012420 con 0x7fc124004810 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc130012580 con 0x7fc124004810 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc13affd700 1 -- 172.21.15.119:0/1636468397 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130012920 con 0x7fc124004810 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc14179d700 1 -- 172.21.15.119:0/1636468397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc124004810 msgr2=0x7fc124004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc14179d700 1 --2- 172.21.15.119:0/1636468397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc124004810 0x7fc124004c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc1300018e0 tx=0x7fc130007d30 comp rx=0 tx=0).stop 2023-10-08T00:56:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc140f9c700 1 -- 172.21.15.119:0/1636468397 reap_dead start 2023-10-08T00:56:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc14179d700 1 -- 172.21.15.119:0/1636468397 shutdown_connections 2023-10-08T00:56:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc14179d700 1 -- 172.21.15.119:0/1636468397 >> 172.21.15.119:0/1636468397 conn(0x7fc13c0d05f0 msgr2=0x7fc13c0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc14179d700 1 -- 172.21.15.119:0/1636468397 shutdown_connections 2023-10-08T00:56:45.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.150+0000 7fc14179d700 1 -- 172.21.15.119:0/1636468397 wait complete. 2023-10-08T00:56:45.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.152+0000 7fc14179d700 1 Processor -- start 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.152+0000 7fc14179d700 1 -- start start 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc14179d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124004810 0x7fc13c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc14179d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc13c150bf0 0x7fc13c14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc14179d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c140d60 0x7fc13c1411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc14179d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c15d520 con 0x7fc13c150bf0 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc14179d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c141720 con 0x7fc124004810 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc14179d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c15cb80 con 0x7fc13c140d60 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc13c150bf0 0x7fc13c14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc13c150bf0 0x7fc13c14ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38440/0 (socket says 172.21.15.119:38440) 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc13b7fe700 1 -- 172.21.15.119:0/1763344624 learned_addr learned my addr 172.21.15.119:0/1763344624 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc13bfff700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124004810 0x7fc13c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.153+0000 7fc140f9c700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c140d60 0x7fc13c1411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc13b7fe700 1 -- 172.21.15.119:0/1763344624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c140d60 msgr2=0x7fc13c1411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc13b7fe700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c140d60 0x7fc13c1411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc13b7fe700 1 -- 172.21.15.119:0/1763344624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124004810 msgr2=0x7fc13c14db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc13b7fe700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124004810 0x7fc13c14db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc13b7fe700 1 -- 172.21.15.119:0/1763344624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c15ce60 con 0x7fc13c150bf0 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc13b7fe700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc13c150bf0 0x7fc13c14ec60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc130002000 tx=0x7fc130015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc123fff700 1 -- 172.21.15.119:0/1763344624 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc130001200 con 0x7fc13c150bf0 2023-10-08T00:56:45.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc123fff700 1 -- 172.21.15.119:0/1763344624 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc13000ca90 con 0x7fc13c150bf0 2023-10-08T00:56:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c15d140 con 0x7fc13c150bf0 2023-10-08T00:56:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc123fff700 1 -- 172.21.15.119:0/1763344624 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13000ce30 con 0x7fc13c150bf0 2023-10-08T00:56:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.154+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c14e060 con 0x7fc13c150bf0 2023-10-08T00:56:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.155+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13c04c3c0 con 0x7fc13c150bf0 2023-10-08T00:56:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.156+0000 7fc123fff700 1 -- 172.21.15.119:0/1763344624 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc130019070 con 0x7fc13c150bf0 2023-10-08T00:56:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.157+0000 7fc123fff700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc128066f30 0x7fc1280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.157+0000 7fc123fff700 1 -- 172.21.15.119:0/1763344624 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc130085660 con 0x7fc13c150bf0 2023-10-08T00:56:45.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.157+0000 7fc13bfff700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc128066f30 0x7fc1280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.158+0000 7fc13bfff700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc128066f30 0x7fc1280693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006ea0 tx=0x7fc12c009120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:45.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.161+0000 7fc123fff700 1 -- 172.21.15.119:0/1763344624 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1300528d0 con 0x7fc13c150bf0 2023-10-08T00:56:45.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.426+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc13c0b3860 con 0x7fc13c150bf0 2023-10-08T00:56:45.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.430+0000 7fc123fff700 1 -- 172.21.15.119:0/1763344624 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc130055ef0 con 0x7fc13c150bf0 2023-10-08T00:56:45.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc128066f30 msgr2=0x7fc1280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc128066f30 0x7fc1280693e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc12c006ea0 tx=0x7fc12c009120 comp rx=0 tx=0).stop 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc13c150bf0 msgr2=0x7fc13c14ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc13c150bf0 0x7fc13c14ec60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc130002000 tx=0x7fc130015000 comp rx=0 tx=0).stop 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 shutdown_connections 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc13c140d60 0x7fc13c1411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc128066f30 0x7fc1280693e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc124004810 0x7fc13c14db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 --2- 172.21.15.119:0/1763344624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc13c150bf0 0x7fc13c14ec60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 >> 172.21.15.119:0/1763344624 conn(0x7fc13c0d05f0 msgr2=0x7fc13c15a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 shutdown_connections 2023-10-08T00:56:45.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.433+0000 7fc14179d700 1 -- 172.21.15.119:0/1763344624 wait complete. 2023-10-08T00:56:45.436 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T00:56:45.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T00:56:45.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.633+0000 7f72b539d700 1 Processor -- start 2023-10-08T00:56:45.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.633+0000 7f72b539d700 1 -- start start 2023-10-08T00:56:45.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.634+0000 7f72b539d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72b0159420 0x7f72b00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.634+0000 7f72b539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b01597f0 0x7f72b00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.634+0000 7f72b539d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72b015d230 0x7f72b015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.634+0000 7f72b539d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b015dbd0 con 0x7f72b015d230 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.634+0000 7f72b539d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72b00b11d0 con 0x7f72b0159420 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.634+0000 7f72b539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00b14b0 con 0x7f72b01597f0 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72b015d230 0x7f72b015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72b015d230 0x7f72b015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38446/0 (socket says 172.21.15.119:38446) 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 -- 172.21.15.119:0/877290477 learned_addr learned my addr 172.21.15.119:0/877290477 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72affff700 1 --2- 172.21.15.119:0/877290477 >> v2:172.21.15.184:3300/0 conn(0x7f72b0159420 0x7f72b00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72af7fe700 1 --2- 172.21.15.119:0/877290477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b01597f0 0x7f72b00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 -- 172.21.15.119:0/877290477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b01597f0 msgr2=0x7f72b00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 --2- 172.21.15.119:0/877290477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b01597f0 0x7f72b00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 -- 172.21.15.119:0/877290477 >> v2:172.21.15.184:3300/0 conn(0x7f72b0159420 msgr2=0x7f72b00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 --2- 172.21.15.119:0/877290477 >> v2:172.21.15.184:3300/0 conn(0x7f72b0159420 0x7f72b00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 -- 172.21.15.119:0/877290477 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00b1790 con 0x7f72b015d230 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.635+0000 7f72b4b9c700 1 --2- 172.21.15.119:0/877290477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b015d230 0x7f72b015d690 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72b00b2f20 tx=0x7f72a8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71552975d6d71a9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72aeffd700 1 -- 172.21.15.119:0/877290477 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a800e070 con 0x7f72b015d230 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72aeffd700 1 -- 172.21.15.119:0/877290477 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a800b970 con 0x7f72b015d230 2023-10-08T00:56:45.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72aeffd700 1 -- 172.21.15.119:0/877290477 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a80092e0 con 0x7f72b015d230 2023-10-08T00:56:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 -- 172.21.15.119:0/877290477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b015d230 msgr2=0x7f72b015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 --2- 172.21.15.119:0/877290477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b015d230 0x7f72b015d690 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f72b00b2f20 tx=0x7f72a8009840 comp rx=0 tx=0).stop 2023-10-08T00:56:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 -- 172.21.15.119:0/877290477 shutdown_connections 2023-10-08T00:56:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 --2- 172.21.15.119:0/877290477 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b015d230 0x7f72b015d690 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 --2- 172.21.15.119:0/877290477 >> v2:172.21.15.184:3300/0 conn(0x7f72b0159420 0x7f72b00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 --2- 172.21.15.119:0/877290477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b01597f0 0x7f72b00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 -- 172.21.15.119:0/877290477 >> 172.21.15.119:0/877290477 conn(0x7f72b00d0660 msgr2=0x7f72b00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:45.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 -- 172.21.15.119:0/877290477 shutdown_connections 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.636+0000 7f72b539d700 1 -- 172.21.15.119:0/877290477 wait complete. 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.638+0000 7f72b539d700 1 Processor -- start 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.638+0000 7f72b539d700 1 -- start start 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72b539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b0159420 0x7f72b0155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72b539d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b01597f0 0x7f72b015c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72b539d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b015d230 0x7f72b015c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72b539d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b015bf70 con 0x7f72b01597f0 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72b539d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00c7150 con 0x7f72b015d230 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72b539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b00c75e0 con 0x7f72b0159420 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b01597f0 0x7f72b015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b01597f0 0x7f72b015c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38458/0 (socket says 172.21.15.119:38458) 2023-10-08T00:56:45.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 -- 172.21.15.119:0/376793174 learned_addr learned my addr 172.21.15.119:0/376793174 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 -- 172.21.15.119:0/376793174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b0159420 msgr2=0x7f72b0155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b0159420 0x7f72b0155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 -- 172.21.15.119:0/376793174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b015d230 msgr2=0x7f72b015c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b015d230 0x7f72b015c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 -- 172.21.15.119:0/376793174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a8009ae0 con 0x7f72b01597f0 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.639+0000 7f72af7fe700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b01597f0 0x7f72b015c180 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f72a400ec80 tx=0x7f72a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.640+0000 7f7297fff700 1 -- 172.21.15.119:0/376793174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a400c9d0 con 0x7f72b01597f0 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.640+0000 7f7297fff700 1 -- 172.21.15.119:0/376793174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a400cb50 con 0x7f72b01597f0 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.640+0000 7f7297fff700 1 -- 172.21.15.119:0/376793174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4013470 con 0x7f72b01597f0 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.640+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b00c6540 con 0x7f72b01597f0 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.640+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b00c6b00 con 0x7f72b01597f0 2023-10-08T00:56:45.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.641+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b004c3c0 con 0x7f72b01597f0 2023-10-08T00:56:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.642+0000 7f7297fff700 1 -- 172.21.15.119:0/376793174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72a4013860 con 0x7f72b01597f0 2023-10-08T00:56:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.643+0000 7f7297fff700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f729c066f30 0x7f729c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:45.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.643+0000 7f7297fff700 1 -- 172.21.15.119:0/376793174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f72a40849a0 con 0x7f72b01597f0 2023-10-08T00:56:45.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.643+0000 7f72affff700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f729c066f30 0x7f729c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:45.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.643+0000 7f72affff700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f729c066f30 0x7f729c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f72b0155b10 tx=0x7f72a00091a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:45.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.647+0000 7f7297fff700 1 -- 172.21.15.119:0/376793174 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72a4051b70 con 0x7f72b01597f0 2023-10-08T00:56:45.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.879+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72b00b1a70 con 0x7f72b01597f0 2023-10-08T00:56:45.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.880+0000 7f7297fff700 1 -- 172.21.15.119:0/376793174 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4606 (secure 0 0 0) 0x7f72a4055190 con 0x7f72b01597f0 2023-10-08T00:56:45.881 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T00:56:45.881 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/768996961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":768996961},{"type":"v1","addr":"172.21.15.184:6834","nonce":768996961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1876644893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1876644893},{"type":"v1","addr":"172.21.15.119:6837","nonce":1876644893}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1187803776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1187803776},{"type":"v1","addr":"172.21.15.184:6835","nonce":1187803776}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:56:39.582864+0000","modified":"2023-10-08T00:56:43.160226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3785845701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3785845701},{"type":"v1","addr":"172.21.15.119:6836","nonce":3785845701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T00:56:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f729c066f30 msgr2=0x7f729c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f729c066f30 0x7f729c0693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f72b0155b10 tx=0x7f72a00091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b01597f0 msgr2=0x7f72b015c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b01597f0 0x7f72b015c180 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f72a400ec80 tx=0x7f72a40091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 shutdown_connections 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b0159420 0x7f72b0155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f729c066f30 0x7f729c0693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b015d230 0x7f72b015c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 --2- 172.21.15.119:0/376793174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b01597f0 0x7f72b015c180 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 >> 172.21.15.119:0/376793174 conn(0x7f72b00d0660 msgr2=0x7f72b0069bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 shutdown_connections 2023-10-08T00:56:45.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:45.882+0000 7f72b539d700 1 -- 172.21.15.119:0/376793174 wait complete. 2023-10-08T00:56:45.885 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2023-10-08T00:56:46.303 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T00:56:39.582864+0000', 'modified': '2023-10-08T00:56:43.160226+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4938}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4938': {'gid': 4938, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.119:6836/3785845701', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 3785845701}, {'type': 'v1', 'addr': '172.21.15.119:6836', 'nonce': 3785845701}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T00:56:46.303 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T00:56:46.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T00:56:46.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.487+0000 7fd7c29ff700 1 Processor -- start 2023-10-08T00:56:46.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.487+0000 7fd7c29ff700 1 -- start start 2023-10-08T00:56:46.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.488+0000 7fd7c29ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0afc40 0x7fd7bc0b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.488+0000 7fd7c29ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc0b0750 con 0x7fd7bc0b2140 2023-10-08T00:56:46.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.488+0000 7fd7c29ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7bc0aca70 con 0x7fd7bc0b0010 2023-10-08T00:56:46.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.489+0000 7fd7c29ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc0acd50 con 0x7fd7bc0afc40 2023-10-08T00:56:46.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.489+0000 7fd7bb7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd7bc0b2140 0x7fd7bc15d850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37420/0 (socket says 172.21.15.119:37420) 2023-10-08T00:56:46.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.489+0000 7fd7bb7fe700 1 -- 172.21.15.119:0/3200615237 learned_addr learned my addr 172.21.15.119:0/3200615237 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:46.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.489+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/3200615237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0afc40 0x7fd7bc0b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.489+0000 7fd7baffd700 1 -- 172.21.15.119:0/3200615237 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290893871 0 0) 0x7fd7bc0b0750 con 0x7fd7bc0b2140 2023-10-08T00:56:46.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.489+0000 7fd7baffd700 1 -- 172.21.15.119:0/3200615237 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a4003570 con 0x7fd7bc0b2140 2023-10-08T00:56:46.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.490+0000 7fd7baffd700 1 -- 172.21.15.119:0/3200615237 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3595026342 0 0) 0x7fd7a4003570 con 0x7fd7bc0b2140 2023-10-08T00:56:46.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.490+0000 7fd7bbfff700 1 -- 172.21.15.119:0/3200615237 >> v1:172.21.15.184:6789/0 conn(0x7fd7bc0b0010 legacy=0x7fd7bc0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.490+0000 7fd7bbfff700 1 -- 172.21.15.119:0/3200615237 >> v1:172.21.15.119:6789/0 conn(0x7fd7bc0b2140 legacy=0x7fd7bc15d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.490+0000 7fd7bbfff700 1 -- 172.21.15.119:0/3200615237 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7bc0ad030 con 0x7fd7bc0afc40 2023-10-08T00:56:46.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.490+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/3200615237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0afc40 0x7fd7bc0b1c00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009910 tx=0x7fd7ac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=510f077c1f2df796 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.490+0000 7fd7baffd700 1 -- 172.21.15.119:0/3200615237 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0001470 con 0x7fd7bc0afc40 2023-10-08T00:56:46.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.490+0000 7fd7baffd700 1 -- 172.21.15.119:0/3200615237 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac011420 con 0x7fd7bc0afc40 2023-10-08T00:56:46.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.490+0000 7fd7baffd700 1 -- 172.21.15.119:0/3200615237 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0117c0 con 0x7fd7bc0afc40 2023-10-08T00:56:46.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.491+0000 7fd7c29ff700 1 -- 172.21.15.119:0/3200615237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0afc40 msgr2=0x7fd7bc0b1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.491+0000 7fd7c29ff700 1 --2- 172.21.15.119:0/3200615237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0afc40 0x7fd7bc0b1c00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac009910 tx=0x7fd7ac0091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.491+0000 7fd7c29ff700 1 -- 172.21.15.119:0/3200615237 shutdown_connections 2023-10-08T00:56:46.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.491+0000 7fd7c29ff700 1 --2- 172.21.15.119:0/3200615237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0afc40 0x7fd7bc0b1c00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.491+0000 7fd7c29ff700 1 -- 172.21.15.119:0/3200615237 >> 172.21.15.119:0/3200615237 conn(0x7fd7bc06a320 msgr2=0x7fd7bc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:46.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.491+0000 7fd7c29ff700 1 -- 172.21.15.119:0/3200615237 shutdown_connections 2023-10-08T00:56:46.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.491+0000 7fd7c29ff700 1 -- 172.21.15.119:0/3200615237 wait complete. 2023-10-08T00:56:46.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.492+0000 7fd7c29ff700 1 Processor -- start 2023-10-08T00:56:46.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.492+0000 7fd7c29ff700 1 -- start start 2023-10-08T00:56:46.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.493+0000 7fd7c29ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7bc0afc40 0x7fd7bc16eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.493+0000 7fd7c29ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0b0010 0x7fd7bc1528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.493+0000 7fd7c29ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7bc0b2140 0x7fd7bc15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.493+0000 7fd7c29ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc16f2d0 con 0x7fd7bc0afc40 2023-10-08T00:56:46.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.493+0000 7fd7c29ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc154b00 con 0x7fd7bc0b2140 2023-10-08T00:56:46.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.493+0000 7fd7c29ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7bc154dc0 con 0x7fd7bc0b0010 2023-10-08T00:56:46.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7bc0afc40 0x7fd7bc16eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7bc0afc40 0x7fd7bc16eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38462/0 (socket says 172.21.15.119:38462) 2023-10-08T00:56:46.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 -- 172.21.15.119:0/2236900151 learned_addr learned my addr 172.21.15.119:0/2236900151 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:46.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 -- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0b0010 msgr2=0x7fd7bc1528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0b0010 0x7fd7bc1528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7bc0b2140 0x7fd7bc15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0b0010 0x7fd7bc1528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 -- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7bc0b2140 msgr2=0x7fd7bc15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7bc0b2140 0x7fd7bc15c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 -- 172.21.15.119:0/2236900151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac009420 con 0x7fd7bc0afc40 2023-10-08T00:56:46.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0b0010 0x7fd7bc1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:46.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7bc0b2140 0x7fd7bc15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:46.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7bc0afc40 0x7fd7bc16eb20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0030c0 tx=0x7fd7ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.494+0000 7fd7a3fff700 1 -- 172.21.15.119:0/2236900151 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac010c20 con 0x7fd7bc0afc40 2023-10-08T00:56:46.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.495+0000 7fd7a3fff700 1 -- 172.21.15.119:0/2236900151 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac010d80 con 0x7fd7bc0afc40 2023-10-08T00:56:46.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.495+0000 7fd7a3fff700 1 -- 172.21.15.119:0/2236900151 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac01d680 con 0x7fd7bc0afc40 2023-10-08T00:56:46.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.495+0000 7fd7c29ff700 1 -- 172.21.15.119:0/2236900151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7bc1550a0 con 0x7fd7bc0afc40 2023-10-08T00:56:46.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.495+0000 7fd7c29ff700 1 -- 172.21.15.119:0/2236900151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc16f060 con 0x7fd7bc0afc40 2023-10-08T00:56:46.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.497+0000 7fd7a3fff700 1 -- 172.21.15.119:0/2236900151 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd7ac01d9f0 con 0x7fd7bc0afc40 2023-10-08T00:56:46.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.498+0000 7fd7a3fff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7a8066f30 0x7fd7a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.498+0000 7fd7a3fff700 1 -- 172.21.15.119:0/2236900151 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd7ac085ce0 con 0x7fd7bc0afc40 2023-10-08T00:56:46.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.498+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7a8066f30 0x7fd7a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.498+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7a8066f30 0x7fd7a80693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0001ed0 tx=0x7fd7b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.499+0000 7fd7d1134b80 1 Processor -- start 2023-10-08T00:56:46.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.499+0000 7fd7d1134b80 1 -- start start 2023-10-08T00:56:46.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.500+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773d79e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.500+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773dc3d00 0x55a773da2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.500+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773d75cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.500+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a773cd0a70 con 0x55a773d79a50 2023-10-08T00:56:46.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.500+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a773cce270 con 0x55a773da2f60 2023-10-08T00:56:46.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.500+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a773cca060 con 0x55a773dc3d00 2023-10-08T00:56:46.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773d79e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773d79e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38470/0 (socket says 172.21.15.119:38470) 2023-10-08T00:56:46.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 -- 172.21.15.119:0/3255987435 learned_addr learned my addr 172.21.15.119:0/3255987435 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:46.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773dc3d00 0x55a773da2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773d75cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 -- 172.21.15.119:0/3255987435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773dc3d00 msgr2=0x55a773da2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773dc3d00 0x55a773da2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 -- 172.21.15.119:0/3255987435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 msgr2=0x55a773d75cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773d75cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 -- 172.21.15.119:0/3255987435 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a773b64270 con 0x55a773d79a50 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773d79e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac084130 tx=0x7fd7ac0840c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4952e4b09d975a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7a17fa700 1 -- 172.21.15.119:0/3255987435 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac056b70 con 0x55a773d79a50 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7a17fa700 1 -- 172.21.15.119:0/3255987435 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac059dc0 con 0x55a773d79a50 2023-10-08T00:56:46.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.501+0000 7fd7a17fa700 1 -- 172.21.15.119:0/3255987435 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac05db10 con 0x55a773d79a50 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 -- 172.21.15.119:0/3255987435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 msgr2=0x55a773d79e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773d79e20 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac084130 tx=0x7fd7ac0840c0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 -- 172.21.15.119:0/3255987435 shutdown_connections 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773d75cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773d79e20 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/3255987435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773dc3d00 0x55a773da2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 -- 172.21.15.119:0/3255987435 >> 172.21.15.119:0/3255987435 conn(0x55a773d7b370 msgr2=0x55a773d7b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 -- 172.21.15.119:0/3255987435 shutdown_connections 2023-10-08T00:56:46.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.502+0000 7fd7d1134b80 1 -- 172.21.15.119:0/3255987435 wait complete. 2023-10-08T00:56:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.503+0000 7fd7d1134b80 1 -- start start 2023-10-08T00:56:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.503+0000 7fd7d1134b80 1 Processor -- start 2023-10-08T00:56:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773d79a50 0x55a773da5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773dd0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773dba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a773c75890 con 0x55a773dc3d00 2023-10-08T00:56:46.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a773c90ea0 con 0x55a773d79a50 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a773b4ce30 con 0x55a773da2f60 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7d1134b80 10 client.0 _ll_register_callbacks cb 0x55a773d3d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7c0f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773dba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7c0f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773dba450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38482/0 (socket says 172.21.15.119:38482) 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7c0f9c700 1 -- 172.21.15.119:0/1281870686 learned_addr learned my addr 172.21.15.119:0/1281870686 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773dd0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.504+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773d79a50 0x55a773da5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7c0f9c700 1 -- 172.21.15.119:0/1281870686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 msgr2=0x55a773dd0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773dd0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7c0f9c700 1 -- 172.21.15.119:0/1281870686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773d79a50 msgr2=0x55a773da5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773d79a50 0x55a773da5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7c0f9c700 1 -- 172.21.15.119:0/1281870686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7bc16f060 con 0x55a773dc3d00 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773dba450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400f700 tx=0x7fd7b4009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46615b7c2a2e9f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd79a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd79a7fc700 1 -- 172.21.15.119:0/1281870686 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40032e0 con 0x55a773dc3d00 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd79a7fc700 1 -- 172.21.15.119:0/1281870686 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd7b4003440 con 0x55a773dc3d00 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1281870686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a773d6eb70 con 0x55a773dc3d00 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd79a7fc700 1 -- 172.21.15.119:0/1281870686 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b4003a50 con 0x55a773dc3d00 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7d1134b80 10 client.5106 fetch_fsmap learned FSMap version 16 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1281870686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55a773b06fe0 con 0x55a773dc3d00 2023-10-08T00:56:46.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.505+0000 7fd79a7fc700 1 -- 172.21.15.119:0/1281870686 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a773d6eb70 con 0x55a773dc3d00 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd79a7fc700 1 -- 172.21.15.119:0/1281870686 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd7b4012070 con 0x55a773dc3d00 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd7d1134b80 10 client.5106 fetch_fsmap finished waiting for FSMap version 16 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd7d1134b80 10 client.5106 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd7d1134b80 20 client.5106 populate_metadata read hostname 'smithi119' 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] conn(0x55a773d74c20 0x55a773d74ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd7d1134b80 4 client.5106 mds_command: new command op to 4938 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1281870686 --> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a773d4baa0 con 0x55a773d74c20 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] conn(0x55a773d74c20 0x55a773d74ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] conn(0x55a773d74c20 0x55a773d74ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b79bc4ab48ad659a server_cookie=e5a798e025b08e74 in_seq=0 out_seq=0 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.506+0000 7fd79a7fc700 10 client.5106 ms_handle_connect on v2:172.21.15.119:6834/3785845701 2023-10-08T00:56:46.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd79a7fc700 1 -- 172.21.15.119:0/1281870686 <== mds.0 v2:172.21.15.119:6834/3785845701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a773d4baa0 con 0x55a773d74c20 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd79a7fc700 10 client.5106 handle_command_reply: tid=0 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 client.5106 shutdown 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1281870686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 msgr2=0x55a773dba450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773dba450 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400f700 tx=0x7fd7b4009040 comp rx=0 tx=0).stop 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1281870686 shutdown_connections 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] conn(0x55a773d74c20 0x55a773d74ff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773dba450 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773d79a50 0x55a773da5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1281870686 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773dd0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd79a7fc700 0 client.5106 ms_handle_reset on v2:172.21.15.119:6834/3785845701 2023-10-08T00:56:46.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1281870686 >> 172.21.15.119:0/1281870686 conn(0x55a773d7b370 msgr2=0x55a773dbf550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:46.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1281870686 shutdown_connections 2023-10-08T00:56:46.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.507+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1281870686 wait complete. 2023-10-08T00:56:46.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.508+0000 7fd7d1134b80 20 client.5106 trim_cache size 0 max 16384 2023-10-08T00:56:46.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.524+0000 7fd7d1134b80 1 Processor -- start 2023-10-08T00:56:46.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.524+0000 7fd7d1134b80 1 -- start start 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.525+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773da3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.525+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773d757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.525+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773dc3d00 0x55a773dc2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.525+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a773d7cdb0 con 0x55a773d79a50 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.525+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a773d75120 con 0x55a773dc3d00 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.525+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a773d59510 con 0x55a773da2f60 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773da3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773da3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38494/0 (socket says 172.21.15.119:38494) 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 -- 172.21.15.119:0/1995810444 learned_addr learned my addr 172.21.15.119:0/1995810444 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:46.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773d757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 -- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 msgr2=0x55a773d757f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773d757f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773dc3d00 0x55a773dc2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 -- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773dc3d00 msgr2=0x55a773dc2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773dc3d00 0x55a773dc2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 -- 172.21.15.119:0/1995810444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a773d77390 con 0x55a773d79a50 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773d757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773dc3d00 0x55a773dc2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773da3430 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac012fd0 tx=0x7fd7ac06d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491a04c871f94d0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7a1ffb700 1 -- 172.21.15.119:0/1995810444 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac056b70 con 0x55a773d79a50 2023-10-08T00:56:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7a1ffb700 1 -- 172.21.15.119:0/1995810444 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac05db10 con 0x55a773d79a50 2023-10-08T00:56:46.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.526+0000 7fd7a1ffb700 1 -- 172.21.15.119:0/1995810444 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac010a10 con 0x55a773d79a50 2023-10-08T00:56:46.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1995810444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 msgr2=0x55a773da3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773da3430 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac012fd0 tx=0x7fd7ac06d040 comp rx=0 tx=0).stop 2023-10-08T00:56:46.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1995810444 shutdown_connections 2023-10-08T00:56:46.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773dc3d00 0x55a773dc2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773d79a50 0x55a773da3430 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/1995810444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773da2f60 0x55a773d757f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1995810444 >> 172.21.15.119:0/1995810444 conn(0x55a773d7b370 msgr2=0x55a773d724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1995810444 shutdown_connections 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.527+0000 7fd7d1134b80 1 -- 172.21.15.119:0/1995810444 wait complete. 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.528+0000 7fd7d1134b80 1 -- start start 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.528+0000 7fd7d1134b80 1 Processor -- start 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773d79a50 0x55a773e26390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773e22dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7d1134b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773db6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a773dcf290 con 0x55a773dc3d00 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a773da37a0 con 0x55a773da2f60 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7d1134b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a773da2470 con 0x55a773d79a50 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7d1134b80 10 client.0 _ll_register_callbacks cb 0x55a773d3d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7c0f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773db6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7c0f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773db6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38496/0 (socket says 172.21.15.119:38496) 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7c0f9c700 1 -- 172.21.15.119:0/621900672 learned_addr learned my addr 172.21.15.119:0/621900672 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773d79a50 0x55a773e26390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773e22dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7c0f9c700 1 -- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773d79a50 msgr2=0x55a773e26390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773d79a50 0x55a773e26390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7c0f9c700 1 -- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 msgr2=0x55a773e22dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773e22dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7c0f9c700 1 -- 172.21.15.119:0/621900672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a773b64270 con 0x55a773dc3d00 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.529+0000 7fd7bb7fe700 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773e22dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd7c0f9c700 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773db6560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400fd70 tx=0x7fd7b401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=289b10879648ca9e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd79a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd79a7fc700 1 -- 172.21.15.119:0/621900672 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b40038f0 con 0x55a773dc3d00 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd7d1134b80 1 -- 172.21.15.119:0/621900672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd7ac059dc0 con 0x55a773dc3d00 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd79a7fc700 1 -- 172.21.15.119:0/621900672 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fd7b4001b00 con 0x55a773dc3d00 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd79a7fc700 1 -- 172.21.15.119:0/621900672 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd7b4003a50 con 0x55a773dc3d00 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd7d1134b80 10 client.5112 fetch_fsmap learned FSMap version 16 2023-10-08T00:56:46.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd7d1134b80 1 -- 172.21.15.119:0/621900672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55a773c98780 con 0x55a773dc3d00 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd79a7fc700 1 -- 172.21.15.119:0/621900672 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac059dc0 con 0x55a773dc3d00 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.530+0000 7fd79a7fc700 1 -- 172.21.15.119:0/621900672 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 16) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd7b40133f0 con 0x55a773dc3d00 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7d1134b80 10 client.5112 fetch_fsmap finished waiting for FSMap version 16 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7d1134b80 10 client.5112 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7d1134b80 20 client.5112 populate_metadata read hostname 'smithi119' 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] conn(0x55a773dbcfd0 0x55a773dbd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7d1134b80 4 client.5112 mds_command: new command op to 4938 tid=0[{"prefix": "status"}] 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7d1134b80 1 -- 172.21.15.119:0/621900672 --> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a773c6ce00 con 0x55a773dbcfd0 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] conn(0x55a773dbcfd0 0x55a773dbd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7bbfff700 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] conn(0x55a773dbcfd0 0x55a773dbd3a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5728ca68915f5bb5 server_cookie=cf608aeb59020fc8 in_seq=0 out_seq=0 2023-10-08T00:56:46.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd79a7fc700 10 client.5112 ms_handle_connect on v2:172.21.15.119:6834/3785845701 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd79a7fc700 1 -- 172.21.15.119:0/621900672 <== mds.0 v2:172.21.15.119:6834/3785845701 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55a773c6ce00 con 0x55a773dbcfd0 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd79a7fc700 10 client.5112 handle_command_reply: tid=0 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.531+0000 7fd7d1134b80 1 client.5112 shutdown 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 -- 172.21.15.119:0/621900672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 msgr2=0x55a773db6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773db6560 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400fd70 tx=0x7fd7b401e000 comp rx=0 tx=0).stop 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 -- 172.21.15.119:0/621900672 shutdown_connections 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a773d79a50 0x55a773e26390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.119:6834/3785845701,v1:172.21.15.119:6836/3785845701] conn(0x55a773dbcfd0 0x55a773dbd3a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a773dc3d00 0x55a773db6560 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 --2- 172.21.15.119:0/621900672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a773da2f60 0x55a773e22dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd79a7fc700 0 client.5112 ms_handle_reset on v2:172.21.15.119:6834/3785845701 2023-10-08T00:56:46.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 -- 172.21.15.119:0/621900672 >> 172.21.15.119:0/621900672 conn(0x55a773d7b370 msgr2=0x55a773dd9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:46.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 -- 172.21.15.119:0/621900672 shutdown_connections 2023-10-08T00:56:46.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 1 -- 172.21.15.119:0/621900672 wait complete. 2023-10-08T00:56:46.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7d1134b80 20 client.5112 trim_cache size 0 max 16384 2023-10-08T00:56:46.543 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T00:56:46.543 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T00:56:46.543 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T00:56:46.543 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4938, 2023-10-08T00:56:46.543 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T00:56:46.544 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T00:56:46.544 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T00:56:46.544 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.397400266, 2023-10-08T00:56:46.544 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 16, 2023-10-08T00:56:46.544 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 31, 2023-10-08T00:56:46.544 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 31, 2023-10-08T00:56:46.544 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 15.228154988 2023-10-08T00:56:46.544 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T00:56:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7c29ff700 1 -- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7a8066f30 msgr2=0x7fd7a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7c29ff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7a8066f30 0x7fd7a80693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0001ed0 tx=0x7fd7b0009040 comp rx=0 tx=0).stop 2023-10-08T00:56:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7c29ff700 1 -- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7bc0afc40 msgr2=0x7fd7bc16eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.532+0000 7fd7c29ff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7bc0afc40 0x7fd7bc16eb20 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd7ac0030c0 tx=0x7fd7ac017000 comp rx=0 tx=0).stop 2023-10-08T00:56:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.533+0000 7fd7c29ff700 1 -- 172.21.15.119:0/2236900151 shutdown_connections 2023-10-08T00:56:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.533+0000 7fd7c29ff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd7a8066f30 0x7fd7a80693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.533+0000 7fd7c29ff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7bc0b2140 0x7fd7bc15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.533+0000 7fd7c29ff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7bc0afc40 0x7fd7bc16eb20 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.533+0000 7fd7c29ff700 1 --2- 172.21.15.119:0/2236900151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7bc0b0010 0x7fd7bc1528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.533+0000 7fd7c29ff700 1 -- 172.21.15.119:0/2236900151 >> 172.21.15.119:0/2236900151 conn(0x7fd7bc06a320 msgr2=0x7fd7bc15a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.533+0000 7fd7c29ff700 1 -- 172.21.15.119:0/2236900151 shutdown_connections 2023-10-08T00:56:46.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.533+0000 7fd7c29ff700 1 -- 172.21.15.119:0/2236900151 wait complete. 2023-10-08T00:56:46.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T00:56:46.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.731+0000 7fc5601ff700 1 Processor -- start 2023-10-08T00:56:46.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.731+0000 7fc5601ff700 1 -- start start 2023-10-08T00:56:46.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.732+0000 7fc5601ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.732+0000 7fc5601ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55815d7e0 con 0x7fc55815df50 2023-10-08T00:56:46.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc5601ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55815dac0 con 0x7fc55806c0a0 2023-10-08T00:56:46.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc5601ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55815aa30 con 0x7fc55806bcd0 2023-10-08T00:56:46.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc55e79c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc55815df50 0x7fc5580c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37436/0 (socket says 172.21.15.119:37436) 2023-10-08T00:56:46.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc55e79c700 1 -- 172.21.15.119:0/2919593138 learned_addr learned my addr 172.21.15.119:0/2919593138 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:46.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc55df9b700 1 --2- 172.21.15.119:0/2919593138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc55cf99700 1 -- 172.21.15.119:0/2919593138 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139555453 0 0) 0x7fc55815d7e0 con 0x7fc55815df50 2023-10-08T00:56:46.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc55cf99700 1 -- 172.21.15.119:0/2919593138 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc544003570 con 0x7fc55815df50 2023-10-08T00:56:46.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc55df9b700 1 -- 172.21.15.119:0/2919593138 >> v1:172.21.15.184:6789/0 conn(0x7fc55806c0a0 legacy=0x7fc55815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc55df9b700 1 -- 172.21.15.119:0/2919593138 >> v1:172.21.15.119:6789/0 conn(0x7fc55815df50 legacy=0x7fc5580c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.733+0000 7fc55df9b700 1 -- 172.21.15.119:0/2919593138 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55815ad10 con 0x7fc55806bcd0 2023-10-08T00:56:46.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.734+0000 7fc55df9b700 1 --2- 172.21.15.119:0/2919593138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580caa90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc554009910 tx=0x7fc5540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f469d9e1d11b6e2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.734+0000 7fc55cf99700 1 -- 172.21.15.119:0/2919593138 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55400b680 con 0x7fc55806bcd0 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.734+0000 7fc55cf99700 1 -- 172.21.15.119:0/2919593138 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55400b7e0 con 0x7fc55806bcd0 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.734+0000 7fc55cf99700 1 -- 172.21.15.119:0/2919593138 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55400bba0 con 0x7fc55806bcd0 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.735+0000 7fc5601ff700 1 -- 172.21.15.119:0/2919593138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 msgr2=0x7fc5580caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.735+0000 7fc5601ff700 1 --2- 172.21.15.119:0/2919593138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580caa90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc554009910 tx=0x7fc5540091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.735+0000 7fc5601ff700 1 -- 172.21.15.119:0/2919593138 shutdown_connections 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.735+0000 7fc5601ff700 1 --2- 172.21.15.119:0/2919593138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580caa90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.735+0000 7fc5601ff700 1 -- 172.21.15.119:0/2919593138 >> 172.21.15.119:0/2919593138 conn(0x7fc5580ce600 msgr2=0x7fc5580cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.735+0000 7fc5601ff700 1 -- 172.21.15.119:0/2919593138 shutdown_connections 2023-10-08T00:56:46.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.735+0000 7fc5601ff700 1 -- 172.21.15.119:0/2919593138 wait complete. 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.736+0000 7fc5601ff700 1 Processor -- start 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.736+0000 7fc5601ff700 1 -- start start 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc5601ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc5601ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55806c0a0 0x7fc55816c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc5601ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc55815df50 0x7fc55816cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc5601ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc558158950 con 0x7fc55815df50 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc5601ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55816d1b0 con 0x7fc55806c0a0 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc5601ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5580b40a0 con 0x7fc55806bcd0 2023-10-08T00:56:46.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc55e79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc55815df50 0x7fc55816cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc55e79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc55815df50 0x7fc55816cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38500/0 (socket says 172.21.15.119:38500) 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc55e79c700 1 -- 172.21.15.119:0/3065878336 learned_addr learned my addr 172.21.15.119:0/3065878336 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc55df9b700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.737+0000 7fc55d79a700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55806c0a0 0x7fc55816c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc55e79c700 1 -- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 msgr2=0x7fc5580b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc55e79c700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc55e79c700 1 -- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55806c0a0 msgr2=0x7fc55816c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc55e79c700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55806c0a0 0x7fc55816c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc55e79c700 1 -- 172.21.15.119:0/3065878336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc554009420 con 0x7fc55815df50 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc55d79a700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55806c0a0 0x7fc55816c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc55e79c700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc55815df50 0x7fc55816cc70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc550001460 tx=0x7fc550001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc54dffb700 1 -- 172.21.15.119:0/3065878336 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55000a940 con 0x7fc55815df50 2023-10-08T00:56:46.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc54dffb700 1 -- 172.21.15.119:0/3065878336 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55000aaa0 con 0x7fc55815df50 2023-10-08T00:56:46.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5580b4380 con 0x7fc55815df50 2023-10-08T00:56:46.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc54dffb700 1 -- 172.21.15.119:0/3065878336 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55000ae20 con 0x7fc55815df50 2023-10-08T00:56:46.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.738+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5581585e0 con 0x7fc55815df50 2023-10-08T00:56:46.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.739+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc558158b00 con 0x7fc55815df50 2023-10-08T00:56:46.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.740+0000 7fc54dffb700 1 -- 172.21.15.119:0/3065878336 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc550007300 con 0x7fc55815df50 2023-10-08T00:56:46.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.741+0000 7fc54dffb700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc53c066f30 0x7fc53c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:46.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.741+0000 7fc54dffb700 1 -- 172.21.15.119:0/3065878336 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fc5500849e0 con 0x7fc55815df50 2023-10-08T00:56:46.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.742+0000 7fc55df9b700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc53c066f30 0x7fc53c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:46.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.742+0000 7fc55df9b700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc53c066f30 0x7fc53c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc5540093f0 tx=0x7fc554017000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:46.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.745+0000 7fc54dffb700 1 -- 172.21.15.119:0/3065878336 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc558158b00 con 0x7fc55815df50 2023-10-08T00:56:46.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.977+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc55804c3c0 con 0x7fc55815df50 2023-10-08T00:56:46.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.978+0000 7fc54dffb700 1 -- 172.21.15.119:0/3065878336 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc550051c30 con 0x7fc55815df50 2023-10-08T00:56:46.979 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T00:56:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc53c066f30 msgr2=0x7fc53c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc53c066f30 0x7fc53c0693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fc5540093f0 tx=0x7fc554017000 comp rx=0 tx=0).stop 2023-10-08T00:56:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc55815df50 msgr2=0x7fc55816cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc55815df50 0x7fc55816cc70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc550001460 tx=0x7fc550001ae0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 shutdown_connections 2023-10-08T00:56:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc55806bcd0 0x7fc5580b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc53c066f30 0x7fc53c0693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc55815df50 0x7fc55816cc70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 --2- 172.21.15.119:0/3065878336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc55806c0a0 0x7fc55816c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.980+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 >> 172.21.15.119:0/3065878336 conn(0x7fc5580ce600 msgr2=0x7fc558157530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.981+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 shutdown_connections 2023-10-08T00:56:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:46.981+0000 7fc5601ff700 1 -- 172.21.15.119:0/3065878336 wait complete. 2023-10-08T00:56:46.999 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T00:56:46.999 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T00:56:46.999 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T00:56:46.999 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T00:56:47.000 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T00:56:47.000 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T00:56:47.000 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T00:56:47.000 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T00:56:47.000 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:56:47.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:47.000 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T00:56:47.037 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T00:56:47.037 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T00:56:47.037 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T00:56:47.038 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42244sec preferred_lft 42244sec 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T00:56:47.039 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T00:56:47.040 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T00:56:47.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:47.041 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T00:56:47.138 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T00:56:47.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:47.141 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T00:56:47.181 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T00:56:47.181 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T00:56:47.181 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T00:56:47.181 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T00:56:47.181 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T00:56:47.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:47.181 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T00:56:47.182 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T00:56:47.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T00:56:47.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T00:56:47.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T00:56:47.182 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T00:56:47.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:47.375 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T00:56:47.411 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T00:56:47.412 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T00:56:47.412 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:56:47.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:47.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T00:56:47.513 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:47.513 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T00:56:47.549 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T00:56:47.549 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T00:56:47.549 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:56:47.624 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T00:56:47.625 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T00:56:47.625 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:47.755 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:47.755 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:47.783 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T00:56:47.784 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T00:56:47.784 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T00:56:47.784 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T00:56:47.784 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T00:56:47.790 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T00:56:47.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T00:56:47.790 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T00:56:47.851 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T00:56:48.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.032+0000 7f872afff700 1 Processor -- start 2023-10-08T00:56:48.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.033+0000 7f872afff700 1 -- start start 2023-10-08T00:56:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f872afff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240b1420 0x7f87240ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f872afff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87240ccf90 0x7f87240c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f872afff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240cc300 con 0x7f87240cbf30 2023-10-08T00:56:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f872afff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8724156bf0 con 0x7f87240b1420 2023-10-08T00:56:48.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f872afff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240ae950 con 0x7f87240ccf90 2023-10-08T00:56:48.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f8723fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240b1420 0x7f87240ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f8723fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240b1420 0x7f87240ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38504/0 (socket says 172.21.15.119:38504) 2023-10-08T00:56:48.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f8723fff700 1 -- 172.21.15.119:0/2269247657 learned_addr learned my addr 172.21.15.119:0/2269247657 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:48.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.034+0000 7f872959c700 1 --2- 172.21.15.119:0/2269247657 >> v2:172.21.15.184:3300/0 conn(0x7f87240ccf90 0x7f87240c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.035+0000 7f8723fff700 1 -- 172.21.15.119:0/2269247657 >> v2:172.21.15.184:3300/0 conn(0x7f87240ccf90 msgr2=0x7f87240c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:48.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.035+0000 7f8723fff700 1 --2- 172.21.15.119:0/2269247657 >> v2:172.21.15.184:3300/0 conn(0x7f87240ccf90 0x7f87240c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.035+0000 7f8723fff700 1 -- 172.21.15.119:0/2269247657 >> v1:172.21.15.184:6789/0 conn(0x7f87240cbf30 legacy=0x7f87240b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:48.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.035+0000 7f8723fff700 1 -- 172.21.15.119:0/2269247657 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240ad370 con 0x7f87240b1420 2023-10-08T00:56:48.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.035+0000 7f8723fff700 1 --2- 172.21.15.119:0/2269247657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 0x7f87240ae410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f87240c8fd0 tx=0x7f8710009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca15bb910946cab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.035+0000 7f87237fe700 1 -- 172.21.15.119:0/2269247657 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871000b7e0 con 0x7f87240b1420 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.035+0000 7f87237fe700 1 -- 172.21.15.119:0/2269247657 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871000b940 con 0x7f87240b1420 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.035+0000 7f87237fe700 1 -- 172.21.15.119:0/2269247657 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871000bce0 con 0x7f87240b1420 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.036+0000 7f872afff700 1 -- 172.21.15.119:0/2269247657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 msgr2=0x7f87240ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.036+0000 7f872afff700 1 --2- 172.21.15.119:0/2269247657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 0x7f87240ae410 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f87240c8fd0 tx=0x7f8710009250 comp rx=0 tx=0).stop 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.036+0000 7f872afff700 1 -- 172.21.15.119:0/2269247657 shutdown_connections 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.036+0000 7f872afff700 1 --2- 172.21.15.119:0/2269247657 >> v2:172.21.15.184:3300/0 conn(0x7f87240ccf90 0x7f87240c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.036+0000 7f872afff700 1 --2- 172.21.15.119:0/2269247657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 0x7f87240ae410 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.036+0000 7f872afff700 1 -- 172.21.15.119:0/2269247657 >> 172.21.15.119:0/2269247657 conn(0x7f872406a320 msgr2=0x7f87240cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.036+0000 7f872afff700 1 -- 172.21.15.119:0/2269247657 shutdown_connections 2023-10-08T00:56:48.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.036+0000 7f872afff700 1 -- 172.21.15.119:0/2269247657 wait complete. 2023-10-08T00:56:48.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.037+0000 7f872afff700 1 Processor -- start 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.037+0000 7f872afff700 1 -- start start 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.038+0000 7f872afff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240b1420 0x7f87240aa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.038+0000 7f872afff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.038+0000 7f872afff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240ccf90 0x7f87240abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.038+0000 7f872afff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240aad40 con 0x7f87240cbf30 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.038+0000 7f872afff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240ab020 con 0x7f87240b1420 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.038+0000 7f872afff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240ab300 con 0x7f87240ccf90 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38516/0 (socket says 172.21.15.119:38516) 2023-10-08T00:56:48.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 -- 172.21.15.119:0/3117622335 learned_addr learned my addr 172.21.15.119:0/3117622335 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8728d9b700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240b1420 0x7f87240aa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f872959c700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240ccf90 0x7f87240abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 -- 172.21.15.119:0/3117622335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240ccf90 msgr2=0x7f87240abde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240ccf90 0x7f87240abde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 -- 172.21.15.119:0/3117622335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240b1420 msgr2=0x7f87240aa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240b1420 0x7f87240aa410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 -- 172.21.15.119:0/3117622335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87100094f0 con 0x7f87240cbf30 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8723fff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240a54c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8710009290 tx=0x7f8710016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8720ff9700 1 -- 172.21.15.119:0/3117622335 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710011d10 con 0x7f87240cbf30 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8720ff9700 1 -- 172.21.15.119:0/3117622335 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8710020070 con 0x7f87240cbf30 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87240ab5e0 con 0x7f87240cbf30 2023-10-08T00:56:48.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.039+0000 7f8720ff9700 1 -- 172.21.15.119:0/3117622335 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871001b6a0 con 0x7f87240cbf30 2023-10-08T00:56:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.040+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8724169f00 con 0x7f87240cbf30 2023-10-08T00:56:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.040+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87240abb00 con 0x7f87240cbf30 2023-10-08T00:56:48.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.041+0000 7f8720ff9700 1 -- 172.21.15.119:0/3117622335 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8710022030 con 0x7f87240cbf30 2023-10-08T00:56:48.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.042+0000 7f8720ff9700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.042+0000 7f8720ff9700 1 -- 172.21.15.119:0/3117622335 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f87100863c0 con 0x7f87240cbf30 2023-10-08T00:56:48.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.042+0000 7f8728d9b700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.043+0000 7f8728d9b700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8708066f30 0x7f87080693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8718003870 tx=0x7f8718009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:48.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.047+0000 7f8720ff9700 1 -- 172.21.15.119:0/3117622335 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87240abb00 con 0x7f87240cbf30 2023-10-08T00:56:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.272+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f87240abb00 con 0x7f8708066f30 2023-10-08T00:56:48.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.274+0000 7f8720ff9700 1 -- 172.21.15.119:0/3117622335 <== mgr.4101 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f87240abb00 con 0x7f8708066f30 2023-10-08T00:56:48.276 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T00:56:48.276 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T00:56:48.276 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T00:56:48.276 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T00:56:48.276 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T00:56:48.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8708066f30 msgr2=0x7f87080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:48.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8708066f30 0x7f87080693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8718003870 tx=0x7f8718009040 comp rx=0 tx=0).stop 2023-10-08T00:56:48.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 msgr2=0x7f87240a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240a54c0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8710009290 tx=0x7f8710016000 comp rx=0 tx=0).stop 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 shutdown_connections 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240ccf90 0x7f87240abde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240b1420 0x7f87240aa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 --2- 172.21.15.119:0/3117622335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240a54c0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 >> 172.21.15.119:0/3117622335 conn(0x7f872406a320 msgr2=0x7f87240c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 shutdown_connections 2023-10-08T00:56:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.276+0000 7f872afff700 1 -- 172.21.15.119:0/3117622335 wait complete. 2023-10-08T00:56:48.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T00:56:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.477+0000 7fdfc09fe700 1 Processor -- start 2023-10-08T00:56:48.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.477+0000 7fdfc09fe700 1 -- start start 2023-10-08T00:56:48.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfc09fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdfbc15a3b0 0x7fdfbc15a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfc09fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc0aef00 0x7fdfbc15d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfc09fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfbc06e470 0x7fdfbc06e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfc09fe700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0aea70 con 0x7fdfbc06e470 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfc09fe700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc06ee00 con 0x7fdfbc15a3b0 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfc09fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0ae480 con 0x7fdfbc0aef00 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfbb59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfbc06e470 0x7fdfbc06e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfbb59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfbc06e470 0x7fdfbc06e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38524/0 (socket says 172.21.15.119:38524) 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.478+0000 7fdfbb59e700 1 -- 172.21.15.119:0/2240491287 learned_addr learned my addr 172.21.15.119:0/2240491287 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfba59c700 1 --2- 172.21.15.119:0/2240491287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc0aef00 0x7fdfbc15d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfbad9d700 1 --2- 172.21.15.119:0/2240491287 >> v2:172.21.15.184:3300/0 conn(0x7fdfbc15a3b0 0x7fdfbc15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfbb59e700 1 -- 172.21.15.119:0/2240491287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc0aef00 msgr2=0x7fdfbc15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfbb59e700 1 --2- 172.21.15.119:0/2240491287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc0aef00 0x7fdfbc15d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfbb59e700 1 -- 172.21.15.119:0/2240491287 >> v2:172.21.15.184:3300/0 conn(0x7fdfbc15a3b0 msgr2=0x7fdfbc15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfbb59e700 1 --2- 172.21.15.119:0/2240491287 >> v2:172.21.15.184:3300/0 conn(0x7fdfbc15a3b0 0x7fdfbc15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfbb59e700 1 -- 172.21.15.119:0/2240491287 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc15acc0 con 0x7fdfbc06e470 2023-10-08T00:56:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfbb59e700 1 --2- 172.21.15.119:0/2240491287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc06e8c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc063350 tx=0x7fdfb4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca3cd6f997b1800 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfb9d9b700 1 -- 172.21.15.119:0/2240491287 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb400e070 con 0x7fdfbc06e470 2023-10-08T00:56:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfb9d9b700 1 -- 172.21.15.119:0/2240491287 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb400ba50 con 0x7fdfbc06e470 2023-10-08T00:56:48.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.479+0000 7fdfb9d9b700 1 -- 172.21.15.119:0/2240491287 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40092e0 con 0x7fdfbc06e470 2023-10-08T00:56:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2240491287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 msgr2=0x7fdfbc06e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2240491287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc06e8c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc063350 tx=0x7fdfb4009840 comp rx=0 tx=0).stop 2023-10-08T00:56:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2240491287 shutdown_connections 2023-10-08T00:56:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2240491287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc06e8c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2240491287 >> v2:172.21.15.184:3300/0 conn(0x7fdfbc15a3b0 0x7fdfbc15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2240491287 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc0aef00 0x7fdfbc15d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2240491287 >> 172.21.15.119:0/2240491287 conn(0x7fdfbc0c9f70 msgr2=0x7fdfbc06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:48.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2240491287 shutdown_connections 2023-10-08T00:56:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.480+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2240491287 wait complete. 2023-10-08T00:56:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.481+0000 7fdfc09fe700 1 Processor -- start 2023-10-08T00:56:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.482+0000 7fdfc09fe700 1 -- start start 2023-10-08T00:56:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.482+0000 7fdfc09fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.482+0000 7fdfc09fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0aef00 0x7fdfbc1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.482+0000 7fdfc09fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc15a3b0 0x7fdfbc156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.482+0000 7fdfc09fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc159090 con 0x7fdfbc06e470 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.482+0000 7fdfc09fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0b2490 con 0x7fdfbc0aef00 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.482+0000 7fdfc09fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc159a60 con 0x7fdfbc15a3b0 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38538/0 (socket says 172.21.15.119:38538) 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 -- 172.21.15.119:0/2601299158 learned_addr learned my addr 172.21.15.119:0/2601299158 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfba59c700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0aef00 0x7fdfbc1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbb59e700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc15a3b0 0x7fdfbc156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 -- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc15a3b0 msgr2=0x7fdfbc156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc15a3b0 0x7fdfbc156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 -- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0aef00 msgr2=0x7fdfbc1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0aef00 0x7fdfbc1535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 -- 172.21.15.119:0/2601299158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb4009ae0 con 0x7fdfbc06e470 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfba59c700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0aef00 0x7fdfbc1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbb59e700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc15a3b0 0x7fdfbc156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfbad9d700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc159520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d890 tx=0x7fdfac008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfa2ffd700 1 -- 172.21.15.119:0/2601299158 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac009610 con 0x7fdfbc06e470 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfa2ffd700 1 -- 172.21.15.119:0/2601299158 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac009770 con 0x7fdfbc06e470 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc154860 con 0x7fdfbc06e470 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.483+0000 7fdfa2ffd700 1 -- 172.21.15.119:0/2601299158 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac009b30 con 0x7fdfbc06e470 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.484+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc158760 con 0x7fdfbc06e470 2023-10-08T00:56:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.485+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc159370 con 0x7fdfbc06e470 2023-10-08T00:56:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.485+0000 7fdfa2ffd700 1 -- 172.21.15.119:0/2601299158 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfac01e460 con 0x7fdfbc06e470 2023-10-08T00:56:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.486+0000 7fdfa2ffd700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.486+0000 7fdfa2ffd700 1 -- 172.21.15.119:0/2601299158 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fdfac084c40 con 0x7fdfbc06e470 2023-10-08T00:56:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.487+0000 7fdfba59c700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:48.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.487+0000 7fdfba59c700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0ca440 tx=0x7fdfb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:48.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.490+0000 7fdfa2ffd700 1 -- 172.21.15.119:0/2601299158 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfac051e10 con 0x7fdfbc06e470 2023-10-08T00:56:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:48.731+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fdfbc15dd30 con 0x7fdfbc06e470 2023-10-08T00:56:49.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.183+0000 7fdfa2ffd700 1 -- 172.21.15.119:0/2601299158 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 6) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fdfac0521e0 con 0x7fdfbc06e470 2023-10-08T00:56:49.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.185+0000 7fdfa2ffd700 1 -- 172.21.15.119:0/2601299158 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7fdfac051e10 con 0x7fdfbc06e470 2023-10-08T00:56:49.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.187+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdfa8066f30 msgr2=0x7fdfa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.187+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0ca440 tx=0x7fdfb0008040 comp rx=0 tx=0).stop 2023-10-08T00:56:49.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.187+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 msgr2=0x7fdfbc159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.187+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc159520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00d890 tx=0x7fdfac008040 comp rx=0 tx=0).stop 2023-10-08T00:56:49.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.187+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 shutdown_connections 2023-10-08T00:56:49.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.187+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc15a3b0 0x7fdfbc156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.188+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.188+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc06e470 0x7fdfbc159520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.188+0000 7fdfc09fe700 1 --2- 172.21.15.119:0/2601299158 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0aef00 0x7fdfbc1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.188+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 >> 172.21.15.119:0/2601299158 conn(0x7fdfbc0c9f70 msgr2=0x7fdfbc15d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:49.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.188+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 shutdown_connections 2023-10-08T00:56:49.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.188+0000 7fdfc09fe700 1 -- 172.21.15.119:0/2601299158 wait complete. 2023-10-08T00:56:49.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T00:56:49.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.399+0000 7f2d0759e700 1 Processor -- start 2023-10-08T00:56:49.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.399+0000 7f2d0759e700 1 -- start start 2023-10-08T00:56:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d0759e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2d080b16d0 0x7f2d080b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d0759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d080b33b0 0x7f2d080b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d0759e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d080b2080 con 0x7f2d080b4e70 2023-10-08T00:56:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d0759e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080cfb00 con 0x7f2d080b33b0 2023-10-08T00:56:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d0759e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d080b5c40 con 0x7f2d080b16d0 2023-10-08T00:56:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d06d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d080b33b0 0x7f2d080b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d06d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d080b33b0 0x7f2d080b37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38576/0 (socket says 172.21.15.119:38576) 2023-10-08T00:56:49.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d06d9d700 1 -- 172.21.15.119:0/1892837868 learned_addr learned my addr 172.21.15.119:0/1892837868 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:49.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d0659c700 1 --1- 172.21.15.119:0/1892837868 >> v1:172.21.15.119:6789/0 conn(0x7f2d080b4e70 0x7f2d080ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37446/0 (socket says 172.21.15.119:37446) 2023-10-08T00:56:49.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d05d9b700 1 --2- 172.21.15.119:0/1892837868 >> v2:172.21.15.184:3300/0 conn(0x7f2d080b16d0 0x7f2d080b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d06d9d700 1 -- 172.21.15.119:0/1892837868 >> v2:172.21.15.184:3300/0 conn(0x7f2d080b16d0 msgr2=0x7f2d080b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:49.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d06d9d700 1 --2- 172.21.15.119:0/1892837868 >> v2:172.21.15.184:3300/0 conn(0x7f2d080b16d0 0x7f2d080b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d06d9d700 1 -- 172.21.15.119:0/1892837868 >> v1:172.21.15.119:6789/0 conn(0x7f2d080b4e70 legacy=0x7f2d080ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.401+0000 7f2d06d9d700 1 -- 172.21.15.119:0/1892837868 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080b5f20 con 0x7f2d080b33b0 2023-10-08T00:56:49.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.402+0000 7f2d06d9d700 1 --2- 172.21.15.119:0/1892837868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b33b0 0x7f2d080b37d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d08089830 tx=0x7f2d00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1f1ef72ddb1cbc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:49.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.402+0000 7f2d0559a700 1 -- 172.21.15.119:0/1892837868 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0000e070 con 0x7f2d080b33b0 2023-10-08T00:56:49.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.402+0000 7f2d0559a700 1 -- 172.21.15.119:0/1892837868 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d0000ba20 con 0x7f2d080b33b0 2023-10-08T00:56:49.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.402+0000 7f2d0559a700 1 -- 172.21.15.119:0/1892837868 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d000092e0 con 0x7f2d080b33b0 2023-10-08T00:56:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.402+0000 7f2d0759e700 1 -- 172.21.15.119:0/1892837868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b33b0 msgr2=0x7f2d080b37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.402+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1892837868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b33b0 0x7f2d080b37d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2d08089830 tx=0x7f2d00009840 comp rx=0 tx=0).stop 2023-10-08T00:56:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.403+0000 7f2d0759e700 1 -- 172.21.15.119:0/1892837868 shutdown_connections 2023-10-08T00:56:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.403+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1892837868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b33b0 0x7f2d080b37d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.403+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1892837868 >> v2:172.21.15.184:3300/0 conn(0x7f2d080b16d0 0x7f2d080b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.403+0000 7f2d0759e700 1 -- 172.21.15.119:0/1892837868 >> 172.21.15.119:0/1892837868 conn(0x7f2d080d10c0 msgr2=0x7f2d080affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.403+0000 7f2d0759e700 1 -- 172.21.15.119:0/1892837868 shutdown_connections 2023-10-08T00:56:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.403+0000 7f2d0759e700 1 -- 172.21.15.119:0/1892837868 wait complete. 2023-10-08T00:56:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.404+0000 7f2d0759e700 1 Processor -- start 2023-10-08T00:56:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.404+0000 7f2d0759e700 1 -- start start 2023-10-08T00:56:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b16d0 0x7f2d08158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080b33b0 0x7f2d080a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0759e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d080b4e70 0x7f2d080a79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0759e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080c7b20 con 0x7f2d080b16d0 2023-10-08T00:56:49.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0759e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080cbb00 con 0x7f2d080b4e70 2023-10-08T00:56:49.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d080cb260 con 0x7f2d080b33b0 2023-10-08T00:56:49.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b16d0 0x7f2d08158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b16d0 0x7f2d08158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38590/0 (socket says 172.21.15.119:38590) 2023-10-08T00:56:49.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.405+0000 7f2d0659c700 1 -- 172.21.15.119:0/1292547072 learned_addr learned my addr 172.21.15.119:0/1292547072 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:49.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d05d9b700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080b33b0 0x7f2d080a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d06d9d700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d080b4e70 0x7f2d080a79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d0659c700 1 -- 172.21.15.119:0/1292547072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080b33b0 msgr2=0x7f2d080a6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d0659c700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080b33b0 0x7f2d080a6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d0659c700 1 -- 172.21.15.119:0/1292547072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d080b4e70 msgr2=0x7f2d080a79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d0659c700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d080b4e70 0x7f2d080a79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d0659c700 1 -- 172.21.15.119:0/1292547072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00009ae0 con 0x7f2d080b16d0 2023-10-08T00:56:49.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d0659c700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b16d0 0x7f2d08158210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c7c0 tx=0x7f2cf8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:49.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2cf67fc700 1 -- 172.21.15.119:0/1292547072 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8005a70 con 0x7f2d080b16d0 2023-10-08T00:56:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2cf67fc700 1 -- 172.21.15.119:0/1292547072 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf8005bd0 con 0x7f2d080b16d0 2023-10-08T00:56:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d080cb540 con 0x7f2d080b16d0 2023-10-08T00:56:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.406+0000 7f2cf67fc700 1 -- 172.21.15.119:0/1292547072 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80070d0 con 0x7f2d080b16d0 2023-10-08T00:56:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.407+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d08158750 con 0x7f2d080b16d0 2023-10-08T00:56:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.407+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d08088990 con 0x7f2d080b16d0 2023-10-08T00:56:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.408+0000 7f2cf67fc700 1 -- 172.21.15.119:0/1292547072 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 6) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f2cf8007480 con 0x7f2d080b16d0 2023-10-08T00:56:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.409+0000 7f2cf67fc700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ce8066f80 0x7f2ce8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.409+0000 7f2cf67fc700 1 -- 172.21.15.119:0/1292547072 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f2cf80839a0 con 0x7f2d080b16d0 2023-10-08T00:56:49.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.410+0000 7f2d05d9b700 1 -- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ce8066f80 msgr2=0x7f2ce8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:49.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.410+0000 7f2d05d9b700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ce8066f80 0x7f2ce8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T00:56:49.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.417+0000 7f2cf67fc700 1 -- 172.21.15.119:0/1292547072 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2cf8050c10 con 0x7f2d080b16d0 2023-10-08T00:56:49.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.611+0000 7f2d05d9b700 1 -- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ce8066f80 msgr2=0x7f2ce8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:49.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.611+0000 7f2d05d9b700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ce8066f80 0x7f2ce8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T00:56:49.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.620+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f2d0815cf60 con 0x7f2d080b16d0 2023-10-08T00:56:49.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.624+0000 7f2cf67fc700 1 -- 172.21.15.119:0/1292547072 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2cf8017020 con 0x7f2d080b16d0 2023-10-08T00:56:49.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ce8066f80 msgr2=0x7f2ce8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:56:49.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ce8066f80 0x7f2ce8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b16d0 msgr2=0x7f2d08158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b16d0 0x7f2d08158210 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2cf800c7c0 tx=0x7f2cf8007f70 comp rx=0 tx=0).stop 2023-10-08T00:56:49.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 shutdown_connections 2023-10-08T00:56:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ce8066f80 0x7f2ce8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d080b4e70 0x7f2d080a79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d080b16d0 0x7f2d08158210 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 --2- 172.21.15.119:0/1292547072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d080b33b0 0x7f2d080a6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 >> 172.21.15.119:0/1292547072 conn(0x7f2d080d10c0 msgr2=0x7f2d08158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 shutdown_connections 2023-10-08T00:56:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.627+0000 7f2d0759e700 1 -- 172.21.15.119:0/1292547072 wait complete. 2023-10-08T00:56:49.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T00:56:49.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.833+0000 7f78b239d700 1 Processor -- start 2023-10-08T00:56:49.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.833+0000 7f78b239d700 1 -- start start 2023-10-08T00:56:49.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b239d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac0ccc00 0x7f78ac069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b239d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f78ac06a020 0x7f78ac0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b239d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78ac0c9f30 con 0x7f78ac159a60 2023-10-08T00:56:49.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b239d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78ac06dfc0 con 0x7f78ac06a020 2023-10-08T00:56:49.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b239d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0ca470 con 0x7f78ac0ccc00 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b1b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f78ac06a020 0x7f78ac0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b1b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f78ac06a020 0x7f78ac0cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38596/0 (socket says 172.21.15.119:38596) 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b1b9c700 1 -- 172.21.15.119:0/3733107020 learned_addr learned my addr 172.21.15.119:0/3733107020 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b0b9a700 1 --2- 172.21.15.119:0/3733107020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac0ccc00 0x7f78ac069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78a3fff700 1 -- 172.21.15.119:0/3733107020 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3610570677 0 0) 0x7f78ac0c9f30 con 0x7f78ac159a60 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b1b9c700 1 -- 172.21.15.119:0/3733107020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac0ccc00 msgr2=0x7f78ac069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b1b9c700 1 --2- 172.21.15.119:0/3733107020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac0ccc00 0x7f78ac069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.835+0000 7f78b1b9c700 1 -- 172.21.15.119:0/3733107020 >> v1:172.21.15.184:6789/0 conn(0x7f78ac159a60 legacy=0x7f78ac0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.836+0000 7f78b1b9c700 1 -- 172.21.15.119:0/3733107020 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac0ca750 con 0x7f78ac06a020 2023-10-08T00:56:49.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.836+0000 7f78b0b9a700 1 --2- 172.21.15.119:0/3733107020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac0ccc00 0x7f78ac069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:49.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.836+0000 7f78b1b9c700 1 --2- 172.21.15.119:0/3733107020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac06a020 0x7f78ac0cb230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009f20 tx=0x7f78a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a27c44329210d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:49.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.836+0000 7f78a3fff700 1 -- 172.21.15.119:0/3733107020 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a400e070 con 0x7f78ac06a020 2023-10-08T00:56:49.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.836+0000 7f78a3fff700 1 -- 172.21.15.119:0/3733107020 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a400bb10 con 0x7f78ac06a020 2023-10-08T00:56:49.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.836+0000 7f78a3fff700 1 -- 172.21.15.119:0/3733107020 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a40092e0 con 0x7f78ac06a020 2023-10-08T00:56:49.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.836+0000 7f78b239d700 1 -- 172.21.15.119:0/3733107020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac06a020 msgr2=0x7f78ac0cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.836+0000 7f78b239d700 1 --2- 172.21.15.119:0/3733107020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac06a020 0x7f78ac0cb230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009f20 tx=0x7f78a4009840 comp rx=0 tx=0).stop 2023-10-08T00:56:49.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.837+0000 7f78b239d700 1 -- 172.21.15.119:0/3733107020 shutdown_connections 2023-10-08T00:56:49.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.837+0000 7f78b239d700 1 --2- 172.21.15.119:0/3733107020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac06a020 0x7f78ac0cb230 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.837+0000 7f78b239d700 1 --2- 172.21.15.119:0/3733107020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac0ccc00 0x7f78ac069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.837+0000 7f78b239d700 1 -- 172.21.15.119:0/3733107020 >> 172.21.15.119:0/3733107020 conn(0x7f78ac06baa0 msgr2=0x7f78ac06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:49.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.837+0000 7f78b239d700 1 -- 172.21.15.119:0/3733107020 shutdown_connections 2023-10-08T00:56:49.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.837+0000 7f78b239d700 1 -- 172.21.15.119:0/3733107020 wait complete. 2023-10-08T00:56:49.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.838+0000 7f78b239d700 1 Processor -- start 2023-10-08T00:56:49.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.838+0000 7f78b239d700 1 -- start start 2023-10-08T00:56:49.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b239d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac06a020 0x7f78ac0b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b239d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac0ccc00 0x7f78ac1546e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b239d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac159a60 0x7f78ac0b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b239d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0b3e80 con 0x7f78ac0ccc00 2023-10-08T00:56:49.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b239d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac154c20 con 0x7f78ac159a60 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b239d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78ac0b2780 con 0x7f78ac06a020 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b0b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac0ccc00 0x7f78ac1546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b0b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac0ccc00 0x7f78ac1546e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38610/0 (socket says 172.21.15.119:38610) 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b0b9a700 1 -- 172.21.15.119:0/122762713 learned_addr learned my addr 172.21.15.119:0/122762713 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b1b9c700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac159a60 0x7f78ac0b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.839+0000 7f78b139b700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac06a020 0x7f78ac0b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78b0b9a700 1 -- 172.21.15.119:0/122762713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac06a020 msgr2=0x7f78ac0b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78b0b9a700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac06a020 0x7f78ac0b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78b0b9a700 1 -- 172.21.15.119:0/122762713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac159a60 msgr2=0x7f78ac0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:49.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78b0b9a700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac159a60 0x7f78ac0b2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:49.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78b0b9a700 1 -- 172.21.15.119:0/122762713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a4009ae0 con 0x7f78ac0ccc00 2023-10-08T00:56:49.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78b0b9a700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac0ccc00 0x7f78ac1546e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f789c00e8d0 tx=0x7f789c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:49.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78a17fa700 1 -- 172.21.15.119:0/122762713 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c013410 con 0x7f78ac0ccc00 2023-10-08T00:56:49.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78a17fa700 1 -- 172.21.15.119:0/122762713 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789c013570 con 0x7f78ac0ccc00 2023-10-08T00:56:49.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78a17fa700 1 -- 172.21.15.119:0/122762713 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789c013910 con 0x7f78ac0ccc00 2023-10-08T00:56:49.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78ac0b3270 con 0x7f78ac0ccc00 2023-10-08T00:56:49.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.840+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78ac0b3830 con 0x7f78ac0ccc00 2023-10-08T00:56:49.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.841+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78ac06b620 con 0x7f78ac0ccc00 2023-10-08T00:56:49.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.842+0000 7f78a17fa700 1 -- 172.21.15.119:0/122762713 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 6) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f789c013d00 con 0x7f78ac0ccc00 2023-10-08T00:56:49.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.843+0000 7f78a17fa700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7890066f80 0x7f7890069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:49.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.843+0000 7f78a17fa700 1 -- 172.21.15.119:0/122762713 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7f789c0842b0 con 0x7f78ac0ccc00 2023-10-08T00:56:49.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.843+0000 7f78b139b700 1 -- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7890066f80 msgr2=0x7f7890069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:49.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.843+0000 7f78b139b700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7890066f80 0x7f7890069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T00:56:49.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:49.848+0000 7f78a17fa700 1 -- 172.21.15.119:0/122762713 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f789c051520 con 0x7f78ac0ccc00 2023-10-08T00:56:50.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.044+0000 7f78b139b700 1 -- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7890066f80 msgr2=0x7f7890069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:50.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.044+0000 7f78b139b700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7890066f80 0x7f7890069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T00:56:50.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.047+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f78ac0b3fc0 con 0x7f78ac0ccc00 2023-10-08T00:56:50.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.051+0000 7f78a17fa700 1 -- 172.21.15.119:0/122762713 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7f789c05a070 con 0x7f78ac0ccc00 2023-10-08T00:56:50.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7890066f80 msgr2=0x7f7890069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:56:50.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7890066f80 0x7f7890069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac0ccc00 msgr2=0x7f78ac1546e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac0ccc00 0x7f78ac1546e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f789c00e8d0 tx=0x7f789c008040 comp rx=0 tx=0).stop 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 shutdown_connections 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f78ac06a020 0x7f78ac0b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7890066f80 0x7f7890069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f78ac159a60 0x7f78ac0b2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 --2- 172.21.15.119:0/122762713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78ac0ccc00 0x7f78ac1546e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 >> 172.21.15.119:0/122762713 conn(0x7f78ac06baa0 msgr2=0x7f78ac1564b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 shutdown_connections 2023-10-08T00:56:50.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.055+0000 7f78b239d700 1 -- 172.21.15.119:0/122762713 wait complete. 2023-10-08T00:56:50.074 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:50.074 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T00:56:50.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:50.121 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-10-08T00:56:50.204 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-10-08T00:56:50.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.432+0000 7fa1cf59d700 1 Processor -- start 2023-10-08T00:56:50.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.433+0000 7fa1cf59d700 1 -- start start 2023-10-08T00:56:50.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cf59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa1c80b03c0 0x7fa1c80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cf59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c81591e0 0x7fa1c8159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cf59d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1c80cfa00 con 0x7fa1c80b0790 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cf59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1c80cb580 con 0x7fa1c80b03c0 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cf59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c80cf160 con 0x7fa1c81591e0 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cdd9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa1c80b0790 0x7fa1c80b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37448/0 (socket says 172.21.15.119:37448) 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cdd9a700 1 -- 172.21.15.119:0/2232707558 learned_addr learned my addr 172.21.15.119:0/2232707558 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2232707558 >> v2:172.21.15.184:3300/0 conn(0x7fa1c80b03c0 0x7fa1c80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1ced9c700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c81591e0 0x7fa1c8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263119602 0 0) 0x7fa1c80cfa00 con 0x7fa1c80b0790 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.434+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1b4003570 con 0x7fa1c80b0790 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1552432943 0 0) 0x7fa1b4003570 con 0x7fa1c80b0790 2023-10-08T00:56:50.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c81591e0 msgr2=0x7fa1c8159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:50.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c81591e0 0x7fa1c8159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 >> v2:172.21.15.184:3300/0 conn(0x7fa1c80b03c0 msgr2=0x7fa1c80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:50.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 --2- 172.21.15.119:0/2232707558 >> v2:172.21.15.184:3300/0 conn(0x7fa1c80b03c0 0x7fa1c80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c80cf440 con 0x7fa1c80b0790 2023-10-08T00:56:50.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fa1b8001470 con 0x7fa1c80b0790 2023-10-08T00:56:50.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2232707558 >> v2:172.21.15.184:3300/0 conn(0x7fa1c80b03c0 0x7fa1c80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:50.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 >> v1:172.21.15.119:6789/0 conn(0x7fa1c80b0790 legacy=0x7fa1c80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1b4004810 0x7fa1b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c81591e0 0x7fa1b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c80cb580 con 0x7fa1b4003a60 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c80cf160 con 0x7fa1b4004810 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1b400ba60 con 0x7fa1c81591e0 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1b4004810 0x7fa1b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:50.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.435+0000 7fa1ced9c700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c81591e0 0x7fa1b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:50.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1ce59b700 1 -- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c81591e0 msgr2=0x7fa1b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:50.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c81591e0 0x7fa1b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1ce59b700 1 -- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1b4004810 msgr2=0x7fa1b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:50.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1b4004810 0x7fa1b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1ce59b700 1 -- 172.21.15.119:0/2232707558 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1b4003570 con 0x7fa1b4003a60 2023-10-08T00:56:50.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1b4003e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009e20 tx=0x7fa1c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90197d82b2d2d94 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:50.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400e920 con 0x7fa1b4003a60 2023-10-08T00:56:50.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c400ea80 con 0x7fa1b4003a60 2023-10-08T00:56:50.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.436+0000 7fa1cd599700 1 -- 172.21.15.119:0/2232707558 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c400ee20 con 0x7fa1b4003a60 2023-10-08T00:56:50.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.437+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2232707558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 msgr2=0x7fa1b4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:50.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.437+0000 7fa1cf59d700 1 --2- 172.21.15.119:0/2232707558 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1b4003e70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009e20 tx=0x7fa1c40091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.437+0000 7fa1ced9c700 1 -- 172.21.15.119:0/2232707558 reap_dead start 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.437+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2232707558 shutdown_connections 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.437+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2232707558 >> 172.21.15.119:0/2232707558 conn(0x7fa1c8063230 msgr2=0x7fa1c8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.437+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2232707558 shutdown_connections 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.437+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2232707558 wait complete. 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cf59d700 1 Processor -- start 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cf59d700 1 -- start start 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cf59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1c8157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cf59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c815da90 0x7fa1c815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cf59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1c80af610 0x7fa1c8068df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cf59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c815cda0 con 0x7fa1b4003a60 2023-10-08T00:56:50.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cf59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c8157eb0 con 0x7fa1c80af610 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cf59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c815b880 con 0x7fa1c815da90 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1ce59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1c8157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1ce59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1c8157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38622/0 (socket says 172.21.15.119:38622) 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1ce59b700 1 -- 172.21.15.119:0/2997449096 learned_addr learned my addr 172.21.15.119:0/2997449096 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c815da90 0x7fa1c815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.439+0000 7fa1ced9c700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1c80af610 0x7fa1c8068df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1ce59b700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c815da90 msgr2=0x7fa1c815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c815da90 0x7fa1c815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1ce59b700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1c80af610 msgr2=0x7fa1c8068df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1c80af610 0x7fa1c8068df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1ce59b700 1 -- 172.21.15.119:0/2997449096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c815d600 con 0x7fa1b4003a60 2023-10-08T00:56:50.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1ce59b700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1c8157970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40139b0 tx=0x7fa1c401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4022070 con 0x7fa1b4003a60 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c4013b10 con 0x7fa1b4003a60 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1c815b2d0 con 0x7fa1b4003a60 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.440+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c401c450 con 0x7fa1b4003a60 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.441+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1c815cfb0 con 0x7fa1b4003a60 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.442+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1c804c3c0 con 0x7fa1b4003a60 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.442+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 6) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fa1c4026070 con 0x7fa1b4003a60 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.443+0000 7fa1be7fc700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.443+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5705+0+0 (secure 0 0 0) 0x7fa1c40857c0 con 0x7fa1b4003a60 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.443+0000 7fa1cdd9a700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 msgr2=0x7fa1ac069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:50.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.443+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T00:56:50.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.447+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1c4052ab0 con 0x7fa1b4003a60 2023-10-08T00:56:50.461 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:50.460+0000 7f73c70e2180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T00:56:50.532 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:50.532+0000 7f4585b1b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T00:56:50.623 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:50.622+0000 7f73c70e2180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T00:56:50.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.645+0000 7fa1cdd9a700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 msgr2=0x7fa1ac069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:50.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.645+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T00:56:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:50.669+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1c815cda0 con 0x7fa1ac066f80 2023-10-08T00:56:50.698 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:50.698+0000 7f4585b1b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T00:56:50.709 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:50.708+0000 7f73c70e2180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T00:56:50.785 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:50.785+0000 7f4585b1b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T00:56:50.931 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:50.930+0000 7f73c70e2180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:56:51.009 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:51.008+0000 7f4585b1b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:56:51.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:51.046+0000 7fa1cdd9a700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 msgr2=0x7fa1ac069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:51.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:51.046+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T00:56:51.108 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:51.107+0000 7f73c70e2180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T00:56:51.188 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:51.188+0000 7f4585b1b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T00:56:51.200 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:51.199+0000 7f73c70e2180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T00:56:51.279 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:51.279+0000 7f4585b1b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T00:56:51.363 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:51.362+0000 7f73c70e2180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T00:56:51.442 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:51.442+0000 7f4585b1b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T00:56:51.464 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:51.462+0000 7f73c70e2180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T00:56:51.544 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:51.544+0000 7f4585b1b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T00:56:51.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:51.847+0000 7fa1cdd9a700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 msgr2=0x7fa1ac069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:51.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:51.847+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T00:56:51.946 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:51.944+0000 7f73c70e2180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T00:56:52.028 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:52.028+0000 7f4585b1b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T00:56:52.040 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:52.038+0000 7f73c70e2180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T00:56:52.122 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:52.122+0000 7f4585b1b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T00:56:52.129 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:52.127+0000 7f73c70e2180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T00:56:52.213 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:52.213+0000 7f4585b1b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T00:56:52.751 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:52.750+0000 7f73c70e2180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T00:56:52.835 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:52.835+0000 7f4585b1b180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T00:56:52.923 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:52.922+0000 7f73c70e2180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:56:53.007 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:53.006+0000 7f4585b1b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T00:56:53.020 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:53.019+0000 7f73c70e2180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T00:56:53.102 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:53.102+0000 7f4585b1b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T00:56:53.322 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:53.321+0000 7f73c70e2180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T00:56:53.395 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:53.394+0000 7f4585b1b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T00:56:53.417 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:53.416+0000 7f73c70e2180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T00:56:53.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:53.448+0000 7fa1cdd9a700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 msgr2=0x7fa1ac069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T00:56:53.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:53.448+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T00:56:53.490 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:53.490+0000 7f4585b1b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T00:56:54.113 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:54.112+0000 7f73c70e2180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T00:56:54.191 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:54.191+0000 7f4585b1b180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T00:56:54.400 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:54.399+0000 7f73c70e2180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T00:56:54.474 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:54.474+0000 7f4585b1b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T00:56:54.492 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:54.491+0000 7f73c70e2180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T00:56:54.566 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:54.566+0000 7f4585b1b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T00:56:54.645 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:54.643+0000 7f73c70e2180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T00:56:54.722 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:54.722+0000 7f4585b1b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T00:56:54.748 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:54.746+0000 7f73c70e2180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T00:56:54.828 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:54.828+0000 7f4585b1b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T00:56:54.933 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:54.931+0000 7f73c70e2180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T00:56:55.016 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.016+0000 7f4585b1b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T00:56:55.022 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:55.021+0000 7f73c70e2180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T00:56:55.108 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.108+0000 7f4585b1b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T00:56:55.112 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:55.111+0000 7f73c70e2180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T00:56:55.202 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.202+0000 7f4585b1b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T00:56:55.207 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T00:56:55.206+0000 7f73c70e2180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T00:56:55.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:55.267+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 7) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fa1c4052e80 con 0x7fa1b4003a60 2023-10-08T00:56:55.297 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.296+0000 7f4585b1b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T00:56:55.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:55.307+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 8) v1 ==== 84468+0+0 (secure 0 0 0) 0x7fa1c4054520 con 0x7fa1b4003a60 2023-10-08T00:56:55.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:55.308+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 msgr2=0x7fa1ac069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T00:56:55.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:55.308+0000 7fa1be7fc700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:55.427 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44fae77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.427 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44fae77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.427 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44fae77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.427 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44fae77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.427 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44fae77700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.427 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44f7e71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.427 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44f7e71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.427 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44f7e71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.428 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44f7e71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:55.428 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:56:55.426+0000 7f44f7e71700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:56:56.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.312+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mon.0 v2:172.21.15.119:3300/0 9 ==== mgrmap(e 9) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fa1c4053750 con 0x7fa1b4003a60 2023-10-08T00:56:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.312+0000 7fa1be7fc700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.312+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1c815cda0 con 0x7fa1ac066f80 2023-10-08T00:56:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.316+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:56.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.318+0000 7fa1cdd9a700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac0693b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa1c815dea0 tx=0x7fa1c80cb6c0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:56.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.404+0000 7fa1be7fc700 1 -- 172.21.15.119:0/2997449096 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa1c815cda0 con 0x7fa1ac066f80 2023-10-08T00:56:56.404 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir0 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 msgr2=0x7fa1ac0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac0693b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa1c815dea0 tx=0x7fa1c80cb6c0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 msgr2=0x7fa1c8157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1c8157970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa1c40139b0 tx=0x7fa1c401a000 comp rx=0 tx=0).stop 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 shutdown_connections 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa1ac066f80 0x7fa1ac0693b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1c80af610 0x7fa1c8068df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1b4003a60 0x7fa1c8157970 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 --2- 172.21.15.119:0/2997449096 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1c815da90 0x7fa1c815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 >> 172.21.15.119:0/2997449096 conn(0x7fa1c8063230 msgr2=0x7fa1c80b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:56.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 shutdown_connections 2023-10-08T00:56:56.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.406+0000 7fa1cf59d700 1 -- 172.21.15.119:0/2997449096 wait complete. 2023-10-08T00:56:56.425 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-10-08T00:56:56.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:56.425 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-10-08T00:56:56.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-10-08T00:56:56.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.689+0000 7fc64f59e700 1 Processor -- start 2023-10-08T00:56:56.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.690+0000 7fc64f59e700 1 -- start start 2023-10-08T00:56:56.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc650156350 0x7fc650157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:56.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc64f59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6500b0bd0 0x7fc65015a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:56.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc64f59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6500b0570 con 0x7fc6500b0760 2023-10-08T00:56:56.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc64f59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6501578b0 con 0x7fc6500b0bd0 2023-10-08T00:56:56.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc64f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500af4d0 con 0x7fc650156350 2023-10-08T00:56:56.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc647fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc6500b0760 0x7fc650159f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54838/0 (socket says 172.21.15.119:54838) 2023-10-08T00:56:56.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc647fff700 1 -- 172.21.15.119:0/144211062 learned_addr learned my addr 172.21.15.119:0/144211062 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:56.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc64ed9d700 1 --2- 172.21.15.119:0/144211062 >> v2:172.21.15.119:3300/0 conn(0x7fc6500b0bd0 0x7fc65015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:56.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.691+0000 7fc64e59c700 1 --2- 172.21.15.119:0/144211062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc650156350 0x7fc650157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:56.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64ed9d700 1 -- 172.21.15.119:0/144211062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc650156350 msgr2=0x7fc650157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64ed9d700 1 --2- 172.21.15.119:0/144211062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc650156350 0x7fc650157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64ed9d700 1 -- 172.21.15.119:0/144211062 >> v1:172.21.15.119:6789/0 conn(0x7fc6500b0760 legacy=0x7fc650159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64ed9d700 1 -- 172.21.15.119:0/144211062 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500af7b0 con 0x7fc6500b0bd0 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64e59c700 1 --2- 172.21.15.119:0/144211062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc650156350 0x7fc650157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64ed9d700 1 --2- 172.21.15.119:0/144211062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6500b0bd0 0x7fc65015a630 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc648009e60 tx=0x7fc648009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9850bcbf5823c94 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64dd9b700 1 -- 172.21.15.119:0/144211062 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64800e070 con 0x7fc6500b0bd0 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64dd9b700 1 -- 172.21.15.119:0/144211062 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64800bc30 con 0x7fc6500b0bd0 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64dd9b700 1 -- 172.21.15.119:0/144211062 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc648012430 con 0x7fc6500b0bd0 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64f59e700 1 -- 172.21.15.119:0/144211062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6500b0bd0 msgr2=0x7fc65015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:56.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.692+0000 7fc64f59e700 1 --2- 172.21.15.119:0/144211062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6500b0bd0 0x7fc65015a630 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc648009e60 tx=0x7fc648009780 comp rx=0 tx=0).stop 2023-10-08T00:56:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.693+0000 7fc64f59e700 1 -- 172.21.15.119:0/144211062 shutdown_connections 2023-10-08T00:56:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.693+0000 7fc64f59e700 1 --2- 172.21.15.119:0/144211062 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6500b0bd0 0x7fc65015a630 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.693+0000 7fc64f59e700 1 --2- 172.21.15.119:0/144211062 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc650156350 0x7fc650157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.693+0000 7fc64f59e700 1 -- 172.21.15.119:0/144211062 >> 172.21.15.119:0/144211062 conn(0x7fc650068e20 msgr2=0x7fc6500cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.693+0000 7fc64f59e700 1 -- 172.21.15.119:0/144211062 shutdown_connections 2023-10-08T00:56:56.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.693+0000 7fc64f59e700 1 -- 172.21.15.119:0/144211062 wait complete. 2023-10-08T00:56:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.694+0000 7fc64f59e700 1 Processor -- start 2023-10-08T00:56:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.694+0000 7fc64f59e700 1 -- start start 2023-10-08T00:56:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.695+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6500b0760 0x7fc6501526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:56.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.695+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6500b0bd0 0x7fc650152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.695+0000 7fc64f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc650156350 0x7fc6500cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.695+0000 7fc64f59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65014ef70 con 0x7fc650156350 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.695+0000 7fc64f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015c1e0 con 0x7fc6500b0760 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.695+0000 7fc64f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015c4c0 con 0x7fc6500b0bd0 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc650156350 0x7fc6500cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc650156350 0x7fc6500cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44560/0 (socket says 172.21.15.119:44560) 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 -- 172.21.15.119:0/2600586795 learned_addr learned my addr 172.21.15.119:0/2600586795 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc647fff700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6500b0bd0 0x7fc650152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64e59c700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6500b0760 0x7fc6501526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 -- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6500b0bd0 msgr2=0x7fc650152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:56.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6500b0bd0 0x7fc650152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 -- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6500b0760 msgr2=0x7fc6501526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6500b0760 0x7fc6501526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 -- 172.21.15.119:0/2600586795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648009a20 con 0x7fc650156350 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64e59c700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6500b0760 0x7fc6501526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc64ed9d700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc650156350 0x7fc6500cb1e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc648013000 tx=0x7fc648019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.696+0000 7fc6467fc700 1 -- 172.21.15.119:0/2600586795 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc648012e90 con 0x7fc650156350 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.697+0000 7fc6467fc700 1 -- 172.21.15.119:0/2600586795 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64801e630 con 0x7fc650156350 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.697+0000 7fc6467fc700 1 -- 172.21.15.119:0/2600586795 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64801e9d0 con 0x7fc650156350 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.697+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65015c7a0 con 0x7fc650156350 2023-10-08T00:56:56.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.697+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65015be20 con 0x7fc650156350 2023-10-08T00:56:56.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.698+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65015bfb0 con 0x7fc650156350 2023-10-08T00:56:56.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.698+0000 7fc6467fc700 1 -- 172.21.15.119:0/2600586795 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fc648024070 con 0x7fc650156350 2023-10-08T00:56:56.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.699+0000 7fc6467fc700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc630066dd0 0x7fc630069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:56.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.699+0000 7fc6467fc700 1 -- 172.21.15.119:0/2600586795 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fc648086ce0 con 0x7fc650156350 2023-10-08T00:56:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.700+0000 7fc64e59c700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc630066dd0 0x7fc630069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:56.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.700+0000 7fc64e59c700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc630066dd0 0x7fc630069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc64000ac10 tx=0x7fc640006f90 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:56.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.704+0000 7fc6467fc700 1 -- 172.21.15.119:0/2600586795 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc64808b070 con 0x7fc650156350 2023-10-08T00:56:56.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.917+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc65015cdf0 con 0x7fc630066dd0 2023-10-08T00:56:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.993+0000 7fc6467fc700 1 -- 172.21.15.119:0/2600586795 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc65015cdf0 con 0x7fc630066dd0 2023-10-08T00:56:56.994 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir1 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc630066dd0 msgr2=0x7fc630069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc630066dd0 0x7fc630069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc64000ac10 tx=0x7fc640006f90 comp rx=0 tx=0).stop 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc650156350 msgr2=0x7fc6500cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc650156350 0x7fc6500cb1e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc648013000 tx=0x7fc648019000 comp rx=0 tx=0).stop 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 shutdown_connections 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc630066dd0 0x7fc630069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc650156350 0x7fc6500cb1e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6500b0760 0x7fc6501526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 --2- 172.21.15.119:0/2600586795 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6500b0bd0 0x7fc650152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:56.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 >> 172.21.15.119:0/2600586795 conn(0x7fc650068e20 msgr2=0x7fc6501519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:56.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.995+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 shutdown_connections 2023-10-08T00:56:56.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:56.996+0000 7fc64f59e700 1 -- 172.21.15.119:0/2600586795 wait complete. 2023-10-08T00:56:57.013 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-10-08T00:56:57.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:57.014 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-10-08T00:56:57.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-10-08T00:56:57.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.278+0000 7fea463ff700 1 Processor -- start 2023-10-08T00:56:57.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.279+0000 7fea463ff700 1 -- start start 2023-10-08T00:56:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.280+0000 7fea463ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea401563b0 0x7fea40156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.280+0000 7fea463ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4015a9b0 con 0x7fea4015a440 2023-10-08T00:56:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.280+0000 7fea463ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea4015ac90 con 0x7fea40068890 2023-10-08T00:56:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.280+0000 7fea463ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea4015af70 con 0x7fea401563b0 2023-10-08T00:56:57.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.280+0000 7fea37fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fea4015a440 0x7fea400acf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54850/0 (socket says 172.21.15.119:54850) 2023-10-08T00:56:57.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.280+0000 7fea37fff700 1 -- 172.21.15.119:0/886345882 learned_addr learned my addr 172.21.15.119:0/886345882 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:56:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.280+0000 7fea3ffff700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea401563b0 0x7fea40156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.280+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403908686 0 0) 0x7fea4015a9b0 con 0x7fea4015a440 2023-10-08T00:56:57.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea2c003570 con 0x7fea4015a440 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119021814 0 0) 0x7fea2c003570 con 0x7fea4015a440 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea401563b0 msgr2=0x7fea40156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea401563b0 0x7fea40156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 >> v1:172.21.15.184:6789/0 conn(0x7fea40068890 legacy=0x7fea400c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4015b250 con 0x7fea4015a440 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fea28001470 con 0x7fea4015a440 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 >> v1:172.21.15.119:6789/0 conn(0x7fea4015a440 legacy=0x7fea400acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2c003a90 0x7fea2c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2c004810 0x7fea2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea401563b0 0x7fea2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4015af70 con 0x7fea2c004810 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c00bb10 con 0x7fea2c003a90 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c00dee0 con 0x7fea401563b0 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea37fff700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2c004810 0x7fea2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.281+0000 7fea3ffff700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2c003a90 0x7fea2c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea4499c700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea401563b0 0x7fea2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea37fff700 1 -- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea401563b0 msgr2=0x7fea2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea37fff700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea401563b0 0x7fea2c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea37fff700 1 -- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2c003a90 msgr2=0x7fea2c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea37fff700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2c003a90 0x7fea2c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea37fff700 1 -- 172.21.15.119:0/886345882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c003570 con 0x7fea2c004810 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea37fff700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2c004810 0x7fea2c004c80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c3c0 tx=0x7fea28001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c43bc5f34fb6be1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:57.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800cc90 con 0x7fea2c004810 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2800cdf0 con 0x7fea2c004810 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.282+0000 7fea3f7fe700 1 -- 172.21.15.119:0/886345882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28005b70 con 0x7fea2c004810 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.283+0000 7fea463ff700 1 -- 172.21.15.119:0/886345882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2c004810 msgr2=0x7fea2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.283+0000 7fea463ff700 1 --2- 172.21.15.119:0/886345882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea2c004810 0x7fea2c004c80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c3c0 tx=0x7fea28001e70 comp rx=0 tx=0).stop 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.283+0000 7fea4499c700 1 -- 172.21.15.119:0/886345882 reap_dead start 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.283+0000 7fea463ff700 1 -- 172.21.15.119:0/886345882 shutdown_connections 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.283+0000 7fea463ff700 1 -- 172.21.15.119:0/886345882 >> 172.21.15.119:0/886345882 conn(0x7fea400c8f90 msgr2=0x7fea400c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.283+0000 7fea463ff700 1 -- 172.21.15.119:0/886345882 shutdown_connections 2023-10-08T00:56:57.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.283+0000 7fea463ff700 1 -- 172.21.15.119:0/886345882 wait complete. 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.284+0000 7fea463ff700 1 Processor -- start 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.284+0000 7fea463ff700 1 -- start start 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.285+0000 7fea463ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2c004810 0x7fea401559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.285+0000 7fea463ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40157b80 0x7fea40158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.285+0000 7fea463ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea40156fd0 0x7fea40157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.285+0000 7fea463ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea40159b00 con 0x7fea40157b80 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.285+0000 7fea463ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea401579b0 con 0x7fea40156fd0 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.285+0000 7fea463ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea40159540 con 0x7fea2c004810 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.285+0000 7fea37fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40157b80 0x7fea40158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea37fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40157b80 0x7fea40158c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44584/0 (socket says 172.21.15.119:44584) 2023-10-08T00:56:57.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea37fff700 1 -- 172.21.15.119:0/882124159 learned_addr learned my addr 172.21.15.119:0/882124159 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea3ffff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2c004810 0x7fea401559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea4499c700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea40156fd0 0x7fea40157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea37fff700 1 -- 172.21.15.119:0/882124159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2c004810 msgr2=0x7fea401559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea37fff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2c004810 0x7fea401559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea37fff700 1 -- 172.21.15.119:0/882124159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea40156fd0 msgr2=0x7fea40157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea37fff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea40156fd0 0x7fea40157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea37fff700 1 -- 172.21.15.119:0/882124159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea40159820 con 0x7fea40157b80 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea37fff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40157b80 0x7fea40158c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c9b0 tx=0x7fea28015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea3cff9700 1 -- 172.21.15.119:0/882124159 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800cc90 con 0x7fea40157b80 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea3cff9700 1 -- 172.21.15.119:0/882124159 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea280070d0 con 0x7fea40157b80 2023-10-08T00:56:57.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea400c1040 con 0x7fea40157b80 2023-10-08T00:56:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.286+0000 7fea3cff9700 1 -- 172.21.15.119:0/882124159 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea280074b0 con 0x7fea40157b80 2023-10-08T00:56:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.287+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea40155f20 con 0x7fea40157b80 2023-10-08T00:56:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.288+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea400c14e0 con 0x7fea40157b80 2023-10-08T00:56:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.288+0000 7fea3cff9700 1 -- 172.21.15.119:0/882124159 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 9) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fea28019030 con 0x7fea40157b80 2023-10-08T00:56:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.290+0000 7fea3cff9700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066dd0 0x7fea20069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.290+0000 7fea3cff9700 1 -- 172.21.15.119:0/882124159 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fea28085510 con 0x7fea40157b80 2023-10-08T00:56:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.290+0000 7fea3ffff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066dd0 0x7fea20069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.291+0000 7fea3ffff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066dd0 0x7fea20069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fea30006f00 tx=0x7fea300091a0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:57.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.294+0000 7fea3cff9700 1 -- 172.21.15.119:0/882124159 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea280529e0 con 0x7fea40157b80 2023-10-08T00:56:57.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.321+0000 7fea3cff9700 1 -- 172.21.15.119:0/882124159 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 10) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fea28052db0 con 0x7fea40157b80 2023-10-08T00:56:57.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.514+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fea400cd420 con 0x7fea20066dd0 2023-10-08T00:56:57.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.582+0000 7fea3cff9700 1 -- 172.21.15.119:0/882124159 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fea400cd420 con 0x7fea20066dd0 2023-10-08T00:56:57.583 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir2 2023-10-08T00:56:57.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066dd0 msgr2=0x7fea20069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066dd0 0x7fea20069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fea30006f00 tx=0x7fea300091a0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40157b80 msgr2=0x7fea40158c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40157b80 0x7fea40158c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fea2800c9b0 tx=0x7fea28015000 comp rx=0 tx=0).stop 2023-10-08T00:56:57.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 shutdown_connections 2023-10-08T00:56:57.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2c004810 0x7fea401559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fea20066dd0 0x7fea20069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea40156fd0 0x7fea40157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 --2- 172.21.15.119:0/882124159 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea40157b80 0x7fea40158c40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 >> 172.21.15.119:0/882124159 conn(0x7fea400c8f90 msgr2=0x7fea4006dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:57.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 shutdown_connections 2023-10-08T00:56:57.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.585+0000 7fea463ff700 1 -- 172.21.15.119:0/882124159 wait complete. 2023-10-08T00:56:57.603 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-10-08T00:56:57.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:57.603 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-10-08T00:56:57.641 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-10-08T00:56:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.867+0000 7f90a2dff700 1 Processor -- start 2023-10-08T00:56:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.868+0000 7f90a2dff700 1 -- start start 2023-10-08T00:56:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.869+0000 7f90a2dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c159ec0 0x7f909c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.869+0000 7f90a2dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f909c0cf350 0x7f909c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.869+0000 7f90a2dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0b5b00 con 0x7f909c0b53d0 2023-10-08T00:56:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.869+0000 7f90a2dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0b5da0 con 0x7f909c0cf350 2023-10-08T00:56:57.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.869+0000 7f90a2dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0b6080 con 0x7f909c159ec0 2023-10-08T00:56:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.869+0000 7f90a139c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f909c0cf350 0x7f909c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.869+0000 7f90a0b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c159ec0 0x7f909c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a139c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f909c0cf350 0x7f909c0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:55514/0 (socket says 172.21.15.119:55514) 2023-10-08T00:56:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a0b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c159ec0 0x7f909c0b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:54642/0 (socket says 172.21.15.119:54642) 2023-10-08T00:56:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a139c700 1 -- 172.21.15.119:0/3163621728 learned_addr learned my addr 172.21.15.119:0/3163621728 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f909bfff700 1 --1- 172.21.15.119:0/3163621728 >> v1:172.21.15.184:6789/0 conn(0x7f909c0b53d0 0x7f909c0cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:39250/0 (socket says 172.21.15.119:39250) 2023-10-08T00:56:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a139c700 1 -- 172.21.15.119:0/3163621728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c159ec0 msgr2=0x7f909c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a139c700 1 --2- 172.21.15.119:0/3163621728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c159ec0 0x7f909c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a139c700 1 -- 172.21.15.119:0/3163621728 >> v1:172.21.15.184:6789/0 conn(0x7f909c0b53d0 legacy=0x7f909c0cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a139c700 1 -- 172.21.15.119:0/3163621728 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0cc100 con 0x7f909c0cf350 2023-10-08T00:56:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a0b9b700 1 --2- 172.21.15.119:0/3163621728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c159ec0 0x7f909c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.870+0000 7f90a139c700 1 --2- 172.21.15.119:0/3163621728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c0cf350 0x7f909c0cd010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9094009f20 tx=0x7f9094009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4f864ec6b75ff47 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f909b7fe700 1 -- 172.21.15.119:0/3163621728 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400e070 con 0x7f909c0cf350 2023-10-08T00:56:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f909b7fe700 1 -- 172.21.15.119:0/3163621728 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400b9d0 con 0x7f909c0cf350 2023-10-08T00:56:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f909b7fe700 1 -- 172.21.15.119:0/3163621728 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90940092e0 con 0x7f909c0cf350 2023-10-08T00:56:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f90a2dff700 1 -- 172.21.15.119:0/3163621728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c0cf350 msgr2=0x7f909c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3163621728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c0cf350 0x7f909c0cd010 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9094009f20 tx=0x7f9094009840 comp rx=0 tx=0).stop 2023-10-08T00:56:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f90a2dff700 1 -- 172.21.15.119:0/3163621728 shutdown_connections 2023-10-08T00:56:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3163621728 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c0cf350 0x7f909c0cd010 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3163621728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c159ec0 0x7f909c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f90a2dff700 1 -- 172.21.15.119:0/3163621728 >> 172.21.15.119:0/3163621728 conn(0x7f909c0999f0 msgr2=0x7f909c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f90a2dff700 1 -- 172.21.15.119:0/3163621728 shutdown_connections 2023-10-08T00:56:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.871+0000 7f90a2dff700 1 -- 172.21.15.119:0/3163621728 wait complete. 2023-10-08T00:56:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.873+0000 7f90a2dff700 1 Processor -- start 2023-10-08T00:56:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.873+0000 7f90a2dff700 1 -- start start 2023-10-08T00:56:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a2dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f909c0b53d0 0x7f909c157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a2dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c0cf350 0x7f909c069d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a2dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c159ec0 0x7f909c09c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a2dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c169e60 con 0x7f909c0b53d0 2023-10-08T00:56:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a2dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c09d790 con 0x7f909c159ec0 2023-10-08T00:56:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a2dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c06a2b0 con 0x7f909c0cf350 2023-10-08T00:56:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a0b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f909c0b53d0 0x7f909c157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a0b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f909c0b53d0 0x7f909c157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44588/0 (socket says 172.21.15.119:44588) 2023-10-08T00:56:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a0b9b700 1 -- 172.21.15.119:0/3251939592 learned_addr learned my addr 172.21.15.119:0/3251939592 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f909bfff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c0cf350 0x7f909c069d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a139c700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c159ec0 0x7f909c09c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a0b9b700 1 -- 172.21.15.119:0/3251939592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c0cf350 msgr2=0x7f909c069d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a0b9b700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c0cf350 0x7f909c069d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a0b9b700 1 -- 172.21.15.119:0/3251939592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c159ec0 msgr2=0x7f909c09c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a0b9b700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c159ec0 0x7f909c09c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.874+0000 7f90a0b9b700 1 -- 172.21.15.119:0/3251939592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90900093c0 con 0x7f909c0b53d0 2023-10-08T00:56:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.875+0000 7f90a0b9b700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f909c0b53d0 0x7f909c157360 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f909000ec80 tx=0x7f9090009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.875+0000 7f9098ff9700 1 -- 172.21.15.119:0/3251939592 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9090005a00 con 0x7f909c0b53d0 2023-10-08T00:56:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.875+0000 7f9098ff9700 1 -- 172.21.15.119:0/3251939592 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9090005b60 con 0x7f909c0b53d0 2023-10-08T00:56:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.875+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9094009ae0 con 0x7f909c0b53d0 2023-10-08T00:56:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.875+0000 7f9098ff9700 1 -- 172.21.15.119:0/3251939592 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909000c850 con 0x7f909c0b53d0 2023-10-08T00:56:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.875+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c06a830 con 0x7f909c0b53d0 2023-10-08T00:56:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.876+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c063c90 con 0x7f909c0b53d0 2023-10-08T00:56:57.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.877+0000 7f9098ff9700 1 -- 172.21.15.119:0/3251939592 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 10) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f909000cbc0 con 0x7f909c0b53d0 2023-10-08T00:56:57.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.878+0000 7f9098ff9700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9080067130 0x7f90800695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:57.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.878+0000 7f9098ff9700 1 -- 172.21.15.119:0/3251939592 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f9090084e20 con 0x7f909c0b53d0 2023-10-08T00:56:57.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.878+0000 7f909bfff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9080067130 0x7f90800695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:57.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.879+0000 7f909bfff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9080067130 0x7f90800695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f909c0b2610 tx=0x7f9088007eb0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:57.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:57.882+0000 7f9098ff9700 1 -- 172.21.15.119:0/3251939592 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90900557a0 con 0x7f909c0b53d0 2023-10-08T00:56:58.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.101+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f909c169fa0 con 0x7f9080067130 2023-10-08T00:56:58.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.167+0000 7f9098ff9700 1 -- 172.21.15.119:0/3251939592 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f9088004c60 con 0x7f9080067130 2023-10-08T00:56:58.168 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir3 2023-10-08T00:56:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9080067130 msgr2=0x7f90800695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:58.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9080067130 0x7f90800695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f909c0b2610 tx=0x7f9088007eb0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f909c0b53d0 msgr2=0x7f909c157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f909c0b53d0 0x7f909c157360 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f909000ec80 tx=0x7f9090009120 comp rx=0 tx=0).stop 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 shutdown_connections 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9080067130 0x7f90800695e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f909c159ec0 0x7f909c09c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f909c0b53d0 0x7f909c157360 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 --2- 172.21.15.119:0/3251939592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f909c0cf350 0x7f909c069d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 >> 172.21.15.119:0/3251939592 conn(0x7f909c0999f0 msgr2=0x7f909c0c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 shutdown_connections 2023-10-08T00:56:58.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.170+0000 7f90a2dff700 1 -- 172.21.15.119:0/3251939592 wait complete. 2023-10-08T00:56:58.188 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-10-08T00:56:58.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:58.188 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-10-08T00:56:58.226 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-10-08T00:56:58.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.454+0000 7f6a3699d700 1 Processor -- start 2023-10-08T00:56:58.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.455+0000 7f6a3699d700 1 -- start start 2023-10-08T00:56:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3699d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6a3015b050 0x7f6a300ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3699d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6a300cc8d0 0x7f6a3015b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3699d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a30157a50 con 0x7f6a3015b960 2023-10-08T00:56:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3699d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a301591b0 con 0x7f6a3015b050 2023-10-08T00:56:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3699d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a300ca9e0 con 0x7f6a300cc8d0 2023-10-08T00:56:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3599b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6a3015b050 0x7f6a300ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:58.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3599b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6a3015b050 0x7f6a300ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44596/0 (socket says 172.21.15.119:44596) 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3599b700 1 -- 172.21.15.119:0/2704602693 learned_addr learned my addr 172.21.15.119:0/2704602693 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3599b700 1 -- 172.21.15.119:0/2704602693 >> v2:172.21.15.184:3300/0 conn(0x7f6a300cc8d0 msgr2=0x7f6a3015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3599b700 1 --2- 172.21.15.119:0/2704602693 >> v2:172.21.15.184:3300/0 conn(0x7f6a300cc8d0 0x7f6a3015b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3599b700 1 -- 172.21.15.119:0/2704602693 >> v1:172.21.15.184:6789/0 conn(0x7f6a3015b960 legacy=0x7f6a30157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.456+0000 7f6a3599b700 1 -- 172.21.15.119:0/2704602693 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a300cacc0 con 0x7f6a3015b050 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.457+0000 7f6a3599b700 1 --2- 172.21.15.119:0/2704602693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a3015b050 0x7f6a300ca100 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6a3006fdd0 tx=0x7f6a2c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e4881b9bb61397 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.457+0000 7f6a34999700 1 -- 172.21.15.119:0/2704602693 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00b820 con 0x7f6a3015b050 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.457+0000 7f6a34999700 1 -- 172.21.15.119:0/2704602693 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c00b980 con 0x7f6a3015b050 2023-10-08T00:56:58.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.457+0000 7f6a34999700 1 -- 172.21.15.119:0/2704602693 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c00bd20 con 0x7f6a3015b050 2023-10-08T00:56:58.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.457+0000 7f6a3699d700 1 -- 172.21.15.119:0/2704602693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a3015b050 msgr2=0x7f6a300ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:58.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.457+0000 7f6a3699d700 1 --2- 172.21.15.119:0/2704602693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a3015b050 0x7f6a300ca100 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6a3006fdd0 tx=0x7f6a2c009250 comp rx=0 tx=0).stop 2023-10-08T00:56:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.458+0000 7f6a3699d700 1 -- 172.21.15.119:0/2704602693 shutdown_connections 2023-10-08T00:56:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.458+0000 7f6a3699d700 1 --2- 172.21.15.119:0/2704602693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a3015b050 0x7f6a300ca100 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.458+0000 7f6a3699d700 1 --2- 172.21.15.119:0/2704602693 >> v2:172.21.15.184:3300/0 conn(0x7f6a300cc8d0 0x7f6a3015b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.458+0000 7f6a3699d700 1 -- 172.21.15.119:0/2704602693 >> 172.21.15.119:0/2704602693 conn(0x7f6a30068e20 msgr2=0x7f6a300cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.458+0000 7f6a3699d700 1 -- 172.21.15.119:0/2704602693 shutdown_connections 2023-10-08T00:56:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.458+0000 7f6a3699d700 1 -- 172.21.15.119:0/2704602693 wait complete. 2023-10-08T00:56:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.459+0000 7f6a3699d700 1 Processor -- start 2023-10-08T00:56:58.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.459+0000 7f6a3699d700 1 -- start start 2023-10-08T00:56:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3699d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a300cc8d0 0x7f6a3016da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3699d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a3015b050 0x7f6a30144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3699d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a3015b960 0x7f6a3006ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3699d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a30152a00 con 0x7f6a300cc8d0 2023-10-08T00:56:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3699d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3016a810 con 0x7f6a3015b050 2023-10-08T00:56:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3699d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3016d580 con 0x7f6a3015b960 2023-10-08T00:56:58.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3599b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a300cc8d0 0x7f6a3016da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3599b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a300cc8d0 0x7f6a3016da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44606/0 (socket says 172.21.15.119:44606) 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.460+0000 7f6a3599b700 1 -- 172.21.15.119:0/3645925414 learned_addr learned my addr 172.21.15.119:0/3645925414 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3619c700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a3015b960 0x7f6a3006ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3519a700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a3015b050 0x7f6a30144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3599b700 1 -- 172.21.15.119:0/3645925414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a3015b960 msgr2=0x7f6a3006ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3599b700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a3015b960 0x7f6a3006ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3599b700 1 -- 172.21.15.119:0/3645925414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a3015b050 msgr2=0x7f6a30144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3599b700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a3015b050 0x7f6a30144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3599b700 1 -- 172.21.15.119:0/3645925414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c0094f0 con 0x7f6a300cc8d0 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3599b700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a300cc8d0 0x7f6a3016da10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c003080 tx=0x7f6a2c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a25ffb700 1 -- 172.21.15.119:0/3645925414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c011e20 con 0x7f6a300cc8d0 2023-10-08T00:56:58.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a25ffb700 1 -- 172.21.15.119:0/3645925414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c01f070 con 0x7f6a300cc8d0 2023-10-08T00:56:58.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a25ffb700 1 -- 172.21.15.119:0/3645925414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c01a8a0 con 0x7f6a300cc8d0 2023-10-08T00:56:58.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a3016a530 con 0x7f6a300cc8d0 2023-10-08T00:56:58.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.461+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a3016d210 con 0x7f6a300cc8d0 2023-10-08T00:56:58.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.462+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a3016d860 con 0x7f6a300cc8d0 2023-10-08T00:56:58.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.463+0000 7f6a25ffb700 1 -- 172.21.15.119:0/3645925414 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 10) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f6a2c01ac90 con 0x7f6a300cc8d0 2023-10-08T00:56:58.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.464+0000 7f6a25ffb700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6a14067130 0x7f6a140695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:58.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.464+0000 7f6a25ffb700 1 -- 172.21.15.119:0/3645925414 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f6a2c021070 con 0x7f6a300cc8d0 2023-10-08T00:56:58.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.464+0000 7f6a3519a700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6a14067130 0x7f6a140695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:58.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.465+0000 7f6a3519a700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6a14067130 0x7f6a140695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6a300b13d0 tx=0x7f6a20008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:58.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.468+0000 7f6a25ffb700 1 -- 172.21.15.119:0/3645925414 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a2c053c20 con 0x7f6a300cc8d0 2023-10-08T00:56:58.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.681+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a3004c3c0 con 0x7f6a14067130 2023-10-08T00:56:58.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.765+0000 7f6a25ffb700 1 -- 172.21.15.119:0/3645925414 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6a3004c3c0 con 0x7f6a14067130 2023-10-08T00:56:58.766 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir4 2023-10-08T00:56:58.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6a14067130 msgr2=0x7f6a140695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:58.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6a14067130 0x7f6a140695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6a300b13d0 tx=0x7f6a20008040 comp rx=0 tx=0).stop 2023-10-08T00:56:58.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a300cc8d0 msgr2=0x7f6a3016da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:58.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a300cc8d0 0x7f6a3016da10 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c003080 tx=0x7f6a2c016000 comp rx=0 tx=0).stop 2023-10-08T00:56:58.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 shutdown_connections 2023-10-08T00:56:58.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a3015b960 0x7f6a3006ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6a14067130 0x7f6a140695e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a300cc8d0 0x7f6a3016da10 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 --2- 172.21.15.119:0/3645925414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a3015b050 0x7f6a30144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:58.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 >> 172.21.15.119:0/3645925414 conn(0x7f6a30068e20 msgr2=0x7f6a30154430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:58.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 shutdown_connections 2023-10-08T00:56:58.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:58.768+0000 7f6a3699d700 1 -- 172.21.15.119:0/3645925414 wait complete. 2023-10-08T00:56:58.785 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-10-08T00:56:58.785 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:58.786 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-10-08T00:56:58.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-10-08T00:56:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.050+0000 7f8115b9d700 1 Processor -- start 2023-10-08T00:56:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.051+0000 7f8115b9d700 1 -- start start 2023-10-08T00:56:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.052+0000 7f8115b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110156350 0x7f8110157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.052+0000 7f8115b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81100b0760 0x7f81100b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.052+0000 7f8115b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100b1110 con 0x7f8110159f20 2023-10-08T00:56:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.052+0000 7f8115b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81100b0570 con 0x7f81100b0760 2023-10-08T00:56:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.052+0000 7f8115b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81101578b0 con 0x7f8110156350 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.052+0000 7f810ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81100b0760 0x7f81100b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.052+0000 7f8114b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110156350 0x7f8110157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f810ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81100b0760 0x7f81100b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:55560/0 (socket says 172.21.15.119:55560) 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f8114b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110156350 0x7f8110157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:54652/0 (socket says 172.21.15.119:54652) 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f810ffff700 1 -- 172.21.15.119:0/2546659987 learned_addr learned my addr 172.21.15.119:0/2546659987 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f810ffff700 1 -- 172.21.15.119:0/2546659987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110156350 msgr2=0x7f8110157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f810ffff700 1 --2- 172.21.15.119:0/2546659987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110156350 0x7f8110157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f810ffff700 1 -- 172.21.15.119:0/2546659987 >> v1:172.21.15.184:6789/0 conn(0x7f8110159f20 legacy=0x7f811015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f810ffff700 1 -- 172.21.15.119:0/2546659987 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81100af4d0 con 0x7f81100b0760 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f8114b9b700 1 --2- 172.21.15.119:0/2546659987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110156350 0x7f8110157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:56:59.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f810f7fe700 1 -- 172.21.15.119:0/2546659987 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483205041 0 0) 0x7f81100b1110 con 0x7f8110159f20 2023-10-08T00:56:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.053+0000 7f810ffff700 1 --2- 172.21.15.119:0/2546659987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81100b0760 0x7f81100b0bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0099b0 tx=0x7f80fc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d142555beede27bb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f810f7fe700 1 -- 172.21.15.119:0/2546659987 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc009ea0 con 0x7f81100b0760 2023-10-08T00:56:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f810f7fe700 1 -- 172.21.15.119:0/2546659987 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc00b860 con 0x7f81100b0760 2023-10-08T00:56:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f810f7fe700 1 -- 172.21.15.119:0/2546659987 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00bc00 con 0x7f81100b0760 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f8115b9d700 1 -- 172.21.15.119:0/2546659987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81100b0760 msgr2=0x7f81100b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f8115b9d700 1 --2- 172.21.15.119:0/2546659987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81100b0760 0x7f81100b0bd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0099b0 tx=0x7f80fc009250 comp rx=0 tx=0).stop 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f8115b9d700 1 -- 172.21.15.119:0/2546659987 shutdown_connections 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f8115b9d700 1 --2- 172.21.15.119:0/2546659987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110156350 0x7f8110157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f8115b9d700 1 --2- 172.21.15.119:0/2546659987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81100b0760 0x7f81100b0bd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f8115b9d700 1 -- 172.21.15.119:0/2546659987 >> 172.21.15.119:0/2546659987 conn(0x7f8110068e20 msgr2=0x7f81100cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f8115b9d700 1 -- 172.21.15.119:0/2546659987 shutdown_connections 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.054+0000 7f8115b9d700 1 -- 172.21.15.119:0/2546659987 wait complete. 2023-10-08T00:56:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.056+0000 7f8115b9d700 1 Processor -- start 2023-10-08T00:56:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.056+0000 7f8115b9d700 1 -- start start 2023-10-08T00:56:59.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f8115b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100b0760 0x7f81100c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f8115b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110156350 0x7f81100cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f8115b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110159f20 0x7f8110149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f8115b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811016bed0 con 0x7f8110156350 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f8115b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100c9600 con 0x7f8110159f20 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f8115b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811016bbf0 con 0x7f81100b0760 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f810ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110156350 0x7f81100cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f810ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110156350 0x7f81100cb6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44608/0 (socket says 172.21.15.119:44608) 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f810ffff700 1 -- 172.21.15.119:0/3097274680 learned_addr learned my addr 172.21.15.119:0/3097274680 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f8114b9b700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100b0760 0x7f81100c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f811539c700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110159f20 0x7f8110149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f810ffff700 1 -- 172.21.15.119:0/3097274680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100b0760 msgr2=0x7f81100c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f810ffff700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100b0760 0x7f81100c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f810ffff700 1 -- 172.21.15.119:0/3097274680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110159f20 msgr2=0x7f8110149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f810ffff700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110159f20 0x7f8110149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.057+0000 7f810ffff700 1 -- 172.21.15.119:0/3097274680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8104009420 con 0x7f8110156350 2023-10-08T00:56:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.058+0000 7f810ffff700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110156350 0x7f81100cb6d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f81100714a0 tx=0x7f80fc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.058+0000 7f810cff9700 1 -- 172.21.15.119:0/3097274680 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00bbe0 con 0x7f8110156350 2023-10-08T00:56:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.058+0000 7f810cff9700 1 -- 172.21.15.119:0/3097274680 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc011420 con 0x7f8110156350 2023-10-08T00:56:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.058+0000 7f810cff9700 1 -- 172.21.15.119:0/3097274680 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc0117c0 con 0x7f8110156350 2023-10-08T00:56:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.058+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc0094f0 con 0x7f8110156350 2023-10-08T00:56:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.058+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81100caed0 con 0x7f8110156350 2023-10-08T00:56:59.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.059+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811004c3c0 con 0x7f8110156350 2023-10-08T00:56:59.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.060+0000 7f810cff9700 1 -- 172.21.15.119:0/3097274680 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 10) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f80fc011bf0 con 0x7f8110156350 2023-10-08T00:56:59.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.061+0000 7f810cff9700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f80f4067130 0x7f80f40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.061+0000 7f810cff9700 1 -- 172.21.15.119:0/3097274680 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f80fc085ec0 con 0x7f8110156350 2023-10-08T00:56:59.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.061+0000 7f8114b9b700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f80f4067130 0x7f80f40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.061+0000 7f8114b9b700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f80f4067130 0x7f80f40695e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f81040093f0 tx=0x7f8104008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:59.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.065+0000 7f810cff9700 1 -- 172.21.15.119:0/3097274680 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80fc052e30 con 0x7f8110156350 2023-10-08T00:56:59.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.282+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f81100af7b0 con 0x7f80f4067130 2023-10-08T00:56:59.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.352+0000 7f810cff9700 1 -- 172.21.15.119:0/3097274680 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f81100af7b0 con 0x7f80f4067130 2023-10-08T00:56:59.353 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir5 2023-10-08T00:56:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f80f4067130 msgr2=0x7f80f40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f80f4067130 0x7f80f40695e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f81040093f0 tx=0x7f8104008040 comp rx=0 tx=0).stop 2023-10-08T00:56:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110156350 msgr2=0x7f81100cb6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110156350 0x7f81100cb6d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f81100714a0 tx=0x7f80fc017000 comp rx=0 tx=0).stop 2023-10-08T00:56:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 shutdown_connections 2023-10-08T00:56:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100b0760 0x7f81100c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f80f4067130 0x7f80f40695e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110159f20 0x7f8110149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 --2- 172.21.15.119:0/3097274680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110156350 0x7f81100cb6d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 >> 172.21.15.119:0/3097274680 conn(0x7f8110068e20 msgr2=0x7f8110154e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:59.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 shutdown_connections 2023-10-08T00:56:59.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.355+0000 7f8115b9d700 1 -- 172.21.15.119:0/3097274680 wait complete. 2023-10-08T00:56:59.373 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-10-08T00:56:59.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:59.373 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-10-08T00:56:59.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-10-08T00:56:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.637+0000 7fa61b5ff700 1 Processor -- start 2023-10-08T00:56:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.638+0000 7fa61b5ff700 1 -- start start 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa61b5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6140c8490 0x7fa6140c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa61b5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa614069e20 0x7fa6141569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa61b5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6140ac030 con 0x7fa614069a50 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa61b5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6140ab820 con 0x7fa614069e20 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa61b5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61406dbb0 con 0x7fa6140c8490 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa619b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa614069e20 0x7fa6141569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa619b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa614069e20 0x7fa6141569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44620/0 (socket says 172.21.15.119:44620) 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa619b9c700 1 -- 172.21.15.119:0/4156492311 learned_addr learned my addr 172.21.15.119:0/4156492311 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:59.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa618b9a700 1 --2- 172.21.15.119:0/4156492311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6140c8490 0x7fa6140c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa619b9c700 1 -- 172.21.15.119:0/4156492311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6140c8490 msgr2=0x7fa6140c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.639+0000 7fa619b9c700 1 --2- 172.21.15.119:0/4156492311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6140c8490 0x7fa6140c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.640+0000 7fa619b9c700 1 -- 172.21.15.119:0/4156492311 >> v1:172.21.15.184:6789/0 conn(0x7fa614069a50 legacy=0x7fa61415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.640+0000 7fa619b9c700 1 -- 172.21.15.119:0/4156492311 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6140aba70 con 0x7fa614069e20 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.640+0000 7fa619b9c700 1 --2- 172.21.15.119:0/4156492311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa614069e20 0x7fa6141569b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009f50 tx=0x7fa60c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d48571fbba4c22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.640+0000 7fa60bfff700 1 -- 172.21.15.119:0/4156492311 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0092e0 con 0x7fa614069e20 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.640+0000 7fa60bfff700 1 -- 172.21.15.119:0/4156492311 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c00ba10 con 0x7fa614069e20 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.640+0000 7fa60bfff700 1 -- 172.21.15.119:0/4156492311 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c00bdd0 con 0x7fa614069e20 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.641+0000 7fa61b5ff700 1 -- 172.21.15.119:0/4156492311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa614069e20 msgr2=0x7fa6141569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.641+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/4156492311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa614069e20 0x7fa6141569b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009f50 tx=0x7fa60c009840 comp rx=0 tx=0).stop 2023-10-08T00:56:59.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.641+0000 7fa61b5ff700 1 -- 172.21.15.119:0/4156492311 shutdown_connections 2023-10-08T00:56:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.641+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/4156492311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa614069e20 0x7fa6141569b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.641+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/4156492311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa6140c8490 0x7fa6140c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.641+0000 7fa61b5ff700 1 -- 172.21.15.119:0/4156492311 >> 172.21.15.119:0/4156492311 conn(0x7fa6140c8f90 msgr2=0x7fa6140c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.641+0000 7fa61b5ff700 1 -- 172.21.15.119:0/4156492311 shutdown_connections 2023-10-08T00:56:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.641+0000 7fa61b5ff700 1 -- 172.21.15.119:0/4156492311 wait complete. 2023-10-08T00:56:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.642+0000 7fa61b5ff700 1 Processor -- start 2023-10-08T00:56:59.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.642+0000 7fa61b5ff700 1 -- start start 2023-10-08T00:56:59.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa61b5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa614069a50 0x7fa6140af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa61b5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa614069e20 0x7fa6140c3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa61b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6140c8490 0x7fa6140c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa61b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6140af9b0 con 0x7fa6140c8490 2023-10-08T00:56:59.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa61b5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6140ac680 con 0x7fa614069e20 2023-10-08T00:56:59.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa61b5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6140b1b10 con 0x7fa614069a50 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa619b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6140c8490 0x7fa6140c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa619b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6140c8490 0x7fa6140c40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44630/0 (socket says 172.21.15.119:44630) 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.643+0000 7fa619b9c700 1 -- 172.21.15.119:0/3789552941 learned_addr learned my addr 172.21.15.119:0/3789552941 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa61939b700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa614069a50 0x7fa6140af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa618b9a700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa614069e20 0x7fa6140c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa619b9c700 1 -- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa614069a50 msgr2=0x7fa6140af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa619b9c700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa614069a50 0x7fa6140af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa619b9c700 1 -- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa614069e20 msgr2=0x7fa6140c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa619b9c700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa614069e20 0x7fa6140c3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa619b9c700 1 -- 172.21.15.119:0/3789552941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa60c009ae0 con 0x7fa6140c8490 2023-10-08T00:56:59.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa61939b700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa614069a50 0x7fa6140af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa619b9c700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6140c8490 0x7fa6140c40d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6140cc0b0 tx=0x7fa60c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa6097fa700 1 -- 172.21.15.119:0/3789552941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0092e0 con 0x7fa6140c8490 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa6097fa700 1 -- 172.21.15.119:0/3789552941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa60c019640 con 0x7fa6140c8490 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa6097fa700 1 -- 172.21.15.119:0/3789552941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0199e0 con 0x7fa6140c8490 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6140b1df0 con 0x7fa6140c8490 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.644+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6141553a0 con 0x7fa6140c8490 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.645+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6140ac7f0 con 0x7fa6140c8490 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.646+0000 7fa6097fa700 1 -- 172.21.15.119:0/3789552941 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 10) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fa60c003040 con 0x7fa6140c8490 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.647+0000 7fa6097fa700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa5f8067130 0x7fa5f80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:56:59.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.647+0000 7fa6097fa700 1 -- 172.21.15.119:0/3789552941 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fa60c00e070 con 0x7fa6140c8490 2023-10-08T00:56:59.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.647+0000 7fa61939b700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa5f8067130 0x7fa5f80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:56:59.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.648+0000 7fa61939b700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa5f8067130 0x7fa5f80695e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa610007fd0 tx=0x7fa610009040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:56:59.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.651+0000 7fa6097fa700 1 -- 172.21.15.119:0/3789552941 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa60c053180 con 0x7fa6140c8490 2023-10-08T00:56:59.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.868+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6140718b0 con 0x7fa5f8067130 2023-10-08T00:56:59.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.934+0000 7fa6097fa700 1 -- 172.21.15.119:0/3789552941 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa6140718b0 con 0x7fa5f8067130 2023-10-08T00:56:59.935 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir6 2023-10-08T00:56:59.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa5f8067130 msgr2=0x7fa5f80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa5f8067130 0x7fa5f80695e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa610007fd0 tx=0x7fa610009040 comp rx=0 tx=0).stop 2023-10-08T00:56:59.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6140c8490 msgr2=0x7fa6140c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6140c8490 0x7fa6140c40d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6140cc0b0 tx=0x7fa60c01a000 comp rx=0 tx=0).stop 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 shutdown_connections 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa614069a50 0x7fa6140af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fa5f8067130 0x7fa5f80695e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6140c8490 0x7fa6140c40d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 --2- 172.21.15.119:0/3789552941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa614069e20 0x7fa6140c3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 >> 172.21.15.119:0/3789552941 conn(0x7fa6140c8f90 msgr2=0x7fa614156870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 shutdown_connections 2023-10-08T00:56:59.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:56:59.937+0000 7fa61b5ff700 1 -- 172.21.15.119:0/3789552941 wait complete. 2023-10-08T00:56:59.955 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-10-08T00:56:59.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:56:59.955 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-10-08T00:56:59.993 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-10-08T00:57:00.180 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:57:00.179+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:57:00.180 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:57:00.179+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:57:00.180 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:57:00.180+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:57:00.180 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:57:00.180+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:57:00.180 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T00:57:00.180+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T00:57:00.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.227+0000 7f415f59e700 1 Processor -- start 2023-10-08T00:57:00.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.227+0000 7f415f59e700 1 -- start start 2023-10-08T00:57:00.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415f59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41600b03c0 0x7f41600cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41601591e0 0x7f4160159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415f59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41600cfa00 con 0x7f41600b0790 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415f59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41600cb580 con 0x7f41600b03c0 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41600cf160 con 0x7f41601591e0 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415e59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41600b03c0 0x7f41600cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415e59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f41600b03c0 0x7f41600cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44646/0 (socket says 172.21.15.119:44646) 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415e59c700 1 -- 172.21.15.119:0/1986265640 learned_addr learned my addr 172.21.15.119:0/1986265640 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415dd9b700 1 --1- 172.21.15.119:0/1986265640 >> v1:172.21.15.119:6789/0 conn(0x7f41600b0790 0x7f41600b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54854/0 (socket says 172.21.15.119:54854) 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415ed9d700 1 --2- 172.21.15.119:0/1986265640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41601591e0 0x7f4160159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415e59c700 1 -- 172.21.15.119:0/1986265640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41601591e0 msgr2=0x7f4160159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:00.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415e59c700 1 --2- 172.21.15.119:0/1986265640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41601591e0 0x7f4160159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415e59c700 1 -- 172.21.15.119:0/1986265640 >> v1:172.21.15.119:6789/0 conn(0x7f41600b0790 legacy=0x7f41600b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.229+0000 7f415e59c700 1 -- 172.21.15.119:0/1986265640 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41600cf440 con 0x7f41600b03c0 2023-10-08T00:57:00.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.230+0000 7f415e59c700 1 --2- 172.21.15.119:0/1986265640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41600b03c0 0x7f41600cea70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f41500099e0 tx=0x7f4150009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eefe468db39e94af server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:00.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.230+0000 7f415d59a700 1 -- 172.21.15.119:0/1986265640 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415000b820 con 0x7f41600b03c0 2023-10-08T00:57:00.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.230+0000 7f415d59a700 1 -- 172.21.15.119:0/1986265640 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415000b980 con 0x7f41600b03c0 2023-10-08T00:57:00.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.230+0000 7f415d59a700 1 -- 172.21.15.119:0/1986265640 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415000bd20 con 0x7f41600b03c0 2023-10-08T00:57:00.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.231+0000 7f415f59e700 1 -- 172.21.15.119:0/1986265640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41600b03c0 msgr2=0x7f41600cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.231+0000 7f415f59e700 1 --2- 172.21.15.119:0/1986265640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41600b03c0 0x7f41600cea70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f41500099e0 tx=0x7f4150009250 comp rx=0 tx=0).stop 2023-10-08T00:57:00.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.231+0000 7f415f59e700 1 -- 172.21.15.119:0/1986265640 shutdown_connections 2023-10-08T00:57:00.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.231+0000 7f415f59e700 1 --2- 172.21.15.119:0/1986265640 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41601591e0 0x7f4160159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.231+0000 7f415f59e700 1 --2- 172.21.15.119:0/1986265640 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41600b03c0 0x7f41600cea70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.231+0000 7f415f59e700 1 -- 172.21.15.119:0/1986265640 >> 172.21.15.119:0/1986265640 conn(0x7f4160063230 msgr2=0x7f4160063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:00.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.231+0000 7f415f59e700 1 -- 172.21.15.119:0/1986265640 shutdown_connections 2023-10-08T00:57:00.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.231+0000 7f415f59e700 1 -- 172.21.15.119:0/1986265640 wait complete. 2023-10-08T00:57:00.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.233+0000 7f415f59e700 1 Processor -- start 2023-10-08T00:57:00.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.233+0000 7f415f59e700 1 -- start start 2023-10-08T00:57:00.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41600b03c0 0x7f41600c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41600b0790 0x7f416014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41601591e0 0x7f41600694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415f59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416015a610 con 0x7f41601591e0 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41600c9a30 con 0x7f41600b0790 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416015a330 con 0x7f41600b03c0 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41601591e0 0x7f41600694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41601591e0 0x7f41600694c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44648/0 (socket says 172.21.15.119:44648) 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415ed9d700 1 -- 172.21.15.119:0/689493986 learned_addr learned my addr 172.21.15.119:0/689493986 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415e59c700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41600b03c0 0x7f41600c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415dd9b700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41600b0790 0x7f416014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415ed9d700 1 -- 172.21.15.119:0/689493986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41600b03c0 msgr2=0x7f41600c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415ed9d700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41600b03c0 0x7f41600c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415ed9d700 1 -- 172.21.15.119:0/689493986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41600b0790 msgr2=0x7f416014b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.234+0000 7f415ed9d700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41600b0790 0x7f416014b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.235+0000 7f415ed9d700 1 -- 172.21.15.119:0/689493986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41500094f0 con 0x7f41601591e0 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.235+0000 7f415ed9d700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41601591e0 0x7f41600694c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4158009fa0 tx=0x7f4158009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.235+0000 7f414e7fc700 1 -- 172.21.15.119:0/689493986 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415800ebc0 con 0x7f41601591e0 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.235+0000 7f414e7fc700 1 -- 172.21.15.119:0/689493986 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415800ed20 con 0x7f41601591e0 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.235+0000 7f414e7fc700 1 -- 172.21.15.119:0/689493986 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158014670 con 0x7f41601591e0 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.235+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41600c93e0 con 0x7f41601591e0 2023-10-08T00:57:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.235+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416014bf20 con 0x7f41601591e0 2023-10-08T00:57:00.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.237+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41600c9240 con 0x7f41601591e0 2023-10-08T00:57:00.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.237+0000 7f414e7fc700 1 -- 172.21.15.119:0/689493986 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 10) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f41580149e0 con 0x7f41601591e0 2023-10-08T00:57:00.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.239+0000 7f414e7fc700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4140067130 0x7f41400695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.239+0000 7f414e7fc700 1 -- 172.21.15.119:0/689493986 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f41580850b0 con 0x7f41601591e0 2023-10-08T00:57:00.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.239+0000 7f415e59c700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4140067130 0x7f41400695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.240+0000 7f415e59c700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4140067130 0x7f41400695e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f41500099e0 tx=0x7f4150017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:00.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.244+0000 7f414e7fc700 1 -- 172.21.15.119:0/689493986 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f41600c9240 con 0x7f41601591e0 2023-10-08T00:57:00.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.495+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f41600c9240 con 0x7f4140067130 2023-10-08T00:57:00.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.576+0000 7f414e7fc700 1 -- 172.21.15.119:0/689493986 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f41600c9240 con 0x7f4140067130 2023-10-08T00:57:00.576 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir7 2023-10-08T00:57:00.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4140067130 msgr2=0x7f41400695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4140067130 0x7f41400695e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f41500099e0 tx=0x7f4150017000 comp rx=0 tx=0).stop 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41601591e0 msgr2=0x7f41600694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41601591e0 0x7f41600694c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4158009fa0 tx=0x7f4158009900 comp rx=0 tx=0).stop 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 shutdown_connections 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f41600b03c0 0x7f41600c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4140067130 0x7f41400695e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41601591e0 0x7f41600694c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 --2- 172.21.15.119:0/689493986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f41600b0790 0x7f416014b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 >> 172.21.15.119:0/689493986 conn(0x7f4160063230 msgr2=0x7f4160068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 shutdown_connections 2023-10-08T00:57:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.578+0000 7f415f59e700 1 -- 172.21.15.119:0/689493986 wait complete. 2023-10-08T00:57:00.597 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-10-08T00:57:00.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:57:00.598 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-10-08T00:57:00.640 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-10-08T00:57:00.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.881+0000 7f4b62bff700 1 Processor -- start 2023-10-08T00:57:00.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.881+0000 7f4b62bff700 1 -- start start 2023-10-08T00:57:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b62bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b62bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4b5c0688d0 0x7f4b5c06e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b62bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5c0ae6d0 con 0x7f4b5c157740 2023-10-08T00:57:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b62bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c0acc00 con 0x7f4b5c0688d0 2023-10-08T00:57:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b62bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c06d6a0 con 0x7f4b5c0b2100 2023-10-08T00:57:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b6119c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4b5c0688d0 0x7f4b5c06e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b5bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b5bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:54704/0 (socket says 172.21.15.119:54704) 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b5bfff700 1 -- 172.21.15.119:0/1706191787 learned_addr learned my addr 172.21.15.119:0/1706191787 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.883+0000 7f4b6099b700 1 --1- 172.21.15.119:0/1706191787 >> v1:172.21.15.184:6789/0 conn(0x7f4b5c157740 0x7f4b5c0adfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:39292/0 (socket says 172.21.15.119:39292) 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.884+0000 7f4b5bfff700 1 -- 172.21.15.119:0/1706191787 >> v2:172.21.15.184:3300/0 conn(0x7f4b5c0688d0 msgr2=0x7f4b5c06e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.884+0000 7f4b5bfff700 1 --2- 172.21.15.119:0/1706191787 >> v2:172.21.15.184:3300/0 conn(0x7f4b5c0688d0 0x7f4b5c06e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.884+0000 7f4b5bfff700 1 -- 172.21.15.119:0/1706191787 >> v1:172.21.15.184:6789/0 conn(0x7f4b5c157740 legacy=0x7f4b5c0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.884+0000 7f4b5bfff700 1 -- 172.21.15.119:0/1706191787 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c06dc20 con 0x7f4b5c0b2100 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.884+0000 7f4b6119c700 1 --2- 172.21.15.119:0/1706191787 >> v2:172.21.15.184:3300/0 conn(0x7f4b5c0688d0 0x7f4b5c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T00:57:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.884+0000 7f4b5bfff700 1 --2- 172.21.15.119:0/1706191787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c068390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b480099b0 tx=0x7f4b48009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b85cbe5b403aa340 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:00.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.884+0000 7f4b5b7fe700 1 -- 172.21.15.119:0/1706191787 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b48009ea0 con 0x7f4b5c0b2100 2023-10-08T00:57:00.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b5b7fe700 1 -- 172.21.15.119:0/1706191787 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4800b860 con 0x7f4b5c0b2100 2023-10-08T00:57:00.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b5b7fe700 1 -- 172.21.15.119:0/1706191787 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800bc00 con 0x7f4b5c0b2100 2023-10-08T00:57:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b62bff700 1 -- 172.21.15.119:0/1706191787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 msgr2=0x7f4b5c068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1706191787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c068390 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4b480099b0 tx=0x7f4b48009250 comp rx=0 tx=0).stop 2023-10-08T00:57:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b62bff700 1 -- 172.21.15.119:0/1706191787 shutdown_connections 2023-10-08T00:57:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1706191787 >> v2:172.21.15.184:3300/0 conn(0x7f4b5c0688d0 0x7f4b5c06e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1706191787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c068390 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b62bff700 1 -- 172.21.15.119:0/1706191787 >> 172.21.15.119:0/1706191787 conn(0x7f4b5c0c8f90 msgr2=0x7f4b5c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b62bff700 1 -- 172.21.15.119:0/1706191787 shutdown_connections 2023-10-08T00:57:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.885+0000 7f4b62bff700 1 -- 172.21.15.119:0/1706191787 wait complete. 2023-10-08T00:57:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.887+0000 7f4b62bff700 1 Processor -- start 2023-10-08T00:57:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.887+0000 7f4b62bff700 1 -- start start 2023-10-08T00:57:00.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b62bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b5c0688d0 0x7f4b5c154ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b62bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c0c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b62bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c157740 0x7f4b5c0c4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b62bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c0c5b20 con 0x7f4b5c157740 2023-10-08T00:57:00.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b62bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c159eb0 con 0x7f4b5c0688d0 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b62bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c15a470 con 0x7f4b5c0b2100 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6119c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c157740 0x7f4b5c0c4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6119c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c157740 0x7f4b5c0c4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44650/0 (socket says 172.21.15.119:44650) 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6119c700 1 -- 172.21.15.119:0/1403202868 learned_addr learned my addr 172.21.15.119:0/1403202868 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6099b700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b5c0688d0 0x7f4b5c154ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b5bfff700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c0c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6119c700 1 -- 172.21.15.119:0/1403202868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 msgr2=0x7f4b5c0c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6119c700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c0c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6119c700 1 -- 172.21.15.119:0/1403202868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b5c0688d0 msgr2=0x7f4b5c154ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6119c700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b5c0688d0 0x7f4b5c154ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:00.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.888+0000 7f4b6119c700 1 -- 172.21.15.119:0/1403202868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54009a20 con 0x7f4b5c157740 2023-10-08T00:57:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.889+0000 7f4b6119c700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c157740 0x7f4b5c0c4f50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400c0f0 tx=0x7f4b5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.889+0000 7f4b58ff9700 1 -- 172.21.15.119:0/1403202868 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5400cc30 con 0x7f4b5c157740 2023-10-08T00:57:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.889+0000 7f4b58ff9700 1 -- 172.21.15.119:0/1403202868 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5400f040 con 0x7f4b5c157740 2023-10-08T00:57:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.889+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b480094f0 con 0x7f4b5c157740 2023-10-08T00:57:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.889+0000 7f4b58ff9700 1 -- 172.21.15.119:0/1403202868 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54014480 con 0x7f4b5c157740 2023-10-08T00:57:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.889+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c0c5560 con 0x7f4b5c157740 2023-10-08T00:57:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.890+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c0ac7a0 con 0x7f4b5c157740 2023-10-08T00:57:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.891+0000 7f4b58ff9700 1 -- 172.21.15.119:0/1403202868 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 10) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f4b54003020 con 0x7f4b5c157740 2023-10-08T00:57:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.892+0000 7f4b58ff9700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4b40067130 0x7f4b400695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.892+0000 7f4b58ff9700 1 -- 172.21.15.119:0/1403202868 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f4b540853d0 con 0x7f4b5c157740 2023-10-08T00:57:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.892+0000 7f4b6099b700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4b40067130 0x7f4b400695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.892+0000 7f4b6099b700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4b40067130 0x7f4b400695e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4b50004fd0 tx=0x7f4b50007eb0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:00.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:00.896+0000 7f4b58ff9700 1 -- 172.21.15.119:0/1403202868 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4b540521b0 con 0x7f4b5c157740 2023-10-08T00:57:01.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.143+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b5c0718b0 con 0x7f4b40067130 2023-10-08T00:57:01.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.213+0000 7f4b58ff9700 1 -- 172.21.15.119:0/1403202868 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f4b5c0718b0 con 0x7f4b40067130 2023-10-08T00:57:01.214 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir8 2023-10-08T00:57:01.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4b40067130 msgr2=0x7f4b400695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4b40067130 0x7f4b400695e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4b50004fd0 tx=0x7f4b50007eb0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c157740 msgr2=0x7f4b5c0c4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c157740 0x7f4b5c0c4f50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4b5400c0f0 tx=0x7f4b5400b040 comp rx=0 tx=0).stop 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 shutdown_connections 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4b40067130 0x7f4b400695e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c157740 0x7f4b5c0c4f50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4b5c0688d0 0x7f4b5c154ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 --2- 172.21.15.119:0/1403202868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4b5c0b2100 0x7f4b5c0c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 >> 172.21.15.119:0/1403202868 conn(0x7f4b5c0c8f90 msgr2=0x7f4b5c156a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:01.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 shutdown_connections 2023-10-08T00:57:01.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.216+0000 7f4b62bff700 1 -- 172.21.15.119:0/1403202868 wait complete. 2023-10-08T00:57:01.233 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-10-08T00:57:01.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T00:57:01.233 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-10-08T00:57:01.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-10-08T00:57:01.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.502+0000 7ff28c7ff700 1 Processor -- start 2023-10-08T00:57:01.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.502+0000 7ff28c7ff700 1 -- start start 2023-10-08T00:57:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.503+0000 7ff28c7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 0x7ff28406d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.503+0000 7ff28c7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff28415a190 0x7ff28406d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.503+0000 7ff28c7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2840b4570 con 0x7ff284153f50 2023-10-08T00:57:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.503+0000 7ff28c7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2840b2710 con 0x7ff28415a190 2023-10-08T00:57:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.503+0000 7ff28c7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28415a730 con 0x7ff284154320 2023-10-08T00:57:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff28a59b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff284153f50 0x7ff2840b2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54864/0 (socket says 172.21.15.119:54864) 2023-10-08T00:57:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff28a59b700 1 -- 172.21.15.119:0/1720270573 learned_addr learned my addr 172.21.15.119:0/1720270573 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:57:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff28ad9c700 1 --2- 172.21.15.119:0/1720270573 >> v2:172.21.15.184:3300/0 conn(0x7ff28415a190 0x7ff28406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181261210 0 0) 0x7ff2840b4570 con 0x7ff284153f50 2023-10-08T00:57:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289d9a700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 0x7ff28406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff270003570 con 0x7ff284153f50 2023-10-08T00:57:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2632317754 0 0) 0x7ff270003570 con 0x7ff284153f50 2023-10-08T00:57:01.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 msgr2=0x7ff28406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 0x7ff28406d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 >> v2:172.21.15.184:3300/0 conn(0x7ff28415a190 msgr2=0x7ff28406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 --2- 172.21.15.119:0/1720270573 >> v2:172.21.15.184:3300/0 conn(0x7ff28415a190 0x7ff28406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28415aa10 con 0x7ff284153f50 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7ff280001470 con 0x7ff284153f50 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 >> v1:172.21.15.119:6789/0 conn(0x7ff284153f50 legacy=0x7ff2840b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.504+0000 7ff289599700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 0x7ff270003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289599700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff270003fe0 0x7ff270004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289599700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff270004d50 0x7ff2700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840b2710 con 0x7ff270003fe0 2023-10-08T00:57:01.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28415a730 con 0x7ff270004d50 2023-10-08T00:57:01.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27000ba90 con 0x7ff284154320 2023-10-08T00:57:01.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289d9a700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff270003fe0 0x7ff270004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:01.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff28a59b700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 0x7ff270003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:01.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff28ad9c700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff270004d50 0x7ff2700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:01.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289d9a700 1 -- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 msgr2=0x7ff270003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:01.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289d9a700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 0x7ff270003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289d9a700 1 -- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff270004d50 msgr2=0x7ff2700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:01.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289d9a700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff270004d50 0x7ff2700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289d9a700 1 -- 172.21.15.119:0/1720270573 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff270003570 con 0x7ff270003fe0 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff28a59b700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284154320 0x7ff270003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.505+0000 7ff289d9a700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff270003fe0 0x7ff270004810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff284154770 tx=0x7ff2740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3b93eac5eb788c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400e6b0 con 0x7ff270003fe0 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27400e810 con 0x7ff270003fe0 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff289599700 1 -- 172.21.15.119:0/1720270573 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27400ebd0 con 0x7ff270003fe0 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff28c7ff700 1 -- 172.21.15.119:0/1720270573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff270003fe0 msgr2=0x7ff270004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff28c7ff700 1 --2- 172.21.15.119:0/1720270573 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff270003fe0 0x7ff270004810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff284154770 tx=0x7ff2740091a0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff28ad9c700 1 -- 172.21.15.119:0/1720270573 reap_dead start 2023-10-08T00:57:01.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff28c7ff700 1 -- 172.21.15.119:0/1720270573 shutdown_connections 2023-10-08T00:57:01.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff28c7ff700 1 -- 172.21.15.119:0/1720270573 >> 172.21.15.119:0/1720270573 conn(0x7ff28406baf0 msgr2=0x7ff28406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:01.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff28c7ff700 1 -- 172.21.15.119:0/1720270573 shutdown_connections 2023-10-08T00:57:01.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.506+0000 7ff28c7ff700 1 -- 172.21.15.119:0/1720270573 wait complete. 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.508+0000 7ff28c7ff700 1 Processor -- start 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.508+0000 7ff28c7ff700 1 -- start start 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff28c7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff270003fe0 0x7ff284151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff28c7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840b0040 0x7ff2840b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff28c7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284159370 0x7ff284159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff28c7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff28416a510 con 0x7ff2840b0040 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff28c7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2840b09f0 con 0x7ff270003fe0 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff28c7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28416b300 con 0x7ff284159370 2023-10-08T00:57:01.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840b0040 0x7ff2840b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840b0040 0x7ff2840b04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44666/0 (socket says 172.21.15.119:44666) 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 -- 172.21.15.119:0/135482135 learned_addr learned my addr 172.21.15.119:0/135482135 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 -- 172.21.15.119:0/135482135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284159370 msgr2=0x7ff284159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284159370 0x7ff284159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 -- 172.21.15.119:0/135482135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff270003fe0 msgr2=0x7ff284151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff270003fe0 0x7ff284151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 -- 172.21.15.119:0/135482135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284151950 con 0x7ff2840b0040 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.509+0000 7ff289d9a700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840b0040 0x7ff2840b04b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff270004730 tx=0x7ff27401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:01.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.510+0000 7ff27a7fc700 1 -- 172.21.15.119:0/135482135 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274022070 con 0x7ff2840b0040 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.510+0000 7ff27a7fc700 1 -- 172.21.15.119:0/135482135 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff27400e980 con 0x7ff2840b0040 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.510+0000 7ff27a7fc700 1 -- 172.21.15.119:0/135482135 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274027470 con 0x7ff2840b0040 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.510+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff284159d50 con 0x7ff2840b0040 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.510+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff28416a9d0 con 0x7ff2840b0040 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.511+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2840718b0 con 0x7ff2840b0040 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.512+0000 7ff27a7fc700 1 -- 172.21.15.119:0/135482135 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7ff27401c450 con 0x7ff2840b0040 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.513+0000 7ff27a7fc700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2680671c0 0x7ff268069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.513+0000 7ff27a7fc700 1 -- 172.21.15.119:0/135482135 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7ff274086570 con 0x7ff2840b0040 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.513+0000 7ff28a59b700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2680671c0 0x7ff268069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:01.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.514+0000 7ff28a59b700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2680671c0 0x7ff268069670 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff2800015e0 tx=0x7ff2840b3110 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:01.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.517+0000 7ff27a7fc700 1 -- 172.21.15.119:0/135482135 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2740533f0 con 0x7ff2840b0040 2023-10-08T00:57:01.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.731+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff28416b440 con 0x7ff2680671c0 2023-10-08T00:57:01.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.802+0000 7ff27a7fc700 1 -- 172.21.15.119:0/135482135 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff28416b440 con 0x7ff2680671c0 2023-10-08T00:57:01.804 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/dir9 2023-10-08T00:57:01.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2680671c0 msgr2=0x7ff268069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:01.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2680671c0 0x7ff268069670 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff2800015e0 tx=0x7ff2840b3110 comp rx=0 tx=0).stop 2023-10-08T00:57:01.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840b0040 msgr2=0x7ff2840b04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:01.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840b0040 0x7ff2840b04b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff270004730 tx=0x7ff27401a000 comp rx=0 tx=0).stop 2023-10-08T00:57:01.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 shutdown_connections 2023-10-08T00:57:01.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff284159370 0x7ff284159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2680671c0 0x7ff268069670 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff270003fe0 0x7ff284151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 --2- 172.21.15.119:0/135482135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2840b0040 0x7ff2840b04b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:01.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.804+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 >> 172.21.15.119:0/135482135 conn(0x7ff28406baf0 msgr2=0x7ff2840c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:01.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.805+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 shutdown_connections 2023-10-08T00:57:01.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:01.805+0000 7ff28c7ff700 1 -- 172.21.15.119:0/135482135 wait complete. 2023-10-08T00:57:01.822 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-10-08T00:57:01.822 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-10-08T00:57:02.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.010+0000 7f868df9d700 1 Processor -- start 2023-10-08T00:57:02.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.010+0000 7f868df9d700 1 -- start start 2023-10-08T00:57:02.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.011+0000 7f868df9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8688153850 0x7f86881596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.011+0000 7f868df9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868815a030 con 0x7f8688153480 2023-10-08T00:57:02.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.011+0000 7f868df9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868815a480 con 0x7f8688159bf0 2023-10-08T00:57:02.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.011+0000 7f868df9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86880afb40 con 0x7f8688153850 2023-10-08T00:57:02.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.011+0000 7f868cf9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8688153480 0x7f86880ae750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54880/0 (socket says 172.21.15.119:54880) 2023-10-08T00:57:02.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.011+0000 7f868cf9b700 1 -- 172.21.15.119:0/3056755255 learned_addr learned my addr 172.21.15.119:0/3056755255 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:57:02.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.011+0000 7f8687fff700 1 --2- 172.21.15.119:0/3056755255 >> v2:172.21.15.184:3300/0 conn(0x7f8688153850 0x7f86881596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.012+0000 7f86877fe700 1 -- 172.21.15.119:0/3056755255 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895175458 0 0) 0x7f868815a030 con 0x7f8688153480 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.012+0000 7f86877fe700 1 -- 172.21.15.119:0/3056755255 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8678003570 con 0x7f8688153480 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.012+0000 7f86877fe700 1 -- 172.21.15.119:0/3056755255 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3575042712 0 0) 0x7f8678003570 con 0x7f8688153480 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.012+0000 7f8687fff700 1 -- 172.21.15.119:0/3056755255 >> v1:172.21.15.184:6789/0 conn(0x7f8688159bf0 legacy=0x7f8688157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.012+0000 7f8687fff700 1 -- 172.21.15.119:0/3056755255 >> v1:172.21.15.119:6789/0 conn(0x7f8688153480 legacy=0x7f86880ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.012+0000 7f8687fff700 1 -- 172.21.15.119:0/3056755255 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86880afe20 con 0x7f8688153850 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.012+0000 7f8687fff700 1 --2- 172.21.15.119:0/3056755255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688153850 0x7f86881596b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86740098e0 tx=0x7f86740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a64e1eec0c7ff68 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f86877fe700 1 -- 172.21.15.119:0/3056755255 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c001470 con 0x7f8688153850 2023-10-08T00:57:02.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f86877fe700 1 -- 172.21.15.119:0/3056755255 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674011420 con 0x7f8688153850 2023-10-08T00:57:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f86877fe700 1 -- 172.21.15.119:0/3056755255 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86740117c0 con 0x7f8688153850 2023-10-08T00:57:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f868df9d700 1 -- 172.21.15.119:0/3056755255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688153850 msgr2=0x7f86881596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f868df9d700 1 --2- 172.21.15.119:0/3056755255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688153850 0x7f86881596b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f86740098e0 tx=0x7f86740091a0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f868df9d700 1 -- 172.21.15.119:0/3056755255 shutdown_connections 2023-10-08T00:57:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f868df9d700 1 --2- 172.21.15.119:0/3056755255 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688153850 0x7f86881596b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f868df9d700 1 -- 172.21.15.119:0/3056755255 >> 172.21.15.119:0/3056755255 conn(0x7f8688068e20 msgr2=0x7f86880cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f868df9d700 1 -- 172.21.15.119:0/3056755255 shutdown_connections 2023-10-08T00:57:02.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.013+0000 7f868df9d700 1 -- 172.21.15.119:0/3056755255 wait complete. 2023-10-08T00:57:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.015+0000 7f868df9d700 1 Processor -- start 2023-10-08T00:57:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.015+0000 7f868df9d700 1 -- start start 2023-10-08T00:57:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f868df9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8688153480 0x7f8688149d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f868df9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8688153850 0x7f8688150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f868df9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688159bf0 0x7f868814cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f868df9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8688169860 con 0x7f8688153850 2023-10-08T00:57:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f868df9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86880c9610 con 0x7f8688159bf0 2023-10-08T00:57:02.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f868df9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86880c98f0 con 0x7f8688153480 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f8687fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8688153850 0x7f8688150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f8687fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8688153850 0x7f8688150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44668/0 (socket says 172.21.15.119:44668) 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f8687fff700 1 -- 172.21.15.119:0/126314962 learned_addr learned my addr 172.21.15.119:0/126314962 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f868d79c700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688159bf0 0x7f868814cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f868cf9b700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8688153480 0x7f8688149d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f8687fff700 1 -- 172.21.15.119:0/126314962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8688153480 msgr2=0x7f8688149d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f8687fff700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8688153480 0x7f8688149d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f8687fff700 1 -- 172.21.15.119:0/126314962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688159bf0 msgr2=0x7f868814cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f8687fff700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688159bf0 0x7f868814cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.016+0000 7f8687fff700 1 -- 172.21.15.119:0/126314962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8674009420 con 0x7f8688153850 2023-10-08T00:57:02.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.017+0000 7f8687fff700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8688153850 0x7f8688150790 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f86740030c0 tx=0x7f8674017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:02.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.017+0000 7f8684ff9700 1 -- 172.21.15.119:0/126314962 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674010b70 con 0x7f8688153850 2023-10-08T00:57:02.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.017+0000 7f8684ff9700 1 -- 172.21.15.119:0/126314962 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674010cd0 con 0x7f8688153850 2023-10-08T00:57:02.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.017+0000 7f8684ff9700 1 -- 172.21.15.119:0/126314962 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867401d660 con 0x7f8688153850 2023-10-08T00:57:02.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.017+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86880c9bd0 con 0x7f8688153850 2023-10-08T00:57:02.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.017+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868814a290 con 0x7f8688153850 2023-10-08T00:57:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.018+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86881699b0 con 0x7f8688153850 2023-10-08T00:57:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.019+0000 7f8684ff9700 1 -- 172.21.15.119:0/126314962 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f867401da10 con 0x7f8688153850 2023-10-08T00:57:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.020+0000 7f8684ff9700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f866c0671c0 0x7f866c069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.020+0000 7f8684ff9700 1 -- 172.21.15.119:0/126314962 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f86740862f0 con 0x7f8688153850 2023-10-08T00:57:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.020+0000 7f868cf9b700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f866c0671c0 0x7f866c069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.021+0000 7f868cf9b700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f866c0671c0 0x7f866c069670 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f867c004dd0 tx=0x7f867c001a00 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:02.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.024+0000 7f8684ff9700 1 -- 172.21.15.119:0/126314962 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86881699b0 con 0x7f8688153850 2023-10-08T00:57:02.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.242+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f86881699b0 con 0x7f866c0671c0 2023-10-08T00:57:02.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.295+0000 7f8684ff9700 1 -- 172.21.15.119:0/126314962 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f86881699b0 con 0x7f866c0671c0 2023-10-08T00:57:02.295 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-10-08T00:57:02.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f866c0671c0 msgr2=0x7f866c069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f866c0671c0 0x7f866c069670 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f867c004dd0 tx=0x7f867c001a00 comp rx=0 tx=0).stop 2023-10-08T00:57:02.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8688153850 msgr2=0x7f8688150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8688153850 0x7f8688150790 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f86740030c0 tx=0x7f8674017000 comp rx=0 tx=0).stop 2023-10-08T00:57:02.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 shutdown_connections 2023-10-08T00:57:02.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8688153480 0x7f8688149d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f866c0671c0 0x7f866c069670 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8688159bf0 0x7f868814cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 --2- 172.21.15.119:0/126314962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8688153850 0x7f8688150790 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 >> 172.21.15.119:0/126314962 conn(0x7f8688068e20 msgr2=0x7f868806e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 shutdown_connections 2023-10-08T00:57:02.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.297+0000 7f868df9d700 1 -- 172.21.15.119:0/126314962 wait complete. 2023-10-08T00:57:02.314 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-10-08T00:57:02.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-10-08T00:57:02.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.501+0000 7fdcef59e700 1 Processor -- start 2023-10-08T00:57:02.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.502+0000 7fdcef59e700 1 -- start start 2023-10-08T00:57:02.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.503+0000 7fdcef59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdcf00b0c00 0x7fdcf00ad570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.503+0000 7fdcef59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf00b0570 con 0x7fdcf00b0760 2023-10-08T00:57:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.503+0000 7fdcef59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf00af4d0 con 0x7fdcf0156350 2023-10-08T00:57:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.503+0000 7fdcef59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf00af7b0 con 0x7fdcf00b0c00 2023-10-08T00:57:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.503+0000 7fdcedd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdcf00b0760 0x7fdcf00ace60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54886/0 (socket says 172.21.15.119:54886) 2023-10-08T00:57:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.503+0000 7fdcedd9b700 1 -- 172.21.15.119:0/2229440614 learned_addr learned my addr 172.21.15.119:0/2229440614 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:57:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.503+0000 7fdceed9d700 1 --2- 172.21.15.119:0/2229440614 >> v2:172.21.15.184:3300/0 conn(0x7fdcf00b0c00 0x7fdcf00ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3291312999 0 0) 0x7fdcf00b0570 con 0x7fdcf00b0760 2023-10-08T00:57:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd8003570 con 0x7fdcf00b0760 2023-10-08T00:57:02.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3736852681 0 0) 0x7fdcd8003570 con 0x7fdcf00b0760 2023-10-08T00:57:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 >> v2:172.21.15.184:3300/0 conn(0x7fdcf00b0c00 msgr2=0x7fdcf00ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 --2- 172.21.15.119:0/2229440614 >> v2:172.21.15.184:3300/0 conn(0x7fdcf00b0c00 0x7fdcf00ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 >> v1:172.21.15.184:6789/0 conn(0x7fdcf0156350 legacy=0x7fdcf0157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf00afa90 con 0x7fdcf00b0760 2023-10-08T00:57:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fdce4001470 con 0x7fdcf00b0760 2023-10-08T00:57:02.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 >> v1:172.21.15.119:6789/0 conn(0x7fdcf00b0760 legacy=0x7fdcf00ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd8003a60 0x7fdcd8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8004810 0x7fdcd8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcd80051c0 0x7fdcd800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf00af7b0 con 0x7fdcd8003a60 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd800c130 con 0x7fdcd8004810 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd800e500 con 0x7fdcd80051c0 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdcee59c700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd8003a60 0x7fdcd8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdcedd9b700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8004810 0x7fdcd8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.504+0000 7fdceed9d700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcd80051c0 0x7fdcd800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdcee59c700 1 -- 172.21.15.119:0/2229440614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcd80051c0 msgr2=0x7fdcd800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdcee59c700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcd80051c0 0x7fdcd800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdcee59c700 1 -- 172.21.15.119:0/2229440614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8004810 msgr2=0x7fdcd8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdcee59c700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8004810 0x7fdcd8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdcee59c700 1 -- 172.21.15.119:0/2229440614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd8003570 con 0x7fdcd8003a60 2023-10-08T00:57:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdceed9d700 1 -- 172.21.15.119:0/2229440614 reap_dead start 2023-10-08T00:57:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdcee59c700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd8003a60 0x7fdcd8003e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdce0002070 tx=0x7fdce00016d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fac945cf7e6f70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce00019e0 con 0x7fdcd8003a60 2023-10-08T00:57:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce000c8b0 con 0x7fdcd8003a60 2023-10-08T00:57:02.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.505+0000 7fdced59a700 1 -- 172.21.15.119:0/2229440614 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce000cc50 con 0x7fdcd8003a60 2023-10-08T00:57:02.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.506+0000 7fdcef59e700 1 -- 172.21.15.119:0/2229440614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd8003a60 msgr2=0x7fdcd8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.506+0000 7fdcef59e700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd8003a60 0x7fdcd8003e30 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdce0002070 tx=0x7fdce00016d0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.506+0000 7fdcef59e700 1 -- 172.21.15.119:0/2229440614 shutdown_connections 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.506+0000 7fdcef59e700 1 --2- 172.21.15.119:0/2229440614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcd8003a60 0x7fdcd8003e30 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.506+0000 7fdcef59e700 1 -- 172.21.15.119:0/2229440614 >> 172.21.15.119:0/2229440614 conn(0x7fdcf0068e20 msgr2=0x7fdcf00cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.506+0000 7fdcef59e700 1 -- 172.21.15.119:0/2229440614 shutdown_connections 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.506+0000 7fdcef59e700 1 -- 172.21.15.119:0/2229440614 wait complete. 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.507+0000 7fdcef59e700 1 Processor -- start 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.507+0000 7fdcef59e700 1 -- start start 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcef59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8003a60 0x7fdcf0150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcef59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcf014d7f0 0x7fdcf014dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcef59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcf013a760 0x7fdcf013abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcef59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf013b100 con 0x7fdcf014d7f0 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcef59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf014b4b0 con 0x7fdcd8003a60 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcef59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf014e290 con 0x7fdcf013a760 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcedd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcf014d7f0 0x7fdcf014dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcedd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcf014d7f0 0x7fdcf014dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44680/0 (socket says 172.21.15.119:44680) 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.508+0000 7fdcedd9b700 1 -- 172.21.15.119:0/262170982 learned_addr learned my addr 172.21.15.119:0/262170982 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcee59c700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8003a60 0x7fdcf0150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdceed9d700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcf013a760 0x7fdcf013abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcedd9b700 1 -- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcf013a760 msgr2=0x7fdcf013abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcedd9b700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcf013a760 0x7fdcf013abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcedd9b700 1 -- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8003a60 msgr2=0x7fdcf0150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcedd9b700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8003a60 0x7fdcf0150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcedd9b700 1 -- 172.21.15.119:0/262170982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf014b770 con 0x7fdcf014d7f0 2023-10-08T00:57:02.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcee59c700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8003a60 0x7fdcf0150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:57:02.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdceed9d700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcf013a760 0x7fdcf013abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T00:57:02.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcedd9b700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcf014d7f0 0x7fdcf014dc60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdce40070e0 tx=0x7fdce4001da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:02.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcde7fc700 1 -- 172.21.15.119:0/262170982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400f5b0 con 0x7fdcf014d7f0 2023-10-08T00:57:02.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcde7fc700 1 -- 172.21.15.119:0/262170982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce400f710 con 0x7fdcf014d7f0 2023-10-08T00:57:02.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf014ba50 con 0x7fdcf014d7f0 2023-10-08T00:57:02.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcde7fc700 1 -- 172.21.15.119:0/262170982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400fa90 con 0x7fdcf014d7f0 2023-10-08T00:57:02.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.509+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf00caf20 con 0x7fdcf014d7f0 2023-10-08T00:57:02.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.510+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf014bf70 con 0x7fdcf014d7f0 2023-10-08T00:57:02.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.511+0000 7fdcde7fc700 1 -- 172.21.15.119:0/262170982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7fdce401d460 con 0x7fdcf014d7f0 2023-10-08T00:57:02.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.512+0000 7fdcde7fc700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdcd00671c0 0x7fdcd0069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.512+0000 7fdcde7fc700 1 -- 172.21.15.119:0/262170982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fdce4084b70 con 0x7fdcf014d7f0 2023-10-08T00:57:02.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.512+0000 7fdcee59c700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdcd00671c0 0x7fdcd0069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.513+0000 7fdcee59c700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdcd00671c0 0x7fdcd0069670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdce0002070 tx=0x7fdce0015000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:02.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.516+0000 7fdcde7fc700 1 -- 172.21.15.119:0/262170982 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcf014bf70 con 0x7fdcf014d7f0 2023-10-08T00:57:02.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.730+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcf014bf70 con 0x7fdcd00671c0 2023-10-08T00:57:02.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.781+0000 7fdcde7fc700 1 -- 172.21.15.119:0/262170982 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdcf014bf70 con 0x7fdcd00671c0 2023-10-08T00:57:02.782 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-10-08T00:57:02.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.783+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdcd00671c0 msgr2=0x7fdcd0069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdcd00671c0 0x7fdcd0069670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdce0002070 tx=0x7fdce0015000 comp rx=0 tx=0).stop 2023-10-08T00:57:02.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcf014d7f0 msgr2=0x7fdcf014dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcf014d7f0 0x7fdcf014dc60 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdce40070e0 tx=0x7fdce4001da0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 shutdown_connections 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdcf013a760 0x7fdcf013abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdcd00671c0 0x7fdcd0069670 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdcd8003a60 0x7fdcf0150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 --2- 172.21.15.119:0/262170982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdcf014d7f0 0x7fdcf014dc60 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 >> 172.21.15.119:0/262170982 conn(0x7fdcf0068e20 msgr2=0x7fdcf0156720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 shutdown_connections 2023-10-08T00:57:02.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.784+0000 7fdcef59e700 1 -- 172.21.15.119:0/262170982 wait complete. 2023-10-08T00:57:02.802 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-10-08T00:57:02.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-10-08T00:57:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.988+0000 7f926599d700 1 Processor -- start 2023-10-08T00:57:02.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.989+0000 7f926599d700 1 -- start start 2023-10-08T00:57:02.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f926599d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f92600685e0 0x7f926015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f926599d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f926015d3f0 0x7f92601538f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f926599d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92600c9480 con 0x7f92600689b0 2023-10-08T00:57:02.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f926599d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926015d880 con 0x7f926015d3f0 2023-10-08T00:57:02.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f926599d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926015db60 con 0x7f92600685e0 2023-10-08T00:57:02.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f925ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f926015d3f0 0x7f92601538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f925ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f926015d3f0 0x7f92601538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44682/0 (socket says 172.21.15.119:44682) 2023-10-08T00:57:02.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f925ffff700 1 -- 172.21.15.119:0/1431153436 learned_addr learned my addr 172.21.15.119:0/1431153436 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:02.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.990+0000 7f926499b700 1 --2- 172.21.15.119:0/1431153436 >> v2:172.21.15.184:3300/0 conn(0x7f92600685e0 0x7f926015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.991+0000 7f925ffff700 1 -- 172.21.15.119:0/1431153436 >> v2:172.21.15.184:3300/0 conn(0x7f92600685e0 msgr2=0x7f926015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:02.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.991+0000 7f925ffff700 1 --2- 172.21.15.119:0/1431153436 >> v2:172.21.15.184:3300/0 conn(0x7f92600685e0 0x7f926015a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.991+0000 7f925ffff700 1 -- 172.21.15.119:0/1431153436 >> v1:172.21.15.184:6789/0 conn(0x7f92600689b0 legacy=0x7f926006eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.991+0000 7f925ffff700 1 -- 172.21.15.119:0/1431153436 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926015de40 con 0x7f926015d3f0 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.991+0000 7f925ffff700 1 --2- 172.21.15.119:0/1431153436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926015d3f0 0x7f92601538f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9260065920 tx=0x7f924c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1c6839298eff64 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.991+0000 7f925f7fe700 1 -- 172.21.15.119:0/1431153436 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00b8f0 con 0x7f926015d3f0 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.991+0000 7f925f7fe700 1 -- 172.21.15.119:0/1431153436 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c00ba50 con 0x7f926015d3f0 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.991+0000 7f925f7fe700 1 -- 172.21.15.119:0/1431153436 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00bdf0 con 0x7f926015d3f0 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.992+0000 7f926599d700 1 -- 172.21.15.119:0/1431153436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926015d3f0 msgr2=0x7f92601538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.992+0000 7f926599d700 1 --2- 172.21.15.119:0/1431153436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926015d3f0 0x7f92601538f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9260065920 tx=0x7f924c009250 comp rx=0 tx=0).stop 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.992+0000 7f926599d700 1 -- 172.21.15.119:0/1431153436 shutdown_connections 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.992+0000 7f926599d700 1 --2- 172.21.15.119:0/1431153436 >> v2:172.21.15.184:3300/0 conn(0x7f92600685e0 0x7f926015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.992+0000 7f926599d700 1 --2- 172.21.15.119:0/1431153436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f926015d3f0 0x7f92601538f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.992+0000 7f926599d700 1 -- 172.21.15.119:0/1431153436 >> 172.21.15.119:0/1431153436 conn(0x7f926006b420 msgr2=0x7f92600cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:02.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.992+0000 7f926599d700 1 -- 172.21.15.119:0/1431153436 shutdown_connections 2023-10-08T00:57:02.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.992+0000 7f926599d700 1 -- 172.21.15.119:0/1431153436 wait complete. 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.993+0000 7f926599d700 1 Processor -- start 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.993+0000 7f926599d700 1 -- start start 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.994+0000 7f926599d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92600685e0 0x7f92600b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.994+0000 7f926599d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92600689b0 0x7f92600b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.994+0000 7f926599d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f926015d3f0 0x7f9260155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.994+0000 7f926599d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b3210 con 0x7f92600685e0 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.994+0000 7f926599d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600c4d70 con 0x7f92600689b0 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.994+0000 7f926599d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92600b2840 con 0x7f926015d3f0 2023-10-08T00:57:02.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92600685e0 0x7f92600b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92600685e0 0x7f92600b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44690/0 (socket says 172.21.15.119:44690) 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 -- 172.21.15.119:0/900107399 learned_addr learned my addr 172.21.15.119:0/900107399 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f925ffff700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92600689b0 0x7f92600b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926519c700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f926015d3f0 0x7f9260155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 -- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f926015d3f0 msgr2=0x7f9260155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f926015d3f0 0x7f9260155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 -- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92600689b0 msgr2=0x7f92600b1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92600689b0 0x7f92600b1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 -- 172.21.15.119:0/900107399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c009490 con 0x7f92600685e0 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926519c700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f926015d3f0 0x7f9260155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:57:02.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926499b700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92600685e0 0x7f92600b2cd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f925400d8c0 tx=0x7f9254008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:02.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f925cff9700 1 -- 172.21.15.119:0/900107399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400de60 con 0x7f92600685e0 2023-10-08T00:57:02.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f925cff9700 1 -- 172.21.15.119:0/900107399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925400b780 con 0x7f92600685e0 2023-10-08T00:57:02.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f925cff9700 1 -- 172.21.15.119:0/900107399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925400bb20 con 0x7f92600685e0 2023-10-08T00:57:02.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.995+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92600b2510 con 0x7f92600685e0 2023-10-08T00:57:02.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.996+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92600c4fc0 con 0x7f92600685e0 2023-10-08T00:57:02.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.997+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926004c3c0 con 0x7f92600685e0 2023-10-08T00:57:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.998+0000 7f925cff9700 1 -- 172.21.15.119:0/900107399 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f925401e460 con 0x7f92600685e0 2023-10-08T00:57:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.999+0000 7f925cff9700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92440671c0 0x7f9244069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.999+0000 7f925cff9700 1 -- 172.21.15.119:0/900107399 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f9254085080 con 0x7f92600685e0 2023-10-08T00:57:02.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:02.999+0000 7f925ffff700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92440671c0 0x7f9244069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.000+0000 7f925ffff700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92440671c0 0x7f9244069670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f92600b3410 tx=0x7f924c017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:03.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.003+0000 7f925cff9700 1 -- 172.21.15.119:0/900107399 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9254051f60 con 0x7f92600685e0 2023-10-08T00:57:03.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.224+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f92600b2980 con 0x7f92440671c0 2023-10-08T00:57:03.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.275+0000 7f925cff9700 1 -- 172.21.15.119:0/900107399 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f924c00bbc0 con 0x7f92440671c0 2023-10-08T00:57:03.276 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-10-08T00:57:03.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.277+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92440671c0 msgr2=0x7f9244069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.277+0000 7f926599d700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92440671c0 0x7f9244069670 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f92600b3410 tx=0x7f924c017000 comp rx=0 tx=0).stop 2023-10-08T00:57:03.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92600685e0 msgr2=0x7f92600b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92600685e0 0x7f92600b2cd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f925400d8c0 tx=0x7f9254008040 comp rx=0 tx=0).stop 2023-10-08T00:57:03.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 shutdown_connections 2023-10-08T00:57:03.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f926015d3f0 0x7f9260155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92440671c0 0x7f9244069670 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92600685e0 0x7f92600b2cd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 --2- 172.21.15.119:0/900107399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92600689b0 0x7f92600b1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 >> 172.21.15.119:0/900107399 conn(0x7f926006b420 msgr2=0x7f92600c31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:03.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 shutdown_connections 2023-10-08T00:57:03.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.278+0000 7f926599d700 1 -- 172.21.15.119:0/900107399 wait complete. 2023-10-08T00:57:03.295 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-10-08T00:57:03.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-10-08T00:57:03.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.480+0000 7fe67d5ff700 1 Processor -- start 2023-10-08T00:57:03.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.480+0000 7fe67d5ff700 1 -- start start 2023-10-08T00:57:03.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.481+0000 7fe67d5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe678069a50 0x7fe6780c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.481+0000 7fe67d5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe678069e20 0x7fe6781569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.481+0000 7fe67d5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6780ac030 con 0x7fe6780c8490 2023-10-08T00:57:03.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe67d5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6780ab820 con 0x7fe678069e20 2023-10-08T00:57:03.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe67d5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67806dbb0 con 0x7fe678069a50 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe6777fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe678069e20 0x7fe6781569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe6777fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe678069e20 0x7fe6781569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44706/0 (socket says 172.21.15.119:44706) 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe6777fe700 1 -- 172.21.15.119:0/3698497979 learned_addr learned my addr 172.21.15.119:0/3698497979 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe676ffd700 1 --2- 172.21.15.119:0/3698497979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe678069a50 0x7fe6780c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe6777fe700 1 -- 172.21.15.119:0/3698497979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe678069a50 msgr2=0x7fe6780c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe6777fe700 1 --2- 172.21.15.119:0/3698497979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe678069a50 0x7fe6780c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe6777fe700 1 -- 172.21.15.119:0/3698497979 >> v1:172.21.15.119:6789/0 conn(0x7fe6780c8490 legacy=0x7fe67815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe6777fe700 1 -- 172.21.15.119:0/3698497979 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6780aba70 con 0x7fe678069e20 2023-10-08T00:57:03.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.482+0000 7fe6767fc700 1 -- 172.21.15.119:0/3698497979 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283590751 0 0) 0x7fe660001110 con 0x7fe6780c8490 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe6777fe700 1 --2- 172.21.15.119:0/3698497979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069e20 0x7fe6781569b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe670009f50 tx=0x7fe670009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1a53330e32f972 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe6767fc700 1 -- 172.21.15.119:0/3698497979 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67000e070 con 0x7fe678069e20 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe6767fc700 1 -- 172.21.15.119:0/3698497979 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67000ba70 con 0x7fe678069e20 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe6767fc700 1 -- 172.21.15.119:0/3698497979 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6700092e0 con 0x7fe678069e20 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe67d5ff700 1 -- 172.21.15.119:0/3698497979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069e20 msgr2=0x7fe6781569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/3698497979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069e20 0x7fe6781569b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe670009f50 tx=0x7fe670009840 comp rx=0 tx=0).stop 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe67d5ff700 1 -- 172.21.15.119:0/3698497979 shutdown_connections 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/3698497979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069e20 0x7fe6781569b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/3698497979 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe678069a50 0x7fe6780c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe67d5ff700 1 -- 172.21.15.119:0/3698497979 >> 172.21.15.119:0/3698497979 conn(0x7fe6780c8f90 msgr2=0x7fe6780c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:03.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.483+0000 7fe67d5ff700 1 -- 172.21.15.119:0/3698497979 shutdown_connections 2023-10-08T00:57:03.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.484+0000 7fe67d5ff700 1 -- 172.21.15.119:0/3698497979 wait complete. 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.485+0000 7fe67d5ff700 1 Processor -- start 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.485+0000 7fe67d5ff700 1 -- start start 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe67d5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069a50 0x7fe6781547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe67d5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe678069e20 0x7fe6780aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe67d5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6780c8490 0x7fe678153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe67d5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780c6820 con 0x7fe678069a50 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe67d5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780c6390 con 0x7fe678069e20 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe67d5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6780c5780 con 0x7fe6780c8490 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe676ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069a50 0x7fe6781547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe676ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069a50 0x7fe6781547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44712/0 (socket says 172.21.15.119:44712) 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe676ffd700 1 -- 172.21.15.119:0/1558846360 learned_addr learned my addr 172.21.15.119:0/1558846360 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe676ffd700 1 -- 172.21.15.119:0/1558846360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6780c8490 msgr2=0x7fe678153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe676ffd700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6780c8490 0x7fe678153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe676ffd700 1 -- 172.21.15.119:0/1558846360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe678069e20 msgr2=0x7fe6780aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.486+0000 7fe676ffd700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe678069e20 0x7fe6780aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.487+0000 7fe676ffd700 1 -- 172.21.15.119:0/1558846360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe670009ae0 con 0x7fe678069a50 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.487+0000 7fe676ffd700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069a50 0x7fe6781547d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe66800d890 tx=0x7fe668008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.487+0000 7fe66effd700 1 -- 172.21.15.119:0/1558846360 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800b600 con 0x7fe678069a50 2023-10-08T00:57:03.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.487+0000 7fe66effd700 1 -- 172.21.15.119:0/1558846360 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe66800b760 con 0x7fe678069a50 2023-10-08T00:57:03.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.487+0000 7fe66effd700 1 -- 172.21.15.119:0/1558846360 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe66800bae0 con 0x7fe678069a50 2023-10-08T00:57:03.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.487+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6780c5a60 con 0x7fe678069a50 2023-10-08T00:57:03.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.487+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6780c6020 con 0x7fe678069a50 2023-10-08T00:57:03.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.488+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6780718b0 con 0x7fe678069a50 2023-10-08T00:57:03.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.489+0000 7fe66effd700 1 -- 172.21.15.119:0/1558846360 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7fe66801e460 con 0x7fe678069a50 2023-10-08T00:57:03.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.490+0000 7fe66effd700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe6580671c0 0x7fe658069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.490+0000 7fe66effd700 1 -- 172.21.15.119:0/1558846360 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fe668085150 con 0x7fe678069a50 2023-10-08T00:57:03.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.490+0000 7fe66ffff700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe6580671c0 0x7fe658069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.491+0000 7fe66ffff700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe6580671c0 0x7fe658069670 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6780b23f0 tx=0x7fe660007b40 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:03.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.494+0000 7fe66effd700 1 -- 172.21.15.119:0/1558846360 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe668051f90 con 0x7fe678069a50 2023-10-08T00:57:03.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.710+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6780c64d0 con 0x7fe6580671c0 2023-10-08T00:57:03.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.758+0000 7fe66effd700 1 -- 172.21.15.119:0/1558846360 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe660014410 con 0x7fe6580671c0 2023-10-08T00:57:03.759 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-10-08T00:57:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe6580671c0 msgr2=0x7fe658069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe6580671c0 0x7fe658069670 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6780b23f0 tx=0x7fe660007b40 comp rx=0 tx=0).stop 2023-10-08T00:57:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069a50 msgr2=0x7fe6781547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069a50 0x7fe6781547d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe66800d890 tx=0x7fe668008040 comp rx=0 tx=0).stop 2023-10-08T00:57:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 shutdown_connections 2023-10-08T00:57:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6780c8490 0x7fe678153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe6580671c0 0x7fe658069670 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe678069a50 0x7fe6781547d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 --2- 172.21.15.119:0/1558846360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe678069e20 0x7fe6780aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.760+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 >> 172.21.15.119:0/1558846360 conn(0x7fe6780c8f90 msgr2=0x7fe6780cb160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.761+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 shutdown_connections 2023-10-08T00:57:03.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.761+0000 7fe67d5ff700 1 -- 172.21.15.119:0/1558846360 wait complete. 2023-10-08T00:57:03.778 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-10-08T00:57:03.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-10-08T00:57:03.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.967+0000 7f92e919d700 1 Processor -- start 2023-10-08T00:57:03.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.967+0000 7f92e919d700 1 -- start start 2023-10-08T00:57:03.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.968+0000 7f92e919d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92e40b03c0 0x7f92e40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.968+0000 7f92e919d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f92e40b0790 0x7f92e40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.968+0000 7f92e919d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e40cfa00 con 0x7f92e41591e0 2023-10-08T00:57:03.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.968+0000 7f92e919d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40cb580 con 0x7f92e40b03c0 2023-10-08T00:57:03.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.968+0000 7f92e919d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e40b2de0 con 0x7f92e40b0790 2023-10-08T00:57:03.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92e40b03c0 0x7f92e40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92e40b03c0 0x7f92e40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44716/0 (socket says 172.21.15.119:44716) 2023-10-08T00:57:03.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e3fff700 1 -- 172.21.15.119:0/660135119 learned_addr learned my addr 172.21.15.119:0/660135119 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:03.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e37fe700 1 --2- 172.21.15.119:0/660135119 >> v2:172.21.15.184:3300/0 conn(0x7f92e40b0790 0x7f92e40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e3fff700 1 -- 172.21.15.119:0/660135119 >> v2:172.21.15.184:3300/0 conn(0x7f92e40b0790 msgr2=0x7f92e40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:03.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e3fff700 1 --2- 172.21.15.119:0/660135119 >> v2:172.21.15.184:3300/0 conn(0x7f92e40b0790 0x7f92e40b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e3fff700 1 -- 172.21.15.119:0/660135119 >> v1:172.21.15.184:6789/0 conn(0x7f92e41591e0 legacy=0x7f92e4159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e3fff700 1 -- 172.21.15.119:0/660135119 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e40cf160 con 0x7f92e40b03c0 2023-10-08T00:57:03.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.969+0000 7f92e3fff700 1 --2- 172.21.15.119:0/660135119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e40b03c0 0x7f92e40cea70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f92d40099b0 tx=0x7f92d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa98b27c137cdf07 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:03.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e2ffd700 1 -- 172.21.15.119:0/660135119 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4009ea0 con 0x7f92e40b03c0 2023-10-08T00:57:03.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e2ffd700 1 -- 172.21.15.119:0/660135119 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d400b820 con 0x7f92e40b03c0 2023-10-08T00:57:03.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e2ffd700 1 -- 172.21.15.119:0/660135119 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d400bbe0 con 0x7f92e40b03c0 2023-10-08T00:57:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e919d700 1 -- 172.21.15.119:0/660135119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e40b03c0 msgr2=0x7f92e40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e919d700 1 --2- 172.21.15.119:0/660135119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e40b03c0 0x7f92e40cea70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f92d40099b0 tx=0x7f92d4009250 comp rx=0 tx=0).stop 2023-10-08T00:57:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e919d700 1 -- 172.21.15.119:0/660135119 shutdown_connections 2023-10-08T00:57:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e919d700 1 --2- 172.21.15.119:0/660135119 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e40b03c0 0x7f92e40cea70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e919d700 1 --2- 172.21.15.119:0/660135119 >> v2:172.21.15.184:3300/0 conn(0x7f92e40b0790 0x7f92e40b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e919d700 1 -- 172.21.15.119:0/660135119 >> 172.21.15.119:0/660135119 conn(0x7f92e4063230 msgr2=0x7f92e4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e919d700 1 -- 172.21.15.119:0/660135119 shutdown_connections 2023-10-08T00:57:03.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.970+0000 7f92e919d700 1 -- 172.21.15.119:0/660135119 wait complete. 2023-10-08T00:57:03.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.972+0000 7f92e919d700 1 Processor -- start 2023-10-08T00:57:03.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.972+0000 7f92e919d700 1 -- start start 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e919d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e40b03c0 0x7f92e40ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e919d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e40b0790 0x7f92e4157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e919d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e41591e0 0x7f92e4169d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e919d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40c9410 con 0x7f92e41591e0 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e919d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e415a190 con 0x7f92e40b03c0 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e919d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e416a5d0 con 0x7f92e40b0790 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e899c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e41591e0 0x7f92e4169d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e899c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e41591e0 0x7f92e4169d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44730/0 (socket says 172.21.15.119:44730) 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e899c700 1 -- 172.21.15.119:0/1611943464 learned_addr learned my addr 172.21.15.119:0/1611943464 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e3fff700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e40b03c0 0x7f92e40ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e37fe700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e40b0790 0x7f92e4157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e899c700 1 -- 172.21.15.119:0/1611943464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e40b0790 msgr2=0x7f92e4157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e899c700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e40b0790 0x7f92e4157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e899c700 1 -- 172.21.15.119:0/1611943464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e40b03c0 msgr2=0x7f92e40ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e899c700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e40b03c0 0x7f92e40ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.973+0000 7f92e899c700 1 -- 172.21.15.119:0/1611943464 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40094f0 con 0x7f92e41591e0 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.974+0000 7f92e899c700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e41591e0 0x7f92e4169d90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c870 tx=0x7f92dc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.974+0000 7f92cbfff700 1 -- 172.21.15.119:0/1611943464 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc012630 con 0x7f92e41591e0 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.974+0000 7f92cbfff700 1 -- 172.21.15.119:0/1611943464 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc0127b0 con 0x7f92e41591e0 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.974+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e416a8b0 con 0x7f92e41591e0 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.974+0000 7f92cbfff700 1 -- 172.21.15.119:0/1611943464 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc012b70 con 0x7f92e41591e0 2023-10-08T00:57:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.974+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e4159e20 con 0x7f92e41591e0 2023-10-08T00:57:03.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.975+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e4157be0 con 0x7f92e41591e0 2023-10-08T00:57:03.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.976+0000 7f92cbfff700 1 -- 172.21.15.119:0/1611943464 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f92dc003030 con 0x7f92e41591e0 2023-10-08T00:57:03.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.977+0000 7f92cbfff700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92d00671c0 0x7f92d0069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.977+0000 7f92cbfff700 1 -- 172.21.15.119:0/1611943464 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f92dc084880 con 0x7f92e41591e0 2023-10-08T00:57:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.977+0000 7f92e3fff700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92d00671c0 0x7f92d0069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.978+0000 7f92e3fff700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92d00671c0 0x7f92d0069670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f92d40099b0 tx=0x7f92d4017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:03.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:03.981+0000 7f92cbfff700 1 -- 172.21.15.119:0/1611943464 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92dc051760 con 0x7f92e41591e0 2023-10-08T00:57:04.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.203+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f92e404c3c0 con 0x7f92d00671c0 2023-10-08T00:57:04.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.256+0000 7f92cbfff700 1 -- 172.21.15.119:0/1611943464 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f92e404c3c0 con 0x7f92d00671c0 2023-10-08T00:57:04.257 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-10-08T00:57:04.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92d00671c0 msgr2=0x7f92d0069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92d00671c0 0x7f92d0069670 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f92d40099b0 tx=0x7f92d4017000 comp rx=0 tx=0).stop 2023-10-08T00:57:04.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e41591e0 msgr2=0x7f92e4169d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e41591e0 0x7f92e4169d90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00c870 tx=0x7f92dc001670 comp rx=0 tx=0).stop 2023-10-08T00:57:04.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 shutdown_connections 2023-10-08T00:57:04.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f92d00671c0 0x7f92d0069670 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92e41591e0 0x7f92e4169d90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f92e40b03c0 0x7f92e40ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 --2- 172.21.15.119:0/1611943464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f92e40b0790 0x7f92e4157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 >> 172.21.15.119:0/1611943464 conn(0x7f92e4063230 msgr2=0x7f92e4069740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:04.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 shutdown_connections 2023-10-08T00:57:04.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.259+0000 7f92e919d700 1 -- 172.21.15.119:0/1611943464 wait complete. 2023-10-08T00:57:04.276 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-10-08T00:57:04.276 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-10-08T00:57:04.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.461+0000 7f1c6ddff700 1 Processor -- start 2023-10-08T00:57:04.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.462+0000 7f1c6ddff700 1 -- start start 2023-10-08T00:57:04.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c6ddff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c68069a50 0x7f1c680c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c6ddff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1c680c8490 0x7f1c6815a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c6ddff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c680ac030 con 0x7f1c68069e20 2023-10-08T00:57:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c6ddff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c680ab820 con 0x7f1c68069a50 2023-10-08T00:57:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c6ddff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6806dbb0 con 0x7f1c680c8490 2023-10-08T00:57:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c677fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c68069a50 0x7f1c680c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c677fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c68069a50 0x7f1c680c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44744/0 (socket says 172.21.15.119:44744) 2023-10-08T00:57:04.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c677fe700 1 -- 172.21.15.119:0/2100846789 learned_addr learned my addr 172.21.15.119:0/2100846789 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:04.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.463+0000 7f1c66ffd700 1 --2- 172.21.15.119:0/2100846789 >> v2:172.21.15.184:3300/0 conn(0x7f1c680c8490 0x7f1c6815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.464+0000 7f1c677fe700 1 -- 172.21.15.119:0/2100846789 >> v2:172.21.15.184:3300/0 conn(0x7f1c680c8490 msgr2=0x7f1c6815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:04.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.464+0000 7f1c677fe700 1 --2- 172.21.15.119:0/2100846789 >> v2:172.21.15.184:3300/0 conn(0x7f1c680c8490 0x7f1c6815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.464+0000 7f1c677fe700 1 -- 172.21.15.119:0/2100846789 >> v1:172.21.15.184:6789/0 conn(0x7f1c68069e20 legacy=0x7f1c6806e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.464+0000 7f1c677fe700 1 -- 172.21.15.119:0/2100846789 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6815a9d0 con 0x7f1c68069a50 2023-10-08T00:57:04.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.464+0000 7f1c677fe700 1 --2- 172.21.15.119:0/2100846789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069a50 0x7f1c680c6ac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1c6806b920 tx=0x7f1c58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb41843aa7b3452 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:04.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.464+0000 7f1c667fc700 1 -- 172.21.15.119:0/2100846789 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58009e70 con 0x7f1c68069a50 2023-10-08T00:57:04.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.464+0000 7f1c667fc700 1 -- 172.21.15.119:0/2100846789 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5800b860 con 0x7f1c68069a50 2023-10-08T00:57:04.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.464+0000 7f1c667fc700 1 -- 172.21.15.119:0/2100846789 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800bc00 con 0x7f1c68069a50 2023-10-08T00:57:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.465+0000 7f1c6ddff700 1 -- 172.21.15.119:0/2100846789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069a50 msgr2=0x7f1c680c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.465+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/2100846789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069a50 0x7f1c680c6ac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f1c6806b920 tx=0x7f1c58009250 comp rx=0 tx=0).stop 2023-10-08T00:57:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.465+0000 7f1c6ddff700 1 -- 172.21.15.119:0/2100846789 shutdown_connections 2023-10-08T00:57:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.465+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/2100846789 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069a50 0x7f1c680c6ac0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.465+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/2100846789 >> v2:172.21.15.184:3300/0 conn(0x7f1c680c8490 0x7f1c6815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.465+0000 7f1c6ddff700 1 -- 172.21.15.119:0/2100846789 >> 172.21.15.119:0/2100846789 conn(0x7f1c680c8f90 msgr2=0x7f1c680c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:04.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.465+0000 7f1c6ddff700 1 -- 172.21.15.119:0/2100846789 shutdown_connections 2023-10-08T00:57:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.465+0000 7f1c6ddff700 1 -- 172.21.15.119:0/2100846789 wait complete. 2023-10-08T00:57:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.466+0000 7f1c6ddff700 1 Processor -- start 2023-10-08T00:57:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.466+0000 7f1c6ddff700 1 -- start start 2023-10-08T00:57:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c6ddff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c68069a50 0x7f1c68153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c6ddff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069e20 0x7f1c68167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c6ddff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c680c8490 0x7f1c68156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c6ddff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c680c5a60 con 0x7f1c68069e20 2023-10-08T00:57:04.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c6ddff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c681686d0 con 0x7f1c680c8490 2023-10-08T00:57:04.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c6ddff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c681689b0 con 0x7f1c68069a50 2023-10-08T00:57:04.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c66ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069e20 0x7f1c68167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c66ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069e20 0x7f1c68167bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44758/0 (socket says 172.21.15.119:44758) 2023-10-08T00:57:04.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.467+0000 7f1c66ffd700 1 -- 172.21.15.119:0/866412307 learned_addr learned my addr 172.21.15.119:0/866412307 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:04.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c66ffd700 1 -- 172.21.15.119:0/866412307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c68069a50 msgr2=0x7f1c68153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c66ffd700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c68069a50 0x7f1c68153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c66ffd700 1 -- 172.21.15.119:0/866412307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c680c8490 msgr2=0x7f1c68156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c66ffd700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c680c8490 0x7f1c68156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c66ffd700 1 -- 172.21.15.119:0/866412307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c580094f0 con 0x7f1c68069e20 2023-10-08T00:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c66ffd700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069e20 0x7f1c68167bc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e950 tx=0x7f1c5c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/866412307 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00c850 con 0x7f1c68069e20 2023-10-08T00:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/866412307 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c00c9b0 con 0x7f1c68069e20 2023-10-08T00:57:04.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c680c56f0 con 0x7f1c68069e20 2023-10-08T00:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/866412307 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00cdb0 con 0x7f1c68069e20 2023-10-08T00:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.468+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c680c53a0 con 0x7f1c68069e20 2023-10-08T00:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.469+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c68168bd0 con 0x7f1c68069e20 2023-10-08T00:57:04.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.470+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/866412307 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f1c5c013680 con 0x7f1c68069e20 2023-10-08T00:57:04.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.471+0000 7f1c4f7fe700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1c540671c0 0x7f1c54069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.471+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/866412307 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f1c5c084c20 con 0x7f1c68069e20 2023-10-08T00:57:04.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.471+0000 7f1c677fe700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1c540671c0 0x7f1c54069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.472+0000 7f1c677fe700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1c540671c0 0x7f1c54069670 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c6806b920 tx=0x7f1c58017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:04.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.475+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/866412307 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c68168bd0 con 0x7f1c68069e20 2023-10-08T00:57:04.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.689+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c680c5ba0 con 0x7f1c540671c0 2023-10-08T00:57:04.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.732+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/866412307 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f1c680c5ba0 con 0x7f1c540671c0 2023-10-08T00:57:04.732 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-10-08T00:57:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1c540671c0 msgr2=0x7f1c54069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1c540671c0 0x7f1c54069670 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c6806b920 tx=0x7f1c58017000 comp rx=0 tx=0).stop 2023-10-08T00:57:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069e20 msgr2=0x7f1c68167bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069e20 0x7f1c68167bc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c00e950 tx=0x7f1c5c0091a0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 shutdown_connections 2023-10-08T00:57:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c68069a50 0x7f1c68153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1c540671c0 0x7f1c54069670 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c680c8490 0x7f1c68156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 --2- 172.21.15.119:0/866412307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c68069e20 0x7f1c68167bc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 >> 172.21.15.119:0/866412307 conn(0x7f1c680c8f90 msgr2=0x7f1c68155770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 shutdown_connections 2023-10-08T00:57:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.734+0000 7f1c6ddff700 1 -- 172.21.15.119:0/866412307 wait complete. 2023-10-08T00:57:04.752 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-10-08T00:57:04.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-10-08T00:57:04.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.938+0000 7fb985bff700 1 Processor -- start 2023-10-08T00:57:04.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.938+0000 7fb985bff700 1 -- start start 2023-10-08T00:57:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.939+0000 7fb985bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb9800c6ac0 0x7fb980068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.939+0000 7fb985bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb980069a50 0x7fb980156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb985bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9800ac030 con 0x7fb9800c8490 2023-10-08T00:57:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb985bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9800ab820 con 0x7fb9800c6ac0 2023-10-08T00:57:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb985bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb98006dbb0 con 0x7fb980069a50 2023-10-08T00:57:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb977fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb9800c6ac0 0x7fb980068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb977fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb9800c6ac0 0x7fb980068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55382/0 (socket says 172.21.15.119:55382) 2023-10-08T00:57:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb977fff700 1 -- 172.21.15.119:0/4119021268 learned_addr learned my addr 172.21.15.119:0/4119021268 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb97ffff700 1 --2- 172.21.15.119:0/4119021268 >> v2:172.21.15.184:3300/0 conn(0x7fb980069a50 0x7fb980156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb977fff700 1 -- 172.21.15.119:0/4119021268 >> v2:172.21.15.184:3300/0 conn(0x7fb980069a50 msgr2=0x7fb980156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb977fff700 1 --2- 172.21.15.119:0/4119021268 >> v2:172.21.15.184:3300/0 conn(0x7fb980069a50 0x7fb980156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb977fff700 1 -- 172.21.15.119:0/4119021268 >> v1:172.21.15.184:6789/0 conn(0x7fb9800c8490 legacy=0x7fb98015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.940+0000 7fb977fff700 1 -- 172.21.15.119:0/4119021268 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9800c6f10 con 0x7fb9800c6ac0 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb977fff700 1 --2- 172.21.15.119:0/4119021268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c6ac0 0x7fb980068390 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb98006f1a0 tx=0x7fb968009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d6643bcd452081 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb97effd700 1 -- 172.21.15.119:0/4119021268 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96800b7e0 con 0x7fb9800c6ac0 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb97effd700 1 -- 172.21.15.119:0/4119021268 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb96800b940 con 0x7fb9800c6ac0 2023-10-08T00:57:04.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb97effd700 1 -- 172.21.15.119:0/4119021268 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb96800bce0 con 0x7fb9800c6ac0 2023-10-08T00:57:04.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb985bff700 1 -- 172.21.15.119:0/4119021268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c6ac0 msgr2=0x7fb980068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb985bff700 1 --2- 172.21.15.119:0/4119021268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c6ac0 0x7fb980068390 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb98006f1a0 tx=0x7fb968009250 comp rx=0 tx=0).stop 2023-10-08T00:57:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb985bff700 1 -- 172.21.15.119:0/4119021268 shutdown_connections 2023-10-08T00:57:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb985bff700 1 --2- 172.21.15.119:0/4119021268 >> v2:172.21.15.184:3300/0 conn(0x7fb980069a50 0x7fb980156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb985bff700 1 --2- 172.21.15.119:0/4119021268 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c6ac0 0x7fb980068390 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.941+0000 7fb985bff700 1 -- 172.21.15.119:0/4119021268 >> 172.21.15.119:0/4119021268 conn(0x7fb9800c8f90 msgr2=0x7fb9800c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.942+0000 7fb985bff700 1 -- 172.21.15.119:0/4119021268 shutdown_connections 2023-10-08T00:57:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.942+0000 7fb985bff700 1 -- 172.21.15.119:0/4119021268 wait complete. 2023-10-08T00:57:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.943+0000 7fb985bff700 1 Processor -- start 2023-10-08T00:57:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.943+0000 7fb985bff700 1 -- start start 2023-10-08T00:57:04.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb985bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980069a50 0x7fb980154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb985bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9800c6ac0 0x7fb9800cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb985bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c8490 0x7fb9800cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb985bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800c4af0 con 0x7fb9800c8490 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb985bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800b0070 con 0x7fb980069a50 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb985bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9800b0350 con 0x7fb9800c6ac0 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb97ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c8490 0x7fb9800cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb97ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c8490 0x7fb9800cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55394/0 (socket says 172.21.15.119:55394) 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb97ffff700 1 -- 172.21.15.119:0/2005514992 learned_addr learned my addr 172.21.15.119:0/2005514992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.944+0000 7fb97f7fe700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980069a50 0x7fb980154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb977fff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9800c6ac0 0x7fb9800cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb97ffff700 1 -- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9800c6ac0 msgr2=0x7fb9800cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb97ffff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9800c6ac0 0x7fb9800cb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb97ffff700 1 -- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980069a50 msgr2=0x7fb980154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb97ffff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980069a50 0x7fb980154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb97ffff700 1 -- 172.21.15.119:0/2005514992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9680094f0 con 0x7fb9800c8490 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb97f7fe700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980069a50 0x7fb980154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb977fff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9800c6ac0 0x7fb9800cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb97ffff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c8490 0x7fb9800cb560 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb978009fa0 tx=0x7fb978009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb9777fe700 1 -- 172.21.15.119:0/2005514992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97800ed20 con 0x7fb9800c8490 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb9777fe700 1 -- 172.21.15.119:0/2005514992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97800ee80 con 0x7fb9800c8490 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9800b0630 con 0x7fb9800c8490 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb9777fe700 1 -- 172.21.15.119:0/2005514992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb978014670 con 0x7fb9800c8490 2023-10-08T00:57:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.945+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb980154f70 con 0x7fb9800c8490 2023-10-08T00:57:04.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.946+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9800c48c0 con 0x7fb9800c8490 2023-10-08T00:57:04.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.947+0000 7fb9777fe700 1 -- 172.21.15.119:0/2005514992 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7fb9780149e0 con 0x7fb9800c8490 2023-10-08T00:57:04.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.948+0000 7fb9777fe700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb9600671c0 0x7fb960069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:04.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.948+0000 7fb9777fe700 1 -- 172.21.15.119:0/2005514992 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fb978085170 con 0x7fb9800c8490 2023-10-08T00:57:04.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.948+0000 7fb97f7fe700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb9600671c0 0x7fb960069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:04.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.949+0000 7fb97f7fe700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb9600671c0 0x7fb960069670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb970006fd0 tx=0x7fb970008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:04.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:04.952+0000 7fb9777fe700 1 -- 172.21.15.119:0/2005514992 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9800c48c0 con 0x7fb9800c8490 2023-10-08T00:57:05.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.171+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9800c48c0 con 0x7fb9600671c0 2023-10-08T00:57:05.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.219+0000 7fb9777fe700 1 -- 172.21.15.119:0/2005514992 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb9800c48c0 con 0x7fb9600671c0 2023-10-08T00:57:05.220 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-10-08T00:57:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.221+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb9600671c0 msgr2=0x7fb960069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.221+0000 7fb985bff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb9600671c0 0x7fb960069670 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb970006fd0 tx=0x7fb970008040 comp rx=0 tx=0).stop 2023-10-08T00:57:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.221+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c8490 msgr2=0x7fb9800cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.221+0000 7fb985bff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c8490 0x7fb9800cb560 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb978009fa0 tx=0x7fb978009900 comp rx=0 tx=0).stop 2023-10-08T00:57:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.221+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 shutdown_connections 2023-10-08T00:57:05.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.222+0000 7fb985bff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb9600671c0 0x7fb960069670 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.222+0000 7fb985bff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9800c8490 0x7fb9800cb560 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.222+0000 7fb985bff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980069a50 0x7fb980154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.222+0000 7fb985bff700 1 --2- 172.21.15.119:0/2005514992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9800c6ac0 0x7fb9800cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.222+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 >> 172.21.15.119:0/2005514992 conn(0x7fb9800c8f90 msgr2=0x7fb980156cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.222+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 shutdown_connections 2023-10-08T00:57:05.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.222+0000 7fb985bff700 1 -- 172.21.15.119:0/2005514992 wait complete. 2023-10-08T00:57:05.239 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-10-08T00:57:05.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-10-08T00:57:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.426+0000 7f27be39d700 1 Processor -- start 2023-10-08T00:57:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.426+0000 7f27be39d700 1 -- start start 2023-10-08T00:57:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.427+0000 7f27be39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b03c0 0x7f27b80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.427+0000 7f27be39d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f27b80b0790 0x7f27b80b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.427+0000 7f27be39d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f27b81591e0 0x7f27b8159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.427+0000 7f27be39d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b80cfa00 con 0x7f27b81591e0 2023-10-08T00:57:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.427+0000 7f27be39d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b80cb580 con 0x7f27b80b0790 2023-10-08T00:57:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.427+0000 7f27be39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80b2de0 con 0x7f27b80b03c0 2023-10-08T00:57:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.427+0000 7f27bdb9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f27b81591e0 0x7f27b8159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.427+0000 7f27bdb9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f27b81591e0 0x7f27b8159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55410/0 (socket says 172.21.15.119:55410) 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bdb9c700 1 -- 172.21.15.119:0/2533985458 learned_addr learned my addr 172.21.15.119:0/2533985458 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bd39b700 1 --2- 172.21.15.119:0/2533985458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b03c0 0x7f27b80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bcb9a700 1 --2- 172.21.15.119:0/2533985458 >> v2:172.21.15.184:3300/0 conn(0x7f27b80b0790 0x7f27b80b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bdb9c700 1 -- 172.21.15.119:0/2533985458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b03c0 msgr2=0x7f27b80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bdb9c700 1 --2- 172.21.15.119:0/2533985458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b03c0 0x7f27b80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bdb9c700 1 -- 172.21.15.119:0/2533985458 >> v2:172.21.15.184:3300/0 conn(0x7f27b80b0790 msgr2=0x7f27b80b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bdb9c700 1 --2- 172.21.15.119:0/2533985458 >> v2:172.21.15.184:3300/0 conn(0x7f27b80b0790 0x7f27b80b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bdb9c700 1 -- 172.21.15.119:0/2533985458 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b80cf160 con 0x7f27b81591e0 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27bdb9c700 1 --2- 172.21.15.119:0/2533985458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b81591e0 0x7f27b8159620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f27b8067fa0 tx=0x7f27b0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37035f60a0c8af1f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.428+0000 7f27affff700 1 -- 172.21.15.119:0/2533985458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b000e070 con 0x7f27b81591e0 2023-10-08T00:57:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27affff700 1 -- 172.21.15.119:0/2533985458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b000ba50 con 0x7f27b81591e0 2023-10-08T00:57:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27affff700 1 -- 172.21.15.119:0/2533985458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b00092e0 con 0x7f27b81591e0 2023-10-08T00:57:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 -- 172.21.15.119:0/2533985458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b81591e0 msgr2=0x7f27b8159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 --2- 172.21.15.119:0/2533985458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b81591e0 0x7f27b8159620 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f27b8067fa0 tx=0x7f27b0009840 comp rx=0 tx=0).stop 2023-10-08T00:57:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 -- 172.21.15.119:0/2533985458 shutdown_connections 2023-10-08T00:57:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 --2- 172.21.15.119:0/2533985458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b81591e0 0x7f27b8159620 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 --2- 172.21.15.119:0/2533985458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b03c0 0x7f27b80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 --2- 172.21.15.119:0/2533985458 >> v2:172.21.15.184:3300/0 conn(0x7f27b80b0790 0x7f27b80b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 -- 172.21.15.119:0/2533985458 >> 172.21.15.119:0/2533985458 conn(0x7f27b8063230 msgr2=0x7f27b8063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 -- 172.21.15.119:0/2533985458 shutdown_connections 2023-10-08T00:57:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.429+0000 7f27be39d700 1 -- 172.21.15.119:0/2533985458 wait complete. 2023-10-08T00:57:05.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.431+0000 7f27be39d700 1 Processor -- start 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.431+0000 7f27be39d700 1 -- start start 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27be39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b80b03c0 0x7f27b80c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27be39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b0790 0x7f27b80b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27be39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b81591e0 0x7f27b80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27be39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b8151e30 con 0x7f27b80b03c0 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27be39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80c8020 con 0x7f27b81591e0 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27be39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b80ca4c0 con 0x7f27b80b0790 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bd39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b80b03c0 0x7f27b80c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bd39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b80b03c0 0x7f27b80c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55424/0 (socket says 172.21.15.119:55424) 2023-10-08T00:57:05.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bd39b700 1 -- 172.21.15.119:0/2497100157 learned_addr learned my addr 172.21.15.119:0/2497100157 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:05.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bcb9a700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b0790 0x7f27b80b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bdb9c700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b81591e0 0x7f27b80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bd39b700 1 -- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b0790 msgr2=0x7f27b80b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bd39b700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b0790 0x7f27b80b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bd39b700 1 -- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b81591e0 msgr2=0x7f27b80b4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bd39b700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b81591e0 0x7f27b80b4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bd39b700 1 -- 172.21.15.119:0/2497100157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0009ae0 con 0x7f27b80b03c0 2023-10-08T00:57:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.432+0000 7f27bdb9c700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b81591e0 0x7f27b80b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T00:57:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.433+0000 7f27bd39b700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b80b03c0 0x7f27b80c9f80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f27b400d890 tx=0x7f27b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.433+0000 7f27ad7fa700 1 -- 172.21.15.119:0/2497100157 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b40125f0 con 0x7f27b80b03c0 2023-10-08T00:57:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.433+0000 7f27ad7fa700 1 -- 172.21.15.119:0/2497100157 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b4012770 con 0x7f27b80b03c0 2023-10-08T00:57:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.433+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b80c69e0 con 0x7f27b80b03c0 2023-10-08T00:57:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.433+0000 7f27ad7fa700 1 -- 172.21.15.119:0/2497100157 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4012b30 con 0x7f27b80b03c0 2023-10-08T00:57:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.433+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b80c6fa0 con 0x7f27b80b03c0 2023-10-08T00:57:05.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.434+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b804c3c0 con 0x7f27b80b03c0 2023-10-08T00:57:05.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.435+0000 7f27ad7fa700 1 -- 172.21.15.119:0/2497100157 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f27b401e460 con 0x7f27b80b03c0 2023-10-08T00:57:05.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.436+0000 7f27ad7fa700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f279c0671c0 0x7f279c069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.436+0000 7f27ad7fa700 1 -- 172.21.15.119:0/2497100157 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f27b4085130 con 0x7f27b80b03c0 2023-10-08T00:57:05.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.436+0000 7f27bcb9a700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f279c0671c0 0x7f279c069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.437+0000 7f27bcb9a700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f279c0671c0 0x7f279c069670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27b81522c0 tx=0x7f27a8008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:05.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.439+0000 7f27ad7fa700 1 -- 172.21.15.119:0/2497100157 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f27b4052010 con 0x7f27b80b03c0 2023-10-08T00:57:05.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.656+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f27b80cad50 con 0x7f279c0671c0 2023-10-08T00:57:05.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.701+0000 7f27ad7fa700 1 -- 172.21.15.119:0/2497100157 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f27b80cad50 con 0x7f279c0671c0 2023-10-08T00:57:05.702 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-10-08T00:57:05.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.703+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f279c0671c0 msgr2=0x7f279c069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.703+0000 7f27be39d700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f279c0671c0 0x7f279c069670 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f27b81522c0 tx=0x7f27a8008040 comp rx=0 tx=0).stop 2023-10-08T00:57:05.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.703+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b80b03c0 msgr2=0x7f27b80c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.703+0000 7f27be39d700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b80b03c0 0x7f27b80c9f80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f27b400d890 tx=0x7f27b4008040 comp rx=0 tx=0).stop 2023-10-08T00:57:05.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.703+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 shutdown_connections 2023-10-08T00:57:05.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.704+0000 7f27be39d700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f279c0671c0 0x7f279c069670 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.704+0000 7f27be39d700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f27b81591e0 0x7f27b80b4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.704+0000 7f27be39d700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b80b03c0 0x7f27b80c9f80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.704+0000 7f27be39d700 1 --2- 172.21.15.119:0/2497100157 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f27b80b0790 0x7f27b80b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.704+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 >> 172.21.15.119:0/2497100157 conn(0x7f27b8063230 msgr2=0x7f27b8069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:05.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.704+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 shutdown_connections 2023-10-08T00:57:05.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.704+0000 7f27be39d700 1 -- 172.21.15.119:0/2497100157 wait complete. 2023-10-08T00:57:05.721 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-10-08T00:57:05.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-10-08T00:57:05.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.907+0000 7f77e919d700 1 Processor -- start 2023-10-08T00:57:05.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.907+0000 7f77e919d700 1 -- start start 2023-10-08T00:57:05.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.908+0000 7f77e919d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77e40b03c0 0x7f77e40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.908+0000 7f77e919d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e40b0790 0x7f77e40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.908+0000 7f77e919d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e40cfa00 con 0x7f77e41591e0 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.908+0000 7f77e919d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e40cb580 con 0x7f77e40b03c0 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.908+0000 7f77e919d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40b2de0 con 0x7f77e40b0790 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.908+0000 7f77e3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77e40b03c0 0x7f77e40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.908+0000 7f77e3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77e40b03c0 0x7f77e40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55440/0 (socket says 172.21.15.119:55440) 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.908+0000 7f77e3fff700 1 -- 172.21.15.119:0/2376064095 learned_addr learned my addr 172.21.15.119:0/2376064095 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e37fe700 1 --2- 172.21.15.119:0/2376064095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e40b0790 0x7f77e40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e3fff700 1 -- 172.21.15.119:0/2376064095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e40b0790 msgr2=0x7f77e40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e3fff700 1 --2- 172.21.15.119:0/2376064095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e40b0790 0x7f77e40b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e3fff700 1 -- 172.21.15.119:0/2376064095 >> v1:172.21.15.119:6789/0 conn(0x7f77e41591e0 legacy=0x7f77e4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e3fff700 1 -- 172.21.15.119:0/2376064095 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40cf160 con 0x7f77e40b03c0 2023-10-08T00:57:05.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e2ffd700 1 -- 172.21.15.119:0/2376064095 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024573654 0 0) 0x7f77e40cfa00 con 0x7f77e41591e0 2023-10-08T00:57:05.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e3fff700 1 --2- 172.21.15.119:0/2376064095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b03c0 0x7f77e40cea70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f77d40099b0 tx=0x7f77d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235fc16539a53dc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:05.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e2ffd700 1 -- 172.21.15.119:0/2376064095 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4009ea0 con 0x7f77e40b03c0 2023-10-08T00:57:05.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e2ffd700 1 -- 172.21.15.119:0/2376064095 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d400b860 con 0x7f77e40b03c0 2023-10-08T00:57:05.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.909+0000 7f77e2ffd700 1 -- 172.21.15.119:0/2376064095 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d400bc00 con 0x7f77e40b03c0 2023-10-08T00:57:05.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.910+0000 7f77e919d700 1 -- 172.21.15.119:0/2376064095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b03c0 msgr2=0x7f77e40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.910+0000 7f77e919d700 1 --2- 172.21.15.119:0/2376064095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b03c0 0x7f77e40cea70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f77d40099b0 tx=0x7f77d4009250 comp rx=0 tx=0).stop 2023-10-08T00:57:05.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.910+0000 7f77e919d700 1 -- 172.21.15.119:0/2376064095 shutdown_connections 2023-10-08T00:57:05.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.910+0000 7f77e919d700 1 --2- 172.21.15.119:0/2376064095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b03c0 0x7f77e40cea70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.910+0000 7f77e919d700 1 --2- 172.21.15.119:0/2376064095 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e40b0790 0x7f77e40b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.910+0000 7f77e919d700 1 -- 172.21.15.119:0/2376064095 >> 172.21.15.119:0/2376064095 conn(0x7f77e4063230 msgr2=0x7f77e4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:05.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.910+0000 7f77e919d700 1 -- 172.21.15.119:0/2376064095 shutdown_connections 2023-10-08T00:57:05.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.910+0000 7f77e919d700 1 -- 172.21.15.119:0/2376064095 wait complete. 2023-10-08T00:57:05.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.911+0000 7f77e919d700 1 Processor -- start 2023-10-08T00:57:05.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.912+0000 7f77e919d700 1 -- start start 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.912+0000 7f77e919d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e40b03c0 0x7f77e4152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.912+0000 7f77e919d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b0790 0x7f77e4151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.912+0000 7f77e919d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e41591e0 0x7f77e4156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.912+0000 7f77e919d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4153830 con 0x7f77e40b0790 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.912+0000 7f77e919d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4155920 con 0x7f77e40b03c0 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.912+0000 7f77e919d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40ca660 con 0x7f77e41591e0 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b0790 0x7f77e4151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b0790 0x7f77e4151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55442/0 (socket says 172.21.15.119:55442) 2023-10-08T00:57:05.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 -- 172.21.15.119:0/1555734235 learned_addr learned my addr 172.21.15.119:0/1555734235 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e3fff700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e40b03c0 0x7f77e4152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e899c700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e41591e0 0x7f77e4156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 -- 172.21.15.119:0/1555734235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e41591e0 msgr2=0x7f77e4156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e41591e0 0x7f77e4156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 -- 172.21.15.119:0/1555734235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e40b03c0 msgr2=0x7f77e4152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e40b03c0 0x7f77e4152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 -- 172.21.15.119:0/1555734235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40094f0 con 0x7f77e40b0790 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.913+0000 7f77e37fe700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b0790 0x7f77e4151de0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f77d800ec80 tx=0x7f77d80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.914+0000 7f77cbfff700 1 -- 172.21.15.119:0/1555734235 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d800c9d0 con 0x7f77e40b0790 2023-10-08T00:57:05.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.914+0000 7f77cbfff700 1 -- 172.21.15.119:0/1555734235 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d800cb30 con 0x7f77e40b0790 2023-10-08T00:57:05.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.914+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e4154ce0 con 0x7f77e40b0790 2023-10-08T00:57:05.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.914+0000 7f77cbfff700 1 -- 172.21.15.119:0/1555734235 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d8013470 con 0x7f77e40b0790 2023-10-08T00:57:05.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.914+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e4155260 con 0x7f77e40b0790 2023-10-08T00:57:05.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.915+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e404c3c0 con 0x7f77e40b0790 2023-10-08T00:57:05.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.916+0000 7f77cbfff700 1 -- 172.21.15.119:0/1555734235 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f77d8013820 con 0x7f77e40b0790 2023-10-08T00:57:05.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.917+0000 7f77cbfff700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f77d00671c0 0x7f77d0069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:05.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.917+0000 7f77cbfff700 1 -- 172.21.15.119:0/1555734235 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f77d8085130 con 0x7f77e40b0790 2023-10-08T00:57:05.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.917+0000 7f77e3fff700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f77d00671c0 0x7f77d0069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:05.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.918+0000 7f77e3fff700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f77d00671c0 0x7f77d0069670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f77d40099b0 tx=0x7f77d4017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:05.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:05.921+0000 7f77cbfff700 1 -- 172.21.15.119:0/1555734235 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77d8052010 con 0x7f77e40b0790 2023-10-08T00:57:06.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.139+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f77e40c8c80 con 0x7f77d00671c0 2023-10-08T00:57:06.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.182+0000 7f77cbfff700 1 -- 172.21.15.119:0/1555734235 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f77e40c8c80 con 0x7f77d00671c0 2023-10-08T00:57:06.182 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-10-08T00:57:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.184+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f77d00671c0 msgr2=0x7f77d0069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.184+0000 7f77e919d700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f77d00671c0 0x7f77d0069670 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f77d40099b0 tx=0x7f77d4017000 comp rx=0 tx=0).stop 2023-10-08T00:57:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b0790 msgr2=0x7f77e4151de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b0790 0x7f77e4151de0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f77d800ec80 tx=0x7f77d80091a0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 shutdown_connections 2023-10-08T00:57:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77e41591e0 0x7f77e4156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f77d00671c0 0x7f77d0069670 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77e40b03c0 0x7f77e4152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 --2- 172.21.15.119:0/1555734235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77e40b0790 0x7f77e4151de0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 >> 172.21.15.119:0/1555734235 conn(0x7f77e4063230 msgr2=0x7f77e40704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:06.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 shutdown_connections 2023-10-08T00:57:06.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.185+0000 7f77e919d700 1 -- 172.21.15.119:0/1555734235 wait complete. 2023-10-08T00:57:06.202 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-10-08T00:57:06.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-10-08T00:57:06.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.390+0000 7fae40bff700 1 Processor -- start 2023-10-08T00:57:06.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.390+0000 7fae40bff700 1 -- start start 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.391+0000 7fae40bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae3c1570c0 0x7fae3c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.391+0000 7fae40bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0ac030 con 0x7fae3c069a50 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.391+0000 7fae40bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0ab820 con 0x7fae3c0c8490 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.391+0000 7fae40bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c06dbb0 con 0x7fae3c1570c0 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.392+0000 7fae3a59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fae3c069a50 0x7fae3c15a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50662/0 (socket says 172.21.15.119:50662) 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.392+0000 7fae3a59c700 1 -- 172.21.15.119:0/377480549 learned_addr learned my addr 172.21.15.119:0/377480549 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.392+0000 7fae3ad9d700 1 --2- 172.21.15.119:0/377480549 >> v2:172.21.15.119:3300/0 conn(0x7fae3c1570c0 0x7fae3c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.392+0000 7fae39d9b700 1 -- 172.21.15.119:0/377480549 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574928257 0 0) 0x7fae2c001110 con 0x7fae3c069a50 2023-10-08T00:57:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.392+0000 7fae39d9b700 1 -- 172.21.15.119:0/377480549 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae28003570 con 0x7fae3c069a50 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.392+0000 7fae3ad9d700 1 -- 172.21.15.119:0/377480549 >> v1:172.21.15.184:6789/0 conn(0x7fae3c0c8490 legacy=0x7fae3c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.392+0000 7fae3ad9d700 1 -- 172.21.15.119:0/377480549 >> v1:172.21.15.119:6789/0 conn(0x7fae3c069a50 legacy=0x7fae3c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.392+0000 7fae3ad9d700 1 -- 172.21.15.119:0/377480549 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0aba70 con 0x7fae3c1570c0 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae3ad9d700 1 --2- 172.21.15.119:0/377480549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c1570c0 0x7fae3c0c6ac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae34009e60 tx=0x7fae34009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930a729cd9addac1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae39d9b700 1 -- 172.21.15.119:0/377480549 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3400e070 con 0x7fae3c1570c0 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae39d9b700 1 -- 172.21.15.119:0/377480549 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3400bb60 con 0x7fae3c1570c0 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae39d9b700 1 -- 172.21.15.119:0/377480549 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34012430 con 0x7fae3c1570c0 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae40bff700 1 -- 172.21.15.119:0/377480549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c1570c0 msgr2=0x7fae3c0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae40bff700 1 --2- 172.21.15.119:0/377480549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c1570c0 0x7fae3c0c6ac0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fae34009e60 tx=0x7fae34009780 comp rx=0 tx=0).stop 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae40bff700 1 -- 172.21.15.119:0/377480549 shutdown_connections 2023-10-08T00:57:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae40bff700 1 --2- 172.21.15.119:0/377480549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c1570c0 0x7fae3c0c6ac0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae40bff700 1 -- 172.21.15.119:0/377480549 >> 172.21.15.119:0/377480549 conn(0x7fae3c0c8f90 msgr2=0x7fae3c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:06.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae40bff700 1 -- 172.21.15.119:0/377480549 shutdown_connections 2023-10-08T00:57:06.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.393+0000 7fae40bff700 1 -- 172.21.15.119:0/377480549 wait complete. 2023-10-08T00:57:06.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.395+0000 7fae40bff700 1 Processor -- start 2023-10-08T00:57:06.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.395+0000 7fae40bff700 1 -- start start 2023-10-08T00:57:06.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae40bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae3c069a50 0x7fae3c147090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:06.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae40bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c0c8490 0x7fae3c0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:06.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae40bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae3c1570c0 0x7fae3c0a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:06.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae40bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c154e90 con 0x7fae3c0c8490 2023-10-08T00:57:06.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae40bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0acdc0 con 0x7fae3c069a50 2023-10-08T00:57:06.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae40bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c0c37d0 con 0x7fae3c1570c0 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c0c8490 0x7fae3c0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c0c8490 0x7fae3c0a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55468/0 (socket says 172.21.15.119:55468) 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae33fff700 1 -- 172.21.15.119:0/2825811661 learned_addr learned my addr 172.21.15.119:0/2825811661 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae3a59c700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae3c069a50 0x7fae3c147090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.396+0000 7fae3ad9d700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae3c1570c0 0x7fae3c0a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae33fff700 1 -- 172.21.15.119:0/2825811661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae3c1570c0 msgr2=0x7fae3c0a9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae33fff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae3c1570c0 0x7fae3c0a9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae33fff700 1 -- 172.21.15.119:0/2825811661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae3c069a50 msgr2=0x7fae3c147090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae33fff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae3c069a50 0x7fae3c147090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae33fff700 1 -- 172.21.15.119:0/2825811661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009a20 con 0x7fae3c0c8490 2023-10-08T00:57:06.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae33fff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c0c8490 0x7fae3c0a98e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c590 tx=0x7fae2400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:06.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae327fc700 1 -- 172.21.15.119:0/2825811661 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24011590 con 0x7fae3c0c8490 2023-10-08T00:57:06.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae327fc700 1 -- 172.21.15.119:0/2825811661 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24011710 con 0x7fae3c0c8490 2023-10-08T00:57:06.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c0aa430 con 0x7fae3c0c8490 2023-10-08T00:57:06.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae327fc700 1 -- 172.21.15.119:0/2825811661 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24011ad0 con 0x7fae3c0c8490 2023-10-08T00:57:06.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.397+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c0ac770 con 0x7fae3c0c8490 2023-10-08T00:57:06.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.398+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c0acf30 con 0x7fae3c0c8490 2023-10-08T00:57:06.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.399+0000 7fae327fc700 1 -- 172.21.15.119:0/2825811661 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 11) v1 ==== 84973+0+0 (secure 0 0 0) 0x7fae2401b3b0 con 0x7fae3c0c8490 2023-10-08T00:57:06.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.400+0000 7fae327fc700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fae1c0671c0 0x7fae1c069670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T00:57:06.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.400+0000 7fae327fc700 1 -- 172.21.15.119:0/2825811661 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fae24083e70 con 0x7fae3c0c8490 2023-10-08T00:57:06.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.401+0000 7fae3a59c700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fae1c0671c0 0x7fae1c069670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T00:57:06.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.401+0000 7fae3a59c700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fae1c0671c0 0x7fae1c069670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae3c156970 tx=0x7fae2c007bf0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T00:57:06.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.404+0000 7fae327fc700 1 -- 172.21.15.119:0/2825811661 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae24050d50 con 0x7fae3c0c8490 2023-10-08T00:57:06.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.621+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fae3c0718b0 con 0x7fae1c0671c0 2023-10-08T00:57:06.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.667+0000 7fae327fc700 1 -- 172.21.15.119:0/2825811661 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fae3c0718b0 con 0x7fae1c0671c0 2023-10-08T00:57:06.667 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-10-08T00:57:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fae1c0671c0 msgr2=0x7fae1c069670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fae1c0671c0 0x7fae1c069670 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae3c156970 tx=0x7fae2c007bf0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c0c8490 msgr2=0x7fae3c0a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c0c8490 0x7fae3c0a98e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c590 tx=0x7fae2400c8a0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 shutdown_connections 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae3c1570c0 0x7fae3c0a9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fae1c0671c0 0x7fae1c069670 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae3c069a50 0x7fae3c147090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 --2- 172.21.15.119:0/2825811661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae3c0c8490 0x7fae3c0a98e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 >> 172.21.15.119:0/2825811661 conn(0x7fae3c0c8f90 msgr2=0x7fae3c069e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 shutdown_connections 2023-10-08T00:57:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T00:57:06.669+0000 7fae40bff700 1 -- 172.21.15.119:0/2825811661 wait complete. 2023-10-08T00:57:06.686 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-10-08T00:57:13.588 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:57:13.593 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:57:43.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:57:43.708 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:58:13.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:58:13.820 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:58:43.922 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:58:43.927 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:59:14.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:59:14.037 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:59:44.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T00:59:44.151 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:00:14.266 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:00:14.272 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:00:44.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:00:44.378 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:01:14.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:01:14.494 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:01:44.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:01:44.612 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:02:00.832 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:02:00.832+0000 7f44ea656700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:02:00.832 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:02:00.832+0000 7f44ea656700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:02:00.832 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:02:00.832+0000 7f44ea656700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:02:00.832 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:02:00.832+0000 7f44ea656700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:02:00.832 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:02:00.832+0000 7f44ea656700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:02:14.720 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:02:14.726 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:02:44.845 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:02:44.850 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:03:14.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:03:14.950 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:03:45.050 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:03:45.056 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:04:15.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:04:15.165 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:04:45.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:04:45.273 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:05:00.355 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:05:00.354+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:05:00.356 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:05:00.354+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:05:00.356 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:05:00.354+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:05:00.356 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:05:00.354+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:05:00.356 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:05:00.354+0000 7f44e8e53700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:05:15.398 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:05:15.404 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:05:45.527 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:05:45.533 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:06:15.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:06:15.774 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:06:45.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:06:45.878 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:07:00.506 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:07:00.505+0000 7f44eb658700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:07:00.506 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:07:00.505+0000 7f44eb658700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:07:00.506 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:07:00.505+0000 7f44eb658700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:07:00.506 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:07:00.505+0000 7f44eb658700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:07:00.507 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:07:00.505+0000 7f44eb658700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:07:16.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:07:16.029 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:07:46.130 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:07:46.136 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:08:16.260 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:08:16.266 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:08:20.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-10-08T01:08:20.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.882+0000 7f3430d9d700 1 Processor -- start 2023-10-08T01:08:20.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.883+0000 7f3430d9d700 1 -- start start 2023-10-08T01:08:20.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.884+0000 7f3430d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f342c156350 0x7f342c157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:20.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.884+0000 7f3430d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c0b0570 con 0x7f342c0b0760 2023-10-08T01:08:20.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.884+0000 7f3430d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342c15ad40 con 0x7f342c0b0c00 2023-10-08T01:08:20.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.884+0000 7f3430d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342c1578b0 con 0x7f342c156350 2023-10-08T01:08:20.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.884+0000 7f342b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f342c156350 0x7f342c157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:20.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.884+0000 7f342affd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f342c0b0760 0x7f342c159f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53308/0 (socket says 172.21.15.119:53308) 2023-10-08T01:08:20.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.884+0000 7f342affd700 1 -- 172.21.15.119:0/2407868619 learned_addr learned my addr 172.21.15.119:0/2407868619 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:20.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.884+0000 7f342b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f342c156350 0x7f342c157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55658/0 (socket says 172.21.15.119:55658) 2023-10-08T01:08:20.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.885+0000 7f342b7fe700 1 -- 172.21.15.119:0/2407868619 >> v1:172.21.15.184:6789/0 conn(0x7f342c0b0c00 legacy=0x7f342c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:20.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.885+0000 7f342b7fe700 1 -- 172.21.15.119:0/2407868619 >> v1:172.21.15.119:6789/0 conn(0x7f342c0b0760 legacy=0x7f342c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:20.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.885+0000 7f342b7fe700 1 -- 172.21.15.119:0/2407868619 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342c0af4d0 con 0x7f342c156350 2023-10-08T01:08:20.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.885+0000 7f342a7fc700 1 -- 172.21.15.119:0/2407868619 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121576279 0 0) 0x7f342c0b0570 con 0x7f342c0b0760 2023-10-08T01:08:20.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.885+0000 7f342b7fe700 1 --2- 172.21.15.119:0/2407868619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c156350 0x7f342c157370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f341c0098e0 tx=0x7f341c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4871e4b33b370a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:20.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.885+0000 7f342a7fc700 1 -- 172.21.15.119:0/2407868619 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c009e80 con 0x7f342c156350 2023-10-08T01:08:20.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.885+0000 7f342a7fc700 1 -- 172.21.15.119:0/2407868619 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f341c00b820 con 0x7f342c156350 2023-10-08T01:08:20.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.885+0000 7f342a7fc700 1 -- 172.21.15.119:0/2407868619 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f341c00bba0 con 0x7f342c156350 2023-10-08T01:08:20.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.886+0000 7f3430d9d700 1 -- 172.21.15.119:0/2407868619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c156350 msgr2=0x7f342c157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:20.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.886+0000 7f3430d9d700 1 --2- 172.21.15.119:0/2407868619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c156350 0x7f342c157370 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f341c0098e0 tx=0x7f341c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:20.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.886+0000 7f3430d9d700 1 -- 172.21.15.119:0/2407868619 shutdown_connections 2023-10-08T01:08:20.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.886+0000 7f3430d9d700 1 --2- 172.21.15.119:0/2407868619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c156350 0x7f342c157370 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:20.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.886+0000 7f3430d9d700 1 -- 172.21.15.119:0/2407868619 >> 172.21.15.119:0/2407868619 conn(0x7f342c068e20 msgr2=0x7f342c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.886+0000 7f3430d9d700 1 -- 172.21.15.119:0/2407868619 shutdown_connections 2023-10-08T01:08:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.886+0000 7f3430d9d700 1 -- 172.21.15.119:0/2407868619 wait complete. 2023-10-08T01:08:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.887+0000 7f3430d9d700 1 Processor -- start 2023-10-08T01:08:20.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.888+0000 7f3430d9d700 1 -- start start 2023-10-08T01:08:20.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.888+0000 7f3430d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f342c0b0760 0x7f342c148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:20.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.888+0000 7f3430d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c0b0c00 0x7f342c0ad9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:20.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.888+0000 7f3430d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f342c156350 0x7f342c0a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:20.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.888+0000 7f3430d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0ad530 con 0x7f342c0b0c00 2023-10-08T01:08:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.888+0000 7f3430d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c149170 con 0x7f342c156350 2023-10-08T01:08:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.888+0000 7f3430d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0acf10 con 0x7f342c0b0760 2023-10-08T01:08:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c0b0c00 0x7f342c0ad9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c0b0c00 0x7f342c0ad9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55668/0 (socket says 172.21.15.119:55668) 2023-10-08T01:08:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 -- 172.21.15.119:0/1283729925 learned_addr learned my addr 172.21.15.119:0/1283729925 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342b7fe700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f342c0b0760 0x7f342c148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 -- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f342c0b0760 msgr2=0x7f342c148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:20.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342bfff700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f342c156350 0x7f342c0a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f342c0b0760 0x7f342c148c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 -- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f342c156350 msgr2=0x7f342c0a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f342c156350 0x7f342c0a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 -- 172.21.15.119:0/1283729925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341c009420 con 0x7f342c0b0c00 2023-10-08T01:08:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342b7fe700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f342c0b0760 0x7f342c148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342bfff700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f342c156350 0x7f342c0a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.889+0000 7f342affd700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c0b0c00 0x7f342c0ad9c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f342000c5b0 tx=0x7f3420001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:20.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.890+0000 7f34137fe700 1 -- 172.21.15.119:0/1283729925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420005b30 con 0x7f342c0b0c00 2023-10-08T01:08:20.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.890+0000 7f34137fe700 1 -- 172.21.15.119:0/1283729925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3420005c90 con 0x7f342c0b0c00 2023-10-08T01:08:20.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.890+0000 7f34137fe700 1 -- 172.21.15.119:0/1283729925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3420007300 con 0x7f342c0b0c00 2023-10-08T01:08:20.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.890+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c152e70 con 0x7f342c0b0c00 2023-10-08T01:08:20.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.890+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c0ad1c0 con 0x7f342c0b0c00 2023-10-08T01:08:20.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.891+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c153390 con 0x7f342c0b0c00 2023-10-08T01:08:20.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.892+0000 7f34137fe700 1 -- 172.21.15.119:0/1283729925 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f342001b3b0 con 0x7f342c0b0c00 2023-10-08T01:08:20.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.893+0000 7f34137fe700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3418067130 0x7f34180695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:20.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.893+0000 7f34137fe700 1 -- 172.21.15.119:0/1283729925 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f3420083f50 con 0x7f342c0b0c00 2023-10-08T01:08:20.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.893+0000 7f342b7fe700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3418067130 0x7f34180695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:20.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.894+0000 7f342b7fe700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3418067130 0x7f34180695e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f341c0030c0 tx=0x7f341c016000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:20.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:20.898+0000 7f34137fe700 1 -- 172.21.15.119:0/1283729925 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3420050da0 con 0x7f342c0b0c00 2023-10-08T01:08:21.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.112+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f342c04c3c0 con 0x7f3418067130 2023-10-08T01:08:21.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.180+0000 7f34137fe700 1 -- 172.21.15.119:0/1283729925 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f342c04c3c0 con 0x7f3418067130 2023-10-08T01:08:21.181 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-10-08T01:08:21.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3418067130 msgr2=0x7f34180695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3418067130 0x7f34180695e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f341c0030c0 tx=0x7f341c016000 comp rx=0 tx=0).stop 2023-10-08T01:08:21.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c0b0c00 msgr2=0x7f342c0ad9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c0b0c00 0x7f342c0ad9c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f342000c5b0 tx=0x7f3420001070 comp rx=0 tx=0).stop 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 shutdown_connections 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f342c0b0760 0x7f342c148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3418067130 0x7f34180695e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f342c156350 0x7f342c0a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 --2- 172.21.15.119:0/1283729925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f342c0b0c00 0x7f342c0ad9c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 >> 172.21.15.119:0/1283729925 conn(0x7f342c068e20 msgr2=0x7f342c154670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 shutdown_connections 2023-10-08T01:08:21.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.182+0000 7f3430d9d700 1 -- 172.21.15.119:0/1283729925 wait complete. 2023-10-08T01:08:21.200 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-10-08T01:08:21.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-10-08T01:08:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.386+0000 7f1572d9d700 1 Processor -- start 2023-10-08T01:08:21.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.386+0000 7f1572d9d700 1 -- start start 2023-10-08T01:08:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.388+0000 7f1572d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f156c0b0760 0x7f156c0b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.388+0000 7f1572d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0b0570 con 0x7f156c0ace60 2023-10-08T01:08:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.388+0000 7f1572d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f156c0af4d0 con 0x7f156c156350 2023-10-08T01:08:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.388+0000 7f1572d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f156c0b1140 con 0x7f156c0b0760 2023-10-08T01:08:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.388+0000 7f157159a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f156c0b0760 0x7f156c0b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.388+0000 7f157259c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f156c0ace60 0x7f156c0ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53310/0 (socket says 172.21.15.119:53310) 2023-10-08T01:08:21.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.388+0000 7f157159a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f156c0b0760 0x7f156c0b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55682/0 (socket says 172.21.15.119:55682) 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.388+0000 7f157259c700 1 -- 172.21.15.119:0/2003863293 learned_addr learned my addr 172.21.15.119:0/2003863293 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.389+0000 7f157159a700 1 -- 172.21.15.119:0/2003863293 >> v1:172.21.15.184:6789/0 conn(0x7f156c156350 legacy=0x7f156c157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.389+0000 7f157159a700 1 -- 172.21.15.119:0/2003863293 >> v1:172.21.15.119:6789/0 conn(0x7f156c0ace60 legacy=0x7f156c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.389+0000 7f157159a700 1 -- 172.21.15.119:0/2003863293 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0af7b0 con 0x7f156c0b0760 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.389+0000 7f1570d99700 1 -- 172.21.15.119:0/2003863293 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880410219 0 0) 0x7f156c0b0570 con 0x7f156c0ace60 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.389+0000 7f157159a700 1 --2- 172.21.15.119:0/2003863293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c0b0c00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f155c0098e0 tx=0x7f155c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc7104be4e4470 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.389+0000 7f1570d99700 1 -- 172.21.15.119:0/2003863293 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c009e80 con 0x7f156c0b0760 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.389+0000 7f1570d99700 1 -- 172.21.15.119:0/2003863293 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c00b860 con 0x7f156c0b0760 2023-10-08T01:08:21.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.389+0000 7f1570d99700 1 -- 172.21.15.119:0/2003863293 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c00bbe0 con 0x7f156c0b0760 2023-10-08T01:08:21.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.390+0000 7f1572d9d700 1 -- 172.21.15.119:0/2003863293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 msgr2=0x7f156c0b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.390+0000 7f1572d9d700 1 --2- 172.21.15.119:0/2003863293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c0b0c00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f155c0098e0 tx=0x7f155c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.390+0000 7f1572d9d700 1 -- 172.21.15.119:0/2003863293 shutdown_connections 2023-10-08T01:08:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.390+0000 7f1572d9d700 1 --2- 172.21.15.119:0/2003863293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c0b0c00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.390+0000 7f1572d9d700 1 -- 172.21.15.119:0/2003863293 >> 172.21.15.119:0/2003863293 conn(0x7f156c068e20 msgr2=0x7f156c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.390+0000 7f1572d9d700 1 -- 172.21.15.119:0/2003863293 shutdown_connections 2023-10-08T01:08:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.390+0000 7f1572d9d700 1 -- 172.21.15.119:0/2003863293 wait complete. 2023-10-08T01:08:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.391+0000 7f1572d9d700 1 Processor -- start 2023-10-08T01:08:21.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.391+0000 7f1572d9d700 1 -- start start 2023-10-08T01:08:21.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.392+0000 7f1572d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f156c0ace60 0x7f156c0aaf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.392+0000 7f1572d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c1698a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.392+0000 7f1572d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f156c156350 0x7f156c0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.392+0000 7f1572d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0a98d0 con 0x7f156c0b0760 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.392+0000 7f1572d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c16a0d0 con 0x7f156c156350 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.392+0000 7f1572d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f156c16a3b0 con 0x7f156c0ace60 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c1698a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c1698a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55686/0 (socket says 172.21.15.119:55686) 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 -- 172.21.15.119:0/1258451293 learned_addr learned my addr 172.21.15.119:0/1258451293 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157259c700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f156c156350 0x7f156c0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f1571d9b700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f156c0ace60 0x7f156c0aaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 -- 172.21.15.119:0/1258451293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f156c0ace60 msgr2=0x7f156c0aaf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f156c0ace60 0x7f156c0aaf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 -- 172.21.15.119:0/1258451293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f156c156350 msgr2=0x7f156c0c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f156c156350 0x7f156c0c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 -- 172.21.15.119:0/1258451293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c009420 con 0x7f156c0b0760 2023-10-08T01:08:21.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.393+0000 7f157159a700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c1698a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f156c0714a0 tx=0x7f155c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:21.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.394+0000 7f1561ffb700 1 -- 172.21.15.119:0/1258451293 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c01f070 con 0x7f156c0b0760 2023-10-08T01:08:21.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.394+0000 7f1561ffb700 1 -- 172.21.15.119:0/1258451293 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f155c000c90 con 0x7f156c0b0760 2023-10-08T01:08:21.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.394+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c16a690 con 0x7f156c0b0760 2023-10-08T01:08:21.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.394+0000 7f1561ffb700 1 -- 172.21.15.119:0/1258451293 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155c025420 con 0x7f156c0b0760 2023-10-08T01:08:21.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.394+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c169de0 con 0x7f156c0b0760 2023-10-08T01:08:21.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.395+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c0a9db0 con 0x7f156c0b0760 2023-10-08T01:08:21.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.396+0000 7f1561ffb700 1 -- 172.21.15.119:0/1258451293 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f155c023070 con 0x7f156c0b0760 2023-10-08T01:08:21.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.396+0000 7f1561ffb700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1550067130 0x7f15500695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.397+0000 7f1561ffb700 1 -- 172.21.15.119:0/1258451293 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f155c085d50 con 0x7f156c0b0760 2023-10-08T01:08:21.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.397+0000 7f1571d9b700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1550067130 0x7f15500695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.397+0000 7f1571d9b700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1550067130 0x7f15500695e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f156c0ca7e0 tx=0x7f1568009040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:21.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.401+0000 7f1561ffb700 1 -- 172.21.15.119:0/1258451293 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f156c0a9db0 con 0x7f156c0b0760 2023-10-08T01:08:21.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.619+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f156c0a9db0 con 0x7f1550067130 2023-10-08T01:08:21.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.684+0000 7f1561ffb700 1 -- 172.21.15.119:0/1258451293 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f156c0a9db0 con 0x7f1550067130 2023-10-08T01:08:21.685 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-10-08T01:08:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1550067130 msgr2=0x7f15500695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1550067130 0x7f15500695e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f156c0ca7e0 tx=0x7f1568009040 comp rx=0 tx=0).stop 2023-10-08T01:08:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 msgr2=0x7f156c1698a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c1698a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f156c0714a0 tx=0x7f155c017000 comp rx=0 tx=0).stop 2023-10-08T01:08:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 shutdown_connections 2023-10-08T01:08:21.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f156c0ace60 0x7f156c0aaf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f1550067130 0x7f15500695e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f156c156350 0x7f156c0c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 --2- 172.21.15.119:0/1258451293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f156c0b0760 0x7f156c1698a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.686+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 >> 172.21.15.119:0/1258451293 conn(0x7f156c068e20 msgr2=0x7f156c156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:21.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.687+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 shutdown_connections 2023-10-08T01:08:21.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.687+0000 7f1572d9d700 1 -- 172.21.15.119:0/1258451293 wait complete. 2023-10-08T01:08:21.705 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-10-08T01:08:21.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-10-08T01:08:21.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.890+0000 7f47819ff700 1 Processor -- start 2023-10-08T01:08:21.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.890+0000 7f47819ff700 1 -- start start 2023-10-08T01:08:21.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.891+0000 7f47819ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c0c8490 0x7f477c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.891+0000 7f47819ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f477c069e20 0x7f477c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.891+0000 7f47819ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f477c0ac030 con 0x7f477c069a50 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.891+0000 7f47819ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477c0ab820 con 0x7f477c069e20 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.891+0000 7f47819ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c06dbb0 con 0x7f477c0c8490 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f477c069e20 0x7f477c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f477c069e20 0x7f477c1569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55690/0 (socket says 172.21.15.119:55690) 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477b7fe700 1 -- 172.21.15.119:0/4151526662 learned_addr learned my addr 172.21.15.119:0/4151526662 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477b7fe700 1 -- 172.21.15.119:0/4151526662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c0c8490 msgr2=0x7f477c0c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477a7fc700 1 --2- 172.21.15.119:0/4151526662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c0c8490 0x7f477c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477b7fe700 1 --2- 172.21.15.119:0/4151526662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c0c8490 0x7f477c0c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477b7fe700 1 -- 172.21.15.119:0/4151526662 >> v1:172.21.15.184:6789/0 conn(0x7f477c069a50 legacy=0x7f477c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:21.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477b7fe700 1 -- 172.21.15.119:0/4151526662 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c0aba70 con 0x7f477c069e20 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477a7fc700 1 --2- 172.21.15.119:0/4151526662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c0c8490 0x7f477c0c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.892+0000 7f477b7fe700 1 --2- 172.21.15.119:0/4151526662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c069e20 0x7f477c1569b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f477400c100 tx=0x7f477400a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb39d9ae1920b9b7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f4779ffb700 1 -- 172.21.15.119:0/4151526662 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400f070 con 0x7f477c069e20 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f4779ffb700 1 -- 172.21.15.119:0/4151526662 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f477400a2e0 con 0x7f477c069e20 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f4779ffb700 1 -- 172.21.15.119:0/4151526662 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400cde0 con 0x7f477c069e20 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f47819ff700 1 -- 172.21.15.119:0/4151526662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c069e20 msgr2=0x7f477c1569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f47819ff700 1 --2- 172.21.15.119:0/4151526662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c069e20 0x7f477c1569b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f477400c100 tx=0x7f477400a950 comp rx=0 tx=0).stop 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f47819ff700 1 -- 172.21.15.119:0/4151526662 shutdown_connections 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f47819ff700 1 --2- 172.21.15.119:0/4151526662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c069e20 0x7f477c1569b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f47819ff700 1 --2- 172.21.15.119:0/4151526662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c0c8490 0x7f477c0c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f47819ff700 1 -- 172.21.15.119:0/4151526662 >> 172.21.15.119:0/4151526662 conn(0x7f477c0c8f90 msgr2=0x7f477c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f47819ff700 1 -- 172.21.15.119:0/4151526662 shutdown_connections 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.893+0000 7f47819ff700 1 -- 172.21.15.119:0/4151526662 wait complete. 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.895+0000 7f47819ff700 1 Processor -- start 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.895+0000 7f47819ff700 1 -- start start 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f47819ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f477c069a50 0x7f477c0c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f47819ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c069e20 0x7f477c154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f47819ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c0c8490 0x7f477c155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f47819ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c0c3330 con 0x7f477c0c8490 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f47819ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f477c157820 con 0x7f477c069a50 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f47819ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f477c157df0 con 0x7f477c069e20 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c0c8490 0x7f477c155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c0c8490 0x7f477c155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55698/0 (socket says 172.21.15.119:55698) 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477b7fe700 1 -- 172.21.15.119:0/1519776278 learned_addr learned my addr 172.21.15.119:0/1519776278 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477b7fe700 1 -- 172.21.15.119:0/1519776278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c069e20 msgr2=0x7f477c154880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477b7fe700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c069e20 0x7f477c154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477b7fe700 1 -- 172.21.15.119:0/1519776278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f477c069a50 msgr2=0x7f477c0c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:21.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477affd700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f477c069a50 0x7f477c0c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477b7fe700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f477c069a50 0x7f477c0c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477b7fe700 1 -- 172.21.15.119:0/1519776278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477400abf0 con 0x7f477c0c8490 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.896+0000 7f477affd700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f477c069a50 0x7f477c0c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.897+0000 7f477b7fe700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c0c8490 0x7f477c155770 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f477400abc0 tx=0x7f4774018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.897+0000 7f4762ffd700 1 -- 172.21.15.119:0/1519776278 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477400a5f0 con 0x7f477c0c8490 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.897+0000 7f4762ffd700 1 -- 172.21.15.119:0/1519776278 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4774008290 con 0x7f477c0c8490 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.897+0000 7f4762ffd700 1 -- 172.21.15.119:0/1519776278 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774013680 con 0x7f477c0c8490 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.897+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477c1580d0 con 0x7f477c0c8490 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.897+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477c158690 con 0x7f477c0c8490 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.898+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f477c0718b0 con 0x7f477c0c8490 2023-10-08T01:08:21.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.899+0000 7f4762ffd700 1 -- 172.21.15.119:0/1519776278 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f477401a070 con 0x7f477c0c8490 2023-10-08T01:08:21.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.901+0000 7f4762ffd700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4768067130 0x7f47680695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:21.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.901+0000 7f4762ffd700 1 -- 172.21.15.119:0/1519776278 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f4774086090 con 0x7f477c0c8490 2023-10-08T01:08:21.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.901+0000 7f477affd700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4768067130 0x7f47680695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:21.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.901+0000 7f477affd700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4768067130 0x7f47680695e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f476c002880 tx=0x7f476c009040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:21.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:21.905+0000 7f4762ffd700 1 -- 172.21.15.119:0/1519776278 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4774052dd0 con 0x7f477c0c8490 2023-10-08T01:08:22.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.123+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f477c0ac030 con 0x7f4768067130 2023-10-08T01:08:22.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.187+0000 7f4762ffd700 1 -- 172.21.15.119:0/1519776278 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f477c0ac030 con 0x7f4768067130 2023-10-08T01:08:22.189 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-10-08T01:08:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.189+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4768067130 msgr2=0x7f47680695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.189+0000 7f47819ff700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4768067130 0x7f47680695e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f476c002880 tx=0x7f476c009040 comp rx=0 tx=0).stop 2023-10-08T01:08:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.189+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c0c8490 msgr2=0x7f477c155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.189+0000 7f47819ff700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c0c8490 0x7f477c155770 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f477400abc0 tx=0x7f4774018000 comp rx=0 tx=0).stop 2023-10-08T01:08:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.190+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 shutdown_connections 2023-10-08T01:08:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.190+0000 7f47819ff700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4768067130 0x7f47680695e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.190+0000 7f47819ff700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f477c0c8490 0x7f477c155770 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.190+0000 7f47819ff700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f477c069a50 0x7f477c0c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.190+0000 7f47819ff700 1 --2- 172.21.15.119:0/1519776278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f477c069e20 0x7f477c154880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.190+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 >> 172.21.15.119:0/1519776278 conn(0x7f477c0c8f90 msgr2=0x7f477c0cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.190+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 shutdown_connections 2023-10-08T01:08:22.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.190+0000 7f47819ff700 1 -- 172.21.15.119:0/1519776278 wait complete. 2023-10-08T01:08:22.208 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-10-08T01:08:22.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-10-08T01:08:22.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.394+0000 7f5bc47ff700 1 Processor -- start 2023-10-08T01:08:22.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.394+0000 7f5bc47ff700 1 -- start start 2023-10-08T01:08:22.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.395+0000 7f5bc47ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5bbc0c8490 0x7f5bbc0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.395+0000 7f5bc47ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5bbc069e20 0x7f5bbc1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.395+0000 7f5bc47ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bbc0ac030 con 0x7f5bbc069a50 2023-10-08T01:08:22.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.395+0000 7f5bc47ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc0ab820 con 0x7f5bbc0c8490 2023-10-08T01:08:22.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.395+0000 7f5bc47ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bbc06dbb0 con 0x7f5bbc069e20 2023-10-08T01:08:22.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc259b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5bbc069a50 0x7f5bbc15a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53324/0 (socket says 172.21.15.119:53324) 2023-10-08T01:08:22.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc259b700 1 -- 172.21.15.119:0/406223146 learned_addr learned my addr 172.21.15.119:0/406223146 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:22.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc1d9a700 1 --2- 172.21.15.119:0/406223146 >> v2:172.21.15.119:3300/0 conn(0x7f5bbc0c8490 0x7f5bbc0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc2d9c700 1 --2- 172.21.15.119:0/406223146 >> v2:172.21.15.184:3300/0 conn(0x7f5bbc069e20 0x7f5bbc1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc1599700 1 -- 172.21.15.119:0/406223146 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253585466 0 0) 0x7f5bb8001110 con 0x7f5bbc069a50 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc1599700 1 -- 172.21.15.119:0/406223146 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ba8003570 con 0x7f5bbc069a50 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc1d9a700 1 -- 172.21.15.119:0/406223146 >> v2:172.21.15.184:3300/0 conn(0x7f5bbc069e20 msgr2=0x7f5bbc1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc1d9a700 1 --2- 172.21.15.119:0/406223146 >> v2:172.21.15.184:3300/0 conn(0x7f5bbc069e20 0x7f5bbc1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc1d9a700 1 -- 172.21.15.119:0/406223146 >> v1:172.21.15.119:6789/0 conn(0x7f5bbc069a50 legacy=0x7f5bbc15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc1d9a700 1 -- 172.21.15.119:0/406223146 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc0aba70 con 0x7f5bbc0c8490 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.396+0000 7f5bc1d9a700 1 --2- 172.21.15.119:0/406223146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc0c8490 0x7f5bbc0c6ac0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc06db20 tx=0x7f5bac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237c4a475bf07538 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc1599700 1 -- 172.21.15.119:0/406223146 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011630 con 0x7f5bbc0c8490 2023-10-08T01:08:22.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc1599700 1 -- 172.21.15.119:0/406223146 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac011790 con 0x7f5bbc0c8490 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc1599700 1 -- 172.21.15.119:0/406223146 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011b30 con 0x7f5bbc0c8490 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc47ff700 1 -- 172.21.15.119:0/406223146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc0c8490 msgr2=0x7f5bbc0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/406223146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc0c8490 0x7f5bbc0c6ac0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc06db20 tx=0x7f5bac0091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc47ff700 1 -- 172.21.15.119:0/406223146 shutdown_connections 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/406223146 >> v2:172.21.15.184:3300/0 conn(0x7f5bbc069e20 0x7f5bbc1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/406223146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc0c8490 0x7f5bbc0c6ac0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc47ff700 1 -- 172.21.15.119:0/406223146 >> 172.21.15.119:0/406223146 conn(0x7f5bbc0c8f90 msgr2=0x7f5bbc0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc47ff700 1 -- 172.21.15.119:0/406223146 shutdown_connections 2023-10-08T01:08:22.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.397+0000 7f5bc47ff700 1 -- 172.21.15.119:0/406223146 wait complete. 2023-10-08T01:08:22.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.399+0000 7f5bc47ff700 1 Processor -- start 2023-10-08T01:08:22.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.399+0000 7f5bc47ff700 1 -- start start 2023-10-08T01:08:22.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc47ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bbc069a50 0x7f5bbc167f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc47ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc069e20 0x7f5bbc1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc47ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bbc0c8490 0x7f5bbc0c0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc47ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0c1310 con 0x7f5bbc069e20 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc47ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc1550e0 con 0x7f5bbc069a50 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc47ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bbc0b0970 con 0x7f5bbc0c8490 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc1d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc069e20 0x7f5bbc1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc1d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc069e20 0x7f5bbc1544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55712/0 (socket says 172.21.15.119:55712) 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc1d9a700 1 -- 172.21.15.119:0/1203357081 learned_addr learned my addr 172.21.15.119:0/1203357081 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc259b700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bbc069a50 0x7f5bbc167f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc1d9a700 1 -- 172.21.15.119:0/1203357081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bbc0c8490 msgr2=0x7f5bbc0c0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc1d9a700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bbc0c8490 0x7f5bbc0c0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc1d9a700 1 -- 172.21.15.119:0/1203357081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bbc069a50 msgr2=0x7f5bbc167f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc1d9a700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bbc069a50 0x7f5bbc167f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc1d9a700 1 -- 172.21.15.119:0/1203357081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac009420 con 0x7f5bbc069e20 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.400+0000 7f5bc259b700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bbc069a50 0x7f5bbc167f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.401+0000 7f5bc1d9a700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc069e20 0x7f5bbc1544f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0cc0b0 tx=0x7f5bac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.401+0000 7f5bb27fc700 1 -- 172.21.15.119:0/1203357081 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011b10 con 0x7f5bbc069e20 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.401+0000 7f5bb27fc700 1 -- 172.21.15.119:0/1203357081 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac0233f0 con 0x7f5bbc069e20 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.401+0000 7f5bb27fc700 1 -- 172.21.15.119:0/1203357081 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac023770 con 0x7f5bbc069e20 2023-10-08T01:08:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.401+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bbc0b0c50 con 0x7f5bbc069e20 2023-10-08T01:08:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.401+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bbc0b1290 con 0x7f5bbc069e20 2023-10-08T01:08:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.402+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bbc0b1420 con 0x7f5bbc069e20 2023-10-08T01:08:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.403+0000 7f5bb27fc700 1 -- 172.21.15.119:0/1203357081 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f5bac0193b0 con 0x7f5bbc069e20 2023-10-08T01:08:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.403+0000 7f5bb27fc700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ba0067130 0x7f5ba00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.404+0000 7f5bb27fc700 1 -- 172.21.15.119:0/1203357081 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f5bac085f80 con 0x7f5bbc069e20 2023-10-08T01:08:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.404+0000 7f5bc259b700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ba0067130 0x7f5ba00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.404+0000 7f5bc259b700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ba0067130 0x7f5ba00695e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80058d0 tx=0x7f5bb8009040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.408+0000 7f5bb27fc700 1 -- 172.21.15.119:0/1203357081 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5bac0b75a0 con 0x7f5bbc069e20 2023-10-08T01:08:22.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.624+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bbc0b0ae0 con 0x7f5ba0067130 2023-10-08T01:08:22.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.691+0000 7f5bb27fc700 1 -- 172.21.15.119:0/1203357081 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5bbc0b0ae0 con 0x7f5ba0067130 2023-10-08T01:08:22.693 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-10-08T01:08:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ba0067130 msgr2=0x7f5ba00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ba0067130 0x7f5ba00695e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80058d0 tx=0x7f5bb8009040 comp rx=0 tx=0).stop 2023-10-08T01:08:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc069e20 msgr2=0x7f5bbc1544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc069e20 0x7f5bbc1544f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0cc0b0 tx=0x7f5bac017000 comp rx=0 tx=0).stop 2023-10-08T01:08:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 shutdown_connections 2023-10-08T01:08:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5bbc0c8490 0x7f5bbc0c0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f5ba0067130 0x7f5ba00695e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5bbc069a50 0x7f5bbc167f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 --2- 172.21.15.119:0/1203357081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5bbc069e20 0x7f5bbc1544f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 >> 172.21.15.119:0/1203357081 conn(0x7f5bbc0c8f90 msgr2=0x7f5bbc156710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 shutdown_connections 2023-10-08T01:08:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.694+0000 7f5bc47ff700 1 -- 172.21.15.119:0/1203357081 wait complete. 2023-10-08T01:08:22.712 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-10-08T01:08:22.712 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-10-08T01:08:22.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.898+0000 7ff072dff700 1 Processor -- start 2023-10-08T01:08:22.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.898+0000 7ff072dff700 1 -- start start 2023-10-08T01:08:22.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.899+0000 7ff072dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c159ec0 0x7ff06c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.899+0000 7ff072dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff06c0b53d0 0x7ff06c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.899+0000 7ff072dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c0b5b00 con 0x7ff06c0cce40 2023-10-08T01:08:22.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.899+0000 7ff072dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c0b5da0 con 0x7ff06c0b53d0 2023-10-08T01:08:22.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.899+0000 7ff072dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c0b6080 con 0x7ff06c159ec0 2023-10-08T01:08:22.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff070b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c159ec0 0x7ff06c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff06bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff06c0b53d0 0x7ff06c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff07139c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7ff06c0cce40 0x7ff06c0cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:54192/0 (socket says 172.21.15.119:54192) 2023-10-08T01:08:22.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff070b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c159ec0 0x7ff06c0b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:51674/0 (socket says 172.21.15.119:51674) 2023-10-08T01:08:22.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff07139c700 1 -- 172.21.15.119:0/233725477 learned_addr learned my addr 172.21.15.119:0/233725477 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:22.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff06bfff700 1 -- 172.21.15.119:0/233725477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c159ec0 msgr2=0x7ff06c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:22.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff06bfff700 1 --2- 172.21.15.119:0/233725477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c159ec0 0x7ff06c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff06bfff700 1 -- 172.21.15.119:0/233725477 >> v1:172.21.15.184:6789/0 conn(0x7ff06c0cce40 legacy=0x7ff06c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff06bfff700 1 -- 172.21.15.119:0/233725477 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c0cc100 con 0x7ff06c0b53d0 2023-10-08T01:08:22.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.900+0000 7ff070b9b700 1 --2- 172.21.15.119:0/233725477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c159ec0 0x7ff06c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:22.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.901+0000 7ff06bfff700 1 --2- 172.21.15.119:0/233725477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 0x7ff06c0cc900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0580098e0 tx=0x7ff0580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e406908145dc495 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:22.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.901+0000 7ff06b7fe700 1 -- 172.21.15.119:0/233725477 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05800b640 con 0x7ff06c0b53d0 2023-10-08T01:08:22.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.901+0000 7ff06b7fe700 1 -- 172.21.15.119:0/233725477 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05800b7a0 con 0x7ff06c0b53d0 2023-10-08T01:08:22.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.901+0000 7ff06b7fe700 1 -- 172.21.15.119:0/233725477 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05800bb40 con 0x7ff06c0b53d0 2023-10-08T01:08:22.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.901+0000 7ff072dff700 1 -- 172.21.15.119:0/233725477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 msgr2=0x7ff06c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.901+0000 7ff072dff700 1 --2- 172.21.15.119:0/233725477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 0x7ff06c0cc900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0580098e0 tx=0x7ff0580091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.902+0000 7ff072dff700 1 -- 172.21.15.119:0/233725477 shutdown_connections 2023-10-08T01:08:22.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.902+0000 7ff072dff700 1 --2- 172.21.15.119:0/233725477 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c159ec0 0x7ff06c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.902+0000 7ff072dff700 1 --2- 172.21.15.119:0/233725477 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 0x7ff06c0cc900 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.902+0000 7ff072dff700 1 -- 172.21.15.119:0/233725477 >> 172.21.15.119:0/233725477 conn(0x7ff06c0999f0 msgr2=0x7ff06c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:22.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.902+0000 7ff072dff700 1 -- 172.21.15.119:0/233725477 shutdown_connections 2023-10-08T01:08:22.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.902+0000 7ff072dff700 1 -- 172.21.15.119:0/233725477 wait complete. 2023-10-08T01:08:22.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.903+0000 7ff072dff700 1 Processor -- start 2023-10-08T01:08:22.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.903+0000 7ff072dff700 1 -- start start 2023-10-08T01:08:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff072dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 0x7ff06c157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff072dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c0cce40 0x7ff06c144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff072dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff06c159ec0 0x7ff06c155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff072dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c1584e0 con 0x7ff06c159ec0 2023-10-08T01:08:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff072dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c157680 con 0x7ff06c0b53d0 2023-10-08T01:08:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff072dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c1578d0 con 0x7ff06c0cce40 2023-10-08T01:08:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff07139c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff06c159ec0 0x7ff06c155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff07139c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff06c159ec0 0x7ff06c155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55714/0 (socket says 172.21.15.119:55714) 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff07139c700 1 -- 172.21.15.119:0/1188256236 learned_addr learned my addr 172.21.15.119:0/1188256236 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff06bfff700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c0cce40 0x7ff06c144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.904+0000 7ff070b9b700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 0x7ff06c157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff07139c700 1 -- 172.21.15.119:0/1188256236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c0cce40 msgr2=0x7ff06c144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff07139c700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c0cce40 0x7ff06c144020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff07139c700 1 -- 172.21.15.119:0/1188256236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 msgr2=0x7ff06c157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff07139c700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 0x7ff06c157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff07139c700 1 -- 172.21.15.119:0/1188256236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0600093c0 con 0x7ff06c159ec0 2023-10-08T01:08:22.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff07139c700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff06c159ec0 0x7ff06c155070 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff06400cc40 tx=0x7ff064001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff068ff9700 1 -- 172.21.15.119:0/1188256236 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400a8b0 con 0x7ff06c159ec0 2023-10-08T01:08:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff068ff9700 1 -- 172.21.15.119:0/1188256236 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06400aa10 con 0x7ff06c159ec0 2023-10-08T01:08:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff058009420 con 0x7ff06c159ec0 2023-10-08T01:08:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff068ff9700 1 -- 172.21.15.119:0/1188256236 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400add0 con 0x7ff06c159ec0 2023-10-08T01:08:22.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.905+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c157c60 con 0x7ff06c159ec0 2023-10-08T01:08:22.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.907+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c158290 con 0x7ff06c159ec0 2023-10-08T01:08:22.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.907+0000 7ff068ff9700 1 -- 172.21.15.119:0/1188256236 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7ff064007300 con 0x7ff06c159ec0 2023-10-08T01:08:22.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.908+0000 7ff068ff9700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff050067130 0x7ff0500695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:22.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.908+0000 7ff068ff9700 1 -- 172.21.15.119:0/1188256236 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7ff064084bd0 con 0x7ff06c159ec0 2023-10-08T01:08:22.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.908+0000 7ff070b9b700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff050067130 0x7ff0500695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:22.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.909+0000 7ff070b9b700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff050067130 0x7ff0500695e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff060009e90 tx=0x7ff060009120 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:22.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:22.912+0000 7ff068ff9700 1 -- 172.21.15.119:0/1188256236 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff064051b40 con 0x7ff06c159ec0 2023-10-08T01:08:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.131+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff06c063c90 con 0x7ff050067130 2023-10-08T01:08:23.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.195+0000 7ff068ff9700 1 -- 172.21.15.119:0/1188256236 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff06c063c90 con 0x7ff050067130 2023-10-08T01:08:23.196 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-10-08T01:08:23.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff050067130 msgr2=0x7ff0500695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff050067130 0x7ff0500695e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff060009e90 tx=0x7ff060009120 comp rx=0 tx=0).stop 2023-10-08T01:08:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff06c159ec0 msgr2=0x7ff06c155070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff06c159ec0 0x7ff06c155070 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff06400cc40 tx=0x7ff064001a50 comp rx=0 tx=0).stop 2023-10-08T01:08:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 shutdown_connections 2023-10-08T01:08:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff050067130 0x7ff0500695e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff06c159ec0 0x7ff06c155070 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff06c0b53d0 0x7ff06c157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 --2- 172.21.15.119:0/1188256236 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff06c0cce40 0x7ff06c144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 >> 172.21.15.119:0/1188256236 conn(0x7ff06c0999f0 msgr2=0x7ff06c0c5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:23.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 shutdown_connections 2023-10-08T01:08:23.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.197+0000 7ff072dff700 1 -- 172.21.15.119:0/1188256236 wait complete. 2023-10-08T01:08:23.215 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-10-08T01:08:23.216 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-10-08T01:08:23.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.402+0000 7f404f19d700 1 Processor -- start 2023-10-08T01:08:23.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.402+0000 7f404f19d700 1 -- start start 2023-10-08T01:08:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.403+0000 7f404f19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f40480ae180 0x7f40480acb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.403+0000 7f404f19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40480aeb10 con 0x7f404806b290 2023-10-08T01:08:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.403+0000 7f404f19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40480af540 con 0x7f40480656e0 2023-10-08T01:08:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.403+0000 7f404f19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40480aedf0 con 0x7f40480ae180 2023-10-08T01:08:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.403+0000 7f404e99c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f404806b290 0x7f40480b1110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53338/0 (socket says 172.21.15.119:53338) 2023-10-08T01:08:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.403+0000 7f404e99c700 1 -- 172.21.15.119:0/3613505027 learned_addr learned my addr 172.21.15.119:0/3613505027 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.403+0000 7f404e19b700 1 --2- 172.21.15.119:0/3613505027 >> v2:172.21.15.184:3300/0 conn(0x7f40480ae180 0x7f40480acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446297038 0 0) 0x7f40480aeb10 con 0x7f404806b290 2023-10-08T01:08:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4034003570 con 0x7f404806b290 2023-10-08T01:08:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3357631037 0 0) 0x7f40480af540 con 0x7f40480656e0 2023-10-08T01:08:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40480aeb10 con 0x7f40480656e0 2023-10-08T01:08:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2151709236 0 0) 0x7f4034003570 con 0x7f404806b290 2023-10-08T01:08:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 >> v2:172.21.15.184:3300/0 conn(0x7f40480ae180 msgr2=0x7f40480acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 --2- 172.21.15.119:0/3613505027 >> v2:172.21.15.184:3300/0 conn(0x7f40480ae180 0x7f40480acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 >> v1:172.21.15.184:6789/0 conn(0x7f40480656e0 legacy=0x7f40480ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480af820 con 0x7f404806b290 2023-10-08T01:08:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f40400017f0 con 0x7f404806b290 2023-10-08T01:08:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 >> v1:172.21.15.119:6789/0 conn(0x7f404806b290 legacy=0x7f40480b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4034003c40 0x7f4034004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034004810 0x7f4034004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4034005210 0x7f403400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40480aedf0 con 0x7f4034005210 2023-10-08T01:08:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403400c1b0 con 0x7f4034003c40 2023-10-08T01:08:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403400c370 con 0x7f4034004810 2023-10-08T01:08:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404e99c700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4034005210 0x7f403400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404e19b700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4034003c40 0x7f4034004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.404+0000 7f404d99a700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034004810 0x7f4034004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404e99c700 1 -- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034004810 msgr2=0x7f4034004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404e99c700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034004810 0x7f4034004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404e99c700 1 -- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4034003c40 msgr2=0x7f4034004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404e99c700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4034003c40 0x7f4034004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404e99c700 1 -- 172.21.15.119:0/3613505027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4034003570 con 0x7f4034005210 2023-10-08T01:08:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404d99a700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034004810 0x7f4034004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404e99c700 1 -- 172.21.15.119:0/3613505027 reap_dead start 2023-10-08T01:08:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404e99c700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4034005210 0x7f403400ba60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f40480642c0 tx=0x7f40400020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0973cebb01da0e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40400070d0 con 0x7f4034005210 2023-10-08T01:08:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4040007250 con 0x7f4034005210 2023-10-08T01:08:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.405+0000 7f404d199700 1 -- 172.21.15.119:0/3613505027 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4040007610 con 0x7f4034005210 2023-10-08T01:08:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.406+0000 7f404f19d700 1 -- 172.21.15.119:0/3613505027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4034005210 msgr2=0x7f403400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.406+0000 7f404f19d700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4034005210 0x7f403400ba60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f40480642c0 tx=0x7f40400020c0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.406+0000 7f404f19d700 1 -- 172.21.15.119:0/3613505027 shutdown_connections 2023-10-08T01:08:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.406+0000 7f404f19d700 1 --2- 172.21.15.119:0/3613505027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4034005210 0x7f403400ba60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.406+0000 7f404f19d700 1 -- 172.21.15.119:0/3613505027 >> 172.21.15.119:0/3613505027 conn(0x7f404806f6a0 msgr2=0x7f404806ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.406+0000 7f404f19d700 1 -- 172.21.15.119:0/3613505027 shutdown_connections 2023-10-08T01:08:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.406+0000 7f404f19d700 1 -- 172.21.15.119:0/3613505027 wait complete. 2023-10-08T01:08:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.407+0000 7f404f19d700 1 Processor -- start 2023-10-08T01:08:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.407+0000 7f404f19d700 1 -- start start 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.408+0000 7f404f19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034005210 0x7f4048095310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.408+0000 7f404f19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404816c980 0x7f40480c1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.408+0000 7f404f19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40480c18c0 0x7f404815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.408+0000 7f404f19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4048095850 con 0x7f404816c980 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.408+0000 7f404f19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40480c1d40 con 0x7f40480c18c0 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.408+0000 7f404f19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404816a4e0 con 0x7f4034005210 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.408+0000 7f404d99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404816c980 0x7f40480c1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404d99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404816c980 0x7f40480c1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55730/0 (socket says 172.21.15.119:55730) 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404d99a700 1 -- 172.21.15.119:0/3756671187 learned_addr learned my addr 172.21.15.119:0/3756671187 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404e99c700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40480c18c0 0x7f404815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404e19b700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034005210 0x7f4048095310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404d99a700 1 -- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034005210 msgr2=0x7f4048095310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404d99a700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034005210 0x7f4048095310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404d99a700 1 -- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40480c18c0 msgr2=0x7f404815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404d99a700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40480c18c0 0x7f404815c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404d99a700 1 -- 172.21.15.119:0/3756671187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4048169bb0 con 0x7f404816c980 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404e99c700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40480c18c0 0x7f404815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404e19b700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034005210 0x7f4048095310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404d99a700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404816c980 0x7f40480c1380 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f403800e7b0 tx=0x7f4038009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f403e7fc700 1 -- 172.21.15.119:0/3756671187 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038018070 con 0x7f404816c980 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f403e7fc700 1 -- 172.21.15.119:0/3756671187 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038010040 con 0x7f404816c980 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4048169e90 con 0x7f404816c980 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.409+0000 7f403e7fc700 1 -- 172.21.15.119:0/3756671187 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038006a70 con 0x7f404816c980 2023-10-08T01:08:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.410+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4048169890 con 0x7f404816c980 2023-10-08T01:08:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.410+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f404804c3c0 con 0x7f404816c980 2023-10-08T01:08:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.411+0000 7f403e7fc700 1 -- 172.21.15.119:0/3756671187 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f4038007020 con 0x7f404816c980 2023-10-08T01:08:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.412+0000 7f403e7fc700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f402c067130 0x7f402c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.412+0000 7f403e7fc700 1 -- 172.21.15.119:0/3756671187 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f4038084460 con 0x7f404816c980 2023-10-08T01:08:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.412+0000 7f404e19b700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f402c067130 0x7f402c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.413+0000 7f404e19b700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f402c067130 0x7f402c0695e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4044006ea0 tx=0x7f4044008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:23.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.417+0000 7f403e7fc700 1 -- 172.21.15.119:0/3756671187 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40380511a0 con 0x7f404816c980 2023-10-08T01:08:23.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.637+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f404816a620 con 0x7f402c067130 2023-10-08T01:08:23.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.704+0000 7f403e7fc700 1 -- 172.21.15.119:0/3756671187 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f404816a620 con 0x7f402c067130 2023-10-08T01:08:23.706 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-10-08T01:08:23.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f402c067130 msgr2=0x7f402c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f402c067130 0x7f402c0695e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4044006ea0 tx=0x7f4044008040 comp rx=0 tx=0).stop 2023-10-08T01:08:23.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404816c980 msgr2=0x7f40480c1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404816c980 0x7f40480c1380 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f403800e7b0 tx=0x7f4038009040 comp rx=0 tx=0).stop 2023-10-08T01:08:23.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 shutdown_connections 2023-10-08T01:08:23.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4034005210 0x7f4048095310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f402c067130 0x7f402c0695e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40480c18c0 0x7f404815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 --2- 172.21.15.119:0/3756671187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f404816c980 0x7f40480c1380 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 >> 172.21.15.119:0/3756671187 conn(0x7f404806f6a0 msgr2=0x7f40480b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:23.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 shutdown_connections 2023-10-08T01:08:23.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.707+0000 7f404f19d700 1 -- 172.21.15.119:0/3756671187 wait complete. 2023-10-08T01:08:23.726 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-10-08T01:08:23.726 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-10-08T01:08:23.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.912+0000 7f9b48fff700 1 Processor -- start 2023-10-08T01:08:23.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.912+0000 7f9b48fff700 1 -- start start 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b48fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9b440c8490 0x7f9b440c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b48fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440ac030 con 0x7f9b44069e20 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b48fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b440ab820 con 0x7f9b44069a50 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b48fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b4406dbb0 con 0x7f9b440c8490 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b41d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9b440c8490 0x7f9b440c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b41d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9b440c8490 0x7f9b440c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55736/0 (socket says 172.21.15.119:55736) 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b41d9b700 1 -- 172.21.15.119:0/3178544526 learned_addr learned my addr 172.21.15.119:0/3178544526 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b42d9d700 1 --1- 172.21.15.119:0/3178544526 >> v1:172.21.15.119:6789/0 conn(0x7f9b44069e20 0x7f9b441569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53346/0 (socket says 172.21.15.119:53346) 2023-10-08T01:08:23.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b4159a700 1 -- 172.21.15.119:0/3178544526 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2198625845 0 0) 0x7f9b3c0013f0 con 0x7f9b44069e20 2023-10-08T01:08:23.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b4159a700 1 -- 172.21.15.119:0/3178544526 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b2c003570 con 0x7f9b44069e20 2023-10-08T01:08:23.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b41d9b700 1 -- 172.21.15.119:0/3178544526 >> v1:172.21.15.184:6789/0 conn(0x7f9b44069a50 legacy=0x7f9b4415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b41d9b700 1 -- 172.21.15.119:0/3178544526 >> v1:172.21.15.119:6789/0 conn(0x7f9b44069e20 legacy=0x7f9b441569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.914+0000 7f9b41d9b700 1 -- 172.21.15.119:0/3178544526 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b440aba70 con 0x7f9b440c8490 2023-10-08T01:08:23.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b41d9b700 1 --2- 172.21.15.119:0/3178544526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440c6ac0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800bc50 tx=0x7f9b3800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdea66c8d85e67b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:23.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b4159a700 1 -- 172.21.15.119:0/3178544526 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3800dbf0 con 0x7f9b440c8490 2023-10-08T01:08:23.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b4159a700 1 -- 172.21.15.119:0/3178544526 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3800dd50 con 0x7f9b440c8490 2023-10-08T01:08:23.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b4159a700 1 -- 172.21.15.119:0/3178544526 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b380196b0 con 0x7f9b440c8490 2023-10-08T01:08:23.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b48fff700 1 -- 172.21.15.119:0/3178544526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 msgr2=0x7f9b440c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b48fff700 1 --2- 172.21.15.119:0/3178544526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440c6ac0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9b3800bc50 tx=0x7f9b3800b4f0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b48fff700 1 -- 172.21.15.119:0/3178544526 shutdown_connections 2023-10-08T01:08:23.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b48fff700 1 --2- 172.21.15.119:0/3178544526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440c6ac0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.915+0000 7f9b48fff700 1 -- 172.21.15.119:0/3178544526 >> 172.21.15.119:0/3178544526 conn(0x7f9b440c8f90 msgr2=0x7f9b440c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:23.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.916+0000 7f9b48fff700 1 -- 172.21.15.119:0/3178544526 shutdown_connections 2023-10-08T01:08:23.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.916+0000 7f9b48fff700 1 -- 172.21.15.119:0/3178544526 wait complete. 2023-10-08T01:08:23.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.917+0000 7f9b48fff700 1 Processor -- start 2023-10-08T01:08:23.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.917+0000 7f9b48fff700 1 -- start start 2023-10-08T01:08:23.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b48fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b44069a50 0x7f9b441549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b48fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b44069e20 0x7f9b440cc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b48fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b48fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440aa940 con 0x7f9b440c8490 2023-10-08T01:08:23.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b48fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44154ef0 con 0x7f9b44069e20 2023-10-08T01:08:23.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b48fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b440c4bb0 con 0x7f9b44069a50 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b42d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b42d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55748/0 (socket says 172.21.15.119:55748) 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.918+0000 7f9b42d9d700 1 -- 172.21.15.119:0/3462227479 learned_addr learned my addr 172.21.15.119:0/3462227479 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b41d9b700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b44069e20 0x7f9b440cc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b4259c700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b44069a50 0x7f9b441549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b42d9d700 1 -- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b44069a50 msgr2=0x7f9b441549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b42d9d700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b44069a50 0x7f9b441549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b42d9d700 1 -- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b44069e20 msgr2=0x7f9b440cc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b42d9d700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b44069e20 0x7f9b440cc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:23.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b42d9d700 1 -- 172.21.15.119:0/3462227479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3800b790 con 0x7f9b440c8490 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b4259c700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b44069a50 0x7f9b441549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b42d9d700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440cca40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00cbc0 tx=0x7f9b3c0019d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b327fc700 1 -- 172.21.15.119:0/3462227479 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00a850 con 0x7f9b440c8490 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b327fc700 1 -- 172.21.15.119:0/3462227479 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b3c00a9b0 con 0x7f9b440c8490 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b440a9d30 con 0x7f9b440c8490 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b327fc700 1 -- 172.21.15.119:0/3462227479 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3c00ad50 con 0x7f9b440c8490 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.919+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b440aa2f0 con 0x7f9b440c8490 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.921+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b440718b0 con 0x7f9b440c8490 2023-10-08T01:08:23.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.921+0000 7f9b327fc700 1 -- 172.21.15.119:0/3462227479 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f9b3c012630 con 0x7f9b440c8490 2023-10-08T01:08:23.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.922+0000 7f9b327fc700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9b24067130 0x7f9b240695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:23.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.922+0000 7f9b327fc700 1 -- 172.21.15.119:0/3462227479 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f9b3c084a90 con 0x7f9b440c8490 2023-10-08T01:08:23.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.923+0000 7f9b4259c700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9b24067130 0x7f9b240695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:23.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.923+0000 7f9b4259c700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9b24067130 0x7f9b240695e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b34007fd0 tx=0x7f9b34009040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:23.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:23.926+0000 7f9b327fc700 1 -- 172.21.15.119:0/3462227479 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9b3c051980 con 0x7f9b440c8490 2023-10-08T01:08:24.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.138+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b440ac030 con 0x7f9b24067130 2023-10-08T01:08:24.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.209+0000 7f9b327fc700 1 -- 172.21.15.119:0/3462227479 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9b440ac030 con 0x7f9b24067130 2023-10-08T01:08:24.210 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-10-08T01:08:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9b24067130 msgr2=0x7f9b240695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9b24067130 0x7f9b240695e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9b34007fd0 tx=0x7f9b34009040 comp rx=0 tx=0).stop 2023-10-08T01:08:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 msgr2=0x7f9b440cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440cca40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c00cbc0 tx=0x7f9b3c0019d0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 shutdown_connections 2023-10-08T01:08:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b44069a50 0x7f9b441549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f9b24067130 0x7f9b240695e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b440c8490 0x7f9b440cca40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 --2- 172.21.15.119:0/3462227479 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b44069e20 0x7f9b440cc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 >> 172.21.15.119:0/3462227479 conn(0x7f9b440c8f90 msgr2=0x7f9b440ca490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:24.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 shutdown_connections 2023-10-08T01:08:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.211+0000 7f9b48fff700 1 -- 172.21.15.119:0/3462227479 wait complete. 2023-10-08T01:08:24.229 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-10-08T01:08:24.230 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-10-08T01:08:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.419+0000 7f4c4d1ff700 1 Processor -- start 2023-10-08T01:08:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.419+0000 7f4c4d1ff700 1 -- start start 2023-10-08T01:08:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.420+0000 7f4c4d1ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4c481563b0 0x7f4c48156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.420+0000 7f4c4d1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4815a440 0x7f4c480acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.420+0000 7f4c4d1ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4815a9b0 con 0x7f4c48068890 2023-10-08T01:08:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.420+0000 7f4c4d1ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4815ac90 con 0x7f4c481563b0 2023-10-08T01:08:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.420+0000 7f4c4d1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815af70 con 0x7f4c4815a440 2023-10-08T01:08:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c4759e700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4c48068890 0x7f4c480c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53362/0 (socket says 172.21.15.119:53362) 2023-10-08T01:08:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c4759e700 1 -- 172.21.15.119:0/2799865120 learned_addr learned my addr 172.21.15.119:0/2799865120 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c46d9d700 1 --2- 172.21.15.119:0/2799865120 >> v2:172.21.15.184:3300/0 conn(0x7f4c481563b0 0x7f4c48156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c4659c700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4815a440 0x7f4c480acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215325615 0 0) 0x7f4c4815a9b0 con 0x7f4c48068890 2023-10-08T01:08:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c30003570 con 0x7f4c48068890 2023-10-08T01:08:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3220510558 0 0) 0x7f4c30003570 con 0x7f4c48068890 2023-10-08T01:08:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4815a440 msgr2=0x7f4c480acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4815a440 0x7f4c480acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 >> v2:172.21.15.184:3300/0 conn(0x7f4c481563b0 msgr2=0x7f4c48156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 --2- 172.21.15.119:0/2799865120 >> v2:172.21.15.184:3300/0 conn(0x7f4c481563b0 0x7f4c48156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4815b250 con 0x7f4c48068890 2023-10-08T01:08:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.421+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f4c400017f0 con 0x7f4c48068890 2023-10-08T01:08:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 >> v1:172.21.15.119:6789/0 conn(0x7f4c48068890 legacy=0x7f4c480c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c45d9b700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c30003a60 0x7f4c30003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c45d9b700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4815a440 0x7f4c30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c45d9b700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815ac90 con 0x7f4c30004d50 2023-10-08T01:08:24.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4815af70 con 0x7f4c30003a60 2023-10-08T01:08:24.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3000ba60 con 0x7f4c4815a440 2023-10-08T01:08:24.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c46d9d700 1 --2- 172.21.15.119:0/2799865120 >> v2:172.21.15.184:3300/0 conn(0x7f4c481563b0 0x7f4c48156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:24.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c4759e700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c4659c700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4815a440 0x7f4c30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c46d9d700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c30003a60 0x7f4c30003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c4759e700 1 -- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4815a440 msgr2=0x7f4c30004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c4759e700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c4815a440 0x7f4c30004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c4759e700 1 -- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c30003a60 msgr2=0x7f4c30003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c4759e700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c30003a60 0x7f4c30003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c4759e700 1 -- 172.21.15.119:0/2799865120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c30003570 con 0x7f4c30004d50 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c46d9d700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c30003a60 0x7f4c30003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.422+0000 7f4c4759e700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c300051c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000a970 tx=0x7f4c40002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffd0b373be51432 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c400070d0 con 0x7f4c30004d50 2023-10-08T01:08:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c40007270 con 0x7f4c30004d50 2023-10-08T01:08:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c45d9b700 1 -- 172.21.15.119:0/2799865120 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c40007630 con 0x7f4c30004d50 2023-10-08T01:08:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/2799865120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 msgr2=0x7f4c300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c4d1ff700 1 --2- 172.21.15.119:0/2799865120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c300051c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4c4000a970 tx=0x7f4c40002210 comp rx=0 tx=0).stop 2023-10-08T01:08:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c4759e700 1 -- 172.21.15.119:0/2799865120 reap_dead start 2023-10-08T01:08:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/2799865120 shutdown_connections 2023-10-08T01:08:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/2799865120 >> 172.21.15.119:0/2799865120 conn(0x7f4c480c8f90 msgr2=0x7f4c480c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/2799865120 shutdown_connections 2023-10-08T01:08:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.423+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/2799865120 wait complete. 2023-10-08T01:08:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.425+0000 7f4c4d1ff700 1 Processor -- start 2023-10-08T01:08:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.425+0000 7f4c4d1ff700 1 -- start start 2023-10-08T01:08:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4d1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c480afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4d1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c48147840 0x7f4c480caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4d1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c48147cb0 0x7f4c480cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4d1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c480c2f60 con 0x7f4c30004d50 2023-10-08T01:08:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4d1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4816ad50 con 0x7f4c48147840 2023-10-08T01:08:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4d1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4816b030 con 0x7f4c48147cb0 2023-10-08T01:08:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c46d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c480afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c46d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c480afdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55768/0 (socket says 172.21.15.119:55768) 2023-10-08T01:08:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c46d9d700 1 -- 172.21.15.119:0/1645306489 learned_addr learned my addr 172.21.15.119:0/1645306489 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4759e700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c48147cb0 0x7f4c480cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4659c700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c48147840 0x7f4c480caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c46d9d700 1 -- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c48147cb0 msgr2=0x7f4c480cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c46d9d700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c48147cb0 0x7f4c480cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c46d9d700 1 -- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c48147840 msgr2=0x7f4c480caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c46d9d700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c48147840 0x7f4c480caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c46d9d700 1 -- 172.21.15.119:0/1645306489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480af2d0 con 0x7f4c30004d50 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.426+0000 7f4c4659c700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c48147840 0x7f4c480caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.427+0000 7f4c46d9d700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c480afdb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4c38011990 tx=0x7f4c38008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.427+0000 7f4c2effd700 1 -- 172.21.15.119:0/1645306489 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800f8e0 con 0x7f4c30004d50 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.427+0000 7f4c2effd700 1 -- 172.21.15.119:0/1645306489 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3800fa40 con 0x7f4c30004d50 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.427+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c480af5b0 con 0x7f4c30004d50 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.427+0000 7f4c2effd700 1 -- 172.21.15.119:0/1645306489 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3800fdc0 con 0x7f4c30004d50 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.427+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c480b02f0 con 0x7f4c30004d50 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.428+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c480afc00 con 0x7f4c30004d50 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.429+0000 7f4c2effd700 1 -- 172.21.15.119:0/1645306489 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f4c38015670 con 0x7f4c30004d50 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.430+0000 7f4c2effd700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4c34067130 0x7f4c340695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.430+0000 7f4c2effd700 1 -- 172.21.15.119:0/1645306489 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f4c38085400 con 0x7f4c30004d50 2023-10-08T01:08:24.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.430+0000 7f4c4659c700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4c34067130 0x7f4c340695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.430+0000 7f4c4659c700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4c34067130 0x7f4c340695e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009d50 tx=0x7f4c3c008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:24.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.433+0000 7f4c2effd700 1 -- 172.21.15.119:0/1645306489 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c38052250 con 0x7f4c30004d50 2023-10-08T01:08:24.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.652+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c480718b0 con 0x7f4c34067130 2023-10-08T01:08:24.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.718+0000 7f4c2effd700 1 -- 172.21.15.119:0/1645306489 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4c480718b0 con 0x7f4c34067130 2023-10-08T01:08:24.721 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-10-08T01:08:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4c34067130 msgr2=0x7f4c340695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4c34067130 0x7f4c340695e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009d50 tx=0x7f4c3c008040 comp rx=0 tx=0).stop 2023-10-08T01:08:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 msgr2=0x7f4c480afdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c480afdb0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4c38011990 tx=0x7f4c38008040 comp rx=0 tx=0).stop 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 shutdown_connections 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c48147cb0 0x7f4c480cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4c34067130 0x7f4c340695e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c30004d50 0x7f4c480afdb0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 --2- 172.21.15.119:0/1645306489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c48147840 0x7f4c480caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 >> 172.21.15.119:0/1645306489 conn(0x7f4c480c8f90 msgr2=0x7f4c48155630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 shutdown_connections 2023-10-08T01:08:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.721+0000 7f4c4d1ff700 1 -- 172.21.15.119:0/1645306489 wait complete. 2023-10-08T01:08:24.740 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-10-08T01:08:24.740 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-10-08T01:08:24.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.925+0000 7f17d43ff700 1 Processor -- start 2023-10-08T01:08:24.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.925+0000 7f17d43ff700 1 -- start start 2023-10-08T01:08:24.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.926+0000 7f17d43ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc1563b0 0x7f17cc156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.926+0000 7f17d43ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc068890 0x7f17cc0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.926+0000 7f17d43ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc15a9b0 con 0x7f17cc15a440 2023-10-08T01:08:24.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.926+0000 7f17d43ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc15ac90 con 0x7f17cc1563b0 2023-10-08T01:08:24.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.926+0000 7f17d43ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc15af70 con 0x7f17cc068890 2023-10-08T01:08:24.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d219b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc1563b0 0x7f17cc156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d199a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f17cc15a440 0x7f17cc0acf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56620/0 (socket says 172.21.15.119:56620) 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d219b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17cc1563b0 0x7f17cc156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49796/0 (socket says 172.21.15.119:49796) 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d199a700 1 -- 172.21.15.119:0/2364997958 learned_addr learned my addr 172.21.15.119:0/2364997958 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d299c700 1 --2- 172.21.15.119:0/2364997958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc068890 0x7f17cc0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d219b700 1 -- 172.21.15.119:0/2364997958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc068890 msgr2=0x7f17cc0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d219b700 1 --2- 172.21.15.119:0/2364997958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc068890 0x7f17cc0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d219b700 1 -- 172.21.15.119:0/2364997958 >> v1:172.21.15.119:6789/0 conn(0x7f17cc15a440 legacy=0x7f17cc0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d219b700 1 -- 172.21.15.119:0/2364997958 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc15b250 con 0x7f17cc1563b0 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d1199700 1 -- 172.21.15.119:0/2364997958 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196363120 0 0) 0x7f17cc15a9b0 con 0x7f17cc15a440 2023-10-08T01:08:24.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d299c700 1 --2- 172.21.15.119:0/2364997958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc068890 0x7f17cc0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.927+0000 7f17d219b700 1 --2- 172.21.15.119:0/2364997958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc1563b0 0x7f17cc156780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098e0 tx=0x7f17c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764ed36608f80f15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d1199700 1 -- 172.21.15.119:0/2364997958 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8009e80 con 0x7f17cc1563b0 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d1199700 1 -- 172.21.15.119:0/2364997958 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c800b860 con 0x7f17cc1563b0 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d1199700 1 -- 172.21.15.119:0/2364997958 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c800bbe0 con 0x7f17cc1563b0 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d43ff700 1 -- 172.21.15.119:0/2364997958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc1563b0 msgr2=0x7f17cc156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2364997958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc1563b0 0x7f17cc156780 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f17c80098e0 tx=0x7f17c80091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d43ff700 1 -- 172.21.15.119:0/2364997958 shutdown_connections 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2364997958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc068890 0x7f17cc0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2364997958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc1563b0 0x7f17cc156780 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d43ff700 1 -- 172.21.15.119:0/2364997958 >> 172.21.15.119:0/2364997958 conn(0x7f17cc0c8f90 msgr2=0x7f17cc0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:24.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d43ff700 1 -- 172.21.15.119:0/2364997958 shutdown_connections 2023-10-08T01:08:24.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.928+0000 7f17d43ff700 1 -- 172.21.15.119:0/2364997958 wait complete. 2023-10-08T01:08:24.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.930+0000 7f17d43ff700 1 Processor -- start 2023-10-08T01:08:24.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.930+0000 7f17d43ff700 1 -- start start 2023-10-08T01:08:24.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d43ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc068890 0x7f17cc1482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d43ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc1563b0 0x7f17cc0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d43ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc15a440 0x7f17cc0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d43ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0b1a40 con 0x7f17cc068890 2023-10-08T01:08:24.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d43ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc148830 con 0x7f17cc15a440 2023-10-08T01:08:24.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d43ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc0b0e30 con 0x7f17cc1563b0 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d219b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc068890 0x7f17cc1482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d219b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc068890 0x7f17cc1482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49798/0 (socket says 172.21.15.119:49798) 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d219b700 1 -- 172.21.15.119:0/2947374036 learned_addr learned my addr 172.21.15.119:0/2947374036 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d199a700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc1563b0 0x7f17cc0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d299c700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc15a440 0x7f17cc0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d219b700 1 -- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc1563b0 msgr2=0x7f17cc0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d219b700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc1563b0 0x7f17cc0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d219b700 1 -- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc15a440 msgr2=0x7f17cc0c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d219b700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc15a440 0x7f17cc0c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d219b700 1 -- 172.21.15.119:0/2947374036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c8009420 con 0x7f17cc068890 2023-10-08T01:08:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.931+0000 7f17d299c700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc15a440 0x7f17cc0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.932+0000 7f17d219b700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc068890 0x7f17cc1482f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0bff50 tx=0x7f17c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.932+0000 7f17c27fc700 1 -- 172.21.15.119:0/2947374036 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8011420 con 0x7f17cc068890 2023-10-08T01:08:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.932+0000 7f17c27fc700 1 -- 172.21.15.119:0/2947374036 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c80115a0 con 0x7f17cc068890 2023-10-08T01:08:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.932+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc0b1110 con 0x7f17cc068890 2023-10-08T01:08:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.932+0000 7f17c27fc700 1 -- 172.21.15.119:0/2947374036 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8011960 con 0x7f17cc068890 2023-10-08T01:08:24.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.932+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc0b16d0 con 0x7f17cc068890 2023-10-08T01:08:24.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.933+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc0718b0 con 0x7f17cc068890 2023-10-08T01:08:24.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.934+0000 7f17c27fc700 1 -- 172.21.15.119:0/2947374036 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f17c80253f0 con 0x7f17cc068890 2023-10-08T01:08:24.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.935+0000 7f17c27fc700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b0067130 0x7f17b00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:24.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.935+0000 7f17c27fc700 1 -- 172.21.15.119:0/2947374036 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f17c8086490 con 0x7f17cc068890 2023-10-08T01:08:24.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.935+0000 7f17d199a700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b0067130 0x7f17b00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.936+0000 7f17d199a700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b0067130 0x7f17b00695e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f17cc16d260 tx=0x7f17bc007f50 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:24.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:24.939+0000 7f17c27fc700 1 -- 172.21.15.119:0/2947374036 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f17c8053380 con 0x7f17cc068890 2023-10-08T01:08:25.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.157+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f17cc0b1b80 con 0x7f17b0067130 2023-10-08T01:08:25.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.221+0000 7f17c27fc700 1 -- 172.21.15.119:0/2947374036 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f17bc006d40 con 0x7f17b0067130 2023-10-08T01:08:25.223 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-10-08T01:08:25.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b0067130 msgr2=0x7f17b00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:25.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b0067130 0x7f17b00695e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f17cc16d260 tx=0x7f17bc007f50 comp rx=0 tx=0).stop 2023-10-08T01:08:25.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc068890 msgr2=0x7f17cc1482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:25.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc068890 0x7f17cc1482f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0bff50 tx=0x7f17c8017000 comp rx=0 tx=0).stop 2023-10-08T01:08:25.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 shutdown_connections 2023-10-08T01:08:25.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f17b0067130 0x7f17b00695e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17cc15a440 0x7f17cc0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17cc068890 0x7f17cc1482f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 --2- 172.21.15.119:0/2947374036 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17cc1563b0 0x7f17cc0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 >> 172.21.15.119:0/2947374036 conn(0x7f17cc0c8f90 msgr2=0x7f17cc06e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:25.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 shutdown_connections 2023-10-08T01:08:25.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.224+0000 7f17d43ff700 1 -- 172.21.15.119:0/2947374036 wait complete. 2023-10-08T01:08:25.242 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-10-08T01:08:25.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-10-08T01:08:25.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.430+0000 7f25009fe700 1 Processor -- start 2023-10-08T01:08:25.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.430+0000 7f25009fe700 1 -- start start 2023-10-08T01:08:25.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.431+0000 7f25009fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24fc0b03c0 0x7f24fc0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:25.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.431+0000 7f25009fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f24fc0b0790 0x7f24fc0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:25.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.431+0000 7f25009fe700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24fc0cfa00 con 0x7f24fc1591e0 2023-10-08T01:08:25.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.431+0000 7f25009fe700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24fc0cb580 con 0x7f24fc0b03c0 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.431+0000 7f25009fe700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24fc0b2de0 con 0x7f24fc0b0790 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.431+0000 7f24fad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24fc0b03c0 0x7f24fc0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24fc0b03c0 0x7f24fc0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49812/0 (socket says 172.21.15.119:49812) 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fad9d700 1 -- 172.21.15.119:0/2493696569 learned_addr learned my addr 172.21.15.119:0/2493696569 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fb59e700 1 --1- 172.21.15.119:0/2493696569 >> v1:172.21.15.119:6789/0 conn(0x7f24fc1591e0 0x7f24fc159600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56622/0 (socket says 172.21.15.119:56622) 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fa59c700 1 --2- 172.21.15.119:0/2493696569 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0b0790 0x7f24fc0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fad9d700 1 -- 172.21.15.119:0/2493696569 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0b0790 msgr2=0x7f24fc0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fad9d700 1 --2- 172.21.15.119:0/2493696569 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0b0790 0x7f24fc0b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fad9d700 1 -- 172.21.15.119:0/2493696569 >> v1:172.21.15.119:6789/0 conn(0x7f24fc1591e0 legacy=0x7f24fc159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fad9d700 1 -- 172.21.15.119:0/2493696569 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc0cf160 con 0x7f24fc0b03c0 2023-10-08T01:08:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24f9d9b700 1 -- 172.21.15.119:0/2493696569 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482399160 0 0) 0x7f24fc0cfa00 con 0x7f24fc1591e0 2023-10-08T01:08:25.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24fad9d700 1 --2- 172.21.15.119:0/2493696569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc0cea70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0099b0 tx=0x7f24ec009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f1dbf9af7df243 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:25.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.432+0000 7f24f9d9b700 1 -- 172.21.15.119:0/2493696569 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec009ea0 con 0x7f24fc0b03c0 2023-10-08T01:08:25.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f24f9d9b700 1 -- 172.21.15.119:0/2493696569 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec00b820 con 0x7f24fc0b03c0 2023-10-08T01:08:25.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f24f9d9b700 1 -- 172.21.15.119:0/2493696569 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00bbe0 con 0x7f24fc0b03c0 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f25009fe700 1 -- 172.21.15.119:0/2493696569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 msgr2=0x7f24fc0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f25009fe700 1 --2- 172.21.15.119:0/2493696569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc0cea70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f24ec0099b0 tx=0x7f24ec009250 comp rx=0 tx=0).stop 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f25009fe700 1 -- 172.21.15.119:0/2493696569 shutdown_connections 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f25009fe700 1 --2- 172.21.15.119:0/2493696569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc0cea70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f25009fe700 1 --2- 172.21.15.119:0/2493696569 >> v2:172.21.15.184:3300/0 conn(0x7f24fc0b0790 0x7f24fc0b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f25009fe700 1 -- 172.21.15.119:0/2493696569 >> 172.21.15.119:0/2493696569 conn(0x7f24fc063230 msgr2=0x7f24fc063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f25009fe700 1 -- 172.21.15.119:0/2493696569 shutdown_connections 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.433+0000 7f25009fe700 1 -- 172.21.15.119:0/2493696569 wait complete. 2023-10-08T01:08:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.435+0000 7f25009fe700 1 Processor -- start 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.435+0000 7f25009fe700 1 -- start start 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f25009fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f25009fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b0790 0x7f24fc0c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f25009fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc1591e0 0x7f24fc1585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f25009fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc0c8350 con 0x7f24fc0b03c0 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f25009fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc0c7740 con 0x7f24fc0b0790 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f25009fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc0c7a20 con 0x7f24fc1591e0 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc1575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49826/0 (socket says 172.21.15.119:49826) 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fad9d700 1 -- 172.21.15.119:0/973825043 learned_addr learned my addr 172.21.15.119:0/973825043 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fa59c700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b0790 0x7f24fc0c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fb59e700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc1591e0 0x7f24fc1585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fad9d700 1 -- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc1591e0 msgr2=0x7f24fc1585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fad9d700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc1591e0 0x7f24fc1585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fad9d700 1 -- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b0790 msgr2=0x7f24fc0c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fad9d700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b0790 0x7f24fc0c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fad9d700 1 -- 172.21.15.119:0/973825043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec0094f0 con 0x7f24fc0b03c0 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.436+0000 7f24fb59e700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc1591e0 0x7f24fc1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.437+0000 7f24fad9d700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc1575b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0cfe40 tx=0x7f24ec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.437+0000 7f24e2ffd700 1 -- 172.21.15.119:0/973825043 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec01f070 con 0x7f24fc0b03c0 2023-10-08T01:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.437+0000 7f24e2ffd700 1 -- 172.21.15.119:0/973825043 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec006b10 con 0x7f24fc0b03c0 2023-10-08T01:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.437+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc0c7d00 con 0x7f24fc0b03c0 2023-10-08T01:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.437+0000 7f24e2ffd700 1 -- 172.21.15.119:0/973825043 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00b990 con 0x7f24fc0b03c0 2023-10-08T01:08:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.437+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc157af0 con 0x7f24fc0b03c0 2023-10-08T01:08:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.438+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc04c3c0 con 0x7f24fc0b03c0 2023-10-08T01:08:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.439+0000 7f24e2ffd700 1 -- 172.21.15.119:0/973825043 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f24ec023070 con 0x7f24fc0b03c0 2023-10-08T01:08:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.440+0000 7f24e2ffd700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f24e8067130 0x7f24e80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.440+0000 7f24e2ffd700 1 -- 172.21.15.119:0/973825043 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f24ec085af0 con 0x7f24fc0b03c0 2023-10-08T01:08:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.440+0000 7f24fa59c700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f24e8067130 0x7f24e80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.440+0000 7f24fa59c700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f24e8067130 0x7f24e80695e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0693d0 tx=0x7f24f00091a0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.444+0000 7f24e2ffd700 1 -- 172.21.15.119:0/973825043 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24ec052a90 con 0x7f24fc0b03c0 2023-10-08T01:08:25.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.663+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f24fc0c8490 con 0x7f24e8067130 2023-10-08T01:08:25.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.736+0000 7f24e2ffd700 1 -- 172.21.15.119:0/973825043 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f24f0009ad0 con 0x7f24e8067130 2023-10-08T01:08:25.738 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-10-08T01:08:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f24e8067130 msgr2=0x7f24e80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f24e8067130 0x7f24e80695e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0693d0 tx=0x7f24f00091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 msgr2=0x7f24fc1575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc1575b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0cfe40 tx=0x7f24ec017000 comp rx=0 tx=0).stop 2023-10-08T01:08:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 shutdown_connections 2023-10-08T01:08:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f24fc1591e0 0x7f24fc1585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f24e8067130 0x7f24e80695e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24fc0b03c0 0x7f24fc1575b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 --2- 172.21.15.119:0/973825043 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f24fc0b0790 0x7f24fc0c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 >> 172.21.15.119:0/973825043 conn(0x7f24fc063230 msgr2=0x7f24fc0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 shutdown_connections 2023-10-08T01:08:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:25.739+0000 7f25009fe700 1 -- 172.21.15.119:0/973825043 wait complete. 2023-10-08T01:08:25.757 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-10-08T01:08:25.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:25.758 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-08T01:08:25.798 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:25.798 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:25.798 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:25.798 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:25.799 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:25.799 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:25.799 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:25.799 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:25.799 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:25.799 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:25.799 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:25.800 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:25.801 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_57_00_UTC', 'scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:25.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-10-08T01:08:26.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.023+0000 7fdedf39d700 1 Processor -- start 2023-10-08T01:08:26.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.024+0000 7fdedf39d700 1 -- start start 2023-10-08T01:08:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdedf39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80b0760 0x7fded80b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdedf39d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fded80ace60 0x7fded80ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdedf39d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fded80b1110 con 0x7fded8156350 2023-10-08T01:08:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdedf39d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded80b0570 con 0x7fded80ace60 2023-10-08T01:08:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdedf39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded80af4d0 con 0x7fded80b0760 2023-10-08T01:08:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdedeb9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fded80ace60 0x7fded80ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdedeb9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fded80ace60 0x7fded80ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49832/0 (socket says 172.21.15.119:49832) 2023-10-08T01:08:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdedeb9c700 1 -- 172.21.15.119:0/3836778709 learned_addr learned my addr 172.21.15.119:0/3836778709 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.025+0000 7fdeddb9a700 1 --2- 172.21.15.119:0/3836778709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80b0760 0x7fded80b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdedeb9c700 1 -- 172.21.15.119:0/3836778709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80b0760 msgr2=0x7fded80b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdedeb9c700 1 --2- 172.21.15.119:0/3836778709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80b0760 0x7fded80b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdedeb9c700 1 -- 172.21.15.119:0/3836778709 >> v1:172.21.15.184:6789/0 conn(0x7fded8156350 legacy=0x7fded8157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdedeb9c700 1 -- 172.21.15.119:0/3836778709 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded80af7b0 con 0x7fded80ace60 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdeddb9a700 1 --2- 172.21.15.119:0/3836778709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80b0760 0x7fded80b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdedeb9c700 1 --2- 172.21.15.119:0/3836778709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded80ace60 0x7fded80ad2a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fded0009f20 tx=0x7fded0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcea9ceeaa237fa7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdedd399700 1 -- 172.21.15.119:0/3836778709 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded000e070 con 0x7fded80ace60 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdedd399700 1 -- 172.21.15.119:0/3836778709 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded000ba80 con 0x7fded80ace60 2023-10-08T01:08:26.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.026+0000 7fdedd399700 1 -- 172.21.15.119:0/3836778709 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00092e0 con 0x7fded80ace60 2023-10-08T01:08:26.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.027+0000 7fdedf39d700 1 -- 172.21.15.119:0/3836778709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded80ace60 msgr2=0x7fded80ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.027+0000 7fdedf39d700 1 --2- 172.21.15.119:0/3836778709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded80ace60 0x7fded80ad2a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fded0009f20 tx=0x7fded0009840 comp rx=0 tx=0).stop 2023-10-08T01:08:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.027+0000 7fdedf39d700 1 -- 172.21.15.119:0/3836778709 shutdown_connections 2023-10-08T01:08:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.027+0000 7fdedf39d700 1 --2- 172.21.15.119:0/3836778709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded80ace60 0x7fded80ad2a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.027+0000 7fdedf39d700 1 --2- 172.21.15.119:0/3836778709 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80b0760 0x7fded80b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.027+0000 7fdedf39d700 1 -- 172.21.15.119:0/3836778709 >> 172.21.15.119:0/3836778709 conn(0x7fded8068e20 msgr2=0x7fded80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.027+0000 7fdedf39d700 1 -- 172.21.15.119:0/3836778709 shutdown_connections 2023-10-08T01:08:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.027+0000 7fdedf39d700 1 -- 172.21.15.119:0/3836778709 wait complete. 2023-10-08T01:08:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.028+0000 7fdedf39d700 1 Processor -- start 2023-10-08T01:08:26.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.028+0000 7fdedf39d700 1 -- start start 2023-10-08T01:08:26.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.029+0000 7fdedf39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80ace60 0x7fded816d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.029+0000 7fdedf39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fded80b0760 0x7fded816d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.029+0000 7fdedf39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded8156350 0x7fded80ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.029+0000 7fdedf39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded806f000 con 0x7fded8156350 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.029+0000 7fdedf39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded806e3f0 con 0x7fded80b0760 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.029+0000 7fdedf39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded806e6d0 con 0x7fded80ace60 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded8156350 0x7fded80ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded8156350 0x7fded80ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49844/0 (socket says 172.21.15.119:49844) 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 -- 172.21.15.119:0/678070926 learned_addr learned my addr 172.21.15.119:0/678070926 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 -- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80ace60 msgr2=0x7fded816d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdede39b700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80ace60 0x7fded816d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdeddb9a700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fded80b0760 0x7fded816d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80ace60 0x7fded816d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 -- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fded80b0760 msgr2=0x7fded816d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fded80b0760 0x7fded816d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 -- 172.21.15.119:0/678070926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded0009ae0 con 0x7fded8156350 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdede39b700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80ace60 0x7fded816d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdeddb9a700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fded80b0760 0x7fded816d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.030+0000 7fdedeb9c700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded8156350 0x7fded80ca990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fded000b6c0 tx=0x7fded0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.031+0000 7fdece7fc700 1 -- 172.21.15.119:0/678070926 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00094e0 con 0x7fded8156350 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.031+0000 7fdece7fc700 1 -- 172.21.15.119:0/678070926 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0025460 con 0x7fded8156350 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.031+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded806e9b0 con 0x7fded8156350 2023-10-08T01:08:26.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.031+0000 7fdece7fc700 1 -- 172.21.15.119:0/678070926 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0025820 con 0x7fded8156350 2023-10-08T01:08:26.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.031+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded806f490 con 0x7fded8156350 2023-10-08T01:08:26.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.032+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded804c3c0 con 0x7fded8156350 2023-10-08T01:08:26.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.033+0000 7fdece7fc700 1 -- 172.21.15.119:0/678070926 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fded0003040 con 0x7fded8156350 2023-10-08T01:08:26.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.033+0000 7fdece7fc700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdebc067130 0x7fdebc0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.033+0000 7fdece7fc700 1 -- 172.21.15.119:0/678070926 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fded0086870 con 0x7fded8156350 2023-10-08T01:08:26.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.034+0000 7fdede39b700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdebc067130 0x7fdebc0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.034+0000 7fdede39b700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdebc067130 0x7fdebc0695e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fded40017a0 tx=0x7fded4009040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:26.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.038+0000 7fdece7fc700 1 -- 172.21.15.119:0/678070926 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fded00535b0 con 0x7fded8156350 2023-10-08T01:08:26.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.251+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fded80c9f20 con 0x7fdebc067130 2023-10-08T01:08:26.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.260+0000 7fdece7fc700 1 -- 172.21.15.119:0/678070926 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fded80c9f20 con 0x7fdebc067130 2023-10-08T01:08:26.261 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:26.261 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:56", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdebc067130 msgr2=0x7fdebc0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdebc067130 0x7fdebc0695e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fded40017a0 tx=0x7fded4009040 comp rx=0 tx=0).stop 2023-10-08T01:08:26.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded8156350 msgr2=0x7fded80ca990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded8156350 0x7fded80ca990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fded000b6c0 tx=0x7fded0018000 comp rx=0 tx=0).stop 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 shutdown_connections 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fded80ace60 0x7fded816d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fdebc067130 0x7fdebc0695e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fded8156350 0x7fded80ca990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 --2- 172.21.15.119:0/678070926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fded80b0760 0x7fded816d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 >> 172.21.15.119:0/678070926 conn(0x7fded8068e20 msgr2=0x7fded80704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 shutdown_connections 2023-10-08T01:08:26.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.262+0000 7fdedf39d700 1 -- 172.21.15.119:0/678070926 wait complete. 2023-10-08T01:08:26.281 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:56", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:26.281 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:56:56', 'first': '2023-10-08T00:57:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:26.281 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2023-10-08T01:08:26.281 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2023-10-08T01:08:26.281 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:26.282 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-08T01:08:26.319 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_01_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_01_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:26.320 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:26.321 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_57_01_UTC', 'scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_01_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:26.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-10-08T01:08:26.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.548+0000 7efde199d700 1 Processor -- start 2023-10-08T01:08:26.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.548+0000 7efde199d700 1 -- start start 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.549+0000 7efde199d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efddc15b050 0x7efddc0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.549+0000 7efde199d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efddc0cc8d0 0x7efddc15b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.549+0000 7efde199d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efddc157a50 con 0x7efddc15b960 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.549+0000 7efde199d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc1591b0 con 0x7efddc15b050 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.549+0000 7efde199d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efddc0ca9e0 con 0x7efddc0cc8d0 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.549+0000 7efde099b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efddc15b050 0x7efddc0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.549+0000 7efde099b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efddc15b050 0x7efddc0ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49858/0 (socket says 172.21.15.119:49858) 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.549+0000 7efde099b700 1 -- 172.21.15.119:0/1965424420 learned_addr learned my addr 172.21.15.119:0/1965424420 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efddbfff700 1 --2- 172.21.15.119:0/1965424420 >> v2:172.21.15.184:3300/0 conn(0x7efddc0cc8d0 0x7efddc15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efde099b700 1 -- 172.21.15.119:0/1965424420 >> v2:172.21.15.184:3300/0 conn(0x7efddc0cc8d0 msgr2=0x7efddc15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:26.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efde099b700 1 --2- 172.21.15.119:0/1965424420 >> v2:172.21.15.184:3300/0 conn(0x7efddc0cc8d0 0x7efddc15b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efde099b700 1 -- 172.21.15.119:0/1965424420 >> v1:172.21.15.119:6789/0 conn(0x7efddc15b960 legacy=0x7efddc157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efde099b700 1 -- 172.21.15.119:0/1965424420 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc0cacc0 con 0x7efddc15b050 2023-10-08T01:08:26.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efde099b700 1 --2- 172.21.15.119:0/1965424420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc15b050 0x7efddc0ca100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efddc06fdd0 tx=0x7efdd0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd269d19ef9086ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:26.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efddb7fe700 1 -- 172.21.15.119:0/1965424420 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000b860 con 0x7efddc15b050 2023-10-08T01:08:26.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efddb7fe700 1 -- 172.21.15.119:0/1965424420 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd000b9c0 con 0x7efddc15b050 2023-10-08T01:08:26.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.550+0000 7efddb7fe700 1 -- 172.21.15.119:0/1965424420 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd000bd60 con 0x7efddc15b050 2023-10-08T01:08:26.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.551+0000 7efde199d700 1 -- 172.21.15.119:0/1965424420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc15b050 msgr2=0x7efddc0ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.551+0000 7efde199d700 1 --2- 172.21.15.119:0/1965424420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc15b050 0x7efddc0ca100 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7efddc06fdd0 tx=0x7efdd0009250 comp rx=0 tx=0).stop 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.551+0000 7efde199d700 1 -- 172.21.15.119:0/1965424420 shutdown_connections 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.551+0000 7efde199d700 1 --2- 172.21.15.119:0/1965424420 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc15b050 0x7efddc0ca100 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.551+0000 7efde199d700 1 --2- 172.21.15.119:0/1965424420 >> v2:172.21.15.184:3300/0 conn(0x7efddc0cc8d0 0x7efddc15b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.551+0000 7efde199d700 1 -- 172.21.15.119:0/1965424420 >> 172.21.15.119:0/1965424420 conn(0x7efddc068e20 msgr2=0x7efddc0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.551+0000 7efde199d700 1 -- 172.21.15.119:0/1965424420 shutdown_connections 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.551+0000 7efde199d700 1 -- 172.21.15.119:0/1965424420 wait complete. 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.552+0000 7efde199d700 1 Processor -- start 2023-10-08T01:08:26.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.553+0000 7efde199d700 1 -- start start 2023-10-08T01:08:26.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.553+0000 7efde199d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc0cc8d0 0x7efddc0c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.553+0000 7efde199d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efddc15b050 0x7efddc16a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.553+0000 7efde199d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efddc15b960 0x7efddc0b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.553+0000 7efde199d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc0c8520 con 0x7efddc0cc8d0 2023-10-08T01:08:26.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.553+0000 7efde199d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efddc16b640 con 0x7efddc15b050 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.553+0000 7efde199d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efddc16ad10 con 0x7efddc15b960 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc0cc8d0 0x7efddc0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc0cc8d0 0x7efddc0c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49874/0 (socket says 172.21.15.119:49874) 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 -- 172.21.15.119:0/3367325421 learned_addr learned my addr 172.21.15.119:0/3367325421 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efddbfff700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efddc15b050 0x7efddc16a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde119c700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efddc15b960 0x7efddc0b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 -- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efddc15b960 msgr2=0x7efddc0b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efddc15b960 0x7efddc0b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 -- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efddc15b050 msgr2=0x7efddc16a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efddc15b050 0x7efddc16a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 -- 172.21.15.119:0/3367325421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00094f0 con 0x7efddc0cc8d0 2023-10-08T01:08:26.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efddbfff700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efddc15b050 0x7efddc16a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:26.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde119c700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efddc15b960 0x7efddc0b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:26.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.554+0000 7efde099b700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc0cc8d0 0x7efddc0c7a60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006fd0 tx=0x7efdd0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:26.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.555+0000 7efdd8ff9700 1 -- 172.21.15.119:0/3367325421 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0011e20 con 0x7efddc0cc8d0 2023-10-08T01:08:26.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.555+0000 7efdd8ff9700 1 -- 172.21.15.119:0/3367325421 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd001a630 con 0x7efddc0cc8d0 2023-10-08T01:08:26.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.555+0000 7efdd8ff9700 1 -- 172.21.15.119:0/3367325421 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd001a9f0 con 0x7efddc0cc8d0 2023-10-08T01:08:26.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.555+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efddc16aff0 con 0x7efddc0cc8d0 2023-10-08T01:08:26.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.555+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efddc16a6e0 con 0x7efddc0cc8d0 2023-10-08T01:08:26.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.556+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efddc16a9c0 con 0x7efddc0cc8d0 2023-10-08T01:08:26.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.557+0000 7efdd8ff9700 1 -- 172.21.15.119:0/3367325421 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7efdd002d440 con 0x7efddc0cc8d0 2023-10-08T01:08:26.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.558+0000 7efdd8ff9700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efdc0067130 0x7efdc00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:26.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.558+0000 7efdd8ff9700 1 -- 172.21.15.119:0/3367325421 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7efdd00864e0 con 0x7efddc0cc8d0 2023-10-08T01:08:26.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.558+0000 7efddbfff700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efdc0067130 0x7efdc00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:26.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.559+0000 7efddbfff700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efdc0067130 0x7efdc00695e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efddc0ae520 tx=0x7efdc8008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:26.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.562+0000 7efdd8ff9700 1 -- 172.21.15.119:0/3367325421 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efdd00533b0 con 0x7efddc0cc8d0 2023-10-08T01:08:26.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.774+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efddc04c3c0 con 0x7efdc0067130 2023-10-08T01:08:26.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.783+0000 7efdd8ff9700 1 -- 172.21.15.119:0/3367325421 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7efddc04c3c0 con 0x7efdc0067130 2023-10-08T01:08:26.784 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:26.784 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:56", "first": "2023-10-08T00:57:01", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efdc0067130 msgr2=0x7efdc00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efdc0067130 0x7efdc00695e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efddc0ae520 tx=0x7efdc8008040 comp rx=0 tx=0).stop 2023-10-08T01:08:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc0cc8d0 msgr2=0x7efddc0c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc0cc8d0 0x7efddc0c7a60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efdd0006fd0 tx=0x7efdd0017000 comp rx=0 tx=0).stop 2023-10-08T01:08:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 shutdown_connections 2023-10-08T01:08:26.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efddc15b960 0x7efddc0b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7efdc0067130 0x7efdc00695e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efddc0cc8d0 0x7efddc0c7a60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 --2- 172.21.15.119:0/3367325421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efddc15b050 0x7efddc16a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:26.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 >> 172.21.15.119:0/3367325421 conn(0x7efddc068e20 msgr2=0x7efddc156b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:26.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 shutdown_connections 2023-10-08T01:08:26.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:26.785+0000 7efde199d700 1 -- 172.21.15.119:0/3367325421 wait complete. 2023-10-08T01:08:26.803 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:56", "first": "2023-10-08T00:57:01", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:26.803 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:56:56', 'first': '2023-10-08T00:57:01', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:26.804 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2023-10-08T01:08:26.804 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2023-10-08T01:08:26.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:26.804 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-08T01:08:26.842 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:26.842 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:26.842 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:26.842 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:26.843 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:26.843 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:26.843 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:26.843 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:26.843 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:26.843 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:26.843 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:26.843 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:26.844 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_57_00_UTC', 'scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:26.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-10-08T01:08:27.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.068+0000 7f745639d700 1 Processor -- start 2023-10-08T01:08:27.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.068+0000 7f745639d700 1 -- start start 2023-10-08T01:08:27.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.069+0000 7f745639d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500cc8d0 0x7f74501596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.069+0000 7f745639d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f74500ccd40 0x7f7450159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.069+0000 7f745639d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f745015a480 con 0x7f74500ae750 2023-10-08T01:08:27.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.069+0000 7f745639d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7450153b90 con 0x7f74500ccd40 2023-10-08T01:08:27.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.069+0000 7f745639d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74500afb40 con 0x7f74500cc8d0 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7454b9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500cc8d0 0x7f74501596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7454b9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500cc8d0 0x7f74501596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:48606/0 (socket says 172.21.15.119:48606) 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7454b9a700 1 -- 172.21.15.119:0/3713571491 learned_addr learned my addr 172.21.15.119:0/3713571491 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f745539b700 1 --1- 172.21.15.119:0/3713571491 >> v1:172.21.15.184:6789/0 conn(0x7f74500ae750 0x7f7450153480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:57644/0 (socket says 172.21.15.119:57644) 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7455b9c700 1 --2- 172.21.15.119:0/3713571491 >> v2:172.21.15.184:3300/0 conn(0x7f74500ccd40 0x7f7450159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7447fff700 1 -- 172.21.15.119:0/3713571491 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3326741884 0 0) 0x7f745015a480 con 0x7f74500ae750 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7447fff700 1 -- 172.21.15.119:0/3713571491 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f743c003570 con 0x7f74500ae750 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7454b9a700 1 -- 172.21.15.119:0/3713571491 >> v2:172.21.15.184:3300/0 conn(0x7f74500ccd40 msgr2=0x7f7450159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:27.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7454b9a700 1 --2- 172.21.15.119:0/3713571491 >> v2:172.21.15.184:3300/0 conn(0x7f74500ccd40 0x7f7450159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7454b9a700 1 -- 172.21.15.119:0/3713571491 >> v1:172.21.15.184:6789/0 conn(0x7f74500ae750 legacy=0x7f7450153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7454b9a700 1 -- 172.21.15.119:0/3713571491 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74500afe20 con 0x7f74500cc8d0 2023-10-08T01:08:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.070+0000 7f7455b9c700 1 --2- 172.21.15.119:0/3713571491 >> v2:172.21.15.184:3300/0 conn(0x7f74500ccd40 0x7f7450159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.071+0000 7f7454b9a700 1 --2- 172.21.15.119:0/3713571491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500cc8d0 0x7f74501596b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f74400099b0 tx=0x7f7440009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ee77729e0e45816 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.071+0000 7f7447fff700 1 -- 172.21.15.119:0/3713571491 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000b680 con 0x7f74500cc8d0 2023-10-08T01:08:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.071+0000 7f7447fff700 1 -- 172.21.15.119:0/3713571491 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744000b7e0 con 0x7f74500cc8d0 2023-10-08T01:08:27.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.071+0000 7f7447fff700 1 -- 172.21.15.119:0/3713571491 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000bb80 con 0x7f74500cc8d0 2023-10-08T01:08:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.071+0000 7f745639d700 1 -- 172.21.15.119:0/3713571491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500cc8d0 msgr2=0x7f74501596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.071+0000 7f745639d700 1 --2- 172.21.15.119:0/3713571491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500cc8d0 0x7f74501596b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f74400099b0 tx=0x7f7440009250 comp rx=0 tx=0).stop 2023-10-08T01:08:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.072+0000 7f745639d700 1 -- 172.21.15.119:0/3713571491 shutdown_connections 2023-10-08T01:08:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.072+0000 7f745639d700 1 --2- 172.21.15.119:0/3713571491 >> v2:172.21.15.184:3300/0 conn(0x7f74500ccd40 0x7f7450159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.072+0000 7f745639d700 1 --2- 172.21.15.119:0/3713571491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500cc8d0 0x7f74501596b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.072+0000 7f745639d700 1 -- 172.21.15.119:0/3713571491 >> 172.21.15.119:0/3713571491 conn(0x7f7450068e20 msgr2=0x7f74500cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.072+0000 7f745639d700 1 -- 172.21.15.119:0/3713571491 shutdown_connections 2023-10-08T01:08:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.072+0000 7f745639d700 1 -- 172.21.15.119:0/3713571491 wait complete. 2023-10-08T01:08:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.073+0000 7f745639d700 1 Processor -- start 2023-10-08T01:08:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.073+0000 7f745639d700 1 -- start start 2023-10-08T01:08:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f745639d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500ae750 0x7f7450155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f745639d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74500cc8d0 0x7f745016a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f745639d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74500ccd40 0x7f745016ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f745639d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7450153cd0 con 0x7f74500cc8d0 2023-10-08T01:08:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f745639d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7450153f70 con 0x7f74500ccd40 2023-10-08T01:08:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f745639d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7450154250 con 0x7f74500ae750 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f7454b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74500cc8d0 0x7f745016a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f7454b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74500cc8d0 0x7f745016a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49878/0 (socket says 172.21.15.119:49878) 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.074+0000 7f7454b9a700 1 -- 172.21.15.119:0/3521199716 learned_addr learned my addr 172.21.15.119:0/3521199716 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f7455b9c700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74500ccd40 0x7f745016ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f745539b700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500ae750 0x7f7450155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f7454b9a700 1 -- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500ae750 msgr2=0x7f7450155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f7454b9a700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500ae750 0x7f7450155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f7454b9a700 1 -- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74500ccd40 msgr2=0x7f745016ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f7454b9a700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74500ccd40 0x7f745016ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f7454b9a700 1 -- 172.21.15.119:0/3521199716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7448009a20 con 0x7f74500cc8d0 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f7455b9c700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74500ccd40 0x7f745016ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:27.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f745539b700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500ae750 0x7f7450155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f7454b9a700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74500cc8d0 0x7f745016a910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7440009290 tx=0x7f7440017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f74457fa700 1 -- 172.21.15.119:0/3521199716 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744000b950 con 0x7f74500cc8d0 2023-10-08T01:08:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f74457fa700 1 -- 172.21.15.119:0/3521199716 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f744000bab0 con 0x7f74500cc8d0 2023-10-08T01:08:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74400094f0 con 0x7f74500cc8d0 2023-10-08T01:08:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.075+0000 7f74457fa700 1 -- 172.21.15.119:0/3521199716 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f744001a680 con 0x7f74500cc8d0 2023-10-08T01:08:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.076+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7450154810 con 0x7f74500cc8d0 2023-10-08T01:08:27.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.076+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745004c3c0 con 0x7f74500cc8d0 2023-10-08T01:08:27.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.077+0000 7f74457fa700 1 -- 172.21.15.119:0/3521199716 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f744001aa30 con 0x7f74500cc8d0 2023-10-08T01:08:27.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.078+0000 7f74457fa700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7434067130 0x7f74340695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.078+0000 7f74457fa700 1 -- 172.21.15.119:0/3521199716 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f74400866d0 con 0x7f74500cc8d0 2023-10-08T01:08:27.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.078+0000 7f745539b700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7434067130 0x7f74340695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.079+0000 7f745539b700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7434067130 0x7f74340695e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f744c0018e0 tx=0x7f744c009040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:27.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.082+0000 7f74457fa700 1 -- 172.21.15.119:0/3521199716 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f74400535c0 con 0x7f74500cc8d0 2023-10-08T01:08:27.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.294+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f74500b0100 con 0x7f7434067130 2023-10-08T01:08:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.303+0000 7f74457fa700 1 -- 172.21.15.119:0/3521199716 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f74500b0100 con 0x7f7434067130 2023-10-08T01:08:27.304 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:27.304 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:57", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7434067130 msgr2=0x7f74340695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7434067130 0x7f74340695e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f744c0018e0 tx=0x7f744c009040 comp rx=0 tx=0).stop 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74500cc8d0 msgr2=0x7f745016a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74500cc8d0 0x7f745016a910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7440009290 tx=0x7f7440017000 comp rx=0 tx=0).stop 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 shutdown_connections 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f74500ae750 0x7f7450155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7434067130 0x7f74340695e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f74500ccd40 0x7f745016ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 --2- 172.21.15.119:0/3521199716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74500cc8d0 0x7f745016a910 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 >> 172.21.15.119:0/3521199716 conn(0x7f7450068e20 msgr2=0x7f74500c7590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 shutdown_connections 2023-10-08T01:08:27.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.305+0000 7f745639d700 1 -- 172.21.15.119:0/3521199716 wait complete. 2023-10-08T01:08:27.323 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:57", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:27.324 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:56:57', 'first': '2023-10-08T00:57:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:27.324 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2023-10-08T01:08:27.324 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2023-10-08T01:08:27.324 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:27.324 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-08T01:08:27.362 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:27.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:27.364 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:27.364 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_57_00_UTC', 'scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:27.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-10-08T01:08:27.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.590+0000 7f567f7ff700 1 Processor -- start 2023-10-08T01:08:27.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.590+0000 7f567f7ff700 1 -- start start 2023-10-08T01:08:27.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.591+0000 7f567f7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5678068890 0x7f56780c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f567f7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567815a9b0 con 0x7f567815a440 2023-10-08T01:08:27.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f567f7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567815ac90 con 0x7f56781563b0 2023-10-08T01:08:27.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f567f7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567815af70 con 0x7f5678068890 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f567cd9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f567815a440 0x7f56780acf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56638/0 (socket says 172.21.15.119:56638) 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f567cd9a700 1 -- 172.21.15.119:0/416297777 learned_addr learned my addr 172.21.15.119:0/416297777 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f567dd9c700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5678068890 0x7f56780c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815484231 0 0) 0x7f567815a9b0 con 0x7f567815a440 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5664003570 con 0x7f567815a440 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344459479 0 0) 0x7f567815ac90 con 0x7f56781563b0 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f567815a9b0 con 0x7f56781563b0 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1038643628 0 0) 0x7f5664003570 con 0x7f567815a440 2023-10-08T01:08:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5678068890 msgr2=0x7f56780c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f566ffff700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5678068890 0x7f56780c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.592+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 >> v1:172.21.15.184:6789/0 conn(0x7f56781563b0 legacy=0x7f5678156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567815b250 con 0x7f567815a440 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f5668001470 con 0x7f567815a440 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 >> v1:172.21.15.119:6789/0 conn(0x7f567815a440 legacy=0x7f56780acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5664003c20 0x7f5664004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5678068890 0x7f5664004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5664004d50 0x7f56640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567815af70 con 0x7f5664004d50 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566400bbf0 con 0x7f5664003c20 2023-10-08T01:08:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566400bd60 con 0x7f5678068890 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f567dd9c700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5664004d50 0x7f56640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f567cd9a700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5678068890 0x7f5664004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f567d59b700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5664003c20 0x7f5664004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f567dd9c700 1 -- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5678068890 msgr2=0x7f5664004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f567dd9c700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5678068890 0x7f5664004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f567dd9c700 1 -- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5664003c20 msgr2=0x7f5664004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f567dd9c700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5664003c20 0x7f5664004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.593+0000 7f567dd9c700 1 -- 172.21.15.119:0/416297777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664003570 con 0x7f5664004d50 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f567dd9c700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5664004d50 0x7f56640051c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f56700094d0 tx=0x7f567000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b65a6946fa188c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000ea90 con 0x7f5664004d50 2023-10-08T01:08:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567000f040 con 0x7f5664004d50 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f566ffff700 1 -- 172.21.15.119:0/416297777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567000ee20 con 0x7f5664004d50 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f567f7ff700 1 -- 172.21.15.119:0/416297777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5664004d50 msgr2=0x7f56640051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f567f7ff700 1 --2- 172.21.15.119:0/416297777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5664004d50 0x7f56640051c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f56700094d0 tx=0x7f567000b040 comp rx=0 tx=0).stop 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f567dd9c700 1 -- 172.21.15.119:0/416297777 reap_dead start 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f567f7ff700 1 -- 172.21.15.119:0/416297777 shutdown_connections 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f567f7ff700 1 -- 172.21.15.119:0/416297777 >> 172.21.15.119:0/416297777 conn(0x7f56780c8f90 msgr2=0x7f56780c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f567f7ff700 1 -- 172.21.15.119:0/416297777 shutdown_connections 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.594+0000 7f567f7ff700 1 -- 172.21.15.119:0/416297777 wait complete. 2023-10-08T01:08:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.596+0000 7f567f7ff700 1 Processor -- start 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.596+0000 7f567f7ff700 1 -- start start 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567f7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5664004d50 0x7f567816abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567f7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567816e0e0 0x7f56781515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567f7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56781550d0 0x7f5678155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567f7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56780ab020 con 0x7f567816e0e0 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567f7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567816a750 con 0x7f56781550d0 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567f7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567816e550 con 0x7f5664004d50 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567cd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567816e0e0 0x7f56781515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567cd9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567816e0e0 0x7f56781515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49894/0 (socket says 172.21.15.119:49894) 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567cd9a700 1 -- 172.21.15.119:0/129600734 learned_addr learned my addr 172.21.15.119:0/129600734 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567d59b700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5664004d50 0x7f567816abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.597+0000 7f567dd9c700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56781550d0 0x7f5678155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567cd9a700 1 -- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5664004d50 msgr2=0x7f567816abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567cd9a700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5664004d50 0x7f567816abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567cd9a700 1 -- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56781550d0 msgr2=0x7f5678155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567cd9a700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56781550d0 0x7f5678155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567cd9a700 1 -- 172.21.15.119:0/129600734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678154b10 con 0x7f567816e0e0 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567d59b700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5664004d50 0x7f567816abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567dd9c700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56781550d0 0x7f5678155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567cd9a700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567816e0e0 0x7f56781515b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5668001e30 tx=0x7f5668001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f566d7fa700 1 -- 172.21.15.119:0/129600734 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56680077d0 con 0x7f567816e0e0 2023-10-08T01:08:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f566d7fa700 1 -- 172.21.15.119:0/129600734 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566800f8d0 con 0x7f567816e0e0 2023-10-08T01:08:27.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f566d7fa700 1 -- 172.21.15.119:0/129600734 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566800fc90 con 0x7f567816e0e0 2023-10-08T01:08:27.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5678154df0 con 0x7f567816e0e0 2023-10-08T01:08:27.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.598+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5678155a80 con 0x7f567816e0e0 2023-10-08T01:08:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.599+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567816a8e0 con 0x7f567816e0e0 2023-10-08T01:08:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.600+0000 7f566d7fa700 1 -- 172.21.15.119:0/129600734 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f5668013600 con 0x7f567816e0e0 2023-10-08T01:08:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.601+0000 7f566d7fa700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f565c067120 0x7f565c0695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.601+0000 7f566d7fa700 1 -- 172.21.15.119:0/129600734 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f5668084bf0 con 0x7f567816e0e0 2023-10-08T01:08:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.601+0000 7f567d59b700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f565c067120 0x7f565c0695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.602+0000 7f567d59b700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f565c067120 0x7f565c0695d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5674000f60 tx=0x7f5674009040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.605+0000 7f566d7fa700 1 -- 172.21.15.119:0/129600734 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5668051a40 con 0x7f567816e0e0 2023-10-08T01:08:27.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.818+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56780718b0 con 0x7f565c067120 2023-10-08T01:08:27.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.827+0000 7f566d7fa700 1 -- 172.21.15.119:0/129600734 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f56780718b0 con 0x7f565c067120 2023-10-08T01:08:27.828 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:27.828 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:58", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f565c067120 msgr2=0x7f565c0695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f565c067120 0x7f565c0695d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5674000f60 tx=0x7f5674009040 comp rx=0 tx=0).stop 2023-10-08T01:08:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567816e0e0 msgr2=0x7f56781515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567816e0e0 0x7f56781515b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5668001e30 tx=0x7f5668001d20 comp rx=0 tx=0).stop 2023-10-08T01:08:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 shutdown_connections 2023-10-08T01:08:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5664004d50 0x7f567816abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f565c067120 0x7f565c0695d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56781550d0 0x7f5678155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 --2- 172.21.15.119:0/129600734 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567816e0e0 0x7f56781515b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:27.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 >> 172.21.15.119:0/129600734 conn(0x7f56780c8f90 msgr2=0x7f5678152300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:27.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 shutdown_connections 2023-10-08T01:08:27.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:27.829+0000 7f567f7ff700 1 -- 172.21.15.119:0/129600734 wait complete. 2023-10-08T01:08:27.847 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:58", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:27.848 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:56:58', 'first': '2023-10-08T00:57:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:27.848 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2023-10-08T01:08:27.848 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2023-10-08T01:08:27.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:27.848 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-08T01:08:27.886 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:27.886 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:27.887 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:27.888 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_57_00_UTC', 'scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:27.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-10-08T01:08:28.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.113+0000 7fd0d5b9d700 1 Processor -- start 2023-10-08T01:08:28.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.113+0000 7fd0d5b9d700 1 -- start start 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.114+0000 7fd0d5b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd0d00685e0 0x7fd0d015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.114+0000 7fd0d5b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd0d0154000 0x7fd0d006eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.114+0000 7fd0d5b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d00c9480 con 0x7fd0d015d3f0 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.114+0000 7fd0d5b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d015d880 con 0x7fd0d0154000 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.114+0000 7fd0d5b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d015db60 con 0x7fd0d00685e0 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.114+0000 7fd0d539c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd0d0154000 0x7fd0d006eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.114+0000 7fd0d539c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd0d0154000 0x7fd0d006eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49904/0 (socket says 172.21.15.119:49904) 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.114+0000 7fd0d539c700 1 -- 172.21.15.119:0/3587271043 learned_addr learned my addr 172.21.15.119:0/3587271043 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0d4b9b700 1 --2- 172.21.15.119:0/3587271043 >> v2:172.21.15.184:3300/0 conn(0x7fd0d00685e0 0x7fd0d015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0d539c700 1 -- 172.21.15.119:0/3587271043 >> v2:172.21.15.184:3300/0 conn(0x7fd0d00685e0 msgr2=0x7fd0d015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:28.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0d539c700 1 --2- 172.21.15.119:0/3587271043 >> v2:172.21.15.184:3300/0 conn(0x7fd0d00685e0 0x7fd0d015a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0d539c700 1 -- 172.21.15.119:0/3587271043 >> v1:172.21.15.184:6789/0 conn(0x7fd0d015d3f0 legacy=0x7fd0d01538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:28.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0d539c700 1 -- 172.21.15.119:0/3587271043 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d015de40 con 0x7fd0d0154000 2023-10-08T01:08:28.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0d539c700 1 --2- 172.21.15.119:0/3587271043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d0154000 0x7fd0d006eb00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0065920 tx=0x7fd0c8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5fa69e45ebc1ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:28.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0cf7fe700 1 -- 172.21.15.119:0/3587271043 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c800e070 con 0x7fd0d0154000 2023-10-08T01:08:28.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0cf7fe700 1 -- 172.21.15.119:0/3587271043 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c800bae0 con 0x7fd0d0154000 2023-10-08T01:08:28.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.115+0000 7fd0cf7fe700 1 -- 172.21.15.119:0/3587271043 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80092e0 con 0x7fd0d0154000 2023-10-08T01:08:28.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.116+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/3587271043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d0154000 msgr2=0x7fd0d006eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.116+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/3587271043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d0154000 0x7fd0d006eb00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0065920 tx=0x7fd0c8009840 comp rx=0 tx=0).stop 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.116+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/3587271043 shutdown_connections 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.116+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/3587271043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d0154000 0x7fd0d006eb00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.116+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/3587271043 >> v2:172.21.15.184:3300/0 conn(0x7fd0d00685e0 0x7fd0d015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.116+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/3587271043 >> 172.21.15.119:0/3587271043 conn(0x7fd0d006b420 msgr2=0x7fd0d00cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.116+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/3587271043 shutdown_connections 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.116+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/3587271043 wait complete. 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.117+0000 7fd0d5b9d700 1 Processor -- start 2023-10-08T01:08:28.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.118+0000 7fd0d5b9d700 1 -- start start 2023-10-08T01:08:28.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.118+0000 7fd0d5b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d00685e0 0x7fd0d00c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.118+0000 7fd0d5b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0154000 0x7fd0d016dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.118+0000 7fd0d5b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d015d3f0 0x7fd0d0145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.118+0000 7fd0d5b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d0146490 con 0x7fd0d015d3f0 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.118+0000 7fd0d5b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d016d750 con 0x7fd0d0154000 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.118+0000 7fd0d5b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d00b1940 con 0x7fd0d00685e0 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d015d3f0 0x7fd0d0145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d015d3f0 0x7fd0d0145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49912/0 (socket says 172.21.15.119:49912) 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 -- 172.21.15.119:0/2056624992 learned_addr learned my addr 172.21.15.119:0/2056624992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0cffff700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0154000 0x7fd0d016dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d4b9b700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d00685e0 0x7fd0d00c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 -- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d00685e0 msgr2=0x7fd0d00c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d00685e0 0x7fd0d00c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 -- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0154000 msgr2=0x7fd0d016dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0154000 0x7fd0d016dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 -- 172.21.15.119:0/2056624992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c8009ae0 con 0x7fd0d015d3f0 2023-10-08T01:08:28.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0cffff700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0154000 0x7fd0d016dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:28.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d4b9b700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d00685e0 0x7fd0d00c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:28.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.119+0000 7fd0d539c700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d015d3f0 0x7fd0d0145f50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0070fe0 tx=0x7fd0c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:28.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.120+0000 7fd0ccff9700 1 -- 172.21.15.119:0/2056624992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80032f0 con 0x7fd0d015d3f0 2023-10-08T01:08:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.120+0000 7fd0ccff9700 1 -- 172.21.15.119:0/2056624992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c8008e80 con 0x7fd0d015d3f0 2023-10-08T01:08:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.120+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d00b1be0 con 0x7fd0d015d3f0 2023-10-08T01:08:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.120+0000 7fd0ccff9700 1 -- 172.21.15.119:0/2056624992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8026690 con 0x7fd0d015d3f0 2023-10-08T01:08:28.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.120+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d016d3e0 con 0x7fd0d015d3f0 2023-10-08T01:08:28.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.121+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d004c3c0 con 0x7fd0d015d3f0 2023-10-08T01:08:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.123+0000 7fd0ccff9700 1 -- 172.21.15.119:0/2056624992 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fd0c8003040 con 0x7fd0d015d3f0 2023-10-08T01:08:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.124+0000 7fd0ccff9700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd0b4067130 0x7fd0b40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.124+0000 7fd0ccff9700 1 -- 172.21.15.119:0/2056624992 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fd0c8087080 con 0x7fd0d015d3f0 2023-10-08T01:08:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.124+0000 7fd0d4b9b700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd0b4067130 0x7fd0b40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.125+0000 7fd0d4b9b700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd0b4067130 0x7fd0b40695e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006fd0 tx=0x7fd0c4008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:28.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.127+0000 7fd0ccff9700 1 -- 172.21.15.119:0/2056624992 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0c8053dc0 con 0x7fd0d015d3f0 2023-10-08T01:08:28.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.341+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0d015e0f0 con 0x7fd0b4067130 2023-10-08T01:08:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.349+0000 7fd0ccff9700 1 -- 172.21.15.119:0/2056624992 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fd0d015e0f0 con 0x7fd0b4067130 2023-10-08T01:08:28.350 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:28.350 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:58", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:28.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd0b4067130 msgr2=0x7fd0b40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd0b4067130 0x7fd0b40695e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4006fd0 tx=0x7fd0c4008040 comp rx=0 tx=0).stop 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d015d3f0 msgr2=0x7fd0d0145f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d015d3f0 0x7fd0d0145f50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0070fe0 tx=0x7fd0c8018000 comp rx=0 tx=0).stop 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 shutdown_connections 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0d00685e0 0x7fd0d00c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fd0b4067130 0x7fd0b40695e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0d015d3f0 0x7fd0d0145f50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 --2- 172.21.15.119:0/2056624992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0d0154000 0x7fd0d016dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 >> 172.21.15.119:0/2056624992 conn(0x7fd0d006b420 msgr2=0x7fd0d015c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:28.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 shutdown_connections 2023-10-08T01:08:28.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.351+0000 7fd0d5b9d700 1 -- 172.21.15.119:0/2056624992 wait complete. 2023-10-08T01:08:28.370 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:58", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:28.371 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:56:58', 'first': '2023-10-08T00:57:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:28.371 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2023-10-08T01:08:28.371 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2023-10-08T01:08:28.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:28.371 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-08T01:08:28.409 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:28.409 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:28.410 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:28.411 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_57_00_UTC', 'scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:28.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-10-08T01:08:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.635+0000 7fe68999d700 1 Processor -- start 2023-10-08T01:08:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.635+0000 7fe68999d700 1 -- start start 2023-10-08T01:08:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.636+0000 7fe68999d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe684068aa0 0x7fe6840b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.636+0000 7fe68999d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6840b0210 con 0x7fe684157870 2023-10-08T01:08:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.636+0000 7fe68999d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68406c180 con 0x7fe6841574a0 2023-10-08T01:08:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.636+0000 7fe68999d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68406c460 con 0x7fe684068aa0 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.636+0000 7fe683fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe684157870 0x7fe6840b06a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56644/0 (socket says 172.21.15.119:56644) 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.636+0000 7fe683fff700 1 -- 172.21.15.119:0/746436365 learned_addr learned my addr 172.21.15.119:0/746436365 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe68919c700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe684068aa0 0x7fe6840b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241373400 0 0) 0x7fe6840b0210 con 0x7fe684157870 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe674003570 con 0x7fe684157870 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251748671 0 0) 0x7fe68406c180 con 0x7fe6841574a0 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6840b0210 con 0x7fe6841574a0 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1176948122 0 0) 0x7fe674003570 con 0x7fe684157870 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe684068aa0 msgr2=0x7fe6840b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe684068aa0 0x7fe6840b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 >> v1:172.21.15.184:6789/0 conn(0x7fe6841574a0 legacy=0x7fe684068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68406c740 con 0x7fe684157870 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fe6700015f0 con 0x7fe684157870 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 >> v1:172.21.15.119:6789/0 conn(0x7fe684157870 legacy=0x7fe6840b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674003c20 0x7fe674004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe674004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe684068aa0 0x7fe6740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68406c460 con 0x7fe674004810 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400bb40 con 0x7fe674003c20 2023-10-08T01:08:28.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.637+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400bd00 con 0x7fe684068aa0 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe683fff700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe674004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe68899b700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674003c20 0x7fe674004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe68919c700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe684068aa0 0x7fe6740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe683fff700 1 -- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe684068aa0 msgr2=0x7fe6740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe683fff700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe684068aa0 0x7fe6740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe683fff700 1 -- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674003c20 msgr2=0x7fe674004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe683fff700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674003c20 0x7fe674004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe683fff700 1 -- 172.21.15.119:0/746436365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674003570 con 0x7fe674004810 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe68899b700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe674003c20 0x7fe674004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe683fff700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe674004c80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c2a0 tx=0x7fe6700078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c49f80a677347a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67000cbd0 con 0x7fe674004810 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67000cd30 con 0x7fe674004810 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.638+0000 7fe6837fe700 1 -- 172.21.15.119:0/746436365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67000cea0 con 0x7fe674004810 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.639+0000 7fe68999d700 1 -- 172.21.15.119:0/746436365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 msgr2=0x7fe674004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.639+0000 7fe68999d700 1 --2- 172.21.15.119:0/746436365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe674004c80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe67000c2a0 tx=0x7fe6700078a0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.639+0000 7fe68919c700 1 -- 172.21.15.119:0/746436365 reap_dead start 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.639+0000 7fe68999d700 1 -- 172.21.15.119:0/746436365 shutdown_connections 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.639+0000 7fe68999d700 1 -- 172.21.15.119:0/746436365 >> 172.21.15.119:0/746436365 conn(0x7fe68406b420 msgr2=0x7fe6840cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.639+0000 7fe68999d700 1 -- 172.21.15.119:0/746436365 shutdown_connections 2023-10-08T01:08:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.639+0000 7fe68999d700 1 -- 172.21.15.119:0/746436365 wait complete. 2023-10-08T01:08:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.641+0000 7fe68999d700 1 Processor -- start 2023-10-08T01:08:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.641+0000 7fe68999d700 1 -- start start 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.641+0000 7fe68999d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe6840c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68999d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe684167500 0x7fe684152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68999d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68415c1f0 0x7fe684159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68999d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe684152560 con 0x7fe674004810 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68999d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840c44c0 con 0x7fe684167500 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68999d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6840c4a80 con 0x7fe68415c1f0 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe6840c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe6840c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49922/0 (socket says 172.21.15.119:49922) 2023-10-08T01:08:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 -- 172.21.15.119:0/3773900203 learned_addr learned my addr 172.21.15.119:0/3773900203 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe683fff700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe684167500 0x7fe684152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68919c700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68415c1f0 0x7fe684159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 -- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68415c1f0 msgr2=0x7fe684159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68415c1f0 0x7fe684159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 -- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe684167500 msgr2=0x7fe684152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe684167500 0x7fe684152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 -- 172.21.15.119:0/3773900203 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68415c690 con 0x7fe674004810 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe683fff700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe684167500 0x7fe684152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68919c700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68415c1f0 0x7fe684159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.642+0000 7fe68899b700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe6840c6f00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe67800fc00 tx=0x7fe678007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:28.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.643+0000 7fe680ff9700 1 -- 172.21.15.119:0/3773900203 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800d8c0 con 0x7fe674004810 2023-10-08T01:08:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.643+0000 7fe680ff9700 1 -- 172.21.15.119:0/3773900203 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67800da20 con 0x7fe674004810 2023-10-08T01:08:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.643+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68415c940 con 0x7fe674004810 2023-10-08T01:08:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.643+0000 7fe680ff9700 1 -- 172.21.15.119:0/3773900203 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67800dda0 con 0x7fe674004810 2023-10-08T01:08:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.643+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6840c4710 con 0x7fe674004810 2023-10-08T01:08:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.644+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68404c3c0 con 0x7fe674004810 2023-10-08T01:08:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.645+0000 7fe680ff9700 1 -- 172.21.15.119:0/3773900203 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fe678006a30 con 0x7fe674004810 2023-10-08T01:08:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.645+0000 7fe680ff9700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe668067130 0x7fe6680695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.646+0000 7fe680ff9700 1 -- 172.21.15.119:0/3773900203 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fe6780849d0 con 0x7fe674004810 2023-10-08T01:08:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.646+0000 7fe683fff700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe668067130 0x7fe6680695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.646+0000 7fe683fff700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe668067130 0x7fe6680695e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe6840b2710 tx=0x7fe670015000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.650+0000 7fe680ff9700 1 -- 172.21.15.119:0/3773900203 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6780518c0 con 0x7fe674004810 2023-10-08T01:08:28.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.863+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe68406ca20 con 0x7fe668067130 2023-10-08T01:08:28.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.871+0000 7fe680ff9700 1 -- 172.21.15.119:0/3773900203 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fe68406ca20 con 0x7fe668067130 2023-10-08T01:08:28.872 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:28.872 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:59", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:28.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe668067130 msgr2=0x7fe6680695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe668067130 0x7fe6680695e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe6840b2710 tx=0x7fe670015000 comp rx=0 tx=0).stop 2023-10-08T01:08:28.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 msgr2=0x7fe6840c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:28.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe6840c6f00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe67800fc00 tx=0x7fe678007d30 comp rx=0 tx=0).stop 2023-10-08T01:08:28.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 shutdown_connections 2023-10-08T01:08:28.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68415c1f0 0x7fe684159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe668067130 0x7fe6680695e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe674004810 0x7fe6840c6f00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 --2- 172.21.15.119:0/3773900203 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe684167500 0x7fe684152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:28.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 >> 172.21.15.119:0/3773900203 conn(0x7fe68406b420 msgr2=0x7fe684153e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:28.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 shutdown_connections 2023-10-08T01:08:28.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:28.873+0000 7fe68999d700 1 -- 172.21.15.119:0/3773900203 wait complete. 2023-10-08T01:08:28.891 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:59", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:28.891 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:56:59', 'first': '2023-10-08T00:57:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:28.892 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2023-10-08T01:08:28.892 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2023-10-08T01:08:28.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:28.892 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-08T01:08:28.929 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:28.930 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:28.931 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:28.931 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_57_00_UTC', 'scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:28.932 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-10-08T01:08:29.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.155+0000 7f6137fff700 1 Processor -- start 2023-10-08T01:08:29.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.155+0000 7f6137fff700 1 -- start start 2023-10-08T01:08:29.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.156+0000 7f6137fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61300c8490 0x7f61300c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6137fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61300ac030 con 0x7f6130069a50 2023-10-08T01:08:29.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6137fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61300ab820 con 0x7f6130069e20 2023-10-08T01:08:29.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6137fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f613006dbb0 con 0x7f61300c8490 2023-10-08T01:08:29.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6135d9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6130069a50 0x7f613015a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56660/0 (socket says 172.21.15.119:56660) 2023-10-08T01:08:29.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6135d9b700 1 -- 172.21.15.119:0/653025198 learned_addr learned my addr 172.21.15.119:0/653025198 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:29.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639615161 0 0) 0x7f612c001110 con 0x7f6130069a50 2023-10-08T01:08:29.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f611c003570 con 0x7f6130069a50 2023-10-08T01:08:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f613559a700 1 --2- 172.21.15.119:0/653025198 >> v2:172.21.15.184:3300/0 conn(0x7f61300c8490 0x7f61300c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1089614898 0 0) 0x7f611c003570 con 0x7f6130069a50 2023-10-08T01:08:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 >> v2:172.21.15.184:3300/0 conn(0x7f61300c8490 msgr2=0x7f61300c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6134d99700 1 --2- 172.21.15.119:0/653025198 >> v2:172.21.15.184:3300/0 conn(0x7f61300c8490 0x7f61300c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 >> v1:172.21.15.184:6789/0 conn(0x7f6130069e20 legacy=0x7f61301569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61300aba70 con 0x7f6130069a50 2023-10-08T01:08:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.157+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f612c001600 con 0x7f6130069a50 2023-10-08T01:08:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 >> v1:172.21.15.119:6789/0 conn(0x7f6130069a50 legacy=0x7f613015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f6134d99700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f611c003a60 0x7f611c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f6134d99700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c004810 0x7f611c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f6134d99700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f611c0052e0 0x7f611c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613006dbb0 con 0x7f611c004810 2023-10-08T01:08:29.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f611c00c230 con 0x7f611c0052e0 2023-10-08T01:08:29.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f611c00c3f0 con 0x7f611c003a60 2023-10-08T01:08:29.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613559a700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c004810 0x7f611c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613659c700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f611c0052e0 0x7f611c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f6135d9b700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f611c003a60 0x7f611c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613559a700 1 -- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f611c003a60 msgr2=0x7f611c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:29.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613559a700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f611c003a60 0x7f611c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613559a700 1 -- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f611c0052e0 msgr2=0x7f611c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:29.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613559a700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f611c0052e0 0x7f611c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613559a700 1 -- 172.21.15.119:0/653025198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c003570 con 0x7f611c004810 2023-10-08T01:08:29.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613659c700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f611c0052e0 0x7f611c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:29.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613659c700 1 -- 172.21.15.119:0/653025198 reap_dead start 2023-10-08T01:08:29.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.158+0000 7f613559a700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c004810 0x7f611c004c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f612000bb90 tx=0x7f6120008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f298d02e52d0623 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:29.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612000d5d0 con 0x7f611c004810 2023-10-08T01:08:29.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612000d730 con 0x7f611c004810 2023-10-08T01:08:29.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6134d99700 1 -- 172.21.15.119:0/653025198 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612000daf0 con 0x7f611c004810 2023-10-08T01:08:29.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6137fff700 1 -- 172.21.15.119:0/653025198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c004810 msgr2=0x7f611c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6137fff700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c004810 0x7f611c004c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f612000bb90 tx=0x7f6120008040 comp rx=0 tx=0).stop 2023-10-08T01:08:29.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6137fff700 1 -- 172.21.15.119:0/653025198 shutdown_connections 2023-10-08T01:08:29.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6137fff700 1 --2- 172.21.15.119:0/653025198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f611c004810 0x7f611c004c80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6137fff700 1 -- 172.21.15.119:0/653025198 >> 172.21.15.119:0/653025198 conn(0x7f61300c8f90 msgr2=0x7f61300c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:29.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6137fff700 1 -- 172.21.15.119:0/653025198 shutdown_connections 2023-10-08T01:08:29.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.159+0000 7f6137fff700 1 -- 172.21.15.119:0/653025198 wait complete. 2023-10-08T01:08:29.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.161+0000 7f6137fff700 1 Processor -- start 2023-10-08T01:08:29.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.161+0000 7f6137fff700 1 -- start start 2023-10-08T01:08:29.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f6137fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f611c004810 0x7f61300c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f6137fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300ac680 0x7f6130152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f6137fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f613014a000 0x7f61300c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f6137fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61300c43e0 con 0x7f613014a000 2023-10-08T01:08:29.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f6137fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613014aaa0 con 0x7f61300ac680 2023-10-08T01:08:29.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f6137fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61300c40b0 con 0x7f611c004810 2023-10-08T01:08:29.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f613014a000 0x7f61300c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f613014a000 0x7f61300c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49926/0 (socket says 172.21.15.119:49926) 2023-10-08T01:08:29.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613659c700 1 -- 172.21.15.119:0/1660049379 learned_addr learned my addr 172.21.15.119:0/1660049379 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:29.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613659c700 1 -- 172.21.15.119:0/1660049379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f611c004810 msgr2=0x7f61300c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613659c700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f611c004810 0x7f61300c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613659c700 1 -- 172.21.15.119:0/1660049379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300ac680 msgr2=0x7f6130152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613559a700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300ac680 0x7f6130152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613659c700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300ac680 0x7f6130152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.162+0000 7f613659c700 1 -- 172.21.15.119:0/1660049379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61300acc40 con 0x7f613014a000 2023-10-08T01:08:29.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.163+0000 7f613659c700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f613014a000 0x7f61300c6f40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f61300c9fd0 tx=0x7f6128009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:29.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.163+0000 7f6125ffb700 1 -- 172.21.15.119:0/1660049379 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128013630 con 0x7f613014a000 2023-10-08T01:08:29.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.163+0000 7f6125ffb700 1 -- 172.21.15.119:0/1660049379 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6128013790 con 0x7f613014a000 2023-10-08T01:08:29.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.163+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61300acf20 con 0x7f613014a000 2023-10-08T01:08:29.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.163+0000 7f6125ffb700 1 -- 172.21.15.119:0/1660049379 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128013b50 con 0x7f613014a000 2023-10-08T01:08:29.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.163+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613014a470 con 0x7f613014a000 2023-10-08T01:08:29.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.164+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61300ad440 con 0x7f613014a000 2023-10-08T01:08:29.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.165+0000 7f6125ffb700 1 -- 172.21.15.119:0/1660049379 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f6128003030 con 0x7f613014a000 2023-10-08T01:08:29.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.166+0000 7f6125ffb700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6114067130 0x7f61140695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.166+0000 7f6125ffb700 1 -- 172.21.15.119:0/1660049379 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f61280847b0 con 0x7f613014a000 2023-10-08T01:08:29.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.166+0000 7f6135d9b700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6114067130 0x7f61140695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.167+0000 7f6135d9b700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6114067130 0x7f61140695e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f612c005c30 tx=0x7f613006dcf0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:29.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.170+0000 7f6125ffb700 1 -- 172.21.15.119:0/1660049379 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61300ad440 con 0x7f613014a000 2023-10-08T01:08:29.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.385+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f61300c4240 con 0x7f6114067130 2023-10-08T01:08:29.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.393+0000 7f6125ffb700 1 -- 172.21.15.119:0/1660049379 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f612c00e710 con 0x7f6114067130 2023-10-08T01:08:29.394 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:29.394 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:59", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.395+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6114067130 msgr2=0x7f61140695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.395+0000 7f6137fff700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6114067130 0x7f61140695e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f612c005c30 tx=0x7f613006dcf0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.395+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f613014a000 msgr2=0x7f61300c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.395+0000 7f6137fff700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f613014a000 0x7f61300c6f40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f61300c9fd0 tx=0x7f6128009040 comp rx=0 tx=0).stop 2023-10-08T01:08:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.396+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 shutdown_connections 2023-10-08T01:08:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.396+0000 7f6137fff700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f611c004810 0x7f61300c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.396+0000 7f6137fff700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6114067130 0x7f61140695e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.396+0000 7f6137fff700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f613014a000 0x7f61300c6f40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.396+0000 7f6137fff700 1 --2- 172.21.15.119:0/1660049379 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61300ac680 0x7f6130152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.396+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 >> 172.21.15.119:0/1660049379 conn(0x7f61300c8f90 msgr2=0x7f613006e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.396+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 shutdown_connections 2023-10-08T01:08:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.396+0000 7f6137fff700 1 -- 172.21.15.119:0/1660049379 wait complete. 2023-10-08T01:08:29.415 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:56:59", "first": "2023-10-08T00:57:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 12, "pruned_count": 0, "active": false}] 2023-10-08T01:08:29.415 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:56:59', 'first': '2023-10-08T00:57:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 12, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:29.416 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 12 2023-10-08T01:08:29.416 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 12 2023-10-08T01:08:29.416 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:29.416 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-08T01:08:29.454 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:29.454 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:29.454 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:29.455 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:29.455 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:29.455 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:29.455 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:29.455 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:29.455 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:29.455 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:29.456 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:29.457 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:29.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-10-08T01:08:29.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.680+0000 7f095a59d700 1 Processor -- start 2023-10-08T01:08:29.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.680+0000 7f095a59d700 1 -- start start 2023-10-08T01:08:29.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.681+0000 7f095a59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09540cb560 0x7f09540cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.681+0000 7f095a59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cc970 0x7f09540697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.681+0000 7f095a59d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09540ccde0 con 0x7f0954069cf0 2023-10-08T01:08:29.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.681+0000 7f095a59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095406aa30 con 0x7f09540cb560 2023-10-08T01:08:29.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.681+0000 7f095a59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0954159db0 con 0x7f09540cc970 2023-10-08T01:08:29.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.681+0000 7f095959b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09540cb560 0x7f09540cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f0959d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0954069cf0 0x7f095406d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56672/0 (socket says 172.21.15.119:56672) 2023-10-08T01:08:29.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f0959d9c700 1 -- 172.21.15.119:0/312948904 learned_addr learned my addr 172.21.15.119:0/312948904 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:29.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f094bfff700 1 -- 172.21.15.119:0/312948904 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 739775836 0 0) 0x7f09540ccde0 con 0x7f0954069cf0 2023-10-08T01:08:29.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f0958d9a700 1 --2- 172.21.15.119:0/312948904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cc970 0x7f09540697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f094bfff700 1 -- 172.21.15.119:0/312948904 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0940003570 con 0x7f0954069cf0 2023-10-08T01:08:29.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f095959b700 1 -- 172.21.15.119:0/312948904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cc970 msgr2=0x7f09540697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:29.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f095959b700 1 --2- 172.21.15.119:0/312948904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cc970 0x7f09540697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f095959b700 1 -- 172.21.15.119:0/312948904 >> v1:172.21.15.119:6789/0 conn(0x7f0954069cf0 legacy=0x7f095406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f095959b700 1 -- 172.21.15.119:0/312948904 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095406a470 con 0x7f09540cb560 2023-10-08T01:08:29.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.682+0000 7f095959b700 1 --2- 172.21.15.119:0/312948904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cb560 0x7f09540cc430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f09500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2557e4b2c8b97e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f094bfff700 1 -- 172.21.15.119:0/312948904 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000b640 con 0x7f09540cb560 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f094bfff700 1 -- 172.21.15.119:0/312948904 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095000b7a0 con 0x7f09540cb560 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f094bfff700 1 -- 172.21.15.119:0/312948904 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000bb40 con 0x7f09540cb560 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f095a59d700 1 -- 172.21.15.119:0/312948904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cb560 msgr2=0x7f09540cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f095a59d700 1 --2- 172.21.15.119:0/312948904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cb560 0x7f09540cc430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f09500098e0 tx=0x7f09500091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f095a59d700 1 -- 172.21.15.119:0/312948904 shutdown_connections 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f095a59d700 1 --2- 172.21.15.119:0/312948904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cb560 0x7f09540cc430 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f095a59d700 1 --2- 172.21.15.119:0/312948904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cc970 0x7f09540697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f095a59d700 1 -- 172.21.15.119:0/312948904 >> 172.21.15.119:0/312948904 conn(0x7f095406baf0 msgr2=0x7f095406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f095a59d700 1 -- 172.21.15.119:0/312948904 shutdown_connections 2023-10-08T01:08:29.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.683+0000 7f095a59d700 1 -- 172.21.15.119:0/312948904 wait complete. 2023-10-08T01:08:29.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.685+0000 7f095a59d700 1 Processor -- start 2023-10-08T01:08:29.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.685+0000 7f095a59d700 1 -- start start 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f095a59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0954069cf0 0x7f09540c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f095a59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cb560 0x7f09540c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f095a59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cc970 0x7f0954156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f095a59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540c7af0 con 0x7f09540cc970 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f095a59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09540c65b0 con 0x7f0954069cf0 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f095a59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09540c7570 con 0x7f09540cb560 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f0959d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cc970 0x7f0954156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f0959d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cc970 0x7f0954156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49944/0 (socket says 172.21.15.119:49944) 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f0959d9c700 1 -- 172.21.15.119:0/1844748619 learned_addr learned my addr 172.21.15.119:0/1844748619 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:29.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f0958d9a700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cb560 0x7f09540c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.686+0000 7f095959b700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0954069cf0 0x7f09540c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f0959d9c700 1 -- 172.21.15.119:0/1844748619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cb560 msgr2=0x7f09540c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f0959d9c700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cb560 0x7f09540c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f0959d9c700 1 -- 172.21.15.119:0/1844748619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0954069cf0 msgr2=0x7f09540c7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f0959d9c700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0954069cf0 0x7f09540c7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f0959d9c700 1 -- 172.21.15.119:0/1844748619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950009420 con 0x7f09540cc970 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f0959d9c700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cc970 0x7f0954156380 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f094c001460 tx=0x7f094c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f09497fa700 1 -- 172.21.15.119:0/1844748619 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00a890 con 0x7f09540cc970 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f09497fa700 1 -- 172.21.15.119:0/1844748619 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c00a9f0 con 0x7f09540cc970 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09540c83a0 con 0x7f09540cc970 2023-10-08T01:08:29.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f09497fa700 1 -- 172.21.15.119:0/1844748619 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00ad70 con 0x7f09540cc970 2023-10-08T01:08:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.687+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09540c6240 con 0x7f09540cc970 2023-10-08T01:08:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.688+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09540c6720 con 0x7f09540cc970 2023-10-08T01:08:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.689+0000 7f09497fa700 1 -- 172.21.15.119:0/1844748619 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f094c007300 con 0x7f09540cc970 2023-10-08T01:08:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.690+0000 7f09497fa700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0938067130 0x7f09380695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.690+0000 7f09497fa700 1 -- 172.21.15.119:0/1844748619 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f094c084d40 con 0x7f09540cc970 2023-10-08T01:08:29.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.690+0000 7f095959b700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0938067130 0x7f09380695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:29.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.691+0000 7f095959b700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0938067130 0x7f09380695e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f09500093f0 tx=0x7f0950017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:29.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.694+0000 7f09497fa700 1 -- 172.21.15.119:0/1844748619 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f094c051c10 con 0x7f09540cc970 2023-10-08T01:08:29.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.909+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f095404c3c0 con 0x7f0938067130 2023-10-08T01:08:29.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.917+0000 7f09497fa700 1 -- 172.21.15.119:0/1844748619 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f095404c3c0 con 0x7f0938067130 2023-10-08T01:08:29.919 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:29.919 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:57:00", "first": "2023-10-08T00:58:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T01:08:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0938067130 msgr2=0x7f09380695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0938067130 0x7f09380695e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f09500093f0 tx=0x7f0950017000 comp rx=0 tx=0).stop 2023-10-08T01:08:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cc970 msgr2=0x7f0954156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cc970 0x7f0954156380 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f094c001460 tx=0x7f094c001ae0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 shutdown_connections 2023-10-08T01:08:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f0938067130 0x7f09380695e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09540cc970 0x7f0954156380 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0954069cf0 0x7f09540c7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 --2- 172.21.15.119:0/1844748619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09540cb560 0x7f09540c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:29.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 >> 172.21.15.119:0/1844748619 conn(0x7f095406baf0 msgr2=0x7f09540ce540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:29.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 shutdown_connections 2023-10-08T01:08:29.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:29.920+0000 7f095a59d700 1 -- 172.21.15.119:0/1844748619 wait complete. 2023-10-08T01:08:29.938 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:57:00", "first": "2023-10-08T00:58:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T01:08:29.938 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:57:00', 'first': '2023-10-08T00:58:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:29.939 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T01:08:29.939 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T01:08:29.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:29.939 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-08T01:08:29.977 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:29.977 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:29.977 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:29.977 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:29.978 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:29.978 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:29.978 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:29.978 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:29.978 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:29.978 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:29.978 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:29.979 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:29.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-10-08T01:08:30.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.205+0000 7faba19ff700 1 Processor -- start 2023-10-08T01:08:30.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.206+0000 7faba19ff700 1 -- start start 2023-10-08T01:08:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7faba19ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab9c069a50 0x7fab9c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7faba19ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069e20 0x7fab9c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7faba19ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab9c0ac030 con 0x7fab9c0c8490 2023-10-08T01:08:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7faba19ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab9c0ab820 con 0x7fab9c069a50 2023-10-08T01:08:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7faba19ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c06dbb0 con 0x7fab9c069e20 2023-10-08T01:08:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7fab9affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab9c069a50 0x7fab9c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7fab9affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab9c069a50 0x7fab9c0c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49954/0 (socket says 172.21.15.119:49954) 2023-10-08T01:08:30.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7fab9affd700 1 -- 172.21.15.119:0/878548466 learned_addr learned my addr 172.21.15.119:0/878548466 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7fab9b7fe700 1 --2- 172.21.15.119:0/878548466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069e20 0x7fab9c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7fab9affd700 1 -- 172.21.15.119:0/878548466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069e20 msgr2=0x7fab9c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7fab9affd700 1 --2- 172.21.15.119:0/878548466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069e20 0x7fab9c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7fab9affd700 1 -- 172.21.15.119:0/878548466 >> v1:172.21.15.184:6789/0 conn(0x7fab9c0c8490 legacy=0x7fab9c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.207+0000 7fab9affd700 1 -- 172.21.15.119:0/878548466 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab9c0aba70 con 0x7fab9c069a50 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.208+0000 7fab9affd700 1 --2- 172.21.15.119:0/878548466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c069a50 0x7fab9c0c6ac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0099b0 tx=0x7fab8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2eb3a0816ddb16 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.208+0000 7fab99ffb700 1 -- 172.21.15.119:0/878548466 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00b860 con 0x7fab9c069a50 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.208+0000 7fab99ffb700 1 -- 172.21.15.119:0/878548466 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8c00b9c0 con 0x7fab9c069a50 2023-10-08T01:08:30.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.208+0000 7fab99ffb700 1 -- 172.21.15.119:0/878548466 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab8c00bd60 con 0x7fab9c069a50 2023-10-08T01:08:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.208+0000 7faba19ff700 1 -- 172.21.15.119:0/878548466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c069a50 msgr2=0x7fab9c0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.208+0000 7faba19ff700 1 --2- 172.21.15.119:0/878548466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c069a50 0x7fab9c0c6ac0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0099b0 tx=0x7fab8c009250 comp rx=0 tx=0).stop 2023-10-08T01:08:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.209+0000 7faba19ff700 1 -- 172.21.15.119:0/878548466 shutdown_connections 2023-10-08T01:08:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.209+0000 7faba19ff700 1 --2- 172.21.15.119:0/878548466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069e20 0x7fab9c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.209+0000 7faba19ff700 1 --2- 172.21.15.119:0/878548466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c069a50 0x7fab9c0c6ac0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.209+0000 7faba19ff700 1 -- 172.21.15.119:0/878548466 >> 172.21.15.119:0/878548466 conn(0x7fab9c0c8f90 msgr2=0x7fab9c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:30.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.209+0000 7faba19ff700 1 -- 172.21.15.119:0/878548466 shutdown_connections 2023-10-08T01:08:30.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.209+0000 7faba19ff700 1 -- 172.21.15.119:0/878548466 wait complete. 2023-10-08T01:08:30.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.210+0000 7faba19ff700 1 Processor -- start 2023-10-08T01:08:30.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.210+0000 7faba19ff700 1 -- start start 2023-10-08T01:08:30.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.211+0000 7faba19ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069a50 0x7fab9c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.211+0000 7faba19ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab9c069e20 0x7fab9c154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.211+0000 7faba19ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c0c8490 0x7fab9c0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.211+0000 7faba19ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c155000 con 0x7fab9c0c8490 2023-10-08T01:08:30.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.211+0000 7faba19ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0b1b60 con 0x7fab9c069e20 2023-10-08T01:08:30.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.211+0000 7faba19ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0b0f50 con 0x7fab9c069a50 2023-10-08T01:08:30.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.211+0000 7fab9b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c0c8490 0x7fab9c0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.211+0000 7fab9b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c0c8490 0x7fab9c0afd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49964/0 (socket says 172.21.15.119:49964) 2023-10-08T01:08:30.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9b7fe700 1 -- 172.21.15.119:0/2442564416 learned_addr learned my addr 172.21.15.119:0/2442564416 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:30.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9affd700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069a50 0x7fab9c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9a7fc700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab9c069e20 0x7fab9c154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9b7fe700 1 -- 172.21.15.119:0/2442564416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069a50 msgr2=0x7fab9c0b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9b7fe700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069a50 0x7fab9c0b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9b7fe700 1 -- 172.21.15.119:0/2442564416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab9c069e20 msgr2=0x7fab9c154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9b7fe700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab9c069e20 0x7fab9c154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9b7fe700 1 -- 172.21.15.119:0/2442564416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab8c0094f0 con 0x7fab9c0c8490 2023-10-08T01:08:30.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab9b7fe700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c0c8490 0x7fab9c0afd20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fab94009fa0 tx=0x7fab94009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:30.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab82ffd700 1 -- 172.21.15.119:0/2442564416 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9400ed20 con 0x7fab9c0c8490 2023-10-08T01:08:30.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab82ffd700 1 -- 172.21.15.119:0/2442564416 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9400ee80 con 0x7fab9c0c8490 2023-10-08T01:08:30.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c0b1230 con 0x7fab9c0c8490 2023-10-08T01:08:30.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.212+0000 7fab82ffd700 1 -- 172.21.15.119:0/2442564416 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab94014670 con 0x7fab9c0c8490 2023-10-08T01:08:30.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.213+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c0b17f0 con 0x7fab9c0c8490 2023-10-08T01:08:30.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.214+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c0b1d10 con 0x7fab9c0c8490 2023-10-08T01:08:30.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.214+0000 7fab82ffd700 1 -- 172.21.15.119:0/2442564416 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fab940149e0 con 0x7fab9c0c8490 2023-10-08T01:08:30.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.215+0000 7fab82ffd700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fab88067130 0x7fab880695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.215+0000 7fab82ffd700 1 -- 172.21.15.119:0/2442564416 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7fab94085150 con 0x7fab9c0c8490 2023-10-08T01:08:30.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.215+0000 7fab9affd700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fab88067130 0x7fab880695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.216+0000 7fab9affd700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fab88067130 0x7fab880695e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0099b0 tx=0x7fab8c018000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:30.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.219+0000 7fab82ffd700 1 -- 172.21.15.119:0/2442564416 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab94052040 con 0x7fab9c0c8490 2023-10-08T01:08:30.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.436+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fab9c06fc30 con 0x7fab88067130 2023-10-08T01:08:30.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.444+0000 7fab82ffd700 1 -- 172.21.15.119:0/2442564416 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fab9c06fc30 con 0x7fab88067130 2023-10-08T01:08:30.445 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:30.445 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:57:01", "first": "2023-10-08T00:58:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T01:08:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.446+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fab88067130 msgr2=0x7fab880695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.446+0000 7faba19ff700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fab88067130 0x7fab880695e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab8c0099b0 tx=0x7fab8c018000 comp rx=0 tx=0).stop 2023-10-08T01:08:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c0c8490 msgr2=0x7fab9c0afd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c0c8490 0x7fab9c0afd20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fab94009fa0 tx=0x7fab94009900 comp rx=0 tx=0).stop 2023-10-08T01:08:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 shutdown_connections 2023-10-08T01:08:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab9c069a50 0x7fab9c0b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fab88067130 0x7fab880695e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab9c0c8490 0x7fab9c0afd20 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 --2- 172.21.15.119:0/2442564416 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab9c069e20 0x7fab9c154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 >> 172.21.15.119:0/2442564416 conn(0x7fab9c0c8f90 msgr2=0x7fab9c156310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:30.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 shutdown_connections 2023-10-08T01:08:30.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.447+0000 7faba19ff700 1 -- 172.21.15.119:0/2442564416 wait complete. 2023-10-08T01:08:30.466 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:57:01", "first": "2023-10-08T00:58:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T01:08:30.466 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:57:01', 'first': '2023-10-08T00:58:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:30.466 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T01:08:30.466 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T01:08:30.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:30.467 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-08T01:08:30.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:30.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:30.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:30.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:30.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:30.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:30.507 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:30.507 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:30.507 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:30.507 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:30.507 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:30.508 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-00_58_00_UTC', 'scheduled-2023-10-08-00_59_00_UTC', 'scheduled-2023-10-08-01_00_00_UTC', 'scheduled-2023-10-08-01_01_00_UTC', 'scheduled-2023-10-08-01_02_00_UTC', 'scheduled-2023-10-08-01_03_00_UTC', 'scheduled-2023-10-08-01_04_00_UTC', 'scheduled-2023-10-08-01_05_00_UTC', 'scheduled-2023-10-08-01_06_00_UTC', 'scheduled-2023-10-08-01_07_00_UTC', 'scheduled-2023-10-08-01_08_00_UTC'] 2023-10-08T01:08:30.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-10-08T01:08:30.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.732+0000 7f742b3ff700 1 Processor -- start 2023-10-08T01:08:30.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.732+0000 7f742b3ff700 1 -- start start 2023-10-08T01:08:30.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.733+0000 7f742b3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7424157740 0x7f74240adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.733+0000 7f742b3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 0x7f742406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.733+0000 7f742b3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742406dc20 con 0x7f74240b2100 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.733+0000 7f742b3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74240acc00 con 0x7f7424157740 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.733+0000 7f742b3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742406d6a0 con 0x7f7424068aa0 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.733+0000 7f742919b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7424157740 0x7f74240adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f742919b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7424157740 0x7f74240adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49970/0 (socket says 172.21.15.119:49970) 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f742919b700 1 -- 172.21.15.119:0/1656566353 learned_addr learned my addr 172.21.15.119:0/1656566353 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f742999c700 1 --2- 172.21.15.119:0/1656566353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 0x7f742406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f742919b700 1 -- 172.21.15.119:0/1656566353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 msgr2=0x7f742406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f742919b700 1 --2- 172.21.15.119:0/1656566353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 0x7f742406e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f742919b700 1 -- 172.21.15.119:0/1656566353 >> v1:172.21.15.184:6789/0 conn(0x7f74240b2100 legacy=0x7f7424068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f742919b700 1 -- 172.21.15.119:0/1656566353 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240ae910 con 0x7f7424157740 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f742919b700 1 --2- 172.21.15.119:0/1656566353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7424157740 0x7f74240adfc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f74200099b0 tx=0x7f7420009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7400e364fca0296e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.734+0000 7f741bfff700 1 -- 172.21.15.119:0/1656566353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000b860 con 0x7f7424157740 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f741bfff700 1 -- 172.21.15.119:0/1656566353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742000b9c0 con 0x7f7424157740 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f741bfff700 1 -- 172.21.15.119:0/1656566353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742000bd60 con 0x7f7424157740 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f742b3ff700 1 -- 172.21.15.119:0/1656566353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7424157740 msgr2=0x7f74240adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1656566353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7424157740 0x7f74240adfc0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f74200099b0 tx=0x7f7420009250 comp rx=0 tx=0).stop 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f742b3ff700 1 -- 172.21.15.119:0/1656566353 shutdown_connections 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1656566353 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 0x7f742406e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1656566353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7424157740 0x7f74240adfc0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f742b3ff700 1 -- 172.21.15.119:0/1656566353 >> 172.21.15.119:0/1656566353 conn(0x7f74240c8f90 msgr2=0x7f74240c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:30.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f742b3ff700 1 -- 172.21.15.119:0/1656566353 shutdown_connections 2023-10-08T01:08:30.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.735+0000 7f742b3ff700 1 -- 172.21.15.119:0/1656566353 wait complete. 2023-10-08T01:08:30.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.737+0000 7f742b3ff700 1 Processor -- start 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.737+0000 7f742b3ff700 1 -- start start 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742b3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 0x7f7424153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742b3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74240b2100 0x7f742416dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742b3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7424157740 0x7f742415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742b3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742416a7a0 con 0x7f74240b2100 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742b3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742416ed10 con 0x7f7424157740 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742b3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7424151ee0 con 0x7f7424068aa0 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742899a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74240b2100 0x7f742416dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742899a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74240b2100 0x7f742416dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49972/0 (socket says 172.21.15.119:49972) 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742899a700 1 -- 172.21.15.119:0/1303047760 learned_addr learned my addr 172.21.15.119:0/1303047760 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:30.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742919b700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 0x7f7424153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742999c700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7424157740 0x7f742415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742899a700 1 -- 172.21.15.119:0/1303047760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 msgr2=0x7f7424153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742899a700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 0x7f7424153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742899a700 1 -- 172.21.15.119:0/1303047760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7424157740 msgr2=0x7f742415c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742899a700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7424157740 0x7f742415c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.738+0000 7f742899a700 1 -- 172.21.15.119:0/1303047760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74200094f0 con 0x7f74240b2100 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.739+0000 7f742899a700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74240b2100 0x7f742416dde0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f741400c590 tx=0x7f741400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.739+0000 7f74197fa700 1 -- 172.21.15.119:0/1303047760 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414011590 con 0x7f74240b2100 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.739+0000 7f74197fa700 1 -- 172.21.15.119:0/1303047760 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7414011710 con 0x7f74240b2100 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.739+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742416e470 con 0x7f74240b2100 2023-10-08T01:08:30.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.739+0000 7f74197fa700 1 -- 172.21.15.119:0/1303047760 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7414011ad0 con 0x7f74240b2100 2023-10-08T01:08:30.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.739+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742416ea30 con 0x7f74240b2100 2023-10-08T01:08:30.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.740+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74240ae410 con 0x7f74240b2100 2023-10-08T01:08:30.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.741+0000 7f74197fa700 1 -- 172.21.15.119:0/1303047760 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f741401b3b0 con 0x7f74240b2100 2023-10-08T01:08:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.742+0000 7f74197fa700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7408067130 0x7f74080695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.742+0000 7f74197fa700 1 -- 172.21.15.119:0/1303047760 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f7414083e60 con 0x7f74240b2100 2023-10-08T01:08:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.742+0000 7f742919b700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7408067130 0x7f74080695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:30.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.743+0000 7f742919b700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7408067130 0x7f74080695e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f74200099b0 tx=0x7f7420017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:30.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.746+0000 7f74197fa700 1 -- 172.21.15.119:0/1303047760 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7414050cb0 con 0x7f74240b2100 2023-10-08T01:08:30.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.959+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f742406b900 con 0x7f7408067130 2023-10-08T01:08:30.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.968+0000 7f74197fa700 1 -- 172.21.15.119:0/1303047760 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f742406b900 con 0x7f7408067130 2023-10-08T01:08:30.970 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:30.970 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:57:01", "first": "2023-10-08T00:58:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T01:08:30.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7408067130 msgr2=0x7f74080695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7408067130 0x7f74080695e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f74200099b0 tx=0x7f7420017000 comp rx=0 tx=0).stop 2023-10-08T01:08:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74240b2100 msgr2=0x7f742416dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74240b2100 0x7f742416dde0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f741400c590 tx=0x7f741400c8a0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 shutdown_connections 2023-10-08T01:08:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7424068aa0 0x7f7424153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f7408067130 0x7f74080695e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7424157740 0x7f742415c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 --2- 172.21.15.119:0/1303047760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74240b2100 0x7f742416dde0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:30.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 >> 172.21.15.119:0/1303047760 conn(0x7f74240c8f90 msgr2=0x7f742415d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:30.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 shutdown_connections 2023-10-08T01:08:30.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:30.971+0000 7f742b3ff700 1 -- 172.21.15.119:0/1303047760 wait complete. 2023-10-08T01:08:30.989 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T00:57:01", "first": "2023-10-08T00:58:00", "last": "2023-10-08T01:08:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-10-08T01:08:30.989 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T00:57:01', 'first': '2023-10-08T00:58:00', 'last': '2023-10-08T01:08:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-10-08T01:08:30.990 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T01:08:30.990 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T01:08:30.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-10-08T01:08:31.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.176+0000 7f3d397ff700 1 Processor -- start 2023-10-08T01:08:31.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.176+0000 7f3d397ff700 1 -- start start 2023-10-08T01:08:31.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.177+0000 7f3d397ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34157740 0x7f3d340adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:31.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.177+0000 7f3d397ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3d34068aa0 0x7f3d3406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:31.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.177+0000 7f3d397ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d3406dc20 con 0x7f3d340b2100 2023-10-08T01:08:31.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.177+0000 7f3d397ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d340acc00 con 0x7f3d34068aa0 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.177+0000 7f3d397ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3406d6a0 con 0x7f3d34157740 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d327fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3d340b2100 0x7f3d34068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56682/0 (socket says 172.21.15.119:56682) 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d327fc700 1 -- 172.21.15.119:0/1200461127 learned_addr learned my addr 172.21.15.119:0/1200461127 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d32ffd700 1 --2- 172.21.15.119:0/1200461127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34157740 0x7f3d340adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d337fe700 1 --2- 172.21.15.119:0/1200461127 >> v2:172.21.15.184:3300/0 conn(0x7f3d34068aa0 0x7f3d3406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d31ffb700 1 -- 172.21.15.119:0/1200461127 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824100896 0 0) 0x7f3d3406dc20 con 0x7f3d340b2100 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d31ffb700 1 -- 172.21.15.119:0/1200461127 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d1c003570 con 0x7f3d340b2100 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d337fe700 1 -- 172.21.15.119:0/1200461127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34157740 msgr2=0x7f3d340adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d337fe700 1 --2- 172.21.15.119:0/1200461127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34157740 0x7f3d340adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d337fe700 1 -- 172.21.15.119:0/1200461127 >> v1:172.21.15.119:6789/0 conn(0x7f3d340b2100 legacy=0x7f3d34068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.178+0000 7f3d337fe700 1 -- 172.21.15.119:0/1200461127 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d340ae910 con 0x7f3d34068aa0 2023-10-08T01:08:31.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.179+0000 7f3d337fe700 1 --2- 172.21.15.119:0/1200461127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34068aa0 0x7f3d3406e0b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009e60 tx=0x7f3d2c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b09e5aec164fcea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:31.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.179+0000 7f3d31ffb700 1 -- 172.21.15.119:0/1200461127 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c00e070 con 0x7f3d34068aa0 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.179+0000 7f3d31ffb700 1 -- 172.21.15.119:0/1200461127 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c00bab0 con 0x7f3d34068aa0 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.179+0000 7f3d31ffb700 1 -- 172.21.15.119:0/1200461127 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c00be70 con 0x7f3d34068aa0 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.180+0000 7f3d397ff700 1 -- 172.21.15.119:0/1200461127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34068aa0 msgr2=0x7f3d3406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.180+0000 7f3d397ff700 1 --2- 172.21.15.119:0/1200461127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34068aa0 0x7f3d3406e0b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009e60 tx=0x7f3d2c009780 comp rx=0 tx=0).stop 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.180+0000 7f3d397ff700 1 -- 172.21.15.119:0/1200461127 shutdown_connections 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.180+0000 7f3d397ff700 1 --2- 172.21.15.119:0/1200461127 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d34068aa0 0x7f3d3406e0b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.180+0000 7f3d397ff700 1 --2- 172.21.15.119:0/1200461127 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34157740 0x7f3d340adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.180+0000 7f3d397ff700 1 -- 172.21.15.119:0/1200461127 >> 172.21.15.119:0/1200461127 conn(0x7f3d340c8f90 msgr2=0x7f3d340c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.180+0000 7f3d397ff700 1 -- 172.21.15.119:0/1200461127 shutdown_connections 2023-10-08T01:08:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.180+0000 7f3d397ff700 1 -- 172.21.15.119:0/1200461127 wait complete. 2023-10-08T01:08:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.181+0000 7f3d397ff700 1 Processor -- start 2023-10-08T01:08:31.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.181+0000 7f3d397ff700 1 -- start start 2023-10-08T01:08:31.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.182+0000 7f3d397ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068aa0 0x7f3d3415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:31.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.182+0000 7f3d397ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d340b2100 0x7f3d3415d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:31.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.182+0000 7f3d397ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d34157740 0x7f3d340b0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:31.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.182+0000 7f3d397ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3415cd50 con 0x7f3d34157740 2023-10-08T01:08:31.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.182+0000 7f3d397ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3415c790 con 0x7f3d340b2100 2023-10-08T01:08:31.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.182+0000 7f3d397ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d3415ca70 con 0x7f3d34068aa0 2023-10-08T01:08:31.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.182+0000 7f3d337fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d34157740 0x7f3d340b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d337fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d34157740 0x7f3d340b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49980/0 (socket says 172.21.15.119:49980) 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d337fe700 1 -- 172.21.15.119:0/4133512725 learned_addr learned my addr 172.21.15.119:0/4133512725 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d32ffd700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068aa0 0x7f3d3415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d327fc700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d340b2100 0x7f3d3415d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d337fe700 1 -- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068aa0 msgr2=0x7f3d3415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d337fe700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068aa0 0x7f3d3415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d337fe700 1 -- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d340b2100 msgr2=0x7f3d3415d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d337fe700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d340b2100 0x7f3d3415d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d337fe700 1 -- 172.21.15.119:0/4133512725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c009a20 con 0x7f3d34157740 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d327fc700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d340b2100 0x7f3d3415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.183+0000 7f3d337fe700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d34157740 0x7f3d340b0460 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c013000 tx=0x7f3d2c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:31.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.184+0000 7f3d1affd700 1 -- 172.21.15.119:0/4133512725 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c027470 con 0x7f3d34157740 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.184+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d3415a7b0 con 0x7f3d34157740 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.184+0000 7f3d1affd700 1 -- 172.21.15.119:0/4133512725 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c0275f0 con 0x7f3d34157740 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.184+0000 7f3d1affd700 1 -- 172.21.15.119:0/4133512725 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0279b0 con 0x7f3d34157740 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.184+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3415c3f0 con 0x7f3d34157740 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.185+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d340718b0 con 0x7f3d34157740 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.186+0000 7f3d1affd700 1 -- 172.21.15.119:0/4133512725 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f3d2c024070 con 0x7f3d34157740 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.186+0000 7f3d1affd700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d20067130 0x7f3d200695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.187+0000 7f3d1affd700 1 -- 172.21.15.119:0/4133512725 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f3d2c087410 con 0x7f3d34157740 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.187+0000 7f3d32ffd700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d20067130 0x7f3d200695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:31.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.187+0000 7f3d32ffd700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d20067130 0x7f3d200695e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009d10 tx=0x7f3d240091a0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:31.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.190+0000 7f3d1affd700 1 -- 172.21.15.119:0/4133512725 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d2c054300 con 0x7f3d34157740 2023-10-08T01:08:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.408+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d3415a630 con 0x7f3d20067130 2023-10-08T01:08:31.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.485+0000 7f3d1affd700 1 -- 172.21.15.119:0/4133512725 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f3d24011a50 con 0x7f3d20067130 2023-10-08T01:08:31.486 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-10-08T01:08:31.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.487+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d20067130 msgr2=0x7f3d200695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:31.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.487+0000 7f3d397ff700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d20067130 0x7f3d200695e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d24009d10 tx=0x7f3d240091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.487+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d34157740 msgr2=0x7f3d340b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:31.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.487+0000 7f3d397ff700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d34157740 0x7f3d340b0460 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c013000 tx=0x7f3d2c019000 comp rx=0 tx=0).stop 2023-10-08T01:08:31.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.488+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 shutdown_connections 2023-10-08T01:08:31.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.488+0000 7f3d397ff700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d34068aa0 0x7f3d3415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.488+0000 7f3d397ff700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f3d20067130 0x7f3d200695e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.488+0000 7f3d397ff700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d34157740 0x7f3d340b0460 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.488+0000 7f3d397ff700 1 --2- 172.21.15.119:0/4133512725 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d340b2100 0x7f3d3415d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:31.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.488+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 >> 172.21.15.119:0/4133512725 conn(0x7f3d340c8f90 msgr2=0x7f3d340aa840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:31.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.488+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 shutdown_connections 2023-10-08T01:08:31.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:31.488+0000 7f3d397ff700 1 -- 172.21.15.119:0/4133512725 wait complete. 2023-10-08T01:08:31.506 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-10-08T01:08:31.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:31.507 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-10-08T01:08:31.544 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:31.544 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:31.545 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:31.545 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:31.545 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:31.545 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:31.545 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:31.545 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:31.545 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:31.546 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:31.546 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:31.546 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:31.547 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:31.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:31.547 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-00_57_00_UTC) 2023-10-08T01:08:31.632 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:31.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:31.633 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:31.716 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:31.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:31.716 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:31.800 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:31.800 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:31.801 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:31.886 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:31.886 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:31.887 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:31.970 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:31.970 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:31.970 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:32.057 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:32.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:32.057 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:32.103 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:32.103 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:32.103 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:32.188 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:32.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:32.188 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:32.272 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:32.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:32.273 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:32.357 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:32.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:32.357 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:32.441 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:32.441 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:32.441 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:32.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:32.526 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-10-08T01:08:32.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-10-08T01:08:32.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.832+0000 7f57aebff700 1 Processor -- start 2023-10-08T01:08:32.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.832+0000 7f57aebff700 1 -- start start 2023-10-08T01:08:32.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.833+0000 7f57aebff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f57a8159ec0 0x7f57a80b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:32.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.833+0000 7f57aebff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57a80b53d0 0x7f57a80cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:32.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.833+0000 7f57aebff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a80b5b00 con 0x7f57a80cce40 2023-10-08T01:08:32.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.833+0000 7f57aebff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a80b5850 con 0x7f57a80b53d0 2023-10-08T01:08:32.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.833+0000 7f57aebff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a80b5da0 con 0x7f57a8159ec0 2023-10-08T01:08:32.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57a7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57a80b53d0 0x7f57a80cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:32.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57ad19c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f57a80cce40 0x7f57a80cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56692/0 (socket says 172.21.15.119:56692) 2023-10-08T01:08:32.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57ad19c700 1 -- 172.21.15.119:0/3660636658 learned_addr learned my addr 172.21.15.119:0/3660636658 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:32.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57a7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57a80b53d0 0x7f57a80cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49990/0 (socket says 172.21.15.119:49990) 2023-10-08T01:08:32.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57ac99b700 1 --2- 172.21.15.119:0/3660636658 >> v2:172.21.15.184:3300/0 conn(0x7f57a8159ec0 0x7f57a80b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:32.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57a7fff700 1 -- 172.21.15.119:0/3660636658 >> v2:172.21.15.184:3300/0 conn(0x7f57a8159ec0 msgr2=0x7f57a80b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57a7fff700 1 --2- 172.21.15.119:0/3660636658 >> v2:172.21.15.184:3300/0 conn(0x7f57a8159ec0 0x7f57a80b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57a7fff700 1 -- 172.21.15.119:0/3660636658 >> v1:172.21.15.119:6789/0 conn(0x7f57a80cce40 legacy=0x7f57a80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57a7fff700 1 -- 172.21.15.119:0/3660636658 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80b6080 con 0x7f57a80b53d0 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57ac99b700 1 --2- 172.21.15.119:0/3660636658 >> v2:172.21.15.184:3300/0 conn(0x7f57a8159ec0 0x7f57a80b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.834+0000 7f57a7fff700 1 --2- 172.21.15.119:0/3660636658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80b53d0 0x7f57a80cc900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57a8089120 tx=0x7f57940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfae4be5993b744b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57a77fe700 1 -- 172.21.15.119:0/3660636658 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579400b6b0 con 0x7f57a80b53d0 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57a77fe700 1 -- 172.21.15.119:0/3660636658 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579400b810 con 0x7f57a80b53d0 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57a77fe700 1 -- 172.21.15.119:0/3660636658 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579400bbb0 con 0x7f57a80b53d0 2023-10-08T01:08:32.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57aebff700 1 -- 172.21.15.119:0/3660636658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80b53d0 msgr2=0x7f57a80cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:32.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57aebff700 1 --2- 172.21.15.119:0/3660636658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80b53d0 0x7f57a80cc900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f57a8089120 tx=0x7f57940091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:32.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57aebff700 1 -- 172.21.15.119:0/3660636658 shutdown_connections 2023-10-08T01:08:32.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57aebff700 1 --2- 172.21.15.119:0/3660636658 >> v2:172.21.15.184:3300/0 conn(0x7f57a8159ec0 0x7f57a80b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:32.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57aebff700 1 --2- 172.21.15.119:0/3660636658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80b53d0 0x7f57a80cc900 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:32.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57aebff700 1 -- 172.21.15.119:0/3660636658 >> 172.21.15.119:0/3660636658 conn(0x7f57a80999f0 msgr2=0x7f57a80b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:32.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57aebff700 1 -- 172.21.15.119:0/3660636658 shutdown_connections 2023-10-08T01:08:32.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.835+0000 7f57aebff700 1 -- 172.21.15.119:0/3660636658 wait complete. 2023-10-08T01:08:32.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.837+0000 7f57aebff700 1 Processor -- start 2023-10-08T01:08:32.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.837+0000 7f57aebff700 1 -- start start 2023-10-08T01:08:32.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57aebff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a80b53d0 0x7f57a8151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:32.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57aebff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80cce40 0x7f57a8158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:32.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57aebff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8159ec0 0x7f57a80c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:32.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57aebff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8159520 con 0x7f57a80cce40 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57aebff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81542d0 con 0x7f57a80b53d0 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57aebff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a81539a0 con 0x7f57a8159ec0 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80cce40 0x7f57a8158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80cce40 0x7f57a8158cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49994/0 (socket says 172.21.15.119:49994) 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57a7fff700 1 -- 172.21.15.119:0/681513614 learned_addr learned my addr 172.21.15.119:0/681513614 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57ac99b700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a80b53d0 0x7f57a8151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.838+0000 7f57ad19c700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8159ec0 0x7f57a80c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a7fff700 1 -- 172.21.15.119:0/681513614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8159ec0 msgr2=0x7f57a80c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:32.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a7fff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8159ec0 0x7f57a80c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a7fff700 1 -- 172.21.15.119:0/681513614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a80b53d0 msgr2=0x7f57a8151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a7fff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a80b53d0 0x7f57a8151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a7fff700 1 -- 172.21.15.119:0/681513614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794009420 con 0x7f57a80cce40 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a7fff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80cce40 0x7f57a8158cf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f57a8089390 tx=0x7f5794017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a4ff9700 1 -- 172.21.15.119:0/681513614 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579400bb90 con 0x7f57a80cce40 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a4ff9700 1 -- 172.21.15.119:0/681513614 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5794011420 con 0x7f57a80cce40 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57a4ff9700 1 -- 172.21.15.119:0/681513614 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57940117a0 con 0x7f57a80cce40 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a8153c80 con 0x7f57a80cce40 2023-10-08T01:08:32.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.839+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a8159230 con 0x7f57a80cce40 2023-10-08T01:08:32.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.840+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a8063c90 con 0x7f57a80cce40 2023-10-08T01:08:32.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.841+0000 7f57a4ff9700 1 -- 172.21.15.119:0/681513614 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f5794011b90 con 0x7f57a80cce40 2023-10-08T01:08:32.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.842+0000 7f57a4ff9700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f578c067130 0x7f578c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:32.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.842+0000 7f57a4ff9700 1 -- 172.21.15.119:0/681513614 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f57940861b0 con 0x7f57a80cce40 2023-10-08T01:08:32.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.842+0000 7f57ac99b700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f578c067130 0x7f578c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:32.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.843+0000 7f57ac99b700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f578c067130 0x7f578c0695e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f579c006ea0 tx=0x7f579c008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:32.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:32.846+0000 7f57a4ff9700 1 -- 172.21.15.119:0/681513614 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5794053000 con 0x7f57a80cce40 2023-10-08T01:08:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.062+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f57a8154440 con 0x7f578c067130 2023-10-08T01:08:33.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.142+0000 7f57a4ff9700 1 -- 172.21.15.119:0/681513614 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f57a8154440 con 0x7f578c067130 2023-10-08T01:08:33.143 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-10-08T01:08:33.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f578c067130 msgr2=0x7f578c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:33.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f578c067130 0x7f578c0695e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f579c006ea0 tx=0x7f579c008040 comp rx=0 tx=0).stop 2023-10-08T01:08:33.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80cce40 msgr2=0x7f57a8158cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:33.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80cce40 0x7f57a8158cf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f57a8089390 tx=0x7f5794017000 comp rx=0 tx=0).stop 2023-10-08T01:08:33.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 shutdown_connections 2023-10-08T01:08:33.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8159ec0 0x7f57a80c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:33.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f578c067130 0x7f578c0695e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:33.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a80b53d0 0x7f57a8151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:33.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 --2- 172.21.15.119:0/681513614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80cce40 0x7f57a8158cf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:33.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 >> 172.21.15.119:0/681513614 conn(0x7f57a80999f0 msgr2=0x7f57a80c59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:33.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 shutdown_connections 2023-10-08T01:08:33.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:33.145+0000 7f57aebff700 1 -- 172.21.15.119:0/681513614 wait complete. 2023-10-08T01:08:33.163 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-10-08T01:08:33.164 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.164 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-10-08T01:08:33.201 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_01_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_01_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:33.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:33.203 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:33.203 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:33.205 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-00_57_01_UTC 2023-10-08T01:08:33.205 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.205 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-00_57_01_UTC) 2023-10-08T01:08:33.290 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:33.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.290 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:33.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-00_59_01_UTC 2023-10-08T01:08:33.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.375 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-00_59_01_UTC) 2023-10-08T01:08:33.459 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:33.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.459 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:33.544 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:33.545 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.545 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:33.628 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:33.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.629 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:33.712 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:33.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.713 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:33.797 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:33.797 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.797 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:33.881 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:33.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.882 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:33.966 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:33.966 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:33.967 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:34.051 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:34.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:34.051 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:34.135 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:34.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:34.135 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:34.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:34.219 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-10-08T01:08:34.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-10-08T01:08:34.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.523+0000 7ff89b59e700 1 Processor -- start 2023-10-08T01:08:34.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.523+0000 7ff89b59e700 1 -- start start 2023-10-08T01:08:34.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89b59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff89c1597d0 0x7ff89c0afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:34.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c0ac0e0 0x7ff89c06e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:34.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff89c06eee0 0x7ff89c06f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:34.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89b59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c06e510 con 0x7ff89c06eee0 2023-10-08T01:08:34.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89b59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c069180 con 0x7ff89c1597d0 2023-10-08T01:08:34.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c06dec0 con 0x7ff89c0ac0e0 2023-10-08T01:08:34.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff89c06eee0 0x7ff89c06f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:34.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff89c06eee0 0x7ff89c06f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49998/0 (socket says 172.21.15.119:49998) 2023-10-08T01:08:34.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.524+0000 7ff89ad9d700 1 -- 172.21.15.119:0/1529340652 learned_addr learned my addr 172.21.15.119:0/1529340652 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:34.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89ad9d700 1 -- 172.21.15.119:0/1529340652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c0ac0e0 msgr2=0x7ff89c06e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:34.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff899d9b700 1 --2- 172.21.15.119:0/1529340652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c0ac0e0 0x7ff89c06e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:34.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89a59c700 1 --2- 172.21.15.119:0/1529340652 >> v2:172.21.15.184:3300/0 conn(0x7ff89c1597d0 0x7ff89c0afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:34.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89ad9d700 1 --2- 172.21.15.119:0/1529340652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c0ac0e0 0x7ff89c06e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89ad9d700 1 -- 172.21.15.119:0/1529340652 >> v2:172.21.15.184:3300/0 conn(0x7ff89c1597d0 msgr2=0x7ff89c0afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89ad9d700 1 --2- 172.21.15.119:0/1529340652 >> v2:172.21.15.184:3300/0 conn(0x7ff89c1597d0 0x7ff89c0afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89ad9d700 1 -- 172.21.15.119:0/1529340652 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c06e1a0 con 0x7ff89c06eee0 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89a59c700 1 --2- 172.21.15.119:0/1529340652 >> v2:172.21.15.184:3300/0 conn(0x7ff89c1597d0 0x7ff89c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff899d9b700 1 --2- 172.21.15.119:0/1529340652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c0ac0e0 0x7ff89c06e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89ad9d700 1 --2- 172.21.15.119:0/1529340652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c06f340 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff89c06d510 tx=0x7ff894009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53a96869df9c196 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89959a700 1 -- 172.21.15.119:0/1529340652 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89400e070 con 0x7ff89c06eee0 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89959a700 1 -- 172.21.15.119:0/1529340652 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89400ba50 con 0x7ff89c06eee0 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.525+0000 7ff89959a700 1 -- 172.21.15.119:0/1529340652 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8940092e0 con 0x7ff89c06eee0 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 -- 172.21.15.119:0/1529340652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 msgr2=0x7ff89c06f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:34.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 --2- 172.21.15.119:0/1529340652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c06f340 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff89c06d510 tx=0x7ff894009840 comp rx=0 tx=0).stop 2023-10-08T01:08:34.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 -- 172.21.15.119:0/1529340652 shutdown_connections 2023-10-08T01:08:34.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 --2- 172.21.15.119:0/1529340652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c06f340 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 --2- 172.21.15.119:0/1529340652 >> v2:172.21.15.184:3300/0 conn(0x7ff89c1597d0 0x7ff89c0afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 --2- 172.21.15.119:0/1529340652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c0ac0e0 0x7ff89c06e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 -- 172.21.15.119:0/1529340652 >> 172.21.15.119:0/1529340652 conn(0x7ff89c06adf0 msgr2=0x7ff89c0ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:34.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 -- 172.21.15.119:0/1529340652 shutdown_connections 2023-10-08T01:08:34.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.526+0000 7ff89b59e700 1 -- 172.21.15.119:0/1529340652 wait complete. 2023-10-08T01:08:34.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.528+0000 7ff89b59e700 1 Processor -- start 2023-10-08T01:08:34.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.528+0000 7ff89b59e700 1 -- start start 2023-10-08T01:08:34.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c0c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:34.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c1597d0 0x7ff89c0c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:34.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1454f0 0x7ff89c169a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:34.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89b59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c15b1c0 con 0x7ff89c06eee0 2023-10-08T01:08:34.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89b59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c15a5b0 con 0x7ff89c1454f0 2023-10-08T01:08:34.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c15a890 con 0x7ff89c1597d0 2023-10-08T01:08:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c0c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c0c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50012/0 (socket says 172.21.15.119:50012) 2023-10-08T01:08:34.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89a59c700 1 -- 172.21.15.119:0/274110475 learned_addr learned my addr 172.21.15.119:0/274110475 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:34.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff899d9b700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c1597d0 0x7ff89c0c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:34.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89ad9d700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1454f0 0x7ff89c169a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:34.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89a59c700 1 -- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c1597d0 msgr2=0x7ff89c0c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:34.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89a59c700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c1597d0 0x7ff89c0c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89a59c700 1 -- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1454f0 msgr2=0x7ff89c169a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:34.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89a59c700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1454f0 0x7ff89c169a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89a59c700 1 -- 172.21.15.119:0/274110475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894009ae0 con 0x7ff89c06eee0 2023-10-08T01:08:34.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff89ad9d700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1454f0 0x7ff89c169a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:34.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.529+0000 7ff899d9b700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c1597d0 0x7ff89c0c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:34.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.530+0000 7ff89a59c700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c0c19c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c850 tx=0x7ff88c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:34.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.530+0000 7ff88a7fc700 1 -- 172.21.15.119:0/274110475 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c011600 con 0x7ff89c06eee0 2023-10-08T01:08:34.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.530+0000 7ff88a7fc700 1 -- 172.21.15.119:0/274110475 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff88c011760 con 0x7ff89c06eee0 2023-10-08T01:08:34.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.530+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89c15ab70 con 0x7ff89c06eee0 2023-10-08T01:08:34.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.530+0000 7ff88a7fc700 1 -- 172.21.15.119:0/274110475 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88c011b00 con 0x7ff89c06eee0 2023-10-08T01:08:34.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.530+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c145990 con 0x7ff89c06eee0 2023-10-08T01:08:34.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.531+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89c04c3c0 con 0x7ff89c06eee0 2023-10-08T01:08:34.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.532+0000 7ff88a7fc700 1 -- 172.21.15.119:0/274110475 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7ff88c0073b0 con 0x7ff89c06eee0 2023-10-08T01:08:34.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.532+0000 7ff88a7fc700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff87c067130 0x7ff87c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:34.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.533+0000 7ff88a7fc700 1 -- 172.21.15.119:0/274110475 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7ff88c084ad0 con 0x7ff89c06eee0 2023-10-08T01:08:34.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.533+0000 7ff899d9b700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff87c067130 0x7ff87c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:34.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.533+0000 7ff899d9b700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff87c067130 0x7ff87c0695e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0c2190 tx=0x7ff890008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:34.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.537+0000 7ff88a7fc700 1 -- 172.21.15.119:0/274110475 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff88c051810 con 0x7ff89c06eee0 2023-10-08T01:08:34.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.750+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff89c15b300 con 0x7ff87c067130 2023-10-08T01:08:34.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.830+0000 7ff88a7fc700 1 -- 172.21.15.119:0/274110475 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7ff89000c040 con 0x7ff87c067130 2023-10-08T01:08:34.831 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-10-08T01:08:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff87c067130 msgr2=0x7ff87c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff87c067130 0x7ff87c0695e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0c2190 tx=0x7ff890008040 comp rx=0 tx=0).stop 2023-10-08T01:08:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 msgr2=0x7ff89c0c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c0c19c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff88c00c850 tx=0x7ff88c008040 comp rx=0 tx=0).stop 2023-10-08T01:08:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 shutdown_connections 2023-10-08T01:08:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff87c067130 0x7ff87c0695e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c1454f0 0x7ff89c169a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c06eee0 0x7ff89c0c19c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 --2- 172.21.15.119:0/274110475 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c1597d0 0x7ff89c0c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.832+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 >> 172.21.15.119:0/274110475 conn(0x7ff89c06adf0 msgr2=0x7ff89c1588a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.833+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 shutdown_connections 2023-10-08T01:08:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:34.833+0000 7ff89b59e700 1 -- 172.21.15.119:0/274110475 wait complete. 2023-10-08T01:08:34.851 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-10-08T01:08:34.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:34.851 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-10-08T01:08:34.889 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:34.889 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:34.889 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:34.889 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:34.889 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:34.889 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:34.890 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:34.890 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:34.890 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:34.890 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:34.890 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:34.891 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:34.892 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:34.892 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:34.892 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-00_57_00_UTC) 2023-10-08T01:08:34.978 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:34.978 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:34.979 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:35.063 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:35.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.063 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:35.147 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:35.147 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.147 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:35.231 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:35.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.231 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:35.316 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:35.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.316 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:35.403 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:35.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.404 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:35.489 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:35.489 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.489 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:35.576 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:35.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.577 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:35.663 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:35.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.664 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:35.749 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:35.749 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.749 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:35.833 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:35.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.834 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:35.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:35.918 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-10-08T01:08:35.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-10-08T01:08:36.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.222+0000 7f4dab5ff700 1 Processor -- start 2023-10-08T01:08:36.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.223+0000 7f4dab5ff700 1 -- start start 2023-10-08T01:08:36.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4dab5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da4157740 0x7f4da40adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:36.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4dab5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4da40b2100 0x7f4da4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:36.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4dab5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da406dc20 con 0x7f4da40688d0 2023-10-08T01:08:36.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4dab5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da40acc00 con 0x7f4da40b2100 2023-10-08T01:08:36.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4dab5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da406d6a0 con 0x7f4da4157740 2023-10-08T01:08:36.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4da939b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da4157740 0x7f4da40adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:36.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4da9b9c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f4da40688d0 0x7f4da4069800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:47854/0 (socket says 172.21.15.119:47854) 2023-10-08T01:08:36.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4da9b9c700 1 -- 172.21.15.119:0/4045851876 learned_addr learned my addr 172.21.15.119:0/4045851876 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:36.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.224+0000 7f4da8b9a700 1 --2- 172.21.15.119:0/4045851876 >> v2:172.21.15.184:3300/0 conn(0x7f4da40b2100 0x7f4da4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:36.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4d9bfff700 1 -- 172.21.15.119:0/4045851876 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186284757 0 0) 0x7f4da406dc20 con 0x7f4da40688d0 2023-10-08T01:08:36.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4d9bfff700 1 -- 172.21.15.119:0/4045851876 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d90003570 con 0x7f4da40688d0 2023-10-08T01:08:36.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4da939b700 1 -- 172.21.15.119:0/4045851876 >> v2:172.21.15.184:3300/0 conn(0x7f4da40b2100 msgr2=0x7f4da4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4da939b700 1 --2- 172.21.15.119:0/4045851876 >> v2:172.21.15.184:3300/0 conn(0x7f4da40b2100 0x7f4da4068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4da939b700 1 -- 172.21.15.119:0/4045851876 >> v1:172.21.15.184:6789/0 conn(0x7f4da40688d0 legacy=0x7f4da4069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4da939b700 1 -- 172.21.15.119:0/4045851876 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da40ae910 con 0x7f4da4157740 2023-10-08T01:08:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4da8b9a700 1 --2- 172.21.15.119:0/4045851876 >> v2:172.21.15.184:3300/0 conn(0x7f4da40b2100 0x7f4da4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4da939b700 1 --2- 172.21.15.119:0/4045851876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da4157740 0x7f4da40adfc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4da00098e0 tx=0x7f4da00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48c8954a2e6156a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.225+0000 7f4d9bfff700 1 -- 172.21.15.119:0/4045851876 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0011630 con 0x7f4da4157740 2023-10-08T01:08:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4d9bfff700 1 -- 172.21.15.119:0/4045851876 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da0011790 con 0x7f4da4157740 2023-10-08T01:08:36.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4d9bfff700 1 -- 172.21.15.119:0/4045851876 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0011b30 con 0x7f4da4157740 2023-10-08T01:08:36.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4dab5ff700 1 -- 172.21.15.119:0/4045851876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da4157740 msgr2=0x7f4da40adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:36.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/4045851876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da4157740 0x7f4da40adfc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4da00098e0 tx=0x7f4da00091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4dab5ff700 1 -- 172.21.15.119:0/4045851876 shutdown_connections 2023-10-08T01:08:36.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/4045851876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da4157740 0x7f4da40adfc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/4045851876 >> v2:172.21.15.184:3300/0 conn(0x7f4da40b2100 0x7f4da4068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4dab5ff700 1 -- 172.21.15.119:0/4045851876 >> 172.21.15.119:0/4045851876 conn(0x7f4da40c8f90 msgr2=0x7f4da40c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:36.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4dab5ff700 1 -- 172.21.15.119:0/4045851876 shutdown_connections 2023-10-08T01:08:36.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.226+0000 7f4dab5ff700 1 -- 172.21.15.119:0/4045851876 wait complete. 2023-10-08T01:08:36.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.228+0000 7f4dab5ff700 1 Processor -- start 2023-10-08T01:08:36.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.228+0000 7f4dab5ff700 1 -- start start 2023-10-08T01:08:36.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.228+0000 7f4dab5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da40688d0 0x7f4da4155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4dab5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da40b2100 0x7f4da40c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4dab5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da4157740 0x7f4da40aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4dab5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da4156230 con 0x7f4da4157740 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4dab5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40aa590 con 0x7f4da40b2100 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4dab5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da40a9980 con 0x7f4da40688d0 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da9b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da4157740 0x7f4da40aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da939b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da40688d0 0x7f4da4155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da9b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da4157740 0x7f4da40aaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41978/0 (socket says 172.21.15.119:41978) 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da9b9c700 1 -- 172.21.15.119:0/275386785 learned_addr learned my addr 172.21.15.119:0/275386785 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da8b9a700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da40b2100 0x7f4da40c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da9b9c700 1 -- 172.21.15.119:0/275386785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da40688d0 msgr2=0x7f4da4155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da9b9c700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da40688d0 0x7f4da4155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da9b9c700 1 -- 172.21.15.119:0/275386785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da40b2100 msgr2=0x7f4da40c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da9b9c700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da40b2100 0x7f4da40c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.229+0000 7f4da9b9c700 1 -- 172.21.15.119:0/275386785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d94009420 con 0x7f4da4157740 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.230+0000 7f4da9b9c700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da4157740 0x7f4da40aaa20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c001460 tx=0x7f4d9c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.230+0000 7f4d997fa700 1 -- 172.21.15.119:0/275386785 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00a870 con 0x7f4da4157740 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.230+0000 7f4d997fa700 1 -- 172.21.15.119:0/275386785 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c00a9d0 con 0x7f4da4157740 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.230+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da0009420 con 0x7f4da4157740 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.230+0000 7f4d997fa700 1 -- 172.21.15.119:0/275386785 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00ad50 con 0x7f4da4157740 2023-10-08T01:08:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.230+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da40a9f40 con 0x7f4da4157740 2023-10-08T01:08:36.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.231+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da40718b0 con 0x7f4da4157740 2023-10-08T01:08:36.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.232+0000 7f4d997fa700 1 -- 172.21.15.119:0/275386785 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f4d9c007300 con 0x7f4da4157740 2023-10-08T01:08:36.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.233+0000 7f4d997fa700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4d88067130 0x7f4d880695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:36.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.233+0000 7f4d997fa700 1 -- 172.21.15.119:0/275386785 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5963+0+0 (secure 0 0 0) 0x7f4d9c084d70 con 0x7f4da4157740 2023-10-08T01:08:36.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.233+0000 7f4da939b700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4d88067130 0x7f4d880695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:36.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.233+0000 7f4da939b700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4d88067130 0x7f4d880695e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4da00091e0 tx=0x7f4da0017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:36.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.237+0000 7f4d997fa700 1 -- 172.21.15.119:0/275386785 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4d9c051bb0 con 0x7f4da4157740 2023-10-08T01:08:36.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.457+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4da40678d0 con 0x7f4d88067130 2023-10-08T01:08:36.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.536+0000 7f4d997fa700 1 -- 172.21.15.119:0/275386785 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f4da40678d0 con 0x7f4d88067130 2023-10-08T01:08:36.537 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-10-08T01:08:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4d88067130 msgr2=0x7f4d880695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4d88067130 0x7f4d880695e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4da00091e0 tx=0x7f4da0017000 comp rx=0 tx=0).stop 2023-10-08T01:08:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da4157740 msgr2=0x7f4da40aaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da4157740 0x7f4da40aaa20 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c001460 tx=0x7f4d9c001a40 comp rx=0 tx=0).stop 2023-10-08T01:08:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 shutdown_connections 2023-10-08T01:08:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4da40688d0 0x7f4da4155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4d88067130 0x7f4d880695e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4da4157740 0x7f4da40aaa20 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 --2- 172.21.15.119:0/275386785 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4da40b2100 0x7f4da40c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 >> 172.21.15.119:0/275386785 conn(0x7f4da40c8f90 msgr2=0x7f4da40cb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 shutdown_connections 2023-10-08T01:08:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:36.538+0000 7f4dab5ff700 1 -- 172.21.15.119:0/275386785 wait complete. 2023-10-08T01:08:36.556 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-10-08T01:08:36.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:36.557 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-10-08T01:08:36.595 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:36.595 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:36.595 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:36.595 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:36.595 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:36.595 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:36.596 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:36.596 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:36.596 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:36.596 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:36.596 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:36.596 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:36.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:36.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:36.597 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-00_57_00_UTC) 2023-10-08T01:08:36.682 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:36.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:36.682 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:36.769 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:36.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:36.770 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:36.853 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:36.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:36.853 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:36.940 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:36.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:36.940 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:37.026 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:37.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:37.027 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:37.110 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:37.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:37.111 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:37.234 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:37.235 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:37.235 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:37.309 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:37.309 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:37.309 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:37.372 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:37.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:37.372 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:37.423 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:37.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:37.424 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:37.484 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:37.484 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:37.484 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:37.553 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:37.553 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-10-08T01:08:37.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-10-08T01:08:37.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.824+0000 7fc06d19d700 1 Processor -- start 2023-10-08T01:08:37.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.824+0000 7fc06d19d700 1 -- start start 2023-10-08T01:08:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.825+0000 7fc06d19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.825+0000 7fc06d19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc0680b0790 0x7fc0680b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.825+0000 7fc06d19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0681591e0 0x7fc068159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.825+0000 7fc06d19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680cfa00 con 0x7fc0681591e0 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.825+0000 7fc06d19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680cb580 con 0x7fc0680b0790 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.825+0000 7fc06d19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b2de0 con 0x7fc0680b03c0 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.825+0000 7fc06c99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0681591e0 0x7fc068159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc06c99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0681591e0 0x7fc068159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41982/0 (socket says 172.21.15.119:41982) 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc06c99c700 1 -- 172.21.15.119:0/18833515 learned_addr learned my addr 172.21.15.119:0/18833515 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc067fff700 1 --2- 172.21.15.119:0/18833515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc0677fe700 1 --2- 172.21.15.119:0/18833515 >> v2:172.21.15.184:3300/0 conn(0x7fc0680b0790 0x7fc0680b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:37.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc06c99c700 1 -- 172.21.15.119:0/18833515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 msgr2=0x7fc0680cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc06c99c700 1 --2- 172.21.15.119:0/18833515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc06c99c700 1 -- 172.21.15.119:0/18833515 >> v2:172.21.15.184:3300/0 conn(0x7fc0680b0790 msgr2=0x7fc0680b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc06c99c700 1 --2- 172.21.15.119:0/18833515 >> v2:172.21.15.184:3300/0 conn(0x7fc0680b0790 0x7fc0680b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc06c99c700 1 -- 172.21.15.119:0/18833515 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680cf160 con 0x7fc0681591e0 2023-10-08T01:08:37.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc06c99c700 1 --2- 172.21.15.119:0/18833515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc068159620 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc068067fa0 tx=0x7fc060009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd55eec6342e55b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.826+0000 7fc066ffd700 1 -- 172.21.15.119:0/18833515 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06000e070 con 0x7fc0681591e0 2023-10-08T01:08:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc066ffd700 1 -- 172.21.15.119:0/18833515 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06000ba50 con 0x7fc0681591e0 2023-10-08T01:08:37.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc066ffd700 1 -- 172.21.15.119:0/18833515 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600092e0 con 0x7fc0681591e0 2023-10-08T01:08:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 -- 172.21.15.119:0/18833515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 msgr2=0x7fc068159620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 --2- 172.21.15.119:0/18833515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc068159620 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc068067fa0 tx=0x7fc060009840 comp rx=0 tx=0).stop 2023-10-08T01:08:37.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 -- 172.21.15.119:0/18833515 shutdown_connections 2023-10-08T01:08:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 --2- 172.21.15.119:0/18833515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc068159620 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 --2- 172.21.15.119:0/18833515 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 --2- 172.21.15.119:0/18833515 >> v2:172.21.15.184:3300/0 conn(0x7fc0680b0790 0x7fc0680b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 -- 172.21.15.119:0/18833515 >> 172.21.15.119:0/18833515 conn(0x7fc068063230 msgr2=0x7fc068063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:37.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 -- 172.21.15.119:0/18833515 shutdown_connections 2023-10-08T01:08:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.827+0000 7fc06d19d700 1 -- 172.21.15.119:0/18833515 wait complete. 2023-10-08T01:08:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.829+0000 7fc06d19d700 1 Processor -- start 2023-10-08T01:08:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.829+0000 7fc06d19d700 1 -- start start 2023-10-08T01:08:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06d19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:37.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06d19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0680b0790 0x7fc0681576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06d19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc0681580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06d19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068169e70 con 0x7fc0681591e0 2023-10-08T01:08:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06d19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06816a140 con 0x7fc0680b0790 2023-10-08T01:08:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06d19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc068157c20 con 0x7fc0680b03c0 2023-10-08T01:08:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06c99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc0681580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06c99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc0681580a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41990/0 (socket says 172.21.15.119:41990) 2023-10-08T01:08:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06c99c700 1 -- 172.21.15.119:0/3036616342 learned_addr learned my addr 172.21.15.119:0/3036616342 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:37.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc0677fe700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0680b0790 0x7fc0681576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc067fff700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06c99c700 1 -- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 msgr2=0x7fc0680c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06c99c700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06c99c700 1 -- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0680b0790 msgr2=0x7fc0681576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06c99c700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0680b0790 0x7fc0681576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc06c99c700 1 -- 172.21.15.119:0/3036616342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc060009ae0 con 0x7fc0681591e0 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc0677fe700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0680b0790 0x7fc0681576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.830+0000 7fc067fff700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.831+0000 7fc06c99c700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc0681580a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc060009fd0 tx=0x7fc060018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.831+0000 7fc04ffff700 1 -- 172.21.15.119:0/3036616342 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0600094e0 con 0x7fc0681591e0 2023-10-08T01:08:37.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.831+0000 7fc04ffff700 1 -- 172.21.15.119:0/3036616342 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0600032f0 con 0x7fc0681591e0 2023-10-08T01:08:37.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.831+0000 7fc04ffff700 1 -- 172.21.15.119:0/3036616342 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060026690 con 0x7fc0681591e0 2023-10-08T01:08:37.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.831+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0680c8d20 con 0x7fc0681591e0 2023-10-08T01:08:37.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.831+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0680c4940 con 0x7fc0681591e0 2023-10-08T01:08:37.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.832+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06804c3c0 con 0x7fc0681591e0 2023-10-08T01:08:37.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.833+0000 7fc04ffff700 1 -- 172.21.15.119:0/3036616342 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fc060003040 con 0x7fc0681591e0 2023-10-08T01:08:37.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.833+0000 7fc04ffff700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc054067130 0x7fc0540695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:37.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.834+0000 7fc04ffff700 1 -- 172.21.15.119:0/3036616342 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6755+0+0 (secure 0 0 0) 0x7fc060086e50 con 0x7fc0681591e0 2023-10-08T01:08:37.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.834+0000 7fc067fff700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc054067130 0x7fc0540695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:37.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.834+0000 7fc067fff700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc054067130 0x7fc0540695e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc058009990 tx=0x7fc058008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:37.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:37.838+0000 7fc04ffff700 1 -- 172.21.15.119:0/3036616342 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc060053870 con 0x7fc0681591e0 2023-10-08T01:08:38.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.115+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc06816a280 con 0x7fc054067130 2023-10-08T01:08:38.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.210+0000 7fc04ffff700 1 -- 172.21.15.119:0/3036616342 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc06816a280 con 0x7fc054067130 2023-10-08T01:08:38.212 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-10-08T01:08:38.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc054067130 msgr2=0x7fc0540695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:38.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc054067130 0x7fc0540695e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc058009990 tx=0x7fc058008040 comp rx=0 tx=0).stop 2023-10-08T01:08:38.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 msgr2=0x7fc0681580a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:38.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc0681580a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc060009fd0 tx=0x7fc060018000 comp rx=0 tx=0).stop 2023-10-08T01:08:38.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 shutdown_connections 2023-10-08T01:08:38.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc0680b03c0 0x7fc0680c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:38.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc054067130 0x7fc0540695e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:38.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0681591e0 0x7fc0681580a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:38.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 --2- 172.21.15.119:0/3036616342 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc0680b0790 0x7fc0681576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:38.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 >> 172.21.15.119:0/3036616342 conn(0x7fc068063230 msgr2=0x7fc0680cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:38.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 shutdown_connections 2023-10-08T01:08:38.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:38.213+0000 7fc06d19d700 1 -- 172.21.15.119:0/3036616342 wait complete. 2023-10-08T01:08:38.231 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-10-08T01:08:38.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.232 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-10-08T01:08:38.281 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:38.282 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:38.282 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:38.282 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:38.282 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:38.282 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:38.283 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:38.283 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:38.283 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:38.283 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:38.283 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:38.283 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:38.284 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:38.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.285 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-00_57_00_UTC) 2023-10-08T01:08:38.337 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:38.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.337 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:38.380 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:38.380 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.380 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:38.465 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:38.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.466 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:38.549 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:38.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.549 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:38.632 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:38.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.632 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:38.714 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:38.715 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.715 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:38.798 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:38.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.798 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:38.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:38.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.881 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:38.964 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:38.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:38.964 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:39.048 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:39.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:39.048 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:39.131 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:39.131 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:39.131 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:39.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:39.229 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-10-08T01:08:39.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-10-08T01:08:39.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.513+0000 7f79c359e700 1 Processor -- start 2023-10-08T01:08:39.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.513+0000 7f79c359e700 1 -- start start 2023-10-08T01:08:39.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.514+0000 7f79c359e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79c4156350 0x7f79c4157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:39.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.514+0000 7f79c359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0c00 0x7f79c415a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:39.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.514+0000 7f79c359e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79c40b0570 con 0x7f79c40b0760 2023-10-08T01:08:39.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.514+0000 7f79c359e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79c41578b0 con 0x7f79c4156350 2023-10-08T01:08:39.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.514+0000 7f79c359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40af4d0 con 0x7f79c40b0c00 2023-10-08T01:08:39.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.514+0000 7f79c259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79c4156350 0x7f79c4157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.514+0000 7f79c259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79c4156350 0x7f79c4157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42002/0 (socket says 172.21.15.119:42002) 2023-10-08T01:08:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.514+0000 7f79c259c700 1 -- 172.21.15.119:0/1104044523 learned_addr learned my addr 172.21.15.119:0/1104044523 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c259c700 1 -- 172.21.15.119:0/1104044523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0c00 msgr2=0x7f79c415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c2d9d700 1 --2- 172.21.15.119:0/1104044523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0c00 0x7f79c415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c259c700 1 --2- 172.21.15.119:0/1104044523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0c00 0x7f79c415a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c259c700 1 -- 172.21.15.119:0/1104044523 >> v1:172.21.15.119:6789/0 conn(0x7f79c40b0760 legacy=0x7f79c4159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c159a700 1 -- 172.21.15.119:0/1104044523 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1093874356 0 0) 0x7f79c40b0570 con 0x7f79c40b0760 2023-10-08T01:08:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c259c700 1 -- 172.21.15.119:0/1104044523 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c40af7b0 con 0x7f79c4156350 2023-10-08T01:08:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c2d9d700 1 --2- 172.21.15.119:0/1104044523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0c00 0x7f79c415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c259c700 1 --2- 172.21.15.119:0/1104044523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c4157370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f79b400d1a0 tx=0x7f79b400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4149c8f3a4d2d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c159a700 1 -- 172.21.15.119:0/1104044523 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b4010070 con 0x7f79c4156350 2023-10-08T01:08:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c159a700 1 -- 172.21.15.119:0/1104044523 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b400b0d0 con 0x7f79c4156350 2023-10-08T01:08:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.515+0000 7f79c159a700 1 -- 172.21.15.119:0/1104044523 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b400b490 con 0x7f79c4156350 2023-10-08T01:08:39.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.516+0000 7f79c359e700 1 -- 172.21.15.119:0/1104044523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 msgr2=0x7f79c4157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.516+0000 7f79c359e700 1 --2- 172.21.15.119:0/1104044523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c4157370 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f79b400d1a0 tx=0x7f79b400ba50 comp rx=0 tx=0).stop 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.516+0000 7f79c359e700 1 -- 172.21.15.119:0/1104044523 shutdown_connections 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.516+0000 7f79c359e700 1 --2- 172.21.15.119:0/1104044523 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0c00 0x7f79c415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.516+0000 7f79c359e700 1 --2- 172.21.15.119:0/1104044523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c4157370 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.516+0000 7f79c359e700 1 -- 172.21.15.119:0/1104044523 >> 172.21.15.119:0/1104044523 conn(0x7f79c4068e20 msgr2=0x7f79c40cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.516+0000 7f79c359e700 1 -- 172.21.15.119:0/1104044523 shutdown_connections 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.516+0000 7f79c359e700 1 -- 172.21.15.119:0/1104044523 wait complete. 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.517+0000 7f79c359e700 1 Processor -- start 2023-10-08T01:08:39.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.517+0000 7f79c359e700 1 -- start start 2023-10-08T01:08:39.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.518+0000 7f79c359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0760 0x7f79c40ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:39.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.518+0000 7f79c359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c40b0c00 0x7f79c40ac880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:39.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.518+0000 7f79c359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c40c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:39.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.518+0000 7f79c359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40acf60 con 0x7f79c4156350 2023-10-08T01:08:39.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.518+0000 7f79c359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40ad240 con 0x7f79c40b0c00 2023-10-08T01:08:39.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.518+0000 7f79c359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c40ad520 con 0x7f79c40b0760 2023-10-08T01:08:39.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c40c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:39.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c40c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42016/0 (socket says 172.21.15.119:42016) 2023-10-08T01:08:39.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 -- 172.21.15.119:0/2019927981 learned_addr learned my addr 172.21.15.119:0/2019927981 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:39.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c259c700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0760 0x7f79c40ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:39.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 -- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0760 msgr2=0x7f79c40ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:39.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0760 0x7f79c40ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 -- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c40b0c00 msgr2=0x7f79c40ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c1d9b700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c40b0c00 0x7f79c40ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c40b0c00 0x7f79c40ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 -- 172.21.15.119:0/2019927981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b400bcf0 con 0x7f79c4156350 2023-10-08T01:08:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c259c700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0760 0x7f79c40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c1d9b700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c40b0c00 0x7f79c40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.519+0000 7f79c2d9d700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c40c99f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c850 tx=0x7f79bc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.520+0000 7f79b27fc700 1 -- 172.21.15.119:0/2019927981 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc011600 con 0x7f79c4156350 2023-10-08T01:08:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.520+0000 7f79b27fc700 1 -- 172.21.15.119:0/2019927981 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc011760 con 0x7f79c4156350 2023-10-08T01:08:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.520+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79c40ad800 con 0x7f79c4156350 2023-10-08T01:08:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.520+0000 7f79b27fc700 1 -- 172.21.15.119:0/2019927981 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc011b20 con 0x7f79c4156350 2023-10-08T01:08:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.520+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79c4150b90 con 0x7f79c4156350 2023-10-08T01:08:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.521+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79c406a360 con 0x7f79c4156350 2023-10-08T01:08:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.522+0000 7f79b27fc700 1 -- 172.21.15.119:0/2019927981 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f79bc018350 con 0x7f79c4156350 2023-10-08T01:08:39.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.522+0000 7f79b27fc700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f79a4067130 0x7f79a40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:39.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.522+0000 7f79b27fc700 1 -- 172.21.15.119:0/2019927981 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f79bc084d70 con 0x7f79c4156350 2023-10-08T01:08:39.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.523+0000 7f79c259c700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f79a4067130 0x7f79a40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:39.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.523+0000 7f79c259c700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f79a4067130 0x7f79a40695e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79b400ba90 tx=0x7f79b401a000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:39.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.527+0000 7f79b27fc700 1 -- 172.21.15.119:0/2019927981 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79bc055330 con 0x7f79c4156350 2023-10-08T01:08:39.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.746+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f79c404c3c0 con 0x7f79a4067130 2023-10-08T01:08:39.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.822+0000 7f79b27fc700 1 -- 172.21.15.119:0/2019927981 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f79c404c3c0 con 0x7f79a4067130 2023-10-08T01:08:39.823 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-10-08T01:08:39.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f79a4067130 msgr2=0x7f79a40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f79a4067130 0x7f79a40695e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f79b400ba90 tx=0x7f79b401a000 comp rx=0 tx=0).stop 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 msgr2=0x7f79c40c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c40c99f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c850 tx=0x7f79bc008040 comp rx=0 tx=0).stop 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 shutdown_connections 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79c40b0760 0x7f79c40ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f79a4067130 0x7f79a40695e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79c4156350 0x7f79c40c99f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.824+0000 7f79c359e700 1 --2- 172.21.15.119:0/2019927981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79c40b0c00 0x7f79c40ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.825+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 >> 172.21.15.119:0/2019927981 conn(0x7f79c4068e20 msgr2=0x7f79c415f920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:39.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.825+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 shutdown_connections 2023-10-08T01:08:39.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:39.825+0000 7f79c359e700 1 -- 172.21.15.119:0/2019927981 wait complete. 2023-10-08T01:08:39.843 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-10-08T01:08:39.843 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:39.843 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:39.881 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:39.882 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:39.882 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:39.882 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:39.883 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:39.883 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:39.883 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-00_57_00_UTC) 2023-10-08T01:08:39.969 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:39.969 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:39.970 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:40.053 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:40.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.053 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:40.136 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:40.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.136 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:40.258 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:40.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.258 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:40.318 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:40.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.319 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:40.362 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:40.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.363 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:40.446 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:40.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.447 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:40.534 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:40.534 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.534 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:40.624 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:40.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.624 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:40.711 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:40.712 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.712 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:40.798 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:40.798 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.798 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:40.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:40.885 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-10-08T01:08:40.965 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-10-08T01:08:41.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.198+0000 7f4f33fff700 1 Processor -- start 2023-10-08T01:08:41.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.198+0000 7f4f33fff700 1 -- start start 2023-10-08T01:08:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.199+0000 7f4f33fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f340cc8d0 0x7f4f3415b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.199+0000 7f4f33fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340ada30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:41.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.199+0000 7f4f33fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f34157a50 con 0x7f4f3415b050 2023-10-08T01:08:41.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.199+0000 7f4f33fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f34158f10 con 0x7f4f340cc8d0 2023-10-08T01:08:41.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.199+0000 7f4f33fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f341591b0 con 0x7f4f3415b960 2023-10-08T01:08:41.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.199+0000 7f4f327fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f340cc8d0 0x7f4f3415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:41.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f327fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f340cc8d0 0x7f4f3415b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42028/0 (socket says 172.21.15.119:42028) 2023-10-08T01:08:41.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f327fc700 1 -- 172.21.15.119:0/918125968 learned_addr learned my addr 172.21.15.119:0/918125968 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:41.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f337fe700 1 --2- 172.21.15.119:0/918125968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:41.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f327fc700 1 -- 172.21.15.119:0/918125968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 msgr2=0x7f4f340ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:41.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f327fc700 1 --2- 172.21.15.119:0/918125968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340ada30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f327fc700 1 -- 172.21.15.119:0/918125968 >> v1:172.21.15.184:6789/0 conn(0x7f4f3415b050 legacy=0x7f4f340ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:41.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f327fc700 1 -- 172.21.15.119:0/918125968 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f340ca9e0 con 0x7f4f340cc8d0 2023-10-08T01:08:41.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f327fc700 1 --2- 172.21.15.119:0/918125968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f3415b420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4f280099b0 tx=0x7f4f28009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef2b0eb62b06a71 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:41.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.200+0000 7f4f31ffb700 1 -- 172.21.15.119:0/918125968 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f28009ea0 con 0x7f4f340cc8d0 2023-10-08T01:08:41.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f31ffb700 1 -- 172.21.15.119:0/918125968 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f2800b860 con 0x7f4f340cc8d0 2023-10-08T01:08:41.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f31ffb700 1 -- 172.21.15.119:0/918125968 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2800bc00 con 0x7f4f340cc8d0 2023-10-08T01:08:41.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f33fff700 1 -- 172.21.15.119:0/918125968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 msgr2=0x7f4f3415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:41.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f33fff700 1 --2- 172.21.15.119:0/918125968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f3415b420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4f280099b0 tx=0x7f4f28009250 comp rx=0 tx=0).stop 2023-10-08T01:08:41.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f33fff700 1 -- 172.21.15.119:0/918125968 shutdown_connections 2023-10-08T01:08:41.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f33fff700 1 --2- 172.21.15.119:0/918125968 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f33fff700 1 --2- 172.21.15.119:0/918125968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f3415b420 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f33fff700 1 -- 172.21.15.119:0/918125968 >> 172.21.15.119:0/918125968 conn(0x7f4f34068e20 msgr2=0x7f4f340cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:41.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f33fff700 1 -- 172.21.15.119:0/918125968 shutdown_connections 2023-10-08T01:08:41.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.201+0000 7f4f33fff700 1 -- 172.21.15.119:0/918125968 wait complete. 2023-10-08T01:08:41.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.203+0000 7f4f33fff700 1 Processor -- start 2023-10-08T01:08:41.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.203+0000 7f4f33fff700 1 -- start start 2023-10-08T01:08:41.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f34141570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:41.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f33fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f3415b050 0x7f4f34143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:41.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f33fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:41.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f33fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f341521c0 con 0x7f4f340cc8d0 2023-10-08T01:08:41.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f33fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f341444d0 con 0x7f4f3415b050 2023-10-08T01:08:41.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f33fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f340c6160 con 0x7f4f3415b960 2023-10-08T01:08:41.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f32ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f34141570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:41.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f32ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f34141570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42038/0 (socket says 172.21.15.119:42038) 2023-10-08T01:08:41.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f32ffd700 1 -- 172.21.15.119:0/3525651892 learned_addr learned my addr 172.21.15.119:0/3525651892 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:41.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f327fc700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f3415b050 0x7f4f34143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:41.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f337fe700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:41.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f32ffd700 1 -- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 msgr2=0x7f4f340c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:41.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f32ffd700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f32ffd700 1 -- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f3415b050 msgr2=0x7f4f34143f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:41.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f32ffd700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f3415b050 0x7f4f34143f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f32ffd700 1 -- 172.21.15.119:0/3525651892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f280094f0 con 0x7f4f340cc8d0 2023-10-08T01:08:41.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f337fe700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:41.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.204+0000 7f4f327fc700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f3415b050 0x7f4f34143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:41.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.205+0000 7f4f32ffd700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f34141570 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c4e0 tx=0x7f4f2400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:41.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.205+0000 7f4f1affd700 1 -- 172.21.15.119:0/3525651892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24011590 con 0x7f4f340cc8d0 2023-10-08T01:08:41.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.205+0000 7f4f1affd700 1 -- 172.21.15.119:0/3525651892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f24011710 con 0x7f4f340cc8d0 2023-10-08T01:08:41.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.205+0000 7f4f1affd700 1 -- 172.21.15.119:0/3525651892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24011ad0 con 0x7f4f340cc8d0 2023-10-08T01:08:41.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.205+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f340c6440 con 0x7f4f340cc8d0 2023-10-08T01:08:41.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.205+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f340c6a00 con 0x7f4f340cc8d0 2023-10-08T01:08:41.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.206+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3404c3c0 con 0x7f4f340cc8d0 2023-10-08T01:08:41.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.207+0000 7f4f1affd700 1 -- 172.21.15.119:0/3525651892 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f4f2401b3b0 con 0x7f4f340cc8d0 2023-10-08T01:08:41.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.208+0000 7f4f1affd700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4f20067130 0x7f4f200695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:41.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.208+0000 7f4f1affd700 1 -- 172.21.15.119:0/3525651892 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6435+0+0 (secure 0 0 0) 0x7f4f240840a0 con 0x7f4f340cc8d0 2023-10-08T01:08:41.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.208+0000 7f4f327fc700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4f20067130 0x7f4f200695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:41.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.209+0000 7f4f327fc700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4f20067130 0x7f4f200695e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4f280099b0 tx=0x7f4f28017000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:41.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.212+0000 7f4f1affd700 1 -- 172.21.15.119:0/3525651892 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f24050d10 con 0x7f4f340cc8d0 2023-10-08T01:08:41.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.457+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f34154dc0 con 0x7f4f20067130 2023-10-08T01:08:41.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.529+0000 7f4f1affd700 1 -- 172.21.15.119:0/3525651892 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f4f34154dc0 con 0x7f4f20067130 2023-10-08T01:08:41.530 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4f20067130 msgr2=0x7f4f200695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4f20067130 0x7f4f200695e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4f280099b0 tx=0x7f4f28017000 comp rx=0 tx=0).stop 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 msgr2=0x7f4f34141570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f34141570 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c4e0 tx=0x7f4f2400c8a0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 shutdown_connections 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f3415b960 0x7f4f340c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f4f20067130 0x7f4f200695e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f340cc8d0 0x7f4f34141570 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 --2- 172.21.15.119:0/3525651892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f3415b050 0x7f4f34143f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:41.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.531+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 >> 172.21.15.119:0/3525651892 conn(0x7f4f34068e20 msgr2=0x7f4f340b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:41.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.532+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 shutdown_connections 2023-10-08T01:08:41.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:41.532+0000 7f4f33fff700 1 -- 172.21.15.119:0/3525651892 wait complete. 2023-10-08T01:08:41.550 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-10-08T01:08:41.550 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:41.550 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-10-08T01:08:41.588 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:41.588 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:41.588 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:41.588 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:41.588 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:41.588 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:41.588 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:41.588 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:41.589 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:41.589 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:41.589 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:41.589 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:41.590 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-00_57_00_UTC 2023-10-08T01:08:41.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:41.590 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-00_57_00_UTC) 2023-10-08T01:08:41.674 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:41.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:41.674 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:41.757 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:41.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:41.758 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:41.841 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:41.841 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:41.841 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:41.925 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:41.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:41.925 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:42.008 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:42.008 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:42.008 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:42.093 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:42.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:42.093 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:42.188 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:42.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:42.188 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:42.246 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:42.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:42.246 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:42.312 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:42.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:42.313 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:42.370 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:42.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:42.370 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:42.414 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:42.414 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:42.414 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:42.497 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:42.497 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-10-08T01:08:42.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-10-08T01:08:42.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.803+0000 7f21aefff700 1 Processor -- start 2023-10-08T01:08:42.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.803+0000 7f21aefff700 1 -- start start 2023-10-08T01:08:42.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.803+0000 7f21aefff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f21a81597b0 0x7f21a80af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:42.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.803+0000 7f21aefff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a8159b80 0x7f21a806a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:42.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.803+0000 7f21aefff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21a806a7f0 0x7f21a8068d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:42.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.803+0000 7f21aefff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a8068390 con 0x7f21a806a7f0 2023-10-08T01:08:42.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.803+0000 7f21aefff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a80ac530 con 0x7f21a81597b0 2023-10-08T01:08:42.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.803+0000 7f21aefff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a806ac30 con 0x7f21a8159b80 2023-10-08T01:08:42.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21a806a7f0 0x7f21a8068d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:42.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21a806a7f0 0x7f21a8068d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42052/0 (socket says 172.21.15.119:42052) 2023-10-08T01:08:42.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 -- 172.21.15.119:0/2599458156 learned_addr learned my addr 172.21.15.119:0/2599458156 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:42.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21acd9b700 1 --2- 172.21.15.119:0/2599458156 >> v2:172.21.15.184:3300/0 conn(0x7f21a81597b0 0x7f21a80af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:42.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21a7fff700 1 --2- 172.21.15.119:0/2599458156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a8159b80 0x7f21a806a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:42.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 -- 172.21.15.119:0/2599458156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a8159b80 msgr2=0x7f21a806a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:42.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 --2- 172.21.15.119:0/2599458156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a8159b80 0x7f21a806a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 -- 172.21.15.119:0/2599458156 >> v2:172.21.15.184:3300/0 conn(0x7f21a81597b0 msgr2=0x7f21a80af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 --2- 172.21.15.119:0/2599458156 >> v2:172.21.15.184:3300/0 conn(0x7f21a81597b0 0x7f21a80af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 -- 172.21.15.119:0/2599458156 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a80aa680 con 0x7f21a806a7f0 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21a7fff700 1 --2- 172.21.15.119:0/2599458156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a8159b80 0x7f21a806a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.804+0000 7f21ad59c700 1 --2- 172.21.15.119:0/2599458156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806a7f0 0x7f21a8068d70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f21a806db50 tx=0x7f21a0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4c0aa0b56f4d8c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21a77fe700 1 -- 172.21.15.119:0/2599458156 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000e070 con 0x7f21a806a7f0 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21a77fe700 1 -- 172.21.15.119:0/2599458156 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a000b990 con 0x7f21a806a7f0 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21a77fe700 1 -- 172.21.15.119:0/2599458156 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a00092e0 con 0x7f21a806a7f0 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 -- 172.21.15.119:0/2599458156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806a7f0 msgr2=0x7f21a8068d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 --2- 172.21.15.119:0/2599458156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806a7f0 0x7f21a8068d70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f21a806db50 tx=0x7f21a0009840 comp rx=0 tx=0).stop 2023-10-08T01:08:42.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 -- 172.21.15.119:0/2599458156 shutdown_connections 2023-10-08T01:08:42.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 --2- 172.21.15.119:0/2599458156 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a806a7f0 0x7f21a8068d70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:42.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 --2- 172.21.15.119:0/2599458156 >> v2:172.21.15.184:3300/0 conn(0x7f21a81597b0 0x7f21a80af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:42.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 --2- 172.21.15.119:0/2599458156 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a8159b80 0x7f21a806a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:42.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 -- 172.21.15.119:0/2599458156 >> 172.21.15.119:0/2599458156 conn(0x7f21a80cd880 msgr2=0x7f21a80ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:42.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 -- 172.21.15.119:0/2599458156 shutdown_connections 2023-10-08T01:08:42.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.805+0000 7f21aefff700 1 -- 172.21.15.119:0/2599458156 wait complete. 2023-10-08T01:08:42.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.807+0000 7f21aefff700 1 Processor -- start 2023-10-08T01:08:42.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.807+0000 7f21aefff700 1 -- start start 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21aefff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a806a7f0 0x7f21a815c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21aefff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a81597b0 0x7f21a815d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21aefff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a8159b80 0x7f21a815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21aefff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a816aef0 con 0x7f21a81597b0 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21aefff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a816a2e0 con 0x7f21a8159b80 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21aefff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a816a5c0 con 0x7f21a806a7f0 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a81597b0 0x7f21a815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a81597b0 0x7f21a815d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42062/0 (socket says 172.21.15.119:42062) 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21a7fff700 1 -- 172.21.15.119:0/349625625 learned_addr learned my addr 172.21.15.119:0/349625625 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21a7fff700 1 -- 172.21.15.119:0/349625625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a806a7f0 msgr2=0x7f21a815c3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21a7fff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a806a7f0 0x7f21a815c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:42.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21a7fff700 1 -- 172.21.15.119:0/349625625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a8159b80 msgr2=0x7f21a815dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:42.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21a7fff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a8159b80 0x7f21a815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:42.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.808+0000 7f21a7fff700 1 -- 172.21.15.119:0/349625625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a0009ae0 con 0x7f21a81597b0 2023-10-08T01:08:42.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.809+0000 7f21a7fff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a81597b0 0x7f21a815d590 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f219400d860 tx=0x7f2194008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:42.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.809+0000 7f21a4ff9700 1 -- 172.21.15.119:0/349625625 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21940125f0 con 0x7f21a81597b0 2023-10-08T01:08:42.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.809+0000 7f21a4ff9700 1 -- 172.21.15.119:0/349625625 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194012750 con 0x7f21a81597b0 2023-10-08T01:08:42.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.809+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a816a8a0 con 0x7f21a81597b0 2023-10-08T01:08:42.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.809+0000 7f21a4ff9700 1 -- 172.21.15.119:0/349625625 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194012ad0 con 0x7f21a81597b0 2023-10-08T01:08:42.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.809+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a80c31f0 con 0x7f21a81597b0 2023-10-08T01:08:42.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.810+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21a80ac920 con 0x7f21a81597b0 2023-10-08T01:08:42.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.811+0000 7f21a4ff9700 1 -- 172.21.15.119:0/349625625 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f2194014430 con 0x7f21a81597b0 2023-10-08T01:08:42.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.812+0000 7f21a4ff9700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f218c067130 0x7f218c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:42.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.812+0000 7f21a4ff9700 1 -- 172.21.15.119:0/349625625 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f21940850f0 con 0x7f21a81597b0 2023-10-08T01:08:42.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.812+0000 7f21acd9b700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f218c067130 0x7f218c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:42.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.813+0000 7f21acd9b700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f218c067130 0x7f218c0695e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21a816d0c0 tx=0x7f219c0091a0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:42.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:42.816+0000 7f21a4ff9700 1 -- 172.21.15.119:0/349625625 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21940557b0 con 0x7f21a81597b0 2023-10-08T01:08:43.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.032+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f21a816b030 con 0x7f218c067130 2023-10-08T01:08:43.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.113+0000 7f21a4ff9700 1 -- 172.21.15.119:0/349625625 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f21a816b030 con 0x7f218c067130 2023-10-08T01:08:43.115 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-10-08T01:08:43.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f218c067130 msgr2=0x7f218c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:43.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f218c067130 0x7f218c0695e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f21a816d0c0 tx=0x7f219c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a81597b0 msgr2=0x7f21a815d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a81597b0 0x7f21a815d590 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f219400d860 tx=0x7f2194008040 comp rx=0 tx=0).stop 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 shutdown_connections 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f21a806a7f0 0x7f21a815c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f218c067130 0x7f218c0695e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f21a8159b80 0x7f21a815dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 --2- 172.21.15.119:0/349625625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21a81597b0 0x7f21a815d590 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 >> 172.21.15.119:0/349625625 conn(0x7f21a80cd880 msgr2=0x7f21a81585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:43.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 shutdown_connections 2023-10-08T01:08:43.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:43.116+0000 7f21aefff700 1 -- 172.21.15.119:0/349625625 wait complete. 2023-10-08T01:08:43.141 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-10-08T01:08:43.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.142 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-10-08T01:08:43.198 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:43.198 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:43.199 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:43.200 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:43.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.201 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:43.258 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:43.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.259 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:43.315 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:43.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.316 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:43.364 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:43.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.365 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:43.408 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:43.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.409 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:43.493 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:43.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.493 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:43.578 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:43.578 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.579 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:43.662 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:43.663 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.663 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:43.747 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:43.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.747 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:43.832 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:43.832 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.832 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:43.916 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:43.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:43.916 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:43.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:44.000 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-10-08T01:08:44.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-10-08T01:08:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.330+0000 7f6798fff700 1 Processor -- start 2023-10-08T01:08:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.331+0000 7f6798fff700 1 -- start start 2023-10-08T01:08:44.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6798fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f67941563b0 0x7f6794156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6798fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f679415a440 0x7f67940acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6798fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679415a9b0 con 0x7f6794068890 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6798fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679415ac90 con 0x7f679415a440 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6798fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679415af70 con 0x7f67941563b0 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6791d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f679415a440 0x7f67940acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6792d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6794068890 0x7f67940c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43486/0 (socket says 172.21.15.119:43486) 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6791d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f679415a440 0x7f67940acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42074/0 (socket says 172.21.15.119:42074) 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6792d9d700 1 -- 172.21.15.119:0/1674414767 learned_addr learned my addr 172.21.15.119:0/1674414767 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:44.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f679259c700 1 --2- 172.21.15.119:0/1674414767 >> v2:172.21.15.184:3300/0 conn(0x7f67941563b0 0x7f6794156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6791d9b700 1 -- 172.21.15.119:0/1674414767 >> v2:172.21.15.184:3300/0 conn(0x7f67941563b0 msgr2=0x7f6794156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6791d9b700 1 --2- 172.21.15.119:0/1674414767 >> v2:172.21.15.184:3300/0 conn(0x7f67941563b0 0x7f6794156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6791d9b700 1 -- 172.21.15.119:0/1674414767 >> v1:172.21.15.119:6789/0 conn(0x7f6794068890 legacy=0x7f67940c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f6791d9b700 1 -- 172.21.15.119:0/1674414767 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679415b250 con 0x7f679415a440 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.332+0000 7f679159a700 1 -- 172.21.15.119:0/1674414767 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687792018 0 0) 0x7f679415a9b0 con 0x7f6794068890 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.333+0000 7f6791d9b700 1 --2- 172.21.15.119:0/1674414767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679415a440 0x7f67940acf30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f679406a9f0 tx=0x7f67880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7725d90ccf3ac66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.333+0000 7f679159a700 1 -- 172.21.15.119:0/1674414767 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678800b6b0 con 0x7f679415a440 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.333+0000 7f679159a700 1 -- 172.21.15.119:0/1674414767 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678800b810 con 0x7f679415a440 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.333+0000 7f679159a700 1 -- 172.21.15.119:0/1674414767 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678800bbb0 con 0x7f679415a440 2023-10-08T01:08:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.333+0000 7f6798fff700 1 -- 172.21.15.119:0/1674414767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679415a440 msgr2=0x7f67940acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:44.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.333+0000 7f6798fff700 1 --2- 172.21.15.119:0/1674414767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679415a440 0x7f67940acf30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f679406a9f0 tx=0x7f67880091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.334+0000 7f6798fff700 1 -- 172.21.15.119:0/1674414767 shutdown_connections 2023-10-08T01:08:44.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.334+0000 7f6798fff700 1 --2- 172.21.15.119:0/1674414767 >> v2:172.21.15.184:3300/0 conn(0x7f67941563b0 0x7f6794156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.334+0000 7f6798fff700 1 --2- 172.21.15.119:0/1674414767 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f679415a440 0x7f67940acf30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.334+0000 7f6798fff700 1 -- 172.21.15.119:0/1674414767 >> 172.21.15.119:0/1674414767 conn(0x7f67940c8f90 msgr2=0x7f67940c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:44.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.334+0000 7f6798fff700 1 -- 172.21.15.119:0/1674414767 shutdown_connections 2023-10-08T01:08:44.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.334+0000 7f6798fff700 1 -- 172.21.15.119:0/1674414767 wait complete. 2023-10-08T01:08:44.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.335+0000 7f6798fff700 1 Processor -- start 2023-10-08T01:08:44.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.335+0000 7f6798fff700 1 -- start start 2023-10-08T01:08:44.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.336+0000 7f6798fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6794068890 0x7f679414a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:44.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.336+0000 7f6798fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67941563b0 0x7f67940afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:44.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.336+0000 7f6798fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f679415a440 0x7f67940cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:44.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.336+0000 7f6798fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679414ae90 con 0x7f67941563b0 2023-10-08T01:08:44.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.336+0000 7f6798fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679416df60 con 0x7f679415a440 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.336+0000 7f6798fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67940af400 con 0x7f6794068890 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.336+0000 7f6791d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67941563b0 0x7f67940afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6791d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67941563b0 0x7f67940afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42088/0 (socket says 172.21.15.119:42088) 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6791d9b700 1 -- 172.21.15.119:0/2138777419 learned_addr learned my addr 172.21.15.119:0/2138777419 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f679259c700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6794068890 0x7f679414a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6792d9d700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f679415a440 0x7f67940cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6791d9b700 1 -- 172.21.15.119:0/2138777419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6794068890 msgr2=0x7f679414a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6791d9b700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6794068890 0x7f679414a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6791d9b700 1 -- 172.21.15.119:0/2138777419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f679415a440 msgr2=0x7f67940cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6791d9b700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f679415a440 0x7f67940cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6791d9b700 1 -- 172.21.15.119:0/2138777419 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6788009420 con 0x7f67941563b0 2023-10-08T01:08:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6791d9b700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67941563b0 0x7f67940afee0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f679406bbb0 tx=0x7f6788017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:44.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f67827fc700 1 -- 172.21.15.119:0/2138777419 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788011420 con 0x7f67941563b0 2023-10-08T01:08:44.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f67827fc700 1 -- 172.21.15.119:0/2138777419 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6788011580 con 0x7f67941563b0 2023-10-08T01:08:44.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f67827fc700 1 -- 172.21.15.119:0/2138777419 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788011940 con 0x7f67941563b0 2023-10-08T01:08:44.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.337+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67940af6e0 con 0x7f67941563b0 2023-10-08T01:08:44.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.338+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67940cb9f0 con 0x7f67941563b0 2023-10-08T01:08:44.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.339+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67940718b0 con 0x7f67941563b0 2023-10-08T01:08:44.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.339+0000 7f67827fc700 1 -- 172.21.15.119:0/2138777419 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f6788011d30 con 0x7f67941563b0 2023-10-08T01:08:44.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.340+0000 7f67827fc700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6774067130 0x7f67740695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:44.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.341+0000 7f67827fc700 1 -- 172.21.15.119:0/2138777419 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6435+0+0 (secure 0 0 0) 0x7f6788085f30 con 0x7f67941563b0 2023-10-08T01:08:44.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.341+0000 7f679259c700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6774067130 0x7f67740695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:44.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.341+0000 7f679259c700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6774067130 0x7f67740695e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6784006fd0 tx=0x7f6784008040 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:44.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.345+0000 7f67827fc700 1 -- 172.21.15.119:0/2138777419 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6788052c80 con 0x7f67941563b0 2023-10-08T01:08:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.579+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f679406d6a0 con 0x7f6774067130 2023-10-08T01:08:44.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.657+0000 7f67827fc700 1 -- 172.21.15.119:0/2138777419 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f679406d6a0 con 0x7f6774067130 2023-10-08T01:08:44.657 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-10-08T01:08:44.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6774067130 msgr2=0x7f67740695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:44.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6774067130 0x7f67740695e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6784006fd0 tx=0x7f6784008040 comp rx=0 tx=0).stop 2023-10-08T01:08:44.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67941563b0 msgr2=0x7f67940afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:44.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67941563b0 0x7f67940afee0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f679406bbb0 tx=0x7f6788017000 comp rx=0 tx=0).stop 2023-10-08T01:08:44.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 shutdown_connections 2023-10-08T01:08:44.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6794068890 0x7f679414a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f6774067130 0x7f67740695e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f679415a440 0x7f67940cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 --2- 172.21.15.119:0/2138777419 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67941563b0 0x7f67940afee0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:44.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 >> 172.21.15.119:0/2138777419 conn(0x7f67940c8f90 msgr2=0x7f67940ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:44.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 shutdown_connections 2023-10-08T01:08:44.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:44.659+0000 7f6798fff700 1 -- 172.21.15.119:0/2138777419 wait complete. 2023-10-08T01:08:44.676 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-10-08T01:08:44.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:44.677 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-10-08T01:08:44.714 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:44.714 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:44.714 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:44.714 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:44.714 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:44.714 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:44.714 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:44.715 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:44.715 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:44.715 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:44.715 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:44.716 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:44.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:44.716 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:44.812 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:44.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:44.813 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:44.858 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:44.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:44.858 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:44.944 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:44.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:44.945 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:45.031 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:45.031 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:45.031 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:45.117 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:45.118 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:45.118 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:45.207 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:45.207 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:45.207 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:45.255 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:45.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:45.255 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:45.339 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:45.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:45.340 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:45.425 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:45.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:45.426 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:45.511 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:45.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:45.512 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:45.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:45.616 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-10-08T01:08:45.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-10-08T01:08:45.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.892+0000 7f2bc13ff700 1 Processor -- start 2023-10-08T01:08:45.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.892+0000 7f2bc13ff700 1 -- start start 2023-10-08T01:08:45.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.893+0000 7f2bc13ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2bbc1563b0 0x7f2bbc156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:45.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.893+0000 7f2bc13ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 0x7f2bbc0acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:45.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.893+0000 7f2bc13ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc15a9b0 con 0x7f2bbc068890 2023-10-08T01:08:45.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.893+0000 7f2bc13ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc15ac90 con 0x7f2bbc1563b0 2023-10-08T01:08:45.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.893+0000 7f2bc13ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc15af70 con 0x7f2bbc15a440 2023-10-08T01:08:45.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bbaffd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2bbc1563b0 0x7f2bbc156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:45.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bbb7fe700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f2bbc068890 0x7f2bbc0c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:59680/0 (socket says 172.21.15.119:59680) 2023-10-08T01:08:45.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bbb7fe700 1 -- 172.21.15.119:0/2516335926 learned_addr learned my addr 172.21.15.119:0/2516335926 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:45.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bba7fc700 1 --2- 172.21.15.119:0/2516335926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 0x7f2bbc0acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:45.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bb9ffb700 1 -- 172.21.15.119:0/2516335926 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613799487 0 0) 0x7f2bbc15a9b0 con 0x7f2bbc068890 2023-10-08T01:08:45.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bbaffd700 1 -- 172.21.15.119:0/2516335926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 msgr2=0x7f2bbc0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:45.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bbaffd700 1 --2- 172.21.15.119:0/2516335926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 0x7f2bbc0acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:45.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bbaffd700 1 -- 172.21.15.119:0/2516335926 >> v1:172.21.15.184:6789/0 conn(0x7f2bbc068890 legacy=0x7f2bbc0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:45.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.894+0000 7f2bbaffd700 1 -- 172.21.15.119:0/2516335926 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc15b250 con 0x7f2bbc1563b0 2023-10-08T01:08:45.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.895+0000 7f2bbaffd700 1 --2- 172.21.15.119:0/2516335926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc1563b0 0x7f2bbc156780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0098e0 tx=0x7f2bac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e5c426ddcd6f05 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:45.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.895+0000 7f2bb9ffb700 1 -- 172.21.15.119:0/2516335926 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00b640 con 0x7f2bbc1563b0 2023-10-08T01:08:45.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.895+0000 7f2bb9ffb700 1 -- 172.21.15.119:0/2516335926 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac00b7a0 con 0x7f2bbc1563b0 2023-10-08T01:08:45.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.895+0000 7f2bb9ffb700 1 -- 172.21.15.119:0/2516335926 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00bb40 con 0x7f2bbc1563b0 2023-10-08T01:08:45.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.895+0000 7f2bc13ff700 1 -- 172.21.15.119:0/2516335926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc1563b0 msgr2=0x7f2bbc156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:45.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.895+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/2516335926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc1563b0 0x7f2bbc156780 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0098e0 tx=0x7f2bac0091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:45.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.896+0000 7f2bc13ff700 1 -- 172.21.15.119:0/2516335926 shutdown_connections 2023-10-08T01:08:45.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.896+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/2516335926 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc1563b0 0x7f2bbc156780 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:45.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.896+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/2516335926 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 0x7f2bbc0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:45.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.896+0000 7f2bc13ff700 1 -- 172.21.15.119:0/2516335926 >> 172.21.15.119:0/2516335926 conn(0x7f2bbc0c8f90 msgr2=0x7f2bbc0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:45.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.896+0000 7f2bc13ff700 1 -- 172.21.15.119:0/2516335926 shutdown_connections 2023-10-08T01:08:45.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.896+0000 7f2bc13ff700 1 -- 172.21.15.119:0/2516335926 wait complete. 2023-10-08T01:08:45.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.897+0000 7f2bc13ff700 1 Processor -- start 2023-10-08T01:08:45.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.897+0000 7f2bc13ff700 1 -- start start 2023-10-08T01:08:45.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.898+0000 7f2bc13ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc068890 0x7f2bbc159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:45.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.898+0000 7f2bc13ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bbc1563b0 0x7f2bbc151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:45.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.898+0000 7f2bc13ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 0x7f2bbc06de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:45.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.898+0000 7f2bc13ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc16c2f0 con 0x7f2bbc1563b0 2023-10-08T01:08:45.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.898+0000 7f2bc13ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc16b6e0 con 0x7f2bbc068890 2023-10-08T01:08:45.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.898+0000 7f2bc13ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc16b9c0 con 0x7f2bbc15a440 2023-10-08T01:08:45.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.898+0000 7f2bbaffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc068890 0x7f2bbc159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.898+0000 7f2bba7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bbc1563b0 0x7f2bbc151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbaffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc068890 0x7f2bbc159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44578/0 (socket says 172.21.15.119:44578) 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbaffd700 1 -- 172.21.15.119:0/3923302372 learned_addr learned my addr 172.21.15.119:0/3923302372 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbb7fe700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 0x7f2bbc06de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbaffd700 1 -- 172.21.15.119:0/3923302372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 msgr2=0x7f2bbc06de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbaffd700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 0x7f2bbc06de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbaffd700 1 -- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bbc1563b0 msgr2=0x7f2bbc151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbaffd700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bbc1563b0 0x7f2bbc151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbaffd700 1 -- 172.21.15.119:0/3923302372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac009420 con 0x7f2bbc068890 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2bbaffd700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc068890 0x7f2bbc159600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc06bbb0 tx=0x7f2bac017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:45.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.899+0000 7f2ba2ffd700 1 -- 172.21.15.119:0/3923302372 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00bb20 con 0x7f2bbc068890 2023-10-08T01:08:45.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.900+0000 7f2ba2ffd700 1 -- 172.21.15.119:0/3923302372 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac000c90 con 0x7f2bbc068890 2023-10-08T01:08:45.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.900+0000 7f2ba2ffd700 1 -- 172.21.15.119:0/3923302372 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac011650 con 0x7f2bbc068890 2023-10-08T01:08:45.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.900+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc16bca0 con 0x7f2bbc068890 2023-10-08T01:08:45.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.900+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc159190 con 0x7f2bbc068890 2023-10-08T01:08:45.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.901+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc0cb250 con 0x7f2bbc068890 2023-10-08T01:08:45.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.901+0000 7f2ba2ffd700 1 -- 172.21.15.119:0/3923302372 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7f2bac011a00 con 0x7f2bbc068890 2023-10-08T01:08:45.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.902+0000 7f2ba2ffd700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ba8067130 0x7f2ba80695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:45.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.903+0000 7f2ba2ffd700 1 -- 172.21.15.119:0/3923302372 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2bac086250 con 0x7f2bbc068890 2023-10-08T01:08:45.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.903+0000 7f2bba7fc700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ba8067130 0x7f2ba80695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:45.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.903+0000 7f2bba7fc700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ba8067130 0x7f2ba80695e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009d10 tx=0x7f2bb00091a0 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:45.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:45.907+0000 7f2ba2ffd700 1 -- 172.21.15.119:0/3923302372 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bac052e50 con 0x7f2bbc068890 2023-10-08T01:08:46.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.125+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 --> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bbc0718b0 con 0x7f2ba8067130 2023-10-08T01:08:46.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.220+0000 7f2ba2ffd700 1 -- 172.21.15.119:0/3923302372 <== mgr.5142 v2:172.21.15.119:6832/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f2bbc0718b0 con 0x7f2ba8067130 2023-10-08T01:08:46.220 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ba8067130 msgr2=0x7f2ba80695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ba8067130 0x7f2ba80695e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009d10 tx=0x7f2bb00091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc068890 msgr2=0x7f2bbc159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc068890 0x7f2bbc159600 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc06bbb0 tx=0x7f2bac017000 comp rx=0 tx=0).stop 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 shutdown_connections 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bbc15a440 0x7f2bbc06de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f2ba8067130 0x7f2ba80695e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bbc068890 0x7f2bbc159600 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 --2- 172.21.15.119:0/3923302372 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bbc1563b0 0x7f2bbc151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:46.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 >> 172.21.15.119:0/3923302372 conn(0x7f2bbc0c8f90 msgr2=0x7f2bbc0ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:46.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.222+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 shutdown_connections 2023-10-08T01:08:46.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:46.223+0000 7f2bc13ff700 1 -- 172.21.15.119:0/3923302372 wait complete. 2023-10-08T01:08:46.239 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-10-08T01:08:46.240 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.240 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-10-08T01:08:46.277 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:46.277 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:46.277 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:46.277 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:46.277 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:46.278 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:46.278 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:46.278 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:46.278 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:46.278 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:46.278 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:46.279 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-00_58_00_UTC 2023-10-08T01:08:46.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.279 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-00_58_00_UTC) 2023-10-08T01:08:46.374 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-00_59_00_UTC 2023-10-08T01:08:46.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.375 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-00_59_00_UTC) 2023-10-08T01:08:46.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:08:46.420 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_00_00_UTC 2023-10-08T01:08:46.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.420 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_00_00_UTC) 2023-10-08T01:08:46.464 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:08:46.520 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_01_00_UTC 2023-10-08T01:08:46.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.520 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_01_00_UTC) 2023-10-08T01:08:46.605 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_02_00_UTC 2023-10-08T01:08:46.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.605 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_02_00_UTC) 2023-10-08T01:08:46.689 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_03_00_UTC 2023-10-08T01:08:46.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.689 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_03_00_UTC) 2023-10-08T01:08:46.773 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_04_00_UTC 2023-10-08T01:08:46.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.773 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_04_00_UTC) 2023-10-08T01:08:46.858 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_05_00_UTC 2023-10-08T01:08:46.858 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.858 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_05_00_UTC) 2023-10-08T01:08:46.943 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_06_00_UTC 2023-10-08T01:08:46.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:46.943 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_06_00_UTC) 2023-10-08T01:08:47.028 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_07_00_UTC 2023-10-08T01:08:47.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:47.028 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_07_00_UTC) 2023-10-08T01:08:47.113 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_08_00_UTC 2023-10-08T01:08:47.113 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:47.113 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-10-08-01_08_00_UTC) 2023-10-08T01:08:47.222 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:47.223 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-10-08T01:08:47.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T01:08:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.491+0000 7fe98e9ff700 1 Processor -- start 2023-10-08T01:08:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.491+0000 7fe98e9ff700 1 -- start start 2023-10-08T01:08:47.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe98e9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe98815ac80 0x7fe988069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe98e9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe98815b050 0x7fe988069cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe98e9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe9880afa60 0x7fe9880aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe98e9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9880af2f0 con 0x7fe98815ac80 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe98e9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9880af5d0 con 0x7fe9880afa60 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe98e9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880c8660 con 0x7fe98815b050 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe987fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe98815ac80 0x7fe988069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe987fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe98815ac80 0x7fe988069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48318/0 (socket says 172.21.15.119:48318) 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe987fff700 1 -- 172.21.15.119:0/1482498730 learned_addr learned my addr 172.21.15.119:0/1482498730 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe98cf9c700 1 --2- 172.21.15.119:0/1482498730 >> v2:172.21.15.184:3300/0 conn(0x7fe9880afa60 0x7fe9880aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:47.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe97f7fe700 1 --2- 172.21.15.119:0/1482498730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe98815b050 0x7fe988069cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe987fff700 1 -- 172.21.15.119:0/1482498730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe98815b050 msgr2=0x7fe988069cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe987fff700 1 --2- 172.21.15.119:0/1482498730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe98815b050 0x7fe988069cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe987fff700 1 -- 172.21.15.119:0/1482498730 >> v2:172.21.15.184:3300/0 conn(0x7fe9880afa60 msgr2=0x7fe9880aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe987fff700 1 --2- 172.21.15.119:0/1482498730 >> v2:172.21.15.184:3300/0 conn(0x7fe9880afa60 0x7fe9880aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.493+0000 7fe987fff700 1 -- 172.21.15.119:0/1482498730 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9880c8940 con 0x7fe98815ac80 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.494+0000 7fe987fff700 1 --2- 172.21.15.119:0/1482498730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815ac80 0x7fe988069770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe98806da80 tx=0x7fe978009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283dd4bbc5bc1d46 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.494+0000 7fe9877fe700 1 -- 172.21.15.119:0/1482498730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe97800e070 con 0x7fe98815ac80 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.494+0000 7fe9877fe700 1 -- 172.21.15.119:0/1482498730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe97800b970 con 0x7fe98815ac80 2023-10-08T01:08:47.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.494+0000 7fe9877fe700 1 -- 172.21.15.119:0/1482498730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9780092e0 con 0x7fe98815ac80 2023-10-08T01:08:47.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.494+0000 7fe98e9ff700 1 -- 172.21.15.119:0/1482498730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815ac80 msgr2=0x7fe988069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:47.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.494+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/1482498730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815ac80 0x7fe988069770 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fe98806da80 tx=0x7fe978009840 comp rx=0 tx=0).stop 2023-10-08T01:08:47.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.495+0000 7fe98e9ff700 1 -- 172.21.15.119:0/1482498730 shutdown_connections 2023-10-08T01:08:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.495+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/1482498730 >> v2:172.21.15.184:3300/0 conn(0x7fe9880afa60 0x7fe9880aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.495+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/1482498730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815ac80 0x7fe988069770 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.495+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/1482498730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe98815b050 0x7fe988069cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.495+0000 7fe98e9ff700 1 -- 172.21.15.119:0/1482498730 >> 172.21.15.119:0/1482498730 conn(0x7fe988068e10 msgr2=0x7fe9880cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.495+0000 7fe98e9ff700 1 -- 172.21.15.119:0/1482498730 shutdown_connections 2023-10-08T01:08:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.495+0000 7fe98e9ff700 1 -- 172.21.15.119:0/1482498730 wait complete. 2023-10-08T01:08:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.496+0000 7fe98e9ff700 1 Processor -- start 2023-10-08T01:08:47.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.496+0000 7fe98e9ff700 1 -- start start 2023-10-08T01:08:47.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98e9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9880afa60 0x7fe9880c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:47.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98e9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe98815ac80 0x7fe9880b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:47.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98e9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815b050 0x7fe9880b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:47.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98e9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880c5eb0 con 0x7fe98815b050 2023-10-08T01:08:47.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98e9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe988168c80 con 0x7fe98815ac80 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98e9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9880ac190 con 0x7fe9880afa60 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98cf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815b050 0x7fe9880b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98cf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815b050 0x7fe9880b1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48330/0 (socket says 172.21.15.119:48330) 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe98cf9c700 1 -- 172.21.15.119:0/2103083040 learned_addr learned my addr 172.21.15.119:0/2103083040 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe97f7fe700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe98815ac80 0x7fe9880b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.497+0000 7fe987fff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9880afa60 0x7fe9880c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe98cf9c700 1 -- 172.21.15.119:0/2103083040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9880afa60 msgr2=0x7fe9880c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe98cf9c700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9880afa60 0x7fe9880c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe98cf9c700 1 -- 172.21.15.119:0/2103083040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe98815ac80 msgr2=0x7fe9880b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe98cf9c700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe98815ac80 0x7fe9880b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:47.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe98cf9c700 1 -- 172.21.15.119:0/2103083040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe978009ae0 con 0x7fe98815b050 2023-10-08T01:08:47.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe98cf9c700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815b050 0x7fe9880b1410 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe98000ea50 tx=0x7fe9800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:47.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe984ff9700 1 -- 172.21.15.119:0/2103083040 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98000ee90 con 0x7fe98815b050 2023-10-08T01:08:47.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe984ff9700 1 -- 172.21.15.119:0/2103083040 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98000c850 con 0x7fe98815b050 2023-10-08T01:08:47.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe984ff9700 1 -- 172.21.15.119:0/2103083040 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98000cc10 con 0x7fe98815b050 2023-10-08T01:08:47.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9880c4f50 con 0x7fe98815b050 2023-10-08T01:08:47.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.498+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9880abb40 con 0x7fe98815b050 2023-10-08T01:08:47.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.499+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9880ac340 con 0x7fe98815b050 2023-10-08T01:08:47.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.500+0000 7fe984ff9700 1 -- 172.21.15.119:0/2103083040 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 17) v1 ==== 84928+0+0 (secure 0 0 0) 0x7fe980013470 con 0x7fe98815b050 2023-10-08T01:08:47.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.501+0000 7fe984ff9700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe968067130 0x7fe9680695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:47.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.501+0000 7fe984ff9700 1 -- 172.21.15.119:0/2103083040 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fe980084df0 con 0x7fe98815b050 2023-10-08T01:08:47.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.501+0000 7fe987fff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe968067130 0x7fe9680695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:47.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.502+0000 7fe987fff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe968067130 0x7fe9680695e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe978009880 tx=0x7fe978018000 comp rx=0 tx=0).ready entity=mgr.5142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:47.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.505+0000 7fe984ff9700 1 -- 172.21.15.119:0/2103083040 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe980051a30 con 0x7fe98815b050 2023-10-08T01:08:47.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:47.749+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fe98804c3c0 con 0x7fe98815b050 2023-10-08T01:08:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.229+0000 7fe984ff9700 1 -- 172.21.15.119:0/2103083040 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fe980051e00 con 0x7fe98815b050 2023-10-08T01:08:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.234+0000 7fe984ff9700 1 -- 172.21.15.119:0/2103083040 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v18) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe980051a30 con 0x7fe98815b050 2023-10-08T01:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.238+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe968067130 msgr2=0x7fe9680695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.238+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe968067130 0x7fe9680695e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe978009880 tx=0x7fe978018000 comp rx=0 tx=0).stop 2023-10-08T01:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815b050 msgr2=0x7fe9880b1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815b050 0x7fe9880b1410 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fe98000ea50 tx=0x7fe9800091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 shutdown_connections 2023-10-08T01:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9880afa60 0x7fe9880c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fe968067130 0x7fe9680695e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe98815b050 0x7fe9880b1410 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 --2- 172.21.15.119:0/2103083040 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe98815ac80 0x7fe9880b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 >> 172.21.15.119:0/2103083040 conn(0x7fe988068e10 msgr2=0x7fe988159170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 shutdown_connections 2023-10-08T01:08:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.239+0000 7fe98e9ff700 1 -- 172.21.15.119:0/2103083040 wait complete. 2023-10-08T01:08:48.266 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:08:48.266 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:08:48.359 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:08:48.440 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:08:48.440 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T01:08:48.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:08:48.440 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T01:08:48.527 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T01:08:48.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:08:48.527 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T01:08:48.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:08:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.792+0000 7fef3ffff700 1 Processor -- start 2023-10-08T01:08:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.792+0000 7fef3ffff700 1 -- start start 2023-10-08T01:08:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.793+0000 7fef3ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40159420 0x7fef400b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.793+0000 7fef3ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fef401597f0 0x7fef400b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.793+0000 7fef3ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef4015d230 0x7fef4015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.793+0000 7fef3ffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef4015dbd0 con 0x7fef4015d230 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.793+0000 7fef3ffff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef400b11d0 con 0x7fef401597f0 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.793+0000 7fef3ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef400b14b0 con 0x7fef40159420 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.793+0000 7fef3f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef4015d230 0x7fef4015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef4015d230 0x7fef4015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48352/0 (socket says 172.21.15.119:48352) 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3f7fe700 1 -- 172.21.15.119:0/75825487 learned_addr learned my addr 172.21.15.119:0/75825487 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3e7fc700 1 --2- 172.21.15.119:0/75825487 >> v2:172.21.15.184:3300/0 conn(0x7fef401597f0 0x7fef400b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3effd700 1 --2- 172.21.15.119:0/75825487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40159420 0x7fef400b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3f7fe700 1 -- 172.21.15.119:0/75825487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40159420 msgr2=0x7fef400b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3f7fe700 1 --2- 172.21.15.119:0/75825487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40159420 0x7fef400b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3f7fe700 1 -- 172.21.15.119:0/75825487 >> v2:172.21.15.184:3300/0 conn(0x7fef401597f0 msgr2=0x7fef400b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3f7fe700 1 --2- 172.21.15.119:0/75825487 >> v2:172.21.15.184:3300/0 conn(0x7fef401597f0 0x7fef400b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3f7fe700 1 -- 172.21.15.119:0/75825487 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400b1790 con 0x7fef4015d230 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3f7fe700 1 --2- 172.21.15.119:0/75825487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef4015d230 0x7fef4015d690 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fef400d1770 tx=0x7fef38009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66cb11a5a0e8979 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.794+0000 7fef3dffb700 1 -- 172.21.15.119:0/75825487 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3800e070 con 0x7fef4015d230 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3dffb700 1 -- 172.21.15.119:0/75825487 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3800b970 con 0x7fef4015d230 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3dffb700 1 -- 172.21.15.119:0/75825487 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef380092e0 con 0x7fef4015d230 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 -- 172.21.15.119:0/75825487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef4015d230 msgr2=0x7fef4015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 --2- 172.21.15.119:0/75825487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef4015d230 0x7fef4015d690 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fef400d1770 tx=0x7fef38009840 comp rx=0 tx=0).stop 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 -- 172.21.15.119:0/75825487 shutdown_connections 2023-10-08T01:08:48.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 --2- 172.21.15.119:0/75825487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef4015d230 0x7fef4015d690 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 --2- 172.21.15.119:0/75825487 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40159420 0x7fef400b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 --2- 172.21.15.119:0/75825487 >> v2:172.21.15.184:3300/0 conn(0x7fef401597f0 0x7fef400b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 -- 172.21.15.119:0/75825487 >> 172.21.15.119:0/75825487 conn(0x7fef400d0660 msgr2=0x7fef400d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:48.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 -- 172.21.15.119:0/75825487 shutdown_connections 2023-10-08T01:08:48.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.795+0000 7fef3ffff700 1 -- 172.21.15.119:0/75825487 wait complete. 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.797+0000 7fef3ffff700 1 Processor -- start 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.797+0000 7fef3ffff700 1 -- start start 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40159420 0x7fef4016a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401597f0 0x7fef4016c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4015d230 0x7fef400cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef400cc390 con 0x7fef40159420 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef400cb810 con 0x7fef401597f0 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef400cbaf0 con 0x7fef4015d230 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40159420 0x7fef4016a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40159420 0x7fef4016a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48360/0 (socket says 172.21.15.119:48360) 2023-10-08T01:08:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3effd700 1 -- 172.21.15.119:0/2579451004 learned_addr learned my addr 172.21.15.119:0/2579451004 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3e7fc700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401597f0 0x7fef4016c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3f7fe700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4015d230 0x7fef400cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3effd700 1 -- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4015d230 msgr2=0x7fef400cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3effd700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4015d230 0x7fef400cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3effd700 1 -- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401597f0 msgr2=0x7fef4016c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3effd700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401597f0 0x7fef4016c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3effd700 1 -- 172.21.15.119:0/2579451004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef38009ae0 con 0x7fef40159420 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.798+0000 7fef3e7fc700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401597f0 0x7fef4016c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.799+0000 7fef3effd700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40159420 0x7fef4016a660 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fef3000ec80 tx=0x7fef300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.799+0000 7fef26ffd700 1 -- 172.21.15.119:0/2579451004 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3000c9d0 con 0x7fef40159420 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.799+0000 7fef26ffd700 1 -- 172.21.15.119:0/2579451004 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3000cb50 con 0x7fef40159420 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.799+0000 7fef26ffd700 1 -- 172.21.15.119:0/2579451004 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef30013470 con 0x7fef40159420 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.799+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef400cbdd0 con 0x7fef40159420 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.799+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef400cb510 con 0x7fef40159420 2023-10-08T01:08:48.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.800+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4016ac70 con 0x7fef40159420 2023-10-08T01:08:48.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.801+0000 7fef26ffd700 1 -- 172.21.15.119:0/2579451004 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fef30013860 con 0x7fef40159420 2023-10-08T01:08:48.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.802+0000 7fef26ffd700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fef2c0670e0 0x7fef2c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:48.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.802+0000 7fef26ffd700 1 -- 172.21.15.119:0/2579451004 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6531+0+0 (secure 0 0 0) 0x7fef30084f60 con 0x7fef40159420 2023-10-08T01:08:48.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.802+0000 7fef3e7fc700 1 -- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fef2c0670e0 msgr2=0x7fef2c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:48.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.802+0000 7fef3e7fc700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fef2c0670e0 0x7fef2c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:48.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:48.810+0000 7fef26ffd700 1 -- 172.21.15.119:0/2579451004 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef30051c10 con 0x7fef40159420 2023-10-08T01:08:49.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.003+0000 7fef3e7fc700 1 -- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fef2c0670e0 msgr2=0x7fef2c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:49.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.003+0000 7fef3e7fc700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fef2c0670e0 0x7fef2c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:49.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.048+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef4004c3c0 con 0x7fef40159420 2023-10-08T01:08:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.049+0000 7fef26ffd700 1 -- 172.21.15.119:0/2579451004 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fef30055230 con 0x7fef40159420 2023-10-08T01:08:49.049 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:49.049 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/768996961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":768996961},{"type":"v1","addr":"172.21.15.184:6834","nonce":768996961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1876644893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1876644893},{"type":"v1","addr":"172.21.15.119:6837","nonce":1876644893}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1187803776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1187803776},{"type":"v1","addr":"172.21.15.184:6835","nonce":1187803776}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:56:39.582864+0000","modified":"2023-10-08T00:56:43.160226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3785845701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3785845701},{"type":"v1","addr":"172.21.15.119:6836","nonce":3785845701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T01:08:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fef2c0670e0 msgr2=0x7fef2c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fef2c0670e0 0x7fef2c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40159420 msgr2=0x7fef4016a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40159420 0x7fef4016a660 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fef3000ec80 tx=0x7fef300091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 shutdown_connections 2023-10-08T01:08:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef4015d230 0x7fef400cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fef2c0670e0 0x7fef2c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40159420 0x7fef4016a660 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 --2- 172.21.15.119:0/2579451004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef401597f0 0x7fef4016c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 >> 172.21.15.119:0/2579451004 conn(0x7fef400d0660 msgr2=0x7fef400b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:49.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 shutdown_connections 2023-10-08T01:08:49.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.052+0000 7fef3ffff700 1 -- 172.21.15.119:0/2579451004 wait complete. 2023-10-08T01:08:49.053 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2023-10-08T01:08:49.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:08:49.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.269+0000 7fcebf59e700 1 Processor -- start 2023-10-08T01:08:49.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.269+0000 7fcebf59e700 1 -- start start 2023-10-08T01:08:49.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.270+0000 7fcebf59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcec0159420 0x7fcec00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.270+0000 7fcebf59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcec015d230 0x7fcec015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.270+0000 7fcebf59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec00b2980 con 0x7fcec01597f0 2023-10-08T01:08:49.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.270+0000 7fcebf59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec015db70 con 0x7fcec015d230 2023-10-08T01:08:49.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.270+0000 7fcebf59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec00b11d0 con 0x7fcec0159420 2023-10-08T01:08:49.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.270+0000 7fcebed9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcec015d230 0x7fcec015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebed9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcec015d230 0x7fcec015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48372/0 (socket says 172.21.15.119:48372) 2023-10-08T01:08:49.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebed9d700 1 -- 172.21.15.119:0/902241703 learned_addr learned my addr 172.21.15.119:0/902241703 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:49.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebe59c700 1 --2- 172.21.15.119:0/902241703 >> v2:172.21.15.184:3300/0 conn(0x7fcec0159420 0x7fcec00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebed9d700 1 -- 172.21.15.119:0/902241703 >> v2:172.21.15.184:3300/0 conn(0x7fcec0159420 msgr2=0x7fcec00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebd59a700 1 -- 172.21.15.119:0/902241703 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421723731 0 0) 0x7fcec00b2980 con 0x7fcec01597f0 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebed9d700 1 --2- 172.21.15.119:0/902241703 >> v2:172.21.15.184:3300/0 conn(0x7fcec0159420 0x7fcec00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebed9d700 1 -- 172.21.15.119:0/902241703 >> v1:172.21.15.184:6789/0 conn(0x7fcec01597f0 legacy=0x7fcec00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebed9d700 1 -- 172.21.15.119:0/902241703 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec00b14b0 con 0x7fcec015d230 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.271+0000 7fcebed9d700 1 --2- 172.21.15.119:0/902241703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec015d230 0x7fcec015d630 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcec00b2f20 tx=0x7fceb8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34151d965c3f49ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebd59a700 1 -- 172.21.15.119:0/902241703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb800e070 con 0x7fcec015d230 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebd59a700 1 -- 172.21.15.119:0/902241703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb800ba20 con 0x7fcec015d230 2023-10-08T01:08:49.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebd59a700 1 -- 172.21.15.119:0/902241703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb80092e0 con 0x7fcec015d230 2023-10-08T01:08:49.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebf59e700 1 -- 172.21.15.119:0/902241703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec015d230 msgr2=0x7fcec015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebf59e700 1 --2- 172.21.15.119:0/902241703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec015d230 0x7fcec015d630 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fcec00b2f20 tx=0x7fceb8009840 comp rx=0 tx=0).stop 2023-10-08T01:08:49.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebf59e700 1 -- 172.21.15.119:0/902241703 shutdown_connections 2023-10-08T01:08:49.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebf59e700 1 --2- 172.21.15.119:0/902241703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec015d230 0x7fcec015d630 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebf59e700 1 --2- 172.21.15.119:0/902241703 >> v2:172.21.15.184:3300/0 conn(0x7fcec0159420 0x7fcec00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebf59e700 1 -- 172.21.15.119:0/902241703 >> 172.21.15.119:0/902241703 conn(0x7fcec00d0660 msgr2=0x7fcec00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:49.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebf59e700 1 -- 172.21.15.119:0/902241703 shutdown_connections 2023-10-08T01:08:49.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.272+0000 7fcebf59e700 1 -- 172.21.15.119:0/902241703 wait complete. 2023-10-08T01:08:49.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.274+0000 7fcebf59e700 1 Processor -- start 2023-10-08T01:08:49.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.274+0000 7fcebf59e700 1 -- start start 2023-10-08T01:08:49.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebf59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcec0159420 0x7fcec016d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebf59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec01597f0 0x7fcec0168d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebf59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec015d230 0x7fcec01543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebf59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec016d1f0 con 0x7fcec01597f0 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebf59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec016cbc0 con 0x7fcec015d230 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebf59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec0154e10 con 0x7fcec0159420 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebdd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec01597f0 0x7fcec0168d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebdd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec01597f0 0x7fcec0168d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48380/0 (socket says 172.21.15.119:48380) 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebdd9b700 1 -- 172.21.15.119:0/12702138 learned_addr learned my addr 172.21.15.119:0/12702138 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebed9d700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec015d230 0x7fcec01543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.275+0000 7fcebe59c700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcec0159420 0x7fcec016d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebdd9b700 1 -- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcec0159420 msgr2=0x7fcec016d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebdd9b700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcec0159420 0x7fcec016d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebdd9b700 1 -- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec015d230 msgr2=0x7fcec01543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebdd9b700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec015d230 0x7fcec01543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebdd9b700 1 -- 172.21.15.119:0/12702138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb8009ae0 con 0x7fcec01597f0 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebed9d700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec015d230 0x7fcec01543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebdd9b700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec01597f0 0x7fcec0168d10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fceb400c7e0 tx=0x7fceb4007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fceae7fc700 1 -- 172.21.15.119:0/12702138 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4011590 con 0x7fcec01597f0 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fceae7fc700 1 -- 172.21.15.119:0/12702138 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4011710 con 0x7fcec01597f0 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fceae7fc700 1 -- 172.21.15.119:0/12702138 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4011ad0 con 0x7fcec01597f0 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec01550f0 con 0x7fcec01597f0 2023-10-08T01:08:49.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.276+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec00b4dd0 con 0x7fcec01597f0 2023-10-08T01:08:49.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.277+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec016d360 con 0x7fcec01597f0 2023-10-08T01:08:49.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.278+0000 7fceae7fc700 1 -- 172.21.15.119:0/12702138 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fceb401b3b0 con 0x7fcec01597f0 2023-10-08T01:08:49.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.279+0000 7fceae7fc700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcea00670e0 0x7fcea0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.279+0000 7fceae7fc700 1 -- 172.21.15.119:0/12702138 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6419+0+0 (secure 0 0 0) 0x7fceb4084070 con 0x7fcec01597f0 2023-10-08T01:08:49.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.279+0000 7fcebe59c700 1 -- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcea00670e0 msgr2=0x7fcea0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:49.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.279+0000 7fcebe59c700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcea00670e0 0x7fcea0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:49.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.283+0000 7fceae7fc700 1 -- 172.21.15.119:0/12702138 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fceb4050cf0 con 0x7fcec01597f0 2023-10-08T01:08:49.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.480+0000 7fcebe59c700 1 -- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcea00670e0 msgr2=0x7fcea0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:49.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.480+0000 7fcebe59c700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcea00670e0 0x7fcea0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.486+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcec016d040 con 0x7fcec01597f0 2023-10-08T01:08:49.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.487+0000 7fceae7fc700 1 -- 172.21.15.119:0/12702138 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16489 (secure 0 0 0) 0x7fceb4017070 con 0x7fcec01597f0 2023-10-08T01:08:49.487 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:49.488 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":45,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:08:49.135623+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:56:37.182342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:56:38.569786+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":235,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/538579956":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/2510601793":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/4274210250":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/1792355627":"2023-10-09T00:56:55.298790+0000","172.21.15.119:6832/85511":"2023-10-09T00:56:55.298790+0000","172.21.15.119:6833/85511":"2023-10-09T00:56:55.298790+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":9,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":24,"length":3},{"begin":31,"length":2},{"begin":36,"length":4},{"begin":42,"length":2},{"begin":47,"length":4},{"begin":53,"length":2},{"begin":58,"length":3},{"begin":63,"length":2},{"begin":68,"length":4},{"begin":74,"length":2},{"begin":80,"length":3},{"begin":85,"length":2},{"begin":91,"length":3},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":102,"length":3},{"begin":107,"length":3},{"begin":112,"length":3},{"begin":117,"length":2},{"begin":177,"length":59}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":102,"length":1},{"begin":112,"length":1},{"begin":234,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:08:49.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcea00670e0 msgr2=0x7fcea0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcea00670e0 0x7fcea0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec01597f0 msgr2=0x7fcec0168d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec01597f0 0x7fcec0168d10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fceb400c7e0 tx=0x7fceb4007dc0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 shutdown_connections 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcec0159420 0x7fcec016d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fcea00670e0 0x7fcea0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcec015d230 0x7fcec01543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 --2- 172.21.15.119:0/12702138 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcec01597f0 0x7fcec0168d10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 >> 172.21.15.119:0/12702138 conn(0x7fcec00d0660 msgr2=0x7fcec0069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 shutdown_connections 2023-10-08T01:08:49.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.490+0000 7fcebf59e700 1 -- 172.21.15.119:0/12702138 wait complete. 2023-10-08T01:08:49.509 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T00:56:37.182342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T00:56:38.569786+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 235, 'snap_epoch': 45, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T01:08:49.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:08:49.540 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:49.540+0000 7f21ab0f7180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:08:49.607 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:49.607+0000 7f662c6b2180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:08:49.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.700+0000 7ffa6ffff700 1 Processor -- start 2023-10-08T01:08:49.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.700+0000 7ffa6ffff700 1 -- start start 2023-10-08T01:08:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.701+0000 7ffa6ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa700b4e70 0x7ffa700ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.701+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa700b16d0 0x7ffa700b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.701+0000 7ffa6ffff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700b2080 con 0x7ffa7015d240 2023-10-08T01:08:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.701+0000 7ffa6ffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700cfb00 con 0x7ffa700b4e70 2023-10-08T01:08:49.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.701+0000 7ffa6ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b5c40 con 0x7ffa700b16d0 2023-10-08T01:08:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.701+0000 7ffa6effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa700b4e70 0x7ffa700ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.701+0000 7ffa6effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa700b4e70 0x7ffa700ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48392/0 (socket says 172.21.15.119:48392) 2023-10-08T01:08:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.701+0000 7ffa6effd700 1 -- 172.21.15.119:0/686645163 learned_addr learned my addr 172.21.15.119:0/686645163 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6e7fc700 1 --2- 172.21.15.119:0/686645163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa700b16d0 0x7ffa700b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6effd700 1 -- 172.21.15.119:0/686645163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa700b16d0 msgr2=0x7ffa700b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:49.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6effd700 1 --2- 172.21.15.119:0/686645163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa700b16d0 0x7ffa700b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6effd700 1 -- 172.21.15.119:0/686645163 >> v1:172.21.15.184:6789/0 conn(0x7ffa7015d240 legacy=0x7ffa7015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6effd700 1 -- 172.21.15.119:0/686645163 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700b5f20 con 0x7ffa700b4e70 2023-10-08T01:08:49.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6effd700 1 --2- 172.21.15.119:0/686645163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa700ccff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffa600099b0 tx=0x7ffa60009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa526129838c2566 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:49.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6dffb700 1 -- 172.21.15.119:0/686645163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000b860 con 0x7ffa700b4e70 2023-10-08T01:08:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6dffb700 1 -- 172.21.15.119:0/686645163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6000b9c0 con 0x7ffa700b4e70 2023-10-08T01:08:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.702+0000 7ffa6dffb700 1 -- 172.21.15.119:0/686645163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000bd60 con 0x7ffa700b4e70 2023-10-08T01:08:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.703+0000 7ffa6ffff700 1 -- 172.21.15.119:0/686645163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 msgr2=0x7ffa700ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.703+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/686645163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa700ccff0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ffa600099b0 tx=0x7ffa60009250 comp rx=0 tx=0).stop 2023-10-08T01:08:49.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.703+0000 7ffa6ffff700 1 -- 172.21.15.119:0/686645163 shutdown_connections 2023-10-08T01:08:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.703+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/686645163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa700ccff0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.703+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/686645163 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa700b16d0 0x7ffa700b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.703+0000 7ffa6ffff700 1 -- 172.21.15.119:0/686645163 >> 172.21.15.119:0/686645163 conn(0x7ffa700d10c0 msgr2=0x7ffa700affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.703+0000 7ffa6ffff700 1 -- 172.21.15.119:0/686645163 shutdown_connections 2023-10-08T01:08:49.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.703+0000 7ffa6ffff700 1 -- 172.21.15.119:0/686645163 wait complete. 2023-10-08T01:08:49.708 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:49.703+0000 7f21ab0f7180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:08:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.705+0000 7ffa6ffff700 1 Processor -- start 2023-10-08T01:08:49.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.705+0000 7ffa6ffff700 1 -- start start 2023-10-08T01:08:49.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa700b16d0 0x7ffa700ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa7014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa7015d240 0x7ffa70068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa7014b600 con 0x7ffa700b4e70 2023-10-08T01:08:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700caf10 con 0x7ffa700b16d0 2023-10-08T01:08:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700cb1f0 con 0x7ffa7015d240 2023-10-08T01:08:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa7014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6effd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa700b16d0 0x7ffa700ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa7014b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48406/0 (socket says 172.21.15.119:48406) 2023-10-08T01:08:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6e7fc700 1 -- 172.21.15.119:0/94063228 learned_addr learned my addr 172.21.15.119:0/94063228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6f7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa7015d240 0x7ffa70068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6effd700 1 -- 172.21.15.119:0/94063228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa7015d240 msgr2=0x7ffa70068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6effd700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa7015d240 0x7ffa70068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6effd700 1 -- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 msgr2=0x7ffa7014b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6effd700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa7014b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6effd700 1 -- 172.21.15.119:0/94063228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600094f0 con 0x7ffa700b16d0 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.706+0000 7ffa6e7fc700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa7014b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.707+0000 7ffa6effd700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa700b16d0 0x7ffa700ca8a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa7015d180 tx=0x7ffa60017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.707+0000 7ffa56ffd700 1 -- 172.21.15.119:0/94063228 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60011e40 con 0x7ffa700b16d0 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.707+0000 7ffa56ffd700 1 -- 172.21.15.119:0/94063228 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60024440 con 0x7ffa700b16d0 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.707+0000 7ffa56ffd700 1 -- 172.21.15.119:0/94063228 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa600247e0 con 0x7ffa700b16d0 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.707+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa700cb4d0 con 0x7ffa700b16d0 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.707+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa700cba90 con 0x7ffa700b16d0 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.708+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa70088990 con 0x7ffa700b16d0 2023-10-08T01:08:49.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.709+0000 7ffa56ffd700 1 -- 172.21.15.119:0/94063228 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7ffa6001a420 con 0x7ffa700b16d0 2023-10-08T01:08:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.710+0000 7ffa56ffd700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffa5c0670e0 0x7ffa5c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.710+0000 7ffa56ffd700 1 -- 172.21.15.119:0/94063228 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6419+0+0 (secure 0 0 0) 0x7ffa60086600 con 0x7ffa700b16d0 2023-10-08T01:08:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.710+0000 7ffa6e7fc700 1 -- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffa5c0670e0 msgr2=0x7ffa5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.710+0000 7ffa6e7fc700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffa5c0670e0 0x7ffa5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:49.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.716+0000 7ffa56ffd700 1 -- 172.21.15.119:0/94063228 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffa60053280 con 0x7ffa700b16d0 2023-10-08T01:08:49.778 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:49.778+0000 7f662c6b2180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:08:49.793 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:49.790+0000 7f21ab0f7180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:08:49.863 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:49.863+0000 7f662c6b2180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:08:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.911+0000 7ffa6e7fc700 1 -- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffa5c0670e0 msgr2=0x7ffa5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.911+0000 7ffa6e7fc700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffa5c0670e0 0x7ffa5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.912+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa700cbc80 con 0x7ffa700b16d0 2023-10-08T01:08:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.912+0000 7ffa56ffd700 1 -- 172.21.15.119:0/94063228 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa600568a0 con 0x7ffa700b16d0 2023-10-08T01:08:49.912 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:08:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffa5c0670e0 msgr2=0x7ffa5c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffa5c0670e0 0x7ffa5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa700b16d0 msgr2=0x7ffa700ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa700b16d0 0x7ffa700ca8a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa7015d180 tx=0x7ffa60017000 comp rx=0 tx=0).stop 2023-10-08T01:08:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 shutdown_connections 2023-10-08T01:08:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffa7015d240 0x7ffa70068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffa5c0670e0 0x7ffa5c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffa700b16d0 0x7ffa700ca8a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 --2- 172.21.15.119:0/94063228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa700b4e70 0x7ffa7014b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 >> 172.21.15.119:0/94063228 conn(0x7ffa700d10c0 msgr2=0x7ffa701590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 shutdown_connections 2023-10-08T01:08:49.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:49.915+0000 7ffa6ffff700 1 -- 172.21.15.119:0/94063228 wait complete. 2023-10-08T01:08:49.937 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T01:08:50.015 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:50.015+0000 7f21ab0f7180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:08:50.096 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:50.096+0000 7f662c6b2180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:08:50.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.126+0000 7fb7ebfff700 1 Processor -- start 2023-10-08T01:08:50.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.127+0000 7fb7ebfff700 1 -- start start 2023-10-08T01:08:50.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7ebfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7ec0b16d0 0x7fb7ec0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7ebfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7ec0b2080 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7ebfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7ec0cfb00 con 0x7fb7ec0b16d0 2023-10-08T01:08:50.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7ebfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec0b5c40 con 0x7fb7ec0b33b0 2023-10-08T01:08:50.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7ea7fc700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7ec0b16d0 0x7fb7ec0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7eb7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7eaffd700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7fb7ec0b4e70 0x7fb7ec0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:59712/0 (socket says 172.21.15.119:59712) 2023-10-08T01:08:50.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.128+0000 7fb7eaffd700 1 -- 172.21.15.119:0/738694125 learned_addr learned my addr 172.21.15.119:0/738694125 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:50.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7e9ffb700 1 -- 172.21.15.119:0/738694125 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106789587 0 0) 0x7fb7ec0b2080 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7e9ffb700 1 -- 172.21.15.119:0/738694125 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d4003570 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7eb7fe700 1 -- 172.21.15.119:0/738694125 >> v2:172.21.15.184:3300/0 conn(0x7fb7ec0b16d0 msgr2=0x7fb7ec0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:50.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7eb7fe700 1 --2- 172.21.15.119:0/738694125 >> v2:172.21.15.184:3300/0 conn(0x7fb7ec0b16d0 0x7fb7ec0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7eb7fe700 1 -- 172.21.15.119:0/738694125 >> v1:172.21.15.184:6789/0 conn(0x7fb7ec0b4e70 legacy=0x7fb7ec0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7eb7fe700 1 -- 172.21.15.119:0/738694125 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec0b5f20 con 0x7fb7ec0b33b0 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7ea7fc700 1 --2- 172.21.15.119:0/738694125 >> v2:172.21.15.184:3300/0 conn(0x7fb7ec0b16d0 0x7fb7ec0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7eb7fe700 1 --2- 172.21.15.119:0/738694125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0b3810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009e60 tx=0x7fb7e4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1cd0d4bf316900 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7e9ffb700 1 -- 172.21.15.119:0/738694125 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400e070 con 0x7fb7ec0b33b0 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7e9ffb700 1 -- 172.21.15.119:0/738694125 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e400bab0 con 0x7fb7ec0b33b0 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.129+0000 7fb7e9ffb700 1 -- 172.21.15.119:0/738694125 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e400be50 con 0x7fb7ec0b33b0 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.130+0000 7fb7ebfff700 1 -- 172.21.15.119:0/738694125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 msgr2=0x7fb7ec0b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.130+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/738694125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0b3810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4009e60 tx=0x7fb7e4009780 comp rx=0 tx=0).stop 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.130+0000 7fb7ebfff700 1 -- 172.21.15.119:0/738694125 shutdown_connections 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.130+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/738694125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0b3810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.130+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/738694125 >> v2:172.21.15.184:3300/0 conn(0x7fb7ec0b16d0 0x7fb7ec0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.130+0000 7fb7ebfff700 1 -- 172.21.15.119:0/738694125 >> 172.21.15.119:0/738694125 conn(0x7fb7ec0d10c0 msgr2=0x7fb7ec0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:50.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.130+0000 7fb7ebfff700 1 -- 172.21.15.119:0/738694125 shutdown_connections 2023-10-08T01:08:50.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.130+0000 7fb7ebfff700 1 -- 172.21.15.119:0/738694125 wait complete. 2023-10-08T01:08:50.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.132+0000 7fb7ebfff700 1 Processor -- start 2023-10-08T01:08:50.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.132+0000 7fb7ebfff700 1 -- start start 2023-10-08T01:08:50.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec0b16d0 0x7fb7ec149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7ebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7ec0b4e70 0x7fb7ec0ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7ebfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec149670 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7ebfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec0c67d0 con 0x7fb7ec0b16d0 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7ebfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec0c5bc0 con 0x7fb7ec0b33b0 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eaffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec0b16d0 0x7fb7ec149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7ec0b4e70 0x7fb7ec0ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eaffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec0b16d0 0x7fb7ec149130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44668/0 (socket says 172.21.15.119:44668) 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eaffd700 1 -- 172.21.15.119:0/2126273636 learned_addr learned my addr 172.21.15.119:0/2126273636 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7ea7fc700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eb7fe700 1 -- 172.21.15.119:0/2126273636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 msgr2=0x7fb7ec0c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eb7fe700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eb7fe700 1 -- 172.21.15.119:0/2126273636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec0b16d0 msgr2=0x7fb7ec149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eb7fe700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec0b16d0 0x7fb7ec149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.133+0000 7fb7eb7fe700 1 -- 172.21.15.119:0/2126273636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009420 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.134+0000 7fb7eb7fe700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7ec0b4e70 0x7fb7ec0ca9b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4016000 tx=0x7fb7e4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:50.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.134+0000 7fb7d2ffd700 1 -- 172.21.15.119:0/2126273636 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4007000 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.134+0000 7fb7d2ffd700 1 -- 172.21.15.119:0/2126273636 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e4012d80 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.134+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e4009a20 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.134+0000 7fb7d2ffd700 1 -- 172.21.15.119:0/2126273636 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4026640 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.134+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7ec0c6180 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.135+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7ec088990 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.136+0000 7fb7d2ffd700 1 -- 172.21.15.119:0/2126273636 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fb7e40269b0 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.137+0000 7fb7d2ffd700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb7d80670e0 0x7fb7d8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.137+0000 7fb7d2ffd700 1 -- 172.21.15.119:0/2126273636 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6419+0+0 (secure 0 0 0) 0x7fb7e4086b10 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.137+0000 7fb7eaffd700 1 -- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb7d80670e0 msgr2=0x7fb7d8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:50.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.137+0000 7fb7eaffd700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb7d80670e0 0x7fb7d8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:50.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.141+0000 7fb7d2ffd700 1 -- 172.21.15.119:0/2126273636 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7e4053680 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.193 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:50.193+0000 7f21ab0f7180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:08:50.281 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:50.281+0000 7f662c6b2180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:08:50.287 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:50.287+0000 7f21ab0f7180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:08:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.337+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7ec04bb80 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.337+0000 7fb7d2ffd700 1 -- 172.21.15.119:0/2126273636 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7e4056ca0 con 0x7fb7ec0b4e70 2023-10-08T01:08:50.337 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T01:08:50.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.338+0000 7fb7eaffd700 1 -- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb7d80670e0 msgr2=0x7fb7d8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:50.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.338+0000 7fb7eaffd700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb7d80670e0 0x7fb7d8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.339+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb7d80670e0 msgr2=0x7fb7d8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.339+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb7d80670e0 0x7fb7d8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.339+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7ec0b4e70 msgr2=0x7fb7ec0ca9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.339+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7ec0b4e70 0x7fb7ec0ca9b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb7e4016000 tx=0x7fb7e4019000 comp rx=0 tx=0).stop 2023-10-08T01:08:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.340+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 shutdown_connections 2023-10-08T01:08:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.340+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fb7d80670e0 0x7fb7d8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.340+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7ec0b4e70 0x7fb7ec0ca9b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.340+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7ec0b16d0 0x7fb7ec149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.340+0000 7fb7ebfff700 1 --2- 172.21.15.119:0/2126273636 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7ec0b33b0 0x7fb7ec0c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.340+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 >> 172.21.15.119:0/2126273636 conn(0x7fb7ec0d10c0 msgr2=0x7fb7ec158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.340+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 shutdown_connections 2023-10-08T01:08:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.340+0000 7fb7ebfff700 1 -- 172.21.15.119:0/2126273636 wait complete. 2023-10-08T01:08:50.358 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T01:08:50.375 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:50.375+0000 7f662c6b2180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:08:50.454 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:50.453+0000 7f21ab0f7180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:08:50.542 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:50.542+0000 7f662c6b2180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:08:50.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.548+0000 7f609ed9d700 1 Processor -- start 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.548+0000 7f609ed9d700 1 -- start start 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.549+0000 7f609ed9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f60980b16d0 0x7f60980b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.549+0000 7f609ed9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60980b2080 con 0x7f60980b33b0 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.549+0000 7f609ed9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60980cfb00 con 0x7f60980b4e70 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.549+0000 7f609ed9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60980b5c40 con 0x7f60980b16d0 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.550+0000 7f609e59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f60980b33b0 0x7f60980b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37672/0 (socket says 172.21.15.119:37672) 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.550+0000 7f609e59c700 1 -- 172.21.15.119:0/2663979986 learned_addr learned my addr 172.21.15.119:0/2663979986 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.550+0000 7f609d59a700 1 --2- 172.21.15.119:0/2663979986 >> v2:172.21.15.184:3300/0 conn(0x7f60980b16d0 0x7f60980b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.550+0000 7f609cd99700 1 -- 172.21.15.119:0/2663979986 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290216865 0 0) 0x7f60980b2080 con 0x7f60980b33b0 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.550+0000 7f609cd99700 1 -- 172.21.15.119:0/2663979986 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6084003570 con 0x7f60980b33b0 2023-10-08T01:08:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.550+0000 7f609d59a700 1 -- 172.21.15.119:0/2663979986 >> v1:172.21.15.184:6789/0 conn(0x7f60980b4e70 legacy=0x7f60980ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.550+0000 7f609d59a700 1 -- 172.21.15.119:0/2663979986 >> v1:172.21.15.119:6789/0 conn(0x7f60980b33b0 legacy=0x7f60980b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.550+0000 7f609d59a700 1 -- 172.21.15.119:0/2663979986 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980b5f20 con 0x7f60980b16d0 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609d59a700 1 --2- 172.21.15.119:0/2663979986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 0x7f60980b1b40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f60880098e0 tx=0x7f60880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fb9745472253ce5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609cd99700 1 -- 172.21.15.119:0/2663979986 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608800b640 con 0x7f60980b16d0 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609cd99700 1 -- 172.21.15.119:0/2663979986 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608800b7a0 con 0x7f60980b16d0 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609cd99700 1 -- 172.21.15.119:0/2663979986 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608800bb40 con 0x7f60980b16d0 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609ed9d700 1 -- 172.21.15.119:0/2663979986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 msgr2=0x7f60980b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609ed9d700 1 --2- 172.21.15.119:0/2663979986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 0x7f60980b1b40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f60880098e0 tx=0x7f60880091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609ed9d700 1 -- 172.21.15.119:0/2663979986 shutdown_connections 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609ed9d700 1 --2- 172.21.15.119:0/2663979986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 0x7f60980b1b40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.551+0000 7f609ed9d700 1 -- 172.21.15.119:0/2663979986 >> 172.21.15.119:0/2663979986 conn(0x7f60980d10c0 msgr2=0x7f60980affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:50.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.552+0000 7f609ed9d700 1 -- 172.21.15.119:0/2663979986 shutdown_connections 2023-10-08T01:08:50.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.552+0000 7f609ed9d700 1 -- 172.21.15.119:0/2663979986 wait complete. 2023-10-08T01:08:50.554 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:50.553+0000 7f21ab0f7180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:08:50.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.553+0000 7f609ed9d700 1 Processor -- start 2023-10-08T01:08:50.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.553+0000 7f609ed9d700 1 -- start start 2023-10-08T01:08:50.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 0x7f609815ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980b33b0 0x7f60980af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980b4e70 0x7f609815d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609ed9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60980cbdc0 con 0x7f60980b4e70 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609ed9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60980cc0a0 con 0x7f60980b16d0 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609ed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60980cc380 con 0x7f60980b33b0 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980b4e70 0x7f609815d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980b4e70 0x7f609815d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48424/0 (socket says 172.21.15.119:48424) 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609e59c700 1 -- 172.21.15.119:0/3683213225 learned_addr learned my addr 172.21.15.119:0/3683213225 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609d59a700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980b33b0 0x7f60980af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.554+0000 7f609dd9b700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 0x7f609815ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:50.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f609e59c700 1 -- 172.21.15.119:0/3683213225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980b33b0 msgr2=0x7f60980af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f609e59c700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980b33b0 0x7f60980af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f609e59c700 1 -- 172.21.15.119:0/3683213225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 msgr2=0x7f609815ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:50.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f609e59c700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 0x7f609815ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:50.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f609e59c700 1 -- 172.21.15.119:0/3683213225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6088009420 con 0x7f60980b4e70 2023-10-08T01:08:50.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f609e59c700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980b4e70 0x7f609815d680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6090001460 tx=0x7f6090001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:50.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f608dffb700 1 -- 172.21.15.119:0/3683213225 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000a890 con 0x7f60980b4e70 2023-10-08T01:08:50.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f608dffb700 1 -- 172.21.15.119:0/3683213225 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000a9f0 con 0x7f60980b4e70 2023-10-08T01:08:50.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f608dffb700 1 -- 172.21.15.119:0/3683213225 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000ad70 con 0x7f60980b4e70 2023-10-08T01:08:50.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60980cc660 con 0x7f60980b4e70 2023-10-08T01:08:50.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.555+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60980cba50 con 0x7f60980b4e70 2023-10-08T01:08:50.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.556+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098088990 con 0x7f60980b4e70 2023-10-08T01:08:50.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.557+0000 7f608dffb700 1 -- 172.21.15.119:0/3683213225 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7f6090007300 con 0x7f60980b4e70 2023-10-08T01:08:50.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.558+0000 7f608dffb700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f607c0670e0 0x7f607c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:50.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.558+0000 7f609dd9b700 1 -- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f607c0670e0 msgr2=0x7f607c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:50.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.558+0000 7f608dffb700 1 -- 172.21.15.119:0/3683213225 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6419+0+0 (secure 0 0 0) 0x7f6090084e50 con 0x7f60980b4e70 2023-10-08T01:08:50.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.558+0000 7f609dd9b700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f607c0670e0 0x7f607c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:50.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.562+0000 7f608dffb700 1 -- 172.21.15.119:0/3683213225 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6090051bf0 con 0x7f60980b4e70 2023-10-08T01:08:50.648 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:50.648+0000 7f662c6b2180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:08:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.758+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f609804bb80 con 0x7f60980b4e70 2023-10-08T01:08:51.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.758+0000 7f608dffb700 1 -- 172.21.15.119:0/3683213225 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6090055210 con 0x7f60980b4e70 2023-10-08T01:08:51.333 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T01:08:51.333 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:51.034+0000 7f21ab0f7180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:08:51.334 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:51.127+0000 7f21ab0f7180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:08:51.334 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:51.216+0000 7f21ab0f7180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.759+0000 7f609dd9b700 1 -- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f607c0670e0 msgr2=0x7f607c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.759+0000 7f609dd9b700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f607c0670e0 0x7f607c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.760+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f607c0670e0 msgr2=0x7f607c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f607c0670e0 0x7f607c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980b4e70 msgr2=0x7f609815d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980b4e70 0x7f609815d680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6090001460 tx=0x7f6090001ae0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 shutdown_connections 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f607c0670e0 0x7f607c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980b4e70 0x7f609815d680 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60980b16d0 0x7f609815ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 --2- 172.21.15.119:0/3683213225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60980b33b0 0x7f60980af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 >> 172.21.15.119:0/3683213225 conn(0x7f60980d10c0 msgr2=0x7f6098068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 shutdown_connections 2023-10-08T01:08:51.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:50.761+0000 7f609ed9d700 1 -- 172.21.15.119:0/3683213225 wait complete. 2023-10-08T01:08:51.337 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:08:51.338 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:51.129+0000 7f662c6b2180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:08:51.338 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:51.223+0000 7f662c6b2180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:08:51.338 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:51.312+0000 7f662c6b2180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:08:51.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.564+0000 7fc80679d700 1 Processor -- start 2023-10-08T01:08:51.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.564+0000 7fc80679d700 1 -- start start 2023-10-08T01:08:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.565+0000 7fc80679d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc800159420 0x7fc8000b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.565+0000 7fc80679d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc8001597f0 0x7fc8000b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.565+0000 7fc80679d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8000b11d0 con 0x7fc80015d230 2023-10-08T01:08:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.565+0000 7fc80679d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8000b14b0 con 0x7fc8001597f0 2023-10-08T01:08:51.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.565+0000 7fc80679d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000b1790 con 0x7fc800159420 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc804f9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc8001597f0 0x7fc8000b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc804f9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc8001597f0 0x7fc8000b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48426/0 (socket says 172.21.15.119:48426) 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc804f9a700 1 -- 172.21.15.119:0/3563119298 learned_addr learned my addr 172.21.15.119:0/3563119298 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc80579b700 1 --2- 172.21.15.119:0/3563119298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc800159420 0x7fc8000b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc804f9a700 1 -- 172.21.15.119:0/3563119298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc800159420 msgr2=0x7fc8000b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc804f9a700 1 --2- 172.21.15.119:0/3563119298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc800159420 0x7fc8000b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc804f9a700 1 -- 172.21.15.119:0/3563119298 >> v1:172.21.15.184:6789/0 conn(0x7fc80015d230 legacy=0x7fc8000b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc804f9a700 1 -- 172.21.15.119:0/3563119298 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000b1a70 con 0x7fc8001597f0 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc7f7fff700 1 -- 172.21.15.119:0/3563119298 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231373535 0 0) 0x7fc8000b11d0 con 0x7fc80015d230 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.566+0000 7fc804f9a700 1 --2- 172.21.15.119:0/3563119298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 0x7fc8000b2270 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00099b0 tx=0x7fc7f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5078f37327de25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc7f7fff700 1 -- 172.21.15.119:0/3563119298 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0009ea0 con 0x7fc8001597f0 2023-10-08T01:08:51.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc7f7fff700 1 -- 172.21.15.119:0/3563119298 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f000b860 con 0x7fc8001597f0 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc7f7fff700 1 -- 172.21.15.119:0/3563119298 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000bc00 con 0x7fc8001597f0 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc80679d700 1 -- 172.21.15.119:0/3563119298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 msgr2=0x7fc8000b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc80679d700 1 --2- 172.21.15.119:0/3563119298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 0x7fc8000b2270 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc7f00099b0 tx=0x7fc7f0009250 comp rx=0 tx=0).stop 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc80679d700 1 -- 172.21.15.119:0/3563119298 shutdown_connections 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc80679d700 1 --2- 172.21.15.119:0/3563119298 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc800159420 0x7fc8000b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc80679d700 1 --2- 172.21.15.119:0/3563119298 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 0x7fc8000b2270 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc80679d700 1 -- 172.21.15.119:0/3563119298 >> 172.21.15.119:0/3563119298 conn(0x7fc8000d0660 msgr2=0x7fc8000d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc80679d700 1 -- 172.21.15.119:0/3563119298 shutdown_connections 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.567+0000 7fc80679d700 1 -- 172.21.15.119:0/3563119298 wait complete. 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.569+0000 7fc80679d700 1 Processor -- start 2023-10-08T01:08:51.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.569+0000 7fc80679d700 1 -- start start 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80679d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc800159420 0x7fc80014adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80679d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 0x7fc800154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80679d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80015d230 0x7fc80014f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80679d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001527d0 con 0x7fc8001597f0 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80679d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc800154740 con 0x7fc800159420 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80679d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc800151bc0 con 0x7fc80015d230 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80579b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc800159420 0x7fc80014adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc804f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 0x7fc800154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80579b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc800159420 0x7fc80014adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44694/0 (socket says 172.21.15.119:44694) 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc80579b700 1 -- 172.21.15.119:0/1874693250 learned_addr learned my addr 172.21.15.119:0/1874693250 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:51.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.570+0000 7fc805f9c700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80015d230 0x7fc80014f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc80579b700 1 -- 172.21.15.119:0/1874693250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80015d230 msgr2=0x7fc80014f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc80579b700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80015d230 0x7fc80014f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc80579b700 1 -- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 msgr2=0x7fc800154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc80579b700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 0x7fc800154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc80579b700 1 -- 172.21.15.119:0/1874693250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00094f0 con 0x7fc800159420 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc80579b700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc800159420 0x7fc80014adf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00ec80 tx=0x7fc7fc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc7f57fa700 1 -- 172.21.15.119:0/1874693250 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00c850 con 0x7fc800159420 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc7f57fa700 1 -- 172.21.15.119:0/1874693250 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7fc00c9b0 con 0x7fc800159420 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc800151ea0 con 0x7fc800159420 2023-10-08T01:08:51.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.571+0000 7fc7f57fa700 1 -- 172.21.15.119:0/1874693250 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00cd50 con 0x7fc800159420 2023-10-08T01:08:51.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.572+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc800152460 con 0x7fc800159420 2023-10-08T01:08:51.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.572+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80004c3c0 con 0x7fc800159420 2023-10-08T01:08:51.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.574+0000 7fc7f57fa700 1 -- 172.21.15.119:0/1874693250 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fc7fc005a00 con 0x7fc800159420 2023-10-08T01:08:51.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.575+0000 7fc7f57fa700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7e40670e0 0x7fc7e4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:51.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.575+0000 7fc7f57fa700 1 -- 172.21.15.119:0/1874693250 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6419+0+0 (secure 0 0 0) 0x7fc7fc084d80 con 0x7fc800159420 2023-10-08T01:08:51.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.575+0000 7fc804f9a700 1 -- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7e40670e0 msgr2=0x7fc7e4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:51.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.575+0000 7fc804f9a700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7e40670e0 0x7fc7e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:51.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.578+0000 7fc7f57fa700 1 -- 172.21.15.119:0/1874693250 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7fc051be0 con 0x7fc800159420 2023-10-08T01:08:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.776+0000 7fc804f9a700 1 -- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7e40670e0 msgr2=0x7fc7e4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.776+0000 7fc804f9a700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7e40670e0 0x7fc7e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:51.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.814+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8000b1d50 con 0x7fc800159420 2023-10-08T01:08:51.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.815+0000 7fc7f57fa700 1 -- 172.21.15.119:0/1874693250 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fc7fc021090 con 0x7fc800159420 2023-10-08T01:08:51.815 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:51.815 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/768996961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":768996961},{"type":"v1","addr":"172.21.15.184:6834","nonce":768996961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1876644893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1876644893},{"type":"v1","addr":"172.21.15.119:6837","nonce":1876644893}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1187803776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1187803776},{"type":"v1","addr":"172.21.15.184:6835","nonce":1187803776}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:56:39.582864+0000","modified":"2023-10-08T00:56:43.160226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3785845701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3785845701},{"type":"v1","addr":"172.21.15.119:6836","nonce":3785845701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.818+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7e40670e0 msgr2=0x7fc7e4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.818+0000 7fc80679d700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7e40670e0 0x7fc7e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.818+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc800159420 msgr2=0x7fc80014adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.818+0000 7fc80679d700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc800159420 0x7fc80014adf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc00ec80 tx=0x7fc7fc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.818+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 shutdown_connections 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.818+0000 7fc80679d700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc80015d230 0x7fc80014f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.818+0000 7fc80679d700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc7e40670e0 0x7fc7e4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.818+0000 7fc80679d700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc800159420 0x7fc80014adf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.819+0000 7fc80679d700 1 --2- 172.21.15.119:0/1874693250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc8001597f0 0x7fc800154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:51.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.819+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 >> 172.21.15.119:0/1874693250 conn(0x7fc8000d0660 msgr2=0x7fc800156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:51.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.819+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 shutdown_connections 2023-10-08T01:08:51.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:51.819+0000 7fc80679d700 1 -- 172.21.15.119:0/1874693250 wait complete. 2023-10-08T01:08:51.821 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2023-10-08T01:08:51.842 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:51.841+0000 7f21ab0f7180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:08:51.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:08:51.955 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:51.955+0000 7f662c6b2180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:08:52.012 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:52.012+0000 7f21ab0f7180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:08:52.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.049+0000 7f185f59e700 1 Processor -- start 2023-10-08T01:08:52.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.049+0000 7f185f59e700 1 -- start start 2023-10-08T01:08:52.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.050+0000 7f185f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f18600b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.050+0000 7f185f59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f186015d230 0x7f186015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.050+0000 7f185f59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186015dbd0 con 0x7f18601597f0 2023-10-08T01:08:52.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.050+0000 7f185f59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600b2980 con 0x7f186015d230 2023-10-08T01:08:52.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.050+0000 7f185f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600b11d0 con 0x7f1860159420 2023-10-08T01:08:52.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185ed9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f186015d230 0x7f186015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185e59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f18600b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185ed9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f186015d230 0x7f186015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44706/0 (socket says 172.21.15.119:44706) 2023-10-08T01:08:52.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185ed9d700 1 -- 172.21.15.119:0/2889953914 learned_addr learned my addr 172.21.15.119:0/2889953914 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:52.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185dd9b700 1 --1- 172.21.15.119:0/2889953914 >> v1:172.21.15.184:6789/0 conn(0x7f18601597f0 0x7f18600b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:59734/0 (socket says 172.21.15.119:59734) 2023-10-08T01:08:52.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185ed9d700 1 -- 172.21.15.119:0/2889953914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 msgr2=0x7f18600b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:52.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185ed9d700 1 --2- 172.21.15.119:0/2889953914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f18600b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185ed9d700 1 -- 172.21.15.119:0/2889953914 >> v1:172.21.15.184:6789/0 conn(0x7f18601597f0 legacy=0x7f18600b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185ed9d700 1 -- 172.21.15.119:0/2889953914 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600b14b0 con 0x7f186015d230 2023-10-08T01:08:52.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185e59c700 1 --2- 172.21.15.119:0/2889953914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f18600b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:52.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185d59a700 1 -- 172.21.15.119:0/2889953914 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999820774 0 0) 0x7f1854001110 con 0x7f18601597f0 2023-10-08T01:08:52.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.051+0000 7f185ed9d700 1 --2- 172.21.15.119:0/2889953914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186015d230 0x7f186015d690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1858009f20 tx=0x7f1858009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82d3be552b9cd54f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:52.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185d59a700 1 -- 172.21.15.119:0/2889953914 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800e070 con 0x7f186015d230 2023-10-08T01:08:52.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185d59a700 1 -- 172.21.15.119:0/2889953914 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185800b9d0 con 0x7f186015d230 2023-10-08T01:08:52.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185d59a700 1 -- 172.21.15.119:0/2889953914 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18580092e0 con 0x7f186015d230 2023-10-08T01:08:52.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185f59e700 1 -- 172.21.15.119:0/2889953914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186015d230 msgr2=0x7f186015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185f59e700 1 --2- 172.21.15.119:0/2889953914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186015d230 0x7f186015d690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1858009f20 tx=0x7f1858009840 comp rx=0 tx=0).stop 2023-10-08T01:08:52.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185f59e700 1 -- 172.21.15.119:0/2889953914 shutdown_connections 2023-10-08T01:08:52.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185f59e700 1 --2- 172.21.15.119:0/2889953914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f186015d230 0x7f186015d690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185f59e700 1 --2- 172.21.15.119:0/2889953914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f18600b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185f59e700 1 -- 172.21.15.119:0/2889953914 >> 172.21.15.119:0/2889953914 conn(0x7f18600d0660 msgr2=0x7f18600d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:52.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185f59e700 1 -- 172.21.15.119:0/2889953914 shutdown_connections 2023-10-08T01:08:52.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.052+0000 7f185f59e700 1 -- 172.21.15.119:0/2889953914 wait complete. 2023-10-08T01:08:52.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.054+0000 7f185f59e700 1 Processor -- start 2023-10-08T01:08:52.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.054+0000 7f185f59e700 1 -- start start 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f186015cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18601597f0 0x7f18601532b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186015d230 0x7f18600d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185f59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186015c6c0 con 0x7f186015d230 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186015bab0 con 0x7f18601597f0 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186015bd90 con 0x7f1860159420 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186015d230 0x7f18600d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185e59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f186015cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186015d230 0x7f18600d0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48448/0 (socket says 172.21.15.119:48448) 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185ed9d700 1 -- 172.21.15.119:0/10971083 learned_addr learned my addr 172.21.15.119:0/10971083 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185dd9b700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18601597f0 0x7f18601532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185ed9d700 1 -- 172.21.15.119:0/10971083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 msgr2=0x7f186015cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185ed9d700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f186015cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185ed9d700 1 -- 172.21.15.119:0/10971083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18601597f0 msgr2=0x7f18601532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185ed9d700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18601597f0 0x7f18601532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185ed9d700 1 -- 172.21.15.119:0/10971083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1850009420 con 0x7f186015d230 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.055+0000 7f185dd9b700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18601597f0 0x7f18601532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.056+0000 7f185ed9d700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186015d230 0x7f18600d0a30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1860089340 tx=0x7f1858018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.056+0000 7f184e7fc700 1 -- 172.21.15.119:0/10971083 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858008b10 con 0x7f186015d230 2023-10-08T01:08:52.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.056+0000 7f184e7fc700 1 -- 172.21.15.119:0/10971083 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858008c90 con 0x7f186015d230 2023-10-08T01:08:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.056+0000 7f184e7fc700 1 -- 172.21.15.119:0/10971083 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18580256b0 con 0x7f186015d230 2023-10-08T01:08:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.056+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1858009ae0 con 0x7f186015d230 2023-10-08T01:08:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.056+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186015c350 con 0x7f186015d230 2023-10-08T01:08:52.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.057+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186004c3c0 con 0x7f186015d230 2023-10-08T01:08:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.058+0000 7f184e7fc700 1 -- 172.21.15.119:0/10971083 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7f1858003040 con 0x7f186015d230 2023-10-08T01:08:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.059+0000 7f184e7fc700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18400670e0 0x7f1840069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.059+0000 7f184e7fc700 1 -- 172.21.15.119:0/10971083 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6419+0+0 (secure 0 0 0) 0x7f18580860c0 con 0x7f186015d230 2023-10-08T01:08:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.059+0000 7f185e59c700 1 -- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18400670e0 msgr2=0x7f1840069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:52.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.059+0000 7f185e59c700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18400670e0 0x7f1840069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:52.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.063+0000 7f184e7fc700 1 -- 172.21.15.119:0/10971083 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1858052d80 con 0x7f186015d230 2023-10-08T01:08:52.109 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:52.109+0000 7f21ab0f7180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:08:52.134 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:52.134+0000 7f662c6b2180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:08:52.233 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:52.234+0000 7f662c6b2180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:08:52.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.260+0000 7f185e59c700 1 -- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18400670e0 msgr2=0x7f1840069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:52.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.260+0000 7f185e59c700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18400670e0 0x7f1840069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:52.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.264+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18600b1790 con 0x7f186015d230 2023-10-08T01:08:52.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.265+0000 7f184e7fc700 1 -- 172.21.15.119:0/10971083 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16489 (secure 0 0 0) 0x7f18580563a0 con 0x7f186015d230 2023-10-08T01:08:52.265 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:52.266 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":45,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:08:49.135623+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:56:37.182342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:56:38.569786+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":235,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/538579956":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/2510601793":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/4274210250":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/1792355627":"2023-10-09T00:56:55.298790+0000","172.21.15.119:6832/85511":"2023-10-09T00:56:55.298790+0000","172.21.15.119:6833/85511":"2023-10-09T00:56:55.298790+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":9,"length":2},{"begin":14,"length":2},{"begin":19,"length":2},{"begin":24,"length":3},{"begin":31,"length":2},{"begin":36,"length":4},{"begin":42,"length":2},{"begin":47,"length":4},{"begin":53,"length":2},{"begin":58,"length":3},{"begin":63,"length":2},{"begin":68,"length":4},{"begin":74,"length":2},{"begin":80,"length":3},{"begin":85,"length":2},{"begin":91,"length":3},{"begin":96,"length":2},{"begin":99,"length":1},{"begin":102,"length":3},{"begin":107,"length":3},{"begin":112,"length":3},{"begin":117,"length":2},{"begin":177,"length":59}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":102,"length":1},{"begin":112,"length":1},{"begin":234,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:08:52.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18400670e0 msgr2=0x7f1840069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:52.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18400670e0 0x7f1840069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186015d230 msgr2=0x7f18600d0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186015d230 0x7f18600d0a30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1860089340 tx=0x7f1858018000 comp rx=0 tx=0).stop 2023-10-08T01:08:52.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 shutdown_connections 2023-10-08T01:08:52.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1860159420 0x7f186015cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f18400670e0 0x7f1840069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f186015d230 0x7f18600d0a30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 --2- 172.21.15.119:0/10971083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18601597f0 0x7f18601532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.267+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 >> 172.21.15.119:0/10971083 conn(0x7f18600d0660 msgr2=0x7f1860154230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:52.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.268+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 shutdown_connections 2023-10-08T01:08:52.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.268+0000 7f185f59e700 1 -- 172.21.15.119:0/10971083 wait complete. 2023-10-08T01:08:52.286 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T01:08:52.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:08:52.406 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:52.406+0000 7f21ab0f7180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:08:52.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.477+0000 7fc9b339d700 1 Processor -- start 2023-10-08T01:08:52.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.477+0000 7fc9b339d700 1 -- start start 2023-10-08T01:08:52.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.478+0000 7fc9b339d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc9ac1597f0 0x7fc9ac0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.478+0000 7fc9b339d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac0b11d0 con 0x7fc9ac15d230 2023-10-08T01:08:52.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.478+0000 7fc9b339d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9ac0b14b0 con 0x7fc9ac159420 2023-10-08T01:08:52.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.478+0000 7fc9b339d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9ac0b1790 con 0x7fc9ac1597f0 2023-10-08T01:08:52.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b2b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc9ac15d230 0x7fc9ac15d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37678/0 (socket says 172.21.15.119:37678) 2023-10-08T01:08:52.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b2b9c700 1 -- 172.21.15.119:0/1379706633 learned_addr learned my addr 172.21.15.119:0/1379706633 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:52.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b1b9a700 1 --2- 172.21.15.119:0/1379706633 >> v2:172.21.15.184:3300/0 conn(0x7fc9ac1597f0 0x7fc9ac0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b1399700 1 -- 172.21.15.119:0/1379706633 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039922948 0 0) 0x7fc9ac0b11d0 con 0x7fc9ac15d230 2023-10-08T01:08:52.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b1399700 1 -- 172.21.15.119:0/1379706633 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc998003570 con 0x7fc9ac15d230 2023-10-08T01:08:52.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b1399700 1 -- 172.21.15.119:0/1379706633 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 915849149 0 0) 0x7fc9ac0b14b0 con 0x7fc9ac159420 2023-10-08T01:08:52.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b1399700 1 -- 172.21.15.119:0/1379706633 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ac0b11d0 con 0x7fc9ac159420 2023-10-08T01:08:52.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b1b9a700 1 -- 172.21.15.119:0/1379706633 >> v1:172.21.15.184:6789/0 conn(0x7fc9ac159420 legacy=0x7fc9ac0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b1b9a700 1 -- 172.21.15.119:0/1379706633 >> v1:172.21.15.119:6789/0 conn(0x7fc9ac15d230 legacy=0x7fc9ac15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.479+0000 7fc9b1b9a700 1 -- 172.21.15.119:0/1379706633 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9ac0b1a70 con 0x7fc9ac1597f0 2023-10-08T01:08:52.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.480+0000 7fc9b1b9a700 1 --2- 172.21.15.119:0/1379706633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 0x7fc9ac0b2270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0098e0 tx=0x7fc99c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=813cade0688632db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:52.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.480+0000 7fc9b1399700 1 -- 172.21.15.119:0/1379706633 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c011630 con 0x7fc9ac1597f0 2023-10-08T01:08:52.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.480+0000 7fc9b1399700 1 -- 172.21.15.119:0/1379706633 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c011790 con 0x7fc9ac1597f0 2023-10-08T01:08:52.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.480+0000 7fc9b1399700 1 -- 172.21.15.119:0/1379706633 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c011b30 con 0x7fc9ac1597f0 2023-10-08T01:08:52.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.480+0000 7fc9b339d700 1 -- 172.21.15.119:0/1379706633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 msgr2=0x7fc9ac0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.480+0000 7fc9b339d700 1 --2- 172.21.15.119:0/1379706633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 0x7fc9ac0b2270 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc99c0098e0 tx=0x7fc99c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.481+0000 7fc9b339d700 1 -- 172.21.15.119:0/1379706633 shutdown_connections 2023-10-08T01:08:52.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.481+0000 7fc9b339d700 1 --2- 172.21.15.119:0/1379706633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 0x7fc9ac0b2270 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.481+0000 7fc9b339d700 1 -- 172.21.15.119:0/1379706633 >> 172.21.15.119:0/1379706633 conn(0x7fc9ac0d0660 msgr2=0x7fc9ac0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:52.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.481+0000 7fc9b339d700 1 -- 172.21.15.119:0/1379706633 shutdown_connections 2023-10-08T01:08:52.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.481+0000 7fc9b339d700 1 -- 172.21.15.119:0/1379706633 wait complete. 2023-10-08T01:08:52.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.482+0000 7fc9b339d700 1 Processor -- start 2023-10-08T01:08:52.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.482+0000 7fc9b339d700 1 -- start start 2023-10-08T01:08:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b339d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9ac159420 0x7fc9ac15c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b339d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 0x7fc9ac068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9ac15d230 0x7fc9ac0690f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b339d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0c6fe0 con 0x7fc9ac159420 2023-10-08T01:08:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b339d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0c6360 con 0x7fc9ac1597f0 2023-10-08T01:08:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ac0c6640 con 0x7fc9ac15d230 2023-10-08T01:08:52.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b239b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9ac159420 0x7fc9ac15c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b239b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9ac159420 0x7fc9ac15c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48456/0 (socket says 172.21.15.119:48456) 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.483+0000 7fc9b239b700 1 -- 172.21.15.119:0/873029932 learned_addr learned my addr 172.21.15.119:0/873029932 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b1b9a700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 0x7fc9ac068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b2b9c700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9ac15d230 0x7fc9ac0690f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b239b700 1 -- 172.21.15.119:0/873029932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9ac15d230 msgr2=0x7fc9ac0690f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b239b700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9ac15d230 0x7fc9ac0690f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b239b700 1 -- 172.21.15.119:0/873029932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 msgr2=0x7fc9ac068bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b239b700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 0x7fc9ac068bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b239b700 1 -- 172.21.15.119:0/873029932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc99c009420 con 0x7fc9ac159420 2023-10-08T01:08:52.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b239b700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9ac159420 0x7fc9ac15c750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c790 tx=0x7fc9a8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9a27fc700 1 -- 172.21.15.119:0/873029932 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8010510 con 0x7fc9ac159420 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9a27fc700 1 -- 172.21.15.119:0/873029932 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9a8010670 con 0x7fc9ac159420 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ac0c6920 con 0x7fc9ac159420 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.484+0000 7fc9a27fc700 1 -- 172.21.15.119:0/873029932 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8010a10 con 0x7fc9ac159420 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.485+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ac15cc90 con 0x7fc9ac159420 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.485+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9ac0c7120 con 0x7fc9ac159420 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.486+0000 7fc9a27fc700 1 -- 172.21.15.119:0/873029932 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fc9a801b3b0 con 0x7fc9ac159420 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.487+0000 7fc9a27fc700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc9900670e0 0x7fc990069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.487+0000 7fc9a27fc700 1 -- 172.21.15.119:0/873029932 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6419+0+0 (secure 0 0 0) 0x7fc9a80843c0 con 0x7fc9ac159420 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.487+0000 7fc9b1b9a700 1 -- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc9900670e0 msgr2=0x7fc990069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.487+0000 7fc9b1b9a700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc9900670e0 0x7fc990069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.495+0000 7fc9a27fc700 1 -- 172.21.15.119:0/873029932 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9a80510e0 con 0x7fc9ac159420 2023-10-08T01:08:52.502 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:52.502+0000 7f21ab0f7180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:08:52.542 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:52.542+0000 7f662c6b2180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:08:52.640 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:52.640+0000 7f662c6b2180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:08:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.688+0000 7fc9b1b9a700 1 -- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc9900670e0 msgr2=0x7fc990069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.688+0000 7fc9b1b9a700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc9900670e0 0x7fc990069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:52.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.736+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9ac04c3c0 con 0x7fc9ac159420 2023-10-08T01:08:52.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.737+0000 7fc9a27fc700 1 -- 172.21.15.119:0/873029932 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4606 (secure 0 0 0) 0x7fc9a8054700 con 0x7fc9ac159420 2023-10-08T01:08:52.737 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:52.737 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/768996961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":768996961},{"type":"v1","addr":"172.21.15.184:6834","nonce":768996961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1876644893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1876644893},{"type":"v1","addr":"172.21.15.119:6837","nonce":1876644893}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1187803776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1187803776},{"type":"v1","addr":"172.21.15.184:6835","nonce":1187803776}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:56:39.582864+0000","modified":"2023-10-08T00:56:43.160226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3785845701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3785845701},{"type":"v1","addr":"172.21.15.119:6836","nonce":3785845701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T01:08:52.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc9900670e0 msgr2=0x7fc990069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:52.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc9900670e0 0x7fc990069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9ac159420 msgr2=0x7fc9ac15c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9ac159420 0x7fc9ac15c750 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800c790 tx=0x7fc9a8007e10 comp rx=0 tx=0).stop 2023-10-08T01:08:52.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 shutdown_connections 2023-10-08T01:08:52.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc9ac15d230 0x7fc9ac0690f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fc9900670e0 0x7fc990069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9ac159420 0x7fc9ac15c750 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 --2- 172.21.15.119:0/873029932 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc9ac1597f0 0x7fc9ac068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 >> 172.21.15.119:0/873029932 conn(0x7fc9ac0d0660 msgr2=0x7fc9ac155240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:52.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 shutdown_connections 2023-10-08T01:08:52.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.740+0000 7fc9b339d700 1 -- 172.21.15.119:0/873029932 wait complete. 2023-10-08T01:08:52.743 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2023-10-08T01:08:52.757 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:08:52.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.944+0000 7ffb91b9d700 1 Processor -- start 2023-10-08T01:08:52.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.944+0000 7ffb91b9d700 1 -- start start 2023-10-08T01:08:52.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.945+0000 7ffb91b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c1597f0 0x7ffb8c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.945+0000 7ffb91b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffb8c15d230 0x7ffb8c15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.945+0000 7ffb91b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c15dbd0 con 0x7ffb8c159420 2023-10-08T01:08:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.945+0000 7ffb91b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b11d0 con 0x7ffb8c15d230 2023-10-08T01:08:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.945+0000 7ffb91b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b14b0 con 0x7ffb8c1597f0 2023-10-08T01:08:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb90b9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffb8c159420 0x7ffb8c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37680/0 (socket says 172.21.15.119:37680) 2023-10-08T01:08:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb90b9b700 1 -- 172.21.15.119:0/795716280 learned_addr learned my addr 172.21.15.119:0/795716280 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb8bfff700 1 --2- 172.21.15.119:0/795716280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c1597f0 0x7ffb8c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb9139c700 1 --2- 172.21.15.119:0/795716280 >> v2:172.21.15.184:3300/0 conn(0x7ffb8c15d230 0x7ffb8c15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb8b7fe700 1 -- 172.21.15.119:0/795716280 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516951206 0 0) 0x7ffb80001110 con 0x7ffb8c159420 2023-10-08T01:08:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb8b7fe700 1 -- 172.21.15.119:0/795716280 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb7c003570 con 0x7ffb8c159420 2023-10-08T01:08:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb8bfff700 1 -- 172.21.15.119:0/795716280 >> v2:172.21.15.184:3300/0 conn(0x7ffb8c15d230 msgr2=0x7ffb8c15d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb8bfff700 1 --2- 172.21.15.119:0/795716280 >> v2:172.21.15.184:3300/0 conn(0x7ffb8c15d230 0x7ffb8c15d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb8bfff700 1 -- 172.21.15.119:0/795716280 >> v1:172.21.15.119:6789/0 conn(0x7ffb8c159420 legacy=0x7ffb8c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb8bfff700 1 -- 172.21.15.119:0/795716280 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0b1790 con 0x7ffb8c1597f0 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.946+0000 7ffb9139c700 1 --2- 172.21.15.119:0/795716280 >> v2:172.21.15.184:3300/0 conn(0x7ffb8c15d230 0x7ffb8c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.947+0000 7ffb8bfff700 1 --2- 172.21.15.119:0/795716280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c1597f0 0x7ffb8c0b2270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb780098e0 tx=0x7ffb780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=913ecdff1f93693d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.947+0000 7ffb8b7fe700 1 -- 172.21.15.119:0/795716280 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78011630 con 0x7ffb8c1597f0 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.947+0000 7ffb8b7fe700 1 -- 172.21.15.119:0/795716280 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb78011790 con 0x7ffb8c1597f0 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.947+0000 7ffb8b7fe700 1 -- 172.21.15.119:0/795716280 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb78011b30 con 0x7ffb8c1597f0 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.947+0000 7ffb91b9d700 1 -- 172.21.15.119:0/795716280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c1597f0 msgr2=0x7ffb8c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.947+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/795716280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c1597f0 0x7ffb8c0b2270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb780098e0 tx=0x7ffb780091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.948+0000 7ffb91b9d700 1 -- 172.21.15.119:0/795716280 shutdown_connections 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.948+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/795716280 >> v2:172.21.15.184:3300/0 conn(0x7ffb8c15d230 0x7ffb8c15d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.948+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/795716280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c1597f0 0x7ffb8c0b2270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.948+0000 7ffb91b9d700 1 -- 172.21.15.119:0/795716280 >> 172.21.15.119:0/795716280 conn(0x7ffb8c0d0660 msgr2=0x7ffb8c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.948+0000 7ffb91b9d700 1 -- 172.21.15.119:0/795716280 shutdown_connections 2023-10-08T01:08:52.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.948+0000 7ffb91b9d700 1 -- 172.21.15.119:0/795716280 wait complete. 2023-10-08T01:08:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.949+0000 7ffb91b9d700 1 Processor -- start 2023-10-08T01:08:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.949+0000 7ffb91b9d700 1 -- start start 2023-10-08T01:08:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb91b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c159420 0x7ffb8c0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb91b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb8c1597f0 0x7ffb8c148f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb91b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb8c15d230 0x7ffb8c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb91b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0c67f0 con 0x7ffb8c15d230 2023-10-08T01:08:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb91b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0c5940 con 0x7ffb8c1597f0 2023-10-08T01:08:52.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb91b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0c5be0 con 0x7ffb8c159420 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb9139c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb8c15d230 0x7ffb8c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb9139c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb8c15d230 0x7ffb8c155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48468/0 (socket says 172.21.15.119:48468) 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb9139c700 1 -- 172.21.15.119:0/2844954937 learned_addr learned my addr 172.21.15.119:0/2844954937 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb90b9b700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c159420 0x7ffb8c0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.950+0000 7ffb8bfff700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb8c1597f0 0x7ffb8c148f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb9139c700 1 -- 172.21.15.119:0/2844954937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c159420 msgr2=0x7ffb8c0c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb9139c700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c159420 0x7ffb8c0c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb9139c700 1 -- 172.21.15.119:0/2844954937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb8c1597f0 msgr2=0x7ffb8c148f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb9139c700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb8c1597f0 0x7ffb8c148f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:52.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb9139c700 1 -- 172.21.15.119:0/2844954937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb84009a20 con 0x7ffb8c15d230 2023-10-08T01:08:52.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb9139c700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb8c15d230 0x7ffb8c155cf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009f20 tx=0x7ffb8400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:52.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb88ff9700 1 -- 172.21.15.119:0/2844954937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8400cb70 con 0x7ffb8c15d230 2023-10-08T01:08:52.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb88ff9700 1 -- 172.21.15.119:0/2844954937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8400f040 con 0x7ffb8c15d230 2023-10-08T01:08:52.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb78009420 con 0x7ffb8c15d230 2023-10-08T01:08:52.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.951+0000 7ffb88ff9700 1 -- 172.21.15.119:0/2844954937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb84014480 con 0x7ffb8c15d230 2023-10-08T01:08:52.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.952+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0c61a0 con 0x7ffb8c15d230 2023-10-08T01:08:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.953+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c1696a0 con 0x7ffb8c15d230 2023-10-08T01:08:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.953+0000 7ffb88ff9700 1 -- 172.21.15.119:0/2844954937 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7ffb84003020 con 0x7ffb8c15d230 2023-10-08T01:08:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.954+0000 7ffb88ff9700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffb700670e0 0x7ffb70069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.954+0000 7ffb88ff9700 1 -- 172.21.15.119:0/2844954937 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6419+0+0 (secure 0 0 0) 0x7ffb84085590 con 0x7ffb8c15d230 2023-10-08T01:08:52.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.954+0000 7ffb90b9b700 1 -- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffb700670e0 msgr2=0x7ffb70069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:52.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.954+0000 7ffb90b9b700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffb700670e0 0x7ffb70069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:52.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:52.958+0000 7ffb88ff9700 1 -- 172.21.15.119:0/2844954937 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffb840521a0 con 0x7ffb8c15d230 2023-10-08T01:08:53.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.155+0000 7ffb90b9b700 1 -- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffb700670e0 msgr2=0x7ffb70069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:53.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.155+0000 7ffb90b9b700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffb700670e0 0x7ffb70069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:53.205 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:53.204+0000 7f21ab0f7180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:08:53.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.228+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb8c04c3c0 con 0x7ffb8c15d230 2023-10-08T01:08:53.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.229+0000 7ffb88ff9700 1 -- 172.21.15.119:0/2844954937 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4606 (secure 0 0 0) 0x7ffb840557c0 con 0x7ffb8c15d230 2023-10-08T01:08:53.229 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:53.229 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/768996961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":768996961},{"type":"v1","addr":"172.21.15.184:6834","nonce":768996961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1876644893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1876644893},{"type":"v1","addr":"172.21.15.119:6837","nonce":1876644893}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1187803776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1187803776},{"type":"v1","addr":"172.21.15.184:6835","nonce":1187803776}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T00:56:39.582864+0000","modified":"2023-10-08T00:56:43.160226+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4938":{"gid":4938,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.119:6836/3785845701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3785845701},{"type":"v1","addr":"172.21.15.119:6836","nonce":3785845701}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-08T01:08:53.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffb700670e0 msgr2=0x7ffb70069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffb700670e0 0x7ffb70069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb8c15d230 msgr2=0x7ffb8c155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb8c15d230 0x7ffb8c155cf0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009f20 tx=0x7ffb8400b040 comp rx=0 tx=0).stop 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 shutdown_connections 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffb8c159420 0x7ffb8c0c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ffb700670e0 0x7ffb70069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffb8c15d230 0x7ffb8c155cf0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 --2- 172.21.15.119:0/2844954937 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffb8c1597f0 0x7ffb8c148f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 >> 172.21.15.119:0/2844954937 conn(0x7ffb8c0d0660 msgr2=0x7ffb8c0b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:53.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.231+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 shutdown_connections 2023-10-08T01:08:53.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.232+0000 7ffb91b9d700 1 -- 172.21.15.119:0/2844954937 wait complete. 2023-10-08T01:08:53.233 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 16 2023-10-08T01:08:53.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:08:53.396 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:53.396+0000 7f662c6b2180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:08:53.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.437+0000 7ff22759d700 1 Processor -- start 2023-10-08T01:08:53.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.437+0000 7ff22759d700 1 -- start start 2023-10-08T01:08:53.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22759d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff220159420 0x7ff2200b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22759d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2201597f0 0x7ff2200b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22759d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2200b11d0 con 0x7ff22015d230 2023-10-08T01:08:53.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22759d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2200b14b0 con 0x7ff220159420 2023-10-08T01:08:53.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22759d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200b1790 con 0x7ff2201597f0 2023-10-08T01:08:53.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22659b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff220159420 0x7ff2200b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22659b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff220159420 0x7ff2200b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48484/0 (socket says 172.21.15.119:48484) 2023-10-08T01:08:53.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff225d9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2201597f0 0x7ff2200b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22659b700 1 -- 172.21.15.119:0/195054396 learned_addr learned my addr 172.21.15.119:0/195054396 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22659b700 1 -- 172.21.15.119:0/195054396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2201597f0 msgr2=0x7ff2200b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22659b700 1 --2- 172.21.15.119:0/195054396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2201597f0 0x7ff2200b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff22659b700 1 -- 172.21.15.119:0/195054396 >> v1:172.21.15.184:6789/0 conn(0x7ff22015d230 legacy=0x7ff2200b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.439+0000 7ff225599700 1 -- 172.21.15.119:0/195054396 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2043289531 0 0) 0x7ff2200b11d0 con 0x7ff22015d230 2023-10-08T01:08:53.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.440+0000 7ff22659b700 1 -- 172.21.15.119:0/195054396 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2200b1a70 con 0x7ff220159420 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.440+0000 7ff22659b700 1 --2- 172.21.15.119:0/195054396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff220159420 0x7ff2200b5f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0099b0 tx=0x7ff21c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981f33f2f09cc4d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.440+0000 7ff225599700 1 -- 172.21.15.119:0/195054396 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c009ea0 con 0x7ff220159420 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.440+0000 7ff225599700 1 -- 172.21.15.119:0/195054396 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c00b860 con 0x7ff220159420 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.440+0000 7ff225599700 1 -- 172.21.15.119:0/195054396 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c00bc00 con 0x7ff220159420 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.441+0000 7ff22759d700 1 -- 172.21.15.119:0/195054396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff220159420 msgr2=0x7ff2200b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.441+0000 7ff22759d700 1 --2- 172.21.15.119:0/195054396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff220159420 0x7ff2200b5f40 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff21c0099b0 tx=0x7ff21c009250 comp rx=0 tx=0).stop 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.441+0000 7ff22759d700 1 -- 172.21.15.119:0/195054396 shutdown_connections 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.441+0000 7ff22759d700 1 --2- 172.21.15.119:0/195054396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff220159420 0x7ff2200b5f40 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.441+0000 7ff22759d700 1 --2- 172.21.15.119:0/195054396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff2201597f0 0x7ff2200b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.441+0000 7ff22759d700 1 -- 172.21.15.119:0/195054396 >> 172.21.15.119:0/195054396 conn(0x7ff2200d0660 msgr2=0x7ff2200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:53.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.441+0000 7ff22759d700 1 -- 172.21.15.119:0/195054396 shutdown_connections 2023-10-08T01:08:53.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.441+0000 7ff22759d700 1 -- 172.21.15.119:0/195054396 wait complete. 2023-10-08T01:08:53.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.442+0000 7ff22759d700 1 Processor -- start 2023-10-08T01:08:53.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.442+0000 7ff22759d700 1 -- start start 2023-10-08T01:08:53.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.443+0000 7ff22759d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff220159420 0x7ff2200c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.443+0000 7ff22759d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2201597f0 0x7ff2200cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.443+0000 7ff22759d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff22015d230 0x7ff2200c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.443+0000 7ff22759d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200c8090 con 0x7ff2201597f0 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.443+0000 7ff22759d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200c7480 con 0x7ff220159420 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.443+0000 7ff22759d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2200c7760 con 0x7ff22015d230 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.443+0000 7ff225d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2201597f0 0x7ff2200cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff225d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2201597f0 0x7ff2200cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48486/0 (socket says 172.21.15.119:48486) 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff225d9a700 1 -- 172.21.15.119:0/2628363126 learned_addr learned my addr 172.21.15.119:0/2628363126 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff22659b700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff220159420 0x7ff2200c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff226d9c700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff22015d230 0x7ff2200c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff225d9a700 1 -- 172.21.15.119:0/2628363126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff22015d230 msgr2=0x7ff2200c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff225d9a700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff22015d230 0x7ff2200c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff225d9a700 1 -- 172.21.15.119:0/2628363126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff220159420 msgr2=0x7ff2200c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff225d9a700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff220159420 0x7ff2200c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff225d9a700 1 -- 172.21.15.119:0/2628363126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0094f0 con 0x7ff2201597f0 2023-10-08T01:08:53.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff225d9a700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2201597f0 0x7ff2200cb450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff21000ec80 tx=0x7ff2100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:53.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff2167fc700 1 -- 172.21.15.119:0/2628363126 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000c850 con 0x7ff2201597f0 2023-10-08T01:08:53.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff2167fc700 1 -- 172.21.15.119:0/2628363126 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21000c9b0 con 0x7ff2201597f0 2023-10-08T01:08:53.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2200c7a40 con 0x7ff2201597f0 2023-10-08T01:08:53.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.444+0000 7ff2167fc700 1 -- 172.21.15.119:0/2628363126 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21000cd50 con 0x7ff2201597f0 2023-10-08T01:08:53.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.445+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2200c5d00 con 0x7ff2201597f0 2023-10-08T01:08:53.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.446+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2200c8200 con 0x7ff2201597f0 2023-10-08T01:08:53.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.446+0000 7ff2167fc700 1 -- 172.21.15.119:0/2628363126 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7ff210013680 con 0x7ff2201597f0 2023-10-08T01:08:53.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.447+0000 7ff2167fc700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2040670e0 0x7ff204069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.447+0000 7ff2167fc700 1 -- 172.21.15.119:0/2628363126 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6723+0+0 (secure 0 0 0) 0x7ff210084eb0 con 0x7ff2201597f0 2023-10-08T01:08:53.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.447+0000 7ff22659b700 1 -- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2040670e0 msgr2=0x7ff204069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:53.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.447+0000 7ff22659b700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2040670e0 0x7ff204069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:53.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.452+0000 7ff2167fc700 1 -- 172.21.15.119:0/2628363126 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff210051aa0 con 0x7ff2201597f0 2023-10-08T01:08:53.494 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:53.494+0000 7f21ab0f7180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:08:53.589 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:53.589+0000 7f21ab0f7180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:08:53.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.648+0000 7ff22659b700 1 -- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2040670e0 msgr2=0x7ff204069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:53.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.648+0000 7ff22659b700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2040670e0 0x7ff204069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:53.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.656+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff22004c3c0 con 0x7ff2201597f0 2023-10-08T01:08:53.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.657+0000 7ff2167fc700 1 -- 172.21.15.119:0/2628363126 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16948 (secure 0 0 0) 0x7ff2100550c0 con 0x7ff2201597f0 2023-10-08T01:08:53.658 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:08:53.658 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":46,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:08:53.126199+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-10-08T00:56:37.182342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-10-08T00:56:38.569786+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":235,"snap_epoch":45,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/538579956":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/2510601793":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/4274210250":"2023-10-09T00:56:55.298790+0000","172.21.15.119:0/1792355627":"2023-10-09T00:56:55.298790+0000","172.21.15.119:6832/85511":"2023-10-09T00:56:55.298790+0000","172.21.15.119:6833/85511":"2023-10-09T00:56:55.298790+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":7,"length":1},{"begin":10,"length":1},{"begin":15,"length":1},{"begin":20,"length":1},{"begin":25,"length":2},{"begin":32,"length":1},{"begin":37,"length":3},{"begin":43,"length":1},{"begin":48,"length":3},{"begin":54,"length":1},{"begin":59,"length":2},{"begin":64,"length":1},{"begin":70,"length":2},{"begin":75,"length":1},{"begin":81,"length":2},{"begin":86,"length":1},{"begin":92,"length":2},{"begin":97,"length":1},{"begin":99,"length":1},{"begin":103,"length":2},{"begin":108,"length":2},{"begin":114,"length":1},{"begin":118,"length":1},{"begin":177,"length":36}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:08:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2040670e0 msgr2=0x7ff204069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2040670e0 0x7ff204069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2201597f0 msgr2=0x7ff2200cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2201597f0 0x7ff2200cb450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff21000ec80 tx=0x7ff2100091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 shutdown_connections 2023-10-08T01:08:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff22015d230 0x7ff2200c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7ff2040670e0 0x7ff204069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff220159420 0x7ff2200c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 --2- 172.21.15.119:0/2628363126 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2201597f0 0x7ff2200cb450 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 >> 172.21.15.119:0/2628363126 conn(0x7ff2200d0660 msgr2=0x7ff2200b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:53.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 shutdown_connections 2023-10-08T01:08:53.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.660+0000 7ff22759d700 1 -- 172.21.15.119:0/2628363126 wait complete. 2023-10-08T01:08:53.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T01:08:53.693 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:53.693+0000 7f662c6b2180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:08:53.743 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:53.743+0000 7f21ab0f7180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:08:53.791 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:53.791+0000 7f662c6b2180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:08:53.848 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:53.847+0000 7f21ab0f7180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:08:53.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.865+0000 7fee4bfff700 1 Processor -- start 2023-10-08T01:08:53.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.865+0000 7fee4bfff700 1 -- start start 2023-10-08T01:08:53.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.866+0000 7fee4bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee4c159420 0x7fee4c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4bfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c0b2980 con 0x7fee4c1597f0 2023-10-08T01:08:53.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4bfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee4c15dbb0 con 0x7fee4c159420 2023-10-08T01:08:53.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4bfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c0b11d0 con 0x7fee4c15d230 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee4c159420 0x7fee4c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4a7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fee4c1597f0 0x7fee4c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37688/0 (socket says 172.21.15.119:37688) 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4a7fc700 1 -- 172.21.15.119:0/474098068 learned_addr learned my addr 172.21.15.119:0/474098068 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4b7fe700 1 --2- 172.21.15.119:0/474098068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee49ffb700 1 -- 172.21.15.119:0/474098068 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561311437 0 0) 0x7fee4c0b2980 con 0x7fee4c1597f0 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee49ffb700 1 -- 172.21.15.119:0/474098068 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee34003570 con 0x7fee4c1597f0 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4affd700 1 -- 172.21.15.119:0/474098068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 msgr2=0x7fee4c15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4affd700 1 --2- 172.21.15.119:0/474098068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4affd700 1 -- 172.21.15.119:0/474098068 >> v1:172.21.15.119:6789/0 conn(0x7fee4c1597f0 legacy=0x7fee4c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.867+0000 7fee4affd700 1 -- 172.21.15.119:0/474098068 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0b14b0 con 0x7fee4c159420 2023-10-08T01:08:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.868+0000 7fee4affd700 1 --2- 172.21.15.119:0/474098068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c0b5f40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0098e0 tx=0x7fee3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90b08c502ad7df15 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:53.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.868+0000 7fee49ffb700 1 -- 172.21.15.119:0/474098068 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c011630 con 0x7fee4c159420 2023-10-08T01:08:53.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.868+0000 7fee49ffb700 1 -- 172.21.15.119:0/474098068 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c011790 con 0x7fee4c159420 2023-10-08T01:08:53.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.868+0000 7fee49ffb700 1 -- 172.21.15.119:0/474098068 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c011b30 con 0x7fee4c159420 2023-10-08T01:08:53.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.869+0000 7fee4bfff700 1 -- 172.21.15.119:0/474098068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 msgr2=0x7fee4c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.869+0000 7fee4bfff700 1 --2- 172.21.15.119:0/474098068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c0b5f40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fee3c0098e0 tx=0x7fee3c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.869+0000 7fee4bfff700 1 -- 172.21.15.119:0/474098068 shutdown_connections 2023-10-08T01:08:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.869+0000 7fee4bfff700 1 --2- 172.21.15.119:0/474098068 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.869+0000 7fee4bfff700 1 --2- 172.21.15.119:0/474098068 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c0b5f40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.869+0000 7fee4bfff700 1 -- 172.21.15.119:0/474098068 >> 172.21.15.119:0/474098068 conn(0x7fee4c0d0660 msgr2=0x7fee4c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.869+0000 7fee4bfff700 1 -- 172.21.15.119:0/474098068 shutdown_connections 2023-10-08T01:08:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.869+0000 7fee4bfff700 1 -- 172.21.15.119:0/474098068 wait complete. 2023-10-08T01:08:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.870+0000 7fee4bfff700 1 Processor -- start 2023-10-08T01:08:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.870+0000 7fee4bfff700 1 -- start start 2023-10-08T01:08:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee4c1597f0 0x7fee4c0c7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c0c7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4bfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15b560 con 0x7fee4c159420 2023-10-08T01:08:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4bfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c157810 con 0x7fee4c1597f0 2023-10-08T01:08:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4bfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c15a950 con 0x7fee4c15d230 2023-10-08T01:08:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48498/0 (socket says 172.21.15.119:48498) 2023-10-08T01:08:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.871+0000 7fee4affd700 1 -- 172.21.15.119:0/616264592 learned_addr learned my addr 172.21.15.119:0/616264592 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4a7fc700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee4c1597f0 0x7fee4c0c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4b7fe700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c0c7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4affd700 1 -- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 msgr2=0x7fee4c0c7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4affd700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c0c7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4affd700 1 -- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee4c1597f0 msgr2=0x7fee4c0c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4affd700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee4c1597f0 0x7fee4c0c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4affd700 1 -- 172.21.15.119:0/616264592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c009420 con 0x7fee4c159420 2023-10-08T01:08:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4a7fc700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee4c1597f0 0x7fee4c0c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4b7fe700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c0c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4affd700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c156f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fee4c089340 tx=0x7fee3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee32ffd700 1 -- 172.21.15.119:0/616264592 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c011b10 con 0x7fee4c159420 2023-10-08T01:08:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee32ffd700 1 -- 172.21.15.119:0/616264592 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c0253f0 con 0x7fee4c159420 2023-10-08T01:08:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee4c15ac30 con 0x7fee4c159420 2023-10-08T01:08:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.872+0000 7fee32ffd700 1 -- 172.21.15.119:0/616264592 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0257b0 con 0x7fee4c159420 2023-10-08T01:08:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.873+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c15b1f0 con 0x7fee4c159420 2023-10-08T01:08:53.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.874+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c04c3c0 con 0x7fee4c159420 2023-10-08T01:08:53.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.875+0000 7fee32ffd700 1 -- 172.21.15.119:0/616264592 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7fee3c025ba0 con 0x7fee4c159420 2023-10-08T01:08:53.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.875+0000 7fee32ffd700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee380670e0 0x7fee38069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:53.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.875+0000 7fee32ffd700 1 -- 172.21.15.119:0/616264592 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6723+0+0 (secure 0 0 0) 0x7fee3c086580 con 0x7fee4c159420 2023-10-08T01:08:53.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.876+0000 7fee4a7fc700 1 -- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee380670e0 msgr2=0x7fee38069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:53.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.876+0000 7fee4a7fc700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee380670e0 0x7fee38069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:08:53.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:53.884+0000 7fee32ffd700 1 -- 172.21.15.119:0/616264592 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee3c053170 con 0x7fee4c159420 2023-10-08T01:08:53.953 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:53.953+0000 7f662c6b2180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:08:54.035 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:54.035+0000 7f21ab0f7180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:08:54.057 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.057+0000 7f662c6b2180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:08:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.077+0000 7fee4a7fc700 1 -- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee380670e0 msgr2=0x7fee38069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6832/85511 2023-10-08T01:08:54.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.077+0000 7fee4a7fc700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee380670e0 0x7fee38069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:08:54.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.124+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fee4c157a40 con 0x7fee4c159420 2023-10-08T01:08:54.125 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T01:08:54.125+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T01:08:54.127 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:54.126+0000 7f21ab0f7180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:08:54.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.132+0000 7fee32ffd700 1 -- 172.21.15.119:0/616264592 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7fee3c056790 con 0x7fee4c159420 2023-10-08T01:08:54.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee380670e0 msgr2=0x7fee38069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:08:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee380670e0 0x7fee38069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 msgr2=0x7fee4c156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c156f40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fee4c089340 tx=0x7fee3c017000 comp rx=0 tx=0).stop 2023-10-08T01:08:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 shutdown_connections 2023-10-08T01:08:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee4c15d230 0x7fee4c0c7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7fee380670e0 0x7fee38069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee4c159420 0x7fee4c156f40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 --2- 172.21.15.119:0/616264592 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee4c1597f0 0x7fee4c0c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 >> 172.21.15.119:0/616264592 conn(0x7fee4c0d0660 msgr2=0x7fee4c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:54.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.134+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 shutdown_connections 2023-10-08T01:08:54.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.135+0000 7fee4bfff700 1 -- 172.21.15.119:0/616264592 wait complete. 2023-10-08T01:08:54.136 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T01:08:54.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T01:08:54.189 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:08:54.220 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:54.220+0000 7f21ab0f7180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:08:54.261 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.262+0000 7f662c6b2180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:08:54.315 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:08:54.315+0000 7f21ab0f7180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:08:54.357 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.357+0000 7f662c6b2180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:08:54.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.381+0000 7f2549b9d700 1 Processor -- start 2023-10-08T01:08:54.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.382+0000 7f2549b9d700 1 -- start start 2023-10-08T01:08:54.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.383+0000 7f2549b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f25440b16d0 0x7f25440b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:54.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.383+0000 7f2549b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25440b2080 con 0x7f25440b4e70 2023-10-08T01:08:54.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.383+0000 7f2549b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25440cfb00 con 0x7f25440b33b0 2023-10-08T01:08:54.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.383+0000 7f2549b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25440b5c40 con 0x7f25440b16d0 2023-10-08T01:08:54.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.383+0000 7f2548b9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f25440b4e70 0x7f25440ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37698/0 (socket says 172.21.15.119:37698) 2023-10-08T01:08:54.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.383+0000 7f2548b9b700 1 -- 172.21.15.119:0/2727740210 learned_addr learned my addr 172.21.15.119:0/2727740210 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:54.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.383+0000 7f2543fff700 1 --2- 172.21.15.119:0/2727740210 >> v2:172.21.15.184:3300/0 conn(0x7f25440b16d0 0x7f25440b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:54.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.383+0000 7f25437fe700 1 -- 172.21.15.119:0/2727740210 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774759884 0 0) 0x7f25440b2080 con 0x7f25440b4e70 2023-10-08T01:08:54.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f25437fe700 1 -- 172.21.15.119:0/2727740210 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2534003570 con 0x7f25440b4e70 2023-10-08T01:08:54.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f25437fe700 1 -- 172.21.15.119:0/2727740210 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693108262 0 0) 0x7f25440cfb00 con 0x7f25440b33b0 2023-10-08T01:08:54.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f25437fe700 1 -- 172.21.15.119:0/2727740210 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25440b2080 con 0x7f25440b33b0 2023-10-08T01:08:54.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f2543fff700 1 -- 172.21.15.119:0/2727740210 >> v1:172.21.15.184:6789/0 conn(0x7f25440b33b0 legacy=0x7f25440b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:54.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f2543fff700 1 -- 172.21.15.119:0/2727740210 >> v1:172.21.15.119:6789/0 conn(0x7f25440b4e70 legacy=0x7f25440ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f2543fff700 1 -- 172.21.15.119:0/2727740210 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25440b5f20 con 0x7f25440b16d0 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f2543fff700 1 --2- 172.21.15.119:0/2727740210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b16d0 0x7f25440b1b40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f25300098e0 tx=0x7f25300091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9e0d383077921e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f25437fe700 1 -- 172.21.15.119:0/2727740210 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530011630 con 0x7f25440b16d0 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f25437fe700 1 -- 172.21.15.119:0/2727740210 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2530011790 con 0x7f25440b16d0 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.384+0000 7f25437fe700 1 -- 172.21.15.119:0/2727740210 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530011b30 con 0x7f25440b16d0 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.385+0000 7f2549b9d700 1 -- 172.21.15.119:0/2727740210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b16d0 msgr2=0x7f25440b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.385+0000 7f2549b9d700 1 --2- 172.21.15.119:0/2727740210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b16d0 0x7f25440b1b40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f25300098e0 tx=0x7f25300091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.385+0000 7f2549b9d700 1 -- 172.21.15.119:0/2727740210 shutdown_connections 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.385+0000 7f2549b9d700 1 --2- 172.21.15.119:0/2727740210 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b16d0 0x7f25440b1b40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.385+0000 7f2549b9d700 1 -- 172.21.15.119:0/2727740210 >> 172.21.15.119:0/2727740210 conn(0x7f25440d10c0 msgr2=0x7f25440affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:54.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.385+0000 7f2549b9d700 1 -- 172.21.15.119:0/2727740210 shutdown_connections 2023-10-08T01:08:54.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.385+0000 7f2549b9d700 1 -- 172.21.15.119:0/2727740210 wait complete. 2023-10-08T01:08:54.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.386+0000 7f2549b9d700 1 Processor -- start 2023-10-08T01:08:54.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.386+0000 7f2549b9d700 1 -- start start 2023-10-08T01:08:54.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.387+0000 7f2549b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25440b16d0 0x7f25440c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:54.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.387+0000 7f2549b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b33b0 0x7f2544153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:54.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.387+0000 7f2549b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25440b4e70 0x7f25440c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:54.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.387+0000 7f2549b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440cafb0 con 0x7f25440b16d0 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.387+0000 7f2549b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440c9290 con 0x7f25440b33b0 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.387+0000 7f2549b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25440cb200 con 0x7f25440b4e70 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25440b16d0 0x7f25440c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25440b16d0 0x7f25440c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48524/0 (socket says 172.21.15.119:48524) 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 -- 172.21.15.119:0/2473283649 learned_addr learned my addr 172.21.15.119:0/2473283649 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2543fff700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b33b0 0x7f2544153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 -- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25440b4e70 msgr2=0x7f25440c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f254939c700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25440b4e70 0x7f25440c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25440b4e70 0x7f25440c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 -- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b33b0 msgr2=0x7f2544153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b33b0 0x7f2544153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 -- 172.21.15.119:0/2473283649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2530009420 con 0x7f25440b16d0 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2543fff700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b33b0 0x7f2544153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f254939c700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25440b4e70 0x7f25440c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2548b9b700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25440b16d0 0x7f25440c8d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f253800c790 tx=0x7f2538007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538005b30 con 0x7f25440b16d0 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538005c90 con 0x7f25440b16d0 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.388+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25440cb4e0 con 0x7f25440b16d0 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.389+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25380072e0 con 0x7f25440b16d0 2023-10-08T01:08:54.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.389+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25440cbaa0 con 0x7f25440b16d0 2023-10-08T01:08:54.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.390+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2544088990 con 0x7f25440b16d0 2023-10-08T01:08:54.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.391+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 18) v1 ==== 84911+0+0 (secure 0 0 0) 0x7f2538007690 con 0x7f25440b16d0 2023-10-08T01:08:54.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.391+0000 7f2540ff9700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f25280670e0 0x7f2528069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:54.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.391+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6413+0+0 (secure 0 0 0) 0x7f2538083f70 con 0x7f25440b16d0 2023-10-08T01:08:54.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.392+0000 7f2543fff700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f25280670e0 0x7f2528069590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:54.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.392+0000 7f2543fff700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f25280670e0 0x7f2528069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello connection peer type does not match what peer advertises 16 != 2 2023-10-08T01:08:54.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.392+0000 7f2543fff700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f25280670e0 0x7f2528069590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.392+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f25280670e0 msgr2=0x7f2528069590 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-10-08T01:08:54.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.392+0000 7f2540ff9700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f25280670e0 0x7f2528069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:54.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.395+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f253804b5a0 con 0x7f25440b16d0 2023-10-08T01:08:54.455 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.455+0000 7f662c6b2180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:08:54.552 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.552+0000 7f662c6b2180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:08:54.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.567+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 19) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f2538051120 con 0x7f25440b16d0 2023-10-08T01:08:54.638 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.635+0000 7f65a7a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.638 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.635+0000 7f65a7a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.638 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.635+0000 7f65a7a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.638 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.635+0000 7f65a7a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.639 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.635+0000 7f65a7a83700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.639 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.636+0000 7f65aaa89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.639 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.636+0000 7f65aaa89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.639 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.636+0000 7f65aaa89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.639 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.636+0000 7f65aaa89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.639 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:08:54.636+0000 7f65aaa89700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:08:54.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:54.671+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f254404bb80 con 0x7f25440b16d0 2023-10-08T01:08:55.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.131+0000 7f2540ff9700 1 -- 172.21.15.119:0/2473283649 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f25380547e0 con 0x7f25440b16d0 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25440b16d0 msgr2=0x7f25440c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25440b16d0 0x7f25440c8d50 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f253800c790 tx=0x7f2538007ea0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 shutdown_connections 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25440b4e70 0x7f25440c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:6832/85511,v1:172.21.15.119:6833/85511] conn(0x7f25280670e0 0x7f2528069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25440b16d0 0x7f25440c8d50 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 --2- 172.21.15.119:0/2473283649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25440b33b0 0x7f2544153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 >> 172.21.15.119:0/2473283649 conn(0x7f25440d10c0 msgr2=0x7f25440c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:55.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 shutdown_connections 2023-10-08T01:08:55.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.133+0000 7f2549b9d700 1 -- 172.21.15.119:0/2473283649 wait complete. 2023-10-08T01:08:55.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T01:08:55.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.338+0000 7fdb3b5ff700 1 Processor -- start 2023-10-08T01:08:55.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.338+0000 7fdb3b5ff700 1 -- start start 2023-10-08T01:08:55.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb3b5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb34156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:55.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb3b5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb34068890 0x7fdb340c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb3b5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3415a9b0 con 0x7fdb3415a440 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb3b5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3415ac90 con 0x7fdb34068890 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb3b5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3415af70 con 0x7fdb341563b0 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb39b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb34068890 0x7fdb340c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb39b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb34068890 0x7fdb340c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58312/0 (socket says 172.21.15.119:58312) 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb39b9c700 1 -- 172.21.15.119:0/1051967069 learned_addr learned my addr 172.21.15.119:0/1051967069 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.339+0000 7fdb3939b700 1 --2- 172.21.15.119:0/1051967069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb34156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb39b9c700 1 -- 172.21.15.119:0/1051967069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 msgr2=0x7fdb34156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb39b9c700 1 --2- 172.21.15.119:0/1051967069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb34156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb39b9c700 1 -- 172.21.15.119:0/1051967069 >> v1:172.21.15.119:6789/0 conn(0x7fdb3415a440 legacy=0x7fdb340acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:55.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb39b9c700 1 -- 172.21.15.119:0/1051967069 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3415b250 con 0x7fdb34068890 2023-10-08T01:08:55.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb2bfff700 1 -- 172.21.15.119:0/1051967069 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698566242 0 0) 0x7fdb3415a9b0 con 0x7fdb3415a440 2023-10-08T01:08:55.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb39b9c700 1 --2- 172.21.15.119:0/1051967069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb34068890 0x7fdb340c8230 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00d1a0 tx=0x7fdb2c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faaef34dfd208ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:55.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb2bfff700 1 -- 172.21.15.119:0/1051967069 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c010070 con 0x7fdb34068890 2023-10-08T01:08:55.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb2bfff700 1 -- 172.21.15.119:0/1051967069 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c00de10 con 0x7fdb34068890 2023-10-08T01:08:55.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.340+0000 7fdb2bfff700 1 -- 172.21.15.119:0/1051967069 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c00b320 con 0x7fdb34068890 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.341+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/1051967069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb34068890 msgr2=0x7fdb340c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.341+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/1051967069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb34068890 0x7fdb340c8230 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00d1a0 tx=0x7fdb2c00ba50 comp rx=0 tx=0).stop 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.341+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/1051967069 shutdown_connections 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.341+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/1051967069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb34068890 0x7fdb340c8230 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.341+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/1051967069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb34156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.341+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/1051967069 >> 172.21.15.119:0/1051967069 conn(0x7fdb340c8f90 msgr2=0x7fdb340c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.341+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/1051967069 shutdown_connections 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.341+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/1051967069 wait complete. 2023-10-08T01:08:55.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.342+0000 7fdb3b5ff700 1 Processor -- start 2023-10-08T01:08:55.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.342+0000 7fdb3b5ff700 1 -- start start 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.343+0000 7fdb3b5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb34068890 0x7fdb34158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.343+0000 7fdb3b5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb3406e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.343+0000 7fdb3b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb3415a440 0x7fdb3406e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.343+0000 7fdb3b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34157be0 con 0x7fdb3415a440 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.343+0000 7fdb3b5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb341572b0 con 0x7fdb34068890 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.343+0000 7fdb3b5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb34157590 con 0x7fdb341563b0 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb3415a440 0x7fdb3406e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb3415a440 0x7fdb3406e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58322/0 (socket says 172.21.15.119:58322) 2023-10-08T01:08:55.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 -- 172.21.15.119:0/2909205612 learned_addr learned my addr 172.21.15.119:0/2909205612 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb3939b700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb34068890 0x7fdb34158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb38b9a700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb3406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 -- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 msgr2=0x7fdb3406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb3406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 -- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb34068890 msgr2=0x7fdb34158070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb34068890 0x7fdb34158070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 -- 172.21.15.119:0/2909205612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c00bcf0 con 0x7fdb3415a440 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb38b9a700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb3406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb39b9c700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb3415a440 0x7fdb3406e540 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00d840 tx=0x7fdb2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb297fa700 1 -- 172.21.15.119:0/2909205612 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c00ac10 con 0x7fdb3415a440 2023-10-08T01:08:55.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.344+0000 7fdb297fa700 1 -- 172.21.15.119:0/2909205612 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c00ad70 con 0x7fdb3415a440 2023-10-08T01:08:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.345+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb34157870 con 0x7fdb3415a440 2023-10-08T01:08:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.345+0000 7fdb297fa700 1 -- 172.21.15.119:0/2909205612 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c01e670 con 0x7fdb3415a440 2023-10-08T01:08:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.345+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb341587f0 con 0x7fdb3415a440 2023-10-08T01:08:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.345+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb34157d90 con 0x7fdb3415a440 2023-10-08T01:08:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.346+0000 7fdb297fa700 1 -- 172.21.15.119:0/2909205612 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 19) v1 ==== 84451+0+0 (secure 0 0 0) 0x7fdb2c003040 con 0x7fdb3415a440 2023-10-08T01:08:55.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.347+0000 7fdb297fa700 1 -- 172.21.15.119:0/2909205612 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6714+0+0 (secure 0 0 0) 0x7fdb2c085dd0 con 0x7fdb3415a440 2023-10-08T01:08:55.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.352+0000 7fdb297fa700 1 -- 172.21.15.119:0/2909205612 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb2c052f20 con 0x7fdb3415a440 2023-10-08T01:08:55.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.542+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdb340718b0 con 0x7fdb3415a440 2023-10-08T01:08:55.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.568+0000 7fdb297fa700 1 -- 172.21.15.119:0/2909205612 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 20) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fdb2c085020 con 0x7fdb3415a440 2023-10-08T01:08:55.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.569+0000 7fdb297fa700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdb1806c980 0x7fdb1806ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:55.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.569+0000 7fdb3939b700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdb1806c980 0x7fdb1806ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:55.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:55.574+0000 7fdb3939b700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdb1806c980 0x7fdb1806ed60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009db0 tx=0x7fdb300091a0 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:56.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.239+0000 7fdb297fa700 1 -- 172.21.15.119:0/2909205612 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdb2c029270 con 0x7fdb3415a440 2023-10-08T01:08:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdb1806c980 msgr2=0x7fdb1806ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdb1806c980 0x7fdb1806ed60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009db0 tx=0x7fdb300091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb3415a440 msgr2=0x7fdb3406e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb3415a440 0x7fdb3406e540 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c00d840 tx=0x7fdb2c01a000 comp rx=0 tx=0).stop 2023-10-08T01:08:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 shutdown_connections 2023-10-08T01:08:56.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdb1806c980 0x7fdb1806ed60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb3415a440 0x7fdb3406e540 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb34068890 0x7fdb34158070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 --2- 172.21.15.119:0/2909205612 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb341563b0 0x7fdb3406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.243+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 >> 172.21.15.119:0/2909205612 conn(0x7fdb340c8f90 msgr2=0x7fdb340ca410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:56.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.246+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 shutdown_connections 2023-10-08T01:08:56.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.246+0000 7fdb3b5ff700 1 -- 172.21.15.119:0/2909205612 wait complete. 2023-10-08T01:08:56.248 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T01:08:56.268 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T01:08:56.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.455+0000 7fe5f35ff700 1 Processor -- start 2023-10-08T01:08:56.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.455+0000 7fe5f35ff700 1 -- start start 2023-10-08T01:08:56.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.456+0000 7fe5f35ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec1563b0 0x7fe5ec156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:56.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.456+0000 7fe5f35ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5ec15a440 0x7fe5ec0acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:56.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.456+0000 7fe5f35ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec15a9b0 con 0x7fe5ec068890 2023-10-08T01:08:56.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.456+0000 7fe5f35ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec15ac90 con 0x7fe5ec15a440 2023-10-08T01:08:56.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.456+0000 7fe5f35ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec15af70 con 0x7fe5ec1563b0 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f0b9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5ec15a440 0x7fe5ec0acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f139b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec1563b0 0x7fe5ec156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f0b9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5ec15a440 0x7fe5ec0acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58782/0 (socket says 172.21.15.119:58782) 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f0b9a700 1 -- 172.21.15.119:0/2615294442 learned_addr learned my addr 172.21.15.119:0/2615294442 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f1b9c700 1 --1- 172.21.15.119:0/2615294442 >> v1:172.21.15.184:6789/0 conn(0x7fe5ec068890 0x7fe5ec0c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:42502/0 (socket says 172.21.15.119:42502) 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f139b700 1 -- 172.21.15.119:0/2615294442 >> v2:172.21.15.184:3300/0 conn(0x7fe5ec15a440 msgr2=0x7fe5ec0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f139b700 1 --2- 172.21.15.119:0/2615294442 >> v2:172.21.15.184:3300/0 conn(0x7fe5ec15a440 0x7fe5ec0acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f139b700 1 -- 172.21.15.119:0/2615294442 >> v1:172.21.15.184:6789/0 conn(0x7fe5ec068890 legacy=0x7fe5ec0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f139b700 1 -- 172.21.15.119:0/2615294442 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec15b250 con 0x7fe5ec1563b0 2023-10-08T01:08:56.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.457+0000 7fe5f0b9a700 1 --2- 172.21.15.119:0/2615294442 >> v2:172.21.15.184:3300/0 conn(0x7fe5ec15a440 0x7fe5ec0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:56.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.458+0000 7fe5f139b700 1 --2- 172.21.15.119:0/2615294442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec1563b0 0x7fe5ec156780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800dd10 tx=0x7fe5e800d630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a232e782ef38535 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:56.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.458+0000 7fe5e3fff700 1 -- 172.21.15.119:0/2615294442 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8012070 con 0x7fe5ec1563b0 2023-10-08T01:08:56.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.458+0000 7fe5e3fff700 1 -- 172.21.15.119:0/2615294442 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e800de70 con 0x7fe5ec1563b0 2023-10-08T01:08:56.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.458+0000 7fe5e3fff700 1 -- 172.21.15.119:0/2615294442 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800fb60 con 0x7fe5ec1563b0 2023-10-08T01:08:56.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.459+0000 7fe5f35ff700 1 -- 172.21.15.119:0/2615294442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec1563b0 msgr2=0x7fe5ec156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:56.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.459+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/2615294442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec1563b0 0x7fe5ec156780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800dd10 tx=0x7fe5e800d630 comp rx=0 tx=0).stop 2023-10-08T01:08:56.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.459+0000 7fe5f35ff700 1 -- 172.21.15.119:0/2615294442 shutdown_connections 2023-10-08T01:08:56.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.459+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/2615294442 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec1563b0 0x7fe5ec156780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.459+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/2615294442 >> v2:172.21.15.184:3300/0 conn(0x7fe5ec15a440 0x7fe5ec0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.459+0000 7fe5f35ff700 1 -- 172.21.15.119:0/2615294442 >> 172.21.15.119:0/2615294442 conn(0x7fe5ec0c8f90 msgr2=0x7fe5ec0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:56.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.459+0000 7fe5f35ff700 1 -- 172.21.15.119:0/2615294442 shutdown_connections 2023-10-08T01:08:56.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.459+0000 7fe5f35ff700 1 -- 172.21.15.119:0/2615294442 wait complete. 2023-10-08T01:08:56.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.460+0000 7fe5f35ff700 1 Processor -- start 2023-10-08T01:08:56.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.460+0000 7fe5f35ff700 1 -- start start 2023-10-08T01:08:56.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.461+0000 7fe5f35ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec068890 0x7fe5ec151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:56.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.461+0000 7fe5f35ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5ec1563b0 0x7fe5ec0ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:56.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.461+0000 7fe5f35ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5ec15a440 0x7fe5ec155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:56.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.461+0000 7fe5f35ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec159d50 con 0x7fe5ec15a440 2023-10-08T01:08:56.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.461+0000 7fe5f35ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec1520f0 con 0x7fe5ec1563b0 2023-10-08T01:08:56.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.461+0000 7fe5f35ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec0adc40 con 0x7fe5ec068890 2023-10-08T01:08:56.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5ec15a440 0x7fe5ec155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:56.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5ec15a440 0x7fe5ec155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58326/0 (socket says 172.21.15.119:58326) 2023-10-08T01:08:56.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 -- 172.21.15.119:0/318278136 learned_addr learned my addr 172.21.15.119:0/318278136 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:56.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f139b700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec068890 0x7fe5ec151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:56.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f0b9a700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5ec1563b0 0x7fe5ec0ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 -- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec068890 msgr2=0x7fe5ec151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec068890 0x7fe5ec151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 -- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5ec1563b0 msgr2=0x7fe5ec0ad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5ec1563b0 0x7fe5ec0ad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 -- 172.21.15.119:0/318278136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc0094f0 con 0x7fe5ec15a440 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f139b700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec068890 0x7fe5ec151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f0b9a700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5ec1563b0 0x7fe5ec0ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f1b9c700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5ec15a440 0x7fe5ec155550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400c890 tx=0x7fe5e4001880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5e17fa700 1 -- 172.21.15.119:0/318278136 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4004da0 con 0x7fe5ec15a440 2023-10-08T01:08:56.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5e17fa700 1 -- 172.21.15.119:0/318278136 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e40060f0 con 0x7fe5ec15a440 2023-10-08T01:08:56.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5e17fa700 1 -- 172.21.15.119:0/318278136 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4006470 con 0x7fe5ec15a440 2023-10-08T01:08:56.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.462+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e800d8d0 con 0x7fe5ec15a440 2023-10-08T01:08:56.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.463+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec0afc00 con 0x7fe5ec15a440 2023-10-08T01:08:56.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.463+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec0afd90 con 0x7fe5ec15a440 2023-10-08T01:08:56.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.464+0000 7fe5e17fa700 1 -- 172.21.15.119:0/318278136 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 20) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fe5e4006820 con 0x7fe5ec15a440 2023-10-08T01:08:56.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.465+0000 7fe5e17fa700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe5d0066d80 0x7fe5d0069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:56.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.465+0000 7fe5e17fa700 1 -- 172.21.15.119:0/318278136 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6276+0+0 (secure 0 0 0) 0x7fe5e40840b0 con 0x7fe5ec15a440 2023-10-08T01:08:56.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.466+0000 7fe5f139b700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe5d0066d80 0x7fe5d0069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:56.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.466+0000 7fe5f139b700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe5d0066d80 0x7fe5d0069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800d670 tx=0x7fe5e801c000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:56.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.469+0000 7fe5e17fa700 1 -- 172.21.15.119:0/318278136 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe5e4051410 con 0x7fe5ec15a440 2023-10-08T01:08:56.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:56.670+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe5ec0c7de0 con 0x7fe5ec15a440 2023-10-08T01:08:57.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.248+0000 7fe5e17fa700 1 -- 172.21.15.119:0/318278136 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5e40517e0 con 0x7fe5ec15a440 2023-10-08T01:08:57.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.573+0000 7fe5e17fa700 1 -- 172.21.15.119:0/318278136 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe5e4054ea0 con 0x7fe5ec15a440 2023-10-08T01:08:57.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe5d0066d80 msgr2=0x7fe5d0069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:57.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe5d0066d80 0x7fe5d0069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800d670 tx=0x7fe5e801c000 comp rx=0 tx=0).stop 2023-10-08T01:08:57.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5ec15a440 msgr2=0x7fe5ec155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:57.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5ec15a440 0x7fe5ec155550 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400c890 tx=0x7fe5e4001880 comp rx=0 tx=0).stop 2023-10-08T01:08:57.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 shutdown_connections 2023-10-08T01:08:57.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe5d0066d80 0x7fe5d0069230 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800d670 tx=0x7fe5e801c000 comp rx=0 tx=0).stop 2023-10-08T01:08:57.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5ec068890 0x7fe5ec151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5ec15a440 0x7fe5ec155550 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 --2- 172.21.15.119:0/318278136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5ec1563b0 0x7fe5ec0ad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.583+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 >> 172.21.15.119:0/318278136 conn(0x7fe5ec0c8f90 msgr2=0x7fe5ec0ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:57.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.585+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 shutdown_connections 2023-10-08T01:08:57.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.587+0000 7fe5f35ff700 1 -- 172.21.15.119:0/318278136 wait complete. 2023-10-08T01:08:57.590 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T01:08:57.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T01:08:57.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.797+0000 7fd13b19d700 1 Processor -- start 2023-10-08T01:08:57.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.797+0000 7fd13b19d700 1 -- start start 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13b19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd134153900 0x7fd1340c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13b19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1340c9db0 0x7fd13406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13b19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13406e150 con 0x7fd134157130 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13b19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1340c7800 con 0x7fd1340c9db0 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13b19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1340c7ae0 con 0x7fd134153900 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13999a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1340c9db0 0x7fd13406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13999a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1340c9db0 0x7fd13406a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58334/0 (socket says 172.21.15.119:58334) 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13999a700 1 -- 172.21.15.119:0/4213948018 learned_addr learned my addr 172.21.15.119:0/4213948018 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:57.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13a19b700 1 --2- 172.21.15.119:0/4213948018 >> v2:172.21.15.184:3300/0 conn(0x7fd134153900 0x7fd1340c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13999a700 1 -- 172.21.15.119:0/4213948018 >> v2:172.21.15.184:3300/0 conn(0x7fd134153900 msgr2=0x7fd1340c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13999a700 1 --2- 172.21.15.119:0/4213948018 >> v2:172.21.15.184:3300/0 conn(0x7fd134153900 0x7fd1340c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13999a700 1 -- 172.21.15.119:0/4213948018 >> v1:172.21.15.119:6789/0 conn(0x7fd134157130 legacy=0x7fd134159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd13999a700 1 -- 172.21.15.119:0/4213948018 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340c7dc0 con 0x7fd1340c9db0 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.799+0000 7fd139199700 1 -- 172.21.15.119:0/4213948018 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445416962 0 0) 0x7fd13406e150 con 0x7fd134157130 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.800+0000 7fd13999a700 1 --2- 172.21.15.119:0/4213948018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340c9db0 0x7fd13406a000 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd13406aed0 tx=0x7fd12400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5533a5ea0ed81ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.800+0000 7fd139199700 1 -- 172.21.15.119:0/4213948018 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12400dc50 con 0x7fd1340c9db0 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.800+0000 7fd139199700 1 -- 172.21.15.119:0/4213948018 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12400ddb0 con 0x7fd1340c9db0 2023-10-08T01:08:57.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.800+0000 7fd139199700 1 -- 172.21.15.119:0/4213948018 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1240196d0 con 0x7fd1340c9db0 2023-10-08T01:08:57.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.800+0000 7fd13b19d700 1 -- 172.21.15.119:0/4213948018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340c9db0 msgr2=0x7fd13406a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:57.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.800+0000 7fd13b19d700 1 --2- 172.21.15.119:0/4213948018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340c9db0 0x7fd13406a000 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd13406aed0 tx=0x7fd12400b4f0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.801+0000 7fd13b19d700 1 -- 172.21.15.119:0/4213948018 shutdown_connections 2023-10-08T01:08:57.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.801+0000 7fd13b19d700 1 --2- 172.21.15.119:0/4213948018 >> v2:172.21.15.184:3300/0 conn(0x7fd134153900 0x7fd1340c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.801+0000 7fd13b19d700 1 --2- 172.21.15.119:0/4213948018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340c9db0 0x7fd13406a000 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.801+0000 7fd13b19d700 1 -- 172.21.15.119:0/4213948018 >> 172.21.15.119:0/4213948018 conn(0x7fd1340cc1e0 msgr2=0x7fd1340cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:57.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.801+0000 7fd13b19d700 1 -- 172.21.15.119:0/4213948018 shutdown_connections 2023-10-08T01:08:57.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.801+0000 7fd13b19d700 1 -- 172.21.15.119:0/4213948018 wait complete. 2023-10-08T01:08:57.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.802+0000 7fd13b19d700 1 Processor -- start 2023-10-08T01:08:57.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.802+0000 7fd13b19d700 1 -- start start 2023-10-08T01:08:57.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13b19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1340c9db0 0x7fd1340c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:57.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13b19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd134153900 0x7fd134150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:57.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13b19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd134157130 0x7fd134154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:57.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13b19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134153cd0 con 0x7fd134153900 2023-10-08T01:08:57.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13b19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134154af0 con 0x7fd1340c9db0 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13b19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340c6600 con 0x7fd134157130 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd134153900 0x7fd134150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd134153900 0x7fd134150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58346/0 (socket says 172.21.15.119:58346) 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.803+0000 7fd13999a700 1 -- 172.21.15.119:0/1785826774 learned_addr learned my addr 172.21.15.119:0/1785826774 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13a19b700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1340c9db0 0x7fd1340c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13a99c700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd134157130 0x7fd134154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13999a700 1 -- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd134157130 msgr2=0x7fd134154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13999a700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd134157130 0x7fd134154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13999a700 1 -- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1340c9db0 msgr2=0x7fd1340c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13999a700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1340c9db0 0x7fd1340c60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13999a700 1 -- 172.21.15.119:0/1785826774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12400b790 con 0x7fd134153900 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13a99c700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd134157130 0x7fd134154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13999a700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd134153900 0x7fd134150c30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd124006fd0 tx=0x7fd12401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd12a7fc700 1 -- 172.21.15.119:0/1785826774 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124010470 con 0x7fd134153900 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd12a7fc700 1 -- 172.21.15.119:0/1785826774 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1240105d0 con 0x7fd134153900 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd12a7fc700 1 -- 172.21.15.119:0/1785826774 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124010950 con 0x7fd134153900 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd134153ee0 con 0x7fd134153900 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.804+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1341544a0 con 0x7fd134153900 2023-10-08T01:08:57.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.805+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13406c450 con 0x7fd134153900 2023-10-08T01:08:57.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.806+0000 7fd12a7fc700 1 -- 172.21.15.119:0/1785826774 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd124022070 con 0x7fd134153900 2023-10-08T01:08:57.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.807+0000 7fd12a7fc700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:57.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.807+0000 7fd12a7fc700 1 -- 172.21.15.119:0/1785826774 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fd124014070 con 0x7fd134153900 2023-10-08T01:08:57.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.807+0000 7fd13a19b700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:57.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.808+0000 7fd13a19b700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd118066f30 0x7fd1180693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd130006fd0 tx=0x7fd130008040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:57.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:57.811+0000 7fd12a7fc700 1 -- 172.21.15.119:0/1785826774 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd124055340 con 0x7fd134153900 2023-10-08T01:08:58.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.005+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd1341597e0 con 0x7fd134153900 2023-10-08T01:08:58.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.010+0000 7fd12a7fc700 1 -- 172.21.15.119:0/1785826774 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd124058960 con 0x7fd134153900 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd118066f30 msgr2=0x7fd1180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd118066f30 0x7fd1180693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd130006fd0 tx=0x7fd130008040 comp rx=0 tx=0).stop 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd134153900 msgr2=0x7fd134150c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd134153900 0x7fd134150c30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd124006fd0 tx=0x7fd12401b000 comp rx=0 tx=0).stop 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 shutdown_connections 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd118066f30 0x7fd1180693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd134157130 0x7fd134154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1340c9db0 0x7fd1340c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 --2- 172.21.15.119:0/1785826774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd134153900 0x7fd134150c30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.012+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 >> 172.21.15.119:0/1785826774 conn(0x7fd1340cc1e0 msgr2=0x7fd1340c12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:58.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.013+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 shutdown_connections 2023-10-08T01:08:58.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.013+0000 7fd13b19d700 1 -- 172.21.15.119:0/1785826774 wait complete. 2023-10-08T01:08:58.030 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T01:08:58.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.215+0000 7f7e33bff700 1 Processor -- start 2023-10-08T01:08:58.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.216+0000 7f7e33bff700 1 -- start start 2023-10-08T01:08:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.217+0000 7f7e33bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7e2c0c83f0 0x7f7e2c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.217+0000 7f7e33bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c15b0d0 con 0x7f7e2c0ccee0 2023-10-08T01:08:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.217+0000 7f7e33bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e2c0c7a60 con 0x7f7e2c1568e0 2023-10-08T01:08:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.217+0000 7f7e33bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e2c15a4c0 con 0x7f7e2c0c83f0 2023-10-08T01:08:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.217+0000 7f7e3119a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7e2c0ccee0 0x7f7e2c1561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49676/0 (socket says 172.21.15.119:49676) 2023-10-08T01:08:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.217+0000 7f7e3119a700 1 -- 172.21.15.119:0/1455048541 learned_addr learned my addr 172.21.15.119:0/1455048541 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:58.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.217+0000 7f7e3199b700 1 --2- 172.21.15.119:0/1455048541 >> v2:172.21.15.184:3300/0 conn(0x7f7e2c0c83f0 0x7f7e2c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.217+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916464559 0 0) 0x7f7e2c15b0d0 con 0x7f7e2c0ccee0 2023-10-08T01:08:58.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e18003570 con 0x7f7e2c0ccee0 2023-10-08T01:08:58.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3784386678 0 0) 0x7f7e18003570 con 0x7f7e2c0ccee0 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 >> v2:172.21.15.184:3300/0 conn(0x7f7e2c0c83f0 msgr2=0x7f7e2c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 --2- 172.21.15.119:0/1455048541 >> v2:172.21.15.184:3300/0 conn(0x7f7e2c0c83f0 0x7f7e2c0c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 >> v1:172.21.15.184:6789/0 conn(0x7f7e2c1568e0 legacy=0x7f7e2c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c15a7a0 con 0x7f7e2c0ccee0 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f7e1c001470 con 0x7f7e2c0ccee0 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 >> v1:172.21.15.119:6789/0 conn(0x7f7e2c0ccee0 legacy=0x7f7e2c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e18003a60 0x7f7e18003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18004810 0x7f7e18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e1800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c15a4c0 con 0x7f7e180051c0 2023-10-08T01:08:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1800c0d0 con 0x7f7e18004810 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1800e4a0 con 0x7f7e18003a60 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e3219c700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e1800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e3119a700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18004810 0x7f7e18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.218+0000 7f7e3199b700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e18003a60 0x7f7e18003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e3219c700 1 -- 172.21.15.119:0/1455048541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e18003a60 msgr2=0x7f7e18003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e3219c700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e18003a60 0x7f7e18003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e3219c700 1 -- 172.21.15.119:0/1455048541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18004810 msgr2=0x7f7e18004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e3219c700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e18004810 0x7f7e18004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e3219c700 1 -- 172.21.15.119:0/1455048541 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e18003570 con 0x7f7e180051c0 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e3219c700 1 -- 172.21.15.119:0/1455048541 reap_dead start 2023-10-08T01:08:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e3219c700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e1800ba60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7e24001fc0 tx=0x7f7e24001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1963017b008b0a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24001950 con 0x7f7e180051c0 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2400c8b0 con 0x7f7e180051c0 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.219+0000 7f7e30999700 1 -- 172.21.15.119:0/1455048541 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2400cc50 con 0x7f7e180051c0 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.220+0000 7f7e33bff700 1 -- 172.21.15.119:0/1455048541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 msgr2=0x7f7e1800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.220+0000 7f7e33bff700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e1800ba60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7e24001fc0 tx=0x7f7e24001640 comp rx=0 tx=0).stop 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.220+0000 7f7e33bff700 1 -- 172.21.15.119:0/1455048541 shutdown_connections 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.220+0000 7f7e33bff700 1 --2- 172.21.15.119:0/1455048541 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e1800ba60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.220+0000 7f7e33bff700 1 -- 172.21.15.119:0/1455048541 >> 172.21.15.119:0/1455048541 conn(0x7f7e2c068e10 msgr2=0x7f7e2c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.220+0000 7f7e33bff700 1 -- 172.21.15.119:0/1455048541 shutdown_connections 2023-10-08T01:08:58.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.220+0000 7f7e33bff700 1 -- 172.21.15.119:0/1455048541 wait complete. 2023-10-08T01:08:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.221+0000 7f7e33bff700 1 Processor -- start 2023-10-08T01:08:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.221+0000 7f7e33bff700 1 -- start start 2023-10-08T01:08:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.222+0000 7f7e33bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e2c14f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.222+0000 7f7e33bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c14f850 0x7f7e2c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.222+0000 7f7e33bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c1515d0 0x7f7e2c151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.222+0000 7f7e33bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c0c5ef0 con 0x7f7e180051c0 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.222+0000 7f7e33bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c1538f0 con 0x7f7e2c14f850 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.222+0000 7f7e33bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e2c152fc0 con 0x7f7e2c1515d0 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e2c14f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e2c14f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58364/0 (socket says 172.21.15.119:58364) 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 -- 172.21.15.119:0/2834806136 learned_addr learned my addr 172.21.15.119:0/2834806136 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3119a700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c14f850 0x7f7e2c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3219c700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c1515d0 0x7f7e2c151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 -- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c1515d0 msgr2=0x7f7e2c151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c1515d0 0x7f7e2c151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 -- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c14f850 msgr2=0x7f7e2c151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c14f850 0x7f7e2c151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 -- 172.21.15.119:0/2834806136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e2c1532a0 con 0x7f7e180051c0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3119a700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c14f850 0x7f7e2c151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e3199b700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e2c14f310 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e28011b10 tx=0x7f7e280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e21ffb700 1 -- 172.21.15.119:0/2834806136 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800f990 con 0x7f7e180051c0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e21ffb700 1 -- 172.21.15.119:0/2834806136 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800faf0 con 0x7f7e180051c0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e2c153580 con 0x7f7e180051c0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.223+0000 7f7e21ffb700 1 -- 172.21.15.119:0/2834806136 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800fe70 con 0x7f7e180051c0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.224+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e2c153d80 con 0x7f7e180051c0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.224+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e2c04c3c0 con 0x7f7e180051c0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.225+0000 7f7e21ffb700 1 -- 172.21.15.119:0/2834806136 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e28014650 con 0x7f7e180051c0 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.226+0000 7f7e21ffb700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7e10066f60 0x7f7e10069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.226+0000 7f7e21ffb700 1 -- 172.21.15.119:0/2834806136 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f7e28018030 con 0x7f7e180051c0 2023-10-08T01:08:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.226+0000 7f7e3119a700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7e10066f60 0x7f7e10069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.227+0000 7f7e3119a700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7e10066f60 0x7f7e10069410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c001ce0 tx=0x7f7e1c009040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:58.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.230+0000 7f7e21ffb700 1 -- 172.21.15.119:0/2834806136 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e28055be0 con 0x7f7e180051c0 2023-10-08T01:08:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.423+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f7e2c15aa80 con 0x7f7e180051c0 2023-10-08T01:08:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.429+0000 7f7e21ffb700 1 -- 172.21.15.119:0/2834806136 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7f7e280220b0 con 0x7f7e180051c0 2023-10-08T01:08:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7e10066f60 msgr2=0x7f7e10069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7e10066f60 0x7f7e10069410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c001ce0 tx=0x7f7e1c009040 comp rx=0 tx=0).stop 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 msgr2=0x7f7e2c14f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e2c14f310 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7e28011b10 tx=0x7f7e280091a0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 shutdown_connections 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7e10066f60 0x7f7e10069410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e2c1515d0 0x7f7e2c151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e180051c0 0x7f7e2c14f310 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 --2- 172.21.15.119:0/2834806136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e2c14f850 0x7f7e2c151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 >> 172.21.15.119:0/2834806136 conn(0x7f7e2c068e10 msgr2=0x7f7e2c0b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 shutdown_connections 2023-10-08T01:08:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.432+0000 7f7e33bff700 1 -- 172.21.15.119:0/2834806136 wait complete. 2023-10-08T01:08:58.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T01:08:58.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.633+0000 7fd91abff700 1 Processor -- start 2023-10-08T01:08:58.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.633+0000 7fd91abff700 1 -- start start 2023-10-08T01:08:58.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.634+0000 7fd91abff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd914159890 0x7fd91415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.634+0000 7fd91abff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd914069560 0x7fd9140699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.634+0000 7fd91abff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd914069ee0 con 0x7fd914159c60 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.634+0000 7fd91abff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91415a160 con 0x7fd914159890 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.634+0000 7fd91abff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140d13a0 con 0x7fd914069560 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91899b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd914159890 0x7fd91415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91899b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd914159890 0x7fd91415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58376/0 (socket says 172.21.15.119:58376) 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91899b700 1 -- 172.21.15.119:0/3078206800 learned_addr learned my addr 172.21.15.119:0/3078206800 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd913fff700 1 --1- 172.21.15.119:0/3078206800 >> v1:172.21.15.119:6789/0 conn(0x7fd914159c60 0x7fd91415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49678/0 (socket says 172.21.15.119:49678) 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91919c700 1 --2- 172.21.15.119:0/3078206800 >> v2:172.21.15.184:3300/0 conn(0x7fd914069560 0x7fd9140699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91899b700 1 -- 172.21.15.119:0/3078206800 >> v2:172.21.15.184:3300/0 conn(0x7fd914069560 msgr2=0x7fd9140699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91899b700 1 --2- 172.21.15.119:0/3078206800 >> v2:172.21.15.184:3300/0 conn(0x7fd914069560 0x7fd9140699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91899b700 1 -- 172.21.15.119:0/3078206800 >> v1:172.21.15.119:6789/0 conn(0x7fd914159c60 legacy=0x7fd91415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91899b700 1 -- 172.21.15.119:0/3078206800 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91415e140 con 0x7fd914159890 2023-10-08T01:08:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd9137fe700 1 -- 172.21.15.119:0/3078206800 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451527189 0 0) 0x7fd914069ee0 con 0x7fd914159c60 2023-10-08T01:08:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.635+0000 7fd91899b700 1 --2- 172.21.15.119:0/3078206800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159890 0x7fd91415abd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd91406a890 tx=0x7fd908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95a055cc4c74143 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd9137fe700 1 -- 172.21.15.119:0/3078206800 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800b860 con 0x7fd914159890 2023-10-08T01:08:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd9137fe700 1 -- 172.21.15.119:0/3078206800 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90800b9c0 con 0x7fd914159890 2023-10-08T01:08:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd9137fe700 1 -- 172.21.15.119:0/3078206800 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90800bd60 con 0x7fd914159890 2023-10-08T01:08:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd91abff700 1 -- 172.21.15.119:0/3078206800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159890 msgr2=0x7fd91415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd91abff700 1 --2- 172.21.15.119:0/3078206800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159890 0x7fd91415abd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fd91406a890 tx=0x7fd908009250 comp rx=0 tx=0).stop 2023-10-08T01:08:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd91abff700 1 -- 172.21.15.119:0/3078206800 shutdown_connections 2023-10-08T01:08:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd91abff700 1 --2- 172.21.15.119:0/3078206800 >> v2:172.21.15.184:3300/0 conn(0x7fd914069560 0x7fd9140699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd91abff700 1 --2- 172.21.15.119:0/3078206800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159890 0x7fd91415abd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd91abff700 1 -- 172.21.15.119:0/3078206800 >> 172.21.15.119:0/3078206800 conn(0x7fd914068e20 msgr2=0x7fd9140d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:58.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd91abff700 1 -- 172.21.15.119:0/3078206800 shutdown_connections 2023-10-08T01:08:58.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.636+0000 7fd91abff700 1 -- 172.21.15.119:0/3078206800 wait complete. 2023-10-08T01:08:58.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.638+0000 7fd91abff700 1 Processor -- start 2023-10-08T01:08:58.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.638+0000 7fd91abff700 1 -- start start 2023-10-08T01:08:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91abff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd914069560 0x7fd9140cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91abff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd914159890 0x7fd914153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91abff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159c60 0x7fd9140ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91abff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914155c90 con 0x7fd914159c60 2023-10-08T01:08:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91abff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914156810 con 0x7fd914069560 2023-10-08T01:08:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91abff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd914155ee0 con 0x7fd914159890 2023-10-08T01:08:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91919c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159c60 0x7fd9140ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91919c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159c60 0x7fd9140ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58384/0 (socket says 172.21.15.119:58384) 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91919c700 1 -- 172.21.15.119:0/3492123768 learned_addr learned my addr 172.21.15.119:0/3492123768 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91899b700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd914069560 0x7fd9140cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd913fff700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd914159890 0x7fd914153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91919c700 1 -- 172.21.15.119:0/3492123768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd914159890 msgr2=0x7fd914153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91919c700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd914159890 0x7fd914153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91919c700 1 -- 172.21.15.119:0/3492123768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd914069560 msgr2=0x7fd9140cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91919c700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd914069560 0x7fd9140cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.639+0000 7fd91919c700 1 -- 172.21.15.119:0/3492123768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080094f0 con 0x7fd914159c60 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.640+0000 7fd91919c700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159c60 0x7fd9140ceda0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009fa0 tx=0x7fd90c009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.640+0000 7fd910ff9700 1 -- 172.21.15.119:0/3492123768 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00ebc0 con 0x7fd914159c60 2023-10-08T01:08:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.640+0000 7fd910ff9700 1 -- 172.21.15.119:0/3492123768 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c00ed20 con 0x7fd914159c60 2023-10-08T01:08:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.640+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9141561c0 con 0x7fd914159c60 2023-10-08T01:08:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.640+0000 7fd910ff9700 1 -- 172.21.15.119:0/3492123768 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c014670 con 0x7fd914159c60 2023-10-08T01:08:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.640+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9140cd600 con 0x7fd914159c60 2023-10-08T01:08:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.641+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91404c3c0 con 0x7fd914159c60 2023-10-08T01:08:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.642+0000 7fd910ff9700 1 -- 172.21.15.119:0/3492123768 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd90c0149e0 con 0x7fd914159c60 2023-10-08T01:08:58.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.643+0000 7fd910ff9700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd8f8066f30 0x7fd8f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:58.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.643+0000 7fd910ff9700 1 -- 172.21.15.119:0/3492123768 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fd90c084cd0 con 0x7fd914159c60 2023-10-08T01:08:58.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.643+0000 7fd91899b700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd8f8066f30 0x7fd8f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:58.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.644+0000 7fd91899b700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd8f8066f30 0x7fd8f80693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd91406a890 tx=0x7fd908017000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:58.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.647+0000 7fd910ff9700 1 -- 172.21.15.119:0/3492123768 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd90c051e00 con 0x7fd914159c60 2023-10-08T01:08:58.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.835+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd91406a5b0 con 0x7fd914159c60 2023-10-08T01:08:58.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.841+0000 7fd910ff9700 1 -- 172.21.15.119:0/3492123768 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v15) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd90c055420 con 0x7fd914159c60 2023-10-08T01:08:58.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd8f8066f30 msgr2=0x7fd8f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd8f8066f30 0x7fd8f80693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd91406a890 tx=0x7fd908017000 comp rx=0 tx=0).stop 2023-10-08T01:08:58.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159c60 msgr2=0x7fd9140ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:58.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159c60 0x7fd9140ceda0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd90c009fa0 tx=0x7fd90c009900 comp rx=0 tx=0).stop 2023-10-08T01:08:58.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 shutdown_connections 2023-10-08T01:08:58.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd8f8066f30 0x7fd8f80693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd914159c60 0x7fd9140ceda0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd914069560 0x7fd9140cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 --2- 172.21.15.119:0/3492123768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd914159890 0x7fd914153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:58.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 >> 172.21.15.119:0/3492123768 conn(0x7fd914068e20 msgr2=0x7fd9140d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:58.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 shutdown_connections 2023-10-08T01:08:58.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:58.844+0000 7fd91abff700 1 -- 172.21.15.119:0/3492123768 wait complete. 2023-10-08T01:08:58.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-10-08T01:08:59.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.044+0000 7fe68cb9d700 1 Processor -- start 2023-10-08T01:08:59.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.044+0000 7fe68cb9d700 1 -- start start 2023-10-08T01:08:59.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.045+0000 7fe68cb9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe688159420 0x7fe6880b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe68cb9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6880b2980 con 0x7fe6881597f0 2023-10-08T01:08:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe68cb9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6880b11d0 con 0x7fe68815d230 2023-10-08T01:08:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe68cb9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6880b14b0 con 0x7fe688159420 2023-10-08T01:08:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe686ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe6881597f0 0x7fe6880b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49688/0 (socket says 172.21.15.119:49688) 2023-10-08T01:08:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe686ffd700 1 -- 172.21.15.119:0/3771050705 learned_addr learned my addr 172.21.15.119:0/3771050705 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:08:59.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe6877fe700 1 --2- 172.21.15.119:0/3771050705 >> v2:172.21.15.184:3300/0 conn(0x7fe688159420 0x7fe6880b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990343178 0 0) 0x7fe6880b2980 con 0x7fe6881597f0 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe670003570 con 0x7fe6881597f0 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1760987790 0 0) 0x7fe670003570 con 0x7fe6881597f0 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 >> v2:172.21.15.184:3300/0 conn(0x7fe688159420 msgr2=0x7fe6880b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe6867fc700 1 --2- 172.21.15.119:0/3771050705 >> v2:172.21.15.184:3300/0 conn(0x7fe688159420 0x7fe6880b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.046+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 >> v1:172.21.15.184:6789/0 conn(0x7fe68815d230 legacy=0x7fe68815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880b1790 con 0x7fe6881597f0 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fe67c001470 con 0x7fe6881597f0 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6877fe700 1 --2- 172.21.15.119:0/3771050705 >> v2:172.21.15.184:3300/0 conn(0x7fe688159420 0x7fe6880b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:08:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 >> v1:172.21.15.119:6789/0 conn(0x7fe6881597f0 legacy=0x7fe6880b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe670003a60 0x7fe670003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe670004810 0x7fe670004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6700051c0 0x7fe67000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6880b14b0 con 0x7fe670004810 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67000c0d0 con 0x7fe6700051c0 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67000c290 con 0x7fe670003a60 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe686ffd700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe670004810 0x7fe670004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe6877fe700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe670003a60 0x7fe670003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe687fff700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6700051c0 0x7fe67000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe686ffd700 1 -- 172.21.15.119:0/3771050705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe670003a60 msgr2=0x7fe670003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe686ffd700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe670003a60 0x7fe670003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe686ffd700 1 -- 172.21.15.119:0/3771050705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6700051c0 msgr2=0x7fe67000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe686ffd700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6700051c0 0x7fe67000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe686ffd700 1 -- 172.21.15.119:0/3771050705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe670003570 con 0x7fe670004810 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.047+0000 7fe687fff700 1 -- 172.21.15.119:0/3771050705 reap_dead start 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe686ffd700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe670004810 0x7fe670004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00c390 tx=0x7fe67c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94abc12d5d8283b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00cc60 con 0x7fe670004810 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c00cdc0 con 0x7fe670004810 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe6867fc700 1 -- 172.21.15.119:0/3771050705 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c005b50 con 0x7fe670004810 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe68cb9d700 1 -- 172.21.15.119:0/3771050705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe670004810 msgr2=0x7fe670004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe68cb9d700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe670004810 0x7fe670004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fe67c00c390 tx=0x7fe67c001e50 comp rx=0 tx=0).stop 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe68cb9d700 1 -- 172.21.15.119:0/3771050705 shutdown_connections 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe68cb9d700 1 --2- 172.21.15.119:0/3771050705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe670004810 0x7fe670004c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.048+0000 7fe68cb9d700 1 -- 172.21.15.119:0/3771050705 >> 172.21.15.119:0/3771050705 conn(0x7fe6880d0660 msgr2=0x7fe6880d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:08:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.049+0000 7fe68cb9d700 1 -- 172.21.15.119:0/3771050705 shutdown_connections 2023-10-08T01:08:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.049+0000 7fe68cb9d700 1 -- 172.21.15.119:0/3771050705 wait complete. 2023-10-08T01:08:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.050+0000 7fe68cb9d700 1 Processor -- start 2023-10-08T01:08:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.050+0000 7fe68cb9d700 1 -- start start 2023-10-08T01:08:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe68cb9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670004810 0x7fe68815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe68cb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe688156130 0x7fe68814f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe68cb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688069360 0x7fe6880697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe68cb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe688069d20 con 0x7fe688069360 2023-10-08T01:08:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe68cb9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe688159ca0 con 0x7fe670004810 2023-10-08T01:08:59.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe68cb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6881565c0 con 0x7fe688156130 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe687fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688069360 0x7fe6880697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe687fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688069360 0x7fe6880697e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58414/0 (socket says 172.21.15.119:58414) 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe687fff700 1 -- 172.21.15.119:0/2108668346 learned_addr learned my addr 172.21.15.119:0/2108668346 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe6877fe700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670004810 0x7fe68815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.051+0000 7fe686ffd700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe688156130 0x7fe68814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe687fff700 1 -- 172.21.15.119:0/2108668346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe688156130 msgr2=0x7fe68814f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe687fff700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe688156130 0x7fe68814f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe687fff700 1 -- 172.21.15.119:0/2108668346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670004810 msgr2=0x7fe68815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe687fff700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670004810 0x7fe68815c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe687fff700 1 -- 172.21.15.119:0/2108668346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6881697d0 con 0x7fe688069360 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe687fff700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688069360 0x7fe6880697e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe6880af930 tx=0x7fe680009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:59.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe66f7fe700 1 -- 172.21.15.119:0/2108668346 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680006db0 con 0x7fe688069360 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe66f7fe700 1 -- 172.21.15.119:0/2108668346 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680003070 con 0x7fe688069360 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe66f7fe700 1 -- 172.21.15.119:0/2108668346 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680003430 con 0x7fe688069360 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe688169ab0 con 0x7fe688069360 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.052+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68816a070 con 0x7fe688069360 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.053+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68816a290 con 0x7fe688069360 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.054+0000 7fe66f7fe700 1 -- 172.21.15.119:0/2108668346 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe680001790 con 0x7fe688069360 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.055+0000 7fe66f7fe700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe674066f30 0x7fe6740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.055+0000 7fe66f7fe700 1 -- 172.21.15.119:0/2108668346 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fe680083be0 con 0x7fe688069360 2023-10-08T01:08:59.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.055+0000 7fe6877fe700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe674066f30 0x7fe6740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:08:59.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.056+0000 7fe6877fe700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe674066f30 0x7fe6740693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe678006f00 tx=0x7fe678008040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:08:59.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.059+0000 7fe66f7fe700 1 -- 172.21.15.119:0/2108668346 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe680050d10 con 0x7fe688069360 2023-10-08T01:08:59.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:08:59.329+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fe68804c3c0 con 0x7fe688069360 2023-10-08T01:09:00.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.256+0000 7fe66f7fe700 1 -- 172.21.15.119:0/2108668346 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v471) v1 ==== 158+0+0 (secure 0 0 0) 0x7fe680019070 con 0x7fe688069360 2023-10-08T01:09:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe674066f30 msgr2=0x7fe6740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe674066f30 0x7fe6740693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe678006f00 tx=0x7fe678008040 comp rx=0 tx=0).stop 2023-10-08T01:09:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688069360 msgr2=0x7fe6880697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688069360 0x7fe6880697e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fe6880af930 tx=0x7fe680009040 comp rx=0 tx=0).stop 2023-10-08T01:09:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 shutdown_connections 2023-10-08T01:09:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe674066f30 0x7fe6740693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe688069360 0x7fe6880697e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe670004810 0x7fe68815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 --2- 172.21.15.119:0/2108668346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe688156130 0x7fe68814f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 >> 172.21.15.119:0/2108668346 conn(0x7fe6880d0660 msgr2=0x7fe6880cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 shutdown_connections 2023-10-08T01:09:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.259+0000 7fe68cb9d700 1 -- 172.21.15.119:0/2108668346 wait complete. 2023-10-08T01:09:00.280 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-10-08T01:09:00.281 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-10-08T01:09:00.281 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T01:09:00.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-08T01:09:00.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.466+0000 7fa2b27ff700 1 Processor -- start 2023-10-08T01:09:00.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.467+0000 7fa2b27ff700 1 -- start start 2023-10-08T01:09:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.468+0000 7fa2b27ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0cbf30 0x7fa2ac0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.468+0000 7fa2b27ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa2ac0b1420 0x7fa2ac0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.468+0000 7fa2b27ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2ac0cc300 con 0x7fa2ac0ccf90 2023-10-08T01:09:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.468+0000 7fa2b27ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2ac156bf0 con 0x7fa2ac0b1420 2023-10-08T01:09:00.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.468+0000 7fa2b27ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac0ae950 con 0x7fa2ac0cbf30 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0cbf30 0x7fa2ac0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2b0d9c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa2ac0b1420 0x7fa2ac0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0cbf30 0x7fa2ac0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:44554/0 (socket says 172.21.15.119:44554) 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2abfff700 1 -- 172.21.15.119:0/183146762 learned_addr learned my addr 172.21.15.119:0/183146762 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2b0d9c700 1 -- 172.21.15.119:0/183146762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0cbf30 msgr2=0x7fa2ac0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2b0d9c700 1 --2- 172.21.15.119:0/183146762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0cbf30 0x7fa2ac0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2b0d9c700 1 -- 172.21.15.119:0/183146762 >> v1:172.21.15.184:6789/0 conn(0x7fa2ac0ccf90 legacy=0x7fa2ac0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2b0d9c700 1 -- 172.21.15.119:0/183146762 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2ac0ad370 con 0x7fa2ac0b1420 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2abfff700 1 --2- 172.21.15.119:0/183146762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0cbf30 0x7fa2ac0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.469+0000 7fa2b0d9c700 1 --2- 172.21.15.119:0/183146762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac0c9740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009e90 tx=0x7fa2a4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcdd228ac09bec19 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:00.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2ab7fe700 1 -- 172.21.15.119:0/183146762 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a400e070 con 0x7fa2ac0b1420 2023-10-08T01:09:00.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2ab7fe700 1 -- 172.21.15.119:0/183146762 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a400bbf0 con 0x7fa2ac0b1420 2023-10-08T01:09:00.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2ab7fe700 1 -- 172.21.15.119:0/183146762 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4012430 con 0x7fa2ac0b1420 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2b27ff700 1 -- 172.21.15.119:0/183146762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0b1420 msgr2=0x7fa2ac0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/183146762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac0c9740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4009e90 tx=0x7fa2a4009780 comp rx=0 tx=0).stop 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2b27ff700 1 -- 172.21.15.119:0/183146762 shutdown_connections 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/183146762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac0c9740 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/183146762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0cbf30 0x7fa2ac0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2b27ff700 1 -- 172.21.15.119:0/183146762 >> 172.21.15.119:0/183146762 conn(0x7fa2ac06a320 msgr2=0x7fa2ac0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.470+0000 7fa2b27ff700 1 -- 172.21.15.119:0/183146762 shutdown_connections 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.471+0000 7fa2b27ff700 1 -- 172.21.15.119:0/183146762 wait complete. 2023-10-08T01:09:00.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.472+0000 7fa2b27ff700 1 Processor -- start 2023-10-08T01:09:00.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.472+0000 7fa2b27ff700 1 -- start start 2023-10-08T01:09:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.473+0000 7fa2b27ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.473+0000 7fa2b27ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0cbf30 0x7fa2ac0a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.473+0000 7fa2b27ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0ccf90 0x7fa2ac159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.473+0000 7fa2b27ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac16cfd0 con 0x7fa2ac0b1420 2023-10-08T01:09:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.473+0000 7fa2b27ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac169f90 con 0x7fa2ac0cbf30 2023-10-08T01:09:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.473+0000 7fa2b27ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2ac16a270 con 0x7fa2ac0ccf90 2023-10-08T01:09:00.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac141920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58416/0 (socket says 172.21.15.119:58416) 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 -- 172.21.15.119:0/1237713228 learned_addr learned my addr 172.21.15.119:0/1237713228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2b0d9c700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0ccf90 0x7fa2ac159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2a3fff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0cbf30 0x7fa2ac0a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 -- 172.21.15.119:0/1237713228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0ccf90 msgr2=0x7fa2ac159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0ccf90 0x7fa2ac159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 -- 172.21.15.119:0/1237713228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0cbf30 msgr2=0x7fa2ac0a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0cbf30 0x7fa2ac0a9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 -- 172.21.15.119:0/1237713228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c0094f0 con 0x7fa2ac0b1420 2023-10-08T01:09:00.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2abfff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac141920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00ec50 tx=0x7fa29c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:00.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.474+0000 7fa2a8ff9700 1 -- 172.21.15.119:0/1237713228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00c850 con 0x7fa2ac0b1420 2023-10-08T01:09:00.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.475+0000 7fa2a8ff9700 1 -- 172.21.15.119:0/1237713228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c00c9b0 con 0x7fa2ac0b1420 2023-10-08T01:09:00.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.475+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a4009a20 con 0x7fa2ac0b1420 2023-10-08T01:09:00.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.475+0000 7fa2a8ff9700 1 -- 172.21.15.119:0/1237713228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00cd50 con 0x7fa2ac0b1420 2023-10-08T01:09:00.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.475+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2ac16a830 con 0x7fa2ac0b1420 2023-10-08T01:09:00.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.476+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2ac04c3c0 con 0x7fa2ac0b1420 2023-10-08T01:09:00.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.476+0000 7fa2a8ff9700 1 -- 172.21.15.119:0/1237713228 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa29c013680 con 0x7fa2ac0b1420 2023-10-08T01:09:00.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.477+0000 7fa2a8ff9700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:00.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.477+0000 7fa2a8ff9700 1 -- 172.21.15.119:0/1237713228 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fa29c0846f0 con 0x7fa2ac0b1420 2023-10-08T01:09:00.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.477+0000 7fa2a3fff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:00.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.478+0000 7fa2a3fff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa28c066f30 0x7fa28c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0a7050 tx=0x7fa294007ec0 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:00.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.481+0000 7fa2a8ff9700 1 -- 172.21.15.119:0/1237713228 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa29c051820 con 0x7fa2ac0b1420 2023-10-08T01:09:00.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:00.751+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7fa2ac0ad650 con 0x7fa2ac0b1420 2023-10-08T01:09:01.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.261+0000 7fa2a8ff9700 1 -- 172.21.15.119:0/1237713228 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v472) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa29c054e40 con 0x7fa2ac0b1420 2023-10-08T01:09:01.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa28c066f30 msgr2=0x7fa28c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:01.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa28c066f30 0x7fa28c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac0a7050 tx=0x7fa294007ec0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2ac0b1420 msgr2=0x7fa2ac141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac141920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00ec50 tx=0x7fa29c009250 comp rx=0 tx=0).stop 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 shutdown_connections 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa28c066f30 0x7fa28c0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2ac0ccf90 0x7fa2ac159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2ac0b1420 0x7fa2ac141920 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 --2- 172.21.15.119:0/1237713228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2ac0cbf30 0x7fa2ac0a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 >> 172.21.15.119:0/1237713228 conn(0x7fa2ac06a320 msgr2=0x7fa2ac0caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 shutdown_connections 2023-10-08T01:09:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.264+0000 7fa2b27ff700 1 -- 172.21.15.119:0/1237713228 wait complete. 2023-10-08T01:09:01.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T01:09:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.468+0000 7f00115ff700 1 Processor -- start 2023-10-08T01:09:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.468+0000 7f00115ff700 1 -- start start 2023-10-08T01:09:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f00115ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f000c0c83f0 0x7f000c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f00115ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 0x7f000c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f00115ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000c15b0d0 con 0x7f000c0c87c0 2023-10-08T01:09:01.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f00115ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000c0c7a60 con 0x7f000c0c83f0 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f00115ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c15a4c0 con 0x7f000c0ccee0 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f000a7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 0x7f000c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f000affd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f000c0c83f0 0x7f000c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f000affd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f000c0c83f0 0x7f000c0c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58932/0 (socket says 172.21.15.119:58932) 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.470+0000 7f000affd700 1 -- 172.21.15.119:0/2518922966 learned_addr learned my addr 172.21.15.119:0/2518922966 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.471+0000 7f000affd700 1 -- 172.21.15.119:0/2518922966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 msgr2=0x7f000c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.471+0000 7f000affd700 1 --2- 172.21.15.119:0/2518922966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 0x7f000c1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.471+0000 7f000affd700 1 -- 172.21.15.119:0/2518922966 >> v1:172.21.15.184:6789/0 conn(0x7f000c0c87c0 legacy=0x7f000c06eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.471+0000 7f000affd700 1 -- 172.21.15.119:0/2518922966 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000c15a7a0 con 0x7f000c0c83f0 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.471+0000 7f000affd700 1 --2- 172.21.15.119:0/2518922966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 0x7f000c0c7520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efffc0099b0 tx=0x7efffc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a8835f89efec451 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:01.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.471+0000 7f0009ffb700 1 -- 172.21.15.119:0/2518922966 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc009ea0 con 0x7f000c0c83f0 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.471+0000 7f0009ffb700 1 -- 172.21.15.119:0/2518922966 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc00b860 con 0x7f000c0c83f0 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.471+0000 7f0009ffb700 1 -- 172.21.15.119:0/2518922966 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc00bc00 con 0x7f000c0c83f0 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.472+0000 7f00115ff700 1 -- 172.21.15.119:0/2518922966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 msgr2=0x7f000c0c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.472+0000 7f00115ff700 1 --2- 172.21.15.119:0/2518922966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 0x7f000c0c7520 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efffc0099b0 tx=0x7efffc009250 comp rx=0 tx=0).stop 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.472+0000 7f00115ff700 1 -- 172.21.15.119:0/2518922966 shutdown_connections 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.472+0000 7f00115ff700 1 --2- 172.21.15.119:0/2518922966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 0x7f000c0c7520 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.472+0000 7f00115ff700 1 --2- 172.21.15.119:0/2518922966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 0x7f000c1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.472+0000 7f00115ff700 1 -- 172.21.15.119:0/2518922966 >> 172.21.15.119:0/2518922966 conn(0x7f000c068e10 msgr2=0x7f000c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.472+0000 7f00115ff700 1 -- 172.21.15.119:0/2518922966 shutdown_connections 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.472+0000 7f00115ff700 1 -- 172.21.15.119:0/2518922966 wait complete. 2023-10-08T01:09:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.473+0000 7f00115ff700 1 Processor -- start 2023-10-08T01:09:01.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.473+0000 7f00115ff700 1 -- start start 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.474+0000 7f00115ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 0x7f000c14a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.474+0000 7f00115ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f000c0c87c0 0x7f000c16b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.474+0000 7f00115ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 0x7f000c16be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.474+0000 7f00115ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0c5260 con 0x7f000c0c87c0 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.474+0000 7f00115ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0c5540 con 0x7f000c0c83f0 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.474+0000 7f00115ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000c0c5820 con 0x7f000c0ccee0 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f000c0c87c0 0x7f000c16b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f000c0c87c0 0x7f000c16b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58432/0 (socket says 172.21.15.119:58432) 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 -- 172.21.15.119:0/2141556996 learned_addr learned my addr 172.21.15.119:0/2141556996 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:01.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000affd700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 0x7f000c14a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000b7fe700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 0x7f000c16be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 -- 172.21.15.119:0/2141556996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 msgr2=0x7f000c16be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 0x7f000c16be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 -- 172.21.15.119:0/2141556996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 msgr2=0x7f000c14a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 0x7f000c14a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 -- 172.21.15.119:0/2141556996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efffc0094f0 con 0x7f000c0c87c0 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.475+0000 7f000a7fc700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f000c0c87c0 0x7f000c16b950 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f000000e930 tx=0x7f00000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.476+0000 7efff2ffd700 1 -- 172.21.15.119:0/2141556996 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000000eed0 con 0x7f000c0c87c0 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.476+0000 7efff2ffd700 1 -- 172.21.15.119:0/2141556996 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000000c850 con 0x7f000c0c87c0 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.476+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000c0c5b00 con 0x7f000c0c87c0 2023-10-08T01:09:01.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.476+0000 7efff2ffd700 1 -- 172.21.15.119:0/2141556996 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000000cbf0 con 0x7f000c0c87c0 2023-10-08T01:09:01.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.476+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000c1478d0 con 0x7f000c0c87c0 2023-10-08T01:09:01.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.477+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000c04c3c0 con 0x7f000c0c87c0 2023-10-08T01:09:01.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.478+0000 7efff2ffd700 1 -- 172.21.15.119:0/2141556996 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0000013470 con 0x7f000c0c87c0 2023-10-08T01:09:01.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.478+0000 7efff2ffd700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7efff8066f30 0x7efff80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:01.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.479+0000 7efff2ffd700 1 -- 172.21.15.119:0/2141556996 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5862+0+0 (secure 0 0 0) 0x7f00000848e0 con 0x7f000c0c87c0 2023-10-08T01:09:01.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.479+0000 7f000affd700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7efff8066f30 0x7efff80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:01.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.479+0000 7f000affd700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7efff8066f30 0x7efff80693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efffc0099b0 tx=0x7efffc017000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:01.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.482+0000 7efff2ffd700 1 -- 172.21.15.119:0/2141556996 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0000051a10 con 0x7f000c0c87c0 2023-10-08T01:09:01.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.674+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f000c0c53a0 con 0x7f000c0c87c0 2023-10-08T01:09:01.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.679+0000 7efff2ffd700 1 -- 172.21.15.119:0/2141556996 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f000c0c53a0 con 0x7f000c0c87c0 2023-10-08T01:09:01.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7efff8066f30 msgr2=0x7efff80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7efff8066f30 0x7efff80693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efffc0099b0 tx=0x7efffc017000 comp rx=0 tx=0).stop 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f000c0c87c0 msgr2=0x7f000c16b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f000c0c87c0 0x7f000c16b950 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f000000e930 tx=0x7f00000091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 shutdown_connections 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7efff8066f30 0x7efff80693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f000c0ccee0 0x7f000c16be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f000c0c83f0 0x7f000c14a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 --2- 172.21.15.119:0/2141556996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f000c0c87c0 0x7f000c16b950 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 >> 172.21.15.119:0/2141556996 conn(0x7f000c068e10 msgr2=0x7f000c0b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:01.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 shutdown_connections 2023-10-08T01:09:01.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:01.682+0000 7f00115ff700 1 -- 172.21.15.119:0/2141556996 wait complete. 2023-10-08T01:09:01.700 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:09:01.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:09:01.779 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:09:01.863 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:09:01.863 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T01:09:01.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:09:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.042+0000 7faf04d9d700 1 Processor -- start 2023-10-08T01:09:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.042+0000 7faf04d9d700 1 -- start start 2023-10-08T01:09:02.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.043+0000 7faf04d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf00150550 0x7faf00157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.043+0000 7faf04d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf000c7410 0x7faf00069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.043+0000 7faf04d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0006e2a0 con 0x7faf00150180 2023-10-08T01:09:02.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.043+0000 7faf04d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0006b2e0 con 0x7faf00150550 2023-10-08T01:09:02.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.043+0000 7faf04d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf000c78e0 con 0x7faf000c7410 2023-10-08T01:09:02.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.043+0000 7faefeffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf00150550 0x7faf00157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.043+0000 7faefeffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faf00150550 0x7faf00157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58448/0 (socket says 172.21.15.119:58448) 2023-10-08T01:09:02.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.043+0000 7faefeffd700 1 -- 172.21.15.119:0/3814179948 learned_addr learned my addr 172.21.15.119:0/3814179948 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:02.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefeffd700 1 -- 172.21.15.119:0/3814179948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf000c7410 msgr2=0x7faf00069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:02.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefeffd700 1 --2- 172.21.15.119:0/3814179948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf000c7410 0x7faf00069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefeffd700 1 -- 172.21.15.119:0/3814179948 >> v1:172.21.15.119:6789/0 conn(0x7faf00150180 legacy=0x7faf000c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefeffd700 1 -- 172.21.15.119:0/3814179948 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000c7bc0 con 0x7faf00150550 2023-10-08T01:09:02.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefe7fc700 1 -- 172.21.15.119:0/3814179948 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069110189 0 0) 0x7faf0006e2a0 con 0x7faf00150180 2023-10-08T01:09:02.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefeffd700 1 --2- 172.21.15.119:0/3814179948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf00150550 0x7faf00157770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faef400ab50 tx=0x7faef400a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b8da89ff468112 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:02.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefe7fc700 1 -- 172.21.15.119:0/3814179948 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400ca60 con 0x7faf00150550 2023-10-08T01:09:02.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefe7fc700 1 -- 172.21.15.119:0/3814179948 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef400cbc0 con 0x7faf00150550 2023-10-08T01:09:02.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.044+0000 7faefe7fc700 1 -- 172.21.15.119:0/3814179948 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4018400 con 0x7faf00150550 2023-10-08T01:09:02.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.045+0000 7faf04d9d700 1 -- 172.21.15.119:0/3814179948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf00150550 msgr2=0x7faf00157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.045+0000 7faf04d9d700 1 --2- 172.21.15.119:0/3814179948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf00150550 0x7faf00157770 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faef400ab50 tx=0x7faef400a3f0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.045+0000 7faf04d9d700 1 -- 172.21.15.119:0/3814179948 shutdown_connections 2023-10-08T01:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.045+0000 7faf04d9d700 1 --2- 172.21.15.119:0/3814179948 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf000c7410 0x7faf00069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.045+0000 7faf04d9d700 1 --2- 172.21.15.119:0/3814179948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf00150550 0x7faf00157770 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.045+0000 7faf04d9d700 1 -- 172.21.15.119:0/3814179948 >> 172.21.15.119:0/3814179948 conn(0x7faf000cc1f0 msgr2=0x7faf000cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.045+0000 7faf04d9d700 1 -- 172.21.15.119:0/3814179948 shutdown_connections 2023-10-08T01:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.045+0000 7faf04d9d700 1 -- 172.21.15.119:0/3814179948 wait complete. 2023-10-08T01:09:02.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.046+0000 7faf04d9d700 1 Processor -- start 2023-10-08T01:09:02.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.046+0000 7faf04d9d700 1 -- start start 2023-10-08T01:09:02.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.047+0000 7faf04d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf000c7410 0x7faf000ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.047+0000 7faf04d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00150180 0x7faf00147aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.047+0000 7faf04d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf00150550 0x7faf000aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.047+0000 7faf04d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf000afff0 con 0x7faf000c7410 2023-10-08T01:09:02.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.047+0000 7faf04d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0014c860 con 0x7faf00150180 2023-10-08T01:09:02.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.047+0000 7faf04d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf00147fe0 con 0x7faf00150550 2023-10-08T01:09:02.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf000c7410 0x7faf000ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf000c7410 0x7faf000ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58464/0 (socket says 172.21.15.119:58464) 2023-10-08T01:09:02.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 -- 172.21.15.119:0/920604607 learned_addr learned my addr 172.21.15.119:0/920604607 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:02.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faefeffd700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00150180 0x7faf00147aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faefffff700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf00150550 0x7faf000aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 -- 172.21.15.119:0/920604607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf00150550 msgr2=0x7faf000aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf00150550 0x7faf000aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 -- 172.21.15.119:0/920604607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00150180 msgr2=0x7faf00147aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00150180 0x7faf00147aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 -- 172.21.15.119:0/920604607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef400a690 con 0x7faf000c7410 2023-10-08T01:09:02.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faeff7fe700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf000c7410 0x7faf000ad760 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faef000cb60 tx=0x7faef00017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:02.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faee77fe700 1 -- 172.21.15.119:0/920604607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000a7b0 con 0x7faf000c7410 2023-10-08T01:09:02.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faee77fe700 1 -- 172.21.15.119:0/920604607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef000a930 con 0x7faf000c7410 2023-10-08T01:09:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.048+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf000af4d0 con 0x7faf000c7410 2023-10-08T01:09:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.049+0000 7faee77fe700 1 -- 172.21.15.119:0/920604607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000acf0 con 0x7faf000c7410 2023-10-08T01:09:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.049+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf000afa30 con 0x7faf000c7410 2023-10-08T01:09:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.049+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0014c630 con 0x7faf000c7410 2023-10-08T01:09:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.050+0000 7faee77fe700 1 -- 172.21.15.119:0/920604607 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faef0003030 con 0x7faf000c7410 2023-10-08T01:09:02.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.051+0000 7faee77fe700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7faeec066f30 0x7faeec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.051+0000 7faee77fe700 1 -- 172.21.15.119:0/920604607 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5862+0+0 (secure 0 0 0) 0x7faef00847c0 con 0x7faf000c7410 2023-10-08T01:09:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.051+0000 7faefeffd700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7faeec066f30 0x7faeec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.052+0000 7faefeffd700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7faeec066f30 0x7faeec0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faef400ab50 tx=0x7faef401a000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:02.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.055+0000 7faee77fe700 1 -- 172.21.15.119:0/920604607 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faef00519a0 con 0x7faf000c7410 2023-10-08T01:09:02.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.287+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf0004c3c0 con 0x7faf000c7410 2023-10-08T01:09:02.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.288+0000 7faee77fe700 1 -- 172.21.15.119:0/920604607 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3535 (secure 0 0 0) 0x7faef0054fc0 con 0x7faf000c7410 2023-10-08T01:09:02.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:02.289 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4610,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/768996961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":768996961},{"type":"v1","addr":"172.21.15.184:6834","nonce":768996961}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4819,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1876644893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1876644893},{"type":"v1","addr":"172.21.15.119:6837","nonce":1876644893}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4825,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1187803776","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1187803776},{"type":"v1","addr":"172.21.15.184:6835","nonce":1187803776}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5640,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/959101113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":959101113},{"type":"v1","addr":"172.21.15.119:6833","nonce":959101113}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-10-08T01:09:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7faeec066f30 msgr2=0x7faeec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7faeec066f30 0x7faeec0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faef400ab50 tx=0x7faef401a000 comp rx=0 tx=0).stop 2023-10-08T01:09:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf000c7410 msgr2=0x7faf000ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf000c7410 0x7faf000ad760 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7faef000cb60 tx=0x7faef00017a0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 shutdown_connections 2023-10-08T01:09:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7faeec066f30 0x7faeec0693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faf00150550 0x7faf000aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf000c7410 0x7faf000ad760 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 --2- 172.21.15.119:0/920604607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faf00150180 0x7faf00147aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 >> 172.21.15.119:0/920604607 conn(0x7faf000cc1f0 msgr2=0x7faf00157900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 shutdown_connections 2023-10-08T01:09:02.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.290+0000 7faf04d9d700 1 -- 172.21.15.119:0/920604607 wait complete. 2023-10-08T01:09:02.293 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2023-10-08T01:09:02.309 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T01:09:02.309 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T01:09:02.309 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T01:09:02.309 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:09:02.309 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T01:09:02.309 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T01:09:02.310 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T01:09:02.310 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:09:02.310 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T01:09:02.310 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T01:09:02.310 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T01:09:02.310 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:09:02.311 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T01:09:02.311 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T01:09:02.311 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T01:09:02.311 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:09:02.324 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T01:09:02.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T01:09:02.326 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T01:09:02.326 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T01:09:02.328 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T01:09:02.329 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T01:09:02.329 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T01:09:02.329 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T01:09:02.370 INFO:tasks.ceph.mds.a:Started 2023-10-08T01:09:02.371 DEBUG:teuthology.parallel:result is None 2023-10-08T01:09:02.374 INFO:tasks.ceph.mds.c:Started 2023-10-08T01:09:02.374 DEBUG:teuthology.parallel:result is None 2023-10-08T01:09:02.375 INFO:tasks.ceph.mds.d:Started 2023-10-08T01:09:02.376 DEBUG:teuthology.parallel:result is None 2023-10-08T01:09:02.378 INFO:tasks.ceph.mds.b:Started 2023-10-08T01:09:02.379 DEBUG:teuthology.parallel:result is None 2023-10-08T01:09:02.379 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T01:09:02.505 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:09:02.505 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T01:09:02.513 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T01:09:02.513 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T01:09:02.515 INFO:tasks.ceph.mds.b.smithi184.stderr:2023-10-08T01:09:02.515+0000 7f70ff7fd700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-10-08T01:09:02.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.573+0000 7fa7d67ff700 1 Processor -- start 2023-10-08T01:09:02.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.573+0000 7fa7d67ff700 1 -- start start 2023-10-08T01:09:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7d67ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa7d0155c10 0x7fa7d0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7d67ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 0x7fa7d00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7d67ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d00acac0 con 0x7fa7d00ac230 2023-10-08T01:09:02.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7d67ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00ad930 con 0x7fa7d0155c10 2023-10-08T01:09:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7d67ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00acd20 con 0x7fa7d00ac6d0 2023-10-08T01:09:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7cffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa7d0155c10 0x7fa7d0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7c7fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa7d00ac230 0x7fa7d01568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49718/0 (socket says 172.21.15.119:49718) 2023-10-08T01:09:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7c7fff700 1 -- 172.21.15.119:0/2987665463 learned_addr learned my addr 172.21.15.119:0/2987665463 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.575+0000 7fa7d4d9c700 1 --2- 172.21.15.119:0/2987665463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 0x7fa7d00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cffff700 1 -- 172.21.15.119:0/2987665463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 msgr2=0x7fa7d00b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cffff700 1 --2- 172.21.15.119:0/2987665463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 0x7fa7d00b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cffff700 1 -- 172.21.15.119:0/2987665463 >> v1:172.21.15.119:6789/0 conn(0x7fa7d00ac230 legacy=0x7fa7d01568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cffff700 1 -- 172.21.15.119:0/2987665463 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00ad000 con 0x7fa7d0155c10 2023-10-08T01:09:02.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cf7fe700 1 -- 172.21.15.119:0/2987665463 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973635567 0 0) 0x7fa7d00acac0 con 0x7fa7d00ac230 2023-10-08T01:09:02.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cffff700 1 --2- 172.21.15.119:0/2987665463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0155fe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00acc50 tx=0x7fa7c000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf9598e6ee9961a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:02.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cf7fe700 1 -- 172.21.15.119:0/2987665463 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000c970 con 0x7fa7d0155c10 2023-10-08T01:09:02.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cf7fe700 1 -- 172.21.15.119:0/2987665463 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c000cad0 con 0x7fa7d0155c10 2023-10-08T01:09:02.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.576+0000 7fa7cf7fe700 1 -- 172.21.15.119:0/2987665463 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000ce90 con 0x7fa7d0155c10 2023-10-08T01:09:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.577+0000 7fa7d67ff700 1 -- 172.21.15.119:0/2987665463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 msgr2=0x7fa7d0155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.577+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/2987665463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0155fe0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00acc50 tx=0x7fa7c000a330 comp rx=0 tx=0).stop 2023-10-08T01:09:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.577+0000 7fa7d67ff700 1 -- 172.21.15.119:0/2987665463 shutdown_connections 2023-10-08T01:09:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.577+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/2987665463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 0x7fa7d00b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.577+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/2987665463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0155fe0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.577+0000 7fa7d67ff700 1 -- 172.21.15.119:0/2987665463 >> 172.21.15.119:0/2987665463 conn(0x7fa7d006a320 msgr2=0x7fa7d00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:02.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.577+0000 7fa7d67ff700 1 -- 172.21.15.119:0/2987665463 shutdown_connections 2023-10-08T01:09:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.577+0000 7fa7d67ff700 1 -- 172.21.15.119:0/2987665463 wait complete. 2023-10-08T01:09:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.578+0000 7fa7d67ff700 1 Processor -- start 2023-10-08T01:09:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.578+0000 7fa7d67ff700 1 -- start start 2023-10-08T01:09:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d67ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d00ac230 0x7fa7d00a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d67ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 0x7fa7d016a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d67ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d67ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d016a860 con 0x7fa7d0155c10 2023-10-08T01:09:02.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d67ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d0151360 con 0x7fa7d00ac230 2023-10-08T01:09:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d67ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00a67b0 con 0x7fa7d00ac6d0 2023-10-08T01:09:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d4d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d4d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58502/0 (socket says 172.21.15.119:58502) 2023-10-08T01:09:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.579+0000 7fa7d4d9c700 1 -- 172.21.15.119:0/3251874704 learned_addr learned my addr 172.21.15.119:0/3251874704 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7cffff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d00ac230 0x7fa7d00a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7c7fff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 0x7fa7d016a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7d4d9c700 1 -- 172.21.15.119:0/3251874704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 msgr2=0x7fa7d016a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7d4d9c700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 0x7fa7d016a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7d4d9c700 1 -- 172.21.15.119:0/3251874704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d00ac230 msgr2=0x7fa7d00a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7d4d9c700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d00ac230 0x7fa7d00a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7d4d9c700 1 -- 172.21.15.119:0/3251874704 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c000a5d0 con 0x7fa7d0155c10 2023-10-08T01:09:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7d4d9c700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0150e20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009e90 tx=0x7fa7c800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:02.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7ccff9700 1 -- 172.21.15.119:0/3251874704 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c800cb70 con 0x7fa7d0155c10 2023-10-08T01:09:02.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7ccff9700 1 -- 172.21.15.119:0/3251874704 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c800f040 con 0x7fa7d0155c10 2023-10-08T01:09:02.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7ccff9700 1 -- 172.21.15.119:0/3251874704 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8014480 con 0x7fa7d0155c10 2023-10-08T01:09:02.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d00a5ba0 con 0x7fa7d0155c10 2023-10-08T01:09:02.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.580+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d00a6160 con 0x7fa7d0155c10 2023-10-08T01:09:02.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.581+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d004c3c0 con 0x7fa7d0155c10 2023-10-08T01:09:02.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.582+0000 7fa7ccff9700 1 -- 172.21.15.119:0/3251874704 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7c8014830 con 0x7fa7d0155c10 2023-10-08T01:09:02.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.583+0000 7fa7ccff9700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa7b0066f30 0x7fa7b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:02.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.583+0000 7fa7ccff9700 1 -- 172.21.15.119:0/3251874704 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5862+0+0 (secure 0 0 0) 0x7fa7c8084fe0 con 0x7fa7d0155c10 2023-10-08T01:09:02.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.583+0000 7fa7cffff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa7b0066f30 0x7fa7b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:02.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.584+0000 7fa7cffff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa7b0066f30 0x7fa7b00693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00a9720 tx=0x7fa7c0019000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:02.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.587+0000 7fa7ccff9700 1 -- 172.21.15.119:0/3251874704 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7c8052170 con 0x7fa7d0155c10 2023-10-08T01:09:02.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:02.794+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa7d00a68f0 con 0x7fa7d0155c10 2023-10-08T01:09:04.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.163+0000 7fa7ccff9700 1 -- 172.21.15.119:0/3251874704 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v55) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa7c8055790 con 0x7fa7d0155c10 2023-10-08T01:09:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa7b0066f30 msgr2=0x7fa7b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa7b0066f30 0x7fa7b00693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00a9720 tx=0x7fa7c0019000 comp rx=0 tx=0).stop 2023-10-08T01:09:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 msgr2=0x7fa7d0150e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0150e20 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009e90 tx=0x7fa7c800b040 comp rx=0 tx=0).stop 2023-10-08T01:09:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 shutdown_connections 2023-10-08T01:09:04.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fa7b0066f30 0x7fa7b00693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7d0155c10 0x7fa7d0150e20 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7d00ac230 0x7fa7d00a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 --2- 172.21.15.119:0/3251874704 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7d00ac6d0 0x7fa7d016a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 >> 172.21.15.119:0/3251874704 conn(0x7fa7d006a320 msgr2=0x7fa7d01523c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:04.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 shutdown_connections 2023-10-08T01:09:04.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.166+0000 7fa7d67ff700 1 -- 172.21.15.119:0/3251874704 wait complete. 2023-10-08T01:09:04.169 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T01:09:04.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T01:09:04.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.368+0000 7f80b91ff700 1 Processor -- start 2023-10-08T01:09:04.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.368+0000 7f80b91ff700 1 -- start start 2023-10-08T01:09:04.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.369+0000 7f80b91ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b406bcd0 0x7f80b40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.369+0000 7f80b91ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b415d7e0 con 0x7f80b406c0a0 2023-10-08T01:09:04.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.369+0000 7f80b91ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b415dac0 con 0x7f80b415df50 2023-10-08T01:09:04.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.369+0000 7f80b91ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b415aa30 con 0x7f80b406bcd0 2023-10-08T01:09:04.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b259c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f80b406c0a0 0x7f80b415a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49728/0 (socket says 172.21.15.119:49728) 2023-10-08T01:09:04.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b259c700 1 -- 172.21.15.119:0/2603817502 learned_addr learned my addr 172.21.15.119:0/2603817502 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:04.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b2d9d700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b406bcd0 0x7f80b40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873027853 0 0) 0x7f80b415d7e0 con 0x7f80b406c0a0 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f809c003570 con 0x7f80b406c0a0 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3068966548 0 0) 0x7f809c003570 con 0x7f80b406c0a0 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b406bcd0 msgr2=0x7f80b40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b406bcd0 0x7f80b40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 >> v1:172.21.15.184:6789/0 conn(0x7f80b415df50 legacy=0x7f80b40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b415ad10 con 0x7f80b406c0a0 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f80a8001470 con 0x7f80b406c0a0 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 >> v1:172.21.15.119:6789/0 conn(0x7f80b406c0a0 legacy=0x7f80b415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.370+0000 7f80b1d9b700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b406bcd0 0x7f809c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b1d9b700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c003fd0 0x7f809c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b1d9b700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c004d50 0x7f809c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b415aa30 con 0x7f809c003fd0 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c00bbf0 con 0x7f809c004d50 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c00bdb0 con 0x7f80b406bcd0 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b259c700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c003fd0 0x7f809c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b2d9d700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b406bcd0 0x7f809c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b359e700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c004d50 0x7f809c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b259c700 1 -- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b406bcd0 msgr2=0x7f809c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b259c700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f80b406bcd0 0x7f809c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b259c700 1 -- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c004d50 msgr2=0x7f809c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b259c700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f809c004d50 0x7f809c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b259c700 1 -- 172.21.15.119:0/2603817502 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c003570 con 0x7f809c003fd0 2023-10-08T01:09:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.371+0000 7f80b259c700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c003fd0 0x7f809c004810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c390 tx=0x7f80b40ce4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8504c29b62f7e5ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800cc60 con 0x7f809c003fd0 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a800ce00 con 0x7f809c003fd0 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b1d9b700 1 -- 172.21.15.119:0/2603817502 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8005b70 con 0x7f809c003fd0 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b91ff700 1 -- 172.21.15.119:0/2603817502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c003fd0 msgr2=0x7f809c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b91ff700 1 --2- 172.21.15.119:0/2603817502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f809c003fd0 0x7f809c004810 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f80a800c390 tx=0x7f80b40ce4d0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b359e700 1 -- 172.21.15.119:0/2603817502 reap_dead start 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b91ff700 1 -- 172.21.15.119:0/2603817502 shutdown_connections 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b91ff700 1 -- 172.21.15.119:0/2603817502 >> 172.21.15.119:0/2603817502 conn(0x7f80b40ce600 msgr2=0x7f80b40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b91ff700 1 -- 172.21.15.119:0/2603817502 shutdown_connections 2023-10-08T01:09:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.372+0000 7f80b91ff700 1 -- 172.21.15.119:0/2603817502 wait complete. 2023-10-08T01:09:04.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.374+0000 7f80b91ff700 1 Processor -- start 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.374+0000 7f80b91ff700 1 -- start start 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b91ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c003fd0 0x7f80b40c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b91ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80b4169850 0x7f80b414f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b91ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b40cac80 0x7f80b40cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b91ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40cb660 con 0x7f80b4169850 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b91ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c4290 con 0x7f80b40cac80 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b91ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40b2fe0 con 0x7f809c003fd0 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80b4169850 0x7f80b414f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80b4169850 0x7f80b414f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58520/0 (socket says 172.21.15.119:58520) 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b259c700 1 -- 172.21.15.119:0/3881048595 learned_addr learned my addr 172.21.15.119:0/3881048595 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b2d9d700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c003fd0 0x7f80b40c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b359e700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b40cac80 0x7f80b40cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b259c700 1 -- 172.21.15.119:0/3881048595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c003fd0 msgr2=0x7f80b40c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b259c700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c003fd0 0x7f80b40c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b259c700 1 -- 172.21.15.119:0/3881048595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b40cac80 msgr2=0x7f80b40cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b259c700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b40cac80 0x7f80b40cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.375+0000 7f80b259c700 1 -- 172.21.15.119:0/3881048595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b40b1780 con 0x7f80b4169850 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.376+0000 7f80b259c700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80b4169850 0x7f80b414f4f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f80a8007d10 tx=0x7f80a8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.376+0000 7f809affd700 1 -- 172.21.15.119:0/3881048595 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a80070d0 con 0x7f80b4169850 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.376+0000 7f809affd700 1 -- 172.21.15.119:0/3881048595 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a8007230 con 0x7f80b4169850 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.376+0000 7f809affd700 1 -- 172.21.15.119:0/3881048595 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a80075f0 con 0x7f80b4169850 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.376+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b4159cc0 con 0x7f80b4169850 2023-10-08T01:09:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.376+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b415a280 con 0x7f80b4169850 2023-10-08T01:09:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.377+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b40b2db0 con 0x7f80b4169850 2023-10-08T01:09:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.378+0000 7f809affd700 1 -- 172.21.15.119:0/3881048595 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f80a8019070 con 0x7f80b4169850 2023-10-08T01:09:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.379+0000 7f809affd700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.379+0000 7f809affd700 1 -- 172.21.15.119:0/3881048595 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5217+0+0 (secure 0 0 0) 0x7f80a8085340 con 0x7f80b4169850 2023-10-08T01:09:04.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.379+0000 7f80b2d9d700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.380+0000 7f80b2d9d700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f80a0066f30 0x7f80a00693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009170 tx=0x7f80a4008040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.383+0000 7f809affd700 1 -- 172.21.15.119:0/3881048595 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f80b40b2db0 con 0x7f80b4169850 2023-10-08T01:09:04.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.650+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f80b40b2db0 con 0x7f80b4169850 2023-10-08T01:09:04.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.650+0000 7f809affd700 1 -- 172.21.15.119:0/3881048595 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f80b40b2db0 con 0x7f80b4169850 2023-10-08T01:09:04.651 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:09:04.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.654 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T01:09:04.657 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:09:04.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.660 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T01:09:04.660 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T01:09:04.660 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:09:04.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.663 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T01:09:04.664 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:09:04.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:09:04.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:09:04.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T01:09:04.668 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:09:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f80a0066f30 msgr2=0x7f80a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f80a0066f30 0x7f80a00693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009170 tx=0x7f80a4008040 comp rx=0 tx=0).stop 2023-10-08T01:09:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80b4169850 msgr2=0x7f80b414f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80b4169850 0x7f80b414f4f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f80a8007d10 tx=0x7f80a8015000 comp rx=0 tx=0).stop 2023-10-08T01:09:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 shutdown_connections 2023-10-08T01:09:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f80a0066f30 0x7f80a00693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f809c003fd0 0x7f80b40c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f80b40cac80 0x7f80b40cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 --2- 172.21.15.119:0/3881048595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80b4169850 0x7f80b414f4f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 >> 172.21.15.119:0/3881048595 conn(0x7f80b40ce600 msgr2=0x7f80b406d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:04.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 shutdown_connections 2023-10-08T01:09:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.653+0000 7f80b91ff700 1 -- 172.21.15.119:0/3881048595 wait complete. 2023-10-08T01:09:04.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:09:04.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.854+0000 7f0aa59ff700 1 Processor -- start 2023-10-08T01:09:04.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.855+0000 7f0aa59ff700 1 -- start start 2023-10-08T01:09:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.856+0000 7f0aa59ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0aa00b1420 0x7f0aa00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.856+0000 7f0aa59ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa00cc300 con 0x7f0aa00ccf90 2023-10-08T01:09:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.856+0000 7f0aa59ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa00ad370 con 0x7f0aa00cbf30 2023-10-08T01:09:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.856+0000 7f0aa59ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa0156bf0 con 0x7f0aa00b1420 2023-10-08T01:09:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.856+0000 7f0a9f7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0aa00ccf90 0x7f0aa00c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49740/0 (socket says 172.21.15.119:49740) 2023-10-08T01:09:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.856+0000 7f0a9f7fe700 1 -- 172.21.15.119:0/3798383629 learned_addr learned my addr 172.21.15.119:0/3798383629 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.856+0000 7f0a97fff700 1 --2- 172.21.15.119:0/3798383629 >> v2:172.21.15.184:3300/0 conn(0x7f0aa00b1420 0x7f0aa00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a9e7fc700 1 -- 172.21.15.119:0/3798383629 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481090820 0 0) 0x7f0aa00cc300 con 0x7f0aa00ccf90 2023-10-08T01:09:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a9e7fc700 1 -- 172.21.15.119:0/3798383629 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a8c003570 con 0x7f0aa00ccf90 2023-10-08T01:09:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a97fff700 1 -- 172.21.15.119:0/3798383629 >> v1:172.21.15.184:6789/0 conn(0x7f0aa00cbf30 legacy=0x7f0aa00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a97fff700 1 -- 172.21.15.119:0/3798383629 >> v1:172.21.15.119:6789/0 conn(0x7f0aa00ccf90 legacy=0x7f0aa00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a9e7fc700 1 -- 172.21.15.119:0/3798383629 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 20916929 0 0) 0x7f0a8c003570 con 0x7f0aa00ccf90 2023-10-08T01:09:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a97fff700 1 -- 172.21.15.119:0/3798383629 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa00ad650 con 0x7f0aa00b1420 2023-10-08T01:09:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a97fff700 1 --2- 172.21.15.119:0/3798383629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa00b1420 0x7f0aa00ae410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0a880098e0 tx=0x7f0a880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b2cc4c0c3a106e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a9e7fc700 1 -- 172.21.15.119:0/3798383629 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a980017f0 con 0x7f0aa00b1420 2023-10-08T01:09:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.857+0000 7f0a9e7fc700 1 -- 172.21.15.119:0/3798383629 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a88011420 con 0x7f0aa00b1420 2023-10-08T01:09:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.858+0000 7f0a9e7fc700 1 -- 172.21.15.119:0/3798383629 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a880117c0 con 0x7f0aa00b1420 2023-10-08T01:09:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.858+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3798383629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa00b1420 msgr2=0x7f0aa00ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.858+0000 7f0aa59ff700 1 --2- 172.21.15.119:0/3798383629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa00b1420 0x7f0aa00ae410 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0a880098e0 tx=0x7f0a880091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.858+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3798383629 shutdown_connections 2023-10-08T01:09:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.858+0000 7f0aa59ff700 1 --2- 172.21.15.119:0/3798383629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa00b1420 0x7f0aa00ae410 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.858+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3798383629 >> 172.21.15.119:0/3798383629 conn(0x7f0aa006a320 msgr2=0x7f0aa00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.858+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3798383629 shutdown_connections 2023-10-08T01:09:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.858+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3798383629 wait complete. 2023-10-08T01:09:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.859+0000 7f0aa59ff700 1 Processor -- start 2023-10-08T01:09:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.860+0000 7f0aa59ff700 1 -- start start 2023-10-08T01:09:04.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.860+0000 7f0aa59ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0aa00b1420 0x7f0aa0155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.860+0000 7f0aa59ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00cbf30 0x7f0aa0169af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0aa59ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa00ccf90 0x7f0aa00a8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0aa59ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa00a84a0 con 0x7f0aa00b1420 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0aa59ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa016a3e0 con 0x7f0aa00ccf90 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0aa59ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa016a6c0 con 0x7f0aa00cbf30 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0aa00b1420 0x7f0aa0155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0aa00b1420 0x7f0aa0155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55006/0 (socket says 172.21.15.119:55006) 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 -- 172.21.15.119:0/3744396923 learned_addr learned my addr 172.21.15.119:0/3744396923 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 -- 172.21.15.119:0/3744396923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00cbf30 msgr2=0x7f0aa0169af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00cbf30 0x7f0aa0169af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 -- 172.21.15.119:0/3744396923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa00ccf90 msgr2=0x7f0aa00a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa00ccf90 0x7f0aa00a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 -- 172.21.15.119:0/3744396923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88009420 con 0x7f0aa00b1420 2023-10-08T01:09:04.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a9effd700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0aa00b1420 0x7f0aa0155b10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000c590 tx=0x7f0a9000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.861+0000 7f0a96ffd700 1 -- 172.21.15.119:0/3744396923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90011590 con 0x7f0aa00b1420 2023-10-08T01:09:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.862+0000 7f0a96ffd700 1 -- 172.21.15.119:0/3744396923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a900116f0 con 0x7f0aa00b1420 2023-10-08T01:09:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.862+0000 7f0a96ffd700 1 -- 172.21.15.119:0/3744396923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90011ab0 con 0x7f0aa00b1420 2023-10-08T01:09:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.862+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa016a9a0 con 0x7f0aa00b1420 2023-10-08T01:09:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.862+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa0156050 con 0x7f0aa00b1420 2023-10-08T01:09:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.863+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa004c3c0 con 0x7f0aa00b1420 2023-10-08T01:09:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.864+0000 7f0a96ffd700 1 -- 172.21.15.119:0/3744396923 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a9001b3b0 con 0x7f0aa00b1420 2023-10-08T01:09:04.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.864+0000 7f0a96ffd700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f0a80066f30 0x7f0a800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.864+0000 7f0a96ffd700 1 -- 172.21.15.119:0/3744396923 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5217+0+0 (secure 0 0 0) 0x7f0a900839a0 con 0x7f0aa00b1420 2023-10-08T01:09:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.865+0000 7f0a97fff700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f0a80066f30 0x7f0a800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.865+0000 7f0a97fff700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f0a80066f30 0x7f0a800693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00b1820 tx=0x7f0a88016000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:04.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:04.868+0000 7f0a96ffd700 1 -- 172.21.15.119:0/3744396923 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a90050d50 con 0x7f0aa00b1420 2023-10-08T01:09:05.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.062+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0aa006f8a0 con 0x7f0aa00b1420 2023-10-08T01:09:05.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.063+0000 7f0a96ffd700 1 -- 172.21.15.119:0/3744396923 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+12735 (secure 0 0 0) 0x7f0a90017020 con 0x7f0aa00b1420 2023-10-08T01:09:05.065 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:05.065 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:09:04.157471+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:09:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.065+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f0a80066f30 msgr2=0x7f0a800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.065+0000 7f0aa59ff700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f0a80066f30 0x7f0a800693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00b1820 tx=0x7f0a88016000 comp rx=0 tx=0).stop 2023-10-08T01:09:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0aa00b1420 msgr2=0x7f0aa0155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0aa00b1420 0x7f0aa0155b10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0a9000c590 tx=0x7f0a9000c950 comp rx=0 tx=0).stop 2023-10-08T01:09:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 shutdown_connections 2023-10-08T01:09:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f0a80066f30 0x7f0a800693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0aa00ccf90 0x7f0aa00a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0aa00b1420 0x7f0aa0155b10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 --2- 172.21.15.119:0/3744396923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0aa00cbf30 0x7f0aa0169af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 >> 172.21.15.119:0/3744396923 conn(0x7f0aa006a320 msgr2=0x7f0aa00cb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 shutdown_connections 2023-10-08T01:09:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.066+0000 7f0aa59ff700 1 -- 172.21.15.119:0/3744396923 wait complete. 2023-10-08T01:09:05.085 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T01:09:05.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:09:05.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.270+0000 7fe3ef59d700 1 Processor -- start 2023-10-08T01:09:05.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.270+0000 7fe3ef59d700 1 -- start start 2023-10-08T01:09:05.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.271+0000 7fe3ef59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3e80b4e70 0x7fe3e80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.271+0000 7fe3ef59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe3e80b16d0 0x7fe3e80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.271+0000 7fe3ef59d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e80b2080 con 0x7fe3e815d240 2023-10-08T01:09:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.271+0000 7fe3ef59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e80cfb00 con 0x7fe3e80b4e70 2023-10-08T01:09:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.271+0000 7fe3ef59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e80b5c40 con 0x7fe3e80b16d0 2023-10-08T01:09:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3ee59b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3e80b4e70 0x7fe3e80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3eed9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe3e815d240 0x7fe3e815d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44390/0 (socket says 172.21.15.119:44390) 2023-10-08T01:09:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3eed9c700 1 -- 172.21.15.119:0/2104389181 learned_addr learned my addr 172.21.15.119:0/2104389181 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3ee59b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3e80b4e70 0x7fe3e80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55012/0 (socket says 172.21.15.119:55012) 2023-10-08T01:09:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3edd9a700 1 --2- 172.21.15.119:0/2104389181 >> v2:172.21.15.184:3300/0 conn(0x7fe3e80b16d0 0x7fe3e80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3ee59b700 1 -- 172.21.15.119:0/2104389181 >> v2:172.21.15.184:3300/0 conn(0x7fe3e80b16d0 msgr2=0x7fe3e80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3ee59b700 1 --2- 172.21.15.119:0/2104389181 >> v2:172.21.15.184:3300/0 conn(0x7fe3e80b16d0 0x7fe3e80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3ee59b700 1 -- 172.21.15.119:0/2104389181 >> v1:172.21.15.119:6789/0 conn(0x7fe3e815d240 legacy=0x7fe3e815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3ee59b700 1 -- 172.21.15.119:0/2104389181 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80b5f20 con 0x7fe3e80b4e70 2023-10-08T01:09:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.272+0000 7fe3ee59b700 1 --2- 172.21.15.119:0/2104389181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e80b4e70 0x7fe3e80ccff0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8089830 tx=0x7fe3e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5152cda2e2ce3a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:05.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.274+0000 7fe3ed599700 1 -- 172.21.15.119:0/2104389181 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4011630 con 0x7fe3e80b4e70 2023-10-08T01:09:05.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.274+0000 7fe3ed599700 1 -- 172.21.15.119:0/2104389181 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e4011790 con 0x7fe3e80b4e70 2023-10-08T01:09:05.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.274+0000 7fe3ed599700 1 -- 172.21.15.119:0/2104389181 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4011b30 con 0x7fe3e80b4e70 2023-10-08T01:09:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.275+0000 7fe3ef59d700 1 -- 172.21.15.119:0/2104389181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e80b4e70 msgr2=0x7fe3e80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.275+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/2104389181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e80b4e70 0x7fe3e80ccff0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8089830 tx=0x7fe3e40091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.275+0000 7fe3ef59d700 1 -- 172.21.15.119:0/2104389181 shutdown_connections 2023-10-08T01:09:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.275+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/2104389181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e80b4e70 0x7fe3e80ccff0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.275+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/2104389181 >> v2:172.21.15.184:3300/0 conn(0x7fe3e80b16d0 0x7fe3e80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.275+0000 7fe3ef59d700 1 -- 172.21.15.119:0/2104389181 >> 172.21.15.119:0/2104389181 conn(0x7fe3e80d10c0 msgr2=0x7fe3e80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.275+0000 7fe3ef59d700 1 -- 172.21.15.119:0/2104389181 shutdown_connections 2023-10-08T01:09:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.275+0000 7fe3ef59d700 1 -- 172.21.15.119:0/2104389181 wait complete. 2023-10-08T01:09:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.276+0000 7fe3ef59d700 1 Processor -- start 2023-10-08T01:09:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.277+0000 7fe3ef59d700 1 -- start start 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.277+0000 7fe3ef59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b16d0 0x7fe3e816d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.277+0000 7fe3ef59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e80b4e70 0x7fe3e8155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.277+0000 7fe3ef59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e815d240 0x7fe3e8169cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.277+0000 7fe3ef59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80c5730 con 0x7fe3e815d240 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.277+0000 7fe3ef59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e80c5430 con 0x7fe3e80b4e70 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.277+0000 7fe3ef59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e816dac0 con 0x7fe3e80b16d0 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e815d240 0x7fe3e8169cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e815d240 0x7fe3e8169cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55016/0 (socket says 172.21.15.119:55016) 2023-10-08T01:09:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 -- 172.21.15.119:0/1716198629 learned_addr learned my addr 172.21.15.119:0/1716198629 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3ee59b700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b16d0 0x7fe3e816d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3edd9a700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e80b4e70 0x7fe3e8155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 -- 172.21.15.119:0/1716198629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b16d0 msgr2=0x7fe3e816d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b16d0 0x7fe3e816d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 -- 172.21.15.119:0/1716198629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e80b4e70 msgr2=0x7fe3e8155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e80b4e70 0x7fe3e8155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 -- 172.21.15.119:0/1716198629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e4009420 con 0x7fe3e815d240 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.278+0000 7fe3eed9c700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e815d240 0x7fe3e8169cf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000cbf0 tx=0x7fe3e00019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.279+0000 7fe3de7fc700 1 -- 172.21.15.119:0/1716198629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000a870 con 0x7fe3e815d240 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.279+0000 7fe3de7fc700 1 -- 172.21.15.119:0/1716198629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3e000a9d0 con 0x7fe3e815d240 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.279+0000 7fe3de7fc700 1 -- 172.21.15.119:0/1716198629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e000ad50 con 0x7fe3e815d240 2023-10-08T01:09:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.279+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e816dda0 con 0x7fe3e815d240 2023-10-08T01:09:05.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.279+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e81564f0 con 0x7fe3e815d240 2023-10-08T01:09:05.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.280+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e80c58e0 con 0x7fe3e815d240 2023-10-08T01:09:05.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.281+0000 7fe3de7fc700 1 -- 172.21.15.119:0/1716198629 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe3e0007300 con 0x7fe3e815d240 2023-10-08T01:09:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.281+0000 7fe3de7fc700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe3cc066f30 0x7fe3cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.282+0000 7fe3de7fc700 1 -- 172.21.15.119:0/1716198629 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5217+0+0 (secure 0 0 0) 0x7fe3e0084660 con 0x7fe3e815d240 2023-10-08T01:09:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.282+0000 7fe3ee59b700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe3cc066f30 0x7fe3cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.282+0000 7fe3ee59b700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe3cc066f30 0x7fe3cc0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40093f0 tx=0x7fe3e4017000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:05.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.286+0000 7fe3de7fc700 1 -- 172.21.15.119:0/1716198629 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe3e80c58e0 con 0x7fe3e815d240 2023-10-08T01:09:05.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.475+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe3e80c58e0 con 0x7fe3e815d240 2023-10-08T01:09:05.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.475+0000 7fe3de7fc700 1 -- 172.21.15.119:0/1716198629 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe3e80c58e0 con 0x7fe3e815d240 2023-10-08T01:09:05.477 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:09:05.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe3cc066f30 msgr2=0x7fe3cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe3cc066f30 0x7fe3cc0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40093f0 tx=0x7fe3e4017000 comp rx=0 tx=0).stop 2023-10-08T01:09:05.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e815d240 msgr2=0x7fe3e8169cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e815d240 0x7fe3e8169cf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000cbf0 tx=0x7fe3e00019c0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 shutdown_connections 2023-10-08T01:09:05.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe3cc066f30 0x7fe3cc0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3e80b16d0 0x7fe3e816d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3e815d240 0x7fe3e8169cf0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 --2- 172.21.15.119:0/1716198629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3e80b4e70 0x7fe3e8155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 >> 172.21.15.119:0/1716198629 conn(0x7fe3e80d10c0 msgr2=0x7fe3e80698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:05.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 shutdown_connections 2023-10-08T01:09:05.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.478+0000 7fe3ef59d700 1 -- 172.21.15.119:0/1716198629 wait complete. 2023-10-08T01:09:05.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T01:09:05.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.677+0000 7f1eaaf9d700 1 Processor -- start 2023-10-08T01:09:05.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.677+0000 7f1eaaf9d700 1 -- start start 2023-10-08T01:09:05.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.678+0000 7f1eaaf9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1ea4150180 0x7f1ea4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.678+0000 7f1eaaf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea40c7410 0x7f1ea4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.678+0000 7f1eaaf9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea406e2a0 con 0x7f1ea4150550 2023-10-08T01:09:05.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.678+0000 7f1eaaf9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea406b2e0 con 0x7f1ea4150180 2023-10-08T01:09:05.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.678+0000 7f1eaaf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40c78e0 con 0x7f1ea40c7410 2023-10-08T01:09:05.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.678+0000 7f1ea979a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1ea4150550 0x7f1ea40c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44406/0 (socket says 172.21.15.119:44406) 2023-10-08T01:09:05.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.678+0000 7f1ea979a700 1 -- 172.21.15.119:0/1770385212 learned_addr learned my addr 172.21.15.119:0/1770385212 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:05.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1ea9f9b700 1 --2- 172.21.15.119:0/1770385212 >> v2:172.21.15.184:3300/0 conn(0x7f1ea4150180 0x7f1ea4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1eaa79c700 1 --2- 172.21.15.119:0/1770385212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea40c7410 0x7f1ea4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1ea8f99700 1 -- 172.21.15.119:0/1770385212 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1446094002 0 0) 0x7f1ea406e2a0 con 0x7f1ea4150550 2023-10-08T01:09:05.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1ea8f99700 1 -- 172.21.15.119:0/1770385212 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e90003570 con 0x7f1ea4150550 2023-10-08T01:09:05.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1eaa79c700 1 -- 172.21.15.119:0/1770385212 >> v2:172.21.15.184:3300/0 conn(0x7f1ea4150180 msgr2=0x7f1ea4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:05.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1eaa79c700 1 --2- 172.21.15.119:0/1770385212 >> v2:172.21.15.184:3300/0 conn(0x7f1ea4150180 0x7f1ea4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1eaa79c700 1 -- 172.21.15.119:0/1770385212 >> v1:172.21.15.119:6789/0 conn(0x7f1ea4150550 legacy=0x7f1ea40c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1eaa79c700 1 -- 172.21.15.119:0/1770385212 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40c7bc0 con 0x7f1ea40c7410 2023-10-08T01:09:05.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1eaa79c700 1 --2- 172.21.15.119:0/1770385212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea40c7410 0x7f1ea4069f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009e60 tx=0x7f1e9c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daa4127128e0da12 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:05.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.679+0000 7f1ea8f99700 1 -- 172.21.15.119:0/1770385212 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c00e070 con 0x7f1ea40c7410 2023-10-08T01:09:05.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1ea8f99700 1 -- 172.21.15.119:0/1770385212 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c00bab0 con 0x7f1ea40c7410 2023-10-08T01:09:05.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1ea8f99700 1 -- 172.21.15.119:0/1770385212 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c00be70 con 0x7f1ea40c7410 2023-10-08T01:09:05.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/1770385212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea40c7410 msgr2=0x7f1ea4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/1770385212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea40c7410 0x7f1ea4069f50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009e60 tx=0x7f1e9c009780 comp rx=0 tx=0).stop 2023-10-08T01:09:05.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/1770385212 shutdown_connections 2023-10-08T01:09:05.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/1770385212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea40c7410 0x7f1ea4069f50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/1770385212 >> v2:172.21.15.184:3300/0 conn(0x7f1ea4150180 0x7f1ea4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/1770385212 >> 172.21.15.119:0/1770385212 conn(0x7f1ea40cc1f0 msgr2=0x7f1ea40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:05.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/1770385212 shutdown_connections 2023-10-08T01:09:05.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.680+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/1770385212 wait complete. 2023-10-08T01:09:05.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.682+0000 7f1eaaf9d700 1 Processor -- start 2023-10-08T01:09:05.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.682+0000 7f1eaaf9d700 1 -- start start 2023-10-08T01:09:05.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaaf9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea40c7410 0x7f1ea40ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaaf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea4150180 0x7f1ea4144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaaf9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea4150550 0x7f1ea40c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaaf9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40b0f40 con 0x7f1ea4150550 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaaf9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40afea0 con 0x7f1ea40c7410 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaaf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40b0180 con 0x7f1ea4150180 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaa79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea4150550 0x7f1ea40c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaa79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea4150550 0x7f1ea40c39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55024/0 (socket says 172.21.15.119:55024) 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaa79c700 1 -- 172.21.15.119:0/4174939498 learned_addr learned my addr 172.21.15.119:0/4174939498 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1ea9f9b700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea40c7410 0x7f1ea40ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1ea979a700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea4150180 0x7f1ea4144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaa79c700 1 -- 172.21.15.119:0/4174939498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea4150180 msgr2=0x7f1ea4144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaa79c700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea4150180 0x7f1ea4144b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaa79c700 1 -- 172.21.15.119:0/4174939498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea40c7410 msgr2=0x7f1ea40ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaa79c700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea40c7410 0x7f1ea40ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.683+0000 7f1eaa79c700 1 -- 172.21.15.119:0/4174939498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c009a20 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.684+0000 7f1eaa79c700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea4150550 0x7f1ea40c39a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ea406d1e0 tx=0x7f1e9c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.684+0000 7f1e99ffb700 1 -- 172.21.15.119:0/4174939498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c012e10 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.684+0000 7f1e99ffb700 1 -- 172.21.15.119:0/4174939498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c027470 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.684+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea40b0460 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.684+0000 7f1e99ffb700 1 -- 172.21.15.119:0/4174939498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c027830 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.684+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea40b0aa0 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.685+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea40b0c30 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.686+0000 7f1e99ffb700 1 -- 172.21.15.119:0/4174939498 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e9c024070 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.687+0000 7f1e99ffb700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f1e88066f30 0x7f1e880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.687+0000 7f1e99ffb700 1 -- 172.21.15.119:0/4174939498 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5217+0+0 (secure 0 0 0) 0x7f1e9c086960 con 0x7f1ea4150550 2023-10-08T01:09:05.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.687+0000 7f1ea9f9b700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f1e88066f30 0x7f1e880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:05.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.687+0000 7f1ea9f9b700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f1e88066f30 0x7f1e880693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009db0 tx=0x7f1ea00091a0 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:05.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.691+0000 7f1e99ffb700 1 -- 172.21.15.119:0/4174939498 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e9c053d90 con 0x7f1ea4150550 2023-10-08T01:09:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.884+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1ea404c3c0 con 0x7f1ea4150550 2023-10-08T01:09:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.884+0000 7f1e99ffb700 1 -- 172.21.15.119:0/4174939498 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v55) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1e9c02c020 con 0x7f1ea4150550 2023-10-08T01:09:05.886 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:05.886 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:09:05.886 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:05.886 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T01:09:05.886 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T01:09:05.886 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:05.886 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:09:05.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.886+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f1e88066f30 msgr2=0x7f1e880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.886+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f1e88066f30 0x7f1e880693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009db0 tx=0x7f1ea00091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.886+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea4150550 msgr2=0x7f1ea40c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:05.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.886+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea4150550 0x7f1ea40c39a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1ea406d1e0 tx=0x7f1e9c019000 comp rx=0 tx=0).stop 2023-10-08T01:09:05.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.887+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 shutdown_connections 2023-10-08T01:09:05.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.887+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f1e88066f30 0x7f1e880693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.887+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ea4150550 0x7f1ea40c39a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.887+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ea40c7410 0x7f1ea40ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.887+0000 7f1eaaf9d700 1 --2- 172.21.15.119:0/4174939498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ea4150180 0x7f1ea4144b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:05.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.887+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 >> 172.21.15.119:0/4174939498 conn(0x7f1ea40cc1f0 msgr2=0x7f1ea4158ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:05.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.887+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 shutdown_connections 2023-10-08T01:09:05.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:05.887+0000 7f1eaaf9d700 1 -- 172.21.15.119:0/4174939498 wait complete. 2023-10-08T01:09:05.905 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T01:09:05.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T01:09:06.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.089+0000 7fce8119d700 1 Processor -- start 2023-10-08T01:09:06.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.089+0000 7fce8119d700 1 -- start start 2023-10-08T01:09:06.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.090+0000 7fce8119d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce7c0cb560 0x7fce7c0cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:06.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.090+0000 7fce8119d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c0697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:06.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.090+0000 7fce8119d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c0ccde0 con 0x7fce7c069cf0 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.090+0000 7fce8119d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c06aa30 con 0x7fce7c0cb560 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.090+0000 7fce8119d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c159db0 con 0x7fce7c0cc970 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce7c0cb560 0x7fce7c0cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce7c0cb560 0x7fce7c0cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55036/0 (socket says 172.21.15.119:55036) 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7bfff700 1 -- 172.21.15.119:0/3155753288 learned_addr learned my addr 172.21.15.119:0/3155753288 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce73fff700 1 --2- 172.21.15.119:0/3155753288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c0697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7bfff700 1 -- 172.21.15.119:0/3155753288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 msgr2=0x7fce7c0697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7bfff700 1 --2- 172.21.15.119:0/3155753288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c0697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7bfff700 1 -- 172.21.15.119:0/3155753288 >> v1:172.21.15.184:6789/0 conn(0x7fce7c069cf0 legacy=0x7fce7c06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:06.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7bfff700 1 -- 172.21.15.119:0/3155753288 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c06a470 con 0x7fce7c0cb560 2023-10-08T01:09:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7bfff700 1 --2- 172.21.15.119:0/3155753288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c0cc430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099b0 tx=0x7fce6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ae8be1ec097c9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.091+0000 7fce7b7fe700 1 -- 172.21.15.119:0/3155753288 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c00b820 con 0x7fce7c0cb560 2023-10-08T01:09:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce7b7fe700 1 -- 172.21.15.119:0/3155753288 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c00b980 con 0x7fce7c0cb560 2023-10-08T01:09:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce7b7fe700 1 -- 172.21.15.119:0/3155753288 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c00bd20 con 0x7fce7c0cb560 2023-10-08T01:09:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce8119d700 1 -- 172.21.15.119:0/3155753288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 msgr2=0x7fce7c0cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce8119d700 1 --2- 172.21.15.119:0/3155753288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c0cc430 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099b0 tx=0x7fce6c009250 comp rx=0 tx=0).stop 2023-10-08T01:09:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce8119d700 1 -- 172.21.15.119:0/3155753288 shutdown_connections 2023-10-08T01:09:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce8119d700 1 --2- 172.21.15.119:0/3155753288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c0cc430 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce8119d700 1 --2- 172.21.15.119:0/3155753288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c0697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce8119d700 1 -- 172.21.15.119:0/3155753288 >> 172.21.15.119:0/3155753288 conn(0x7fce7c06baf0 msgr2=0x7fce7c06bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce8119d700 1 -- 172.21.15.119:0/3155753288 shutdown_connections 2023-10-08T01:09:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.092+0000 7fce8119d700 1 -- 172.21.15.119:0/3155753288 wait complete. 2023-10-08T01:09:06.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.094+0000 7fce8119d700 1 Processor -- start 2023-10-08T01:09:06.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.094+0000 7fce8119d700 1 -- start start 2023-10-08T01:09:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce8119d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce7c069cf0 0x7fce7c1480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce8119d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c15b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce8119d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c1499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce8119d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c149510 con 0x7fce7c0cb560 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce8119d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c1699a0 con 0x7fce7c069cf0 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce8119d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c148900 con 0x7fce7c0cc970 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c15b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c15b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55050/0 (socket says 172.21.15.119:55050) 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 -- 172.21.15.119:0/3512460634 learned_addr learned my addr 172.21.15.119:0/3512460634 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce7bfff700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce7c069cf0 0x7fce7c1480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:06.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce8099c700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c1499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 -- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 msgr2=0x7fce7c1499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c1499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 -- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce7c069cf0 msgr2=0x7fce7c1480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce7c069cf0 0x7fce7c1480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 -- 172.21.15.119:0/3512460634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0094f0 con 0x7fce7c0cb560 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce8099c700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c1499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.095+0000 7fce73fff700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c15b960 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d890 tx=0x7fce64008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.096+0000 7fce78ff9700 1 -- 172.21.15.119:0/3512460634 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64009610 con 0x7fce7c0cb560 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.096+0000 7fce78ff9700 1 -- 172.21.15.119:0/3512460634 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64009770 con 0x7fce7c0cb560 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.096+0000 7fce78ff9700 1 -- 172.21.15.119:0/3512460634 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64009b30 con 0x7fce7c0cb560 2023-10-08T01:09:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.096+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c148be0 con 0x7fce7c0cb560 2023-10-08T01:09:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.096+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c1491a0 con 0x7fce7c0cb560 2023-10-08T01:09:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.097+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c04c3c0 con 0x7fce7c0cb560 2023-10-08T01:09:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.098+0000 7fce78ff9700 1 -- 172.21.15.119:0/3512460634 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fce6401e460 con 0x7fce7c0cb560 2023-10-08T01:09:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.099+0000 7fce78ff9700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fce5c066f30 0x7fce5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.099+0000 7fce78ff9700 1 -- 172.21.15.119:0/3512460634 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5217+0+0 (secure 0 0 0) 0x7fce64084a60 con 0x7fce7c0cb560 2023-10-08T01:09:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.099+0000 7fce7bfff700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fce5c066f30 0x7fce5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.100+0000 7fce7bfff700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fce5c066f30 0x7fce5c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099b0 tx=0x7fce6c016000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:06.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.103+0000 7fce78ff9700 1 -- 172.21.15.119:0/3512460634 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fce64051e10 con 0x7fce7c0cb560 2023-10-08T01:09:06.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:06.294+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fce7c06a750 con 0x7fce7c0cb560 2023-10-08T01:09:07.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.295+0000 7fce78ff9700 1 -- 172.21.15.119:0/3512460634 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v56) v1 ==== 138+0+0 (secure 0 0 0) 0x7fce64055430 con 0x7fce7c0cb560 2023-10-08T01:09:07.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fce5c066f30 msgr2=0x7fce5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:07.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fce5c066f30 0x7fce5c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0099b0 tx=0x7fce6c016000 comp rx=0 tx=0).stop 2023-10-08T01:09:07.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 msgr2=0x7fce7c15b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:07.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c15b960 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fce6400d890 tx=0x7fce64008040 comp rx=0 tx=0).stop 2023-10-08T01:09:07.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 shutdown_connections 2023-10-08T01:09:07.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fce5c066f30 0x7fce5c0693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fce7c0cc970 0x7fce7c1499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fce7c069cf0 0x7fce7c1480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 --2- 172.21.15.119:0/3512460634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce7c0cb560 0x7fce7c15b960 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.300+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 >> 172.21.15.119:0/3512460634 conn(0x7fce7c06baf0 msgr2=0x7fce7c0c33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:07.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.301+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 shutdown_connections 2023-10-08T01:09:07.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.301+0000 7fce8119d700 1 -- 172.21.15.119:0/3512460634 wait complete. 2023-10-08T01:09:07.305 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T01:09:07.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T01:09:07.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.512+0000 7ff51719d700 1 Processor -- start 2023-10-08T01:09:07.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.512+0000 7ff51719d700 1 -- start start 2023-10-08T01:09:07.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.513+0000 7ff51719d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff51006d840 0x7ff5100c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:07.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.513+0000 7ff51719d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100c9860 con 0x7ff51006af90 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.513+0000 7ff51719d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5100c8cd0 con 0x7ff51006b360 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.513+0000 7ff51719d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100cd200 con 0x7ff51006d840 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff51619b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff51006af90 0x7ff51006d130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44414/0 (socket says 172.21.15.119:44414) 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff51619b700 1 -- 172.21.15.119:0/830861283 learned_addr learned my addr 172.21.15.119:0/830861283 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff51599a700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff51006d840 0x7ff5100c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277014414 0 0) 0x7ff5100c9860 con 0x7ff51006af90 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4fc003570 con 0x7ff51006af90 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335676185 0 0) 0x7ff5100c8cd0 con 0x7ff51006b360 2023-10-08T01:09:07.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5100c9860 con 0x7ff51006b360 2023-10-08T01:09:07.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1357258679 0 0) 0x7ff4fc003570 con 0x7ff51006af90 2023-10-08T01:09:07.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff51006d840 msgr2=0x7ff5100c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:07.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.514+0000 7ff515199700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff51006d840 0x7ff5100c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 >> v1:172.21.15.184:6789/0 conn(0x7ff51006b360 legacy=0x7ff5100ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:07.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100c8f30 con 0x7ff51006af90 2023-10-08T01:09:07.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7ff50c001470 con 0x7ff51006af90 2023-10-08T01:09:07.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 >> v1:172.21.15.119:6789/0 conn(0x7ff51006af90 legacy=0x7ff51006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:07.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff51006d840 0x7ff4fc003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff4fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4fc004d50 0x7ff4fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100cd200 con 0x7ff4fc0041d0 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc00bbf0 con 0x7ff4fc004d50 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4fc00bdb0 con 0x7ff51006d840 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff51599a700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff4fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff51619b700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff51006d840 0x7ff4fc003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff51699c700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4fc004d50 0x7ff4fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff51599a700 1 -- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff51006d840 msgr2=0x7ff4fc003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff51599a700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff51006d840 0x7ff4fc003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff51599a700 1 -- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4fc004d50 msgr2=0x7ff4fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff51599a700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4fc004d50 0x7ff4fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.515+0000 7ff51599a700 1 -- 172.21.15.119:0/830861283 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4fc003570 con 0x7ff4fc0041d0 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff51599a700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff4fc004810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff500009c00 tx=0x7ff5000091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89fba199a6aa908 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50000e710 con 0x7ff4fc0041d0 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50000e870 con 0x7ff4fc0041d0 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff515199700 1 -- 172.21.15.119:0/830861283 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50000ec30 con 0x7ff4fc0041d0 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff51719d700 1 -- 172.21.15.119:0/830861283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 msgr2=0x7ff4fc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff51719d700 1 --2- 172.21.15.119:0/830861283 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff4fc004810 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff500009c00 tx=0x7ff5000091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff51699c700 1 -- 172.21.15.119:0/830861283 reap_dead start 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff51719d700 1 -- 172.21.15.119:0/830861283 shutdown_connections 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff51719d700 1 -- 172.21.15.119:0/830861283 >> 172.21.15.119:0/830861283 conn(0x7ff51006baa0 msgr2=0x7ff51006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:07.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff51719d700 1 -- 172.21.15.119:0/830861283 shutdown_connections 2023-10-08T01:09:07.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.516+0000 7ff51719d700 1 -- 172.21.15.119:0/830861283 wait complete. 2023-10-08T01:09:07.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.518+0000 7ff51719d700 1 Processor -- start 2023-10-08T01:09:07.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.518+0000 7ff51719d700 1 -- start start 2023-10-08T01:09:07.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51719d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff5100b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:07.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51719d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5100c87d0 0x7ff5100b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:07.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51719d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5100b0700 0x7ff51016d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:07.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51719d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100b0ba0 con 0x7ff4fc0041d0 2023-10-08T01:09:07.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51719d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100b3f40 con 0x7ff5100b0700 2023-10-08T01:09:07.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51719d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51016afc0 con 0x7ff5100c87d0 2023-10-08T01:09:07.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff5100b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:07.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51619b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff5100b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55064/0 (socket says 172.21.15.119:55064) 2023-10-08T01:09:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51619b700 1 -- 172.21.15.119:0/1471001311 learned_addr learned my addr 172.21.15.119:0/1471001311 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51619b700 1 -- 172.21.15.119:0/1471001311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5100c87d0 msgr2=0x7ff5100b01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51619b700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5100c87d0 0x7ff5100b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51699c700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5100b0700 0x7ff51016d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51619b700 1 -- 172.21.15.119:0/1471001311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5100b0700 msgr2=0x7ff51016d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:07.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51619b700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5100b0700 0x7ff51016d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51619b700 1 -- 172.21.15.119:0/1471001311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5100c8c40 con 0x7ff4fc0041d0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.519+0000 7ff51699c700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5100b0700 0x7ff51016d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.520+0000 7ff51619b700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff5100b2f90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00e820 tx=0x7ff5100c97d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.520+0000 7ff5067fc700 1 -- 172.21.15.119:0/1471001311 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c001b80 con 0x7ff4fc0041d0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.520+0000 7ff5067fc700 1 -- 172.21.15.119:0/1471001311 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c00c5a0 con 0x7ff4fc0041d0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.520+0000 7ff5067fc700 1 -- 172.21.15.119:0/1471001311 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c00c960 con 0x7ff4fc0041d0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.520+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5100b36a0 con 0x7ff4fc0041d0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.520+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5100b3c60 con 0x7ff4fc0041d0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.521+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51004c3c0 con 0x7ff4fc0041d0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.522+0000 7ff5067fc700 1 -- 172.21.15.119:0/1471001311 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff50c00ccd0 con 0x7ff4fc0041d0 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.523+0000 7ff5067fc700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff4f4066f30 0x7ff4f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:07.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.523+0000 7ff5067fc700 1 -- 172.21.15.119:0/1471001311 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5587+0+0 (secure 0 0 0) 0x7ff50c084440 con 0x7ff4fc0041d0 2023-10-08T01:09:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.523+0000 7ff51599a700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff4f4066f30 0x7ff4f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.524+0000 7ff51599a700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff4f4066f30 0x7ff4f40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5100cb770 tx=0x7ff50001a000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:07.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.528+0000 7ff5067fc700 1 -- 172.21.15.119:0/1471001311 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff50c051680 con 0x7ff4fc0041d0 2023-10-08T01:09:07.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:07.720+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff5100c9210 con 0x7ff4fc0041d0 2023-10-08T01:09:08.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.297+0000 7ff5067fc700 1 -- 172.21.15.119:0/1471001311 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v57) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff50c054ca0 con 0x7ff4fc0041d0 2023-10-08T01:09:08.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.309+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff4f4066f30 msgr2=0x7ff4f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:08.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.309+0000 7ff51719d700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff4f4066f30 0x7ff4f40693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff5100cb770 tx=0x7ff50001a000 comp rx=0 tx=0).stop 2023-10-08T01:09:08.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.309+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 msgr2=0x7ff5100b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:08.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.309+0000 7ff51719d700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff5100b2f90 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00e820 tx=0x7ff5100c97d0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.310+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 shutdown_connections 2023-10-08T01:09:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.310+0000 7ff51719d700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff4f4066f30 0x7ff4f40693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.310+0000 7ff51719d700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5100b0700 0x7ff51016d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.310+0000 7ff51719d700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff4fc0041d0 0x7ff5100b2f90 secure :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff50c00e820 tx=0x7ff5100c97d0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.310+0000 7ff51719d700 1 --2- 172.21.15.119:0/1471001311 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5100c87d0 0x7ff5100b01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.310+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 >> 172.21.15.119:0/1471001311 conn(0x7ff51006baa0 msgr2=0x7ff5100c34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.310+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 shutdown_connections 2023-10-08T01:09:08.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.311+0000 7ff51719d700 1 -- 172.21.15.119:0/1471001311 wait complete. 2023-10-08T01:09:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T01:09:08.337 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T01:09:08.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.521+0000 7fd93bfff700 1 Processor -- start 2023-10-08T01:09:08.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.522+0000 7fd93bfff700 1 -- start start 2023-10-08T01:09:08.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.523+0000 7fd93bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd9340c83f0 0x7fd9340c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:08.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.523+0000 7fd93bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9341568e0 0x7fd9340ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.523+0000 7fd93bfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93415b0d0 con 0x7fd9340ccee0 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.523+0000 7fd93bfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9340c7a60 con 0x7fd9340c83f0 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.523+0000 7fd93bfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93415a4c0 con 0x7fd9341568e0 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.523+0000 7fd939d9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd9340c83f0 0x7fd9340c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.523+0000 7fd93a59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9341568e0 0x7fd9340ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd939d9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd9340c83f0 0x7fd9340c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:52746/0 (socket says 172.21.15.119:52746) 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd939d9b700 1 -- 172.21.15.119:0/2443339567 learned_addr learned my addr 172.21.15.119:0/2443339567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd939d9b700 1 -- 172.21.15.119:0/2443339567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9341568e0 msgr2=0x7fd9340ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd939d9b700 1 --2- 172.21.15.119:0/2443339567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9341568e0 0x7fd9340ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd939d9b700 1 -- 172.21.15.119:0/2443339567 >> v1:172.21.15.184:6789/0 conn(0x7fd9340ccee0 legacy=0x7fd9341561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd939d9b700 1 -- 172.21.15.119:0/2443339567 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93415a7a0 con 0x7fd9340c83f0 2023-10-08T01:09:08.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd93a59c700 1 --2- 172.21.15.119:0/2443339567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9341568e0 0x7fd9340ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd939d9b700 1 --2- 172.21.15.119:0/2443339567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9340c83f0 0x7fd9340c7520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd93000bc50 tx=0x7fd93000b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aca8bd0b41860479 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.524+0000 7fd938d99700 1 -- 172.21.15.119:0/2443339567 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93000dbf0 con 0x7fd9340c83f0 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd938d99700 1 -- 172.21.15.119:0/2443339567 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93000dd50 con 0x7fd9340c83f0 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd938d99700 1 -- 172.21.15.119:0/2443339567 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9300196b0 con 0x7fd9340c83f0 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd93bfff700 1 -- 172.21.15.119:0/2443339567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9340c83f0 msgr2=0x7fd9340c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2443339567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9340c83f0 0x7fd9340c7520 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd93000bc50 tx=0x7fd93000b4f0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd93bfff700 1 -- 172.21.15.119:0/2443339567 shutdown_connections 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2443339567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9341568e0 0x7fd9340ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2443339567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9340c83f0 0x7fd9340c7520 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd93bfff700 1 -- 172.21.15.119:0/2443339567 >> 172.21.15.119:0/2443339567 conn(0x7fd934068e10 msgr2=0x7fd9340cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd93bfff700 1 -- 172.21.15.119:0/2443339567 shutdown_connections 2023-10-08T01:09:08.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.525+0000 7fd93bfff700 1 -- 172.21.15.119:0/2443339567 wait complete. 2023-10-08T01:09:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.527+0000 7fd93bfff700 1 Processor -- start 2023-10-08T01:09:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.527+0000 7fd93bfff700 1 -- start start 2023-10-08T01:09:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9340c83f0 0x7fd93414c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9340ccee0 0x7fd9340c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9341568e0 0x7fd9340c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93bfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93416b110 con 0x7fd9340ccee0 2023-10-08T01:09:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93bfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93414cd40 con 0x7fd9341568e0 2023-10-08T01:09:08.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93bfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93414b7a0 con 0x7fd9340c83f0 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93959a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9340ccee0 0x7fd9340c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93959a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9340ccee0 0x7fd9340c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55078/0 (socket says 172.21.15.119:55078) 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93959a700 1 -- 172.21.15.119:0/2526371629 learned_addr learned my addr 172.21.15.119:0/2526371629 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93a59c700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9341568e0 0x7fd9340c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd939d9b700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9340c83f0 0x7fd93414c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93959a700 1 -- 172.21.15.119:0/2526371629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9340c83f0 msgr2=0x7fd93414c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93959a700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9340c83f0 0x7fd93414c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93959a700 1 -- 172.21.15.119:0/2526371629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9341568e0 msgr2=0x7fd9340c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93959a700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9341568e0 0x7fd9340c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.528+0000 7fd93959a700 1 -- 172.21.15.119:0/2526371629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c009a20 con 0x7fd9340ccee0 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.529+0000 7fd93959a700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9340ccee0 0x7fd9340c5cb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c590 tx=0x7fd92400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:08.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.529+0000 7fd929ffb700 1 -- 172.21.15.119:0/2526371629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924011590 con 0x7fd9340ccee0 2023-10-08T01:09:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.529+0000 7fd929ffb700 1 -- 172.21.15.119:0/2526371629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd924011710 con 0x7fd9340ccee0 2023-10-08T01:09:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.529+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd93000b790 con 0x7fd9340ccee0 2023-10-08T01:09:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.529+0000 7fd929ffb700 1 -- 172.21.15.119:0/2526371629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924011ad0 con 0x7fd9340ccee0 2023-10-08T01:09:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.529+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd93414bdd0 con 0x7fd9340ccee0 2023-10-08T01:09:08.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.530+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd93414bbe0 con 0x7fd9340ccee0 2023-10-08T01:09:08.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.531+0000 7fd929ffb700 1 -- 172.21.15.119:0/2526371629 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd92401c440 con 0x7fd9340ccee0 2023-10-08T01:09:08.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.532+0000 7fd929ffb700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd918066f30 0x7fd9180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:08.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.532+0000 7fd929ffb700 1 -- 172.21.15.119:0/2526371629 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5969+0+0 (secure 0 0 0) 0x7fd924083b40 con 0x7fd9340ccee0 2023-10-08T01:09:08.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.532+0000 7fd939d9b700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd918066f30 0x7fd9180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:08.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.533+0000 7fd939d9b700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd918066f30 0x7fd9180693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd93000bc50 tx=0x7fd93001b000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:08.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.537+0000 7fd929ffb700 1 -- 172.21.15.119:0/2526371629 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd93414bbe0 con 0x7fd9340ccee0 2023-10-08T01:09:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:08.769+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd9340c68e0 con 0x7fd9340ccee0 2023-10-08T01:09:09.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.317+0000 7fd929ffb700 1 -- 172.21.15.119:0/2526371629 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd924017070 con 0x7fd9340ccee0 2023-10-08T01:09:09.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.321+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd918066f30 msgr2=0x7fd9180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.321+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd918066f30 0x7fd9180693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd93000bc50 tx=0x7fd93001b000 comp rx=0 tx=0).stop 2023-10-08T01:09:09.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.321+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9340ccee0 msgr2=0x7fd9340c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.321+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9340ccee0 0x7fd9340c5cb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd92400c590 tx=0x7fd92400c8a0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.327+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 shutdown_connections 2023-10-08T01:09:09.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.327+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd918066f30 0x7fd9180693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.327+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9340c83f0 0x7fd93414c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.327+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9341568e0 0x7fd9340c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.327+0000 7fd93bfff700 1 --2- 172.21.15.119:0/2526371629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9340ccee0 0x7fd9340c5cb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.327+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 >> 172.21.15.119:0/2526371629 conn(0x7fd934068e10 msgr2=0x7fd9340c2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:09.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.329+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 shutdown_connections 2023-10-08T01:09:09.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.331+0000 7fd93bfff700 1 -- 172.21.15.119:0/2526371629 wait complete. 2023-10-08T01:09:09.336 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 6 and data pool 7 2023-10-08T01:09:09.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:09:09.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.537+0000 7f683c9ff700 1 Processor -- start 2023-10-08T01:09:09.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.537+0000 7f683c9ff700 1 -- start start 2023-10-08T01:09:09.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.538+0000 7f683c9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f68380b1420 0x7f68380ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.538+0000 7f683c9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68380ccf90 0x7f68380c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.538+0000 7f683c9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380cc300 con 0x7f68380cbf30 2023-10-08T01:09:09.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.538+0000 7f683c9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6838156bf0 con 0x7f68380b1420 2023-10-08T01:09:09.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.538+0000 7f683c9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380ae950 con 0x7f68380ccf90 2023-10-08T01:09:09.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.538+0000 7f683659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f68380cbf30 0x7f68380b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44430/0 (socket says 172.21.15.119:44430) 2023-10-08T01:09:09.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.538+0000 7f683659c700 1 -- 172.21.15.119:0/735180356 learned_addr learned my addr 172.21.15.119:0/735180356 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:09.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f6836d9d700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68380ccf90 0x7f68380c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f6835d9b700 1 --2- 172.21.15.119:0/735180356 >> v2:172.21.15.184:3300/0 conn(0x7f68380b1420 0x7f68380ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683718340 0 0) 0x7f68380cc300 con 0x7f68380cbf30 2023-10-08T01:09:09.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6820003570 con 0x7f68380cbf30 2023-10-08T01:09:09.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533626067 0 0) 0x7f6820003570 con 0x7f68380cbf30 2023-10-08T01:09:09.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68380ccf90 msgr2=0x7f68380c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:09.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68380ccf90 0x7f68380c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 >> v2:172.21.15.184:3300/0 conn(0x7f68380b1420 msgr2=0x7f68380ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:09.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 --2- 172.21.15.119:0/735180356 >> v2:172.21.15.184:3300/0 conn(0x7f68380b1420 0x7f68380ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380ad370 con 0x7f68380cbf30 2023-10-08T01:09:09.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f6828001470 con 0x7f68380cbf30 2023-10-08T01:09:09.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 >> v1:172.21.15.119:6789/0 conn(0x7f68380cbf30 legacy=0x7f68380b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68380ccf90 0x7f6820003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68200040e0 0x7f6820004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6820004d50 0x7f68200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.539+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6838156bf0 con 0x7f6820004d50 2023-10-08T01:09:09.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380ae950 con 0x7f68200040e0 2023-10-08T01:09:09.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f682000bb90 con 0x7f68380ccf90 2023-10-08T01:09:09.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f6836d9d700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6820004d50 0x7f68200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f683659c700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68380ccf90 0x7f6820003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f6835d9b700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68200040e0 0x7f6820004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f6836d9d700 1 -- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68380ccf90 msgr2=0x7f6820003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f6836d9d700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68380ccf90 0x7f6820003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f6836d9d700 1 -- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68200040e0 msgr2=0x7f6820004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f6836d9d700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68200040e0 0x7f6820004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f6836d9d700 1 -- 172.21.15.119:0/735180356 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6820003570 con 0x7f6820004d50 2023-10-08T01:09:09.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.540+0000 7f6836d9d700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6820004d50 0x7f68200051c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f68300094d0 tx=0x7f683000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed815ec9db11c66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683000ea90 con 0x7f6820004d50 2023-10-08T01:09:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683000f040 con 0x7f6820004d50 2023-10-08T01:09:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683559a700 1 -- 172.21.15.119:0/735180356 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683000ee20 con 0x7f6820004d50 2023-10-08T01:09:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683c9ff700 1 -- 172.21.15.119:0/735180356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6820004d50 msgr2=0x7f68200051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683c9ff700 1 --2- 172.21.15.119:0/735180356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6820004d50 0x7f68200051c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f68300094d0 tx=0x7f683000b040 comp rx=0 tx=0).stop 2023-10-08T01:09:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f6836d9d700 1 -- 172.21.15.119:0/735180356 reap_dead start 2023-10-08T01:09:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683c9ff700 1 -- 172.21.15.119:0/735180356 shutdown_connections 2023-10-08T01:09:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683c9ff700 1 -- 172.21.15.119:0/735180356 >> 172.21.15.119:0/735180356 conn(0x7f683806a320 msgr2=0x7f68380cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683c9ff700 1 -- 172.21.15.119:0/735180356 shutdown_connections 2023-10-08T01:09:09.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.541+0000 7f683c9ff700 1 -- 172.21.15.119:0/735180356 wait complete. 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.543+0000 7f683c9ff700 1 Processor -- start 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.543+0000 7f683c9ff700 1 -- start start 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f683c9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6820004d50 0x7f683814a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f683c9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6838149f30 0x7f68380c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f683c9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68380c3d10 0x7f68380aa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f683c9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c4180 con 0x7f68380c3d10 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f683c9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68380c2110 con 0x7f6820004d50 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f683c9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f683814ada0 con 0x7f6838149f30 2023-10-08T01:09:09.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68380c3d10 0x7f68380aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68380c3d10 0x7f68380aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55096/0 (socket says 172.21.15.119:55096) 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 -- 172.21.15.119:0/3076827381 learned_addr learned my addr 172.21.15.119:0/3076827381 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 -- 172.21.15.119:0/3076827381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6838149f30 msgr2=0x7f68380c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6838149f30 0x7f68380c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 -- 172.21.15.119:0/3076827381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6820004d50 msgr2=0x7f683814a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6820004d50 0x7f683814a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 -- 172.21.15.119:0/3076827381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380c1500 con 0x7f68380c3d10 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.544+0000 7f6836d9d700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68380c3d10 0x7f68380aa980 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6820004790 tx=0x7f683001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.545+0000 7f68267fc700 1 -- 172.21.15.119:0/3076827381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683001e640 con 0x7f68380c3d10 2023-10-08T01:09:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.545+0000 7f68267fc700 1 -- 172.21.15.119:0/3076827381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683001e7a0 con 0x7f68380c3d10 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.545+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68380c17e0 con 0x7f68380c3d10 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.545+0000 7f68267fc700 1 -- 172.21.15.119:0/3076827381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683001eba0 con 0x7f68380c3d10 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.545+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68380c1da0 con 0x7f68380c3d10 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.546+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683804c3c0 con 0x7f68380c3d10 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.546+0000 7f68267fc700 1 -- 172.21.15.119:0/3076827381 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6830003040 con 0x7f68380c3d10 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.547+0000 7f68267fc700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6818066f30 0x7f68180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.547+0000 7f68267fc700 1 -- 172.21.15.119:0/3076827381 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f68300868a0 con 0x7f68380c3d10 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.548+0000 7f683659c700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6818066f30 0x7f68180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.548+0000 7f683659c700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6818066f30 0x7f68180693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6828001e80 tx=0x7f68380c8fd0 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.552+0000 7f68267fc700 1 -- 172.21.15.119:0/3076827381 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6830053900 con 0x7f68380c3d10 2023-10-08T01:09:09.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.744+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68380ad650 con 0x7f68380c3d10 2023-10-08T01:09:09.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.745+0000 7f68267fc700 1 -- 172.21.15.119:0/3076827381 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+15964 (secure 0 0 0) 0x7f6830056f20 con 0x7f68380c3d10 2023-10-08T01:09:09.746 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:09.746 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":58,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:09:08.771001+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:09:06.334038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T01:09:07.759893+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.747+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6818066f30 msgr2=0x7f68180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.747+0000 7f683c9ff700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6818066f30 0x7f68180693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6828001e80 tx=0x7f68380c8fd0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.747+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68380c3d10 msgr2=0x7f68380aa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.747+0000 7f683c9ff700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68380c3d10 0x7f68380aa980 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f6820004790 tx=0x7f683001a000 comp rx=0 tx=0).stop 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.748+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 shutdown_connections 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.748+0000 7f683c9ff700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6818066f30 0x7f68180693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.748+0000 7f683c9ff700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68380c3d10 0x7f68380aa980 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.748+0000 7f683c9ff700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6820004d50 0x7f683814a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.748+0000 7f683c9ff700 1 --2- 172.21.15.119:0/3076827381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6838149f30 0x7f68380c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.748+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 >> 172.21.15.119:0/3076827381 conn(0x7f683806a320 msgr2=0x7f68380ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:09.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.748+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 shutdown_connections 2023-10-08T01:09:09.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.748+0000 7f683c9ff700 1 -- 172.21.15.119:0/3076827381 wait complete. 2023-10-08T01:09:09.766 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:09:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.946+0000 7fc913dff700 1 Processor -- start 2023-10-08T01:09:09.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.946+0000 7fc913dff700 1 -- start start 2023-10-08T01:09:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.947+0000 7fc913dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc90c155c10 0x7fc90c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.947+0000 7fc913dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc90c0ac230 0x7fc90c0ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.947+0000 7fc913dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc90c0ad930 con 0x7fc90c1568a0 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.947+0000 7fc913dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90c0acd20 con 0x7fc90c155c10 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.947+0000 7fc913dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc90c0ad000 con 0x7fc90c0ac230 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc911b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc90c155c10 0x7fc90c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc911b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc90c155c10 0x7fc90c155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55108/0 (socket says 172.21.15.119:55108) 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc911b9b700 1 -- 172.21.15.119:0/4114671679 learned_addr learned my addr 172.21.15.119:0/4114671679 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc91139a700 1 --2- 172.21.15.119:0/4114671679 >> v2:172.21.15.184:3300/0 conn(0x7fc90c0ac230 0x7fc90c0ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc911b9b700 1 -- 172.21.15.119:0/4114671679 >> v2:172.21.15.184:3300/0 conn(0x7fc90c0ac230 msgr2=0x7fc90c0ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc911b9b700 1 --2- 172.21.15.119:0/4114671679 >> v2:172.21.15.184:3300/0 conn(0x7fc90c0ac230 0x7fc90c0ac6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc911b9b700 1 -- 172.21.15.119:0/4114671679 >> v1:172.21.15.184:6789/0 conn(0x7fc90c1568a0 legacy=0x7fc90c156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc911b9b700 1 -- 172.21.15.119:0/4114671679 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0ad2e0 con 0x7fc90c155c10 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc91139a700 1 --2- 172.21.15.119:0/4114671679 >> v2:172.21.15.184:3300/0 conn(0x7fc90c0ac230 0x7fc90c0ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc911b9b700 1 --2- 172.21.15.119:0/4114671679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c155c10 0x7fc90c155fe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc90c065bb0 tx=0x7fc908009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a51db513ee5dfc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.948+0000 7fc910b99700 1 -- 172.21.15.119:0/4114671679 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90800b860 con 0x7fc90c155c10 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc910b99700 1 -- 172.21.15.119:0/4114671679 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90800b9c0 con 0x7fc90c155c10 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc910b99700 1 -- 172.21.15.119:0/4114671679 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90800bd60 con 0x7fc90c155c10 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc913dff700 1 -- 172.21.15.119:0/4114671679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c155c10 msgr2=0x7fc90c155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc913dff700 1 --2- 172.21.15.119:0/4114671679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c155c10 0x7fc90c155fe0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc90c065bb0 tx=0x7fc908009250 comp rx=0 tx=0).stop 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc913dff700 1 -- 172.21.15.119:0/4114671679 shutdown_connections 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc913dff700 1 --2- 172.21.15.119:0/4114671679 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c155c10 0x7fc90c155fe0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc913dff700 1 --2- 172.21.15.119:0/4114671679 >> v2:172.21.15.184:3300/0 conn(0x7fc90c0ac230 0x7fc90c0ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc913dff700 1 -- 172.21.15.119:0/4114671679 >> 172.21.15.119:0/4114671679 conn(0x7fc90c06a320 msgr2=0x7fc90c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc913dff700 1 -- 172.21.15.119:0/4114671679 shutdown_connections 2023-10-08T01:09:09.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.949+0000 7fc913dff700 1 -- 172.21.15.119:0/4114671679 wait complete. 2023-10-08T01:09:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.951+0000 7fc913dff700 1 Processor -- start 2023-10-08T01:09:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.951+0000 7fc913dff700 1 -- start start 2023-10-08T01:09:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc913dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc90c0ac230 0x7fc90c1691d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc913dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c155c10 0x7fc90c152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc913dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c1568a0 0x7fc90c0b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc913dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0a54c0 con 0x7fc90c1568a0 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc913dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0a57a0 con 0x7fc90c155c10 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc913dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc90c0a5a80 con 0x7fc90c0ac230 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91239c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c1568a0 0x7fc90c0b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91239c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c1568a0 0x7fc90c0b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55112/0 (socket says 172.21.15.119:55112) 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91239c700 1 -- 172.21.15.119:0/1535777325 learned_addr learned my addr 172.21.15.119:0/1535777325 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc911b9b700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc90c0ac230 0x7fc90c1691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91139a700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c155c10 0x7fc90c152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91239c700 1 -- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc90c0ac230 msgr2=0x7fc90c1691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91239c700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc90c0ac230 0x7fc90c1691d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91239c700 1 -- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c155c10 msgr2=0x7fc90c152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91239c700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c155c10 0x7fc90c152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.952+0000 7fc91239c700 1 -- 172.21.15.119:0/1535777325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9080094f0 con 0x7fc90c1568a0 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.953+0000 7fc911b9b700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc90c0ac230 0x7fc90c1691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.953+0000 7fc91139a700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c155c10 0x7fc90c152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.953+0000 7fc91239c700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c1568a0 0x7fc90c0b02a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc90400c840 tx=0x7fc904001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.953+0000 7fc901ffb700 1 -- 172.21.15.119:0/1535777325 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904012630 con 0x7fc90c1568a0 2023-10-08T01:09:09.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.953+0000 7fc901ffb700 1 -- 172.21.15.119:0/1535777325 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9040127b0 con 0x7fc90c1568a0 2023-10-08T01:09:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.953+0000 7fc901ffb700 1 -- 172.21.15.119:0/1535777325 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904012b70 con 0x7fc90c1568a0 2023-10-08T01:09:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.953+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c0a5d60 con 0x7fc90c1568a0 2023-10-08T01:09:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.953+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c152ac0 con 0x7fc90c1568a0 2023-10-08T01:09:09.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.954+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90c04c3c0 con 0x7fc90c1568a0 2023-10-08T01:09:09.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.955+0000 7fc901ffb700 1 -- 172.21.15.119:0/1535777325 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc904003030 con 0x7fc90c1568a0 2023-10-08T01:09:09.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.956+0000 7fc901ffb700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fc8f0066f30 0x7fc8f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:09.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.956+0000 7fc901ffb700 1 -- 172.21.15.119:0/1535777325 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fc904084700 con 0x7fc90c1568a0 2023-10-08T01:09:09.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.956+0000 7fc911b9b700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fc8f0066f30 0x7fc8f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:09.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.957+0000 7fc911b9b700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fc8f0066f30 0x7fc8f00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc90c065bb0 tx=0x7fc908017000 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:09.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:09.960+0000 7fc901ffb700 1 -- 172.21.15.119:0/1535777325 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc904051760 con 0x7fc90c1568a0 2023-10-08T01:09:10.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.150+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc90c0ad5c0 con 0x7fc90c1568a0 2023-10-08T01:09:10.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.151+0000 7fc901ffb700 1 -- 172.21.15.119:0/1535777325 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+15964 (secure 0 0 0) 0x7fc904054d80 con 0x7fc90c1568a0 2023-10-08T01:09:10.152 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:10.153 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":58,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:09:08.771001+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:09:06.334038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T01:09:07.759893+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:09:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.153+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fc8f0066f30 msgr2=0x7fc8f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:10.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.153+0000 7fc913dff700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fc8f0066f30 0x7fc8f00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc90c065bb0 tx=0x7fc908017000 comp rx=0 tx=0).stop 2023-10-08T01:09:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.153+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c1568a0 msgr2=0x7fc90c0b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.153+0000 7fc913dff700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c1568a0 0x7fc90c0b02a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc90400c840 tx=0x7fc904001670 comp rx=0 tx=0).stop 2023-10-08T01:09:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.154+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 shutdown_connections 2023-10-08T01:09:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.154+0000 7fc913dff700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fc8f0066f30 0x7fc8f00693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.154+0000 7fc913dff700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc90c0ac230 0x7fc90c1691d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.154+0000 7fc913dff700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc90c1568a0 0x7fc90c0b02a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.154+0000 7fc913dff700 1 --2- 172.21.15.119:0/1535777325 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc90c155c10 0x7fc90c152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.154+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 >> 172.21.15.119:0/1535777325 conn(0x7fc90c06a320 msgr2=0x7fc90c0cab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:10.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.154+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 shutdown_connections 2023-10-08T01:09:10.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.154+0000 7fc913dff700 1 -- 172.21.15.119:0/1535777325 wait complete. 2023-10-08T01:09:10.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T01:09:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.357+0000 7f89ff59e700 1 Processor -- start 2023-10-08T01:09:10.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.357+0000 7f89ff59e700 1 -- start start 2023-10-08T01:09:10.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.358+0000 7f89ff59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a00159a60 0x7f8a000c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:10.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.358+0000 7f89ff59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a000c9cb0 con 0x7f8a000ccc00 2023-10-08T01:09:10.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.358+0000 7f89ff59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a000c9f30 con 0x7f8a000cb230 2023-10-08T01:09:10.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.358+0000 7f89ff59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0006dfc0 con 0x7f8a00159a60 2023-10-08T01:09:10.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.358+0000 7f89fdd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8a000ccc00 0x7f8a00069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44434/0 (socket says 172.21.15.119:44434) 2023-10-08T01:09:10.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.358+0000 7f89fdd9b700 1 -- 172.21.15.119:0/1197017237 learned_addr learned my addr 172.21.15.119:0/1197017237 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fe59c700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a00159a60 0x7f8a000c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164720075 0 0) 0x7f8a000c9cb0 con 0x7f8a000ccc00 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e8003570 con 0x7f8a000ccc00 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2647191324 0 0) 0x7f89e8003570 con 0x7f8a000ccc00 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a00159a60 msgr2=0x7f8a000c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a00159a60 0x7f8a000c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 >> v1:172.21.15.184:6789/0 conn(0x7f8a000cb230 legacy=0x7f8a0006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a000ca470 con 0x7f8a000ccc00 2023-10-08T01:09:10.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f89f4001470 con 0x7f8a000ccc00 2023-10-08T01:09:10.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 >> v1:172.21.15.119:6789/0 conn(0x7f8a000ccc00 legacy=0x7f8a00069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:10.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e8003a90 0x7f89e8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:10.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8004810 0x7f89e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:10.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a00159a60 0x7f89e80051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:10.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0006dfc0 con 0x7f89e8004810 2023-10-08T01:09:10.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e800bb60 con 0x7f89e8003a90 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.359+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e800bd20 con 0x7f8a00159a60 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fe59c700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e8003a90 0x7f89e8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fdd9b700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8004810 0x7f89e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fed9d700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a00159a60 0x7f89e80051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fe59c700 1 -- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a00159a60 msgr2=0x7f89e80051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fe59c700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a00159a60 0x7f89e80051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fe59c700 1 -- 172.21.15.119:0/1197017237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8004810 msgr2=0x7f89e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fe59c700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8004810 0x7f89e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fe59c700 1 -- 172.21.15.119:0/1197017237 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e8003570 con 0x7f89e8003a90 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fdd9b700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8004810 0x7f89e8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.360+0000 7f89fe59c700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e8003a90 0x7f89e8003ea0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009c00 tx=0x7f89f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b618f8956e1f18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f000e710 con 0x7f89e8003a90 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f000e870 con 0x7f89e8003a90 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89fd59a700 1 -- 172.21.15.119:0/1197017237 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f000ec30 con 0x7f89e8003a90 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89ff59e700 1 -- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e8003a90 msgr2=0x7f89e8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89ff59e700 1 --2- 172.21.15.119:0/1197017237 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89e8003a90 0x7f89e8003ea0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f89f0009c00 tx=0x7f89f00091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89fed9d700 1 -- 172.21.15.119:0/1197017237 reap_dead start 2023-10-08T01:09:10.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89ff59e700 1 -- 172.21.15.119:0/1197017237 shutdown_connections 2023-10-08T01:09:10.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89ff59e700 1 -- 172.21.15.119:0/1197017237 >> 172.21.15.119:0/1197017237 conn(0x7f8a0006baa0 msgr2=0x7f8a0006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:10.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89ff59e700 1 -- 172.21.15.119:0/1197017237 shutdown_connections 2023-10-08T01:09:10.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.361+0000 7f89ff59e700 1 -- 172.21.15.119:0/1197017237 wait complete. 2023-10-08T01:09:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.363+0000 7f89ff59e700 1 Processor -- start 2023-10-08T01:09:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.363+0000 7f89ff59e700 1 -- start start 2023-10-08T01:09:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.363+0000 7f89ff59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8003a90 0x7f8a000c5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89ff59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000c7fb0 0x7f8a00169f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89ff59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0016a480 0x7f8a0016a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89ff59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a000c69a0 con 0x7f89e8003a90 2023-10-08T01:09:10.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89ff59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0016ae30 con 0x7f8a0016a480 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89ff59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a00161dd0 con 0x7f8a000c7fb0 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8003a90 0x7f8a000c5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8003a90 0x7f8a000c5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55128/0 (socket says 172.21.15.119:55128) 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 -- 172.21.15.119:0/1031522796 learned_addr learned my addr 172.21.15.119:0/1031522796 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fdd9b700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000c7fb0 0x7f8a00169f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 -- 172.21.15.119:0/1031522796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000c7fb0 msgr2=0x7f8a00169f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000c7fb0 0x7f8a00169f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 -- 172.21.15.119:0/1031522796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0016a480 msgr2=0x7f8a0016a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:10.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0016a480 0x7f8a0016a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 -- 172.21.15.119:0/1031522796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a000c5d90 con 0x7f89e8003a90 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fdd9b700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000c7fb0 0x7f8a00169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.364+0000 7f89fe59c700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8003a90 0x7f8a000c5740 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f89f0013ef0 tx=0x7f89f001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.365+0000 7f89ee7fc700 1 -- 172.21.15.119:0/1031522796 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0022070 con 0x7f89e8003a90 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.365+0000 7f89ee7fc700 1 -- 172.21.15.119:0/1031522796 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f000e9e0 con 0x7f89e8003a90 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.365+0000 7f89ee7fc700 1 -- 172.21.15.119:0/1031522796 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f0027470 con 0x7f89e8003a90 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.365+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a000c6070 con 0x7f89e8003a90 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.365+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a000c6630 con 0x7f89e8003a90 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.366+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0006eba0 con 0x7f89e8003a90 2023-10-08T01:09:10.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.366+0000 7f89ee7fc700 1 -- 172.21.15.119:0/1031522796 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89f001c450 con 0x7f89e8003a90 2023-10-08T01:09:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.367+0000 7f89ee7fc700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f89e0066f30 0x7f89e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.367+0000 7f89ee7fc700 1 -- 172.21.15.119:0/1031522796 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f89f00862e0 con 0x7f89e8003a90 2023-10-08T01:09:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.368+0000 7f89fdd9b700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f89e0066f30 0x7f89e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.368+0000 7f89fdd9b700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f89e0066f30 0x7f89e00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89f4000f60 tx=0x7f89f4009040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:10.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.372+0000 7f89ee7fc700 1 -- 172.21.15.119:0/1031522796 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89f0053340 con 0x7f89e8003a90 2023-10-08T01:09:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:10.613+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8a000ca750 con 0x7f89e8003a90 2023-10-08T01:09:11.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.324+0000 7f89ee7fc700 1 -- 172.21.15.119:0/1031522796 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f89f0056960 con 0x7f89e8003a90 2023-10-08T01:09:11.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f89e0066f30 msgr2=0x7f89e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:11.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f89e0066f30 0x7f89e00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f89f4000f60 tx=0x7f89f4009040 comp rx=0 tx=0).stop 2023-10-08T01:09:11.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8003a90 msgr2=0x7f8a000c5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:11.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8003a90 0x7f8a000c5740 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f89f0013ef0 tx=0x7f89f001a000 comp rx=0 tx=0).stop 2023-10-08T01:09:11.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 shutdown_connections 2023-10-08T01:09:11.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f89e0066f30 0x7f89e00693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a0016a480 0x7f8a0016a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89e8003a90 0x7f8a000c5740 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 --2- 172.21.15.119:0/1031522796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a000c7fb0 0x7f8a00169f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.327+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 >> 172.21.15.119:0/1031522796 conn(0x7f8a0006baa0 msgr2=0x7f8a000c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:11.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.328+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 shutdown_connections 2023-10-08T01:09:11.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.328+0000 7f89ff59e700 1 -- 172.21.15.119:0/1031522796 wait complete. 2023-10-08T01:09:11.347 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T01:09:11.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.531+0000 7f6724d9d700 1 Processor -- start 2023-10-08T01:09:11.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.531+0000 7f6724d9d700 1 -- start start 2023-10-08T01:09:11.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.532+0000 7f6724d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6720153900 0x7f67200c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f6724d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f672006a710 0x7f672006e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f6724d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f672006e150 con 0x7f67200c9db0 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f6724d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67200c7800 con 0x7f672006a710 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f6724d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200c7ae0 con 0x7f6720153900 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f671ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f672006a710 0x7f672006e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f671ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f672006a710 0x7f672006e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55142/0 (socket says 172.21.15.119:55142) 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f671ffff700 1 -- 172.21.15.119:0/1930030051 learned_addr learned my addr 172.21.15.119:0/1930030051 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f671f7fe700 1 --2- 172.21.15.119:0/1930030051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6720153900 0x7f67200c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f671ffff700 1 -- 172.21.15.119:0/1930030051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6720153900 msgr2=0x7f67200c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f671ffff700 1 --2- 172.21.15.119:0/1930030051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6720153900 0x7f67200c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f671ffff700 1 -- 172.21.15.119:0/1930030051 >> v1:172.21.15.184:6789/0 conn(0x7f67200c9db0 legacy=0x7f672006a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:11.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.533+0000 7f671ffff700 1 -- 172.21.15.119:0/1930030051 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67200c7dc0 con 0x7f672006a710 2023-10-08T01:09:11.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.534+0000 7f671ffff700 1 --2- 172.21.15.119:0/1930030051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672006e5e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6718009f20 tx=0x7f6718009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f0b086f9ebd5eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:11.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.534+0000 7f671e7fc700 1 -- 172.21.15.119:0/1930030051 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671800e070 con 0x7f672006a710 2023-10-08T01:09:11.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.534+0000 7f671e7fc700 1 -- 172.21.15.119:0/1930030051 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671800b9d0 con 0x7f672006a710 2023-10-08T01:09:11.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.534+0000 7f671e7fc700 1 -- 172.21.15.119:0/1930030051 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67180092e0 con 0x7f672006a710 2023-10-08T01:09:11.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.534+0000 7f6724d9d700 1 -- 172.21.15.119:0/1930030051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 msgr2=0x7f672006e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:11.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.534+0000 7f6724d9d700 1 --2- 172.21.15.119:0/1930030051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672006e5e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6718009f20 tx=0x7f6718009840 comp rx=0 tx=0).stop 2023-10-08T01:09:11.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.534+0000 7f6724d9d700 1 -- 172.21.15.119:0/1930030051 shutdown_connections 2023-10-08T01:09:11.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.535+0000 7f6724d9d700 1 --2- 172.21.15.119:0/1930030051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672006e5e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.535+0000 7f6724d9d700 1 --2- 172.21.15.119:0/1930030051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6720153900 0x7f67200c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.535+0000 7f6724d9d700 1 -- 172.21.15.119:0/1930030051 >> 172.21.15.119:0/1930030051 conn(0x7f67200cc1e0 msgr2=0x7f67200cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.535+0000 7f6724d9d700 1 -- 172.21.15.119:0/1930030051 shutdown_connections 2023-10-08T01:09:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.535+0000 7f6724d9d700 1 -- 172.21.15.119:0/1930030051 wait complete. 2023-10-08T01:09:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.536+0000 7f6724d9d700 1 Processor -- start 2023-10-08T01:09:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.536+0000 7f6724d9d700 1 -- start start 2023-10-08T01:09:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f6724d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f6724d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c9db0 0x7f67200acda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f6724d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6720153900 0x7f67201671e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f6724d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67200afa50 con 0x7f672006a710 2023-10-08T01:09:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f6724d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67200ad2e0 con 0x7f6720153900 2023-10-08T01:09:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f6724d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67200ad580 con 0x7f67200c9db0 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f671f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672015a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55152/0 (socket says 172.21.15.119:55152) 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f671f7fe700 1 -- 172.21.15.119:0/44540463 learned_addr learned my addr 172.21.15.119:0/44540463 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f671ffff700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6720153900 0x7f67201671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.537+0000 7f671effd700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c9db0 0x7f67200acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f671f7fe700 1 -- 172.21.15.119:0/44540463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c9db0 msgr2=0x7f67200acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f671f7fe700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c9db0 0x7f67200acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f671f7fe700 1 -- 172.21.15.119:0/44540463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6720153900 msgr2=0x7f67201671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f671f7fe700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6720153900 0x7f67201671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f671f7fe700 1 -- 172.21.15.119:0/44540463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6718009ae0 con 0x7f672006a710 2023-10-08T01:09:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f671f7fe700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672015a4b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f671000ec80 tx=0x7f67100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f67077fe700 1 -- 172.21.15.119:0/44540463 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000c850 con 0x7f672006a710 2023-10-08T01:09:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f67077fe700 1 -- 172.21.15.119:0/44540463 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671000c9b0 con 0x7f672006a710 2023-10-08T01:09:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67200ad860 con 0x7f672006a710 2023-10-08T01:09:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f67077fe700 1 -- 172.21.15.119:0/44540463 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671000cd50 con 0x7f672006a710 2023-10-08T01:09:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.538+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67200ade20 con 0x7f672006a710 2023-10-08T01:09:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.539+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672006c450 con 0x7f672006a710 2023-10-08T01:09:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.540+0000 7f67077fe700 1 -- 172.21.15.119:0/44540463 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6710013680 con 0x7f672006a710 2023-10-08T01:09:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.541+0000 7f67077fe700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f670c066f30 0x7f670c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.541+0000 7f67077fe700 1 -- 172.21.15.119:0/44540463 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f6710084a00 con 0x7f672006a710 2023-10-08T01:09:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.541+0000 7f671effd700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f670c066f30 0x7f670c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:11.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.542+0000 7f671effd700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f670c066f30 0x7f670c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67200aba50 tx=0x7f6714009040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:11.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.545+0000 7f67077fe700 1 -- 172.21.15.119:0/44540463 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6710051a60 con 0x7f672006a710 2023-10-08T01:09:11.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:11.778+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f672006dd20 con 0x7f672006a710 2023-10-08T01:09:12.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.330+0000 7f67077fe700 1 -- 172.21.15.119:0/44540463 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6710055080 con 0x7f672006a710 2023-10-08T01:09:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f670c066f30 msgr2=0x7f670c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f670c066f30 0x7f670c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67200aba50 tx=0x7f6714009040 comp rx=0 tx=0).stop 2023-10-08T01:09:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 msgr2=0x7f672015a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672015a4b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f671000ec80 tx=0x7f67100091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 shutdown_connections 2023-10-08T01:09:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f670c066f30 0x7f670c0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6720153900 0x7f67201671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f672006a710 0x7f672015a4b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 --2- 172.21.15.119:0/44540463 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67200c9db0 0x7f67200acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 >> 172.21.15.119:0/44540463 conn(0x7f67200cc1e0 msgr2=0x7f672006ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 shutdown_connections 2023-10-08T01:09:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.333+0000 7f6724d9d700 1 -- 172.21.15.119:0/44540463 wait complete. 2023-10-08T01:09:12.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:09:12.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.533+0000 7f9fd7bff700 1 Processor -- start 2023-10-08T01:09:12.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.533+0000 7f9fd7bff700 1 -- start start 2023-10-08T01:09:12.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.534+0000 7f9fd7bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9fd01578b0 0x7f9fd0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.534+0000 7f9fd7bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd00ad930 con 0x7f9fd0155c10 2023-10-08T01:09:12.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.534+0000 7f9fd7bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd00acd20 con 0x7f9fd00af2f0 2023-10-08T01:09:12.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.534+0000 7f9fd7bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd00ad000 con 0x7f9fd01578b0 2023-10-08T01:09:12.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.535+0000 7f9fd519a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9fd01578b0 0x7f9fd0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.535+0000 7f9fd599b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9fd0155c10 0x7f9fd00ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44438/0 (socket says 172.21.15.119:44438) 2023-10-08T01:09:12.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.535+0000 7f9fd599b700 1 -- 172.21.15.119:0/1755685417 learned_addr learned my addr 172.21.15.119:0/1755685417 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:12.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.535+0000 7f9fd519a700 1 -- 172.21.15.119:0/1755685417 >> v1:172.21.15.184:6789/0 conn(0x7f9fd00af2f0 legacy=0x7f9fd00af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:12.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.535+0000 7f9fd519a700 1 -- 172.21.15.119:0/1755685417 >> v1:172.21.15.119:6789/0 conn(0x7f9fd0155c10 legacy=0x7f9fd00ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.535+0000 7f9fd519a700 1 -- 172.21.15.119:0/1755685417 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd00ad2e0 con 0x7f9fd01578b0 2023-10-08T01:09:12.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.535+0000 7f9fd4999700 1 -- 172.21.15.119:0/1755685417 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198950529 0 0) 0x7f9fd00ad930 con 0x7f9fd0155c10 2023-10-08T01:09:12.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.535+0000 7f9fd519a700 1 --2- 172.21.15.119:0/1755685417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd0155fe0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00098e0 tx=0x7f9fc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14905a184fd4eb54 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:12.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd4999700 1 -- 172.21.15.119:0/1755685417 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0009e80 con 0x7f9fd01578b0 2023-10-08T01:09:12.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd4999700 1 -- 172.21.15.119:0/1755685417 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc000b820 con 0x7f9fd01578b0 2023-10-08T01:09:12.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd4999700 1 -- 172.21.15.119:0/1755685417 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000bba0 con 0x7f9fd01578b0 2023-10-08T01:09:12.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1755685417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 msgr2=0x7f9fd0155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd7bff700 1 --2- 172.21.15.119:0/1755685417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd0155fe0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00098e0 tx=0x7f9fc00091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1755685417 shutdown_connections 2023-10-08T01:09:12.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd7bff700 1 --2- 172.21.15.119:0/1755685417 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd0155fe0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1755685417 >> 172.21.15.119:0/1755685417 conn(0x7f9fd006a320 msgr2=0x7f9fd00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:12.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1755685417 shutdown_connections 2023-10-08T01:09:12.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.536+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1755685417 wait complete. 2023-10-08T01:09:12.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.538+0000 7f9fd7bff700 1 Processor -- start 2023-10-08T01:09:12.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.538+0000 7f9fd7bff700 1 -- start start 2023-10-08T01:09:12.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd7bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd00af2f0 0x7f9fd016de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd7bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd0155c10 0x7f9fd015b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd7bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd015bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd7bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0151ba0 con 0x7f9fd01578b0 2023-10-08T01:09:12.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd7bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0169e70 con 0x7f9fd00af2f0 2023-10-08T01:09:12.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd7bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd006ee60 con 0x7f9fd0155c10 2023-10-08T01:09:12.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd619c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd015bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd619c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd015bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55174/0 (socket says 172.21.15.119:55174) 2023-10-08T01:09:12.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd619c700 1 -- 172.21.15.119:0/1303376031 learned_addr learned my addr 172.21.15.119:0/1303376031 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:12.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd599b700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd00af2f0 0x7f9fd016de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.539+0000 7f9fd519a700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd0155c10 0x7f9fd015b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fd619c700 1 -- 172.21.15.119:0/1303376031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd0155c10 msgr2=0x7f9fd015b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fd619c700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd0155c10 0x7f9fd015b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fd619c700 1 -- 172.21.15.119:0/1303376031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd00af2f0 msgr2=0x7f9fd016de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fd619c700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd00af2f0 0x7f9fd016de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fd619c700 1 -- 172.21.15.119:0/1303376031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc0009420 con 0x7f9fd01578b0 2023-10-08T01:09:12.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fd619c700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd015bc90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800c930 tx=0x7f9fc80026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:12.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fc5ffb700 1 -- 172.21.15.119:0/1303376031 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8012630 con 0x7f9fd01578b0 2023-10-08T01:09:12.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fc5ffb700 1 -- 172.21.15.119:0/1303376031 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc80127d0 con 0x7f9fd01578b0 2023-10-08T01:09:12.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd016a080 con 0x7f9fd01578b0 2023-10-08T01:09:12.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fc5ffb700 1 -- 172.21.15.119:0/1303376031 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8012b90 con 0x7f9fd01578b0 2023-10-08T01:09:12.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.540+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd016a640 con 0x7f9fd01578b0 2023-10-08T01:09:12.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.541+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd004c3c0 con 0x7f9fd01578b0 2023-10-08T01:09:12.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.542+0000 7f9fc5ffb700 1 -- 172.21.15.119:0/1303376031 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9fc801a3b0 con 0x7f9fd01578b0 2023-10-08T01:09:12.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.543+0000 7f9fc5ffb700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.544+0000 7f9fc5ffb700 1 -- 172.21.15.119:0/1303376031 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f9fc8084390 con 0x7f9fd01578b0 2023-10-08T01:09:12.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.544+0000 7f9fd599b700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.544+0000 7f9fd599b700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9fb4066f30 0x7f9fb40693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc007fd0 tx=0x7f9fcc007d30 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:12.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.547+0000 7f9fc5ffb700 1 -- 172.21.15.119:0/1303376031 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9fc8054e80 con 0x7f9fd01578b0 2023-10-08T01:09:12.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.779+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fd00ad5c0 con 0x7f9fd01578b0 2023-10-08T01:09:12.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.780+0000 7f9fc5ffb700 1 -- 172.21.15.119:0/1303376031 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f9fc8054a10 con 0x7f9fd01578b0 2023-10-08T01:09:12.781 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:12.781 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1019078588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1019078588},{"type":"v1","addr":"172.21.15.119:6835","nonce":1019078588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1353749238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1353749238},{"type":"v1","addr":"172.21.15.184:6834","nonce":1353749238}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/186953052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":186953052},{"type":"v1","addr":"172.21.15.184:6835","nonce":186953052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:09:08.771490+0000","modified":"2023-10-08T01:09:12.322278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/920039174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":920039174},{"type":"v1","addr":"172.21.15.119:6837","nonce":920039174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9fb4066f30 msgr2=0x7f9fb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9fb4066f30 0x7f9fb40693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc007fd0 tx=0x7f9fcc007d30 comp rx=0 tx=0).stop 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 msgr2=0x7f9fd015bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd015bc90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f9fc800c930 tx=0x7f9fc80026e0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 shutdown_connections 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9fb4066f30 0x7f9fb40693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9fd01578b0 0x7f9fd015bc90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9fd00af2f0 0x7f9fd016de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 --2- 172.21.15.119:0/1303376031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9fd0155c10 0x7f9fd015b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.782+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 >> 172.21.15.119:0/1303376031 conn(0x7f9fd006a320 msgr2=0x7f9fd0068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:12.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.783+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 shutdown_connections 2023-10-08T01:09:12.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.783+0000 7f9fd7bff700 1 -- 172.21.15.119:0/1303376031 wait complete. 2023-10-08T01:09:12.785 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 24 2023-10-08T01:09:12.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:09:12.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.981+0000 7ffaf4b9d700 1 Processor -- start 2023-10-08T01:09:12.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.981+0000 7ffaf4b9d700 1 -- start start 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.982+0000 7ffaf4b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffaf0159420 0x7ffaf00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.982+0000 7ffaf4b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf00b2980 con 0x7ffaf015d230 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.982+0000 7ffaf4b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf00b11d0 con 0x7ffaf01597f0 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.982+0000 7ffaf4b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf00b14b0 con 0x7ffaf0159420 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.982+0000 7ffaef7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffaf0159420 0x7ffaf00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.982+0000 7ffaeffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffaf015d230 0x7ffaf015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44452/0 (socket says 172.21.15.119:44452) 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.982+0000 7ffaeffff700 1 -- 172.21.15.119:0/2450411296 learned_addr learned my addr 172.21.15.119:0/2450411296 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.983+0000 7ffaef7fe700 1 -- 172.21.15.119:0/2450411296 >> v1:172.21.15.184:6789/0 conn(0x7ffaf01597f0 legacy=0x7ffaf00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:12.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.983+0000 7ffaef7fe700 1 -- 172.21.15.119:0/2450411296 >> v1:172.21.15.119:6789/0 conn(0x7ffaf015d230 legacy=0x7ffaf015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.983+0000 7ffaef7fe700 1 -- 172.21.15.119:0/2450411296 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf00b1790 con 0x7ffaf0159420 2023-10-08T01:09:12.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.983+0000 7ffaee7fc700 1 -- 172.21.15.119:0/2450411296 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291249198 0 0) 0x7ffaf00b2980 con 0x7ffaf015d230 2023-10-08T01:09:12.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.983+0000 7ffaef7fe700 1 --2- 172.21.15.119:0/2450411296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00b5f40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ffae00098e0 tx=0x7ffae00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9888fd1c31b1ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:12.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.983+0000 7ffaee7fc700 1 -- 172.21.15.119:0/2450411296 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0009e80 con 0x7ffaf0159420 2023-10-08T01:09:12.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.983+0000 7ffaee7fc700 1 -- 172.21.15.119:0/2450411296 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae000b820 con 0x7ffaf0159420 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.983+0000 7ffaee7fc700 1 -- 172.21.15.119:0/2450411296 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000bba0 con 0x7ffaf0159420 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.984+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/2450411296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 msgr2=0x7ffaf00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.984+0000 7ffaf4b9d700 1 --2- 172.21.15.119:0/2450411296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00b5f40 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ffae00098e0 tx=0x7ffae00091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.984+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/2450411296 shutdown_connections 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.984+0000 7ffaf4b9d700 1 --2- 172.21.15.119:0/2450411296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00b5f40 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.984+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/2450411296 >> 172.21.15.119:0/2450411296 conn(0x7ffaf00d0660 msgr2=0x7ffaf00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.984+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/2450411296 shutdown_connections 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.984+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/2450411296 wait complete. 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.985+0000 7ffaf4b9d700 1 Processor -- start 2023-10-08T01:09:12.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.986+0000 7ffaf4b9d700 1 -- start start 2023-10-08T01:09:12.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.986+0000 7ffaf4b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.986+0000 7ffaf4b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf01597f0 0x7ffaf015ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.986+0000 7ffaf4b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf015d230 0x7ffaf00c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaf4b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00cad80 con 0x7ffaf0159420 2023-10-08T01:09:12.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaf4b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00c8d00 con 0x7ffaf015d230 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaf4b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf00c8fe0 con 0x7ffaf01597f0 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55192/0 (socket says 172.21.15.119:55192) 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 -- 172.21.15.119:0/3496014892 learned_addr learned my addr 172.21.15.119:0/3496014892 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaeffff700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf015d230 0x7ffaf00c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaeeffd700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf01597f0 0x7ffaf015ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 -- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf01597f0 msgr2=0x7ffaf015ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf01597f0 0x7ffaf015ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 -- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf015d230 msgr2=0x7ffaf00c84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf015d230 0x7ffaf00c84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:12.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 -- 172.21.15.119:0/3496014892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae0009420 con 0x7ffaf0159420 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaeeffd700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf01597f0 0x7ffaf015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaeffff700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf015d230 0x7ffaf00c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.987+0000 7ffaef7fe700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00c5f00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ffae00030c0 tx=0x7ffae0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.988+0000 7ffad77fe700 1 -- 172.21.15.119:0/3496014892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae000ba10 con 0x7ffaf0159420 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.988+0000 7ffad77fe700 1 -- 172.21.15.119:0/3496014892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae0020070 con 0x7ffaf0159420 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.988+0000 7ffad77fe700 1 -- 172.21.15.119:0/3496014892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae001b6a0 con 0x7ffaf0159420 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.988+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf00ca730 con 0x7ffaf0159420 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.988+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf015b1b0 con 0x7ffaf0159420 2023-10-08T01:09:12.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.989+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf00ca590 con 0x7ffaf0159420 2023-10-08T01:09:12.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.990+0000 7ffad77fe700 1 -- 172.21.15.119:0/3496014892 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffae0022030 con 0x7ffaf0159420 2023-10-08T01:09:12.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.990+0000 7ffad77fe700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ffadc066f30 0x7ffadc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:12.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.991+0000 7ffad77fe700 1 -- 172.21.15.119:0/3496014892 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7ffae00867d0 con 0x7ffaf0159420 2023-10-08T01:09:12.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.991+0000 7ffaeeffd700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ffadc066f30 0x7ffadc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:12.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.991+0000 7ffaeeffd700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ffadc066f30 0x7ffadc0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00c5680 tx=0x7ffae4009040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:12.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:12.995+0000 7ffad77fe700 1 -- 172.21.15.119:0/3496014892 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffaf00ca590 con 0x7ffaf0159420 2023-10-08T01:09:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.187+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffaf00ca590 con 0x7ffaf0159420 2023-10-08T01:09:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.188+0000 7ffad77fe700 1 -- 172.21.15.119:0/3496014892 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+15951 (secure 0 0 0) 0x7ffaf00ca590 con 0x7ffaf0159420 2023-10-08T01:09:13.189 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:13.189 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":59,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:09:10.308204+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:09:06.334038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T01:09:07.759893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:09:13.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ffadc066f30 msgr2=0x7ffadc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ffadc066f30 0x7ffadc0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ffaf00c5680 tx=0x7ffae4009040 comp rx=0 tx=0).stop 2023-10-08T01:09:13.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 msgr2=0x7ffaf00c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00c5f00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ffae00030c0 tx=0x7ffae0016000 comp rx=0 tx=0).stop 2023-10-08T01:09:13.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 shutdown_connections 2023-10-08T01:09:13.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ffadc066f30 0x7ffadc0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf015d230 0x7ffaf00c84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf0159420 0x7ffaf00c5f00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 --2- 172.21.15.119:0/3496014892 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf01597f0 0x7ffaf015ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 >> 172.21.15.119:0/3496014892 conn(0x7ffaf00d0660 msgr2=0x7ffaf0156100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:13.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.190+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 shutdown_connections 2023-10-08T01:09:13.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.191+0000 7ffaf4b9d700 1 -- 172.21.15.119:0/3496014892 wait complete. 2023-10-08T01:09:13.209 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T01:09:13.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:09:13.274 INFO:tasks.ceph.mds.c.smithi119.stderr:2023-10-08T01:09:13.273+0000 7fb70fdf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:09:13.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.391+0000 7f28829ff700 1 Processor -- start 2023-10-08T01:09:13.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.391+0000 7f28829ff700 1 -- start start 2023-10-08T01:09:13.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f28829ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0cbf30 0x7f287c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f28829ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f287c0b1420 0x7f287c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f28829ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f287c0cc300 con 0x7f287c0ccf90 2023-10-08T01:09:13.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f28829ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f287c156bf0 con 0x7f287c0b1420 2023-10-08T01:09:13.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f28829ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c0ae950 con 0x7f287c0cbf30 2023-10-08T01:09:13.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f287b7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f287c0ccf90 0x7f287c0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44468/0 (socket says 172.21.15.119:44468) 2023-10-08T01:09:13.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f287b7fe700 1 -- 172.21.15.119:0/2934110928 learned_addr learned my addr 172.21.15.119:0/2934110928 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:13.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f2880f9c700 1 --2- 172.21.15.119:0/2934110928 >> v2:172.21.15.119:3300/0 conn(0x7f287c0b1420 0x7f287c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f287bfff700 1 --2- 172.21.15.119:0/2934110928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0cbf30 0x7f287c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f2880f9c700 1 -- 172.21.15.119:0/2934110928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0cbf30 msgr2=0x7f287c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:13.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f287affd700 1 -- 172.21.15.119:0/2934110928 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2834571789 0 0) 0x7f287c0cc300 con 0x7f287c0ccf90 2023-10-08T01:09:13.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f2880f9c700 1 --2- 172.21.15.119:0/2934110928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0cbf30 0x7f287c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f2880f9c700 1 -- 172.21.15.119:0/2934110928 >> v1:172.21.15.119:6789/0 conn(0x7f287c0ccf90 legacy=0x7f287c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.393+0000 7f2880f9c700 1 -- 172.21.15.119:0/2934110928 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c0ad370 con 0x7f287c0b1420 2023-10-08T01:09:13.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f2880f9c700 1 --2- 172.21.15.119:0/2934110928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0b1420 0x7f287c0c9740 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2874009e90 tx=0x7f2874009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1894d7094510601d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:13.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f287affd700 1 -- 172.21.15.119:0/2934110928 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287400e070 con 0x7f287c0b1420 2023-10-08T01:09:13.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f287affd700 1 -- 172.21.15.119:0/2934110928 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287400bbf0 con 0x7f287c0b1420 2023-10-08T01:09:13.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f287affd700 1 -- 172.21.15.119:0/2934110928 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874012430 con 0x7f287c0b1420 2023-10-08T01:09:13.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f28829ff700 1 -- 172.21.15.119:0/2934110928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0b1420 msgr2=0x7f287c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f28829ff700 1 --2- 172.21.15.119:0/2934110928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0b1420 0x7f287c0c9740 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2874009e90 tx=0x7f2874009780 comp rx=0 tx=0).stop 2023-10-08T01:09:13.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f28829ff700 1 -- 172.21.15.119:0/2934110928 shutdown_connections 2023-10-08T01:09:13.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f28829ff700 1 --2- 172.21.15.119:0/2934110928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0b1420 0x7f287c0c9740 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f28829ff700 1 --2- 172.21.15.119:0/2934110928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0cbf30 0x7f287c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.394+0000 7f28829ff700 1 -- 172.21.15.119:0/2934110928 >> 172.21.15.119:0/2934110928 conn(0x7f287c06a320 msgr2=0x7f287c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:13.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.395+0000 7f28829ff700 1 -- 172.21.15.119:0/2934110928 shutdown_connections 2023-10-08T01:09:13.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.395+0000 7f28829ff700 1 -- 172.21.15.119:0/2934110928 wait complete. 2023-10-08T01:09:13.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.396+0000 7f28829ff700 1 Processor -- start 2023-10-08T01:09:13.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.396+0000 7f28829ff700 1 -- start start 2023-10-08T01:09:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f28829ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f287c0b1420 0x7f287c154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f28829ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0cbf30 0x7f287c0a8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f28829ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0ccf90 0x7f287c15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f28829ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1557b0 con 0x7f287c0cbf30 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f28829ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c1689d0 con 0x7f287c0b1420 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f28829ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c167c00 con 0x7f287c0ccf90 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f287b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0cbf30 0x7f287c0a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f287b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0cbf30 0x7f287c0a8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55208/0 (socket says 172.21.15.119:55208) 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f287b7fe700 1 -- 172.21.15.119:0/1280323944 learned_addr learned my addr 172.21.15.119:0/1280323944 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f287bfff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f287c0b1420 0x7f287c154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.397+0000 7f2880f9c700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0ccf90 0x7f287c15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f287b7fe700 1 -- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0ccf90 msgr2=0x7f287c15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f287b7fe700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0ccf90 0x7f287c15a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f287b7fe700 1 -- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f287c0b1420 msgr2=0x7f287c154e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f287b7fe700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f287c0b1420 0x7f287c154e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f287b7fe700 1 -- 172.21.15.119:0/1280323944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874009a20 con 0x7f287c0cbf30 2023-10-08T01:09:13.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f2880f9c700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0ccf90 0x7f287c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:09:13.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f287b7fe700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0cbf30 0x7f287c0a8210 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f287000c5e0 tx=0x7f2870001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:13.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f2863fff700 1 -- 172.21.15.119:0/1280323944 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870005b30 con 0x7f287c0cbf30 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f2863fff700 1 -- 172.21.15.119:0/1280323944 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2870005c90 con 0x7f287c0cbf30 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287c167ee0 con 0x7f287c0cbf30 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f2863fff700 1 -- 172.21.15.119:0/1280323944 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2870007300 con 0x7f287c0cbf30 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.398+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287c1684a0 con 0x7f287c0cbf30 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.399+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287c168810 con 0x7f287c0cbf30 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.400+0000 7f2863fff700 1 -- 172.21.15.119:0/1280323944 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f287001b3b0 con 0x7f287c0cbf30 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.401+0000 7f2863fff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2868066f30 0x7f28680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.401+0000 7f2863fff700 1 -- 172.21.15.119:0/1280323944 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2870083c20 con 0x7f287c0cbf30 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.401+0000 7f287bfff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2868066f30 0x7f28680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.402+0000 7f287bfff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2868066f30 0x7f28680693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f286c006fd0 tx=0x7f286c008040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:13.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.405+0000 7f2863fff700 1 -- 172.21.15.119:0/1280323944 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f287c168810 con 0x7f287c0cbf30 2023-10-08T01:09:13.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.591+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 --> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f287c168810 con 0x7f2868066f30 2023-10-08T01:09:13.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.601+0000 7f2863fff700 1 -- 172.21.15.119:0/1280323944 <== mgr.5571 v2:172.21.15.119:6834/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288968 (secure 0 0 0) 0x7f287c168810 con 0x7f2868066f30 2023-10-08T01:09:13.607 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2868066f30 msgr2=0x7f28680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2868066f30 0x7f28680693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f286c006fd0 tx=0x7f286c008040 comp rx=0 tx=0).stop 2023-10-08T01:09:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0cbf30 msgr2=0x7f287c0a8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0cbf30 0x7f287c0a8210 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f287000c5e0 tx=0x7f2870001070 comp rx=0 tx=0).stop 2023-10-08T01:09:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 shutdown_connections 2023-10-08T01:09:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2868066f30 0x7f28680693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f287c0ccf90 0x7f287c15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f287c0b1420 0x7f287c154e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 --2- 172.21.15.119:0/1280323944 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f287c0cbf30 0x7f287c0a8210 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.605+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 >> 172.21.15.119:0/1280323944 conn(0x7f287c06a320 msgr2=0x7f287c155d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.606+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 shutdown_connections 2023-10-08T01:09:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.606+0000 7f28829ff700 1 -- 172.21.15.119:0/1280323944 wait complete. 2023-10-08T01:09:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:09:13.624 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T01:09:12.582001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425924,"kb_used_data":6660,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752548796,"statfs":{"total":773094113280,"available":770609967104,"internally_reserved":0,"allocated":6819840,"data_stored":3779304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.288290"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323476+0000","last_change":"2023-10-08T01:09:09.322200+0000","last_active":"2023-10-08T01:09:09.323476+0000","last_peered":"2023-10-08T01:09:09.323476+0000","last_clean":"2023-10-08T01:09:09.323476+0000","last_became_active":"2023-10-08T01:09:09.321292+0000","last_became_peered":"2023-10-08T01:09:09.321292+0000","last_unstale":"2023-10-08T01:09:09.323476+0000","last_undegraded":"2023-10-08T01:09:09.323476+0000","last_fullsized":"2023-10-08T01:09:09.323476+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:47.695405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322358+0000","last_change":"2023-10-08T01:09:09.320320+0000","last_active":"2023-10-08T01:09:09.322358+0000","last_peered":"2023-10-08T01:09:09.322358+0000","last_clean":"2023-10-08T01:09:09.322358+0000","last_became_active":"2023-10-08T01:09:09.318044+0000","last_became_peered":"2023-10-08T01:09:09.318044+0000","last_unstale":"2023-10-08T01:09:09.322358+0000","last_undegraded":"2023-10-08T01:09:09.322358+0000","last_fullsized":"2023-10-08T01:09:09.322358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:38:18.765115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.330197+0000","last_change":"2023-10-08T01:09:09.329288+0000","last_active":"2023-10-08T01:09:09.330197+0000","last_peered":"2023-10-08T01:09:09.330197+0000","last_clean":"2023-10-08T01:09:09.330197+0000","last_became_active":"2023-10-08T01:09:09.329006+0000","last_became_peered":"2023-10-08T01:09:09.329006+0000","last_unstale":"2023-10-08T01:09:09.330197+0000","last_undegraded":"2023-10-08T01:09:09.330197+0000","last_fullsized":"2023-10-08T01:09:09.330197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:13.215258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318725+0000","last_change":"2023-10-08T01:09:09.315238+0000","last_active":"2023-10-08T01:09:09.318725+0000","last_peered":"2023-10-08T01:09:09.318725+0000","last_clean":"2023-10-08T01:09:09.318725+0000","last_became_active":"2023-10-08T01:09:09.314933+0000","last_became_peered":"2023-10-08T01:09:09.314933+0000","last_unstale":"2023-10-08T01:09:09.318725+0000","last_undegraded":"2023-10-08T01:09:09.318725+0000","last_fullsized":"2023-10-08T01:09:09.318725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:27.277557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322416+0000","last_change":"2023-10-08T01:09:09.320731+0000","last_active":"2023-10-08T01:09:09.322416+0000","last_peered":"2023-10-08T01:09:09.322416+0000","last_clean":"2023-10-08T01:09:09.322416+0000","last_became_active":"2023-10-08T01:09:09.320467+0000","last_became_peered":"2023-10-08T01:09:09.320467+0000","last_unstale":"2023-10-08T01:09:09.322416+0000","last_undegraded":"2023-10-08T01:09:09.322416+0000","last_fullsized":"2023-10-08T01:09:09.322416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:50.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.329571+0000","last_change":"2023-10-08T01:09:09.325430+0000","last_active":"2023-10-08T01:09:09.329571+0000","last_peered":"2023-10-08T01:09:09.329571+0000","last_clean":"2023-10-08T01:09:09.329571+0000","last_became_active":"2023-10-08T01:09:09.323547+0000","last_became_peered":"2023-10-08T01:09:09.323547+0000","last_unstale":"2023-10-08T01:09:09.329571+0000","last_undegraded":"2023-10-08T01:09:09.329571+0000","last_fullsized":"2023-10-08T01:09:09.329571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:15.829749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327347+0000","last_change":"2023-10-08T01:09:09.326101+0000","last_active":"2023-10-08T01:09:09.327347+0000","last_peered":"2023-10-08T01:09:09.327347+0000","last_clean":"2023-10-08T01:09:09.327347+0000","last_became_active":"2023-10-08T01:09:09.325822+0000","last_became_peered":"2023-10-08T01:09:09.325822+0000","last_unstale":"2023-10-08T01:09:09.327347+0000","last_undegraded":"2023-10-08T01:09:09.327347+0000","last_fullsized":"2023-10-08T01:09:09.327347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:38.721617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318897+0000","last_change":"2023-10-08T01:09:09.316183+0000","last_active":"2023-10-08T01:09:09.318897+0000","last_peered":"2023-10-08T01:09:09.318897+0000","last_clean":"2023-10-08T01:09:09.318897+0000","last_became_active":"2023-10-08T01:09:09.315862+0000","last_became_peered":"2023-10-08T01:09:09.315862+0000","last_unstale":"2023-10-08T01:09:09.318897+0000","last_undegraded":"2023-10-08T01:09:09.318897+0000","last_fullsized":"2023-10-08T01:09:09.318897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:30.804190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.332804+0000","last_change":"2023-10-08T01:09:09.328609+0000","last_active":"2023-10-08T01:09:09.332804+0000","last_peered":"2023-10-08T01:09:09.332804+0000","last_clean":"2023-10-08T01:09:09.332804+0000","last_became_active":"2023-10-08T01:09:09.328330+0000","last_became_peered":"2023-10-08T01:09:09.328330+0000","last_unstale":"2023-10-08T01:09:09.332804+0000","last_undegraded":"2023-10-08T01:09:09.332804+0000","last_fullsized":"2023-10-08T01:09:09.332804+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:08.943473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.319156+0000","last_change":"2023-10-08T01:09:09.316904+0000","last_active":"2023-10-08T01:09:09.319156+0000","last_peered":"2023-10-08T01:09:09.319156+0000","last_clean":"2023-10-08T01:09:09.319156+0000","last_became_active":"2023-10-08T01:09:09.316639+0000","last_became_peered":"2023-10-08T01:09:09.316639+0000","last_unstale":"2023-10-08T01:09:09.319156+0000","last_undegraded":"2023-10-08T01:09:09.319156+0000","last_fullsized":"2023-10-08T01:09:09.319156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:08.574405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.331164+0000","last_change":"2023-10-08T01:09:09.328672+0000","last_active":"2023-10-08T01:09:09.331164+0000","last_peered":"2023-10-08T01:09:09.331164+0000","last_clean":"2023-10-08T01:09:09.331164+0000","last_became_active":"2023-10-08T01:09:09.328318+0000","last_became_peered":"2023-10-08T01:09:09.328318+0000","last_unstale":"2023-10-08T01:09:09.331164+0000","last_undegraded":"2023-10-08T01:09:09.331164+0000","last_fullsized":"2023-10-08T01:09:09.331164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:59.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.330585+0000","last_change":"2023-10-08T01:09:09.329592+0000","last_active":"2023-10-08T01:09:09.330585+0000","last_peered":"2023-10-08T01:09:09.330585+0000","last_clean":"2023-10-08T01:09:09.330585+0000","last_became_active":"2023-10-08T01:09:09.329141+0000","last_became_peered":"2023-10-08T01:09:09.329141+0000","last_unstale":"2023-10-08T01:09:09.330585+0000","last_undegraded":"2023-10-08T01:09:09.330585+0000","last_fullsized":"2023-10-08T01:09:09.330585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:08.943473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.335204+0000","last_change":"2023-10-08T01:09:09.327822+0000","last_active":"2023-10-08T01:09:09.335204+0000","last_peered":"2023-10-08T01:09:09.335204+0000","last_clean":"2023-10-08T01:09:09.335204+0000","last_became_active":"2023-10-08T01:09:09.327568+0000","last_became_peered":"2023-10-08T01:09:09.327568+0000","last_unstale":"2023-10-08T01:09:09.335204+0000","last_undegraded":"2023-10-08T01:09:09.335204+0000","last_fullsized":"2023-10-08T01:09:09.335204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:17.910526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.331376+0000","last_change":"2023-10-08T01:09:09.328529+0000","last_active":"2023-10-08T01:09:09.331376+0000","last_peered":"2023-10-08T01:09:09.331376+0000","last_clean":"2023-10-08T01:09:09.331376+0000","last_became_active":"2023-10-08T01:09:09.328228+0000","last_became_peered":"2023-10-08T01:09:09.328228+0000","last_unstale":"2023-10-08T01:09:09.331376+0000","last_undegraded":"2023-10-08T01:09:09.331376+0000","last_fullsized":"2023-10-08T01:09:09.331376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:38.532994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327303+0000","last_change":"2023-10-08T01:09:09.322917+0000","last_active":"2023-10-08T01:09:09.327303+0000","last_peered":"2023-10-08T01:09:09.327303+0000","last_clean":"2023-10-08T01:09:09.327303+0000","last_became_active":"2023-10-08T01:09:09.322603+0000","last_became_peered":"2023-10-08T01:09:09.322603+0000","last_unstale":"2023-10-08T01:09:09.327303+0000","last_undegraded":"2023-10-08T01:09:09.327303+0000","last_fullsized":"2023-10-08T01:09:09.327303+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:10.905840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327284+0000","last_change":"2023-10-08T01:09:09.323953+0000","last_active":"2023-10-08T01:09:09.327284+0000","last_peered":"2023-10-08T01:09:09.327284+0000","last_clean":"2023-10-08T01:09:09.327284+0000","last_became_active":"2023-10-08T01:09:09.323267+0000","last_became_peered":"2023-10-08T01:09:09.323267+0000","last_unstale":"2023-10-08T01:09:09.327284+0000","last_undegraded":"2023-10-08T01:09:09.327284+0000","last_fullsized":"2023-10-08T01:09:09.327284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:13.215258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325495+0000","last_change":"2023-10-08T01:09:09.321689+0000","last_active":"2023-10-08T01:09:09.325495+0000","last_peered":"2023-10-08T01:09:09.325495+0000","last_clean":"2023-10-08T01:09:09.325495+0000","last_became_active":"2023-10-08T01:09:09.321436+0000","last_became_peered":"2023-10-08T01:09:09.321436+0000","last_unstale":"2023-10-08T01:09:09.325495+0000","last_undegraded":"2023-10-08T01:09:09.325495+0000","last_fullsized":"2023-10-08T01:09:09.325495+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:12.528510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325532+0000","last_change":"2023-10-08T01:09:09.321163+0000","last_active":"2023-10-08T01:09:09.325532+0000","last_peered":"2023-10-08T01:09:09.325532+0000","last_clean":"2023-10-08T01:09:09.325532+0000","last_became_active":"2023-10-08T01:09:09.320718+0000","last_became_peered":"2023-10-08T01:09:09.320718+0000","last_unstale":"2023-10-08T01:09:09.325532+0000","last_undegraded":"2023-10-08T01:09:09.325532+0000","last_fullsized":"2023-10-08T01:09:09.325532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:47.187822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327242+0000","last_change":"2023-10-08T01:09:09.323783+0000","last_active":"2023-10-08T01:09:09.327242+0000","last_peered":"2023-10-08T01:09:09.327242+0000","last_clean":"2023-10-08T01:09:09.327242+0000","last_became_active":"2023-10-08T01:09:09.322684+0000","last_became_peered":"2023-10-08T01:09:09.322684+0000","last_unstale":"2023-10-08T01:09:09.327242+0000","last_undegraded":"2023-10-08T01:09:09.327242+0000","last_fullsized":"2023-10-08T01:09:09.327242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:27.277557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322921+0000","last_change":"2023-10-08T01:09:09.320248+0000","last_active":"2023-10-08T01:09:09.322921+0000","last_peered":"2023-10-08T01:09:09.322921+0000","last_clean":"2023-10-08T01:09:09.322921+0000","last_became_active":"2023-10-08T01:09:09.319981+0000","last_became_peered":"2023-10-08T01:09:09.319981+0000","last_unstale":"2023-10-08T01:09:09.322921+0000","last_undegraded":"2023-10-08T01:09:09.322921+0000","last_fullsized":"2023-10-08T01:09:09.322921+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:51.512961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.326297+0000","last_change":"2023-10-08T01:09:09.321400+0000","last_active":"2023-10-08T01:09:09.326297+0000","last_peered":"2023-10-08T01:09:09.326297+0000","last_clean":"2023-10-08T01:09:09.326297+0000","last_became_active":"2023-10-08T01:09:09.321120+0000","last_became_peered":"2023-10-08T01:09:09.321120+0000","last_unstale":"2023-10-08T01:09:09.326297+0000","last_undegraded":"2023-10-08T01:09:09.326297+0000","last_fullsized":"2023-10-08T01:09:09.326297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:08.553237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318847+0000","last_change":"2023-10-08T01:09:09.316924+0000","last_active":"2023-10-08T01:09:09.318847+0000","last_peered":"2023-10-08T01:09:09.318847+0000","last_clean":"2023-10-08T01:09:09.318847+0000","last_became_active":"2023-10-08T01:09:09.316601+0000","last_became_peered":"2023-10-08T01:09:09.316601+0000","last_unstale":"2023-10-08T01:09:09.318847+0000","last_undegraded":"2023-10-08T01:09:09.318847+0000","last_fullsized":"2023-10-08T01:09:09.318847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:13.215258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.315432+0000","last_change":"2023-10-08T01:09:09.314521+0000","last_active":"2023-10-08T01:09:09.315432+0000","last_peered":"2023-10-08T01:09:09.315432+0000","last_clean":"2023-10-08T01:09:09.315432+0000","last_became_active":"2023-10-08T01:09:09.314269+0000","last_became_peered":"2023-10-08T01:09:09.314269+0000","last_unstale":"2023-10-08T01:09:09.315432+0000","last_undegraded":"2023-10-08T01:09:09.315432+0000","last_fullsized":"2023-10-08T01:09:09.315432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:17.910526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.315920+0000","last_change":"2023-10-08T01:09:09.313540+0000","last_active":"2023-10-08T01:09:09.315920+0000","last_peered":"2023-10-08T01:09:09.315920+0000","last_clean":"2023-10-08T01:09:09.315920+0000","last_became_active":"2023-10-08T01:09:09.313271+0000","last_became_peered":"2023-10-08T01:09:09.313271+0000","last_unstale":"2023-10-08T01:09:09.315920+0000","last_undegraded":"2023-10-08T01:09:09.315920+0000","last_fullsized":"2023-10-08T01:09:09.315920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:37.644751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327529+0000","last_change":"2023-10-08T01:09:09.325817+0000","last_active":"2023-10-08T01:09:09.327529+0000","last_peered":"2023-10-08T01:09:09.327529+0000","last_clean":"2023-10-08T01:09:09.327529+0000","last_became_active":"2023-10-08T01:09:09.324410+0000","last_became_peered":"2023-10-08T01:09:09.324410+0000","last_unstale":"2023-10-08T01:09:09.327529+0000","last_undegraded":"2023-10-08T01:09:09.327529+0000","last_fullsized":"2023-10-08T01:09:09.327529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:03.235186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.335315+0000","last_change":"2023-10-08T01:09:09.328382+0000","last_active":"2023-10-08T01:09:09.335315+0000","last_peered":"2023-10-08T01:09:09.335315+0000","last_clean":"2023-10-08T01:09:09.335315+0000","last_became_active":"2023-10-08T01:09:09.328116+0000","last_became_peered":"2023-10-08T01:09:09.328116+0000","last_unstale":"2023-10-08T01:09:09.335315+0000","last_undegraded":"2023-10-08T01:09:09.335315+0000","last_fullsized":"2023-10-08T01:09:09.335315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:37.644751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323042+0000","last_change":"2023-10-08T01:09:09.316967+0000","last_active":"2023-10-08T01:09:09.323042+0000","last_peered":"2023-10-08T01:09:09.323042+0000","last_clean":"2023-10-08T01:09:09.323042+0000","last_became_active":"2023-10-08T01:09:09.316711+0000","last_became_peered":"2023-10-08T01:09:09.316711+0000","last_unstale":"2023-10-08T01:09:09.323042+0000","last_undegraded":"2023-10-08T01:09:09.323042+0000","last_fullsized":"2023-10-08T01:09:09.323042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:23.143628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.330605+0000","last_change":"2023-10-08T01:09:09.329674+0000","last_active":"2023-10-08T01:09:09.330605+0000","last_peered":"2023-10-08T01:09:09.330605+0000","last_clean":"2023-10-08T01:09:09.330605+0000","last_became_active":"2023-10-08T01:09:09.329400+0000","last_became_peered":"2023-10-08T01:09:09.329400+0000","last_unstale":"2023-10-08T01:09:09.330605+0000","last_undegraded":"2023-10-08T01:09:09.330605+0000","last_fullsized":"2023-10-08T01:09:09.330605+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:20.710780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327145+0000","last_change":"2023-10-08T01:09:09.323054+0000","last_active":"2023-10-08T01:09:09.327145+0000","last_peered":"2023-10-08T01:09:09.327145+0000","last_clean":"2023-10-08T01:09:09.327145+0000","last_became_active":"2023-10-08T01:09:09.321930+0000","last_became_peered":"2023-10-08T01:09:09.321930+0000","last_unstale":"2023-10-08T01:09:09.327145+0000","last_undegraded":"2023-10-08T01:09:09.327145+0000","last_fullsized":"2023-10-08T01:09:09.327145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:47:28.247811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323010+0000","last_change":"2023-10-08T01:09:09.317908+0000","last_active":"2023-10-08T01:09:09.323010+0000","last_peered":"2023-10-08T01:09:09.323010+0000","last_clean":"2023-10-08T01:09:09.323010+0000","last_became_active":"2023-10-08T01:09:09.317631+0000","last_became_peered":"2023-10-08T01:09:09.317631+0000","last_unstale":"2023-10-08T01:09:09.323010+0000","last_undegraded":"2023-10-08T01:09:09.323010+0000","last_fullsized":"2023-10-08T01:09:09.323010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:08.943473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.320294+0000","last_change":"2023-10-08T01:09:09.318549+0000","last_active":"2023-10-08T01:09:09.320294+0000","last_peered":"2023-10-08T01:09:09.320294+0000","last_clean":"2023-10-08T01:09:09.320294+0000","last_became_active":"2023-10-08T01:09:09.318233+0000","last_became_peered":"2023-10-08T01:09:09.318233+0000","last_unstale":"2023-10-08T01:09:09.320294+0000","last_undegraded":"2023-10-08T01:09:09.320294+0000","last_fullsized":"2023-10-08T01:09:09.320294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:47.695405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.331447+0000","last_change":"2023-10-08T01:09:09.326220+0000","last_active":"2023-10-08T01:09:09.331447+0000","last_peered":"2023-10-08T01:09:09.331447+0000","last_clean":"2023-10-08T01:09:09.331447+0000","last_became_active":"2023-10-08T01:09:09.325942+0000","last_became_peered":"2023-10-08T01:09:09.325942+0000","last_unstale":"2023-10-08T01:09:09.331447+0000","last_undegraded":"2023-10-08T01:09:09.331447+0000","last_fullsized":"2023-10-08T01:09:09.331447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:03.235186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.326277+0000","last_change":"2023-10-08T01:09:09.321483+0000","last_active":"2023-10-08T01:09:09.326277+0000","last_peered":"2023-10-08T01:09:09.326277+0000","last_clean":"2023-10-08T01:09:09.326277+0000","last_became_active":"2023-10-08T01:09:09.321207+0000","last_became_peered":"2023-10-08T01:09:09.321207+0000","last_unstale":"2023-10-08T01:09:09.326277+0000","last_undegraded":"2023-10-08T01:09:09.326277+0000","last_fullsized":"2023-10-08T01:09:09.326277+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:30.804190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"58'4","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.345174+0000","last_change":"2023-10-08T01:09:08.313826+0000","last_active":"2023-10-08T01:09:09.345174+0000","last_peered":"2023-10-08T01:09:09.345174+0000","last_clean":"2023-10-08T01:09:09.345174+0000","last_became_active":"2023-10-08T01:09:08.313333+0000","last_became_peered":"2023-10-08T01:09:08.313333+0000","last_unstale":"2023-10-08T01:09:09.345174+0000","last_undegraded":"2023-10-08T01:09:09.345174+0000","last_fullsized":"2023-10-08T01:09:09.345174+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:58.460395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325954+0000","last_change":"2023-10-08T01:09:09.316052+0000","last_active":"2023-10-08T01:09:09.325954+0000","last_peered":"2023-10-08T01:09:09.325954+0000","last_clean":"2023-10-08T01:09:09.325954+0000","last_became_active":"2023-10-08T01:09:09.315769+0000","last_became_peered":"2023-10-08T01:09:09.315769+0000","last_unstale":"2023-10-08T01:09:09.325954+0000","last_undegraded":"2023-10-08T01:09:09.325954+0000","last_fullsized":"2023-10-08T01:09:09.325954+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:51.512961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.337597+0000","last_change":"2023-10-08T01:09:08.319687+0000","last_active":"2023-10-08T01:09:09.337597+0000","last_peered":"2023-10-08T01:09:09.337597+0000","last_clean":"2023-10-08T01:09:09.337597+0000","last_became_active":"2023-10-08T01:09:08.319371+0000","last_became_peered":"2023-10-08T01:09:08.319371+0000","last_unstale":"2023-10-08T01:09:09.337597+0000","last_undegraded":"2023-10-08T01:09:09.337597+0000","last_fullsized":"2023-10-08T01:09:09.337597+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:02.224693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322457+0000","last_change":"2023-10-08T01:09:09.321527+0000","last_active":"2023-10-08T01:09:09.322457+0000","last_peered":"2023-10-08T01:09:09.322457+0000","last_clean":"2023-10-08T01:09:09.322457+0000","last_became_active":"2023-10-08T01:09:09.321236+0000","last_became_peered":"2023-10-08T01:09:09.321236+0000","last_unstale":"2023-10-08T01:09:09.322457+0000","last_undegraded":"2023-10-08T01:09:09.322457+0000","last_fullsized":"2023-10-08T01:09:09.322457+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:47.187822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.344054+0000","last_change":"2023-10-08T01:09:08.312918+0000","last_active":"2023-10-08T01:09:09.344054+0000","last_peered":"2023-10-08T01:09:09.344054+0000","last_clean":"2023-10-08T01:09:09.344054+0000","last_became_active":"2023-10-08T01:09:08.312643+0000","last_became_peered":"2023-10-08T01:09:08.312643+0000","last_unstale":"2023-10-08T01:09:09.344054+0000","last_undegraded":"2023-10-08T01:09:09.344054+0000","last_fullsized":"2023-10-08T01:09:09.344054+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:36.634257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327352+0000","last_change":"2023-10-08T01:09:09.325801+0000","last_active":"2023-10-08T01:09:09.327352+0000","last_peered":"2023-10-08T01:09:09.327352+0000","last_clean":"2023-10-08T01:09:09.327352+0000","last_became_active":"2023-10-08T01:09:09.324754+0000","last_became_peered":"2023-10-08T01:09:09.324754+0000","last_unstale":"2023-10-08T01:09:09.327352+0000","last_undegraded":"2023-10-08T01:09:09.327352+0000","last_fullsized":"2023-10-08T01:09:09.327352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:27.277557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.336271+0000","last_change":"2023-10-08T01:09:08.316194+0000","last_active":"2023-10-08T01:09:09.336271+0000","last_peered":"2023-10-08T01:09:09.336271+0000","last_clean":"2023-10-08T01:09:09.336271+0000","last_became_active":"2023-10-08T01:09:08.315901+0000","last_became_peered":"2023-10-08T01:09:08.315901+0000","last_unstale":"2023-10-08T01:09:09.336271+0000","last_undegraded":"2023-10-08T01:09:09.336271+0000","last_fullsized":"2023-10-08T01:09:09.336271+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:11.518017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322991+0000","last_change":"2023-10-08T01:09:09.316050+0000","last_active":"2023-10-08T01:09:09.322991+0000","last_peered":"2023-10-08T01:09:09.322991+0000","last_clean":"2023-10-08T01:09:09.322991+0000","last_became_active":"2023-10-08T01:09:09.315805+0000","last_became_peered":"2023-10-08T01:09:09.315805+0000","last_unstale":"2023-10-08T01:09:09.322991+0000","last_undegraded":"2023-10-08T01:09:09.322991+0000","last_fullsized":"2023-10-08T01:09:09.322991+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:59.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.345304+0000","last_change":"2023-10-08T01:09:08.315821+0000","last_active":"2023-10-08T01:09:09.345304+0000","last_peered":"2023-10-08T01:09:09.345304+0000","last_clean":"2023-10-08T01:09:09.345304+0000","last_became_active":"2023-10-08T01:09:08.315587+0000","last_became_peered":"2023-10-08T01:09:08.315587+0000","last_unstale":"2023-10-08T01:09:09.345304+0000","last_undegraded":"2023-10-08T01:09:09.345304+0000","last_fullsized":"2023-10-08T01:09:09.345304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:19.700287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.317297+0000","last_change":"2023-10-08T01:09:09.313549+0000","last_active":"2023-10-08T01:09:09.317297+0000","last_peered":"2023-10-08T01:09:09.317297+0000","last_clean":"2023-10-08T01:09:09.317297+0000","last_became_active":"2023-10-08T01:09:09.310250+0000","last_became_peered":"2023-10-08T01:09:09.310250+0000","last_unstale":"2023-10-08T01:09:09.317297+0000","last_undegraded":"2023-10-08T01:09:09.317297+0000","last_fullsized":"2023-10-08T01:09:09.317297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:23.143628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.336208+0000","last_change":"2023-10-08T01:09:08.316185+0000","last_active":"2023-10-08T01:09:09.336208+0000","last_peered":"2023-10-08T01:09:09.336208+0000","last_clean":"2023-10-08T01:09:09.336208+0000","last_became_active":"2023-10-08T01:09:08.315086+0000","last_became_peered":"2023-10-08T01:09:08.315086+0000","last_unstale":"2023-10-08T01:09:09.336208+0000","last_undegraded":"2023-10-08T01:09:09.336208+0000","last_fullsized":"2023-10-08T01:09:09.336208+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:21:17.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318915+0000","last_change":"2023-10-08T01:09:09.316377+0000","last_active":"2023-10-08T01:09:09.318915+0000","last_peered":"2023-10-08T01:09:09.318915+0000","last_clean":"2023-10-08T01:09:09.318915+0000","last_became_active":"2023-10-08T01:09:09.316118+0000","last_became_peered":"2023-10-08T01:09:09.316118+0000","last_unstale":"2023-10-08T01:09:09.318915+0000","last_undegraded":"2023-10-08T01:09:09.318915+0000","last_fullsized":"2023-10-08T01:09:09.318915+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:14:59.082595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.310530+0000","last_change":"2023-10-08T01:09:08.310728+0000","last_active":"2023-10-08T01:09:09.310530+0000","last_peered":"2023-10-08T01:09:09.310530+0000","last_clean":"2023-10-08T01:09:09.310530+0000","last_became_active":"2023-10-08T01:09:08.310445+0000","last_became_peered":"2023-10-08T01:09:08.310445+0000","last_unstale":"2023-10-08T01:09:09.310530+0000","last_undegraded":"2023-10-08T01:09:09.310530+0000","last_fullsized":"2023-10-08T01:09:09.310530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:12.204765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323956+0000","last_change":"2023-10-08T01:09:09.315344+0000","last_active":"2023-10-08T01:09:09.323956+0000","last_peered":"2023-10-08T01:09:09.323956+0000","last_clean":"2023-10-08T01:09:09.323956+0000","last_became_active":"2023-10-08T01:09:09.315024+0000","last_became_peered":"2023-10-08T01:09:09.315024+0000","last_unstale":"2023-10-08T01:09:09.323956+0000","last_undegraded":"2023-10-08T01:09:09.323956+0000","last_fullsized":"2023-10-08T01:09:09.323956+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:21:18.795393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.306506+0000","last_change":"2023-10-08T01:09:08.318464+0000","last_active":"2023-10-08T01:09:09.306506+0000","last_peered":"2023-10-08T01:09:09.306506+0000","last_clean":"2023-10-08T01:09:09.306506+0000","last_became_active":"2023-10-08T01:09:08.318159+0000","last_became_peered":"2023-10-08T01:09:08.318159+0000","last_unstale":"2023-10-08T01:09:09.306506+0000","last_undegraded":"2023-10-08T01:09:09.306506+0000","last_fullsized":"2023-10-08T01:09:09.306506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:45.525204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325511+0000","last_change":"2023-10-08T01:09:09.320974+0000","last_active":"2023-10-08T01:09:09.325511+0000","last_peered":"2023-10-08T01:09:09.325511+0000","last_clean":"2023-10-08T01:09:09.325511+0000","last_became_active":"2023-10-08T01:09:09.320375+0000","last_became_peered":"2023-10-08T01:09:09.320375+0000","last_unstale":"2023-10-08T01:09:09.325511+0000","last_undegraded":"2023-10-08T01:09:09.325511+0000","last_fullsized":"2023-10-08T01:09:09.325511+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:36.546997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.312154+0000","last_change":"2023-10-08T01:09:08.314452+0000","last_active":"2023-10-08T01:09:09.312154+0000","last_peered":"2023-10-08T01:09:09.312154+0000","last_clean":"2023-10-08T01:09:09.312154+0000","last_became_active":"2023-10-08T01:09:08.314129+0000","last_became_peered":"2023-10-08T01:09:08.314129+0000","last_unstale":"2023-10-08T01:09:09.312154+0000","last_undegraded":"2023-10-08T01:09:09.312154+0000","last_fullsized":"2023-10-08T01:09:09.312154+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:37.731362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322540+0000","last_change":"2023-10-08T01:09:09.319386+0000","last_active":"2023-10-08T01:09:09.322540+0000","last_peered":"2023-10-08T01:09:09.322540+0000","last_clean":"2023-10-08T01:09:09.322540+0000","last_became_active":"2023-10-08T01:09:09.318572+0000","last_became_peered":"2023-10-08T01:09:09.318572+0000","last_unstale":"2023-10-08T01:09:09.322540+0000","last_undegraded":"2023-10-08T01:09:09.322540+0000","last_fullsized":"2023-10-08T01:09:09.322540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:31.033623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.306937+0000","last_change":"2023-10-08T01:09:08.323551+0000","last_active":"2023-10-08T01:09:09.306937+0000","last_peered":"2023-10-08T01:09:09.306937+0000","last_clean":"2023-10-08T01:09:09.306937+0000","last_became_active":"2023-10-08T01:09:08.323246+0000","last_became_peered":"2023-10-08T01:09:08.323246+0000","last_unstale":"2023-10-08T01:09:09.306937+0000","last_undegraded":"2023-10-08T01:09:09.306937+0000","last_fullsized":"2023-10-08T01:09:09.306937+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:21.726634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.338574+0000","last_change":"2023-10-08T01:09:09.332911+0000","last_active":"2023-10-08T01:09:09.338574+0000","last_peered":"2023-10-08T01:09:09.338574+0000","last_clean":"2023-10-08T01:09:09.338574+0000","last_became_active":"2023-10-08T01:09:09.332663+0000","last_became_peered":"2023-10-08T01:09:09.332663+0000","last_unstale":"2023-10-08T01:09:09.338574+0000","last_undegraded":"2023-10-08T01:09:09.338574+0000","last_fullsized":"2023-10-08T01:09:09.338574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:07:17.048807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"58'3","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.337528+0000","last_change":"2023-10-08T01:09:08.321891+0000","last_active":"2023-10-08T01:09:09.337528+0000","last_peered":"2023-10-08T01:09:09.337528+0000","last_clean":"2023-10-08T01:09:09.337528+0000","last_became_active":"2023-10-08T01:09:08.321057+0000","last_became_peered":"2023-10-08T01:09:08.321057+0000","last_unstale":"2023-10-08T01:09:09.337528+0000","last_undegraded":"2023-10-08T01:09:09.337528+0000","last_fullsized":"2023-10-08T01:09:09.337528+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:58.460395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323443+0000","last_change":"2023-10-08T01:09:09.322019+0000","last_active":"2023-10-08T01:09:09.323443+0000","last_peered":"2023-10-08T01:09:09.323443+0000","last_clean":"2023-10-08T01:09:09.323443+0000","last_became_active":"2023-10-08T01:09:09.321726+0000","last_became_peered":"2023-10-08T01:09:09.321726+0000","last_unstale":"2023-10-08T01:09:09.323443+0000","last_undegraded":"2023-10-08T01:09:09.323443+0000","last_fullsized":"2023-10-08T01:09:09.323443+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:18.065010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"58'6","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.347108+0000","last_change":"2023-10-08T01:09:08.316773+0000","last_active":"2023-10-08T01:09:09.347108+0000","last_peered":"2023-10-08T01:09:09.347108+0000","last_clean":"2023-10-08T01:09:09.347108+0000","last_became_active":"2023-10-08T01:09:08.316557+0000","last_became_peered":"2023-10-08T01:09:08.316557+0000","last_unstale":"2023-10-08T01:09:09.347108+0000","last_undegraded":"2023-10-08T01:09:09.347108+0000","last_fullsized":"2023-10-08T01:09:09.347108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:21.726634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.324155+0000","last_change":"2023-10-08T01:09:09.319780+0000","last_active":"2023-10-08T01:09:09.324155+0000","last_peered":"2023-10-08T01:09:09.324155+0000","last_clean":"2023-10-08T01:09:09.324155+0000","last_became_active":"2023-10-08T01:09:09.319259+0000","last_became_peered":"2023-10-08T01:09:09.319259+0000","last_unstale":"2023-10-08T01:09:09.324155+0000","last_undegraded":"2023-10-08T01:09:09.324155+0000","last_fullsized":"2023-10-08T01:09:09.324155+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:38.741855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.345219+0000","last_change":"2023-10-08T01:09:08.319537+0000","last_active":"2023-10-08T01:09:09.345219+0000","last_peered":"2023-10-08T01:09:09.345219+0000","last_clean":"2023-10-08T01:09:09.345219+0000","last_became_active":"2023-10-08T01:09:08.319258+0000","last_became_peered":"2023-10-08T01:09:08.319258+0000","last_unstale":"2023-10-08T01:09:09.345219+0000","last_undegraded":"2023-10-08T01:09:09.345219+0000","last_fullsized":"2023-10-08T01:09:09.345219+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:46.177328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318960+0000","last_change":"2023-10-08T01:09:09.314911+0000","last_active":"2023-10-08T01:09:09.318960+0000","last_peered":"2023-10-08T01:09:09.318960+0000","last_clean":"2023-10-08T01:09:09.318960+0000","last_became_active":"2023-10-08T01:09:09.314631+0000","last_became_peered":"2023-10-08T01:09:09.314631+0000","last_unstale":"2023-10-08T01:09:09.318960+0000","last_undegraded":"2023-10-08T01:09:09.318960+0000","last_fullsized":"2023-10-08T01:09:09.318960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:36.546997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.303993+0000","last_change":"2023-10-08T01:09:08.315297+0000","last_active":"2023-10-08T01:09:09.303993+0000","last_peered":"2023-10-08T01:09:09.303993+0000","last_clean":"2023-10-08T01:09:09.303993+0000","last_became_active":"2023-10-08T01:09:08.315028+0000","last_became_peered":"2023-10-08T01:09:08.315028+0000","last_unstale":"2023-10-08T01:09:09.303993+0000","last_undegraded":"2023-10-08T01:09:09.303993+0000","last_fullsized":"2023-10-08T01:09:09.303993+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:07.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.341540+0000","last_change":"2023-10-08T01:09:09.333801+0000","last_active":"2023-10-08T01:09:09.341540+0000","last_peered":"2023-10-08T01:09:09.341540+0000","last_clean":"2023-10-08T01:09:09.341540+0000","last_became_active":"2023-10-08T01:09:09.333502+0000","last_became_peered":"2023-10-08T01:09:09.333502+0000","last_unstale":"2023-10-08T01:09:09.341540+0000","last_undegraded":"2023-10-08T01:09:09.341540+0000","last_fullsized":"2023-10-08T01:09:09.341540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:31.033623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.311771+0000","last_change":"2023-10-08T01:09:08.320239+0000","last_active":"2023-10-08T01:09:09.311771+0000","last_peered":"2023-10-08T01:09:09.311771+0000","last_clean":"2023-10-08T01:09:09.311771+0000","last_became_active":"2023-10-08T01:09:08.319955+0000","last_became_peered":"2023-10-08T01:09:08.319955+0000","last_unstale":"2023-10-08T01:09:09.311771+0000","last_undegraded":"2023-10-08T01:09:09.311771+0000","last_fullsized":"2023-10-08T01:09:09.311771+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:35.536504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.317547+0000","last_change":"2023-10-08T01:09:09.313493+0000","last_active":"2023-10-08T01:09:09.317547+0000","last_peered":"2023-10-08T01:09:09.317547+0000","last_clean":"2023-10-08T01:09:09.317547+0000","last_became_active":"2023-10-08T01:09:09.313229+0000","last_became_peered":"2023-10-08T01:09:09.313229+0000","last_unstale":"2023-10-08T01:09:09.317547+0000","last_undegraded":"2023-10-08T01:09:09.317547+0000","last_fullsized":"2023-10-08T01:09:09.317547+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:17.244899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"58'8","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.344025+0000","last_change":"2023-10-08T01:09:08.317533+0000","last_active":"2023-10-08T01:09:09.344025+0000","last_peered":"2023-10-08T01:09:09.344025+0000","last_clean":"2023-10-08T01:09:09.344025+0000","last_became_active":"2023-10-08T01:09:08.317259+0000","last_became_peered":"2023-10-08T01:09:08.317259+0000","last_unstale":"2023-10-08T01:09:09.344025+0000","last_undegraded":"2023-10-08T01:09:09.344025+0000","last_fullsized":"2023-10-08T01:09:09.344025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:56:19.417230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.326851+0000","last_change":"2023-10-08T01:09:09.324109+0000","last_active":"2023-10-08T01:09:09.326851+0000","last_peered":"2023-10-08T01:09:09.326851+0000","last_clean":"2023-10-08T01:09:09.326851+0000","last_became_active":"2023-10-08T01:09:09.323838+0000","last_became_peered":"2023-10-08T01:09:09.323838+0000","last_unstale":"2023-10-08T01:09:09.326851+0000","last_undegraded":"2023-10-08T01:09:09.326851+0000","last_fullsized":"2023-10-08T01:09:09.326851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:38.741855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.342258+0000","last_change":"2023-10-08T01:09:08.312702+0000","last_active":"2023-10-08T01:09:09.342258+0000","last_peered":"2023-10-08T01:09:09.342258+0000","last_clean":"2023-10-08T01:09:09.342258+0000","last_became_active":"2023-10-08T01:09:08.312233+0000","last_became_peered":"2023-10-08T01:09:08.312233+0000","last_unstale":"2023-10-08T01:09:09.342258+0000","last_undegraded":"2023-10-08T01:09:09.342258+0000","last_fullsized":"2023-10-08T01:09:09.342258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:38.858148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322757+0000","last_change":"2023-10-08T01:09:09.318140+0000","last_active":"2023-10-08T01:09:09.322757+0000","last_peered":"2023-10-08T01:09:09.322757+0000","last_clean":"2023-10-08T01:09:09.322757+0000","last_became_active":"2023-10-08T01:09:09.317668+0000","last_became_peered":"2023-10-08T01:09:09.317668+0000","last_unstale":"2023-10-08T01:09:09.322757+0000","last_undegraded":"2023-10-08T01:09:09.322757+0000","last_fullsized":"2023-10-08T01:09:09.322757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:37.644751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.338725+0000","last_change":"2023-10-08T01:09:08.318961+0000","last_active":"2023-10-08T01:09:09.338725+0000","last_peered":"2023-10-08T01:09:09.338725+0000","last_clean":"2023-10-08T01:09:09.338725+0000","last_became_active":"2023-10-08T01:09:08.318478+0000","last_became_peered":"2023-10-08T01:09:08.318478+0000","last_unstale":"2023-10-08T01:09:09.338725+0000","last_undegraded":"2023-10-08T01:09:09.338725+0000","last_fullsized":"2023-10-08T01:09:09.338725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:50.502468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327200+0000","last_change":"2023-10-08T01:09:09.323492+0000","last_active":"2023-10-08T01:09:09.327200+0000","last_peered":"2023-10-08T01:09:09.327200+0000","last_clean":"2023-10-08T01:09:09.327200+0000","last_became_active":"2023-10-08T01:09:09.323107+0000","last_became_peered":"2023-10-08T01:09:09.323107+0000","last_unstale":"2023-10-08T01:09:09.327200+0000","last_undegraded":"2023-10-08T01:09:09.327200+0000","last_fullsized":"2023-10-08T01:09:09.327200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:35.306878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.338558+0000","last_change":"2023-10-08T01:09:08.311946+0000","last_active":"2023-10-08T01:09:09.338558+0000","last_peered":"2023-10-08T01:09:09.338558+0000","last_clean":"2023-10-08T01:09:09.338558+0000","last_became_active":"2023-10-08T01:09:08.311697+0000","last_became_peered":"2023-10-08T01:09:08.311697+0000","last_unstale":"2023-10-08T01:09:09.338558+0000","last_undegraded":"2023-10-08T01:09:09.338558+0000","last_fullsized":"2023-10-08T01:09:09.338558+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:22.133135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327299+0000","last_change":"2023-10-08T01:09:09.324004+0000","last_active":"2023-10-08T01:09:09.327299+0000","last_peered":"2023-10-08T01:09:09.327299+0000","last_clean":"2023-10-08T01:09:09.327299+0000","last_became_active":"2023-10-08T01:09:09.323701+0000","last_became_peered":"2023-10-08T01:09:09.323701+0000","last_unstale":"2023-10-08T01:09:09.327299+0000","last_undegraded":"2023-10-08T01:09:09.327299+0000","last_fullsized":"2023-10-08T01:09:09.327299+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:39.868642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.313507+0000","last_change":"2023-10-08T01:09:08.314277+0000","last_active":"2023-10-08T01:09:09.313507+0000","last_peered":"2023-10-08T01:09:09.313507+0000","last_clean":"2023-10-08T01:09:09.313507+0000","last_became_active":"2023-10-08T01:09:08.314019+0000","last_became_peered":"2023-10-08T01:09:08.314019+0000","last_unstale":"2023-10-08T01:09:09.313507+0000","last_undegraded":"2023-10-08T01:09:09.313507+0000","last_fullsized":"2023-10-08T01:09:09.313507+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:07.563912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.336575+0000","last_change":"2023-10-08T01:09:09.330962+0000","last_active":"2023-10-08T01:09:09.336575+0000","last_peered":"2023-10-08T01:09:09.336575+0000","last_clean":"2023-10-08T01:09:09.336575+0000","last_became_active":"2023-10-08T01:09:09.330718+0000","last_became_peered":"2023-10-08T01:09:09.330718+0000","last_unstale":"2023-10-08T01:09:09.336575+0000","last_undegraded":"2023-10-08T01:09:09.336575+0000","last_fullsized":"2023-10-08T01:09:09.336575+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:36.035056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.345231+0000","last_change":"2023-10-08T01:09:08.315258+0000","last_active":"2023-10-08T01:09:09.345231+0000","last_peered":"2023-10-08T01:09:09.345231+0000","last_clean":"2023-10-08T01:09:09.345231+0000","last_became_active":"2023-10-08T01:09:08.315017+0000","last_became_peered":"2023-10-08T01:09:08.315017+0000","last_unstale":"2023-10-08T01:09:09.345231+0000","last_undegraded":"2023-10-08T01:09:09.345231+0000","last_fullsized":"2023-10-08T01:09:09.345231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:07.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.328948+0000","last_change":"2023-10-08T01:09:09.326623+0000","last_active":"2023-10-08T01:09:09.328948+0000","last_peered":"2023-10-08T01:09:09.328948+0000","last_clean":"2023-10-08T01:09:09.328948+0000","last_became_active":"2023-10-08T01:09:09.325498+0000","last_became_peered":"2023-10-08T01:09:09.325498+0000","last_unstale":"2023-10-08T01:09:09.328948+0000","last_undegraded":"2023-10-08T01:09:09.328948+0000","last_fullsized":"2023-10-08T01:09:09.328948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:23.143628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.310716+0000","last_change":"2023-10-08T01:09:08.306175+0000","last_active":"2023-10-08T01:09:09.310716+0000","last_peered":"2023-10-08T01:09:09.310716+0000","last_clean":"2023-10-08T01:09:09.310716+0000","last_became_active":"2023-10-08T01:09:08.305878+0000","last_became_peered":"2023-10-08T01:09:08.305878+0000","last_unstale":"2023-10-08T01:09:09.310716+0000","last_undegraded":"2023-10-08T01:09:09.310716+0000","last_fullsized":"2023-10-08T01:09:09.310716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:14:58.072102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.320402+0000","last_change":"2023-10-08T01:09:09.319509+0000","last_active":"2023-10-08T01:09:09.320402+0000","last_peered":"2023-10-08T01:09:09.320402+0000","last_clean":"2023-10-08T01:09:09.320402+0000","last_became_active":"2023-10-08T01:09:09.319182+0000","last_became_peered":"2023-10-08T01:09:09.319182+0000","last_unstale":"2023-10-08T01:09:09.320402+0000","last_undegraded":"2023-10-08T01:09:09.320402+0000","last_fullsized":"2023-10-08T01:09:09.320402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:21:18.795393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.305588+0000","last_change":"2023-10-08T01:09:08.321444+0000","last_active":"2023-10-08T01:09:09.305588+0000","last_peered":"2023-10-08T01:09:09.305588+0000","last_clean":"2023-10-08T01:09:09.305588+0000","last_became_active":"2023-10-08T01:09:08.321107+0000","last_became_peered":"2023-10-08T01:09:08.321107+0000","last_unstale":"2023-10-08T01:09:09.305588+0000","last_undegraded":"2023-10-08T01:09:09.305588+0000","last_fullsized":"2023-10-08T01:09:09.305588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:07.038593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.331130+0000","last_change":"2023-10-08T01:09:09.328674+0000","last_active":"2023-10-08T01:09:09.331130+0000","last_peered":"2023-10-08T01:09:09.331130+0000","last_clean":"2023-10-08T01:09:09.331130+0000","last_became_active":"2023-10-08T01:09:09.328357+0000","last_became_peered":"2023-10-08T01:09:09.328357+0000","last_unstale":"2023-10-08T01:09:09.331130+0000","last_undegraded":"2023-10-08T01:09:09.331130+0000","last_fullsized":"2023-10-08T01:09:09.331130+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:20.710780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.303058+0000","last_change":"2023-10-08T01:09:08.315463+0000","last_active":"2023-10-08T01:09:09.303058+0000","last_peered":"2023-10-08T01:09:09.303058+0000","last_clean":"2023-10-08T01:09:09.303058+0000","last_became_active":"2023-10-08T01:09:08.315193+0000","last_became_peered":"2023-10-08T01:09:08.315193+0000","last_unstale":"2023-10-08T01:09:09.303058+0000","last_undegraded":"2023-10-08T01:09:09.303058+0000","last_fullsized":"2023-10-08T01:09:09.303058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:19.700287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325975+0000","last_change":"2023-10-08T01:09:09.320956+0000","last_active":"2023-10-08T01:09:09.325975+0000","last_peered":"2023-10-08T01:09:09.325975+0000","last_clean":"2023-10-08T01:09:09.325975+0000","last_became_active":"2023-10-08T01:09:09.320592+0000","last_became_peered":"2023-10-08T01:09:09.320592+0000","last_unstale":"2023-10-08T01:09:09.325975+0000","last_undegraded":"2023-10-08T01:09:09.325975+0000","last_fullsized":"2023-10-08T01:09:09.325975+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:40.558251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.314033+0000","last_change":"2023-10-08T01:09:08.319445+0000","last_active":"2023-10-08T01:09:09.314033+0000","last_peered":"2023-10-08T01:09:09.314033+0000","last_clean":"2023-10-08T01:09:09.314033+0000","last_became_active":"2023-10-08T01:09:08.319158+0000","last_became_peered":"2023-10-08T01:09:08.319158+0000","last_unstale":"2023-10-08T01:09:09.314033+0000","last_undegraded":"2023-10-08T01:09:09.314033+0000","last_fullsized":"2023-10-08T01:09:09.314033+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:59.432265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323304+0000","last_change":"2023-10-08T01:09:09.319198+0000","last_active":"2023-10-08T01:09:09.323304+0000","last_peered":"2023-10-08T01:09:09.323304+0000","last_clean":"2023-10-08T01:09:09.323304+0000","last_became_active":"2023-10-08T01:09:09.318952+0000","last_became_peered":"2023-10-08T01:09:09.318952+0000","last_unstale":"2023-10-08T01:09:09.323304+0000","last_undegraded":"2023-10-08T01:09:09.323304+0000","last_fullsized":"2023-10-08T01:09:09.323304+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:15:00.442758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.305085+0000","last_change":"2023-10-08T01:09:08.322197+0000","last_active":"2023-10-08T01:09:09.305085+0000","last_peered":"2023-10-08T01:09:09.305085+0000","last_clean":"2023-10-08T01:09:09.305085+0000","last_became_active":"2023-10-08T01:09:08.321567+0000","last_became_peered":"2023-10-08T01:09:08.321567+0000","last_unstale":"2023-10-08T01:09:09.305085+0000","last_undegraded":"2023-10-08T01:09:09.305085+0000","last_fullsized":"2023-10-08T01:09:09.305085+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:59.432265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.314587+0000","last_change":"2023-10-08T01:09:09.310737+0000","last_active":"2023-10-08T01:09:09.314587+0000","last_peered":"2023-10-08T01:09:09.314587+0000","last_clean":"2023-10-08T01:09:09.314587+0000","last_became_active":"2023-10-08T01:09:09.310389+0000","last_became_peered":"2023-10-08T01:09:09.310389+0000","last_unstale":"2023-10-08T01:09:09.314587+0000","last_undegraded":"2023-10-08T01:09:09.314587+0000","last_fullsized":"2023-10-08T01:09:09.314587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:50:36.089765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.310560+0000","last_change":"2023-10-08T01:09:08.308607+0000","last_active":"2023-10-08T01:09:09.310560+0000","last_peered":"2023-10-08T01:09:09.310560+0000","last_clean":"2023-10-08T01:09:09.310560+0000","last_became_active":"2023-10-08T01:09:08.308305+0000","last_became_peered":"2023-10-08T01:09:08.308305+0000","last_unstale":"2023-10-08T01:09:09.310560+0000","last_undegraded":"2023-10-08T01:09:09.310560+0000","last_fullsized":"2023-10-08T01:09:09.310560+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:21:17.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.329617+0000","last_change":"2023-10-08T01:09:09.326662+0000","last_active":"2023-10-08T01:09:09.329617+0000","last_peered":"2023-10-08T01:09:09.329617+0000","last_clean":"2023-10-08T01:09:09.329617+0000","last_became_active":"2023-10-08T01:09:09.326407+0000","last_became_peered":"2023-10-08T01:09:09.326407+0000","last_unstale":"2023-10-08T01:09:09.329617+0000","last_undegraded":"2023-10-08T01:09:09.329617+0000","last_fullsized":"2023-10-08T01:09:09.329617+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:01.286642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.305324+0000","last_change":"2023-10-08T01:09:08.313396+0000","last_active":"2023-10-08T01:09:09.305324+0000","last_peered":"2023-10-08T01:09:09.305324+0000","last_clean":"2023-10-08T01:09:09.305324+0000","last_became_active":"2023-10-08T01:09:08.313087+0000","last_became_peered":"2023-10-08T01:09:08.313087+0000","last_unstale":"2023-10-08T01:09:09.305324+0000","last_undegraded":"2023-10-08T01:09:09.305324+0000","last_fullsized":"2023-10-08T01:09:09.305324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:07.563912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.332594+0000","last_change":"2023-10-08T01:09:09.330360+0000","last_active":"2023-10-08T01:09:09.332594+0000","last_peered":"2023-10-08T01:09:09.332594+0000","last_clean":"2023-10-08T01:09:09.332594+0000","last_became_active":"2023-10-08T01:09:09.330100+0000","last_became_peered":"2023-10-08T01:09:09.330100+0000","last_unstale":"2023-10-08T01:09:09.332594+0000","last_undegraded":"2023-10-08T01:09:09.332594+0000","last_fullsized":"2023-10-08T01:09:09.332594+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:41.537708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.309062+0000","last_change":"2023-10-08T01:09:08.319584+0000","last_active":"2023-10-08T01:09:09.309062+0000","last_peered":"2023-10-08T01:09:09.309062+0000","last_clean":"2023-10-08T01:09:09.309062+0000","last_became_active":"2023-10-08T01:09:08.319313+0000","last_became_peered":"2023-10-08T01:09:08.319313+0000","last_unstale":"2023-10-08T01:09:09.309062+0000","last_undegraded":"2023-10-08T01:09:09.309062+0000","last_fullsized":"2023-10-08T01:09:09.309062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:37.522501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327358+0000","last_change":"2023-10-08T01:09:09.326359+0000","last_active":"2023-10-08T01:09:09.327358+0000","last_peered":"2023-10-08T01:09:09.327358+0000","last_clean":"2023-10-08T01:09:09.327358+0000","last_became_active":"2023-10-08T01:09:09.326071+0000","last_became_peered":"2023-10-08T01:09:09.326071+0000","last_unstale":"2023-10-08T01:09:09.327358+0000","last_undegraded":"2023-10-08T01:09:09.327358+0000","last_fullsized":"2023-10-08T01:09:09.327358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:36.546997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.310602+0000","last_change":"2023-10-08T01:09:08.310261+0000","last_active":"2023-10-08T01:09:09.310602+0000","last_peered":"2023-10-08T01:09:09.310602+0000","last_clean":"2023-10-08T01:09:09.310602+0000","last_became_active":"2023-10-08T01:09:08.309985+0000","last_became_peered":"2023-10-08T01:09:08.309985+0000","last_unstale":"2023-10-08T01:09:09.310602+0000","last_undegraded":"2023-10-08T01:09:09.310602+0000","last_fullsized":"2023-10-08T01:09:09.310602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:50.502468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323376+0000","last_change":"2023-10-08T01:09:09.318228+0000","last_active":"2023-10-08T01:09:09.323376+0000","last_peered":"2023-10-08T01:09:09.323376+0000","last_clean":"2023-10-08T01:09:09.323376+0000","last_became_active":"2023-10-08T01:09:09.317951+0000","last_became_peered":"2023-10-08T01:09:09.317951+0000","last_unstale":"2023-10-08T01:09:09.323376+0000","last_undegraded":"2023-10-08T01:09:09.323376+0000","last_fullsized":"2023-10-08T01:09:09.323376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:20.710780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.306503+0000","last_change":"2023-10-08T01:09:08.314603+0000","last_active":"2023-10-08T01:09:09.306503+0000","last_peered":"2023-10-08T01:09:09.306503+0000","last_clean":"2023-10-08T01:09:09.306503+0000","last_became_active":"2023-10-08T01:09:08.311835+0000","last_became_peered":"2023-10-08T01:09:08.311835+0000","last_unstale":"2023-10-08T01:09:09.306503+0000","last_undegraded":"2023-10-08T01:09:09.306503+0000","last_fullsized":"2023-10-08T01:09:09.306503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:05:44.356964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.319131+0000","last_change":"2023-10-08T01:09:09.315799+0000","last_active":"2023-10-08T01:09:09.319131+0000","last_peered":"2023-10-08T01:09:09.319131+0000","last_clean":"2023-10-08T01:09:09.319131+0000","last_became_active":"2023-10-08T01:09:09.315509+0000","last_became_peered":"2023-10-08T01:09:09.315509+0000","last_unstale":"2023-10-08T01:09:09.319131+0000","last_undegraded":"2023-10-08T01:09:09.319131+0000","last_fullsized":"2023-10-08T01:09:09.319131+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:47.187822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.342448+0000","last_change":"2023-10-08T01:09:08.314174+0000","last_active":"2023-10-08T01:09:09.342448+0000","last_peered":"2023-10-08T01:09:09.342448+0000","last_clean":"2023-10-08T01:09:09.342448+0000","last_became_active":"2023-10-08T01:09:08.313917+0000","last_became_peered":"2023-10-08T01:09:08.313917+0000","last_unstale":"2023-10-08T01:09:09.342448+0000","last_undegraded":"2023-10-08T01:09:09.342448+0000","last_fullsized":"2023-10-08T01:09:09.342448+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:26.267064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"48'139","reported_seq":276,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.303160+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:09:09.303160+0000","last_peered":"2023-10-08T01:09:09.303160+0000","last_clean":"2023-10-08T01:09:09.303160+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:09:09.303160+0000","last_undegraded":"2023-10-08T01:09:09.303160+0000","last_fullsized":"2023-10-08T01:09:09.303160+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575013,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302652,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069188,"statfs":{"total":96636764160,"available":96326848512,"internally_reserved":0,"allocated":1298432,"data_stored":917197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90100000000000002,"5min":0.996,"15min":0.996},"last":0.84999999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.87,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79800000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82899999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86799999999999999,"5min":1.054,"15min":1.054},"last":0.93000000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.91600000000000004,"15min":0.998},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.57499999999999996,"5min":0.52000000000000002,"15min":0.50600000000000001},"max":{"1min":0.95299999999999996,"5min":1.03,"15min":1.03},"last":0.94599999999999995}]},{"osd":4,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.44,"5min":0.439,"15min":0.40899999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.0780000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65500000000000003,"15min":0.64800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40000000000000002},"max":{"1min":0.86899999999999999,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":1.01,"15min":1.1259999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.70999999999999996},"min":{"1min":0.50600000000000001,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.80600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575013,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302064,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069776,"statfs":{"total":96636764160,"available":96327450624,"internally_reserved":0,"allocated":696320,"data_stored":320731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.872,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.68799999999999994,"15min":0.69299999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.88700000000000001,"15min":1.093},"last":0.47199999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":2.984},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70599999999999996,"15min":0.72699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.96899999999999997,"15min":3.0190000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84399999999999997,"5min":0.85499999999999998,"15min":1.9199999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":0.73799999999999999},"min":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.4810000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.621},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.75900000000000001,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.55800000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":1.163},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.361},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63400000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68100000000000005,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":1.21,"5min":1.21,"15min":1.4119999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.73799999999999999},"min":{"1min":0.501,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.88,"5min":0.98699999999999999,"15min":2.9489999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.749,"15min":0.747},"min":{"1min":0.47099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302072,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069768,"statfs":{"total":96636764160,"available":96327442432,"internally_reserved":0,"allocated":704512,"data_stored":326809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.47699999999999998,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.78900000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51600000000000001,"15min":0.495},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.28199999999999997},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.58899999999999997,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.37},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.66100000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.437},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.312},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.50600000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.661},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.51900000000000002,"5min":0.45100000000000001,"15min":0.434},"max":{"1min":1.069,"5min":1.069,"15min":1.5569999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.409},"last":0.875},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.617},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.53300000000000003},"max":{"1min":1.002,"5min":1.1020000000000001,"15min":1.4790000000000001},"last":0.83399999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84299999999999997,"15min":0.82999999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.84499999999999997,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.86399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298544,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073296,"statfs":{"total":96636764160,"available":96331055104,"internally_reserved":0,"allocated":1286144,"data_stored":910489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.315},"max":{"1min":0.64400000000000002,"5min":0.73999999999999999,"15min":1.238},"last":0.36799999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.47799999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.439},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.69999999999999996,"15min":1.7370000000000001},"last":0.41399999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.0489999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41799999999999998},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.503}]},{"osd":4,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.755,"15min":0.754},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.45100000000000001},"max":{"1min":1.0009999999999999,"5min":1.1899999999999999,"15min":1.617},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.48899999999999999},"max":{"1min":0.95499999999999996,"5min":1.0700000000000001,"15min":1.4099999999999999},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.51000000000000001},"max":{"1min":1.2729999999999999,"5min":1.3440000000000001,"15min":1.5960000000000001},"last":0.745},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.302,"15min":1.5289999999999999},"last":0.73799999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.46600000000000003},"max":{"1min":1.0129999999999999,"5min":1.165,"15min":1.3779999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51200000000000001},"max":{"1min":1.2390000000000001,"5min":1.2549999999999999,"15min":1.6459999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":1.0349999999999999,"5min":1.071,"15min":1.5740000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.84299999999999997,"15min":0.83399999999999996},"min":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.008,"5min":1.222,"15min":1.3160000000000001},"last":0.86599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306180,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065660,"statfs":{"total":96636764160,"available":96323235840,"internally_reserved":0,"allocated":716800,"data_stored":328297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":1.2689999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52100000000000002,"15min":0.501},"min":{"1min":0.318,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.33800000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.505,"15min":0.47899999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.28399999999999997},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60899999999999999,"15min":0.59599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76100000000000001,"15min":0.76600000000000001},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.079,"5min":1.079,"15min":1.1180000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.78800000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.755}]},{"osd":6,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80800000000000005,"15min":0.79500000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.79800000000000004,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80200000000000005,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.90400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306176,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065664,"statfs":{"total":96636764160,"available":96323239936,"internally_reserved":0,"allocated":712704,"data_stored":327349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47299999999999998,"15min":0.45800000000000002},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.249},"max":{"1min":0.69999999999999996,"5min":0.77600000000000002,"15min":0.80100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.86599999999999999},"last":0.503}]},{"osd":1,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46899999999999997,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66400000000000003,"5min":0.872,"15min":1.5629999999999999},"last":0.69199999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.38700000000000001,"15min":0.34100000000000003},"max":{"1min":0.60099999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63700000000000001,"5min":0.89000000000000001,"15min":1.3899999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.52400000000000002,"5min":0.5,"15min":0.46000000000000002},"max":{"1min":1.117,"5min":1.413,"15min":1.413},"last":0.73299999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.1899999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.94399999999999995,"5min":1.089,"15min":1.431},"last":0.74299999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.012,"5min":1.379,"15min":1.458},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.80000000000000004},"min":{"1min":0.58299999999999996,"5min":0.505,"15min":0.44500000000000001},"max":{"1min":1.1499999999999999,"5min":1.486,"15min":1.52},"last":0.84299999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.042,"5min":1.389,"15min":1.389},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80500000000000005},"min":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.5},"max":{"1min":0.97199999999999998,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.80100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306176,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065664,"statfs":{"total":96636764160,"available":96323239936,"internally_reserved":0,"allocated":712704,"data_stored":328275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.68300000000000005},"min":{"1min":0.56999999999999995,"5min":0.35599999999999998,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.91900000000000004,"15min":0.95599999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.73299999999999998,"15min":0.68600000000000005},"min":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.36599999999999999},"max":{"1min":0.94799999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66400000000000003}]},{"osd":1,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72299999999999998,"15min":0.68600000000000005},"min":{"1min":0.59099999999999997,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93200000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.74099999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.77800000000000002,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.502,"15min":0.45600000000000002},"max":{"1min":0.91000000000000003,"5min":1.002,"15min":1.024},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79100000000000004,"15min":0.76100000000000001},"min":{"1min":0.68799999999999994,"5min":0.56399999999999995,"15min":0.45400000000000001},"max":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70999999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79400000000000004,"15min":0.77500000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47999999999999998},"max":{"1min":0.99099999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.82299999999999995,"15min":0.79400000000000004},"min":{"1min":0.64000000000000001,"5min":0.53900000000000003,"15min":0.51800000000000002},"max":{"1min":0.91200000000000003,"5min":1.454,"15min":1.454},"last":0.77000000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.65900000000000003},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.95299999999999996,"15min":1.1459999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.42399999999999999},"max":{"1min":0.86799999999999999,"5min":0.998,"15min":1.3420000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65000000000000002,"15min":0.67800000000000005},"min":{"1min":0.47499999999999998,"5min":0.433,"15min":0.40799999999999997},"max":{"1min":0.88,"5min":0.97699999999999998,"15min":1.1140000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65700000000000003,"15min":0.70299999999999996},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":1.206},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67500000000000004,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.97699999999999998,"15min":1.5209999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.75900000000000001},"min":{"1min":0.51000000000000001,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.86499999999999999,"5min":1.0049999999999999,"15min":1.26},"last":0.79900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302060,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069780,"statfs":{"total":96636764160,"available":96327454720,"internally_reserved":0,"allocated":692224,"data_stored":320157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.499,"5min":0.42099999999999999,"15min":0.33700000000000002},"max":{"1min":0.874,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73199999999999998,"15min":0.70699999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.38},"max":{"1min":0.85499999999999998,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70699999999999996,"15min":0.71099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.01},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.626,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81000000000000005,"5min":0.92100000000000004,"15min":1.077},"last":0.68500000000000005}]},{"osd":2,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71899999999999997,"15min":0.71099999999999997},"min":{"1min":0.52400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.72599999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.90100000000000002,"5min":0.996,"15min":1.0449999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91800000000000004,"5min":0.94399999999999995,"15min":1.1339999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.74199999999999999,"15min":0.753},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93000000000000005,"5min":0.95099999999999996,"15min":1.107},"last":0.622}]},{"osd":4,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59599999999999997,"15min":0.625},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":1.042},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.51200000000000001,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.081},"last":0.71099999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65800000000000003,"15min":0.68100000000000005},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.40699999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.80700000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.59999999999999998,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.165},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.749},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.3879999999999999},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:09:13.632 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:09:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.845+0000 7fef44fff700 1 Processor -- start 2023-10-08T01:09:13.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.845+0000 7fef44fff700 1 -- start start 2023-10-08T01:09:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef44fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fef400b0a10 0x7fef40155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef44fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef400ad930 con 0x7fef401578b0 2023-10-08T01:09:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef44fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef400acd20 con 0x7fef40155c10 2023-10-08T01:09:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef44fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef400ad000 con 0x7fef400b0a10 2023-10-08T01:09:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef37fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fef401578b0 0x7fef400af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44484/0 (socket says 172.21.15.119:44484) 2023-10-08T01:09:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef37fff700 1 -- 172.21.15.119:0/2258847314 learned_addr learned my addr 172.21.15.119:0/2258847314 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:13.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef3ed9d700 1 --2- 172.21.15.119:0/2258847314 >> v2:172.21.15.184:3300/0 conn(0x7fef400b0a10 0x7fef40155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435065050 0 0) 0x7fef400ad930 con 0x7fef401578b0 2023-10-08T01:09:13.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.847+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef2c003570 con 0x7fef401578b0 2023-10-08T01:09:13.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983175685 0 0) 0x7fef2c003570 con 0x7fef401578b0 2023-10-08T01:09:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 >> v2:172.21.15.184:3300/0 conn(0x7fef400b0a10 msgr2=0x7fef40155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 --2- 172.21.15.119:0/2258847314 >> v2:172.21.15.184:3300/0 conn(0x7fef400b0a10 0x7fef40155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 >> v1:172.21.15.184:6789/0 conn(0x7fef40155c10 legacy=0x7fef400ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef400ad2e0 con 0x7fef401578b0 2023-10-08T01:09:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fef28001470 con 0x7fef401578b0 2023-10-08T01:09:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 >> v1:172.21.15.119:6789/0 conn(0x7fef401578b0 legacy=0x7fef400af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c003ab0 0x7fef2c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2c004810 0x7fef2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2c005210 0x7fef2c00cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef400ad000 con 0x7fef2c004810 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c00d240 con 0x7fef2c003ab0 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef2c00d400 con 0x7fef2c005210 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3ed9d700 1 --2- 172.21.15.119:0/2258847314 >> v2:172.21.15.184:3300/0 conn(0x7fef400b0a10 0x7fef40155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef37fff700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2c004810 0x7fef2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3e59c700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c003ab0 0x7fef2c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef3ed9d700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2c005210 0x7fef2c00cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef37fff700 1 -- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2c005210 msgr2=0x7fef2c00cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:13.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef37fff700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef2c005210 0x7fef2c00cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef37fff700 1 -- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c003ab0 msgr2=0x7fef2c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:13.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef37fff700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c003ab0 0x7fef2c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.848+0000 7fef37fff700 1 -- 172.21.15.119:0/2258847314 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c003570 con 0x7fef2c004810 2023-10-08T01:09:13.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.849+0000 7fef3ed9d700 1 -- 172.21.15.119:0/2258847314 reap_dead start 2023-10-08T01:09:13.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.849+0000 7fef3e59c700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c003ab0 0x7fef2c003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:09:13.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.849+0000 7fef37fff700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2c004810 0x7fef2c004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fef28007bf0 tx=0x7fef28001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98eb5646a164c33 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:13.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.849+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2800cb70 con 0x7fef2c004810 2023-10-08T01:09:13.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.849+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2800ccf0 con 0x7fef2c004810 2023-10-08T01:09:13.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.849+0000 7fef3dd9b700 1 -- 172.21.15.119:0/2258847314 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28005b70 con 0x7fef2c004810 2023-10-08T01:09:13.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.849+0000 7fef44fff700 1 -- 172.21.15.119:0/2258847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2c004810 msgr2=0x7fef2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.849+0000 7fef44fff700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2c004810 0x7fef2c004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fef28007bf0 tx=0x7fef28001d80 comp rx=0 tx=0).stop 2023-10-08T01:09:13.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.850+0000 7fef44fff700 1 -- 172.21.15.119:0/2258847314 shutdown_connections 2023-10-08T01:09:13.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.850+0000 7fef44fff700 1 --2- 172.21.15.119:0/2258847314 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef2c004810 0x7fef2c004c80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.850+0000 7fef44fff700 1 -- 172.21.15.119:0/2258847314 >> 172.21.15.119:0/2258847314 conn(0x7fef4006a320 msgr2=0x7fef400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:13.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.850+0000 7fef44fff700 1 -- 172.21.15.119:0/2258847314 shutdown_connections 2023-10-08T01:09:13.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.850+0000 7fef44fff700 1 -- 172.21.15.119:0/2258847314 wait complete. 2023-10-08T01:09:13.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.851+0000 7fef44fff700 1 Processor -- start 2023-10-08T01:09:13.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.851+0000 7fef44fff700 1 -- start start 2023-10-08T01:09:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef44fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c004810 0x7fef40166450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef44fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40166bf0 0x7fef40150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef44fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40168470 0x7fef401688b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef44fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40168df0 con 0x7fef40166bf0 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef44fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef40158f50 con 0x7fef2c004810 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef44fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef40166990 con 0x7fef40168470 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef37fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40166bf0 0x7fef40150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef37fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40166bf0 0x7fef40150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55230/0 (socket says 172.21.15.119:55230) 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef37fff700 1 -- 172.21.15.119:0/847547888 learned_addr learned my addr 172.21.15.119:0/847547888 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef3e59c700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c004810 0x7fef40166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.852+0000 7fef3ed9d700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40168470 0x7fef401688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef37fff700 1 -- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40168470 msgr2=0x7fef401688b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef37fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40168470 0x7fef401688b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef37fff700 1 -- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c004810 msgr2=0x7fef40166450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:13.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef37fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c004810 0x7fef40166450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef37fff700 1 -- 172.21.15.119:0/847547888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef4014eec0 con 0x7fef40166bf0 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef3ed9d700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40168470 0x7fef401688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef37fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40166bf0 0x7fef40150d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fef28000f50 tx=0x7fef28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef367fc700 1 -- 172.21.15.119:0/847547888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2800ce60 con 0x7fef40166bf0 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef367fc700 1 -- 172.21.15.119:0/847547888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef280070d0 con 0x7fef40166bf0 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef367fc700 1 -- 172.21.15.119:0/847547888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef28007490 con 0x7fef40166bf0 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef4014f1a0 con 0x7fef40166bf0 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.853+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef401594d0 con 0x7fef40166bf0 2023-10-08T01:09:13.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.854+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef4004c3c0 con 0x7fef40166bf0 2023-10-08T01:09:13.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.855+0000 7fef367fc700 1 -- 172.21.15.119:0/847547888 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fef28011470 con 0x7fef40166bf0 2023-10-08T01:09:13.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.856+0000 7fef367fc700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fef20066f30 0x7fef200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:13.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.856+0000 7fef367fc700 1 -- 172.21.15.119:0/847547888 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fef28087720 con 0x7fef40166bf0 2023-10-08T01:09:13.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.856+0000 7fef3e59c700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fef20066f30 0x7fef200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:13.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.857+0000 7fef3e59c700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fef20066f30 0x7fef200693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fef40061160 tx=0x7fef30009040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:13.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:13.860+0000 7fef367fc700 1 -- 172.21.15.119:0/847547888 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fef28054830 con 0x7fef40166bf0 2023-10-08T01:09:14.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.049+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 --> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef400ad5c0 con 0x7fef20066f30 2023-10-08T01:09:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.057+0000 7fef367fc700 1 -- 172.21.15.119:0/847547888 <== mgr.5571 v2:172.21.15.119:6834/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288968 (secure 0 0 0) 0x7fef400ad5c0 con 0x7fef20066f30 2023-10-08T01:09:14.065 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fef20066f30 msgr2=0x7fef200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fef20066f30 0x7fef200693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fef40061160 tx=0x7fef30009040 comp rx=0 tx=0).stop 2023-10-08T01:09:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40166bf0 msgr2=0x7fef40150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40166bf0 0x7fef40150d50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fef28000f50 tx=0x7fef28017000 comp rx=0 tx=0).stop 2023-10-08T01:09:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 shutdown_connections 2023-10-08T01:09:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fef20066f30 0x7fef200693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fef40168470 0x7fef401688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fef2c004810 0x7fef40166450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 --2- 172.21.15.119:0/847547888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef40166bf0 0x7fef40150d50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 >> 172.21.15.119:0/847547888 conn(0x7fef4006a320 msgr2=0x7fef4014a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 shutdown_connections 2023-10-08T01:09:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.062+0000 7fef44fff700 1 -- 172.21.15.119:0/847547888 wait complete. 2023-10-08T01:09:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:09:14.081 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T01:09:12.582001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":92,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2425924,"kb_used_data":6660,"kb_used_omap":0,"kb_used_meta":2419200,"kb_avail":752548796,"statfs":{"total":773094113280,"available":770609967104,"internally_reserved":0,"allocated":6819840,"data_stored":3779304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2477260800},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.288290"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323476+0000","last_change":"2023-10-08T01:09:09.322200+0000","last_active":"2023-10-08T01:09:09.323476+0000","last_peered":"2023-10-08T01:09:09.323476+0000","last_clean":"2023-10-08T01:09:09.323476+0000","last_became_active":"2023-10-08T01:09:09.321292+0000","last_became_peered":"2023-10-08T01:09:09.321292+0000","last_unstale":"2023-10-08T01:09:09.323476+0000","last_undegraded":"2023-10-08T01:09:09.323476+0000","last_fullsized":"2023-10-08T01:09:09.323476+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:47.695405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322358+0000","last_change":"2023-10-08T01:09:09.320320+0000","last_active":"2023-10-08T01:09:09.322358+0000","last_peered":"2023-10-08T01:09:09.322358+0000","last_clean":"2023-10-08T01:09:09.322358+0000","last_became_active":"2023-10-08T01:09:09.318044+0000","last_became_peered":"2023-10-08T01:09:09.318044+0000","last_unstale":"2023-10-08T01:09:09.322358+0000","last_undegraded":"2023-10-08T01:09:09.322358+0000","last_fullsized":"2023-10-08T01:09:09.322358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:38:18.765115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.330197+0000","last_change":"2023-10-08T01:09:09.329288+0000","last_active":"2023-10-08T01:09:09.330197+0000","last_peered":"2023-10-08T01:09:09.330197+0000","last_clean":"2023-10-08T01:09:09.330197+0000","last_became_active":"2023-10-08T01:09:09.329006+0000","last_became_peered":"2023-10-08T01:09:09.329006+0000","last_unstale":"2023-10-08T01:09:09.330197+0000","last_undegraded":"2023-10-08T01:09:09.330197+0000","last_fullsized":"2023-10-08T01:09:09.330197+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:13.215258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318725+0000","last_change":"2023-10-08T01:09:09.315238+0000","last_active":"2023-10-08T01:09:09.318725+0000","last_peered":"2023-10-08T01:09:09.318725+0000","last_clean":"2023-10-08T01:09:09.318725+0000","last_became_active":"2023-10-08T01:09:09.314933+0000","last_became_peered":"2023-10-08T01:09:09.314933+0000","last_unstale":"2023-10-08T01:09:09.318725+0000","last_undegraded":"2023-10-08T01:09:09.318725+0000","last_fullsized":"2023-10-08T01:09:09.318725+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:27.277557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322416+0000","last_change":"2023-10-08T01:09:09.320731+0000","last_active":"2023-10-08T01:09:09.322416+0000","last_peered":"2023-10-08T01:09:09.322416+0000","last_clean":"2023-10-08T01:09:09.322416+0000","last_became_active":"2023-10-08T01:09:09.320467+0000","last_became_peered":"2023-10-08T01:09:09.320467+0000","last_unstale":"2023-10-08T01:09:09.322416+0000","last_undegraded":"2023-10-08T01:09:09.322416+0000","last_fullsized":"2023-10-08T01:09:09.322416+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:50.992816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.329571+0000","last_change":"2023-10-08T01:09:09.325430+0000","last_active":"2023-10-08T01:09:09.329571+0000","last_peered":"2023-10-08T01:09:09.329571+0000","last_clean":"2023-10-08T01:09:09.329571+0000","last_became_active":"2023-10-08T01:09:09.323547+0000","last_became_peered":"2023-10-08T01:09:09.323547+0000","last_unstale":"2023-10-08T01:09:09.329571+0000","last_undegraded":"2023-10-08T01:09:09.329571+0000","last_fullsized":"2023-10-08T01:09:09.329571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:15.829749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327347+0000","last_change":"2023-10-08T01:09:09.326101+0000","last_active":"2023-10-08T01:09:09.327347+0000","last_peered":"2023-10-08T01:09:09.327347+0000","last_clean":"2023-10-08T01:09:09.327347+0000","last_became_active":"2023-10-08T01:09:09.325822+0000","last_became_peered":"2023-10-08T01:09:09.325822+0000","last_unstale":"2023-10-08T01:09:09.327347+0000","last_undegraded":"2023-10-08T01:09:09.327347+0000","last_fullsized":"2023-10-08T01:09:09.327347+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:38.721617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318897+0000","last_change":"2023-10-08T01:09:09.316183+0000","last_active":"2023-10-08T01:09:09.318897+0000","last_peered":"2023-10-08T01:09:09.318897+0000","last_clean":"2023-10-08T01:09:09.318897+0000","last_became_active":"2023-10-08T01:09:09.315862+0000","last_became_peered":"2023-10-08T01:09:09.315862+0000","last_unstale":"2023-10-08T01:09:09.318897+0000","last_undegraded":"2023-10-08T01:09:09.318897+0000","last_fullsized":"2023-10-08T01:09:09.318897+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:30.804190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.332804+0000","last_change":"2023-10-08T01:09:09.328609+0000","last_active":"2023-10-08T01:09:09.332804+0000","last_peered":"2023-10-08T01:09:09.332804+0000","last_clean":"2023-10-08T01:09:09.332804+0000","last_became_active":"2023-10-08T01:09:09.328330+0000","last_became_peered":"2023-10-08T01:09:09.328330+0000","last_unstale":"2023-10-08T01:09:09.332804+0000","last_undegraded":"2023-10-08T01:09:09.332804+0000","last_fullsized":"2023-10-08T01:09:09.332804+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:08.943473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.319156+0000","last_change":"2023-10-08T01:09:09.316904+0000","last_active":"2023-10-08T01:09:09.319156+0000","last_peered":"2023-10-08T01:09:09.319156+0000","last_clean":"2023-10-08T01:09:09.319156+0000","last_became_active":"2023-10-08T01:09:09.316639+0000","last_became_peered":"2023-10-08T01:09:09.316639+0000","last_unstale":"2023-10-08T01:09:09.319156+0000","last_undegraded":"2023-10-08T01:09:09.319156+0000","last_fullsized":"2023-10-08T01:09:09.319156+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:08.574405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.331164+0000","last_change":"2023-10-08T01:09:09.328672+0000","last_active":"2023-10-08T01:09:09.331164+0000","last_peered":"2023-10-08T01:09:09.331164+0000","last_clean":"2023-10-08T01:09:09.331164+0000","last_became_active":"2023-10-08T01:09:09.328318+0000","last_became_peered":"2023-10-08T01:09:09.328318+0000","last_unstale":"2023-10-08T01:09:09.331164+0000","last_undegraded":"2023-10-08T01:09:09.331164+0000","last_fullsized":"2023-10-08T01:09:09.331164+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:59.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.330585+0000","last_change":"2023-10-08T01:09:09.329592+0000","last_active":"2023-10-08T01:09:09.330585+0000","last_peered":"2023-10-08T01:09:09.330585+0000","last_clean":"2023-10-08T01:09:09.330585+0000","last_became_active":"2023-10-08T01:09:09.329141+0000","last_became_peered":"2023-10-08T01:09:09.329141+0000","last_unstale":"2023-10-08T01:09:09.330585+0000","last_undegraded":"2023-10-08T01:09:09.330585+0000","last_fullsized":"2023-10-08T01:09:09.330585+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:08.943473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.335204+0000","last_change":"2023-10-08T01:09:09.327822+0000","last_active":"2023-10-08T01:09:09.335204+0000","last_peered":"2023-10-08T01:09:09.335204+0000","last_clean":"2023-10-08T01:09:09.335204+0000","last_became_active":"2023-10-08T01:09:09.327568+0000","last_became_peered":"2023-10-08T01:09:09.327568+0000","last_unstale":"2023-10-08T01:09:09.335204+0000","last_undegraded":"2023-10-08T01:09:09.335204+0000","last_fullsized":"2023-10-08T01:09:09.335204+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:17.910526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.331376+0000","last_change":"2023-10-08T01:09:09.328529+0000","last_active":"2023-10-08T01:09:09.331376+0000","last_peered":"2023-10-08T01:09:09.331376+0000","last_clean":"2023-10-08T01:09:09.331376+0000","last_became_active":"2023-10-08T01:09:09.328228+0000","last_became_peered":"2023-10-08T01:09:09.328228+0000","last_unstale":"2023-10-08T01:09:09.331376+0000","last_undegraded":"2023-10-08T01:09:09.331376+0000","last_fullsized":"2023-10-08T01:09:09.331376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:38.532994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327303+0000","last_change":"2023-10-08T01:09:09.322917+0000","last_active":"2023-10-08T01:09:09.327303+0000","last_peered":"2023-10-08T01:09:09.327303+0000","last_clean":"2023-10-08T01:09:09.327303+0000","last_became_active":"2023-10-08T01:09:09.322603+0000","last_became_peered":"2023-10-08T01:09:09.322603+0000","last_unstale":"2023-10-08T01:09:09.327303+0000","last_undegraded":"2023-10-08T01:09:09.327303+0000","last_fullsized":"2023-10-08T01:09:09.327303+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:38:10.905840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327284+0000","last_change":"2023-10-08T01:09:09.323953+0000","last_active":"2023-10-08T01:09:09.327284+0000","last_peered":"2023-10-08T01:09:09.327284+0000","last_clean":"2023-10-08T01:09:09.327284+0000","last_became_active":"2023-10-08T01:09:09.323267+0000","last_became_peered":"2023-10-08T01:09:09.323267+0000","last_unstale":"2023-10-08T01:09:09.327284+0000","last_undegraded":"2023-10-08T01:09:09.327284+0000","last_fullsized":"2023-10-08T01:09:09.327284+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:13.215258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325495+0000","last_change":"2023-10-08T01:09:09.321689+0000","last_active":"2023-10-08T01:09:09.325495+0000","last_peered":"2023-10-08T01:09:09.325495+0000","last_clean":"2023-10-08T01:09:09.325495+0000","last_became_active":"2023-10-08T01:09:09.321436+0000","last_became_peered":"2023-10-08T01:09:09.321436+0000","last_unstale":"2023-10-08T01:09:09.325495+0000","last_undegraded":"2023-10-08T01:09:09.325495+0000","last_fullsized":"2023-10-08T01:09:09.325495+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:12.528510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325532+0000","last_change":"2023-10-08T01:09:09.321163+0000","last_active":"2023-10-08T01:09:09.325532+0000","last_peered":"2023-10-08T01:09:09.325532+0000","last_clean":"2023-10-08T01:09:09.325532+0000","last_became_active":"2023-10-08T01:09:09.320718+0000","last_became_peered":"2023-10-08T01:09:09.320718+0000","last_unstale":"2023-10-08T01:09:09.325532+0000","last_undegraded":"2023-10-08T01:09:09.325532+0000","last_fullsized":"2023-10-08T01:09:09.325532+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:47.187822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327242+0000","last_change":"2023-10-08T01:09:09.323783+0000","last_active":"2023-10-08T01:09:09.327242+0000","last_peered":"2023-10-08T01:09:09.327242+0000","last_clean":"2023-10-08T01:09:09.327242+0000","last_became_active":"2023-10-08T01:09:09.322684+0000","last_became_peered":"2023-10-08T01:09:09.322684+0000","last_unstale":"2023-10-08T01:09:09.327242+0000","last_undegraded":"2023-10-08T01:09:09.327242+0000","last_fullsized":"2023-10-08T01:09:09.327242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:27.277557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322921+0000","last_change":"2023-10-08T01:09:09.320248+0000","last_active":"2023-10-08T01:09:09.322921+0000","last_peered":"2023-10-08T01:09:09.322921+0000","last_clean":"2023-10-08T01:09:09.322921+0000","last_became_active":"2023-10-08T01:09:09.319981+0000","last_became_peered":"2023-10-08T01:09:09.319981+0000","last_unstale":"2023-10-08T01:09:09.322921+0000","last_undegraded":"2023-10-08T01:09:09.322921+0000","last_fullsized":"2023-10-08T01:09:09.322921+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:51.512961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.326297+0000","last_change":"2023-10-08T01:09:09.321400+0000","last_active":"2023-10-08T01:09:09.326297+0000","last_peered":"2023-10-08T01:09:09.326297+0000","last_clean":"2023-10-08T01:09:09.326297+0000","last_became_active":"2023-10-08T01:09:09.321120+0000","last_became_peered":"2023-10-08T01:09:09.321120+0000","last_unstale":"2023-10-08T01:09:09.326297+0000","last_undegraded":"2023-10-08T01:09:09.326297+0000","last_fullsized":"2023-10-08T01:09:09.326297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:08:08.553237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318847+0000","last_change":"2023-10-08T01:09:09.316924+0000","last_active":"2023-10-08T01:09:09.318847+0000","last_peered":"2023-10-08T01:09:09.318847+0000","last_clean":"2023-10-08T01:09:09.318847+0000","last_became_active":"2023-10-08T01:09:09.316601+0000","last_became_peered":"2023-10-08T01:09:09.316601+0000","last_unstale":"2023-10-08T01:09:09.318847+0000","last_undegraded":"2023-10-08T01:09:09.318847+0000","last_fullsized":"2023-10-08T01:09:09.318847+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:13.215258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.315432+0000","last_change":"2023-10-08T01:09:09.314521+0000","last_active":"2023-10-08T01:09:09.315432+0000","last_peered":"2023-10-08T01:09:09.315432+0000","last_clean":"2023-10-08T01:09:09.315432+0000","last_became_active":"2023-10-08T01:09:09.314269+0000","last_became_peered":"2023-10-08T01:09:09.314269+0000","last_unstale":"2023-10-08T01:09:09.315432+0000","last_undegraded":"2023-10-08T01:09:09.315432+0000","last_fullsized":"2023-10-08T01:09:09.315432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:17.910526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.315920+0000","last_change":"2023-10-08T01:09:09.313540+0000","last_active":"2023-10-08T01:09:09.315920+0000","last_peered":"2023-10-08T01:09:09.315920+0000","last_clean":"2023-10-08T01:09:09.315920+0000","last_became_active":"2023-10-08T01:09:09.313271+0000","last_became_peered":"2023-10-08T01:09:09.313271+0000","last_unstale":"2023-10-08T01:09:09.315920+0000","last_undegraded":"2023-10-08T01:09:09.315920+0000","last_fullsized":"2023-10-08T01:09:09.315920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:37.644751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327529+0000","last_change":"2023-10-08T01:09:09.325817+0000","last_active":"2023-10-08T01:09:09.327529+0000","last_peered":"2023-10-08T01:09:09.327529+0000","last_clean":"2023-10-08T01:09:09.327529+0000","last_became_active":"2023-10-08T01:09:09.324410+0000","last_became_peered":"2023-10-08T01:09:09.324410+0000","last_unstale":"2023-10-08T01:09:09.327529+0000","last_undegraded":"2023-10-08T01:09:09.327529+0000","last_fullsized":"2023-10-08T01:09:09.327529+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:03.235186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.335315+0000","last_change":"2023-10-08T01:09:09.328382+0000","last_active":"2023-10-08T01:09:09.335315+0000","last_peered":"2023-10-08T01:09:09.335315+0000","last_clean":"2023-10-08T01:09:09.335315+0000","last_became_active":"2023-10-08T01:09:09.328116+0000","last_became_peered":"2023-10-08T01:09:09.328116+0000","last_unstale":"2023-10-08T01:09:09.335315+0000","last_undegraded":"2023-10-08T01:09:09.335315+0000","last_fullsized":"2023-10-08T01:09:09.335315+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:37.644751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323042+0000","last_change":"2023-10-08T01:09:09.316967+0000","last_active":"2023-10-08T01:09:09.323042+0000","last_peered":"2023-10-08T01:09:09.323042+0000","last_clean":"2023-10-08T01:09:09.323042+0000","last_became_active":"2023-10-08T01:09:09.316711+0000","last_became_peered":"2023-10-08T01:09:09.316711+0000","last_unstale":"2023-10-08T01:09:09.323042+0000","last_undegraded":"2023-10-08T01:09:09.323042+0000","last_fullsized":"2023-10-08T01:09:09.323042+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:23.143628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.330605+0000","last_change":"2023-10-08T01:09:09.329674+0000","last_active":"2023-10-08T01:09:09.330605+0000","last_peered":"2023-10-08T01:09:09.330605+0000","last_clean":"2023-10-08T01:09:09.330605+0000","last_became_active":"2023-10-08T01:09:09.329400+0000","last_became_peered":"2023-10-08T01:09:09.329400+0000","last_unstale":"2023-10-08T01:09:09.330605+0000","last_undegraded":"2023-10-08T01:09:09.330605+0000","last_fullsized":"2023-10-08T01:09:09.330605+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:20.710780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327145+0000","last_change":"2023-10-08T01:09:09.323054+0000","last_active":"2023-10-08T01:09:09.327145+0000","last_peered":"2023-10-08T01:09:09.327145+0000","last_clean":"2023-10-08T01:09:09.327145+0000","last_became_active":"2023-10-08T01:09:09.321930+0000","last_became_peered":"2023-10-08T01:09:09.321930+0000","last_unstale":"2023-10-08T01:09:09.327145+0000","last_undegraded":"2023-10-08T01:09:09.327145+0000","last_fullsized":"2023-10-08T01:09:09.327145+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:47:28.247811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323010+0000","last_change":"2023-10-08T01:09:09.317908+0000","last_active":"2023-10-08T01:09:09.323010+0000","last_peered":"2023-10-08T01:09:09.323010+0000","last_clean":"2023-10-08T01:09:09.323010+0000","last_became_active":"2023-10-08T01:09:09.317631+0000","last_became_peered":"2023-10-08T01:09:09.317631+0000","last_unstale":"2023-10-08T01:09:09.323010+0000","last_undegraded":"2023-10-08T01:09:09.323010+0000","last_fullsized":"2023-10-08T01:09:09.323010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:08.943473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.320294+0000","last_change":"2023-10-08T01:09:09.318549+0000","last_active":"2023-10-08T01:09:09.320294+0000","last_peered":"2023-10-08T01:09:09.320294+0000","last_clean":"2023-10-08T01:09:09.320294+0000","last_became_active":"2023-10-08T01:09:09.318233+0000","last_became_peered":"2023-10-08T01:09:09.318233+0000","last_unstale":"2023-10-08T01:09:09.320294+0000","last_undegraded":"2023-10-08T01:09:09.320294+0000","last_fullsized":"2023-10-08T01:09:09.320294+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:47.695405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.331447+0000","last_change":"2023-10-08T01:09:09.326220+0000","last_active":"2023-10-08T01:09:09.331447+0000","last_peered":"2023-10-08T01:09:09.331447+0000","last_clean":"2023-10-08T01:09:09.331447+0000","last_became_active":"2023-10-08T01:09:09.325942+0000","last_became_peered":"2023-10-08T01:09:09.325942+0000","last_unstale":"2023-10-08T01:09:09.331447+0000","last_undegraded":"2023-10-08T01:09:09.331447+0000","last_fullsized":"2023-10-08T01:09:09.331447+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:03.235186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.326277+0000","last_change":"2023-10-08T01:09:09.321483+0000","last_active":"2023-10-08T01:09:09.326277+0000","last_peered":"2023-10-08T01:09:09.326277+0000","last_clean":"2023-10-08T01:09:09.326277+0000","last_became_active":"2023-10-08T01:09:09.321207+0000","last_became_peered":"2023-10-08T01:09:09.321207+0000","last_unstale":"2023-10-08T01:09:09.326277+0000","last_undegraded":"2023-10-08T01:09:09.326277+0000","last_fullsized":"2023-10-08T01:09:09.326277+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:44:30.804190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"58'4","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.345174+0000","last_change":"2023-10-08T01:09:08.313826+0000","last_active":"2023-10-08T01:09:09.345174+0000","last_peered":"2023-10-08T01:09:09.345174+0000","last_clean":"2023-10-08T01:09:09.345174+0000","last_became_active":"2023-10-08T01:09:08.313333+0000","last_became_peered":"2023-10-08T01:09:08.313333+0000","last_unstale":"2023-10-08T01:09:09.345174+0000","last_undegraded":"2023-10-08T01:09:09.345174+0000","last_fullsized":"2023-10-08T01:09:09.345174+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:58.460395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325954+0000","last_change":"2023-10-08T01:09:09.316052+0000","last_active":"2023-10-08T01:09:09.325954+0000","last_peered":"2023-10-08T01:09:09.325954+0000","last_clean":"2023-10-08T01:09:09.325954+0000","last_became_active":"2023-10-08T01:09:09.315769+0000","last_became_peered":"2023-10-08T01:09:09.315769+0000","last_unstale":"2023-10-08T01:09:09.325954+0000","last_undegraded":"2023-10-08T01:09:09.325954+0000","last_fullsized":"2023-10-08T01:09:09.325954+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:51.512961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.337597+0000","last_change":"2023-10-08T01:09:08.319687+0000","last_active":"2023-10-08T01:09:09.337597+0000","last_peered":"2023-10-08T01:09:09.337597+0000","last_clean":"2023-10-08T01:09:09.337597+0000","last_became_active":"2023-10-08T01:09:08.319371+0000","last_became_peered":"2023-10-08T01:09:08.319371+0000","last_unstale":"2023-10-08T01:09:09.337597+0000","last_undegraded":"2023-10-08T01:09:09.337597+0000","last_fullsized":"2023-10-08T01:09:09.337597+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:02.224693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322457+0000","last_change":"2023-10-08T01:09:09.321527+0000","last_active":"2023-10-08T01:09:09.322457+0000","last_peered":"2023-10-08T01:09:09.322457+0000","last_clean":"2023-10-08T01:09:09.322457+0000","last_became_active":"2023-10-08T01:09:09.321236+0000","last_became_peered":"2023-10-08T01:09:09.321236+0000","last_unstale":"2023-10-08T01:09:09.322457+0000","last_undegraded":"2023-10-08T01:09:09.322457+0000","last_fullsized":"2023-10-08T01:09:09.322457+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:47.187822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.344054+0000","last_change":"2023-10-08T01:09:08.312918+0000","last_active":"2023-10-08T01:09:09.344054+0000","last_peered":"2023-10-08T01:09:09.344054+0000","last_clean":"2023-10-08T01:09:09.344054+0000","last_became_active":"2023-10-08T01:09:08.312643+0000","last_became_peered":"2023-10-08T01:09:08.312643+0000","last_unstale":"2023-10-08T01:09:09.344054+0000","last_undegraded":"2023-10-08T01:09:09.344054+0000","last_fullsized":"2023-10-08T01:09:09.344054+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:36.634257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327352+0000","last_change":"2023-10-08T01:09:09.325801+0000","last_active":"2023-10-08T01:09:09.327352+0000","last_peered":"2023-10-08T01:09:09.327352+0000","last_clean":"2023-10-08T01:09:09.327352+0000","last_became_active":"2023-10-08T01:09:09.324754+0000","last_became_peered":"2023-10-08T01:09:09.324754+0000","last_unstale":"2023-10-08T01:09:09.327352+0000","last_undegraded":"2023-10-08T01:09:09.327352+0000","last_fullsized":"2023-10-08T01:09:09.327352+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:27.277557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.336271+0000","last_change":"2023-10-08T01:09:08.316194+0000","last_active":"2023-10-08T01:09:09.336271+0000","last_peered":"2023-10-08T01:09:09.336271+0000","last_clean":"2023-10-08T01:09:09.336271+0000","last_became_active":"2023-10-08T01:09:08.315901+0000","last_became_peered":"2023-10-08T01:09:08.315901+0000","last_unstale":"2023-10-08T01:09:09.336271+0000","last_undegraded":"2023-10-08T01:09:09.336271+0000","last_fullsized":"2023-10-08T01:09:09.336271+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:11.518017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322991+0000","last_change":"2023-10-08T01:09:09.316050+0000","last_active":"2023-10-08T01:09:09.322991+0000","last_peered":"2023-10-08T01:09:09.322991+0000","last_clean":"2023-10-08T01:09:09.322991+0000","last_became_active":"2023-10-08T01:09:09.315805+0000","last_became_peered":"2023-10-08T01:09:09.315805+0000","last_unstale":"2023-10-08T01:09:09.322991+0000","last_undegraded":"2023-10-08T01:09:09.322991+0000","last_fullsized":"2023-10-08T01:09:09.322991+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:59.470888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.345304+0000","last_change":"2023-10-08T01:09:08.315821+0000","last_active":"2023-10-08T01:09:09.345304+0000","last_peered":"2023-10-08T01:09:09.345304+0000","last_clean":"2023-10-08T01:09:09.345304+0000","last_became_active":"2023-10-08T01:09:08.315587+0000","last_became_peered":"2023-10-08T01:09:08.315587+0000","last_unstale":"2023-10-08T01:09:09.345304+0000","last_undegraded":"2023-10-08T01:09:09.345304+0000","last_fullsized":"2023-10-08T01:09:09.345304+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:19.700287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.317297+0000","last_change":"2023-10-08T01:09:09.313549+0000","last_active":"2023-10-08T01:09:09.317297+0000","last_peered":"2023-10-08T01:09:09.317297+0000","last_clean":"2023-10-08T01:09:09.317297+0000","last_became_active":"2023-10-08T01:09:09.310250+0000","last_became_peered":"2023-10-08T01:09:09.310250+0000","last_unstale":"2023-10-08T01:09:09.317297+0000","last_undegraded":"2023-10-08T01:09:09.317297+0000","last_fullsized":"2023-10-08T01:09:09.317297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:23.143628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.336208+0000","last_change":"2023-10-08T01:09:08.316185+0000","last_active":"2023-10-08T01:09:09.336208+0000","last_peered":"2023-10-08T01:09:09.336208+0000","last_clean":"2023-10-08T01:09:09.336208+0000","last_became_active":"2023-10-08T01:09:08.315086+0000","last_became_peered":"2023-10-08T01:09:08.315086+0000","last_unstale":"2023-10-08T01:09:09.336208+0000","last_undegraded":"2023-10-08T01:09:09.336208+0000","last_fullsized":"2023-10-08T01:09:09.336208+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:21:17.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318915+0000","last_change":"2023-10-08T01:09:09.316377+0000","last_active":"2023-10-08T01:09:09.318915+0000","last_peered":"2023-10-08T01:09:09.318915+0000","last_clean":"2023-10-08T01:09:09.318915+0000","last_became_active":"2023-10-08T01:09:09.316118+0000","last_became_peered":"2023-10-08T01:09:09.316118+0000","last_unstale":"2023-10-08T01:09:09.318915+0000","last_undegraded":"2023-10-08T01:09:09.318915+0000","last_fullsized":"2023-10-08T01:09:09.318915+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:14:59.082595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.310530+0000","last_change":"2023-10-08T01:09:08.310728+0000","last_active":"2023-10-08T01:09:09.310530+0000","last_peered":"2023-10-08T01:09:09.310530+0000","last_clean":"2023-10-08T01:09:09.310530+0000","last_became_active":"2023-10-08T01:09:08.310445+0000","last_became_peered":"2023-10-08T01:09:08.310445+0000","last_unstale":"2023-10-08T01:09:09.310530+0000","last_undegraded":"2023-10-08T01:09:09.310530+0000","last_fullsized":"2023-10-08T01:09:09.310530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:16:12.204765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323956+0000","last_change":"2023-10-08T01:09:09.315344+0000","last_active":"2023-10-08T01:09:09.323956+0000","last_peered":"2023-10-08T01:09:09.323956+0000","last_clean":"2023-10-08T01:09:09.323956+0000","last_became_active":"2023-10-08T01:09:09.315024+0000","last_became_peered":"2023-10-08T01:09:09.315024+0000","last_unstale":"2023-10-08T01:09:09.323956+0000","last_undegraded":"2023-10-08T01:09:09.323956+0000","last_fullsized":"2023-10-08T01:09:09.323956+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:21:18.795393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.306506+0000","last_change":"2023-10-08T01:09:08.318464+0000","last_active":"2023-10-08T01:09:09.306506+0000","last_peered":"2023-10-08T01:09:09.306506+0000","last_clean":"2023-10-08T01:09:09.306506+0000","last_became_active":"2023-10-08T01:09:08.318159+0000","last_became_peered":"2023-10-08T01:09:08.318159+0000","last_unstale":"2023-10-08T01:09:09.306506+0000","last_undegraded":"2023-10-08T01:09:09.306506+0000","last_fullsized":"2023-10-08T01:09:09.306506+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:31:45.525204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325511+0000","last_change":"2023-10-08T01:09:09.320974+0000","last_active":"2023-10-08T01:09:09.325511+0000","last_peered":"2023-10-08T01:09:09.325511+0000","last_clean":"2023-10-08T01:09:09.325511+0000","last_became_active":"2023-10-08T01:09:09.320375+0000","last_became_peered":"2023-10-08T01:09:09.320375+0000","last_unstale":"2023-10-08T01:09:09.325511+0000","last_undegraded":"2023-10-08T01:09:09.325511+0000","last_fullsized":"2023-10-08T01:09:09.325511+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:36.546997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.312154+0000","last_change":"2023-10-08T01:09:08.314452+0000","last_active":"2023-10-08T01:09:09.312154+0000","last_peered":"2023-10-08T01:09:09.312154+0000","last_clean":"2023-10-08T01:09:09.312154+0000","last_became_active":"2023-10-08T01:09:08.314129+0000","last_became_peered":"2023-10-08T01:09:08.314129+0000","last_unstale":"2023-10-08T01:09:09.312154+0000","last_undegraded":"2023-10-08T01:09:09.312154+0000","last_fullsized":"2023-10-08T01:09:09.312154+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:37.731362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322540+0000","last_change":"2023-10-08T01:09:09.319386+0000","last_active":"2023-10-08T01:09:09.322540+0000","last_peered":"2023-10-08T01:09:09.322540+0000","last_clean":"2023-10-08T01:09:09.322540+0000","last_became_active":"2023-10-08T01:09:09.318572+0000","last_became_peered":"2023-10-08T01:09:09.318572+0000","last_unstale":"2023-10-08T01:09:09.322540+0000","last_undegraded":"2023-10-08T01:09:09.322540+0000","last_fullsized":"2023-10-08T01:09:09.322540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:31.033623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.306937+0000","last_change":"2023-10-08T01:09:08.323551+0000","last_active":"2023-10-08T01:09:09.306937+0000","last_peered":"2023-10-08T01:09:09.306937+0000","last_clean":"2023-10-08T01:09:09.306937+0000","last_became_active":"2023-10-08T01:09:08.323246+0000","last_became_peered":"2023-10-08T01:09:08.323246+0000","last_unstale":"2023-10-08T01:09:09.306937+0000","last_undegraded":"2023-10-08T01:09:09.306937+0000","last_fullsized":"2023-10-08T01:09:09.306937+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:21.726634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.338574+0000","last_change":"2023-10-08T01:09:09.332911+0000","last_active":"2023-10-08T01:09:09.338574+0000","last_peered":"2023-10-08T01:09:09.338574+0000","last_clean":"2023-10-08T01:09:09.338574+0000","last_became_active":"2023-10-08T01:09:09.332663+0000","last_became_peered":"2023-10-08T01:09:09.332663+0000","last_unstale":"2023-10-08T01:09:09.338574+0000","last_undegraded":"2023-10-08T01:09:09.338574+0000","last_fullsized":"2023-10-08T01:09:09.338574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:07:17.048807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"58'3","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.337528+0000","last_change":"2023-10-08T01:09:08.321891+0000","last_active":"2023-10-08T01:09:09.337528+0000","last_peered":"2023-10-08T01:09:09.337528+0000","last_clean":"2023-10-08T01:09:09.337528+0000","last_became_active":"2023-10-08T01:09:08.321057+0000","last_became_peered":"2023-10-08T01:09:08.321057+0000","last_unstale":"2023-10-08T01:09:09.337528+0000","last_undegraded":"2023-10-08T01:09:09.337528+0000","last_fullsized":"2023-10-08T01:09:09.337528+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:06:58.460395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323443+0000","last_change":"2023-10-08T01:09:09.322019+0000","last_active":"2023-10-08T01:09:09.323443+0000","last_peered":"2023-10-08T01:09:09.323443+0000","last_clean":"2023-10-08T01:09:09.323443+0000","last_became_active":"2023-10-08T01:09:09.321726+0000","last_became_peered":"2023-10-08T01:09:09.321726+0000","last_unstale":"2023-10-08T01:09:09.323443+0000","last_undegraded":"2023-10-08T01:09:09.323443+0000","last_fullsized":"2023-10-08T01:09:09.323443+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:18.065010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"58'6","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.347108+0000","last_change":"2023-10-08T01:09:08.316773+0000","last_active":"2023-10-08T01:09:09.347108+0000","last_peered":"2023-10-08T01:09:09.347108+0000","last_clean":"2023-10-08T01:09:09.347108+0000","last_became_active":"2023-10-08T01:09:08.316557+0000","last_became_peered":"2023-10-08T01:09:08.316557+0000","last_unstale":"2023-10-08T01:09:09.347108+0000","last_undegraded":"2023-10-08T01:09:09.347108+0000","last_fullsized":"2023-10-08T01:09:09.347108+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:21.726634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.324155+0000","last_change":"2023-10-08T01:09:09.319780+0000","last_active":"2023-10-08T01:09:09.324155+0000","last_peered":"2023-10-08T01:09:09.324155+0000","last_clean":"2023-10-08T01:09:09.324155+0000","last_became_active":"2023-10-08T01:09:09.319259+0000","last_became_peered":"2023-10-08T01:09:09.319259+0000","last_unstale":"2023-10-08T01:09:09.324155+0000","last_undegraded":"2023-10-08T01:09:09.324155+0000","last_fullsized":"2023-10-08T01:09:09.324155+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:38.741855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.345219+0000","last_change":"2023-10-08T01:09:08.319537+0000","last_active":"2023-10-08T01:09:09.345219+0000","last_peered":"2023-10-08T01:09:09.345219+0000","last_clean":"2023-10-08T01:09:09.345219+0000","last_became_active":"2023-10-08T01:09:08.319258+0000","last_became_peered":"2023-10-08T01:09:08.319258+0000","last_unstale":"2023-10-08T01:09:09.345219+0000","last_undegraded":"2023-10-08T01:09:09.345219+0000","last_fullsized":"2023-10-08T01:09:09.345219+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:46.177328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.318960+0000","last_change":"2023-10-08T01:09:09.314911+0000","last_active":"2023-10-08T01:09:09.318960+0000","last_peered":"2023-10-08T01:09:09.318960+0000","last_clean":"2023-10-08T01:09:09.318960+0000","last_became_active":"2023-10-08T01:09:09.314631+0000","last_became_peered":"2023-10-08T01:09:09.314631+0000","last_unstale":"2023-10-08T01:09:09.318960+0000","last_undegraded":"2023-10-08T01:09:09.318960+0000","last_fullsized":"2023-10-08T01:09:09.318960+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:36.546997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.303993+0000","last_change":"2023-10-08T01:09:08.315297+0000","last_active":"2023-10-08T01:09:09.303993+0000","last_peered":"2023-10-08T01:09:09.303993+0000","last_clean":"2023-10-08T01:09:09.303993+0000","last_became_active":"2023-10-08T01:09:08.315028+0000","last_became_peered":"2023-10-08T01:09:08.315028+0000","last_unstale":"2023-10-08T01:09:09.303993+0000","last_undegraded":"2023-10-08T01:09:09.303993+0000","last_fullsized":"2023-10-08T01:09:09.303993+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:07.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.341540+0000","last_change":"2023-10-08T01:09:09.333801+0000","last_active":"2023-10-08T01:09:09.341540+0000","last_peered":"2023-10-08T01:09:09.341540+0000","last_clean":"2023-10-08T01:09:09.341540+0000","last_became_active":"2023-10-08T01:09:09.333502+0000","last_became_peered":"2023-10-08T01:09:09.333502+0000","last_unstale":"2023-10-08T01:09:09.341540+0000","last_undegraded":"2023-10-08T01:09:09.341540+0000","last_fullsized":"2023-10-08T01:09:09.341540+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:53:31.033623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.311771+0000","last_change":"2023-10-08T01:09:08.320239+0000","last_active":"2023-10-08T01:09:09.311771+0000","last_peered":"2023-10-08T01:09:09.311771+0000","last_clean":"2023-10-08T01:09:09.311771+0000","last_became_active":"2023-10-08T01:09:08.319955+0000","last_became_peered":"2023-10-08T01:09:08.319955+0000","last_unstale":"2023-10-08T01:09:09.311771+0000","last_undegraded":"2023-10-08T01:09:09.311771+0000","last_fullsized":"2023-10-08T01:09:09.311771+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:35.536504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.317547+0000","last_change":"2023-10-08T01:09:09.313493+0000","last_active":"2023-10-08T01:09:09.317547+0000","last_peered":"2023-10-08T01:09:09.317547+0000","last_clean":"2023-10-08T01:09:09.317547+0000","last_became_active":"2023-10-08T01:09:09.313229+0000","last_became_peered":"2023-10-08T01:09:09.313229+0000","last_unstale":"2023-10-08T01:09:09.317547+0000","last_undegraded":"2023-10-08T01:09:09.317547+0000","last_fullsized":"2023-10-08T01:09:09.317547+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:17.244899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"58'8","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.344025+0000","last_change":"2023-10-08T01:09:08.317533+0000","last_active":"2023-10-08T01:09:09.344025+0000","last_peered":"2023-10-08T01:09:09.344025+0000","last_clean":"2023-10-08T01:09:09.344025+0000","last_became_active":"2023-10-08T01:09:08.317259+0000","last_became_peered":"2023-10-08T01:09:08.317259+0000","last_unstale":"2023-10-08T01:09:09.344025+0000","last_undegraded":"2023-10-08T01:09:09.344025+0000","last_fullsized":"2023-10-08T01:09:09.344025+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:56:19.417230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.326851+0000","last_change":"2023-10-08T01:09:09.324109+0000","last_active":"2023-10-08T01:09:09.326851+0000","last_peered":"2023-10-08T01:09:09.326851+0000","last_clean":"2023-10-08T01:09:09.326851+0000","last_became_active":"2023-10-08T01:09:09.323838+0000","last_became_peered":"2023-10-08T01:09:09.323838+0000","last_unstale":"2023-10-08T01:09:09.326851+0000","last_undegraded":"2023-10-08T01:09:09.326851+0000","last_fullsized":"2023-10-08T01:09:09.326851+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:38.741855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.342258+0000","last_change":"2023-10-08T01:09:08.312702+0000","last_active":"2023-10-08T01:09:09.342258+0000","last_peered":"2023-10-08T01:09:09.342258+0000","last_clean":"2023-10-08T01:09:09.342258+0000","last_became_active":"2023-10-08T01:09:08.312233+0000","last_became_peered":"2023-10-08T01:09:08.312233+0000","last_unstale":"2023-10-08T01:09:09.342258+0000","last_undegraded":"2023-10-08T01:09:09.342258+0000","last_fullsized":"2023-10-08T01:09:09.342258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:38.858148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.322757+0000","last_change":"2023-10-08T01:09:09.318140+0000","last_active":"2023-10-08T01:09:09.322757+0000","last_peered":"2023-10-08T01:09:09.322757+0000","last_clean":"2023-10-08T01:09:09.322757+0000","last_became_active":"2023-10-08T01:09:09.317668+0000","last_became_peered":"2023-10-08T01:09:09.317668+0000","last_unstale":"2023-10-08T01:09:09.322757+0000","last_undegraded":"2023-10-08T01:09:09.322757+0000","last_fullsized":"2023-10-08T01:09:09.322757+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:37.644751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.338725+0000","last_change":"2023-10-08T01:09:08.318961+0000","last_active":"2023-10-08T01:09:09.338725+0000","last_peered":"2023-10-08T01:09:09.338725+0000","last_clean":"2023-10-08T01:09:09.338725+0000","last_became_active":"2023-10-08T01:09:08.318478+0000","last_became_peered":"2023-10-08T01:09:08.318478+0000","last_unstale":"2023-10-08T01:09:09.338725+0000","last_undegraded":"2023-10-08T01:09:09.338725+0000","last_fullsized":"2023-10-08T01:09:09.338725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:50.502468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327200+0000","last_change":"2023-10-08T01:09:09.323492+0000","last_active":"2023-10-08T01:09:09.327200+0000","last_peered":"2023-10-08T01:09:09.327200+0000","last_clean":"2023-10-08T01:09:09.327200+0000","last_became_active":"2023-10-08T01:09:09.323107+0000","last_became_peered":"2023-10-08T01:09:09.323107+0000","last_unstale":"2023-10-08T01:09:09.327200+0000","last_undegraded":"2023-10-08T01:09:09.327200+0000","last_fullsized":"2023-10-08T01:09:09.327200+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:11:35.306878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"58'1","reported_seq":11,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.338558+0000","last_change":"2023-10-08T01:09:08.311946+0000","last_active":"2023-10-08T01:09:09.338558+0000","last_peered":"2023-10-08T01:09:09.338558+0000","last_clean":"2023-10-08T01:09:09.338558+0000","last_became_active":"2023-10-08T01:09:08.311697+0000","last_became_peered":"2023-10-08T01:09:08.311697+0000","last_unstale":"2023-10-08T01:09:09.338558+0000","last_undegraded":"2023-10-08T01:09:09.338558+0000","last_fullsized":"2023-10-08T01:09:09.338558+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:22.133135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327299+0000","last_change":"2023-10-08T01:09:09.324004+0000","last_active":"2023-10-08T01:09:09.327299+0000","last_peered":"2023-10-08T01:09:09.327299+0000","last_clean":"2023-10-08T01:09:09.327299+0000","last_became_active":"2023-10-08T01:09:09.323701+0000","last_became_peered":"2023-10-08T01:09:09.323701+0000","last_unstale":"2023-10-08T01:09:09.327299+0000","last_undegraded":"2023-10-08T01:09:09.327299+0000","last_fullsized":"2023-10-08T01:09:09.327299+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:18:39.868642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.313507+0000","last_change":"2023-10-08T01:09:08.314277+0000","last_active":"2023-10-08T01:09:09.313507+0000","last_peered":"2023-10-08T01:09:09.313507+0000","last_clean":"2023-10-08T01:09:09.313507+0000","last_became_active":"2023-10-08T01:09:08.314019+0000","last_became_peered":"2023-10-08T01:09:08.314019+0000","last_unstale":"2023-10-08T01:09:09.313507+0000","last_undegraded":"2023-10-08T01:09:09.313507+0000","last_fullsized":"2023-10-08T01:09:09.313507+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:07.563912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.336575+0000","last_change":"2023-10-08T01:09:09.330962+0000","last_active":"2023-10-08T01:09:09.336575+0000","last_peered":"2023-10-08T01:09:09.336575+0000","last_clean":"2023-10-08T01:09:09.336575+0000","last_became_active":"2023-10-08T01:09:09.330718+0000","last_became_peered":"2023-10-08T01:09:09.330718+0000","last_unstale":"2023-10-08T01:09:09.336575+0000","last_undegraded":"2023-10-08T01:09:09.336575+0000","last_fullsized":"2023-10-08T01:09:09.336575+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:11:36.035056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.345231+0000","last_change":"2023-10-08T01:09:08.315258+0000","last_active":"2023-10-08T01:09:09.345231+0000","last_peered":"2023-10-08T01:09:09.345231+0000","last_clean":"2023-10-08T01:09:09.345231+0000","last_became_active":"2023-10-08T01:09:08.315017+0000","last_became_peered":"2023-10-08T01:09:08.315017+0000","last_unstale":"2023-10-08T01:09:09.345231+0000","last_undegraded":"2023-10-08T01:09:09.345231+0000","last_fullsized":"2023-10-08T01:09:09.345231+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:07.932980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.328948+0000","last_change":"2023-10-08T01:09:09.326623+0000","last_active":"2023-10-08T01:09:09.328948+0000","last_peered":"2023-10-08T01:09:09.328948+0000","last_clean":"2023-10-08T01:09:09.328948+0000","last_became_active":"2023-10-08T01:09:09.325498+0000","last_became_peered":"2023-10-08T01:09:09.325498+0000","last_unstale":"2023-10-08T01:09:09.328948+0000","last_undegraded":"2023-10-08T01:09:09.328948+0000","last_fullsized":"2023-10-08T01:09:09.328948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:23.143628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.310716+0000","last_change":"2023-10-08T01:09:08.306175+0000","last_active":"2023-10-08T01:09:09.310716+0000","last_peered":"2023-10-08T01:09:09.310716+0000","last_clean":"2023-10-08T01:09:09.310716+0000","last_became_active":"2023-10-08T01:09:08.305878+0000","last_became_peered":"2023-10-08T01:09:08.305878+0000","last_unstale":"2023-10-08T01:09:09.310716+0000","last_undegraded":"2023-10-08T01:09:09.310716+0000","last_fullsized":"2023-10-08T01:09:09.310716+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:14:58.072102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.320402+0000","last_change":"2023-10-08T01:09:09.319509+0000","last_active":"2023-10-08T01:09:09.320402+0000","last_peered":"2023-10-08T01:09:09.320402+0000","last_clean":"2023-10-08T01:09:09.320402+0000","last_became_active":"2023-10-08T01:09:09.319182+0000","last_became_peered":"2023-10-08T01:09:09.319182+0000","last_unstale":"2023-10-08T01:09:09.320402+0000","last_undegraded":"2023-10-08T01:09:09.320402+0000","last_fullsized":"2023-10-08T01:09:09.320402+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:21:18.795393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.305588+0000","last_change":"2023-10-08T01:09:08.321444+0000","last_active":"2023-10-08T01:09:09.305588+0000","last_peered":"2023-10-08T01:09:09.305588+0000","last_clean":"2023-10-08T01:09:09.305588+0000","last_became_active":"2023-10-08T01:09:08.321107+0000","last_became_peered":"2023-10-08T01:09:08.321107+0000","last_unstale":"2023-10-08T01:09:09.305588+0000","last_undegraded":"2023-10-08T01:09:09.305588+0000","last_fullsized":"2023-10-08T01:09:09.305588+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:07.038593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.331130+0000","last_change":"2023-10-08T01:09:09.328674+0000","last_active":"2023-10-08T01:09:09.331130+0000","last_peered":"2023-10-08T01:09:09.331130+0000","last_clean":"2023-10-08T01:09:09.331130+0000","last_became_active":"2023-10-08T01:09:09.328357+0000","last_became_peered":"2023-10-08T01:09:09.328357+0000","last_unstale":"2023-10-08T01:09:09.331130+0000","last_undegraded":"2023-10-08T01:09:09.331130+0000","last_fullsized":"2023-10-08T01:09:09.331130+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:20.710780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.303058+0000","last_change":"2023-10-08T01:09:08.315463+0000","last_active":"2023-10-08T01:09:09.303058+0000","last_peered":"2023-10-08T01:09:09.303058+0000","last_clean":"2023-10-08T01:09:09.303058+0000","last_became_active":"2023-10-08T01:09:08.315193+0000","last_became_peered":"2023-10-08T01:09:08.315193+0000","last_unstale":"2023-10-08T01:09:09.303058+0000","last_undegraded":"2023-10-08T01:09:09.303058+0000","last_fullsized":"2023-10-08T01:09:09.303058+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:19.700287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.325975+0000","last_change":"2023-10-08T01:09:09.320956+0000","last_active":"2023-10-08T01:09:09.325975+0000","last_peered":"2023-10-08T01:09:09.325975+0000","last_clean":"2023-10-08T01:09:09.325975+0000","last_became_active":"2023-10-08T01:09:09.320592+0000","last_became_peered":"2023-10-08T01:09:09.320592+0000","last_unstale":"2023-10-08T01:09:09.325975+0000","last_undegraded":"2023-10-08T01:09:09.325975+0000","last_fullsized":"2023-10-08T01:09:09.325975+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:03:40.558251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.314033+0000","last_change":"2023-10-08T01:09:08.319445+0000","last_active":"2023-10-08T01:09:09.314033+0000","last_peered":"2023-10-08T01:09:09.314033+0000","last_clean":"2023-10-08T01:09:09.314033+0000","last_became_active":"2023-10-08T01:09:08.319158+0000","last_became_peered":"2023-10-08T01:09:08.319158+0000","last_unstale":"2023-10-08T01:09:09.314033+0000","last_undegraded":"2023-10-08T01:09:09.314033+0000","last_fullsized":"2023-10-08T01:09:09.314033+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:59.432265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323304+0000","last_change":"2023-10-08T01:09:09.319198+0000","last_active":"2023-10-08T01:09:09.323304+0000","last_peered":"2023-10-08T01:09:09.323304+0000","last_clean":"2023-10-08T01:09:09.323304+0000","last_became_active":"2023-10-08T01:09:09.318952+0000","last_became_peered":"2023-10-08T01:09:09.318952+0000","last_unstale":"2023-10-08T01:09:09.323304+0000","last_undegraded":"2023-10-08T01:09:09.323304+0000","last_fullsized":"2023-10-08T01:09:09.323304+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:15:00.442758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.305085+0000","last_change":"2023-10-08T01:09:08.322197+0000","last_active":"2023-10-08T01:09:09.305085+0000","last_peered":"2023-10-08T01:09:09.305085+0000","last_clean":"2023-10-08T01:09:09.305085+0000","last_became_active":"2023-10-08T01:09:08.321567+0000","last_became_peered":"2023-10-08T01:09:08.321567+0000","last_unstale":"2023-10-08T01:09:09.305085+0000","last_undegraded":"2023-10-08T01:09:09.305085+0000","last_fullsized":"2023-10-08T01:09:09.305085+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:14:59.432265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.314587+0000","last_change":"2023-10-08T01:09:09.310737+0000","last_active":"2023-10-08T01:09:09.314587+0000","last_peered":"2023-10-08T01:09:09.314587+0000","last_clean":"2023-10-08T01:09:09.314587+0000","last_became_active":"2023-10-08T01:09:09.310389+0000","last_became_peered":"2023-10-08T01:09:09.310389+0000","last_unstale":"2023-10-08T01:09:09.314587+0000","last_undegraded":"2023-10-08T01:09:09.314587+0000","last_fullsized":"2023-10-08T01:09:09.314587+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:50:36.089765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.310560+0000","last_change":"2023-10-08T01:09:08.308607+0000","last_active":"2023-10-08T01:09:09.310560+0000","last_peered":"2023-10-08T01:09:09.310560+0000","last_clean":"2023-10-08T01:09:09.310560+0000","last_became_active":"2023-10-08T01:09:08.308305+0000","last_became_peered":"2023-10-08T01:09:08.308305+0000","last_unstale":"2023-10-08T01:09:09.310560+0000","last_undegraded":"2023-10-08T01:09:09.310560+0000","last_fullsized":"2023-10-08T01:09:09.310560+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:21:17.784900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.329617+0000","last_change":"2023-10-08T01:09:09.326662+0000","last_active":"2023-10-08T01:09:09.329617+0000","last_peered":"2023-10-08T01:09:09.329617+0000","last_clean":"2023-10-08T01:09:09.329617+0000","last_became_active":"2023-10-08T01:09:09.326407+0000","last_became_peered":"2023-10-08T01:09:09.326407+0000","last_unstale":"2023-10-08T01:09:09.329617+0000","last_undegraded":"2023-10-08T01:09:09.329617+0000","last_fullsized":"2023-10-08T01:09:09.329617+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:01.286642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.305324+0000","last_change":"2023-10-08T01:09:08.313396+0000","last_active":"2023-10-08T01:09:09.305324+0000","last_peered":"2023-10-08T01:09:09.305324+0000","last_clean":"2023-10-08T01:09:09.305324+0000","last_became_active":"2023-10-08T01:09:08.313087+0000","last_became_peered":"2023-10-08T01:09:08.313087+0000","last_unstale":"2023-10-08T01:09:09.305324+0000","last_undegraded":"2023-10-08T01:09:09.305324+0000","last_fullsized":"2023-10-08T01:09:09.305324+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:20:07.563912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.332594+0000","last_change":"2023-10-08T01:09:09.330360+0000","last_active":"2023-10-08T01:09:09.332594+0000","last_peered":"2023-10-08T01:09:09.332594+0000","last_clean":"2023-10-08T01:09:09.332594+0000","last_became_active":"2023-10-08T01:09:09.330100+0000","last_became_peered":"2023-10-08T01:09:09.330100+0000","last_unstale":"2023-10-08T01:09:09.332594+0000","last_undegraded":"2023-10-08T01:09:09.332594+0000","last_fullsized":"2023-10-08T01:09:09.332594+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:41.537708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.309062+0000","last_change":"2023-10-08T01:09:08.319584+0000","last_active":"2023-10-08T01:09:09.309062+0000","last_peered":"2023-10-08T01:09:09.309062+0000","last_clean":"2023-10-08T01:09:09.309062+0000","last_became_active":"2023-10-08T01:09:08.319313+0000","last_became_peered":"2023-10-08T01:09:08.319313+0000","last_unstale":"2023-10-08T01:09:09.309062+0000","last_undegraded":"2023-10-08T01:09:09.309062+0000","last_fullsized":"2023-10-08T01:09:09.309062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:37.522501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.327358+0000","last_change":"2023-10-08T01:09:09.326359+0000","last_active":"2023-10-08T01:09:09.327358+0000","last_peered":"2023-10-08T01:09:09.327358+0000","last_clean":"2023-10-08T01:09:09.327358+0000","last_became_active":"2023-10-08T01:09:09.326071+0000","last_became_peered":"2023-10-08T01:09:09.326071+0000","last_unstale":"2023-10-08T01:09:09.327358+0000","last_undegraded":"2023-10-08T01:09:09.327358+0000","last_fullsized":"2023-10-08T01:09:09.327358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:36.546997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.310602+0000","last_change":"2023-10-08T01:09:08.310261+0000","last_active":"2023-10-08T01:09:09.310602+0000","last_peered":"2023-10-08T01:09:09.310602+0000","last_clean":"2023-10-08T01:09:09.310602+0000","last_became_active":"2023-10-08T01:09:08.309985+0000","last_became_peered":"2023-10-08T01:09:08.309985+0000","last_unstale":"2023-10-08T01:09:09.310602+0000","last_undegraded":"2023-10-08T01:09:09.310602+0000","last_fullsized":"2023-10-08T01:09:09.310602+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:50.502468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.323376+0000","last_change":"2023-10-08T01:09:09.318228+0000","last_active":"2023-10-08T01:09:09.323376+0000","last_peered":"2023-10-08T01:09:09.323376+0000","last_clean":"2023-10-08T01:09:09.323376+0000","last_became_active":"2023-10-08T01:09:09.317951+0000","last_became_peered":"2023-10-08T01:09:09.317951+0000","last_unstale":"2023-10-08T01:09:09.323376+0000","last_undegraded":"2023-10-08T01:09:09.323376+0000","last_fullsized":"2023-10-08T01:09:09.323376+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:20.710780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.306503+0000","last_change":"2023-10-08T01:09:08.314603+0000","last_active":"2023-10-08T01:09:09.306503+0000","last_peered":"2023-10-08T01:09:09.306503+0000","last_clean":"2023-10-08T01:09:09.306503+0000","last_became_active":"2023-10-08T01:09:08.311835+0000","last_became_peered":"2023-10-08T01:09:08.311835+0000","last_unstale":"2023-10-08T01:09:09.306503+0000","last_undegraded":"2023-10-08T01:09:09.306503+0000","last_fullsized":"2023-10-08T01:09:09.306503+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:05:44.356964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.319131+0000","last_change":"2023-10-08T01:09:09.315799+0000","last_active":"2023-10-08T01:09:09.319131+0000","last_peered":"2023-10-08T01:09:09.319131+0000","last_clean":"2023-10-08T01:09:09.319131+0000","last_became_active":"2023-10-08T01:09:09.315509+0000","last_became_peered":"2023-10-08T01:09:09.315509+0000","last_unstale":"2023-10-08T01:09:09.319131+0000","last_undegraded":"2023-10-08T01:09:09.319131+0000","last_fullsized":"2023-10-08T01:09:09.319131+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:08.291329+0000","last_clean_scrub_stamp":"2023-10-08T01:09:08.291329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:47.187822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"58'2","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.342448+0000","last_change":"2023-10-08T01:09:08.314174+0000","last_active":"2023-10-08T01:09:09.342448+0000","last_peered":"2023-10-08T01:09:09.342448+0000","last_clean":"2023-10-08T01:09:09.342448+0000","last_became_active":"2023-10-08T01:09:08.313917+0000","last_became_peered":"2023-10-08T01:09:08.313917+0000","last_unstale":"2023-10-08T01:09:09.342448+0000","last_undegraded":"2023-10-08T01:09:09.342448+0000","last_fullsized":"2023-10-08T01:09:09.342448+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:09:07.280835+0000","last_clean_scrub_stamp":"2023-10-08T01:09:07.280835+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:26.267064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"48'139","reported_seq":276,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-10-08T01:09:09.303160+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:09:09.303160+0000","last_peered":"2023-10-08T01:09:09.303160+0000","last_clean":"2023-10-08T01:09:09.303160+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:09:09.303160+0000","last_undegraded":"2023-10-08T01:09:09.303160+0000","last_fullsized":"2023-10-08T01:09:09.303160+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575013,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302652,"kb_used_data":1268,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069188,"statfs":{"total":96636764160,"available":96326848512,"internally_reserved":0,"allocated":1298432,"data_stored":917197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.70099999999999996,"15min":0.68500000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.878,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.71199999999999997,"15min":0.69799999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90100000000000002,"5min":0.996,"15min":0.996},"last":0.84999999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.70699999999999996,"15min":0.68899999999999995},"min":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.87,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.79800000000000004,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.82899999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.75,"15min":0.72999999999999998},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.47199999999999998},"max":{"1min":0.92500000000000004,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.86799999999999999,"5min":1.054,"15min":1.054},"last":0.93000000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.71099999999999997,"15min":0.71399999999999997},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81999999999999995,"5min":0.91600000000000004,"15min":0.998},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.57499999999999996,"5min":0.52000000000000002,"15min":0.50600000000000001},"max":{"1min":0.95299999999999996,"5min":1.03,"15min":1.03},"last":0.94599999999999995}]},{"osd":4,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.40899999999999997,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.86899999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.621,"15min":0.624},"min":{"1min":0.44,"5min":0.439,"15min":0.40899999999999997},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":1.0780000000000001},"last":0.63900000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65500000000000003,"15min":0.64800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40000000000000002},"max":{"1min":0.86899999999999999,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67300000000000004,"15min":0.67700000000000005},"min":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.44800000000000001},"max":{"1min":0.91500000000000004,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.66500000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.58499999999999996,"5min":0.58099999999999996,"15min":0.40400000000000003},"max":{"1min":0.96599999999999997,"5min":1.01,"15min":1.1259999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72099999999999997,"15min":0.70999999999999996},"min":{"1min":0.50600000000000001,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.88200000000000001,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.80600000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575013,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302064,"kb_used_data":680,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069776,"statfs":{"total":96636764160,"available":96327450624,"internally_reserved":0,"allocated":696320,"data_stored":320731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.70399999999999996,"15min":0.69699999999999995},"min":{"1min":0.40400000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.872,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.68799999999999994,"15min":0.69299999999999995},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.85699999999999998,"5min":0.88700000000000001,"15min":1.093},"last":0.47199999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.70699999999999996,"15min":0.72699999999999998},"min":{"1min":0.40600000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":2.984},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.70599999999999996,"15min":0.72699999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.96899999999999997,"15min":3.0190000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.51400000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.84399999999999997,"5min":0.85499999999999998,"15min":1.9199999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.72099999999999997,"15min":0.73799999999999999},"min":{"1min":0.49299999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.4810000000000001},"last":0.63800000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.621},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.73799999999999999,"15min":0.73999999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.75900000000000001,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.55800000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.875,"5min":0.95599999999999996,"15min":1.163},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.64300000000000002,"15min":0.64500000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.361},"last":0.53400000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63400000000000001,"15min":0.63700000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68100000000000005,"15min":0.69499999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.443},"max":{"1min":1.21,"5min":1.21,"15min":1.4119999999999999},"last":0.72599999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.73799999999999999},"min":{"1min":0.501,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.88,"5min":0.98699999999999999,"15min":2.9489999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.749,"15min":0.747},"min":{"1min":0.47099999999999997,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.60499999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302072,"kb_used_data":688,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069768,"statfs":{"total":96636764160,"available":96327442432,"internally_reserved":0,"allocated":704512,"data_stored":326809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.47699999999999998,"15min":0.46300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.78900000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51600000000000001,"15min":0.495},"min":{"1min":0.33700000000000002,"5min":0.32700000000000001,"15min":0.28199999999999997},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.48099999999999998,"15min":0.47599999999999998},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.58899999999999997,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.37},{"interface":"front","average":{"1min":0.503,"5min":0.54300000000000004,"15min":0.53000000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.66100000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.58099999999999996,"15min":0.57199999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.76500000000000001,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.437},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.38900000000000001,"15min":0.312},"max":{"1min":1.5449999999999999,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.77600000000000002,"15min":0.76000000000000001},"min":{"1min":0.50600000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.452,"5min":1.452,"15min":1.661},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.747,"5min":0.76100000000000001,"15min":0.76900000000000002},"min":{"1min":0.51900000000000002,"5min":0.45100000000000001,"15min":0.434},"max":{"1min":1.069,"5min":1.069,"15min":1.5569999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76800000000000002,"15min":0.77100000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.127,"5min":1.127,"15min":1.409},"last":0.875},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.77200000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.79700000000000004,"15min":0.79400000000000004},"min":{"1min":0.497,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.617},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.79400000000000004,"15min":0.79200000000000004},"min":{"1min":0.56000000000000005,"5min":0.54000000000000004,"15min":0.53300000000000003},"max":{"1min":1.002,"5min":1.1020000000000001,"15min":1.4790000000000001},"last":0.83399999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:09:02 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84299999999999997,"15min":0.82999999999999996},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.87,"5min":0.84499999999999997,"15min":0.84199999999999997},"min":{"1min":0.624,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.86399999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298544,"kb_used_data":1256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073296,"statfs":{"total":96636764160,"available":96331055104,"internally_reserved":0,"allocated":1286144,"data_stored":910489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46100000000000002,"15min":0.45000000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.997},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.439,"5min":0.49299999999999999,"15min":0.48699999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.315},"max":{"1min":0.64400000000000002,"5min":0.73999999999999999,"15min":1.238},"last":0.36799999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.47799999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.439},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.52500000000000002,"15min":0.52400000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.64200000000000002,"5min":0.69999999999999996,"15min":1.7370000000000001},"last":0.41399999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34399999999999997},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":1.0489999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41799999999999998},"max":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.503}]},{"osd":4,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.755,"15min":0.754},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.45100000000000001},"max":{"1min":1.0009999999999999,"5min":1.1899999999999999,"15min":1.617},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.48899999999999999},"max":{"1min":0.95499999999999996,"5min":1.0700000000000001,"15min":1.4099999999999999},"last":0.72599999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.60599999999999998,"5min":0.57899999999999996,"15min":0.51000000000000001},"max":{"1min":1.2729999999999999,"5min":1.3440000000000001,"15min":1.5960000000000001},"last":0.745},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.81499999999999995,"15min":0.81200000000000006},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.45300000000000001},"max":{"1min":0.94899999999999995,"5min":1.302,"15min":1.5289999999999999},"last":0.73799999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.81299999999999994,"15min":0.79700000000000004},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.46600000000000003},"max":{"1min":1.0129999999999999,"5min":1.165,"15min":1.3779999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.51200000000000001},"max":{"1min":1.2390000000000001,"5min":1.2549999999999999,"15min":1.6459999999999999},"last":0.70399999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":1.0349999999999999,"5min":1.071,"15min":1.5740000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.84299999999999997,"15min":0.83399999999999996},"min":{"1min":0.64600000000000002,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.008,"5min":1.222,"15min":1.3160000000000001},"last":0.86599999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306180,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065660,"statfs":{"total":96636764160,"available":96323235840,"internally_reserved":0,"allocated":716800,"data_stored":328297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.68999999999999995,"15min":1.2689999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.52100000000000002,"15min":0.501},"min":{"1min":0.318,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.33800000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.505,"15min":0.47899999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.28399999999999997},"max":{"1min":2.786,"5min":2.786,"15min":2.786},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.56899999999999995,"15min":0.54500000000000004},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.50800000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60899999999999999,"15min":0.59599999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57399999999999995,"15min":0.55500000000000005},"min":{"1min":0.40899999999999997,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.76100000000000001,"15min":0.76600000000000001},"min":{"1min":0.53600000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.079,"5min":1.079,"15min":1.1180000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80900000000000005,"15min":0.78800000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":2.3559999999999999,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.83499999999999996,"15min":0.84099999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.81399999999999995,"15min":0.81599999999999995},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.755}]},{"osd":6,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.80800000000000005,"15min":0.79500000000000004},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.653,"5min":2.653,"15min":2.653},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.751,"5min":0.79800000000000004,"15min":0.79100000000000004},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.80200000000000005,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":2.7490000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.90400000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306176,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065664,"statfs":{"total":96636764160,"available":96323239936,"internally_reserved":0,"allocated":712704,"data_stored":327349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47299999999999998,"15min":0.45800000000000002},"min":{"1min":0.311,"5min":0.30499999999999999,"15min":0.249},"max":{"1min":0.69999999999999996,"5min":0.77600000000000002,"15min":0.80100000000000005},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.48099999999999998,"15min":0.47699999999999998},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.86599999999999999},"last":0.503}]},{"osd":1,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.46899999999999997,"15min":0.47399999999999998},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54200000000000004,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.51600000000000001,"15min":0.52600000000000002},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.66400000000000003,"5min":0.872,"15min":1.5629999999999999},"last":0.69199999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.40500000000000003,"5min":0.38700000000000001,"15min":0.34100000000000003},"max":{"1min":0.60099999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.63},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.63700000000000001,"5min":0.89000000000000001,"15min":1.3899999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.751,"15min":0.76400000000000001},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.93700000000000006,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.52400000000000002,"5min":0.5,"15min":0.46000000000000002},"max":{"1min":1.117,"5min":1.413,"15min":1.413},"last":0.73299999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.78100000000000003,"15min":0.77900000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.1899999999999999,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.77800000000000002,"15min":0.78400000000000003},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.94399999999999995,"5min":1.089,"15min":1.431},"last":0.74299999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.012,"5min":1.379,"15min":1.458},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80400000000000005,"15min":0.80000000000000004},"min":{"1min":0.58299999999999996,"5min":0.505,"15min":0.44500000000000001},"max":{"1min":1.1499999999999999,"5min":1.486,"15min":1.52},"last":0.84299999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:08:59 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.60099999999999998,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.042,"5min":1.389,"15min":1.389},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.80600000000000005,"15min":0.80500000000000005},"min":{"1min":0.57799999999999996,"5min":0.57299999999999995,"15min":0.5},"max":{"1min":0.97199999999999998,"5min":1.4730000000000001,"15min":1.4730000000000001},"last":0.80100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306176,"kb_used_data":696,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065664,"statfs":{"total":96636764160,"available":96323239936,"internally_reserved":0,"allocated":712704,"data_stored":328275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.68300000000000005},"min":{"1min":0.56999999999999995,"5min":0.35599999999999998,"15min":0.30099999999999999},"max":{"1min":0.84799999999999998,"5min":0.91900000000000004,"15min":0.95599999999999996},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.752,"5min":0.73299999999999998,"15min":0.68600000000000005},"min":{"1min":0.56399999999999995,"5min":0.45500000000000002,"15min":0.36599999999999999},"max":{"1min":0.94799999999999995,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66400000000000003}]},{"osd":1,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72299999999999998,"15min":0.68600000000000005},"min":{"1min":0.59099999999999997,"5min":0.48399999999999999,"15min":0.438},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.71399999999999997},"min":{"1min":0.51500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.93200000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.74099999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.77800000000000002,"15min":0.747},"min":{"1min":0.61099999999999999,"5min":0.502,"15min":0.45600000000000002},"max":{"1min":0.91000000000000003,"5min":1.002,"15min":1.024},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.79100000000000004,"15min":0.76100000000000001},"min":{"1min":0.68799999999999994,"5min":0.56399999999999995,"15min":0.45400000000000001},"max":{"1min":0.96799999999999997,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.70999999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79400000000000004,"15min":0.77500000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47999999999999998},"max":{"1min":0.99099999999999999,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.82299999999999995,"15min":0.79400000000000004},"min":{"1min":0.64000000000000001,"5min":0.53900000000000003,"15min":0.51800000000000002},"max":{"1min":0.91200000000000003,"5min":1.454,"15min":1.454},"last":0.77000000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61299999999999999,"15min":0.65900000000000003},"min":{"1min":0.377,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82699999999999996,"5min":0.95299999999999996,"15min":1.1459999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.747,"15min":0.76300000000000001},"min":{"1min":0.53700000000000003,"5min":0.52900000000000003,"15min":0.42399999999999999},"max":{"1min":0.86799999999999999,"5min":0.998,"15min":1.3420000000000001},"last":0.81000000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65000000000000002,"15min":0.67800000000000005},"min":{"1min":0.47499999999999998,"5min":0.433,"15min":0.40799999999999997},"max":{"1min":0.88,"5min":0.97699999999999998,"15min":1.1140000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65700000000000003,"15min":0.70299999999999996},"min":{"1min":0.43099999999999999,"5min":0.41299999999999998,"15min":0.40200000000000002},"max":{"1min":0.89500000000000002,"5min":0.94199999999999995,"15min":1.206},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67500000000000004,"15min":0.72399999999999998},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.89100000000000001,"5min":0.97699999999999998,"15min":1.5209999999999999},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.72299999999999998,"15min":0.75900000000000001},"min":{"1min":0.51000000000000001,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.86499999999999999,"5min":1.0049999999999999,"15min":1.26},"last":0.79900000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302060,"kb_used_data":676,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069780,"statfs":{"total":96636764160,"available":96327454720,"internally_reserved":0,"allocated":692224,"data_stored":320157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.68200000000000005,"15min":0.68799999999999994},"min":{"1min":0.499,"5min":0.42099999999999999,"15min":0.33700000000000002},"max":{"1min":0.874,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.73199999999999998,"15min":0.70699999999999996},"min":{"1min":0.502,"5min":0.502,"15min":0.38},"max":{"1min":0.85499999999999998,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.59299999999999997}]},{"osd":1,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.70699999999999996,"15min":0.71099999999999997},"min":{"1min":0.442,"5min":0.442,"15min":0.39500000000000002},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.01},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.626,"5min":0.68899999999999995,"15min":0.69199999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81000000000000005,"5min":0.92100000000000004,"15min":1.077},"last":0.68500000000000005}]},{"osd":2,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.71899999999999997,"15min":0.71099999999999997},"min":{"1min":0.52400000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.88600000000000001,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.745,"15min":0.72599999999999998},"min":{"1min":0.53600000000000003,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.90100000000000002,"5min":0.996,"15min":1.0449999999999999},"last":0.66200000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.91800000000000004,"5min":0.94399999999999995,"15min":1.1339999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.74199999999999999,"15min":0.753},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93000000000000005,"5min":0.95099999999999996,"15min":1.107},"last":0.622}]},{"osd":4,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59599999999999997,"15min":0.625},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.83299999999999996,"5min":0.86699999999999999,"15min":1.042},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66000000000000003,"15min":0.67300000000000004},"min":{"1min":0.51200000000000001,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":1.048,"5min":1.048,"15min":1.081},"last":0.71099999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65800000000000003,"15min":0.68100000000000005},"min":{"1min":0.48699999999999999,"5min":0.45300000000000001,"15min":0.40699999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.66900000000000004,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.80700000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:09:03 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.70099999999999996,"15min":0.70499999999999996},"min":{"1min":0.59999999999999998,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.165},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.71099999999999997,"15min":0.749},"min":{"1min":0.54200000000000004,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.024,"5min":1.024,"15min":1.3879999999999999},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:09:14.090 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T01:09:14.090 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T01:09:14.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.309+0000 7fe26f3ff700 1 Processor -- start 2023-10-08T01:09:14.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.309+0000 7fe26f3ff700 1 -- start start 2023-10-08T01:09:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.310+0000 7fe26f3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe268156620 0x7fe2680c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.310+0000 7fe26f3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe268068390 0x7fe268068830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.310+0000 7fe26f3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26815a6f0 con 0x7fe2681569f0 2023-10-08T01:09:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.310+0000 7fe26f3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2680c8d30 con 0x7fe268068390 2023-10-08T01:09:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.310+0000 7fe26f3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe268068d70 con 0x7fe268156620 2023-10-08T01:09:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe268068390 0x7fe268068830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26c99a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2681569f0 0x7fe2680ace70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44496/0 (socket says 172.21.15.119:44496) 2023-10-08T01:09:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe268068390 0x7fe268068830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55238/0 (socket says 172.21.15.119:55238) 2023-10-08T01:09:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d99c700 1 -- 172.21.15.119:0/2682418774 learned_addr learned my addr 172.21.15.119:0/2682418774 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d19b700 1 --2- 172.21.15.119:0/2682418774 >> v2:172.21.15.184:3300/0 conn(0x7fe268156620 0x7fe2680c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d99c700 1 -- 172.21.15.119:0/2682418774 >> v2:172.21.15.184:3300/0 conn(0x7fe268156620 msgr2=0x7fe2680c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d99c700 1 --2- 172.21.15.119:0/2682418774 >> v2:172.21.15.184:3300/0 conn(0x7fe268156620 0x7fe2680c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d99c700 1 -- 172.21.15.119:0/2682418774 >> v1:172.21.15.119:6789/0 conn(0x7fe2681569f0 legacy=0x7fe2680ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe25ffff700 1 -- 172.21.15.119:0/2682418774 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773891032 0 0) 0x7fe26815a6f0 con 0x7fe2681569f0 2023-10-08T01:09:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d99c700 1 -- 172.21.15.119:0/2682418774 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26815a9d0 con 0x7fe268068390 2023-10-08T01:09:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.311+0000 7fe26d99c700 1 --2- 172.21.15.119:0/2682418774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268068390 0x7fe268068830 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe2680c8f80 tx=0x7fe260009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afd1e91dbcfa3c90 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe25ffff700 1 -- 172.21.15.119:0/2682418774 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26000e070 con 0x7fe268068390 2023-10-08T01:09:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe25ffff700 1 -- 172.21.15.119:0/2682418774 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26000ba50 con 0x7fe268068390 2023-10-08T01:09:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe25ffff700 1 -- 172.21.15.119:0/2682418774 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2600092e0 con 0x7fe268068390 2023-10-08T01:09:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe26f3ff700 1 -- 172.21.15.119:0/2682418774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268068390 msgr2=0x7fe268068830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/2682418774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268068390 0x7fe268068830 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe2680c8f80 tx=0x7fe260009840 comp rx=0 tx=0).stop 2023-10-08T01:09:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe26f3ff700 1 -- 172.21.15.119:0/2682418774 shutdown_connections 2023-10-08T01:09:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/2682418774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268068390 0x7fe268068830 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/2682418774 >> v2:172.21.15.184:3300/0 conn(0x7fe268156620 0x7fe2680c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe26f3ff700 1 -- 172.21.15.119:0/2682418774 >> 172.21.15.119:0/2682418774 conn(0x7fe2680cc540 msgr2=0x7fe2680cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe26f3ff700 1 -- 172.21.15.119:0/2682418774 shutdown_connections 2023-10-08T01:09:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.312+0000 7fe26f3ff700 1 -- 172.21.15.119:0/2682418774 wait complete. 2023-10-08T01:09:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.314+0000 7fe26f3ff700 1 Processor -- start 2023-10-08T01:09:14.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.314+0000 7fe26f3ff700 1 -- start start 2023-10-08T01:09:14.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26f3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe268068390 0x7fe268155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26f3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268156620 0x7fe2680b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26f3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2681569f0 0x7fe268150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26f3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26814ff80 con 0x7fe268156620 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26f3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680b22e0 con 0x7fe268068390 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26f3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680b00e0 con 0x7fe2681569f0 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268156620 0x7fe2680b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268156620 0x7fe2680b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55246/0 (socket says 172.21.15.119:55246) 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26c99a700 1 -- 172.21.15.119:0/1409170584 learned_addr learned my addr 172.21.15.119:0/1409170584 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26d19b700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe268068390 0x7fe268155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26d99c700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2681569f0 0x7fe268150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26c99a700 1 -- 172.21.15.119:0/1409170584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2681569f0 msgr2=0x7fe268150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26c99a700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2681569f0 0x7fe268150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26c99a700 1 -- 172.21.15.119:0/1409170584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe268068390 msgr2=0x7fe268155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26c99a700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe268068390 0x7fe268155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.315+0000 7fe26c99a700 1 -- 172.21.15.119:0/1409170584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe260009ae0 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.316+0000 7fe26c99a700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268156620 0x7fe2680b1da0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe2580011e0 tx=0x7fe258007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.316+0000 7fe25d7fa700 1 -- 172.21.15.119:0/1409170584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258010260 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.316+0000 7fe25d7fa700 1 -- 172.21.15.119:0/1409170584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2580103c0 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.316+0000 7fe25d7fa700 1 -- 172.21.15.119:0/1409170584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258007320 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.316+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26814f930 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.316+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe268150950 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.317+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26814f770 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.318+0000 7fe25d7fa700 1 -- 172.21.15.119:0/1409170584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe25800a590 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.319+0000 7fe25d7fa700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe24c066f30 0x7fe24c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.319+0000 7fe25d7fa700 1 -- 172.21.15.119:0/1409170584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe258087ef0 con 0x7fe268156620 2023-10-08T01:09:14.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.319+0000 7fe26d19b700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe24c066f30 0x7fe24c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.320+0000 7fe26d19b700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe24c066f30 0x7fe24c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe264006fd0 tx=0x7fe264008040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:14.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.323+0000 7fe25d7fa700 1 -- 172.21.15.119:0/1409170584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe258054ff0 con 0x7fe268156620 2023-10-08T01:09:14.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.592+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe2680718b0 con 0x7fe268156620 2023-10-08T01:09:14.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.598+0000 7fe25d7fa700 1 -- 172.21.15.119:0/1409170584 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe25801f070 con 0x7fe268156620 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe24c066f30 msgr2=0x7fe24c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe24c066f30 0x7fe24c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe264006fd0 tx=0x7fe264008040 comp rx=0 tx=0).stop 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268156620 msgr2=0x7fe2680b1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268156620 0x7fe2680b1da0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fe2580011e0 tx=0x7fe258007e10 comp rx=0 tx=0).stop 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 shutdown_connections 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe24c066f30 0x7fe24c0693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2681569f0 0x7fe268150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe268068390 0x7fe268155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 --2- 172.21.15.119:0/1409170584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe268156620 0x7fe2680b1da0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 >> 172.21.15.119:0/1409170584 conn(0x7fe2680cc540 msgr2=0x7fe2680ad0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 shutdown_connections 2023-10-08T01:09:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.601+0000 7fe26f3ff700 1 -- 172.21.15.119:0/1409170584 wait complete. 2023-10-08T01:09:14.603 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T01:09:14.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:09:14.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.799+0000 7fb8f13ff700 1 Processor -- start 2023-10-08T01:09:14.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.799+0000 7fb8f13ff700 1 -- start start 2023-10-08T01:09:14.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.800+0000 7fb8f13ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8ec0cbf30 0x7fb8ec0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.800+0000 7fb8f13ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8ec0b1420 0x7fb8ec0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.800+0000 7fb8f13ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0cc300 con 0x7fb8ec0ccf90 2023-10-08T01:09:14.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.800+0000 7fb8f13ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec156bf0 con 0x7fb8ec0cbf30 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.800+0000 7fb8f13ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8ec0ae950 con 0x7fb8ec0b1420 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8eaffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8ec0cbf30 0x7fb8ec0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8ea7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb8ec0ccf90 0x7fb8ec0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44508/0 (socket says 172.21.15.119:44508) 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8eaffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8ec0cbf30 0x7fb8ec0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55254/0 (socket says 172.21.15.119:55254) 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8ea7fc700 1 -- 172.21.15.119:0/313177724 learned_addr learned my addr 172.21.15.119:0/313177724 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8eb7fe700 1 --2- 172.21.15.119:0/313177724 >> v2:172.21.15.184:3300/0 conn(0x7fb8ec0b1420 0x7fb8ec0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8eaffd700 1 -- 172.21.15.119:0/313177724 >> v2:172.21.15.184:3300/0 conn(0x7fb8ec0b1420 msgr2=0x7fb8ec0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8eaffd700 1 --2- 172.21.15.119:0/313177724 >> v2:172.21.15.184:3300/0 conn(0x7fb8ec0b1420 0x7fb8ec0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8eaffd700 1 -- 172.21.15.119:0/313177724 >> v1:172.21.15.119:6789/0 conn(0x7fb8ec0ccf90 legacy=0x7fb8ec0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8eaffd700 1 -- 172.21.15.119:0/313177724 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec0ad370 con 0x7fb8ec0cbf30 2023-10-08T01:09:14.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8e9ffb700 1 -- 172.21.15.119:0/313177724 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745971513 0 0) 0x7fb8ec0cc300 con 0x7fb8ec0ccf90 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.801+0000 7fb8eaffd700 1 --2- 172.21.15.119:0/313177724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0cbf30 0x7fb8ec0ae410 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0b19e0 tx=0x7fb8dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d50d4f4b6954c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8e9ffb700 1 -- 172.21.15.119:0/313177724 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc009e80 con 0x7fb8ec0cbf30 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8e9ffb700 1 -- 172.21.15.119:0/313177724 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc00b820 con 0x7fb8ec0cbf30 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8e9ffb700 1 -- 172.21.15.119:0/313177724 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00bba0 con 0x7fb8ec0cbf30 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8f13ff700 1 -- 172.21.15.119:0/313177724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0cbf30 msgr2=0x7fb8ec0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/313177724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0cbf30 0x7fb8ec0ae410 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec0b19e0 tx=0x7fb8dc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8f13ff700 1 -- 172.21.15.119:0/313177724 shutdown_connections 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/313177724 >> v2:172.21.15.184:3300/0 conn(0x7fb8ec0b1420 0x7fb8ec0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/313177724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0cbf30 0x7fb8ec0ae410 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8f13ff700 1 -- 172.21.15.119:0/313177724 >> 172.21.15.119:0/313177724 conn(0x7fb8ec06a320 msgr2=0x7fb8ec0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8f13ff700 1 -- 172.21.15.119:0/313177724 shutdown_connections 2023-10-08T01:09:14.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.802+0000 7fb8f13ff700 1 -- 172.21.15.119:0/313177724 wait complete. 2023-10-08T01:09:14.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.804+0000 7fb8f13ff700 1 Processor -- start 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.804+0000 7fb8f13ff700 1 -- start start 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8f13ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0b1420 0x7fb8ec1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8f13ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8ec0cbf30 0x7fb8ec15db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8f13ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8ec0ccf90 0x7fb8ec0a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8f13ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0ab4a0 con 0x7fb8ec0b1420 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8f13ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0aabe0 con 0x7fb8ec0cbf30 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8f13ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec0aae50 con 0x7fb8ec0ccf90 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0b1420 0x7fb8ec1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0b1420 0x7fb8ec1559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55262/0 (socket says 172.21.15.119:55262) 2023-10-08T01:09:14.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eaffd700 1 -- 172.21.15.119:0/2677652900 learned_addr learned my addr 172.21.15.119:0/2677652900 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8ea7fc700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8ec0cbf30 0x7fb8ec15db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eb7fe700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8ec0ccf90 0x7fb8ec0a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eaffd700 1 -- 172.21.15.119:0/2677652900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8ec0ccf90 msgr2=0x7fb8ec0a8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eaffd700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8ec0ccf90 0x7fb8ec0a8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eaffd700 1 -- 172.21.15.119:0/2677652900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8ec0cbf30 msgr2=0x7fb8ec15db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eaffd700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8ec0cbf30 0x7fb8ec15db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.805+0000 7fb8eaffd700 1 -- 172.21.15.119:0/2677652900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc009420 con 0x7fb8ec0b1420 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.806+0000 7fb8eaffd700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0b1420 0x7fb8ec1559e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00b3c0 tx=0x7fb8dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.806+0000 7fb8d2ffd700 1 -- 172.21.15.119:0/2677652900 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc011420 con 0x7fb8ec0b1420 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.806+0000 7fb8d2ffd700 1 -- 172.21.15.119:0/2677652900 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc0115c0 con 0x7fb8ec0b1420 2023-10-08T01:09:14.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.806+0000 7fb8d2ffd700 1 -- 172.21.15.119:0/2677652900 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc011980 con 0x7fb8ec0b1420 2023-10-08T01:09:14.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.806+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec0ab130 con 0x7fb8ec0b1420 2023-10-08T01:09:14.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.806+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec0a9470 con 0x7fb8ec0b1420 2023-10-08T01:09:14.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.807+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec0b08c0 con 0x7fb8ec0b1420 2023-10-08T01:09:14.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.808+0000 7fb8d2ffd700 1 -- 172.21.15.119:0/2677652900 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8dc011cf0 con 0x7fb8ec0b1420 2023-10-08T01:09:14.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.809+0000 7fb8d2ffd700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:14.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.809+0000 7fb8d2ffd700 1 -- 172.21.15.119:0/2677652900 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fb8dc0860e0 con 0x7fb8ec0b1420 2023-10-08T01:09:14.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.809+0000 7fb8ea7fc700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:14.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.810+0000 7fb8ea7fc700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fb8d8066f30 0x7fb8d80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c8f0 tx=0x7fb8e0009040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:14.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:14.813+0000 7fb8d2ffd700 1 -- 172.21.15.119:0/2677652900 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8dc053140 con 0x7fb8ec0b1420 2023-10-08T01:09:15.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.051+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8ec04c3c0 con 0x7fb8ec0b1420 2023-10-08T01:09:15.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.051+0000 7fb8d2ffd700 1 -- 172.21.15.119:0/2677652900 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fb8dc056760 con 0x7fb8ec0b1420 2023-10-08T01:09:15.053 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:15.053 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1019078588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1019078588},{"type":"v1","addr":"172.21.15.119:6835","nonce":1019078588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1353749238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1353749238},{"type":"v1","addr":"172.21.15.184:6834","nonce":1353749238}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/186953052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":186953052},{"type":"v1","addr":"172.21.15.184:6835","nonce":186953052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:09:08.771490+0000","modified":"2023-10-08T01:09:12.322278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/920039174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":920039174},{"type":"v1","addr":"172.21.15.119:6837","nonce":920039174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T01:09:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fb8d8066f30 msgr2=0x7fb8d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fb8d8066f30 0x7fb8d80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000c8f0 tx=0x7fb8e0009040 comp rx=0 tx=0).stop 2023-10-08T01:09:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0b1420 msgr2=0x7fb8ec1559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0b1420 0x7fb8ec1559e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00b3c0 tx=0x7fb8dc017000 comp rx=0 tx=0).stop 2023-10-08T01:09:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 shutdown_connections 2023-10-08T01:09:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fb8d8066f30 0x7fb8d80693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8ec0ccf90 0x7fb8ec0a8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8ec0b1420 0x7fb8ec1559e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 --2- 172.21.15.119:0/2677652900 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8ec0cbf30 0x7fb8ec15db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 >> 172.21.15.119:0/2677652900 conn(0x7fb8ec06a320 msgr2=0x7fb8ec06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:15.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 shutdown_connections 2023-10-08T01:09:15.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.054+0000 7fb8f13ff700 1 -- 172.21.15.119:0/2677652900 wait complete. 2023-10-08T01:09:15.056 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 24 2023-10-08T01:09:15.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T01:09:08.771490+0000', 'modified': '2023-10-08T01:09:12.322278+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5757}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5757': {'gid': 5757, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/920039174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6833', 'nonce': 920039174}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 920039174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T01:09:15.475 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T01:09:15.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T01:09:15.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.659+0000 7fd8bfdff700 1 Processor -- start 2023-10-08T01:09:15.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.659+0000 7fd8bfdff700 1 -- start start 2023-10-08T01:09:15.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.660+0000 7fd8bfdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b8159c60 0x7fd8b815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.660+0000 7fd8bfdff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd8b815db10 0x7fd8b8069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.660+0000 7fd8bfdff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b815a160 con 0x7fd8b8159890 2023-10-08T01:09:15.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.660+0000 7fd8bfdff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b80d13a0 con 0x7fd8b815db10 2023-10-08T01:09:15.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.660+0000 7fd8bfdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b815e140 con 0x7fd8b8159c60 2023-10-08T01:09:15.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.660+0000 7fd8bdb9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd8b8159890 0x7fd8b815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54670/0 (socket says 172.21.15.119:54670) 2023-10-08T01:09:15.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.660+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/2806417771 learned_addr learned my addr 172.21.15.119:0/2806417771 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:15.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/2806417771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b8159c60 0x7fd8b815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8be39c700 1 --2- 172.21.15.119:0/2806417771 >> v2:172.21.15.184:3300/0 conn(0x7fd8b815db10 0x7fd8b8069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8bcb99700 1 -- 172.21.15.119:0/2806417771 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1949146955 0 0) 0x7fd8b815a160 con 0x7fd8b8159890 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8bcb99700 1 -- 172.21.15.119:0/2806417771 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a4003570 con 0x7fd8b8159890 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8bcb99700 1 -- 172.21.15.119:0/2806417771 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2120994935 0 0) 0x7fd8a4003570 con 0x7fd8b8159890 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8be39c700 1 -- 172.21.15.119:0/2806417771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b8159c60 msgr2=0x7fd8b815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8be39c700 1 --2- 172.21.15.119:0/2806417771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b8159c60 0x7fd8b815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8be39c700 1 -- 172.21.15.119:0/2806417771 >> v1:172.21.15.119:6789/0 conn(0x7fd8b8159890 legacy=0x7fd8b815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8be39c700 1 -- 172.21.15.119:0/2806417771 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b815a3d0 con 0x7fd8b815db10 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.661+0000 7fd8be39c700 1 --2- 172.21.15.119:0/2806417771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b815db10 0x7fd8b8069560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009e60 tx=0x7fd8b0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e2480d4c47c5845 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bcb99700 1 -- 172.21.15.119:0/2806417771 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000e070 con 0x7fd8b815db10 2023-10-08T01:09:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bcb99700 1 -- 172.21.15.119:0/2806417771 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b000b7a0 con 0x7fd8b815db10 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bcb99700 1 -- 172.21.15.119:0/2806417771 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4001470 con 0x7fd8b815db10 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bfdff700 1 -- 172.21.15.119:0/2806417771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b815db10 msgr2=0x7fd8b8069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/2806417771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b815db10 0x7fd8b8069560 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0009e60 tx=0x7fd8b0009780 comp rx=0 tx=0).stop 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bfdff700 1 -- 172.21.15.119:0/2806417771 shutdown_connections 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/2806417771 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b815db10 0x7fd8b8069560 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/2806417771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b8159c60 0x7fd8b815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bfdff700 1 -- 172.21.15.119:0/2806417771 >> 172.21.15.119:0/2806417771 conn(0x7fd8b8068e20 msgr2=0x7fd8b80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bfdff700 1 -- 172.21.15.119:0/2806417771 shutdown_connections 2023-10-08T01:09:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.662+0000 7fd8bfdff700 1 -- 172.21.15.119:0/2806417771 wait complete. 2023-10-08T01:09:15.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.664+0000 7fd8bfdff700 1 Processor -- start 2023-10-08T01:09:15.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.664+0000 7fd8bfdff700 1 -- start start 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bfdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8b8159890 0x7fd8b814c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bfdff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b8159c60 0x7fd8b815c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bfdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b815db10 0x7fd8b815cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bfdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b80b3590 con 0x7fd8b8159890 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bfdff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b80cfe20 con 0x7fd8b8159c60 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bfdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b80cf210 con 0x7fd8b815db10 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bdb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8b8159890 0x7fd8b814c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bdb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8b8159890 0x7fd8b814c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46310/0 (socket says 172.21.15.119:46310) 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/102865949 learned_addr learned my addr 172.21.15.119:0/102865949 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8be39c700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b815db10 0x7fd8b815cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b8159c60 0x7fd8b815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/102865949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b815db10 msgr2=0x7fd8b815cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b815db10 0x7fd8b815cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/102865949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b8159c60 msgr2=0x7fd8b815c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b8159c60 0x7fd8b815c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.665+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/102865949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0009a20 con 0x7fd8b8159890 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.666+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8b8159890 0x7fd8b814c670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400cb90 tx=0x7fd8b4001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.666+0000 7fd8adffb700 1 -- 172.21.15.119:0/102865949 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4007e90 con 0x7fd8b8159890 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.666+0000 7fd8adffb700 1 -- 172.21.15.119:0/102865949 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4007360 con 0x7fd8b8159890 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.666+0000 7fd8bfdff700 1 -- 172.21.15.119:0/102865949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b80cf4f0 con 0x7fd8b8159890 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.666+0000 7fd8adffb700 1 -- 172.21.15.119:0/102865949 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4007700 con 0x7fd8b8159890 2023-10-08T01:09:15.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.666+0000 7fd8bfdff700 1 -- 172.21.15.119:0/102865949 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b80cfab0 con 0x7fd8b8159890 2023-10-08T01:09:15.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.668+0000 7fd8adffb700 1 -- 172.21.15.119:0/102865949 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8b4004a40 con 0x7fd8b8159890 2023-10-08T01:09:15.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.669+0000 7fd8adffb700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd89c066f60 0x7fd89c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.669+0000 7fd8adffb700 1 -- 172.21.15.119:0/102865949 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd8b4083ef0 con 0x7fd8b8159890 2023-10-08T01:09:15.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.669+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd89c066f60 0x7fd89c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.670+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd89c066f60 0x7fd89c069410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8b815a090 tx=0x7fd8a80091a0 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.671+0000 7fd8ce582b80 1 Processor -- start 2023-10-08T01:09:15.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.671+0000 7fd8ce582b80 1 -- start start 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f2235bd00 0x560f2233aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2230dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f22268a70 con 0x560f2233af60 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f22266270 con 0x560f2235bd00 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f22262060 con 0x560f22311a50 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8be39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2230dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8be39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2230dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46312/0 (socket says 172.21.15.119:46312) 2023-10-08T01:09:15.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8be39c700 1 -- 172.21.15.119:0/596395628 learned_addr learned my addr 172.21.15.119:0/596395628 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f2235bd00 0x560f2233aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8be39c700 1 -- 172.21.15.119:0/596395628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f22311a50 msgr2=0x560f22311e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8be39c700 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8be39c700 1 -- 172.21.15.119:0/596395628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f2235bd00 msgr2=0x560f2233aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8be39c700 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f2235bd00 0x560f2233aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.672+0000 7fd8be39c700 1 -- 172.21.15.119:0/596395628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f220fc270 con 0x560f2233af60 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.673+0000 7fd8be39c700 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2230dcf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00033d0 tx=0x7fd8b0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24cbf4e101449515 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.673+0000 7fd8a2ffd700 1 -- 172.21.15.119:0/596395628 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000be40 con 0x560f2233af60 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.673+0000 7fd8a2ffd700 1 -- 172.21.15.119:0/596395628 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b0023070 con 0x560f2233af60 2023-10-08T01:09:15.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.673+0000 7fd8a2ffd700 1 -- 172.21.15.119:0/596395628 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0028480 con 0x560f2233af60 2023-10-08T01:09:15.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 -- 172.21.15.119:0/596395628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 msgr2=0x560f2230dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2230dcf0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00033d0 tx=0x7fd8b0018000 comp rx=0 tx=0).stop 2023-10-08T01:09:15.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 -- 172.21.15.119:0/596395628 shutdown_connections 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2230dcf0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/596395628 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f2235bd00 0x560f2233aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 -- 172.21.15.119:0/596395628 >> 172.21.15.119:0/596395628 conn(0x560f22313370 msgr2=0x560f22313740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 -- 172.21.15.119:0/596395628 shutdown_connections 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 -- 172.21.15.119:0/596395628 wait complete. 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.674+0000 7fd8ce582b80 1 -- start start 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.675+0000 7fd8ce582b80 1 Processor -- start 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.675+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f2233e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2233bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2235bd00 0x560f2230cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2220d890 con 0x560f2233af60 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f22228ea0 con 0x560f22311a50 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f220e4e30 con 0x560f2235bd00 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8ce582b80 10 client.0 _ll_register_callbacks cb 0x560f222d5110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2233bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2233bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46314/0 (socket says 172.21.15.119:46314) 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 -- 172.21.15.119:0/203143275 learned_addr learned my addr 172.21.15.119:0/203143275 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8be39c700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2235bd00 0x560f2230cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f2233e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 -- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2235bd00 msgr2=0x560f2230cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2235bd00 0x560f2230cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 -- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 msgr2=0x560f2233e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f2233e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 -- 172.21.15.119:0/203143275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b815a3d0 con 0x560f2233af60 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f2233e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8be39c700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2235bd00 0x560f2230cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.676+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2233bdc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8014a10 tx=0x7fd8a8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=175f047f60c4e4fb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd897fff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd897fff700 1 -- 172.21.15.119:0/203143275 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8022710 con 0x560f2233af60 2023-10-08T01:09:15.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd897fff700 1 -- 172.21.15.119:0/203143275 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd8a801a040 con 0x560f2233af60 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 1 -- 172.21.15.119:0/203143275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f22306b70 con 0x560f2233af60 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd897fff700 1 -- 172.21.15.119:0/203143275 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8a8022a80 con 0x560f2233af60 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 10 client.5898 fetch_fsmap learned FSMap version 24 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 1 -- 172.21.15.119:0/203143275 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560f2209efe0 con 0x560f2233af60 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd897fff700 1 -- 172.21.15.119:0/203143275 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560f22306b70 con 0x560f2233af60 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd897fff700 1 -- 172.21.15.119:0/203143275 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd8a8022be0 con 0x560f2233af60 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 10 client.5898 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 10 client.5898 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 20 client.5898 populate_metadata read hostname 'smithi119' 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] conn(0x560f22353700 0x560f22353ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 4 client.5898 mds_command: new command op to 5757 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.677+0000 7fd8ce582b80 1 -- 172.21.15.119:0/203143275 --> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f222e3aa0 con 0x560f22353700 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.678+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] conn(0x560f22353700 0x560f22353ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.678+0000 7fd897fff700 10 client.5898 ms_handle_connect on v2:172.21.15.119:6833/920039174 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.678+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] conn(0x560f22353700 0x560f22353ad0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb7d1bf9b77ed85e server_cookie=f5c897dc31b5897d in_seq=0 out_seq=0 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd897fff700 1 -- 172.21.15.119:0/203143275 <== mds.0 v2:172.21.15.119:6833/920039174 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560f222e3aa0 con 0x560f22353700 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd897fff700 10 client.5898 handle_command_reply: tid=0 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 client.5898 shutdown 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 -- 172.21.15.119:0/203143275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 msgr2=0x560f2233bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2233bdc0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8014a10 tx=0x7fd8a8019000 comp rx=0 tx=0).stop 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 -- 172.21.15.119:0/203143275 shutdown_connections 2023-10-08T01:09:15.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] conn(0x560f22353700 0x560f22353ad0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2235bd00 0x560f2230cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f2233e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/203143275 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f2233af60 0x560f2233bdc0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd897fff700 0 client.5898 ms_handle_reset on v2:172.21.15.119:6833/920039174 2023-10-08T01:09:15.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 -- 172.21.15.119:0/203143275 >> 172.21.15.119:0/203143275 conn(0x560f22313370 msgr2=0x560f22357550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:15.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 -- 172.21.15.119:0/203143275 shutdown_connections 2023-10-08T01:09:15.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 1 -- 172.21.15.119:0/203143275 wait complete. 2023-10-08T01:09:15.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.679+0000 7fd8ce582b80 20 client.5898 trim_cache size 0 max 16384 2023-10-08T01:09:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.696+0000 7fd8ce582b80 1 Processor -- start 2023-10-08T01:09:15.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.696+0000 7fd8ce582b80 1 -- start start 2023-10-08T01:09:15.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.697+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22313370 0x560f22355f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.697+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.697+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 0x560f22314b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.697+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f2230c5a0 con 0x560f22313370 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.697+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f22339d30 con 0x560f22311a50 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.697+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f2233c120 con 0x560f2233af60 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22313370 0x560f22355f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22313370 0x560f22355f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46330/0 (socket says 172.21.15.119:46330) 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/530253646 learned_addr learned my addr 172.21.15.119:0/530253646 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8be39c700 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 0x560f22314b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 msgr2=0x560f22314b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 0x560f22314b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 msgr2=0x560f22311e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/530253646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f22230780 con 0x560f22313370 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22313370 0x560f22355f30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400a340 tx=0x7fd8b4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a3486f05462118 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8a37fe700 1 -- 172.21.15.119:0/530253646 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b405bc30 con 0x560f22313370 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8a37fe700 1 -- 172.21.15.119:0/530253646 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4052dd0 con 0x560f22313370 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.698+0000 7fd8a37fe700 1 -- 172.21.15.119:0/530253646 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4082310 con 0x560f22313370 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 -- 172.21.15.119:0/530253646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22313370 msgr2=0x560f22355f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22313370 0x560f22355f30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400a340 tx=0x7fd8b4014040 comp rx=0 tx=0).stop 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 -- 172.21.15.119:0/530253646 shutdown_connections 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 0x560f22314b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22313370 0x560f22355f30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/530253646 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22311a50 0x560f22311e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 -- 172.21.15.119:0/530253646 >> 172.21.15.119:0/530253646 conn(0x560f2233d9f0 msgr2=0x560f2230a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:15.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 -- 172.21.15.119:0/530253646 shutdown_connections 2023-10-08T01:09:15.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.699+0000 7fd8ce582b80 1 -- 172.21.15.119:0/530253646 wait complete. 2023-10-08T01:09:15.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.700+0000 7fd8ce582b80 1 -- start start 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.700+0000 7fd8ce582b80 1 Processor -- start 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22311a50 0x560f223b0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22313370 0x560f223c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8ce582b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 0x560f22357550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f22340d80 con 0x560f22311a50 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f223408b0 con 0x560f22313370 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8ce582b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f223c80e0 con 0x560f2233af60 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8ce582b80 10 client.0 _ll_register_callbacks cb 0x560f222d5110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:09:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bdb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22311a50 0x560f223b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bdb9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22311a50 0x560f223b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46334/0 (socket says 172.21.15.119:46334) 2023-10-08T01:09:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/3788386916 learned_addr learned my addr 172.21.15.119:0/3788386916 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8be39c700 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 0x560f22357550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22313370 0x560f223c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/3788386916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 msgr2=0x560f22357550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 0x560f22357550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/3788386916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22313370 msgr2=0x560f223c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22313370 0x560f223c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.701+0000 7fd8bdb9b700 1 -- 172.21.15.119:0/3788386916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b80cfab0 con 0x560f22311a50 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8bdb9b700 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22311a50 0x560f223b0360 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4054c20 tx=0x7fd8b401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3006d7fde2821dfc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd897fff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd897fff700 1 -- 172.21.15.119:0/3788386916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b405bc30 con 0x560f22311a50 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd897fff700 1 -- 172.21.15.119:0/3788386916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd8b40671b0 con 0x560f22311a50 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 1 -- 172.21.15.119:0/3788386916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8b80b24b0 con 0x560f22311a50 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd897fff700 1 -- 172.21.15.119:0/3788386916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd8b4082310 con 0x560f22311a50 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 10 client.5904 fetch_fsmap learned FSMap version 24 2023-10-08T01:09:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 1 -- 172.21.15.119:0/3788386916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x560f22217470 con 0x560f22311a50 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd897fff700 1 -- 172.21.15.119:0/3788386916 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b80b24b0 con 0x560f22311a50 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd897fff700 1 -- 172.21.15.119:0/3788386916 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 24) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd8b406c020 con 0x560f22311a50 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 10 client.5904 fetch_fsmap finished waiting for FSMap version 24 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 10 client.5904 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 20 client.5904 populate_metadata read hostname 'smithi119' 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] conn(0x560f2235bd00 0x560f223384a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 4 client.5904 mds_command: new command op to 5757 tid=0[{"prefix": "status"}] 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8ce582b80 1 -- 172.21.15.119:0/3788386916 --> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd8b0023070 con 0x560f2235bd00 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.702+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] conn(0x560f2235bd00 0x560f223384a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd897fff700 10 client.5904 ms_handle_connect on v2:172.21.15.119:6833/920039174 2023-10-08T01:09:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8bd39a700 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] conn(0x560f2235bd00 0x560f223384a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dcc9fd652f157b9 server_cookie=2959b92addb13525 in_seq=0 out_seq=0 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd897fff700 1 -- 172.21.15.119:0/3788386916 <== mds.0 v2:172.21.15.119:6833/920039174 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7fd8b0023070 con 0x560f2235bd00 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd897fff700 10 client.5904 handle_command_reply: tid=0 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 client.5904 shutdown 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 -- 172.21.15.119:0/3788386916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22311a50 msgr2=0x560f223b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22311a50 0x560f223b0360 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4054c20 tx=0x7fd8b401e000 comp rx=0 tx=0).stop 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 -- 172.21.15.119:0/3788386916 shutdown_connections 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.119:6833/920039174,v1:172.21.15.119:6837/920039174] conn(0x560f2235bd00 0x560f223384a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x560f2233af60 0x560f22357550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560f22311a50 0x560f223b0360 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 --2- 172.21.15.119:0/3788386916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x560f22313370 0x560f223c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd897fff700 0 client.5904 ms_handle_reset on v2:172.21.15.119:6833/920039174 2023-10-08T01:09:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.703+0000 7fd8ce582b80 1 -- 172.21.15.119:0/3788386916 >> 172.21.15.119:0/3788386916 conn(0x560f2233d9f0 msgr2=0x560f22353ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8ce582b80 1 -- 172.21.15.119:0/3788386916 shutdown_connections 2023-10-08T01:09:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8ce582b80 1 -- 172.21.15.119:0/3788386916 wait complete. 2023-10-08T01:09:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8ce582b80 20 client.5904 trim_cache size 0 max 16384 2023-10-08T01:09:15.712 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:09:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T01:09:15.712 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5757, 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.3961606700000004, 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 24, 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 59, 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 59, 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 13.19400139 2023-10-08T01:09:15.713 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:09:15.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 -- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd89c066f60 msgr2=0x7fd89c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd89c066f60 0x7fd89c069410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd8b815a090 tx=0x7fd8a80091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 -- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8b8159890 msgr2=0x7fd8b814c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8b8159890 0x7fd8b814c670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fd8b400cb90 tx=0x7fd8b4001b50 comp rx=0 tx=0).stop 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 -- 172.21.15.119:0/102865949 shutdown_connections 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd89c066f60 0x7fd89c069410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8b815db10 0x7fd8b815cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8b8159890 0x7fd8b814c670 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 --2- 172.21.15.119:0/102865949 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8b8159c60 0x7fd8b815c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.704+0000 7fd8bfdff700 1 -- 172.21.15.119:0/102865949 >> 172.21.15.119:0/102865949 conn(0x7fd8b8068e20 msgr2=0x7fd8b814d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.705+0000 7fd8bfdff700 1 -- 172.21.15.119:0/102865949 shutdown_connections 2023-10-08T01:09:15.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.705+0000 7fd8bfdff700 1 -- 172.21.15.119:0/102865949 wait complete. 2023-10-08T01:09:15.722 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T01:09:15.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.903+0000 7fdc393ff700 1 Processor -- start 2023-10-08T01:09:15.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.903+0000 7fdc393ff700 1 -- start start 2023-10-08T01:09:15.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.904+0000 7fdc393ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc34159c10 0x7fdc3415b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.904+0000 7fdc393ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc340b5ea0 0x7fdc34068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc393ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3415c480 con 0x7fdc34068bb0 2023-10-08T01:09:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc393ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc340b2a40 con 0x7fdc34159c10 2023-10-08T01:09:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc393ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3415b5d0 con 0x7fdc340b5ea0 2023-10-08T01:09:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc32ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc34159c10 0x7fdc3415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc337fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdc34068bb0 0x7fdc3415c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54674/0 (socket says 172.21.15.119:54674) 2023-10-08T01:09:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc32ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc34159c10 0x7fdc3415b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46346/0 (socket says 172.21.15.119:46346) 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc337fe700 1 -- 172.21.15.119:0/3056223061 learned_addr learned my addr 172.21.15.119:0/3056223061 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc327fc700 1 --2- 172.21.15.119:0/3056223061 >> v2:172.21.15.184:3300/0 conn(0x7fdc340b5ea0 0x7fdc34068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc32ffd700 1 -- 172.21.15.119:0/3056223061 >> v2:172.21.15.184:3300/0 conn(0x7fdc340b5ea0 msgr2=0x7fdc34068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc32ffd700 1 --2- 172.21.15.119:0/3056223061 >> v2:172.21.15.184:3300/0 conn(0x7fdc340b5ea0 0x7fdc34068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc32ffd700 1 -- 172.21.15.119:0/3056223061 >> v1:172.21.15.119:6789/0 conn(0x7fdc34068bb0 legacy=0x7fdc3415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc32ffd700 1 -- 172.21.15.119:0/3056223061 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3415b870 con 0x7fdc34159c10 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.905+0000 7fdc32ffd700 1 --2- 172.21.15.119:0/3056223061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34159c10 0x7fdc3415b090 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdc3409a150 tx=0x7fdc240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e466fe74ba136b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc31ffb700 1 -- 172.21.15.119:0/3056223061 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400b6b0 con 0x7fdc34159c10 2023-10-08T01:09:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc31ffb700 1 -- 172.21.15.119:0/3056223061 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2400b810 con 0x7fdc34159c10 2023-10-08T01:09:15.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc31ffb700 1 -- 172.21.15.119:0/3056223061 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2400bbb0 con 0x7fdc34159c10 2023-10-08T01:09:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc393ff700 1 -- 172.21.15.119:0/3056223061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34159c10 msgr2=0x7fdc3415b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc393ff700 1 --2- 172.21.15.119:0/3056223061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34159c10 0x7fdc3415b090 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fdc3409a150 tx=0x7fdc240091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc393ff700 1 -- 172.21.15.119:0/3056223061 shutdown_connections 2023-10-08T01:09:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc393ff700 1 --2- 172.21.15.119:0/3056223061 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34159c10 0x7fdc3415b090 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc393ff700 1 --2- 172.21.15.119:0/3056223061 >> v2:172.21.15.184:3300/0 conn(0x7fdc340b5ea0 0x7fdc34068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc393ff700 1 -- 172.21.15.119:0/3056223061 >> 172.21.15.119:0/3056223061 conn(0x7fdc34099a20 msgr2=0x7fdc340af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc393ff700 1 -- 172.21.15.119:0/3056223061 shutdown_connections 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.906+0000 7fdc393ff700 1 -- 172.21.15.119:0/3056223061 wait complete. 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.908+0000 7fdc393ff700 1 Processor -- start 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.908+0000 7fdc393ff700 1 -- start start 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc393ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34068bb0 0x7fdc340b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc393ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc340b5ea0 0x7fdc340c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc393ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc34159c10 0x7fdc340c8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc393ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3416a9c0 con 0x7fdc34068bb0 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc393ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3416ac10 con 0x7fdc34159c10 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc393ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3416aef0 con 0x7fdc340b5ea0 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc32ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34068bb0 0x7fdc340b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc32ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34068bb0 0x7fdc340b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46350/0 (socket says 172.21.15.119:46350) 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc32ffd700 1 -- 172.21.15.119:0/848550565 learned_addr learned my addr 172.21.15.119:0/848550565 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc32ffd700 1 -- 172.21.15.119:0/848550565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc340b5ea0 msgr2=0x7fdc340c8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc32ffd700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc340b5ea0 0x7fdc340c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc32ffd700 1 -- 172.21.15.119:0/848550565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc34159c10 msgr2=0x7fdc340c8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc337fe700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc34159c10 0x7fdc340c8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc32ffd700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc34159c10 0x7fdc340c8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc32ffd700 1 -- 172.21.15.119:0/848550565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009420 con 0x7fdc34068bb0 2023-10-08T01:09:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.909+0000 7fdc337fe700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc34159c10 0x7fdc340c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.910+0000 7fdc32ffd700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34068bb0 0x7fdc340b0a20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdc34063db0 tx=0x7fdc24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.910+0000 7fdc1affd700 1 -- 172.21.15.119:0/848550565 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24011420 con 0x7fdc34068bb0 2023-10-08T01:09:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.910+0000 7fdc1affd700 1 -- 172.21.15.119:0/848550565 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc24011580 con 0x7fdc34068bb0 2023-10-08T01:09:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.910+0000 7fdc1affd700 1 -- 172.21.15.119:0/848550565 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24011940 con 0x7fdc34068bb0 2023-10-08T01:09:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.910+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3416b1d0 con 0x7fdc34068bb0 2023-10-08T01:09:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.910+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc340b0f60 con 0x7fdc34068bb0 2023-10-08T01:09:15.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.911+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3416b6f0 con 0x7fdc34068bb0 2023-10-08T01:09:15.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.912+0000 7fdc1affd700 1 -- 172.21.15.119:0/848550565 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc24011d30 con 0x7fdc34068bb0 2023-10-08T01:09:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.913+0000 7fdc1affd700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdc20066f30 0x7fdc200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.913+0000 7fdc1affd700 1 -- 172.21.15.119:0/848550565 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fdc24086110 con 0x7fdc34068bb0 2023-10-08T01:09:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.913+0000 7fdc327fc700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdc20066f30 0x7fdc200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.914+0000 7fdc327fc700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdc20066f30 0x7fdc200693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc340d1280 tx=0x7fdc28008040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:15.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:15.917+0000 7fdc1affd700 1 -- 172.21.15.119:0/848550565 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc3416b6f0 con 0x7fdc34068bb0 2023-10-08T01:09:16.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.152+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc3416b6f0 con 0x7fdc34068bb0 2023-10-08T01:09:16.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.153+0000 7fdc1affd700 1 -- 172.21.15.119:0/848550565 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc3416b6f0 con 0x7fdc34068bb0 2023-10-08T01:09:16.154 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T01:09:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdc20066f30 msgr2=0x7fdc200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdc20066f30 0x7fdc200693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdc340d1280 tx=0x7fdc28008040 comp rx=0 tx=0).stop 2023-10-08T01:09:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34068bb0 msgr2=0x7fdc340b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34068bb0 0x7fdc340b0a20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fdc34063db0 tx=0x7fdc24017000 comp rx=0 tx=0).stop 2023-10-08T01:09:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 shutdown_connections 2023-10-08T01:09:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdc20066f30 0x7fdc200693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc34159c10 0x7fdc340c8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc34068bb0 0x7fdc340b0a20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 --2- 172.21.15.119:0/848550565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc340b5ea0 0x7fdc340c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 >> 172.21.15.119:0/848550565 conn(0x7fdc34099a20 msgr2=0x7fdc3409a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:16.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 shutdown_connections 2023-10-08T01:09:16.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:16.155+0000 7fdc393ff700 1 -- 172.21.15.119:0/848550565 wait complete. 2023-10-08T01:09:16.173 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T01:09:16.173 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T01:09:16.173 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T01:09:16.173 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T01:09:16.174 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T01:09:16.174 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T01:09:16.174 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T01:09:16.174 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T01:09:16.174 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:09:16.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:09:16.174 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T01:09:16.211 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T01:09:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T01:09:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T01:09:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:09:16.211 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T01:09:16.212 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41495sec preferred_lft 41495sec 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:09:16.213 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T01:09:16.214 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:09:16.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:09:16.215 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T01:09:16.310 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T01:09:16.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:09:16.314 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T01:09:16.354 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T01:09:16.354 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T01:09:16.354 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T01:09:16.354 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T01:09:16.354 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T01:09:16.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:09:16.355 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T01:09:16.355 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T01:09:16.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T01:09:16.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T01:09:16.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T01:09:16.355 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T01:09:16.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:09:16.550 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T01:09:16.578 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:09:16.586 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T01:09:16.586 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T01:09:16.586 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:09:16.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:09:16.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T01:09:16.630 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:09:16.692 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:09:16.692 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T01:09:16.799 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:09:16.799 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:09:16.800 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:09:16.875 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T01:09:16.875 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T01:09:16.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:09:17.006 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:09:17.006 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:09:17.035 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:09:17.035 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:09:17.035 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T01:09:17.035 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T01:09:17.035 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T01:09:17.039 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T01:09:17.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:09:17.040 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:09:17.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T01:09:17.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.282+0000 7f732a19d700 1 Processor -- start 2023-10-08T01:09:17.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.282+0000 7f732a19d700 1 -- start start 2023-10-08T01:09:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.283+0000 7f732a19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f73241595a0 0x7f73240b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.283+0000 7f732a19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c8830 0x7f7324159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.283+0000 7f732a19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73240b1410 con 0x7f73240c6c60 2023-10-08T01:09:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.283+0000 7f732a19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73240b2b90 con 0x7f73241595a0 2023-10-08T01:09:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.283+0000 7f732a19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7324063230 con 0x7f73240c8830 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.284+0000 7f732999c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c8830 0x7f7324159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.284+0000 7f732999c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c8830 0x7f7324159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:59820/0 (socket says 172.21.15.119:59820) 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.284+0000 7f732999c700 1 -- 172.21.15.119:0/3587021280 learned_addr learned my addr 172.21.15.119:0/3587021280 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.284+0000 7f732919b700 1 --2- 172.21.15.119:0/3587021280 >> v2:172.21.15.184:3300/0 conn(0x7f73241595a0 0x7f73240b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.284+0000 7f732999c700 1 -- 172.21.15.119:0/3587021280 >> v2:172.21.15.184:3300/0 conn(0x7f73241595a0 msgr2=0x7f73240b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.284+0000 7f732999c700 1 --2- 172.21.15.119:0/3587021280 >> v2:172.21.15.184:3300/0 conn(0x7f73241595a0 0x7f73240b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.284+0000 7f732999c700 1 -- 172.21.15.119:0/3587021280 >> v1:172.21.15.184:6789/0 conn(0x7f73240c6c60 legacy=0x7f73240c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.284+0000 7f732999c700 1 -- 172.21.15.119:0/3587021280 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73240634f0 con 0x7f73240c8830 2023-10-08T01:09:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f732999c700 1 --2- 172.21.15.119:0/3587021280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c8830 0x7f7324159970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f731c009f20 tx=0x7f731c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c879a5d3b30887c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:17.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f731bfff700 1 -- 172.21.15.119:0/3587021280 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00e070 con 0x7f73240c8830 2023-10-08T01:09:17.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f731bfff700 1 -- 172.21.15.119:0/3587021280 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c00b9d0 con 0x7f73240c8830 2023-10-08T01:09:17.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f731bfff700 1 -- 172.21.15.119:0/3587021280 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0092e0 con 0x7f73240c8830 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f732a19d700 1 -- 172.21.15.119:0/3587021280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c8830 msgr2=0x7f7324159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f732a19d700 1 --2- 172.21.15.119:0/3587021280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c8830 0x7f7324159970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f731c009f20 tx=0x7f731c009840 comp rx=0 tx=0).stop 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f732a19d700 1 -- 172.21.15.119:0/3587021280 shutdown_connections 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f732a19d700 1 --2- 172.21.15.119:0/3587021280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c8830 0x7f7324159970 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f732a19d700 1 --2- 172.21.15.119:0/3587021280 >> v2:172.21.15.184:3300/0 conn(0x7f73241595a0 0x7f73240b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.285+0000 7f732a19d700 1 -- 172.21.15.119:0/3587021280 >> 172.21.15.119:0/3587021280 conn(0x7f732406ddb0 msgr2=0x7f732406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.286+0000 7f732a19d700 1 -- 172.21.15.119:0/3587021280 shutdown_connections 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.286+0000 7f732a19d700 1 -- 172.21.15.119:0/3587021280 wait complete. 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.287+0000 7f732a19d700 1 Processor -- start 2023-10-08T01:09:17.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.287+0000 7f732a19d700 1 -- start start 2023-10-08T01:09:17.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732a19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c6c60 0x7f73240c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732a19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73240c8830 0x7f73240b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732a19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73241595a0 0x7f73240b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732a19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324140e20 con 0x7f73240c8830 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732a19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240c6620 con 0x7f73241595a0 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732a19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73240c36e0 con 0x7f73240c6c60 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732899a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73240c8830 0x7f73240b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732899a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73240c8830 0x7f73240b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46358/0 (socket says 172.21.15.119:46358) 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.288+0000 7f732899a700 1 -- 172.21.15.119:0/606913917 learned_addr learned my addr 172.21.15.119:0/606913917 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732919b700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c6c60 0x7f73240c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732999c700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73241595a0 0x7f73240b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732899a700 1 -- 172.21.15.119:0/606913917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c6c60 msgr2=0x7f73240c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732899a700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c6c60 0x7f73240c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732899a700 1 -- 172.21.15.119:0/606913917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73241595a0 msgr2=0x7f73240b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732899a700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73241595a0 0x7f73240b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732899a700 1 -- 172.21.15.119:0/606913917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c009ae0 con 0x7f73240c8830 2023-10-08T01:09:17.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732899a700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73240c8830 0x7f73240b17a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f731400c7c0 tx=0x7f7314007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:17.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f73197fa700 1 -- 172.21.15.119:0/606913917 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314011590 con 0x7f73240c8830 2023-10-08T01:09:17.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f73197fa700 1 -- 172.21.15.119:0/606913917 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73140116f0 con 0x7f73240c8830 2023-10-08T01:09:17.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f73197fa700 1 -- 172.21.15.119:0/606913917 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314011ab0 con 0x7f73240c8830 2023-10-08T01:09:17.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.289+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7324159e00 con 0x7f73240c8830 2023-10-08T01:09:17.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.290+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732415a3c0 con 0x7f73240c8830 2023-10-08T01:09:17.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.290+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732404c3c0 con 0x7f73240c8830 2023-10-08T01:09:17.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.291+0000 7f73197fa700 1 -- 172.21.15.119:0/606913917 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f731401b3b0 con 0x7f73240c8830 2023-10-08T01:09:17.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.292+0000 7f73197fa700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7308066f30 0x7f73080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.292+0000 7f73197fa700 1 -- 172.21.15.119:0/606913917 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f7314083b60 con 0x7f73240c8830 2023-10-08T01:09:17.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.292+0000 7f732919b700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7308066f30 0x7f73080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.293+0000 7f732919b700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7308066f30 0x7f73080693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7320009d10 tx=0x7f73200091a0 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:17.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.296+0000 7f73197fa700 1 -- 172.21.15.119:0/606913917 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7314050bc0 con 0x7f73240c8830 2023-10-08T01:09:17.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.517+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 --> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f73240637d0 con 0x7f7308066f30 2023-10-08T01:09:17.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.519+0000 7f73197fa700 1 -- 172.21.15.119:0/606913917 <== mgr.5571 v2:172.21.15.119:6834/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f73240637d0 con 0x7f7308066f30 2023-10-08T01:09:17.520 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:09:17.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:09:17.520 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T01:09:17.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:09:17.520 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:09:17.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7308066f30 msgr2=0x7f73080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7308066f30 0x7f73080693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7320009d10 tx=0x7f73200091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73240c8830 msgr2=0x7f73240b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73240c8830 0x7f73240b17a0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f731400c7c0 tx=0x7f7314007f50 comp rx=0 tx=0).stop 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 shutdown_connections 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f7308066f30 0x7f73080693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73240c6c60 0x7f73240c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73241595a0 0x7f73240b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 --2- 172.21.15.119:0/606913917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73240c8830 0x7f73240b17a0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 >> 172.21.15.119:0/606913917 conn(0x7f732406ddb0 msgr2=0x7f732415c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 shutdown_connections 2023-10-08T01:09:17.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.521+0000 7f732a19d700 1 -- 172.21.15.119:0/606913917 wait complete. 2023-10-08T01:09:17.540 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T01:09:17.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.722+0000 7f29257ff700 1 Processor -- start 2023-10-08T01:09:17.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.722+0000 7f29257ff700 1 -- start start 2023-10-08T01:09:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f29257ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2920069770 0x7f2920069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f29257ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292006e950 con 0x7f2920155eb0 2023-10-08T01:09:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f29257ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2920156320 con 0x7f29200c70b0 2023-10-08T01:09:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f29257ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29200cd350 con 0x7f2920069770 2023-10-08T01:09:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f2917fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2920155eb0 0x7f292006e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54676/0 (socket says 172.21.15.119:54676) 2023-10-08T01:09:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f2917fff700 1 -- 172.21.15.119:0/3245154098 learned_addr learned my addr 172.21.15.119:0/3245154098 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:17.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f291f7fe700 1 --2- 172.21.15.119:0/3245154098 >> v2:172.21.15.119:3300/0 conn(0x7f2920069770 0x7f2920069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f291e7fc700 1 -- 172.21.15.119:0/3245154098 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884968872 0 0) 0x7f2908001110 con 0x7f2920155eb0 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f291f7fe700 1 -- 172.21.15.119:0/3245154098 >> v1:172.21.15.184:6789/0 conn(0x7f29200c70b0 legacy=0x7f29200ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f291f7fe700 1 -- 172.21.15.119:0/3245154098 >> v1:172.21.15.119:6789/0 conn(0x7f2920155eb0 legacy=0x7f292006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.724+0000 7f291f7fe700 1 -- 172.21.15.119:0/3245154098 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292006eb90 con 0x7f2920069770 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f291f7fe700 1 --2- 172.21.15.119:0/3245154098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2920069770 0x7f2920069bf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2918009e60 tx=0x7f2918009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d6d8ef631fa5e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f291e7fc700 1 -- 172.21.15.119:0/3245154098 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291800bbf0 con 0x7f2920069770 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f291e7fc700 1 -- 172.21.15.119:0/3245154098 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291800bd50 con 0x7f2920069770 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f291e7fc700 1 -- 172.21.15.119:0/3245154098 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29180116b0 con 0x7f2920069770 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f29257ff700 1 -- 172.21.15.119:0/3245154098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2920069770 msgr2=0x7f2920069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f29257ff700 1 --2- 172.21.15.119:0/3245154098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2920069770 0x7f2920069bf0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2918009e60 tx=0x7f2918009780 comp rx=0 tx=0).stop 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f29257ff700 1 -- 172.21.15.119:0/3245154098 shutdown_connections 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f29257ff700 1 --2- 172.21.15.119:0/3245154098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2920069770 0x7f2920069bf0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f29257ff700 1 -- 172.21.15.119:0/3245154098 >> 172.21.15.119:0/3245154098 conn(0x7f2920068e10 msgr2=0x7f29200cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:17.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f29257ff700 1 -- 172.21.15.119:0/3245154098 shutdown_connections 2023-10-08T01:09:17.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.725+0000 7f29257ff700 1 -- 172.21.15.119:0/3245154098 wait complete. 2023-10-08T01:09:17.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.727+0000 7f29257ff700 1 Processor -- start 2023-10-08T01:09:17.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.727+0000 7f29257ff700 1 -- start start 2023-10-08T01:09:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f29257ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920069770 0x7f29200ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f29257ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200c70b0 0x7f29200c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f29257ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2920155eb0 0x7f29200c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f29257ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200c5720 con 0x7f29200c70b0 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f29257ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29200c5a00 con 0x7f2920069770 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f29257ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29200c5ce0 con 0x7f2920155eb0 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f2917fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200c70b0 0x7f29200c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f2917fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200c70b0 0x7f29200c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46372/0 (socket says 172.21.15.119:46372) 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f2917fff700 1 -- 172.21.15.119:0/554459889 learned_addr learned my addr 172.21.15.119:0/554459889 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f291effd700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920069770 0x7f29200ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f2917fff700 1 -- 172.21.15.119:0/554459889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2920155eb0 msgr2=0x7f29200c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f2917fff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2920155eb0 0x7f29200c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f2917fff700 1 -- 172.21.15.119:0/554459889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920069770 msgr2=0x7f29200ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:17.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f2917fff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920069770 0x7f29200ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f2917fff700 1 -- 172.21.15.119:0/554459889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918009a20 con 0x7f29200c70b0 2023-10-08T01:09:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.728+0000 7f291effd700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920069770 0x7f29200ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.729+0000 7f2917fff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200c70b0 0x7f29200c5020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f290800c5b0 tx=0x7f2908001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.729+0000 7f2916ffd700 1 -- 172.21.15.119:0/554459889 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908005b30 con 0x7f29200c70b0 2023-10-08T01:09:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.729+0000 7f2916ffd700 1 -- 172.21.15.119:0/554459889 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908005c90 con 0x7f29200c70b0 2023-10-08T01:09:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.729+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29200b0310 con 0x7f29200c70b0 2023-10-08T01:09:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.729+0000 7f2916ffd700 1 -- 172.21.15.119:0/554459889 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908007300 con 0x7f29200c70b0 2023-10-08T01:09:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.729+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29200c8140 con 0x7f29200c70b0 2023-10-08T01:09:17.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.730+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29200b0170 con 0x7f29200c70b0 2023-10-08T01:09:17.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.731+0000 7f2916ffd700 1 -- 172.21.15.119:0/554459889 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 21) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f290801b3b0 con 0x7f29200c70b0 2023-10-08T01:09:17.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.732+0000 7f2916ffd700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2900066f30 0x7f29000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:17.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.732+0000 7f2916ffd700 1 -- 172.21.15.119:0/554459889 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f2908083b00 con 0x7f29200c70b0 2023-10-08T01:09:17.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.732+0000 7f291effd700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2900066f30 0x7f29000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:17.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.732+0000 7f291effd700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2900066f30 0x7f29000693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2910002880 tx=0x7f2910009040 comp rx=0 tx=0).ready entity=mgr.5571 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:17.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.736+0000 7f2916ffd700 1 -- 172.21.15.119:0/554459889 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29200b0170 con 0x7f29200c70b0 2023-10-08T01:09:17.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:17.978+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f292006e950 con 0x7f29200c70b0 2023-10-08T01:09:18.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.362+0000 7f2916ffd700 1 -- 172.21.15.119:0/554459889 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 22) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f2908050f30 con 0x7f29200c70b0 2023-10-08T01:09:18.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.363+0000 7f2916ffd700 1 -- 172.21.15.119:0/554459889 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v22) v1 ==== 92+0+0 (secure 0 0 0) 0x7f29080545f0 con 0x7f29200c70b0 2023-10-08T01:09:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2900066f30 msgr2=0x7f29000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2900066f30 0x7f29000693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2910002880 tx=0x7f2910009040 comp rx=0 tx=0).stop 2023-10-08T01:09:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200c70b0 msgr2=0x7f29200c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200c70b0 0x7f29200c5020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f290800c5b0 tx=0x7f2908001070 comp rx=0 tx=0).stop 2023-10-08T01:09:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 shutdown_connections 2023-10-08T01:09:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f2900066f30 0x7f29000693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2920155eb0 0x7f29200c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2920069770 0x7f29200ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 --2- 172.21.15.119:0/554459889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29200c70b0 0x7f29200c5020 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 >> 172.21.15.119:0/554459889 conn(0x7f2920068e10 msgr2=0x7f29200c2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:18.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 shutdown_connections 2023-10-08T01:09:18.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.366+0000 7f29257ff700 1 -- 172.21.15.119:0/554459889 wait complete. 2023-10-08T01:09:18.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T01:09:18.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.584+0000 7fd4db59d700 1 Processor -- start 2023-10-08T01:09:18.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.584+0000 7fd4db59d700 1 -- start start 2023-10-08T01:09:18.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.585+0000 7fd4db59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 0x7fd4d415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:18.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.585+0000 7fd4db59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4d40aef00 0x7fd4d415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:18.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.585+0000 7fd4db59d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d40aea70 con 0x7fd4d406e470 2023-10-08T01:09:18.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.585+0000 7fd4db59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40ae480 con 0x7fd4d40aef00 2023-10-08T01:09:18.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.585+0000 7fd4db59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d415acc0 con 0x7fd4d415a3b0 2023-10-08T01:09:18.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9d9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4d40aef00 0x7fd4d415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:18.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9d9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4d40aef00 0x7fd4d415d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46400/0 (socket says 172.21.15.119:46400) 2023-10-08T01:09:18.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9d9a700 1 -- 172.21.15.119:0/2677419594 learned_addr learned my addr 172.21.15.119:0/2677419594 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:18.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4da59b700 1 --2- 172.21.15.119:0/2677419594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 0x7fd4d415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:18.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9d9a700 1 -- 172.21.15.119:0/2677419594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 msgr2=0x7fd4d415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9d9a700 1 --2- 172.21.15.119:0/2677419594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 0x7fd4d415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9d9a700 1 -- 172.21.15.119:0/2677419594 >> v1:172.21.15.184:6789/0 conn(0x7fd4d406e470 legacy=0x7fd4d40c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9d9a700 1 -- 172.21.15.119:0/2677419594 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d415dd30 con 0x7fd4d40aef00 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9d9a700 1 --2- 172.21.15.119:0/2677419594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40aef00 0x7fd4d415d3c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40099b0 tx=0x7fd4c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33d4e41bec44fc8f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.586+0000 7fd4d9599700 1 -- 172.21.15.119:0/2677419594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4009ea0 con 0x7fd4d40aef00 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4d9599700 1 -- 172.21.15.119:0/2677419594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c400b860 con 0x7fd4d40aef00 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4d9599700 1 -- 172.21.15.119:0/2677419594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400bc00 con 0x7fd4d40aef00 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4db59d700 1 -- 172.21.15.119:0/2677419594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40aef00 msgr2=0x7fd4d415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4db59d700 1 --2- 172.21.15.119:0/2677419594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40aef00 0x7fd4d415d3c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd4c40099b0 tx=0x7fd4c4009250 comp rx=0 tx=0).stop 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4db59d700 1 -- 172.21.15.119:0/2677419594 shutdown_connections 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4db59d700 1 --2- 172.21.15.119:0/2677419594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 0x7fd4d415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4db59d700 1 --2- 172.21.15.119:0/2677419594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d40aef00 0x7fd4d415d3c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4db59d700 1 -- 172.21.15.119:0/2677419594 >> 172.21.15.119:0/2677419594 conn(0x7fd4d40c9f70 msgr2=0x7fd4d406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:18.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4db59d700 1 -- 172.21.15.119:0/2677419594 shutdown_connections 2023-10-08T01:09:18.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.587+0000 7fd4db59d700 1 -- 172.21.15.119:0/2677419594 wait complete. 2023-10-08T01:09:18.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.589+0000 7fd4db59d700 1 Processor -- start 2023-10-08T01:09:18.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.589+0000 7fd4db59d700 1 -- start start 2023-10-08T01:09:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4db59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d406e470 0x7fd4d4155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4db59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d40aef00 0x7fd4d41543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4db59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 0x7fd4d41522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4db59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4157e60 con 0x7fd4d406e470 2023-10-08T01:09:18.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4db59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4154e90 con 0x7fd4d40aef00 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4db59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4155170 con 0x7fd4d415a3b0 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4da59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d406e470 0x7fd4d4155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4da59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d406e470 0x7fd4d4155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46410/0 (socket says 172.21.15.119:46410) 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4da59b700 1 -- 172.21.15.119:0/1377793768 learned_addr learned my addr 172.21.15.119:0/1377793768 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4d9d9a700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d40aef00 0x7fd4d41543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4dad9c700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 0x7fd4d41522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.590+0000 7fd4da59b700 1 -- 172.21.15.119:0/1377793768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 msgr2=0x7fd4d41522f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4da59b700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 0x7fd4d41522f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4da59b700 1 -- 172.21.15.119:0/1377793768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d40aef00 msgr2=0x7fd4d41543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4da59b700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d40aef00 0x7fd4d41543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4da59b700 1 -- 172.21.15.119:0/1377793768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c40094f0 con 0x7fd4d406e470 2023-10-08T01:09:18.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4da59b700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d406e470 0x7fd4d4155a10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000e930 tx=0x7fd4d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:18.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4ca7fc700 1 -- 172.21.15.119:0/1377793768 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0013680 con 0x7fd4d406e470 2023-10-08T01:09:18.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4ca7fc700 1 -- 172.21.15.119:0/1377793768 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d00137e0 con 0x7fd4d406e470 2023-10-08T01:09:18.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4ca7fc700 1 -- 172.21.15.119:0/1377793768 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0013b80 con 0x7fd4d406e470 2023-10-08T01:09:18.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d4155450 con 0x7fd4d406e470 2023-10-08T01:09:18.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.591+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d40c5f80 con 0x7fd4d406e470 2023-10-08T01:09:18.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.592+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d406cfa0 con 0x7fd4d406e470 2023-10-08T01:09:18.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.594+0000 7fd4ca7fc700 1 -- 172.21.15.119:0/1377793768 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 22) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fd4d000c460 con 0x7fd4d406e470 2023-10-08T01:09:18.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.595+0000 7fd4ca7fc700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd4b8066f80 0x7fd4b8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:18.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.595+0000 7fd4ca7fc700 1 -- 172.21.15.119:0/1377793768 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fd4d0084870 con 0x7fd4d406e470 2023-10-08T01:09:18.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.595+0000 7fd4d9d9a700 1 -- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd4b8066f80 msgr2=0x7fd4b8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:18.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.595+0000 7fd4d9d9a700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd4b8066f80 0x7fd4b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:09:18.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.599+0000 7fd4ca7fc700 1 -- 172.21.15.119:0/1377793768 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4d0051970 con 0x7fd4d406e470 2023-10-08T01:09:18.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.796+0000 7fd4d9d9a700 1 -- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd4b8066f80 msgr2=0x7fd4b8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:18.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.796+0000 7fd4d9d9a700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd4b8066f80 0x7fd4b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:09:18.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.799+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd4d415e010 con 0x7fd4d406e470 2023-10-08T01:09:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.803+0000 7fd4ca7fc700 1 -- 172.21.15.119:0/1377793768 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd4d0054f90 con 0x7fd4d406e470 2023-10-08T01:09:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd4b8066f80 msgr2=0x7fd4b8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:09:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd4b8066f80 0x7fd4b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d406e470 msgr2=0x7fd4d4155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d406e470 0x7fd4d4155a10 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000e930 tx=0x7fd4d00091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 shutdown_connections 2023-10-08T01:09:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd4b8066f80 0x7fd4b8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4d415a3b0 0x7fd4d41522f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4d406e470 0x7fd4d4155a10 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 --2- 172.21.15.119:0/1377793768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4d40aef00 0x7fd4d41543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 >> 172.21.15.119:0/1377793768 conn(0x7fd4d40c9f70 msgr2=0x7fd4d4153630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 shutdown_connections 2023-10-08T01:09:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:18.806+0000 7fd4db59d700 1 -- 172.21.15.119:0/1377793768 wait complete. 2023-10-08T01:09:18.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T01:09:19.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.012+0000 7fe58179d700 1 Processor -- start 2023-10-08T01:09:19.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.012+0000 7fe58179d700 1 -- start start 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.013+0000 7fe58179d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe57c0b16d0 0x7fe57c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.013+0000 7fe58179d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe57c0b33b0 0x7fe57c0b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.013+0000 7fe58179d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57c0b2080 con 0x7fe57c0b4e70 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.013+0000 7fe58179d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c0cfb00 con 0x7fe57c0b16d0 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.013+0000 7fe58179d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe57c0b5c40 con 0x7fe57c0b33b0 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe57b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe57c0b16d0 0x7fe57c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe57bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe57c0b4e70 0x7fe57c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54678/0 (socket says 172.21.15.119:54678) 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe57bfff700 1 -- 172.21.15.119:0/2907360724 learned_addr learned my addr 172.21.15.119:0/2907360724 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:19.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe580f9c700 1 --2- 172.21.15.119:0/2907360724 >> v2:172.21.15.184:3300/0 conn(0x7fe57c0b33b0 0x7fe57c0b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe57b7fe700 1 -- 172.21.15.119:0/2907360724 >> v2:172.21.15.184:3300/0 conn(0x7fe57c0b33b0 msgr2=0x7fe57c0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:19.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe57b7fe700 1 --2- 172.21.15.119:0/2907360724 >> v2:172.21.15.184:3300/0 conn(0x7fe57c0b33b0 0x7fe57c0b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe57b7fe700 1 -- 172.21.15.119:0/2907360724 >> v1:172.21.15.119:6789/0 conn(0x7fe57c0b4e70 legacy=0x7fe57c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe57b7fe700 1 -- 172.21.15.119:0/2907360724 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c0b5f20 con 0x7fe57c0b16d0 2023-10-08T01:09:19.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.014+0000 7fe57affd700 1 -- 172.21.15.119:0/2907360724 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502575232 0 0) 0x7fe57c0b2080 con 0x7fe57c0b4e70 2023-10-08T01:09:19.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.015+0000 7fe57b7fe700 1 --2- 172.21.15.119:0/2907360724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c0b1b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe57c071380 tx=0x7fe5700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f56572d7da88f6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:19.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.015+0000 7fe57affd700 1 -- 172.21.15.119:0/2907360724 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570011630 con 0x7fe57c0b16d0 2023-10-08T01:09:19.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.015+0000 7fe57affd700 1 -- 172.21.15.119:0/2907360724 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570011790 con 0x7fe57c0b16d0 2023-10-08T01:09:19.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.015+0000 7fe57affd700 1 -- 172.21.15.119:0/2907360724 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570011b30 con 0x7fe57c0b16d0 2023-10-08T01:09:19.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.015+0000 7fe58179d700 1 -- 172.21.15.119:0/2907360724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 msgr2=0x7fe57c0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.015+0000 7fe58179d700 1 --2- 172.21.15.119:0/2907360724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c0b1b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe57c071380 tx=0x7fe5700091a0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.016+0000 7fe58179d700 1 -- 172.21.15.119:0/2907360724 shutdown_connections 2023-10-08T01:09:19.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.016+0000 7fe58179d700 1 --2- 172.21.15.119:0/2907360724 >> v2:172.21.15.184:3300/0 conn(0x7fe57c0b33b0 0x7fe57c0b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.016+0000 7fe58179d700 1 --2- 172.21.15.119:0/2907360724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c0b1b40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.016+0000 7fe58179d700 1 -- 172.21.15.119:0/2907360724 >> 172.21.15.119:0/2907360724 conn(0x7fe57c0d10c0 msgr2=0x7fe57c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:19.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.016+0000 7fe58179d700 1 -- 172.21.15.119:0/2907360724 shutdown_connections 2023-10-08T01:09:19.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.016+0000 7fe58179d700 1 -- 172.21.15.119:0/2907360724 wait complete. 2023-10-08T01:09:19.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.017+0000 7fe58179d700 1 Processor -- start 2023-10-08T01:09:19.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.017+0000 7fe58179d700 1 -- start start 2023-10-08T01:09:19.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.018+0000 7fe58179d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c16d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.018+0000 7fe58179d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0b33b0 0x7fe57c1699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.018+0000 7fe58179d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0b4e70 0x7fe57c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.018+0000 7fe58179d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c7020 con 0x7fe57c0b16d0 2023-10-08T01:09:19.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.018+0000 7fe58179d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c7270 con 0x7fe57c0b4e70 2023-10-08T01:09:19.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.018+0000 7fe58179d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57c0c7550 con 0x7fe57c0b33b0 2023-10-08T01:09:19.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.018+0000 7fe57bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c16d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe580f9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0b4e70 0x7fe57c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe57bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c16d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46434/0 (socket says 172.21.15.119:46434) 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe57bfff700 1 -- 172.21.15.119:0/3281228770 learned_addr learned my addr 172.21.15.119:0/3281228770 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe57b7fe700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0b33b0 0x7fe57c1699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe580f9c700 1 -- 172.21.15.119:0/3281228770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0b33b0 msgr2=0x7fe57c1699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe580f9c700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0b33b0 0x7fe57c1699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe580f9c700 1 -- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 msgr2=0x7fe57c16d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe580f9c700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c16d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe580f9c700 1 -- 172.21.15.119:0/3281228770 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe570009420 con 0x7fe57c0b4e70 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe57bfff700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c16d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:09:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe580f9c700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0b4e70 0x7fe57c0cb430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe574009e50 tx=0x7fe57400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.019+0000 7fe563fff700 1 -- 172.21.15.119:0/3281228770 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57400caa0 con 0x7fe57c0b4e70 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.020+0000 7fe563fff700 1 -- 172.21.15.119:0/3281228770 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57400f040 con 0x7fe57c0b4e70 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.020+0000 7fe563fff700 1 -- 172.21.15.119:0/3281228770 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57400ce50 con 0x7fe57c0b4e70 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.020+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57c0c7830 con 0x7fe57c0b4e70 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.020+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57c0c7e70 con 0x7fe57c0b4e70 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.021+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57c088990 con 0x7fe57c0b4e70 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.022+0000 7fe563fff700 1 -- 172.21.15.119:0/3281228770 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fe574014690 con 0x7fe57c0b4e70 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.023+0000 7fe563fff700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe568066f80 0x7fe568069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.023+0000 7fe563fff700 1 -- 172.21.15.119:0/3281228770 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7fe574085320 con 0x7fe57c0b4e70 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.023+0000 7fe57bfff700 1 -- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe568066f80 msgr2=0x7fe568069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.023+0000 7fe57bfff700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe568066f80 0x7fe568069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:09:19.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.028+0000 7fe563fff700 1 -- 172.21.15.119:0/3281228770 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe574055eb0 con 0x7fe57c0b4e70 2023-10-08T01:09:19.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.224+0000 7fe57bfff700 1 -- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe568066f80 msgr2=0x7fe568069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:19.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.224+0000 7fe57bfff700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe568066f80 0x7fe568069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:09:19.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.230+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe57c04bb80 con 0x7fe57c0b4e70 2023-10-08T01:09:19.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.236+0000 7fe563fff700 1 -- 172.21.15.119:0/3281228770 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe574055a40 con 0x7fe57c0b4e70 2023-10-08T01:09:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe568066f80 msgr2=0x7fe568069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:09:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe568066f80 0x7fe568069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0b4e70 msgr2=0x7fe57c0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0b4e70 0x7fe57c0cb430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe574009e50 tx=0x7fe57400b040 comp rx=0 tx=0).stop 2023-10-08T01:09:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 shutdown_connections 2023-10-08T01:09:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe568066f80 0x7fe568069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe57c0b4e70 0x7fe57c0cb430 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe57c0b16d0 0x7fe57c16d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 --2- 172.21.15.119:0/3281228770 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe57c0b33b0 0x7fe57c1699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 >> 172.21.15.119:0/3281228770 conn(0x7fe57c0d10c0 msgr2=0x7fe57c069710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 shutdown_connections 2023-10-08T01:09:19.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.239+0000 7fe58179d700 1 -- 172.21.15.119:0/3281228770 wait complete. 2023-10-08T01:09:19.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:19.258 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T01:09:19.299 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-08T01:09:19.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.528+0000 7f13ec99d700 1 Processor -- start 2023-10-08T01:09:19.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.528+0000 7f13ec99d700 1 -- start start 2023-10-08T01:09:19.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13ec99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13e80cb560 0x7f13e80cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13ec99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e8069ec0 0x7f13e815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13ec99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13e8159db0 con 0x7f13e80cc970 2023-10-08T01:09:19.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13ec99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13e806aa30 con 0x7f13e80cb560 2023-10-08T01:09:19.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13ec99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80cce10 con 0x7f13e8069ec0 2023-10-08T01:09:19.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13e77fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13e80cb560 0x7f13e80cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13e7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e8069ec0 0x7f13e815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13e77fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13e80cb560 0x7f13e80cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:48042/0 (socket says 172.21.15.119:48042) 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.530+0000 7f13e77fe700 1 -- 172.21.15.119:0/3167457550 learned_addr learned my addr 172.21.15.119:0/3167457550 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e67fc700 1 -- 172.21.15.119:0/3167457550 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018223334 0 0) 0x7f13e8159db0 con 0x7f13e80cc970 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e67fc700 1 -- 172.21.15.119:0/3167457550 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d0003570 con 0x7f13e80cc970 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e77fe700 1 -- 172.21.15.119:0/3167457550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e8069ec0 msgr2=0x7f13e815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e77fe700 1 --2- 172.21.15.119:0/3167457550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e8069ec0 0x7f13e815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e77fe700 1 -- 172.21.15.119:0/3167457550 >> v1:172.21.15.184:6789/0 conn(0x7f13e80cc970 legacy=0x7f13e80697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e77fe700 1 -- 172.21.15.119:0/3167457550 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e806a470 con 0x7f13e80cb560 2023-10-08T01:09:19.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e77fe700 1 --2- 172.21.15.119:0/3167457550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e80cb560 0x7f13e80cc430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f13d80099e0 tx=0x7f13d8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc051b3c766ca30a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e67fc700 1 -- 172.21.15.119:0/3167457550 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800b640 con 0x7f13e80cb560 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e67fc700 1 -- 172.21.15.119:0/3167457550 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d800b7a0 con 0x7f13e80cb560 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.531+0000 7f13e67fc700 1 -- 172.21.15.119:0/3167457550 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800bb40 con 0x7f13e80cb560 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.532+0000 7f13ec99d700 1 -- 172.21.15.119:0/3167457550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e80cb560 msgr2=0x7f13e80cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.532+0000 7f13ec99d700 1 --2- 172.21.15.119:0/3167457550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e80cb560 0x7f13e80cc430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f13d80099e0 tx=0x7f13d8009250 comp rx=0 tx=0).stop 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.532+0000 7f13ec99d700 1 -- 172.21.15.119:0/3167457550 shutdown_connections 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.532+0000 7f13ec99d700 1 --2- 172.21.15.119:0/3167457550 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e8069ec0 0x7f13e815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.532+0000 7f13ec99d700 1 --2- 172.21.15.119:0/3167457550 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e80cb560 0x7f13e80cc430 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.532+0000 7f13ec99d700 1 -- 172.21.15.119:0/3167457550 >> 172.21.15.119:0/3167457550 conn(0x7f13e806baf0 msgr2=0x7f13e806bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.532+0000 7f13ec99d700 1 -- 172.21.15.119:0/3167457550 shutdown_connections 2023-10-08T01:09:19.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.532+0000 7f13ec99d700 1 -- 172.21.15.119:0/3167457550 wait complete. 2023-10-08T01:09:19.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.533+0000 7f13ec99d700 1 Processor -- start 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.533+0000 7f13ec99d700 1 -- start start 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.534+0000 7f13ec99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e8069ec0 0x7f13e814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.534+0000 7f13ec99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e80cb560 0x7f13e80ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.534+0000 7f13ec99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e80cc970 0x7f13e80ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.534+0000 7f13ec99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80aa670 con 0x7f13e80cc970 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.534+0000 7f13ec99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80aa950 con 0x7f13e8069ec0 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.534+0000 7f13ec99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e80c58a0 con 0x7f13e80cb560 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.534+0000 7f13e77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e8069ec0 0x7f13e814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e77fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e8069ec0 0x7f13e814c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:48054/0 (socket says 172.21.15.119:48054) 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e77fe700 1 -- 172.21.15.119:0/730197272 learned_addr learned my addr 172.21.15.119:0/730197272 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e6ffd700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e80cb560 0x7f13e80ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e7fff700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e80cc970 0x7f13e80ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e77fe700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e80cb560 msgr2=0x7f13e80ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e77fe700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e80cb560 0x7f13e80ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e77fe700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e80cc970 msgr2=0x7f13e80ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e77fe700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e80cc970 0x7f13e80ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e77fe700 1 -- 172.21.15.119:0/730197272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80094f0 con 0x7f13e8069ec0 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13e77fe700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e8069ec0 0x7f13e814c050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f13e806ce10 tx=0x7f13d8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d800b910 con 0x7f13e8069ec0 2023-10-08T01:09:19.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d800ba90 con 0x7f13e8069ec0 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.535+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e80c5b80 con 0x7f13e8069ec0 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.536+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d8024690 con 0x7f13e8069ec0 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.536+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e816a900 con 0x7f13e8069ec0 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.537+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e80c56e0 con 0x7f13e8069ec0 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.538+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 22) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f13d802d440 con 0x7f13e8069ec0 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.538+0000 7f13cf7fe700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d4069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.539+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f13d8085ff0 con 0x7f13e8069ec0 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.539+0000 7f13e6ffd700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 msgr2=0x7f13d4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:19.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.539+0000 7f13e6ffd700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:09:19.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.543+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13d80b75a0 con 0x7f13e8069ec0 2023-10-08T01:09:19.630 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:19.629+0000 7f902ab12180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:09:19.711 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:19.710+0000 7f1cbb8f2180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:09:19.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.740+0000 7f13e6ffd700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 msgr2=0x7f13d4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:19.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.740+0000 7f13e6ffd700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:09:19.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:19.768+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 --> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f13e804c3c0 con 0x7f13d4066f80 2023-10-08T01:09:19.790 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:19.789+0000 7f902ab12180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:09:19.874 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:19.873+0000 7f902ab12180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:09:19.878 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:19.877+0000 7f1cbb8f2180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:09:19.962 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:19.961+0000 7f1cbb8f2180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:09:20.096 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:20.095+0000 7f902ab12180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:09:20.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:20.141+0000 7f13e6ffd700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 msgr2=0x7f13d4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:20.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:20.141+0000 7f13e6ffd700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:09:20.184 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:20.183+0000 7f1cbb8f2180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:09:20.270 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:20.269+0000 7f902ab12180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:09:20.360 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:20.359+0000 7f1cbb8f2180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:09:20.361 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:20.359+0000 7f902ab12180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:09:20.452 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:20.451+0000 7f1cbb8f2180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:09:20.521 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:20.520+0000 7f902ab12180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:09:20.613 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:20.612+0000 7f1cbb8f2180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:09:20.620 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:20.619+0000 7f902ab12180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:09:20.713 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:20.712+0000 7f1cbb8f2180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:09:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:20.942+0000 7f13e6ffd700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 msgr2=0x7f13d4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:20.942+0000 7f13e6ffd700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T01:09:21.097 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:21.096+0000 7f902ab12180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:09:21.190 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:21.189+0000 7f902ab12180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:09:21.194 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:21.192+0000 7f1cbb8f2180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:09:21.278 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:21.277+0000 7f902ab12180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:09:21.287 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:21.286+0000 7f1cbb8f2180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:09:21.377 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:21.376+0000 7f1cbb8f2180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:09:21.893 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:21.892+0000 7f902ab12180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:09:21.993 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:21.992+0000 7f1cbb8f2180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:09:22.062 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:22.061+0000 7f902ab12180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:09:22.156 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:22.155+0000 7f902ab12180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:09:22.165 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:22.164+0000 7f1cbb8f2180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:09:22.260 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:22.260+0000 7f1cbb8f2180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:09:22.449 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:22.448+0000 7f902ab12180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:09:22.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:22.544+0000 7f13e6ffd700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 msgr2=0x7f13d4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:09:22.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:22.544+0000 7f13e6ffd700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T01:09:22.545 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:22.544+0000 7f902ab12180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:09:22.554 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:22.554+0000 7f1cbb8f2180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:09:22.649 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:22.648+0000 7f1cbb8f2180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:09:23.235 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:23.234+0000 7f902ab12180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:09:23.345 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:23.344+0000 7f1cbb8f2180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:09:23.521 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:23.520+0000 7f902ab12180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:09:23.611 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:23.610+0000 7f902ab12180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:09:23.627 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:23.626+0000 7f1cbb8f2180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:09:23.719 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:23.718+0000 7f1cbb8f2180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:09:23.763 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:23.761+0000 7f902ab12180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:09:23.865 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:23.864+0000 7f902ab12180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:09:23.873 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:23.872+0000 7f1cbb8f2180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:09:23.976 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:23.975+0000 7f1cbb8f2180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:09:24.050 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:24.048+0000 7f902ab12180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:09:24.140 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:24.138+0000 7f902ab12180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:09:24.163 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.162+0000 7f1cbb8f2180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:09:24.229 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:24.227+0000 7f902ab12180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:09:24.254 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.254+0000 7f1cbb8f2180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:09:24.321 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:09:24.320+0000 7f902ab12180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:09:24.347 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.346+0000 7f1cbb8f2180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:09:24.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:24.380+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mgrmap(e 23) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f13d8053440 con 0x7f13e8069ec0 2023-10-08T01:09:24.443 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.443+0000 7f1cbb8f2180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:09:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:24.454+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 8 ==== mgrmap(e 24) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f13d8056220 con 0x7f13e8069ec0 2023-10-08T01:09:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:24.454+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 msgr2=0x7f13d4069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:09:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:24.454+0000 7f13cf7fe700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:24.579 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c30c37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.579 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c30c37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.579 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c30c37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.579 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c30c37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.580 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c30c37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.580 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c2cc2f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.580 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c2cc2f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.580 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c2cc2f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.580 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c2cc2f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:24.581 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:09:24.577+0000 7f1c2cc2f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:09:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.461+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mon.1 v2:172.21.15.184:3300/0 9 ==== mgrmap(e 25) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f13d8008d50 con 0x7f13e8069ec0 2023-10-08T01:09:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.462+0000 7f13cf7fe700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d40693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.462+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 --> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f13e804c3c0 con 0x7f13d4066f80 2023-10-08T01:09:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.463+0000 7f13e6ffd700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d40693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.466+0000 7f13e6ffd700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d40693b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13e8167320 tx=0x7f13dc007d50 comp rx=0 tx=0).ready entity=mgr.5940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:25.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.524+0000 7f13cf7fe700 1 -- 172.21.15.119:0/730197272 <== mgr.5940 v2:172.21.15.119:6834/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f13e804c3c0 con 0x7f13d4066f80 2023-10-08T01:09:25.525 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path snap_test_dir1 2023-10-08T01:09:25.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 msgr2=0x7f13d40693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:25.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d40693b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f13e8167320 tx=0x7f13dc007d50 comp rx=0 tx=0).stop 2023-10-08T01:09:25.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e8069ec0 msgr2=0x7f13e814c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:25.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e8069ec0 0x7f13e814c050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f13e806ce10 tx=0x7f13d8017000 comp rx=0 tx=0).stop 2023-10-08T01:09:25.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 shutdown_connections 2023-10-08T01:09:25.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f13d4066f80 0x7f13d40693b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13e80cc970 0x7f13e80ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13e8069ec0 0x7f13e814c050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 --2- 172.21.15.119:0/730197272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13e80cb560 0x7f13e80ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 >> 172.21.15.119:0/730197272 conn(0x7f13e806baf0 msgr2=0x7f13e80ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:25.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 shutdown_connections 2023-10-08T01:09:25.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.527+0000 7f13ec99d700 1 -- 172.21.15.119:0/730197272 wait complete. 2023-10-08T01:09:25.545 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T01:09:25.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-10-08T01:09:25.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.731+0000 7fcbfa1ff700 1 Processor -- start 2023-10-08T01:09:25.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.731+0000 7fcbfa1ff700 1 -- start start 2023-10-08T01:09:25.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbfa1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbf4069a50 0x7fcbf40c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:25.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbfa1ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcbf40c8490 0x7fcbf415a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:25.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbfa1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069e20 0x7fcbf406e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:25.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbfa1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf40ac030 con 0x7fcbf4069a50 2023-10-08T01:09:25.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbfa1ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf40ab820 con 0x7fcbf40c8490 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbfa1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf406dbb0 con 0x7fcbf4069e20 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbf37fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbf4069a50 0x7fcbf40c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbf37fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcbf4069a50 0x7fcbf40c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42182/0 (socket says 172.21.15.119:42182) 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.732+0000 7fcbf37fe700 1 -- 172.21.15.119:0/3448548263 learned_addr learned my addr 172.21.15.119:0/3448548263 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf2ffd700 1 --2- 172.21.15.119:0/3448548263 >> v2:172.21.15.184:3300/0 conn(0x7fcbf40c8490 0x7fcbf415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf3fff700 1 --2- 172.21.15.119:0/3448548263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069e20 0x7fcbf406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf37fe700 1 -- 172.21.15.119:0/3448548263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069e20 msgr2=0x7fcbf406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf37fe700 1 --2- 172.21.15.119:0/3448548263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069e20 0x7fcbf406e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf37fe700 1 -- 172.21.15.119:0/3448548263 >> v2:172.21.15.184:3300/0 conn(0x7fcbf40c8490 msgr2=0x7fcbf415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf37fe700 1 --2- 172.21.15.119:0/3448548263 >> v2:172.21.15.184:3300/0 conn(0x7fcbf40c8490 0x7fcbf415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf37fe700 1 -- 172.21.15.119:0/3448548263 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf415a9d0 con 0x7fcbf4069a50 2023-10-08T01:09:25.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf37fe700 1 --2- 172.21.15.119:0/3448548263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf4069a50 0x7fcbf40c6ac0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcbf406b920 tx=0x7fcbe400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744c5a52304c12e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:25.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf27fc700 1 -- 172.21.15.119:0/3448548263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe400be60 con 0x7fcbf4069a50 2023-10-08T01:09:25.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf27fc700 1 -- 172.21.15.119:0/3448548263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe400ddc0 con 0x7fcbf4069a50 2023-10-08T01:09:25.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.733+0000 7fcbf27fc700 1 -- 172.21.15.119:0/3448548263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe40196b0 con 0x7fcbf4069a50 2023-10-08T01:09:25.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/3448548263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf4069a50 msgr2=0x7fcbf40c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:25.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/3448548263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf4069a50 0x7fcbf40c6ac0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fcbf406b920 tx=0x7fcbe400b4f0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/3448548263 shutdown_connections 2023-10-08T01:09:25.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/3448548263 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069e20 0x7fcbf406e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/3448548263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf4069a50 0x7fcbf40c6ac0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/3448548263 >> v2:172.21.15.184:3300/0 conn(0x7fcbf40c8490 0x7fcbf415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/3448548263 >> 172.21.15.119:0/3448548263 conn(0x7fcbf40c8f90 msgr2=0x7fcbf40c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/3448548263 shutdown_connections 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.734+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/3448548263 wait complete. 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.735+0000 7fcbfa1ff700 1 Processor -- start 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.736+0000 7fcbfa1ff700 1 -- start start 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.736+0000 7fcbfa1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069a50 0x7fcbf41691a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.736+0000 7fcbfa1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf4069e20 0x7fcbf41696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.736+0000 7fcbfa1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf40c8490 0x7fcbf40c1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.736+0000 7fcbfa1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4155ac0 con 0x7fcbf40c8490 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.736+0000 7fcbfa1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4167420 con 0x7fcbf4069e20 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.736+0000 7fcbfa1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf4166e10 con 0x7fcbf4069a50 2023-10-08T01:09:25.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf40c8490 0x7fcbf40c1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf40c8490 0x7fcbf40c1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42196/0 (socket says 172.21.15.119:42196) 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 -- 172.21.15.119:0/2795966421 learned_addr learned my addr 172.21.15.119:0/2795966421 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf2ffd700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf4069e20 0x7fcbf41696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf37fe700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069a50 0x7fcbf41691a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 -- 172.21.15.119:0/2795966421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069a50 msgr2=0x7fcbf41691a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069a50 0x7fcbf41691a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 -- 172.21.15.119:0/2795966421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf4069e20 msgr2=0x7fcbf41696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf4069e20 0x7fcbf41696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 -- 172.21.15.119:0/2795966421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe400b790 con 0x7fcbf40c8490 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.737+0000 7fcbf3fff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf40c8490 0x7fcbf40c1250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009fa0 tx=0x7fcbec009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:25.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.738+0000 7fcbdb7fe700 1 -- 172.21.15.119:0/2795966421 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00ebc0 con 0x7fcbf40c8490 2023-10-08T01:09:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.738+0000 7fcbdb7fe700 1 -- 172.21.15.119:0/2795966421 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec00ed20 con 0x7fcbf40c8490 2023-10-08T01:09:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.738+0000 7fcbdb7fe700 1 -- 172.21.15.119:0/2795966421 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec014670 con 0x7fcbf40c8490 2023-10-08T01:09:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.738+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf40c2090 con 0x7fcbf40c8490 2023-10-08T01:09:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.738+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf40c2370 con 0x7fcbf40c8490 2023-10-08T01:09:25.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.739+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf41675b0 con 0x7fcbf40c8490 2023-10-08T01:09:25.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.740+0000 7fcbdb7fe700 1 -- 172.21.15.119:0/2795966421 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fcbec0149e0 con 0x7fcbf40c8490 2023-10-08T01:09:25.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.741+0000 7fcbdb7fe700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcbe0066dd0 0x7fcbe0069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:25.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.741+0000 7fcbdb7fe700 1 -- 172.21.15.119:0/2795966421 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6327+0+0 (secure 0 0 0) 0x7fcbec084c20 con 0x7fcbf40c8490 2023-10-08T01:09:25.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.741+0000 7fcbf37fe700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcbe0066dd0 0x7fcbe0069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:25.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.742+0000 7fcbf37fe700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcbe0066dd0 0x7fcbe0069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcbf406b920 tx=0x7fcbe401b000 comp rx=0 tx=0).ready entity=mgr.5940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:25.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.746+0000 7fcbdb7fe700 1 -- 172.21.15.119:0/2795966421 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbec051e00 con 0x7fcbf40c8490 2023-10-08T01:09:25.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:25.965+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 --> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbf40718b0 con 0x7fcbe0066dd0 2023-10-08T01:09:26.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.028+0000 7fcbdb7fe700 1 -- 172.21.15.119:0/2795966421 <== mgr.5940 v2:172.21.15.119:6834/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcbf40718b0 con 0x7fcbe0066dd0 2023-10-08T01:09:26.028 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path snap_test_dir1 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcbe0066dd0 msgr2=0x7fcbe0069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcbe0066dd0 0x7fcbe0069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fcbf406b920 tx=0x7fcbe401b000 comp rx=0 tx=0).stop 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf40c8490 msgr2=0x7fcbf40c1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf40c8490 0x7fcbf40c1250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009fa0 tx=0x7fcbec009900 comp rx=0 tx=0).stop 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 shutdown_connections 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcbe0066dd0 0x7fcbe0069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbf4069a50 0x7fcbf41691a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbf40c8490 0x7fcbf40c1250 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 --2- 172.21.15.119:0/2795966421 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbf4069e20 0x7fcbf41696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 >> 172.21.15.119:0/2795966421 conn(0x7fcbf40c8f90 msgr2=0x7fcbf40b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:26.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.030+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 shutdown_connections 2023-10-08T01:09:26.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.031+0000 7fcbfa1ff700 1 -- 172.21.15.119:0/2795966421 wait complete. 2023-10-08T01:09:26.048 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T01:09:26.048 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-01_10_00 in ~34s... 2023-10-08T01:09:26.048 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-01_11_00 in ~94s... 2023-10-08T01:09:26.048 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-10-08T01:09:26.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-08T01:09:26.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.235+0000 7ff6393ff700 1 Processor -- start 2023-10-08T01:09:26.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.235+0000 7ff6393ff700 1 -- start start 2023-10-08T01:09:26.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.236+0000 7ff6393ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff634154320 0x7ff63406d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:26.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.236+0000 7ff6393ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6340b4570 con 0x7ff634153f50 2023-10-08T01:09:26.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.236+0000 7ff6393ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6340b2710 con 0x7ff63415a190 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.236+0000 7ff6393ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63415a730 con 0x7ff634154320 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff632ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff634153f50 0x7ff6340b2000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48666/0 (socket says 172.21.15.119:48666) 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff632ffd700 1 -- 172.21.15.119:0/850551207 learned_addr learned my addr 172.21.15.119:0/850551207 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff6327fc700 1 --2- 172.21.15.119:0/850551207 >> v2:172.21.15.184:3300/0 conn(0x7ff634154320 0x7ff63406d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884441030 0 0) 0x7ff6340b4570 con 0x7ff634153f50 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff61c003570 con 0x7ff634153f50 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2639365404 0 0) 0x7ff61c003570 con 0x7ff634153f50 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 >> v2:172.21.15.184:3300/0 conn(0x7ff634154320 msgr2=0x7ff63406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 --2- 172.21.15.119:0/850551207 >> v2:172.21.15.184:3300/0 conn(0x7ff634154320 0x7ff63406d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 >> v1:172.21.15.184:6789/0 conn(0x7ff63415a190 legacy=0x7ff634069e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63415aa10 con 0x7ff634153f50 2023-10-08T01:09:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7ff624001470 con 0x7ff634153f50 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 >> v1:172.21.15.119:6789/0 conn(0x7ff634153f50 legacy=0x7ff6340b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.237+0000 7ff631ffb700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff61c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff631ffb700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff61c004810 0x7ff61c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff631ffb700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff61c005210 0x7ff61c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63415a730 con 0x7ff61c003ab0 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c00c130 con 0x7ff61c005210 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c00c2f0 con 0x7ff61c004810 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff632ffd700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff61c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff6337fe700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff61c005210 0x7ff61c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff6327fc700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff61c004810 0x7ff61c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:26.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff632ffd700 1 -- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff61c004810 msgr2=0x7ff61c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff632ffd700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff61c004810 0x7ff61c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff632ffd700 1 -- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff61c005210 msgr2=0x7ff61c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff632ffd700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff61c005210 0x7ff61c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff632ffd700 1 -- 172.21.15.119:0/850551207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c003570 con 0x7ff61c003ab0 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff6337fe700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff61c005210 0x7ff61c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff6337fe700 1 -- 172.21.15.119:0/850551207 reap_dead start 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.238+0000 7ff632ffd700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff61c003ec0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c2d0 tx=0x7ff624001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7630a51d8e8f9481 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62400cba0 con 0x7ff61c003ab0 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff62400cd20 con 0x7ff61c003ab0 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff631ffb700 1 -- 172.21.15.119:0/850551207 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff624005b70 con 0x7ff61c003ab0 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff6393ff700 1 -- 172.21.15.119:0/850551207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 msgr2=0x7ff61c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff6393ff700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff61c003ec0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c2d0 tx=0x7ff624001d80 comp rx=0 tx=0).stop 2023-10-08T01:09:26.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff6393ff700 1 -- 172.21.15.119:0/850551207 shutdown_connections 2023-10-08T01:09:26.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff6393ff700 1 --2- 172.21.15.119:0/850551207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff61c003ec0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff6393ff700 1 -- 172.21.15.119:0/850551207 >> 172.21.15.119:0/850551207 conn(0x7ff63406baf0 msgr2=0x7ff63406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:26.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff6393ff700 1 -- 172.21.15.119:0/850551207 shutdown_connections 2023-10-08T01:09:26.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.239+0000 7ff6393ff700 1 -- 172.21.15.119:0/850551207 wait complete. 2023-10-08T01:09:26.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.240+0000 7ff6393ff700 1 Processor -- start 2023-10-08T01:09:26.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.241+0000 7ff6393ff700 1 -- start start 2023-10-08T01:09:26.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.241+0000 7ff6393ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff63413cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:26.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.241+0000 7ff6393ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6340c99d0 0x7ff6340c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.241+0000 7ff6393ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6340cd020 0x7ff6340cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff6393ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63413d320 con 0x7ff61c003ab0 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff6393ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340ca380 con 0x7ff6340cd020 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff6393ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6340cde50 con 0x7ff6340c99d0 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff63413cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff63413cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42210/0 (socket says 172.21.15.119:42210) 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 -- 172.21.15.119:0/3950226439 learned_addr learned my addr 172.21.15.119:0/3950226439 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff6327fc700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6340c99d0 0x7ff6340c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 -- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6340c99d0 msgr2=0x7ff6340c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff6337fe700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6340cd020 0x7ff6340cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6340c99d0 0x7ff6340c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 -- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6340cd020 msgr2=0x7ff6340cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6340cd020 0x7ff6340cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 -- 172.21.15.119:0/3950226439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6340ce130 con 0x7ff61c003ab0 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff6327fc700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6340c99d0 0x7ff6340c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff6337fe700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6340cd020 0x7ff6340cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff632ffd700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff63413cde0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c8c0 tx=0x7ff624015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff61affd700 1 -- 172.21.15.119:0/3950226439 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62400c8f0 con 0x7ff61c003ab0 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff61affd700 1 -- 172.21.15.119:0/3950226439 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff624004e70 con 0x7ff61c003ab0 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.242+0000 7ff61affd700 1 -- 172.21.15.119:0/3950226439 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6240074f0 con 0x7ff61c003ab0 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.243+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6340ce410 con 0x7ff61c003ab0 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.243+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6340cda00 con 0x7ff61c003ab0 2023-10-08T01:09:26.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.244+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6340718b0 con 0x7ff61c003ab0 2023-10-08T01:09:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.244+0000 7ff61affd700 1 -- 172.21.15.119:0/3950226439 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 25) v1 ==== 84595+0+0 (secure 0 0 0) 0x7ff624012400 con 0x7ff61c003ab0 2023-10-08T01:09:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.246+0000 7ff61affd700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff620066e00 0x7ff6200692b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:09:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.246+0000 7ff61affd700 1 -- 172.21.15.119:0/3950226439 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6327+0+0 (secure 0 0 0) 0x7ff624084e90 con 0x7ff61c003ab0 2023-10-08T01:09:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.246+0000 7ff6327fc700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff620066e00 0x7ff6200692b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:09:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.247+0000 7ff6327fc700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff620066e00 0x7ff6200692b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff628006f00 tx=0x7ff628008040 comp rx=0 tx=0).ready entity=mgr.5940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:09:26.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.250+0000 7ff61affd700 1 -- 172.21.15.119:0/3950226439 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff624052120 con 0x7ff61c003ab0 2023-10-08T01:09:26.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.466+0000 7ff61affd700 1 -- 172.21.15.119:0/3950226439 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 26) v1 ==== 84916+0+0 (secure 0 0 0) 0x7ff6240524f0 con 0x7ff61c003ab0 2023-10-08T01:09:26.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.471+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 --> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff63415acf0 con 0x7ff620066e00 2023-10-08T01:09:26.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.480+0000 7ff61affd700 1 -- 172.21.15.119:0/3950226439 <== mgr.5940 v2:172.21.15.119:6834/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7ff63415acf0 con 0x7ff620066e00 2023-10-08T01:09:26.480 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:09:26.480 INFO:teuthology.orchestra.run.smithi119.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-10-08T01:09:26.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff620066e00 msgr2=0x7ff6200692b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff620066e00 0x7ff6200692b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff628006f00 tx=0x7ff628008040 comp rx=0 tx=0).stop 2023-10-08T01:09:26.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 msgr2=0x7ff63413cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff63413cde0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff62400c8c0 tx=0x7ff624015000 comp rx=0 tx=0).stop 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 shutdown_connections 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7ff620066e00 0x7ff6200692b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6340cd020 0x7ff6340cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff61c003ab0 0x7ff63413cde0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 --2- 172.21.15.119:0/3950226439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff6340c99d0 0x7ff6340c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 >> 172.21.15.119:0/3950226439 conn(0x7ff63406baf0 msgr2=0x7ff634069d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 shutdown_connections 2023-10-08T01:09:26.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:09:26.482+0000 7ff6393ff700 1 -- 172.21.15.119:0/3950226439 wait complete. 2023-10-08T01:09:26.499 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-10-08T01:09:26.499 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-10-08T01:09:26.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:26.500 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:26.539 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:26.539 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:26.539 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:27.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:27.540 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:27.583 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:27.584 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:27.584 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:28.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:28.585 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:28.628 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:28.628 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:28.628 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:29.629 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:29.629 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:29.673 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:29.673 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:29.673 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:30.674 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:30.674 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:30.717 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:30.717 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:30.718 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:31.718 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:31.718 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:31.761 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:31.761 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:31.761 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:32.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:32.762 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:32.805 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:32.805 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:32.805 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:33.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:33.806 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:33.850 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:33.850 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:33.850 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:34.851 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:34.852 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:34.894 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:34.895 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:34.895 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:35.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:35.896 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:35.939 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:35.939 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:35.939 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:36.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:36.941 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:36.983 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:36.984 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:36.984 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:37.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:37.985 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:38.027 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:38.028 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:38.028 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:39.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:39.028 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:39.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:39.072 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:39.072 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:40.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:40.073 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:40.116 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:40.116 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:40.116 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:41.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:41.117 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:41.160 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:41.160 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:41.160 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:42.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:42.161 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:42.205 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:42.205 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:42.205 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:43.206 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:43.206 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:43.249 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:43.249 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:43.249 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:44.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:44.250 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:44.293 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:44.293 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:44.293 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:45.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:45.295 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:45.339 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:45.339 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:45.339 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:46.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:46.341 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:46.385 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:46.385 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:46.385 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:46.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:09:46.739 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:09:47.386 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:47.386 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:47.429 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:47.429 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:47.429 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:48.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:48.430 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:48.473 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:48.473 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:48.473 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:49.474 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:49.474 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:49.517 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:49.518 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:49.518 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:50.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:50.519 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:50.562 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:50.562 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:50.562 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:51.563 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:51.563 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:51.606 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:51.606 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:51.606 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:52.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:52.607 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:52.650 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:52.650 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:52.650 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:53.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:53.651 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:53.694 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:53.694 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:53.694 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:54.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:54.696 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:54.738 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:54.738 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:54.738 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:55.739 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:55.739 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:55.782 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:55.782 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:55.782 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:56.783 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:56.784 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:56.827 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:56.827 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:56.827 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:57.828 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:57.828 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:57.871 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:57.871 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:57.871 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:58.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:58.872 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:58.915 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:58.915 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:58.915 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:09:59.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:09:59.916 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:09:59.960 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:09:59.960 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:09:59.960 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:00.040 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:10:00.039+0000 7f1c1fc15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:10:00.040 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:10:00.039+0000 7f1c1fc15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:10:00.040 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:10:00.039+0000 7f1c1fc15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:10:00.040 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:10:00.039+0000 7f1c1fc15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:10:00.040 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:10:00.039+0000 7f1c1fc15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:10:00.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:00.961 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:01.003 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:01.004 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:01.004 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:01.004 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:01.004 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-08-01_10_00_UTC'] 2023-10-08T01:10:01.005 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696727366.048355, now = 1696727401.0050614, timo = 34 2023-10-08T01:10:02.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:02.006 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:02.048 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:02.049 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:02.050 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:02.050 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:03.051 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:03.051 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:03.093 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:03.094 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:03.094 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:03.095 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:04.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:04.095 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:04.137 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:04.138 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:04.138 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:04.138 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:05.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:05.140 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:05.181 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:05.182 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:05.182 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:05.183 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:06.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:06.183 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:06.225 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:06.226 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:06.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:06.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:07.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:07.228 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:07.270 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:07.271 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:07.273 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:07.273 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:08.274 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:08.275 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:08.317 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:08.318 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:08.318 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:08.318 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:09.321 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:09.321 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:09.363 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:09.364 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:09.364 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:09.364 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:10.366 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:10.366 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:10.407 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:10.408 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:10.408 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:10.409 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:11.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:11.410 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:11.452 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:11.453 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:11.453 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:11.453 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:12.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:12.455 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:12.496 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:12.497 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:12.497 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:12.498 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:13.498 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:13.498 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:13.540 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:13.541 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:13.541 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:13.541 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:14.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:14.542 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:14.584 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:14.586 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:14.586 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:14.586 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:15.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:15.587 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:15.629 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:15.630 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:15.630 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:15.630 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:16.631 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:16.632 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:16.673 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:16.674 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:16.674 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:16.674 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:16.921 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:10:16.926 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:10:17.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:17.676 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:17.718 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:17.719 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:17.719 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:17.719 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:18.720 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:18.720 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:18.764 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:18.764 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:18.764 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:18.764 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:19.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:19.765 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:19.807 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:19.808 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:19.808 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:19.808 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:20.809 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:20.809 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:20.851 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:20.852 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:20.852 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:20.852 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:21.853 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:21.853 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:21.895 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:21.896 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:21.896 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:21.896 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:22.897 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:22.898 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:22.939 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:22.941 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:22.941 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:22.941 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:23.941 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:23.942 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:23.983 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:23.984 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:23.984 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:23.985 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:24.986 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:24.986 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:25.028 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:25.029 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:25.029 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:25.029 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:26.030 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:26.031 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:26.072 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:26.073 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:26.074 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:26.074 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:27.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:27.075 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:27.117 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:27.118 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:27.118 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:27.118 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:28.119 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:28.120 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:28.161 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:28.162 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:28.163 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:28.163 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:29.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:29.164 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:29.207 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:29.208 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:29.208 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:29.208 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:30.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:30.210 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:30.253 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:30.254 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:30.254 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:30.254 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:31.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:31.255 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:31.298 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:31.299 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:31.299 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:31.299 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:32.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:32.300 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:32.342 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:32.343 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:32.343 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:32.344 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:33.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:33.345 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:33.387 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:33.389 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:33.389 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:33.389 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:34.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:34.390 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:34.431 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:34.432 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:34.433 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:34.433 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:35.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:35.434 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:35.477 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:35.478 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:35.478 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:35.478 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:36.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:36.480 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:36.521 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:36.522 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:36.522 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:36.522 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:37.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:37.524 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:37.566 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:37.567 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:37.567 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:37.567 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:38.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:38.568 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:38.610 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:38.611 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:38.611 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:38.611 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:39.612 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:39.612 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:39.655 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:39.655 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:39.656 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:39.656 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:40.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:40.657 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:40.698 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:40.699 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:40.700 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:40.700 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:41.701 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:41.701 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:41.743 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:41.744 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:41.744 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:41.744 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:42.745 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:42.746 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:42.787 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:42.788 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:42.788 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:42.788 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:43.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:43.789 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:43.830 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:43.832 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:43.832 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:43.832 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:44.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:44.833 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:44.875 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:44.876 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:44.876 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:44.876 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:45.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:45.877 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:45.918 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:45.919 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:45.920 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:45.920 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:46.921 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:46.921 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:46.963 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:46.964 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:46.964 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:46.964 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:47.034 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:10:47.040 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:10:47.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:47.965 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:48.006 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:48.026 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:48.026 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:48.026 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:49.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:49.034 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:49.077 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:49.078 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:49.078 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:49.078 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:50.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:50.079 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:50.121 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:50.122 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:50.122 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:50.122 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:51.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:51.193 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:51.234 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:51.235 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:51.235 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:51.235 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:52.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:52.243 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:52.284 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:52.285 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:52.285 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:52.285 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:53.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:53.289 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:53.333 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:53.335 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:53.335 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:53.335 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:54.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:54.337 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:54.380 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:54.380 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:54.380 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:54.380 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:55.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:55.381 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:55.422 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:55.424 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:55.424 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:55.424 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:56.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:56.425 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:56.468 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:56.469 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:56.469 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:56.469 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:57.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:57.470 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:57.512 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:57.513 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:57.513 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:57.514 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:58.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:58.515 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:58.557 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:58.558 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:58.558 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:58.558 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:10:59.559 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:10:59.559 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:10:59.601 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:10:59.602 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:10:59.602 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:10:59.602 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:11:00.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:00.603 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:11:00.645 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:11:00.645 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_11_00_UTC 2023-10-08T01:11:00.646 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_11_00_UTC', 'scheduled-2023-10-08-01_10_00_UTC'} 2023-10-08T01:11:00.646 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-01_11_00_UTC'} 2023-10-08T01:11:00.646 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:11:00.646 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-08-01_11_00_UTC'] 2023-10-08T01:11:00.646 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696727366.048355, now = 1696727460.646932, timo = 94 2023-10-08T01:11:01.648 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-08T01:11:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.843+0000 7fd545fff700 1 Processor -- start 2023-10-08T01:11:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.844+0000 7fd545fff700 1 -- start start 2023-10-08T01:11:01.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd545fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd540068890 0x7fd5400c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:01.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd545fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54015a9b0 con 0x7fd54015a440 2023-10-08T01:11:01.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd545fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54015ac90 con 0x7fd5401563b0 2023-10-08T01:11:01.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd545fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd54015af70 con 0x7fd540068890 2023-10-08T01:11:01.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd537fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd54015a440 0x7fd5400acf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58334/0 (socket says 172.21.15.119:58334) 2023-10-08T01:11:01.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd537fff700 1 -- 172.21.15.119:0/1519048604 learned_addr learned my addr 172.21.15.119:0/1519048604 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:01.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd53ffff700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd540068890 0x7fd5400c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:01.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014020456 0 0) 0x7fd54015a9b0 con 0x7fd54015a440 2023-10-08T01:11:01.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.845+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd52c003570 con 0x7fd54015a440 2023-10-08T01:11:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1111184283 0 0) 0x7fd52c003570 con 0x7fd54015a440 2023-10-08T01:11:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd540068890 msgr2=0x7fd5400c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd540068890 0x7fd5400c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 >> v1:172.21.15.184:6789/0 conn(0x7fd5401563b0 legacy=0x7fd540156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54015b250 con 0x7fd54015a440 2023-10-08T01:11:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fd528001470 con 0x7fd54015a440 2023-10-08T01:11:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 >> v1:172.21.15.119:6789/0 conn(0x7fd54015a440 legacy=0x7fd5400acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c003a90 0x7fd52c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c004810 0x7fd52c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd540068890 0x7fd52c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54015af70 con 0x7fd52c003a90 2023-10-08T01:11:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c00bb10 con 0x7fd52c004810 2023-10-08T01:11:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd52c00dee0 con 0x7fd540068890 2023-10-08T01:11:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53f7fe700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c003a90 0x7fd52c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd537fff700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c004810 0x7fd52c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.846+0000 7fd53ffff700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd540068890 0x7fd52c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53f7fe700 1 -- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd540068890 msgr2=0x7fd52c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53f7fe700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd540068890 0x7fd52c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53f7fe700 1 -- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c004810 msgr2=0x7fd52c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53f7fe700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd52c004810 0x7fd52c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53f7fe700 1 -- 172.21.15.119:0/1519048604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd52c003570 con 0x7fd52c003a90 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53f7fe700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c003a90 0x7fd52c003e60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd5300019a0 tx=0x7fd530007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bd0dc23eb18a1b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000c5d0 con 0x7fd52c003a90 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53000c730 con 0x7fd52c003a90 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53effd700 1 -- 172.21.15.119:0/1519048604 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53000cad0 con 0x7fd52c003a90 2023-10-08T01:11:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd545fff700 1 -- 172.21.15.119:0/1519048604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c003a90 msgr2=0x7fd52c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:01.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd545fff700 1 --2- 172.21.15.119:0/1519048604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd52c003a90 0x7fd52c003e60 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fd5300019a0 tx=0x7fd530007d10 comp rx=0 tx=0).stop 2023-10-08T01:11:01.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.847+0000 7fd53ffff700 1 -- 172.21.15.119:0/1519048604 reap_dead start 2023-10-08T01:11:01.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.848+0000 7fd545fff700 1 -- 172.21.15.119:0/1519048604 shutdown_connections 2023-10-08T01:11:01.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.848+0000 7fd545fff700 1 -- 172.21.15.119:0/1519048604 >> 172.21.15.119:0/1519048604 conn(0x7fd5400c8f90 msgr2=0x7fd5400c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:01.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.848+0000 7fd545fff700 1 -- 172.21.15.119:0/1519048604 shutdown_connections 2023-10-08T01:11:01.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.848+0000 7fd545fff700 1 -- 172.21.15.119:0/1519048604 wait complete. 2023-10-08T01:11:01.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.849+0000 7fd545fff700 1 Processor -- start 2023-10-08T01:11:01.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.849+0000 7fd545fff700 1 -- start start 2023-10-08T01:11:01.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.850+0000 7fd545fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c003a90 0x7fd5400c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:01.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.850+0000 7fd545fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5400af2d0 0x7fd54014fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:01.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.850+0000 7fd545fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd540150450 0x7fd5401508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.850+0000 7fd545fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd540150e10 con 0x7fd540150450 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.850+0000 7fd545fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400b0740 con 0x7fd5400af2d0 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.850+0000 7fd545fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5400afae0 con 0x7fd52c003a90 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd540150450 0x7fd5401508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd540150450 0x7fd5401508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60422/0 (socket says 172.21.15.119:60422) 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 -- 172.21.15.119:0/739002384 learned_addr learned my addr 172.21.15.119:0/739002384 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53f7fe700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c003a90 0x7fd5400c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd537fff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5400af2d0 0x7fd54014fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 -- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c003a90 msgr2=0x7fd5400c2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c003a90 0x7fd5400c2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:01.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 -- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5400af2d0 msgr2=0x7fd54014fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5400af2d0 0x7fd54014fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 -- 172.21.15.119:0/739002384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400afdc0 con 0x7fd540150450 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53f7fe700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c003a90 0x7fd5400c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd537fff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5400af2d0 0x7fd54014fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd53ffff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd540150450 0x7fd5401508d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd5400c34c0 tx=0x7fd53800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd5377fe700 1 -- 172.21.15.119:0/739002384 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800e070 con 0x7fd540150450 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd5377fe700 1 -- 172.21.15.119:0/739002384 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd538011d00 con 0x7fd540150450 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd5377fe700 1 -- 172.21.15.119:0/739002384 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5380166c0 con 0x7fd540150450 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.851+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5400b00a0 con 0x7fd540150450 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.852+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5400af740 con 0x7fd540150450 2023-10-08T01:11:01.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.853+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5400b08b0 con 0x7fd540150450 2023-10-08T01:11:01.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.853+0000 7fd5377fe700 1 -- 172.21.15.119:0/739002384 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 28) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fd538016a70 con 0x7fd540150450 2023-10-08T01:11:01.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.855+0000 7fd5377fe700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd5200670a0 0x7fd520069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:01.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.855+0000 7fd5377fe700 1 -- 172.21.15.119:0/739002384 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6385+0+0 (secure 0 0 0) 0x7fd538085620 con 0x7fd540150450 2023-10-08T01:11:01.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.855+0000 7fd53f7fe700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd5200670a0 0x7fd520069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:01.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.856+0000 7fd53f7fe700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd5200670a0 0x7fd520069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd530008fd0 tx=0x7fd530015000 comp rx=0 tx=0).ready entity=mgr.5940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:01.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:01.860+0000 7fd5377fe700 1 -- 172.21.15.119:0/739002384 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd538052320 con 0x7fd540150450 2023-10-08T01:11:02.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.074+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 --> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd54006d6a0 con 0x7fd5200670a0 2023-10-08T01:11:02.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.145+0000 7fd5377fe700 1 -- 172.21.15.119:0/739002384 <== mgr.5940 v2:172.21.15.119:6834/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd54006d6a0 con 0x7fd5200670a0 2023-10-08T01:11:02.145 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path snap_test_dir1 2023-10-08T01:11:02.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd5200670a0 msgr2=0x7fd520069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd5200670a0 0x7fd520069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd530008fd0 tx=0x7fd530015000 comp rx=0 tx=0).stop 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd540150450 msgr2=0x7fd5401508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd540150450 0x7fd5401508d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fd5400c34c0 tx=0x7fd53800b040 comp rx=0 tx=0).stop 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 shutdown_connections 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd5200670a0 0x7fd520069550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd52c003a90 0x7fd5400c2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd540150450 0x7fd5401508d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 --2- 172.21.15.119:0/739002384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5400af2d0 0x7fd54014fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 >> 172.21.15.119:0/739002384 conn(0x7fd5400c8f90 msgr2=0x7fd5400c8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:02.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 shutdown_connections 2023-10-08T01:11:02.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.147+0000 7fd545fff700 1 -- 172.21.15.119:0/739002384 wait complete. 2023-10-08T01:11:02.164 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T01:11:02.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:02.165 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:11:02.201 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:11:02.201 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_11_00_UTC 2023-10-08T01:11:02.202 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_10_00_UTC 2023-10-08T01:11:02.203 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:02.203 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_10_00_UTC) 2023-10-08T01:11:02.285 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_11_00_UTC 2023-10-08T01:11:02.285 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:02.285 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_11_00_UTC) 2023-10-08T01:11:02.368 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:02.368 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T01:11:02.448 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T01:11:02.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.671+0000 7f81459ff700 1 Processor -- start 2023-10-08T01:11:02.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.672+0000 7f81459ff700 1 -- start start 2023-10-08T01:11:02.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.673+0000 7f81459ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81400c83f0 0x7f81400c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:02.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.673+0000 7f81459ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400ccee0 0x7f81401561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:02.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.673+0000 7f81459ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814015b0d0 con 0x7f81400c87c0 2023-10-08T01:11:02.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.673+0000 7f81459ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81400c7a60 con 0x7f81400c83f0 2023-10-08T01:11:02.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.673+0000 7f81459ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814015a4c0 con 0x7f81400ccee0 2023-10-08T01:11:02.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.673+0000 7f813e7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400ccee0 0x7f81401561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.673+0000 7f813effd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f81400c83f0 0x7f81400c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813e7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400ccee0 0x7f81401561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:59236/0 (socket says 172.21.15.119:59236) 2023-10-08T01:11:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813e7fc700 1 -- 172.21.15.119:0/1552034865 learned_addr learned my addr 172.21.15.119:0/1552034865 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813f7fe700 1 --1- 172.21.15.119:0/1552034865 >> v1:172.21.15.184:6789/0 conn(0x7f81400c87c0 0x7f814006eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:39422/0 (socket says 172.21.15.119:39422) 2023-10-08T01:11:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813dffb700 1 -- 172.21.15.119:0/1552034865 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263135011 0 0) 0x7f814015b0d0 con 0x7f81400c87c0 2023-10-08T01:11:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813dffb700 1 -- 172.21.15.119:0/1552034865 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8128003570 con 0x7f81400c87c0 2023-10-08T01:11:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813e7fc700 1 -- 172.21.15.119:0/1552034865 >> v2:172.21.15.184:3300/0 conn(0x7f81400c83f0 msgr2=0x7f81400c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813e7fc700 1 --2- 172.21.15.119:0/1552034865 >> v2:172.21.15.184:3300/0 conn(0x7f81400c83f0 0x7f81400c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813e7fc700 1 -- 172.21.15.119:0/1552034865 >> v1:172.21.15.184:6789/0 conn(0x7f81400c87c0 legacy=0x7f814006eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813e7fc700 1 -- 172.21.15.119:0/1552034865 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814015a7a0 con 0x7f81400ccee0 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813effd700 1 --2- 172.21.15.119:0/1552034865 >> v2:172.21.15.184:3300/0 conn(0x7f81400c83f0 0x7f81400c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.674+0000 7f813e7fc700 1 --2- 172.21.15.119:0/1552034865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400ccee0 0x7f81401561d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81340099b0 tx=0x7f8134009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae7792a51e9d92d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f813dffb700 1 -- 172.21.15.119:0/1552034865 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400b680 con 0x7f81400ccee0 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f813dffb700 1 -- 172.21.15.119:0/1552034865 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813400b7e0 con 0x7f81400ccee0 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f813dffb700 1 -- 172.21.15.119:0/1552034865 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400bb80 con 0x7f81400ccee0 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f81459ff700 1 -- 172.21.15.119:0/1552034865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400ccee0 msgr2=0x7f81401561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:02.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f81459ff700 1 --2- 172.21.15.119:0/1552034865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400ccee0 0x7f81401561d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81340099b0 tx=0x7f8134009250 comp rx=0 tx=0).stop 2023-10-08T01:11:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f81459ff700 1 -- 172.21.15.119:0/1552034865 shutdown_connections 2023-10-08T01:11:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f81459ff700 1 --2- 172.21.15.119:0/1552034865 >> v2:172.21.15.184:3300/0 conn(0x7f81400c83f0 0x7f81400c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f81459ff700 1 --2- 172.21.15.119:0/1552034865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400ccee0 0x7f81401561d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f81459ff700 1 -- 172.21.15.119:0/1552034865 >> 172.21.15.119:0/1552034865 conn(0x7f8140068e10 msgr2=0x7f81400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f81459ff700 1 -- 172.21.15.119:0/1552034865 shutdown_connections 2023-10-08T01:11:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.675+0000 7f81459ff700 1 -- 172.21.15.119:0/1552034865 wait complete. 2023-10-08T01:11:02.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.677+0000 7f81459ff700 1 Processor -- start 2023-10-08T01:11:02.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.677+0000 7f81459ff700 1 -- start start 2023-10-08T01:11:02.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f81459ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81400c83f0 0x7f81400b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:02.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f81459ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400c87c0 0x7f81400b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:02.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f81459ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81400ccee0 0x7f814014cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:02.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f81459ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8140155880 con 0x7f81400ccee0 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f81459ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81400b20a0 con 0x7f81400c83f0 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f81459ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81400af500 con 0x7f81400c87c0 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f813f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81400ccee0 0x7f814014cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f813f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81400ccee0 0x7f814014cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60428/0 (socket says 172.21.15.119:60428) 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f813f7fe700 1 -- 172.21.15.119:0/1439236414 learned_addr learned my addr 172.21.15.119:0/1439236414 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f813effd700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81400c83f0 0x7f81400b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.678+0000 7f813e7fc700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400c87c0 0x7f81400b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f813f7fe700 1 -- 172.21.15.119:0/1439236414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400c87c0 msgr2=0x7f81400b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f813f7fe700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400c87c0 0x7f81400b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f813f7fe700 1 -- 172.21.15.119:0/1439236414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81400c83f0 msgr2=0x7f81400b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f813f7fe700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81400c83f0 0x7f81400b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f813f7fe700 1 -- 172.21.15.119:0/1439236414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8130009420 con 0x7f81400ccee0 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f813f7fe700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81400ccee0 0x7f814014cd70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f813800c9b0 tx=0x7f8138001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f8126ffd700 1 -- 172.21.15.119:0/1439236414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800cea0 con 0x7f81400ccee0 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f8126ffd700 1 -- 172.21.15.119:0/1439236414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8138004da0 con 0x7f81400ccee0 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f8126ffd700 1 -- 172.21.15.119:0/1439236414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138006300 con 0x7f81400ccee0 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81340094f0 con 0x7f81400ccee0 2023-10-08T01:11:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.679+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814014a660 con 0x7f81400ccee0 2023-10-08T01:11:02.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.680+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81400af2d0 con 0x7f81400ccee0 2023-10-08T01:11:02.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.681+0000 7f8126ffd700 1 -- 172.21.15.119:0/1439236414 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 28) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f81380066f0 con 0x7f81400ccee0 2023-10-08T01:11:02.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.683+0000 7f8126ffd700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f812c0670a0 0x7f812c069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:02.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.683+0000 7f8126ffd700 1 -- 172.21.15.119:0/1439236414 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6385+0+0 (secure 0 0 0) 0x7f81380847f0 con 0x7f81400ccee0 2023-10-08T01:11:02.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.683+0000 7f813effd700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f812c0670a0 0x7f812c069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:02.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.684+0000 7f813effd700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f812c0670a0 0x7f812c069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8130009f40 tx=0x7f81300091a0 comp rx=0 tx=0).ready entity=mgr.5940 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:02.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.687+0000 7f8126ffd700 1 -- 172.21.15.119:0/1439236414 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81400af2d0 con 0x7f81400ccee0 2023-10-08T01:11:02.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:02.932+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f81400af2d0 con 0x7f81400ccee0 2023-10-08T01:11:03.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.280+0000 7f8126ffd700 1 -- 172.21.15.119:0/1439236414 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f81380519c0 con 0x7f81400ccee0 2023-10-08T01:11:03.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.281+0000 7f8126ffd700 1 -- 172.21.15.119:0/1439236414 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v29) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8138082580 con 0x7f81400ccee0 2023-10-08T01:11:03.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.284+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f812c0670a0 msgr2=0x7f812c069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:03.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.284+0000 7f81459ff700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f812c0670a0 0x7f812c069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8130009f40 tx=0x7f81300091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.284+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81400ccee0 msgr2=0x7f814014cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.284+0000 7f81459ff700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81400ccee0 0x7f814014cd70 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f813800c9b0 tx=0x7f8138001980 comp rx=0 tx=0).stop 2023-10-08T01:11:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.285+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 shutdown_connections 2023-10-08T01:11:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.285+0000 7f81459ff700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f812c0670a0 0x7f812c069550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.285+0000 7f81459ff700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81400ccee0 0x7f814014cd70 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.285+0000 7f81459ff700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f81400c83f0 0x7f81400b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.285+0000 7f81459ff700 1 --2- 172.21.15.119:0/1439236414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81400c87c0 0x7f81400b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:03.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.285+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 >> 172.21.15.119:0/1439236414 conn(0x7f8140068e10 msgr2=0x7f81400cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:03.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.285+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 shutdown_connections 2023-10-08T01:11:03.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.285+0000 7f81459ff700 1 -- 172.21.15.119:0/1439236414 wait complete. 2023-10-08T01:11:03.305 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:11:03.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:11:03.392 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:11:03.473 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:11:03.473 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T01:11:03.473 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:03.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T01:11:03.559 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T01:11:03.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:03.559 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T01:11:03.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.826+0000 7fd01199d700 1 Processor -- start 2023-10-08T01:11:03.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.826+0000 7fd01199d700 1 -- start start 2023-10-08T01:11:03.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.827+0000 7fd01199d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd00c0b2c80 0x7fd00c0cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.827+0000 7fd01199d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c0b1500 con 0x7fd00c06c0f0 2023-10-08T01:11:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.827+0000 7fd01199d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd00c0cda20 con 0x7fd00c140cd0 2023-10-08T01:11:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.827+0000 7fd01199d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00c0cdd00 con 0x7fd00c0b2c80 2023-10-08T01:11:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.827+0000 7fd00bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd00c06c0f0 0x7fd00c0ccd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58358/0 (socket says 172.21.15.119:58358) 2023-10-08T01:11:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.827+0000 7fd01119c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd00c0b2c80 0x7fd00c0cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.827+0000 7fd00bfff700 1 -- 172.21.15.119:0/2795502390 learned_addr learned my addr 172.21.15.119:0/2795502390 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.828+0000 7fd01119c700 1 -- 172.21.15.119:0/2795502390 >> v1:172.21.15.184:6789/0 conn(0x7fd00c140cd0 legacy=0x7fd00c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.828+0000 7fd01119c700 1 -- 172.21.15.119:0/2795502390 >> v1:172.21.15.119:6789/0 conn(0x7fd00c06c0f0 legacy=0x7fd00c0ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.828+0000 7fd00b7fe700 1 -- 172.21.15.119:0/2795502390 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968496790 0 0) 0x7fd00c0b1500 con 0x7fd00c06c0f0 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.828+0000 7fd01119c700 1 -- 172.21.15.119:0/2795502390 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c0cdfe0 con 0x7fd00c0b2c80 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.828+0000 7fd01119c700 1 --2- 172.21.15.119:0/2795502390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c0cd420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd004009e90 tx=0x7fd004009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113f168dc4c78264 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.828+0000 7fd00b7fe700 1 -- 172.21.15.119:0/2795502390 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00400e070 con 0x7fd00c0b2c80 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd00b7fe700 1 -- 172.21.15.119:0/2795502390 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00400bbf0 con 0x7fd00c0b2c80 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd00b7fe700 1 -- 172.21.15.119:0/2795502390 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd004012430 con 0x7fd00c0b2c80 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd01199d700 1 -- 172.21.15.119:0/2795502390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 msgr2=0x7fd00c0cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:03.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd01199d700 1 --2- 172.21.15.119:0/2795502390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c0cd420 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd004009e90 tx=0x7fd004009780 comp rx=0 tx=0).stop 2023-10-08T01:11:03.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd01199d700 1 -- 172.21.15.119:0/2795502390 shutdown_connections 2023-10-08T01:11:03.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd01199d700 1 --2- 172.21.15.119:0/2795502390 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c0cd420 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:03.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd01199d700 1 -- 172.21.15.119:0/2795502390 >> 172.21.15.119:0/2795502390 conn(0x7fd00c0ce430 msgr2=0x7fd00c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:03.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd01199d700 1 -- 172.21.15.119:0/2795502390 shutdown_connections 2023-10-08T01:11:03.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.829+0000 7fd01199d700 1 -- 172.21.15.119:0/2795502390 wait complete. 2023-10-08T01:11:03.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.831+0000 7fd01199d700 1 Processor -- start 2023-10-08T01:11:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.831+0000 7fd01199d700 1 -- start start 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd01199d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c06c0f0 0x7fd00c1387c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd01199d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c13a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd01199d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c140cd0 0x7fd00c137740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd01199d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c16d060 con 0x7fd00c0b2c80 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd01199d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c16d340 con 0x7fd00c06c0f0 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd01199d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00c16d620 con 0x7fd00c140cd0 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd00bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c13a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd00bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c13a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60444/0 (socket says 172.21.15.119:60444) 2023-10-08T01:11:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd00bfff700 1 -- 172.21.15.119:0/1774078047 learned_addr learned my addr 172.21.15.119:0/1774078047 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd01099b700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c06c0f0 0x7fd00c1387c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd00bfff700 1 -- 172.21.15.119:0/1774078047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c140cd0 msgr2=0x7fd00c137740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd00bfff700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c140cd0 0x7fd00c137740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd00bfff700 1 -- 172.21.15.119:0/1774078047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c06c0f0 msgr2=0x7fd00c1387c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd00bfff700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c06c0f0 0x7fd00c1387c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.832+0000 7fd00bfff700 1 -- 172.21.15.119:0/1774078047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd004009a20 con 0x7fd00c0b2c80 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.833+0000 7fd00bfff700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c13a1d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c7e0 tx=0x7fcff8007d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.833+0000 7fd008ff9700 1 -- 172.21.15.119:0/1774078047 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8005c00 con 0x7fd00c0b2c80 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.833+0000 7fd008ff9700 1 -- 172.21.15.119:0/1774078047 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8005d80 con 0x7fd00c0b2c80 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.833+0000 7fd008ff9700 1 -- 172.21.15.119:0/1774078047 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8007320 con 0x7fd00c0b2c80 2023-10-08T01:11:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.833+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00c16d900 con 0x7fd00c0b2c80 2023-10-08T01:11:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.833+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd00c13a710 con 0x7fd00c0b2c80 2023-10-08T01:11:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.834+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd00c0c9520 con 0x7fd00c0b2c80 2023-10-08T01:11:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.836+0000 7fd008ff9700 1 -- 172.21.15.119:0/1774078047 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fcff801b3b0 con 0x7fd00c0b2c80 2023-10-08T01:11:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.837+0000 7fd008ff9700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcff0067050 0x7fcff0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.837+0000 7fd008ff9700 1 -- 172.21.15.119:0/1774078047 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6473+0+0 (secure 0 0 0) 0x7fcff8083f50 con 0x7fd00c0b2c80 2023-10-08T01:11:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.837+0000 7fd01099b700 1 -- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcff0067050 msgr2=0x7fcff0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.837+0000 7fd01099b700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcff0067050 0x7fcff0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:03.842+0000 7fd008ff9700 1 -- 172.21.15.119:0/1774078047 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd00c0c9520 con 0x7fd00c0b2c80 2023-10-08T01:11:04.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.038+0000 7fd01099b700 1 -- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcff0067050 msgr2=0x7fcff0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:04.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.038+0000 7fd01099b700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcff0067050 0x7fcff0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:04.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.085+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd00c0c9520 con 0x7fd00c0b2c80 2023-10-08T01:11:04.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.085+0000 7fd008ff9700 1 -- 172.21.15.119:0/1774078047 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fd00c0c9520 con 0x7fd00c0b2c80 2023-10-08T01:11:04.087 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:04.087 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1019078588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1019078588},{"type":"v1","addr":"172.21.15.119:6835","nonce":1019078588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1353749238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1353749238},{"type":"v1","addr":"172.21.15.184:6834","nonce":1353749238}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/186953052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":186953052},{"type":"v1","addr":"172.21.15.184:6835","nonce":186953052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:09:08.771490+0000","modified":"2023-10-08T01:09:12.322278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/920039174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":920039174},{"type":"v1","addr":"172.21.15.119:6837","nonce":920039174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T01:11:04.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcff0067050 msgr2=0x7fcff0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:04.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcff0067050 0x7fcff0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 msgr2=0x7fd00c13a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c13a1d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcff800c7e0 tx=0x7fcff8007d70 comp rx=0 tx=0).stop 2023-10-08T01:11:04.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 shutdown_connections 2023-10-08T01:11:04.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fcff0067050 0x7fcff0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd00c140cd0 0x7fd00c137740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd00c06c0f0 0x7fd00c1387c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 --2- 172.21.15.119:0/1774078047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd00c0b2c80 0x7fd00c13a1d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 >> 172.21.15.119:0/1774078047 conn(0x7fd00c0ce430 msgr2=0x7fd00c13ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:04.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 shutdown_connections 2023-10-08T01:11:04.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.088+0000 7fd01199d700 1 -- 172.21.15.119:0/1774078047 wait complete. 2023-10-08T01:11:04.092 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 24 2023-10-08T01:11:04.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.299+0000 7f538539d700 1 Processor -- start 2023-10-08T01:11:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.299+0000 7f538539d700 1 -- start start 2023-10-08T01:11:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.300+0000 7f538539d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5380159420 0x7f53800b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.300+0000 7f538539d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f538015d230 0x7f538015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.300+0000 7f538539d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53800b2980 con 0x7f53801597f0 2023-10-08T01:11:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.300+0000 7f538539d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538015db70 con 0x7f538015d230 2023-10-08T01:11:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.300+0000 7f538539d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53800b11d0 con 0x7f5380159420 2023-10-08T01:11:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.301+0000 7f5384b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f538015d230 0x7f538015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.301+0000 7f5384b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f538015d230 0x7f538015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60446/0 (socket says 172.21.15.119:60446) 2023-10-08T01:11:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.301+0000 7f5384b9c700 1 -- 172.21.15.119:0/2636828750 learned_addr learned my addr 172.21.15.119:0/2636828750 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.301+0000 7f537ffff700 1 --2- 172.21.15.119:0/2636828750 >> v2:172.21.15.184:3300/0 conn(0x7f5380159420 0x7f53800b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.301+0000 7f5384b9c700 1 -- 172.21.15.119:0/2636828750 >> v2:172.21.15.184:3300/0 conn(0x7f5380159420 msgr2=0x7f53800b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.301+0000 7f5384b9c700 1 --2- 172.21.15.119:0/2636828750 >> v2:172.21.15.184:3300/0 conn(0x7f5380159420 0x7f53800b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.301+0000 7f5384b9c700 1 -- 172.21.15.119:0/2636828750 >> v1:172.21.15.184:6789/0 conn(0x7f53801597f0 legacy=0x7f53800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.301+0000 7f5384b9c700 1 -- 172.21.15.119:0/2636828750 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53800b14b0 con 0x7f538015d230 2023-10-08T01:11:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f5384b9c700 1 --2- 172.21.15.119:0/2636828750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f538015d630 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f53800b2f20 tx=0x7f5378009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc23710245166485 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f537effd700 1 -- 172.21.15.119:0/2636828750 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537800e070 con 0x7f538015d230 2023-10-08T01:11:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f537effd700 1 -- 172.21.15.119:0/2636828750 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537800ba20 con 0x7f538015d230 2023-10-08T01:11:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f537effd700 1 -- 172.21.15.119:0/2636828750 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780092e0 con 0x7f538015d230 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f538539d700 1 -- 172.21.15.119:0/2636828750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 msgr2=0x7f538015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f538539d700 1 --2- 172.21.15.119:0/2636828750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f538015d630 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f53800b2f20 tx=0x7f5378009840 comp rx=0 tx=0).stop 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f538539d700 1 -- 172.21.15.119:0/2636828750 shutdown_connections 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f538539d700 1 --2- 172.21.15.119:0/2636828750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f538015d630 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f538539d700 1 --2- 172.21.15.119:0/2636828750 >> v2:172.21.15.184:3300/0 conn(0x7f5380159420 0x7f53800b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.302+0000 7f538539d700 1 -- 172.21.15.119:0/2636828750 >> 172.21.15.119:0/2636828750 conn(0x7f53800d0660 msgr2=0x7f53800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.303+0000 7f538539d700 1 -- 172.21.15.119:0/2636828750 shutdown_connections 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.303+0000 7f538539d700 1 -- 172.21.15.119:0/2636828750 wait complete. 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.304+0000 7f538539d700 1 Processor -- start 2023-10-08T01:11:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.304+0000 7f538539d700 1 -- start start 2023-10-08T01:11:04.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f538539d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5380159420 0x7f5380155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f538539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53801597f0 0x7f53800c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f538539d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f53800c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f538539d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53800c9740 con 0x7f538015d230 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f538539d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53800c5400 con 0x7f5380159420 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f538539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53800c5650 con 0x7f53801597f0 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f5384b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f53800c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f5384b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f53800c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60456/0 (socket says 172.21.15.119:60456) 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f5384b9c700 1 -- 172.21.15.119:0/4139574245 learned_addr learned my addr 172.21.15.119:0/4139574245 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.305+0000 7f537ffff700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5380159420 0x7f5380155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f537f7fe700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53801597f0 0x7f53800c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5384b9c700 1 -- 172.21.15.119:0/4139574245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53801597f0 msgr2=0x7f53800c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5384b9c700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53801597f0 0x7f53800c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5384b9c700 1 -- 172.21.15.119:0/4139574245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5380159420 msgr2=0x7f5380155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5384b9c700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5380159420 0x7f5380155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5384b9c700 1 -- 172.21.15.119:0/4139574245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5378009ae0 con 0x7f538015d230 2023-10-08T01:11:04.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5384b9c700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f53800c7dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5378009880 tx=0x7f5378018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:04.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5367fff700 1 -- 172.21.15.119:0/4139574245 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378008a80 con 0x7f538015d230 2023-10-08T01:11:04.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5367fff700 1 -- 172.21.15.119:0/4139574245 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5378008c00 con 0x7f538015d230 2023-10-08T01:11:04.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53800c5930 con 0x7f538015d230 2023-10-08T01:11:04.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.306+0000 7f5367fff700 1 -- 172.21.15.119:0/4139574245 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537801c410 con 0x7f538015d230 2023-10-08T01:11:04.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.307+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53800c5ef0 con 0x7f538015d230 2023-10-08T01:11:04.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.308+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538004c3c0 con 0x7f538015d230 2023-10-08T01:11:04.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.308+0000 7f5367fff700 1 -- 172.21.15.119:0/4139574245 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f5378003040 con 0x7f538015d230 2023-10-08T01:11:04.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.309+0000 7f5367fff700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f536c067050 0x7f536c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.309+0000 7f5367fff700 1 -- 172.21.15.119:0/4139574245 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f5378086250 con 0x7f538015d230 2023-10-08T01:11:04.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.309+0000 7f537ffff700 1 -- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f536c067050 msgr2=0x7f536c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:04.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.309+0000 7f537ffff700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f536c067050 0x7f536c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:04.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.313+0000 7f5367fff700 1 -- 172.21.15.119:0/4139574245 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5378052f10 con 0x7f538015d230 2023-10-08T01:11:04.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.510+0000 7f537ffff700 1 -- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f536c067050 msgr2=0x7f536c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:04.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.510+0000 7f537ffff700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f536c067050 0x7f536c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:04.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.527+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53800b1790 con 0x7f538015d230 2023-10-08T01:11:04.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.528+0000 7f5367fff700 1 -- 172.21.15.119:0/4139574245 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16548 (secure 0 0 0) 0x7f5378056530 con 0x7f538015d230 2023-10-08T01:11:04.529 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:04.529 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":64,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:04.273139+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:09:06.334038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T01:09:07.759893+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.3c","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/593291478":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/4075293652":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/3952125277":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/91830571":"2023-10-09T01:09:24.445449+0000","172.21.15.119:6836/85511":"2023-10-09T01:09:24.445449+0000","172.21.15.119:6834/85511":"2023-10-09T01:09:24.445449+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:04.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f536c067050 msgr2=0x7f536c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f536c067050 0x7f536c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 msgr2=0x7f53800c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f53800c7dc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5378009880 tx=0x7f5378018000 comp rx=0 tx=0).stop 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 shutdown_connections 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f536c067050 0x7f536c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538015d230 0x7f53800c7dc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5380159420 0x7f5380155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 --2- 172.21.15.119:0/4139574245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f53801597f0 0x7f53800c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 >> 172.21.15.119:0/4139574245 conn(0x7f53800d0660 msgr2=0x7f53801586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 shutdown_connections 2023-10-08T01:11:04.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.530+0000 7f538539d700 1 -- 172.21.15.119:0/4139574245 wait complete. 2023-10-08T01:11:04.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T01:09:06.334038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T01:09:07.759893+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 64, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T01:11:04.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:11:04.579 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:04.578+0000 7f3f4cc64180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:11:04.646 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:04.644+0000 7f695a0ab180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:11:04.742 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:04.741+0000 7f3f4cc64180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:04.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.745+0000 7f6f4ed9d700 1 Processor -- start 2023-10-08T01:11:04.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.745+0000 7f6f4ed9d700 1 -- start start 2023-10-08T01:11:04.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f480c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4ed9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480cd240 con 0x7f6f48157680 2023-10-08T01:11:04.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4ed9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f480c9a20 con 0x7f6f480cb590 2023-10-08T01:11:04.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4ed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480cc700 con 0x7f6f48157a50 2023-10-08T01:11:04.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4dd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6f48157680 0x7f6f480cae80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58364/0 (socket says 172.21.15.119:58364) 2023-10-08T01:11:04.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4dd9b700 1 -- 172.21.15.119:0/248294255 learned_addr learned my addr 172.21.15.119:0/248294255 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:04.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4d59a700 1 --2- 172.21.15.119:0/248294255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f480c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:04.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4cd99700 1 -- 172.21.15.119:0/248294255 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1157690888 0 0) 0x7f6f480cd240 con 0x7f6f48157680 2023-10-08T01:11:04.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4cd99700 1 -- 172.21.15.119:0/248294255 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f34003570 con 0x7f6f48157680 2023-10-08T01:11:04.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4cd99700 1 -- 172.21.15.119:0/248294255 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 153418447 0 0) 0x7f6f480c9a20 con 0x7f6f480cb590 2023-10-08T01:11:04.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4cd99700 1 -- 172.21.15.119:0/248294255 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f480cd240 con 0x7f6f480cb590 2023-10-08T01:11:04.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4d59a700 1 -- 172.21.15.119:0/248294255 >> v1:172.21.15.184:6789/0 conn(0x7f6f480cb590 legacy=0x7f6f480c9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4d59a700 1 -- 172.21.15.119:0/248294255 >> v1:172.21.15.119:6789/0 conn(0x7f6f48157680 legacy=0x7f6f480cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.747+0000 7f6f4d59a700 1 -- 172.21.15.119:0/248294255 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480cc910 con 0x7f6f48157a50 2023-10-08T01:11:04.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.748+0000 7f6f4d59a700 1 --2- 172.21.15.119:0/248294255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f480c8cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009910 tx=0x7f6f380091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f36c891ce92d79 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:04.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.748+0000 7f6f4cd99700 1 -- 172.21.15.119:0/248294255 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38011630 con 0x7f6f48157a50 2023-10-08T01:11:04.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.748+0000 7f6f4cd99700 1 -- 172.21.15.119:0/248294255 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38011790 con 0x7f6f48157a50 2023-10-08T01:11:04.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.748+0000 7f6f4cd99700 1 -- 172.21.15.119:0/248294255 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38011b30 con 0x7f6f48157a50 2023-10-08T01:11:04.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.749+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/248294255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 msgr2=0x7f6f480c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.749+0000 7f6f4ed9d700 1 --2- 172.21.15.119:0/248294255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f480c8cd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009910 tx=0x7f6f380091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.749+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/248294255 shutdown_connections 2023-10-08T01:11:04.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.749+0000 7f6f4ed9d700 1 --2- 172.21.15.119:0/248294255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f480c8cd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.749+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/248294255 >> 172.21.15.119:0/248294255 conn(0x7f6f4806baa0 msgr2=0x7f6f4806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:04.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.749+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/248294255 shutdown_connections 2023-10-08T01:11:04.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.749+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/248294255 wait complete. 2023-10-08T01:11:04.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.750+0000 7f6f4ed9d700 1 Processor -- start 2023-10-08T01:11:04.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.750+0000 7f6f4ed9d700 1 -- start start 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.751+0000 7f6f4ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f480cb590 0x7f6f48151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.751+0000 7f6f4ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f48157680 0x7f6f48169ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.751+0000 7f6f4ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f4815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.751+0000 7f6f4ed9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4816aa50 con 0x7f6f48157680 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.751+0000 7f6f4ed9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4816d5e0 con 0x7f6f480cb590 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.751+0000 7f6f4ed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4816a490 con 0x7f6f48157a50 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.751+0000 7f6f4d59a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f48157680 0x7f6f48169ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4d59a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f48157680 0x7f6f48169ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60460/0 (socket says 172.21.15.119:60460) 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4d59a700 1 -- 172.21.15.119:0/3517205627 learned_addr learned my addr 172.21.15.119:0/3517205627 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4e59c700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f4815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:04.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4d59a700 1 -- 172.21.15.119:0/3517205627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 msgr2=0x7f6f4815b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4d59a700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f4815b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4d59a700 1 -- 172.21.15.119:0/3517205627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f480cb590 msgr2=0x7f6f48151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4d59a700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f480cb590 0x7f6f48151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4d59a700 1 -- 172.21.15.119:0/3517205627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f38009420 con 0x7f6f48157680 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.752+0000 7f6f4d59a700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f48157680 0x7f6f48169ad0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6f4806cdc0 tx=0x7f6f38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.753+0000 7f6f3dffb700 1 -- 172.21.15.119:0/3517205627 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3801f070 con 0x7f6f48157680 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.753+0000 7f6f3dffb700 1 -- 172.21.15.119:0/3517205627 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38000bc0 con 0x7f6f48157680 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.753+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4816a770 con 0x7f6f48157680 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.753+0000 7f6f3dffb700 1 -- 172.21.15.119:0/3517205627 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38025420 con 0x7f6f48157680 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.753+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4816a010 con 0x7f6f48157680 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.754+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4816d0d0 con 0x7f6f48157680 2023-10-08T01:11:04.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.756+0000 7f6f3dffb700 1 -- 172.21.15.119:0/3517205627 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f6f38023070 con 0x7f6f48157680 2023-10-08T01:11:04.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.757+0000 7f6f3dffb700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6f2c067050 0x7f6f2c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:04.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.757+0000 7f6f3dffb700 1 -- 172.21.15.119:0/3517205627 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f6f38085bf0 con 0x7f6f48157680 2023-10-08T01:11:04.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.757+0000 7f6f4dd9b700 1 -- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6f2c067050 msgr2=0x7f6f2c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:04.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.757+0000 7f6f4dd9b700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6f2c067050 0x7f6f2c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:04.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.761+0000 7f6f3dffb700 1 -- 172.21.15.119:0/3517205627 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f4816d0d0 con 0x7f6f48157680 2023-10-08T01:11:04.821 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:04.820+0000 7f695a0ab180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:04.827 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:04.826+0000 7f3f4cc64180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:11:04.909 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:04.908+0000 7f695a0ab180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:11:04.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.958+0000 7f6f4dd9b700 1 -- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6f2c067050 msgr2=0x7f6f2c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:04.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.958+0000 7f6f4dd9b700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6f2c067050 0x7f6f2c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:04.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.959+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f4816d0d0 con 0x7f6f48157680 2023-10-08T01:11:04.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.959+0000 7f6f3dffb700 1 -- 172.21.15.119:0/3517205627 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f4816d0d0 con 0x7f6f48157680 2023-10-08T01:11:04.961 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:11:04.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6f2c067050 msgr2=0x7f6f2c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:04.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6f2c067050 0x7f6f2c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f48157680 msgr2=0x7f6f48169ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:04.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f48157680 0x7f6f48169ad0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6f4806cdc0 tx=0x7f6f38017000 comp rx=0 tx=0).stop 2023-10-08T01:11:04.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 shutdown_connections 2023-10-08T01:11:04.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f6f2c067050 0x7f6f2c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f48157a50 0x7f6f4815b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f480cb590 0x7f6f48151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 --2- 172.21.15.119:0/3517205627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f48157680 0x7f6f48169ad0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:04.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 >> 172.21.15.119:0/3517205627 conn(0x7f6f4806baa0 msgr2=0x7f6f480cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:04.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 shutdown_connections 2023-10-08T01:11:04.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:04.962+0000 7f6f4ed9d700 1 -- 172.21.15.119:0/3517205627 wait complete. 2023-10-08T01:11:04.982 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T01:11:05.058 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:05.054+0000 7f3f4cc64180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:05.145 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:05.143+0000 7f695a0ab180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:05.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.171+0000 7f956339d700 1 Processor -- start 2023-10-08T01:11:05.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.171+0000 7f956339d700 1 -- start start 2023-10-08T01:11:05.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f956339d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f955c0b4e70 0x7f955c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f956339d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c0cfb00 con 0x7f955c0b16d0 2023-10-08T01:11:05.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f956339d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955c15e060 con 0x7f955c0b1b40 2023-10-08T01:11:05.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f956339d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f955c0b5c40 con 0x7f955c0b4e70 2023-10-08T01:11:05.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f9561b9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f955c0b16d0 0x7f955c15d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54526/0 (socket says 172.21.15.119:54526) 2023-10-08T01:11:05.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f9561b9a700 1 -- 172.21.15.119:0/3988206231 learned_addr learned my addr 172.21.15.119:0/3988206231 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:05.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f956239b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f955c0b4e70 0x7f955c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f956239b700 1 -- 172.21.15.119:0/3988206231 >> v1:172.21.15.184:6789/0 conn(0x7f955c0b1b40 legacy=0x7f955c15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f956239b700 1 -- 172.21.15.119:0/3988206231 >> v1:172.21.15.119:6789/0 conn(0x7f955c0b16d0 legacy=0x7f955c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f956239b700 1 -- 172.21.15.119:0/3988206231 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955c0b5f20 con 0x7f955c0b4e70 2023-10-08T01:11:05.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.173+0000 7f9561399700 1 -- 172.21.15.119:0/3988206231 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614289443 0 0) 0x7f955c15e060 con 0x7f955c0b1b40 2023-10-08T01:11:05.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.174+0000 7f956239b700 1 --2- 172.21.15.119:0/3988206231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b4e70 0x7f955c0ccff0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f95580098e0 tx=0x7f95580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9efd9bcdbc36323 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:05.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.174+0000 7f9561399700 1 -- 172.21.15.119:0/3988206231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9558009e80 con 0x7f955c0b4e70 2023-10-08T01:11:05.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.174+0000 7f9561399700 1 -- 172.21.15.119:0/3988206231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955800b860 con 0x7f955c0b4e70 2023-10-08T01:11:05.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.174+0000 7f9561399700 1 -- 172.21.15.119:0/3988206231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955800bbe0 con 0x7f955c0b4e70 2023-10-08T01:11:05.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.175+0000 7f956339d700 1 -- 172.21.15.119:0/3988206231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b4e70 msgr2=0x7f955c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.175+0000 7f956339d700 1 --2- 172.21.15.119:0/3988206231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b4e70 0x7f955c0ccff0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f95580098e0 tx=0x7f95580091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.175+0000 7f956339d700 1 -- 172.21.15.119:0/3988206231 shutdown_connections 2023-10-08T01:11:05.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.175+0000 7f956339d700 1 --2- 172.21.15.119:0/3988206231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b4e70 0x7f955c0ccff0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.175+0000 7f956339d700 1 -- 172.21.15.119:0/3988206231 >> 172.21.15.119:0/3988206231 conn(0x7f955c0d10c0 msgr2=0x7f955c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:05.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.175+0000 7f956339d700 1 -- 172.21.15.119:0/3988206231 shutdown_connections 2023-10-08T01:11:05.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.175+0000 7f956339d700 1 -- 172.21.15.119:0/3988206231 wait complete. 2023-10-08T01:11:05.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.176+0000 7f956339d700 1 Processor -- start 2023-10-08T01:11:05.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.176+0000 7f956339d700 1 -- start start 2023-10-08T01:11:05.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956339d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b16d0 0x7f955c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f955c0b1b40 0x7f955c155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956339d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f955c0b4e70 0x7f955c0b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956339d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c154d30 con 0x7f955c0b16d0 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956339d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955c153e40 con 0x7f955c0b4e70 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f955c154120 con 0x7f955c0b1b40 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956239b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b16d0 0x7f955c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956239b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b16d0 0x7f955c0c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39594/0 (socket says 172.21.15.119:39594) 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.177+0000 7f956239b700 1 -- 172.21.15.119:0/3481848773 learned_addr learned my addr 172.21.15.119:0/3481848773 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f9561b9a700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f955c0b1b40 0x7f955c155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f9562b9c700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f955c0b4e70 0x7f955c0b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f956239b700 1 -- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f955c0b1b40 msgr2=0x7f955c155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f956239b700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f955c0b1b40 0x7f955c155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f956239b700 1 -- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f955c0b4e70 msgr2=0x7f955c0b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f956239b700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f955c0b4e70 0x7f955c0b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f956239b700 1 -- 172.21.15.119:0/3481848773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9558009420 con 0x7f955c0b16d0 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f9562b9c700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f955c0b4e70 0x7f955c0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f956239b700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b16d0 0x7f955c0c5940 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f955c070dd0 tx=0x7f9558017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f95527fc700 1 -- 172.21.15.119:0/3481848773 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955801f070 con 0x7f955c0b16d0 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f95527fc700 1 -- 172.21.15.119:0/3481848773 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9558011420 con 0x7f955c0b16d0 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f95527fc700 1 -- 172.21.15.119:0/3481848773 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f955800bbe0 con 0x7f955c0b16d0 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.178+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955c154400 con 0x7f955c0b16d0 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.179+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955c1549c0 con 0x7f955c0b16d0 2023-10-08T01:11:05.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.179+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955c088990 con 0x7f955c0b16d0 2023-10-08T01:11:05.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.180+0000 7f95527fc700 1 -- 172.21.15.119:0/3481848773 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f9558011a50 con 0x7f955c0b16d0 2023-10-08T01:11:05.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.181+0000 7f95527fc700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9540067050 0x7f9540069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.181+0000 7f95527fc700 1 -- 172.21.15.119:0/3481848773 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f9558085f00 con 0x7f955c0b16d0 2023-10-08T01:11:05.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.181+0000 7f9561b9a700 1 -- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9540067050 msgr2=0x7f9540069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:05.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.181+0000 7f9561b9a700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9540067050 0x7f9540069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:05.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.185+0000 7f95527fc700 1 -- 172.21.15.119:0/3481848773 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9558052cd0 con 0x7f955c0b16d0 2023-10-08T01:11:05.233 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:05.231+0000 7f3f4cc64180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:11:05.325 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:05.323+0000 7f3f4cc64180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:11:05.330 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:05.330+0000 7f695a0ab180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:11:05.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.382+0000 7f9561b9a700 1 -- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9540067050 msgr2=0x7f9540069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:05.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.382+0000 7f9561b9a700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9540067050 0x7f9540069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:05.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.388+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f955c04bb80 con 0x7f955c0b16d0 2023-10-08T01:11:05.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.389+0000 7f95527fc700 1 -- 172.21.15.119:0/3481848773 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9558015020 con 0x7f955c0b16d0 2023-10-08T01:11:05.390 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T01:11:05.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9540067050 msgr2=0x7f9540069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:05.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9540067050 0x7f9540069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b16d0 msgr2=0x7f955c0c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b16d0 0x7f955c0c5940 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f955c070dd0 tx=0x7f9558017000 comp rx=0 tx=0).stop 2023-10-08T01:11:05.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 shutdown_connections 2023-10-08T01:11:05.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f9540067050 0x7f9540069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f955c0b4e70 0x7f955c0b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f955c0b16d0 0x7f955c0c5940 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 --2- 172.21.15.119:0/3481848773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f955c0b1b40 0x7f955c155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 >> 172.21.15.119:0/3481848773 conn(0x7f955c0d10c0 msgr2=0x7f955c0ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:05.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 shutdown_connections 2023-10-08T01:11:05.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.391+0000 7f956339d700 1 -- 172.21.15.119:0/3481848773 wait complete. 2023-10-08T01:11:05.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T01:11:05.428 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:05.427+0000 7f695a0ab180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:11:05.487 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:05.486+0000 7f3f4cc64180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:11:05.588 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:05.587+0000 7f3f4cc64180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:05.597 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:05.596+0000 7f695a0ab180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:11:05.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.602+0000 7fe20919d700 1 Processor -- start 2023-10-08T01:11:05.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.602+0000 7fe20919d700 1 -- start start 2023-10-08T01:11:05.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe20919d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe2040ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe20919d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2040b16d0 0x7fe2040b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe20919d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2040b2080 con 0x7fe20415d240 2023-10-08T01:11:05.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe20919d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2040cfb00 con 0x7fe2040b16d0 2023-10-08T01:11:05.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe20919d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2040b5c40 con 0x7fe2040b4e70 2023-10-08T01:11:05.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe20899c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe20415d240 0x7fe20415d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54538/0 (socket says 172.21.15.119:54538) 2023-10-08T01:11:05.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe20899c700 1 -- 172.21.15.119:0/1226902652 learned_addr learned my addr 172.21.15.119:0/1226902652 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:05.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe2037fe700 1 --2- 172.21.15.119:0/1226902652 >> v2:172.21.15.184:3300/0 conn(0x7fe2040b16d0 0x7fe2040b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.603+0000 7fe203fff700 1 --2- 172.21.15.119:0/1226902652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe2040ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.604+0000 7fe202ffd700 1 -- 172.21.15.119:0/1226902652 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1747872300 0 0) 0x7fe2040b2080 con 0x7fe20415d240 2023-10-08T01:11:05.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.604+0000 7fe202ffd700 1 -- 172.21.15.119:0/1226902652 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ec003570 con 0x7fe20415d240 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.604+0000 7fe2037fe700 1 -- 172.21.15.119:0/1226902652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 msgr2=0x7fe2040ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.604+0000 7fe2037fe700 1 --2- 172.21.15.119:0/1226902652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe2040ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.604+0000 7fe2037fe700 1 -- 172.21.15.119:0/1226902652 >> v1:172.21.15.119:6789/0 conn(0x7fe20415d240 legacy=0x7fe20415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.604+0000 7fe2037fe700 1 -- 172.21.15.119:0/1226902652 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2040b5f20 con 0x7fe2040b16d0 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.604+0000 7fe2037fe700 1 --2- 172.21.15.119:0/1226902652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe2040b1b40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80098e0 tx=0x7fe1f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28985698b059be85 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.604+0000 7fe202ffd700 1 -- 172.21.15.119:0/1226902652 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8011630 con 0x7fe2040b16d0 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe202ffd700 1 -- 172.21.15.119:0/1226902652 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f8011790 con 0x7fe2040b16d0 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe202ffd700 1 -- 172.21.15.119:0/1226902652 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8011b30 con 0x7fe2040b16d0 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe20919d700 1 -- 172.21.15.119:0/1226902652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 msgr2=0x7fe2040b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe20919d700 1 --2- 172.21.15.119:0/1226902652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe2040b1b40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe1f80098e0 tx=0x7fe1f80091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe20919d700 1 -- 172.21.15.119:0/1226902652 shutdown_connections 2023-10-08T01:11:05.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe20919d700 1 --2- 172.21.15.119:0/1226902652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe2040ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe20919d700 1 --2- 172.21.15.119:0/1226902652 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe2040b1b40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe20919d700 1 -- 172.21.15.119:0/1226902652 >> 172.21.15.119:0/1226902652 conn(0x7fe2040d10c0 msgr2=0x7fe2040affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:05.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe20919d700 1 -- 172.21.15.119:0/1226902652 shutdown_connections 2023-10-08T01:11:05.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.605+0000 7fe20919d700 1 -- 172.21.15.119:0/1226902652 wait complete. 2023-10-08T01:11:05.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.607+0000 7fe20919d700 1 Processor -- start 2023-10-08T01:11:05.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.607+0000 7fe20919d700 1 -- start start 2023-10-08T01:11:05.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe20919d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe20415f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe20919d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe20416a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe20919d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20415d240 0x7fe20415dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe20919d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20416ae40 con 0x7fe20415d240 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe20919d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20416b0e0 con 0x7fe2040b16d0 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe20919d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20416b3c0 con 0x7fe2040b4e70 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe203fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe20415f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe2037fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe20416a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe20899c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20415d240 0x7fe20415dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe203fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe20415f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:51070/0 (socket says 172.21.15.119:51070) 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe203fff700 1 -- 172.21.15.119:0/285118817 learned_addr learned my addr 172.21.15.119:0/285118817 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:05.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe2037fe700 1 -- 172.21.15.119:0/285118817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 msgr2=0x7fe20415f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.608+0000 7fe2037fe700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe20415f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe2037fe700 1 -- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20415d240 msgr2=0x7fe20415dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe2037fe700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20415d240 0x7fe20415dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe2037fe700 1 -- 172.21.15.119:0/285118817 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f8009420 con 0x7fe2040b4e70 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe203fff700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe20415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe20899c700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20415d240 0x7fe20415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe2037fe700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe20416a900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8016000 tx=0x7fe1f8017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe1ebfff700 1 -- 172.21.15.119:0/285118817 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f801f070 con 0x7fe2040b4e70 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe1ebfff700 1 -- 172.21.15.119:0/285118817 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f8000bc0 con 0x7fe2040b4e70 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe1ebfff700 1 -- 172.21.15.119:0/285118817 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8011b30 con 0x7fe2040b4e70 2023-10-08T01:11:05.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20416b6a0 con 0x7fe2040b4e70 2023-10-08T01:11:05.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.609+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20416bc60 con 0x7fe2040b4e70 2023-10-08T01:11:05.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.610+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2040b4650 con 0x7fe2040b4e70 2023-10-08T01:11:05.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.611+0000 7fe1ebfff700 1 -- 172.21.15.119:0/285118817 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fe1f8023070 con 0x7fe2040b4e70 2023-10-08T01:11:05.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.612+0000 7fe1ebfff700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe1f0067050 0x7fe1f0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:05.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.612+0000 7fe1ebfff700 1 -- 172.21.15.119:0/285118817 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fe1f8085820 con 0x7fe2040b4e70 2023-10-08T01:11:05.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.612+0000 7fe203fff700 1 -- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe1f0067050 msgr2=0x7fe1f0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:05.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.612+0000 7fe203fff700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe1f0067050 0x7fe1f0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:05.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.617+0000 7fe1ebfff700 1 -- 172.21.15.119:0/285118817 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1f80525f0 con 0x7fe2040b4e70 2023-10-08T01:11:05.703 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:05.702+0000 7f695a0ab180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:05.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.813+0000 7fe203fff700 1 -- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe1f0067050 msgr2=0x7fe1f0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:05.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.813+0000 7fe203fff700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe1f0067050 0x7fe1f0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:05.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.814+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe20416be50 con 0x7fe2040b4e70 2023-10-08T01:11:05.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.815+0000 7fe1ebfff700 1 -- 172.21.15.119:0/285118817 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe1f8055c10 con 0x7fe2040b4e70 2023-10-08T01:11:05.816 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T01:11:05.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe1f0067050 msgr2=0x7fe1f0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:05.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe1f0067050 0x7fe1f0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 msgr2=0x7fe20416a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe20416a900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8016000 tx=0x7fe1f8017000 comp rx=0 tx=0).stop 2023-10-08T01:11:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 shutdown_connections 2023-10-08T01:11:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fe1f0067050 0x7fe1f0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe20415d240 0x7fe20415dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2040b16d0 0x7fe20415f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 --2- 172.21.15.119:0/285118817 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2040b4e70 0x7fe20416a900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:05.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 >> 172.21.15.119:0/285118817 conn(0x7fe2040d10c0 msgr2=0x7fe2040b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:05.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 shutdown_connections 2023-10-08T01:11:05.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:05.819+0000 7fe20919d700 1 -- 172.21.15.119:0/285118817 wait complete. 2023-10-08T01:11:05.845 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:06.071 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:06.070+0000 7f3f4cc64180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:11:06.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.074+0000 7fd878f9d700 1 Processor -- start 2023-10-08T01:11:06.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.074+0000 7fd878f9d700 1 -- start start 2023-10-08T01:11:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd878f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8741597f0 0x7fd8740b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd878f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd87415d230 0x7fd87415d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd878f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87415dbb0 con 0x7fd874159420 2023-10-08T01:11:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd878f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8740b11d0 con 0x7fd8741597f0 2023-10-08T01:11:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd878f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8740b14b0 con 0x7fd87415d230 2023-10-08T01:11:06.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd8737fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd874159420 0x7fd8740b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54554/0 (socket says 172.21.15.119:54554) 2023-10-08T01:11:06.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd8737fe700 1 -- 172.21.15.119:0/3276185691 learned_addr learned my addr 172.21.15.119:0/3276185691 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:06.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd872ffd700 1 --2- 172.21.15.119:0/3276185691 >> v2:172.21.15.119:3300/0 conn(0x7fd8741597f0 0x7fd8740b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.076+0000 7fd873fff700 1 --2- 172.21.15.119:0/3276185691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd87415d230 0x7fd87415d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd8727fc700 1 -- 172.21.15.119:0/3276185691 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990180715 0 0) 0x7fd87415dbb0 con 0x7fd874159420 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd8727fc700 1 -- 172.21.15.119:0/3276185691 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd85c003570 con 0x7fd874159420 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd872ffd700 1 -- 172.21.15.119:0/3276185691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd87415d230 msgr2=0x7fd87415d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd872ffd700 1 --2- 172.21.15.119:0/3276185691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd87415d230 0x7fd87415d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd872ffd700 1 -- 172.21.15.119:0/3276185691 >> v1:172.21.15.119:6789/0 conn(0x7fd874159420 legacy=0x7fd8740b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd872ffd700 1 -- 172.21.15.119:0/3276185691 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8740b1790 con 0x7fd8741597f0 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd872ffd700 1 --2- 172.21.15.119:0/3276185691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8741597f0 0x7fd8740b2270 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd8680098e0 tx=0x7fd8680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396b608a0b714416 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd8727fc700 1 -- 172.21.15.119:0/3276185691 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868011630 con 0x7fd8741597f0 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd8727fc700 1 -- 172.21.15.119:0/3276185691 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd868011790 con 0x7fd8741597f0 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.077+0000 7fd8727fc700 1 -- 172.21.15.119:0/3276185691 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868011b30 con 0x7fd8741597f0 2023-10-08T01:11:06.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.078+0000 7fd878f9d700 1 -- 172.21.15.119:0/3276185691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8741597f0 msgr2=0x7fd8740b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.078+0000 7fd878f9d700 1 --2- 172.21.15.119:0/3276185691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8741597f0 0x7fd8740b2270 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd8680098e0 tx=0x7fd8680091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.078+0000 7fd878f9d700 1 -- 172.21.15.119:0/3276185691 shutdown_connections 2023-10-08T01:11:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.078+0000 7fd878f9d700 1 --2- 172.21.15.119:0/3276185691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd87415d230 0x7fd87415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.078+0000 7fd878f9d700 1 --2- 172.21.15.119:0/3276185691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8741597f0 0x7fd8740b2270 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.078+0000 7fd878f9d700 1 -- 172.21.15.119:0/3276185691 >> 172.21.15.119:0/3276185691 conn(0x7fd8740d0660 msgr2=0x7fd8740d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.078+0000 7fd878f9d700 1 -- 172.21.15.119:0/3276185691 shutdown_connections 2023-10-08T01:11:06.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.078+0000 7fd878f9d700 1 -- 172.21.15.119:0/3276185691 wait complete. 2023-10-08T01:11:06.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.079+0000 7fd878f9d700 1 Processor -- start 2023-10-08T01:11:06.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.080+0000 7fd878f9d700 1 -- start start 2023-10-08T01:11:06.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.080+0000 7fd878f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874159420 0x7fd874146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.080+0000 7fd878f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8741597f0 0x7fd87415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd878f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87415d230 0x7fd8740c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd878f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd874169e10 con 0x7fd87415d230 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd878f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87415b400 con 0x7fd874159420 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd878f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87415a7f0 con 0x7fd8741597f0 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87415d230 0x7fd8740c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87415d230 0x7fd8740c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39618/0 (socket says 172.21.15.119:39618) 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 -- 172.21.15.119:0/611458904 learned_addr learned my addr 172.21.15.119:0/611458904 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd872ffd700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8741597f0 0x7fd87415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd8737fe700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874159420 0x7fd874146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 -- 172.21.15.119:0/611458904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8741597f0 msgr2=0x7fd87415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8741597f0 0x7fd87415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 -- 172.21.15.119:0/611458904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874159420 msgr2=0x7fd874146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874159420 0x7fd874146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 -- 172.21.15.119:0/611458904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009420 con 0x7fd87415d230 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.081+0000 7fd873fff700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87415d230 0x7fd8740c5400 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009e20 tx=0x7fd86c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.082+0000 7fd85b7fe700 1 -- 172.21.15.119:0/611458904 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00ca40 con 0x7fd87415d230 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.082+0000 7fd85b7fe700 1 -- 172.21.15.119:0/611458904 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86c00f040 con 0x7fd87415d230 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.082+0000 7fd85b7fe700 1 -- 172.21.15.119:0/611458904 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c00ce10 con 0x7fd87415d230 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.082+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87415aad0 con 0x7fd87415d230 2023-10-08T01:11:06.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.082+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87415b090 con 0x7fd87415d230 2023-10-08T01:11:06.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.083+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87404c3c0 con 0x7fd87415d230 2023-10-08T01:11:06.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.085+0000 7fd85b7fe700 1 -- 172.21.15.119:0/611458904 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fd86c014690 con 0x7fd87415d230 2023-10-08T01:11:06.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.086+0000 7fd85b7fe700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd860067050 0x7fd860069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.086+0000 7fd85b7fe700 1 -- 172.21.15.119:0/611458904 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fd86c0852d0 con 0x7fd87415d230 2023-10-08T01:11:06.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.086+0000 7fd8737fe700 1 -- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd860067050 msgr2=0x7fd860069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:06.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.086+0000 7fd8737fe700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd860067050 0x7fd860069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:06.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.090+0000 7fd85b7fe700 1 -- 172.21.15.119:0/611458904 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd86c052030 con 0x7fd87415d230 2023-10-08T01:11:06.166 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:06.165+0000 7f3f4cc64180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:11:06.217 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:06.216+0000 7f695a0ab180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:11:06.256 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:06.255+0000 7f3f4cc64180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:11:06.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.287+0000 7fd8737fe700 1 -- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd860067050 msgr2=0x7fd860069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:06.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.287+0000 7fd8737fe700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd860067050 0x7fd860069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:06.313 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:06.312+0000 7f695a0ab180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:11:06.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.330+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd87406a160 con 0x7fd87415d230 2023-10-08T01:11:06.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.331+0000 7fd85b7fe700 1 -- 172.21.15.119:0/611458904 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fd86c055650 con 0x7fd87415d230 2023-10-08T01:11:06.332 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:06.332 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1019078588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1019078588},{"type":"v1","addr":"172.21.15.119:6835","nonce":1019078588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1353749238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1353749238},{"type":"v1","addr":"172.21.15.184:6834","nonce":1353749238}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/186953052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":186953052},{"type":"v1","addr":"172.21.15.184:6835","nonce":186953052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:09:08.771490+0000","modified":"2023-10-08T01:09:12.322278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/920039174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":920039174},{"type":"v1","addr":"172.21.15.119:6837","nonce":920039174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T01:11:06.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.333+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd860067050 msgr2=0x7fd860069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:06.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.333+0000 7fd878f9d700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd860067050 0x7fd860069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.333+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87415d230 msgr2=0x7fd8740c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.333+0000 7fd878f9d700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87415d230 0x7fd8740c5400 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009e20 tx=0x7fd86c00b040 comp rx=0 tx=0).stop 2023-10-08T01:11:06.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.333+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 shutdown_connections 2023-10-08T01:11:06.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.333+0000 7fd878f9d700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fd860067050 0x7fd860069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.333+0000 7fd878f9d700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd87415d230 0x7fd8740c5400 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.333+0000 7fd878f9d700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd874159420 0x7fd874146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.334+0000 7fd878f9d700 1 --2- 172.21.15.119:0/611458904 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8741597f0 0x7fd87415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.334+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 >> 172.21.15.119:0/611458904 conn(0x7fd8740d0660 msgr2=0x7fd8740b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:06.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.334+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 shutdown_connections 2023-10-08T01:11:06.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.334+0000 7fd878f9d700 1 -- 172.21.15.119:0/611458904 wait complete. 2023-10-08T01:11:06.337 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 24 2023-10-08T01:11:06.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:06.410 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:06.409+0000 7f695a0ab180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:11:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.539+0000 7fadbe79d700 1 Processor -- start 2023-10-08T01:11:06.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.539+0000 7fadbe79d700 1 -- start start 2023-10-08T01:11:06.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.540+0000 7fadbe79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.540+0000 7fadbe79d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fadb81597f0 0x7fadb80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.540+0000 7fadbe79d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb80b11d0 con 0x7fadb815d230 2023-10-08T01:11:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.540+0000 7fadbe79d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb80b14b0 con 0x7fadb81597f0 2023-10-08T01:11:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.540+0000 7fadbe79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80b1790 con 0x7fadb8159420 2023-10-08T01:11:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbdf9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fadb815d230 0x7fadb80b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54568/0 (socket says 172.21.15.119:54568) 2023-10-08T01:11:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbdf9c700 1 -- 172.21.15.119:0/3122775723 learned_addr learned my addr 172.21.15.119:0/3122775723 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbd79b700 1 --2- 172.21.15.119:0/3122775723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbcf9a700 1 --2- 172.21.15.119:0/3122775723 >> v2:172.21.15.184:3300/0 conn(0x7fadb81597f0 0x7fadb80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadaffff700 1 -- 172.21.15.119:0/3122775723 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368015305 0 0) 0x7fadb80b11d0 con 0x7fadb815d230 2023-10-08T01:11:06.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadaffff700 1 -- 172.21.15.119:0/3122775723 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada4003570 con 0x7fadb815d230 2023-10-08T01:11:06.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbd79b700 1 -- 172.21.15.119:0/3122775723 >> v2:172.21.15.184:3300/0 conn(0x7fadb81597f0 msgr2=0x7fadb80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:06.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbd79b700 1 --2- 172.21.15.119:0/3122775723 >> v2:172.21.15.184:3300/0 conn(0x7fadb81597f0 0x7fadb80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbd79b700 1 -- 172.21.15.119:0/3122775723 >> v1:172.21.15.119:6789/0 conn(0x7fadb815d230 legacy=0x7fadb80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbd79b700 1 -- 172.21.15.119:0/3122775723 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80b1a70 con 0x7fadb8159420 2023-10-08T01:11:06.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.541+0000 7fadbcf9a700 1 --2- 172.21.15.119:0/3122775723 >> v2:172.21.15.184:3300/0 conn(0x7fadb81597f0 0x7fadb80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.542+0000 7fadbd79b700 1 --2- 172.21.15.119:0/3122775723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80b5f40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadb40098e0 tx=0x7fadb40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f8cedddfcae49b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.542+0000 7fadaffff700 1 -- 172.21.15.119:0/3122775723 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4011630 con 0x7fadb8159420 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.542+0000 7fadaffff700 1 -- 172.21.15.119:0/3122775723 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb4011790 con 0x7fadb8159420 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.542+0000 7fadaffff700 1 -- 172.21.15.119:0/3122775723 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb4011b30 con 0x7fadb8159420 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.542+0000 7fadbe79d700 1 -- 172.21.15.119:0/3122775723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 msgr2=0x7fadb80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.542+0000 7fadbe79d700 1 --2- 172.21.15.119:0/3122775723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80b5f40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fadb40098e0 tx=0x7fadb40091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.543+0000 7fadbe79d700 1 -- 172.21.15.119:0/3122775723 shutdown_connections 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.543+0000 7fadbe79d700 1 --2- 172.21.15.119:0/3122775723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80b5f40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.543+0000 7fadbe79d700 1 --2- 172.21.15.119:0/3122775723 >> v2:172.21.15.184:3300/0 conn(0x7fadb81597f0 0x7fadb80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.543+0000 7fadbe79d700 1 -- 172.21.15.119:0/3122775723 >> 172.21.15.119:0/3122775723 conn(0x7fadb80d0660 msgr2=0x7fadb80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:06.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.543+0000 7fadbe79d700 1 -- 172.21.15.119:0/3122775723 shutdown_connections 2023-10-08T01:11:06.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.543+0000 7fadbe79d700 1 -- 172.21.15.119:0/3122775723 wait complete. 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.544+0000 7fadbe79d700 1 Processor -- start 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.544+0000 7fadbe79d700 1 -- start start 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbe79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbe79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb81597f0 0x7fadb814bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbe79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb815d230 0x7fadb80cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbe79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb80c7710 con 0x7fadb81597f0 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbe79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb816d070 con 0x7fadb815d230 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbe79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb816d350 con 0x7fadb8159420 2023-10-08T01:11:06.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbcf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb81597f0 0x7fadb814bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbcf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb81597f0 0x7fadb814bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39628/0 (socket says 172.21.15.119:39628) 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbcf9a700 1 -- 172.21.15.119:0/114733323 learned_addr learned my addr 172.21.15.119:0/114733323 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbdf9c700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb815d230 0x7fadb80cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.545+0000 7fadbd79b700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadbcf9a700 1 -- 172.21.15.119:0/114733323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 msgr2=0x7fadb80c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadbcf9a700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadbcf9a700 1 -- 172.21.15.119:0/114733323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb815d230 msgr2=0x7fadb80cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadbcf9a700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb815d230 0x7fadb80cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadbcf9a700 1 -- 172.21.15.119:0/114733323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada8009420 con 0x7fadb81597f0 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadbcf9a700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb81597f0 0x7fadb814bfd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fada800ec80 tx=0x7fada80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:06.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadad7fa700 1 -- 172.21.15.119:0/114733323 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800c920 con 0x7fadb81597f0 2023-10-08T01:11:06.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadad7fa700 1 -- 172.21.15.119:0/114733323 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada800ca80 con 0x7fadb81597f0 2023-10-08T01:11:06.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb4009420 con 0x7fadb81597f0 2023-10-08T01:11:06.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadad7fa700 1 -- 172.21.15.119:0/114733323 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada800ce00 con 0x7fadb81597f0 2023-10-08T01:11:06.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.546+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb816d910 con 0x7fadb81597f0 2023-10-08T01:11:06.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.548+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb804c3c0 con 0x7fadb81597f0 2023-10-08T01:11:06.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.548+0000 7fadad7fa700 1 -- 172.21.15.119:0/114733323 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fada8013680 con 0x7fadb81597f0 2023-10-08T01:11:06.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.550+0000 7fadad7fa700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fad9c067050 0x7fad9c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:06.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.550+0000 7fadad7fa700 1 -- 172.21.15.119:0/114733323 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fada8084d80 con 0x7fadb81597f0 2023-10-08T01:11:06.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.550+0000 7fadbd79b700 1 -- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fad9c067050 msgr2=0x7fad9c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:06.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.550+0000 7fadbd79b700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fad9c067050 0x7fad9c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:06.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.554+0000 7fadad7fa700 1 -- 172.21.15.119:0/114733323 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fada8051bf0 con 0x7fadb81597f0 2023-10-08T01:11:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.751+0000 7fadbd79b700 1 -- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fad9c067050 msgr2=0x7fad9c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.751+0000 7fadbd79b700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fad9c067050 0x7fad9c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:06.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.753+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadb80b1d50 con 0x7fadb81597f0 2023-10-08T01:11:06.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.754+0000 7fadad7fa700 1 -- 172.21.15.119:0/114733323 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16548 (secure 0 0 0) 0x7fada8055210 con 0x7fadb81597f0 2023-10-08T01:11:06.756 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:06.756 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":64,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:04.273139+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:09:06.334038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T01:09:07.759893+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.3c","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/593291478":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/4075293652":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/3952125277":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/91830571":"2023-10-09T01:09:24.445449+0000","172.21.15.119:6836/85511":"2023-10-09T01:09:24.445449+0000","172.21.15.119:6834/85511":"2023-10-09T01:09:24.445449+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fad9c067050 msgr2=0x7fad9c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fad9c067050 0x7fad9c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb81597f0 msgr2=0x7fadb814bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb81597f0 0x7fadb814bfd0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fada800ec80 tx=0x7fada80091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 shutdown_connections 2023-10-08T01:11:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fad9c067050 0x7fad9c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fadb8159420 0x7fadb80c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fadb815d230 0x7fadb80cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 --2- 172.21.15.119:0/114733323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadb81597f0 0x7fadb814bfd0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.758+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 >> 172.21.15.119:0/114733323 conn(0x7fadb80d0660 msgr2=0x7fadb80b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.759+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 shutdown_connections 2023-10-08T01:11:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.759+0000 7fadbe79d700 1 -- 172.21.15.119:0/114733323 wait complete. 2023-10-08T01:11:06.785 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T01:11:06.785 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:06.879 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:06.878+0000 7f3f4cc64180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:11:06.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.994+0000 7f261359e700 1 Processor -- start 2023-10-08T01:11:07.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.994+0000 7f261359e700 1 -- start start 2023-10-08T01:11:07.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.996+0000 7f261359e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f261415daa0 0x7f261406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.996+0000 7f261359e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261406e8c0 con 0x7f26140696c0 2023-10-08T01:11:07.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.996+0000 7f261359e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26140ca740 con 0x7f261415de70 2023-10-08T01:11:07.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.996+0000 7f261359e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261406e5e0 con 0x7f261415daa0 2023-10-08T01:11:07.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.997+0000 7f261259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f261415daa0 0x7f261406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.997+0000 7f2612d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f26140696c0 0x7f261406ecd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54576/0 (socket says 172.21.15.119:54576) 2023-10-08T01:11:07.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.997+0000 7f2612d9d700 1 -- 172.21.15.119:0/2462849200 learned_addr learned my addr 172.21.15.119:0/2462849200 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:07.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.997+0000 7f261259c700 1 -- 172.21.15.119:0/2462849200 >> v1:172.21.15.184:6789/0 conn(0x7f261415de70 legacy=0x7f26140c7110 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-08T01:11:07.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.998+0000 7f261259c700 1 -- 172.21.15.119:0/2462849200 >> v1:172.21.15.119:6789/0 conn(0x7f26140696c0 legacy=0x7f261406ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.998+0000 7f261259c700 1 -- 172.21.15.119:0/2462849200 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140ca0f0 con 0x7f261415daa0 2023-10-08T01:11:07.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.998+0000 7f261259c700 1 --2- 172.21.15.119:0/2462849200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f261415daa0 0x7f261406e000 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f260c0098e0 tx=0x7f260c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf82d86490bbc05 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:07.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.998+0000 7f261159a700 1 -- 172.21.15.119:0/2462849200 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c009e80 con 0x7f261415daa0 2023-10-08T01:11:07.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.998+0000 7f261159a700 1 -- 172.21.15.119:0/2462849200 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260c00b820 con 0x7f261415daa0 2023-10-08T01:11:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.998+0000 7f261159a700 1 -- 172.21.15.119:0/2462849200 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260c00bba0 con 0x7f261415daa0 2023-10-08T01:11:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.999+0000 7f261359e700 1 -- 172.21.15.119:0/2462849200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f261415daa0 msgr2=0x7f261406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.999+0000 7f261359e700 1 --2- 172.21.15.119:0/2462849200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f261415daa0 0x7f261406e000 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f260c0098e0 tx=0x7f260c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.999+0000 7f261359e700 1 -- 172.21.15.119:0/2462849200 shutdown_connections 2023-10-08T01:11:07.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.999+0000 7f261359e700 1 --2- 172.21.15.119:0/2462849200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f261415daa0 0x7f261406e000 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:06.999+0000 7f261359e700 1 -- 172.21.15.119:0/2462849200 >> 172.21.15.119:0/2462849200 conn(0x7f26140cc1f0 msgr2=0x7f26140cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:07.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.000+0000 7f261359e700 1 -- 172.21.15.119:0/2462849200 shutdown_connections 2023-10-08T01:11:07.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.000+0000 7f261359e700 1 -- 172.21.15.119:0/2462849200 wait complete. 2023-10-08T01:11:07.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.002+0000 7f261359e700 1 Processor -- start 2023-10-08T01:11:07.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.002+0000 7f261359e700 1 -- start start 2023-10-08T01:11:07.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.003+0000 7f261359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140696c0 0x7f26140c5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.003+0000 7f261359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f261415de70 0x7f2614146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.003+0000 7f261359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26141465b0 0x7f26140b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.003+0000 7f261359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2614146a30 con 0x7f26141465b0 2023-10-08T01:11:07.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f261359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26140c65d0 con 0x7f261415de70 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f261359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26140c4480 con 0x7f26140696c0 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f2611d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f261415de70 0x7f2614146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f261259c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140696c0 0x7f26140c5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f2611d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f261415de70 0x7f2614146070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:51110/0 (socket says 172.21.15.119:51110) 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f2612d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26141465b0 0x7f26140b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f2611d9b700 1 -- 172.21.15.119:0/1610930349 learned_addr learned my addr 172.21.15.119:0/1610930349 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f2611d9b700 1 -- 172.21.15.119:0/1610930349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140696c0 msgr2=0x7f26140c5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f2611d9b700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140696c0 0x7f26140c5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.004+0000 7f2611d9b700 1 -- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26141465b0 msgr2=0x7f26140b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.005+0000 7f2611d9b700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26141465b0 0x7f26140b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.005+0000 7f2611d9b700 1 -- 172.21.15.119:0/1610930349 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260c009420 con 0x7f261415de70 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.005+0000 7f2612d9d700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26141465b0 0x7f26140b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.005+0000 7f2611d9b700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f261415de70 0x7f2614146070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00bbe0 tx=0x7f25fc009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.005+0000 7f260a7fc700 1 -- 172.21.15.119:0/1610930349 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc0092e0 con 0x7f261415de70 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.005+0000 7f260a7fc700 1 -- 172.21.15.119:0/1610930349 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25fc00c630 con 0x7f261415de70 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.006+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26140c4760 con 0x7f261415de70 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.006+0000 7f260a7fc700 1 -- 172.21.15.119:0/1610930349 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25fc00c9d0 con 0x7f261415de70 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.006+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26140c4d20 con 0x7f261415de70 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.007+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26140c4f40 con 0x7f261415de70 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.008+0000 7f260a7fc700 1 -- 172.21.15.119:0/1610930349 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f25fc012070 con 0x7f261415de70 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.009+0000 7f260a7fc700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f25f4067050 0x7f25f4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.009+0000 7f261259c700 1 -- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f25f4067050 msgr2=0x7f25f4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:07.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.009+0000 7f260a7fc700 1 -- 172.21.15.119:0/1610930349 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f25fc084680 con 0x7f261415de70 2023-10-08T01:11:07.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.009+0000 7f261259c700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f25f4067050 0x7f25f4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:07.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.018+0000 7f260a7fc700 1 -- 172.21.15.119:0/1610930349 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25fc051500 con 0x7f261415de70 2023-10-08T01:11:07.056 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:07.055+0000 7f3f4cc64180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:07.069 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:07.068+0000 7f695a0ab180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:11:07.152 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:07.151+0000 7f3f4cc64180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:11:07.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.210+0000 7f261259c700 1 -- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f25f4067050 msgr2=0x7f25f4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:07.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.210+0000 7f261259c700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f25f4067050 0x7f25f4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:07.243 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:07.242+0000 7f695a0ab180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.259+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f261404c3c0 con 0x7f261415de70 2023-10-08T01:11:07.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.260+0000 7f260a7fc700 1 -- 172.21.15.119:0/1610930349 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f25fc054b20 con 0x7f261415de70 2023-10-08T01:11:07.261 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:07.261 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1019078588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1019078588},{"type":"v1","addr":"172.21.15.119:6835","nonce":1019078588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1353749238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1353749238},{"type":"v1","addr":"172.21.15.184:6834","nonce":1353749238}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/186953052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":186953052},{"type":"v1","addr":"172.21.15.184:6835","nonce":186953052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:09:08.771490+0000","modified":"2023-10-08T01:09:12.322278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/920039174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":920039174},{"type":"v1","addr":"172.21.15.119:6837","nonce":920039174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T01:11:07.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f25f4067050 msgr2=0x7f25f4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:07.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f25f4067050 0x7f25f4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f261415de70 msgr2=0x7f2614146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f261415de70 0x7f2614146070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00bbe0 tx=0x7f25fc009840 comp rx=0 tx=0).stop 2023-10-08T01:11:07.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 shutdown_connections 2023-10-08T01:11:07.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f25f4067050 0x7f25f4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f26140696c0 0x7f26140c5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26141465b0 0x7f26140b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 --2- 172.21.15.119:0/1610930349 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f261415de70 0x7f2614146070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 >> 172.21.15.119:0/1610930349 conn(0x7f26140cc1f0 msgr2=0x7f2614149600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:07.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 shutdown_connections 2023-10-08T01:11:07.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.262+0000 7f261359e700 1 -- 172.21.15.119:0/1610930349 wait complete. 2023-10-08T01:11:07.267 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 24 2023-10-08T01:11:07.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:07.345 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:07.343+0000 7f695a0ab180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:11:07.450 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:07.448+0000 7f3f4cc64180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:11:07.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.469+0000 7fdb0cbff700 1 Processor -- start 2023-10-08T01:11:07.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.469+0000 7fdb0cbff700 1 -- start start 2023-10-08T01:11:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.470+0000 7fdb0cbff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb08159c60 0x7fdb0815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.470+0000 7fdb0cbff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdb0815db10 0x7fdb08069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.470+0000 7fdb0cbff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb0815a160 con 0x7fdb08159890 2023-10-08T01:11:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.470+0000 7fdb0cbff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0815df50 con 0x7fdb08159c60 2023-10-08T01:11:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.470+0000 7fdb0cbff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080d13a0 con 0x7fdb0815db10 2023-10-08T01:11:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb05d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb08159c60 0x7fdb0815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb0659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdb08159890 0x7fdb0815abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54592/0 (socket says 172.21.15.119:54592) 2023-10-08T01:11:07.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb0659c700 1 -- 172.21.15.119:0/732085683 learned_addr learned my addr 172.21.15.119:0/732085683 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:07.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb06d9d700 1 --2- 172.21.15.119:0/732085683 >> v2:172.21.15.184:3300/0 conn(0x7fdb0815db10 0x7fdb08069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb05d9b700 1 -- 172.21.15.119:0/732085683 >> v2:172.21.15.184:3300/0 conn(0x7fdb0815db10 msgr2=0x7fdb08069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb05d9b700 1 --2- 172.21.15.119:0/732085683 >> v2:172.21.15.184:3300/0 conn(0x7fdb0815db10 0x7fdb08069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb05d9b700 1 -- 172.21.15.119:0/732085683 >> v1:172.21.15.119:6789/0 conn(0x7fdb08159890 legacy=0x7fdb0815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb05d9b700 1 -- 172.21.15.119:0/732085683 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0815e140 con 0x7fdb08159c60 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.471+0000 7fdb05d9b700 1 --2- 172.21.15.119:0/732085683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb08159c60 0x7fdb0815d5d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdb0806a890 tx=0x7fdafc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9c75de218bff4d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.472+0000 7fdb0559a700 1 -- 172.21.15.119:0/732085683 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc011630 con 0x7fdb08159c60 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.472+0000 7fdb0559a700 1 -- 172.21.15.119:0/732085683 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc011790 con 0x7fdb08159c60 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.472+0000 7fdb0559a700 1 -- 172.21.15.119:0/732085683 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc011b30 con 0x7fdb08159c60 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.472+0000 7fdb0cbff700 1 -- 172.21.15.119:0/732085683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb08159c60 msgr2=0x7fdb0815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.473+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/732085683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb08159c60 0x7fdb0815d5d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fdb0806a890 tx=0x7fdafc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.473+0000 7fdb0cbff700 1 -- 172.21.15.119:0/732085683 shutdown_connections 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.473+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/732085683 >> v2:172.21.15.184:3300/0 conn(0x7fdb0815db10 0x7fdb08069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.473+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/732085683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb08159c60 0x7fdb0815d5d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.473+0000 7fdb0cbff700 1 -- 172.21.15.119:0/732085683 >> 172.21.15.119:0/732085683 conn(0x7fdb08068e20 msgr2=0x7fdb080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:07.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.473+0000 7fdb0cbff700 1 -- 172.21.15.119:0/732085683 shutdown_connections 2023-10-08T01:11:07.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.473+0000 7fdb0cbff700 1 -- 172.21.15.119:0/732085683 wait complete. 2023-10-08T01:11:07.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.474+0000 7fdb0cbff700 1 Processor -- start 2023-10-08T01:11:07.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.474+0000 7fdb0cbff700 1 -- start start 2023-10-08T01:11:07.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.475+0000 7fdb0cbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb08159890 0x7fdb0814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.475+0000 7fdb0cbff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08159c60 0x7fdb08154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.475+0000 7fdb0cbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0815db10 0x7fdb08155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.475+0000 7fdb0cbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080ce980 con 0x7fdb0815db10 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.475+0000 7fdb0cbff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080cd0c0 con 0x7fdb08159c60 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.475+0000 7fdb0cbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080cdc00 con 0x7fdb08159890 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb06d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0815db10 0x7fdb08155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08159c60 0x7fdb08154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb0659c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb08159890 0x7fdb0814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08159c60 0x7fdb08154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:51132/0 (socket says 172.21.15.119:51132) 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 -- 172.21.15.119:0/1716078783 learned_addr learned my addr 172.21.15.119:0/1716078783 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 -- 172.21.15.119:0/1716078783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb08159890 msgr2=0x7fdb0814c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb08159890 0x7fdb0814c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 -- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0815db10 msgr2=0x7fdb08155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0815db10 0x7fdb08155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 -- 172.21.15.119:0/1716078783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009420 con 0x7fdb08159c60 2023-10-08T01:11:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb06d9d700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0815db10 0x7fdb08155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.476+0000 7fdb05d9b700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08159c60 0x7fdb08154ff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0030c0 tx=0x7fdafc016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.477+0000 7fdaf67fc700 1 -- 172.21.15.119:0/1716078783 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc01cc20 con 0x7fdb08159c60 2023-10-08T01:11:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.477+0000 7fdaf67fc700 1 -- 172.21.15.119:0/1716078783 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc01cd80 con 0x7fdb08159c60 2023-10-08T01:11:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.477+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb080cd2d0 con 0x7fdb08159c60 2023-10-08T01:11:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.477+0000 7fdaf67fc700 1 -- 172.21.15.119:0/1716078783 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc01a6a0 con 0x7fdb08159c60 2023-10-08T01:11:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.477+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb080cd890 con 0x7fdb08159c60 2023-10-08T01:11:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.478+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb080b08a0 con 0x7fdb08159c60 2023-10-08T01:11:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.479+0000 7fdaf67fc700 1 -- 172.21.15.119:0/1716078783 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fdafc01aad0 con 0x7fdb08159c60 2023-10-08T01:11:07.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.480+0000 7fdaf67fc700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdae8067050 0x7fdae8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.480+0000 7fdaf67fc700 1 -- 172.21.15.119:0/1716078783 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fdafc086920 con 0x7fdb08159c60 2023-10-08T01:11:07.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.480+0000 7fdb0659c700 1 -- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdae8067050 msgr2=0x7fdae8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:07.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.480+0000 7fdb0659c700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdae8067050 0x7fdae8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.488+0000 7fdaf67fc700 1 -- 172.21.15.119:0/1716078783 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdafc0536f0 con 0x7fdb08159c60 2023-10-08T01:11:07.551 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:07.549+0000 7f3f4cc64180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:11:07.648 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:07.647+0000 7f695a0ab180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:11:07.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.681+0000 7fdb0659c700 1 -- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdae8067050 msgr2=0x7fdae8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:07.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.681+0000 7fdb0659c700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdae8067050 0x7fdae8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:07.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.735+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdb080b08a0 con 0x7fdb08159c60 2023-10-08T01:11:07.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.736+0000 7fdaf67fc700 1 -- 172.21.15.119:0/1716078783 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fdafc056d10 con 0x7fdb08159c60 2023-10-08T01:11:07.737 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:07.738 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1019078588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1019078588},{"type":"v1","addr":"172.21.15.119:6835","nonce":1019078588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1353749238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1353749238},{"type":"v1","addr":"172.21.15.184:6834","nonce":1353749238}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/186953052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":186953052},{"type":"v1","addr":"172.21.15.184:6835","nonce":186953052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:09:08.771490+0000","modified":"2023-10-08T01:09:12.322278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5757},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5757":{"gid":5757,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/920039174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":920039174},{"type":"v1","addr":"172.21.15.119:6837","nonce":920039174}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-08T01:11:07.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdae8067050 msgr2=0x7fdae8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:07.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdae8067050 0x7fdae8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08159c60 msgr2=0x7fdb08154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08159c60 0x7fdb08154ff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0030c0 tx=0x7fdafc016000 comp rx=0 tx=0).stop 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 shutdown_connections 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fdae8067050 0x7fdae8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb08159890 0x7fdb0814c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb0815db10 0x7fdb08155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 --2- 172.21.15.119:0/1716078783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb08159c60 0x7fdb08154ff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 >> 172.21.15.119:0/1716078783 conn(0x7fdb08068e20 msgr2=0x7fdb08157c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 shutdown_connections 2023-10-08T01:11:07.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.739+0000 7fdb0cbff700 1 -- 172.21.15.119:0/1716078783 wait complete. 2023-10-08T01:11:07.742 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 24 2023-10-08T01:11:07.747 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:07.747+0000 7f695a0ab180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:11:07.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:07.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.943+0000 7f69be99d700 1 Processor -- start 2023-10-08T01:11:07.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.944+0000 7f69be99d700 1 -- start start 2023-10-08T01:11:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69be99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f69b8159420 0x7f69b80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69be99d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69b81597f0 0x7f69b80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69be99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 0x7f69b815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69be99d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b815dbb0 con 0x7f69b81597f0 2023-10-08T01:11:07.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69be99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b80b11d0 con 0x7f69b8159420 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69be99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b80b14b0 con 0x7f69b815d230 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69bd19a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69b81597f0 0x7f69b80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69bd19a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69b81597f0 0x7f69b80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39660/0 (socket says 172.21.15.119:39660) 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69bd19a700 1 -- 172.21.15.119:0/1018116932 learned_addr learned my addr 172.21.15.119:0/1018116932 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69bd99b700 1 --2- 172.21.15.119:0/1018116932 >> v2:172.21.15.184:3300/0 conn(0x7f69b8159420 0x7f69b80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.945+0000 7f69be19c700 1 --2- 172.21.15.119:0/1018116932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 0x7f69b815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bd19a700 1 -- 172.21.15.119:0/1018116932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 msgr2=0x7f69b815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bd19a700 1 --2- 172.21.15.119:0/1018116932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 0x7f69b815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bd19a700 1 -- 172.21.15.119:0/1018116932 >> v2:172.21.15.184:3300/0 conn(0x7f69b8159420 msgr2=0x7f69b80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:07.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bd19a700 1 --2- 172.21.15.119:0/1018116932 >> v2:172.21.15.184:3300/0 conn(0x7f69b8159420 0x7f69b80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bd19a700 1 -- 172.21.15.119:0/1018116932 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b80b1790 con 0x7f69b81597f0 2023-10-08T01:11:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bd99b700 1 --2- 172.21.15.119:0/1018116932 >> v2:172.21.15.184:3300/0 conn(0x7f69b8159420 0x7f69b80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bd19a700 1 --2- 172.21.15.119:0/1018116932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b81597f0 0x7f69b80b2270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f69b80b2f20 tx=0x7f69a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc5012ebe913d14 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bc999700 1 -- 172.21.15.119:0/1018116932 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a800b640 con 0x7f69b81597f0 2023-10-08T01:11:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bc999700 1 -- 172.21.15.119:0/1018116932 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a800b7a0 con 0x7f69b81597f0 2023-10-08T01:11:07.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.946+0000 7f69bc999700 1 -- 172.21.15.119:0/1018116932 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a800bb60 con 0x7f69b81597f0 2023-10-08T01:11:07.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 -- 172.21.15.119:0/1018116932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b81597f0 msgr2=0x7f69b80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 --2- 172.21.15.119:0/1018116932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b81597f0 0x7f69b80b2270 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f69b80b2f20 tx=0x7f69a8009250 comp rx=0 tx=0).stop 2023-10-08T01:11:07.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 -- 172.21.15.119:0/1018116932 shutdown_connections 2023-10-08T01:11:07.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 --2- 172.21.15.119:0/1018116932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 0x7f69b815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 --2- 172.21.15.119:0/1018116932 >> v2:172.21.15.184:3300/0 conn(0x7f69b8159420 0x7f69b80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 --2- 172.21.15.119:0/1018116932 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b81597f0 0x7f69b80b2270 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 -- 172.21.15.119:0/1018116932 >> 172.21.15.119:0/1018116932 conn(0x7f69b80d0660 msgr2=0x7f69b80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 -- 172.21.15.119:0/1018116932 shutdown_connections 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.947+0000 7f69be99d700 1 -- 172.21.15.119:0/1018116932 wait complete. 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.948+0000 7f69be99d700 1 Processor -- start 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.948+0000 7f69be99d700 1 -- start start 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.949+0000 7f69be99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b8159420 0x7f69b80cbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.949+0000 7f69be99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b81597f0 0x7f69b80b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.949+0000 7f69be99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 0x7f69b80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.949+0000 7f69be99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b815a7e0 con 0x7f69b8159420 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.949+0000 7f69be99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b80cc150 con 0x7f69b81597f0 2023-10-08T01:11:07.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.949+0000 7f69be99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b815a1d0 con 0x7f69b815d230 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b8159420 0x7f69b80cbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b8159420 0x7f69b80cbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39666/0 (socket says 172.21.15.119:39666) 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd19a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b81597f0 0x7f69b80b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd99b700 1 -- 172.21.15.119:0/1986139107 learned_addr learned my addr 172.21.15.119:0/1986139107 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69be19c700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 0x7f69b80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd19a700 1 -- 172.21.15.119:0/1986139107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 msgr2=0x7f69b80b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd19a700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 0x7f69b80b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd19a700 1 -- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b8159420 msgr2=0x7f69b80cbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd19a700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b8159420 0x7f69b80cbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd19a700 1 -- 172.21.15.119:0/1986139107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b4009420 con 0x7f69b81597f0 2023-10-08T01:11:07.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.950+0000 7f69bd19a700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b81597f0 0x7f69b80b4830 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f69b8089750 tx=0x7f69a8017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.951+0000 7f69adffb700 1 -- 172.21.15.119:0/1986139107 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a800b910 con 0x7f69b81597f0 2023-10-08T01:11:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.951+0000 7f69adffb700 1 -- 172.21.15.119:0/1986139107 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a800ba70 con 0x7f69b81597f0 2023-10-08T01:11:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.951+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69a80094f0 con 0x7f69b81597f0 2023-10-08T01:11:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.951+0000 7f69adffb700 1 -- 172.21.15.119:0/1986139107 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8011650 con 0x7f69b81597f0 2023-10-08T01:11:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.951+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b80b53d0 con 0x7f69b81597f0 2023-10-08T01:11:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.952+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b815a970 con 0x7f69b81597f0 2023-10-08T01:11:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.953+0000 7f69adffb700 1 -- 172.21.15.119:0/1986139107 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f69a8011a00 con 0x7f69b81597f0 2023-10-08T01:11:07.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.954+0000 7f69adffb700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f699c067050 0x7f699c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:07.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.954+0000 7f69adffb700 1 -- 172.21.15.119:0/1986139107 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f69a8086130 con 0x7f69b81597f0 2023-10-08T01:11:07.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.954+0000 7f69bd99b700 1 -- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f699c067050 msgr2=0x7f699c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:07.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.954+0000 7f69bd99b700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f699c067050 0x7f699c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:07.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:07.962+0000 7f69adffb700 1 -- 172.21.15.119:0/1986139107 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69a8052f00 con 0x7f69b81597f0 2023-10-08T01:11:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.155+0000 7f69bd99b700 1 -- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f699c067050 msgr2=0x7f699c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.155+0000 7f69bd99b700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f699c067050 0x7f699c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.162+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69b804c3c0 con 0x7f69b81597f0 2023-10-08T01:11:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.169+0000 7f69adffb700 1 -- 172.21.15.119:0/1986139107 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16548 (secure 0 0 0) 0x7f69a8011cb0 con 0x7f69b81597f0 2023-10-08T01:11:08.170 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:08.170 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":64,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:04.273139+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:09:06.334038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-10-08T01:09:07.759893+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.9","mappings":[{"from":0,"to":4}]},{"pgid":"7.3c","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/593291478":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/4075293652":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/3952125277":"2023-10-09T01:09:24.445449+0000","172.21.15.119:0/91830571":"2023-10-09T01:09:24.445449+0000","172.21.15.119:6836/85511":"2023-10-09T01:09:24.445449+0000","172.21.15.119:6834/85511":"2023-10-09T01:09:24.445449+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:08.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f699c067050 msgr2=0x7f699c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:08.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f699c067050 0x7f699c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b81597f0 msgr2=0x7f69b80b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b81597f0 0x7f69b80b4830 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f69b8089750 tx=0x7f69a8017000 comp rx=0 tx=0).stop 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 shutdown_connections 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f699c067050 0x7f699c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69b815d230 0x7f69b80b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69b8159420 0x7f69b80cbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 --2- 172.21.15.119:0/1986139107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69b81597f0 0x7f69b80b4830 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 >> 172.21.15.119:0/1986139107 conn(0x7f69b80d0660 msgr2=0x7f69b80ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 shutdown_connections 2023-10-08T01:11:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.171+0000 7f69be99d700 1 -- 172.21.15.119:0/1986139107 wait complete. 2023-10-08T01:11:08.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T01:11:08.256 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:08.254+0000 7f3f4cc64180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:11:08.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.378+0000 7fbd6ab9d700 1 Processor -- start 2023-10-08T01:11:08.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.379+0000 7fbd6ab9d700 1 -- start start 2023-10-08T01:11:08.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.380+0000 7fbd6ab9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd64156310 0x7fbd640c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.380+0000 7fbd6ab9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640ca860 con 0x7fbd641566e0 2023-10-08T01:11:08.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.380+0000 7fbd6ab9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640cab40 con 0x7fbd640ae710 2023-10-08T01:11:08.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.380+0000 7fbd6ab9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640cae20 con 0x7fbd64156310 2023-10-08T01:11:08.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.380+0000 7fbd6939a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbd641566e0 0x7fbd640ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54594/0 (socket says 172.21.15.119:54594) 2023-10-08T01:11:08.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.380+0000 7fbd6939a700 1 -- 172.21.15.119:0/3233990302 learned_addr learned my addr 172.21.15.119:0/3233990302 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:08.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.380+0000 7fbd69b9b700 1 --2- 172.21.15.119:0/3233990302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd64156310 0x7fbd640c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.380+0000 7fbd68b99700 1 -- 172.21.15.119:0/3233990302 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468208826 0 0) 0x7fbd640ca860 con 0x7fbd641566e0 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd68b99700 1 -- 172.21.15.119:0/3233990302 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd50003570 con 0x7fbd641566e0 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd68b99700 1 -- 172.21.15.119:0/3233990302 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 840589795 0 0) 0x7fbd640cab40 con 0x7fbd640ae710 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd68b99700 1 -- 172.21.15.119:0/3233990302 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd640ca860 con 0x7fbd640ae710 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd69b9b700 1 -- 172.21.15.119:0/3233990302 >> v1:172.21.15.184:6789/0 conn(0x7fbd640ae710 legacy=0x7fbd640c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd69b9b700 1 -- 172.21.15.119:0/3233990302 >> v1:172.21.15.119:6789/0 conn(0x7fbd641566e0 legacy=0x7fbd640ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd69b9b700 1 -- 172.21.15.119:0/3233990302 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640cb100 con 0x7fbd64156310 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd69b9b700 1 --2- 172.21.15.119:0/3233990302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd64156310 0x7fbd640c9dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009910 tx=0x7fbd600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca22751210132ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd68b99700 1 -- 172.21.15.119:0/3233990302 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60011630 con 0x7fbd64156310 2023-10-08T01:11:08.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd68b99700 1 -- 172.21.15.119:0/3233990302 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60011790 con 0x7fbd64156310 2023-10-08T01:11:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.381+0000 7fbd68b99700 1 -- 172.21.15.119:0/3233990302 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60011b30 con 0x7fbd64156310 2023-10-08T01:11:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.382+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3233990302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd64156310 msgr2=0x7fbd640c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.382+0000 7fbd6ab9d700 1 --2- 172.21.15.119:0/3233990302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd64156310 0x7fbd640c9dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009910 tx=0x7fbd600091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.382+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3233990302 shutdown_connections 2023-10-08T01:11:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.382+0000 7fbd6ab9d700 1 --2- 172.21.15.119:0/3233990302 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd64156310 0x7fbd640c9dc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.382+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3233990302 >> 172.21.15.119:0/3233990302 conn(0x7fbd640cc1f0 msgr2=0x7fbd640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.382+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3233990302 shutdown_connections 2023-10-08T01:11:08.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.382+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3233990302 wait complete. 2023-10-08T01:11:08.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.384+0000 7fbd6ab9d700 1 Processor -- start 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.384+0000 7fbd6ab9d700 1 -- start start 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6ab9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd640ae710 0x7fbd64159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6ab9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd64156310 0x7fbd64146dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6ab9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd641566e0 0x7fbd640c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6ab9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640c3cc0 con 0x7fbd64156310 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6ab9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64159df0 con 0x7fbd640ae710 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6ab9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640c33d0 con 0x7fbd641566e0 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd64156310 0x7fbd64146dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd64156310 0x7fbd64146dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39678/0 (socket says 172.21.15.119:39678) 2023-10-08T01:11:08.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6939a700 1 -- 172.21.15.119:0/3306799056 learned_addr learned my addr 172.21.15.119:0/3306799056 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6a39c700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd641566e0 0x7fbd640c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd69b9b700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd640ae710 0x7fbd64159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6939a700 1 -- 172.21.15.119:0/3306799056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd641566e0 msgr2=0x7fbd640c4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6939a700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd641566e0 0x7fbd640c4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6939a700 1 -- 172.21.15.119:0/3306799056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd640ae710 msgr2=0x7fbd64159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6939a700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd640ae710 0x7fbd64159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.385+0000 7fbd6939a700 1 -- 172.21.15.119:0/3306799056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd60009420 con 0x7fbd64156310 2023-10-08T01:11:08.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.386+0000 7fbd6939a700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd64156310 0x7fbd64146dc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400c580 tx=0x7fbd54007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.386+0000 7fbd59ffb700 1 -- 172.21.15.119:0/3306799056 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54005b30 con 0x7fbd64156310 2023-10-08T01:11:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.386+0000 7fbd59ffb700 1 -- 172.21.15.119:0/3306799056 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54005cb0 con 0x7fbd64156310 2023-10-08T01:11:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.386+0000 7fbd59ffb700 1 -- 172.21.15.119:0/3306799056 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54007320 con 0x7fbd64156310 2023-10-08T01:11:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.386+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd640c3670 con 0x7fbd64156310 2023-10-08T01:11:08.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.386+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd641511f0 con 0x7fbd64156310 2023-10-08T01:11:08.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.387+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd640c3e70 con 0x7fbd64156310 2023-10-08T01:11:08.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.388+0000 7fbd59ffb700 1 -- 172.21.15.119:0/3306799056 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fbd5401b3b0 con 0x7fbd64156310 2023-10-08T01:11:08.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.389+0000 7fbd59ffb700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fbd48067050 0x7fbd48069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.389+0000 7fbd69b9b700 1 -- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fbd48067050 msgr2=0x7fbd48069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:08.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.389+0000 7fbd59ffb700 1 -- 172.21.15.119:0/3306799056 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fbd54083d80 con 0x7fbd64156310 2023-10-08T01:11:08.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.389+0000 7fbd69b9b700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fbd48067050 0x7fbd48069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:08.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.394+0000 7fbd59ffb700 1 -- 172.21.15.119:0/3306799056 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd640c3e70 con 0x7fbd64156310 2023-10-08T01:11:08.488 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:08.487+0000 7f695a0ab180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:11:08.548 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:08.547+0000 7f3f4cc64180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:11:08.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.590+0000 7fbd69b9b700 1 -- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fbd48067050 msgr2=0x7fbd48069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6834/85511 2023-10-08T01:11:08.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.590+0000 7fbd69b9b700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fbd48067050 0x7fbd48069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:08.640 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:08.639+0000 7f3f4cc64180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:11:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.642+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbd6404c3c0 con 0x7fbd64156310 2023-10-08T01:11:08.643 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T01:11:08.643+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T01:11:08.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.647+0000 7fbd59ffb700 1 -- 172.21.15.119:0/3306799056 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbd54017020 con 0x7fbd64156310 2023-10-08T01:11:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fbd48067050 msgr2=0x7fbd48069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fbd48067050 0x7fbd48069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd64156310 msgr2=0x7fbd64146dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd64156310 0x7fbd64146dc0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400c580 tx=0x7fbd54007e20 comp rx=0 tx=0).stop 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 shutdown_connections 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7fbd48067050 0x7fbd48069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd641566e0 0x7fbd640c4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd640ae710 0x7fbd64159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 --2- 172.21.15.119:0/3306799056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd64156310 0x7fbd64146dc0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 >> 172.21.15.119:0/3306799056 conn(0x7fbd640cc1f0 msgr2=0x7fbd64151f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 shutdown_connections 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.650+0000 7fbd6ab9d700 1 -- 172.21.15.119:0/3306799056 wait complete. 2023-10-08T01:11:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T01:11:08.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T01:11:08.724 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T01:11:08.794 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:08.793+0000 7f695a0ab180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:11:08.799 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:08.798+0000 7f3f4cc64180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:11:08.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.865+0000 7f5eaef9d700 1 Processor -- start 2023-10-08T01:11:08.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.865+0000 7f5eaef9d700 1 -- start start 2023-10-08T01:11:08.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eaef9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ea8159a60 0x7f5ea80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eaef9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5ea80ccc00 0x7f5ea8069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eaef9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea80c9cb0 con 0x7f5ea806a020 2023-10-08T01:11:08.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eaef9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea806dfc0 con 0x7f5ea8159a60 2023-10-08T01:11:08.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eaef9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea80c9f30 con 0x7f5ea80ccc00 2023-10-08T01:11:08.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eadf9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ea8159a60 0x7f5ea80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eae79c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5ea806a020 0x7f5ea80cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54612/0 (socket says 172.21.15.119:54612) 2023-10-08T01:11:08.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eadf9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ea8159a60 0x7f5ea80c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39686/0 (socket says 172.21.15.119:39686) 2023-10-08T01:11:08.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5eae79c700 1 -- 172.21.15.119:0/1314022229 learned_addr learned my addr 172.21.15.119:0/1314022229 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:08.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.867+0000 7f5ead79a700 1 --2- 172.21.15.119:0/1314022229 >> v2:172.21.15.184:3300/0 conn(0x7f5ea80ccc00 0x7f5ea8069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eadf9b700 1 -- 172.21.15.119:0/1314022229 >> v2:172.21.15.184:3300/0 conn(0x7f5ea80ccc00 msgr2=0x7f5ea8069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eadf9b700 1 --2- 172.21.15.119:0/1314022229 >> v2:172.21.15.184:3300/0 conn(0x7f5ea80ccc00 0x7f5ea8069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eadf9b700 1 -- 172.21.15.119:0/1314022229 >> v1:172.21.15.119:6789/0 conn(0x7f5ea806a020 legacy=0x7f5ea80cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eadf9b700 1 -- 172.21.15.119:0/1314022229 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea80ca470 con 0x7f5ea8159a60 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eacf99700 1 -- 172.21.15.119:0/1314022229 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1998938552 0 0) 0x7f5ea80c9cb0 con 0x7f5ea806a020 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eadf9b700 1 --2- 172.21.15.119:0/1314022229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea8159a60 0x7f5ea80c9770 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80cf6a0 tx=0x7f5ea40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f57e286e86f16c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eacf99700 1 -- 172.21.15.119:0/1314022229 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400b690 con 0x7f5ea8159a60 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eacf99700 1 -- 172.21.15.119:0/1314022229 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea400b7f0 con 0x7f5ea8159a60 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.868+0000 7f5eacf99700 1 -- 172.21.15.119:0/1314022229 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400bb90 con 0x7f5ea8159a60 2023-10-08T01:11:08.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.869+0000 7f5eaef9d700 1 -- 172.21.15.119:0/1314022229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea8159a60 msgr2=0x7f5ea80c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.869+0000 7f5eaef9d700 1 --2- 172.21.15.119:0/1314022229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea8159a60 0x7f5ea80c9770 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80cf6a0 tx=0x7f5ea40091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.869+0000 7f5eaef9d700 1 -- 172.21.15.119:0/1314022229 shutdown_connections 2023-10-08T01:11:08.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.869+0000 7f5eaef9d700 1 --2- 172.21.15.119:0/1314022229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea8159a60 0x7f5ea80c9770 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.869+0000 7f5eaef9d700 1 --2- 172.21.15.119:0/1314022229 >> v2:172.21.15.184:3300/0 conn(0x7f5ea80ccc00 0x7f5ea8069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.869+0000 7f5eaef9d700 1 -- 172.21.15.119:0/1314022229 >> 172.21.15.119:0/1314022229 conn(0x7f5ea806baa0 msgr2=0x7f5ea806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:08.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.869+0000 7f5eaef9d700 1 -- 172.21.15.119:0/1314022229 shutdown_connections 2023-10-08T01:11:08.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.869+0000 7f5eaef9d700 1 -- 172.21.15.119:0/1314022229 wait complete. 2023-10-08T01:11:08.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.871+0000 7f5eaef9d700 1 Processor -- start 2023-10-08T01:11:08.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.871+0000 7f5eaef9d700 1 -- start start 2023-10-08T01:11:08.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eaef9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ea806a020 0x7f5ea8146a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eaef9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea80ccc00 0x7f5ea80b36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eaef9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ea8159a60 0x7f5ea80a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eaef9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea80c6d90 con 0x7f5ea80ccc00 2023-10-08T01:11:08.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eaef9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea80c6180 con 0x7f5ea8159a60 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eaef9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea80c6460 con 0x7f5ea806a020 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eadf9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ea806a020 0x7f5ea8146a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5ead79a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea80ccc00 0x7f5ea80b36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eadf9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ea806a020 0x7f5ea8146a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34548/0 (socket says 172.21.15.119:34548) 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eadf9b700 1 -- 172.21.15.119:0/293144887 learned_addr learned my addr 172.21.15.119:0/293144887 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.873+0000 7f5eae79c700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ea8159a60 0x7f5ea80a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5eadf9b700 1 -- 172.21.15.119:0/293144887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ea8159a60 msgr2=0x7f5ea80a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5eadf9b700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ea8159a60 0x7f5ea80a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5eadf9b700 1 -- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea80ccc00 msgr2=0x7f5ea80b36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5eadf9b700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea80ccc00 0x7f5ea80b36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:08.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5eadf9b700 1 -- 172.21.15.119:0/293144887 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea4009420 con 0x7f5ea806a020 2023-10-08T01:11:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5eadf9b700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ea806a020 0x7f5ea8146a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ea806cdc0 tx=0x7f5ea4017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea400bb70 con 0x7f5ea806a020 2023-10-08T01:11:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea4011420 con 0x7f5ea806a020 2023-10-08T01:11:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40117c0 con 0x7f5ea806a020 2023-10-08T01:11:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea80c6740 con 0x7f5ea806a020 2023-10-08T01:11:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.874+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea8146f70 con 0x7f5ea806a020 2023-10-08T01:11:08.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.875+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea806eba0 con 0x7f5ea806a020 2023-10-08T01:11:08.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.877+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 29) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f5ea4011bb0 con 0x7f5ea806a020 2023-10-08T01:11:08.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.878+0000 7f5e9dffb700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:08.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.878+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6499+0+0 (secure 0 0 0) 0x7f5ea4086520 con 0x7f5ea806a020 2023-10-08T01:11:08.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.878+0000 7f5ead79a700 -1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:08.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.878+0000 7f5ead79a700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:08.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:08.881+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ea4053340 con 0x7f5ea806a020 2023-10-08T01:11:08.893 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:08.892+0000 7f695a0ab180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:11:08.903 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:08.901+0000 7f3f4cc64180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:11:09.054 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:09.053+0000 7f695a0ab180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:11:09.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.079+0000 7f5ead79a700 -1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.079+0000 7f5ead79a700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:09.090 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:09.089+0000 7f3f4cc64180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:11:09.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.127+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5ea80ca750 con 0x7f5ea806a020 2023-10-08T01:11:09.161 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:09.160+0000 7f695a0ab180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:11:09.181 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:09.180+0000 7f3f4cc64180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:11:09.272 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:09.271+0000 7f3f4cc64180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:11:09.354 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:09.353+0000 7f695a0ab180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:11:09.368 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:09.367+0000 7f3f4cc64180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:11:09.371 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:09.370+0000 7f3f42dfc700 -1 --2- 172.21.15.184:0/85410 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x56527899a000 0x5652789a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.448 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:09.447+0000 7f695a0ab180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:11:09.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.480+0000 7f5ead79a700 -1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.480+0000 7f5ead79a700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:11:09.540 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:09.540+0000 7f695a0ab180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:11:09.547 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T01:11:09.546+0000 7f27dc5ee700 -1 --2- 172.21.15.119:0/85510 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55b66184c000 0x55b661df2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.547 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-10-08T01:11:09.546+0000 7fbf013fb700 -1 --2- [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x558770040000 0x55876b158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.547 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-10-08T01:11:09.546+0000 7f10e01fb700 -1 --2- [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x561ef8d9c400 0x561ef1be4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.548 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-10-08T01:11:09.546+0000 7f922d3fd700 -1 --2- [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55a641b03000 0x55a638bbec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.548 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-10-08T01:11:09.546+0000 7fe0741fd700 -1 --2- [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55b545771c00 0x55b53982c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.549 INFO:tasks.ceph.osd.4.smithi184.stderr:2023-10-08T01:11:09.546+0000 7fb44b3fc700 -1 --2- [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x56313a362400 0x563134056000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.549 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-10-08T01:11:09.548+0000 7f834e9fc700 -1 --2- [v2:172.21.15.119:6832/1019078588,v1:172.21.15.119:6835/1019078588] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55f21d577800 0x55f21c939080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.549 INFO:tasks.ceph.mon.c.smithi184.stderr:2023-10-08T01:11:09.546+0000 7fa7b85f9700 -1 --2- 172.21.15.184:0/85409 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55e075785000 0x55e0761cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.550 INFO:tasks.ceph.osd.5.smithi184.stderr:2023-10-08T01:11:09.546+0000 7f78ebffd700 -1 --2- [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55c779eb3000 0x55c7755e8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.550 INFO:tasks.ceph.osd.7.smithi184.stderr:2023-10-08T01:11:09.546+0000 7fad6c9fc700 -1 --2- [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x5638b5d0ac00 0x5638badf3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.550 INFO:tasks.ceph.osd.6.smithi184.stderr:2023-10-08T01:11:09.547+0000 7f313b1fb700 -1 --2- [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55e2bf861000 0x55e2b93fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.551 INFO:tasks.ceph.mon.b.smithi184.stderr:2023-10-08T01:11:09.547+0000 7f27425f9700 -1 --2- 172.21.15.184:0/85408 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55a61d096000 0x55a61b9fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.551 INFO:tasks.ceph.mds.b.smithi184.stderr:2023-10-08T01:11:09.549+0000 7f70feffc700 -1 --2- [v2:172.21.15.184:6833/186953052,v1:172.21.15.184:6835/186953052] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x5575cad9d800 0x5575ca163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.551 INFO:tasks.ceph.mds.d.smithi184.stderr:2023-10-08T01:11:09.549+0000 7f762fdfb700 -1 --2- [v2:172.21.15.184:6832/1353749238,v1:172.21.15.184:6834/1353749238] >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x55e3dc997800 0x55e3dbd5d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.572 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:09.571+0000 7f3f42dfc700 -1 --2- 172.21.15.184:0/85410 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x56527899a000 0x5652789a0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.636 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:09.635+0000 7f695a0ab180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:11:09.639 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:09.638+0000 7f69501fc700 -1 --2- 172.21.15.119:0/85511 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x559f40378000 0x559f4037e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner peer [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] is using msgr V1 protocol 2023-10-08T01:11:09.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.649+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mgrmap(e 30) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f5ea4053710 con 0x7f5ea806a020 2023-10-08T01:11:09.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.649+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 msgr2=0x7f5e8c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:09.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.649+0000 7f5e9dffb700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.653+0000 7f5e9dffb700 1 -- 172.21.15.119:0/293144887 <== mon.2 v2:172.21.15.184:3301/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5ea4030210 con 0x7f5ea806a020 2023-10-08T01:11:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ea806a020 msgr2=0x7f5ea8146a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ea806a020 0x7f5ea8146a30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5ea806cdc0 tx=0x7f5ea4017000 comp rx=0 tx=0).stop 2023-10-08T01:11:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 shutdown_connections 2023-10-08T01:11:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:6834/85511,v1:172.21.15.119:6836/85511] conn(0x7f5e8c067050 0x7f5e8c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ea806a020 0x7f5ea8146a30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ea8159a60 0x7f5ea80a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 --2- 172.21.15.119:0/293144887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ea80ccc00 0x7f5ea80b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 >> 172.21.15.119:0/293144887 conn(0x7f5ea806baa0 msgr2=0x7f5ea815a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 shutdown_connections 2023-10-08T01:11:09.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.657+0000 7f5eaef9d700 1 -- 172.21.15.119:0/293144887 wait complete. 2023-10-08T01:11:09.679 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T01:11:09.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.868+0000 7f963c5ff700 1 Processor -- start 2023-10-08T01:11:09.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.869+0000 7f963c5ff700 1 -- start start 2023-10-08T01:11:09.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f963c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f9634156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:09.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f963c5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f963415a440 0x7f96340acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:09.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f963c5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963415a9b0 con 0x7f9634068890 2023-10-08T01:11:09.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f963c5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963415ac90 con 0x7f963415a440 2023-10-08T01:11:09.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f963c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963415af70 con 0x7f96341563b0 2023-10-08T01:11:09.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f9639b9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f963415a440 0x7f96340acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:09.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f963a39b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f9634156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f9639b9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f963415a440 0x7f96340acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:51274/0 (socket says 172.21.15.119:51274) 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f963ab9c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f9634068890 0x7f96340c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:48852/0 (socket says 172.21.15.119:48852) 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f9639b9a700 1 -- 172.21.15.119:0/137146759 learned_addr learned my addr 172.21.15.119:0/137146759 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.870+0000 7f9639399700 1 -- 172.21.15.119:0/137146759 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3515614728 0 0) 0x7f963415a9b0 con 0x7f9634068890 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f9639b9a700 1 -- 172.21.15.119:0/137146759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 msgr2=0x7f9634156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f9639b9a700 1 --2- 172.21.15.119:0/137146759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f9634156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f9639b9a700 1 -- 172.21.15.119:0/137146759 >> v1:172.21.15.184:6789/0 conn(0x7f9634068890 legacy=0x7f96340c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f9639b9a700 1 -- 172.21.15.119:0/137146759 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963415b250 con 0x7f963415a440 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f963a39b700 1 --2- 172.21.15.119:0/137146759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f9634156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:09.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f9639b9a700 1 --2- 172.21.15.119:0/137146759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 0x7f96340acf30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96240099b0 tx=0x7f9624009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9a0822e8b06dca9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f9639399700 1 -- 172.21.15.119:0/137146759 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624009ea0 con 0x7f963415a440 2023-10-08T01:11:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f9639399700 1 -- 172.21.15.119:0/137146759 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f962400b860 con 0x7f963415a440 2023-10-08T01:11:09.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.871+0000 7f9639399700 1 -- 172.21.15.119:0/137146759 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f962400bc00 con 0x7f963415a440 2023-10-08T01:11:09.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.872+0000 7f963c5ff700 1 -- 172.21.15.119:0/137146759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 msgr2=0x7f96340acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.872+0000 7f963c5ff700 1 --2- 172.21.15.119:0/137146759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 0x7f96340acf30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f96240099b0 tx=0x7f9624009250 comp rx=0 tx=0).stop 2023-10-08T01:11:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.872+0000 7f963c5ff700 1 -- 172.21.15.119:0/137146759 shutdown_connections 2023-10-08T01:11:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.872+0000 7f963c5ff700 1 --2- 172.21.15.119:0/137146759 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f9634156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.872+0000 7f963c5ff700 1 --2- 172.21.15.119:0/137146759 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 0x7f96340acf30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.872+0000 7f963c5ff700 1 -- 172.21.15.119:0/137146759 >> 172.21.15.119:0/137146759 conn(0x7f96340c8f90 msgr2=0x7f96340c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:09.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.872+0000 7f963c5ff700 1 -- 172.21.15.119:0/137146759 shutdown_connections 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.872+0000 7f963c5ff700 1 -- 172.21.15.119:0/137146759 wait complete. 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.873+0000 7f963c5ff700 1 Processor -- start 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.873+0000 7f963c5ff700 1 -- start start 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.874+0000 7f963c5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9634068890 0x7f9634150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.874+0000 7f963c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f96341541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.874+0000 7f963c5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 0x7f9634151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.874+0000 7f963c5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96340bf9d0 con 0x7f9634068890 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.874+0000 7f963c5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96340afc40 con 0x7f963415a440 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.874+0000 7f963c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96340aff20 con 0x7f96341563b0 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.874+0000 7f963a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9634068890 0x7f9634150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:09.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9634068890 0x7f9634150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39774/0 (socket says 172.21.15.119:39774) 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963a39b700 1 -- 172.21.15.119:0/1001041677 learned_addr learned my addr 172.21.15.119:0/1001041677 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963ab9c700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 0x7f9634151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963a39b700 1 -- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 msgr2=0x7f96341541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f9639b9a700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f96341541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963a39b700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f96341541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963a39b700 1 -- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 msgr2=0x7f9634151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963a39b700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 0x7f9634151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963a39b700 1 -- 172.21.15.119:0/1001041677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009420 con 0x7f9634068890 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963ab9c700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 0x7f9634151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:09.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f9639b9a700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f96341541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:09.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963a39b700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9634068890 0x7f9634150650 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f963000ec80 tx=0x7f96300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:09.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f962a7fc700 1 -- 172.21.15.119:0/1001041677 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963000c9d0 con 0x7f9634068890 2023-10-08T01:11:09.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f962a7fc700 1 -- 172.21.15.119:0/1001041677 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f963000cb30 con 0x7f9634068890 2023-10-08T01:11:09.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96240094f0 con 0x7f9634068890 2023-10-08T01:11:09.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.875+0000 7f962a7fc700 1 -- 172.21.15.119:0/1001041677 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963000ceb0 con 0x7f9634068890 2023-10-08T01:11:09.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.876+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96340b04e0 con 0x7f9634068890 2023-10-08T01:11:09.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.877+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96340718b0 con 0x7f9634068890 2023-10-08T01:11:09.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.877+0000 7f962a7fc700 1 -- 172.21.15.119:0/1001041677 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 30) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f9630013680 con 0x7f9634068890 2023-10-08T01:11:09.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.878+0000 7f962a7fc700 1 -- 172.21.15.119:0/1001041677 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6757+0+0 (secure 0 0 0) 0x7f96300849a0 con 0x7f9634068890 2023-10-08T01:11:09.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:09.882+0000 7f962a7fc700 1 -- 172.21.15.119:0/1001041677 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9630051ac0 con 0x7f9634068890 2023-10-08T01:11:10.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.073+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f963406d6a0 con 0x7f9634068890 2023-10-08T01:11:10.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.660+0000 7f962a7fc700 1 -- 172.21.15.119:0/1001041677 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 31) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f9630051570 con 0x7f9634068890 2023-10-08T01:11:10.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.664+0000 7f962a7fc700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f961806ca50 0x7f961806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:10.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.665+0000 7f9639b9a700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f961806ca50 0x7f961806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:10.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.668+0000 7f9639b9a700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f961806ca50 0x7f961806ee30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f96240099b0 tx=0x7f9624017000 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:10.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.676+0000 7f962a7fc700 1 -- 172.21.15.119:0/1001041677 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v67) v1 ==== 181+0+0 (secure 0 0 0) 0x7f963006fe20 con 0x7f9634068890 2023-10-08T01:11:10.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f961806ca50 msgr2=0x7f961806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:10.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f961806ca50 0x7f961806ee30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f96240099b0 tx=0x7f9624017000 comp rx=0 tx=0).stop 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9634068890 msgr2=0x7f9634150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9634068890 0x7f9634150650 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f963000ec80 tx=0x7f96300091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 shutdown_connections 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f961806ca50 0x7f961806ee30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f963415a440 0x7f9634151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9634068890 0x7f9634150650 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 --2- 172.21.15.119:0/1001041677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96341563b0 0x7f96341541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 >> 172.21.15.119:0/1001041677 conn(0x7f96340c8f90 msgr2=0x7f9634160040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 shutdown_connections 2023-10-08T01:11:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.682+0000 7f963c5ff700 1 -- 172.21.15.119:0/1001041677 wait complete. 2023-10-08T01:11:10.685 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T01:11:10.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T01:11:10.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.887+0000 7f157e9ff700 1 Processor -- start 2023-10-08T01:11:10.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.888+0000 7f157e9ff700 1 -- start start 2023-10-08T01:11:10.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f157e9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f157815a440 0x7f15780acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f157e9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f157e9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157815a9b0 con 0x7f15781563b0 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f157e9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157815ac90 con 0x7f157815a440 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f157e9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157815af70 con 0x7f1578068890 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f15777fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f157815a440 0x7f15780acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f15777fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f157815a440 0x7f15780acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39776/0 (socket says 172.21.15.119:39776) 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f15777fe700 1 -- 172.21.15.119:0/2070503139 learned_addr learned my addr 172.21.15.119:0/2070503139 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.889+0000 7f157cf9c700 1 --2- 172.21.15.119:0/2070503139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.890+0000 7f15777fe700 1 -- 172.21.15.119:0/2070503139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 msgr2=0x7f15780c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.890+0000 7f15777fe700 1 --2- 172.21.15.119:0/2070503139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.890+0000 7f15777fe700 1 -- 172.21.15.119:0/2070503139 >> v1:172.21.15.184:6789/0 conn(0x7f15781563b0 legacy=0x7f1578156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:10.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.890+0000 7f15777fe700 1 -- 172.21.15.119:0/2070503139 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157815b250 con 0x7f157815a440 2023-10-08T01:11:10.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.890+0000 7f15777fe700 1 --2- 172.21.15.119:0/2070503139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15780acf30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f156c0099b0 tx=0x7f156c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61be49fea0e04b92 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.890+0000 7f1576ffd700 1 -- 172.21.15.119:0/2070503139 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c009ea0 con 0x7f157815a440 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.890+0000 7f1576ffd700 1 -- 172.21.15.119:0/2070503139 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c00b860 con 0x7f157815a440 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.890+0000 7f1576ffd700 1 -- 172.21.15.119:0/2070503139 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00bc00 con 0x7f157815a440 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.891+0000 7f157e9ff700 1 -- 172.21.15.119:0/2070503139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 msgr2=0x7f15780acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.891+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2070503139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15780acf30 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f156c0099b0 tx=0x7f156c009250 comp rx=0 tx=0).stop 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.891+0000 7f157e9ff700 1 -- 172.21.15.119:0/2070503139 shutdown_connections 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.891+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2070503139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.891+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2070503139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15780acf30 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.891+0000 7f157e9ff700 1 -- 172.21.15.119:0/2070503139 >> 172.21.15.119:0/2070503139 conn(0x7f15780c8f90 msgr2=0x7f15780c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:10.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.891+0000 7f157e9ff700 1 -- 172.21.15.119:0/2070503139 shutdown_connections 2023-10-08T01:11:10.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.891+0000 7f157e9ff700 1 -- 172.21.15.119:0/2070503139 wait complete. 2023-10-08T01:11:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.892+0000 7f157e9ff700 1 Processor -- start 2023-10-08T01:11:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.892+0000 7f157e9ff700 1 -- start start 2023-10-08T01:11:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.893+0000 7f157e9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.893+0000 7f157e9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15781563b0 0x7f15780bff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.893+0000 7f157e9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15781478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.893+0000 7f157e9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780b1b60 con 0x7f157815a440 2023-10-08T01:11:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.893+0000 7f157e9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15780b0f50 con 0x7f15781563b0 2023-10-08T01:11:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.893+0000 7f157e9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15780b1230 con 0x7f1578068890 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15781478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15781478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39786/0 (socket says 172.21.15.119:39786) 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 -- 172.21.15.119:0/2396254249 learned_addr learned my addr 172.21.15.119:0/2396254249 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f15777fe700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15781563b0 0x7f15780bff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f1577fff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 -- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 msgr2=0x7f15780b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 -- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15781563b0 msgr2=0x7f15780bff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15781563b0 0x7f15780bff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 -- 172.21.15.119:0/2396254249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c0094f0 con 0x7f157815a440 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f1577fff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:10.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157cf9c700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15781478e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1570009df0 tx=0x7f157000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:10.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f155ffff700 1 -- 172.21.15.119:0/2396254249 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157000cb50 con 0x7f157815a440 2023-10-08T01:11:10.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f155ffff700 1 -- 172.21.15.119:0/2396254249 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157000f040 con 0x7f157815a440 2023-10-08T01:11:10.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15780b1510 con 0x7f157815a440 2023-10-08T01:11:10.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.894+0000 7f155ffff700 1 -- 172.21.15.119:0/2396254249 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1570014480 con 0x7f157815a440 2023-10-08T01:11:10.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.895+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1578147e20 con 0x7f157815a440 2023-10-08T01:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.896+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15780b1cd0 con 0x7f157815a440 2023-10-08T01:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.896+0000 7f155ffff700 1 -- 172.21.15.119:0/2396254249 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 31) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f1570014830 con 0x7f157815a440 2023-10-08T01:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.897+0000 7f155ffff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1564066d80 0x7f1564069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.897+0000 7f155ffff700 1 -- 172.21.15.119:0/2396254249 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6319+0+0 (secure 0 0 0) 0x7f1570084e00 con 0x7f157815a440 2023-10-08T01:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.897+0000 7f1577fff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1564066d80 0x7f1564069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.898+0000 7f1577fff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1564066d80 0x7f1564069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1568001820 tx=0x7f1568009040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:10.902+0000 7f155ffff700 1 -- 172.21.15.119:0/2396254249 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1570052080 con 0x7f157815a440 2023-10-08T01:11:11.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.092+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f15780718b0 con 0x7f157815a440 2023-10-08T01:11:11.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.673+0000 7f155ffff700 1 -- 172.21.15.119:0/2396254249 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v68) v1 ==== 169+0+0 (secure 0 0 0) 0x7f15700556a0 con 0x7f157815a440 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1564066d80 msgr2=0x7f1564069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1564066d80 0x7f1564069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1568001820 tx=0x7f1568009040 comp rx=0 tx=0).stop 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 msgr2=0x7f15781478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15781478e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1570009df0 tx=0x7f157000b040 comp rx=0 tx=0).stop 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 shutdown_connections 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1564066d80 0x7f1564069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1578068890 0x7f15780b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f157815a440 0x7f15781478e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 --2- 172.21.15.119:0/2396254249 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15781563b0 0x7f15780bff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 >> 172.21.15.119:0/2396254249 conn(0x7f15780c8f90 msgr2=0x7f15780adfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:11.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 shutdown_connections 2023-10-08T01:11:11.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.676+0000 7f157e9ff700 1 -- 172.21.15.119:0/2396254249 wait complete. 2023-10-08T01:11:11.679 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T01:11:11.699 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T01:11:11.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.879+0000 7f4bee9ff700 1 Processor -- start 2023-10-08T01:11:11.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.879+0000 7f4bee9ff700 1 -- start start 2023-10-08T01:11:11.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.880+0000 7f4bee9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4be8154160 0x7f4be81577b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:11.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.880+0000 7f4bee9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 0x7f4be80c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.880+0000 7f4bee9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be815b2e0 con 0x7f4be8154530 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.880+0000 7f4bee9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be815a6d0 con 0x7f4be8154160 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.880+0000 7f4bee9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be815a9b0 con 0x7f4be80ca120 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4be8154160 0x7f4be81577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4be8154160 0x7f4be81577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39788/0 (socket says 172.21.15.119:39788) 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4becf9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4be8154530 0x7f4be80c6e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54626/0 (socket says 172.21.15.119:54626) 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be7fff700 1 -- 172.21.15.119:0/169284778 learned_addr learned my addr 172.21.15.119:0/169284778 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be77fe700 1 --2- 172.21.15.119:0/169284778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 0x7f4be80c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be7fff700 1 -- 172.21.15.119:0/169284778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 msgr2=0x7f4be80c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be7fff700 1 --2- 172.21.15.119:0/169284778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 0x7f4be80c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be7fff700 1 -- 172.21.15.119:0/169284778 >> v1:172.21.15.119:6789/0 conn(0x7f4be8154530 legacy=0x7f4be80c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:11.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be7fff700 1 -- 172.21.15.119:0/169284778 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be815ac90 con 0x7f4be8154160 2023-10-08T01:11:11.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.881+0000 7f4be7fff700 1 --2- 172.21.15.119:0/169284778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be81577b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009990 tx=0x7f4bd8009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dace76dae683dd3c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:11.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4be6ffd700 1 -- 172.21.15.119:0/169284778 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd8005c10 con 0x7f4be8154160 2023-10-08T01:11:11.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4be6ffd700 1 -- 172.21.15.119:0/169284778 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bd8005d70 con 0x7f4be8154160 2023-10-08T01:11:11.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4be6ffd700 1 -- 172.21.15.119:0/169284778 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bd800b880 con 0x7f4be8154160 2023-10-08T01:11:11.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4bee9ff700 1 -- 172.21.15.119:0/169284778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 msgr2=0x7f4be81577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:11.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/169284778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be81577b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8009990 tx=0x7f4bd8009220 comp rx=0 tx=0).stop 2023-10-08T01:11:11.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4bee9ff700 1 -- 172.21.15.119:0/169284778 shutdown_connections 2023-10-08T01:11:11.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/169284778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be81577b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/169284778 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 0x7f4be80c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4bee9ff700 1 -- 172.21.15.119:0/169284778 >> 172.21.15.119:0/169284778 conn(0x7f4be806a360 msgr2=0x7f4be80cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4bee9ff700 1 -- 172.21.15.119:0/169284778 shutdown_connections 2023-10-08T01:11:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.882+0000 7f4bee9ff700 1 -- 172.21.15.119:0/169284778 wait complete. 2023-10-08T01:11:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.884+0000 7f4bee9ff700 1 Processor -- start 2023-10-08T01:11:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.884+0000 7f4bee9ff700 1 -- start start 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4bee9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 0x7f4be814ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4bee9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be80ad300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4bee9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4be8154530 0x7f4be8158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4bee9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be814f4b0 con 0x7f4be8154160 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4bee9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be816adb0 con 0x7f4be8154530 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4bee9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be816a1a0 con 0x7f4be80ca120 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be80ad300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be80ad300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39804/0 (socket says 172.21.15.119:39804) 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be77fe700 1 -- 172.21.15.119:0/1481275689 learned_addr learned my addr 172.21.15.119:0/1481275689 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:11.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be7fff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 0x7f4be814ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4becf9c700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4be8154530 0x7f4be8158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be77fe700 1 -- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 msgr2=0x7f4be814ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be77fe700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 0x7f4be814ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be77fe700 1 -- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4be8154530 msgr2=0x7f4be8158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be77fe700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4be8154530 0x7f4be8158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4be77fe700 1 -- 172.21.15.119:0/1481275689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd8009480 con 0x7f4be8154160 2023-10-08T01:11:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.885+0000 7f4becf9c700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4be8154530 0x7f4be8158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.886+0000 7f4be77fe700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be80ad300 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00d890 tx=0x7f4bdc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.886+0000 7f4bcffff700 1 -- 172.21.15.119:0/1481275689 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00b600 con 0x7f4be8154160 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.886+0000 7f4bcffff700 1 -- 172.21.15.119:0/1481275689 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc00b760 con 0x7f4be8154160 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.886+0000 7f4bcffff700 1 -- 172.21.15.119:0/1481275689 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc00bae0 con 0x7f4be8154160 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.886+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be816a480 con 0x7f4be8154160 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.886+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be816aa40 con 0x7f4be8154160 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.887+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be80c9b90 con 0x7f4be8154160 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.888+0000 7f4bcffff700 1 -- 172.21.15.119:0/1481275689 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bdc01e460 con 0x7f4be8154160 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.889+0000 7f4bcffff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4bd4066f30 0x7f4bd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.889+0000 7f4bcffff700 1 -- 172.21.15.119:0/1481275689 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f4bdc084c70 con 0x7f4be8154160 2023-10-08T01:11:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.889+0000 7f4be7fff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4bd4066f30 0x7f4bd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:11.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.890+0000 7f4be7fff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4bd4066f30 0x7f4bd40693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4be8153ab0 tx=0x7f4bd8016000 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:11.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:11.893+0000 7f4bcffff700 1 -- 172.21.15.119:0/1481275689 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bdc051e50 con 0x7f4be8154160 2023-10-08T01:11:12.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.083+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4be804c3c0 con 0x7f4be8154160 2023-10-08T01:11:12.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.087+0000 7f4bcffff700 1 -- 172.21.15.119:0/1481275689 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4bdc055470 con 0x7f4be8154160 2023-10-08T01:11:12.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4bd4066f30 msgr2=0x7f4bd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4bd4066f30 0x7f4bd40693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4be8153ab0 tx=0x7f4bd8016000 comp rx=0 tx=0).stop 2023-10-08T01:11:12.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 msgr2=0x7f4be80ad300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be80ad300 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00d890 tx=0x7f4bdc008040 comp rx=0 tx=0).stop 2023-10-08T01:11:12.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 shutdown_connections 2023-10-08T01:11:12.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4bd4066f30 0x7f4bd40693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4be80ca120 0x7f4be814ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4be8154530 0x7f4be8158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 --2- 172.21.15.119:0/1481275689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4be8154160 0x7f4be80ad300 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 >> 172.21.15.119:0/1481275689 conn(0x7f4be806a360 msgr2=0x7f4be80c6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:12.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 shutdown_connections 2023-10-08T01:11:12.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.090+0000 7f4bee9ff700 1 -- 172.21.15.119:0/1481275689 wait complete. 2023-10-08T01:11:12.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T01:11:12.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.301+0000 7f88af59e700 1 Processor -- start 2023-10-08T01:11:12.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.301+0000 7f88af59e700 1 -- start start 2023-10-08T01:11:12.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.302+0000 7f88af59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b0159a60 0x7f88b00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.302+0000 7f88af59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f88b00cb230 0x7f88b00cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.302+0000 7f88af59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b00c9cb0 con 0x7f88b00ccc00 2023-10-08T01:11:12.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.302+0000 7f88af59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b00cbbd0 con 0x7f88b00cb230 2023-10-08T01:11:12.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.302+0000 7f88af59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b006dfc0 con 0x7f88b0159a60 2023-10-08T01:11:12.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.302+0000 7f88add9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f88b00ccc00 0x7f88b0069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54628/0 (socket says 172.21.15.119:54628) 2023-10-08T01:11:12.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.302+0000 7f88add9b700 1 -- 172.21.15.119:0/647425344 learned_addr learned my addr 172.21.15.119:0/647425344 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:12.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ae59c700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b0159a60 0x7f88b00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88aed9d700 1 --2- 172.21.15.119:0/647425344 >> v2:172.21.15.184:3300/0 conn(0x7f88b00cb230 0x7f88b00cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275673995 0 0) 0x7f88b00c9cb0 con 0x7f88b00ccc00 2023-10-08T01:11:12.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8898003570 con 0x7f88b00ccc00 2023-10-08T01:11:12.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2274240149 0 0) 0x7f8898003570 con 0x7f88b00ccc00 2023-10-08T01:11:12.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b0159a60 msgr2=0x7f88b00c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:12.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b0159a60 0x7f88b00c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 >> v2:172.21.15.184:3300/0 conn(0x7f88b00cb230 msgr2=0x7f88b00cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:12.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 --2- 172.21.15.119:0/647425344 >> v2:172.21.15.184:3300/0 conn(0x7f88b00cb230 0x7f88b00cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00c9f30 con 0x7f88b00ccc00 2023-10-08T01:11:12.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f88a4001470 con 0x7f88b00ccc00 2023-10-08T01:11:12.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 >> v1:172.21.15.119:6789/0 conn(0x7f88b00ccc00 legacy=0x7f88b0069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8898003a90 0x7f8898003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8898004810 0x7f8898004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b0159a60 0x7f88980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b006dfc0 con 0x7f8898004810 2023-10-08T01:11:12.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889800ba60 con 0x7f8898003a90 2023-10-08T01:11:12.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.303+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f889800bba0 con 0x7f88b0159a60 2023-10-08T01:11:12.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88add9b700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8898004810 0x7f8898004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88ae59c700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8898003a90 0x7f8898003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88aed9d700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b0159a60 0x7f88980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88add9b700 1 -- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b0159a60 msgr2=0x7f88980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:12.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88add9b700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b0159a60 0x7f88980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88add9b700 1 -- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8898003a90 msgr2=0x7f8898003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:12.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88add9b700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8898003a90 0x7f8898003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88add9b700 1 -- 172.21.15.119:0/647425344 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898003570 con 0x7f8898004810 2023-10-08T01:11:12.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88add9b700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8898004810 0x7f8898004c80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c3f0 tx=0x7f88a4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713230ab676b906e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:12.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a400cc80 con 0x7f8898004810 2023-10-08T01:11:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a400cde0 con 0x7f8898004810 2023-10-08T01:11:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.304+0000 7f88ad59a700 1 -- 172.21.15.119:0/647425344 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4005b50 con 0x7f8898004810 2023-10-08T01:11:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.305+0000 7f88af59e700 1 -- 172.21.15.119:0/647425344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8898004810 msgr2=0x7f8898004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.305+0000 7f88af59e700 1 --2- 172.21.15.119:0/647425344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8898004810 0x7f8898004c80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f88a400c3f0 tx=0x7f88a4001e50 comp rx=0 tx=0).stop 2023-10-08T01:11:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.305+0000 7f88aed9d700 1 -- 172.21.15.119:0/647425344 reap_dead start 2023-10-08T01:11:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.305+0000 7f88af59e700 1 -- 172.21.15.119:0/647425344 shutdown_connections 2023-10-08T01:11:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.305+0000 7f88af59e700 1 -- 172.21.15.119:0/647425344 >> 172.21.15.119:0/647425344 conn(0x7f88b006baa0 msgr2=0x7f88b006bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.305+0000 7f88af59e700 1 -- 172.21.15.119:0/647425344 shutdown_connections 2023-10-08T01:11:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.305+0000 7f88af59e700 1 -- 172.21.15.119:0/647425344 wait complete. 2023-10-08T01:11:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.306+0000 7f88af59e700 1 Processor -- start 2023-10-08T01:11:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.306+0000 7f88af59e700 1 -- start start 2023-10-08T01:11:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.307+0000 7f88af59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8898004810 0x7f88b00b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.307+0000 7f88af59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b00c8200 0x7f88b0146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.307+0000 7f88af59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b0157400 0x7f88b0157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.307+0000 7f88af59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00b4180 con 0x7f88b0157400 2023-10-08T01:11:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.307+0000 7f88af59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b0157d90 con 0x7f8898004810 2023-10-08T01:11:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.307+0000 7f88af59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00c8670 con 0x7f88b00c8200 2023-10-08T01:11:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b0157400 0x7f88b0157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b0157400 0x7f88b0157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39822/0 (socket says 172.21.15.119:39822) 2023-10-08T01:11:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 -- 172.21.15.119:0/2398134278 learned_addr learned my addr 172.21.15.119:0/2398134278 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 -- 172.21.15.119:0/2398134278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b00c8200 msgr2=0x7f88b0146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b00c8200 0x7f88b0146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 -- 172.21.15.119:0/2398134278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8898004810 msgr2=0x7f88b00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8898004810 0x7f88b00b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 -- 172.21.15.119:0/2398134278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b015c290 con 0x7f88b0157400 2023-10-08T01:11:12.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88aed9d700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b0157400 0x7f88b0157850 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f88b006e510 tx=0x7f88a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f889e7fc700 1 -- 172.21.15.119:0/2398134278 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a800e070 con 0x7f88b0157400 2023-10-08T01:11:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f889e7fc700 1 -- 172.21.15.119:0/2398134278 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a8011d00 con 0x7f88b0157400 2023-10-08T01:11:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f889e7fc700 1 -- 172.21.15.119:0/2398134278 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a80166e0 con 0x7f88b0157400 2023-10-08T01:11:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.308+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b00b3ea0 con 0x7f88b0157400 2023-10-08T01:11:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.309+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b00c5d10 con 0x7f88b0157400 2023-10-08T01:11:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.310+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b00c5860 con 0x7f88b0157400 2023-10-08T01:11:12.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.310+0000 7f889e7fc700 1 -- 172.21.15.119:0/2398134278 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f88a8016a90 con 0x7f88b0157400 2023-10-08T01:11:12.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.311+0000 7f889e7fc700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8890066f30 0x7f88900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.311+0000 7f889e7fc700 1 -- 172.21.15.119:0/2398134278 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f88a8085270 con 0x7f88b0157400 2023-10-08T01:11:12.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.312+0000 7f88ae59c700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8890066f30 0x7f88900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.312+0000 7f88ae59c700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8890066f30 0x7f88900693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006f00 tx=0x7f88a00091a0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:12.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.315+0000 7f889e7fc700 1 -- 172.21.15.119:0/2398134278 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88a80523b0 con 0x7f88b0157400 2023-10-08T01:11:12.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.507+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f88b00ca470 con 0x7f88b0157400 2023-10-08T01:11:12.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.512+0000 7f889e7fc700 1 -- 172.21.15.119:0/2398134278 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f88a80559d0 con 0x7f88b0157400 2023-10-08T01:11:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8890066f30 msgr2=0x7f88900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8890066f30 0x7f88900693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f88a0006f00 tx=0x7f88a00091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b0157400 msgr2=0x7f88b0157850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b0157400 0x7f88b0157850 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f88b006e510 tx=0x7f88a800b040 comp rx=0 tx=0).stop 2023-10-08T01:11:12.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 shutdown_connections 2023-10-08T01:11:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8890066f30 0x7f88900693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88b0157400 0x7f88b0157850 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8898004810 0x7f88b00b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 --2- 172.21.15.119:0/2398134278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f88b00c8200 0x7f88b0146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 >> 172.21.15.119:0/2398134278 conn(0x7f88b006baa0 msgr2=0x7f88b0156530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 shutdown_connections 2023-10-08T01:11:12.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.514+0000 7f88af59e700 1 -- 172.21.15.119:0/2398134278 wait complete. 2023-10-08T01:11:12.534 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T01:11:12.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.718+0000 7fa2aa5ff700 1 Processor -- start 2023-10-08T01:11:12.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.718+0000 7fa2aa5ff700 1 -- start start 2023-10-08T01:11:12.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.720+0000 7fa2aa5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2a4155eb0 0x7fa2a406e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.720+0000 7fa2aa5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a406eb90 con 0x7fa2a4069e80 2023-10-08T01:11:12.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.720+0000 7fa2aa5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a406ee70 con 0x7fa2a40c70b0 2023-10-08T01:11:12.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.720+0000 7fa2aa5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a406f150 con 0x7fa2a4155eb0 2023-10-08T01:11:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.720+0000 7fa2a3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2a4155eb0 0x7fa2a406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.720+0000 7fa2a8b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa2a4069e80 0x7fa2a40ca4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54632/0 (socket says 172.21.15.119:54632) 2023-10-08T01:11:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.720+0000 7fa2a8b9c700 1 -- 172.21.15.119:0/1213768914 learned_addr learned my addr 172.21.15.119:0/1213768914 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.720+0000 7fa2a3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2a4155eb0 0x7fa2a406e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39824/0 (socket says 172.21.15.119:39824) 2023-10-08T01:11:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.721+0000 7fa2a3fff700 1 -- 172.21.15.119:0/1213768914 >> v1:172.21.15.184:6789/0 conn(0x7fa2a40c70b0 legacy=0x7fa2a4069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.721+0000 7fa2a3fff700 1 -- 172.21.15.119:0/1213768914 >> v1:172.21.15.119:6789/0 conn(0x7fa2a4069e80 legacy=0x7fa2a40ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.721+0000 7fa2a3fff700 1 -- 172.21.15.119:0/1213768914 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a406f430 con 0x7fa2a4155eb0 2023-10-08T01:11:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.721+0000 7fa2a3fff700 1 --2- 172.21.15.119:0/1213768914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a406e240 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa2940098e0 tx=0x7fa2940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411f2bc5c66a42b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.721+0000 7fa2a37fe700 1 -- 172.21.15.119:0/1213768914 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa294009e80 con 0x7fa2a4155eb0 2023-10-08T01:11:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.721+0000 7fa2a37fe700 1 -- 172.21.15.119:0/1213768914 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29400b860 con 0x7fa2a4155eb0 2023-10-08T01:11:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.721+0000 7fa2a37fe700 1 -- 172.21.15.119:0/1213768914 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29400bbe0 con 0x7fa2a4155eb0 2023-10-08T01:11:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.722+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/1213768914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 msgr2=0x7fa2a406e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.722+0000 7fa2aa5ff700 1 --2- 172.21.15.119:0/1213768914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a406e240 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa2940098e0 tx=0x7fa2940091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.722+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/1213768914 shutdown_connections 2023-10-08T01:11:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.722+0000 7fa2aa5ff700 1 --2- 172.21.15.119:0/1213768914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a406e240 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.722+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/1213768914 >> 172.21.15.119:0/1213768914 conn(0x7fa2a4068e10 msgr2=0x7fa2a40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.722+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/1213768914 shutdown_connections 2023-10-08T01:11:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.722+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/1213768914 wait complete. 2023-10-08T01:11:12.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.723+0000 7fa2aa5ff700 1 Processor -- start 2023-10-08T01:11:12.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.724+0000 7fa2aa5ff700 1 -- start start 2023-10-08T01:11:12.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.724+0000 7fa2aa5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4069e80 0x7fa2a4152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.724+0000 7fa2aa5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a40c70b0 0x7fa2a40c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.724+0000 7fa2aa5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a40c2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.724+0000 7fa2aa5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a41535e0 con 0x7fa2a4155eb0 2023-10-08T01:11:12.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.724+0000 7fa2aa5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a416a090 con 0x7fa2a4069e80 2023-10-08T01:11:12.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.724+0000 7fa2aa5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a416a370 con 0x7fa2a40c70b0 2023-10-08T01:11:12.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a40c2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a40c2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39838/0 (socket says 172.21.15.119:39838) 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 -- 172.21.15.119:0/3864726781 learned_addr learned my addr 172.21.15.119:0/3864726781 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a3fff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4069e80 0x7fa2a4152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa29bfff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a40c70b0 0x7fa2a40c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 -- 172.21.15.119:0/3864726781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a40c70b0 msgr2=0x7fa2a40c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a40c70b0 0x7fa2a40c6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 -- 172.21.15.119:0/3864726781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4069e80 msgr2=0x7fa2a4152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4069e80 0x7fa2a4152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 -- 172.21.15.119:0/3864726781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa294009420 con 0x7fa2a4155eb0 2023-10-08T01:11:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.725+0000 7fa2a8b9c700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a40c2170 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00cbc0 tx=0x7fa29c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.726+0000 7fa2a0ff9700 1 -- 172.21.15.119:0/3864726781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00a890 con 0x7fa2a4155eb0 2023-10-08T01:11:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.726+0000 7fa2a0ff9700 1 -- 172.21.15.119:0/3864726781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c00a9f0 con 0x7fa2a4155eb0 2023-10-08T01:11:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.726+0000 7fa2a0ff9700 1 -- 172.21.15.119:0/3864726781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00adb0 con 0x7fa2a4155eb0 2023-10-08T01:11:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.726+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a416a650 con 0x7fa2a4155eb0 2023-10-08T01:11:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.726+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a416ac90 con 0x7fa2a4155eb0 2023-10-08T01:11:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.727+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a404c3c0 con 0x7fa2a4155eb0 2023-10-08T01:11:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.728+0000 7fa2a0ff9700 1 -- 172.21.15.119:0/3864726781 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa29c007300 con 0x7fa2a4155eb0 2023-10-08T01:11:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.728+0000 7fa2a0ff9700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa284066f30 0x7fa2840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.729+0000 7fa2a0ff9700 1 -- 172.21.15.119:0/3864726781 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fa29c084940 con 0x7fa2a4155eb0 2023-10-08T01:11:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.729+0000 7fa2a3fff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa284066f30 0x7fa2840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.729+0000 7fa2a3fff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa284066f30 0x7fa2840693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2940091e0 tx=0x7fa294018000 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.733+0000 7fa2a0ff9700 1 -- 172.21.15.119:0/3864726781 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa29c051b00 con 0x7fa2a4155eb0 2023-10-08T01:11:12.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.924+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa2a41562e0 con 0x7fa2a4155eb0 2023-10-08T01:11:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.931+0000 7fa2a0ff9700 1 -- 172.21.15.119:0/3864726781 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v21) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa29c055120 con 0x7fa2a4155eb0 2023-10-08T01:11:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.933+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa284066f30 msgr2=0x7fa2840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.933+0000 7fa2aa5ff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa284066f30 0x7fa2840693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa2940091e0 tx=0x7fa294018000 comp rx=0 tx=0).stop 2023-10-08T01:11:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.933+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 msgr2=0x7fa2a40c2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.933+0000 7fa2aa5ff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a40c2170 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00cbc0 tx=0x7fa29c0019c0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.934+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 shutdown_connections 2023-10-08T01:11:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.934+0000 7fa2aa5ff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa284066f30 0x7fa2840693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.934+0000 7fa2aa5ff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2a4155eb0 0x7fa2a40c2170 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.934+0000 7fa2aa5ff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2a4069e80 0x7fa2a4152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.934+0000 7fa2aa5ff700 1 --2- 172.21.15.119:0/3864726781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2a40c70b0 0x7fa2a40c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.934+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 >> 172.21.15.119:0/3864726781 conn(0x7fa2a4068e10 msgr2=0x7fa2a4155140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.934+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 shutdown_connections 2023-10-08T01:11:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:12.934+0000 7fa2aa5ff700 1 -- 172.21.15.119:0/3864726781 wait complete. 2023-10-08T01:11:12.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-10-08T01:11:13.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.133+0000 7f2fb49fe700 1 Processor -- start 2023-10-08T01:11:13.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.133+0000 7f2fb49fe700 1 -- start start 2023-10-08T01:11:13.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.134+0000 7f2fb49fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2fb006a660 0x7f2fb0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.134+0000 7f2fb49fe700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb006e2a0 con 0x7f2fb0150550 2023-10-08T01:11:13.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.134+0000 7f2fb49fe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb006b2e0 con 0x7f2fb0150180 2023-10-08T01:11:13.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.134+0000 7f2fb49fe700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb00c78e0 con 0x7f2fb006a660 2023-10-08T01:11:13.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.135+0000 7f2fae59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2fb0150550 0x7f2fb0069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54646/0 (socket says 172.21.15.119:54646) 2023-10-08T01:11:13.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.135+0000 7f2fae59c700 1 -- 172.21.15.119:0/610062682 learned_addr learned my addr 172.21.15.119:0/610062682 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.135+0000 7f2faf59e700 1 --2- 172.21.15.119:0/610062682 >> v2:172.21.15.184:3300/0 conn(0x7f2fb006a660 0x7f2fb0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.135+0000 7f2fadd9b700 1 -- 172.21.15.119:0/610062682 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266794366 0 0) 0x7f2fb006e2a0 con 0x7f2fb0150550 2023-10-08T01:11:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.135+0000 7f2fadd9b700 1 -- 172.21.15.119:0/610062682 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f98003570 con 0x7f2fb0150550 2023-10-08T01:11:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.135+0000 7f2faf59e700 1 -- 172.21.15.119:0/610062682 >> v1:172.21.15.184:6789/0 conn(0x7f2fb0150180 legacy=0x7f2fb00c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.135+0000 7f2faf59e700 1 -- 172.21.15.119:0/610062682 >> v1:172.21.15.119:6789/0 conn(0x7f2fb0150550 legacy=0x7f2fb0069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.135+0000 7f2faf59e700 1 -- 172.21.15.119:0/610062682 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00c7bc0 con 0x7f2fb006a660 2023-10-08T01:11:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.136+0000 7f2faf59e700 1 --2- 172.21.15.119:0/610062682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb006a660 0x7f2fb0157770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009e90 tx=0x7f2fa8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fec46f8e29e20228 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.136+0000 7f2fadd9b700 1 -- 172.21.15.119:0/610062682 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa800bc40 con 0x7f2fb006a660 2023-10-08T01:11:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.136+0000 7f2fadd9b700 1 -- 172.21.15.119:0/610062682 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa800bda0 con 0x7f2fb006a660 2023-10-08T01:11:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.136+0000 7f2fadd9b700 1 -- 172.21.15.119:0/610062682 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8011650 con 0x7f2fb006a660 2023-10-08T01:11:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.137+0000 7f2fb49fe700 1 -- 172.21.15.119:0/610062682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb006a660 msgr2=0x7f2fb0157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.137+0000 7f2fb49fe700 1 --2- 172.21.15.119:0/610062682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb006a660 0x7f2fb0157770 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009e90 tx=0x7f2fa8009780 comp rx=0 tx=0).stop 2023-10-08T01:11:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.137+0000 7f2fb49fe700 1 -- 172.21.15.119:0/610062682 shutdown_connections 2023-10-08T01:11:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.137+0000 7f2fb49fe700 1 --2- 172.21.15.119:0/610062682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb006a660 0x7f2fb0157770 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.137+0000 7f2fb49fe700 1 -- 172.21.15.119:0/610062682 >> 172.21.15.119:0/610062682 conn(0x7f2fb00cc1f0 msgr2=0x7f2fb00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.137+0000 7f2fb49fe700 1 -- 172.21.15.119:0/610062682 shutdown_connections 2023-10-08T01:11:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.137+0000 7f2fb49fe700 1 -- 172.21.15.119:0/610062682 wait complete. 2023-10-08T01:11:13.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.138+0000 7f2fb49fe700 1 Processor -- start 2023-10-08T01:11:13.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.138+0000 7f2fb49fe700 1 -- start start 2023-10-08T01:11:13.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.139+0000 7f2fb49fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb006a660 0x7f2fb0143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.139+0000 7f2fb49fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fb0150180 0x7f2fb00c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.139+0000 7f2fb49fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0150550 0x7f2fb00c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.139+0000 7f2fb49fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0169600 con 0x7f2fb006a660 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.139+0000 7f2fb49fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb016c9e0 con 0x7f2fb0150550 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.139+0000 7f2fb49fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb016a420 con 0x7f2fb0150180 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.139+0000 7f2faed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb006a660 0x7f2fb0143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2faed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb006a660 0x7f2fb0143600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39850/0 (socket says 172.21.15.119:39850) 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2faed9d700 1 -- 172.21.15.119:0/1890999914 learned_addr learned my addr 172.21.15.119:0/1890999914 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2faed9d700 1 -- 172.21.15.119:0/1890999914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fb0150180 msgr2=0x7f2fb00c58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2faed9d700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fb0150180 0x7f2fb00c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2faed9d700 1 -- 172.21.15.119:0/1890999914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0150550 msgr2=0x7f2fb00c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2faed9d700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0150550 0x7f2fb00c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2faed9d700 1 -- 172.21.15.119:0/1890999914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa8009a20 con 0x7f2fb006a660 2023-10-08T01:11:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2faed9d700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb006a660 0x7f2fb0143600 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000c590 tx=0x7f2fa000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2f96ffd700 1 -- 172.21.15.119:0/1890999914 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0011590 con 0x7f2fb006a660 2023-10-08T01:11:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2f96ffd700 1 -- 172.21.15.119:0/1890999914 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa00116f0 con 0x7f2fb006a660 2023-10-08T01:11:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2f96ffd700 1 -- 172.21.15.119:0/1890999914 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0011ab0 con 0x7f2fb006a660 2023-10-08T01:11:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.140+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb0169810 con 0x7f2fb006a660 2023-10-08T01:11:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.141+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb0169dd0 con 0x7f2fb006a660 2023-10-08T01:11:13.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.142+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb004c3c0 con 0x7f2fb006a660 2023-10-08T01:11:13.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.142+0000 7f2f96ffd700 1 -- 172.21.15.119:0/1890999914 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fa001b3b0 con 0x7f2fb006a660 2023-10-08T01:11:13.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.143+0000 7f2f96ffd700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2f9c066f30 0x7f2f9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.143+0000 7f2f96ffd700 1 -- 172.21.15.119:0/1890999914 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f2fa0083a70 con 0x7f2fb006a660 2023-10-08T01:11:13.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.144+0000 7f2fae59c700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2f9c066f30 0x7f2f9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.144+0000 7f2fae59c700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2f9c066f30 0x7f2f9c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40058d0 tx=0x7f2fa40079d0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:13.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.147+0000 7f2f96ffd700 1 -- 172.21.15.119:0/1890999914 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fa0050c50 con 0x7f2fb006a660 2023-10-08T01:11:13.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.414+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f2fb006ada0 con 0x7f2fb006a660 2023-10-08T01:11:13.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.682+0000 7f2f96ffd700 1 -- 172.21.15.119:0/1890999914 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v556) v1 ==== 154+0+0 (secure 0 0 0) 0x7f2fa0017070 con 0x7f2fb006a660 2023-10-08T01:11:13.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2f9c066f30 msgr2=0x7f2f9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2f9c066f30 0x7f2f9c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40058d0 tx=0x7f2fa40079d0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb006a660 msgr2=0x7f2fb0143600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb006a660 0x7f2fb0143600 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000c590 tx=0x7f2fa000c950 comp rx=0 tx=0).stop 2023-10-08T01:11:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 shutdown_connections 2023-10-08T01:11:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2f9c066f30 0x7f2f9c0693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fb0150550 0x7f2fb00c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb006a660 0x7f2fb0143600 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 --2- 172.21.15.119:0/1890999914 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fb0150180 0x7f2fb00c58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 >> 172.21.15.119:0/1890999914 conn(0x7f2fb00cc1f0 msgr2=0x7f2fb0069f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 shutdown_connections 2023-10-08T01:11:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.685+0000 7f2fb49fe700 1 -- 172.21.15.119:0/1890999914 wait complete. 2023-10-08T01:11:13.704 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-10-08T01:11:13.704 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-10-08T01:11:13.704 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T01:11:13.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-08T01:11:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.885+0000 7f61769ff700 1 Processor -- start 2023-10-08T01:11:13.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.886+0000 7f61769ff700 1 -- start start 2023-10-08T01:11:13.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.887+0000 7f61769ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 0x7f61700ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.887+0000 7f61769ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61700ccf90 0x7f61700b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.887+0000 7f61769ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61700cc300 con 0x7f617015a270 2023-10-08T01:11:13.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.887+0000 7f61769ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6170156bf0 con 0x7f61700ccf90 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.887+0000 7f61769ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61700ae950 con 0x7f61700cbf30 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.887+0000 7f6167fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61700ccf90 0x7f61700b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f6167fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f61700ccf90 0x7f61700b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39860/0 (socket says 172.21.15.119:39860) 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f6167fff700 1 -- 172.21.15.119:0/2315373454 learned_addr learned my addr 172.21.15.119:0/2315373454 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f616ffff700 1 --2- 172.21.15.119:0/2315373454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 0x7f61700ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f6167fff700 1 -- 172.21.15.119:0/2315373454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 msgr2=0x7f61700ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f6167fff700 1 --2- 172.21.15.119:0/2315373454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 0x7f61700ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f6167fff700 1 -- 172.21.15.119:0/2315373454 >> v1:172.21.15.184:6789/0 conn(0x7f617015a270 legacy=0x7f61700c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f6167fff700 1 -- 172.21.15.119:0/2315373454 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61700ad370 con 0x7f61700ccf90 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f6167fff700 1 --2- 172.21.15.119:0/2315373454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61700ccf90 0x7f61700b0d10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f61580099e0 tx=0x7f6158009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcc7ec18dfafcc8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:13.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.888+0000 7f616f7fe700 1 -- 172.21.15.119:0/2315373454 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800b820 con 0x7f61700ccf90 2023-10-08T01:11:13.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f616f7fe700 1 -- 172.21.15.119:0/2315373454 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615800b980 con 0x7f61700ccf90 2023-10-08T01:11:13.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f616f7fe700 1 -- 172.21.15.119:0/2315373454 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615800bd20 con 0x7f61700ccf90 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f61769ff700 1 -- 172.21.15.119:0/2315373454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61700ccf90 msgr2=0x7f61700b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f61769ff700 1 --2- 172.21.15.119:0/2315373454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61700ccf90 0x7f61700b0d10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f61580099e0 tx=0x7f6158009250 comp rx=0 tx=0).stop 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f61769ff700 1 -- 172.21.15.119:0/2315373454 shutdown_connections 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f61769ff700 1 --2- 172.21.15.119:0/2315373454 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 0x7f61700ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f61769ff700 1 --2- 172.21.15.119:0/2315373454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61700ccf90 0x7f61700b0d10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f61769ff700 1 -- 172.21.15.119:0/2315373454 >> 172.21.15.119:0/2315373454 conn(0x7f617006a320 msgr2=0x7f61700cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f61769ff700 1 -- 172.21.15.119:0/2315373454 shutdown_connections 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.889+0000 7f61769ff700 1 -- 172.21.15.119:0/2315373454 wait complete. 2023-10-08T01:11:13.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.891+0000 7f61769ff700 1 Processor -- start 2023-10-08T01:11:13.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.891+0000 7f61769ff700 1 -- start start 2023-10-08T01:11:13.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f61769ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 0x7f6170158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f61769ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61700ccf90 0x7f6170151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f61769ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f617015a270 0x7f61700ac740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f61769ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6170151ab0 con 0x7f617015a270 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f61769ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6170157f00 con 0x7f61700ccf90 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f61769ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61701572f0 con 0x7f61700cbf30 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6174f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f617015a270 0x7f61700ac740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6174f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f617015a270 0x7f61700ac740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39864/0 (socket says 172.21.15.119:39864) 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6174f9c700 1 -- 172.21.15.119:0/381962217 learned_addr learned my addr 172.21.15.119:0/381962217 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:13.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6167fff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61700ccf90 0x7f6170151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f616ffff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 0x7f6170158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6174f9c700 1 -- 172.21.15.119:0/381962217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 msgr2=0x7f6170158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6174f9c700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 0x7f6170158390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6174f9c700 1 -- 172.21.15.119:0/381962217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61700ccf90 msgr2=0x7f6170151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6174f9c700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61700ccf90 0x7f6170151570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.892+0000 7f6174f9c700 1 -- 172.21.15.119:0/381962217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580094f0 con 0x7f617015a270 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.893+0000 7f6174f9c700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f617015a270 0x7f61700ac740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f616800c8e0 tx=0x7f6168001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.893+0000 7f616cff9700 1 -- 172.21.15.119:0/381962217 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168012630 con 0x7f617015a270 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.893+0000 7f616cff9700 1 -- 172.21.15.119:0/381962217 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61680127b0 con 0x7f617015a270 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.893+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61701575d0 con 0x7f617015a270 2023-10-08T01:11:13.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.893+0000 7f616cff9700 1 -- 172.21.15.119:0/381962217 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6168012b70 con 0x7f617015a270 2023-10-08T01:11:13.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.893+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6170157b90 con 0x7f617015a270 2023-10-08T01:11:13.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.894+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6170158070 con 0x7f617015a270 2023-10-08T01:11:13.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.895+0000 7f616cff9700 1 -- 172.21.15.119:0/381962217 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6168003030 con 0x7f617015a270 2023-10-08T01:11:13.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.896+0000 7f616cff9700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6150066f30 0x7f61500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:13.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.896+0000 7f616cff9700 1 -- 172.21.15.119:0/381962217 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5847+0+0 (secure 0 0 0) 0x7f6168084490 con 0x7f617015a270 2023-10-08T01:11:13.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.896+0000 7f616ffff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6150066f30 0x7f61500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:13.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.896+0000 7f616ffff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6150066f30 0x7f61500693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6160009d10 tx=0x7f61600091a0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:13.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:13.900+0000 7f616cff9700 1 -- 172.21.15.119:0/381962217 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61680515d0 con 0x7f617015a270 2023-10-08T01:11:14.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.164+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f617004c3c0 con 0x7f617015a270 2023-10-08T01:11:14.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.682+0000 7f616cff9700 1 -- 172.21.15.119:0/381962217 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v557) v1 ==== 159+0+0 (secure 0 0 0) 0x7f6168054bf0 con 0x7f617015a270 2023-10-08T01:11:14.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6150066f30 msgr2=0x7f61500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:14.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6150066f30 0x7f61500693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6160009d10 tx=0x7f61600091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f617015a270 msgr2=0x7f61700ac740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:14.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f617015a270 0x7f61700ac740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f616800c8e0 tx=0x7f6168001710 comp rx=0 tx=0).stop 2023-10-08T01:11:14.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 shutdown_connections 2023-10-08T01:11:14.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6150066f30 0x7f61500693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61700cbf30 0x7f6170158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f617015a270 0x7f61700ac740 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 --2- 172.21.15.119:0/381962217 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61700ccf90 0x7f6170151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 >> 172.21.15.119:0/381962217 conn(0x7f617006a320 msgr2=0x7f61700b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:14.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 shutdown_connections 2023-10-08T01:11:14.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.685+0000 7f61769ff700 1 -- 172.21.15.119:0/381962217 wait complete. 2023-10-08T01:11:14.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T01:11:14.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.890+0000 7fcfb719d700 1 Processor -- start 2023-10-08T01:11:14.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.890+0000 7fcfb719d700 1 -- start start 2023-10-08T01:11:14.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.891+0000 7fcfb719d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.891+0000 7fcfb719d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcfb00b33b0 0x7fcfb00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.891+0000 7fcfb719d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfb00b2080 con 0x7fcfb00b4e70 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.891+0000 7fcfb719d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfb00cfb00 con 0x7fcfb00b33b0 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.891+0000 7fcfb719d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00b5c40 con 0x7fcfb00b16d0 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb699c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcfb00b33b0 0x7fcfb00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb619b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcfb00b4e70 0x7fcfb00ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59974/0 (socket says 172.21.15.119:59974) 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb699c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcfb00b33b0 0x7fcfb00b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53402/0 (socket says 172.21.15.119:53402) 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb619b700 1 -- 172.21.15.119:0/2813775802 learned_addr learned my addr 172.21.15.119:0/2813775802 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb599a700 1 --2- 172.21.15.119:0/2813775802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:14.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb699c700 1 -- 172.21.15.119:0/2813775802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 msgr2=0x7fcfb00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb699c700 1 --2- 172.21.15.119:0/2813775802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb699c700 1 -- 172.21.15.119:0/2813775802 >> v1:172.21.15.119:6789/0 conn(0x7fcfb00b4e70 legacy=0x7fcfb00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb699c700 1 -- 172.21.15.119:0/2813775802 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb00b5f20 con 0x7fcfb00b33b0 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb599a700 1 --2- 172.21.15.119:0/2813775802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb5199700 1 -- 172.21.15.119:0/2813775802 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717930072 0 0) 0x7fcfb00b2080 con 0x7fcfb00b4e70 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.892+0000 7fcfb699c700 1 --2- 172.21.15.119:0/2813775802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00b3830 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009e60 tx=0x7fcfa8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a726fa5946573 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb5199700 1 -- 172.21.15.119:0/2813775802 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800e070 con 0x7fcfb00b33b0 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb5199700 1 -- 172.21.15.119:0/2813775802 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa800bc30 con 0x7fcfb00b33b0 2023-10-08T01:11:14.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb5199700 1 -- 172.21.15.119:0/2813775802 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8012430 con 0x7fcfb00b33b0 2023-10-08T01:11:14.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb719d700 1 -- 172.21.15.119:0/2813775802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 msgr2=0x7fcfb00b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:14.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb719d700 1 --2- 172.21.15.119:0/2813775802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00b3830 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009e60 tx=0x7fcfa8009780 comp rx=0 tx=0).stop 2023-10-08T01:11:14.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb719d700 1 -- 172.21.15.119:0/2813775802 shutdown_connections 2023-10-08T01:11:14.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb719d700 1 --2- 172.21.15.119:0/2813775802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00b3830 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb719d700 1 --2- 172.21.15.119:0/2813775802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb719d700 1 -- 172.21.15.119:0/2813775802 >> 172.21.15.119:0/2813775802 conn(0x7fcfb00d10c0 msgr2=0x7fcfb00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb719d700 1 -- 172.21.15.119:0/2813775802 shutdown_connections 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.893+0000 7fcfb719d700 1 -- 172.21.15.119:0/2813775802 wait complete. 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.895+0000 7fcfb719d700 1 Processor -- start 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.895+0000 7fcfb719d700 1 -- start start 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb719d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb719d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb719d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b4e70 0x7fcfb015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb719d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb01584c0 con 0x7fcfb00b33b0 2023-10-08T01:11:14.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb719d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00cb1e0 con 0x7fcfb00b4e70 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb719d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb00cb4c0 con 0x7fcfb00b16d0 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53408/0 (socket says 172.21.15.119:53408) 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb599a700 1 -- 172.21.15.119:0/4125672760 learned_addr learned my addr 172.21.15.119:0/4125672760 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb619b700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb699c700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b4e70 0x7fcfb015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb599a700 1 -- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 msgr2=0x7fcfb00ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb599a700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb599a700 1 -- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b4e70 msgr2=0x7fcfb015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb599a700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b4e70 0x7fcfb015dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:14.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.896+0000 7fcfb599a700 1 -- 172.21.15.119:0/4125672760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8009a20 con 0x7fcfb00b33b0 2023-10-08T01:11:14.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.897+0000 7fcfb619b700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:14.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.897+0000 7fcfb599a700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00cab40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e8d0 tx=0x7fcfa0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:14.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.897+0000 7fcfa67fc700 1 -- 172.21.15.119:0/4125672760 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000ee70 con 0x7fcfb00b33b0 2023-10-08T01:11:14.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.897+0000 7fcfa67fc700 1 -- 172.21.15.119:0/4125672760 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa000c5e0 con 0x7fcfb00b33b0 2023-10-08T01:11:14.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.897+0000 7fcfa67fc700 1 -- 172.21.15.119:0/4125672760 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa000c980 con 0x7fcfb00b33b0 2023-10-08T01:11:14.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.897+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb00cb7a0 con 0x7fcfb00b33b0 2023-10-08T01:11:14.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.897+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb00cbd60 con 0x7fcfb00b33b0 2023-10-08T01:11:14.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.898+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfb0088990 con 0x7fcfb00b33b0 2023-10-08T01:11:14.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.899+0000 7fcfa67fc700 1 -- 172.21.15.119:0/4125672760 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfa0013410 con 0x7fcfb00b33b0 2023-10-08T01:11:14.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.900+0000 7fcfa67fc700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcf94066f30 0x7fcf940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:14.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.900+0000 7fcfa67fc700 1 -- 172.21.15.119:0/4125672760 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fcfa0084470 con 0x7fcfb00b33b0 2023-10-08T01:11:14.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.900+0000 7fcfb619b700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcf94066f30 0x7fcf940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:14.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.901+0000 7fcfb619b700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcf94066f30 0x7fcf940693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00b4de0 tx=0x7fcfac009040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:14.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:14.904+0000 7fcfa67fc700 1 -- 172.21.15.119:0/4125672760 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfa00515b0 con 0x7fcfb00b33b0 2023-10-08T01:11:15.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.095+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcfb004bb80 con 0x7fcfb00b33b0 2023-10-08T01:11:15.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.100+0000 7fcfa67fc700 1 -- 172.21.15.119:0/4125672760 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcfa0054bd0 con 0x7fcfb00b33b0 2023-10-08T01:11:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcf94066f30 msgr2=0x7fcf940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcf94066f30 0x7fcf940693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00b4de0 tx=0x7fcfac009040 comp rx=0 tx=0).stop 2023-10-08T01:11:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 msgr2=0x7fcfb00cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00cab40 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000e8d0 tx=0x7fcfa0008040 comp rx=0 tx=0).stop 2023-10-08T01:11:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 shutdown_connections 2023-10-08T01:11:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcf94066f30 0x7fcf940693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcfb00b16d0 0x7fcfb00ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcfb00b4e70 0x7fcfb015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 --2- 172.21.15.119:0/4125672760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcfb00b33b0 0x7fcfb00cab40 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 >> 172.21.15.119:0/4125672760 conn(0x7fcfb00d10c0 msgr2=0x7fcfb0158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 shutdown_connections 2023-10-08T01:11:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.103+0000 7fcfb719d700 1 -- 172.21.15.119:0/4125672760 wait complete. 2023-10-08T01:11:15.123 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:11:15.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:11:15.208 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:11:15.294 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:11:15.294 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T01:11:15.294 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:15.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.474+0000 7ffbcf9ff700 1 Processor -- start 2023-10-08T01:11:15.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.474+0000 7ffbcf9ff700 1 -- start start 2023-10-08T01:11:15.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.475+0000 7ffbcf9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbc806bcd0 0x7ffbc80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:15.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.475+0000 7ffbcf9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffbc806c0a0 0x7ffbc815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:15.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.475+0000 7ffbcf9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:15.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.475+0000 7ffbcf9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc815d7e0 con 0x7ffbc806bcd0 2023-10-08T01:11:15.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.475+0000 7ffbcf9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc815dac0 con 0x7ffbc806c0a0 2023-10-08T01:11:15.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.475+0000 7ffbcf9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc815aa30 con 0x7ffbc815df50 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.475+0000 7ffbcd79b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbc806bcd0 0x7ffbc80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcd79b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffbc806bcd0 0x7ffbc80caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53422/0 (socket says 172.21.15.119:53422) 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcd79b700 1 -- 172.21.15.119:0/1495405102 learned_addr learned my addr 172.21.15.119:0/1495405102 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbccf9a700 1 --2- 172.21.15.119:0/1495405102 >> v2:172.21.15.184:3300/0 conn(0x7ffbc806c0a0 0x7ffbc815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcdf9c700 1 --2- 172.21.15.119:0/1495405102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcd79b700 1 -- 172.21.15.119:0/1495405102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 msgr2=0x7ffbc815e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcd79b700 1 --2- 172.21.15.119:0/1495405102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcd79b700 1 -- 172.21.15.119:0/1495405102 >> v2:172.21.15.184:3300/0 conn(0x7ffbc806c0a0 msgr2=0x7ffbc815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcd79b700 1 --2- 172.21.15.119:0/1495405102 >> v2:172.21.15.184:3300/0 conn(0x7ffbc806c0a0 0x7ffbc815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcd79b700 1 -- 172.21.15.119:0/1495405102 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc815ad10 con 0x7ffbc806bcd0 2023-10-08T01:11:15.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbccf9a700 1 --2- 172.21.15.119:0/1495405102 >> v2:172.21.15.184:3300/0 conn(0x7ffbc806c0a0 0x7ffbc815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:15.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.476+0000 7ffbcd79b700 1 --2- 172.21.15.119:0/1495405102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80caa90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffbc806b4e0 tx=0x7ffbc4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c4fd48f08eec93 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:15.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbbffff700 1 -- 172.21.15.119:0/1495405102 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400b860 con 0x7ffbc806bcd0 2023-10-08T01:11:15.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbbffff700 1 -- 172.21.15.119:0/1495405102 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc400b9c0 con 0x7ffbc806bcd0 2023-10-08T01:11:15.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbbffff700 1 -- 172.21.15.119:0/1495405102 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc400bd80 con 0x7ffbc806bcd0 2023-10-08T01:11:15.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/1495405102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 msgr2=0x7ffbc80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:15.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/1495405102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80caa90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ffbc806b4e0 tx=0x7ffbc4009250 comp rx=0 tx=0).stop 2023-10-08T01:11:15.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/1495405102 shutdown_connections 2023-10-08T01:11:15.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/1495405102 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/1495405102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80caa90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/1495405102 >> v2:172.21.15.184:3300/0 conn(0x7ffbc806c0a0 0x7ffbc815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/1495405102 >> 172.21.15.119:0/1495405102 conn(0x7ffbc80ce600 msgr2=0x7ffbc80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:15.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/1495405102 shutdown_connections 2023-10-08T01:11:15.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.477+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/1495405102 wait complete. 2023-10-08T01:11:15.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.479+0000 7ffbcf9ff700 1 Processor -- start 2023-10-08T01:11:15.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.479+0000 7ffbcf9ff700 1 -- start start 2023-10-08T01:11:15.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcf9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:15.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcf9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc806c0a0 0x7ffbc80c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:15.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcf9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:15.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcf9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80c87b0 con 0x7ffbc806bcd0 2023-10-08T01:11:15.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcf9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80c6340 con 0x7ffbc806c0a0 2023-10-08T01:11:15.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcf9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc80c89a0 con 0x7ffbc815df50 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53438/0 (socket says 172.21.15.119:53438) 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 -- 172.21.15.119:0/2428331384 learned_addr learned my addr 172.21.15.119:0/2428331384 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbccf9a700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc806c0a0 0x7ffbc80c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcdf9c700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 -- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 msgr2=0x7ffbc815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 -- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc806c0a0 msgr2=0x7ffbc80c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:15.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc806c0a0 0x7ffbc80c8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 -- 172.21.15.119:0/2428331384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40094f0 con 0x7ffbc806bcd0 2023-10-08T01:11:15.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbccf9a700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc806c0a0 0x7ffbc80c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:15.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcdf9c700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:15.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.480+0000 7ffbcd79b700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80c7d30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8069220 tx=0x7ffbc4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:15.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.481+0000 7ffbbd7fa700 1 -- 172.21.15.119:0/2428331384 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4026450 con 0x7ffbc806bcd0 2023-10-08T01:11:15.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.481+0000 7ffbbd7fa700 1 -- 172.21.15.119:0/2428331384 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc40265b0 con 0x7ffbc806bcd0 2023-10-08T01:11:15.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.481+0000 7ffbbd7fa700 1 -- 172.21.15.119:0/2428331384 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4026970 con 0x7ffbc806bcd0 2023-10-08T01:11:15.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.481+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc80c8c80 con 0x7ffbc806bcd0 2023-10-08T01:11:15.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.481+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc80c9240 con 0x7ffbc806bcd0 2023-10-08T01:11:15.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.482+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc804c3c0 con 0x7ffbc806bcd0 2023-10-08T01:11:15.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.483+0000 7ffbbd7fa700 1 -- 172.21.15.119:0/2428331384 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbc4026d20 con 0x7ffbc806bcd0 2023-10-08T01:11:15.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.483+0000 7ffbbd7fa700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffbac066f30 0x7ffbac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:15.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.484+0000 7ffbbd7fa700 1 -- 172.21.15.119:0/2428331384 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5847+0+0 (secure 0 0 0) 0x7ffbc4086130 con 0x7ffbc806bcd0 2023-10-08T01:11:15.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.484+0000 7ffbccf9a700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffbac066f30 0x7ffbac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:15.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.484+0000 7ffbccf9a700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffbac066f30 0x7ffbac0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8006f00 tx=0x7ffbb8008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:15.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.487+0000 7ffbbd7fa700 1 -- 172.21.15.119:0/2428331384 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbc40532f0 con 0x7ffbc806bcd0 2023-10-08T01:11:15.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.722+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffbc815b020 con 0x7ffbc806bcd0 2023-10-08T01:11:15.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.723+0000 7ffbbd7fa700 1 -- 172.21.15.119:0/2428331384 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3538 (secure 0 0 0) 0x7ffbc4015df0 con 0x7ffbc806bcd0 2023-10-08T01:11:15.724 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:15.724 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1019078588","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1019078588},{"type":"v1","addr":"172.21.15.119:6835","nonce":1019078588}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5126,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/1353749238","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":1353749238},{"type":"v1","addr":"172.21.15.184:6834","nonce":1353749238}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5524,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/186953052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":186953052},{"type":"v1","addr":"172.21.15.184:6835","nonce":186953052}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6123,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1363795730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1363795730},{"type":"v1","addr":"172.21.15.119:6834","nonce":1363795730}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-10-08T01:11:15.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffbac066f30 msgr2=0x7ffbac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffbac066f30 0x7ffbac0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffbb8006f00 tx=0x7ffbb8008040 comp rx=0 tx=0).stop 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 msgr2=0x7ffbc80c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80c7d30 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8069220 tx=0x7ffbc4017000 comp rx=0 tx=0).stop 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 shutdown_connections 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffbac066f30 0x7ffbac0693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffbc815df50 0x7ffbc815b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffbc806bcd0 0x7ffbc80c7d30 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 --2- 172.21.15.119:0/2428331384 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffbc806c0a0 0x7ffbc80c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 >> 172.21.15.119:0/2428331384 conn(0x7ffbc80ce600 msgr2=0x7ffbc8156d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 shutdown_connections 2023-10-08T01:11:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:15.725+0000 7ffbcf9ff700 1 -- 172.21.15.119:0/2428331384 wait complete. 2023-10-08T01:11:15.728 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2023-10-08T01:11:15.744 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T01:11:15.745 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T01:11:15.745 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T01:11:15.745 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:11:15.745 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T01:11:15.745 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T01:11:15.746 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T01:11:15.746 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:11:15.746 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T01:11:15.746 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T01:11:15.746 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T01:11:15.746 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:11:15.747 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T01:11:15.747 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T01:11:15.747 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T01:11:15.747 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:11:15.758 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T01:11:15.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T01:11:15.759 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T01:11:15.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T01:11:15.763 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T01:11:15.764 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T01:11:15.764 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T01:11:15.765 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T01:11:15.802 INFO:tasks.ceph.mds.a:Started 2023-10-08T01:11:15.803 DEBUG:teuthology.parallel:result is None 2023-10-08T01:11:15.805 INFO:tasks.ceph.mds.c:Started 2023-10-08T01:11:15.806 DEBUG:teuthology.parallel:result is None 2023-10-08T01:11:15.809 INFO:tasks.ceph.mds.d:Started 2023-10-08T01:11:15.809 DEBUG:teuthology.parallel:result is None 2023-10-08T01:11:15.812 INFO:tasks.ceph.mds.b:Started 2023-10-08T01:11:15.812 DEBUG:teuthology.parallel:result is None 2023-10-08T01:11:15.812 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T01:11:15.938 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:11:15.938 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T01:11:15.958 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T01:11:15.958 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T01:11:16.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.005+0000 7fc2cd19d700 1 Processor -- start 2023-10-08T01:11:16.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.005+0000 7fc2cd19d700 1 -- start start 2023-10-08T01:11:16.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.006+0000 7fc2cd19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc2c81597f0 0x7fc2c80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:16.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.006+0000 7fc2cd19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c815d230 0x7fc2c815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:16.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.006+0000 7fc2cd19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c815dbb0 con 0x7fc2c8159420 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.006+0000 7fc2cd19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2c80b11d0 con 0x7fc2c81597f0 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.006+0000 7fc2cd19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80b14b0 con 0x7fc2c815d230 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.006+0000 7fc2c77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc2c81597f0 0x7fc2c80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2c77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc2c81597f0 0x7fc2c80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53460/0 (socket says 172.21.15.119:53460) 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2c77fe700 1 -- 172.21.15.119:0/86529158 learned_addr learned my addr 172.21.15.119:0/86529158 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2cc99c700 1 --2- 172.21.15.119:0/86529158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c815d230 0x7fc2c815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2c77fe700 1 -- 172.21.15.119:0/86529158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c815d230 msgr2=0x7fc2c815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2c77fe700 1 --2- 172.21.15.119:0/86529158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c815d230 0x7fc2c815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2c77fe700 1 -- 172.21.15.119:0/86529158 >> v1:172.21.15.184:6789/0 conn(0x7fc2c8159420 legacy=0x7fc2c80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2c77fe700 1 -- 172.21.15.119:0/86529158 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c80b1790 con 0x7fc2c81597f0 2023-10-08T01:11:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2cc99c700 1 --2- 172.21.15.119:0/86529158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c815d230 0x7fc2c815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2c77fe700 1 --2- 172.21.15.119:0/86529158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c81597f0 0x7fc2c80b2270 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0099b0 tx=0x7fc2bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27e2a14c000d302e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.007+0000 7fc2c6ffd700 1 -- 172.21.15.119:0/86529158 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00b860 con 0x7fc2c81597f0 2023-10-08T01:11:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2c6ffd700 1 -- 172.21.15.119:0/86529158 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc00b9c0 con 0x7fc2c81597f0 2023-10-08T01:11:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2c6ffd700 1 -- 172.21.15.119:0/86529158 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc00bd60 con 0x7fc2c81597f0 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2cd19d700 1 -- 172.21.15.119:0/86529158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c81597f0 msgr2=0x7fc2c80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/86529158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c81597f0 0x7fc2c80b2270 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc0099b0 tx=0x7fc2bc009250 comp rx=0 tx=0).stop 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2cd19d700 1 -- 172.21.15.119:0/86529158 shutdown_connections 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/86529158 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c815d230 0x7fc2c815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/86529158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c81597f0 0x7fc2c80b2270 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2cd19d700 1 -- 172.21.15.119:0/86529158 >> 172.21.15.119:0/86529158 conn(0x7fc2c80d0660 msgr2=0x7fc2c80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2cd19d700 1 -- 172.21.15.119:0/86529158 shutdown_connections 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.008+0000 7fc2cd19d700 1 -- 172.21.15.119:0/86529158 wait complete. 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.009+0000 7fc2cd19d700 1 Processor -- start 2023-10-08T01:11:16.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.010+0000 7fc2cd19d700 1 -- start start 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.010+0000 7fc2cd19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c8159420 0x7fc2c815c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.010+0000 7fc2cd19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c81597f0 0x7fc2c8156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cd19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c815d230 0x7fc2c80c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cd19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c815cae0 con 0x7fc2c815d230 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cd19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80c61b0 con 0x7fc2c8159420 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cd19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c80c55a0 con 0x7fc2c81597f0 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c815d230 0x7fc2c80c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c815d230 0x7fc2c80c7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53464/0 (socket says 172.21.15.119:53464) 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 -- 172.21.15.119:0/1093943285 learned_addr learned my addr 172.21.15.119:0/1093943285 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2c7fff700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c8159420 0x7fc2c815c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:16.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2c77fe700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c81597f0 0x7fc2c8156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 -- 172.21.15.119:0/1093943285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c81597f0 msgr2=0x7fc2c8156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c81597f0 0x7fc2c8156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 -- 172.21.15.119:0/1093943285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c8159420 msgr2=0x7fc2c815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c8159420 0x7fc2c815c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 -- 172.21.15.119:0/1093943285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc0094f0 con 0x7fc2c815d230 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.011+0000 7fc2cc99c700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c815d230 0x7fc2c80c7130 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009fc0 tx=0x7fc2c00099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.012+0000 7fc2affff700 1 -- 172.21.15.119:0/1093943285 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000dbe0 con 0x7fc2c815d230 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.012+0000 7fc2affff700 1 -- 172.21.15.119:0/1093943285 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c000f040 con 0x7fc2c815d230 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.012+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c80c5880 con 0x7fc2c815d230 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.012+0000 7fc2affff700 1 -- 172.21.15.119:0/1093943285 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0015440 con 0x7fc2c815d230 2023-10-08T01:11:16.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.012+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c80c5e40 con 0x7fc2c815d230 2023-10-08T01:11:16.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.013+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c80c6540 con 0x7fc2c815d230 2023-10-08T01:11:16.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.014+0000 7fc2affff700 1 -- 172.21.15.119:0/1093943285 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2c001e3b0 con 0x7fc2c815d230 2023-10-08T01:11:16.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.014+0000 7fc2affff700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b4066f30 0x7fc2b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.015+0000 7fc2affff700 1 -- 172.21.15.119:0/1093943285 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5847+0+0 (secure 0 0 0) 0x7fc2c0084a70 con 0x7fc2c815d230 2023-10-08T01:11:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.015+0000 7fc2c7fff700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b4066f30 0x7fc2b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:16.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.015+0000 7fc2c7fff700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b4066f30 0x7fc2b40693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80039f0 tx=0x7fc2b8007ec0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:16.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.018+0000 7fc2affff700 1 -- 172.21.15.119:0/1093943285 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2c80c6540 con 0x7fc2c815d230 2023-10-08T01:11:16.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:16.233+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc2c80c6540 con 0x7fc2c815d230 2023-10-08T01:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.130+0000 7fc2affff700 1 -- 172.21.15.119:0/1093943285 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v73) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc2c80c6540 con 0x7fc2c815d230 2023-10-08T01:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b4066f30 msgr2=0x7fc2b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b4066f30 0x7fc2b40693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc2b80039f0 tx=0x7fc2b8007ec0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c815d230 msgr2=0x7fc2c80c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c815d230 0x7fc2c80c7130 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009fc0 tx=0x7fc2c00099b0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 shutdown_connections 2023-10-08T01:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b4066f30 0x7fc2b40693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c815d230 0x7fc2c80c7130 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c8159420 0x7fc2c815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 --2- 172.21.15.119:0/1093943285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c81597f0 0x7fc2c8156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.133+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 >> 172.21.15.119:0/1093943285 conn(0x7fc2c80d0660 msgr2=0x7fc2c80b20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.134+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 shutdown_connections 2023-10-08T01:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.134+0000 7fc2cd19d700 1 -- 172.21.15.119:0/1093943285 wait complete. 2023-10-08T01:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T01:11:17.147 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:11:17.153 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:11:17.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T01:11:17.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.379+0000 7f32d97ff700 1 Processor -- start 2023-10-08T01:11:17.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.380+0000 7f32d97ff700 1 -- start start 2023-10-08T01:11:17.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d97ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f32d40cbf30 0x7f32d40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d97ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40cc300 con 0x7f32d40ccf90 2023-10-08T01:11:17.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d97ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d40ad370 con 0x7f32d40b1420 2023-10-08T01:11:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d97ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32d4156bf0 con 0x7f32d40cbf30 2023-10-08T01:11:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d2ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f32d40cbf30 0x7f32d40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32cbfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f32d40ccf90 0x7f32d40b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60004/0 (socket says 172.21.15.119:60004) 2023-10-08T01:11:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32cbfff700 1 -- 172.21.15.119:0/739580888 learned_addr learned my addr 172.21.15.119:0/739580888 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d2ffd700 1 -- 172.21.15.119:0/739580888 >> v1:172.21.15.184:6789/0 conn(0x7f32d40b1420 legacy=0x7f32d40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d2ffd700 1 -- 172.21.15.119:0/739580888 >> v1:172.21.15.119:6789/0 conn(0x7f32d40ccf90 legacy=0x7f32d40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d2ffd700 1 -- 172.21.15.119:0/739580888 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d40ad650 con 0x7f32d40cbf30 2023-10-08T01:11:17.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.381+0000 7f32d27fc700 1 -- 172.21.15.119:0/739580888 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268104424 0 0) 0x7f32d40cc300 con 0x7f32d40ccf90 2023-10-08T01:11:17.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.382+0000 7f32d2ffd700 1 --2- 172.21.15.119:0/739580888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40cbf30 0x7f32d40ae410 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f32c40098e0 tx=0x7f32c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6607c1b17697fa9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:17.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.382+0000 7f32d27fc700 1 -- 172.21.15.119:0/739580888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c4009e80 con 0x7f32d40cbf30 2023-10-08T01:11:17.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.382+0000 7f32d27fc700 1 -- 172.21.15.119:0/739580888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c400b820 con 0x7f32d40cbf30 2023-10-08T01:11:17.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.382+0000 7f32d27fc700 1 -- 172.21.15.119:0/739580888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c400bba0 con 0x7f32d40cbf30 2023-10-08T01:11:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.382+0000 7f32d97ff700 1 -- 172.21.15.119:0/739580888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40cbf30 msgr2=0x7f32d40ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.382+0000 7f32d97ff700 1 --2- 172.21.15.119:0/739580888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40cbf30 0x7f32d40ae410 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f32c40098e0 tx=0x7f32c40091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.383+0000 7f32d97ff700 1 -- 172.21.15.119:0/739580888 shutdown_connections 2023-10-08T01:11:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.383+0000 7f32d97ff700 1 --2- 172.21.15.119:0/739580888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40cbf30 0x7f32d40ae410 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.383+0000 7f32d97ff700 1 -- 172.21.15.119:0/739580888 >> 172.21.15.119:0/739580888 conn(0x7f32d406a320 msgr2=0x7f32d40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.383+0000 7f32d97ff700 1 -- 172.21.15.119:0/739580888 shutdown_connections 2023-10-08T01:11:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.383+0000 7f32d97ff700 1 -- 172.21.15.119:0/739580888 wait complete. 2023-10-08T01:11:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.384+0000 7f32d97ff700 1 Processor -- start 2023-10-08T01:11:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.384+0000 7f32d97ff700 1 -- start start 2023-10-08T01:11:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d97ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d40b1420 0x7f32d4144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d97ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d40cbf30 0x7f32d41596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d97ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40ccf90 0x7f32d4159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d97ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d416bba0 con 0x7f32d40ccf90 2023-10-08T01:11:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d97ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d416af90 con 0x7f32d40b1420 2023-10-08T01:11:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d97ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d416b270 con 0x7f32d40cbf30 2023-10-08T01:11:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40ccf90 0x7f32d4159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40ccf90 0x7f32d4159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53478/0 (socket says 172.21.15.119:53478) 2023-10-08T01:11:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.385+0000 7f32d37fe700 1 -- 172.21.15.119:0/738003367 learned_addr learned my addr 172.21.15.119:0/738003367 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:17.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d2ffd700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d40b1420 0x7f32d4144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32cbfff700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d40cbf30 0x7f32d41596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d37fe700 1 -- 172.21.15.119:0/738003367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d40cbf30 msgr2=0x7f32d41596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d37fe700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d40cbf30 0x7f32d41596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d37fe700 1 -- 172.21.15.119:0/738003367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d40b1420 msgr2=0x7f32d4144ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d37fe700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d40b1420 0x7f32d4144ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d37fe700 1 -- 172.21.15.119:0/738003367 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c4009420 con 0x7f32d40ccf90 2023-10-08T01:11:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d37fe700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40ccf90 0x7f32d4159c20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00bbb0 tx=0x7f32cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32caffd700 1 -- 172.21.15.119:0/738003367 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc0092e0 con 0x7f32d40ccf90 2023-10-08T01:11:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32caffd700 1 -- 172.21.15.119:0/738003367 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc00c6f0 con 0x7f32d40ccf90 2023-10-08T01:11:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d416b550 con 0x7f32d40ccf90 2023-10-08T01:11:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32caffd700 1 -- 172.21.15.119:0/738003367 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc00ca90 con 0x7f32d40ccf90 2023-10-08T01:11:17.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.386+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d4167960 con 0x7f32d40ccf90 2023-10-08T01:11:17.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.387+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d416bd50 con 0x7f32d40ccf90 2023-10-08T01:11:17.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.388+0000 7f32caffd700 1 -- 172.21.15.119:0/738003367 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32cc012070 con 0x7f32d40ccf90 2023-10-08T01:11:17.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.390+0000 7f32caffd700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.390+0000 7f32caffd700 1 -- 172.21.15.119:0/738003367 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f32cc083fd0 con 0x7f32d40ccf90 2023-10-08T01:11:17.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.390+0000 7f32d2ffd700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.390+0000 7f32d2ffd700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32b4066f30 0x7f32b40693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f32c40030c0 tx=0x7f32c4016000 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:17.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.394+0000 7f32caffd700 1 -- 172.21.15.119:0/738003367 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32d416bd50 con 0x7f32d40ccf90 2023-10-08T01:11:17.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.658+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f32d404c3c0 con 0x7f32d40ccf90 2023-10-08T01:11:17.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.659+0000 7f32caffd700 1 -- 172.21.15.119:0/738003367 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f32cc051370 con 0x7f32d40ccf90 2023-10-08T01:11:17.660 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:17.660 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:11:17.660 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T01:11:17.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.662 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T01:11:17.662 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T01:11:17.662 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:17.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:17.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:17.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:17.665 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.665 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T01:11:17.665 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T01:11:17.666 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:17.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:17.666 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.668 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T01:11:17.668 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T01:11:17.668 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:17.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:17.668 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.669 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T01:11:17.669 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T01:11:17.669 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.669 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:17.669 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:17.669 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.672 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.673 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T01:11:17.673 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T01:11:17.673 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.673 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.674 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:17.675 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T01:11:17.675 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T01:11:17.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.675 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T01:11:17.675 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T01:11:17.675 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T01:11:17.676 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T01:11:17.677 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:17.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:11:17.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:11:17.678 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:11:17.678 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.678 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.678 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.678 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.679 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:17.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:11:17.681 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:11:17.681 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:17.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:17.681 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T01:11:17.681 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:11:17.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32b4066f30 msgr2=0x7f32b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32b4066f30 0x7f32b40693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f32c40030c0 tx=0x7f32c4016000 comp rx=0 tx=0).stop 2023-10-08T01:11:17.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40ccf90 msgr2=0x7f32d4159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40ccf90 0x7f32d4159c20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00bbb0 tx=0x7f32cc009840 comp rx=0 tx=0).stop 2023-10-08T01:11:17.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 shutdown_connections 2023-10-08T01:11:17.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32b4066f30 0x7f32b40693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32d40ccf90 0x7f32d4159c20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32d40b1420 0x7f32d4144ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 --2- 172.21.15.119:0/738003367 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32d40cbf30 0x7f32d41596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 >> 172.21.15.119:0/738003367 conn(0x7f32d406a320 msgr2=0x7f32d40c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:17.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 shutdown_connections 2023-10-08T01:11:17.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.661+0000 7f32d97ff700 1 -- 172.21.15.119:0/738003367 wait complete. 2023-10-08T01:11:17.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:17.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.869+0000 7f87e7fff700 1 Processor -- start 2023-10-08T01:11:17.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.869+0000 7f87e7fff700 1 -- start start 2023-10-08T01:11:17.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.870+0000 7f87e7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00cd010 0x7f87e00ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.870+0000 7f87e7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87e00ac770 0x7f87e00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.870+0000 7f87e7fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e00ad930 con 0x7f87e00ccc40 2023-10-08T01:11:17.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.870+0000 7f87e7fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e00acd20 con 0x7f87e00ac770 2023-10-08T01:11:17.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.870+0000 7f87e7fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e00ad000 con 0x7f87e00cd010 2023-10-08T01:11:17.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.871+0000 7f87e659c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87e00ac770 0x7f87e00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.871+0000 7f87e559a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00cd010 0x7f87e00ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.871+0000 7f87e659c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87e00ac770 0x7f87e00b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:34708/0 (socket says 172.21.15.119:34708) 2023-10-08T01:11:17.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.871+0000 7f87e659c700 1 -- 172.21.15.119:0/161588844 learned_addr learned my addr 172.21.15.119:0/161588844 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:17.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.871+0000 7f87e559a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00cd010 0x7f87e00ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:57452/0 (socket says 172.21.15.119:57452) 2023-10-08T01:11:17.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.871+0000 7f87e5d9b700 1 --1- 172.21.15.119:0/161588844 >> v1:172.21.15.184:6789/0 conn(0x7f87e00ccc40 0x7f87e0155c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:45960/0 (socket says 172.21.15.119:45960) 2023-10-08T01:11:17.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e659c700 1 -- 172.21.15.119:0/161588844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00cd010 msgr2=0x7f87e00ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:17.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e659c700 1 --2- 172.21.15.119:0/161588844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00cd010 0x7f87e00ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e659c700 1 -- 172.21.15.119:0/161588844 >> v1:172.21.15.184:6789/0 conn(0x7f87e00ccc40 legacy=0x7f87e0155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e659c700 1 -- 172.21.15.119:0/161588844 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e00ad2e0 con 0x7f87e00ac770 2023-10-08T01:11:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e559a700 1 --2- 172.21.15.119:0/161588844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00cd010 0x7f87e00ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e4d99700 1 -- 172.21.15.119:0/161588844 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318848061 0 0) 0x7f87e00ad930 con 0x7f87e00ccc40 2023-10-08T01:11:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e659c700 1 --2- 172.21.15.119:0/161588844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00ac770 0x7f87e00b0300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87d800d1a0 tx=0x7f87d800ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2febeb7713b0a9da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e4d99700 1 -- 172.21.15.119:0/161588844 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d8010070 con 0x7f87e00ac770 2023-10-08T01:11:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e4d99700 1 -- 172.21.15.119:0/161588844 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d800de10 con 0x7f87e00ac770 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.872+0000 7f87e4d99700 1 -- 172.21.15.119:0/161588844 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d800b320 con 0x7f87e00ac770 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.873+0000 7f87e7fff700 1 -- 172.21.15.119:0/161588844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00ac770 msgr2=0x7f87e00b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.873+0000 7f87e7fff700 1 --2- 172.21.15.119:0/161588844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00ac770 0x7f87e00b0300 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87d800d1a0 tx=0x7f87d800ba50 comp rx=0 tx=0).stop 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.873+0000 7f87e7fff700 1 -- 172.21.15.119:0/161588844 shutdown_connections 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.873+0000 7f87e7fff700 1 --2- 172.21.15.119:0/161588844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00ac770 0x7f87e00b0300 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.873+0000 7f87e7fff700 1 --2- 172.21.15.119:0/161588844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00cd010 0x7f87e00ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.873+0000 7f87e7fff700 1 -- 172.21.15.119:0/161588844 >> 172.21.15.119:0/161588844 conn(0x7f87e006a320 msgr2=0x7f87e00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.873+0000 7f87e7fff700 1 -- 172.21.15.119:0/161588844 shutdown_connections 2023-10-08T01:11:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.873+0000 7f87e7fff700 1 -- 172.21.15.119:0/161588844 wait complete. 2023-10-08T01:11:17.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.874+0000 7f87e7fff700 1 Processor -- start 2023-10-08T01:11:17.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.874+0000 7f87e7fff700 1 -- start start 2023-10-08T01:11:17.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.875+0000 7f87e7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e00ac770 0x7f87e013c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.875+0000 7f87e7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00ccc40 0x7f87e015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.875+0000 7f87e7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00cd010 0x7f87e00aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.875+0000 7f87e7fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e0151490 con 0x7f87e00ac770 2023-10-08T01:11:17.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.875+0000 7f87e7fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e0151770 con 0x7f87e00cd010 2023-10-08T01:11:17.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.875+0000 7f87e7fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e00cb150 con 0x7f87e00ccc40 2023-10-08T01:11:17.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.875+0000 7f87e5d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e00ac770 0x7f87e013c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e5d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e00ac770 0x7f87e013c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53484/0 (socket says 172.21.15.119:53484) 2023-10-08T01:11:17.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e5d9b700 1 -- 172.21.15.119:0/2900541295 learned_addr learned my addr 172.21.15.119:0/2900541295 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:17.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e559a700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00ccc40 0x7f87e015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e659c700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00cd010 0x7f87e00aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e5d9b700 1 -- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00ccc40 msgr2=0x7f87e015cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e5d9b700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00ccc40 0x7f87e015cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e5d9b700 1 -- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00cd010 msgr2=0x7f87e00aa450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e5d9b700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00cd010 0x7f87e00aa450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e5d9b700 1 -- 172.21.15.119:0/2900541295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0009420 con 0x7f87e00ac770 2023-10-08T01:11:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e559a700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00ccc40 0x7f87e015cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e5d9b700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e00ac770 0x7f87e013c680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0011e0 tx=0x7f87dc007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87d5ffb700 1 -- 172.21.15.119:0/2900541295 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc005a70 con 0x7f87e00ac770 2023-10-08T01:11:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87d5ffb700 1 -- 172.21.15.119:0/2900541295 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc005bd0 con 0x7f87e00ac770 2023-10-08T01:11:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d800bcf0 con 0x7f87e00ac770 2023-10-08T01:11:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.876+0000 7f87d5ffb700 1 -- 172.21.15.119:0/2900541295 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0070d0 con 0x7f87e00ac770 2023-10-08T01:11:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.877+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e00cb710 con 0x7f87e00ac770 2023-10-08T01:11:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.878+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e004c3c0 con 0x7f87e00ac770 2023-10-08T01:11:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.878+0000 7f87d5ffb700 1 -- 172.21.15.119:0/2900541295 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87dc007480 con 0x7f87e00ac770 2023-10-08T01:11:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.879+0000 7f87d5ffb700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.879+0000 7f87d5ffb700 1 -- 172.21.15.119:0/2900541295 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f87dc083850 con 0x7f87e00ac770 2023-10-08T01:11:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.879+0000 7f87e559a700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.880+0000 7f87e559a700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87c4066f30 0x7f87c40693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f87d00093f0 tx=0x7f87d0008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:17.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:17.883+0000 7f87d5ffb700 1 -- 172.21.15.119:0/2900541295 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87dc04b5a0 con 0x7f87e00ac770 2023-10-08T01:11:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.074+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87e00ad5c0 con 0x7f87e00ac770 2023-10-08T01:11:18.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.075+0000 7f87d5ffb700 1 -- 172.21.15.119:0/2900541295 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+12779 (secure 0 0 0) 0x7f87dc017020 con 0x7f87e00ac770 2023-10-08T01:11:18.077 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:18.077 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:17.127452+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:18.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87c4066f30 msgr2=0x7f87c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87c4066f30 0x7f87c40693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f87d00093f0 tx=0x7f87d0008040 comp rx=0 tx=0).stop 2023-10-08T01:11:18.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e00ac770 msgr2=0x7f87e013c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e00ac770 0x7f87e013c680 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f87dc0011e0 tx=0x7f87dc007e10 comp rx=0 tx=0).stop 2023-10-08T01:11:18.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 shutdown_connections 2023-10-08T01:11:18.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87e00cd010 0x7f87e00aa450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87e00ac770 0x7f87e013c680 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 --2- 172.21.15.119:0/2900541295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87e00ccc40 0x7f87e015cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.077+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 >> 172.21.15.119:0/2900541295 conn(0x7f87e006a320 msgr2=0x7f87e00a98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:18.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.078+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 shutdown_connections 2023-10-08T01:11:18.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.078+0000 7f87e7fff700 1 -- 172.21.15.119:0/2900541295 wait complete. 2023-10-08T01:11:18.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T01:11:18.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:11:18.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.278+0000 7f7cd1f9d700 1 Processor -- start 2023-10-08T01:11:18.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.279+0000 7f7cd1f9d700 1 -- start start 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd1f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7ccc153900 0x7f7ccc0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd1f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 0x7f7ccc06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd1f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ccc157130 0x7f7ccc06a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd1f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc06e150 con 0x7f7ccc157130 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd1f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ccc0c7800 con 0x7f7ccc153900 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd1f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0c7ae0 con 0x7f7ccc0c9db0 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd179c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ccc157130 0x7f7ccc06a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd179c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ccc157130 0x7f7ccc06a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53486/0 (socket says 172.21.15.119:53486) 2023-10-08T01:11:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.280+0000 7f7cd179c700 1 -- 172.21.15.119:0/2704320368 learned_addr learned my addr 172.21.15.119:0/2704320368 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:18.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7ccbfff700 1 --2- 172.21.15.119:0/2704320368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 0x7f7ccc06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7cd0f9b700 1 --2- 172.21.15.119:0/2704320368 >> v2:172.21.15.184:3300/0 conn(0x7f7ccc153900 0x7f7ccc0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7cd179c700 1 -- 172.21.15.119:0/2704320368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 msgr2=0x7f7ccc06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:18.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7cd179c700 1 --2- 172.21.15.119:0/2704320368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 0x7f7ccc06a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7cd179c700 1 -- 172.21.15.119:0/2704320368 >> v2:172.21.15.184:3300/0 conn(0x7f7ccc153900 msgr2=0x7f7ccc0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:18.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7cd179c700 1 --2- 172.21.15.119:0/2704320368 >> v2:172.21.15.184:3300/0 conn(0x7f7ccc153900 0x7f7ccc0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7cd179c700 1 -- 172.21.15.119:0/2704320368 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ccc0c7dc0 con 0x7f7ccc157130 2023-10-08T01:11:18.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7cd179c700 1 --2- 172.21.15.119:0/2704320368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc06a540 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc06aed0 tx=0x7f7cc4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8145d06717d542eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7ccb7fe700 1 -- 172.21.15.119:0/2704320368 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc400e070 con 0x7f7ccc157130 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7ccb7fe700 1 -- 172.21.15.119:0/2704320368 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc400ba50 con 0x7f7ccc157130 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.281+0000 7f7ccb7fe700 1 -- 172.21.15.119:0/2704320368 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40092e0 con 0x7f7ccc157130 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/2704320368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 msgr2=0x7f7ccc06a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/2704320368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc06a540 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc06aed0 tx=0x7f7cc4009840 comp rx=0 tx=0).stop 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/2704320368 shutdown_connections 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/2704320368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc06a540 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/2704320368 >> v2:172.21.15.184:3300/0 conn(0x7f7ccc153900 0x7f7ccc0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/2704320368 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 0x7f7ccc06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/2704320368 >> 172.21.15.119:0/2704320368 conn(0x7f7ccc0cc1e0 msgr2=0x7f7ccc0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/2704320368 shutdown_connections 2023-10-08T01:11:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.282+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/2704320368 wait complete. 2023-10-08T01:11:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.283+0000 7f7cd1f9d700 1 Processor -- start 2023-10-08T01:11:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.284+0000 7f7cd1f9d700 1 -- start start 2023-10-08T01:11:18.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.284+0000 7f7cd1f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 0x7f7ccc148a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.284+0000 7f7cd1f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ccc153900 0x7f7ccc14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.284+0000 7f7cd1f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.284+0000 7f7cd1f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc14d5a0 con 0x7f7ccc157130 2023-10-08T01:11:18.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.284+0000 7f7cd1f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0ac0d0 con 0x7f7ccc153900 2023-10-08T01:11:18.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.284+0000 7f7cd1f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ccc0abda0 con 0x7f7ccc0c9db0 2023-10-08T01:11:18.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc144000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53492/0 (socket says 172.21.15.119:53492) 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 -- 172.21.15.119:0/1437895537 learned_addr learned my addr 172.21.15.119:0/1437895537 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 -- 172.21.15.119:0/1437895537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 msgr2=0x7f7ccc148a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 0x7f7ccc148a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 -- 172.21.15.119:0/1437895537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ccc153900 msgr2=0x7f7ccc14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ccc153900 0x7f7ccc14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 -- 172.21.15.119:0/1437895537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4009ae0 con 0x7f7ccc157130 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.285+0000 7f7cd179c700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc144000 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc06ce80 tx=0x7f7cc4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.286+0000 7f7cc8ff9700 1 -- 172.21.15.119:0/1437895537 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc40032f0 con 0x7f7ccc157130 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.286+0000 7f7cc8ff9700 1 -- 172.21.15.119:0/1437895537 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc4008e80 con 0x7f7ccc157130 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.286+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ccc14cd00 con 0x7f7ccc157130 2023-10-08T01:11:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.286+0000 7f7cc8ff9700 1 -- 172.21.15.119:0/1437895537 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc401c660 con 0x7f7ccc157130 2023-10-08T01:11:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.286+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ccc14d340 con 0x7f7ccc157130 2023-10-08T01:11:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.287+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ccc06c450 con 0x7f7ccc157130 2023-10-08T01:11:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.288+0000 7f7cc8ff9700 1 -- 172.21.15.119:0/1437895537 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7cc4003040 con 0x7f7ccc157130 2023-10-08T01:11:18.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.288+0000 7f7cc8ff9700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7cb0066f30 0x7f7cb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.289+0000 7f7cc8ff9700 1 -- 172.21.15.119:0/1437895537 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f7cc4085ca0 con 0x7f7ccc157130 2023-10-08T01:11:18.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.289+0000 7f7cd0f9b700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7cb0066f30 0x7f7cb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.289+0000 7f7cd0f9b700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7cb0066f30 0x7f7cb00693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009a60 tx=0x7f7cc0008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:18.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.293+0000 7f7cc8ff9700 1 -- 172.21.15.119:0/1437895537 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7cc40530e0 con 0x7f7ccc157130 2023-10-08T01:11:18.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.485+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ccc0ac240 con 0x7f7ccc157130 2023-10-08T01:11:18.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.486+0000 7f7cc8ff9700 1 -- 172.21.15.119:0/1437895537 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7cc4009940 con 0x7f7ccc157130 2023-10-08T01:11:18.487 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:11:18.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7cb0066f30 msgr2=0x7f7cb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7cb0066f30 0x7f7cb00693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0009a60 tx=0x7f7cc0008040 comp rx=0 tx=0).stop 2023-10-08T01:11:18.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 msgr2=0x7f7ccc144000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc144000 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc06ce80 tx=0x7f7cc4018000 comp rx=0 tx=0).stop 2023-10-08T01:11:18.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 shutdown_connections 2023-10-08T01:11:18.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7cb0066f30 0x7f7cb00693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ccc0c9db0 0x7f7ccc148a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ccc157130 0x7f7ccc144000 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 --2- 172.21.15.119:0/1437895537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ccc153900 0x7f7ccc14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 >> 172.21.15.119:0/1437895537 conn(0x7f7ccc0cc1e0 msgr2=0x7f7ccc06e890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:18.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 shutdown_connections 2023-10-08T01:11:18.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.488+0000 7f7cd1f9d700 1 -- 172.21.15.119:0/1437895537 wait complete. 2023-10-08T01:11:18.507 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T01:11:18.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.687+0000 7fad749ff700 1 Processor -- start 2023-10-08T01:11:18.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.688+0000 7fad749ff700 1 -- start start 2023-10-08T01:11:18.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.689+0000 7fad749ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad700b1420 0x7fad700ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.689+0000 7fad749ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad700cc300 con 0x7fad700ccf90 2023-10-08T01:11:18.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.689+0000 7fad749ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad700ad370 con 0x7fad700cbf30 2023-10-08T01:11:18.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.689+0000 7fad749ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad70156bf0 con 0x7fad700b1420 2023-10-08T01:11:18.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.689+0000 7fad6ed9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fad700ccf90 0x7fad700c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60020/0 (socket says 172.21.15.119:60020) 2023-10-08T01:11:18.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.689+0000 7fad6ed9d700 1 -- 172.21.15.119:0/2560402501 learned_addr learned my addr 172.21.15.119:0/2560402501 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.689+0000 7fad67fff700 1 --2- 172.21.15.119:0/2560402501 >> v2:172.21.15.119:3300/0 conn(0x7fad700b1420 0x7fad700ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad6dd9b700 1 -- 172.21.15.119:0/2560402501 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350490080 0 0) 0x7fad700cc300 con 0x7fad700ccf90 2023-10-08T01:11:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad6dd9b700 1 -- 172.21.15.119:0/2560402501 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad5c003570 con 0x7fad700ccf90 2023-10-08T01:11:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad67fff700 1 -- 172.21.15.119:0/2560402501 >> v1:172.21.15.184:6789/0 conn(0x7fad700cbf30 legacy=0x7fad700b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad67fff700 1 -- 172.21.15.119:0/2560402501 >> v1:172.21.15.119:6789/0 conn(0x7fad700ccf90 legacy=0x7fad700c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad67fff700 1 -- 172.21.15.119:0/2560402501 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad700ad650 con 0x7fad700b1420 2023-10-08T01:11:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad67fff700 1 --2- 172.21.15.119:0/2560402501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad700ae410 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fad580098e0 tx=0x7fad580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5fbf59f29c3306 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:18.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad6dd9b700 1 -- 172.21.15.119:0/2560402501 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5800b640 con 0x7fad700b1420 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad6dd9b700 1 -- 172.21.15.119:0/2560402501 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5800b7a0 con 0x7fad700b1420 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.690+0000 7fad6dd9b700 1 -- 172.21.15.119:0/2560402501 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5800bb40 con 0x7fad700b1420 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.691+0000 7fad749ff700 1 -- 172.21.15.119:0/2560402501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 msgr2=0x7fad700ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.691+0000 7fad749ff700 1 --2- 172.21.15.119:0/2560402501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad700ae410 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fad580098e0 tx=0x7fad580091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.691+0000 7fad749ff700 1 -- 172.21.15.119:0/2560402501 shutdown_connections 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.691+0000 7fad749ff700 1 --2- 172.21.15.119:0/2560402501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad700ae410 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.691+0000 7fad749ff700 1 -- 172.21.15.119:0/2560402501 >> 172.21.15.119:0/2560402501 conn(0x7fad7006a320 msgr2=0x7fad700cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.691+0000 7fad749ff700 1 -- 172.21.15.119:0/2560402501 shutdown_connections 2023-10-08T01:11:18.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.691+0000 7fad749ff700 1 -- 172.21.15.119:0/2560402501 wait complete. 2023-10-08T01:11:18.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.692+0000 7fad749ff700 1 Processor -- start 2023-10-08T01:11:18.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.692+0000 7fad749ff700 1 -- start start 2023-10-08T01:11:18.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.693+0000 7fad749ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad701556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.693+0000 7fad749ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad700cbf30 0x7fad700a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.693+0000 7fad749ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad700ccf90 0x7fad700c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.693+0000 7fad749ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c7af0 con 0x7fad700b1420 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.693+0000 7fad749ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c6c00 con 0x7fad700ccf90 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.693+0000 7fad749ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad700c6ee0 con 0x7fad700cbf30 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad701556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad701556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53504/0 (socket says 172.21.15.119:53504) 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 -- 172.21.15.119:0/393938604 learned_addr learned my addr 172.21.15.119:0/393938604 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:18.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad67fff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad700cbf30 0x7fad700a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6ed9d700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad700ccf90 0x7fad700c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 -- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad700cbf30 msgr2=0x7fad700a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad700cbf30 0x7fad700a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 -- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad700ccf90 msgr2=0x7fad700c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad700ccf90 0x7fad700c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 -- 172.21.15.119:0/393938604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad58009420 con 0x7fad700b1420 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad67fff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad700cbf30 0x7fad700a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6ed9d700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad700ccf90 0x7fad700c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:18.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.694+0000 7fad6e59c700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad701556b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c590 tx=0x7fad6000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:18.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.698+0000 7fad667fc700 1 -- 172.21.15.119:0/393938604 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60011590 con 0x7fad700b1420 2023-10-08T01:11:18.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.698+0000 7fad667fc700 1 -- 172.21.15.119:0/393938604 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad600116f0 con 0x7fad700b1420 2023-10-08T01:11:18.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.698+0000 7fad667fc700 1 -- 172.21.15.119:0/393938604 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad60011ab0 con 0x7fad700b1420 2023-10-08T01:11:18.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.698+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad700c71c0 con 0x7fad700b1420 2023-10-08T01:11:18.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.698+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad700c7780 con 0x7fad700b1420 2023-10-08T01:11:18.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.699+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad70155bf0 con 0x7fad700b1420 2023-10-08T01:11:18.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.700+0000 7fad667fc700 1 -- 172.21.15.119:0/393938604 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad6001b3b0 con 0x7fad700b1420 2023-10-08T01:11:18.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.701+0000 7fad667fc700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fad50066fb0 0x7fad50069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:18.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.701+0000 7fad667fc700 1 -- 172.21.15.119:0/393938604 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fad600839b0 con 0x7fad700b1420 2023-10-08T01:11:18.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.701+0000 7fad67fff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fad50066fb0 0x7fad50069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:18.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.702+0000 7fad67fff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fad50066fb0 0x7fad50069460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad700b1820 tx=0x7fad58017000 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:18.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.705+0000 7fad667fc700 1 -- 172.21.15.119:0/393938604 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad60050d50 con 0x7fad700b1420 2023-10-08T01:11:18.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.897+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fad7004c3c0 con 0x7fad700b1420 2023-10-08T01:11:18.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.897+0000 7fad667fc700 1 -- 172.21.15.119:0/393938604 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) v1 ==== 84+0+65 (secure 0 0 0) 0x7fad60017020 con 0x7fad700b1420 2023-10-08T01:11:18.898 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:18.898 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:11:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T01:11:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T01:11:18.899 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:18.899 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:11:18.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fad50066fb0 msgr2=0x7fad50069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fad50066fb0 0x7fad50069460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fad700b1820 tx=0x7fad58017000 comp rx=0 tx=0).stop 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 msgr2=0x7fad701556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad701556b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fad6000c590 tx=0x7fad6000c950 comp rx=0 tx=0).stop 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 shutdown_connections 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fad50066fb0 0x7fad50069460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad700ccf90 0x7fad700c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad700b1420 0x7fad701556b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 --2- 172.21.15.119:0/393938604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad700cbf30 0x7fad700a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 >> 172.21.15.119:0/393938604 conn(0x7fad7006a320 msgr2=0x7fad700c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 shutdown_connections 2023-10-08T01:11:18.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:18.899+0000 7fad749ff700 1 -- 172.21.15.119:0/393938604 wait complete. 2023-10-08T01:11:18.918 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T01:11:18.918 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T01:11:19.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.102+0000 7fd30b7ff700 1 Processor -- start 2023-10-08T01:11:19.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.102+0000 7fd30b7ff700 1 -- start start 2023-10-08T01:11:19.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.103+0000 7fd30b7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd30415a440 0x7fd3040acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.103+0000 7fd30b7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30415a9b0 con 0x7fd304068890 2023-10-08T01:11:19.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.103+0000 7fd30b7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd30415ac90 con 0x7fd3041563b0 2023-10-08T01:11:19.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.103+0000 7fd30b7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30415af70 con 0x7fd30415a440 2023-10-08T01:11:19.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd309d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd304068890 0x7fd3040c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60024/0 (socket says 172.21.15.119:60024) 2023-10-08T01:11:19.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd309d9c700 1 -- 172.21.15.119:0/2524623336 learned_addr learned my addr 172.21.15.119:0/2524623336 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd308d9a700 1 --2- 172.21.15.119:0/2524623336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd30415a440 0x7fd3040acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd2fbfff700 1 -- 172.21.15.119:0/2524623336 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948303978 0 0) 0x7fd30415a9b0 con 0x7fd304068890 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd2fbfff700 1 -- 172.21.15.119:0/2524623336 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f0003570 con 0x7fd304068890 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd2fbfff700 1 -- 172.21.15.119:0/2524623336 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184366359 0 0) 0x7fd30415ac90 con 0x7fd3041563b0 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd2fbfff700 1 -- 172.21.15.119:0/2524623336 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd30415a9b0 con 0x7fd3041563b0 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd308d9a700 1 -- 172.21.15.119:0/2524623336 >> v1:172.21.15.184:6789/0 conn(0x7fd3041563b0 legacy=0x7fd304156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd308d9a700 1 -- 172.21.15.119:0/2524623336 >> v1:172.21.15.119:6789/0 conn(0x7fd304068890 legacy=0x7fd3040c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.104+0000 7fd308d9a700 1 -- 172.21.15.119:0/2524623336 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30415b250 con 0x7fd30415a440 2023-10-08T01:11:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd308d9a700 1 --2- 172.21.15.119:0/2524623336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd30415a440 0x7fd3040acf30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009910 tx=0x7fd2f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=116393547b2d276a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd2fbfff700 1 -- 172.21.15.119:0/2524623336 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4011630 con 0x7fd30415a440 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd2fbfff700 1 -- 172.21.15.119:0/2524623336 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f4011790 con 0x7fd30415a440 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd2fbfff700 1 -- 172.21.15.119:0/2524623336 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4011b10 con 0x7fd30415a440 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd30b7ff700 1 -- 172.21.15.119:0/2524623336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd30415a440 msgr2=0x7fd3040acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd30b7ff700 1 --2- 172.21.15.119:0/2524623336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd30415a440 0x7fd3040acf30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009910 tx=0x7fd2f40091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd30b7ff700 1 -- 172.21.15.119:0/2524623336 shutdown_connections 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd30b7ff700 1 --2- 172.21.15.119:0/2524623336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd30415a440 0x7fd3040acf30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.105+0000 7fd30b7ff700 1 -- 172.21.15.119:0/2524623336 >> 172.21.15.119:0/2524623336 conn(0x7fd3040c8f90 msgr2=0x7fd3040c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.106+0000 7fd30b7ff700 1 -- 172.21.15.119:0/2524623336 shutdown_connections 2023-10-08T01:11:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.106+0000 7fd30b7ff700 1 -- 172.21.15.119:0/2524623336 wait complete. 2023-10-08T01:11:19.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.107+0000 7fd30b7ff700 1 Processor -- start 2023-10-08T01:11:19.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.107+0000 7fd30b7ff700 1 -- start start 2023-10-08T01:11:19.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30b7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd304068890 0x7fd304153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30b7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041563b0 0x7fd3040b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30b7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30415a440 0x7fd304158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30b7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040afee0 con 0x7fd304068890 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30b7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040af2d0 con 0x7fd30415a440 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30b7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040af5b0 con 0x7fd3041563b0 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd304068890 0x7fd304153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd304068890 0x7fd304153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53520/0 (socket says 172.21.15.119:53520) 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30959b700 1 -- 172.21.15.119:0/1949316195 learned_addr learned my addr 172.21.15.119:0/1949316195 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30959b700 1 -- 172.21.15.119:0/1949316195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041563b0 msgr2=0x7fd3040b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30959b700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041563b0 0x7fd3040b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30959b700 1 -- 172.21.15.119:0/1949316195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30415a440 msgr2=0x7fd304158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30959b700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30415a440 0x7fd304158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.108+0000 7fd30959b700 1 -- 172.21.15.119:0/1949316195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f4009420 con 0x7fd304068890 2023-10-08T01:11:19.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.109+0000 7fd30959b700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd304068890 0x7fd304153de0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c7c0 tx=0x7fd300007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:19.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.109+0000 7fd2f97fa700 1 -- 172.21.15.119:0/1949316195 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300010510 con 0x7fd304068890 2023-10-08T01:11:19.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.109+0000 7fd2f97fa700 1 -- 172.21.15.119:0/1949316195 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300010670 con 0x7fd304068890 2023-10-08T01:11:19.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.109+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3040af890 con 0x7fd304068890 2023-10-08T01:11:19.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.109+0000 7fd2f97fa700 1 -- 172.21.15.119:0/1949316195 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300010a30 con 0x7fd304068890 2023-10-08T01:11:19.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.109+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3040b0ff0 con 0x7fd304068890 2023-10-08T01:11:19.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.110+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3040718b0 con 0x7fd304068890 2023-10-08T01:11:19.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.111+0000 7fd2f97fa700 1 -- 172.21.15.119:0/1949316195 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd30001b400 con 0x7fd304068890 2023-10-08T01:11:19.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.112+0000 7fd2f97fa700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2e8066f30 0x7fd2e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.112+0000 7fd2f97fa700 1 -- 172.21.15.119:0/1949316195 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fd300084000 con 0x7fd304068890 2023-10-08T01:11:19.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.112+0000 7fd308d9a700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2e8066f30 0x7fd2e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:19.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.113+0000 7fd308d9a700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2e8066f30 0x7fd2e80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd3040b0840 tx=0x7fd2f4017000 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:19.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.116+0000 7fd2f97fa700 1 -- 172.21.15.119:0/1949316195 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd300051440 con 0x7fd304068890 2023-10-08T01:11:19.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.308+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd30406d6a0 con 0x7fd304068890 2023-10-08T01:11:19.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.710+0000 7fd2f97fa700 1 -- 172.21.15.119:0/1949316195 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd300054a60 con 0x7fd304068890 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2e8066f30 msgr2=0x7fd2e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2e8066f30 0x7fd2e80693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd3040b0840 tx=0x7fd2f4017000 comp rx=0 tx=0).stop 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd304068890 msgr2=0x7fd304153de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd304068890 0x7fd304153de0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c7c0 tx=0x7fd300007dc0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 shutdown_connections 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2e8066f30 0x7fd2e80693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd30415a440 0x7fd304158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd304068890 0x7fd304153de0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 --2- 172.21.15.119:0/1949316195 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd3041563b0 0x7fd3040b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 >> 172.21.15.119:0/1949316195 conn(0x7fd3040c8f90 msgr2=0x7fd304068c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:19.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 shutdown_connections 2023-10-08T01:11:19.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.714+0000 7fd30b7ff700 1 -- 172.21.15.119:0/1949316195 wait complete. 2023-10-08T01:11:19.717 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T01:11:19.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T01:11:19.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.919+0000 7fb203bff700 1 Processor -- start 2023-10-08T01:11:19.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.919+0000 7fb203bff700 1 -- start start 2023-10-08T01:11:19.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.920+0000 7fb203bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.920+0000 7fb203bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc15a6f0 con 0x7fb1fc1569f0 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb203bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc15a9d0 con 0x7fb1fc156620 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb203bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0c8d30 con 0x7fb1fc068aa0 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb20119a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb1fc1569f0 0x7fb1fc068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60038/0 (socket says 172.21.15.119:60038) 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb20119a700 1 -- 172.21.15.119:0/691974525 learned_addr learned my addr 172.21.15.119:0/691974525 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb20219c700 1 --2- 172.21.15.119:0/691974525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb200999700 1 -- 172.21.15.119:0/691974525 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369917057 0 0) 0x7fb1fc15a6f0 con 0x7fb1fc1569f0 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb200999700 1 -- 172.21.15.119:0/691974525 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e8003570 con 0x7fb1fc1569f0 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb200999700 1 -- 172.21.15.119:0/691974525 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748857809 0 0) 0x7fb1fc15a9d0 con 0x7fb1fc156620 2023-10-08T01:11:19.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.921+0000 7fb200999700 1 -- 172.21.15.119:0/691974525 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1fc15a6f0 con 0x7fb1fc156620 2023-10-08T01:11:19.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.922+0000 7fb20219c700 1 -- 172.21.15.119:0/691974525 >> v1:172.21.15.184:6789/0 conn(0x7fb1fc156620 legacy=0x7fb1fc0ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.922+0000 7fb20219c700 1 -- 172.21.15.119:0/691974525 >> v1:172.21.15.119:6789/0 conn(0x7fb1fc1569f0 legacy=0x7fb1fc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.922+0000 7fb20219c700 1 -- 172.21.15.119:0/691974525 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc15acb0 con 0x7fb1fc068aa0 2023-10-08T01:11:19.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.922+0000 7fb20219c700 1 --2- 172.21.15.119:0/691974525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c87f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009e90 tx=0x7fb1f4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95e2024cfa84768b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:19.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.922+0000 7fb200999700 1 -- 172.21.15.119:0/691974525 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400e070 con 0x7fb1fc068aa0 2023-10-08T01:11:19.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.922+0000 7fb200999700 1 -- 172.21.15.119:0/691974525 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f400bb80 con 0x7fb1fc068aa0 2023-10-08T01:11:19.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.922+0000 7fb200999700 1 -- 172.21.15.119:0/691974525 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4012430 con 0x7fb1fc068aa0 2023-10-08T01:11:19.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.923+0000 7fb203bff700 1 -- 172.21.15.119:0/691974525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 msgr2=0x7fb1fc0c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.923+0000 7fb203bff700 1 --2- 172.21.15.119:0/691974525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c87f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009e90 tx=0x7fb1f4009780 comp rx=0 tx=0).stop 2023-10-08T01:11:19.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.923+0000 7fb203bff700 1 -- 172.21.15.119:0/691974525 shutdown_connections 2023-10-08T01:11:19.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.923+0000 7fb203bff700 1 --2- 172.21.15.119:0/691974525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c87f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.923+0000 7fb203bff700 1 -- 172.21.15.119:0/691974525 >> 172.21.15.119:0/691974525 conn(0x7fb1fc0cc540 msgr2=0x7fb1fc0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:19.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.923+0000 7fb203bff700 1 -- 172.21.15.119:0/691974525 shutdown_connections 2023-10-08T01:11:19.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.923+0000 7fb203bff700 1 -- 172.21.15.119:0/691974525 wait complete. 2023-10-08T01:11:19.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.924+0000 7fb203bff700 1 Processor -- start 2023-10-08T01:11:19.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.924+0000 7fb203bff700 1 -- start start 2023-10-08T01:11:19.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.925+0000 7fb203bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.925+0000 7fb203bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1fc156620 0x7fb1fc154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.925+0000 7fb203bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc1569f0 0x7fb1fc0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.925+0000 7fb203bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0c5d30 con 0x7fb1fc1569f0 2023-10-08T01:11:19.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.925+0000 7fb203bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc165530 con 0x7fb1fc156620 2023-10-08T01:11:19.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.925+0000 7fb203bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc0c4860 con 0x7fb1fc068aa0 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc1569f0 0x7fb1fc0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc1569f0 0x7fb1fc0cb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53526/0 (socket says 172.21.15.119:53526) 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 -- 172.21.15.119:0/719204228 learned_addr learned my addr 172.21.15.119:0/719204228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20119a700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1fc156620 0x7fb1fc154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20199b700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 -- 172.21.15.119:0/719204228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 msgr2=0x7fb1fc0c3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 -- 172.21.15.119:0/719204228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1fc156620 msgr2=0x7fb1fc154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1fc156620 0x7fb1fc154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 -- 172.21.15.119:0/719204228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f4009a20 con 0x7fb1fc1569f0 2023-10-08T01:11:19.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.926+0000 7fb20219c700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc1569f0 0x7fb1fc0cb1f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4013000 tx=0x7fb1f4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:19.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.927+0000 7fb1f1ffb700 1 -- 172.21.15.119:0/719204228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4027470 con 0x7fb1fc1569f0 2023-10-08T01:11:19.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.927+0000 7fb1f1ffb700 1 -- 172.21.15.119:0/719204228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f40275f0 con 0x7fb1fc1569f0 2023-10-08T01:11:19.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.927+0000 7fb1f1ffb700 1 -- 172.21.15.119:0/719204228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f40279b0 con 0x7fb1fc1569f0 2023-10-08T01:11:19.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.927+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc0c4b40 con 0x7fb1fc1569f0 2023-10-08T01:11:19.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.927+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc0c5100 con 0x7fb1fc1569f0 2023-10-08T01:11:19.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.928+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc165680 con 0x7fb1fc1569f0 2023-10-08T01:11:19.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.929+0000 7fb1f1ffb700 1 -- 172.21.15.119:0/719204228 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb1f4024070 con 0x7fb1fc1569f0 2023-10-08T01:11:19.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.930+0000 7fb1f1ffb700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb1e0066f30 0x7fb1e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:19.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.930+0000 7fb1f1ffb700 1 -- 172.21.15.119:0/719204228 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5615+0+0 (secure 0 0 0) 0x7fb1f4086bc0 con 0x7fb1fc1569f0 2023-10-08T01:11:19.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.930+0000 7fb20199b700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb1e0066f30 0x7fb1e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:19.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.931+0000 7fb20199b700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb1e0066f30 0x7fb1e00693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007fd0 tx=0x7fb1f8007d10 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:19.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:19.934+0000 7fb1f1ffb700 1 -- 172.21.15.119:0/719204228 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb1fc165680 con 0x7fb1fc1569f0 2023-10-08T01:11:20.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.125+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb1fc165680 con 0x7fb1fc1569f0 2023-10-08T01:11:20.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.711+0000 7fb1f1ffb700 1 -- 172.21.15.119:0/719204228 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v75) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb1fc165680 con 0x7fb1fc1569f0 2023-10-08T01:11:20.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.718+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb1e0066f30 msgr2=0x7fb1e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:20.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.718+0000 7fb203bff700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb1e0066f30 0x7fb1e00693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007fd0 tx=0x7fb1f8007d10 comp rx=0 tx=0).stop 2023-10-08T01:11:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.718+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc1569f0 msgr2=0x7fb1fc0cb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.718+0000 7fb203bff700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc1569f0 0x7fb1fc0cb1f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4013000 tx=0x7fb1f4019000 comp rx=0 tx=0).stop 2023-10-08T01:11:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.719+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 shutdown_connections 2023-10-08T01:11:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.719+0000 7fb203bff700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb1e0066f30 0x7fb1e00693e0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007fd0 tx=0x7fb1f8007d10 comp rx=0 tx=0).stop 2023-10-08T01:11:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.719+0000 7fb203bff700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb1fc068aa0 0x7fb1fc0c3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.719+0000 7fb203bff700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb1fc1569f0 0x7fb1fc0cb1f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.719+0000 7fb203bff700 1 --2- 172.21.15.119:0/719204228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb1fc156620 0x7fb1fc154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.719+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 >> 172.21.15.119:0/719204228 conn(0x7fb1fc0cc540 msgr2=0x7fb1fc1556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:20.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.723+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 shutdown_connections 2023-10-08T01:11:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.725+0000 7fb203bff700 1 -- 172.21.15.119:0/719204228 wait complete. 2023-10-08T01:11:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T01:11:20.750 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T01:11:20.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.933+0000 7fb2e35ff700 1 Processor -- start 2023-10-08T01:11:20.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.933+0000 7fb2e35ff700 1 -- start start 2023-10-08T01:11:20.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e35ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb2dc0c83f0 0x7fb2dc0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:20.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e35ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0ccee0 0x7fb2dc1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:20.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e35ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2dc0c87c0 0x7fb2dc156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:20.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e35ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc156c50 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e35ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc15b0d0 con 0x7fb2dc0c87c0 2023-10-08T01:11:20.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e35ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0c7a60 con 0x7fb2dc0ccee0 2023-10-08T01:11:20.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e139b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb2dc0c83f0 0x7fb2dc0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:20.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e139b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb2dc0c83f0 0x7fb2dc0c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53538/0 (socket says 172.21.15.119:53538) 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e139b700 1 -- 172.21.15.119:0/3121301077 learned_addr learned my addr 172.21.15.119:0/3121301077 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e1b9c700 1 --2- 172.21.15.119:0/3121301077 >> v2:172.21.15.184:3300/0 conn(0x7fb2dc0c87c0 0x7fb2dc156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e0b9a700 1 --2- 172.21.15.119:0/3121301077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0ccee0 0x7fb2dc1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e139b700 1 -- 172.21.15.119:0/3121301077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0ccee0 msgr2=0x7fb2dc1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e139b700 1 --2- 172.21.15.119:0/3121301077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0ccee0 0x7fb2dc1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e139b700 1 -- 172.21.15.119:0/3121301077 >> v2:172.21.15.184:3300/0 conn(0x7fb2dc0c87c0 msgr2=0x7fb2dc156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e139b700 1 --2- 172.21.15.119:0/3121301077 >> v2:172.21.15.184:3300/0 conn(0x7fb2dc0c87c0 0x7fb2dc156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.935+0000 7fb2e139b700 1 -- 172.21.15.119:0/3121301077 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc15a4c0 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.936+0000 7fb2e139b700 1 --2- 172.21.15.119:0/3121301077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc0c7520 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0cc340 tx=0x7fb2d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cd0d54091a98e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.936+0000 7fb2d3fff700 1 -- 172.21.15.119:0/3121301077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800b860 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.936+0000 7fb2d3fff700 1 -- 172.21.15.119:0/3121301077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d800b9c0 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.936+0000 7fb2d3fff700 1 -- 172.21.15.119:0/3121301077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800bd60 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.936+0000 7fb2e35ff700 1 -- 172.21.15.119:0/3121301077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 msgr2=0x7fb2dc0c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.936+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/3121301077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc0c7520 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc0cc340 tx=0x7fb2d8009250 comp rx=0 tx=0).stop 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.937+0000 7fb2e35ff700 1 -- 172.21.15.119:0/3121301077 shutdown_connections 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.937+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/3121301077 >> v2:172.21.15.184:3300/0 conn(0x7fb2dc0c87c0 0x7fb2dc156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.937+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/3121301077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc0c7520 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.937+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/3121301077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0ccee0 0x7fb2dc1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.937+0000 7fb2e35ff700 1 -- 172.21.15.119:0/3121301077 >> 172.21.15.119:0/3121301077 conn(0x7fb2dc068e10 msgr2=0x7fb2dc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.937+0000 7fb2e35ff700 1 -- 172.21.15.119:0/3121301077 shutdown_connections 2023-10-08T01:11:20.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.937+0000 7fb2e35ff700 1 -- 172.21.15.119:0/3121301077 wait complete. 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.938+0000 7fb2e35ff700 1 Processor -- start 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.938+0000 7fb2e35ff700 1 -- start start 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e35ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e35ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0c87c0 0x7fb2dc1561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e35ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc0ccee0 0x7fb2dc144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e35ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1548b0 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e35ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc159b70 con 0x7fb2dc0ccee0 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e35ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc153f80 con 0x7fb2dc0c87c0 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e139b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:20.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e139b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53552/0 (socket says 172.21.15.119:53552) 2023-10-08T01:11:20.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.939+0000 7fb2e139b700 1 -- 172.21.15.119:0/855205295 learned_addr learned my addr 172.21.15.119:0/855205295 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:20.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e1b9c700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc0ccee0 0x7fb2dc144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:20.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e0b9a700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0c87c0 0x7fb2dc1561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:20.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e139b700 1 -- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0c87c0 msgr2=0x7fb2dc1561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:20.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e139b700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0c87c0 0x7fb2dc1561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e139b700 1 -- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc0ccee0 msgr2=0x7fb2dc144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:20.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e139b700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc0ccee0 0x7fb2dc144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:20.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e139b700 1 -- 172.21.15.119:0/855205295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80094f0 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e1b9c700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc0ccee0 0x7fb2dc144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e139b700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc159200 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc063d40 tx=0x7fb2d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2d17fa700 1 -- 172.21.15.119:0/855205295 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8024440 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2d17fa700 1 -- 172.21.15.119:0/855205295 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d80245a0 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2d17fa700 1 -- 172.21.15.119:0/855205295 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8024960 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc154260 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.940+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc156d90 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.941+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2dc0c8c70 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.943+0000 7fb2d17fa700 1 -- 172.21.15.119:0/855205295 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb2d801a420 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.943+0000 7fb2d17fa700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2c0066f30 0x7fb2c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.943+0000 7fb2d17fa700 1 -- 172.21.15.119:0/855205295 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fb2d80868b0 con 0x7fb2dc0c83f0 2023-10-08T01:11:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.944+0000 7fb2e0b9a700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2c0066f30 0x7fb2c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.944+0000 7fb2e0b9a700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2c0066f30 0x7fb2c00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc15b2d0 tx=0x7fb2cc008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:20.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:20.947+0000 7fb2d17fa700 1 -- 172.21.15.119:0/855205295 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2d8053a80 con 0x7fb2dc0c83f0 2023-10-08T01:11:21.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.181+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb2dc04c3c0 con 0x7fb2dc0c83f0 2023-10-08T01:11:21.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.741+0000 7fb2d17fa700 1 -- 172.21.15.119:0/855205295 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb2d80570a0 con 0x7fb2dc0c83f0 2023-10-08T01:11:21.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.746+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2c0066f30 msgr2=0x7fb2c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:21.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.746+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2c0066f30 0x7fb2c00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc15b2d0 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2023-10-08T01:11:21.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.746+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 msgr2=0x7fb2dc159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:21.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.746+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc159200 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc063d40 tx=0x7fb2d8017000 comp rx=0 tx=0).stop 2023-10-08T01:11:21.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.749+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 shutdown_connections 2023-10-08T01:11:21.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.749+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2c0066f30 0x7fb2c00693e0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc15b2d0 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2023-10-08T01:11:21.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.749+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2dc0ccee0 0x7fb2dc144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:21.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.749+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2dc0c83f0 0x7fb2dc159200 secure :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc063d40 tx=0x7fb2d8017000 comp rx=0 tx=0).stop 2023-10-08T01:11:21.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.749+0000 7fb2e35ff700 1 --2- 172.21.15.119:0/855205295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2dc0c87c0 0x7fb2dc1561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:21.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.749+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 >> 172.21.15.119:0/855205295 conn(0x7fb2dc068e10 msgr2=0x7fb2dc0cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.754+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 shutdown_connections 2023-10-08T01:11:21.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.754+0000 7fb2e35ff700 1 -- 172.21.15.119:0/855205295 wait complete. 2023-10-08T01:11:21.759 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 8 and data pool 9 2023-10-08T01:11:21.777 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.958+0000 7fd3357ff700 1 Processor -- start 2023-10-08T01:11:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.958+0000 7fd3357ff700 1 -- start start 2023-10-08T01:11:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.959+0000 7fd3357ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd330159890 0x7fd33015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.959+0000 7fd3357ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330069560 0x7fd3300699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.959+0000 7fd3357ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd330069ee0 con 0x7fd330159c60 2023-10-08T01:11:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.959+0000 7fd3357ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33015a160 con 0x7fd330159890 2023-10-08T01:11:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.959+0000 7fd3357ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300d13a0 con 0x7fd330069560 2023-10-08T01:11:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd330159890 0x7fd33015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd330159890 0x7fd33015abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53560/0 (socket says 172.21.15.119:53560) 2023-10-08T01:11:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32effd700 1 -- 172.21.15.119:0/2963896109 learned_addr learned my addr 172.21.15.119:0/2963896109 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:21.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32f7fe700 1 --2- 172.21.15.119:0/2963896109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330069560 0x7fd3300699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32effd700 1 -- 172.21.15.119:0/2963896109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330069560 msgr2=0x7fd3300699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32effd700 1 --2- 172.21.15.119:0/2963896109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330069560 0x7fd3300699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32effd700 1 -- 172.21.15.119:0/2963896109 >> v1:172.21.15.184:6789/0 conn(0x7fd330159c60 legacy=0x7fd33015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32effd700 1 -- 172.21.15.119:0/2963896109 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33015e140 con 0x7fd330159890 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.960+0000 7fd32effd700 1 --2- 172.21.15.119:0/2963896109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330159890 0x7fd33015abd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd3200099b0 tx=0x7fd320009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d92f0fe9e097ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd32dffb700 1 -- 172.21.15.119:0/2963896109 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32000b820 con 0x7fd330159890 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd32dffb700 1 -- 172.21.15.119:0/2963896109 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32000b980 con 0x7fd330159890 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd32dffb700 1 -- 172.21.15.119:0/2963896109 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32000bd20 con 0x7fd330159890 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd3357ff700 1 -- 172.21.15.119:0/2963896109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330159890 msgr2=0x7fd33015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2963896109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330159890 0x7fd33015abd0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd3200099b0 tx=0x7fd320009250 comp rx=0 tx=0).stop 2023-10-08T01:11:21.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd3357ff700 1 -- 172.21.15.119:0/2963896109 shutdown_connections 2023-10-08T01:11:21.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2963896109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330069560 0x7fd3300699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:21.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2963896109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330159890 0x7fd33015abd0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:21.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd3357ff700 1 -- 172.21.15.119:0/2963896109 >> 172.21.15.119:0/2963896109 conn(0x7fd330068e20 msgr2=0x7fd3300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:21.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd3357ff700 1 -- 172.21.15.119:0/2963896109 shutdown_connections 2023-10-08T01:11:21.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.961+0000 7fd3357ff700 1 -- 172.21.15.119:0/2963896109 wait complete. 2023-10-08T01:11:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.963+0000 7fd3357ff700 1 Processor -- start 2023-10-08T01:11:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.963+0000 7fd3357ff700 1 -- start start 2023-10-08T01:11:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd3357ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330069560 0x7fd330169830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:21.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd3357ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330159890 0x7fd330147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd3357ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330159c60 0x7fd33016cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd3357ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3300b3630 con 0x7fd330069560 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd3357ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33016d6f0 con 0x7fd330159890 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd3357ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33016d9d0 con 0x7fd330159c60 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330069560 0x7fd330169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330069560 0x7fd330169830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53574/0 (socket says 172.21.15.119:53574) 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32effd700 1 -- 172.21.15.119:0/2371494253 learned_addr learned my addr 172.21.15.119:0/2371494253 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32e7fc700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330159890 0x7fd330147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32f7fe700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330159c60 0x7fd33016cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32effd700 1 -- 172.21.15.119:0/2371494253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330159c60 msgr2=0x7fd33016cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:21.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32effd700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330159c60 0x7fd33016cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32effd700 1 -- 172.21.15.119:0/2371494253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330159890 msgr2=0x7fd330147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32effd700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330159890 0x7fd330147160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.964+0000 7fd32effd700 1 -- 172.21.15.119:0/2371494253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3200094f0 con 0x7fd330069560 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.965+0000 7fd32effd700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330069560 0x7fd330169830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd320009290 tx=0x7fd320016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.965+0000 7fd316ffd700 1 -- 172.21.15.119:0/2371494253 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd320011b40 con 0x7fd330069560 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.965+0000 7fd316ffd700 1 -- 172.21.15.119:0/2371494253 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd320020070 con 0x7fd330069560 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.965+0000 7fd316ffd700 1 -- 172.21.15.119:0/2371494253 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32001b450 con 0x7fd330069560 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.965+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3300b38d0 con 0x7fd330069560 2023-10-08T01:11:21.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.965+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33016dcb0 con 0x7fd330069560 2023-10-08T01:11:21.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.966+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33004c3c0 con 0x7fd330069560 2023-10-08T01:11:21.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.967+0000 7fd316ffd700 1 -- 172.21.15.119:0/2371494253 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd320022030 con 0x7fd330069560 2023-10-08T01:11:21.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.968+0000 7fd316ffd700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd31c066f30 0x7fd31c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:21.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.968+0000 7fd316ffd700 1 -- 172.21.15.119:0/2371494253 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fd3200863e0 con 0x7fd330069560 2023-10-08T01:11:21.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.968+0000 7fd32e7fc700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd31c066f30 0x7fd31c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:21.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.969+0000 7fd32e7fc700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd31c066f30 0x7fd31c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd330158d10 tx=0x7fd324007ea0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:21.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:21.972+0000 7fd316ffd700 1 -- 172.21.15.119:0/2371494253 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd3200534c0 con 0x7fd330069560 2023-10-08T01:11:22.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.162+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd33004c3c0 con 0x7fd330069560 2023-10-08T01:11:22.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.163+0000 7fd316ffd700 1 -- 172.21.15.119:0/2371494253 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16008 (secure 0 0 0) 0x7fd320056ae0 con 0x7fd330069560 2023-10-08T01:11:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:22.164 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":76,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:21.183503+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:11:19.347044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T01:11:20.164667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:22.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd31c066f30 msgr2=0x7fd31c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd31c066f30 0x7fd31c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd330158d10 tx=0x7fd324007ea0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330069560 msgr2=0x7fd330169830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330069560 0x7fd330169830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fd320009290 tx=0x7fd320016000 comp rx=0 tx=0).stop 2023-10-08T01:11:22.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 shutdown_connections 2023-10-08T01:11:22.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd31c066f30 0x7fd31c0693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd330159c60 0x7fd33016cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd330069560 0x7fd330169830 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 --2- 172.21.15.119:0/2371494253 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd330159890 0x7fd330147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 >> 172.21.15.119:0/2371494253 conn(0x7fd330068e20 msgr2=0x7fd3301534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:22.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 shutdown_connections 2023-10-08T01:11:22.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.165+0000 7fd3357ff700 1 -- 172.21.15.119:0/2371494253 wait complete. 2023-10-08T01:11:22.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.364+0000 7efccddff700 1 Processor -- start 2023-10-08T01:11:22.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.364+0000 7efccddff700 1 -- start start 2023-10-08T01:11:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.365+0000 7efccddff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efcc81579f0 0x7efcc8162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.365+0000 7efccddff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc80cb300 con 0x7efcc80b1850 2023-10-08T01:11:22.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.365+0000 7efccddff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc80cb5e0 con 0x7efcc80b1c20 2023-10-08T01:11:22.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.365+0000 7efccddff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc806fc30 con 0x7efcc81579f0 2023-10-08T01:11:22.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc77fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efcc80b1850 0x7efcc80ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60052/0 (socket says 172.21.15.119:60052) 2023-10-08T01:11:22.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc77fe700 1 -- 172.21.15.119:0/737424881 learned_addr learned my addr 172.21.15.119:0/737424881 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:22.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc7fff700 1 --2- 172.21.15.119:0/737424881 >> v2:172.21.15.184:3300/0 conn(0x7efcc81579f0 0x7efcc8162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744352668 0 0) 0x7efcc80cb300 con 0x7efcc80b1850 2023-10-08T01:11:22.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcb0003570 con 0x7efcc80b1850 2023-10-08T01:11:22.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4263911556 0 0) 0x7efcb0003570 con 0x7efcc80b1850 2023-10-08T01:11:22.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 >> v2:172.21.15.184:3300/0 conn(0x7efcc81579f0 msgr2=0x7efcc8162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc67fc700 1 --2- 172.21.15.119:0/737424881 >> v2:172.21.15.184:3300/0 conn(0x7efcc81579f0 0x7efcc8162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 >> v1:172.21.15.184:6789/0 conn(0x7efcc80b1c20 legacy=0x7efcc81572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.366+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc80cb8c0 con 0x7efcc80b1850 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7efcb8001470 con 0x7efcc80b1850 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 >> v1:172.21.15.119:6789/0 conn(0x7efcc80b1850 legacy=0x7efcc80ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc67fc700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcb0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc67fc700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb0004810 0x7efcb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc67fc700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb00051c0 0x7efcb000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc806fc30 con 0x7efcb0003a60 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb000c0d0 con 0x7efcb00051c0 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb000c290 con 0x7efcb0004810 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc77fe700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcb0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc7fff700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb00051c0 0x7efcb000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc6ffd700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb0004810 0x7efcb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc77fe700 1 -- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb0004810 msgr2=0x7efcb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc77fe700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcb0004810 0x7efcb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc77fe700 1 -- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb00051c0 msgr2=0x7efcb000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc77fe700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb00051c0 0x7efcb000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc77fe700 1 -- 172.21.15.119:0/737424881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb0003570 con 0x7efcb0003a60 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc7fff700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcb00051c0 0x7efcb000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.367+0000 7efcc7fff700 1 -- 172.21.15.119:0/737424881 reap_dead start 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efcc77fe700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcb0003e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c3c0 tx=0x7efcb8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4868e40f1b06ea35 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb800cc90 con 0x7efcb0003a60 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb800cdf0 con 0x7efcb0003a60 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efcc67fc700 1 -- 172.21.15.119:0/737424881 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8005b50 con 0x7efcb0003a60 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efccddff700 1 -- 172.21.15.119:0/737424881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 msgr2=0x7efcb0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efccddff700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcb0003e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c3c0 tx=0x7efcb8001e50 comp rx=0 tx=0).stop 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efccddff700 1 -- 172.21.15.119:0/737424881 shutdown_connections 2023-10-08T01:11:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efccddff700 1 --2- 172.21.15.119:0/737424881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcb0003e70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efccddff700 1 -- 172.21.15.119:0/737424881 >> 172.21.15.119:0/737424881 conn(0x7efcc806a320 msgr2=0x7efcc80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efccddff700 1 -- 172.21.15.119:0/737424881 shutdown_connections 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.368+0000 7efccddff700 1 -- 172.21.15.119:0/737424881 wait complete. 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.370+0000 7efccddff700 1 Processor -- start 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.370+0000 7efccddff700 1 -- start start 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efccddff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcc814b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efccddff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc80b0940 0x7efcc80b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efccddff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc814dc10 0x7efcc814e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efccddff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80cc260 con 0x7efcb0003a60 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efccddff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc80b12f0 con 0x7efcc80b0940 2023-10-08T01:11:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efccddff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc814e5f0 con 0x7efcc814dc10 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcc814b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efcc77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcc814b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53594/0 (socket says 172.21.15.119:53594) 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efcc77fe700 1 -- 172.21.15.119:0/52854782 learned_addr learned my addr 172.21.15.119:0/52854782 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efcc6ffd700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc80b0940 0x7efcc80b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efcc7fff700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc814dc10 0x7efcc814e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.371+0000 7efcc77fe700 1 -- 172.21.15.119:0/52854782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc814dc10 msgr2=0x7efcc814e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efcc77fe700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc814dc10 0x7efcc814e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efcc77fe700 1 -- 172.21.15.119:0/52854782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc80b0940 msgr2=0x7efcc80b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efcc77fe700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc80b0940 0x7efcc80b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efcc77fe700 1 -- 172.21.15.119:0/52854782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc815aa40 con 0x7efcb0003a60 2023-10-08T01:11:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efcc77fe700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcc814b470 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c9b0 tx=0x7efcb8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:22.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efcaf7fe700 1 -- 172.21.15.119:0/52854782 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8022070 con 0x7efcb0003a60 2023-10-08T01:11:22.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efcaf7fe700 1 -- 172.21.15.119:0/52854782 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb800c9e0 con 0x7efcb0003a60 2023-10-08T01:11:22.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc814e880 con 0x7efcb0003a60 2023-10-08T01:11:22.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efcaf7fe700 1 -- 172.21.15.119:0/52854782 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb800cc90 con 0x7efcb0003a60 2023-10-08T01:11:22.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.372+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc814ee40 con 0x7efcb0003a60 2023-10-08T01:11:22.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.373+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc80a9b00 con 0x7efcb0003a60 2023-10-08T01:11:22.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.374+0000 7efcaf7fe700 1 -- 172.21.15.119:0/52854782 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcb8007550 con 0x7efcb0003a60 2023-10-08T01:11:22.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.375+0000 7efcaf7fe700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcb4066f30 0x7efcb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.375+0000 7efcaf7fe700 1 -- 172.21.15.119:0/52854782 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6097+0+0 (secure 0 0 0) 0x7efcb80857f0 con 0x7efcb0003a60 2023-10-08T01:11:22.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.375+0000 7efcc6ffd700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcb4066f30 0x7efcb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.376+0000 7efcc6ffd700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcb4066f30 0x7efcb40693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efcc8159680 tx=0x7efcbc009040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:22.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.379+0000 7efcaf7fe700 1 -- 172.21.15.119:0/52854782 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcb80528d0 con 0x7efcb0003a60 2023-10-08T01:11:22.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.571+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcc804c3c0 con 0x7efcb0003a60 2023-10-08T01:11:22.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.572+0000 7efcaf7fe700 1 -- 172.21.15.119:0/52854782 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+16008 (secure 0 0 0) 0x7efcb8055ef0 con 0x7efcb0003a60 2023-10-08T01:11:22.573 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:22.573 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":76,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:21.183503+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:11:19.347044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T01:11:20.164667+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:22.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcb4066f30 msgr2=0x7efcb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcb4066f30 0x7efcb40693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efcc8159680 tx=0x7efcbc009040 comp rx=0 tx=0).stop 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 msgr2=0x7efcc814b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcc814b470 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7efcb800c9b0 tx=0x7efcb8015000 comp rx=0 tx=0).stop 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 shutdown_connections 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcb4066f30 0x7efcb40693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc814dc10 0x7efcc814e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb0003a60 0x7efcc814b470 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 --2- 172.21.15.119:0/52854782 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc80b0940 0x7efcc80b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 >> 172.21.15.119:0/52854782 conn(0x7efcc806a320 msgr2=0x7efcc806f210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:22.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 shutdown_connections 2023-10-08T01:11:22.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.574+0000 7efccddff700 1 -- 172.21.15.119:0/52854782 wait complete. 2023-10-08T01:11:22.593 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T01:11:22.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.780+0000 7f25f1dff700 1 Processor -- start 2023-10-08T01:11:22.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.780+0000 7f25f1dff700 1 -- start start 2023-10-08T01:11:22.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.781+0000 7f25f1dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06bcd0 0x7f25ec0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.781+0000 7f25f1dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f25ec06c0a0 0x7f25ec15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.781+0000 7f25f1dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25ec15df50 0x7f25ec15e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.781+0000 7f25f1dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec15d7e0 con 0x7f25ec15df50 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.781+0000 7f25f1dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ec15dac0 con 0x7f25ec06c0a0 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.781+0000 7f25f1dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec15aa30 con 0x7f25ec06bcd0 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25ec15df50 0x7f25ec15e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25ec15df50 0x7f25ec15e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53596/0 (socket says 172.21.15.119:53596) 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 -- 172.21.15.119:0/1683193685 learned_addr learned my addr 172.21.15.119:0/1683193685 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25eaffd700 1 --2- 172.21.15.119:0/1683193685 >> v2:172.21.15.184:3300/0 conn(0x7f25ec06c0a0 0x7f25ec15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25eb7fe700 1 --2- 172.21.15.119:0/1683193685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06bcd0 0x7f25ec0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 -- 172.21.15.119:0/1683193685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06bcd0 msgr2=0x7f25ec0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 --2- 172.21.15.119:0/1683193685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06bcd0 0x7f25ec0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 -- 172.21.15.119:0/1683193685 >> v2:172.21.15.184:3300/0 conn(0x7f25ec06c0a0 msgr2=0x7f25ec15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:22.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 --2- 172.21.15.119:0/1683193685 >> v2:172.21.15.184:3300/0 conn(0x7f25ec06c0a0 0x7f25ec15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 -- 172.21.15.119:0/1683193685 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec15ad10 con 0x7f25ec15df50 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25eaffd700 1 --2- 172.21.15.119:0/1683193685 >> v2:172.21.15.184:3300/0 conn(0x7f25ec06c0a0 0x7f25ec15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25eb7fe700 1 --2- 172.21.15.119:0/1683193685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06bcd0 0x7f25ec0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ebfff700 1 --2- 172.21.15.119:0/1683193685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec15e3b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0c5660 tx=0x7f25e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23016b3858474d64 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.782+0000 7f25ea7fc700 1 -- 172.21.15.119:0/1683193685 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e400e070 con 0x7f25ec15df50 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25ea7fc700 1 -- 172.21.15.119:0/1683193685 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e400ba50 con 0x7f25ec15df50 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25ea7fc700 1 -- 172.21.15.119:0/1683193685 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e40092e0 con 0x7f25ec15df50 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 -- 172.21.15.119:0/1683193685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 msgr2=0x7f25ec15e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 --2- 172.21.15.119:0/1683193685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec15e3b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0c5660 tx=0x7f25e4009840 comp rx=0 tx=0).stop 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 -- 172.21.15.119:0/1683193685 shutdown_connections 2023-10-08T01:11:22.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 --2- 172.21.15.119:0/1683193685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec15e3b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 --2- 172.21.15.119:0/1683193685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06bcd0 0x7f25ec0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 --2- 172.21.15.119:0/1683193685 >> v2:172.21.15.184:3300/0 conn(0x7f25ec06c0a0 0x7f25ec15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 -- 172.21.15.119:0/1683193685 >> 172.21.15.119:0/1683193685 conn(0x7f25ec0ce600 msgr2=0x7f25ec0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:22.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 -- 172.21.15.119:0/1683193685 shutdown_connections 2023-10-08T01:11:22.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.783+0000 7f25f1dff700 1 -- 172.21.15.119:0/1683193685 wait complete. 2023-10-08T01:11:22.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.785+0000 7f25f1dff700 1 Processor -- start 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.785+0000 7f25f1dff700 1 -- start start 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25f1dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25ec06bcd0 0x7f25ec154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25f1dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06c0a0 0x7f25ec0c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25f1dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec16a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25f1dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec159c80 con 0x7f25ec15df50 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25f1dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec0c5bf0 con 0x7f25ec06bcd0 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25f1dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ec0b1820 con 0x7f25ec06c0a0 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec16a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec16a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53608/0 (socket says 172.21.15.119:53608) 2023-10-08T01:11:22.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 -- 172.21.15.119:0/198847885 learned_addr learned my addr 172.21.15.119:0/198847885 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25eaffd700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06c0a0 0x7f25ec0c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25eb7fe700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25ec06bcd0 0x7f25ec154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 -- 172.21.15.119:0/198847885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06c0a0 msgr2=0x7f25ec0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06c0a0 0x7f25ec0c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 -- 172.21.15.119:0/198847885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25ec06bcd0 msgr2=0x7f25ec154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25ec06bcd0 0x7f25ec154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 -- 172.21.15.119:0/198847885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e4009ae0 con 0x7f25ec15df50 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.786+0000 7f25ebfff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec16a1c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f25ec069220 tx=0x7f25e4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.787+0000 7f25d37fe700 1 -- 172.21.15.119:0/198847885 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e40070a0 con 0x7f25ec15df50 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.787+0000 7f25d37fe700 1 -- 172.21.15.119:0/198847885 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25e401d420 con 0x7f25ec15df50 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.787+0000 7f25d37fe700 1 -- 172.21.15.119:0/198847885 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e401d800 con 0x7f25ec15df50 2023-10-08T01:11:22.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.787+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ec16b040 con 0x7f25ec15df50 2023-10-08T01:11:22.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.787+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ec0c8020 con 0x7f25ec15df50 2023-10-08T01:11:22.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.788+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25ec04c3c0 con 0x7f25ec15df50 2023-10-08T01:11:22.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.789+0000 7f25d37fe700 1 -- 172.21.15.119:0/198847885 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f25e4003050 con 0x7f25ec15df50 2023-10-08T01:11:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.789+0000 7f25d37fe700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25d8066f30 0x7f25d80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.790+0000 7f25d37fe700 1 -- 172.21.15.119:0/198847885 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f25e40865f0 con 0x7f25ec15df50 2023-10-08T01:11:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.790+0000 7f25eb7fe700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25d8066f30 0x7f25d80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:22.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.790+0000 7f25eb7fe700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25d8066f30 0x7f25d80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25dc006fd0 tx=0x7f25dc008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:22.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:22.794+0000 7f25d37fe700 1 -- 172.21.15.119:0/198847885 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f25e4053630 con 0x7f25ec15df50 2023-10-08T01:11:23.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.031+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f25ec15aff0 con 0x7f25ec15df50 2023-10-08T01:11:23.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.752+0000 7f25d37fe700 1 -- 172.21.15.119:0/198847885 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) v1 ==== 118+0+0 (secure 0 0 0) 0x7f25e4056c50 con 0x7f25ec15df50 2023-10-08T01:11:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25d8066f30 msgr2=0x7f25d80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25d8066f30 0x7f25d80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f25dc006fd0 tx=0x7f25dc008040 comp rx=0 tx=0).stop 2023-10-08T01:11:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 msgr2=0x7f25ec16a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec16a1c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f25ec069220 tx=0x7f25e4018000 comp rx=0 tx=0).stop 2023-10-08T01:11:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 shutdown_connections 2023-10-08T01:11:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25d8066f30 0x7f25d80693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25ec15df50 0x7f25ec16a1c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25ec06bcd0 0x7f25ec154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 --2- 172.21.15.119:0/198847885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f25ec06c0a0 0x7f25ec0c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 >> 172.21.15.119:0/198847885 conn(0x7f25ec0ce600 msgr2=0x7f25ec0c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 shutdown_connections 2023-10-08T01:11:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.755+0000 7f25f1dff700 1 -- 172.21.15.119:0/198847885 wait complete. 2023-10-08T01:11:23.775 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T01:11:23.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.959+0000 7f5c7c5ff700 1 Processor -- start 2023-10-08T01:11:23.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.959+0000 7f5c7c5ff700 1 -- start start 2023-10-08T01:11:23.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.960+0000 7f5c7c5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c740c83f0 0x7f5c740c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:23.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.960+0000 7f5c7c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 0x7f5c741561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:23.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7c5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5c740c87c0 0x7f5c74156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:23.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7c5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c74156c50 con 0x7f5c740c83f0 2023-10-08T01:11:23.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7c5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c7415b0d0 con 0x7f5c740c87c0 2023-10-08T01:11:23.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740c7a60 con 0x7f5c740ccee0 2023-10-08T01:11:23.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7a39b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c740c83f0 0x7f5c740c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:23.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7a39b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c740c83f0 0x7f5c740c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53616/0 (socket says 172.21.15.119:53616) 2023-10-08T01:11:23.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7a39b700 1 -- 172.21.15.119:0/769812688 learned_addr learned my addr 172.21.15.119:0/769812688 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c79b9a700 1 --2- 172.21.15.119:0/769812688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 0x7f5c741561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7ab9c700 1 --2- 172.21.15.119:0/769812688 >> v2:172.21.15.184:3300/0 conn(0x7f5c740c87c0 0x7f5c74156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7a39b700 1 -- 172.21.15.119:0/769812688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 msgr2=0x7f5c741561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7a39b700 1 --2- 172.21.15.119:0/769812688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 0x7f5c741561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7a39b700 1 -- 172.21.15.119:0/769812688 >> v2:172.21.15.184:3300/0 conn(0x7f5c740c87c0 msgr2=0x7f5c74156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7a39b700 1 --2- 172.21.15.119:0/769812688 >> v2:172.21.15.184:3300/0 conn(0x7f5c740c87c0 0x7f5c74156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.961+0000 7f5c7a39b700 1 -- 172.21.15.119:0/769812688 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7415a4c0 con 0x7f5c740c83f0 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c7a39b700 1 --2- 172.21.15.119:0/769812688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c740c7520 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5c740cc340 tx=0x7f5c70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1827dc9b6169b1e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c79399700 1 -- 172.21.15.119:0/769812688 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000b820 con 0x7f5c740c83f0 2023-10-08T01:11:23.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c79399700 1 -- 172.21.15.119:0/769812688 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7000b980 con 0x7f5c740c83f0 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c79399700 1 -- 172.21.15.119:0/769812688 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7000bd20 con 0x7f5c740c83f0 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/769812688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 msgr2=0x7f5c740c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/769812688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c740c7520 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5c740cc340 tx=0x7f5c70009250 comp rx=0 tx=0).stop 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/769812688 shutdown_connections 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/769812688 >> v2:172.21.15.184:3300/0 conn(0x7f5c740c87c0 0x7f5c74156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/769812688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c740c7520 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/769812688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 0x7f5c741561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.962+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/769812688 >> 172.21.15.119:0/769812688 conn(0x7f5c74068e10 msgr2=0x7f5c740cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.963+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/769812688 shutdown_connections 2023-10-08T01:11:23.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.963+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/769812688 wait complete. 2023-10-08T01:11:23.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.964+0000 7f5c7c5ff700 1 Processor -- start 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.964+0000 7f5c7c5ff700 1 -- start start 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7c5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c74154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7c5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c740c87c0 0x7f5c7413df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 0x7f5c7416aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7c5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74157f10 con 0x7f5c740c83f0 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7c5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7413e4b0 con 0x7f5c740c87c0 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c741551c0 con 0x7f5c740ccee0 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c74154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:23.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7a39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c74154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53628/0 (socket says 172.21.15.119:53628) 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.965+0000 7f5c7a39b700 1 -- 172.21.15.119:0/1363338470 learned_addr learned my addr 172.21.15.119:0/1363338470 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c7a39b700 1 -- 172.21.15.119:0/1363338470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 msgr2=0x7f5c7416aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c7a39b700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 0x7f5c7416aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c79b9a700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c740c87c0 0x7f5c7413df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c7a39b700 1 -- 172.21.15.119:0/1363338470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c740c87c0 msgr2=0x7f5c7413df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c7a39b700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c740c87c0 0x7f5c7413df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c7a39b700 1 -- 172.21.15.119:0/1363338470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700094f0 con 0x7f5c740c83f0 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c7a39b700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c74154b00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5c700094c0 tx=0x7f5c70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:23.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c6a7fc700 1 -- 172.21.15.119:0/1363338470 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70024440 con 0x7f5c740c83f0 2023-10-08T01:11:23.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c6a7fc700 1 -- 172.21.15.119:0/1363338470 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c700245e0 con 0x7f5c740c83f0 2023-10-08T01:11:23.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c6a7fc700 1 -- 172.21.15.119:0/1363338470 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c700249a0 con 0x7f5c740c83f0 2023-10-08T01:11:23.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c741554a0 con 0x7f5c740c83f0 2023-10-08T01:11:23.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.966+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c74155a60 con 0x7f5c740c83f0 2023-10-08T01:11:23.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.967+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7404c3c0 con 0x7f5c740c83f0 2023-10-08T01:11:23.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.968+0000 7f5c6a7fc700 1 -- 172.21.15.119:0/1363338470 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5c7001a420 con 0x7f5c740c83f0 2023-10-08T01:11:23.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.969+0000 7f5c6a7fc700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5c58066f30 0x7f5c580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:23.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.969+0000 7f5c6a7fc700 1 -- 172.21.15.119:0/1363338470 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f5c70086580 con 0x7f5c740c83f0 2023-10-08T01:11:23.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.969+0000 7f5c79b9a700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5c58066f30 0x7f5c580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:23.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.970+0000 7f5c79b9a700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5c58066f30 0x7f5c580693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c74159e50 tx=0x7f5c64008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:23.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:23.974+0000 7f5c6a7fc700 1 -- 172.21.15.119:0/1363338470 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5c700535c0 con 0x7f5c740c83f0 2023-10-08T01:11:24.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:24.215+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5c7415a7a0 con 0x7f5c740c83f0 2023-10-08T01:11:25.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.171+0000 7f5c6a7fc700 1 -- 172.21.15.119:0/1363338470 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5c70056be0 con 0x7f5c740c83f0 2023-10-08T01:11:25.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5c58066f30 msgr2=0x7f5c580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5c58066f30 0x7f5c580693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5c74159e50 tx=0x7f5c64008040 comp rx=0 tx=0).stop 2023-10-08T01:11:25.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 msgr2=0x7f5c74154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c74154b00 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5c700094c0 tx=0x7f5c70017000 comp rx=0 tx=0).stop 2023-10-08T01:11:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 shutdown_connections 2023-10-08T01:11:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5c58066f30 0x7f5c580693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5c740ccee0 0x7f5c7416aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c740c83f0 0x7f5c74154b00 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 --2- 172.21.15.119:0/1363338470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5c740c87c0 0x7f5c7413df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 >> 172.21.15.119:0/1363338470 conn(0x7f5c74068e10 msgr2=0x7f5c7406eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 shutdown_connections 2023-10-08T01:11:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.175+0000 7f5c7c5ff700 1 -- 172.21.15.119:0/1363338470 wait complete. 2023-10-08T01:11:25.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:25.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.383+0000 7fe6921ff700 1 Processor -- start 2023-10-08T01:11:25.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.383+0000 7fe6921ff700 1 -- start start 2023-10-08T01:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.384+0000 7fe6921ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c155c10 0x7fe68c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.384+0000 7fe6921ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe68c0ac6a0 0x7fe68c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.384+0000 7fe6921ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c0acac0 con 0x7fe68c0ac230 2023-10-08T01:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.384+0000 7fe6921ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68c0ad930 con 0x7fe68c0ac6a0 2023-10-08T01:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.384+0000 7fe6921ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0acd20 con 0x7fe68c155c10 2023-10-08T01:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.384+0000 7fe68bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe68c0ac6a0 0x7fe68c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.384+0000 7fe68bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe68c0ac6a0 0x7fe68c0b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52304/0 (socket says 172.21.15.119:52304) 2023-10-08T01:11:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.384+0000 7fe68bfff700 1 -- 172.21.15.119:0/602876499 learned_addr learned my addr 172.21.15.119:0/602876499 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.385+0000 7fe68bfff700 1 -- 172.21.15.119:0/602876499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c155c10 msgr2=0x7fe68c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.385+0000 7fe68bfff700 1 --2- 172.21.15.119:0/602876499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c155c10 0x7fe68c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.385+0000 7fe68bfff700 1 -- 172.21.15.119:0/602876499 >> v1:172.21.15.184:6789/0 conn(0x7fe68c0ac230 legacy=0x7fe68c1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.385+0000 7fe68bfff700 1 -- 172.21.15.119:0/602876499 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c0ad000 con 0x7fe68c0ac6a0 2023-10-08T01:11:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.385+0000 7fe68bfff700 1 --2- 172.21.15.119:0/602876499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0ac6a0 0x7fe68c0b0300 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe68400d1a0 tx=0x7fe68400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9e18a7ac527e2a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.385+0000 7fe68a7fc700 1 -- 172.21.15.119:0/602876499 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684010070 con 0x7fe68c0ac6a0 2023-10-08T01:11:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.385+0000 7fe68a7fc700 1 -- 172.21.15.119:0/602876499 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400dc70 con 0x7fe68c0ac6a0 2023-10-08T01:11:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.385+0000 7fe68a7fc700 1 -- 172.21.15.119:0/602876499 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684016440 con 0x7fe68c0ac6a0 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.386+0000 7fe6921ff700 1 -- 172.21.15.119:0/602876499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0ac6a0 msgr2=0x7fe68c0b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.386+0000 7fe6921ff700 1 --2- 172.21.15.119:0/602876499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0ac6a0 0x7fe68c0b0300 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe68400d1a0 tx=0x7fe68400ba50 comp rx=0 tx=0).stop 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.386+0000 7fe6921ff700 1 -- 172.21.15.119:0/602876499 shutdown_connections 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.386+0000 7fe6921ff700 1 --2- 172.21.15.119:0/602876499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0ac6a0 0x7fe68c0b0300 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.386+0000 7fe6921ff700 1 --2- 172.21.15.119:0/602876499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c155c10 0x7fe68c155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.386+0000 7fe6921ff700 1 -- 172.21.15.119:0/602876499 >> 172.21.15.119:0/602876499 conn(0x7fe68c06a320 msgr2=0x7fe68c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.386+0000 7fe6921ff700 1 -- 172.21.15.119:0/602876499 shutdown_connections 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.386+0000 7fe6921ff700 1 -- 172.21.15.119:0/602876499 wait complete. 2023-10-08T01:11:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.388+0000 7fe6921ff700 1 Processor -- start 2023-10-08T01:11:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.388+0000 7fe6921ff700 1 -- start start 2023-10-08T01:11:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe6921ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0ac230 0x7fe68c153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe6921ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0ac6a0 0x7fe68c153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe6921ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c155c10 0x7fe68c144140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe6921ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c140770 con 0x7fe68c155c10 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe6921ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c140a50 con 0x7fe68c0ac6a0 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe6921ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c140d30 con 0x7fe68c0ac230 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c155c10 0x7fe68c144140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c155c10 0x7fe68c144140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52320/0 (socket says 172.21.15.119:52320) 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68bfff700 1 -- 172.21.15.119:0/3253659578 learned_addr learned my addr 172.21.15.119:0/3253659578 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68b7fe700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0ac230 0x7fe68c153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68affd700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0ac6a0 0x7fe68c153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68bfff700 1 -- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0ac230 msgr2=0x7fe68c153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68bfff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0ac230 0x7fe68c153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68bfff700 1 -- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0ac6a0 msgr2=0x7fe68c153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68bfff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0ac6a0 0x7fe68c153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68bfff700 1 -- 172.21.15.119:0/3253659578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68400bcf0 con 0x7fe68c155c10 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68b7fe700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0ac230 0x7fe68c153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.389+0000 7fe68affd700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0ac6a0 0x7fe68c153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.390+0000 7fe68bfff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c155c10 0x7fe68c144140 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe68400ba90 tx=0x7fe68401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.390+0000 7fe6737fe700 1 -- 172.21.15.119:0/3253659578 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684016440 con 0x7fe68c155c10 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.390+0000 7fe6737fe700 1 -- 172.21.15.119:0/3253659578 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe684020670 con 0x7fe68c155c10 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.390+0000 7fe6737fe700 1 -- 172.21.15.119:0/3253659578 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684020a10 con 0x7fe68c155c10 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.390+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c141010 con 0x7fe68c155c10 2023-10-08T01:11:25.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.390+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c141810 con 0x7fe68c155c10 2023-10-08T01:11:25.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.391+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c141530 con 0x7fe68c155c10 2023-10-08T01:11:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.392+0000 7fe6737fe700 1 -- 172.21.15.119:0/3253659578 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe684008410 con 0x7fe68c155c10 2023-10-08T01:11:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.393+0000 7fe6737fe700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe678066f30 0x7fe6780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.393+0000 7fe6737fe700 1 -- 172.21.15.119:0/3253659578 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe684086be0 con 0x7fe68c155c10 2023-10-08T01:11:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.393+0000 7fe68b7fe700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe678066f30 0x7fe6780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.393+0000 7fe68b7fe700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe678066f30 0x7fe6780693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe67c006fd0 tx=0x7fe67c008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:25.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.397+0000 7fe6737fe700 1 -- 172.21.15.119:0/3253659578 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe68c141530 con 0x7fe68c155c10 2023-10-08T01:11:25.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.628+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe68c141530 con 0x7fe68c155c10 2023-10-08T01:11:25.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.629+0000 7fe6737fe700 1 -- 172.21.15.119:0/3253659578 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fe68c141530 con 0x7fe68c155c10 2023-10-08T01:11:25.630 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:25.630 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4024548231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":4024548231},{"type":"v1","addr":"172.21.15.184:6835","nonce":4024548231}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1924614082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1924614082},{"type":"v1","addr":"172.21.15.119:6835","nonce":1924614082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5902,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/296234355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":296234355},{"type":"v1","addr":"172.21.15.184:6834","nonce":296234355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:11:21.183828+0000","modified":"2023-10-08T01:11:25.168981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6225":{"gid":6225,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6834/926130691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":926130691},{"type":"v1","addr":"172.21.15.119:6834","nonce":926130691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T01:11:25.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe678066f30 msgr2=0x7fe6780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe678066f30 0x7fe6780693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe67c006fd0 tx=0x7fe67c008040 comp rx=0 tx=0).stop 2023-10-08T01:11:25.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c155c10 msgr2=0x7fe68c144140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c155c10 0x7fe68c144140 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe68400ba90 tx=0x7fe68401a000 comp rx=0 tx=0).stop 2023-10-08T01:11:25.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 shutdown_connections 2023-10-08T01:11:25.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe678066f30 0x7fe6780693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0ac230 0x7fe68c153910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c155c10 0x7fe68c144140 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 --2- 172.21.15.119:0/3253659578 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0ac6a0 0x7fe68c153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.631+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 >> 172.21.15.119:0/3253659578 conn(0x7fe68c06a320 msgr2=0x7fe68c154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:25.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.632+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 shutdown_connections 2023-10-08T01:11:25.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.632+0000 7fe6921ff700 1 -- 172.21.15.119:0/3253659578 wait complete. 2023-10-08T01:11:25.634 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 32 2023-10-08T01:11:25.650 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:25.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.831+0000 7fd41359e700 1 Processor -- start 2023-10-08T01:11:25.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.832+0000 7fd41359e700 1 -- start start 2023-10-08T01:11:25.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd41359e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd4141566e0 0x7fd4140ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd41359e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4140cb900 0x7fd4140ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd41359e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4140ca860 con 0x7fd414156310 2023-10-08T01:11:25.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd41359e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4140cab40 con 0x7fd4140cb900 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd41359e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4140cae20 con 0x7fd4141566e0 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd412d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4140cb900 0x7fd4140ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd41259c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd414156310 0x7fd4140c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42476/0 (socket says 172.21.15.119:42476) 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd41259c700 1 -- 172.21.15.119:0/1518509386 learned_addr learned my addr 172.21.15.119:0/1518509386 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd412d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4140cb900 0x7fd4140ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52332/0 (socket says 172.21.15.119:52332) 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.833+0000 7fd411d9b700 1 --2- 172.21.15.119:0/1518509386 >> v2:172.21.15.184:3300/0 conn(0x7fd4141566e0 0x7fd4140ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd412d9d700 1 -- 172.21.15.119:0/1518509386 >> v2:172.21.15.184:3300/0 conn(0x7fd4141566e0 msgr2=0x7fd4140ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd412d9d700 1 --2- 172.21.15.119:0/1518509386 >> v2:172.21.15.184:3300/0 conn(0x7fd4141566e0 0x7fd4140ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd412d9d700 1 -- 172.21.15.119:0/1518509386 >> v1:172.21.15.119:6789/0 conn(0x7fd414156310 legacy=0x7fd4140c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd412d9d700 1 -- 172.21.15.119:0/1518509386 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140cb100 con 0x7fd4140cb900 2023-10-08T01:11:25.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd41159a700 1 -- 172.21.15.119:0/1518509386 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547484433 0 0) 0x7fd4140ca860 con 0x7fd414156310 2023-10-08T01:11:25.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd411d9b700 1 --2- 172.21.15.119:0/1518509386 >> v2:172.21.15.184:3300/0 conn(0x7fd4141566e0 0x7fd4140ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:25.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd412d9d700 1 --2- 172.21.15.119:0/1518509386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4140cb900 0x7fd4140ae540 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd41406c000 tx=0x7fd40c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1925b98b92da6045 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:25.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd41159a700 1 -- 172.21.15.119:0/1518509386 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00e070 con 0x7fd4140cb900 2023-10-08T01:11:25.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd41159a700 1 -- 172.21.15.119:0/1518509386 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c00bc30 con 0x7fd4140cb900 2023-10-08T01:11:25.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.834+0000 7fd41159a700 1 -- 172.21.15.119:0/1518509386 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c012430 con 0x7fd4140cb900 2023-10-08T01:11:25.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.835+0000 7fd41359e700 1 -- 172.21.15.119:0/1518509386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4140cb900 msgr2=0x7fd4140ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.835+0000 7fd41359e700 1 --2- 172.21.15.119:0/1518509386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4140cb900 0x7fd4140ae540 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd41406c000 tx=0x7fd40c009780 comp rx=0 tx=0).stop 2023-10-08T01:11:25.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.835+0000 7fd41359e700 1 -- 172.21.15.119:0/1518509386 shutdown_connections 2023-10-08T01:11:25.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.835+0000 7fd41359e700 1 --2- 172.21.15.119:0/1518509386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4140cb900 0x7fd4140ae540 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.835+0000 7fd41359e700 1 --2- 172.21.15.119:0/1518509386 >> v2:172.21.15.184:3300/0 conn(0x7fd4141566e0 0x7fd4140ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.835+0000 7fd41359e700 1 -- 172.21.15.119:0/1518509386 >> 172.21.15.119:0/1518509386 conn(0x7fd4140cc1f0 msgr2=0x7fd4140cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:25.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.835+0000 7fd41359e700 1 -- 172.21.15.119:0/1518509386 shutdown_connections 2023-10-08T01:11:25.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.835+0000 7fd41359e700 1 -- 172.21.15.119:0/1518509386 wait complete. 2023-10-08T01:11:25.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.836+0000 7fd41359e700 1 Processor -- start 2023-10-08T01:11:25.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.836+0000 7fd41359e700 1 -- start start 2023-10-08T01:11:25.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd41359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4140cb900 0x7fd414150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd41359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd414156310 0x7fd4140b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd41359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4141566e0 0x7fd41415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd41359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41415c860 con 0x7fd414156310 2023-10-08T01:11:25.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd41359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41415b7c0 con 0x7fd4141566e0 2023-10-08T01:11:25.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd41359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41415baa0 con 0x7fd4140cb900 2023-10-08T01:11:25.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd411d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd414156310 0x7fd4140b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd411d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd414156310 0x7fd4140b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52346/0 (socket says 172.21.15.119:52346) 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.837+0000 7fd411d9b700 1 -- 172.21.15.119:0/1283972137 learned_addr learned my addr 172.21.15.119:0/1283972137 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd41259c700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4140cb900 0x7fd414150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd412d9d700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4141566e0 0x7fd41415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd411d9b700 1 -- 172.21.15.119:0/1283972137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4140cb900 msgr2=0x7fd414150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd411d9b700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4140cb900 0x7fd414150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd411d9b700 1 -- 172.21.15.119:0/1283972137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4141566e0 msgr2=0x7fd41415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd411d9b700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4141566e0 0x7fd41415d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd411d9b700 1 -- 172.21.15.119:0/1283972137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd40c009a20 con 0x7fd414156310 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd411d9b700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd414156310 0x7fd4140b1770 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e8f0 tx=0x7fd408008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:25.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd4027fc700 1 -- 172.21.15.119:0/1283972137 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408013410 con 0x7fd414156310 2023-10-08T01:11:25.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd4027fc700 1 -- 172.21.15.119:0/1283972137 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd408013570 con 0x7fd414156310 2023-10-08T01:11:25.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd4027fc700 1 -- 172.21.15.119:0/1283972137 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd408013930 con 0x7fd414156310 2023-10-08T01:11:25.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.838+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41415bd80 con 0x7fd414156310 2023-10-08T01:11:25.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.839+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41415c340 con 0x7fd414156310 2023-10-08T01:11:25.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.840+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41404c3c0 con 0x7fd414156310 2023-10-08T01:11:25.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.840+0000 7fd4027fc700 1 -- 172.21.15.119:0/1283972137 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd408013ca0 con 0x7fd414156310 2023-10-08T01:11:25.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.841+0000 7fd4027fc700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd3f4066f30 0x7fd3f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:25.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.841+0000 7fd4027fc700 1 -- 172.21.15.119:0/1283972137 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fd4080844b0 con 0x7fd414156310 2023-10-08T01:11:25.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.842+0000 7fd41259c700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd3f4066f30 0x7fd3f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:25.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.842+0000 7fd41259c700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd3f4066f30 0x7fd3f40693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd404007fd0 tx=0x7fd404007d30 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:25.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:25.845+0000 7fd4027fc700 1 -- 172.21.15.119:0/1283972137 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd408055020 con 0x7fd414156310 2023-10-08T01:11:26.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.039+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4140cd0b0 con 0x7fd414156310 2023-10-08T01:11:26.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.039+0000 7fd4027fc700 1 -- 172.21.15.119:0/1283972137 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+15995 (secure 0 0 0) 0x7fd408054bb0 con 0x7fd414156310 2023-10-08T01:11:26.041 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:26.041 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":77,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:22.720810+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:11:19.347044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T01:11:20.164667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:26.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd3f4066f30 msgr2=0x7fd3f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd3f4066f30 0x7fd3f40693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd404007fd0 tx=0x7fd404007d30 comp rx=0 tx=0).stop 2023-10-08T01:11:26.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd414156310 msgr2=0x7fd4140b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd414156310 0x7fd4140b1770 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e8f0 tx=0x7fd408008040 comp rx=0 tx=0).stop 2023-10-08T01:11:26.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 shutdown_connections 2023-10-08T01:11:26.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd3f4066f30 0x7fd3f40693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd4140cb900 0x7fd414150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4141566e0 0x7fd41415d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 --2- 172.21.15.119:0/1283972137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd414156310 0x7fd4140b1770 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 >> 172.21.15.119:0/1283972137 conn(0x7fd4140cc1f0 msgr2=0x7fd41406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:26.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 shutdown_connections 2023-10-08T01:11:26.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.042+0000 7fd41359e700 1 -- 172.21.15.119:0/1283972137 wait complete. 2023-10-08T01:11:26.061 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T01:11:26.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:11:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.243+0000 7fecfdbff700 1 Processor -- start 2023-10-08T01:11:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.243+0000 7fecfdbff700 1 -- start start 2023-10-08T01:11:26.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.244+0000 7fecfdbff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fecf806c0a0 0x7fecf80caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.244+0000 7fecfdbff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fecf815df50 0x7fecf815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.244+0000 7fecfdbff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf815d7e0 con 0x7fecf806bcd0 2023-10-08T01:11:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.244+0000 7fecfdbff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf815dac0 con 0x7fecf806c0a0 2023-10-08T01:11:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.244+0000 7fecfdbff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecf815aa30 con 0x7fecf815df50 2023-10-08T01:11:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf77fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fecf806bcd0 0x7fecf815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42482/0 (socket says 172.21.15.119:42482) 2023-10-08T01:11:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf77fe700 1 -- 172.21.15.119:0/1176726955 learned_addr learned my addr 172.21.15.119:0/1176726955 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:26.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf6ffd700 1 --2- 172.21.15.119:0/1176726955 >> v2:172.21.15.119:3300/0 conn(0x7fecf806c0a0 0x7fecf80caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf67fc700 1 -- 172.21.15.119:0/1176726955 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728166212 0 0) 0x7fecf815d7e0 con 0x7fecf806bcd0 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf7fff700 1 --2- 172.21.15.119:0/1176726955 >> v2:172.21.15.184:3300/0 conn(0x7fecf815df50 0x7fecf815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf67fc700 1 -- 172.21.15.119:0/1176726955 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece0003570 con 0x7fecf806bcd0 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf6ffd700 1 -- 172.21.15.119:0/1176726955 >> v2:172.21.15.184:3300/0 conn(0x7fecf815df50 msgr2=0x7fecf815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf6ffd700 1 --2- 172.21.15.119:0/1176726955 >> v2:172.21.15.184:3300/0 conn(0x7fecf815df50 0x7fecf815e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf6ffd700 1 -- 172.21.15.119:0/1176726955 >> v1:172.21.15.119:6789/0 conn(0x7fecf806bcd0 legacy=0x7fecf815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf6ffd700 1 -- 172.21.15.119:0/1176726955 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf815ad10 con 0x7fecf806c0a0 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.245+0000 7fecf6ffd700 1 --2- 172.21.15.119:0/1176726955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80caa90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fecf815d720 tx=0x7fecec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fab76ac6552e8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:26.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecf67fc700 1 -- 172.21.15.119:0/1176726955 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec011630 con 0x7fecf806c0a0 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecf67fc700 1 -- 172.21.15.119:0/1176726955 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec011790 con 0x7fecf806c0a0 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecf67fc700 1 -- 172.21.15.119:0/1176726955 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec011b30 con 0x7fecf806c0a0 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecfdbff700 1 -- 172.21.15.119:0/1176726955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 msgr2=0x7fecf80caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1176726955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80caa90 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fecf815d720 tx=0x7fecec0091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecfdbff700 1 -- 172.21.15.119:0/1176726955 shutdown_connections 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1176726955 >> v2:172.21.15.184:3300/0 conn(0x7fecf815df50 0x7fecf815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1176726955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80caa90 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecfdbff700 1 -- 172.21.15.119:0/1176726955 >> 172.21.15.119:0/1176726955 conn(0x7fecf80ce600 msgr2=0x7fecf80cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecfdbff700 1 -- 172.21.15.119:0/1176726955 shutdown_connections 2023-10-08T01:11:26.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.246+0000 7fecfdbff700 1 -- 172.21.15.119:0/1176726955 wait complete. 2023-10-08T01:11:26.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.248+0000 7fecfdbff700 1 Processor -- start 2023-10-08T01:11:26.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.248+0000 7fecfdbff700 1 -- start start 2023-10-08T01:11:26.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecfdbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecf806bcd0 0x7fecf814cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecfdbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecfdbff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecf815df50 0x7fecf8158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecfdbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80c71d0 con 0x7fecf806c0a0 2023-10-08T01:11:26.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecfdbff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf814d0e0 con 0x7fecf815df50 2023-10-08T01:11:26.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecfdbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf80c65c0 con 0x7fecf806bcd0 2023-10-08T01:11:26.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52376/0 (socket says 172.21.15.119:52376) 2023-10-08T01:11:26.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 -- 172.21.15.119:0/1170470164 learned_addr learned my addr 172.21.15.119:0/1170470164 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 -- 172.21.15.119:0/1170470164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecf806bcd0 msgr2=0x7fecf814cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecf806bcd0 0x7fecf814cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 -- 172.21.15.119:0/1170470164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecf815df50 msgr2=0x7fecf8158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecf815df50 0x7fecf8158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 -- 172.21.15.119:0/1170470164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec009420 con 0x7fecf806c0a0 2023-10-08T01:11:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.249+0000 7fecf6ffd700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80c7d90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fecf8069220 tx=0x7fecec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.250+0000 7fecdf7fe700 1 -- 172.21.15.119:0/1170470164 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec011b10 con 0x7fecf806c0a0 2023-10-08T01:11:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.250+0000 7fecdf7fe700 1 -- 172.21.15.119:0/1170470164 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec0253f0 con 0x7fecf806c0a0 2023-10-08T01:11:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.250+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf80c68a0 con 0x7fecf806c0a0 2023-10-08T01:11:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.250+0000 7fecdf7fe700 1 -- 172.21.15.119:0/1170470164 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0257d0 con 0x7fecf806c0a0 2023-10-08T01:11:26.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.250+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf80c6e60 con 0x7fecf806c0a0 2023-10-08T01:11:26.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.251+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf804c3c0 con 0x7fecf806c0a0 2023-10-08T01:11:26.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.252+0000 7fecdf7fe700 1 -- 172.21.15.119:0/1170470164 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecec025c00 con 0x7fecf806c0a0 2023-10-08T01:11:26.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.252+0000 7fecdf7fe700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fece4066f30 0x7fece40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.253+0000 7fecdf7fe700 1 -- 172.21.15.119:0/1170470164 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fecec085f50 con 0x7fecf806c0a0 2023-10-08T01:11:26.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.253+0000 7fecf77fe700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fece4066f30 0x7fece40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.253+0000 7fecf77fe700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fece4066f30 0x7fece40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fece80058d0 tx=0x7fece80079d0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:26.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.257+0000 7fecdf7fe700 1 -- 172.21.15.119:0/1170470164 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecec052f90 con 0x7fecf806c0a0 2023-10-08T01:11:26.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.444+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecf80c7310 con 0x7fece4066f30 2023-10-08T01:11:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.454+0000 7fecdf7fe700 1 -- 172.21.15.119:0/1170470164 <== mgr.6036 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278393 (secure 0 0 0) 0x7fecf80c7310 con 0x7fece4066f30 2023-10-08T01:11:26.459 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:26.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fece4066f30 msgr2=0x7fece40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fece4066f30 0x7fece40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fece80058d0 tx=0x7fece80079d0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 msgr2=0x7fecf80c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80c7d90 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fecf8069220 tx=0x7fecec017000 comp rx=0 tx=0).stop 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 shutdown_connections 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fece4066f30 0x7fece40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecf806bcd0 0x7fecf814cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecf815df50 0x7fecf8158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 --2- 172.21.15.119:0/1170470164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecf806c0a0 0x7fecf80c7d90 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 >> 172.21.15.119:0/1170470164 conn(0x7fecf80ce600 msgr2=0x7fecf815b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 shutdown_connections 2023-10-08T01:11:26.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.458+0000 7fecfdbff700 1 -- 172.21.15.119:0/1170470164 wait complete. 2023-10-08T01:11:26.463 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:11:26.477 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:11:25.658802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2435592,"kb_used_data":8136,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752539128,"statfs":{"total":773094113280,"available":770600067072,"internally_reserved":0,"allocated":8331264,"data_stored":4578576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952619"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.714381+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:38.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711167+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:07:46.793070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713886+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712131+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:48.896753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707326+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:12.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.714161+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711676+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707697+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707295+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:52.202279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710708+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:57.061923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707296+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:35.034982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.708520+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711077+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:36:59.754487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713164+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:04.435348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711131+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707607+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:33.628997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710023+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:56:08.516088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713601+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711109+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713510+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710611+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:36:02.053732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.709892+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:37.752999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711048+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.709935+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:04.435348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712856+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:12.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713824+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:37.752999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711519+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:56.568531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710130+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:30.461191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707220+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:58.819788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710407+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:46.734068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710209+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:11.596616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712853+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:14:07.853881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"66'161","reported_seq":366,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-08T01:11:17.136357+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:11:17.136357+0000","last_peered":"2023-10-08T01:11:17.136357+0000","last_clean":"2023-10-08T01:11:17.136357+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:11:17.136357+0000","last_undegraded":"2023-10-08T01:11:17.136357+0000","last_fullsized":"2023-10-08T01:11:17.136357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575037,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":1015098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.503},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.93000000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.876},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.85299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.66900000000000004}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.92800000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.54800000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.434},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46200000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.45300000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.68400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":1015098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.238},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.7370000000000001},"last":0.38}]},{"osd":3,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":0.90700000000000003,"15min":1.0489999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.49299999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.629,"5min":0.53000000000000003,"15min":0.45100000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.617},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.48899999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.81899999999999995}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2729999999999999,"15min":1.5960000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.71299999999999997,"5min":0.54200000000000004,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.5289999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.3779999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.629,"5min":0.52700000000000002,"15min":0.51200000000000001},"max":{"1min":1.129,"5min":1.2390000000000001,"15min":1.6459999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.80300000000000005,"15min":0.81999999999999995},"min":{"1min":0.67500000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.82899999999999996,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.86899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306352,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065488,"statfs":{"total":96636764160,"available":96323059712,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.41499999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.437}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.80300000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.81499999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.76300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306352,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065488,"statfs":{"total":96636764160,"available":96323059712,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.46100000000000002},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.627,"5min":0.76000000000000001,"15min":0.80100000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.42399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.67400000000000004,"5min":0.82999999999999996,"15min":1.5629999999999999},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.34100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77200000000000002,"5min":0.89000000000000001,"15min":1.3899999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.55600000000000005,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":1.095,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.3240000000000001,"15min":1.413},"last":1.0580000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.64100000000000001,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.56799999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":0.997,"15min":1.431},"last":0.76100000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.032,"5min":1.0600000000000001,"15min":1.458},"last":1.101},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.44500000000000001},"max":{"1min":1.0629999999999999,"5min":1.1499999999999999,"15min":1.52},"last":0.88300000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.46500000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.389},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.65800000000000003,"5min":0.57799999999999996,"15min":0.5},"max":{"1min":1.054,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.91000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310448,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061392,"statfs":{"total":96636764160,"available":96318865408,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.63200000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.56999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":1.0469999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.77400000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.91300000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.87}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.84499999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.89800000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.622},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:11:26.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:11:26.693 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-10-08T01:11:26.693+0000 7f9ce7ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:11:26.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.696+0000 7ff7edf9d700 1 Processor -- start 2023-10-08T01:11:26.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.696+0000 7ff7edf9d700 1 -- start start 2023-10-08T01:11:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7edf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7edf9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7e8150550 0x7ff7e80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7edf9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e806e2a0 con 0x7ff7e80c7240 2023-10-08T01:11:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7edf9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e80c76e0 con 0x7ff7e8150550 2023-10-08T01:11:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7edf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e806b2e0 con 0x7ff7e8150180 2023-10-08T01:11:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7e7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7e8150550 0x7ff7e80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7ecf9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7ecf9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e8157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:32928/0 (socket says 172.21.15.119:32928) 2023-10-08T01:11:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7e7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff7e8150550 0x7ff7e80c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:42688/0 (socket says 172.21.15.119:42688) 2023-10-08T01:11:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7ecf9b700 1 -- 172.21.15.119:0/2034510300 learned_addr learned my addr 172.21.15.119:0/2034510300 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:26.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.698+0000 7ff7e77fe700 1 -- 172.21.15.119:0/2034510300 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777555378 0 0) 0x7ff7e806e2a0 con 0x7ff7e80c7240 2023-10-08T01:11:26.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7e77fe700 1 -- 172.21.15.119:0/2034510300 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7d8003570 con 0x7ff7e80c7240 2023-10-08T01:11:26.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7ecf9b700 1 -- 172.21.15.119:0/2034510300 >> v2:172.21.15.184:3300/0 conn(0x7ff7e8150550 msgr2=0x7ff7e80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:26.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7ecf9b700 1 --2- 172.21.15.119:0/2034510300 >> v2:172.21.15.184:3300/0 conn(0x7ff7e8150550 0x7ff7e80c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7ecf9b700 1 -- 172.21.15.119:0/2034510300 >> v1:172.21.15.184:6789/0 conn(0x7ff7e80c7240 legacy=0x7ff7e8069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7ecf9b700 1 -- 172.21.15.119:0/2034510300 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e80c78e0 con 0x7ff7e8150180 2023-10-08T01:11:26.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7e7fff700 1 --2- 172.21.15.119:0/2034510300 >> v2:172.21.15.184:3300/0 conn(0x7ff7e8150550 0x7ff7e80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7ecf9b700 1 --2- 172.21.15.119:0/2034510300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e8157770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00bc50 tx=0x7ff7dc00b4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a405db15e8d00a26 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7e77fe700 1 -- 172.21.15.119:0/2034510300 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc00db40 con 0x7ff7e8150180 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7e77fe700 1 -- 172.21.15.119:0/2034510300 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc00dca0 con 0x7ff7e8150180 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.699+0000 7ff7e77fe700 1 -- 172.21.15.119:0/2034510300 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc0196b0 con 0x7ff7e8150180 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.700+0000 7ff7edf9d700 1 -- 172.21.15.119:0/2034510300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 msgr2=0x7ff7e8157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.700+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/2034510300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e8157770 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc00bc50 tx=0x7ff7dc00b4f0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.700+0000 7ff7edf9d700 1 -- 172.21.15.119:0/2034510300 shutdown_connections 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.700+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/2034510300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e8157770 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.700+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/2034510300 >> v2:172.21.15.184:3300/0 conn(0x7ff7e8150550 0x7ff7e80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.700+0000 7ff7edf9d700 1 -- 172.21.15.119:0/2034510300 >> 172.21.15.119:0/2034510300 conn(0x7ff7e80cc1f0 msgr2=0x7ff7e80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:26.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.700+0000 7ff7edf9d700 1 -- 172.21.15.119:0/2034510300 shutdown_connections 2023-10-08T01:11:26.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.700+0000 7ff7edf9d700 1 -- 172.21.15.119:0/2034510300 wait complete. 2023-10-08T01:11:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.701+0000 7ff7edf9d700 1 Processor -- start 2023-10-08T01:11:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.701+0000 7ff7edf9d700 1 -- start start 2023-10-08T01:11:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.702+0000 7ff7edf9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80c7240 0x7ff7e8149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.702+0000 7ff7edf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e814a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.702+0000 7ff7edf9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e8150550 0x7ff7e80cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.702+0000 7ff7edf9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e80aef90 con 0x7ff7e80c7240 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.702+0000 7ff7edf9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e80afdb0 con 0x7ff7e8150550 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.702+0000 7ff7edf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e80af1a0 con 0x7ff7e8150180 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80c7240 0x7ff7e8149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80c7240 0x7ff7e8149fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52392/0 (socket says 172.21.15.119:52392) 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 -- 172.21.15.119:0/3116807205 learned_addr learned my addr 172.21.15.119:0/3116807205 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7e7fff700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e814a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ed79c700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e8150550 0x7ff7e80cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 -- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 msgr2=0x7ff7e814a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e814a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 -- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e8150550 msgr2=0x7ff7e80cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e8150550 0x7ff7e80cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 -- 172.21.15.119:0/3116807205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009420 con 0x7ff7e80c7240 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ed79c700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e8150550 0x7ff7e80cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.703+0000 7ff7ecf9b700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80c7240 0x7ff7e8149fb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff7e806b420 tx=0x7ff7dc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.704+0000 7ff7e4ff9700 1 -- 172.21.15.119:0/3116807205 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc010680 con 0x7ff7e80c7240 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.704+0000 7ff7e4ff9700 1 -- 172.21.15.119:0/3116807205 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc0107e0 con 0x7ff7e80c7240 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.704+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc00b790 con 0x7ff7e80c7240 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.704+0000 7ff7e4ff9700 1 -- 172.21.15.119:0/3116807205 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc010ba0 con 0x7ff7e80c7240 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.704+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e80af760 con 0x7ff7e80c7240 2023-10-08T01:11:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.705+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e804c3c0 con 0x7ff7e80c7240 2023-10-08T01:11:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.706+0000 7ff7e4ff9700 1 -- 172.21.15.119:0/3116807205 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7dc0233f0 con 0x7ff7e80c7240 2023-10-08T01:11:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.707+0000 7ff7e4ff9700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7cc066f30 0x7ff7cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.707+0000 7ff7e4ff9700 1 -- 172.21.15.119:0/3116807205 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff7dc014070 con 0x7ff7e80c7240 2023-10-08T01:11:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.707+0000 7ff7e7fff700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7cc066f30 0x7ff7cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.707+0000 7ff7e7fff700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7cc066f30 0x7ff7cc0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff7e814fa90 tx=0x7ff7d40091a0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:26.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.712+0000 7ff7e4ff9700 1 -- 172.21.15.119:0/3116807205 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7dc053530 con 0x7ff7e80c7240 2023-10-08T01:11:26.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.900+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7e806d650 con 0x7ff7cc066f30 2023-10-08T01:11:26.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.908+0000 7ff7e4ff9700 1 -- 172.21.15.119:0/3116807205 <== mgr.6036 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278393 (secure 0 0 0) 0x7ff7e806d650 con 0x7ff7cc066f30 2023-10-08T01:11:26.912 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:26.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7cc066f30 msgr2=0x7ff7cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7cc066f30 0x7ff7cc0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff7e814fa90 tx=0x7ff7d40091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80c7240 msgr2=0x7ff7e8149fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80c7240 0x7ff7e8149fb0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff7e806b420 tx=0x7ff7dc01b000 comp rx=0 tx=0).stop 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 shutdown_connections 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7cc066f30 0x7ff7cc0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7e8150550 0x7ff7e80cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7e80c7240 0x7ff7e8149fb0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 --2- 172.21.15.119:0/3116807205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7e8150180 0x7ff7e814a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 >> 172.21.15.119:0/3116807205 conn(0x7ff7e80cc1f0 msgr2=0x7ff7e8154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 shutdown_connections 2023-10-08T01:11:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:26.912+0000 7ff7edf9d700 1 -- 172.21.15.119:0/3116807205 wait complete. 2023-10-08T01:11:26.916 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:11:26.930 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:11:25.658802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2435592,"kb_used_data":8136,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752539128,"statfs":{"total":773094113280,"available":770600067072,"internally_reserved":0,"allocated":8331264,"data_stored":4578576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952619"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.714381+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:38.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711167+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:07:46.793070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713886+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712131+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:48.896753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707326+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:12.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.714161+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711676+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707697+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707295+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:52.202279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710708+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:57.061923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707296+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:35.034982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.708520+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711077+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:36:59.754487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713164+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:04.435348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711131+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707607+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:33.628997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710023+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:56:08.516088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713601+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711109+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713510+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710611+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:36:02.053732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.709892+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:37.752999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711048+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.709935+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:04.435348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712856+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:12.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713824+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:37.752999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711519+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:56.568531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710130+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:30.461191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707220+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:58.819788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710407+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:46.734068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710209+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:11.596616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712853+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:14:07.853881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"66'161","reported_seq":366,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-08T01:11:17.136357+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:11:17.136357+0000","last_peered":"2023-10-08T01:11:17.136357+0000","last_clean":"2023-10-08T01:11:17.136357+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:11:17.136357+0000","last_undegraded":"2023-10-08T01:11:17.136357+0000","last_fullsized":"2023-10-08T01:11:17.136357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575037,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":1015098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.503},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.93000000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.876},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.85299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.66900000000000004}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.92800000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.54800000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.434},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46200000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.45300000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.68400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":1015098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.238},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.7370000000000001},"last":0.38}]},{"osd":3,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":0.90700000000000003,"15min":1.0489999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.49299999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.629,"5min":0.53000000000000003,"15min":0.45100000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.617},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.48899999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.81899999999999995}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2729999999999999,"15min":1.5960000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.71299999999999997,"5min":0.54200000000000004,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.5289999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.3779999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.629,"5min":0.52700000000000002,"15min":0.51200000000000001},"max":{"1min":1.129,"5min":1.2390000000000001,"15min":1.6459999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.80300000000000005,"15min":0.81999999999999995},"min":{"1min":0.67500000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.82899999999999996,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.86899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306352,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065488,"statfs":{"total":96636764160,"available":96323059712,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.41499999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.437}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.80300000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.81499999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.76300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306352,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065488,"statfs":{"total":96636764160,"available":96323059712,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.46100000000000002},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.627,"5min":0.76000000000000001,"15min":0.80100000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.42399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.67400000000000004,"5min":0.82999999999999996,"15min":1.5629999999999999},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.34100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77200000000000002,"5min":0.89000000000000001,"15min":1.3899999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.55600000000000005,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":1.095,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.3240000000000001,"15min":1.413},"last":1.0580000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.64100000000000001,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.56799999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":0.997,"15min":1.431},"last":0.76100000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.032,"5min":1.0600000000000001,"15min":1.458},"last":1.101},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.44500000000000001},"max":{"1min":1.0629999999999999,"5min":1.1499999999999999,"15min":1.52},"last":0.88300000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.46500000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.389},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.65800000000000003,"5min":0.57799999999999996,"15min":0.5},"max":{"1min":1.054,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.91000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310448,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061392,"statfs":{"total":96636764160,"available":96318865408,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.63200000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.56999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":1.0469999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.77400000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.91300000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.87}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.84499999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.89800000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.622},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:11:26.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:11:27.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.152+0000 7f5df89ff700 1 Processor -- start 2023-10-08T01:11:27.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.152+0000 7f5df89ff700 1 -- start start 2023-10-08T01:11:27.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.153+0000 7f5df89ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5df4069560 0x7f5df40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.153+0000 7f5df89ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df4069ee0 con 0x7f5df4159890 2023-10-08T01:11:27.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.153+0000 7f5df89ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df415a160 con 0x7f5df4159c60 2023-10-08T01:11:27.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.153+0000 7f5df89ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df40d13a0 con 0x7f5df4069560 2023-10-08T01:11:27.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df259c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5df4159890 0x7f5df415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42484/0 (socket says 172.21.15.119:42484) 2023-10-08T01:11:27.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df259c700 1 -- 172.21.15.119:0/1392846379 learned_addr learned my addr 172.21.15.119:0/1392846379 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:27.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df2d9d700 1 --2- 172.21.15.119:0/1392846379 >> v2:172.21.15.119:3300/0 conn(0x7f5df4069560 0x7f5df40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df159a700 1 -- 172.21.15.119:0/1392846379 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026254603 0 0) 0x7f5df4069ee0 con 0x7f5df4159890 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df159a700 1 -- 172.21.15.119:0/1392846379 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ddc003570 con 0x7f5df4159890 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df2d9d700 1 -- 172.21.15.119:0/1392846379 >> v1:172.21.15.184:6789/0 conn(0x7f5df4159c60 legacy=0x7f5df415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df2d9d700 1 -- 172.21.15.119:0/1392846379 >> v1:172.21.15.119:6789/0 conn(0x7f5df4159890 legacy=0x7f5df415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df2d9d700 1 -- 172.21.15.119:0/1392846379 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df415e140 con 0x7f5df4069560 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.154+0000 7f5df2d9d700 1 --2- 172.21.15.119:0/1392846379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4069560 0x7f5df40699a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009e60 tx=0x7f5dec009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8b9462d748e3c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df159a700 1 -- 172.21.15.119:0/1392846379 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec00e070 con 0x7f5df4069560 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df159a700 1 -- 172.21.15.119:0/1392846379 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dec00bb60 con 0x7f5df4069560 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df159a700 1 -- 172.21.15.119:0/1392846379 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dec012430 con 0x7f5df4069560 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df89ff700 1 -- 172.21.15.119:0/1392846379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4069560 msgr2=0x7f5df40699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df89ff700 1 --2- 172.21.15.119:0/1392846379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4069560 0x7f5df40699a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5dec009e60 tx=0x7f5dec009780 comp rx=0 tx=0).stop 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df89ff700 1 -- 172.21.15.119:0/1392846379 shutdown_connections 2023-10-08T01:11:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df89ff700 1 --2- 172.21.15.119:0/1392846379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4069560 0x7f5df40699a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df89ff700 1 -- 172.21.15.119:0/1392846379 >> 172.21.15.119:0/1392846379 conn(0x7f5df4068e20 msgr2=0x7f5df40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:27.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df89ff700 1 -- 172.21.15.119:0/1392846379 shutdown_connections 2023-10-08T01:11:27.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.155+0000 7f5df89ff700 1 -- 172.21.15.119:0/1392846379 wait complete. 2023-10-08T01:11:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.157+0000 7f5df89ff700 1 Processor -- start 2023-10-08T01:11:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.157+0000 7f5df89ff700 1 -- start start 2023-10-08T01:11:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df89ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df4069560 0x7f5df40c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df89ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4159890 0x7f5df40b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df89ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df4159c60 0x7f5df41401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df89ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df4155ca0 con 0x7f5df4159890 2023-10-08T01:11:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df89ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df4156220 con 0x7f5df4159c60 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df89ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df40b5ff0 con 0x7f5df4069560 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4159890 0x7f5df40b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4159890 0x7f5df40b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52412/0 (socket says 172.21.15.119:52412) 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 -- 172.21.15.119:0/846121137 learned_addr learned my addr 172.21.15.119:0/846121137 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 -- 172.21.15.119:0/846121137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df4069560 msgr2=0x7f5df40c67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df4069560 0x7f5df40c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 -- 172.21.15.119:0/846121137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df4159c60 msgr2=0x7f5df41401f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df4159c60 0x7f5df41401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 -- 172.21.15.119:0/846121137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dec009a20 con 0x7f5df4159890 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.158+0000 7f5df1d9b700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4159890 0x7f5df40b54e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c4e0 tx=0x7f5de800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:27.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.159+0000 7f5de27fc700 1 -- 172.21.15.119:0/846121137 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8011590 con 0x7f5df4159890 2023-10-08T01:11:27.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.159+0000 7f5de27fc700 1 -- 172.21.15.119:0/846121137 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de8011710 con 0x7f5df4159890 2023-10-08T01:11:27.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.159+0000 7f5de27fc700 1 -- 172.21.15.119:0/846121137 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8011ad0 con 0x7f5df4159890 2023-10-08T01:11:27.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.159+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df40b62d0 con 0x7f5df4159890 2023-10-08T01:11:27.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.159+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df4152180 con 0x7f5df4159890 2023-10-08T01:11:27.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.160+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df40b64c0 con 0x7f5df4159890 2023-10-08T01:11:27.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.161+0000 7f5de27fc700 1 -- 172.21.15.119:0/846121137 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5de801b3b0 con 0x7f5df4159890 2023-10-08T01:11:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.162+0000 7f5de27fc700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5dd4066f30 0x7f5dd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.162+0000 7f5de27fc700 1 -- 172.21.15.119:0/846121137 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f5de8083c20 con 0x7f5df4159890 2023-10-08T01:11:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.162+0000 7f5df259c700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5dd4066f30 0x7f5dd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.163+0000 7f5df259c700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5dd4066f30 0x7f5dd40693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5de40058d0 tx=0x7f5de40079d0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:27.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.166+0000 7f5de27fc700 1 -- 172.21.15.119:0/846121137 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5df40b64c0 con 0x7f5df4159890 2023-10-08T01:11:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.355+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5df40b64c0 con 0x7f5dd4066f30 2023-10-08T01:11:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.364+0000 7f5de27fc700 1 -- 172.21.15.119:0/846121137 <== mgr.6036 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278393 (secure 0 0 0) 0x7f5df40b64c0 con 0x7f5dd4066f30 2023-10-08T01:11:27.367 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5dd4066f30 msgr2=0x7f5dd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5dd4066f30 0x7f5dd40693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5de40058d0 tx=0x7f5de40079d0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4159890 msgr2=0x7f5df40b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4159890 0x7f5df40b54e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c4e0 tx=0x7f5de800c8a0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 shutdown_connections 2023-10-08T01:11:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5dd4066f30 0x7f5dd40693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5df4069560 0x7f5df40c67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5df4159c60 0x7f5df41401f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 --2- 172.21.15.119:0/846121137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5df4159890 0x7f5df40b54e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 >> 172.21.15.119:0/846121137 conn(0x7f5df4068e20 msgr2=0x7f5df4152e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 shutdown_connections 2023-10-08T01:11:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.367+0000 7f5df89ff700 1 -- 172.21.15.119:0/846121137 wait complete. 2023-10-08T01:11:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:11:27.386 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:11:25.658802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2435592,"kb_used_data":8136,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752539128,"statfs":{"total":773094113280,"available":770600067072,"internally_reserved":0,"allocated":8331264,"data_stored":4578576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.952619"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.714381+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:38.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711167+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:07:46.793070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713886+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712131+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:48.896753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707326+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:12.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.714161+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711676+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707697+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707295+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:52.202279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710708+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:57.061923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707296+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:35.034982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.708520+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711077+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:36:59.754487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713164+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:04.435348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711131+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707607+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:33.628997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710023+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:56:08.516088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713601+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711109+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713510+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710611+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:36:02.053732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.709892+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:37.752999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711048+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.709935+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:04.435348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712856+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:12.152323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.713824+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:37.752999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.711519+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:56.568531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710130+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:30.461191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.707220+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:58.819788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710407+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:46.734068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.710209+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:11.596616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:11:20.703453+0000","last_change":"2023-10-08T01:11:20.703453+0000","last_active":"2023-10-08T01:11:20.703453+0000","last_peered":"2023-10-08T01:11:20.703453+0000","last_clean":"2023-10-08T01:11:20.703453+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:20.703453+0000","last_undegraded":"2023-10-08T01:11:20.703453+0000","last_fullsized":"2023-10-08T01:11:20.703453+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:11:19.696010+0000","last_change":"2023-10-08T01:11:19.712853+0000","last_active":"2023-10-08T01:11:19.696010+0000","last_peered":"2023-10-08T01:11:19.696010+0000","last_clean":"2023-10-08T01:11:19.696010+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:11:19.696010+0000","last_undegraded":"2023-10-08T01:11:19.696010+0000","last_fullsized":"2023-10-08T01:11:19.696010+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:14:07.853881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"66'161","reported_seq":366,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-10-08T01:11:17.136357+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:11:17.136357+0000","last_peered":"2023-10-08T01:11:17.136357+0000","last_clean":"2023-10-08T01:11:17.136357+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:11:17.136357+0000","last_undegraded":"2023-10-08T01:11:17.136357+0000","last_fullsized":"2023-10-08T01:11:17.136357+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575037,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":1015098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.72799999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.77100000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.503},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.93000000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.81699999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.876},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.85299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575037,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.66900000000000004}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.86399999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.92800000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.54800000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.58899999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575037,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.434},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.46200000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.45300000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.65800000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.56599999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.68400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575036,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":1015098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.238},"last":0.42099999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.7370000000000001},"last":0.38}]},{"osd":3,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":0.90700000000000003,"15min":1.0489999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.49299999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.629,"5min":0.53000000000000003,"15min":0.45100000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.617},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.48899999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.81899999999999995}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2729999999999999,"15min":1.5960000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.71299999999999997,"5min":0.54200000000000004,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.5289999999999999},"last":0.77400000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.3779999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.629,"5min":0.52700000000000002,"15min":0.51200000000000001},"max":{"1min":1.129,"5min":1.2390000000000001,"15min":1.6459999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.80300000000000005,"15min":0.81999999999999995},"min":{"1min":0.67500000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.82899999999999996,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.86899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":306352,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065488,"statfs":{"total":96636764160,"available":96323059712,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.41499999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.437}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.73699999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.80300000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.81499999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.76300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306352,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065488,"statfs":{"total":96636764160,"available":96323059712,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.46100000000000002},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.627,"5min":0.76000000000000001,"15min":0.80100000000000005},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.36299999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.42399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.67400000000000004,"5min":0.82999999999999996,"15min":1.5629999999999999},"last":0.438}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.34100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77200000000000002,"5min":0.89000000000000001,"15min":1.3899999999999999},"last":0.58799999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.55600000000000005,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":1.095,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.3240000000000001,"15min":1.413},"last":1.0580000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.64100000000000001,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.56799999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":0.997,"15min":1.431},"last":0.76100000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.032,"5min":1.0600000000000001,"15min":1.458},"last":1.101},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.44500000000000001},"max":{"1min":1.0629999999999999,"5min":1.1499999999999999,"15min":1.52},"last":0.88300000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.46500000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.389},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.65800000000000003,"5min":0.57799999999999996,"15min":0.5},"max":{"1min":1.054,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.91000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310448,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061392,"statfs":{"total":96636764160,"available":96318865408,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.63200000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.68000000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.70099999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.56999999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302256,"kb_used_data":872,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069584,"statfs":{"total":96636764160,"available":96327254016,"internally_reserved":0,"allocated":892928,"data_stored":424730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":1.0469999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.77400000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.91300000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.87}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.84499999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.89800000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.622},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.66300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:11:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2023-10-08T01:11:27.392 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.393 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-10-08T01:11:27.394 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-10-08T01:11:27.395 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-10-08T01:11:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2023-10-08T01:11:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-10-08T01:11:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.396 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2023-10-08T01:11:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2023-10-08T01:11:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-10-08T01:11:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2023-10-08T01:11:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.397 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-10-08T01:11:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-10-08T01:11:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2023-10-08T01:11:27.398 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-10-08T01:11:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.399 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-10-08T01:11:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2023-10-08T01:11:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.401 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-10-08T01:11:27.402 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-10-08T01:11:27.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.714381+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:27:38.586089+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:11:27.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-10-08T01:11:27.403 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.403 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-10-08T01:11:27.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.711167+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:07:46.793070+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:11:27.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-10-08T01:11:27.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-10-08T01:11:27.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.713886+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:11:27.404 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-10-08T01:11:27.404 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-10-08T01:11:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.712131+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:39:48.896753+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:11:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-10-08T01:11:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-10-08T01:11:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.707326+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:10:12.152323+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:11:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-10-08T01:11:27.405 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-10-08T01:11:27.406 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.714161+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:11:27.406 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-10-08T01:11:27.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-10-08T01:11:27.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.711676+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:11:27.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-10-08T01:11:27.407 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.407 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.707697+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.707295+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:41:52.202279+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-10-08T01:11:27.408 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.710708+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:52:57.061923+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.707296+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:20:35.034982+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.708520+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:11:27.409 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-10-08T01:11:27.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-10-08T01:11:27.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.711077+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:36:59.754487+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:11:27.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-10-08T01:11:27.410 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.410 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.713164+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:43:04.435348+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.711131+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-10-08T01:11:27.411 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.707607+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:27:33.628997+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:11:27.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-10-08T01:11:27.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-10-08T01:11:27.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.710023+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:56:08.516088+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:11:27.412 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-10-08T01:11:27.412 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-10-08T01:11:27.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.713601+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:11:27.413 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-10-08T01:11:27.413 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-10-08T01:11:27.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.711109+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:49:12.630230+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:11:27.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-10-08T01:11:27.414 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.414 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.713510+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.710611+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:36:02.053732+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-10-08T01:11:27.415 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.709892+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:45:37.752999+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:11:27.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-10-08T01:11:27.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-10-08T01:11:27.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.711048+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:06:55.453416+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:11:27.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-10-08T01:11:27.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.709935+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:43:04.435348+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.712856+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:10:12.152323+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.713824+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:45:37.752999+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.711519+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:42:56.568531+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-10-08T01:11:27.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.710130+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:31:30.461191+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.707220+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:34:58.819788+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.710407+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:22:46.734068+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-10-08T01:11:27.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-10-08T01:11:27.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.710209+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:34:11.596616+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:11:27.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-10-08T01:11:27.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:11:20.703453+0000', 'last_change': '2023-10-08T01:11:20.703453+0000', 'last_active': '2023-10-08T01:11:20.703453+0000', 'last_peered': '2023-10-08T01:11:20.703453+0000', 'last_clean': '2023-10-08T01:11:20.703453+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:20.703453+0000', 'last_undegraded': '2023-10-08T01:11:20.703453+0000', 'last_fullsized': '2023-10-08T01:11:20.703453+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:20.703453+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:11:27.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-10-08T01:11:27.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:11:19.696010+0000', 'last_change': '2023-10-08T01:11:19.712853+0000', 'last_active': '2023-10-08T01:11:19.696010+0000', 'last_peered': '2023-10-08T01:11:19.696010+0000', 'last_clean': '2023-10-08T01:11:19.696010+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:11:19.696010+0000', 'last_undegraded': '2023-10-08T01:11:19.696010+0000', 'last_fullsized': '2023-10-08T01:11:19.696010+0000', 'mapping_epoch': 74, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 74, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'last_clean_scrub_stamp': '2023-10-08T01:11:19.696010+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:14:07.853881+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:11:27.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T01:11:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42f59e700 1 Processor -- start 2023-10-08T01:11:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42f59e700 1 -- start start 2023-10-08T01:11:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42f59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb4301559a0 0x7fb430069870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300ac210 0x7fb430069db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42f59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4300c6810 con 0x7fb430155d70 2023-10-08T01:11:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42f59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4300c7390 con 0x7fb4301559a0 2023-10-08T01:11:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300c6a60 con 0x7fb4300ac210 2023-10-08T01:11:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42ed9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb430155d70 0x7fb43006deb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42498/0 (socket says 172.21.15.119:42498) 2023-10-08T01:11:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42ed9d700 1 -- 172.21.15.119:0/3673069077 learned_addr learned my addr 172.21.15.119:0/3673069077 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42e59c700 1 --2- 172.21.15.119:0/3673069077 >> v2:172.21.15.184:3300/0 conn(0x7fb4301559a0 0x7fb430069870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb427fff700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300ac210 0x7fb430069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.607+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3367136145 0 0) 0x7fb4300c6810 con 0x7fb430155d70 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb41c003570 con 0x7fb430155d70 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1077954622 0 0) 0x7fb41c003570 con 0x7fb430155d70 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300ac210 msgr2=0x7fb430069db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300ac210 0x7fb430069db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 >> v2:172.21.15.184:3300/0 conn(0x7fb4301559a0 msgr2=0x7fb430069870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 --2- 172.21.15.119:0/3673069077 >> v2:172.21.15.184:3300/0 conn(0x7fb4301559a0 0x7fb430069870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4300c6d40 con 0x7fb430155d70 2023-10-08T01:11:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fb4280017f0 con 0x7fb430155d70 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 >> v1:172.21.15.119:6789/0 conn(0x7fb430155d70 legacy=0x7fb43006deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb41c003a60 0x7fb41c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41c004810 0x7fb41c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300ac210 0x7fb41c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300c7390 con 0x7fb41c004810 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300c6a60 con 0x7fb41c003a60 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c00ba60 con 0x7fb4300ac210 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb427fff700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41c004810 0x7fb41c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42e59c700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb41c003a60 0x7fb41c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.608+0000 7fb42ed9d700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300ac210 0x7fb41c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb427fff700 1 -- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300ac210 msgr2=0x7fb41c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb427fff700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4300ac210 0x7fb41c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb427fff700 1 -- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb41c003a60 msgr2=0x7fb41c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb427fff700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb41c003a60 0x7fb41c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb427fff700 1 -- 172.21.15.119:0/3673069077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c003570 con 0x7fb41c004810 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb427fff700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41c004810 0x7fb41c004c80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb41800ade0 tx=0x7fb41800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6315818aa582ff6a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41800fa30 con 0x7fb41c004810 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41800fb90 con 0x7fb41c004810 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.609+0000 7fb42dd9b700 1 -- 172.21.15.119:0/3673069077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41800fd20 con 0x7fb41c004810 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.610+0000 7fb42f59e700 1 -- 172.21.15.119:0/3673069077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41c004810 msgr2=0x7fb41c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.610+0000 7fb42f59e700 1 --2- 172.21.15.119:0/3673069077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb41c004810 0x7fb41c004c80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb41800ade0 tx=0x7fb41800a330 comp rx=0 tx=0).stop 2023-10-08T01:11:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.610+0000 7fb42ed9d700 1 -- 172.21.15.119:0/3673069077 reap_dead start 2023-10-08T01:11:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.610+0000 7fb42f59e700 1 -- 172.21.15.119:0/3673069077 shutdown_connections 2023-10-08T01:11:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.610+0000 7fb42f59e700 1 -- 172.21.15.119:0/3673069077 >> 172.21.15.119:0/3673069077 conn(0x7fb4300ccca0 msgr2=0x7fb4300c8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.610+0000 7fb42f59e700 1 -- 172.21.15.119:0/3673069077 shutdown_connections 2023-10-08T01:11:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.610+0000 7fb42f59e700 1 -- 172.21.15.119:0/3673069077 wait complete. 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.611+0000 7fb42f59e700 1 Processor -- start 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.611+0000 7fb42f59e700 1 -- start start 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb42f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb41c004810 0x7fb4300c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb42f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4300c3c90 0x7fb430166aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb42f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4300a6550 0x7fb4300a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb42f59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300c51b0 con 0x7fb4300c3c90 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb42f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300a6f00 con 0x7fb4300a6550 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb42f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4300c4200 con 0x7fb41c004810 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb427fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4300c3c90 0x7fb430166aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb427fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4300c3c90 0x7fb430166aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52436/0 (socket says 172.21.15.119:52436) 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb427fff700 1 -- 172.21.15.119:0/1562185866 learned_addr learned my addr 172.21.15.119:0/1562185866 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb42e59c700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb41c004810 0x7fb4300c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.612+0000 7fb42ed9d700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4300a6550 0x7fb4300a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb427fff700 1 -- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb41c004810 msgr2=0x7fb4300c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb427fff700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb41c004810 0x7fb4300c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb427fff700 1 -- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4300a6550 msgr2=0x7fb4300a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb427fff700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4300a6550 0x7fb4300a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb427fff700 1 -- 172.21.15.119:0/1562185866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4300c44e0 con 0x7fb4300c3c90 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb42e59c700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb41c004810 0x7fb4300c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb42ed9d700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4300a6550 0x7fb4300a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb427fff700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4300c3c90 0x7fb430166aa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb41800f8a0 tx=0x7fb41801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb4267fc700 1 -- 172.21.15.119:0/1562185866 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418012460 con 0x7fb4300c3c90 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb4267fc700 1 -- 172.21.15.119:0/1562185866 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4180125c0 con 0x7fb4300c3c90 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb4267fc700 1 -- 172.21.15.119:0/1562185866 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418012980 con 0x7fb4300c3c90 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4300c47c0 con 0x7fb4300c3c90 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.613+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb430166fe0 con 0x7fb4300c3c90 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.614+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4300c4c60 con 0x7fb4300c3c90 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.615+0000 7fb4267fc700 1 -- 172.21.15.119:0/1562185866 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb418025400 con 0x7fb4300c3c90 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.616+0000 7fb4267fc700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb410066f30 0x7fb4100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.616+0000 7fb4267fc700 1 -- 172.21.15.119:0/1562185866 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb418016070 con 0x7fb4300c3c90 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.616+0000 7fb42e59c700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb410066f30 0x7fb4100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.617+0000 7fb42e59c700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb410066f30 0x7fb4100693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb420006f00 tx=0x7fb420008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:27.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.620+0000 7fb4267fc700 1 -- 172.21.15.119:0/1562185866 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb418053790 con 0x7fb4300c3c90 2023-10-08T01:11:27.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.889+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb43004c3c0 con 0x7fb4300c3c90 2023-10-08T01:11:27.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.889+0000 7fb4267fc700 1 -- 172.21.15.119:0/1562185866 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1111 (secure 0 0 0) 0x7fb418084e60 con 0x7fb4300c3c90 2023-10-08T01:11:27.889 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:27.889 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":964,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":77,"num_osds":8,"num_up_osds":8,"osd_up_since":1696726535,"num_in_osds":8,"osd_in_since":1696726526,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2494046208,"bytes_avail":770600067072,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":32,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6225}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-08T01:10:26.477565+0000","services":{}},"progress_events":{}} 2023-10-08T01:11:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.891+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb410066f30 msgr2=0x7fb4100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.891+0000 7fb42f59e700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb410066f30 0x7fb4100693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb420006f00 tx=0x7fb420008040 comp rx=0 tx=0).stop 2023-10-08T01:11:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.891+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4300c3c90 msgr2=0x7fb430166aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.891+0000 7fb42f59e700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4300c3c90 0x7fb430166aa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb41800f8a0 tx=0x7fb41801d000 comp rx=0 tx=0).stop 2023-10-08T01:11:27.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.892+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 shutdown_connections 2023-10-08T01:11:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.892+0000 7fb42f59e700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb410066f30 0x7fb4100693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.892+0000 7fb42f59e700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb41c004810 0x7fb4300c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.892+0000 7fb42f59e700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4300a6550 0x7fb4300a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.892+0000 7fb42f59e700 1 --2- 172.21.15.119:0/1562185866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4300c3c90 0x7fb430166aa0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.892+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 >> 172.21.15.119:0/1562185866 conn(0x7fb4300ccca0 msgr2=0x7fb4300cb380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.892+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 shutdown_connections 2023-10-08T01:11:27.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:27.892+0000 7fb42f59e700 1 -- 172.21.15.119:0/1562185866 wait complete. 2023-10-08T01:11:27.909 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T01:11:27.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:11:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.090+0000 7f1a5e19d700 1 Processor -- start 2023-10-08T01:11:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.090+0000 7f1a5e19d700 1 -- start start 2023-10-08T01:11:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.091+0000 7f1a5e19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a580b0bd0 0x7f1a581646a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.091+0000 7f1a5e19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5815a980 con 0x7f1a58157270 2023-10-08T01:11:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.091+0000 7f1a5e19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a5815a050 con 0x7f1a58068aa0 2023-10-08T01:11:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.091+0000 7f1a5e19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5815e310 con 0x7f1a580b0bd0 2023-10-08T01:11:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.091+0000 7f1a5d19b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1a58157270 0x7f1a58068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42512/0 (socket says 172.21.15.119:42512) 2023-10-08T01:11:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.091+0000 7f1a5d19b700 1 -- 172.21.15.119:0/1213280027 learned_addr learned my addr 172.21.15.119:0/1213280027 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:28.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804109367 0 0) 0x7f1a5815a980 con 0x7f1a58157270 2023-10-08T01:11:28.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a44003570 con 0x7f1a58157270 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a5d99c700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a580b0bd0 0x7f1a581646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3803583123 0 0) 0x7f1a44003570 con 0x7f1a58157270 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a580b0bd0 msgr2=0x7f1a581646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a580b0bd0 0x7f1a581646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 >> v1:172.21.15.184:6789/0 conn(0x7f1a58068aa0 legacy=0x7f1a580b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5815a330 con 0x7f1a58157270 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f1a54001470 con 0x7f1a58157270 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 >> v1:172.21.15.119:6789/0 conn(0x7f1a58157270 legacy=0x7f1a58068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a44003a90 0x7f1a44003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:28.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a580b0bd0 0x7f1a44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:28.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a44004d50 0x7f1a440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:28.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4400bbf0 con 0x7f1a44003a90 2023-10-08T01:11:28.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4400bd30 con 0x7f1a44004d50 2023-10-08T01:11:28.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4400bec0 con 0x7f1a580b0bd0 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.092+0000 7f1a5d19b700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a44003a90 0x7f1a44003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a5d99c700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a44004d50 0x7f1a440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a5c99a700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a580b0bd0 0x7f1a44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a5d19b700 1 -- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a580b0bd0 msgr2=0x7f1a44004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a5d19b700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a580b0bd0 0x7f1a44004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a5d19b700 1 -- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a44004d50 msgr2=0x7f1a440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a5d19b700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a44004d50 0x7f1a440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a5d19b700 1 -- 172.21.15.119:0/1213280027 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a44003570 con 0x7f1a44003a90 2023-10-08T01:11:28.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a5d19b700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a44003a90 0x7f1a44003ea0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400c2d0 tx=0x7f1a54001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6c6279c3b9f795 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400cba0 con 0x7f1a44003a90 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5400cd20 con 0x7f1a44003a90 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.093+0000 7f1a4ffff700 1 -- 172.21.15.119:0/1213280027 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54005b70 con 0x7f1a44003a90 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.094+0000 7f1a5e19d700 1 -- 172.21.15.119:0/1213280027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a44003a90 msgr2=0x7f1a44003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.094+0000 7f1a5e19d700 1 --2- 172.21.15.119:0/1213280027 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a44003a90 0x7f1a44003ea0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1a5400c2d0 tx=0x7f1a54001d80 comp rx=0 tx=0).stop 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.094+0000 7f1a5d99c700 1 -- 172.21.15.119:0/1213280027 reap_dead start 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.094+0000 7f1a5e19d700 1 -- 172.21.15.119:0/1213280027 shutdown_connections 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.094+0000 7f1a5e19d700 1 -- 172.21.15.119:0/1213280027 >> 172.21.15.119:0/1213280027 conn(0x7f1a580cd310 msgr2=0x7f1a580cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.094+0000 7f1a5e19d700 1 -- 172.21.15.119:0/1213280027 shutdown_connections 2023-10-08T01:11:28.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.094+0000 7f1a5e19d700 1 -- 172.21.15.119:0/1213280027 wait complete. 2023-10-08T01:11:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.095+0000 7f1a5e19d700 1 Processor -- start 2023-10-08T01:11:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.095+0000 7f1a5e19d700 1 -- start start 2023-10-08T01:11:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.096+0000 7f1a5e19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a44003a90 0x7f1a580ae830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:28.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.096+0000 7f1a5e19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a581534f0 0x7f1a5815eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.096+0000 7f1a5e19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a5815cb70 0x7f1a5815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.096+0000 7f1a5e19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a580cc990 con 0x7f1a581534f0 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.096+0000 7f1a5e19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5815d530 con 0x7f1a5815cb70 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.096+0000 7f1a5e19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5815c690 con 0x7f1a44003a90 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.096+0000 7f1a5c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a581534f0 0x7f1a5815eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a581534f0 0x7f1a5815eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52444/0 (socket says 172.21.15.119:52444) 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5c99a700 1 -- 172.21.15.119:0/3108090989 learned_addr learned my addr 172.21.15.119:0/3108090989 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5d99c700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a5815cb70 0x7f1a5815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5d19b700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a44003a90 0x7f1a580ae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:28.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5c99a700 1 -- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a44003a90 msgr2=0x7f1a580ae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5c99a700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a44003a90 0x7f1a580ae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5c99a700 1 -- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a5815cb70 msgr2=0x7f1a5815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5c99a700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a5815cb70 0x7f1a5815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5c99a700 1 -- 172.21.15.119:0/3108090989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a580ae2f0 con 0x7f1a581534f0 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5d99c700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a5815cb70 0x7f1a5815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5d19b700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a44003a90 0x7f1a580ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5c99a700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a581534f0 0x7f1a5815eb60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800e8a0 tx=0x7f1a48009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a4d7fa700 1 -- 172.21.15.119:0/3108090989 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48013630 con 0x7f1a581534f0 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a4d7fa700 1 -- 172.21.15.119:0/3108090989 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a48013790 con 0x7f1a581534f0 2023-10-08T01:11:28.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a4d7fa700 1 -- 172.21.15.119:0/3108090989 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48013b30 con 0x7f1a581534f0 2023-10-08T01:11:28.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.097+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a580ae5d0 con 0x7f1a581534f0 2023-10-08T01:11:28.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.098+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5815c320 con 0x7f1a581534f0 2023-10-08T01:11:28.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.098+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5804c3c0 con 0x7f1a581534f0 2023-10-08T01:11:28.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.099+0000 7f1a4d7fa700 1 -- 172.21.15.119:0/3108090989 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a4801c400 con 0x7f1a581534f0 2023-10-08T01:11:28.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.100+0000 7f1a4d7fa700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a3c066f30 0x7f1a3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:28.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.100+0000 7f1a4d7fa700 1 -- 172.21.15.119:0/3108090989 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f1a48084ae0 con 0x7f1a581534f0 2023-10-08T01:11:28.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.100+0000 7f1a5d19b700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a3c066f30 0x7f1a3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:28.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.101+0000 7f1a5d19b700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a3c066f30 0x7f1a3c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a54007c00 tx=0x7f1a54015000 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:28.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.104+0000 7f1a4d7fa700 1 -- 172.21.15.119:0/3108090989 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a48051b20 con 0x7f1a581534f0 2023-10-08T01:11:28.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.296+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1a5815a610 con 0x7f1a3c066f30 2023-10-08T01:11:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.305+0000 7f1a4d7fa700 1 -- 172.21.15.119:0/3108090989 <== mgr.6036 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289216 (secure 0 0 0) 0x7f1a5815a610 con 0x7f1a3c066f30 2023-10-08T01:11:28.309 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a3c066f30 msgr2=0x7f1a3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a3c066f30 0x7f1a3c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a54007c00 tx=0x7f1a54015000 comp rx=0 tx=0).stop 2023-10-08T01:11:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a581534f0 msgr2=0x7f1a5815eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:28.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a581534f0 0x7f1a5815eb60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1a4800e8a0 tx=0x7f1a48009040 comp rx=0 tx=0).stop 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 shutdown_connections 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a3c066f30 0x7f1a3c0693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1a44003a90 0x7f1a580ae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1a5815cb70 0x7f1a5815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 --2- 172.21.15.119:0/3108090989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a581534f0 0x7f1a5815eb60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 >> 172.21.15.119:0/3108090989 conn(0x7f1a580cd310 msgr2=0x7f1a58156500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 shutdown_connections 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:28.309+0000 7f1a5e19d700 1 -- 172.21.15.119:0/3108090989 wait complete. 2023-10-08T01:11:28.312 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:11:28.327 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T01:11:27.659139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2435944,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752538776,"statfs":{"total":773094113280,"available":770599706624,"internally_reserved":0,"allocated":8691712,"data_stored":4747044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.950204"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.742949+0000","last_change":"2023-10-08T01:11:21.734625+0000","last_active":"2023-10-08T01:11:21.742949+0000","last_peered":"2023-10-08T01:11:21.742949+0000","last_clean":"2023-10-08T01:11:21.742949+0000","last_became_active":"2023-10-08T01:11:21.734288+0000","last_became_peered":"2023-10-08T01:11:21.734288+0000","last_unstale":"2023-10-08T01:11:21.742949+0000","last_undegraded":"2023-10-08T01:11:21.742949+0000","last_fullsized":"2023-10-08T01:11:21.742949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:47.741510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.749754+0000","last_change":"2023-10-08T01:11:21.748540+0000","last_active":"2023-10-08T01:11:21.749754+0000","last_peered":"2023-10-08T01:11:21.749754+0000","last_clean":"2023-10-08T01:11:21.749754+0000","last_became_active":"2023-10-08T01:11:21.748276+0000","last_became_peered":"2023-10-08T01:11:21.748276+0000","last_unstale":"2023-10-08T01:11:21.749754+0000","last_undegraded":"2023-10-08T01:11:21.749754+0000","last_fullsized":"2023-10-08T01:11:21.749754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:47.557664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744557+0000","last_change":"2023-10-08T01:11:21.743208+0000","last_active":"2023-10-08T01:11:21.744557+0000","last_peered":"2023-10-08T01:11:21.744557+0000","last_clean":"2023-10-08T01:11:21.744557+0000","last_became_active":"2023-10-08T01:11:21.742955+0000","last_became_peered":"2023-10-08T01:11:21.742955+0000","last_unstale":"2023-10-08T01:11:21.744557+0000","last_undegraded":"2023-10-08T01:11:21.744557+0000","last_fullsized":"2023-10-08T01:11:21.744557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:13.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.749986+0000","last_change":"2023-10-08T01:11:21.744538+0000","last_active":"2023-10-08T01:11:21.749986+0000","last_peered":"2023-10-08T01:11:21.749986+0000","last_clean":"2023-10-08T01:11:21.749986+0000","last_became_active":"2023-10-08T01:11:21.744220+0000","last_became_peered":"2023-10-08T01:11:21.744220+0000","last_unstale":"2023-10-08T01:11:21.749986+0000","last_undegraded":"2023-10-08T01:11:21.749986+0000","last_fullsized":"2023-10-08T01:11:21.749986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:53:19.668372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744350+0000","last_change":"2023-10-08T01:11:21.742526+0000","last_active":"2023-10-08T01:11:21.744350+0000","last_peered":"2023-10-08T01:11:21.744350+0000","last_clean":"2023-10-08T01:11:21.744350+0000","last_became_active":"2023-10-08T01:11:21.742287+0000","last_became_peered":"2023-10-08T01:11:21.742287+0000","last_unstale":"2023-10-08T01:11:21.744350+0000","last_undegraded":"2023-10-08T01:11:21.744350+0000","last_fullsized":"2023-10-08T01:11:21.744350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:14:08.861323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737640+0000","last_change":"2023-10-08T01:11:21.733074+0000","last_active":"2023-10-08T01:11:21.737640+0000","last_peered":"2023-10-08T01:11:21.737640+0000","last_clean":"2023-10-08T01:11:21.737640+0000","last_became_active":"2023-10-08T01:11:21.732795+0000","last_became_peered":"2023-10-08T01:11:21.732795+0000","last_unstale":"2023-10-08T01:11:21.737640+0000","last_undegraded":"2023-10-08T01:11:21.737640+0000","last_fullsized":"2023-10-08T01:11:21.737640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:55:03.140046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.746884+0000","last_change":"2023-10-08T01:11:21.743612+0000","last_active":"2023-10-08T01:11:21.746884+0000","last_peered":"2023-10-08T01:11:21.746884+0000","last_clean":"2023-10-08T01:11:21.746884+0000","last_became_active":"2023-10-08T01:11:21.743367+0000","last_became_peered":"2023-10-08T01:11:21.743367+0000","last_unstale":"2023-10-08T01:11:21.746884+0000","last_undegraded":"2023-10-08T01:11:21.746884+0000","last_fullsized":"2023-10-08T01:11:21.746884+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:58.069365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737603+0000","last_change":"2023-10-08T01:11:21.732607+0000","last_active":"2023-10-08T01:11:21.737603+0000","last_peered":"2023-10-08T01:11:21.737603+0000","last_clean":"2023-10-08T01:11:21.737603+0000","last_became_active":"2023-10-08T01:11:21.732313+0000","last_became_peered":"2023-10-08T01:11:21.732313+0000","last_unstale":"2023-10-08T01:11:21.737603+0000","last_undegraded":"2023-10-08T01:11:21.737603+0000","last_fullsized":"2023-10-08T01:11:21.737603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:50:24.923577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.739546+0000","last_change":"2023-10-08T01:11:21.737895+0000","last_active":"2023-10-08T01:11:21.739546+0000","last_peered":"2023-10-08T01:11:21.739546+0000","last_clean":"2023-10-08T01:11:21.739546+0000","last_became_active":"2023-10-08T01:11:21.737574+0000","last_became_peered":"2023-10-08T01:11:21.737574+0000","last_unstale":"2023-10-08T01:11:21.739546+0000","last_undegraded":"2023-10-08T01:11:21.739546+0000","last_fullsized":"2023-10-08T01:11:21.739546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:12.604059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754595+0000","last_change":"2023-10-08T01:11:21.750279+0000","last_active":"2023-10-08T01:11:21.754595+0000","last_peered":"2023-10-08T01:11:21.754595+0000","last_clean":"2023-10-08T01:11:21.754595+0000","last_became_active":"2023-10-08T01:11:21.749755+0000","last_became_peered":"2023-10-08T01:11:21.749755+0000","last_unstale":"2023-10-08T01:11:21.754595+0000","last_undegraded":"2023-10-08T01:11:21.754595+0000","last_fullsized":"2023-10-08T01:11:21.754595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:57.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.758784+0000","last_change":"2023-10-08T01:11:21.754039+0000","last_active":"2023-10-08T01:11:21.758784+0000","last_peered":"2023-10-08T01:11:21.758784+0000","last_clean":"2023-10-08T01:11:21.758784+0000","last_became_active":"2023-10-08T01:11:21.753655+0000","last_became_peered":"2023-10-08T01:11:21.753655+0000","last_unstale":"2023-10-08T01:11:21.758784+0000","last_undegraded":"2023-10-08T01:11:21.758784+0000","last_fullsized":"2023-10-08T01:11:21.758784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:39.593532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.749961+0000","last_change":"2023-10-08T01:11:21.741533+0000","last_active":"2023-10-08T01:11:21.749961+0000","last_peered":"2023-10-08T01:11:21.749961+0000","last_clean":"2023-10-08T01:11:21.749961+0000","last_became_active":"2023-10-08T01:11:21.741269+0000","last_became_peered":"2023-10-08T01:11:21.741269+0000","last_unstale":"2023-10-08T01:11:21.749961+0000","last_undegraded":"2023-10-08T01:11:21.749961+0000","last_fullsized":"2023-10-08T01:11:21.749961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:38:37.419465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737212+0000","last_change":"2023-10-08T01:11:21.734119+0000","last_active":"2023-10-08T01:11:21.737212+0000","last_peered":"2023-10-08T01:11:21.737212+0000","last_clean":"2023-10-08T01:11:21.737212+0000","last_became_active":"2023-10-08T01:11:21.733808+0000","last_became_peered":"2023-10-08T01:11:21.733808+0000","last_unstale":"2023-10-08T01:11:21.737212+0000","last_undegraded":"2023-10-08T01:11:21.737212+0000","last_fullsized":"2023-10-08T01:11:21.737212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:59.827230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.740644+0000","last_change":"2023-10-08T01:11:21.738874+0000","last_active":"2023-10-08T01:11:21.740644+0000","last_peered":"2023-10-08T01:11:21.740644+0000","last_clean":"2023-10-08T01:11:21.740644+0000","last_became_active":"2023-10-08T01:11:21.736697+0000","last_became_peered":"2023-10-08T01:11:21.736697+0000","last_unstale":"2023-10-08T01:11:21.740644+0000","last_undegraded":"2023-10-08T01:11:21.740644+0000","last_fullsized":"2023-10-08T01:11:21.740644+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:07:47.800512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750130+0000","last_change":"2023-10-08T01:11:21.748555+0000","last_active":"2023-10-08T01:11:21.750130+0000","last_peered":"2023-10-08T01:11:21.750130+0000","last_clean":"2023-10-08T01:11:21.750130+0000","last_became_active":"2023-10-08T01:11:21.748276+0000","last_became_peered":"2023-10-08T01:11:21.748276+0000","last_unstale":"2023-10-08T01:11:21.750130+0000","last_undegraded":"2023-10-08T01:11:21.750130+0000","last_fullsized":"2023-10-08T01:11:21.750130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:00.761929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.735680+0000","last_change":"2023-10-08T01:11:21.729641+0000","last_active":"2023-10-08T01:11:21.735680+0000","last_peered":"2023-10-08T01:11:21.735680+0000","last_clean":"2023-10-08T01:11:21.735680+0000","last_became_active":"2023-10-08T01:11:21.729379+0000","last_became_peered":"2023-10-08T01:11:21.729379+0000","last_unstale":"2023-10-08T01:11:21.735680+0000","last_undegraded":"2023-10-08T01:11:21.735680+0000","last_fullsized":"2023-10-08T01:11:21.735680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:13.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.743117+0000","last_change":"2023-10-08T01:11:21.740889+0000","last_active":"2023-10-08T01:11:21.743117+0000","last_peered":"2023-10-08T01:11:21.743117+0000","last_clean":"2023-10-08T01:11:21.743117+0000","last_became_active":"2023-10-08T01:11:21.740575+0000","last_became_peered":"2023-10-08T01:11:21.740575+0000","last_unstale":"2023-10-08T01:11:21.743117+0000","last_undegraded":"2023-10-08T01:11:21.743117+0000","last_fullsized":"2023-10-08T01:11:21.743117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:33.627492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744028+0000","last_change":"2023-10-08T01:11:21.741742+0000","last_active":"2023-10-08T01:11:21.744028+0000","last_peered":"2023-10-08T01:11:21.744028+0000","last_clean":"2023-10-08T01:11:21.744028+0000","last_became_active":"2023-10-08T01:11:21.741447+0000","last_became_peered":"2023-10-08T01:11:21.741447+0000","last_unstale":"2023-10-08T01:11:21.744028+0000","last_undegraded":"2023-10-08T01:11:21.744028+0000","last_fullsized":"2023-10-08T01:11:21.744028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:36.169190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.741640+0000","last_change":"2023-10-08T01:11:21.734404+0000","last_active":"2023-10-08T01:11:21.741640+0000","last_peered":"2023-10-08T01:11:21.741640+0000","last_clean":"2023-10-08T01:11:21.741640+0000","last_became_active":"2023-10-08T01:11:21.734076+0000","last_became_peered":"2023-10-08T01:11:21.734076+0000","last_unstale":"2023-10-08T01:11:21.741640+0000","last_undegraded":"2023-10-08T01:11:21.741640+0000","last_fullsized":"2023-10-08T01:11:21.741640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:14.450422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.748483+0000","last_change":"2023-10-08T01:11:21.742567+0000","last_active":"2023-10-08T01:11:21.748483+0000","last_peered":"2023-10-08T01:11:21.748483+0000","last_clean":"2023-10-08T01:11:21.748483+0000","last_became_active":"2023-10-08T01:11:21.740331+0000","last_became_peered":"2023-10-08T01:11:21.740331+0000","last_unstale":"2023-10-08T01:11:21.748483+0000","last_undegraded":"2023-10-08T01:11:21.748483+0000","last_fullsized":"2023-10-08T01:11:21.748483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:16.987048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750933+0000","last_change":"2023-10-08T01:11:21.749066+0000","last_active":"2023-10-08T01:11:21.750933+0000","last_peered":"2023-10-08T01:11:21.750933+0000","last_clean":"2023-10-08T01:11:21.750933+0000","last_became_active":"2023-10-08T01:11:21.748559+0000","last_became_peered":"2023-10-08T01:11:21.748559+0000","last_unstale":"2023-10-08T01:11:21.750933+0000","last_undegraded":"2023-10-08T01:11:21.750933+0000","last_fullsized":"2023-10-08T01:11:21.750933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:55.596379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.748211+0000","last_change":"2023-10-08T01:11:21.742287+0000","last_active":"2023-10-08T01:11:21.748211+0000","last_peered":"2023-10-08T01:11:21.748211+0000","last_clean":"2023-10-08T01:11:21.748211+0000","last_became_active":"2023-10-08T01:11:21.741967+0000","last_became_peered":"2023-10-08T01:11:21.741967+0000","last_unstale":"2023-10-08T01:11:21.748211+0000","last_undegraded":"2023-10-08T01:11:21.748211+0000","last_fullsized":"2023-10-08T01:11:21.748211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:50.899302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.748047+0000","last_change":"2023-10-08T01:11:21.744406+0000","last_active":"2023-10-08T01:11:21.748047+0000","last_peered":"2023-10-08T01:11:21.748047+0000","last_clean":"2023-10-08T01:11:21.748047+0000","last_became_active":"2023-10-08T01:11:21.744150+0000","last_became_peered":"2023-10-08T01:11:21.744150+0000","last_unstale":"2023-10-08T01:11:21.748047+0000","last_undegraded":"2023-10-08T01:11:21.748047+0000","last_fullsized":"2023-10-08T01:11:21.748047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:44:01.648076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.736837+0000","last_change":"2023-10-08T01:11:21.733418+0000","last_active":"2023-10-08T01:11:21.736837+0000","last_peered":"2023-10-08T01:11:21.736837+0000","last_clean":"2023-10-08T01:11:21.736837+0000","last_became_active":"2023-10-08T01:11:21.733130+0000","last_became_peered":"2023-10-08T01:11:21.733130+0000","last_unstale":"2023-10-08T01:11:21.736837+0000","last_undegraded":"2023-10-08T01:11:21.736837+0000","last_fullsized":"2023-10-08T01:11:21.736837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:14.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750683+0000","last_change":"2023-10-08T01:11:21.747795+0000","last_active":"2023-10-08T01:11:21.750683+0000","last_peered":"2023-10-08T01:11:21.750683+0000","last_clean":"2023-10-08T01:11:21.750683+0000","last_became_active":"2023-10-08T01:11:21.747532+0000","last_became_peered":"2023-10-08T01:11:21.747532+0000","last_unstale":"2023-10-08T01:11:21.750683+0000","last_undegraded":"2023-10-08T01:11:21.750683+0000","last_fullsized":"2023-10-08T01:11:21.750683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:38.760441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750561+0000","last_change":"2023-10-08T01:11:21.748380+0000","last_active":"2023-10-08T01:11:21.750561+0000","last_peered":"2023-10-08T01:11:21.750561+0000","last_clean":"2023-10-08T01:11:21.750561+0000","last_became_active":"2023-10-08T01:11:21.747916+0000","last_became_peered":"2023-10-08T01:11:21.747916+0000","last_unstale":"2023-10-08T01:11:21.750561+0000","last_undegraded":"2023-10-08T01:11:21.750561+0000","last_fullsized":"2023-10-08T01:11:21.750561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:44.365874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737232+0000","last_change":"2023-10-08T01:11:21.731384+0000","last_active":"2023-10-08T01:11:21.737232+0000","last_peered":"2023-10-08T01:11:21.737232+0000","last_clean":"2023-10-08T01:11:21.737232+0000","last_became_active":"2023-10-08T01:11:21.730695+0000","last_became_peered":"2023-10-08T01:11:21.730695+0000","last_unstale":"2023-10-08T01:11:21.737232+0000","last_undegraded":"2023-10-08T01:11:21.737232+0000","last_fullsized":"2023-10-08T01:11:21.737232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:38:37.419465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.753700+0000","last_change":"2023-10-08T01:11:21.750748+0000","last_active":"2023-10-08T01:11:21.753700+0000","last_peered":"2023-10-08T01:11:21.753700+0000","last_clean":"2023-10-08T01:11:21.753700+0000","last_became_active":"2023-10-08T01:11:21.750500+0000","last_became_peered":"2023-10-08T01:11:21.750500+0000","last_unstale":"2023-10-08T01:11:21.753700+0000","last_undegraded":"2023-10-08T01:11:21.753700+0000","last_fullsized":"2023-10-08T01:11:21.753700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:31.468634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750858+0000","last_change":"2023-10-08T01:11:21.749661+0000","last_active":"2023-10-08T01:11:21.750858+0000","last_peered":"2023-10-08T01:11:21.750858+0000","last_clean":"2023-10-08T01:11:21.750858+0000","last_became_active":"2023-10-08T01:11:21.749297+0000","last_became_peered":"2023-10-08T01:11:21.749297+0000","last_unstale":"2023-10-08T01:11:21.750858+0000","last_undegraded":"2023-10-08T01:11:21.750858+0000","last_fullsized":"2023-10-08T01:11:21.750858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:51.269770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.743965+0000","last_change":"2023-10-08T01:11:21.740395+0000","last_active":"2023-10-08T01:11:21.743965+0000","last_peered":"2023-10-08T01:11:21.743965+0000","last_clean":"2023-10-08T01:11:21.743965+0000","last_became_active":"2023-10-08T01:11:21.740122+0000","last_became_peered":"2023-10-08T01:11:21.740122+0000","last_unstale":"2023-10-08T01:11:21.743965+0000","last_undegraded":"2023-10-08T01:11:21.743965+0000","last_fullsized":"2023-10-08T01:11:21.743965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:52.567751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750652+0000","last_change":"2023-10-08T01:11:21.748445+0000","last_active":"2023-10-08T01:11:21.750652+0000","last_peered":"2023-10-08T01:11:21.750652+0000","last_clean":"2023-10-08T01:11:21.750652+0000","last_became_active":"2023-10-08T01:11:21.748193+0000","last_became_peered":"2023-10-08T01:11:21.748193+0000","last_unstale":"2023-10-08T01:11:21.750652+0000","last_undegraded":"2023-10-08T01:11:21.750652+0000","last_fullsized":"2023-10-08T01:11:21.750652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:05.442790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737327+0000","last_change":"2023-10-08T01:11:21.734342+0000","last_active":"2023-10-08T01:11:21.737327+0000","last_peered":"2023-10-08T01:11:21.737327+0000","last_clean":"2023-10-08T01:11:21.737327+0000","last_became_active":"2023-10-08T01:11:21.734048+0000","last_became_peered":"2023-10-08T01:11:21.734048+0000","last_unstale":"2023-10-08T01:11:21.737327+0000","last_undegraded":"2023-10-08T01:11:21.737327+0000","last_fullsized":"2023-10-08T01:11:21.737327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:52.567751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751083+0000","last_change":"2023-10-08T01:11:21.743433+0000","last_active":"2023-10-08T01:11:21.751083+0000","last_peered":"2023-10-08T01:11:21.751083+0000","last_clean":"2023-10-08T01:11:21.751083+0000","last_became_active":"2023-10-08T01:11:21.742926+0000","last_became_peered":"2023-10-08T01:11:21.742926+0000","last_unstale":"2023-10-08T01:11:21.751083+0000","last_undegraded":"2023-10-08T01:11:21.751083+0000","last_fullsized":"2023-10-08T01:11:21.751083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:47:13.443993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.760540+0000","last_change":"2023-10-08T01:11:20.731718+0000","last_active":"2023-10-08T01:11:21.760540+0000","last_peered":"2023-10-08T01:11:21.760540+0000","last_clean":"2023-10-08T01:11:21.760540+0000","last_became_active":"2023-10-08T01:11:20.731471+0000","last_became_peered":"2023-10-08T01:11:20.731471+0000","last_unstale":"2023-10-08T01:11:21.760540+0000","last_undegraded":"2023-10-08T01:11:21.760540+0000","last_fullsized":"2023-10-08T01:11:21.760540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:33.628997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751641+0000","last_change":"2023-10-08T01:11:21.745894+0000","last_active":"2023-10-08T01:11:21.751641+0000","last_peered":"2023-10-08T01:11:21.751641+0000","last_clean":"2023-10-08T01:11:21.751641+0000","last_became_active":"2023-10-08T01:11:21.745513+0000","last_became_peered":"2023-10-08T01:11:21.745513+0000","last_unstale":"2023-10-08T01:11:21.751641+0000","last_undegraded":"2023-10-08T01:11:21.751641+0000","last_fullsized":"2023-10-08T01:11:21.751641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:57.575973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763890+0000","last_change":"2023-10-08T01:11:20.735524+0000","last_active":"2023-10-08T01:11:21.763890+0000","last_peered":"2023-10-08T01:11:21.763890+0000","last_clean":"2023-10-08T01:11:21.763890+0000","last_became_active":"2023-10-08T01:11:20.735143+0000","last_became_peered":"2023-10-08T01:11:20.735143+0000","last_unstale":"2023-10-08T01:11:21.763890+0000","last_undegraded":"2023-10-08T01:11:21.763890+0000","last_fullsized":"2023-10-08T01:11:21.763890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:48.915842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751650+0000","last_change":"2023-10-08T01:11:21.747675+0000","last_active":"2023-10-08T01:11:21.751650+0000","last_peered":"2023-10-08T01:11:21.751650+0000","last_clean":"2023-10-08T01:11:21.751650+0000","last_became_active":"2023-10-08T01:11:21.747420+0000","last_became_peered":"2023-10-08T01:11:21.747420+0000","last_unstale":"2023-10-08T01:11:21.751650+0000","last_undegraded":"2023-10-08T01:11:21.751650+0000","last_fullsized":"2023-10-08T01:11:21.751650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:53:19.668372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.762247+0000","last_change":"2023-10-08T01:11:20.725920+0000","last_active":"2023-10-08T01:11:21.762247+0000","last_peered":"2023-10-08T01:11:21.762247+0000","last_clean":"2023-10-08T01:11:21.762247+0000","last_became_active":"2023-10-08T01:11:20.725675+0000","last_became_peered":"2023-10-08T01:11:20.725675+0000","last_unstale":"2023-10-08T01:11:21.762247+0000","last_undegraded":"2023-10-08T01:11:21.762247+0000","last_fullsized":"2023-10-08T01:11:21.762247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:52.202279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751197+0000","last_change":"2023-10-08T01:11:21.744564+0000","last_active":"2023-10-08T01:11:21.751197+0000","last_peered":"2023-10-08T01:11:21.751197+0000","last_clean":"2023-10-08T01:11:21.751197+0000","last_became_active":"2023-10-08T01:11:21.744327+0000","last_became_peered":"2023-10-08T01:11:21.744327+0000","last_unstale":"2023-10-08T01:11:21.751197+0000","last_undegraded":"2023-10-08T01:11:21.751197+0000","last_fullsized":"2023-10-08T01:11:21.751197+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:19:49.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.762758+0000","last_change":"2023-10-08T01:11:20.726390+0000","last_active":"2023-10-08T01:11:21.762758+0000","last_peered":"2023-10-08T01:11:21.762758+0000","last_clean":"2023-10-08T01:11:21.762758+0000","last_became_active":"2023-10-08T01:11:20.726113+0000","last_became_peered":"2023-10-08T01:11:20.726113+0000","last_unstale":"2023-10-08T01:11:21.762758+0000","last_undegraded":"2023-10-08T01:11:21.762758+0000","last_fullsized":"2023-10-08T01:11:21.762758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:08:53.978733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.748230+0000","last_change":"2023-10-08T01:11:21.742743+0000","last_active":"2023-10-08T01:11:21.748230+0000","last_peered":"2023-10-08T01:11:21.748230+0000","last_clean":"2023-10-08T01:11:21.748230+0000","last_became_active":"2023-10-08T01:11:21.742308+0000","last_became_peered":"2023-10-08T01:11:21.742308+0000","last_unstale":"2023-10-08T01:11:21.748230+0000","last_undegraded":"2023-10-08T01:11:21.748230+0000","last_fullsized":"2023-10-08T01:11:21.748230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:20.346478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.755935+0000","last_change":"2023-10-08T01:11:20.720139+0000","last_active":"2023-10-08T01:11:21.755935+0000","last_peered":"2023-10-08T01:11:21.755935+0000","last_clean":"2023-10-08T01:11:21.755935+0000","last_became_active":"2023-10-08T01:11:20.719850+0000","last_became_peered":"2023-10-08T01:11:20.719850+0000","last_unstale":"2023-10-08T01:11:21.755935+0000","last_undegraded":"2023-10-08T01:11:21.755935+0000","last_fullsized":"2023-10-08T01:11:21.755935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:48.896753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.741683+0000","last_change":"2023-10-08T01:11:21.736777+0000","last_active":"2023-10-08T01:11:21.741683+0000","last_peered":"2023-10-08T01:11:21.741683+0000","last_clean":"2023-10-08T01:11:21.741683+0000","last_became_active":"2023-10-08T01:11:21.736490+0000","last_became_peered":"2023-10-08T01:11:21.736490+0000","last_unstale":"2023-10-08T01:11:21.741683+0000","last_undegraded":"2023-10-08T01:11:21.741683+0000","last_fullsized":"2023-10-08T01:11:21.741683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:58.737586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754964+0000","last_change":"2023-10-08T01:11:20.722971+0000","last_active":"2023-10-08T01:11:21.754964+0000","last_peered":"2023-10-08T01:11:21.754964+0000","last_clean":"2023-10-08T01:11:21.754964+0000","last_became_active":"2023-10-08T01:11:20.722707+0000","last_became_peered":"2023-10-08T01:11:20.722707+0000","last_unstale":"2023-10-08T01:11:21.754964+0000","last_undegraded":"2023-10-08T01:11:21.754964+0000","last_fullsized":"2023-10-08T01:11:21.754964+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:35.034982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.749921+0000","last_change":"2023-10-08T01:11:21.743476+0000","last_active":"2023-10-08T01:11:21.749921+0000","last_peered":"2023-10-08T01:11:21.749921+0000","last_clean":"2023-10-08T01:11:21.749921+0000","last_became_active":"2023-10-08T01:11:21.743172+0000","last_became_peered":"2023-10-08T01:11:21.743172+0000","last_unstale":"2023-10-08T01:11:21.749921+0000","last_undegraded":"2023-10-08T01:11:21.749921+0000","last_fullsized":"2023-10-08T01:11:21.749921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:39.477645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.733839+0000","last_change":"2023-10-08T01:11:20.725543+0000","last_active":"2023-10-08T01:11:21.733839+0000","last_peered":"2023-10-08T01:11:21.733839+0000","last_clean":"2023-10-08T01:11:21.733839+0000","last_became_active":"2023-10-08T01:11:20.725186+0000","last_became_peered":"2023-10-08T01:11:20.725186+0000","last_unstale":"2023-10-08T01:11:21.733839+0000","last_undegraded":"2023-10-08T01:11:21.733839+0000","last_fullsized":"2023-10-08T01:11:21.733839+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:39.137134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744054+0000","last_change":"2023-10-08T01:11:21.741335+0000","last_active":"2023-10-08T01:11:21.744054+0000","last_peered":"2023-10-08T01:11:21.744054+0000","last_clean":"2023-10-08T01:11:21.744054+0000","last_became_active":"2023-10-08T01:11:21.741175+0000","last_became_peered":"2023-10-08T01:11:21.741175+0000","last_unstale":"2023-10-08T01:11:21.744054+0000","last_undegraded":"2023-10-08T01:11:21.744054+0000","last_fullsized":"2023-10-08T01:11:21.744054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:00.724551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.728249+0000","last_change":"2023-10-08T01:11:20.732390+0000","last_active":"2023-10-08T01:11:21.728249+0000","last_peered":"2023-10-08T01:11:21.728249+0000","last_clean":"2023-10-08T01:11:21.728249+0000","last_became_active":"2023-10-08T01:11:20.732100+0000","last_became_peered":"2023-10-08T01:11:20.732100+0000","last_unstale":"2023-10-08T01:11:21.728249+0000","last_undegraded":"2023-10-08T01:11:21.728249+0000","last_fullsized":"2023-10-08T01:11:21.728249+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751017+0000","last_change":"2023-10-08T01:11:21.749481+0000","last_active":"2023-10-08T01:11:21.751017+0000","last_peered":"2023-10-08T01:11:21.751017+0000","last_clean":"2023-10-08T01:11:21.751017+0000","last_became_active":"2023-10-08T01:11:21.749221+0000","last_became_peered":"2023-10-08T01:11:21.749221+0000","last_unstale":"2023-10-08T01:11:21.751017+0000","last_undegraded":"2023-10-08T01:11:21.751017+0000","last_fullsized":"2023-10-08T01:11:21.751017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:27:16.126354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.732044+0000","last_change":"2023-10-08T01:11:20.729480+0000","last_active":"2023-10-08T01:11:21.732044+0000","last_peered":"2023-10-08T01:11:21.732044+0000","last_clean":"2023-10-08T01:11:21.732044+0000","last_became_active":"2023-10-08T01:11:20.729194+0000","last_became_peered":"2023-10-08T01:11:20.729194+0000","last_unstale":"2023-10-08T01:11:21.732044+0000","last_undegraded":"2023-10-08T01:11:21.732044+0000","last_fullsized":"2023-10-08T01:11:21.732044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:08:53.978733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750907+0000","last_change":"2023-10-08T01:11:21.748596+0000","last_active":"2023-10-08T01:11:21.750907+0000","last_peered":"2023-10-08T01:11:21.750907+0000","last_clean":"2023-10-08T01:11:21.750907+0000","last_became_active":"2023-10-08T01:11:21.747472+0000","last_became_peered":"2023-10-08T01:11:21.747472+0000","last_unstale":"2023-10-08T01:11:21.750907+0000","last_undegraded":"2023-10-08T01:11:21.750907+0000","last_fullsized":"2023-10-08T01:11:21.750907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:19:49.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.731074+0000","last_change":"2023-10-08T01:11:20.733957+0000","last_active":"2023-10-08T01:11:21.731074+0000","last_peered":"2023-10-08T01:11:21.731074+0000","last_clean":"2023-10-08T01:11:21.731074+0000","last_became_active":"2023-10-08T01:11:20.733708+0000","last_became_peered":"2023-10-08T01:11:20.733708+0000","last_unstale":"2023-10-08T01:11:21.731074+0000","last_undegraded":"2023-10-08T01:11:21.731074+0000","last_fullsized":"2023-10-08T01:11:21.731074+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:13.831127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754824+0000","last_change":"2023-10-08T01:11:21.747556+0000","last_active":"2023-10-08T01:11:21.754824+0000","last_peered":"2023-10-08T01:11:21.754824+0000","last_clean":"2023-10-08T01:11:21.754824+0000","last_became_active":"2023-10-08T01:11:21.747306+0000","last_became_peered":"2023-10-08T01:11:21.747306+0000","last_unstale":"2023-10-08T01:11:21.754824+0000","last_undegraded":"2023-10-08T01:11:21.754824+0000","last_fullsized":"2023-10-08T01:11:21.754824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:49.904196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"76'3","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.769097+0000","last_change":"2023-10-08T01:11:20.724641+0000","last_active":"2023-10-08T01:11:21.769097+0000","last_peered":"2023-10-08T01:11:21.769097+0000","last_clean":"2023-10-08T01:11:21.769097+0000","last_became_active":"2023-10-08T01:11:20.723826+0000","last_became_peered":"2023-10-08T01:11:20.723826+0000","last_unstale":"2023-10-08T01:11:21.769097+0000","last_undegraded":"2023-10-08T01:11:21.769097+0000","last_fullsized":"2023-10-08T01:11:21.769097+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:56:08.516088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754355+0000","last_change":"2023-10-08T01:11:21.752835+0000","last_active":"2023-10-08T01:11:21.754355+0000","last_peered":"2023-10-08T01:11:21.754355+0000","last_clean":"2023-10-08T01:11:21.754355+0000","last_became_active":"2023-10-08T01:11:21.752512+0000","last_became_peered":"2023-10-08T01:11:21.752512+0000","last_unstale":"2023-10-08T01:11:21.754355+0000","last_undegraded":"2023-10-08T01:11:21.754355+0000","last_fullsized":"2023-10-08T01:11:21.754355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:32.447333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"76'6","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.764191+0000","last_change":"2023-10-08T01:11:20.720914+0000","last_active":"2023-10-08T01:11:21.764191+0000","last_peered":"2023-10-08T01:11:21.764191+0000","last_clean":"2023-10-08T01:11:21.764191+0000","last_became_active":"2023-10-08T01:11:20.720623+0000","last_became_peered":"2023-10-08T01:11:20.720623+0000","last_unstale":"2023-10-08T01:11:21.764191+0000","last_undegraded":"2023-10-08T01:11:21.764191+0000","last_fullsized":"2023-10-08T01:11:21.764191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:56.568531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.757017+0000","last_change":"2023-10-08T01:11:21.755116+0000","last_active":"2023-10-08T01:11:21.757017+0000","last_peered":"2023-10-08T01:11:21.757017+0000","last_clean":"2023-10-08T01:11:21.757017+0000","last_became_active":"2023-10-08T01:11:21.754817+0000","last_became_peered":"2023-10-08T01:11:21.754817+0000","last_unstale":"2023-10-08T01:11:21.757017+0000","last_undegraded":"2023-10-08T01:11:21.757017+0000","last_fullsized":"2023-10-08T01:11:21.757017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:55:03.140046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763929+0000","last_change":"2023-10-08T01:11:20.733962+0000","last_active":"2023-10-08T01:11:21.763929+0000","last_peered":"2023-10-08T01:11:21.763929+0000","last_clean":"2023-10-08T01:11:21.763929+0000","last_became_active":"2023-10-08T01:11:20.733675+0000","last_became_peered":"2023-10-08T01:11:20.733675+0000","last_unstale":"2023-10-08T01:11:21.763929+0000","last_undegraded":"2023-10-08T01:11:21.763929+0000","last_fullsized":"2023-10-08T01:11:21.763929+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:08.865881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750117+0000","last_change":"2023-10-08T01:11:21.746734+0000","last_active":"2023-10-08T01:11:21.750117+0000","last_peered":"2023-10-08T01:11:21.750117+0000","last_clean":"2023-10-08T01:11:21.750117+0000","last_became_active":"2023-10-08T01:11:21.746067+0000","last_became_peered":"2023-10-08T01:11:21.746067+0000","last_unstale":"2023-10-08T01:11:21.750117+0000","last_undegraded":"2023-10-08T01:11:21.750117+0000","last_fullsized":"2023-10-08T01:11:21.750117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:25.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.727317+0000","last_change":"2023-10-08T01:11:20.721718+0000","last_active":"2023-10-08T01:11:21.727317+0000","last_peered":"2023-10-08T01:11:21.727317+0000","last_clean":"2023-10-08T01:11:21.727317+0000","last_became_active":"2023-10-08T01:11:20.721450+0000","last_became_peered":"2023-10-08T01:11:20.721450+0000","last_unstale":"2023-10-08T01:11:21.727317+0000","last_undegraded":"2023-10-08T01:11:21.727317+0000","last_fullsized":"2023-10-08T01:11:21.727317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:30.461191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.739596+0000","last_change":"2023-10-08T01:11:21.737484+0000","last_active":"2023-10-08T01:11:21.739596+0000","last_peered":"2023-10-08T01:11:21.739596+0000","last_clean":"2023-10-08T01:11:21.739596+0000","last_became_active":"2023-10-08T01:11:21.737217+0000","last_became_peered":"2023-10-08T01:11:21.737217+0000","last_unstale":"2023-10-08T01:11:21.739596+0000","last_undegraded":"2023-10-08T01:11:21.739596+0000","last_fullsized":"2023-10-08T01:11:21.739596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:36:03.061174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.733944+0000","last_change":"2023-10-08T01:11:20.727796+0000","last_active":"2023-10-08T01:11:21.733944+0000","last_peered":"2023-10-08T01:11:21.733944+0000","last_clean":"2023-10-08T01:11:21.733944+0000","last_became_active":"2023-10-08T01:11:20.727530+0000","last_became_peered":"2023-10-08T01:11:20.727530+0000","last_unstale":"2023-10-08T01:11:21.733944+0000","last_undegraded":"2023-10-08T01:11:21.733944+0000","last_fullsized":"2023-10-08T01:11:21.733944+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:52.202279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754300+0000","last_change":"2023-10-08T01:11:21.752232+0000","last_active":"2023-10-08T01:11:21.754300+0000","last_peered":"2023-10-08T01:11:21.754300+0000","last_clean":"2023-10-08T01:11:21.754300+0000","last_became_active":"2023-10-08T01:11:21.751917+0000","last_became_peered":"2023-10-08T01:11:21.751917+0000","last_unstale":"2023-10-08T01:11:21.754300+0000","last_undegraded":"2023-10-08T01:11:21.754300+0000","last_fullsized":"2023-10-08T01:11:21.754300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:55.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"76'8","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763406+0000","last_change":"2023-10-08T01:11:20.725738+0000","last_active":"2023-10-08T01:11:21.763406+0000","last_peered":"2023-10-08T01:11:21.763406+0000","last_clean":"2023-10-08T01:11:21.763406+0000","last_became_active":"2023-10-08T01:11:20.725430+0000","last_became_peered":"2023-10-08T01:11:20.725430+0000","last_unstale":"2023-10-08T01:11:21.763406+0000","last_undegraded":"2023-10-08T01:11:21.763406+0000","last_fullsized":"2023-10-08T01:11:21.763406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:11.596616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.746488+0000","last_change":"2023-10-08T01:11:21.743946+0000","last_active":"2023-10-08T01:11:21.746488+0000","last_peered":"2023-10-08T01:11:21.746488+0000","last_clean":"2023-10-08T01:11:21.746488+0000","last_became_active":"2023-10-08T01:11:21.743573+0000","last_became_peered":"2023-10-08T01:11:21.743573+0000","last_unstale":"2023-10-08T01:11:21.746488+0000","last_undegraded":"2023-10-08T01:11:21.746488+0000","last_fullsized":"2023-10-08T01:11:21.746488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:55:03.140046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763853+0000","last_change":"2023-10-08T01:11:20.733998+0000","last_active":"2023-10-08T01:11:21.763853+0000","last_peered":"2023-10-08T01:11:21.763853+0000","last_clean":"2023-10-08T01:11:21.763853+0000","last_became_active":"2023-10-08T01:11:20.733730+0000","last_became_peered":"2023-10-08T01:11:20.733730+0000","last_unstale":"2023-10-08T01:11:21.763853+0000","last_undegraded":"2023-10-08T01:11:21.763853+0000","last_fullsized":"2023-10-08T01:11:21.763853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:10.519648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.755151+0000","last_change":"2023-10-08T01:11:21.752891+0000","last_active":"2023-10-08T01:11:21.755151+0000","last_peered":"2023-10-08T01:11:21.755151+0000","last_clean":"2023-10-08T01:11:21.755151+0000","last_became_active":"2023-10-08T01:11:21.752609+0000","last_became_peered":"2023-10-08T01:11:21.752609+0000","last_unstale":"2023-10-08T01:11:21.755151+0000","last_undegraded":"2023-10-08T01:11:21.755151+0000","last_fullsized":"2023-10-08T01:11:21.755151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:13.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.755982+0000","last_change":"2023-10-08T01:11:20.718240+0000","last_active":"2023-10-08T01:11:21.755982+0000","last_peered":"2023-10-08T01:11:21.755982+0000","last_clean":"2023-10-08T01:11:21.755982+0000","last_became_active":"2023-10-08T01:11:20.717987+0000","last_became_peered":"2023-10-08T01:11:20.717987+0000","last_unstale":"2023-10-08T01:11:21.755982+0000","last_undegraded":"2023-10-08T01:11:21.755982+0000","last_fullsized":"2023-10-08T01:11:21.755982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:39.137134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.747376+0000","last_change":"2023-10-08T01:11:21.741530+0000","last_active":"2023-10-08T01:11:21.747376+0000","last_peered":"2023-10-08T01:11:21.747376+0000","last_clean":"2023-10-08T01:11:21.747376+0000","last_became_active":"2023-10-08T01:11:21.740579+0000","last_became_peered":"2023-10-08T01:11:21.740579+0000","last_unstale":"2023-10-08T01:11:21.747376+0000","last_undegraded":"2023-10-08T01:11:21.747376+0000","last_fullsized":"2023-10-08T01:11:21.747376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:20.017188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.762764+0000","last_change":"2023-10-08T01:11:20.720816+0000","last_active":"2023-10-08T01:11:21.762764+0000","last_peered":"2023-10-08T01:11:21.762764+0000","last_clean":"2023-10-08T01:11:21.762764+0000","last_became_active":"2023-10-08T01:11:20.720541+0000","last_became_peered":"2023-10-08T01:11:20.720541+0000","last_unstale":"2023-10-08T01:11:21.762764+0000","last_undegraded":"2023-10-08T01:11:21.762764+0000","last_fullsized":"2023-10-08T01:11:21.762764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:35.034982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754336+0000","last_change":"2023-10-08T01:11:21.752519+0000","last_active":"2023-10-08T01:11:21.754336+0000","last_peered":"2023-10-08T01:11:21.754336+0000","last_clean":"2023-10-08T01:11:21.754336+0000","last_became_active":"2023-10-08T01:11:21.752257+0000","last_became_peered":"2023-10-08T01:11:21.752257+0000","last_unstale":"2023-10-08T01:11:21.754336+0000","last_undegraded":"2023-10-08T01:11:21.754336+0000","last_fullsized":"2023-10-08T01:11:21.754336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:45.501970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.725351+0000","last_change":"2023-10-08T01:11:20.728496+0000","last_active":"2023-10-08T01:11:21.725351+0000","last_peered":"2023-10-08T01:11:21.725351+0000","last_clean":"2023-10-08T01:11:21.725351+0000","last_became_active":"2023-10-08T01:11:20.728220+0000","last_became_peered":"2023-10-08T01:11:20.728220+0000","last_unstale":"2023-10-08T01:11:21.725351+0000","last_undegraded":"2023-10-08T01:11:21.725351+0000","last_fullsized":"2023-10-08T01:11:21.725351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:48.896753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.755199+0000","last_change":"2023-10-08T01:11:21.752958+0000","last_active":"2023-10-08T01:11:21.755199+0000","last_peered":"2023-10-08T01:11:21.755199+0000","last_clean":"2023-10-08T01:11:21.755199+0000","last_became_active":"2023-10-08T01:11:21.752623+0000","last_became_peered":"2023-10-08T01:11:21.752623+0000","last_unstale":"2023-10-08T01:11:21.755199+0000","last_undegraded":"2023-10-08T01:11:21.755199+0000","last_fullsized":"2023-10-08T01:11:21.755199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:47:13.443993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.764045+0000","last_change":"2023-10-08T01:11:20.730842+0000","last_active":"2023-10-08T01:11:21.764045+0000","last_peered":"2023-10-08T01:11:21.764045+0000","last_clean":"2023-10-08T01:11:21.764045+0000","last_became_active":"2023-10-08T01:11:20.727733+0000","last_became_peered":"2023-10-08T01:11:20.727733+0000","last_unstale":"2023-10-08T01:11:21.764045+0000","last_undegraded":"2023-10-08T01:11:21.764045+0000","last_fullsized":"2023-10-08T01:11:21.764045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:27:15.118911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.735588+0000","last_change":"2023-10-08T01:11:21.730330+0000","last_active":"2023-10-08T01:11:21.735588+0000","last_peered":"2023-10-08T01:11:21.735588+0000","last_clean":"2023-10-08T01:11:21.735588+0000","last_became_active":"2023-10-08T01:11:21.730042+0000","last_became_peered":"2023-10-08T01:11:21.730042+0000","last_unstale":"2023-10-08T01:11:21.735588+0000","last_undegraded":"2023-10-08T01:11:21.735588+0000","last_fullsized":"2023-10-08T01:11:21.735588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:49.923284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.745109+0000","last_change":"2023-10-08T01:11:20.724381+0000","last_active":"2023-10-08T01:11:21.745109+0000","last_peered":"2023-10-08T01:11:21.745109+0000","last_clean":"2023-10-08T01:11:21.745109+0000","last_became_active":"2023-10-08T01:11:20.724176+0000","last_became_peered":"2023-10-08T01:11:20.724176+0000","last_unstale":"2023-10-08T01:11:21.745109+0000","last_undegraded":"2023-10-08T01:11:21.745109+0000","last_fullsized":"2023-10-08T01:11:21.745109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:42.276926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.740190+0000","last_change":"2023-10-08T01:11:21.739225+0000","last_active":"2023-10-08T01:11:21.740190+0000","last_peered":"2023-10-08T01:11:21.740190+0000","last_clean":"2023-10-08T01:11:21.740190+0000","last_became_active":"2023-10-08T01:11:21.738948+0000","last_became_peered":"2023-10-08T01:11:21.738948+0000","last_unstale":"2023-10-08T01:11:21.740190+0000","last_undegraded":"2023-10-08T01:11:21.740190+0000","last_fullsized":"2023-10-08T01:11:21.740190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:44.365874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.736228+0000","last_change":"2023-10-08T01:11:20.732180+0000","last_active":"2023-10-08T01:11:21.736228+0000","last_peered":"2023-10-08T01:11:21.736228+0000","last_clean":"2023-10-08T01:11:21.736228+0000","last_became_active":"2023-10-08T01:11:20.731915+0000","last_became_peered":"2023-10-08T01:11:20.731915+0000","last_unstale":"2023-10-08T01:11:21.736228+0000","last_undegraded":"2023-10-08T01:11:21.736228+0000","last_fullsized":"2023-10-08T01:11:21.736228+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:33.628997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750865+0000","last_change":"2023-10-08T01:11:21.749436+0000","last_active":"2023-10-08T01:11:21.750865+0000","last_peered":"2023-10-08T01:11:21.750865+0000","last_clean":"2023-10-08T01:11:21.750865+0000","last_became_active":"2023-10-08T01:11:21.748525+0000","last_became_peered":"2023-10-08T01:11:21.748525+0000","last_unstale":"2023-10-08T01:11:21.750865+0000","last_undegraded":"2023-10-08T01:11:21.750865+0000","last_fullsized":"2023-10-08T01:11:21.750865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:09.873323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.764323+0000","last_change":"2023-10-08T01:11:20.734568+0000","last_active":"2023-10-08T01:11:21.764323+0000","last_peered":"2023-10-08T01:11:21.764323+0000","last_clean":"2023-10-08T01:11:21.764323+0000","last_became_active":"2023-10-08T01:11:20.734290+0000","last_became_peered":"2023-10-08T01:11:20.734290+0000","last_unstale":"2023-10-08T01:11:21.764323+0000","last_undegraded":"2023-10-08T01:11:21.764323+0000","last_fullsized":"2023-10-08T01:11:21.764323+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:48:58.092204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750407+0000","last_change":"2023-10-08T01:11:21.747744+0000","last_active":"2023-10-08T01:11:21.750407+0000","last_peered":"2023-10-08T01:11:21.750407+0000","last_clean":"2023-10-08T01:11:21.750407+0000","last_became_active":"2023-10-08T01:11:21.746922+0000","last_became_peered":"2023-10-08T01:11:21.746922+0000","last_unstale":"2023-10-08T01:11:21.750407+0000","last_undegraded":"2023-10-08T01:11:21.750407+0000","last_fullsized":"2023-10-08T01:11:21.750407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:49.923284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.726372+0000","last_change":"2023-10-08T01:11:20.726552+0000","last_active":"2023-10-08T01:11:21.726372+0000","last_peered":"2023-10-08T01:11:21.726372+0000","last_clean":"2023-10-08T01:11:21.726372+0000","last_became_active":"2023-10-08T01:11:20.726287+0000","last_became_peered":"2023-10-08T01:11:20.726287+0000","last_unstale":"2023-10-08T01:11:21.726372+0000","last_undegraded":"2023-10-08T01:11:21.726372+0000","last_fullsized":"2023-10-08T01:11:21.726372+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:56.568531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.753302+0000","last_change":"2023-10-08T01:11:21.745510+0000","last_active":"2023-10-08T01:11:21.753302+0000","last_peered":"2023-10-08T01:11:21.753302+0000","last_clean":"2023-10-08T01:11:21.753302+0000","last_became_active":"2023-10-08T01:11:21.745239+0000","last_became_peered":"2023-10-08T01:11:21.745239+0000","last_unstale":"2023-10-08T01:11:21.753302+0000","last_undegraded":"2023-10-08T01:11:21.753302+0000","last_fullsized":"2023-10-08T01:11:21.753302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:11.370692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.727832+0000","last_change":"2023-10-08T01:11:20.720757+0000","last_active":"2023-10-08T01:11:21.727832+0000","last_peered":"2023-10-08T01:11:21.727832+0000","last_clean":"2023-10-08T01:11:21.727832+0000","last_became_active":"2023-10-08T01:11:20.720491+0000","last_became_peered":"2023-10-08T01:11:20.720491+0000","last_unstale":"2023-10-08T01:11:21.727832+0000","last_undegraded":"2023-10-08T01:11:21.727832+0000","last_fullsized":"2023-10-08T01:11:21.727832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:46.734068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751178+0000","last_change":"2023-10-08T01:11:21.744010+0000","last_active":"2023-10-08T01:11:21.751178+0000","last_peered":"2023-10-08T01:11:21.751178+0000","last_clean":"2023-10-08T01:11:21.751178+0000","last_became_active":"2023-10-08T01:11:21.743772+0000","last_became_peered":"2023-10-08T01:11:21.743772+0000","last_unstale":"2023-10-08T01:11:21.751178+0000","last_undegraded":"2023-10-08T01:11:21.751178+0000","last_fullsized":"2023-10-08T01:11:21.751178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:55.596379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.736696+0000","last_change":"2023-10-08T01:11:20.726583+0000","last_active":"2023-10-08T01:11:21.736696+0000","last_peered":"2023-10-08T01:11:21.736696+0000","last_clean":"2023-10-08T01:11:21.736696+0000","last_became_active":"2023-10-08T01:11:20.725722+0000","last_became_peered":"2023-10-08T01:11:20.725722+0000","last_unstale":"2023-10-08T01:11:21.736696+0000","last_undegraded":"2023-10-08T01:11:21.736696+0000","last_fullsized":"2023-10-08T01:11:21.736696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:38.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763749+0000","last_change":"2023-10-08T01:11:21.756555+0000","last_active":"2023-10-08T01:11:21.763749+0000","last_peered":"2023-10-08T01:11:21.763749+0000","last_clean":"2023-10-08T01:11:21.763749+0000","last_became_active":"2023-10-08T01:11:21.756259+0000","last_became_peered":"2023-10-08T01:11:21.756259+0000","last_unstale":"2023-10-08T01:11:21.763749+0000","last_undegraded":"2023-10-08T01:11:21.763749+0000","last_fullsized":"2023-10-08T01:11:21.763749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:28.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.738477+0000","last_change":"2023-10-08T01:11:20.727665+0000","last_active":"2023-10-08T01:11:21.738477+0000","last_peered":"2023-10-08T01:11:21.738477+0000","last_clean":"2023-10-08T01:11:21.738477+0000","last_became_active":"2023-10-08T01:11:20.727352+0000","last_became_peered":"2023-10-08T01:11:20.727352+0000","last_unstale":"2023-10-08T01:11:21.738477+0000","last_undegraded":"2023-10-08T01:11:21.738477+0000","last_fullsized":"2023-10-08T01:11:21.738477+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:13.442980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754472+0000","last_change":"2023-10-08T01:11:21.752252+0000","last_active":"2023-10-08T01:11:21.754472+0000","last_peered":"2023-10-08T01:11:21.754472+0000","last_clean":"2023-10-08T01:11:21.754472+0000","last_became_active":"2023-10-08T01:11:21.751961+0000","last_became_peered":"2023-10-08T01:11:21.751961+0000","last_unstale":"2023-10-08T01:11:21.754472+0000","last_undegraded":"2023-10-08T01:11:21.754472+0000","last_fullsized":"2023-10-08T01:11:21.754472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:44.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.728804+0000","last_change":"2023-10-08T01:11:20.722841+0000","last_active":"2023-10-08T01:11:21.728804+0000","last_peered":"2023-10-08T01:11:21.728804+0000","last_clean":"2023-10-08T01:11:21.728804+0000","last_became_active":"2023-10-08T01:11:20.722582+0000","last_became_peered":"2023-10-08T01:11:20.722582+0000","last_unstale":"2023-10-08T01:11:21.728804+0000","last_undegraded":"2023-10-08T01:11:21.728804+0000","last_fullsized":"2023-10-08T01:11:21.728804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754217+0000","last_change":"2023-10-08T01:11:21.746429+0000","last_active":"2023-10-08T01:11:21.754217+0000","last_peered":"2023-10-08T01:11:21.754217+0000","last_clean":"2023-10-08T01:11:21.754217+0000","last_became_active":"2023-10-08T01:11:21.746153+0000","last_became_peered":"2023-10-08T01:11:21.746153+0000","last_unstale":"2023-10-08T01:11:21.754217+0000","last_undegraded":"2023-10-08T01:11:21.754217+0000","last_fullsized":"2023-10-08T01:11:21.754217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:13.637672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.738835+0000","last_change":"2023-10-08T01:11:20.730285+0000","last_active":"2023-10-08T01:11:21.738835+0000","last_peered":"2023-10-08T01:11:21.738835+0000","last_clean":"2023-10-08T01:11:21.738835+0000","last_became_active":"2023-10-08T01:11:20.730030+0000","last_became_peered":"2023-10-08T01:11:20.730030+0000","last_unstale":"2023-10-08T01:11:21.738835+0000","last_undegraded":"2023-10-08T01:11:21.738835+0000","last_fullsized":"2023-10-08T01:11:21.738835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:46:54.188581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744036+0000","last_change":"2023-10-08T01:11:21.740890+0000","last_active":"2023-10-08T01:11:21.744036+0000","last_peered":"2023-10-08T01:11:21.744036+0000","last_clean":"2023-10-08T01:11:21.744036+0000","last_became_active":"2023-10-08T01:11:21.740620+0000","last_became_peered":"2023-10-08T01:11:21.740620+0000","last_unstale":"2023-10-08T01:11:21.744036+0000","last_undegraded":"2023-10-08T01:11:21.744036+0000","last_fullsized":"2023-10-08T01:11:21.744036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:00.075685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.729749+0000","last_change":"2023-10-08T01:11:20.731212+0000","last_active":"2023-10-08T01:11:21.729749+0000","last_peered":"2023-10-08T01:11:21.729749+0000","last_clean":"2023-10-08T01:11:21.729749+0000","last_became_active":"2023-10-08T01:11:20.730951+0000","last_became_peered":"2023-10-08T01:11:20.730951+0000","last_unstale":"2023-10-08T01:11:21.729749+0000","last_undegraded":"2023-10-08T01:11:21.729749+0000","last_fullsized":"2023-10-08T01:11:21.729749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:38:36.412023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.740780+0000","last_change":"2023-10-08T01:11:21.737714+0000","last_active":"2023-10-08T01:11:21.740780+0000","last_peered":"2023-10-08T01:11:21.740780+0000","last_clean":"2023-10-08T01:11:21.740780+0000","last_became_active":"2023-10-08T01:11:21.737099+0000","last_became_peered":"2023-10-08T01:11:21.737099+0000","last_unstale":"2023-10-08T01:11:21.740780+0000","last_undegraded":"2023-10-08T01:11:21.740780+0000","last_fullsized":"2023-10-08T01:11:21.740780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:44.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.727171+0000","last_change":"2023-10-08T01:11:20.723634+0000","last_active":"2023-10-08T01:11:21.727171+0000","last_peered":"2023-10-08T01:11:21.727171+0000","last_clean":"2023-10-08T01:11:21.727171+0000","last_became_active":"2023-10-08T01:11:20.723384+0000","last_became_peered":"2023-10-08T01:11:20.723384+0000","last_unstale":"2023-10-08T01:11:21.727171+0000","last_undegraded":"2023-10-08T01:11:21.727171+0000","last_fullsized":"2023-10-08T01:11:21.727171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:13.442980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"66'161","reported_seq":369,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.732900+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:11:21.732900+0000","last_peered":"2023-10-08T01:11:21.732900+0000","last_clean":"2023-10-08T01:11:21.732900+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:11:21.732900+0000","last_undegraded":"2023-10-08T01:11:21.732900+0000","last_fullsized":"2023-10-08T01:11:21.732900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575038,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302880,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068960,"statfs":{"total":96636764160,"available":96326615040,"internally_reserved":0,"allocated":1531904,"data_stored":1035740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.627}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.51100000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.56100000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.54400000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.57699999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575038,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302292,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069548,"statfs":{"total":96636764160,"available":96327217152,"internally_reserved":0,"allocated":929792,"data_stored":445212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.64000000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.69599999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.71799999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.79800000000000004}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.51200000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.54000000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.70699999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575038,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302296,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069544,"statfs":{"total":96636764160,"available":96327213056,"internally_reserved":0,"allocated":933888,"data_stored":445302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.501},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.49099999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.63}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.754}]}]},{"osd":0,"up_from":13,"seq":55834575037,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302888,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068952,"statfs":{"total":96636764160,"available":96326606848,"internally_reserved":0,"allocated":1540096,"data_stored":1037670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.997},"last":0.442},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.238},"last":0.377}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.7370000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":0.90700000000000003,"15min":1.0489999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.496}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.629,"5min":0.53000000000000003,"15min":0.45100000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.617},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.48899999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.70099999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2729999999999999,"15min":1.5960000000000001},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.71299999999999997,"5min":0.54200000000000004,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.5289999999999999},"last":0.67700000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.3779999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.629,"5min":0.52700000000000002,"15min":0.51200000000000001},"max":{"1min":1.129,"5min":1.2390000000000001,"15min":1.6459999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.80300000000000005,"15min":0.81999999999999995},"min":{"1min":0.67500000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.82899999999999996,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.78600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575037,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306396,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065444,"statfs":{"total":96636764160,"available":96323014656,"internally_reserved":0,"allocated":937984,"data_stored":445268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.625},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.76900000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.64200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575037,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306396,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065444,"statfs":{"total":96636764160,"available":96323014656,"internally_reserved":0,"allocated":937984,"data_stored":445268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.46100000000000002},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.627,"5min":0.76000000000000001,"15min":0.80100000000000005},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.443}]},{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.42399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.67400000000000004,"5min":0.82999999999999996,"15min":1.5629999999999999},"last":0.40100000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.34100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77200000000000002,"5min":0.89000000000000001,"15min":1.3899999999999999},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.55600000000000005,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":1.095,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.3240000000000001,"15min":1.413},"last":0.59799999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.64100000000000001,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.56799999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":0.997,"15min":1.431},"last":0.61399999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.032,"5min":1.0600000000000001,"15min":1.458},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.44500000000000001},"max":{"1min":1.0629999999999999,"5min":1.1499999999999999,"15min":1.52},"last":0.71299999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.46500000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.389},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.65800000000000003,"5min":0.57799999999999996,"15min":0.5},"max":{"1min":1.054,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.55400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575037,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310496,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061344,"statfs":{"total":96636764160,"available":96318816256,"internally_reserved":0,"allocated":942080,"data_stored":446748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.76300000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.5},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.65000000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.67100000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.71499999999999997}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.746},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.81399999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.78200000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575037,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":937984,"data_stored":445836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.98699999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.78400000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.91900000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":1.034}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.754},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.90200000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.80800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:11:31.334 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:11:31.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.525+0000 7f51f03ff700 1 Processor -- start 2023-10-08T01:11:31.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.525+0000 7f51f03ff700 1 -- start start 2023-10-08T01:11:31.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.526+0000 7f51f03ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51e80ccf90 0x7f51e80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.526+0000 7f51f03ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e80cc300 con 0x7f51e80b1420 2023-10-08T01:11:31.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.526+0000 7f51f03ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e80ad370 con 0x7f51e80cbf30 2023-10-08T01:11:31.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.526+0000 7f51f03ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e8156bf0 con 0x7f51e80ccf90 2023-10-08T01:11:31.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ee99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51e80ccf90 0x7f51e80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ee99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51e80ccf90 0x7f51e80ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52452/0 (socket says 172.21.15.119:52452) 2023-10-08T01:11:31.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ee99c700 1 -- 172.21.15.119:0/1956421164 learned_addr learned my addr 172.21.15.119:0/1956421164 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:31.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ed99a700 1 --1- 172.21.15.119:0/1956421164 >> v1:172.21.15.119:6789/0 conn(0x7f51e80b1420 0x7f51e80c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42528/0 (socket says 172.21.15.119:42528) 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ee99c700 1 -- 172.21.15.119:0/1956421164 >> v1:172.21.15.184:6789/0 conn(0x7f51e80cbf30 legacy=0x7f51e80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ee99c700 1 -- 172.21.15.119:0/1956421164 >> v1:172.21.15.119:6789/0 conn(0x7f51e80b1420 legacy=0x7f51e80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ee99c700 1 -- 172.21.15.119:0/1956421164 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e80ad650 con 0x7f51e80ccf90 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ed199700 1 -- 172.21.15.119:0/1956421164 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402818353 0 0) 0x7f51e80cc300 con 0x7f51e80b1420 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.527+0000 7f51ee99c700 1 --2- 172.21.15.119:0/1956421164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80ccf90 0x7f51e80ae410 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009f20 tx=0x7f51e0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f4a504f32b360 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51ed199700 1 -- 172.21.15.119:0/1956421164 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e000e070 con 0x7f51e80ccf90 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51ed199700 1 -- 172.21.15.119:0/1956421164 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e000ba80 con 0x7f51e80ccf90 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51ed199700 1 -- 172.21.15.119:0/1956421164 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00092e0 con 0x7f51e80ccf90 2023-10-08T01:11:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51f03ff700 1 -- 172.21.15.119:0/1956421164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80ccf90 msgr2=0x7f51e80ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51f03ff700 1 --2- 172.21.15.119:0/1956421164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80ccf90 0x7f51e80ae410 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f51e0009f20 tx=0x7f51e0009840 comp rx=0 tx=0).stop 2023-10-08T01:11:31.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51f03ff700 1 -- 172.21.15.119:0/1956421164 shutdown_connections 2023-10-08T01:11:31.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51f03ff700 1 --2- 172.21.15.119:0/1956421164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80ccf90 0x7f51e80ae410 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51f03ff700 1 -- 172.21.15.119:0/1956421164 >> 172.21.15.119:0/1956421164 conn(0x7f51e806a320 msgr2=0x7f51e80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:31.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51f03ff700 1 -- 172.21.15.119:0/1956421164 shutdown_connections 2023-10-08T01:11:31.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.528+0000 7f51f03ff700 1 -- 172.21.15.119:0/1956421164 wait complete. 2023-10-08T01:11:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.530+0000 7f51f03ff700 1 Processor -- start 2023-10-08T01:11:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.530+0000 7f51f03ff700 1 -- start start 2023-10-08T01:11:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51f03ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80b1420 0x7f51e816a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51f03ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80cbf30 0x7f51e814a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51f03ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80ccf90 0x7f51e80c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51f03ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e816a930 con 0x7f51e80cbf30 2023-10-08T01:11:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51f03ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e80c75a0 con 0x7f51e80b1420 2023-10-08T01:11:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51f03ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e814a580 con 0x7f51e80ccf90 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51ed99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80cbf30 0x7f51e814a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51ed99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80cbf30 0x7f51e814a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52458/0 (socket says 172.21.15.119:52458) 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51ed99a700 1 -- 172.21.15.119:0/2492538624 learned_addr learned my addr 172.21.15.119:0/2492538624 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51ed99a700 1 -- 172.21.15.119:0/2492538624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80ccf90 msgr2=0x7f51e80c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51ed99a700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80ccf90 0x7f51e80c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51ed99a700 1 -- 172.21.15.119:0/2492538624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80b1420 msgr2=0x7f51e816a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51ed99a700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80b1420 0x7f51e816a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.531+0000 7f51ed99a700 1 -- 172.21.15.119:0/2492538624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e0009ae0 con 0x7f51e80cbf30 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.532+0000 7f51ed99a700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80cbf30 0x7f51e814a040 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c5e0 tx=0x7f51d8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.532+0000 7f51de7fc700 1 -- 172.21.15.119:0/2492538624 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8005a70 con 0x7f51e80cbf30 2023-10-08T01:11:31.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.532+0000 7f51de7fc700 1 -- 172.21.15.119:0/2492538624 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8005bd0 con 0x7f51e80cbf30 2023-10-08T01:11:31.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.532+0000 7f51de7fc700 1 -- 172.21.15.119:0/2492538624 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80070d0 con 0x7f51e80cbf30 2023-10-08T01:11:31.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.532+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e80c66b0 con 0x7f51e80cbf30 2023-10-08T01:11:31.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.532+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e80c6c70 con 0x7f51e80cbf30 2023-10-08T01:11:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.533+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e804c3c0 con 0x7f51e80cbf30 2023-10-08T01:11:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.534+0000 7f51de7fc700 1 -- 172.21.15.119:0/2492538624 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51d8007440 con 0x7f51e80cbf30 2023-10-08T01:11:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.534+0000 7f51de7fc700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f51cc066f30 0x7f51cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.535+0000 7f51de7fc700 1 -- 172.21.15.119:0/2492538624 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f51d8083cb0 con 0x7f51e80cbf30 2023-10-08T01:11:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.535+0000 7f51ee19b700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f51cc066f30 0x7f51cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.535+0000 7f51ee19b700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f51cc066f30 0x7f51cc0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51e80c88b0 tx=0x7f51e4007b40 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:31.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.539+0000 7f51de7fc700 1 -- 172.21.15.119:0/2492538624 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51d8050cf0 con 0x7f51e80cbf30 2023-10-08T01:11:31.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.727+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51e806f8a0 con 0x7f51cc066f30 2023-10-08T01:11:31.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.736+0000 7f51de7fc700 1 -- 172.21.15.119:0/2492538624 <== mgr.6036 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289117 (secure 0 0 0) 0x7f51e806f8a0 con 0x7f51cc066f30 2023-10-08T01:11:31.739 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f51cc066f30 msgr2=0x7f51cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f51cc066f30 0x7f51cc0693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51e80c88b0 tx=0x7f51e4007b40 comp rx=0 tx=0).stop 2023-10-08T01:11:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80cbf30 msgr2=0x7f51e814a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80cbf30 0x7f51e814a040 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f51d800c5e0 tx=0x7f51d8007d30 comp rx=0 tx=0).stop 2023-10-08T01:11:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 shutdown_connections 2023-10-08T01:11:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f51cc066f30 0x7f51cc0693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51e80ccf90 0x7f51e80c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51e80b1420 0x7f51e816a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 --2- 172.21.15.119:0/2492538624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51e80cbf30 0x7f51e814a040 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 >> 172.21.15.119:0/2492538624 conn(0x7f51e806a320 msgr2=0x7f51e815a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:31.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 shutdown_connections 2023-10-08T01:11:31.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.740+0000 7f51f03ff700 1 -- 172.21.15.119:0/2492538624 wait complete. 2023-10-08T01:11:31.743 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:11:31.758 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T01:11:31.659912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":284,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":200,"ondisk_log_size":200,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2435944,"kb_used_data":8488,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752538776,"statfs":{"total":773094113280,"available":770599706624,"internally_reserved":0,"allocated":8691712,"data_stored":4747044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.938606"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.742949+0000","last_change":"2023-10-08T01:11:21.734625+0000","last_active":"2023-10-08T01:11:21.742949+0000","last_peered":"2023-10-08T01:11:21.742949+0000","last_clean":"2023-10-08T01:11:21.742949+0000","last_became_active":"2023-10-08T01:11:21.734288+0000","last_became_peered":"2023-10-08T01:11:21.734288+0000","last_unstale":"2023-10-08T01:11:21.742949+0000","last_undegraded":"2023-10-08T01:11:21.742949+0000","last_fullsized":"2023-10-08T01:11:21.742949+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:47.741510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.749754+0000","last_change":"2023-10-08T01:11:21.748540+0000","last_active":"2023-10-08T01:11:21.749754+0000","last_peered":"2023-10-08T01:11:21.749754+0000","last_clean":"2023-10-08T01:11:21.749754+0000","last_became_active":"2023-10-08T01:11:21.748276+0000","last_became_peered":"2023-10-08T01:11:21.748276+0000","last_unstale":"2023-10-08T01:11:21.749754+0000","last_undegraded":"2023-10-08T01:11:21.749754+0000","last_fullsized":"2023-10-08T01:11:21.749754+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:08:47.557664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744557+0000","last_change":"2023-10-08T01:11:21.743208+0000","last_active":"2023-10-08T01:11:21.744557+0000","last_peered":"2023-10-08T01:11:21.744557+0000","last_clean":"2023-10-08T01:11:21.744557+0000","last_became_active":"2023-10-08T01:11:21.742955+0000","last_became_peered":"2023-10-08T01:11:21.742955+0000","last_unstale":"2023-10-08T01:11:21.744557+0000","last_undegraded":"2023-10-08T01:11:21.744557+0000","last_fullsized":"2023-10-08T01:11:21.744557+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:13.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.749986+0000","last_change":"2023-10-08T01:11:21.744538+0000","last_active":"2023-10-08T01:11:21.749986+0000","last_peered":"2023-10-08T01:11:21.749986+0000","last_clean":"2023-10-08T01:11:21.749986+0000","last_became_active":"2023-10-08T01:11:21.744220+0000","last_became_peered":"2023-10-08T01:11:21.744220+0000","last_unstale":"2023-10-08T01:11:21.749986+0000","last_undegraded":"2023-10-08T01:11:21.749986+0000","last_fullsized":"2023-10-08T01:11:21.749986+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:53:19.668372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744350+0000","last_change":"2023-10-08T01:11:21.742526+0000","last_active":"2023-10-08T01:11:21.744350+0000","last_peered":"2023-10-08T01:11:21.744350+0000","last_clean":"2023-10-08T01:11:21.744350+0000","last_became_active":"2023-10-08T01:11:21.742287+0000","last_became_peered":"2023-10-08T01:11:21.742287+0000","last_unstale":"2023-10-08T01:11:21.744350+0000","last_undegraded":"2023-10-08T01:11:21.744350+0000","last_fullsized":"2023-10-08T01:11:21.744350+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:14:08.861323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737640+0000","last_change":"2023-10-08T01:11:21.733074+0000","last_active":"2023-10-08T01:11:21.737640+0000","last_peered":"2023-10-08T01:11:21.737640+0000","last_clean":"2023-10-08T01:11:21.737640+0000","last_became_active":"2023-10-08T01:11:21.732795+0000","last_became_peered":"2023-10-08T01:11:21.732795+0000","last_unstale":"2023-10-08T01:11:21.737640+0000","last_undegraded":"2023-10-08T01:11:21.737640+0000","last_fullsized":"2023-10-08T01:11:21.737640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:55:03.140046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.746884+0000","last_change":"2023-10-08T01:11:21.743612+0000","last_active":"2023-10-08T01:11:21.746884+0000","last_peered":"2023-10-08T01:11:21.746884+0000","last_clean":"2023-10-08T01:11:21.746884+0000","last_became_active":"2023-10-08T01:11:21.743367+0000","last_became_peered":"2023-10-08T01:11:21.743367+0000","last_unstale":"2023-10-08T01:11:21.746884+0000","last_undegraded":"2023-10-08T01:11:21.746884+0000","last_fullsized":"2023-10-08T01:11:21.746884+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:52:58.069365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737603+0000","last_change":"2023-10-08T01:11:21.732607+0000","last_active":"2023-10-08T01:11:21.737603+0000","last_peered":"2023-10-08T01:11:21.737603+0000","last_clean":"2023-10-08T01:11:21.737603+0000","last_became_active":"2023-10-08T01:11:21.732313+0000","last_became_peered":"2023-10-08T01:11:21.732313+0000","last_unstale":"2023-10-08T01:11:21.737603+0000","last_undegraded":"2023-10-08T01:11:21.737603+0000","last_fullsized":"2023-10-08T01:11:21.737603+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:50:24.923577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.739546+0000","last_change":"2023-10-08T01:11:21.737895+0000","last_active":"2023-10-08T01:11:21.739546+0000","last_peered":"2023-10-08T01:11:21.739546+0000","last_clean":"2023-10-08T01:11:21.739546+0000","last_became_active":"2023-10-08T01:11:21.737574+0000","last_became_peered":"2023-10-08T01:11:21.737574+0000","last_unstale":"2023-10-08T01:11:21.739546+0000","last_undegraded":"2023-10-08T01:11:21.739546+0000","last_fullsized":"2023-10-08T01:11:21.739546+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:12.604059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754595+0000","last_change":"2023-10-08T01:11:21.750279+0000","last_active":"2023-10-08T01:11:21.754595+0000","last_peered":"2023-10-08T01:11:21.754595+0000","last_clean":"2023-10-08T01:11:21.754595+0000","last_became_active":"2023-10-08T01:11:21.749755+0000","last_became_peered":"2023-10-08T01:11:21.749755+0000","last_unstale":"2023-10-08T01:11:21.754595+0000","last_undegraded":"2023-10-08T01:11:21.754595+0000","last_fullsized":"2023-10-08T01:11:21.754595+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:57.554427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.758784+0000","last_change":"2023-10-08T01:11:21.754039+0000","last_active":"2023-10-08T01:11:21.758784+0000","last_peered":"2023-10-08T01:11:21.758784+0000","last_clean":"2023-10-08T01:11:21.758784+0000","last_became_active":"2023-10-08T01:11:21.753655+0000","last_became_peered":"2023-10-08T01:11:21.753655+0000","last_unstale":"2023-10-08T01:11:21.758784+0000","last_undegraded":"2023-10-08T01:11:21.758784+0000","last_fullsized":"2023-10-08T01:11:21.758784+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:39.593532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.749961+0000","last_change":"2023-10-08T01:11:21.741533+0000","last_active":"2023-10-08T01:11:21.749961+0000","last_peered":"2023-10-08T01:11:21.749961+0000","last_clean":"2023-10-08T01:11:21.749961+0000","last_became_active":"2023-10-08T01:11:21.741269+0000","last_became_peered":"2023-10-08T01:11:21.741269+0000","last_unstale":"2023-10-08T01:11:21.749961+0000","last_undegraded":"2023-10-08T01:11:21.749961+0000","last_fullsized":"2023-10-08T01:11:21.749961+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:38:37.419465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737212+0000","last_change":"2023-10-08T01:11:21.734119+0000","last_active":"2023-10-08T01:11:21.737212+0000","last_peered":"2023-10-08T01:11:21.737212+0000","last_clean":"2023-10-08T01:11:21.737212+0000","last_became_active":"2023-10-08T01:11:21.733808+0000","last_became_peered":"2023-10-08T01:11:21.733808+0000","last_unstale":"2023-10-08T01:11:21.737212+0000","last_undegraded":"2023-10-08T01:11:21.737212+0000","last_fullsized":"2023-10-08T01:11:21.737212+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:59.827230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.740644+0000","last_change":"2023-10-08T01:11:21.738874+0000","last_active":"2023-10-08T01:11:21.740644+0000","last_peered":"2023-10-08T01:11:21.740644+0000","last_clean":"2023-10-08T01:11:21.740644+0000","last_became_active":"2023-10-08T01:11:21.736697+0000","last_became_peered":"2023-10-08T01:11:21.736697+0000","last_unstale":"2023-10-08T01:11:21.740644+0000","last_undegraded":"2023-10-08T01:11:21.740644+0000","last_fullsized":"2023-10-08T01:11:21.740644+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:07:47.800512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750130+0000","last_change":"2023-10-08T01:11:21.748555+0000","last_active":"2023-10-08T01:11:21.750130+0000","last_peered":"2023-10-08T01:11:21.750130+0000","last_clean":"2023-10-08T01:11:21.750130+0000","last_became_active":"2023-10-08T01:11:21.748276+0000","last_became_peered":"2023-10-08T01:11:21.748276+0000","last_unstale":"2023-10-08T01:11:21.750130+0000","last_undegraded":"2023-10-08T01:11:21.750130+0000","last_fullsized":"2023-10-08T01:11:21.750130+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:00.761929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.735680+0000","last_change":"2023-10-08T01:11:21.729641+0000","last_active":"2023-10-08T01:11:21.735680+0000","last_peered":"2023-10-08T01:11:21.735680+0000","last_clean":"2023-10-08T01:11:21.735680+0000","last_became_active":"2023-10-08T01:11:21.729379+0000","last_became_peered":"2023-10-08T01:11:21.729379+0000","last_unstale":"2023-10-08T01:11:21.735680+0000","last_undegraded":"2023-10-08T01:11:21.735680+0000","last_fullsized":"2023-10-08T01:11:21.735680+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:13.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.743117+0000","last_change":"2023-10-08T01:11:21.740889+0000","last_active":"2023-10-08T01:11:21.743117+0000","last_peered":"2023-10-08T01:11:21.743117+0000","last_clean":"2023-10-08T01:11:21.743117+0000","last_became_active":"2023-10-08T01:11:21.740575+0000","last_became_peered":"2023-10-08T01:11:21.740575+0000","last_unstale":"2023-10-08T01:11:21.743117+0000","last_undegraded":"2023-10-08T01:11:21.743117+0000","last_fullsized":"2023-10-08T01:11:21.743117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:33.627492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744028+0000","last_change":"2023-10-08T01:11:21.741742+0000","last_active":"2023-10-08T01:11:21.744028+0000","last_peered":"2023-10-08T01:11:21.744028+0000","last_clean":"2023-10-08T01:11:21.744028+0000","last_became_active":"2023-10-08T01:11:21.741447+0000","last_became_peered":"2023-10-08T01:11:21.741447+0000","last_unstale":"2023-10-08T01:11:21.744028+0000","last_undegraded":"2023-10-08T01:11:21.744028+0000","last_fullsized":"2023-10-08T01:11:21.744028+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:36.169190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.741640+0000","last_change":"2023-10-08T01:11:21.734404+0000","last_active":"2023-10-08T01:11:21.741640+0000","last_peered":"2023-10-08T01:11:21.741640+0000","last_clean":"2023-10-08T01:11:21.741640+0000","last_became_active":"2023-10-08T01:11:21.734076+0000","last_became_peered":"2023-10-08T01:11:21.734076+0000","last_unstale":"2023-10-08T01:11:21.741640+0000","last_undegraded":"2023-10-08T01:11:21.741640+0000","last_fullsized":"2023-10-08T01:11:21.741640+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:14.450422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.748483+0000","last_change":"2023-10-08T01:11:21.742567+0000","last_active":"2023-10-08T01:11:21.748483+0000","last_peered":"2023-10-08T01:11:21.748483+0000","last_clean":"2023-10-08T01:11:21.748483+0000","last_became_active":"2023-10-08T01:11:21.740331+0000","last_became_peered":"2023-10-08T01:11:21.740331+0000","last_unstale":"2023-10-08T01:11:21.748483+0000","last_undegraded":"2023-10-08T01:11:21.748483+0000","last_fullsized":"2023-10-08T01:11:21.748483+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:16.987048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750933+0000","last_change":"2023-10-08T01:11:21.749066+0000","last_active":"2023-10-08T01:11:21.750933+0000","last_peered":"2023-10-08T01:11:21.750933+0000","last_clean":"2023-10-08T01:11:21.750933+0000","last_became_active":"2023-10-08T01:11:21.748559+0000","last_became_peered":"2023-10-08T01:11:21.748559+0000","last_unstale":"2023-10-08T01:11:21.750933+0000","last_undegraded":"2023-10-08T01:11:21.750933+0000","last_fullsized":"2023-10-08T01:11:21.750933+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:55.596379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.748211+0000","last_change":"2023-10-08T01:11:21.742287+0000","last_active":"2023-10-08T01:11:21.748211+0000","last_peered":"2023-10-08T01:11:21.748211+0000","last_clean":"2023-10-08T01:11:21.748211+0000","last_became_active":"2023-10-08T01:11:21.741967+0000","last_became_peered":"2023-10-08T01:11:21.741967+0000","last_unstale":"2023-10-08T01:11:21.748211+0000","last_undegraded":"2023-10-08T01:11:21.748211+0000","last_fullsized":"2023-10-08T01:11:21.748211+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:28:50.899302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.748047+0000","last_change":"2023-10-08T01:11:21.744406+0000","last_active":"2023-10-08T01:11:21.748047+0000","last_peered":"2023-10-08T01:11:21.748047+0000","last_clean":"2023-10-08T01:11:21.748047+0000","last_became_active":"2023-10-08T01:11:21.744150+0000","last_became_peered":"2023-10-08T01:11:21.744150+0000","last_unstale":"2023-10-08T01:11:21.748047+0000","last_undegraded":"2023-10-08T01:11:21.748047+0000","last_fullsized":"2023-10-08T01:11:21.748047+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:44:01.648076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.736837+0000","last_change":"2023-10-08T01:11:21.733418+0000","last_active":"2023-10-08T01:11:21.736837+0000","last_peered":"2023-10-08T01:11:21.736837+0000","last_clean":"2023-10-08T01:11:21.736837+0000","last_became_active":"2023-10-08T01:11:21.733130+0000","last_became_peered":"2023-10-08T01:11:21.733130+0000","last_unstale":"2023-10-08T01:11:21.736837+0000","last_undegraded":"2023-10-08T01:11:21.736837+0000","last_fullsized":"2023-10-08T01:11:21.736837+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:14.319695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750683+0000","last_change":"2023-10-08T01:11:21.747795+0000","last_active":"2023-10-08T01:11:21.750683+0000","last_peered":"2023-10-08T01:11:21.750683+0000","last_clean":"2023-10-08T01:11:21.750683+0000","last_became_active":"2023-10-08T01:11:21.747532+0000","last_became_peered":"2023-10-08T01:11:21.747532+0000","last_unstale":"2023-10-08T01:11:21.750683+0000","last_undegraded":"2023-10-08T01:11:21.750683+0000","last_fullsized":"2023-10-08T01:11:21.750683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:45:38.760441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750561+0000","last_change":"2023-10-08T01:11:21.748380+0000","last_active":"2023-10-08T01:11:21.750561+0000","last_peered":"2023-10-08T01:11:21.750561+0000","last_clean":"2023-10-08T01:11:21.750561+0000","last_became_active":"2023-10-08T01:11:21.747916+0000","last_became_peered":"2023-10-08T01:11:21.747916+0000","last_unstale":"2023-10-08T01:11:21.750561+0000","last_undegraded":"2023-10-08T01:11:21.750561+0000","last_fullsized":"2023-10-08T01:11:21.750561+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:44.365874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737232+0000","last_change":"2023-10-08T01:11:21.731384+0000","last_active":"2023-10-08T01:11:21.737232+0000","last_peered":"2023-10-08T01:11:21.737232+0000","last_clean":"2023-10-08T01:11:21.737232+0000","last_became_active":"2023-10-08T01:11:21.730695+0000","last_became_peered":"2023-10-08T01:11:21.730695+0000","last_unstale":"2023-10-08T01:11:21.737232+0000","last_undegraded":"2023-10-08T01:11:21.737232+0000","last_fullsized":"2023-10-08T01:11:21.737232+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:38:37.419465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.753700+0000","last_change":"2023-10-08T01:11:21.750748+0000","last_active":"2023-10-08T01:11:21.753700+0000","last_peered":"2023-10-08T01:11:21.753700+0000","last_clean":"2023-10-08T01:11:21.753700+0000","last_became_active":"2023-10-08T01:11:21.750500+0000","last_became_peered":"2023-10-08T01:11:21.750500+0000","last_unstale":"2023-10-08T01:11:21.753700+0000","last_undegraded":"2023-10-08T01:11:21.753700+0000","last_fullsized":"2023-10-08T01:11:21.753700+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:31.468634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750858+0000","last_change":"2023-10-08T01:11:21.749661+0000","last_active":"2023-10-08T01:11:21.750858+0000","last_peered":"2023-10-08T01:11:21.750858+0000","last_clean":"2023-10-08T01:11:21.750858+0000","last_became_active":"2023-10-08T01:11:21.749297+0000","last_became_peered":"2023-10-08T01:11:21.749297+0000","last_unstale":"2023-10-08T01:11:21.750858+0000","last_undegraded":"2023-10-08T01:11:21.750858+0000","last_fullsized":"2023-10-08T01:11:21.750858+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:51.269770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.743965+0000","last_change":"2023-10-08T01:11:21.740395+0000","last_active":"2023-10-08T01:11:21.743965+0000","last_peered":"2023-10-08T01:11:21.743965+0000","last_clean":"2023-10-08T01:11:21.743965+0000","last_became_active":"2023-10-08T01:11:21.740122+0000","last_became_peered":"2023-10-08T01:11:21.740122+0000","last_unstale":"2023-10-08T01:11:21.743965+0000","last_undegraded":"2023-10-08T01:11:21.743965+0000","last_fullsized":"2023-10-08T01:11:21.743965+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:52.567751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750652+0000","last_change":"2023-10-08T01:11:21.748445+0000","last_active":"2023-10-08T01:11:21.750652+0000","last_peered":"2023-10-08T01:11:21.750652+0000","last_clean":"2023-10-08T01:11:21.750652+0000","last_became_active":"2023-10-08T01:11:21.748193+0000","last_became_peered":"2023-10-08T01:11:21.748193+0000","last_unstale":"2023-10-08T01:11:21.750652+0000","last_undegraded":"2023-10-08T01:11:21.750652+0000","last_fullsized":"2023-10-08T01:11:21.750652+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:05.442790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.737327+0000","last_change":"2023-10-08T01:11:21.734342+0000","last_active":"2023-10-08T01:11:21.737327+0000","last_peered":"2023-10-08T01:11:21.737327+0000","last_clean":"2023-10-08T01:11:21.737327+0000","last_became_active":"2023-10-08T01:11:21.734048+0000","last_became_peered":"2023-10-08T01:11:21.734048+0000","last_unstale":"2023-10-08T01:11:21.737327+0000","last_undegraded":"2023-10-08T01:11:21.737327+0000","last_fullsized":"2023-10-08T01:11:21.737327+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:52.567751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751083+0000","last_change":"2023-10-08T01:11:21.743433+0000","last_active":"2023-10-08T01:11:21.751083+0000","last_peered":"2023-10-08T01:11:21.751083+0000","last_clean":"2023-10-08T01:11:21.751083+0000","last_became_active":"2023-10-08T01:11:21.742926+0000","last_became_peered":"2023-10-08T01:11:21.742926+0000","last_unstale":"2023-10-08T01:11:21.751083+0000","last_undegraded":"2023-10-08T01:11:21.751083+0000","last_fullsized":"2023-10-08T01:11:21.751083+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:47:13.443993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"76'4","reported_seq":14,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.760540+0000","last_change":"2023-10-08T01:11:20.731718+0000","last_active":"2023-10-08T01:11:21.760540+0000","last_peered":"2023-10-08T01:11:21.760540+0000","last_clean":"2023-10-08T01:11:21.760540+0000","last_became_active":"2023-10-08T01:11:20.731471+0000","last_became_peered":"2023-10-08T01:11:20.731471+0000","last_unstale":"2023-10-08T01:11:21.760540+0000","last_undegraded":"2023-10-08T01:11:21.760540+0000","last_fullsized":"2023-10-08T01:11:21.760540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:33.628997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751641+0000","last_change":"2023-10-08T01:11:21.745894+0000","last_active":"2023-10-08T01:11:21.751641+0000","last_peered":"2023-10-08T01:11:21.751641+0000","last_clean":"2023-10-08T01:11:21.751641+0000","last_became_active":"2023-10-08T01:11:21.745513+0000","last_became_peered":"2023-10-08T01:11:21.745513+0000","last_unstale":"2023-10-08T01:11:21.751641+0000","last_undegraded":"2023-10-08T01:11:21.751641+0000","last_fullsized":"2023-10-08T01:11:21.751641+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:57.575973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763890+0000","last_change":"2023-10-08T01:11:20.735524+0000","last_active":"2023-10-08T01:11:21.763890+0000","last_peered":"2023-10-08T01:11:21.763890+0000","last_clean":"2023-10-08T01:11:21.763890+0000","last_became_active":"2023-10-08T01:11:20.735143+0000","last_became_peered":"2023-10-08T01:11:20.735143+0000","last_unstale":"2023-10-08T01:11:21.763890+0000","last_undegraded":"2023-10-08T01:11:21.763890+0000","last_fullsized":"2023-10-08T01:11:21.763890+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:48.915842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751650+0000","last_change":"2023-10-08T01:11:21.747675+0000","last_active":"2023-10-08T01:11:21.751650+0000","last_peered":"2023-10-08T01:11:21.751650+0000","last_clean":"2023-10-08T01:11:21.751650+0000","last_became_active":"2023-10-08T01:11:21.747420+0000","last_became_peered":"2023-10-08T01:11:21.747420+0000","last_unstale":"2023-10-08T01:11:21.751650+0000","last_undegraded":"2023-10-08T01:11:21.751650+0000","last_fullsized":"2023-10-08T01:11:21.751650+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:53:19.668372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.762247+0000","last_change":"2023-10-08T01:11:20.725920+0000","last_active":"2023-10-08T01:11:21.762247+0000","last_peered":"2023-10-08T01:11:21.762247+0000","last_clean":"2023-10-08T01:11:21.762247+0000","last_became_active":"2023-10-08T01:11:20.725675+0000","last_became_peered":"2023-10-08T01:11:20.725675+0000","last_unstale":"2023-10-08T01:11:21.762247+0000","last_undegraded":"2023-10-08T01:11:21.762247+0000","last_fullsized":"2023-10-08T01:11:21.762247+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:52.202279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751197+0000","last_change":"2023-10-08T01:11:21.744564+0000","last_active":"2023-10-08T01:11:21.751197+0000","last_peered":"2023-10-08T01:11:21.751197+0000","last_clean":"2023-10-08T01:11:21.751197+0000","last_became_active":"2023-10-08T01:11:21.744327+0000","last_became_peered":"2023-10-08T01:11:21.744327+0000","last_unstale":"2023-10-08T01:11:21.751197+0000","last_undegraded":"2023-10-08T01:11:21.751197+0000","last_fullsized":"2023-10-08T01:11:21.751197+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:19:49.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.762758+0000","last_change":"2023-10-08T01:11:20.726390+0000","last_active":"2023-10-08T01:11:21.762758+0000","last_peered":"2023-10-08T01:11:21.762758+0000","last_clean":"2023-10-08T01:11:21.762758+0000","last_became_active":"2023-10-08T01:11:20.726113+0000","last_became_peered":"2023-10-08T01:11:20.726113+0000","last_unstale":"2023-10-08T01:11:21.762758+0000","last_undegraded":"2023-10-08T01:11:21.762758+0000","last_fullsized":"2023-10-08T01:11:21.762758+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:08:53.978733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.748230+0000","last_change":"2023-10-08T01:11:21.742743+0000","last_active":"2023-10-08T01:11:21.748230+0000","last_peered":"2023-10-08T01:11:21.748230+0000","last_clean":"2023-10-08T01:11:21.748230+0000","last_became_active":"2023-10-08T01:11:21.742308+0000","last_became_peered":"2023-10-08T01:11:21.742308+0000","last_unstale":"2023-10-08T01:11:21.748230+0000","last_undegraded":"2023-10-08T01:11:21.748230+0000","last_fullsized":"2023-10-08T01:11:21.748230+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:06:20.346478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.755935+0000","last_change":"2023-10-08T01:11:20.720139+0000","last_active":"2023-10-08T01:11:21.755935+0000","last_peered":"2023-10-08T01:11:21.755935+0000","last_clean":"2023-10-08T01:11:21.755935+0000","last_became_active":"2023-10-08T01:11:20.719850+0000","last_became_peered":"2023-10-08T01:11:20.719850+0000","last_unstale":"2023-10-08T01:11:21.755935+0000","last_undegraded":"2023-10-08T01:11:21.755935+0000","last_fullsized":"2023-10-08T01:11:21.755935+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:48.896753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.741683+0000","last_change":"2023-10-08T01:11:21.736777+0000","last_active":"2023-10-08T01:11:21.741683+0000","last_peered":"2023-10-08T01:11:21.741683+0000","last_clean":"2023-10-08T01:11:21.741683+0000","last_became_active":"2023-10-08T01:11:21.736490+0000","last_became_peered":"2023-10-08T01:11:21.736490+0000","last_unstale":"2023-10-08T01:11:21.741683+0000","last_undegraded":"2023-10-08T01:11:21.741683+0000","last_fullsized":"2023-10-08T01:11:21.741683+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:58.737586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754964+0000","last_change":"2023-10-08T01:11:20.722971+0000","last_active":"2023-10-08T01:11:21.754964+0000","last_peered":"2023-10-08T01:11:21.754964+0000","last_clean":"2023-10-08T01:11:21.754964+0000","last_became_active":"2023-10-08T01:11:20.722707+0000","last_became_peered":"2023-10-08T01:11:20.722707+0000","last_unstale":"2023-10-08T01:11:21.754964+0000","last_undegraded":"2023-10-08T01:11:21.754964+0000","last_fullsized":"2023-10-08T01:11:21.754964+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:35.034982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.749921+0000","last_change":"2023-10-08T01:11:21.743476+0000","last_active":"2023-10-08T01:11:21.749921+0000","last_peered":"2023-10-08T01:11:21.749921+0000","last_clean":"2023-10-08T01:11:21.749921+0000","last_became_active":"2023-10-08T01:11:21.743172+0000","last_became_peered":"2023-10-08T01:11:21.743172+0000","last_unstale":"2023-10-08T01:11:21.749921+0000","last_undegraded":"2023-10-08T01:11:21.749921+0000","last_fullsized":"2023-10-08T01:11:21.749921+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:39.477645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.733839+0000","last_change":"2023-10-08T01:11:20.725543+0000","last_active":"2023-10-08T01:11:21.733839+0000","last_peered":"2023-10-08T01:11:21.733839+0000","last_clean":"2023-10-08T01:11:21.733839+0000","last_became_active":"2023-10-08T01:11:20.725186+0000","last_became_peered":"2023-10-08T01:11:20.725186+0000","last_unstale":"2023-10-08T01:11:21.733839+0000","last_undegraded":"2023-10-08T01:11:21.733839+0000","last_fullsized":"2023-10-08T01:11:21.733839+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:39.137134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744054+0000","last_change":"2023-10-08T01:11:21.741335+0000","last_active":"2023-10-08T01:11:21.744054+0000","last_peered":"2023-10-08T01:11:21.744054+0000","last_clean":"2023-10-08T01:11:21.744054+0000","last_became_active":"2023-10-08T01:11:21.741175+0000","last_became_peered":"2023-10-08T01:11:21.741175+0000","last_unstale":"2023-10-08T01:11:21.744054+0000","last_undegraded":"2023-10-08T01:11:21.744054+0000","last_fullsized":"2023-10-08T01:11:21.744054+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:00.724551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.728249+0000","last_change":"2023-10-08T01:11:20.732390+0000","last_active":"2023-10-08T01:11:21.728249+0000","last_peered":"2023-10-08T01:11:21.728249+0000","last_clean":"2023-10-08T01:11:21.728249+0000","last_became_active":"2023-10-08T01:11:20.732100+0000","last_became_peered":"2023-10-08T01:11:20.732100+0000","last_unstale":"2023-10-08T01:11:21.728249+0000","last_undegraded":"2023-10-08T01:11:21.728249+0000","last_fullsized":"2023-10-08T01:11:21.728249+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:10.363250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751017+0000","last_change":"2023-10-08T01:11:21.749481+0000","last_active":"2023-10-08T01:11:21.751017+0000","last_peered":"2023-10-08T01:11:21.751017+0000","last_clean":"2023-10-08T01:11:21.751017+0000","last_became_active":"2023-10-08T01:11:21.749221+0000","last_became_peered":"2023-10-08T01:11:21.749221+0000","last_unstale":"2023-10-08T01:11:21.751017+0000","last_undegraded":"2023-10-08T01:11:21.751017+0000","last_fullsized":"2023-10-08T01:11:21.751017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:27:16.126354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.732044+0000","last_change":"2023-10-08T01:11:20.729480+0000","last_active":"2023-10-08T01:11:21.732044+0000","last_peered":"2023-10-08T01:11:21.732044+0000","last_clean":"2023-10-08T01:11:21.732044+0000","last_became_active":"2023-10-08T01:11:20.729194+0000","last_became_peered":"2023-10-08T01:11:20.729194+0000","last_unstale":"2023-10-08T01:11:21.732044+0000","last_undegraded":"2023-10-08T01:11:21.732044+0000","last_fullsized":"2023-10-08T01:11:21.732044+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:08:53.978733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750907+0000","last_change":"2023-10-08T01:11:21.748596+0000","last_active":"2023-10-08T01:11:21.750907+0000","last_peered":"2023-10-08T01:11:21.750907+0000","last_clean":"2023-10-08T01:11:21.750907+0000","last_became_active":"2023-10-08T01:11:21.747472+0000","last_became_peered":"2023-10-08T01:11:21.747472+0000","last_unstale":"2023-10-08T01:11:21.750907+0000","last_undegraded":"2023-10-08T01:11:21.750907+0000","last_fullsized":"2023-10-08T01:11:21.750907+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:19:49.042000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.731074+0000","last_change":"2023-10-08T01:11:20.733957+0000","last_active":"2023-10-08T01:11:21.731074+0000","last_peered":"2023-10-08T01:11:21.731074+0000","last_clean":"2023-10-08T01:11:21.731074+0000","last_became_active":"2023-10-08T01:11:20.733708+0000","last_became_peered":"2023-10-08T01:11:20.733708+0000","last_unstale":"2023-10-08T01:11:21.731074+0000","last_undegraded":"2023-10-08T01:11:21.731074+0000","last_fullsized":"2023-10-08T01:11:21.731074+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:19:13.831127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754824+0000","last_change":"2023-10-08T01:11:21.747556+0000","last_active":"2023-10-08T01:11:21.754824+0000","last_peered":"2023-10-08T01:11:21.754824+0000","last_clean":"2023-10-08T01:11:21.754824+0000","last_became_active":"2023-10-08T01:11:21.747306+0000","last_became_peered":"2023-10-08T01:11:21.747306+0000","last_unstale":"2023-10-08T01:11:21.754824+0000","last_undegraded":"2023-10-08T01:11:21.754824+0000","last_fullsized":"2023-10-08T01:11:21.754824+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:49.904196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"76'3","reported_seq":13,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.769097+0000","last_change":"2023-10-08T01:11:20.724641+0000","last_active":"2023-10-08T01:11:21.769097+0000","last_peered":"2023-10-08T01:11:21.769097+0000","last_clean":"2023-10-08T01:11:21.769097+0000","last_became_active":"2023-10-08T01:11:20.723826+0000","last_became_peered":"2023-10-08T01:11:20.723826+0000","last_unstale":"2023-10-08T01:11:21.769097+0000","last_undegraded":"2023-10-08T01:11:21.769097+0000","last_fullsized":"2023-10-08T01:11:21.769097+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:56:08.516088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754355+0000","last_change":"2023-10-08T01:11:21.752835+0000","last_active":"2023-10-08T01:11:21.754355+0000","last_peered":"2023-10-08T01:11:21.754355+0000","last_clean":"2023-10-08T01:11:21.754355+0000","last_became_active":"2023-10-08T01:11:21.752512+0000","last_became_peered":"2023-10-08T01:11:21.752512+0000","last_unstale":"2023-10-08T01:11:21.754355+0000","last_undegraded":"2023-10-08T01:11:21.754355+0000","last_fullsized":"2023-10-08T01:11:21.754355+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:32.447333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"76'6","reported_seq":16,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.764191+0000","last_change":"2023-10-08T01:11:20.720914+0000","last_active":"2023-10-08T01:11:21.764191+0000","last_peered":"2023-10-08T01:11:21.764191+0000","last_clean":"2023-10-08T01:11:21.764191+0000","last_became_active":"2023-10-08T01:11:20.720623+0000","last_became_peered":"2023-10-08T01:11:20.720623+0000","last_unstale":"2023-10-08T01:11:21.764191+0000","last_undegraded":"2023-10-08T01:11:21.764191+0000","last_fullsized":"2023-10-08T01:11:21.764191+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:56.568531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.757017+0000","last_change":"2023-10-08T01:11:21.755116+0000","last_active":"2023-10-08T01:11:21.757017+0000","last_peered":"2023-10-08T01:11:21.757017+0000","last_clean":"2023-10-08T01:11:21.757017+0000","last_became_active":"2023-10-08T01:11:21.754817+0000","last_became_peered":"2023-10-08T01:11:21.754817+0000","last_unstale":"2023-10-08T01:11:21.757017+0000","last_undegraded":"2023-10-08T01:11:21.757017+0000","last_fullsized":"2023-10-08T01:11:21.757017+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:55:03.140046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763929+0000","last_change":"2023-10-08T01:11:20.733962+0000","last_active":"2023-10-08T01:11:21.763929+0000","last_peered":"2023-10-08T01:11:21.763929+0000","last_clean":"2023-10-08T01:11:21.763929+0000","last_became_active":"2023-10-08T01:11:20.733675+0000","last_became_peered":"2023-10-08T01:11:20.733675+0000","last_unstale":"2023-10-08T01:11:21.763929+0000","last_undegraded":"2023-10-08T01:11:21.763929+0000","last_fullsized":"2023-10-08T01:11:21.763929+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:08.865881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750117+0000","last_change":"2023-10-08T01:11:21.746734+0000","last_active":"2023-10-08T01:11:21.750117+0000","last_peered":"2023-10-08T01:11:21.750117+0000","last_clean":"2023-10-08T01:11:21.750117+0000","last_became_active":"2023-10-08T01:11:21.746067+0000","last_became_peered":"2023-10-08T01:11:21.746067+0000","last_unstale":"2023-10-08T01:11:21.750117+0000","last_undegraded":"2023-10-08T01:11:21.750117+0000","last_fullsized":"2023-10-08T01:11:21.750117+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:51:25.914521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.727317+0000","last_change":"2023-10-08T01:11:20.721718+0000","last_active":"2023-10-08T01:11:21.727317+0000","last_peered":"2023-10-08T01:11:21.727317+0000","last_clean":"2023-10-08T01:11:21.727317+0000","last_became_active":"2023-10-08T01:11:20.721450+0000","last_became_peered":"2023-10-08T01:11:20.721450+0000","last_unstale":"2023-10-08T01:11:21.727317+0000","last_undegraded":"2023-10-08T01:11:21.727317+0000","last_fullsized":"2023-10-08T01:11:21.727317+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:31:30.461191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.739596+0000","last_change":"2023-10-08T01:11:21.737484+0000","last_active":"2023-10-08T01:11:21.739596+0000","last_peered":"2023-10-08T01:11:21.739596+0000","last_clean":"2023-10-08T01:11:21.739596+0000","last_became_active":"2023-10-08T01:11:21.737217+0000","last_became_peered":"2023-10-08T01:11:21.737217+0000","last_unstale":"2023-10-08T01:11:21.739596+0000","last_undegraded":"2023-10-08T01:11:21.739596+0000","last_fullsized":"2023-10-08T01:11:21.739596+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:36:03.061174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.733944+0000","last_change":"2023-10-08T01:11:20.727796+0000","last_active":"2023-10-08T01:11:21.733944+0000","last_peered":"2023-10-08T01:11:21.733944+0000","last_clean":"2023-10-08T01:11:21.733944+0000","last_became_active":"2023-10-08T01:11:20.727530+0000","last_became_peered":"2023-10-08T01:11:20.727530+0000","last_unstale":"2023-10-08T01:11:21.733944+0000","last_undegraded":"2023-10-08T01:11:21.733944+0000","last_fullsized":"2023-10-08T01:11:21.733944+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:52.202279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754300+0000","last_change":"2023-10-08T01:11:21.752232+0000","last_active":"2023-10-08T01:11:21.754300+0000","last_peered":"2023-10-08T01:11:21.754300+0000","last_clean":"2023-10-08T01:11:21.754300+0000","last_became_active":"2023-10-08T01:11:21.751917+0000","last_became_peered":"2023-10-08T01:11:21.751917+0000","last_unstale":"2023-10-08T01:11:21.754300+0000","last_undegraded":"2023-10-08T01:11:21.754300+0000","last_fullsized":"2023-10-08T01:11:21.754300+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:23:55.610314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"76'8","reported_seq":18,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763406+0000","last_change":"2023-10-08T01:11:20.725738+0000","last_active":"2023-10-08T01:11:21.763406+0000","last_peered":"2023-10-08T01:11:21.763406+0000","last_clean":"2023-10-08T01:11:21.763406+0000","last_became_active":"2023-10-08T01:11:20.725430+0000","last_became_peered":"2023-10-08T01:11:20.725430+0000","last_unstale":"2023-10-08T01:11:21.763406+0000","last_undegraded":"2023-10-08T01:11:21.763406+0000","last_fullsized":"2023-10-08T01:11:21.763406+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:34:11.596616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.746488+0000","last_change":"2023-10-08T01:11:21.743946+0000","last_active":"2023-10-08T01:11:21.746488+0000","last_peered":"2023-10-08T01:11:21.746488+0000","last_clean":"2023-10-08T01:11:21.746488+0000","last_became_active":"2023-10-08T01:11:21.743573+0000","last_became_peered":"2023-10-08T01:11:21.743573+0000","last_unstale":"2023-10-08T01:11:21.746488+0000","last_undegraded":"2023-10-08T01:11:21.746488+0000","last_fullsized":"2023-10-08T01:11:21.746488+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:55:03.140046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763853+0000","last_change":"2023-10-08T01:11:20.733998+0000","last_active":"2023-10-08T01:11:21.763853+0000","last_peered":"2023-10-08T01:11:21.763853+0000","last_clean":"2023-10-08T01:11:21.763853+0000","last_became_active":"2023-10-08T01:11:20.733730+0000","last_became_peered":"2023-10-08T01:11:20.733730+0000","last_unstale":"2023-10-08T01:11:21.763853+0000","last_undegraded":"2023-10-08T01:11:21.763853+0000","last_fullsized":"2023-10-08T01:11:21.763853+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:10.519648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.755151+0000","last_change":"2023-10-08T01:11:21.752891+0000","last_active":"2023-10-08T01:11:21.755151+0000","last_peered":"2023-10-08T01:11:21.755151+0000","last_clean":"2023-10-08T01:11:21.755151+0000","last_became_active":"2023-10-08T01:11:21.752609+0000","last_became_peered":"2023-10-08T01:11:21.752609+0000","last_unstale":"2023-10-08T01:11:21.755151+0000","last_undegraded":"2023-10-08T01:11:21.755151+0000","last_fullsized":"2023-10-08T01:11:21.755151+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:13.284594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.755982+0000","last_change":"2023-10-08T01:11:20.718240+0000","last_active":"2023-10-08T01:11:21.755982+0000","last_peered":"2023-10-08T01:11:21.755982+0000","last_clean":"2023-10-08T01:11:21.755982+0000","last_became_active":"2023-10-08T01:11:20.717987+0000","last_became_peered":"2023-10-08T01:11:20.717987+0000","last_unstale":"2023-10-08T01:11:21.755982+0000","last_undegraded":"2023-10-08T01:11:21.755982+0000","last_fullsized":"2023-10-08T01:11:21.755982+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:39.137134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.747376+0000","last_change":"2023-10-08T01:11:21.741530+0000","last_active":"2023-10-08T01:11:21.747376+0000","last_peered":"2023-10-08T01:11:21.747376+0000","last_clean":"2023-10-08T01:11:21.747376+0000","last_became_active":"2023-10-08T01:11:21.740579+0000","last_became_peered":"2023-10-08T01:11:21.740579+0000","last_unstale":"2023-10-08T01:11:21.747376+0000","last_undegraded":"2023-10-08T01:11:21.747376+0000","last_fullsized":"2023-10-08T01:11:21.747376+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:20.017188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"76'1","reported_seq":11,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.762764+0000","last_change":"2023-10-08T01:11:20.720816+0000","last_active":"2023-10-08T01:11:21.762764+0000","last_peered":"2023-10-08T01:11:21.762764+0000","last_clean":"2023-10-08T01:11:21.762764+0000","last_became_active":"2023-10-08T01:11:20.720541+0000","last_became_peered":"2023-10-08T01:11:20.720541+0000","last_unstale":"2023-10-08T01:11:21.762764+0000","last_undegraded":"2023-10-08T01:11:21.762764+0000","last_fullsized":"2023-10-08T01:11:21.762764+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:20:35.034982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754336+0000","last_change":"2023-10-08T01:11:21.752519+0000","last_active":"2023-10-08T01:11:21.754336+0000","last_peered":"2023-10-08T01:11:21.754336+0000","last_clean":"2023-10-08T01:11:21.754336+0000","last_became_active":"2023-10-08T01:11:21.752257+0000","last_became_peered":"2023-10-08T01:11:21.752257+0000","last_unstale":"2023-10-08T01:11:21.754336+0000","last_undegraded":"2023-10-08T01:11:21.754336+0000","last_fullsized":"2023-10-08T01:11:21.754336+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:45.501970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.725351+0000","last_change":"2023-10-08T01:11:20.728496+0000","last_active":"2023-10-08T01:11:21.725351+0000","last_peered":"2023-10-08T01:11:21.725351+0000","last_clean":"2023-10-08T01:11:21.725351+0000","last_became_active":"2023-10-08T01:11:20.728220+0000","last_became_peered":"2023-10-08T01:11:20.728220+0000","last_unstale":"2023-10-08T01:11:21.725351+0000","last_undegraded":"2023-10-08T01:11:21.725351+0000","last_fullsized":"2023-10-08T01:11:21.725351+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:39:48.896753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.755199+0000","last_change":"2023-10-08T01:11:21.752958+0000","last_active":"2023-10-08T01:11:21.755199+0000","last_peered":"2023-10-08T01:11:21.755199+0000","last_clean":"2023-10-08T01:11:21.755199+0000","last_became_active":"2023-10-08T01:11:21.752623+0000","last_became_peered":"2023-10-08T01:11:21.752623+0000","last_unstale":"2023-10-08T01:11:21.755199+0000","last_undegraded":"2023-10-08T01:11:21.755199+0000","last_fullsized":"2023-10-08T01:11:21.755199+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:47:13.443993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.764045+0000","last_change":"2023-10-08T01:11:20.730842+0000","last_active":"2023-10-08T01:11:21.764045+0000","last_peered":"2023-10-08T01:11:21.764045+0000","last_clean":"2023-10-08T01:11:21.764045+0000","last_became_active":"2023-10-08T01:11:20.727733+0000","last_became_peered":"2023-10-08T01:11:20.727733+0000","last_unstale":"2023-10-08T01:11:21.764045+0000","last_undegraded":"2023-10-08T01:11:21.764045+0000","last_fullsized":"2023-10-08T01:11:21.764045+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:27:15.118911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.735588+0000","last_change":"2023-10-08T01:11:21.730330+0000","last_active":"2023-10-08T01:11:21.735588+0000","last_peered":"2023-10-08T01:11:21.735588+0000","last_clean":"2023-10-08T01:11:21.735588+0000","last_became_active":"2023-10-08T01:11:21.730042+0000","last_became_peered":"2023-10-08T01:11:21.730042+0000","last_unstale":"2023-10-08T01:11:21.735588+0000","last_undegraded":"2023-10-08T01:11:21.735588+0000","last_fullsized":"2023-10-08T01:11:21.735588+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:49.923284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.745109+0000","last_change":"2023-10-08T01:11:20.724381+0000","last_active":"2023-10-08T01:11:21.745109+0000","last_peered":"2023-10-08T01:11:21.745109+0000","last_clean":"2023-10-08T01:11:21.745109+0000","last_became_active":"2023-10-08T01:11:20.724176+0000","last_became_peered":"2023-10-08T01:11:20.724176+0000","last_unstale":"2023-10-08T01:11:21.745109+0000","last_undegraded":"2023-10-08T01:11:21.745109+0000","last_fullsized":"2023-10-08T01:11:21.745109+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:42.276926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.740190+0000","last_change":"2023-10-08T01:11:21.739225+0000","last_active":"2023-10-08T01:11:21.740190+0000","last_peered":"2023-10-08T01:11:21.740190+0000","last_clean":"2023-10-08T01:11:21.740190+0000","last_became_active":"2023-10-08T01:11:21.738948+0000","last_became_peered":"2023-10-08T01:11:21.738948+0000","last_unstale":"2023-10-08T01:11:21.740190+0000","last_undegraded":"2023-10-08T01:11:21.740190+0000","last_fullsized":"2023-10-08T01:11:21.740190+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:53:44.365874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.736228+0000","last_change":"2023-10-08T01:11:20.732180+0000","last_active":"2023-10-08T01:11:21.736228+0000","last_peered":"2023-10-08T01:11:21.736228+0000","last_clean":"2023-10-08T01:11:21.736228+0000","last_became_active":"2023-10-08T01:11:20.731915+0000","last_became_peered":"2023-10-08T01:11:20.731915+0000","last_unstale":"2023-10-08T01:11:21.736228+0000","last_undegraded":"2023-10-08T01:11:21.736228+0000","last_fullsized":"2023-10-08T01:11:21.736228+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:27:33.628997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750865+0000","last_change":"2023-10-08T01:11:21.749436+0000","last_active":"2023-10-08T01:11:21.750865+0000","last_peered":"2023-10-08T01:11:21.750865+0000","last_clean":"2023-10-08T01:11:21.750865+0000","last_became_active":"2023-10-08T01:11:21.748525+0000","last_became_peered":"2023-10-08T01:11:21.748525+0000","last_unstale":"2023-10-08T01:11:21.750865+0000","last_undegraded":"2023-10-08T01:11:21.750865+0000","last_fullsized":"2023-10-08T01:11:21.750865+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:09.873323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"76'2","reported_seq":12,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.764323+0000","last_change":"2023-10-08T01:11:20.734568+0000","last_active":"2023-10-08T01:11:21.764323+0000","last_peered":"2023-10-08T01:11:21.764323+0000","last_clean":"2023-10-08T01:11:21.764323+0000","last_became_active":"2023-10-08T01:11:20.734290+0000","last_became_peered":"2023-10-08T01:11:20.734290+0000","last_unstale":"2023-10-08T01:11:21.764323+0000","last_undegraded":"2023-10-08T01:11:21.764323+0000","last_fullsized":"2023-10-08T01:11:21.764323+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:48:58.092204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.750407+0000","last_change":"2023-10-08T01:11:21.747744+0000","last_active":"2023-10-08T01:11:21.750407+0000","last_peered":"2023-10-08T01:11:21.750407+0000","last_clean":"2023-10-08T01:11:21.750407+0000","last_became_active":"2023-10-08T01:11:21.746922+0000","last_became_peered":"2023-10-08T01:11:21.746922+0000","last_unstale":"2023-10-08T01:11:21.750407+0000","last_undegraded":"2023-10-08T01:11:21.750407+0000","last_fullsized":"2023-10-08T01:11:21.750407+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:49.923284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"77'1","reported_seq":11,"reported_epoch":77,"state":"active+clean","last_fresh":"2023-10-08T01:11:22.867926+0000","last_change":"2023-10-08T01:11:20.726552+0000","last_active":"2023-10-08T01:11:22.867926+0000","last_peered":"2023-10-08T01:11:22.867926+0000","last_clean":"2023-10-08T01:11:22.867926+0000","last_became_active":"2023-10-08T01:11:20.726287+0000","last_became_peered":"2023-10-08T01:11:20.726287+0000","last_unstale":"2023-10-08T01:11:22.867926+0000","last_undegraded":"2023-10-08T01:11:22.867926+0000","last_fullsized":"2023-10-08T01:11:22.867926+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:56.568531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.753302+0000","last_change":"2023-10-08T01:11:21.745510+0000","last_active":"2023-10-08T01:11:21.753302+0000","last_peered":"2023-10-08T01:11:21.753302+0000","last_clean":"2023-10-08T01:11:21.753302+0000","last_became_active":"2023-10-08T01:11:21.745239+0000","last_became_peered":"2023-10-08T01:11:21.745239+0000","last_unstale":"2023-10-08T01:11:21.753302+0000","last_undegraded":"2023-10-08T01:11:21.753302+0000","last_fullsized":"2023-10-08T01:11:21.753302+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:11.370692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.727832+0000","last_change":"2023-10-08T01:11:20.720757+0000","last_active":"2023-10-08T01:11:21.727832+0000","last_peered":"2023-10-08T01:11:21.727832+0000","last_clean":"2023-10-08T01:11:21.727832+0000","last_became_active":"2023-10-08T01:11:20.720491+0000","last_became_peered":"2023-10-08T01:11:20.720491+0000","last_unstale":"2023-10-08T01:11:21.727832+0000","last_undegraded":"2023-10-08T01:11:21.727832+0000","last_fullsized":"2023-10-08T01:11:21.727832+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:46.734068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.751178+0000","last_change":"2023-10-08T01:11:21.744010+0000","last_active":"2023-10-08T01:11:21.751178+0000","last_peered":"2023-10-08T01:11:21.751178+0000","last_clean":"2023-10-08T01:11:21.751178+0000","last_became_active":"2023-10-08T01:11:21.743772+0000","last_became_peered":"2023-10-08T01:11:21.743772+0000","last_unstale":"2023-10-08T01:11:21.751178+0000","last_undegraded":"2023-10-08T01:11:21.751178+0000","last_fullsized":"2023-10-08T01:11:21.751178+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:55.596379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.736696+0000","last_change":"2023-10-08T01:11:20.726583+0000","last_active":"2023-10-08T01:11:21.736696+0000","last_peered":"2023-10-08T01:11:21.736696+0000","last_clean":"2023-10-08T01:11:21.736696+0000","last_became_active":"2023-10-08T01:11:20.725722+0000","last_became_peered":"2023-10-08T01:11:20.725722+0000","last_unstale":"2023-10-08T01:11:21.736696+0000","last_undegraded":"2023-10-08T01:11:21.736696+0000","last_fullsized":"2023-10-08T01:11:21.736696+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:27:38.586089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.763749+0000","last_change":"2023-10-08T01:11:21.756555+0000","last_active":"2023-10-08T01:11:21.763749+0000","last_peered":"2023-10-08T01:11:21.763749+0000","last_clean":"2023-10-08T01:11:21.763749+0000","last_became_active":"2023-10-08T01:11:21.756259+0000","last_became_peered":"2023-10-08T01:11:21.756259+0000","last_unstale":"2023-10-08T01:11:21.763749+0000","last_undegraded":"2023-10-08T01:11:21.763749+0000","last_fullsized":"2023-10-08T01:11:21.763749+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:28.116582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.738477+0000","last_change":"2023-10-08T01:11:20.727665+0000","last_active":"2023-10-08T01:11:21.738477+0000","last_peered":"2023-10-08T01:11:21.738477+0000","last_clean":"2023-10-08T01:11:21.738477+0000","last_became_active":"2023-10-08T01:11:20.727352+0000","last_became_peered":"2023-10-08T01:11:20.727352+0000","last_unstale":"2023-10-08T01:11:21.738477+0000","last_undegraded":"2023-10-08T01:11:21.738477+0000","last_fullsized":"2023-10-08T01:11:21.738477+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:13.442980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754472+0000","last_change":"2023-10-08T01:11:21.752252+0000","last_active":"2023-10-08T01:11:21.754472+0000","last_peered":"2023-10-08T01:11:21.754472+0000","last_clean":"2023-10-08T01:11:21.754472+0000","last_became_active":"2023-10-08T01:11:21.751961+0000","last_became_peered":"2023-10-08T01:11:21.751961+0000","last_unstale":"2023-10-08T01:11:21.754472+0000","last_undegraded":"2023-10-08T01:11:21.754472+0000","last_fullsized":"2023-10-08T01:11:21.754472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:44.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.728804+0000","last_change":"2023-10-08T01:11:20.722841+0000","last_active":"2023-10-08T01:11:21.728804+0000","last_peered":"2023-10-08T01:11:21.728804+0000","last_clean":"2023-10-08T01:11:21.728804+0000","last_became_active":"2023-10-08T01:11:20.722582+0000","last_became_peered":"2023-10-08T01:11:20.722582+0000","last_unstale":"2023-10-08T01:11:21.728804+0000","last_undegraded":"2023-10-08T01:11:21.728804+0000","last_fullsized":"2023-10-08T01:11:21.728804+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:32:56.546985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.754217+0000","last_change":"2023-10-08T01:11:21.746429+0000","last_active":"2023-10-08T01:11:21.754217+0000","last_peered":"2023-10-08T01:11:21.754217+0000","last_clean":"2023-10-08T01:11:21.754217+0000","last_became_active":"2023-10-08T01:11:21.746153+0000","last_became_peered":"2023-10-08T01:11:21.746153+0000","last_unstale":"2023-10-08T01:11:21.754217+0000","last_undegraded":"2023-10-08T01:11:21.754217+0000","last_fullsized":"2023-10-08T01:11:21.754217+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:49:13.637672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.738835+0000","last_change":"2023-10-08T01:11:20.730285+0000","last_active":"2023-10-08T01:11:21.738835+0000","last_peered":"2023-10-08T01:11:21.738835+0000","last_clean":"2023-10-08T01:11:21.738835+0000","last_became_active":"2023-10-08T01:11:20.730030+0000","last_became_peered":"2023-10-08T01:11:20.730030+0000","last_unstale":"2023-10-08T01:11:21.738835+0000","last_undegraded":"2023-10-08T01:11:21.738835+0000","last_fullsized":"2023-10-08T01:11:21.738835+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:46:54.188581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.744036+0000","last_change":"2023-10-08T01:11:21.740890+0000","last_active":"2023-10-08T01:11:21.744036+0000","last_peered":"2023-10-08T01:11:21.744036+0000","last_clean":"2023-10-08T01:11:21.744036+0000","last_became_active":"2023-10-08T01:11:21.740620+0000","last_became_peered":"2023-10-08T01:11:21.740620+0000","last_unstale":"2023-10-08T01:11:21.744036+0000","last_undegraded":"2023-10-08T01:11:21.744036+0000","last_fullsized":"2023-10-08T01:11:21.744036+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:00.075685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.729749+0000","last_change":"2023-10-08T01:11:20.731212+0000","last_active":"2023-10-08T01:11:21.729749+0000","last_peered":"2023-10-08T01:11:21.729749+0000","last_clean":"2023-10-08T01:11:21.729749+0000","last_became_active":"2023-10-08T01:11:20.730951+0000","last_became_peered":"2023-10-08T01:11:20.730951+0000","last_unstale":"2023-10-08T01:11:21.729749+0000","last_undegraded":"2023-10-08T01:11:21.729749+0000","last_fullsized":"2023-10-08T01:11:21.729749+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:38:36.412023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.740780+0000","last_change":"2023-10-08T01:11:21.737714+0000","last_active":"2023-10-08T01:11:21.740780+0000","last_peered":"2023-10-08T01:11:21.740780+0000","last_clean":"2023-10-08T01:11:21.740780+0000","last_became_active":"2023-10-08T01:11:21.737099+0000","last_became_peered":"2023-10-08T01:11:21.737099+0000","last_unstale":"2023-10-08T01:11:21.740780+0000","last_undegraded":"2023-10-08T01:11:21.740780+0000","last_fullsized":"2023-10-08T01:11:21.740780+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":75,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:20.703453+0000","last_clean_scrub_stamp":"2023-10-08T01:11:20.703453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:44.804580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.727171+0000","last_change":"2023-10-08T01:11:20.723634+0000","last_active":"2023-10-08T01:11:21.727171+0000","last_peered":"2023-10-08T01:11:21.727171+0000","last_clean":"2023-10-08T01:11:21.727171+0000","last_became_active":"2023-10-08T01:11:20.723384+0000","last_became_peered":"2023-10-08T01:11:20.723384+0000","last_unstale":"2023-10-08T01:11:21.727171+0000","last_undegraded":"2023-10-08T01:11:21.727171+0000","last_fullsized":"2023-10-08T01:11:21.727171+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:11:19.696010+0000","last_clean_scrub_stamp":"2023-10-08T01:11:19.696010+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:02:13.442980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"66'161","reported_seq":369,"reported_epoch":76,"state":"active+clean","last_fresh":"2023-10-08T01:11:21.732900+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:11:21.732900+0000","last_peered":"2023-10-08T01:11:21.732900+0000","last_clean":"2023-10-08T01:11:21.732900+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:11:21.732900+0000","last_undegraded":"2023-10-08T01:11:21.732900+0000","last_fullsized":"2023-10-08T01:11:21.732900+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575039,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302880,"kb_used_data":1496,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068960,"statfs":{"total":96636764160,"available":96326615040,"internally_reserved":0,"allocated":1531904,"data_stored":1035740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66000000000000003}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.73899999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.61599999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.64500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.57899999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.751},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.63300000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575039,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302292,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069548,"statfs":{"total":96636764160,"available":96327217152,"internally_reserved":0,"allocated":929792,"data_stored":445212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.47399999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.54800000000000004}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.5}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.625},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.52700000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.73999999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575039,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302296,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069544,"statfs":{"total":96636764160,"available":96327213056,"internally_reserved":0,"allocated":933888,"data_stored":445302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.501},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.49099999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.67700000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.63}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.58499999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.754}]}]},{"osd":0,"up_from":13,"seq":55834575038,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302888,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068952,"statfs":{"total":96636764160,"available":96326606848,"internally_reserved":0,"allocated":1540096,"data_stored":1037670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.46200000000000002,"15min":0.45900000000000002},"min":{"1min":0.32600000000000001,"5min":0.28499999999999998,"15min":0.28399999999999997},"max":{"1min":0.66100000000000003,"5min":0.68300000000000005,"15min":0.997},"last":0.437},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.499,"15min":0.497},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.238},"last":0.32300000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.38600000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.63100000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.51800000000000002,"15min":0.53200000000000003},"min":{"1min":0.42299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":1.7370000000000001},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.41799999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.72599999999999998,"5min":0.90700000000000003,"15min":1.0489999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.59799999999999998},"min":{"1min":0.498,"5min":0.42199999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.3640000000000001,"15min":1.3640000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.753,"15min":0.75600000000000001},"min":{"1min":0.629,"5min":0.53000000000000003,"15min":0.45100000000000001},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.617},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.77100000000000002,"15min":0.77500000000000002},"min":{"1min":0.64200000000000002,"5min":0.52800000000000002,"15min":0.48899999999999999},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.64200000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.58899999999999997,"5min":0.56999999999999995,"15min":0.51000000000000001},"max":{"1min":1.1950000000000001,"5min":1.2729999999999999,"15min":1.5960000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.80100000000000005,"15min":0.81699999999999995},"min":{"1min":0.71299999999999997,"5min":0.54200000000000004,"15min":0.45300000000000001},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.5289999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.79800000000000004,"15min":0.80300000000000005},"min":{"1min":0.61299999999999999,"5min":0.50900000000000001,"15min":0.46600000000000003},"max":{"1min":1.361,"5min":1.361,"15min":1.3779999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.629,"5min":0.52700000000000002,"15min":0.51200000000000001},"max":{"1min":1.129,"5min":1.2390000000000001,"15min":1.6459999999999999},"last":0.73899999999999999}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.80300000000000005,"15min":0.81999999999999995},"min":{"1min":0.67500000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.82899999999999996,"15min":0.84099999999999997},"min":{"1min":0.61199999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.75}]}]},{"osd":2,"up_from":13,"seq":55834575038,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306396,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065444,"statfs":{"total":96636764160,"available":96323014656,"internally_reserved":0,"allocated":937984,"data_stored":445268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.52900000000000003}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.625},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.57199999999999995}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.76900000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.64200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575038,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306396,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065444,"statfs":{"total":96636764160,"available":96323014656,"internally_reserved":0,"allocated":937984,"data_stored":445268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.45000000000000001,"15min":0.46100000000000002},"min":{"1min":0.33700000000000002,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.627,"5min":0.76000000000000001,"15min":0.80100000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.48399999999999999},"min":{"1min":0.40100000000000002,"5min":0.32000000000000001,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.35499999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.46999999999999997,"15min":0.47599999999999998},"min":{"1min":0.39600000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.64900000000000002,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.52500000000000002},"min":{"1min":0.42399999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.67400000000000004,"5min":0.82999999999999996,"15min":1.5629999999999999},"last":0.372}]},{"osd":2,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.58499999999999996,"15min":0.58599999999999997},"min":{"1min":0.51400000000000001,"5min":0.40500000000000003,"15min":0.34100000000000003},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.55600000000000005,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.77200000000000002,"5min":0.89000000000000001,"15min":1.3899999999999999},"last":0.44400000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.75600000000000001,"15min":0.76400000000000001},"min":{"1min":0.55600000000000005,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":1.095,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.753,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.58699999999999997,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":0.96099999999999997,"5min":1.3240000000000001,"15min":1.413},"last":0.58299999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.77000000000000002,"15min":0.78100000000000003},"min":{"1min":0.64100000000000001,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":0.93300000000000005,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76100000000000001,"15min":0.77900000000000003},"min":{"1min":0.56799999999999995,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.92100000000000004,"5min":0.997,"15min":1.431},"last":0.66300000000000003}]},{"osd":6,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78000000000000003,"15min":0.80200000000000005},"min":{"1min":0.59899999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.032,"5min":1.0600000000000001,"15min":1.458},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79300000000000004,"15min":0.79600000000000004},"min":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.44500000000000001},"max":{"1min":1.0629999999999999,"5min":1.1499999999999999,"15min":1.52},"last":0.55300000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79300000000000004,"15min":0.80100000000000005},"min":{"1min":0.59799999999999998,"5min":0.53900000000000003,"15min":0.46500000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.389},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.79700000000000004,"15min":0.80300000000000005},"min":{"1min":0.65800000000000003,"5min":0.57799999999999996,"15min":0.5},"max":{"1min":1.054,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.67400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575038,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310496,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061344,"statfs":{"total":96636764160,"available":96318816256,"internally_reserved":0,"allocated":942080,"data_stored":446748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.90000000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.73999999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.90900000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.88400000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.87}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.79300000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575038,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":937984,"data_stored":445836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.745},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.78800000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.83399999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.80200000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.51300000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.53800000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.65700000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:11:31.764 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T01:11:31.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T01:11:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.985+0000 7f89f739d700 1 Processor -- start 2023-10-08T01:11:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.985+0000 7f89f739d700 1 -- start start 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.986+0000 7f89f739d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f00c7840 0x7f89f0159920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.986+0000 7f89f739d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89f00c7c10 0x7f89f0159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.986+0000 7f89f739d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f006e510 con 0x7f89f015a3a0 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.986+0000 7f89f739d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00c7220 con 0x7f89f00c7c10 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.986+0000 7f89f739d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f006df20 con 0x7f89f00c7840 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.986+0000 7f89f5b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89f00c7c10 0x7f89f0159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.986+0000 7f89f5b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89f00c7c10 0x7f89f0159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52472/0 (socket says 172.21.15.119:52472) 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.986+0000 7f89f5b9a700 1 -- 172.21.15.119:0/2152863592 learned_addr learned my addr 172.21.15.119:0/2152863592 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f639b700 1 --2- 172.21.15.119:0/2152863592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f00c7840 0x7f89f0159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f5b9a700 1 -- 172.21.15.119:0/2152863592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f00c7840 msgr2=0x7f89f0159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f5b9a700 1 --2- 172.21.15.119:0/2152863592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f00c7840 0x7f89f0159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f5b9a700 1 -- 172.21.15.119:0/2152863592 >> v1:172.21.15.184:6789/0 conn(0x7f89f015a3a0 legacy=0x7f89f006e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f5b9a700 1 -- 172.21.15.119:0/2152863592 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0157040 con 0x7f89f00c7c10 2023-10-08T01:11:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f5b9a700 1 --2- 172.21.15.119:0/2152863592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f0159e60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f89e00099b0 tx=0x7f89e0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6ca1b04425d55e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f5399700 1 -- 172.21.15.119:0/2152863592 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000b8f0 con 0x7f89f00c7c10 2023-10-08T01:11:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f5399700 1 -- 172.21.15.119:0/2152863592 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000ba50 con 0x7f89f00c7c10 2023-10-08T01:11:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.987+0000 7f89f5399700 1 -- 172.21.15.119:0/2152863592 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e000bdf0 con 0x7f89f00c7c10 2023-10-08T01:11:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.988+0000 7f89f739d700 1 -- 172.21.15.119:0/2152863592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 msgr2=0x7f89f0159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.988+0000 7f89f739d700 1 --2- 172.21.15.119:0/2152863592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f0159e60 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f89e00099b0 tx=0x7f89e0009250 comp rx=0 tx=0).stop 2023-10-08T01:11:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.988+0000 7f89f739d700 1 -- 172.21.15.119:0/2152863592 shutdown_connections 2023-10-08T01:11:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.988+0000 7f89f739d700 1 --2- 172.21.15.119:0/2152863592 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f00c7840 0x7f89f0159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.988+0000 7f89f739d700 1 --2- 172.21.15.119:0/2152863592 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f0159e60 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.988+0000 7f89f739d700 1 -- 172.21.15.119:0/2152863592 >> 172.21.15.119:0/2152863592 conn(0x7f89f0068e20 msgr2=0x7f89f00cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.988+0000 7f89f739d700 1 -- 172.21.15.119:0/2152863592 shutdown_connections 2023-10-08T01:11:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.988+0000 7f89f739d700 1 -- 172.21.15.119:0/2152863592 wait complete. 2023-10-08T01:11:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.989+0000 7f89f739d700 1 Processor -- start 2023-10-08T01:11:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.990+0000 7f89f739d700 1 -- start start 2023-10-08T01:11:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.990+0000 7f89f739d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f00c7840 0x7f89f00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.990+0000 7f89f739d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f00c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.990+0000 7f89f739d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f015a3a0 0x7f89f00acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.990+0000 7f89f739d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f01542f0 con 0x7f89f00c7c10 2023-10-08T01:11:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.990+0000 7f89f739d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00ad160 con 0x7f89f00c7840 2023-10-08T01:11:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.990+0000 7f89f739d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00adfa0 con 0x7f89f015a3a0 2023-10-08T01:11:31.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f00c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f00c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52484/0 (socket says 172.21.15.119:52484) 2023-10-08T01:11:31.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 -- 172.21.15.119:0/1735812308 learned_addr learned my addr 172.21.15.119:0/1735812308 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:31.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f639b700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f00c7840 0x7f89f00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 -- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f015a3a0 msgr2=0x7f89f00acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f6b9c700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f015a3a0 0x7f89f00acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f015a3a0 0x7f89f00acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 -- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f00c7840 msgr2=0x7f89f00b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f00c7840 0x7f89f00b15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 -- 172.21.15.119:0/1735812308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e00094f0 con 0x7f89f00c7c10 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f639b700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f00c7840 0x7f89f00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f6b9c700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f015a3a0 0x7f89f00acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.991+0000 7f89f5b9a700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f00c5330 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f89f0064f90 tx=0x7f89e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.992+0000 7f89e67fc700 1 -- 172.21.15.119:0/1735812308 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0024440 con 0x7f89f00c7c10 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.992+0000 7f89e67fc700 1 -- 172.21.15.119:0/1735812308 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e00245a0 con 0x7f89f00c7c10 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.992+0000 7f89e67fc700 1 -- 172.21.15.119:0/1735812308 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0024940 con 0x7f89f00c7c10 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.992+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f00ad390 con 0x7f89f00c7c10 2023-10-08T01:11:31.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.992+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f00ad950 con 0x7f89f00c7c10 2023-10-08T01:11:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.993+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f00ae110 con 0x7f89f00c7c10 2023-10-08T01:11:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.994+0000 7f89e67fc700 1 -- 172.21.15.119:0/1735812308 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f89e001a420 con 0x7f89f00c7c10 2023-10-08T01:11:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.994+0000 7f89e67fc700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f89d4066f30 0x7f89d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:31.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.994+0000 7f89e67fc700 1 -- 172.21.15.119:0/1735812308 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f89e0086420 con 0x7f89f00c7c10 2023-10-08T01:11:31.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.995+0000 7f89f639b700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f89d4066f30 0x7f89d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:31.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.995+0000 7f89f639b700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f89d4066f30 0x7f89d40693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006fd0 tx=0x7f89ec008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:31.998+0000 7f89e67fc700 1 -- 172.21.15.119:0/1735812308 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89e00534e0 con 0x7f89f00c7c10 2023-10-08T01:11:32.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.265+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f89f004c3c0 con 0x7f89f00c7c10 2023-10-08T01:11:32.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.270+0000 7f89e67fc700 1 -- 172.21.15.119:0/1735812308 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f89e0056b00 con 0x7f89f00c7c10 2023-10-08T01:11:32.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f89d4066f30 msgr2=0x7f89d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:32.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f89d4066f30 0x7f89d40693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f89ec006fd0 tx=0x7f89ec008040 comp rx=0 tx=0).stop 2023-10-08T01:11:32.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 msgr2=0x7f89f00c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:32.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f00c5330 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f89f0064f90 tx=0x7f89e0017000 comp rx=0 tx=0).stop 2023-10-08T01:11:32.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 shutdown_connections 2023-10-08T01:11:32.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f89d4066f30 0x7f89d40693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89f015a3a0 0x7f89f00acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89f00c7840 0x7f89f00b15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 --2- 172.21.15.119:0/1735812308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89f00c7c10 0x7f89f00c5330 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 >> 172.21.15.119:0/1735812308 conn(0x7f89f0068e20 msgr2=0x7f89f0159830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:32.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 shutdown_connections 2023-10-08T01:11:32.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.273+0000 7f89f739d700 1 -- 172.21.15.119:0/1735812308 wait complete. 2023-10-08T01:11:32.275 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T01:11:32.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:32.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.471+0000 7fb951f9d700 1 Processor -- start 2023-10-08T01:11:32.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.471+0000 7fb951f9d700 1 -- start start 2023-10-08T01:11:32.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.472+0000 7fb951f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb94c150180 0x7fb94c157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:32.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.472+0000 7fb951f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0c7410 0x7fb94c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:32.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.472+0000 7fb951f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb94c06e2a0 con 0x7fb94c150550 2023-10-08T01:11:32.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.472+0000 7fb951f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb94c06b2e0 con 0x7fb94c150180 2023-10-08T01:11:32.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.472+0000 7fb951f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0c78e0 con 0x7fb94c0c7410 2023-10-08T01:11:32.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.472+0000 7fb950f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb94c150180 0x7fb94c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.472+0000 7fb950f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb94c150180 0x7fb94c157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52490/0 (socket says 172.21.15.119:52490) 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.472+0000 7fb950f9b700 1 -- 172.21.15.119:0/1592103079 learned_addr learned my addr 172.21.15.119:0/1592103079 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb95179c700 1 --2- 172.21.15.119:0/1592103079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0c7410 0x7fb94c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb950f9b700 1 -- 172.21.15.119:0/1592103079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0c7410 msgr2=0x7fb94c069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb950f9b700 1 --2- 172.21.15.119:0/1592103079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0c7410 0x7fb94c069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb950f9b700 1 -- 172.21.15.119:0/1592103079 >> v1:172.21.15.119:6789/0 conn(0x7fb94c150550 legacy=0x7fb94c0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb950f9b700 1 -- 172.21.15.119:0/1592103079 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0c7bc0 con 0x7fb94c150180 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb94b7fe700 1 -- 172.21.15.119:0/1592103079 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 528209665 0 0) 0x7fb94c06e2a0 con 0x7fb94c150550 2023-10-08T01:11:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb95179c700 1 --2- 172.21.15.119:0/1592103079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0c7410 0x7fb94c069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb950f9b700 1 --2- 172.21.15.119:0/1592103079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c150180 0x7fb94c157770 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb94000bc50 tx=0x7fb94000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af54c62d101281c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb94b7fe700 1 -- 172.21.15.119:0/1592103079 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94000be60 con 0x7fb94c150180 2023-10-08T01:11:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb94b7fe700 1 -- 172.21.15.119:0/1592103079 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94000de00 con 0x7fb94c150180 2023-10-08T01:11:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.473+0000 7fb94b7fe700 1 -- 172.21.15.119:0/1592103079 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9400196b0 con 0x7fb94c150180 2023-10-08T01:11:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.474+0000 7fb951f9d700 1 -- 172.21.15.119:0/1592103079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c150180 msgr2=0x7fb94c157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.474+0000 7fb951f9d700 1 --2- 172.21.15.119:0/1592103079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c150180 0x7fb94c157770 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb94000bc50 tx=0x7fb94000b4f0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.474+0000 7fb951f9d700 1 -- 172.21.15.119:0/1592103079 shutdown_connections 2023-10-08T01:11:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.474+0000 7fb951f9d700 1 --2- 172.21.15.119:0/1592103079 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c0c7410 0x7fb94c069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.474+0000 7fb951f9d700 1 --2- 172.21.15.119:0/1592103079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c150180 0x7fb94c157770 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.474+0000 7fb951f9d700 1 -- 172.21.15.119:0/1592103079 >> 172.21.15.119:0/1592103079 conn(0x7fb94c0cc1f0 msgr2=0x7fb94c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.474+0000 7fb951f9d700 1 -- 172.21.15.119:0/1592103079 shutdown_connections 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.474+0000 7fb951f9d700 1 -- 172.21.15.119:0/1592103079 wait complete. 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.475+0000 7fb951f9d700 1 Processor -- start 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.475+0000 7fb951f9d700 1 -- start start 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.476+0000 7fb951f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c0c7410 0x7fb94c0c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.476+0000 7fb951f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c150180 0x7fb94c0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.476+0000 7fb951f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c150550 0x7fb94c168ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.476+0000 7fb951f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c1698c0 con 0x7fb94c0c7410 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.476+0000 7fb951f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0b13d0 con 0x7fb94c150180 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.476+0000 7fb951f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb94c0c5eb0 con 0x7fb94c150550 2023-10-08T01:11:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c0c7410 0x7fb94c0c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:32.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c0c7410 0x7fb94c0c5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52504/0 (socket says 172.21.15.119:52504) 2023-10-08T01:11:32.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 -- 172.21.15.119:0/997052601 learned_addr learned my addr 172.21.15.119:0/997052601 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:32.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 -- 172.21.15.119:0/997052601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c150550 msgr2=0x7fb94c168ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:32.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c150550 0x7fb94c168ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 -- 172.21.15.119:0/997052601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c150180 msgr2=0x7fb94c0ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:32.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c150180 0x7fb94c0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 -- 172.21.15.119:0/997052601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94000b790 con 0x7fb94c0c7410 2023-10-08T01:11:32.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb950f9b700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c0c7410 0x7fb94c0c5970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb94c06fe40 tx=0x7fb94001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:32.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb948ff9700 1 -- 172.21.15.119:0/997052601 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940010540 con 0x7fb94c0c7410 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb948ff9700 1 -- 172.21.15.119:0/997052601 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9400106a0 con 0x7fb94c0c7410 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.477+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb94c0b0b30 con 0x7fb94c0c7410 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.478+0000 7fb948ff9700 1 -- 172.21.15.119:0/997052601 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb940010a60 con 0x7fb94c0c7410 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.478+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb94c0b10f0 con 0x7fb94c0c7410 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.479+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb94c04c3c0 con 0x7fb94c0c7410 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.479+0000 7fb948ff9700 1 -- 172.21.15.119:0/997052601 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb940022030 con 0x7fb94c0c7410 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.480+0000 7fb948ff9700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb930066f30 0x7fb9300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.480+0000 7fb948ff9700 1 -- 172.21.15.119:0/997052601 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb940014070 con 0x7fb94c0c7410 2023-10-08T01:11:32.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.481+0000 7fb94bfff700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb930066f30 0x7fb9300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:32.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.481+0000 7fb94bfff700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb930066f30 0x7fb9300693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb94c166660 tx=0x7fb938007b40 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:32.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.484+0000 7fb948ff9700 1 -- 172.21.15.119:0/997052601 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9400559d0 con 0x7fb94c0c7410 2023-10-08T01:11:32.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.718+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb94c06ada0 con 0x7fb94c0c7410 2023-10-08T01:11:32.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.718+0000 7fb948ff9700 1 -- 172.21.15.119:0/997052601 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fb940058fb0 con 0x7fb94c0c7410 2023-10-08T01:11:32.718 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:32.718 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4024548231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":4024548231},{"type":"v1","addr":"172.21.15.184:6835","nonce":4024548231}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1924614082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1924614082},{"type":"v1","addr":"172.21.15.119:6835","nonce":1924614082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5902,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/296234355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":296234355},{"type":"v1","addr":"172.21.15.184:6834","nonce":296234355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:11:21.183828+0000","modified":"2023-10-08T01:11:25.168981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6225":{"gid":6225,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6834/926130691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":926130691},{"type":"v1","addr":"172.21.15.119:6834","nonce":926130691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T01:11:32.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb930066f30 msgr2=0x7fb9300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:32.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb930066f30 0x7fb9300693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb94c166660 tx=0x7fb938007b40 comp rx=0 tx=0).stop 2023-10-08T01:11:32.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c0c7410 msgr2=0x7fb94c0c5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:32.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c0c7410 0x7fb94c0c5970 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb94c06fe40 tx=0x7fb94001b000 comp rx=0 tx=0).stop 2023-10-08T01:11:32.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 shutdown_connections 2023-10-08T01:11:32.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb930066f30 0x7fb9300693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb94c150550 0x7fb94c168ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb94c0c7410 0x7fb94c0c5970 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 --2- 172.21.15.119:0/997052601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb94c150180 0x7fb94c0ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:32.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 >> 172.21.15.119:0/997052601 conn(0x7fb94c0cc1f0 msgr2=0x7fb94c157770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:32.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 shutdown_connections 2023-10-08T01:11:32.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:32.721+0000 7fb951f9d700 1 -- 172.21.15.119:0/997052601 wait complete. 2023-10-08T01:11:32.722 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 32 2023-10-08T01:11:33.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T01:11:21.183828+0000', 'modified': '2023-10-08T01:11:25.168981+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6225}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6225': {'gid': 6225, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6834/926130691', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6832', 'nonce': 926130691}, {'type': 'v1', 'addr': '172.21.15.119:6834', 'nonce': 926130691}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T01:11:33.142 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T01:11:33.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T01:11:33.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.326+0000 7fe5cfbff700 1 Processor -- start 2023-10-08T01:11:33.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.326+0000 7fe5cfbff700 1 -- start start 2023-10-08T01:11:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.327+0000 7fe5cfbff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe5c8159890 0x7fe5c815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.327+0000 7fe5cfbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c8159c60 0x7fe5c815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.327+0000 7fe5cfbff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c815a160 con 0x7fe5c815db10 2023-10-08T01:11:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.327+0000 7fe5cfbff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c80d13a0 con 0x7fe5c8159890 2023-10-08T01:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.327+0000 7fe5cfbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c815e140 con 0x7fe5c8159c60 2023-10-08T01:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5ce19c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe5c815db10 0x7fe5c8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42546/0 (socket says 172.21.15.119:42546) 2023-10-08T01:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5ce19c700 1 -- 172.21.15.119:0/571993374 learned_addr learned my addr 172.21.15.119:0/571993374 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/571993374 >> v2:172.21.15.184:3300/0 conn(0x7fe5c8159890 0x7fe5c815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c8159c60 0x7fe5c815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1497283362 0 0) 0x7fe5c815a160 con 0x7fe5c815db10 2023-10-08T01:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b4003570 con 0x7fe5c815db10 2023-10-08T01:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2918939379 0 0) 0x7fe5b4003570 con 0x7fe5c815db10 2023-10-08T01:11:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c8159c60 msgr2=0x7fe5c815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c8159c60 0x7fe5c815d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 >> v2:172.21.15.184:3300/0 conn(0x7fe5c8159890 msgr2=0x7fe5c815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 --2- 172.21.15.119:0/571993374 >> v2:172.21.15.184:3300/0 conn(0x7fe5c8159890 0x7fe5c815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c815a3d0 con 0x7fe5c815db10 2023-10-08T01:11:33.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fe5c00017f0 con 0x7fe5c815db10 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/571993374 >> v2:172.21.15.184:3300/0 conn(0x7fe5c8159890 0x7fe5c815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 >> v1:172.21.15.119:6789/0 conn(0x7fe5c815db10 legacy=0x7fe5c8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.328+0000 7fe5cc999700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c8159c60 0x7fe5b4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cc999700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5b4003fe0 0x7fe5b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cc999700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4004d50 0x7fe5b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c80d13a0 con 0x7fe5b4003fe0 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c815e140 con 0x7fe5b4004d50 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b400ba90 con 0x7fe5c8159c60 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5b4003fe0 0x7fe5b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c8159c60 0x7fe5b4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5ce19c700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4004d50 0x7fe5b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cd19a700 1 -- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c8159c60 msgr2=0x7fe5b4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c8159c60 0x7fe5b4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cd19a700 1 -- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4004d50 msgr2=0x7fe5b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4004d50 0x7fe5b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cd19a700 1 -- 172.21.15.119:0/571993374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b4003570 con 0x7fe5b4003fe0 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.329+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5b4003fe0 0x7fe5b4004810 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe5c815a090 tx=0x7fe5b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb515ec2a90cd6d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800e6b0 con 0x7fe5b4003fe0 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b800e810 con 0x7fe5b4003fe0 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cc999700 1 -- 172.21.15.119:0/571993374 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b800ebd0 con 0x7fe5b4003fe0 2023-10-08T01:11:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cfbff700 1 -- 172.21.15.119:0/571993374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5b4003fe0 msgr2=0x7fe5b4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cfbff700 1 --2- 172.21.15.119:0/571993374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5b4003fe0 0x7fe5b4004810 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe5c815a090 tx=0x7fe5b80091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5ce19c700 1 -- 172.21.15.119:0/571993374 reap_dead start 2023-10-08T01:11:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cfbff700 1 -- 172.21.15.119:0/571993374 shutdown_connections 2023-10-08T01:11:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cfbff700 1 -- 172.21.15.119:0/571993374 >> 172.21.15.119:0/571993374 conn(0x7fe5c8068e20 msgr2=0x7fe5c80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cfbff700 1 -- 172.21.15.119:0/571993374 shutdown_connections 2023-10-08T01:11:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.330+0000 7fe5cfbff700 1 -- 172.21.15.119:0/571993374 wait complete. 2023-10-08T01:11:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.332+0000 7fe5cfbff700 1 Processor -- start 2023-10-08T01:11:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.332+0000 7fe5cfbff700 1 -- start start 2023-10-08T01:11:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.332+0000 7fe5cfbff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4003fe0 0x7fe5c80b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5cfbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c814d270 0x7fe5c80ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5cfbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c8151ea0 0x7fe5c8152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5cfbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c8152850 con 0x7fe5c8151ea0 2023-10-08T01:11:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5cfbff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c816d180 con 0x7fe5b4003fe0 2023-10-08T01:11:33.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5cfbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c816dfe0 con 0x7fe5c814d270 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c8151ea0 0x7fe5c8152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c8151ea0 0x7fe5c8152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52516/0 (socket says 172.21.15.119:52516) 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 -- 172.21.15.119:0/1834479846 learned_addr learned my addr 172.21.15.119:0/1834479846 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4003fe0 0x7fe5c80b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c814d270 0x7fe5c80ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 -- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c814d270 msgr2=0x7fe5c80ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c814d270 0x7fe5c80ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 -- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4003fe0 msgr2=0x7fe5c80b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4003fe0 0x7fe5c80b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 -- 172.21.15.119:0/1834479846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c816d3d0 con 0x7fe5c8151ea0 2023-10-08T01:11:33.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c814d270 0x7fe5c80ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.333+0000 7fe5ce19c700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c8151ea0 0x7fe5c8152310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00013d0 tx=0x7fe5c0001910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.334+0000 7fe5bdffb700 1 -- 172.21.15.119:0/1834479846 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0017070 con 0x7fe5c8151ea0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.334+0000 7fe5bdffb700 1 -- 172.21.15.119:0/1834479846 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c000ebc0 con 0x7fe5c8151ea0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.334+0000 7fe5bdffb700 1 -- 172.21.15.119:0/1834479846 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0013420 con 0x7fe5c8151ea0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.334+0000 7fe5cfbff700 1 -- 172.21.15.119:0/1834479846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c816d6b0 con 0x7fe5c8151ea0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.334+0000 7fe5cfbff700 1 -- 172.21.15.119:0/1834479846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c816dc70 con 0x7fe5c8151ea0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.336+0000 7fe5bdffb700 1 -- 172.21.15.119:0/1834479846 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5c0013790 con 0x7fe5c8151ea0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.337+0000 7fe5bdffb700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe5ac066f30 0x7fe5ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.337+0000 7fe5bdffb700 1 -- 172.21.15.119:0/1834479846 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe5c0084680 con 0x7fe5c8151ea0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.338+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe5ac066f30 0x7fe5ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.338+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe5ac066f30 0x7fe5ac0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006f00 tx=0x7fe5c40091a0 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.339+0000 7fe5de37bb80 1 Processor -- start 2023-10-08T01:11:33.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.339+0000 7fe5de37bb80 1 -- start start 2023-10-08T01:11:33.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 0x562f5ed8be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edd5d00 0x562f5edb4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5ed87cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5ece2a70 con 0x562f5edd5d00 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5ece0270 con 0x562f5ed8ba50 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f5ecdc060 con 0x562f5edb4f60 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5cd19a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edd5d00 0x562f5edb4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5cd19a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edd5d00 0x562f5edb4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52522/0 (socket says 172.21.15.119:52522) 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5cd19a700 1 -- 172.21.15.119:0/3803857649 learned_addr learned my addr 172.21.15.119:0/3803857649 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.340+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 0x562f5ed8be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5ce19c700 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5ed87cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5cd19a700 1 -- 172.21.15.119:0/3803857649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 msgr2=0x562f5ed87cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5ed87cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5cd19a700 1 -- 172.21.15.119:0/3803857649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 msgr2=0x562f5ed8be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 0x562f5ed8be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5cd19a700 1 -- 172.21.15.119:0/3803857649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5eb76270 con 0x562f5edd5d00 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edd5d00 0x562f5edb4a20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800e4b0 tx=0x7fe5b801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebed103f01dab359 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5aaffd700 1 -- 172.21.15.119:0/3803857649 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8006c90 con 0x562f5edd5d00 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5aaffd700 1 -- 172.21.15.119:0/3803857649 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b8023040 con 0x562f5edd5d00 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.341+0000 7fe5aaffd700 1 -- 172.21.15.119:0/3803857649 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b80095b0 con 0x562f5edd5d00 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3803857649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edd5d00 msgr2=0x562f5edb4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edd5d00 0x562f5edb4a20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe5b800e4b0 tx=0x7fe5b801a000 comp rx=0 tx=0).stop 2023-10-08T01:11:33.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3803857649 shutdown_connections 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5ed87cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 0x562f5ed8be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3803857649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edd5d00 0x562f5edb4a20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3803857649 >> 172.21.15.119:0/3803857649 conn(0x562f5ed8d370 msgr2=0x562f5ed8d740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3803857649 shutdown_connections 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.342+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3803857649 wait complete. 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.343+0000 7fe5de37bb80 1 -- start start 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.343+0000 7fe5de37bb80 1 Processor -- start 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5edd28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5edcbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edd5d00 0x562f5edcc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5ec87890 con 0x562f5ed8ba50 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5eca2ea0 con 0x562f5edd5d00 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f5eb5ee30 con 0x562f5edb4f60 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5de37bb80 10 client.0 _ll_register_callbacks cb 0x562f5ed4f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5cd99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5edd28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5cd99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5edd28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52530/0 (socket says 172.21.15.119:52530) 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5cd99b700 1 -- 172.21.15.119:0/2458929051 learned_addr learned my addr 172.21.15.119:0/2458929051 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5ce19c700 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edd5d00 0x562f5edcc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.344+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5edcbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5cd99b700 1 -- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 msgr2=0x562f5edcbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5edcbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5cd99b700 1 -- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edd5d00 msgr2=0x562f5edcc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edd5d00 0x562f5edcc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5cd99b700 1 -- 172.21.15.119:0/2458929051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b4003570 con 0x562f5ed8ba50 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5edcbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5edd28d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4015f80 tx=0x7fe5c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe655a4de685dba1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5a3fff700 1 -- 172.21.15.119:0/2458929051 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4014e80 con 0x562f5ed8ba50 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5a3fff700 1 -- 172.21.15.119:0/2458929051 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe5c4022bc0 con 0x562f5ed8ba50 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2458929051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f5ed80b70 con 0x562f5ed8ba50 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5a3fff700 1 -- 172.21.15.119:0/2458929051 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5c402c480 con 0x562f5ed8ba50 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5de37bb80 10 client.6402 fetch_fsmap learned FSMap version 32 2023-10-08T01:11:33.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2458929051 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562f5eb18fe0 con 0x562f5ed8ba50 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.345+0000 7fe5a3fff700 1 -- 172.21.15.119:0/2458929051 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562f5ed80b70 con 0x562f5ed8ba50 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5a3fff700 1 -- 172.21.15.119:0/2458929051 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe5c402c5e0 con 0x562f5ed8ba50 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5de37bb80 10 client.6402 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5de37bb80 10 client.6402 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5de37bb80 20 client.6402 populate_metadata read hostname 'smithi119' 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] conn(0x562f5edcdfa0 0x562f5edce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5de37bb80 4 client.6402 mds_command: new command op to 6225 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2458929051 --> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f5ed5daa0 con 0x562f5edcdfa0 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] conn(0x562f5edcdfa0 0x562f5edce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] conn(0x562f5edcdfa0 0x562f5edce370 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=461ca7c66e246165 server_cookie=866b01489c98cde1 in_seq=0 out_seq=0 2023-10-08T01:11:33.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.346+0000 7fe5a3fff700 10 client.6402 ms_handle_connect on v2:172.21.15.119:6832/926130691 2023-10-08T01:11:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.347+0000 7fe5a3fff700 1 -- 172.21.15.119:0/2458929051 <== mds.0 v2:172.21.15.119:6832/926130691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562f5ed5daa0 con 0x562f5edcdfa0 2023-10-08T01:11:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.347+0000 7fe5a3fff700 10 client.6402 handle_command_reply: tid=0 2023-10-08T01:11:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.347+0000 7fe5de37bb80 1 client.6402 shutdown 2023-10-08T01:11:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.347+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2458929051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 msgr2=0x562f5edd28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.347+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5edd28d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4015f80 tx=0x7fe5c4019000 comp rx=0 tx=0).stop 2023-10-08T01:11:33.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.347+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2458929051 shutdown_connections 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.347+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] conn(0x562f5edcdfa0 0x562f5edce370 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.348+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edd5d00 0x562f5edcc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.348+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5edd28d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.348+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2458929051 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edb4f60 0x562f5edcbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.348+0000 7fe5a3fff700 0 client.6402 ms_handle_reset on v2:172.21.15.119:6832/926130691 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.348+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2458929051 >> 172.21.15.119:0/2458929051 conn(0x562f5ed8d370 msgr2=0x562f5edd1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.348+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2458929051 shutdown_connections 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.348+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2458929051 wait complete. 2023-10-08T01:11:33.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.348+0000 7fe5de37bb80 20 client.6402 trim_cache size 0 max 16384 2023-10-08T01:11:33.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.365+0000 7fe5de37bb80 1 Processor -- start 2023-10-08T01:11:33.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.365+0000 7fe5de37bb80 1 -- start start 2023-10-08T01:11:33.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 0x562f5edb3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edb4f60 0x562f5ed8e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 0x562f5ed8ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5edbbe20 con 0x562f5edb4f60 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5edcefe0 con 0x562f5ed8ba50 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f5ed64410 con 0x562f5edd5d00 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edb4f60 0x562f5ed8e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edb4f60 0x562f5ed8e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52534/0 (socket says 172.21.15.119:52534) 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 -- 172.21.15.119:0/2844066459 learned_addr learned my addr 172.21.15.119:0/2844066459 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 -- 172.21.15.119:0/2844066459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 msgr2=0x562f5ed8ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/2844066459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 0x562f5ed8ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 -- 172.21.15.119:0/2844066459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 msgr2=0x562f5edb3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/2844066459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 0x562f5edb3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/2844066459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 0x562f5edb3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 -- 172.21.15.119:0/2844066459 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5edcf170 con 0x562f5edb4f60 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.366+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/2844066459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edb4f60 0x562f5ed8e9c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80099c0 tx=0x7fe5b8037250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9950d7812fa7f8be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5ab7fe700 1 -- 172.21.15.119:0/2844066459 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8000bc0 con 0x562f5edb4f60 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5ab7fe700 1 -- 172.21.15.119:0/2844066459 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5b800eb90 con 0x562f5edb4f60 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5ab7fe700 1 -- 172.21.15.119:0/2844066459 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5b8000d20 con 0x562f5edb4f60 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2844066459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edb4f60 msgr2=0x562f5ed8e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2844066459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edb4f60 0x562f5ed8e9c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80099c0 tx=0x7fe5b8037250 comp rx=0 tx=0).stop 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2844066459 shutdown_connections 2023-10-08T01:11:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2844066459 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 0x562f5ed8ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2844066459 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5ed8ba50 0x562f5edb3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/2844066459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5edb4f60 0x562f5ed8e9c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2844066459 >> 172.21.15.119:0/2844066459 conn(0x562f5ed8d370 msgr2=0x562f5ed878b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2844066459 shutdown_connections 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.367+0000 7fe5de37bb80 1 -- 172.21.15.119:0/2844066459 wait complete. 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.368+0000 7fe5de37bb80 1 -- start start 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.368+0000 7fe5de37bb80 1 Processor -- start 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5ee3a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edb4f60 0x562f5edef620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5de37bb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 0x562f5edefb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5edf1760 con 0x562f5ed8ba50 2023-10-08T01:11:33.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5ee41440 con 0x562f5edb4f60 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5de37bb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x562f5edd1a20 con 0x562f5edd5d00 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5de37bb80 10 client.0 _ll_register_callbacks cb 0x562f5ed4f110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5cd99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5ee3a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5cd99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5ee3a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52548/0 (socket says 172.21.15.119:52548) 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5cd99b700 1 -- 172.21.15.119:0/3881673345 learned_addr learned my addr 172.21.15.119:0/3881673345 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5ce19c700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 0x562f5edefb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.369+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edb4f60 0x562f5edef620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5cd99b700 1 -- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 msgr2=0x562f5edefb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 0x562f5edefb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5cd99b700 1 -- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edb4f60 msgr2=0x562f5edef620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edb4f60 0x562f5edef620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5cd99b700 1 -- 172.21.15.119:0/3881673345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f5eb76270 con 0x562f5ed8ba50 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5ce19c700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 0x562f5edefb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edb4f60 0x562f5edef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5cd99b700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5ee3a510 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000c00 tx=0x7fe5c401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c2778cad65c8dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5a3fff700 1 -- 172.21.15.119:0/3881673345 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4014e80 con 0x562f5ed8ba50 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5a3fff700 1 -- 172.21.15.119:0/3881673345 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe5c402caf0 con 0x562f5ed8ba50 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3881673345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5c8169ca0 con 0x562f5ed8ba50 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5a3fff700 1 -- 172.21.15.119:0/3881673345 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5c4022c20 con 0x562f5ed8ba50 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5de37bb80 10 client.6408 fetch_fsmap learned FSMap version 32 2023-10-08T01:11:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3881673345 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x562f5ecaa780 con 0x562f5ed8ba50 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.370+0000 7fe5a3fff700 1 -- 172.21.15.119:0/3881673345 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c8169ca0 con 0x562f5ed8ba50 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5a3fff700 1 -- 172.21.15.119:0/3881673345 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 32) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fe5c402c480 con 0x562f5ed8ba50 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5de37bb80 10 client.6408 fetch_fsmap finished waiting for FSMap version 32 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5de37bb80 10 client.6408 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5de37bb80 20 client.6408 populate_metadata read hostname 'smithi119' 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] conn(0x562f5edf2190 0x562f5edf2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5de37bb80 4 client.6408 mds_command: new command op to 6225 tid=0[{"prefix": "status"}] 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3881673345 --> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fe5b8023040 con 0x562f5edf2190 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] conn(0x562f5edf2190 0x562f5edf2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5cd19a700 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] conn(0x562f5edf2190 0x562f5edf2560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e911090716a1275 server_cookie=e8784cde421f92c3 in_seq=0 out_seq=0 2023-10-08T01:11:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.371+0000 7fe5a3fff700 10 client.6408 ms_handle_connect on v2:172.21.15.119:6832/926130691 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5a3fff700 1 -- 172.21.15.119:0/3881673345 <== mds.0 v2:172.21.15.119:6832/926130691 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7fe5b8023040 con 0x562f5edf2190 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5a3fff700 10 client.6408 handle_command_reply: tid=0 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 client.6408 shutdown 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3881673345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 msgr2=0x562f5ee3a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5ee3a510 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000c00 tx=0x7fe5c401d000 comp rx=0 tx=0).stop 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3881673345 shutdown_connections 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x562f5edd5d00 0x562f5edefb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.119:6832/926130691,v1:172.21.15.119:6834/926130691] conn(0x562f5edf2190 0x562f5edf2560 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562f5ed8ba50 0x562f5ee3a510 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 --2- 172.21.15.119:0/3881673345 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562f5edb4f60 0x562f5edef620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5a3fff700 0 client.6408 ms_handle_reset on v2:172.21.15.119:6832/926130691 2023-10-08T01:11:33.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3881673345 >> 172.21.15.119:0/3881673345 conn(0x562f5ed8d370 msgr2=0x562f5ed87800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:33.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3881673345 shutdown_connections 2023-10-08T01:11:33.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 1 -- 172.21.15.119:0/3881673345 wait complete. 2023-10-08T01:11:33.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.372+0000 7fe5de37bb80 20 client.6408 trim_cache size 0 max 16384 2023-10-08T01:11:33.384 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:11:33.384 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 6225, 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 11.638483256000001, 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 32, 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 77, 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 77, 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 17.430651257000001 2023-10-08T01:11:33.385 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 -- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe5ac066f30 msgr2=0x7fe5ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe5ac066f30 0x7fe5ac0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4006f00 tx=0x7fe5c40091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 -- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c8151ea0 msgr2=0x7fe5c8152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c8151ea0 0x7fe5c8152310 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00013d0 tx=0x7fe5c0001910 comp rx=0 tx=0).stop 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 -- 172.21.15.119:0/1834479846 shutdown_connections 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe5ac066f30 0x7fe5ac0693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c8151ea0 0x7fe5c8152310 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe5b4003fe0 0x7fe5c80b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 --2- 172.21.15.119:0/1834479846 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe5c814d270 0x7fe5c80ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 -- 172.21.15.119:0/1834479846 >> 172.21.15.119:0/1834479846 conn(0x7fe5c8068e20 msgr2=0x7fe5c8154660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 -- 172.21.15.119:0/1834479846 shutdown_connections 2023-10-08T01:11:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.373+0000 7fe5cfbff700 1 -- 172.21.15.119:0/1834479846 wait complete. 2023-10-08T01:11:33.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T01:11:33.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.570+0000 7f811579d700 1 Processor -- start 2023-10-08T01:11:33.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.571+0000 7f811579d700 1 -- start start 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.572+0000 7f811579d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8110150180 0x7f8110157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.572+0000 7f811579d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110150550 0x7f81100c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.572+0000 7f811579d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81100c7240 0x7f8110069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.572+0000 7f811579d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81100c76c0 con 0x7f81100c7240 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.572+0000 7f811579d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f811006e2a0 con 0x7f8110150180 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.572+0000 7f811579d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811006b2e0 con 0x7f8110150550 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.572+0000 7f8114f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81100c7240 0x7f8110069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8114f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81100c7240 0x7f8110069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52558/0 (socket says 172.21.15.119:52558) 2023-10-08T01:11:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8114f9c700 1 -- 172.21.15.119:0/934543577 learned_addr learned my addr 172.21.15.119:0/934543577 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f810ffff700 1 --2- 172.21.15.119:0/934543577 >> v2:172.21.15.184:3300/0 conn(0x7f8110150180 0x7f8110157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8107fff700 1 --2- 172.21.15.119:0/934543577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110150550 0x7f81100c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8114f9c700 1 -- 172.21.15.119:0/934543577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110150550 msgr2=0x7f81100c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8114f9c700 1 --2- 172.21.15.119:0/934543577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110150550 0x7f81100c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8114f9c700 1 -- 172.21.15.119:0/934543577 >> v2:172.21.15.184:3300/0 conn(0x7f8110150180 msgr2=0x7f8110157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8114f9c700 1 --2- 172.21.15.119:0/934543577 >> v2:172.21.15.184:3300/0 conn(0x7f8110150180 0x7f8110157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8114f9c700 1 -- 172.21.15.119:0/934543577 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81100c78e0 con 0x7f81100c7240 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f810ffff700 1 --2- 172.21.15.119:0/934543577 >> v2:172.21.15.184:3300/0 conn(0x7f8110150180 0x7f8110157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f8114f9c700 1 --2- 172.21.15.119:0/934543577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81100c7240 0x7f8110069f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8108009f20 tx=0x7f8108009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d1f21f7d96d61c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f810f7fe700 1 -- 172.21.15.119:0/934543577 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810800e070 con 0x7f81100c7240 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f810f7fe700 1 -- 172.21.15.119:0/934543577 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810800ba40 con 0x7f81100c7240 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.573+0000 7f810f7fe700 1 -- 172.21.15.119:0/934543577 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81080092e0 con 0x7f81100c7240 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 -- 172.21.15.119:0/934543577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81100c7240 msgr2=0x7f8110069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 --2- 172.21.15.119:0/934543577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81100c7240 0x7f8110069f50 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8108009f20 tx=0x7f8108009840 comp rx=0 tx=0).stop 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 -- 172.21.15.119:0/934543577 shutdown_connections 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 --2- 172.21.15.119:0/934543577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81100c7240 0x7f8110069f50 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 --2- 172.21.15.119:0/934543577 >> v2:172.21.15.184:3300/0 conn(0x7f8110150180 0x7f8110157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 --2- 172.21.15.119:0/934543577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8110150550 0x7f81100c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 -- 172.21.15.119:0/934543577 >> 172.21.15.119:0/934543577 conn(0x7f81100cc1f0 msgr2=0x7f81100cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 -- 172.21.15.119:0/934543577 shutdown_connections 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.574+0000 7f811579d700 1 -- 172.21.15.119:0/934543577 wait complete. 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.575+0000 7f811579d700 1 Processor -- start 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.576+0000 7f811579d700 1 -- start start 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.576+0000 7f811579d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100c7240 0x7f81100af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.576+0000 7f811579d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110150180 0x7f81100aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f811579d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110150550 0x7f8110148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f811579d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100ad180 con 0x7f8110150550 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f811579d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100ad460 con 0x7f8110150180 2023-10-08T01:11:33.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f811579d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81100ad740 con 0x7f81100c7240 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8114f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110150550 0x7f8110148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8114f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110150550 0x7f8110148b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52568/0 (socket says 172.21.15.119:52568) 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8114f9c700 1 -- 172.21.15.119:0/2299422228 learned_addr learned my addr 172.21.15.119:0/2299422228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8107fff700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110150180 0x7f81100aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f810ffff700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100c7240 0x7f81100af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8114f9c700 1 -- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100c7240 msgr2=0x7f81100af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8114f9c700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100c7240 0x7f81100af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8114f9c700 1 -- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110150180 msgr2=0x7f81100aca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8114f9c700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110150180 0x7f81100aca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f8114f9c700 1 -- 172.21.15.119:0/2299422228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8108009ae0 con 0x7f8110150550 2023-10-08T01:11:33.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.577+0000 7f810ffff700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100c7240 0x7f81100af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.578+0000 7f8114f9c700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110150550 0x7f8110148b30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f811006fe40 tx=0x7f8108018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.578+0000 7f810cff9700 1 -- 172.21.15.119:0/2299422228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81080032f0 con 0x7f8110150550 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.578+0000 7f810cff9700 1 -- 172.21.15.119:0/2299422228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8108008d90 con 0x7f8110150550 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.578+0000 7f810cff9700 1 -- 172.21.15.119:0/2299422228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108026690 con 0x7f8110150550 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.578+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81100ada20 con 0x7f8110150550 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.578+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8110149070 con 0x7f8110150550 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.579+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81100adf40 con 0x7f8110150550 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.580+0000 7f810cff9700 1 -- 172.21.15.119:0/2299422228 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8108003040 con 0x7f8110150550 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.581+0000 7f810cff9700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f80f0066f30 0x7f80f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.581+0000 7f810cff9700 1 -- 172.21.15.119:0/2299422228 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f8108086310 con 0x7f8110150550 2023-10-08T01:11:33.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.581+0000 7f810ffff700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f80f0066f30 0x7f80f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:33.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.582+0000 7f810ffff700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f80f0066f30 0x7f80f00693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8100006f00 tx=0x7f8100008040 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:33.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.585+0000 7f810cff9700 1 -- 172.21.15.119:0/2299422228 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81100adf40 con 0x7f8110150550 2023-10-08T01:11:33.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.821+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f81100adf40 con 0x7f8110150550 2023-10-08T01:11:33.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.821+0000 7f810cff9700 1 -- 172.21.15.119:0/2299422228 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+73 (secure 0 0 0) 0x7f81100adf40 con 0x7f8110150550 2023-10-08T01:11:33.821 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T01:11:33.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f80f0066f30 msgr2=0x7f80f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f80f0066f30 0x7f80f00693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8100006f00 tx=0x7f8100008040 comp rx=0 tx=0).stop 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110150550 msgr2=0x7f8110148b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110150550 0x7f8110148b30 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f811006fe40 tx=0x7f8108018000 comp rx=0 tx=0).stop 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 shutdown_connections 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f80f0066f30 0x7f80f00693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81100c7240 0x7f81100af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8110150550 0x7f8110148b30 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 --2- 172.21.15.119:0/2299422228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8110150180 0x7f81100aca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 >> 172.21.15.119:0/2299422228 conn(0x7f81100cc1f0 msgr2=0x7f81101588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:33.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.823+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 shutdown_connections 2023-10-08T01:11:33.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:33.824+0000 7f811579d700 1 -- 172.21.15.119:0/2299422228 wait complete. 2023-10-08T01:11:33.841 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T01:11:33.841 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T01:11:33.842 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T01:11:33.842 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T01:11:33.842 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T01:11:33.842 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T01:11:33.842 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T01:11:33.842 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T01:11:33.843 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:11:33.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:33.843 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T01:11:33.881 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T01:11:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T01:11:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T01:11:33.881 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:11:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T01:11:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:11:33.882 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:11:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:11:33.882 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T01:11:33.882 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:11:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:11:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T01:11:33.883 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:11:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T01:11:33.883 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T01:11:33.883 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T01:11:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T01:11:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41358sec preferred_lft 41358sec 2023-10-08T01:11:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T01:11:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:11:33.884 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T01:11:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T01:11:33.884 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T01:11:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:11:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T01:11:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:11:33.885 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:11:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T01:11:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T01:11:33.885 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:11:33.885 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:11:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T01:11:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T01:11:33.886 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:11:33.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:33.886 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T01:11:33.979 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T01:11:33.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:33.982 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T01:11:34.021 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T01:11:34.022 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T01:11:34.022 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T01:11:34.022 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T01:11:34.022 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T01:11:34.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:34.023 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T01:11:34.023 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T01:11:34.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T01:11:34.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T01:11:34.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T01:11:34.024 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T01:11:34.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:34.216 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T01:11:34.253 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T01:11:34.253 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T01:11:34.254 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:11:34.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:34.254 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T01:11:34.355 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:11:34.355 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T01:11:34.391 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:11:34.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:11:34.391 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:11:34.467 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T01:11:34.467 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T01:11:34.467 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:11:34.598 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:11:34.599 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:11:34.628 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:11:34.628 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:11:34.628 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T01:11:34.628 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T01:11:34.628 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T01:11:34.632 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T01:11:34.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:34.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:11:34.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T01:11:34.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.875+0000 7f3070d9d700 1 Processor -- start 2023-10-08T01:11:34.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.875+0000 7f3070d9d700 1 -- start start 2023-10-08T01:11:34.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.876+0000 7f3070d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f306c140cd0 0x7f306c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:34.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.876+0000 7f3070d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c1410a0 0x7f306c06bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:34.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.876+0000 7f3070d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306c0b1500 con 0x7f306c0ccd10 2023-10-08T01:11:34.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.877+0000 7f3070d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306c0cda20 con 0x7f306c140cd0 2023-10-08T01:11:34.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.877+0000 7f3070d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c0cdd00 con 0x7f306c1410a0 2023-10-08T01:11:34.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.877+0000 7f306b7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f306c140cd0 0x7f306c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:34.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.877+0000 7f306affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c1410a0 0x7f306c06bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:34.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.877+0000 7f306b7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f306c140cd0 0x7f306c06b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:42922/0 (socket says 172.21.15.119:42922) 2023-10-08T01:11:34.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.877+0000 7f306b7fe700 1 -- 172.21.15.119:0/2258855860 learned_addr learned my addr 172.21.15.119:0/2258855860 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:34.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306affd700 1 -- 172.21.15.119:0/2258855860 >> v2:172.21.15.184:3300/0 conn(0x7f306c140cd0 msgr2=0x7f306c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:34.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306affd700 1 --2- 172.21.15.119:0/2258855860 >> v2:172.21.15.184:3300/0 conn(0x7f306c140cd0 0x7f306c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:34.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306affd700 1 -- 172.21.15.119:0/2258855860 >> v1:172.21.15.184:6789/0 conn(0x7f306c0ccd10 legacy=0x7f306c0cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:34.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306affd700 1 -- 172.21.15.119:0/2258855860 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c0cdfe0 con 0x7f306c1410a0 2023-10-08T01:11:34.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306b7fe700 1 --2- 172.21.15.119:0/2258855860 >> v2:172.21.15.184:3300/0 conn(0x7f306c140cd0 0x7f306c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:34.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306affd700 1 --2- 172.21.15.119:0/2258855860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c1410a0 0x7f306c06bf20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f30600098e0 tx=0x7f30600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f3fd024eff5dddf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:34.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306a7fc700 1 -- 172.21.15.119:0/2258855860 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000b640 con 0x7f306c1410a0 2023-10-08T01:11:34.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306a7fc700 1 -- 172.21.15.119:0/2258855860 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306000b7a0 con 0x7f306c1410a0 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.878+0000 7f306a7fc700 1 -- 172.21.15.119:0/2258855860 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306000bb40 con 0x7f306c1410a0 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.879+0000 7f3070d9d700 1 -- 172.21.15.119:0/2258855860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c1410a0 msgr2=0x7f306c06bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.879+0000 7f3070d9d700 1 --2- 172.21.15.119:0/2258855860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c1410a0 0x7f306c06bf20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f30600098e0 tx=0x7f30600091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.879+0000 7f3070d9d700 1 -- 172.21.15.119:0/2258855860 shutdown_connections 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.879+0000 7f3070d9d700 1 --2- 172.21.15.119:0/2258855860 >> v2:172.21.15.184:3300/0 conn(0x7f306c140cd0 0x7f306c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.879+0000 7f3070d9d700 1 --2- 172.21.15.119:0/2258855860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c1410a0 0x7f306c06bf20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.879+0000 7f3070d9d700 1 -- 172.21.15.119:0/2258855860 >> 172.21.15.119:0/2258855860 conn(0x7f306c0ce430 msgr2=0x7f306c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.879+0000 7f3070d9d700 1 -- 172.21.15.119:0/2258855860 shutdown_connections 2023-10-08T01:11:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.879+0000 7f3070d9d700 1 -- 172.21.15.119:0/2258855860 wait complete. 2023-10-08T01:11:34.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.880+0000 7f3070d9d700 1 Processor -- start 2023-10-08T01:11:34.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.880+0000 7f3070d9d700 1 -- start start 2023-10-08T01:11:34.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f3070d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f306c0ccd10 0x7f306c0afdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f3070d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c140cd0 0x7f306c139180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f3070d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c1410a0 0x7f306c155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f3070d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c13a550 con 0x7f306c0ccd10 2023-10-08T01:11:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f3070d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306c1396c0 con 0x7f306c1410a0 2023-10-08T01:11:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f3070d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306c139940 con 0x7f306c140cd0 2023-10-08T01:11:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f306b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f306c0ccd10 0x7f306c0afdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f306b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f306c0ccd10 0x7f306c0afdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36242/0 (socket says 172.21.15.119:36242) 2023-10-08T01:11:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.881+0000 7f306b7fe700 1 -- 172.21.15.119:0/824741251 learned_addr learned my addr 172.21.15.119:0/824741251 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f306b7fe700 1 -- 172.21.15.119:0/824741251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c140cd0 msgr2=0x7f306c139180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f306b7fe700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c140cd0 0x7f306c139180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f306bfff700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c1410a0 0x7f306c155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f306b7fe700 1 -- 172.21.15.119:0/824741251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c1410a0 msgr2=0x7f306c155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f306b7fe700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c1410a0 0x7f306c155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f306b7fe700 1 -- 172.21.15.119:0/824741251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c0094f0 con 0x7f306c0ccd10 2023-10-08T01:11:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f306bfff700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c1410a0 0x7f306c155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f306b7fe700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f306c0ccd10 0x7f306c0afdd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f305c00ec80 tx=0x7f305c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f30537fe700 1 -- 172.21.15.119:0/824741251 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c00c9d0 con 0x7f306c0ccd10 2023-10-08T01:11:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f30537fe700 1 -- 172.21.15.119:0/824741251 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305c00cb30 con 0x7f306c0ccd10 2023-10-08T01:11:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3060009420 con 0x7f306c0ccd10 2023-10-08T01:11:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f30537fe700 1 -- 172.21.15.119:0/824741251 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305c00ceb0 con 0x7f306c0ccd10 2023-10-08T01:11:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.882+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c139f00 con 0x7f306c0ccd10 2023-10-08T01:11:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.883+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f306c13a830 con 0x7f306c0ccd10 2023-10-08T01:11:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.884+0000 7f30537fe700 1 -- 172.21.15.119:0/824741251 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f305c013680 con 0x7f306c0ccd10 2023-10-08T01:11:34.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.885+0000 7f30537fe700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3058066f30 0x7f30580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:34.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.885+0000 7f30537fe700 1 -- 172.21.15.119:0/824741251 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f305c084eb0 con 0x7f306c0ccd10 2023-10-08T01:11:34.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.885+0000 7f306affd700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3058066f30 0x7f30580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:34.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.886+0000 7f306affd700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3058066f30 0x7f30580693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f306c16b120 tx=0x7f3060017000 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:34.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:34.889+0000 7f30537fe700 1 -- 172.21.15.119:0/824741251 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f305c051ef0 con 0x7f306c0ccd10 2023-10-08T01:11:35.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.106+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f306c13a690 con 0x7f3058066f30 2023-10-08T01:11:35.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.108+0000 7f30537fe700 1 -- 172.21.15.119:0/824741251 <== mgr.6036 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3060000c50 con 0x7f3058066f30 2023-10-08T01:11:35.108 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:11:35.108 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:35.108 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T01:11:35.108 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:35.109 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:11:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3058066f30 msgr2=0x7f30580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3058066f30 0x7f30580693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f306c16b120 tx=0x7f3060017000 comp rx=0 tx=0).stop 2023-10-08T01:11:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f306c0ccd10 msgr2=0x7f306c0afdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f306c0ccd10 0x7f306c0afdd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f305c00ec80 tx=0x7f305c009250 comp rx=0 tx=0).stop 2023-10-08T01:11:35.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 shutdown_connections 2023-10-08T01:11:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3058066f30 0x7f30580693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f306c1410a0 0x7f306c155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f306c0ccd10 0x7f306c0afdd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 --2- 172.21.15.119:0/824741251 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f306c140cd0 0x7f306c139180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 >> 172.21.15.119:0/824741251 conn(0x7f306c0ce430 msgr2=0x7f306c157240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 shutdown_connections 2023-10-08T01:11:35.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.111+0000 7f3070d9d700 1 -- 172.21.15.119:0/824741251 wait complete. 2023-10-08T01:11:35.128 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T01:11:35.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.311+0000 7f432c3ff700 1 Processor -- start 2023-10-08T01:11:35.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.311+0000 7f432c3ff700 1 -- start start 2023-10-08T01:11:35.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.312+0000 7f432c3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4324069770 0x7f4324069bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:35.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432c3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432406e950 con 0x7f43240c70b0 2023-10-08T01:11:35.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432c3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4324156320 con 0x7f4324155eb0 2023-10-08T01:11:35.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432c3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43240cd350 con 0x7f4324069770 2023-10-08T01:11:35.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432a99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4324069770 0x7f4324069bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:35.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432a99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4324069770 0x7f4324069bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36246/0 (socket says 172.21.15.119:36246) 2023-10-08T01:11:35.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432a19b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f43240c70b0 0x7f43240ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47172/0 (socket says 172.21.15.119:47172) 2023-10-08T01:11:35.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432a99c700 1 -- 172.21.15.119:0/584390597 learned_addr learned my addr 172.21.15.119:0/584390597 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432a99c700 1 -- 172.21.15.119:0/584390597 >> v1:172.21.15.184:6789/0 conn(0x7f4324155eb0 legacy=0x7f432406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432a99c700 1 -- 172.21.15.119:0/584390597 >> v1:172.21.15.119:6789/0 conn(0x7f43240c70b0 legacy=0x7f43240ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432a99c700 1 -- 172.21.15.119:0/584390597 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432406eb90 con 0x7f4324069770 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.313+0000 7f432a99c700 1 --2- 172.21.15.119:0/584390597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324069770 0x7f4324069bf0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f431c00d1a0 tx=0x7f431c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=facd1538f12820df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f4329199700 1 -- 172.21.15.119:0/584390597 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c010070 con 0x7f4324069770 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f4329199700 1 -- 172.21.15.119:0/584390597 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c00b0d0 con 0x7f4324069770 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f4329199700 1 -- 172.21.15.119:0/584390597 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c00b490 con 0x7f4324069770 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f432c3ff700 1 -- 172.21.15.119:0/584390597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324069770 msgr2=0x7f4324069bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:35.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f432c3ff700 1 --2- 172.21.15.119:0/584390597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324069770 0x7f4324069bf0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f431c00d1a0 tx=0x7f431c00ba50 comp rx=0 tx=0).stop 2023-10-08T01:11:35.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f432c3ff700 1 -- 172.21.15.119:0/584390597 shutdown_connections 2023-10-08T01:11:35.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f432c3ff700 1 --2- 172.21.15.119:0/584390597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324069770 0x7f4324069bf0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:35.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f432c3ff700 1 -- 172.21.15.119:0/584390597 >> 172.21.15.119:0/584390597 conn(0x7f4324068e10 msgr2=0x7f43240cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:35.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f432c3ff700 1 -- 172.21.15.119:0/584390597 shutdown_connections 2023-10-08T01:11:35.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.314+0000 7f432c3ff700 1 -- 172.21.15.119:0/584390597 wait complete. 2023-10-08T01:11:35.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.316+0000 7f432c3ff700 1 Processor -- start 2023-10-08T01:11:35.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.316+0000 7f432c3ff700 1 -- start start 2023-10-08T01:11:35.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432c3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4324069770 0x7f432414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:35.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432c3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43240c70b0 0x7f43240aa8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432c3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324155eb0 0x7f43240aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432c3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240c5dd0 con 0x7f4324155eb0 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432c3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43240c35e0 con 0x7f43240c70b0 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432c3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43240c4120 con 0x7f4324069770 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432a99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324155eb0 0x7f43240aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432a99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324155eb0 0x7f43240aae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36250/0 (socket says 172.21.15.119:36250) 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432a99c700 1 -- 172.21.15.119:0/1670483721 learned_addr learned my addr 172.21.15.119:0/1670483721 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432a19b700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4324069770 0x7f432414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.317+0000 7f432999a700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43240c70b0 0x7f43240aa8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432a99c700 1 -- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4324069770 msgr2=0x7f432414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432a99c700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4324069770 0x7f432414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:35.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432a99c700 1 -- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43240c70b0 msgr2=0x7f43240aa8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432a99c700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43240c70b0 0x7f43240aa8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432a99c700 1 -- 172.21.15.119:0/1670483721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c00bcf0 con 0x7f4324155eb0 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432a19b700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4324069770 0x7f432414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432a99c700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324155eb0 0x7f43240aae30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f431c00ba90 tx=0x7f431c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f431a7fc700 1 -- 172.21.15.119:0/1670483721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c00ade0 con 0x7f4324155eb0 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f431a7fc700 1 -- 172.21.15.119:0/1670483721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431c01e440 con 0x7f4324155eb0 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f431a7fc700 1 -- 172.21.15.119:0/1670483721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431c01e800 con 0x7f4324155eb0 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43240c37f0 con 0x7f4324155eb0 2023-10-08T01:11:35.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.318+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43240c3db0 con 0x7f4324155eb0 2023-10-08T01:11:35.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.319+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43240c42b0 con 0x7f4324155eb0 2023-10-08T01:11:35.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.320+0000 7f431a7fc700 1 -- 172.21.15.119:0/1670483721 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 32) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f431c003040 con 0x7f4324155eb0 2023-10-08T01:11:35.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.321+0000 7f431a7fc700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4308066f30 0x7f43080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:35.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.321+0000 7f431a7fc700 1 -- 172.21.15.119:0/1670483721 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f431c0866a0 con 0x7f4324155eb0 2023-10-08T01:11:35.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.321+0000 7f432a19b700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4308066f30 0x7f43080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:35.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.322+0000 7f432a19b700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4308066f30 0x7f43080693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4320007fd0 tx=0x7f4320007e30 comp rx=0 tx=0).ready entity=mgr.6036 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:35.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.325+0000 7f431a7fc700 1 -- 172.21.15.119:0/1670483721 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f431c0536e0 con 0x7f4324155eb0 2023-10-08T01:11:35.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:35.569+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f432404c3c0 con 0x7f4324155eb0 2023-10-08T01:11:36.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.217+0000 7f431a7fc700 1 -- 172.21.15.119:0/1670483721 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 33) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f431c053ab0 con 0x7f4324155eb0 2023-10-08T01:11:36.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.218+0000 7f431a7fc700 1 -- 172.21.15.119:0/1670483721 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v33) v1 ==== 92+0+0 (secure 0 0 0) 0x7f431c057170 con 0x7f4324155eb0 2023-10-08T01:11:36.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.220+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4308066f30 msgr2=0x7f43080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.220+0000 7f432c3ff700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4308066f30 0x7f43080693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4320007fd0 tx=0x7f4320007e30 comp rx=0 tx=0).stop 2023-10-08T01:11:36.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.220+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324155eb0 msgr2=0x7f43240aae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.220+0000 7f432c3ff700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324155eb0 0x7f43240aae30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f431c00ba90 tx=0x7f431c01a000 comp rx=0 tx=0).stop 2023-10-08T01:11:36.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.221+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 shutdown_connections 2023-10-08T01:11:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.221+0000 7f432c3ff700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4308066f30 0x7f43080693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.221+0000 7f432c3ff700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4324069770 0x7f432414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.221+0000 7f432c3ff700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4324155eb0 0x7f43240aae30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.221+0000 7f432c3ff700 1 --2- 172.21.15.119:0/1670483721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43240c70b0 0x7f43240aa8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.221+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 >> 172.21.15.119:0/1670483721 conn(0x7f4324068e10 msgr2=0x7f43240ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.221+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 shutdown_connections 2023-10-08T01:11:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.221+0000 7f432c3ff700 1 -- 172.21.15.119:0/1670483721 wait complete. 2023-10-08T01:11:36.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T01:11:36.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.432+0000 7f42d159d700 1 Processor -- start 2023-10-08T01:11:36.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.432+0000 7f42d159d700 1 -- start start 2023-10-08T01:11:36.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42d159d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42d159d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f42cc0b16d0 0x7f42cc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42d159d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f42cc15d240 0x7f42cc15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42d159d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42cc0b2080 con 0x7f42cc15d240 2023-10-08T01:11:36.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42d159d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42cc0cfb00 con 0x7f42cc0b16d0 2023-10-08T01:11:36.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42d159d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc0b5c40 con 0x7f42cc0b4e70 2023-10-08T01:11:36.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42d0d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f42cc15d240 0x7f42cc15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42cb7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f42cc0b16d0 0x7f42cc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42cb7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f42cc0b16d0 0x7f42cc0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:42980/0 (socket says 172.21.15.119:42980) 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:37298/0 (socket says 172.21.15.119:37298) 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.433+0000 7f42cb7fe700 1 -- 172.21.15.119:0/1060343882 learned_addr learned my addr 172.21.15.119:0/1060343882 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.434+0000 7f42cb7fe700 1 -- 172.21.15.119:0/1060343882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 msgr2=0x7f42cc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.434+0000 7f42cb7fe700 1 --2- 172.21.15.119:0/1060343882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.434+0000 7f42cb7fe700 1 -- 172.21.15.119:0/1060343882 >> v2:172.21.15.119:3300/0 conn(0x7f42cc15d240 msgr2=0x7f42cc15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.434+0000 7f42cb7fe700 1 --2- 172.21.15.119:0/1060343882 >> v2:172.21.15.119:3300/0 conn(0x7f42cc15d240 0x7f42cc15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.434+0000 7f42cb7fe700 1 -- 172.21.15.119:0/1060343882 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42cc0b5f20 con 0x7f42cc0b16d0 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.434+0000 7f42d0d9c700 1 --2- 172.21.15.119:0/1060343882 >> v2:172.21.15.119:3300/0 conn(0x7f42cc15d240 0x7f42cc15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.434+0000 7f42cb7fe700 1 --2- 172.21.15.119:0/1060343882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 0x7f42cc0b1b40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42cc089440 tx=0x7f42c0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e346c8e36b45f5e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.434+0000 7f42caffd700 1 -- 172.21.15.119:0/1060343882 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000b640 con 0x7f42cc0b16d0 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42caffd700 1 -- 172.21.15.119:0/1060343882 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c000b7a0 con 0x7f42cc0b16d0 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42caffd700 1 -- 172.21.15.119:0/1060343882 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c000bb60 con 0x7f42cc0b16d0 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 -- 172.21.15.119:0/1060343882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 msgr2=0x7f42cc0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 --2- 172.21.15.119:0/1060343882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 0x7f42cc0b1b40 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f42cc089440 tx=0x7f42c0009250 comp rx=0 tx=0).stop 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 -- 172.21.15.119:0/1060343882 shutdown_connections 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 --2- 172.21.15.119:0/1060343882 >> v2:172.21.15.119:3300/0 conn(0x7f42cc15d240 0x7f42cc15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 --2- 172.21.15.119:0/1060343882 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 --2- 172.21.15.119:0/1060343882 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 0x7f42cc0b1b40 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 -- 172.21.15.119:0/1060343882 >> 172.21.15.119:0/1060343882 conn(0x7f42cc0d10c0 msgr2=0x7f42cc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 -- 172.21.15.119:0/1060343882 shutdown_connections 2023-10-08T01:11:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.435+0000 7f42d159d700 1 -- 172.21.15.119:0/1060343882 wait complete. 2023-10-08T01:11:36.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.437+0000 7f42d159d700 1 Processor -- start 2023-10-08T01:11:36.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.437+0000 7f42d159d700 1 -- start start 2023-10-08T01:11:36.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d159d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 0x7f42cc1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d159d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d159d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42cc15d240 0x7f42cc068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d159d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc145f20 con 0x7f42cc15d240 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d159d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc147800 con 0x7f42cc0b16d0 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d159d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42cc146bf0 con 0x7f42cc0b4e70 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42cbfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 0x7f42cc1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d0d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42cc15d240 0x7f42cc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d0d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42cc15d240 0x7f42cc068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36284/0 (socket says 172.21.15.119:36284) 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d0d9c700 1 -- 172.21.15.119:0/2758407857 learned_addr learned my addr 172.21.15.119:0/2758407857 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42cb7fe700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d0d9c700 1 -- 172.21.15.119:0/2758407857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 msgr2=0x7f42cc146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d0d9c700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d0d9c700 1 -- 172.21.15.119:0/2758407857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 msgr2=0x7f42cc1459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d0d9c700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 0x7f42cc1459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42d0d9c700 1 -- 172.21.15.119:0/2758407857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c4009a20 con 0x7f42cc15d240 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.438+0000 7f42cbfff700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 0x7f42cc1459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.439+0000 7f42d0d9c700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42cc15d240 0x7f42cc068390 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009f20 tx=0x7f42c400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.439+0000 7f42b3fff700 1 -- 172.21.15.119:0/2758407857 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c400cb70 con 0x7f42cc15d240 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.439+0000 7f42b3fff700 1 -- 172.21.15.119:0/2758407857 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c400f040 con 0x7f42cc15d240 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.439+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c00094f0 con 0x7f42cc15d240 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.439+0000 7f42b3fff700 1 -- 172.21.15.119:0/2758407857 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4014480 con 0x7f42cc15d240 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.439+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42cc1471b0 con 0x7f42cc15d240 2023-10-08T01:11:36.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.440+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42cc088990 con 0x7f42cc15d240 2023-10-08T01:11:36.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.441+0000 7f42b3fff700 1 -- 172.21.15.119:0/2758407857 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f42c4003020 con 0x7f42cc15d240 2023-10-08T01:11:36.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.442+0000 7f42b3fff700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f42b8066f80 0x7f42b8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.442+0000 7f42b3fff700 1 -- 172.21.15.119:0/2758407857 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f42c4085130 con 0x7f42cc15d240 2023-10-08T01:11:36.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.442+0000 7f42cbfff700 1 -- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f42b8066f80 msgr2=0x7f42b8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:36.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.442+0000 7f42cbfff700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f42b8066f80 0x7f42b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:36.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.450+0000 7f42b3fff700 1 -- 172.21.15.119:0/2758407857 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f42c4052170 con 0x7f42cc15d240 2023-10-08T01:11:36.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.643+0000 7f42cbfff700 1 -- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f42b8066f80 msgr2=0x7f42b8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:36.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.643+0000 7f42cbfff700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f42b8066f80 0x7f42b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.653+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f42cc04bb80 con 0x7f42cc15d240 2023-10-08T01:11:36.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.657+0000 7f42b3fff700 1 -- 172.21.15.119:0/2758407857 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f42c4055790 con 0x7f42cc15d240 2023-10-08T01:11:36.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f42b8066f80 msgr2=0x7f42b8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f42b8066f80 0x7f42b8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42cc15d240 msgr2=0x7f42cc068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42cc15d240 0x7f42cc068390 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009f20 tx=0x7f42c400b040 comp rx=0 tx=0).stop 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 shutdown_connections 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f42b8066f80 0x7f42b8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42cc15d240 0x7f42cc068390 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42cc0b16d0 0x7f42cc1459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 --2- 172.21.15.119:0/2758407857 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42cc0b4e70 0x7f42cc146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.659+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 >> 172.21.15.119:0/2758407857 conn(0x7f42cc0d10c0 msgr2=0x7f42cc069a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.660+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 shutdown_connections 2023-10-08T01:11:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.660+0000 7f42d159d700 1 -- 172.21.15.119:0/2758407857 wait complete. 2023-10-08T01:11:36.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T01:11:36.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.867+0000 7f426579d700 1 Processor -- start 2023-10-08T01:11:36.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.868+0000 7f426579d700 1 -- start start 2023-10-08T01:11:36.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f426579d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f42600b16d0 0x7f42600b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f426579d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f42600b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f426579d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42600b2080 con 0x7f42600b4e70 2023-10-08T01:11:36.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f426579d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42600cfb00 con 0x7f42600b16d0 2023-10-08T01:11:36.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f426579d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600b5c40 con 0x7f42600b33b0 2023-10-08T01:11:36.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f425ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f42600b4e70 0x7f42600ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47188/0 (socket says 172.21.15.119:47188) 2023-10-08T01:11:36.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f425ffff700 1 -- 172.21.15.119:0/2424261451 learned_addr learned my addr 172.21.15.119:0/2424261451 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:36.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f425f7fe700 1 --2- 172.21.15.119:0/2424261451 >> v2:172.21.15.184:3300/0 conn(0x7f42600b16d0 0x7f42600b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.869+0000 7f4264f9c700 1 --2- 172.21.15.119:0/2424261451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f42600b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f425effd700 1 -- 172.21.15.119:0/2424261451 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325711726 0 0) 0x7f42600b2080 con 0x7f42600b4e70 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f425effd700 1 -- 172.21.15.119:0/2424261451 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4248003570 con 0x7f42600b4e70 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f4264f9c700 1 -- 172.21.15.119:0/2424261451 >> v2:172.21.15.184:3300/0 conn(0x7f42600b16d0 msgr2=0x7f42600b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f4264f9c700 1 --2- 172.21.15.119:0/2424261451 >> v2:172.21.15.184:3300/0 conn(0x7f42600b16d0 0x7f42600b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f4264f9c700 1 -- 172.21.15.119:0/2424261451 >> v1:172.21.15.119:6789/0 conn(0x7f42600b4e70 legacy=0x7f42600ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f4264f9c700 1 -- 172.21.15.119:0/2424261451 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600b5f20 con 0x7f42600b33b0 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f4264f9c700 1 --2- 172.21.15.119:0/2424261451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f42600b3810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4258009e60 tx=0x7f4258009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5308e9c7903605f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f425effd700 1 -- 172.21.15.119:0/2424261451 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800e070 con 0x7f42600b33b0 2023-10-08T01:11:36.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f425effd700 1 -- 172.21.15.119:0/2424261451 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425800bab0 con 0x7f42600b33b0 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.870+0000 7f425effd700 1 -- 172.21.15.119:0/2424261451 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800be30 con 0x7f42600b33b0 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.871+0000 7f426579d700 1 -- 172.21.15.119:0/2424261451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 msgr2=0x7f42600b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.871+0000 7f426579d700 1 --2- 172.21.15.119:0/2424261451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f42600b3810 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4258009e60 tx=0x7f4258009780 comp rx=0 tx=0).stop 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.871+0000 7f426579d700 1 -- 172.21.15.119:0/2424261451 shutdown_connections 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.871+0000 7f426579d700 1 --2- 172.21.15.119:0/2424261451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f42600b3810 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.871+0000 7f426579d700 1 --2- 172.21.15.119:0/2424261451 >> v2:172.21.15.184:3300/0 conn(0x7f42600b16d0 0x7f42600b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.871+0000 7f426579d700 1 -- 172.21.15.119:0/2424261451 >> 172.21.15.119:0/2424261451 conn(0x7f42600d10c0 msgr2=0x7f42600affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.871+0000 7f426579d700 1 -- 172.21.15.119:0/2424261451 shutdown_connections 2023-10-08T01:11:36.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.871+0000 7f426579d700 1 -- 172.21.15.119:0/2424261451 wait complete. 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.872+0000 7f426579d700 1 Processor -- start 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.873+0000 7f426579d700 1 -- start start 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.873+0000 7f426579d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42600b16d0 0x7f42600c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.873+0000 7f426579d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f4260145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.873+0000 7f426579d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b4e70 0x7f4260145fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f426579d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600c95e0 con 0x7f42600b4e70 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f426579d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600c8730 con 0x7f42600b16d0 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f426579d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4260145cd0 con 0x7f42600b33b0 2023-10-08T01:11:36.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b4e70 0x7f4260145fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b4e70 0x7f4260145fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36290/0 (socket says 172.21.15.119:36290) 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 -- 172.21.15.119:0/3651988986 learned_addr learned my addr 172.21.15.119:0/3651988986 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f425ffff700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42600b16d0 0x7f42600c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f425f7fe700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f4260145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 -- 172.21.15.119:0/3651988986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 msgr2=0x7f4260145790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f4260145790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 -- 172.21.15.119:0/3651988986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42600b16d0 msgr2=0x7f42600c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42600b16d0 0x7f42600c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 -- 172.21.15.119:0/3651988986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4258009a20 con 0x7f42600b4e70 2023-10-08T01:11:36.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.874+0000 7f4264f9c700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b4e70 0x7f4260145fd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f425800b610 tx=0x7f4258019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:36.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.875+0000 7f4247fff700 1 -- 172.21.15.119:0/3651988986 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258012d80 con 0x7f42600b4e70 2023-10-08T01:11:36.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.875+0000 7f4247fff700 1 -- 172.21.15.119:0/3651988986 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42580263f0 con 0x7f42600b4e70 2023-10-08T01:11:36.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.875+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42600c89d0 con 0x7f42600b4e70 2023-10-08T01:11:36.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.875+0000 7f4247fff700 1 -- 172.21.15.119:0/3651988986 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258026790 con 0x7f42600b4e70 2023-10-08T01:11:36.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.875+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42600c8f90 con 0x7f42600b4e70 2023-10-08T01:11:36.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.876+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4260088990 con 0x7f42600b4e70 2023-10-08T01:11:36.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.877+0000 7f4247fff700 1 -- 172.21.15.119:0/3651988986 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 33) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f4258026b40 con 0x7f42600b4e70 2023-10-08T01:11:36.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.878+0000 7f4247fff700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f424c066f80 0x7f424c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:36.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.878+0000 7f4247fff700 1 -- 172.21.15.119:0/3651988986 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f4258086160 con 0x7f42600b4e70 2023-10-08T01:11:36.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.878+0000 7f425ffff700 1 -- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f424c066f80 msgr2=0x7f424c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:36.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.878+0000 7f425ffff700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f424c066f80 0x7f424c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:36.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:36.882+0000 7f4247fff700 1 -- 172.21.15.119:0/3651988986 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4258053240 con 0x7f42600b4e70 2023-10-08T01:11:37.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.079+0000 7f425ffff700 1 -- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f424c066f80 msgr2=0x7f424c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:37.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.079+0000 7f425ffff700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f424c066f80 0x7f424c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:37.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.080+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f426004bb80 con 0x7f42600b4e70 2023-10-08T01:11:37.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.085+0000 7f4247fff700 1 -- 172.21.15.119:0/3651988986 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f4258056860 con 0x7f42600b4e70 2023-10-08T01:11:37.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f424c066f80 msgr2=0x7f424c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f424c066f80 0x7f424c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b4e70 msgr2=0x7f4260145fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b4e70 0x7f4260145fd0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f425800b610 tx=0x7f4258019000 comp rx=0 tx=0).stop 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 shutdown_connections 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f424c066f80 0x7f424c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f42600b4e70 0x7f4260145fd0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42600b16d0 0x7f42600c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 --2- 172.21.15.119:0/3651988986 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42600b33b0 0x7f4260145790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 >> 172.21.15.119:0/3651988986 conn(0x7f42600d10c0 msgr2=0x7f426015c850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:37.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 shutdown_connections 2023-10-08T01:11:37.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.087+0000 7f426579d700 1 -- 172.21.15.119:0/3651988986 wait complete. 2023-10-08T01:11:37.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:37.106 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T01:11:37.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-08T01:11:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.381+0000 7f77fa59d700 1 Processor -- start 2023-10-08T01:11:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.382+0000 7f77fa59d700 1 -- start start 2023-10-08T01:11:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77fa59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77f40b03c0 0x7f77f40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77fa59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b0790 0x7f77f40b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77fa59d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f40cfa00 con 0x7f77f41591e0 2023-10-08T01:11:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77fa59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f40cb580 con 0x7f77f40b03c0 2023-10-08T01:11:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77fa59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40b2de0 con 0x7f77f40b0790 2023-10-08T01:11:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77f9d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f77f41591e0 0x7f77f4159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47198/0 (socket says 172.21.15.119:47198) 2023-10-08T01:11:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77f9d9c700 1 -- 172.21.15.119:0/1602949387 learned_addr learned my addr 172.21.15.119:0/1602949387 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77f959b700 1 --2- 172.21.15.119:0/1602949387 >> v2:172.21.15.184:3300/0 conn(0x7f77f40b03c0 0x7f77f40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.383+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/1602949387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b0790 0x7f77f40b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:37.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77ebfff700 1 -- 172.21.15.119:0/1602949387 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068631890 0 0) 0x7f77f40cfa00 con 0x7f77f41591e0 2023-10-08T01:11:37.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77ebfff700 1 -- 172.21.15.119:0/1602949387 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77e0003570 con 0x7f77f41591e0 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77f959b700 1 -- 172.21.15.119:0/1602949387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b0790 msgr2=0x7f77f40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77f959b700 1 --2- 172.21.15.119:0/1602949387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b0790 0x7f77f40b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77f959b700 1 -- 172.21.15.119:0/1602949387 >> v1:172.21.15.119:6789/0 conn(0x7f77f41591e0 legacy=0x7f77f4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77f959b700 1 -- 172.21.15.119:0/1602949387 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f40cf160 con 0x7f77f40b03c0 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77f959b700 1 --2- 172.21.15.119:0/1602949387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f40b03c0 0x7f77f40cea70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098e0 tx=0x7f77f00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd9e2b715ff706b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77ebfff700 1 -- 172.21.15.119:0/1602949387 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0011630 con 0x7f77f40b03c0 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77ebfff700 1 -- 172.21.15.119:0/1602949387 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f0011790 con 0x7f77f40b03c0 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.384+0000 7f77ebfff700 1 -- 172.21.15.119:0/1602949387 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0011b30 con 0x7f77f40b03c0 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.385+0000 7f77fa59d700 1 -- 172.21.15.119:0/1602949387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f40b03c0 msgr2=0x7f77f40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:37.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.385+0000 7f77fa59d700 1 --2- 172.21.15.119:0/1602949387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f40b03c0 0x7f77f40cea70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f77f00098e0 tx=0x7f77f00091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.385+0000 7f77fa59d700 1 -- 172.21.15.119:0/1602949387 shutdown_connections 2023-10-08T01:11:37.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.385+0000 7f77fa59d700 1 --2- 172.21.15.119:0/1602949387 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f40b03c0 0x7f77f40cea70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.385+0000 7f77fa59d700 1 --2- 172.21.15.119:0/1602949387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b0790 0x7f77f40b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.385+0000 7f77fa59d700 1 -- 172.21.15.119:0/1602949387 >> 172.21.15.119:0/1602949387 conn(0x7f77f4063230 msgr2=0x7f77f4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:37.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.385+0000 7f77fa59d700 1 -- 172.21.15.119:0/1602949387 shutdown_connections 2023-10-08T01:11:37.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.385+0000 7f77fa59d700 1 -- 172.21.15.119:0/1602949387 wait complete. 2023-10-08T01:11:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.386+0000 7f77fa59d700 1 Processor -- start 2023-10-08T01:11:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.387+0000 7f77fa59d700 1 -- start start 2023-10-08T01:11:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.387+0000 7f77fa59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b03c0 0x7f77f40c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.387+0000 7f77fa59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f40b0790 0x7f77f4150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.387+0000 7f77fa59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f41591e0 0x7f77f40c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.387+0000 7f77fa59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40c50b0 con 0x7f77f41591e0 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.387+0000 7f77fa59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40c46e0 con 0x7f77f40b0790 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.387+0000 7f77fa59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f40c6fd0 con 0x7f77f40b03c0 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f9d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f41591e0 0x7f77f40c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f959b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b03c0 0x7f77f40c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f959b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b03c0 0x7f77f40c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:37334/0 (socket says 172.21.15.119:37334) 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f959b700 1 -- 172.21.15.119:0/3109538069 learned_addr learned my addr 172.21.15.119:0/3109538069 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f40b0790 0x7f77f4150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f959b700 1 -- 172.21.15.119:0/3109538069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f40b0790 msgr2=0x7f77f4150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f959b700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77f40b0790 0x7f77f4150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f959b700 1 -- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f41591e0 msgr2=0x7f77f40c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f959b700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f41591e0 0x7f77f40c6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f959b700 1 -- 172.21.15.119:0/3109538069 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f0009420 con 0x7f77f40b03c0 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.388+0000 7f77f9d9c700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77f41591e0 0x7f77f40c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.389+0000 7f77f959b700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77f40b03c0 0x7f77f40c4b70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f77f0011f20 tx=0x7f77f0017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.389+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0011900 con 0x7f77f40b03c0 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.389+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f0011a60 con 0x7f77f40b03c0 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.389+0000 7f77fa59d700 1 -- 172.21.15.119:0/3109538069 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77f40c4090 con 0x7f77f40b03c0 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.389+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0023620 con 0x7f77f40b03c0 2023-10-08T01:11:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.389+0000 7f77fa59d700 1 -- 172.21.15.119:0/3109538069 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f40c72b0 con 0x7f77f40b03c0 2023-10-08T01:11:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.390+0000 7f77fa59d700 1 -- 172.21.15.119:0/3109538069 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77f404c3c0 con 0x7f77f40b03c0 2023-10-08T01:11:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.392+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 33) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f77f002c3b0 con 0x7f77f40b03c0 2023-10-08T01:11:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.393+0000 7f77e97fa700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.393+0000 7f77f8d9a700 1 -- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 msgr2=0x7f77d8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.393+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f77f0085a50 con 0x7f77f40b03c0 2023-10-08T01:11:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.393+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:37.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.398+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77f0052b10 con 0x7f77f40b03c0 2023-10-08T01:11:37.504 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:37.504+0000 7f7aa385b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:11:37.562 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:37.563+0000 7f4f6828f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:11:37.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.594+0000 7f77f8d9a700 1 -- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 msgr2=0x7f77d8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:37.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.594+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:37.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.625+0000 7f77fa59d700 1 -- 172.21.15.119:0/3109538069 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f77f40c4820 con 0x7f77d8066f80 2023-10-08T01:11:37.665 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:37.665+0000 7f7aa385b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:37.728 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:37.729+0000 7f4f6828f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:37.753 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:37.753+0000 7f7aa385b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:11:37.817 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:37.817+0000 7f4f6828f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:11:37.980 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:37.979+0000 7f7aa385b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.995+0000 7f77f8d9a700 1 -- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 msgr2=0x7f77d8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:37.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:37.995+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:11:38.041 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:38.041+0000 7f4f6828f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:38.156 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:38.156+0000 7f7aa385b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:11:38.220 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:38.220+0000 7f4f6828f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:11:38.248 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:38.248+0000 7f7aa385b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:11:38.311 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:38.311+0000 7f4f6828f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:11:38.410 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:38.409+0000 7f7aa385b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:11:38.472 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:38.473+0000 7f4f6828f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:11:38.510 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:38.510+0000 7f7aa385b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:38.573 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:38.573+0000 7f4f6828f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:38.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:38.796+0000 7f77f8d9a700 1 -- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 msgr2=0x7f77d8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:38.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:38.796+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T01:11:38.994 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:38.993+0000 7f7aa385b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:11:39.054 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:39.054+0000 7f4f6828f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:11:39.088 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:39.087+0000 7f7aa385b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:11:39.148 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:39.148+0000 7f4f6828f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:11:39.178 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:39.178+0000 7f7aa385b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:11:39.237 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:39.237+0000 7f4f6828f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:11:39.800 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:39.800+0000 7f7aa385b180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:11:39.855 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:39.855+0000 7f4f6828f180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:11:39.972 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:39.972+0000 7f7aa385b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:40.027 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:40.028+0000 7f4f6828f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:40.067 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:40.067+0000 7f7aa385b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:11:40.123 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:40.123+0000 7f4f6828f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:11:40.359 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:40.359+0000 7f7aa385b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:11:40.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:40.398+0000 7f77f8d9a700 1 -- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 msgr2=0x7f77d8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:40.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:40.398+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T01:11:40.415 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:40.416+0000 7f4f6828f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:11:40.455 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:40.454+0000 7f7aa385b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:11:40.511 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:40.511+0000 7f4f6828f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:11:41.157 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:41.155+0000 7f7aa385b180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:11:41.210 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:41.210+0000 7f4f6828f180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:11:41.442 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:41.442+0000 7f7aa385b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:11:41.494 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:41.494+0000 7f4f6828f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:11:41.536 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:41.536+0000 7f7aa385b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:11:41.587 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:41.587+0000 7f4f6828f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:11:41.690 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:41.690+0000 7f7aa385b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:11:41.741 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:41.741+0000 7f4f6828f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:11:41.795 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:41.795+0000 7f7aa385b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:11:41.846 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:41.846+0000 7f4f6828f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:11:41.980 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:41.980+0000 7f7aa385b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:11:42.035 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.034+0000 7f4f6828f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:11:42.071 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:42.071+0000 7f7aa385b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:11:42.127 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.127+0000 7f4f6828f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:11:42.162 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:42.162+0000 7f7aa385b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:11:42.218 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.218+0000 7f4f6828f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:11:42.255 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:42.255+0000 7f7aa385b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:11:42.312 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.312+0000 7f4f6828f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:11:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:42.315+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mgrmap(e 34) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f77f0052ee0 con 0x7f77f40b03c0 2023-10-08T01:11:42.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:42.329+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 8 ==== mgrmap(e 35) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f77f0055cc0 con 0x7f77f40b03c0 2023-10-08T01:11:42.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:42.330+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 msgr2=0x7f77d8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:42.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:42.330+0000 7f77e97fa700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:42.444 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.444+0000 7f4eda5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.444 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.444+0000 7f4eda5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.444 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.444+0000 7f4eda5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.445 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.444+0000 7f4eda5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.445 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.444+0000 7f4eda5f1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.445 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.445+0000 7f4edd5f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.445 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.445+0000 7f4edd5f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.445 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.445+0000 7f4edd5f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.446 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.445+0000 7f4edd5f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:42.446 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:42.445+0000 7f4edd5f7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.341+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mon.2 v2:172.21.15.184:3301/0 9 ==== mgrmap(e 36) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f77f00537b0 con 0x7f77f40b03c0 2023-10-08T01:11:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.341+0000 7f77e97fa700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d80693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:43.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.341+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f77d8033320 con 0x7f77d8066f80 2023-10-08T01:11:43.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.342+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d80693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:43.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.344+0000 7f77f8d9a700 1 --2- 172.21.15.119:0/3109538069 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77d8066f80 0x7f77d80693b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f77f40c3cc0 tx=0x7f77e40091a0 comp rx=0 tx=0).ready entity=mgr.6438 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:43.352 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:43.352+0000 7f4ee4605700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-08T01:11:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.353+0000 7f77e97fa700 1 -- 172.21.15.119:0/3109538069 <== mgr.6438 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f77d8033320 con 0x7f77d8066f80 2023-10-08T01:11:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-08T01:11:43.364 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T01:11:43.364 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:43.364 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T01:11:43.406 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T01:11:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.630+0000 7f7e747ff700 1 Processor -- start 2023-10-08T01:11:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.631+0000 7f7e747ff700 1 -- start start 2023-10-08T01:11:43.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e747ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 0x7f7e6c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:43.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e747ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e6c0ccee0 0x7f7e6c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:43.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e747ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c15b0d0 con 0x7f7e6c0c87c0 2023-10-08T01:11:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e747ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c0c7a60 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e747ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c15a4c0 con 0x7f7e6c0c83f0 2023-10-08T01:11:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e71d9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e6c0ccee0 0x7f7e6c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e72d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7e6c0c87c0 0x7f7e6c06eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47204/0 (socket says 172.21.15.119:47204) 2023-10-08T01:11:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e71d9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e6c0ccee0 0x7f7e6c1561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36428/0 (socket says 172.21.15.119:36428) 2023-10-08T01:11:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e72d9c700 1 -- 172.21.15.119:0/780021571 learned_addr learned my addr 172.21.15.119:0/780021571 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.632+0000 7f7e7259b700 1 --2- 172.21.15.119:0/780021571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 0x7f7e6c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71599700 1 -- 172.21.15.119:0/780021571 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188422962 0 0) 0x7f7e6c15b0d0 con 0x7f7e6c0c87c0 2023-10-08T01:11:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71d9a700 1 -- 172.21.15.119:0/780021571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 msgr2=0x7f7e6c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71d9a700 1 --2- 172.21.15.119:0/780021571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 0x7f7e6c0c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71d9a700 1 -- 172.21.15.119:0/780021571 >> v1:172.21.15.119:6789/0 conn(0x7f7e6c0c87c0 legacy=0x7f7e6c06eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71d9a700 1 -- 172.21.15.119:0/780021571 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c15a7a0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71d9a700 1 --2- 172.21.15.119:0/780021571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c1561d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00bb90 tx=0x7f7e5c00b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5135610f4a1a56e1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71599700 1 -- 172.21.15.119:0/780021571 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c00db40 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71599700 1 -- 172.21.15.119:0/780021571 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c00dca0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.633+0000 7f7e71599700 1 -- 172.21.15.119:0/780021571 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c0196b0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.634+0000 7f7e747ff700 1 -- 172.21.15.119:0/780021571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 msgr2=0x7f7e6c1561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.634+0000 7f7e747ff700 1 --2- 172.21.15.119:0/780021571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c1561d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00bb90 tx=0x7f7e5c00b430 comp rx=0 tx=0).stop 2023-10-08T01:11:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.634+0000 7f7e747ff700 1 -- 172.21.15.119:0/780021571 shutdown_connections 2023-10-08T01:11:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.634+0000 7f7e747ff700 1 --2- 172.21.15.119:0/780021571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 0x7f7e6c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.634+0000 7f7e747ff700 1 --2- 172.21.15.119:0/780021571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c1561d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.634+0000 7f7e747ff700 1 -- 172.21.15.119:0/780021571 >> 172.21.15.119:0/780021571 conn(0x7f7e6c068e10 msgr2=0x7f7e6c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.634+0000 7f7e747ff700 1 -- 172.21.15.119:0/780021571 shutdown_connections 2023-10-08T01:11:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.634+0000 7f7e747ff700 1 -- 172.21.15.119:0/780021571 wait complete. 2023-10-08T01:11:43.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.635+0000 7f7e747ff700 1 Processor -- start 2023-10-08T01:11:43.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.635+0000 7f7e747ff700 1 -- start start 2023-10-08T01:11:43.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.636+0000 7f7e747ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 0x7f7e6c14ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:43.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.636+0000 7f7e747ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e6c0c87c0 0x7f7e6c0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:43.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.636+0000 7f7e747ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.636+0000 7f7e747ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c16d570 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.636+0000 7f7e747ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c16e3d0 con 0x7f7e6c0c87c0 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.636+0000 7f7e747ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c16d7c0 con 0x7f7e6c0c83f0 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c156750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36444/0 (socket says 172.21.15.119:36444) 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 -- 172.21.15.119:0/2068714692 learned_addr learned my addr 172.21.15.119:0/2068714692 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e7259b700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 0x7f7e6c14ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e71d9a700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e6c0c87c0 0x7f7e6c0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 -- 172.21.15.119:0/2068714692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 msgr2=0x7f7e6c14ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 0x7f7e6c14ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 -- 172.21.15.119:0/2068714692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e6c0c87c0 msgr2=0x7f7e6c0c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e6c0c87c0 0x7f7e6c0c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 -- 172.21.15.119:0/2068714692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c00b6d0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.637+0000 7f7e72d9c700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c156750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ccc0 tx=0x7f7e64001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.638+0000 7f7e627fc700 1 -- 172.21.15.119:0/2068714692 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400a8a0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.638+0000 7f7e627fc700 1 -- 172.21.15.119:0/2068714692 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6400aa00 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.638+0000 7f7e627fc700 1 -- 172.21.15.119:0/2068714692 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6400adc0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.638+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c16daa0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.638+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c16e060 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.639+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c04c3c0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.640+0000 7f7e627fc700 1 -- 172.21.15.119:0/2068714692 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 36) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f7e64007300 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.640+0000 7f7e627fc700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7e50066dd0 0x7f7e50069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.641+0000 7f7e627fc700 1 -- 172.21.15.119:0/2068714692 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f7e640848c0 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.641+0000 7f7e7259b700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7e50066dd0 0x7f7e50069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.641+0000 7f7e7259b700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7e50066dd0 0x7f7e50069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009db0 tx=0x7f7e680091a0 comp rx=0 tx=0).ready entity=mgr.6438 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.645+0000 7f7e627fc700 1 -- 172.21.15.119:0/2068714692 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7e64051b00 con 0x7f7e6c0ccee0 2023-10-08T01:11:43.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:43.887+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f7e6c16e510 con 0x7f7e6c0ccee0 2023-10-08T01:11:44.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.343+0000 7f7e627fc700 1 -- 172.21.15.119:0/2068714692 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f7e64051ed0 con 0x7f7e6c0ccee0 2023-10-08T01:11:44.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.344+0000 7f7e627fc700 1 -- 172.21.15.119:0/2068714692 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v37) v1 ==== 93+0+0 (secure 0 0 0) 0x7f7e64051b00 con 0x7f7e6c0ccee0 2023-10-08T01:11:44.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7e50066dd0 msgr2=0x7f7e50069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7e50066dd0 0x7f7e50069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009db0 tx=0x7f7e680091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 msgr2=0x7f7e6c156750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c156750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400ccc0 tx=0x7f7e64001a80 comp rx=0 tx=0).stop 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 shutdown_connections 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7e50066dd0 0x7f7e50069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e6c0c83f0 0x7f7e6c14ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e6c0ccee0 0x7f7e6c156750 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 --2- 172.21.15.119:0/2068714692 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e6c0c87c0 0x7f7e6c0c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.347+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 >> 172.21.15.119:0/2068714692 conn(0x7f7e6c068e10 msgr2=0x7f7e6c0c2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:44.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.348+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 shutdown_connections 2023-10-08T01:11:44.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.348+0000 7f7e747ff700 1 -- 172.21.15.119:0/2068714692 wait complete. 2023-10-08T01:11:44.375 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:11:44.375 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:11:44.461 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:11:44.542 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:11:44.542 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T01:11:44.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:11:44.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T01:11:44.632 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T01:11:44.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:11:44.632 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T01:11:44.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.904+0000 7f36c9b9d700 1 Processor -- start 2023-10-08T01:11:44.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.904+0000 7f36c9b9d700 1 -- start start 2023-10-08T01:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.905+0000 7f36c9b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c40b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:44.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.905+0000 7f36c9b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36c41597f0 0x7f36c40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.905+0000 7f36c9b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c40b11d0 con 0x7f36c415d230 2023-10-08T01:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.905+0000 7f36c9b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c40b14b0 con 0x7f36c41597f0 2023-10-08T01:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.905+0000 7f36c9b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40b1790 con 0x7f36c4159420 2023-10-08T01:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c939c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f36c415d230 0x7f36c40b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60624/0 (socket says 172.21.15.119:60624) 2023-10-08T01:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c939c700 1 -- 172.21.15.119:0/3510706437 learned_addr learned my addr 172.21.15.119:0/3510706437 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c8b9b700 1 --2- 172.21.15.119:0/3510706437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c40b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:44.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c3fff700 1 --2- 172.21.15.119:0/3510706437 >> v2:172.21.15.184:3300/0 conn(0x7f36c41597f0 0x7f36c40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:44.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c37fe700 1 -- 172.21.15.119:0/3510706437 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767043414 0 0) 0x7f36c40b11d0 con 0x7f36c415d230 2023-10-08T01:11:44.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c37fe700 1 -- 172.21.15.119:0/3510706437 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36b4003570 con 0x7f36c415d230 2023-10-08T01:11:44.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c8b9b700 1 -- 172.21.15.119:0/3510706437 >> v2:172.21.15.184:3300/0 conn(0x7f36c41597f0 msgr2=0x7f36c40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:44.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c8b9b700 1 --2- 172.21.15.119:0/3510706437 >> v2:172.21.15.184:3300/0 conn(0x7f36c41597f0 0x7f36c40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c8b9b700 1 -- 172.21.15.119:0/3510706437 >> v1:172.21.15.119:6789/0 conn(0x7f36c415d230 legacy=0x7f36c40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:44.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c8b9b700 1 -- 172.21.15.119:0/3510706437 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40b1a70 con 0x7f36c4159420 2023-10-08T01:11:44.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.906+0000 7f36c3fff700 1 --2- 172.21.15.119:0/3510706437 >> v2:172.21.15.184:3300/0 conn(0x7f36c41597f0 0x7f36c40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:44.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c8b9b700 1 --2- 172.21.15.119:0/3510706437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c40b5f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f36b80098e0 tx=0x7f36b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8a5411d7019b59e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:44.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c37fe700 1 -- 172.21.15.119:0/3510706437 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8011630 con 0x7f36c4159420 2023-10-08T01:11:44.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c37fe700 1 -- 172.21.15.119:0/3510706437 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b8011790 con 0x7f36c4159420 2023-10-08T01:11:44.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c37fe700 1 -- 172.21.15.119:0/3510706437 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b8011b30 con 0x7f36c4159420 2023-10-08T01:11:44.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c9b9d700 1 -- 172.21.15.119:0/3510706437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 msgr2=0x7f36c40b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:44.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/3510706437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c40b5f40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f36b80098e0 tx=0x7f36b80091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c9b9d700 1 -- 172.21.15.119:0/3510706437 shutdown_connections 2023-10-08T01:11:44.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/3510706437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c40b5f40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/3510706437 >> v2:172.21.15.184:3300/0 conn(0x7f36c41597f0 0x7f36c40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.907+0000 7f36c9b9d700 1 -- 172.21.15.119:0/3510706437 >> 172.21.15.119:0/3510706437 conn(0x7f36c40d0660 msgr2=0x7f36c40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:44.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.908+0000 7f36c9b9d700 1 -- 172.21.15.119:0/3510706437 shutdown_connections 2023-10-08T01:11:44.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.908+0000 7f36c9b9d700 1 -- 172.21.15.119:0/3510706437 wait complete. 2023-10-08T01:11:44.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.909+0000 7f36c9b9d700 1 Processor -- start 2023-10-08T01:11:44.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.909+0000 7f36c9b9d700 1 -- start start 2023-10-08T01:11:44.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c9b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c4156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:44.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c9b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36c41597f0 0x7f36c415e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:44.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c9b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c415d230 0x7f36c415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:44.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c9b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c415f100 con 0x7f36c41597f0 2023-10-08T01:11:44.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c9b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40c8fe0 con 0x7f36c415d230 2023-10-08T01:11:44.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c9b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c415eb40 con 0x7f36c4159420 2023-10-08T01:11:44.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36c41597f0 0x7f36c415e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:44.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c8b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c4156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:44.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36c41597f0 0x7f36c415e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44728/0 (socket says 172.21.15.119:44728) 2023-10-08T01:11:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c3fff700 1 -- 172.21.15.119:0/4177220982 learned_addr learned my addr 172.21.15.119:0/4177220982 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.910+0000 7f36c939c700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c415d230 0x7f36c415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c3fff700 1 -- 172.21.15.119:0/4177220982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 msgr2=0x7f36c4156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c3fff700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c4156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c3fff700 1 -- 172.21.15.119:0/4177220982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c415d230 msgr2=0x7f36c415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c3fff700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c415d230 0x7f36c415f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c3fff700 1 -- 172.21.15.119:0/4177220982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b0009420 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c3fff700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36c41597f0 0x7f36c415e050 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f36b000ec80 tx=0x7f36b00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c0ff9700 1 -- 172.21.15.119:0/4177220982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b000c8c0 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c0ff9700 1 -- 172.21.15.119:0/4177220982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b000ca20 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b8009420 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c0ff9700 1 -- 172.21.15.119:0/4177220982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b000cda0 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.911+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c40c9210 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.913+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c404c3c0 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.913+0000 7f36c0ff9700 1 -- 172.21.15.119:0/4177220982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f36b0013680 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.914+0000 7f36c0ff9700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36a80670e0 0x7f36a8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.914+0000 7f36c0ff9700 1 -- 172.21.15.119:0/4177220982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f36b0084e50 con 0x7f36c41597f0 2023-10-08T01:11:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.914+0000 7f36c8b9b700 1 -- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36a80670e0 msgr2=0x7f36a8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:44.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.914+0000 7f36c8b9b700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36a80670e0 0x7f36a8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:44.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:44.918+0000 7f36c0ff9700 1 -- 172.21.15.119:0/4177220982 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f36b0051bb0 con 0x7f36c41597f0 2023-10-08T01:11:45.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.115+0000 7f36c8b9b700 1 -- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36a80670e0 msgr2=0x7f36a8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:45.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.115+0000 7f36c8b9b700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36a80670e0 0x7f36a8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:45.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.162+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36c40b1d50 con 0x7f36c41597f0 2023-10-08T01:11:45.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.163+0000 7f36c0ff9700 1 -- 172.21.15.119:0/4177220982 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f36b00551d0 con 0x7f36c41597f0 2023-10-08T01:11:45.164 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:45.164 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4024548231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":4024548231},{"type":"v1","addr":"172.21.15.184:6835","nonce":4024548231}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1924614082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1924614082},{"type":"v1","addr":"172.21.15.119:6835","nonce":1924614082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5902,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/296234355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":296234355},{"type":"v1","addr":"172.21.15.184:6834","nonce":296234355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:11:21.183828+0000","modified":"2023-10-08T01:11:25.168981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6225":{"gid":6225,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6834/926130691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":926130691},{"type":"v1","addr":"172.21.15.119:6834","nonce":926130691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T01:11:45.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.165+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36a80670e0 msgr2=0x7f36a8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:45.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.165+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36a80670e0 0x7f36a8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.165+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36c41597f0 msgr2=0x7f36c415e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.165+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36c41597f0 0x7f36c415e050 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f36b000ec80 tx=0x7f36b00091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.165+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 shutdown_connections 2023-10-08T01:11:45.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.166+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36a80670e0 0x7f36a8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.166+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36c4159420 0x7f36c4156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.166+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36c415d230 0x7f36c415f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.166+0000 7f36c9b9d700 1 --2- 172.21.15.119:0/4177220982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36c41597f0 0x7f36c415e050 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.166+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 >> 172.21.15.119:0/4177220982 conn(0x7f36c40d0660 msgr2=0x7f36c40b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:45.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.166+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 shutdown_connections 2023-10-08T01:11:45.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.166+0000 7f36c9b9d700 1 -- 172.21.15.119:0/4177220982 wait complete. 2023-10-08T01:11:45.168 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 32 2023-10-08T01:11:45.186 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:45.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.375+0000 7f77a939d700 1 Processor -- start 2023-10-08T01:11:45.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.375+0000 7f77a939d700 1 -- start start 2023-10-08T01:11:45.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.376+0000 7f77a939d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77a40b1e50 0x7f77a4159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a939d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40cfe20 con 0x7f77a41593b0 2023-10-08T01:11:45.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a939d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40cf210 con 0x7f77a40ceaa0 2023-10-08T01:11:45.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a939d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40cf4f0 con 0x7f77a40b1e50 2023-10-08T01:11:45.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f77a41593b0 0x7f77a40b1740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60632/0 (socket says 172.21.15.119:60632) 2023-10-08T01:11:45.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a3fff700 1 -- 172.21.15.119:0/41524262 learned_addr learned my addr 172.21.15.119:0/41524262 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:45.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a37fe700 1 --2- 172.21.15.119:0/41524262 >> v2:172.21.15.184:3300/0 conn(0x7f77a40b1e50 0x7f77a4159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a2ffd700 1 -- 172.21.15.119:0/41524262 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451948379 0 0) 0x7f77a40cfe20 con 0x7f77a41593b0 2023-10-08T01:11:45.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a2ffd700 1 -- 172.21.15.119:0/41524262 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778c003570 con 0x7f77a41593b0 2023-10-08T01:11:45.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a37fe700 1 -- 172.21.15.119:0/41524262 >> v1:172.21.15.184:6789/0 conn(0x7f77a40ceaa0 legacy=0x7f77a40b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a37fe700 1 -- 172.21.15.119:0/41524262 >> v1:172.21.15.119:6789/0 conn(0x7f77a41593b0 legacy=0x7f77a40b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.377+0000 7f77a37fe700 1 -- 172.21.15.119:0/41524262 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40cf7d0 con 0x7f77a40b1e50 2023-10-08T01:11:45.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.378+0000 7f77a37fe700 1 --2- 172.21.15.119:0/41524262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a40b1e50 0x7f77a4159780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f77980098e0 tx=0x7f77980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1afe366263507d50 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:45.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.378+0000 7f77a2ffd700 1 -- 172.21.15.119:0/41524262 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800b640 con 0x7f77a40b1e50 2023-10-08T01:11:45.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.378+0000 7f77a2ffd700 1 -- 172.21.15.119:0/41524262 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779800b7a0 con 0x7f77a40b1e50 2023-10-08T01:11:45.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.378+0000 7f77a2ffd700 1 -- 172.21.15.119:0/41524262 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779800bb40 con 0x7f77a40b1e50 2023-10-08T01:11:45.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.379+0000 7f77a939d700 1 -- 172.21.15.119:0/41524262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a40b1e50 msgr2=0x7f77a4159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.379+0000 7f77a939d700 1 --2- 172.21.15.119:0/41524262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a40b1e50 0x7f77a4159780 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f77980098e0 tx=0x7f77980091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.379+0000 7f77a939d700 1 -- 172.21.15.119:0/41524262 shutdown_connections 2023-10-08T01:11:45.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.379+0000 7f77a939d700 1 --2- 172.21.15.119:0/41524262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a40b1e50 0x7f77a4159780 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.379+0000 7f77a939d700 1 -- 172.21.15.119:0/41524262 >> 172.21.15.119:0/41524262 conn(0x7f77a40d1130 msgr2=0x7f77a40b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:45.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.379+0000 7f77a939d700 1 -- 172.21.15.119:0/41524262 shutdown_connections 2023-10-08T01:11:45.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.379+0000 7f77a939d700 1 -- 172.21.15.119:0/41524262 wait complete. 2023-10-08T01:11:45.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.380+0000 7f77a939d700 1 Processor -- start 2023-10-08T01:11:45.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.380+0000 7f77a939d700 1 -- start start 2023-10-08T01:11:45.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.381+0000 7f77a939d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a40b1e50 0x7f77a41697e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.381+0000 7f77a939d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a40ceaa0 0x7f77a416c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.381+0000 7f77a939d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a41593b0 0x7f77a415b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.381+0000 7f77a939d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a415a700 con 0x7f77a40b1e50 2023-10-08T01:11:45.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.381+0000 7f77a939d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40c55a0 con 0x7f77a41593b0 2023-10-08T01:11:45.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.381+0000 7f77a939d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40c5880 con 0x7f77a40ceaa0 2023-10-08T01:11:45.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a40b1e50 0x7f77a41697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a40b1e50 0x7f77a41697e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44734/0 (socket says 172.21.15.119:44734) 2023-10-08T01:11:45.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 -- 172.21.15.119:0/3986181346 learned_addr learned my addr 172.21.15.119:0/3986181346 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:45.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a37fe700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a40ceaa0 0x7f77a416c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a8b9c700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a41593b0 0x7f77a415b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 -- 172.21.15.119:0/3986181346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a40ceaa0 msgr2=0x7f77a416c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a40ceaa0 0x7f77a416c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 -- 172.21.15.119:0/3986181346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a41593b0 msgr2=0x7f77a415b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a41593b0 0x7f77a415b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 -- 172.21.15.119:0/3986181346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7798009420 con 0x7f77a40b1e50 2023-10-08T01:11:45.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.382+0000 7f77a3fff700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a40b1e50 0x7f77a41697e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f779400c590 tx=0x7f7794007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:45.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.383+0000 7f778bfff700 1 -- 172.21.15.119:0/3986181346 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794005b30 con 0x7f77a40b1e50 2023-10-08T01:11:45.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.383+0000 7f778bfff700 1 -- 172.21.15.119:0/3986181346 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794005cb0 con 0x7f77a40b1e50 2023-10-08T01:11:45.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.383+0000 7f778bfff700 1 -- 172.21.15.119:0/3986181346 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7794007320 con 0x7f77a40b1e50 2023-10-08T01:11:45.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.383+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a40c5b60 con 0x7f77a40b1e50 2023-10-08T01:11:45.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.383+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a416caa0 con 0x7f77a40b1e50 2023-10-08T01:11:45.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.384+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a404c3c0 con 0x7f77a40b1e50 2023-10-08T01:11:45.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.385+0000 7f778bfff700 1 -- 172.21.15.119:0/3986181346 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f779401b3b0 con 0x7f77a40b1e50 2023-10-08T01:11:45.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.386+0000 7f778bfff700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77900670e0 0x7f7790069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.386+0000 7f778bfff700 1 -- 172.21.15.119:0/3986181346 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f7794083ff0 con 0x7f77a40b1e50 2023-10-08T01:11:45.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.386+0000 7f77a37fe700 1 -- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77900670e0 msgr2=0x7f7790069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:45.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.386+0000 7f77a37fe700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77900670e0 0x7f7790069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:45.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.390+0000 7f778bfff700 1 -- 172.21.15.119:0/3986181346 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f77940b55a0 con 0x7f77a40b1e50 2023-10-08T01:11:45.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.587+0000 7f77a37fe700 1 -- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77900670e0 msgr2=0x7f7790069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:45.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.587+0000 7f77a37fe700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77900670e0 0x7f7790069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:45.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.588+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77a40cfab0 con 0x7f77a40b1e50 2023-10-08T01:11:45.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.589+0000 7f778bfff700 1 -- 172.21.15.119:0/3986181346 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f7794017020 con 0x7f77a40b1e50 2023-10-08T01:11:45.590 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:45.590 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":78,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:42.316661+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:11:19.347044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T01:11:20.164667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/322083914":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/4063763211":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/2918246125":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/1513448491":"2023-10-09T01:11:42.316619+0000","172.21.15.119:6837/85511":"2023-10-09T01:11:42.316619+0000","172.21.15.119:6836/85511":"2023-10-09T01:11:42.316619+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:45.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77900670e0 msgr2=0x7f7790069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:45.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77900670e0 0x7f7790069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a40b1e50 msgr2=0x7f77a41697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a40b1e50 0x7f77a41697e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f779400c590 tx=0x7f7794007e00 comp rx=0 tx=0).stop 2023-10-08T01:11:45.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 shutdown_connections 2023-10-08T01:11:45.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f77900670e0 0x7f7790069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a41593b0 0x7f77a415b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a40b1e50 0x7f77a41697e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 --2- 172.21.15.119:0/3986181346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a40ceaa0 0x7f77a416c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 >> 172.21.15.119:0/3986181346 conn(0x7f77a40d1130 msgr2=0x7f77a4157e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:45.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 shutdown_connections 2023-10-08T01:11:45.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.591+0000 7f77a939d700 1 -- 172.21.15.119:0/3986181346 wait complete. 2023-10-08T01:11:45.611 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T01:11:19.347044+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T01:11:20.164667+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T01:11:45.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:11:45.637 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:45.637+0000 7f67b3774180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:11:45.697 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:45.695+0000 7f9aba263180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:11:45.804 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:45.803+0000 7f67b3774180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:45.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.805+0000 7f872a39d700 1 Processor -- start 2023-10-08T01:11:45.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.805+0000 7f872a39d700 1 -- start start 2023-10-08T01:11:45.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.806+0000 7f872a39d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240b4e70 0x7f87240ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.806+0000 7f872a39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b16d0 0x7f87240b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.806+0000 7f872a39d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240b2080 con 0x7f872415d240 2023-10-08T01:11:45.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.806+0000 7f872a39d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240cfb00 con 0x7f87240b4e70 2023-10-08T01:11:45.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.806+0000 7f872a39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240b5c40 con 0x7f87240b16d0 2023-10-08T01:11:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.806+0000 7f872939b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240b4e70 0x7f87240ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f872939b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240b4e70 0x7f87240ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44748/0 (socket says 172.21.15.119:44748) 2023-10-08T01:11:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f872939b700 1 -- 172.21.15.119:0/2863374900 learned_addr learned my addr 172.21.15.119:0/2863374900 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f8728b9a700 1 --2- 172.21.15.119:0/2863374900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b16d0 0x7f87240b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f872939b700 1 -- 172.21.15.119:0/2863374900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b16d0 msgr2=0x7f87240b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f872939b700 1 --2- 172.21.15.119:0/2863374900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b16d0 0x7f87240b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f872939b700 1 -- 172.21.15.119:0/2863374900 >> v1:172.21.15.184:6789/0 conn(0x7f872415d240 legacy=0x7f872415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f872939b700 1 -- 172.21.15.119:0/2863374900 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240b5f20 con 0x7f87240b4e70 2023-10-08T01:11:45.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f872939b700 1 --2- 172.21.15.119:0/2863374900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b4e70 0x7f87240ccff0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f87200099b0 tx=0x7f8720009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636bc899e4d03718 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:45.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.807+0000 7f871bfff700 1 -- 172.21.15.119:0/2863374900 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720009ea0 con 0x7f87240b4e70 2023-10-08T01:11:45.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f871bfff700 1 -- 172.21.15.119:0/2863374900 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872000b860 con 0x7f87240b4e70 2023-10-08T01:11:45.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f871bfff700 1 -- 172.21.15.119:0/2863374900 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000bc00 con 0x7f87240b4e70 2023-10-08T01:11:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f872a39d700 1 -- 172.21.15.119:0/2863374900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b4e70 msgr2=0x7f87240ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f872a39d700 1 --2- 172.21.15.119:0/2863374900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b4e70 0x7f87240ccff0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f87200099b0 tx=0x7f8720009250 comp rx=0 tx=0).stop 2023-10-08T01:11:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f872a39d700 1 -- 172.21.15.119:0/2863374900 shutdown_connections 2023-10-08T01:11:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f872a39d700 1 --2- 172.21.15.119:0/2863374900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b4e70 0x7f87240ccff0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f872a39d700 1 --2- 172.21.15.119:0/2863374900 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b16d0 0x7f87240b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f872a39d700 1 -- 172.21.15.119:0/2863374900 >> 172.21.15.119:0/2863374900 conn(0x7f87240d10c0 msgr2=0x7f87240affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:45.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f872a39d700 1 -- 172.21.15.119:0/2863374900 shutdown_connections 2023-10-08T01:11:45.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.808+0000 7f872a39d700 1 -- 172.21.15.119:0/2863374900 wait complete. 2023-10-08T01:11:45.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.810+0000 7f872a39d700 1 Processor -- start 2023-10-08T01:11:45.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.810+0000 7f872a39d700 1 -- start start 2023-10-08T01:11:45.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872a39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b16d0 0x7f87240ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872a39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b4e70 0x7f872414b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872a39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f872415d240 0x7f8724068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872a39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f872414b600 con 0x7f87240b16d0 2023-10-08T01:11:45.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872a39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240caf10 con 0x7f872415d240 2023-10-08T01:11:45.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872a39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87240cb1f0 con 0x7f87240b4e70 2023-10-08T01:11:45.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b16d0 0x7f87240ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b16d0 0x7f87240ca8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44754/0 (socket says 172.21.15.119:44754) 2023-10-08T01:11:45.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872939b700 1 -- 172.21.15.119:0/3367281223 learned_addr learned my addr 172.21.15.119:0/3367281223 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f8728b9a700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b4e70 0x7f872414b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f8729b9c700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f872415d240 0x7f8724068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872939b700 1 -- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b4e70 msgr2=0x7f872414b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872939b700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b4e70 0x7f872414b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872939b700 1 -- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f872415d240 msgr2=0x7f8724068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872939b700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f872415d240 0x7f8724068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f872939b700 1 -- 172.21.15.119:0/3367281223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87200094f0 con 0x7f87240b16d0 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.811+0000 7f8729b9c700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f872415d240 0x7f8724068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.812+0000 7f872939b700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b16d0 0x7f87240ca8a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8724070dd0 tx=0x7f8720017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.812+0000 7f87197fa700 1 -- 172.21.15.119:0/3367281223 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000bbe0 con 0x7f87240b16d0 2023-10-08T01:11:45.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.812+0000 7f87197fa700 1 -- 172.21.15.119:0/3367281223 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8720011420 con 0x7f87240b16d0 2023-10-08T01:11:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.812+0000 7f87197fa700 1 -- 172.21.15.119:0/3367281223 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200117c0 con 0x7f87240b16d0 2023-10-08T01:11:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.812+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87240cb4d0 con 0x7f87240b16d0 2023-10-08T01:11:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.812+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87240cba90 con 0x7f87240b16d0 2023-10-08T01:11:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.813+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87240cbcb0 con 0x7f87240b16d0 2023-10-08T01:11:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.814+0000 7f87197fa700 1 -- 172.21.15.119:0/3367281223 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f8720011b70 con 0x7f87240b16d0 2023-10-08T01:11:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.815+0000 7f87197fa700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:45.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.815+0000 7f87197fa700 1 -- 172.21.15.119:0/3367281223 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f8720086300 con 0x7f87240b16d0 2023-10-08T01:11:45.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.815+0000 7f8728b9a700 1 -- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 msgr2=0x7f8708069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:45.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.815+0000 7f8728b9a700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:45.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:45.819+0000 7f87197fa700 1 -- 172.21.15.119:0/3367281223 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8720053060 con 0x7f87240b16d0 2023-10-08T01:11:45.873 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:45.872+0000 7f9aba263180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:45.891 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:45.891+0000 7f67b3774180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:11:45.962 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:45.961+0000 7f9aba263180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:11:46.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.016+0000 7f8728b9a700 1 -- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 msgr2=0x7f8708069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:46.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.016+0000 7f8728b9a700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:46.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.019+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f872404bb80 con 0x7f87240b16d0 2023-10-08T01:11:46.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.020+0000 7f87197fa700 1 -- 172.21.15.119:0/3367281223 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8720056680 con 0x7f87240b16d0 2023-10-08T01:11:46.021 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:11:46.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 msgr2=0x7f8708069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b16d0 msgr2=0x7f87240ca8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b16d0 0x7f87240ca8a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8724070dd0 tx=0x7f8720017000 comp rx=0 tx=0).stop 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 shutdown_connections 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f872415d240 0x7f8724068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b16d0 0x7f87240ca8a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 --2- 172.21.15.119:0/3367281223 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240b4e70 0x7f872414b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 >> 172.21.15.119:0/3367281223 conn(0x7f87240d10c0 msgr2=0x7f87241590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 shutdown_connections 2023-10-08T01:11:46.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.022+0000 7f872a39d700 1 -- 172.21.15.119:0/3367281223 wait complete. 2023-10-08T01:11:46.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T01:11:46.116 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:46.116+0000 7f67b3774180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:46.196 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:46.195+0000 7f9aba263180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:46.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.230+0000 7fad0b19d700 1 Processor -- start 2023-10-08T01:11:46.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.230+0000 7fad0b19d700 1 -- start start 2023-10-08T01:11:46.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.231+0000 7fad0b19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 0x7fad0415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.231+0000 7fad0b19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad040aef00 0x7fad0415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.231+0000 7fad0b19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fad0406e470 0x7fad0406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.231+0000 7fad0b19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad0406ee30 con 0x7fad040aef00 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.231+0000 7fad0b19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad040aea70 con 0x7fad0406e470 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.231+0000 7fad0b19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad040ae480 con 0x7fad0415a3b0 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad040aef00 0x7fad0415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fad040aef00 0x7fad0415d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44768/0 (socket says 172.21.15.119:44768) 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 -- 172.21.15.119:0/1955498281 learned_addr learned my addr 172.21.15.119:0/1955498281 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0a99c700 1 --2- 172.21.15.119:0/1955498281 >> v2:172.21.15.184:3300/0 conn(0x7fad0406e470 0x7fad0406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0a19b700 1 --2- 172.21.15.119:0/1955498281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 0x7fad0415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 -- 172.21.15.119:0/1955498281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 msgr2=0x7fad0415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:46.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 --2- 172.21.15.119:0/1955498281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 0x7fad0415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 -- 172.21.15.119:0/1955498281 >> v2:172.21.15.184:3300/0 conn(0x7fad0406e470 msgr2=0x7fad0406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:46.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 --2- 172.21.15.119:0/1955498281 >> v2:172.21.15.184:3300/0 conn(0x7fad0406e470 0x7fad0406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 -- 172.21.15.119:0/1955498281 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad0415acc0 con 0x7fad040aef00 2023-10-08T01:11:46.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.232+0000 7fad0999a700 1 --2- 172.21.15.119:0/1955498281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 0x7fad0415d3c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fad040c9470 tx=0x7facf4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5666c8bb92a8044 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:46.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad09199700 1 -- 172.21.15.119:0/1955498281 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400b640 con 0x7fad040aef00 2023-10-08T01:11:46.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad09199700 1 -- 172.21.15.119:0/1955498281 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facf400b7a0 con 0x7fad040aef00 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad09199700 1 -- 172.21.15.119:0/1955498281 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf400bb60 con 0x7fad040aef00 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 -- 172.21.15.119:0/1955498281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 msgr2=0x7fad0415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 --2- 172.21.15.119:0/1955498281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 0x7fad0415d3c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fad040c9470 tx=0x7facf4009250 comp rx=0 tx=0).stop 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 -- 172.21.15.119:0/1955498281 shutdown_connections 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 --2- 172.21.15.119:0/1955498281 >> v2:172.21.15.184:3300/0 conn(0x7fad0406e470 0x7fad0406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 --2- 172.21.15.119:0/1955498281 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 0x7fad0415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 --2- 172.21.15.119:0/1955498281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 0x7fad0415d3c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 -- 172.21.15.119:0/1955498281 >> 172.21.15.119:0/1955498281 conn(0x7fad040c9f70 msgr2=0x7fad0406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 -- 172.21.15.119:0/1955498281 shutdown_connections 2023-10-08T01:11:46.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.233+0000 7fad0b19d700 1 -- 172.21.15.119:0/1955498281 wait complete. 2023-10-08T01:11:46.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.235+0000 7fad0b19d700 1 Processor -- start 2023-10-08T01:11:46.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.235+0000 7fad0b19d700 1 -- start start 2023-10-08T01:11:46.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0b19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0406e470 0x7fad040c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0b19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 0x7fad040c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0b19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 0x7fad04152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0b19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad04158300 con 0x7fad040aef00 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0b19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad040c4ad0 con 0x7fad0406e470 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0b19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad040c4db0 con 0x7fad0415a3b0 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a19b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0406e470 0x7fad040c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 0x7fad040c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a19b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0406e470 0x7fad040c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58660/0 (socket says 172.21.15.119:58660) 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a19b700 1 -- 172.21.15.119:0/2481692192 learned_addr learned my addr 172.21.15.119:0/2481692192 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a99c700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 0x7fad04152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a19b700 1 -- 172.21.15.119:0/2481692192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 msgr2=0x7fad04152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a19b700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 0x7fad04152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a19b700 1 -- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 msgr2=0x7fad040c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a19b700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 0x7fad040c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.236+0000 7fad0a19b700 1 -- 172.21.15.119:0/2481692192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf40094f0 con 0x7fad0406e470 2023-10-08T01:11:46.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.237+0000 7fad0a19b700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0406e470 0x7fad040c44f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad0000e950 tx=0x7fad000091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:46.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.237+0000 7facfa7fc700 1 -- 172.21.15.119:0/2481692192 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00013680 con 0x7fad0406e470 2023-10-08T01:11:46.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.237+0000 7facfa7fc700 1 -- 172.21.15.119:0/2481692192 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad000137e0 con 0x7fad0406e470 2023-10-08T01:11:46.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.237+0000 7facfa7fc700 1 -- 172.21.15.119:0/2481692192 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad00013ba0 con 0x7fad0406e470 2023-10-08T01:11:46.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.237+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad040c5090 con 0x7fad0406e470 2023-10-08T01:11:46.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.237+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad040c69b0 con 0x7fad0406e470 2023-10-08T01:11:46.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.238+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad041580d0 con 0x7fad0406e470 2023-10-08T01:11:46.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.239+0000 7facfa7fc700 1 -- 172.21.15.119:0/2481692192 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fad000123d0 con 0x7fad0406e470 2023-10-08T01:11:46.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.240+0000 7facfa7fc700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7face80670e0 0x7face8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.240+0000 7facfa7fc700 1 -- 172.21.15.119:0/2481692192 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fad00084d80 con 0x7fad0406e470 2023-10-08T01:11:46.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.240+0000 7fad0999a700 1 -- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7face80670e0 msgr2=0x7face8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:46.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.240+0000 7fad0999a700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7face80670e0 0x7face8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:46.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.245+0000 7facfa7fc700 1 -- 172.21.15.119:0/2481692192 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad041580d0 con 0x7fad0406e470 2023-10-08T01:11:46.295 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:46.295+0000 7f67b3774180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:11:46.380 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:46.379+0000 7f9aba263180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:11:46.388 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:46.388+0000 7f67b3774180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:11:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.441+0000 7fad0999a700 1 -- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7face80670e0 msgr2=0x7face8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.441+0000 7fad0999a700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7face80670e0 0x7face8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:46.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.443+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fad041580d0 con 0x7fad0406e470 2023-10-08T01:11:46.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.444+0000 7facfa7fc700 1 -- 172.21.15.119:0/2481692192 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v78) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad041580d0 con 0x7fad0406e470 2023-10-08T01:11:46.445 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.446+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7face80670e0 msgr2=0x7face8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.446+0000 7fad0b19d700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7face80670e0 0x7face8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.446+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0406e470 msgr2=0x7fad040c44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.446+0000 7fad0b19d700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0406e470 0x7fad040c44f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fad0000e950 tx=0x7fad000091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.447+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 shutdown_connections 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.447+0000 7fad0b19d700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7face80670e0 0x7face8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.447+0000 7fad0b19d700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fad0415a3b0 0x7fad04152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.447+0000 7fad0b19d700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fad0406e470 0x7fad040c44f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.447+0000 7fad0b19d700 1 --2- 172.21.15.119:0/2481692192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad040aef00 0x7fad040c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.447+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 >> 172.21.15.119:0/2481692192 conn(0x7fad040c9f70 msgr2=0x7fad0415aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:46.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.447+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 shutdown_connections 2023-10-08T01:11:46.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.447+0000 7fad0b19d700 1 -- 172.21.15.119:0/2481692192 wait complete. 2023-10-08T01:11:46.466 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T01:11:46.476 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:46.475+0000 7f9aba263180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:11:46.551 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:46.551+0000 7f67b3774180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:11:46.646 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:46.645+0000 7f9aba263180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:11:46.651 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:46.651+0000 7f67b3774180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.655+0000 7f0bcbfff700 1 Processor -- start 2023-10-08T01:11:46.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.655+0000 7f0bcbfff700 1 -- start start 2023-10-08T01:11:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bcbfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b16d0 0x7f0bcc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bcbfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0bcc0b33b0 0x7f0bcc0b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bcbfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bcc0b2080 con 0x7f0bcc0b4e70 2023-10-08T01:11:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bcbfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bcc0cfb00 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bcbfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0b5c40 con 0x7f0bcc0b16d0 2023-10-08T01:11:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bca7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b16d0 0x7f0bcc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bcb7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0bcc0b33b0 0x7f0bcc0b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bcb7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0bcc0b33b0 0x7f0bcc0b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58662/0 (socket says 172.21.15.119:58662) 2023-10-08T01:11:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.657+0000 7f0bcb7fe700 1 -- 172.21.15.119:0/3663173259 learned_addr learned my addr 172.21.15.119:0/3663173259 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bcb7fe700 1 -- 172.21.15.119:0/3663173259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b16d0 msgr2=0x7f0bcc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bcb7fe700 1 --2- 172.21.15.119:0/3663173259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b16d0 0x7f0bcc0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bcb7fe700 1 -- 172.21.15.119:0/3663173259 >> v1:172.21.15.184:6789/0 conn(0x7f0bcc0b4e70 legacy=0x7f0bcc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bcb7fe700 1 -- 172.21.15.119:0/3663173259 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bcc0b5f20 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bca7fc700 1 --2- 172.21.15.119:0/3663173259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b16d0 0x7f0bcc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bcb7fe700 1 --2- 172.21.15.119:0/3663173259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc0b3830 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009f20 tx=0x7f0bc4009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=601f4e894e3e7971 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bc9ffb700 1 -- 172.21.15.119:0/3663173259 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400e070 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bc9ffb700 1 -- 172.21.15.119:0/3663173259 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc400b9d0 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.658+0000 7f0bc9ffb700 1 -- 172.21.15.119:0/3663173259 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc40092e0 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.659+0000 7f0bcbfff700 1 -- 172.21.15.119:0/3663173259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 msgr2=0x7f0bcc0b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.659+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/3663173259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc0b3830 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4009f20 tx=0x7f0bc4009840 comp rx=0 tx=0).stop 2023-10-08T01:11:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.659+0000 7f0bcbfff700 1 -- 172.21.15.119:0/3663173259 shutdown_connections 2023-10-08T01:11:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.659+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/3663173259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc0b3830 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.659+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/3663173259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b16d0 0x7f0bcc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.659+0000 7f0bcbfff700 1 -- 172.21.15.119:0/3663173259 >> 172.21.15.119:0/3663173259 conn(0x7f0bcc0d10c0 msgr2=0x7f0bcc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:46.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.659+0000 7f0bcbfff700 1 -- 172.21.15.119:0/3663173259 shutdown_connections 2023-10-08T01:11:46.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.659+0000 7f0bcbfff700 1 -- 172.21.15.119:0/3663173259 wait complete. 2023-10-08T01:11:46.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.660+0000 7f0bcbfff700 1 Processor -- start 2023-10-08T01:11:46.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.660+0000 7f0bcbfff700 1 -- start start 2023-10-08T01:11:46.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.661+0000 7f0bcbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bcc0b16d0 0x7f0bcc15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.661+0000 7f0bcbfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc16d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.661+0000 7f0bcbfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b4e70 0x7f0bcc16d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.661+0000 7f0bcbfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0c7e00 con 0x7f0bcc0b16d0 2023-10-08T01:11:46.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.661+0000 7f0bcbfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0c80e0 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.661+0000 7f0bcbfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc0c83c0 con 0x7f0bcc0b4e70 2023-10-08T01:11:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bcaffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bcc0b16d0 0x7f0bcc15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bca7fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc16d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bcaffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bcc0b16d0 0x7f0bcc15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44776/0 (socket says 172.21.15.119:44776) 2023-10-08T01:11:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bca7fc700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc16d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58670/0 (socket says 172.21.15.119:58670) 2023-10-08T01:11:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bcaffd700 1 -- 172.21.15.119:0/790390140 learned_addr learned my addr 172.21.15.119:0/790390140 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bcb7fe700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b4e70 0x7f0bcc16d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bca7fc700 1 -- 172.21.15.119:0/790390140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b4e70 msgr2=0x7f0bcc16d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bca7fc700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b4e70 0x7f0bcc16d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bca7fc700 1 -- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bcc0b16d0 msgr2=0x7f0bcc15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bca7fc700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bcc0b16d0 0x7f0bcc15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bca7fc700 1 -- 172.21.15.119:0/790390140 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc0009420 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.662+0000 7f0bca7fc700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc16d310 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000ec80 tx=0x7f0bc00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.663+0000 7f0bb2ffd700 1 -- 172.21.15.119:0/790390140 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000c850 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.663+0000 7f0bb2ffd700 1 -- 172.21.15.119:0/790390140 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bc000c9b0 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.663+0000 7f0bb2ffd700 1 -- 172.21.15.119:0/790390140 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000cd70 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.663+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc4009ae0 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.663+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc0c8980 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.664+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc088990 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.665+0000 7f0bb2ffd700 1 -- 172.21.15.119:0/790390140 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f0bc0013680 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.666+0000 7f0bb2ffd700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bb80670e0 0x7f0bb8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:46.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.666+0000 7f0bb2ffd700 1 -- 172.21.15.119:0/790390140 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f0bc0084d50 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.666+0000 7f0bcaffd700 1 -- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bb80670e0 msgr2=0x7f0bb8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:46.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.666+0000 7f0bcaffd700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bb80670e0 0x7f0bb8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:46.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.670+0000 7f0bb2ffd700 1 -- 172.21.15.119:0/790390140 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bc0051a10 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.751 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:46.750+0000 7f9aba263180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:11:46.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.867+0000 7f0bcaffd700 1 -- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bb80670e0 msgr2=0x7f0bb8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:46.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.867+0000 7f0bcaffd700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bb80670e0 0x7f0bb8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:46.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.867+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0bcc0c8220 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.868+0000 7f0bb2ffd700 1 -- 172.21.15.119:0/790390140 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v78) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0bcc0c8220 con 0x7f0bcc0b33b0 2023-10-08T01:11:46.869 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T01:11:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bb80670e0 msgr2=0x7f0bb8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bb80670e0 0x7f0bb8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 msgr2=0x7f0bcc16d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc16d310 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000ec80 tx=0x7f0bc00091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 shutdown_connections 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bb80670e0 0x7f0bb8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bcc0b4e70 0x7f0bcc16d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bcc0b16d0 0x7f0bcc15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 --2- 172.21.15.119:0/790390140 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bcc0b33b0 0x7f0bcc16d310 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 >> 172.21.15.119:0/790390140 conn(0x7f0bcc0d10c0 msgr2=0x7f0bcc158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:46.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 shutdown_connections 2023-10-08T01:11:46.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:46.870+0000 7f0bcbfff700 1 -- 172.21.15.119:0/790390140 wait complete. 2023-10-08T01:11:46.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:47.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.117+0000 7f6a6099d700 1 Processor -- start 2023-10-08T01:11:47.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.117+0000 7f6a6099d700 1 -- start start 2023-10-08T01:11:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.118+0000 7f6a6099d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c1595a0 0x7f6a5c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.118+0000 7f6a6099d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6a5c0c6c60 0x7f6a5c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.118+0000 7f6a6099d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c0b1410 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.118+0000 7f6a6099d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c0b2b90 con 0x7f6a5c0c6c60 2023-10-08T01:11:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.118+0000 7f6a6099d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c063230 con 0x7f6a5c1595a0 2023-10-08T01:11:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6a5c0b1de0 0x7f6a5c15d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60646/0 (socket says 172.21.15.119:60646) 2023-10-08T01:11:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5bfff700 1 -- 172.21.15.119:0/3851901730 learned_addr learned my addr 172.21.15.119:0/3851901730 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5b7fe700 1 --2- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c1595a0 0x7f6a5c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5affd700 1 --2- 172.21.15.119:0/3851901730 >> v2:172.21.15.184:3300/0 conn(0x7f6a5c0c6c60 0x7f6a5c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5a7fc700 1 -- 172.21.15.119:0/3851901730 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126921578 0 0) 0x7f6a5c0b1410 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5a7fc700 1 -- 172.21.15.119:0/3851901730 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a44003570 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5affd700 1 -- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c1595a0 msgr2=0x7f6a5c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:47.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5affd700 1 --2- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c1595a0 0x7f6a5c0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5affd700 1 -- 172.21.15.119:0/3851901730 >> v1:172.21.15.119:6789/0 conn(0x7f6a5c0b1de0 legacy=0x7f6a5c15d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5affd700 1 -- 172.21.15.119:0/3851901730 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0634f0 con 0x7f6a5c0c6c60 2023-10-08T01:11:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.119+0000 7f6a5b7fe700 1 --2- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c1595a0 0x7f6a5c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.120+0000 7f6a5affd700 1 --2- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0c6c60 0x7f6a5c0c8120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a500098e0 tx=0x7f6a500091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3efa2229b3ca220a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.120+0000 7f6a5a7fc700 1 -- 172.21.15.119:0/3851901730 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50011630 con 0x7f6a5c0c6c60 2023-10-08T01:11:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.120+0000 7f6a5a7fc700 1 -- 172.21.15.119:0/3851901730 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a50011790 con 0x7f6a5c0c6c60 2023-10-08T01:11:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.120+0000 7f6a5a7fc700 1 -- 172.21.15.119:0/3851901730 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a50011b30 con 0x7f6a5c0c6c60 2023-10-08T01:11:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.121+0000 7f6a6099d700 1 -- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0c6c60 msgr2=0x7f6a5c0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.121+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0c6c60 0x7f6a5c0c8120 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6a500098e0 tx=0x7f6a500091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.121+0000 7f6a6099d700 1 -- 172.21.15.119:0/3851901730 shutdown_connections 2023-10-08T01:11:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.121+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c1595a0 0x7f6a5c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.121+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3851901730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0c6c60 0x7f6a5c0c8120 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.121+0000 7f6a6099d700 1 -- 172.21.15.119:0/3851901730 >> 172.21.15.119:0/3851901730 conn(0x7f6a5c06ddb0 msgr2=0x7f6a5c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.121+0000 7f6a6099d700 1 -- 172.21.15.119:0/3851901730 shutdown_connections 2023-10-08T01:11:47.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.121+0000 7f6a6099d700 1 -- 172.21.15.119:0/3851901730 wait complete. 2023-10-08T01:11:47.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.122+0000 7f6a6099d700 1 Processor -- start 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.122+0000 7f6a6099d700 1 -- start start 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.123+0000 7f6a6099d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0b1de0 0x7f6a5c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.123+0000 7f6a6099d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c0c6c60 0x7f6a5c14e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.123+0000 7f6a6099d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c1595a0 0x7f6a5c13e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.123+0000 7f6a6099d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c15bda0 con 0x7f6a5c1595a0 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.123+0000 7f6a6099d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c15b3d0 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.123+0000 7f6a6099d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c15a7c0 con 0x7f6a5c0c6c60 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.123+0000 7f6a5bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c1595a0 0x7f6a5c13e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0b1de0 0x7f6a5c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0b1de0 0x7f6a5c15b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58696/0 (socket says 172.21.15.119:58696) 2023-10-08T01:11:47.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 -- 172.21.15.119:0/3635672677 learned_addr learned my addr 172.21.15.119:0/3635672677 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:47.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5affd700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c0c6c60 0x7f6a5c14e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 -- 172.21.15.119:0/3635672677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c0c6c60 msgr2=0x7f6a5c14e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c0c6c60 0x7f6a5c14e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 -- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c1595a0 msgr2=0x7f6a5c13e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c1595a0 0x7f6a5c13e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 -- 172.21.15.119:0/3635672677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c009420 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a5b7fe700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0b1de0 0x7f6a5c15b860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00ecb0 tx=0x7f6a4c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:47.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a437fe700 1 -- 172.21.15.119:0/3635672677 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00c850 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.124+0000 7f6a437fe700 1 -- 172.21.15.119:0/3635672677 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c00c9b0 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.125+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a50009420 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.125+0000 7f6a437fe700 1 -- 172.21.15.119:0/3635672677 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00cd30 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.125+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c15b060 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.126+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c04c3c0 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.127+0000 7f6a437fe700 1 -- 172.21.15.119:0/3635672677 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f6a4c013680 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.127+0000 7f6a437fe700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6a480670e0 0x7f6a48069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.128+0000 7f6a5affd700 1 -- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6a480670e0 msgr2=0x7f6a48069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.128+0000 7f6a437fe700 1 -- 172.21.15.119:0/3635672677 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f6a4c084e00 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.128+0000 7f6a5affd700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6a480670e0 0x7f6a48069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:47.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.132+0000 7f6a437fe700 1 -- 172.21.15.119:0/3635672677 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6a4c055550 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.138 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:47.138+0000 7f67b3774180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:11:47.232 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:47.232+0000 7f67b3774180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:11:47.256 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:47.255+0000 7f9aba263180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:11:47.261 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:11:47.266 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:11:47.325 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:47.325+0000 7f67b3774180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:11:47.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.329+0000 7f6a5affd700 1 -- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6a480670e0 msgr2=0x7f6a48069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:47.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.329+0000 7f6a5affd700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6a480670e0 0x7f6a48069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:47.355 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:47.354+0000 7f9aba263180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:11:47.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.378+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a5c0637d0 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.378+0000 7f6a437fe700 1 -- 172.21.15.119:0/3635672677 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f6a4c0550e0 con 0x7f6a5c0b1de0 2023-10-08T01:11:47.380 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:47.380 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4024548231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":4024548231},{"type":"v1","addr":"172.21.15.184:6835","nonce":4024548231}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1924614082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1924614082},{"type":"v1","addr":"172.21.15.119:6835","nonce":1924614082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5902,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/296234355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":296234355},{"type":"v1","addr":"172.21.15.184:6834","nonce":296234355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:11:21.183828+0000","modified":"2023-10-08T01:11:25.168981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6225":{"gid":6225,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6834/926130691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":926130691},{"type":"v1","addr":"172.21.15.119:6834","nonce":926130691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T01:11:47.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6a480670e0 msgr2=0x7f6a48069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:47.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6a480670e0 0x7f6a48069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0b1de0 msgr2=0x7f6a5c15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0b1de0 0x7f6a5c15b860 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00ecb0 tx=0x7f6a4c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 shutdown_connections 2023-10-08T01:11:47.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6a480670e0 0x7f6a48069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6a5c1595a0 0x7f6a5c13e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6a5c0b1de0 0x7f6a5c15b860 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 --2- 172.21.15.119:0/3635672677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6a5c0c6c60 0x7f6a5c14e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 >> 172.21.15.119:0/3635672677 conn(0x7f6a5c06ddb0 msgr2=0x7f6a5c14af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:47.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 shutdown_connections 2023-10-08T01:11:47.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.381+0000 7f6a6099d700 1 -- 172.21.15.119:0/3635672677 wait complete. 2023-10-08T01:11:47.385 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 32 2023-10-08T01:11:47.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:47.450 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:47.449+0000 7f9aba263180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:11:47.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.627+0000 7f775ffff700 1 Processor -- start 2023-10-08T01:11:47.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.628+0000 7f775ffff700 1 -- start start 2023-10-08T01:11:47.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77580b1420 0x7f77580ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77580ccf90 0x7f77580c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775ffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77580cc300 con 0x7f77580cbf30 2023-10-08T01:11:47.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775ffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7758156bf0 con 0x7f77580ccf90 2023-10-08T01:11:47.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775ffff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77580ae950 con 0x7f77580b1420 2023-10-08T01:11:47.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775e59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77580ccf90 0x7f77580c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775dd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f77580cbf30 0x7f77580b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60662/0 (socket says 172.21.15.119:60662) 2023-10-08T01:11:47.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775dd9b700 1 -- 172.21.15.119:0/299487797 learned_addr learned my addr 172.21.15.119:0/299487797 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:47.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.629+0000 7f775d59a700 1 --2- 172.21.15.119:0/299487797 >> v2:172.21.15.184:3300/0 conn(0x7f77580b1420 0x7f77580ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775e59c700 1 -- 172.21.15.119:0/299487797 >> v2:172.21.15.184:3300/0 conn(0x7f77580b1420 msgr2=0x7f77580ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775e59c700 1 --2- 172.21.15.119:0/299487797 >> v2:172.21.15.184:3300/0 conn(0x7f77580b1420 0x7f77580ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775e59c700 1 -- 172.21.15.119:0/299487797 >> v1:172.21.15.119:6789/0 conn(0x7f77580cbf30 legacy=0x7f77580b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775cd99700 1 -- 172.21.15.119:0/299487797 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2718853208 0 0) 0x7f77580cc300 con 0x7f77580cbf30 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775e59c700 1 -- 172.21.15.119:0/299487797 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580ad370 con 0x7f77580ccf90 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775e59c700 1 --2- 172.21.15.119:0/299487797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580ccf90 0x7f77580c9740 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f77580c9280 tx=0x7f7750009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67785e6a011d3b05 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775cd99700 1 -- 172.21.15.119:0/299487797 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000e070 con 0x7f77580ccf90 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775cd99700 1 -- 172.21.15.119:0/299487797 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775000bb20 con 0x7f77580ccf90 2023-10-08T01:11:47.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.630+0000 7f775cd99700 1 -- 172.21.15.119:0/299487797 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750012430 con 0x7f77580ccf90 2023-10-08T01:11:47.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.631+0000 7f775ffff700 1 -- 172.21.15.119:0/299487797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580ccf90 msgr2=0x7f77580c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.631+0000 7f775ffff700 1 --2- 172.21.15.119:0/299487797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580ccf90 0x7f77580c9740 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f77580c9280 tx=0x7f7750009780 comp rx=0 tx=0).stop 2023-10-08T01:11:47.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.631+0000 7f775ffff700 1 -- 172.21.15.119:0/299487797 shutdown_connections 2023-10-08T01:11:47.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.631+0000 7f775ffff700 1 --2- 172.21.15.119:0/299487797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580ccf90 0x7f77580c9740 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.631+0000 7f775ffff700 1 --2- 172.21.15.119:0/299487797 >> v2:172.21.15.184:3300/0 conn(0x7f77580b1420 0x7f77580ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.631+0000 7f775ffff700 1 -- 172.21.15.119:0/299487797 >> 172.21.15.119:0/299487797 conn(0x7f775806a320 msgr2=0x7f77580cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:47.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.631+0000 7f775ffff700 1 -- 172.21.15.119:0/299487797 shutdown_connections 2023-10-08T01:11:47.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.631+0000 7f775ffff700 1 -- 172.21.15.119:0/299487797 wait complete. 2023-10-08T01:11:47.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.633+0000 7f775ffff700 1 Processor -- start 2023-10-08T01:11:47.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.633+0000 7f775ffff700 1 -- start start 2023-10-08T01:11:47.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77580b1420 0x7f775816aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580cbf30 0x7f7758150100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77580ccf90 0x7f7758168f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775816a5a0 con 0x7f77580cbf30 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7758150640 con 0x7f77580ccf90 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7758169990 con 0x7f77580b1420 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775d59a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580cbf30 0x7f7758150100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775dd9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77580b1420 0x7f775816aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775e59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77580ccf90 0x7f7758168f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775e59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77580ccf90 0x7f7758168f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58714/0 (socket says 172.21.15.119:58714) 2023-10-08T01:11:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775e59c700 1 -- 172.21.15.119:0/3318329985 learned_addr learned my addr 172.21.15.119:0/3318329985 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775d59a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580cbf30 0x7f7758150100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44800/0 (socket says 172.21.15.119:44800) 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.634+0000 7f775dd9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77580b1420 0x7f775816aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:42824/0 (socket says 172.21.15.119:42824) 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f775e59c700 1 -- 172.21.15.119:0/3318329985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77580b1420 msgr2=0x7f775816aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f775e59c700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77580b1420 0x7f775816aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f775e59c700 1 -- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580cbf30 msgr2=0x7f7758150100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f775e59c700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580cbf30 0x7f7758150100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f775e59c700 1 -- 172.21.15.119:0/3318329985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750009a20 con 0x7f77580ccf90 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f775e59c700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77580ccf90 0x7f7758168f40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f775000b6c0 tx=0x7f7750019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f774dffb700 1 -- 172.21.15.119:0/3318329985 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7750026480 con 0x7f77580ccf90 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f774dffb700 1 -- 172.21.15.119:0/3318329985 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750026600 con 0x7f77580ccf90 2023-10-08T01:11:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f774dffb700 1 -- 172.21.15.119:0/3318329985 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77500269c0 con 0x7f77580ccf90 2023-10-08T01:11:47.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7758169c70 con 0x7f77580ccf90 2023-10-08T01:11:47.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.635+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775816a230 con 0x7f77580ccf90 2023-10-08T01:11:47.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.636+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775816a750 con 0x7f77580ccf90 2023-10-08T01:11:47.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.637+0000 7f774dffb700 1 -- 172.21.15.119:0/3318329985 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f775001c400 con 0x7f77580ccf90 2023-10-08T01:11:47.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.638+0000 7f774dffb700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f773c0670e0 0x7f773c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:47.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.638+0000 7f774dffb700 1 -- 172.21.15.119:0/3318329985 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f7750086f50 con 0x7f77580ccf90 2023-10-08T01:11:47.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.638+0000 7f775dd9b700 1 -- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f773c0670e0 msgr2=0x7f773c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:47.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.638+0000 7f775dd9b700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f773c0670e0 0x7f773c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:47.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.642+0000 7f774dffb700 1 -- 172.21.15.119:0/3318329985 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f775816a750 con 0x7f77580ccf90 2023-10-08T01:11:47.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.839+0000 7f775dd9b700 1 -- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f773c0670e0 msgr2=0x7f773c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:47.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.839+0000 7f775dd9b700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f773c0670e0 0x7f773c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:47.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.844+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77580cb600 con 0x7f77580ccf90 2023-10-08T01:11:47.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.846+0000 7f774dffb700 1 -- 172.21.15.119:0/3318329985 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f775002b090 con 0x7f77580ccf90 2023-10-08T01:11:47.847 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:47.847 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":78,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:42.316661+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:11:19.347044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T01:11:20.164667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/322083914":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/4063763211":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/2918246125":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/1513448491":"2023-10-09T01:11:42.316619+0000","172.21.15.119:6837/85511":"2023-10-09T01:11:42.316619+0000","172.21.15.119:6836/85511":"2023-10-09T01:11:42.316619+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:47.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f773c0670e0 msgr2=0x7f773c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f773c0670e0 0x7f773c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77580ccf90 msgr2=0x7f7758168f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77580ccf90 0x7f7758168f40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f775000b6c0 tx=0x7f7750019000 comp rx=0 tx=0).stop 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 shutdown_connections 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f773c0670e0 0x7f773c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77580b1420 0x7f775816aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77580ccf90 0x7f7758168f40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 --2- 172.21.15.119:0/3318329985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77580cbf30 0x7f7758150100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.848+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 >> 172.21.15.119:0/3318329985 conn(0x7f775806a320 msgr2=0x7f7758155d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:47.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.849+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 shutdown_connections 2023-10-08T01:11:47.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:47.849+0000 7f775ffff700 1 -- 172.21.15.119:0/3318329985 wait complete. 2023-10-08T01:11:47.868 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T01:11:47.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:47.948 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:47.948+0000 7f67b3774180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:11:48.098 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:48.097+0000 7f9aba263180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:11:48.119 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:48.119+0000 7f67b3774180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:48.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.130+0000 7f07cffff700 1 Processor -- start 2023-10-08T01:11:48.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.130+0000 7f07cffff700 1 -- start start 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.131+0000 7f07cffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159b30 0x7f07d00697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.131+0000 7f07cffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f07d006ba80 0x7f07d006bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.131+0000 7f07cffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07d00c8e20 con 0x7f07d0159f00 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.131+0000 7f07cffff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07d00c8c90 con 0x7f07d006ba80 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.131+0000 7f07cffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d006c440 con 0x7f07d0159b30 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07ce7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f07d0159f00 0x7f07d00ca7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60678/0 (socket says 172.21.15.119:60678) 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07ce7fc700 1 -- 172.21.15.119:0/2339418131 learned_addr learned my addr 172.21.15.119:0/2339418131 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07ceffd700 1 --2- 172.21.15.119:0/2339418131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159b30 0x7f07d00697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07cf7fe700 1 --2- 172.21.15.119:0/2339418131 >> v2:172.21.15.184:3300/0 conn(0x7f07d006ba80 0x7f07d006bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07cdffb700 1 -- 172.21.15.119:0/2339418131 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168630532 0 0) 0x7f07d00c8e20 con 0x7f07d0159f00 2023-10-08T01:11:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07cdffb700 1 -- 172.21.15.119:0/2339418131 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07b8003570 con 0x7f07d0159f00 2023-10-08T01:11:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07ceffd700 1 -- 172.21.15.119:0/2339418131 >> v2:172.21.15.184:3300/0 conn(0x7f07d006ba80 msgr2=0x7f07d006bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07ceffd700 1 --2- 172.21.15.119:0/2339418131 >> v2:172.21.15.184:3300/0 conn(0x7f07d006ba80 0x7f07d006bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07ceffd700 1 -- 172.21.15.119:0/2339418131 >> v1:172.21.15.119:6789/0 conn(0x7f07d0159f00 legacy=0x7f07d00ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07ceffd700 1 -- 172.21.15.119:0/2339418131 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d015a7c0 con 0x7f07d0159b30 2023-10-08T01:11:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.132+0000 7f07cf7fe700 1 --2- 172.21.15.119:0/2339418131 >> v2:172.21.15.184:3300/0 conn(0x7f07d006ba80 0x7f07d006bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.133+0000 7f07ceffd700 1 --2- 172.21.15.119:0/2339418131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159b30 0x7f07d00697f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07c00098e0 tx=0x7f07c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64645a26df2571d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.133+0000 7f07cdffb700 1 -- 172.21.15.119:0/2339418131 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0011630 con 0x7f07d0159b30 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.133+0000 7f07cdffb700 1 -- 172.21.15.119:0/2339418131 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c0011790 con 0x7f07d0159b30 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.133+0000 7f07cdffb700 1 -- 172.21.15.119:0/2339418131 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0011b30 con 0x7f07d0159b30 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.134+0000 7f07cffff700 1 -- 172.21.15.119:0/2339418131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159b30 msgr2=0x7f07d00697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.134+0000 7f07cffff700 1 --2- 172.21.15.119:0/2339418131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159b30 0x7f07d00697f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07c00098e0 tx=0x7f07c00091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.134+0000 7f07cffff700 1 -- 172.21.15.119:0/2339418131 shutdown_connections 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.134+0000 7f07cffff700 1 --2- 172.21.15.119:0/2339418131 >> v2:172.21.15.184:3300/0 conn(0x7f07d006ba80 0x7f07d006bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.134+0000 7f07cffff700 1 --2- 172.21.15.119:0/2339418131 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159b30 0x7f07d00697f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.134+0000 7f07cffff700 1 -- 172.21.15.119:0/2339418131 >> 172.21.15.119:0/2339418131 conn(0x7f07d00ce650 msgr2=0x7f07d00cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.134+0000 7f07cffff700 1 -- 172.21.15.119:0/2339418131 shutdown_connections 2023-10-08T01:11:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.134+0000 7f07cffff700 1 -- 172.21.15.119:0/2339418131 wait complete. 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.135+0000 7f07cffff700 1 Processor -- start 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.135+0000 7f07cffff700 1 -- start start 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07cffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d006ba80 0x7f07d014e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07cffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07d0159b30 0x7f07d01557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07cffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159f00 0x7f07d0155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07cffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d00c5c10 con 0x7f07d006ba80 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07cffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d00b3d00 con 0x7f07d0159b30 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07cffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d00b3fe0 con 0x7f07d0159f00 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07ceffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d006ba80 0x7f07d014e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07ceffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d006ba80 0x7f07d014e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44808/0 (socket says 172.21.15.119:44808) 2023-10-08T01:11:48.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.136+0000 7f07ceffd700 1 -- 172.21.15.119:0/144105241 learned_addr learned my addr 172.21.15.119:0/144105241 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07ce7fc700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07d0159b30 0x7f07d01557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07cf7fe700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159f00 0x7f07d0155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07ceffd700 1 -- 172.21.15.119:0/144105241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159f00 msgr2=0x7f07d0155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07ceffd700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159f00 0x7f07d0155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07ceffd700 1 -- 172.21.15.119:0/144105241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07d0159b30 msgr2=0x7f07d01557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07ceffd700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07d0159b30 0x7f07d01557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07ceffd700 1 -- 172.21.15.119:0/144105241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c8009a20 con 0x7f07d006ba80 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07ceffd700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d006ba80 0x7f07d014e300 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f07d00c8dd0 tx=0x7f07c0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07b6ffd700 1 -- 172.21.15.119:0/144105241 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0011b10 con 0x7f07d006ba80 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07b6ffd700 1 -- 172.21.15.119:0/144105241 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c00253f0 con 0x7f07d006ba80 2023-10-08T01:11:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c0009420 con 0x7f07d006ba80 2023-10-08T01:11:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.137+0000 7f07b6ffd700 1 -- 172.21.15.119:0/144105241 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c0025790 con 0x7f07d006ba80 2023-10-08T01:11:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.138+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07d014e840 con 0x7f07d006ba80 2023-10-08T01:11:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.138+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07d00b4220 con 0x7f07d006ba80 2023-10-08T01:11:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.139+0000 7f07b6ffd700 1 -- 172.21.15.119:0/144105241 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f07c0025b80 con 0x7f07d006ba80 2023-10-08T01:11:48.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.140+0000 7f07b6ffd700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f07bc0670e0 0x7f07bc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.140+0000 7f07ce7fc700 1 -- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f07bc0670e0 msgr2=0x7f07bc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:48.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.140+0000 7f07b6ffd700 1 -- 172.21.15.119:0/144105241 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f07c0086400 con 0x7f07d006ba80 2023-10-08T01:11:48.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.140+0000 7f07ce7fc700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f07bc0670e0 0x7f07bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.149+0000 7f07b6ffd700 1 -- 172.21.15.119:0/144105241 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07d00b4220 con 0x7f07d006ba80 2023-10-08T01:11:48.216 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:48.216+0000 7f67b3774180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:11:48.281 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:48.280+0000 7f9aba263180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:11:48.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.341+0000 7f07ce7fc700 1 -- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f07bc0670e0 msgr2=0x7f07bc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:48.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.341+0000 7f07ce7fc700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f07bc0670e0 0x7f07bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:48.378 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:48.377+0000 7f9aba263180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:11:48.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.393+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07d00c56e0 con 0x7f07d006ba80 2023-10-08T01:11:48.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.393+0000 7f07b6ffd700 1 -- 172.21.15.119:0/144105241 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4603 (secure 0 0 0) 0x7f07c00530c0 con 0x7f07d006ba80 2023-10-08T01:11:48.395 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:48.395 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4024548231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":4024548231},{"type":"v1","addr":"172.21.15.184:6835","nonce":4024548231}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1924614082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1924614082},{"type":"v1","addr":"172.21.15.119:6835","nonce":1924614082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5902,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/296234355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":296234355},{"type":"v1","addr":"172.21.15.184:6834","nonce":296234355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:11:21.183828+0000","modified":"2023-10-08T01:11:25.168981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6225":{"gid":6225,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6834/926130691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":926130691},{"type":"v1","addr":"172.21.15.119:6834","nonce":926130691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T01:11:48.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f07bc0670e0 msgr2=0x7f07bc069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:48.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f07bc0670e0 0x7f07bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d006ba80 msgr2=0x7f07d014e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d006ba80 0x7f07d014e300 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f07d00c8dd0 tx=0x7f07c0017000 comp rx=0 tx=0).stop 2023-10-08T01:11:48.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 shutdown_connections 2023-10-08T01:11:48.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f07bc0670e0 0x7f07bc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07d0159f00 0x7f07d0155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07d006ba80 0x7f07d014e300 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 --2- 172.21.15.119:0/144105241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07d0159b30 0x7f07d01557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 >> 172.21.15.119:0/144105241 conn(0x7f07d00ce650 msgr2=0x7f07d006db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:48.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 shutdown_connections 2023-10-08T01:11:48.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.396+0000 7f07cffff700 1 -- 172.21.15.119:0/144105241 wait complete. 2023-10-08T01:11:48.399 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 32 2023-10-08T01:11:48.417 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:48.512 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:48.511+0000 7f67b3774180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:11:48.606 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:48.605+0000 7f67b3774180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:11:48.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.607+0000 7fb7cf59e700 1 Processor -- start 2023-10-08T01:11:48.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.607+0000 7fb7cf59e700 1 -- start start 2023-10-08T01:11:48.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.608+0000 7fb7cf59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d01595a0 0x7fb7d00b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.608+0000 7fb7cf59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7d00c8830 0x7fb7d0159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.608+0000 7fb7cf59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7d00b1410 con 0x7fb7d00c6c60 2023-10-08T01:11:48.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.608+0000 7fb7cf59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7d00b2b90 con 0x7fb7d00c8830 2023-10-08T01:11:48.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.608+0000 7fb7cf59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d0063230 con 0x7fb7d01595a0 2023-10-08T01:11:48.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.608+0000 7fb7ced9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7d00c8830 0x7fb7d0159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.608+0000 7fb7ce59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d01595a0 0x7fb7d00b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ce59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d01595a0 0x7fb7d00b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:42850/0 (socket says 172.21.15.119:42850) 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ced9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7d00c8830 0x7fb7d0159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:58748/0 (socket says 172.21.15.119:58748) 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ce59c700 1 -- 172.21.15.119:0/3863774520 learned_addr learned my addr 172.21.15.119:0/3863774520 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ce59c700 1 -- 172.21.15.119:0/3863774520 >> v2:172.21.15.184:3300/0 conn(0x7fb7d00c8830 msgr2=0x7fb7d0159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ce59c700 1 --2- 172.21.15.119:0/3863774520 >> v2:172.21.15.184:3300/0 conn(0x7fb7d00c8830 0x7fb7d0159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ce59c700 1 -- 172.21.15.119:0/3863774520 >> v1:172.21.15.184:6789/0 conn(0x7fb7d00c6c60 legacy=0x7fb7d00c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ce59c700 1 -- 172.21.15.119:0/3863774520 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d00634f0 con 0x7fb7d01595a0 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ced9d700 1 --2- 172.21.15.119:0/3863774520 >> v2:172.21.15.184:3300/0 conn(0x7fb7d00c8830 0x7fb7d0159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:48.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.609+0000 7fb7ce59c700 1 --2- 172.21.15.119:0/3863774520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d01595a0 0x7fb7d00b18a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00099b0 tx=0x7fb7c0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dc53e49a9e55696 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cd59a700 1 -- 172.21.15.119:0/3863774520 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0009ea0 con 0x7fb7d01595a0 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cd59a700 1 -- 172.21.15.119:0/3863774520 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c000b860 con 0x7fb7d01595a0 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cd59a700 1 -- 172.21.15.119:0/3863774520 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c000bc00 con 0x7fb7d01595a0 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cf59e700 1 -- 172.21.15.119:0/3863774520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d01595a0 msgr2=0x7fb7d00b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/3863774520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d01595a0 0x7fb7d00b18a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb7c00099b0 tx=0x7fb7c0009250 comp rx=0 tx=0).stop 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cf59e700 1 -- 172.21.15.119:0/3863774520 shutdown_connections 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/3863774520 >> v2:172.21.15.184:3300/0 conn(0x7fb7d00c8830 0x7fb7d0159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/3863774520 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d01595a0 0x7fb7d00b18a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cf59e700 1 -- 172.21.15.119:0/3863774520 >> 172.21.15.119:0/3863774520 conn(0x7fb7d006ddb0 msgr2=0x7fb7d006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cf59e700 1 -- 172.21.15.119:0/3863774520 shutdown_connections 2023-10-08T01:11:48.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.610+0000 7fb7cf59e700 1 -- 172.21.15.119:0/3863774520 wait complete. 2023-10-08T01:11:48.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.612+0000 7fb7cf59e700 1 Processor -- start 2023-10-08T01:11:48.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.612+0000 7fb7cf59e700 1 -- start start 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7cf59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d00c6c60 0x7fb7d0151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7cf59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d00c8830 0x7fb7d016ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7cf59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7d01595a0 0x7fb7d00c5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7cf59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d014c740 con 0x7fb7d01595a0 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7cf59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d01517d0 con 0x7fb7d00c6c60 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7cf59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d0151ab0 con 0x7fb7d00c8830 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7ced9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7d01595a0 0x7fb7d00c5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7ce59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d00c6c60 0x7fb7d0151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7ced9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7d01595a0 0x7fb7d00c5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44814/0 (socket says 172.21.15.119:44814) 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7ced9d700 1 -- 172.21.15.119:0/1874455205 learned_addr learned my addr 172.21.15.119:0/1874455205 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.613+0000 7fb7cdd9b700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d00c8830 0x7fb7d016ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7ce59c700 1 -- 172.21.15.119:0/1874455205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d00c8830 msgr2=0x7fb7d016ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7ce59c700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d00c8830 0x7fb7d016ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7ce59c700 1 -- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7d01595a0 msgr2=0x7fb7d00c5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7ce59c700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7d01595a0 0x7fb7d00c5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7ce59c700 1 -- 172.21.15.119:0/1874455205 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c8009a20 con 0x7fb7d00c6c60 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7ce59c700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d00c6c60 0x7fb7d0151130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb7d006d4e0 tx=0x7fb7c0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7be7fc700 1 -- 172.21.15.119:0/1874455205 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0000d10 con 0x7fb7d00c6c60 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7be7fc700 1 -- 172.21.15.119:0/1874455205 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7c000bbe0 con 0x7fb7d00c6c60 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c00094f0 con 0x7fb7d00c6c60 2023-10-08T01:11:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.614+0000 7fb7be7fc700 1 -- 172.21.15.119:0/1874455205 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7c0025670 con 0x7fb7d00c6c60 2023-10-08T01:11:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.615+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d0152070 con 0x7fb7d00c6c60 2023-10-08T01:11:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.615+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d004c3c0 con 0x7fb7d00c6c60 2023-10-08T01:11:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.617+0000 7fb7be7fc700 1 -- 172.21.15.119:0/1874455205 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fb7c0025a20 con 0x7fb7d00c6c60 2023-10-08T01:11:48.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.618+0000 7fb7be7fc700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb7b00670e0 0x7fb7b0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:48.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.618+0000 7fb7be7fc700 1 -- 172.21.15.119:0/1874455205 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fb7c0086840 con 0x7fb7d00c6c60 2023-10-08T01:11:48.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.618+0000 7fb7cdd9b700 1 -- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb7b00670e0 msgr2=0x7fb7b0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:48.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.618+0000 7fb7cdd9b700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb7b00670e0 0x7fb7b0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.622+0000 7fb7be7fc700 1 -- 172.21.15.119:0/1874455205 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7c0053580 con 0x7fb7d00c6c60 2023-10-08T01:11:48.690 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:48.689+0000 7f9aba263180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:11:48.789 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:48.788+0000 7f9aba263180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:11:48.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.819+0000 7fb7cdd9b700 1 -- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb7b00670e0 msgr2=0x7fb7b0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:48.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.819+0000 7fb7cdd9b700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb7b00670e0 0x7fb7b0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:48.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.867+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7d0151bf0 con 0x7fb7d00c6c60 2023-10-08T01:11:48.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.867+0000 7fb7be7fc700 1 -- 172.21.15.119:0/1874455205 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4603 (secure 0 0 0) 0x7fb7d0151bf0 con 0x7fb7d00c6c60 2023-10-08T01:11:48.868 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:48.869 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":32,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4024548231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":4024548231},{"type":"v1","addr":"172.21.15.184:6835","nonce":4024548231}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1924614082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1924614082},{"type":"v1","addr":"172.21.15.119:6835","nonce":1924614082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5902,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/296234355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":296234355},{"type":"v1","addr":"172.21.15.184:6834","nonce":296234355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:11:21.183828+0000","modified":"2023-10-08T01:11:25.168981+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6225":{"gid":6225,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6834/926130691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":926130691},{"type":"v1","addr":"172.21.15.119:6834","nonce":926130691}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-10-08T01:11:48.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb7b00670e0 msgr2=0x7fb7b0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:48.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb7b00670e0 0x7fb7b0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d00c6c60 msgr2=0x7fb7d0151130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:48.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d00c6c60 0x7fb7d0151130 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb7d006d4e0 tx=0x7fb7c0017000 comp rx=0 tx=0).stop 2023-10-08T01:11:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 shutdown_connections 2023-10-08T01:11:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb7b00670e0 0x7fb7b0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7d01595a0 0x7fb7d00c5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7d00c6c60 0x7fb7d0151130 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 --2- 172.21.15.119:0/1874455205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7d00c8830 0x7fb7d016ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 >> 172.21.15.119:0/1874455205 conn(0x7fb7d006ddb0 msgr2=0x7fb7d015c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 shutdown_connections 2023-10-08T01:11:48.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:48.870+0000 7fb7cf59e700 1 -- 172.21.15.119:0/1874455205 wait complete. 2023-10-08T01:11:48.873 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 32 2023-10-08T01:11:48.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:49.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.076+0000 7f716539d700 1 Processor -- start 2023-10-08T01:11:49.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.077+0000 7f716539d700 1 -- start start 2023-10-08T01:11:49.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.078+0000 7f716539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71601597f0 0x7f71600b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.078+0000 7f716539d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71600b11d0 con 0x7f716015d230 2023-10-08T01:11:49.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.078+0000 7f716539d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71600b14b0 con 0x7f7160159420 2023-10-08T01:11:49.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.078+0000 7f716539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71600b1790 con 0x7f71601597f0 2023-10-08T01:11:49.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.078+0000 7f7164b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f716015d230 0x7f71600b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60694/0 (socket says 172.21.15.119:60694) 2023-10-08T01:11:49.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.078+0000 7f7164b9c700 1 -- 172.21.15.119:0/1300530533 learned_addr learned my addr 172.21.15.119:0/1300530533 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:49.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.078+0000 7f715f7fe700 1 --2- 172.21.15.119:0/1300530533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71601597f0 0x7f71600b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715effd700 1 -- 172.21.15.119:0/1300530533 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930086286 0 0) 0x7f71600b11d0 con 0x7f716015d230 2023-10-08T01:11:49.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715effd700 1 -- 172.21.15.119:0/1300530533 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7148003570 con 0x7f716015d230 2023-10-08T01:11:49.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715effd700 1 -- 172.21.15.119:0/1300530533 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455048813 0 0) 0x7f71600b14b0 con 0x7f7160159420 2023-10-08T01:11:49.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715effd700 1 -- 172.21.15.119:0/1300530533 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71600b11d0 con 0x7f7160159420 2023-10-08T01:11:49.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715f7fe700 1 -- 172.21.15.119:0/1300530533 >> v1:172.21.15.184:6789/0 conn(0x7f7160159420 legacy=0x7f71600b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715f7fe700 1 -- 172.21.15.119:0/1300530533 >> v1:172.21.15.119:6789/0 conn(0x7f716015d230 legacy=0x7f71600b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715f7fe700 1 -- 172.21.15.119:0/1300530533 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71600b1a70 con 0x7f71601597f0 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715f7fe700 1 --2- 172.21.15.119:0/1300530533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71601597f0 0x7f71600b2270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7154009910 tx=0x7f71540091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=896fa378daee847f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715effd700 1 -- 172.21.15.119:0/1300530533 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154011630 con 0x7f71601597f0 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715effd700 1 -- 172.21.15.119:0/1300530533 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154011790 con 0x7f71601597f0 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.079+0000 7f715effd700 1 -- 172.21.15.119:0/1300530533 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154011b30 con 0x7f71601597f0 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.080+0000 7f716539d700 1 -- 172.21.15.119:0/1300530533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71601597f0 msgr2=0x7f71600b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.080+0000 7f716539d700 1 --2- 172.21.15.119:0/1300530533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71601597f0 0x7f71600b2270 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7154009910 tx=0x7f71540091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.080+0000 7f716539d700 1 -- 172.21.15.119:0/1300530533 shutdown_connections 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.080+0000 7f716539d700 1 --2- 172.21.15.119:0/1300530533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71601597f0 0x7f71600b2270 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.080+0000 7f716539d700 1 -- 172.21.15.119:0/1300530533 >> 172.21.15.119:0/1300530533 conn(0x7f71600d0660 msgr2=0x7f71600d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:49.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.080+0000 7f716539d700 1 -- 172.21.15.119:0/1300530533 shutdown_connections 2023-10-08T01:11:49.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.080+0000 7f716539d700 1 -- 172.21.15.119:0/1300530533 wait complete. 2023-10-08T01:11:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.081+0000 7f716539d700 1 Processor -- start 2023-10-08T01:11:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.082+0000 7f716539d700 1 -- start start 2023-10-08T01:11:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.082+0000 7f716539d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160159420 0x7f7160155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.082+0000 7f716539d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71601597f0 0x7f7160068d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.082+0000 7f716539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f716015d230 0x7f7160069250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.082+0000 7f716539d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716015c420 con 0x7f71601597f0 2023-10-08T01:11:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.082+0000 7f716539d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7160155a40 con 0x7f7160159420 2023-10-08T01:11:49.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.082+0000 7f716539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71600c61e0 con 0x7f716015d230 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71601597f0 0x7f7160068d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71601597f0 0x7f7160068d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44824/0 (socket says 172.21.15.119:44824) 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 -- 172.21.15.119:0/667033961 learned_addr learned my addr 172.21.15.119:0/667033961 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715ffff700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160159420 0x7f7160155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f7164b9c700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f716015d230 0x7f7160069250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 -- 172.21.15.119:0/667033961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f716015d230 msgr2=0x7f7160069250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f716015d230 0x7f7160069250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 -- 172.21.15.119:0/667033961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160159420 msgr2=0x7f7160155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160159420 0x7f7160155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 -- 172.21.15.119:0/667033961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154009420 con 0x7f71601597f0 2023-10-08T01:11:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.083+0000 7f715f7fe700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71601597f0 0x7f7160068d10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7154009f00 tx=0x7f7154017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.084+0000 7f7147fff700 1 -- 172.21.15.119:0/667033961 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715401f070 con 0x7f71601597f0 2023-10-08T01:11:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.084+0000 7f7147fff700 1 -- 172.21.15.119:0/667033961 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154000bc0 con 0x7f71601597f0 2023-10-08T01:11:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.084+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716015baf0 con 0x7f71601597f0 2023-10-08T01:11:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.084+0000 7f7147fff700 1 -- 172.21.15.119:0/667033961 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154025420 con 0x7f71601597f0 2023-10-08T01:11:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.084+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716015c0b0 con 0x7f71601597f0 2023-10-08T01:11:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.085+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716004c3c0 con 0x7f71601597f0 2023-10-08T01:11:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.086+0000 7f7147fff700 1 -- 172.21.15.119:0/667033961 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f7154023070 con 0x7f71601597f0 2023-10-08T01:11:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.086+0000 7f7147fff700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f714c0670e0 0x7f714c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.087+0000 7f7147fff700 1 -- 172.21.15.119:0/667033961 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f7154085ef0 con 0x7f71601597f0 2023-10-08T01:11:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.087+0000 7f715ffff700 1 -- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f714c0670e0 msgr2=0x7f714c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.087+0000 7f715ffff700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f714c0670e0 0x7f714c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:49.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.095+0000 7f7147fff700 1 -- 172.21.15.119:0/667033961 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7154052bb0 con 0x7f71601597f0 2023-10-08T01:11:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.288+0000 7f715ffff700 1 -- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f714c0670e0 msgr2=0x7f714c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:49.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.288+0000 7f715ffff700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f714c0670e0 0x7f714c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:49.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.296+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f716015c560 con 0x7f71601597f0 2023-10-08T01:11:49.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.297+0000 7f7147fff700 1 -- 172.21.15.119:0/667033961 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16367 (secure 0 0 0) 0x7f715402a090 con 0x7f71601597f0 2023-10-08T01:11:49.298 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:49.299 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":78,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:42.316661+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:11:19.347044+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-10-08T01:11:20.164667+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/322083914":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/4063763211":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/2918246125":"2023-10-09T01:11:42.316619+0000","172.21.15.119:0/1513448491":"2023-10-09T01:11:42.316619+0000","172.21.15.119:6837/85511":"2023-10-09T01:11:42.316619+0000","172.21.15.119:6836/85511":"2023-10-09T01:11:42.316619+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:49.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f714c0670e0 msgr2=0x7f714c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:49.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f714c0670e0 0x7f714c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71601597f0 msgr2=0x7f7160068d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71601597f0 0x7f7160068d10 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7154009f00 tx=0x7f7154017000 comp rx=0 tx=0).stop 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 shutdown_connections 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f714c0670e0 0x7f714c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f716015d230 0x7f7160069250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7160159420 0x7f7160155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 --2- 172.21.15.119:0/667033961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71601597f0 0x7f7160068d10 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 >> 172.21.15.119:0/667033961 conn(0x7f71600d0660 msgr2=0x7f71601582e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 shutdown_connections 2023-10-08T01:11:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.300+0000 7f716539d700 1 -- 172.21.15.119:0/667033961 wait complete. 2023-10-08T01:11:49.315 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:49.314+0000 7f67b3774180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:11:49.319 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T01:11:49.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.505+0000 7f332a39d700 1 Processor -- start 2023-10-08T01:11:49.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.505+0000 7f332a39d700 1 -- start start 2023-10-08T01:11:49.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.506+0000 7f332a39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324159420 0x7f33240b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.506+0000 7f332a39d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f33241597f0 0x7f33240b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.506+0000 7f332a39d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33240b11d0 con 0x7f332415d230 2023-10-08T01:11:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.506+0000 7f332a39d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33240b14b0 con 0x7f33241597f0 2023-10-08T01:11:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.506+0000 7f332a39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33240b1790 con 0x7f3324159420 2023-10-08T01:11:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f3329b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f332415d230 0x7f33240b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60708/0 (socket says 172.21.15.119:60708) 2023-10-08T01:11:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f3329b9c700 1 -- 172.21.15.119:0/3951485797 learned_addr learned my addr 172.21.15.119:0/3951485797 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f332939b700 1 --2- 172.21.15.119:0/3951485797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324159420 0x7f33240b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f3328b9a700 1 --2- 172.21.15.119:0/3951485797 >> v2:172.21.15.184:3300/0 conn(0x7f33241597f0 0x7f33240b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f331bfff700 1 -- 172.21.15.119:0/3951485797 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470274767 0 0) 0x7f33240b11d0 con 0x7f332415d230 2023-10-08T01:11:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f331bfff700 1 -- 172.21.15.119:0/3951485797 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3310003570 con 0x7f332415d230 2023-10-08T01:11:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f332939b700 1 -- 172.21.15.119:0/3951485797 >> v2:172.21.15.184:3300/0 conn(0x7f33241597f0 msgr2=0x7f33240b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f332939b700 1 --2- 172.21.15.119:0/3951485797 >> v2:172.21.15.184:3300/0 conn(0x7f33241597f0 0x7f33240b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f332939b700 1 -- 172.21.15.119:0/3951485797 >> v1:172.21.15.119:6789/0 conn(0x7f332415d230 legacy=0x7f33240b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.507+0000 7f332939b700 1 -- 172.21.15.119:0/3951485797 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33240b1a70 con 0x7f3324159420 2023-10-08T01:11:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.508+0000 7f332939b700 1 --2- 172.21.15.119:0/3951485797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324159420 0x7f33240b5f40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33200098e0 tx=0x7f33200091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0d72cba9a190f95 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.508+0000 7f331bfff700 1 -- 172.21.15.119:0/3951485797 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320011630 con 0x7f3324159420 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.508+0000 7f331bfff700 1 -- 172.21.15.119:0/3951485797 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3320011790 con 0x7f3324159420 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.508+0000 7f331bfff700 1 -- 172.21.15.119:0/3951485797 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320011b30 con 0x7f3324159420 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.508+0000 7f332a39d700 1 -- 172.21.15.119:0/3951485797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324159420 msgr2=0x7f33240b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.508+0000 7f332a39d700 1 --2- 172.21.15.119:0/3951485797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324159420 0x7f33240b5f40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f33200098e0 tx=0x7f33200091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.508+0000 7f332a39d700 1 -- 172.21.15.119:0/3951485797 shutdown_connections 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.509+0000 7f332a39d700 1 --2- 172.21.15.119:0/3951485797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324159420 0x7f33240b5f40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.509+0000 7f332a39d700 1 --2- 172.21.15.119:0/3951485797 >> v2:172.21.15.184:3300/0 conn(0x7f33241597f0 0x7f33240b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.509+0000 7f332a39d700 1 -- 172.21.15.119:0/3951485797 >> 172.21.15.119:0/3951485797 conn(0x7f33240d0660 msgr2=0x7f33240d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.509+0000 7f332a39d700 1 -- 172.21.15.119:0/3951485797 shutdown_connections 2023-10-08T01:11:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.509+0000 7f332a39d700 1 -- 172.21.15.119:0/3951485797 wait complete. 2023-10-08T01:11:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.510+0000 7f332a39d700 1 Processor -- start 2023-10-08T01:11:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.510+0000 7f332a39d700 1 -- start start 2023-10-08T01:11:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332a39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3324159420 0x7f332415bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332a39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33241597f0 0x7f332416aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332a39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332415d230 0x7f332416af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332a39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332415b930 con 0x7f3324159420 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332a39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332415ad60 con 0x7f33241597f0 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332a39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332415b000 con 0x7f332415d230 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3324159420 0x7f332415bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3324159420 0x7f332415bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44832/0 (socket says 172.21.15.119:44832) 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f332939b700 1 -- 172.21.15.119:0/2604485684 learned_addr learned my addr 172.21.15.119:0/2604485684 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f3328b9a700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33241597f0 0x7f332416aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.511+0000 7f3329b9c700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332415d230 0x7f332416af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f332939b700 1 -- 172.21.15.119:0/2604485684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332415d230 msgr2=0x7f332416af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f332939b700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332415d230 0x7f332416af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f332939b700 1 -- 172.21.15.119:0/2604485684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33241597f0 msgr2=0x7f332416aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f332939b700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33241597f0 0x7f332416aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f332939b700 1 -- 172.21.15.119:0/2604485684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3320009420 con 0x7f3324159420 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f332939b700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3324159420 0x7f332415bdc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f33240b18d0 tx=0x7f3320017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f33197fa700 1 -- 172.21.15.119:0/2604485684 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320011900 con 0x7f3324159420 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f33197fa700 1 -- 172.21.15.119:0/2604485684 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3320011a60 con 0x7f3324159420 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.512+0000 7f33197fa700 1 -- 172.21.15.119:0/2604485684 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3320025640 con 0x7f3324159420 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.513+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332415b2e0 con 0x7f3324159420 2023-10-08T01:11:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.513+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332415c300 con 0x7f3324159420 2023-10-08T01:11:49.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.514+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332415c5e0 con 0x7f3324159420 2023-10-08T01:11:49.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.515+0000 7f33197fa700 1 -- 172.21.15.119:0/2604485684 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f3320025a30 con 0x7f3324159420 2023-10-08T01:11:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.516+0000 7f33197fa700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f33080670e0 0x7f3308069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.516+0000 7f33197fa700 1 -- 172.21.15.119:0/2604485684 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f3320086270 con 0x7f3324159420 2023-10-08T01:11:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.516+0000 7f3328b9a700 1 -- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f33080670e0 msgr2=0x7f3308069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.516+0000 7f3328b9a700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f33080670e0 0x7f3308069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:49.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.521+0000 7f33197fa700 1 -- 172.21.15.119:0/2604485684 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3320052f30 con 0x7f3324159420 2023-10-08T01:11:49.535 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:49.535+0000 7f9aba263180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:11:49.606 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:49.604+0000 7f67b3774180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:11:49.698 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:49.697+0000 7f67b3774180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:11:49.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.717+0000 7f3328b9a700 1 -- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f33080670e0 msgr2=0x7f3308069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:49.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.717+0000 7f3328b9a700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f33080670e0 0x7f3308069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:49.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.765+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f332404c3c0 con 0x7f3324159420 2023-10-08T01:11:49.767 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T01:11:49.766+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T01:11:49.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.770+0000 7f33197fa700 1 -- 172.21.15.119:0/2604485684 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3320056550 con 0x7f3324159420 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.772+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f33080670e0 msgr2=0x7f3308069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.772+0000 7f332a39d700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f33080670e0 0x7f3308069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.772+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3324159420 msgr2=0x7f332415bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.772+0000 7f332a39d700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3324159420 0x7f332415bdc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f33240b18d0 tx=0x7f3320017000 comp rx=0 tx=0).stop 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.773+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 shutdown_connections 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.773+0000 7f332a39d700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f33080670e0 0x7f3308069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.773+0000 7f332a39d700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332415d230 0x7f332416af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.773+0000 7f332a39d700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3324159420 0x7f332415bdc0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.773+0000 7f332a39d700 1 --2- 172.21.15.119:0/2604485684 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33241597f0 0x7f332416aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.773+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 >> 172.21.15.119:0/2604485684 conn(0x7f33240d0660 msgr2=0x7f33241573e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:49.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.773+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 shutdown_connections 2023-10-08T01:11:49.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.773+0000 7f332a39d700 1 -- 172.21.15.119:0/2604485684 wait complete. 2023-10-08T01:11:49.775 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T01:11:49.777 INFO:tasks.ceph.mds.a.smithi119.stderr: -2240> 2023-10-08T01:11:26.693+0000 7f9ce7ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:11:49.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T01:11:49.821 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:11:49.838 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:49.837+0000 7f9aba263180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:11:49.855 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:49.854+0000 7f67b3774180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:11:49.935 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:49.934+0000 7f9aba263180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:11:49.959 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:49.958+0000 7f67b3774180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:11:49.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.985+0000 7fdf7b59e700 1 Processor -- start 2023-10-08T01:11:49.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.985+0000 7fdf7b59e700 1 -- start start 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.986+0000 7fdf7b59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdf7c15e070 0x7fdf7c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.986+0000 7fdf7b59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c15ab70 con 0x7fdf7c157150 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.986+0000 7fdf7b59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c15d590 con 0x7fdf7c06eb90 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.986+0000 7fdf7b59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7c0ad1b0 con 0x7fdf7c15e070 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.986+0000 7fdf7a59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdf7c157150 0x7fdf7c06e480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60728/0 (socket says 172.21.15.119:60728) 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.986+0000 7fdf7a59c700 1 -- 172.21.15.119:0/2062295787 learned_addr learned my addr 172.21.15.119:0/2062295787 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.986+0000 7fdf7ad9d700 1 --2- 172.21.15.119:0/2062295787 >> v2:172.21.15.184:3300/0 conn(0x7fdf7c15e070 0x7fdf7c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.987+0000 7fdf7959a700 1 -- 172.21.15.119:0/2062295787 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668020938 0 0) 0x7fdf7c15ab70 con 0x7fdf7c157150 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.987+0000 7fdf7959a700 1 -- 172.21.15.119:0/2062295787 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf64003570 con 0x7fdf7c157150 2023-10-08T01:11:49.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.987+0000 7fdf7ad9d700 1 -- 172.21.15.119:0/2062295787 >> v1:172.21.15.184:6789/0 conn(0x7fdf7c06eb90 legacy=0x7fdf7c159df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.987+0000 7fdf7ad9d700 1 -- 172.21.15.119:0/2062295787 >> v1:172.21.15.119:6789/0 conn(0x7fdf7c157150 legacy=0x7fdf7c06e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.987+0000 7fdf7ad9d700 1 -- 172.21.15.119:0/2062295787 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c15d870 con 0x7fdf7c15e070 2023-10-08T01:11:49.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.987+0000 7fdf7ad9d700 1 --2- 172.21.15.119:0/2062295787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c15e070 0x7fdf7c15a500 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009e60 tx=0x7fdf74009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d00d2896df9e9b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:49.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.987+0000 7fdf7959a700 1 -- 172.21.15.119:0/2062295787 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7400e070 con 0x7fdf7c15e070 2023-10-08T01:11:49.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7959a700 1 -- 172.21.15.119:0/2062295787 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7400bb60 con 0x7fdf7c15e070 2023-10-08T01:11:49.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7959a700 1 -- 172.21.15.119:0/2062295787 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf74012430 con 0x7fdf7c15e070 2023-10-08T01:11:49.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2062295787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c15e070 msgr2=0x7fdf7c15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7b59e700 1 --2- 172.21.15.119:0/2062295787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c15e070 0x7fdf7c15a500 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009e60 tx=0x7fdf74009780 comp rx=0 tx=0).stop 2023-10-08T01:11:49.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2062295787 shutdown_connections 2023-10-08T01:11:49.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7b59e700 1 --2- 172.21.15.119:0/2062295787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c15e070 0x7fdf7c15a500 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2062295787 >> 172.21.15.119:0/2062295787 conn(0x7fdf7c0c9fb0 msgr2=0x7fdf7c06dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:49.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2062295787 shutdown_connections 2023-10-08T01:11:49.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.988+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2062295787 wait complete. 2023-10-08T01:11:49.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.990+0000 7fdf7b59e700 1 Processor -- start 2023-10-08T01:11:49.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.990+0000 7fdf7b59e700 1 -- start start 2023-10-08T01:11:49.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf7b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c06eb90 0x7fdf7c1464c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf7b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf7c157150 0x7fdf7c0c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf7b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf7c15e070 0x7fdf7c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf7b59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0b0fd0 con 0x7fdf7c157150 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf7b59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0b12b0 con 0x7fdf7c06eb90 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf7b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0b1590 con 0x7fdf7c15e070 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf79d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf7c157150 0x7fdf7c0c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf7a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c06eb90 0x7fdf7c1464c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf79d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf7c157150 0x7fdf7c0c3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44856/0 (socket says 172.21.15.119:44856) 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf79d9b700 1 -- 172.21.15.119:0/2857825391 learned_addr learned my addr 172.21.15.119:0/2857825391 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf7ad9d700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf7c15e070 0x7fdf7c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf79d9b700 1 -- 172.21.15.119:0/2857825391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf7c15e070 msgr2=0x7fdf7c0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf79d9b700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf7c15e070 0x7fdf7c0b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf79d9b700 1 -- 172.21.15.119:0/2857825391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c06eb90 msgr2=0x7fdf7c1464c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:49.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf79d9b700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c06eb90 0x7fdf7c1464c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:49.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.991+0000 7fdf79d9b700 1 -- 172.21.15.119:0/2857825391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74009a20 con 0x7fdf7c157150 2023-10-08T01:11:49.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.992+0000 7fdf79d9b700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf7c157150 0x7fdf7c0c3a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000c4e0 tx=0x7fdf7000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:49.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.992+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70011590 con 0x7fdf7c157150 2023-10-08T01:11:49.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.992+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf70011710 con 0x7fdf7c157150 2023-10-08T01:11:49.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.992+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70011ad0 con 0x7fdf7c157150 2023-10-08T01:11:49.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.992+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c0b1870 con 0x7fdf7c157150 2023-10-08T01:11:49.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.992+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c146a00 con 0x7fdf7c157150 2023-10-08T01:11:49.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.993+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7c06cfd0 con 0x7fdf7c157150 2023-10-08T01:11:49.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.994+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 37) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fdf7001b3b0 con 0x7fdf7c157150 2023-10-08T01:11:49.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.995+0000 7fdf6a7fc700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 0x7fdf5c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:49.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.995+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fdf70083f90 con 0x7fdf7c157150 2023-10-08T01:11:49.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.995+0000 7fdf7a59c700 1 -- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 msgr2=0x7fdf5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:49.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:49.995+0000 7fdf7a59c700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 0x7fdf5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:11:50.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.002+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf70050c00 con 0x7fdf7c157150 2023-10-08T01:11:50.099 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.098+0000 7f9aba263180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:11:50.149 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:50.148+0000 7f67b3774180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:11:50.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.196+0000 7fdf7a59c700 1 -- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 msgr2=0x7fdf5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:50.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.196+0000 7fdf7a59c700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 0x7fdf5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:11:50.207 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.206+0000 7f9aba263180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:11:50.241 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:50.240+0000 7f67b3774180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:11:50.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.248+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdf7c0c3090 con 0x7fdf7c157150 2023-10-08T01:11:50.331 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:50.330+0000 7f67b3774180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:11:50.400 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.400+0000 7f9aba263180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:11:50.425 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:11:50.424+0000 7f67b3774180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:11:50.494 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.494+0000 7f9aba263180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:11:50.587 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.586+0000 7f9aba263180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:11:50.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.597+0000 7fdf7a59c700 1 -- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 msgr2=0x7fdf5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:11:50.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.597+0000 7fdf7a59c700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 0x7fdf5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:11:50.681 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.680+0000 7f9aba263180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:11:50.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.693+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 38) v1 ==== 84451+0+0 (secure 0 0 0) 0x7fdf70050fd0 con 0x7fdf7c157150 2023-10-08T01:11:50.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.693+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 msgr2=0x7fdf5c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:11:50.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.693+0000 7fdf6a7fc700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 0x7fdf5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:50.760 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a33dc0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.760 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a33dc0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.760 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a33dc0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.760 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a33dc0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.760 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a33dc0700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.760 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a385c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.761 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a385c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.761 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a385c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.761 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a385c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.761 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:11:50.758+0000 7f9a385c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:11:50.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.771+0000 7fdf6a7fc700 1 -- 172.21.15.119:0/2857825391 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdf7006ebb0 con 0x7fdf7c157150 2023-10-08T01:11:50.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf7c157150 msgr2=0x7fdf7c0c3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:50.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf7c157150 0x7fdf7c0c3a80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000c4e0 tx=0x7fdf7000c7f0 comp rx=0 tx=0).stop 2023-10-08T01:11:50.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 shutdown_connections 2023-10-08T01:11:50.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf5c0670e0 0x7fdf5c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:50.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdf7c15e070 0x7fdf7c0b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:50.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdf7c06eb90 0x7fdf7c1464c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:50.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 --2- 172.21.15.119:0/2857825391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf7c157150 0x7fdf7c0c3a80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:50.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 >> 172.21.15.119:0/2857825391 conn(0x7fdf7c0c9fb0 msgr2=0x7fdf7c1559b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:50.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 shutdown_connections 2023-10-08T01:11:50.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:50.775+0000 7fdf7b59e700 1 -- 172.21.15.119:0/2857825391 wait complete. 2023-10-08T01:11:50.798 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T01:11:51.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.023+0000 7f5ff47ff700 1 Processor -- start 2023-10-08T01:11:51.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.024+0000 7f5ff47ff700 1 -- start start 2023-10-08T01:11:51.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.025+0000 7f5ff47ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec068aa0 0x7f5fec162460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.025+0000 7f5ff47ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec0ae6d0 con 0x7f5fec0b2100 2023-10-08T01:11:51.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.025+0000 7f5ff47ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec06dc20 con 0x7f5fec157740 2023-10-08T01:11:51.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.025+0000 7f5ff47ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0acc00 con 0x7f5fec068aa0 2023-10-08T01:11:51.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.025+0000 7f5ff1d9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5fec0b2100 0x7f5fec068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60742/0 (socket says 172.21.15.119:60742) 2023-10-08T01:11:51.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.025+0000 7f5ff1d9a700 1 -- 172.21.15.119:0/1284724880 learned_addr learned my addr 172.21.15.119:0/1284724880 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:51.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.025+0000 7f5ff2d9c700 1 --2- 172.21.15.119:0/1284724880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec068aa0 0x7f5fec162460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.025+0000 7f5ff1599700 1 -- 172.21.15.119:0/1284724880 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 558289507 0 0) 0x7f5fdc001110 con 0x7f5fec0b2100 2023-10-08T01:11:51.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff1599700 1 -- 172.21.15.119:0/1284724880 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd8003570 con 0x7f5fec0b2100 2023-10-08T01:11:51.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff1599700 1 -- 172.21.15.119:0/1284724880 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376303452 0 0) 0x7f5fec06dc20 con 0x7f5fec157740 2023-10-08T01:11:51.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff1599700 1 -- 172.21.15.119:0/1284724880 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fdc001110 con 0x7f5fec157740 2023-10-08T01:11:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff2d9c700 1 -- 172.21.15.119:0/1284724880 >> v1:172.21.15.184:6789/0 conn(0x7f5fec157740 legacy=0x7f5fec0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff2d9c700 1 -- 172.21.15.119:0/1284724880 >> v1:172.21.15.119:6789/0 conn(0x7f5fec0b2100 legacy=0x7f5fec068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff2d9c700 1 -- 172.21.15.119:0/1284724880 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0ae910 con 0x7f5fec068aa0 2023-10-08T01:11:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff2d9c700 1 --2- 172.21.15.119:0/1284724880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec068aa0 0x7f5fec162460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009e90 tx=0x7f5fe4009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0f0c4c8fa8884c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff1599700 1 -- 172.21.15.119:0/1284724880 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe400e070 con 0x7f5fec068aa0 2023-10-08T01:11:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff1599700 1 -- 172.21.15.119:0/1284724880 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe400bb80 con 0x7f5fec068aa0 2023-10-08T01:11:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.026+0000 7f5ff1599700 1 -- 172.21.15.119:0/1284724880 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4012430 con 0x7f5fec068aa0 2023-10-08T01:11:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.027+0000 7f5ff47ff700 1 -- 172.21.15.119:0/1284724880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec068aa0 msgr2=0x7f5fec162460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.027+0000 7f5ff47ff700 1 --2- 172.21.15.119:0/1284724880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec068aa0 0x7f5fec162460 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009e90 tx=0x7f5fe4009780 comp rx=0 tx=0).stop 2023-10-08T01:11:51.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.027+0000 7f5ff47ff700 1 -- 172.21.15.119:0/1284724880 shutdown_connections 2023-10-08T01:11:51.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.027+0000 7f5ff47ff700 1 --2- 172.21.15.119:0/1284724880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec068aa0 0x7f5fec162460 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.027+0000 7f5ff47ff700 1 -- 172.21.15.119:0/1284724880 >> 172.21.15.119:0/1284724880 conn(0x7f5fec0c8f90 msgr2=0x7f5fec0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:51.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.027+0000 7f5ff47ff700 1 -- 172.21.15.119:0/1284724880 shutdown_connections 2023-10-08T01:11:51.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.027+0000 7f5ff47ff700 1 -- 172.21.15.119:0/1284724880 wait complete. 2023-10-08T01:11:51.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.028+0000 7f5ff47ff700 1 Processor -- start 2023-10-08T01:11:51.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.028+0000 7f5ff47ff700 1 -- start start 2023-10-08T01:11:51.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.029+0000 7f5ff47ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec068aa0 0x7f5fec14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.029+0000 7f5ff47ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b2100 0x7f5fec0c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.029+0000 7f5ff47ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec157740 0x7f5fec1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.029+0000 7f5ff47ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec14dc90 con 0x7f5fec068aa0 2023-10-08T01:11:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.029+0000 7f5ff47ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec159d10 con 0x7f5fec157740 2023-10-08T01:11:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.029+0000 7f5ff47ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec159ff0 con 0x7f5fec0b2100 2023-10-08T01:11:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.029+0000 7f5ff259b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec068aa0 0x7f5fec14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff259b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec068aa0 0x7f5fec14d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44968/0 (socket says 172.21.15.119:44968) 2023-10-08T01:11:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff259b700 1 -- 172.21.15.119:0/2914941653 learned_addr learned my addr 172.21.15.119:0/2914941653 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff2d9c700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec157740 0x7f5fec1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff1d9a700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b2100 0x7f5fec0c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff259b700 1 -- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b2100 msgr2=0x7f5fec0c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff259b700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b2100 0x7f5fec0c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff259b700 1 -- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec157740 msgr2=0x7f5fec1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff259b700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec157740 0x7f5fec1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff259b700 1 -- 172.21.15.119:0/2914941653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe4009a20 con 0x7f5fec068aa0 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff1d9a700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b2100 0x7f5fec0c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5ff259b700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec068aa0 0x7f5fec14d750 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800c5b0 tx=0x7f5fe8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5fe27fc700 1 -- 172.21.15.119:0/2914941653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8010510 con 0x7f5fec068aa0 2023-10-08T01:11:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5fe27fc700 1 -- 172.21.15.119:0/2914941653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8010670 con 0x7f5fec068aa0 2023-10-08T01:11:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.030+0000 7f5fe27fc700 1 -- 172.21.15.119:0/2914941653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8010a30 con 0x7f5fec068aa0 2023-10-08T01:11:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.031+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec0c6150 con 0x7f5fec068aa0 2023-10-08T01:11:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.031+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec0c6710 con 0x7f5fec068aa0 2023-10-08T01:11:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.031+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec0718b0 con 0x7f5fec068aa0 2023-10-08T01:11:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.032+0000 7f5fe27fc700 1 -- 172.21.15.119:0/2914941653 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 38) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f5fe801b400 con 0x7f5fec068aa0 2023-10-08T01:11:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.033+0000 7f5fe27fc700 1 -- 172.21.15.119:0/2914941653 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6742+0+0 (secure 0 0 0) 0x7f5fe80840f0 con 0x7f5fec068aa0 2023-10-08T01:11:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.037+0000 7f5fe27fc700 1 -- 172.21.15.119:0/2914941653 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fe8051220 con 0x7f5fec068aa0 2023-10-08T01:11:51.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.227+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5fec0678d0 con 0x7f5fec068aa0 2023-10-08T01:11:51.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.695+0000 7f5fe27fc700 1 -- 172.21.15.119:0/2914941653 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 39) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f5fe8050cd0 con 0x7f5fec068aa0 2023-10-08T01:11:51.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.696+0000 7f5fe27fc700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5fd006c9e0 0x7f5fd006edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.698+0000 7f5ff1d9a700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5fd006c9e0 0x7f5fd006edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.701+0000 7f5ff1d9a700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5fd006c9e0 0x7f5fd006edc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5fec15cd10 tx=0x7f5fdc009040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.774+0000 7f5fe27fc700 1 -- 172.21.15.119:0/2914941653 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v81) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5fe806f730 con 0x7f5fec068aa0 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.779+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5fd006c9e0 msgr2=0x7f5fd006edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.779+0000 7f5ff47ff700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5fd006c9e0 0x7f5fd006edc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5fec15cd10 tx=0x7f5fdc009040 comp rx=0 tx=0).stop 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.779+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec068aa0 msgr2=0x7f5fec14d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.779+0000 7f5ff47ff700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec068aa0 0x7f5fec14d750 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f5fe800c5b0 tx=0x7f5fe8007e10 comp rx=0 tx=0).stop 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.780+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 shutdown_connections 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.780+0000 7f5ff47ff700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5fd006c9e0 0x7f5fd006edc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.780+0000 7f5ff47ff700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5fec157740 0x7f5fec1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.780+0000 7f5ff47ff700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fec068aa0 0x7f5fec14d750 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.780+0000 7f5ff47ff700 1 --2- 172.21.15.119:0/2914941653 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fec0b2100 0x7f5fec0c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.780+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 >> 172.21.15.119:0/2914941653 conn(0x7f5fec0c8f90 msgr2=0x7f5fec0b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.780+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 shutdown_connections 2023-10-08T01:11:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.781+0000 7f5ff47ff700 1 -- 172.21.15.119:0/2914941653 wait complete. 2023-10-08T01:11:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T01:11:51.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T01:11:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.985+0000 7f6d395ff700 1 Processor -- start 2023-10-08T01:11:51.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.985+0000 7f6d395ff700 1 -- start start 2023-10-08T01:11:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.986+0000 7f6d395ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d341563b0 0x7f6d34156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.986+0000 7f6d395ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d3415a440 0x7f6d340acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d395ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3415a9b0 con 0x7f6d34068890 2023-10-08T01:11:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d395ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3415ac90 con 0x7f6d341563b0 2023-10-08T01:11:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d395ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3415af70 con 0x7f6d3415a440 2023-10-08T01:11:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d337fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6d34068890 0x7f6d340c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60744/0 (socket says 172.21.15.119:60744) 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d337fe700 1 -- 172.21.15.119:0/976473434 learned_addr learned my addr 172.21.15.119:0/976473434 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d327fc700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d3415a440 0x7f6d340acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d32ffd700 1 --2- 172.21.15.119:0/976473434 >> v2:172.21.15.184:3300/0 conn(0x7f6d341563b0 0x7f6d34156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455935098 0 0) 0x7f6d3415a9b0 con 0x7f6d34068890 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.987+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d1c003570 con 0x7f6d34068890 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2865630508 0 0) 0x7f6d1c003570 con 0x7f6d34068890 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d3415a440 msgr2=0x7f6d340acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d3415a440 0x7f6d340acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 >> v2:172.21.15.184:3300/0 conn(0x7f6d341563b0 msgr2=0x7f6d34156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 --2- 172.21.15.119:0/976473434 >> v2:172.21.15.184:3300/0 conn(0x7f6d341563b0 0x7f6d34156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3415b250 con 0x7f6d34068890 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f6d2c0017f0 con 0x7f6d34068890 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 >> v1:172.21.15.119:6789/0 conn(0x7f6d34068890 legacy=0x7f6d340c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d3415a440 0x7f6d1c003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0040b0 0x7f6d1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d1c004d50 0x7f6d1c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3415ac90 con 0x7f6d1c0040b0 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3415af70 con 0x7f6d1c004d50 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c00bb60 con 0x7f6d3415a440 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d327fc700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0040b0 0x7f6d1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d32ffd700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d3415a440 0x7f6d1c003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d337fe700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d1c004d50 0x7f6d1c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:51.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d327fc700 1 -- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d3415a440 msgr2=0x7f6d1c003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:51.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d327fc700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d3415a440 0x7f6d1c003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d327fc700 1 -- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d1c004d50 msgr2=0x7f6d1c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:51.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d327fc700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d1c004d50 0x7f6d1c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.988+0000 7f6d327fc700 1 -- 172.21.15.119:0/976473434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c003570 con 0x7f6d1c0040b0 2023-10-08T01:11:51.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.989+0000 7f6d327fc700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0040b0 0x7f6d1c004810 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006f00 tx=0x7f6d280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4084fceb33dea5b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:51.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.989+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800e710 con 0x7f6d1c0040b0 2023-10-08T01:11:51.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.989+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2800e870 con 0x7f6d1c0040b0 2023-10-08T01:11:51.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.989+0000 7f6d31ffb700 1 -- 172.21.15.119:0/976473434 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800ec30 con 0x7f6d1c0040b0 2023-10-08T01:11:51.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.989+0000 7f6d395ff700 1 -- 172.21.15.119:0/976473434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0040b0 msgr2=0x7f6d1c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:51.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.989+0000 7f6d395ff700 1 --2- 172.21.15.119:0/976473434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d1c0040b0 0x7f6d1c004810 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6d28006f00 tx=0x7f6d280091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:51.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.990+0000 7f6d337fe700 1 -- 172.21.15.119:0/976473434 reap_dead start 2023-10-08T01:11:51.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.990+0000 7f6d395ff700 1 -- 172.21.15.119:0/976473434 shutdown_connections 2023-10-08T01:11:51.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.990+0000 7f6d395ff700 1 -- 172.21.15.119:0/976473434 >> 172.21.15.119:0/976473434 conn(0x7f6d340c8f90 msgr2=0x7f6d340c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:51.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.990+0000 7f6d395ff700 1 -- 172.21.15.119:0/976473434 shutdown_connections 2023-10-08T01:11:51.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.990+0000 7f6d395ff700 1 -- 172.21.15.119:0/976473434 wait complete. 2023-10-08T01:11:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.991+0000 7f6d395ff700 1 Processor -- start 2023-10-08T01:11:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.991+0000 7f6d395ff700 1 -- start start 2023-10-08T01:11:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d395ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d1c0040b0 0x7f6d340aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d395ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d34155330 0x7f6d3415f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:52.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d395ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d340b1790 0x7f6d340b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:52.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d395ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3416b410 con 0x7f6d340b1790 2023-10-08T01:11:52.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d395ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3415fc60 con 0x7f6d34155330 2023-10-08T01:11:52.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d395ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3415ff40 con 0x7f6d1c0040b0 2023-10-08T01:11:52.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d337fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d340b1790 0x7f6d340b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:52.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d337fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d340b1790 0x7f6d340b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44986/0 (socket says 172.21.15.119:44986) 2023-10-08T01:11:52.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.992+0000 7f6d337fe700 1 -- 172.21.15.119:0/812566334 learned_addr learned my addr 172.21.15.119:0/812566334 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:52.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d327fc700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d34155330 0x7f6d3415f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:52.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d32ffd700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d1c0040b0 0x7f6d340aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:52.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d337fe700 1 -- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d1c0040b0 msgr2=0x7f6d340aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:52.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d337fe700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d1c0040b0 0x7f6d340aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:52.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d337fe700 1 -- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d34155330 msgr2=0x7f6d3415f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d337fe700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d34155330 0x7f6d3415f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d337fe700 1 -- 172.21.15.119:0/812566334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34160220 con 0x7f6d340b1790 2023-10-08T01:11:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d327fc700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d34155330 0x7f6d3415f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d337fe700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d340b1790 0x7f6d340b1c10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6d3406b950 tx=0x7f6d2c002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d1affd700 1 -- 172.21.15.119:0/812566334 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c00acb0 con 0x7f6d340b1790 2023-10-08T01:11:52.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d1affd700 1 -- 172.21.15.119:0/812566334 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c00ae30 con 0x7f6d340b1790 2023-10-08T01:11:52.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d1affd700 1 -- 172.21.15.119:0/812566334 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c013640 con 0x7f6d340b1790 2023-10-08T01:11:52.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3416b0a0 con 0x7f6d340b1790 2023-10-08T01:11:52.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.993+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d341557a0 con 0x7f6d340b1790 2023-10-08T01:11:52.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.994+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3416b260 con 0x7f6d340b1790 2023-10-08T01:11:52.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.995+0000 7f6d1affd700 1 -- 172.21.15.119:0/812566334 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 39) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f6d2c0139f0 con 0x7f6d340b1790 2023-10-08T01:11:52.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.996+0000 7f6d1affd700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6d20066d80 0x7f6d20069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.996+0000 7f6d1affd700 1 -- 172.21.15.119:0/812566334 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6304+0+0 (secure 0 0 0) 0x7f6d2c084df0 con 0x7f6d340b1790 2023-10-08T01:11:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.997+0000 7f6d32ffd700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6d20066d80 0x7f6d20069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:51.997+0000 7f6d32ffd700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6d20066d80 0x7f6d20069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6d340b1580 tx=0x7f6d2400b430 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:52.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.000+0000 7f6d1affd700 1 -- 172.21.15.119:0/812566334 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d2c052010 con 0x7f6d340b1790 2023-10-08T01:11:52.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.196+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6d340718b0 con 0x7f6d340b1790 2023-10-08T01:11:52.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.778+0000 7f6d1affd700 1 -- 172.21.15.119:0/812566334 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d2c0523e0 con 0x7f6d340b1790 2023-10-08T01:11:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.787+0000 7f6d1affd700 1 -- 172.21.15.119:0/812566334 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v82) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6d2c052010 con 0x7f6d340b1790 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6d20066d80 msgr2=0x7f6d20069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6d20066d80 0x7f6d20069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6d340b1580 tx=0x7f6d2400b430 comp rx=0 tx=0).stop 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d340b1790 msgr2=0x7f6d340b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d340b1790 0x7f6d340b1c10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6d3406b950 tx=0x7f6d2c002140 comp rx=0 tx=0).stop 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 shutdown_connections 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6d20066d80 0x7f6d20069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d1c0040b0 0x7f6d340aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d340b1790 0x7f6d340b1c10 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.790+0000 7f6d395ff700 1 --2- 172.21.15.119:0/812566334 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d34155330 0x7f6d3415f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:52.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.791+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 >> 172.21.15.119:0/812566334 conn(0x7f6d340c8f90 msgr2=0x7f6d340adf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:52.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.791+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 shutdown_connections 2023-10-08T01:11:52.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:52.791+0000 7f6d395ff700 1 -- 172.21.15.119:0/812566334 wait complete. 2023-10-08T01:11:52.793 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T01:11:52.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T01:11:53.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.005+0000 7fc9405ff700 1 Processor -- start 2023-10-08T01:11:53.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.006+0000 7fc9405ff700 1 -- start start 2023-10-08T01:11:53.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.007+0000 7fc9405ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc938069e80 0x7fc938160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.007+0000 7fc9405ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93806e950 con 0x7fc938155eb0 2023-10-08T01:11:53.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.007+0000 7fc9405ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93806eb90 con 0x7fc9380c70b0 2023-10-08T01:11:53.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.007+0000 7fc9405ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc93806ee70 con 0x7fc938069e80 2023-10-08T01:11:53.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.007+0000 7fc93e39b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc938155eb0 0x7fc93806e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60746/0 (socket says 172.21.15.119:60746) 2023-10-08T01:11:53.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.007+0000 7fc93e39b700 1 -- 172.21.15.119:0/49897116 learned_addr learned my addr 172.21.15.119:0/49897116 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:53.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.007+0000 7fc93eb9c700 1 --2- 172.21.15.119:0/49897116 >> v2:172.21.15.184:3300/0 conn(0x7fc938069e80 0x7fc938160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.007+0000 7fc93d399700 1 -- 172.21.15.119:0/49897116 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451392027 0 0) 0x7fc934001110 con 0x7fc938155eb0 2023-10-08T01:11:53.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93d399700 1 -- 172.21.15.119:0/49897116 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc924003570 con 0x7fc938155eb0 2023-10-08T01:11:53.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93d399700 1 -- 172.21.15.119:0/49897116 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987328457 0 0) 0x7fc93806eb90 con 0x7fc9380c70b0 2023-10-08T01:11:53.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93d399700 1 -- 172.21.15.119:0/49897116 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc934001110 con 0x7fc9380c70b0 2023-10-08T01:11:53.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93eb9c700 1 -- 172.21.15.119:0/49897116 >> v1:172.21.15.184:6789/0 conn(0x7fc9380c70b0 legacy=0x7fc938069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93eb9c700 1 -- 172.21.15.119:0/49897116 >> v1:172.21.15.119:6789/0 conn(0x7fc938155eb0 legacy=0x7fc93806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93d399700 1 -- 172.21.15.119:0/49897116 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1302411033 0 0) 0x7fc924003570 con 0x7fc938155eb0 2023-10-08T01:11:53.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93eb9c700 1 -- 172.21.15.119:0/49897116 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93806f150 con 0x7fc938069e80 2023-10-08T01:11:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93eb9c700 1 --2- 172.21.15.119:0/49897116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938069e80 0x7fc938160bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc930009d50 tx=0x7fc930009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81aca530d471e213 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93d399700 1 -- 172.21.15.119:0/49897116 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc934001600 con 0x7fc938069e80 2023-10-08T01:11:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93d399700 1 -- 172.21.15.119:0/49897116 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc930009eb0 con 0x7fc938069e80 2023-10-08T01:11:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.008+0000 7fc93d399700 1 -- 172.21.15.119:0/49897116 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc93000cae0 con 0x7fc938069e80 2023-10-08T01:11:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.009+0000 7fc9405ff700 1 -- 172.21.15.119:0/49897116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938069e80 msgr2=0x7fc938160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.009+0000 7fc9405ff700 1 --2- 172.21.15.119:0/49897116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938069e80 0x7fc938160bf0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc930009d50 tx=0x7fc930009780 comp rx=0 tx=0).stop 2023-10-08T01:11:53.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.009+0000 7fc9405ff700 1 -- 172.21.15.119:0/49897116 shutdown_connections 2023-10-08T01:11:53.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.009+0000 7fc9405ff700 1 --2- 172.21.15.119:0/49897116 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938069e80 0x7fc938160bf0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.009+0000 7fc9405ff700 1 -- 172.21.15.119:0/49897116 >> 172.21.15.119:0/49897116 conn(0x7fc938068e10 msgr2=0x7fc9380cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:53.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.009+0000 7fc9405ff700 1 -- 172.21.15.119:0/49897116 shutdown_connections 2023-10-08T01:11:53.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.009+0000 7fc9405ff700 1 -- 172.21.15.119:0/49897116 wait complete. 2023-10-08T01:11:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.010+0000 7fc9405ff700 1 Processor -- start 2023-10-08T01:11:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.011+0000 7fc9405ff700 1 -- start start 2023-10-08T01:11:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.011+0000 7fc9405ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc938069e80 0x7fc938143030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.011+0000 7fc9405ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9380c70b0 0x7fc9380abbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.011+0000 7fc9405ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938155eb0 0x7fc9380ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.011+0000 7fc9405ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93816b070 con 0x7fc9380c70b0 2023-10-08T01:11:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.011+0000 7fc9405ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380c55a0 con 0x7fc938155eb0 2023-10-08T01:11:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.011+0000 7fc9405ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380c5880 con 0x7fc938069e80 2023-10-08T01:11:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9380c70b0 0x7fc9380abbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9380c70b0 0x7fc9380abbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44990/0 (socket says 172.21.15.119:44990) 2023-10-08T01:11:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 -- 172.21.15.119:0/3394355773 learned_addr learned my addr 172.21.15.119:0/3394355773 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93eb9c700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938155eb0 0x7fc9380ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93e39b700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc938069e80 0x7fc938143030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 -- 172.21.15.119:0/3394355773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc938069e80 msgr2=0x7fc938143030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc938069e80 0x7fc938143030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 -- 172.21.15.119:0/3394355773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938155eb0 msgr2=0x7fc9380ac100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938155eb0 0x7fc9380ac100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 -- 172.21.15.119:0/3394355773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93000d040 con 0x7fc9380c70b0 2023-10-08T01:11:53.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.012+0000 7fc93db9a700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9380c70b0 0x7fc9380abbc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c790 tx=0x7fc928007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.013+0000 7fc92e7fc700 1 -- 172.21.15.119:0/3394355773 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928010510 con 0x7fc9380c70b0 2023-10-08T01:11:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.013+0000 7fc92e7fc700 1 -- 172.21.15.119:0/3394355773 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc928010670 con 0x7fc9380c70b0 2023-10-08T01:11:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.013+0000 7fc92e7fc700 1 -- 172.21.15.119:0/3394355773 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928010a10 con 0x7fc9380c70b0 2023-10-08T01:11:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.013+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9380c5b60 con 0x7fc9380c70b0 2023-10-08T01:11:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.013+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc938143570 con 0x7fc9380c70b0 2023-10-08T01:11:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.014+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc93816b1e0 con 0x7fc9380c70b0 2023-10-08T01:11:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.015+0000 7fc92e7fc700 1 -- 172.21.15.119:0/3394355773 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc92801b3b0 con 0x7fc9380c70b0 2023-10-08T01:11:53.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.016+0000 7fc92e7fc700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc91c066f30 0x7fc91c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.016+0000 7fc92e7fc700 1 -- 172.21.15.119:0/3394355773 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fc928083e90 con 0x7fc9380c70b0 2023-10-08T01:11:53.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.016+0000 7fc93e39b700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc91c066f30 0x7fc91c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.017+0000 7fc93e39b700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc91c066f30 0x7fc91c0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc934001010 tx=0x7fc934001ac0 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.020+0000 7fc92e7fc700 1 -- 172.21.15.119:0/3394355773 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc928051040 con 0x7fc9380c70b0 2023-10-08T01:11:53.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.213+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc93804c3c0 con 0x7fc9380c70b0 2023-10-08T01:11:53.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.217+0000 7fc92e7fc700 1 -- 172.21.15.119:0/3394355773 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9280170a0 con 0x7fc9380c70b0 2023-10-08T01:11:53.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc91c066f30 msgr2=0x7fc91c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc91c066f30 0x7fc91c0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc934001010 tx=0x7fc934001ac0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9380c70b0 msgr2=0x7fc9380abbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9380c70b0 0x7fc9380abbc0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c790 tx=0x7fc928007e10 comp rx=0 tx=0).stop 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 shutdown_connections 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc91c066f30 0x7fc91c0693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc938069e80 0x7fc938143030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc938155eb0 0x7fc9380ac100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 --2- 172.21.15.119:0/3394355773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9380c70b0 0x7fc9380abbc0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.219+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 >> 172.21.15.119:0/3394355773 conn(0x7fc938068e10 msgr2=0x7fc9380a98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:53.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.220+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 shutdown_connections 2023-10-08T01:11:53.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.220+0000 7fc9405ff700 1 -- 172.21.15.119:0/3394355773 wait complete. 2023-10-08T01:11:53.238 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T01:11:53.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.420+0000 7f903d1ff700 1 Processor -- start 2023-10-08T01:11:53.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.420+0000 7f903d1ff700 1 -- start start 2023-10-08T01:11:53.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.421+0000 7f903d1ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f90380c83f0 0x7f90380c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.421+0000 7f903d1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90381568e0 0x7f90380ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.421+0000 7f903d1ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903815b0d0 con 0x7f90380ccee0 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.421+0000 7f903d1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90380c7a60 con 0x7f90381568e0 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.421+0000 7f903d1ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f903815a4c0 con 0x7f90380c83f0 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f903759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90381568e0 0x7f90380ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f903759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f90381568e0 0x7f90380ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44992/0 (socket says 172.21.15.119:44992) 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f903759e700 1 -- 172.21.15.119:0/3049267685 learned_addr learned my addr 172.21.15.119:0/3049267685 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f9036d9d700 1 --2- 172.21.15.119:0/3049267685 >> v2:172.21.15.184:3300/0 conn(0x7f90380c83f0 0x7f90380c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f903759e700 1 -- 172.21.15.119:0/3049267685 >> v2:172.21.15.184:3300/0 conn(0x7f90380c83f0 msgr2=0x7f90380c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:53.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f903759e700 1 --2- 172.21.15.119:0/3049267685 >> v2:172.21.15.184:3300/0 conn(0x7f90380c83f0 0x7f90380c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f903759e700 1 -- 172.21.15.119:0/3049267685 >> v1:172.21.15.184:6789/0 conn(0x7f90380ccee0 legacy=0x7f90381561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f903759e700 1 -- 172.21.15.119:0/3049267685 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903815a7a0 con 0x7f90381568e0 2023-10-08T01:11:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f9036d9d700 1 --2- 172.21.15.119:0/3049267685 >> v2:172.21.15.184:3300/0 conn(0x7f90380c83f0 0x7f90380c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.422+0000 7f903759e700 1 --2- 172.21.15.119:0/3049267685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380ca1e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f90380c9fd0 tx=0x7f9030009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fb736446be9069 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f9035d9b700 1 -- 172.21.15.119:0/3049267685 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903000e070 con 0x7f90381568e0 2023-10-08T01:11:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f9035d9b700 1 -- 172.21.15.119:0/3049267685 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903000ba10 con 0x7f90381568e0 2023-10-08T01:11:53.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f9035d9b700 1 -- 172.21.15.119:0/3049267685 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90300092e0 con 0x7f90381568e0 2023-10-08T01:11:53.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f903d1ff700 1 -- 172.21.15.119:0/3049267685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 msgr2=0x7f90380ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f903d1ff700 1 --2- 172.21.15.119:0/3049267685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380ca1e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f90380c9fd0 tx=0x7f9030009840 comp rx=0 tx=0).stop 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f903d1ff700 1 -- 172.21.15.119:0/3049267685 shutdown_connections 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f903d1ff700 1 --2- 172.21.15.119:0/3049267685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380ca1e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f903d1ff700 1 --2- 172.21.15.119:0/3049267685 >> v2:172.21.15.184:3300/0 conn(0x7f90380c83f0 0x7f90380c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f903d1ff700 1 -- 172.21.15.119:0/3049267685 >> 172.21.15.119:0/3049267685 conn(0x7f9038068e10 msgr2=0x7f90380cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f903d1ff700 1 -- 172.21.15.119:0/3049267685 shutdown_connections 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.423+0000 7f903d1ff700 1 -- 172.21.15.119:0/3049267685 wait complete. 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.425+0000 7f903d1ff700 1 Processor -- start 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.425+0000 7f903d1ff700 1 -- start start 2023-10-08T01:11:53.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903d1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90380c83f0 0x7f90380b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903d1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90380ccee0 0x7f90380c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903d1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903d1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9038157380 con 0x7f90381568e0 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903d1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90380b1b30 con 0x7f90380ccee0 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903d1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90380b1e10 con 0x7f90380c83f0 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45004/0 (socket says 172.21.15.119:45004) 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903759e700 1 -- 172.21.15.119:0/2157906625 learned_addr learned my addr 172.21.15.119:0/2157906625 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f9036d9d700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90380c83f0 0x7f90380b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903659c700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90380ccee0 0x7f90380c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903759e700 1 -- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90380c83f0 msgr2=0x7f90380b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903759e700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90380c83f0 0x7f90380b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903759e700 1 -- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90380ccee0 msgr2=0x7f90380c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903759e700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90380ccee0 0x7f90380c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903759e700 1 -- 172.21.15.119:0/2157906625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9030009ae0 con 0x7f90381568e0 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.426+0000 7f903659c700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90380ccee0 0x7f90380c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.427+0000 7f903759e700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380c9630 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9038063d40 tx=0x7f9030017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.427+0000 7f901effd700 1 -- 172.21.15.119:0/2157906625 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9030012db0 con 0x7f90381568e0 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.427+0000 7f901effd700 1 -- 172.21.15.119:0/2157906625 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903001d420 con 0x7f90381568e0 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.427+0000 7f901effd700 1 -- 172.21.15.119:0/2157906625 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903001d7e0 con 0x7f90381568e0 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.427+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9038156d30 con 0x7f90381568e0 2023-10-08T01:11:53.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.427+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90380b1700 con 0x7f90381568e0 2023-10-08T01:11:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.428+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903804c3c0 con 0x7f90381568e0 2023-10-08T01:11:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.429+0000 7f901effd700 1 -- 172.21.15.119:0/2157906625 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f903001dbd0 con 0x7f90381568e0 2023-10-08T01:11:53.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.430+0000 7f901effd700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9024066f60 0x7f9024069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.430+0000 7f901effd700 1 -- 172.21.15.119:0/2157906625 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f9030085f80 con 0x7f90381568e0 2023-10-08T01:11:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.430+0000 7f9036d9d700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9024066f60 0x7f9024069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.430+0000 7f9036d9d700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9024066f60 0x7f9024069410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f903806a1d0 tx=0x7f9028006f90 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.434+0000 7f901effd700 1 -- 172.21.15.119:0/2157906625 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9030053090 con 0x7f90381568e0 2023-10-08T01:11:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.626+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f903815aa80 con 0x7f90381568e0 2023-10-08T01:11:53.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.630+0000 7f901effd700 1 -- 172.21.15.119:0/2157906625 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v26) v1 ==== 115+0+0 (secure 0 0 0) 0x7f90300566b0 con 0x7f90381568e0 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9024066f60 msgr2=0x7f9024069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9024066f60 0x7f9024069410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f903806a1d0 tx=0x7f9028006f90 comp rx=0 tx=0).stop 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 msgr2=0x7f90380c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380c9630 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9038063d40 tx=0x7f9030017000 comp rx=0 tx=0).stop 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 shutdown_connections 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9024066f60 0x7f9024069410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90380c83f0 0x7f90380b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90381568e0 0x7f90380c9630 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.633+0000 7f903d1ff700 1 --2- 172.21.15.119:0/2157906625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90380ccee0 0x7f90380c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.634+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 >> 172.21.15.119:0/2157906625 conn(0x7f9038068e10 msgr2=0x7f90380ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:53.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.634+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 shutdown_connections 2023-10-08T01:11:53.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.634+0000 7f903d1ff700 1 -- 172.21.15.119:0/2157906625 wait complete. 2023-10-08T01:11:53.652 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T01:11:53.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.838+0000 7f162359e700 1 Processor -- start 2023-10-08T01:11:53.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.838+0000 7f162359e700 1 -- start start 2023-10-08T01:11:53.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.839+0000 7f162359e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1624153900 0x7f16240c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.839+0000 7f162359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 0x7f162406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.839+0000 7f162359e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162406e150 con 0x7f16240c9db0 2023-10-08T01:11:53.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.839+0000 7f162359e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16240c7800 con 0x7f1624153900 2023-10-08T01:11:53.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.839+0000 7f162359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240c7ae0 con 0x7f162406a710 2023-10-08T01:11:53.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.839+0000 7f162259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1624153900 0x7f16240c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.839+0000 7f162259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1624153900 0x7f16240c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45008/0 (socket says 172.21.15.119:45008) 2023-10-08T01:11:53.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.839+0000 7f162259c700 1 -- 172.21.15.119:0/1741969217 learned_addr learned my addr 172.21.15.119:0/1741969217 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:53.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f1622d9d700 1 --2- 172.21.15.119:0/1741969217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 0x7f162406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f162259c700 1 -- 172.21.15.119:0/1741969217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 msgr2=0x7f162406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:53.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f162259c700 1 --2- 172.21.15.119:0/1741969217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 0x7f162406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f162259c700 1 -- 172.21.15.119:0/1741969217 >> v1:172.21.15.184:6789/0 conn(0x7f16240c9db0 legacy=0x7f162406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f162259c700 1 -- 172.21.15.119:0/1741969217 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16240c7dc0 con 0x7f1624153900 2023-10-08T01:11:53.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f162259c700 1 --2- 172.21.15.119:0/1741969217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c6f00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f16140099b0 tx=0x7f1614009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22684dce51f78e23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f1621d9b700 1 -- 172.21.15.119:0/1741969217 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400b860 con 0x7f1624153900 2023-10-08T01:11:53.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f1621d9b700 1 -- 172.21.15.119:0/1741969217 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400b9c0 con 0x7f1624153900 2023-10-08T01:11:53.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.840+0000 7f1621d9b700 1 -- 172.21.15.119:0/1741969217 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400bd60 con 0x7f1624153900 2023-10-08T01:11:53.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.841+0000 7f162359e700 1 -- 172.21.15.119:0/1741969217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 msgr2=0x7f16240c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.841+0000 7f162359e700 1 --2- 172.21.15.119:0/1741969217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c6f00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f16140099b0 tx=0x7f1614009250 comp rx=0 tx=0).stop 2023-10-08T01:11:53.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.841+0000 7f162359e700 1 -- 172.21.15.119:0/1741969217 shutdown_connections 2023-10-08T01:11:53.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.841+0000 7f162359e700 1 --2- 172.21.15.119:0/1741969217 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 0x7f162406e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.841+0000 7f162359e700 1 --2- 172.21.15.119:0/1741969217 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c6f00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.841+0000 7f162359e700 1 -- 172.21.15.119:0/1741969217 >> 172.21.15.119:0/1741969217 conn(0x7f16240cc1e0 msgr2=0x7f16240cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:53.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.841+0000 7f162359e700 1 -- 172.21.15.119:0/1741969217 shutdown_connections 2023-10-08T01:11:53.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.841+0000 7f162359e700 1 -- 172.21.15.119:0/1741969217 wait complete. 2023-10-08T01:11:53.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.842+0000 7f162359e700 1 Processor -- start 2023-10-08T01:11:53.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.843+0000 7f162359e700 1 -- start start 2023-10-08T01:11:53.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.843+0000 7f162359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 0x7f162416d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f162359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16240c9db0 0x7f16240abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f162359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f162359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162416d580 con 0x7f1624153900 2023-10-08T01:11:53.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f162359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240ae790 con 0x7f16240c9db0 2023-10-08T01:11:53.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f162359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240ad8e0 con 0x7f162406a710 2023-10-08T01:11:53.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f1622d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f1622d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45024/0 (socket says 172.21.15.119:45024) 2023-10-08T01:11:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f1622d9d700 1 -- 172.21.15.119:0/4099678429 learned_addr learned my addr 172.21.15.119:0/4099678429 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f162259c700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 0x7f162416d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f161bfff700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16240c9db0 0x7f16240abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f1622d9d700 1 -- 172.21.15.119:0/4099678429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 msgr2=0x7f162416d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f1622d9d700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 0x7f162416d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f1622d9d700 1 -- 172.21.15.119:0/4099678429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16240c9db0 msgr2=0x7f16240abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f1622d9d700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16240c9db0 0x7f16240abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.844+0000 7f1622d9d700 1 -- 172.21.15.119:0/4099678429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16140094f0 con 0x7f1624153900 2023-10-08T01:11:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.845+0000 7f1622d9d700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c23e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f161c007ed0 tx=0x7f161c009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.845+0000 7f161a7fc700 1 -- 172.21.15.119:0/4099678429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00eb90 con 0x7f1624153900 2023-10-08T01:11:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.845+0000 7f161a7fc700 1 -- 172.21.15.119:0/4099678429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c00ed10 con 0x7f1624153900 2023-10-08T01:11:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.845+0000 7f161a7fc700 1 -- 172.21.15.119:0/4099678429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c014670 con 0x7f1624153900 2023-10-08T01:11:53.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.845+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16240adb80 con 0x7f1624153900 2023-10-08T01:11:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.845+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16240ae420 con 0x7f1624153900 2023-10-08T01:11:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.846+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16240ae940 con 0x7f1624153900 2023-10-08T01:11:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.847+0000 7f161a7fc700 1 -- 172.21.15.119:0/4099678429 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f161c014a60 con 0x7f1624153900 2023-10-08T01:11:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.848+0000 7f161a7fc700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1604066f30 0x7f16040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.848+0000 7f161a7fc700 1 -- 172.21.15.119:0/4099678429 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f161c084c10 con 0x7f1624153900 2023-10-08T01:11:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.848+0000 7f162259c700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1604066f30 0x7f16040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:53.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.849+0000 7f162259c700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1604066f30 0x7f16040693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16140099b0 tx=0x7f1614017000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:53.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:53.852+0000 7f161a7fc700 1 -- 172.21.15.119:0/4099678429 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f16240ae940 con 0x7f1624153900 2023-10-08T01:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.042+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f16240ae900 con 0x7f1624153900 2023-10-08T01:11:54.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.047+0000 7f161a7fc700 1 -- 172.21.15.119:0/4099678429 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v27) v1 ==== 120+0+0 (secure 0 0 0) 0x7f161c051d20 con 0x7f1624153900 2023-10-08T01:11:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1604066f30 msgr2=0x7f16040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1604066f30 0x7f16040693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16140099b0 tx=0x7f1614017000 comp rx=0 tx=0).stop 2023-10-08T01:11:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 msgr2=0x7f16240c23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c23e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f161c007ed0 tx=0x7f161c009880 comp rx=0 tx=0).stop 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 shutdown_connections 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1604066f30 0x7f16040693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f162406a710 0x7f162416d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1624153900 0x7f16240c23e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 --2- 172.21.15.119:0/4099678429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16240c9db0 0x7f16240abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 >> 172.21.15.119:0/4099678429 conn(0x7f16240cc1e0 msgr2=0x7f16240ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 shutdown_connections 2023-10-08T01:11:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.049+0000 7f162359e700 1 -- 172.21.15.119:0/4099678429 wait complete. 2023-10-08T01:11:54.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-10-08T01:11:54.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.246+0000 7f41a41ff700 1 Processor -- start 2023-10-08T01:11:54.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.247+0000 7f41a41ff700 1 -- start start 2023-10-08T01:11:54.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a41ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f419c0b1850 0x7f419c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a41ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f419c0ae680 0x7f419c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a41ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0aeb00 0x7f419c157820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a41ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419c06fc30 con 0x7f419c0ae680 2023-10-08T01:11:54.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a41ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f419c0cc260 con 0x7f419c0b1850 2023-10-08T01:11:54.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a41ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c0cb300 con 0x7f419c0aeb00 2023-10-08T01:11:54.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a179a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f419c0ae680 0x7f419c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a179a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f419c0ae680 0x7f419c1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45026/0 (socket says 172.21.15.119:45026) 2023-10-08T01:11:54.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.248+0000 7f41a179a700 1 -- 172.21.15.119:0/349086494 learned_addr learned my addr 172.21.15.119:0/349086494 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a1f9b700 1 --2- 172.21.15.119:0/349086494 >> v2:172.21.15.184:3300/0 conn(0x7f419c0b1850 0x7f419c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a179a700 1 -- 172.21.15.119:0/349086494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0aeb00 msgr2=0x7f419c157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a279c700 1 --2- 172.21.15.119:0/349086494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0aeb00 0x7f419c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a179a700 1 --2- 172.21.15.119:0/349086494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0aeb00 0x7f419c157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a179a700 1 -- 172.21.15.119:0/349086494 >> v2:172.21.15.184:3300/0 conn(0x7f419c0b1850 msgr2=0x7f419c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a179a700 1 --2- 172.21.15.119:0/349086494 >> v2:172.21.15.184:3300/0 conn(0x7f419c0b1850 0x7f419c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a179a700 1 -- 172.21.15.119:0/349086494 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c0cb5e0 con 0x7f419c0ae680 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a1f9b700 1 --2- 172.21.15.119:0/349086494 >> v2:172.21.15.184:3300/0 conn(0x7f419c0b1850 0x7f419c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:54.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a279c700 1 --2- 172.21.15.119:0/349086494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0aeb00 0x7f419c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:54.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a179a700 1 --2- 172.21.15.119:0/349086494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0ae680 0x7f419c1572e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f419c0641e0 tx=0x7f418c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58019acdc2af8a6b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:54.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a0f99700 1 -- 172.21.15.119:0/349086494 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00b820 con 0x7f419c0ae680 2023-10-08T01:11:54.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a0f99700 1 -- 172.21.15.119:0/349086494 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f418c00b980 con 0x7f419c0ae680 2023-10-08T01:11:54.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.249+0000 7f41a0f99700 1 -- 172.21.15.119:0/349086494 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c00bd20 con 0x7f419c0ae680 2023-10-08T01:11:54.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 -- 172.21.15.119:0/349086494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0ae680 msgr2=0x7f419c1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 --2- 172.21.15.119:0/349086494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0ae680 0x7f419c1572e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f419c0641e0 tx=0x7f418c009250 comp rx=0 tx=0).stop 2023-10-08T01:11:54.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 -- 172.21.15.119:0/349086494 shutdown_connections 2023-10-08T01:11:54.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 --2- 172.21.15.119:0/349086494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0aeb00 0x7f419c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 --2- 172.21.15.119:0/349086494 >> v2:172.21.15.184:3300/0 conn(0x7f419c0b1850 0x7f419c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 --2- 172.21.15.119:0/349086494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0ae680 0x7f419c1572e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 -- 172.21.15.119:0/349086494 >> 172.21.15.119:0/349086494 conn(0x7f419c06a320 msgr2=0x7f419c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:54.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 -- 172.21.15.119:0/349086494 shutdown_connections 2023-10-08T01:11:54.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.250+0000 7f41a41ff700 1 -- 172.21.15.119:0/349086494 wait complete. 2023-10-08T01:11:54.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.251+0000 7f41a41ff700 1 Processor -- start 2023-10-08T01:11:54.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.252+0000 7f41a41ff700 1 -- start start 2023-10-08T01:11:54.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.252+0000 7f41a41ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0ae680 0x7f419c06f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.252+0000 7f41a41ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f419c0aeb00 0x7f419c155540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.252+0000 7f41a41ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0b1850 0x7f419c0afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.252+0000 7f41a41ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c169e50 con 0x7f419c0b1850 2023-10-08T01:11:54.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.252+0000 7f41a41ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419c155a80 con 0x7f419c0aeb00 2023-10-08T01:11:54.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.252+0000 7f41a41ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f419c155d00 con 0x7f419c0ae680 2023-10-08T01:11:54.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0b1850 0x7f419c0afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0b1850 0x7f419c0afae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45036/0 (socket says 172.21.15.119:45036) 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 -- 172.21.15.119:0/36935034 learned_addr learned my addr 172.21.15.119:0/36935034 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a179a700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f419c0aeb00 0x7f419c155540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a1f9b700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0ae680 0x7f419c06f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 -- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0ae680 msgr2=0x7f419c06f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0ae680 0x7f419c06f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 -- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f419c0aeb00 msgr2=0x7f419c155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f419c0aeb00 0x7f419c155540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 -- 172.21.15.119:0/36935034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c0094f0 con 0x7f419c0b1850 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a1f9b700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0ae680 0x7f419c06f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:54.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f41a279c700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0b1850 0x7f419c0afae0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f419400cf90 tx=0x7f4194007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.253+0000 7f4191ffb700 1 -- 172.21.15.119:0/36935034 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194015440 con 0x7f419c0b1850 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.254+0000 7f4191ffb700 1 -- 172.21.15.119:0/36935034 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41940155a0 con 0x7f419c0b1850 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.254+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419c155fe0 con 0x7f419c0b1850 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.254+0000 7f4191ffb700 1 -- 172.21.15.119:0/36935034 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4194015960 con 0x7f419c0b1850 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.254+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419c169740 con 0x7f419c0b1850 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.255+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f419c16a000 con 0x7f419c0b1850 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.255+0000 7f4191ffb700 1 -- 172.21.15.119:0/36935034 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f419400d440 con 0x7f419c0b1850 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.256+0000 7f4191ffb700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.256+0000 7f4191ffb700 1 -- 172.21.15.119:0/36935034 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f4194085320 con 0x7f419c0b1850 2023-10-08T01:11:54.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.257+0000 7f41a1f9b700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.257+0000 7f41a1f9b700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4180066f30 0x7f41800693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f419800ac10 tx=0x7f4198006f90 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:54.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.260+0000 7f4191ffb700 1 -- 172.21.15.119:0/36935034 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f419c16a000 con 0x7f419c0b1850 2023-10-08T01:11:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.525+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7f419c16a000 con 0x7f419c0b1850 2023-10-08T01:11:54.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.780+0000 7f4191ffb700 1 -- 172.21.15.119:0/36935034 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v590) v1 ==== 156+0+0 (secure 0 0 0) 0x7f419c16a000 con 0x7f419c0b1850 2023-10-08T01:11:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4180066f30 msgr2=0x7f41800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4180066f30 0x7f41800693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f419800ac10 tx=0x7f4198006f90 comp rx=0 tx=0).stop 2023-10-08T01:11:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0b1850 msgr2=0x7f419c0afae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0b1850 0x7f419c0afae0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f419400cf90 tx=0x7f4194007870 comp rx=0 tx=0).stop 2023-10-08T01:11:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 shutdown_connections 2023-10-08T01:11:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4180066f30 0x7f41800693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f419c0ae680 0x7f419c06f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f419c0b1850 0x7f419c0afae0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 --2- 172.21.15.119:0/36935034 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f419c0aeb00 0x7f419c155540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 >> 172.21.15.119:0/36935034 conn(0x7f419c06a320 msgr2=0x7f419c15f430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 shutdown_connections 2023-10-08T01:11:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.783+0000 7f41a41ff700 1 -- 172.21.15.119:0/36935034 wait complete. 2023-10-08T01:11:54.801 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-10-08T01:11:54.802 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-10-08T01:11:54.802 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T01:11:54.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-08T01:11:54.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.981+0000 7fbc2139d700 1 Processor -- start 2023-10-08T01:11:54.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.981+0000 7fbc2139d700 1 -- start start 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc2139d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc1c150550 0x7fbc1c157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc2139d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc1c0c7410 0x7fbc1c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc2139d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc1c06e2a0 con 0x7fbc1c150180 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc2139d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c06b2e0 con 0x7fbc1c0c7410 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc2139d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc1c0c78e0 con 0x7fbc1c150550 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc20b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc1c0c7410 0x7fbc1c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc1bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbc1c150180 0x7fbc1c0c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41598/0 (socket says 172.21.15.119:41598) 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc1bfff700 1 -- 172.21.15.119:0/3533278984 learned_addr learned my addr 172.21.15.119:0/3533278984 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.983+0000 7fbc1b7fe700 1 --2- 172.21.15.119:0/3533278984 >> v2:172.21.15.184:3300/0 conn(0x7fbc1c150550 0x7fbc1c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.984+0000 7fbc20b9c700 1 -- 172.21.15.119:0/3533278984 >> v2:172.21.15.184:3300/0 conn(0x7fbc1c150550 msgr2=0x7fbc1c157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.984+0000 7fbc20b9c700 1 --2- 172.21.15.119:0/3533278984 >> v2:172.21.15.184:3300/0 conn(0x7fbc1c150550 0x7fbc1c157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.984+0000 7fbc20b9c700 1 -- 172.21.15.119:0/3533278984 >> v1:172.21.15.119:6789/0 conn(0x7fbc1c150180 legacy=0x7fbc1c0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.984+0000 7fbc20b9c700 1 -- 172.21.15.119:0/3533278984 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c0c7bc0 con 0x7fbc1c0c7410 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.984+0000 7fbc20b9c700 1 --2- 172.21.15.119:0/3533278984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c0c7410 0x7fbc1c069f50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c06d1e0 tx=0x7fbc140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60de8bf161f27245 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.984+0000 7fbc1affd700 1 -- 172.21.15.119:0/3533278984 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14010600 con 0x7fbc1c0c7410 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.984+0000 7fbc1affd700 1 -- 172.21.15.119:0/3533278984 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc14010760 con 0x7fbc1c0c7410 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.984+0000 7fbc1affd700 1 -- 172.21.15.119:0/3533278984 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14010b00 con 0x7fbc1c0c7410 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.985+0000 7fbc2139d700 1 -- 172.21.15.119:0/3533278984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c0c7410 msgr2=0x7fbc1c069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.985+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3533278984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c0c7410 0x7fbc1c069f50 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c06d1e0 tx=0x7fbc140091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.985+0000 7fbc2139d700 1 -- 172.21.15.119:0/3533278984 shutdown_connections 2023-10-08T01:11:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.985+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3533278984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c0c7410 0x7fbc1c069f50 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.985+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3533278984 >> v2:172.21.15.184:3300/0 conn(0x7fbc1c150550 0x7fbc1c157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.985+0000 7fbc2139d700 1 -- 172.21.15.119:0/3533278984 >> 172.21.15.119:0/3533278984 conn(0x7fbc1c0cc1f0 msgr2=0x7fbc1c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:54.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.985+0000 7fbc2139d700 1 -- 172.21.15.119:0/3533278984 shutdown_connections 2023-10-08T01:11:54.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.985+0000 7fbc2139d700 1 -- 172.21.15.119:0/3533278984 wait complete. 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.986+0000 7fbc2139d700 1 Processor -- start 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.986+0000 7fbc2139d700 1 -- start start 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc2139d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0c7410 0x7fbc1c13dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc2139d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c150180 0x7fbc1c0c4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc2139d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c150550 0x7fbc1c0aca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc2139d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0af5e0 con 0x7fbc1c150550 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc2139d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0ad5f0 con 0x7fbc1c150180 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc2139d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c0ad8d0 con 0x7fbc1c0c7410 2023-10-08T01:11:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc20b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c150550 0x7fbc1c0aca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc20b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c150550 0x7fbc1c0aca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51532/0 (socket says 172.21.15.119:51532) 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.987+0000 7fbc20b9c700 1 -- 172.21.15.119:0/3862466476 learned_addr learned my addr 172.21.15.119:0/3862466476 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc20b9c700 1 -- 172.21.15.119:0/3862466476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0c7410 msgr2=0x7fbc1c13dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc20b9c700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0c7410 0x7fbc1c13dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc20b9c700 1 -- 172.21.15.119:0/3862466476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c150180 msgr2=0x7fbc1c0c4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc20b9c700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c150180 0x7fbc1c0c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc20b9c700 1 -- 172.21.15.119:0/3862466476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14009420 con 0x7fbc1c150550 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc20b9c700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c150550 0x7fbc1c0aca80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbc14014000 tx=0x7fbc14018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc03fff700 1 -- 172.21.15.119:0/3862466476 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc140233f0 con 0x7fbc1c150550 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc03fff700 1 -- 172.21.15.119:0/3862466476 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc14023550 con 0x7fbc1c150550 2023-10-08T01:11:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc03fff700 1 -- 172.21.15.119:0/3862466476 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc14023910 con 0x7fbc1c150550 2023-10-08T01:11:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c0aef90 con 0x7fbc1c150550 2023-10-08T01:11:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.988+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c0ae2f0 con 0x7fbc1c150550 2023-10-08T01:11:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.990+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c06f770 con 0x7fbc1c150550 2023-10-08T01:11:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.990+0000 7fbc03fff700 1 -- 172.21.15.119:0/3862466476 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc14023d40 con 0x7fbc1c150550 2023-10-08T01:11:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.991+0000 7fbc03fff700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.991+0000 7fbc03fff700 1 -- 172.21.15.119:0/3862466476 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fbc14085e50 con 0x7fbc1c150550 2023-10-08T01:11:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.991+0000 7fbc1bfff700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.992+0000 7fbc1bfff700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc08066f30 0x7fbc080693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c002040 tx=0x7fbc0c0015f0 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:54.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:54.995+0000 7fbc03fff700 1 -- 172.21.15.119:0/3862466476 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc14053000 con 0x7fbc1c150550 2023-10-08T01:11:55.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.263+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7fbc1c04c3c0 con 0x7fbc1c150550 2023-10-08T01:11:55.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.782+0000 7fbc03fff700 1 -- 172.21.15.119:0/3862466476 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v591) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbc14056620 con 0x7fbc1c150550 2023-10-08T01:11:55.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc08066f30 msgr2=0x7fbc080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc08066f30 0x7fbc080693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c002040 tx=0x7fbc0c0015f0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c150550 msgr2=0x7fbc1c0aca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c150550 0x7fbc1c0aca80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbc14014000 tx=0x7fbc14018000 comp rx=0 tx=0).stop 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 shutdown_connections 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc08066f30 0x7fbc080693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc1c0c7410 0x7fbc1c13dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc1c150550 0x7fbc1c0aca80 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 --2- 172.21.15.119:0/3862466476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc1c150180 0x7fbc1c0c4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 >> 172.21.15.119:0/3862466476 conn(0x7fbc1c0cc1f0 msgr2=0x7fbc1c0696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:55.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 shutdown_connections 2023-10-08T01:11:55.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.784+0000 7fbc2139d700 1 -- 172.21.15.119:0/3862466476 wait complete. 2023-10-08T01:11:55.803 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T01:11:55.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.987+0000 7fe80d9ff700 1 Processor -- start 2023-10-08T01:11:55.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.988+0000 7fe80d9ff700 1 -- start start 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe80d9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8080c83f0 0x7fe8080c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe80d9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe8081568e0 0x7fe8080ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe80d9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe80815b0d0 con 0x7fe8080ccee0 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe80d9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8080c7a60 con 0x7fe8081568e0 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe80d9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80815a4c0 con 0x7fe8080c83f0 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe8077fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe8081568e0 0x7fe8080ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe8077fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe8081568e0 0x7fe8080ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51544/0 (socket says 172.21.15.119:51544) 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe8077fe700 1 -- 172.21.15.119:0/3224517430 learned_addr learned my addr 172.21.15.119:0/3224517430 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe8067fc700 1 --1- 172.21.15.119:0/3224517430 >> v1:172.21.15.119:6789/0 conn(0x7fe8080ccee0 0x7fe8081561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41604/0 (socket says 172.21.15.119:41604) 2023-10-08T01:11:55.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.989+0000 7fe806ffd700 1 --2- 172.21.15.119:0/3224517430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8080c83f0 0x7fe8080c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe8077fe700 1 -- 172.21.15.119:0/3224517430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8080c83f0 msgr2=0x7fe8080c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe8077fe700 1 --2- 172.21.15.119:0/3224517430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8080c83f0 0x7fe8080c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe8077fe700 1 -- 172.21.15.119:0/3224517430 >> v1:172.21.15.119:6789/0 conn(0x7fe8080ccee0 legacy=0x7fe8081561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe805ffb700 1 -- 172.21.15.119:0/3224517430 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172717849 0 0) 0x7fe80815b0d0 con 0x7fe8080ccee0 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe8077fe700 1 -- 172.21.15.119:0/3224517430 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe80815a7a0 con 0x7fe8081568e0 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe8077fe700 1 --2- 172.21.15.119:0/3224517430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8081568e0 0x7fe8080ca1e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe800009f20 tx=0x7fe800009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f7a48d9ecba830 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe805ffb700 1 -- 172.21.15.119:0/3224517430 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80000e070 con 0x7fe8081568e0 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe805ffb700 1 -- 172.21.15.119:0/3224517430 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80000ba80 con 0x7fe8081568e0 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.990+0000 7fe805ffb700 1 -- 172.21.15.119:0/3224517430 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8000092e0 con 0x7fe8081568e0 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.991+0000 7fe80d9ff700 1 -- 172.21.15.119:0/3224517430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8081568e0 msgr2=0x7fe8080ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.991+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/3224517430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8081568e0 0x7fe8080ca1e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe800009f20 tx=0x7fe800009840 comp rx=0 tx=0).stop 2023-10-08T01:11:55.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.991+0000 7fe80d9ff700 1 -- 172.21.15.119:0/3224517430 shutdown_connections 2023-10-08T01:11:55.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.991+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/3224517430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8081568e0 0x7fe8080ca1e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.991+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/3224517430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8080c83f0 0x7fe8080c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.991+0000 7fe80d9ff700 1 -- 172.21.15.119:0/3224517430 >> 172.21.15.119:0/3224517430 conn(0x7fe808068e10 msgr2=0x7fe8080cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:55.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.991+0000 7fe80d9ff700 1 -- 172.21.15.119:0/3224517430 shutdown_connections 2023-10-08T01:11:55.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.991+0000 7fe80d9ff700 1 -- 172.21.15.119:0/3224517430 wait complete. 2023-10-08T01:11:55.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.992+0000 7fe80d9ff700 1 Processor -- start 2023-10-08T01:11:55.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.992+0000 7fe80d9ff700 1 -- start start 2023-10-08T01:11:55.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe80d9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8080c83f0 0x7fe808158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:55.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe80d9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8080ccee0 0x7fe8080c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:55.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe80d9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8081568e0 0x7fe80814cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:55.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe80d9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80814dd30 con 0x7fe8080c83f0 2023-10-08T01:11:55.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe80d9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe80814e010 con 0x7fe8080ccee0 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe80d9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80814e2f0 con 0x7fe8081568e0 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe806ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8080c83f0 0x7fe808158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe806ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8080c83f0 0x7fe808158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51554/0 (socket says 172.21.15.119:51554) 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.993+0000 7fe806ffd700 1 -- 172.21.15.119:0/2758594518 learned_addr learned my addr 172.21.15.119:0/2758594518 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe806ffd700 1 -- 172.21.15.119:0/2758594518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8081568e0 msgr2=0x7fe80814cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe806ffd700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8081568e0 0x7fe80814cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe806ffd700 1 -- 172.21.15.119:0/2758594518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8080ccee0 msgr2=0x7fe8080c26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe806ffd700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8080ccee0 0x7fe8080c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe806ffd700 1 -- 172.21.15.119:0/2758594518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009ae0 con 0x7fe8080c83f0 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe806ffd700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8080c83f0 0x7fe808158df0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800d890 tx=0x7fe7f8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:55.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe7eeffd700 1 -- 172.21.15.119:0/2758594518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f80125f0 con 0x7fe8080c83f0 2023-10-08T01:11:55.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe7eeffd700 1 -- 172.21.15.119:0/2758594518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f8012770 con 0x7fe8080c83f0 2023-10-08T01:11:55.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe7eeffd700 1 -- 172.21.15.119:0/2758594518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8012b30 con 0x7fe8080c83f0 2023-10-08T01:11:55.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80814e5d0 con 0x7fe8080c83f0 2023-10-08T01:11:55.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.994+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8080c2c00 con 0x7fe8080c83f0 2023-10-08T01:11:55.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.995+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8080c2e20 con 0x7fe8080c83f0 2023-10-08T01:11:55.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.996+0000 7fe7eeffd700 1 -- 172.21.15.119:0/2758594518 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe7f801e460 con 0x7fe8080c83f0 2023-10-08T01:11:55.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.997+0000 7fe7eeffd700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe7f4066f30 0x7fe7f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:55.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.997+0000 7fe7eeffd700 1 -- 172.21.15.119:0/2758594518 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fe7f8084d90 con 0x7fe8080c83f0 2023-10-08T01:11:55.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.998+0000 7fe8067fc700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe7f4066f30 0x7fe7f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:55.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:55.998+0000 7fe8067fc700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe7f4066f30 0x7fe7f40693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0011e0 tx=0x7fe7fc0079b0 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:56.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.001+0000 7fe7eeffd700 1 -- 172.21.15.119:0/2758594518 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe7f8051ea0 con 0x7fe8080c83f0 2023-10-08T01:11:56.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.192+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe80804c3c0 con 0x7fe8080c83f0 2023-10-08T01:11:56.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.196+0000 7fe7eeffd700 1 -- 172.21.15.119:0/2758594518 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe7f80554c0 con 0x7fe8080c83f0 2023-10-08T01:11:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.198+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe7f4066f30 msgr2=0x7fe7f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.198+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe7f4066f30 0x7fe7f40693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0011e0 tx=0x7fe7fc0079b0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.198+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8080c83f0 msgr2=0x7fe808158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.198+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8080c83f0 0x7fe808158df0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800d890 tx=0x7fe7f8008040 comp rx=0 tx=0).stop 2023-10-08T01:11:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.199+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 shutdown_connections 2023-10-08T01:11:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.199+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe7f4066f30 0x7fe7f40693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.199+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe8081568e0 0x7fe80814cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.199+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe8080c83f0 0x7fe808158df0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.199+0000 7fe80d9ff700 1 --2- 172.21.15.119:0/2758594518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe8080ccee0 0x7fe8080c26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.199+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 >> 172.21.15.119:0/2758594518 conn(0x7fe808068e10 msgr2=0x7fe808150e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.199+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 shutdown_connections 2023-10-08T01:11:56.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.199+0000 7fe80d9ff700 1 -- 172.21.15.119:0/2758594518 wait complete. 2023-10-08T01:11:56.217 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:11:56.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:11:56.295 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:11:56.386 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:11:56.386 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T01:11:56.386 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:11:56.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.565+0000 7f1b4919d700 1 Processor -- start 2023-10-08T01:11:56.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.566+0000 7f1b4919d700 1 -- start start 2023-10-08T01:11:56.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.567+0000 7f1b4919d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b44157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:56.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.567+0000 7f1b4919d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4406e2a0 con 0x7f1b44150180 2023-10-08T01:11:56.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.567+0000 7f1b4919d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4406b2e0 con 0x7f1b440c7410 2023-10-08T01:11:56.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.567+0000 7f1b4919d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440c78e0 con 0x7f1b44150550 2023-10-08T01:11:56.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.567+0000 7f1b43fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1b44150180 0x7f1b440c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41612/0 (socket says 172.21.15.119:41612) 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.567+0000 7f1b43fff700 1 -- 172.21.15.119:0/1605506955 learned_addr learned my addr 172.21.15.119:0/1605506955 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.567+0000 7f1b437fe700 1 --2- 172.21.15.119:0/1605506955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b44157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.567+0000 7f1b42ffd700 1 -- 172.21.15.119:0/1605506955 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052343893 0 0) 0x7f1b4406e2a0 con 0x7f1b44150180 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b42ffd700 1 -- 172.21.15.119:0/1605506955 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b2c003570 con 0x7f1b44150180 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b42ffd700 1 -- 172.21.15.119:0/1605506955 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152624031 0 0) 0x7f1b4406b2e0 con 0x7f1b440c7410 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b42ffd700 1 -- 172.21.15.119:0/1605506955 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b4406e2a0 con 0x7f1b440c7410 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b42ffd700 1 -- 172.21.15.119:0/1605506955 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3057627415 0 0) 0x7f1b2c003570 con 0x7f1b44150180 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b437fe700 1 -- 172.21.15.119:0/1605506955 >> v1:172.21.15.184:6789/0 conn(0x7f1b440c7410 legacy=0x7f1b44069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b437fe700 1 -- 172.21.15.119:0/1605506955 >> v1:172.21.15.119:6789/0 conn(0x7f1b44150180 legacy=0x7f1b440c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b437fe700 1 -- 172.21.15.119:0/1605506955 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440c7bc0 con 0x7f1b44150550 2023-10-08T01:11:56.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b437fe700 1 --2- 172.21.15.119:0/1605506955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b44157770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800aaf0 tx=0x7f1b3800a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=161257dfc4455f0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b42ffd700 1 -- 172.21.15.119:0/1605506955 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34001470 con 0x7f1b44150550 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b42ffd700 1 -- 172.21.15.119:0/1605506955 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3800c660 con 0x7f1b44150550 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.568+0000 7f1b42ffd700 1 -- 172.21.15.119:0/1605506955 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3800ca00 con 0x7f1b44150550 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.569+0000 7f1b4919d700 1 -- 172.21.15.119:0/1605506955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 msgr2=0x7f1b44157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.569+0000 7f1b4919d700 1 --2- 172.21.15.119:0/1605506955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b44157770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800aaf0 tx=0x7f1b3800a330 comp rx=0 tx=0).stop 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.569+0000 7f1b4919d700 1 -- 172.21.15.119:0/1605506955 shutdown_connections 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.569+0000 7f1b4919d700 1 --2- 172.21.15.119:0/1605506955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b44157770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.569+0000 7f1b4919d700 1 -- 172.21.15.119:0/1605506955 >> 172.21.15.119:0/1605506955 conn(0x7f1b440cc1f0 msgr2=0x7f1b440cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.569+0000 7f1b4919d700 1 -- 172.21.15.119:0/1605506955 shutdown_connections 2023-10-08T01:11:56.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.569+0000 7f1b4919d700 1 -- 172.21.15.119:0/1605506955 wait complete. 2023-10-08T01:11:56.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.570+0000 7f1b4919d700 1 Processor -- start 2023-10-08T01:11:56.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.571+0000 7f1b4919d700 1 -- start start 2023-10-08T01:11:56.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.571+0000 7f1b4919d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b440c7410 0x7f1b440c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:56.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.571+0000 7f1b4919d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b44150180 0x7f1b4416a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:56.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.571+0000 7f1b4919d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b440c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:56.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b4919d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440aa6e0 con 0x7f1b440c7410 2023-10-08T01:11:56.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b4919d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440c4920 con 0x7f1b44150180 2023-10-08T01:11:56.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b4919d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b440c1a10 con 0x7f1b44150550 2023-10-08T01:11:56.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b440c7410 0x7f1b440c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:56.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b440c7410 0x7f1b440c43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51558/0 (socket says 172.21.15.119:51558) 2023-10-08T01:11:56.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 -- 172.21.15.119:0/1615607247 learned_addr learned my addr 172.21.15.119:0/1615607247 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:56.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b437fe700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b44150180 0x7f1b4416a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:56.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b4899c700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b440c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:56.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 -- 172.21.15.119:0/1615607247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 msgr2=0x7f1b440c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b440c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 -- 172.21.15.119:0/1615607247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b44150180 msgr2=0x7f1b4416a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b44150180 0x7f1b4416a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 -- 172.21.15.119:0/1615607247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3800a5d0 con 0x7f1b440c7410 2023-10-08T01:11:56.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.572+0000 7f1b43fff700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b440c7410 0x7f1b440c43e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400cb70 tx=0x7f1b34001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:56.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.573+0000 7f1b2bfff700 1 -- 172.21.15.119:0/1615607247 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34007e90 con 0x7f1b440c7410 2023-10-08T01:11:56.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.573+0000 7f1b2bfff700 1 -- 172.21.15.119:0/1615607247 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3400ce80 con 0x7f1b440c7410 2023-10-08T01:11:56.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.573+0000 7f1b2bfff700 1 -- 172.21.15.119:0/1615607247 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34007570 con 0x7f1b440c7410 2023-10-08T01:11:56.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.573+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b440c1cf0 con 0x7f1b440c7410 2023-10-08T01:11:56.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.573+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b440aa300 con 0x7f1b440c7410 2023-10-08T01:11:56.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.574+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4404c3c0 con 0x7f1b440c7410 2023-10-08T01:11:56.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.575+0000 7f1b2bfff700 1 -- 172.21.15.119:0/1615607247 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b34004a40 con 0x7f1b440c7410 2023-10-08T01:11:56.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.575+0000 7f1b2bfff700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b30066f30 0x7f1b300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:56.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.575+0000 7f1b2bfff700 1 -- 172.21.15.119:0/1615607247 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f1b34083ca0 con 0x7f1b440c7410 2023-10-08T01:11:56.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.576+0000 7f1b437fe700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b30066f30 0x7f1b300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:56.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.576+0000 7f1b437fe700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b30066f30 0x7f1b300693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800aac0 tx=0x7f1b38019000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:56.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.580+0000 7f1b2bfff700 1 -- 172.21.15.119:0/1615607247 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b3404c5a0 con 0x7f1b440c7410 2023-10-08T01:11:56.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.814+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b4406ada0 con 0x7f1b440c7410 2023-10-08T01:11:56.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.814+0000 7f1b2bfff700 1 -- 172.21.15.119:0/1615607247 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f1b34018070 con 0x7f1b440c7410 2023-10-08T01:11:56.816 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:56.816 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5444,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/4024548231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":4024548231},{"type":"v1","addr":"172.21.15.184:6835","nonce":4024548231}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1924614082","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1924614082},{"type":"v1","addr":"172.21.15.119:6835","nonce":1924614082}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":5902,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/296234355","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":296234355},{"type":"v1","addr":"172.21.15.184:6834","nonce":296234355}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6582,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1958559124","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1958559124},{"type":"v1","addr":"172.21.15.119:6834","nonce":1958559124}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[]} 2023-10-08T01:11:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b30066f30 msgr2=0x7f1b300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b30066f30 0x7f1b300693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800aac0 tx=0x7f1b38019000 comp rx=0 tx=0).stop 2023-10-08T01:11:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b440c7410 msgr2=0x7f1b440c43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b440c7410 0x7f1b440c43e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400cb70 tx=0x7f1b34001b00 comp rx=0 tx=0).stop 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 shutdown_connections 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b30066f30 0x7f1b300693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b44150550 0x7f1b440c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b440c7410 0x7f1b440c43e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 --2- 172.21.15.119:0/1615607247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b44150180 0x7f1b4416a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 >> 172.21.15.119:0/1615607247 conn(0x7f1b440cc1f0 msgr2=0x7f1b441597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 shutdown_connections 2023-10-08T01:11:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:56.817+0000 7f1b4919d700 1 -- 172.21.15.119:0/1615607247 wait complete. 2023-10-08T01:11:56.820 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 34 2023-10-08T01:11:56.835 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T01:11:56.836 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T01:11:56.836 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T01:11:56.836 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:11:56.836 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T01:11:56.836 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T01:11:56.837 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T01:11:56.837 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:11:56.837 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T01:11:56.837 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T01:11:56.837 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T01:11:56.838 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:11:56.838 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T01:11:56.838 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T01:11:56.838 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T01:11:56.838 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:11:56.849 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T01:11:56.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T01:11:56.850 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T01:11:56.851 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T01:11:56.853 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T01:11:56.853 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T01:11:56.854 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T01:11:56.855 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T01:11:56.895 INFO:tasks.ceph.mds.a:Started 2023-10-08T01:11:56.895 DEBUG:teuthology.parallel:result is None 2023-10-08T01:11:56.898 INFO:tasks.ceph.mds.b:Started 2023-10-08T01:11:56.898 DEBUG:teuthology.parallel:result is None 2023-10-08T01:11:56.899 INFO:tasks.ceph.mds.c:Started 2023-10-08T01:11:56.899 DEBUG:teuthology.parallel:result is None 2023-10-08T01:11:56.900 INFO:tasks.ceph.mds.d:Started 2023-10-08T01:11:56.901 DEBUG:teuthology.parallel:result is None 2023-10-08T01:11:56.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T01:11:57.027 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:11:57.030 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T01:11:57.034 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T01:11:57.034 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T01:11:57.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.092+0000 7fb5ad79d700 1 Processor -- start 2023-10-08T01:11:57.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.092+0000 7fb5ad79d700 1 -- start start 2023-10-08T01:11:57.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.093+0000 7fb5ad79d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb5a806d950 0x7fb5a80c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:57.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.093+0000 7fb5ad79d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5a806a0a0 0x7fb5a80c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.093+0000 7fb5ad79d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5a806ae70 con 0x7fb5a806dd20 2023-10-08T01:11:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.093+0000 7fb5ad79d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a806a540 con 0x7fb5a806a0a0 2023-10-08T01:11:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.093+0000 7fb5ad79d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5a806a820 con 0x7fb5a806d950 2023-10-08T01:11:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5acf9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb5a806dd20 0x7fb5a80c7150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41640/0 (socket says 172.21.15.119:41640) 2023-10-08T01:11:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5a77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5a806a0a0 0x7fb5a80c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5acf9c700 1 -- 172.21.15.119:0/4189905414 learned_addr learned my addr 172.21.15.119:0/4189905414 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:57.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5a7fff700 1 --2- 172.21.15.119:0/4189905414 >> v2:172.21.15.184:3300/0 conn(0x7fb5a806d950 0x7fb5a80c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5a77fe700 1 -- 172.21.15.119:0/4189905414 >> v2:172.21.15.184:3300/0 conn(0x7fb5a806d950 msgr2=0x7fb5a80c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5a77fe700 1 --2- 172.21.15.119:0/4189905414 >> v2:172.21.15.184:3300/0 conn(0x7fb5a806d950 0x7fb5a80c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5a77fe700 1 -- 172.21.15.119:0/4189905414 >> v1:172.21.15.119:6789/0 conn(0x7fb5a806dd20 legacy=0x7fb5a80c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5a77fe700 1 -- 172.21.15.119:0/4189905414 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5a806ab00 con 0x7fb5a806a0a0 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5a77fe700 1 --2- 172.21.15.119:0/4189905414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806a0a0 0x7fb5a80c6c10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80c9c70 tx=0x7fb59c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa95140b987d7eb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.094+0000 7fb5a6ffd700 1 -- 172.21.15.119:0/4189905414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c011630 con 0x7fb5a806a0a0 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5a6ffd700 1 -- 172.21.15.119:0/4189905414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c011790 con 0x7fb5a806a0a0 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5a6ffd700 1 -- 172.21.15.119:0/4189905414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c011b30 con 0x7fb5a806a0a0 2023-10-08T01:11:57.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4189905414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806a0a0 msgr2=0x7fb5a80c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:57.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4189905414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806a0a0 0x7fb5a80c6c10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80c9c70 tx=0x7fb59c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:57.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4189905414 shutdown_connections 2023-10-08T01:11:57.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4189905414 >> v2:172.21.15.184:3300/0 conn(0x7fb5a806d950 0x7fb5a80c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:57.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4189905414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806a0a0 0x7fb5a80c6c10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:57.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4189905414 >> 172.21.15.119:0/4189905414 conn(0x7fb5a80cc1f0 msgr2=0x7fb5a80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:57.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4189905414 shutdown_connections 2023-10-08T01:11:57.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.095+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4189905414 wait complete. 2023-10-08T01:11:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.097+0000 7fb5ad79d700 1 Processor -- start 2023-10-08T01:11:57.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.097+0000 7fb5ad79d700 1 -- start start 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5ad79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a806a0a0 0x7fb5a80ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5ad79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5a806d950 0x7fb5a80cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5ad79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806dd20 0x7fb5a80adf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5ad79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80ca520 con 0x7fb5a806dd20 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5ad79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80c5f00 con 0x7fb5a806a0a0 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5ad79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a80cbdc0 con 0x7fb5a806d950 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5acf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806dd20 0x7fb5a80adf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5acf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806dd20 0x7fb5a80adf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51610/0 (socket says 172.21.15.119:51610) 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5acf9c700 1 -- 172.21.15.119:0/4247946543 learned_addr learned my addr 172.21.15.119:0/4247946543 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:57.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5a7fff700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a806a0a0 0x7fb5a80ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5a77fe700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5a806d950 0x7fb5a80cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5acf9c700 1 -- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5a806d950 msgr2=0x7fb5a80cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5acf9c700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5a806d950 0x7fb5a80cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5acf9c700 1 -- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a806a0a0 msgr2=0x7fb5a80ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5acf9c700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a806a0a0 0x7fb5a80ca940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5acf9c700 1 -- 172.21.15.119:0/4247946543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c009420 con 0x7fb5a806dd20 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.098+0000 7fb5a7fff700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a806a0a0 0x7fb5a80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.099+0000 7fb5acf9c700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806dd20 0x7fb5a80adf10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000cbc0 tx=0x7fb5a00019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.099+0000 7fb58ffff700 1 -- 172.21.15.119:0/4247946543 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000a7e0 con 0x7fb5a806dd20 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.099+0000 7fb58ffff700 1 -- 172.21.15.119:0/4247946543 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a000a940 con 0x7fb5a806dd20 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.099+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a80af500 con 0x7fb5a806dd20 2023-10-08T01:11:57.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.099+0000 7fb58ffff700 1 -- 172.21.15.119:0/4247946543 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a000acc0 con 0x7fb5a806dd20 2023-10-08T01:11:57.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.099+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a80af7e0 con 0x7fb5a806dd20 2023-10-08T01:11:57.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.100+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a80ca660 con 0x7fb5a806dd20 2023-10-08T01:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.101+0000 7fb58ffff700 1 -- 172.21.15.119:0/4247946543 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb5a0007300 con 0x7fb5a806dd20 2023-10-08T01:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.102+0000 7fb58ffff700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb594066f30 0x7fb5940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.102+0000 7fb58ffff700 1 -- 172.21.15.119:0/4247946543 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fb5a0084940 con 0x7fb5a806dd20 2023-10-08T01:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.102+0000 7fb5a7fff700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb594066f30 0x7fb5940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:57.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.103+0000 7fb5a7fff700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb594066f30 0x7fb5940693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb598011480 tx=0x7fb5980091a0 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:57.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.106+0000 7fb58ffff700 1 -- 172.21.15.119:0/4247946543 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb5a80ca660 con 0x7fb5a806dd20 2023-10-08T01:11:57.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:57.316+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb5a80ca660 con 0x7fb5a806dd20 2023-10-08T01:11:58.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.216+0000 7fb58ffff700 1 -- 172.21.15.119:0/4247946543 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v87) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb5a80ca660 con 0x7fb5a806dd20 2023-10-08T01:11:58.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb594066f30 msgr2=0x7fb5940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb594066f30 0x7fb5940693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb598011480 tx=0x7fb5980091a0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806dd20 msgr2=0x7fb5a80adf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806dd20 0x7fb5a80adf10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000cbc0 tx=0x7fb5a00019c0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 shutdown_connections 2023-10-08T01:11:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb594066f30 0x7fb5940693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5a806dd20 0x7fb5a80adf10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb5a806a0a0 0x7fb5a80ca940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 --2- 172.21.15.119:0/4247946543 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb5a806d950 0x7fb5a80cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 >> 172.21.15.119:0/4247946543 conn(0x7fb5a80cc1f0 msgr2=0x7fb5a8155cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 shutdown_connections 2023-10-08T01:11:58.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.219+0000 7fb5ad79d700 1 -- 172.21.15.119:0/4247946543 wait complete. 2023-10-08T01:11:58.222 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T01:11:58.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T01:11:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.422+0000 7f86d37ff700 1 Processor -- start 2023-10-08T01:11:58.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.422+0000 7f86d37ff700 1 -- start start 2023-10-08T01:11:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.423+0000 7f86d37ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86cc06bcd0 0x7f86cc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.423+0000 7f86d37ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 0x7f86cc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.423+0000 7f86d37ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86cc15df50 0x7f86cc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.423+0000 7f86d37ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86cc15d7e0 con 0x7f86cc06bcd0 2023-10-08T01:11:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.423+0000 7f86d37ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86cc15dac0 con 0x7f86cc15df50 2023-10-08T01:11:58.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.423+0000 7f86d37ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc15aa30 con 0x7f86cc06c0a0 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86cc06bcd0 0x7f86cc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86cc06bcd0 0x7f86cc0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51620/0 (socket says 172.21.15.119:51620) 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 -- 172.21.15.119:0/4142276728 learned_addr learned my addr 172.21.15.119:0/4142276728 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d0d9a700 1 --2- 172.21.15.119:0/4142276728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 0x7f86cc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d1d9c700 1 --2- 172.21.15.119:0/4142276728 >> v2:172.21.15.184:3300/0 conn(0x7f86cc15df50 0x7f86cc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 -- 172.21.15.119:0/4142276728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 msgr2=0x7f86cc15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 --2- 172.21.15.119:0/4142276728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 0x7f86cc15a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 -- 172.21.15.119:0/4142276728 >> v2:172.21.15.184:3300/0 conn(0x7f86cc15df50 msgr2=0x7f86cc15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 --2- 172.21.15.119:0/4142276728 >> v2:172.21.15.184:3300/0 conn(0x7f86cc15df50 0x7f86cc15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 -- 172.21.15.119:0/4142276728 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc15ad10 con 0x7f86cc06bcd0 2023-10-08T01:11:58.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d1d9c700 1 --2- 172.21.15.119:0/4142276728 >> v2:172.21.15.184:3300/0 conn(0x7f86cc15df50 0x7f86cc15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86d159b700 1 --2- 172.21.15.119:0/4142276728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc06bcd0 0x7f86cc0caa90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b4600 tx=0x7f86c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68d28b160a414c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.424+0000 7f86c3fff700 1 -- 172.21.15.119:0/4142276728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c800b860 con 0x7f86cc06bcd0 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86c3fff700 1 -- 172.21.15.119:0/4142276728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c800b9c0 con 0x7f86cc06bcd0 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86c3fff700 1 -- 172.21.15.119:0/4142276728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c800bd80 con 0x7f86cc06bcd0 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 -- 172.21.15.119:0/4142276728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc06bcd0 msgr2=0x7f86cc0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 --2- 172.21.15.119:0/4142276728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc06bcd0 0x7f86cc0caa90 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b4600 tx=0x7f86c8009250 comp rx=0 tx=0).stop 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 -- 172.21.15.119:0/4142276728 shutdown_connections 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 --2- 172.21.15.119:0/4142276728 >> v2:172.21.15.184:3300/0 conn(0x7f86cc15df50 0x7f86cc15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 --2- 172.21.15.119:0/4142276728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc06bcd0 0x7f86cc0caa90 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 --2- 172.21.15.119:0/4142276728 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 0x7f86cc15a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 -- 172.21.15.119:0/4142276728 >> 172.21.15.119:0/4142276728 conn(0x7f86cc0ce600 msgr2=0x7f86cc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:58.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 -- 172.21.15.119:0/4142276728 shutdown_connections 2023-10-08T01:11:58.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.425+0000 7f86d37ff700 1 -- 172.21.15.119:0/4142276728 wait complete. 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.427+0000 7f86d37ff700 1 Processor -- start 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.427+0000 7f86d37ff700 1 -- start start 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d37ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc06bcd0 0x7f86cc155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d37ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 0x7f86cc0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d37ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc15df50 0x7f86cc159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d37ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc145d00 con 0x7f86cc15df50 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d37ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc156520 con 0x7f86cc06bcd0 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d37ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc16e010 con 0x7f86cc06c0a0 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d1d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc15df50 0x7f86cc159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d1d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc15df50 0x7f86cc159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51622/0 (socket says 172.21.15.119:51622) 2023-10-08T01:11:58.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d1d9c700 1 -- 172.21.15.119:0/794073896 learned_addr learned my addr 172.21.15.119:0/794073896 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d159b700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc06bcd0 0x7f86cc155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d0d9a700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 0x7f86cc0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d1d9c700 1 -- 172.21.15.119:0/794073896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 msgr2=0x7f86cc0b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d1d9c700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 0x7f86cc0b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d1d9c700 1 -- 172.21.15.119:0/794073896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc06bcd0 msgr2=0x7f86cc155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d1d9c700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc06bcd0 0x7f86cc155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.428+0000 7f86d1d9c700 1 -- 172.21.15.119:0/794073896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c80094f0 con 0x7f86cc15df50 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.429+0000 7f86d1d9c700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc15df50 0x7f86cc159990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f86c400a380 tx=0x7f86c4007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.429+0000 7f86c17fa700 1 -- 172.21.15.119:0/794073896 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4015440 con 0x7f86cc15df50 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.429+0000 7f86c17fa700 1 -- 172.21.15.119:0/794073896 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c40155a0 con 0x7f86cc15df50 2023-10-08T01:11:58.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.429+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86cc16e2f0 con 0x7f86cc15df50 2023-10-08T01:11:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.429+0000 7f86c17fa700 1 -- 172.21.15.119:0/794073896 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c4015960 con 0x7f86cc15df50 2023-10-08T01:11:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.429+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86cc16e8b0 con 0x7f86cc15df50 2023-10-08T01:11:58.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.430+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86cc04c3c0 con 0x7f86cc15df50 2023-10-08T01:11:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.431+0000 7f86c17fa700 1 -- 172.21.15.119:0/794073896 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86c400d440 con 0x7f86cc15df50 2023-10-08T01:11:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.432+0000 7f86c17fa700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86b0066f30 0x7f86b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.432+0000 7f86c17fa700 1 -- 172.21.15.119:0/794073896 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f86c4084e60 con 0x7f86cc15df50 2023-10-08T01:11:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.432+0000 7f86d159b700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86b0066f30 0x7f86b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.433+0000 7f86d159b700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86b0066f30 0x7f86b00693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b4600 tx=0x7f86c8017000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:58.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.436+0000 7f86c17fa700 1 -- 172.21.15.119:0/794073896 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86c40522a0 con 0x7f86cc15df50 2023-10-08T01:11:58.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.704+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f86cc06b1d0 con 0x7f86cc15df50 2023-10-08T01:11:58.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.705+0000 7f86c17fa700 1 -- 172.21.15.119:0/794073896 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f86c401a090 con 0x7f86cc15df50 2023-10-08T01:11:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:58.706 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:11:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T01:11:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.706 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T01:11:58.707 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:58.708 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.710 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:58.711 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T01:11:58.713 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.714 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T01:11:58.716 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T01:11:58.717 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T01:11:58.718 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T01:11:58.719 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.720 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:11:58.721 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:11:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:58.722 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T01:11:58.722 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:11:58.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86b0066f30 msgr2=0x7f86b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86b0066f30 0x7f86b00693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f86cc0b4600 tx=0x7f86c8017000 comp rx=0 tx=0).stop 2023-10-08T01:11:58.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc15df50 msgr2=0x7f86cc159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc15df50 0x7f86cc159990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f86c400a380 tx=0x7f86c4007900 comp rx=0 tx=0).stop 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 shutdown_connections 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86b0066f30 0x7f86b00693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc15df50 0x7f86cc159990 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc06bcd0 0x7f86cc155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 --2- 172.21.15.119:0/794073896 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc06c0a0 0x7f86cc0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 >> 172.21.15.119:0/794073896 conn(0x7f86cc0ce600 msgr2=0x7f86cc06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 shutdown_connections 2023-10-08T01:11:58.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.707+0000 7f86d37ff700 1 -- 172.21.15.119:0/794073896 wait complete. 2023-10-08T01:11:58.726 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:11:58.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.905+0000 7f136edff700 1 Processor -- start 2023-10-08T01:11:58.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.905+0000 7f136edff700 1 -- start start 2023-10-08T01:11:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.906+0000 7f136edff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13681578b0 0x7f1368155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.906+0000 7f136edff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 0x7f13680af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.906+0000 7f136edff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13680ad930 con 0x7f1368155c10 2023-10-08T01:11:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.906+0000 7f136edff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13680b0c80 con 0x7f13681578b0 2023-10-08T01:11:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.906+0000 7f136edff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13680acd20 con 0x7f13680af2f0 2023-10-08T01:11:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f1367fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13681578b0 0x7f1368155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f136cb9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1368155c10 0x7f13680ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41650/0 (socket says 172.21.15.119:41650) 2023-10-08T01:11:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f136cb9b700 1 -- 172.21.15.119:0/976120811 learned_addr learned my addr 172.21.15.119:0/976120811 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:58.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f136d39c700 1 --2- 172.21.15.119:0/976120811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 0x7f13680af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f13677fe700 1 -- 172.21.15.119:0/976120811 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204005059 0 0) 0x7f13680ad930 con 0x7f1368155c10 2023-10-08T01:11:58.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f1367fff700 1 -- 172.21.15.119:0/976120811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 msgr2=0x7f13680af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:58.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f1367fff700 1 --2- 172.21.15.119:0/976120811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 0x7f13680af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f1367fff700 1 -- 172.21.15.119:0/976120811 >> v1:172.21.15.119:6789/0 conn(0x7f1368155c10 legacy=0x7f13680ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f1367fff700 1 -- 172.21.15.119:0/976120811 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13680ad000 con 0x7f13681578b0 2023-10-08T01:11:58.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f1367fff700 1 --2- 172.21.15.119:0/976120811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13681578b0 0x7f1368155fe0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f135400aa90 tx=0x7f135400a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c04f62943a251c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:58.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.907+0000 7f13677fe700 1 -- 172.21.15.119:0/976120811 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135400ca40 con 0x7f13681578b0 2023-10-08T01:11:58.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f13677fe700 1 -- 172.21.15.119:0/976120811 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135400cba0 con 0x7f13681578b0 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f13677fe700 1 -- 172.21.15.119:0/976120811 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354018400 con 0x7f13681578b0 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f136edff700 1 -- 172.21.15.119:0/976120811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13681578b0 msgr2=0x7f1368155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f136edff700 1 --2- 172.21.15.119:0/976120811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13681578b0 0x7f1368155fe0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f135400aa90 tx=0x7f135400a330 comp rx=0 tx=0).stop 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f136edff700 1 -- 172.21.15.119:0/976120811 shutdown_connections 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f136edff700 1 --2- 172.21.15.119:0/976120811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 0x7f13680af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f136edff700 1 --2- 172.21.15.119:0/976120811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13681578b0 0x7f1368155fe0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f136edff700 1 -- 172.21.15.119:0/976120811 >> 172.21.15.119:0/976120811 conn(0x7f136806a320 msgr2=0x7f13680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f136edff700 1 -- 172.21.15.119:0/976120811 shutdown_connections 2023-10-08T01:11:58.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.908+0000 7f136edff700 1 -- 172.21.15.119:0/976120811 wait complete. 2023-10-08T01:11:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.910+0000 7f136edff700 1 Processor -- start 2023-10-08T01:11:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.910+0000 7f136edff700 1 -- start start 2023-10-08T01:11:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f136edff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 0x7f136814dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f136edff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1368155c10 0x7f136814c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f136edff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13681578b0 0x7f1368165a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f136edff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136814d820 con 0x7f1368155c10 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f136edff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136814e1f0 con 0x7f13681578b0 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f136edff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136814cc10 con 0x7f13680af2f0 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f1367fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1368155c10 0x7f136814c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f1367fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1368155c10 0x7f136814c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51626/0 (socket says 172.21.15.119:51626) 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f1367fff700 1 -- 172.21.15.119:0/1353164174 learned_addr learned my addr 172.21.15.119:0/1353164174 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f136cb9b700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 0x7f136814dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f136d39c700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13681578b0 0x7f1368165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f1367fff700 1 -- 172.21.15.119:0/1353164174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 msgr2=0x7f136814dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f1367fff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 0x7f136814dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f1367fff700 1 -- 172.21.15.119:0/1353164174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13681578b0 msgr2=0x7f1368165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f1367fff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13681578b0 0x7f1368165a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.911+0000 7f1367fff700 1 -- 172.21.15.119:0/1353164174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135400a5d0 con 0x7f1368155c10 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.912+0000 7f1367fff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1368155c10 0x7f136814c130 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f13680ae0f0 tx=0x7f135401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.912+0000 7f1364ff9700 1 -- 172.21.15.119:0/1353164174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135401c040 con 0x7f1368155c10 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.912+0000 7f1364ff9700 1 -- 172.21.15.119:0/1353164174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135400ce60 con 0x7f1368155c10 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.912+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136814cef0 con 0x7f1368155c10 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.912+0000 7f1364ff9700 1 -- 172.21.15.119:0/1353164174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354010870 con 0x7f1368155c10 2023-10-08T01:11:58.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.912+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136814d4b0 con 0x7f1368155c10 2023-10-08T01:11:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.913+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1368159c00 con 0x7f1368155c10 2023-10-08T01:11:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.914+0000 7f1364ff9700 1 -- 172.21.15.119:0/1353164174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1354010c20 con 0x7f1368155c10 2023-10-08T01:11:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.915+0000 7f1364ff9700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f134c066f30 0x7f134c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.915+0000 7f1364ff9700 1 -- 172.21.15.119:0/1353164174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f1354085a80 con 0x7f1368155c10 2023-10-08T01:11:58.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.915+0000 7f136cb9b700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f134c066f30 0x7f134c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:58.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.916+0000 7f136cb9b700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f134c066f30 0x7f134c0693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13680b0c10 tx=0x7f135c007d30 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:58.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:58.919+0000 7f1364ff9700 1 -- 172.21.15.119:0/1353164174 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1354052e20 con 0x7f1368155c10 2023-10-08T01:11:59.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.108+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f136804c3c0 con 0x7f1368155c10 2023-10-08T01:11:59.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.109+0000 7f1364ff9700 1 -- 172.21.15.119:0/1353164174 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+12779 (secure 0 0 0) 0x7f1354056440 con 0x7f1368155c10 2023-10-08T01:11:59.110 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:59.110 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:11:58.214084+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:11:59.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f134c066f30 msgr2=0x7f134c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f134c066f30 0x7f134c0693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f13680b0c10 tx=0x7f135c007d30 comp rx=0 tx=0).stop 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1368155c10 msgr2=0x7f136814c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1368155c10 0x7f136814c130 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f13680ae0f0 tx=0x7f135401a000 comp rx=0 tx=0).stop 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 shutdown_connections 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f134c066f30 0x7f134c0693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13680af2f0 0x7f136814dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13681578b0 0x7f1368165a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 --2- 172.21.15.119:0/1353164174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1368155c10 0x7f136814c130 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.111+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 >> 172.21.15.119:0/1353164174 conn(0x7f136806a320 msgr2=0x7f1368068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.112+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 shutdown_connections 2023-10-08T01:11:59.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.112+0000 7f136edff700 1 -- 172.21.15.119:0/1353164174 wait complete. 2023-10-08T01:11:59.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T01:11:59.131 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:11:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.315+0000 7f718b9ff700 1 Processor -- start 2023-10-08T01:11:59.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.315+0000 7f718b9ff700 1 -- start start 2023-10-08T01:11:59.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.316+0000 7f718b9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71840c83f0 0x7f71840c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.316+0000 7f718b9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71841568e0 0x7f71840ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.316+0000 7f718b9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718415b0d0 con 0x7f71840ccee0 2023-10-08T01:11:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.316+0000 7f718b9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71840c7a60 con 0x7f71841568e0 2023-10-08T01:11:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.316+0000 7f718b9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718415a4c0 con 0x7f71840c83f0 2023-10-08T01:11:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f7189f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71841568e0 0x7f71840ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f7188f9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f71840ccee0 0x7f71841561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41658/0 (socket says 172.21.15.119:41658) 2023-10-08T01:11:59.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f7188f9a700 1 -- 172.21.15.119:0/2104640552 learned_addr learned my addr 172.21.15.119:0/2104640552 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:11:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f718979b700 1 --2- 172.21.15.119:0/2104640552 >> v2:172.21.15.184:3300/0 conn(0x7f71840c83f0 0x7f71840c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f7189f9c700 1 -- 172.21.15.119:0/2104640552 >> v2:172.21.15.184:3300/0 conn(0x7f71840c83f0 msgr2=0x7f71840c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f7189f9c700 1 --2- 172.21.15.119:0/2104640552 >> v2:172.21.15.184:3300/0 conn(0x7f71840c83f0 0x7f71840c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f7189f9c700 1 -- 172.21.15.119:0/2104640552 >> v1:172.21.15.119:6789/0 conn(0x7f71840ccee0 legacy=0x7f71841561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f7189f9c700 1 -- 172.21.15.119:0/2104640552 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718415a7a0 con 0x7f71841568e0 2023-10-08T01:11:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f718979b700 1 --2- 172.21.15.119:0/2104640552 >> v2:172.21.15.184:3300/0 conn(0x7f71840c83f0 0x7f71840c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:11:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.317+0000 7f717bfff700 1 -- 172.21.15.119:0/2104640552 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421973646 0 0) 0x7f718415b0d0 con 0x7f71840ccee0 2023-10-08T01:11:59.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f7189f9c700 1 --2- 172.21.15.119:0/2104640552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71841568e0 0x7f71840ca1e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f71840c9fd0 tx=0x7f717c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ef2db7e989c937 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f717bfff700 1 -- 172.21.15.119:0/2104640552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c00e070 con 0x7f71841568e0 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f717bfff700 1 -- 172.21.15.119:0/2104640552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00bc30 con 0x7f71841568e0 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f717bfff700 1 -- 172.21.15.119:0/2104640552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c012430 con 0x7f71841568e0 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f718b9ff700 1 -- 172.21.15.119:0/2104640552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71841568e0 msgr2=0x7f71840ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f718b9ff700 1 --2- 172.21.15.119:0/2104640552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71841568e0 0x7f71840ca1e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f71840c9fd0 tx=0x7f717c009780 comp rx=0 tx=0).stop 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f718b9ff700 1 -- 172.21.15.119:0/2104640552 shutdown_connections 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f718b9ff700 1 --2- 172.21.15.119:0/2104640552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71841568e0 0x7f71840ca1e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f718b9ff700 1 --2- 172.21.15.119:0/2104640552 >> v2:172.21.15.184:3300/0 conn(0x7f71840c83f0 0x7f71840c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f718b9ff700 1 -- 172.21.15.119:0/2104640552 >> 172.21.15.119:0/2104640552 conn(0x7f7184068e10 msgr2=0x7f71840cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.318+0000 7f718b9ff700 1 -- 172.21.15.119:0/2104640552 shutdown_connections 2023-10-08T01:11:59.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.319+0000 7f718b9ff700 1 -- 172.21.15.119:0/2104640552 wait complete. 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.320+0000 7f718b9ff700 1 Processor -- start 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.320+0000 7f718b9ff700 1 -- start start 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718b9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840c83f0 0x7f7184169450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718b9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71840ccee0 0x7f71841590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718b9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71841568e0 0x7f71841595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718b9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184169990 con 0x7f71840c83f0 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718b9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7184159b20 con 0x7f71841568e0 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718b9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7184165550 con 0x7f71840ccee0 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718979b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840c83f0 0x7f7184169450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718979b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840c83f0 0x7f7184169450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51632/0 (socket says 172.21.15.119:51632) 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f718979b700 1 -- 172.21.15.119:0/1809554261 learned_addr learned my addr 172.21.15.119:0/1809554261 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f7189f9c700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71841568e0 0x7f71841595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.321+0000 7f7188f9a700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71840ccee0 0x7f71841590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f718979b700 1 -- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71840ccee0 msgr2=0x7f71841590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f718979b700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71840ccee0 0x7f71841590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f718979b700 1 -- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71841568e0 msgr2=0x7f71841595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f718979b700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71841568e0 0x7f71841595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f718979b700 1 -- 172.21.15.119:0/1809554261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c009a20 con 0x7f71840c83f0 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f7189f9c700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71841568e0 0x7f71841595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f718979b700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840c83f0 0x7f7184169450 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f718000c830 tx=0x7f7180008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f71797fa700 1 -- 172.21.15.119:0/1809554261 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180011600 con 0x7f71840c83f0 2023-10-08T01:11:59.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f71797fa700 1 -- 172.21.15.119:0/1809554261 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7180011760 con 0x7f71840c83f0 2023-10-08T01:11:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7184165830 con 0x7f71840c83f0 2023-10-08T01:11:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f71797fa700 1 -- 172.21.15.119:0/1809554261 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7180011b20 con 0x7f71840c83f0 2023-10-08T01:11:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.322+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7184165e70 con 0x7f71840c83f0 2023-10-08T01:11:59.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.323+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718406bb10 con 0x7f71840c83f0 2023-10-08T01:11:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.324+0000 7f71797fa700 1 -- 172.21.15.119:0/1809554261 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f718000a3e0 con 0x7f71840c83f0 2023-10-08T01:11:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.325+0000 7f71797fa700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7168066f30 0x7f71680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.325+0000 7f71797fa700 1 -- 172.21.15.119:0/1809554261 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f71800848c0 con 0x7f71840c83f0 2023-10-08T01:11:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.325+0000 7f7188f9a700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7168066f30 0x7f71680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.326+0000 7f7188f9a700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7168066f30 0x7f71680693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71840c34b0 tx=0x7f7174009040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:59.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.329+0000 7f71797fa700 1 -- 172.21.15.119:0/1809554261 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7180051c60 con 0x7f71840c83f0 2023-10-08T01:11:59.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.519+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f718404c3c0 con 0x7f71840c83f0 2023-10-08T01:11:59.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.520+0000 7f71797fa700 1 -- 172.21.15.119:0/1809554261 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f71800150a0 con 0x7f71840c83f0 2023-10-08T01:11:59.521 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:11:59.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7168066f30 msgr2=0x7f71680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7168066f30 0x7f71680693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71840c34b0 tx=0x7f7174009040 comp rx=0 tx=0).stop 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840c83f0 msgr2=0x7f7184169450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840c83f0 0x7f7184169450 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f718000c830 tx=0x7f7180008040 comp rx=0 tx=0).stop 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 shutdown_connections 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7168066f30 0x7f71680693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71841568e0 0x7f71841595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71840c83f0 0x7f7184169450 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 --2- 172.21.15.119:0/1809554261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71840ccee0 0x7f71841590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 >> 172.21.15.119:0/1809554261 conn(0x7f7184068e10 msgr2=0x7f7184151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:59.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 shutdown_connections 2023-10-08T01:11:59.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.522+0000 7f718b9ff700 1 -- 172.21.15.119:0/1809554261 wait complete. 2023-10-08T01:11:59.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T01:11:59.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.721+0000 7fc1db59e700 1 Processor -- start 2023-10-08T01:11:59.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.721+0000 7fc1db59e700 1 -- start start 2023-10-08T01:11:59.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.722+0000 7fc1db59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc1dc159420 0x7fc1dc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.722+0000 7fc1db59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc1dc15d230 0x7fc1dc15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.722+0000 7fc1db59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1dc0b2980 con 0x7fc1dc1597f0 2023-10-08T01:11:59.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.722+0000 7fc1db59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1dc15db70 con 0x7fc1dc159420 2023-10-08T01:11:59.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.722+0000 7fc1db59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1dc0b11d0 con 0x7fc1dc15d230 2023-10-08T01:11:59.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.722+0000 7fc1da59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc1dc159420 0x7fc1dc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.722+0000 7fc1da59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc1dc159420 0x7fc1dc0b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51644/0 (socket says 172.21.15.119:51644) 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.722+0000 7fc1da59c700 1 -- 172.21.15.119:0/3397061073 learned_addr learned my addr 172.21.15.119:0/3397061073 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1dad9d700 1 --2- 172.21.15.119:0/3397061073 >> v2:172.21.15.184:3300/0 conn(0x7fc1dc15d230 0x7fc1dc15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1da59c700 1 -- 172.21.15.119:0/3397061073 >> v2:172.21.15.184:3300/0 conn(0x7fc1dc15d230 msgr2=0x7fc1dc15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1da59c700 1 --2- 172.21.15.119:0/3397061073 >> v2:172.21.15.184:3300/0 conn(0x7fc1dc15d230 0x7fc1dc15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1da59c700 1 -- 172.21.15.119:0/3397061073 >> v1:172.21.15.184:6789/0 conn(0x7fc1dc1597f0 legacy=0x7fc1dc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1da59c700 1 -- 172.21.15.119:0/3397061073 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1dc0b14b0 con 0x7fc1dc159420 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1da59c700 1 --2- 172.21.15.119:0/3397061073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc159420 0x7fc1dc0b5f40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15d170 tx=0x7fc1cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d54493f1c0fa9c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1d959a700 1 -- 172.21.15.119:0/3397061073 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00b820 con 0x7fc1dc159420 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1d959a700 1 -- 172.21.15.119:0/3397061073 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1cc00b980 con 0x7fc1dc159420 2023-10-08T01:11:59.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.723+0000 7fc1d959a700 1 -- 172.21.15.119:0/3397061073 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00bd20 con 0x7fc1dc159420 2023-10-08T01:11:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.724+0000 7fc1db59e700 1 -- 172.21.15.119:0/3397061073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc159420 msgr2=0x7fc1dc0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.724+0000 7fc1db59e700 1 --2- 172.21.15.119:0/3397061073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc159420 0x7fc1dc0b5f40 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15d170 tx=0x7fc1cc009250 comp rx=0 tx=0).stop 2023-10-08T01:11:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.724+0000 7fc1db59e700 1 -- 172.21.15.119:0/3397061073 shutdown_connections 2023-10-08T01:11:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.724+0000 7fc1db59e700 1 --2- 172.21.15.119:0/3397061073 >> v2:172.21.15.184:3300/0 conn(0x7fc1dc15d230 0x7fc1dc15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.724+0000 7fc1db59e700 1 --2- 172.21.15.119:0/3397061073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc159420 0x7fc1dc0b5f40 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.724+0000 7fc1db59e700 1 -- 172.21.15.119:0/3397061073 >> 172.21.15.119:0/3397061073 conn(0x7fc1dc0d0660 msgr2=0x7fc1dc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.724+0000 7fc1db59e700 1 -- 172.21.15.119:0/3397061073 shutdown_connections 2023-10-08T01:11:59.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.724+0000 7fc1db59e700 1 -- 172.21.15.119:0/3397061073 wait complete. 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.725+0000 7fc1db59e700 1 Processor -- start 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.726+0000 7fc1db59e700 1 -- start start 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.726+0000 7fc1db59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1dc159420 0x7fc1dc0c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.726+0000 7fc1db59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1597f0 0x7fc1dc0c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.726+0000 7fc1db59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1dc15d230 0x7fc1dc169bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.726+0000 7fc1db59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc0cf8e0 con 0x7fc1dc1597f0 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.726+0000 7fc1db59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc0d02a0 con 0x7fc1dc159420 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.726+0000 7fc1db59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1dc0ced60 con 0x7fc1dc15d230 2023-10-08T01:11:59.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1597f0 0x7fc1dc0c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1597f0 0x7fc1dc0c6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51650/0 (socket says 172.21.15.119:51650) 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 -- 172.21.15.119:0/2839380783 learned_addr learned my addr 172.21.15.119:0/2839380783 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1da59c700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1dc159420 0x7fc1dc0c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1dad9d700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1dc15d230 0x7fc1dc169bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 -- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1dc15d230 msgr2=0x7fc1dc169bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1dc15d230 0x7fc1dc169bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 -- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1dc159420 msgr2=0x7fc1dc0c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1dc159420 0x7fc1dc0c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 -- 172.21.15.119:0/2839380783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc0094f0 con 0x7fc1dc1597f0 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1da59c700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1dc159420 0x7fc1dc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:11:59.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.727+0000 7fc1d9d9b700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1597f0 0x7fc1dc0c6f90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c590 tx=0x7fc1d0007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:59.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.728+0000 7fc1ca7fc700 1 -- 172.21.15.119:0/2839380783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0011660 con 0x7fc1dc1597f0 2023-10-08T01:11:59.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.728+0000 7fc1ca7fc700 1 -- 172.21.15.119:0/2839380783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d00117c0 con 0x7fc1dc1597f0 2023-10-08T01:11:59.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.728+0000 7fc1ca7fc700 1 -- 172.21.15.119:0/2839380783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0011b80 con 0x7fc1dc1597f0 2023-10-08T01:11:59.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.728+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1dc0cf040 con 0x7fc1dc1597f0 2023-10-08T01:11:59.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.728+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1dc0cf600 con 0x7fc1dc1597f0 2023-10-08T01:11:59.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.729+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1dc04c3c0 con 0x7fc1dc1597f0 2023-10-08T01:11:59.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.730+0000 7fc1ca7fc700 1 -- 172.21.15.119:0/2839380783 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1d001c460 con 0x7fc1dc1597f0 2023-10-08T01:11:59.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.730+0000 7fc1ca7fc700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:11:59.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.730+0000 7fc1ca7fc700 1 -- 172.21.15.119:0/2839380783 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fc1d00842b0 con 0x7fc1dc1597f0 2023-10-08T01:11:59.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.731+0000 7fc1da59c700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:11:59.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.731+0000 7fc1da59c700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc1bc066f30 0x7fc1bc0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15d170 tx=0x7fc1cc016000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:11:59.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.735+0000 7fc1ca7fc700 1 -- 172.21.15.119:0/2839380783 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1d00516f0 con 0x7fc1dc1597f0 2023-10-08T01:11:59.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.929+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc1dc0b1790 con 0x7fc1dc1597f0 2023-10-08T01:11:59.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.929+0000 7fc1ca7fc700 1 -- 172.21.15.119:0/2839380783 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v87) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc1d0054d10 con 0x7fc1dc1597f0 2023-10-08T01:11:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:11:59.931 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:11:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:11:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T01:11:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T01:11:59.931 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:11:59.931 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:11:59.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc1bc066f30 msgr2=0x7fc1bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc1bc066f30 0x7fc1bc0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc15d170 tx=0x7fc1cc016000 comp rx=0 tx=0).stop 2023-10-08T01:11:59.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1597f0 msgr2=0x7fc1dc0c6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:11:59.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1597f0 0x7fc1dc0c6f90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c590 tx=0x7fc1d0007f30 comp rx=0 tx=0).stop 2023-10-08T01:11:59.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 shutdown_connections 2023-10-08T01:11:59.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc1bc066f30 0x7fc1bc0693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc1dc15d230 0x7fc1dc169bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc1dc159420 0x7fc1dc0c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 --2- 172.21.15.119:0/2839380783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc1dc1597f0 0x7fc1dc0c6f90 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:11:59.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 >> 172.21.15.119:0/2839380783 conn(0x7fc1dc0d0660 msgr2=0x7fc1dc1578a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:11:59.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 shutdown_connections 2023-10-08T01:11:59.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:11:59.931+0000 7fc1db59e700 1 -- 172.21.15.119:0/2839380783 wait complete. 2023-10-08T01:11:59.950 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T01:11:59.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T01:12:00.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.134+0000 7fee93fff700 1 Processor -- start 2023-10-08T01:12:00.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.134+0000 7fee93fff700 1 -- start start 2023-10-08T01:12:00.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.135+0000 7fee93fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee940b03c0 0x7fee940cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:00.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.135+0000 7fee93fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 0x7fee94159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:00.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.135+0000 7fee93fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee940cfa00 con 0x7fee940b0790 2023-10-08T01:12:00.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.135+0000 7fee93fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee940cb580 con 0x7fee940b03c0 2023-10-08T01:12:00.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.135+0000 7fee93fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee940cf160 con 0x7fee941591e0 2023-10-08T01:12:00.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.135+0000 7fee92ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee940b03c0 0x7fee940cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:00.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.135+0000 7fee8bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fee940b0790 0x7fee940b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41666/0 (socket says 172.21.15.119:41666) 2023-10-08T01:12:00.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee8bfff700 1 -- 172.21.15.119:0/3352771343 learned_addr learned my addr 172.21.15.119:0/3352771343 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:00.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee937fe700 1 --2- 172.21.15.119:0/3352771343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 0x7fee94159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:00.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee927fc700 1 -- 172.21.15.119:0/3352771343 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3624738235 0 0) 0x7fee940cfa00 con 0x7fee940b0790 2023-10-08T01:12:00.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee92ffd700 1 -- 172.21.15.119:0/3352771343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 msgr2=0x7fee94159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:00.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee92ffd700 1 --2- 172.21.15.119:0/3352771343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 0x7fee94159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee92ffd700 1 -- 172.21.15.119:0/3352771343 >> v1:172.21.15.119:6789/0 conn(0x7fee940b0790 legacy=0x7fee940b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:00.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee92ffd700 1 -- 172.21.15.119:0/3352771343 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940cf440 con 0x7fee940b03c0 2023-10-08T01:12:00.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee92ffd700 1 --2- 172.21.15.119:0/3352771343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b03c0 0x7fee940cea70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fee84009910 tx=0x7fee840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff51a3c6948567c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:00.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.136+0000 7fee927fc700 1 -- 172.21.15.119:0/3352771343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8400b820 con 0x7fee940b03c0 2023-10-08T01:12:00.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee927fc700 1 -- 172.21.15.119:0/3352771343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8400b980 con 0x7fee940b03c0 2023-10-08T01:12:00.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee927fc700 1 -- 172.21.15.119:0/3352771343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8400bd20 con 0x7fee940b03c0 2023-10-08T01:12:00.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee93fff700 1 -- 172.21.15.119:0/3352771343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b03c0 msgr2=0x7fee940cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:00.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee93fff700 1 --2- 172.21.15.119:0/3352771343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b03c0 0x7fee940cea70 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fee84009910 tx=0x7fee840091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee93fff700 1 -- 172.21.15.119:0/3352771343 shutdown_connections 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee93fff700 1 --2- 172.21.15.119:0/3352771343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 0x7fee94159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee93fff700 1 --2- 172.21.15.119:0/3352771343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b03c0 0x7fee940cea70 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee93fff700 1 -- 172.21.15.119:0/3352771343 >> 172.21.15.119:0/3352771343 conn(0x7fee94063230 msgr2=0x7fee94063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee93fff700 1 -- 172.21.15.119:0/3352771343 shutdown_connections 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.137+0000 7fee93fff700 1 -- 172.21.15.119:0/3352771343 wait complete. 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.139+0000 7fee93fff700 1 Processor -- start 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.139+0000 7fee93fff700 1 -- start start 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.139+0000 7fee93fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee940b03c0 0x7fee940c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:00.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee93fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b0790 0x7fee9414b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:00.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee93fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 0x7fee940c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:00.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee93fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9415c4c0 con 0x7fee940b0790 2023-10-08T01:12:00.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee93fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9415c650 con 0x7fee940b03c0 2023-10-08T01:12:00.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee93fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee941598b0 con 0x7fee941591e0 2023-10-08T01:12:00.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee8bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b0790 0x7fee9414b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:00.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee8bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b0790 0x7fee9414b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51672/0 (socket says 172.21.15.119:51672) 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee8bfff700 1 -- 172.21.15.119:0/682332953 learned_addr learned my addr 172.21.15.119:0/682332953 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee937fe700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 0x7fee940c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee92ffd700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee940b03c0 0x7fee940c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee8bfff700 1 -- 172.21.15.119:0/682332953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 msgr2=0x7fee940c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee8bfff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 0x7fee940c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee8bfff700 1 -- 172.21.15.119:0/682332953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee940b03c0 msgr2=0x7fee940c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee8bfff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee940b03c0 0x7fee940c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.140+0000 7fee8bfff700 1 -- 172.21.15.119:0/682332953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84009420 con 0x7fee940b0790 2023-10-08T01:12:00.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.141+0000 7fee8bfff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b0790 0x7fee9414b840 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c7e0 tx=0x7fee7c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:00.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.141+0000 7fee8affd700 1 -- 172.21.15.119:0/682332953 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c005b30 con 0x7fee940b0790 2023-10-08T01:12:00.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.141+0000 7fee8affd700 1 -- 172.21.15.119:0/682332953 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7c005c90 con 0x7fee940b0790 2023-10-08T01:12:00.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.141+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee940c4550 con 0x7fee940b0790 2023-10-08T01:12:00.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.141+0000 7fee8affd700 1 -- 172.21.15.119:0/682332953 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7c007300 con 0x7fee940b0790 2023-10-08T01:12:00.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.141+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee940c4b10 con 0x7fee940b0790 2023-10-08T01:12:00.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.142+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9415c800 con 0x7fee940b0790 2023-10-08T01:12:00.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.143+0000 7fee8affd700 1 -- 172.21.15.119:0/682332953 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee7c01b3b0 con 0x7fee940b0790 2023-10-08T01:12:00.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.144+0000 7fee8affd700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee74066f30 0x7fee740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:00.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.144+0000 7fee8affd700 1 -- 172.21.15.119:0/682332953 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fee7c083920 con 0x7fee940b0790 2023-10-08T01:12:00.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.144+0000 7fee92ffd700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee74066f30 0x7fee740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:00.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.145+0000 7fee92ffd700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee74066f30 0x7fee740693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee840091e0 tx=0x7fee84017000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:00.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.148+0000 7fee8affd700 1 -- 172.21.15.119:0/682332953 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee9415c800 con 0x7fee940b0790 2023-10-08T01:12:00.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.336+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fee9415c7c0 con 0x7fee940b0790 2023-10-08T01:12:00.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.801+0000 7fee8affd700 1 -- 172.21.15.119:0/682332953 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v88) v1 ==== 138+0+0 (secure 0 0 0) 0x7fee7c017020 con 0x7fee940b0790 2023-10-08T01:12:00.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee74066f30 msgr2=0x7fee740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:00.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee74066f30 0x7fee740693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fee840091e0 tx=0x7fee84017000 comp rx=0 tx=0).stop 2023-10-08T01:12:00.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b0790 msgr2=0x7fee9414b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:00.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b0790 0x7fee9414b840 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00c7e0 tx=0x7fee7c007df0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 shutdown_connections 2023-10-08T01:12:00.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee74066f30 0x7fee740693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee941591e0 0x7fee940c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee940b03c0 0x7fee940c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 --2- 172.21.15.119:0/682332953 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee940b0790 0x7fee9414b840 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:00.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 >> 172.21.15.119:0/682332953 conn(0x7fee94063230 msgr2=0x7fee940cba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:00.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 shutdown_connections 2023-10-08T01:12:00.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:00.804+0000 7fee93fff700 1 -- 172.21.15.119:0/682332953 wait complete. 2023-10-08T01:12:00.809 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T01:12:00.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T01:12:01.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.013+0000 7f68aafff700 1 Processor -- start 2023-10-08T01:12:01.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.013+0000 7f68aafff700 1 -- start start 2023-10-08T01:12:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.014+0000 7f68aafff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68a4157630 0x7f68a4158a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.014+0000 7f68aafff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f68a4157a00 0x7f68a40abd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68aafff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a4158590 con 0x7f68a40ac290 2023-10-08T01:12:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68aafff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a4157f80 con 0x7f68a4157630 2023-10-08T01:12:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68aafff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a40ac7a0 con 0x7f68a4157a00 2023-10-08T01:12:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a959c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f68a40ac290 0x7f68a4068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41682/0 (socket says 172.21.15.119:41682) 2023-10-08T01:12:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a8d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68a4157630 0x7f68a4158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:01.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a959c700 1 -- 172.21.15.119:0/1918919974 learned_addr learned my addr 172.21.15.119:0/1918919974 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:01.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a3fff700 1 --2- 172.21.15.119:0/1918919974 >> v2:172.21.15.184:3300/0 conn(0x7f68a4157a00 0x7f68a40abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:01.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a37fe700 1 -- 172.21.15.119:0/1918919974 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773021754 0 0) 0x7f68a4158590 con 0x7f68a40ac290 2023-10-08T01:12:01.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a8d9b700 1 -- 172.21.15.119:0/1918919974 >> v2:172.21.15.184:3300/0 conn(0x7f68a4157a00 msgr2=0x7f68a40abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a8d9b700 1 --2- 172.21.15.119:0/1918919974 >> v2:172.21.15.184:3300/0 conn(0x7f68a4157a00 0x7f68a40abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a8d9b700 1 -- 172.21.15.119:0/1918919974 >> v1:172.21.15.119:6789/0 conn(0x7f68a40ac290 legacy=0x7f68a4068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.015+0000 7f68a8d9b700 1 -- 172.21.15.119:0/1918919974 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4158220 con 0x7f68a4157630 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68a8d9b700 1 --2- 172.21.15.119:0/1918919974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a4157630 0x7f68a4158a20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f68a40c8cf0 tx=0x7f68980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1827404f121d23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68a37fe700 1 -- 172.21.15.119:0/1918919974 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800b6b0 con 0x7f68a4157630 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68a37fe700 1 -- 172.21.15.119:0/1918919974 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689800b810 con 0x7f68a4157630 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68a37fe700 1 -- 172.21.15.119:0/1918919974 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800bbb0 con 0x7f68a4157630 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68aafff700 1 -- 172.21.15.119:0/1918919974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a4157630 msgr2=0x7f68a4158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68aafff700 1 --2- 172.21.15.119:0/1918919974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a4157630 0x7f68a4158a20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f68a40c8cf0 tx=0x7f68980091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68aafff700 1 -- 172.21.15.119:0/1918919974 shutdown_connections 2023-10-08T01:12:01.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68aafff700 1 --2- 172.21.15.119:0/1918919974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a4157630 0x7f68a4158a20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68aafff700 1 --2- 172.21.15.119:0/1918919974 >> v2:172.21.15.184:3300/0 conn(0x7f68a4157a00 0x7f68a40abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.016+0000 7f68aafff700 1 -- 172.21.15.119:0/1918919974 >> 172.21.15.119:0/1918919974 conn(0x7f68a40cc540 msgr2=0x7f68a40cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:01.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.017+0000 7f68aafff700 1 -- 172.21.15.119:0/1918919974 shutdown_connections 2023-10-08T01:12:01.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.017+0000 7f68aafff700 1 -- 172.21.15.119:0/1918919974 wait complete. 2023-10-08T01:12:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.018+0000 7f68aafff700 1 Processor -- start 2023-10-08T01:12:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.018+0000 7f68aafff700 1 -- start start 2023-10-08T01:12:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68aafff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a40ac290 0x7f68a416aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68aafff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a4157630 0x7f68a415b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68aafff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68a4157a00 0x7f68a40c4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68aafff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a41553c0 con 0x7f68a40ac290 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68aafff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a40a54d0 con 0x7f68a4157630 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68aafff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a416af60 con 0x7f68a4157a00 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68a8d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a40ac290 0x7f68a416aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68a8d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a40ac290 0x7f68a416aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51692/0 (socket says 172.21.15.119:51692) 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68a8d9b700 1 -- 172.21.15.119:0/4099752961 learned_addr learned my addr 172.21.15.119:0/4099752961 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68a3fff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a4157630 0x7f68a415b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68a959c700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68a4157a00 0x7f68a40c4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.019+0000 7f68a8d9b700 1 -- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68a4157a00 msgr2=0x7f68a40c4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a8d9b700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68a4157a00 0x7f68a40c4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a8d9b700 1 -- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a4157630 msgr2=0x7f68a415b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a8d9b700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a4157630 0x7f68a415b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a8d9b700 1 -- 172.21.15.119:0/4099752961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898009420 con 0x7f68a40ac290 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a3fff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a4157630 0x7f68a415b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a8d9b700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a40ac290 0x7f68a416aa20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f68a406a9e0 tx=0x7f6898017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a0ff9700 1 -- 172.21.15.119:0/4099752961 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898011420 con 0x7f68a40ac290 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a0ff9700 1 -- 172.21.15.119:0/4099752961 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6898011580 con 0x7f68a40ac290 2023-10-08T01:12:01.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68a0ff9700 1 -- 172.21.15.119:0/4099752961 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898011940 con 0x7f68a40ac290 2023-10-08T01:12:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a415ca40 con 0x7f68a40ac290 2023-10-08T01:12:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.020+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a415c190 con 0x7f68a40ac290 2023-10-08T01:12:01.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.021+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a40718b0 con 0x7f68a40ac290 2023-10-08T01:12:01.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.022+0000 7f68a0ff9700 1 -- 172.21.15.119:0/4099752961 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6898011d30 con 0x7f68a40ac290 2023-10-08T01:12:01.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.023+0000 7f68a0ff9700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6888066f60 0x7f6888069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:01.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.023+0000 7f68a0ff9700 1 -- 172.21.15.119:0/4099752961 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5615+0+0 (secure 0 0 0) 0x7f6898085fc0 con 0x7f68a40ac290 2023-10-08T01:12:01.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.023+0000 7f68a3fff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6888066f60 0x7f6888069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:01.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.024+0000 7f68a3fff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6888066f60 0x7f6888069410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68a40b09d0 tx=0x7f6890008040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:01.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.028+0000 7f68a0ff9700 1 -- 172.21.15.119:0/4099752961 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68980531f0 con 0x7f68a40ac290 2023-10-08T01:12:01.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.221+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f68a40aca80 con 0x7f68a40ac290 2023-10-08T01:12:01.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.808+0000 7f68a0ff9700 1 -- 172.21.15.119:0/4099752961 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6898056810 con 0x7f68a40ac290 2023-10-08T01:12:01.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6888066f60 msgr2=0x7f6888069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:01.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6888066f60 0x7f6888069410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68a40b09d0 tx=0x7f6890008040 comp rx=0 tx=0).stop 2023-10-08T01:12:01.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a40ac290 msgr2=0x7f68a416aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:01.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a40ac290 0x7f68a416aa20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f68a406a9e0 tx=0x7f6898017000 comp rx=0 tx=0).stop 2023-10-08T01:12:01.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 shutdown_connections 2023-10-08T01:12:01.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6888066f60 0x7f6888069410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f68a4157a00 0x7f68a40c4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68a40ac290 0x7f68a416aa20 secure :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f68a406a9e0 tx=0x7f6898017000 comp rx=0 tx=0).stop 2023-10-08T01:12:01.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 --2- 172.21.15.119:0/4099752961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68a4157630 0x7f68a415b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:01.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.816+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 >> 172.21.15.119:0/4099752961 conn(0x7f68a40cc540 msgr2=0x7f68a4155680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:01.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.820+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 shutdown_connections 2023-10-08T01:12:01.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:01.823+0000 7f68aafff700 1 -- 172.21.15.119:0/4099752961 wait complete. 2023-10-08T01:12:01.828 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T01:12:01.847 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T01:12:02.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.030+0000 7f9d7a3ff700 1 Processor -- start 2023-10-08T01:12:02.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.030+0000 7f9d7a3ff700 1 -- start start 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.031+0000 7f9d7a3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9d74159890 0x7f9d7415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.031+0000 7f9d7a3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d7415a160 con 0x7f9d74159c60 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.031+0000 7f9d7a3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d740d13a0 con 0x7f9d74069560 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.032+0000 7f9d7a3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d7415e140 con 0x7f9d74159890 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.032+0000 7f9d737fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9d74159c60 0x7f9d7415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41684/0 (socket says 172.21.15.119:41684) 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.032+0000 7f9d737fe700 1 -- 172.21.15.119:0/4025599968 learned_addr learned my addr 172.21.15.119:0/4025599968 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.032+0000 7f9d73fff700 1 --2- 172.21.15.119:0/4025599968 >> v2:172.21.15.184:3300/0 conn(0x7f9d74159890 0x7f9d7415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.032+0000 7f9d72ffd700 1 -- 172.21.15.119:0/4025599968 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2516508016 0 0) 0x7f9d7415a160 con 0x7f9d74159c60 2023-10-08T01:12:02.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.032+0000 7f9d72ffd700 1 -- 172.21.15.119:0/4025599968 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d5c003570 con 0x7f9d74159c60 2023-10-08T01:12:02.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.032+0000 7f9d72ffd700 1 -- 172.21.15.119:0/4025599968 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142615540 0 0) 0x7f9d740d13a0 con 0x7f9d74069560 2023-10-08T01:12:02.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.032+0000 7f9d72ffd700 1 -- 172.21.15.119:0/4025599968 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d7415a160 con 0x7f9d74069560 2023-10-08T01:12:02.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.033+0000 7f9d73fff700 1 -- 172.21.15.119:0/4025599968 >> v1:172.21.15.184:6789/0 conn(0x7f9d74069560 legacy=0x7f9d740ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:02.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.033+0000 7f9d73fff700 1 -- 172.21.15.119:0/4025599968 >> v1:172.21.15.119:6789/0 conn(0x7f9d74159c60 legacy=0x7f9d7415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:02.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.033+0000 7f9d72ffd700 1 -- 172.21.15.119:0/4025599968 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 182560631 0 0) 0x7f9d5c003570 con 0x7f9d74159c60 2023-10-08T01:12:02.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.033+0000 7f9d73fff700 1 -- 172.21.15.119:0/4025599968 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7415a3d0 con 0x7f9d74159890 2023-10-08T01:12:02.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.033+0000 7f9d73fff700 1 --2- 172.21.15.119:0/4025599968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159890 0x7f9d7415abd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d640098e0 tx=0x7f9d640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c746278280c893 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:02.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.033+0000 7f9d72ffd700 1 -- 172.21.15.119:0/4025599968 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68001470 con 0x7f9d74159890 2023-10-08T01:12:02.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.033+0000 7f9d72ffd700 1 -- 172.21.15.119:0/4025599968 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d64011420 con 0x7f9d74159890 2023-10-08T01:12:02.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.033+0000 7f9d72ffd700 1 -- 172.21.15.119:0/4025599968 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d640117c0 con 0x7f9d74159890 2023-10-08T01:12:02.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.034+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/4025599968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159890 msgr2=0x7f9d7415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:02.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.034+0000 7f9d7a3ff700 1 --2- 172.21.15.119:0/4025599968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159890 0x7f9d7415abd0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9d640098e0 tx=0x7f9d640091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:02.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.034+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/4025599968 shutdown_connections 2023-10-08T01:12:02.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.034+0000 7f9d7a3ff700 1 --2- 172.21.15.119:0/4025599968 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159890 0x7f9d7415abd0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:02.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.034+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/4025599968 >> 172.21.15.119:0/4025599968 conn(0x7f9d74068e20 msgr2=0x7f9d740d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:02.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.034+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/4025599968 shutdown_connections 2023-10-08T01:12:02.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.034+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/4025599968 wait complete. 2023-10-08T01:12:02.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.035+0000 7f9d7a3ff700 1 Processor -- start 2023-10-08T01:12:02.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.035+0000 7f9d7a3ff700 1 -- start start 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.036+0000 7f9d7a3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74069560 0x7f9d740ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.036+0000 7f9d7a3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74159890 0x7f9d7415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.036+0000 7f9d7a3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159c60 0x7f9d740b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.036+0000 7f9d7a3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7416a250 con 0x7f9d74069560 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.036+0000 7f9d7a3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7416a530 con 0x7f9d74159c60 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.036+0000 7f9d7a3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d7416a810 con 0x7f9d74159890 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74069560 0x7f9d740ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74069560 0x7f9d740ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51704/0 (socket says 172.21.15.119:51704) 2023-10-08T01:12:02.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 -- 172.21.15.119:0/1105686977 learned_addr learned my addr 172.21.15.119:0/1105686977 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 -- 172.21.15.119:0/1105686977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74159890 msgr2=0x7f9d7415cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74159890 0x7f9d7415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 -- 172.21.15.119:0/1105686977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159c60 msgr2=0x7f9d740b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d7899c700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159c60 0x7f9d740b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159c60 0x7f9d740b3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 -- 172.21.15.119:0/1105686977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d64009420 con 0x7f9d74069560 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d7899c700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159c60 0x7f9d740b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d73fff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74069560 0x7f9d740ce9e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9d74070e20 tx=0x7f9d64016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:02.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.037+0000 7f9d5bfff700 1 -- 172.21.15.119:0/1105686977 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6401cc90 con 0x7f9d74069560 2023-10-08T01:12:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.038+0000 7f9d5bfff700 1 -- 172.21.15.119:0/1105686977 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6401cdf0 con 0x7f9d74069560 2023-10-08T01:12:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.038+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7416aaf0 con 0x7f9d74069560 2023-10-08T01:12:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.038+0000 7f9d5bfff700 1 -- 172.21.15.119:0/1105686977 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6401a6a0 con 0x7f9d74069560 2023-10-08T01:12:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.038+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d740cef20 con 0x7f9d74069560 2023-10-08T01:12:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.039+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7404c3c0 con 0x7f9d74069560 2023-10-08T01:12:02.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.040+0000 7f9d5bfff700 1 -- 172.21.15.119:0/1105686977 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d6401aa50 con 0x7f9d74069560 2023-10-08T01:12:02.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.041+0000 7f9d5bfff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d60066f30 0x7f9d600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:02.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.041+0000 7f9d5bfff700 1 -- 172.21.15.119:0/1105686977 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5997+0+0 (secure 0 0 0) 0x7f9d64086320 con 0x7f9d74069560 2023-10-08T01:12:02.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.041+0000 7f9d737fe700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d60066f30 0x7f9d600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:02.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.041+0000 7f9d737fe700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d60066f30 0x7f9d600693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d68000ff0 tx=0x7f9d68001430 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:02.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.045+0000 7f9d5bfff700 1 -- 172.21.15.119:0/1105686977 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d640534f0 con 0x7f9d74069560 2023-10-08T01:12:02.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.277+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9d7416a390 con 0x7f9d74069560 2023-10-08T01:12:02.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.840+0000 7f9d5bfff700 1 -- 172.21.15.119:0/1105686977 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v36) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9d7416a390 con 0x7f9d74069560 2023-10-08T01:12:02.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d60066f30 msgr2=0x7f9d600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:02.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d60066f30 0x7f9d600693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d68000ff0 tx=0x7f9d68001430 comp rx=0 tx=0).stop 2023-10-08T01:12:02.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74069560 msgr2=0x7f9d740ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:02.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74069560 0x7f9d740ce9e0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9d74070e20 tx=0x7f9d64016000 comp rx=0 tx=0).stop 2023-10-08T01:12:02.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 shutdown_connections 2023-10-08T01:12:02.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d60066f30 0x7f9d600693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:02.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d74159c60 0x7f9d740b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:02.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d74069560 0x7f9d740ce9e0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:02.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 --2- 172.21.15.119:0/1105686977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d74159890 0x7f9d7415cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:02.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.848+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 >> 172.21.15.119:0/1105686977 conn(0x7f9d74068e20 msgr2=0x7f9d740c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:02.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.849+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 shutdown_connections 2023-10-08T01:12:02.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:02.849+0000 7f9d7a3ff700 1 -- 172.21.15.119:0/1105686977 wait complete. 2023-10-08T01:12:02.852 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 10 and data pool 11 2023-10-08T01:12:02.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:03.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.053+0000 7f6eb8bff700 1 Processor -- start 2023-10-08T01:12:03.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.053+0000 7f6eb8bff700 1 -- start start 2023-10-08T01:12:03.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.055+0000 7f6eb8bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.055+0000 7f6eb8bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40cc300 con 0x7f6eb40ccf90 2023-10-08T01:12:03.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.055+0000 7f6eb8bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb40ad370 con 0x7f6eb40b1420 2023-10-08T01:12:03.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.055+0000 7f6eb8bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4156bf0 con 0x7f6eb40cbf30 2023-10-08T01:12:03.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.055+0000 7f6eabfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6eb40ccf90 0x7f6eb40b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41686/0 (socket says 172.21.15.119:41686) 2023-10-08T01:12:03.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.055+0000 7f6eabfff700 1 -- 172.21.15.119:0/4184600467 learned_addr learned my addr 172.21.15.119:0/4184600467 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:03.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.055+0000 7f6eb259c700 1 --2- 172.21.15.119:0/4184600467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.055+0000 7f6eb1d9b700 1 -- 172.21.15.119:0/4184600467 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817219081 0 0) 0x7f6eb40cc300 con 0x7f6eb40ccf90 2023-10-08T01:12:03.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb1d9b700 1 -- 172.21.15.119:0/4184600467 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea0003570 con 0x7f6eb40ccf90 2023-10-08T01:12:03.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb1d9b700 1 -- 172.21.15.119:0/4184600467 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127557011 0 0) 0x7f6eb40ad370 con 0x7f6eb40b1420 2023-10-08T01:12:03.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb1d9b700 1 -- 172.21.15.119:0/4184600467 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eb40cc300 con 0x7f6eb40b1420 2023-10-08T01:12:03.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb259c700 1 -- 172.21.15.119:0/4184600467 >> v1:172.21.15.184:6789/0 conn(0x7f6eb40b1420 legacy=0x7f6eb40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb259c700 1 -- 172.21.15.119:0/4184600467 >> v1:172.21.15.119:6789/0 conn(0x7f6eb40ccf90 legacy=0x7f6eb40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb259c700 1 -- 172.21.15.119:0/4184600467 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb40ad650 con 0x7f6eb40cbf30 2023-10-08T01:12:03.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb259c700 1 --2- 172.21.15.119:0/4184600467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb40ae410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009910 tx=0x7f6ea40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9bb5ec891add1c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb1d9b700 1 -- 172.21.15.119:0/4184600467 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011630 con 0x7f6eb40cbf30 2023-10-08T01:12:03.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb1d9b700 1 -- 172.21.15.119:0/4184600467 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4011790 con 0x7f6eb40cbf30 2023-10-08T01:12:03.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.056+0000 7f6eb1d9b700 1 -- 172.21.15.119:0/4184600467 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011b30 con 0x7f6eb40cbf30 2023-10-08T01:12:03.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.057+0000 7f6eb8bff700 1 -- 172.21.15.119:0/4184600467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 msgr2=0x7f6eb40ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.057+0000 7f6eb8bff700 1 --2- 172.21.15.119:0/4184600467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb40ae410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009910 tx=0x7f6ea40091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.057+0000 7f6eb8bff700 1 -- 172.21.15.119:0/4184600467 shutdown_connections 2023-10-08T01:12:03.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.057+0000 7f6eb8bff700 1 --2- 172.21.15.119:0/4184600467 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb40ae410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.057+0000 7f6eb8bff700 1 -- 172.21.15.119:0/4184600467 >> 172.21.15.119:0/4184600467 conn(0x7f6eb406a320 msgr2=0x7f6eb40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:03.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.057+0000 7f6eb8bff700 1 -- 172.21.15.119:0/4184600467 shutdown_connections 2023-10-08T01:12:03.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.057+0000 7f6eb8bff700 1 -- 172.21.15.119:0/4184600467 wait complete. 2023-10-08T01:12:03.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.058+0000 7f6eb8bff700 1 Processor -- start 2023-10-08T01:12:03.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.058+0000 7f6eb8bff700 1 -- start start 2023-10-08T01:12:03.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.059+0000 7f6eb8bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eb40b1420 0x7f6eb4150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.059+0000 7f6eb8bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb4151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.059+0000 7f6eb8bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb40ccf90 0x7f6eb4151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.059+0000 7f6eb8bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4157740 con 0x7f6eb40ccf90 2023-10-08T01:12:03.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.059+0000 7f6eb8bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4157990 con 0x7f6eb40b1420 2023-10-08T01:12:03.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.059+0000 7f6eb8bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb4157c70 con 0x7f6eb40cbf30 2023-10-08T01:12:03.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb40ccf90 0x7f6eb4151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb40ccf90 0x7f6eb4151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51720/0 (socket says 172.21.15.119:51720) 2023-10-08T01:12:03.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 -- 172.21.15.119:0/3753570632 learned_addr learned my addr 172.21.15.119:0/3753570632 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:03.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eabfff700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb4151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb259c700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eb40b1420 0x7f6eb4150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 -- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 msgr2=0x7f6eb4151530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb4151530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 -- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eb40b1420 msgr2=0x7f6eb4150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eb40b1420 0x7f6eb4150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 -- 172.21.15.119:0/3753570632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea4009420 con 0x7f6eb40ccf90 2023-10-08T01:12:03.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb259c700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eb40b1420 0x7f6eb4150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:03.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eabfff700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb4151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:03.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb2d9d700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb40ccf90 0x7f6eb4151a70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00c910 tx=0x7f6eac007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3753570632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00a7a0 con 0x7f6eb40ccf90 2023-10-08T01:12:03.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3753570632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac00a920 con 0x7f6eb40ccf90 2023-10-08T01:12:03.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb4157f50 con 0x7f6eb40ccf90 2023-10-08T01:12:03.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.060+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3753570632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00ace0 con 0x7f6eb40ccf90 2023-10-08T01:12:03.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.061+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb4158510 con 0x7f6eb40ccf90 2023-10-08T01:12:03.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.062+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb41586d0 con 0x7f6eb40ccf90 2023-10-08T01:12:03.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.063+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3753570632 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6eac012630 con 0x7f6eb40ccf90 2023-10-08T01:12:03.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.063+0000 7f6eaa7fc700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e94066f30 0x7f6e940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.063+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3753570632 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f6eac084a90 con 0x7f6eb40ccf90 2023-10-08T01:12:03.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.064+0000 7f6eb259c700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e94066f30 0x7f6e940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.064+0000 7f6eb259c700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e94066f30 0x7f6e940693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009860 tx=0x7f6ea4017000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.067+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3753570632 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6eac051b70 con 0x7f6eb40ccf90 2023-10-08T01:12:03.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.263+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6eb404c3c0 con 0x7f6eb40ccf90 2023-10-08T01:12:03.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.264+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3753570632 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16011 (secure 0 0 0) 0x7f6eac055190 con 0x7f6eb40ccf90 2023-10-08T01:12:03.265 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:03.265 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":90,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:02.279333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:00.376380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:01.260893+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:03.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e94066f30 msgr2=0x7f6e940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e94066f30 0x7f6e940693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4009860 tx=0x7f6ea4017000 comp rx=0 tx=0).stop 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb40ccf90 msgr2=0x7f6eb4151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb40ccf90 0x7f6eb4151a70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00c910 tx=0x7f6eac007f90 comp rx=0 tx=0).stop 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 shutdown_connections 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e94066f30 0x7f6e940693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6eb40ccf90 0x7f6eb4151a70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6eb40b1420 0x7f6eb4150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 --2- 172.21.15.119:0/3753570632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6eb40cbf30 0x7f6eb4151530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 >> 172.21.15.119:0/3753570632 conn(0x7f6eb406a320 msgr2=0x7f6eb40ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 shutdown_connections 2023-10-08T01:12:03.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.266+0000 7f6eb8bff700 1 -- 172.21.15.119:0/3753570632 wait complete. 2023-10-08T01:12:03.285 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:03.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.465+0000 7f32ea5ff700 1 Processor -- start 2023-10-08T01:12:03.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.466+0000 7f32ea5ff700 1 -- start start 2023-10-08T01:12:03.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.467+0000 7f32ea5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f32e41579f0 0x7f32e4162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.467+0000 7f32ea5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e40cb300 con 0x7f32e40b1850 2023-10-08T01:12:03.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.467+0000 7f32ea5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e40cb5e0 con 0x7f32e40b1c20 2023-10-08T01:12:03.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.467+0000 7f32ea5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e406fc30 con 0x7f32e41579f0 2023-10-08T01:12:03.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.467+0000 7f32e8b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f32e41579f0 0x7f32e4162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.467+0000 7f32e3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f32e40b1850 0x7f32e40ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41694/0 (socket says 172.21.15.119:41694) 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.467+0000 7f32e3fff700 1 -- 172.21.15.119:0/3328700668 learned_addr learned my addr 172.21.15.119:0/3328700668 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.467+0000 7f32e2ffd700 1 -- 172.21.15.119:0/3328700668 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727005050 0 0) 0x7f32e40cb300 con 0x7f32e40b1850 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.468+0000 7f32e2ffd700 1 -- 172.21.15.119:0/3328700668 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32cc003570 con 0x7f32e40b1850 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.468+0000 7f32e8b9c700 1 -- 172.21.15.119:0/3328700668 >> v1:172.21.15.184:6789/0 conn(0x7f32e40b1c20 legacy=0x7f32e41572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.468+0000 7f32e8b9c700 1 -- 172.21.15.119:0/3328700668 >> v1:172.21.15.119:6789/0 conn(0x7f32e40b1850 legacy=0x7f32e40ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.468+0000 7f32e8b9c700 1 -- 172.21.15.119:0/3328700668 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e40cb8c0 con 0x7f32e41579f0 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.468+0000 7f32e8b9c700 1 --2- 172.21.15.119:0/3328700668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e41579f0 0x7f32e4162530 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0098e0 tx=0x7f32dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d06b34c31cd849 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.468+0000 7f32e2ffd700 1 -- 172.21.15.119:0/3328700668 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00b640 con 0x7f32e41579f0 2023-10-08T01:12:03.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.468+0000 7f32e2ffd700 1 -- 172.21.15.119:0/3328700668 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32dc00b7a0 con 0x7f32e41579f0 2023-10-08T01:12:03.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.468+0000 7f32e2ffd700 1 -- 172.21.15.119:0/3328700668 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00bb20 con 0x7f32e41579f0 2023-10-08T01:12:03.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.469+0000 7f32ea5ff700 1 -- 172.21.15.119:0/3328700668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e41579f0 msgr2=0x7f32e4162530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.469+0000 7f32ea5ff700 1 --2- 172.21.15.119:0/3328700668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e41579f0 0x7f32e4162530 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0098e0 tx=0x7f32dc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.469+0000 7f32ea5ff700 1 -- 172.21.15.119:0/3328700668 shutdown_connections 2023-10-08T01:12:03.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.469+0000 7f32ea5ff700 1 --2- 172.21.15.119:0/3328700668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e41579f0 0x7f32e4162530 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.469+0000 7f32ea5ff700 1 -- 172.21.15.119:0/3328700668 >> 172.21.15.119:0/3328700668 conn(0x7f32e406a320 msgr2=0x7f32e40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:03.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.469+0000 7f32ea5ff700 1 -- 172.21.15.119:0/3328700668 shutdown_connections 2023-10-08T01:12:03.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.469+0000 7f32ea5ff700 1 -- 172.21.15.119:0/3328700668 wait complete. 2023-10-08T01:12:03.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.470+0000 7f32ea5ff700 1 Processor -- start 2023-10-08T01:12:03.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.470+0000 7f32ea5ff700 1 -- start start 2023-10-08T01:12:03.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.471+0000 7f32ea5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32e40b1850 0x7f32e41402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.471+0000 7f32ea5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40b1c20 0x7f32e40af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.471+0000 7f32ea5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32e41579f0 0x7f32e40a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.471+0000 7f32ea5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e40cc260 con 0x7f32e40b1c20 2023-10-08T01:12:03.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.471+0000 7f32ea5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e415cc10 con 0x7f32e40b1850 2023-10-08T01:12:03.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.471+0000 7f32ea5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e4140800 con 0x7f32e41579f0 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.471+0000 7f32e37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40b1c20 0x7f32e40af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40b1c20 0x7f32e40af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51748/0 (socket says 172.21.15.119:51748) 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e37fe700 1 -- 172.21.15.119:0/957187315 learned_addr learned my addr 172.21.15.119:0/957187315 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e3fff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32e40b1850 0x7f32e41402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e8b9c700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32e41579f0 0x7f32e40a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e37fe700 1 -- 172.21.15.119:0/957187315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32e41579f0 msgr2=0x7f32e40a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e37fe700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32e41579f0 0x7f32e40a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e37fe700 1 -- 172.21.15.119:0/957187315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32e40b1850 msgr2=0x7f32e41402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e37fe700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32e40b1850 0x7f32e41402c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e37fe700 1 -- 172.21.15.119:0/957187315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc009420 con 0x7f32e40b1c20 2023-10-08T01:12:03.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32e37fe700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40b1c20 0x7f32e40af3f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f32d800c590 tx=0x7f32d800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32cbfff700 1 -- 172.21.15.119:0/957187315 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8011590 con 0x7f32e40b1c20 2023-10-08T01:12:03.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32cbfff700 1 -- 172.21.15.119:0/957187315 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d80116f0 con 0x7f32e40b1c20 2023-10-08T01:12:03.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e40aa490 con 0x7f32e40b1c20 2023-10-08T01:12:03.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.472+0000 7f32cbfff700 1 -- 172.21.15.119:0/957187315 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8011ab0 con 0x7f32e40b1c20 2023-10-08T01:12:03.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.473+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e415b370 con 0x7f32e40b1c20 2023-10-08T01:12:03.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.474+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e404c3c0 con 0x7f32e40b1c20 2023-10-08T01:12:03.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.475+0000 7f32cbfff700 1 -- 172.21.15.119:0/957187315 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f32d801a360 con 0x7f32e40b1c20 2023-10-08T01:12:03.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.475+0000 7f32cbfff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32d0066f30 0x7f32d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.475+0000 7f32cbfff700 1 -- 172.21.15.119:0/957187315 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f32d8083780 con 0x7f32e40b1c20 2023-10-08T01:12:03.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.476+0000 7f32e3fff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32d0066f30 0x7f32d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.476+0000 7f32e3fff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32d0066f30 0x7f32d00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f32d4005c50 tx=0x7f32d4009040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.480+0000 7f32cbfff700 1 -- 172.21.15.119:0/957187315 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f32d80507c0 con 0x7f32e40b1c20 2023-10-08T01:12:03.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.671+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32e40678d0 con 0x7f32e40b1c20 2023-10-08T01:12:03.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.672+0000 7f32cbfff700 1 -- 172.21.15.119:0/957187315 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+16011 (secure 0 0 0) 0x7f32d8053de0 con 0x7f32e40b1c20 2023-10-08T01:12:03.673 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:03.674 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":90,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:02.279333+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:00.376380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:01.260893+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.674+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32d0066f30 msgr2=0x7f32d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.674+0000 7f32ea5ff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32d0066f30 0x7f32d00693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f32d4005c50 tx=0x7f32d4009040 comp rx=0 tx=0).stop 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.674+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40b1c20 msgr2=0x7f32e40af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.674+0000 7f32ea5ff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40b1c20 0x7f32e40af3f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f32d800c590 tx=0x7f32d800c950 comp rx=0 tx=0).stop 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.675+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 shutdown_connections 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.675+0000 7f32ea5ff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f32d0066f30 0x7f32d00693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.675+0000 7f32ea5ff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f32e41579f0 0x7f32e40a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.675+0000 7f32ea5ff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f32e40b1850 0x7f32e41402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.675+0000 7f32ea5ff700 1 --2- 172.21.15.119:0/957187315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f32e40b1c20 0x7f32e40af3f0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.675+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 >> 172.21.15.119:0/957187315 conn(0x7f32e406a320 msgr2=0x7f32e4152d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:03.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.675+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 shutdown_connections 2023-10-08T01:12:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.675+0000 7f32ea5ff700 1 -- 172.21.15.119:0/957187315 wait complete. 2023-10-08T01:12:03.694 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T01:12:03.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.880+0000 7fb6d89ff700 1 Processor -- start 2023-10-08T01:12:03.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.880+0000 7fb6d89ff700 1 -- start start 2023-10-08T01:12:03.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d89ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb6d4069e80 0x7fb6d4160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d89ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d406e950 con 0x7fb6d4155eb0 2023-10-08T01:12:03.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d89ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d406eb90 con 0x7fb6d40c70b0 2023-10-08T01:12:03.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d89ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d406ee70 con 0x7fb6d4069e80 2023-10-08T01:12:03.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d259c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb6d4155eb0 0x7fb6d406e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41708/0 (socket says 172.21.15.119:41708) 2023-10-08T01:12:03.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d259c700 1 -- 172.21.15.119:0/3766520168 learned_addr learned my addr 172.21.15.119:0/3766520168 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d2d9d700 1 --2- 172.21.15.119:0/3766520168 >> v2:172.21.15.184:3300/0 conn(0x7fb6d4069e80 0x7fb6d4160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323544192 0 0) 0x7fb6c4001110 con 0x7fb6d4155eb0 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.882+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6c0004680 con 0x7fb6d4155eb0 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140381517 0 0) 0x7fb6c0004680 con 0x7fb6d4155eb0 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 >> v2:172.21.15.184:3300/0 conn(0x7fb6d4069e80 msgr2=0x7fb6d4160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 --2- 172.21.15.119:0/3766520168 >> v2:172.21.15.184:3300/0 conn(0x7fb6d4069e80 0x7fb6d4160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 >> v1:172.21.15.184:6789/0 conn(0x7fb6d40c70b0 legacy=0x7fb6d4069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d406f150 con 0x7fb6d4155eb0 2023-10-08T01:12:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fb6c4001600 con 0x7fb6d4155eb0 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 >> v1:172.21.15.119:6789/0 conn(0x7fb6d4155eb0 legacy=0x7fb6d406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6c0004bc0 0x7fb6c0004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0005920 0x7fb6c0005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6c0006320 0x7fb6c0003570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d406ee70 con 0x7fb6c0004bc0 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c0003cc0 con 0x7fb6c0005920 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c0003e80 con 0x7fb6c0006320 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d259c700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6c0004bc0 0x7fb6c0004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6d2d9d700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6c0006320 0x7fb6c0003570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.883+0000 7fb6cbfff700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0005920 0x7fb6c0005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d259c700 1 -- 172.21.15.119:0/3766520168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6c0006320 msgr2=0x7fb6c0003570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d259c700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6c0006320 0x7fb6c0003570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d259c700 1 -- 172.21.15.119:0/3766520168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0005920 msgr2=0x7fb6c0005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d259c700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0005920 0x7fb6c0005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d259c700 1 -- 172.21.15.119:0/3766520168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c0004680 con 0x7fb6c0004bc0 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d2d9d700 1 -- 172.21.15.119:0/3766520168 reap_dead start 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d259c700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6c0004bc0 0x7fb6c0004fd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400c3c0 tx=0x7fb6d406f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c61d7fa25ceb136 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400ccf0 con 0x7fb6c0004bc0 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c400ce50 con 0x7fb6c0004bc0 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.884+0000 7fb6d1d9b700 1 -- 172.21.15.119:0/3766520168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4005b70 con 0x7fb6c0004bc0 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.885+0000 7fb6d89ff700 1 -- 172.21.15.119:0/3766520168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6c0004bc0 msgr2=0x7fb6c0004fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.885+0000 7fb6d89ff700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6c0004bc0 0x7fb6c0004fd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400c3c0 tx=0x7fb6d406f7a0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.885+0000 7fb6d89ff700 1 -- 172.21.15.119:0/3766520168 shutdown_connections 2023-10-08T01:12:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.885+0000 7fb6d89ff700 1 --2- 172.21.15.119:0/3766520168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6c0004bc0 0x7fb6c0004fd0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.885+0000 7fb6d89ff700 1 -- 172.21.15.119:0/3766520168 >> 172.21.15.119:0/3766520168 conn(0x7fb6d4068e10 msgr2=0x7fb6d40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:03.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.885+0000 7fb6d89ff700 1 -- 172.21.15.119:0/3766520168 shutdown_connections 2023-10-08T01:12:03.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.885+0000 7fb6d89ff700 1 -- 172.21.15.119:0/3766520168 wait complete. 2023-10-08T01:12:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.886+0000 7fb6d89ff700 1 Processor -- start 2023-10-08T01:12:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.886+0000 7fb6d89ff700 1 -- start start 2023-10-08T01:12:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.887+0000 7fb6d89ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0004bc0 0x7fb6d40c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.887+0000 7fb6d89ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6d4159ce0 0x7fb6d4154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.887+0000 7fb6d89ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d415b030 0x7fb6d415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.887+0000 7fb6d89ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d415b9e0 con 0x7fb6d415b030 2023-10-08T01:12:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.887+0000 7fb6d89ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d40c3680 con 0x7fb6c0004bc0 2023-10-08T01:12:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.887+0000 7fb6d89ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d415a230 con 0x7fb6d4159ce0 2023-10-08T01:12:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.887+0000 7fb6d2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d415b030 0x7fb6d415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.887+0000 7fb6d2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d415b030 0x7fb6d415b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51760/0 (socket says 172.21.15.119:51760) 2023-10-08T01:12:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d2d9d700 1 -- 172.21.15.119:0/926661852 learned_addr learned my addr 172.21.15.119:0/926661852 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d259c700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0004bc0 0x7fb6d40c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6cbfff700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6d4159ce0 0x7fb6d4154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d2d9d700 1 -- 172.21.15.119:0/926661852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6d4159ce0 msgr2=0x7fb6d4154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d2d9d700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6d4159ce0 0x7fb6d4154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d2d9d700 1 -- 172.21.15.119:0/926661852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0004bc0 msgr2=0x7fb6d40c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:03.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d2d9d700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0004bc0 0x7fb6d40c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:03.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d2d9d700 1 -- 172.21.15.119:0/926661852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d415a510 con 0x7fb6d415b030 2023-10-08T01:12:03.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d2d9d700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d415b030 0x7fb6d415b4a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb6d406a280 tx=0x7fb6cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6ca7fc700 1 -- 172.21.15.119:0/926661852 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00e040 con 0x7fb6d415b030 2023-10-08T01:12:03.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6ca7fc700 1 -- 172.21.15.119:0/926661852 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc00f040 con 0x7fb6d415b030 2023-10-08T01:12:03.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6ca7fc700 1 -- 172.21.15.119:0/926661852 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc00fd70 con 0x7fb6d415b030 2023-10-08T01:12:03.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.888+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d415a7f0 con 0x7fb6d415b030 2023-10-08T01:12:03.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.889+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d40c6470 con 0x7fb6d415b030 2023-10-08T01:12:03.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.890+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d40c3850 con 0x7fb6d415b030 2023-10-08T01:12:03.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.890+0000 7fb6ca7fc700 1 -- 172.21.15.119:0/926661852 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb6cc015640 con 0x7fb6d415b030 2023-10-08T01:12:03.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.891+0000 7fb6ca7fc700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb6b4066f20 0x7fb6b40693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:03.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.891+0000 7fb6ca7fc700 1 -- 172.21.15.119:0/926661852 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb6cc084dd0 con 0x7fb6d415b030 2023-10-08T01:12:03.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.892+0000 7fb6d259c700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb6b4066f20 0x7fb6b40693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:03.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.892+0000 7fb6d259c700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb6b4066f20 0x7fb6b40693d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4007d80 tx=0x7fb6c4015000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:03.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:03.895+0000 7fb6ca7fc700 1 -- 172.21.15.119:0/926661852 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb6d40c3850 con 0x7fb6d415b030 2023-10-08T01:12:04.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.129+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb6d40c37f0 con 0x7fb6d415b030 2023-10-08T01:12:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.853+0000 7fb6ca7fc700 1 -- 172.21.15.119:0/926661852 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb6cc086020 con 0x7fb6d415b030 2023-10-08T01:12:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb6b4066f20 msgr2=0x7fb6b40693d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb6b4066f20 0x7fb6b40693d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb6c4007d80 tx=0x7fb6c4015000 comp rx=0 tx=0).stop 2023-10-08T01:12:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d415b030 msgr2=0x7fb6d415b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d415b030 0x7fb6d415b4a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb6d406a280 tx=0x7fb6cc00b040 comp rx=0 tx=0).stop 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 shutdown_connections 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb6b4066f20 0x7fb6b40693d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d415b030 0x7fb6d415b4a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb6c0004bc0 0x7fb6d40c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 --2- 172.21.15.119:0/926661852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6d4159ce0 0x7fb6d4154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.855+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 >> 172.21.15.119:0/926661852 conn(0x7fb6d4068e10 msgr2=0x7fb6d40c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.856+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 shutdown_connections 2023-10-08T01:12:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:04.856+0000 7fb6d89ff700 1 -- 172.21.15.119:0/926661852 wait complete. 2023-10-08T01:12:04.877 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T01:12:05.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.065+0000 7f6e363ff700 1 Processor -- start 2023-10-08T01:12:05.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.065+0000 7f6e363ff700 1 -- start start 2023-10-08T01:12:05.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e363ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e301561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e363ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e300c87c0 0x7f6e30156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e363ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3015b0d0 con 0x7f6e300c83f0 2023-10-08T01:12:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e363ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e30156c50 con 0x7f6e300c87c0 2023-10-08T01:12:05.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e363ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3015a4c0 con 0x7f6e300ccee0 2023-10-08T01:12:05.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e2f7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e301561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:05.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e3499c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e300c87c0 0x7f6e30156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:05.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e2f7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e301561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:42770/0 (socket says 172.21.15.119:42770) 2023-10-08T01:12:05.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e2f7fe700 1 -- 172.21.15.119:0/3266737675 learned_addr learned my addr 172.21.15.119:0/3266737675 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:05.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.067+0000 7f6e2ffff700 1 --1- 172.21.15.119:0/3266737675 >> v1:172.21.15.184:6789/0 conn(0x7f6e300c83f0 0x7f6e300c7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:50964/0 (socket says 172.21.15.119:50964) 2023-10-08T01:12:05.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e3499c700 1 -- 172.21.15.119:0/3266737675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 msgr2=0x7f6e301561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e3499c700 1 --2- 172.21.15.119:0/3266737675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e301561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e3499c700 1 -- 172.21.15.119:0/3266737675 >> v1:172.21.15.184:6789/0 conn(0x7f6e300c83f0 legacy=0x7f6e300c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e3499c700 1 -- 172.21.15.119:0/3266737675 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3015a7a0 con 0x7f6e300c87c0 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e2f7fe700 1 --2- 172.21.15.119:0/3266737675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e301561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e3499c700 1 --2- 172.21.15.119:0/3266737675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 0x7f6e30156710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800d0d0 tx=0x7f6e2800b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=826efa5ffe35d116 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e2effd700 1 -- 172.21.15.119:0/3266737675 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28010070 con 0x7f6e300c87c0 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e2effd700 1 -- 172.21.15.119:0/3266737675 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2800b2e0 con 0x7f6e300c87c0 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.068+0000 7f6e2effd700 1 -- 172.21.15.119:0/3266737675 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2800dd80 con 0x7f6e300c87c0 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.069+0000 7f6e363ff700 1 -- 172.21.15.119:0/3266737675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 msgr2=0x7f6e30156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.069+0000 7f6e363ff700 1 --2- 172.21.15.119:0/3266737675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 0x7f6e30156710 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800d0d0 tx=0x7f6e2800b990 comp rx=0 tx=0).stop 2023-10-08T01:12:05.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.069+0000 7f6e363ff700 1 -- 172.21.15.119:0/3266737675 shutdown_connections 2023-10-08T01:12:05.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.069+0000 7f6e363ff700 1 --2- 172.21.15.119:0/3266737675 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 0x7f6e30156710 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.069+0000 7f6e363ff700 1 --2- 172.21.15.119:0/3266737675 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e301561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.069+0000 7f6e363ff700 1 -- 172.21.15.119:0/3266737675 >> 172.21.15.119:0/3266737675 conn(0x7f6e30068e10 msgr2=0x7f6e300cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:05.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.069+0000 7f6e363ff700 1 -- 172.21.15.119:0/3266737675 shutdown_connections 2023-10-08T01:12:05.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.069+0000 7f6e363ff700 1 -- 172.21.15.119:0/3266737675 wait complete. 2023-10-08T01:12:05.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.070+0000 7f6e363ff700 1 Processor -- start 2023-10-08T01:12:05.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.071+0000 7f6e363ff700 1 -- start start 2023-10-08T01:12:05.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.071+0000 7f6e363ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e300c83f0 0x7f6e301577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:05.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.071+0000 7f6e363ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 0x7f6e301515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:05.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.071+0000 7f6e363ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e3013fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:05.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.071+0000 7f6e363ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30158f40 con 0x7f6e300c83f0 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.071+0000 7f6e363ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30157ea0 con 0x7f6e300c87c0 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.071+0000 7f6e363ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e30158180 con 0x7f6e300ccee0 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e300c83f0 0x7f6e301577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e300c83f0 0x7f6e301577a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55114/0 (socket says 172.21.15.119:55114) 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 -- 172.21.15.119:0/4152842742 learned_addr learned my addr 172.21.15.119:0/4152842742 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e3499c700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e3013fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2f7fe700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 0x7f6e301515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 -- 172.21.15.119:0/4152842742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 msgr2=0x7f6e3013fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e3013fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 -- 172.21.15.119:0/4152842742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 msgr2=0x7f6e301515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:05.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 0x7f6e301515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 -- 172.21.15.119:0/4152842742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e240094f0 con 0x7f6e300c83f0 2023-10-08T01:12:05.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.072+0000 7f6e2ffff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e300c83f0 0x7f6e301577a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000c7c0 tx=0x7f6e20007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:05.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.073+0000 7f6e17fff700 1 -- 172.21.15.119:0/4152842742 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20005a70 con 0x7f6e300c83f0 2023-10-08T01:12:05.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.073+0000 7f6e17fff700 1 -- 172.21.15.119:0/4152842742 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e20005bf0 con 0x7f6e300c83f0 2023-10-08T01:12:05.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.073+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2800bc30 con 0x7f6e300c83f0 2023-10-08T01:12:05.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.073+0000 7f6e17fff700 1 -- 172.21.15.119:0/4152842742 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e200070f0 con 0x7f6e300c83f0 2023-10-08T01:12:05.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.073+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30158740 con 0x7f6e300c83f0 2023-10-08T01:12:05.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.074+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3004c3c0 con 0x7f6e300c83f0 2023-10-08T01:12:05.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.075+0000 7f6e17fff700 1 -- 172.21.15.119:0/4152842742 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e20007460 con 0x7f6e300c83f0 2023-10-08T01:12:05.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.075+0000 7f6e17fff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e1c066f30 0x7f6e1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:05.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.076+0000 7f6e17fff700 1 -- 172.21.15.119:0/4152842742 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f6e20084120 con 0x7f6e300c83f0 2023-10-08T01:12:05.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.076+0000 7f6e2f7fe700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e1c066f30 0x7f6e1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:05.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.076+0000 7f6e2f7fe700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e1c066f30 0x7f6e1c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006fd0 tx=0x7f6e24008040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:05.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.080+0000 7f6e17fff700 1 -- 172.21.15.119:0/4152842742 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e20051280 con 0x7f6e300c83f0 2023-10-08T01:12:05.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.311+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6e3015aa80 con 0x7f6e300c83f0 2023-10-08T01:12:05.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.854+0000 7f6e17fff700 1 -- 172.21.15.119:0/4152842742 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6e200548a0 con 0x7f6e300c83f0 2023-10-08T01:12:05.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e1c066f30 msgr2=0x7f6e1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:05.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e1c066f30 0x7f6e1c0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6e24006fd0 tx=0x7f6e24008040 comp rx=0 tx=0).stop 2023-10-08T01:12:05.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e300c83f0 msgr2=0x7f6e301577a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:05.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e300c83f0 0x7f6e301577a0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000c7c0 tx=0x7f6e20007f70 comp rx=0 tx=0).stop 2023-10-08T01:12:05.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 shutdown_connections 2023-10-08T01:12:05.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e1c066f30 0x7f6e1c0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e300ccee0 0x7f6e3013fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e300c83f0 0x7f6e301577a0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 --2- 172.21.15.119:0/4152842742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e300c87c0 0x7f6e301515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:05.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.856+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 >> 172.21.15.119:0/4152842742 conn(0x7f6e30068e10 msgr2=0x7f6e300cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:05.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.857+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 shutdown_connections 2023-10-08T01:12:05.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:05.857+0000 7f6e363ff700 1 -- 172.21.15.119:0/4152842742 wait complete. 2023-10-08T01:12:05.877 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:06.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.058+0000 7f735b59e700 1 Processor -- start 2023-10-08T01:12:06.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.058+0000 7f735b59e700 1 -- start start 2023-10-08T01:12:06.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.059+0000 7f735b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f735c0c6c60 0x7f735c0c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.059+0000 7f735b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c06e8c0 0x7f735c0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.059+0000 7f735b59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c0b1410 con 0x7f735c1595a0 2023-10-08T01:12:06.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.059+0000 7f735b59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f735c0b2b90 con 0x7f735c0c6c60 2023-10-08T01:12:06.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.059+0000 7f735b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c063230 con 0x7f735c06e8c0 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f7359d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f735c0c6c60 0x7f735c0c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f735a59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f735c1595a0 0x7f735c0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52210/0 (socket says 172.21.15.119:52210) 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f735a59c700 1 -- 172.21.15.119:0/2073926128 learned_addr learned my addr 172.21.15.119:0/2073926128 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f735ad9d700 1 --2- 172.21.15.119:0/2073926128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c06e8c0 0x7f735c0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f7359d9b700 1 -- 172.21.15.119:0/2073926128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c06e8c0 msgr2=0x7f735c0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f7359d9b700 1 --2- 172.21.15.119:0/2073926128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c06e8c0 0x7f735c0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f7359d9b700 1 -- 172.21.15.119:0/2073926128 >> v1:172.21.15.119:6789/0 conn(0x7f735c1595a0 legacy=0x7f735c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f7359d9b700 1 -- 172.21.15.119:0/2073926128 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c0634f0 con 0x7f735c0c6c60 2023-10-08T01:12:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f735959a700 1 -- 172.21.15.119:0/2073926128 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27968878 0 0) 0x7f735c0b1410 con 0x7f735c1595a0 2023-10-08T01:12:06.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f735ad9d700 1 --2- 172.21.15.119:0/2073926128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c06e8c0 0x7f735c0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:06.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.060+0000 7f7359d9b700 1 --2- 172.21.15.119:0/2073926128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c0c6c60 0x7f735c0c8120 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f73500098e0 tx=0x7f73500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a9aa88575090c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735959a700 1 -- 172.21.15.119:0/2073926128 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350009e80 con 0x7f735c0c6c60 2023-10-08T01:12:06.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735959a700 1 -- 172.21.15.119:0/2073926128 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735000b860 con 0x7f735c0c6c60 2023-10-08T01:12:06.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735959a700 1 -- 172.21.15.119:0/2073926128 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735000bbe0 con 0x7f735c0c6c60 2023-10-08T01:12:06.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735b59e700 1 -- 172.21.15.119:0/2073926128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c0c6c60 msgr2=0x7f735c0c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735b59e700 1 --2- 172.21.15.119:0/2073926128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c0c6c60 0x7f735c0c8120 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f73500098e0 tx=0x7f73500091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735b59e700 1 -- 172.21.15.119:0/2073926128 shutdown_connections 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735b59e700 1 --2- 172.21.15.119:0/2073926128 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c06e8c0 0x7f735c0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735b59e700 1 --2- 172.21.15.119:0/2073926128 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c0c6c60 0x7f735c0c8120 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735b59e700 1 -- 172.21.15.119:0/2073926128 >> 172.21.15.119:0/2073926128 conn(0x7f735c06ddb0 msgr2=0x7f735c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735b59e700 1 -- 172.21.15.119:0/2073926128 shutdown_connections 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.061+0000 7f735b59e700 1 -- 172.21.15.119:0/2073926128 wait complete. 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.063+0000 7f735b59e700 1 Processor -- start 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.063+0000 7f735b59e700 1 -- start start 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f735c06e8c0 0x7f735c1425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c0c6c60 0x7f735c0cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c1595a0 0x7f735c0cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735b59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c142b00 con 0x7f735c1595a0 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735b59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c15e320 con 0x7f735c06e8c0 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c15e600 con 0x7f735c0c6c60 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c1595a0 0x7f735c0cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c1595a0 0x7f735c0cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55124/0 (socket says 172.21.15.119:55124) 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735ad9d700 1 -- 172.21.15.119:0/1813505234 learned_addr learned my addr 172.21.15.119:0/1813505234 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f735a59c700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f735c06e8c0 0x7f735c1425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.064+0000 7f7359d9b700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c0c6c60 0x7f735c0cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735ad9d700 1 -- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c0c6c60 msgr2=0x7f735c0cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735ad9d700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c0c6c60 0x7f735c0cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735ad9d700 1 -- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f735c06e8c0 msgr2=0x7f735c1425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735ad9d700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f735c06e8c0 0x7f735c1425c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735ad9d700 1 -- 172.21.15.119:0/1813505234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350009420 con 0x7f735c1595a0 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735a59c700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f735c06e8c0 0x7f735c1425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735ad9d700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c1595a0 0x7f735c0cbcf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f735400d180 tx=0x7f7354009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f734a7fc700 1 -- 172.21.15.119:0/1813505234 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735400dd70 con 0x7f735c1595a0 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f734a7fc700 1 -- 172.21.15.119:0/1813505234 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735400f040 con 0x7f735c1595a0 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c15e8e0 con 0x7f735c1595a0 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f734a7fc700 1 -- 172.21.15.119:0/1813505234 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7354015690 con 0x7f735c1595a0 2023-10-08T01:12:06.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.065+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c15ddd0 con 0x7f735c1595a0 2023-10-08T01:12:06.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.066+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c15e080 con 0x7f735c1595a0 2023-10-08T01:12:06.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.067+0000 7f734a7fc700 1 -- 172.21.15.119:0/1813505234 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f735401e3b0 con 0x7f735c1595a0 2023-10-08T01:12:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.068+0000 7f734a7fc700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f733c066f30 0x7f733c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.068+0000 7f734a7fc700 1 -- 172.21.15.119:0/1813505234 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7354084c00 con 0x7f735c1595a0 2023-10-08T01:12:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.068+0000 7f735a59c700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f733c066f30 0x7f733c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.069+0000 7f735a59c700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f733c066f30 0x7f733c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f734c001890 tx=0x7f734c009040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.072+0000 7f734a7fc700 1 -- 172.21.15.119:0/1813505234 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f735408b080 con 0x7f735c1595a0 2023-10-08T01:12:06.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.305+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f735c04c3c0 con 0x7f735c1595a0 2023-10-08T01:12:06.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.305+0000 7f734a7fc700 1 -- 172.21.15.119:0/1813505234 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f735401a090 con 0x7f735c1595a0 2023-10-08T01:12:06.307 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:06.307 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3441253316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3441253316},{"type":"v1","addr":"172.21.15.184:6834","nonce":3441253316}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/3613948578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3613948578},{"type":"v1","addr":"172.21.15.119:6833","nonce":3613948578}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/765135141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":765135141},{"type":"v1","addr":"172.21.15.119:6835","nonce":765135141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:02.279676+0000","modified":"2023-10-08T01:12:05.851791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/910230897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":910230897},{"type":"v1","addr":"172.21.15.184:6835","nonce":910230897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T01:12:06.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.307+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f733c066f30 msgr2=0x7f733c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.307+0000 7f735b59e700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f733c066f30 0x7f733c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f734c001890 tx=0x7f734c009040 comp rx=0 tx=0).stop 2023-10-08T01:12:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.307+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c1595a0 msgr2=0x7f735c0cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.307+0000 7f735b59e700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c1595a0 0x7f735c0cbcf0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f735400d180 tx=0x7f7354009a50 comp rx=0 tx=0).stop 2023-10-08T01:12:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.308+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 shutdown_connections 2023-10-08T01:12:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.308+0000 7f735b59e700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f733c066f30 0x7f733c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.308+0000 7f735b59e700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f735c1595a0 0x7f735c0cbcf0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.308+0000 7f735b59e700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f735c06e8c0 0x7f735c1425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.308+0000 7f735b59e700 1 --2- 172.21.15.119:0/1813505234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f735c0c6c60 0x7f735c0cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.308+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 >> 172.21.15.119:0/1813505234 conn(0x7f735c06ddb0 msgr2=0x7f735c158550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.308+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 shutdown_connections 2023-10-08T01:12:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.308+0000 7f735b59e700 1 -- 172.21.15.119:0/1813505234 wait complete. 2023-10-08T01:12:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 39 2023-10-08T01:12:06.326 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.508+0000 7f7eed5ff700 1 Processor -- start 2023-10-08T01:12:06.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.508+0000 7f7eed5ff700 1 -- start start 2023-10-08T01:12:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7eed5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ee8155c10 0x7f7ee8155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7eed5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ac6d0 0x7f7ee80b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7eed5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ee80acac0 con 0x7f7ee80ac230 2023-10-08T01:12:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7eed5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee80ad930 con 0x7f7ee8155c10 2023-10-08T01:12:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7eed5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee80acd20 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7ee6ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ee8155c10 0x7f7ee8155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7edffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7ee80ac230 0x7f7ee81568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52220/0 (socket says 172.21.15.119:52220) 2023-10-08T01:12:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7edffff700 1 -- 172.21.15.119:0/1306968289 learned_addr learned my addr 172.21.15.119:0/1306968289 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7ee77fe700 1 --2- 172.21.15.119:0/1306968289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ac6d0 0x7f7ee80b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7ee6ffd700 1 -- 172.21.15.119:0/1306968289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ac6d0 msgr2=0x7f7ee80b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7ee6ffd700 1 --2- 172.21.15.119:0/1306968289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ac6d0 0x7f7ee80b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7ee6ffd700 1 -- 172.21.15.119:0/1306968289 >> v1:172.21.15.119:6789/0 conn(0x7f7ee80ac230 legacy=0x7f7ee81568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7ee6ffd700 1 -- 172.21.15.119:0/1306968289 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee80ad000 con 0x7f7ee8155c10 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.510+0000 7f7ee67fc700 1 -- 172.21.15.119:0/1306968289 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811158360 0 0) 0x7f7ee80acac0 con 0x7f7ee80ac230 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.511+0000 7f7ee6ffd700 1 --2- 172.21.15.119:0/1306968289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee8155c10 0x7f7ee8155fe0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80acc50 tx=0x7f7ed80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be09ea0581a14636 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.511+0000 7f7ee67fc700 1 -- 172.21.15.119:0/1306968289 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8011630 con 0x7f7ee8155c10 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.511+0000 7f7ee67fc700 1 -- 172.21.15.119:0/1306968289 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed8011790 con 0x7f7ee8155c10 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.511+0000 7f7ee67fc700 1 -- 172.21.15.119:0/1306968289 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed8011b50 con 0x7f7ee8155c10 2023-10-08T01:12:06.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.511+0000 7f7eed5ff700 1 -- 172.21.15.119:0/1306968289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee8155c10 msgr2=0x7f7ee8155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.511+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/1306968289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee8155c10 0x7f7ee8155fe0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80acc50 tx=0x7f7ed80091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.512+0000 7f7eed5ff700 1 -- 172.21.15.119:0/1306968289 shutdown_connections 2023-10-08T01:12:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.512+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/1306968289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee80ac6d0 0x7f7ee80b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.512+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/1306968289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee8155c10 0x7f7ee8155fe0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.512+0000 7f7eed5ff700 1 -- 172.21.15.119:0/1306968289 >> 172.21.15.119:0/1306968289 conn(0x7f7ee806a320 msgr2=0x7f7ee80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.512+0000 7f7eed5ff700 1 -- 172.21.15.119:0/1306968289 shutdown_connections 2023-10-08T01:12:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.512+0000 7f7eed5ff700 1 -- 172.21.15.119:0/1306968289 wait complete. 2023-10-08T01:12:06.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.513+0000 7f7eed5ff700 1 Processor -- start 2023-10-08T01:12:06.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.513+0000 7f7eed5ff700 1 -- start start 2023-10-08T01:12:06.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7eed5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80ac230 0x7f7ee815e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7eed5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ac6d0 0x7f7ee814e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7eed5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee8155c10 0x7f7ee8169aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7eed5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee815a5e0 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7eed5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee815f050 con 0x7f7ee80ac230 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7eed5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee8159690 con 0x7f7ee8155c10 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7edffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ac6d0 0x7f7ee814e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7edffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ac6d0 0x7f7ee814e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55136/0 (socket says 172.21.15.119:55136) 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.514+0000 7f7edffff700 1 -- 172.21.15.119:0/3382081388 learned_addr learned my addr 172.21.15.119:0/3382081388 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edffff700 1 -- 172.21.15.119:0/3382081388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee8155c10 msgr2=0x7f7ee8169aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edffff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee8155c10 0x7f7ee8169aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edffff700 1 -- 172.21.15.119:0/3382081388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80ac230 msgr2=0x7f7ee815e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edffff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80ac230 0x7f7ee815e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edffff700 1 -- 172.21.15.119:0/3382081388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed8009420 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edffff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ac6d0 0x7f7ee814e7a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000c7e0 tx=0x7f7ed0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edeffd700 1 -- 172.21.15.119:0/3382081388 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0005b30 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edeffd700 1 -- 172.21.15.119:0/3382081388 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0005c90 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7edeffd700 1 -- 172.21.15.119:0/3382081388 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0007300 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee814f360 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.515+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee815b7e0 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.516+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee804c3c0 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.517+0000 7f7edeffd700 1 -- 172.21.15.119:0/3382081388 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ed001b400 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.518+0000 7f7edeffd700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.518+0000 7f7edeffd700 1 -- 172.21.15.119:0/3382081388 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7ed0084270 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.518+0000 7f7ee6ffd700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.519+0000 7f7ee6ffd700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80acc50 tx=0x7f7ed8016000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.522+0000 7f7edeffd700 1 -- 172.21.15.119:0/3382081388 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ed0051330 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.715+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ee80ad310 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.716+0000 7f7edeffd700 1 -- 172.21.15.119:0/3382081388 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+15998 (secure 0 0 0) 0x7f7ed0054950 con 0x7f7ee80ac6d0 2023-10-08T01:12:06.717 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:06.717 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:03.836067+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:00.376380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:01.260893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7ec8066f30 msgr2=0x7f7ec80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7ee80acc50 tx=0x7f7ed8016000 comp rx=0 tx=0).stop 2023-10-08T01:12:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ac6d0 msgr2=0x7f7ee814e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ac6d0 0x7f7ee814e7a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000c7e0 tx=0x7f7ed0007df0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 shutdown_connections 2023-10-08T01:12:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7ec8066f30 0x7f7ec80693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ee8155c10 0x7f7ee8169aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ee80ac230 0x7f7ee815e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 --2- 172.21.15.119:0/3382081388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ee80ac6d0 0x7f7ee814e7a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 >> 172.21.15.119:0/3382081388 conn(0x7f7ee806a320 msgr2=0x7f7ee814fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:06.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 shutdown_connections 2023-10-08T01:12:06.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.719+0000 7f7eed5ff700 1 -- 172.21.15.119:0/3382081388 wait complete. 2023-10-08T01:12:06.737 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T01:12:06.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:12:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.920+0000 7f057a59d700 1 Processor -- start 2023-10-08T01:12:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.920+0000 7f057a59d700 1 -- start start 2023-10-08T01:12:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.921+0000 7f057a59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f05741595a0 0x7f05740b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.921+0000 7f057a59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f05740c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.921+0000 7f057a59d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05740b1410 con 0x7f05740b1de0 2023-10-08T01:12:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.921+0000 7f057a59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05740b2b90 con 0x7f05741595a0 2023-10-08T01:12:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.921+0000 7f057a59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0574063230 con 0x7f05740c6c60 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f0579d9c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f05740b1de0 0x7f057415d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:50974/0 (socket says 172.21.15.119:50974) 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f0578d9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f05740c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f057959b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f05741595a0 0x7f05740b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f0579d9c700 1 -- 172.21.15.119:0/1196727254 learned_addr learned my addr 172.21.15.119:0/1196727254 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f057959b700 1 -- 172.21.15.119:0/1196727254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 msgr2=0x7f05740c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f057959b700 1 --2- 172.21.15.119:0/1196727254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f05740c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f057959b700 1 -- 172.21.15.119:0/1196727254 >> v1:172.21.15.184:6789/0 conn(0x7f05740b1de0 legacy=0x7f057415d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f057959b700 1 -- 172.21.15.119:0/1196727254 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05740634f0 con 0x7f05741595a0 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f0578d9a700 1 --2- 172.21.15.119:0/1196727254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f05740c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.922+0000 7f056bfff700 1 -- 172.21.15.119:0/1196727254 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143830407 0 0) 0x7f05740b1410 con 0x7f05740b1de0 2023-10-08T01:12:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f057959b700 1 --2- 172.21.15.119:0/1196727254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05741595a0 0x7f05740b18a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f05700098e0 tx=0x7f05700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=670df768efa5f14e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f056bfff700 1 -- 172.21.15.119:0/1196727254 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057000b640 con 0x7f05741595a0 2023-10-08T01:12:06.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f056bfff700 1 -- 172.21.15.119:0/1196727254 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057000b7a0 con 0x7f05741595a0 2023-10-08T01:12:06.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f056bfff700 1 -- 172.21.15.119:0/1196727254 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057000bb40 con 0x7f05741595a0 2023-10-08T01:12:06.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f057a59d700 1 -- 172.21.15.119:0/1196727254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05741595a0 msgr2=0x7f05740b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f057a59d700 1 --2- 172.21.15.119:0/1196727254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05741595a0 0x7f05740b18a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f05700098e0 tx=0x7f05700091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f057a59d700 1 -- 172.21.15.119:0/1196727254 shutdown_connections 2023-10-08T01:12:06.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f057a59d700 1 --2- 172.21.15.119:0/1196727254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05741595a0 0x7f05740b18a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f057a59d700 1 --2- 172.21.15.119:0/1196727254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f05740c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.923+0000 7f057a59d700 1 -- 172.21.15.119:0/1196727254 >> 172.21.15.119:0/1196727254 conn(0x7f057406ddb0 msgr2=0x7f057406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:06.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.924+0000 7f057a59d700 1 -- 172.21.15.119:0/1196727254 shutdown_connections 2023-10-08T01:12:06.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.924+0000 7f057a59d700 1 -- 172.21.15.119:0/1196727254 wait complete. 2023-10-08T01:12:06.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.925+0000 7f057a59d700 1 Processor -- start 2023-10-08T01:12:06.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.925+0000 7f057a59d700 1 -- start start 2023-10-08T01:12:06.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f057a59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05740b1de0 0x7f05740c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f057a59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f0574151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f057a59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05741595a0 0x7f05740c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f057a59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05740c2940 con 0x7f05741595a0 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f057a59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05740c1d30 con 0x7f05740b1de0 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f057a59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05740c2010 con 0x7f05740c6c60 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f0579d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05741595a0 0x7f05740c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f0579d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05741595a0 0x7f05740c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55144/0 (socket says 172.21.15.119:55144) 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f057959b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05740b1de0 0x7f05740c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f0578d9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f0574151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.926+0000 7f0579d9c700 1 -- 172.21.15.119:0/1727898492 learned_addr learned my addr 172.21.15.119:0/1727898492 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f0579d9c700 1 -- 172.21.15.119:0/1727898492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 msgr2=0x7f0574151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f0579d9c700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f0574151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f0579d9c700 1 -- 172.21.15.119:0/1727898492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05740b1de0 msgr2=0x7f05740c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f0579d9c700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05740b1de0 0x7f05740c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f0579d9c700 1 -- 172.21.15.119:0/1727898492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0564009420 con 0x7f05741595a0 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f0579d9c700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05741595a0 0x7f05740c5990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f056c00ccc0 tx=0x7f056c001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f05697fa700 1 -- 172.21.15.119:0/1727898492 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00a880 con 0x7f05741595a0 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f05697fa700 1 -- 172.21.15.119:0/1727898492 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f056c00a9e0 con 0x7f05741595a0 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0570009420 con 0x7f05741595a0 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f05697fa700 1 -- 172.21.15.119:0/1727898492 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056c00ad80 con 0x7f05741595a0 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.927+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05740c25d0 con 0x7f05741595a0 2023-10-08T01:12:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.928+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057404c3c0 con 0x7f05741595a0 2023-10-08T01:12:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.929+0000 7f05697fa700 1 -- 172.21.15.119:0/1727898492 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f056c007300 con 0x7f05741595a0 2023-10-08T01:12:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.930+0000 7f05697fa700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0558066f30 0x7f05580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.930+0000 7f05697fa700 1 -- 172.21.15.119:0/1727898492 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f056c084aa0 con 0x7f05741595a0 2023-10-08T01:12:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.930+0000 7f057959b700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0558066f30 0x7f05580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:06.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.931+0000 7f057959b700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0558066f30 0x7f05580693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05700093f0 tx=0x7f0570017000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:06.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:06.934+0000 7f05697fa700 1 -- 172.21.15.119:0/1727898492 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f056c051b80 con 0x7f05741595a0 2023-10-08T01:12:07.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.121+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05740c2a80 con 0x7f0558066f30 2023-10-08T01:12:07.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.131+0000 7f05697fa700 1 -- 172.21.15.119:0/1727898492 <== mgr.6516 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284039 (secure 0 0 0) 0x7f05740c2a80 con 0x7f0558066f30 2023-10-08T01:12:07.136 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:07.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0558066f30 msgr2=0x7f05580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0558066f30 0x7f05580693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05700093f0 tx=0x7f0570017000 comp rx=0 tx=0).stop 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05741595a0 msgr2=0x7f05740c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05741595a0 0x7f05740c5990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f056c00ccc0 tx=0x7f056c001a20 comp rx=0 tx=0).stop 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 shutdown_connections 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0558066f30 0x7f05580693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05741595a0 0x7f05740c5990 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f05740b1de0 0x7f05740c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 --2- 172.21.15.119:0/1727898492 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f05740c6c60 0x7f0574151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 >> 172.21.15.119:0/1727898492 conn(0x7f057406ddb0 msgr2=0x7f0574153e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:07.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 shutdown_connections 2023-10-08T01:12:07.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.136+0000 7f057a59d700 1 -- 172.21.15.119:0/1727898492 wait complete. 2023-10-08T01:12:07.140 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:12:07.154 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:12:06.707159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590550,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":292,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2437172,"kb_used_data":9716,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752537548,"statfs":{"total":773094113280,"available":770598449152,"internally_reserved":0,"allocated":9949184,"data_stored":5392356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":182,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.906649"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866793+0000","last_change":"2023-10-08T01:12:02.861488+0000","last_active":"2023-10-08T01:12:02.866793+0000","last_peered":"2023-10-08T01:12:02.866793+0000","last_clean":"2023-10-08T01:12:02.866793+0000","last_became_active":"2023-10-08T01:12:02.860018+0000","last_became_peered":"2023-10-08T01:12:02.860018+0000","last_unstale":"2023-10-08T01:12:02.866793+0000","last_undegraded":"2023-10-08T01:12:02.866793+0000","last_fullsized":"2023-10-08T01:12:02.866793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.163695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846099+0000","last_change":"2023-10-08T01:12:02.841414+0000","last_active":"2023-10-08T01:12:02.846099+0000","last_peered":"2023-10-08T01:12:02.846099+0000","last_clean":"2023-10-08T01:12:02.846099+0000","last_became_active":"2023-10-08T01:12:02.841091+0000","last_became_peered":"2023-10-08T01:12:02.841091+0000","last_unstale":"2023-10-08T01:12:02.846099+0000","last_undegraded":"2023-10-08T01:12:02.846099+0000","last_fullsized":"2023-10-08T01:12:02.846099+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:03.550299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845916+0000","last_change":"2023-10-08T01:12:02.841353+0000","last_active":"2023-10-08T01:12:02.845916+0000","last_peered":"2023-10-08T01:12:02.845916+0000","last_clean":"2023-10-08T01:12:02.845916+0000","last_became_active":"2023-10-08T01:12:02.841117+0000","last_became_peered":"2023-10-08T01:12:02.841117+0000","last_unstale":"2023-10-08T01:12:02.845916+0000","last_undegraded":"2023-10-08T01:12:02.845916+0000","last_fullsized":"2023-10-08T01:12:02.845916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:32.447875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859766+0000","last_change":"2023-10-08T01:12:02.851025+0000","last_active":"2023-10-08T01:12:02.859766+0000","last_peered":"2023-10-08T01:12:02.859766+0000","last_clean":"2023-10-08T01:12:02.859766+0000","last_became_active":"2023-10-08T01:12:02.850765+0000","last_became_peered":"2023-10-08T01:12:02.850765+0000","last_unstale":"2023-10-08T01:12:02.859766+0000","last_undegraded":"2023-10-08T01:12:02.859766+0000","last_fullsized":"2023-10-08T01:12:02.859766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:09.116366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860705+0000","last_change":"2023-10-08T01:12:02.845674+0000","last_active":"2023-10-08T01:12:02.860705+0000","last_peered":"2023-10-08T01:12:02.860705+0000","last_clean":"2023-10-08T01:12:02.860705+0000","last_became_active":"2023-10-08T01:12:02.845358+0000","last_became_peered":"2023-10-08T01:12:02.845358+0000","last_unstale":"2023-10-08T01:12:02.860705+0000","last_undegraded":"2023-10-08T01:12:02.860705+0000","last_fullsized":"2023-10-08T01:12:02.860705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:47.976571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844181+0000","last_change":"2023-10-08T01:12:02.839332+0000","last_active":"2023-10-08T01:12:02.844181+0000","last_peered":"2023-10-08T01:12:02.844181+0000","last_clean":"2023-10-08T01:12:02.844181+0000","last_became_active":"2023-10-08T01:12:02.838641+0000","last_became_peered":"2023-10-08T01:12:02.838641+0000","last_unstale":"2023-10-08T01:12:02.844181+0000","last_undegraded":"2023-10-08T01:12:02.844181+0000","last_fullsized":"2023-10-08T01:12:02.844181+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:24.293015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851974+0000","last_change":"2023-10-08T01:12:02.849866+0000","last_active":"2023-10-08T01:12:02.851974+0000","last_peered":"2023-10-08T01:12:02.851974+0000","last_clean":"2023-10-08T01:12:02.851974+0000","last_became_active":"2023-10-08T01:12:02.849568+0000","last_became_peered":"2023-10-08T01:12:02.849568+0000","last_unstale":"2023-10-08T01:12:02.851974+0000","last_undegraded":"2023-10-08T01:12:02.851974+0000","last_fullsized":"2023-10-08T01:12:02.851974+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:30.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846415+0000","last_change":"2023-10-08T01:12:02.843295+0000","last_active":"2023-10-08T01:12:02.846415+0000","last_peered":"2023-10-08T01:12:02.846415+0000","last_clean":"2023-10-08T01:12:02.846415+0000","last_became_active":"2023-10-08T01:12:02.842892+0000","last_became_peered":"2023-10-08T01:12:02.842892+0000","last_unstale":"2023-10-08T01:12:02.846415+0000","last_undegraded":"2023-10-08T01:12:02.846415+0000","last_fullsized":"2023-10-08T01:12:02.846415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:20.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851263+0000","last_change":"2023-10-08T01:12:02.847055+0000","last_active":"2023-10-08T01:12:02.851263+0000","last_peered":"2023-10-08T01:12:02.851263+0000","last_clean":"2023-10-08T01:12:02.851263+0000","last_became_active":"2023-10-08T01:12:02.846767+0000","last_became_peered":"2023-10-08T01:12:02.846767+0000","last_unstale":"2023-10-08T01:12:02.851263+0000","last_undegraded":"2023-10-08T01:12:02.851263+0000","last_fullsized":"2023-10-08T01:12:02.851263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:52.932416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844213+0000","last_change":"2023-10-08T01:12:02.840294+0000","last_active":"2023-10-08T01:12:02.844213+0000","last_peered":"2023-10-08T01:12:02.844213+0000","last_clean":"2023-10-08T01:12:02.844213+0000","last_became_active":"2023-10-08T01:12:02.838656+0000","last_became_peered":"2023-10-08T01:12:02.838656+0000","last_unstale":"2023-10-08T01:12:02.844213+0000","last_undegraded":"2023-10-08T01:12:02.844213+0000","last_fullsized":"2023-10-08T01:12:02.844213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:57.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866715+0000","last_change":"2023-10-08T01:12:02.860578+0000","last_active":"2023-10-08T01:12:02.866715+0000","last_peered":"2023-10-08T01:12:02.866715+0000","last_clean":"2023-10-08T01:12:02.866715+0000","last_became_active":"2023-10-08T01:12:02.859906+0000","last_became_peered":"2023-10-08T01:12:02.859906+0000","last_unstale":"2023-10-08T01:12:02.866715+0000","last_undegraded":"2023-10-08T01:12:02.866715+0000","last_fullsized":"2023-10-08T01:12:02.866715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:48.742822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853436+0000","last_change":"2023-10-08T01:12:02.851814+0000","last_active":"2023-10-08T01:12:02.853436+0000","last_peered":"2023-10-08T01:12:02.853436+0000","last_clean":"2023-10-08T01:12:02.853436+0000","last_became_active":"2023-10-08T01:12:02.851488+0000","last_became_peered":"2023-10-08T01:12:02.851488+0000","last_unstale":"2023-10-08T01:12:02.853436+0000","last_undegraded":"2023-10-08T01:12:02.853436+0000","last_fullsized":"2023-10-08T01:12:02.853436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:37.330422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846116+0000","last_change":"2023-10-08T01:12:02.845033+0000","last_active":"2023-10-08T01:12:02.846116+0000","last_peered":"2023-10-08T01:12:02.846116+0000","last_clean":"2023-10-08T01:12:02.846116+0000","last_became_active":"2023-10-08T01:12:02.844545+0000","last_became_peered":"2023-10-08T01:12:02.844545+0000","last_unstale":"2023-10-08T01:12:02.846116+0000","last_undegraded":"2023-10-08T01:12:02.846116+0000","last_fullsized":"2023-10-08T01:12:02.846116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:36.576301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845074+0000","last_change":"2023-10-08T01:12:02.839789+0000","last_active":"2023-10-08T01:12:02.845074+0000","last_peered":"2023-10-08T01:12:02.845074+0000","last_clean":"2023-10-08T01:12:02.845074+0000","last_became_active":"2023-10-08T01:12:02.839516+0000","last_became_peered":"2023-10-08T01:12:02.839516+0000","last_unstale":"2023-10-08T01:12:02.845074+0000","last_undegraded":"2023-10-08T01:12:02.845074+0000","last_fullsized":"2023-10-08T01:12:02.845074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:42.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860840+0000","last_change":"2023-10-08T01:12:02.854035+0000","last_active":"2023-10-08T01:12:02.860840+0000","last_peered":"2023-10-08T01:12:02.860840+0000","last_clean":"2023-10-08T01:12:02.860840+0000","last_became_active":"2023-10-08T01:12:02.853704+0000","last_became_peered":"2023-10-08T01:12:02.853704+0000","last_unstale":"2023-10-08T01:12:02.860840+0000","last_undegraded":"2023-10-08T01:12:02.860840+0000","last_fullsized":"2023-10-08T01:12:02.860840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:55.363740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807321+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844198+0000","last_change":"2023-10-08T01:12:02.839586+0000","last_active":"2023-10-08T01:12:02.844198+0000","last_peered":"2023-10-08T01:12:02.844198+0000","last_clean":"2023-10-08T01:12:02.844198+0000","last_became_active":"2023-10-08T01:12:02.839033+0000","last_became_peered":"2023-10-08T01:12:02.839033+0000","last_unstale":"2023-10-08T01:12:02.844198+0000","last_undegraded":"2023-10-08T01:12:02.844198+0000","last_fullsized":"2023-10-08T01:12:02.844198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:43.348391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856349+0000","last_change":"2023-10-08T01:12:01.818263+0000","last_active":"2023-10-08T01:12:02.856349+0000","last_peered":"2023-10-08T01:12:02.856349+0000","last_clean":"2023-10-08T01:12:02.856349+0000","last_became_active":"2023-10-08T01:12:01.818011+0000","last_became_peered":"2023-10-08T01:12:01.818011+0000","last_unstale":"2023-10-08T01:12:02.856349+0000","last_undegraded":"2023-10-08T01:12:02.856349+0000","last_fullsized":"2023-10-08T01:12:02.856349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860732+0000","last_change":"2023-10-08T01:12:02.857609+0000","last_active":"2023-10-08T01:12:02.860732+0000","last_peered":"2023-10-08T01:12:02.860732+0000","last_clean":"2023-10-08T01:12:02.860732+0000","last_became_active":"2023-10-08T01:12:02.857277+0000","last_became_peered":"2023-10-08T01:12:02.857277+0000","last_unstale":"2023-10-08T01:12:02.860732+0000","last_undegraded":"2023-10-08T01:12:02.860732+0000","last_fullsized":"2023-10-08T01:12:02.860732+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.729095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804484+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:03.061391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.868186+0000","last_change":"2023-10-08T01:12:01.831258+0000","last_active":"2023-10-08T01:12:02.868186+0000","last_peered":"2023-10-08T01:12:02.868186+0000","last_clean":"2023-10-08T01:12:02.868186+0000","last_became_active":"2023-10-08T01:12:01.830943+0000","last_became_peered":"2023-10-08T01:12:01.830943+0000","last_unstale":"2023-10-08T01:12:02.868186+0000","last_undegraded":"2023-10-08T01:12:02.868186+0000","last_fullsized":"2023-10-08T01:12:02.868186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853340+0000","last_change":"2023-10-08T01:12:01.826432+0000","last_active":"2023-10-08T01:12:02.853340+0000","last_peered":"2023-10-08T01:12:02.853340+0000","last_clean":"2023-10-08T01:12:02.853340+0000","last_became_active":"2023-10-08T01:12:01.826156+0000","last_became_peered":"2023-10-08T01:12:01.826156+0000","last_unstale":"2023-10-08T01:12:02.853340+0000","last_undegraded":"2023-10-08T01:12:02.853340+0000","last_fullsized":"2023-10-08T01:12:02.853340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:26.787866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867313+0000","last_change":"2023-10-08T01:12:01.833334+0000","last_active":"2023-10-08T01:12:02.867313+0000","last_peered":"2023-10-08T01:12:02.867313+0000","last_clean":"2023-10-08T01:12:02.867313+0000","last_became_active":"2023-10-08T01:12:01.832064+0000","last_became_peered":"2023-10-08T01:12:01.832064+0000","last_unstale":"2023-10-08T01:12:02.867313+0000","last_undegraded":"2023-10-08T01:12:02.867313+0000","last_fullsized":"2023-10-08T01:12:02.867313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:49.406724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846186+0000","last_change":"2023-10-08T01:12:02.843587+0000","last_active":"2023-10-08T01:12:02.846186+0000","last_peered":"2023-10-08T01:12:02.846186+0000","last_clean":"2023-10-08T01:12:02.846186+0000","last_became_active":"2023-10-08T01:12:02.843202+0000","last_became_peered":"2023-10-08T01:12:02.843202+0000","last_unstale":"2023-10-08T01:12:02.846186+0000","last_undegraded":"2023-10-08T01:12:02.846186+0000","last_fullsized":"2023-10-08T01:12:02.846186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:50.412239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809293+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:23:16.271266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851277+0000","last_change":"2023-10-08T01:12:02.847761+0000","last_active":"2023-10-08T01:12:02.851277+0000","last_peered":"2023-10-08T01:12:02.851277+0000","last_clean":"2023-10-08T01:12:02.851277+0000","last_became_active":"2023-10-08T01:12:02.847425+0000","last_became_peered":"2023-10-08T01:12:02.847425+0000","last_unstale":"2023-10-08T01:12:02.851277+0000","last_undegraded":"2023-10-08T01:12:02.851277+0000","last_fullsized":"2023-10-08T01:12:02.851277+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:12:47.987269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.826270+0000","last_change":"2023-10-08T01:12:01.822270+0000","last_active":"2023-10-08T01:12:02.826270+0000","last_peered":"2023-10-08T01:12:02.826270+0000","last_clean":"2023-10-08T01:12:02.826270+0000","last_became_active":"2023-10-08T01:12:01.821995+0000","last_became_peered":"2023-10-08T01:12:01.821995+0000","last_unstale":"2023-10-08T01:12:02.826270+0000","last_undegraded":"2023-10-08T01:12:02.826270+0000","last_fullsized":"2023-10-08T01:12:02.826270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:40.166452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804453+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:38.169246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852304+0000","last_change":"2023-10-08T01:12:01.832041+0000","last_active":"2023-10-08T01:12:02.852304+0000","last_peered":"2023-10-08T01:12:02.852304+0000","last_clean":"2023-10-08T01:12:02.852304+0000","last_became_active":"2023-10-08T01:12:01.831603+0000","last_became_peered":"2023-10-08T01:12:01.831603+0000","last_unstale":"2023-10-08T01:12:02.852304+0000","last_undegraded":"2023-10-08T01:12:02.852304+0000","last_fullsized":"2023-10-08T01:12:02.852304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:42.342876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860424+0000","last_change":"2023-10-08T01:12:02.855912+0000","last_active":"2023-10-08T01:12:02.860424+0000","last_peered":"2023-10-08T01:12:02.860424+0000","last_clean":"2023-10-08T01:12:02.860424+0000","last_became_active":"2023-10-08T01:12:02.855628+0000","last_became_peered":"2023-10-08T01:12:02.855628+0000","last_unstale":"2023-10-08T01:12:02.860424+0000","last_undegraded":"2023-10-08T01:12:02.860424+0000","last_fullsized":"2023-10-08T01:12:02.860424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:48.158503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"90'3","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856557+0000","last_change":"2023-10-08T01:12:01.825659+0000","last_active":"2023-10-08T01:12:02.856557+0000","last_peered":"2023-10-08T01:12:02.856557+0000","last_clean":"2023-10-08T01:12:02.856557+0000","last_became_active":"2023-10-08T01:12:01.825371+0000","last_became_peered":"2023-10-08T01:12:01.825371+0000","last_unstale":"2023-10-08T01:12:02.856557+0000","last_undegraded":"2023-10-08T01:12:02.856557+0000","last_fullsized":"2023-10-08T01:12:02.856557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:46.049146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843348+0000","last_change":"2023-10-08T01:12:02.837217+0000","last_active":"2023-10-08T01:12:02.843348+0000","last_peered":"2023-10-08T01:12:02.843348+0000","last_clean":"2023-10-08T01:12:02.843348+0000","last_became_active":"2023-10-08T01:12:02.836983+0000","last_became_peered":"2023-10-08T01:12:02.836983+0000","last_unstale":"2023-10-08T01:12:02.843348+0000","last_undegraded":"2023-10-08T01:12:02.843348+0000","last_fullsized":"2023-10-08T01:12:02.843348+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:31.604511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809400+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809631+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850403+0000","last_change":"2023-10-08T01:12:01.829191+0000","last_active":"2023-10-08T01:12:02.850403+0000","last_peered":"2023-10-08T01:12:02.850403+0000","last_clean":"2023-10-08T01:12:02.850403+0000","last_became_active":"2023-10-08T01:12:01.828873+0000","last_became_peered":"2023-10-08T01:12:01.828873+0000","last_unstale":"2023-10-08T01:12:02.850403+0000","last_undegraded":"2023-10-08T01:12:02.850403+0000","last_fullsized":"2023-10-08T01:12:02.850403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:31.442360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851325+0000","last_change":"2023-10-08T01:12:02.848989+0000","last_active":"2023-10-08T01:12:02.851325+0000","last_peered":"2023-10-08T01:12:02.851325+0000","last_clean":"2023-10-08T01:12:02.851325+0000","last_became_active":"2023-10-08T01:12:02.848775+0000","last_became_peered":"2023-10-08T01:12:02.848775+0000","last_unstale":"2023-10-08T01:12:02.851325+0000","last_undegraded":"2023-10-08T01:12:02.851325+0000","last_fullsized":"2023-10-08T01:12:02.851325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:30.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.808808+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:47.152989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843699+0000","last_change":"2023-10-08T01:12:02.837963+0000","last_active":"2023-10-08T01:12:02.843699+0000","last_peered":"2023-10-08T01:12:02.843699+0000","last_clean":"2023-10-08T01:12:02.843699+0000","last_became_active":"2023-10-08T01:12:02.837114+0000","last_became_peered":"2023-10-08T01:12:02.837114+0000","last_unstale":"2023-10-08T01:12:02.843699+0000","last_undegraded":"2023-10-08T01:12:02.843699+0000","last_fullsized":"2023-10-08T01:12:02.843699+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:42.075297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804185+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:38.855490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867962+0000","last_change":"2023-10-08T01:12:01.831319+0000","last_active":"2023-10-08T01:12:02.867962+0000","last_peered":"2023-10-08T01:12:02.867962+0000","last_clean":"2023-10-08T01:12:02.867962+0000","last_became_active":"2023-10-08T01:12:01.831046+0000","last_became_peered":"2023-10-08T01:12:01.831046+0000","last_unstale":"2023-10-08T01:12:02.867962+0000","last_undegraded":"2023-10-08T01:12:02.867962+0000","last_fullsized":"2023-10-08T01:12:02.867962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804397+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:54.295030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867409+0000","last_change":"2023-10-08T01:12:01.829452+0000","last_active":"2023-10-08T01:12:02.867409+0000","last_peered":"2023-10-08T01:12:02.867409+0000","last_clean":"2023-10-08T01:12:02.867409+0000","last_became_active":"2023-10-08T01:12:01.829140+0000","last_became_peered":"2023-10-08T01:12:01.829140+0000","last_unstale":"2023-10-08T01:12:02.867409+0000","last_undegraded":"2023-10-08T01:12:02.867409+0000","last_fullsized":"2023-10-08T01:12:02.867409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:51.755961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843809+0000","last_change":"2023-10-08T01:12:02.840709+0000","last_active":"2023-10-08T01:12:02.843809+0000","last_peered":"2023-10-08T01:12:02.843809+0000","last_clean":"2023-10-08T01:12:02.843809+0000","last_became_active":"2023-10-08T01:12:02.840250+0000","last_became_peered":"2023-10-08T01:12:02.840250+0000","last_unstale":"2023-10-08T01:12:02.843809+0000","last_undegraded":"2023-10-08T01:12:02.843809+0000","last_fullsized":"2023-10-08T01:12:02.843809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:26.311091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807321+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.838339+0000","last_change":"2023-10-08T01:12:02.836913+0000","last_active":"2023-10-08T01:12:02.838339+0000","last_peered":"2023-10-08T01:12:02.838339+0000","last_clean":"2023-10-08T01:12:02.838339+0000","last_became_active":"2023-10-08T01:12:02.836618+0000","last_became_peered":"2023-10-08T01:12:02.836618+0000","last_unstale":"2023-10-08T01:12:02.838339+0000","last_undegraded":"2023-10-08T01:12:02.838339+0000","last_fullsized":"2023-10-08T01:12:02.838339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:05.095868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850521+0000","last_change":"2023-10-08T01:12:01.831714+0000","last_active":"2023-10-08T01:12:02.850521+0000","last_peered":"2023-10-08T01:12:02.850521+0000","last_clean":"2023-10-08T01:12:02.850521+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850521+0000","last_undegraded":"2023-10-08T01:12:02.850521+0000","last_fullsized":"2023-10-08T01:12:02.850521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:56.535520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.862945+0000","last_change":"2023-10-08T01:12:02.862541+0000","last_active":"2023-10-08T01:12:02.862945+0000","last_peered":"2023-10-08T01:12:02.862945+0000","last_clean":"2023-10-08T01:12:02.862945+0000","last_became_active":"2023-10-08T01:12:02.862076+0000","last_became_peered":"2023-10-08T01:12:02.862076+0000","last_unstale":"2023-10-08T01:12:02.862945+0000","last_undegraded":"2023-10-08T01:12:02.862945+0000","last_fullsized":"2023-10-08T01:12:02.862945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:26.919061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807911+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:46.971056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852339+0000","last_change":"2023-10-08T01:12:02.849161+0000","last_active":"2023-10-08T01:12:02.852339+0000","last_peered":"2023-10-08T01:12:02.852339+0000","last_clean":"2023-10-08T01:12:02.852339+0000","last_became_active":"2023-10-08T01:12:02.848913+0000","last_became_peered":"2023-10-08T01:12:02.848913+0000","last_unstale":"2023-10-08T01:12:02.852339+0000","last_undegraded":"2023-10-08T01:12:02.852339+0000","last_fullsized":"2023-10-08T01:12:02.852339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:58:40.796959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.832971+0000","last_change":"2023-10-08T01:12:01.819602+0000","last_active":"2023-10-08T01:12:02.832971+0000","last_peered":"2023-10-08T01:12:02.832971+0000","last_clean":"2023-10-08T01:12:02.832971+0000","last_became_active":"2023-10-08T01:12:01.819304+0000","last_became_peered":"2023-10-08T01:12:01.819304+0000","last_unstale":"2023-10-08T01:12:02.832971+0000","last_undegraded":"2023-10-08T01:12:02.832971+0000","last_fullsized":"2023-10-08T01:12:02.832971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:55.795619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846380+0000","last_change":"2023-10-08T01:12:02.844295+0000","last_active":"2023-10-08T01:12:02.846380+0000","last_peered":"2023-10-08T01:12:02.846380+0000","last_clean":"2023-10-08T01:12:02.846380+0000","last_became_active":"2023-10-08T01:12:02.844001+0000","last_became_peered":"2023-10-08T01:12:02.844001+0000","last_unstale":"2023-10-08T01:12:02.846380+0000","last_undegraded":"2023-10-08T01:12:02.846380+0000","last_fullsized":"2023-10-08T01:12:02.846380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:39:36.571615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867968+0000","last_change":"2023-10-08T01:12:01.831354+0000","last_active":"2023-10-08T01:12:02.867968+0000","last_peered":"2023-10-08T01:12:02.867968+0000","last_clean":"2023-10-08T01:12:02.867968+0000","last_became_active":"2023-10-08T01:12:01.830964+0000","last_became_peered":"2023-10-08T01:12:01.830964+0000","last_unstale":"2023-10-08T01:12:02.867968+0000","last_undegraded":"2023-10-08T01:12:02.867968+0000","last_fullsized":"2023-10-08T01:12:02.867968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809329+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:08.110852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828282+0000","last_change":"2023-10-08T01:12:01.823222+0000","last_active":"2023-10-08T01:12:02.828282+0000","last_peered":"2023-10-08T01:12:02.828282+0000","last_clean":"2023-10-08T01:12:02.828282+0000","last_became_active":"2023-10-08T01:12:01.822970+0000","last_became_peered":"2023-10-08T01:12:01.822970+0000","last_unstale":"2023-10-08T01:12:02.828282+0000","last_undegraded":"2023-10-08T01:12:02.828282+0000","last_fullsized":"2023-10-08T01:12:02.828282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:00.437245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834238+0000","last_change":"2023-10-08T01:12:01.815771+0000","last_active":"2023-10-08T01:12:02.834238+0000","last_peered":"2023-10-08T01:12:02.834238+0000","last_clean":"2023-10-08T01:12:02.834238+0000","last_became_active":"2023-10-08T01:12:01.815502+0000","last_became_peered":"2023-10-08T01:12:01.815502+0000","last_unstale":"2023-10-08T01:12:02.834238+0000","last_undegraded":"2023-10-08T01:12:02.834238+0000","last_fullsized":"2023-10-08T01:12:02.834238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843426+0000","last_change":"2023-10-08T01:12:02.838180+0000","last_active":"2023-10-08T01:12:02.843426+0000","last_peered":"2023-10-08T01:12:02.843426+0000","last_clean":"2023-10-08T01:12:02.843426+0000","last_became_active":"2023-10-08T01:12:02.837945+0000","last_became_peered":"2023-10-08T01:12:02.837945+0000","last_unstale":"2023-10-08T01:12:02.843426+0000","last_undegraded":"2023-10-08T01:12:02.843426+0000","last_fullsized":"2023-10-08T01:12:02.843426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:21.100043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.827017+0000","last_change":"2023-10-08T01:12:01.827250+0000","last_active":"2023-10-08T01:12:02.827017+0000","last_peered":"2023-10-08T01:12:02.827017+0000","last_clean":"2023-10-08T01:12:02.827017+0000","last_became_active":"2023-10-08T01:12:01.826935+0000","last_became_peered":"2023-10-08T01:12:01.826935+0000","last_unstale":"2023-10-08T01:12:02.827017+0000","last_undegraded":"2023-10-08T01:12:02.827017+0000","last_fullsized":"2023-10-08T01:12:02.827017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:40.815318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860494+0000","last_change":"2023-10-08T01:12:02.856747+0000","last_active":"2023-10-08T01:12:02.860494+0000","last_peered":"2023-10-08T01:12:02.860494+0000","last_clean":"2023-10-08T01:12:02.860494+0000","last_became_active":"2023-10-08T01:12:02.856476+0000","last_became_peered":"2023-10-08T01:12:02.856476+0000","last_unstale":"2023-10-08T01:12:02.860494+0000","last_undegraded":"2023-10-08T01:12:02.860494+0000","last_fullsized":"2023-10-08T01:12:02.860494+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:00.451849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809989+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804185+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:50.137372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859915+0000","last_change":"2023-10-08T01:12:02.855156+0000","last_active":"2023-10-08T01:12:02.859915+0000","last_peered":"2023-10-08T01:12:02.859915+0000","last_clean":"2023-10-08T01:12:02.859915+0000","last_became_active":"2023-10-08T01:12:02.854809+0000","last_became_peered":"2023-10-08T01:12:02.854809+0000","last_unstale":"2023-10-08T01:12:02.859915+0000","last_undegraded":"2023-10-08T01:12:02.859915+0000","last_fullsized":"2023-10-08T01:12:02.859915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:46.539103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850388+0000","last_change":"2023-10-08T01:12:01.831667+0000","last_active":"2023-10-08T01:12:02.850388+0000","last_peered":"2023-10-08T01:12:02.850388+0000","last_clean":"2023-10-08T01:12:02.850388+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850388+0000","last_undegraded":"2023-10-08T01:12:02.850388+0000","last_fullsized":"2023-10-08T01:12:02.850388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:23.287500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852439+0000","last_change":"2023-10-08T01:12:02.849116+0000","last_active":"2023-10-08T01:12:02.852439+0000","last_peered":"2023-10-08T01:12:02.852439+0000","last_clean":"2023-10-08T01:12:02.852439+0000","last_became_active":"2023-10-08T01:12:02.848833+0000","last_became_peered":"2023-10-08T01:12:02.848833+0000","last_unstale":"2023-10-08T01:12:02.852439+0000","last_undegraded":"2023-10-08T01:12:02.852439+0000","last_fullsized":"2023-10-08T01:12:02.852439+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:35.798359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.870650+0000","last_change":"2023-10-08T01:12:01.828087+0000","last_active":"2023-10-08T01:12:02.870650+0000","last_peered":"2023-10-08T01:12:02.870650+0000","last_clean":"2023-10-08T01:12:02.870650+0000","last_became_active":"2023-10-08T01:12:01.827688+0000","last_became_peered":"2023-10-08T01:12:01.827688+0000","last_unstale":"2023-10-08T01:12:02.870650+0000","last_undegraded":"2023-10-08T01:12:02.870650+0000","last_fullsized":"2023-10-08T01:12:02.870650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:25.305577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"80'183","reported_seq":457,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T01:11:58.219508+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:11:58.219508+0000","last_peered":"2023-10-08T01:11:58.219508+0000","last_clean":"2023-10-08T01:11:58.219508+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:11:58.219508+0000","last_undegraded":"2023-10-08T01:11:58.219508+0000","last_fullsized":"2023-10-08T01:11:58.219508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":4},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":182,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1667072,"data_stored":1106358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.58599999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.72599999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.623}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1118208,"data_stored":536572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.497},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.74199999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.83699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.28599999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.495},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.85099999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":1.026},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.92800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303048,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068792,"statfs":{"total":96636764160,"available":96326443008,"internally_reserved":0,"allocated":1703936,"data_stored":1126848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50800000000000001,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.617},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.5289999999999999},"last":0.754}]},{"osd":6,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.208,"5min":1.361,"15min":1.3779999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1719999999999999,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.65000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306576,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065264,"statfs":{"total":96636764160,"available":96322830336,"internally_reserved":0,"allocated":1122304,"data_stored":538016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306572,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065268,"statfs":{"total":96636764160,"available":96322834432,"internally_reserved":0,"allocated":1118208,"data_stored":536592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.69999999999999996,"15min":0.80100000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.57399999999999995,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.378}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61699999999999999,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.82999999999999996,"15min":0.88100000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.79400000000000004,"15min":1.1060000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":1.095,"15min":1.3029999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74299999999999999,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.117,"15min":1.413},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.77800000000000002},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.0920000000000001,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":1.089},"last":0.61899999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.379},"last":0.751},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.486},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97099999999999997,"5min":1.0840000000000001,"15min":1.389},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.79700000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.5},"max":{"1min":0.93700000000000006,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.65500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310624,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061216,"statfs":{"total":96636764160,"available":96318685184,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.72199999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.75}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.80900000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.66700000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.88300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.66100000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.622}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.749},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.76300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:12:07.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:12:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.374+0000 7ff9bd39d700 1 Processor -- start 2023-10-08T01:12:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.374+0000 7ff9bd39d700 1 -- start start 2023-10-08T01:12:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.375+0000 7ff9bd39d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff9b8156310 0x7ff9b80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.375+0000 7ff9bd39d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b80ca860 con 0x7ff9b81566e0 2023-10-08T01:12:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.375+0000 7ff9bd39d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b80cab40 con 0x7ff9b80ae710 2023-10-08T01:12:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.375+0000 7ff9bd39d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b80cae20 con 0x7ff9b8156310 2023-10-08T01:12:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.375+0000 7ff9b77fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff9b81566e0 0x7ff9b80ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52222/0 (socket says 172.21.15.119:52222) 2023-10-08T01:12:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.375+0000 7ff9b77fe700 1 -- 172.21.15.119:0/2306478391 learned_addr learned my addr 172.21.15.119:0/2306478391 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.375+0000 7ff9b7fff700 1 --2- 172.21.15.119:0/2306478391 >> v2:172.21.15.184:3300/0 conn(0x7ff9b8156310 0x7ff9b80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b6ffd700 1 -- 172.21.15.119:0/2306478391 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570332754 0 0) 0x7ff9b80ca860 con 0x7ff9b81566e0 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b6ffd700 1 -- 172.21.15.119:0/2306478391 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a0003570 con 0x7ff9b81566e0 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b6ffd700 1 -- 172.21.15.119:0/2306478391 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370581241 0 0) 0x7ff9b80cab40 con 0x7ff9b80ae710 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b6ffd700 1 -- 172.21.15.119:0/2306478391 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b80ca860 con 0x7ff9b80ae710 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b7fff700 1 -- 172.21.15.119:0/2306478391 >> v1:172.21.15.184:6789/0 conn(0x7ff9b80ae710 legacy=0x7ff9b80c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b7fff700 1 -- 172.21.15.119:0/2306478391 >> v1:172.21.15.119:6789/0 conn(0x7ff9b81566e0 legacy=0x7ff9b80ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b7fff700 1 -- 172.21.15.119:0/2306478391 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80cb100 con 0x7ff9b8156310 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b7fff700 1 --2- 172.21.15.119:0/2306478391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 0x7ff9b80c9dc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80098e0 tx=0x7ff9a80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90e65756c1492453 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.376+0000 7ff9b6ffd700 1 -- 172.21.15.119:0/2306478391 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8011630 con 0x7ff9b8156310 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9b6ffd700 1 -- 172.21.15.119:0/2306478391 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a8011790 con 0x7ff9b8156310 2023-10-08T01:12:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9b6ffd700 1 -- 172.21.15.119:0/2306478391 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8011b30 con 0x7ff9b8156310 2023-10-08T01:12:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9bd39d700 1 -- 172.21.15.119:0/2306478391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 msgr2=0x7ff9b80c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9bd39d700 1 --2- 172.21.15.119:0/2306478391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 0x7ff9b80c9dc0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80098e0 tx=0x7ff9a80091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9bd39d700 1 -- 172.21.15.119:0/2306478391 shutdown_connections 2023-10-08T01:12:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9bd39d700 1 --2- 172.21.15.119:0/2306478391 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 0x7ff9b80c9dc0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9bd39d700 1 -- 172.21.15.119:0/2306478391 >> 172.21.15.119:0/2306478391 conn(0x7ff9b80cc1f0 msgr2=0x7ff9b80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9bd39d700 1 -- 172.21.15.119:0/2306478391 shutdown_connections 2023-10-08T01:12:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.377+0000 7ff9bd39d700 1 -- 172.21.15.119:0/2306478391 wait complete. 2023-10-08T01:12:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.379+0000 7ff9bd39d700 1 Processor -- start 2023-10-08T01:12:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.379+0000 7ff9bd39d700 1 -- start start 2023-10-08T01:12:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.379+0000 7ff9bd39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b80ae710 0x7ff9b815a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.379+0000 7ff9bd39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 0x7ff9b80c2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bd39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9b81566e0 0x7ff9b80c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bd39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b815a9e0 con 0x7ff9b81566e0 2023-10-08T01:12:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bd39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80c2f80 con 0x7ff9b8156310 2023-10-08T01:12:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bd39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8159a00 con 0x7ff9b80ae710 2023-10-08T01:12:07.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bcb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9b81566e0 0x7ff9b80c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bcb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9b81566e0 0x7ff9b80c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55152/0 (socket says 172.21.15.119:55152) 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bcb9c700 1 -- 172.21.15.119:0/3222558762 learned_addr learned my addr 172.21.15.119:0/3222558762 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9b77fe700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 0x7ff9b80c2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9b7fff700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b80ae710 0x7ff9b815a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bcb9c700 1 -- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b80ae710 msgr2=0x7ff9b815a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bcb9c700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b80ae710 0x7ff9b815a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bcb9c700 1 -- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 msgr2=0x7ff9b80c2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bcb9c700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 0x7ff9b80c2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9bcb9c700 1 -- 172.21.15.119:0/3222558762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a8009420 con 0x7ff9b81566e0 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.380+0000 7ff9b7fff700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b80ae710 0x7ff9b815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.381+0000 7ff9bcb9c700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9b81566e0 0x7ff9b80c76b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000c830 tx=0x7ff9b0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:07.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.381+0000 7ff99ffff700 1 -- 172.21.15.119:0/3222558762 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0012630 con 0x7ff9b81566e0 2023-10-08T01:12:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.381+0000 7ff99ffff700 1 -- 172.21.15.119:0/3222558762 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b0012790 con 0x7ff9b81566e0 2023-10-08T01:12:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.381+0000 7ff99ffff700 1 -- 172.21.15.119:0/3222558762 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b0012b50 con 0x7ff9b81566e0 2023-10-08T01:12:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.381+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b816aa60 con 0x7ff9b81566e0 2023-10-08T01:12:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.381+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b8159f80 con 0x7ff9b81566e0 2023-10-08T01:12:07.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.382+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b804c3c0 con 0x7ff9b81566e0 2023-10-08T01:12:07.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.383+0000 7ff99ffff700 1 -- 172.21.15.119:0/3222558762 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff9b001b400 con 0x7ff9b81566e0 2023-10-08T01:12:07.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.384+0000 7ff99ffff700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff9a4066f30 0x7ff9a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.384+0000 7ff99ffff700 1 -- 172.21.15.119:0/3222558762 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff9b00849c0 con 0x7ff9b81566e0 2023-10-08T01:12:07.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.384+0000 7ff9b7fff700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff9a4066f30 0x7ff9a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.385+0000 7ff9b7fff700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff9a4066f30 0x7ff9a40693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80093f0 tx=0x7ff9a8017000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:07.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.388+0000 7ff99ffff700 1 -- 172.21.15.119:0/3222558762 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff9b0051aa0 con 0x7ff9b81566e0 2023-10-08T01:12:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.574+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff9b8159860 con 0x7ff9a4066f30 2023-10-08T01:12:07.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.583+0000 7ff99ffff700 1 -- 172.21.15.119:0/3222558762 <== mgr.6516 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284039 (secure 0 0 0) 0x7ff9b8159860 con 0x7ff9a4066f30 2023-10-08T01:12:07.588 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:07.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff9a4066f30 msgr2=0x7ff9a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff9a4066f30 0x7ff9a40693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9a80093f0 tx=0x7ff9a8017000 comp rx=0 tx=0).stop 2023-10-08T01:12:07.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9b81566e0 msgr2=0x7ff9b80c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9b81566e0 0x7ff9b80c76b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000c830 tx=0x7ff9b0007f90 comp rx=0 tx=0).stop 2023-10-08T01:12:07.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 shutdown_connections 2023-10-08T01:12:07.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff9a4066f30 0x7ff9a40693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff9b80ae710 0x7ff9b815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9b81566e0 0x7ff9b80c76b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 --2- 172.21.15.119:0/3222558762 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff9b8156310 0x7ff9b80c2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 >> 172.21.15.119:0/3222558762 conn(0x7ff9b80cc1f0 msgr2=0x7ff9b806ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:07.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 shutdown_connections 2023-10-08T01:12:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.587+0000 7ff9bd39d700 1 -- 172.21.15.119:0/3222558762 wait complete. 2023-10-08T01:12:07.594 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:12:07.605 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:12:06.707159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590550,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":292,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2437172,"kb_used_data":9716,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752537548,"statfs":{"total":773094113280,"available":770598449152,"internally_reserved":0,"allocated":9949184,"data_stored":5392356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":182,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.906649"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866793+0000","last_change":"2023-10-08T01:12:02.861488+0000","last_active":"2023-10-08T01:12:02.866793+0000","last_peered":"2023-10-08T01:12:02.866793+0000","last_clean":"2023-10-08T01:12:02.866793+0000","last_became_active":"2023-10-08T01:12:02.860018+0000","last_became_peered":"2023-10-08T01:12:02.860018+0000","last_unstale":"2023-10-08T01:12:02.866793+0000","last_undegraded":"2023-10-08T01:12:02.866793+0000","last_fullsized":"2023-10-08T01:12:02.866793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.163695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846099+0000","last_change":"2023-10-08T01:12:02.841414+0000","last_active":"2023-10-08T01:12:02.846099+0000","last_peered":"2023-10-08T01:12:02.846099+0000","last_clean":"2023-10-08T01:12:02.846099+0000","last_became_active":"2023-10-08T01:12:02.841091+0000","last_became_peered":"2023-10-08T01:12:02.841091+0000","last_unstale":"2023-10-08T01:12:02.846099+0000","last_undegraded":"2023-10-08T01:12:02.846099+0000","last_fullsized":"2023-10-08T01:12:02.846099+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:03.550299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845916+0000","last_change":"2023-10-08T01:12:02.841353+0000","last_active":"2023-10-08T01:12:02.845916+0000","last_peered":"2023-10-08T01:12:02.845916+0000","last_clean":"2023-10-08T01:12:02.845916+0000","last_became_active":"2023-10-08T01:12:02.841117+0000","last_became_peered":"2023-10-08T01:12:02.841117+0000","last_unstale":"2023-10-08T01:12:02.845916+0000","last_undegraded":"2023-10-08T01:12:02.845916+0000","last_fullsized":"2023-10-08T01:12:02.845916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:32.447875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859766+0000","last_change":"2023-10-08T01:12:02.851025+0000","last_active":"2023-10-08T01:12:02.859766+0000","last_peered":"2023-10-08T01:12:02.859766+0000","last_clean":"2023-10-08T01:12:02.859766+0000","last_became_active":"2023-10-08T01:12:02.850765+0000","last_became_peered":"2023-10-08T01:12:02.850765+0000","last_unstale":"2023-10-08T01:12:02.859766+0000","last_undegraded":"2023-10-08T01:12:02.859766+0000","last_fullsized":"2023-10-08T01:12:02.859766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:09.116366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860705+0000","last_change":"2023-10-08T01:12:02.845674+0000","last_active":"2023-10-08T01:12:02.860705+0000","last_peered":"2023-10-08T01:12:02.860705+0000","last_clean":"2023-10-08T01:12:02.860705+0000","last_became_active":"2023-10-08T01:12:02.845358+0000","last_became_peered":"2023-10-08T01:12:02.845358+0000","last_unstale":"2023-10-08T01:12:02.860705+0000","last_undegraded":"2023-10-08T01:12:02.860705+0000","last_fullsized":"2023-10-08T01:12:02.860705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:47.976571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844181+0000","last_change":"2023-10-08T01:12:02.839332+0000","last_active":"2023-10-08T01:12:02.844181+0000","last_peered":"2023-10-08T01:12:02.844181+0000","last_clean":"2023-10-08T01:12:02.844181+0000","last_became_active":"2023-10-08T01:12:02.838641+0000","last_became_peered":"2023-10-08T01:12:02.838641+0000","last_unstale":"2023-10-08T01:12:02.844181+0000","last_undegraded":"2023-10-08T01:12:02.844181+0000","last_fullsized":"2023-10-08T01:12:02.844181+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:24.293015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851974+0000","last_change":"2023-10-08T01:12:02.849866+0000","last_active":"2023-10-08T01:12:02.851974+0000","last_peered":"2023-10-08T01:12:02.851974+0000","last_clean":"2023-10-08T01:12:02.851974+0000","last_became_active":"2023-10-08T01:12:02.849568+0000","last_became_peered":"2023-10-08T01:12:02.849568+0000","last_unstale":"2023-10-08T01:12:02.851974+0000","last_undegraded":"2023-10-08T01:12:02.851974+0000","last_fullsized":"2023-10-08T01:12:02.851974+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:30.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846415+0000","last_change":"2023-10-08T01:12:02.843295+0000","last_active":"2023-10-08T01:12:02.846415+0000","last_peered":"2023-10-08T01:12:02.846415+0000","last_clean":"2023-10-08T01:12:02.846415+0000","last_became_active":"2023-10-08T01:12:02.842892+0000","last_became_peered":"2023-10-08T01:12:02.842892+0000","last_unstale":"2023-10-08T01:12:02.846415+0000","last_undegraded":"2023-10-08T01:12:02.846415+0000","last_fullsized":"2023-10-08T01:12:02.846415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:20.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851263+0000","last_change":"2023-10-08T01:12:02.847055+0000","last_active":"2023-10-08T01:12:02.851263+0000","last_peered":"2023-10-08T01:12:02.851263+0000","last_clean":"2023-10-08T01:12:02.851263+0000","last_became_active":"2023-10-08T01:12:02.846767+0000","last_became_peered":"2023-10-08T01:12:02.846767+0000","last_unstale":"2023-10-08T01:12:02.851263+0000","last_undegraded":"2023-10-08T01:12:02.851263+0000","last_fullsized":"2023-10-08T01:12:02.851263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:52.932416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844213+0000","last_change":"2023-10-08T01:12:02.840294+0000","last_active":"2023-10-08T01:12:02.844213+0000","last_peered":"2023-10-08T01:12:02.844213+0000","last_clean":"2023-10-08T01:12:02.844213+0000","last_became_active":"2023-10-08T01:12:02.838656+0000","last_became_peered":"2023-10-08T01:12:02.838656+0000","last_unstale":"2023-10-08T01:12:02.844213+0000","last_undegraded":"2023-10-08T01:12:02.844213+0000","last_fullsized":"2023-10-08T01:12:02.844213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:57.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866715+0000","last_change":"2023-10-08T01:12:02.860578+0000","last_active":"2023-10-08T01:12:02.866715+0000","last_peered":"2023-10-08T01:12:02.866715+0000","last_clean":"2023-10-08T01:12:02.866715+0000","last_became_active":"2023-10-08T01:12:02.859906+0000","last_became_peered":"2023-10-08T01:12:02.859906+0000","last_unstale":"2023-10-08T01:12:02.866715+0000","last_undegraded":"2023-10-08T01:12:02.866715+0000","last_fullsized":"2023-10-08T01:12:02.866715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:48.742822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853436+0000","last_change":"2023-10-08T01:12:02.851814+0000","last_active":"2023-10-08T01:12:02.853436+0000","last_peered":"2023-10-08T01:12:02.853436+0000","last_clean":"2023-10-08T01:12:02.853436+0000","last_became_active":"2023-10-08T01:12:02.851488+0000","last_became_peered":"2023-10-08T01:12:02.851488+0000","last_unstale":"2023-10-08T01:12:02.853436+0000","last_undegraded":"2023-10-08T01:12:02.853436+0000","last_fullsized":"2023-10-08T01:12:02.853436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:37.330422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846116+0000","last_change":"2023-10-08T01:12:02.845033+0000","last_active":"2023-10-08T01:12:02.846116+0000","last_peered":"2023-10-08T01:12:02.846116+0000","last_clean":"2023-10-08T01:12:02.846116+0000","last_became_active":"2023-10-08T01:12:02.844545+0000","last_became_peered":"2023-10-08T01:12:02.844545+0000","last_unstale":"2023-10-08T01:12:02.846116+0000","last_undegraded":"2023-10-08T01:12:02.846116+0000","last_fullsized":"2023-10-08T01:12:02.846116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:36.576301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845074+0000","last_change":"2023-10-08T01:12:02.839789+0000","last_active":"2023-10-08T01:12:02.845074+0000","last_peered":"2023-10-08T01:12:02.845074+0000","last_clean":"2023-10-08T01:12:02.845074+0000","last_became_active":"2023-10-08T01:12:02.839516+0000","last_became_peered":"2023-10-08T01:12:02.839516+0000","last_unstale":"2023-10-08T01:12:02.845074+0000","last_undegraded":"2023-10-08T01:12:02.845074+0000","last_fullsized":"2023-10-08T01:12:02.845074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:42.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860840+0000","last_change":"2023-10-08T01:12:02.854035+0000","last_active":"2023-10-08T01:12:02.860840+0000","last_peered":"2023-10-08T01:12:02.860840+0000","last_clean":"2023-10-08T01:12:02.860840+0000","last_became_active":"2023-10-08T01:12:02.853704+0000","last_became_peered":"2023-10-08T01:12:02.853704+0000","last_unstale":"2023-10-08T01:12:02.860840+0000","last_undegraded":"2023-10-08T01:12:02.860840+0000","last_fullsized":"2023-10-08T01:12:02.860840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:55.363740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807321+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844198+0000","last_change":"2023-10-08T01:12:02.839586+0000","last_active":"2023-10-08T01:12:02.844198+0000","last_peered":"2023-10-08T01:12:02.844198+0000","last_clean":"2023-10-08T01:12:02.844198+0000","last_became_active":"2023-10-08T01:12:02.839033+0000","last_became_peered":"2023-10-08T01:12:02.839033+0000","last_unstale":"2023-10-08T01:12:02.844198+0000","last_undegraded":"2023-10-08T01:12:02.844198+0000","last_fullsized":"2023-10-08T01:12:02.844198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:43.348391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856349+0000","last_change":"2023-10-08T01:12:01.818263+0000","last_active":"2023-10-08T01:12:02.856349+0000","last_peered":"2023-10-08T01:12:02.856349+0000","last_clean":"2023-10-08T01:12:02.856349+0000","last_became_active":"2023-10-08T01:12:01.818011+0000","last_became_peered":"2023-10-08T01:12:01.818011+0000","last_unstale":"2023-10-08T01:12:02.856349+0000","last_undegraded":"2023-10-08T01:12:02.856349+0000","last_fullsized":"2023-10-08T01:12:02.856349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860732+0000","last_change":"2023-10-08T01:12:02.857609+0000","last_active":"2023-10-08T01:12:02.860732+0000","last_peered":"2023-10-08T01:12:02.860732+0000","last_clean":"2023-10-08T01:12:02.860732+0000","last_became_active":"2023-10-08T01:12:02.857277+0000","last_became_peered":"2023-10-08T01:12:02.857277+0000","last_unstale":"2023-10-08T01:12:02.860732+0000","last_undegraded":"2023-10-08T01:12:02.860732+0000","last_fullsized":"2023-10-08T01:12:02.860732+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.729095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804484+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:03.061391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.868186+0000","last_change":"2023-10-08T01:12:01.831258+0000","last_active":"2023-10-08T01:12:02.868186+0000","last_peered":"2023-10-08T01:12:02.868186+0000","last_clean":"2023-10-08T01:12:02.868186+0000","last_became_active":"2023-10-08T01:12:01.830943+0000","last_became_peered":"2023-10-08T01:12:01.830943+0000","last_unstale":"2023-10-08T01:12:02.868186+0000","last_undegraded":"2023-10-08T01:12:02.868186+0000","last_fullsized":"2023-10-08T01:12:02.868186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853340+0000","last_change":"2023-10-08T01:12:01.826432+0000","last_active":"2023-10-08T01:12:02.853340+0000","last_peered":"2023-10-08T01:12:02.853340+0000","last_clean":"2023-10-08T01:12:02.853340+0000","last_became_active":"2023-10-08T01:12:01.826156+0000","last_became_peered":"2023-10-08T01:12:01.826156+0000","last_unstale":"2023-10-08T01:12:02.853340+0000","last_undegraded":"2023-10-08T01:12:02.853340+0000","last_fullsized":"2023-10-08T01:12:02.853340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:26.787866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867313+0000","last_change":"2023-10-08T01:12:01.833334+0000","last_active":"2023-10-08T01:12:02.867313+0000","last_peered":"2023-10-08T01:12:02.867313+0000","last_clean":"2023-10-08T01:12:02.867313+0000","last_became_active":"2023-10-08T01:12:01.832064+0000","last_became_peered":"2023-10-08T01:12:01.832064+0000","last_unstale":"2023-10-08T01:12:02.867313+0000","last_undegraded":"2023-10-08T01:12:02.867313+0000","last_fullsized":"2023-10-08T01:12:02.867313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:49.406724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846186+0000","last_change":"2023-10-08T01:12:02.843587+0000","last_active":"2023-10-08T01:12:02.846186+0000","last_peered":"2023-10-08T01:12:02.846186+0000","last_clean":"2023-10-08T01:12:02.846186+0000","last_became_active":"2023-10-08T01:12:02.843202+0000","last_became_peered":"2023-10-08T01:12:02.843202+0000","last_unstale":"2023-10-08T01:12:02.846186+0000","last_undegraded":"2023-10-08T01:12:02.846186+0000","last_fullsized":"2023-10-08T01:12:02.846186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:50.412239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809293+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:23:16.271266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851277+0000","last_change":"2023-10-08T01:12:02.847761+0000","last_active":"2023-10-08T01:12:02.851277+0000","last_peered":"2023-10-08T01:12:02.851277+0000","last_clean":"2023-10-08T01:12:02.851277+0000","last_became_active":"2023-10-08T01:12:02.847425+0000","last_became_peered":"2023-10-08T01:12:02.847425+0000","last_unstale":"2023-10-08T01:12:02.851277+0000","last_undegraded":"2023-10-08T01:12:02.851277+0000","last_fullsized":"2023-10-08T01:12:02.851277+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:12:47.987269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.826270+0000","last_change":"2023-10-08T01:12:01.822270+0000","last_active":"2023-10-08T01:12:02.826270+0000","last_peered":"2023-10-08T01:12:02.826270+0000","last_clean":"2023-10-08T01:12:02.826270+0000","last_became_active":"2023-10-08T01:12:01.821995+0000","last_became_peered":"2023-10-08T01:12:01.821995+0000","last_unstale":"2023-10-08T01:12:02.826270+0000","last_undegraded":"2023-10-08T01:12:02.826270+0000","last_fullsized":"2023-10-08T01:12:02.826270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:40.166452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804453+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:38.169246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852304+0000","last_change":"2023-10-08T01:12:01.832041+0000","last_active":"2023-10-08T01:12:02.852304+0000","last_peered":"2023-10-08T01:12:02.852304+0000","last_clean":"2023-10-08T01:12:02.852304+0000","last_became_active":"2023-10-08T01:12:01.831603+0000","last_became_peered":"2023-10-08T01:12:01.831603+0000","last_unstale":"2023-10-08T01:12:02.852304+0000","last_undegraded":"2023-10-08T01:12:02.852304+0000","last_fullsized":"2023-10-08T01:12:02.852304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:42.342876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860424+0000","last_change":"2023-10-08T01:12:02.855912+0000","last_active":"2023-10-08T01:12:02.860424+0000","last_peered":"2023-10-08T01:12:02.860424+0000","last_clean":"2023-10-08T01:12:02.860424+0000","last_became_active":"2023-10-08T01:12:02.855628+0000","last_became_peered":"2023-10-08T01:12:02.855628+0000","last_unstale":"2023-10-08T01:12:02.860424+0000","last_undegraded":"2023-10-08T01:12:02.860424+0000","last_fullsized":"2023-10-08T01:12:02.860424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:48.158503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"90'3","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856557+0000","last_change":"2023-10-08T01:12:01.825659+0000","last_active":"2023-10-08T01:12:02.856557+0000","last_peered":"2023-10-08T01:12:02.856557+0000","last_clean":"2023-10-08T01:12:02.856557+0000","last_became_active":"2023-10-08T01:12:01.825371+0000","last_became_peered":"2023-10-08T01:12:01.825371+0000","last_unstale":"2023-10-08T01:12:02.856557+0000","last_undegraded":"2023-10-08T01:12:02.856557+0000","last_fullsized":"2023-10-08T01:12:02.856557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:46.049146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843348+0000","last_change":"2023-10-08T01:12:02.837217+0000","last_active":"2023-10-08T01:12:02.843348+0000","last_peered":"2023-10-08T01:12:02.843348+0000","last_clean":"2023-10-08T01:12:02.843348+0000","last_became_active":"2023-10-08T01:12:02.836983+0000","last_became_peered":"2023-10-08T01:12:02.836983+0000","last_unstale":"2023-10-08T01:12:02.843348+0000","last_undegraded":"2023-10-08T01:12:02.843348+0000","last_fullsized":"2023-10-08T01:12:02.843348+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:31.604511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809400+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809631+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850403+0000","last_change":"2023-10-08T01:12:01.829191+0000","last_active":"2023-10-08T01:12:02.850403+0000","last_peered":"2023-10-08T01:12:02.850403+0000","last_clean":"2023-10-08T01:12:02.850403+0000","last_became_active":"2023-10-08T01:12:01.828873+0000","last_became_peered":"2023-10-08T01:12:01.828873+0000","last_unstale":"2023-10-08T01:12:02.850403+0000","last_undegraded":"2023-10-08T01:12:02.850403+0000","last_fullsized":"2023-10-08T01:12:02.850403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:31.442360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851325+0000","last_change":"2023-10-08T01:12:02.848989+0000","last_active":"2023-10-08T01:12:02.851325+0000","last_peered":"2023-10-08T01:12:02.851325+0000","last_clean":"2023-10-08T01:12:02.851325+0000","last_became_active":"2023-10-08T01:12:02.848775+0000","last_became_peered":"2023-10-08T01:12:02.848775+0000","last_unstale":"2023-10-08T01:12:02.851325+0000","last_undegraded":"2023-10-08T01:12:02.851325+0000","last_fullsized":"2023-10-08T01:12:02.851325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:30.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.808808+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:47.152989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843699+0000","last_change":"2023-10-08T01:12:02.837963+0000","last_active":"2023-10-08T01:12:02.843699+0000","last_peered":"2023-10-08T01:12:02.843699+0000","last_clean":"2023-10-08T01:12:02.843699+0000","last_became_active":"2023-10-08T01:12:02.837114+0000","last_became_peered":"2023-10-08T01:12:02.837114+0000","last_unstale":"2023-10-08T01:12:02.843699+0000","last_undegraded":"2023-10-08T01:12:02.843699+0000","last_fullsized":"2023-10-08T01:12:02.843699+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:42.075297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804185+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:38.855490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867962+0000","last_change":"2023-10-08T01:12:01.831319+0000","last_active":"2023-10-08T01:12:02.867962+0000","last_peered":"2023-10-08T01:12:02.867962+0000","last_clean":"2023-10-08T01:12:02.867962+0000","last_became_active":"2023-10-08T01:12:01.831046+0000","last_became_peered":"2023-10-08T01:12:01.831046+0000","last_unstale":"2023-10-08T01:12:02.867962+0000","last_undegraded":"2023-10-08T01:12:02.867962+0000","last_fullsized":"2023-10-08T01:12:02.867962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804397+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:54.295030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867409+0000","last_change":"2023-10-08T01:12:01.829452+0000","last_active":"2023-10-08T01:12:02.867409+0000","last_peered":"2023-10-08T01:12:02.867409+0000","last_clean":"2023-10-08T01:12:02.867409+0000","last_became_active":"2023-10-08T01:12:01.829140+0000","last_became_peered":"2023-10-08T01:12:01.829140+0000","last_unstale":"2023-10-08T01:12:02.867409+0000","last_undegraded":"2023-10-08T01:12:02.867409+0000","last_fullsized":"2023-10-08T01:12:02.867409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:51.755961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843809+0000","last_change":"2023-10-08T01:12:02.840709+0000","last_active":"2023-10-08T01:12:02.843809+0000","last_peered":"2023-10-08T01:12:02.843809+0000","last_clean":"2023-10-08T01:12:02.843809+0000","last_became_active":"2023-10-08T01:12:02.840250+0000","last_became_peered":"2023-10-08T01:12:02.840250+0000","last_unstale":"2023-10-08T01:12:02.843809+0000","last_undegraded":"2023-10-08T01:12:02.843809+0000","last_fullsized":"2023-10-08T01:12:02.843809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:26.311091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807321+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.838339+0000","last_change":"2023-10-08T01:12:02.836913+0000","last_active":"2023-10-08T01:12:02.838339+0000","last_peered":"2023-10-08T01:12:02.838339+0000","last_clean":"2023-10-08T01:12:02.838339+0000","last_became_active":"2023-10-08T01:12:02.836618+0000","last_became_peered":"2023-10-08T01:12:02.836618+0000","last_unstale":"2023-10-08T01:12:02.838339+0000","last_undegraded":"2023-10-08T01:12:02.838339+0000","last_fullsized":"2023-10-08T01:12:02.838339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:05.095868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850521+0000","last_change":"2023-10-08T01:12:01.831714+0000","last_active":"2023-10-08T01:12:02.850521+0000","last_peered":"2023-10-08T01:12:02.850521+0000","last_clean":"2023-10-08T01:12:02.850521+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850521+0000","last_undegraded":"2023-10-08T01:12:02.850521+0000","last_fullsized":"2023-10-08T01:12:02.850521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:56.535520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.862945+0000","last_change":"2023-10-08T01:12:02.862541+0000","last_active":"2023-10-08T01:12:02.862945+0000","last_peered":"2023-10-08T01:12:02.862945+0000","last_clean":"2023-10-08T01:12:02.862945+0000","last_became_active":"2023-10-08T01:12:02.862076+0000","last_became_peered":"2023-10-08T01:12:02.862076+0000","last_unstale":"2023-10-08T01:12:02.862945+0000","last_undegraded":"2023-10-08T01:12:02.862945+0000","last_fullsized":"2023-10-08T01:12:02.862945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:26.919061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807911+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:46.971056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852339+0000","last_change":"2023-10-08T01:12:02.849161+0000","last_active":"2023-10-08T01:12:02.852339+0000","last_peered":"2023-10-08T01:12:02.852339+0000","last_clean":"2023-10-08T01:12:02.852339+0000","last_became_active":"2023-10-08T01:12:02.848913+0000","last_became_peered":"2023-10-08T01:12:02.848913+0000","last_unstale":"2023-10-08T01:12:02.852339+0000","last_undegraded":"2023-10-08T01:12:02.852339+0000","last_fullsized":"2023-10-08T01:12:02.852339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:58:40.796959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.832971+0000","last_change":"2023-10-08T01:12:01.819602+0000","last_active":"2023-10-08T01:12:02.832971+0000","last_peered":"2023-10-08T01:12:02.832971+0000","last_clean":"2023-10-08T01:12:02.832971+0000","last_became_active":"2023-10-08T01:12:01.819304+0000","last_became_peered":"2023-10-08T01:12:01.819304+0000","last_unstale":"2023-10-08T01:12:02.832971+0000","last_undegraded":"2023-10-08T01:12:02.832971+0000","last_fullsized":"2023-10-08T01:12:02.832971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:55.795619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846380+0000","last_change":"2023-10-08T01:12:02.844295+0000","last_active":"2023-10-08T01:12:02.846380+0000","last_peered":"2023-10-08T01:12:02.846380+0000","last_clean":"2023-10-08T01:12:02.846380+0000","last_became_active":"2023-10-08T01:12:02.844001+0000","last_became_peered":"2023-10-08T01:12:02.844001+0000","last_unstale":"2023-10-08T01:12:02.846380+0000","last_undegraded":"2023-10-08T01:12:02.846380+0000","last_fullsized":"2023-10-08T01:12:02.846380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:39:36.571615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867968+0000","last_change":"2023-10-08T01:12:01.831354+0000","last_active":"2023-10-08T01:12:02.867968+0000","last_peered":"2023-10-08T01:12:02.867968+0000","last_clean":"2023-10-08T01:12:02.867968+0000","last_became_active":"2023-10-08T01:12:01.830964+0000","last_became_peered":"2023-10-08T01:12:01.830964+0000","last_unstale":"2023-10-08T01:12:02.867968+0000","last_undegraded":"2023-10-08T01:12:02.867968+0000","last_fullsized":"2023-10-08T01:12:02.867968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809329+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:08.110852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828282+0000","last_change":"2023-10-08T01:12:01.823222+0000","last_active":"2023-10-08T01:12:02.828282+0000","last_peered":"2023-10-08T01:12:02.828282+0000","last_clean":"2023-10-08T01:12:02.828282+0000","last_became_active":"2023-10-08T01:12:01.822970+0000","last_became_peered":"2023-10-08T01:12:01.822970+0000","last_unstale":"2023-10-08T01:12:02.828282+0000","last_undegraded":"2023-10-08T01:12:02.828282+0000","last_fullsized":"2023-10-08T01:12:02.828282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:00.437245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834238+0000","last_change":"2023-10-08T01:12:01.815771+0000","last_active":"2023-10-08T01:12:02.834238+0000","last_peered":"2023-10-08T01:12:02.834238+0000","last_clean":"2023-10-08T01:12:02.834238+0000","last_became_active":"2023-10-08T01:12:01.815502+0000","last_became_peered":"2023-10-08T01:12:01.815502+0000","last_unstale":"2023-10-08T01:12:02.834238+0000","last_undegraded":"2023-10-08T01:12:02.834238+0000","last_fullsized":"2023-10-08T01:12:02.834238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843426+0000","last_change":"2023-10-08T01:12:02.838180+0000","last_active":"2023-10-08T01:12:02.843426+0000","last_peered":"2023-10-08T01:12:02.843426+0000","last_clean":"2023-10-08T01:12:02.843426+0000","last_became_active":"2023-10-08T01:12:02.837945+0000","last_became_peered":"2023-10-08T01:12:02.837945+0000","last_unstale":"2023-10-08T01:12:02.843426+0000","last_undegraded":"2023-10-08T01:12:02.843426+0000","last_fullsized":"2023-10-08T01:12:02.843426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:21.100043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.827017+0000","last_change":"2023-10-08T01:12:01.827250+0000","last_active":"2023-10-08T01:12:02.827017+0000","last_peered":"2023-10-08T01:12:02.827017+0000","last_clean":"2023-10-08T01:12:02.827017+0000","last_became_active":"2023-10-08T01:12:01.826935+0000","last_became_peered":"2023-10-08T01:12:01.826935+0000","last_unstale":"2023-10-08T01:12:02.827017+0000","last_undegraded":"2023-10-08T01:12:02.827017+0000","last_fullsized":"2023-10-08T01:12:02.827017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:40.815318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860494+0000","last_change":"2023-10-08T01:12:02.856747+0000","last_active":"2023-10-08T01:12:02.860494+0000","last_peered":"2023-10-08T01:12:02.860494+0000","last_clean":"2023-10-08T01:12:02.860494+0000","last_became_active":"2023-10-08T01:12:02.856476+0000","last_became_peered":"2023-10-08T01:12:02.856476+0000","last_unstale":"2023-10-08T01:12:02.860494+0000","last_undegraded":"2023-10-08T01:12:02.860494+0000","last_fullsized":"2023-10-08T01:12:02.860494+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:00.451849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809989+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804185+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:50.137372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859915+0000","last_change":"2023-10-08T01:12:02.855156+0000","last_active":"2023-10-08T01:12:02.859915+0000","last_peered":"2023-10-08T01:12:02.859915+0000","last_clean":"2023-10-08T01:12:02.859915+0000","last_became_active":"2023-10-08T01:12:02.854809+0000","last_became_peered":"2023-10-08T01:12:02.854809+0000","last_unstale":"2023-10-08T01:12:02.859915+0000","last_undegraded":"2023-10-08T01:12:02.859915+0000","last_fullsized":"2023-10-08T01:12:02.859915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:46.539103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850388+0000","last_change":"2023-10-08T01:12:01.831667+0000","last_active":"2023-10-08T01:12:02.850388+0000","last_peered":"2023-10-08T01:12:02.850388+0000","last_clean":"2023-10-08T01:12:02.850388+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850388+0000","last_undegraded":"2023-10-08T01:12:02.850388+0000","last_fullsized":"2023-10-08T01:12:02.850388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:23.287500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852439+0000","last_change":"2023-10-08T01:12:02.849116+0000","last_active":"2023-10-08T01:12:02.852439+0000","last_peered":"2023-10-08T01:12:02.852439+0000","last_clean":"2023-10-08T01:12:02.852439+0000","last_became_active":"2023-10-08T01:12:02.848833+0000","last_became_peered":"2023-10-08T01:12:02.848833+0000","last_unstale":"2023-10-08T01:12:02.852439+0000","last_undegraded":"2023-10-08T01:12:02.852439+0000","last_fullsized":"2023-10-08T01:12:02.852439+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:35.798359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.870650+0000","last_change":"2023-10-08T01:12:01.828087+0000","last_active":"2023-10-08T01:12:02.870650+0000","last_peered":"2023-10-08T01:12:02.870650+0000","last_clean":"2023-10-08T01:12:02.870650+0000","last_became_active":"2023-10-08T01:12:01.827688+0000","last_became_peered":"2023-10-08T01:12:01.827688+0000","last_unstale":"2023-10-08T01:12:02.870650+0000","last_undegraded":"2023-10-08T01:12:02.870650+0000","last_fullsized":"2023-10-08T01:12:02.870650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:25.305577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"80'183","reported_seq":457,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T01:11:58.219508+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:11:58.219508+0000","last_peered":"2023-10-08T01:11:58.219508+0000","last_clean":"2023-10-08T01:11:58.219508+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:11:58.219508+0000","last_undegraded":"2023-10-08T01:11:58.219508+0000","last_fullsized":"2023-10-08T01:11:58.219508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":4},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":182,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1667072,"data_stored":1106358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.58599999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.72599999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.623}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1118208,"data_stored":536572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.497},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.74199999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.83699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.28599999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.495},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.85099999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":1.026},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.92800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303048,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068792,"statfs":{"total":96636764160,"available":96326443008,"internally_reserved":0,"allocated":1703936,"data_stored":1126848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50800000000000001,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.617},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.5289999999999999},"last":0.754}]},{"osd":6,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.208,"5min":1.361,"15min":1.3779999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1719999999999999,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.65000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306576,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065264,"statfs":{"total":96636764160,"available":96322830336,"internally_reserved":0,"allocated":1122304,"data_stored":538016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306572,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065268,"statfs":{"total":96636764160,"available":96322834432,"internally_reserved":0,"allocated":1118208,"data_stored":536592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.69999999999999996,"15min":0.80100000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.57399999999999995,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.378}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61699999999999999,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.82999999999999996,"15min":0.88100000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.79400000000000004,"15min":1.1060000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":1.095,"15min":1.3029999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74299999999999999,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.117,"15min":1.413},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.77800000000000002},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.0920000000000001,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":1.089},"last":0.61899999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.379},"last":0.751},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.486},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97099999999999997,"5min":1.0840000000000001,"15min":1.389},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.79700000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.5},"max":{"1min":0.93700000000000006,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.65500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310624,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061216,"statfs":{"total":96636764160,"available":96318685184,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.72199999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.75}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.80900000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.66700000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.88300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.66100000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.622}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.749},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.76300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:12:07.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:12:07.798 INFO:tasks.ceph.mds.b.smithi184.stderr:2023-10-08T01:12:07.797+0000 7fe29f5f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:12:07.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.827+0000 7f79eb7ff700 1 Processor -- start 2023-10-08T01:12:07.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.827+0000 7f79eb7ff700 1 -- start start 2023-10-08T01:12:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.828+0000 7f79eb7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79e4159890 0x7f79e415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79eb7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159c60 0x7f79e415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79eb7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e415a160 con 0x7f79e415db10 2023-10-08T01:12:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79eb7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e40d13a0 con 0x7f79e4159890 2023-10-08T01:12:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79eb7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415e140 con 0x7f79e4159c60 2023-10-08T01:12:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79e959b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79e4159890 0x7f79e415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79e9d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f79e415db10 0x7f79e4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52234/0 (socket says 172.21.15.119:52234) 2023-10-08T01:12:07.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79e9d9c700 1 -- 172.21.15.119:0/3939843084 learned_addr learned my addr 172.21.15.119:0/3939843084 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:07.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79e8d9a700 1 --2- 172.21.15.119:0/3939843084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159c60 0x7f79e415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79e959b700 1 -- 172.21.15.119:0/3939843084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159c60 msgr2=0x7f79e415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:07.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79e959b700 1 --2- 172.21.15.119:0/3939843084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159c60 0x7f79e415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79e959b700 1 -- 172.21.15.119:0/3939843084 >> v1:172.21.15.119:6789/0 conn(0x7f79e415db10 legacy=0x7f79e4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79e959b700 1 -- 172.21.15.119:0/3939843084 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e415a3d0 con 0x7f79e4159890 2023-10-08T01:12:07.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.829+0000 7f79dbfff700 1 -- 172.21.15.119:0/3939843084 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474803207 0 0) 0x7f79e415a160 con 0x7f79e415db10 2023-10-08T01:12:07.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79e959b700 1 --2- 172.21.15.119:0/3939843084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159890 0x7f79e415abd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f79e00098e0 tx=0x7f79e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1456601014204b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:07.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79dbfff700 1 -- 172.21.15.119:0/3939843084 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000b640 con 0x7f79e4159890 2023-10-08T01:12:07.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79dbfff700 1 -- 172.21.15.119:0/3939843084 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000b7a0 con 0x7f79e4159890 2023-10-08T01:12:07.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79dbfff700 1 -- 172.21.15.119:0/3939843084 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000bb40 con 0x7f79e4159890 2023-10-08T01:12:07.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79eb7ff700 1 -- 172.21.15.119:0/3939843084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159890 msgr2=0x7f79e415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/3939843084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159890 0x7f79e415abd0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f79e00098e0 tx=0x7f79e00091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79eb7ff700 1 -- 172.21.15.119:0/3939843084 shutdown_connections 2023-10-08T01:12:07.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/3939843084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159890 0x7f79e415abd0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/3939843084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159c60 0x7f79e415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79eb7ff700 1 -- 172.21.15.119:0/3939843084 >> 172.21.15.119:0/3939843084 conn(0x7f79e4068e20 msgr2=0x7f79e40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:07.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79eb7ff700 1 -- 172.21.15.119:0/3939843084 shutdown_connections 2023-10-08T01:12:07.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.830+0000 7f79eb7ff700 1 -- 172.21.15.119:0/3939843084 wait complete. 2023-10-08T01:12:07.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.832+0000 7f79eb7ff700 1 Processor -- start 2023-10-08T01:12:07.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.832+0000 7f79eb7ff700 1 -- start start 2023-10-08T01:12:07.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79eb7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159890 0x7f79e415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79eb7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159c60 0x7f79e40ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79eb7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79e415db10 0x7f79e414d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79eb7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415c850 con 0x7f79e4159c60 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79eb7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415bc40 con 0x7f79e415db10 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79eb7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415bf20 con 0x7f79e4159890 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79e8d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159c60 0x7f79e40ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79e8d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159c60 0x7f79e40ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55176/0 (socket says 172.21.15.119:55176) 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79e8d9a700 1 -- 172.21.15.119:0/2366138826 learned_addr learned my addr 172.21.15.119:0/2366138826 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79e959b700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159890 0x7f79e415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79e9d9c700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79e415db10 0x7f79e414d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79e8d9a700 1 -- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159890 msgr2=0x7f79e415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.833+0000 7f79e8d9a700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159890 0x7f79e415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79e8d9a700 1 -- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79e415db10 msgr2=0x7f79e414d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79e8d9a700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79e415db10 0x7f79e414d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79e8d9a700 1 -- 172.21.15.119:0/2366138826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0009420 con 0x7f79e4159c60 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79e9d9c700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79e415db10 0x7f79e414d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79e8d9a700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159c60 0x7f79e40ce010 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f79d400ec50 tx=0x7f79d40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79d97fa700 1 -- 172.21.15.119:0/2366138826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400c850 con 0x7f79e4159c60 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79d97fa700 1 -- 172.21.15.119:0/2366138826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d400c9b0 con 0x7f79e4159c60 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79d97fa700 1 -- 172.21.15.119:0/2366138826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d400cd70 con 0x7f79e4159c60 2023-10-08T01:12:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e415c200 con 0x7f79e4159c60 2023-10-08T01:12:07.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.834+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e40ce550 con 0x7f79e4159c60 2023-10-08T01:12:07.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.835+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e404c3c0 con 0x7f79e4159c60 2023-10-08T01:12:07.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.836+0000 7f79d97fa700 1 -- 172.21.15.119:0/2366138826 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f79d4013680 con 0x7f79e4159c60 2023-10-08T01:12:07.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.837+0000 7f79d97fa700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f79c8066f30 0x7f79c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:07.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.837+0000 7f79d97fa700 1 -- 172.21.15.119:0/2366138826 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f79d4084a30 con 0x7f79e4159c60 2023-10-08T01:12:07.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.837+0000 7f79e959b700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f79c8066f30 0x7f79c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:07.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.838+0000 7f79e959b700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f79c8066f30 0x7f79c80693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79e00093f0 tx=0x7f79e0017000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:07.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:07.842+0000 7f79d97fa700 1 -- 172.21.15.119:0/2366138826 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79d4051a70 con 0x7f79e4159c60 2023-10-08T01:12:08.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.032+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f79e415c090 con 0x7f79c8066f30 2023-10-08T01:12:08.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.040+0000 7f79d97fa700 1 -- 172.21.15.119:0/2366138826 <== mgr.6516 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284039 (secure 0 0 0) 0x7f79e415c090 con 0x7f79c8066f30 2023-10-08T01:12:08.046 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f79c8066f30 msgr2=0x7f79c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f79c8066f30 0x7f79c80693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f79e00093f0 tx=0x7f79e0017000 comp rx=0 tx=0).stop 2023-10-08T01:12:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159c60 msgr2=0x7f79e40ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159c60 0x7f79e40ce010 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f79d400ec50 tx=0x7f79d40091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 shutdown_connections 2023-10-08T01:12:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f79c8066f30 0x7f79c80693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f79e4159890 0x7f79e415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f79e415db10 0x7f79e414d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 --2- 172.21.15.119:0/2366138826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79e4159c60 0x7f79e40ce010 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 >> 172.21.15.119:0/2366138826 conn(0x7f79e4068e20 msgr2=0x7f79e40b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:08.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 shutdown_connections 2023-10-08T01:12:08.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.045+0000 7f79eb7ff700 1 -- 172.21.15.119:0/2366138826 wait complete. 2023-10-08T01:12:08.050 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:12:08.064 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:12:06.707159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590550,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":292,"num_write_kb":2419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":197,"ondisk_log_size":197,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2437172,"kb_used_data":9716,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752537548,"statfs":{"total":773094113280,"available":770598449152,"internally_reserved":0,"allocated":9949184,"data_stored":5392356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":182,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.906649"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866793+0000","last_change":"2023-10-08T01:12:02.861488+0000","last_active":"2023-10-08T01:12:02.866793+0000","last_peered":"2023-10-08T01:12:02.866793+0000","last_clean":"2023-10-08T01:12:02.866793+0000","last_became_active":"2023-10-08T01:12:02.860018+0000","last_became_peered":"2023-10-08T01:12:02.860018+0000","last_unstale":"2023-10-08T01:12:02.866793+0000","last_undegraded":"2023-10-08T01:12:02.866793+0000","last_fullsized":"2023-10-08T01:12:02.866793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.163695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846099+0000","last_change":"2023-10-08T01:12:02.841414+0000","last_active":"2023-10-08T01:12:02.846099+0000","last_peered":"2023-10-08T01:12:02.846099+0000","last_clean":"2023-10-08T01:12:02.846099+0000","last_became_active":"2023-10-08T01:12:02.841091+0000","last_became_peered":"2023-10-08T01:12:02.841091+0000","last_unstale":"2023-10-08T01:12:02.846099+0000","last_undegraded":"2023-10-08T01:12:02.846099+0000","last_fullsized":"2023-10-08T01:12:02.846099+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:03.550299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845916+0000","last_change":"2023-10-08T01:12:02.841353+0000","last_active":"2023-10-08T01:12:02.845916+0000","last_peered":"2023-10-08T01:12:02.845916+0000","last_clean":"2023-10-08T01:12:02.845916+0000","last_became_active":"2023-10-08T01:12:02.841117+0000","last_became_peered":"2023-10-08T01:12:02.841117+0000","last_unstale":"2023-10-08T01:12:02.845916+0000","last_undegraded":"2023-10-08T01:12:02.845916+0000","last_fullsized":"2023-10-08T01:12:02.845916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:32.447875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859766+0000","last_change":"2023-10-08T01:12:02.851025+0000","last_active":"2023-10-08T01:12:02.859766+0000","last_peered":"2023-10-08T01:12:02.859766+0000","last_clean":"2023-10-08T01:12:02.859766+0000","last_became_active":"2023-10-08T01:12:02.850765+0000","last_became_peered":"2023-10-08T01:12:02.850765+0000","last_unstale":"2023-10-08T01:12:02.859766+0000","last_undegraded":"2023-10-08T01:12:02.859766+0000","last_fullsized":"2023-10-08T01:12:02.859766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:09.116366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860705+0000","last_change":"2023-10-08T01:12:02.845674+0000","last_active":"2023-10-08T01:12:02.860705+0000","last_peered":"2023-10-08T01:12:02.860705+0000","last_clean":"2023-10-08T01:12:02.860705+0000","last_became_active":"2023-10-08T01:12:02.845358+0000","last_became_peered":"2023-10-08T01:12:02.845358+0000","last_unstale":"2023-10-08T01:12:02.860705+0000","last_undegraded":"2023-10-08T01:12:02.860705+0000","last_fullsized":"2023-10-08T01:12:02.860705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:47.976571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844181+0000","last_change":"2023-10-08T01:12:02.839332+0000","last_active":"2023-10-08T01:12:02.844181+0000","last_peered":"2023-10-08T01:12:02.844181+0000","last_clean":"2023-10-08T01:12:02.844181+0000","last_became_active":"2023-10-08T01:12:02.838641+0000","last_became_peered":"2023-10-08T01:12:02.838641+0000","last_unstale":"2023-10-08T01:12:02.844181+0000","last_undegraded":"2023-10-08T01:12:02.844181+0000","last_fullsized":"2023-10-08T01:12:02.844181+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:24.293015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851974+0000","last_change":"2023-10-08T01:12:02.849866+0000","last_active":"2023-10-08T01:12:02.851974+0000","last_peered":"2023-10-08T01:12:02.851974+0000","last_clean":"2023-10-08T01:12:02.851974+0000","last_became_active":"2023-10-08T01:12:02.849568+0000","last_became_peered":"2023-10-08T01:12:02.849568+0000","last_unstale":"2023-10-08T01:12:02.851974+0000","last_undegraded":"2023-10-08T01:12:02.851974+0000","last_fullsized":"2023-10-08T01:12:02.851974+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:30.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846415+0000","last_change":"2023-10-08T01:12:02.843295+0000","last_active":"2023-10-08T01:12:02.846415+0000","last_peered":"2023-10-08T01:12:02.846415+0000","last_clean":"2023-10-08T01:12:02.846415+0000","last_became_active":"2023-10-08T01:12:02.842892+0000","last_became_peered":"2023-10-08T01:12:02.842892+0000","last_unstale":"2023-10-08T01:12:02.846415+0000","last_undegraded":"2023-10-08T01:12:02.846415+0000","last_fullsized":"2023-10-08T01:12:02.846415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:20.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851263+0000","last_change":"2023-10-08T01:12:02.847055+0000","last_active":"2023-10-08T01:12:02.851263+0000","last_peered":"2023-10-08T01:12:02.851263+0000","last_clean":"2023-10-08T01:12:02.851263+0000","last_became_active":"2023-10-08T01:12:02.846767+0000","last_became_peered":"2023-10-08T01:12:02.846767+0000","last_unstale":"2023-10-08T01:12:02.851263+0000","last_undegraded":"2023-10-08T01:12:02.851263+0000","last_fullsized":"2023-10-08T01:12:02.851263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:52.932416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844213+0000","last_change":"2023-10-08T01:12:02.840294+0000","last_active":"2023-10-08T01:12:02.844213+0000","last_peered":"2023-10-08T01:12:02.844213+0000","last_clean":"2023-10-08T01:12:02.844213+0000","last_became_active":"2023-10-08T01:12:02.838656+0000","last_became_peered":"2023-10-08T01:12:02.838656+0000","last_unstale":"2023-10-08T01:12:02.844213+0000","last_undegraded":"2023-10-08T01:12:02.844213+0000","last_fullsized":"2023-10-08T01:12:02.844213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:57.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866715+0000","last_change":"2023-10-08T01:12:02.860578+0000","last_active":"2023-10-08T01:12:02.866715+0000","last_peered":"2023-10-08T01:12:02.866715+0000","last_clean":"2023-10-08T01:12:02.866715+0000","last_became_active":"2023-10-08T01:12:02.859906+0000","last_became_peered":"2023-10-08T01:12:02.859906+0000","last_unstale":"2023-10-08T01:12:02.866715+0000","last_undegraded":"2023-10-08T01:12:02.866715+0000","last_fullsized":"2023-10-08T01:12:02.866715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:48.742822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853436+0000","last_change":"2023-10-08T01:12:02.851814+0000","last_active":"2023-10-08T01:12:02.853436+0000","last_peered":"2023-10-08T01:12:02.853436+0000","last_clean":"2023-10-08T01:12:02.853436+0000","last_became_active":"2023-10-08T01:12:02.851488+0000","last_became_peered":"2023-10-08T01:12:02.851488+0000","last_unstale":"2023-10-08T01:12:02.853436+0000","last_undegraded":"2023-10-08T01:12:02.853436+0000","last_fullsized":"2023-10-08T01:12:02.853436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:37.330422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846116+0000","last_change":"2023-10-08T01:12:02.845033+0000","last_active":"2023-10-08T01:12:02.846116+0000","last_peered":"2023-10-08T01:12:02.846116+0000","last_clean":"2023-10-08T01:12:02.846116+0000","last_became_active":"2023-10-08T01:12:02.844545+0000","last_became_peered":"2023-10-08T01:12:02.844545+0000","last_unstale":"2023-10-08T01:12:02.846116+0000","last_undegraded":"2023-10-08T01:12:02.846116+0000","last_fullsized":"2023-10-08T01:12:02.846116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:36.576301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845074+0000","last_change":"2023-10-08T01:12:02.839789+0000","last_active":"2023-10-08T01:12:02.845074+0000","last_peered":"2023-10-08T01:12:02.845074+0000","last_clean":"2023-10-08T01:12:02.845074+0000","last_became_active":"2023-10-08T01:12:02.839516+0000","last_became_peered":"2023-10-08T01:12:02.839516+0000","last_unstale":"2023-10-08T01:12:02.845074+0000","last_undegraded":"2023-10-08T01:12:02.845074+0000","last_fullsized":"2023-10-08T01:12:02.845074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:42.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860840+0000","last_change":"2023-10-08T01:12:02.854035+0000","last_active":"2023-10-08T01:12:02.860840+0000","last_peered":"2023-10-08T01:12:02.860840+0000","last_clean":"2023-10-08T01:12:02.860840+0000","last_became_active":"2023-10-08T01:12:02.853704+0000","last_became_peered":"2023-10-08T01:12:02.853704+0000","last_unstale":"2023-10-08T01:12:02.860840+0000","last_undegraded":"2023-10-08T01:12:02.860840+0000","last_fullsized":"2023-10-08T01:12:02.860840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:55.363740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807321+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844198+0000","last_change":"2023-10-08T01:12:02.839586+0000","last_active":"2023-10-08T01:12:02.844198+0000","last_peered":"2023-10-08T01:12:02.844198+0000","last_clean":"2023-10-08T01:12:02.844198+0000","last_became_active":"2023-10-08T01:12:02.839033+0000","last_became_peered":"2023-10-08T01:12:02.839033+0000","last_unstale":"2023-10-08T01:12:02.844198+0000","last_undegraded":"2023-10-08T01:12:02.844198+0000","last_fullsized":"2023-10-08T01:12:02.844198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:43.348391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856349+0000","last_change":"2023-10-08T01:12:01.818263+0000","last_active":"2023-10-08T01:12:02.856349+0000","last_peered":"2023-10-08T01:12:02.856349+0000","last_clean":"2023-10-08T01:12:02.856349+0000","last_became_active":"2023-10-08T01:12:01.818011+0000","last_became_peered":"2023-10-08T01:12:01.818011+0000","last_unstale":"2023-10-08T01:12:02.856349+0000","last_undegraded":"2023-10-08T01:12:02.856349+0000","last_fullsized":"2023-10-08T01:12:02.856349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860732+0000","last_change":"2023-10-08T01:12:02.857609+0000","last_active":"2023-10-08T01:12:02.860732+0000","last_peered":"2023-10-08T01:12:02.860732+0000","last_clean":"2023-10-08T01:12:02.860732+0000","last_became_active":"2023-10-08T01:12:02.857277+0000","last_became_peered":"2023-10-08T01:12:02.857277+0000","last_unstale":"2023-10-08T01:12:02.860732+0000","last_undegraded":"2023-10-08T01:12:02.860732+0000","last_fullsized":"2023-10-08T01:12:02.860732+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.729095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804484+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:03.061391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.868186+0000","last_change":"2023-10-08T01:12:01.831258+0000","last_active":"2023-10-08T01:12:02.868186+0000","last_peered":"2023-10-08T01:12:02.868186+0000","last_clean":"2023-10-08T01:12:02.868186+0000","last_became_active":"2023-10-08T01:12:01.830943+0000","last_became_peered":"2023-10-08T01:12:01.830943+0000","last_unstale":"2023-10-08T01:12:02.868186+0000","last_undegraded":"2023-10-08T01:12:02.868186+0000","last_fullsized":"2023-10-08T01:12:02.868186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853340+0000","last_change":"2023-10-08T01:12:01.826432+0000","last_active":"2023-10-08T01:12:02.853340+0000","last_peered":"2023-10-08T01:12:02.853340+0000","last_clean":"2023-10-08T01:12:02.853340+0000","last_became_active":"2023-10-08T01:12:01.826156+0000","last_became_peered":"2023-10-08T01:12:01.826156+0000","last_unstale":"2023-10-08T01:12:02.853340+0000","last_undegraded":"2023-10-08T01:12:02.853340+0000","last_fullsized":"2023-10-08T01:12:02.853340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:26.787866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867313+0000","last_change":"2023-10-08T01:12:01.833334+0000","last_active":"2023-10-08T01:12:02.867313+0000","last_peered":"2023-10-08T01:12:02.867313+0000","last_clean":"2023-10-08T01:12:02.867313+0000","last_became_active":"2023-10-08T01:12:01.832064+0000","last_became_peered":"2023-10-08T01:12:01.832064+0000","last_unstale":"2023-10-08T01:12:02.867313+0000","last_undegraded":"2023-10-08T01:12:02.867313+0000","last_fullsized":"2023-10-08T01:12:02.867313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:49.406724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846186+0000","last_change":"2023-10-08T01:12:02.843587+0000","last_active":"2023-10-08T01:12:02.846186+0000","last_peered":"2023-10-08T01:12:02.846186+0000","last_clean":"2023-10-08T01:12:02.846186+0000","last_became_active":"2023-10-08T01:12:02.843202+0000","last_became_peered":"2023-10-08T01:12:02.843202+0000","last_unstale":"2023-10-08T01:12:02.846186+0000","last_undegraded":"2023-10-08T01:12:02.846186+0000","last_fullsized":"2023-10-08T01:12:02.846186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:50.412239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809293+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:23:16.271266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851277+0000","last_change":"2023-10-08T01:12:02.847761+0000","last_active":"2023-10-08T01:12:02.851277+0000","last_peered":"2023-10-08T01:12:02.851277+0000","last_clean":"2023-10-08T01:12:02.851277+0000","last_became_active":"2023-10-08T01:12:02.847425+0000","last_became_peered":"2023-10-08T01:12:02.847425+0000","last_unstale":"2023-10-08T01:12:02.851277+0000","last_undegraded":"2023-10-08T01:12:02.851277+0000","last_fullsized":"2023-10-08T01:12:02.851277+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:12:47.987269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.826270+0000","last_change":"2023-10-08T01:12:01.822270+0000","last_active":"2023-10-08T01:12:02.826270+0000","last_peered":"2023-10-08T01:12:02.826270+0000","last_clean":"2023-10-08T01:12:02.826270+0000","last_became_active":"2023-10-08T01:12:01.821995+0000","last_became_peered":"2023-10-08T01:12:01.821995+0000","last_unstale":"2023-10-08T01:12:02.826270+0000","last_undegraded":"2023-10-08T01:12:02.826270+0000","last_fullsized":"2023-10-08T01:12:02.826270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:40.166452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804453+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:38.169246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852304+0000","last_change":"2023-10-08T01:12:01.832041+0000","last_active":"2023-10-08T01:12:02.852304+0000","last_peered":"2023-10-08T01:12:02.852304+0000","last_clean":"2023-10-08T01:12:02.852304+0000","last_became_active":"2023-10-08T01:12:01.831603+0000","last_became_peered":"2023-10-08T01:12:01.831603+0000","last_unstale":"2023-10-08T01:12:02.852304+0000","last_undegraded":"2023-10-08T01:12:02.852304+0000","last_fullsized":"2023-10-08T01:12:02.852304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:42.342876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860424+0000","last_change":"2023-10-08T01:12:02.855912+0000","last_active":"2023-10-08T01:12:02.860424+0000","last_peered":"2023-10-08T01:12:02.860424+0000","last_clean":"2023-10-08T01:12:02.860424+0000","last_became_active":"2023-10-08T01:12:02.855628+0000","last_became_peered":"2023-10-08T01:12:02.855628+0000","last_unstale":"2023-10-08T01:12:02.860424+0000","last_undegraded":"2023-10-08T01:12:02.860424+0000","last_fullsized":"2023-10-08T01:12:02.860424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:48.158503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"90'3","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856557+0000","last_change":"2023-10-08T01:12:01.825659+0000","last_active":"2023-10-08T01:12:02.856557+0000","last_peered":"2023-10-08T01:12:02.856557+0000","last_clean":"2023-10-08T01:12:02.856557+0000","last_became_active":"2023-10-08T01:12:01.825371+0000","last_became_peered":"2023-10-08T01:12:01.825371+0000","last_unstale":"2023-10-08T01:12:02.856557+0000","last_undegraded":"2023-10-08T01:12:02.856557+0000","last_fullsized":"2023-10-08T01:12:02.856557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:46.049146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843348+0000","last_change":"2023-10-08T01:12:02.837217+0000","last_active":"2023-10-08T01:12:02.843348+0000","last_peered":"2023-10-08T01:12:02.843348+0000","last_clean":"2023-10-08T01:12:02.843348+0000","last_became_active":"2023-10-08T01:12:02.836983+0000","last_became_peered":"2023-10-08T01:12:02.836983+0000","last_unstale":"2023-10-08T01:12:02.843348+0000","last_undegraded":"2023-10-08T01:12:02.843348+0000","last_fullsized":"2023-10-08T01:12:02.843348+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:31.604511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809400+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809631+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850403+0000","last_change":"2023-10-08T01:12:01.829191+0000","last_active":"2023-10-08T01:12:02.850403+0000","last_peered":"2023-10-08T01:12:02.850403+0000","last_clean":"2023-10-08T01:12:02.850403+0000","last_became_active":"2023-10-08T01:12:01.828873+0000","last_became_peered":"2023-10-08T01:12:01.828873+0000","last_unstale":"2023-10-08T01:12:02.850403+0000","last_undegraded":"2023-10-08T01:12:02.850403+0000","last_fullsized":"2023-10-08T01:12:02.850403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:31.442360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851325+0000","last_change":"2023-10-08T01:12:02.848989+0000","last_active":"2023-10-08T01:12:02.851325+0000","last_peered":"2023-10-08T01:12:02.851325+0000","last_clean":"2023-10-08T01:12:02.851325+0000","last_became_active":"2023-10-08T01:12:02.848775+0000","last_became_peered":"2023-10-08T01:12:02.848775+0000","last_unstale":"2023-10-08T01:12:02.851325+0000","last_undegraded":"2023-10-08T01:12:02.851325+0000","last_fullsized":"2023-10-08T01:12:02.851325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:30.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.808808+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:47.152989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843699+0000","last_change":"2023-10-08T01:12:02.837963+0000","last_active":"2023-10-08T01:12:02.843699+0000","last_peered":"2023-10-08T01:12:02.843699+0000","last_clean":"2023-10-08T01:12:02.843699+0000","last_became_active":"2023-10-08T01:12:02.837114+0000","last_became_peered":"2023-10-08T01:12:02.837114+0000","last_unstale":"2023-10-08T01:12:02.843699+0000","last_undegraded":"2023-10-08T01:12:02.843699+0000","last_fullsized":"2023-10-08T01:12:02.843699+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:42.075297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804185+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:01:38.855490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867962+0000","last_change":"2023-10-08T01:12:01.831319+0000","last_active":"2023-10-08T01:12:02.867962+0000","last_peered":"2023-10-08T01:12:02.867962+0000","last_clean":"2023-10-08T01:12:02.867962+0000","last_became_active":"2023-10-08T01:12:01.831046+0000","last_became_peered":"2023-10-08T01:12:01.831046+0000","last_unstale":"2023-10-08T01:12:02.867962+0000","last_undegraded":"2023-10-08T01:12:02.867962+0000","last_fullsized":"2023-10-08T01:12:02.867962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804397+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:54.295030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867409+0000","last_change":"2023-10-08T01:12:01.829452+0000","last_active":"2023-10-08T01:12:02.867409+0000","last_peered":"2023-10-08T01:12:02.867409+0000","last_clean":"2023-10-08T01:12:02.867409+0000","last_became_active":"2023-10-08T01:12:01.829140+0000","last_became_peered":"2023-10-08T01:12:01.829140+0000","last_unstale":"2023-10-08T01:12:02.867409+0000","last_undegraded":"2023-10-08T01:12:02.867409+0000","last_fullsized":"2023-10-08T01:12:02.867409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:51.755961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843809+0000","last_change":"2023-10-08T01:12:02.840709+0000","last_active":"2023-10-08T01:12:02.843809+0000","last_peered":"2023-10-08T01:12:02.843809+0000","last_clean":"2023-10-08T01:12:02.843809+0000","last_became_active":"2023-10-08T01:12:02.840250+0000","last_became_peered":"2023-10-08T01:12:02.840250+0000","last_unstale":"2023-10-08T01:12:02.843809+0000","last_undegraded":"2023-10-08T01:12:02.843809+0000","last_fullsized":"2023-10-08T01:12:02.843809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:26.311091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807321+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.838339+0000","last_change":"2023-10-08T01:12:02.836913+0000","last_active":"2023-10-08T01:12:02.838339+0000","last_peered":"2023-10-08T01:12:02.838339+0000","last_clean":"2023-10-08T01:12:02.838339+0000","last_became_active":"2023-10-08T01:12:02.836618+0000","last_became_peered":"2023-10-08T01:12:02.836618+0000","last_unstale":"2023-10-08T01:12:02.838339+0000","last_undegraded":"2023-10-08T01:12:02.838339+0000","last_fullsized":"2023-10-08T01:12:02.838339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:05.095868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850521+0000","last_change":"2023-10-08T01:12:01.831714+0000","last_active":"2023-10-08T01:12:02.850521+0000","last_peered":"2023-10-08T01:12:02.850521+0000","last_clean":"2023-10-08T01:12:02.850521+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850521+0000","last_undegraded":"2023-10-08T01:12:02.850521+0000","last_fullsized":"2023-10-08T01:12:02.850521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:56.535520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.862945+0000","last_change":"2023-10-08T01:12:02.862541+0000","last_active":"2023-10-08T01:12:02.862945+0000","last_peered":"2023-10-08T01:12:02.862945+0000","last_clean":"2023-10-08T01:12:02.862945+0000","last_became_active":"2023-10-08T01:12:02.862076+0000","last_became_peered":"2023-10-08T01:12:02.862076+0000","last_unstale":"2023-10-08T01:12:02.862945+0000","last_undegraded":"2023-10-08T01:12:02.862945+0000","last_fullsized":"2023-10-08T01:12:02.862945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:26.919061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.807911+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:46.971056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852339+0000","last_change":"2023-10-08T01:12:02.849161+0000","last_active":"2023-10-08T01:12:02.852339+0000","last_peered":"2023-10-08T01:12:02.852339+0000","last_clean":"2023-10-08T01:12:02.852339+0000","last_became_active":"2023-10-08T01:12:02.848913+0000","last_became_peered":"2023-10-08T01:12:02.848913+0000","last_unstale":"2023-10-08T01:12:02.852339+0000","last_undegraded":"2023-10-08T01:12:02.852339+0000","last_fullsized":"2023-10-08T01:12:02.852339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:58:40.796959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.832971+0000","last_change":"2023-10-08T01:12:01.819602+0000","last_active":"2023-10-08T01:12:02.832971+0000","last_peered":"2023-10-08T01:12:02.832971+0000","last_clean":"2023-10-08T01:12:02.832971+0000","last_became_active":"2023-10-08T01:12:01.819304+0000","last_became_peered":"2023-10-08T01:12:01.819304+0000","last_unstale":"2023-10-08T01:12:02.832971+0000","last_undegraded":"2023-10-08T01:12:02.832971+0000","last_fullsized":"2023-10-08T01:12:02.832971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:55.795619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846380+0000","last_change":"2023-10-08T01:12:02.844295+0000","last_active":"2023-10-08T01:12:02.846380+0000","last_peered":"2023-10-08T01:12:02.846380+0000","last_clean":"2023-10-08T01:12:02.846380+0000","last_became_active":"2023-10-08T01:12:02.844001+0000","last_became_peered":"2023-10-08T01:12:02.844001+0000","last_unstale":"2023-10-08T01:12:02.846380+0000","last_undegraded":"2023-10-08T01:12:02.846380+0000","last_fullsized":"2023-10-08T01:12:02.846380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:39:36.571615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867968+0000","last_change":"2023-10-08T01:12:01.831354+0000","last_active":"2023-10-08T01:12:02.867968+0000","last_peered":"2023-10-08T01:12:02.867968+0000","last_clean":"2023-10-08T01:12:02.867968+0000","last_became_active":"2023-10-08T01:12:01.830964+0000","last_became_peered":"2023-10-08T01:12:01.830964+0000","last_unstale":"2023-10-08T01:12:02.867968+0000","last_undegraded":"2023-10-08T01:12:02.867968+0000","last_fullsized":"2023-10-08T01:12:02.867968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809329+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:08.110852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828282+0000","last_change":"2023-10-08T01:12:01.823222+0000","last_active":"2023-10-08T01:12:02.828282+0000","last_peered":"2023-10-08T01:12:02.828282+0000","last_clean":"2023-10-08T01:12:02.828282+0000","last_became_active":"2023-10-08T01:12:01.822970+0000","last_became_peered":"2023-10-08T01:12:01.822970+0000","last_unstale":"2023-10-08T01:12:02.828282+0000","last_undegraded":"2023-10-08T01:12:02.828282+0000","last_fullsized":"2023-10-08T01:12:02.828282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:00.437245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834238+0000","last_change":"2023-10-08T01:12:01.815771+0000","last_active":"2023-10-08T01:12:02.834238+0000","last_peered":"2023-10-08T01:12:02.834238+0000","last_clean":"2023-10-08T01:12:02.834238+0000","last_became_active":"2023-10-08T01:12:01.815502+0000","last_became_peered":"2023-10-08T01:12:01.815502+0000","last_unstale":"2023-10-08T01:12:02.834238+0000","last_undegraded":"2023-10-08T01:12:02.834238+0000","last_fullsized":"2023-10-08T01:12:02.834238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843426+0000","last_change":"2023-10-08T01:12:02.838180+0000","last_active":"2023-10-08T01:12:02.843426+0000","last_peered":"2023-10-08T01:12:02.843426+0000","last_clean":"2023-10-08T01:12:02.843426+0000","last_became_active":"2023-10-08T01:12:02.837945+0000","last_became_peered":"2023-10-08T01:12:02.837945+0000","last_unstale":"2023-10-08T01:12:02.843426+0000","last_undegraded":"2023-10-08T01:12:02.843426+0000","last_fullsized":"2023-10-08T01:12:02.843426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:21.100043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.827017+0000","last_change":"2023-10-08T01:12:01.827250+0000","last_active":"2023-10-08T01:12:02.827017+0000","last_peered":"2023-10-08T01:12:02.827017+0000","last_clean":"2023-10-08T01:12:02.827017+0000","last_became_active":"2023-10-08T01:12:01.826935+0000","last_became_peered":"2023-10-08T01:12:01.826935+0000","last_unstale":"2023-10-08T01:12:02.827017+0000","last_undegraded":"2023-10-08T01:12:02.827017+0000","last_fullsized":"2023-10-08T01:12:02.827017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:40.815318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860494+0000","last_change":"2023-10-08T01:12:02.856747+0000","last_active":"2023-10-08T01:12:02.860494+0000","last_peered":"2023-10-08T01:12:02.860494+0000","last_clean":"2023-10-08T01:12:02.860494+0000","last_became_active":"2023-10-08T01:12:02.856476+0000","last_became_peered":"2023-10-08T01:12:02.856476+0000","last_unstale":"2023-10-08T01:12:02.860494+0000","last_undegraded":"2023-10-08T01:12:02.860494+0000","last_fullsized":"2023-10-08T01:12:02.860494+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:00.451849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.809989+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:01.799734+0000","last_change":"2023-10-08T01:12:01.799734+0000","last_active":"2023-10-08T01:12:01.799734+0000","last_peered":"2023-10-08T01:12:01.799734+0000","last_clean":"2023-10-08T01:12:01.799734+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:01.799734+0000","last_undegraded":"2023-10-08T01:12:01.799734+0000","last_fullsized":"2023-10-08T01:12:01.799734+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:12:00.794219+0000","last_change":"2023-10-08T01:12:00.804185+0000","last_active":"2023-10-08T01:12:00.794219+0000","last_peered":"2023-10-08T01:12:00.794219+0000","last_clean":"2023-10-08T01:12:00.794219+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:00.794219+0000","last_undegraded":"2023-10-08T01:12:00.794219+0000","last_fullsized":"2023-10-08T01:12:00.794219+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:50.137372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859915+0000","last_change":"2023-10-08T01:12:02.855156+0000","last_active":"2023-10-08T01:12:02.859915+0000","last_peered":"2023-10-08T01:12:02.859915+0000","last_clean":"2023-10-08T01:12:02.859915+0000","last_became_active":"2023-10-08T01:12:02.854809+0000","last_became_peered":"2023-10-08T01:12:02.854809+0000","last_unstale":"2023-10-08T01:12:02.859915+0000","last_undegraded":"2023-10-08T01:12:02.859915+0000","last_fullsized":"2023-10-08T01:12:02.859915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:46.539103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850388+0000","last_change":"2023-10-08T01:12:01.831667+0000","last_active":"2023-10-08T01:12:02.850388+0000","last_peered":"2023-10-08T01:12:02.850388+0000","last_clean":"2023-10-08T01:12:02.850388+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850388+0000","last_undegraded":"2023-10-08T01:12:02.850388+0000","last_fullsized":"2023-10-08T01:12:02.850388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:23.287500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852439+0000","last_change":"2023-10-08T01:12:02.849116+0000","last_active":"2023-10-08T01:12:02.852439+0000","last_peered":"2023-10-08T01:12:02.852439+0000","last_clean":"2023-10-08T01:12:02.852439+0000","last_became_active":"2023-10-08T01:12:02.848833+0000","last_became_peered":"2023-10-08T01:12:02.848833+0000","last_unstale":"2023-10-08T01:12:02.852439+0000","last_undegraded":"2023-10-08T01:12:02.852439+0000","last_fullsized":"2023-10-08T01:12:02.852439+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:35.798359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.870650+0000","last_change":"2023-10-08T01:12:01.828087+0000","last_active":"2023-10-08T01:12:02.870650+0000","last_peered":"2023-10-08T01:12:02.870650+0000","last_clean":"2023-10-08T01:12:02.870650+0000","last_became_active":"2023-10-08T01:12:01.827688+0000","last_became_peered":"2023-10-08T01:12:01.827688+0000","last_unstale":"2023-10-08T01:12:02.870650+0000","last_undegraded":"2023-10-08T01:12:02.870650+0000","last_fullsized":"2023-10-08T01:12:02.870650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:25.305577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"80'183","reported_seq":457,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-10-08T01:11:58.219508+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:11:58.219508+0000","last_peered":"2023-10-08T01:11:58.219508+0000","last_clean":"2023-10-08T01:11:58.219508+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:11:58.219508+0000","last_undegraded":"2023-10-08T01:11:58.219508+0000","last_fullsized":"2023-10-08T01:11:58.219508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":4},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":182,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":14,"ondisk_log_size":14,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303012,"kb_used_data":1628,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068828,"statfs":{"total":96636764160,"available":96326479872,"internally_reserved":0,"allocated":1667072,"data_stored":1106358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.65600000000000003}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.58599999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.72599999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.504},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.60299999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.623}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.746}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1118208,"data_stored":536572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.497},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.74199999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.83699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.28599999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.495},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.47199999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.95599999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.85099999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":1.026},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.92800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303048,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068792,"statfs":{"total":96636764160,"available":96326443008,"internally_reserved":0,"allocated":1703936,"data_stored":1126848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50800000000000001,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.617},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.5289999999999999},"last":0.754}]},{"osd":6,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.208,"5min":1.361,"15min":1.3779999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1719999999999999,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.65000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306576,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065264,"statfs":{"total":96636764160,"available":96322830336,"internally_reserved":0,"allocated":1122304,"data_stored":538016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306572,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065268,"statfs":{"total":96636764160,"available":96322834432,"internally_reserved":0,"allocated":1118208,"data_stored":536592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.69999999999999996,"15min":0.80100000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.57399999999999995,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.378}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61699999999999999,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.82999999999999996,"15min":0.88100000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.79400000000000004,"15min":1.1060000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":1.095,"15min":1.3029999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74299999999999999,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.117,"15min":1.413},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.77800000000000002},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.0920000000000001,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":1.089},"last":0.61899999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.379},"last":0.751},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.486},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97099999999999997,"5min":1.0840000000000001,"15min":1.389},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.79700000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.5},"max":{"1min":0.93700000000000006,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.65500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575043,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310624,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061216,"statfs":{"total":96636764160,"available":96318685184,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.72199999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.75}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.80900000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.90400000000000003}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.66700000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.88300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575043,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302432,"kb_used_data":1048,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069408,"statfs":{"total":96636764160,"available":96327073792,"internally_reserved":0,"allocated":1073152,"data_stored":515990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.69099999999999995}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.66100000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.70899999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.622}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.57199999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.749},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.76300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:12:08.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-10-08T01:12:08.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.074 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-10-08T01:12:08.074 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-10-08T01:12:08.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-10-08T01:12:08.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.075 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-10-08T01:12:08.075 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-10-08T01:12:08.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.076 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-10-08T01:12:08.076 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-10-08T01:12:08.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-10-08T01:12:08.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-10-08T01:12:08.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.077 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-10-08T01:12:08.077 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.078 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-10-08T01:12:08.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-10-08T01:12:08.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-10-08T01:12:08.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.079 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-10-08T01:12:08.079 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.807321+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:12:08.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-10-08T01:12:08.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.804484+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:25:03.061391+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:12:08.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-10-08T01:12:08.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-10-08T01:12:08.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.080 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-10-08T01:12:08.080 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-10-08T01:12:08.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.809293+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:23:16.271266+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:12:08.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-10-08T01:12:08.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-10-08T01:12:08.081 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.804453+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:07:38.169246+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:12:08.081 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-10-08T01:12:08.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-10-08T01:12:08.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.809400+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:12:08.082 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-10-08T01:12:08.082 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-10-08T01:12:08.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.809631+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:12:08.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-10-08T01:12:08.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-10-08T01:12:08.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.808808+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:37:47.152989+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:12:08.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.804185+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:01:38.855490+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.804397+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:42:54.295030+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-10-08T01:12:08.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-10-08T01:12:08.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.807321+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:12:08.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-10-08T01:12:08.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.807911+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:18:46.971056+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:12:08.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-10-08T01:12:08.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-10-08T01:12:08.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.809329+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:10:08.110852+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:12:08.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-10-08T01:12:08.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-10-08T01:12:08.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-10-08T01:12:08.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.809989+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:12:08.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-10-08T01:12:08.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:12:01.799734+0000', 'last_change': '2023-10-08T01:12:01.799734+0000', 'last_active': '2023-10-08T01:12:01.799734+0000', 'last_peered': '2023-10-08T01:12:01.799734+0000', 'last_clean': '2023-10-08T01:12:01.799734+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:01.799734+0000', 'last_undegraded': '2023-10-08T01:12:01.799734+0000', 'last_fullsized': '2023-10-08T01:12:01.799734+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:01.799734+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:12:08.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-10-08T01:12:08.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:12:00.794219+0000', 'last_change': '2023-10-08T01:12:00.804185+0000', 'last_active': '2023-10-08T01:12:00.794219+0000', 'last_peered': '2023-10-08T01:12:00.794219+0000', 'last_clean': '2023-10-08T01:12:00.794219+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:12:00.794219+0000', 'last_undegraded': '2023-10-08T01:12:00.794219+0000', 'last_fullsized': '2023-10-08T01:12:00.794219+0000', 'mapping_epoch': 88, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 88, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'last_clean_scrub_stamp': '2023-10-08T01:12:00.794219+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:20:50.137372+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:12:08.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T01:12:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.283+0000 7f69de3ff700 1 Processor -- start 2023-10-08T01:12:08.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.283+0000 7f69de3ff700 1 -- start start 2023-10-08T01:12:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69de3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69d8156510 0x7f69d80ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69de3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d80ad1e0 con 0x7f69d80b0ca0 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69de3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69d8153f10 con 0x7f69d80acd40 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69de3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69d80ad9d0 con 0x7f69d8156510 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69d7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69d8156510 0x7f69d80ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69dc99c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f69d80b0ca0 0x7f69d815fad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52244/0 (socket says 172.21.15.119:52244) 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69dc99c700 1 -- 172.21.15.119:0/1021301958 learned_addr learned my addr 172.21.15.119:0/1021301958 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69d7fff700 1 -- 172.21.15.119:0/1021301958 >> v1:172.21.15.184:6789/0 conn(0x7f69d80acd40 legacy=0x7f69d81568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69d7fff700 1 -- 172.21.15.119:0/1021301958 >> v1:172.21.15.119:6789/0 conn(0x7f69d80b0ca0 legacy=0x7f69d815fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69d7fff700 1 -- 172.21.15.119:0/1021301958 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80af800 con 0x7f69d8156510 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.285+0000 7f69d77fe700 1 -- 172.21.15.119:0/1021301958 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933728440 0 0) 0x7f69d80ad1e0 con 0x7f69d80b0ca0 2023-10-08T01:12:08.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.286+0000 7f69d7fff700 1 --2- 172.21.15.119:0/1021301958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d8156510 0x7f69d80ac800 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f69c800aa90 tx=0x7f69c800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4d2b6a9f8869cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:08.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.286+0000 7f69d77fe700 1 -- 172.21.15.119:0/1021301958 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c800ca40 con 0x7f69d8156510 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.286+0000 7f69d77fe700 1 -- 172.21.15.119:0/1021301958 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c800cba0 con 0x7f69d8156510 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.286+0000 7f69d77fe700 1 -- 172.21.15.119:0/1021301958 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8018400 con 0x7f69d8156510 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.286+0000 7f69de3ff700 1 -- 172.21.15.119:0/1021301958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d8156510 msgr2=0x7f69d80ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.286+0000 7f69de3ff700 1 --2- 172.21.15.119:0/1021301958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d8156510 0x7f69d80ac800 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f69c800aa90 tx=0x7f69c800a330 comp rx=0 tx=0).stop 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.287+0000 7f69de3ff700 1 -- 172.21.15.119:0/1021301958 shutdown_connections 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.287+0000 7f69de3ff700 1 --2- 172.21.15.119:0/1021301958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d8156510 0x7f69d80ac800 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.287+0000 7f69de3ff700 1 -- 172.21.15.119:0/1021301958 >> 172.21.15.119:0/1021301958 conn(0x7f69d806a310 msgr2=0x7f69d80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.287+0000 7f69de3ff700 1 -- 172.21.15.119:0/1021301958 shutdown_connections 2023-10-08T01:12:08.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.287+0000 7f69de3ff700 1 -- 172.21.15.119:0/1021301958 wait complete. 2023-10-08T01:12:08.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.288+0000 7f69de3ff700 1 Processor -- start 2023-10-08T01:12:08.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.288+0000 7f69de3ff700 1 -- start start 2023-10-08T01:12:08.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69de3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d80acd40 0x7f69d8169b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69de3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69d80b0ca0 0x7f69d806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69de3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69d8156510 0x7f69d8144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69de3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d816e090 con 0x7f69d80acd40 2023-10-08T01:12:08.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69de3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80ca620 con 0x7f69d80b0ca0 2023-10-08T01:12:08.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69de3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d80ca900 con 0x7f69d8156510 2023-10-08T01:12:08.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d80acd40 0x7f69d8169b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69d7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d80acd40 0x7f69d8169b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55180/0 (socket says 172.21.15.119:55180) 2023-10-08T01:12:08.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69d7fff700 1 -- 172.21.15.119:0/451270296 learned_addr learned my addr 172.21.15.119:0/451270296 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:08.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69dc99c700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69d8156510 0x7f69d8144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.289+0000 7f69cffff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69d80b0ca0 0x7f69d806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d7fff700 1 -- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69d8156510 msgr2=0x7f69d8144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d7fff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69d8156510 0x7f69d8144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d7fff700 1 -- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69d80b0ca0 msgr2=0x7f69d806eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d7fff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69d80b0ca0 0x7f69d806eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d7fff700 1 -- 172.21.15.119:0/451270296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c800a5d0 con 0x7f69d80acd40 2023-10-08T01:12:08.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69cffff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69d80b0ca0 0x7f69d806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:08.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d7fff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d80acd40 0x7f69d8169b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f69d80cd610 tx=0x7f69c801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:08.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d4ff9700 1 -- 172.21.15.119:0/451270296 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8010600 con 0x7f69d80acd40 2023-10-08T01:12:08.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d4ff9700 1 -- 172.21.15.119:0/451270296 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69c8010780 con 0x7f69d80acd40 2023-10-08T01:12:08.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69d80cabe0 con 0x7f69d80acd40 2023-10-08T01:12:08.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69d4ff9700 1 -- 172.21.15.119:0/451270296 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69c8010b40 con 0x7f69d80acd40 2023-10-08T01:12:08.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.290+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69d816a0b0 con 0x7f69d80acd40 2023-10-08T01:12:08.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.291+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69d816a2d0 con 0x7f69d80acd40 2023-10-08T01:12:08.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.292+0000 7f69d4ff9700 1 -- 172.21.15.119:0/451270296 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f69c8021420 con 0x7f69d80acd40 2023-10-08T01:12:08.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.293+0000 7f69d4ff9700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f69b8066f30 0x7f69b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.293+0000 7f69d4ff9700 1 -- 172.21.15.119:0/451270296 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f69c8086080 con 0x7f69d80acd40 2023-10-08T01:12:08.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.293+0000 7f69cffff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f69b8066f30 0x7f69b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.294+0000 7f69cffff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f69b8066f30 0x7f69b80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f69d8069330 tx=0x7f69c0009040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:08.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.297+0000 7f69d4ff9700 1 -- 172.21.15.119:0/451270296 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69c80b75a0 con 0x7f69d80acd40 2023-10-08T01:12:08.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.564+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f69d804c3c0 con 0x7f69d80acd40 2023-10-08T01:12:08.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.564+0000 7f69d4ff9700 1 -- 172.21.15.119:0/451270296 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1176 (secure 0 0 0) 0x7f69c80530c0 con 0x7f69d80acd40 2023-10-08T01:12:08.565 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:08.566 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1005,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1696726535,"num_in_osds":8,"osd_in_since":1696726526,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":51},{"state_name":"unknown","count":32},{"state_name":"creating+peering","count":14}],"num_pgs":97,"num_pools":3,"num_objects":9,"data_bytes":590550,"bytes_used":2495664128,"bytes_avail":770598449152,"bytes_total":773094113280,"unknown_pgs_ratio":0.32989689707756042,"inactive_pgs_ratio":0.14432989060878754,"write_bytes_sec":520,"read_op_per_sec":0,"write_op_per_sec":2},"fsmap":{"epoch":39,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"b","status":"up:active","gid":6708}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-10-08T01:10:26.477565+0000","services":{}},"progress_events":{}} 2023-10-08T01:12:08.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f69b8066f30 msgr2=0x7f69b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f69b8066f30 0x7f69b80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f69d8069330 tx=0x7f69c0009040 comp rx=0 tx=0).stop 2023-10-08T01:12:08.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d80acd40 msgr2=0x7f69d8169b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d80acd40 0x7f69d8169b70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f69d80cd610 tx=0x7f69c801a000 comp rx=0 tx=0).stop 2023-10-08T01:12:08.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 shutdown_connections 2023-10-08T01:12:08.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f69b8066f30 0x7f69b80693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69d8156510 0x7f69d8144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69d80acd40 0x7f69d8169b70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 --2- 172.21.15.119:0/451270296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69d80b0ca0 0x7f69d806eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.566+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 >> 172.21.15.119:0/451270296 conn(0x7f69d806a310 msgr2=0x7f69d81568e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:08.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.567+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 shutdown_connections 2023-10-08T01:12:08.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.567+0000 7f69de3ff700 1 -- 172.21.15.119:0/451270296 wait complete. 2023-10-08T01:12:08.585 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T01:12:08.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:12:08.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.767+0000 7f0c8519d700 1 Processor -- start 2023-10-08T01:12:08.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.767+0000 7f0c8519d700 1 -- start start 2023-10-08T01:12:08.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c8519d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0c801597f0 0x7f0c800b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c8519d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800b11d0 con 0x7f0c80159420 2023-10-08T01:12:08.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c8519d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c800b14b0 con 0x7f0c8015d230 2023-10-08T01:12:08.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c8519d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c800b1790 con 0x7f0c801597f0 2023-10-08T01:12:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0c80159420 0x7f0c800b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52248/0 (socket says 172.21.15.119:52248) 2023-10-08T01:12:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7ffff700 1 -- 172.21.15.119:0/2959424532 learned_addr learned my addr 172.21.15.119:0/2959424532 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7f7fe700 1 --2- 172.21.15.119:0/2959424532 >> v2:172.21.15.119:3300/0 conn(0x7f0c801597f0 0x7f0c800b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7effd700 1 -- 172.21.15.119:0/2959424532 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654832615 0 0) 0x7f0c800b11d0 con 0x7f0c80159420 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7effd700 1 -- 172.21.15.119:0/2959424532 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c68003570 con 0x7f0c80159420 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7f7fe700 1 -- 172.21.15.119:0/2959424532 >> v1:172.21.15.184:6789/0 conn(0x7f0c8015d230 legacy=0x7f0c8015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7f7fe700 1 -- 172.21.15.119:0/2959424532 >> v1:172.21.15.119:6789/0 conn(0x7f0c80159420 legacy=0x7f0c800b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7f7fe700 1 -- 172.21.15.119:0/2959424532 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c800b1a70 con 0x7f0c801597f0 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7f7fe700 1 --2- 172.21.15.119:0/2959424532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c800b2270 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f0c740098e0 tx=0x7f0c740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc50679234677f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7effd700 1 -- 172.21.15.119:0/2959424532 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400b640 con 0x7f0c801597f0 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.768+0000 7f0c7effd700 1 -- 172.21.15.119:0/2959424532 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7400b7a0 con 0x7f0c801597f0 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.769+0000 7f0c7effd700 1 -- 172.21.15.119:0/2959424532 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400bb40 con 0x7f0c801597f0 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.769+0000 7f0c8519d700 1 -- 172.21.15.119:0/2959424532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 msgr2=0x7f0c800b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.769+0000 7f0c8519d700 1 --2- 172.21.15.119:0/2959424532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c800b2270 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f0c740098e0 tx=0x7f0c740091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.769+0000 7f0c8519d700 1 -- 172.21.15.119:0/2959424532 shutdown_connections 2023-10-08T01:12:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.769+0000 7f0c8519d700 1 --2- 172.21.15.119:0/2959424532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c800b2270 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.769+0000 7f0c8519d700 1 -- 172.21.15.119:0/2959424532 >> 172.21.15.119:0/2959424532 conn(0x7f0c800d0660 msgr2=0x7f0c800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.769+0000 7f0c8519d700 1 -- 172.21.15.119:0/2959424532 shutdown_connections 2023-10-08T01:12:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.769+0000 7f0c8519d700 1 -- 172.21.15.119:0/2959424532 wait complete. 2023-10-08T01:12:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.771+0000 7f0c8519d700 1 Processor -- start 2023-10-08T01:12:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.771+0000 7f0c8519d700 1 -- start start 2023-10-08T01:12:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c8519d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c80159420 0x7f0c80153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c8519d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c8016a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c8519d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015d230 0x7f0c80068ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c8519d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c801546c0 con 0x7f0c801597f0 2023-10-08T01:12:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c8519d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8015bd10 con 0x7f0c80159420 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c8519d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8016a610 con 0x7f0c8015d230 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c8016a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c8016a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55192/0 (socket says 172.21.15.119:55192) 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7f7fe700 1 -- 172.21.15.119:0/3703353088 learned_addr learned my addr 172.21.15.119:0/3703353088 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7ffff700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c80159420 0x7f0c80153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c8499c700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015d230 0x7f0c80068ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7f7fe700 1 -- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015d230 msgr2=0x7f0c80068ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7f7fe700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015d230 0x7f0c80068ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7f7fe700 1 -- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c80159420 msgr2=0x7f0c80153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7f7fe700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c80159420 0x7f0c80153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7f7fe700 1 -- 172.21.15.119:0/3703353088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c74009420 con 0x7f0c801597f0 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c7ffff700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c80159420 0x7f0c80153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.772+0000 7f0c8499c700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015d230 0x7f0c80068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.773+0000 7f0c7f7fe700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c8016a0d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0c74013000 tx=0x7f0c74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.773+0000 7f0c67fff700 1 -- 172.21.15.119:0/3703353088 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7400bb20 con 0x7f0c801597f0 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.773+0000 7f0c67fff700 1 -- 172.21.15.119:0/3703353088 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c74011420 con 0x7f0c801597f0 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.773+0000 7f0c67fff700 1 -- 172.21.15.119:0/3703353088 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c740117c0 con 0x7f0c801597f0 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.773+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8015b100 con 0x7f0c801597f0 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.773+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8015b6c0 con 0x7f0c801597f0 2023-10-08T01:12:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.774+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8015be80 con 0x7f0c801597f0 2023-10-08T01:12:08.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.775+0000 7f0c67fff700 1 -- 172.21.15.119:0/3703353088 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0c74011b70 con 0x7f0c801597f0 2023-10-08T01:12:08.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.776+0000 7f0c67fff700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c6c066f30 0x7f0c6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:08.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.776+0000 7f0c67fff700 1 -- 172.21.15.119:0/3703353088 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f0c74085e60 con 0x7f0c801597f0 2023-10-08T01:12:08.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.776+0000 7f0c7ffff700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c6c066f30 0x7f0c6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:08.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.777+0000 7f0c7ffff700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c6c066f30 0x7f0c6c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c800cda40 tx=0x7f0c70009040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:08.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.781+0000 7f0c67fff700 1 -- 172.21.15.119:0/3703353088 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c74052ea0 con 0x7f0c801597f0 2023-10-08T01:12:08.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.968+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c8004c3c0 con 0x7f0c6c066f30 2023-10-08T01:12:08.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.977+0000 7f0c67fff700 1 -- 172.21.15.119:0/3703353088 <== mgr.6516 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289236 (secure 0 0 0) 0x7f0c8004c3c0 con 0x7f0c6c066f30 2023-10-08T01:12:08.983 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.981+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c6c066f30 msgr2=0x7f0c6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.981+0000 7f0c8519d700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c6c066f30 0x7f0c6c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0c800cda40 tx=0x7f0c70009040 comp rx=0 tx=0).stop 2023-10-08T01:12:08.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.981+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 msgr2=0x7f0c8016a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.981+0000 7f0c8519d700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c8016a0d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f0c74013000 tx=0x7f0c74017000 comp rx=0 tx=0).stop 2023-10-08T01:12:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.981+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 shutdown_connections 2023-10-08T01:12:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.981+0000 7f0c8519d700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c6c066f30 0x7f0c6c0693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.981+0000 7f0c8519d700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c8015d230 0x7f0c80068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.981+0000 7f0c8519d700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c80159420 0x7f0c80153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.982+0000 7f0c8519d700 1 --2- 172.21.15.119:0/3703353088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c801597f0 0x7f0c8016a0d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:08.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.982+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 >> 172.21.15.119:0/3703353088 conn(0x7f0c800d0660 msgr2=0x7f0c800b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:08.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.982+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 shutdown_connections 2023-10-08T01:12:08.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:08.982+0000 7f0c8519d700 1 -- 172.21.15.119:0/3703353088 wait complete. 2023-10-08T01:12:08.989 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:12:08.999 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T01:12:08.707559+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2437352,"kb_used_data":9896,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752537368,"statfs":{"total":773094113280,"available":770598264832,"internally_reserved":0,"allocated":10133504,"data_stored":5477188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.901611"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852804+0000","last_change":"2023-10-08T01:12:02.847769+0000","last_active":"2023-10-08T01:12:02.852804+0000","last_peered":"2023-10-08T01:12:02.852804+0000","last_clean":"2023-10-08T01:12:02.852804+0000","last_became_active":"2023-10-08T01:12:02.847457+0000","last_became_peered":"2023-10-08T01:12:02.847457+0000","last_unstale":"2023-10-08T01:12:02.852804+0000","last_undegraded":"2023-10-08T01:12:02.852804+0000","last_fullsized":"2023-10-08T01:12:02.852804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:27.793380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866793+0000","last_change":"2023-10-08T01:12:02.861488+0000","last_active":"2023-10-08T01:12:02.866793+0000","last_peered":"2023-10-08T01:12:02.866793+0000","last_clean":"2023-10-08T01:12:02.866793+0000","last_became_active":"2023-10-08T01:12:02.860018+0000","last_became_peered":"2023-10-08T01:12:02.860018+0000","last_unstale":"2023-10-08T01:12:02.866793+0000","last_undegraded":"2023-10-08T01:12:02.866793+0000","last_fullsized":"2023-10-08T01:12:02.866793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.163695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846099+0000","last_change":"2023-10-08T01:12:02.841414+0000","last_active":"2023-10-08T01:12:02.846099+0000","last_peered":"2023-10-08T01:12:02.846099+0000","last_clean":"2023-10-08T01:12:02.846099+0000","last_became_active":"2023-10-08T01:12:02.841091+0000","last_became_peered":"2023-10-08T01:12:02.841091+0000","last_unstale":"2023-10-08T01:12:02.846099+0000","last_undegraded":"2023-10-08T01:12:02.846099+0000","last_fullsized":"2023-10-08T01:12:02.846099+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:03.550299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.862643+0000","last_change":"2023-10-08T01:12:02.860263+0000","last_active":"2023-10-08T01:12:02.862643+0000","last_peered":"2023-10-08T01:12:02.862643+0000","last_clean":"2023-10-08T01:12:02.862643+0000","last_became_active":"2023-10-08T01:12:02.860067+0000","last_became_peered":"2023-10-08T01:12:02.860067+0000","last_unstale":"2023-10-08T01:12:02.862643+0000","last_undegraded":"2023-10-08T01:12:02.862643+0000","last_fullsized":"2023-10-08T01:12:02.862643+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:18.546972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845916+0000","last_change":"2023-10-08T01:12:02.841353+0000","last_active":"2023-10-08T01:12:02.845916+0000","last_peered":"2023-10-08T01:12:02.845916+0000","last_clean":"2023-10-08T01:12:02.845916+0000","last_became_active":"2023-10-08T01:12:02.841117+0000","last_became_peered":"2023-10-08T01:12:02.841117+0000","last_unstale":"2023-10-08T01:12:02.845916+0000","last_undegraded":"2023-10-08T01:12:02.845916+0000","last_fullsized":"2023-10-08T01:12:02.845916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:32.447875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859766+0000","last_change":"2023-10-08T01:12:02.851025+0000","last_active":"2023-10-08T01:12:02.859766+0000","last_peered":"2023-10-08T01:12:02.859766+0000","last_clean":"2023-10-08T01:12:02.859766+0000","last_became_active":"2023-10-08T01:12:02.850765+0000","last_became_peered":"2023-10-08T01:12:02.850765+0000","last_unstale":"2023-10-08T01:12:02.859766+0000","last_undegraded":"2023-10-08T01:12:02.859766+0000","last_fullsized":"2023-10-08T01:12:02.859766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:09.116366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850784+0000","last_change":"2023-10-08T01:12:02.838614+0000","last_active":"2023-10-08T01:12:02.850784+0000","last_peered":"2023-10-08T01:12:02.850784+0000","last_clean":"2023-10-08T01:12:02.850784+0000","last_became_active":"2023-10-08T01:12:02.838296+0000","last_became_peered":"2023-10-08T01:12:02.838296+0000","last_unstale":"2023-10-08T01:12:02.850784+0000","last_undegraded":"2023-10-08T01:12:02.850784+0000","last_fullsized":"2023-10-08T01:12:02.850784+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:52.932416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860705+0000","last_change":"2023-10-08T01:12:02.845674+0000","last_active":"2023-10-08T01:12:02.860705+0000","last_peered":"2023-10-08T01:12:02.860705+0000","last_clean":"2023-10-08T01:12:02.860705+0000","last_became_active":"2023-10-08T01:12:02.845358+0000","last_became_peered":"2023-10-08T01:12:02.845358+0000","last_unstale":"2023-10-08T01:12:02.860705+0000","last_undegraded":"2023-10-08T01:12:02.860705+0000","last_fullsized":"2023-10-08T01:12:02.860705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:47.976571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844181+0000","last_change":"2023-10-08T01:12:02.839332+0000","last_active":"2023-10-08T01:12:02.844181+0000","last_peered":"2023-10-08T01:12:02.844181+0000","last_clean":"2023-10-08T01:12:02.844181+0000","last_became_active":"2023-10-08T01:12:02.838641+0000","last_became_peered":"2023-10-08T01:12:02.838641+0000","last_unstale":"2023-10-08T01:12:02.844181+0000","last_undegraded":"2023-10-08T01:12:02.844181+0000","last_fullsized":"2023-10-08T01:12:02.844181+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:24.293015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.842677+0000","last_change":"2023-10-08T01:12:02.836233+0000","last_active":"2023-10-08T01:12:02.842677+0000","last_peered":"2023-10-08T01:12:02.842677+0000","last_clean":"2023-10-08T01:12:02.842677+0000","last_became_active":"2023-10-08T01:12:02.835917+0000","last_became_peered":"2023-10-08T01:12:02.835917+0000","last_unstale":"2023-10-08T01:12:02.842677+0000","last_undegraded":"2023-10-08T01:12:02.842677+0000","last_fullsized":"2023-10-08T01:12:02.842677+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:21.312491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.848016+0000","last_change":"2023-10-08T01:12:02.842004+0000","last_active":"2023-10-08T01:12:02.848016+0000","last_peered":"2023-10-08T01:12:02.848016+0000","last_clean":"2023-10-08T01:12:02.848016+0000","last_became_active":"2023-10-08T01:12:02.841401+0000","last_became_peered":"2023-10-08T01:12:02.841401+0000","last_unstale":"2023-10-08T01:12:02.848016+0000","last_undegraded":"2023-10-08T01:12:02.848016+0000","last_fullsized":"2023-10-08T01:12:02.848016+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:19:33.997729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845982+0000","last_change":"2023-10-08T01:12:02.841051+0000","last_active":"2023-10-08T01:12:02.845982+0000","last_peered":"2023-10-08T01:12:02.845982+0000","last_clean":"2023-10-08T01:12:02.845982+0000","last_became_active":"2023-10-08T01:12:02.840797+0000","last_became_peered":"2023-10-08T01:12:02.840797+0000","last_unstale":"2023-10-08T01:12:02.845982+0000","last_undegraded":"2023-10-08T01:12:02.845982+0000","last_fullsized":"2023-10-08T01:12:02.845982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:04.066905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851974+0000","last_change":"2023-10-08T01:12:02.849866+0000","last_active":"2023-10-08T01:12:02.851974+0000","last_peered":"2023-10-08T01:12:02.851974+0000","last_clean":"2023-10-08T01:12:02.851974+0000","last_became_active":"2023-10-08T01:12:02.849568+0000","last_became_peered":"2023-10-08T01:12:02.849568+0000","last_unstale":"2023-10-08T01:12:02.851974+0000","last_undegraded":"2023-10-08T01:12:02.851974+0000","last_fullsized":"2023-10-08T01:12:02.851974+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:30.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846415+0000","last_change":"2023-10-08T01:12:02.843295+0000","last_active":"2023-10-08T01:12:02.846415+0000","last_peered":"2023-10-08T01:12:02.846415+0000","last_clean":"2023-10-08T01:12:02.846415+0000","last_became_active":"2023-10-08T01:12:02.842892+0000","last_became_peered":"2023-10-08T01:12:02.842892+0000","last_unstale":"2023-10-08T01:12:02.846415+0000","last_undegraded":"2023-10-08T01:12:02.846415+0000","last_fullsized":"2023-10-08T01:12:02.846415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:20.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851263+0000","last_change":"2023-10-08T01:12:02.847055+0000","last_active":"2023-10-08T01:12:02.851263+0000","last_peered":"2023-10-08T01:12:02.851263+0000","last_clean":"2023-10-08T01:12:02.851263+0000","last_became_active":"2023-10-08T01:12:02.846767+0000","last_became_peered":"2023-10-08T01:12:02.846767+0000","last_unstale":"2023-10-08T01:12:02.851263+0000","last_undegraded":"2023-10-08T01:12:02.851263+0000","last_fullsized":"2023-10-08T01:12:02.851263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:52.932416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866121+0000","last_change":"2023-10-08T01:12:02.864825+0000","last_active":"2023-10-08T01:12:02.866121+0000","last_peered":"2023-10-08T01:12:02.866121+0000","last_clean":"2023-10-08T01:12:02.866121+0000","last_became_active":"2023-10-08T01:12:02.864505+0000","last_became_peered":"2023-10-08T01:12:02.864505+0000","last_unstale":"2023-10-08T01:12:02.866121+0000","last_undegraded":"2023-10-08T01:12:02.866121+0000","last_fullsized":"2023-10-08T01:12:02.866121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:56:06.717153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844213+0000","last_change":"2023-10-08T01:12:02.840294+0000","last_active":"2023-10-08T01:12:02.844213+0000","last_peered":"2023-10-08T01:12:02.844213+0000","last_clean":"2023-10-08T01:12:02.844213+0000","last_became_active":"2023-10-08T01:12:02.838656+0000","last_became_peered":"2023-10-08T01:12:02.838656+0000","last_unstale":"2023-10-08T01:12:02.844213+0000","last_undegraded":"2023-10-08T01:12:02.844213+0000","last_fullsized":"2023-10-08T01:12:02.844213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:57.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867850+0000","last_change":"2023-10-08T01:12:02.863797+0000","last_active":"2023-10-08T01:12:02.867850+0000","last_peered":"2023-10-08T01:12:02.867850+0000","last_clean":"2023-10-08T01:12:02.867850+0000","last_became_active":"2023-10-08T01:12:02.863597+0000","last_became_peered":"2023-10-08T01:12:02.863597+0000","last_unstale":"2023-10-08T01:12:02.867850+0000","last_undegraded":"2023-10-08T01:12:02.867850+0000","last_fullsized":"2023-10-08T01:12:02.867850+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.520669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866715+0000","last_change":"2023-10-08T01:12:02.860578+0000","last_active":"2023-10-08T01:12:02.866715+0000","last_peered":"2023-10-08T01:12:02.866715+0000","last_clean":"2023-10-08T01:12:02.866715+0000","last_became_active":"2023-10-08T01:12:02.859906+0000","last_became_peered":"2023-10-08T01:12:02.859906+0000","last_unstale":"2023-10-08T01:12:02.866715+0000","last_undegraded":"2023-10-08T01:12:02.866715+0000","last_fullsized":"2023-10-08T01:12:02.866715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:48.742822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853436+0000","last_change":"2023-10-08T01:12:02.851814+0000","last_active":"2023-10-08T01:12:02.853436+0000","last_peered":"2023-10-08T01:12:02.853436+0000","last_clean":"2023-10-08T01:12:02.853436+0000","last_became_active":"2023-10-08T01:12:02.851488+0000","last_became_peered":"2023-10-08T01:12:02.851488+0000","last_unstale":"2023-10-08T01:12:02.853436+0000","last_undegraded":"2023-10-08T01:12:02.853436+0000","last_fullsized":"2023-10-08T01:12:02.853436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:37.330422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846971+0000","last_change":"2023-10-08T01:12:02.843829+0000","last_active":"2023-10-08T01:12:02.846971+0000","last_peered":"2023-10-08T01:12:02.846971+0000","last_clean":"2023-10-08T01:12:02.846971+0000","last_became_active":"2023-10-08T01:12:02.843570+0000","last_became_peered":"2023-10-08T01:12:02.843570+0000","last_unstale":"2023-10-08T01:12:02.846971+0000","last_undegraded":"2023-10-08T01:12:02.846971+0000","last_fullsized":"2023-10-08T01:12:02.846971+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:25.168516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.837237+0000","last_change":"2023-10-08T01:12:02.835029+0000","last_active":"2023-10-08T01:12:02.837237+0000","last_peered":"2023-10-08T01:12:02.837237+0000","last_clean":"2023-10-08T01:12:02.837237+0000","last_became_active":"2023-10-08T01:12:02.834601+0000","last_became_peered":"2023-10-08T01:12:02.834601+0000","last_unstale":"2023-10-08T01:12:02.837237+0000","last_undegraded":"2023-10-08T01:12:02.837237+0000","last_fullsized":"2023-10-08T01:12:02.837237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:29.210434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853918+0000","last_change":"2023-10-08T01:12:02.852791+0000","last_active":"2023-10-08T01:12:02.853918+0000","last_peered":"2023-10-08T01:12:02.853918+0000","last_clean":"2023-10-08T01:12:02.853918+0000","last_became_active":"2023-10-08T01:12:02.852501+0000","last_became_peered":"2023-10-08T01:12:02.852501+0000","last_unstale":"2023-10-08T01:12:02.853918+0000","last_undegraded":"2023-10-08T01:12:02.853918+0000","last_fullsized":"2023-10-08T01:12:02.853918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:25:25.157341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.865134+0000","last_change":"2023-10-08T01:12:02.862014+0000","last_active":"2023-10-08T01:12:02.865134+0000","last_peered":"2023-10-08T01:12:02.865134+0000","last_clean":"2023-10-08T01:12:02.865134+0000","last_became_active":"2023-10-08T01:12:02.861727+0000","last_became_peered":"2023-10-08T01:12:02.861727+0000","last_unstale":"2023-10-08T01:12:02.865134+0000","last_undegraded":"2023-10-08T01:12:02.865134+0000","last_fullsized":"2023-10-08T01:12:02.865134+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:32:11.334178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.837294+0000","last_change":"2023-10-08T01:12:02.835365+0000","last_active":"2023-10-08T01:12:02.837294+0000","last_peered":"2023-10-08T01:12:02.837294+0000","last_clean":"2023-10-08T01:12:02.837294+0000","last_became_active":"2023-10-08T01:12:02.835097+0000","last_became_peered":"2023-10-08T01:12:02.835097+0000","last_unstale":"2023-10-08T01:12:02.837294+0000","last_undegraded":"2023-10-08T01:12:02.837294+0000","last_fullsized":"2023-10-08T01:12:02.837294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:48.481704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.865035+0000","last_change":"2023-10-08T01:12:02.863933+0000","last_active":"2023-10-08T01:12:02.865035+0000","last_peered":"2023-10-08T01:12:02.865035+0000","last_clean":"2023-10-08T01:12:02.865035+0000","last_became_active":"2023-10-08T01:12:02.863612+0000","last_became_peered":"2023-10-08T01:12:02.863612+0000","last_unstale":"2023-10-08T01:12:02.865035+0000","last_undegraded":"2023-10-08T01:12:02.865035+0000","last_fullsized":"2023-10-08T01:12:02.865035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:39:36.571615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846116+0000","last_change":"2023-10-08T01:12:02.845033+0000","last_active":"2023-10-08T01:12:02.846116+0000","last_peered":"2023-10-08T01:12:02.846116+0000","last_clean":"2023-10-08T01:12:02.846116+0000","last_became_active":"2023-10-08T01:12:02.844545+0000","last_became_peered":"2023-10-08T01:12:02.844545+0000","last_unstale":"2023-10-08T01:12:02.846116+0000","last_undegraded":"2023-10-08T01:12:02.846116+0000","last_fullsized":"2023-10-08T01:12:02.846116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:36.576301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.842633+0000","last_change":"2023-10-08T01:12:02.840449+0000","last_active":"2023-10-08T01:12:02.842633+0000","last_peered":"2023-10-08T01:12:02.842633+0000","last_clean":"2023-10-08T01:12:02.842633+0000","last_became_active":"2023-10-08T01:12:02.840167+0000","last_became_peered":"2023-10-08T01:12:02.840167+0000","last_unstale":"2023-10-08T01:12:02.842633+0000","last_undegraded":"2023-10-08T01:12:02.842633+0000","last_fullsized":"2023-10-08T01:12:02.842633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:36:37.500845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845074+0000","last_change":"2023-10-08T01:12:02.839789+0000","last_active":"2023-10-08T01:12:02.845074+0000","last_peered":"2023-10-08T01:12:02.845074+0000","last_clean":"2023-10-08T01:12:02.845074+0000","last_became_active":"2023-10-08T01:12:02.839516+0000","last_became_peered":"2023-10-08T01:12:02.839516+0000","last_unstale":"2023-10-08T01:12:02.845074+0000","last_undegraded":"2023-10-08T01:12:02.845074+0000","last_fullsized":"2023-10-08T01:12:02.845074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:42.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856916+0000","last_change":"2023-10-08T01:12:02.855973+0000","last_active":"2023-10-08T01:12:02.856916+0000","last_peered":"2023-10-08T01:12:02.856916+0000","last_clean":"2023-10-08T01:12:02.856916+0000","last_became_active":"2023-10-08T01:12:02.855712+0000","last_became_peered":"2023-10-08T01:12:02.855712+0000","last_unstale":"2023-10-08T01:12:02.856916+0000","last_undegraded":"2023-10-08T01:12:02.856916+0000","last_fullsized":"2023-10-08T01:12:02.856916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:30.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850807+0000","last_change":"2023-10-08T01:12:02.842126+0000","last_active":"2023-10-08T01:12:02.850807+0000","last_peered":"2023-10-08T01:12:02.850807+0000","last_clean":"2023-10-08T01:12:02.850807+0000","last_became_active":"2023-10-08T01:12:02.841671+0000","last_became_peered":"2023-10-08T01:12:02.841671+0000","last_unstale":"2023-10-08T01:12:02.850807+0000","last_undegraded":"2023-10-08T01:12:02.850807+0000","last_fullsized":"2023-10-08T01:12:02.850807+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:12:47.987269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860840+0000","last_change":"2023-10-08T01:12:02.854035+0000","last_active":"2023-10-08T01:12:02.860840+0000","last_peered":"2023-10-08T01:12:02.860840+0000","last_clean":"2023-10-08T01:12:02.860840+0000","last_became_active":"2023-10-08T01:12:02.853704+0000","last_became_peered":"2023-10-08T01:12:02.853704+0000","last_unstale":"2023-10-08T01:12:02.860840+0000","last_undegraded":"2023-10-08T01:12:02.860840+0000","last_fullsized":"2023-10-08T01:12:02.860840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:55.363740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.864742+0000","last_change":"2023-10-08T01:12:02.862777+0000","last_active":"2023-10-08T01:12:02.864742+0000","last_peered":"2023-10-08T01:12:02.864742+0000","last_clean":"2023-10-08T01:12:02.864742+0000","last_became_active":"2023-10-08T01:12:02.862503+0000","last_became_peered":"2023-10-08T01:12:02.862503+0000","last_unstale":"2023-10-08T01:12:02.864742+0000","last_undegraded":"2023-10-08T01:12:02.864742+0000","last_fullsized":"2023-10-08T01:12:02.864742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:45.646920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"90'4","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.865487+0000","last_change":"2023-10-08T01:12:01.826162+0000","last_active":"2023-10-08T01:12:02.865487+0000","last_peered":"2023-10-08T01:12:02.865487+0000","last_clean":"2023-10-08T01:12:02.865487+0000","last_became_active":"2023-10-08T01:12:01.825717+0000","last_became_peered":"2023-10-08T01:12:01.825717+0000","last_unstale":"2023-10-08T01:12:02.865487+0000","last_undegraded":"2023-10-08T01:12:02.865487+0000","last_fullsized":"2023-10-08T01:12:02.865487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:47.476190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844198+0000","last_change":"2023-10-08T01:12:02.839586+0000","last_active":"2023-10-08T01:12:02.844198+0000","last_peered":"2023-10-08T01:12:02.844198+0000","last_clean":"2023-10-08T01:12:02.844198+0000","last_became_active":"2023-10-08T01:12:02.839033+0000","last_became_peered":"2023-10-08T01:12:02.839033+0000","last_unstale":"2023-10-08T01:12:02.844198+0000","last_undegraded":"2023-10-08T01:12:02.844198+0000","last_fullsized":"2023-10-08T01:12:02.844198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:43.348391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856349+0000","last_change":"2023-10-08T01:12:01.818263+0000","last_active":"2023-10-08T01:12:02.856349+0000","last_peered":"2023-10-08T01:12:02.856349+0000","last_clean":"2023-10-08T01:12:02.856349+0000","last_became_active":"2023-10-08T01:12:01.818011+0000","last_became_peered":"2023-10-08T01:12:01.818011+0000","last_unstale":"2023-10-08T01:12:02.856349+0000","last_undegraded":"2023-10-08T01:12:02.856349+0000","last_fullsized":"2023-10-08T01:12:02.856349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860732+0000","last_change":"2023-10-08T01:12:02.857609+0000","last_active":"2023-10-08T01:12:02.860732+0000","last_peered":"2023-10-08T01:12:02.860732+0000","last_clean":"2023-10-08T01:12:02.860732+0000","last_became_active":"2023-10-08T01:12:02.857277+0000","last_became_peered":"2023-10-08T01:12:02.857277+0000","last_unstale":"2023-10-08T01:12:02.860732+0000","last_undegraded":"2023-10-08T01:12:02.860732+0000","last_fullsized":"2023-10-08T01:12:02.860732+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.729095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.863916+0000","last_change":"2023-10-08T01:12:01.831998+0000","last_active":"2023-10-08T01:12:02.863916+0000","last_peered":"2023-10-08T01:12:02.863916+0000","last_clean":"2023-10-08T01:12:02.863916+0000","last_became_active":"2023-10-08T01:12:01.830751+0000","last_became_peered":"2023-10-08T01:12:01.830751+0000","last_unstale":"2023-10-08T01:12:02.863916+0000","last_undegraded":"2023-10-08T01:12:02.863916+0000","last_fullsized":"2023-10-08T01:12:02.863916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:17.541458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852014+0000","last_change":"2023-10-08T01:12:02.846830+0000","last_active":"2023-10-08T01:12:02.852014+0000","last_peered":"2023-10-08T01:12:02.852014+0000","last_clean":"2023-10-08T01:12:02.852014+0000","last_became_active":"2023-10-08T01:12:02.846528+0000","last_became_peered":"2023-10-08T01:12:02.846528+0000","last_unstale":"2023-10-08T01:12:02.852014+0000","last_undegraded":"2023-10-08T01:12:02.852014+0000","last_fullsized":"2023-10-08T01:12:02.852014+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:38.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.868186+0000","last_change":"2023-10-08T01:12:01.831258+0000","last_active":"2023-10-08T01:12:02.868186+0000","last_peered":"2023-10-08T01:12:02.868186+0000","last_clean":"2023-10-08T01:12:02.868186+0000","last_became_active":"2023-10-08T01:12:01.830943+0000","last_became_peered":"2023-10-08T01:12:01.830943+0000","last_unstale":"2023-10-08T01:12:02.868186+0000","last_undegraded":"2023-10-08T01:12:02.868186+0000","last_fullsized":"2023-10-08T01:12:02.868186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850668+0000","last_change":"2023-10-08T01:12:02.839727+0000","last_active":"2023-10-08T01:12:02.850668+0000","last_peered":"2023-10-08T01:12:02.850668+0000","last_clean":"2023-10-08T01:12:02.850668+0000","last_became_active":"2023-10-08T01:12:02.836234+0000","last_became_peered":"2023-10-08T01:12:02.836234+0000","last_unstale":"2023-10-08T01:12:02.850668+0000","last_undegraded":"2023-10-08T01:12:02.850668+0000","last_fullsized":"2023-10-08T01:12:02.850668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:06:09.924996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853340+0000","last_change":"2023-10-08T01:12:01.826432+0000","last_active":"2023-10-08T01:12:02.853340+0000","last_peered":"2023-10-08T01:12:02.853340+0000","last_clean":"2023-10-08T01:12:02.853340+0000","last_became_active":"2023-10-08T01:12:01.826156+0000","last_became_peered":"2023-10-08T01:12:01.826156+0000","last_unstale":"2023-10-08T01:12:02.853340+0000","last_undegraded":"2023-10-08T01:12:02.853340+0000","last_fullsized":"2023-10-08T01:12:02.853340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:26.787866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.837320+0000","last_change":"2023-10-08T01:12:02.835389+0000","last_active":"2023-10-08T01:12:02.837320+0000","last_peered":"2023-10-08T01:12:02.837320+0000","last_clean":"2023-10-08T01:12:02.837320+0000","last_became_active":"2023-10-08T01:12:02.834837+0000","last_became_peered":"2023-10-08T01:12:02.834837+0000","last_unstale":"2023-10-08T01:12:02.837320+0000","last_undegraded":"2023-10-08T01:12:02.837320+0000","last_fullsized":"2023-10-08T01:12:02.837320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:56.801134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867313+0000","last_change":"2023-10-08T01:12:01.833334+0000","last_active":"2023-10-08T01:12:02.867313+0000","last_peered":"2023-10-08T01:12:02.867313+0000","last_clean":"2023-10-08T01:12:02.867313+0000","last_became_active":"2023-10-08T01:12:01.832064+0000","last_became_peered":"2023-10-08T01:12:01.832064+0000","last_unstale":"2023-10-08T01:12:02.867313+0000","last_undegraded":"2023-10-08T01:12:02.867313+0000","last_fullsized":"2023-10-08T01:12:02.867313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:49.406724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846186+0000","last_change":"2023-10-08T01:12:02.843587+0000","last_active":"2023-10-08T01:12:02.846186+0000","last_peered":"2023-10-08T01:12:02.846186+0000","last_clean":"2023-10-08T01:12:02.846186+0000","last_became_active":"2023-10-08T01:12:02.843202+0000","last_became_peered":"2023-10-08T01:12:02.843202+0000","last_unstale":"2023-10-08T01:12:02.846186+0000","last_undegraded":"2023-10-08T01:12:02.846186+0000","last_fullsized":"2023-10-08T01:12:02.846186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:50.412239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834563+0000","last_change":"2023-10-08T01:12:01.827032+0000","last_active":"2023-10-08T01:12:02.834563+0000","last_peered":"2023-10-08T01:12:02.834563+0000","last_clean":"2023-10-08T01:12:02.834563+0000","last_became_active":"2023-10-08T01:12:01.826774+0000","last_became_peered":"2023-10-08T01:12:01.826774+0000","last_unstale":"2023-10-08T01:12:02.834563+0000","last_undegraded":"2023-10-08T01:12:02.834563+0000","last_fullsized":"2023-10-08T01:12:02.834563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:25.305577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851277+0000","last_change":"2023-10-08T01:12:02.847761+0000","last_active":"2023-10-08T01:12:02.851277+0000","last_peered":"2023-10-08T01:12:02.851277+0000","last_clean":"2023-10-08T01:12:02.851277+0000","last_became_active":"2023-10-08T01:12:02.847425+0000","last_became_peered":"2023-10-08T01:12:02.847425+0000","last_unstale":"2023-10-08T01:12:02.851277+0000","last_undegraded":"2023-10-08T01:12:02.851277+0000","last_fullsized":"2023-10-08T01:12:02.851277+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:12:47.987269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.826270+0000","last_change":"2023-10-08T01:12:01.822270+0000","last_active":"2023-10-08T01:12:02.826270+0000","last_peered":"2023-10-08T01:12:02.826270+0000","last_clean":"2023-10-08T01:12:02.826270+0000","last_became_active":"2023-10-08T01:12:01.821995+0000","last_became_peered":"2023-10-08T01:12:01.821995+0000","last_unstale":"2023-10-08T01:12:02.826270+0000","last_undegraded":"2023-10-08T01:12:02.826270+0000","last_fullsized":"2023-10-08T01:12:02.826270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:40.166452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850742+0000","last_change":"2023-10-08T01:12:02.843828+0000","last_active":"2023-10-08T01:12:02.850742+0000","last_peered":"2023-10-08T01:12:02.850742+0000","last_clean":"2023-10-08T01:12:02.850742+0000","last_became_active":"2023-10-08T01:12:02.843560+0000","last_became_peered":"2023-10-08T01:12:02.843560+0000","last_unstale":"2023-10-08T01:12:02.850742+0000","last_undegraded":"2023-10-08T01:12:02.850742+0000","last_fullsized":"2023-10-08T01:12:02.850742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:14.871146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828552+0000","last_change":"2023-10-08T01:12:01.821727+0000","last_active":"2023-10-08T01:12:02.828552+0000","last_peered":"2023-10-08T01:12:02.828552+0000","last_clean":"2023-10-08T01:12:02.828552+0000","last_became_active":"2023-10-08T01:12:01.821455+0000","last_became_peered":"2023-10-08T01:12:01.821455+0000","last_unstale":"2023-10-08T01:12:02.828552+0000","last_undegraded":"2023-10-08T01:12:02.828552+0000","last_fullsized":"2023-10-08T01:12:02.828552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:58.515154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853785+0000","last_change":"2023-10-08T01:12:02.850817+0000","last_active":"2023-10-08T01:12:02.853785+0000","last_peered":"2023-10-08T01:12:02.853785+0000","last_clean":"2023-10-08T01:12:02.853785+0000","last_became_active":"2023-10-08T01:12:02.849008+0000","last_became_peered":"2023-10-08T01:12:02.849008+0000","last_unstale":"2023-10-08T01:12:02.853785+0000","last_undegraded":"2023-10-08T01:12:02.853785+0000","last_fullsized":"2023-10-08T01:12:02.853785+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:09.986293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852304+0000","last_change":"2023-10-08T01:12:01.832041+0000","last_active":"2023-10-08T01:12:02.852304+0000","last_peered":"2023-10-08T01:12:02.852304+0000","last_clean":"2023-10-08T01:12:02.852304+0000","last_became_active":"2023-10-08T01:12:01.831603+0000","last_became_peered":"2023-10-08T01:12:01.831603+0000","last_unstale":"2023-10-08T01:12:02.852304+0000","last_undegraded":"2023-10-08T01:12:02.852304+0000","last_fullsized":"2023-10-08T01:12:02.852304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:42.342876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860424+0000","last_change":"2023-10-08T01:12:02.855912+0000","last_active":"2023-10-08T01:12:02.860424+0000","last_peered":"2023-10-08T01:12:02.860424+0000","last_clean":"2023-10-08T01:12:02.860424+0000","last_became_active":"2023-10-08T01:12:02.855628+0000","last_became_peered":"2023-10-08T01:12:02.855628+0000","last_unstale":"2023-10-08T01:12:02.860424+0000","last_undegraded":"2023-10-08T01:12:02.860424+0000","last_fullsized":"2023-10-08T01:12:02.860424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:48.158503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"90'3","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856557+0000","last_change":"2023-10-08T01:12:01.825659+0000","last_active":"2023-10-08T01:12:02.856557+0000","last_peered":"2023-10-08T01:12:02.856557+0000","last_clean":"2023-10-08T01:12:02.856557+0000","last_became_active":"2023-10-08T01:12:01.825371+0000","last_became_peered":"2023-10-08T01:12:01.825371+0000","last_unstale":"2023-10-08T01:12:02.856557+0000","last_undegraded":"2023-10-08T01:12:02.856557+0000","last_fullsized":"2023-10-08T01:12:02.856557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:46.049146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843348+0000","last_change":"2023-10-08T01:12:02.837217+0000","last_active":"2023-10-08T01:12:02.843348+0000","last_peered":"2023-10-08T01:12:02.843348+0000","last_clean":"2023-10-08T01:12:02.843348+0000","last_became_active":"2023-10-08T01:12:02.836983+0000","last_became_peered":"2023-10-08T01:12:02.836983+0000","last_unstale":"2023-10-08T01:12:02.843348+0000","last_undegraded":"2023-10-08T01:12:02.843348+0000","last_fullsized":"2023-10-08T01:12:02.843348+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:31.604511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"90'6","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860024+0000","last_change":"2023-10-08T01:12:01.822637+0000","last_active":"2023-10-08T01:12:02.860024+0000","last_peered":"2023-10-08T01:12:02.860024+0000","last_clean":"2023-10-08T01:12:02.860024+0000","last_became_active":"2023-10-08T01:12:01.822310+0000","last_became_peered":"2023-10-08T01:12:01.822310+0000","last_unstale":"2023-10-08T01:12:02.860024+0000","last_undegraded":"2023-10-08T01:12:02.860024+0000","last_fullsized":"2023-10-08T01:12:02.860024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:23.287500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852020+0000","last_change":"2023-10-08T01:12:02.847081+0000","last_active":"2023-10-08T01:12:02.852020+0000","last_peered":"2023-10-08T01:12:02.852020+0000","last_clean":"2023-10-08T01:12:02.852020+0000","last_became_active":"2023-10-08T01:12:02.846830+0000","last_became_peered":"2023-10-08T01:12:02.846830+0000","last_unstale":"2023-10-08T01:12:02.852020+0000","last_undegraded":"2023-10-08T01:12:02.852020+0000","last_fullsized":"2023-10-08T01:12:02.852020+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:34:39.597593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860497+0000","last_change":"2023-10-08T01:12:01.824364+0000","last_active":"2023-10-08T01:12:02.860497+0000","last_peered":"2023-10-08T01:12:02.860497+0000","last_clean":"2023-10-08T01:12:02.860497+0000","last_became_active":"2023-10-08T01:12:01.824117+0000","last_became_peered":"2023-10-08T01:12:01.824117+0000","last_unstale":"2023-10-08T01:12:02.860497+0000","last_undegraded":"2023-10-08T01:12:02.860497+0000","last_fullsized":"2023-10-08T01:12:02.860497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:42.342876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850711+0000","last_change":"2023-10-08T01:12:02.841236+0000","last_active":"2023-10-08T01:12:02.850711+0000","last_peered":"2023-10-08T01:12:02.850711+0000","last_clean":"2023-10-08T01:12:02.850711+0000","last_became_active":"2023-10-08T01:12:02.840900+0000","last_became_peered":"2023-10-08T01:12:02.840900+0000","last_unstale":"2023-10-08T01:12:02.850711+0000","last_undegraded":"2023-10-08T01:12:02.850711+0000","last_fullsized":"2023-10-08T01:12:02.850711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:37.330422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850403+0000","last_change":"2023-10-08T01:12:01.829191+0000","last_active":"2023-10-08T01:12:02.850403+0000","last_peered":"2023-10-08T01:12:02.850403+0000","last_clean":"2023-10-08T01:12:02.850403+0000","last_became_active":"2023-10-08T01:12:01.828873+0000","last_became_peered":"2023-10-08T01:12:01.828873+0000","last_unstale":"2023-10-08T01:12:02.850403+0000","last_undegraded":"2023-10-08T01:12:02.850403+0000","last_fullsized":"2023-10-08T01:12:02.850403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:31.442360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851325+0000","last_change":"2023-10-08T01:12:02.848989+0000","last_active":"2023-10-08T01:12:02.851325+0000","last_peered":"2023-10-08T01:12:02.851325+0000","last_clean":"2023-10-08T01:12:02.851325+0000","last_became_active":"2023-10-08T01:12:02.848775+0000","last_became_peered":"2023-10-08T01:12:02.848775+0000","last_unstale":"2023-10-08T01:12:02.851325+0000","last_undegraded":"2023-10-08T01:12:02.851325+0000","last_fullsized":"2023-10-08T01:12:02.851325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:30.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.833785+0000","last_change":"2023-10-08T01:12:01.828030+0000","last_active":"2023-10-08T01:12:02.833785+0000","last_peered":"2023-10-08T01:12:02.833785+0000","last_clean":"2023-10-08T01:12:02.833785+0000","last_became_active":"2023-10-08T01:12:01.827589+0000","last_became_peered":"2023-10-08T01:12:01.827589+0000","last_unstale":"2023-10-08T01:12:02.833785+0000","last_undegraded":"2023-10-08T01:12:02.833785+0000","last_fullsized":"2023-10-08T01:12:02.833785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:59.446334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843699+0000","last_change":"2023-10-08T01:12:02.837963+0000","last_active":"2023-10-08T01:12:02.843699+0000","last_peered":"2023-10-08T01:12:02.843699+0000","last_clean":"2023-10-08T01:12:02.843699+0000","last_became_active":"2023-10-08T01:12:02.837114+0000","last_became_peered":"2023-10-08T01:12:02.837114+0000","last_unstale":"2023-10-08T01:12:02.843699+0000","last_undegraded":"2023-10-08T01:12:02.843699+0000","last_fullsized":"2023-10-08T01:12:02.843699+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:42.075297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"90'8","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859108+0000","last_change":"2023-10-08T01:12:01.823157+0000","last_active":"2023-10-08T01:12:02.859108+0000","last_peered":"2023-10-08T01:12:02.859108+0000","last_clean":"2023-10-08T01:12:02.859108+0000","last_became_active":"2023-10-08T01:12:01.822907+0000","last_became_peered":"2023-10-08T01:12:01.822907+0000","last_unstale":"2023-10-08T01:12:02.859108+0000","last_undegraded":"2023-10-08T01:12:02.859108+0000","last_fullsized":"2023-10-08T01:12:02.859108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:58.158180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.842697+0000","last_change":"2023-10-08T01:12:02.837019+0000","last_active":"2023-10-08T01:12:02.842697+0000","last_peered":"2023-10-08T01:12:02.842697+0000","last_clean":"2023-10-08T01:12:02.842697+0000","last_became_active":"2023-10-08T01:12:02.836721+0000","last_became_peered":"2023-10-08T01:12:02.836721+0000","last_unstale":"2023-10-08T01:12:02.842697+0000","last_undegraded":"2023-10-08T01:12:02.842697+0000","last_fullsized":"2023-10-08T01:12:02.842697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:44.223005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867962+0000","last_change":"2023-10-08T01:12:01.831319+0000","last_active":"2023-10-08T01:12:02.867962+0000","last_peered":"2023-10-08T01:12:02.867962+0000","last_clean":"2023-10-08T01:12:02.867962+0000","last_became_active":"2023-10-08T01:12:01.831046+0000","last_became_peered":"2023-10-08T01:12:01.831046+0000","last_unstale":"2023-10-08T01:12:02.867962+0000","last_undegraded":"2023-10-08T01:12:02.867962+0000","last_fullsized":"2023-10-08T01:12:02.867962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.847987+0000","last_change":"2023-10-08T01:12:02.841062+0000","last_active":"2023-10-08T01:12:02.847987+0000","last_peered":"2023-10-08T01:12:02.847987+0000","last_clean":"2023-10-08T01:12:02.847987+0000","last_became_active":"2023-10-08T01:12:02.840782+0000","last_became_peered":"2023-10-08T01:12:02.840782+0000","last_unstale":"2023-10-08T01:12:02.847987+0000","last_undegraded":"2023-10-08T01:12:02.847987+0000","last_fullsized":"2023-10-08T01:12:02.847987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:21.885843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.864739+0000","last_change":"2023-10-08T01:12:01.830454+0000","last_active":"2023-10-08T01:12:02.864739+0000","last_peered":"2023-10-08T01:12:02.864739+0000","last_clean":"2023-10-08T01:12:02.864739+0000","last_became_active":"2023-10-08T01:12:01.830184+0000","last_became_peered":"2023-10-08T01:12:01.830184+0000","last_unstale":"2023-10-08T01:12:02.864739+0000","last_undegraded":"2023-10-08T01:12:02.864739+0000","last_fullsized":"2023-10-08T01:12:02.864739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:47.737308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845961+0000","last_change":"2023-10-08T01:12:02.840968+0000","last_active":"2023-10-08T01:12:02.845961+0000","last_peered":"2023-10-08T01:12:02.845961+0000","last_clean":"2023-10-08T01:12:02.845961+0000","last_became_active":"2023-10-08T01:12:02.840717+0000","last_became_peered":"2023-10-08T01:12:02.840717+0000","last_unstale":"2023-10-08T01:12:02.845961+0000","last_undegraded":"2023-10-08T01:12:02.845961+0000","last_fullsized":"2023-10-08T01:12:02.845961+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:39.174760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867409+0000","last_change":"2023-10-08T01:12:01.829452+0000","last_active":"2023-10-08T01:12:02.867409+0000","last_peered":"2023-10-08T01:12:02.867409+0000","last_clean":"2023-10-08T01:12:02.867409+0000","last_became_active":"2023-10-08T01:12:01.829140+0000","last_became_peered":"2023-10-08T01:12:01.829140+0000","last_unstale":"2023-10-08T01:12:02.867409+0000","last_undegraded":"2023-10-08T01:12:02.867409+0000","last_fullsized":"2023-10-08T01:12:02.867409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:51.755961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843809+0000","last_change":"2023-10-08T01:12:02.840709+0000","last_active":"2023-10-08T01:12:02.843809+0000","last_peered":"2023-10-08T01:12:02.843809+0000","last_clean":"2023-10-08T01:12:02.843809+0000","last_became_active":"2023-10-08T01:12:02.840250+0000","last_became_peered":"2023-10-08T01:12:02.840250+0000","last_unstale":"2023-10-08T01:12:02.843809+0000","last_undegraded":"2023-10-08T01:12:02.843809+0000","last_fullsized":"2023-10-08T01:12:02.843809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:26.311091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.831042+0000","last_change":"2023-10-08T01:12:01.821491+0000","last_active":"2023-10-08T01:12:02.831042+0000","last_peered":"2023-10-08T01:12:02.831042+0000","last_clean":"2023-10-08T01:12:02.831042+0000","last_became_active":"2023-10-08T01:12:01.821135+0000","last_became_peered":"2023-10-08T01:12:01.821135+0000","last_unstale":"2023-10-08T01:12:02.831042+0000","last_undegraded":"2023-10-08T01:12:02.831042+0000","last_fullsized":"2023-10-08T01:12:02.831042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:55.795619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.838339+0000","last_change":"2023-10-08T01:12:02.836913+0000","last_active":"2023-10-08T01:12:02.838339+0000","last_peered":"2023-10-08T01:12:02.838339+0000","last_clean":"2023-10-08T01:12:02.838339+0000","last_became_active":"2023-10-08T01:12:02.836618+0000","last_became_peered":"2023-10-08T01:12:02.836618+0000","last_unstale":"2023-10-08T01:12:02.838339+0000","last_undegraded":"2023-10-08T01:12:02.838339+0000","last_fullsized":"2023-10-08T01:12:02.838339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:05.095868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850521+0000","last_change":"2023-10-08T01:12:01.831714+0000","last_active":"2023-10-08T01:12:02.850521+0000","last_peered":"2023-10-08T01:12:02.850521+0000","last_clean":"2023-10-08T01:12:02.850521+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850521+0000","last_undegraded":"2023-10-08T01:12:02.850521+0000","last_fullsized":"2023-10-08T01:12:02.850521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:56.535520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.862945+0000","last_change":"2023-10-08T01:12:02.862541+0000","last_active":"2023-10-08T01:12:02.862945+0000","last_peered":"2023-10-08T01:12:02.862945+0000","last_clean":"2023-10-08T01:12:02.862945+0000","last_became_active":"2023-10-08T01:12:02.862076+0000","last_became_peered":"2023-10-08T01:12:02.862076+0000","last_unstale":"2023-10-08T01:12:02.862945+0000","last_undegraded":"2023-10-08T01:12:02.862945+0000","last_fullsized":"2023-10-08T01:12:02.862945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:26.919061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.833887+0000","last_change":"2023-10-08T01:12:01.827944+0000","last_active":"2023-10-08T01:12:02.833887+0000","last_peered":"2023-10-08T01:12:02.833887+0000","last_clean":"2023-10-08T01:12:02.833887+0000","last_became_active":"2023-10-08T01:12:01.827116+0000","last_became_peered":"2023-10-08T01:12:01.827116+0000","last_unstale":"2023-10-08T01:12:02.833887+0000","last_undegraded":"2023-10-08T01:12:02.833887+0000","last_fullsized":"2023-10-08T01:12:02.833887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:51.723580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852339+0000","last_change":"2023-10-08T01:12:02.849161+0000","last_active":"2023-10-08T01:12:02.852339+0000","last_peered":"2023-10-08T01:12:02.852339+0000","last_clean":"2023-10-08T01:12:02.852339+0000","last_became_active":"2023-10-08T01:12:02.848913+0000","last_became_peered":"2023-10-08T01:12:02.848913+0000","last_unstale":"2023-10-08T01:12:02.852339+0000","last_undegraded":"2023-10-08T01:12:02.852339+0000","last_fullsized":"2023-10-08T01:12:02.852339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:58:40.796959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.832971+0000","last_change":"2023-10-08T01:12:01.819602+0000","last_active":"2023-10-08T01:12:02.832971+0000","last_peered":"2023-10-08T01:12:02.832971+0000","last_clean":"2023-10-08T01:12:02.832971+0000","last_became_active":"2023-10-08T01:12:01.819304+0000","last_became_peered":"2023-10-08T01:12:01.819304+0000","last_unstale":"2023-10-08T01:12:02.832971+0000","last_undegraded":"2023-10-08T01:12:02.832971+0000","last_fullsized":"2023-10-08T01:12:02.832971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:55.795619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846380+0000","last_change":"2023-10-08T01:12:02.844295+0000","last_active":"2023-10-08T01:12:02.846380+0000","last_peered":"2023-10-08T01:12:02.846380+0000","last_clean":"2023-10-08T01:12:02.846380+0000","last_became_active":"2023-10-08T01:12:02.844001+0000","last_became_peered":"2023-10-08T01:12:02.844001+0000","last_unstale":"2023-10-08T01:12:02.846380+0000","last_undegraded":"2023-10-08T01:12:02.846380+0000","last_fullsized":"2023-10-08T01:12:02.846380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:39:36.571615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867968+0000","last_change":"2023-10-08T01:12:01.831354+0000","last_active":"2023-10-08T01:12:02.867968+0000","last_peered":"2023-10-08T01:12:02.867968+0000","last_clean":"2023-10-08T01:12:02.867968+0000","last_became_active":"2023-10-08T01:12:01.830964+0000","last_became_peered":"2023-10-08T01:12:01.830964+0000","last_unstale":"2023-10-08T01:12:02.867968+0000","last_undegraded":"2023-10-08T01:12:02.867968+0000","last_fullsized":"2023-10-08T01:12:02.867968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850568+0000","last_change":"2023-10-08T01:12:02.845176+0000","last_active":"2023-10-08T01:12:02.850568+0000","last_peered":"2023-10-08T01:12:02.850568+0000","last_clean":"2023-10-08T01:12:02.850568+0000","last_became_active":"2023-10-08T01:12:02.844833+0000","last_became_peered":"2023-10-08T01:12:02.844833+0000","last_unstale":"2023-10-08T01:12:02.850568+0000","last_undegraded":"2023-10-08T01:12:02.850568+0000","last_fullsized":"2023-10-08T01:12:02.850568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:30.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834922+0000","last_change":"2023-10-08T01:12:01.829454+0000","last_active":"2023-10-08T01:12:02.834922+0000","last_peered":"2023-10-08T01:12:02.834922+0000","last_clean":"2023-10-08T01:12:02.834922+0000","last_became_active":"2023-10-08T01:12:01.829133+0000","last_became_peered":"2023-10-08T01:12:01.829133+0000","last_unstale":"2023-10-08T01:12:02.834922+0000","last_undegraded":"2023-10-08T01:12:02.834922+0000","last_fullsized":"2023-10-08T01:12:02.834922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:45.533589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.865079+0000","last_change":"2023-10-08T01:12:02.862111+0000","last_active":"2023-10-08T01:12:02.865079+0000","last_peered":"2023-10-08T01:12:02.865079+0000","last_clean":"2023-10-08T01:12:02.865079+0000","last_became_active":"2023-10-08T01:12:02.861816+0000","last_became_peered":"2023-10-08T01:12:02.861816+0000","last_unstale":"2023-10-08T01:12:02.865079+0000","last_undegraded":"2023-10-08T01:12:02.865079+0000","last_fullsized":"2023-10-08T01:12:02.865079+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:45:20.356518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828282+0000","last_change":"2023-10-08T01:12:01.823222+0000","last_active":"2023-10-08T01:12:02.828282+0000","last_peered":"2023-10-08T01:12:02.828282+0000","last_clean":"2023-10-08T01:12:02.828282+0000","last_became_active":"2023-10-08T01:12:01.822970+0000","last_became_peered":"2023-10-08T01:12:01.822970+0000","last_unstale":"2023-10-08T01:12:02.828282+0000","last_undegraded":"2023-10-08T01:12:02.828282+0000","last_fullsized":"2023-10-08T01:12:02.828282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:00.437245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853947+0000","last_change":"2023-10-08T01:12:02.851360+0000","last_active":"2023-10-08T01:12:02.853947+0000","last_peered":"2023-10-08T01:12:02.853947+0000","last_clean":"2023-10-08T01:12:02.853947+0000","last_became_active":"2023-10-08T01:12:02.851117+0000","last_became_peered":"2023-10-08T01:12:02.851117+0000","last_unstale":"2023-10-08T01:12:02.853947+0000","last_undegraded":"2023-10-08T01:12:02.853947+0000","last_fullsized":"2023-10-08T01:12:02.853947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:47.054661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834238+0000","last_change":"2023-10-08T01:12:01.815771+0000","last_active":"2023-10-08T01:12:02.834238+0000","last_peered":"2023-10-08T01:12:02.834238+0000","last_clean":"2023-10-08T01:12:02.834238+0000","last_became_active":"2023-10-08T01:12:01.815502+0000","last_became_peered":"2023-10-08T01:12:01.815502+0000","last_unstale":"2023-10-08T01:12:02.834238+0000","last_undegraded":"2023-10-08T01:12:02.834238+0000","last_fullsized":"2023-10-08T01:12:02.834238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843426+0000","last_change":"2023-10-08T01:12:02.838180+0000","last_active":"2023-10-08T01:12:02.843426+0000","last_peered":"2023-10-08T01:12:02.843426+0000","last_clean":"2023-10-08T01:12:02.843426+0000","last_became_active":"2023-10-08T01:12:02.837945+0000","last_became_peered":"2023-10-08T01:12:02.837945+0000","last_unstale":"2023-10-08T01:12:02.843426+0000","last_undegraded":"2023-10-08T01:12:02.843426+0000","last_fullsized":"2023-10-08T01:12:02.843426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:21.100043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.827017+0000","last_change":"2023-10-08T01:12:01.827250+0000","last_active":"2023-10-08T01:12:02.827017+0000","last_peered":"2023-10-08T01:12:02.827017+0000","last_clean":"2023-10-08T01:12:02.827017+0000","last_became_active":"2023-10-08T01:12:01.826935+0000","last_became_peered":"2023-10-08T01:12:01.826935+0000","last_unstale":"2023-10-08T01:12:02.827017+0000","last_undegraded":"2023-10-08T01:12:02.827017+0000","last_fullsized":"2023-10-08T01:12:02.827017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:40.815318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860494+0000","last_change":"2023-10-08T01:12:02.856747+0000","last_active":"2023-10-08T01:12:02.860494+0000","last_peered":"2023-10-08T01:12:02.860494+0000","last_clean":"2023-10-08T01:12:02.860494+0000","last_became_active":"2023-10-08T01:12:02.856476+0000","last_became_peered":"2023-10-08T01:12:02.856476+0000","last_unstale":"2023-10-08T01:12:02.860494+0000","last_undegraded":"2023-10-08T01:12:02.860494+0000","last_fullsized":"2023-10-08T01:12:02.860494+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:00.451849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.833345+0000","last_change":"2023-10-08T01:12:01.826441+0000","last_active":"2023-10-08T01:12:02.833345+0000","last_peered":"2023-10-08T01:12:02.833345+0000","last_clean":"2023-10-08T01:12:02.833345+0000","last_became_active":"2023-10-08T01:12:01.826164+0000","last_became_peered":"2023-10-08T01:12:01.826164+0000","last_unstale":"2023-10-08T01:12:02.833345+0000","last_undegraded":"2023-10-08T01:12:02.833345+0000","last_fullsized":"2023-10-08T01:12:02.833345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:56.535520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843986+0000","last_change":"2023-10-08T01:12:02.839714+0000","last_active":"2023-10-08T01:12:02.843986+0000","last_peered":"2023-10-08T01:12:02.843986+0000","last_clean":"2023-10-08T01:12:02.843986+0000","last_became_active":"2023-10-08T01:12:02.839457+0000","last_became_peered":"2023-10-08T01:12:02.839457+0000","last_unstale":"2023-10-08T01:12:02.843986+0000","last_undegraded":"2023-10-08T01:12:02.843986+0000","last_fullsized":"2023-10-08T01:12:02.843986+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:55.300544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828089+0000","last_change":"2023-10-08T01:12:01.822618+0000","last_active":"2023-10-08T01:12:02.828089+0000","last_peered":"2023-10-08T01:12:02.828089+0000","last_clean":"2023-10-08T01:12:02.828089+0000","last_became_active":"2023-10-08T01:12:01.822354+0000","last_became_peered":"2023-10-08T01:12:01.822354+0000","last_unstale":"2023-10-08T01:12:02.828089+0000","last_undegraded":"2023-10-08T01:12:02.828089+0000","last_fullsized":"2023-10-08T01:12:02.828089+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:25:43.082489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859915+0000","last_change":"2023-10-08T01:12:02.855156+0000","last_active":"2023-10-08T01:12:02.859915+0000","last_peered":"2023-10-08T01:12:02.859915+0000","last_clean":"2023-10-08T01:12:02.859915+0000","last_became_active":"2023-10-08T01:12:02.854809+0000","last_became_peered":"2023-10-08T01:12:02.854809+0000","last_unstale":"2023-10-08T01:12:02.859915+0000","last_undegraded":"2023-10-08T01:12:02.859915+0000","last_fullsized":"2023-10-08T01:12:02.859915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:46.539103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850388+0000","last_change":"2023-10-08T01:12:01.831667+0000","last_active":"2023-10-08T01:12:02.850388+0000","last_peered":"2023-10-08T01:12:02.850388+0000","last_clean":"2023-10-08T01:12:02.850388+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850388+0000","last_undegraded":"2023-10-08T01:12:02.850388+0000","last_fullsized":"2023-10-08T01:12:02.850388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:23.287500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852439+0000","last_change":"2023-10-08T01:12:02.849116+0000","last_active":"2023-10-08T01:12:02.852439+0000","last_peered":"2023-10-08T01:12:02.852439+0000","last_clean":"2023-10-08T01:12:02.852439+0000","last_became_active":"2023-10-08T01:12:02.848833+0000","last_became_peered":"2023-10-08T01:12:02.848833+0000","last_unstale":"2023-10-08T01:12:02.852439+0000","last_undegraded":"2023-10-08T01:12:02.852439+0000","last_fullsized":"2023-10-08T01:12:02.852439+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:35.798359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.870650+0000","last_change":"2023-10-08T01:12:01.828087+0000","last_active":"2023-10-08T01:12:02.870650+0000","last_peered":"2023-10-08T01:12:02.870650+0000","last_clean":"2023-10-08T01:12:02.870650+0000","last_became_active":"2023-10-08T01:12:01.827688+0000","last_became_peered":"2023-10-08T01:12:01.827688+0000","last_unstale":"2023-10-08T01:12:02.870650+0000","last_undegraded":"2023-10-08T01:12:02.870650+0000","last_fullsized":"2023-10-08T01:12:02.870650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:25.305577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"80'183","reported_seq":460,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.833196+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:12:02.833196+0000","last_peered":"2023-10-08T01:12:02.833196+0000","last_clean":"2023-10-08T01:12:02.833196+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:12:02.833196+0000","last_undegraded":"2023-10-08T01:12:02.833196+0000","last_fullsized":"2023-10-08T01:12:02.833196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1712128,"data_stored":1127436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.63}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.65600000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.71399999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.5}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.73799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1118208,"data_stored":536572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.497},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.74199999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.83699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1130496,"data_stored":538604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.504},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.433}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44900000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.747},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303048,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068792,"statfs":{"total":96636764160,"available":96326443008,"internally_reserved":0,"allocated":1703936,"data_stored":1126848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50800000000000001,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.617},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.5289999999999999},"last":0.754}]},{"osd":6,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.208,"5min":1.361,"15min":1.3779999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1719999999999999,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.65000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306576,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065264,"statfs":{"total":96636764160,"available":96322830336,"internally_reserved":0,"allocated":1122304,"data_stored":538016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306572,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065268,"statfs":{"total":96636764160,"available":96322834432,"internally_reserved":0,"allocated":1118208,"data_stored":536592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.69999999999999996,"15min":0.80100000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.57399999999999995,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.378}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61699999999999999,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.82999999999999996,"15min":0.88100000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.79400000000000004,"15min":1.1060000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":1.095,"15min":1.3029999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74299999999999999,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.117,"15min":1.413},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.77800000000000002},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.0920000000000001,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":1.089},"last":0.61899999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.379},"last":0.751},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.486},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97099999999999997,"5min":1.0840000000000001,"15min":1.389},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.79700000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.5},"max":{"1min":0.93700000000000006,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.65500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310664,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061176,"statfs":{"total":96636764160,"available":96318644224,"internally_reserved":0,"allocated":1114112,"data_stored":536550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.88100000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.75},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.81299999999999994}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.79800000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.72399999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.63}]}]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":536570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.55900000000000005}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.495},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.58999999999999997}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.623},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.71999999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.874},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.77700000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:12:12.012 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:12:12.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.203+0000 7f9f4d19d700 1 Processor -- start 2023-10-08T01:12:12.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.203+0000 7f9f4d19d700 1 -- start start 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.204+0000 7f9f4d19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9f481595a0 0x7f9f480b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.204+0000 7f9f4d19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f480b1410 con 0x7f9f480c6c60 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.204+0000 7f9f4d19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f48063230 con 0x7f9f480c8830 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.204+0000 7f9f4d19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f480b2b90 con 0x7f9f481595a0 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f477fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9f480c6c60 0x7f9f480c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52250/0 (socket says 172.21.15.119:52250) 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f477fe700 1 -- 172.21.15.119:0/3090517429 learned_addr learned my addr 172.21.15.119:0/3090517429 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f47fff700 1 --2- 172.21.15.119:0/3090517429 >> v2:172.21.15.184:3300/0 conn(0x7f9f481595a0 0x7f9f480b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f46ffd700 1 -- 172.21.15.119:0/3090517429 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599674238 0 0) 0x7f9f480b1410 con 0x7f9f480c6c60 2023-10-08T01:12:12.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f46ffd700 1 -- 172.21.15.119:0/3090517429 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f30003570 con 0x7f9f480c6c60 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f46ffd700 1 -- 172.21.15.119:0/3090517429 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829776216 0 0) 0x7f9f48063230 con 0x7f9f480c8830 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f46ffd700 1 -- 172.21.15.119:0/3090517429 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f480b1410 con 0x7f9f480c8830 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f47fff700 1 -- 172.21.15.119:0/3090517429 >> v1:172.21.15.184:6789/0 conn(0x7f9f480c8830 legacy=0x7f9f48159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f47fff700 1 -- 172.21.15.119:0/3090517429 >> v1:172.21.15.119:6789/0 conn(0x7f9f480c6c60 legacy=0x7f9f480c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.205+0000 7f9f47fff700 1 -- 172.21.15.119:0/3090517429 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480634f0 con 0x7f9f481595a0 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f47fff700 1 --2- 172.21.15.119:0/3090517429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f481595a0 0x7f9f480b18a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9f380098e0 tx=0x7f9f380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=723a2c6b498d96fd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f46ffd700 1 -- 172.21.15.119:0/3090517429 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38011630 con 0x7f9f481595a0 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f46ffd700 1 -- 172.21.15.119:0/3090517429 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f38011790 con 0x7f9f481595a0 2023-10-08T01:12:12.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f46ffd700 1 -- 172.21.15.119:0/3090517429 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38011b30 con 0x7f9f481595a0 2023-10-08T01:12:12.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f4d19d700 1 -- 172.21.15.119:0/3090517429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f481595a0 msgr2=0x7f9f480b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f4d19d700 1 --2- 172.21.15.119:0/3090517429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f481595a0 0x7f9f480b18a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f9f380098e0 tx=0x7f9f380091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f4d19d700 1 -- 172.21.15.119:0/3090517429 shutdown_connections 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f4d19d700 1 --2- 172.21.15.119:0/3090517429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f481595a0 0x7f9f480b18a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f4d19d700 1 -- 172.21.15.119:0/3090517429 >> 172.21.15.119:0/3090517429 conn(0x7f9f4806ddb0 msgr2=0x7f9f4806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.206+0000 7f9f4d19d700 1 -- 172.21.15.119:0/3090517429 shutdown_connections 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.207+0000 7f9f4d19d700 1 -- 172.21.15.119:0/3090517429 wait complete. 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.208+0000 7f9f4d19d700 1 Processor -- start 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.208+0000 7f9f4d19d700 1 -- start start 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f4d19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f480c6c60 0x7f9f48147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f4d19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f480c8830 0x7f9f4816a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f4d19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f481595a0 0x7f9f4815aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f4d19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815d5b0 con 0x7f9f480c6c60 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f4d19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815c9a0 con 0x7f9f480c8830 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f4d19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4815cc80 con 0x7f9f481595a0 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f47fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f480c6c60 0x7f9f48147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f47fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f480c6c60 0x7f9f48147470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55202/0 (socket says 172.21.15.119:55202) 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f47fff700 1 -- 172.21.15.119:0/906441595 learned_addr learned my addr 172.21.15.119:0/906441595 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f477fe700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f480c8830 0x7f9f4816a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.209+0000 7f9f4c99c700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f481595a0 0x7f9f4815aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f47fff700 1 -- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f481595a0 msgr2=0x7f9f4815aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f47fff700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f481595a0 0x7f9f4815aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f47fff700 1 -- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f480c8830 msgr2=0x7f9f4816a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f47fff700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f480c8830 0x7f9f4816a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f47fff700 1 -- 172.21.15.119:0/906441595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38009420 con 0x7f9f480c6c60 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f4c99c700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f481595a0 0x7f9f4815aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f47fff700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f480c6c60 0x7f9f48147470 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9f48064340 tx=0x7f9f38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f2ffff700 1 -- 172.21.15.119:0/906441595 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3801f070 con 0x7f9f480c6c60 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f2ffff700 1 -- 172.21.15.119:0/906441595 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f38000bc0 con 0x7f9f480c6c60 2023-10-08T01:12:12.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4815cf60 con 0x7f9f480c6c60 2023-10-08T01:12:12.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f2ffff700 1 -- 172.21.15.119:0/906441595 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f38025420 con 0x7f9f480c6c60 2023-10-08T01:12:12.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.210+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4815c670 con 0x7f9f480c6c60 2023-10-08T01:12:12.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.211+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4804c3c0 con 0x7f9f480c6c60 2023-10-08T01:12:12.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.212+0000 7f9f2ffff700 1 -- 172.21.15.119:0/906441595 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f38023070 con 0x7f9f480c6c60 2023-10-08T01:12:12.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.213+0000 7f9f2ffff700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f34066f30 0x7f9f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.213+0000 7f9f2ffff700 1 -- 172.21.15.119:0/906441595 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f9f380859c0 con 0x7f9f480c6c60 2023-10-08T01:12:12.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.213+0000 7f9f477fe700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f34066f30 0x7f9f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.214+0000 7f9f477fe700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f34066f30 0x7f9f340693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007fd0 tx=0x7f9f3c009040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:12.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.217+0000 7f9f2ffff700 1 -- 172.21.15.119:0/906441595 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f38052a00 con 0x7f9f480c6c60 2023-10-08T01:12:12.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.403+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f4815d6f0 con 0x7f9f34066f30 2023-10-08T01:12:12.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.412+0000 7f9f2ffff700 1 -- 172.21.15.119:0/906441595 <== mgr.6516 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289236 (secure 0 0 0) 0x7f9f3c00f680 con 0x7f9f34066f30 2023-10-08T01:12:12.416 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.416+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f34066f30 msgr2=0x7f9f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.416+0000 7f9f4d19d700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f34066f30 0x7f9f340693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007fd0 tx=0x7f9f3c009040 comp rx=0 tx=0).stop 2023-10-08T01:12:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.416+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f480c6c60 msgr2=0x7f9f48147470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.416+0000 7f9f4d19d700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f480c6c60 0x7f9f48147470 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f9f48064340 tx=0x7f9f38017000 comp rx=0 tx=0).stop 2023-10-08T01:12:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.417+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 shutdown_connections 2023-10-08T01:12:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.417+0000 7f9f4d19d700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f34066f30 0x7f9f340693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.417+0000 7f9f4d19d700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f481595a0 0x7f9f4815aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.417+0000 7f9f4d19d700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f480c6c60 0x7f9f48147470 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.417+0000 7f9f4d19d700 1 --2- 172.21.15.119:0/906441595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f480c8830 0x7f9f4816a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.417+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 >> 172.21.15.119:0/906441595 conn(0x7f9f4806ddb0 msgr2=0x7f9f480c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:12.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.417+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 shutdown_connections 2023-10-08T01:12:12.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.417+0000 7f9f4d19d700 1 -- 172.21.15.119:0/906441595 wait complete. 2023-10-08T01:12:12.419 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:12:12.434 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-08T01:12:10.708037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":323,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2437352,"kb_used_data":9896,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752537368,"statfs":{"total":773094113280,"available":770598264832,"internally_reserved":0,"allocated":10133504,"data_stored":5477188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001578"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852804+0000","last_change":"2023-10-08T01:12:02.847769+0000","last_active":"2023-10-08T01:12:02.852804+0000","last_peered":"2023-10-08T01:12:02.852804+0000","last_clean":"2023-10-08T01:12:02.852804+0000","last_became_active":"2023-10-08T01:12:02.847457+0000","last_became_peered":"2023-10-08T01:12:02.847457+0000","last_unstale":"2023-10-08T01:12:02.852804+0000","last_undegraded":"2023-10-08T01:12:02.852804+0000","last_fullsized":"2023-10-08T01:12:02.852804+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:27.793380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866793+0000","last_change":"2023-10-08T01:12:02.861488+0000","last_active":"2023-10-08T01:12:02.866793+0000","last_peered":"2023-10-08T01:12:02.866793+0000","last_clean":"2023-10-08T01:12:02.866793+0000","last_became_active":"2023-10-08T01:12:02.860018+0000","last_became_peered":"2023-10-08T01:12:02.860018+0000","last_unstale":"2023-10-08T01:12:02.866793+0000","last_undegraded":"2023-10-08T01:12:02.866793+0000","last_fullsized":"2023-10-08T01:12:02.866793+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.163695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846099+0000","last_change":"2023-10-08T01:12:02.841414+0000","last_active":"2023-10-08T01:12:02.846099+0000","last_peered":"2023-10-08T01:12:02.846099+0000","last_clean":"2023-10-08T01:12:02.846099+0000","last_became_active":"2023-10-08T01:12:02.841091+0000","last_became_peered":"2023-10-08T01:12:02.841091+0000","last_unstale":"2023-10-08T01:12:02.846099+0000","last_undegraded":"2023-10-08T01:12:02.846099+0000","last_fullsized":"2023-10-08T01:12:02.846099+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:42:03.550299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.862643+0000","last_change":"2023-10-08T01:12:02.860263+0000","last_active":"2023-10-08T01:12:02.862643+0000","last_peered":"2023-10-08T01:12:02.862643+0000","last_clean":"2023-10-08T01:12:02.862643+0000","last_became_active":"2023-10-08T01:12:02.860067+0000","last_became_peered":"2023-10-08T01:12:02.860067+0000","last_unstale":"2023-10-08T01:12:02.862643+0000","last_undegraded":"2023-10-08T01:12:02.862643+0000","last_fullsized":"2023-10-08T01:12:02.862643+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:18.546972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845916+0000","last_change":"2023-10-08T01:12:02.841353+0000","last_active":"2023-10-08T01:12:02.845916+0000","last_peered":"2023-10-08T01:12:02.845916+0000","last_clean":"2023-10-08T01:12:02.845916+0000","last_became_active":"2023-10-08T01:12:02.841117+0000","last_became_peered":"2023-10-08T01:12:02.841117+0000","last_unstale":"2023-10-08T01:12:02.845916+0000","last_undegraded":"2023-10-08T01:12:02.845916+0000","last_fullsized":"2023-10-08T01:12:02.845916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:32.447875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859766+0000","last_change":"2023-10-08T01:12:02.851025+0000","last_active":"2023-10-08T01:12:02.859766+0000","last_peered":"2023-10-08T01:12:02.859766+0000","last_clean":"2023-10-08T01:12:02.859766+0000","last_became_active":"2023-10-08T01:12:02.850765+0000","last_became_peered":"2023-10-08T01:12:02.850765+0000","last_unstale":"2023-10-08T01:12:02.859766+0000","last_undegraded":"2023-10-08T01:12:02.859766+0000","last_fullsized":"2023-10-08T01:12:02.859766+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:10:09.116366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850784+0000","last_change":"2023-10-08T01:12:02.838614+0000","last_active":"2023-10-08T01:12:02.850784+0000","last_peered":"2023-10-08T01:12:02.850784+0000","last_clean":"2023-10-08T01:12:02.850784+0000","last_became_active":"2023-10-08T01:12:02.838296+0000","last_became_peered":"2023-10-08T01:12:02.838296+0000","last_unstale":"2023-10-08T01:12:02.850784+0000","last_undegraded":"2023-10-08T01:12:02.850784+0000","last_fullsized":"2023-10-08T01:12:02.850784+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:52.932416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860705+0000","last_change":"2023-10-08T01:12:02.845674+0000","last_active":"2023-10-08T01:12:02.860705+0000","last_peered":"2023-10-08T01:12:02.860705+0000","last_clean":"2023-10-08T01:12:02.860705+0000","last_became_active":"2023-10-08T01:12:02.845358+0000","last_became_peered":"2023-10-08T01:12:02.845358+0000","last_unstale":"2023-10-08T01:12:02.860705+0000","last_undegraded":"2023-10-08T01:12:02.860705+0000","last_fullsized":"2023-10-08T01:12:02.860705+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:18:47.976571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844181+0000","last_change":"2023-10-08T01:12:02.839332+0000","last_active":"2023-10-08T01:12:02.844181+0000","last_peered":"2023-10-08T01:12:02.844181+0000","last_clean":"2023-10-08T01:12:02.844181+0000","last_became_active":"2023-10-08T01:12:02.838641+0000","last_became_peered":"2023-10-08T01:12:02.838641+0000","last_unstale":"2023-10-08T01:12:02.844181+0000","last_undegraded":"2023-10-08T01:12:02.844181+0000","last_fullsized":"2023-10-08T01:12:02.844181+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:24.293015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.842677+0000","last_change":"2023-10-08T01:12:02.836233+0000","last_active":"2023-10-08T01:12:02.842677+0000","last_peered":"2023-10-08T01:12:02.842677+0000","last_clean":"2023-10-08T01:12:02.842677+0000","last_became_active":"2023-10-08T01:12:02.835917+0000","last_became_peered":"2023-10-08T01:12:02.835917+0000","last_unstale":"2023-10-08T01:12:02.842677+0000","last_undegraded":"2023-10-08T01:12:02.842677+0000","last_fullsized":"2023-10-08T01:12:02.842677+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:45:21.312491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.848016+0000","last_change":"2023-10-08T01:12:02.842004+0000","last_active":"2023-10-08T01:12:02.848016+0000","last_peered":"2023-10-08T01:12:02.848016+0000","last_clean":"2023-10-08T01:12:02.848016+0000","last_became_active":"2023-10-08T01:12:02.841401+0000","last_became_peered":"2023-10-08T01:12:02.841401+0000","last_unstale":"2023-10-08T01:12:02.848016+0000","last_undegraded":"2023-10-08T01:12:02.848016+0000","last_fullsized":"2023-10-08T01:12:02.848016+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:19:33.997729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845982+0000","last_change":"2023-10-08T01:12:02.841051+0000","last_active":"2023-10-08T01:12:02.845982+0000","last_peered":"2023-10-08T01:12:02.845982+0000","last_clean":"2023-10-08T01:12:02.845982+0000","last_became_active":"2023-10-08T01:12:02.840797+0000","last_became_peered":"2023-10-08T01:12:02.840797+0000","last_unstale":"2023-10-08T01:12:02.845982+0000","last_undegraded":"2023-10-08T01:12:02.845982+0000","last_fullsized":"2023-10-08T01:12:02.845982+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:25:04.066905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851974+0000","last_change":"2023-10-08T01:12:02.849866+0000","last_active":"2023-10-08T01:12:02.851974+0000","last_peered":"2023-10-08T01:12:02.851974+0000","last_clean":"2023-10-08T01:12:02.851974+0000","last_became_active":"2023-10-08T01:12:02.849568+0000","last_became_peered":"2023-10-08T01:12:02.849568+0000","last_unstale":"2023-10-08T01:12:02.851974+0000","last_undegraded":"2023-10-08T01:12:02.851974+0000","last_fullsized":"2023-10-08T01:12:02.851974+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:30.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846415+0000","last_change":"2023-10-08T01:12:02.843295+0000","last_active":"2023-10-08T01:12:02.846415+0000","last_peered":"2023-10-08T01:12:02.846415+0000","last_clean":"2023-10-08T01:12:02.846415+0000","last_became_active":"2023-10-08T01:12:02.842892+0000","last_became_peered":"2023-10-08T01:12:02.842892+0000","last_unstale":"2023-10-08T01:12:02.846415+0000","last_undegraded":"2023-10-08T01:12:02.846415+0000","last_fullsized":"2023-10-08T01:12:02.846415+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:41:20.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851263+0000","last_change":"2023-10-08T01:12:02.847055+0000","last_active":"2023-10-08T01:12:02.851263+0000","last_peered":"2023-10-08T01:12:02.851263+0000","last_clean":"2023-10-08T01:12:02.851263+0000","last_became_active":"2023-10-08T01:12:02.846767+0000","last_became_peered":"2023-10-08T01:12:02.846767+0000","last_unstale":"2023-10-08T01:12:02.851263+0000","last_undegraded":"2023-10-08T01:12:02.851263+0000","last_fullsized":"2023-10-08T01:12:02.851263+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:52.932416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866121+0000","last_change":"2023-10-08T01:12:02.864825+0000","last_active":"2023-10-08T01:12:02.866121+0000","last_peered":"2023-10-08T01:12:02.866121+0000","last_clean":"2023-10-08T01:12:02.866121+0000","last_became_active":"2023-10-08T01:12:02.864505+0000","last_became_peered":"2023-10-08T01:12:02.864505+0000","last_unstale":"2023-10-08T01:12:02.866121+0000","last_undegraded":"2023-10-08T01:12:02.866121+0000","last_fullsized":"2023-10-08T01:12:02.866121+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:56:06.717153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844213+0000","last_change":"2023-10-08T01:12:02.840294+0000","last_active":"2023-10-08T01:12:02.844213+0000","last_peered":"2023-10-08T01:12:02.844213+0000","last_clean":"2023-10-08T01:12:02.844213+0000","last_became_active":"2023-10-08T01:12:02.838656+0000","last_became_peered":"2023-10-08T01:12:02.838656+0000","last_unstale":"2023-10-08T01:12:02.844213+0000","last_undegraded":"2023-10-08T01:12:02.844213+0000","last_fullsized":"2023-10-08T01:12:02.844213+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:57.541034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867850+0000","last_change":"2023-10-08T01:12:02.863797+0000","last_active":"2023-10-08T01:12:02.867850+0000","last_peered":"2023-10-08T01:12:02.867850+0000","last_clean":"2023-10-08T01:12:02.867850+0000","last_became_active":"2023-10-08T01:12:02.863597+0000","last_became_peered":"2023-10-08T01:12:02.863597+0000","last_unstale":"2023-10-08T01:12:02.867850+0000","last_undegraded":"2023-10-08T01:12:02.867850+0000","last_fullsized":"2023-10-08T01:12:02.867850+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:59.520669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.866715+0000","last_change":"2023-10-08T01:12:02.860578+0000","last_active":"2023-10-08T01:12:02.866715+0000","last_peered":"2023-10-08T01:12:02.866715+0000","last_clean":"2023-10-08T01:12:02.866715+0000","last_became_active":"2023-10-08T01:12:02.859906+0000","last_became_peered":"2023-10-08T01:12:02.859906+0000","last_unstale":"2023-10-08T01:12:02.866715+0000","last_undegraded":"2023-10-08T01:12:02.866715+0000","last_fullsized":"2023-10-08T01:12:02.866715+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:48.742822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853436+0000","last_change":"2023-10-08T01:12:02.851814+0000","last_active":"2023-10-08T01:12:02.853436+0000","last_peered":"2023-10-08T01:12:02.853436+0000","last_clean":"2023-10-08T01:12:02.853436+0000","last_became_active":"2023-10-08T01:12:02.851488+0000","last_became_peered":"2023-10-08T01:12:02.851488+0000","last_unstale":"2023-10-08T01:12:02.853436+0000","last_undegraded":"2023-10-08T01:12:02.853436+0000","last_fullsized":"2023-10-08T01:12:02.853436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:37.330422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846971+0000","last_change":"2023-10-08T01:12:02.843829+0000","last_active":"2023-10-08T01:12:02.846971+0000","last_peered":"2023-10-08T01:12:02.846971+0000","last_clean":"2023-10-08T01:12:02.846971+0000","last_became_active":"2023-10-08T01:12:02.843570+0000","last_became_peered":"2023-10-08T01:12:02.843570+0000","last_unstale":"2023-10-08T01:12:02.846971+0000","last_undegraded":"2023-10-08T01:12:02.846971+0000","last_fullsized":"2023-10-08T01:12:02.846971+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:25.168516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.837237+0000","last_change":"2023-10-08T01:12:02.835029+0000","last_active":"2023-10-08T01:12:02.837237+0000","last_peered":"2023-10-08T01:12:02.837237+0000","last_clean":"2023-10-08T01:12:02.837237+0000","last_became_active":"2023-10-08T01:12:02.834601+0000","last_became_peered":"2023-10-08T01:12:02.834601+0000","last_unstale":"2023-10-08T01:12:02.837237+0000","last_undegraded":"2023-10-08T01:12:02.837237+0000","last_fullsized":"2023-10-08T01:12:02.837237+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:29.210434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853918+0000","last_change":"2023-10-08T01:12:02.852791+0000","last_active":"2023-10-08T01:12:02.853918+0000","last_peered":"2023-10-08T01:12:02.853918+0000","last_clean":"2023-10-08T01:12:02.853918+0000","last_became_active":"2023-10-08T01:12:02.852501+0000","last_became_peered":"2023-10-08T01:12:02.852501+0000","last_unstale":"2023-10-08T01:12:02.853918+0000","last_undegraded":"2023-10-08T01:12:02.853918+0000","last_fullsized":"2023-10-08T01:12:02.853918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:25:25.157341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.865134+0000","last_change":"2023-10-08T01:12:02.862014+0000","last_active":"2023-10-08T01:12:02.865134+0000","last_peered":"2023-10-08T01:12:02.865134+0000","last_clean":"2023-10-08T01:12:02.865134+0000","last_became_active":"2023-10-08T01:12:02.861727+0000","last_became_peered":"2023-10-08T01:12:02.861727+0000","last_unstale":"2023-10-08T01:12:02.865134+0000","last_undegraded":"2023-10-08T01:12:02.865134+0000","last_fullsized":"2023-10-08T01:12:02.865134+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:32:11.334178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.837294+0000","last_change":"2023-10-08T01:12:02.835365+0000","last_active":"2023-10-08T01:12:02.837294+0000","last_peered":"2023-10-08T01:12:02.837294+0000","last_clean":"2023-10-08T01:12:02.837294+0000","last_became_active":"2023-10-08T01:12:02.835097+0000","last_became_peered":"2023-10-08T01:12:02.835097+0000","last_unstale":"2023-10-08T01:12:02.837294+0000","last_undegraded":"2023-10-08T01:12:02.837294+0000","last_fullsized":"2023-10-08T01:12:02.837294+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:48.481704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.865035+0000","last_change":"2023-10-08T01:12:02.863933+0000","last_active":"2023-10-08T01:12:02.865035+0000","last_peered":"2023-10-08T01:12:02.865035+0000","last_clean":"2023-10-08T01:12:02.865035+0000","last_became_active":"2023-10-08T01:12:02.863612+0000","last_became_peered":"2023-10-08T01:12:02.863612+0000","last_unstale":"2023-10-08T01:12:02.865035+0000","last_undegraded":"2023-10-08T01:12:02.865035+0000","last_fullsized":"2023-10-08T01:12:02.865035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:39:36.571615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846116+0000","last_change":"2023-10-08T01:12:02.845033+0000","last_active":"2023-10-08T01:12:02.846116+0000","last_peered":"2023-10-08T01:12:02.846116+0000","last_clean":"2023-10-08T01:12:02.846116+0000","last_became_active":"2023-10-08T01:12:02.844545+0000","last_became_peered":"2023-10-08T01:12:02.844545+0000","last_unstale":"2023-10-08T01:12:02.846116+0000","last_undegraded":"2023-10-08T01:12:02.846116+0000","last_fullsized":"2023-10-08T01:12:02.846116+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:36.576301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.842633+0000","last_change":"2023-10-08T01:12:02.840449+0000","last_active":"2023-10-08T01:12:02.842633+0000","last_peered":"2023-10-08T01:12:02.842633+0000","last_clean":"2023-10-08T01:12:02.842633+0000","last_became_active":"2023-10-08T01:12:02.840167+0000","last_became_peered":"2023-10-08T01:12:02.840167+0000","last_unstale":"2023-10-08T01:12:02.842633+0000","last_undegraded":"2023-10-08T01:12:02.842633+0000","last_fullsized":"2023-10-08T01:12:02.842633+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:36:37.500845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845074+0000","last_change":"2023-10-08T01:12:02.839789+0000","last_active":"2023-10-08T01:12:02.845074+0000","last_peered":"2023-10-08T01:12:02.845074+0000","last_clean":"2023-10-08T01:12:02.845074+0000","last_became_active":"2023-10-08T01:12:02.839516+0000","last_became_peered":"2023-10-08T01:12:02.839516+0000","last_unstale":"2023-10-08T01:12:02.845074+0000","last_undegraded":"2023-10-08T01:12:02.845074+0000","last_fullsized":"2023-10-08T01:12:02.845074+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:42.982688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856916+0000","last_change":"2023-10-08T01:12:02.855973+0000","last_active":"2023-10-08T01:12:02.856916+0000","last_peered":"2023-10-08T01:12:02.856916+0000","last_clean":"2023-10-08T01:12:02.856916+0000","last_became_active":"2023-10-08T01:12:02.855712+0000","last_became_peered":"2023-10-08T01:12:02.855712+0000","last_unstale":"2023-10-08T01:12:02.856916+0000","last_undegraded":"2023-10-08T01:12:02.856916+0000","last_fullsized":"2023-10-08T01:12:02.856916+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:07:30.307443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850807+0000","last_change":"2023-10-08T01:12:02.842126+0000","last_active":"2023-10-08T01:12:02.850807+0000","last_peered":"2023-10-08T01:12:02.850807+0000","last_clean":"2023-10-08T01:12:02.850807+0000","last_became_active":"2023-10-08T01:12:02.841671+0000","last_became_peered":"2023-10-08T01:12:02.841671+0000","last_unstale":"2023-10-08T01:12:02.850807+0000","last_undegraded":"2023-10-08T01:12:02.850807+0000","last_fullsized":"2023-10-08T01:12:02.850807+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:12:47.987269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860840+0000","last_change":"2023-10-08T01:12:02.854035+0000","last_active":"2023-10-08T01:12:02.860840+0000","last_peered":"2023-10-08T01:12:02.860840+0000","last_clean":"2023-10-08T01:12:02.860840+0000","last_became_active":"2023-10-08T01:12:02.853704+0000","last_became_peered":"2023-10-08T01:12:02.853704+0000","last_unstale":"2023-10-08T01:12:02.860840+0000","last_undegraded":"2023-10-08T01:12:02.860840+0000","last_fullsized":"2023-10-08T01:12:02.860840+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:30:55.363740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.864742+0000","last_change":"2023-10-08T01:12:02.862777+0000","last_active":"2023-10-08T01:12:02.864742+0000","last_peered":"2023-10-08T01:12:02.864742+0000","last_clean":"2023-10-08T01:12:02.864742+0000","last_became_active":"2023-10-08T01:12:02.862503+0000","last_became_peered":"2023-10-08T01:12:02.862503+0000","last_unstale":"2023-10-08T01:12:02.864742+0000","last_undegraded":"2023-10-08T01:12:02.864742+0000","last_fullsized":"2023-10-08T01:12:02.864742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:45.646920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"90'4","reported_seq":14,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.865487+0000","last_change":"2023-10-08T01:12:01.826162+0000","last_active":"2023-10-08T01:12:02.865487+0000","last_peered":"2023-10-08T01:12:02.865487+0000","last_clean":"2023-10-08T01:12:02.865487+0000","last_became_active":"2023-10-08T01:12:01.825717+0000","last_became_peered":"2023-10-08T01:12:01.825717+0000","last_unstale":"2023-10-08T01:12:02.865487+0000","last_undegraded":"2023-10-08T01:12:02.865487+0000","last_fullsized":"2023-10-08T01:12:02.865487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:47.476190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.844198+0000","last_change":"2023-10-08T01:12:02.839586+0000","last_active":"2023-10-08T01:12:02.844198+0000","last_peered":"2023-10-08T01:12:02.844198+0000","last_clean":"2023-10-08T01:12:02.844198+0000","last_became_active":"2023-10-08T01:12:02.839033+0000","last_became_peered":"2023-10-08T01:12:02.839033+0000","last_unstale":"2023-10-08T01:12:02.844198+0000","last_undegraded":"2023-10-08T01:12:02.844198+0000","last_fullsized":"2023-10-08T01:12:02.844198+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:43.348391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856349+0000","last_change":"2023-10-08T01:12:01.818263+0000","last_active":"2023-10-08T01:12:02.856349+0000","last_peered":"2023-10-08T01:12:02.856349+0000","last_clean":"2023-10-08T01:12:02.856349+0000","last_became_active":"2023-10-08T01:12:01.818011+0000","last_became_peered":"2023-10-08T01:12:01.818011+0000","last_unstale":"2023-10-08T01:12:02.856349+0000","last_undegraded":"2023-10-08T01:12:02.856349+0000","last_fullsized":"2023-10-08T01:12:02.856349+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:33:07.200626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860732+0000","last_change":"2023-10-08T01:12:02.857609+0000","last_active":"2023-10-08T01:12:02.860732+0000","last_peered":"2023-10-08T01:12:02.860732+0000","last_clean":"2023-10-08T01:12:02.860732+0000","last_became_active":"2023-10-08T01:12:02.857277+0000","last_became_peered":"2023-10-08T01:12:02.857277+0000","last_unstale":"2023-10-08T01:12:02.860732+0000","last_undegraded":"2023-10-08T01:12:02.860732+0000","last_fullsized":"2023-10-08T01:12:02.860732+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.729095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.863916+0000","last_change":"2023-10-08T01:12:01.831998+0000","last_active":"2023-10-08T01:12:02.863916+0000","last_peered":"2023-10-08T01:12:02.863916+0000","last_clean":"2023-10-08T01:12:02.863916+0000","last_became_active":"2023-10-08T01:12:01.830751+0000","last_became_peered":"2023-10-08T01:12:01.830751+0000","last_unstale":"2023-10-08T01:12:02.863916+0000","last_undegraded":"2023-10-08T01:12:02.863916+0000","last_fullsized":"2023-10-08T01:12:02.863916+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:39:17.541458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852014+0000","last_change":"2023-10-08T01:12:02.846830+0000","last_active":"2023-10-08T01:12:02.852014+0000","last_peered":"2023-10-08T01:12:02.852014+0000","last_clean":"2023-10-08T01:12:02.852014+0000","last_became_active":"2023-10-08T01:12:02.846528+0000","last_became_peered":"2023-10-08T01:12:02.846528+0000","last_unstale":"2023-10-08T01:12:02.852014+0000","last_undegraded":"2023-10-08T01:12:02.852014+0000","last_fullsized":"2023-10-08T01:12:02.852014+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:38.330288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.868186+0000","last_change":"2023-10-08T01:12:01.831258+0000","last_active":"2023-10-08T01:12:02.868186+0000","last_peered":"2023-10-08T01:12:02.868186+0000","last_clean":"2023-10-08T01:12:02.868186+0000","last_became_active":"2023-10-08T01:12:01.830943+0000","last_became_peered":"2023-10-08T01:12:01.830943+0000","last_unstale":"2023-10-08T01:12:02.868186+0000","last_undegraded":"2023-10-08T01:12:02.868186+0000","last_fullsized":"2023-10-08T01:12:02.868186+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:30.598996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850668+0000","last_change":"2023-10-08T01:12:02.839727+0000","last_active":"2023-10-08T01:12:02.850668+0000","last_peered":"2023-10-08T01:12:02.850668+0000","last_clean":"2023-10-08T01:12:02.850668+0000","last_became_active":"2023-10-08T01:12:02.836234+0000","last_became_peered":"2023-10-08T01:12:02.836234+0000","last_unstale":"2023-10-08T01:12:02.850668+0000","last_undegraded":"2023-10-08T01:12:02.850668+0000","last_fullsized":"2023-10-08T01:12:02.850668+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:06:09.924996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853340+0000","last_change":"2023-10-08T01:12:01.826432+0000","last_active":"2023-10-08T01:12:02.853340+0000","last_peered":"2023-10-08T01:12:02.853340+0000","last_clean":"2023-10-08T01:12:02.853340+0000","last_became_active":"2023-10-08T01:12:01.826156+0000","last_became_peered":"2023-10-08T01:12:01.826156+0000","last_unstale":"2023-10-08T01:12:02.853340+0000","last_undegraded":"2023-10-08T01:12:02.853340+0000","last_fullsized":"2023-10-08T01:12:02.853340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:20:26.787866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.837320+0000","last_change":"2023-10-08T01:12:02.835389+0000","last_active":"2023-10-08T01:12:02.837320+0000","last_peered":"2023-10-08T01:12:02.837320+0000","last_clean":"2023-10-08T01:12:02.837320+0000","last_became_active":"2023-10-08T01:12:02.834837+0000","last_became_peered":"2023-10-08T01:12:02.834837+0000","last_unstale":"2023-10-08T01:12:02.837320+0000","last_undegraded":"2023-10-08T01:12:02.837320+0000","last_fullsized":"2023-10-08T01:12:02.837320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:56.801134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867313+0000","last_change":"2023-10-08T01:12:01.833334+0000","last_active":"2023-10-08T01:12:02.867313+0000","last_peered":"2023-10-08T01:12:02.867313+0000","last_clean":"2023-10-08T01:12:02.867313+0000","last_became_active":"2023-10-08T01:12:01.832064+0000","last_became_peered":"2023-10-08T01:12:01.832064+0000","last_unstale":"2023-10-08T01:12:02.867313+0000","last_undegraded":"2023-10-08T01:12:02.867313+0000","last_fullsized":"2023-10-08T01:12:02.867313+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:49.406724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846186+0000","last_change":"2023-10-08T01:12:02.843587+0000","last_active":"2023-10-08T01:12:02.846186+0000","last_peered":"2023-10-08T01:12:02.846186+0000","last_clean":"2023-10-08T01:12:02.846186+0000","last_became_active":"2023-10-08T01:12:02.843202+0000","last_became_peered":"2023-10-08T01:12:02.843202+0000","last_unstale":"2023-10-08T01:12:02.846186+0000","last_undegraded":"2023-10-08T01:12:02.846186+0000","last_fullsized":"2023-10-08T01:12:02.846186+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:32:50.412239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834563+0000","last_change":"2023-10-08T01:12:01.827032+0000","last_active":"2023-10-08T01:12:02.834563+0000","last_peered":"2023-10-08T01:12:02.834563+0000","last_clean":"2023-10-08T01:12:02.834563+0000","last_became_active":"2023-10-08T01:12:01.826774+0000","last_became_peered":"2023-10-08T01:12:01.826774+0000","last_unstale":"2023-10-08T01:12:02.834563+0000","last_undegraded":"2023-10-08T01:12:02.834563+0000","last_fullsized":"2023-10-08T01:12:02.834563+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:25.305577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851277+0000","last_change":"2023-10-08T01:12:02.847761+0000","last_active":"2023-10-08T01:12:02.851277+0000","last_peered":"2023-10-08T01:12:02.851277+0000","last_clean":"2023-10-08T01:12:02.851277+0000","last_became_active":"2023-10-08T01:12:02.847425+0000","last_became_peered":"2023-10-08T01:12:02.847425+0000","last_unstale":"2023-10-08T01:12:02.851277+0000","last_undegraded":"2023-10-08T01:12:02.851277+0000","last_fullsized":"2023-10-08T01:12:02.851277+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:12:47.987269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.826270+0000","last_change":"2023-10-08T01:12:01.822270+0000","last_active":"2023-10-08T01:12:02.826270+0000","last_peered":"2023-10-08T01:12:02.826270+0000","last_clean":"2023-10-08T01:12:02.826270+0000","last_became_active":"2023-10-08T01:12:01.821995+0000","last_became_peered":"2023-10-08T01:12:01.821995+0000","last_unstale":"2023-10-08T01:12:02.826270+0000","last_undegraded":"2023-10-08T01:12:02.826270+0000","last_fullsized":"2023-10-08T01:12:02.826270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:40.166452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850742+0000","last_change":"2023-10-08T01:12:02.843828+0000","last_active":"2023-10-08T01:12:02.850742+0000","last_peered":"2023-10-08T01:12:02.850742+0000","last_clean":"2023-10-08T01:12:02.850742+0000","last_became_active":"2023-10-08T01:12:02.843560+0000","last_became_peered":"2023-10-08T01:12:02.843560+0000","last_unstale":"2023-10-08T01:12:02.850742+0000","last_undegraded":"2023-10-08T01:12:02.850742+0000","last_fullsized":"2023-10-08T01:12:02.850742+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:22:14.871146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828552+0000","last_change":"2023-10-08T01:12:01.821727+0000","last_active":"2023-10-08T01:12:02.828552+0000","last_peered":"2023-10-08T01:12:02.828552+0000","last_clean":"2023-10-08T01:12:02.828552+0000","last_became_active":"2023-10-08T01:12:01.821455+0000","last_became_peered":"2023-10-08T01:12:01.821455+0000","last_unstale":"2023-10-08T01:12:02.828552+0000","last_undegraded":"2023-10-08T01:12:02.828552+0000","last_fullsized":"2023-10-08T01:12:02.828552+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:58.515154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853785+0000","last_change":"2023-10-08T01:12:02.850817+0000","last_active":"2023-10-08T01:12:02.853785+0000","last_peered":"2023-10-08T01:12:02.853785+0000","last_clean":"2023-10-08T01:12:02.853785+0000","last_became_active":"2023-10-08T01:12:02.849008+0000","last_became_peered":"2023-10-08T01:12:02.849008+0000","last_unstale":"2023-10-08T01:12:02.853785+0000","last_undegraded":"2023-10-08T01:12:02.853785+0000","last_fullsized":"2023-10-08T01:12:02.853785+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:09.986293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852304+0000","last_change":"2023-10-08T01:12:01.832041+0000","last_active":"2023-10-08T01:12:02.852304+0000","last_peered":"2023-10-08T01:12:02.852304+0000","last_clean":"2023-10-08T01:12:02.852304+0000","last_became_active":"2023-10-08T01:12:01.831603+0000","last_became_peered":"2023-10-08T01:12:01.831603+0000","last_unstale":"2023-10-08T01:12:02.852304+0000","last_undegraded":"2023-10-08T01:12:02.852304+0000","last_fullsized":"2023-10-08T01:12:02.852304+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:42.342876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860424+0000","last_change":"2023-10-08T01:12:02.855912+0000","last_active":"2023-10-08T01:12:02.860424+0000","last_peered":"2023-10-08T01:12:02.860424+0000","last_clean":"2023-10-08T01:12:02.860424+0000","last_became_active":"2023-10-08T01:12:02.855628+0000","last_became_peered":"2023-10-08T01:12:02.855628+0000","last_unstale":"2023-10-08T01:12:02.860424+0000","last_undegraded":"2023-10-08T01:12:02.860424+0000","last_fullsized":"2023-10-08T01:12:02.860424+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:48.158503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"90'3","reported_seq":13,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.856557+0000","last_change":"2023-10-08T01:12:01.825659+0000","last_active":"2023-10-08T01:12:02.856557+0000","last_peered":"2023-10-08T01:12:02.856557+0000","last_clean":"2023-10-08T01:12:02.856557+0000","last_became_active":"2023-10-08T01:12:01.825371+0000","last_became_peered":"2023-10-08T01:12:01.825371+0000","last_unstale":"2023-10-08T01:12:02.856557+0000","last_undegraded":"2023-10-08T01:12:02.856557+0000","last_fullsized":"2023-10-08T01:12:02.856557+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:46.049146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843348+0000","last_change":"2023-10-08T01:12:02.837217+0000","last_active":"2023-10-08T01:12:02.843348+0000","last_peered":"2023-10-08T01:12:02.843348+0000","last_clean":"2023-10-08T01:12:02.843348+0000","last_became_active":"2023-10-08T01:12:02.836983+0000","last_became_peered":"2023-10-08T01:12:02.836983+0000","last_unstale":"2023-10-08T01:12:02.843348+0000","last_undegraded":"2023-10-08T01:12:02.843348+0000","last_fullsized":"2023-10-08T01:12:02.843348+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:22:31.604511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"90'6","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860024+0000","last_change":"2023-10-08T01:12:01.822637+0000","last_active":"2023-10-08T01:12:02.860024+0000","last_peered":"2023-10-08T01:12:02.860024+0000","last_clean":"2023-10-08T01:12:02.860024+0000","last_became_active":"2023-10-08T01:12:01.822310+0000","last_became_peered":"2023-10-08T01:12:01.822310+0000","last_unstale":"2023-10-08T01:12:02.860024+0000","last_undegraded":"2023-10-08T01:12:02.860024+0000","last_fullsized":"2023-10-08T01:12:02.860024+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:23.287500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852020+0000","last_change":"2023-10-08T01:12:02.847081+0000","last_active":"2023-10-08T01:12:02.852020+0000","last_peered":"2023-10-08T01:12:02.852020+0000","last_clean":"2023-10-08T01:12:02.852020+0000","last_became_active":"2023-10-08T01:12:02.846830+0000","last_became_peered":"2023-10-08T01:12:02.846830+0000","last_unstale":"2023-10-08T01:12:02.852020+0000","last_undegraded":"2023-10-08T01:12:02.852020+0000","last_fullsized":"2023-10-08T01:12:02.852020+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:34:39.597593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860497+0000","last_change":"2023-10-08T01:12:01.824364+0000","last_active":"2023-10-08T01:12:02.860497+0000","last_peered":"2023-10-08T01:12:02.860497+0000","last_clean":"2023-10-08T01:12:02.860497+0000","last_became_active":"2023-10-08T01:12:01.824117+0000","last_became_peered":"2023-10-08T01:12:01.824117+0000","last_unstale":"2023-10-08T01:12:02.860497+0000","last_undegraded":"2023-10-08T01:12:02.860497+0000","last_fullsized":"2023-10-08T01:12:02.860497+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:26:42.342876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850711+0000","last_change":"2023-10-08T01:12:02.841236+0000","last_active":"2023-10-08T01:12:02.850711+0000","last_peered":"2023-10-08T01:12:02.850711+0000","last_clean":"2023-10-08T01:12:02.850711+0000","last_became_active":"2023-10-08T01:12:02.840900+0000","last_became_peered":"2023-10-08T01:12:02.840900+0000","last_unstale":"2023-10-08T01:12:02.850711+0000","last_undegraded":"2023-10-08T01:12:02.850711+0000","last_fullsized":"2023-10-08T01:12:02.850711+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:37.330422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850403+0000","last_change":"2023-10-08T01:12:01.829191+0000","last_active":"2023-10-08T01:12:02.850403+0000","last_peered":"2023-10-08T01:12:02.850403+0000","last_clean":"2023-10-08T01:12:02.850403+0000","last_became_active":"2023-10-08T01:12:01.828873+0000","last_became_peered":"2023-10-08T01:12:01.828873+0000","last_unstale":"2023-10-08T01:12:02.850403+0000","last_undegraded":"2023-10-08T01:12:02.850403+0000","last_fullsized":"2023-10-08T01:12:02.850403+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:31.442360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.851325+0000","last_change":"2023-10-08T01:12:02.848989+0000","last_active":"2023-10-08T01:12:02.851325+0000","last_peered":"2023-10-08T01:12:02.851325+0000","last_clean":"2023-10-08T01:12:02.851325+0000","last_became_active":"2023-10-08T01:12:02.848775+0000","last_became_peered":"2023-10-08T01:12:02.848775+0000","last_unstale":"2023-10-08T01:12:02.851325+0000","last_undegraded":"2023-10-08T01:12:02.851325+0000","last_fullsized":"2023-10-08T01:12:02.851325+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:30.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.833785+0000","last_change":"2023-10-08T01:12:01.828030+0000","last_active":"2023-10-08T01:12:02.833785+0000","last_peered":"2023-10-08T01:12:02.833785+0000","last_clean":"2023-10-08T01:12:02.833785+0000","last_became_active":"2023-10-08T01:12:01.827589+0000","last_became_peered":"2023-10-08T01:12:01.827589+0000","last_unstale":"2023-10-08T01:12:02.833785+0000","last_undegraded":"2023-10-08T01:12:02.833785+0000","last_fullsized":"2023-10-08T01:12:02.833785+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:02:59.446334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843699+0000","last_change":"2023-10-08T01:12:02.837963+0000","last_active":"2023-10-08T01:12:02.843699+0000","last_peered":"2023-10-08T01:12:02.843699+0000","last_clean":"2023-10-08T01:12:02.843699+0000","last_became_active":"2023-10-08T01:12:02.837114+0000","last_became_peered":"2023-10-08T01:12:02.837114+0000","last_unstale":"2023-10-08T01:12:02.843699+0000","last_undegraded":"2023-10-08T01:12:02.843699+0000","last_fullsized":"2023-10-08T01:12:02.843699+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:42.075297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"90'8","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859108+0000","last_change":"2023-10-08T01:12:01.823157+0000","last_active":"2023-10-08T01:12:02.859108+0000","last_peered":"2023-10-08T01:12:02.859108+0000","last_clean":"2023-10-08T01:12:02.859108+0000","last_became_active":"2023-10-08T01:12:01.822907+0000","last_became_peered":"2023-10-08T01:12:01.822907+0000","last_unstale":"2023-10-08T01:12:02.859108+0000","last_undegraded":"2023-10-08T01:12:02.859108+0000","last_fullsized":"2023-10-08T01:12:02.859108+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:58.158180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.842697+0000","last_change":"2023-10-08T01:12:02.837019+0000","last_active":"2023-10-08T01:12:02.842697+0000","last_peered":"2023-10-08T01:12:02.842697+0000","last_clean":"2023-10-08T01:12:02.842697+0000","last_became_active":"2023-10-08T01:12:02.836721+0000","last_became_peered":"2023-10-08T01:12:02.836721+0000","last_unstale":"2023-10-08T01:12:02.842697+0000","last_undegraded":"2023-10-08T01:12:02.842697+0000","last_fullsized":"2023-10-08T01:12:02.842697+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:44.223005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867962+0000","last_change":"2023-10-08T01:12:01.831319+0000","last_active":"2023-10-08T01:12:02.867962+0000","last_peered":"2023-10-08T01:12:02.867962+0000","last_clean":"2023-10-08T01:12:02.867962+0000","last_became_active":"2023-10-08T01:12:01.831046+0000","last_became_peered":"2023-10-08T01:12:01.831046+0000","last_unstale":"2023-10-08T01:12:02.867962+0000","last_undegraded":"2023-10-08T01:12:02.867962+0000","last_fullsized":"2023-10-08T01:12:02.867962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:41.069783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.847987+0000","last_change":"2023-10-08T01:12:02.841062+0000","last_active":"2023-10-08T01:12:02.847987+0000","last_peered":"2023-10-08T01:12:02.847987+0000","last_clean":"2023-10-08T01:12:02.847987+0000","last_became_active":"2023-10-08T01:12:02.840782+0000","last_became_peered":"2023-10-08T01:12:02.840782+0000","last_unstale":"2023-10-08T01:12:02.847987+0000","last_undegraded":"2023-10-08T01:12:02.847987+0000","last_fullsized":"2023-10-08T01:12:02.847987+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:21.885843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.864739+0000","last_change":"2023-10-08T01:12:01.830454+0000","last_active":"2023-10-08T01:12:02.864739+0000","last_peered":"2023-10-08T01:12:02.864739+0000","last_clean":"2023-10-08T01:12:02.864739+0000","last_became_active":"2023-10-08T01:12:01.830184+0000","last_became_peered":"2023-10-08T01:12:01.830184+0000","last_unstale":"2023-10-08T01:12:02.864739+0000","last_undegraded":"2023-10-08T01:12:02.864739+0000","last_fullsized":"2023-10-08T01:12:02.864739+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:47.737308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.845961+0000","last_change":"2023-10-08T01:12:02.840968+0000","last_active":"2023-10-08T01:12:02.845961+0000","last_peered":"2023-10-08T01:12:02.845961+0000","last_clean":"2023-10-08T01:12:02.845961+0000","last_became_active":"2023-10-08T01:12:02.840717+0000","last_became_peered":"2023-10-08T01:12:02.840717+0000","last_unstale":"2023-10-08T01:12:02.845961+0000","last_undegraded":"2023-10-08T01:12:02.845961+0000","last_fullsized":"2023-10-08T01:12:02.845961+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:39.174760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"90'1","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867409+0000","last_change":"2023-10-08T01:12:01.829452+0000","last_active":"2023-10-08T01:12:02.867409+0000","last_peered":"2023-10-08T01:12:02.867409+0000","last_clean":"2023-10-08T01:12:02.867409+0000","last_became_active":"2023-10-08T01:12:01.829140+0000","last_became_peered":"2023-10-08T01:12:01.829140+0000","last_unstale":"2023-10-08T01:12:02.867409+0000","last_undegraded":"2023-10-08T01:12:02.867409+0000","last_fullsized":"2023-10-08T01:12:02.867409+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:47:51.755961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843809+0000","last_change":"2023-10-08T01:12:02.840709+0000","last_active":"2023-10-08T01:12:02.843809+0000","last_peered":"2023-10-08T01:12:02.843809+0000","last_clean":"2023-10-08T01:12:02.843809+0000","last_became_active":"2023-10-08T01:12:02.840250+0000","last_became_peered":"2023-10-08T01:12:02.840250+0000","last_unstale":"2023-10-08T01:12:02.843809+0000","last_undegraded":"2023-10-08T01:12:02.843809+0000","last_fullsized":"2023-10-08T01:12:02.843809+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:26.311091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.831042+0000","last_change":"2023-10-08T01:12:01.821491+0000","last_active":"2023-10-08T01:12:02.831042+0000","last_peered":"2023-10-08T01:12:02.831042+0000","last_clean":"2023-10-08T01:12:02.831042+0000","last_became_active":"2023-10-08T01:12:01.821135+0000","last_became_peered":"2023-10-08T01:12:01.821135+0000","last_unstale":"2023-10-08T01:12:02.831042+0000","last_undegraded":"2023-10-08T01:12:02.831042+0000","last_fullsized":"2023-10-08T01:12:02.831042+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:55.795619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.838339+0000","last_change":"2023-10-08T01:12:02.836913+0000","last_active":"2023-10-08T01:12:02.838339+0000","last_peered":"2023-10-08T01:12:02.838339+0000","last_clean":"2023-10-08T01:12:02.838339+0000","last_became_active":"2023-10-08T01:12:02.836618+0000","last_became_peered":"2023-10-08T01:12:02.836618+0000","last_unstale":"2023-10-08T01:12:02.838339+0000","last_undegraded":"2023-10-08T01:12:02.838339+0000","last_fullsized":"2023-10-08T01:12:02.838339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:34:05.095868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850521+0000","last_change":"2023-10-08T01:12:01.831714+0000","last_active":"2023-10-08T01:12:02.850521+0000","last_peered":"2023-10-08T01:12:02.850521+0000","last_clean":"2023-10-08T01:12:02.850521+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850521+0000","last_undegraded":"2023-10-08T01:12:02.850521+0000","last_fullsized":"2023-10-08T01:12:02.850521+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:56.535520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.862945+0000","last_change":"2023-10-08T01:12:02.862541+0000","last_active":"2023-10-08T01:12:02.862945+0000","last_peered":"2023-10-08T01:12:02.862945+0000","last_clean":"2023-10-08T01:12:02.862945+0000","last_became_active":"2023-10-08T01:12:02.862076+0000","last_became_peered":"2023-10-08T01:12:02.862076+0000","last_unstale":"2023-10-08T01:12:02.862945+0000","last_undegraded":"2023-10-08T01:12:02.862945+0000","last_fullsized":"2023-10-08T01:12:02.862945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:11:26.919061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.833887+0000","last_change":"2023-10-08T01:12:01.827944+0000","last_active":"2023-10-08T01:12:02.833887+0000","last_peered":"2023-10-08T01:12:02.833887+0000","last_clean":"2023-10-08T01:12:02.833887+0000","last_became_active":"2023-10-08T01:12:01.827116+0000","last_became_peered":"2023-10-08T01:12:01.827116+0000","last_unstale":"2023-10-08T01:12:02.833887+0000","last_undegraded":"2023-10-08T01:12:02.833887+0000","last_fullsized":"2023-10-08T01:12:02.833887+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:51.723580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852339+0000","last_change":"2023-10-08T01:12:02.849161+0000","last_active":"2023-10-08T01:12:02.852339+0000","last_peered":"2023-10-08T01:12:02.852339+0000","last_clean":"2023-10-08T01:12:02.852339+0000","last_became_active":"2023-10-08T01:12:02.848913+0000","last_became_peered":"2023-10-08T01:12:02.848913+0000","last_unstale":"2023-10-08T01:12:02.852339+0000","last_undegraded":"2023-10-08T01:12:02.852339+0000","last_fullsized":"2023-10-08T01:12:02.852339+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:58:40.796959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.832971+0000","last_change":"2023-10-08T01:12:01.819602+0000","last_active":"2023-10-08T01:12:02.832971+0000","last_peered":"2023-10-08T01:12:02.832971+0000","last_clean":"2023-10-08T01:12:02.832971+0000","last_became_active":"2023-10-08T01:12:01.819304+0000","last_became_peered":"2023-10-08T01:12:01.819304+0000","last_unstale":"2023-10-08T01:12:02.832971+0000","last_undegraded":"2023-10-08T01:12:02.832971+0000","last_fullsized":"2023-10-08T01:12:02.832971+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:55.795619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.846380+0000","last_change":"2023-10-08T01:12:02.844295+0000","last_active":"2023-10-08T01:12:02.846380+0000","last_peered":"2023-10-08T01:12:02.846380+0000","last_clean":"2023-10-08T01:12:02.846380+0000","last_became_active":"2023-10-08T01:12:02.844001+0000","last_became_peered":"2023-10-08T01:12:02.844001+0000","last_unstale":"2023-10-08T01:12:02.846380+0000","last_undegraded":"2023-10-08T01:12:02.846380+0000","last_fullsized":"2023-10-08T01:12:02.846380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:39:36.571615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.867968+0000","last_change":"2023-10-08T01:12:01.831354+0000","last_active":"2023-10-08T01:12:02.867968+0000","last_peered":"2023-10-08T01:12:02.867968+0000","last_clean":"2023-10-08T01:12:02.867968+0000","last_became_active":"2023-10-08T01:12:01.830964+0000","last_became_peered":"2023-10-08T01:12:01.830964+0000","last_unstale":"2023-10-08T01:12:02.867968+0000","last_undegraded":"2023-10-08T01:12:02.867968+0000","last_fullsized":"2023-10-08T01:12:02.867968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:20.094528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850568+0000","last_change":"2023-10-08T01:12:02.845176+0000","last_active":"2023-10-08T01:12:02.850568+0000","last_peered":"2023-10-08T01:12:02.850568+0000","last_clean":"2023-10-08T01:12:02.850568+0000","last_became_active":"2023-10-08T01:12:02.844833+0000","last_became_peered":"2023-10-08T01:12:02.844833+0000","last_unstale":"2023-10-08T01:12:02.850568+0000","last_undegraded":"2023-10-08T01:12:02.850568+0000","last_fullsized":"2023-10-08T01:12:02.850568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:30.072445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834922+0000","last_change":"2023-10-08T01:12:01.829454+0000","last_active":"2023-10-08T01:12:02.834922+0000","last_peered":"2023-10-08T01:12:02.834922+0000","last_clean":"2023-10-08T01:12:02.834922+0000","last_became_active":"2023-10-08T01:12:01.829133+0000","last_became_peered":"2023-10-08T01:12:01.829133+0000","last_unstale":"2023-10-08T01:12:02.834922+0000","last_undegraded":"2023-10-08T01:12:02.834922+0000","last_fullsized":"2023-10-08T01:12:02.834922+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:45.533589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.865079+0000","last_change":"2023-10-08T01:12:02.862111+0000","last_active":"2023-10-08T01:12:02.865079+0000","last_peered":"2023-10-08T01:12:02.865079+0000","last_clean":"2023-10-08T01:12:02.865079+0000","last_became_active":"2023-10-08T01:12:02.861816+0000","last_became_peered":"2023-10-08T01:12:02.861816+0000","last_unstale":"2023-10-08T01:12:02.865079+0000","last_undegraded":"2023-10-08T01:12:02.865079+0000","last_fullsized":"2023-10-08T01:12:02.865079+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:45:20.356518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828282+0000","last_change":"2023-10-08T01:12:01.823222+0000","last_active":"2023-10-08T01:12:02.828282+0000","last_peered":"2023-10-08T01:12:02.828282+0000","last_clean":"2023-10-08T01:12:02.828282+0000","last_became_active":"2023-10-08T01:12:01.822970+0000","last_became_peered":"2023-10-08T01:12:01.822970+0000","last_unstale":"2023-10-08T01:12:02.828282+0000","last_undegraded":"2023-10-08T01:12:02.828282+0000","last_fullsized":"2023-10-08T01:12:02.828282+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:00.437245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.853947+0000","last_change":"2023-10-08T01:12:02.851360+0000","last_active":"2023-10-08T01:12:02.853947+0000","last_peered":"2023-10-08T01:12:02.853947+0000","last_clean":"2023-10-08T01:12:02.853947+0000","last_became_active":"2023-10-08T01:12:02.851117+0000","last_became_peered":"2023-10-08T01:12:02.851117+0000","last_unstale":"2023-10-08T01:12:02.853947+0000","last_undegraded":"2023-10-08T01:12:02.853947+0000","last_fullsized":"2023-10-08T01:12:02.853947+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:47.054661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.834238+0000","last_change":"2023-10-08T01:12:01.815771+0000","last_active":"2023-10-08T01:12:02.834238+0000","last_peered":"2023-10-08T01:12:02.834238+0000","last_clean":"2023-10-08T01:12:02.834238+0000","last_became_active":"2023-10-08T01:12:01.815502+0000","last_became_peered":"2023-10-08T01:12:01.815502+0000","last_unstale":"2023-10-08T01:12:02.834238+0000","last_undegraded":"2023-10-08T01:12:02.834238+0000","last_fullsized":"2023-10-08T01:12:02.834238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:41:24.163002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843426+0000","last_change":"2023-10-08T01:12:02.838180+0000","last_active":"2023-10-08T01:12:02.843426+0000","last_peered":"2023-10-08T01:12:02.843426+0000","last_clean":"2023-10-08T01:12:02.843426+0000","last_became_active":"2023-10-08T01:12:02.837945+0000","last_became_peered":"2023-10-08T01:12:02.837945+0000","last_unstale":"2023-10-08T01:12:02.843426+0000","last_undegraded":"2023-10-08T01:12:02.843426+0000","last_fullsized":"2023-10-08T01:12:02.843426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:56:21.100043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.827017+0000","last_change":"2023-10-08T01:12:01.827250+0000","last_active":"2023-10-08T01:12:02.827017+0000","last_peered":"2023-10-08T01:12:02.827017+0000","last_clean":"2023-10-08T01:12:02.827017+0000","last_became_active":"2023-10-08T01:12:01.826935+0000","last_became_peered":"2023-10-08T01:12:01.826935+0000","last_unstale":"2023-10-08T01:12:02.827017+0000","last_undegraded":"2023-10-08T01:12:02.827017+0000","last_fullsized":"2023-10-08T01:12:02.827017+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:12:40.815318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.860494+0000","last_change":"2023-10-08T01:12:02.856747+0000","last_active":"2023-10-08T01:12:02.860494+0000","last_peered":"2023-10-08T01:12:02.860494+0000","last_clean":"2023-10-08T01:12:02.860494+0000","last_became_active":"2023-10-08T01:12:02.856476+0000","last_became_peered":"2023-10-08T01:12:02.856476+0000","last_unstale":"2023-10-08T01:12:02.860494+0000","last_undegraded":"2023-10-08T01:12:02.860494+0000","last_fullsized":"2023-10-08T01:12:02.860494+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:03:00.451849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.833345+0000","last_change":"2023-10-08T01:12:01.826441+0000","last_active":"2023-10-08T01:12:02.833345+0000","last_peered":"2023-10-08T01:12:02.833345+0000","last_clean":"2023-10-08T01:12:02.833345+0000","last_became_active":"2023-10-08T01:12:01.826164+0000","last_became_peered":"2023-10-08T01:12:01.826164+0000","last_unstale":"2023-10-08T01:12:02.833345+0000","last_undegraded":"2023-10-08T01:12:02.833345+0000","last_fullsized":"2023-10-08T01:12:02.833345+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:05:56.535520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.843986+0000","last_change":"2023-10-08T01:12:02.839714+0000","last_active":"2023-10-08T01:12:02.843986+0000","last_peered":"2023-10-08T01:12:02.843986+0000","last_clean":"2023-10-08T01:12:02.843986+0000","last_became_active":"2023-10-08T01:12:02.839457+0000","last_became_peered":"2023-10-08T01:12:02.839457+0000","last_unstale":"2023-10-08T01:12:02.843986+0000","last_undegraded":"2023-10-08T01:12:02.843986+0000","last_fullsized":"2023-10-08T01:12:02.843986+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:55.300544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.828089+0000","last_change":"2023-10-08T01:12:01.822618+0000","last_active":"2023-10-08T01:12:02.828089+0000","last_peered":"2023-10-08T01:12:02.828089+0000","last_clean":"2023-10-08T01:12:02.828089+0000","last_became_active":"2023-10-08T01:12:01.822354+0000","last_became_peered":"2023-10-08T01:12:01.822354+0000","last_unstale":"2023-10-08T01:12:02.828089+0000","last_undegraded":"2023-10-08T01:12:02.828089+0000","last_fullsized":"2023-10-08T01:12:02.828089+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:25:43.082489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.859915+0000","last_change":"2023-10-08T01:12:02.855156+0000","last_active":"2023-10-08T01:12:02.859915+0000","last_peered":"2023-10-08T01:12:02.859915+0000","last_clean":"2023-10-08T01:12:02.859915+0000","last_became_active":"2023-10-08T01:12:02.854809+0000","last_became_peered":"2023-10-08T01:12:02.854809+0000","last_unstale":"2023-10-08T01:12:02.859915+0000","last_undegraded":"2023-10-08T01:12:02.859915+0000","last_fullsized":"2023-10-08T01:12:02.859915+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:00:46.539103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":10,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.850388+0000","last_change":"2023-10-08T01:12:01.831667+0000","last_active":"2023-10-08T01:12:02.850388+0000","last_peered":"2023-10-08T01:12:02.850388+0000","last_clean":"2023-10-08T01:12:02.850388+0000","last_became_active":"2023-10-08T01:12:01.831363+0000","last_became_peered":"2023-10-08T01:12:01.831363+0000","last_unstale":"2023-10-08T01:12:02.850388+0000","last_undegraded":"2023-10-08T01:12:02.850388+0000","last_fullsized":"2023-10-08T01:12:02.850388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:23.287500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.852439+0000","last_change":"2023-10-08T01:12:02.849116+0000","last_active":"2023-10-08T01:12:02.852439+0000","last_peered":"2023-10-08T01:12:02.852439+0000","last_clean":"2023-10-08T01:12:02.852439+0000","last_became_active":"2023-10-08T01:12:02.848833+0000","last_became_peered":"2023-10-08T01:12:02.848833+0000","last_unstale":"2023-10-08T01:12:02.852439+0000","last_undegraded":"2023-10-08T01:12:02.852439+0000","last_fullsized":"2023-10-08T01:12:02.852439+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:01.799734+0000","last_clean_scrub_stamp":"2023-10-08T01:12:01.799734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:35.798359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"90'2","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.870650+0000","last_change":"2023-10-08T01:12:01.828087+0000","last_active":"2023-10-08T01:12:02.870650+0000","last_peered":"2023-10-08T01:12:02.870650+0000","last_clean":"2023-10-08T01:12:02.870650+0000","last_became_active":"2023-10-08T01:12:01.827688+0000","last_became_peered":"2023-10-08T01:12:01.827688+0000","last_unstale":"2023-10-08T01:12:02.870650+0000","last_undegraded":"2023-10-08T01:12:02.870650+0000","last_fullsized":"2023-10-08T01:12:02.870650+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:00.794219+0000","last_clean_scrub_stamp":"2023-10-08T01:12:00.794219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:25.305577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"80'183","reported_seq":460,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-10-08T01:12:02.833196+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:12:02.833196+0000","last_peered":"2023-10-08T01:12:02.833196+0000","last_clean":"2023-10-08T01:12:02.833196+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:12:02.833196+0000","last_undegraded":"2023-10-08T01:12:02.833196+0000","last_fullsized":"2023-10-08T01:12:02.833196+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":183,"log_dups_size":0,"ondisk_log_size":183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":692,"num_read_kb":1336,"num_write":279,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":183,"ondisk_log_size":183,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303056,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068784,"statfs":{"total":96636764160,"available":96326434816,"internally_reserved":0,"allocated":1712128,"data_stored":1127436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.55800000000000005,"5min":0.40300000000000002,"15min":0.36399999999999999},"max":{"1min":0.91500000000000004,"5min":1.123,"15min":1.123},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.70399999999999996,"15min":0.71099999999999997},"min":{"1min":0.47299999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.56299999999999994}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70399999999999996,"15min":0.70299999999999996},"min":{"1min":0.503,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.872,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.71299999999999997,"15min":0.72399999999999998},"min":{"1min":0.53000000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.93600000000000005,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.63}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73599999999999999,"15min":0.73799999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47199999999999998},"max":{"1min":0.99299999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.74099999999999999,"15min":0.73399999999999999},"min":{"1min":0.503,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.95099999999999996,"5min":1.167,"15min":1.167},"last":0.65600000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.70699999999999996,"15min":0.71899999999999997},"min":{"1min":0.58299999999999996,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93200000000000005,"5min":0.94699999999999995,"15min":0.998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76800000000000002,"15min":0.75800000000000001},"min":{"1min":0.59899999999999998,"5min":0.503,"15min":0.503},"max":{"1min":1.052,"5min":1.079,"15min":1.079},"last":0.71399999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.38},"max":{"1min":0.72499999999999998,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.61499999999999999,"15min":0.61199999999999999},"min":{"1min":0.45000000000000001,"5min":0.44,"15min":0.40899999999999997},"max":{"1min":0.78100000000000003,"5min":0.88,"15min":1.0780000000000001},"last":0.5}]},{"osd":5,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.63200000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.89500000000000002,"5min":0.93400000000000005,"15min":1.1240000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.67000000000000004,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.44800000000000001},"max":{"1min":0.85999999999999999,"5min":0.97599999999999998,"15min":1.0389999999999999},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.752,"15min":0.754},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.40400000000000003},"max":{"1min":0.82699999999999996,"5min":1.01,"15min":1.1259999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.41599999999999998},"max":{"1min":0.95199999999999996,"5min":0.98599999999999999,"15min":1.1439999999999999},"last":0.73799999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1118208,"data_stored":536572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70399999999999996},"min":{"1min":0.40600000000000003,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.90100000000000002,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.497},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68500000000000005,"15min":0.69999999999999996},"min":{"1min":0.55700000000000005,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.093},"last":0.74199999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.54600000000000004,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":2.984},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70099999999999996,"15min":0.72999999999999998},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.018,"15min":3.0190000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.67900000000000005,"15min":0.69599999999999995},"min":{"1min":0.501,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.88600000000000001,"5min":0.91300000000000003,"15min":1.9199999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.72099999999999997,"15min":0.73499999999999999},"min":{"1min":0.59799999999999998,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.89700000000000002,"5min":0.91300000000000003,"15min":1.4810000000000001},"last":0.54900000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.59399999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.95399999999999996,"5min":0.95499999999999996,"15min":1.2290000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72799999999999998,"15min":0.74099999999999999},"min":{"1min":0.60699999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.95599999999999996},"last":0.72099999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59699999999999998,"15min":0.59599999999999997},"min":{"1min":0.38900000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":1.163},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.628,"15min":0.625},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41399999999999998},"max":{"1min":0.877,"5min":1.087,"15min":1.361},"last":0.68500000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.623,"15min":0.621},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.395,"15min":1.395},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.67100000000000004,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.81799999999999995,"5min":1.21,"15min":1.4119999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Sun Oct 8 01:11:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.69899999999999995,"15min":0.71799999999999997},"min":{"1min":0.44500000000000001,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":0.90500000000000003,"5min":0.98999999999999999,"15min":2.9489999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50600000000000001,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":0.96899999999999997,"5min":1.329,"15min":1.329},"last":0.83699999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1104,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1130496,"data_stored":538604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.47699999999999998,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.69799999999999995,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.504},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.503},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.28199999999999997},"max":{"1min":0.76600000000000001,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.433}]},{"osd":2,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.35599999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.69399999999999995,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.41999999999999998,"5min":0.36699999999999999,"15min":0.33700000000000002},"max":{"1min":0.69599999999999995,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.44900000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.57499999999999996},"min":{"1min":0.45700000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73999999999999999,"5min":0.79700000000000004,"15min":0.82099999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.312},"max":{"1min":0.67700000000000005,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.39400000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.76700000000000002},"min":{"1min":0.63400000000000001,"5min":0.505,"15min":0.38500000000000001},"max":{"1min":1.05,"5min":1.452,"15min":1.661},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75600000000000001,"15min":0.76900000000000002},"min":{"1min":0.56399999999999995,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":0.99099999999999999,"5min":1.1499999999999999,"15min":1.5569999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76800000000000002,"15min":0.77500000000000002},"min":{"1min":0.56799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.0820000000000001,"5min":1.274,"15min":1.409},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.79100000000000004},"min":{"1min":0.59699999999999998,"5min":0.54500000000000004,"15min":0.498},"max":{"1min":1.0229999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.60799999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.78800000000000003,"15min":0.79600000000000004},"min":{"1min":0.626,"5min":0.48399999999999999,"15min":0.47799999999999998},"max":{"1min":0.93799999999999994,"5min":1.6020000000000001,"15min":1.617},"last":0.747},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.60399999999999998,"5min":0.56000000000000005,"15min":0.53300000000000003},"max":{"1min":1.0249999999999999,"5min":1.3660000000000001,"15min":1.4790000000000001},"last":0.59599999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.84099999999999997,"15min":0.83199999999999996},"min":{"1min":0.63600000000000001,"5min":0.56000000000000005,"15min":0.53200000000000003},"max":{"1min":1.0640000000000001,"5min":1.621,"15min":1.621},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.66100000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.0620000000000001,"5min":1.4890000000000001,"15min":1.913},"last":0.71499999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303048,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068792,"statfs":{"total":96636764160,"available":96326443008,"internally_reserved":0,"allocated":1703936,"data_stored":1126848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.45200000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50800000000000001,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.36399999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.46500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.617},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.5289999999999999},"last":0.754}]},{"osd":6,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.208,"5min":1.361,"15min":1.3779999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1719999999999999,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.65000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306576,"kb_used_data":1096,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065264,"statfs":{"total":96636764160,"available":96322830336,"internally_reserved":0,"allocated":1122304,"data_stored":538016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.46200000000000002,"15min":0.45800000000000002},"min":{"1min":0.33500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69499999999999995,"5min":0.752,"15min":1.2689999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51900000000000002,"15min":0.51100000000000001},"min":{"1min":0.40200000000000002,"5min":0.318,"15min":0.27500000000000002},"max":{"1min":0.752,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.45700000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.504,"15min":0.48299999999999998},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28399999999999997},"max":{"1min":0.63200000000000001,"5min":2.786,"15min":2.786},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57099999999999995,"15min":0.54800000000000004},"min":{"1min":0.45700000000000002,"5min":0.36799999999999999,"15min":0.35899999999999999},"max":{"1min":0.79600000000000004,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61099999999999999,"15min":0.60099999999999998},"min":{"1min":0.44400000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.84099999999999997,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.39700000000000002,"5min":0.38500000000000001,"15min":0.34999999999999998},"max":{"1min":0.745,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.53700000000000003,"5min":0.51000000000000001,"15min":0.504},"max":{"1min":0.96199999999999997,"5min":1.079,"15min":1.1180000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.624,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.071,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.65500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.82399999999999995,"15min":0.83999999999999997},"min":{"1min":0.65800000000000003,"5min":0.53800000000000003,"15min":0.53100000000000003},"max":{"1min":1.0660000000000001,"5min":2.488,"15min":2.488},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.61099999999999999,"5min":0.57599999999999996,"15min":0.50900000000000001},"max":{"1min":1.0880000000000001,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.74099999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.79300000000000004,"15min":0.79800000000000004},"min":{"1min":0.55700000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0429999999999999,"5min":2.653,"15min":2.653},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79300000000000004,"15min":0.79700000000000004},"min":{"1min":0.57899999999999996,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0309999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:11:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79700000000000004,"15min":0.80000000000000004},"min":{"1min":0.57999999999999996,"5min":0.53400000000000003,"15min":0.51600000000000001},"max":{"1min":1.0289999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.82699999999999996,"15min":0.82099999999999995},"min":{"1min":0.66600000000000004,"5min":0.504,"15min":0.504},"max":{"1min":1.111,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.66100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306572,"kb_used_data":1092,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065268,"statfs":{"total":96636764160,"available":96322834432,"internally_reserved":0,"allocated":1118208,"data_stored":536592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.69999999999999996,"15min":0.80100000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.57399999999999995,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.378}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61699999999999999,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.82999999999999996,"15min":0.88100000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.79400000000000004,"15min":1.1060000000000001},"last":0.47699999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":1.095,"15min":1.3029999999999999},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74299999999999999,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.117,"15min":1.413},"last":0.76100000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.77800000000000002},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.0920000000000001,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":1.089},"last":0.61899999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.379},"last":0.751},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.486},"last":0.63500000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97099999999999997,"5min":1.0840000000000001,"15min":1.389},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.79700000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.5},"max":{"1min":0.93700000000000006,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.65500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310664,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061176,"statfs":{"total":96636764160,"available":96318644224,"internally_reserved":0,"allocated":1114112,"data_stored":536550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.68300000000000005,"15min":0.68400000000000005},"min":{"1min":0.52300000000000002,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69799999999999995,"15min":0.69499999999999995},"min":{"1min":0.50800000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.96599999999999997,"5min":0.998,"15min":1.0049999999999999},"last":0.60899999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.69399999999999995,"15min":0.69899999999999995},"min":{"1min":0.52800000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.05,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.56299999999999994,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.88100000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.91800000000000004,"5min":0.96199999999999997,"15min":1.024},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.75700000000000001,"15min":0.76800000000000002},"min":{"1min":0.65300000000000002,"5min":0.502,"15min":0.45400000000000001},"max":{"1min":0.93500000000000005,"5min":1.323,"15min":1.323},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74299999999999999,"15min":0.76400000000000001},"min":{"1min":0.497,"5min":0.442,"15min":0.442},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.75},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76900000000000002,"15min":0.79400000000000004},"min":{"1min":0.64100000000000001,"5min":0.53300000000000003,"15min":0.51800000000000002},"max":{"1min":1.0109999999999999,"5min":1.2829999999999999,"15min":1.454},"last":0.81299999999999994}]},{"osd":5,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.59199999999999997,"15min":0.61499999999999999},"min":{"1min":0.38900000000000001,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.77800000000000002,"5min":0.98399999999999999,"15min":1.1459999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.72099999999999997,"15min":0.73599999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.42399999999999999},"max":{"1min":0.85899999999999999,"5min":0.95499999999999996,"15min":1.3420000000000001},"last":0.79800000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63300000000000001,"15min":0.65000000000000002},"min":{"1min":0.46899999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.64700000000000002,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.91100000000000003,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.72399999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:11:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.68200000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.90400000000000003,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.70999999999999996,"15min":0.72599999999999998},"min":{"1min":0.53800000000000003,"5min":0.48299999999999998,"15min":0.45300000000000001},"max":{"1min":0.94399999999999995,"5min":1.2450000000000001,"15min":1.26},"last":0.63}]}]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302472,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069368,"statfs":{"total":96636764160,"available":96327032832,"internally_reserved":0,"allocated":1114112,"data_stored":536570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68100000000000005,"15min":0.69299999999999995},"min":{"1min":0.54500000000000004,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.93899999999999995,"15min":1.0289999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58199999999999996,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":0.93999999999999995,"5min":1.0089999999999999,"15min":1.0129999999999999},"last":0.55900000000000005}]},{"osd":1,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.69799999999999995,"15min":0.71499999999999997},"min":{"1min":0.52700000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.495},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68899999999999995,"15min":0.69799999999999995},"min":{"1min":0.47899999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.879,"5min":0.92100000000000004,"15min":1.077},"last":0.58999999999999997}]},{"osd":2,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71699999999999997},"min":{"1min":0.54200000000000004,"5min":0.51700000000000002,"15min":0.38700000000000001},"max":{"1min":0.82999999999999996,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.623},{"interface":"front","average":{"1min":0.747,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.48599999999999999,"5min":0.47999999999999998,"15min":0.46400000000000002},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.72399999999999998,"15min":0.73399999999999999},"min":{"1min":0.56999999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.87,"5min":0.92000000000000004,"15min":1.1339999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.746,"15min":0.75800000000000001},"min":{"1min":0.58799999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.107},"last":0.71999999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.59399999999999997,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.376,"15min":0.35599999999999998},"max":{"1min":0.84199999999999997,"5min":0.86299999999999999,"15min":1.042},"last":0.874},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67300000000000004,"15min":0.66300000000000003},"min":{"1min":0.48799999999999999,"5min":0.45300000000000001,"15min":0.42199999999999999},"max":{"1min":0.80500000000000005,"5min":1.048,"15min":1.081},"last":0.77700000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.64800000000000002,"15min":0.66100000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.1499999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68300000000000005,"15min":0.68100000000000005},"min":{"1min":0.53500000000000003,"5min":0.433,"15min":0.41699999999999998},"max":{"1min":0.874,"5min":0.95699999999999996,"15min":1.0960000000000001},"last":0.80200000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.70399999999999996,"15min":0.69599999999999995},"min":{"1min":0.50800000000000001,"5min":0.49299999999999999,"15min":0.48199999999999998},"max":{"1min":0.89300000000000002,"5min":1.095,"15min":1.165},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.56399999999999995,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.3879999999999999},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:12:12.443 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T01:12:12.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T01:12:12.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.661+0000 7f9f551ff700 1 Processor -- start 2023-10-08T01:12:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.661+0000 7f9f551ff700 1 -- start start 2023-10-08T01:12:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f551ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f50156620 0x7f9f500c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f551ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5015a6f0 con 0x7f9f501569f0 2023-10-08T01:12:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f551ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5015a9d0 con 0x7f9f50068390 2023-10-08T01:12:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f551ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500c8d30 con 0x7f9f50156620 2023-10-08T01:12:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f4e59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9f501569f0 0x7f9f500ace70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52266/0 (socket says 172.21.15.119:52266) 2023-10-08T01:12:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f4e59c700 1 -- 172.21.15.119:0/4243194337 learned_addr learned my addr 172.21.15.119:0/4243194337 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f4ed9d700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f50156620 0x7f9f500c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962766750 0 0) 0x7f9f5015a6f0 con 0x7f9f501569f0 2023-10-08T01:12:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.663+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f38003570 con 0x7f9f501569f0 2023-10-08T01:12:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 284415885 0 0) 0x7f9f38003570 con 0x7f9f501569f0 2023-10-08T01:12:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f50156620 msgr2=0x7f9f500c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f50156620 0x7f9f500c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 >> v1:172.21.15.184:6789/0 conn(0x7f9f50068390 legacy=0x7f9f5006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5015acb0 con 0x7f9f501569f0 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f9f44001470 con 0x7f9f501569f0 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 >> v1:172.21.15.119:6789/0 conn(0x7f9f501569f0 legacy=0x7f9f500ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003a90 0x7f9f38003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f50156620 0x7f9f38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f38004d50 0x7f9f380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500c8d30 con 0x7f9f38003a90 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3800dd20 con 0x7f9f38004d50 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3800de60 con 0x7f9f50156620 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4ed9d700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003a90 0x7f9f38003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4e59c700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f50156620 0x7f9f38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.664+0000 7f9f4f59e700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f38004d50 0x7f9f380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4ed9d700 1 -- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f50156620 msgr2=0x7f9f38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4ed9d700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f50156620 0x7f9f38004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4ed9d700 1 -- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f38004d50 msgr2=0x7f9f380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4ed9d700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f38004d50 0x7f9f380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4ed9d700 1 -- 172.21.15.119:0/4243194337 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f38003570 con 0x7f9f38003a90 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4ed9d700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003a90 0x7f9f38003e60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000e210 tx=0x7f9f4000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eca53766f31f7da6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000eac0 con 0x7f9f38003a90 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f4000f040 con 0x7f9f38003a90 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f4dd9b700 1 -- 172.21.15.119:0/4243194337 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f4000ee70 con 0x7f9f38003a90 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f551ff700 1 -- 172.21.15.119:0/4243194337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003a90 msgr2=0x7f9f38003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.665+0000 7f9f551ff700 1 --2- 172.21.15.119:0/4243194337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f38003a90 0x7f9f38003e60 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f9f4000e210 tx=0x7f9f4000b040 comp rx=0 tx=0).stop 2023-10-08T01:12:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.666+0000 7f9f4f59e700 1 -- 172.21.15.119:0/4243194337 reap_dead start 2023-10-08T01:12:12.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.666+0000 7f9f551ff700 1 -- 172.21.15.119:0/4243194337 shutdown_connections 2023-10-08T01:12:12.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.666+0000 7f9f551ff700 1 -- 172.21.15.119:0/4243194337 >> 172.21.15.119:0/4243194337 conn(0x7f9f500cc540 msgr2=0x7f9f500cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:12.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.666+0000 7f9f551ff700 1 -- 172.21.15.119:0/4243194337 shutdown_connections 2023-10-08T01:12:12.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.666+0000 7f9f551ff700 1 -- 172.21.15.119:0/4243194337 wait complete. 2023-10-08T01:12:12.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.667+0000 7f9f551ff700 1 Processor -- start 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.667+0000 7f9f551ff700 1 -- start start 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f551ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f38003a90 0x7f9f50154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f551ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f500b0ff0 0x7f9f501583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f551ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50158900 0x7f9f500c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f551ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f50155130 con 0x7f9f50158900 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f551ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f50158e80 con 0x7f9f500b0ff0 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f551ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f50159160 con 0x7f9f38003a90 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f4f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50158900 0x7f9f500c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f4f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50158900 0x7f9f500c7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55226/0 (socket says 172.21.15.119:55226) 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.668+0000 7f9f4f59e700 1 -- 172.21.15.119:0/4122024950 learned_addr learned my addr 172.21.15.119:0/4122024950 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f4f59e700 1 -- 172.21.15.119:0/4122024950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f38003a90 msgr2=0x7f9f50154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f4f59e700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f38003a90 0x7f9f50154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f4f59e700 1 -- 172.21.15.119:0/4122024950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f500b0ff0 msgr2=0x7f9f501583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f4f59e700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f500b0ff0 0x7f9f501583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f4f59e700 1 -- 172.21.15.119:0/4122024950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50159440 con 0x7f9f50158900 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f4f59e700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50158900 0x7f9f500c7de0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9f500c7b90 tx=0x7f9f48009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f36ffd700 1 -- 172.21.15.119:0/4122024950 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f480015a0 con 0x7f9f50158900 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f36ffd700 1 -- 172.21.15.119:0/4122024950 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f48001720 con 0x7f9f50158900 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f50159720 con 0x7f9f50158900 2023-10-08T01:12:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f36ffd700 1 -- 172.21.15.119:0/4122024950 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f48001ae0 con 0x7f9f50158900 2023-10-08T01:12:12.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.669+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f500c8320 con 0x7f9f50158900 2023-10-08T01:12:12.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.670+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f500718b0 con 0x7f9f50158900 2023-10-08T01:12:12.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.671+0000 7f9f36ffd700 1 -- 172.21.15.119:0/4122024950 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f4801d460 con 0x7f9f50158900 2023-10-08T01:12:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.672+0000 7f9f36ffd700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.672+0000 7f9f36ffd700 1 -- 172.21.15.119:0/4122024950 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f9f48084840 con 0x7f9f50158900 2023-10-08T01:12:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.672+0000 7f9f4ed9d700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.673+0000 7f9f4ed9d700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f3c066f30 0x7f9f3c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f400094a0 tx=0x7f9f4001a000 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:12.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.676+0000 7f9f36ffd700 1 -- 172.21.15.119:0/4122024950 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f48051880 con 0x7f9f50158900 2023-10-08T01:12:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.943+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9f50068780 con 0x7f9f50158900 2023-10-08T01:12:12.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.948+0000 7f9f36ffd700 1 -- 172.21.15.119:0/4122024950 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7f9f48054ea0 con 0x7f9f50158900 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f3c066f30 msgr2=0x7f9f3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f3c066f30 0x7f9f3c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9f400094a0 tx=0x7f9f4001a000 comp rx=0 tx=0).stop 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50158900 msgr2=0x7f9f500c7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50158900 0x7f9f500c7de0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9f500c7b90 tx=0x7f9f48009040 comp rx=0 tx=0).stop 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 shutdown_connections 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f3c066f30 0x7f9f3c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f38003a90 0x7f9f50154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f50158900 0x7f9f500c7de0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 --2- 172.21.15.119:0/4122024950 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f500b0ff0 0x7f9f501583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.950+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 >> 172.21.15.119:0/4122024950 conn(0x7f9f500cc540 msgr2=0x7f9f500c7460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.951+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 shutdown_connections 2023-10-08T01:12:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:12.951+0000 7f9f551ff700 1 -- 172.21.15.119:0/4122024950 wait complete. 2023-10-08T01:12:12.952 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T01:12:12.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:13.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.147+0000 7ff273fff700 1 Processor -- start 2023-10-08T01:12:13.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.147+0000 7ff273fff700 1 -- start start 2023-10-08T01:12:13.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff273fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff26c0b1850 0x7ff26c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:13.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff273fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff26c0ae680 0x7ff26c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:13.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff273fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c0cb300 con 0x7ff26c0aeb00 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff273fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c06fc30 con 0x7ff26c0ae680 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff273fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c0cc260 con 0x7ff26c0b1850 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff27159a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff26c0ae680 0x7ff26c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff27159a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff26c0ae680 0x7ff26c1572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55234/0 (socket says 172.21.15.119:55234) 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff27159a700 1 -- 172.21.15.119:0/3011446494 learned_addr learned my addr 172.21.15.119:0/3011446494 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff271d9b700 1 --2- 172.21.15.119:0/3011446494 >> v2:172.21.15.184:3300/0 conn(0x7ff26c0b1850 0x7ff26c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff27159a700 1 -- 172.21.15.119:0/3011446494 >> v2:172.21.15.184:3300/0 conn(0x7ff26c0b1850 msgr2=0x7ff26c0b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff27159a700 1 --2- 172.21.15.119:0/3011446494 >> v2:172.21.15.184:3300/0 conn(0x7ff26c0b1850 0x7ff26c0b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff27159a700 1 -- 172.21.15.119:0/3011446494 >> v1:172.21.15.184:6789/0 conn(0x7ff26c0aeb00 legacy=0x7ff26c15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.149+0000 7ff27159a700 1 -- 172.21.15.119:0/3011446494 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c0cb5e0 con 0x7ff26c0ae680 2023-10-08T01:12:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.150+0000 7ff27159a700 1 --2- 172.21.15.119:0/3011446494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0ae680 0x7ff26c1572e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0641e0 tx=0x7ff25c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d25cb6c04075d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:13.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.150+0000 7ff270d99700 1 -- 172.21.15.119:0/3011446494 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00b820 con 0x7ff26c0ae680 2023-10-08T01:12:13.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.150+0000 7ff270d99700 1 -- 172.21.15.119:0/3011446494 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c00b980 con 0x7ff26c0ae680 2023-10-08T01:12:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.150+0000 7ff270d99700 1 -- 172.21.15.119:0/3011446494 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00bd20 con 0x7ff26c0ae680 2023-10-08T01:12:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.150+0000 7ff273fff700 1 -- 172.21.15.119:0/3011446494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0ae680 msgr2=0x7ff26c1572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.150+0000 7ff273fff700 1 --2- 172.21.15.119:0/3011446494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0ae680 0x7ff26c1572e0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0641e0 tx=0x7ff25c009250 comp rx=0 tx=0).stop 2023-10-08T01:12:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.151+0000 7ff273fff700 1 -- 172.21.15.119:0/3011446494 shutdown_connections 2023-10-08T01:12:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.151+0000 7ff273fff700 1 --2- 172.21.15.119:0/3011446494 >> v2:172.21.15.184:3300/0 conn(0x7ff26c0b1850 0x7ff26c0b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.151+0000 7ff273fff700 1 --2- 172.21.15.119:0/3011446494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0ae680 0x7ff26c1572e0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.151+0000 7ff273fff700 1 -- 172.21.15.119:0/3011446494 >> 172.21.15.119:0/3011446494 conn(0x7ff26c06a320 msgr2=0x7ff26c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.151+0000 7ff273fff700 1 -- 172.21.15.119:0/3011446494 shutdown_connections 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.151+0000 7ff273fff700 1 -- 172.21.15.119:0/3011446494 wait complete. 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.152+0000 7ff273fff700 1 Processor -- start 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.152+0000 7ff273fff700 1 -- start start 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff273fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26c0ae680 0x7ff26c15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff273fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0aeb00 0x7ff26c0a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff273fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26c0b1850 0x7ff26c0a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff273fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0b1250 con 0x7ff26c0aeb00 2023-10-08T01:12:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff273fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0a60a0 con 0x7ff26c0b1850 2023-10-08T01:12:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff273fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c0a6380 con 0x7ff26c0ae680 2023-10-08T01:12:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff27159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0aeb00 0x7ff26c0a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff27159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0aeb00 0x7ff26c0a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55238/0 (socket says 172.21.15.119:55238) 2023-10-08T01:12:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.153+0000 7ff27159a700 1 -- 172.21.15.119:0/2325157741 learned_addr learned my addr 172.21.15.119:0/2325157741 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff271d9b700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26c0ae680 0x7ff26c15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:13.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff27259c700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26c0b1850 0x7ff26c0a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff27159a700 1 -- 172.21.15.119:0/2325157741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26c0ae680 msgr2=0x7ff26c15c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff27159a700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26c0ae680 0x7ff26c15c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff27159a700 1 -- 172.21.15.119:0/2325157741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26c0b1850 msgr2=0x7ff26c0a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff27159a700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26c0b1850 0x7ff26c0a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff27159a700 1 -- 172.21.15.119:0/2325157741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0094f0 con 0x7ff26c0aeb00 2023-10-08T01:12:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff27159a700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0aeb00 0x7ff26c0a54c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009290 tx=0x7ff25c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff261ffb700 1 -- 172.21.15.119:0/2325157741 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c011e20 con 0x7ff26c0aeb00 2023-10-08T01:12:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff261ffb700 1 -- 172.21.15.119:0/2325157741 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c020070 con 0x7ff26c0aeb00 2023-10-08T01:12:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c0a6660 con 0x7ff26c0aeb00 2023-10-08T01:12:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.154+0000 7ff261ffb700 1 -- 172.21.15.119:0/2325157741 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c01b6a0 con 0x7ff26c0aeb00 2023-10-08T01:12:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.155+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c0a97d0 con 0x7ff26c0aeb00 2023-10-08T01:12:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.156+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c04c3c0 con 0x7ff26c0aeb00 2023-10-08T01:12:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.156+0000 7ff261ffb700 1 -- 172.21.15.119:0/2325157741 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff25c022030 con 0x7ff26c0aeb00 2023-10-08T01:12:13.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.158+0000 7ff261ffb700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff250066f30 0x7ff2500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.158+0000 7ff261ffb700 1 -- 172.21.15.119:0/2325157741 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff25c086610 con 0x7ff26c0aeb00 2023-10-08T01:12:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.158+0000 7ff271d9b700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff250066f30 0x7ff2500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.158+0000 7ff271d9b700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff250066f30 0x7ff2500693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff268006fd0 tx=0x7ff268008040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.162+0000 7ff261ffb700 1 -- 172.21.15.119:0/2325157741 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff25c0536f0 con 0x7ff26c0aeb00 2023-10-08T01:12:13.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.398+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff26c04c3c0 con 0x7ff26c0aeb00 2023-10-08T01:12:13.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.398+0000 7ff261ffb700 1 -- 172.21.15.119:0/2325157741 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7ff25c056d10 con 0x7ff26c0aeb00 2023-10-08T01:12:13.398 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:13.399 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3441253316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3441253316},{"type":"v1","addr":"172.21.15.184:6834","nonce":3441253316}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/3613948578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3613948578},{"type":"v1","addr":"172.21.15.119:6833","nonce":3613948578}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/765135141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":765135141},{"type":"v1","addr":"172.21.15.119:6835","nonce":765135141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:02.279676+0000","modified":"2023-10-08T01:12:05.851791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/910230897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":910230897},{"type":"v1","addr":"172.21.15.184:6835","nonce":910230897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T01:12:13.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff250066f30 msgr2=0x7ff2500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:13.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff250066f30 0x7ff2500693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff268006fd0 tx=0x7ff268008040 comp rx=0 tx=0).stop 2023-10-08T01:12:13.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0aeb00 msgr2=0x7ff26c0a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:13.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0aeb00 0x7ff26c0a54c0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff25c009290 tx=0x7ff25c016000 comp rx=0 tx=0).stop 2023-10-08T01:12:13.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 shutdown_connections 2023-10-08T01:12:13.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff250066f30 0x7ff2500693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff26c0ae680 0x7ff26c15c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff26c0b1850 0x7ff26c0a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 --2- 172.21.15.119:0/2325157741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff26c0aeb00 0x7ff26c0a54c0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 >> 172.21.15.119:0/2325157741 conn(0x7ff26c06a320 msgr2=0x7ff26c0ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 shutdown_connections 2023-10-08T01:12:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:13.401+0000 7ff273fff700 1 -- 172.21.15.119:0/2325157741 wait complete. 2023-10-08T01:12:13.402 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 39 2023-10-08T01:12:13.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T01:12:02.279676+0000', 'modified': '2023-10-08T01:12:05.851791+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6708}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6708': {'gid': 6708, 'name': 'b', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6835/910230897', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6833', 'nonce': 910230897}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 910230897}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T01:12:13.822 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T01:12:13.822 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T01:12:14.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.005+0000 7f595959d700 1 Processor -- start 2023-10-08T01:12:14.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.006+0000 7f595959d700 1 -- start start 2023-10-08T01:12:14.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f595959d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5954150180 0x7f5954157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f595959d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59540c7410 0x7f5954069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f595959d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595406e2a0 con 0x7f5954150550 2023-10-08T01:12:14.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f595959d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595406b2e0 con 0x7f59540c7410 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f595959d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59540c78e0 con 0x7f5954150180 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f5958d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59540c7410 0x7f5954069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f5958d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59540c7410 0x7f5954069f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55254/0 (socket says 172.21.15.119:55254) 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f5958d9c700 1 -- 172.21.15.119:0/458642206 learned_addr learned my addr 172.21.15.119:0/458642206 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.007+0000 7f5953fff700 1 --2- 172.21.15.119:0/458642206 >> v2:172.21.15.184:3300/0 conn(0x7f5954150180 0x7f5954157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.008+0000 7f5958d9c700 1 -- 172.21.15.119:0/458642206 >> v2:172.21.15.184:3300/0 conn(0x7f5954150180 msgr2=0x7f5954157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.008+0000 7f5958d9c700 1 --2- 172.21.15.119:0/458642206 >> v2:172.21.15.184:3300/0 conn(0x7f5954150180 0x7f5954157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.008+0000 7f5958d9c700 1 -- 172.21.15.119:0/458642206 >> v1:172.21.15.184:6789/0 conn(0x7f5954150550 legacy=0x7f59540c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.008+0000 7f5958d9c700 1 -- 172.21.15.119:0/458642206 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540c7bc0 con 0x7f59540c7410 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.008+0000 7f5958d9c700 1 --2- 172.21.15.119:0/458642206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59540c7410 0x7f5954069f50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f595406fd20 tx=0x7f594c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203e35d82789bccf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.008+0000 7f5952ffd700 1 -- 172.21.15.119:0/458642206 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00e070 con 0x7f59540c7410 2023-10-08T01:12:14.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.008+0000 7f5952ffd700 1 -- 172.21.15.119:0/458642206 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594c00ba50 con 0x7f59540c7410 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.008+0000 7f5952ffd700 1 -- 172.21.15.119:0/458642206 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c0092e0 con 0x7f59540c7410 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.009+0000 7f595959d700 1 -- 172.21.15.119:0/458642206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59540c7410 msgr2=0x7f5954069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.009+0000 7f595959d700 1 --2- 172.21.15.119:0/458642206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59540c7410 0x7f5954069f50 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f595406fd20 tx=0x7f594c009840 comp rx=0 tx=0).stop 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.009+0000 7f595959d700 1 -- 172.21.15.119:0/458642206 shutdown_connections 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.009+0000 7f595959d700 1 --2- 172.21.15.119:0/458642206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59540c7410 0x7f5954069f50 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.009+0000 7f595959d700 1 --2- 172.21.15.119:0/458642206 >> v2:172.21.15.184:3300/0 conn(0x7f5954150180 0x7f5954157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.009+0000 7f595959d700 1 -- 172.21.15.119:0/458642206 >> 172.21.15.119:0/458642206 conn(0x7f59540cc1f0 msgr2=0x7f59540cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.009+0000 7f595959d700 1 -- 172.21.15.119:0/458642206 shutdown_connections 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.009+0000 7f595959d700 1 -- 172.21.15.119:0/458642206 wait complete. 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.010+0000 7f595959d700 1 Processor -- start 2023-10-08T01:12:14.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.010+0000 7f595959d700 1 -- start start 2023-10-08T01:12:14.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.011+0000 7f595959d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59540c7410 0x7f5954149d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.011+0000 7f595959d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5954150180 0x7f595414c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.011+0000 7f595959d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5954150550 0x7f59540afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.011+0000 7f595959d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540af960 con 0x7f5954150180 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.011+0000 7f595959d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59540c4770 con 0x7f59540c7410 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.011+0000 7f595959d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595414a2a0 con 0x7f5954150550 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.011+0000 7f59537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5954150180 0x7f595414c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f59537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5954150180 0x7f595414c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55266/0 (socket says 172.21.15.119:55266) 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f59537fe700 1 -- 172.21.15.119:0/699349863 learned_addr learned my addr 172.21.15.119:0/699349863 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f5953fff700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59540c7410 0x7f5954149d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f5958d9c700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5954150550 0x7f59540afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f59537fe700 1 -- 172.21.15.119:0/699349863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5954150550 msgr2=0x7f59540afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f59537fe700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5954150550 0x7f59540afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f59537fe700 1 -- 172.21.15.119:0/699349863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59540c7410 msgr2=0x7f5954149d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f59537fe700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59540c7410 0x7f5954149d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f59537fe700 1 -- 172.21.15.119:0/699349863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c009ae0 con 0x7f5954150180 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f59537fe700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5954150180 0x7f595414c750 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f594800c590 tx=0x7f594800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f593bfff700 1 -- 172.21.15.119:0/699349863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948011590 con 0x7f5954150180 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f593bfff700 1 -- 172.21.15.119:0/699349863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59480116f0 con 0x7f5954150180 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f595959d700 1 -- 172.21.15.119:0/699349863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59540b1370 con 0x7f5954150180 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f593bfff700 1 -- 172.21.15.119:0/699349863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948011ab0 con 0x7f5954150180 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.012+0000 7f595959d700 1 -- 172.21.15.119:0/699349863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59540af030 con 0x7f5954150180 2023-10-08T01:12:14.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.015+0000 7f593bfff700 1 -- 172.21.15.119:0/699349863 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f594801b3b0 con 0x7f5954150180 2023-10-08T01:12:14.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.015+0000 7f593bfff700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5940066f30 0x7f59400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.015+0000 7f593bfff700 1 -- 172.21.15.119:0/699349863 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f5948083d10 con 0x7f5954150180 2023-10-08T01:12:14.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.016+0000 7f5953fff700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5940066f30 0x7f59400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.016+0000 7f5953fff700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5940066f30 0x7f59400693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5944006fd0 tx=0x7f5944008040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.017+0000 7f5968f4cb80 1 Processor -- start 2023-10-08T01:12:14.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.018+0000 7f5968f4cb80 1 -- start start 2023-10-08T01:12:14.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.018+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1da17480 0x564c1d9f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.018+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ca5f0 0x564c1da17850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.018+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ccb80 0x564c1d9ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.018+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d923a70 con 0x564c1d9ca5f0 2023-10-08T01:12:14.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.018+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d921270 con 0x564c1da17480 2023-10-08T01:12:14.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.018+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c1d91d060 con 0x564c1d9ccb80 2023-10-08T01:12:14.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ca5f0 0x564c1da17850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ca5f0 0x564c1da17850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55276/0 (socket says 172.21.15.119:55276) 2023-10-08T01:12:14.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 -- 172.21.15.119:0/2483166708 learned_addr learned my addr 172.21.15.119:0/2483166708 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:14.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f5958d9c700 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ccb80 0x564c1d9ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f5953fff700 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1da17480 0x564c1d9f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 -- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ccb80 msgr2=0x564c1d9ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ccb80 0x564c1d9ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 -- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1da17480 msgr2=0x564c1d9f67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1da17480 0x564c1d9f67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 -- 172.21.15.119:0/2483166708 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c1d7b7270 con 0x564c1d9ca5f0 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f5958d9c700 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ccb80 0x564c1d9ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f5953fff700 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1da17480 0x564c1d9f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:14.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59537fe700 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ca5f0 0x564c1da17850 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f59480549a0 tx=0x7f5948081f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565b1185904bc044 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59397fa700 1 -- 172.21.15.119:0/2483166708 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948057c60 con 0x564c1d9ca5f0 2023-10-08T01:12:14.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59397fa700 1 -- 172.21.15.119:0/2483166708 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594805b9b0 con 0x564c1d9ca5f0 2023-10-08T01:12:14.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.019+0000 7f59397fa700 1 -- 172.21.15.119:0/2483166708 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948052b50 con 0x564c1d9ca5f0 2023-10-08T01:12:14.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2483166708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ca5f0 msgr2=0x564c1da17850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ca5f0 0x564c1da17850 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f59480549a0 tx=0x7f5948081f60 comp rx=0 tx=0).stop 2023-10-08T01:12:14.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2483166708 shutdown_connections 2023-10-08T01:12:14.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ccb80 0x564c1d9ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1da17480 0x564c1d9f67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2483166708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ca5f0 0x564c1da17850 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2483166708 >> 172.21.15.119:0/2483166708 conn(0x564c1d9ce0c0 msgr2=0x564c1d9ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:14.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2483166708 shutdown_connections 2023-10-08T01:12:14.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.020+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2483166708 wait complete. 2023-10-08T01:12:14.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.021+0000 7f5968f4cb80 1 -- start start 2023-10-08T01:12:14.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.021+0000 7f5968f4cb80 1 Processor -- start 2023-10-08T01:12:14.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.022+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 0x564c1d9f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.022+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1d9c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.022+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 0x564c1d9c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.022+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d8c8890 con 0x564c1d9ccb80 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.022+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d8e3ea0 con 0x564c1d9ca5f0 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.022+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c1d79fe30 con 0x564c1da17480 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.022+0000 7f5968f4cb80 10 client.0 _ll_register_callbacks cb 0x564c1d9902c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.022+0000 7f59537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1d9c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1d9c36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55278/0 (socket says 172.21.15.119:55278) 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59537fe700 1 -- 172.21.15.119:0/1898043721 learned_addr learned my addr 172.21.15.119:0/1898043721 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59537fe700 1 -- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 msgr2=0x564c1d9c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59537fe700 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 0x564c1d9c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59537fe700 1 -- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 msgr2=0x564c1d9f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59537fe700 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 0x564c1d9f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59537fe700 1 -- 172.21.15.119:0/1898043721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59540af030 con 0x564c1d9ccb80 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59537fe700 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1d9c36c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f594800a040 tx=0x7f5948068000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfce5d5d2b70f987 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59327fc700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59327fc700 1 -- 172.21.15.119:0/1898043721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594805b520 con 0x564c1d9ccb80 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59327fc700 1 -- 172.21.15.119:0/1898043721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f594805b680 con 0x564c1d9ccb80 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f5968f4cb80 1 -- 172.21.15.119:0/1898043721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c1d9c1cd0 con 0x564c1d9ccb80 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f59327fc700 1 -- 172.21.15.119:0/1898043721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5948011e20 con 0x564c1d9ccb80 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.023+0000 7f5968f4cb80 10 client.6879 fetch_fsmap learned FSMap version 39 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f5968f4cb80 1 -- 172.21.15.119:0/1898043721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x564c1d759fe0 con 0x564c1d9ccb80 2023-10-08T01:12:14.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f59327fc700 1 -- 172.21.15.119:0/1898043721 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564c1d9c1cd0 con 0x564c1d9ccb80 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f59327fc700 1 -- 172.21.15.119:0/1898043721 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5948050800 con 0x564c1d9ccb80 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f5968f4cb80 10 client.6879 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f5968f4cb80 10 client.6879 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f5968f4cb80 20 client.6879 populate_metadata read hostname 'smithi119' 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] conn(0x564c1da128f0 0x564c1da13e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f5968f4cb80 4 client.6879 mds_command: new command op to 6708 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f5968f4cb80 1 -- 172.21.15.119:0/1898043721 --> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c1d99eaa0 con 0x564c1da128f0 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.024+0000 7f5953fff700 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] conn(0x564c1da128f0 0x564c1da13e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.025+0000 7f5953fff700 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] conn(0x564c1da128f0 0x564c1da13e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8869b35485546b65 server_cookie=384b65455a9e93a2 in_seq=0 out_seq=0 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.025+0000 7f59327fc700 10 client.6879 ms_handle_connect on v2:172.21.15.184:6833/910230897 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f59327fc700 1 -- 172.21.15.119:0/1898043721 <== mds.0 v2:172.21.15.184:6833/910230897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x564c1d99eaa0 con 0x564c1da128f0 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f59327fc700 10 client.6879 handle_command_reply: tid=0 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 client.6879 shutdown 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 -- 172.21.15.119:0/1898043721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 msgr2=0x564c1d9c36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1d9c36c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f594800a040 tx=0x7f5948068000 comp rx=0 tx=0).stop 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 -- 172.21.15.119:0/1898043721 shutdown_connections 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] conn(0x564c1da128f0 0x564c1da13e70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 0x564c1d9c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 0x564c1d9f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f59327fc700 0 client.6879 ms_handle_reset on v2:172.21.15.184:6833/910230897 2023-10-08T01:12:14.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/1898043721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1d9c36c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 -- 172.21.15.119:0/1898043721 >> 172.21.15.119:0/1898043721 conn(0x564c1d9ce0c0 msgr2=0x564c1d9ccff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:14.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 -- 172.21.15.119:0/1898043721 shutdown_connections 2023-10-08T01:12:14.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 1 -- 172.21.15.119:0/1898043721 wait complete. 2023-10-08T01:12:14.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.026+0000 7f5968f4cb80 20 client.6879 trim_cache size 0 max 16384 2023-10-08T01:12:14.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.043+0000 7f5968f4cb80 1 Processor -- start 2023-10-08T01:12:14.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.044+0000 7f5968f4cb80 1 -- start start 2023-10-08T01:12:14.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.044+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 0x564c1d9f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.044+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1da73a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.044+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 0x564c1da73f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.044+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d9d0b20 con 0x564c1d9ccb80 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.044+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1da16700 con 0x564c1d9ca5f0 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.044+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c1d9ca1d0 con 0x564c1da17480 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1da73a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1da73a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55292/0 (socket says 172.21.15.119:55292) 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 -- 172.21.15.119:0/3941178205 learned_addr learned my addr 172.21.15.119:0/3941178205 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f5953fff700 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 0x564c1d9f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f5958d9c700 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 0x564c1da73f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 -- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 msgr2=0x564c1da73f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 0x564c1da73f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 -- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 msgr2=0x564c1d9f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 0x564c1d9f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 -- 172.21.15.119:0/3941178205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c1d8eb780 con 0x564c1d9ccb80 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f5953fff700 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 0x564c1d9f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f5958d9c700 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 0x564c1da73f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f59537fe700 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1da73a10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5948000fb0 tx=0x7f594807e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8aadd5430cfab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f5939ffb700 1 -- 172.21.15.119:0/3941178205 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800e040 con 0x564c1d9ccb80 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f5939ffb700 1 -- 172.21.15.119:0/3941178205 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5948011e20 con 0x564c1d9ccb80 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.045+0000 7f5939ffb700 1 -- 172.21.15.119:0/3941178205 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948050800 con 0x564c1d9ccb80 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 -- 172.21.15.119:0/3941178205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 msgr2=0x564c1da73a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1da73a10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5948000fb0 tx=0x7f594807e040 comp rx=0 tx=0).stop 2023-10-08T01:12:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 -- 172.21.15.119:0/3941178205 shutdown_connections 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1da17480 0x564c1da73f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ca5f0 0x564c1d9f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/3941178205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1d9ccb80 0x564c1da73a10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 -- 172.21.15.119:0/3941178205 >> 172.21.15.119:0/3941178205 conn(0x564c1d9ce0c0 msgr2=0x564c1d9cedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 -- 172.21.15.119:0/3941178205 shutdown_connections 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.046+0000 7f5968f4cb80 1 -- 172.21.15.119:0/3941178205 wait complete. 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.047+0000 7f5968f4cb80 1 -- start start 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.047+0000 7f5968f4cb80 1 Processor -- start 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ca5f0 0x564c1da72ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ccb80 0x564c1d9f9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5968f4cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1da17480 0x564c1d9f9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1d9f13e0 con 0x564c1da17480 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c1da7d310 con 0x564c1d9ccb80 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5968f4cb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x564c1da7d480 con 0x564c1d9ca5f0 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5968f4cb80 10 client.0 _ll_register_callbacks cb 0x7f5948055190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5958d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1da17480 0x564c1d9f9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5958d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1da17480 0x564c1d9f9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55294/0 (socket says 172.21.15.119:55294) 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5958d9c700 1 -- 172.21.15.119:0/2850532350 learned_addr learned my addr 172.21.15.119:0/2850532350 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5953fff700 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ca5f0 0x564c1da72ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f59537fe700 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ccb80 0x564c1d9f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5958d9c700 1 -- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ca5f0 msgr2=0x564c1da72ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5958d9c700 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ca5f0 0x564c1da72ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5958d9c700 1 -- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ccb80 msgr2=0x564c1d9f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.048+0000 7f5958d9c700 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ccb80 0x564c1d9f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f5958d9c700 1 -- 172.21.15.119:0/2850532350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c1d759fe0 con 0x564c1da17480 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f5958d9c700 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1da17480 0x564c1d9f9da0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f594c012000 tx=0x7f594c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09a3150f3d176c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f59327fc700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f59327fc700 1 -- 172.21.15.119:0/2850532350 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c007e80 con 0x564c1da17480 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2850532350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f595416b3f0 con 0x564c1da17480 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f59327fc700 1 -- 172.21.15.119:0/2850532350 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f594c00be60 con 0x564c1da17480 2023-10-08T01:12:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f59327fc700 1 -- 172.21.15.119:0/2850532350 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f594c0092e0 con 0x564c1da17480 2023-10-08T01:12:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f5968f4cb80 10 client.6885 fetch_fsmap learned FSMap version 39 2023-10-08T01:12:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2850532350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x564c1d8d2470 con 0x564c1da17480 2023-10-08T01:12:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.049+0000 7f59327fc700 1 -- 172.21.15.119:0/2850532350 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595416b3f0 con 0x564c1da17480 2023-10-08T01:12:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f59327fc700 1 -- 172.21.15.119:0/2850532350 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 39) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f594c00b050 con 0x564c1da17480 2023-10-08T01:12:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f5968f4cb80 10 client.6885 fetch_fsmap finished waiting for FSMap version 39 2023-10-08T01:12:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f5968f4cb80 10 client.6885 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T01:12:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f5968f4cb80 20 client.6885 populate_metadata read hostname 'smithi119' 2023-10-08T01:12:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] conn(0x564c1da767a0 0x564c1da76b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f5968f4cb80 4 client.6885 mds_command: new command op to 6708 tid=0[{"prefix": "status"}] 2023-10-08T01:12:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2850532350 --> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f594805b9b0 con 0x564c1da767a0 2023-10-08T01:12:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f5953fff700 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] conn(0x564c1da767a0 0x564c1da76b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f59327fc700 10 client.6885 ms_handle_connect on v2:172.21.15.184:6833/910230897 2023-10-08T01:12:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.050+0000 7f5953fff700 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] conn(0x564c1da767a0 0x564c1da76b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473b7a1686ac1dab server_cookie=dfe30a80660440c9 in_seq=0 out_seq=0 2023-10-08T01:12:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f59327fc700 1 -- 172.21.15.119:0/2850532350 <== mds.0 v2:172.21.15.184:6833/910230897 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x7f594805b9b0 con 0x564c1da767a0 2023-10-08T01:12:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f59327fc700 10 client.6885 handle_command_reply: tid=0 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 client.6885 shutdown 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2850532350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1da17480 msgr2=0x564c1d9f9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1da17480 0x564c1d9f9da0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f594c012000 tx=0x7f594c018000 comp rx=0 tx=0).stop 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2850532350 shutdown_connections 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:6833/910230897,v1:172.21.15.184:6835/910230897] conn(0x564c1da767a0 0x564c1da76b70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x564c1d9ca5f0 0x564c1da72ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x564c1da17480 0x564c1d9f9da0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f59327fc700 0 client.6885 ms_handle_reset on v2:172.21.15.184:6833/910230897 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 --2- 172.21.15.119:0/2850532350 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x564c1d9ccb80 0x564c1d9f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2850532350 >> 172.21.15.119:0/2850532350 conn(0x564c1d9ce0c0 msgr2=0x564c1d9ceb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2850532350 shutdown_connections 2023-10-08T01:12:14.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 1 -- 172.21.15.119:0/2850532350 wait complete. 2023-10-08T01:12:14.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.051+0000 7f5968f4cb80 20 client.6885 trim_cache size 0 max 16384 2023-10-08T01:12:14.060 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:12:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T01:12:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T01:12:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 6708, 2023-10-08T01:12:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T01:12:14.060 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T01:12:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T01:12:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 11.214497022, 2023-10-08T01:12:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 39, 2023-10-08T01:12:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 91, 2023-10-08T01:12:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 91, 2023-10-08T01:12:14.061 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 17.013181053 2023-10-08T01:12:14.061 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:12:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 -- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5940066f30 msgr2=0x7f59400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5940066f30 0x7f59400693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5944006fd0 tx=0x7f5944008040 comp rx=0 tx=0).stop 2023-10-08T01:12:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 -- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5954150180 msgr2=0x7f595414c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5954150180 0x7f595414c750 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f594800c590 tx=0x7f594800c950 comp rx=0 tx=0).stop 2023-10-08T01:12:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 -- 172.21.15.119:0/699349863 shutdown_connections 2023-10-08T01:12:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5940066f30 0x7f59400693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5954150550 0x7f59540afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59540c7410 0x7f5954149d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 --2- 172.21.15.119:0/699349863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5954150180 0x7f595414c750 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 -- 172.21.15.119:0/699349863 >> 172.21.15.119:0/699349863 conn(0x7f59540cc1f0 msgr2=0x7f59540c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 -- 172.21.15.119:0/699349863 shutdown_connections 2023-10-08T01:12:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.052+0000 7f595959d700 1 -- 172.21.15.119:0/699349863 wait complete. 2023-10-08T01:12:14.069 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T01:12:14.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.250+0000 7f371a39d700 1 Processor -- start 2023-10-08T01:12:14.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.250+0000 7f371a39d700 1 -- start start 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.251+0000 7f371a39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 0x7f37140b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.251+0000 7f371a39d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37140ad870 0x7f371406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.251+0000 7f371a39d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37140b40f0 con 0x7f37140b3b70 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.251+0000 7f371a39d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3714069640 con 0x7f37140ad870 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.251+0000 7f371a39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3714069920 con 0x7f3714142ea0 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.251+0000 7f3718b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37140ad870 0x7f371406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f3718b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37140ad870 0x7f371406b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55310/0 (socket says 172.21.15.119:55310) 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f3718b9a700 1 -- 172.21.15.119:0/250085221 learned_addr learned my addr 172.21.15.119:0/250085221 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f371939b700 1 --2- 172.21.15.119:0/250085221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 0x7f37140b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f3718b9a700 1 -- 172.21.15.119:0/250085221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 msgr2=0x7f37140b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:14.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f3718b9a700 1 --2- 172.21.15.119:0/250085221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 0x7f37140b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f3718b9a700 1 -- 172.21.15.119:0/250085221 >> v1:172.21.15.184:6789/0 conn(0x7f37140b3b70 legacy=0x7f371406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f3718b9a700 1 -- 172.21.15.119:0/250085221 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3714069c00 con 0x7f37140ad870 2023-10-08T01:12:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f3718b9a700 1 --2- 172.21.15.119:0/250085221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140ad870 0x7f371406b9e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f37040099b0 tx=0x7f3704009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6faef668d0e1422 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.252+0000 7f370bfff700 1 -- 172.21.15.119:0/250085221 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3704009ea0 con 0x7f37140ad870 2023-10-08T01:12:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f370bfff700 1 -- 172.21.15.119:0/250085221 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370400b860 con 0x7f37140ad870 2023-10-08T01:12:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f370bfff700 1 -- 172.21.15.119:0/250085221 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370400bc00 con 0x7f37140ad870 2023-10-08T01:12:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f371a39d700 1 -- 172.21.15.119:0/250085221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140ad870 msgr2=0x7f371406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f371a39d700 1 --2- 172.21.15.119:0/250085221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140ad870 0x7f371406b9e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f37040099b0 tx=0x7f3704009250 comp rx=0 tx=0).stop 2023-10-08T01:12:14.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f371a39d700 1 -- 172.21.15.119:0/250085221 shutdown_connections 2023-10-08T01:12:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f371a39d700 1 --2- 172.21.15.119:0/250085221 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 0x7f37140b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f371a39d700 1 --2- 172.21.15.119:0/250085221 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140ad870 0x7f371406b9e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f371a39d700 1 -- 172.21.15.119:0/250085221 >> 172.21.15.119:0/250085221 conn(0x7f37140ce430 msgr2=0x7f37140ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f371a39d700 1 -- 172.21.15.119:0/250085221 shutdown_connections 2023-10-08T01:12:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.253+0000 7f371a39d700 1 -- 172.21.15.119:0/250085221 wait complete. 2023-10-08T01:12:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.255+0000 7f371a39d700 1 Processor -- start 2023-10-08T01:12:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.255+0000 7f371a39d700 1 -- start start 2023-10-08T01:12:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f371a39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37140b3b70 0x7f3714158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f371a39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 0x7f37140c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f371a39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140af610 0x7f37140afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f371a39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37140c8900 con 0x7f37140af610 2023-10-08T01:12:14.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f371a39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37140c7ff0 con 0x7f37140b3b70 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f371a39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37140b00d0 con 0x7f3714142ea0 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3719b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140af610 0x7f37140afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3719b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140af610 0x7f37140afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55322/0 (socket says 172.21.15.119:55322) 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3719b9c700 1 -- 172.21.15.119:0/2650672940 learned_addr learned my addr 172.21.15.119:0/2650672940 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3718b9a700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 0x7f37140c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f371939b700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37140b3b70 0x7f3714158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3719b9c700 1 -- 172.21.15.119:0/2650672940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 msgr2=0x7f37140c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3719b9c700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 0x7f37140c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3719b9c700 1 -- 172.21.15.119:0/2650672940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37140b3b70 msgr2=0x7f3714158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3719b9c700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37140b3b70 0x7f3714158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.256+0000 7f3719b9c700 1 -- 172.21.15.119:0/2650672940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37040094f0 con 0x7f37140af610 2023-10-08T01:12:14.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.257+0000 7f3719b9c700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140af610 0x7f37140afa90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c8e0 tx=0x7f370c001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.257+0000 7f37097fa700 1 -- 172.21.15.119:0/2650672940 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c012630 con 0x7f37140af610 2023-10-08T01:12:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.257+0000 7f37097fa700 1 -- 172.21.15.119:0/2650672940 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f370c012790 con 0x7f37140af610 2023-10-08T01:12:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.257+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37140c82b0 con 0x7f37140af610 2023-10-08T01:12:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.257+0000 7f37097fa700 1 -- 172.21.15.119:0/2650672940 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f370c012b50 con 0x7f37140af610 2023-10-08T01:12:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.257+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37140aef10 con 0x7f37140af610 2023-10-08T01:12:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.258+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f371404c3c0 con 0x7f37140af610 2023-10-08T01:12:14.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.259+0000 7f37097fa700 1 -- 172.21.15.119:0/2650672940 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f370c01a3a0 con 0x7f37140af610 2023-10-08T01:12:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.260+0000 7f37097fa700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36f8066f30 0x7f36f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.260+0000 7f37097fa700 1 -- 172.21.15.119:0/2650672940 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f370c084830 con 0x7f37140af610 2023-10-08T01:12:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.260+0000 7f371939b700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36f8066f30 0x7f36f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.261+0000 7f371939b700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36f8066f30 0x7f36f80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3710009d10 tx=0x7f37100091a0 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:14.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.264+0000 7f37097fa700 1 -- 172.21.15.119:0/2650672940 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f370c0553a0 con 0x7f37140af610 2023-10-08T01:12:14.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.496+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3714143270 con 0x7f37140af610 2023-10-08T01:12:14.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.496+0000 7f37097fa700 1 -- 172.21.15.119:0/2650672940 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f370c054f30 con 0x7f37140af610 2023-10-08T01:12:14.497 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T01:12:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36f8066f30 msgr2=0x7f36f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36f8066f30 0x7f36f80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3710009d10 tx=0x7f37100091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140af610 msgr2=0x7f37140afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140af610 0x7f37140afa90 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f370c00c8e0 tx=0x7f370c001710 comp rx=0 tx=0).stop 2023-10-08T01:12:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 shutdown_connections 2023-10-08T01:12:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f36f8066f30 0x7f36f80693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37140af610 0x7f37140afa90 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37140b3b70 0x7f3714158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 --2- 172.21.15.119:0/2650672940 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3714142ea0 0x7f37140c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.498+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 >> 172.21.15.119:0/2650672940 conn(0x7f37140ce430 msgr2=0x7f3714141d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.499+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 shutdown_connections 2023-10-08T01:12:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:14.499+0000 7f371a39d700 1 -- 172.21.15.119:0/2650672940 wait complete. 2023-10-08T01:12:14.516 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T01:12:14.517 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T01:12:14.517 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T01:12:14.517 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T01:12:14.517 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:14.517 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T01:12:14.517 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T01:12:14.518 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T01:12:14.518 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:12:14.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:14.518 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T01:12:14.555 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T01:12:14.555 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T01:12:14.555 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T01:12:14.555 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:14.556 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T01:12:14.556 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:14.556 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:12:14.556 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:14.556 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T01:12:14.556 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:12:14.556 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:14.557 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T01:12:14.557 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:12:14.557 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:14.557 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T01:12:14.557 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:14.557 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T01:12:14.558 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41317sec preferred_lft 41317sec 2023-10-08T01:12:14.558 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T01:12:14.558 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:14.558 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T01:12:14.558 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:14.558 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T01:12:14.559 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:14.559 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T01:12:14.559 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:14.559 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:12:14.559 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T01:12:14.559 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T01:12:14.560 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:14.560 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:12:14.560 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T01:12:14.560 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T01:12:14.560 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:14.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:14.561 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T01:12:14.655 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T01:12:14.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:14.658 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T01:12:14.696 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T01:12:14.696 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T01:12:14.696 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T01:12:14.697 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T01:12:14.697 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T01:12:14.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:14.698 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T01:12:14.698 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T01:12:14.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T01:12:14.699 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T01:12:14.699 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T01:12:14.699 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T01:12:14.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:14.893 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T01:12:14.930 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T01:12:14.930 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T01:12:14.930 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:12:14.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:14.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T01:12:15.032 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:12:15.032 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:15.069 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:12:15.069 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:12:15.069 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:15.145 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T01:12:15.145 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T01:12:15.146 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:12:15.276 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:12:15.276 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:12:15.304 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:12:15.304 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:12:15.304 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T01:12:15.305 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T01:12:15.305 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T01:12:15.308 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T01:12:15.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:15.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:15.370 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T01:12:15.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.551+0000 7fdd877ff700 1 Processor -- start 2023-10-08T01:12:15.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.551+0000 7fdd877ff700 1 -- start start 2023-10-08T01:12:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.552+0000 7fdd877ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 0x7fdd800ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.552+0000 7fdd877ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800cc300 con 0x7fdd800ccf90 2023-10-08T01:12:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.552+0000 7fdd877ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800ad370 con 0x7fdd800cbf30 2023-10-08T01:12:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.552+0000 7fdd877ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80156bf0 con 0x7fdd800b1420 2023-10-08T01:12:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.552+0000 7fdd85d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdd800ccf90 0x7fdd800c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58174/0 (socket says 172.21.15.119:58174) 2023-10-08T01:12:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.552+0000 7fdd85d9c700 1 -- 172.21.15.119:0/112077018 learned_addr learned my addr 172.21.15.119:0/112077018 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd84d9a700 1 --2- 172.21.15.119:0/112077018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 0x7fdd800ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd77fff700 1 -- 172.21.15.119:0/112077018 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376855309 0 0) 0x7fdd800cc300 con 0x7fdd800ccf90 2023-10-08T01:12:15.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd77fff700 1 -- 172.21.15.119:0/112077018 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd6c003570 con 0x7fdd800ccf90 2023-10-08T01:12:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd77fff700 1 -- 172.21.15.119:0/112077018 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592406871 0 0) 0x7fdd800ad370 con 0x7fdd800cbf30 2023-10-08T01:12:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd77fff700 1 -- 172.21.15.119:0/112077018 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd800cc300 con 0x7fdd800cbf30 2023-10-08T01:12:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd77fff700 1 -- 172.21.15.119:0/112077018 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 651211279 0 0) 0x7fdd6c003570 con 0x7fdd800ccf90 2023-10-08T01:12:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd84d9a700 1 -- 172.21.15.119:0/112077018 >> v1:172.21.15.184:6789/0 conn(0x7fdd800cbf30 legacy=0x7fdd800b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd84d9a700 1 -- 172.21.15.119:0/112077018 >> v1:172.21.15.119:6789/0 conn(0x7fdd800ccf90 legacy=0x7fdd800c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.553+0000 7fdd84d9a700 1 -- 172.21.15.119:0/112077018 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800ad650 con 0x7fdd800b1420 2023-10-08T01:12:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd84d9a700 1 --2- 172.21.15.119:0/112077018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 0x7fdd800ae410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009910 tx=0x7fdd700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0a990937602599a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd77fff700 1 -- 172.21.15.119:0/112077018 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd780017f0 con 0x7fdd800b1420 2023-10-08T01:12:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd77fff700 1 -- 172.21.15.119:0/112077018 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70011420 con 0x7fdd800b1420 2023-10-08T01:12:15.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd77fff700 1 -- 172.21.15.119:0/112077018 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700117c0 con 0x7fdd800b1420 2023-10-08T01:12:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd877ff700 1 -- 172.21.15.119:0/112077018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 msgr2=0x7fdd800ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd877ff700 1 --2- 172.21.15.119:0/112077018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 0x7fdd800ae410 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdd70009910 tx=0x7fdd700091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd877ff700 1 -- 172.21.15.119:0/112077018 shutdown_connections 2023-10-08T01:12:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd877ff700 1 --2- 172.21.15.119:0/112077018 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 0x7fdd800ae410 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.554+0000 7fdd877ff700 1 -- 172.21.15.119:0/112077018 >> 172.21.15.119:0/112077018 conn(0x7fdd8006a320 msgr2=0x7fdd800cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:15.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.555+0000 7fdd877ff700 1 -- 172.21.15.119:0/112077018 shutdown_connections 2023-10-08T01:12:15.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.555+0000 7fdd877ff700 1 -- 172.21.15.119:0/112077018 wait complete. 2023-10-08T01:12:15.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.556+0000 7fdd877ff700 1 Processor -- start 2023-10-08T01:12:15.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.556+0000 7fdd877ff700 1 -- start start 2023-10-08T01:12:15.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd877ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 0x7fdd80159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd877ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800cbf30 0x7fdd800ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd877ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800ccf90 0x7fdd800ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd877ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801514e0 con 0x7fdd800ccf90 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd877ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80159b60 con 0x7fdd800cbf30 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd877ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800c5ca0 con 0x7fdd800b1420 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd85d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800ccf90 0x7fdd800ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd85d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800ccf90 0x7fdd800ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43322/0 (socket says 172.21.15.119:43322) 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.557+0000 7fdd85d9c700 1 -- 172.21.15.119:0/1274354981 learned_addr learned my addr 172.21.15.119:0/1274354981 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd85d9c700 1 -- 172.21.15.119:0/1274354981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 msgr2=0x7fdd80159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd85d9c700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 0x7fdd80159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd85d9c700 1 -- 172.21.15.119:0/1274354981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800cbf30 msgr2=0x7fdd800ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd85d9c700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800cbf30 0x7fdd800ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd85d9c700 1 -- 172.21.15.119:0/1274354981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd70009420 con 0x7fdd800ccf90 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd85d9c700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800ccf90 0x7fdd800ca5f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800cc10 tx=0x7fdd78007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd757fa700 1 -- 172.21.15.119:0/1274354981 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78004dc0 con 0x7fdd800ccf90 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd757fa700 1 -- 172.21.15.119:0/1274354981 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7800a700 con 0x7fdd800ccf90 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd80150e90 con 0x7fdd800ccf90 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd757fa700 1 -- 172.21.15.119:0/1274354981 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7800aac0 con 0x7fdd800ccf90 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.558+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd80150ba0 con 0x7fdd800ccf90 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.559+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8004c3c0 con 0x7fdd800ccf90 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.560+0000 7fdd757fa700 1 -- 172.21.15.119:0/1274354981 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd78007150 con 0x7fdd800ccf90 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.561+0000 7fdd757fa700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.561+0000 7fdd757fa700 1 -- 172.21.15.119:0/1274354981 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fdd78084340 con 0x7fdd800ccf90 2023-10-08T01:12:15.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.561+0000 7fdd8559b700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.562+0000 7fdd8559b700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0026f0 tx=0x7fdd7c007990 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:15.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.565+0000 7fdd757fa700 1 -- 172.21.15.119:0/1274354981 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd78051400 con 0x7fdd800ccf90 2023-10-08T01:12:15.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.781+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd800c72e0 con 0x7fdd64066f30 2023-10-08T01:12:15.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.782+0000 7fdd757fa700 1 -- 172.21.15.119:0/1274354981 <== mgr.6516 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fdd800c72e0 con 0x7fdd64066f30 2023-10-08T01:12:15.782 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:12:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T01:12:15.782 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:15.783 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:12:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 msgr2=0x7fdd640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0026f0 tx=0x7fdd7c007990 comp rx=0 tx=0).stop 2023-10-08T01:12:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800ccf90 msgr2=0x7fdd800ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800ccf90 0x7fdd800ca5f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800cc10 tx=0x7fdd78007d20 comp rx=0 tx=0).stop 2023-10-08T01:12:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 shutdown_connections 2023-10-08T01:12:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1420 0x7fdd80159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800ccf90 0x7fdd800ca5f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 --2- 172.21.15.119:0/1274354981 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800cbf30 0x7fdd800ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 >> 172.21.15.119:0/1274354981 conn(0x7fdd8006a320 msgr2=0x7fdd800b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:15.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 shutdown_connections 2023-10-08T01:12:15.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.785+0000 7fdd877ff700 1 -- 172.21.15.119:0/1274354981 wait complete. 2023-10-08T01:12:15.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T01:12:15.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.985+0000 7f3002d9d700 1 Processor -- start 2023-10-08T01:12:15.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.985+0000 7f3002d9d700 1 -- start start 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f3002d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2ffc153900 0x7f2ffc0c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f3002d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 0x7f2ffc06a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f3002d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ffc157130 0x7f2ffc06a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f3002d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ffc06e150 con 0x7f2ffc157130 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f3002d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ffc0c7800 con 0x7f2ffc153900 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f3002d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc0c7ae0 con 0x7f2ffc0c9db0 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f300259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ffc157130 0x7f2ffc06a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f300259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2ffc157130 0x7f2ffc06a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43324/0 (socket says 172.21.15.119:43324) 2023-10-08T01:12:15.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.986+0000 7f300259c700 1 -- 172.21.15.119:0/2649814076 learned_addr learned my addr 172.21.15.119:0/2649814076 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f300159a700 1 --2- 172.21.15.119:0/2649814076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 0x7f2ffc06a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f3001d9b700 1 --2- 172.21.15.119:0/2649814076 >> v2:172.21.15.184:3300/0 conn(0x7f2ffc153900 0x7f2ffc0c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f300259c700 1 -- 172.21.15.119:0/2649814076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 msgr2=0x7f2ffc06a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f300259c700 1 --2- 172.21.15.119:0/2649814076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 0x7f2ffc06a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f300259c700 1 -- 172.21.15.119:0/2649814076 >> v2:172.21.15.184:3300/0 conn(0x7f2ffc153900 msgr2=0x7f2ffc0c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f300259c700 1 --2- 172.21.15.119:0/2649814076 >> v2:172.21.15.184:3300/0 conn(0x7f2ffc153900 0x7f2ffc0c6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f300259c700 1 -- 172.21.15.119:0/2649814076 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ffc0c7dc0 con 0x7f2ffc157130 2023-10-08T01:12:15.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f3001d9b700 1 --2- 172.21.15.119:0/2649814076 >> v2:172.21.15.184:3300/0 conn(0x7f2ffc153900 0x7f2ffc0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:15.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f300259c700 1 --2- 172.21.15.119:0/2649814076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc157130 0x7f2ffc06a540 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc06aed0 tx=0x7f2ff4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6835051541268ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f3000d99700 1 -- 172.21.15.119:0/2649814076 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff400e070 con 0x7f2ffc157130 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f3000d99700 1 -- 172.21.15.119:0/2649814076 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff400ba50 con 0x7f2ffc157130 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.987+0000 7f3000d99700 1 -- 172.21.15.119:0/2649814076 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff40092e0 con 0x7f2ffc157130 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 -- 172.21.15.119:0/2649814076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc157130 msgr2=0x7f2ffc06a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 --2- 172.21.15.119:0/2649814076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc157130 0x7f2ffc06a540 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc06aed0 tx=0x7f2ff4009840 comp rx=0 tx=0).stop 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 -- 172.21.15.119:0/2649814076 shutdown_connections 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 --2- 172.21.15.119:0/2649814076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc157130 0x7f2ffc06a540 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 --2- 172.21.15.119:0/2649814076 >> v2:172.21.15.184:3300/0 conn(0x7f2ffc153900 0x7f2ffc0c6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 --2- 172.21.15.119:0/2649814076 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 0x7f2ffc06a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 -- 172.21.15.119:0/2649814076 >> 172.21.15.119:0/2649814076 conn(0x7f2ffc0cc1e0 msgr2=0x7f2ffc0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 -- 172.21.15.119:0/2649814076 shutdown_connections 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.988+0000 7f3002d9d700 1 -- 172.21.15.119:0/2649814076 wait complete. 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.989+0000 7f3002d9d700 1 Processor -- start 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.990+0000 7f3002d9d700 1 -- start start 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.990+0000 7f3002d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 0x7f2ffc153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.990+0000 7f3002d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc153900 0x7f2ffc1473b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.990+0000 7f3002d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc157130 0x7f2ffc151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.990+0000 7f3002d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc156300 con 0x7f2ffc153900 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.990+0000 7f3002d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc1478f0 con 0x7f2ffc157130 2023-10-08T01:12:15.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.990+0000 7f3002d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc1545c0 con 0x7f2ffc0c9db0 2023-10-08T01:12:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc153900 0x7f2ffc1473b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc153900 0x7f2ffc1473b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43336/0 (socket says 172.21.15.119:43336) 2023-10-08T01:12:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300159a700 1 -- 172.21.15.119:0/159858151 learned_addr learned my addr 172.21.15.119:0/159858151 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f3001d9b700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 0x7f2ffc153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300259c700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc157130 0x7f2ffc151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300159a700 1 -- 172.21.15.119:0/159858151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 msgr2=0x7f2ffc153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300159a700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 0x7f2ffc153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300159a700 1 -- 172.21.15.119:0/159858151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc157130 msgr2=0x7f2ffc151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300159a700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc157130 0x7f2ffc151f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.991+0000 7f300159a700 1 -- 172.21.15.119:0/159858151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff4009ae0 con 0x7f2ffc153900 2023-10-08T01:12:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.992+0000 7f300159a700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc153900 0x7f2ffc1473b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00d890 tx=0x7f2fec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.992+0000 7f2ff1ffb700 1 -- 172.21.15.119:0/159858151 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0125f0 con 0x7f2ffc153900 2023-10-08T01:12:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.992+0000 7f2ff1ffb700 1 -- 172.21.15.119:0/159858151 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec012770 con 0x7f2ffc153900 2023-10-08T01:12:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.992+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc1548a0 con 0x7f2ffc153900 2023-10-08T01:12:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.992+0000 7f2ff1ffb700 1 -- 172.21.15.119:0/159858151 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec012b30 con 0x7f2ffc153900 2023-10-08T01:12:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.992+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc147c80 con 0x7f2ffc153900 2023-10-08T01:12:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.993+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc147ae0 con 0x7f2ffc153900 2023-10-08T01:12:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.994+0000 7f2ff1ffb700 1 -- 172.21.15.119:0/159858151 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 40) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fec01e460 con 0x7f2ffc153900 2023-10-08T01:12:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.995+0000 7f2ff1ffb700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fe0066f30 0x7f2fe00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.995+0000 7f2ff1ffb700 1 -- 172.21.15.119:0/159858151 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f2fec084c30 con 0x7f2ffc153900 2023-10-08T01:12:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.995+0000 7f3001d9b700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fe0066f30 0x7f2fe00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.996+0000 7f3001d9b700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fe0066f30 0x7f2fe00693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006f00 tx=0x7f2ff8008040 comp rx=0 tx=0).ready entity=mgr.6516 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:15.999+0000 7f2ff1ffb700 1 -- 172.21.15.119:0/159858151 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2ffc147ae0 con 0x7f2ffc153900 2023-10-08T01:12:16.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:16.244+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2ffc147ae0 con 0x7f2ffc153900 2023-10-08T01:12:17.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.218+0000 7f2ff1ffb700 1 -- 172.21.15.119:0/159858151 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 41) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f2fec0520e0 con 0x7f2ffc153900 2023-10-08T01:12:17.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.219+0000 7f2ff1ffb700 1 -- 172.21.15.119:0/159858151 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v41) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2fec0557a0 con 0x7f2ffc153900 2023-10-08T01:12:17.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fe0066f30 msgr2=0x7f2fe00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fe0066f30 0x7f2fe00693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8006f00 tx=0x7f2ff8008040 comp rx=0 tx=0).stop 2023-10-08T01:12:17.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc153900 msgr2=0x7f2ffc1473b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc153900 0x7f2ffc1473b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00d890 tx=0x7f2fec008040 comp rx=0 tx=0).stop 2023-10-08T01:12:17.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 shutdown_connections 2023-10-08T01:12:17.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fe0066f30 0x7f2fe00693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2ffc0c9db0 0x7f2ffc153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2ffc157130 0x7f2ffc151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 --2- 172.21.15.119:0/159858151 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ffc153900 0x7f2ffc1473b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 >> 172.21.15.119:0/159858151 conn(0x7f2ffc0cc1e0 msgr2=0x7f2ffc0c10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:17.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 shutdown_connections 2023-10-08T01:12:17.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.222+0000 7f3002d9d700 1 -- 172.21.15.119:0/159858151 wait complete. 2023-10-08T01:12:17.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T01:12:17.367 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:12:17.373 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:12:17.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.441+0000 7f5552f9d700 1 Processor -- start 2023-10-08T01:12:17.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.441+0000 7f5552f9d700 1 -- start start 2023-10-08T01:12:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.442+0000 7f5552f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f554c0b4e70 0x7f554c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.442+0000 7f5552f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c0b16d0 0x7f554c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.442+0000 7f5552f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c0b2080 con 0x7f554c15d240 2023-10-08T01:12:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.442+0000 7f5552f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f554c0cfb00 con 0x7f554c0b4e70 2023-10-08T01:12:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.442+0000 7f5552f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0b5c40 con 0x7f554c0b16d0 2023-10-08T01:12:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f5551f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f554c0b4e70 0x7f554c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f555279c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f554c15d240 0x7f554c15d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58196/0 (socket says 172.21.15.119:58196) 2023-10-08T01:12:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f555279c700 1 -- 172.21.15.119:0/1678447337 learned_addr learned my addr 172.21.15.119:0/1678447337 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f555179a700 1 --2- 172.21.15.119:0/1678447337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c0b16d0 0x7f554c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f5550f99700 1 -- 172.21.15.119:0/1678447337 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049110538 0 0) 0x7f554c0b2080 con 0x7f554c15d240 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f5551f9b700 1 -- 172.21.15.119:0/1678447337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c0b16d0 msgr2=0x7f554c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f5551f9b700 1 --2- 172.21.15.119:0/1678447337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c0b16d0 0x7f554c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f5551f9b700 1 -- 172.21.15.119:0/1678447337 >> v1:172.21.15.119:6789/0 conn(0x7f554c15d240 legacy=0x7f554c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f5551f9b700 1 -- 172.21.15.119:0/1678447337 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0b5f20 con 0x7f554c0b4e70 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.443+0000 7f5551f9b700 1 --2- 172.21.15.119:0/1678447337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b4e70 0x7f554c0ccff0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f55480098e0 tx=0x7f55480091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb2b2f813406797 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.444+0000 7f5550f99700 1 -- 172.21.15.119:0/1678447337 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554800b640 con 0x7f554c0b4e70 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.444+0000 7f5550f99700 1 -- 172.21.15.119:0/1678447337 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554800b7a0 con 0x7f554c0b4e70 2023-10-08T01:12:17.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.444+0000 7f5550f99700 1 -- 172.21.15.119:0/1678447337 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554800bb40 con 0x7f554c0b4e70 2023-10-08T01:12:17.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.444+0000 7f5552f9d700 1 -- 172.21.15.119:0/1678447337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b4e70 msgr2=0x7f554c0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.444+0000 7f5552f9d700 1 --2- 172.21.15.119:0/1678447337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b4e70 0x7f554c0ccff0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f55480098e0 tx=0x7f55480091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.445+0000 7f5552f9d700 1 -- 172.21.15.119:0/1678447337 shutdown_connections 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.445+0000 7f5552f9d700 1 --2- 172.21.15.119:0/1678447337 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b4e70 0x7f554c0ccff0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.445+0000 7f5552f9d700 1 --2- 172.21.15.119:0/1678447337 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c0b16d0 0x7f554c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.445+0000 7f5552f9d700 1 -- 172.21.15.119:0/1678447337 >> 172.21.15.119:0/1678447337 conn(0x7f554c0d10c0 msgr2=0x7f554c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.445+0000 7f5552f9d700 1 -- 172.21.15.119:0/1678447337 shutdown_connections 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.445+0000 7f5552f9d700 1 -- 172.21.15.119:0/1678447337 wait complete. 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.446+0000 7f5552f9d700 1 Processor -- start 2023-10-08T01:12:17.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.446+0000 7f5552f9d700 1 -- start start 2023-10-08T01:12:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5552f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b16d0 0x7f554c0c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5552f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f554c0b4e70 0x7f554c157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5552f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c15d240 0x7f554c15c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5552f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0ca140 con 0x7f554c0b16d0 2023-10-08T01:12:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5552f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c158750 con 0x7f554c0b4e70 2023-10-08T01:12:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5552f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0c74b0 con 0x7f554c15d240 2023-10-08T01:12:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5551f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b16d0 0x7f554c0c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5551f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b16d0 0x7f554c0c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43378/0 (socket says 172.21.15.119:43378) 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.447+0000 7f5551f9b700 1 -- 172.21.15.119:0/2802954269 learned_addr learned my addr 172.21.15.119:0/2802954269 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f555179a700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f554c0b4e70 0x7f554c157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f555279c700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c15d240 0x7f554c15c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5551f9b700 1 -- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c15d240 msgr2=0x7f554c15c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5551f9b700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c15d240 0x7f554c15c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5551f9b700 1 -- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f554c0b4e70 msgr2=0x7f554c157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5551f9b700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f554c0b4e70 0x7f554c157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5551f9b700 1 -- 172.21.15.119:0/2802954269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5548009420 con 0x7f554c0b16d0 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f555279c700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c15d240 0x7f554c15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5551f9b700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b16d0 0x7f554c0c6e20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f55480093f0 tx=0x7f5548017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5541ffb700 1 -- 172.21.15.119:0/2802954269 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554801f070 con 0x7f554c0b16d0 2023-10-08T01:12:17.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5541ffb700 1 -- 172.21.15.119:0/2802954269 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554800b910 con 0x7f554c0b16d0 2023-10-08T01:12:17.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5541ffb700 1 -- 172.21.15.119:0/2802954269 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548011420 con 0x7f554c0b16d0 2023-10-08T01:12:17.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.448+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c0c97a0 con 0x7f554c0b16d0 2023-10-08T01:12:17.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.449+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554c0c9d60 con 0x7f554c0b16d0 2023-10-08T01:12:17.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.450+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554c0ca3b0 con 0x7f554c0b16d0 2023-10-08T01:12:17.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.451+0000 7f5541ffb700 1 -- 172.21.15.119:0/2802954269 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 41) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f5548011810 con 0x7f554c0b16d0 2023-10-08T01:12:17.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.452+0000 7f5541ffb700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5530066f80 0x7f5530069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.453+0000 7f5541ffb700 1 -- 172.21.15.119:0/2802954269 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f5548085aa0 con 0x7f554c0b16d0 2023-10-08T01:12:17.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.453+0000 7f555179a700 1 -- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5530066f80 msgr2=0x7f5530069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:17.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.453+0000 7f555179a700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5530066f80 0x7f5530069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:17.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.457+0000 7f5541ffb700 1 -- 172.21.15.119:0/2802954269 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5548052b80 con 0x7f554c0b16d0 2023-10-08T01:12:17.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.654+0000 7f555179a700 1 -- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5530066f80 msgr2=0x7f5530069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:17.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.654+0000 7f555179a700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5530066f80 0x7f5530069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:17.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.659+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f554c04bb80 con 0x7f554c0b16d0 2023-10-08T01:12:17.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.664+0000 7f5541ffb700 1 -- 172.21.15.119:0/2802954269 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f55480561a0 con 0x7f554c0b16d0 2023-10-08T01:12:17.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5530066f80 msgr2=0x7f5530069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:17.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5530066f80 0x7f5530069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b16d0 msgr2=0x7f554c0c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b16d0 0x7f554c0c6e20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f55480093f0 tx=0x7f5548017000 comp rx=0 tx=0).stop 2023-10-08T01:12:17.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 shutdown_connections 2023-10-08T01:12:17.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5530066f80 0x7f5530069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f554c15d240 0x7f554c15c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f554c0b16d0 0x7f554c0c6e20 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 --2- 172.21.15.119:0/2802954269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f554c0b4e70 0x7f554c157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 >> 172.21.15.119:0/2802954269 conn(0x7f554c0d10c0 msgr2=0x7f554c069ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:17.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 shutdown_connections 2023-10-08T01:12:17.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.666+0000 7f5552f9d700 1 -- 172.21.15.119:0/2802954269 wait complete. 2023-10-08T01:12:17.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T01:12:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.876+0000 7f68a2d9d700 1 Processor -- start 2023-10-08T01:12:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.876+0000 7f68a2d9d700 1 -- start start 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.877+0000 7f68a2d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f689c06e470 0x7f689c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.877+0000 7f68a2d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c0aea70 con 0x7f689c0aef00 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.877+0000 7f68a2d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689c15dad0 con 0x7f689c15a3b0 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.877+0000 7f68a2d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f689c06ee30 con 0x7f689c06e470 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a159a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f689c0aef00 0x7f689c15d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58200/0 (socket says 172.21.15.119:58200) 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a159a700 1 -- 172.21.15.119:0/4213614686 learned_addr learned my addr 172.21.15.119:0/4213614686 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a259c700 1 --2- 172.21.15.119:0/4213614686 >> v2:172.21.15.184:3300/0 conn(0x7f689c06e470 0x7f689c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a0d99700 1 -- 172.21.15.119:0/4213614686 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 47581161 0 0) 0x7f689c0aea70 con 0x7f689c0aef00 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a0d99700 1 -- 172.21.15.119:0/4213614686 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6888003570 con 0x7f689c0aef00 2023-10-08T01:12:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a259c700 1 -- 172.21.15.119:0/4213614686 >> v1:172.21.15.184:6789/0 conn(0x7f689c15a3b0 legacy=0x7f689c15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a0d99700 1 -- 172.21.15.119:0/4213614686 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3472898136 0 0) 0x7f689c15dad0 con 0x7f689c15a3b0 2023-10-08T01:12:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a259c700 1 -- 172.21.15.119:0/4213614686 >> v1:172.21.15.119:6789/0 conn(0x7f689c0aef00 legacy=0x7f689c15d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a259c700 1 -- 172.21.15.119:0/4213614686 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c0ae480 con 0x7f689c06e470 2023-10-08T01:12:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.878+0000 7f68a259c700 1 --2- 172.21.15.119:0/4213614686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c06e470 0x7f689c06e8f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6894009e60 tx=0x7f6894009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669bbe4b5b67142c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.879+0000 7f68a0d99700 1 -- 172.21.15.119:0/4213614686 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689400e070 con 0x7f689c06e470 2023-10-08T01:12:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.879+0000 7f68a0d99700 1 -- 172.21.15.119:0/4213614686 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689400bb60 con 0x7f689c06e470 2023-10-08T01:12:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.879+0000 7f68a0d99700 1 -- 172.21.15.119:0/4213614686 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894012430 con 0x7f689c06e470 2023-10-08T01:12:17.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.879+0000 7f68a2d9d700 1 -- 172.21.15.119:0/4213614686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c06e470 msgr2=0x7f689c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.879+0000 7f68a2d9d700 1 --2- 172.21.15.119:0/4213614686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c06e470 0x7f689c06e8f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6894009e60 tx=0x7f6894009780 comp rx=0 tx=0).stop 2023-10-08T01:12:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.879+0000 7f68a2d9d700 1 -- 172.21.15.119:0/4213614686 shutdown_connections 2023-10-08T01:12:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.879+0000 7f68a2d9d700 1 --2- 172.21.15.119:0/4213614686 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c06e470 0x7f689c06e8f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.879+0000 7f68a2d9d700 1 -- 172.21.15.119:0/4213614686 >> 172.21.15.119:0/4213614686 conn(0x7f689c0c9f70 msgr2=0x7f689c06dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.880+0000 7f68a2d9d700 1 -- 172.21.15.119:0/4213614686 shutdown_connections 2023-10-08T01:12:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.880+0000 7f68a2d9d700 1 -- 172.21.15.119:0/4213614686 wait complete. 2023-10-08T01:12:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.881+0000 7f68a2d9d700 1 Processor -- start 2023-10-08T01:12:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.881+0000 7f68a2d9d700 1 -- start start 2023-10-08T01:12:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.882+0000 7f68a2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f689c06e470 0x7f689c0b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.882+0000 7f68a2d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f689c0aef00 0x7f689c1432e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.882+0000 7f68a2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c15a3b0 0x7f689c1426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.882+0000 7f68a2d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c0b1670 con 0x7f689c06e470 2023-10-08T01:12:17.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.882+0000 7f68a2d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c143820 con 0x7f689c15a3b0 2023-10-08T01:12:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.882+0000 7f68a2d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689c153720 con 0x7f689c0aef00 2023-10-08T01:12:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.882+0000 7f68a1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f689c06e470 0x7f689c0b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a159a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f689c0aef00 0x7f689c1432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f689c06e470 0x7f689c0b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43390/0 (socket says 172.21.15.119:43390) 2023-10-08T01:12:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a1d9b700 1 -- 172.21.15.119:0/2333614500 learned_addr learned my addr 172.21.15.119:0/2333614500 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a259c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c15a3b0 0x7f689c1426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:17.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a1d9b700 1 -- 172.21.15.119:0/2333614500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f689c0aef00 msgr2=0x7f689c1432e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a1d9b700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f689c0aef00 0x7f689c1432e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a1d9b700 1 -- 172.21.15.119:0/2333614500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c15a3b0 msgr2=0x7f689c1426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a1d9b700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c15a3b0 0x7f689c1426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a1d9b700 1 -- 172.21.15.119:0/2333614500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6894009a20 con 0x7f689c06e470 2023-10-08T01:12:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.883+0000 7f68a1d9b700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f689c06e470 0x7f689c0b03d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f689800c530 tx=0x7f689800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:17.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.884+0000 7f6891ffb700 1 -- 172.21.15.119:0/2333614500 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898011590 con 0x7f689c06e470 2023-10-08T01:12:17.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.884+0000 7f6891ffb700 1 -- 172.21.15.119:0/2333614500 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6898011710 con 0x7f689c06e470 2023-10-08T01:12:17.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.884+0000 7f6891ffb700 1 -- 172.21.15.119:0/2333614500 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898011ad0 con 0x7f689c06e470 2023-10-08T01:12:17.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.884+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c0b0a60 con 0x7f689c06e470 2023-10-08T01:12:17.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.884+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c0b1020 con 0x7f689c06e470 2023-10-08T01:12:17.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.885+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689c06cfa0 con 0x7f689c06e470 2023-10-08T01:12:17.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.887+0000 7f6891ffb700 1 -- 172.21.15.119:0/2333614500 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 41) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f689801b3b0 con 0x7f689c06e470 2023-10-08T01:12:17.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.887+0000 7f6891ffb700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6880066f80 0x7f6880069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:17.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.887+0000 7f6891ffb700 1 -- 172.21.15.119:0/2333614500 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f6898083b60 con 0x7f689c06e470 2023-10-08T01:12:17.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.887+0000 7f68a159a700 1 -- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6880066f80 msgr2=0x7f6880069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:17.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.887+0000 7f68a159a700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6880066f80 0x7f6880069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:17.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:17.891+0000 7f6891ffb700 1 -- 172.21.15.119:0/2333614500 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6898050c40 con 0x7f689c06e470 2023-10-08T01:12:18.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.087+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f689c15dd30 con 0x7f689c06e470 2023-10-08T01:12:18.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.088+0000 7f68a159a700 1 -- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6880066f80 msgr2=0x7f6880069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:18.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.088+0000 7f68a159a700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6880066f80 0x7f6880069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.093+0000 7f6891ffb700 1 -- 172.21.15.119:0/2333614500 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6898017070 con 0x7f689c06e470 2023-10-08T01:12:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6880066f80 msgr2=0x7f6880069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6880066f80 0x7f6880069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f689c06e470 msgr2=0x7f689c0b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f689c06e470 0x7f689c0b03d0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f689800c530 tx=0x7f689800c840 comp rx=0 tx=0).stop 2023-10-08T01:12:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 shutdown_connections 2023-10-08T01:12:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6880066f80 0x7f6880069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f689c15a3b0 0x7f689c1426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f689c06e470 0x7f689c0b03d0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 --2- 172.21.15.119:0/2333614500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f689c0aef00 0x7f689c1432e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.095+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 >> 172.21.15.119:0/2333614500 conn(0x7f689c0c9f70 msgr2=0x7f689c15c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:18.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.096+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 shutdown_connections 2023-10-08T01:12:18.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.096+0000 7f68a2d9d700 1 -- 172.21.15.119:0/2333614500 wait complete. 2023-10-08T01:12:18.114 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-08T01:12:18.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.305+0000 7f2e2759e700 1 Processor -- start 2023-10-08T01:12:18.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.305+0000 7f2e2759e700 1 -- start start 2023-10-08T01:12:18.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.306+0000 7f2e2759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28153480 0x7f2e280ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:18.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.306+0000 7f2e2759e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e28159dc0 0x7f2e28157280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:18.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.306+0000 7f2e2759e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e280aec90 con 0x7f2e28153850 2023-10-08T01:12:18.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.306+0000 7f2e2759e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2815a480 con 0x7f2e28159dc0 2023-10-08T01:12:18.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.306+0000 7f2e2759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280afb40 con 0x7f2e28153480 2023-10-08T01:12:18.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2659c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28153480 0x7f2e280ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:18.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e26d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e28159dc0 0x7f2e28157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:18.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2659c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28153480 0x7f2e280ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:34968/0 (socket says 172.21.15.119:34968) 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e26d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2e28159dc0 0x7f2e28157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:43692/0 (socket says 172.21.15.119:43692) 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2659c700 1 -- 172.21.15.119:0/3494391268 learned_addr learned my addr 172.21.15.119:0/3494391268 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2559a700 1 -- 172.21.15.119:0/3494391268 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1856175121 0 0) 0x7f2e280aec90 con 0x7f2e28153850 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2559a700 1 -- 172.21.15.119:0/3494391268 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e10003570 con 0x7f2e28153850 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2659c700 1 -- 172.21.15.119:0/3494391268 >> v2:172.21.15.184:3300/0 conn(0x7f2e28159dc0 msgr2=0x7f2e28157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3494391268 >> v2:172.21.15.184:3300/0 conn(0x7f2e28159dc0 0x7f2e28157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2659c700 1 -- 172.21.15.119:0/3494391268 >> v1:172.21.15.184:6789/0 conn(0x7f2e28153850 legacy=0x7f2e281596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2659c700 1 -- 172.21.15.119:0/3494391268 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e280afe20 con 0x7f2e28153480 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e26d9d700 1 --2- 172.21.15.119:0/3494391268 >> v2:172.21.15.184:3300/0 conn(0x7f2e28159dc0 0x7f2e28157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.307+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3494391268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28153480 0x7f2e280ae750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800aa50 tx=0x7f2e1800a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d6abafe9ec7f2e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2559a700 1 -- 172.21.15.119:0/3494391268 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800ae90 con 0x7f2e28153480 2023-10-08T01:12:18.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2559a700 1 -- 172.21.15.119:0/3494391268 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1800b750 con 0x7f2e28153480 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2559a700 1 -- 172.21.15.119:0/3494391268 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800bad0 con 0x7f2e28153480 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2759e700 1 -- 172.21.15.119:0/3494391268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28153480 msgr2=0x7f2e280ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2759e700 1 --2- 172.21.15.119:0/3494391268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28153480 0x7f2e280ae750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800aa50 tx=0x7f2e1800a330 comp rx=0 tx=0).stop 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2759e700 1 -- 172.21.15.119:0/3494391268 shutdown_connections 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2759e700 1 --2- 172.21.15.119:0/3494391268 >> v2:172.21.15.184:3300/0 conn(0x7f2e28159dc0 0x7f2e28157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2759e700 1 --2- 172.21.15.119:0/3494391268 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28153480 0x7f2e280ae750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2759e700 1 -- 172.21.15.119:0/3494391268 >> 172.21.15.119:0/3494391268 conn(0x7f2e28068e20 msgr2=0x7f2e280cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2759e700 1 -- 172.21.15.119:0/3494391268 shutdown_connections 2023-10-08T01:12:18.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.308+0000 7f2e2759e700 1 -- 172.21.15.119:0/3494391268 wait complete. 2023-10-08T01:12:18.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.310+0000 7f2e2759e700 1 Processor -- start 2023-10-08T01:12:18.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.310+0000 7f2e2759e700 1 -- start start 2023-10-08T01:12:18.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e2759e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e28153480 0x7f2e280c1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:18.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e2759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e28153850 0x7f2e280c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e2759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28159dc0 0x7f2e281439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e2759e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280c15b0 con 0x7f2e28153850 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e2759e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280c1880 con 0x7f2e28153480 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e2759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280c1b60 con 0x7f2e28159dc0 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e25d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e28153850 0x7f2e280c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e2659c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e28153480 0x7f2e280c1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e25d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e28153850 0x7f2e280c3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43406/0 (socket says 172.21.15.119:43406) 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e26d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28159dc0 0x7f2e281439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.311+0000 7f2e25d9b700 1 -- 172.21.15.119:0/3577352860 learned_addr learned my addr 172.21.15.119:0/3577352860 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e25d9b700 1 -- 172.21.15.119:0/3577352860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28159dc0 msgr2=0x7f2e281439a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:18.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e25d9b700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e28159dc0 0x7f2e281439a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e25d9b700 1 -- 172.21.15.119:0/3577352860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e28153480 msgr2=0x7f2e280c1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:18.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e25d9b700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e28153480 0x7f2e280c1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:18.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e25d9b700 1 -- 172.21.15.119:0/3577352860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2000bc30 con 0x7f2e28153850 2023-10-08T01:12:18.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e28153480 0x7f2e280c1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:18.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e25d9b700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e28153850 0x7f2e280c3d50 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00c5b0 tx=0x7f2e1c007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:18.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c0105c0 con 0x7f2e28153850 2023-10-08T01:12:18.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c010720 con 0x7f2e28153850 2023-10-08T01:12:18.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c010ae0 con 0x7f2e28153850 2023-10-08T01:12:18.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e2759e700 1 -- 172.21.15.119:0/3577352860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e1800a590 con 0x7f2e28153850 2023-10-08T01:12:18.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.312+0000 7f2e2759e700 1 -- 172.21.15.119:0/3577352860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e280c2120 con 0x7f2e28153850 2023-10-08T01:12:18.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.313+0000 7f2e2759e700 1 -- 172.21.15.119:0/3577352860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2804c3c0 con 0x7f2e28153850 2023-10-08T01:12:18.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.314+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 41) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f2e1c0123e0 con 0x7f2e28153850 2023-10-08T01:12:18.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.315+0000 7f2e167fc700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:18.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.315+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f2e1c084250 con 0x7f2e28153850 2023-10-08T01:12:18.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.315+0000 7f2e2659c700 1 -- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 msgr2=0x7f2e08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:18.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.315+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:18.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.319+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e1c051330 con 0x7f2e28153850 2023-10-08T01:12:18.500 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:18.500+0000 7ff397eba180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:12:18.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.516+0000 7f2e2659c700 1 -- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 msgr2=0x7f2e08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:18.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.516+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:18.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.542+0000 7f2e2759e700 1 -- 172.21.15.119:0/3577352860 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e280b0100 con 0x7f2e08066f80 2023-10-08T01:12:18.554 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:18.555+0000 7f272246e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:12:18.663 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:18.662+0000 7ff397eba180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:18.719 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:18.719+0000 7f272246e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:18.748 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:18.748+0000 7ff397eba180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:12:18.806 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:18.806+0000 7f272246e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:12:18.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.917+0000 7f2e2659c700 1 -- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 msgr2=0x7f2e08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:18.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:18.917+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:12:18.978 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:18.978+0000 7ff397eba180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:19.033 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:19.033+0000 7f272246e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:19.154 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:19.153+0000 7ff397eba180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:12:19.210 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:19.210+0000 7f272246e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:12:19.370 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:19.244+0000 7ff397eba180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:12:19.370 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:19.301+0000 7f272246e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:12:19.406 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:19.406+0000 7ff397eba180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:12:19.462 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:19.462+0000 7f272246e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:12:19.506 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:19.505+0000 7ff397eba180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:19.564 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:19.564+0000 7f272246e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:19.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:19.718+0000 7f2e2659c700 1 -- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 msgr2=0x7f2e08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:19.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:19.718+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T01:12:19.989 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:19.988+0000 7ff397eba180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:12:20.049 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:20.049+0000 7f272246e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:12:20.082 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:20.082+0000 7ff397eba180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:12:20.142 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:20.143+0000 7f272246e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:12:20.172 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:20.172+0000 7ff397eba180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:12:20.231 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:20.231+0000 7f272246e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:12:20.792 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:20.791+0000 7ff397eba180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:12:20.860 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:20.857+0000 7f272246e180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:12:20.963 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:20.963+0000 7ff397eba180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:21.027 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:21.027+0000 7f272246e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:21.058 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:21.058+0000 7ff397eba180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:12:21.139 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:21.122+0000 7f272246e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:12:21.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:21.320+0000 7f2e2659c700 1 -- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 msgr2=0x7f2e08069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:21.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:21.320+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T01:12:21.354 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:21.353+0000 7ff397eba180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:12:21.414 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:21.414+0000 7f272246e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:12:21.450 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:21.450+0000 7ff397eba180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:12:21.509 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:21.509+0000 7f272246e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:12:22.152 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:22.152+0000 7ff397eba180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:12:22.208 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:22.208+0000 7f272246e180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:12:22.438 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:22.438+0000 7ff397eba180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:12:22.492 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:22.492+0000 7f272246e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:12:22.548 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:22.531+0000 7ff397eba180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:12:22.585 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:22.585+0000 7f272246e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:12:22.685 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:22.685+0000 7ff397eba180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:12:22.737 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:22.737+0000 7f272246e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:12:22.790 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:22.789+0000 7ff397eba180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:12:22.858 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:22.841+0000 7f272246e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:12:22.977 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:22.977+0000 7ff397eba180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:12:23.030 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.030+0000 7f272246e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:12:23.070 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:23.070+0000 7ff397eba180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:12:23.123 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.123+0000 7f272246e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:12:23.172 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:23.161+0000 7ff397eba180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:12:23.215 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.215+0000 7f272246e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:12:23.256 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:23.256+0000 7ff397eba180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:12:23.309 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.309+0000 7f272246e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:12:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:23.323+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 42) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f2e1c051700 con 0x7f2e28153850 2023-10-08T01:12:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:23.323+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 msgr2=0x7f2e08069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:23.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:23.323+0000 7f2e167fc700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:23.438 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f2696fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.478 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f2696fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.479 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f2696fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.479 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f2696fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.479 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f2696fb6700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.479 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f26947b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.479 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f26947b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.480 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f26947b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.480 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f26947b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:23.480 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:23.438+0000 7f26947b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:24.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.325+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 43) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f2e1c0544e0 con 0x7f2e28153850 2023-10-08T01:12:24.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.326+0000 7f2e167fc700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:24.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.326+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e280b0100 con 0x7f2e08066f80 2023-10-08T01:12:24.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.329+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:24.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.332+0000 7f2e2659c700 1 --2- 172.21.15.119:0/3577352860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2e08066f80 0x7f2e08069430 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800aa50 tx=0x7f2e18019000 comp rx=0 tx=0).ready entity=mgr.6915 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:24.341 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:24.340+0000 7f269e7c5700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-08T01:12:24.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.341+0000 7f2e167fc700 1 -- 172.21.15.119:0/3577352860 <== mgr.6915 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f2e280b0100 con 0x7f2e08066f80 2023-10-08T01:12:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-08T01:12:24.351 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T01:12:24.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T01:12:24.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.617+0000 7f0ce8bff700 1 Processor -- start 2023-10-08T01:12:24.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.618+0000 7f0ce8bff700 1 -- start start 2023-10-08T01:12:24.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce8bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ce406bcd0 0x7f0ce40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:24.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce8bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0ce406c0a0 0x7f0ce415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:24.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce8bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce415d7e0 con 0x7f0ce415df50 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce8bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce415dac0 con 0x7f0ce406bcd0 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce8bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce415aa30 con 0x7f0ce406c0a0 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce2d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0ce415df50 0x7f0ce40c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58210/0 (socket says 172.21.15.119:58210) 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce2d9d700 1 -- 172.21.15.119:0/2671721531 learned_addr learned my addr 172.21.15.119:0/2671721531 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ce406bcd0 0x7f0ce40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce1d9b700 1 --2- 172.21.15.119:0/2671721531 >> v2:172.21.15.184:3300/0 conn(0x7f0ce406c0a0 0x7f0ce415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.619+0000 7f0ce159a700 1 -- 172.21.15.119:0/2671721531 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401593569 0 0) 0x7f0ce415d7e0 con 0x7f0ce415df50 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce159a700 1 -- 172.21.15.119:0/2671721531 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ccc003570 con 0x7f0ce415df50 2023-10-08T01:12:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce259c700 1 -- 172.21.15.119:0/2671721531 >> v2:172.21.15.184:3300/0 conn(0x7f0ce406c0a0 msgr2=0x7f0ce415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce259c700 1 --2- 172.21.15.119:0/2671721531 >> v2:172.21.15.184:3300/0 conn(0x7f0ce406c0a0 0x7f0ce415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce259c700 1 -- 172.21.15.119:0/2671721531 >> v1:172.21.15.119:6789/0 conn(0x7f0ce415df50 legacy=0x7f0ce40c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce259c700 1 -- 172.21.15.119:0/2671721531 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce415ad10 con 0x7f0ce406bcd0 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce1d9b700 1 --2- 172.21.15.119:0/2671721531 >> v2:172.21.15.184:3300/0 conn(0x7f0ce406c0a0 0x7f0ce415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce259c700 1 --2- 172.21.15.119:0/2671721531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce40caa90 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0ce406b4e0 tx=0x7f0cd4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad318554e78dc86a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce159a700 1 -- 172.21.15.119:0/2671721531 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400e070 con 0x7f0ce406bcd0 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce159a700 1 -- 172.21.15.119:0/2671721531 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd400bb60 con 0x7f0ce406bcd0 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.620+0000 7f0ce159a700 1 -- 172.21.15.119:0/2671721531 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4012430 con 0x7f0ce406bcd0 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.621+0000 7f0ce8bff700 1 -- 172.21.15.119:0/2671721531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 msgr2=0x7f0ce40caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.621+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/2671721531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce40caa90 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f0ce406b4e0 tx=0x7f0cd4009780 comp rx=0 tx=0).stop 2023-10-08T01:12:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.621+0000 7f0ce8bff700 1 -- 172.21.15.119:0/2671721531 shutdown_connections 2023-10-08T01:12:24.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.621+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/2671721531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce40caa90 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:24.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.621+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/2671721531 >> v2:172.21.15.184:3300/0 conn(0x7f0ce406c0a0 0x7f0ce415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:24.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.621+0000 7f0ce8bff700 1 -- 172.21.15.119:0/2671721531 >> 172.21.15.119:0/2671721531 conn(0x7f0ce40ce600 msgr2=0x7f0ce40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:24.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.621+0000 7f0ce8bff700 1 -- 172.21.15.119:0/2671721531 shutdown_connections 2023-10-08T01:12:24.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.621+0000 7f0ce8bff700 1 -- 172.21.15.119:0/2671721531 wait complete. 2023-10-08T01:12:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.622+0000 7f0ce8bff700 1 Processor -- start 2023-10-08T01:12:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.622+0000 7f0ce8bff700 1 -- start start 2023-10-08T01:12:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce8bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce414f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce8bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ce406c0a0 0x7f0ce416cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce8bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ce415df50 0x7f0ce4157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce8bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce414fe80 con 0x7f0ce406bcd0 2023-10-08T01:12:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce8bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4152880 con 0x7f0ce406c0a0 2023-10-08T01:12:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce8bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce4151c40 con 0x7f0ce415df50 2023-10-08T01:12:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce414f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce414f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43594/0 (socket says 172.21.15.119:43594) 2023-10-08T01:12:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.623+0000 7f0ce259c700 1 -- 172.21.15.119:0/3813086152 learned_addr learned my addr 172.21.15.119:0/3813086152 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce1d9b700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ce406c0a0 0x7f0ce416cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce2d9d700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ce415df50 0x7f0ce4157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce259c700 1 -- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ce415df50 msgr2=0x7f0ce4157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce259c700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ce415df50 0x7f0ce4157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce259c700 1 -- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ce406c0a0 msgr2=0x7f0ce416cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:24.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce259c700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ce406c0a0 0x7f0ce416cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:24.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce259c700 1 -- 172.21.15.119:0/3813086152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd4009a20 con 0x7f0ce406bcd0 2023-10-08T01:12:24.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce2d9d700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ce415df50 0x7f0ce4157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce259c700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce414f630 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40099f0 tx=0x7f0cd4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0cd27fc700 1 -- 172.21.15.119:0/3813086152 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4012c50 con 0x7f0ce406bcd0 2023-10-08T01:12:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0cd27fc700 1 -- 172.21.15.119:0/3813086152 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd4012dd0 con 0x7f0ce406bcd0 2023-10-08T01:12:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce4151ee0 con 0x7f0ce406bcd0 2023-10-08T01:12:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0cd27fc700 1 -- 172.21.15.119:0/3813086152 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4026640 con 0x7f0ce406bcd0 2023-10-08T01:12:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.624+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce41524a0 con 0x7f0ce406bcd0 2023-10-08T01:12:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.625+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce414fbd0 con 0x7f0ce406bcd0 2023-10-08T01:12:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.626+0000 7f0cd27fc700 1 -- 172.21.15.119:0/3813086152 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 43) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f0cd4026a30 con 0x7f0ce406bcd0 2023-10-08T01:12:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.627+0000 7f0cd27fc700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cc4066dd0 0x7f0cc4069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.627+0000 7f0cd27fc700 1 -- 172.21.15.119:0/3813086152 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f0cd4085f60 con 0x7f0ce406bcd0 2023-10-08T01:12:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.627+0000 7f0ce1d9b700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cc4066dd0 0x7f0cc4069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.628+0000 7f0ce1d9b700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cc4066dd0 0x7f0cc4069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8006f00 tx=0x7f0cd8008040 comp rx=0 tx=0).ready entity=mgr.6915 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.631+0000 7f0cd27fc700 1 -- 172.21.15.119:0/3813086152 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0cd4053120 con 0x7f0ce406bcd0 2023-10-08T01:12:24.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:24.876+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f0ce404c3c0 con 0x7f0ce406bcd0 2023-10-08T01:12:25.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.332+0000 7f0cd27fc700 1 -- 172.21.15.119:0/3813086152 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f0cd40534f0 con 0x7f0ce406bcd0 2023-10-08T01:12:25.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.333+0000 7f0cd27fc700 1 -- 172.21.15.119:0/3813086152 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v44) v1 ==== 93+0+0 (secure 0 0 0) 0x7f0cd4030d00 con 0x7f0ce406bcd0 2023-10-08T01:12:25.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cc4066dd0 msgr2=0x7f0cc4069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:25.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cc4066dd0 0x7f0cc4069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8006f00 tx=0x7f0cd8008040 comp rx=0 tx=0).stop 2023-10-08T01:12:25.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 msgr2=0x7f0ce414f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:25.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce414f630 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f0cd40099f0 tx=0x7f0cd4019000 comp rx=0 tx=0).stop 2023-10-08T01:12:25.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 shutdown_connections 2023-10-08T01:12:25.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cc4066dd0 0x7f0cc4069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:25.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ce415df50 0x7f0ce4157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:25.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ce406bcd0 0x7f0ce414f630 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:25.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 --2- 172.21.15.119:0/3813086152 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ce406c0a0 0x7f0ce416cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:25.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 >> 172.21.15.119:0/3813086152 conn(0x7f0ce40ce600 msgr2=0x7f0ce41585a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:25.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 shutdown_connections 2023-10-08T01:12:25.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.335+0000 7f0ce8bff700 1 -- 172.21.15.119:0/3813086152 wait complete. 2023-10-08T01:12:25.363 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:12:25.363 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:12:25.451 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:12:25.528 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:12:25.540 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T01:12:25.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:12:25.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:25.628 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T01:12:25.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:25.640 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T01:12:25.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.910+0000 7fa0c299d700 1 Processor -- start 2023-10-08T01:12:25.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.910+0000 7fa0c299d700 1 -- start start 2023-10-08T01:12:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c299d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa0bc15d230 0x7fa0bc15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c299d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc0b2980 con 0x7fa0bc159420 2023-10-08T01:12:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c299d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc15db70 con 0x7fa0bc1597f0 2023-10-08T01:12:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c299d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b11d0 con 0x7fa0bc15d230 2023-10-08T01:12:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c199b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa0bc159420 0x7fa0bc0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47890/0 (socket says 172.21.15.119:47890) 2023-10-08T01:12:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c199b700 1 -- 172.21.15.119:0/806916973 learned_addr learned my addr 172.21.15.119:0/806916973 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c219c700 1 --2- 172.21.15.119:0/806916973 >> v2:172.21.15.184:3300/0 conn(0x7fa0bc15d230 0x7fa0bc15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c0999700 1 -- 172.21.15.119:0/806916973 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067956848 0 0) 0x7fa0bc0b2980 con 0x7fa0bc159420 2023-10-08T01:12:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.912+0000 7fa0c0999700 1 -- 172.21.15.119:0/806916973 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a8003570 con 0x7fa0bc159420 2023-10-08T01:12:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.913+0000 7fa0c219c700 1 -- 172.21.15.119:0/806916973 >> v1:172.21.15.184:6789/0 conn(0x7fa0bc1597f0 legacy=0x7fa0bc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.913+0000 7fa0c219c700 1 -- 172.21.15.119:0/806916973 >> v1:172.21.15.119:6789/0 conn(0x7fa0bc159420 legacy=0x7fa0bc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.913+0000 7fa0c219c700 1 -- 172.21.15.119:0/806916973 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0b14b0 con 0x7fa0bc15d230 2023-10-08T01:12:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.913+0000 7fa0c219c700 1 --2- 172.21.15.119:0/806916973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 0x7fa0bc15d630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009e60 tx=0x7fa0b4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb51675bed6f55a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.913+0000 7fa0c0999700 1 -- 172.21.15.119:0/806916973 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b400e070 con 0x7fa0bc15d230 2023-10-08T01:12:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.913+0000 7fa0c0999700 1 -- 172.21.15.119:0/806916973 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b400bb60 con 0x7fa0bc15d230 2023-10-08T01:12:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.913+0000 7fa0c0999700 1 -- 172.21.15.119:0/806916973 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4012430 con 0x7fa0bc15d230 2023-10-08T01:12:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.914+0000 7fa0c299d700 1 -- 172.21.15.119:0/806916973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 msgr2=0x7fa0bc15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.914+0000 7fa0c299d700 1 --2- 172.21.15.119:0/806916973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 0x7fa0bc15d630 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009e60 tx=0x7fa0b4009780 comp rx=0 tx=0).stop 2023-10-08T01:12:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.914+0000 7fa0c299d700 1 -- 172.21.15.119:0/806916973 shutdown_connections 2023-10-08T01:12:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.914+0000 7fa0c299d700 1 --2- 172.21.15.119:0/806916973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 0x7fa0bc15d630 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.914+0000 7fa0c299d700 1 -- 172.21.15.119:0/806916973 >> 172.21.15.119:0/806916973 conn(0x7fa0bc0d0660 msgr2=0x7fa0bc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:25.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.914+0000 7fa0c299d700 1 -- 172.21.15.119:0/806916973 shutdown_connections 2023-10-08T01:12:25.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.914+0000 7fa0c299d700 1 -- 172.21.15.119:0/806916973 wait complete. 2023-10-08T01:12:25.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.915+0000 7fa0c299d700 1 Processor -- start 2023-10-08T01:12:25.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.915+0000 7fa0c299d700 1 -- start start 2023-10-08T01:12:25.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.916+0000 7fa0c299d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc159420 0x7fa0bc140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.916+0000 7fa0c299d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0bc1597f0 0x7fa0bc15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.916+0000 7fa0c299d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 0x7fa0bc15a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.916+0000 7fa0c299d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc144590 con 0x7fa0bc1597f0 2023-10-08T01:12:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.916+0000 7fa0c299d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15bba0 con 0x7fa0bc15d230 2023-10-08T01:12:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.916+0000 7fa0c299d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc16d930 con 0x7fa0bc159420 2023-10-08T01:12:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.916+0000 7fa0c119a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0bc1597f0 0x7fa0bc15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:25.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c199b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc159420 0x7fa0bc140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c119a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0bc1597f0 0x7fa0bc15a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58738/0 (socket says 172.21.15.119:58738) 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c119a700 1 -- 172.21.15.119:0/3858661496 learned_addr learned my addr 172.21.15.119:0/3858661496 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c219c700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 0x7fa0bc15a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c199b700 1 -- 172.21.15.119:0/3858661496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 msgr2=0x7fa0bc15a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c199b700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 0x7fa0bc15a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c199b700 1 -- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0bc1597f0 msgr2=0x7fa0bc15a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c199b700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0bc1597f0 0x7fa0bc15a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c199b700 1 -- 172.21.15.119:0/3858661496 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4009a20 con 0x7fa0bc159420 2023-10-08T01:12:25.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c119a700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0bc1597f0 0x7fa0bc15a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:25.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.917+0000 7fa0c199b700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc159420 0x7fa0bc140c30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800c580 tx=0x7fa0b8007d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:25.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.918+0000 7fa0b1ffb700 1 -- 172.21.15.119:0/3858661496 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8005b30 con 0x7fa0bc159420 2023-10-08T01:12:25.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.918+0000 7fa0b1ffb700 1 -- 172.21.15.119:0/3858661496 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b8005cb0 con 0x7fa0bc159420 2023-10-08T01:12:25.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.918+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc15af90 con 0x7fa0bc159420 2023-10-08T01:12:25.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.918+0000 7fa0b1ffb700 1 -- 172.21.15.119:0/3858661496 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8007320 con 0x7fa0bc159420 2023-10-08T01:12:25.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.918+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc15b550 con 0x7fa0bc159420 2023-10-08T01:12:25.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.919+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc04c3c0 con 0x7fa0bc159420 2023-10-08T01:12:25.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.920+0000 7fa0b1ffb700 1 -- 172.21.15.119:0/3858661496 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fa0b801b3b0 con 0x7fa0bc159420 2023-10-08T01:12:25.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.921+0000 7fa0b1ffb700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa0a00670e0 0x7fa0a0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:25.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.921+0000 7fa0b1ffb700 1 -- 172.21.15.119:0/3858661496 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fa0b8083f90 con 0x7fa0bc159420 2023-10-08T01:12:25.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.921+0000 7fa0c119a700 1 -- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa0a00670e0 msgr2=0x7fa0a0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:25.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.921+0000 7fa0c119a700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa0a00670e0 0x7fa0a0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:25.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:25.925+0000 7fa0b1ffb700 1 -- 172.21.15.119:0/3858661496 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0b8050c50 con 0x7fa0bc159420 2023-10-08T01:12:26.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.122+0000 7fa0c119a700 1 -- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa0a00670e0 msgr2=0x7fa0a0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:26.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.122+0000 7fa0c119a700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa0a00670e0 0x7fa0a0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:26.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.167+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0bc0b1790 con 0x7fa0bc159420 2023-10-08T01:12:26.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.168+0000 7fa0b1ffb700 1 -- 172.21.15.119:0/3858661496 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fa0b8017020 con 0x7fa0bc159420 2023-10-08T01:12:26.169 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:26.169 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3441253316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3441253316},{"type":"v1","addr":"172.21.15.184:6834","nonce":3441253316}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/3613948578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3613948578},{"type":"v1","addr":"172.21.15.119:6833","nonce":3613948578}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/765135141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":765135141},{"type":"v1","addr":"172.21.15.119:6835","nonce":765135141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:02.279676+0000","modified":"2023-10-08T01:12:05.851791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/910230897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":910230897},{"type":"v1","addr":"172.21.15.184:6835","nonce":910230897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.170+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa0a00670e0 msgr2=0x7fa0a0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.170+0000 7fa0c299d700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa0a00670e0 0x7fa0a0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.170+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc159420 msgr2=0x7fa0bc140c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.170+0000 7fa0c299d700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc159420 0x7fa0bc140c30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800c580 tx=0x7fa0b8007d50 comp rx=0 tx=0).stop 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.171+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 shutdown_connections 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.171+0000 7fa0c299d700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa0a00670e0 0x7fa0a0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.171+0000 7fa0c299d700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa0bc159420 0x7fa0bc140c30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.171+0000 7fa0c299d700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa0bc15d230 0x7fa0bc15a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.171+0000 7fa0c299d700 1 --2- 172.21.15.119:0/3858661496 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0bc1597f0 0x7fa0bc15a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.171+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 >> 172.21.15.119:0/3858661496 conn(0x7fa0bc0d0660 msgr2=0x7fa0bc0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:26.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.171+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 shutdown_connections 2023-10-08T01:12:26.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.171+0000 7fa0c299d700 1 -- 172.21.15.119:0/3858661496 wait complete. 2023-10-08T01:12:26.173 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 39 2023-10-08T01:12:26.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:26.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.377+0000 7ff11ffff700 1 Processor -- start 2023-10-08T01:12:26.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.377+0000 7ff11ffff700 1 -- start start 2023-10-08T01:12:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.378+0000 7ff11ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff12015d230 0x7ff12015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.378+0000 7ff11ffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1200b2980 con 0x7ff120159420 2023-10-08T01:12:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.378+0000 7ff11ffff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12015db70 con 0x7ff1201597f0 2023-10-08T01:12:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.378+0000 7ff11ffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1200b11d0 con 0x7ff12015d230 2023-10-08T01:12:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.378+0000 7ff11effd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff120159420 0x7ff1200b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47898/0 (socket says 172.21.15.119:47898) 2023-10-08T01:12:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.378+0000 7ff11effd700 1 -- 172.21.15.119:0/4207990628 learned_addr learned my addr 172.21.15.119:0/4207990628 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.378+0000 7ff11f7fe700 1 --2- 172.21.15.119:0/4207990628 >> v2:172.21.15.119:3300/0 conn(0x7ff12015d230 0x7ff12015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:26.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11dffb700 1 -- 172.21.15.119:0/4207990628 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749770161 0 0) 0x7ff1200b2980 con 0x7ff120159420 2023-10-08T01:12:26.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11dffb700 1 -- 172.21.15.119:0/4207990628 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff108003570 con 0x7ff120159420 2023-10-08T01:12:26.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11dffb700 1 -- 172.21.15.119:0/4207990628 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976174204 0 0) 0x7ff12015db70 con 0x7ff1201597f0 2023-10-08T01:12:26.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11dffb700 1 -- 172.21.15.119:0/4207990628 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1200b2980 con 0x7ff1201597f0 2023-10-08T01:12:26.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11f7fe700 1 -- 172.21.15.119:0/4207990628 >> v1:172.21.15.184:6789/0 conn(0x7ff1201597f0 legacy=0x7ff1200b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11f7fe700 1 -- 172.21.15.119:0/4207990628 >> v1:172.21.15.119:6789/0 conn(0x7ff120159420 legacy=0x7ff1200b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11f7fe700 1 -- 172.21.15.119:0/4207990628 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1200b14b0 con 0x7ff12015d230 2023-10-08T01:12:26.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11f7fe700 1 --2- 172.21.15.119:0/4207990628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff12015d230 0x7ff12015d630 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff118009e60 tx=0x7ff118009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45920db4d93ca1a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:26.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.379+0000 7ff11dffb700 1 -- 172.21.15.119:0/4207990628 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800e070 con 0x7ff12015d230 2023-10-08T01:12:26.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11dffb700 1 -- 172.21.15.119:0/4207990628 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11800bab0 con 0x7ff12015d230 2023-10-08T01:12:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11dffb700 1 -- 172.21.15.119:0/4207990628 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11800be70 con 0x7ff12015d230 2023-10-08T01:12:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11ffff700 1 -- 172.21.15.119:0/4207990628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff12015d230 msgr2=0x7ff12015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11ffff700 1 --2- 172.21.15.119:0/4207990628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff12015d230 0x7ff12015d630 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff118009e60 tx=0x7ff118009780 comp rx=0 tx=0).stop 2023-10-08T01:12:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11ffff700 1 -- 172.21.15.119:0/4207990628 shutdown_connections 2023-10-08T01:12:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11ffff700 1 --2- 172.21.15.119:0/4207990628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff12015d230 0x7ff12015d630 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11ffff700 1 -- 172.21.15.119:0/4207990628 >> 172.21.15.119:0/4207990628 conn(0x7ff1200d0660 msgr2=0x7ff1200d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11ffff700 1 -- 172.21.15.119:0/4207990628 shutdown_connections 2023-10-08T01:12:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.380+0000 7ff11ffff700 1 -- 172.21.15.119:0/4207990628 wait complete. 2023-10-08T01:12:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.382+0000 7ff11ffff700 1 Processor -- start 2023-10-08T01:12:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.382+0000 7ff11ffff700 1 -- start start 2023-10-08T01:12:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff120159420 0x7ff120150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1201597f0 0x7ff12014f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff12015d230 0x7ff1200b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff120152200 con 0x7ff120159420 2023-10-08T01:12:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201515f0 con 0x7ff12015d230 2023-10-08T01:12:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201518d0 con 0x7ff1201597f0 2023-10-08T01:12:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff120159420 0x7ff120150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff120159420 0x7ff120150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58752/0 (socket says 172.21.15.119:58752) 2023-10-08T01:12:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11effd700 1 -- 172.21.15.119:0/2954294545 learned_addr learned my addr 172.21.15.119:0/2954294545 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:26.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11f7fe700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff12015d230 0x7ff1200b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:26.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11e7fc700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1201597f0 0x7ff12014f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:26.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11effd700 1 -- 172.21.15.119:0/2954294545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1201597f0 msgr2=0x7ff12014f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11effd700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1201597f0 0x7ff12014f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11effd700 1 -- 172.21.15.119:0/2954294545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff12015d230 msgr2=0x7ff1200b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11effd700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff12015d230 0x7ff1200b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.383+0000 7ff11effd700 1 -- 172.21.15.119:0/2954294545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118009a20 con 0x7ff120159420 2023-10-08T01:12:26.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.384+0000 7ff11effd700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff120159420 0x7ff120150d40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c790 tx=0x7ff110007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:26.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.384+0000 7ff106ffd700 1 -- 172.21.15.119:0/2954294545 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110005b30 con 0x7ff120159420 2023-10-08T01:12:26.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.384+0000 7ff106ffd700 1 -- 172.21.15.119:0/2954294545 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff110005cd0 con 0x7ff120159420 2023-10-08T01:12:26.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.384+0000 7ff106ffd700 1 -- 172.21.15.119:0/2954294545 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff110007320 con 0x7ff120159420 2023-10-08T01:12:26.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.384+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff120151bb0 con 0x7ff120159420 2023-10-08T01:12:26.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.384+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff120151280 con 0x7ff120159420 2023-10-08T01:12:26.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.385+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12004c3c0 con 0x7ff120159420 2023-10-08T01:12:26.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.386+0000 7ff106ffd700 1 -- 172.21.15.119:0/2954294545 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7ff110012400 con 0x7ff120159420 2023-10-08T01:12:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.387+0000 7ff106ffd700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff10c0670e0 0x7ff10c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.387+0000 7ff106ffd700 1 -- 172.21.15.119:0/2954294545 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7ff110083f40 con 0x7ff120159420 2023-10-08T01:12:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.387+0000 7ff11e7fc700 1 -- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff10c0670e0 msgr2=0x7ff10c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.387+0000 7ff11e7fc700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff10c0670e0 0x7ff10c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.395+0000 7ff106ffd700 1 -- 172.21.15.119:0/2954294545 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff11004b5a0 con 0x7ff120159420 2023-10-08T01:12:26.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.588+0000 7ff11e7fc700 1 -- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff10c0670e0 msgr2=0x7ff10c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:26.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.588+0000 7ff11e7fc700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff10c0670e0 0x7ff10c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:26.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.594+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1200b1790 con 0x7ff120159420 2023-10-08T01:12:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.595+0000 7ff106ffd700 1 -- 172.21.15.119:0/2954294545 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16371 (secure 0 0 0) 0x7ff110017090 con 0x7ff120159420 2023-10-08T01:12:26.596 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:26.596 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":92,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:23.311955+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:00.376380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:01.260893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2785931715":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/1171405380":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/2173962022":"2023-10-09T01:12:23.311889+0000","172.21.15.119:6837/85511":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/2446352006":"2023-10-09T01:12:23.311889+0000","172.21.15.119:6836/85511":"2023-10-09T01:12:23.311889+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff10c0670e0 msgr2=0x7ff10c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff10c0670e0 0x7ff10c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff120159420 msgr2=0x7ff120150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff120159420 0x7ff120150d40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7ff11000c790 tx=0x7ff110007e80 comp rx=0 tx=0).stop 2023-10-08T01:12:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 shutdown_connections 2023-10-08T01:12:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff10c0670e0 0x7ff10c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff12015d230 0x7ff1200b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff120159420 0x7ff120150d40 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 --2- 172.21.15.119:0/2954294545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1201597f0 0x7ff12014f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 >> 172.21.15.119:0/2954294545 conn(0x7ff1200d0660 msgr2=0x7ff1201570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 shutdown_connections 2023-10-08T01:12:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.598+0000 7ff11ffff700 1 -- 172.21.15.119:0/2954294545 wait complete. 2023-10-08T01:12:26.613 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:26.613+0000 7f5dfd27c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:12:26.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T01:12:00.376380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T01:12:01.260893+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T01:12:26.624 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:12:26.686 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:26.683+0000 7fdff959c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:12:26.776 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:26.775+0000 7f5dfd27c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:26.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.812+0000 7f0b7d99d700 1 Processor -- start 2023-10-08T01:12:26.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.812+0000 7f0b7d99d700 1 -- start start 2023-10-08T01:12:26.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.813+0000 7f0b7d99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0b78159a60 0x7f0b780c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.813+0000 7f0b7d99d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780c9cb0 con 0x7f0b780ccc00 2023-10-08T01:12:26.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.813+0000 7f0b7d99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780c9f30 con 0x7f0b780cb230 2023-10-08T01:12:26.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.813+0000 7f0b7d99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7806dfc0 con 0x7f0b78159a60 2023-10-08T01:12:26.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.813+0000 7f0b77fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0b780ccc00 0x7f0b78069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47908/0 (socket says 172.21.15.119:47908) 2023-10-08T01:12:26.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.813+0000 7f0b77fff700 1 -- 172.21.15.119:0/2463549691 learned_addr learned my addr 172.21.15.119:0/2463549691 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:26.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.814+0000 7f0b7c99b700 1 --2- 172.21.15.119:0/2463549691 >> v2:172.21.15.184:3300/0 conn(0x7f0b78159a60 0x7f0b780c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:26.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.814+0000 7f0b777fe700 1 -- 172.21.15.119:0/2463549691 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464208319 0 0) 0x7f0b780c9cb0 con 0x7f0b780ccc00 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.814+0000 7f0b777fe700 1 -- 172.21.15.119:0/2463549691 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b68003570 con 0x7f0b780ccc00 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.814+0000 7f0b7c99b700 1 -- 172.21.15.119:0/2463549691 >> v1:172.21.15.184:6789/0 conn(0x7f0b780cb230 legacy=0x7f0b7806d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.814+0000 7f0b7c99b700 1 -- 172.21.15.119:0/2463549691 >> v1:172.21.15.119:6789/0 conn(0x7f0b780ccc00 legacy=0x7f0b78069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.814+0000 7f0b7c99b700 1 -- 172.21.15.119:0/2463549691 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780ca470 con 0x7f0b78159a60 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.814+0000 7f0b7c99b700 1 --2- 172.21.15.119:0/2463549691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b78159a60 0x7f0b780c9770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0098e0 tx=0x7f0b6c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e661978588c2db8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b777fe700 1 -- 172.21.15.119:0/2463549691 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00b6d0 con 0x7f0b78159a60 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b777fe700 1 -- 172.21.15.119:0/2463549691 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c00b830 con 0x7f0b78159a60 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b777fe700 1 -- 172.21.15.119:0/2463549691 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c00bbd0 con 0x7f0b78159a60 2023-10-08T01:12:26.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b7d99d700 1 -- 172.21.15.119:0/2463549691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b78159a60 msgr2=0x7f0b780c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b7d99d700 1 --2- 172.21.15.119:0/2463549691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b78159a60 0x7f0b780c9770 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0098e0 tx=0x7f0b6c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b7d99d700 1 -- 172.21.15.119:0/2463549691 shutdown_connections 2023-10-08T01:12:26.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b7d99d700 1 --2- 172.21.15.119:0/2463549691 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b78159a60 0x7f0b780c9770 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b7d99d700 1 -- 172.21.15.119:0/2463549691 >> 172.21.15.119:0/2463549691 conn(0x7f0b7806baa0 msgr2=0x7f0b7806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:26.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b7d99d700 1 -- 172.21.15.119:0/2463549691 shutdown_connections 2023-10-08T01:12:26.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.815+0000 7f0b7d99d700 1 -- 172.21.15.119:0/2463549691 wait complete. 2023-10-08T01:12:26.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.817+0000 7f0b7d99d700 1 Processor -- start 2023-10-08T01:12:26.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.817+0000 7f0b7d99d700 1 -- start start 2023-10-08T01:12:26.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7d99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b780cb230 0x7f0b7814e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7d99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b780ccc00 0x7f0b7816b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7d99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b78159a60 0x7f0b780aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7d99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7814e780 con 0x7f0b780cb230 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7d99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b2560 con 0x7f0b780ccc00 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7d99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b780b27b0 con 0x7f0b78159a60 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7c99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b780cb230 0x7f0b7814e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7c99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b780cb230 0x7f0b7814e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58756/0 (socket says 172.21.15.119:58756) 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7c99b700 1 -- 172.21.15.119:0/3602295836 learned_addr learned my addr 172.21.15.119:0/3602295836 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b77fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b780ccc00 0x7f0b7816b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7d19c700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b78159a60 0x7f0b780aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7c99b700 1 -- 172.21.15.119:0/3602295836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b78159a60 msgr2=0x7f0b780aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7c99b700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b78159a60 0x7f0b780aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7c99b700 1 -- 172.21.15.119:0/3602295836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b780ccc00 msgr2=0x7f0b7816b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7c99b700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b780ccc00 0x7f0b7816b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.818+0000 7f0b7c99b700 1 -- 172.21.15.119:0/3602295836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c009420 con 0x7f0b780cb230 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.819+0000 7f0b7c99b700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b780cb230 0x7f0b7814e240 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0b780c9e60 tx=0x7f0b6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.819+0000 7f0b74ff9700 1 -- 172.21.15.119:0/3602295836 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c000d10 con 0x7f0b780cb230 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.819+0000 7f0b74ff9700 1 -- 172.21.15.119:0/3602295836 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c00bbb0 con 0x7f0b780cb230 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.819+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b780b2a90 con 0x7f0b780cb230 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.819+0000 7f0b74ff9700 1 -- 172.21.15.119:0/3602295836 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c011650 con 0x7f0b780cb230 2023-10-08T01:12:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.819+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b780b3050 con 0x7f0b780cb230 2023-10-08T01:12:26.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.820+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7806eba0 con 0x7f0b780cb230 2023-10-08T01:12:26.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.821+0000 7f0b74ff9700 1 -- 172.21.15.119:0/3602295836 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f0b6c011a00 con 0x7f0b780cb230 2023-10-08T01:12:26.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.822+0000 7f0b74ff9700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0b5c0670e0 0x7f0b5c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:26.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.822+0000 7f0b74ff9700 1 -- 172.21.15.119:0/3602295836 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f0b6c086070 con 0x7f0b780cb230 2023-10-08T01:12:26.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.822+0000 7f0b77fff700 1 -- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0b5c0670e0 msgr2=0x7f0b5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:26.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.822+0000 7f0b77fff700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0b5c0670e0 0x7f0b5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:26.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:26.826+0000 7f0b74ff9700 1 -- 172.21.15.119:0/3602295836 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b6c052d30 con 0x7f0b780cb230 2023-10-08T01:12:26.858 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:26.857+0000 7fdff959c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:26.867 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:26.867+0000 7f5dfd27c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:12:26.946 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:26.945+0000 7fdff959c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:12:27.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.023+0000 7f0b77fff700 1 -- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0b5c0670e0 msgr2=0x7f0b5c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:27.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.023+0000 7f0b77fff700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0b5c0670e0 0x7f0b5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:27.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.027+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b780ca750 con 0x7f0b780cb230 2023-10-08T01:12:27.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.028+0000 7f0b74ff9700 1 -- 172.21.15.119:0/3602295836 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b6c02a0a0 con 0x7f0b780cb230 2023-10-08T01:12:27.057 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:12:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0b5c0670e0 msgr2=0x7f0b5c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0b5c0670e0 0x7f0b5c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b780cb230 msgr2=0x7f0b7814e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b780cb230 0x7f0b7814e240 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0b780c9e60 tx=0x7f0b6c017000 comp rx=0 tx=0).stop 2023-10-08T01:12:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 shutdown_connections 2023-10-08T01:12:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0b5c0670e0 0x7f0b5c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0b78159a60 0x7f0b780aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b780cb230 0x7f0b7814e240 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 --2- 172.21.15.119:0/3602295836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0b780ccc00 0x7f0b7816b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 >> 172.21.15.119:0/3602295836 conn(0x7f0b7806baa0 msgr2=0x7f0b78158800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:27.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 shutdown_connections 2023-10-08T01:12:27.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.030+0000 7f0b7d99d700 1 -- 172.21.15.119:0/3602295836 wait complete. 2023-10-08T01:12:27.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T01:12:27.093 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:27.092+0000 7f5dfd27c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:27.187 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:27.186+0000 7fdff959c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:27.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.247+0000 7fb9ff59e700 1 Processor -- start 2023-10-08T01:12:27.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.247+0000 7fb9ff59e700 1 -- start start 2023-10-08T01:12:27.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9ff59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b4e70 0x7fba000ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9ff59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba000cfb00 con 0x7fba000b16d0 2023-10-08T01:12:27.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9ff59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0015e060 con 0x7fba000b1b40 2023-10-08T01:12:27.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9ff59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba000b5c40 con 0x7fba000b4e70 2023-10-08T01:12:27.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9fdd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fba000b16d0 0x7fba0015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47924/0 (socket says 172.21.15.119:47924) 2023-10-08T01:12:27.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9fdd9b700 1 -- 172.21.15.119:0/1933313307 learned_addr learned my addr 172.21.15.119:0/1933313307 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:27.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9fe59c700 1 --2- 172.21.15.119:0/1933313307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b4e70 0x7fba000ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9fd59a700 1 -- 172.21.15.119:0/1933313307 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862450013 0 0) 0x7fba000cfb00 con 0x7fba000b16d0 2023-10-08T01:12:27.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9fd59a700 1 -- 172.21.15.119:0/1933313307 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e8003570 con 0x7fba000b16d0 2023-10-08T01:12:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.249+0000 7fb9fd59a700 1 -- 172.21.15.119:0/1933313307 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387455025 0 0) 0x7fba0015e060 con 0x7fba000b1b40 2023-10-08T01:12:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.250+0000 7fb9fd59a700 1 -- 172.21.15.119:0/1933313307 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba000cfb00 con 0x7fba000b1b40 2023-10-08T01:12:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.250+0000 7fb9fe59c700 1 -- 172.21.15.119:0/1933313307 >> v1:172.21.15.184:6789/0 conn(0x7fba000b1b40 legacy=0x7fba0015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.250+0000 7fb9fe59c700 1 -- 172.21.15.119:0/1933313307 >> v1:172.21.15.119:6789/0 conn(0x7fba000b16d0 legacy=0x7fba0015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.250+0000 7fb9fe59c700 1 -- 172.21.15.119:0/1933313307 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba000b5f20 con 0x7fba000b4e70 2023-10-08T01:12:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.250+0000 7fb9fe59c700 1 --2- 172.21.15.119:0/1933313307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b4e70 0x7fba000ccff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009910 tx=0x7fb9f00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26facfee9ce7d1b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.250+0000 7fb9fd59a700 1 -- 172.21.15.119:0/1933313307 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0011630 con 0x7fba000b4e70 2023-10-08T01:12:27.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.250+0000 7fb9fd59a700 1 -- 172.21.15.119:0/1933313307 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f0011790 con 0x7fba000b4e70 2023-10-08T01:12:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.250+0000 7fb9fd59a700 1 -- 172.21.15.119:0/1933313307 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0011b30 con 0x7fba000b4e70 2023-10-08T01:12:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.251+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1933313307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b4e70 msgr2=0x7fba000ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.251+0000 7fb9ff59e700 1 --2- 172.21.15.119:0/1933313307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b4e70 0x7fba000ccff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009910 tx=0x7fb9f00091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.251+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1933313307 shutdown_connections 2023-10-08T01:12:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.251+0000 7fb9ff59e700 1 --2- 172.21.15.119:0/1933313307 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b4e70 0x7fba000ccff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.251+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1933313307 >> 172.21.15.119:0/1933313307 conn(0x7fba000d10c0 msgr2=0x7fba000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.251+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1933313307 shutdown_connections 2023-10-08T01:12:27.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.251+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1933313307 wait complete. 2023-10-08T01:12:27.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.252+0000 7fb9ff59e700 1 Processor -- start 2023-10-08T01:12:27.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.253+0000 7fb9ff59e700 1 -- start start 2023-10-08T01:12:27.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.253+0000 7fb9ff59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba000b16d0 0x7fba0015cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.253+0000 7fb9ff59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b1b40 0x7fba00148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9ff59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba000b4e70 0x7fba001493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9ff59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0016a730 con 0x7fba000b4e70 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9ff59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba001498e0 con 0x7fba000b16d0 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9ff59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba00169e40 con 0x7fba000b1b40 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba000b4e70 0x7fba001493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fe59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba000b16d0 0x7fba0015cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba000b4e70 0x7fba001493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58758/0 (socket says 172.21.15.119:58758) 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fed9d700 1 -- 172.21.15.119:0/1564376445 learned_addr learned my addr 172.21.15.119:0/1564376445 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fdd9b700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b1b40 0x7fba00148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fed9d700 1 -- 172.21.15.119:0/1564376445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b1b40 msgr2=0x7fba00148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fed9d700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b1b40 0x7fba00148e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fed9d700 1 -- 172.21.15.119:0/1564376445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba000b16d0 msgr2=0x7fba0015cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fed9d700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba000b16d0 0x7fba0015cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.254+0000 7fb9fed9d700 1 -- 172.21.15.119:0/1564376445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f0009420 con 0x7fba000b4e70 2023-10-08T01:12:27.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.255+0000 7fb9fed9d700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba000b4e70 0x7fba001493a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c910 tx=0x7fb9f80027a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:27.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.255+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/1564376445 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8012630 con 0x7fba000b4e70 2023-10-08T01:12:27.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.255+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/1564376445 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f80127b0 con 0x7fba000b4e70 2023-10-08T01:12:27.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.255+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/1564376445 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8012b70 con 0x7fba000b4e70 2023-10-08T01:12:27.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.255+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0016a0e0 con 0x7fba000b4e70 2023-10-08T01:12:27.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.255+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0014cda0 con 0x7fba000b4e70 2023-10-08T01:12:27.272 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:27.271+0000 7f5dfd27c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:12:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.256+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba00088990 con 0x7fba000b4e70 2023-10-08T01:12:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.257+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/1564376445 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fb9f801b400 con 0x7fba000b4e70 2023-10-08T01:12:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.258+0000 7fb9ee7fc700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb9e00670e0 0x7fb9e0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.258+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/1564376445 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fb9f8084e90 con 0x7fba000b4e70 2023-10-08T01:12:27.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.258+0000 7fb9fe59c700 1 -- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb9e00670e0 msgr2=0x7fb9e0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:27.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.258+0000 7fb9fe59c700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb9e00670e0 0x7fb9e0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:27.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.262+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/1564376445 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9f8051b50 con 0x7fba000b4e70 2023-10-08T01:12:27.366 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:27.365+0000 7f5dfd27c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:12:27.373 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:27.372+0000 7fdff959c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:12:27.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.459+0000 7fb9fe59c700 1 -- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb9e00670e0 msgr2=0x7fb9e0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:27.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.459+0000 7fb9fe59c700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb9e00670e0 0x7fb9e0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:27.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.464+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba0004bb80 con 0x7fba000b4e70 2023-10-08T01:12:27.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.465+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/1564376445 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb9f8055170 con 0x7fba000b4e70 2023-10-08T01:12:27.518 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T01:12:27.519 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:27.468+0000 7fdff959c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:12:27.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb9e00670e0 msgr2=0x7fb9e0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:27.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb9e00670e0 0x7fb9e0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba000b4e70 msgr2=0x7fba001493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba000b4e70 0x7fba001493a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c910 tx=0x7fb9f80027a0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 shutdown_connections 2023-10-08T01:12:27.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb9e00670e0 0x7fb9e0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fba000b4e70 0x7fba001493a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fba000b16d0 0x7fba0015cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 --2- 172.21.15.119:0/1564376445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fba000b1b40 0x7fba00148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 >> 172.21.15.119:0/1564376445 conn(0x7fba000d10c0 msgr2=0x7fba0015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:27.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 shutdown_connections 2023-10-08T01:12:27.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.469+0000 7fb9ff59e700 1 -- 172.21.15.119:0/1564376445 wait complete. 2023-10-08T01:12:27.521 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T01:12:27.528 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:27.528+0000 7f5dfd27c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:12:27.629 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:27.629+0000 7f5dfd27c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:27.652 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:27.645+0000 7fdff959c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:12:27.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.710+0000 7fd505d9d700 1 Processor -- start 2023-10-08T01:12:27.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.710+0000 7fd505d9d700 1 -- start start 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.711+0000 7fd505d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5000b16d0 0x7fd5000b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.711+0000 7fd505d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.711+0000 7fd505d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5000b2080 con 0x7fd5000b4e70 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.711+0000 7fd505d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5000cfb00 con 0x7fd5000b16d0 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.711+0000 7fd505d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000b5c40 con 0x7fd5000b33b0 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd4fffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5000b16d0 0x7fd5000b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd50559c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd50559c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:36828/0 (socket says 172.21.15.119:36828) 2023-10-08T01:12:27.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd50559c700 1 -- 172.21.15.119:0/219068162 learned_addr learned my addr 172.21.15.119:0/219068162 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:27.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd4fffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5000b16d0 0x7fd5000b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:59322/0 (socket says 172.21.15.119:59322) 2023-10-08T01:12:27.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd50559c700 1 -- 172.21.15.119:0/219068162 >> v2:172.21.15.184:3300/0 conn(0x7fd5000b16d0 msgr2=0x7fd5000b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:27.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd50559c700 1 --2- 172.21.15.119:0/219068162 >> v2:172.21.15.184:3300/0 conn(0x7fd5000b16d0 0x7fd5000b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd50559c700 1 -- 172.21.15.119:0/219068162 >> v1:172.21.15.184:6789/0 conn(0x7fd5000b4e70 legacy=0x7fd5000ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd50559c700 1 -- 172.21.15.119:0/219068162 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5000b5f20 con 0x7fd5000b33b0 2023-10-08T01:12:27.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.712+0000 7fd4fffff700 1 --2- 172.21.15.119:0/219068162 >> v2:172.21.15.184:3300/0 conn(0x7fd5000b16d0 0x7fd5000b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.713+0000 7fd50559c700 1 --2- 172.21.15.119:0/219068162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009f20 tx=0x7fd4f8009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ef133841f1675aa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.713+0000 7fd4ff7fe700 1 -- 172.21.15.119:0/219068162 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f800e070 con 0x7fd5000b33b0 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.713+0000 7fd4ff7fe700 1 -- 172.21.15.119:0/219068162 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f800b9d0 con 0x7fd5000b33b0 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.713+0000 7fd4ff7fe700 1 -- 172.21.15.119:0/219068162 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80092e0 con 0x7fd5000b33b0 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.714+0000 7fd505d9d700 1 -- 172.21.15.119:0/219068162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 msgr2=0x7fd5000b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.714+0000 7fd505d9d700 1 --2- 172.21.15.119:0/219068162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009f20 tx=0x7fd4f8009840 comp rx=0 tx=0).stop 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.714+0000 7fd505d9d700 1 -- 172.21.15.119:0/219068162 shutdown_connections 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.714+0000 7fd505d9d700 1 --2- 172.21.15.119:0/219068162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b3810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.714+0000 7fd505d9d700 1 --2- 172.21.15.119:0/219068162 >> v2:172.21.15.184:3300/0 conn(0x7fd5000b16d0 0x7fd5000b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.714+0000 7fd505d9d700 1 -- 172.21.15.119:0/219068162 >> 172.21.15.119:0/219068162 conn(0x7fd5000d10c0 msgr2=0x7fd5000affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:27.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.714+0000 7fd505d9d700 1 -- 172.21.15.119:0/219068162 shutdown_connections 2023-10-08T01:12:27.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.714+0000 7fd505d9d700 1 -- 172.21.15.119:0/219068162 wait complete. 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.715+0000 7fd505d9d700 1 Processor -- start 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.715+0000 7fd505d9d700 1 -- start start 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.716+0000 7fd505d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000b16d0 0x7fd5000c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.716+0000 7fd505d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.716+0000 7fd505d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5000b4e70 0x7fd500152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.716+0000 7fd505d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c8130 con 0x7fd5000b16d0 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.716+0000 7fd505d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c75b0 con 0x7fd5000b4e70 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.716+0000 7fd505d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000c7890 con 0x7fd5000b33b0 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.716+0000 7fd504d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000b16d0 0x7fd5000c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd50559c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5000b4e70 0x7fd500152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd504d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000b16d0 0x7fd5000c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58766/0 (socket says 172.21.15.119:58766) 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd504d9b700 1 -- 172.21.15.119:0/2856659705 learned_addr learned my addr 172.21.15.119:0/2856659705 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd4fffff700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd504d9b700 1 -- 172.21.15.119:0/2856659705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 msgr2=0x7fd5000b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd504d9b700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd504d9b700 1 -- 172.21.15.119:0/2856659705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5000b4e70 msgr2=0x7fd500152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd504d9b700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5000b4e70 0x7fd500152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd504d9b700 1 -- 172.21.15.119:0/2856659705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec009420 con 0x7fd5000b16d0 2023-10-08T01:12:27.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd50559c700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5000b4e70 0x7fd500152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:27.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd504d9b700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000b16d0 0x7fd5000c6cf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c590 tx=0x7fd4f400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:27.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd4fcff9700 1 -- 172.21.15.119:0/2856659705 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4011590 con 0x7fd5000b16d0 2023-10-08T01:12:27.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd4fcff9700 1 -- 172.21.15.119:0/2856659705 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f4011710 con 0x7fd5000b16d0 2023-10-08T01:12:27.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd4fcff9700 1 -- 172.21.15.119:0/2856659705 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f4011ad0 con 0x7fd5000b16d0 2023-10-08T01:12:27.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.717+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f8009ae0 con 0x7fd5000b16d0 2023-10-08T01:12:27.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.718+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5000c7e50 con 0x7fd5000b16d0 2023-10-08T01:12:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.719+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5000c5490 con 0x7fd5000b16d0 2023-10-08T01:12:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.720+0000 7fd4fcff9700 1 -- 172.21.15.119:0/2856659705 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fd4f401b3b0 con 0x7fd5000b16d0 2023-10-08T01:12:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.721+0000 7fd4fcff9700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd4e40670e0 0x7fd4e4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.721+0000 7fd4fcff9700 1 -- 172.21.15.119:0/2856659705 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fd4f4083f50 con 0x7fd5000b16d0 2023-10-08T01:12:27.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.721+0000 7fd4fffff700 1 -- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd4e40670e0 msgr2=0x7fd4e4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.721+0000 7fd4fffff700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd4e40670e0 0x7fd4e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:27.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.725+0000 7fd4fcff9700 1 -- 172.21.15.119:0/2856659705 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5000c5490 con 0x7fd5000b16d0 2023-10-08T01:12:27.753 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:27.752+0000 7fdff959c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.922+0000 7fd4fffff700 1 -- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd4e40670e0 msgr2=0x7fd4e4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.922+0000 7fd4fffff700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd4e40670e0 0x7fd4e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:27.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.923+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd5000c5490 con 0x7fd5000b16d0 2023-10-08T01:12:27.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.924+0000 7fd4fcff9700 1 -- 172.21.15.119:0/2856659705 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd5000c5490 con 0x7fd5000b16d0 2023-10-08T01:12:27.925 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T01:12:27.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.926+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd4e40670e0 msgr2=0x7fd4e4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:27.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.926+0000 7fd505d9d700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd4e40670e0 0x7fd4e4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.926+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000b16d0 msgr2=0x7fd5000c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:27.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.926+0000 7fd505d9d700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000b16d0 0x7fd5000c6cf0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd4f400c590 tx=0x7fd4f400c8a0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.926+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 shutdown_connections 2023-10-08T01:12:27.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.927+0000 7fd505d9d700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd4e40670e0 0x7fd4e4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.927+0000 7fd505d9d700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5000b4e70 0x7fd500152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.927+0000 7fd505d9d700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000b16d0 0x7fd5000c6cf0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.927+0000 7fd505d9d700 1 --2- 172.21.15.119:0/2856659705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5000b33b0 0x7fd5000b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:27.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.927+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 >> 172.21.15.119:0/2856659705 conn(0x7fd5000d10c0 msgr2=0x7fd5000b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:27.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.927+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 shutdown_connections 2023-10-08T01:12:27.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:27.927+0000 7fd505d9d700 1 -- 172.21.15.119:0/2856659705 wait complete. 2023-10-08T01:12:27.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:28.115 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:28.114+0000 7f5dfd27c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:12:28.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.133+0000 7f7c8539d700 1 Processor -- start 2023-10-08T01:12:28.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.133+0000 7f7c8539d700 1 -- start start 2023-10-08T01:12:28.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.134+0000 7f7c8539d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c80159420 0x7f7c800b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.134+0000 7f7c8539d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7c8015d230 0x7f7c8015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.134+0000 7f7c8539d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c800b2980 con 0x7f7c801597f0 2023-10-08T01:12:28.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.134+0000 7f7c8539d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c8015db70 con 0x7f7c80159420 2023-10-08T01:12:28.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.134+0000 7f7c8539d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c800b11d0 con 0x7f7c8015d230 2023-10-08T01:12:28.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c7ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c80159420 0x7f7c800b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c7ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c80159420 0x7f7c800b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58772/0 (socket says 172.21.15.119:58772) 2023-10-08T01:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c7ffff700 1 -- 172.21.15.119:0/2210646070 learned_addr learned my addr 172.21.15.119:0/2210646070 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c84b9c700 1 --2- 172.21.15.119:0/2210646070 >> v2:172.21.15.184:3300/0 conn(0x7f7c8015d230 0x7f7c8015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c7ffff700 1 -- 172.21.15.119:0/2210646070 >> v2:172.21.15.184:3300/0 conn(0x7f7c8015d230 msgr2=0x7f7c8015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c7ffff700 1 --2- 172.21.15.119:0/2210646070 >> v2:172.21.15.184:3300/0 conn(0x7f7c8015d230 0x7f7c8015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c7ffff700 1 -- 172.21.15.119:0/2210646070 >> v1:172.21.15.184:6789/0 conn(0x7f7c801597f0 legacy=0x7f7c800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c7ffff700 1 -- 172.21.15.119:0/2210646070 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c800b14b0 con 0x7f7c80159420 2023-10-08T01:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.135+0000 7f7c7ffff700 1 --2- 172.21.15.119:0/2210646070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c800b5f40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7c8015d170 tx=0x7f7c70009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388d91014a206c3e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c7effd700 1 -- 172.21.15.119:0/2210646070 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70009e70 con 0x7f7c80159420 2023-10-08T01:12:28.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c7effd700 1 -- 172.21.15.119:0/2210646070 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7000b860 con 0x7f7c80159420 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c7effd700 1 -- 172.21.15.119:0/2210646070 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000bc00 con 0x7f7c80159420 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c8539d700 1 -- 172.21.15.119:0/2210646070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 msgr2=0x7f7c800b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c8539d700 1 --2- 172.21.15.119:0/2210646070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c800b5f40 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f7c8015d170 tx=0x7f7c70009250 comp rx=0 tx=0).stop 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c8539d700 1 -- 172.21.15.119:0/2210646070 shutdown_connections 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c8539d700 1 --2- 172.21.15.119:0/2210646070 >> v2:172.21.15.184:3300/0 conn(0x7f7c8015d230 0x7f7c8015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c8539d700 1 --2- 172.21.15.119:0/2210646070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c800b5f40 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c8539d700 1 -- 172.21.15.119:0/2210646070 >> 172.21.15.119:0/2210646070 conn(0x7f7c800d0660 msgr2=0x7f7c800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.136+0000 7f7c8539d700 1 -- 172.21.15.119:0/2210646070 shutdown_connections 2023-10-08T01:12:28.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.137+0000 7f7c8539d700 1 -- 172.21.15.119:0/2210646070 wait complete. 2023-10-08T01:12:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.138+0000 7f7c8539d700 1 Processor -- start 2023-10-08T01:12:28.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.138+0000 7f7c8539d700 1 -- start start 2023-10-08T01:12:28.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c8539d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c80146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c8539d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c801597f0 0x7f7c800cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c8539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8015d230 0x7f7c80158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c8539d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800cb290 con 0x7f7c80159420 2023-10-08T01:12:28.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c8539d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800ca6c0 con 0x7f7c801597f0 2023-10-08T01:12:28.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c8539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800ca960 con 0x7f7c8015d230 2023-10-08T01:12:28.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c7ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c80146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c7f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c801597f0 0x7f7c800cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c7ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c80146fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58778/0 (socket says 172.21.15.119:58778) 2023-10-08T01:12:28.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c7ffff700 1 -- 172.21.15.119:0/3682328890 learned_addr learned my addr 172.21.15.119:0/3682328890 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:28.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.139+0000 7f7c84b9c700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8015d230 0x7f7c80158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c7ffff700 1 -- 172.21.15.119:0/3682328890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8015d230 msgr2=0x7f7c80158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c7ffff700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8015d230 0x7f7c80158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c7ffff700 1 -- 172.21.15.119:0/3682328890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c801597f0 msgr2=0x7f7c800cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c7ffff700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c801597f0 0x7f7c800cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c7ffff700 1 -- 172.21.15.119:0/3682328890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c700094f0 con 0x7f7c80159420 2023-10-08T01:12:28.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c7f7fe700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c801597f0 0x7f7c800cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:28.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c7ffff700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c80146fc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7c80089340 tx=0x7f7c70017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:28.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c67fff700 1 -- 172.21.15.119:0/3682328890 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000bbe0 con 0x7f7c80159420 2023-10-08T01:12:28.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c67fff700 1 -- 172.21.15.119:0/3682328890 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70011420 con 0x7f7c80159420 2023-10-08T01:12:28.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c67fff700 1 -- 172.21.15.119:0/3682328890 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700117c0 con 0x7f7c80159420 2023-10-08T01:12:28.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.140+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c800cac40 con 0x7f7c80159420 2023-10-08T01:12:28.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.141+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c80147500 con 0x7f7c80159420 2023-10-08T01:12:28.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.142+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c800cb570 con 0x7f7c80159420 2023-10-08T01:12:28.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.143+0000 7f7c67fff700 1 -- 172.21.15.119:0/3682328890 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f7c70011bf0 con 0x7f7c80159420 2023-10-08T01:12:28.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.144+0000 7f7c67fff700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7c6c0670e0 0x7f7c6c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.144+0000 7f7c67fff700 1 -- 172.21.15.119:0/3682328890 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f7c70086340 con 0x7f7c80159420 2023-10-08T01:12:28.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.144+0000 7f7c7f7fe700 1 -- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7c6c0670e0 msgr2=0x7f7c6c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:28.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.144+0000 7f7c7f7fe700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7c6c0670e0 0x7f7c6c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:28.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.148+0000 7f7c67fff700 1 -- 172.21.15.119:0/3682328890 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7c700530a0 con 0x7f7c80159420 2023-10-08T01:12:28.210 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:28.210+0000 7f5dfd27c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:12:28.258 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:28.257+0000 7fdff959c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:12:28.300 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:28.300+0000 7f5dfd27c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:12:28.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.345+0000 7f7c7f7fe700 1 -- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7c6c0670e0 msgr2=0x7f7c6c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:28.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.345+0000 7f7c7f7fe700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7c6c0670e0 0x7f7c6c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:28.359 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:28.355+0000 7fdff959c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:12:28.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.390+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c8004c3c0 con 0x7f7c80159420 2023-10-08T01:12:28.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.391+0000 7f7c67fff700 1 -- 172.21.15.119:0/3682328890 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f7c700566c0 con 0x7f7c80159420 2023-10-08T01:12:28.437 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:28.437 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3441253316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3441253316},{"type":"v1","addr":"172.21.15.184:6834","nonce":3441253316}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/3613948578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3613948578},{"type":"v1","addr":"172.21.15.119:6833","nonce":3613948578}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/765135141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":765135141},{"type":"v1","addr":"172.21.15.119:6835","nonce":765135141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:02.279676+0000","modified":"2023-10-08T01:12:05.851791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/910230897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":910230897},{"type":"v1","addr":"172.21.15.184:6835","nonce":910230897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T01:12:28.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7c6c0670e0 msgr2=0x7f7c6c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:28.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7c6c0670e0 0x7f7c6c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 msgr2=0x7f7c80146fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c80146fc0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f7c80089340 tx=0x7f7c70017000 comp rx=0 tx=0).stop 2023-10-08T01:12:28.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 shutdown_connections 2023-10-08T01:12:28.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7c6c0670e0 0x7f7c6c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7c8015d230 0x7f7c80158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c80159420 0x7f7c80146fc0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 --2- 172.21.15.119:0/3682328890 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7c801597f0 0x7f7c800cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 >> 172.21.15.119:0/3682328890 conn(0x7f7c800d0660 msgr2=0x7f7c80069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:28.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 shutdown_connections 2023-10-08T01:12:28.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.393+0000 7f7c8539d700 1 -- 172.21.15.119:0/3682328890 wait complete. 2023-10-08T01:12:28.440 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 39 2023-10-08T01:12:28.440 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:28.452 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:28.451+0000 7fdff959c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:12:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.627+0000 7f872c3ff700 1 Processor -- start 2023-10-08T01:12:28.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.627+0000 7f872c3ff700 1 -- start start 2023-10-08T01:12:28.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.628+0000 7f872c3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240cbf30 0x7f87240ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.628+0000 7f872c3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87240b1420 0x7f87240c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.628+0000 7f872c3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87240cc300 con 0x7f87240ccf90 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.628+0000 7f872c3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8724156bf0 con 0x7f87240cbf30 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.628+0000 7f872c3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240ae950 con 0x7f87240b1420 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a19b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240cbf30 0x7f87240ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a99c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87240b1420 0x7f87240c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a19b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f87240cbf30 0x7f87240ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58782/0 (socket says 172.21.15.119:58782) 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a99c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87240b1420 0x7f87240c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:59354/0 (socket says 172.21.15.119:59354) 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a19b700 1 -- 172.21.15.119:0/2217976321 learned_addr learned my addr 172.21.15.119:0/2217976321 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:28.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872999a700 1 --1- 172.21.15.119:0/2217976321 >> v1:172.21.15.184:6789/0 conn(0x7f87240ccf90 0x7f87240b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:33392/0 (socket says 172.21.15.119:33392) 2023-10-08T01:12:28.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a19b700 1 -- 172.21.15.119:0/2217976321 >> v2:172.21.15.184:3300/0 conn(0x7f87240b1420 msgr2=0x7f87240c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:28.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a19b700 1 --2- 172.21.15.119:0/2217976321 >> v2:172.21.15.184:3300/0 conn(0x7f87240b1420 0x7f87240c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a19b700 1 -- 172.21.15.119:0/2217976321 >> v1:172.21.15.184:6789/0 conn(0x7f87240ccf90 legacy=0x7f87240b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a19b700 1 -- 172.21.15.119:0/2217976321 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87240ad370 con 0x7f87240cbf30 2023-10-08T01:12:28.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.629+0000 7f872a19b700 1 --2- 172.21.15.119:0/2217976321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240ae410 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f87240b19e0 tx=0x7f872000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e40a8daf53cb4f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:28.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.630+0000 7f8729199700 1 -- 172.21.15.119:0/2217976321 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000dbb0 con 0x7f87240cbf30 2023-10-08T01:12:28.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.630+0000 7f8729199700 1 -- 172.21.15.119:0/2217976321 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872000dd10 con 0x7f87240cbf30 2023-10-08T01:12:28.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.630+0000 7f8729199700 1 -- 172.21.15.119:0/2217976321 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87200196b0 con 0x7f87240cbf30 2023-10-08T01:12:28.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.630+0000 7f872c3ff700 1 -- 172.21.15.119:0/2217976321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 msgr2=0x7f87240ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.631+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2217976321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240ae410 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f87240b19e0 tx=0x7f872000b4f0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.631+0000 7f872c3ff700 1 -- 172.21.15.119:0/2217976321 shutdown_connections 2023-10-08T01:12:28.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.631+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2217976321 >> v2:172.21.15.184:3300/0 conn(0x7f87240b1420 0x7f87240c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.631+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2217976321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240cbf30 0x7f87240ae410 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.631+0000 7f872c3ff700 1 -- 172.21.15.119:0/2217976321 >> 172.21.15.119:0/2217976321 conn(0x7f872406a320 msgr2=0x7f87240cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:28.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.631+0000 7f872c3ff700 1 -- 172.21.15.119:0/2217976321 shutdown_connections 2023-10-08T01:12:28.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.631+0000 7f872c3ff700 1 -- 172.21.15.119:0/2217976321 wait complete. 2023-10-08T01:12:28.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.632+0000 7f872c3ff700 1 Processor -- start 2023-10-08T01:12:28.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.632+0000 7f872c3ff700 1 -- start start 2023-10-08T01:12:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.633+0000 7f872c3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 0x7f87240c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.633+0000 7f872c3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240cbf30 0x7f872414b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.633+0000 7f872c3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240ccf90 0x7f872416b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.633+0000 7f872c3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87240ac3f0 con 0x7f87240b1420 2023-10-08T01:12:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.633+0000 7f872c3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87241587b0 con 0x7f87240ccf90 2023-10-08T01:12:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.633+0000 7f872c3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8724158a90 con 0x7f87240cbf30 2023-10-08T01:12:28.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 0x7f87240c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 0x7f87240c12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58788/0 (socket says 172.21.15.119:58788) 2023-10-08T01:12:28.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a19b700 1 -- 172.21.15.119:0/2301425875 learned_addr learned my addr 172.21.15.119:0/2301425875 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872999a700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240cbf30 0x7f872414b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a99c700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240ccf90 0x7f872416b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a99c700 1 -- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240cbf30 msgr2=0x7f872414b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a99c700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240cbf30 0x7f872414b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a99c700 1 -- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 msgr2=0x7f87240c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a99c700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 0x7f87240c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a99c700 1 -- 172.21.15.119:0/2301425875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872000b790 con 0x7f87240ccf90 2023-10-08T01:12:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.634+0000 7f872a99c700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240ccf90 0x7f872416b640 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f871c009e50 tx=0x7f871c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.635+0000 7f871a7fc700 1 -- 172.21.15.119:0/2301425875 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00caa0 con 0x7f87240ccf90 2023-10-08T01:12:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.635+0000 7f871a7fc700 1 -- 172.21.15.119:0/2301425875 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c00f040 con 0x7f87240ccf90 2023-10-08T01:12:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.635+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8724158d70 con 0x7f87240ccf90 2023-10-08T01:12:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.635+0000 7f871a7fc700 1 -- 172.21.15.119:0/2301425875 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00ce50 con 0x7f87240ccf90 2023-10-08T01:12:28.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.635+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f872414b680 con 0x7f87240ccf90 2023-10-08T01:12:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.636+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f872404c3c0 con 0x7f87240ccf90 2023-10-08T01:12:28.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.637+0000 7f871a7fc700 1 -- 172.21.15.119:0/2301425875 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f871c014690 con 0x7f87240ccf90 2023-10-08T01:12:28.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.637+0000 7f871a7fc700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.638+0000 7f871a7fc700 1 -- 172.21.15.119:0/2301425875 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f871c085810 con 0x7f87240ccf90 2023-10-08T01:12:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.638+0000 7f872a19b700 1 -- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 msgr2=0x7f8708069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.638+0000 7f872a19b700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.646+0000 7f871a7fc700 1 -- 172.21.15.119:0/2301425875 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f871c055e50 con 0x7f87240ccf90 2023-10-08T01:12:28.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.839+0000 7f872a19b700 1 -- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 msgr2=0x7f8708069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:28.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.839+0000 7f872a19b700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:28.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.847+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87240ad650 con 0x7f87240ccf90 2023-10-08T01:12:28.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.849+0000 7f871a7fc700 1 -- 172.21.15.119:0/2301425875 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16371 (secure 0 0 0) 0x7f871c0559e0 con 0x7f87240ccf90 2023-10-08T01:12:28.850 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:28.850 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":92,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:23.311955+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:00.376380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:01.260893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2785931715":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/1171405380":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/2173962022":"2023-10-09T01:12:23.311889+0000","172.21.15.119:6837/85511":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/2446352006":"2023-10-09T01:12:23.311889+0000","172.21.15.119:6836/85511":"2023-10-09T01:12:23.311889+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:28.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.851+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 msgr2=0x7f8708069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:28.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.851+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.851+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240ccf90 msgr2=0x7f872416b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:28.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.851+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240ccf90 0x7f872416b640 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f871c009e50 tx=0x7f871c00b040 comp rx=0 tx=0).stop 2023-10-08T01:12:28.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.852+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 shutdown_connections 2023-10-08T01:12:28.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.852+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f87080670e0 0x7f8708069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.852+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87240ccf90 0x7f872416b640 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.852+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f87240b1420 0x7f87240c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.852+0000 7f872c3ff700 1 --2- 172.21.15.119:0/2301425875 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f87240cbf30 0x7f872414b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:28.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.852+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 >> 172.21.15.119:0/2301425875 conn(0x7f872406a320 msgr2=0x7f87240ca620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:28.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.852+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 shutdown_connections 2023-10-08T01:12:28.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:28.852+0000 7f872c3ff700 1 -- 172.21.15.119:0/2301425875 wait complete. 2023-10-08T01:12:28.901 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T01:12:28.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:28.929 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:28.929+0000 7f5dfd27c180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:12:29.096 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:29.095+0000 7fdff959c180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:12:29.100 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:29.100+0000 7f5dfd27c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:29.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.157+0000 7f1286bff700 1 Processor -- start 2023-10-08T01:12:29.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.157+0000 7f1286bff700 1 -- start start 2023-10-08T01:12:29.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f1286bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 0x7f12800cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f1286bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f12800cbb30 0x7f12800ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f1286bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128006a220 con 0x7f12801599c0 2023-10-08T01:12:29.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f1286bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12800cbf60 con 0x7f12800cbb30 2023-10-08T01:12:29.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f1286bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1280069c70 con 0x7f128015ab90 2023-10-08T01:12:29.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f128499b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f12801599c0 0x7f1280069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47934/0 (socket says 172.21.15.119:47934) 2023-10-08T01:12:29.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f128499b700 1 -- 172.21.15.119:0/534834829 learned_addr learned my addr 172.21.15.119:0/534834829 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:29.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f127ffff700 1 --2- 172.21.15.119:0/534834829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 0x7f12800cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f128519c700 1 --2- 172.21.15.119:0/534834829 >> v2:172.21.15.184:3300/0 conn(0x7f12800cbb30 0x7f12800ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f127f7fe700 1 -- 172.21.15.119:0/534834829 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582761378 0 0) 0x7f128006a220 con 0x7f12801599c0 2023-10-08T01:12:29.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.159+0000 7f127f7fe700 1 -- 172.21.15.119:0/534834829 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1270003570 con 0x7f12801599c0 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.160+0000 7f128519c700 1 -- 172.21.15.119:0/534834829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 msgr2=0x7f12800cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.160+0000 7f128519c700 1 --2- 172.21.15.119:0/534834829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 0x7f12800cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.160+0000 7f128519c700 1 -- 172.21.15.119:0/534834829 >> v1:172.21.15.119:6789/0 conn(0x7f12801599c0 legacy=0x7f1280069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.160+0000 7f128519c700 1 -- 172.21.15.119:0/534834829 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280069f40 con 0x7f12800cbb30 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.160+0000 7f128519c700 1 --2- 172.21.15.119:0/534834829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12800cbb30 0x7f12800ce2e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1278009e60 tx=0x7f1278009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65325a6904d38d66 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.160+0000 7f127f7fe700 1 -- 172.21.15.119:0/534834829 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127800e070 con 0x7f12800cbb30 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.160+0000 7f127f7fe700 1 -- 172.21.15.119:0/534834829 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127800bab0 con 0x7f12800cbb30 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.160+0000 7f127f7fe700 1 -- 172.21.15.119:0/534834829 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127800be70 con 0x7f12800cbb30 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.161+0000 7f1286bff700 1 -- 172.21.15.119:0/534834829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12800cbb30 msgr2=0x7f12800ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.161+0000 7f1286bff700 1 --2- 172.21.15.119:0/534834829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12800cbb30 0x7f12800ce2e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1278009e60 tx=0x7f1278009780 comp rx=0 tx=0).stop 2023-10-08T01:12:29.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.161+0000 7f1286bff700 1 -- 172.21.15.119:0/534834829 shutdown_connections 2023-10-08T01:12:29.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.161+0000 7f1286bff700 1 --2- 172.21.15.119:0/534834829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12800cbb30 0x7f12800ce2e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.161+0000 7f1286bff700 1 --2- 172.21.15.119:0/534834829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 0x7f12800cb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.161+0000 7f1286bff700 1 -- 172.21.15.119:0/534834829 >> 172.21.15.119:0/534834829 conn(0x7f1280068e10 msgr2=0x7f12800d0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:29.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.161+0000 7f1286bff700 1 -- 172.21.15.119:0/534834829 shutdown_connections 2023-10-08T01:12:29.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.161+0000 7f1286bff700 1 -- 172.21.15.119:0/534834829 wait complete. 2023-10-08T01:12:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.162+0000 7f1286bff700 1 Processor -- start 2023-10-08T01:12:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.163+0000 7f1286bff700 1 -- start start 2023-10-08T01:12:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.163+0000 7f1286bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12800cbb30 0x7f12800b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.163+0000 7f1286bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12801599c0 0x7f12800c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.163+0000 7f1286bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 0x7f12801525f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.163+0000 7f1286bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12800c6750 con 0x7f12800cbb30 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.163+0000 7f1286bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128014e840 con 0x7f12801599c0 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.163+0000 7f1286bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12800b17e0 con 0x7f128015ab90 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12800cbb30 0x7f12800b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12800cbb30 0x7f12800b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58792/0 (socket says 172.21.15.119:58792) 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 -- 172.21.15.119:0/1522986801 learned_addr learned my addr 172.21.15.119:0/1522986801 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128519c700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 0x7f12801525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f127ffff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12801599c0 0x7f12800c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 -- 172.21.15.119:0/1522986801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 msgr2=0x7f12801525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 0x7f12801525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 -- 172.21.15.119:0/1522986801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12801599c0 msgr2=0x7f12800c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12801599c0 0x7f12800c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 -- 172.21.15.119:0/1522986801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278009a20 con 0x7f12800cbb30 2023-10-08T01:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.164+0000 7f128499b700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12800cbb30 0x7f12800b25a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f127400c7f0 tx=0x7f1274007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.165+0000 7f127cff9700 1 -- 172.21.15.119:0/1522986801 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274005cd0 con 0x7f12800cbb30 2023-10-08T01:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.165+0000 7f127cff9700 1 -- 172.21.15.119:0/1522986801 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1274005e50 con 0x7f12800cbb30 2023-10-08T01:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.165+0000 7f127cff9700 1 -- 172.21.15.119:0/1522986801 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1274007320 con 0x7f12800cbb30 2023-10-08T01:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.165+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12800b1ac0 con 0x7f12800cbb30 2023-10-08T01:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.165+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12800b2080 con 0x7f12800cbb30 2023-10-08T01:12:29.216 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:29.196+0000 7f5dfd27c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.166+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1280064550 con 0x7f12800cbb30 2023-10-08T01:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.167+0000 7f127cff9700 1 -- 172.21.15.119:0/1522986801 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f127401b3b0 con 0x7f12800cbb30 2023-10-08T01:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.168+0000 7f127cff9700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f12640670e0 0x7f1264069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.168+0000 7f127cff9700 1 -- 172.21.15.119:0/1522986801 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f1274084140 con 0x7f12800cbb30 2023-10-08T01:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.168+0000 7f127ffff700 1 -- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f12640670e0 msgr2=0x7f1264069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.168+0000 7f127ffff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f12640670e0 0x7f1264069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.172+0000 7f127cff9700 1 -- 172.21.15.119:0/1522986801 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1274050ea0 con 0x7f12800cbb30 2023-10-08T01:12:29.277 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:29.276+0000 7fdff959c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:29.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.369+0000 7f127ffff700 1 -- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f12640670e0 msgr2=0x7f1264069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:29.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.369+0000 7f127ffff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f12640670e0 0x7f1264069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:29.376 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:29.375+0000 7fdff959c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:12:29.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.415+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f128004c3c0 con 0x7f12800cbb30 2023-10-08T01:12:29.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.416+0000 7f127cff9700 1 -- 172.21.15.119:0/1522986801 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f12740544c0 con 0x7f12800cbb30 2023-10-08T01:12:29.418 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:29.418 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3441253316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3441253316},{"type":"v1","addr":"172.21.15.184:6834","nonce":3441253316}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/3613948578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3613948578},{"type":"v1","addr":"172.21.15.119:6833","nonce":3613948578}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/765135141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":765135141},{"type":"v1","addr":"172.21.15.119:6835","nonce":765135141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:02.279676+0000","modified":"2023-10-08T01:12:05.851791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/910230897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":910230897},{"type":"v1","addr":"172.21.15.184:6835","nonce":910230897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T01:12:29.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f12640670e0 msgr2=0x7f1264069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:29.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f12640670e0 0x7f1264069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12800cbb30 msgr2=0x7f12800b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12800cbb30 0x7f12800b25a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f127400c7f0 tx=0x7f1274007e00 comp rx=0 tx=0).stop 2023-10-08T01:12:29.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 shutdown_connections 2023-10-08T01:12:29.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f12640670e0 0x7f1264069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f128015ab90 0x7f12801525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12800cbb30 0x7f12800b25a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 --2- 172.21.15.119:0/1522986801 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12801599c0 0x7f12800c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 >> 172.21.15.119:0/1522986801 conn(0x7f1280068e10 msgr2=0x7f12800ce1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:29.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.418+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 shutdown_connections 2023-10-08T01:12:29.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.419+0000 7f1286bff700 1 -- 172.21.15.119:0/1522986801 wait complete. 2023-10-08T01:12:29.427 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 39 2023-10-08T01:12:29.438 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:29.490 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:29.490+0000 7f5dfd27c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:12:29.585 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:29.584+0000 7f5dfd27c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:12:29.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.630+0000 7fd5f759d700 1 Processor -- start 2023-10-08T01:12:29.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.630+0000 7fd5f759d700 1 -- start start 2023-10-08T01:12:29.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f759d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0142ea0 0x7fd5f00b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f759d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5f006c0f0 0x7fd5f006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f759d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f00b40f0 con 0x7fd5f00ad870 2023-10-08T01:12:29.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f759d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f0069640 con 0x7fd5f006c0f0 2023-10-08T01:12:29.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f759d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f0069920 con 0x7fd5f0142ea0 2023-10-08T01:12:29.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f6d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5f006c0f0 0x7fd5f006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f659b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0142ea0 0x7fd5f00b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f659b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0142ea0 0x7fd5f00b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:36866/0 (socket says 172.21.15.119:36866) 2023-10-08T01:12:29.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f6d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5f006c0f0 0x7fd5f006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58794/0 (socket says 172.21.15.119:58794) 2023-10-08T01:12:29.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f659b700 1 -- 172.21.15.119:0/1705534772 learned_addr learned my addr 172.21.15.119:0/1705534772 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:29.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.632+0000 7fd5f5d9a700 1 --1- 172.21.15.119:0/1705534772 >> v1:172.21.15.184:6789/0 conn(0x7fd5f00ad870 0x7fd5f006b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:33408/0 (socket says 172.21.15.119:33408) 2023-10-08T01:12:29.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f5599700 1 -- 172.21.15.119:0/1705534772 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 173777664 0 0) 0x7fd5f00b40f0 con 0x7fd5f00ad870 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f6d9c700 1 -- 172.21.15.119:0/1705534772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0142ea0 msgr2=0x7fd5f00b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f6d9c700 1 --2- 172.21.15.119:0/1705534772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0142ea0 0x7fd5f00b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f6d9c700 1 -- 172.21.15.119:0/1705534772 >> v1:172.21.15.184:6789/0 conn(0x7fd5f00ad870 legacy=0x7fd5f006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f6d9c700 1 -- 172.21.15.119:0/1705534772 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f0069c00 con 0x7fd5f006c0f0 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f659b700 1 --2- 172.21.15.119:0/1705534772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0142ea0 0x7fd5f00b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f6d9c700 1 --2- 172.21.15.119:0/1705534772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f006c0f0 0x7fd5f006a120 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009e60 tx=0x7fd5e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eeec7ee4155483c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f5599700 1 -- 172.21.15.119:0/1705534772 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e800e070 con 0x7fd5f006c0f0 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.633+0000 7fd5f5599700 1 -- 172.21.15.119:0/1705534772 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e800bb60 con 0x7fd5f006c0f0 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f5599700 1 -- 172.21.15.119:0/1705534772 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8012430 con 0x7fd5f006c0f0 2023-10-08T01:12:29.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f759d700 1 -- 172.21.15.119:0/1705534772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f006c0f0 msgr2=0x7fd5f006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f759d700 1 --2- 172.21.15.119:0/1705534772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f006c0f0 0x7fd5f006a120 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8009e60 tx=0x7fd5e8009780 comp rx=0 tx=0).stop 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f759d700 1 -- 172.21.15.119:0/1705534772 shutdown_connections 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f759d700 1 --2- 172.21.15.119:0/1705534772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f006c0f0 0x7fd5f006a120 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f759d700 1 --2- 172.21.15.119:0/1705534772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0142ea0 0x7fd5f00b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f759d700 1 -- 172.21.15.119:0/1705534772 >> 172.21.15.119:0/1705534772 conn(0x7fd5f00ce430 msgr2=0x7fd5f00ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f759d700 1 -- 172.21.15.119:0/1705534772 shutdown_connections 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.634+0000 7fd5f759d700 1 -- 172.21.15.119:0/1705534772 wait complete. 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.636+0000 7fd5f759d700 1 Processor -- start 2023-10-08T01:12:29.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.636+0000 7fd5f759d700 1 -- start start 2023-10-08T01:12:29.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f759d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f006c0f0 0x7fd5f01401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f759d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f0142ea0 0x7fd5f00acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f759d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159ea0 0x7fd5f00afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f759d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f01406e0 con 0x7fd5f0159ea0 2023-10-08T01:12:29.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f759d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f0169e60 con 0x7fd5f0142ea0 2023-10-08T01:12:29.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f759d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f016a140 con 0x7fd5f006c0f0 2023-10-08T01:12:29.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f6d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159ea0 0x7fd5f00afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f5d9a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f0142ea0 0x7fd5f00acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f659b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f006c0f0 0x7fd5f01401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:29.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f6d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159ea0 0x7fd5f00afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58798/0 (socket says 172.21.15.119:58798) 2023-10-08T01:12:29.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.637+0000 7fd5f6d9c700 1 -- 172.21.15.119:0/3484625683 learned_addr learned my addr 172.21.15.119:0/3484625683 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:29.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f5d9a700 1 -- 172.21.15.119:0/3484625683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f006c0f0 msgr2=0x7fd5f01401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f5d9a700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f006c0f0 0x7fd5f01401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f5d9a700 1 -- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159ea0 msgr2=0x7fd5f00afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f5d9a700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159ea0 0x7fd5f00afee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f5d9a700 1 -- 172.21.15.119:0/3484625683 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ec0094f0 con 0x7fd5f0142ea0 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f6d9c700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159ea0 0x7fd5f00afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f5d9a700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f0142ea0 0x7fd5f00acc20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c5b0 tx=0x7fd5e0007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5e67fc700 1 -- 172.21.15.119:0/3484625683 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0005b40 con 0x7fd5f0142ea0 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5e67fc700 1 -- 172.21.15.119:0/3484625683 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0005ca0 con 0x7fd5f0142ea0 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5e67fc700 1 -- 172.21.15.119:0/3484625683 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0007320 con 0x7fd5f0142ea0 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e8009a20 con 0x7fd5f0142ea0 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.638+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f016a700 con 0x7fd5f0142ea0 2023-10-08T01:12:29.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.640+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f016a8f0 con 0x7fd5f0142ea0 2023-10-08T01:12:29.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.640+0000 7fd5e67fc700 1 -- 172.21.15.119:0/3484625683 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fd5e001b3b0 con 0x7fd5f0142ea0 2023-10-08T01:12:29.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.641+0000 7fd5e67fc700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5d40670e0 0x7fd5d4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:29.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.641+0000 7fd5e67fc700 1 -- 172.21.15.119:0/3484625683 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fd5e0083f80 con 0x7fd5f0142ea0 2023-10-08T01:12:29.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.641+0000 7fd5f659b700 1 -- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5d40670e0 msgr2=0x7fd5d4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:29.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.641+0000 7fd5f659b700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5d40670e0 0x7fd5d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:29.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.645+0000 7fd5e67fc700 1 -- 172.21.15.119:0/3484625683 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd5e0050c40 con 0x7fd5f0142ea0 2023-10-08T01:12:29.684 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:29.683+0000 7fdff959c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:12:29.782 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:29.781+0000 7fdff959c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:12:29.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.842+0000 7fd5f659b700 1 -- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5d40670e0 msgr2=0x7fd5d4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:29.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.842+0000 7fd5f659b700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5d40670e0 0x7fd5d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:29.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.887+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5f0169fa0 con 0x7fd5f0142ea0 2023-10-08T01:12:29.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.888+0000 7fd5e67fc700 1 -- 172.21.15.119:0/3484625683 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fd5f0169fa0 con 0x7fd5f0142ea0 2023-10-08T01:12:29.890 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:29.890 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3441253316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3441253316},{"type":"v1","addr":"172.21.15.184:6834","nonce":3441253316}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/3613948578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3613948578},{"type":"v1","addr":"172.21.15.119:6833","nonce":3613948578}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/765135141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":765135141},{"type":"v1","addr":"172.21.15.119:6835","nonce":765135141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:02.279676+0000","modified":"2023-10-08T01:12:05.851791+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6708},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6708":{"gid":6708,"name":"b","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/910230897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":910230897},{"type":"v1","addr":"172.21.15.184:6835","nonce":910230897}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-10-08T01:12:29.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5d40670e0 msgr2=0x7fd5d4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:29.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5d40670e0 0x7fd5d4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f0142ea0 msgr2=0x7fd5f00acc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f0142ea0 0x7fd5f00acc20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd5e000c5b0 tx=0x7fd5e0007d50 comp rx=0 tx=0).stop 2023-10-08T01:12:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 shutdown_connections 2023-10-08T01:12:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5d40670e0 0x7fd5d4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f006c0f0 0x7fd5f01401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159ea0 0x7fd5f00afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 --2- 172.21.15.119:0/3484625683 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f0142ea0 0x7fd5f00acc20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:29.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 >> 172.21.15.119:0/3484625683 conn(0x7fd5f00ce430 msgr2=0x7fd5f006a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:29.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 shutdown_connections 2023-10-08T01:12:29.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:29.891+0000 7fd5f759d700 1 -- 172.21.15.119:0/3484625683 wait complete. 2023-10-08T01:12:29.896 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 39 2023-10-08T01:12:29.910 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.092+0000 7f16b0dff700 1 Processor -- start 2023-10-08T01:12:30.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.092+0000 7f16b0dff700 1 -- start start 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.093+0000 7f16b0dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.093+0000 7f16b0dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f16ac06c0a0 0x7f16ac15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.093+0000 7f16b0dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16ac15d7e0 con 0x7f16ac15df50 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.093+0000 7f16b0dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16ac15dac0 con 0x7f16ac06c0a0 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.093+0000 7f16b0dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac15aa30 con 0x7f16ac06bcd0 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16aa59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16a9d9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f16ac06c0a0 0x7f16ac15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16a9d9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f16ac06c0a0 0x7f16ac15a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:59408/0 (socket says 172.21.15.119:59408) 2023-10-08T01:12:30.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16a9d9b700 1 -- 172.21.15.119:0/3855644698 learned_addr learned my addr 172.21.15.119:0/3855644698 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:30.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16aa59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:36882/0 (socket says 172.21.15.119:36882) 2023-10-08T01:12:30.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16aad9d700 1 --1- 172.21.15.119:0/3855644698 >> v1:172.21.15.184:6789/0 conn(0x7f16ac15df50 0x7f16ac0c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:33410/0 (socket says 172.21.15.119:33410) 2023-10-08T01:12:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16a9d9b700 1 -- 172.21.15.119:0/3855644698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 msgr2=0x7f16ac0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16a9d9b700 1 --2- 172.21.15.119:0/3855644698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16a9d9b700 1 -- 172.21.15.119:0/3855644698 >> v1:172.21.15.184:6789/0 conn(0x7f16ac15df50 legacy=0x7f16ac0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16a9d9b700 1 -- 172.21.15.119:0/3855644698 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16ac15ad10 con 0x7f16ac06c0a0 2023-10-08T01:12:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.094+0000 7f16a959a700 1 -- 172.21.15.119:0/3855644698 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1603807098 0 0) 0x7f16ac15d7e0 con 0x7f16ac15df50 2023-10-08T01:12:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.095+0000 7f16aa59c700 1 --2- 172.21.15.119:0/3855644698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.095+0000 7f16a9d9b700 1 --2- 172.21.15.119:0/3855644698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 0x7f16ac15a1e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f16a00099b0 tx=0x7f16a0009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bfb3a2665624a0c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:30.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.095+0000 7f16a959a700 1 -- 172.21.15.119:0/3855644698 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0009ea0 con 0x7f16ac06c0a0 2023-10-08T01:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.095+0000 7f16a959a700 1 -- 172.21.15.119:0/3855644698 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a000b860 con 0x7f16ac06c0a0 2023-10-08T01:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.095+0000 7f16a959a700 1 -- 172.21.15.119:0/3855644698 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a000bc20 con 0x7f16ac06c0a0 2023-10-08T01:12:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.096+0000 7f16b0dff700 1 -- 172.21.15.119:0/3855644698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 msgr2=0x7f16ac15a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.096+0000 7f16b0dff700 1 --2- 172.21.15.119:0/3855644698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 0x7f16ac15a1e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f16a00099b0 tx=0x7f16a0009250 comp rx=0 tx=0).stop 2023-10-08T01:12:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.096+0000 7f16b0dff700 1 -- 172.21.15.119:0/3855644698 shutdown_connections 2023-10-08T01:12:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.096+0000 7f16b0dff700 1 --2- 172.21.15.119:0/3855644698 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.096+0000 7f16b0dff700 1 --2- 172.21.15.119:0/3855644698 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 0x7f16ac15a1e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.096+0000 7f16b0dff700 1 -- 172.21.15.119:0/3855644698 >> 172.21.15.119:0/3855644698 conn(0x7f16ac0ce600 msgr2=0x7f16ac0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.096+0000 7f16b0dff700 1 -- 172.21.15.119:0/3855644698 shutdown_connections 2023-10-08T01:12:30.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.096+0000 7f16b0dff700 1 -- 172.21.15.119:0/3855644698 wait complete. 2023-10-08T01:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.098+0000 7f16b0dff700 1 Processor -- start 2023-10-08T01:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.099+0000 7f16b0dff700 1 -- start start 2023-10-08T01:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16b0dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16b0dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 0x7f16ac169fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16b0dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ac15df50 0x7f16ac0c69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16b0dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac0b13a0 con 0x7f16ac15df50 2023-10-08T01:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16b0dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac16a7f0 con 0x7f16ac06c0a0 2023-10-08T01:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16b0dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac16aad0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16aad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ac15df50 0x7f16ac0c69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16aad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ac15df50 0x7f16ac0c69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58812/0 (socket says 172.21.15.119:58812) 2023-10-08T01:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16aad9d700 1 -- 172.21.15.119:0/2078295500 learned_addr learned my addr 172.21.15.119:0/2078295500 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16aa59c700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.100+0000 7f16a9d9b700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 0x7f16ac169fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f16aa59c700 1 -- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 msgr2=0x7f16ac169fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f16aa59c700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 0x7f16ac169fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f16aa59c700 1 -- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ac15df50 msgr2=0x7f16ac0c69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f16aa59c700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ac15df50 0x7f16ac0c69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f16aa59c700 1 -- 172.21.15.119:0/2078295500 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c009420 con 0x7f16ac06bcd0 2023-10-08T01:12:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f16aa59c700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0c5b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f169c00ea00 tx=0x7f169c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f169a7fc700 1 -- 172.21.15.119:0/2078295500 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00c6a0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f169a7fc700 1 -- 172.21.15.119:0/2078295500 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169c00c800 con 0x7f16ac06bcd0 2023-10-08T01:12:30.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f169a7fc700 1 -- 172.21.15.119:0/2078295500 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169c00cba0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.101+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a00094f0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.102+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac0b16e0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.103+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac16a5a0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.104+0000 7f169a7fc700 1 -- 172.21.15.119:0/2078295500 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f169c013470 con 0x7f16ac06bcd0 2023-10-08T01:12:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.105+0000 7f169a7fc700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f168c0670e0 0x7f168c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.105+0000 7f169a7fc700 1 -- 172.21.15.119:0/2078295500 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f169c084cf0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.105+0000 7f16a9d9b700 1 -- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f168c0670e0 msgr2=0x7f168c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:30.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.105+0000 7f16a9d9b700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f168c0670e0 0x7f168c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:30.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.109+0000 7f169a7fc700 1 -- 172.21.15.119:0/2078295500 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f169c0519b0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.286 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:30.286+0000 7f5dfd27c180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:12:30.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.306+0000 7f16a9d9b700 1 -- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f168c0670e0 msgr2=0x7f168c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:30.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.306+0000 7f16a9d9b700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f168c0670e0 0x7f168c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:30.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.306+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16ac0b1870 con 0x7f16ac06bcd0 2023-10-08T01:12:30.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.308+0000 7f169a7fc700 1 -- 172.21.15.119:0/2078295500 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16371 (secure 0 0 0) 0x7f169c054fd0 con 0x7f16ac06bcd0 2023-10-08T01:12:30.309 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:30.309 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":92,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:23.311955+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:00.376380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:01.260893+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2785931715":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/1171405380":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/2173962022":"2023-10-09T01:12:23.311889+0000","172.21.15.119:6837/85511":"2023-10-09T01:12:23.311889+0000","172.21.15.119:0/2446352006":"2023-10-09T01:12:23.311889+0000","172.21.15.119:6836/85511":"2023-10-09T01:12:23.311889+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:30.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f168c0670e0 msgr2=0x7f168c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:30.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f168c0670e0 0x7f168c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 msgr2=0x7f16ac0c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0c5b40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f169c00ea00 tx=0x7f169c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 shutdown_connections 2023-10-08T01:12:30.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f168c0670e0 0x7f168c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16ac06bcd0 0x7f16ac0c5b40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16ac15df50 0x7f16ac0c69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 --2- 172.21.15.119:0/2078295500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f16ac06c0a0 0x7f16ac169fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 >> 172.21.15.119:0/2078295500 conn(0x7f16ac0ce600 msgr2=0x7f16ac1566a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:30.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 shutdown_connections 2023-10-08T01:12:30.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.310+0000 7f16b0dff700 1 -- 172.21.15.119:0/2078295500 wait complete. 2023-10-08T01:12:30.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T01:12:30.514 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:30.514+0000 7fdff959c180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:12:30.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.518+0000 7f9871b9d700 1 Processor -- start 2023-10-08T01:12:30.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.518+0000 7f9871b9d700 1 -- start start 2023-10-08T01:12:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f9871b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f986c1597f0 0x7f986c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f9871b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f986c15d230 0x7f986c15d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f9871b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c15db70 con 0x7f986c159420 2023-10-08T01:12:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f9871b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986c0b11d0 con 0x7f986c1597f0 2023-10-08T01:12:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f9871b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986c0b14b0 con 0x7f986c15d230 2023-10-08T01:12:30.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f9870b9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f986c159420 0x7f986c0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47944/0 (socket says 172.21.15.119:47944) 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f986bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f986c1597f0 0x7f986c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f9870b9b700 1 -- 172.21.15.119:0/1694688397 learned_addr learned my addr 172.21.15.119:0/1694688397 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f987139c700 1 --2- 172.21.15.119:0/1694688397 >> v2:172.21.15.184:3300/0 conn(0x7f986c15d230 0x7f986c15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f986b7fe700 1 -- 172.21.15.119:0/1694688397 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2295449084 0 0) 0x7f986c15db70 con 0x7f986c159420 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f986bfff700 1 -- 172.21.15.119:0/1694688397 >> v2:172.21.15.184:3300/0 conn(0x7f986c15d230 msgr2=0x7f986c15d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f986bfff700 1 --2- 172.21.15.119:0/1694688397 >> v2:172.21.15.184:3300/0 conn(0x7f986c15d230 0x7f986c15d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f986bfff700 1 -- 172.21.15.119:0/1694688397 >> v1:172.21.15.119:6789/0 conn(0x7f986c159420 legacy=0x7f986c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.520+0000 7f986bfff700 1 -- 172.21.15.119:0/1694688397 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c0b1790 con 0x7f986c1597f0 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.521+0000 7f986bfff700 1 --2- 172.21.15.119:0/1694688397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c1597f0 0x7f986c0b2270 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f986c15d170 tx=0x7f98580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f78f039fbe380b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.521+0000 7f986b7fe700 1 -- 172.21.15.119:0/1694688397 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858011630 con 0x7f986c1597f0 2023-10-08T01:12:30.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.521+0000 7f986b7fe700 1 -- 172.21.15.119:0/1694688397 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9858011790 con 0x7f986c1597f0 2023-10-08T01:12:30.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.521+0000 7f986b7fe700 1 -- 172.21.15.119:0/1694688397 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9858011b30 con 0x7f986c1597f0 2023-10-08T01:12:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.521+0000 7f9871b9d700 1 -- 172.21.15.119:0/1694688397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c1597f0 msgr2=0x7f986c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.521+0000 7f9871b9d700 1 --2- 172.21.15.119:0/1694688397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c1597f0 0x7f986c0b2270 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f986c15d170 tx=0x7f98580091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.522+0000 7f9871b9d700 1 -- 172.21.15.119:0/1694688397 shutdown_connections 2023-10-08T01:12:30.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.522+0000 7f9871b9d700 1 --2- 172.21.15.119:0/1694688397 >> v2:172.21.15.184:3300/0 conn(0x7f986c15d230 0x7f986c15d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.522+0000 7f9871b9d700 1 --2- 172.21.15.119:0/1694688397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c1597f0 0x7f986c0b2270 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.522+0000 7f9871b9d700 1 -- 172.21.15.119:0/1694688397 >> 172.21.15.119:0/1694688397 conn(0x7f986c0d0660 msgr2=0x7f986c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.522+0000 7f9871b9d700 1 -- 172.21.15.119:0/1694688397 shutdown_connections 2023-10-08T01:12:30.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.522+0000 7f9871b9d700 1 -- 172.21.15.119:0/1694688397 wait complete. 2023-10-08T01:12:30.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.524+0000 7f9871b9d700 1 Processor -- start 2023-10-08T01:12:30.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.524+0000 7f9871b9d700 1 -- start start 2023-10-08T01:12:30.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f9871b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986c159420 0x7f986c169c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f9871b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f986c1597f0 0x7f986c16c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f9871b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c15d230 0x7f986c0c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f9871b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c16a1d0 con 0x7f986c15d230 2023-10-08T01:12:30.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f9871b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c15b9f0 con 0x7f986c1597f0 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f9871b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c15ade0 con 0x7f986c159420 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f987139c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c15d230 0x7f986c0c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f987139c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c15d230 0x7f986c0c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58836/0 (socket says 172.21.15.119:58836) 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f987139c700 1 -- 172.21.15.119:0/4179286929 learned_addr learned my addr 172.21.15.119:0/4179286929 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f986bfff700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f986c1597f0 0x7f986c16c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f9870b9b700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986c159420 0x7f986c169c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f987139c700 1 -- 172.21.15.119:0/4179286929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986c159420 msgr2=0x7f986c169c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f987139c700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986c159420 0x7f986c169c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f987139c700 1 -- 172.21.15.119:0/4179286929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f986c1597f0 msgr2=0x7f986c16c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f987139c700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f986c1597f0 0x7f986c16c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.526+0000 7f987139c700 1 -- 172.21.15.119:0/4179286929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858009420 con 0x7f986c15d230 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.527+0000 7f987139c700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c15d230 0x7f986c0c54b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9864009df0 tx=0x7f986400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.527+0000 7f9868ff9700 1 -- 172.21.15.119:0/4179286929 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400caa0 con 0x7f986c15d230 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.527+0000 7f9868ff9700 1 -- 172.21.15.119:0/4179286929 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986400f040 con 0x7f986c15d230 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.527+0000 7f9868ff9700 1 -- 172.21.15.119:0/4179286929 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986400ce70 con 0x7f986c15d230 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.527+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c15b0c0 con 0x7f986c15d230 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.527+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c15b680 con 0x7f986c15d230 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.528+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c15bb60 con 0x7f986c15d230 2023-10-08T01:12:30.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.529+0000 7f9868ff9700 1 -- 172.21.15.119:0/4179286929 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f9864014690 con 0x7f986c15d230 2023-10-08T01:12:30.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.530+0000 7f9868ff9700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98500670e0 0x7f9850069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:30.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.530+0000 7f9868ff9700 1 -- 172.21.15.119:0/4179286929 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f9864085400 con 0x7f986c15d230 2023-10-08T01:12:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.530+0000 7f9870b9b700 1 -- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98500670e0 msgr2=0x7f9850069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:30.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.530+0000 7f9870b9b700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98500670e0 0x7f9850069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:30.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.534+0000 7f9868ff9700 1 -- 172.21.15.119:0/4179286929 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9864055ae0 con 0x7f986c15d230 2023-10-08T01:12:30.575 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:30.575+0000 7f5dfd27c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:12:30.667 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:30.667+0000 7f5dfd27c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:12:30.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.731+0000 7f9870b9b700 1 -- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98500670e0 msgr2=0x7f9850069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:30.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.731+0000 7f9870b9b700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98500670e0 0x7f9850069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:30.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.778+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f986c04c3c0 con 0x7f986c15d230 2023-10-08T01:12:30.790 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T01:12:30.778+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.782+0000 7f9868ff9700 1 -- 172.21.15.119:0/4179286929 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9864055670 con 0x7f986c15d230 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.784+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98500670e0 msgr2=0x7f9850069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.784+0000 7f9871b9d700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98500670e0 0x7f9850069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.784+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c15d230 msgr2=0x7f986c0c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.784+0000 7f9871b9d700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c15d230 0x7f986c0c54b0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9864009df0 tx=0x7f986400b040 comp rx=0 tx=0).stop 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.785+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 shutdown_connections 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.785+0000 7f9871b9d700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98500670e0 0x7f9850069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.785+0000 7f9871b9d700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f986c159420 0x7f986c169c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.785+0000 7f9871b9d700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f986c15d230 0x7f986c0c54b0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.785+0000 7f9871b9d700 1 --2- 172.21.15.119:0/4179286929 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f986c1597f0 0x7f986c16c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:30.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.785+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 >> 172.21.15.119:0/4179286929 conn(0x7f986c0d0660 msgr2=0x7f986c158120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:30.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.785+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 shutdown_connections 2023-10-08T01:12:30.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:30.785+0000 7f9871b9d700 1 -- 172.21.15.119:0/4179286929 wait complete. 2023-10-08T01:12:30.792 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T01:12:30.792 INFO:tasks.ceph.mds.b.smithi184.stderr: -1248> 2023-10-08T01:12:07.797+0000 7fe29f5f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:12:30.805 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T01:12:30.817 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:30.814+0000 7fdff959c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:12:30.829 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:30.829+0000 7f5dfd27c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:12:30.832 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T01:12:30.913 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:30.912+0000 7fdff959c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:12:30.937 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:30.937+0000 7f5dfd27c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:12:31.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.002+0000 7f31b219d700 1 Processor -- start 2023-10-08T01:12:31.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.003+0000 7f31b219d700 1 -- start start 2023-10-08T01:12:31.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b219d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac0c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b219d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f31ac06bfa0 0x7f31ac0c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b219d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31ac0ca920 con 0x7f31ac068390 2023-10-08T01:12:31.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b219d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31ac06af00 con 0x7f31ac06bfa0 2023-10-08T01:12:31.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b219d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac06b1e0 con 0x7f31ac159a70 2023-10-08T01:12:31.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b099a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f31ac06bfa0 0x7f31ac0c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:31.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b199c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f31ac068390 0x7f31ac0caec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:33424/0 (socket says 172.21.15.119:33424) 2023-10-08T01:12:31.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b199c700 1 -- 172.21.15.119:0/2781405344 learned_addr learned my addr 172.21.15.119:0/2781405344 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:31.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.004+0000 7f31b119b700 1 --2- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac0c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:31.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.005+0000 7f31a3fff700 1 -- 172.21.15.119:0/2781405344 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688200480 0 0) 0x7f31ac0ca920 con 0x7f31ac068390 2023-10-08T01:12:31.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.005+0000 7f31b099a700 1 -- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 msgr2=0x7f31ac0c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:31.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.005+0000 7f31b099a700 1 --2- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac0c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.005+0000 7f31b099a700 1 -- 172.21.15.119:0/2781405344 >> v1:172.21.15.184:6789/0 conn(0x7f31ac068390 legacy=0x7f31ac0caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.005+0000 7f31b099a700 1 -- 172.21.15.119:0/2781405344 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac06b4c0 con 0x7f31ac06bfa0 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.005+0000 7f31b119b700 1 --2- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.005+0000 7f31b099a700 1 --2- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 0x7f31ac0c8830 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f319c0098d0 tx=0x7f319c009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e485779edace6bdb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31a3fff700 1 -- 172.21.15.119:0/2781405344 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c005c10 con 0x7f31ac06bfa0 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31a3fff700 1 -- 172.21.15.119:0/2781405344 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c005d70 con 0x7f31ac06bfa0 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31a3fff700 1 -- 172.21.15.119:0/2781405344 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c006280 con 0x7f31ac06bfa0 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31b219d700 1 -- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 msgr2=0x7f31ac0c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31b219d700 1 --2- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 0x7f31ac0c8830 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f319c0098d0 tx=0x7f319c009120 comp rx=0 tx=0).stop 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31b219d700 1 -- 172.21.15.119:0/2781405344 shutdown_connections 2023-10-08T01:12:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31b219d700 1 --2- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac0c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31b219d700 1 --2- 172.21.15.119:0/2781405344 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 0x7f31ac0c8830 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31b219d700 1 -- 172.21.15.119:0/2781405344 >> 172.21.15.119:0/2781405344 conn(0x7f31ac0cddc0 msgr2=0x7f31ac0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31b219d700 1 -- 172.21.15.119:0/2781405344 shutdown_connections 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.006+0000 7f31b219d700 1 -- 172.21.15.119:0/2781405344 wait complete. 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.008+0000 7f31b219d700 1 Processor -- start 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.008+0000 7f31b219d700 1 -- start start 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b219d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31ac068390 0x7f31ac0b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b219d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 0x7f31ac1575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b219d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac169ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b219d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac0b31a0 con 0x7f31ac068390 2023-10-08T01:12:31.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b219d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac158320 con 0x7f31ac06bfa0 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b219d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31ac0c4fa0 con 0x7f31ac159a70 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31ac068390 0x7f31ac0b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31ac068390 0x7f31ac0b2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58852/0 (socket says 172.21.15.119:58852) 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b119b700 1 -- 172.21.15.119:0/1662211183 learned_addr learned my addr 172.21.15.119:0/1662211183 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b099a700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 0x7f31ac1575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.009+0000 7f31b199c700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac169ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b099a700 1 -- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 msgr2=0x7f31ac169ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b099a700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac169ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b099a700 1 -- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31ac068390 msgr2=0x7f31ac0b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b099a700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31ac068390 0x7f31ac0b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b099a700 1 -- 172.21.15.119:0/1662211183 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a8009420 con 0x7f31ac06bfa0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b119b700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31ac068390 0x7f31ac0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b099a700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 0x7f31ac1575a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f319c015000 tx=0x7f319c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c01a700 con 0x7f31ac06bfa0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c01a880 con 0x7f31ac06bfa0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319c0093c0 con 0x7f31ac06bfa0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c01ac40 con 0x7f31ac06bfa0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.010+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31ac0c4950 con 0x7f31ac06bfa0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.011+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31ac0c5110 con 0x7f31ac06bfa0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.012+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 44) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f319c011420 con 0x7f31ac06bfa0 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.013+0000 7f31a17fa700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 0x7f3190069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.013+0000 7f31b119b700 1 -- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 msgr2=0x7f3190069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:31.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.013+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f319c0863e0 con 0x7f31ac06bfa0 2023-10-08T01:12:31.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.013+0000 7f31b119b700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 0x7f3190069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:31.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.017+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f319c053050 con 0x7f31ac06bfa0 2023-10-08T01:12:31.072 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.071+0000 7fdff959c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:12:31.127 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:31.126+0000 7f5dfd27c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:12:31.177 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.176+0000 7fdff959c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:12:31.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.214+0000 7f31b119b700 1 -- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 msgr2=0x7f3190069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:31.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.214+0000 7f31b119b700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 0x7f3190069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:31.217 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:31.217+0000 7f5dfd27c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:12:31.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.257+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f31ac063d10 con 0x7f31ac06bfa0 2023-10-08T01:12:31.308 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:31.307+0000 7f5dfd27c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:12:31.368 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.367+0000 7fdff959c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:12:31.402 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:31.402+0000 7f5dfd27c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:12:31.462 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.461+0000 7fdff959c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:12:31.553 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.552+0000 7fdff959c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:12:31.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.615+0000 7f31b119b700 1 -- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 msgr2=0x7f3190069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:31.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.615+0000 7f31b119b700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 0x7f3190069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:12:31.738 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.646+0000 7fdff959c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:12:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.658+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mgrmap(e 45) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f319c053420 con 0x7f31ac06bfa0 2023-10-08T01:12:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.658+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 msgr2=0x7f3190069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.658+0000 7f31a17fa700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 0x7f3190069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.742 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.721+0000 7fdf749c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.742 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.721+0000 7fdf749c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.742 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.721+0000 7fdf749c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.742 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.721+0000 7fdf749c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.742 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.721+0000 7fdf749c3700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.743 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.722+0000 7fdf779c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.743 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.722+0000 7fdf779c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.743 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.722+0000 7fdf779c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.743 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.722+0000 7fdf779c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.743 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:31.722+0000 7fdf779c9700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:12:31.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.784+0000 7f31a17fa700 1 -- 172.21.15.119:0/1662211183 <== mon.1 v2:172.21.15.184:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f319c06c1e0 con 0x7f31ac06bfa0 2023-10-08T01:12:31.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 msgr2=0x7f31ac1575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:31.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 0x7f31ac1575a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f319c015000 tx=0x7f319c016000 comp rx=0 tx=0).stop 2023-10-08T01:12:31.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 shutdown_connections 2023-10-08T01:12:31.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f31900670e0 0x7f3190069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31ac159a70 0x7f31ac169ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31ac068390 0x7f31ac0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 --2- 172.21.15.119:0/1662211183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31ac06bfa0 0x7f31ac1575a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 >> 172.21.15.119:0/1662211183 conn(0x7f31ac0cddc0 msgr2=0x7f31ac0cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:31.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 shutdown_connections 2023-10-08T01:12:31.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.786+0000 7f31b219d700 1 -- 172.21.15.119:0/1662211183 wait complete. 2023-10-08T01:12:31.806 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T01:12:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.994+0000 7f8b111ff700 1 Processor -- start 2023-10-08T01:12:31.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.994+0000 7f8b111ff700 1 -- start start 2023-10-08T01:12:31.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b111ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c1563b0 0x7f8b0c156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b111ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8b0c15a440 0x7f8b0c0acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b111ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8b0c068890 0x7f8b0c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:31.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b111ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0c15a9b0 con 0x7f8b0c068890 2023-10-08T01:12:31.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b111ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b0c15ac90 con 0x7f8b0c15a440 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b111ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c15af70 con 0x7f8b0c1563b0 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b0b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8b0c068890 0x7f8b0c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b0b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8b0c068890 0x7f8b0c0c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58976/0 (socket says 172.21.15.119:58976) 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b0b59e700 1 -- 172.21.15.119:0/2336115267 learned_addr learned my addr 172.21.15.119:0/2336115267 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b0a59c700 1 --2- 172.21.15.119:0/2336115267 >> v2:172.21.15.184:3300/0 conn(0x7f8b0c15a440 0x7f8b0c0acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.995+0000 7f8b0ad9d700 1 --2- 172.21.15.119:0/2336115267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c1563b0 0x7f8b0c156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b0b59e700 1 -- 172.21.15.119:0/2336115267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c1563b0 msgr2=0x7f8b0c156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b0b59e700 1 --2- 172.21.15.119:0/2336115267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c1563b0 0x7f8b0c156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b0b59e700 1 -- 172.21.15.119:0/2336115267 >> v2:172.21.15.184:3300/0 conn(0x7f8b0c15a440 msgr2=0x7f8b0c0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b0b59e700 1 --2- 172.21.15.119:0/2336115267 >> v2:172.21.15.184:3300/0 conn(0x7f8b0c15a440 0x7f8b0c0acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:31.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b0b59e700 1 -- 172.21.15.119:0/2336115267 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c15b250 con 0x7f8b0c068890 2023-10-08T01:12:31.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b0a59c700 1 --2- 172.21.15.119:0/2336115267 >> v2:172.21.15.184:3300/0 conn(0x7f8b0c15a440 0x7f8b0c0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:31.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b0b59e700 1 --2- 172.21.15.119:0/2336115267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c068890 0x7f8b0c0c8230 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c06aa20 tx=0x7f8b04009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b37a77c58d2ab4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:31.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b09d9b700 1 -- 172.21.15.119:0/2336115267 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0400e070 con 0x7f8b0c068890 2023-10-08T01:12:31.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b09d9b700 1 -- 172.21.15.119:0/2336115267 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0400ba50 con 0x7f8b0c068890 2023-10-08T01:12:31.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.996+0000 7f8b09d9b700 1 -- 172.21.15.119:0/2336115267 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040092e0 con 0x7f8b0c068890 2023-10-08T01:12:32.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 -- 172.21.15.119:0/2336115267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c068890 msgr2=0x7f8b0c0c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:32.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 --2- 172.21.15.119:0/2336115267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c068890 0x7f8b0c0c8230 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c06aa20 tx=0x7f8b04009840 comp rx=0 tx=0).stop 2023-10-08T01:12:32.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 -- 172.21.15.119:0/2336115267 shutdown_connections 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 --2- 172.21.15.119:0/2336115267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c068890 0x7f8b0c0c8230 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 --2- 172.21.15.119:0/2336115267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c1563b0 0x7f8b0c156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 --2- 172.21.15.119:0/2336115267 >> v2:172.21.15.184:3300/0 conn(0x7f8b0c15a440 0x7f8b0c0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 -- 172.21.15.119:0/2336115267 >> 172.21.15.119:0/2336115267 conn(0x7f8b0c0c8f90 msgr2=0x7f8b0c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 -- 172.21.15.119:0/2336115267 shutdown_connections 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.997+0000 7f8b111ff700 1 -- 172.21.15.119:0/2336115267 wait complete. 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.998+0000 7f8b111ff700 1 Processor -- start 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.998+0000 7f8b111ff700 1 -- start start 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.999+0000 7f8b111ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c068890 0x7f8b0c157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.999+0000 7f8b111ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0c1563b0 0x7f8b0c0c2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:32.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.999+0000 7f8b111ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c15a440 0x7f8b0c14c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:32.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.999+0000 7f8b111ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c158170 con 0x7f8b0c15a440 2023-10-08T01:12:32.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.999+0000 7f8b111ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c0c0710 con 0x7f8b0c1563b0 2023-10-08T01:12:32.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.999+0000 7f8b111ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c0c2e70 con 0x7f8b0c068890 2023-10-08T01:12:32.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:31.999+0000 7f8b0b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c15a440 0x7f8b0c14c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:32.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c15a440 0x7f8b0c14c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58986/0 (socket says 172.21.15.119:58986) 2023-10-08T01:12:32.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0b59e700 1 -- 172.21.15.119:0/1539818401 learned_addr learned my addr 172.21.15.119:0/1539818401 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:32.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0ad9d700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c068890 0x7f8b0c157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:32.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0a59c700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0c1563b0 0x7f8b0c0c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:32.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0b59e700 1 -- 172.21.15.119:0/1539818401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c068890 msgr2=0x7f8b0c157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:32.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0b59e700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c068890 0x7f8b0c157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0b59e700 1 -- 172.21.15.119:0/1539818401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0c1563b0 msgr2=0x7f8b0c0c2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0b59e700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0c1563b0 0x7f8b0c0c2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0b59e700 1 -- 172.21.15.119:0/1539818401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b04009ae0 con 0x7f8b0c15a440 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b0b59e700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c15a440 0x7f8b0c14c000 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c06bbb0 tx=0x7f8b04018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8af2ffd700 1 -- 172.21.15.119:0/1539818401 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040032f0 con 0x7f8b0c15a440 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8af2ffd700 1 -- 172.21.15.119:0/1539818401 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b04008ce0 con 0x7f8b0c15a440 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c157e90 con 0x7f8b0c15a440 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.000+0000 7f8af2ffd700 1 -- 172.21.15.119:0/1539818401 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b040266b0 con 0x7f8b0c15a440 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.001+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c0c03a0 con 0x7f8b0c15a440 2023-10-08T01:12:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.002+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b0c0bff20 con 0x7f8b0c15a440 2023-10-08T01:12:32.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.002+0000 7f8af2ffd700 1 -- 172.21.15.119:0/1539818401 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 45) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f8b04003040 con 0x7f8b0c15a440 2023-10-08T01:12:32.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.004+0000 7f8af2ffd700 1 -- 172.21.15.119:0/1539818401 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6742+0+0 (secure 0 0 0) 0x7f8b04086350 con 0x7f8b0c15a440 2023-10-08T01:12:32.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.008+0000 7f8af2ffd700 1 -- 172.21.15.119:0/1539818401 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b0c0bff20 con 0x7f8b0c15a440 2023-10-08T01:12:32.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.200+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b0c0bff20 con 0x7f8b0c15a440 2023-10-08T01:12:32.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.662+0000 7f8af2ffd700 1 -- 172.21.15.119:0/1539818401 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 46) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f8b04086020 con 0x7f8b0c15a440 2023-10-08T01:12:32.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.663+0000 7f8af2ffd700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8af806c9e0 0x7f8af806edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:32.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.666+0000 7f8b0ad9d700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8af806c9e0 0x7f8af806edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:32.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.668+0000 7f8b0ad9d700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8af806c9e0 0x7f8af806edc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8afc006fd0 tx=0x7f8afc008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:32.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.787+0000 7f8af2ffd700 1 -- 172.21.15.119:0/1539818401 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8b04028c50 con 0x7f8b0c15a440 2023-10-08T01:12:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.791+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8af806c9e0 msgr2=0x7f8af806edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.791+0000 7f8b111ff700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8af806c9e0 0x7f8af806edc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8afc006fd0 tx=0x7f8afc008040 comp rx=0 tx=0).stop 2023-10-08T01:12:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.791+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c15a440 msgr2=0x7f8b0c14c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.791+0000 7f8b111ff700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c15a440 0x7f8b0c14c000 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c06bbb0 tx=0x7f8b04018000 comp rx=0 tx=0).stop 2023-10-08T01:12:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.792+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 shutdown_connections 2023-10-08T01:12:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.792+0000 7f8b111ff700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8af806c9e0 0x7f8af806edc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.792+0000 7f8b111ff700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b0c068890 0x7f8b0c157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.792+0000 7f8b111ff700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b0c15a440 0x7f8b0c14c000 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.792+0000 7f8b111ff700 1 --2- 172.21.15.119:0/1539818401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b0c1563b0 0x7f8b0c0c2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.792+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 >> 172.21.15.119:0/1539818401 conn(0x7f8b0c0c8f90 msgr2=0x7f8b0c0ad9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.792+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 shutdown_connections 2023-10-08T01:12:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.792+0000 7f8b111ff700 1 -- 172.21.15.119:0/1539818401 wait complete. 2023-10-08T01:12:32.795 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T01:12:32.812 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T01:12:32.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.996+0000 7f000a9ff700 1 Processor -- start 2023-10-08T01:12:32.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.996+0000 7f000a9ff700 1 -- start start 2023-10-08T01:12:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.997+0000 7f000a9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00040b42c0 0x7f00040ce620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.997+0000 7f000a9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f00040ceb60 0x7f00040cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.997+0000 7f000a9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00040cefd0 con 0x7f0004155360 2023-10-08T01:12:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.997+0000 7f000a9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000415c320 con 0x7f00040b42c0 2023-10-08T01:12:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.997+0000 7f000a9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000415b710 con 0x7f00040ceb60 2023-10-08T01:12:33.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f00037fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00040b42c0 0x7f00040ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:33.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f00037fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00040b42c0 0x7f00040ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58988/0 (socket says 172.21.15.119:58988) 2023-10-08T01:12:33.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f00037fe700 1 -- 172.21.15.119:0/1275487409 learned_addr learned my addr 172.21.15.119:0/1275487409 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:33.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f00037fe700 1 -- 172.21.15.119:0/1275487409 >> v2:172.21.15.184:3300/0 conn(0x7f00040ceb60 msgr2=0x7f00040cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:33.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f0008f9c700 1 --2- 172.21.15.119:0/1275487409 >> v2:172.21.15.184:3300/0 conn(0x7f00040ceb60 0x7f00040cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:33.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f00037fe700 1 --2- 172.21.15.119:0/1275487409 >> v2:172.21.15.184:3300/0 conn(0x7f00040ceb60 0x7f00040cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:33.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f00037fe700 1 -- 172.21.15.119:0/1275487409 >> v1:172.21.15.184:6789/0 conn(0x7f0004155360 legacy=0x7f00040b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:33.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f00037fe700 1 -- 172.21.15.119:0/1275487409 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f000415b9f0 con 0x7f00040b42c0 2023-10-08T01:12:33.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f0008f9c700 1 --2- 172.21.15.119:0/1275487409 >> v2:172.21.15.184:3300/0 conn(0x7f00040ceb60 0x7f00040cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:33.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f00037fe700 1 --2- 172.21.15.119:0/1275487409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f00040ce620 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f000406f5b0 tx=0x7efff8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ea4f05de2c457 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:33.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.998+0000 7f0002ffd700 1 -- 172.21.15.119:0/1275487409 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff800b820 con 0x7f00040b42c0 2023-10-08T01:12:33.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f0002ffd700 1 -- 172.21.15.119:0/1275487409 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff800b980 con 0x7f00040b42c0 2023-10-08T01:12:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f0002ffd700 1 -- 172.21.15.119:0/1275487409 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff800bd20 con 0x7f00040b42c0 2023-10-08T01:12:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f000a9ff700 1 -- 172.21.15.119:0/1275487409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 msgr2=0x7f00040ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f000a9ff700 1 --2- 172.21.15.119:0/1275487409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f00040ce620 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f000406f5b0 tx=0x7efff8009250 comp rx=0 tx=0).stop 2023-10-08T01:12:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f000a9ff700 1 -- 172.21.15.119:0/1275487409 shutdown_connections 2023-10-08T01:12:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f000a9ff700 1 --2- 172.21.15.119:0/1275487409 >> v2:172.21.15.184:3300/0 conn(0x7f00040ceb60 0x7f00040cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:33.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f000a9ff700 1 --2- 172.21.15.119:0/1275487409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f00040ce620 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:33.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f000a9ff700 1 -- 172.21.15.119:0/1275487409 >> 172.21.15.119:0/1275487409 conn(0x7f000406baf0 msgr2=0x7f000406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:33.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f000a9ff700 1 -- 172.21.15.119:0/1275487409 shutdown_connections 2023-10-08T01:12:33.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:32.999+0000 7f000a9ff700 1 -- 172.21.15.119:0/1275487409 wait complete. 2023-10-08T01:12:33.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.001+0000 7f000a9ff700 1 Processor -- start 2023-10-08T01:12:33.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.001+0000 7f000a9ff700 1 -- start start 2023-10-08T01:12:33.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f000a9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f0004069e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:33.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f000a9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00040ceb60 0x7f00040af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:33.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f000a9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0004155360 0x7f00041536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:33.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f000a9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000416a340 con 0x7f00040b42c0 2023-10-08T01:12:33.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f000a9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00040afe30 con 0x7f00040ceb60 2023-10-08T01:12:33.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f000a9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000406a490 con 0x7f0004155360 2023-10-08T01:12:33.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f0004069e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:33.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f0004069e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58992/0 (socket says 172.21.15.119:58992) 2023-10-08T01:12:33.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 -- 172.21.15.119:0/261869494 learned_addr learned my addr 172.21.15.119:0/261869494 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:33.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f00037fe700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00040ceb60 0x7f00040af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:33.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0008f9c700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0004155360 0x7f00041536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:33.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 -- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0004155360 msgr2=0x7f00041536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:33.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0004155360 0x7f00041536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:33.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 -- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00040ceb60 msgr2=0x7f00040af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:33.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00040ceb60 0x7f00040af8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:33.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 -- 172.21.15.119:0/261869494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff80094f0 con 0x7f00040b42c0 2023-10-08T01:12:33.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0008f9c700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0004155360 0x7f00041536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:33.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.002+0000 7f0003fff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f0004069e60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7efff400b5e0 tx=0x7efff4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:33.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.003+0000 7effebfff700 1 -- 172.21.15.119:0/261869494 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400f510 con 0x7f00040b42c0 2023-10-08T01:12:33.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.003+0000 7effebfff700 1 -- 172.21.15.119:0/261869494 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff400f670 con 0x7f00040b42c0 2023-10-08T01:12:33.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.003+0000 7effebfff700 1 -- 172.21.15.119:0/261869494 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff400fa30 con 0x7f00040b42c0 2023-10-08T01:12:33.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.003+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f000406a770 con 0x7f00040b42c0 2023-10-08T01:12:33.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.003+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f000406ad30 con 0x7f00040b42c0 2023-10-08T01:12:33.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.004+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00040718b0 con 0x7f00040b42c0 2023-10-08T01:12:33.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.004+0000 7effebfff700 1 -- 172.21.15.119:0/261869494 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 46) v1 ==== 84578+0+0 (secure 0 0 0) 0x7efff401a400 con 0x7f00040b42c0 2023-10-08T01:12:33.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.005+0000 7effebfff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efff0066d80 0x7efff0069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:33.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.005+0000 7effebfff700 1 -- 172.21.15.119:0/261869494 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6304+0+0 (secure 0 0 0) 0x7efff40840b0 con 0x7f00040b42c0 2023-10-08T01:12:33.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.006+0000 7f00037fe700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efff0066d80 0x7efff0069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:33.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.006+0000 7f00037fe700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efff0066d80 0x7efff0069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f000406f5b0 tx=0x7efff8016000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:33.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.009+0000 7effebfff700 1 -- 172.21.15.119:0/261869494 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efff4051350 con 0x7f00040b42c0 2023-10-08T01:12:33.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:33.205+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f000415bcd0 con 0x7f00040b42c0 2023-10-08T01:12:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.186+0000 7effebfff700 1 -- 172.21.15.119:0/261869494 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) v1 ==== 169+0+0 (secure 0 0 0) 0x7efff4054970 con 0x7f00040b42c0 2023-10-08T01:12:34.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efff0066d80 msgr2=0x7efff0069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:34.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efff0066d80 0x7efff0069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f000406f5b0 tx=0x7efff8016000 comp rx=0 tx=0).stop 2023-10-08T01:12:34.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 msgr2=0x7f0004069e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:34.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f0004069e60 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7efff400b5e0 tx=0x7efff4009120 comp rx=0 tx=0).stop 2023-10-08T01:12:34.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 shutdown_connections 2023-10-08T01:12:34.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efff0066d80 0x7efff0069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:34.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0004155360 0x7f00041536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:34.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00040b42c0 0x7f0004069e60 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:34.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 --2- 172.21.15.119:0/261869494 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00040ceb60 0x7f00040af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:34.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 >> 172.21.15.119:0/261869494 conn(0x7f000406baf0 msgr2=0x7f00040b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:34.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 shutdown_connections 2023-10-08T01:12:34.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.189+0000 7f000a9ff700 1 -- 172.21.15.119:0/261869494 wait complete. 2023-10-08T01:12:34.556 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T01:12:34.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T01:12:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.751+0000 7fd2e279d700 1 Processor -- start 2023-10-08T01:12:34.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.751+0000 7fd2e279d700 1 -- start start 2023-10-08T01:12:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.752+0000 7fd2e279d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd2dc0b16d0 0x7fd2dc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.752+0000 7fd2e279d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2dc0b2080 con 0x7fd2dc0b33b0 2023-10-08T01:12:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.752+0000 7fd2e279d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2dc0cfb00 con 0x7fd2dc0b4e70 2023-10-08T01:12:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.752+0000 7fd2e279d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2dc0b5c40 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.752+0000 7fd2e1f9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd2dc0b33b0 0x7fd2dc0b37d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47952/0 (socket says 172.21.15.119:47952) 2023-10-08T01:12:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.752+0000 7fd2e1f9c700 1 -- 172.21.15.119:0/3198842635 learned_addr learned my addr 172.21.15.119:0/3198842635 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2e0f9a700 1 --2- 172.21.15.119:0/3198842635 >> v2:172.21.15.184:3300/0 conn(0x7fd2dc0b16d0 0x7fd2dc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2d3fff700 1 -- 172.21.15.119:0/3198842635 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813182298 0 0) 0x7fd2dc0b2080 con 0x7fd2dc0b33b0 2023-10-08T01:12:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2d3fff700 1 -- 172.21.15.119:0/3198842635 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c8003570 con 0x7fd2dc0b33b0 2023-10-08T01:12:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2d3fff700 1 -- 172.21.15.119:0/3198842635 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529305233 0 0) 0x7fd2dc0cfb00 con 0x7fd2dc0b4e70 2023-10-08T01:12:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2d3fff700 1 -- 172.21.15.119:0/3198842635 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2dc0b2080 con 0x7fd2dc0b4e70 2023-10-08T01:12:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2e0f9a700 1 -- 172.21.15.119:0/3198842635 >> v1:172.21.15.184:6789/0 conn(0x7fd2dc0b4e70 legacy=0x7fd2dc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2e0f9a700 1 -- 172.21.15.119:0/3198842635 >> v1:172.21.15.119:6789/0 conn(0x7fd2dc0b33b0 legacy=0x7fd2dc0b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2e0f9a700 1 -- 172.21.15.119:0/3198842635 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2dc0b5f20 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.753+0000 7fd2e0f9a700 1 --2- 172.21.15.119:0/3198842635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0b1b40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0098e0 tx=0x7fd2cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5db2a7d581886ca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2d3fff700 1 -- 172.21.15.119:0/3198842635 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc011630 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2d3fff700 1 -- 172.21.15.119:0/3198842635 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc011790 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2d3fff700 1 -- 172.21.15.119:0/3198842635 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc011b30 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2e279d700 1 -- 172.21.15.119:0/3198842635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b16d0 msgr2=0x7fd2dc0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2e279d700 1 --2- 172.21.15.119:0/3198842635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0b1b40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0098e0 tx=0x7fd2cc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2e279d700 1 -- 172.21.15.119:0/3198842635 shutdown_connections 2023-10-08T01:12:35.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2e279d700 1 --2- 172.21.15.119:0/3198842635 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0b1b40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2e279d700 1 -- 172.21.15.119:0/3198842635 >> 172.21.15.119:0/3198842635 conn(0x7fd2dc0d10c0 msgr2=0x7fd2dc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:35.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2e279d700 1 -- 172.21.15.119:0/3198842635 shutdown_connections 2023-10-08T01:12:35.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.754+0000 7fd2e279d700 1 -- 172.21.15.119:0/3198842635 wait complete. 2023-10-08T01:12:35.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.756+0000 7fd2e279d700 1 Processor -- start 2023-10-08T01:12:35.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.756+0000 7fd2e279d700 1 -- start start 2023-10-08T01:12:35.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e279d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e279d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2dc0b33b0 0x7fd2dc15f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e279d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b4e70 0x7fd2dc15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e279d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1583c0 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e279d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0cc070 con 0x7fd2dc0b4e70 2023-10-08T01:12:35.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e279d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc0cc350 con 0x7fd2dc0b33b0 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e179b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0cb9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59002/0 (socket says 172.21.15.119:59002) 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e179b700 1 -- 172.21.15.119:0/3299188235 learned_addr learned my addr 172.21.15.119:0/3299188235 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e0f9a700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2dc0b33b0 0x7fd2dc15f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e1f9c700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b4e70 0x7fd2dc15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e179b700 1 -- 172.21.15.119:0/3299188235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2dc0b33b0 msgr2=0x7fd2dc15f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e179b700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2dc0b33b0 0x7fd2dc15f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e179b700 1 -- 172.21.15.119:0/3299188235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b4e70 msgr2=0x7fd2dc15fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.757+0000 7fd2e179b700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b4e70 0x7fd2dc15fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.758+0000 7fd2e179b700 1 -- 172.21.15.119:0/3299188235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc009420 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.758+0000 7fd2e179b700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0cb9b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c580 tx=0x7fd2d8007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.758+0000 7fd2d17fa700 1 -- 172.21.15.119:0/3299188235 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8010510 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.758+0000 7fd2d17fa700 1 -- 172.21.15.119:0/3299188235 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d8010670 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.758+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc0cc630 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.758+0000 7fd2d17fa700 1 -- 172.21.15.119:0/3299188235 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d8010a30 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.758+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc15add0 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.759+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc15aff0 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.760+0000 7fd2d17fa700 1 -- 172.21.15.119:0/3299188235 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2d801b400 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.761+0000 7fd2d17fa700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2c0066f30 0x7fd2c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.761+0000 7fd2d17fa700 1 -- 172.21.15.119:0/3299188235 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fd2d8084290 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.761+0000 7fd2e0f9a700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2c0066f30 0x7fd2c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.762+0000 7fd2e0f9a700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2c0066f30 0x7fd2c00693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0093f0 tx=0x7fd2cc017000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.765+0000 7fd2d17fa700 1 -- 172.21.15.119:0/3299188235 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2d80513a0 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.964+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd2dc04bb80 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.968+0000 7fd2d17fa700 1 -- 172.21.15.119:0/3299188235 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd2d80549c0 con 0x7fd2dc0b16d0 2023-10-08T01:12:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2c0066f30 msgr2=0x7fd2c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2c0066f30 0x7fd2c00693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0093f0 tx=0x7fd2cc017000 comp rx=0 tx=0).stop 2023-10-08T01:12:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2dc0b16d0 msgr2=0x7fd2dc0cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0cb9b0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd2d800c580 tx=0x7fd2d8007e10 comp rx=0 tx=0).stop 2023-10-08T01:12:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 shutdown_connections 2023-10-08T01:12:35.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd2c0066f30 0x7fd2c00693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2dc0b4e70 0x7fd2dc15fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2dc0b16d0 0x7fd2dc0cb9b0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 --2- 172.21.15.119:0/3299188235 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2dc0b33b0 0x7fd2dc15f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.970+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 >> 172.21.15.119:0/3299188235 conn(0x7fd2dc0d10c0 msgr2=0x7fd2dc0c5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:35.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.971+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 shutdown_connections 2023-10-08T01:12:35.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:34.971+0000 7fd2e279d700 1 -- 172.21.15.119:0/3299188235 wait complete. 2023-10-08T01:12:35.015 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T01:12:35.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.202+0000 7fe12b59d700 1 Processor -- start 2023-10-08T01:12:35.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.202+0000 7fe12b59d700 1 -- start start 2023-10-08T01:12:35.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.203+0000 7fe12b59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe1240cb1a0 0x7fe12415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.203+0000 7fe12b59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12415a8a0 0x7fe12406f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.203+0000 7fe12b59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe12406f5a0 0x7fe1240ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.203+0000 7fe12b59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12415b2e0 con 0x7fe1240cb1a0 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.203+0000 7fe12b59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe12406ebd0 con 0x7fe12406f5a0 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.203+0000 7fe12b59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe12415ad20 con 0x7fe12415a8a0 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe1240cb1a0 0x7fe12415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe1240cb1a0 0x7fe12415a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36882/0 (socket says 172.21.15.119:36882) 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 -- 172.21.15.119:0/1840961309 learned_addr learned my addr 172.21.15.119:0/1840961309 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12ad9c700 1 --2- 172.21.15.119:0/1840961309 >> v2:172.21.15.184:3300/0 conn(0x7fe12406f5a0 0x7fe1240ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe129d9a700 1 --2- 172.21.15.119:0/1840961309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12415a8a0 0x7fe12406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 -- 172.21.15.119:0/1840961309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12415a8a0 msgr2=0x7fe12406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 --2- 172.21.15.119:0/1840961309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12415a8a0 0x7fe12406f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 -- 172.21.15.119:0/1840961309 >> v2:172.21.15.184:3300/0 conn(0x7fe12406f5a0 msgr2=0x7fe1240ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 --2- 172.21.15.119:0/1840961309 >> v2:172.21.15.184:3300/0 conn(0x7fe12406f5a0 0x7fe1240ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 -- 172.21.15.119:0/1840961309 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12415b000 con 0x7fe1240cb1a0 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe12a59b700 1 --2- 172.21.15.119:0/1840961309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe12415a360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe12406d580 tx=0x7fe120009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff53c4752bc2c281 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.204+0000 7fe129599700 1 -- 172.21.15.119:0/1840961309 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000b8f0 con 0x7fe1240cb1a0 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe129599700 1 -- 172.21.15.119:0/1840961309 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12000ba50 con 0x7fe1240cb1a0 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe129599700 1 -- 172.21.15.119:0/1840961309 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000bdf0 con 0x7fe1240cb1a0 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 -- 172.21.15.119:0/1840961309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 msgr2=0x7fe12415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1840961309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe12415a360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe12406d580 tx=0x7fe120009250 comp rx=0 tx=0).stop 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 -- 172.21.15.119:0/1840961309 shutdown_connections 2023-10-08T01:12:35.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1840961309 >> v2:172.21.15.184:3300/0 conn(0x7fe12406f5a0 0x7fe1240ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1840961309 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe12415a360 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1840961309 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12415a8a0 0x7fe12406f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 -- 172.21.15.119:0/1840961309 >> 172.21.15.119:0/1840961309 conn(0x7fe1240cc1e0 msgr2=0x7fe1240cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:35.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 -- 172.21.15.119:0/1840961309 shutdown_connections 2023-10-08T01:12:35.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.205+0000 7fe12b59d700 1 -- 172.21.15.119:0/1840961309 wait complete. 2023-10-08T01:12:35.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.207+0000 7fe12b59d700 1 Processor -- start 2023-10-08T01:12:35.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.207+0000 7fe12b59d700 1 -- start start 2023-10-08T01:12:35.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12b59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12406f5a0 0x7fe12414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12b59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe124159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12b59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12416b4b0 0x7fe1240c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12b59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe12414e590 con 0x7fe1240cb1a0 2023-10-08T01:12:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12b59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1241556d0 con 0x7fe12416b4b0 2023-10-08T01:12:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12b59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe124154ac0 con 0x7fe12406f5a0 2023-10-08T01:12:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe129d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe124159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe129d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe124159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36892/0 (socket says 172.21.15.119:36892) 2023-10-08T01:12:35.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe129d9a700 1 -- 172.21.15.119:0/1125978254 learned_addr learned my addr 172.21.15.119:0/1125978254 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:35.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12a59b700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12406f5a0 0x7fe12414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe129d9a700 1 -- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12406f5a0 msgr2=0x7fe12414e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe129d9a700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12406f5a0 0x7fe12414e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe129d9a700 1 -- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12416b4b0 msgr2=0x7fe1240c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12ad9c700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12416b4b0 0x7fe1240c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe129d9a700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12416b4b0 0x7fe1240c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe129d9a700 1 -- 172.21.15.119:0/1125978254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1200094f0 con 0x7fe1240cb1a0 2023-10-08T01:12:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12a59b700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12406f5a0 0x7fe12414e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.208+0000 7fe12ad9c700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12416b4b0 0x7fe1240c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.209+0000 7fe129d9a700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe124159720 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe11400d860 tx=0x7fe114008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.209+0000 7fe11a7fc700 1 -- 172.21.15.119:0/1125978254 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114009610 con 0x7fe1240cb1a0 2023-10-08T01:12:35.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.209+0000 7fe11a7fc700 1 -- 172.21.15.119:0/1125978254 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe114009770 con 0x7fe1240cb1a0 2023-10-08T01:12:35.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.209+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe124154da0 con 0x7fe1240cb1a0 2023-10-08T01:12:35.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.209+0000 7fe11a7fc700 1 -- 172.21.15.119:0/1125978254 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe114009b30 con 0x7fe1240cb1a0 2023-10-08T01:12:35.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.209+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe124155360 con 0x7fe1240cb1a0 2023-10-08T01:12:35.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.210+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe12406c450 con 0x7fe1240cb1a0 2023-10-08T01:12:35.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.211+0000 7fe11a7fc700 1 -- 172.21.15.119:0/1125978254 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe11401e460 con 0x7fe1240cb1a0 2023-10-08T01:12:35.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.212+0000 7fe11a7fc700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe108066f60 0x7fe108069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.212+0000 7fe11a7fc700 1 -- 172.21.15.119:0/1125978254 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fe114084ce0 con 0x7fe1240cb1a0 2023-10-08T01:12:35.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.212+0000 7fe12a59b700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe108066f60 0x7fe108069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.213+0000 7fe12a59b700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe108066f60 0x7fe108069410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe12406d580 tx=0x7fe120017000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.216+0000 7fe11a7fc700 1 -- 172.21.15.119:0/1125978254 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe114055880 con 0x7fe1240cb1a0 2023-10-08T01:12:35.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.407+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fe124155810 con 0x7fe1240cb1a0 2023-10-08T01:12:35.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.411+0000 7fe11a7fc700 1 -- 172.21.15.119:0/1125978254 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v32) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe114009ea0 con 0x7fe1240cb1a0 2023-10-08T01:12:35.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe108066f60 msgr2=0x7fe108069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe108066f60 0x7fe108069410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe12406d580 tx=0x7fe120017000 comp rx=0 tx=0).stop 2023-10-08T01:12:35.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 msgr2=0x7fe124159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe124159720 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe11400d860 tx=0x7fe114008040 comp rx=0 tx=0).stop 2023-10-08T01:12:35.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 shutdown_connections 2023-10-08T01:12:35.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe108066f60 0x7fe108069410 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe12406f5a0 0x7fe12414e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe12416b4b0 0x7fe1240c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 --2- 172.21.15.119:0/1125978254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1240cb1a0 0x7fe124159720 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 >> 172.21.15.119:0/1125978254 conn(0x7fe1240cc1e0 msgr2=0x7fe1240c2140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:35.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 shutdown_connections 2023-10-08T01:12:35.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.414+0000 7fe12b59d700 1 -- 172.21.15.119:0/1125978254 wait complete. 2023-10-08T01:12:35.433 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T01:12:35.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.614+0000 7fa49bfff700 1 Processor -- start 2023-10-08T01:12:35.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.614+0000 7fa49bfff700 1 -- start start 2023-10-08T01:12:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa49c1407e0 0x7fa49c06a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49c06dc00 0x7fa49c06a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49bfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49c0c6820 con 0x7fa49c140bb0 2023-10-08T01:12:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49bfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa49c0c6de0 con 0x7fa49c1407e0 2023-10-08T01:12:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49bfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c141270 con 0x7fa49c06dc00 2023-10-08T01:12:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49a7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa49c140bb0 0x7fa49c06d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57742/0 (socket says 172.21.15.119:57742) 2023-10-08T01:12:35.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49a7fc700 1 -- 172.21.15.119:0/2420224943 learned_addr learned my addr 172.21.15.119:0/2420224943 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49affd700 1 --2- 172.21.15.119:0/2420224943 >> v2:172.21.15.184:3300/0 conn(0x7fa49c1407e0 0x7fa49c06a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834868709 0 0) 0x7fa49c0c6820 con 0x7fa49c140bb0 2023-10-08T01:12:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa49b7fe700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49c06dc00 0x7fa49c06a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.616+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa484003570 con 0x7fa49c140bb0 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1626133837 0 0) 0x7fa484003570 con 0x7fa49c140bb0 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49c06dc00 msgr2=0x7fa49c06a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49c06dc00 0x7fa49c06a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 >> v2:172.21.15.184:3300/0 conn(0x7fa49c1407e0 msgr2=0x7fa49c06a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 --2- 172.21.15.119:0/2420224943 >> v2:172.21.15.184:3300/0 conn(0x7fa49c1407e0 0x7fa49c06a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c06acf0 con 0x7fa49c140bb0 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fa490001470 con 0x7fa49c140bb0 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 >> v1:172.21.15.119:6789/0 conn(0x7fa49c140bb0 legacy=0x7fa49c06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa484003ae0 0x7fa484003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa484004810 0x7fa484004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49c06dc00 0x7fa4840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0c6de0 con 0x7fa484004810 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c141270 con 0x7fa484003ae0 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48400bb10 con 0x7fa49c06dc00 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa49a7fc700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa484004810 0x7fa484004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa49affd700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa484003ae0 0x7fa484003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.617+0000 7fa49b7fe700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49c06dc00 0x7fa4840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa49a7fc700 1 -- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49c06dc00 msgr2=0x7fa4840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa49a7fc700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa49c06dc00 0x7fa4840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa49a7fc700 1 -- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa484003ae0 msgr2=0x7fa484003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa49a7fc700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa484003ae0 0x7fa484003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa49a7fc700 1 -- 172.21.15.119:0/2420224943 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484003570 con 0x7fa484004810 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa49a7fc700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa484004810 0x7fa484004c80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa49000c2d0 tx=0x7fa490001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dd2881c5abbbf4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49000cba0 con 0x7fa484004810 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49000cd20 con 0x7fa484004810 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa499ffb700 1 -- 172.21.15.119:0/2420224943 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490005b70 con 0x7fa484004810 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa49bfff700 1 -- 172.21.15.119:0/2420224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa484004810 msgr2=0x7fa484004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.618+0000 7fa49bfff700 1 --2- 172.21.15.119:0/2420224943 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa484004810 0x7fa484004c80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fa49000c2d0 tx=0x7fa490001dc0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.619+0000 7fa49b7fe700 1 -- 172.21.15.119:0/2420224943 reap_dead start 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.619+0000 7fa49bfff700 1 -- 172.21.15.119:0/2420224943 shutdown_connections 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.619+0000 7fa49bfff700 1 -- 172.21.15.119:0/2420224943 >> 172.21.15.119:0/2420224943 conn(0x7fa49c0cc1e0 msgr2=0x7fa49c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.619+0000 7fa49bfff700 1 -- 172.21.15.119:0/2420224943 shutdown_connections 2023-10-08T01:12:35.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.619+0000 7fa49bfff700 1 -- 172.21.15.119:0/2420224943 wait complete. 2023-10-08T01:12:35.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.620+0000 7fa49bfff700 1 Processor -- start 2023-10-08T01:12:35.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.620+0000 7fa49bfff700 1 -- start start 2023-10-08T01:12:35.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa484004810 0x7fa49c157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49c1583d0 0x7fa49c0c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c0c55d0 0x7fa49c0c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49bfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c158ea0 con 0x7fa49c0c55d0 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49bfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c0ca4a0 con 0x7fa49c1583d0 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49bfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49c158bc0 con 0x7fa484004810 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c0c55d0 0x7fa49c0c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c0c55d0 0x7fa49c0c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36906/0 (socket says 172.21.15.119:36906) 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49b7fe700 1 -- 172.21.15.119:0/2709042935 learned_addr learned my addr 172.21.15.119:0/2709042935 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49a7fc700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49c1583d0 0x7fa49c0c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.621+0000 7fa49affd700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa484004810 0x7fa49c157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa49b7fe700 1 -- 172.21.15.119:0/2709042935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa484004810 msgr2=0x7fa49c157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa49b7fe700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa484004810 0x7fa49c157e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa49b7fe700 1 -- 172.21.15.119:0/2709042935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49c1583d0 msgr2=0x7fa49c0c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa49b7fe700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49c1583d0 0x7fa49c0c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa49b7fe700 1 -- 172.21.15.119:0/2709042935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49c0c9e50 con 0x7fa49c0c55d0 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa49b7fe700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c0c55d0 0x7fa49c0c5a20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa4940094d0 tx=0x7fa49400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa482ffd700 1 -- 172.21.15.119:0/2709042935 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400e040 con 0x7fa49c0c55d0 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa482ffd700 1 -- 172.21.15.119:0/2709042935 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49400f040 con 0x7fa49c0c55d0 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa482ffd700 1 -- 172.21.15.119:0/2709042935 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa49400fe20 con 0x7fa49c0c55d0 2023-10-08T01:12:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49c0ca130 con 0x7fa49c0c55d0 2023-10-08T01:12:35.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.622+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49c158840 con 0x7fa49c0c55d0 2023-10-08T01:12:35.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.623+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49c06c450 con 0x7fa49c0c55d0 2023-10-08T01:12:35.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.624+0000 7fa482ffd700 1 -- 172.21.15.119:0/2709042935 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa494015640 con 0x7fa49c0c55d0 2023-10-08T01:12:35.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.625+0000 7fa482ffd700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa488066f30 0x7fa4880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:35.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.625+0000 7fa482ffd700 1 -- 172.21.15.119:0/2709042935 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5890+0+0 (secure 0 0 0) 0x7fa494084e40 con 0x7fa49c0c55d0 2023-10-08T01:12:35.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.626+0000 7fa49affd700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa488066f30 0x7fa4880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:35.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.626+0000 7fa49affd700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa488066f30 0x7fa4880693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa48c006f00 tx=0x7fa48c008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:35.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.629+0000 7fa482ffd700 1 -- 172.21.15.119:0/2709042935 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa494051ff0 con 0x7fa49c0c55d0 2023-10-08T01:12:35.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.824+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fa49c0ca2c0 con 0x7fa49c0c55d0 2023-10-08T01:12:35.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.828+0000 7fa482ffd700 1 -- 172.21.15.119:0/2709042935 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v33) v1 ==== 120+0+0 (secure 0 0 0) 0x7fa49c0ca2c0 con 0x7fa49c0c55d0 2023-10-08T01:12:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa488066f30 msgr2=0x7fa4880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa488066f30 0x7fa4880693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa48c006f00 tx=0x7fa48c008040 comp rx=0 tx=0).stop 2023-10-08T01:12:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c0c55d0 msgr2=0x7fa49c0c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c0c55d0 0x7fa49c0c5a20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fa4940094d0 tx=0x7fa49400b040 comp rx=0 tx=0).stop 2023-10-08T01:12:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 shutdown_connections 2023-10-08T01:12:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa488066f30 0x7fa4880693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa484004810 0x7fa49c157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49c0c55d0 0x7fa49c0c5a20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 --2- 172.21.15.119:0/2709042935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa49c1583d0 0x7fa49c0c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 >> 172.21.15.119:0/2709042935 conn(0x7fa49c0cc1e0 msgr2=0x7fa49c13bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 shutdown_connections 2023-10-08T01:12:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:35.831+0000 7fa49bfff700 1 -- 172.21.15.119:0/2709042935 wait complete. 2023-10-08T01:12:35.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-10-08T01:12:36.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.030+0000 7f175f9ff700 1 Processor -- start 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.031+0000 7f175f9ff700 1 -- start start 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175f9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 0x7f17580b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175f9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f17581579f0 0x7f17580abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175f9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17580cb300 con 0x7f17580ae680 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175f9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175806fc30 con 0x7f17581579f0 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175f9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580cc260 con 0x7f17580b1850 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175cf9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f17580ae680 0x7f17581572e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57748/0 (socket says 172.21.15.119:57748) 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175cf9a700 1 -- 172.21.15.119:0/1343263974 learned_addr learned my addr 172.21.15.119:0/1343263974 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175df9c700 1 --2- 172.21.15.119:0/1343263974 >> v2:172.21.15.119:3300/0 conn(0x7f17581579f0 0x7f17580abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.032+0000 7f175d79b700 1 --2- 172.21.15.119:0/1343263974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 0x7f17580b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f174ffff700 1 -- 172.21.15.119:0/1343263974 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830206544 0 0) 0x7f17580cb300 con 0x7f17580ae680 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f174ffff700 1 -- 172.21.15.119:0/1343263974 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1744003570 con 0x7f17580ae680 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f175df9c700 1 -- 172.21.15.119:0/1343263974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 msgr2=0x7f17580b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f175df9c700 1 --2- 172.21.15.119:0/1343263974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 0x7f17580b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f175df9c700 1 -- 172.21.15.119:0/1343263974 >> v1:172.21.15.119:6789/0 conn(0x7f17580ae680 legacy=0x7f17581572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f175df9c700 1 -- 172.21.15.119:0/1343263974 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580cb5e0 con 0x7f17581579f0 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f175df9c700 1 --2- 172.21.15.119:0/1343263974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17581579f0 0x7f17580abf60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e60 tx=0x7f1750009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e664abe21d38497 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f174ffff700 1 -- 172.21.15.119:0/1343263974 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000e070 con 0x7f17581579f0 2023-10-08T01:12:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f174ffff700 1 -- 172.21.15.119:0/1343263974 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175000bab0 con 0x7f17581579f0 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.033+0000 7f174ffff700 1 -- 172.21.15.119:0/1343263974 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175000be70 con 0x7f17581579f0 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.034+0000 7f175f9ff700 1 -- 172.21.15.119:0/1343263974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17581579f0 msgr2=0x7f17580abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.034+0000 7f175f9ff700 1 --2- 172.21.15.119:0/1343263974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17581579f0 0x7f17580abf60 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1750009e60 tx=0x7f1750009780 comp rx=0 tx=0).stop 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.034+0000 7f175f9ff700 1 -- 172.21.15.119:0/1343263974 shutdown_connections 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.034+0000 7f175f9ff700 1 --2- 172.21.15.119:0/1343263974 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17581579f0 0x7f17580abf60 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.034+0000 7f175f9ff700 1 --2- 172.21.15.119:0/1343263974 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 0x7f17580b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.034+0000 7f175f9ff700 1 -- 172.21.15.119:0/1343263974 >> 172.21.15.119:0/1343263974 conn(0x7f175806a320 msgr2=0x7f17580cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.034+0000 7f175f9ff700 1 -- 172.21.15.119:0/1343263974 shutdown_connections 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.034+0000 7f175f9ff700 1 -- 172.21.15.119:0/1343263974 wait complete. 2023-10-08T01:12:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.035+0000 7f175f9ff700 1 Processor -- start 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.036+0000 7f175f9ff700 1 -- start start 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.036+0000 7f175f9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17580ae680 0x7f1758147ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.036+0000 7f175f9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 0x7f175815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.036+0000 7f175f9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17581579f0 0x7f175815cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.036+0000 7f175f9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1758157f00 con 0x7f17580ae680 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.036+0000 7f175f9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f175815b2e0 con 0x7f17581579f0 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.036+0000 7f175f9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f175815b5c0 con 0x7f17580b1850 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17580ae680 0x7f1758147ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17580ae680 0x7f1758147ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36926/0 (socket says 172.21.15.119:36926) 2023-10-08T01:12:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 -- 172.21.15.119:0/2242792301 learned_addr learned my addr 172.21.15.119:0/2242792301 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175cf9a700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 0x7f175815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175df9c700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17581579f0 0x7f175815cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 -- 172.21.15.119:0/2242792301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 msgr2=0x7f175815c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 0x7f175815c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 -- 172.21.15.119:0/2242792301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17581579f0 msgr2=0x7f175815cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17581579f0 0x7f175815cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 -- 172.21.15.119:0/2242792301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1750009a20 con 0x7f17580ae680 2023-10-08T01:12:36.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.037+0000 7f175d79b700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17580ae680 0x7f1758147ef0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f175400ec80 tx=0x7f17540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.038+0000 7f174d7fa700 1 -- 172.21.15.119:0/2242792301 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f175400c9d0 con 0x7f17580ae680 2023-10-08T01:12:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.038+0000 7f174d7fa700 1 -- 172.21.15.119:0/2242792301 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f175400cb30 con 0x7f17580ae680 2023-10-08T01:12:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.038+0000 7f174d7fa700 1 -- 172.21.15.119:0/2242792301 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1754013470 con 0x7f17580ae680 2023-10-08T01:12:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.038+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175815b8a0 con 0x7f17580ae680 2023-10-08T01:12:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.038+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1758148430 con 0x7f17580ae680 2023-10-08T01:12:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.039+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f175804c3c0 con 0x7f17580ae680 2023-10-08T01:12:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.040+0000 7f174d7fa700 1 -- 172.21.15.119:0/2242792301 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1754013820 con 0x7f17580ae680 2023-10-08T01:12:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.041+0000 7f174d7fa700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f173c066f30 0x7f173c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.041+0000 7f174d7fa700 1 -- 172.21.15.119:0/2242792301 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f1754084960 con 0x7f17580ae680 2023-10-08T01:12:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.041+0000 7f175cf9a700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f173c066f30 0x7f173c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:36.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.042+0000 7f175cf9a700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f173c066f30 0x7f173c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f175815c590 tx=0x7f1748009040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:36.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.045+0000 7f174d7fa700 1 -- 172.21.15.119:0/2242792301 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1754051b10 con 0x7f17580ae680 2023-10-08T01:12:36.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:36.311+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f17580cb8c0 con 0x7f17580ae680 2023-10-08T01:12:37.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.190+0000 7f174d7fa700 1 -- 172.21.15.119:0/2242792301 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v625) v1 ==== 166+0+0 (secure 0 0 0) 0x7f1754055130 con 0x7f17580ae680 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f173c066f30 msgr2=0x7f173c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f173c066f30 0x7f173c0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f175815c590 tx=0x7f1748009040 comp rx=0 tx=0).stop 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17580ae680 msgr2=0x7f1758147ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17580ae680 0x7f1758147ef0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f175400ec80 tx=0x7f17540091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 shutdown_connections 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f173c066f30 0x7f173c0693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f17581579f0 0x7f175815cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f17580ae680 0x7f1758147ef0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 --2- 172.21.15.119:0/2242792301 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f17580b1850 0x7f175815c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.193+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 >> 172.21.15.119:0/2242792301 conn(0x7f175806a320 msgr2=0x7f17580ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:37.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.194+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 shutdown_connections 2023-10-08T01:12:37.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.194+0000 7f175f9ff700 1 -- 172.21.15.119:0/2242792301 wait complete. 2023-10-08T01:12:37.215 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-10-08T01:12:37.215 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-10-08T01:12:37.215 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T01:12:37.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-08T01:12:37.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.397+0000 7f8bc939d700 1 Processor -- start 2023-10-08T01:12:37.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.397+0000 7f8bc939d700 1 -- start start 2023-10-08T01:12:37.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.398+0000 7f8bc939d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc4150180 0x7f8bc4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.398+0000 7f8bc939d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bc4150550 0x7f8bc40c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.398+0000 7f8bc939d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8bc40c7240 0x7f8bc4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc939d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40c76e0 con 0x7f8bc4150550 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc939d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc406e2a0 con 0x7f8bc40c7240 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc939d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc406b2e0 con 0x7f8bc4150180 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc37fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bc4150550 0x7f8bc40c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc37fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bc4150550 0x7f8bc40c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36938/0 (socket says 172.21.15.119:36938) 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc37fe700 1 -- 172.21.15.119:0/3628135498 learned_addr learned my addr 172.21.15.119:0/3628135498 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc8b9c700 1 --2- 172.21.15.119:0/3628135498 >> v2:172.21.15.184:3300/0 conn(0x7f8bc40c7240 0x7f8bc4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc3fff700 1 --2- 172.21.15.119:0/3628135498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc4150180 0x7f8bc4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:37.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc37fe700 1 -- 172.21.15.119:0/3628135498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc4150180 msgr2=0x7f8bc4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc37fe700 1 --2- 172.21.15.119:0/3628135498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc4150180 0x7f8bc4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc37fe700 1 -- 172.21.15.119:0/3628135498 >> v2:172.21.15.184:3300/0 conn(0x7f8bc40c7240 msgr2=0x7f8bc4069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc37fe700 1 --2- 172.21.15.119:0/3628135498 >> v2:172.21.15.184:3300/0 conn(0x7f8bc40c7240 0x7f8bc4069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc37fe700 1 -- 172.21.15.119:0/3628135498 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40c78e0 con 0x7f8bc4150550 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.399+0000 7f8bc8b9c700 1 --2- 172.21.15.119:0/3628135498 >> v2:172.21.15.184:3300/0 conn(0x7f8bc40c7240 0x7f8bc4069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc37fe700 1 --2- 172.21.15.119:0/3628135498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6d00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406ad70 tx=0x7f8bb8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f261df6fa5a46f7d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc2ffd700 1 -- 172.21.15.119:0/3628135498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800b820 con 0x7f8bc4150550 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc2ffd700 1 -- 172.21.15.119:0/3628135498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb800b980 con 0x7f8bc4150550 2023-10-08T01:12:37.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc2ffd700 1 -- 172.21.15.119:0/3628135498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800bd20 con 0x7f8bc4150550 2023-10-08T01:12:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc939d700 1 -- 172.21.15.119:0/3628135498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 msgr2=0x7f8bc40c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:37.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc939d700 1 --2- 172.21.15.119:0/3628135498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6d00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406ad70 tx=0x7f8bb8009250 comp rx=0 tx=0).stop 2023-10-08T01:12:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc939d700 1 -- 172.21.15.119:0/3628135498 shutdown_connections 2023-10-08T01:12:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc939d700 1 --2- 172.21.15.119:0/3628135498 >> v2:172.21.15.184:3300/0 conn(0x7f8bc40c7240 0x7f8bc4069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc939d700 1 --2- 172.21.15.119:0/3628135498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc4150180 0x7f8bc4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc939d700 1 --2- 172.21.15.119:0/3628135498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6d00 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.400+0000 7f8bc939d700 1 -- 172.21.15.119:0/3628135498 >> 172.21.15.119:0/3628135498 conn(0x7f8bc40cc1f0 msgr2=0x7f8bc40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.401+0000 7f8bc939d700 1 -- 172.21.15.119:0/3628135498 shutdown_connections 2023-10-08T01:12:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.401+0000 7f8bc939d700 1 -- 172.21.15.119:0/3628135498 wait complete. 2023-10-08T01:12:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.402+0000 7f8bc939d700 1 Processor -- start 2023-10-08T01:12:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.402+0000 7f8bc939d700 1 -- start start 2023-10-08T01:12:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc939d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc40c7240 0x7f8bc41446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:37.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc939d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bc4150180 0x7f8bc40c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc939d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc939d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40afe20 con 0x7f8bc4150550 2023-10-08T01:12:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc939d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40aef90 con 0x7f8bc4150180 2023-10-08T01:12:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc939d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41450c0 con 0x7f8bc40c7240 2023-10-08T01:12:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc8b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc8b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36952/0 (socket says 172.21.15.119:36952) 2023-10-08T01:12:37.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc8b9c700 1 -- 172.21.15.119:0/1956585332 learned_addr learned my addr 172.21.15.119:0/1956585332 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc3fff700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc40c7240 0x7f8bc41446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.403+0000 7f8bc37fe700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bc4150180 0x7f8bc40c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc8b9c700 1 -- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc40c7240 msgr2=0x7f8bc41446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc8b9c700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc40c7240 0x7f8bc41446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc8b9c700 1 -- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bc4150180 msgr2=0x7f8bc40c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc8b9c700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bc4150180 0x7f8bc40c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc8b9c700 1 -- 172.21.15.119:0/1956585332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80094f0 con 0x7f8bc4150550 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc37fe700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bc4150180 0x7f8bc40c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc8b9c700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6410 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009fc0 tx=0x7f8bbc0099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8babfff700 1 -- 172.21.15.119:0/1956585332 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00dc90 con 0x7f8bc4150550 2023-10-08T01:12:37.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8babfff700 1 -- 172.21.15.119:0/1956585332 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc00f040 con 0x7f8bc4150550 2023-10-08T01:12:37.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc40af1a0 con 0x7f8bc4150550 2023-10-08T01:12:37.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8babfff700 1 -- 172.21.15.119:0/1956585332 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc015690 con 0x7f8bc4150550 2023-10-08T01:12:37.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.404+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc40af760 con 0x7f8bc4150550 2023-10-08T01:12:37.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.405+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc40aff60 con 0x7f8bc4150550 2023-10-08T01:12:37.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.406+0000 7f8babfff700 1 -- 172.21.15.119:0/1956585332 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8bbc01e3b0 con 0x7f8bc4150550 2023-10-08T01:12:37.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.407+0000 7f8babfff700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0066f30 0x7f8bb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:37.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.407+0000 7f8babfff700 1 -- 172.21.15.119:0/1956585332 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f8bbc084b40 con 0x7f8bc4150550 2023-10-08T01:12:37.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.407+0000 7f8bc3fff700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0066f30 0x7f8bb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:37.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.408+0000 7f8bc3fff700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0066f30 0x7f8bb00693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4006fd0 tx=0x7f8bb4008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:37.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.411+0000 7f8babfff700 1 -- 172.21.15.119:0/1956585332 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bbc051c50 con 0x7f8bc4150550 2023-10-08T01:12:37.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:37.681+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f8bc404c3c0 con 0x7f8bc4150550 2023-10-08T01:12:38.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.201+0000 7f8babfff700 1 -- 172.21.15.119:0/1956585332 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v626) v1 ==== 186+0+0 (secure 0 0 0) 0x7f8bbc01a100 con 0x7f8bc4150550 2023-10-08T01:12:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0066f30 msgr2=0x7f8bb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0066f30 0x7f8bb00693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8bb4006fd0 tx=0x7f8bb4008040 comp rx=0 tx=0).stop 2023-10-08T01:12:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 msgr2=0x7f8bc40c6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6410 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009fc0 tx=0x7f8bbc0099b0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 shutdown_connections 2023-10-08T01:12:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0066f30 0x7f8bb00693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bc40c7240 0x7f8bc41446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bc4150550 0x7f8bc40c6410 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 --2- 172.21.15.119:0/1956585332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bc4150180 0x7f8bc40c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.203+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 >> 172.21.15.119:0/1956585332 conn(0x7f8bc40cc1f0 msgr2=0x7f8bc40c22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.204+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 shutdown_connections 2023-10-08T01:12:38.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.204+0000 7f8bc939d700 1 -- 172.21.15.119:0/1956585332 wait complete. 2023-10-08T01:12:38.223 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T01:12:38.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.406+0000 7f72ccfff700 1 Processor -- start 2023-10-08T01:12:38.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.406+0000 7f72ccfff700 1 -- start start 2023-10-08T01:12:38.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.407+0000 7f72ccfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f72c815ac80 0x7f72c8069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.408+0000 7f72ccfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c80af2f0 con 0x7f72c815b050 2023-10-08T01:12:38.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.408+0000 7f72ccfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c80af5d0 con 0x7f72c8069cb0 2023-10-08T01:12:38.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.408+0000 7f72ccfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c806a0e0 con 0x7f72c815ac80 2023-10-08T01:12:38.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.408+0000 7f72c5d9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f72c815b050 0x7f72c80afa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57752/0 (socket says 172.21.15.119:57752) 2023-10-08T01:12:38.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.408+0000 7f72c5d9b700 1 -- 172.21.15.119:0/1338304546 learned_addr learned my addr 172.21.15.119:0/1338304546 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:38.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.408+0000 7f72c659c700 1 --2- 172.21.15.119:0/1338304546 >> v2:172.21.15.184:3300/0 conn(0x7f72c815ac80 0x7f72c8069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:38.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.408+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2978369821 0 0) 0x7f72c80af2f0 con 0x7f72c815b050 2023-10-08T01:12:38.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.408+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72b0003570 con 0x7f72c815b050 2023-10-08T01:12:38.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 779908189 0 0) 0x7f72c80af5d0 con 0x7f72c8069cb0 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72c80af2f0 con 0x7f72c8069cb0 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 222285199 0 0) 0x7f72b0003570 con 0x7f72c815b050 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 >> v2:172.21.15.184:3300/0 conn(0x7f72c815ac80 msgr2=0x7f72c8069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 --2- 172.21.15.119:0/1338304546 >> v2:172.21.15.184:3300/0 conn(0x7f72c815ac80 0x7f72c8069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 >> v1:172.21.15.184:6789/0 conn(0x7f72c8069cb0 legacy=0x7f72c80c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c80c8660 con 0x7f72c815b050 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f72bc001470 con 0x7f72c815b050 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 >> v1:172.21.15.119:6789/0 conn(0x7f72c815b050 legacy=0x7f72c80afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72b0004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0004810 0x7f72b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b0005210 0x7f72b000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c806a0e0 con 0x7f72b0003c40 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b000c1b0 con 0x7f72b0004810 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b000c370 con 0x7f72b0005210 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c659c700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72b0004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c5d9b700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0004810 0x7f72b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c6d9d700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b0005210 0x7f72b000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c659c700 1 -- 172.21.15.119:0/1338304546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b0005210 msgr2=0x7f72b000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c659c700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72b0005210 0x7f72b000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c659c700 1 -- 172.21.15.119:0/1338304546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0004810 msgr2=0x7f72b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.409+0000 7f72c659c700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72b0004810 0x7f72b0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.410+0000 7f72c659c700 1 -- 172.21.15.119:0/1338304546 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b0003570 con 0x7f72b0003c40 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.410+0000 7f72c6d9d700 1 -- 172.21.15.119:0/1338304546 reap_dead start 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.410+0000 7f72c659c700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72b0004030 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f72b8006f00 tx=0x7f72b80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a30f0a0fd49443d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.410+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800e710 con 0x7f72b0003c40 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.410+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800e870 con 0x7f72b0003c40 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.410+0000 7f72c559a700 1 -- 172.21.15.119:0/1338304546 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b800ec30 con 0x7f72b0003c40 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.410+0000 7f72ccfff700 1 -- 172.21.15.119:0/1338304546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 msgr2=0x7f72b0004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.410+0000 7f72ccfff700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72b0004030 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f72b8006f00 tx=0x7f72b80091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.411+0000 7f72ccfff700 1 -- 172.21.15.119:0/1338304546 shutdown_connections 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.411+0000 7f72ccfff700 1 --2- 172.21.15.119:0/1338304546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72b0004030 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.411+0000 7f72ccfff700 1 -- 172.21.15.119:0/1338304546 >> 172.21.15.119:0/1338304546 conn(0x7f72c8068e10 msgr2=0x7f72c80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:38.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.411+0000 7f72ccfff700 1 -- 172.21.15.119:0/1338304546 shutdown_connections 2023-10-08T01:12:38.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.411+0000 7f72ccfff700 1 -- 172.21.15.119:0/1338304546 wait complete. 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.412+0000 7f72ccfff700 1 Processor -- start 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.412+0000 7f72ccfff700 1 -- start start 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72ccfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72c80ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72ccfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72c80ac900 0x7f72c80b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72ccfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c814cea0 0x7f72c814d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72ccfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80cbd50 con 0x7f72b0003c40 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72ccfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80cb8c0 con 0x7f72c814cea0 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72ccfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c814d830 con 0x7f72c80ac900 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72c659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72c80ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:38.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72c659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72c80ca2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36970/0 (socket says 172.21.15.119:36970) 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.413+0000 7f72c659c700 1 -- 172.21.15.119:0/3700184925 learned_addr learned my addr 172.21.15.119:0/3700184925 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72c659c700 1 -- 172.21.15.119:0/3700184925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72c80ac900 msgr2=0x7f72c80b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72c659c700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72c80ac900 0x7f72c80b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72c659c700 1 -- 172.21.15.119:0/3700184925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c814cea0 msgr2=0x7f72c814d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72c659c700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c814cea0 0x7f72c814d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72c659c700 1 -- 172.21.15.119:0/3700184925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c80cacb0 con 0x7f72b0003c40 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72c659c700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72c80ca2d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c180 tx=0x7f72b801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72b67fc700 1 -- 172.21.15.119:0/3700184925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8022070 con 0x7f72b0003c40 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72b67fc700 1 -- 172.21.15.119:0/3700184925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b800e9e0 con 0x7f72b0003c40 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72b67fc700 1 -- 172.21.15.119:0/3700184925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b8027470 con 0x7f72b0003c40 2023-10-08T01:12:38.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c80caf90 con 0x7f72b0003c40 2023-10-08T01:12:38.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.414+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c80cb550 con 0x7f72b0003c40 2023-10-08T01:12:38.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.415+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c804c3c0 con 0x7f72b0003c40 2023-10-08T01:12:38.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.416+0000 7f72b67fc700 1 -- 172.21.15.119:0/3700184925 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72b801c450 con 0x7f72b0003c40 2023-10-08T01:12:38.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.417+0000 7f72b67fc700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72a8066f30 0x7f72a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.417+0000 7f72b67fc700 1 -- 172.21.15.119:0/3700184925 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f72b80862c0 con 0x7f72b0003c40 2023-10-08T01:12:38.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.417+0000 7f72c5d9b700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72a8066f30 0x7f72a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:38.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.418+0000 7f72c5d9b700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72a8066f30 0x7f72a80693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f72c80b2030 tx=0x7f72bc001d40 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:38.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.421+0000 7f72b67fc700 1 -- 172.21.15.119:0/3700184925 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72b80533d0 con 0x7f72b0003c40 2023-10-08T01:12:38.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.612+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f72c80c8940 con 0x7f72b0003c40 2023-10-08T01:12:38.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.616+0000 7f72b67fc700 1 -- 172.21.15.119:0/3700184925 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72b80569f0 con 0x7f72b0003c40 2023-10-08T01:12:38.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72a8066f30 msgr2=0x7f72a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72a8066f30 0x7f72a80693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f72c80b2030 tx=0x7f72bc001d40 comp rx=0 tx=0).stop 2023-10-08T01:12:38.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 msgr2=0x7f72c80ca2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72c80ca2d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f72b000c180 tx=0x7f72b801a000 comp rx=0 tx=0).stop 2023-10-08T01:12:38.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 shutdown_connections 2023-10-08T01:12:38.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72a8066f30 0x7f72a80693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72c814cea0 0x7f72c814d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72b0003c40 0x7f72c80ca2d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 --2- 172.21.15.119:0/3700184925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72c80ac900 0x7f72c80b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 >> 172.21.15.119:0/3700184925 conn(0x7f72c8068e10 msgr2=0x7f72c80c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:38.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 shutdown_connections 2023-10-08T01:12:38.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.619+0000 7f72ccfff700 1 -- 172.21.15.119:0/3700184925 wait complete. 2023-10-08T01:12:38.637 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:12:38.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:12:38.719 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:12:38.815 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:12:38.815 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T01:12:38.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:38.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.994+0000 7feb2abff700 1 Processor -- start 2023-10-08T01:12:38.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.995+0000 7feb2abff700 1 -- start start 2023-10-08T01:12:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb2abff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feb24155c10 0x7feb24155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb2abff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb240ac230 0x7feb240ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb2abff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb240ad930 con 0x7feb241568a0 2023-10-08T01:12:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb2abff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb240acd20 con 0x7feb240ac230 2023-10-08T01:12:38.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb2abff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb240ad000 con 0x7feb24155c10 2023-10-08T01:12:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb23fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb240ac230 0x7feb240ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb23fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb240ac230 0x7feb240ac6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36984/0 (socket says 172.21.15.119:36984) 2023-10-08T01:12:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb23fff700 1 -- 172.21.15.119:0/1224746873 learned_addr learned my addr 172.21.15.119:0/1224746873 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.996+0000 7feb2899b700 1 --2- 172.21.15.119:0/1224746873 >> v2:172.21.15.184:3300/0 conn(0x7feb24155c10 0x7feb24155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.997+0000 7feb23fff700 1 -- 172.21.15.119:0/1224746873 >> v2:172.21.15.184:3300/0 conn(0x7feb24155c10 msgr2=0x7feb24155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.997+0000 7feb23fff700 1 --2- 172.21.15.119:0/1224746873 >> v2:172.21.15.184:3300/0 conn(0x7feb24155c10 0x7feb24155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.997+0000 7feb23fff700 1 -- 172.21.15.119:0/1224746873 >> v1:172.21.15.184:6789/0 conn(0x7feb241568a0 legacy=0x7feb24156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:38.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.997+0000 7feb23fff700 1 -- 172.21.15.119:0/1224746873 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb240ad2e0 con 0x7feb240ac230 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.997+0000 7feb23fff700 1 --2- 172.21.15.119:0/1224746873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb240ac230 0x7feb240ac6d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7feb24063d40 tx=0x7feb1000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d599718b43b1d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.997+0000 7feb237fe700 1 -- 172.21.15.119:0/1224746873 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1000dba0 con 0x7feb240ac230 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.997+0000 7feb237fe700 1 -- 172.21.15.119:0/1224746873 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1000dd00 con 0x7feb240ac230 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.997+0000 7feb237fe700 1 -- 172.21.15.119:0/1224746873 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb100196d0 con 0x7feb240ac230 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.998+0000 7feb2abff700 1 -- 172.21.15.119:0/1224746873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb240ac230 msgr2=0x7feb240ac6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.998+0000 7feb2abff700 1 --2- 172.21.15.119:0/1224746873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb240ac230 0x7feb240ac6d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7feb24063d40 tx=0x7feb1000b4f0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.998+0000 7feb2abff700 1 -- 172.21.15.119:0/1224746873 shutdown_connections 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.998+0000 7feb2abff700 1 --2- 172.21.15.119:0/1224746873 >> v2:172.21.15.184:3300/0 conn(0x7feb24155c10 0x7feb24155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.998+0000 7feb2abff700 1 --2- 172.21.15.119:0/1224746873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb240ac230 0x7feb240ac6d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.998+0000 7feb2abff700 1 -- 172.21.15.119:0/1224746873 >> 172.21.15.119:0/1224746873 conn(0x7feb2406a320 msgr2=0x7feb240cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:39.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.998+0000 7feb2abff700 1 -- 172.21.15.119:0/1224746873 shutdown_connections 2023-10-08T01:12:39.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.998+0000 7feb2abff700 1 -- 172.21.15.119:0/1224746873 wait complete. 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.999+0000 7feb2abff700 1 Processor -- start 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:38.999+0000 7feb2abff700 1 -- start start 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.000+0000 7feb2abff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb240ac230 0x7feb24147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.000+0000 7feb2abff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb24155c10 0x7feb24146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.000+0000 7feb2abff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb241568a0 0x7feb24146ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.000+0000 7feb2abff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240a9ff0 con 0x7feb24155c10 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.000+0000 7feb2abff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb241458e0 con 0x7feb240ac230 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.000+0000 7feb2abff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb24145bc0 con 0x7feb241568a0 2023-10-08T01:12:39.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.000+0000 7feb23fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb24155c10 0x7feb24146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb23fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb24155c10 0x7feb24146980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36986/0 (socket says 172.21.15.119:36986) 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb23fff700 1 -- 172.21.15.119:0/148850259 learned_addr learned my addr 172.21.15.119:0/148850259 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb2899b700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb240ac230 0x7feb24147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb2919c700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb241568a0 0x7feb24146ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb23fff700 1 -- 172.21.15.119:0/148850259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb241568a0 msgr2=0x7feb24146ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb23fff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb241568a0 0x7feb24146ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb23fff700 1 -- 172.21.15.119:0/148850259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb240ac230 msgr2=0x7feb24147860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb23fff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb240ac230 0x7feb24147860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb23fff700 1 -- 172.21.15.119:0/148850259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb1000b790 con 0x7feb24155c10 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb23fff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb24155c10 0x7feb24146980 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7feb1000b760 tx=0x7feb1001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:39.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb20ff9700 1 -- 172.21.15.119:0/148850259 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10010470 con 0x7feb24155c10 2023-10-08T01:12:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb20ff9700 1 -- 172.21.15.119:0/148850259 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb100105d0 con 0x7feb24155c10 2023-10-08T01:12:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb24145ea0 con 0x7feb24155c10 2023-10-08T01:12:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.001+0000 7feb20ff9700 1 -- 172.21.15.119:0/148850259 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10010990 con 0x7feb24155c10 2023-10-08T01:12:39.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.002+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb24146460 con 0x7feb24155c10 2023-10-08T01:12:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.002+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb24146650 con 0x7feb24155c10 2023-10-08T01:12:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.003+0000 7feb20ff9700 1 -- 172.21.15.119:0/148850259 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb10022030 con 0x7feb24155c10 2023-10-08T01:12:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.004+0000 7feb20ff9700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feb08066f30 0x7feb080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.004+0000 7feb20ff9700 1 -- 172.21.15.119:0/148850259 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5890+0+0 (secure 0 0 0) 0x7feb10014070 con 0x7feb24155c10 2023-10-08T01:12:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.004+0000 7feb2899b700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feb08066f30 0x7feb080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.005+0000 7feb2899b700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feb08066f30 0x7feb080693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb18009d40 tx=0x7feb18008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:39.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.008+0000 7feb20ff9700 1 -- 172.21.15.119:0/148850259 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb10055900 con 0x7feb24155c10 2023-10-08T01:12:39.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.245+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb2404c3c0 con 0x7feb24155c10 2023-10-08T01:12:39.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.246+0000 7feb20ff9700 1 -- 172.21.15.119:0/148850259 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3538 (secure 0 0 0) 0x7feb10058f20 con 0x7feb24155c10 2023-10-08T01:12:39.247 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:39.247 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6304,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3441253316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3441253316},{"type":"v1","addr":"172.21.15.184:6834","nonce":3441253316}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6628,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3903169231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3903169231},{"type":"v1","addr":"172.21.15.184:6835","nonce":3903169231}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/3613948578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3613948578},{"type":"v1","addr":"172.21.15.119:6833","nonce":3613948578}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6702,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/765135141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":765135141},{"type":"v1","addr":"172.21.15.119:6835","nonce":765135141}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35}],"filesystems":[]} 2023-10-08T01:12:39.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feb08066f30 msgr2=0x7feb080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feb08066f30 0x7feb080693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7feb18009d40 tx=0x7feb18008040 comp rx=0 tx=0).stop 2023-10-08T01:12:39.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb24155c10 msgr2=0x7feb24146980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb24155c10 0x7feb24146980 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7feb1000b760 tx=0x7feb1001b000 comp rx=0 tx=0).stop 2023-10-08T01:12:39.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 shutdown_connections 2023-10-08T01:12:39.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feb08066f30 0x7feb080693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feb241568a0 0x7feb24146ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feb240ac230 0x7feb24147860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 --2- 172.21.15.119:0/148850259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb24155c10 0x7feb24146980 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 >> 172.21.15.119:0/148850259 conn(0x7feb2406a320 msgr2=0x7feb240ab940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:39.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 shutdown_connections 2023-10-08T01:12:39.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.248+0000 7feb2abff700 1 -- 172.21.15.119:0/148850259 wait complete. 2023-10-08T01:12:39.252 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 41 2023-10-08T01:12:39.267 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T01:12:39.267 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T01:12:39.267 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T01:12:39.267 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:12:39.267 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T01:12:39.267 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T01:12:39.268 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T01:12:39.268 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:12:39.268 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T01:12:39.268 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T01:12:39.268 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T01:12:39.268 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:12:39.269 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T01:12:39.269 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T01:12:39.269 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T01:12:39.269 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:12:39.282 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T01:12:39.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T01:12:39.283 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T01:12:39.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T01:12:39.285 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T01:12:39.285 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T01:12:39.286 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T01:12:39.286 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T01:12:39.326 INFO:tasks.ceph.mds.c:Started 2023-10-08T01:12:39.326 DEBUG:teuthology.parallel:result is None 2023-10-08T01:12:39.328 INFO:tasks.ceph.mds.b:Started 2023-10-08T01:12:39.329 DEBUG:teuthology.parallel:result is None 2023-10-08T01:12:39.329 INFO:tasks.ceph.mds.a:Started 2023-10-08T01:12:39.329 DEBUG:teuthology.parallel:result is None 2023-10-08T01:12:39.331 INFO:tasks.ceph.mds.d:Started 2023-10-08T01:12:39.331 DEBUG:teuthology.parallel:result is None 2023-10-08T01:12:39.332 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T01:12:39.460 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T01:12:39.460 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:12:39.467 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T01:12:39.467 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T01:12:39.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.522+0000 7f3ba11ff700 1 Processor -- start 2023-10-08T01:12:39.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.522+0000 7f3ba11ff700 1 -- start start 2023-10-08T01:12:39.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.523+0000 7f3ba11ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3b9c159af0 0x7f3b9c0ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.523+0000 7f3ba11ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 0x7f3b9c0ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.523+0000 7f3ba11ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b9c0adbf0 con 0x7f3b9c159720 2023-10-08T01:12:39.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.523+0000 7f3ba11ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9c0aba20 con 0x7f3b9c159af0 2023-10-08T01:12:39.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.523+0000 7f3ba11ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c0ac020 con 0x7f3b9c0ad250 2023-10-08T01:12:39.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9ad9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3b9c159720 0x7f3b9c0a98d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57782/0 (socket says 172.21.15.119:57782) 2023-10-08T01:12:39.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9ad9d700 1 -- 172.21.15.119:0/1991551016 learned_addr learned my addr 172.21.15.119:0/1991551016 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9a59c700 1 --2- 172.21.15.119:0/1991551016 >> v2:172.21.15.119:3300/0 conn(0x7f3b9c159af0 0x7f3b9c0ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9b59e700 1 --2- 172.21.15.119:0/1991551016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 0x7f3b9c0ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9a59c700 1 -- 172.21.15.119:0/1991551016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 msgr2=0x7f3b9c0ad6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9a59c700 1 --2- 172.21.15.119:0/1991551016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 0x7f3b9c0ad6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9a59c700 1 -- 172.21.15.119:0/1991551016 >> v1:172.21.15.119:6789/0 conn(0x7f3b9c159720 legacy=0x7f3b9c0a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9a59c700 1 -- 172.21.15.119:0/1991551016 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c0aa440 con 0x7f3b9c159af0 2023-10-08T01:12:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b99d9b700 1 -- 172.21.15.119:0/1991551016 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290330129 0 0) 0x7f3b8c001110 con 0x7f3b9c159720 2023-10-08T01:12:39.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b9a59c700 1 --2- 172.21.15.119:0/1991551016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159af0 0x7f3b9c0ab4e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3b900098e0 tx=0x7f3b900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486bbbc01038d662 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.524+0000 7f3b99d9b700 1 -- 172.21.15.119:0/1991551016 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9000b640 con 0x7f3b9c159af0 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3b99d9b700 1 -- 172.21.15.119:0/1991551016 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9000b7a0 con 0x7f3b9c159af0 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3b99d9b700 1 -- 172.21.15.119:0/1991551016 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9000bb40 con 0x7f3b9c159af0 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1991551016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159af0 msgr2=0x7f3b9c0ab4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1991551016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159af0 0x7f3b9c0ab4e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3b900098e0 tx=0x7f3b900091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1991551016 shutdown_connections 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1991551016 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 0x7f3b9c0ad6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1991551016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159af0 0x7f3b9c0ab4e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1991551016 >> 172.21.15.119:0/1991551016 conn(0x7f3b9c06a320 msgr2=0x7f3b9c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:39.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1991551016 shutdown_connections 2023-10-08T01:12:39.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.525+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1991551016 wait complete. 2023-10-08T01:12:39.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.527+0000 7f3ba11ff700 1 Processor -- start 2023-10-08T01:12:39.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.527+0000 7f3ba11ff700 1 -- start start 2023-10-08T01:12:39.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3ba11ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 0x7f3b9c14bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3ba11ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159720 0x7f3b9c14dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3ba11ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b9c159af0 0x7f3b9c156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3ba11ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c14e760 con 0x7f3b9c159720 2023-10-08T01:12:39.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3ba11ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c14ea40 con 0x7f3b9c159af0 2023-10-08T01:12:39.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3ba11ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c14ed20 con 0x7f3b9c0ad250 2023-10-08T01:12:39.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159720 0x7f3b9c14dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159720 0x7f3b9c14dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37038/0 (socket says 172.21.15.119:37038) 2023-10-08T01:12:39.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9a59c700 1 -- 172.21.15.119:0/1195015707 learned_addr learned my addr 172.21.15.119:0/1195015707 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:39.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9ad9d700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 0x7f3b9c14bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9b59e700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b9c159af0 0x7f3b9c156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9a59c700 1 -- 172.21.15.119:0/1195015707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 msgr2=0x7f3b9c14bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9a59c700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 0x7f3b9c14bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9a59c700 1 -- 172.21.15.119:0/1195015707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b9c159af0 msgr2=0x7f3b9c156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:39.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9a59c700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b9c159af0 0x7f3b9c156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:39.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.528+0000 7f3b9a59c700 1 -- 172.21.15.119:0/1195015707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b90009420 con 0x7f3b9c159720 2023-10-08T01:12:39.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.529+0000 7f3b9a59c700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159720 0x7f3b9c14dda0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c064290 tx=0x7f3b90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:39.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.529+0000 7f3b82ffd700 1 -- 172.21.15.119:0/1195015707 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9000b910 con 0x7f3b9c159720 2023-10-08T01:12:39.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.529+0000 7f3b82ffd700 1 -- 172.21.15.119:0/1195015707 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9000ba70 con 0x7f3b9c159720 2023-10-08T01:12:39.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.529+0000 7f3b82ffd700 1 -- 172.21.15.119:0/1195015707 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b90011670 con 0x7f3b9c159720 2023-10-08T01:12:39.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.529+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c14f000 con 0x7f3b9c159720 2023-10-08T01:12:39.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.529+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c14c040 con 0x7f3b9c159720 2023-10-08T01:12:39.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.530+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9c04c3c0 con 0x7f3b9c159720 2023-10-08T01:12:39.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.531+0000 7f3b82ffd700 1 -- 172.21.15.119:0/1195015707 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b90011aa0 con 0x7f3b9c159720 2023-10-08T01:12:39.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.532+0000 7f3b82ffd700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b88066f30 0x7f3b880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:39.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.532+0000 7f3b82ffd700 1 -- 172.21.15.119:0/1195015707 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5890+0+0 (secure 0 0 0) 0x7f3b90085e40 con 0x7f3b9c159720 2023-10-08T01:12:39.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.532+0000 7f3b9ad9d700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b88066f30 0x7f3b880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:39.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.533+0000 7f3b9ad9d700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b88066f30 0x7f3b880693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c006fc0 tx=0x7f3b8c007b90 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:39.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.536+0000 7f3b82ffd700 1 -- 172.21.15.119:0/1195015707 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b90052f50 con 0x7f3b9c159720 2023-10-08T01:12:39.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:39.743+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3b9c0aa720 con 0x7f3b9c159720 2023-10-08T01:12:40.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.221+0000 7f3b82ffd700 1 -- 172.21.15.119:0/1195015707 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3b90056570 con 0x7f3b9c159720 2023-10-08T01:12:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.224+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b88066f30 msgr2=0x7f3b880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.224+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b88066f30 0x7f3b880693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c006fc0 tx=0x7f3b8c007b90 comp rx=0 tx=0).stop 2023-10-08T01:12:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.224+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159720 msgr2=0x7f3b9c14dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.224+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159720 0x7f3b9c14dda0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c064290 tx=0x7f3b90017000 comp rx=0 tx=0).stop 2023-10-08T01:12:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.224+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 shutdown_connections 2023-10-08T01:12:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.224+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b88066f30 0x7f3b880693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.225+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3b9c0ad250 0x7f3b9c14bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.225+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3b9c159af0 0x7f3b9c156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.225+0000 7f3ba11ff700 1 --2- 172.21.15.119:0/1195015707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3b9c159720 0x7f3b9c14dda0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.225+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 >> 172.21.15.119:0/1195015707 conn(0x7f3b9c06a320 msgr2=0x7f3b9c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.225+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 shutdown_connections 2023-10-08T01:12:40.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.225+0000 7f3ba11ff700 1 -- 172.21.15.119:0/1195015707 wait complete. 2023-10-08T01:12:40.227 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T01:12:40.245 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T01:12:40.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.424+0000 7fe9065ff700 1 Processor -- start 2023-10-08T01:12:40.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.424+0000 7fe9065ff700 1 -- start start 2023-10-08T01:12:40.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.425+0000 7fe9065ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe9001578b0 0x7fe900155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.425+0000 7fe9065ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9000af2f0 0x7fe9000af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.425+0000 7fe9065ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9000ad930 con 0x7fe900155c10 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.425+0000 7fe9065ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000acd20 con 0x7fe9000af2f0 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.425+0000 7fe9065ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9000ad000 con 0x7fe9001578b0 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe8fffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe900155c10 0x7fe9000ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57798/0 (socket says 172.21.15.119:57798) 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe8fffff700 1 -- 172.21.15.119:0/1097347038 learned_addr learned my addr 172.21.15.119:0/1097347038 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe904b9c700 1 --2- 172.21.15.119:0/1097347038 >> v2:172.21.15.119:3300/0 conn(0x7fe9000af2f0 0x7fe9000af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe8ff7fe700 1 --2- 172.21.15.119:0/1097347038 >> v2:172.21.15.184:3300/0 conn(0x7fe9001578b0 0x7fe900155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe8feffd700 1 -- 172.21.15.119:0/1097347038 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149205568 0 0) 0x7fe9000ad930 con 0x7fe900155c10 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe8feffd700 1 -- 172.21.15.119:0/1097347038 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8e8003570 con 0x7fe900155c10 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe904b9c700 1 -- 172.21.15.119:0/1097347038 >> v2:172.21.15.184:3300/0 conn(0x7fe9001578b0 msgr2=0x7fe900155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe904b9c700 1 --2- 172.21.15.119:0/1097347038 >> v2:172.21.15.184:3300/0 conn(0x7fe9001578b0 0x7fe900155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe904b9c700 1 -- 172.21.15.119:0/1097347038 >> v1:172.21.15.119:6789/0 conn(0x7fe900155c10 legacy=0x7fe9000ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe904b9c700 1 -- 172.21.15.119:0/1097347038 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9000ad2e0 con 0x7fe9000af2f0 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.426+0000 7fe8ff7fe700 1 --2- 172.21.15.119:0/1097347038 >> v2:172.21.15.184:3300/0 conn(0x7fe9001578b0 0x7fe900155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe904b9c700 1 --2- 172.21.15.119:0/1097347038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9000af730 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe9000c8c60 tx=0x7fe8f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c720b6fb4e7982 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe8feffd700 1 -- 172.21.15.119:0/1097347038 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800e070 con 0x7fe9000af2f0 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe8feffd700 1 -- 172.21.15.119:0/1097347038 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f800bb60 con 0x7fe9000af2f0 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe8feffd700 1 -- 172.21.15.119:0/1097347038 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8012430 con 0x7fe9000af2f0 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe9065ff700 1 -- 172.21.15.119:0/1097347038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 msgr2=0x7fe9000af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe9065ff700 1 --2- 172.21.15.119:0/1097347038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9000af730 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fe9000c8c60 tx=0x7fe8f8009780 comp rx=0 tx=0).stop 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe9065ff700 1 -- 172.21.15.119:0/1097347038 shutdown_connections 2023-10-08T01:12:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe9065ff700 1 --2- 172.21.15.119:0/1097347038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9000af730 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe9065ff700 1 --2- 172.21.15.119:0/1097347038 >> v2:172.21.15.184:3300/0 conn(0x7fe9001578b0 0x7fe900155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe9065ff700 1 -- 172.21.15.119:0/1097347038 >> 172.21.15.119:0/1097347038 conn(0x7fe90006a320 msgr2=0x7fe9000cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.427+0000 7fe9065ff700 1 -- 172.21.15.119:0/1097347038 shutdown_connections 2023-10-08T01:12:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.428+0000 7fe9065ff700 1 -- 172.21.15.119:0/1097347038 wait complete. 2023-10-08T01:12:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.429+0000 7fe9065ff700 1 Processor -- start 2023-10-08T01:12:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.429+0000 7fe9065ff700 1 -- start start 2023-10-08T01:12:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe9065ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9001696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe9065ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe900155c10 0x7fe9000abf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe9065ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9001578b0 0x7fe9000ac4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe9065ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90016e480 con 0x7fe9000af2f0 2023-10-08T01:12:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe9065ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90016aa20 con 0x7fe900155c10 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe9065ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90016aeb0 con 0x7fe9001578b0 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe8fffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9001696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe8fffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9001696b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37058/0 (socket says 172.21.15.119:37058) 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.430+0000 7fe8fffff700 1 -- 172.21.15.119:0/826536784 learned_addr learned my addr 172.21.15.119:0/826536784 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8fffff700 1 -- 172.21.15.119:0/826536784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9001578b0 msgr2=0x7fe9000ac4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8fffff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9001578b0 0x7fe9000ac4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8fffff700 1 -- 172.21.15.119:0/826536784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe900155c10 msgr2=0x7fe9000abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8fffff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe900155c10 0x7fe9000abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8fffff700 1 -- 172.21.15.119:0/826536784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8009a20 con 0x7fe9000af2f0 2023-10-08T01:12:40.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8fffff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9001696b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000c7c0 tx=0x7fe8f0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:40.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8e7fff700 1 -- 172.21.15.119:0/826536784 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0005b30 con 0x7fe9000af2f0 2023-10-08T01:12:40.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8e7fff700 1 -- 172.21.15.119:0/826536784 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f0005c90 con 0x7fe9000af2f0 2023-10-08T01:12:40.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90016a0f0 con 0x7fe9000af2f0 2023-10-08T01:12:40.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe8e7fff700 1 -- 172.21.15.119:0/826536784 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0007340 con 0x7fe9000af2f0 2023-10-08T01:12:40.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.431+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90016a6b0 con 0x7fe9000af2f0 2023-10-08T01:12:40.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.432+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90004c3c0 con 0x7fe9000af2f0 2023-10-08T01:12:40.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.433+0000 7fe8e7fff700 1 -- 172.21.15.119:0/826536784 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8f001b400 con 0x7fe9000af2f0 2023-10-08T01:12:40.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.434+0000 7fe8e7fff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe8ec066f30 0x7fe8ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.434+0000 7fe8e7fff700 1 -- 172.21.15.119:0/826536784 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fe8f0083e70 con 0x7fe9000af2f0 2023-10-08T01:12:40.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.434+0000 7fe8ff7fe700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe8ec066f30 0x7fe8ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.434+0000 7fe8ff7fe700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe8ec066f30 0x7fe8ec0693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006fd0 tx=0x7fe8f4008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:40.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.438+0000 7fe8e7fff700 1 -- 172.21.15.119:0/826536784 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8f0051330 con 0x7fe9000af2f0 2023-10-08T01:12:40.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.699+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe9000ad5c0 con 0x7fe9000af2f0 2023-10-08T01:12:40.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.700+0000 7fe8e7fff700 1 -- 172.21.15.119:0/826536784 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe8f0054950 con 0x7fe9000af2f0 2023-10-08T01:12:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:40.702 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:12:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T01:12:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T01:12:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T01:12:40.702 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T01:12:40.703 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.704 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T01:12:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T01:12:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:12:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:12:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:12:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.705 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T01:12:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T01:12:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:12:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:12:40.706 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:12:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T01:12:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T01:12:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.707 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:12:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:12:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T01:12:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T01:12:40.708 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:12:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:12:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T01:12:40.709 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T01:12:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:12:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:12:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.710 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T01:12:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T01:12:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:12:40.711 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.712 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T01:12:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T01:12:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:12:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:12:40.713 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T01:12:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T01:12:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.714 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:12:40.715 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T01:12:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T01:12:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:12:40.716 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T01:12:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T01:12:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T01:12:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T01:12:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.717 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:12:40.718 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:12:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:12:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T01:12:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T01:12:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.719 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:12:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:12:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:12:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:12:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.720 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T01:12:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T01:12:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:12:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:12:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:12:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.721 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T01:12:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T01:12:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:12:40.722 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:12:40.723 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:12:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:12:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:40.724 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T01:12:40.724 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:12:40.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe8ec066f30 msgr2=0x7fe8ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe8ec066f30 0x7fe8ec0693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006fd0 tx=0x7fe8f4008040 comp rx=0 tx=0).stop 2023-10-08T01:12:40.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 msgr2=0x7fe9001696b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9001696b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000c7c0 tx=0x7fe8f0007e00 comp rx=0 tx=0).stop 2023-10-08T01:12:40.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 shutdown_connections 2023-10-08T01:12:40.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe8ec066f30 0x7fe8ec0693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9001578b0 0x7fe9000ac4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9000af2f0 0x7fe9001696b0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 --2- 172.21.15.119:0/826536784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe900155c10 0x7fe9000abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 >> 172.21.15.119:0/826536784 conn(0x7fe90006a320 msgr2=0x7fe900153690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:40.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.702+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 shutdown_connections 2023-10-08T01:12:40.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.703+0000 7fe9065ff700 1 -- 172.21.15.119:0/826536784 wait complete. 2023-10-08T01:12:40.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:40.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.910+0000 7f42a91ff700 1 Processor -- start 2023-10-08T01:12:40.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.910+0000 7f42a91ff700 1 -- start start 2023-10-08T01:12:40.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.911+0000 7f42a91ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a4069560 0x7f42a40699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.911+0000 7f42a91ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a4069ee0 con 0x7f42a4159890 2023-10-08T01:12:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.911+0000 7f42a91ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a415a160 con 0x7f42a4159c60 2023-10-08T01:12:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.911+0000 7f42a91ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40d13a0 con 0x7f42a4069560 2023-10-08T01:12:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a2d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f42a4159890 0x7f42a415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57800/0 (socket says 172.21.15.119:57800) 2023-10-08T01:12:40.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a2d9d700 1 -- 172.21.15.119:0/1124887756 learned_addr learned my addr 172.21.15.119:0/1124887756 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:40.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a359e700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a4069560 0x7f42a40699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004079756 0 0) 0x7f42a4069ee0 con 0x7f42a4159890 2023-10-08T01:12:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f428c003570 con 0x7f42a4159890 2023-10-08T01:12:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159701463 0 0) 0x7f42a415a160 con 0x7f42a4159c60 2023-10-08T01:12:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a4069ee0 con 0x7f42a4159c60 2023-10-08T01:12:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4128216245 0 0) 0x7f428c003570 con 0x7f42a4159890 2023-10-08T01:12:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a4069560 msgr2=0x7f42a40699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a4069560 0x7f42a40699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 >> v1:172.21.15.184:6789/0 conn(0x7f42a4159c60 legacy=0x7f42a415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a415e140 con 0x7f42a4159890 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f4294001470 con 0x7f42a4159890 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 >> v1:172.21.15.119:6789/0 conn(0x7f42a4159890 legacy=0x7f42a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a4069560 0x7f428c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f428c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c004d50 0x7f428c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40d13a0 con 0x7f428c004180 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428c00bba0 con 0x7f428c004d50 2023-10-08T01:12:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.912+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428c00bd60 con 0x7f42a4069560 2023-10-08T01:12:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a259c700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f428c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a2d9d700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a4069560 0x7f428c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a359e700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c004d50 0x7f428c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a259c700 1 -- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a4069560 msgr2=0x7f428c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a259c700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a4069560 0x7f428c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a259c700 1 -- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c004d50 msgr2=0x7f428c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a259c700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f428c004d50 0x7f428c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a259c700 1 -- 172.21.15.119:0/1124887756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428c003570 con 0x7f428c004180 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a259c700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f428c004810 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f42a406a070 tx=0x7f4298007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64cfc98378627d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.913+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800c5d0 con 0x7f428c004180 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429800c730 con 0x7f428c004180 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a1d9b700 1 -- 172.21.15.119:0/1124887756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800caf0 con 0x7f428c004180 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a91ff700 1 -- 172.21.15.119:0/1124887756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 msgr2=0x7f428c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a91ff700 1 --2- 172.21.15.119:0/1124887756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f428c004810 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f42a406a070 tx=0x7f4298007d10 comp rx=0 tx=0).stop 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a359e700 1 -- 172.21.15.119:0/1124887756 reap_dead start 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a91ff700 1 -- 172.21.15.119:0/1124887756 shutdown_connections 2023-10-08T01:12:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a91ff700 1 -- 172.21.15.119:0/1124887756 >> 172.21.15.119:0/1124887756 conn(0x7f42a4068e20 msgr2=0x7f42a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:40.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a91ff700 1 -- 172.21.15.119:0/1124887756 shutdown_connections 2023-10-08T01:12:40.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.914+0000 7f42a91ff700 1 -- 172.21.15.119:0/1124887756 wait complete. 2023-10-08T01:12:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.916+0000 7f42a91ff700 1 Processor -- start 2023-10-08T01:12:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.916+0000 7f42a91ff700 1 -- start start 2023-10-08T01:12:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.916+0000 7f42a91ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f42a4154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.916+0000 7f42a91ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a40cfce0 0x7f42a41552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.916+0000 7f42a91ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42a414cbd0 0x7f42a414d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a91ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a415bc10 con 0x7f428c004180 2023-10-08T01:12:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a91ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40cd7f0 con 0x7f42a414cbd0 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a91ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a415b600 con 0x7f42a40cfce0 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f42a4154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f42a4154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37072/0 (socket says 172.21.15.119:37072) 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 -- 172.21.15.119:0/1071719107 learned_addr learned my addr 172.21.15.119:0/1071719107 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a259c700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a40cfce0 0x7f42a41552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a359e700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42a414cbd0 0x7f42a414d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 -- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a40cfce0 msgr2=0x7f42a41552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a40cfce0 0x7f42a41552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 -- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42a414cbd0 msgr2=0x7f42a414d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42a414cbd0 0x7f42a414d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 -- 172.21.15.119:0/1071719107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a40cd510 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a359e700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42a414cbd0 0x7f42a414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.917+0000 7f42a2d9d700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f42a4154db0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4294007390 tx=0x7f4294001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.918+0000 7f428affd700 1 -- 172.21.15.119:0/1071719107 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294018030 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.918+0000 7f428affd700 1 -- 172.21.15.119:0/1071719107 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429400f8d0 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.918+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a415b8a0 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.918+0000 7f428affd700 1 -- 172.21.15.119:0/1071719107 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429400fc50 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.918+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a40cd0c0 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.919+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a404c3c0 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.920+0000 7f428affd700 1 -- 172.21.15.119:0/1071719107 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f429401b070 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.921+0000 7f428affd700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4290066f30 0x7f42900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.921+0000 7f428affd700 1 -- 172.21.15.119:0/1071719107 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f4294019070 con 0x7f428c004180 2023-10-08T01:12:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.921+0000 7f42a259c700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4290066f30 0x7f42900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:40.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.922+0000 7f42a259c700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4290066f30 0x7f42900693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4298001520 tx=0x7f4298015000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:40.925+0000 7f428affd700 1 -- 172.21.15.119:0/1071719107 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4294051710 con 0x7f428c004180 2023-10-08T01:12:41.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.115+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42a40cd250 con 0x7f428c004180 2023-10-08T01:12:41.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.116+0000 7f428affd700 1 -- 172.21.15.119:0/1071719107 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+12781 (secure 0 0 0) 0x7f4294054d30 con 0x7f428c004180 2023-10-08T01:12:41.117 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:41.117 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":100,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:40.227254+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.118+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4290066f30 msgr2=0x7f42900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.118+0000 7f42a91ff700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4290066f30 0x7f42900693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4298001520 tx=0x7f4298015000 comp rx=0 tx=0).stop 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.118+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 msgr2=0x7f42a4154db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.118+0000 7f42a91ff700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f42a4154db0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4294007390 tx=0x7f4294001df0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.118+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 shutdown_connections 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.118+0000 7f42a91ff700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4290066f30 0x7f42900693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.118+0000 7f42a91ff700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f42a414cbd0 0x7f42a414d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.118+0000 7f42a91ff700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f428c004180 0x7f42a4154db0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.119+0000 7f42a91ff700 1 --2- 172.21.15.119:0/1071719107 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f42a40cfce0 0x7f42a41552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.119+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 >> 172.21.15.119:0/1071719107 conn(0x7f42a4068e20 msgr2=0x7f42a40b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:41.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.119+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 shutdown_connections 2023-10-08T01:12:41.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.119+0000 7f42a91ff700 1 -- 172.21.15.119:0/1071719107 wait complete. 2023-10-08T01:12:41.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T01:12:41.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:12:41.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.324+0000 7ff66a7ff700 1 Processor -- start 2023-10-08T01:12:41.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.324+0000 7ff66a7ff700 1 -- start start 2023-10-08T01:12:41.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.325+0000 7ff66a7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff664069560 0x7ff6640699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.325+0000 7ff66a7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff664069ee0 con 0x7ff664159c60 2023-10-08T01:12:41.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.325+0000 7ff66a7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff66415a160 con 0x7ff664159890 2023-10-08T01:12:41.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.325+0000 7ff66a7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6640d13a0 con 0x7ff664069560 2023-10-08T01:12:41.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff6637fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff664159c60 0x7ff66415d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57804/0 (socket says 172.21.15.119:57804) 2023-10-08T01:12:41.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff6637fe700 1 -- 172.21.15.119:0/1712610987 learned_addr learned my addr 172.21.15.119:0/1712610987 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:41.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff668d9c700 1 --2- 172.21.15.119:0/1712610987 >> v2:172.21.15.184:3300/0 conn(0x7ff664069560 0x7ff6640699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252727152 0 0) 0x7ff664069ee0 con 0x7ff664159c60 2023-10-08T01:12:41.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff64c003570 con 0x7ff664159c60 2023-10-08T01:12:41.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874349501 0 0) 0x7ff66415a160 con 0x7ff664159890 2023-10-08T01:12:41.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff664069ee0 con 0x7ff664159890 2023-10-08T01:12:41.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1807536577 0 0) 0x7ff64c003570 con 0x7ff664159c60 2023-10-08T01:12:41.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 >> v2:172.21.15.184:3300/0 conn(0x7ff664069560 msgr2=0x7ff6640699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:41.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 --2- 172.21.15.119:0/1712610987 >> v2:172.21.15.184:3300/0 conn(0x7ff664069560 0x7ff6640699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 >> v1:172.21.15.184:6789/0 conn(0x7ff664159890 legacy=0x7ff66415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66415e140 con 0x7ff664159c60 2023-10-08T01:12:41.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7ff658001470 con 0x7ff664159c60 2023-10-08T01:12:41.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 >> v1:172.21.15.119:6789/0 conn(0x7ff664159c60 legacy=0x7ff66415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004c50 0x7ff64c005040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0037b0 0x7ff64c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.326+0000 7ff662ffd700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff64c004160 0x7ff64c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6640d13a0 con 0x7ff64c004160 2023-10-08T01:12:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c00c0b0 con 0x7ff64c004c50 2023-10-08T01:12:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c00c270 con 0x7ff64c0037b0 2023-10-08T01:12:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff668d9c700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff64c004160 0x7ff64c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff663fff700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004c50 0x7ff64c005040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff6637fe700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0037b0 0x7ff64c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff668d9c700 1 -- 172.21.15.119:0/1712610987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0037b0 msgr2=0x7ff64c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:41.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff668d9c700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c0037b0 0x7ff64c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff668d9c700 1 -- 172.21.15.119:0/1712610987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004c50 msgr2=0x7ff64c005040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:41.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff668d9c700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff64c004c50 0x7ff64c005040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff668d9c700 1 -- 172.21.15.119:0/1712610987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c003570 con 0x7ff64c004160 2023-10-08T01:12:41.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff668d9c700 1 -- 172.21.15.119:0/1712610987 reap_dead start 2023-10-08T01:12:41.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.327+0000 7ff668d9c700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff64c004160 0x7ff64c00ba60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00e210 tx=0x7ff65c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3feee8e03c3af746 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:41.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00eac0 con 0x7ff64c004160 2023-10-08T01:12:41.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff65c00f040 con 0x7ff64c004160 2023-10-08T01:12:41.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff662ffd700 1 -- 172.21.15.119:0/1712610987 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65c00ee70 con 0x7ff64c004160 2023-10-08T01:12:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff66a7ff700 1 -- 172.21.15.119:0/1712610987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff64c004160 msgr2=0x7ff64c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff66a7ff700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff64c004160 0x7ff64c00ba60 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00e210 tx=0x7ff65c00b040 comp rx=0 tx=0).stop 2023-10-08T01:12:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff66a7ff700 1 -- 172.21.15.119:0/1712610987 shutdown_connections 2023-10-08T01:12:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff66a7ff700 1 --2- 172.21.15.119:0/1712610987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff64c004160 0x7ff64c00ba60 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff66a7ff700 1 -- 172.21.15.119:0/1712610987 >> 172.21.15.119:0/1712610987 conn(0x7ff664068e20 msgr2=0x7ff6640d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff66a7ff700 1 -- 172.21.15.119:0/1712610987 shutdown_connections 2023-10-08T01:12:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.328+0000 7ff66a7ff700 1 -- 172.21.15.119:0/1712610987 wait complete. 2023-10-08T01:12:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.330+0000 7ff66a7ff700 1 Processor -- start 2023-10-08T01:12:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.330+0000 7ff66a7ff700 1 -- start start 2023-10-08T01:12:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff66a7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c004160 0x7ff6640cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff66a7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6640ced30 0x7ff664150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff66a7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6640cf1a0 0x7ff664150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff66a7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66416ad30 con 0x7ff6640ced30 2023-10-08T01:12:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff66a7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66416a160 con 0x7ff6640cf1a0 2023-10-08T01:12:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff66a7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66416a400 con 0x7ff64c004160 2023-10-08T01:12:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff6637fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6640ced30 0x7ff664150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff6637fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6640ced30 0x7ff664150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37104/0 (socket says 172.21.15.119:37104) 2023-10-08T01:12:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff6637fe700 1 -- 172.21.15.119:0/248417086 learned_addr learned my addr 172.21.15.119:0/248417086 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff663fff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c004160 0x7ff6640cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff668d9c700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6640cf1a0 0x7ff664150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff6637fe700 1 -- 172.21.15.119:0/248417086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c004160 msgr2=0x7ff6640cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff6637fe700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c004160 0x7ff6640cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff6637fe700 1 -- 172.21.15.119:0/248417086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6640cf1a0 msgr2=0x7ff664150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff6637fe700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6640cf1a0 0x7ff664150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.331+0000 7ff6637fe700 1 -- 172.21.15.119:0/248417086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66416a6e0 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.332+0000 7ff6637fe700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6640ced30 0x7ff664150a30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e7b0 tx=0x7ff658001010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.332+0000 7ff64bfff700 1 -- 172.21.15.119:0/248417086 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658012500 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.332+0000 7ff64bfff700 1 -- 172.21.15.119:0/248417086 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658010040 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.332+0000 7ff64bfff700 1 -- 172.21.15.119:0/248417086 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6580128b0 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.332+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66416a9c0 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.332+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff664155230 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.333+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66416aee0 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.334+0000 7ff64bfff700 1 -- 172.21.15.119:0/248417086 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff658012ca0 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.335+0000 7ff64bfff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff650066f30 0x7ff6500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.335+0000 7ff64bfff700 1 -- 172.21.15.119:0/248417086 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5245+0+0 (secure 0 0 0) 0x7ff6580838c0 con 0x7ff6640ced30 2023-10-08T01:12:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.335+0000 7ff663fff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff650066f30 0x7ff6500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.336+0000 7ff663fff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff650066f30 0x7ff6500693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff654007e40 tx=0x7ff654009040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.339+0000 7ff64bfff700 1 -- 172.21.15.119:0/248417086 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff66416aee0 con 0x7ff6640ced30 2023-10-08T01:12:41.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.534+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff66416aee0 con 0x7ff6640ced30 2023-10-08T01:12:41.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.535+0000 7ff64bfff700 1 -- 172.21.15.119:0/248417086 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff66416aee0 con 0x7ff6640ced30 2023-10-08T01:12:41.536 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:12:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff650066f30 msgr2=0x7ff6500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff650066f30 0x7ff6500693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff654007e40 tx=0x7ff654009040 comp rx=0 tx=0).stop 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6640ced30 msgr2=0x7ff664150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6640ced30 0x7ff664150a30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7ff65800e7b0 tx=0x7ff658001010 comp rx=0 tx=0).stop 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 shutdown_connections 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff650066f30 0x7ff6500693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff64c004160 0x7ff6640cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff6640cf1a0 0x7ff664150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 --2- 172.21.15.119:0/248417086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6640ced30 0x7ff664150a30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 >> 172.21.15.119:0/248417086 conn(0x7ff664068e20 msgr2=0x7ff6640b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.537+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 shutdown_connections 2023-10-08T01:12:41.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.538+0000 7ff66a7ff700 1 -- 172.21.15.119:0/248417086 wait complete. 2023-10-08T01:12:41.556 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T01:12:41.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.738+0000 7fea2a9ff700 1 Processor -- start 2023-10-08T01:12:41.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.738+0000 7fea2a9ff700 1 -- start start 2023-10-08T01:12:41.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.739+0000 7fea2a9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fea241599e0 0x7fea24069ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.739+0000 7fea2a9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2406a2b0 0x7fea2406a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.739+0000 7fea2a9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea240c96a0 con 0x7fea24159db0 2023-10-08T01:12:41.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.739+0000 7fea2a9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea240cb9e0 con 0x7fea241599e0 2023-10-08T01:12:41.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.739+0000 7fea2a9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea240cab60 con 0x7fea2406a2b0 2023-10-08T01:12:41.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea23fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fea241599e0 0x7fea24069ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea23fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fea241599e0 0x7fea24069ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37108/0 (socket says 172.21.15.119:37108) 2023-10-08T01:12:41.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea23fff700 1 -- 172.21.15.119:0/87190234 learned_addr learned my addr 172.21.15.119:0/87190234 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:41.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea28f9c700 1 --2- 172.21.15.119:0/87190234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2406a2b0 0x7fea2406a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea23fff700 1 -- 172.21.15.119:0/87190234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2406a2b0 msgr2=0x7fea2406a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea23fff700 1 --2- 172.21.15.119:0/87190234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2406a2b0 0x7fea2406a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea23fff700 1 -- 172.21.15.119:0/87190234 >> v1:172.21.15.184:6789/0 conn(0x7fea24159db0 legacy=0x7fea24069080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea23fff700 1 -- 172.21.15.119:0/87190234 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea240cb420 con 0x7fea241599e0 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea28f9c700 1 --2- 172.21.15.119:0/87190234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2406a2b0 0x7fea2406a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea23fff700 1 --2- 172.21.15.119:0/87190234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea241599e0 0x7fea24069ac0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fea140099b0 tx=0x7fea14009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13333de1c57adead server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.740+0000 7fea22ffd700 1 -- 172.21.15.119:0/87190234 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1400b820 con 0x7fea241599e0 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea22ffd700 1 -- 172.21.15.119:0/87190234 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1400b980 con 0x7fea241599e0 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea22ffd700 1 -- 172.21.15.119:0/87190234 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1400bd20 con 0x7fea241599e0 2023-10-08T01:12:41.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea2a9ff700 1 -- 172.21.15.119:0/87190234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea241599e0 msgr2=0x7fea24069ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/87190234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea241599e0 0x7fea24069ac0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fea140099b0 tx=0x7fea14009250 comp rx=0 tx=0).stop 2023-10-08T01:12:41.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea2a9ff700 1 -- 172.21.15.119:0/87190234 shutdown_connections 2023-10-08T01:12:41.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/87190234 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea2406a2b0 0x7fea2406a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/87190234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea241599e0 0x7fea24069ac0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea2a9ff700 1 -- 172.21.15.119:0/87190234 >> 172.21.15.119:0/87190234 conn(0x7fea240c9860 msgr2=0x7fea240c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:41.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea2a9ff700 1 -- 172.21.15.119:0/87190234 shutdown_connections 2023-10-08T01:12:41.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.741+0000 7fea2a9ff700 1 -- 172.21.15.119:0/87190234 wait complete. 2023-10-08T01:12:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.743+0000 7fea2a9ff700 1 Processor -- start 2023-10-08T01:12:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.743+0000 7fea2a9ff700 1 -- start start 2023-10-08T01:12:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea2a9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2406a2b0 0x7fea240b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea2a9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea241599e0 0x7fea2416a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea2a9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea24159db0 0x7fea24147940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea2a9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2416bd40 con 0x7fea24159db0 2023-10-08T01:12:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea2a9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2416ae90 con 0x7fea2406a2b0 2023-10-08T01:12:41.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea2a9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2416b130 con 0x7fea241599e0 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea28f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea24159db0 0x7fea24147940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea28f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea24159db0 0x7fea24147940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37122/0 (socket says 172.21.15.119:37122) 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea28f9c700 1 -- 172.21.15.119:0/114637928 learned_addr learned my addr 172.21.15.119:0/114637928 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea23fff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2406a2b0 0x7fea240b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea237fe700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea241599e0 0x7fea2416a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea28f9c700 1 -- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea241599e0 msgr2=0x7fea2416a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea28f9c700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea241599e0 0x7fea2416a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea28f9c700 1 -- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2406a2b0 msgr2=0x7fea240b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea28f9c700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2406a2b0 0x7fea240b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea28f9c700 1 -- 172.21.15.119:0/114637928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140094f0 con 0x7fea24159db0 2023-10-08T01:12:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.744+0000 7fea237fe700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea241599e0 0x7fea2416a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.745+0000 7fea28f9c700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea24159db0 0x7fea24147940 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00a380 tx=0x7fea1c007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.745+0000 7fea0bfff700 1 -- 172.21.15.119:0/114637928 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00ae10 con 0x7fea24159db0 2023-10-08T01:12:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.745+0000 7fea0bfff700 1 -- 172.21.15.119:0/114637928 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c00d650 con 0x7fea24159db0 2023-10-08T01:12:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.745+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2416b410 con 0x7fea24159db0 2023-10-08T01:12:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.745+0000 7fea0bfff700 1 -- 172.21.15.119:0/114637928 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c00da10 con 0x7fea24159db0 2023-10-08T01:12:41.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.745+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2416b9d0 con 0x7fea24159db0 2023-10-08T01:12:41.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.746+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea2416beb0 con 0x7fea24159db0 2023-10-08T01:12:41.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.747+0000 7fea0bfff700 1 -- 172.21.15.119:0/114637928 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea1c01e3b0 con 0x7fea24159db0 2023-10-08T01:12:41.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.748+0000 7fea0bfff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fea10066f30 0x7fea100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:41.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.748+0000 7fea0bfff700 1 -- 172.21.15.119:0/114637928 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fea1c084db0 con 0x7fea24159db0 2023-10-08T01:12:41.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.748+0000 7fea23fff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fea10066f30 0x7fea100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:41.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.749+0000 7fea23fff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fea10066f30 0x7fea100693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea140099b0 tx=0x7fea14016000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:41.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.752+0000 7fea0bfff700 1 -- 172.21.15.119:0/114637928 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea1c0521f0 con 0x7fea24159db0 2023-10-08T01:12:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.945+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fea2404c3c0 con 0x7fea24159db0 2023-10-08T01:12:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.945+0000 7fea0bfff700 1 -- 172.21.15.119:0/114637928 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) v1 ==== 84+0+65 (secure 0 0 0) 0x7fea1c01a090 con 0x7fea24159db0 2023-10-08T01:12:41.946 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:41.947 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:12:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T01:12:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T01:12:41.947 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:41.947 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:12:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fea10066f30 msgr2=0x7fea100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fea10066f30 0x7fea100693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fea140099b0 tx=0x7fea14016000 comp rx=0 tx=0).stop 2023-10-08T01:12:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea24159db0 msgr2=0x7fea24147940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea24159db0 0x7fea24147940 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00a380 tx=0x7fea1c007900 comp rx=0 tx=0).stop 2023-10-08T01:12:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 shutdown_connections 2023-10-08T01:12:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fea10066f30 0x7fea100693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea24159db0 0x7fea24147940 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fea2406a2b0 0x7fea240b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 --2- 172.21.15.119:0/114637928 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fea241599e0 0x7fea2416a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.947+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 >> 172.21.15.119:0/114637928 conn(0x7fea240c9860 msgr2=0x7fea24158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.948+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 shutdown_connections 2023-10-08T01:12:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:41.948+0000 7fea2a9ff700 1 -- 172.21.15.119:0/114637928 wait complete. 2023-10-08T01:12:41.966 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T01:12:41.966 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T01:12:42.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.149+0000 7fd94b9ff700 1 Processor -- start 2023-10-08T01:12:42.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.149+0000 7fd94b9ff700 1 -- start start 2023-10-08T01:12:42.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.150+0000 7fd94b9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 0x7fd944150120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:42.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.150+0000 7fd94b9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd944159e70 0x7fd94415a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.150+0000 7fd94b9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd94415da10 con 0x7fd9440689b0 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.150+0000 7fd94b9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd94415dcb0 con 0x7fd944159e70 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.150+0000 7fd94b9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd94415df90 con 0x7fd9440685e0 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd949f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd944159e70 0x7fd94415a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd949f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd944159e70 0x7fd94415a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37132/0 (socket says 172.21.15.119:37132) 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd949f9c700 1 -- 172.21.15.119:0/3905717939 learned_addr learned my addr 172.21.15.119:0/3905717939 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd94979b700 1 --2- 172.21.15.119:0/3905717939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 0x7fd944150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd949f9c700 1 -- 172.21.15.119:0/3905717939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 msgr2=0x7fd944150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:42.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd949f9c700 1 --2- 172.21.15.119:0/3905717939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 0x7fd944150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:42.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd949f9c700 1 -- 172.21.15.119:0/3905717939 >> v1:172.21.15.184:6789/0 conn(0x7fd9440689b0 legacy=0x7fd944150660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:42.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd949f9c700 1 -- 172.21.15.119:0/3905717939 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94415e270 con 0x7fd944159e70 2023-10-08T01:12:42.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd94979b700 1 --2- 172.21.15.119:0/3905717939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 0x7fd944150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:42.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.151+0000 7fd949f9c700 1 --2- 172.21.15.119:0/3905717939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944159e70 0x7fd94415a2d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009f20 tx=0x7fd93c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8507c71bef253618 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:42.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd93bfff700 1 -- 172.21.15.119:0/3905717939 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00e070 con 0x7fd944159e70 2023-10-08T01:12:42.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd93bfff700 1 -- 172.21.15.119:0/3905717939 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93c00bb10 con 0x7fd944159e70 2023-10-08T01:12:42.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd93bfff700 1 -- 172.21.15.119:0/3905717939 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c0092e0 con 0x7fd944159e70 2023-10-08T01:12:42.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd94b9ff700 1 -- 172.21.15.119:0/3905717939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944159e70 msgr2=0x7fd94415a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:42.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/3905717939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944159e70 0x7fd94415a2d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009f20 tx=0x7fd93c009840 comp rx=0 tx=0).stop 2023-10-08T01:12:42.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd94b9ff700 1 -- 172.21.15.119:0/3905717939 shutdown_connections 2023-10-08T01:12:42.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/3905717939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd944159e70 0x7fd94415a2d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:42.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/3905717939 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 0x7fd944150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:42.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd94b9ff700 1 -- 172.21.15.119:0/3905717939 >> 172.21.15.119:0/3905717939 conn(0x7fd9440c8f90 msgr2=0x7fd9440c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:42.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd94b9ff700 1 -- 172.21.15.119:0/3905717939 shutdown_connections 2023-10-08T01:12:42.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.152+0000 7fd94b9ff700 1 -- 172.21.15.119:0/3905717939 wait complete. 2023-10-08T01:12:42.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.154+0000 7fd94b9ff700 1 Processor -- start 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.154+0000 7fd94b9ff700 1 -- start start 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd94b9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 0x7fd944151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd94b9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9440689b0 0x7fd9440ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd94b9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944159e70 0x7fd9440ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd94b9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd944152a30 con 0x7fd9440689b0 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd94b9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9440c0020 con 0x7fd944159e70 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd94b9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9440c0300 con 0x7fd9440685e0 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd948f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9440689b0 0x7fd9440ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd948f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9440689b0 0x7fd9440ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37148/0 (socket says 172.21.15.119:37148) 2023-10-08T01:12:42.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd948f9a700 1 -- 172.21.15.119:0/836139067 learned_addr learned my addr 172.21.15.119:0/836139067 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd948f9a700 1 -- 172.21.15.119:0/836139067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 msgr2=0x7fd944151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd948f9a700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 0x7fd944151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd948f9a700 1 -- 172.21.15.119:0/836139067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944159e70 msgr2=0x7fd9440ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd948f9a700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944159e70 0x7fd9440ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.155+0000 7fd948f9a700 1 -- 172.21.15.119:0/836139067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c009ae0 con 0x7fd9440689b0 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.156+0000 7fd948f9a700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9440689b0 0x7fd9440ca050 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c590 tx=0x7fd93400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.156+0000 7fd9397fa700 1 -- 172.21.15.119:0/836139067 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934011590 con 0x7fd9440689b0 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.156+0000 7fd9397fa700 1 -- 172.21.15.119:0/836139067 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9340116f0 con 0x7fd9440689b0 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.156+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9440c05e0 con 0x7fd9440689b0 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.156+0000 7fd9397fa700 1 -- 172.21.15.119:0/836139067 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934011ab0 con 0x7fd9440689b0 2023-10-08T01:12:42.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.156+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9440c0ba0 con 0x7fd9440689b0 2023-10-08T01:12:42.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.157+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9440718b0 con 0x7fd9440689b0 2023-10-08T01:12:42.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.158+0000 7fd9397fa700 1 -- 172.21.15.119:0/836139067 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd93401b3b0 con 0x7fd9440689b0 2023-10-08T01:12:42.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.159+0000 7fd9397fa700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd928066f30 0x7fd9280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:42.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.159+0000 7fd9397fa700 1 -- 172.21.15.119:0/836139067 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fd934083910 con 0x7fd9440689b0 2023-10-08T01:12:42.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.159+0000 7fd94979b700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd928066f30 0x7fd9280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:42.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.159+0000 7fd94979b700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd928066f30 0x7fd9280693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd940006fd0 tx=0x7fd940008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:42.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.163+0000 7fd9397fa700 1 -- 172.21.15.119:0/836139067 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd934050d50 con 0x7fd9440689b0 2023-10-08T01:12:42.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:42.356+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd9440c8800 con 0x7fd9440689b0 2023-10-08T01:12:43.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.231+0000 7fd9397fa700 1 -- 172.21.15.119:0/836139067 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd934017070 con 0x7fd9440689b0 2023-10-08T01:12:43.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd928066f30 msgr2=0x7fd9280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:43.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd928066f30 0x7fd9280693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd940006fd0 tx=0x7fd940008040 comp rx=0 tx=0).stop 2023-10-08T01:12:43.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9440689b0 msgr2=0x7fd9440ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:43.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9440689b0 0x7fd9440ca050 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd93400c590 tx=0x7fd93400c950 comp rx=0 tx=0).stop 2023-10-08T01:12:43.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 shutdown_connections 2023-10-08T01:12:43.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd928066f30 0x7fd9280693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9440685e0 0x7fd944151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd944159e70 0x7fd9440ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 --2- 172.21.15.119:0/836139067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9440689b0 0x7fd9440ca050 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.237+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 >> 172.21.15.119:0/836139067 conn(0x7fd9440c8f90 msgr2=0x7fd94406d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:43.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.239+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 shutdown_connections 2023-10-08T01:12:43.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.239+0000 7fd94b9ff700 1 -- 172.21.15.119:0/836139067 wait complete. 2023-10-08T01:12:43.242 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T01:12:43.261 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T01:12:43.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.452+0000 7fb99a3ff700 1 Processor -- start 2023-10-08T01:12:43.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.453+0000 7fb99a3ff700 1 -- start start 2023-10-08T01:12:43.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.454+0000 7fb99a3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb994157630 0x7fb994158a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.454+0000 7fb99a3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb994157a00 0x7fb9940abd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.454+0000 7fb99a3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb994158590 con 0x7fb9940ac290 2023-10-08T01:12:43.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.454+0000 7fb99a3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb994157f80 con 0x7fb994157630 2023-10-08T01:12:43.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.454+0000 7fb99a3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9940ac7a0 con 0x7fb994157a00 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.454+0000 7fb99899c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb9940ac290 0x7fb994068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57814/0 (socket says 172.21.15.119:57814) 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.454+0000 7fb99899c700 1 -- 172.21.15.119:0/3381533702 learned_addr learned my addr 172.21.15.119:0/3381533702 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.454+0000 7fb98bfff700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb994157a00 0x7fb9940abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb993fff700 1 --2- 172.21.15.119:0/3381533702 >> v2:172.21.15.184:3300/0 conn(0x7fb994157630 0x7fb994158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552944061 0 0) 0x7fb994158590 con 0x7fb9940ac290 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb980003570 con 0x7fb9940ac290 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409739898 0 0) 0x7fb980003570 con 0x7fb9940ac290 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb994157a00 msgr2=0x7fb9940abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb994157a00 0x7fb9940abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 >> v2:172.21.15.184:3300/0 conn(0x7fb994157630 msgr2=0x7fb994158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 --2- 172.21.15.119:0/3381533702 >> v2:172.21.15.184:3300/0 conn(0x7fb994157630 0x7fb994158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb994158220 con 0x7fb9940ac290 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fb98c0017f0 con 0x7fb9940ac290 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 >> v1:172.21.15.119:6789/0 conn(0x7fb9940ac290 legacy=0x7fb994068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb994157a00 0x7fb980003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb980003fe0 0x7fb980004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980004d50 0x7fb9800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb994157f80 con 0x7fb980003fe0 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9940ac7a0 con 0x7fb980004d50 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98000ba90 con 0x7fb994157a00 2023-10-08T01:12:43.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.455+0000 7fb98bfff700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb980003fe0 0x7fb980004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb993fff700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb994157a00 0x7fb980003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb99899c700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980004d50 0x7fb9800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb98bfff700 1 -- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb994157a00 msgr2=0x7fb980003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:43.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb98bfff700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb994157a00 0x7fb980003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb98bfff700 1 -- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980004d50 msgr2=0x7fb9800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:43.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb98bfff700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980004d50 0x7fb9800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb98bfff700 1 -- 172.21.15.119:0/3381533702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb980003570 con 0x7fb980003fe0 2023-10-08T01:12:43.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb99899c700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb980004d50 0x7fb9800051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:43.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb98bfff700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb980003fe0 0x7fb980004810 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009c00 tx=0x7fb97c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cc59fe01215f8b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:43.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00e6b0 con 0x7fb980003fe0 2023-10-08T01:12:43.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb97c00e810 con 0x7fb980003fe0 2023-10-08T01:12:43.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.456+0000 7fb9937fe700 1 -- 172.21.15.119:0/3381533702 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb97c00ebd0 con 0x7fb980003fe0 2023-10-08T01:12:43.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.457+0000 7fb99a3ff700 1 -- 172.21.15.119:0/3381533702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb980003fe0 msgr2=0x7fb980004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:43.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.457+0000 7fb99a3ff700 1 --2- 172.21.15.119:0/3381533702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb980003fe0 0x7fb980004810 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009c00 tx=0x7fb97c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.457+0000 7fb99899c700 1 -- 172.21.15.119:0/3381533702 reap_dead start 2023-10-08T01:12:43.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.457+0000 7fb99a3ff700 1 -- 172.21.15.119:0/3381533702 shutdown_connections 2023-10-08T01:12:43.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.457+0000 7fb99a3ff700 1 -- 172.21.15.119:0/3381533702 >> 172.21.15.119:0/3381533702 conn(0x7fb9940cc540 msgr2=0x7fb9940cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:43.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.457+0000 7fb99a3ff700 1 -- 172.21.15.119:0/3381533702 shutdown_connections 2023-10-08T01:12:43.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.457+0000 7fb99a3ff700 1 -- 172.21.15.119:0/3381533702 wait complete. 2023-10-08T01:12:43.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.458+0000 7fb99a3ff700 1 Processor -- start 2023-10-08T01:12:43.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.458+0000 7fb99a3ff700 1 -- start start 2023-10-08T01:12:43.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.459+0000 7fb99a3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb980003fe0 0x7fb9940685e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.459+0000 7fb99a3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9940a98e0 0x7fb9940a9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.459+0000 7fb99a3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb994155110 0x7fb994155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.459+0000 7fb99a3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb994155ac0 con 0x7fb994155110 2023-10-08T01:12:43.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.459+0000 7fb99a3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9941565b0 con 0x7fb9940a98e0 2023-10-08T01:12:43.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.459+0000 7fb99a3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9940aa3a0 con 0x7fb980003fe0 2023-10-08T01:12:43.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb994155110 0x7fb994155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb994155110 0x7fb994155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37180/0 (socket says 172.21.15.119:37180) 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 -- 172.21.15.119:0/1654090422 learned_addr learned my addr 172.21.15.119:0/1654090422 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb993fff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb980003fe0 0x7fb9940685e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb98bfff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9940a98e0 0x7fb9940a9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 -- 172.21.15.119:0/1654090422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb980003fe0 msgr2=0x7fb9940685e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb980003fe0 0x7fb9940685e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 -- 172.21.15.119:0/1654090422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9940a98e0 msgr2=0x7fb9940a9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9940a98e0 0x7fb9940a9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 -- 172.21.15.119:0/1654090422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9940aa680 con 0x7fb994155110 2023-10-08T01:12:43.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb99899c700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb994155110 0x7fb994155580 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb9940c34d0 tx=0x7fb98c002180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.460+0000 7fb990ff9700 1 -- 172.21.15.119:0/1654090422 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c00eb10 con 0x7fb994155110 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.461+0000 7fb990ff9700 1 -- 172.21.15.119:0/1654090422 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c00ec70 con 0x7fb994155110 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.461+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9940aa960 con 0x7fb994155110 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.461+0000 7fb990ff9700 1 -- 172.21.15.119:0/1654090422 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c013420 con 0x7fb994155110 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.461+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9940c1240 con 0x7fb994155110 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.462+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9940c10a0 con 0x7fb994155110 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.463+0000 7fb990ff9700 1 -- 172.21.15.119:0/1654090422 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb98c013830 con 0x7fb994155110 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.464+0000 7fb990ff9700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb974066f30 0x7fb9740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.464+0000 7fb990ff9700 1 -- 172.21.15.119:0/1654090422 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5615+0+0 (secure 0 0 0) 0x7fb98c085330 con 0x7fb994155110 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.464+0000 7fb993fff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb974066f30 0x7fb9740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.465+0000 7fb993fff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb974066f30 0x7fb9740693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb984006f00 tx=0x7fb9840091a0 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:43.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.468+0000 7fb990ff9700 1 -- 172.21.15.119:0/1654090422 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb98c052680 con 0x7fb994155110 2023-10-08T01:12:43.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:43.660+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb9940c10a0 con 0x7fb994155110 2023-10-08T01:12:44.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.241+0000 7fb990ff9700 1 -- 172.21.15.119:0/1654090422 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb9940c10a0 con 0x7fb994155110 2023-10-08T01:12:44.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.247+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb974066f30 msgr2=0x7fb9740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:44.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.247+0000 7fb99a3ff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb974066f30 0x7fb9740693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb984006f00 tx=0x7fb9840091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:44.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.247+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb994155110 msgr2=0x7fb994155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:44.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.247+0000 7fb99a3ff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb994155110 0x7fb994155580 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb9940c34d0 tx=0x7fb98c002180 comp rx=0 tx=0).stop 2023-10-08T01:12:44.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.250+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 shutdown_connections 2023-10-08T01:12:44.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.250+0000 7fb99a3ff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb974066f30 0x7fb9740693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:44.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.250+0000 7fb99a3ff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb980003fe0 0x7fb9940685e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:44.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.250+0000 7fb99a3ff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb994155110 0x7fb994155580 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:44.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.250+0000 7fb99a3ff700 1 --2- 172.21.15.119:0/1654090422 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9940a98e0 0x7fb9940a9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:44.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.250+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 >> 172.21.15.119:0/1654090422 conn(0x7fb9940cc540 msgr2=0x7fb9940cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:44.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.251+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 shutdown_connections 2023-10-08T01:12:44.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.252+0000 7fb99a3ff700 1 -- 172.21.15.119:0/1654090422 wait complete. 2023-10-08T01:12:44.263 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T01:12:44.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T01:12:44.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.466+0000 7fd76f3ff700 1 Processor -- start 2023-10-08T01:12:44.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.466+0000 7fd76f3ff700 1 -- start start 2023-10-08T01:12:44.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.467+0000 7fd76f3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7681568e0 0x7fd7681616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:44.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.467+0000 7fd76f3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76815b0d0 con 0x7fd7680ccee0 2023-10-08T01:12:44.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.467+0000 7fd76f3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76815a4c0 con 0x7fd7680c83f0 2023-10-08T01:12:44.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.467+0000 7fd76f3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76815a7a0 con 0x7fd7681568e0 2023-10-08T01:12:44.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd76c99a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd7680ccee0 0x7fd7681561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57820/0 (socket says 172.21.15.119:57820) 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd76c99a700 1 -- 172.21.15.119:0/2189181260 learned_addr learned my addr 172.21.15.119:0/2189181260 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd76d99c700 1 --2- 172.21.15.119:0/2189181260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7681568e0 0x7fd7681616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd75ffff700 1 -- 172.21.15.119:0/2189181260 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277100176 0 0) 0x7fd76815b0d0 con 0x7fd7680ccee0 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd75ffff700 1 -- 172.21.15.119:0/2189181260 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd754003570 con 0x7fd7680ccee0 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd75ffff700 1 -- 172.21.15.119:0/2189181260 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2658255106 0 0) 0x7fd76815a4c0 con 0x7fd7680c83f0 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd76d99c700 1 -- 172.21.15.119:0/2189181260 >> v1:172.21.15.184:6789/0 conn(0x7fd7680c83f0 legacy=0x7fd7680c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd76d99c700 1 -- 172.21.15.119:0/2189181260 >> v1:172.21.15.119:6789/0 conn(0x7fd7680ccee0 legacy=0x7fd7681561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.468+0000 7fd76d99c700 1 -- 172.21.15.119:0/2189181260 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76815aa80 con 0x7fd7681568e0 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.469+0000 7fd76d99c700 1 --2- 172.21.15.119:0/2189181260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7681568e0 0x7fd7681616c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd760009e90 tx=0x7fd760009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afcef819bacccdb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.469+0000 7fd75ffff700 1 -- 172.21.15.119:0/2189181260 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76000e070 con 0x7fd7681568e0 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.469+0000 7fd75ffff700 1 -- 172.21.15.119:0/2189181260 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76000b920 con 0x7fd7681568e0 2023-10-08T01:12:44.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.469+0000 7fd75ffff700 1 -- 172.21.15.119:0/2189181260 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7580013f0 con 0x7fd7681568e0 2023-10-08T01:12:44.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.470+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2189181260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7681568e0 msgr2=0x7fd7681616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:44.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.470+0000 7fd76f3ff700 1 --2- 172.21.15.119:0/2189181260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7681568e0 0x7fd7681616c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd760009e90 tx=0x7fd760009780 comp rx=0 tx=0).stop 2023-10-08T01:12:44.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.470+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2189181260 shutdown_connections 2023-10-08T01:12:44.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.470+0000 7fd76f3ff700 1 --2- 172.21.15.119:0/2189181260 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7681568e0 0x7fd7681616c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:44.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.470+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2189181260 >> 172.21.15.119:0/2189181260 conn(0x7fd768068e10 msgr2=0x7fd7680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:44.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.470+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2189181260 shutdown_connections 2023-10-08T01:12:44.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.470+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2189181260 wait complete. 2023-10-08T01:12:44.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.471+0000 7fd76f3ff700 1 Processor -- start 2023-10-08T01:12:44.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.471+0000 7fd76f3ff700 1 -- start start 2023-10-08T01:12:44.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.472+0000 7fd76f3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7680c83f0 0x7fd7680c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:44.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.472+0000 7fd76f3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7680ccee0 0x7fd76814cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:44.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.472+0000 7fd76f3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7681568e0 0x7fd76814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.472+0000 7fd76f3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76814e8f0 con 0x7fd7681568e0 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.472+0000 7fd76f3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680c56c0 con 0x7fd7680ccee0 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.472+0000 7fd76f3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76814e460 con 0x7fd7680c83f0 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7681568e0 0x7fd76814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76c99a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7680ccee0 0x7fd76814cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d19b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7680c83f0 0x7fd7680c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7681568e0 0x7fd76814d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37192/0 (socket says 172.21.15.119:37192) 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 -- 172.21.15.119:0/2182684244 learned_addr learned my addr 172.21.15.119:0/2182684244 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 -- 172.21.15.119:0/2182684244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7680c83f0 msgr2=0x7fd7680c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7680c83f0 0x7fd7680c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:44.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 -- 172.21.15.119:0/2182684244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7680ccee0 msgr2=0x7fd76814cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:44.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7680ccee0 0x7fd76814cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:44.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 -- 172.21.15.119:0/2182684244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd760009a20 con 0x7fd7681568e0 2023-10-08T01:12:44.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d19b700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7680c83f0 0x7fd7680c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:44.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.473+0000 7fd76d99c700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7681568e0 0x7fd76814d1b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd76806d270 tx=0x7fd760017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:44.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.474+0000 7fd75d7fa700 1 -- 172.21.15.119:0/2182684244 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760012c20 con 0x7fd7681568e0 2023-10-08T01:12:44.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.474+0000 7fd75d7fa700 1 -- 172.21.15.119:0/2182684244 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd760012dc0 con 0x7fd7681568e0 2023-10-08T01:12:44.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.474+0000 7fd75d7fa700 1 -- 172.21.15.119:0/2182684244 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7600266b0 con 0x7fd7681568e0 2023-10-08T01:12:44.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.474+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76814d850 con 0x7fd7681568e0 2023-10-08T01:12:44.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.474+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76814de10 con 0x7fd7681568e0 2023-10-08T01:12:44.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.475+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76814e5f0 con 0x7fd7681568e0 2023-10-08T01:12:44.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.476+0000 7fd75d7fa700 1 -- 172.21.15.119:0/2182684244 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd76001d440 con 0x7fd7681568e0 2023-10-08T01:12:44.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.477+0000 7fd75d7fa700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd74c066f30 0x7fd74c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.477+0000 7fd75d7fa700 1 -- 172.21.15.119:0/2182684244 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fd760086940 con 0x7fd7681568e0 2023-10-08T01:12:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.477+0000 7fd76d19b700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd74c066f30 0x7fd74c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:44.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.478+0000 7fd76d19b700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd74c066f30 0x7fd74c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd764001120 tx=0x7fd764007f90 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:44.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.481+0000 7fd75d7fa700 1 -- 172.21.15.119:0/2182684244 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd760053b10 con 0x7fd7681568e0 2023-10-08T01:12:44.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:44.713+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd76804c3c0 con 0x7fd7681568e0 2023-10-08T01:12:45.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.243+0000 7fd75d7fa700 1 -- 172.21.15.119:0/2182684244 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd760057130 con 0x7fd7681568e0 2023-10-08T01:12:45.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.256+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd74c066f30 msgr2=0x7fd74c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.256+0000 7fd76f3ff700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd74c066f30 0x7fd74c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd764001120 tx=0x7fd764007f90 comp rx=0 tx=0).stop 2023-10-08T01:12:45.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.256+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7681568e0 msgr2=0x7fd76814d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.256+0000 7fd76f3ff700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7681568e0 0x7fd76814d1b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd76806d270 tx=0x7fd760017000 comp rx=0 tx=0).stop 2023-10-08T01:12:45.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.261+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 shutdown_connections 2023-10-08T01:12:45.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.261+0000 7fd76f3ff700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd74c066f30 0x7fd74c0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.261+0000 7fd76f3ff700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7680c83f0 0x7fd7680c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.261+0000 7fd76f3ff700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7681568e0 0x7fd76814d1b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.261+0000 7fd76f3ff700 1 --2- 172.21.15.119:0/2182684244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd7680ccee0 0x7fd76814cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.261+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 >> 172.21.15.119:0/2182684244 conn(0x7fd768068e10 msgr2=0x7fd768150fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:45.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.261+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 shutdown_connections 2023-10-08T01:12:45.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.264+0000 7fd76f3ff700 1 -- 172.21.15.119:0/2182684244 wait complete. 2023-10-08T01:12:45.270 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 12 and data pool 13 2023-10-08T01:12:45.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:45.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.472+0000 7f844339d700 1 Processor -- start 2023-10-08T01:12:45.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.472+0000 7f844339d700 1 -- start start 2023-10-08T01:12:45.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.473+0000 7f844339d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f843c0c8830 0x7f843c164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.473+0000 7f844339d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c0b1410 con 0x7f843c0c6c60 2023-10-08T01:12:45.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.473+0000 7f844339d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c063230 con 0x7f843c1595a0 2023-10-08T01:12:45.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.473+0000 7f844339d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0b2b90 con 0x7f843c0c8830 2023-10-08T01:12:45.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441b9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f843c0c6c60 0x7f843c0c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45362/0 (socket says 172.21.15.119:45362) 2023-10-08T01:12:45.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441b9a700 1 -- 172.21.15.119:0/4093282604 learned_addr learned my addr 172.21.15.119:0/4093282604 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:45.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870354758 0 0) 0x7f843c0b1410 con 0x7f843c0c6c60 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8442b9c700 1 --2- 172.21.15.119:0/4093282604 >> v2:172.21.15.184:3300/0 conn(0x7f843c0c8830 0x7f843c164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8428003570 con 0x7f843c0c6c60 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3379324059 0 0) 0x7f8428003570 con 0x7f843c0c6c60 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 >> v2:172.21.15.184:3300/0 conn(0x7f843c0c8830 msgr2=0x7f843c164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 --2- 172.21.15.119:0/4093282604 >> v2:172.21.15.184:3300/0 conn(0x7f843c0c8830 0x7f843c164800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 >> v1:172.21.15.184:6789/0 conn(0x7f843c1595a0 legacy=0x7f843c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0634f0 con 0x7f843c0c6c60 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f842c001470 con 0x7f843c0c6c60 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 >> v1:172.21.15.119:6789/0 conn(0x7f843c0c6c60 legacy=0x7f843c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428003a60 0x7f8428003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8428004810 0x7f8428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84280051c0 0x7f842800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0b2b90 con 0x7f8428003a60 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842800c160 con 0x7f8428004810 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.474+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f842800c320 con 0x7f84280051c0 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f844239b700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428003a60 0x7f8428003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f8441b9a700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8428004810 0x7f8428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f8442b9c700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84280051c0 0x7f842800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f844239b700 1 -- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84280051c0 msgr2=0x7f842800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f844239b700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f84280051c0 0x7f842800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f844239b700 1 -- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8428004810 msgr2=0x7f8428004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:45.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f844239b700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8428004810 0x7f8428004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f844239b700 1 -- 172.21.15.119:0/4093282604 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428003570 con 0x7f8428003a60 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f8441b9a700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8428004810 0x7f8428004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f8442b9c700 1 -- 172.21.15.119:0/4093282604 reap_dead start 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.475+0000 7f844239b700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428003a60 0x7f8428003e70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f84380018e0 tx=0x7f8438007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7eec0a457aaced server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438012420 con 0x7f8428003a60 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8438012580 con 0x7f8428003a60 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f8441399700 1 -- 172.21.15.119:0/4093282604 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8438012920 con 0x7f8428003a60 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f844339d700 1 -- 172.21.15.119:0/4093282604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428003a60 msgr2=0x7f8428003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f844339d700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428003a60 0x7f8428003e70 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f84380018e0 tx=0x7f8438007d30 comp rx=0 tx=0).stop 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f844339d700 1 -- 172.21.15.119:0/4093282604 shutdown_connections 2023-10-08T01:12:45.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f844339d700 1 --2- 172.21.15.119:0/4093282604 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8428003a60 0x7f8428003e70 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f844339d700 1 -- 172.21.15.119:0/4093282604 >> 172.21.15.119:0/4093282604 conn(0x7f843c06ddb0 msgr2=0x7f843c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:45.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f844339d700 1 -- 172.21.15.119:0/4093282604 shutdown_connections 2023-10-08T01:12:45.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.476+0000 7f844339d700 1 -- 172.21.15.119:0/4093282604 wait complete. 2023-10-08T01:12:45.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.478+0000 7f844339d700 1 Processor -- start 2023-10-08T01:12:45.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.478+0000 7f844339d700 1 -- start start 2023-10-08T01:12:45.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f844339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8428003a60 0x7f843c14e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f844339d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843c148f40 0x7f843c153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f844339d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f843c157470 0x7f843c157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f844339d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c157e50 con 0x7f843c148f40 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f844339d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c152040 con 0x7f843c157470 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f844339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c153da0 con 0x7f8428003a60 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843c148f40 0x7f843c153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843c148f40 0x7f843c153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47184/0 (socket says 172.21.15.119:47184) 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 -- 172.21.15.119:0/1908815941 learned_addr learned my addr 172.21.15.119:0/1908815941 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f844239b700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8428003a60 0x7f843c14e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8442b9c700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f843c157470 0x7f843c157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 -- 172.21.15.119:0/1908815941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8428003a60 msgr2=0x7f843c14e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8428003a60 0x7f843c14e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 -- 172.21.15.119:0/1908815941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f843c157470 msgr2=0x7f843c157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f843c157470 0x7f843c157910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 -- 172.21.15.119:0/1908815941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c154000 con 0x7f843c148f40 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.479+0000 7f8441b9a700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843c148f40 0x7f843c153860 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f842c00ebc0 tx=0x7f842c001430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.480+0000 7f84327fc700 1 -- 172.21.15.119:0/1908815941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c00c680 con 0x7f843c148f40 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.480+0000 7f84327fc700 1 -- 172.21.15.119:0/1908815941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c00c7e0 con 0x7f843c148f40 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.480+0000 7f84327fc700 1 -- 172.21.15.119:0/1908815941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c00cba0 con 0x7f843c148f40 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.480+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c152250 con 0x7f843c148f40 2023-10-08T01:12:45.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.480+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c152810 con 0x7f843c148f40 2023-10-08T01:12:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.481+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c152a30 con 0x7f843c148f40 2023-10-08T01:12:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.482+0000 7f84327fc700 1 -- 172.21.15.119:0/1908815941 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f842c01d460 con 0x7f843c148f40 2023-10-08T01:12:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.483+0000 7f84327fc700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8420066f30 0x7f84200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.483+0000 7f84327fc700 1 -- 172.21.15.119:0/1908815941 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f842c084640 con 0x7f843c148f40 2023-10-08T01:12:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.483+0000 7f844239b700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8420066f30 0x7f84200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.483+0000 7f844239b700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8420066f30 0x7f84200693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84380016c0 tx=0x7f8438015000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:45.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.487+0000 7f84327fc700 1 -- 172.21.15.119:0/1908815941 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f842c051720 con 0x7f843c148f40 2023-10-08T01:12:45.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.678+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f843c04c3c0 con 0x7f843c148f40 2023-10-08T01:12:45.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.679+0000 7f84327fc700 1 -- 172.21.15.119:0/1908815941 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16022 (secure 0 0 0) 0x7f842c054d40 con 0x7f843c148f40 2023-10-08T01:12:45.680 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:45.680 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":103,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:44.714959+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:42.395485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:43.700121+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:45.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8420066f30 msgr2=0x7f84200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8420066f30 0x7f84200693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f84380016c0 tx=0x7f8438015000 comp rx=0 tx=0).stop 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843c148f40 msgr2=0x7f843c153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843c148f40 0x7f843c153860 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f842c00ebc0 tx=0x7f842c001430 comp rx=0 tx=0).stop 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 shutdown_connections 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8420066f30 0x7f84200693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8428003a60 0x7f843c14e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f843c157470 0x7f843c157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 --2- 172.21.15.119:0/1908815941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f843c148f40 0x7f843c153860 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 >> 172.21.15.119:0/1908815941 conn(0x7f843c06ddb0 msgr2=0x7f843c14aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 shutdown_connections 2023-10-08T01:12:45.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.681+0000 7f844339d700 1 -- 172.21.15.119:0/1908815941 wait complete. 2023-10-08T01:12:45.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:45.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.878+0000 7fdd87fff700 1 Processor -- start 2023-10-08T01:12:45.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.878+0000 7fdd87fff700 1 -- start start 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd87fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd800b0480 0x7fdd80155b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd800ae520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd87fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd800aeec0 con 0x7fdd80155740 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd87fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd801567e0 con 0x7fdd800b0480 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd87fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800b2370 con 0x7fdd800b1f40 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd8559a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd800b0480 0x7fdd80155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd8559a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd800b0480 0x7fdd80155b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47200/0 (socket says 172.21.15.119:47200) 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd8559a700 1 -- 172.21.15.119:0/3727065808 learned_addr learned my addr 172.21.15.119:0/3727065808 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.880+0000 7fdd8659c700 1 --2- 172.21.15.119:0/3727065808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd800ae520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd8559a700 1 -- 172.21.15.119:0/3727065808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 msgr2=0x7fdd800ae520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd8559a700 1 --2- 172.21.15.119:0/3727065808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd800ae520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd8559a700 1 -- 172.21.15.119:0/3727065808 >> v1:172.21.15.184:6789/0 conn(0x7fdd80155740 legacy=0x7fdd800afd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd8559a700 1 -- 172.21.15.119:0/3727065808 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd800cc2a0 con 0x7fdd800b0480 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd8659c700 1 --2- 172.21.15.119:0/3727065808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd800ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd8559a700 1 --2- 172.21.15.119:0/3727065808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800b0480 0x7fdd80155b10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000bc50 tx=0x7fdd7000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840e60130753d17e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd84d99700 1 -- 172.21.15.119:0/3727065808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000be60 con 0x7fdd800b0480 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd84d99700 1 -- 172.21.15.119:0/3727065808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7000de00 con 0x7fdd800b0480 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd84d99700 1 -- 172.21.15.119:0/3727065808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700196b0 con 0x7fdd800b0480 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd87fff700 1 -- 172.21.15.119:0/3727065808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800b0480 msgr2=0x7fdd80155b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.881+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3727065808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800b0480 0x7fdd80155b10 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000bc50 tx=0x7fdd7000b4f0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.882+0000 7fdd87fff700 1 -- 172.21.15.119:0/3727065808 shutdown_connections 2023-10-08T01:12:45.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.882+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3727065808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd800ae520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.882+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3727065808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd800b0480 0x7fdd80155b10 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.882+0000 7fdd87fff700 1 -- 172.21.15.119:0/3727065808 >> 172.21.15.119:0/3727065808 conn(0x7fdd8006a320 msgr2=0x7fdd800cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:45.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.882+0000 7fdd87fff700 1 -- 172.21.15.119:0/3727065808 shutdown_connections 2023-10-08T01:12:45.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.882+0000 7fdd87fff700 1 -- 172.21.15.119:0/3727065808 wait complete. 2023-10-08T01:12:45.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.883+0000 7fdd87fff700 1 Processor -- start 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.883+0000 7fdd87fff700 1 -- start start 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.884+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800b0480 0x7fdd80151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.884+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd8014b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.884+0000 7fdd87fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80155740 0x7fdd800ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.884+0000 7fdd87fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8014fff0 con 0x7fdd80155740 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.884+0000 7fdd87fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801502d0 con 0x7fdd800b0480 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.884+0000 7fdd87fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801505b0 con 0x7fdd800b1f40 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.884+0000 7fdd8659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80155740 0x7fdd800ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80155740 0x7fdd800ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47204/0 (socket says 172.21.15.119:47204) 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8659c700 1 -- 172.21.15.119:0/3020853639 learned_addr learned my addr 172.21.15.119:0/3020853639 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:45.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8559a700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd8014b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd85d9b700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800b0480 0x7fdd80151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8659c700 1 -- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 msgr2=0x7fdd8014b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8659c700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd8014b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8659c700 1 -- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800b0480 msgr2=0x7fdd80151090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8659c700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800b0480 0x7fdd80151090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8659c700 1 -- 172.21.15.119:0/3020853639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7000b790 con 0x7fdd80155740 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd85d9b700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800b0480 0x7fdd80151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd8659c700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80155740 0x7fdd800ace20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800d180 tx=0x7fdd78009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd75ffb700 1 -- 172.21.15.119:0/3020853639 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7800de20 con 0x7fdd80155740 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd75ffb700 1 -- 172.21.15.119:0/3020853639 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7800f040 con 0x7fdd80155740 2023-10-08T01:12:45.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.885+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd80150890 con 0x7fdd80155740 2023-10-08T01:12:45.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.886+0000 7fdd75ffb700 1 -- 172.21.15.119:0/3020853639 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd78015690 con 0x7fdd80155740 2023-10-08T01:12:45.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.886+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8014bed0 con 0x7fdd80155740 2023-10-08T01:12:45.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.886+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8004c3c0 con 0x7fdd80155740 2023-10-08T01:12:45.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.887+0000 7fdd75ffb700 1 -- 172.21.15.119:0/3020853639 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdd7801e3b0 con 0x7fdd80155740 2023-10-08T01:12:45.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.888+0000 7fdd75ffb700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:45.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.888+0000 7fdd75ffb700 1 -- 172.21.15.119:0/3020853639 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fdd78084cd0 con 0x7fdd80155740 2023-10-08T01:12:45.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.889+0000 7fdd85d9b700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:45.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.889+0000 7fdd85d9b700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c007fd0 tx=0x7fdd7c009040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:45.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:45.892+0000 7fdd75ffb700 1 -- 172.21.15.119:0/3020853639 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd78051db0 con 0x7fdd80155740 2023-10-08T01:12:46.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.088+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd800aebe0 con 0x7fdd80155740 2023-10-08T01:12:46.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.089+0000 7fdd75ffb700 1 -- 172.21.15.119:0/3020853639 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16022 (secure 0 0 0) 0x7fdd7801a090 con 0x7fdd80155740 2023-10-08T01:12:46.090 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:46.090 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":103,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:44.714959+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:42.395485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:43.700121+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:46.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 msgr2=0x7fdd640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:46.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c007fd0 tx=0x7fdd7c009040 comp rx=0 tx=0).stop 2023-10-08T01:12:46.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80155740 msgr2=0x7fdd800ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:46.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80155740 0x7fdd800ace20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800d180 tx=0x7fdd78009a50 comp rx=0 tx=0).stop 2023-10-08T01:12:46.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 shutdown_connections 2023-10-08T01:12:46.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd64066f30 0x7fdd640693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:46.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd80155740 0x7fdd800ace20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:46.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd800b0480 0x7fdd80151090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:46.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 --2- 172.21.15.119:0/3020853639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd800b1f40 0x7fdd8014b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:46.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 >> 172.21.15.119:0/3020853639 conn(0x7fdd8006a320 msgr2=0x7fdd801542c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:46.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 shutdown_connections 2023-10-08T01:12:46.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.091+0000 7fdd87fff700 1 -- 172.21.15.119:0/3020853639 wait complete. 2023-10-08T01:12:46.110 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T01:12:46.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.297+0000 7fe2f39ff700 1 Processor -- start 2023-10-08T01:12:46.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.297+0000 7fe2f39ff700 1 -- start start 2023-10-08T01:12:46.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.298+0000 7fe2f39ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2ec15c910 0x7fe2ec15cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:46.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.298+0000 7fe2f39ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec15c480 con 0x7fe2ec0b5ea0 2023-10-08T01:12:46.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.298+0000 7fe2f39ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec15d290 con 0x7fe2ec159c10 2023-10-08T01:12:46.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.298+0000 7fe2f39ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ec0b2a40 con 0x7fe2ec15c910 2023-10-08T01:12:46.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.299+0000 7fe2f1f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2ec15c910 0x7fe2ec15cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:46.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.299+0000 7fe2f1f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe2ec15c910 0x7fe2ec15cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47210/0 (socket says 172.21.15.119:47210) 2023-10-08T01:12:46.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.299+0000 7fe2f1f9c700 1 -- 172.21.15.119:0/2171521385 learned_addr learned my addr 172.21.15.119:0/2171521385 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:46.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.299+0000 7fe2f0f9a700 1 --1- 172.21.15.119:0/2171521385 >> v1:172.21.15.119:6789/0 conn(0x7fe2ec0b5ea0 0x7fe2ec068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45368/0 (socket says 172.21.15.119:45368) 2023-10-08T01:12:46.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.299+0000 7fe2f1f9c700 1 -- 172.21.15.119:0/2171521385 >> v1:172.21.15.184:6789/0 conn(0x7fe2ec159c10 legacy=0x7fe2ec15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:46.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.299+0000 7fe2f1f9c700 1 -- 172.21.15.119:0/2171521385 >> v1:172.21.15.119:6789/0 conn(0x7fe2ec0b5ea0 legacy=0x7fe2ec068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:46.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.299+0000 7fe2f1f9c700 1 -- 172.21.15.119:0/2171521385 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec15b870 con 0x7fe2ec15c910 2023-10-08T01:12:46.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.299+0000 7fe2f1f9c700 1 --2- 172.21.15.119:0/2171521385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec15cd50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009f20 tx=0x7fe2e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47438afaa9a5ab9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:46.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2e3fff700 1 -- 172.21.15.119:0/2171521385 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e400e070 con 0x7fe2ec15c910 2023-10-08T01:12:46.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2e3fff700 1 -- 172.21.15.119:0/2171521385 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e400ba80 con 0x7fe2ec15c910 2023-10-08T01:12:46.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2e3fff700 1 -- 172.21.15.119:0/2171521385 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e40092e0 con 0x7fe2ec15c910 2023-10-08T01:12:46.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2f39ff700 1 -- 172.21.15.119:0/2171521385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 msgr2=0x7fe2ec15cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:46.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2f39ff700 1 --2- 172.21.15.119:0/2171521385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec15cd50 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4009f20 tx=0x7fe2e4009840 comp rx=0 tx=0).stop 2023-10-08T01:12:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2f39ff700 1 -- 172.21.15.119:0/2171521385 shutdown_connections 2023-10-08T01:12:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2f39ff700 1 --2- 172.21.15.119:0/2171521385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec15cd50 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2f39ff700 1 -- 172.21.15.119:0/2171521385 >> 172.21.15.119:0/2171521385 conn(0x7fe2ec099a20 msgr2=0x7fe2ec0af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2f39ff700 1 -- 172.21.15.119:0/2171521385 shutdown_connections 2023-10-08T01:12:46.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.300+0000 7fe2f39ff700 1 -- 172.21.15.119:0/2171521385 wait complete. 2023-10-08T01:12:46.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.302+0000 7fe2f39ff700 1 Processor -- start 2023-10-08T01:12:46.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.302+0000 7fe2f39ff700 1 -- start start 2023-10-08T01:12:46.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f39ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2ec0b5ea0 0x7fe2ec16c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:46.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f39ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2ec159c10 0x7fe2ec155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f39ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f39ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec16caa0 con 0x7fe2ec15c910 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f39ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec15e260 con 0x7fe2ec159c10 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f39ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec15e540 con 0x7fe2ec0b5ea0 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f1f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f1f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec155f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47216/0 (socket says 172.21.15.119:47216) 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f1f9c700 1 -- 172.21.15.119:0/3585429706 learned_addr learned my addr 172.21.15.119:0/3585429706 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f1f9c700 1 -- 172.21.15.119:0/3585429706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2ec0b5ea0 msgr2=0x7fe2ec16c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f1f9c700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2ec0b5ea0 0x7fe2ec16c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f1f9c700 1 -- 172.21.15.119:0/3585429706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2ec159c10 msgr2=0x7fe2ec155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:46.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f1f9c700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2ec159c10 0x7fe2ec155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:46.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.303+0000 7fe2f1f9c700 1 -- 172.21.15.119:0/3585429706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4009ae0 con 0x7fe2ec15c910 2023-10-08T01:12:46.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.304+0000 7fe2f1f9c700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec155f50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec063db0 tx=0x7fe2e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:46.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.304+0000 7fe2e17fa700 1 -- 172.21.15.119:0/3585429706 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e4012ea0 con 0x7fe2ec15c910 2023-10-08T01:12:46.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.304+0000 7fe2e17fa700 1 -- 172.21.15.119:0/3585429706 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e40032f0 con 0x7fe2ec15c910 2023-10-08T01:12:46.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.304+0000 7fe2e17fa700 1 -- 172.21.15.119:0/3585429706 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e401d630 con 0x7fe2ec15c910 2023-10-08T01:12:46.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.304+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec15e820 con 0x7fe2ec15c910 2023-10-08T01:12:46.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.304+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec15ede0 con 0x7fe2ec15c910 2023-10-08T01:12:46.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.305+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec169620 con 0x7fe2ec15c910 2023-10-08T01:12:46.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.306+0000 7fe2e17fa700 1 -- 172.21.15.119:0/3585429706 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2e401da60 con 0x7fe2ec15c910 2023-10-08T01:12:46.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.307+0000 7fe2e17fa700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe2d0066f30 0x7fe2d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:46.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.307+0000 7fe2e17fa700 1 -- 172.21.15.119:0/3585429706 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe2e4085ff0 con 0x7fe2ec15c910 2023-10-08T01:12:46.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.307+0000 7fe2f179b700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe2d0066f30 0x7fe2d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:46.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.308+0000 7fe2f179b700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe2d0066f30 0x7fe2d00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8007fd0 tx=0x7fe2e8007b20 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:46.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.311+0000 7fe2e17fa700 1 -- 172.21.15.119:0/3585429706 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2e40530d0 con 0x7fe2ec15c910 2023-10-08T01:12:46.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:46.544+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe2ec15f000 con 0x7fe2ec15c910 2023-10-08T01:12:47.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.271+0000 7fe2e17fa700 1 -- 172.21.15.119:0/3585429706 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v47) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe2e40566f0 con 0x7fe2ec15c910 2023-10-08T01:12:47.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe2d0066f30 msgr2=0x7fe2d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:47.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe2d0066f30 0x7fe2d00693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe2e8007fd0 tx=0x7fe2e8007b20 comp rx=0 tx=0).stop 2023-10-08T01:12:47.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 msgr2=0x7fe2ec155f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:47.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec155f50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec063db0 tx=0x7fe2e4017000 comp rx=0 tx=0).stop 2023-10-08T01:12:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 shutdown_connections 2023-10-08T01:12:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe2d0066f30 0x7fe2d00693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2ec0b5ea0 0x7fe2ec16c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2ec15c910 0x7fe2ec155f50 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 --2- 172.21.15.119:0/3585429706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2ec159c10 0x7fe2ec155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 >> 172.21.15.119:0/3585429706 conn(0x7fe2ec099a20 msgr2=0x7fe2ec09b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 shutdown_connections 2023-10-08T01:12:47.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.274+0000 7fe2f39ff700 1 -- 172.21.15.119:0/3585429706 wait complete. 2023-10-08T01:12:47.293 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T01:12:47.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.477+0000 7fa27539d700 1 Processor -- start 2023-10-08T01:12:47.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.477+0000 7fa27539d700 1 -- start start 2023-10-08T01:12:47.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.478+0000 7fa27539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa27006a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:47.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.478+0000 7fa27539d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa27006dc00 0x7fa27006a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:47.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.478+0000 7fa27539d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2700c6820 con 0x7fa270140bb0 2023-10-08T01:12:47.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.478+0000 7fa27539d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2700c6de0 con 0x7fa27006dc00 2023-10-08T01:12:47.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.478+0000 7fa27539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa270141270 con 0x7fa2701407e0 2023-10-08T01:12:47.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa274b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa27006dc00 0x7fa27006a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:47.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa274b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa27006dc00 0x7fa27006a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47222/0 (socket says 172.21.15.119:47222) 2023-10-08T01:12:47.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa274b9c700 1 -- 172.21.15.119:0/2947913844 learned_addr learned my addr 172.21.15.119:0/2947913844 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:47.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa26ffff700 1 --2- 172.21.15.119:0/2947913844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa27006a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:47.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa274b9c700 1 -- 172.21.15.119:0/2947913844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 msgr2=0x7fa27006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:47.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa274b9c700 1 --2- 172.21.15.119:0/2947913844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa27006a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa274b9c700 1 -- 172.21.15.119:0/2947913844 >> v1:172.21.15.184:6789/0 conn(0x7fa270140bb0 legacy=0x7fa27006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:47.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa274b9c700 1 -- 172.21.15.119:0/2947913844 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27006acf0 con 0x7fa27006dc00 2023-10-08T01:12:47.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa274b9c700 1 --2- 172.21.15.119:0/2947913844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27006a6c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa268009f20 tx=0x7fa268009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b5f2fb95f0a1d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:47.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.479+0000 7fa26f7fe700 1 -- 172.21.15.119:0/2947913844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26800e070 con 0x7fa27006dc00 2023-10-08T01:12:47.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa26f7fe700 1 -- 172.21.15.119:0/2947913844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26800b9d0 con 0x7fa27006dc00 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa26f7fe700 1 -- 172.21.15.119:0/2947913844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2680092e0 con 0x7fa27006dc00 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa27539d700 1 -- 172.21.15.119:0/2947913844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 msgr2=0x7fa27006a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa27539d700 1 --2- 172.21.15.119:0/2947913844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27006a6c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fa268009f20 tx=0x7fa268009840 comp rx=0 tx=0).stop 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa27539d700 1 -- 172.21.15.119:0/2947913844 shutdown_connections 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa27539d700 1 --2- 172.21.15.119:0/2947913844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27006a6c0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa27539d700 1 --2- 172.21.15.119:0/2947913844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa27006a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa27539d700 1 -- 172.21.15.119:0/2947913844 >> 172.21.15.119:0/2947913844 conn(0x7fa2700cc1e0 msgr2=0x7fa2700cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa27539d700 1 -- 172.21.15.119:0/2947913844 shutdown_connections 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.480+0000 7fa27539d700 1 -- 172.21.15.119:0/2947913844 wait complete. 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.482+0000 7fa27539d700 1 Processor -- start 2023-10-08T01:12:47.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.482+0000 7fa27539d700 1 -- start start 2023-10-08T01:12:47.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa27539d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27013f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:47.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa27539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa2700ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:47.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa27539d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa270140bb0 0x7fa2700ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:47.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa27539d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700afd30 con 0x7fa27006dc00 2023-10-08T01:12:47.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa27539d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700b0010 con 0x7fa270140bb0 2023-10-08T01:12:47.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa27539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2700b02f0 con 0x7fa2701407e0 2023-10-08T01:12:47.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27013f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:47.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27013f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47224/0 (socket says 172.21.15.119:47224) 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 -- 172.21.15.119:0/687934378 learned_addr learned my addr 172.21.15.119:0/687934378 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa274b9c700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa270140bb0 0x7fa2700ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa267fff700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa2700ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 -- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 msgr2=0x7fa2700ac1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa2700ac1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 -- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa270140bb0 msgr2=0x7fa2700ac6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa270140bb0 0x7fa2700ac6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 -- 172.21.15.119:0/687934378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa268009ae0 con 0x7fa27006dc00 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa267fff700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa2700ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.483+0000 7fa26ffff700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27013f570 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa26000ec80 tx=0x7fa2600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.484+0000 7fa26cff9700 1 -- 172.21.15.119:0/687934378 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26000c9d0 con 0x7fa27006dc00 2023-10-08T01:12:47.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.484+0000 7fa26cff9700 1 -- 172.21.15.119:0/687934378 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26000cb30 con 0x7fa27006dc00 2023-10-08T01:12:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.484+0000 7fa26cff9700 1 -- 172.21.15.119:0/687934378 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa260013470 con 0x7fa27006dc00 2023-10-08T01:12:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.484+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2700b05d0 con 0x7fa27006dc00 2023-10-08T01:12:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.484+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2700af070 con 0x7fa27006dc00 2023-10-08T01:12:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.485+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2700a98d0 con 0x7fa27006dc00 2023-10-08T01:12:47.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.486+0000 7fa26cff9700 1 -- 172.21.15.119:0/687934378 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa260013820 con 0x7fa27006dc00 2023-10-08T01:12:47.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.486+0000 7fa26cff9700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa250066f30 0x7fa2500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:47.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.486+0000 7fa26cff9700 1 -- 172.21.15.119:0/687934378 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fa260084a50 con 0x7fa27006dc00 2023-10-08T01:12:47.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.487+0000 7fa267fff700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa250066f30 0x7fa2500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:47.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.487+0000 7fa267fff700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa250066f30 0x7fa2500693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa27016ae40 tx=0x7fa258009040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:47.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.490+0000 7fa26cff9700 1 -- 172.21.15.119:0/687934378 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa260051b30 con 0x7fa27006dc00 2023-10-08T01:12:47.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:12:47.509 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:12:47.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:47.735+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa2700b0940 con 0x7fa27006dc00 2023-10-08T01:12:48.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.274+0000 7fa26cff9700 1 -- 172.21.15.119:0/687934378 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v48) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa260055150 con 0x7fa27006dc00 2023-10-08T01:12:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa250066f30 msgr2=0x7fa2500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa250066f30 0x7fa2500693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa27016ae40 tx=0x7fa258009040 comp rx=0 tx=0).stop 2023-10-08T01:12:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 msgr2=0x7fa27013f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27013f570 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fa26000ec80 tx=0x7fa2600091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 shutdown_connections 2023-10-08T01:12:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa250066f30 0x7fa2500693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa270140bb0 0x7fa2700ac6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa27006dc00 0x7fa27013f570 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 --2- 172.21.15.119:0/687934378 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2701407e0 0x7fa2700ac1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.277+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 >> 172.21.15.119:0/687934378 conn(0x7fa2700cc1e0 msgr2=0x7fa2700c0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.278+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 shutdown_connections 2023-10-08T01:12:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.278+0000 7fa27539d700 1 -- 172.21.15.119:0/687934378 wait complete. 2023-10-08T01:12:48.298 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:48.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.479+0000 7f36a9dff700 1 Processor -- start 2023-10-08T01:12:48.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.480+0000 7f36a9dff700 1 -- start start 2023-10-08T01:12:48.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f36a9dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36a406bcd0 0x7f36a40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f36a9dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36a406c0a0 0x7f36a415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f36a9dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a415df50 0x7f36a415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f36a9dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a415d7e0 con 0x7f36a406c0a0 2023-10-08T01:12:48.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f36a9dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a415dac0 con 0x7f36a406bcd0 2023-10-08T01:12:48.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f36a9dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a415aa30 con 0x7f36a415df50 2023-10-08T01:12:48.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f369bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36a406c0a0 0x7f36a415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f369bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36a406c0a0 0x7f36a415a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47228/0 (socket says 172.21.15.119:47228) 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f369bfff700 1 -- 172.21.15.119:0/3773709436 learned_addr learned my addr 172.21.15.119:0/3773709436 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f36a37fe700 1 --2- 172.21.15.119:0/3773709436 >> v2:172.21.15.184:3300/0 conn(0x7f36a406bcd0 0x7f36a40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.481+0000 7f36a3fff700 1 --2- 172.21.15.119:0/3773709436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a415df50 0x7f36a415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f369bfff700 1 -- 172.21.15.119:0/3773709436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a415df50 msgr2=0x7f36a415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f369bfff700 1 --2- 172.21.15.119:0/3773709436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a415df50 0x7f36a415e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f369bfff700 1 -- 172.21.15.119:0/3773709436 >> v2:172.21.15.184:3300/0 conn(0x7f36a406bcd0 msgr2=0x7f36a40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f369bfff700 1 --2- 172.21.15.119:0/3773709436 >> v2:172.21.15.184:3300/0 conn(0x7f36a406bcd0 0x7f36a40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f369bfff700 1 -- 172.21.15.119:0/3773709436 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a415ad10 con 0x7f36a406c0a0 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f36a37fe700 1 --2- 172.21.15.119:0/3773709436 >> v2:172.21.15.184:3300/0 conn(0x7f36a406bcd0 0x7f36a40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:48.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f369bfff700 1 --2- 172.21.15.119:0/3773709436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406c0a0 0x7f36a415a1e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f36a40691a0 tx=0x7f368c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a160e7c33c9e7a17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f36a2ffd700 1 -- 172.21.15.119:0/3773709436 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c00b640 con 0x7f36a406c0a0 2023-10-08T01:12:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f36a2ffd700 1 -- 172.21.15.119:0/3773709436 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c00b7a0 con 0x7f36a406c0a0 2023-10-08T01:12:48.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.482+0000 7f36a2ffd700 1 -- 172.21.15.119:0/3773709436 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c00bb40 con 0x7f36a406c0a0 2023-10-08T01:12:48.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 -- 172.21.15.119:0/3773709436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406c0a0 msgr2=0x7f36a415a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 --2- 172.21.15.119:0/3773709436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406c0a0 0x7f36a415a1e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f36a40691a0 tx=0x7f368c009250 comp rx=0 tx=0).stop 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 -- 172.21.15.119:0/3773709436 shutdown_connections 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 --2- 172.21.15.119:0/3773709436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a415df50 0x7f36a415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 --2- 172.21.15.119:0/3773709436 >> v2:172.21.15.184:3300/0 conn(0x7f36a406bcd0 0x7f36a40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 --2- 172.21.15.119:0/3773709436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406c0a0 0x7f36a415a1e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 -- 172.21.15.119:0/3773709436 >> 172.21.15.119:0/3773709436 conn(0x7f36a40ce600 msgr2=0x7f36a40cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 -- 172.21.15.119:0/3773709436 shutdown_connections 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.483+0000 7f36a9dff700 1 -- 172.21.15.119:0/3773709436 wait complete. 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.484+0000 7f36a9dff700 1 Processor -- start 2023-10-08T01:12:48.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.484+0000 7f36a9dff700 1 -- start start 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.485+0000 7f36a9dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406bcd0 0x7f36a40c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.485+0000 7f36a9dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a406c0a0 0x7f36a40c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.485+0000 7f36a9dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a415df50 0x7f36a40c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.485+0000 7f36a9dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a416e130 con 0x7f36a406bcd0 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.485+0000 7f36a9dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a4169d70 con 0x7f36a415df50 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.485+0000 7f36a9dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a416a050 con 0x7f36a406c0a0 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406bcd0 0x7f36a40c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406bcd0 0x7f36a40c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47234/0 (socket says 172.21.15.119:47234) 2023-10-08T01:12:48.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 -- 172.21.15.119:0/1986717209 learned_addr learned my addr 172.21.15.119:0/1986717209 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a3fff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a415df50 0x7f36a40c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f369bfff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a406c0a0 0x7f36a40c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 -- 172.21.15.119:0/1986717209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a406c0a0 msgr2=0x7f36a40c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a406c0a0 0x7f36a40c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 -- 172.21.15.119:0/1986717209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a415df50 msgr2=0x7f36a40c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a415df50 0x7f36a40c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 -- 172.21.15.119:0/1986717209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694009420 con 0x7f36a406bcd0 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a37fe700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406bcd0 0x7f36a40c5870 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f369400ec80 tx=0x7f36940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f369b7fe700 1 -- 172.21.15.119:0/1986717209 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369400c9d0 con 0x7f36a406bcd0 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f369b7fe700 1 -- 172.21.15.119:0/1986717209 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369400cb30 con 0x7f36a406bcd0 2023-10-08T01:12:48.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c0094f0 con 0x7f36a406bcd0 2023-10-08T01:12:48.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.486+0000 7f369b7fe700 1 -- 172.21.15.119:0/1986717209 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369400ceb0 con 0x7f36a406bcd0 2023-10-08T01:12:48.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.487+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a416a610 con 0x7f36a406bcd0 2023-10-08T01:12:48.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.488+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a416a980 con 0x7f36a406bcd0 2023-10-08T01:12:48.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.488+0000 7f369b7fe700 1 -- 172.21.15.119:0/1986717209 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3694013680 con 0x7f36a406bcd0 2023-10-08T01:12:48.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.489+0000 7f369b7fe700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3684066f30 0x7f36840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.489+0000 7f369b7fe700 1 -- 172.21.15.119:0/1986717209 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f3694084a60 con 0x7f36a406bcd0 2023-10-08T01:12:48.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.490+0000 7f369bfff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3684066f30 0x7f36840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.490+0000 7f369bfff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3684066f30 0x7f36840693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36a40691a0 tx=0x7f368c017000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:48.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.494+0000 7f369b7fe700 1 -- 172.21.15.119:0/1986717209 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3694051b40 con 0x7f36a406bcd0 2023-10-08T01:12:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.727+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36a404c3c0 con 0x7f36a406bcd0 2023-10-08T01:12:48.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.728+0000 7f369b7fe700 1 -- 172.21.15.119:0/1986717209 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f3694055160 con 0x7f36a406bcd0 2023-10-08T01:12:48.729 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:48.729 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1682556092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1682556092},{"type":"v1","addr":"172.21.15.184:6835","nonce":1682556092}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/691036731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":691036731},{"type":"v1","addr":"172.21.15.184:6834","nonce":691036731}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7182,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3636556734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3636556734},{"type":"v1","addr":"172.21.15.119:6834","nonce":3636556734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:44.715284+0000","modified":"2023-10-08T01:12:48.271445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7185":{"gid":7185,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3302512146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3302512146},{"type":"v1","addr":"172.21.15.119:6835","nonce":3302512146}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T01:12:48.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.730+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3684066f30 msgr2=0x7f36840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.730+0000 7f36a9dff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3684066f30 0x7f36840693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f36a40691a0 tx=0x7f368c017000 comp rx=0 tx=0).stop 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.730+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406bcd0 msgr2=0x7f36a40c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.730+0000 7f36a9dff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406bcd0 0x7f36a40c5870 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f369400ec80 tx=0x7f36940091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.730+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 shutdown_connections 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.730+0000 7f36a9dff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3684066f30 0x7f36840693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.730+0000 7f36a9dff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36a415df50 0x7f36a40c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.730+0000 7f36a9dff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36a406bcd0 0x7f36a40c5870 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.731+0000 7f36a9dff700 1 --2- 172.21.15.119:0/1986717209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36a406c0a0 0x7f36a40c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.731+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 >> 172.21.15.119:0/1986717209 conn(0x7f36a40ce600 msgr2=0x7f36a4156160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:48.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.731+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 shutdown_connections 2023-10-08T01:12:48.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.731+0000 7f36a9dff700 1 -- 172.21.15.119:0/1986717209 wait complete. 2023-10-08T01:12:48.733 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 48 2023-10-08T01:12:48.749 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:12:48.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.927+0000 7faadcdff700 1 Processor -- start 2023-10-08T01:12:48.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.928+0000 7faadcdff700 1 -- start start 2023-10-08T01:12:48.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.929+0000 7faadcdff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faad80b1420 0x7faad80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.929+0000 7faadcdff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad80cc300 con 0x7faad80cbf30 2023-10-08T01:12:48.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.929+0000 7faadcdff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad80ad370 con 0x7faad80ccf90 2023-10-08T01:12:48.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.929+0000 7faadcdff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad8156bf0 con 0x7faad80b1420 2023-10-08T01:12:48.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.929+0000 7faad659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7faad80cbf30 0x7faad80b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45380/0 (socket says 172.21.15.119:45380) 2023-10-08T01:12:48.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.929+0000 7faad659c700 1 -- 172.21.15.119:0/3526256572 learned_addr learned my addr 172.21.15.119:0/3526256572 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:48.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.929+0000 7faacffff700 1 --2- 172.21.15.119:0/3526256572 >> v2:172.21.15.184:3300/0 conn(0x7faad80b1420 0x7faad80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.929+0000 7faad5d9b700 1 -- 172.21.15.119:0/3526256572 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283544729 0 0) 0x7faad80cc300 con 0x7faad80cbf30 2023-10-08T01:12:48.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faad5d9b700 1 -- 172.21.15.119:0/3526256572 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faac4003570 con 0x7faad80cbf30 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faad5d9b700 1 -- 172.21.15.119:0/3526256572 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828085083 0 0) 0x7faad80ad370 con 0x7faad80ccf90 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faad5d9b700 1 -- 172.21.15.119:0/3526256572 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faad80cc300 con 0x7faad80ccf90 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faacffff700 1 -- 172.21.15.119:0/3526256572 >> v1:172.21.15.184:6789/0 conn(0x7faad80ccf90 legacy=0x7faad80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faacffff700 1 -- 172.21.15.119:0/3526256572 >> v1:172.21.15.119:6789/0 conn(0x7faad80cbf30 legacy=0x7faad80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faacffff700 1 -- 172.21.15.119:0/3526256572 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad80ad650 con 0x7faad80b1420 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faacffff700 1 --2- 172.21.15.119:0/3526256572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80b1420 0x7faad80ae410 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faac00098e0 tx=0x7faac00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb9f23bb2b4c97b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faad5d9b700 1 -- 172.21.15.119:0/3526256572 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0011630 con 0x7faad80b1420 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faad5d9b700 1 -- 172.21.15.119:0/3526256572 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac0011790 con 0x7faad80b1420 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.930+0000 7faad5d9b700 1 -- 172.21.15.119:0/3526256572 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0011b30 con 0x7faad80b1420 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.931+0000 7faadcdff700 1 -- 172.21.15.119:0/3526256572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80b1420 msgr2=0x7faad80ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.931+0000 7faadcdff700 1 --2- 172.21.15.119:0/3526256572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80b1420 0x7faad80ae410 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7faac00098e0 tx=0x7faac00091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.931+0000 7faadcdff700 1 -- 172.21.15.119:0/3526256572 shutdown_connections 2023-10-08T01:12:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.931+0000 7faadcdff700 1 --2- 172.21.15.119:0/3526256572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80b1420 0x7faad80ae410 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.931+0000 7faadcdff700 1 -- 172.21.15.119:0/3526256572 >> 172.21.15.119:0/3526256572 conn(0x7faad806a320 msgr2=0x7faad80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.931+0000 7faadcdff700 1 -- 172.21.15.119:0/3526256572 shutdown_connections 2023-10-08T01:12:48.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.931+0000 7faadcdff700 1 -- 172.21.15.119:0/3526256572 wait complete. 2023-10-08T01:12:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.932+0000 7faadcdff700 1 Processor -- start 2023-10-08T01:12:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.933+0000 7faadcdff700 1 -- start start 2023-10-08T01:12:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.933+0000 7faadcdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80b1420 0x7faad8143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.933+0000 7faadcdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad80cbf30 0x7faad8154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.933+0000 7faadcdff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80ccf90 0x7faad8154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.933+0000 7faadcdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad816b010 con 0x7faad80b1420 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.933+0000 7faadcdff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8144510 con 0x7faad80ccf90 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.933+0000 7faadcdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad816a400 con 0x7faad80cbf30 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80b1420 0x7faad8143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80b1420 0x7faad8143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47236/0 (socket says 172.21.15.119:47236) 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 -- 172.21.15.119:0/1884930460 learned_addr learned my addr 172.21.15.119:0/1884930460 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad6d9d700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80ccf90 0x7faad8154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faacffff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad80cbf30 0x7faad8154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 -- 172.21.15.119:0/1884930460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad80cbf30 msgr2=0x7faad8154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad80cbf30 0x7faad8154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 -- 172.21.15.119:0/1884930460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80ccf90 msgr2=0x7faad8154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:48.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80ccf90 0x7faad8154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:48.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 -- 172.21.15.119:0/1884930460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0009420 con 0x7faad80b1420 2023-10-08T01:12:48.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.934+0000 7faad659c700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80b1420 0x7faad8143fd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7faac800c790 tx=0x7faac8007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:48.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.935+0000 7faace7fc700 1 -- 172.21.15.119:0/1884930460 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8005b30 con 0x7faad80b1420 2023-10-08T01:12:48.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.935+0000 7faace7fc700 1 -- 172.21.15.119:0/1884930460 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac8005c90 con 0x7faad80b1420 2023-10-08T01:12:48.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.935+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad816a6e0 con 0x7faad80b1420 2023-10-08T01:12:48.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.935+0000 7faace7fc700 1 -- 172.21.15.119:0/1884930460 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8007300 con 0x7faad80b1420 2023-10-08T01:12:48.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.935+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad816aca0 con 0x7faad80b1420 2023-10-08T01:12:48.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.936+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad804c3c0 con 0x7faad80b1420 2023-10-08T01:12:48.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.937+0000 7faace7fc700 1 -- 172.21.15.119:0/1884930460 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faac801b400 con 0x7faad80b1420 2023-10-08T01:12:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.937+0000 7faace7fc700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faab8066f30 0x7faab80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.938+0000 7faace7fc700 1 -- 172.21.15.119:0/1884930460 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7faac80841f0 con 0x7faad80b1420 2023-10-08T01:12:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.938+0000 7faacffff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faab8066f30 0x7faab80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:48.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.938+0000 7faacffff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faab8066f30 0x7faab80693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faad80b1820 tx=0x7faac0017000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:48.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:48.941+0000 7faace7fc700 1 -- 172.21.15.119:0/1884930460 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faac8054e90 con 0x7faad80b1420 2023-10-08T01:12:49.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.131+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faad80ad0b0 con 0x7faad80b1420 2023-10-08T01:12:49.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.132+0000 7faace7fc700 1 -- 172.21.15.119:0/1884930460 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16009 (secure 0 0 0) 0x7faac80076f0 con 0x7faad80b1420 2023-10-08T01:12:49.133 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:49.134 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":104,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:12:46.238462+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:42.395485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:43.700121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:12:49.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faab8066f30 msgr2=0x7faab80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faab8066f30 0x7faab80693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faad80b1820 tx=0x7faac0017000 comp rx=0 tx=0).stop 2023-10-08T01:12:49.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80b1420 msgr2=0x7faad8143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80b1420 0x7faad8143fd0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7faac800c790 tx=0x7faac8007ea0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 shutdown_connections 2023-10-08T01:12:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faab8066f30 0x7faab80693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faad80ccf90 0x7faad8154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faad80b1420 0x7faad8143fd0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 --2- 172.21.15.119:0/1884930460 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faad80cbf30 0x7faad8154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 >> 172.21.15.119:0/1884930460 conn(0x7faad806a320 msgr2=0x7faad80b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 shutdown_connections 2023-10-08T01:12:49.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.135+0000 7faadcdff700 1 -- 172.21.15.119:0/1884930460 wait complete. 2023-10-08T01:12:49.155 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T01:12:49.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:12:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.336+0000 7f01f8dff700 1 Processor -- start 2023-10-08T01:12:49.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.336+0000 7f01f8dff700 1 -- start start 2023-10-08T01:12:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f8dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01f40cbf30 0x7f01f40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f8dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01f40ccf90 0x7f01f40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f8dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f40cc300 con 0x7f01f415a270 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f8dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f4156bf0 con 0x7f01f40ccf90 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f8dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40ae950 con 0x7f01f40cbf30 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f2d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f01f415a270 0x7f01f40c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45384/0 (socket says 172.21.15.119:45384) 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f2d9d700 1 -- 172.21.15.119:0/2863776964 learned_addr learned my addr 172.21.15.119:0/2863776964 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01ebfff700 1 --2- 172.21.15.119:0/2863776964 >> v2:172.21.15.184:3300/0 conn(0x7f01f40ccf90 0x7f01f40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f259c700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01f40cbf30 0x7f01f40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864275253 0 0) 0x7f01f40cc300 con 0x7f01f415a270 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.338+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e0003570 con 0x7f01f415a270 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3319684585 0 0) 0x7f01e0003570 con 0x7f01f415a270 2023-10-08T01:12:49.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01f40cbf30 msgr2=0x7f01f40ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01f40cbf30 0x7f01f40ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 >> v2:172.21.15.184:3300/0 conn(0x7f01f40ccf90 msgr2=0x7f01f40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 --2- 172.21.15.119:0/2863776964 >> v2:172.21.15.184:3300/0 conn(0x7f01f40ccf90 0x7f01f40b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40ad370 con 0x7f01f415a270 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f01ec0017f0 con 0x7f01f415a270 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 >> v1:172.21.15.119:6789/0 conn(0x7f01f415a270 legacy=0x7f01f40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01f40cbf30 0x7f01e0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0004010 0x7f01e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e0004d50 0x7f01e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4156bf0 con 0x7f01e0004010 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40ae950 con 0x7f01e0004d50 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e000bab0 con 0x7f01f40cbf30 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01ebfff700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0004010 0x7f01e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f259c700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01f40cbf30 0x7f01e0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.339+0000 7f01f2d9d700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e0004d50 0x7f01e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01ebfff700 1 -- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01f40cbf30 msgr2=0x7f01e0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01ebfff700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01f40cbf30 0x7f01e0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01ebfff700 1 -- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e0004d50 msgr2=0x7f01e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:49.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01ebfff700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01e0004d50 0x7f01e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01ebfff700 1 -- 172.21.15.119:0/2863776964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0003570 con 0x7f01e0004010 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01ebfff700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0004010 0x7f01e0004810 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009c00 tx=0x7f01dc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfc65e8131d382f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00e710 con 0x7f01e0004010 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc00e870 con 0x7f01e0004010 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.340+0000 7f01f1d9b700 1 -- 172.21.15.119:0/2863776964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00ec30 con 0x7f01e0004010 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.341+0000 7f01f8dff700 1 -- 172.21.15.119:0/2863776964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0004010 msgr2=0x7f01e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.341+0000 7f01f8dff700 1 --2- 172.21.15.119:0/2863776964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0004010 0x7f01e0004810 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f01dc009c00 tx=0x7f01dc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.341+0000 7f01f2d9d700 1 -- 172.21.15.119:0/2863776964 reap_dead start 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.341+0000 7f01f8dff700 1 -- 172.21.15.119:0/2863776964 shutdown_connections 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.341+0000 7f01f8dff700 1 -- 172.21.15.119:0/2863776964 >> 172.21.15.119:0/2863776964 conn(0x7f01f406a320 msgr2=0x7f01f40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:49.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.341+0000 7f01f8dff700 1 -- 172.21.15.119:0/2863776964 shutdown_connections 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.341+0000 7f01f8dff700 1 -- 172.21.15.119:0/2863776964 wait complete. 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.342+0000 7f01f8dff700 1 Processor -- start 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.342+0000 7f01f8dff700 1 -- start start 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f8dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01e0004010 0x7f01f4149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f8dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f4158a30 0x7f01f415d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f8dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4155010 0x7f01f415d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f8dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f415abc0 con 0x7f01f4155010 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f8dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4158f30 con 0x7f01f4158a30 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f8dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4159210 con 0x7f01e0004010 2023-10-08T01:12:49.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4155010 0x7f01f415d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4155010 0x7f01f415d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47254/0 (socket says 172.21.15.119:47254) 2023-10-08T01:12:49.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.343+0000 7f01f2d9d700 1 -- 172.21.15.119:0/3292288919 learned_addr learned my addr 172.21.15.119:0/3292288919 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:49.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01ebfff700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f4158a30 0x7f01f415d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f259c700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01e0004010 0x7f01f4149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f2d9d700 1 -- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01e0004010 msgr2=0x7f01f4149100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f2d9d700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01e0004010 0x7f01f4149100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f2d9d700 1 -- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f4158a30 msgr2=0x7f01f415d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f2d9d700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f4158a30 0x7f01f415d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f2d9d700 1 -- 172.21.15.119:0/3292288919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f41594f0 con 0x7f01f4155010 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f259c700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01e0004010 0x7f01f4149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f2d9d700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4155010 0x7f01f415d940 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f01f40cd430 tx=0x7f01ec0021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01ea7fc700 1 -- 172.21.15.119:0/3292288919 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00ad40 con 0x7f01f4155010 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01ea7fc700 1 -- 172.21.15.119:0/3292288919 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ec0133f0 con 0x7f01f4155010 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f41597d0 con 0x7f01f4155010 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01ea7fc700 1 -- 172.21.15.119:0/3292288919 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00aea0 con 0x7f01f4155010 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.344+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f4155880 con 0x7f01f4155010 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.345+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f4159960 con 0x7f01f4155010 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.346+0000 7f01ea7fc700 1 -- 172.21.15.119:0/3292288919 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01ec013a20 con 0x7f01f4155010 2023-10-08T01:12:49.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.347+0000 7f01ea7fc700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f01d4066f30 0x7f01d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.347+0000 7f01ea7fc700 1 -- 172.21.15.119:0/3292288919 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f01ec084fc0 con 0x7f01f4155010 2023-10-08T01:12:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.347+0000 7f01f259c700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f01d4066f30 0x7f01d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.348+0000 7f01f259c700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f01d4066f30 0x7f01d40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01e4006dc0 tx=0x7f01e4008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:49.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.351+0000 7f01ea7fc700 1 -- 172.21.15.119:0/3292288919 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01ec0520a0 con 0x7f01f4155010 2023-10-08T01:12:49.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.538+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f01f404c3c0 con 0x7f01d4066f30 2023-10-08T01:12:49.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.548+0000 7f01ea7fc700 1 -- 172.21.15.119:0/3292288919 <== mgr.6999 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278860 (secure 0 0 0) 0x7f01f404c3c0 con 0x7f01d4066f30 2023-10-08T01:12:49.553 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:49.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.552+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f01d4066f30 msgr2=0x7f01d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.552+0000 7f01f8dff700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f01d4066f30 0x7f01d40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f01e4006dc0 tx=0x7f01e4008040 comp rx=0 tx=0).stop 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4155010 msgr2=0x7f01f415d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4155010 0x7f01f415d940 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f01f40cd430 tx=0x7f01ec0021d0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 shutdown_connections 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f01d4066f30 0x7f01d40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01e0004010 0x7f01f4149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01f4155010 0x7f01f415d940 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 --2- 172.21.15.119:0/3292288919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01f4158a30 0x7f01f415d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 >> 172.21.15.119:0/3292288919 conn(0x7f01f406a320 msgr2=0x7f01f4155e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 shutdown_connections 2023-10-08T01:12:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.553+0000 7f01f8dff700 1 -- 172.21.15.119:0/3292288919 wait complete. 2023-10-08T01:12:49.557 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:12:49.572 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-08T01:12:47.669749+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":341,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":58,"acting":58,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":2438380,"kb_used_data":10924,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752536340,"statfs":{"total":773094113280,"available":770597212160,"internally_reserved":0,"allocated":11186176,"data_stored":6012433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.439341"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267417+0000","last_change":"2023-10-08T01:12:45.264773+0000","last_active":"2023-10-08T01:12:45.267417+0000","last_peered":"2023-10-08T01:12:45.267417+0000","last_clean":"2023-10-08T01:12:45.267417+0000","last_became_active":"2023-10-08T01:12:45.264198+0000","last_became_peered":"2023-10-08T01:12:45.264198+0000","last_unstale":"2023-10-08T01:12:45.267417+0000","last_undegraded":"2023-10-08T01:12:45.267417+0000","last_fullsized":"2023-10-08T01:12:45.267417+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:47.797058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.262721+0000","last_change":"2023-10-08T01:12:45.256900+0000","last_active":"2023-10-08T01:12:45.262721+0000","last_peered":"2023-10-08T01:12:45.262721+0000","last_clean":"2023-10-08T01:12:45.262721+0000","last_became_active":"2023-10-08T01:12:45.256568+0000","last_became_peered":"2023-10-08T01:12:45.256568+0000","last_unstale":"2023-10-08T01:12:45.262721+0000","last_undegraded":"2023-10-08T01:12:45.262721+0000","last_fullsized":"2023-10-08T01:12:45.262721+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:22:12.485007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267295+0000","last_change":"2023-10-08T01:12:45.263929+0000","last_active":"2023-10-08T01:12:45.267295+0000","last_peered":"2023-10-08T01:12:45.267295+0000","last_clean":"2023-10-08T01:12:45.267295+0000","last_became_active":"2023-10-08T01:12:45.263639+0000","last_became_peered":"2023-10-08T01:12:45.263639+0000","last_unstale":"2023-10-08T01:12:45.267295+0000","last_undegraded":"2023-10-08T01:12:45.267295+0000","last_fullsized":"2023-10-08T01:12:45.267295+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:21.657522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.261546+0000","last_change":"2023-10-08T01:12:45.259791+0000","last_active":"2023-10-08T01:12:45.261546+0000","last_peered":"2023-10-08T01:12:45.261546+0000","last_clean":"2023-10-08T01:12:45.261546+0000","last_became_active":"2023-10-08T01:12:45.259349+0000","last_became_peered":"2023-10-08T01:12:45.259349+0000","last_unstale":"2023-10-08T01:12:45.261546+0000","last_undegraded":"2023-10-08T01:12:45.261546+0000","last_fullsized":"2023-10-08T01:12:45.261546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:27.924521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273138+0000","last_change":"2023-10-08T01:12:45.269555+0000","last_active":"2023-10-08T01:12:45.273138+0000","last_peered":"2023-10-08T01:12:45.273138+0000","last_clean":"2023-10-08T01:12:45.273138+0000","last_became_active":"2023-10-08T01:12:45.269257+0000","last_became_peered":"2023-10-08T01:12:45.269257+0000","last_unstale":"2023-10-08T01:12:45.273138+0000","last_undegraded":"2023-10-08T01:12:45.273138+0000","last_fullsized":"2023-10-08T01:12:45.273138+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:44.683003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.270156+0000","last_change":"2023-10-08T01:12:45.262121+0000","last_active":"2023-10-08T01:12:45.270156+0000","last_peered":"2023-10-08T01:12:45.270156+0000","last_clean":"2023-10-08T01:12:45.270156+0000","last_became_active":"2023-10-08T01:12:45.261872+0000","last_became_peered":"2023-10-08T01:12:45.261872+0000","last_unstale":"2023-10-08T01:12:45.270156+0000","last_undegraded":"2023-10-08T01:12:45.270156+0000","last_fullsized":"2023-10-08T01:12:45.270156+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:12:49.301176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273170+0000","last_change":"2023-10-08T01:12:45.267334+0000","last_active":"2023-10-08T01:12:45.273170+0000","last_peered":"2023-10-08T01:12:45.273170+0000","last_clean":"2023-10-08T01:12:45.273170+0000","last_became_active":"2023-10-08T01:12:45.267086+0000","last_became_peered":"2023-10-08T01:12:45.267086+0000","last_unstale":"2023-10-08T01:12:45.273170+0000","last_undegraded":"2023-10-08T01:12:45.273170+0000","last_fullsized":"2023-10-08T01:12:45.273170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273010+0000","last_change":"2023-10-08T01:12:45.267180+0000","last_active":"2023-10-08T01:12:45.273010+0000","last_peered":"2023-10-08T01:12:45.273010+0000","last_clean":"2023-10-08T01:12:45.273010+0000","last_became_active":"2023-10-08T01:12:45.266278+0000","last_became_peered":"2023-10-08T01:12:45.266278+0000","last_unstale":"2023-10-08T01:12:45.273010+0000","last_undegraded":"2023-10-08T01:12:45.273010+0000","last_fullsized":"2023-10-08T01:12:45.273010+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:38.262593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273112+0000","last_change":"2023-10-08T01:12:45.266691+0000","last_active":"2023-10-08T01:12:45.273112+0000","last_peered":"2023-10-08T01:12:45.273112+0000","last_clean":"2023-10-08T01:12:45.273112+0000","last_became_active":"2023-10-08T01:12:45.266371+0000","last_became_peered":"2023-10-08T01:12:45.266371+0000","last_unstale":"2023-10-08T01:12:45.273112+0000","last_undegraded":"2023-10-08T01:12:45.273112+0000","last_fullsized":"2023-10-08T01:12:45.273112+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:01.279801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.277816+0000","last_change":"2023-10-08T01:12:44.261721+0000","last_active":"2023-10-08T01:12:45.277816+0000","last_peered":"2023-10-08T01:12:45.277816+0000","last_clean":"2023-10-08T01:12:45.277816+0000","last_became_active":"2023-10-08T01:12:44.261441+0000","last_became_peered":"2023-10-08T01:12:44.261441+0000","last_unstale":"2023-10-08T01:12:45.277816+0000","last_undegraded":"2023-10-08T01:12:45.277816+0000","last_fullsized":"2023-10-08T01:12:45.277816+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:52.132770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.277869+0000","last_change":"2023-10-08T01:12:44.260062+0000","last_active":"2023-10-08T01:12:45.277869+0000","last_peered":"2023-10-08T01:12:45.277869+0000","last_clean":"2023-10-08T01:12:45.277869+0000","last_became_active":"2023-10-08T01:12:44.259824+0000","last_became_peered":"2023-10-08T01:12:44.259824+0000","last_unstale":"2023-10-08T01:12:45.277869+0000","last_undegraded":"2023-10-08T01:12:45.277869+0000","last_fullsized":"2023-10-08T01:12:45.277869+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:23.038163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.244565+0000","last_change":"2023-10-08T01:12:44.255146+0000","last_active":"2023-10-08T01:12:45.244565+0000","last_peered":"2023-10-08T01:12:45.244565+0000","last_clean":"2023-10-08T01:12:45.244565+0000","last_became_active":"2023-10-08T01:12:44.254849+0000","last_became_peered":"2023-10-08T01:12:44.254849+0000","last_unstale":"2023-10-08T01:12:45.244565+0000","last_undegraded":"2023-10-08T01:12:45.244565+0000","last_fullsized":"2023-10-08T01:12:45.244565+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:38:59.726189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.261528+0000","last_change":"2023-10-08T01:12:45.259623+0000","last_active":"2023-10-08T01:12:45.261528+0000","last_peered":"2023-10-08T01:12:45.261528+0000","last_clean":"2023-10-08T01:12:45.261528+0000","last_became_active":"2023-10-08T01:12:45.258942+0000","last_became_peered":"2023-10-08T01:12:45.258942+0000","last_unstale":"2023-10-08T01:12:45.261528+0000","last_undegraded":"2023-10-08T01:12:45.261528+0000","last_fullsized":"2023-10-08T01:12:45.261528+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:18:43.749953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.262629+0000","last_change":"2023-10-08T01:12:45.260215+0000","last_active":"2023-10-08T01:12:45.262629+0000","last_peered":"2023-10-08T01:12:45.262629+0000","last_clean":"2023-10-08T01:12:45.262629+0000","last_became_active":"2023-10-08T01:12:45.258758+0000","last_became_peered":"2023-10-08T01:12:45.258758+0000","last_unstale":"2023-10-08T01:12:45.262629+0000","last_undegraded":"2023-10-08T01:12:45.262629+0000","last_fullsized":"2023-10-08T01:12:45.262629+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:17:23.249506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.270104+0000","last_change":"2023-10-08T01:12:45.261467+0000","last_active":"2023-10-08T01:12:45.270104+0000","last_peered":"2023-10-08T01:12:45.270104+0000","last_clean":"2023-10-08T01:12:45.270104+0000","last_became_active":"2023-10-08T01:12:45.261187+0000","last_became_peered":"2023-10-08T01:12:45.261187+0000","last_unstale":"2023-10-08T01:12:45.270104+0000","last_undegraded":"2023-10-08T01:12:45.270104+0000","last_fullsized":"2023-10-08T01:12:45.270104+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:47.947899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273517+0000","last_change":"2023-10-08T01:12:45.265232+0000","last_active":"2023-10-08T01:12:45.273517+0000","last_peered":"2023-10-08T01:12:45.273517+0000","last_clean":"2023-10-08T01:12:45.273517+0000","last_became_active":"2023-10-08T01:12:45.264914+0000","last_became_peered":"2023-10-08T01:12:45.264914+0000","last_unstale":"2023-10-08T01:12:45.273517+0000","last_undegraded":"2023-10-08T01:12:45.273517+0000","last_fullsized":"2023-10-08T01:12:45.273517+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:18:43.749953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.254495+0000","last_change":"2023-10-08T01:12:44.256072+0000","last_active":"2023-10-08T01:12:45.254495+0000","last_peered":"2023-10-08T01:12:45.254495+0000","last_clean":"2023-10-08T01:12:45.254495+0000","last_became_active":"2023-10-08T01:12:44.255819+0000","last_became_peered":"2023-10-08T01:12:44.255819+0000","last_unstale":"2023-10-08T01:12:45.254495+0000","last_undegraded":"2023-10-08T01:12:45.254495+0000","last_fullsized":"2023-10-08T01:12:45.254495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:34.791617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"103'8","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.281535+0000","last_change":"2023-10-08T01:12:44.257004+0000","last_active":"2023-10-08T01:12:45.281535+0000","last_peered":"2023-10-08T01:12:45.281535+0000","last_clean":"2023-10-08T01:12:45.281535+0000","last_became_active":"2023-10-08T01:12:44.256606+0000","last_became_peered":"2023-10-08T01:12:44.256606+0000","last_unstale":"2023-10-08T01:12:45.281535+0000","last_undegraded":"2023-10-08T01:12:45.281535+0000","last_fullsized":"2023-10-08T01:12:45.281535+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:04:03.871176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.280664+0000","last_change":"2023-10-08T01:12:44.261407+0000","last_active":"2023-10-08T01:12:45.280664+0000","last_peered":"2023-10-08T01:12:45.280664+0000","last_clean":"2023-10-08T01:12:45.280664+0000","last_became_active":"2023-10-08T01:12:44.261142+0000","last_became_peered":"2023-10-08T01:12:44.261142+0000","last_unstale":"2023-10-08T01:12:45.280664+0000","last_undegraded":"2023-10-08T01:12:45.280664+0000","last_fullsized":"2023-10-08T01:12:45.280664+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:16:06.145977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.245184+0000","last_change":"2023-10-08T01:12:44.254537+0000","last_active":"2023-10-08T01:12:45.245184+0000","last_peered":"2023-10-08T01:12:45.245184+0000","last_clean":"2023-10-08T01:12:45.245184+0000","last_became_active":"2023-10-08T01:12:44.254189+0000","last_became_peered":"2023-10-08T01:12:44.254189+0000","last_unstale":"2023-10-08T01:12:45.245184+0000","last_undegraded":"2023-10-08T01:12:45.245184+0000","last_fullsized":"2023-10-08T01:12:45.245184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:04.324015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267277+0000","last_change":"2023-10-08T01:12:45.263857+0000","last_active":"2023-10-08T01:12:45.267277+0000","last_peered":"2023-10-08T01:12:45.267277+0000","last_clean":"2023-10-08T01:12:45.267277+0000","last_became_active":"2023-10-08T01:12:45.263608+0000","last_became_peered":"2023-10-08T01:12:45.263608+0000","last_unstale":"2023-10-08T01:12:45.267277+0000","last_undegraded":"2023-10-08T01:12:45.267277+0000","last_fullsized":"2023-10-08T01:12:45.267277+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:03.239767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.274572+0000","last_change":"2023-10-08T01:12:44.259093+0000","last_active":"2023-10-08T01:12:45.274572+0000","last_peered":"2023-10-08T01:12:45.274572+0000","last_clean":"2023-10-08T01:12:45.274572+0000","last_became_active":"2023-10-08T01:12:44.258830+0000","last_became_peered":"2023-10-08T01:12:44.258830+0000","last_unstale":"2023-10-08T01:12:45.274572+0000","last_undegraded":"2023-10-08T01:12:45.274572+0000","last_fullsized":"2023-10-08T01:12:45.274572+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:37.153567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.247114+0000","last_change":"2023-10-08T01:12:44.259595+0000","last_active":"2023-10-08T01:12:45.247114+0000","last_peered":"2023-10-08T01:12:45.247114+0000","last_clean":"2023-10-08T01:12:45.247114+0000","last_became_active":"2023-10-08T01:12:44.259154+0000","last_became_peered":"2023-10-08T01:12:44.259154+0000","last_unstale":"2023-10-08T01:12:45.247114+0000","last_undegraded":"2023-10-08T01:12:45.247114+0000","last_fullsized":"2023-10-08T01:12:45.247114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:53.512756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.255215+0000","last_change":"2023-10-08T01:12:45.253394+0000","last_active":"2023-10-08T01:12:45.255215+0000","last_peered":"2023-10-08T01:12:45.255215+0000","last_clean":"2023-10-08T01:12:45.255215+0000","last_became_active":"2023-10-08T01:12:45.253141+0000","last_became_peered":"2023-10-08T01:12:45.253141+0000","last_unstale":"2023-10-08T01:12:45.255215+0000","last_undegraded":"2023-10-08T01:12:45.255215+0000","last_fullsized":"2023-10-08T01:12:45.255215+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:35:37.810355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.246253+0000","last_change":"2023-10-08T01:12:44.258783+0000","last_active":"2023-10-08T01:12:45.246253+0000","last_peered":"2023-10-08T01:12:45.246253+0000","last_clean":"2023-10-08T01:12:45.246253+0000","last_became_active":"2023-10-08T01:12:44.258514+0000","last_became_peered":"2023-10-08T01:12:44.258514+0000","last_unstale":"2023-10-08T01:12:45.246253+0000","last_undegraded":"2023-10-08T01:12:45.246253+0000","last_fullsized":"2023-10-08T01:12:45.246253+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:41:36.735984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.254102+0000","last_change":"2023-10-08T01:12:44.258530+0000","last_active":"2023-10-08T01:12:45.254102+0000","last_peered":"2023-10-08T01:12:45.254102+0000","last_clean":"2023-10-08T01:12:45.254102+0000","last_became_active":"2023-10-08T01:12:44.258248+0000","last_became_peered":"2023-10-08T01:12:44.258248+0000","last_unstale":"2023-10-08T01:12:45.254102+0000","last_undegraded":"2023-10-08T01:12:45.254102+0000","last_fullsized":"2023-10-08T01:12:45.254102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:23.333638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.245283+0000","last_change":"2023-10-08T01:12:44.252561+0000","last_active":"2023-10-08T01:12:45.245283+0000","last_peered":"2023-10-08T01:12:45.245283+0000","last_clean":"2023-10-08T01:12:45.245283+0000","last_became_active":"2023-10-08T01:12:44.252224+0000","last_became_peered":"2023-10-08T01:12:44.252224+0000","last_unstale":"2023-10-08T01:12:45.245283+0000","last_undegraded":"2023-10-08T01:12:45.245283+0000","last_fullsized":"2023-10-08T01:12:45.245283+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:19.471269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273800+0000","last_change":"2023-10-08T01:12:45.266492+0000","last_active":"2023-10-08T01:12:45.273800+0000","last_peered":"2023-10-08T01:12:45.273800+0000","last_clean":"2023-10-08T01:12:45.273800+0000","last_became_active":"2023-10-08T01:12:45.266235+0000","last_became_peered":"2023-10-08T01:12:45.266235+0000","last_unstale":"2023-10-08T01:12:45.273800+0000","last_undegraded":"2023-10-08T01:12:45.273800+0000","last_fullsized":"2023-10-08T01:12:45.273800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:27.924521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:44.230143+0000","last_change":"2023-10-08T01:12:44.230143+0000","last_active":"2023-10-08T01:12:44.230143+0000","last_peered":"2023-10-08T01:12:44.230143+0000","last_clean":"2023-10-08T01:12:44.230143+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:44.230143+0000","last_undegraded":"2023-10-08T01:12:44.230143+0000","last_fullsized":"2023-10-08T01:12:44.230143+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:12:43.224582+0000","last_change":"2023-10-08T01:12:43.224582+0000","last_active":"2023-10-08T01:12:43.224582+0000","last_peered":"2023-10-08T01:12:43.224582+0000","last_clean":"2023-10-08T01:12:43.224582+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:12:43.224582+0000","last_undegraded":"2023-10-08T01:12:43.224582+0000","last_fullsized":"2023-10-08T01:12:43.224582+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"94'205","reported_seq":542,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-10-08T01:12:34.192699+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:12:34.192699+0000","last_peered":"2023-10-08T01:12:34.192699+0000","last_clean":"2023-10-08T01:12:34.192699+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:12:34.192699+0000","last_undegraded":"2023-10-08T01:12:34.192699+0000","last_fullsized":"2023-10-08T01:12:34.192699+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":32,"acting":32,"num_store_stats":3},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16,"ondisk_log_size":16,"up":24,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575051,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303164,"kb_used_data":1780,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068676,"statfs":{"total":96636764160,"available":96326324224,"internally_reserved":0,"allocated":1822720,"data_stored":1184057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.89500000000000002,"5min":1.123,"15min":1.123},"last":0.748},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.71299999999999997},"min":{"1min":0.52800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89200000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.71899999999999997}]},{"osd":1,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.48199999999999998,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.72599999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.79400000000000004}]},{"osd":2,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.52000000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.73199999999999998},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80600000000000005,"5min":1.167,"15min":1.167},"last":0.77800000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69199999999999995,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":0.94699999999999995,"15min":0.998},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.044,"5min":1.079,"15min":1.079},"last":0.70199999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.38},"max":{"1min":0.753,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.875,"5min":0.88,"15min":0.88300000000000001},"last":0.67200000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.64600000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.40400000000000003},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41599999999999998},"max":{"1min":0.85299999999999998,"5min":0.95199999999999996,"15min":0.98599999999999999},"last":0.56499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575051,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302584,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069256,"statfs":{"total":96636764160,"available":96326918144,"internally_reserved":0,"allocated":1228800,"data_stored":593689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67400000000000004,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66900000000000004,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.77100000000000002,"5min":0.91200000000000003,"15min":0.92800000000000005},"last":0.65100000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.72799999999999998},"min":{"1min":0.52000000000000002,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.88700000000000001,"5min":0.92400000000000004,"15min":2.984},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.74199999999999999,"5min":1.018,"15min":3.0190000000000001},"last":0.45300000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66500000000000004,"15min":0.69299999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.96699999999999997},"last":0.871}]},{"osd":3,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":0.95399999999999996,"15min":0.99199999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.70799999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.62},"min":{"1min":0.45700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.871,"5min":1.087,"15min":1.087},"last":0.69199999999999995}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.47899999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.0189999999999999,"5min":1.395,"15min":1.395},"last":0.754},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0489999999999999,"5min":1.21,"15min":1.21},"last":1.04}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.57699999999999996,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":1.095,"5min":1.095,"15min":2.9489999999999998},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.60099999999999998,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":1.075,"5min":1.329,"15min":1.329},"last":1.012}]}]},{"osd":1,"up_from":13,"seq":55834575051,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302584,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069256,"statfs":{"total":96636764160,"available":96326918144,"internally_reserved":0,"allocated":1228800,"data_stored":593689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.36499999999999999,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.55400000000000005,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.315},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.54700000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46899999999999997,"15min":0.47999999999999998},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.55500000000000005,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.51700000000000002,"15min":0.53400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33700000000000002},"max":{"1min":0.60699999999999998,"5min":0.70599999999999996,"15min":0.84799999999999998},"last":0.40999999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.62,"5min":0.76500000000000001,"15min":0.82099999999999995},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.59799999999999998,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.433}]},{"osd":4,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.40899999999999997},"max":{"1min":0.97799999999999998,"5min":1.452,"15min":1.452},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.747,"15min":0.76700000000000002},"min":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":1.0029999999999999,"5min":1.1499999999999999,"15min":1.1839999999999999},"last":0.83299999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.5,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91900000000000004,"5min":1.274,"15min":1.3959999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.498},"max":{"1min":0.88400000000000001,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.54600000000000004,"5min":0.497,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.77400000000000002,"15min":0.79100000000000004},"min":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.54000000000000004},"max":{"1min":1.054,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.79300000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.81200000000000006,"15min":0.82699999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.53200000000000003},"max":{"1min":1.026,"5min":1.621,"15min":1.621},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.755,"5min":0.81499999999999995,"15min":0.83499999999999996},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.81299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834575051,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303216,"kb_used_data":1832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068624,"statfs":{"total":96636764160,"available":96326270976,"internally_reserved":0,"allocated":1875968,"data_stored":1210778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.33700000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50800000000000001,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.497},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.48499999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.617},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.75}]},{"osd":5,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.5289999999999999},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.208,"5min":1.361,"15min":1.3779999999999999},"last":0.875},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1719999999999999,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575051,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306732,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065108,"statfs":{"total":96636764160,"available":96322670592,"internally_reserved":0,"allocated":1282048,"data_stored":620930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45200000000000001,"15min":0.45700000000000002},"min":{"1min":0.27200000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69699999999999995,"5min":0.752,"15min":0.752},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.27500000000000002},"max":{"1min":0.70499999999999996,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.43099999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68300000000000005,"5min":2.786,"15min":2.786},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.625,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.37}]},{"osd":3,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.78700000000000003,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.74199999999999999,"15min":0.76500000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.004,"5min":1.079,"15min":1.079},"last":0.746},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.76700000000000002,"15min":0.79200000000000004},"min":{"1min":0.46100000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.046,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.88600000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.80200000000000005,"15min":0.83099999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":2.488,"15min":2.488},"last":0.754},{"interface":"front","average":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81699999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.03,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.83399999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.50900000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.038,"5min":2.653,"15min":2.653},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77600000000000002,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.016,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0129999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.81000000000000005,"15min":0.81899999999999995},"min":{"1min":0.51800000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.0600000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.85399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575051,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306740,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065100,"statfs":{"total":96636764160,"available":96322662400,"internally_reserved":0,"allocated":1290240,"data_stored":621912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.69999999999999996,"15min":0.80100000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.57399999999999995,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.438}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61699999999999999,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.82999999999999996,"15min":0.88100000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.79400000000000004,"15min":1.1060000000000001},"last":0.55800000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":1.095,"15min":1.3029999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74299999999999999,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.117,"15min":1.413},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.77800000000000002},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.0920000000000001,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":1.089},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.379},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.486},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97099999999999997,"5min":1.0840000000000001,"15min":1.389},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.79700000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.5},"max":{"1min":0.93700000000000006,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":310776,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061064,"statfs":{"total":96636764160,"available":96318529536,"internally_reserved":0,"allocated":1228800,"data_stored":593689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.82499999999999996,"5min":0.999,"15min":0.999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.66900000000000004,"15min":0.69499999999999995},"min":{"1min":0.42599999999999999,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.56999999999999995}]},{"osd":1,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67400000000000004,"15min":0.70099999999999996},"min":{"1min":0.40100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94299999999999995,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.68500000000000005,"15min":0.70699999999999996},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88100000000000001,"5min":1.04,"15min":1.04},"last":0.71499999999999997}]},{"osd":2,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68500000000000005,"15min":0.73899999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":1.002},"last":0.629},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.76500000000000001},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.90900000000000003,"5min":1.323,"15min":1.323},"last":0.73299999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.53800000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.95799999999999996,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.746,"15min":0.78900000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.51800000000000002},"max":{"1min":0.88400000000000001,"5min":1.2829999999999999,"15min":1.454},"last":0.79500000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.84499999999999997,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90400000000000003,"5min":0.95499999999999996,"15min":0.998},"last":0.82399999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.41299999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.86399999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.81299999999999994}]},{"osd":7,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.501,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.878}]}]},{"osd":5,"up_from":13,"seq":55834575050,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":302584,"kb_used_data":1200,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069256,"statfs":{"total":96636764160,"available":96326918144,"internally_reserved":0,"allocated":1228800,"data_stored":593689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66900000000000004,"15min":0.69199999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.83699999999999997,"5min":0.90600000000000003,"15min":0.95599999999999996},"last":0.503},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.495,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.49099999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.85799999999999998,"5min":1.026,"15min":1.026},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.78800000000000003,"5min":0.91900000000000004,"15min":0.99099999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70099999999999996,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.38700000000000001},"max":{"1min":0.90100000000000002,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":0.91900000000000004,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.51900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.96099999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.58299999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.626}]},{"osd":4,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59299999999999997,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.39100000000000001,"15min":0.35599999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.98199999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.84499999999999997,"5min":1.048,"15min":1.081},"last":0.58199999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.78000000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.41699999999999998},"max":{"1min":0.90200000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71199999999999997,"15min":0.69099999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.93200000000000005,"5min":1.095,"15min":1.095},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":0.86699999999999999,"5min":1.05,"15min":1.1779999999999999},"last":0.752}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:12:49.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:12:49.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.796+0000 7f307ffff700 1 Processor -- start 2023-10-08T01:12:49.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.796+0000 7f307ffff700 1 -- start start 2023-10-08T01:12:49.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f308006a660 0x7f3080157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307ffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308006e2a0 con 0x7f3080150180 2023-10-08T01:12:49.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307ffff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308006b2e0 con 0x7f3080150550 2023-10-08T01:12:49.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307ffff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30800c78e0 con 0x7f308006a660 2023-10-08T01:12:49.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307effd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3080150180 0x7f30800c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45390/0 (socket says 172.21.15.119:45390) 2023-10-08T01:12:49.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307effd700 1 -- 172.21.15.119:0/3375798004 learned_addr learned my addr 172.21.15.119:0/3375798004 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:49.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307f7fe700 1 --2- 172.21.15.119:0/3375798004 >> v2:172.21.15.184:3300/0 conn(0x7f308006a660 0x7f3080157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307dffb700 1 -- 172.21.15.119:0/3375798004 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 447587725 0 0) 0x7f308006e2a0 con 0x7f3080150180 2023-10-08T01:12:49.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307dffb700 1 -- 172.21.15.119:0/3375798004 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3068003570 con 0x7f3080150180 2023-10-08T01:12:49.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307dffb700 1 -- 172.21.15.119:0/3375798004 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284111038 0 0) 0x7f308006b2e0 con 0x7f3080150550 2023-10-08T01:12:49.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307dffb700 1 -- 172.21.15.119:0/3375798004 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f308006e2a0 con 0x7f3080150550 2023-10-08T01:12:49.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307dffb700 1 -- 172.21.15.119:0/3375798004 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3058968796 0 0) 0x7f3068003570 con 0x7f3080150180 2023-10-08T01:12:49.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307f7fe700 1 -- 172.21.15.119:0/3375798004 >> v1:172.21.15.184:6789/0 conn(0x7f3080150550 legacy=0x7f3080069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307f7fe700 1 -- 172.21.15.119:0/3375798004 >> v1:172.21.15.119:6789/0 conn(0x7f3080150180 legacy=0x7f30800c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307f7fe700 1 -- 172.21.15.119:0/3375798004 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30800c7bc0 con 0x7f308006a660 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.797+0000 7f307f7fe700 1 --2- 172.21.15.119:0/3375798004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f308006a660 0x7f3080157770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f307800aac0 tx=0x7f307800a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f266442d6676c86e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307dffb700 1 -- 172.21.15.119:0/3375798004 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30700017f0 con 0x7f308006a660 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307dffb700 1 -- 172.21.15.119:0/3375798004 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307800c660 con 0x7f308006a660 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307dffb700 1 -- 172.21.15.119:0/3375798004 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307800ca00 con 0x7f308006a660 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307ffff700 1 -- 172.21.15.119:0/3375798004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f308006a660 msgr2=0x7f3080157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307ffff700 1 --2- 172.21.15.119:0/3375798004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f308006a660 0x7f3080157770 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f307800aac0 tx=0x7f307800a330 comp rx=0 tx=0).stop 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307ffff700 1 -- 172.21.15.119:0/3375798004 shutdown_connections 2023-10-08T01:12:49.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307ffff700 1 --2- 172.21.15.119:0/3375798004 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f308006a660 0x7f3080157770 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307ffff700 1 -- 172.21.15.119:0/3375798004 >> 172.21.15.119:0/3375798004 conn(0x7f30800cc1f0 msgr2=0x7f30800cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:49.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307ffff700 1 -- 172.21.15.119:0/3375798004 shutdown_connections 2023-10-08T01:12:49.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.798+0000 7f307ffff700 1 -- 172.21.15.119:0/3375798004 wait complete. 2023-10-08T01:12:49.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.800+0000 7f307ffff700 1 Processor -- start 2023-10-08T01:12:49.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.800+0000 7f307ffff700 1 -- start start 2023-10-08T01:12:49.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f308006a660 0x7f308014b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080150180 0x7f30800acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080150550 0x7f30800cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800ae640 con 0x7f3080150180 2023-10-08T01:12:49.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30800ada70 con 0x7f3080150550 2023-10-08T01:12:49.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30800add10 con 0x7f308006a660 2023-10-08T01:12:49.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080150180 0x7f30800acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080150180 0x7f30800acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47270/0 (socket says 172.21.15.119:47270) 2023-10-08T01:12:49.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307e7fc700 1 -- 172.21.15.119:0/2552354791 learned_addr learned my addr 172.21.15.119:0/2552354791 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:49.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307effd700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f308006a660 0x7f308014b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.801+0000 7f307f7fe700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080150550 0x7f30800cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f307e7fc700 1 -- 172.21.15.119:0/2552354791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f308006a660 msgr2=0x7f308014b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f307e7fc700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f308006a660 0x7f308014b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f307e7fc700 1 -- 172.21.15.119:0/2552354791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080150550 msgr2=0x7f30800cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f307e7fc700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080150550 0x7f30800cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f307e7fc700 1 -- 172.21.15.119:0/2552354791 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307800a5d0 con 0x7f3080150180 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f307e7fc700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080150180 0x7f30800acf40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f307400c580 tx=0x7f3074007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f3066ffd700 1 -- 172.21.15.119:0/2552354791 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074010510 con 0x7f3080150180 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f3066ffd700 1 -- 172.21.15.119:0/2552354791 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074010670 con 0x7f3080150180 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f3066ffd700 1 -- 172.21.15.119:0/2552354791 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074010a30 con 0x7f3080150180 2023-10-08T01:12:49.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30800adff0 con 0x7f3080150180 2023-10-08T01:12:49.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.802+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30800ad480 con 0x7f3080150180 2023-10-08T01:12:49.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.803+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30800ae7f0 con 0x7f3080150180 2023-10-08T01:12:49.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.804+0000 7f3066ffd700 1 -- 172.21.15.119:0/2552354791 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f307401b400 con 0x7f3080150180 2023-10-08T01:12:49.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.805+0000 7f3066ffd700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f306c066f30 0x7f306c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:49.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.805+0000 7f3066ffd700 1 -- 172.21.15.119:0/2552354791 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f30740841f0 con 0x7f3080150180 2023-10-08T01:12:49.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.805+0000 7f307effd700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f306c066f30 0x7f306c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:49.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.806+0000 7f307effd700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f306c066f30 0x7f306c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30800c5f30 tx=0x7f3070000f40 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:49.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.809+0000 7f3066ffd700 1 -- 172.21.15.119:0/2552354791 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30800ae7f0 con 0x7f3080150180 2023-10-08T01:12:49.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:49.995+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30800ae7f0 con 0x7f306c066f30 2023-10-08T01:12:50.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.003+0000 7f3066ffd700 1 -- 172.21.15.119:0/2552354791 <== mgr.6999 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289290 (secure 0 0 0) 0x7f30800ae7f0 con 0x7f306c066f30 2023-10-08T01:12:50.009 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:50.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.007+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f306c066f30 msgr2=0x7f306c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.007+0000 7f307ffff700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f306c066f30 0x7f306c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f30800c5f30 tx=0x7f3070000f40 comp rx=0 tx=0).stop 2023-10-08T01:12:50.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.007+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080150180 msgr2=0x7f30800acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.007+0000 7f307ffff700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080150180 0x7f30800acf40 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f307400c580 tx=0x7f3074007e10 comp rx=0 tx=0).stop 2023-10-08T01:12:50.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.009+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 shutdown_connections 2023-10-08T01:12:50.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.009+0000 7f307ffff700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f306c066f30 0x7f306c0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.009+0000 7f307ffff700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f308006a660 0x7f308014b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.009+0000 7f307ffff700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3080150550 0x7f30800cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.009+0000 7f307ffff700 1 --2- 172.21.15.119:0/2552354791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3080150180 0x7f30800acf40 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.009+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 >> 172.21.15.119:0/2552354791 conn(0x7f30800cc1f0 msgr2=0x7f3080159800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:50.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.009+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 shutdown_connections 2023-10-08T01:12:50.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.009+0000 7f307ffff700 1 -- 172.21.15.119:0/2552354791 wait complete. 2023-10-08T01:12:50.013 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:12:50.027 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:12:49.670064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":363,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":243,"ondisk_log_size":243,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2438664,"kb_used_data":11208,"kb_used_omap":0,"kb_used_meta":2427392,"kb_avail":752536056,"statfs":{"total":773094113280,"available":770596921344,"internally_reserved":0,"allocated":11476992,"data_stored":6148356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2485649408},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001019"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.268726+0000","last_change":"2023-10-08T01:12:45.267099+0000","last_active":"2023-10-08T01:12:45.268726+0000","last_peered":"2023-10-08T01:12:45.268726+0000","last_clean":"2023-10-08T01:12:45.268726+0000","last_became_active":"2023-10-08T01:12:45.266810+0000","last_became_peered":"2023-10-08T01:12:45.266810+0000","last_unstale":"2023-10-08T01:12:45.268726+0000","last_undegraded":"2023-10-08T01:12:45.268726+0000","last_fullsized":"2023-10-08T01:12:45.268726+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:24.043725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.263890+0000","last_change":"2023-10-08T01:12:45.260861+0000","last_active":"2023-10-08T01:12:45.263890+0000","last_peered":"2023-10-08T01:12:45.263890+0000","last_clean":"2023-10-08T01:12:45.263890+0000","last_became_active":"2023-10-08T01:12:45.260620+0000","last_became_peered":"2023-10-08T01:12:45.260620+0000","last_unstale":"2023-10-08T01:12:45.263890+0000","last_undegraded":"2023-10-08T01:12:45.263890+0000","last_fullsized":"2023-10-08T01:12:45.263890+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:52.890235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.269065+0000","last_change":"2023-10-08T01:12:45.261666+0000","last_active":"2023-10-08T01:12:45.269065+0000","last_peered":"2023-10-08T01:12:45.269065+0000","last_clean":"2023-10-08T01:12:45.269065+0000","last_became_active":"2023-10-08T01:12:45.261341+0000","last_became_peered":"2023-10-08T01:12:45.261341+0000","last_unstale":"2023-10-08T01:12:45.269065+0000","last_undegraded":"2023-10-08T01:12:45.269065+0000","last_fullsized":"2023-10-08T01:12:45.269065+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:12:49.301176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267417+0000","last_change":"2023-10-08T01:12:45.264773+0000","last_active":"2023-10-08T01:12:45.267417+0000","last_peered":"2023-10-08T01:12:45.267417+0000","last_clean":"2023-10-08T01:12:45.267417+0000","last_became_active":"2023-10-08T01:12:45.264198+0000","last_became_peered":"2023-10-08T01:12:45.264198+0000","last_unstale":"2023-10-08T01:12:45.267417+0000","last_undegraded":"2023-10-08T01:12:45.267417+0000","last_fullsized":"2023-10-08T01:12:45.267417+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:47.797058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267591+0000","last_change":"2023-10-08T01:12:45.264320+0000","last_active":"2023-10-08T01:12:45.267591+0000","last_peered":"2023-10-08T01:12:45.267591+0000","last_clean":"2023-10-08T01:12:45.267591+0000","last_became_active":"2023-10-08T01:12:45.264036+0000","last_became_peered":"2023-10-08T01:12:45.264036+0000","last_unstale":"2023-10-08T01:12:45.267591+0000","last_undegraded":"2023-10-08T01:12:45.267591+0000","last_fullsized":"2023-10-08T01:12:45.267591+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:50.363074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256651+0000","last_change":"2023-10-08T01:12:45.252967+0000","last_active":"2023-10-08T01:12:45.256651+0000","last_peered":"2023-10-08T01:12:45.256651+0000","last_clean":"2023-10-08T01:12:45.256651+0000","last_became_active":"2023-10-08T01:12:45.252649+0000","last_became_peered":"2023-10-08T01:12:45.252649+0000","last_unstale":"2023-10-08T01:12:45.256651+0000","last_undegraded":"2023-10-08T01:12:45.256651+0000","last_fullsized":"2023-10-08T01:12:45.256651+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:38.159129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.262721+0000","last_change":"2023-10-08T01:12:45.256900+0000","last_active":"2023-10-08T01:12:45.262721+0000","last_peered":"2023-10-08T01:12:45.262721+0000","last_clean":"2023-10-08T01:12:45.262721+0000","last_became_active":"2023-10-08T01:12:45.256568+0000","last_became_peered":"2023-10-08T01:12:45.256568+0000","last_unstale":"2023-10-08T01:12:45.262721+0000","last_undegraded":"2023-10-08T01:12:45.262721+0000","last_fullsized":"2023-10-08T01:12:45.262721+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:22:12.485007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.261008+0000","last_change":"2023-10-08T01:12:45.257732+0000","last_active":"2023-10-08T01:12:45.261008+0000","last_peered":"2023-10-08T01:12:45.261008+0000","last_clean":"2023-10-08T01:12:45.261008+0000","last_became_active":"2023-10-08T01:12:45.257440+0000","last_became_peered":"2023-10-08T01:12:45.257440+0000","last_unstale":"2023-10-08T01:12:45.261008+0000","last_undegraded":"2023-10-08T01:12:45.261008+0000","last_fullsized":"2023-10-08T01:12:45.261008+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:00.731750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256811+0000","last_change":"2023-10-08T01:12:45.253753+0000","last_active":"2023-10-08T01:12:45.256811+0000","last_peered":"2023-10-08T01:12:45.256811+0000","last_clean":"2023-10-08T01:12:45.256811+0000","last_became_active":"2023-10-08T01:12:45.253402+0000","last_became_peered":"2023-10-08T01:12:45.253402+0000","last_unstale":"2023-10-08T01:12:45.256811+0000","last_undegraded":"2023-10-08T01:12:45.256811+0000","last_fullsized":"2023-10-08T01:12:45.256811+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:24.043725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267378+0000","last_change":"2023-10-08T01:12:45.260274+0000","last_active":"2023-10-08T01:12:45.267378+0000","last_peered":"2023-10-08T01:12:45.267378+0000","last_clean":"2023-10-08T01:12:45.267378+0000","last_became_active":"2023-10-08T01:12:45.259837+0000","last_became_peered":"2023-10-08T01:12:45.259837+0000","last_unstale":"2023-10-08T01:12:45.267378+0000","last_undegraded":"2023-10-08T01:12:45.267378+0000","last_fullsized":"2023-10-08T01:12:45.267378+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:38.159129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.255147+0000","last_change":"2023-10-08T01:12:45.252046+0000","last_active":"2023-10-08T01:12:45.255147+0000","last_peered":"2023-10-08T01:12:45.255147+0000","last_clean":"2023-10-08T01:12:45.255147+0000","last_became_active":"2023-10-08T01:12:45.251787+0000","last_became_peered":"2023-10-08T01:12:45.251787+0000","last_unstale":"2023-10-08T01:12:45.255147+0000","last_undegraded":"2023-10-08T01:12:45.255147+0000","last_fullsized":"2023-10-08T01:12:45.255147+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:06:37.617043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267567+0000","last_change":"2023-10-08T01:12:45.260860+0000","last_active":"2023-10-08T01:12:45.267567+0000","last_peered":"2023-10-08T01:12:45.267567+0000","last_clean":"2023-10-08T01:12:45.267567+0000","last_became_active":"2023-10-08T01:12:45.260269+0000","last_became_peered":"2023-10-08T01:12:45.260269+0000","last_unstale":"2023-10-08T01:12:45.267567+0000","last_undegraded":"2023-10-08T01:12:45.267567+0000","last_fullsized":"2023-10-08T01:12:45.267567+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:35.797178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.269570+0000","last_change":"2023-10-08T01:12:45.266325+0000","last_active":"2023-10-08T01:12:45.269570+0000","last_peered":"2023-10-08T01:12:45.269570+0000","last_clean":"2023-10-08T01:12:45.269570+0000","last_became_active":"2023-10-08T01:12:45.266039+0000","last_became_peered":"2023-10-08T01:12:45.266039+0000","last_unstale":"2023-10-08T01:12:45.269570+0000","last_undegraded":"2023-10-08T01:12:45.269570+0000","last_fullsized":"2023-10-08T01:12:45.269570+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:40:52.062475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267295+0000","last_change":"2023-10-08T01:12:45.263929+0000","last_active":"2023-10-08T01:12:45.267295+0000","last_peered":"2023-10-08T01:12:45.267295+0000","last_clean":"2023-10-08T01:12:45.267295+0000","last_became_active":"2023-10-08T01:12:45.263639+0000","last_became_peered":"2023-10-08T01:12:45.263639+0000","last_unstale":"2023-10-08T01:12:45.267295+0000","last_undegraded":"2023-10-08T01:12:45.267295+0000","last_fullsized":"2023-10-08T01:12:45.267295+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:21.657522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.261546+0000","last_change":"2023-10-08T01:12:45.259791+0000","last_active":"2023-10-08T01:12:45.261546+0000","last_peered":"2023-10-08T01:12:45.261546+0000","last_clean":"2023-10-08T01:12:45.261546+0000","last_became_active":"2023-10-08T01:12:45.259349+0000","last_became_peered":"2023-10-08T01:12:45.259349+0000","last_unstale":"2023-10-08T01:12:45.261546+0000","last_undegraded":"2023-10-08T01:12:45.261546+0000","last_fullsized":"2023-10-08T01:12:45.261546+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:27.924521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.269170+0000","last_change":"2023-10-08T01:12:45.265310+0000","last_active":"2023-10-08T01:12:45.269170+0000","last_peered":"2023-10-08T01:12:45.269170+0000","last_clean":"2023-10-08T01:12:45.269170+0000","last_became_active":"2023-10-08T01:12:45.265004+0000","last_became_peered":"2023-10-08T01:12:45.265004+0000","last_unstale":"2023-10-08T01:12:45.269170+0000","last_undegraded":"2023-10-08T01:12:45.269170+0000","last_fullsized":"2023-10-08T01:12:45.269170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:24.043725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267559+0000","last_change":"2023-10-08T01:12:45.264214+0000","last_active":"2023-10-08T01:12:45.267559+0000","last_peered":"2023-10-08T01:12:45.267559+0000","last_clean":"2023-10-08T01:12:45.267559+0000","last_became_active":"2023-10-08T01:12:45.263798+0000","last_became_peered":"2023-10-08T01:12:45.263798+0000","last_unstale":"2023-10-08T01:12:45.267559+0000","last_undegraded":"2023-10-08T01:12:45.267559+0000","last_fullsized":"2023-10-08T01:12:45.267559+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:27.924521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273138+0000","last_change":"2023-10-08T01:12:45.269555+0000","last_active":"2023-10-08T01:12:45.273138+0000","last_peered":"2023-10-08T01:12:45.273138+0000","last_clean":"2023-10-08T01:12:45.273138+0000","last_became_active":"2023-10-08T01:12:45.269257+0000","last_became_peered":"2023-10-08T01:12:45.269257+0000","last_unstale":"2023-10-08T01:12:45.273138+0000","last_undegraded":"2023-10-08T01:12:45.273138+0000","last_fullsized":"2023-10-08T01:12:45.273138+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:29:44.683003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.270156+0000","last_change":"2023-10-08T01:12:45.262121+0000","last_active":"2023-10-08T01:12:45.270156+0000","last_peered":"2023-10-08T01:12:45.270156+0000","last_clean":"2023-10-08T01:12:45.270156+0000","last_became_active":"2023-10-08T01:12:45.261872+0000","last_became_peered":"2023-10-08T01:12:45.261872+0000","last_unstale":"2023-10-08T01:12:45.270156+0000","last_undegraded":"2023-10-08T01:12:45.270156+0000","last_fullsized":"2023-10-08T01:12:45.270156+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:12:49.301176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273170+0000","last_change":"2023-10-08T01:12:45.267334+0000","last_active":"2023-10-08T01:12:45.273170+0000","last_peered":"2023-10-08T01:12:45.273170+0000","last_clean":"2023-10-08T01:12:45.273170+0000","last_became_active":"2023-10-08T01:12:45.267086+0000","last_became_peered":"2023-10-08T01:12:45.267086+0000","last_unstale":"2023-10-08T01:12:45.273170+0000","last_undegraded":"2023-10-08T01:12:45.273170+0000","last_fullsized":"2023-10-08T01:12:45.273170+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:14.905006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273010+0000","last_change":"2023-10-08T01:12:45.267180+0000","last_active":"2023-10-08T01:12:45.273010+0000","last_peered":"2023-10-08T01:12:45.273010+0000","last_clean":"2023-10-08T01:12:45.273010+0000","last_became_active":"2023-10-08T01:12:45.266278+0000","last_became_peered":"2023-10-08T01:12:45.266278+0000","last_unstale":"2023-10-08T01:12:45.273010+0000","last_undegraded":"2023-10-08T01:12:45.273010+0000","last_fullsized":"2023-10-08T01:12:45.273010+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:38.262593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.252002+0000","last_change":"2023-10-08T01:12:45.248904+0000","last_active":"2023-10-08T01:12:45.252002+0000","last_peered":"2023-10-08T01:12:45.252002+0000","last_clean":"2023-10-08T01:12:45.252002+0000","last_became_active":"2023-10-08T01:12:45.247588+0000","last_became_peered":"2023-10-08T01:12:45.247588+0000","last_unstale":"2023-10-08T01:12:45.252002+0000","last_undegraded":"2023-10-08T01:12:45.252002+0000","last_fullsized":"2023-10-08T01:12:45.252002+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:05.481947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267475+0000","last_change":"2023-10-08T01:12:45.259228+0000","last_active":"2023-10-08T01:12:45.267475+0000","last_peered":"2023-10-08T01:12:45.267475+0000","last_clean":"2023-10-08T01:12:45.267475+0000","last_became_active":"2023-10-08T01:12:45.258886+0000","last_became_peered":"2023-10-08T01:12:45.258886+0000","last_unstale":"2023-10-08T01:12:45.267475+0000","last_undegraded":"2023-10-08T01:12:45.267475+0000","last_fullsized":"2023-10-08T01:12:45.267475+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:20.476830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273112+0000","last_change":"2023-10-08T01:12:45.266691+0000","last_active":"2023-10-08T01:12:45.273112+0000","last_peered":"2023-10-08T01:12:45.273112+0000","last_clean":"2023-10-08T01:12:45.273112+0000","last_became_active":"2023-10-08T01:12:45.266371+0000","last_became_peered":"2023-10-08T01:12:45.266371+0000","last_unstale":"2023-10-08T01:12:45.273112+0000","last_undegraded":"2023-10-08T01:12:45.273112+0000","last_fullsized":"2023-10-08T01:12:45.273112+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:37:01.279801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.263922+0000","last_change":"2023-10-08T01:12:45.262142+0000","last_active":"2023-10-08T01:12:45.263922+0000","last_peered":"2023-10-08T01:12:45.263922+0000","last_clean":"2023-10-08T01:12:45.263922+0000","last_became_active":"2023-10-08T01:12:45.261914+0000","last_became_peered":"2023-10-08T01:12:45.261914+0000","last_unstale":"2023-10-08T01:12:45.263922+0000","last_undegraded":"2023-10-08T01:12:45.263922+0000","last_fullsized":"2023-10-08T01:12:45.263922+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:50.337681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256336+0000","last_change":"2023-10-08T01:12:45.251090+0000","last_active":"2023-10-08T01:12:45.256336+0000","last_peered":"2023-10-08T01:12:45.256336+0000","last_clean":"2023-10-08T01:12:45.256336+0000","last_became_active":"2023-10-08T01:12:45.250706+0000","last_became_peered":"2023-10-08T01:12:45.250706+0000","last_unstale":"2023-10-08T01:12:45.256336+0000","last_undegraded":"2023-10-08T01:12:45.256336+0000","last_fullsized":"2023-10-08T01:12:45.256336+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:27.924521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.269097+0000","last_change":"2023-10-08T01:12:45.265743+0000","last_active":"2023-10-08T01:12:45.269097+0000","last_peered":"2023-10-08T01:12:45.269097+0000","last_clean":"2023-10-08T01:12:45.269097+0000","last_became_active":"2023-10-08T01:12:45.265415+0000","last_became_peered":"2023-10-08T01:12:45.265415+0000","last_unstale":"2023-10-08T01:12:45.269097+0000","last_undegraded":"2023-10-08T01:12:45.269097+0000","last_fullsized":"2023-10-08T01:12:45.269097+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:12:49.301176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.275186+0000","last_change":"2023-10-08T01:12:45.270010+0000","last_active":"2023-10-08T01:12:45.275186+0000","last_peered":"2023-10-08T01:12:45.275186+0000","last_clean":"2023-10-08T01:12:45.275186+0000","last_became_active":"2023-10-08T01:12:45.269688+0000","last_became_peered":"2023-10-08T01:12:45.269688+0000","last_unstale":"2023-10-08T01:12:45.275186+0000","last_undegraded":"2023-10-08T01:12:45.275186+0000","last_fullsized":"2023-10-08T01:12:45.275186+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:05.329576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.262999+0000","last_change":"2023-10-08T01:12:45.259491+0000","last_active":"2023-10-08T01:12:45.262999+0000","last_peered":"2023-10-08T01:12:45.262999+0000","last_clean":"2023-10-08T01:12:45.262999+0000","last_became_active":"2023-10-08T01:12:45.259148+0000","last_became_peered":"2023-10-08T01:12:45.259148+0000","last_unstale":"2023-10-08T01:12:45.262999+0000","last_undegraded":"2023-10-08T01:12:45.262999+0000","last_fullsized":"2023-10-08T01:12:45.262999+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:11:45.533105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256420+0000","last_change":"2023-10-08T01:12:45.251071+0000","last_active":"2023-10-08T01:12:45.256420+0000","last_peered":"2023-10-08T01:12:45.256420+0000","last_clean":"2023-10-08T01:12:45.256420+0000","last_became_active":"2023-10-08T01:12:45.250685+0000","last_became_peered":"2023-10-08T01:12:45.250685+0000","last_unstale":"2023-10-08T01:12:45.256420+0000","last_undegraded":"2023-10-08T01:12:45.256420+0000","last_fullsized":"2023-10-08T01:12:45.256420+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:14.679776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.275270+0000","last_change":"2023-10-08T01:12:45.271406+0000","last_active":"2023-10-08T01:12:45.275270+0000","last_peered":"2023-10-08T01:12:45.275270+0000","last_clean":"2023-10-08T01:12:45.275270+0000","last_became_active":"2023-10-08T01:12:45.271099+0000","last_became_peered":"2023-10-08T01:12:45.271099+0000","last_unstale":"2023-10-08T01:12:45.275270+0000","last_undegraded":"2023-10-08T01:12:45.275270+0000","last_fullsized":"2023-10-08T01:12:45.275270+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.644937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267540+0000","last_change":"2023-10-08T01:12:45.265131+0000","last_active":"2023-10-08T01:12:45.267540+0000","last_peered":"2023-10-08T01:12:45.267540+0000","last_clean":"2023-10-08T01:12:45.267540+0000","last_became_active":"2023-10-08T01:12:45.264855+0000","last_became_peered":"2023-10-08T01:12:45.264855+0000","last_unstale":"2023-10-08T01:12:45.267540+0000","last_undegraded":"2023-10-08T01:12:45.267540+0000","last_fullsized":"2023-10-08T01:12:45.267540+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:24.339200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.269137+0000","last_change":"2023-10-08T01:12:45.267074+0000","last_active":"2023-10-08T01:12:45.269137+0000","last_peered":"2023-10-08T01:12:45.269137+0000","last_clean":"2023-10-08T01:12:45.269137+0000","last_became_active":"2023-10-08T01:12:45.266799+0000","last_became_peered":"2023-10-08T01:12:45.266799+0000","last_unstale":"2023-10-08T01:12:45.269137+0000","last_undegraded":"2023-10-08T01:12:45.269137+0000","last_fullsized":"2023-10-08T01:12:45.269137+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:24.339200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"103'4","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.278365+0000","last_change":"2023-10-08T01:12:44.260210+0000","last_active":"2023-10-08T01:12:45.278365+0000","last_peered":"2023-10-08T01:12:45.278365+0000","last_clean":"2023-10-08T01:12:45.278365+0000","last_became_active":"2023-10-08T01:12:44.259451+0000","last_became_peered":"2023-10-08T01:12:44.259451+0000","last_unstale":"2023-10-08T01:12:45.278365+0000","last_undegraded":"2023-10-08T01:12:45.278365+0000","last_fullsized":"2023-10-08T01:12:45.278365+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:24:56.728600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267222+0000","last_change":"2023-10-08T01:12:45.260263+0000","last_active":"2023-10-08T01:12:45.267222+0000","last_peered":"2023-10-08T01:12:45.267222+0000","last_clean":"2023-10-08T01:12:45.267222+0000","last_became_active":"2023-10-08T01:12:45.259339+0000","last_became_peered":"2023-10-08T01:12:45.259339+0000","last_unstale":"2023-10-08T01:12:45.267222+0000","last_undegraded":"2023-10-08T01:12:45.267222+0000","last_fullsized":"2023-10-08T01:12:45.267222+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:53.138331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.276946+0000","last_change":"2023-10-08T01:12:44.257763+0000","last_active":"2023-10-08T01:12:45.276946+0000","last_peered":"2023-10-08T01:12:45.276946+0000","last_clean":"2023-10-08T01:12:45.276946+0000","last_became_active":"2023-10-08T01:12:44.257485+0000","last_became_peered":"2023-10-08T01:12:44.257485+0000","last_unstale":"2023-10-08T01:12:45.276946+0000","last_undegraded":"2023-10-08T01:12:45.276946+0000","last_fullsized":"2023-10-08T01:12:45.276946+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:13.674215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267706+0000","last_change":"2023-10-08T01:12:45.266031+0000","last_active":"2023-10-08T01:12:45.267706+0000","last_peered":"2023-10-08T01:12:45.267706+0000","last_clean":"2023-10-08T01:12:45.267706+0000","last_became_active":"2023-10-08T01:12:45.265673+0000","last_became_peered":"2023-10-08T01:12:45.265673+0000","last_unstale":"2023-10-08T01:12:45.267706+0000","last_undegraded":"2023-10-08T01:12:45.267706+0000","last_fullsized":"2023-10-08T01:12:45.267706+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:00.731750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.277816+0000","last_change":"2023-10-08T01:12:44.261721+0000","last_active":"2023-10-08T01:12:45.277816+0000","last_peered":"2023-10-08T01:12:45.277816+0000","last_clean":"2023-10-08T01:12:45.277816+0000","last_became_active":"2023-10-08T01:12:44.261441+0000","last_became_peered":"2023-10-08T01:12:44.261441+0000","last_unstale":"2023-10-08T01:12:45.277816+0000","last_undegraded":"2023-10-08T01:12:45.277816+0000","last_fullsized":"2023-10-08T01:12:45.277816+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:03:52.132770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256393+0000","last_change":"2023-10-08T01:12:45.253684+0000","last_active":"2023-10-08T01:12:45.256393+0000","last_peered":"2023-10-08T01:12:45.256393+0000","last_clean":"2023-10-08T01:12:45.256393+0000","last_became_active":"2023-10-08T01:12:45.253380+0000","last_became_peered":"2023-10-08T01:12:45.253380+0000","last_unstale":"2023-10-08T01:12:45.256393+0000","last_undegraded":"2023-10-08T01:12:45.256393+0000","last_fullsized":"2023-10-08T01:12:45.256393+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:02:36.541714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.277869+0000","last_change":"2023-10-08T01:12:44.260062+0000","last_active":"2023-10-08T01:12:45.277869+0000","last_peered":"2023-10-08T01:12:45.277869+0000","last_clean":"2023-10-08T01:12:45.277869+0000","last_became_active":"2023-10-08T01:12:44.259824+0000","last_became_peered":"2023-10-08T01:12:44.259824+0000","last_unstale":"2023-10-08T01:12:45.277869+0000","last_undegraded":"2023-10-08T01:12:45.277869+0000","last_fullsized":"2023-10-08T01:12:45.277869+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:23.038163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256400+0000","last_change":"2023-10-08T01:12:45.250916+0000","last_active":"2023-10-08T01:12:45.256400+0000","last_peered":"2023-10-08T01:12:45.256400+0000","last_clean":"2023-10-08T01:12:45.256400+0000","last_became_active":"2023-10-08T01:12:45.250345+0000","last_became_peered":"2023-10-08T01:12:45.250345+0000","last_unstale":"2023-10-08T01:12:45.256400+0000","last_undegraded":"2023-10-08T01:12:45.256400+0000","last_fullsized":"2023-10-08T01:12:45.256400+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:15.770121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.277716+0000","last_change":"2023-10-08T01:12:44.256552+0000","last_active":"2023-10-08T01:12:45.277716+0000","last_peered":"2023-10-08T01:12:45.277716+0000","last_clean":"2023-10-08T01:12:45.277716+0000","last_became_active":"2023-10-08T01:12:44.255636+0000","last_became_peered":"2023-10-08T01:12:44.255636+0000","last_unstale":"2023-10-08T01:12:45.277716+0000","last_undegraded":"2023-10-08T01:12:45.277716+0000","last_fullsized":"2023-10-08T01:12:45.277716+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:22:49.332119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.264436+0000","last_change":"2023-10-08T01:12:45.256479+0000","last_active":"2023-10-08T01:12:45.264436+0000","last_peered":"2023-10-08T01:12:45.264436+0000","last_clean":"2023-10-08T01:12:45.264436+0000","last_became_active":"2023-10-08T01:12:45.256045+0000","last_became_peered":"2023-10-08T01:12:45.256045+0000","last_unstale":"2023-10-08T01:12:45.264436+0000","last_undegraded":"2023-10-08T01:12:45.264436+0000","last_fullsized":"2023-10-08T01:12:45.264436+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:17:23.249506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.277162+0000","last_change":"2023-10-08T01:12:44.259650+0000","last_active":"2023-10-08T01:12:45.277162+0000","last_peered":"2023-10-08T01:12:45.277162+0000","last_clean":"2023-10-08T01:12:45.277162+0000","last_became_active":"2023-10-08T01:12:44.257886+0000","last_became_peered":"2023-10-08T01:12:44.257886+0000","last_unstale":"2023-10-08T01:12:45.277162+0000","last_undegraded":"2023-10-08T01:12:45.277162+0000","last_fullsized":"2023-10-08T01:12:45.277162+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:32.967100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.259505+0000","last_change":"2023-10-08T01:12:45.257533+0000","last_active":"2023-10-08T01:12:45.259505+0000","last_peered":"2023-10-08T01:12:45.259505+0000","last_clean":"2023-10-08T01:12:45.259505+0000","last_became_active":"2023-10-08T01:12:45.256693+0000","last_became_peered":"2023-10-08T01:12:45.256693+0000","last_unstale":"2023-10-08T01:12:45.259505+0000","last_undegraded":"2023-10-08T01:12:45.259505+0000","last_fullsized":"2023-10-08T01:12:45.259505+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:17:23.249506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.244565+0000","last_change":"2023-10-08T01:12:44.255146+0000","last_active":"2023-10-08T01:12:45.244565+0000","last_peered":"2023-10-08T01:12:45.244565+0000","last_clean":"2023-10-08T01:12:45.244565+0000","last_became_active":"2023-10-08T01:12:44.254849+0000","last_became_peered":"2023-10-08T01:12:44.254849+0000","last_unstale":"2023-10-08T01:12:45.244565+0000","last_undegraded":"2023-10-08T01:12:45.244565+0000","last_fullsized":"2023-10-08T01:12:45.244565+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:38:59.726189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.261528+0000","last_change":"2023-10-08T01:12:45.259623+0000","last_active":"2023-10-08T01:12:45.261528+0000","last_peered":"2023-10-08T01:12:45.261528+0000","last_clean":"2023-10-08T01:12:45.261528+0000","last_became_active":"2023-10-08T01:12:45.258942+0000","last_became_peered":"2023-10-08T01:12:45.258942+0000","last_unstale":"2023-10-08T01:12:45.261528+0000","last_undegraded":"2023-10-08T01:12:45.261528+0000","last_fullsized":"2023-10-08T01:12:45.261528+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:18:43.749953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.243057+0000","last_change":"2023-10-08T01:12:44.257115+0000","last_active":"2023-10-08T01:12:45.243057+0000","last_peered":"2023-10-08T01:12:45.243057+0000","last_clean":"2023-10-08T01:12:45.243057+0000","last_became_active":"2023-10-08T01:12:44.256867+0000","last_became_peered":"2023-10-08T01:12:44.256867+0000","last_unstale":"2023-10-08T01:12:45.243057+0000","last_undegraded":"2023-10-08T01:12:45.243057+0000","last_fullsized":"2023-10-08T01:12:45.243057+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:26.224539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.262629+0000","last_change":"2023-10-08T01:12:45.260215+0000","last_active":"2023-10-08T01:12:45.262629+0000","last_peered":"2023-10-08T01:12:45.262629+0000","last_clean":"2023-10-08T01:12:45.262629+0000","last_became_active":"2023-10-08T01:12:45.258758+0000","last_became_peered":"2023-10-08T01:12:45.258758+0000","last_unstale":"2023-10-08T01:12:45.262629+0000","last_undegraded":"2023-10-08T01:12:45.262629+0000","last_fullsized":"2023-10-08T01:12:45.262629+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:17:23.249506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.243963+0000","last_change":"2023-10-08T01:12:44.254566+0000","last_active":"2023-10-08T01:12:45.243963+0000","last_peered":"2023-10-08T01:12:45.243963+0000","last_clean":"2023-10-08T01:12:45.243963+0000","last_became_active":"2023-10-08T01:12:44.254136+0000","last_became_peered":"2023-10-08T01:12:44.254136+0000","last_unstale":"2023-10-08T01:12:45.243963+0000","last_undegraded":"2023-10-08T01:12:45.243963+0000","last_fullsized":"2023-10-08T01:12:45.243963+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:18.526189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256794+0000","last_change":"2023-10-08T01:12:45.252207+0000","last_active":"2023-10-08T01:12:45.256794+0000","last_peered":"2023-10-08T01:12:45.256794+0000","last_clean":"2023-10-08T01:12:45.256794+0000","last_became_active":"2023-10-08T01:12:45.251903+0000","last_became_peered":"2023-10-08T01:12:45.251903+0000","last_unstale":"2023-10-08T01:12:45.256794+0000","last_undegraded":"2023-10-08T01:12:45.256794+0000","last_fullsized":"2023-10-08T01:12:45.256794+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:35.797178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.245538+0000","last_change":"2023-10-08T01:12:44.258613+0000","last_active":"2023-10-08T01:12:45.245538+0000","last_peered":"2023-10-08T01:12:45.245538+0000","last_clean":"2023-10-08T01:12:45.245538+0000","last_became_active":"2023-10-08T01:12:44.258346+0000","last_became_peered":"2023-10-08T01:12:44.258346+0000","last_unstale":"2023-10-08T01:12:45.245538+0000","last_undegraded":"2023-10-08T01:12:45.245538+0000","last_fullsized":"2023-10-08T01:12:45.245538+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:06:36.611481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.270104+0000","last_change":"2023-10-08T01:12:45.261467+0000","last_active":"2023-10-08T01:12:45.270104+0000","last_peered":"2023-10-08T01:12:45.270104+0000","last_clean":"2023-10-08T01:12:45.270104+0000","last_became_active":"2023-10-08T01:12:45.261187+0000","last_became_peered":"2023-10-08T01:12:45.261187+0000","last_unstale":"2023-10-08T01:12:45.270104+0000","last_undegraded":"2023-10-08T01:12:45.270104+0000","last_fullsized":"2023-10-08T01:12:45.270104+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:47.947899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"103'3","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.278507+0000","last_change":"2023-10-08T01:12:44.256650+0000","last_active":"2023-10-08T01:12:45.278507+0000","last_peered":"2023-10-08T01:12:45.278507+0000","last_clean":"2023-10-08T01:12:45.278507+0000","last_became_active":"2023-10-08T01:12:44.256397+0000","last_became_peered":"2023-10-08T01:12:44.256397+0000","last_unstale":"2023-10-08T01:12:45.278507+0000","last_undegraded":"2023-10-08T01:12:45.278507+0000","last_fullsized":"2023-10-08T01:12:45.278507+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:11:44.527544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.275337+0000","last_change":"2023-10-08T01:12:45.272120+0000","last_active":"2023-10-08T01:12:45.275337+0000","last_peered":"2023-10-08T01:12:45.275337+0000","last_clean":"2023-10-08T01:12:45.275337+0000","last_became_active":"2023-10-08T01:12:45.271817+0000","last_became_peered":"2023-10-08T01:12:45.271817+0000","last_unstale":"2023-10-08T01:12:45.275337+0000","last_undegraded":"2023-10-08T01:12:45.275337+0000","last_fullsized":"2023-10-08T01:12:45.275337+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:18:43.749953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"103'6","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.280438+0000","last_change":"2023-10-08T01:12:44.258993+0000","last_active":"2023-10-08T01:12:45.280438+0000","last_peered":"2023-10-08T01:12:45.280438+0000","last_clean":"2023-10-08T01:12:45.280438+0000","last_became_active":"2023-10-08T01:12:44.258643+0000","last_became_peered":"2023-10-08T01:12:44.258643+0000","last_unstale":"2023-10-08T01:12:45.280438+0000","last_undegraded":"2023-10-08T01:12:45.280438+0000","last_fullsized":"2023-10-08T01:12:45.280438+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:38.124126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273517+0000","last_change":"2023-10-08T01:12:45.265232+0000","last_active":"2023-10-08T01:12:45.273517+0000","last_peered":"2023-10-08T01:12:45.273517+0000","last_clean":"2023-10-08T01:12:45.273517+0000","last_became_active":"2023-10-08T01:12:45.264914+0000","last_became_peered":"2023-10-08T01:12:45.264914+0000","last_unstale":"2023-10-08T01:12:45.273517+0000","last_undegraded":"2023-10-08T01:12:45.273517+0000","last_fullsized":"2023-10-08T01:12:45.273517+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:18:43.749953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.275773+0000","last_change":"2023-10-08T01:12:44.253139+0000","last_active":"2023-10-08T01:12:45.275773+0000","last_peered":"2023-10-08T01:12:45.275773+0000","last_clean":"2023-10-08T01:12:45.275773+0000","last_became_active":"2023-10-08T01:12:44.252848+0000","last_became_peered":"2023-10-08T01:12:44.252848+0000","last_unstale":"2023-10-08T01:12:45.275773+0000","last_undegraded":"2023-10-08T01:12:45.275773+0000","last_fullsized":"2023-10-08T01:12:45.275773+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:10:13.899444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.257962+0000","last_change":"2023-10-08T01:12:45.256229+0000","last_active":"2023-10-08T01:12:45.257962+0000","last_peered":"2023-10-08T01:12:45.257962+0000","last_clean":"2023-10-08T01:12:45.257962+0000","last_became_active":"2023-10-08T01:12:45.255974+0000","last_became_peered":"2023-10-08T01:12:45.255974+0000","last_unstale":"2023-10-08T01:12:45.257962+0000","last_undegraded":"2023-10-08T01:12:45.257962+0000","last_fullsized":"2023-10-08T01:12:45.257962+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:50:05.001736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.254495+0000","last_change":"2023-10-08T01:12:44.256072+0000","last_active":"2023-10-08T01:12:45.254495+0000","last_peered":"2023-10-08T01:12:45.254495+0000","last_clean":"2023-10-08T01:12:45.254495+0000","last_became_active":"2023-10-08T01:12:44.255819+0000","last_became_peered":"2023-10-08T01:12:44.255819+0000","last_unstale":"2023-10-08T01:12:45.254495+0000","last_undegraded":"2023-10-08T01:12:45.254495+0000","last_fullsized":"2023-10-08T01:12:45.254495+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:34.791617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256490+0000","last_change":"2023-10-08T01:12:45.250574+0000","last_active":"2023-10-08T01:12:45.256490+0000","last_peered":"2023-10-08T01:12:45.256490+0000","last_clean":"2023-10-08T01:12:45.256490+0000","last_became_active":"2023-10-08T01:12:45.250302+0000","last_became_peered":"2023-10-08T01:12:45.250302+0000","last_unstale":"2023-10-08T01:12:45.256490+0000","last_undegraded":"2023-10-08T01:12:45.256490+0000","last_fullsized":"2023-10-08T01:12:45.256490+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:13.043132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.247561+0000","last_change":"2023-10-08T01:12:44.261587+0000","last_active":"2023-10-08T01:12:45.247561+0000","last_peered":"2023-10-08T01:12:45.247561+0000","last_clean":"2023-10-08T01:12:45.247561+0000","last_became_active":"2023-10-08T01:12:44.260843+0000","last_became_peered":"2023-10-08T01:12:44.260843+0000","last_unstale":"2023-10-08T01:12:45.247561+0000","last_undegraded":"2023-10-08T01:12:45.247561+0000","last_fullsized":"2023-10-08T01:12:45.247561+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:32.967100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.263515+0000","last_change":"2023-10-08T01:12:45.261574+0000","last_active":"2023-10-08T01:12:45.263515+0000","last_peered":"2023-10-08T01:12:45.263515+0000","last_clean":"2023-10-08T01:12:45.263515+0000","last_became_active":"2023-10-08T01:12:45.261320+0000","last_became_peered":"2023-10-08T01:12:45.261320+0000","last_unstale":"2023-10-08T01:12:45.263515+0000","last_undegraded":"2023-10-08T01:12:45.263515+0000","last_fullsized":"2023-10-08T01:12:45.263515+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:22.853468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"103'8","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.281535+0000","last_change":"2023-10-08T01:12:44.257004+0000","last_active":"2023-10-08T01:12:45.281535+0000","last_peered":"2023-10-08T01:12:45.281535+0000","last_clean":"2023-10-08T01:12:45.281535+0000","last_became_active":"2023-10-08T01:12:44.256606+0000","last_became_peered":"2023-10-08T01:12:44.256606+0000","last_unstale":"2023-10-08T01:12:45.281535+0000","last_undegraded":"2023-10-08T01:12:45.281535+0000","last_fullsized":"2023-10-08T01:12:45.281535+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:04:03.871176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256356+0000","last_change":"2023-10-08T01:12:45.253802+0000","last_active":"2023-10-08T01:12:45.256356+0000","last_peered":"2023-10-08T01:12:45.256356+0000","last_clean":"2023-10-08T01:12:45.256356+0000","last_became_active":"2023-10-08T01:12:45.253454+0000","last_became_peered":"2023-10-08T01:12:45.253454+0000","last_unstale":"2023-10-08T01:12:45.256356+0000","last_undegraded":"2023-10-08T01:12:45.256356+0000","last_fullsized":"2023-10-08T01:12:45.256356+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:19:36.584592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.278172+0000","last_change":"2023-10-08T01:12:44.254394+0000","last_active":"2023-10-08T01:12:45.278172+0000","last_peered":"2023-10-08T01:12:45.278172+0000","last_clean":"2023-10-08T01:12:45.278172+0000","last_became_active":"2023-10-08T01:12:44.253775+0000","last_became_peered":"2023-10-08T01:12:44.253775+0000","last_unstale":"2023-10-08T01:12:45.278172+0000","last_undegraded":"2023-10-08T01:12:45.278172+0000","last_fullsized":"2023-10-08T01:12:45.278172+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:07:51.884673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267202+0000","last_change":"2023-10-08T01:12:45.260128+0000","last_active":"2023-10-08T01:12:45.267202+0000","last_peered":"2023-10-08T01:12:45.267202+0000","last_clean":"2023-10-08T01:12:45.267202+0000","last_became_active":"2023-10-08T01:12:45.259636+0000","last_became_peered":"2023-10-08T01:12:45.259636+0000","last_unstale":"2023-10-08T01:12:45.267202+0000","last_undegraded":"2023-10-08T01:12:45.267202+0000","last_fullsized":"2023-10-08T01:12:45.267202+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:54:36.621873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.280664+0000","last_change":"2023-10-08T01:12:44.261407+0000","last_active":"2023-10-08T01:12:45.280664+0000","last_peered":"2023-10-08T01:12:45.280664+0000","last_clean":"2023-10-08T01:12:45.280664+0000","last_became_active":"2023-10-08T01:12:44.261142+0000","last_became_peered":"2023-10-08T01:12:44.261142+0000","last_unstale":"2023-10-08T01:12:45.280664+0000","last_undegraded":"2023-10-08T01:12:45.280664+0000","last_fullsized":"2023-10-08T01:12:45.280664+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:16:06.145977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.264358+0000","last_change":"2023-10-08T01:12:45.257258+0000","last_active":"2023-10-08T01:12:45.264358+0000","last_peered":"2023-10-08T01:12:45.264358+0000","last_clean":"2023-10-08T01:12:45.264358+0000","last_became_active":"2023-10-08T01:12:45.256982+0000","last_became_peered":"2023-10-08T01:12:45.256982+0000","last_unstale":"2023-10-08T01:12:45.264358+0000","last_undegraded":"2023-10-08T01:12:45.264358+0000","last_fullsized":"2023-10-08T01:12:45.264358+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:00:35.797178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.245184+0000","last_change":"2023-10-08T01:12:44.254537+0000","last_active":"2023-10-08T01:12:45.245184+0000","last_peered":"2023-10-08T01:12:45.245184+0000","last_clean":"2023-10-08T01:12:45.245184+0000","last_became_active":"2023-10-08T01:12:44.254189+0000","last_became_peered":"2023-10-08T01:12:44.254189+0000","last_unstale":"2023-10-08T01:12:45.245184+0000","last_undegraded":"2023-10-08T01:12:45.245184+0000","last_fullsized":"2023-10-08T01:12:45.245184+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:04.324015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.267277+0000","last_change":"2023-10-08T01:12:45.263857+0000","last_active":"2023-10-08T01:12:45.267277+0000","last_peered":"2023-10-08T01:12:45.267277+0000","last_clean":"2023-10-08T01:12:45.267277+0000","last_became_active":"2023-10-08T01:12:45.263608+0000","last_became_peered":"2023-10-08T01:12:45.263608+0000","last_unstale":"2023-10-08T01:12:45.267277+0000","last_undegraded":"2023-10-08T01:12:45.267277+0000","last_fullsized":"2023-10-08T01:12:45.267277+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:03:03.239767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.274572+0000","last_change":"2023-10-08T01:12:44.259093+0000","last_active":"2023-10-08T01:12:45.274572+0000","last_peered":"2023-10-08T01:12:45.274572+0000","last_clean":"2023-10-08T01:12:45.274572+0000","last_became_active":"2023-10-08T01:12:44.258830+0000","last_became_peered":"2023-10-08T01:12:44.258830+0000","last_unstale":"2023-10-08T01:12:45.274572+0000","last_undegraded":"2023-10-08T01:12:45.274572+0000","last_fullsized":"2023-10-08T01:12:45.274572+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:37.153567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273119+0000","last_change":"2023-10-08T01:12:45.265449+0000","last_active":"2023-10-08T01:12:45.273119+0000","last_peered":"2023-10-08T01:12:45.273119+0000","last_clean":"2023-10-08T01:12:45.273119+0000","last_became_active":"2023-10-08T01:12:45.261198+0000","last_became_peered":"2023-10-08T01:12:45.261198+0000","last_unstale":"2023-10-08T01:12:45.273119+0000","last_undegraded":"2023-10-08T01:12:45.273119+0000","last_fullsized":"2023-10-08T01:12:45.273119+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:21:05.329576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.247114+0000","last_change":"2023-10-08T01:12:44.259595+0000","last_active":"2023-10-08T01:12:45.247114+0000","last_peered":"2023-10-08T01:12:45.247114+0000","last_clean":"2023-10-08T01:12:45.247114+0000","last_became_active":"2023-10-08T01:12:44.259154+0000","last_became_peered":"2023-10-08T01:12:44.259154+0000","last_unstale":"2023-10-08T01:12:45.247114+0000","last_undegraded":"2023-10-08T01:12:45.247114+0000","last_fullsized":"2023-10-08T01:12:45.247114+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:53:53.512756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.255215+0000","last_change":"2023-10-08T01:12:45.253394+0000","last_active":"2023-10-08T01:12:45.255215+0000","last_peered":"2023-10-08T01:12:45.255215+0000","last_clean":"2023-10-08T01:12:45.255215+0000","last_became_active":"2023-10-08T01:12:45.253141+0000","last_became_peered":"2023-10-08T01:12:45.253141+0000","last_unstale":"2023-10-08T01:12:45.255215+0000","last_undegraded":"2023-10-08T01:12:45.255215+0000","last_fullsized":"2023-10-08T01:12:45.255215+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:35:37.810355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.246253+0000","last_change":"2023-10-08T01:12:44.258783+0000","last_active":"2023-10-08T01:12:45.246253+0000","last_peered":"2023-10-08T01:12:45.246253+0000","last_clean":"2023-10-08T01:12:45.246253+0000","last_became_active":"2023-10-08T01:12:44.258514+0000","last_became_peered":"2023-10-08T01:12:44.258514+0000","last_unstale":"2023-10-08T01:12:45.246253+0000","last_undegraded":"2023-10-08T01:12:45.246253+0000","last_fullsized":"2023-10-08T01:12:45.246253+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:41:36.735984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.260425+0000","last_change":"2023-10-08T01:12:45.256389+0000","last_active":"2023-10-08T01:12:45.260425+0000","last_peered":"2023-10-08T01:12:45.260425+0000","last_clean":"2023-10-08T01:12:45.260425+0000","last_became_active":"2023-10-08T01:12:45.256113+0000","last_became_peered":"2023-10-08T01:12:45.256113+0000","last_unstale":"2023-10-08T01:12:45.260425+0000","last_undegraded":"2023-10-08T01:12:45.260425+0000","last_fullsized":"2023-10-08T01:12:45.260425+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:20.476830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.254102+0000","last_change":"2023-10-08T01:12:44.258530+0000","last_active":"2023-10-08T01:12:45.254102+0000","last_peered":"2023-10-08T01:12:45.254102+0000","last_clean":"2023-10-08T01:12:45.254102+0000","last_became_active":"2023-10-08T01:12:44.258248+0000","last_became_peered":"2023-10-08T01:12:44.258248+0000","last_unstale":"2023-10-08T01:12:45.254102+0000","last_undegraded":"2023-10-08T01:12:45.254102+0000","last_fullsized":"2023-10-08T01:12:45.254102+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:23.333638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.264475+0000","last_change":"2023-10-08T01:12:45.257550+0000","last_active":"2023-10-08T01:12:45.264475+0000","last_peered":"2023-10-08T01:12:45.264475+0000","last_clean":"2023-10-08T01:12:45.264475+0000","last_became_active":"2023-10-08T01:12:45.257267+0000","last_became_peered":"2023-10-08T01:12:45.257267+0000","last_unstale":"2023-10-08T01:12:45.264475+0000","last_undegraded":"2023-10-08T01:12:45.264475+0000","last_fullsized":"2023-10-08T01:12:45.264475+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:38.159129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"103'2","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273954+0000","last_change":"2023-10-08T01:12:44.258177+0000","last_active":"2023-10-08T01:12:45.273954+0000","last_peered":"2023-10-08T01:12:45.273954+0000","last_clean":"2023-10-08T01:12:45.273954+0000","last_became_active":"2023-10-08T01:12:44.257913+0000","last_became_peered":"2023-10-08T01:12:44.257913+0000","last_unstale":"2023-10-08T01:12:45.273954+0000","last_undegraded":"2023-10-08T01:12:45.273954+0000","last_fullsized":"2023-10-08T01:12:45.273954+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:47.905818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.268987+0000","last_change":"2023-10-08T01:12:45.261619+0000","last_active":"2023-10-08T01:12:45.268987+0000","last_peered":"2023-10-08T01:12:45.268987+0000","last_clean":"2023-10-08T01:12:45.268987+0000","last_became_active":"2023-10-08T01:12:45.261169+0000","last_became_peered":"2023-10-08T01:12:45.261169+0000","last_unstale":"2023-10-08T01:12:45.268987+0000","last_undegraded":"2023-10-08T01:12:45.268987+0000","last_fullsized":"2023-10-08T01:12:45.268987+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:47.947899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.243579+0000","last_change":"2023-10-08T01:12:44.254541+0000","last_active":"2023-10-08T01:12:45.243579+0000","last_peered":"2023-10-08T01:12:45.243579+0000","last_clean":"2023-10-08T01:12:45.243579+0000","last_became_active":"2023-10-08T01:12:44.254210+0000","last_became_peered":"2023-10-08T01:12:44.254210+0000","last_unstale":"2023-10-08T01:12:45.243579+0000","last_undegraded":"2023-10-08T01:12:45.243579+0000","last_fullsized":"2023-10-08T01:12:45.243579+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:00:40.188131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.275022+0000","last_change":"2023-10-08T01:12:45.268689+0000","last_active":"2023-10-08T01:12:45.275022+0000","last_peered":"2023-10-08T01:12:45.275022+0000","last_clean":"2023-10-08T01:12:45.275022+0000","last_became_active":"2023-10-08T01:12:45.268343+0000","last_became_peered":"2023-10-08T01:12:45.268343+0000","last_unstale":"2023-10-08T01:12:45.275022+0000","last_undegraded":"2023-10-08T01:12:45.275022+0000","last_fullsized":"2023-10-08T01:12:45.275022+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:46.644937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.243929+0000","last_change":"2023-10-08T01:12:44.260816+0000","last_active":"2023-10-08T01:12:45.243929+0000","last_peered":"2023-10-08T01:12:45.243929+0000","last_clean":"2023-10-08T01:12:45.243929+0000","last_became_active":"2023-10-08T01:12:44.260186+0000","last_became_peered":"2023-10-08T01:12:44.260186+0000","last_unstale":"2023-10-08T01:12:45.243929+0000","last_undegraded":"2023-10-08T01:12:45.243929+0000","last_fullsized":"2023-10-08T01:12:45.243929+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:38:55.428507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.275222+0000","last_change":"2023-10-08T01:12:45.271539+0000","last_active":"2023-10-08T01:12:45.275222+0000","last_peered":"2023-10-08T01:12:45.275222+0000","last_clean":"2023-10-08T01:12:45.275222+0000","last_became_active":"2023-10-08T01:12:45.271133+0000","last_became_peered":"2023-10-08T01:12:45.271133+0000","last_unstale":"2023-10-08T01:12:45.275222+0000","last_undegraded":"2023-10-08T01:12:45.275222+0000","last_fullsized":"2023-10-08T01:12:45.275222+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:22:47.947899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.246365+0000","last_change":"2023-10-08T01:12:44.247763+0000","last_active":"2023-10-08T01:12:45.246365+0000","last_peered":"2023-10-08T01:12:45.246365+0000","last_clean":"2023-10-08T01:12:45.246365+0000","last_became_active":"2023-10-08T01:12:44.247478+0000","last_became_peered":"2023-10-08T01:12:44.247478+0000","last_unstale":"2023-10-08T01:12:45.246365+0000","last_undegraded":"2023-10-08T01:12:45.246365+0000","last_fullsized":"2023-10-08T01:12:45.246365+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:04.476385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.264316+0000","last_change":"2023-10-08T01:12:45.261168+0000","last_active":"2023-10-08T01:12:45.264316+0000","last_peered":"2023-10-08T01:12:45.264316+0000","last_clean":"2023-10-08T01:12:45.264316+0000","last_became_active":"2023-10-08T01:12:45.260841+0000","last_became_peered":"2023-10-08T01:12:45.260841+0000","last_unstale":"2023-10-08T01:12:45.264316+0000","last_undegraded":"2023-10-08T01:12:45.264316+0000","last_fullsized":"2023-10-08T01:12:45.264316+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:24.339200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.245283+0000","last_change":"2023-10-08T01:12:44.252561+0000","last_active":"2023-10-08T01:12:45.245283+0000","last_peered":"2023-10-08T01:12:45.245283+0000","last_clean":"2023-10-08T01:12:45.245283+0000","last_became_active":"2023-10-08T01:12:44.252224+0000","last_became_peered":"2023-10-08T01:12:44.252224+0000","last_unstale":"2023-10-08T01:12:45.245283+0000","last_undegraded":"2023-10-08T01:12:45.245283+0000","last_fullsized":"2023-10-08T01:12:45.245283+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:47:19.471269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.273800+0000","last_change":"2023-10-08T01:12:45.266492+0000","last_active":"2023-10-08T01:12:45.273800+0000","last_peered":"2023-10-08T01:12:45.273800+0000","last_clean":"2023-10-08T01:12:45.273800+0000","last_became_active":"2023-10-08T01:12:45.266235+0000","last_became_peered":"2023-10-08T01:12:45.266235+0000","last_unstale":"2023-10-08T01:12:45.273800+0000","last_undegraded":"2023-10-08T01:12:45.273800+0000","last_fullsized":"2023-10-08T01:12:45.273800+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:24:27.924521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.252111+0000","last_change":"2023-10-08T01:12:44.257356+0000","last_active":"2023-10-08T01:12:45.252111+0000","last_peered":"2023-10-08T01:12:45.252111+0000","last_clean":"2023-10-08T01:12:45.252111+0000","last_became_active":"2023-10-08T01:12:44.257097+0000","last_became_peered":"2023-10-08T01:12:44.257097+0000","last_unstale":"2023-10-08T01:12:45.252111+0000","last_undegraded":"2023-10-08T01:12:45.252111+0000","last_fullsized":"2023-10-08T01:12:45.252111+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:38.124126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.256358+0000","last_change":"2023-10-08T01:12:45.251827+0000","last_active":"2023-10-08T01:12:45.256358+0000","last_peered":"2023-10-08T01:12:45.256358+0000","last_clean":"2023-10-08T01:12:45.256358+0000","last_became_active":"2023-10-08T01:12:45.251538+0000","last_became_peered":"2023-10-08T01:12:45.251538+0000","last_unstale":"2023-10-08T01:12:45.256358+0000","last_undegraded":"2023-10-08T01:12:45.256358+0000","last_fullsized":"2023-10-08T01:12:45.256358+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:18:43.749953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.245682+0000","last_change":"2023-10-08T01:12:44.258926+0000","last_active":"2023-10-08T01:12:45.245682+0000","last_peered":"2023-10-08T01:12:45.245682+0000","last_clean":"2023-10-08T01:12:45.245682+0000","last_became_active":"2023-10-08T01:12:44.257448+0000","last_became_peered":"2023-10-08T01:12:44.257448+0000","last_unstale":"2023-10-08T01:12:45.245682+0000","last_undegraded":"2023-10-08T01:12:45.245682+0000","last_fullsized":"2023-10-08T01:12:45.245682+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:24:21.847907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.274990+0000","last_change":"2023-10-08T01:12:45.268532+0000","last_active":"2023-10-08T01:12:45.274990+0000","last_peered":"2023-10-08T01:12:45.274990+0000","last_clean":"2023-10-08T01:12:45.274990+0000","last_became_active":"2023-10-08T01:12:45.268255+0000","last_became_peered":"2023-10-08T01:12:45.268255+0000","last_unstale":"2023-10-08T01:12:45.274990+0000","last_undegraded":"2023-10-08T01:12:45.274990+0000","last_fullsized":"2023-10-08T01:12:45.274990+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:50.363074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.250198+0000","last_change":"2023-10-08T01:12:44.255457+0000","last_active":"2023-10-08T01:12:45.250198+0000","last_peered":"2023-10-08T01:12:45.250198+0000","last_clean":"2023-10-08T01:12:45.250198+0000","last_became_active":"2023-10-08T01:12:44.254799+0000","last_became_peered":"2023-10-08T01:12:44.254799+0000","last_unstale":"2023-10-08T01:12:45.250198+0000","last_undegraded":"2023-10-08T01:12:45.250198+0000","last_fullsized":"2023-10-08T01:12:45.250198+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:14.764560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.263956+0000","last_change":"2023-10-08T01:12:45.259068+0000","last_active":"2023-10-08T01:12:45.263956+0000","last_peered":"2023-10-08T01:12:45.263956+0000","last_clean":"2023-10-08T01:12:45.263956+0000","last_became_active":"2023-10-08T01:12:45.258741+0000","last_became_peered":"2023-10-08T01:12:45.258741+0000","last_unstale":"2023-10-08T01:12:45.263956+0000","last_undegraded":"2023-10-08T01:12:45.263956+0000","last_fullsized":"2023-10-08T01:12:45.263956+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:44.230143+0000","last_clean_scrub_stamp":"2023-10-08T01:12:44.230143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:07:19.531750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"103'1","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.277302+0000","last_change":"2023-10-08T01:12:44.255645+0000","last_active":"2023-10-08T01:12:45.277302+0000","last_peered":"2023-10-08T01:12:45.277302+0000","last_clean":"2023-10-08T01:12:45.277302+0000","last_became_active":"2023-10-08T01:12:44.255361+0000","last_became_peered":"2023-10-08T01:12:44.255361+0000","last_unstale":"2023-10-08T01:12:45.277302+0000","last_undegraded":"2023-10-08T01:12:45.277302+0000","last_fullsized":"2023-10-08T01:12:45.277302+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:12:43.224582+0000","last_clean_scrub_stamp":"2023-10-08T01:12:43.224582+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:25.438819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"94'205","reported_seq":546,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-10-08T01:12:45.245557+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:12:45.245557+0000","last_peered":"2023-10-08T01:12:45.245557+0000","last_clean":"2023-10-08T01:12:45.245557+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:12:45.245557+0000","last_undegraded":"2023-10-08T01:12:45.245557+0000","last_fullsized":"2023-10-08T01:12:45.245557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":205,"log_dups_size":0,"ondisk_log_size":205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":884,"num_read_kb":1748,"num_write":319,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575052,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303216,"kb_used_data":1832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068624,"statfs":{"total":96636764160,"available":96326270976,"internally_reserved":0,"allocated":1875968,"data_stored":1210814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.68700000000000006,"15min":0.70699999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.89500000000000002,"5min":1.123,"15min":1.123},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68799999999999994,"15min":0.71299999999999997},"min":{"1min":0.52800000000000002,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.89200000000000002,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.69099999999999995,"15min":0.70499999999999996},"min":{"1min":0.48199999999999998,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.91200000000000003,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.69399999999999995,"15min":0.72599999999999998},"min":{"1min":0.47199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.74299999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71699999999999997,"15min":0.73899999999999999},"min":{"1min":0.52000000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.133,"5min":1.133,"15min":1.133},"last":0.505},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.71399999999999997,"15min":0.73199999999999998},"min":{"1min":0.51100000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.80600000000000005,"5min":1.167,"15min":1.167},"last":0.68400000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.69199999999999995,"15min":0.71599999999999997},"min":{"1min":0.503,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.81499999999999995,"5min":0.94699999999999995,"15min":0.998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.75900000000000001,"15min":0.75700000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.044,"5min":1.079,"15min":1.079},"last":0.66300000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.58099999999999996,"15min":0.58399999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.38},"max":{"1min":0.753,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.875,"5min":0.88,"15min":0.88300000000000001},"last":0.44800000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63600000000000001,"15min":0.622},"min":{"1min":0.375,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66500000000000004,"15min":0.65500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.53600000000000003}]},{"osd":6,"last update":"Sun Oct 8 01:12:11 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.73999999999999999,"15min":0.749},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.40400000000000003},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.71399999999999997,"15min":0.69799999999999995},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41599999999999998},"max":{"1min":0.85299999999999998,"5min":0.95199999999999996,"15min":0.98599999999999999},"last":0.69499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302644,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069196,"statfs":{"total":96636764160,"available":96326856704,"internally_reserved":0,"allocated":1290240,"data_stored":621042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67400000000000004,"15min":0.70299999999999996},"min":{"1min":0.44,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86299999999999999,"5min":0.93700000000000006,"15min":0.94199999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.66900000000000004,"15min":0.69799999999999995},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.38800000000000001},"max":{"1min":0.77100000000000002,"5min":0.91200000000000003,"15min":0.92800000000000005},"last":0.44400000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68400000000000005,"15min":0.72799999999999998},"min":{"1min":0.52000000000000002,"5min":0.40600000000000003,"15min":0.39000000000000001},"max":{"1min":0.88700000000000001,"5min":0.92400000000000004,"15min":2.984},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.68200000000000005,"15min":0.72599999999999998},"min":{"1min":0.47499999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.74199999999999999,"5min":1.018,"15min":3.0190000000000001},"last":0.63700000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66500000000000004,"15min":0.69299999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.69999999999999996,"15min":0.72899999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.96699999999999997},"last":0.751}]},{"osd":3,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.72299999999999998,"15min":0.753},"min":{"1min":0.56100000000000005,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.85499999999999998,"5min":0.95399999999999996,"15min":0.99199999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.501,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.81200000000000006}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.60599999999999998,"15min":0.59399999999999997},"min":{"1min":0.38600000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.627,"15min":0.62},"min":{"1min":0.45700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.871,"5min":1.087,"15min":1.087},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63100000000000001,"15min":0.61799999999999999},"min":{"1min":0.47899999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.0189999999999999,"5min":1.395,"15min":1.395},"last":0.872},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":1.0489999999999999,"5min":1.21,"15min":1.21},"last":0.88900000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.57699999999999996,"5min":0.44,"15min":0.40200000000000002},"max":{"1min":1.095,"5min":1.095,"15min":2.9489999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.73799999999999999,"15min":0.73399999999999999},"min":{"1min":0.60099999999999998,"5min":0.47099999999999997,"15min":0.45200000000000001},"max":{"1min":1.075,"5min":1.329,"15min":1.329},"last":0.92100000000000004}]}]},{"osd":1,"up_from":13,"seq":55834575052,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302644,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069196,"statfs":{"total":96636764160,"available":96326856704,"internally_reserved":0,"allocated":1290240,"data_stored":620570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.36499999999999999,"5min":0.27600000000000002,"15min":0.25700000000000001},"max":{"1min":0.55400000000000005,"5min":0.78900000000000003,"15min":0.79500000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.48699999999999999,"15min":0.503},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28199999999999997},"max":{"1min":0.54700000000000004,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.33900000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.46899999999999997,"15min":0.47999999999999998},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.26700000000000002},"max":{"1min":0.55500000000000005,"5min":0.72499999999999998,"15min":0.77300000000000002},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.51700000000000002,"15min":0.53400000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33700000000000002},"max":{"1min":0.60699999999999998,"5min":0.70599999999999996,"15min":0.84799999999999998},"last":0.52400000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.41799999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.62,"5min":0.76500000000000001,"15min":0.82099999999999995},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.53700000000000003,"15min":0.53800000000000003},"min":{"1min":0.39200000000000002,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.59799999999999998,"5min":1.5449999999999999,"15min":1.5449999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.52300000000000002,"5min":0.50600000000000001,"15min":0.40899999999999997},"max":{"1min":0.97799999999999998,"5min":1.452,"15min":1.452},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.747,"15min":0.76700000000000002},"min":{"1min":0.54400000000000004,"5min":0.51900000000000002,"15min":0.434},"max":{"1min":1.0029999999999999,"5min":1.1499999999999999,"15min":1.1839999999999999},"last":0.747}]},{"osd":5,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.755,"15min":0.77100000000000002},"min":{"1min":0.5,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.91900000000000004,"5min":1.274,"15min":1.3959999999999999},"last":0.623},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76500000000000001,"15min":0.78600000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.498},"max":{"1min":0.88400000000000001,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.72499999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.54600000000000004,"5min":0.497,"15min":0.48399999999999999},"max":{"1min":0.97499999999999998,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.77400000000000002,"15min":0.79100000000000004},"min":{"1min":0.56599999999999995,"5min":0.56000000000000005,"15min":0.54000000000000004},"max":{"1min":1.054,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.67500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.81200000000000006,"15min":0.82699999999999996},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.53200000000000003},"max":{"1min":1.026,"5min":1.621,"15min":1.621},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.755,"5min":0.81499999999999995,"15min":0.83499999999999996},"min":{"1min":0.59099999999999997,"5min":0.58099999999999996,"15min":0.51300000000000001},"max":{"1min":1.0209999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.69099999999999995}]}]},{"osd":0,"up_from":13,"seq":55834575051,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303216,"kb_used_data":1832,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068624,"statfs":{"total":96636764160,"available":96326270976,"internally_reserved":0,"allocated":1875968,"data_stored":1210778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.46000000000000002,"15min":0.46300000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.49399999999999999,"15min":0.501},"min":{"1min":0.32300000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.33700000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48699999999999999},"min":{"1min":0.28399999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.50800000000000001,"15min":0.53300000000000003},"min":{"1min":0.35699999999999998,"5min":0.33500000000000002,"15min":0.32900000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.40699999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54200000000000004,"15min":0.55000000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.497},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.48499999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.74199999999999999,"15min":0.754},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.45100000000000001},"max":{"1min":1.341,"5min":1.341,"15min":1.617},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76600000000000001,"15min":0.77700000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.244,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.75}]},{"osd":5,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77900000000000003,"15min":0.79300000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.81599999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.381,"5min":1.381,"15min":1.5289999999999999},"last":0.93400000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.78600000000000003,"15min":0.80400000000000005},"min":{"1min":0.52400000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.208,"5min":1.361,"15min":1.3779999999999999},"last":0.875},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.76300000000000001,"15min":0.78500000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.78600000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:12:05 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.81499999999999995},"min":{"1min":0.54900000000000004,"5min":0.51700000000000002,"15min":0.505},"max":{"1min":0.97699999999999998,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.81899999999999995,"15min":0.84199999999999997},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.1719999999999999,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.83399999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575051,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306732,"kb_used_data":1252,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065108,"statfs":{"total":96636764160,"available":96322670592,"internally_reserved":0,"allocated":1282048,"data_stored":620930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45200000000000001,"15min":0.45700000000000002},"min":{"1min":0.27200000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69699999999999995,"5min":0.752,"15min":0.752},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.50800000000000001,"15min":0.51200000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.27500000000000002},"max":{"1min":0.70499999999999996,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.43099999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.49199999999999999,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68300000000000005,"5min":2.786,"15min":2.786},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.55600000000000005,"15min":0.54400000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.625,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.37}]},{"osd":3,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.59399999999999997,"15min":0.59899999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.75,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.57199999999999995,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.78700000000000003,"5min":2.2170000000000001,"15min":2.2170000000000001},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.74199999999999999,"15min":0.76500000000000001},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.004,"5min":1.079,"15min":1.079},"last":0.746},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.76700000000000002,"15min":0.79200000000000004},"min":{"1min":0.46100000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.046,"5min":2.3559999999999999,"15min":2.3559999999999999},"last":0.88600000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.80200000000000005,"15min":0.83099999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.0089999999999999,"5min":2.488,"15min":2.488},"last":0.754},{"interface":"front","average":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81699999999999995},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.03,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.83399999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77900000000000003,"15min":0.79500000000000004},"min":{"1min":0.50900000000000001,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.038,"5min":2.653,"15min":2.653},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.77600000000000002,"15min":0.79600000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.016,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.84299999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:12:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0129999999999999,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.81000000000000005,"15min":0.81899999999999995},"min":{"1min":0.51800000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.0600000000000001,"5min":2.7490000000000001,"15min":2.7490000000000001},"last":0.85399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575051,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306740,"kb_used_data":1260,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94065100,"statfs":{"total":96636764160,"available":96322662400,"internally_reserved":0,"allocated":1290240,"data_stored":621912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.44,"15min":0.46100000000000002},"min":{"1min":0.30599999999999999,"5min":0.27200000000000002,"15min":0.249},"max":{"1min":0.54400000000000004,"5min":0.69999999999999996,"15min":0.80100000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.27900000000000003},"max":{"1min":0.57399999999999995,"5min":0.77100000000000002,"15min":0.86599999999999999},"last":0.438}]},{"osd":1,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46899999999999997,"15min":0.47499999999999998},"min":{"1min":0.32600000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.61699999999999999,"5min":0.69399999999999995,"15min":0.89700000000000002},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.504,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.31,"15min":0.31},"max":{"1min":0.629,"5min":0.82999999999999996,"15min":0.88100000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56799999999999995,"15min":0.58499999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70599999999999996,"5min":0.81200000000000006,"15min":0.89700000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.621},"min":{"1min":0.44400000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72199999999999998,"5min":0.79400000000000004,"15min":1.1060000000000001},"last":0.55800000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.72699999999999998,"15min":0.75700000000000001},"min":{"1min":0.47799999999999998,"5min":0.44400000000000001,"15min":0.41099999999999998},"max":{"1min":0.84599999999999997,"5min":1.095,"15min":1.3029999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.74299999999999999,"15min":0.78000000000000003},"min":{"1min":0.50700000000000001,"5min":0.499,"15min":0.46000000000000002},"max":{"1min":1.0580000000000001,"5min":1.117,"15min":1.413},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.747,"15min":0.77800000000000002},"min":{"1min":0.52600000000000002,"5min":0.51700000000000002,"15min":0.51000000000000001},"max":{"1min":1.0920000000000001,"5min":1.1899999999999999,"15min":1.3580000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.51700000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.874,"5min":0.97999999999999998,"15min":1.089},"last":0.63400000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.76300000000000001,"15min":0.79700000000000004},"min":{"1min":0.58199999999999996,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.379},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.76500000000000001,"15min":0.78700000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.44500000000000001},"max":{"1min":0.88300000000000001,"5min":1.1499999999999999,"15min":1.486},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:12:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97099999999999997,"5min":1.0840000000000001,"15min":1.389},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.77300000000000002,"15min":0.79700000000000004},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.5},"max":{"1min":0.93700000000000006,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310824,"kb_used_data":1248,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94061016,"statfs":{"total":96636764160,"available":96318480384,"internally_reserved":0,"allocated":1277952,"data_stored":620376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.68100000000000005},"min":{"1min":0.435,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.82499999999999996,"5min":0.999,"15min":0.999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.66900000000000004,"15min":0.69499999999999995},"min":{"1min":0.42599999999999999,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.90000000000000002,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.69899999999999995}]},{"osd":1,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.67400000000000004,"15min":0.70099999999999996},"min":{"1min":0.40100000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.94299999999999995,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.68500000000000005,"15min":0.70699999999999996},"min":{"1min":0.46899999999999997,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.88100000000000001,"5min":1.04,"15min":1.04},"last":0.75800000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.68500000000000005,"15min":0.73899999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":1.002},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.76500000000000001},"min":{"1min":0.51700000000000002,"5min":0.502,"15min":0.502},"max":{"1min":0.90900000000000003,"5min":1.323,"15min":1.323},"last":0.89100000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.53800000000000003,"5min":0.442,"15min":0.442},"max":{"1min":0.95799999999999996,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.746,"15min":0.78900000000000003},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.51800000000000002},"max":{"1min":0.88400000000000001,"5min":1.2829999999999999,"15min":1.454},"last":0.80800000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59599999999999997,"15min":0.60099999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.84499999999999997,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72299999999999998,"15min":0.72699999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.90400000000000003,"5min":0.95499999999999996,"15min":0.998},"last":0.65600000000000003}]},{"osd":6,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.84399999999999997,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.65400000000000003,"15min":0.64900000000000002},"min":{"1min":0.41299999999999998,"5min":0.40300000000000002,"15min":0.40200000000000002},"max":{"1min":0.86399999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.504}]},{"osd":7,"last update":"Sun Oct 8 01:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.67400000000000004,"15min":0.66600000000000004},"min":{"1min":0.501,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.70099999999999996,"15min":0.71199999999999997},"min":{"1min":0.503,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.88300000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.67700000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575051,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302648,"kb_used_data":1264,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069192,"statfs":{"total":96636764160,"available":96326852608,"internally_reserved":0,"allocated":1294336,"data_stored":621934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66900000000000004,"15min":0.69199999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.33700000000000002},"max":{"1min":0.83699999999999997,"5min":0.90600000000000003,"15min":0.95599999999999996},"last":0.503},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.71799999999999997,"15min":0.72099999999999997},"min":{"1min":0.495,"5min":0.48299999999999998,"15min":0.38},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67300000000000004}]},{"osd":1,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.49099999999999999,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.85799999999999998,"5min":1.026,"15min":1.026},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68100000000000005,"15min":0.69799999999999995},"min":{"1min":0.47099999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.78800000000000003,"5min":0.91900000000000004,"15min":0.99099999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.70099999999999996,"15min":0.71799999999999997},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.38700000000000001},"max":{"1min":0.90100000000000002,"5min":0.92500000000000004,"15min":0.94399999999999995},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.50700000000000001,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":0.91900000000000004,"5min":0.94799999999999995,"15min":1.0449999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.51900000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.96099999999999997},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.74399999999999999,"15min":0.75600000000000001},"min":{"1min":0.58299999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.626}]},{"osd":4,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59299999999999997,"15min":0.60199999999999998},"min":{"1min":0.40899999999999997,"5min":0.39100000000000001,"15min":0.35599999999999998},"max":{"1min":0.874,"5min":0.874,"15min":0.98199999999999998},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.67000000000000004,"15min":0.65500000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.84499999999999997,"5min":1.048,"15min":1.081},"last":0.58199999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.45300000000000001,"5min":0.44800000000000001,"15min":0.40699999999999997},"max":{"1min":0.78000000000000003,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68200000000000005,"15min":0.67400000000000004},"min":{"1min":0.50700000000000001,"5min":0.46800000000000003,"15min":0.41699999999999998},"max":{"1min":0.90200000000000002,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.65500000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:12:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.71199999999999997,"15min":0.69099999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.93200000000000005,"5min":1.095,"15min":1.095},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.47399999999999998,"15min":0.45800000000000002},"max":{"1min":0.86699999999999999,"5min":1.05,"15min":1.1779999999999999},"last":0.752}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:12:50.034 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T01:12:50.035 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T01:12:50.219 INFO:tasks.ceph.mds.c.smithi119.stderr:2023-10-08T01:12:50.219+0000 7f3365bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:12:50.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.255+0000 7f4737bff700 1 Processor -- start 2023-10-08T01:12:50.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.255+0000 7f4737bff700 1 -- start start 2023-10-08T01:12:50.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.256+0000 7f4737bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f47300c85c0 0x7f473006d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f4737bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730156ee0 0x7f4730069890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f4737bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f473006a210 con 0x7f4730069dd0 2023-10-08T01:12:50.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f4737bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4730156340 con 0x7f47300c85c0 2023-10-08T01:12:50.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f4737bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47300c8990 con 0x7f4730156ee0 2023-10-08T01:12:50.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f473599b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f47300c85c0 0x7f473006d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f473599b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f47300c85c0 0x7f473006d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47286/0 (socket says 172.21.15.119:47286) 2023-10-08T01:12:50.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f473599b700 1 -- 172.21.15.119:0/2958699312 learned_addr learned my addr 172.21.15.119:0/2958699312 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:50.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f473519a700 1 --2- 172.21.15.119:0/2958699312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730156ee0 0x7f4730069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f473599b700 1 -- 172.21.15.119:0/2958699312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730156ee0 msgr2=0x7f4730069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:50.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f473599b700 1 --2- 172.21.15.119:0/2958699312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730156ee0 0x7f4730069890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f473599b700 1 -- 172.21.15.119:0/2958699312 >> v1:172.21.15.119:6789/0 conn(0x7f4730069dd0 legacy=0x7f4730159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f473599b700 1 -- 172.21.15.119:0/2958699312 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473006ee20 con 0x7f47300c85c0 2023-10-08T01:12:50.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.257+0000 7f4734999700 1 -- 172.21.15.119:0/2958699312 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170747881 0 0) 0x7f473006a210 con 0x7f4730069dd0 2023-10-08T01:12:50.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f473599b700 1 --2- 172.21.15.119:0/2958699312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47300c85c0 0x7f473006d6c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099b0 tx=0x7f472c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee9b989a984c572 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:50.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4734999700 1 -- 172.21.15.119:0/2958699312 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00b860 con 0x7f47300c85c0 2023-10-08T01:12:50.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4734999700 1 -- 172.21.15.119:0/2958699312 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472c00b9c0 con 0x7f47300c85c0 2023-10-08T01:12:50.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4734999700 1 -- 172.21.15.119:0/2958699312 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472c00bd60 con 0x7f47300c85c0 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4737bff700 1 -- 172.21.15.119:0/2958699312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47300c85c0 msgr2=0x7f473006d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4737bff700 1 --2- 172.21.15.119:0/2958699312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47300c85c0 0x7f473006d6c0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099b0 tx=0x7f472c009250 comp rx=0 tx=0).stop 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4737bff700 1 -- 172.21.15.119:0/2958699312 shutdown_connections 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4737bff700 1 --2- 172.21.15.119:0/2958699312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47300c85c0 0x7f473006d6c0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4737bff700 1 --2- 172.21.15.119:0/2958699312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730156ee0 0x7f4730069890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4737bff700 1 -- 172.21.15.119:0/2958699312 >> 172.21.15.119:0/2958699312 conn(0x7f47300cc540 msgr2=0x7f47300cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.258+0000 7f4737bff700 1 -- 172.21.15.119:0/2958699312 shutdown_connections 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.259+0000 7f4737bff700 1 -- 172.21.15.119:0/2958699312 wait complete. 2023-10-08T01:12:50.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.260+0000 7f4737bff700 1 Processor -- start 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.260+0000 7f4737bff700 1 -- start start 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f4737bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730069dd0 0x7f4730155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f4737bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47300c85c0 0x7f47300b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f4737bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4730156ee0 0x7f47300b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f4737bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4730155d90 con 0x7f4730156ee0 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f4737bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47300c2a80 con 0x7f47300c85c0 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f4737bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47300a88c0 con 0x7f4730069dd0 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f473619c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4730156ee0 0x7f47300b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f473619c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4730156ee0 0x7f47300b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47292/0 (socket says 172.21.15.119:47292) 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f473619c700 1 -- 172.21.15.119:0/882127226 learned_addr learned my addr 172.21.15.119:0/882127226 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:50.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f473519a700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47300c85c0 0x7f47300b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.261+0000 7f473599b700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730069dd0 0x7f4730155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f473619c700 1 -- 172.21.15.119:0/882127226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730069dd0 msgr2=0x7f4730155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f473619c700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730069dd0 0x7f4730155850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f473619c700 1 -- 172.21.15.119:0/882127226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47300c85c0 msgr2=0x7f47300b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f473619c700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47300c85c0 0x7f47300b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f473619c700 1 -- 172.21.15.119:0/882127226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472c0094f0 con 0x7f4730156ee0 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f473619c700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4730156ee0 0x7f47300b1980 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4728001560 tx=0x7f4728001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f4725ffb700 1 -- 172.21.15.119:0/882127226 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47280040d0 con 0x7f4730156ee0 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f4725ffb700 1 -- 172.21.15.119:0/882127226 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f472800a7e0 con 0x7f4730156ee0 2023-10-08T01:12:50.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47300c0040 con 0x7f4730156ee0 2023-10-08T01:12:50.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f4725ffb700 1 -- 172.21.15.119:0/882127226 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f472800aba0 con 0x7f4730156ee0 2023-10-08T01:12:50.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.262+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47300c0600 con 0x7f4730156ee0 2023-10-08T01:12:50.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.263+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47300718b0 con 0x7f4730156ee0 2023-10-08T01:12:50.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.264+0000 7f4725ffb700 1 -- 172.21.15.119:0/882127226 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4728012420 con 0x7f4730156ee0 2023-10-08T01:12:50.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.265+0000 7f4725ffb700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4714066f30 0x7f47140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.265+0000 7f4725ffb700 1 -- 172.21.15.119:0/882127226 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f4728084350 con 0x7f4730156ee0 2023-10-08T01:12:50.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.265+0000 7f473599b700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4714066f30 0x7f47140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.266+0000 7f473599b700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4714066f30 0x7f47140693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099b0 tx=0x7f472c017000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:50.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.269+0000 7f4725ffb700 1 -- 172.21.15.119:0/882127226 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4728051430 con 0x7f4730156ee0 2023-10-08T01:12:50.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.534+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f473006dd80 con 0x7f4730156ee0 2023-10-08T01:12:50.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.538+0000 7f4725ffb700 1 -- 172.21.15.119:0/882127226 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4728054a50 con 0x7f4730156ee0 2023-10-08T01:12:50.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4714066f30 msgr2=0x7f47140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4714066f30 0x7f47140693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f472c0099b0 tx=0x7f472c017000 comp rx=0 tx=0).stop 2023-10-08T01:12:50.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4730156ee0 msgr2=0x7f47300b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4730156ee0 0x7f47300b1980 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f4728001560 tx=0x7f4728001770 comp rx=0 tx=0).stop 2023-10-08T01:12:50.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 shutdown_connections 2023-10-08T01:12:50.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4714066f30 0x7f47140693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4730069dd0 0x7f4730155850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4730156ee0 0x7f47300b1980 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 --2- 172.21.15.119:0/882127226 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47300c85c0 0x7f47300b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.540+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 >> 172.21.15.119:0/882127226 conn(0x7f47300cc540 msgr2=0x7f47300ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:50.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.541+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 shutdown_connections 2023-10-08T01:12:50.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.541+0000 7f4737bff700 1 -- 172.21.15.119:0/882127226 wait complete. 2023-10-08T01:12:50.543 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T01:12:50.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:12:50.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.739+0000 7f67d49ff700 1 Processor -- start 2023-10-08T01:12:50.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.739+0000 7f67d49ff700 1 -- start start 2023-10-08T01:12:50.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67d49ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00cbf30 0x7f67d00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67d49ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f67d00b1420 0x7f67d00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67d49ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d00cc300 con 0x7f67d00ccf90 2023-10-08T01:12:50.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67d49ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d0156bf0 con 0x7f67d00b1420 2023-10-08T01:12:50.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67d49ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00ae950 con 0x7f67d00cbf30 2023-10-08T01:12:50.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67ced9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f67d00b1420 0x7f67d00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67cdd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f67d00ccf90 0x7f67d00b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45414/0 (socket says 172.21.15.119:45414) 2023-10-08T01:12:50.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67cdd9b700 1 -- 172.21.15.119:0/3076404645 learned_addr learned my addr 172.21.15.119:0/3076404645 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:50.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67ced9d700 1 -- 172.21.15.119:0/3076404645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00cbf30 msgr2=0x7f67d00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:50.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67ce59c700 1 --2- 172.21.15.119:0/3076404645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00cbf30 0x7f67d00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67ced9d700 1 --2- 172.21.15.119:0/3076404645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00cbf30 0x7f67d00ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67ced9d700 1 -- 172.21.15.119:0/3076404645 >> v1:172.21.15.119:6789/0 conn(0x7f67d00ccf90 legacy=0x7f67d00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67ced9d700 1 -- 172.21.15.119:0/3076404645 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00ad370 con 0x7f67d00b1420 2023-10-08T01:12:50.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.741+0000 7f67cd59a700 1 -- 172.21.15.119:0/3076404645 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674242608 0 0) 0x7f67d00cc300 con 0x7f67d00ccf90 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.742+0000 7f67ced9d700 1 --2- 172.21.15.119:0/3076404645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00b1420 0x7f67d00c9740 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009e90 tx=0x7f67c8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76c086a7589064f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.742+0000 7f67cd59a700 1 -- 172.21.15.119:0/3076404645 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c800e070 con 0x7f67d00b1420 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.742+0000 7f67cd59a700 1 -- 172.21.15.119:0/3076404645 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c800bbf0 con 0x7f67d00b1420 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.742+0000 7f67cd59a700 1 -- 172.21.15.119:0/3076404645 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8012430 con 0x7f67d00b1420 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.742+0000 7f67d49ff700 1 -- 172.21.15.119:0/3076404645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00b1420 msgr2=0x7f67d00c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.742+0000 7f67d49ff700 1 --2- 172.21.15.119:0/3076404645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00b1420 0x7f67d00c9740 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009e90 tx=0x7f67c8009780 comp rx=0 tx=0).stop 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.743+0000 7f67d49ff700 1 -- 172.21.15.119:0/3076404645 shutdown_connections 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.743+0000 7f67d49ff700 1 --2- 172.21.15.119:0/3076404645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00b1420 0x7f67d00c9740 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.743+0000 7f67d49ff700 1 --2- 172.21.15.119:0/3076404645 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00cbf30 0x7f67d00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.743+0000 7f67d49ff700 1 -- 172.21.15.119:0/3076404645 >> 172.21.15.119:0/3076404645 conn(0x7f67d006a320 msgr2=0x7f67d00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:50.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.743+0000 7f67d49ff700 1 -- 172.21.15.119:0/3076404645 shutdown_connections 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.743+0000 7f67d49ff700 1 -- 172.21.15.119:0/3076404645 wait complete. 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.744+0000 7f67d49ff700 1 Processor -- start 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.744+0000 7f67d49ff700 1 -- start start 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67d49ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00b1420 0x7f67d006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67d49ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d00cbf30 0x7f67d0140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67d49ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00ccf90 0x7f67d0141300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67d49ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d0158960 con 0x7f67d00ccf90 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67d49ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d0157c40 con 0x7f67d00cbf30 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67d49ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d015d730 con 0x7f67d00b1420 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67ced9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00ccf90 0x7f67d0141300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67ced9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00ccf90 0x7f67d0141300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47316/0 (socket says 172.21.15.119:47316) 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67ced9d700 1 -- 172.21.15.119:0/4123722557 learned_addr learned my addr 172.21.15.119:0/4123722557 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67ce59c700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00b1420 0x7f67d006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.745+0000 7f67cdd9b700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d00cbf30 0x7f67d0140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67ced9d700 1 -- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00b1420 msgr2=0x7f67d006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67ced9d700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00b1420 0x7f67d006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67ced9d700 1 -- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d00cbf30 msgr2=0x7f67d0140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67ced9d700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d00cbf30 0x7f67d0140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67ced9d700 1 -- 172.21.15.119:0/4123722557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c8009a20 con 0x7f67d00ccf90 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67cdd9b700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d00cbf30 0x7f67d0140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67ced9d700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00ccf90 0x7f67d0141300 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f67c8013000 tx=0x7f67c8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:50.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67be7fc700 1 -- 172.21.15.119:0/4123722557 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c801d630 con 0x7f67d00ccf90 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67be7fc700 1 -- 172.21.15.119:0/4123722557 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c801d790 con 0x7f67d00ccf90 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67be7fc700 1 -- 172.21.15.119:0/4123722557 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c801db50 con 0x7f67d00ccf90 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d015da10 con 0x7f67d00ccf90 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.746+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d015d3e0 con 0x7f67d00ccf90 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.747+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d004c3c0 con 0x7f67d00ccf90 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.748+0000 7f67be7fc700 1 -- 172.21.15.119:0/4123722557 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f67c8024070 con 0x7f67d00ccf90 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.749+0000 7f67be7fc700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67b0066f30 0x7f67b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.749+0000 7f67be7fc700 1 -- 172.21.15.119:0/4123722557 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f67c8086fa0 con 0x7f67d00ccf90 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.750+0000 7f67ce59c700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67b0066f30 0x7f67b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:50.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.750+0000 7f67ce59c700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67b0066f30 0x7f67b00693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67c000acb0 tx=0x7f67c0008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:50.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.753+0000 7f67be7fc700 1 -- 172.21.15.119:0/4123722557 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67c8053fe0 con 0x7f67d00ccf90 2023-10-08T01:12:50.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.986+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67d00ad680 con 0x7f67d00ccf90 2023-10-08T01:12:50.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.987+0000 7f67be7fc700 1 -- 172.21.15.119:0/4123722557 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f67c8057600 con 0x7f67d00ccf90 2023-10-08T01:12:50.987 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:12:50.987 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1682556092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1682556092},{"type":"v1","addr":"172.21.15.184:6835","nonce":1682556092}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/691036731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":691036731},{"type":"v1","addr":"172.21.15.184:6834","nonce":691036731}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7182,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3636556734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3636556734},{"type":"v1","addr":"172.21.15.119:6834","nonce":3636556734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:44.715284+0000","modified":"2023-10-08T01:12:48.271445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7185":{"gid":7185,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3302512146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3302512146},{"type":"v1","addr":"172.21.15.119:6835","nonce":3302512146}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T01:12:50.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67b0066f30 msgr2=0x7f67b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67b0066f30 0x7f67b00693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f67c000acb0 tx=0x7f67c0008040 comp rx=0 tx=0).stop 2023-10-08T01:12:50.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00ccf90 msgr2=0x7f67d0141300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:50.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00ccf90 0x7f67d0141300 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f67c8013000 tx=0x7f67c8019000 comp rx=0 tx=0).stop 2023-10-08T01:12:50.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 shutdown_connections 2023-10-08T01:12:50.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67b0066f30 0x7f67b00693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d00b1420 0x7f67d006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d00ccf90 0x7f67d0141300 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 --2- 172.21.15.119:0/4123722557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d00cbf30 0x7f67d0140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:50.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.989+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 >> 172.21.15.119:0/4123722557 conn(0x7f67d006a320 msgr2=0x7f67d00cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:50.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.990+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 shutdown_connections 2023-10-08T01:12:50.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:50.990+0000 7f67d49ff700 1 -- 172.21.15.119:0/4123722557 wait complete. 2023-10-08T01:12:50.991 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 48 2023-10-08T01:12:51.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 48, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T01:12:44.715284+0000', 'modified': '2023-10-08T01:12:48.271445+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7185}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7185': {'gid': 7185, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6835/3302512146', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6833', 'nonce': 3302512146}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 3302512146}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T01:12:51.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T01:12:51.410 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T01:12:51.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.593+0000 7f771a79d700 1 Processor -- start 2023-10-08T01:12:51.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.594+0000 7f771a79d700 1 -- start start 2023-10-08T01:12:51.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f771a79d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7714150180 0x7f7714157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f771a79d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7714150550 0x7f77140c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f771a79d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771406e2a0 con 0x7f77140c7240 2023-10-08T01:12:51.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f771a79d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771406b2e0 con 0x7f7714150180 2023-10-08T01:12:51.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f771a79d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77140c7680 con 0x7f7714150550 2023-10-08T01:12:51.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f771979b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7714150180 0x7f7714157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f771979b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7714150180 0x7f7714157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47326/0 (socket says 172.21.15.119:47326) 2023-10-08T01:12:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f771979b700 1 -- 172.21.15.119:0/3028532732 learned_addr learned my addr 172.21.15.119:0/3028532732 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f7719f9c700 1 --1- 172.21.15.119:0/3028532732 >> v1:172.21.15.119:6789/0 conn(0x7f77140c7240 0x7f7714069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45424/0 (socket says 172.21.15.119:45424) 2023-10-08T01:12:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.595+0000 7f7718f9a700 1 --2- 172.21.15.119:0/3028532732 >> v2:172.21.15.184:3300/0 conn(0x7f7714150550 0x7f77140c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f771979b700 1 -- 172.21.15.119:0/3028532732 >> v2:172.21.15.184:3300/0 conn(0x7f7714150550 msgr2=0x7f77140c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f771979b700 1 --2- 172.21.15.119:0/3028532732 >> v2:172.21.15.184:3300/0 conn(0x7f7714150550 0x7f77140c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f771979b700 1 -- 172.21.15.119:0/3028532732 >> v1:172.21.15.119:6789/0 conn(0x7f77140c7240 legacy=0x7f7714069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f771979b700 1 -- 172.21.15.119:0/3028532732 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140c78e0 con 0x7f7714150180 2023-10-08T01:12:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f7718f9a700 1 --2- 172.21.15.119:0/3028532732 >> v2:172.21.15.184:3300/0 conn(0x7f7714150550 0x7f77140c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f770bfff700 1 -- 172.21.15.119:0/3028532732 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 882638657 0 0) 0x7f771406e2a0 con 0x7f77140c7240 2023-10-08T01:12:51.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f771979b700 1 --2- 172.21.15.119:0/3028532732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150180 0x7f7714157770 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f771406d1e0 tx=0x7f7710009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c95a661f79c04a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f770bfff700 1 -- 172.21.15.119:0/3028532732 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000b860 con 0x7f7714150180 2023-10-08T01:12:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f770bfff700 1 -- 172.21.15.119:0/3028532732 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771000b9c0 con 0x7f7714150180 2023-10-08T01:12:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.596+0000 7f770bfff700 1 -- 172.21.15.119:0/3028532732 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771000bd60 con 0x7f7714150180 2023-10-08T01:12:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.597+0000 7f771a79d700 1 -- 172.21.15.119:0/3028532732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150180 msgr2=0x7f7714157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.597+0000 7f771a79d700 1 --2- 172.21.15.119:0/3028532732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150180 0x7f7714157770 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f771406d1e0 tx=0x7f7710009250 comp rx=0 tx=0).stop 2023-10-08T01:12:51.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.597+0000 7f771a79d700 1 -- 172.21.15.119:0/3028532732 shutdown_connections 2023-10-08T01:12:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.597+0000 7f771a79d700 1 --2- 172.21.15.119:0/3028532732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150180 0x7f7714157770 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.597+0000 7f771a79d700 1 --2- 172.21.15.119:0/3028532732 >> v2:172.21.15.184:3300/0 conn(0x7f7714150550 0x7f77140c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.597+0000 7f771a79d700 1 -- 172.21.15.119:0/3028532732 >> 172.21.15.119:0/3028532732 conn(0x7f77140cc1f0 msgr2=0x7f77140cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.597+0000 7f771a79d700 1 -- 172.21.15.119:0/3028532732 shutdown_connections 2023-10-08T01:12:51.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.597+0000 7f771a79d700 1 -- 172.21.15.119:0/3028532732 wait complete. 2023-10-08T01:12:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.598+0000 7f771a79d700 1 Processor -- start 2023-10-08T01:12:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.598+0000 7f771a79d700 1 -- start start 2023-10-08T01:12:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.599+0000 7f771a79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77140c7240 0x7f77140ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.599+0000 7f771a79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7714150180 0x7f771414ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.599+0000 7f771a79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150550 0x7f77140b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.599+0000 7f771a79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77140ad4b0 con 0x7f7714150550 2023-10-08T01:12:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.599+0000 7f771a79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77140ac8a0 con 0x7f77140c7240 2023-10-08T01:12:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.599+0000 7f771a79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77140acb80 con 0x7f7714150180 2023-10-08T01:12:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150550 0x7f77140b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150550 0x7f77140b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47334/0 (socket says 172.21.15.119:47334) 2023-10-08T01:12:51.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 -- 172.21.15.119:0/922141661 learned_addr learned my addr 172.21.15.119:0/922141661 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7718f9a700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7714150180 0x7f771414ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f771979b700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77140c7240 0x7f77140ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 -- 172.21.15.119:0/922141661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7714150180 msgr2=0x7f771414ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7714150180 0x7f771414ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 -- 172.21.15.119:0/922141661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77140c7240 msgr2=0x7f77140ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77140c7240 0x7f77140ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 -- 172.21.15.119:0/922141661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77100094f0 con 0x7f7714150550 2023-10-08T01:12:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f7719f9c700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150550 0x7f77140b0cc0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c960 tx=0x7f770c001920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.600+0000 7f77097fa700 1 -- 172.21.15.119:0/922141661 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c004da0 con 0x7f7714150550 2023-10-08T01:12:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.601+0000 7f77097fa700 1 -- 172.21.15.119:0/922141661 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c0060f0 con 0x7f7714150550 2023-10-08T01:12:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.601+0000 7f771a79d700 1 -- 172.21.15.119:0/922141661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77140ace60 con 0x7f7714150550 2023-10-08T01:12:51.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.601+0000 7f77097fa700 1 -- 172.21.15.119:0/922141661 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0064b0 con 0x7f7714150550 2023-10-08T01:12:51.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.601+0000 7f771a79d700 1 -- 172.21.15.119:0/922141661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771414b340 con 0x7f7714150550 2023-10-08T01:12:51.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.603+0000 7f77097fa700 1 -- 172.21.15.119:0/922141661 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f770c0068a0 con 0x7f7714150550 2023-10-08T01:12:51.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.603+0000 7f77097fa700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f76f8066f30 0x7f76f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.604+0000 7f77097fa700 1 -- 172.21.15.119:0/922141661 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f770c084450 con 0x7f7714150550 2023-10-08T01:12:51.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.604+0000 7f771979b700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f76f8066f30 0x7f76f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.604+0000 7f771979b700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f76f8066f30 0x7f76f80693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f771406d1e0 tx=0x7f7710017000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.605+0000 7f772a10cb80 1 Processor -- start 2023-10-08T01:12:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.605+0000 7f772a10cb80 1 -- start start 2023-10-08T01:12:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.606+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 0x563aec8d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.606+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 0x563aec8f2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.606+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.606+0000 7f772a10cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aec7fea70 con 0x563aec8a7b80 2023-10-08T01:12:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.606+0000 7f772a10cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aec7fc270 con 0x563aec8a55f0 2023-10-08T01:12:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.606+0000 7f772a10cb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aec7f8060 con 0x563aec8f2480 2023-10-08T01:12:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.606+0000 7f7719f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7719f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47350/0 (socket says 172.21.15.119:47350) 2023-10-08T01:12:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7719f9c700 1 -- 172.21.15.119:0/3617910386 learned_addr learned my addr 172.21.15.119:0/3617910386 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f771979b700 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 0x563aec8d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7718f9a700 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 0x563aec8f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7719f9c700 1 -- 172.21.15.119:0/3617910386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 msgr2=0x563aec8d17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7719f9c700 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 0x563aec8d17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7719f9c700 1 -- 172.21.15.119:0/3617910386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 msgr2=0x563aec8f2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7719f9c700 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 0x563aec8f2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7719f9c700 1 -- 172.21.15.119:0/3617910386 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aec692270 con 0x563aec8a7b80 2023-10-08T01:12:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f7719f9c700 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a7ff0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f770c0516d0 tx=0x7f770c0826a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cefb86028db390 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f76fe7fc700 1 -- 172.21.15.119:0/3617910386 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0551d0 con 0x563aec8a7b80 2023-10-08T01:12:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f76fe7fc700 1 -- 172.21.15.119:0/3617910386 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c058420 con 0x563aec8a7b80 2023-10-08T01:12:51.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.607+0000 7f76fe7fc700 1 -- 172.21.15.119:0/3617910386 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c05c170 con 0x563aec8a7b80 2023-10-08T01:12:51.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 -- 172.21.15.119:0/3617910386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 msgr2=0x563aec8a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a7ff0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f770c0516d0 tx=0x7f770c0826a0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 -- 172.21.15.119:0/3617910386 shutdown_connections 2023-10-08T01:12:51.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 0x563aec8d17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a7ff0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3617910386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 0x563aec8f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 -- 172.21.15.119:0/3617910386 >> 172.21.15.119:0/3617910386 conn(0x563aec8a90c0 msgr2=0x563aec8a98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 -- 172.21.15.119:0/3617910386 shutdown_connections 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.608+0000 7f772a10cb80 1 -- 172.21.15.119:0/3617910386 wait complete. 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.609+0000 7f772a10cb80 1 -- start start 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.609+0000 7f772a10cb80 1 Processor -- start 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a7b80 0x563aec8feb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8f2480 0x563aec9016d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f772a10cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aec7a3890 con 0x563aec8f2480 2023-10-08T01:12:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f772a10cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aec7beea0 con 0x563aec8a7b80 2023-10-08T01:12:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f772a10cb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aec67ae30 con 0x563aec8a55f0 2023-10-08T01:12:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f772a10cb80 10 client.0 _ll_register_callbacks cb 0x563aec86b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:12:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7719f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8f2480 0x563aec9016d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7719f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8f2480 0x563aec9016d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47360/0 (socket says 172.21.15.119:47360) 2023-10-08T01:12:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7719f9c700 1 -- 172.21.15.119:0/3429638290 learned_addr learned my addr 172.21.15.119:0/3429638290 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f771979b700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7718f9a700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a7b80 0x563aec8feb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7719f9c700 1 -- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 msgr2=0x563aec8cbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7719f9c700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8cbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7719f9c700 1 -- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a7b80 msgr2=0x563aec8feb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7719f9c700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a7b80 0x563aec8feb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7719f9c700 1 -- 172.21.15.119:0/3429638290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770c05a040 con 0x563aec8f2480 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f7718f9a700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a7b80 0x563aec8feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.610+0000 7f771979b700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f7719f9c700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8f2480 0x563aec9016d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f770c0516d0 tx=0x7f770c069000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c33aa77972c13dd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f76f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:12:51.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f76f37fe700 1 -- 172.21.15.119:0/3429638290 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c053570 con 0x563aec8f2480 2023-10-08T01:12:51.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f76f37fe700 1 -- 172.21.15.119:0/3429638290 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f770c055430 con 0x563aec8f2480 2023-10-08T01:12:51.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 1 -- 172.21.15.119:0/3429638290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563aec89ccd0 con 0x563aec8f2480 2023-10-08T01:12:51.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f76f37fe700 1 -- 172.21.15.119:0/3429638290 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f770c0546c0 con 0x563aec8f2480 2023-10-08T01:12:51.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 10 client.7329 fetch_fsmap learned FSMap version 48 2023-10-08T01:12:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 1 -- 172.21.15.119:0/3429638290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x563aec634fe0 con 0x563aec8f2480 2023-10-08T01:12:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f76f37fe700 1 -- 172.21.15.119:0/3429638290 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563aec89ccd0 con 0x563aec8f2480 2023-10-08T01:12:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f76f37fe700 1 -- 172.21.15.119:0/3429638290 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f770c05bce0 con 0x563aec8f2480 2023-10-08T01:12:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 10 client.7329 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T01:12:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 10 client.7329 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T01:12:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 20 client.7329 populate_metadata read hostname 'smithi119' 2023-10-08T01:12:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] conn(0x563aec8d1ca0 0x563aec8d6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 4 client.7329 mds_command: new command op to 7185 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T01:12:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.611+0000 7f772a10cb80 1 -- 172.21.15.119:0/3429638290 --> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563aec879aa0 con 0x563aec8d1ca0 2023-10-08T01:12:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.612+0000 7f771979b700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] conn(0x563aec8d1ca0 0x563aec8d6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.612+0000 7f771979b700 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] conn(0x563aec8d1ca0 0x563aec8d6ac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32484972067b47d4 server_cookie=a99cbf8c3d56bdb in_seq=0 out_seq=0 2023-10-08T01:12:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.612+0000 7f76f37fe700 10 client.7329 ms_handle_connect on v2:172.21.15.119:6833/3302512146 2023-10-08T01:12:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f76f37fe700 1 -- 172.21.15.119:0/3429638290 <== mds.0 v2:172.21.15.119:6833/3302512146 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563aec879aa0 con 0x563aec8d1ca0 2023-10-08T01:12:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f76f37fe700 10 client.7329 handle_command_reply: tid=0 2023-10-08T01:12:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 client.7329 shutdown 2023-10-08T01:12:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 -- 172.21.15.119:0/3429638290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8f2480 msgr2=0x563aec9016d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8f2480 0x563aec9016d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f770c0516d0 tx=0x7f770c069000 comp rx=0 tx=0).stop 2023-10-08T01:12:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 -- 172.21.15.119:0/3429638290 shutdown_connections 2023-10-08T01:12:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] conn(0x563aec8d1ca0 0x563aec8d6ac0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8cbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8f2480 0x563aec9016d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3429638290 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a7b80 0x563aec8feb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f76f37fe700 0 client.7329 ms_handle_reset on v2:172.21.15.119:6833/3302512146 2023-10-08T01:12:51.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 -- 172.21.15.119:0/3429638290 >> 172.21.15.119:0/3429638290 conn(0x563aec8a90c0 msgr2=0x563aec8a7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:51.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 -- 172.21.15.119:0/3429638290 shutdown_connections 2023-10-08T01:12:51.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 1 -- 172.21.15.119:0/3429638290 wait complete. 2023-10-08T01:12:51.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.613+0000 7f772a10cb80 20 client.7329 trim_cache size 0 max 16384 2023-10-08T01:12:51.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.631+0000 7f772a10cb80 1 Processor -- start 2023-10-08T01:12:51.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.631+0000 7f772a10cb80 1 -- start start 2023-10-08T01:12:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8d6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8f2480 0x563aec8a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f772a10cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aec8d1610 con 0x563aec8a7b80 2023-10-08T01:12:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f772a10cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aec8a79e0 con 0x563aec8f2480 2023-10-08T01:12:51.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f772a10cb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aec7a3810 con 0x563aec8a55f0 2023-10-08T01:12:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f7718f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f7718f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47362/0 (socket says 172.21.15.119:47362) 2023-10-08T01:12:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f7718f9a700 1 -- 172.21.15.119:0/3852141086 learned_addr learned my addr 172.21.15.119:0/3852141086 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f7719f9c700 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8f2480 0x563aec8a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.632+0000 7f771979b700 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8d6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f7718f9a700 1 -- 172.21.15.119:0/3852141086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 msgr2=0x563aec8d6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f7718f9a700 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8d6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f7718f9a700 1 -- 172.21.15.119:0/3852141086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8f2480 msgr2=0x563aec8a9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f7718f9a700 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8f2480 0x563aec8a9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f7718f9a700 1 -- 172.21.15.119:0/3852141086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aec9018c0 con 0x563aec8a7b80 2023-10-08T01:12:51.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f7718f9a700 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a96c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7704007a00 tx=0x7f7704008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae0642e3824425b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f76feffd700 1 -- 172.21.15.119:0/3852141086 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040148f0 con 0x563aec8a7b80 2023-10-08T01:12:51.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f76feffd700 1 -- 172.21.15.119:0/3852141086 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7704014a70 con 0x563aec8a7b80 2023-10-08T01:12:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.633+0000 7f76feffd700 1 -- 172.21.15.119:0/3852141086 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7704014e30 con 0x563aec8a7b80 2023-10-08T01:12:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 -- 172.21.15.119:0/3852141086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 msgr2=0x563aec8a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a96c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f7704007a00 tx=0x7f7704008040 comp rx=0 tx=0).stop 2023-10-08T01:12:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 -- 172.21.15.119:0/3852141086 shutdown_connections 2023-10-08T01:12:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8a55f0 0x563aec8d6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8f2480 0x563aec8a9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 --2- 172.21.15.119:0/3852141086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a96c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 -- 172.21.15.119:0/3852141086 >> 172.21.15.119:0/3852141086 conn(0x563aec8a90c0 msgr2=0x563aec8d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 -- 172.21.15.119:0/3852141086 shutdown_connections 2023-10-08T01:12:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 -- 172.21.15.119:0/3852141086 wait complete. 2023-10-08T01:12:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.634+0000 7f772a10cb80 1 -- start start 2023-10-08T01:12:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.635+0000 7f772a10cb80 1 Processor -- start 2023-10-08T01:12:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 0x563aec93cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f772a10cb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 0x563aec8a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f772a10cb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aec8ef490 con 0x563aec8a7b80 2023-10-08T01:12:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f772a10cb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aec954f10 con 0x563aec8a55f0 2023-10-08T01:12:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f772a10cb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x563aec8ee810 con 0x563aec8f2480 2023-10-08T01:12:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f772a10cb80 10 client.0 _ll_register_callbacks cb 0x563aec86b2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:12:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7718f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7718f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47368/0 (socket says 172.21.15.119:47368) 2023-10-08T01:12:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7718f9a700 1 -- 172.21.15.119:0/2348649975 learned_addr learned my addr 172.21.15.119:0/2348649975 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7719f9c700 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 0x563aec8a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f771979b700 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 0x563aec93cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7718f9a700 1 -- 172.21.15.119:0/2348649975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 msgr2=0x563aec8a99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7718f9a700 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 0x563aec8a99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7718f9a700 1 -- 172.21.15.119:0/2348649975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 msgr2=0x563aec93cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7718f9a700 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 0x563aec93cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.636+0000 7f7718f9a700 1 -- 172.21.15.119:0/2348649975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f770400e040 con 0x563aec8a7b80 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f7718f9a700 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a9490 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7704007010 tx=0x7f7704020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344cd2126db738b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f76f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f76f37fe700 1 -- 172.21.15.119:0/2348649975 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77040168f0 con 0x563aec8a7b80 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 1 -- 172.21.15.119:0/2348649975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77140afde0 con 0x563aec8a7b80 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f76f37fe700 1 -- 172.21.15.119:0/2348649975 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7704016a50 con 0x563aec8a7b80 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f76f37fe700 1 -- 172.21.15.119:0/2348649975 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7704016e60 con 0x563aec8a7b80 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 10 client.7335 fetch_fsmap learned FSMap version 48 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 1 -- 172.21.15.119:0/2348649975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x563aec7c6780 con 0x563aec8a7b80 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f76f37fe700 1 -- 172.21.15.119:0/2348649975 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77140afde0 con 0x563aec8a7b80 2023-10-08T01:12:51.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f76f37fe700 1 -- 172.21.15.119:0/2348649975 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 48) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f770401e400 con 0x563aec8a7b80 2023-10-08T01:12:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 10 client.7335 fetch_fsmap finished waiting for FSMap version 48 2023-10-08T01:12:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 10 client.7335 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T01:12:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 20 client.7335 populate_metadata read hostname 'smithi119' 2023-10-08T01:12:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] conn(0x563aec89e260 0x563aec89e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 4 client.7335 mds_command: new command op to 7185 tid=0[{"prefix": "status"}] 2023-10-08T01:12:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.637+0000 7f772a10cb80 1 -- 172.21.15.119:0/2348649975 --> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f770c058420 con 0x563aec89e260 2023-10-08T01:12:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f771979b700 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] conn(0x563aec89e260 0x563aec89e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f76f37fe700 10 client.7335 ms_handle_connect on v2:172.21.15.119:6833/3302512146 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f771979b700 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] conn(0x563aec89e260 0x563aec89e630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eeb9d13717c965e8 server_cookie=c86ac1378fbe9e9c in_seq=0 out_seq=0 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f76f37fe700 1 -- 172.21.15.119:0/2348649975 <== mds.0 v2:172.21.15.119:6833/3302512146 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x7f770c058420 con 0x563aec89e260 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f76f37fe700 10 client.7335 handle_command_reply: tid=0 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 client.7335 shutdown 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 -- 172.21.15.119:0/2348649975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 msgr2=0x563aec8a9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a9490 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7704007010 tx=0x7f7704020000 comp rx=0 tx=0).stop 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 -- 172.21.15.119:0/2348649975 shutdown_connections 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.119:6833/3302512146,v1:172.21.15.119:6835/3302512146] conn(0x563aec89e260 0x563aec89e630 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x563aec8f2480 0x563aec8a99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x563aec8a55f0 0x563aec93cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 --2- 172.21.15.119:0/2348649975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563aec8a7b80 0x563aec8a9490 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f76f37fe700 0 client.7335 ms_handle_reset on v2:172.21.15.119:6833/3302512146 2023-10-08T01:12:51.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.638+0000 7f772a10cb80 1 -- 172.21.15.119:0/2348649975 >> 172.21.15.119:0/2348649975 conn(0x563aec8a90c0 msgr2=0x563aec8d6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:51.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f772a10cb80 1 -- 172.21.15.119:0/2348649975 shutdown_connections 2023-10-08T01:12:51.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f772a10cb80 1 -- 172.21.15.119:0/2348649975 wait complete. 2023-10-08T01:12:51.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f772a10cb80 20 client.7335 trim_cache size 0 max 16384 2023-10-08T01:12:51.788 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:12:51.788 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T01:12:51.788 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T01:12:51.788 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7185, 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.3998661720000003, 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 48, 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 104, 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 104, 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 12.174465873000001 2023-10-08T01:12:51.789 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 -- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f76f8066f30 msgr2=0x7f76f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f76f8066f30 0x7f76f80693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f771406d1e0 tx=0x7f7710017000 comp rx=0 tx=0).stop 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 -- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150550 msgr2=0x7f77140b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150550 0x7f77140b0cc0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c960 tx=0x7f770c001920 comp rx=0 tx=0).stop 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 -- 172.21.15.119:0/922141661 shutdown_connections 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f76f8066f30 0x7f76f80693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7714150550 0x7f77140b0cc0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77140c7240 0x7f77140ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.639+0000 7f771a79d700 1 --2- 172.21.15.119:0/922141661 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7714150180 0x7f771414ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.640+0000 7f771a79d700 1 -- 172.21.15.119:0/922141661 >> 172.21.15.119:0/922141661 conn(0x7f77140cc1f0 msgr2=0x7f7714154d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:51.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.640+0000 7f771a79d700 1 -- 172.21.15.119:0/922141661 shutdown_connections 2023-10-08T01:12:51.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.640+0000 7f771a79d700 1 -- 172.21.15.119:0/922141661 wait complete. 2023-10-08T01:12:51.792 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T01:12:51.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.973+0000 7fdb8759e700 1 Processor -- start 2023-10-08T01:12:51.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.974+0000 7fdb8759e700 1 -- start start 2023-10-08T01:12:51.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb8759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb8759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb88150550 0x7fdb880c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb8759e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdb880c7240 0x7fdb88069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb8759e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb880c76e0 con 0x7fdb88150550 2023-10-08T01:12:51.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb8759e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8806e2a0 con 0x7fdb880c7240 2023-10-08T01:12:51.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb8759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8806b2e0 con 0x7fdb88150180 2023-10-08T01:12:51.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb85d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb88150550 0x7fdb880c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb85d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdb88150550 0x7fdb880c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47370/0 (socket says 172.21.15.119:47370) 2023-10-08T01:12:51.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb85d9b700 1 -- 172.21.15.119:0/4016840240 learned_addr learned my addr 172.21.15.119:0/4016840240 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:51.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb8659c700 1 --2- 172.21.15.119:0/4016840240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.975+0000 7fdb86d9d700 1 --2- 172.21.15.119:0/4016840240 >> v2:172.21.15.184:3300/0 conn(0x7fdb880c7240 0x7fdb88069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb85d9b700 1 -- 172.21.15.119:0/4016840240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 msgr2=0x7fdb88157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb85d9b700 1 --2- 172.21.15.119:0/4016840240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb85d9b700 1 -- 172.21.15.119:0/4016840240 >> v2:172.21.15.184:3300/0 conn(0x7fdb880c7240 msgr2=0x7fdb88069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:51.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb85d9b700 1 --2- 172.21.15.119:0/4016840240 >> v2:172.21.15.184:3300/0 conn(0x7fdb880c7240 0x7fdb88069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb85d9b700 1 -- 172.21.15.119:0/4016840240 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb880c78e0 con 0x7fdb88150550 2023-10-08T01:12:51.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb86d9d700 1 --2- 172.21.15.119:0/4016840240 >> v2:172.21.15.184:3300/0 conn(0x7fdb880c7240 0x7fdb88069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:51.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb85d9b700 1 --2- 172.21.15.119:0/4016840240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb88150550 0x7fdb880c6d00 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdb8806ad70 tx=0x7fdb7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452754a71db13ace server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb8559a700 1 -- 172.21.15.119:0/4016840240 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00b860 con 0x7fdb88150550 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb8559a700 1 -- 172.21.15.119:0/4016840240 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c00b9c0 con 0x7fdb88150550 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.976+0000 7fdb8559a700 1 -- 172.21.15.119:0/4016840240 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00bd60 con 0x7fdb88150550 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 -- 172.21.15.119:0/4016840240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb88150550 msgr2=0x7fdb880c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 --2- 172.21.15.119:0/4016840240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb88150550 0x7fdb880c6d00 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdb8806ad70 tx=0x7fdb7c009250 comp rx=0 tx=0).stop 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 -- 172.21.15.119:0/4016840240 shutdown_connections 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 --2- 172.21.15.119:0/4016840240 >> v2:172.21.15.184:3300/0 conn(0x7fdb880c7240 0x7fdb88069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 --2- 172.21.15.119:0/4016840240 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 --2- 172.21.15.119:0/4016840240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb88150550 0x7fdb880c6d00 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 -- 172.21.15.119:0/4016840240 >> 172.21.15.119:0/4016840240 conn(0x7fdb880cc1f0 msgr2=0x7fdb880cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:51.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 -- 172.21.15.119:0/4016840240 shutdown_connections 2023-10-08T01:12:51.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.977+0000 7fdb8759e700 1 -- 172.21.15.119:0/4016840240 wait complete. 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.978+0000 7fdb8759e700 1 Processor -- start 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.978+0000 7fdb8759e700 1 -- start start 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.979+0000 7fdb8759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb880c7240 0x7fdb880b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.979+0000 7fdb8759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.979+0000 7fdb8759e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb88150550 0x7fdb88155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.979+0000 7fdb8759e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880b1130 con 0x7fdb880c7240 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.979+0000 7fdb8759e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88147590 con 0x7fdb88150550 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.979+0000 7fdb8759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb880b0bb0 con 0x7fdb88150180 2023-10-08T01:12:51.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb880c7240 0x7fdb880b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb880c7240 0x7fdb880b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47372/0 (socket says 172.21.15.119:47372) 2023-10-08T01:12:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 -- 172.21.15.119:0/3091337194 learned_addr learned my addr 172.21.15.119:0/3091337194 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb85d9b700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb86d9d700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb88150550 0x7fdb88155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 -- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 msgr2=0x7fdb88154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 -- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb88150550 msgr2=0x7fdb88155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb88150550 0x7fdb88155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 -- 172.21.15.119:0/3091337194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c0094f0 con 0x7fdb880c7240 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb85d9b700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb86d9d700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb88150550 0x7fdb88155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8659c700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb880c7240 0x7fdb880b0670 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800d890 tx=0x7fdb78008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb767fc700 1 -- 172.21.15.119:0/3091337194 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800b600 con 0x7fdb880c7240 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb767fc700 1 -- 172.21.15.119:0/3091337194 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7800b760 con 0x7fdb880c7240 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb767fc700 1 -- 172.21.15.119:0/3091337194 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7800bae0 con 0x7fdb880c7240 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.980+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb880b0e50 con 0x7fdb880c7240 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.981+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8814c9c0 con 0x7fdb880c7240 2023-10-08T01:12:51.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.982+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8814d010 con 0x7fdb880c7240 2023-10-08T01:12:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.982+0000 7fdb767fc700 1 -- 172.21.15.119:0/3091337194 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb7801e460 con 0x7fdb880c7240 2023-10-08T01:12:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.983+0000 7fdb767fc700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb68066f30 0x7fdb680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.984+0000 7fdb767fc700 1 -- 172.21.15.119:0/3091337194 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fdb78084d70 con 0x7fdb880c7240 2023-10-08T01:12:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.984+0000 7fdb85d9b700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb68066f30 0x7fdb680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:51.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.984+0000 7fdb85d9b700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb68066f30 0x7fdb680693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb8806ad70 tx=0x7fdb7c018000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:51.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:51.987+0000 7fdb767fc700 1 -- 172.21.15.119:0/3091337194 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb78051e50 con 0x7fdb880c7240 2023-10-08T01:12:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.223+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdb8814c540 con 0x7fdb880c7240 2023-10-08T01:12:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.223+0000 7fdb767fc700 1 -- 172.21.15.119:0/3091337194 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v48) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdb8814c540 con 0x7fdb880c7240 2023-10-08T01:12:52.224 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T01:12:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.225+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb68066f30 msgr2=0x7fdb680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.225+0000 7fdb8759e700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb68066f30 0x7fdb680693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fdb8806ad70 tx=0x7fdb7c018000 comp rx=0 tx=0).stop 2023-10-08T01:12:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.225+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb880c7240 msgr2=0x7fdb880b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.225+0000 7fdb8759e700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb880c7240 0x7fdb880b0670 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800d890 tx=0x7fdb78008040 comp rx=0 tx=0).stop 2023-10-08T01:12:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.226+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 shutdown_connections 2023-10-08T01:12:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.226+0000 7fdb8759e700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb68066f30 0x7fdb680693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.226+0000 7fdb8759e700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb88150550 0x7fdb88155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:52.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.226+0000 7fdb8759e700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb880c7240 0x7fdb880b0670 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:52.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.226+0000 7fdb8759e700 1 --2- 172.21.15.119:0/3091337194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb88150180 0x7fdb88154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:52.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.226+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 >> 172.21.15.119:0/3091337194 conn(0x7fdb880cc1f0 msgr2=0x7fdb8814bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:52.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.226+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 shutdown_connections 2023-10-08T01:12:52.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:52.226+0000 7fdb8759e700 1 -- 172.21.15.119:0/3091337194 wait complete. 2023-10-08T01:12:52.243 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T01:12:52.243 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T01:12:52.243 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T01:12:52.244 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T01:12:52.244 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:52.244 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T01:12:52.244 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T01:12:52.244 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T01:12:52.244 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:12:52.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:52.245 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T01:12:52.283 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T01:12:52.283 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:12:52.284 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41279sec preferred_lft 41279sec 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T01:12:52.285 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:52.286 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:12:52.286 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T01:12:52.286 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T01:12:52.286 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:52.286 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:12:52.286 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T01:12:52.286 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T01:12:52.286 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:12:52.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:52.287 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T01:12:52.383 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T01:12:52.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:52.386 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T01:12:52.424 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T01:12:52.424 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T01:12:52.424 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T01:12:52.425 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T01:12:52.425 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T01:12:52.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:52.426 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T01:12:52.426 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T01:12:52.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T01:12:52.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T01:12:52.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T01:12:52.426 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T01:12:52.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:52.621 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T01:12:52.657 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T01:12:52.658 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T01:12:52.658 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:12:52.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:52.658 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T01:12:52.760 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:12:52.761 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:52.797 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:12:52.797 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:12:52.798 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:52.873 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T01:12:52.873 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T01:12:52.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:12:53.003 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:12:53.004 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:12:53.032 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:12:53.032 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:12:53.032 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T01:12:53.033 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T01:12:53.033 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T01:12:53.038 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T01:12:53.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:12:53.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:12:53.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T01:12:53.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.287+0000 7feebd99d700 1 Processor -- start 2023-10-08T01:12:53.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.287+0000 7feebd99d700 1 -- start start 2023-10-08T01:12:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.288+0000 7feebd99d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feeb806d950 0x7feeb80c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.288+0000 7feebd99d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb806ae70 con 0x7feeb806dd20 2023-10-08T01:12:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.288+0000 7feebd99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeb80c7a30 con 0x7feeb806a0a0 2023-10-08T01:12:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.288+0000 7feebd99d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeb806a540 con 0x7feeb806d950 2023-10-08T01:12:53.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.288+0000 7feebc99b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feeb806d950 0x7feeb80c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.288+0000 7feebc99b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feeb806d950 0x7feeb80c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47386/0 (socket says 172.21.15.119:47386) 2023-10-08T01:12:53.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.288+0000 7feebc99b700 1 -- 172.21.15.119:0/2401050589 learned_addr learned my addr 172.21.15.119:0/2401050589 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:53.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.288+0000 7feebd19c700 1 --1- 172.21.15.119:0/2401050589 >> v1:172.21.15.119:6789/0 conn(0x7feeb806dd20 0x7feeb80c7320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45438/0 (socket says 172.21.15.119:45438) 2023-10-08T01:12:53.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.289+0000 7feebc99b700 1 -- 172.21.15.119:0/2401050589 >> v1:172.21.15.184:6789/0 conn(0x7feeb806a0a0 legacy=0x7feeb80c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.289+0000 7feebc99b700 1 -- 172.21.15.119:0/2401050589 >> v1:172.21.15.119:6789/0 conn(0x7feeb806dd20 legacy=0x7feeb80c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.289+0000 7feebc99b700 1 -- 172.21.15.119:0/2401050589 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb806a820 con 0x7feeb806d950 2023-10-08T01:12:53.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.289+0000 7feeb77fe700 1 -- 172.21.15.119:0/2401050589 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495234948 0 0) 0x7feeb806ae70 con 0x7feeb806dd20 2023-10-08T01:12:53.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.289+0000 7feebc99b700 1 --2- 172.21.15.119:0/2401050589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806d950 0x7feeb80c8350 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7feeac0099b0 tx=0x7feeac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41619e1cda219366 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:53.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.289+0000 7feeb77fe700 1 -- 172.21.15.119:0/2401050589 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac00b860 con 0x7feeb806d950 2023-10-08T01:12:53.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.289+0000 7feeb77fe700 1 -- 172.21.15.119:0/2401050589 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeac00b9c0 con 0x7feeb806d950 2023-10-08T01:12:53.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.289+0000 7feeb77fe700 1 -- 172.21.15.119:0/2401050589 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeac00bd60 con 0x7feeb806d950 2023-10-08T01:12:53.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.290+0000 7feebd99d700 1 -- 172.21.15.119:0/2401050589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806d950 msgr2=0x7feeb80c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.290+0000 7feebd99d700 1 --2- 172.21.15.119:0/2401050589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806d950 0x7feeb80c8350 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7feeac0099b0 tx=0x7feeac009250 comp rx=0 tx=0).stop 2023-10-08T01:12:53.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.290+0000 7feebd99d700 1 -- 172.21.15.119:0/2401050589 shutdown_connections 2023-10-08T01:12:53.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.290+0000 7feebd99d700 1 --2- 172.21.15.119:0/2401050589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806d950 0x7feeb80c8350 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.290+0000 7feebd99d700 1 -- 172.21.15.119:0/2401050589 >> 172.21.15.119:0/2401050589 conn(0x7feeb80cc1f0 msgr2=0x7feeb80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:53.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.290+0000 7feebd99d700 1 -- 172.21.15.119:0/2401050589 shutdown_connections 2023-10-08T01:12:53.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.290+0000 7feebd99d700 1 -- 172.21.15.119:0/2401050589 wait complete. 2023-10-08T01:12:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.292+0000 7feebd99d700 1 Processor -- start 2023-10-08T01:12:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.292+0000 7feebd99d700 1 -- start start 2023-10-08T01:12:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.293+0000 7feebd99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb806a0a0 0x7feeb80ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb806d950 0x7feeb8156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806dd20 0x7feeb8157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb816adb0 con 0x7feeb806dd20 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb816a1a0 con 0x7feeb806d950 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeb816a480 con 0x7feeb806a0a0 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806dd20 0x7feeb8157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebc99b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb806a0a0 0x7feeb80ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feeb7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb806d950 0x7feeb8156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806dd20 0x7feeb8157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47390/0 (socket says 172.21.15.119:47390) 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebc99b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb806a0a0 0x7feeb80ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:56558/0 (socket says 172.21.15.119:56558) 2023-10-08T01:12:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd19c700 1 -- 172.21.15.119:0/3476093231 learned_addr learned my addr 172.21.15.119:0/3476093231 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd19c700 1 -- 172.21.15.119:0/3476093231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb806a0a0 msgr2=0x7feeb80ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd19c700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb806a0a0 0x7feeb80ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd19c700 1 -- 172.21.15.119:0/3476093231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb806d950 msgr2=0x7feeb8156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd19c700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb806d950 0x7feeb8156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.294+0000 7feebd19c700 1 -- 172.21.15.119:0/3476093231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0094f0 con 0x7feeb806dd20 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.295+0000 7feebd19c700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806dd20 0x7feeb8157340 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7feeb000cd50 tx=0x7feeb0001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.295+0000 7feeb4ff9700 1 -- 172.21.15.119:0/3476093231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0005df0 con 0x7feeb806dd20 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.295+0000 7feeb4ff9700 1 -- 172.21.15.119:0/3476093231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb000a8b0 con 0x7feeb806dd20 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.295+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb816a760 con 0x7feeb806dd20 2023-10-08T01:12:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.295+0000 7feeb4ff9700 1 -- 172.21.15.119:0/3476093231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb000ac70 con 0x7feeb806dd20 2023-10-08T01:12:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.295+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb815aa00 con 0x7feeb806dd20 2023-10-08T01:12:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.296+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb804c3c0 con 0x7feeb806dd20 2023-10-08T01:12:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.297+0000 7feeb4ff9700 1 -- 172.21.15.119:0/3476093231 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feeb0007300 con 0x7feeb806dd20 2023-10-08T01:12:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.298+0000 7feeb4ff9700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.298+0000 7feeb4ff9700 1 -- 172.21.15.119:0/3476093231 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7feeb00847d0 con 0x7feeb806dd20 2023-10-08T01:12:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.298+0000 7feebc99b700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.299+0000 7feebc99b700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee9c066f30 0x7fee9c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feeac0099b0 tx=0x7feeac017000 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:53.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.302+0000 7feeb4ff9700 1 -- 172.21.15.119:0/3476093231 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feeb0051890 con 0x7feeb806dd20 2023-10-08T01:12:53.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.522+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7feeb816aef0 con 0x7fee9c066f30 2023-10-08T01:12:53.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.524+0000 7feeb4ff9700 1 -- 172.21.15.119:0/3476093231 <== mgr.6999 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7feeb816aef0 con 0x7fee9c066f30 2023-10-08T01:12:53.524 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:12:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:12:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T01:12:53.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:12:53.524 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:12:53.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee9c066f30 msgr2=0x7fee9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee9c066f30 0x7fee9c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feeac0099b0 tx=0x7feeac017000 comp rx=0 tx=0).stop 2023-10-08T01:12:53.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806dd20 msgr2=0x7feeb8157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806dd20 0x7feeb8157340 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7feeb000cd50 tx=0x7feeb0001b10 comp rx=0 tx=0).stop 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 shutdown_connections 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fee9c066f30 0x7fee9c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feeb806a0a0 0x7feeb80ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeb806dd20 0x7feeb8157340 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 --2- 172.21.15.119:0/3476093231 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feeb806d950 0x7feeb8156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 >> 172.21.15.119:0/3476093231 conn(0x7feeb80cc1f0 msgr2=0x7feeb80c8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 shutdown_connections 2023-10-08T01:12:53.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.526+0000 7feebd99d700 1 -- 172.21.15.119:0/3476093231 wait complete. 2023-10-08T01:12:53.544 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T01:12:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.727+0000 7f0fca19d700 1 Processor -- start 2023-10-08T01:12:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.727+0000 7f0fca19d700 1 -- start start 2023-10-08T01:12:53.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.728+0000 7f0fca19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0fc40b4e70 0x7f0fc40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.728+0000 7f0fca19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b1b40 0x7f0fc415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.728+0000 7f0fca19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc40cfb00 con 0x7f0fc40b16d0 2023-10-08T01:12:53.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.728+0000 7f0fca19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc40b5c40 con 0x7f0fc40b4e70 2023-10-08T01:12:53.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.728+0000 7f0fca19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40b5f20 con 0x7f0fc40b1b40 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc919b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0fc40b4e70 0x7f0fc40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc899a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0fc40b16d0 0x7f0fc415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45452/0 (socket says 172.21.15.119:45452) 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc899a700 1 -- 172.21.15.119:0/3143549243 learned_addr learned my addr 172.21.15.119:0/3143549243 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc999c700 1 --2- 172.21.15.119:0/3143549243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b1b40 0x7f0fc415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fbbfff700 1 -- 172.21.15.119:0/3143549243 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001373210 0 0) 0x7f0fc40cfb00 con 0x7f0fc40b16d0 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fbbfff700 1 -- 172.21.15.119:0/3143549243 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb0003570 con 0x7f0fc40b16d0 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc999c700 1 -- 172.21.15.119:0/3143549243 >> v2:172.21.15.119:3300/0 conn(0x7f0fc40b4e70 msgr2=0x7f0fc40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc999c700 1 --2- 172.21.15.119:0/3143549243 >> v2:172.21.15.119:3300/0 conn(0x7f0fc40b4e70 0x7f0fc40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc999c700 1 -- 172.21.15.119:0/3143549243 >> v1:172.21.15.119:6789/0 conn(0x7f0fc40b16d0 legacy=0x7f0fc415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc999c700 1 -- 172.21.15.119:0/3143549243 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40b6200 con 0x7f0fc40b1b40 2023-10-08T01:12:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.729+0000 7f0fc919b700 1 --2- 172.21.15.119:0/3143549243 >> v2:172.21.15.119:3300/0 conn(0x7f0fc40b4e70 0x7f0fc40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:53.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fc999c700 1 --2- 172.21.15.119:0/3143549243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b1b40 0x7f0fc415d950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009e60 tx=0x7f0fbc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be8ccf93a2ef8d6e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:53.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fbbfff700 1 -- 172.21.15.119:0/3143549243 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00e070 con 0x7f0fc40b1b40 2023-10-08T01:12:53.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fbbfff700 1 -- 172.21.15.119:0/3143549243 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc00bab0 con 0x7f0fc40b1b40 2023-10-08T01:12:53.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fbbfff700 1 -- 172.21.15.119:0/3143549243 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00be30 con 0x7f0fc40b1b40 2023-10-08T01:12:53.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fca19d700 1 -- 172.21.15.119:0/3143549243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b1b40 msgr2=0x7f0fc415d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fca19d700 1 --2- 172.21.15.119:0/3143549243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b1b40 0x7f0fc415d950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc009e60 tx=0x7f0fbc009780 comp rx=0 tx=0).stop 2023-10-08T01:12:53.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fca19d700 1 -- 172.21.15.119:0/3143549243 shutdown_connections 2023-10-08T01:12:53.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fca19d700 1 --2- 172.21.15.119:0/3143549243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b1b40 0x7f0fc415d950 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fca19d700 1 --2- 172.21.15.119:0/3143549243 >> v2:172.21.15.119:3300/0 conn(0x7f0fc40b4e70 0x7f0fc40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fca19d700 1 -- 172.21.15.119:0/3143549243 >> 172.21.15.119:0/3143549243 conn(0x7f0fc40d10c0 msgr2=0x7f0fc40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:53.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.730+0000 7f0fca19d700 1 -- 172.21.15.119:0/3143549243 shutdown_connections 2023-10-08T01:12:53.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.731+0000 7f0fca19d700 1 -- 172.21.15.119:0/3143549243 wait complete. 2023-10-08T01:12:53.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.732+0000 7f0fca19d700 1 Processor -- start 2023-10-08T01:12:53.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.732+0000 7f0fca19d700 1 -- start start 2023-10-08T01:12:53.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fca19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fc40b16d0 0x7f0fc41449d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fca19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fc40b1b40 0x7f0fc415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fca19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b4e70 0x7f0fc415eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fca19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40c63a0 con 0x7f0fc40b1b40 2023-10-08T01:12:53.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fca19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40ca480 con 0x7f0fc40b16d0 2023-10-08T01:12:53.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fca19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40c5390 con 0x7f0fc40b4e70 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fc899a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fc40b1b40 0x7f0fc415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fc899a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fc40b1b40 0x7f0fc415e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47404/0 (socket says 172.21.15.119:47404) 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fc899a700 1 -- 172.21.15.119:0/555957677 learned_addr learned my addr 172.21.15.119:0/555957677 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fc919b700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fc40b16d0 0x7f0fc41449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fc999c700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b4e70 0x7f0fc415eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.733+0000 7f0fc899a700 1 -- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b4e70 msgr2=0x7f0fc415eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fc899a700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b4e70 0x7f0fc415eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fc899a700 1 -- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fc40b16d0 msgr2=0x7f0fc41449d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fc899a700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fc40b16d0 0x7f0fc41449d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:53.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fc899a700 1 -- 172.21.15.119:0/555957677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc0009420 con 0x7f0fc40b1b40 2023-10-08T01:12:53.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fc919b700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fc40b16d0 0x7f0fc41449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:53.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fc899a700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fc40b1b40 0x7f0fc415e560 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40011e0 tx=0x7f0fb4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:53.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fb97fa700 1 -- 172.21.15.119:0/555957677 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4005b30 con 0x7f0fc40b1b40 2023-10-08T01:12:53.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fb97fa700 1 -- 172.21.15.119:0/555957677 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb4005c90 con 0x7f0fc40b1b40 2023-10-08T01:12:53.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fb97fa700 1 -- 172.21.15.119:0/555957677 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb40072e0 con 0x7f0fc40b1b40 2023-10-08T01:12:53.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fbc009a20 con 0x7f0fc40b1b40 2023-10-08T01:12:53.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.734+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc40c5f10 con 0x7f0fc40b1b40 2023-10-08T01:12:53.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.735+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc40ce180 con 0x7f0fc40b1b40 2023-10-08T01:12:53.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.736+0000 7f0fb97fa700 1 -- 172.21.15.119:0/555957677 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 47) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0fb4007650 con 0x7f0fc40b1b40 2023-10-08T01:12:53.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.737+0000 7f0fb97fa700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0fa8066f30 0x7f0fa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:53.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.737+0000 7f0fb97fa700 1 -- 172.21.15.119:0/555957677 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f0fb4083ae0 con 0x7f0fc40b1b40 2023-10-08T01:12:53.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.738+0000 7f0fc919b700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0fa8066f30 0x7f0fa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:53.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.738+0000 7f0fc919b700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0fa8066f30 0x7f0fa80693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00093f0 tx=0x7f0fc0008040 comp rx=0 tx=0).ready entity=mgr.6999 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:53.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.742+0000 7f0fb97fa700 1 -- 172.21.15.119:0/555957677 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0fb404b5a0 con 0x7f0fc40b1b40 2023-10-08T01:12:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:53.986+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f0fc40c60a0 con 0x7f0fc40b1b40 2023-10-08T01:12:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.303+0000 7f0fb97fa700 1 -- 172.21.15.119:0/555957677 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 48) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f0fb4051040 con 0x7f0fc40b1b40 2023-10-08T01:12:54.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.305+0000 7f0fb97fa700 1 -- 172.21.15.119:0/555957677 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v48) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0fb4054700 con 0x7f0fc40b1b40 2023-10-08T01:12:54.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.307+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0fa8066f30 msgr2=0x7f0fa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.307+0000 7f0fca19d700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0fa8066f30 0x7f0fa80693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0fc00093f0 tx=0x7f0fc0008040 comp rx=0 tx=0).stop 2023-10-08T01:12:54.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.307+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fc40b1b40 msgr2=0x7f0fc415e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.307+0000 7f0fca19d700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fc40b1b40 0x7f0fc415e560 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0fb40011e0 tx=0x7f0fb4007df0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.308+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 shutdown_connections 2023-10-08T01:12:54.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.308+0000 7f0fca19d700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0fa8066f30 0x7f0fa80693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.308+0000 7f0fca19d700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0fc40b4e70 0x7f0fc415eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.308+0000 7f0fca19d700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0fc40b16d0 0x7f0fc41449d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.308+0000 7f0fca19d700 1 --2- 172.21.15.119:0/555957677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0fc40b1b40 0x7f0fc415e560 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.308+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 >> 172.21.15.119:0/555957677 conn(0x7f0fc40d10c0 msgr2=0x7f0fc4159210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:54.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.308+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 shutdown_connections 2023-10-08T01:12:54.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.308+0000 7f0fca19d700 1 -- 172.21.15.119:0/555957677 wait complete. 2023-10-08T01:12:54.326 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T01:12:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.521+0000 7f2296f9d700 1 Processor -- start 2023-10-08T01:12:54.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.521+0000 7f2296f9d700 1 -- start start 2023-10-08T01:12:54.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.522+0000 7f2296f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b4e70 0x7f22900ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.522+0000 7f2296f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22900b1b40 0x7f229015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.522+0000 7f2296f9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22900cfb00 con 0x7f22900b16d0 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.522+0000 7f2296f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22900b5c40 con 0x7f22900b1b40 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.522+0000 7f2296f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22900b5f20 con 0x7f22900b4e70 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.522+0000 7f229679c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22900b1b40 0x7f229015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f2295f9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b4e70 0x7f22900ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f229579a700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f22900b16d0 0x7f229015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:34014/0 (socket says 172.21.15.119:34014) 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f229679c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f22900b1b40 0x7f229015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47416/0 (socket says 172.21.15.119:47416) 2023-10-08T01:12:54.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f229579a700 1 -- 172.21.15.119:0/3531392691 learned_addr learned my addr 172.21.15.119:0/3531392691 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f2294f99700 1 -- 172.21.15.119:0/3531392691 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864745897 0 0) 0x7f22900cfb00 con 0x7f22900b16d0 2023-10-08T01:12:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f2294f99700 1 -- 172.21.15.119:0/3531392691 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f227c003570 con 0x7f22900b16d0 2023-10-08T01:12:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f229679c700 1 -- 172.21.15.119:0/3531392691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b4e70 msgr2=0x7f22900ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f229679c700 1 --2- 172.21.15.119:0/3531392691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b4e70 0x7f22900ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f229679c700 1 -- 172.21.15.119:0/3531392691 >> v1:172.21.15.184:6789/0 conn(0x7f22900b16d0 legacy=0x7f229015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f229679c700 1 -- 172.21.15.119:0/3531392691 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900b6200 con 0x7f22900b1b40 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f2295f9b700 1 --2- 172.21.15.119:0/3531392691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b4e70 0x7f22900ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.523+0000 7f229679c700 1 --2- 172.21.15.119:0/3531392691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b1b40 0x7f229015d950 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2288009e60 tx=0x7f2288009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d420c864180d20b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.524+0000 7f2294f99700 1 -- 172.21.15.119:0/3531392691 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800e070 con 0x7f22900b1b40 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.524+0000 7f2294f99700 1 -- 172.21.15.119:0/3531392691 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228800bab0 con 0x7f22900b1b40 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.524+0000 7f2294f99700 1 -- 172.21.15.119:0/3531392691 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228800be70 con 0x7f22900b1b40 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.524+0000 7f2296f9d700 1 -- 172.21.15.119:0/3531392691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b1b40 msgr2=0x7f229015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.524+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3531392691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b1b40 0x7f229015d950 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2288009e60 tx=0x7f2288009780 comp rx=0 tx=0).stop 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.525+0000 7f2296f9d700 1 -- 172.21.15.119:0/3531392691 shutdown_connections 2023-10-08T01:12:54.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.525+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3531392691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b1b40 0x7f229015d950 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.525+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3531392691 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b4e70 0x7f22900ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.525+0000 7f2296f9d700 1 -- 172.21.15.119:0/3531392691 >> 172.21.15.119:0/3531392691 conn(0x7f22900d10c0 msgr2=0x7f22900affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:54.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.525+0000 7f2296f9d700 1 -- 172.21.15.119:0/3531392691 shutdown_connections 2023-10-08T01:12:54.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.525+0000 7f2296f9d700 1 -- 172.21.15.119:0/3531392691 wait complete. 2023-10-08T01:12:54.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.526+0000 7f2296f9d700 1 Processor -- start 2023-10-08T01:12:54.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.526+0000 7f2296f9d700 1 -- start start 2023-10-08T01:12:54.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.527+0000 7f2296f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22900b16d0 0x7f22900cbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.527+0000 7f2296f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b1b40 0x7f22900c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.527+0000 7f2296f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b4e70 0x7f229014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.527+0000 7f2296f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900c7a80 con 0x7f22900b4e70 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.527+0000 7f2296f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22900c86b0 con 0x7f22900b16d0 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.527+0000 7f2296f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22900c77e0 con 0x7f22900b1b40 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.527+0000 7f229579a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b1b40 0x7f22900c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.527+0000 7f2295f9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22900b16d0 0x7f22900cbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f229579a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b1b40 0x7f22900c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:56616/0 (socket says 172.21.15.119:56616) 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f229579a700 1 -- 172.21.15.119:0/3161580150 learned_addr learned my addr 172.21.15.119:0/3161580150 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f229679c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b4e70 0x7f229014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f2295f9b700 1 -- 172.21.15.119:0/3161580150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b1b40 msgr2=0x7f22900c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f2295f9b700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b1b40 0x7f22900c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f2295f9b700 1 -- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b4e70 msgr2=0x7f229014c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f2295f9b700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b4e70 0x7f229014c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f2295f9b700 1 -- 172.21.15.119:0/3161580150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c009420 con 0x7f22900b16d0 2023-10-08T01:12:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f2295f9b700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22900b16d0 0x7f22900cbc40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f228c00ea30 tx=0x7f228c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.528+0000 7f2285ffb700 1 -- 172.21.15.119:0/3161580150 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c00ee70 con 0x7f22900b16d0 2023-10-08T01:12:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.529+0000 7f2285ffb700 1 -- 172.21.15.119:0/3161580150 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c00c850 con 0x7f22900b16d0 2023-10-08T01:12:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.529+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2288009a20 con 0x7f22900b16d0 2023-10-08T01:12:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.529+0000 7f2285ffb700 1 -- 172.21.15.119:0/3161580150 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c00cbf0 con 0x7f22900b16d0 2023-10-08T01:12:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.529+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22900c7d80 con 0x7f22900b16d0 2023-10-08T01:12:54.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.530+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2290088990 con 0x7f22900b16d0 2023-10-08T01:12:54.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.531+0000 7f2285ffb700 1 -- 172.21.15.119:0/3161580150 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 48) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f228c013470 con 0x7f22900b16d0 2023-10-08T01:12:54.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.531+0000 7f2285ffb700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2274066f80 0x7f2274069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.532+0000 7f2285ffb700 1 -- 172.21.15.119:0/3161580150 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f228c0848c0 con 0x7f22900b16d0 2023-10-08T01:12:54.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.532+0000 7f229579a700 1 -- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2274066f80 msgr2=0x7f2274069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:54.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.532+0000 7f229579a700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2274066f80 0x7f2274069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:54.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.540+0000 7f2285ffb700 1 -- 172.21.15.119:0/3161580150 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f228c0519a0 con 0x7f22900b16d0 2023-10-08T01:12:54.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.733+0000 7f229579a700 1 -- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2274066f80 msgr2=0x7f2274069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:54.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.733+0000 7f229579a700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2274066f80 0x7f2274069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:54.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.741+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f229004bb80 con 0x7f22900b16d0 2023-10-08T01:12:54.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.746+0000 7f2285ffb700 1 -- 172.21.15.119:0/3161580150 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f228c054fc0 con 0x7f22900b16d0 2023-10-08T01:12:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2274066f80 msgr2=0x7f2274069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2274066f80 0x7f2274069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22900b16d0 msgr2=0x7f22900cbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22900b16d0 0x7f22900cbc40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f228c00ea30 tx=0x7f228c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 shutdown_connections 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2274066f80 0x7f2274069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22900b4e70 0x7f229014c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f22900b16d0 0x7f22900cbc40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 --2- 172.21.15.119:0/3161580150 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f22900b1b40 0x7f22900c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 >> 172.21.15.119:0/3161580150 conn(0x7f22900d10c0 msgr2=0x7f2290158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:54.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 shutdown_connections 2023-10-08T01:12:54.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.749+0000 7f2296f9d700 1 -- 172.21.15.119:0/3161580150 wait complete. 2023-10-08T01:12:54.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T01:12:54.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.959+0000 7ff3bed9d700 1 Processor -- start 2023-10-08T01:12:54.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.959+0000 7ff3bed9d700 1 -- start start 2023-10-08T01:12:54.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.960+0000 7ff3bed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.960+0000 7ff3bed9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3b806a020 0x7ff3b80cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.960+0000 7ff3bed9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b80c9f30 con 0x7ff3b8159a60 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.960+0000 7ff3bed9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b806dfc0 con 0x7ff3b806a020 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.960+0000 7ff3bed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b80ca470 con 0x7ff3b80ccc00 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3be59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3b806a020 0x7ff3b80cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3bdd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff3b8159a60 0x7ff3b80c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51530/0 (socket says 172.21.15.119:51530) 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3bdd9b700 1 -- 172.21.15.119:0/3987509381 learned_addr learned my addr 172.21.15.119:0/3987509381 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3bd59a700 1 --2- 172.21.15.119:0/3987509381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3be59c700 1 -- 172.21.15.119:0/3987509381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 msgr2=0x7ff3b8069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3be59c700 1 --2- 172.21.15.119:0/3987509381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8069ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3be59c700 1 -- 172.21.15.119:0/3987509381 >> v1:172.21.15.119:6789/0 conn(0x7ff3b8159a60 legacy=0x7ff3b80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3be59c700 1 -- 172.21.15.119:0/3987509381 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b80ca750 con 0x7ff3b806a020 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3bcd99700 1 -- 172.21.15.119:0/3987509381 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 575831776 0 0) 0x7ff3b80c9f30 con 0x7ff3b8159a60 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.961+0000 7ff3be59c700 1 --2- 172.21.15.119:0/3987509381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b80cb230 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009e60 tx=0x7ff3b0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f8c8000a4494c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bcd99700 1 -- 172.21.15.119:0/3987509381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b000e070 con 0x7ff3b806a020 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bcd99700 1 -- 172.21.15.119:0/3987509381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b000bbf0 con 0x7ff3b806a020 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bcd99700 1 -- 172.21.15.119:0/3987509381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b0012430 con 0x7ff3b806a020 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3987509381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 msgr2=0x7ff3b80cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3987509381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b80cb230 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff3b0009e60 tx=0x7ff3b0009780 comp rx=0 tx=0).stop 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3987509381 shutdown_connections 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3987509381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b80cb230 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3987509381 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8069ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3987509381 >> 172.21.15.119:0/3987509381 conn(0x7ff3b806baa0 msgr2=0x7ff3b806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3987509381 shutdown_connections 2023-10-08T01:12:54.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.962+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3987509381 wait complete. 2023-10-08T01:12:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.964+0000 7ff3bed9d700 1 Processor -- start 2023-10-08T01:12:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.964+0000 7ff3bed9d700 1 -- start start 2023-10-08T01:12:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b8157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8153630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3b8159a60 0x7ff3b8159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bed9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b80c7180 con 0x7ff3b806a020 2023-10-08T01:12:54.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bed9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b80b3b60 con 0x7ff3b8159a60 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b8153b70 con 0x7ff3b80ccc00 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bdd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b8157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bdd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b8157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37664/0 (socket says 172.21.15.119:37664) 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bdd9b700 1 -- 172.21.15.119:0/3213789489 learned_addr learned my addr 172.21.15.119:0/3213789489 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3be59c700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3b8159a60 0x7ff3b8159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.965+0000 7ff3bd59a700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8153630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bdd9b700 1 -- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 msgr2=0x7ff3b8153630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bdd9b700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8153630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bdd9b700 1 -- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3b8159a60 msgr2=0x7ff3b8159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bdd9b700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3b8159a60 0x7ff3b8159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:54.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bdd9b700 1 -- 172.21.15.119:0/3213789489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0009a20 con 0x7ff3b806a020 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bd59a700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bdd9b700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b8157820 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c5b0 tx=0x7ff3b4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3adffb700 1 -- 172.21.15.119:0/3213789489 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4005b30 con 0x7ff3b806a020 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3adffb700 1 -- 172.21.15.119:0/3213789489 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3b4005c90 con 0x7ff3b806a020 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3adffb700 1 -- 172.21.15.119:0/3213789489 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3b4007300 con 0x7ff3b806a020 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b80c6850 con 0x7ff3b806a020 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.966+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b80c6e10 con 0x7ff3b806a020 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.967+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b80c72f0 con 0x7ff3b806a020 2023-10-08T01:12:54.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.968+0000 7ff3adffb700 1 -- 172.21.15.119:0/3213789489 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 48) v1 ==== 84757+0+0 (secure 0 0 0) 0x7ff3b401b3b0 con 0x7ff3b806a020 2023-10-08T01:12:54.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.969+0000 7ff3adffb700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff39c066f80 0x7ff39c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:54.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.969+0000 7ff3adffb700 1 -- 172.21.15.119:0/3213789489 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff3b4083c50 con 0x7ff3b806a020 2023-10-08T01:12:54.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.970+0000 7ff3bd59a700 1 -- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff39c066f80 msgr2=0x7ff39c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:54.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.970+0000 7ff3bd59a700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff39c066f80 0x7ff39c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:54.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:54.974+0000 7ff3adffb700 1 -- 172.21.15.119:0/3213789489 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3b4089020 con 0x7ff3b806a020 2023-10-08T01:12:55.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.171+0000 7ff3bd59a700 1 -- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff39c066f80 msgr2=0x7ff39c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:55.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.171+0000 7ff3bd59a700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff39c066f80 0x7ff39c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:55.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.176+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff3b80caa30 con 0x7ff3b806a020 2023-10-08T01:12:55.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.180+0000 7ff3adffb700 1 -- 172.21.15.119:0/3213789489 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff3b4017020 con 0x7ff3b806a020 2023-10-08T01:12:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff39c066f80 msgr2=0x7ff39c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:12:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff39c066f80 0x7ff39c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 msgr2=0x7ff3b8157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b8157820 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400c5b0 tx=0x7ff3b4001070 comp rx=0 tx=0).stop 2023-10-08T01:12:55.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 shutdown_connections 2023-10-08T01:12:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff39c066f80 0x7ff39c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff3b8159a60 0x7ff3b8159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3b806a020 0x7ff3b8157820 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 --2- 172.21.15.119:0/3213789489 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff3b80ccc00 0x7ff3b8153630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 >> 172.21.15.119:0/3213789489 conn(0x7ff3b806baa0 msgr2=0x7ff3b80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 shutdown_connections 2023-10-08T01:12:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.183+0000 7ff3bed9d700 1 -- 172.21.15.119:0/3213789489 wait complete. 2023-10-08T01:12:55.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-10-08T01:12:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.393+0000 7f52d8f9d700 1 Processor -- start 2023-10-08T01:12:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.393+0000 7f52d8f9d700 1 -- start start 2023-10-08T01:12:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.394+0000 7f52d8f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f52d40cb560 0x7f52d40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.394+0000 7f52d8f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d4069ec0 0x7f52d415ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d8f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d4159db0 con 0x7f52d40cc970 2023-10-08T01:12:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d8f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d406aa30 con 0x7f52d40cb560 2023-10-08T01:12:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d8f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40cce10 con 0x7f52d4069ec0 2023-10-08T01:12:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d2ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f52d40cc970 0x7f52d40697b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51538/0 (socket says 172.21.15.119:51538) 2023-10-08T01:12:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d2ffd700 1 -- 172.21.15.119:0/622171867 learned_addr learned my addr 172.21.15.119:0/622171867 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:12:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d37fe700 1 --2- 172.21.15.119:0/622171867 >> v2:172.21.15.184:3300/0 conn(0x7f52d40cb560 0x7f52d40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d3fff700 1 --2- 172.21.15.119:0/622171867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d4069ec0 0x7f52d415ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d27fc700 1 -- 172.21.15.119:0/622171867 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387271561 0 0) 0x7f52d4159db0 con 0x7f52d40cc970 2023-10-08T01:12:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d27fc700 1 -- 172.21.15.119:0/622171867 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52bc003570 con 0x7f52d40cc970 2023-10-08T01:12:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d37fe700 1 -- 172.21.15.119:0/622171867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d4069ec0 msgr2=0x7f52d415ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:12:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d37fe700 1 --2- 172.21.15.119:0/622171867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d4069ec0 0x7f52d415ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d37fe700 1 -- 172.21.15.119:0/622171867 >> v1:172.21.15.119:6789/0 conn(0x7f52d40cc970 legacy=0x7f52d40697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.395+0000 7f52d37fe700 1 -- 172.21.15.119:0/622171867 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d406a470 con 0x7f52d40cb560 2023-10-08T01:12:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.396+0000 7f52d37fe700 1 --2- 172.21.15.119:0/622171867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d40cb560 0x7f52d40cc430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009910 tx=0x7f52c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0a18abf38b5a12c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.396+0000 7f52d27fc700 1 -- 172.21.15.119:0/622171867 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011630 con 0x7f52d40cb560 2023-10-08T01:12:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.396+0000 7f52d27fc700 1 -- 172.21.15.119:0/622171867 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c4011790 con 0x7f52d40cb560 2023-10-08T01:12:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.396+0000 7f52d27fc700 1 -- 172.21.15.119:0/622171867 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c4011b30 con 0x7f52d40cb560 2023-10-08T01:12:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.397+0000 7f52d8f9d700 1 -- 172.21.15.119:0/622171867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d40cb560 msgr2=0x7f52d40cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.397+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/622171867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d40cb560 0x7f52d40cc430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009910 tx=0x7f52c40091a0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.397+0000 7f52d8f9d700 1 -- 172.21.15.119:0/622171867 shutdown_connections 2023-10-08T01:12:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.397+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/622171867 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d4069ec0 0x7f52d415ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.397+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/622171867 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d40cb560 0x7f52d40cc430 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.397+0000 7f52d8f9d700 1 -- 172.21.15.119:0/622171867 >> 172.21.15.119:0/622171867 conn(0x7f52d406baf0 msgr2=0x7f52d406bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:12:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.397+0000 7f52d8f9d700 1 -- 172.21.15.119:0/622171867 shutdown_connections 2023-10-08T01:12:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.397+0000 7f52d8f9d700 1 -- 172.21.15.119:0/622171867 wait complete. 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.398+0000 7f52d8f9d700 1 Processor -- start 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.398+0000 7f52d8f9d700 1 -- start start 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.399+0000 7f52d8f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d4069ec0 0x7f52d4150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.399+0000 7f52d8f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d40cb560 0x7f52d414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.399+0000 7f52d8f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d40cc970 0x7f52d4153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.399+0000 7f52d8f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41577a0 con 0x7f52d40cb560 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.399+0000 7f52d8f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d40b2b60 con 0x7f52d4069ec0 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.399+0000 7f52d8f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4151400 con 0x7f52d40cc970 2023-10-08T01:12:55.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d40cb560 0x7f52d414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d40cb560 0x7f52d414ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37680/0 (socket says 172.21.15.119:37680) 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 -- 172.21.15.119:0/3155012413 learned_addr learned my addr 172.21.15.119:0/3155012413 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d4069ec0 0x7f52d4150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d3fff700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d40cc970 0x7f52d4153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d40cc970 msgr2=0x7f52d4153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d40cc970 0x7f52d4153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d4069ec0 msgr2=0x7f52d4150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d4069ec0 0x7f52d4150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 -- 172.21.15.119:0/3155012413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009420 con 0x7f52d40cb560 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d4069ec0 0x7f52d4150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:12:55.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52d2ffd700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d40cb560 0x7f52d414ffe0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f52c800c5c0 tx=0x7f52c8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:12:55.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.400+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8005b30 con 0x7f52d40cb560 2023-10-08T01:12:55.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.401+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c8005cb0 con 0x7f52d40cb560 2023-10-08T01:12:55.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.401+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d40b2810 con 0x7f52d40cb560 2023-10-08T01:12:55.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.401+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8007320 con 0x7f52d40cb560 2023-10-08T01:12:55.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.401+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d40cedd0 con 0x7f52d40cb560 2023-10-08T01:12:55.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.402+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d40b2d10 con 0x7f52d40cb560 2023-10-08T01:12:55.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.403+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 48) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f52c801b3b0 con 0x7f52d40cb560 2023-10-08T01:12:55.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.403+0000 7f52bb7fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:12:55.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.404+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f52c8083c80 con 0x7f52d40cb560 2023-10-08T01:12:55.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.404+0000 7f52d37fe700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 msgr2=0x7f52c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:55.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.404+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:12:55.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.408+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f52d40b2d10 con 0x7f52d40cb560 2023-10-08T01:12:55.586 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:55.586+0000 7fcc54458180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:12:55.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.605+0000 7f52d37fe700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 msgr2=0x7f52c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:55.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.605+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:12:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:55.636+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f52d40b2d10 con 0x7f52c0066f80 2023-10-08T01:12:55.637 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:55.637+0000 7facb247f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:12:55.749 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:55.749+0000 7fcc54458180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:55.800 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:55.801+0000 7facb247f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:55.835 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:55.835+0000 7fcc54458180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:12:55.887 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:55.887+0000 7facb247f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:12:56.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:56.006+0000 7f52d37fe700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 msgr2=0x7f52c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:56.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:56.006+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:12:56.059 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:56.059+0000 7fcc54458180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:56.111 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:56.111+0000 7facb247f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:56.235 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:56.235+0000 7fcc54458180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:12:56.288 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:56.288+0000 7facb247f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:12:56.326 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:56.326+0000 7fcc54458180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:12:56.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:56.379+0000 7facb247f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:12:56.491 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:56.491+0000 7fcc54458180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:12:56.542 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:56.542+0000 7facb247f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:12:56.593 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:56.593+0000 7fcc54458180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:56.643 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:56.643+0000 7facb247f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:12:56.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:56.807+0000 7f52d37fe700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 msgr2=0x7f52c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:56.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:56.807+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T01:12:57.077 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:57.077+0000 7fcc54458180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:12:57.125 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:57.125+0000 7facb247f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:12:57.170 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:57.170+0000 7fcc54458180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:12:57.218 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:57.218+0000 7facb247f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:12:57.260 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:57.260+0000 7fcc54458180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:12:57.308 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:57.308+0000 7facb247f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:12:57.882 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:57.882+0000 7fcc54458180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:12:57.926 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:57.926+0000 7facb247f180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:12:58.055 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:58.054+0000 7fcc54458180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:58.101 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:58.100+0000 7facb247f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:12:58.149 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:58.149+0000 7fcc54458180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:12:58.197 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:58.196+0000 7facb247f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:12:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:58.409+0000 7f52d37fe700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 msgr2=0x7f52c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:12:58.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:12:58.409+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T01:12:58.442 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:58.442+0000 7fcc54458180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:12:58.491 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:58.491+0000 7facb247f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:12:58.539 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:58.539+0000 7fcc54458180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:12:58.587 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:58.587+0000 7facb247f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:12:59.241 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:59.241+0000 7fcc54458180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:12:59.292 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:59.292+0000 7facb247f180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:12:59.528 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:59.528+0000 7fcc54458180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:12:59.576 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:59.577+0000 7facb247f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:12:59.621 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:59.621+0000 7fcc54458180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:12:59.669 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:59.670+0000 7facb247f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:12:59.773 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:59.773+0000 7fcc54458180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:12:59.823 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:59.823+0000 7facb247f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:12:59.878 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:12:59.878+0000 7fcc54458180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:12:59.927 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:12:59.927+0000 7facb247f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:13:00.070 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:13:00.069+0000 7fcc54458180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:13:00.118 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.118+0000 7facb247f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:13:00.161 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:13:00.161+0000 7fcc54458180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:13:00.211 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.211+0000 7facb247f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:13:00.253 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:13:00.253+0000 7fcc54458180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:13:00.305 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.305+0000 7facb247f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:13:00.348 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:13:00.348+0000 7fcc54458180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:13:00.399 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.399+0000 7facb247f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:13:00.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:00.407+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 49) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f52c8051130 con 0x7f52d40cb560 2023-10-08T01:13:00.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:00.419+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 50) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f52c8054c80 con 0x7f52d40cb560 2023-10-08T01:13:00.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:00.419+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 msgr2=0x7f52c0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:13:00.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:00.419+0000 7f52bb7fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:00.546 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.546+0000 7fac267b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.547 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.546+0000 7fac267b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.547 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.546+0000 7fac267b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.547 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.546+0000 7fac267b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.547 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.546+0000 7fac267b5700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.549 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.547+0000 7fac247b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.550 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.547+0000 7fac247b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.550 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.547+0000 7fac247b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.550 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.547+0000 7fac247b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:00.550 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:13:00.547+0000 7fac247b1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:13:01.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.428+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mon.0 v2:172.21.15.119:3300/0 9 ==== mgrmap(e 51) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f52c8051a00 con 0x7f52d40cb560 2023-10-08T01:13:01.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.429+0000 7f52bb7fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c00693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:13:01.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.429+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f52d40b2d10 con 0x7f52c0066f80 2023-10-08T01:13:01.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.430+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c00693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:13:01.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.433+0000 7f52d37fe700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c00693b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f52c4006dc0 tx=0x7f52c4017000 comp rx=0 tx=0).ready entity=mgr.6206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:13:01.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.490+0000 7f52bb7fe700 1 -- 172.21.15.119:0/3155012413 <== mgr.6206 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f52d40b2d10 con 0x7f52c0066f80 2023-10-08T01:13:01.490 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /bad-path 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.492+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 msgr2=0x7f52c00693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.492+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c00693b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f52c4006dc0 tx=0x7f52c4017000 comp rx=0 tx=0).stop 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.492+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d40cb560 msgr2=0x7f52d414ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.492+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d40cb560 0x7f52d414ffe0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f52c800c5c0 tx=0x7f52c8007e00 comp rx=0 tx=0).stop 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.492+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 shutdown_connections 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.492+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f52c0066f80 0x7f52c00693b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.492+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f52d40cc970 0x7f52d4153660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.493+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f52d4069ec0 0x7f52d4150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.493+0000 7f52d8f9d700 1 --2- 172.21.15.119:0/3155012413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52d40cb560 0x7f52d414ffe0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.493+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 >> 172.21.15.119:0/3155012413 conn(0x7f52d406baf0 msgr2=0x7f52d41579e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:13:01.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.493+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 shutdown_connections 2023-10-08T01:13:01.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.493+0000 7f52d8f9d700 1 -- 172.21.15.119:0/3155012413 wait complete. 2023-10-08T01:13:01.510 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-10-08T01:13:01.510 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-10-08T01:13:01.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.695+0000 7fa7d17ff700 1 Processor -- start 2023-10-08T01:13:01.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.695+0000 7fa7d17ff700 1 -- start start 2023-10-08T01:13:01.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.696+0000 7fa7d17ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa7cc0c8490 0x7fa7cc0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.696+0000 7fa7d17ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7d17ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7cc0ac030 con 0x7fa7cc15a490 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7d17ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7cc068aa0 con 0x7fa7cc0c8490 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7d17ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc0ab820 con 0x7fa7cc069a50 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7caffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa7cc0c8490 0x7fa7cc0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7c3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa7cc15a490 0x7fa7cc068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51546/0 (socket says 172.21.15.119:51546) 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7c3fff700 1 -- 172.21.15.119:0/2972403429 learned_addr learned my addr 172.21.15.119:0/2972403429 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7cb7fe700 1 --2- 172.21.15.119:0/2972403429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7caffd700 1 -- 172.21.15.119:0/2972403429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 msgr2=0x7fa7cc156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:13:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7caffd700 1 --2- 172.21.15.119:0/2972403429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7caffd700 1 -- 172.21.15.119:0/2972403429 >> v1:172.21.15.119:6789/0 conn(0x7fa7cc15a490 legacy=0x7fa7cc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7caffd700 1 -- 172.21.15.119:0/2972403429 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc06dbb0 con 0x7fa7cc0c8490 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.697+0000 7fa7caffd700 1 --2- 172.21.15.119:0/2972403429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc0c8490 0x7fa7cc0c6ac0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0098e0 tx=0x7fa7bc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca51aae6462b9a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7ca7fc700 1 -- 172.21.15.119:0/2972403429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc009e80 con 0x7fa7cc0c8490 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7ca7fc700 1 -- 172.21.15.119:0/2972403429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7bc00b860 con 0x7fa7cc0c8490 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7ca7fc700 1 -- 172.21.15.119:0/2972403429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc00bbe0 con 0x7fa7cc0c8490 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2972403429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc0c8490 msgr2=0x7fa7cc0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2972403429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc0c8490 0x7fa7cc0c6ac0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0098e0 tx=0x7fa7bc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2972403429 shutdown_connections 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2972403429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2972403429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc0c8490 0x7fa7cc0c6ac0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2972403429 >> 172.21.15.119:0/2972403429 conn(0x7fa7cc0c8f90 msgr2=0x7fa7cc0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:13:01.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2972403429 shutdown_connections 2023-10-08T01:13:01.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.698+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2972403429 wait complete. 2023-10-08T01:13:01.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.700+0000 7fa7d17ff700 1 Processor -- start 2023-10-08T01:13:01.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.700+0000 7fa7d17ff700 1 -- start start 2023-10-08T01:13:01.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7d17ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc0c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:13:01.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7d17ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc0c8490 0x7fa7cc154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:13:01.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7d17ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc15a490 0x7fa7cc0a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:13:01.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7d17ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc0b0e70 con 0x7fa7cc15a490 2023-10-08T01:13:01.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7d17ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc0b02a0 con 0x7fa7cc0c8490 2023-10-08T01:13:01.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7d17ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc0b0540 con 0x7fa7cc069a50 2023-10-08T01:13:01.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7cb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc15a490 0x7fa7cc0a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:13:01.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7cb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc15a490 0x7fa7cc0a98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37810/0 (socket says 172.21.15.119:37810) 2023-10-08T01:13:01.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7cb7fe700 1 -- 172.21.15.119:0/2461414519 learned_addr learned my addr 172.21.15.119:0/2461414519 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:13:01.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7caffd700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc0c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:13:01.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.701+0000 7fa7c3fff700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc0c8490 0x7fa7cc154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:13:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7cb7fe700 1 -- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 msgr2=0x7fa7cc0c3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:13:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7cb7fe700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc0c3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7cb7fe700 1 -- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc0c8490 msgr2=0x7fa7cc154c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:13:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7cb7fe700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc0c8490 0x7fa7cc154c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7cb7fe700 1 -- 172.21.15.119:0/2461414519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc009420 con 0x7fa7cc15a490 2023-10-08T01:13:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7caffd700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc0c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:13:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7cb7fe700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc15a490 0x7fa7cc0a98e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009fa0 tx=0x7fa7c4009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:13:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7c2ffd700 1 -- 172.21.15.119:0/2461414519 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c400ed20 con 0x7fa7cc15a490 2023-10-08T01:13:01.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7c2ffd700 1 -- 172.21.15.119:0/2461414519 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c400ee80 con 0x7fa7cc15a490 2023-10-08T01:13:01.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc0b0820 con 0x7fa7cc15a490 2023-10-08T01:13:01.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7c2ffd700 1 -- 172.21.15.119:0/2461414519 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4014670 con 0x7fa7cc15a490 2023-10-08T01:13:01.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.702+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc0c4410 con 0x7fa7cc15a490 2023-10-08T01:13:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.703+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc0718b0 con 0x7fa7cc15a490 2023-10-08T01:13:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.704+0000 7fa7c2ffd700 1 -- 172.21.15.119:0/2461414519 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 51) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fa7c40149e0 con 0x7fa7cc15a490 2023-10-08T01:13:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.705+0000 7fa7c2ffd700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa7ac066dd0 0x7fa7ac069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:13:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.705+0000 7fa7c2ffd700 1 -- 172.21.15.119:0/2461414519 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fa7c4084cc0 con 0x7fa7cc15a490 2023-10-08T01:13:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.705+0000 7fa7caffd700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa7ac066dd0 0x7fa7ac069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:13:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.706+0000 7fa7caffd700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa7ac066dd0 0x7fa7ac069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0091e0 tx=0x7fa7bc017000 comp rx=0 tx=0).ready entity=mgr.6206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:13:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.709+0000 7fa7c2ffd700 1 -- 172.21.15.119:0/2461414519 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7c4051f50 con 0x7fa7cc15a490 2023-10-08T01:13:01.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.926+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7cc0b0fb0 con 0x7fa7ac066dd0 2023-10-08T01:13:01.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.934+0000 7fa7c2ffd700 1 -- 172.21.15.119:0/2461414519 <== mgr.6206 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fa7cc0b0fb0 con 0x7fa7ac066dd0 2023-10-08T01:13:01.935 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:13:01.935 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T01:13:01", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-08T01:13:01.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa7ac066dd0 msgr2=0x7fa7ac069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:13:01.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa7ac066dd0 0x7fa7ac069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc0091e0 tx=0x7fa7bc017000 comp rx=0 tx=0).stop 2023-10-08T01:13:01.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc15a490 msgr2=0x7fa7cc0a98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc15a490 0x7fa7cc0a98e0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009fa0 tx=0x7fa7c4009900 comp rx=0 tx=0).stop 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 shutdown_connections 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa7ac066dd0 0x7fa7ac069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa7cc069a50 0x7fa7cc0c3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa7cc15a490 0x7fa7cc0a98e0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 --2- 172.21.15.119:0/2461414519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa7cc0c8490 0x7fa7cc154c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 >> 172.21.15.119:0/2461414519 conn(0x7fa7cc0c8f90 msgr2=0x7fa7cc0cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 shutdown_connections 2023-10-08T01:13:01.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:13:01.937+0000 7fa7d17ff700 1 -- 172.21.15.119:0/2461414519 wait complete. 2023-10-08T01:13:01.955 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T01:13:01", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-10-08T01:13:17.611 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:13:17.617 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:13:47.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:13:47.733 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:14:00.502 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:00.501+0000 7fac16f96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:00.502 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:00.501+0000 7fac16f96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:00.502 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:00.501+0000 7fac16f96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:00.502 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:00.501+0000 7fac16f96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:00.502 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:00.501+0000 7fac16f96700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:01.956 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-10-08T01:14:02.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.150+0000 7f7ad03ff700 1 Processor -- start 2023-10-08T01:14:02.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.151+0000 7f7ad03ff700 1 -- start start 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ad03ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac8157740 0x7f7ac80adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ad03ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ac8068aa0 0x7f7ac806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ad03ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac806dc20 con 0x7f7ac80b2100 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ad03ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac80acc00 con 0x7f7ac8068aa0 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ad03ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac806d6a0 con 0x7f7ac8157740 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ace99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ac8068aa0 0x7f7ac806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ace99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ac8068aa0 0x7f7ac806e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52598/0 (socket says 172.21.15.119:52598) 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ace99c700 1 -- 172.21.15.119:0/3598265695 learned_addr learned my addr 172.21.15.119:0/3598265695 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:02.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.152+0000 7f7ace19b700 1 --2- 172.21.15.119:0/3598265695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac8157740 0x7f7ac80adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.153+0000 7f7ace99c700 1 -- 172.21.15.119:0/3598265695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac8157740 msgr2=0x7f7ac80adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:02.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.153+0000 7f7ace99c700 1 --2- 172.21.15.119:0/3598265695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac8157740 0x7f7ac80adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.153+0000 7f7ace99c700 1 -- 172.21.15.119:0/3598265695 >> v1:172.21.15.184:6789/0 conn(0x7f7ac80b2100 legacy=0x7f7ac8068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.153+0000 7f7ace99c700 1 -- 172.21.15.119:0/3598265695 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac80ae910 con 0x7f7ac8068aa0 2023-10-08T01:14:02.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.153+0000 7f7ace99c700 1 --2- 172.21.15.119:0/3598265695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac806e0b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009f20 tx=0x7f7ac0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505259c8dbc12817 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:02.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.153+0000 7f7acd199700 1 -- 172.21.15.119:0/3598265695 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000e070 con 0x7f7ac8068aa0 2023-10-08T01:14:02.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.153+0000 7f7acd199700 1 -- 172.21.15.119:0/3598265695 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac000b9d0 con 0x7f7ac8068aa0 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.153+0000 7f7acd199700 1 -- 172.21.15.119:0/3598265695 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac00092e0 con 0x7f7ac8068aa0 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.154+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3598265695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 msgr2=0x7f7ac806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.154+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3598265695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac806e0b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009f20 tx=0x7f7ac0009840 comp rx=0 tx=0).stop 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.154+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3598265695 shutdown_connections 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.154+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3598265695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac806e0b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.154+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3598265695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac8157740 0x7f7ac80adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.154+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3598265695 >> 172.21.15.119:0/3598265695 conn(0x7f7ac80c8f90 msgr2=0x7f7ac80c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.154+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3598265695 shutdown_connections 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.154+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3598265695 wait complete. 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.155+0000 7f7ad03ff700 1 Processor -- start 2023-10-08T01:14:02.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.155+0000 7f7ad03ff700 1 -- start start 2023-10-08T01:14:02.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.156+0000 7f7ad03ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac8147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.156+0000 7f7ad03ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac80b2100 0x7f7ac815c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.156+0000 7f7ad03ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ac8157740 0x7f7ac815c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.156+0000 7f7ad03ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80c6ac0 con 0x7f7ac8068aa0 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.156+0000 7f7ad03ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac816ab00 con 0x7f7ac8157740 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.156+0000 7f7ad03ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac80c6d10 con 0x7f7ac80b2100 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac8147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac8147840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52610/0 (socket says 172.21.15.119:52610) 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 -- 172.21.15.119:0/3584279355 learned_addr learned my addr 172.21.15.119:0/3584279355 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7acd99a700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac80b2100 0x7f7ac815c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace99c700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ac8157740 0x7f7ac815c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 -- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac80b2100 msgr2=0x7f7ac815c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac80b2100 0x7f7ac815c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 -- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ac8157740 msgr2=0x7f7ac815c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ac8157740 0x7f7ac815c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 -- 172.21.15.119:0/3584279355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac0009ae0 con 0x7f7ac8068aa0 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace99c700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ac8157740 0x7f7ac815c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7acd99a700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac80b2100 0x7f7ac815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ace19b700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac8147840 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e930 tx=0x7f7ac40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7abe7fc700 1 -- 172.21.15.119:0/3584279355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4009510 con 0x7f7ac8068aa0 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7abe7fc700 1 -- 172.21.15.119:0/3584279355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac400c850 con 0x7f7ac8068aa0 2023-10-08T01:14:02.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7abe7fc700 1 -- 172.21.15.119:0/3584279355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400cc10 con 0x7f7ac8068aa0 2023-10-08T01:14:02.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.157+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac80c6ff0 con 0x7f7ac8068aa0 2023-10-08T01:14:02.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.158+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac8147d80 con 0x7f7ac8068aa0 2023-10-08T01:14:02.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.159+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac80718b0 con 0x7f7ac8068aa0 2023-10-08T01:14:02.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.159+0000 7f7abe7fc700 1 -- 172.21.15.119:0/3584279355 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 52) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f7ac4013470 con 0x7f7ac8068aa0 2023-10-08T01:14:02.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.160+0000 7f7abe7fc700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7aac067130 0x7f7aac0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.160+0000 7f7abe7fc700 1 -- 172.21.15.119:0/3584279355 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f7ac4084d40 con 0x7f7ac8068aa0 2023-10-08T01:14:02.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.160+0000 7f7acd99a700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7aac067130 0x7f7aac0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.161+0000 7f7acd99a700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7aac067130 0x7f7aac0695e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80cb810 tx=0x7f7ab8009040 comp rx=0 tx=0).ready entity=mgr.6206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:02.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.165+0000 7f7abe7fc700 1 -- 172.21.15.119:0/3584279355 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ac4051a10 con 0x7f7ac8068aa0 2023-10-08T01:14:02.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.381+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ac80aebf0 con 0x7f7aac067130 2023-10-08T01:14:02.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.391+0000 7f7abe7fc700 1 -- 172.21.15.119:0/3584279355 <== mgr.6206 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f7ac80aebf0 con 0x7f7aac067130 2023-10-08T01:14:02.392 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:02.392 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T01:13:01", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-08T01:14:02.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7aac067130 msgr2=0x7f7aac0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7aac067130 0x7f7aac0695e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80cb810 tx=0x7f7ab8009040 comp rx=0 tx=0).stop 2023-10-08T01:14:02.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 msgr2=0x7f7ac8147840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac8147840 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e930 tx=0x7f7ac40091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 shutdown_connections 2023-10-08T01:14:02.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7aac067130 0x7f7aac0695e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ac8157740 0x7f7ac815c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ac8068aa0 0x7f7ac8147840 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 --2- 172.21.15.119:0/3584279355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ac80b2100 0x7f7ac815c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 >> 172.21.15.119:0/3584279355 conn(0x7f7ac80c8f90 msgr2=0x7f7ac81565e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:02.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 shutdown_connections 2023-10-08T01:14:02.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.393+0000 7f7ad03ff700 1 -- 172.21.15.119:0/3584279355 wait complete. 2023-10-08T01:14:02.411 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T01:13:01", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-10-08T01:14:02.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-10-08T01:14:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.597+0000 7f8bd2f9d700 1 Processor -- start 2023-10-08T01:14:02.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.597+0000 7f8bd2f9d700 1 -- start start 2023-10-08T01:14:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.598+0000 7f8bd2f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 0x7f8bcc0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.598+0000 7f8bd2f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bcc0b0790 0x7f8bcc0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.598+0000 7f8bd2f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bcc0cfa00 con 0x7f8bcc1591e0 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.598+0000 7f8bd2f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bcc0cb580 con 0x7f8bcc0b0790 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.598+0000 7f8bd2f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0b2de0 con 0x7f8bcc0b03c0 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.598+0000 7f8bd179a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bcc0b0790 0x7f8bcc0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd179a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8bcc0b0790 0x7f8bcc0b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52612/0 (socket says 172.21.15.119:52612) 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd179a700 1 -- 172.21.15.119:0/2485457777 learned_addr learned my addr 172.21.15.119:0/2485457777 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd279c700 1 --1- 172.21.15.119:0/2485457777 >> v1:172.21.15.119:6789/0 conn(0x7f8bcc1591e0 0x7f8bcc159620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48952/0 (socket says 172.21.15.119:48952) 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd1f9b700 1 --2- 172.21.15.119:0/2485457777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 0x7f8bcc0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd179a700 1 -- 172.21.15.119:0/2485457777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 msgr2=0x7f8bcc0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd179a700 1 --2- 172.21.15.119:0/2485457777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 0x7f8bcc0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd179a700 1 -- 172.21.15.119:0/2485457777 >> v1:172.21.15.119:6789/0 conn(0x7f8bcc1591e0 legacy=0x7f8bcc159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd179a700 1 -- 172.21.15.119:0/2485457777 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0cf160 con 0x7f8bcc0b0790 2023-10-08T01:14:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd0f99700 1 -- 172.21.15.119:0/2485457777 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112541825 0 0) 0x7f8bcc0cfa00 con 0x7f8bcc1591e0 2023-10-08T01:14:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd179a700 1 --2- 172.21.15.119:0/2485457777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc0b0790 0x7f8bcc0b28a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0099b0 tx=0x7f8bbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40e424bb80f8863 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.599+0000 7f8bd0f99700 1 -- 172.21.15.119:0/2485457777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc009ea0 con 0x7f8bcc0b0790 2023-10-08T01:14:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd0f99700 1 -- 172.21.15.119:0/2485457777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bbc00b860 con 0x7f8bcc0b0790 2023-10-08T01:14:02.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd0f99700 1 -- 172.21.15.119:0/2485457777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc00bc00 con 0x7f8bcc0b0790 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/2485457777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc0b0790 msgr2=0x7f8bcc0b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/2485457777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc0b0790 0x7f8bcc0b28a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0099b0 tx=0x7f8bbc009250 comp rx=0 tx=0).stop 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/2485457777 shutdown_connections 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/2485457777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 0x7f8bcc0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/2485457777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc0b0790 0x7f8bcc0b28a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/2485457777 >> 172.21.15.119:0/2485457777 conn(0x7f8bcc063230 msgr2=0x7f8bcc063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/2485457777 shutdown_connections 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.600+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/2485457777 wait complete. 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.602+0000 7f8bd2f9d700 1 Processor -- start 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.602+0000 7f8bd2f9d700 1 -- start start 2023-10-08T01:14:02.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd2f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 0x7f8bcc15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd2f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bcc0b0790 0x7f8bcc15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd2f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc1591e0 0x7f8bcc15c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd2f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc15ae00 con 0x7f8bcc1591e0 2023-10-08T01:14:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd2f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0c64e0 con 0x7f8bcc0b0790 2023-10-08T01:14:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd2f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0c5bb0 con 0x7f8bcc0b03c0 2023-10-08T01:14:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc1591e0 0x7f8bcc15c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc1591e0 0x7f8bcc15c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52616/0 (socket says 172.21.15.119:52616) 2023-10-08T01:14:02.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd279c700 1 -- 172.21.15.119:0/269337577 learned_addr learned my addr 172.21.15.119:0/269337577 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd179a700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bcc0b0790 0x7f8bcc15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd1f9b700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 0x7f8bcc15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd279c700 1 -- 172.21.15.119:0/269337577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 msgr2=0x7f8bcc15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd279c700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 0x7f8bcc15dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd279c700 1 -- 172.21.15.119:0/269337577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bcc0b0790 msgr2=0x7f8bcc15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd279c700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bcc0b0790 0x7f8bcc15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.603+0000 7f8bd279c700 1 -- 172.21.15.119:0/269337577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0094f0 con 0x7f8bcc1591e0 2023-10-08T01:14:02.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.604+0000 7f8bd279c700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc1591e0 0x7f8bcc15c330 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400cd80 tx=0x7f8bc4001b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.604+0000 7f8bc1ffb700 1 -- 172.21.15.119:0/269337577 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4005df0 con 0x7f8bcc1591e0 2023-10-08T01:14:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.604+0000 7f8bc1ffb700 1 -- 172.21.15.119:0/269337577 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc400a920 con 0x7f8bcc1591e0 2023-10-08T01:14:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.604+0000 7f8bc1ffb700 1 -- 172.21.15.119:0/269337577 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc400ace0 con 0x7f8bcc1591e0 2023-10-08T01:14:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.604+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc0c5e90 con 0x7f8bcc1591e0 2023-10-08T01:14:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.604+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc0c6970 con 0x7f8bcc1591e0 2023-10-08T01:14:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.605+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc0cc9b0 con 0x7f8bcc1591e0 2023-10-08T01:14:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.606+0000 7f8bc1ffb700 1 -- 172.21.15.119:0/269337577 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 52) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f8bc4007300 con 0x7f8bcc1591e0 2023-10-08T01:14:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.607+0000 7f8bc1ffb700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0067130 0x7f8bb00695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.607+0000 7f8bc1ffb700 1 -- 172.21.15.119:0/269337577 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f8bc4084de0 con 0x7f8bcc1591e0 2023-10-08T01:14:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.607+0000 7f8bd1f9b700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0067130 0x7f8bb00695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:02.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.608+0000 7f8bd1f9b700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0067130 0x7f8bb00695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009ca0 tx=0x7f8bc8008040 comp rx=0 tx=0).ready entity=mgr.6206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:02.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.611+0000 7f8bc1ffb700 1 -- 172.21.15.119:0/269337577 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bc4051b30 con 0x7f8bcc1591e0 2023-10-08T01:14:02.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.825+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bcc04c3c0 con 0x7f8bb0067130 2023-10-08T01:14:02.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.891+0000 7f8bc1ffb700 1 -- 172.21.15.119:0/269337577 <== mgr.6206 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f8bcc04c3c0 con 0x7f8bb0067130 2023-10-08T01:14:02.892 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /bad-path 2023-10-08T01:14:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0067130 msgr2=0x7f8bb00695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0067130 0x7f8bb00695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009ca0 tx=0x7f8bc8008040 comp rx=0 tx=0).stop 2023-10-08T01:14:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc1591e0 msgr2=0x7f8bcc15c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc1591e0 0x7f8bcc15c330 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400cd80 tx=0x7f8bc4001b10 comp rx=0 tx=0).stop 2023-10-08T01:14:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 shutdown_connections 2023-10-08T01:14:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8bb0067130 0x7f8bb00695e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8bcc0b03c0 0x7f8bcc15dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8bcc1591e0 0x7f8bcc15c330 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 --2- 172.21.15.119:0/269337577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8bcc0b0790 0x7f8bcc15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.893+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 >> 172.21.15.119:0/269337577 conn(0x7f8bcc063230 msgr2=0x7f8bcc0cc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.894+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 shutdown_connections 2023-10-08T01:14:02.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:02.894+0000 7f8bd2f9d700 1 -- 172.21.15.119:0/269337577 wait complete. 2023-10-08T01:14:02.912 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-10-08T01:14:02.913 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T01:14:03.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.094+0000 7f816f59d700 1 Processor -- start 2023-10-08T01:14:03.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.094+0000 7f816f59d700 1 -- start start 2023-10-08T01:14:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.095+0000 7f816f59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f816806a710 0x7f8168160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.095+0000 7f816f59d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f816806e150 con 0x7f81680c9db0 2023-10-08T01:14:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.095+0000 7f816f59d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81680c7800 con 0x7f8168153900 2023-10-08T01:14:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.095+0000 7f816f59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81680c7610 con 0x7f816806a710 2023-10-08T01:14:03.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.095+0000 7f816dd9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f81680c9db0 0x7f816806a000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48954/0 (socket says 172.21.15.119:48954) 2023-10-08T01:14:03.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.095+0000 7f816dd9a700 1 -- 172.21.15.119:0/1507728003 learned_addr learned my addr 172.21.15.119:0/1507728003 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:03.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.095+0000 7f816ed9c700 1 --2- 172.21.15.119:0/1507728003 >> v2:172.21.15.119:3300/0 conn(0x7f816806a710 0x7f8168160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816d599700 1 -- 172.21.15.119:0/1507728003 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 54871898 0 0) 0x7f816806e150 con 0x7f81680c9db0 2023-10-08T01:14:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816d599700 1 -- 172.21.15.119:0/1507728003 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8154003570 con 0x7f81680c9db0 2023-10-08T01:14:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816ed9c700 1 -- 172.21.15.119:0/1507728003 >> v1:172.21.15.184:6789/0 conn(0x7f8168153900 legacy=0x7f81680c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816ed9c700 1 -- 172.21.15.119:0/1507728003 >> v1:172.21.15.119:6789/0 conn(0x7f81680c9db0 legacy=0x7f816806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816ed9c700 1 -- 172.21.15.119:0/1507728003 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81680c7ae0 con 0x7f816806a710 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816ed9c700 1 --2- 172.21.15.119:0/1507728003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f816806a710 0x7f8168160e70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f816000af70 tx=0x7f816000a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=549c47a6edd888a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816d599700 1 -- 172.21.15.119:0/1507728003 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816000f070 con 0x7f816806a710 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816d599700 1 -- 172.21.15.119:0/1507728003 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816000a2e0 con 0x7f816806a710 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.096+0000 7f816d599700 1 -- 172.21.15.119:0/1507728003 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816000cc70 con 0x7f816806a710 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.097+0000 7f816f59d700 1 -- 172.21.15.119:0/1507728003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f816806a710 msgr2=0x7f8168160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.097+0000 7f816f59d700 1 --2- 172.21.15.119:0/1507728003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f816806a710 0x7f8168160e70 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f816000af70 tx=0x7f816000a890 comp rx=0 tx=0).stop 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.097+0000 7f816f59d700 1 -- 172.21.15.119:0/1507728003 shutdown_connections 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.097+0000 7f816f59d700 1 --2- 172.21.15.119:0/1507728003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f816806a710 0x7f8168160e70 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.097+0000 7f816f59d700 1 -- 172.21.15.119:0/1507728003 >> 172.21.15.119:0/1507728003 conn(0x7f81680cc1e0 msgr2=0x7f81680cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.097+0000 7f816f59d700 1 -- 172.21.15.119:0/1507728003 shutdown_connections 2023-10-08T01:14:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.097+0000 7f816f59d700 1 -- 172.21.15.119:0/1507728003 wait complete. 2023-10-08T01:14:03.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.098+0000 7f816f59d700 1 Processor -- start 2023-10-08T01:14:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.099+0000 7f816f59d700 1 -- start start 2023-10-08T01:14:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.099+0000 7f816f59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816806a710 0x7f81681388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.099+0000 7f816f59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81680c9db0 0x7f81680acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.099+0000 7f816f59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8168153900 0x7f81680c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.099+0000 7f816f59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8168138e30 con 0x7f8168153900 2023-10-08T01:14:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.099+0000 7f816f59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81680b1e90 con 0x7f816806a710 2023-10-08T01:14:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.099+0000 7f816f59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81680ae5b0 con 0x7f81680c9db0 2023-10-08T01:14:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8168153900 0x7f81680c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8168153900 0x7f81680c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52624/0 (socket says 172.21.15.119:52624) 2023-10-08T01:14:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 -- 172.21.15.119:0/429607912 learned_addr learned my addr 172.21.15.119:0/429607912 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816e59b700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816806a710 0x7f81681388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816dd9a700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81680c9db0 0x7f81680acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 -- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81680c9db0 msgr2=0x7f81680acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81680c9db0 0x7f81680acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 -- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816806a710 msgr2=0x7f81681388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816806a710 0x7f81681388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 -- 172.21.15.119:0/429607912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816000ab30 con 0x7f8168153900 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816e59b700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816806a710 0x7f81681388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.100+0000 7f816ed9c700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8168153900 0x7f81680c5640 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8160013000 tx=0x7f8160019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.101+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160009c30 con 0x7f8168153900 2023-10-08T01:14:03.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.101+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8160009d90 con 0x7f8168153900 2023-10-08T01:14:03.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.101+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160027660 con 0x7f8168153900 2023-10-08T01:14:03.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.101+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81680ad9a0 con 0x7f8168153900 2023-10-08T01:14:03.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.101+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81680adf60 con 0x7f8168153900 2023-10-08T01:14:03.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.102+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81680ae940 con 0x7f8168153900 2023-10-08T01:14:03.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.102+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 52) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f816001d410 con 0x7f8168153900 2023-10-08T01:14:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.103+0000 7f815e7fc700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f814c067130 0x7f814c0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.103+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f81600865f0 con 0x7f8168153900 2023-10-08T01:14:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.104+0000 7f816e59b700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f814c067130 0x7f814c0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.104+0000 7f816e59b700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f814c067130 0x7f814c0695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81640017a0 tx=0x7f8164009040 comp rx=0 tx=0).ready entity=mgr.6206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.108+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81680ae940 con 0x7f8168153900 2023-10-08T01:14:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.324+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 53) v1 ==== 84973+0+0 (secure 0 0 0) 0x7f8160053580 con 0x7f8168153900 2023-10-08T01:14:03.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:03.358+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f81680ae940 con 0x7f8168153900 2023-10-08T01:14:04.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.327+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f8160003020 con 0x7f8168153900 2023-10-08T01:14:04.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.328+0000 7f815e7fc700 1 -- 172.21.15.119:0/429607912 <== mon.0 v2:172.21.15.119:3300/0 9 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v54) v1 ==== 93+0+0 (secure 0 0 0) 0x7f81600541c0 con 0x7f8168153900 2023-10-08T01:14:04.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.331+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f814c067130 msgr2=0x7f814c0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:04.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.331+0000 7f816f59d700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f814c067130 0x7f814c0695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f81640017a0 tx=0x7f8164009040 comp rx=0 tx=0).stop 2023-10-08T01:14:04.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.331+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8168153900 msgr2=0x7f81680c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:04.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.331+0000 7f816f59d700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8168153900 0x7f81680c5640 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f8160013000 tx=0x7f8160019000 comp rx=0 tx=0).stop 2023-10-08T01:14:04.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.332+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 shutdown_connections 2023-10-08T01:14:04.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.332+0000 7f816f59d700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f814c067130 0x7f814c0695e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.332+0000 7f816f59d700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8168153900 0x7f81680c5640 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.332+0000 7f816f59d700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f816806a710 0x7f81681388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.332+0000 7f816f59d700 1 --2- 172.21.15.119:0/429607912 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f81680c9db0 0x7f81680acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.332+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 >> 172.21.15.119:0/429607912 conn(0x7f81680cc1e0 msgr2=0x7f8168153ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:04.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.332+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 shutdown_connections 2023-10-08T01:14:04.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.332+0000 7f816f59d700 1 -- 172.21.15.119:0/429607912 wait complete. 2023-10-08T01:14:04.355 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:14:04.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:14:04.449 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:14:04.529 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:14:04.529 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T01:14:04.529 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:04.530 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T01:14:04.614 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T01:14:04.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:14:04.614 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T01:14:04.651 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:14:04.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.876+0000 7fbbb1b9d700 1 Processor -- start 2023-10-08T01:14:04.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.876+0000 7fbbb1b9d700 1 -- start start 2023-10-08T01:14:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.877+0000 7fbbb1b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbbac142ea0 0x7fbbac0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.877+0000 7fbbb1b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbbac06c0f0 0x7fbbac06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.877+0000 7fbbb1b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbac0b40f0 con 0x7fbbac0ad870 2023-10-08T01:14:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.877+0000 7fbbb1b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac069640 con 0x7fbbac142ea0 2023-10-08T01:14:04.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.877+0000 7fbbb1b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac069920 con 0x7fbbac06c0f0 2023-10-08T01:14:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbb0b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbbac142ea0 0x7fbbac0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbb0b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbbac142ea0 0x7fbbac0b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34628/0 (socket says 172.21.15.119:34628) 2023-10-08T01:14:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbb0b9b700 1 -- 172.21.15.119:0/1163307106 learned_addr learned my addr 172.21.15.119:0/1163307106 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbb139c700 1 --2- 172.21.15.119:0/1163307106 >> v2:172.21.15.184:3300/0 conn(0x7fbbac06c0f0 0x7fbbac06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbabfff700 1 --1- 172.21.15.119:0/1163307106 >> v1:172.21.15.184:6789/0 conn(0x7fbbac0ad870 0x7fbbac06b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:40714/0 (socket says 172.21.15.119:40714) 2023-10-08T01:14:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbb0b9b700 1 -- 172.21.15.119:0/1163307106 >> v2:172.21.15.184:3300/0 conn(0x7fbbac06c0f0 msgr2=0x7fbbac06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:04.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbb0b9b700 1 --2- 172.21.15.119:0/1163307106 >> v2:172.21.15.184:3300/0 conn(0x7fbbac06c0f0 0x7fbbac06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbb0b9b700 1 -- 172.21.15.119:0/1163307106 >> v1:172.21.15.184:6789/0 conn(0x7fbbac0ad870 legacy=0x7fbbac06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.878+0000 7fbbb0b9b700 1 -- 172.21.15.119:0/1163307106 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac069c00 con 0x7fbbac142ea0 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbb0b9b700 1 --2- 172.21.15.119:0/1163307106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac142ea0 0x7fbbac0b3630 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbbac071330 tx=0x7fbba0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33657afa494473ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbab7fe700 1 -- 172.21.15.119:0/1163307106 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0009e70 con 0x7fbbac142ea0 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbab7fe700 1 -- 172.21.15.119:0/1163307106 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba000b860 con 0x7fbbac142ea0 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbab7fe700 1 -- 172.21.15.119:0/1163307106 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba000bc00 con 0x7fbbac142ea0 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/1163307106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac142ea0 msgr2=0x7fbbac0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/1163307106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac142ea0 0x7fbbac0b3630 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbbac071330 tx=0x7fbba0009250 comp rx=0 tx=0).stop 2023-10-08T01:14:04.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/1163307106 shutdown_connections 2023-10-08T01:14:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/1163307106 >> v2:172.21.15.184:3300/0 conn(0x7fbbac06c0f0 0x7fbbac06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.879+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/1163307106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac142ea0 0x7fbbac0b3630 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.880+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/1163307106 >> 172.21.15.119:0/1163307106 conn(0x7fbbac0ce430 msgr2=0x7fbbac0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.880+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/1163307106 shutdown_connections 2023-10-08T01:14:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.880+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/1163307106 wait complete. 2023-10-08T01:14:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.881+0000 7fbbb1b9d700 1 Processor -- start 2023-10-08T01:14:04.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.881+0000 7fbbb1b9d700 1 -- start start 2023-10-08T01:14:04.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb1b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbac06c0f0 0x7fbbac15ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:04.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb1b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbac142ea0 0x7fbbac15a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:04.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb1b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac140150 0x7fbbac158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb1b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0c75e0 con 0x7fbbac140150 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb1b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0c78c0 con 0x7fbbac06c0f0 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb1b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac0c7ba0 con 0x7fbbac142ea0 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb139c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac140150 0x7fbbac158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb0b9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbac06c0f0 0x7fbbac15ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb139c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac140150 0x7fbbac158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34630/0 (socket says 172.21.15.119:34630) 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.882+0000 7fbbb139c700 1 -- 172.21.15.119:0/182210663 learned_addr learned my addr 172.21.15.119:0/182210663 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbabfff700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbac142ea0 0x7fbbac15a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbb139c700 1 -- 172.21.15.119:0/182210663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbac142ea0 msgr2=0x7fbbac15a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:04.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbb139c700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbac142ea0 0x7fbbac15a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbb139c700 1 -- 172.21.15.119:0/182210663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbac06c0f0 msgr2=0x7fbbac15ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:04.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbb139c700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbac06c0f0 0x7fbbac15ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:04.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbb139c700 1 -- 172.21.15.119:0/182210663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba00094f0 con 0x7fbbac140150 2023-10-08T01:14:04.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbb139c700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac140150 0x7fbbac158b80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009ef0 tx=0x7fbba400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:04.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbba8ff9700 1 -- 172.21.15.119:0/182210663 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400cb70 con 0x7fbbac140150 2023-10-08T01:14:04.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbba8ff9700 1 -- 172.21.15.119:0/182210663 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400f040 con 0x7fbbac140150 2023-10-08T01:14:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbba8ff9700 1 -- 172.21.15.119:0/182210663 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4014480 con 0x7fbbac140150 2023-10-08T01:14:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac0c7e80 con 0x7fbbac140150 2023-10-08T01:14:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.883+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac15f110 con 0x7fbbac140150 2023-10-08T01:14:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.885+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac0c83a0 con 0x7fbbac140150 2023-10-08T01:14:04.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.885+0000 7fbba8ff9700 1 -- 172.21.15.119:0/182210663 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fbba4014830 con 0x7fbbac140150 2023-10-08T01:14:04.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.886+0000 7fbba8ff9700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbb90067170 0x7fbb90069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:04.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.886+0000 7fbba8ff9700 1 -- 172.21.15.119:0/182210663 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7fbba40858e0 con 0x7fbbac140150 2023-10-08T01:14:04.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.886+0000 7fbbb0b9b700 1 -- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbb90067170 msgr2=0x7fbb90069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:04.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.886+0000 7fbbb0b9b700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbb90067170 0x7fbb90069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:04.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:04.890+0000 7fbba8ff9700 1 -- 172.21.15.119:0/182210663 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbac0c83a0 con 0x7fbbac140150 2023-10-08T01:14:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.087+0000 7fbbb0b9b700 1 -- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbb90067170 msgr2=0x7fbb90069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.087+0000 7fbbb0b9b700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbb90067170 0x7fbb90069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.151+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbac0c83a0 con 0x7fbbac140150 2023-10-08T01:14:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.152+0000 7fbba8ff9700 1 -- 172.21.15.119:0/182210663 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fbbac0c83a0 con 0x7fbbac140150 2023-10-08T01:14:05.153 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:05.153 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1682556092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1682556092},{"type":"v1","addr":"172.21.15.184:6835","nonce":1682556092}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/691036731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":691036731},{"type":"v1","addr":"172.21.15.184:6834","nonce":691036731}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7182,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3636556734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3636556734},{"type":"v1","addr":"172.21.15.119:6834","nonce":3636556734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:44.715284+0000","modified":"2023-10-08T01:12:48.271445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7185":{"gid":7185,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3302512146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3302512146},{"type":"v1","addr":"172.21.15.119:6835","nonce":3302512146}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbb90067170 msgr2=0x7fbb90069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbb90067170 0x7fbb90069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac140150 msgr2=0x7fbbac158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac140150 0x7fbbac158b80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fbba4009ef0 tx=0x7fbba400b040 comp rx=0 tx=0).stop 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 shutdown_connections 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbb90067170 0x7fbb90069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbac140150 0x7fbbac158b80 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbac06c0f0 0x7fbbac15ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 --2- 172.21.15.119:0/182210663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbac142ea0 0x7fbbac15a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 >> 172.21.15.119:0/182210663 conn(0x7fbbac0ce430 msgr2=0x7fbbac1420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 shutdown_connections 2023-10-08T01:14:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.154+0000 7fbbb1b9d700 1 -- 172.21.15.119:0/182210663 wait complete. 2023-10-08T01:14:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 48 2023-10-08T01:14:05.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:14:05.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.364+0000 7efd3e79d700 1 Processor -- start 2023-10-08T01:14:05.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.364+0000 7efd3e79d700 1 -- start start 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3e79d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd38159420 0x7efd380b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3e79d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efd3815d230 0x7efd3815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3e79d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380b2980 con 0x7efd381597f0 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3e79d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3815db70 con 0x7efd38159420 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3e79d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380b11d0 con 0x7efd3815d230 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3d79b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd38159420 0x7efd380b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3d79b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efd38159420 0x7efd380b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34640/0 (socket says 172.21.15.119:34640) 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3d79b700 1 -- 172.21.15.119:0/46777923 learned_addr learned my addr 172.21.15.119:0/46777923 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.365+0000 7efd3df9c700 1 --2- 172.21.15.119:0/46777923 >> v2:172.21.15.184:3300/0 conn(0x7efd3815d230 0x7efd3815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.366+0000 7efd3d79b700 1 -- 172.21.15.119:0/46777923 >> v2:172.21.15.184:3300/0 conn(0x7efd3815d230 msgr2=0x7efd3815d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:05.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.366+0000 7efd3d79b700 1 --2- 172.21.15.119:0/46777923 >> v2:172.21.15.184:3300/0 conn(0x7efd3815d230 0x7efd3815d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.366+0000 7efd3d79b700 1 -- 172.21.15.119:0/46777923 >> v1:172.21.15.184:6789/0 conn(0x7efd381597f0 legacy=0x7efd380b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.366+0000 7efd3d79b700 1 -- 172.21.15.119:0/46777923 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd380b14b0 con 0x7efd38159420 2023-10-08T01:14:05.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.366+0000 7efd3d79b700 1 --2- 172.21.15.119:0/46777923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd38159420 0x7efd380b5f40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7efd3815d170 tx=0x7efd34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271cd2f68e49ca2c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:05.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.366+0000 7efd2ffff700 1 -- 172.21.15.119:0/46777923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd34009e70 con 0x7efd38159420 2023-10-08T01:14:05.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.366+0000 7efd2ffff700 1 -- 172.21.15.119:0/46777923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3400b860 con 0x7efd38159420 2023-10-08T01:14:05.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.366+0000 7efd2ffff700 1 -- 172.21.15.119:0/46777923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3400bc00 con 0x7efd38159420 2023-10-08T01:14:05.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.367+0000 7efd3e79d700 1 -- 172.21.15.119:0/46777923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd38159420 msgr2=0x7efd380b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.367+0000 7efd3e79d700 1 --2- 172.21.15.119:0/46777923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd38159420 0x7efd380b5f40 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7efd3815d170 tx=0x7efd34009250 comp rx=0 tx=0).stop 2023-10-08T01:14:05.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.367+0000 7efd3e79d700 1 -- 172.21.15.119:0/46777923 shutdown_connections 2023-10-08T01:14:05.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.367+0000 7efd3e79d700 1 --2- 172.21.15.119:0/46777923 >> v2:172.21.15.184:3300/0 conn(0x7efd3815d230 0x7efd3815d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.367+0000 7efd3e79d700 1 --2- 172.21.15.119:0/46777923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd38159420 0x7efd380b5f40 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.367+0000 7efd3e79d700 1 -- 172.21.15.119:0/46777923 >> 172.21.15.119:0/46777923 conn(0x7efd380d0660 msgr2=0x7efd380d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:05.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.367+0000 7efd3e79d700 1 -- 172.21.15.119:0/46777923 shutdown_connections 2023-10-08T01:14:05.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.367+0000 7efd3e79d700 1 -- 172.21.15.119:0/46777923 wait complete. 2023-10-08T01:14:05.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.370+0000 7efd3e79d700 1 Processor -- start 2023-10-08T01:14:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.370+0000 7efd3e79d700 1 -- start start 2023-10-08T01:14:05.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.371+0000 7efd3e79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd38159420 0x7efd38146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.371+0000 7efd3e79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd381597f0 0x7efd380cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.371+0000 7efd3e79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3815d230 0x7efd38158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.371+0000 7efd3e79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380cb290 con 0x7efd381597f0 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.371+0000 7efd3e79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380ca6c0 con 0x7efd3815d230 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.371+0000 7efd3e79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd380ca960 con 0x7efd38159420 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.371+0000 7efd3cf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd381597f0 0x7efd380cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3cf9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd381597f0 0x7efd380cb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34652/0 (socket says 172.21.15.119:34652) 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3cf9a700 1 -- 172.21.15.119:0/4042885112 learned_addr learned my addr 172.21.15.119:0/4042885112 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3d79b700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd38159420 0x7efd38146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3df9c700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3815d230 0x7efd38158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3cf9a700 1 -- 172.21.15.119:0/4042885112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd38159420 msgr2=0x7efd38146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3cf9a700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd38159420 0x7efd38146fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3cf9a700 1 -- 172.21.15.119:0/4042885112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3815d230 msgr2=0x7efd38158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3cf9a700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3815d230 0x7efd38158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3cf9a700 1 -- 172.21.15.119:0/4042885112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd340094f0 con 0x7efd381597f0 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3cf9a700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd381597f0 0x7efd380cb720 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c7c0 tx=0x7efd28007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd2d7fa700 1 -- 172.21.15.119:0/4042885112 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28011590 con 0x7efd381597f0 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd2d7fa700 1 -- 172.21.15.119:0/4042885112 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd280116f0 con 0x7efd381597f0 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd2d7fa700 1 -- 172.21.15.119:0/4042885112 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28011ab0 con 0x7efd381597f0 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.372+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd380cac40 con 0x7efd381597f0 2023-10-08T01:14:05.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.373+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd38147500 con 0x7efd381597f0 2023-10-08T01:14:05.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.374+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3804c3c0 con 0x7efd381597f0 2023-10-08T01:14:05.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.374+0000 7efd2d7fa700 1 -- 172.21.15.119:0/4042885112 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7efd2801b3b0 con 0x7efd381597f0 2023-10-08T01:14:05.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.375+0000 7efd2d7fa700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efd1c067170 0x7efd1c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.375+0000 7efd2d7fa700 1 -- 172.21.15.119:0/4042885112 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7efd28083f60 con 0x7efd381597f0 2023-10-08T01:14:05.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.375+0000 7efd3d79b700 1 -- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efd1c067170 msgr2=0x7efd1c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:05.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.375+0000 7efd3d79b700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efd1c067170 0x7efd1c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:05.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.379+0000 7efd2d7fa700 1 -- 172.21.15.119:0/4042885112 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd28050c20 con 0x7efd381597f0 2023-10-08T01:14:05.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.576+0000 7efd3d79b700 1 -- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efd1c067170 msgr2=0x7efd1c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:05.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.576+0000 7efd3d79b700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efd1c067170 0x7efd1c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:05.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.579+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd380cb3d0 con 0x7efd381597f0 2023-10-08T01:14:05.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.580+0000 7efd2d7fa700 1 -- 172.21.15.119:0/4042885112 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16430 (secure 0 0 0) 0x7efd28017070 con 0x7efd381597f0 2023-10-08T01:14:05.581 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:05.581 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":107,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:02.302915+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:42.395485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:43.700121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.c","mappings":[{"from":1,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1425177173":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/3497965416":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/3790087460":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/898896673":"2023-10-09T01:13:00.409133+0000","172.21.15.119:6837/85511":"2023-10-09T01:13:00.409133+0000","172.21.15.119:6836/85511":"2023-10-09T01:13:00.409133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:14:05.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efd1c067170 msgr2=0x7efd1c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:05.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efd1c067170 0x7efd1c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd381597f0 msgr2=0x7efd380cb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd381597f0 0x7efd380cb720 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7efd2800c7c0 tx=0x7efd28007f50 comp rx=0 tx=0).stop 2023-10-08T01:14:05.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 shutdown_connections 2023-10-08T01:14:05.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efd1c067170 0x7efd1c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efd38159420 0x7efd38146fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efd3815d230 0x7efd38158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 --2- 172.21.15.119:0/4042885112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd381597f0 0x7efd380cb720 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 >> 172.21.15.119:0/4042885112 conn(0x7efd380d0660 msgr2=0x7efd38069ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:05.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 shutdown_connections 2023-10-08T01:14:05.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.582+0000 7efd3e79d700 1 -- 172.21.15.119:0/4042885112 wait complete. 2023-10-08T01:14:05.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T01:12:42.395485+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T01:12:43.700121+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '104', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T01:14:05.603 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:14:05.604 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:05.603+0000 7fa27bef4180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:14:05.678 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:05.677+0000 7f75e330a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:14:05.764 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:05.764+0000 7fa27bef4180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:14:05.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.794+0000 7f7ffab9d700 1 Processor -- start 2023-10-08T01:14:05.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.794+0000 7f7ffab9d700 1 -- start start 2023-10-08T01:14:05.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.796+0000 7f7ffab9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7ff40c9db0 0x7f7ff406a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffab9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ff4157130 0x7f7ff406a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffab9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff406e150 con 0x7f7ff4153900 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffab9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff40c7610 con 0x7f7ff4157130 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffab9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff40c7800 con 0x7f7ff40c9db0 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ff9b9b700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f7ff4153900 0x7f7ff40c6f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:40738/0 (socket says 172.21.15.119:40738) 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffa39c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7ff4157130 0x7f7ff406a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ff9b9b700 1 -- 172.21.15.119:0/2503959168 learned_addr learned my addr 172.21.15.119:0/2503959168 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ff939a700 1 --2- 172.21.15.119:0/2503959168 >> v2:172.21.15.184:3300/0 conn(0x7f7ff40c9db0 0x7f7ff406a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ff8b99700 1 -- 172.21.15.119:0/2503959168 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227488476 0 0) 0x7f7ff406e150 con 0x7f7ff4153900 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ff8b99700 1 -- 172.21.15.119:0/2503959168 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fe0003570 con 0x7f7ff4153900 2023-10-08T01:14:05.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffa39c700 1 -- 172.21.15.119:0/2503959168 >> v2:172.21.15.184:3300/0 conn(0x7f7ff40c9db0 msgr2=0x7f7ff406a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffa39c700 1 --2- 172.21.15.119:0/2503959168 >> v2:172.21.15.184:3300/0 conn(0x7f7ff40c9db0 0x7f7ff406a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffa39c700 1 -- 172.21.15.119:0/2503959168 >> v1:172.21.15.184:6789/0 conn(0x7f7ff4153900 legacy=0x7f7ff40c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.797+0000 7f7ffa39c700 1 -- 172.21.15.119:0/2503959168 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff40c7ae0 con 0x7f7ff4157130 2023-10-08T01:14:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.798+0000 7f7ffa39c700 1 --2- 172.21.15.119:0/2503959168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4157130 0x7f7ff406a540 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40aed40 tx=0x7f7fec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b284fadfaaafb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:05.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.798+0000 7f7ff8b99700 1 -- 172.21.15.119:0/2503959168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec011630 con 0x7f7ff4157130 2023-10-08T01:14:05.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.798+0000 7f7ff8b99700 1 -- 172.21.15.119:0/2503959168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fec0117b0 con 0x7f7ff4157130 2023-10-08T01:14:05.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.798+0000 7f7ff8b99700 1 -- 172.21.15.119:0/2503959168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fec011b70 con 0x7f7ff4157130 2023-10-08T01:14:05.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.799+0000 7f7ffab9d700 1 -- 172.21.15.119:0/2503959168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4157130 msgr2=0x7f7ff406a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.799+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/2503959168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4157130 0x7f7ff406a540 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40aed40 tx=0x7f7fec0091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.799+0000 7f7ffab9d700 1 -- 172.21.15.119:0/2503959168 shutdown_connections 2023-10-08T01:14:05.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.799+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/2503959168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4157130 0x7f7ff406a540 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.799+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/2503959168 >> v2:172.21.15.184:3300/0 conn(0x7f7ff40c9db0 0x7f7ff406a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.799+0000 7f7ffab9d700 1 -- 172.21.15.119:0/2503959168 >> 172.21.15.119:0/2503959168 conn(0x7f7ff40cc1e0 msgr2=0x7f7ff40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:05.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.799+0000 7f7ffab9d700 1 -- 172.21.15.119:0/2503959168 shutdown_connections 2023-10-08T01:14:05.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.799+0000 7f7ffab9d700 1 -- 172.21.15.119:0/2503959168 wait complete. 2023-10-08T01:14:05.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.800+0000 7f7ffab9d700 1 Processor -- start 2023-10-08T01:14:05.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.801+0000 7f7ffab9d700 1 -- start start 2023-10-08T01:14:05.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.801+0000 7f7ffab9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ff40c9db0 0x7f7ff4153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.801+0000 7f7ffab9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4153900 0x7f7ff40ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.801+0000 7f7ffab9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ff4157130 0x7f7ff4155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.801+0000 7f7ffab9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4147b60 con 0x7f7ff4153900 2023-10-08T01:14:05.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.801+0000 7f7ffab9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff40abfe0 con 0x7f7ff40c9db0 2023-10-08T01:14:05.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.801+0000 7f7ffab9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4147510 con 0x7f7ff4157130 2023-10-08T01:14:05.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4153900 0x7f7ff40ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4153900 0x7f7ff40ae2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34674/0 (socket says 172.21.15.119:34674) 2023-10-08T01:14:05.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 -- 172.21.15.119:0/3458758815 learned_addr learned my addr 172.21.15.119:0/3458758815 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:05.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff9b9b700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ff40c9db0 0x7f7ff4153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ffa39c700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ff4157130 0x7f7ff4155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:05.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 -- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ff4157130 msgr2=0x7f7ff4155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ff4157130 0x7f7ff4155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 -- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ff40c9db0 msgr2=0x7f7ff4153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:05.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ff40c9db0 0x7f7ff4153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:05.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 -- 172.21.15.119:0/3458758815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fec009420 con 0x7f7ff4153900 2023-10-08T01:14:05.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff9b9b700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ff40c9db0 0x7f7ff4153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:05.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.802+0000 7f7ff939a700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4153900 0x7f7ff40ae2d0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400ec50 tx=0x7f7fe40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:05.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.803+0000 7f7fe9ffb700 1 -- 172.21.15.119:0/3458758815 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400c850 con 0x7f7ff4153900 2023-10-08T01:14:05.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.803+0000 7f7fe9ffb700 1 -- 172.21.15.119:0/3458758815 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe400c9b0 con 0x7f7ff4153900 2023-10-08T01:14:05.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.803+0000 7f7fe9ffb700 1 -- 172.21.15.119:0/3458758815 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe400cd70 con 0x7f7ff4153900 2023-10-08T01:14:05.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.803+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff41477f0 con 0x7f7ff4153900 2023-10-08T01:14:05.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.803+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff40ac2c0 con 0x7f7ff4153900 2023-10-08T01:14:05.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.804+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff4147cd0 con 0x7f7ff4153900 2023-10-08T01:14:05.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.805+0000 7f7fe9ffb700 1 -- 172.21.15.119:0/3458758815 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f7fe4013680 con 0x7f7ff4153900 2023-10-08T01:14:05.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.806+0000 7f7fe9ffb700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fd8067170 0x7f7fd8069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:05.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.806+0000 7f7fe9ffb700 1 -- 172.21.15.119:0/3458758815 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f7fe4084e70 con 0x7f7ff4153900 2023-10-08T01:14:05.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.806+0000 7f7ff9b9b700 1 -- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fd8067170 msgr2=0x7f7fd8069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:05.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.806+0000 7f7ff9b9b700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fd8067170 0x7f7fd8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:05.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:05.810+0000 7f7fe9ffb700 1 -- 172.21.15.119:0/3458758815 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fe4051b30 con 0x7f7ff4153900 2023-10-08T01:14:05.850 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:05.850+0000 7fa27bef4180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:14:05.851 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:05.850+0000 7f75e330a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:14:05.941 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:05.938+0000 7f75e330a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:14:06.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.007+0000 7f7ff9b9b700 1 -- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fd8067170 msgr2=0x7f7fd8069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:06.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.007+0000 7f7ff9b9b700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fd8067170 0x7f7fd8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.008+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ff40c7dc0 con 0x7f7ff4153900 2023-10-08T01:14:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.008+0000 7f7fe9ffb700 1 -- 172.21.15.119:0/3458758815 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fe4055150 con 0x7f7ff4153900 2023-10-08T01:14:06.009 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:14:06.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fd8067170 msgr2=0x7f7fd8069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fd8067170 0x7f7fd8069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4153900 msgr2=0x7f7ff40ae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4153900 0x7f7ff40ae2d0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f7fe400ec50 tx=0x7f7fe40091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 shutdown_connections 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fd8067170 0x7f7fd8069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ff4157130 0x7f7ff4155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ff40c9db0 0x7f7ff4153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 --2- 172.21.15.119:0/3458758815 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ff4153900 0x7f7ff40ae2d0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 >> 172.21.15.119:0/3458758815 conn(0x7f7ff40cc1e0 msgr2=0x7f7ff406eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 shutdown_connections 2023-10-08T01:14:06.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.011+0000 7f7ffab9d700 1 -- 172.21.15.119:0/3458758815 wait complete. 2023-10-08T01:14:06.031 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T01:14:06.072 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:06.072+0000 7fa27bef4180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:14:06.172 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:06.171+0000 7f75e330a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:14:06.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.219+0000 7f138f39d700 1 Processor -- start 2023-10-08T01:14:06.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.219+0000 7f138f39d700 1 -- start start 2023-10-08T01:14:06.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.220+0000 7f138f39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13880b4e70 0x7f13880ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.220+0000 7f138f39d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13880b16d0 0x7f13880b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.220+0000 7f138f39d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13880b2080 con 0x7f138815d240 2023-10-08T01:14:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.220+0000 7f138f39d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880cfb00 con 0x7f13880b16d0 2023-10-08T01:14:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.220+0000 7f138f39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880b5c40 con 0x7f13880b4e70 2023-10-08T01:14:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138eb9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f138815d240 0x7f138815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47022/0 (socket says 172.21.15.119:47022) 2023-10-08T01:14:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138eb9c700 1 -- 172.21.15.119:0/1279638054 learned_addr learned my addr 172.21.15.119:0/1279638054 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138e39b700 1 --2- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13880b4e70 0x7f13880ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138db9a700 1 --2- 172.21.15.119:0/1279638054 >> v2:172.21.15.184:3300/0 conn(0x7f13880b16d0 0x7f13880b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138d399700 1 -- 172.21.15.119:0/1279638054 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589777406 0 0) 0x7f13880b2080 con 0x7f138815d240 2023-10-08T01:14:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138d399700 1 -- 172.21.15.119:0/1279638054 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1374003570 con 0x7f138815d240 2023-10-08T01:14:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138db9a700 1 -- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13880b4e70 msgr2=0x7f13880ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138db9a700 1 --2- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13880b4e70 0x7f13880ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138db9a700 1 -- 172.21.15.119:0/1279638054 >> v1:172.21.15.119:6789/0 conn(0x7f138815d240 legacy=0x7f138815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138db9a700 1 -- 172.21.15.119:0/1279638054 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880b5f20 con 0x7f13880b16d0 2023-10-08T01:14:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.221+0000 7f138e39b700 1 --2- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13880b4e70 0x7f13880ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.222+0000 7f138db9a700 1 --2- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 0x7f13880b1b40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f13780098e0 tx=0x7f13780091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc8eab2942d72939 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.222+0000 7f138d399700 1 -- 172.21.15.119:0/1279638054 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378011630 con 0x7f13880b16d0 2023-10-08T01:14:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.222+0000 7f138d399700 1 -- 172.21.15.119:0/1279638054 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378011790 con 0x7f13880b16d0 2023-10-08T01:14:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.222+0000 7f138d399700 1 -- 172.21.15.119:0/1279638054 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378011b10 con 0x7f13880b16d0 2023-10-08T01:14:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.223+0000 7f138f39d700 1 -- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 msgr2=0x7f13880b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.223+0000 7f138f39d700 1 --2- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 0x7f13880b1b40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f13780098e0 tx=0x7f13780091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.223+0000 7f138f39d700 1 -- 172.21.15.119:0/1279638054 shutdown_connections 2023-10-08T01:14:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.223+0000 7f138f39d700 1 --2- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13880b4e70 0x7f13880ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.223+0000 7f138f39d700 1 --2- 172.21.15.119:0/1279638054 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 0x7f13880b1b40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.223+0000 7f138f39d700 1 -- 172.21.15.119:0/1279638054 >> 172.21.15.119:0/1279638054 conn(0x7f13880d10c0 msgr2=0x7f13880affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.223+0000 7f138f39d700 1 -- 172.21.15.119:0/1279638054 shutdown_connections 2023-10-08T01:14:06.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.223+0000 7f138f39d700 1 -- 172.21.15.119:0/1279638054 wait complete. 2023-10-08T01:14:06.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.224+0000 7f138f39d700 1 Processor -- start 2023-10-08T01:14:06.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.224+0000 7f138f39d700 1 -- start start 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138f39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 0x7f138815f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138f39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13880b4e70 0x7f138816a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138f39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138815d240 0x7f138815dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138f39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138816ae40 con 0x7f13880b4e70 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138f39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138816b0e0 con 0x7f13880b16d0 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138f39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138816b3c0 con 0x7f138815d240 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138db9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13880b4e70 0x7f138816a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138db9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13880b4e70 0x7f138816a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34682/0 (socket says 172.21.15.119:34682) 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138db9a700 1 -- 172.21.15.119:0/1710380699 learned_addr learned my addr 172.21.15.119:0/1710380699 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.225+0000 7f138e39b700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 0x7f138815f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f138eb9c700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138815d240 0x7f138815dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f138db9a700 1 -- 172.21.15.119:0/1710380699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138815d240 msgr2=0x7f138815dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f138db9a700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138815d240 0x7f138815dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f138db9a700 1 -- 172.21.15.119:0/1710380699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 msgr2=0x7f138815f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f138db9a700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 0x7f138815f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f138db9a700 1 -- 172.21.15.119:0/1710380699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384009420 con 0x7f13880b4e70 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f138db9a700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13880b4e70 0x7f138816a900 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f1388070dd0 tx=0x7f1378017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f137e7fc700 1 -- 172.21.15.119:0/1710380699 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1378011900 con 0x7f13880b4e70 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f137e7fc700 1 -- 172.21.15.119:0/1710380699 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1378011a80 con 0x7f13880b4e70 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1378009420 con 0x7f13880b4e70 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.226+0000 7f137e7fc700 1 -- 172.21.15.119:0/1710380699 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137801a680 con 0x7f13880b4e70 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.227+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138816b980 con 0x7f13880b4e70 2023-10-08T01:14:06.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.228+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13880b4650 con 0x7f13880b4e70 2023-10-08T01:14:06.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.228+0000 7f137e7fc700 1 -- 172.21.15.119:0/1710380699 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f137801aab0 con 0x7f13880b4e70 2023-10-08T01:14:06.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.229+0000 7f137e7fc700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f136c067170 0x7f136c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.229+0000 7f138e39b700 1 -- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f136c067170 msgr2=0x7f136c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:06.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.229+0000 7f137e7fc700 1 -- 172.21.15.119:0/1710380699 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f1378086790 con 0x7f13880b4e70 2023-10-08T01:14:06.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.229+0000 7f138e39b700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f136c067170 0x7f136c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:06.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.233+0000 7f137e7fc700 1 -- 172.21.15.119:0/1710380699 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1378053450 con 0x7f13880b4e70 2023-10-08T01:14:06.247 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:06.247+0000 7fa27bef4180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:14:06.339 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:06.339+0000 7fa27bef4180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:14:06.362 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:06.361+0000 7f75e330a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:14:06.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.430+0000 7f138e39b700 1 -- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f136c067170 msgr2=0x7f136c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:06.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.430+0000 7f138e39b700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f136c067170 0x7f136c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:06.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.437+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f138804bb80 con 0x7f13880b4e70 2023-10-08T01:14:06.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.438+0000 7f137e7fc700 1 -- 172.21.15.119:0/1710380699 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1378056a70 con 0x7f13880b4e70 2023-10-08T01:14:06.439 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T01:14:06.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f136c067170 msgr2=0x7f136c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:06.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f136c067170 0x7f136c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13880b4e70 msgr2=0x7f138816a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13880b4e70 0x7f138816a900 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f1388070dd0 tx=0x7f1378017000 comp rx=0 tx=0).stop 2023-10-08T01:14:06.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 shutdown_connections 2023-10-08T01:14:06.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f136c067170 0x7f136c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f138815d240 0x7f138815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13880b16d0 0x7f138815f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 --2- 172.21.15.119:0/1710380699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13880b4e70 0x7f138816a900 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 >> 172.21.15.119:0/1710380699 conn(0x7f13880d10c0 msgr2=0x7f13880b52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 shutdown_connections 2023-10-08T01:14:06.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.440+0000 7f138f39d700 1 -- 172.21.15.119:0/1710380699 wait complete. 2023-10-08T01:14:06.456 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:06.455+0000 7f75e330a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:14:06.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T01:14:06.501 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:06.501+0000 7fa27bef4180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:14:06.599 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:06.599+0000 7fa27bef4180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:14:06.626 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:06.625+0000 7f75e330a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:14:06.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.648+0000 7f494559d700 1 Processor -- start 2023-10-08T01:14:06.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.649+0000 7f494559d700 1 -- start start 2023-10-08T01:14:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.650+0000 7f494559d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f49400b1b40 0x7f49400b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.650+0000 7f494559d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49400cfb00 con 0x7f49400b4e70 2023-10-08T01:14:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.650+0000 7f494559d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49400b5c40 con 0x7f49400b16d0 2023-10-08T01:14:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.650+0000 7f494559d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49400b5f20 con 0x7f49400b1b40 2023-10-08T01:14:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.650+0000 7f493ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f49400b4e70 0x7f49400ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47026/0 (socket says 172.21.15.119:47026) 2023-10-08T01:14:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.650+0000 7f493ffff700 1 -- 172.21.15.119:0/2025582715 learned_addr learned my addr 172.21.15.119:0/2025582715 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.650+0000 7f4944d9c700 1 --2- 172.21.15.119:0/2025582715 >> v2:172.21.15.184:3300/0 conn(0x7f49400b1b40 0x7f49400b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.650+0000 7f493effd700 1 -- 172.21.15.119:0/2025582715 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4171028433 0 0) 0x7f49400cfb00 con 0x7f49400b4e70 2023-10-08T01:14:06.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f493effd700 1 -- 172.21.15.119:0/2025582715 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4928003570 con 0x7f49400b4e70 2023-10-08T01:14:06.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f493effd700 1 -- 172.21.15.119:0/2025582715 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1133889705 0 0) 0x7f49400b5c40 con 0x7f49400b16d0 2023-10-08T01:14:06.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f493effd700 1 -- 172.21.15.119:0/2025582715 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49400cfb00 con 0x7f49400b16d0 2023-10-08T01:14:06.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f493effd700 1 -- 172.21.15.119:0/2025582715 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1802309967 0 0) 0x7f4928003570 con 0x7f49400b4e70 2023-10-08T01:14:06.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f4944d9c700 1 -- 172.21.15.119:0/2025582715 >> v1:172.21.15.184:6789/0 conn(0x7f49400b16d0 legacy=0x7f49400b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f4944d9c700 1 -- 172.21.15.119:0/2025582715 >> v1:172.21.15.119:6789/0 conn(0x7f49400b4e70 legacy=0x7f49400ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f4944d9c700 1 -- 172.21.15.119:0/2025582715 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49400b6200 con 0x7f49400b1b40 2023-10-08T01:14:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f4944d9c700 1 --2- 172.21.15.119:0/2025582715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b1b40 0x7f49400b3ac0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4938009e60 tx=0x7f4938009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f0156b005c17d1b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f493effd700 1 -- 172.21.15.119:0/2025582715 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493800e070 con 0x7f49400b1b40 2023-10-08T01:14:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f493effd700 1 -- 172.21.15.119:0/2025582715 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493800b7a0 con 0x7f49400b1b40 2023-10-08T01:14:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.651+0000 7f493effd700 1 -- 172.21.15.119:0/2025582715 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4930001470 con 0x7f49400b1b40 2023-10-08T01:14:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.652+0000 7f494559d700 1 -- 172.21.15.119:0/2025582715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b1b40 msgr2=0x7f49400b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.652+0000 7f494559d700 1 --2- 172.21.15.119:0/2025582715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b1b40 0x7f49400b3ac0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4938009e60 tx=0x7f4938009780 comp rx=0 tx=0).stop 2023-10-08T01:14:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.652+0000 7f494559d700 1 -- 172.21.15.119:0/2025582715 shutdown_connections 2023-10-08T01:14:06.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.652+0000 7f494559d700 1 --2- 172.21.15.119:0/2025582715 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b1b40 0x7f49400b3ac0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.652+0000 7f494559d700 1 -- 172.21.15.119:0/2025582715 >> 172.21.15.119:0/2025582715 conn(0x7f49400d10c0 msgr2=0x7f49400affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:06.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.652+0000 7f494559d700 1 -- 172.21.15.119:0/2025582715 shutdown_connections 2023-10-08T01:14:06.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.652+0000 7f494559d700 1 -- 172.21.15.119:0/2025582715 wait complete. 2023-10-08T01:14:06.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.654+0000 7f494559d700 1 Processor -- start 2023-10-08T01:14:06.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.654+0000 7f494559d700 1 -- start start 2023-10-08T01:14:06.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.654+0000 7f494559d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49400b16d0 0x7f4940148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f494559d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49400b1b40 0x7f49400b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f494559d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b4e70 0x7f49400b5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f494559d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400cbd10 con 0x7f49400b1b40 2023-10-08T01:14:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f494559d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49401485d0 con 0x7f49400b4e70 2023-10-08T01:14:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f494559d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400cb140 con 0x7f49400b16d0 2023-10-08T01:14:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49400b1b40 0x7f49400b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49400b1b40 0x7f49400b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34688/0 (socket says 172.21.15.119:34688) 2023-10-08T01:14:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493f7fe700 1 -- 172.21.15.119:0/3674404456 learned_addr learned my addr 172.21.15.119:0/3674404456 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493ffff700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49400b16d0 0x7f4940148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f4944d9c700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b4e70 0x7f49400b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493f7fe700 1 -- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49400b16d0 msgr2=0x7f4940148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493f7fe700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49400b16d0 0x7f4940148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493f7fe700 1 -- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b4e70 msgr2=0x7f49400b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493f7fe700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b4e70 0x7f49400b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f493f7fe700 1 -- 172.21.15.119:0/3674404456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938009a20 con 0x7f49400b1b40 2023-10-08T01:14:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.655+0000 7f4944d9c700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b4e70 0x7f49400b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.656+0000 7f493f7fe700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49400b1b40 0x7f49400b5240 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f493400c790 tx=0x7f4934007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.656+0000 7f4927fff700 1 -- 172.21.15.119:0/3674404456 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934010510 con 0x7f49400b1b40 2023-10-08T01:14:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.656+0000 7f4927fff700 1 -- 172.21.15.119:0/3674404456 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4934010670 con 0x7f49400b1b40 2023-10-08T01:14:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.656+0000 7f4927fff700 1 -- 172.21.15.119:0/3674404456 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4934010a30 con 0x7f49400b1b40 2023-10-08T01:14:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.656+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49400cc1a0 con 0x7f49400b1b40 2023-10-08T01:14:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.656+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49400cb6c0 con 0x7f49400b1b40 2023-10-08T01:14:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.657+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49400cbe80 con 0x7f49400b1b40 2023-10-08T01:14:06.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.658+0000 7f4927fff700 1 -- 172.21.15.119:0/3674404456 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f493401b3b0 con 0x7f49400b1b40 2023-10-08T01:14:06.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.659+0000 7f4927fff700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f492c067170 0x7f492c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:06.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.659+0000 7f4927fff700 1 -- 172.21.15.119:0/3674404456 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f4934084470 con 0x7f49400b1b40 2023-10-08T01:14:06.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.659+0000 7f493ffff700 1 -- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f492c067170 msgr2=0x7f492c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:06.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.659+0000 7f493ffff700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f492c067170 0x7f492c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.663+0000 7f4927fff700 1 -- 172.21.15.119:0/3674404456 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4934051130 con 0x7f49400b1b40 2023-10-08T01:14:06.747 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:06.746+0000 7f75e330a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:14:06.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.860+0000 7f493ffff700 1 -- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f492c067170 msgr2=0x7f492c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:06.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.860+0000 7f493ffff700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f492c067170 0x7f492c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:06.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.877+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f494004bb80 con 0x7f49400b1b40 2023-10-08T01:14:06.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.878+0000 7f4927fff700 1 -- 172.21.15.119:0/3674404456 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4934017020 con 0x7f49400b1b40 2023-10-08T01:14:06.879 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T01:14:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f492c067170 msgr2=0x7f492c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f492c067170 0x7f492c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49400b1b40 msgr2=0x7f49400b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49400b1b40 0x7f49400b5240 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f493400c790 tx=0x7f4934007e10 comp rx=0 tx=0).stop 2023-10-08T01:14:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 shutdown_connections 2023-10-08T01:14:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f492c067170 0x7f492c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f49400b16d0 0x7f4940148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f49400b4e70 0x7f49400b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 --2- 172.21.15.119:0/3674404456 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49400b1b40 0x7f49400b5240 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 >> 172.21.15.119:0/3674404456 conn(0x7f49400d10c0 msgr2=0x7f49400b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:06.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 shutdown_connections 2023-10-08T01:14:06.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:06.880+0000 7f494559d700 1 -- 172.21.15.119:0/3674404456 wait complete. 2023-10-08T01:14:06.900 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:14:07.079 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:07.078+0000 7fa27bef4180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:14:07.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.088+0000 7f48ef59e700 1 Processor -- start 2023-10-08T01:14:07.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.088+0000 7f48ef59e700 1 -- start start 2023-10-08T01:14:07.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48ef59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f0159420 0x7f48f00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48ef59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f00b2980 con 0x7f48f01597f0 2023-10-08T01:14:07.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48ef59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f00b11d0 con 0x7f48f015d230 2023-10-08T01:14:07.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48ef59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00b14b0 con 0x7f48f0159420 2023-10-08T01:14:07.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48edd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f48f01597f0 0x7f48f00b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47036/0 (socket says 172.21.15.119:47036) 2023-10-08T01:14:07.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48edd9b700 1 -- 172.21.15.119:0/3900243513 learned_addr learned my addr 172.21.15.119:0/3900243513 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:07.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48ee59c700 1 --2- 172.21.15.119:0/3900243513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f0159420 0x7f48f00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:07.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48ed59a700 1 -- 172.21.15.119:0/3900243513 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454315946 0 0) 0x7f48f00b2980 con 0x7f48f01597f0 2023-10-08T01:14:07.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48ed59a700 1 -- 172.21.15.119:0/3900243513 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48d8003570 con 0x7f48f01597f0 2023-10-08T01:14:07.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.090+0000 7f48ed59a700 1 -- 172.21.15.119:0/3900243513 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747862749 0 0) 0x7f48f00b11d0 con 0x7f48f015d230 2023-10-08T01:14:07.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.091+0000 7f48ed59a700 1 -- 172.21.15.119:0/3900243513 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f00b2980 con 0x7f48f015d230 2023-10-08T01:14:07.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.091+0000 7f48ee59c700 1 -- 172.21.15.119:0/3900243513 >> v1:172.21.15.184:6789/0 conn(0x7f48f015d230 legacy=0x7f48f00b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.091+0000 7f48ee59c700 1 -- 172.21.15.119:0/3900243513 >> v1:172.21.15.119:6789/0 conn(0x7f48f01597f0 legacy=0x7f48f00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.091+0000 7f48ee59c700 1 -- 172.21.15.119:0/3900243513 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f00b1790 con 0x7f48f0159420 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.091+0000 7f48ee59c700 1 --2- 172.21.15.119:0/3900243513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f0159420 0x7f48f00b5f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009910 tx=0x7f48e00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=476bc9444d274983 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.091+0000 7f48ed59a700 1 -- 172.21.15.119:0/3900243513 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0011630 con 0x7f48f0159420 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.091+0000 7f48ed59a700 1 -- 172.21.15.119:0/3900243513 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e0011790 con 0x7f48f0159420 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.091+0000 7f48ed59a700 1 -- 172.21.15.119:0/3900243513 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0011b30 con 0x7f48f0159420 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.092+0000 7f48ef59e700 1 -- 172.21.15.119:0/3900243513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f0159420 msgr2=0x7f48f00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.092+0000 7f48ef59e700 1 --2- 172.21.15.119:0/3900243513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f0159420 0x7f48f00b5f40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009910 tx=0x7f48e00091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.092+0000 7f48ef59e700 1 -- 172.21.15.119:0/3900243513 shutdown_connections 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.092+0000 7f48ef59e700 1 --2- 172.21.15.119:0/3900243513 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f0159420 0x7f48f00b5f40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.092+0000 7f48ef59e700 1 -- 172.21.15.119:0/3900243513 >> 172.21.15.119:0/3900243513 conn(0x7f48f00d0660 msgr2=0x7f48f00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:07.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.092+0000 7f48ef59e700 1 -- 172.21.15.119:0/3900243513 shutdown_connections 2023-10-08T01:14:07.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.092+0000 7f48ef59e700 1 -- 172.21.15.119:0/3900243513 wait complete. 2023-10-08T01:14:07.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.093+0000 7f48ef59e700 1 Processor -- start 2023-10-08T01:14:07.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.094+0000 7f48ef59e700 1 -- start start 2023-10-08T01:14:07.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.094+0000 7f48ef59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f0159420 0x7f48f00cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.094+0000 7f48ef59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01597f0 0x7f48f00c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.094+0000 7f48ef59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f015d230 0x7f48f00c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.094+0000 7f48ef59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00cb0a0 con 0x7f48f01597f0 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.094+0000 7f48ef59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00ca1e0 con 0x7f48f0159420 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.094+0000 7f48ef59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00ca490 con 0x7f48f015d230 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48edd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01597f0 0x7f48f00c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48edd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01597f0 0x7f48f00c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34692/0 (socket says 172.21.15.119:34692) 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48edd9b700 1 -- 172.21.15.119:0/1390686225 learned_addr learned my addr 172.21.15.119:0/1390686225 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48ee59c700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f0159420 0x7f48f00cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48eed9d700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f015d230 0x7f48f00c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48ee59c700 1 -- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f015d230 msgr2=0x7f48f00c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48ee59c700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f015d230 0x7f48f00c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48ee59c700 1 -- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01597f0 msgr2=0x7f48f00c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48ee59c700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01597f0 0x7f48f00c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48ee59c700 1 -- 172.21.15.119:0/1390686225 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0009420 con 0x7f48f0159420 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.095+0000 7f48edd9b700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01597f0 0x7f48f00c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.096+0000 7f48ee59c700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f0159420 0x7f48f00cb530 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f48e0016000 tx=0x7f48e0017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.096+0000 7f48de7fc700 1 -- 172.21.15.119:0/1390686225 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e001f070 con 0x7f48f0159420 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.096+0000 7f48de7fc700 1 -- 172.21.15.119:0/1390686225 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e0002f70 con 0x7f48f0159420 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.096+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f00ca770 con 0x7f48f0159420 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.096+0000 7f48de7fc700 1 -- 172.21.15.119:0/1390686225 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e0011900 con 0x7f48f0159420 2023-10-08T01:14:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.096+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f00cad30 con 0x7f48f0159420 2023-10-08T01:14:07.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.097+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f00cb380 con 0x7f48f0159420 2023-10-08T01:14:07.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.098+0000 7f48de7fc700 1 -- 172.21.15.119:0/1390686225 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f48e0023070 con 0x7f48f0159420 2023-10-08T01:14:07.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.099+0000 7f48de7fc700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f48d0067170 0x7f48d0069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.099+0000 7f48de7fc700 1 -- 172.21.15.119:0/1390686225 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f48e0085c90 con 0x7f48f0159420 2023-10-08T01:14:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.099+0000 7f48edd9b700 1 -- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f48d0067170 msgr2=0x7f48d0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:07.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.099+0000 7f48edd9b700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f48d0067170 0x7f48d0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.103+0000 7f48de7fc700 1 -- 172.21.15.119:0/1390686225 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48e0052950 con 0x7f48f0159420 2023-10-08T01:14:07.173 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:07.172+0000 7fa27bef4180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:14:07.251 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:07.250+0000 7f75e330a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:14:07.260 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:07.260+0000 7fa27bef4180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:14:07.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.300+0000 7f48edd9b700 1 -- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f48d0067170 msgr2=0x7f48d0069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:07.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.300+0000 7f48edd9b700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f48d0067170 0x7f48d0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:07.348 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:07.347+0000 7f75e330a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:14:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.352+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48f004c3c0 con 0x7f48f0159420 2023-10-08T01:14:07.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.352+0000 7f48de7fc700 1 -- 172.21.15.119:0/1390686225 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f48e0055f70 con 0x7f48f0159420 2023-10-08T01:14:07.353 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:07.353 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1682556092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1682556092},{"type":"v1","addr":"172.21.15.184:6835","nonce":1682556092}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/691036731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":691036731},{"type":"v1","addr":"172.21.15.184:6834","nonce":691036731}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7182,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3636556734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3636556734},{"type":"v1","addr":"172.21.15.119:6834","nonce":3636556734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:44.715284+0000","modified":"2023-10-08T01:12:48.271445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7185":{"gid":7185,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3302512146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3302512146},{"type":"v1","addr":"172.21.15.119:6835","nonce":3302512146}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T01:14:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f48d0067170 msgr2=0x7f48d0069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f48d0067170 0x7f48d0069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f0159420 msgr2=0x7f48f00cb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f0159420 0x7f48f00cb530 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f48e0016000 tx=0x7f48e0017000 comp rx=0 tx=0).stop 2023-10-08T01:14:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 shutdown_connections 2023-10-08T01:14:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f48d0067170 0x7f48d0069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f48f015d230 0x7f48f00c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48f0159420 0x7f48f00cb530 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 --2- 172.21.15.119:0/1390686225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01597f0 0x7f48f00c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 >> 172.21.15.119:0/1390686225 conn(0x7f48f00d0660 msgr2=0x7f48f00b3e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 shutdown_connections 2023-10-08T01:14:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.355+0000 7f48ef59e700 1 -- 172.21.15.119:0/1390686225 wait complete. 2023-10-08T01:14:07.357 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 48 2023-10-08T01:14:07.374 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:14:07.448 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:07.447+0000 7f75e330a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:14:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.563+0000 7f8607fff700 1 Processor -- start 2023-10-08T01:14:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.563+0000 7f8607fff700 1 -- start start 2023-10-08T01:14:07.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.564+0000 7f8607fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86081596b0 0x7f86080aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.564+0000 7f8607fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86080c81c0 con 0x7f8608063230 2023-10-08T01:14:07.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.564+0000 7f8607fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860815ab20 con 0x7f86080c1760 2023-10-08T01:14:07.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.565+0000 7f8607fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86080c22f0 con 0x7f86081596b0 2023-10-08T01:14:07.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.565+0000 7f8606ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86081596b0 0x7f86080aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:07.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.565+0000 7f86067fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8608063230 0x7f86080c8650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47050/0 (socket says 172.21.15.119:47050) 2023-10-08T01:14:07.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.565+0000 7f86067fc700 1 -- 172.21.15.119:0/1484682389 learned_addr learned my addr 172.21.15.119:0/1484682389 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:07.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.565+0000 7f8606ffd700 1 -- 172.21.15.119:0/1484682389 >> v1:172.21.15.184:6789/0 conn(0x7f86080c1760 legacy=0x7f86080c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.565+0000 7f8606ffd700 1 -- 172.21.15.119:0/1484682389 >> v1:172.21.15.119:6789/0 conn(0x7f8608063230 legacy=0x7f86080c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.565+0000 7f8606ffd700 1 -- 172.21.15.119:0/1484682389 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86080c7b90 con 0x7f86081596b0 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.565+0000 7f8606ffd700 1 --2- 172.21.15.119:0/1484682389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86081596b0 0x7f86080aef90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f85f80098e0 tx=0x7f85f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca9b1141eab0c7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8605ffb700 1 -- 172.21.15.119:0/1484682389 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8009e80 con 0x7f86081596b0 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8605ffb700 1 -- 172.21.15.119:0/1484682389 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f800b860 con 0x7f86081596b0 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8605ffb700 1 -- 172.21.15.119:0/1484682389 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f800bbe0 con 0x7f86081596b0 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8607fff700 1 -- 172.21.15.119:0/1484682389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86081596b0 msgr2=0x7f86080aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8607fff700 1 --2- 172.21.15.119:0/1484682389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86081596b0 0x7f86080aef90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f85f80098e0 tx=0x7f85f80091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8607fff700 1 -- 172.21.15.119:0/1484682389 shutdown_connections 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8607fff700 1 --2- 172.21.15.119:0/1484682389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86081596b0 0x7f86080aef90 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8607fff700 1 -- 172.21.15.119:0/1484682389 >> 172.21.15.119:0/1484682389 conn(0x7f86080cd490 msgr2=0x7f86080c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.566+0000 7f8607fff700 1 -- 172.21.15.119:0/1484682389 shutdown_connections 2023-10-08T01:14:07.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.567+0000 7f8607fff700 1 -- 172.21.15.119:0/1484682389 wait complete. 2023-10-08T01:14:07.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.568+0000 7f8607fff700 1 Processor -- start 2023-10-08T01:14:07.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.568+0000 7f8607fff700 1 -- start start 2023-10-08T01:14:07.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f8607fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8608063230 0x7f8608153650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f8607fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86080c1760 0x7f86081557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f8607fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86081596b0 0x7f8608159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f8607fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8608153b90 con 0x7f86080c1760 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f8607fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860815bea0 con 0x7f8608063230 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f8607fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860815c180 con 0x7f86081596b0 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f86067fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86080c1760 0x7f86081557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f86067fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86080c1760 0x7f86081557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34712/0 (socket says 172.21.15.119:34712) 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f86067fc700 1 -- 172.21.15.119:0/2346467161 learned_addr learned my addr 172.21.15.119:0/2346467161 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f8606ffd700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8608063230 0x7f8608153650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.569+0000 7f86077fe700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86081596b0 0x7f8608159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f86067fc700 1 -- 172.21.15.119:0/2346467161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86081596b0 msgr2=0x7f8608159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f86067fc700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86081596b0 0x7f8608159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f86067fc700 1 -- 172.21.15.119:0/2346467161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8608063230 msgr2=0x7f8608153650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f86067fc700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8608063230 0x7f8608153650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f86067fc700 1 -- 172.21.15.119:0/2346467161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f8009420 con 0x7f86080c1760 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f86067fc700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86080c1760 0x7f86081557d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00c790 tx=0x7f85fc007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f85eeffd700 1 -- 172.21.15.119:0/2346467161 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc005b30 con 0x7f86080c1760 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f85eeffd700 1 -- 172.21.15.119:0/2346467161 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc005c90 con 0x7f86080c1760 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f85eeffd700 1 -- 172.21.15.119:0/2346467161 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc004a80 con 0x7f86080c1760 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860815c460 con 0x7f86080c1760 2023-10-08T01:14:07.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.570+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86081549c0 con 0x7f86080c1760 2023-10-08T01:14:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.571+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860815ca90 con 0x7f86080c1760 2023-10-08T01:14:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.572+0000 7f85eeffd700 1 -- 172.21.15.119:0/2346467161 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f85fc0070d0 con 0x7f86080c1760 2023-10-08T01:14:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.573+0000 7f85eeffd700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f4067170 0x7f85f4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.573+0000 7f85eeffd700 1 -- 172.21.15.119:0/2346467161 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f85fc083f50 con 0x7f86080c1760 2023-10-08T01:14:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.573+0000 7f8606ffd700 1 -- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f4067170 msgr2=0x7f85f4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:07.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.573+0000 7f8606ffd700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f4067170 0x7f85f4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:07.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.577+0000 7f85eeffd700 1 -- 172.21.15.119:0/2346467161 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85fc04b5a0 con 0x7f86080c1760 2023-10-08T01:14:07.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.774+0000 7f8606ffd700 1 -- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f4067170 msgr2=0x7f85f4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:07.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.774+0000 7f8606ffd700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f4067170 0x7f85f4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.778+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f860804c3c0 con 0x7f86080c1760 2023-10-08T01:14:07.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.779+0000 7f85eeffd700 1 -- 172.21.15.119:0/2346467161 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16430 (secure 0 0 0) 0x7f85fc050c10 con 0x7f86080c1760 2023-10-08T01:14:07.782 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:07.783 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":107,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:02.302915+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:42.395485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:43.700121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.c","mappings":[{"from":1,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1425177173":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/3497965416":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/3790087460":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/898896673":"2023-10-09T01:13:00.409133+0000","172.21.15.119:6837/85511":"2023-10-09T01:13:00.409133+0000","172.21.15.119:6836/85511":"2023-10-09T01:13:00.409133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:14:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f4067170 msgr2=0x7f85f4069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f4067170 0x7f85f4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86080c1760 msgr2=0x7f86081557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86080c1760 0x7f86081557d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f85fc00c790 tx=0x7f85fc007f40 comp rx=0 tx=0).stop 2023-10-08T01:14:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 shutdown_connections 2023-10-08T01:14:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f4067170 0x7f85f4069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86081596b0 0x7f8608159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8608063230 0x7f8608153650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 --2- 172.21.15.119:0/2346467161 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86080c1760 0x7f86081557d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 >> 172.21.15.119:0/2346467161 conn(0x7f86080cd490 msgr2=0x7f86080aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 shutdown_connections 2023-10-08T01:14:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:07.782+0000 7f8607fff700 1 -- 172.21.15.119:0/2346467161 wait complete. 2023-10-08T01:14:07.801 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T01:14:07.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:14:07.881 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:07.881+0000 7fa27bef4180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:14:08.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.004+0000 7f0d9339d700 1 Processor -- start 2023-10-08T01:14:08.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.004+0000 7f0d9339d700 1 -- start start 2023-10-08T01:14:08.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.006+0000 7f0d9339d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d8c166530 0x7f0d8c0c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.006+0000 7f0d9339d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0d8c0caad0 0x7f0d8c0872f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.006+0000 7f0d9339d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8c069cd0 con 0x7f0d8c087830 2023-10-08T01:14:08.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.006+0000 7f0d9339d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8c166900 con 0x7f0d8c166530 2023-10-08T01:14:08.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.006+0000 7f0d9339d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8c0696c0 con 0x7f0d8c0caad0 2023-10-08T01:14:08.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.006+0000 7f0d9239b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d8c166530 0x7f0d8c0c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.006+0000 7f0d92b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0d8c087830 0x7f0d8c06a160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47058/0 (socket says 172.21.15.119:47058) 2023-10-08T01:14:08.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.006+0000 7f0d9239b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d8c166530 0x7f0d8c0c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34716/0 (socket says 172.21.15.119:34716) 2023-10-08T01:14:08.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.007+0000 7f0d92b9c700 1 -- 172.21.15.119:0/3565577602 learned_addr learned my addr 172.21.15.119:0/3565577602 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.007+0000 7f0d9239b700 1 -- 172.21.15.119:0/3565577602 >> v2:172.21.15.184:3300/0 conn(0x7f0d8c0caad0 msgr2=0x7f0d8c0872f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.007+0000 7f0d9239b700 1 --2- 172.21.15.119:0/3565577602 >> v2:172.21.15.184:3300/0 conn(0x7f0d8c0caad0 0x7f0d8c0872f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.007+0000 7f0d9239b700 1 -- 172.21.15.119:0/3565577602 >> v1:172.21.15.119:6789/0 conn(0x7f0d8c087830 legacy=0x7f0d8c06a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.007+0000 7f0d9239b700 1 -- 172.21.15.119:0/3565577602 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c087c30 con 0x7f0d8c166530 2023-10-08T01:14:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.007+0000 7f0d9239b700 1 --2- 172.21.15.119:0/3565577602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d8c166530 0x7f0d8c0c7260 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0aed10 tx=0x7f0d840090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729654a88cc86ffd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.008+0000 7f0d91399700 1 -- 172.21.15.119:0/3565577602 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d840100d0 con 0x7f0d8c166530 2023-10-08T01:14:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.008+0000 7f0d91399700 1 -- 172.21.15.119:0/3565577602 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d84010230 con 0x7f0d8c166530 2023-10-08T01:14:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.009+0000 7f0d9339d700 1 -- 172.21.15.119:0/3565577602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d8c166530 msgr2=0x7f0d8c0c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.009+0000 7f0d9339d700 1 --2- 172.21.15.119:0/3565577602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d8c166530 0x7f0d8c0c7260 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c0aed10 tx=0x7f0d840090f0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.009+0000 7f0d9339d700 1 -- 172.21.15.119:0/3565577602 shutdown_connections 2023-10-08T01:14:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.009+0000 7f0d9339d700 1 --2- 172.21.15.119:0/3565577602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d8c166530 0x7f0d8c0c7260 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.009+0000 7f0d9339d700 1 --2- 172.21.15.119:0/3565577602 >> v2:172.21.15.184:3300/0 conn(0x7f0d8c0caad0 0x7f0d8c0872f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.009+0000 7f0d9339d700 1 -- 172.21.15.119:0/3565577602 >> 172.21.15.119:0/3565577602 conn(0x7f0d8c0cc1f0 msgr2=0x7f0d8c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.010+0000 7f0d9339d700 1 -- 172.21.15.119:0/3565577602 shutdown_connections 2023-10-08T01:14:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.010+0000 7f0d9339d700 1 -- 172.21.15.119:0/3565577602 wait complete. 2023-10-08T01:14:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.012+0000 7f0d9339d700 1 Processor -- start 2023-10-08T01:14:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.012+0000 7f0d9339d700 1 -- start start 2023-10-08T01:14:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d9339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c087830 0x7f0d8c163990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d9339d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d8c0caad0 0x7f0d8c0c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d9339d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c161aa0 0x7f0d8c165130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d9339d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c0c5c00 con 0x7f0d8c0caad0 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d9339d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c164ca0 con 0x7f0d8c161aa0 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d9339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c164090 con 0x7f0d8c087830 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d9239b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c087830 0x7f0d8c163990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d92b9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c161aa0 0x7f0d8c165130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d92b9c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c161aa0 0x7f0d8c165130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:51688/0 (socket says 172.21.15.119:51688) 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d9239b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c087830 0x7f0d8c163990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:50710/0 (socket says 172.21.15.119:50710) 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.014+0000 7f0d92b9c700 1 -- 172.21.15.119:0/1462261992 learned_addr learned my addr 172.21.15.119:0/1462261992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.015+0000 7f0d9239b700 1 -- 172.21.15.119:0/1462261992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c161aa0 msgr2=0x7f0d8c165130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.015+0000 7f0d9239b700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c161aa0 0x7f0d8c165130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.015+0000 7f0d9239b700 1 -- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d8c0caad0 msgr2=0x7f0d8c0c56c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:08.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.015+0000 7f0d9239b700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d8c0caad0 0x7f0d8c0c56c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.015+0000 7f0d9239b700 1 -- 172.21.15.119:0/1462261992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d840107d0 con 0x7f0d8c087830 2023-10-08T01:14:08.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.015+0000 7f0d92b9c700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c161aa0 0x7f0d8c165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:08.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.015+0000 7f0d9239b700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c087830 0x7f0d8c163990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d8401e000 tx=0x7f0d8401f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:08.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.016+0000 7f0d8a7fc700 1 -- 172.21.15.119:0/1462261992 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84019630 con 0x7f0d8c087830 2023-10-08T01:14:08.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.016+0000 7f0d8a7fc700 1 -- 172.21.15.119:0/1462261992 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d84016040 con 0x7f0d8c087830 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.016+0000 7f0d8a7fc700 1 -- 172.21.15.119:0/1462261992 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8400e740 con 0x7f0d8c087830 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.016+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c164370 con 0x7f0d8c087830 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.016+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c164930 con 0x7f0d8c087830 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.017+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8c04c3c0 con 0x7f0d8c087830 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.019+0000 7f0d8a7fc700 1 -- 172.21.15.119:0/1462261992 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f0d8400eb70 con 0x7f0d8c087830 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.020+0000 7f0d8a7fc700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0d780671f0 0x7f0d780696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.020+0000 7f0d8a7fc700 1 -- 172.21.15.119:0/1462261992 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f0d84086610 con 0x7f0d8c087830 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.020+0000 7f0d91b9a700 1 -- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0d780671f0 msgr2=0x7f0d780696a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:08.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.021+0000 7f0d91b9a700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0d780671f0 0x7f0d780696a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:08.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.024+0000 7f0d8a7fc700 1 -- 172.21.15.119:0/1462261992 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0d840531c0 con 0x7f0d8c087830 2023-10-08T01:14:08.053 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:08.053+0000 7fa27bef4180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:14:08.098 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:08.097+0000 7f75e330a180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:14:08.147 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:08.146+0000 7fa27bef4180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:14:08.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.222+0000 7f0d91b9a700 1 -- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0d780671f0 msgr2=0x7f0d780696a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:08.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.222+0000 7f0d91b9a700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0d780671f0 0x7f0d780696a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:08.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.271+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d8c0ccb80 con 0x7f0d8c087830 2023-10-08T01:14:08.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.272+0000 7f0d8a7fc700 1 -- 172.21.15.119:0/1462261992 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f0d840567e0 con 0x7f0d8c087830 2023-10-08T01:14:08.274 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:08.274 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1682556092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1682556092},{"type":"v1","addr":"172.21.15.184:6835","nonce":1682556092}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/691036731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":691036731},{"type":"v1","addr":"172.21.15.184:6834","nonce":691036731}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7182,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3636556734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3636556734},{"type":"v1","addr":"172.21.15.119:6834","nonce":3636556734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:44.715284+0000","modified":"2023-10-08T01:12:48.271445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7185":{"gid":7185,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3302512146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3302512146},{"type":"v1","addr":"172.21.15.119:6835","nonce":3302512146}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T01:14:08.275 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:08.274+0000 7f75e330a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:14:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0d780671f0 msgr2=0x7f0d780696a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0d780671f0 0x7f0d780696a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c087830 msgr2=0x7f0d8c163990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c087830 0x7f0d8c163990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0d8401e000 tx=0x7f0d8401f000 comp rx=0 tx=0).stop 2023-10-08T01:14:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 shutdown_connections 2023-10-08T01:14:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0d780671f0 0x7f0d780696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0d8c087830 0x7f0d8c163990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0d8c161aa0 0x7f0d8c165130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 --2- 172.21.15.119:0/1462261992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d8c0caad0 0x7f0d8c0c56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 >> 172.21.15.119:0/1462261992 conn(0x7f0d8c0cc1f0 msgr2=0x7f0d8c0ca2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 shutdown_connections 2023-10-08T01:14:08.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.275+0000 7f0d9339d700 1 -- 172.21.15.119:0/1462261992 wait complete. 2023-10-08T01:14:08.278 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 48 2023-10-08T01:14:08.295 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:14:08.377 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:08.376+0000 7f75e330a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:14:08.440 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:08.440+0000 7fa27bef4180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:14:08.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.484+0000 7f94aa5ff700 1 Processor -- start 2023-10-08T01:14:08.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.484+0000 7f94aa5ff700 1 -- start start 2023-10-08T01:14:08.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.485+0000 7f94aa5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f94a4159c60 0x7f94a415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.485+0000 7f94aa5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a415a160 con 0x7f94a4159890 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.485+0000 7f94aa5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a40d13a0 con 0x7f94a415db10 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.485+0000 7f94aa5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a415df50 con 0x7f94a4159c60 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.485+0000 7f94a3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f94a4159890 0x7f94a415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47066/0 (socket says 172.21.15.119:47066) 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.485+0000 7f94a3fff700 1 -- 172.21.15.119:0/3945996966 learned_addr learned my addr 172.21.15.119:0/3945996966 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.485+0000 7f94a37fe700 1 --2- 172.21.15.119:0/3945996966 >> v2:172.21.15.184:3300/0 conn(0x7f94a4159c60 0x7f94a415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a2ffd700 1 -- 172.21.15.119:0/3945996966 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296691687 0 0) 0x7f94a40d13a0 con 0x7f94a415db10 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a2ffd700 1 -- 172.21.15.119:0/3945996966 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f948c003570 con 0x7f94a415db10 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a2ffd700 1 -- 172.21.15.119:0/3945996966 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4131814430 0 0) 0x7f94a415a160 con 0x7f94a4159890 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a2ffd700 1 -- 172.21.15.119:0/3945996966 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94a40d13a0 con 0x7f94a4159890 2023-10-08T01:14:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a37fe700 1 -- 172.21.15.119:0/3945996966 >> v1:172.21.15.184:6789/0 conn(0x7f94a415db10 legacy=0x7f94a4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a37fe700 1 -- 172.21.15.119:0/3945996966 >> v1:172.21.15.119:6789/0 conn(0x7f94a4159890 legacy=0x7f94a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a37fe700 1 -- 172.21.15.119:0/3945996966 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a415e140 con 0x7f94a4159c60 2023-10-08T01:14:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a37fe700 1 --2- 172.21.15.119:0/3945996966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a4159c60 0x7f94a415d5d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f94980098e0 tx=0x7f94980091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d09ba8a5f44f1874 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.486+0000 7f94a2ffd700 1 -- 172.21.15.119:0/3945996966 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498011630 con 0x7f94a4159c60 2023-10-08T01:14:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.487+0000 7f94a2ffd700 1 -- 172.21.15.119:0/3945996966 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498011790 con 0x7f94a4159c60 2023-10-08T01:14:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.487+0000 7f94a2ffd700 1 -- 172.21.15.119:0/3945996966 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498011b30 con 0x7f94a4159c60 2023-10-08T01:14:08.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.487+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3945996966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a4159c60 msgr2=0x7f94a415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.487+0000 7f94aa5ff700 1 --2- 172.21.15.119:0/3945996966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a4159c60 0x7f94a415d5d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f94980098e0 tx=0x7f94980091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.487+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3945996966 shutdown_connections 2023-10-08T01:14:08.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.487+0000 7f94aa5ff700 1 --2- 172.21.15.119:0/3945996966 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a4159c60 0x7f94a415d5d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.487+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3945996966 >> 172.21.15.119:0/3945996966 conn(0x7f94a4068e20 msgr2=0x7f94a40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:08.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.487+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3945996966 shutdown_connections 2023-10-08T01:14:08.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.488+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3945996966 wait complete. 2023-10-08T01:14:08.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.489+0000 7f94aa5ff700 1 Processor -- start 2023-10-08T01:14:08.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.489+0000 7f94aa5ff700 1 -- start start 2023-10-08T01:14:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94aa5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94a4159890 0x7f94a416d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94aa5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4159c60 0x7f94a40b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94aa5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a415db10 0x7f94a416a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94aa5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40cf210 con 0x7f94a4159c60 2023-10-08T01:14:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94aa5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40cec50 con 0x7f94a415db10 2023-10-08T01:14:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94aa5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a40cef30 con 0x7f94a4159890 2023-10-08T01:14:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4159c60 0x7f94a40b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94a37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4159c60 0x7f94a40b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34726/0 (socket says 172.21.15.119:34726) 2023-10-08T01:14:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94a37fe700 1 -- 172.21.15.119:0/3405514616 learned_addr learned my addr 172.21.15.119:0/3405514616 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.490+0000 7f94a3fff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94a4159890 0x7f94a416d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.491+0000 7f94a8b9c700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a415db10 0x7f94a416a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.491+0000 7f94a37fe700 1 -- 172.21.15.119:0/3405514616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94a4159890 msgr2=0x7f94a416d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.491+0000 7f94a37fe700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94a4159890 0x7f94a416d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.491+0000 7f94a37fe700 1 -- 172.21.15.119:0/3405514616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a415db10 msgr2=0x7f94a416a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.491+0000 7f94a37fe700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a415db10 0x7f94a416a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.491+0000 7f94a37fe700 1 -- 172.21.15.119:0/3405514616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498009420 con 0x7f94a4159c60 2023-10-08T01:14:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.491+0000 7f94a37fe700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4159c60 0x7f94a40b2350 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f94a4070e20 tx=0x7f9498017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.491+0000 7f948bfff700 1 -- 172.21.15.119:0/3405514616 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949801f070 con 0x7f94a4159c60 2023-10-08T01:14:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.492+0000 7f948bfff700 1 -- 172.21.15.119:0/3405514616 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498000bc0 con 0x7f94a4159c60 2023-10-08T01:14:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.492+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a4169b20 con 0x7f94a4159c60 2023-10-08T01:14:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.492+0000 7f948bfff700 1 -- 172.21.15.119:0/3405514616 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498025420 con 0x7f94a4159c60 2023-10-08T01:14:08.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.492+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a416a860 con 0x7f94a4159c60 2023-10-08T01:14:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.493+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a4169980 con 0x7f94a4159c60 2023-10-08T01:14:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.494+0000 7f948bfff700 1 -- 172.21.15.119:0/3405514616 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f9498023070 con 0x7f94a4159c60 2023-10-08T01:14:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.494+0000 7f948bfff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9490067170 0x7f9490069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.495+0000 7f948bfff700 1 -- 172.21.15.119:0/3405514616 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f9498085e10 con 0x7f94a4159c60 2023-10-08T01:14:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.495+0000 7f94a3fff700 1 -- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9490067170 msgr2=0x7f9490069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.495+0000 7f94a3fff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9490067170 0x7f9490069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.499+0000 7f948bfff700 1 -- 172.21.15.119:0/3405514616 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f94a4169980 con 0x7f94a4159c60 2023-10-08T01:14:08.536 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:08.536+0000 7fa27bef4180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:14:08.682 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:08.681+0000 7f75e330a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:14:08.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.696+0000 7f94a3fff700 1 -- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9490067170 msgr2=0x7f9490069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:08.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.696+0000 7f94a3fff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9490067170 0x7f9490069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:08.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.741+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94a4169980 con 0x7f94a4159c60 2023-10-08T01:14:08.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.741+0000 7f948bfff700 1 -- 172.21.15.119:0/3405514616 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f94a4169980 con 0x7f94a4159c60 2023-10-08T01:14:08.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:08.743 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":48,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1682556092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1682556092},{"type":"v1","addr":"172.21.15.184:6835","nonce":1682556092}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/691036731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":691036731},{"type":"v1","addr":"172.21.15.184:6834","nonce":691036731}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7182,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3636556734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3636556734},{"type":"v1","addr":"172.21.15.119:6834","nonce":3636556734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:12:44.715284+0000","modified":"2023-10-08T01:12:48.271445+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7185},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7185":{"gid":7185,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/3302512146","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":3302512146},{"type":"v1","addr":"172.21.15.119:6835","nonce":3302512146}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-10-08T01:14:08.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9490067170 msgr2=0x7f9490069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9490067170 0x7f9490069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4159c60 msgr2=0x7f94a40b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4159c60 0x7f94a40b2350 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f94a4070e20 tx=0x7f9498017000 comp rx=0 tx=0).stop 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 shutdown_connections 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9490067170 0x7f9490069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94a4159890 0x7f94a416d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94a415db10 0x7f94a416a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 --2- 172.21.15.119:0/3405514616 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94a4159c60 0x7f94a40b2350 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 >> 172.21.15.119:0/3405514616 conn(0x7f94a4068e20 msgr2=0x7f94a41534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:08.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 shutdown_connections 2023-10-08T01:14:08.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.744+0000 7f94aa5ff700 1 -- 172.21.15.119:0/3405514616 wait complete. 2023-10-08T01:14:08.747 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 48 2023-10-08T01:14:08.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:14:08.783 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:08.782+0000 7f75e330a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:14:08.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.948+0000 7f685739d700 1 Processor -- start 2023-10-08T01:14:08.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.948+0000 7f685739d700 1 -- start start 2023-10-08T01:14:08.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f685739d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6850159420 0x7f68500b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f685739d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68501597f0 0x7f68500b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f685739d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68500b11d0 con 0x7f685015d230 2023-10-08T01:14:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f685739d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500b14b0 con 0x7f68501597f0 2023-10-08T01:14:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f685739d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68500b1790 con 0x7f6850159420 2023-10-08T01:14:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f6855b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68501597f0 0x7f68500b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f6855b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68501597f0 0x7f68500b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34734/0 (socket says 172.21.15.119:34734) 2023-10-08T01:14:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f6856b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f685015d230 0x7f685015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47082/0 (socket says 172.21.15.119:47082) 2023-10-08T01:14:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f6855b9a700 1 -- 172.21.15.119:0/458876168 learned_addr learned my addr 172.21.15.119:0/458876168 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.949+0000 7f685639b700 1 --2- 172.21.15.119:0/458876168 >> v2:172.21.15.184:3300/0 conn(0x7f6850159420 0x7f68500b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.950+0000 7f6855b9a700 1 -- 172.21.15.119:0/458876168 >> v2:172.21.15.184:3300/0 conn(0x7f6850159420 msgr2=0x7f68500b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.950+0000 7f6855b9a700 1 --2- 172.21.15.119:0/458876168 >> v2:172.21.15.184:3300/0 conn(0x7f6850159420 0x7f68500b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.950+0000 7f6855b9a700 1 -- 172.21.15.119:0/458876168 >> v1:172.21.15.119:6789/0 conn(0x7f685015d230 legacy=0x7f685015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.950+0000 7f6855b9a700 1 -- 172.21.15.119:0/458876168 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500b1a70 con 0x7f68501597f0 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.950+0000 7f6855b9a700 1 --2- 172.21.15.119:0/458876168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f68500b2270 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f68500b2f20 tx=0x7f6840009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f70f63da4d0654 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.950+0000 7f6855399700 1 -- 172.21.15.119:0/458876168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840009e70 con 0x7f68501597f0 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.950+0000 7f6855399700 1 -- 172.21.15.119:0/458876168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684000b860 con 0x7f68501597f0 2023-10-08T01:14:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.950+0000 7f6855399700 1 -- 172.21.15.119:0/458876168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684000bc00 con 0x7f68501597f0 2023-10-08T01:14:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.951+0000 7f685739d700 1 -- 172.21.15.119:0/458876168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 msgr2=0x7f68500b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.951+0000 7f685739d700 1 --2- 172.21.15.119:0/458876168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f68500b2270 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f68500b2f20 tx=0x7f6840009250 comp rx=0 tx=0).stop 2023-10-08T01:14:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.951+0000 7f685739d700 1 -- 172.21.15.119:0/458876168 shutdown_connections 2023-10-08T01:14:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.951+0000 7f685739d700 1 --2- 172.21.15.119:0/458876168 >> v2:172.21.15.184:3300/0 conn(0x7f6850159420 0x7f68500b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.951+0000 7f685739d700 1 --2- 172.21.15.119:0/458876168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f68500b2270 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.951+0000 7f685739d700 1 -- 172.21.15.119:0/458876168 >> 172.21.15.119:0/458876168 conn(0x7f68500d0660 msgr2=0x7f68500d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.951+0000 7f685739d700 1 -- 172.21.15.119:0/458876168 shutdown_connections 2023-10-08T01:14:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.951+0000 7f685739d700 1 -- 172.21.15.119:0/458876168 wait complete. 2023-10-08T01:14:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.953+0000 7f685739d700 1 Processor -- start 2023-10-08T01:14:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.953+0000 7f685739d700 1 -- start start 2023-10-08T01:14:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f685739d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6850159420 0x7f685016a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f685739d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f6850153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f685739d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685015d230 0x7f685015c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f685739d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685016d1c0 con 0x7f68501597f0 2023-10-08T01:14:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f685739d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685016d480 con 0x7f6850159420 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f685739d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685016d760 con 0x7f685015d230 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6855b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f6850153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6855b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f6850153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34750/0 (socket says 172.21.15.119:34750) 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6855b9a700 1 -- 172.21.15.119:0/4276145860 learned_addr learned my addr 172.21.15.119:0/4276145860 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f685639b700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6850159420 0x7f685016a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6856b9c700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685015d230 0x7f685015c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6855b9a700 1 -- 172.21.15.119:0/4276145860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685015d230 msgr2=0x7f685015c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6855b9a700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685015d230 0x7f685015c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6855b9a700 1 -- 172.21.15.119:0/4276145860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6850159420 msgr2=0x7f685016a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6855b9a700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6850159420 0x7f685016a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.954+0000 7f6855b9a700 1 -- 172.21.15.119:0/4276145860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400094f0 con 0x7f68501597f0 2023-10-08T01:14:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.955+0000 7f6855b9a700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f6850153e00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6850089340 tx=0x7f6840017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.955+0000 7f68467fc700 1 -- 172.21.15.119:0/4276145860 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684000b9d0 con 0x7f68501597f0 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.955+0000 7f68467fc700 1 -- 172.21.15.119:0/4276145860 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684000bb30 con 0x7f68501597f0 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.955+0000 7f68467fc700 1 -- 172.21.15.119:0/4276145860 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840025670 con 0x7f68501597f0 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.955+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685016da40 con 0x7f68501597f0 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.955+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68500c6d40 con 0x7f68501597f0 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.956+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685004c3c0 con 0x7f68501597f0 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.957+0000 7f68467fc700 1 -- 172.21.15.119:0/4276145860 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f6840023070 con 0x7f68501597f0 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.958+0000 7f68467fc700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6834067170 0x7f6834069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.958+0000 7f68467fc700 1 -- 172.21.15.119:0/4276145860 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f6840086100 con 0x7f68501597f0 2023-10-08T01:14:08.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.958+0000 7f685639b700 1 -- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6834067170 msgr2=0x7f6834069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:08.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.958+0000 7f685639b700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6834067170 0x7f6834069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:08.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:08.962+0000 7f68467fc700 1 -- 172.21.15.119:0/4276145860 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6840052dc0 con 0x7f68501597f0 2023-10-08T01:14:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.159+0000 7f685639b700 1 -- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6834067170 msgr2=0x7f6834069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.159+0000 7f685639b700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6834067170 0x7f6834069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:09.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.161+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68500b1d50 con 0x7f68501597f0 2023-10-08T01:14:09.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.162+0000 7f68467fc700 1 -- 172.21.15.119:0/4276145860 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16430 (secure 0 0 0) 0x7f68400563e0 con 0x7f68501597f0 2023-10-08T01:14:09.164 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:09.164 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":107,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:02.302915+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:12:42.395485+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-10-08T01:12:43.700121+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.c","mappings":[{"from":1,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1425177173":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/3497965416":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/3790087460":"2023-10-09T01:13:00.409133+0000","172.21.15.119:0/898896673":"2023-10-09T01:13:00.409133+0000","172.21.15.119:6837/85511":"2023-10-09T01:13:00.409133+0000","172.21.15.119:6836/85511":"2023-10-09T01:13:00.409133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:14:09.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6834067170 msgr2=0x7f6834069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:09.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6834067170 0x7f6834069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 msgr2=0x7f6850153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f6850153e00 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6850089340 tx=0x7f6840017000 comp rx=0 tx=0).stop 2023-10-08T01:14:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 shutdown_connections 2023-10-08T01:14:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6834067170 0x7f6834069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f685015d230 0x7f685015c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6850159420 0x7f685016a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 --2- 172.21.15.119:0/4276145860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68501597f0 0x7f6850153e00 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 >> 172.21.15.119:0/4276145860 conn(0x7f68500d0660 msgr2=0x7f68500b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 shutdown_connections 2023-10-08T01:14:09.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.165+0000 7f685739d700 1 -- 172.21.15.119:0/4276145860 wait complete. 2023-10-08T01:14:09.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T01:14:09.232 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:09.232+0000 7fa27bef4180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:14:09.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.413+0000 7f349dd9d700 1 Processor -- start 2023-10-08T01:14:09.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.413+0000 7f349dd9d700 1 -- start start 2023-10-08T01:14:09.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.414+0000 7f349dd9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34980c6c60 0x7f34980c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.414+0000 7f349dd9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f349806e8c0 0x7f34980c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.414+0000 7f349dd9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34980b1410 con 0x7f34981595a0 2023-10-08T01:14:09.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.414+0000 7f349dd9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34980b2b90 con 0x7f34980c6c60 2023-10-08T01:14:09.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.414+0000 7f349dd9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3498063230 con 0x7f349806e8c0 2023-10-08T01:14:09.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f3497fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34980c6c60 0x7f34980c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f3497fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34980c6c60 0x7f34980c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34760/0 (socket says 172.21.15.119:34760) 2023-10-08T01:14:09.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f3497fff700 1 -- 172.21.15.119:0/1239785012 learned_addr learned my addr 172.21.15.119:0/1239785012 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:09.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f349d59c700 1 --2- 172.21.15.119:0/1239785012 >> v2:172.21.15.184:3300/0 conn(0x7f349806e8c0 0x7f34980c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f3497fff700 1 -- 172.21.15.119:0/1239785012 >> v2:172.21.15.184:3300/0 conn(0x7f349806e8c0 msgr2=0x7f34980c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f3497fff700 1 --2- 172.21.15.119:0/1239785012 >> v2:172.21.15.184:3300/0 conn(0x7f349806e8c0 0x7f34980c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f3497fff700 1 -- 172.21.15.119:0/1239785012 >> v1:172.21.15.184:6789/0 conn(0x7f34981595a0 legacy=0x7f34980b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f3497fff700 1 -- 172.21.15.119:0/1239785012 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34980634f0 con 0x7f34980c6c60 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f34977fe700 1 -- 172.21.15.119:0/1239785012 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754060692 0 0) 0x7f34980b1410 con 0x7f34981595a0 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.415+0000 7f3497fff700 1 --2- 172.21.15.119:0/1239785012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34980c6c60 0x7f34980c8120 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f349815d390 tx=0x7f3484009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ca6d39f06ba76c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f34977fe700 1 -- 172.21.15.119:0/1239785012 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484009e70 con 0x7f34980c6c60 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f34977fe700 1 -- 172.21.15.119:0/1239785012 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348400b860 con 0x7f34980c6c60 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f34977fe700 1 -- 172.21.15.119:0/1239785012 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348400bc00 con 0x7f34980c6c60 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f349dd9d700 1 -- 172.21.15.119:0/1239785012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34980c6c60 msgr2=0x7f34980c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f349dd9d700 1 --2- 172.21.15.119:0/1239785012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34980c6c60 0x7f34980c8120 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f349815d390 tx=0x7f3484009250 comp rx=0 tx=0).stop 2023-10-08T01:14:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f349dd9d700 1 -- 172.21.15.119:0/1239785012 shutdown_connections 2023-10-08T01:14:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f349dd9d700 1 --2- 172.21.15.119:0/1239785012 >> v2:172.21.15.184:3300/0 conn(0x7f349806e8c0 0x7f34980c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f349dd9d700 1 --2- 172.21.15.119:0/1239785012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34980c6c60 0x7f34980c8120 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f349dd9d700 1 -- 172.21.15.119:0/1239785012 >> 172.21.15.119:0/1239785012 conn(0x7f349806ddb0 msgr2=0x7f349806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f349dd9d700 1 -- 172.21.15.119:0/1239785012 shutdown_connections 2023-10-08T01:14:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.416+0000 7f349dd9d700 1 -- 172.21.15.119:0/1239785012 wait complete. 2023-10-08T01:14:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.418+0000 7f349dd9d700 1 Processor -- start 2023-10-08T01:14:09.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.418+0000 7f349dd9d700 1 -- start start 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349dd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f349806e8c0 0x7f3498169de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349dd9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34980c6c60 0x7f349813e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349dd9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34981595a0 0x7f349814eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349dd9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349815a740 con 0x7f34981595a0 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349dd9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349816a3f0 con 0x7f34980c6c60 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349dd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349816a6d0 con 0x7f349806e8c0 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349d59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34981595a0 0x7f349814eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349d59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34981595a0 0x7f349814eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34766/0 (socket says 172.21.15.119:34766) 2023-10-08T01:14:09.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349d59c700 1 -- 172.21.15.119:0/2500566216 learned_addr learned my addr 172.21.15.119:0/2500566216 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f3497fff700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34980c6c60 0x7f349813e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349cd9b700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f349806e8c0 0x7f3498169de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349d59c700 1 -- 172.21.15.119:0/2500566216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f349806e8c0 msgr2=0x7f3498169de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349d59c700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f349806e8c0 0x7f3498169de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349d59c700 1 -- 172.21.15.119:0/2500566216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34980c6c60 msgr2=0x7f349813e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349d59c700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34980c6c60 0x7f349813e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.419+0000 7f349d59c700 1 -- 172.21.15.119:0/2500566216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840094f0 con 0x7f34981595a0 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.420+0000 7f349d59c700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34981595a0 0x7f349814eec0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f3490009e90 tx=0x7f349000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:09.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.420+0000 7f3494ff9700 1 -- 172.21.15.119:0/2500566216 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000cb70 con 0x7f34981595a0 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.420+0000 7f3494ff9700 1 -- 172.21.15.119:0/2500566216 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000f040 con 0x7f34981595a0 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.420+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349816a9b0 con 0x7f34981595a0 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.420+0000 7f3494ff9700 1 -- 172.21.15.119:0/2500566216 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490014480 con 0x7f34981595a0 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.420+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349813e9e0 con 0x7f34981595a0 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.421+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349815a960 con 0x7f34981595a0 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.422+0000 7f3494ff9700 1 -- 172.21.15.119:0/2500566216 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f3490014830 con 0x7f34981595a0 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.423+0000 7f3494ff9700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f347c067170 0x7f347c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.423+0000 7f3494ff9700 1 -- 172.21.15.119:0/2500566216 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6384+0+0 (secure 0 0 0) 0x7f3490085690 con 0x7f34981595a0 2023-10-08T01:14:09.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.423+0000 7f349cd9b700 1 -- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f347c067170 msgr2=0x7f347c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:09.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.423+0000 7f349cd9b700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f347c067170 0x7f347c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:09.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.427+0000 7f3494ff9700 1 -- 172.21.15.119:0/2500566216 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f349815a960 con 0x7f34981595a0 2023-10-08T01:14:09.519 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:09.519+0000 7fa27bef4180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:14:09.528 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:09.527+0000 7f75e330a180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:14:09.610 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:09.610+0000 7fa27bef4180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:14:09.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.624+0000 7f349cd9b700 1 -- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f347c067170 msgr2=0x7f347c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:09.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.624+0000 7f349cd9b700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f347c067170 0x7f347c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.667+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f349815a960 con 0x7f34981595a0 2023-10-08T01:14:09.668 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T01:14:09.667+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T01:14:09.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.675+0000 7f3494ff9700 1 -- 172.21.15.119:0/2500566216 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7f349815a960 con 0x7f34981595a0 2023-10-08T01:14:09.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.678+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f347c067170 msgr2=0x7f347c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:09.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.678+0000 7f349dd9d700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f347c067170 0x7f347c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.678+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34981595a0 msgr2=0x7f349814eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34981595a0 0x7f349814eec0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f3490009e90 tx=0x7f349000b040 comp rx=0 tx=0).stop 2023-10-08T01:14:09.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 shutdown_connections 2023-10-08T01:14:09.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f347c067170 0x7f347c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f349806e8c0 0x7f3498169de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34981595a0 0x7f349814eec0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 --2- 172.21.15.119:0/2500566216 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f34980c6c60 0x7f349813e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 >> 172.21.15.119:0/2500566216 conn(0x7f349806ddb0 msgr2=0x7f34980cb240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:09.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 shutdown_connections 2023-10-08T01:14:09.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.679+0000 7f349dd9d700 1 -- 172.21.15.119:0/2500566216 wait complete. 2023-10-08T01:14:09.681 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T01:14:09.688 INFO:tasks.ceph.mds.c.smithi119.stderr: -5278> 2023-10-08T01:12:50.219+0000 7f3365bf1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:14:09.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T01:14:09.741 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T01:14:09.764 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:09.764+0000 7fa27bef4180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:14:09.828 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:09.825+0000 7f75e330a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:14:09.868 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:09.868+0000 7fa27bef4180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:14:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.899+0000 7fab6d99d700 1 Processor -- start 2023-10-08T01:14:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.899+0000 7fab6d99d700 1 -- start start 2023-10-08T01:14:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.900+0000 7fab6d99d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab680b16d0 0x7fab680b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.900+0000 7fab6d99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fab680b33b0 0x7fab680b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.900+0000 7fab6d99d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab680b2080 con 0x7fab680b4e70 2023-10-08T01:14:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.900+0000 7fab6d99d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab680cfb00 con 0x7fab680b16d0 2023-10-08T01:14:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.900+0000 7fab6d99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab680b5c40 con 0x7fab680b33b0 2023-10-08T01:14:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab67fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab680b16d0 0x7fab680b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab6c99b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fab680b4e70 0x7fab680ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47102/0 (socket says 172.21.15.119:47102) 2023-10-08T01:14:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab6c99b700 1 -- 172.21.15.119:0/841794331 learned_addr learned my addr 172.21.15.119:0/841794331 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab6d19c700 1 --2- 172.21.15.119:0/841794331 >> v2:172.21.15.184:3300/0 conn(0x7fab680b33b0 0x7fab680b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab67fff700 1 -- 172.21.15.119:0/841794331 >> v2:172.21.15.184:3300/0 conn(0x7fab680b33b0 msgr2=0x7fab680b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:09.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab67fff700 1 --2- 172.21.15.119:0/841794331 >> v2:172.21.15.184:3300/0 conn(0x7fab680b33b0 0x7fab680b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab677fe700 1 -- 172.21.15.119:0/841794331 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 859321708 0 0) 0x7fab680b2080 con 0x7fab680b4e70 2023-10-08T01:14:09.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab67fff700 1 -- 172.21.15.119:0/841794331 >> v1:172.21.15.119:6789/0 conn(0x7fab680b4e70 legacy=0x7fab680ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab67fff700 1 -- 172.21.15.119:0/841794331 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680b5f20 con 0x7fab680b16d0 2023-10-08T01:14:09.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.901+0000 7fab67fff700 1 --2- 172.21.15.119:0/841794331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b16d0 0x7fab680b1b40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fab68071380 tx=0x7fab540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f568d4739528a18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:09.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab677fe700 1 -- 172.21.15.119:0/841794331 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54011630 con 0x7fab680b16d0 2023-10-08T01:14:09.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab677fe700 1 -- 172.21.15.119:0/841794331 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab54011790 con 0x7fab680b16d0 2023-10-08T01:14:09.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab677fe700 1 -- 172.21.15.119:0/841794331 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54011b30 con 0x7fab680b16d0 2023-10-08T01:14:09.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab6d99d700 1 -- 172.21.15.119:0/841794331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b16d0 msgr2=0x7fab680b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab6d99d700 1 --2- 172.21.15.119:0/841794331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b16d0 0x7fab680b1b40 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fab68071380 tx=0x7fab540091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab6d99d700 1 -- 172.21.15.119:0/841794331 shutdown_connections 2023-10-08T01:14:09.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab6d99d700 1 --2- 172.21.15.119:0/841794331 >> v2:172.21.15.184:3300/0 conn(0x7fab680b33b0 0x7fab680b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab6d99d700 1 --2- 172.21.15.119:0/841794331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b16d0 0x7fab680b1b40 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab6d99d700 1 -- 172.21.15.119:0/841794331 >> 172.21.15.119:0/841794331 conn(0x7fab680d10c0 msgr2=0x7fab680affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.902+0000 7fab6d99d700 1 -- 172.21.15.119:0/841794331 shutdown_connections 2023-10-08T01:14:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.903+0000 7fab6d99d700 1 -- 172.21.15.119:0/841794331 wait complete. 2023-10-08T01:14:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.904+0000 7fab6d99d700 1 Processor -- start 2023-10-08T01:14:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.904+0000 7fab6d99d700 1 -- start start 2023-10-08T01:14:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab6d99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab680b16d0 0x7fab6816d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab6d99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b33b0 0x7fab681699c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab6d99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab680b4e70 0x7fab680cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab6d99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680c7020 con 0x7fab680b33b0 2023-10-08T01:14:09.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab6d99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680c7270 con 0x7fab680b16d0 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab6d99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab680c7550 con 0x7fab680b4e70 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b33b0 0x7fab681699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b33b0 0x7fab681699c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34800/0 (socket says 172.21.15.119:34800) 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab67fff700 1 -- 172.21.15.119:0/4073260113 learned_addr learned my addr 172.21.15.119:0/4073260113 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab6c99b700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab680b16d0 0x7fab6816d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.905+0000 7fab6d19c700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab680b4e70 0x7fab680cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab67fff700 1 -- 172.21.15.119:0/4073260113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab680b4e70 msgr2=0x7fab680cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab67fff700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab680b4e70 0x7fab680cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab67fff700 1 -- 172.21.15.119:0/4073260113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab680b16d0 msgr2=0x7fab6816d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab67fff700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab680b16d0 0x7fab6816d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab67fff700 1 -- 172.21.15.119:0/4073260113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab54009420 con 0x7fab680b33b0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab67fff700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b33b0 0x7fab681699c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fab68070dd0 tx=0x7fab54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab64ff9700 1 -- 172.21.15.119:0/4073260113 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54011900 con 0x7fab680b33b0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab64ff9700 1 -- 172.21.15.119:0/4073260113 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab54011a60 con 0x7fab680b33b0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab64ff9700 1 -- 172.21.15.119:0/4073260113 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab54025640 con 0x7fab680b33b0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab680c7830 con 0x7fab680b33b0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.906+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab680c7df0 con 0x7fab680b33b0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.908+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab680c8010 con 0x7fab680b33b0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.908+0000 7fab64ff9700 1 -- 172.21.15.119:0/4073260113 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 54) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fab54025a70 con 0x7fab680b33b0 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.909+0000 7fab64ff9700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 0x7fab4c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.909+0000 7fab6c99b700 1 -- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 msgr2=0x7fab4c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.910+0000 7fab64ff9700 1 -- 172.21.15.119:0/4073260113 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6470+0+0 (secure 0 0 0) 0x7fab540862a0 con 0x7fab680b33b0 2023-10-08T01:14:09.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.910+0000 7fab6c99b700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 0x7fab4c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:09.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:09.914+0000 7fab64ff9700 1 -- 172.21.15.119:0/4073260113 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fab54052ee0 con 0x7fab680b33b0 2023-10-08T01:14:09.931 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:09.930+0000 7f75e330a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:14:10.055 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:10.055+0000 7fa27bef4180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:14:10.098 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:10.097+0000 7f75e330a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:14:10.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.111+0000 7fab6c99b700 1 -- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 msgr2=0x7fab4c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:10.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.111+0000 7fab6c99b700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 0x7fab4c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:10.146 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:10.146+0000 7fa27bef4180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:14:10.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.179+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fab6804bb80 con 0x7fab680b33b0 2023-10-08T01:14:10.206 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:10.206+0000 7f75e330a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:14:10.236 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:10.236+0000 7fa27bef4180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:14:10.328 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:10.328+0000 7fa27bef4180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:14:10.398 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:10.397+0000 7f75e330a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:14:10.491 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:10.490+0000 7f75e330a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:14:10.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.512+0000 7fab6c99b700 1 -- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 msgr2=0x7fab4c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:10.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.512+0000 7fab6c99b700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 0x7fab4c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:14:10.583 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:10.583+0000 7f75e330a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:14:10.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.676+0000 7fab64ff9700 1 -- 172.21.15.119:0/4073260113 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7fab54056500 con 0x7fab680b33b0 2023-10-08T01:14:10.679 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:10.678+0000 7f75e330a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 msgr2=0x7fab4c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 0x7fab4c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b33b0 msgr2=0x7fab681699c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b33b0 0x7fab681699c0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fab68070dd0 tx=0x7fab54017000 comp rx=0 tx=0).stop 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 shutdown_connections 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fab4c067170 0x7fab4c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fab680b4e70 0x7fab680cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fab680b16d0 0x7fab6816d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 --2- 172.21.15.119:0/4073260113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab680b33b0 0x7fab681699c0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.679+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 >> 172.21.15.119:0/4073260113 conn(0x7fab680d10c0 msgr2=0x7fab68069710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:10.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.681+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 shutdown_connections 2023-10-08T01:14:10.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.682+0000 7fab6d99d700 1 -- 172.21.15.119:0/4073260113 wait complete. 2023-10-08T01:14:10.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T01:14:10.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.933+0000 7f03bf19d700 1 Processor -- start 2023-10-08T01:14:10.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.933+0000 7f03bf19d700 1 -- start start 2023-10-08T01:14:10.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.934+0000 7f03bf19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03b80656e0 0x7f03b80ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:10.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.934+0000 7f03bf19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b80aeb10 con 0x7f03b80ae180 2023-10-08T01:14:10.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03bf19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b80af540 con 0x7f03b806b290 2023-10-08T01:14:10.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03bf19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b80aedf0 con 0x7f03b80656e0 2023-10-08T01:14:10.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03bd99a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f03b80656e0 0x7f03b80ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:10.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03be19b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f03b80ae180 0x7f03b80acb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47118/0 (socket says 172.21.15.119:47118) 2023-10-08T01:14:10.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03be19b700 1 -- 172.21.15.119:0/3434075719 learned_addr learned my addr 172.21.15.119:0/3434075719 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:10.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03bd99a700 1 -- 172.21.15.119:0/3434075719 >> v1:172.21.15.184:6789/0 conn(0x7f03b806b290 legacy=0x7f03b80b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03bd99a700 1 -- 172.21.15.119:0/3434075719 >> v1:172.21.15.119:6789/0 conn(0x7f03b80ae180 legacy=0x7f03b80acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03bd99a700 1 -- 172.21.15.119:0/3434075719 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80af820 con 0x7f03b80656e0 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.935+0000 7f03bd99a700 1 --2- 172.21.15.119:0/3434075719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b80ad260 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098e0 tx=0x7f03a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bb1882b1a1ddd3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bd199700 1 -- 172.21.15.119:0/3434075719 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8009e80 con 0x7f03b80656e0 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bd199700 1 -- 172.21.15.119:0/3434075719 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a800b860 con 0x7f03b80656e0 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bd199700 1 -- 172.21.15.119:0/3434075719 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a800bbe0 con 0x7f03b80656e0 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bf19d700 1 -- 172.21.15.119:0/3434075719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 msgr2=0x7f03b80ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bf19d700 1 --2- 172.21.15.119:0/3434075719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b80ad260 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f03a80098e0 tx=0x7f03a80091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bf19d700 1 -- 172.21.15.119:0/3434075719 shutdown_connections 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bf19d700 1 --2- 172.21.15.119:0/3434075719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b80ad260 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bf19d700 1 -- 172.21.15.119:0/3434075719 >> 172.21.15.119:0/3434075719 conn(0x7f03b806f6a0 msgr2=0x7f03b806ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:10.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bf19d700 1 -- 172.21.15.119:0/3434075719 shutdown_connections 2023-10-08T01:14:10.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.936+0000 7f03bf19d700 1 -- 172.21.15.119:0/3434075719 wait complete. 2023-10-08T01:14:10.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.938+0000 7f03bf19d700 1 Processor -- start 2023-10-08T01:14:10.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.938+0000 7f03bf19d700 1 -- start start 2023-10-08T01:14:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03bf19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b8166c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03bf19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03b806b290 0x7f03b815f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03bf19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03b80ae180 0x7f03b815df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03bf19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b8169a10 con 0x7f03b80656e0 2023-10-08T01:14:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03bf19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b8169cf0 con 0x7f03b80ae180 2023-10-08T01:14:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03bf19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b8169fd0 con 0x7f03b806b290 2023-10-08T01:14:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b8166c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be19b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b8166c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34916/0 (socket says 172.21.15.119:34916) 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be19b700 1 -- 172.21.15.119:0/3882431678 learned_addr learned my addr 172.21.15.119:0/3882431678 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03bd99a700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03b806b290 0x7f03b815f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be99c700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03b80ae180 0x7f03b815df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be19b700 1 -- 172.21.15.119:0/3882431678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03b806b290 msgr2=0x7f03b815f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be19b700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03b806b290 0x7f03b815f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be19b700 1 -- 172.21.15.119:0/3882431678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03b80ae180 msgr2=0x7f03b815df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be19b700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03b80ae180 0x7f03b815df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.939+0000 7f03be19b700 1 -- 172.21.15.119:0/3882431678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a8009420 con 0x7f03b80656e0 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.940+0000 7f03be19b700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b8166c90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c590 tx=0x7f03b4007e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.940+0000 7f03ae7fc700 1 -- 172.21.15.119:0/3882431678 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b4001160 con 0x7f03b80656e0 2023-10-08T01:14:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.940+0000 7f03ae7fc700 1 -- 172.21.15.119:0/3882431678 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b40072e0 con 0x7f03b80656e0 2023-10-08T01:14:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.940+0000 7f03ae7fc700 1 -- 172.21.15.119:0/3882431678 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b40057f0 con 0x7f03b80656e0 2023-10-08T01:14:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.940+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b816a2b0 con 0x7f03b80656e0 2023-10-08T01:14:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.940+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b81671d0 con 0x7f03b80656e0 2023-10-08T01:14:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.941+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b804c3c0 con 0x7f03b80656e0 2023-10-08T01:14:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.942+0000 7f03ae7fc700 1 -- 172.21.15.119:0/3882431678 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 56) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f03b4005c20 con 0x7f03b80656e0 2023-10-08T01:14:10.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.943+0000 7f03ae7fc700 1 -- 172.21.15.119:0/3882431678 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f03b4083810 con 0x7f03b80656e0 2023-10-08T01:14:10.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:10.947+0000 7f03ae7fc700 1 -- 172.21.15.119:0/3882431678 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03b40509f0 con 0x7f03b80656e0 2023-10-08T01:14:11.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.136+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03b806caa0 con 0x7f03b80656e0 2023-10-08T01:14:11.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.698+0000 7f03ae7fc700 1 -- 172.21.15.119:0/3882431678 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 57) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f03b40504a0 con 0x7f03b80656e0 2023-10-08T01:14:11.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.698+0000 7f03ae7fc700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f039c06ca90 0x7f039c06ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:11.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.699+0000 7f03bd99a700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f039c06ca90 0x7f039c06ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:11.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.701+0000 7f03bd99a700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f039c06ca90 0x7f039c06ee70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f03b8065ab0 tx=0x7f03a8017000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:11.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.706+0000 7f03ae7fc700 1 -- 172.21.15.119:0/3882431678 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) v1 ==== 181+0+0 (secure 0 0 0) 0x7f03b406fd40 con 0x7f03b80656e0 2023-10-08T01:14:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f039c06ca90 msgr2=0x7f039c06ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f039c06ca90 0x7f039c06ee70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f03b8065ab0 tx=0x7f03a8017000 comp rx=0 tx=0).stop 2023-10-08T01:14:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 msgr2=0x7f03b8166c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b8166c90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c590 tx=0x7f03b4007e40 comp rx=0 tx=0).stop 2023-10-08T01:14:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 shutdown_connections 2023-10-08T01:14:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f039c06ca90 0x7f039c06ee70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f03b80ae180 0x7f03b815df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f03b80656e0 0x7f03b8166c90 secure :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c590 tx=0x7f03b4007e40 comp rx=0 tx=0).stop 2023-10-08T01:14:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 --2- 172.21.15.119:0/3882431678 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f03b806b290 0x7f03b815f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 >> 172.21.15.119:0/3882431678 conn(0x7f03b806f6a0 msgr2=0x7f03b8069230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 shutdown_connections 2023-10-08T01:14:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.710+0000 7f03bf19d700 1 -- 172.21.15.119:0/3882431678 wait complete. 2023-10-08T01:14:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T01:14:11.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T01:14:11.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.918+0000 7f936d19d700 1 Processor -- start 2023-10-08T01:14:11.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.918+0000 7f936d19d700 1 -- start start 2023-10-08T01:14:11.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.919+0000 7f936d19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93680b03c0 0x7f93680cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:11.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.919+0000 7f936d19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b0790 0x7f93680b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:11.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.919+0000 7f936d19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93680cfa00 con 0x7f93681591e0 2023-10-08T01:14:11.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.919+0000 7f936d19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93680cb580 con 0x7f93680b03c0 2023-10-08T01:14:11.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.919+0000 7f936d19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93680b2de0 con 0x7f93680b0790 2023-10-08T01:14:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f93677fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b0790 0x7f93680b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f9367fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93680b03c0 0x7f93680cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f93677fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b0790 0x7f93680b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:50890/0 (socket says 172.21.15.119:50890) 2023-10-08T01:14:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f9367fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93680b03c0 0x7f93680cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:51852/0 (socket says 172.21.15.119:51852) 2023-10-08T01:14:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f93677fe700 1 -- 172.21.15.119:0/256255589 learned_addr learned my addr 172.21.15.119:0/256255589 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f9366ffd700 1 -- 172.21.15.119:0/256255589 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238524397 0 0) 0x7f93680cfa00 con 0x7f93681591e0 2023-10-08T01:14:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f9366ffd700 1 -- 172.21.15.119:0/256255589 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9350003570 con 0x7f93681591e0 2023-10-08T01:14:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f93677fe700 1 -- 172.21.15.119:0/256255589 >> v2:172.21.15.184:3300/0 conn(0x7f93680b03c0 msgr2=0x7f93680cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f93677fe700 1 --2- 172.21.15.119:0/256255589 >> v2:172.21.15.184:3300/0 conn(0x7f93680b03c0 0x7f93680cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f93677fe700 1 -- 172.21.15.119:0/256255589 >> v1:172.21.15.184:6789/0 conn(0x7f93681591e0 legacy=0x7f9368159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f93677fe700 1 -- 172.21.15.119:0/256255589 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93680cf160 con 0x7f93680b0790 2023-10-08T01:14:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.920+0000 7f9367fff700 1 --2- 172.21.15.119:0/256255589 >> v2:172.21.15.184:3300/0 conn(0x7f93680b03c0 0x7f93680cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.921+0000 7f93677fe700 1 --2- 172.21.15.119:0/256255589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b0790 0x7f93680b28a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f935c0099b0 tx=0x7f935c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6449e99f4feb9413 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.921+0000 7f9366ffd700 1 -- 172.21.15.119:0/256255589 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00b680 con 0x7f93680b0790 2023-10-08T01:14:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.921+0000 7f9366ffd700 1 -- 172.21.15.119:0/256255589 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c00b7e0 con 0x7f93680b0790 2023-10-08T01:14:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.921+0000 7f9366ffd700 1 -- 172.21.15.119:0/256255589 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00bb60 con 0x7f93680b0790 2023-10-08T01:14:11.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.921+0000 7f936d19d700 1 -- 172.21.15.119:0/256255589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b0790 msgr2=0x7f93680b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:11.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.922+0000 7f936d19d700 1 --2- 172.21.15.119:0/256255589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b0790 0x7f93680b28a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f935c0099b0 tx=0x7f935c009250 comp rx=0 tx=0).stop 2023-10-08T01:14:11.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.922+0000 7f936d19d700 1 -- 172.21.15.119:0/256255589 shutdown_connections 2023-10-08T01:14:11.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.922+0000 7f936d19d700 1 --2- 172.21.15.119:0/256255589 >> v2:172.21.15.184:3300/0 conn(0x7f93680b03c0 0x7f93680cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.922+0000 7f936d19d700 1 --2- 172.21.15.119:0/256255589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b0790 0x7f93680b28a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.922+0000 7f936d19d700 1 -- 172.21.15.119:0/256255589 >> 172.21.15.119:0/256255589 conn(0x7f9368063230 msgr2=0x7f9368063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.922+0000 7f936d19d700 1 -- 172.21.15.119:0/256255589 shutdown_connections 2023-10-08T01:14:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.922+0000 7f936d19d700 1 -- 172.21.15.119:0/256255589 wait complete. 2023-10-08T01:14:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.923+0000 7f936d19d700 1 Processor -- start 2023-10-08T01:14:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.923+0000 7f936d19d700 1 -- start start 2023-10-08T01:14:11.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.924+0000 7f936d19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b03c0 0x7f9368155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:11.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.924+0000 7f936d19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93680b0790 0x7f93680b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:11.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.924+0000 7f936d19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93681591e0 0x7f93680b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:11.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.924+0000 7f936d19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9368152470 con 0x7f93681591e0 2023-10-08T01:14:11.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.924+0000 7f936d19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93680c7120 con 0x7f93680b0790 2023-10-08T01:14:11.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.924+0000 7f936d19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93680c6880 con 0x7f93680b03c0 2023-10-08T01:14:11.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.924+0000 7f936c99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93681591e0 0x7f93680b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:11.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.924+0000 7f9367fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b03c0 0x7f9368155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936c99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93681591e0 0x7f93680b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34926/0 (socket says 172.21.15.119:34926) 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936c99c700 1 -- 172.21.15.119:0/338065014 learned_addr learned my addr 172.21.15.119:0/338065014 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f93677fe700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93680b0790 0x7f93680b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936c99c700 1 -- 172.21.15.119:0/338065014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b03c0 msgr2=0x7f9368155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936c99c700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b03c0 0x7f9368155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936c99c700 1 -- 172.21.15.119:0/338065014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93680b0790 msgr2=0x7f93680b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936c99c700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93680b0790 0x7f93680b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936c99c700 1 -- 172.21.15.119:0/338065014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358009420 con 0x7f93681591e0 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936c99c700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93681591e0 0x7f93680b47e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f936000c910 tx=0x7f9360002720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f934ffff700 1 -- 172.21.15.119:0/338065014 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360012630 con 0x7f93681591e0 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f934ffff700 1 -- 172.21.15.119:0/338065014 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9360012790 con 0x7f93681591e0 2023-10-08T01:14:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f934ffff700 1 -- 172.21.15.119:0/338065014 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360012b50 con 0x7f93681591e0 2023-10-08T01:14:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.925+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c0094f0 con 0x7f93681591e0 2023-10-08T01:14:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.926+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93680c6e40 con 0x7f93681591e0 2023-10-08T01:14:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.927+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936804c3c0 con 0x7f93681591e0 2023-10-08T01:14:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.927+0000 7f934ffff700 1 -- 172.21.15.119:0/338065014 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 57) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f936001a3a0 con 0x7f93681591e0 2023-10-08T01:14:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.928+0000 7f934ffff700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9354066d80 0x7f9354069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.928+0000 7f934ffff700 1 -- 172.21.15.119:0/338065014 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6376+0+0 (secure 0 0 0) 0x7f9360084660 con 0x7f93681591e0 2023-10-08T01:14:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.929+0000 7f9367fff700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9354066d80 0x7f9354069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.929+0000 7f9367fff700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9354066d80 0x7f9354069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9358009ea0 tx=0x7f93580091a0 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:11.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:11.933+0000 7f934ffff700 1 -- 172.21.15.119:0/338065014 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93600518e0 con 0x7f93681591e0 2023-10-08T01:14:12.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.126+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f93680ccab0 con 0x7f93681591e0 2023-10-08T01:14:12.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.709+0000 7f934ffff700 1 -- 172.21.15.119:0/338065014 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9360054f00 con 0x7f93681591e0 2023-10-08T01:14:12.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9354066d80 msgr2=0x7f9354069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9354066d80 0x7f9354069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9358009ea0 tx=0x7f93580091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93681591e0 msgr2=0x7f93680b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93681591e0 0x7f93680b47e0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f936000c910 tx=0x7f9360002720 comp rx=0 tx=0).stop 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 shutdown_connections 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9354066d80 0x7f9354069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93680b03c0 0x7f9368155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93681591e0 0x7f93680b47e0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 --2- 172.21.15.119:0/338065014 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93680b0790 0x7f93680b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.713+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 >> 172.21.15.119:0/338065014 conn(0x7f9368063230 msgr2=0x7f93680b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:12.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.714+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 shutdown_connections 2023-10-08T01:14:12.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.714+0000 7f936d19d700 1 -- 172.21.15.119:0/338065014 wait complete. 2023-10-08T01:14:12.718 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T01:14:12.741 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T01:14:12.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.925+0000 7fe1a4bff700 1 Processor -- start 2023-10-08T01:14:12.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.925+0000 7fe1a4bff700 1 -- start start 2023-10-08T01:14:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.926+0000 7fe1a4bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe1a00c83f0 0x7fe1a00c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.926+0000 7fe1a4bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe1a00ccee0 0x7fe1a01561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.926+0000 7fe1a4bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a015b0d0 con 0x7fe1a00c87c0 2023-10-08T01:14:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.926+0000 7fe1a4bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a00c7a60 con 0x7fe1a00ccee0 2023-10-08T01:14:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.926+0000 7fe1a4bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a015a4c0 con 0x7fe1a00c83f0 2023-10-08T01:14:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19ed9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe1a00c87c0 0x7fe1a006eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47122/0 (socket says 172.21.15.119:47122) 2023-10-08T01:14:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19ed9d700 1 -- 172.21.15.119:0/1428589118 learned_addr learned my addr 172.21.15.119:0/1428589118 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19dd9b700 1 --2- 172.21.15.119:0/1428589118 >> v2:172.21.15.119:3300/0 conn(0x7fe1a00ccee0 0x7fe1a01561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19e59c700 1 --2- 172.21.15.119:0/1428589118 >> v2:172.21.15.184:3300/0 conn(0x7fe1a00c83f0 0x7fe1a00c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19d59a700 1 -- 172.21.15.119:0/1428589118 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339926287 0 0) 0x7fe1a015b0d0 con 0x7fe1a00c87c0 2023-10-08T01:14:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19dd9b700 1 -- 172.21.15.119:0/1428589118 >> v2:172.21.15.184:3300/0 conn(0x7fe1a00c83f0 msgr2=0x7fe1a00c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19dd9b700 1 --2- 172.21.15.119:0/1428589118 >> v2:172.21.15.184:3300/0 conn(0x7fe1a00c83f0 0x7fe1a00c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19dd9b700 1 -- 172.21.15.119:0/1428589118 >> v1:172.21.15.119:6789/0 conn(0x7fe1a00c87c0 legacy=0x7fe1a006eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19dd9b700 1 -- 172.21.15.119:0/1428589118 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a015a7a0 con 0x7fe1a00ccee0 2023-10-08T01:14:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19e59c700 1 --2- 172.21.15.119:0/1428589118 >> v2:172.21.15.184:3300/0 conn(0x7fe1a00c83f0 0x7fe1a00c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.927+0000 7fe19dd9b700 1 --2- 172.21.15.119:0/1428589118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00ccee0 0x7fe1a01561d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00c9fd0 tx=0x7fe1940091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1ad40e1045da18 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe19d59a700 1 -- 172.21.15.119:0/1428589118 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19400b6b0 con 0x7fe1a00ccee0 2023-10-08T01:14:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe19d59a700 1 -- 172.21.15.119:0/1428589118 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19400b810 con 0x7fe1a00ccee0 2023-10-08T01:14:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe19d59a700 1 -- 172.21.15.119:0/1428589118 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19400bbb0 con 0x7fe1a00ccee0 2023-10-08T01:14:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe1a4bff700 1 -- 172.21.15.119:0/1428589118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00ccee0 msgr2=0x7fe1a01561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/1428589118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00ccee0 0x7fe1a01561d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe1a00c9fd0 tx=0x7fe1940091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe1a4bff700 1 -- 172.21.15.119:0/1428589118 shutdown_connections 2023-10-08T01:14:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/1428589118 >> v2:172.21.15.184:3300/0 conn(0x7fe1a00c83f0 0x7fe1a00c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/1428589118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00ccee0 0x7fe1a01561d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe1a4bff700 1 -- 172.21.15.119:0/1428589118 >> 172.21.15.119:0/1428589118 conn(0x7fe1a0068e10 msgr2=0x7fe1a00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe1a4bff700 1 -- 172.21.15.119:0/1428589118 shutdown_connections 2023-10-08T01:14:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.928+0000 7fe1a4bff700 1 -- 172.21.15.119:0/1428589118 wait complete. 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.930+0000 7fe1a4bff700 1 Processor -- start 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.930+0000 7fe1a4bff700 1 -- start start 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe1a4bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1a00c83f0 0x7fe1a0157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe1a4bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00c87c0 0x7fe1a0156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe1a4bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1a00ccee0 0x7fe1a0168e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe1a4bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a013f770 con 0x7fe1a00c87c0 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe1a4bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0157520 con 0x7fe1a00c83f0 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe1a4bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0158ac0 con 0x7fe1a00ccee0 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19dd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00c87c0 0x7fe1a0156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19dd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00c87c0 0x7fe1a0156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34940/0 (socket says 172.21.15.119:34940) 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19dd9b700 1 -- 172.21.15.119:0/2603252343 learned_addr learned my addr 172.21.15.119:0/2603252343 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19e59c700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1a00c83f0 0x7fe1a0157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19ed9d700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1a00ccee0 0x7fe1a0168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19dd9b700 1 -- 172.21.15.119:0/2603252343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1a00ccee0 msgr2=0x7fe1a0168e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19dd9b700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1a00ccee0 0x7fe1a0168e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19dd9b700 1 -- 172.21.15.119:0/2603252343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1a00c83f0 msgr2=0x7fe1a0157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19dd9b700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1a00c83f0 0x7fe1a0157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.931+0000 7fe19dd9b700 1 -- 172.21.15.119:0/2603252343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe194009420 con 0x7fe1a00c87c0 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.932+0000 7fe19dd9b700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00c87c0 0x7fe1a0156fe0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0063d40 tx=0x7fe194017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.932+0000 7fe18e7fc700 1 -- 172.21.15.119:0/2603252343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194011420 con 0x7fe1a00c87c0 2023-10-08T01:14:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.932+0000 7fe18e7fc700 1 -- 172.21.15.119:0/2603252343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1940115a0 con 0x7fe1a00c87c0 2023-10-08T01:14:12.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.932+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a0157eb0 con 0x7fe1a00c87c0 2023-10-08T01:14:12.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.932+0000 7fe18e7fc700 1 -- 172.21.15.119:0/2603252343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194011960 con 0x7fe1a00c87c0 2023-10-08T01:14:12.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.932+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a0158470 con 0x7fe1a00c87c0 2023-10-08T01:14:12.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.933+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a004c3c0 con 0x7fe1a00c87c0 2023-10-08T01:14:12.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.934+0000 7fe18e7fc700 1 -- 172.21.15.119:0/2603252343 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe194011d50 con 0x7fe1a00c87c0 2023-10-08T01:14:12.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.935+0000 7fe18e7fc700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe180066f30 0x7fe1800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:12.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.935+0000 7fe18e7fc700 1 -- 172.21.15.119:0/2603252343 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5933+0+0 (secure 0 0 0) 0x7fe1940861a0 con 0x7fe1a00c87c0 2023-10-08T01:14:12.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.936+0000 7fe19e59c700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe180066f30 0x7fe1800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:12.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.936+0000 7fe19e59c700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe180066f30 0x7fe1800693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe190006fd0 tx=0x7fe190008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:12.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:12.940+0000 7fe18e7fc700 1 -- 172.21.15.119:0/2603252343 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe194053290 con 0x7fe1a00c87c0 2023-10-08T01:14:13.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.130+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe1a0158c00 con 0x7fe1a00c87c0 2023-10-08T01:14:13.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.134+0000 7fe18e7fc700 1 -- 172.21.15.119:0/2603252343 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe1940568b0 con 0x7fe1a00c87c0 2023-10-08T01:14:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe180066f30 msgr2=0x7fe1800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe180066f30 0x7fe1800693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe190006fd0 tx=0x7fe190008040 comp rx=0 tx=0).stop 2023-10-08T01:14:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00c87c0 msgr2=0x7fe1a0156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00c87c0 0x7fe1a0156fe0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0063d40 tx=0x7fe194017000 comp rx=0 tx=0).stop 2023-10-08T01:14:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 shutdown_connections 2023-10-08T01:14:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe180066f30 0x7fe1800693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe1a00ccee0 0x7fe1a0168e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1a00c83f0 0x7fe1a0157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 --2- 172.21.15.119:0/2603252343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1a00c87c0 0x7fe1a0156fe0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 >> 172.21.15.119:0/2603252343 conn(0x7fe1a0068e10 msgr2=0x7fe1a00b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 shutdown_connections 2023-10-08T01:14:13.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.137+0000 7fe1a4bff700 1 -- 172.21.15.119:0/2603252343 wait complete. 2023-10-08T01:14:13.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T01:14:13.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.348+0000 7ff0bffff700 1 Processor -- start 2023-10-08T01:14:13.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.349+0000 7ff0bffff700 1 -- start start 2023-10-08T01:14:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0bffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b8155eb0 0x7ff0b806e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0bffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff0b80c70b0 0x7ff0b8069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0bffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b806eb90 con 0x7ff0b8069cb0 2023-10-08T01:14:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0bffff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b806a130 con 0x7ff0b80c70b0 2023-10-08T01:14:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0bffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b806ee70 con 0x7ff0b8155eb0 2023-10-08T01:14:13.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0be59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff0b8069cb0 0x7ff0b8159a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47132/0 (socket says 172.21.15.119:47132) 2023-10-08T01:14:13.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0be59c700 1 -- 172.21.15.119:0/443318272 learned_addr learned my addr 172.21.15.119:0/443318272 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:13.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0bdd9b700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b8155eb0 0x7ff0b806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0bd59a700 1 --2- 172.21.15.119:0/443318272 >> v2:172.21.15.184:3300/0 conn(0x7ff0b80c70b0 0x7ff0b8069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.350+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142359593 0 0) 0x7ff0b806eb90 con 0x7ff0b8069cb0 2023-10-08T01:14:13.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a4004680 con 0x7ff0b8069cb0 2023-10-08T01:14:13.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 195671126 0 0) 0x7ff0a4004680 con 0x7ff0b8069cb0 2023-10-08T01:14:13.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b8155eb0 msgr2=0x7ff0b806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:13.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b8155eb0 0x7ff0b806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 >> v2:172.21.15.184:3300/0 conn(0x7ff0b80c70b0 msgr2=0x7ff0b8069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:13.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 --2- 172.21.15.119:0/443318272 >> v2:172.21.15.184:3300/0 conn(0x7ff0b80c70b0 0x7ff0b8069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b806f150 con 0x7ff0b8069cb0 2023-10-08T01:14:13.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7ff0b00017f0 con 0x7ff0b8069cb0 2023-10-08T01:14:13.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bd59a700 1 --2- 172.21.15.119:0/443318272 >> v2:172.21.15.184:3300/0 conn(0x7ff0b80c70b0 0x7ff0b8069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:13.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 >> v1:172.21.15.119:6789/0 conn(0x7ff0b8069cb0 legacy=0x7ff0b8159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a4004b70 0x7ff0a4004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0a4005920 0x7ff0a4005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0a40062d0 0x7ff0a4003490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b806a130 con 0x7ff0a40062d0 2023-10-08T01:14:13.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b806ee70 con 0x7ff0a4004b70 2023-10-08T01:14:13.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a40039d0 con 0x7ff0a4005920 2023-10-08T01:14:13.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bdd9b700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b8155eb0 0x7ff0b806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:13.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0be59c700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0a40062d0 0x7ff0a4003490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bd59a700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0a4005920 0x7ff0a4005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.351+0000 7ff0bdd9b700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a4004b70 0x7ff0a4004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0be59c700 1 -- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0a4005920 msgr2=0x7ff0a4005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:13.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0be59c700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0a4005920 0x7ff0a4005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0be59c700 1 -- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a4004b70 msgr2=0x7ff0a4004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:13.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0be59c700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a4004b70 0x7ff0a4004f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0be59c700 1 -- 172.21.15.119:0/443318272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a4004680 con 0x7ff0a40062d0 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0be59c700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0a40062d0 0x7ff0a4003490 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000a970 tx=0x7ff0b0002210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e94243fe55ffdfd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b00070d0 con 0x7ff0a40062d0 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b0007270 con 0x7ff0a40062d0 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.352+0000 7ff0bcd99700 1 -- 172.21.15.119:0/443318272 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0007630 con 0x7ff0a40062d0 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.353+0000 7ff0bffff700 1 -- 172.21.15.119:0/443318272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0a40062d0 msgr2=0x7ff0a4003490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.353+0000 7ff0bffff700 1 --2- 172.21.15.119:0/443318272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0a40062d0 0x7ff0a4003490 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000a970 tx=0x7ff0b0002210 comp rx=0 tx=0).stop 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.353+0000 7ff0be59c700 1 -- 172.21.15.119:0/443318272 reap_dead start 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.353+0000 7ff0bffff700 1 -- 172.21.15.119:0/443318272 shutdown_connections 2023-10-08T01:14:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.353+0000 7ff0bffff700 1 -- 172.21.15.119:0/443318272 >> 172.21.15.119:0/443318272 conn(0x7ff0b8068e10 msgr2=0x7ff0b80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.353+0000 7ff0bffff700 1 -- 172.21.15.119:0/443318272 shutdown_connections 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.353+0000 7ff0bffff700 1 -- 172.21.15.119:0/443318272 wait complete. 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.354+0000 7ff0bffff700 1 Processor -- start 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.354+0000 7ff0bffff700 1 -- start start 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a40062d0 0x7ff0b8140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b8152b20 0x7ff0b8169ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b80c5840 0x7ff0b80c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b8140790 con 0x7ff0b8152b20 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b816a0e0 con 0x7ff0a40062d0 2023-10-08T01:14:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b80c4340 con 0x7ff0b80c5840 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bd59a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b8152b20 0x7ff0b8169ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bd59a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b8152b20 0x7ff0b8169ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34964/0 (socket says 172.21.15.119:34964) 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.355+0000 7ff0bd59a700 1 -- 172.21.15.119:0/4137419717 learned_addr learned my addr 172.21.15.119:0/4137419717 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0be59c700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b80c5840 0x7ff0b80c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bdd9b700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a40062d0 0x7ff0b8140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bd59a700 1 -- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b80c5840 msgr2=0x7ff0b80c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bd59a700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b80c5840 0x7ff0b80c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bd59a700 1 -- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a40062d0 msgr2=0x7ff0b8140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bd59a700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a40062d0 0x7ff0b8140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bd59a700 1 -- 172.21.15.119:0/4137419717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b816ab90 con 0x7ff0b8152b20 2023-10-08T01:14:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bdd9b700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a40062d0 0x7ff0b8140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bd59a700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b8152b20 0x7ff0b8169ba0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8011d70 tx=0x7ff0a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0adffb700 1 -- 172.21.15.119:0/4137419717 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a800fcd0 con 0x7ff0b8152b20 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0adffb700 1 -- 172.21.15.119:0/4137419717 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a800fe30 con 0x7ff0b8152b20 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0adffb700 1 -- 172.21.15.119:0/4137419717 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a801d6d0 con 0x7ff0b8152b20 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b80ca580 con 0x7ff0b8152b20 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.356+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b816a5d0 con 0x7ff0b8152b20 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.357+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b80ca710 con 0x7ff0b8152b20 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.359+0000 7ff0adffb700 1 -- 172.21.15.119:0/4137419717 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0a801da80 con 0x7ff0b8152b20 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.360+0000 7ff0adffb700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff09c066f30 0x7ff09c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.360+0000 7ff0adffb700 1 -- 172.21.15.119:0/4137419717 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5933+0+0 (secure 0 0 0) 0x7ff0a8018070 con 0x7ff0b8152b20 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.360+0000 7ff0bdd9b700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff09c066f30 0x7ff09c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.361+0000 7ff0bdd9b700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff09c066f30 0x7ff09c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4006f00 tx=0x7ff0b4008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:13.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.364+0000 7ff0adffb700 1 -- 172.21.15.119:0/4137419717 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0a8052960 con 0x7ff0b8152b20 2023-10-08T01:14:13.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.554+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff0b804c3c0 con 0x7ff0b8152b20 2023-10-08T01:14:13.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.560+0000 7ff0adffb700 1 -- 172.21.15.119:0/4137419717 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v38) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff0a8055f80 con 0x7ff0b8152b20 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff09c066f30 msgr2=0x7ff09c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff09c066f30 0x7ff09c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4006f00 tx=0x7ff0b4008040 comp rx=0 tx=0).stop 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b8152b20 msgr2=0x7ff0b8169ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b8152b20 0x7ff0b8169ba0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8011d70 tx=0x7ff0a80091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 shutdown_connections 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff09c066f30 0x7ff09c0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0b80c5840 0x7ff0b80c5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0a40062d0 0x7ff0b8140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 --2- 172.21.15.119:0/4137419717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0b8152b20 0x7ff0b8169ba0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.562+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 >> 172.21.15.119:0/4137419717 conn(0x7ff0b8068e10 msgr2=0x7ff0b8154be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:13.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.563+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 shutdown_connections 2023-10-08T01:14:13.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.563+0000 7ff0bffff700 1 -- 172.21.15.119:0/4137419717 wait complete. 2023-10-08T01:14:13.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T01:14:13.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.765+0000 7ff3633ff700 1 Processor -- start 2023-10-08T01:14:13.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.765+0000 7ff3633ff700 1 -- start start 2023-10-08T01:14:13.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff3633ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c0c83f0 0x7ff35c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff3633ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff35c1568e0 0x7ff35c0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff3633ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35c15b0d0 con 0x7ff35c0ccee0 2023-10-08T01:14:13.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff3633ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35c0c7a60 con 0x7ff35c1568e0 2023-10-08T01:14:13.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff3633ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c15a4c0 con 0x7ff35c0c83f0 2023-10-08T01:14:13.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff36099a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff35c0ccee0 0x7ff35c1561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47144/0 (socket says 172.21.15.119:47144) 2023-10-08T01:14:13.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff36099a700 1 -- 172.21.15.119:0/1531841055 learned_addr learned my addr 172.21.15.119:0/1531841055 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:13.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff36199c700 1 --2- 172.21.15.119:0/1531841055 >> v2:172.21.15.119:3300/0 conn(0x7ff35c1568e0 0x7ff35c0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff36119b700 1 --2- 172.21.15.119:0/1531841055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c0c83f0 0x7ff35c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff353fff700 1 -- 172.21.15.119:0/1531841055 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4109865426 0 0) 0x7ff35c15b0d0 con 0x7ff35c0ccee0 2023-10-08T01:14:13.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.767+0000 7ff353fff700 1 -- 172.21.15.119:0/1531841055 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff348003570 con 0x7ff35c0ccee0 2023-10-08T01:14:13.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff36199c700 1 -- 172.21.15.119:0/1531841055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c0c83f0 msgr2=0x7ff35c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:13.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff36199c700 1 --2- 172.21.15.119:0/1531841055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c0c83f0 0x7ff35c0c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff36199c700 1 -- 172.21.15.119:0/1531841055 >> v1:172.21.15.119:6789/0 conn(0x7ff35c0ccee0 legacy=0x7ff35c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff36199c700 1 -- 172.21.15.119:0/1531841055 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c15a7a0 con 0x7ff35c1568e0 2023-10-08T01:14:13.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff36119b700 1 --2- 172.21.15.119:0/1531841055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c0c83f0 0x7ff35c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:13.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff36199c700 1 --2- 172.21.15.119:0/1531841055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c1568e0 0x7ff35c0ca1e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff354009e60 tx=0x7ff354009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56748c792193e541 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:13.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff353fff700 1 -- 172.21.15.119:0/1531841055 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35400e070 con 0x7ff35c1568e0 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff353fff700 1 -- 172.21.15.119:0/1531841055 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35400bab0 con 0x7ff35c1568e0 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.768+0000 7ff353fff700 1 -- 172.21.15.119:0/1531841055 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35400be70 con 0x7ff35c1568e0 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.769+0000 7ff3633ff700 1 -- 172.21.15.119:0/1531841055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c1568e0 msgr2=0x7ff35c0ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.769+0000 7ff3633ff700 1 --2- 172.21.15.119:0/1531841055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c1568e0 0x7ff35c0ca1e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff354009e60 tx=0x7ff354009780 comp rx=0 tx=0).stop 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.769+0000 7ff3633ff700 1 -- 172.21.15.119:0/1531841055 shutdown_connections 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.769+0000 7ff3633ff700 1 --2- 172.21.15.119:0/1531841055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c1568e0 0x7ff35c0ca1e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.769+0000 7ff3633ff700 1 --2- 172.21.15.119:0/1531841055 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c0c83f0 0x7ff35c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.769+0000 7ff3633ff700 1 -- 172.21.15.119:0/1531841055 >> 172.21.15.119:0/1531841055 conn(0x7ff35c068e10 msgr2=0x7ff35c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.769+0000 7ff3633ff700 1 -- 172.21.15.119:0/1531841055 shutdown_connections 2023-10-08T01:14:13.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.769+0000 7ff3633ff700 1 -- 172.21.15.119:0/1531841055 wait complete. 2023-10-08T01:14:13.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.770+0000 7ff3633ff700 1 Processor -- start 2023-10-08T01:14:13.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.770+0000 7ff3633ff700 1 -- start start 2023-10-08T01:14:13.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff3633ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0c83f0 0x7ff35c0c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff3633ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c0ccee0 0x7ff35c147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff3633ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c1568e0 0x7ff35c1619c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff3633ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c160c10 con 0x7ff35c0c83f0 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff3633ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c0c5c60 con 0x7ff35c0ccee0 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff3633ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c1602e0 con 0x7ff35c1568e0 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff36119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0c83f0 0x7ff35c0c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff36119b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0c83f0 0x7ff35c0c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34984/0 (socket says 172.21.15.119:34984) 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.771+0000 7ff36119b700 1 -- 172.21.15.119:0/866818839 learned_addr learned my addr 172.21.15.119:0/866818839 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff36119b700 1 -- 172.21.15.119:0/866818839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c1568e0 msgr2=0x7ff35c1619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff36119b700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c1568e0 0x7ff35c1619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff36119b700 1 -- 172.21.15.119:0/866818839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c0ccee0 msgr2=0x7ff35c147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff36119b700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c0ccee0 0x7ff35c147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff36119b700 1 -- 172.21.15.119:0/866818839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff358009420 con 0x7ff35c0c83f0 2023-10-08T01:14:13.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff36119b700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0c83f0 0x7ff35c0c6110 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff35800ec80 tx=0x7ff3580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff3517fa700 1 -- 172.21.15.119:0/866818839 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35800c9d0 con 0x7ff35c0c83f0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff3517fa700 1 -- 172.21.15.119:0/866818839 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35800cb30 con 0x7ff35c0c83f0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff354009a20 con 0x7ff35c0c83f0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff3517fa700 1 -- 172.21.15.119:0/866818839 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35800ceb0 con 0x7ff35c0c83f0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.772+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c1608a0 con 0x7ff35c0c83f0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.773+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35c04c3c0 con 0x7ff35c0c83f0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.774+0000 7ff3517fa700 1 -- 172.21.15.119:0/866818839 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff358013680 con 0x7ff35c0c83f0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.775+0000 7ff3517fa700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff340066f30 0x7ff3400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.776+0000 7ff3517fa700 1 -- 172.21.15.119:0/866818839 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5933+0+0 (secure 0 0 0) 0x7ff358084ea0 con 0x7ff35c0c83f0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.776+0000 7ff36099a700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff340066f30 0x7ff3400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:13.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.776+0000 7ff36099a700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff340066f30 0x7ff3400693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0058d0 tx=0x7ff34c0079d0 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:13.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.780+0000 7ff3517fa700 1 -- 172.21.15.119:0/866818839 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff358051f90 con 0x7ff35c0c83f0 2023-10-08T01:14:13.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.970+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7ff35c15aa80 con 0x7ff35c0c83f0 2023-10-08T01:14:13.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.974+0000 7ff3517fa700 1 -- 172.21.15.119:0/866818839 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v39) v1 ==== 120+0+0 (secure 0 0 0) 0x7ff3580555b0 con 0x7ff35c0c83f0 2023-10-08T01:14:13.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff340066f30 msgr2=0x7ff3400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff340066f30 0x7ff3400693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0058d0 tx=0x7ff34c0079d0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0c83f0 msgr2=0x7ff35c0c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0c83f0 0x7ff35c0c6110 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff35800ec80 tx=0x7ff3580091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 shutdown_connections 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff340066f30 0x7ff3400693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff35c1568e0 0x7ff35c1619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0c83f0 0x7ff35c0c6110 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 --2- 172.21.15.119:0/866818839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff35c0ccee0 0x7ff35c147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 >> 172.21.15.119:0/866818839 conn(0x7ff35c068e10 msgr2=0x7ff35c0c94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 shutdown_connections 2023-10-08T01:14:13.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:13.977+0000 7ff3633ff700 1 -- 172.21.15.119:0/866818839 wait complete. 2023-10-08T01:14:13.996 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-10-08T01:14:14.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.175+0000 7f91383ff700 1 Processor -- start 2023-10-08T01:14:14.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.176+0000 7f91383ff700 1 -- start start 2023-10-08T01:14:14.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f91383ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91300b1850 0x7f91300b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f91383ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91301579f0 0x7f91300abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f91383ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91300cb300 con 0x7f91300ae680 2023-10-08T01:14:14.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f91383ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913006fc30 con 0x7f91300b1850 2023-10-08T01:14:14.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f91383ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91300cc260 con 0x7f91301579f0 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f913599a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f91300ae680 0x7f91301572e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47156/0 (socket says 172.21.15.119:47156) 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f913619b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91300b1850 0x7f91300b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f913599a700 1 -- 172.21.15.119:0/1845044145 learned_addr learned my addr 172.21.15.119:0/1845044145 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f913699c700 1 --2- 172.21.15.119:0/1845044145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91301579f0 0x7f91300abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.178+0000 7f913619b700 1 -- 172.21.15.119:0/1845044145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91301579f0 msgr2=0x7f91300abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.178+0000 7f913619b700 1 --2- 172.21.15.119:0/1845044145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91301579f0 0x7f91300abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.177+0000 7f9135199700 1 -- 172.21.15.119:0/1845044145 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833987484 0 0) 0x7f91300cb300 con 0x7f91300ae680 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.178+0000 7f913619b700 1 -- 172.21.15.119:0/1845044145 >> v1:172.21.15.119:6789/0 conn(0x7f91300ae680 legacy=0x7f91301572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.178+0000 7f913619b700 1 -- 172.21.15.119:0/1845044145 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91300cb5e0 con 0x7f91300b1850 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.178+0000 7f913619b700 1 --2- 172.21.15.119:0/1845044145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f91300b1c20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f912c009910 tx=0x7f912c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3735318badbbb266 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:14.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.178+0000 7f9135199700 1 -- 172.21.15.119:0/1845044145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00b820 con 0x7f91300b1850 2023-10-08T01:14:14.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.178+0000 7f9135199700 1 -- 172.21.15.119:0/1845044145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c00b980 con 0x7f91300b1850 2023-10-08T01:14:14.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.178+0000 7f9135199700 1 -- 172.21.15.119:0/1845044145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c00bd20 con 0x7f91300b1850 2023-10-08T01:14:14.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.179+0000 7f91383ff700 1 -- 172.21.15.119:0/1845044145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 msgr2=0x7f91300b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.179+0000 7f91383ff700 1 --2- 172.21.15.119:0/1845044145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f91300b1c20 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f912c009910 tx=0x7f912c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.179+0000 7f91383ff700 1 -- 172.21.15.119:0/1845044145 shutdown_connections 2023-10-08T01:14:14.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.179+0000 7f91383ff700 1 --2- 172.21.15.119:0/1845044145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91301579f0 0x7f91300abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.179+0000 7f91383ff700 1 --2- 172.21.15.119:0/1845044145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f91300b1c20 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.179+0000 7f91383ff700 1 -- 172.21.15.119:0/1845044145 >> 172.21.15.119:0/1845044145 conn(0x7f913006a320 msgr2=0x7f91300cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:14.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.179+0000 7f91383ff700 1 -- 172.21.15.119:0/1845044145 shutdown_connections 2023-10-08T01:14:14.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.179+0000 7f91383ff700 1 -- 172.21.15.119:0/1845044145 wait complete. 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.180+0000 7f91383ff700 1 Processor -- start 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.180+0000 7f91383ff700 1 -- start start 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f91383ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91300ae680 0x7f913016d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f91383ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f913015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f91383ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91301579f0 0x7f913015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f91383ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9130169dd0 con 0x7f91300b1850 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f91383ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913014c540 con 0x7f91301579f0 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f91383ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91300ab5b0 con 0x7f91300ae680 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f913599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f913015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f913599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f913015c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34998/0 (socket says 172.21.15.119:34998) 2023-10-08T01:14:14.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f913599a700 1 -- 172.21.15.119:0/242993519 learned_addr learned my addr 172.21.15.119:0/242993519 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.181+0000 7f913619b700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91300ae680 0x7f913016d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f913699c700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91301579f0 0x7f913015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f913599a700 1 -- 172.21.15.119:0/242993519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91300ae680 msgr2=0x7f913016d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f913599a700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91300ae680 0x7f913016d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f913599a700 1 -- 172.21.15.119:0/242993519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91301579f0 msgr2=0x7f913015c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f913599a700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91301579f0 0x7f913015c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f913599a700 1 -- 172.21.15.119:0/242993519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c009420 con 0x7f91300b1850 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f913599a700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f913015c260 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f912000c7e0 tx=0x7f9120007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f91267fc700 1 -- 172.21.15.119:0/242993519 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120005b30 con 0x7f91300b1850 2023-10-08T01:14:14.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f91267fc700 1 -- 172.21.15.119:0/242993519 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9120005c90 con 0x7f91300b1850 2023-10-08T01:14:14.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913014c7e0 con 0x7f91300b1850 2023-10-08T01:14:14.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f91267fc700 1 -- 172.21.15.119:0/242993519 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9120007300 con 0x7f91300b1850 2023-10-08T01:14:14.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.182+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91300ab1d0 con 0x7f91300b1850 2023-10-08T01:14:14.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.183+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9130169f40 con 0x7f91300b1850 2023-10-08T01:14:14.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.184+0000 7f91267fc700 1 -- 172.21.15.119:0/242993519 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f91200076b0 con 0x7f91300b1850 2023-10-08T01:14:14.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.185+0000 7f91267fc700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9114066f30 0x7f91140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.185+0000 7f91267fc700 1 -- 172.21.15.119:0/242993519 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f9120083b80 con 0x7f91300b1850 2023-10-08T01:14:14.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.185+0000 7f913619b700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9114066f30 0x7f91140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.186+0000 7f913619b700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9114066f30 0x7f91140693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f912c009860 tx=0x7f912c017000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:14.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.189+0000 7f91267fc700 1 -- 172.21.15.119:0/242993519 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9120050d10 con 0x7f91300b1850 2023-10-08T01:14:14.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.456+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f913004c3c0 con 0x7f91300b1850 2023-10-08T01:14:14.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.724+0000 7f91267fc700 1 -- 172.21.15.119:0/242993519 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v688) v1 ==== 183+0+0 (secure 0 0 0) 0x7f91200170f0 con 0x7f91300b1850 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.726+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9114066f30 msgr2=0x7f91140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.726+0000 7f91383ff700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9114066f30 0x7f91140693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f912c009860 tx=0x7f912c017000 comp rx=0 tx=0).stop 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.726+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 msgr2=0x7f913015c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.726+0000 7f91383ff700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f913015c260 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f912000c7e0 tx=0x7f9120007df0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.727+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 shutdown_connections 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.727+0000 7f91383ff700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9114066f30 0x7f91140693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.727+0000 7f91383ff700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91300ae680 0x7f913016d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.727+0000 7f91383ff700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91301579f0 0x7f913015c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.727+0000 7f91383ff700 1 --2- 172.21.15.119:0/242993519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91300b1850 0x7f913015c260 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.727+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 >> 172.21.15.119:0/242993519 conn(0x7f913006a320 msgr2=0x7f913006edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:14.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.727+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 shutdown_connections 2023-10-08T01:14:14.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.727+0000 7f91383ff700 1 -- 172.21.15.119:0/242993519 wait complete. 2023-10-08T01:14:14.744 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-10-08T01:14:14.744 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-10-08T01:14:14.745 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T01:14:14.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-08T01:14:14.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.925+0000 7f0a8ffff700 1 Processor -- start 2023-10-08T01:14:14.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.925+0000 7f0a8ffff700 1 -- start start 2023-10-08T01:14:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a9015de50 0x7f0a9006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a8ffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a900ca410 con 0x7f0a900696c0 2023-10-08T01:14:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a8ffff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a900c7220 con 0x7f0a9015da80 2023-10-08T01:14:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a8ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900c7500 con 0x7f0a9015de50 2023-10-08T01:14:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a8f7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0a900696c0 0x7f0a9006ec60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54724/0 (socket says 172.21.15.119:54724) 2023-10-08T01:14:14.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a8f7fe700 1 -- 172.21.15.119:0/4216384660 learned_addr learned my addr 172.21.15.119:0/4216384660 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a87fff700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a9015de50 0x7f0a9006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2703916570 0 0) 0x7f0a900ca410 con 0x7f0a900696c0 2023-10-08T01:14:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.927+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a7c003570 con 0x7f0a900696c0 2023-10-08T01:14:14.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 111293643 0 0) 0x7f0a7c003570 con 0x7f0a900696c0 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a9015de50 msgr2=0x7f0a9006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a9015de50 0x7f0a9006e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 >> v1:172.21.15.184:6789/0 conn(0x7f0a9015da80 legacy=0x7f0a900c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a900c9dc0 con 0x7f0a900696c0 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f0a880017f0 con 0x7f0a900696c0 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 >> v1:172.21.15.119:6789/0 conn(0x7f0a900696c0 legacy=0x7f0a9006ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c003a90 0x7f0a7c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a7c004810 0x7f0a7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a9015de50 0x7f0a7c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900c7500 con 0x7f0a7c003a90 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00bbf0 con 0x7f0a7c004810 2023-10-08T01:14:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c00bdb0 con 0x7f0a9015de50 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8effd700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c003a90 0x7f0a7c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a87fff700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a7c004810 0x7f0a7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.928+0000 7f0a8f7fe700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a9015de50 0x7f0a7c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8effd700 1 -- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a9015de50 msgr2=0x7f0a7c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8effd700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a9015de50 0x7f0a7c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8effd700 1 -- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a7c004810 msgr2=0x7f0a7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8effd700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a7c004810 0x7f0a7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8effd700 1 -- 172.21.15.119:0/4216384660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c003570 con 0x7f0a7c003a90 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8effd700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c003a90 0x7f0a7c003e60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0a800019a0 tx=0x7f0a80007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9281c066c0c91e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8000c5d0 con 0x7f0a7c003a90 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8000c730 con 0x7f0a7c003a90 2023-10-08T01:14:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.929+0000 7f0a8e7fc700 1 -- 172.21.15.119:0/4216384660 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a8000cad0 con 0x7f0a7c003a90 2023-10-08T01:14:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.930+0000 7f0a8ffff700 1 -- 172.21.15.119:0/4216384660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c003a90 msgr2=0x7f0a7c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.930+0000 7f0a8ffff700 1 --2- 172.21.15.119:0/4216384660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a7c003a90 0x7f0a7c003e60 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0a800019a0 tx=0x7f0a80007d10 comp rx=0 tx=0).stop 2023-10-08T01:14:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.930+0000 7f0a8f7fe700 1 -- 172.21.15.119:0/4216384660 reap_dead start 2023-10-08T01:14:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.930+0000 7f0a8ffff700 1 -- 172.21.15.119:0/4216384660 shutdown_connections 2023-10-08T01:14:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.930+0000 7f0a8ffff700 1 -- 172.21.15.119:0/4216384660 >> 172.21.15.119:0/4216384660 conn(0x7f0a900cc1f0 msgr2=0x7f0a900cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.930+0000 7f0a8ffff700 1 -- 172.21.15.119:0/4216384660 shutdown_connections 2023-10-08T01:14:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.930+0000 7f0a8ffff700 1 -- 172.21.15.119:0/4216384660 wait complete. 2023-10-08T01:14:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.931+0000 7f0a8ffff700 1 Processor -- start 2023-10-08T01:14:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.931+0000 7f0a8ffff700 1 -- start start 2023-10-08T01:14:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a7c003a90 0x7f0a9016e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90170150 0x7f0a90161a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a8ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a901705c0 0x7f0a90161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a8ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a900c4fb0 con 0x7f0a90170150 2023-10-08T01:14:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a8ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a9015ffd0 con 0x7f0a901705c0 2023-10-08T01:14:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a8ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a901602b0 con 0x7f0a7c003a90 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a87fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90170150 0x7f0a90161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a87fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90170150 0x7f0a90161a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48642/0 (socket says 172.21.15.119:48642) 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.932+0000 7f0a87fff700 1 -- 172.21.15.119:0/3114349002 learned_addr learned my addr 172.21.15.119:0/3114349002 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a87fff700 1 -- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a7c003a90 msgr2=0x7f0a9016e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a8effd700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a7c003a90 0x7f0a9016e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a8f7fe700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a901705c0 0x7f0a90161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a87fff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a7c003a90 0x7f0a9016e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a87fff700 1 -- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a901705c0 msgr2=0x7f0a90161fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:14.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a87fff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a901705c0 0x7f0a90161fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a87fff700 1 -- 172.21.15.119:0/3114349002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a900c4c40 con 0x7f0a90170150 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a8f7fe700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a901705c0 0x7f0a90161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a8effd700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a7c003a90 0x7f0a9016e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a87fff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90170150 0x7f0a90161a60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0a78011b40 tx=0x7f0a780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a86ffd700 1 -- 172.21.15.119:0/3114349002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7800fa70 con 0x7f0a90170150 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a86ffd700 1 -- 172.21.15.119:0/3114349002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7800fbd0 con 0x7f0a90170150 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a86ffd700 1 -- 172.21.15.119:0/3114349002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a78014440 con 0x7f0a90170150 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a900c4860 con 0x7f0a90170150 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.933+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a9015f4e0 con 0x7f0a90170150 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.934+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9004c3c0 con 0x7f0a90170150 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.935+0000 7f0a86ffd700 1 -- 172.21.15.119:0/3114349002 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0a78014870 con 0x7f0a90170150 2023-10-08T01:14:14.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.936+0000 7f0a86ffd700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0a70066f30 0x7f0a700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:14.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.936+0000 7f0a86ffd700 1 -- 172.21.15.119:0/3114349002 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f0a78018030 con 0x7f0a90170150 2023-10-08T01:14:14.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.936+0000 7f0a8effd700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0a70066f30 0x7f0a700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:14.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.937+0000 7f0a8effd700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0a70066f30 0x7f0a700693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0a80008fd0 tx=0x7f0a80015000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:14.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:14.940+0000 7f0a86ffd700 1 -- 172.21.15.119:0/3114349002 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0a78052210 con 0x7f0a90170150 2023-10-08T01:14:15.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.207+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f0a900ca0a0 con 0x7f0a90170150 2023-10-08T01:14:15.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.726+0000 7f0a86ffd700 1 -- 172.21.15.119:0/3114349002 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v689) v1 ==== 151+0+0 (secure 0 0 0) 0x7f0a780220d0 con 0x7f0a90170150 2023-10-08T01:14:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0a70066f30 msgr2=0x7f0a700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0a70066f30 0x7f0a700693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0a80008fd0 tx=0x7f0a80015000 comp rx=0 tx=0).stop 2023-10-08T01:14:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90170150 msgr2=0x7f0a90161a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90170150 0x7f0a90161a60 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f0a78011b40 tx=0x7f0a780091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 shutdown_connections 2023-10-08T01:14:15.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0a70066f30 0x7f0a700693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0a7c003a90 0x7f0a9016e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0a901705c0 0x7f0a90161fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 --2- 172.21.15.119:0/3114349002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0a90170150 0x7f0a90161a60 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 >> 172.21.15.119:0/3114349002 conn(0x7f0a900cc1f0 msgr2=0x7f0a90069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:15.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 shutdown_connections 2023-10-08T01:14:15.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.729+0000 7f0a8ffff700 1 -- 172.21.15.119:0/3114349002 wait complete. 2023-10-08T01:14:15.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T01:14:15.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.931+0000 7f1fa17ff700 1 Processor -- start 2023-10-08T01:14:15.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.931+0000 7f1fa17ff700 1 -- start start 2023-10-08T01:14:15.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.932+0000 7f1fa17ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1f9c069e80 0x7f1f9c0c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:15.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.932+0000 7f1fa17ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c0af2f0 con 0x7f1f9c15ac80 2023-10-08T01:14:15.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.932+0000 7f1fa17ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9c0af5d0 con 0x7f1f9c0b0a70 2023-10-08T01:14:15.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.932+0000 7f1fa17ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c0c8660 con 0x7f1f9c069e80 2023-10-08T01:14:15.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.932+0000 7f1f9affd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1f9c15ac80 0x7f1f9c069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54738/0 (socket says 172.21.15.119:54738) 2023-10-08T01:14:15.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.932+0000 7f1f9affd700 1 -- 172.21.15.119:0/1099094108 learned_addr learned my addr 172.21.15.119:0/1099094108 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:15.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f9a7fc700 1 --2- 172.21.15.119:0/1099094108 >> v2:172.21.15.184:3300/0 conn(0x7f1f9c069e80 0x7f1f9c0c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982300986 0 0) 0x7f1f9c0af2f0 con 0x7f1f9c15ac80 2023-10-08T01:14:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f84003570 con 0x7f1f9c15ac80 2023-10-08T01:14:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1996994527 0 0) 0x7f1f84003570 con 0x7f1f9c15ac80 2023-10-08T01:14:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 >> v2:172.21.15.184:3300/0 conn(0x7f1f9c069e80 msgr2=0x7f1f9c0c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 --2- 172.21.15.119:0/1099094108 >> v2:172.21.15.184:3300/0 conn(0x7f1f9c069e80 0x7f1f9c0c6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 >> v1:172.21.15.184:6789/0 conn(0x7f1f9c0b0a70 legacy=0x7f1f9c0c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0c8940 con 0x7f1f9c15ac80 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f1f8c001470 con 0x7f1f9c15ac80 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 >> v1:172.21.15.119:6789/0 conn(0x7f1f9c15ac80 legacy=0x7f1f9c069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f84003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f84004810 0x7f1f84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84005210 0x7f1f8400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0c8660 con 0x7f1f84003ab0 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8400c130 con 0x7f1f84005210 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.933+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8400c2f0 con 0x7f1f84004810 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9affd700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f84003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9a7fc700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f84004810 0x7f1f84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9b7fe700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84005210 0x7f1f8400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9affd700 1 -- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f84004810 msgr2=0x7f1f84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9affd700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f84004810 0x7f1f84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9affd700 1 -- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84005210 msgr2=0x7f1f8400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9affd700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84005210 0x7f1f8400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9affd700 1 -- 172.21.15.119:0/1099094108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f84003570 con 0x7f1f84003ab0 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9b7fe700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f84005210 0x7f1f8400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9b7fe700 1 -- 172.21.15.119:0/1099094108 reap_dead start 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f9affd700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f84003ec0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c2a0 tx=0x7f1f9c0af540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=553fca90c91a1ef5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00cb70 con 0x7f1f84003ab0 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c00ccd0 con 0x7f1f84003ab0 2023-10-08T01:14:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.934+0000 7f1f99ffb700 1 -- 172.21.15.119:0/1099094108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c005b70 con 0x7f1f84003ab0 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.935+0000 7f1fa17ff700 1 -- 172.21.15.119:0/1099094108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 msgr2=0x7f1f84003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.935+0000 7f1fa17ff700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f84003ec0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c2a0 tx=0x7f1f9c0af540 comp rx=0 tx=0).stop 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.935+0000 7f1fa17ff700 1 -- 172.21.15.119:0/1099094108 shutdown_connections 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.935+0000 7f1fa17ff700 1 --2- 172.21.15.119:0/1099094108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f84003ec0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.935+0000 7f1fa17ff700 1 -- 172.21.15.119:0/1099094108 >> 172.21.15.119:0/1099094108 conn(0x7f1f9c068e10 msgr2=0x7f1f9c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.935+0000 7f1fa17ff700 1 -- 172.21.15.119:0/1099094108 shutdown_connections 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.935+0000 7f1fa17ff700 1 -- 172.21.15.119:0/1099094108 wait complete. 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.937+0000 7f1fa17ff700 1 Processor -- start 2023-10-08T01:14:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.937+0000 7f1fa17ff700 1 -- start start 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1fa17ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f9c0c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1fa17ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c0ae1c0 0x7f1f9c1658e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1fa17ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9c0ae630 0x7f1f9c155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1fa17ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0ac970 con 0x7f1f84003ab0 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1fa17ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0c7900 con 0x7f1f9c0ae1c0 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1fa17ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0c7be0 con 0x7f1f9c0ae630 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f9c0c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f9c0c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48652/0 (socket says 172.21.15.119:48652) 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 -- 172.21.15.119:0/90085706 learned_addr learned my addr 172.21.15.119:0/90085706 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 -- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9c0ae630 msgr2=0x7f1f9c155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9b7fe700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9c0ae630 0x7f1f9c155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9a7fc700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c0ae1c0 0x7f1f9c1658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9c0ae630 0x7f1f9c155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 -- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c0ae1c0 msgr2=0x7f1f9c1658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c0ae1c0 0x7f1f9c1658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 -- 172.21.15.119:0/90085706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c0c7ec0 con 0x7f1f84003ab0 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9b7fe700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9c0ae630 0x7f1f9c155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9a7fc700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c0ae1c0 0x7f1f9c1658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.938+0000 7f1f9affd700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f9c0c7290 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c890 tx=0x7f1f8c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.939+0000 7f1f82ffd700 1 -- 172.21.15.119:0/90085706 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c022070 con 0x7f1f84003ab0 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.939+0000 7f1f82ffd700 1 -- 172.21.15.119:0/90085706 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c004e70 con 0x7f1f84003ab0 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.939+0000 7f1f82ffd700 1 -- 172.21.15.119:0/90085706 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c007510 con 0x7f1f84003ab0 2023-10-08T01:14:15.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.939+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c0c81a0 con 0x7f1f84003ab0 2023-10-08T01:14:15.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.939+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c155d50 con 0x7f1f84003ab0 2023-10-08T01:14:15.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.940+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c0acae0 con 0x7f1f84003ab0 2023-10-08T01:14:15.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.941+0000 7f1f82ffd700 1 -- 172.21.15.119:0/90085706 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1f8c01e430 con 0x7f1f84003ab0 2023-10-08T01:14:15.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.942+0000 7f1f82ffd700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1f88066f30 0x7f1f880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:15.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.942+0000 7f1f82ffd700 1 -- 172.21.15.119:0/90085706 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f1f8c085150 con 0x7f1f84003ab0 2023-10-08T01:14:15.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.942+0000 7f1f9a7fc700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1f88066f30 0x7f1f880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:15.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.943+0000 7f1f9a7fc700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1f88066f30 0x7f1f880693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f90006dc0 tx=0x7f1f90008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:15.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:15.946+0000 7f1f82ffd700 1 -- 172.21.15.119:0/90085706 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1f8c052340 con 0x7f1f84003ab0 2023-10-08T01:14:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.134+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1f9c04c3c0 con 0x7f1f84003ab0 2023-10-08T01:14:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.138+0000 7f1f82ffd700 1 -- 172.21.15.119:0/90085706 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1f8c00ce40 con 0x7f1f84003ab0 2023-10-08T01:14:16.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1f88066f30 msgr2=0x7f1f880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:16.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1f88066f30 0x7f1f880693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f1f90006dc0 tx=0x7f1f90008040 comp rx=0 tx=0).stop 2023-10-08T01:14:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 msgr2=0x7f1f9c0c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f9c0c7290 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00c890 tx=0x7f1f8c015000 comp rx=0 tx=0).stop 2023-10-08T01:14:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 shutdown_connections 2023-10-08T01:14:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1f88066f30 0x7f1f880693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1f9c0ae630 0x7f1f9c155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1f84003ab0 0x7f1f9c0c7290 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 --2- 172.21.15.119:0/90085706 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1f9c0ae1c0 0x7f1f9c1658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 >> 172.21.15.119:0/90085706 conn(0x7f1f9c068e10 msgr2=0x7f1f9c159da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 shutdown_connections 2023-10-08T01:14:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.141+0000 7f1fa17ff700 1 -- 172.21.15.119:0/90085706 wait complete. 2023-10-08T01:14:16.158 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:14:16.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:14:16.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:14:16.330 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:14:16.330 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T01:14:16.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:14:16.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.512+0000 7f3ba3fff700 1 Processor -- start 2023-10-08T01:14:16.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.513+0000 7f3ba3fff700 1 -- start start 2023-10-08T01:14:16.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba3fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3ba4150180 0x7f3ba4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:16.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ba4150550 0x7f3ba40c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:16.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 0x7f3ba4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:16.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba3fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40c76a0 con 0x7f3ba4150550 2023-10-08T01:14:16.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba3fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba406e2a0 con 0x7f3ba4150180 2023-10-08T01:14:16.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba406b2e0 con 0x7f3ba40c7240 2023-10-08T01:14:16.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba27fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ba4150550 0x7f3ba40c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:16.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba27fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3ba4150550 0x7f3ba40c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48656/0 (socket says 172.21.15.119:48656) 2023-10-08T01:14:16.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba27fc700 1 -- 172.21.15.119:0/75903252 learned_addr learned my addr 172.21.15.119:0/75903252 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:16.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba37fe700 1 --2- 172.21.15.119:0/75903252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 0x7f3ba4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:16.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.514+0000 7f3ba2ffd700 1 --2- 172.21.15.119:0/75903252 >> v2:172.21.15.184:3300/0 conn(0x7f3ba4150180 0x7f3ba4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:16.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba27fc700 1 -- 172.21.15.119:0/75903252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 msgr2=0x7f3ba4069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:16.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba27fc700 1 --2- 172.21.15.119:0/75903252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 0x7f3ba4069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba27fc700 1 -- 172.21.15.119:0/75903252 >> v2:172.21.15.184:3300/0 conn(0x7f3ba4150180 msgr2=0x7f3ba4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:16.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba27fc700 1 --2- 172.21.15.119:0/75903252 >> v2:172.21.15.184:3300/0 conn(0x7f3ba4150180 0x7f3ba4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba27fc700 1 -- 172.21.15.119:0/75903252 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40c78e0 con 0x7f3ba4150550 2023-10-08T01:14:16.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba27fc700 1 --2- 172.21.15.119:0/75903252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150550 0x7f3ba40c6d00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3ba406fd20 tx=0x7f3b98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e7db5e4649dc9c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:16.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba1ffb700 1 -- 172.21.15.119:0/75903252 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800b860 con 0x7f3ba4150550 2023-10-08T01:14:16.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba1ffb700 1 -- 172.21.15.119:0/75903252 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9800b9c0 con 0x7f3ba4150550 2023-10-08T01:14:16.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.515+0000 7f3ba1ffb700 1 -- 172.21.15.119:0/75903252 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800bd60 con 0x7f3ba4150550 2023-10-08T01:14:16.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 -- 172.21.15.119:0/75903252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150550 msgr2=0x7f3ba40c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:16.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/75903252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150550 0x7f3ba40c6d00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f3ba406fd20 tx=0x7f3b98009250 comp rx=0 tx=0).stop 2023-10-08T01:14:16.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 -- 172.21.15.119:0/75903252 shutdown_connections 2023-10-08T01:14:16.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/75903252 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 0x7f3ba4069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/75903252 >> v2:172.21.15.184:3300/0 conn(0x7f3ba4150180 0x7f3ba4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/75903252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150550 0x7f3ba40c6d00 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 -- 172.21.15.119:0/75903252 >> 172.21.15.119:0/75903252 conn(0x7f3ba40cc1f0 msgr2=0x7f3ba40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:16.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 -- 172.21.15.119:0/75903252 shutdown_connections 2023-10-08T01:14:16.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.516+0000 7f3ba3fff700 1 -- 172.21.15.119:0/75903252 wait complete. 2023-10-08T01:14:16.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.517+0000 7f3ba3fff700 1 Processor -- start 2023-10-08T01:14:16.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.518+0000 7f3ba3fff700 1 -- start start 2023-10-08T01:14:16.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.518+0000 7f3ba3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 0x7f3ba416a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:16.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.518+0000 7f3ba3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150180 0x7f3ba40af1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:16.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.518+0000 7f3ba3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba4150550 0x7f3ba40ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:16.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.518+0000 7f3ba3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba414b5f0 con 0x7f3ba4150180 2023-10-08T01:14:16.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.518+0000 7f3ba3fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba40af6e0 con 0x7f3ba4150550 2023-10-08T01:14:16.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.518+0000 7f3ba3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba416a790 con 0x7f3ba40c7240 2023-10-08T01:14:16.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150180 0x7f3ba40af1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:16.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150180 0x7f3ba40af1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48660/0 (socket says 172.21.15.119:48660) 2023-10-08T01:14:16.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 -- 172.21.15.119:0/2890239137 learned_addr learned my addr 172.21.15.119:0/2890239137 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:16.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba2ffd700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 0x7f3ba416a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:16.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba37fe700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba4150550 0x7f3ba40ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:16.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 -- 172.21.15.119:0/2890239137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 msgr2=0x7f3ba416a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:16.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 0x7f3ba416a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 -- 172.21.15.119:0/2890239137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba4150550 msgr2=0x7f3ba40ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:16.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba4150550 0x7f3ba40ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 -- 172.21.15.119:0/2890239137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980094f0 con 0x7f3ba4150180 2023-10-08T01:14:16.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.519+0000 7f3ba27fc700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150180 0x7f3ba40af1a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3ba406fe40 tx=0x7f3b98017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:16.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.520+0000 7f3b8affd700 1 -- 172.21.15.119:0/2890239137 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98024440 con 0x7f3ba4150180 2023-10-08T01:14:16.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.520+0000 7f3b8affd700 1 -- 172.21.15.119:0/2890239137 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b980245a0 con 0x7f3ba4150180 2023-10-08T01:14:16.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.520+0000 7f3b8affd700 1 -- 172.21.15.119:0/2890239137 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98024960 con 0x7f3ba4150180 2023-10-08T01:14:16.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.520+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba40af960 con 0x7f3ba4150180 2023-10-08T01:14:16.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.520+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba40affa0 con 0x7f3ba4150180 2023-10-08T01:14:16.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.521+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba416a950 con 0x7f3ba4150180 2023-10-08T01:14:16.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.522+0000 7f3b8affd700 1 -- 172.21.15.119:0/2890239137 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b9801a420 con 0x7f3ba4150180 2023-10-08T01:14:16.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.523+0000 7f3b8affd700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b90066f30 0x7f3b900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:16.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.523+0000 7f3b8affd700 1 -- 172.21.15.119:0/2890239137 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5933+0+0 (secure 0 0 0) 0x7f3b98086410 con 0x7f3ba4150180 2023-10-08T01:14:16.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.523+0000 7f3ba2ffd700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b90066f30 0x7f3b900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:16.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.523+0000 7f3ba2ffd700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b90066f30 0x7f3b900693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3b94006fd0 tx=0x7f3b94008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:16.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.527+0000 7f3b8affd700 1 -- 172.21.15.119:0/2890239137 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b98053500 con 0x7f3ba4150180 2023-10-08T01:14:16.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.765+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ba404c3c0 con 0x7f3ba4150180 2023-10-08T01:14:16.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.765+0000 7f3b8affd700 1 -- 172.21.15.119:0/2890239137 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f3b98015df0 con 0x7f3ba4150180 2023-10-08T01:14:16.766 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:16.766 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6745,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1682556092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1682556092},{"type":"v1","addr":"172.21.15.184:6835","nonce":1682556092}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6748,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/691036731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":691036731},{"type":"v1","addr":"172.21.15.184:6834","nonce":691036731}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7182,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3636556734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3636556734},{"type":"v1","addr":"172.21.15.119:6834","nonce":3636556734}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7566,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/4235340051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":4235340051},{"type":"v1","addr":"172.21.15.119:6835","nonce":4235340051}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":50}],"filesystems":[]} 2023-10-08T01:14:16.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b90066f30 msgr2=0x7f3b900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:16.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b90066f30 0x7f3b900693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3b94006fd0 tx=0x7f3b94008040 comp rx=0 tx=0).stop 2023-10-08T01:14:16.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150180 msgr2=0x7f3ba40af1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:16.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150180 0x7f3ba40af1a0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f3ba406fe40 tx=0x7f3b98017000 comp rx=0 tx=0).stop 2023-10-08T01:14:16.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 shutdown_connections 2023-10-08T01:14:16.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3b90066f30 0x7f3b900693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3ba40c7240 0x7f3ba416a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3ba4150550 0x7f3ba40ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 --2- 172.21.15.119:0/2890239137 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ba4150180 0x7f3ba40af1a0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.767+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 >> 172.21.15.119:0/2890239137 conn(0x7f3ba40cc1f0 msgr2=0x7f3ba4158a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.768+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 shutdown_connections 2023-10-08T01:14:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:16.768+0000 7f3ba3fff700 1 -- 172.21.15.119:0/2890239137 wait complete. 2023-10-08T01:14:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 50 2023-10-08T01:14:16.786 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T01:14:16.786 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T01:14:16.787 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T01:14:16.787 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:14:16.787 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T01:14:16.787 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T01:14:16.787 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T01:14:16.787 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:14:16.788 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T01:14:16.788 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T01:14:16.788 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T01:14:16.788 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:14:16.788 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T01:14:16.788 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T01:14:16.789 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T01:14:16.789 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:14:16.800 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T01:14:16.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T01:14:16.802 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T01:14:16.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T01:14:16.805 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T01:14:16.805 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T01:14:16.806 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T01:14:16.807 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T01:14:16.846 INFO:tasks.ceph.mds.a:Started 2023-10-08T01:14:16.846 DEBUG:teuthology.parallel:result is None 2023-10-08T01:14:16.848 INFO:tasks.ceph.mds.c:Started 2023-10-08T01:14:16.848 DEBUG:teuthology.parallel:result is None 2023-10-08T01:14:16.851 INFO:tasks.ceph.mds.b:Started 2023-10-08T01:14:16.851 DEBUG:teuthology.parallel:result is None 2023-10-08T01:14:16.854 INFO:tasks.ceph.mds.d:Started 2023-10-08T01:14:16.855 DEBUG:teuthology.parallel:result is None 2023-10-08T01:14:16.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T01:14:16.981 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:14:16.981 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T01:14:16.999 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T01:14:16.999 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T01:14:17.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.050+0000 7fa90979d700 1 Processor -- start 2023-10-08T01:14:17.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.050+0000 7fa90979d700 1 -- start start 2023-10-08T01:14:17.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa90979d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa9041597f0 0x7fa9040b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:17.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa90979d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa90415d230 0x7fa90415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:17.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa90979d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90415db70 con 0x7fa904159420 2023-10-08T01:14:17.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa90979d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040b11d0 con 0x7fa90415d230 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa90979d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9040b14b0 con 0x7fa9041597f0 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa908f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa90415d230 0x7fa90415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa908f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa90415d230 0x7fa90415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48720/0 (socket says 172.21.15.119:48720) 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa908f9c700 1 -- 172.21.15.119:0/3538237338 learned_addr learned my addr 172.21.15.119:0/3538237338 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.051+0000 7fa903fff700 1 --1- 172.21.15.119:0/3538237338 >> v1:172.21.15.119:6789/0 conn(0x7fa904159420 0x7fa9040b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54764/0 (socket says 172.21.15.119:54764) 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa9037fe700 1 --2- 172.21.15.119:0/3538237338 >> v2:172.21.15.184:3300/0 conn(0x7fa9041597f0 0x7fa9040b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa908f9c700 1 -- 172.21.15.119:0/3538237338 >> v2:172.21.15.184:3300/0 conn(0x7fa9041597f0 msgr2=0x7fa9040b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa908f9c700 1 --2- 172.21.15.119:0/3538237338 >> v2:172.21.15.184:3300/0 conn(0x7fa9041597f0 0x7fa9040b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa908f9c700 1 -- 172.21.15.119:0/3538237338 >> v1:172.21.15.119:6789/0 conn(0x7fa904159420 legacy=0x7fa9040b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa908f9c700 1 -- 172.21.15.119:0/3538237338 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040b1790 con 0x7fa90415d230 2023-10-08T01:14:17.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa902ffd700 1 -- 172.21.15.119:0/3538237338 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060517948 0 0) 0x7fa90415db70 con 0x7fa904159420 2023-10-08T01:14:17.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa908f9c700 1 --2- 172.21.15.119:0/3538237338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa90415d630 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa904089750 tx=0x7fa8fc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ae4672126e637f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:17.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa902ffd700 1 -- 172.21.15.119:0/3538237338 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc00e070 con 0x7fa90415d230 2023-10-08T01:14:17.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa902ffd700 1 -- 172.21.15.119:0/3538237338 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc00ba20 con 0x7fa90415d230 2023-10-08T01:14:17.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.052+0000 7fa902ffd700 1 -- 172.21.15.119:0/3538237338 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0092e0 con 0x7fa90415d230 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.053+0000 7fa90979d700 1 -- 172.21.15.119:0/3538237338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 msgr2=0x7fa90415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.053+0000 7fa90979d700 1 --2- 172.21.15.119:0/3538237338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa90415d630 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fa904089750 tx=0x7fa8fc009840 comp rx=0 tx=0).stop 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.053+0000 7fa90979d700 1 -- 172.21.15.119:0/3538237338 shutdown_connections 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.053+0000 7fa90979d700 1 --2- 172.21.15.119:0/3538237338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa90415d630 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.053+0000 7fa90979d700 1 --2- 172.21.15.119:0/3538237338 >> v2:172.21.15.184:3300/0 conn(0x7fa9041597f0 0x7fa9040b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.053+0000 7fa90979d700 1 -- 172.21.15.119:0/3538237338 >> 172.21.15.119:0/3538237338 conn(0x7fa9040d0660 msgr2=0x7fa9040d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.053+0000 7fa90979d700 1 -- 172.21.15.119:0/3538237338 shutdown_connections 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.053+0000 7fa90979d700 1 -- 172.21.15.119:0/3538237338 wait complete. 2023-10-08T01:14:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.054+0000 7fa90979d700 1 Processor -- start 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.055+0000 7fa90979d700 1 -- start start 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.055+0000 7fa90979d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa904159420 0x7fa904154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.055+0000 7fa90979d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9041597f0 0x7fa90415ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.055+0000 7fa90979d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa9040cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.055+0000 7fa90979d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90415c220 con 0x7fa90415d230 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.055+0000 7fa90979d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90415b610 con 0x7fa904159420 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.055+0000 7fa90979d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90415b8f0 con 0x7fa9041597f0 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa9040cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa9040cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48724/0 (socket says 172.21.15.119:48724) 2023-10-08T01:14:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 -- 172.21.15.119:0/3109356297 learned_addr learned my addr 172.21.15.119:0/3109356297 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 -- 172.21.15.119:0/3109356297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9041597f0 msgr2=0x7fa90415ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9041597f0 0x7fa90415ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa903fff700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa904159420 0x7fa904154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 -- 172.21.15.119:0/3109356297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa904159420 msgr2=0x7fa904154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa904159420 0x7fa904154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 -- 172.21.15.119:0/3109356297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc009ae0 con 0x7fa90415d230 2023-10-08T01:14:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa903fff700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa904159420 0x7fa904154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:17.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.056+0000 7fa908f9c700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa9040cf3e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009ab0 tx=0x7fa8fc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:17.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.057+0000 7fa8ebfff700 1 -- 172.21.15.119:0/3109356297 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc008c60 con 0x7fa90415d230 2023-10-08T01:14:17.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.057+0000 7fa8ebfff700 1 -- 172.21.15.119:0/3109356297 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8fc008dc0 con 0x7fa90415d230 2023-10-08T01:14:17.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.057+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa90415bbd0 con 0x7fa90415d230 2023-10-08T01:14:17.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.057+0000 7fa8ebfff700 1 -- 172.21.15.119:0/3109356297 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc0256b0 con 0x7fa90415d230 2023-10-08T01:14:17.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.057+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa90415cbc0 con 0x7fa90415d230 2023-10-08T01:14:17.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.058+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90415c3d0 con 0x7fa90415d230 2023-10-08T01:14:17.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.059+0000 7fa8ebfff700 1 -- 172.21.15.119:0/3109356297 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8fc003040 con 0x7fa90415d230 2023-10-08T01:14:17.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.059+0000 7fa8ebfff700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa8f0066f30 0x7fa8f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:17.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.060+0000 7fa8ebfff700 1 -- 172.21.15.119:0/3109356297 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5933+0+0 (secure 0 0 0) 0x7fa8fc086310 con 0x7fa90415d230 2023-10-08T01:14:17.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.060+0000 7fa903fff700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa8f0066f30 0x7fa8f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:17.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.060+0000 7fa903fff700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa8f0066f30 0x7fa8f00693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40018d0 tx=0x7fa8f4009040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:17.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.063+0000 7fa8ebfff700 1 -- 172.21.15.119:0/3109356297 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa90415c3d0 con 0x7fa90415d230 2023-10-08T01:14:17.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:17.278+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa90415c3d0 con 0x7fa90415d230 2023-10-08T01:14:17.879 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:14:17.884 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:14:18.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.163+0000 7fa8ebfff700 1 -- 172.21.15.119:0/3109356297 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v116) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa90415c3d0 con 0x7fa90415d230 2023-10-08T01:14:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa8f0066f30 msgr2=0x7fa8f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa8f0066f30 0x7fa8f00693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa8f40018d0 tx=0x7fa8f4009040 comp rx=0 tx=0).stop 2023-10-08T01:14:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 msgr2=0x7fa9040cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa9040cf3e0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc009ab0 tx=0x7fa8fc018000 comp rx=0 tx=0).stop 2023-10-08T01:14:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 shutdown_connections 2023-10-08T01:14:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa8f0066f30 0x7fa8f00693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa90415d230 0x7fa9040cf3e0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa904159420 0x7fa904154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 --2- 172.21.15.119:0/3109356297 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa9041597f0 0x7fa90415ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 >> 172.21.15.119:0/3109356297 conn(0x7fa9040d0660 msgr2=0x7fa904157190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 shutdown_connections 2023-10-08T01:14:18.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.166+0000 7fa90979d700 1 -- 172.21.15.119:0/3109356297 wait complete. 2023-10-08T01:14:18.168 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T01:14:18.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T01:14:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.367+0000 7f51977ff700 1 Processor -- start 2023-10-08T01:14:18.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.367+0000 7f51977ff700 1 -- start start 2023-10-08T01:14:18.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.368+0000 7f51977ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51900cbf30 0x7f51900ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.368+0000 7f51977ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f51900b1420 0x7f51900c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.368+0000 7f51977ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51900cc300 con 0x7f51900ccf90 2023-10-08T01:14:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.368+0000 7f51977ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5190156bf0 con 0x7f51900cbf30 2023-10-08T01:14:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.368+0000 7f51977ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51900ae950 con 0x7f51900b1420 2023-10-08T01:14:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f519559b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51900cbf30 0x7f51900ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f519559b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51900cbf30 0x7f51900ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48740/0 (socket says 172.21.15.119:48740) 2023-10-08T01:14:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f519559b700 1 -- 172.21.15.119:0/2411575050 learned_addr learned my addr 172.21.15.119:0/2411575050 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:18.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f5195d9c700 1 --2- 172.21.15.119:0/2411575050 >> v2:172.21.15.184:3300/0 conn(0x7f51900b1420 0x7f51900c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f519559b700 1 -- 172.21.15.119:0/2411575050 >> v2:172.21.15.184:3300/0 conn(0x7f51900b1420 msgr2=0x7f51900c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f519559b700 1 --2- 172.21.15.119:0/2411575050 >> v2:172.21.15.184:3300/0 conn(0x7f51900b1420 0x7f51900c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f519559b700 1 -- 172.21.15.119:0/2411575050 >> v1:172.21.15.184:6789/0 conn(0x7f51900ccf90 legacy=0x7f51900b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f519559b700 1 -- 172.21.15.119:0/2411575050 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900ad370 con 0x7f51900cbf30 2023-10-08T01:14:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f5195d9c700 1 --2- 172.21.15.119:0/2411575050 >> v2:172.21.15.184:3300/0 conn(0x7f51900b1420 0x7f51900c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.369+0000 7f519559b700 1 --2- 172.21.15.119:0/2411575050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900cbf30 0x7f51900ae410 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f51900b19e0 tx=0x7f518c00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c01baea3b50119 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f5187fff700 1 -- 172.21.15.119:0/2411575050 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00be60 con 0x7f51900cbf30 2023-10-08T01:14:18.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f5187fff700 1 -- 172.21.15.119:0/2411575050 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c00ddc0 con 0x7f51900cbf30 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f5187fff700 1 -- 172.21.15.119:0/2411575050 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0196b0 con 0x7f51900cbf30 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f51977ff700 1 -- 172.21.15.119:0/2411575050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900cbf30 msgr2=0x7f51900ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f51977ff700 1 --2- 172.21.15.119:0/2411575050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900cbf30 0x7f51900ae410 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f51900b19e0 tx=0x7f518c00b4f0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f51977ff700 1 -- 172.21.15.119:0/2411575050 shutdown_connections 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f51977ff700 1 --2- 172.21.15.119:0/2411575050 >> v2:172.21.15.184:3300/0 conn(0x7f51900b1420 0x7f51900c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f51977ff700 1 --2- 172.21.15.119:0/2411575050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900cbf30 0x7f51900ae410 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f51977ff700 1 -- 172.21.15.119:0/2411575050 >> 172.21.15.119:0/2411575050 conn(0x7f519006a320 msgr2=0x7f51900cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f51977ff700 1 -- 172.21.15.119:0/2411575050 shutdown_connections 2023-10-08T01:14:18.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.370+0000 7f51977ff700 1 -- 172.21.15.119:0/2411575050 wait complete. 2023-10-08T01:14:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.372+0000 7f51977ff700 1 Processor -- start 2023-10-08T01:14:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.372+0000 7f51977ff700 1 -- start start 2023-10-08T01:14:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f51977ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51900b1420 0x7f51901406c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f51977ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51900cbf30 0x7f5190158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f51977ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900ccf90 0x7f51901591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f51977ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5190159730 con 0x7f51900ccf90 2023-10-08T01:14:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f51977ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51900a9b70 con 0x7f51900cbf30 2023-10-08T01:14:18.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f51977ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51900a6200 con 0x7f51900b1420 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5195d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900ccf90 0x7f51901591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5195d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900ccf90 0x7f51901591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48744/0 (socket says 172.21.15.119:48744) 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5195d9c700 1 -- 172.21.15.119:0/303960687 learned_addr learned my addr 172.21.15.119:0/303960687 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f519559b700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51900b1420 0x7f51901406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5194d9a700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51900cbf30 0x7f5190158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5195d9c700 1 -- 172.21.15.119:0/303960687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51900b1420 msgr2=0x7f51901406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5195d9c700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51900b1420 0x7f51901406c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5195d9c700 1 -- 172.21.15.119:0/303960687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51900cbf30 msgr2=0x7f5190158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5195d9c700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51900cbf30 0x7f5190158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.373+0000 7f5195d9c700 1 -- 172.21.15.119:0/303960687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c00b790 con 0x7f51900ccf90 2023-10-08T01:14:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.374+0000 7f5195d9c700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900ccf90 0x7f51901591f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f518800d180 tx=0x7f5188009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.374+0000 7f51857fa700 1 -- 172.21.15.119:0/303960687 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518800dd70 con 0x7f51900ccf90 2023-10-08T01:14:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.374+0000 7f51857fa700 1 -- 172.21.15.119:0/303960687 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518800f040 con 0x7f51900ccf90 2023-10-08T01:14:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.374+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51900a55f0 con 0x7f51900ccf90 2023-10-08T01:14:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.374+0000 7f51857fa700 1 -- 172.21.15.119:0/303960687 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188015690 con 0x7f51900ccf90 2023-10-08T01:14:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.374+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51900a5bb0 con 0x7f51900ccf90 2023-10-08T01:14:18.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.375+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51900a63b0 con 0x7f51900ccf90 2023-10-08T01:14:18.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.376+0000 7f51857fa700 1 -- 172.21.15.119:0/303960687 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51880143a0 con 0x7f51900ccf90 2023-10-08T01:14:18.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.377+0000 7f51857fa700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5174066f60 0x7f5174069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.377+0000 7f51857fa700 1 -- 172.21.15.119:0/303960687 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f5188084590 con 0x7f51900ccf90 2023-10-08T01:14:18.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.377+0000 7f519559b700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5174066f60 0x7f5174069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.378+0000 7f519559b700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5174066f60 0x7f5174069410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f51900b19e0 tx=0x7f518c01b000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:18.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.381+0000 7f51857fa700 1 -- 172.21.15.119:0/303960687 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51900a63b0 con 0x7f51900ccf90 2023-10-08T01:14:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.647+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f51900a6370 con 0x7f51900ccf90 2023-10-08T01:14:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.647+0000 7f51857fa700 1 -- 172.21.15.119:0/303960687 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5188051930 con 0x7f51900ccf90 2023-10-08T01:14:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:18.648 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:14:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T01:14:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.648 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T01:14:18.649 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:14:18.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:14:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.652 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T01:14:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T01:14:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:14:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:14:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T01:14:18.655 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.656 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:14:18.657 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:14:18.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:14:18.659 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.660 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T01:14:18.663 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T01:14:18.664 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T01:14:18.665 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T01:14:18.666 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:14:18.667 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T01:14:18.668 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.649+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5174066f60 msgr2=0x7f5174069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.649+0000 7f51977ff700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5174066f60 0x7f5174069410 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f51900b19e0 tx=0x7f518c01b000 comp rx=0 tx=0).stop 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.649+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900ccf90 msgr2=0x7f51901591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.649+0000 7f51977ff700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900ccf90 0x7f51901591f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f518800d180 tx=0x7f5188009a50 comp rx=0 tx=0).stop 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.650+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 shutdown_connections 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.650+0000 7f51977ff700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5174066f60 0x7f5174069410 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.650+0000 7f51977ff700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51900b1420 0x7f51901406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.650+0000 7f51977ff700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51900ccf90 0x7f51901591f0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.650+0000 7f51977ff700 1 --2- 172.21.15.119:0/303960687 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f51900cbf30 0x7f5190158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.650+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 >> 172.21.15.119:0/303960687 conn(0x7f519006a320 msgr2=0x7f51900ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.650+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 shutdown_connections 2023-10-08T01:14:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.650+0000 7f51977ff700 1 -- 172.21.15.119:0/303960687 wait complete. 2023-10-08T01:14:18.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:14:18.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.852+0000 7fbcc29ff700 1 Processor -- start 2023-10-08T01:14:18.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.852+0000 7fbcc29ff700 1 -- start start 2023-10-08T01:14:18.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.853+0000 7fbcc29ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbcbc06bcd0 0x7fbcbc0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.853+0000 7fbcc29ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbcbc06c0a0 0x7fbcbc15a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.853+0000 7fbcc29ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc15e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.853+0000 7fbcc29ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc15d7e0 con 0x7fbcbc06c0a0 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.853+0000 7fbcc29ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc15dac0 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.853+0000 7fbcc29ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc15aa30 con 0x7fbcbc15df50 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbcbc06c0a0 0x7fbcbc15a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbcbc06c0a0 0x7fbcbc15a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48752/0 (socket says 172.21.15.119:48752) 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 -- 172.21.15.119:0/3358487710 learned_addr learned my addr 172.21.15.119:0/3358487710 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcc0f9c700 1 --2- 172.21.15.119:0/3358487710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc15e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbbfff700 1 --2- 172.21.15.119:0/3358487710 >> v2:172.21.15.184:3300/0 conn(0x7fbcbc06bcd0 0x7fbcbc0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 -- 172.21.15.119:0/3358487710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 msgr2=0x7fbcbc15e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 --2- 172.21.15.119:0/3358487710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc15e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 -- 172.21.15.119:0/3358487710 >> v2:172.21.15.184:3300/0 conn(0x7fbcbc06bcd0 msgr2=0x7fbcbc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 --2- 172.21.15.119:0/3358487710 >> v2:172.21.15.184:3300/0 conn(0x7fbcbc06bcd0 0x7fbcbc0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 -- 172.21.15.119:0/3358487710 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc15ad10 con 0x7fbcbc06c0a0 2023-10-08T01:14:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.854+0000 7fbcbb7fe700 1 --2- 172.21.15.119:0/3358487710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06c0a0 0x7fbcbc15a1e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0691a0 tx=0x7fbcb0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73847902ae3900f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcbaffd700 1 -- 172.21.15.119:0/3358487710 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000b640 con 0x7fbcbc06c0a0 2023-10-08T01:14:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcbaffd700 1 -- 172.21.15.119:0/3358487710 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb000b7a0 con 0x7fbcbc06c0a0 2023-10-08T01:14:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcbaffd700 1 -- 172.21.15.119:0/3358487710 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000bb40 con 0x7fbcbc06c0a0 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 -- 172.21.15.119:0/3358487710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06c0a0 msgr2=0x7fbcbc15a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/3358487710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06c0a0 0x7fbcbc15a1e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0691a0 tx=0x7fbcb0009250 comp rx=0 tx=0).stop 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 -- 172.21.15.119:0/3358487710 shutdown_connections 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/3358487710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc15e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/3358487710 >> v2:172.21.15.184:3300/0 conn(0x7fbcbc06bcd0 0x7fbcbc0caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/3358487710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06c0a0 0x7fbcbc15a1e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 -- 172.21.15.119:0/3358487710 >> 172.21.15.119:0/3358487710 conn(0x7fbcbc0ce600 msgr2=0x7fbcbc0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 -- 172.21.15.119:0/3358487710 shutdown_connections 2023-10-08T01:14:18.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.855+0000 7fbcc29ff700 1 -- 172.21.15.119:0/3358487710 wait complete. 2023-10-08T01:14:18.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.857+0000 7fbcc29ff700 1 Processor -- start 2023-10-08T01:14:18.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.857+0000 7fbcc29ff700 1 -- start start 2023-10-08T01:14:18.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcc29ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06bcd0 0x7fbcbc0c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcc29ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc06c0a0 0x7fbcbc0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcc29ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc0c45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcc29ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc16e130 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcc29ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc169d70 con 0x7fbcbc06c0a0 2023-10-08T01:14:18.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcc29ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc16a050 con 0x7fbcbc15df50 2023-10-08T01:14:18.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06bcd0 0x7fbcbc0c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06bcd0 0x7fbcbc0c5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48760/0 (socket says 172.21.15.119:48760) 2023-10-08T01:14:18.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbbfff700 1 -- 172.21.15.119:0/242502792 learned_addr learned my addr 172.21.15.119:0/242502792 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:18.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbb7fe700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc06c0a0 0x7fbcbc0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcc0f9c700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc0c45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbbfff700 1 -- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 msgr2=0x7fbcbc0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbbfff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc0c45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbbfff700 1 -- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc06c0a0 msgr2=0x7fbcbc0c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:18.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbbfff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc06c0a0 0x7fbcbc0c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:18.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbbfff700 1 -- 172.21.15.119:0/242502792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb00094f0 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcc0f9c700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:18.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.858+0000 7fbcbb7fe700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc06c0a0 0x7fbcbc0c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:18.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.859+0000 7fbcbbfff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06bcd0 0x7fbcbc0c5870 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00e930 tx=0x7fbcac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:18.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.859+0000 7fbca3fff700 1 -- 172.21.15.119:0/242502792 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00eed0 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.859+0000 7fbca3fff700 1 -- 172.21.15.119:0/242502792 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac00c850 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.859+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc16a330 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.859+0000 7fbca3fff700 1 -- 172.21.15.119:0/242502792 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac00cbf0 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.859+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc16ab30 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.860+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc16a980 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.861+0000 7fbca3fff700 1 -- 172.21.15.119:0/242502792 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcac013470 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.862+0000 7fbca3fff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbca8066f30 0x7fbca80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.862+0000 7fbca3fff700 1 -- 172.21.15.119:0/242502792 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fbcac084630 con 0x7fbcbc06bcd0 2023-10-08T01:14:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.862+0000 7fbcbb7fe700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbca8066f30 0x7fbca80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.863+0000 7fbcbb7fe700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbca8066f30 0x7fbca80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0691a0 tx=0x7fbcb0017000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:18.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:18.866+0000 7fbca3fff700 1 -- 172.21.15.119:0/242502792 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcac0519d0 con 0x7fbcbc06bcd0 2023-10-08T01:14:19.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.057+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcbc04c3c0 con 0x7fbcbc06bcd0 2023-10-08T01:14:19.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.058+0000 7fbca3fff700 1 -- 172.21.15.119:0/242502792 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+12789 (secure 0 0 0) 0x7fbcac054ff0 con 0x7fbcbc06bcd0 2023-10-08T01:14:19.058 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:19.058 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":116,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:18.159596+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:14:19.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbca8066f30 msgr2=0x7fbca80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbca8066f30 0x7fbca80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0691a0 tx=0x7fbcb0017000 comp rx=0 tx=0).stop 2023-10-08T01:14:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06bcd0 msgr2=0x7fbcbc0c5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06bcd0 0x7fbcbc0c5870 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00e930 tx=0x7fbcac0091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 shutdown_connections 2023-10-08T01:14:19.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbca8066f30 0x7fbca80693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbcbc15df50 0x7fbcbc0c45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbcbc06bcd0 0x7fbcbc0c5870 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 --2- 172.21.15.119:0/242502792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbcbc06c0a0 0x7fbcbc0c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 >> 172.21.15.119:0/242502792 conn(0x7fbcbc0ce600 msgr2=0x7fbcbc156160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:19.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 shutdown_connections 2023-10-08T01:14:19.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.060+0000 7fbcc29ff700 1 -- 172.21.15.119:0/242502792 wait complete. 2023-10-08T01:14:19.079 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T01:14:19.080 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:14:19.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.264+0000 7f7373fff700 1 Processor -- start 2023-10-08T01:14:19.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.265+0000 7f7373fff700 1 -- start start 2023-10-08T01:14:19.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.266+0000 7f7373fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f737406f060 0x7f73740ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.266+0000 7f7373fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737406ebd0 con 0x7f73740cb1a0 2023-10-08T01:14:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.266+0000 7f7373fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f737415aa70 con 0x7f737406f4b0 2023-10-08T01:14:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.266+0000 7f7373fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f737415b2e0 con 0x7f737406f060 2023-10-08T01:14:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.266+0000 7f7372ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f73740cb1a0 0x7f737415a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54766/0 (socket says 172.21.15.119:54766) 2023-10-08T01:14:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.266+0000 7f7372ffd700 1 -- 172.21.15.119:0/1925761214 learned_addr learned my addr 172.21.15.119:0/1925761214 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411601720 0 0) 0x7f737406ebd0 con 0x7f73740cb1a0 2023-10-08T01:14:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7360003570 con 0x7f73740cb1a0 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f736bfff700 1 --2- 172.21.15.119:0/1925761214 >> v2:172.21.15.184:3300/0 conn(0x7f737406f060 0x7f73740ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 270613315 0 0) 0x7f7360003570 con 0x7f73740cb1a0 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 >> v2:172.21.15.184:3300/0 conn(0x7f737406f060 msgr2=0x7f73740ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 --2- 172.21.15.119:0/1925761214 >> v2:172.21.15.184:3300/0 conn(0x7f737406f060 0x7f73740ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 >> v1:172.21.15.184:6789/0 conn(0x7f737406f4b0 legacy=0x7f73740c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737415ad20 con 0x7f73740cb1a0 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f7364001470 con 0x7f73740cb1a0 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 >> v1:172.21.15.119:6789/0 conn(0x7f73740cb1a0 legacy=0x7f737415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7360003b30 0x7f7360003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360004810 0x7f7360004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73600052e0 0x7f736000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737415b2e0 con 0x7f7360004810 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f736000e3a0 con 0x7f7360003b30 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.267+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f736000e560 con 0x7f73600052e0 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f736bfff700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360004810 0x7f7360004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f7372ffd700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7360003b30 0x7f7360003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f73737fe700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73600052e0 0x7f736000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f736bfff700 1 -- 172.21.15.119:0/1925761214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73600052e0 msgr2=0x7f736000dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:19.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f736bfff700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73600052e0 0x7f736000dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f736bfff700 1 -- 172.21.15.119:0/1925761214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7360003b30 msgr2=0x7f7360003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f736bfff700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7360003b30 0x7f7360003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f736bfff700 1 -- 172.21.15.119:0/1925761214 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7360003570 con 0x7f7360004810 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f73737fe700 1 -- 172.21.15.119:0/1925761214 reap_dead start 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f736bfff700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360004810 0x7f7360004c80 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f735c009c00 tx=0x7f735c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6321c3baeeecb156 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00e710 con 0x7f7360004810 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c00e870 con 0x7f7360004810 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.268+0000 7f73727fc700 1 -- 172.21.15.119:0/1925761214 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00ec30 con 0x7f7360004810 2023-10-08T01:14:19.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.269+0000 7f7373fff700 1 -- 172.21.15.119:0/1925761214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360004810 msgr2=0x7f7360004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.269+0000 7f7373fff700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360004810 0x7f7360004c80 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f735c009c00 tx=0x7f735c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.269+0000 7f7373fff700 1 -- 172.21.15.119:0/1925761214 shutdown_connections 2023-10-08T01:14:19.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.269+0000 7f7373fff700 1 --2- 172.21.15.119:0/1925761214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7360004810 0x7f7360004c80 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.269+0000 7f7373fff700 1 -- 172.21.15.119:0/1925761214 >> 172.21.15.119:0/1925761214 conn(0x7f73740cc1e0 msgr2=0x7f73740cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:19.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.269+0000 7f7373fff700 1 -- 172.21.15.119:0/1925761214 shutdown_connections 2023-10-08T01:14:19.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.269+0000 7f7373fff700 1 -- 172.21.15.119:0/1925761214 wait complete. 2023-10-08T01:14:19.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.270+0000 7f7373fff700 1 Processor -- start 2023-10-08T01:14:19.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.271+0000 7f7373fff700 1 -- start start 2023-10-08T01:14:19.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.271+0000 7f7373fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7360004810 0x7f737414e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.271+0000 7f7373fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7374154fc0 0x7f7374159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.271+0000 7f7373fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73740c0f80 0x7f73740c1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.271+0000 7f7373fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73740c1960 con 0x7f7374154fc0 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.271+0000 7f7373fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7374156100 con 0x7f73740c0f80 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.271+0000 7f7373fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73741554f0 con 0x7f7360004810 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7374154fc0 0x7f7374159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7374154fc0 0x7f7374159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48790/0 (socket says 172.21.15.119:48790) 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 -- 172.21.15.119:0/137348098 learned_addr learned my addr 172.21.15.119:0/137348098 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f73737fe700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73740c0f80 0x7f73740c1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f7372ffd700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7360004810 0x7f737414e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 -- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7360004810 msgr2=0x7f737414e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7360004810 0x7f737414e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 -- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73740c0f80 msgr2=0x7f73740c1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73740c0f80 0x7f73740c1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 -- 172.21.15.119:0/137348098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73741557d0 con 0x7f7374154fc0 2023-10-08T01:14:19.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f73737fe700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73740c0f80 0x7f73740c1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:19.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f7372ffd700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7360004810 0x7f737414e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736bfff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7374154fc0 0x7f7374159560 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f736000e370 tx=0x7f735c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736affd700 1 -- 172.21.15.119:0/137348098 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c022070 con 0x7f7374154fc0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f736affd700 1 -- 172.21.15.119:0/137348098 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c00e9e0 con 0x7f7374154fc0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.272+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7374155ab0 con 0x7f7374154fc0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.273+0000 7f736affd700 1 -- 172.21.15.119:0/137348098 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c01c450 con 0x7f7374154fc0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.273+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737414e720 con 0x7f7374154fc0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.274+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7374156d90 con 0x7f7374154fc0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.274+0000 7f736affd700 1 -- 172.21.15.119:0/137348098 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f735c01c840 con 0x7f7374154fc0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.275+0000 7f736affd700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7354066f30 0x7f73540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.275+0000 7f736affd700 1 -- 172.21.15.119:0/137348098 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f735c085b20 con 0x7f7374154fc0 2023-10-08T01:14:19.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.275+0000 7f7372ffd700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7354066f30 0x7f73540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.276+0000 7f7372ffd700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7354066f30 0x7f73540693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f73640015a0 tx=0x7f73740c8be0 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:19.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.279+0000 7f736affd700 1 -- 172.21.15.119:0/137348098 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7374156d90 con 0x7f7374154fc0 2023-10-08T01:14:19.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.473+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7374156d90 con 0x7f7374154fc0 2023-10-08T01:14:19.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.473+0000 7f736affd700 1 -- 172.21.15.119:0/137348098 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7374156d90 con 0x7f7374154fc0 2023-10-08T01:14:19.474 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:14:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.475+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7354066f30 msgr2=0x7f73540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.475+0000 7f7373fff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7354066f30 0x7f73540693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f73640015a0 tx=0x7f73740c8be0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.475+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7374154fc0 msgr2=0x7f7374159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.475+0000 7f7373fff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7374154fc0 0x7f7374159560 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f736000e370 tx=0x7f735c01a000 comp rx=0 tx=0).stop 2023-10-08T01:14:19.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.476+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 shutdown_connections 2023-10-08T01:14:19.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.476+0000 7f7373fff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7354066f30 0x7f73540693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.476+0000 7f7373fff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7360004810 0x7f737414e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.476+0000 7f7373fff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73740c0f80 0x7f73740c1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.476+0000 7f7373fff700 1 --2- 172.21.15.119:0/137348098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7374154fc0 0x7f7374159560 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.476+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 >> 172.21.15.119:0/137348098 conn(0x7f73740cc1e0 msgr2=0x7f73740c20c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:19.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.476+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 shutdown_connections 2023-10-08T01:14:19.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.476+0000 7f7373fff700 1 -- 172.21.15.119:0/137348098 wait complete. 2023-10-08T01:14:19.493 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T01:14:19.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.673+0000 7f9d6339d700 1 Processor -- start 2023-10-08T01:14:19.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.674+0000 7f9d6339d700 1 -- start start 2023-10-08T01:14:19.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d6339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c157770 0x7f9d5c150180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d6339d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9d5c0c6d00 0x7f9d5c0c7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d6339d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d5c06e730 0x7f9d5c06ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d6339d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c0c76b0 con 0x7f9d5c06e730 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d6339d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c06e2a0 con 0x7f9d5c0c6d00 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d6339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c157b40 con 0x7f9d5c157770 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d62b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d5c06e730 0x7f9d5c06ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d62b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d5c06e730 0x7f9d5c06ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48794/0 (socket says 172.21.15.119:48794) 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d62b9c700 1 -- 172.21.15.119:0/1765882389 learned_addr learned my addr 172.21.15.119:0/1765882389 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d61b9a700 1 --2- 172.21.15.119:0/1765882389 >> v2:172.21.15.184:3300/0 conn(0x7f9d5c0c6d00 0x7f9d5c0c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.675+0000 7f9d6239b700 1 --2- 172.21.15.119:0/1765882389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c157770 0x7f9d5c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d62b9c700 1 -- 172.21.15.119:0/1765882389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c157770 msgr2=0x7f9d5c150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d62b9c700 1 --2- 172.21.15.119:0/1765882389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c157770 0x7f9d5c150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d62b9c700 1 -- 172.21.15.119:0/1765882389 >> v2:172.21.15.184:3300/0 conn(0x7f9d5c0c6d00 msgr2=0x7f9d5c0c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:19.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d62b9c700 1 --2- 172.21.15.119:0/1765882389 >> v2:172.21.15.184:3300/0 conn(0x7f9d5c0c6d00 0x7f9d5c0c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d62b9c700 1 -- 172.21.15.119:0/1765882389 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c06b2e0 con 0x7f9d5c06e730 2023-10-08T01:14:19.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d62b9c700 1 --2- 172.21.15.119:0/1765882389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c06e730 0x7f9d5c06ebb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c06c030 tx=0x7f9d54009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c044b80ac17402 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:19.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d61399700 1 -- 172.21.15.119:0/1765882389 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400e070 con 0x7f9d5c06e730 2023-10-08T01:14:19.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d61399700 1 -- 172.21.15.119:0/1765882389 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5400b970 con 0x7f9d5c06e730 2023-10-08T01:14:19.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.676+0000 7f9d61399700 1 -- 172.21.15.119:0/1765882389 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d540092e0 con 0x7f9d5c06e730 2023-10-08T01:14:19.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 -- 172.21.15.119:0/1765882389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c06e730 msgr2=0x7f9d5c06ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1765882389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c06e730 0x7f9d5c06ebb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c06c030 tx=0x7f9d54009840 comp rx=0 tx=0).stop 2023-10-08T01:14:19.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 -- 172.21.15.119:0/1765882389 shutdown_connections 2023-10-08T01:14:19.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1765882389 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c06e730 0x7f9d5c06ebb0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1765882389 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c157770 0x7f9d5c150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1765882389 >> v2:172.21.15.184:3300/0 conn(0x7f9d5c0c6d00 0x7f9d5c0c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 -- 172.21.15.119:0/1765882389 >> 172.21.15.119:0/1765882389 conn(0x7f9d5c0cc1f0 msgr2=0x7f9d5c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 -- 172.21.15.119:0/1765882389 shutdown_connections 2023-10-08T01:14:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.677+0000 7f9d6339d700 1 -- 172.21.15.119:0/1765882389 wait complete. 2023-10-08T01:14:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.679+0000 7f9d6339d700 1 Processor -- start 2023-10-08T01:14:19.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.679+0000 7f9d6339d700 1 -- start start 2023-10-08T01:14:19.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d6339d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c06e730 0x7f9d5c14e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d6339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c0c6d00 0x7f9d5c0c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d6339d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c157770 0x7f9d5c0c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d6339d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0c6520 con 0x7f9d5c157770 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d6339d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c14eed0 con 0x7f9d5c06e730 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d6339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0abeb0 con 0x7f9d5c0c6d00 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d62b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c157770 0x7f9d5c0c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d62b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c157770 0x7f9d5c0c5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48796/0 (socket says 172.21.15.119:48796) 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d62b9c700 1 -- 172.21.15.119:0/1451722167 learned_addr learned my addr 172.21.15.119:0/1451722167 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d61b9a700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c0c6d00 0x7f9d5c0c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d6239b700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c06e730 0x7f9d5c14e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d62b9c700 1 -- 172.21.15.119:0/1451722167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c0c6d00 msgr2=0x7f9d5c0c4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d62b9c700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c0c6d00 0x7f9d5c0c4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d62b9c700 1 -- 172.21.15.119:0/1451722167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c06e730 msgr2=0x7f9d5c14e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d62b9c700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c06e730 0x7f9d5c14e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.680+0000 7f9d62b9c700 1 -- 172.21.15.119:0/1451722167 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d54009ae0 con 0x7f9d5c157770 2023-10-08T01:14:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.681+0000 7f9d62b9c700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c157770 0x7f9d5c0c5140 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009880 tx=0x7f9d54018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.681+0000 7f9d527fc700 1 -- 172.21.15.119:0/1451722167 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d540032f0 con 0x7f9d5c157770 2023-10-08T01:14:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.681+0000 7f9d527fc700 1 -- 172.21.15.119:0/1451722167 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d54008bc0 con 0x7f9d5c157770 2023-10-08T01:14:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.681+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5c0c5f60 con 0x7f9d5c157770 2023-10-08T01:14:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.681+0000 7f9d527fc700 1 -- 172.21.15.119:0/1451722167 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d54025460 con 0x7f9d5c157770 2023-10-08T01:14:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.681+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c0b1ac0 con 0x7f9d5c157770 2023-10-08T01:14:19.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.682+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5c0ac060 con 0x7f9d5c157770 2023-10-08T01:14:19.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.683+0000 7f9d527fc700 1 -- 172.21.15.119:0/1451722167 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9d54003040 con 0x7f9d5c157770 2023-10-08T01:14:19.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.684+0000 7f9d527fc700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d40066f30 0x7f9d400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:19.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.684+0000 7f9d527fc700 1 -- 172.21.15.119:0/1451722167 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5245+0+0 (secure 0 0 0) 0x7f9d54085ef0 con 0x7f9d5c157770 2023-10-08T01:14:19.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.684+0000 7f9d6239b700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d40066f30 0x7f9d400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:19.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.685+0000 7f9d6239b700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d40066f30 0x7f9d400693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009d40 tx=0x7f9d58008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:19.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.688+0000 7f9d527fc700 1 -- 172.21.15.119:0/1451722167 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d5c0ac060 con 0x7f9d5c157770 2023-10-08T01:14:19.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.884+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9d5c158bd0 con 0x7f9d5c157770 2023-10-08T01:14:19.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.884+0000 7f9d527fc700 1 -- 172.21.15.119:0/1451722167 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9d54053290 con 0x7f9d5c157770 2023-10-08T01:14:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:19.884 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:14:19.884 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T01:14:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T01:14:19.885 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:19.885 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d40066f30 msgr2=0x7f9d400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d40066f30 0x7f9d400693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009d40 tx=0x7f9d58008040 comp rx=0 tx=0).stop 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c157770 msgr2=0x7f9d5c0c5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c157770 0x7f9d5c0c5140 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009880 tx=0x7f9d54018000 comp rx=0 tx=0).stop 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 shutdown_connections 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d40066f30 0x7f9d400693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d5c157770 0x7f9d5c0c5140 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d5c06e730 0x7f9d5c14e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 --2- 172.21.15.119:0/1451722167 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d5c0c6d00 0x7f9d5c0c4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:19.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.886+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 >> 172.21.15.119:0/1451722167 conn(0x7f9d5c0cc1f0 msgr2=0x7f9d5c155b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:19.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.887+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 shutdown_connections 2023-10-08T01:14:19.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:19.887+0000 7f9d6339d700 1 -- 172.21.15.119:0/1451722167 wait complete. 2023-10-08T01:14:19.904 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T01:14:19.904 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T01:14:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.089+0000 7fe016d9d700 1 Processor -- start 2023-10-08T01:14:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.089+0000 7fe016d9d700 1 -- start start 2023-10-08T01:14:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.090+0000 7fe016d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe01015bb30 0x7fe0101623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.090+0000 7fe016d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe010157a50 con 0x7fe0100cc8d0 2023-10-08T01:14:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.090+0000 7fe016d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe010158f10 con 0x7fe01015b050 2023-10-08T01:14:20.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.090+0000 7fe016d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0101591b0 con 0x7fe01015bb30 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.090+0000 7fe01659c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe01015bb30 0x7fe0101623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe01659c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe01015bb30 0x7fe0101623f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48804/0 (socket says 172.21.15.119:48804) 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe01659c700 1 -- 172.21.15.119:0/2356512216 learned_addr learned my addr 172.21.15.119:0/2356512216 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe01559a700 1 --1- 172.21.15.119:0/2356512216 >> v1:172.21.15.119:6789/0 conn(0x7fe0100cc8d0 0x7fe01015b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54776/0 (socket says 172.21.15.119:54776) 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe01659c700 1 -- 172.21.15.119:0/2356512216 >> v1:172.21.15.184:6789/0 conn(0x7fe01015b050 legacy=0x7fe0100ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe01659c700 1 -- 172.21.15.119:0/2356512216 >> v1:172.21.15.119:6789/0 conn(0x7fe0100cc8d0 legacy=0x7fe01015b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe01659c700 1 -- 172.21.15.119:0/2356512216 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0100ca9e0 con 0x7fe01015bb30 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe014d99700 1 -- 172.21.15.119:0/2356512216 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306922542 0 0) 0x7fe010157a50 con 0x7fe0100cc8d0 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe01659c700 1 --2- 172.21.15.119:0/2356512216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101623f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe008009f20 tx=0x7fe008009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10032de1c1663b68 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.091+0000 7fe014d99700 1 -- 172.21.15.119:0/2356512216 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00800e070 con 0x7fe01015bb30 2023-10-08T01:14:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe014d99700 1 -- 172.21.15.119:0/2356512216 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00800ba80 con 0x7fe01015bb30 2023-10-08T01:14:20.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe014d99700 1 -- 172.21.15.119:0/2356512216 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080092e0 con 0x7fe01015bb30 2023-10-08T01:14:20.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe016d9d700 1 -- 172.21.15.119:0/2356512216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 msgr2=0x7fe0101623f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe016d9d700 1 --2- 172.21.15.119:0/2356512216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101623f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe008009f20 tx=0x7fe008009840 comp rx=0 tx=0).stop 2023-10-08T01:14:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe016d9d700 1 -- 172.21.15.119:0/2356512216 shutdown_connections 2023-10-08T01:14:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe016d9d700 1 --2- 172.21.15.119:0/2356512216 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101623f0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe016d9d700 1 -- 172.21.15.119:0/2356512216 >> 172.21.15.119:0/2356512216 conn(0x7fe010068e20 msgr2=0x7fe0100cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe016d9d700 1 -- 172.21.15.119:0/2356512216 shutdown_connections 2023-10-08T01:14:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.092+0000 7fe016d9d700 1 -- 172.21.15.119:0/2356512216 wait complete. 2023-10-08T01:14:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.094+0000 7fe016d9d700 1 Processor -- start 2023-10-08T01:14:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.094+0000 7fe016d9d700 1 -- start start 2023-10-08T01:14:20.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe016d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0100cc8d0 0x7fe0100ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe016d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01015b050 0x7fe01015e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe016d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe016d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01015e380 con 0x7fe01015bb30 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe016d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01015d770 con 0x7fe01015b050 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe016d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe01015da50 con 0x7fe0100cc8d0 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe01659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe01659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48810/0 (socket says 172.21.15.119:48810) 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe01659c700 1 -- 172.21.15.119:0/3588785587 learned_addr learned my addr 172.21.15.119:0/3588785587 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe015d9b700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0100cc8d0 0x7fe0100ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe01559a700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01015b050 0x7fe01015e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe01659c700 1 -- 172.21.15.119:0/3588785587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0100cc8d0 msgr2=0x7fe0100ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.095+0000 7fe01659c700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0100cc8d0 0x7fe0100ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe01659c700 1 -- 172.21.15.119:0/3588785587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01015b050 msgr2=0x7fe01015e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe01659c700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01015b050 0x7fe01015e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe01659c700 1 -- 172.21.15.119:0/3588785587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe008009ae0 con 0x7fe01015bb30 2023-10-08T01:14:20.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe01659c700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101427c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe0100714a0 tx=0x7fe008018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:20.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe005ffb700 1 -- 172.21.15.119:0/3588785587 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe008007d10 con 0x7fe01015bb30 2023-10-08T01:14:20.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe005ffb700 1 -- 172.21.15.119:0/3588785587 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe008007e90 con 0x7fe01015bb30 2023-10-08T01:14:20.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01015dd30 con 0x7fe01015bb30 2023-10-08T01:14:20.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe005ffb700 1 -- 172.21.15.119:0/3588785587 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0080266b0 con 0x7fe01015bb30 2023-10-08T01:14:20.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.096+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01015ed50 con 0x7fe01015bb30 2023-10-08T01:14:20.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.097+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01015ef70 con 0x7fe01015bb30 2023-10-08T01:14:20.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.098+0000 7fe005ffb700 1 -- 172.21.15.119:0/3588785587 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe00801d420 con 0x7fe01015bb30 2023-10-08T01:14:20.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.099+0000 7fe005ffb700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdff4066f30 0x7fdff40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.099+0000 7fe005ffb700 1 -- 172.21.15.119:0/3588785587 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5245+0+0 (secure 0 0 0) 0x7fe008086480 con 0x7fe01015bb30 2023-10-08T01:14:20.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.099+0000 7fe015d9b700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdff4066f30 0x7fdff40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.100+0000 7fe015d9b700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdff4066f30 0x7fdff40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe00c003a80 tx=0x7fe00c007ec0 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:20.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.103+0000 7fe005ffb700 1 -- 172.21.15.119:0/3588785587 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe008053820 con 0x7fe01015bb30 2023-10-08T01:14:20.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.292+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe01004c3c0 con 0x7fe01015bb30 2023-10-08T01:14:20.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.743+0000 7fe005ffb700 1 -- 172.21.15.119:0/3588785587 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe008056e40 con 0x7fe01015bb30 2023-10-08T01:14:20.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdff4066f30 msgr2=0x7fdff40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdff4066f30 0x7fdff40693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe00c003a80 tx=0x7fe00c007ec0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 msgr2=0x7fe0101427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101427c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fe0100714a0 tx=0x7fe008018000 comp rx=0 tx=0).stop 2023-10-08T01:14:20.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 shutdown_connections 2023-10-08T01:14:20.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdff4066f30 0x7fdff40693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe0100cc8d0 0x7fe0100ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe01015bb30 0x7fe0101427c0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 --2- 172.21.15.119:0/3588785587 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe01015b050 0x7fe01015e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.746+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 >> 172.21.15.119:0/3588785587 conn(0x7fe010068e20 msgr2=0x7fe01006dd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:20.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.747+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 shutdown_connections 2023-10-08T01:14:20.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.747+0000 7fe016d9d700 1 -- 172.21.15.119:0/3588785587 wait complete. 2023-10-08T01:14:20.752 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T01:14:20.769 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T01:14:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.957+0000 7f70043ff700 1 Processor -- start 2023-10-08T01:14:20.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.957+0000 7f70043ff700 1 -- start start 2023-10-08T01:14:20.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.958+0000 7f70043ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6ffc1569f0 0x7f6ffc0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.959+0000 7f70043ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc15a6f0 con 0x7f6ffc068390 2023-10-08T01:14:20.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.959+0000 7f70043ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc15a9d0 con 0x7f6ffc156620 2023-10-08T01:14:20.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.959+0000 7f70043ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ffc0c8d30 con 0x7f6ffc1569f0 2023-10-08T01:14:20.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.959+0000 7f700299c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6ffc068390 0x7f6ffc06da30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54784/0 (socket says 172.21.15.119:54784) 2023-10-08T01:14:20.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.959+0000 7f700299c700 1 -- 172.21.15.119:0/4158377885 learned_addr learned my addr 172.21.15.119:0/4158377885 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:20.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.959+0000 7f700199a700 1 --2- 172.21.15.119:0/4158377885 >> v2:172.21.15.184:3300/0 conn(0x7f6ffc1569f0 0x7f6ffc0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.959+0000 7f7001199700 1 -- 172.21.15.119:0/4158377885 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386968197 0 0) 0x7f6ffc15a6f0 con 0x7f6ffc068390 2023-10-08T01:14:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.959+0000 7f7001199700 1 -- 172.21.15.119:0/4158377885 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fe8003570 con 0x7f6ffc068390 2023-10-08T01:14:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.960+0000 7f700199a700 1 -- 172.21.15.119:0/4158377885 >> v1:172.21.15.184:6789/0 conn(0x7f6ffc156620 legacy=0x7f6ffc0ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.960+0000 7f700199a700 1 -- 172.21.15.119:0/4158377885 >> v1:172.21.15.119:6789/0 conn(0x7f6ffc068390 legacy=0x7f6ffc06da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.960+0000 7f700199a700 1 -- 172.21.15.119:0/4158377885 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc15acb0 con 0x7f6ffc1569f0 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.960+0000 7f700199a700 1 --2- 172.21.15.119:0/4158377885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc1569f0 0x7f6ffc0c87f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0098e0 tx=0x7f6fec0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50d341e377fc549 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.960+0000 7f7001199700 1 -- 172.21.15.119:0/4158377885 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00b640 con 0x7f6ffc1569f0 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.960+0000 7f7001199700 1 -- 172.21.15.119:0/4158377885 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec00b7a0 con 0x7f6ffc1569f0 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.960+0000 7f7001199700 1 -- 172.21.15.119:0/4158377885 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec00bb40 con 0x7f6ffc1569f0 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.961+0000 7f70043ff700 1 -- 172.21.15.119:0/4158377885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc1569f0 msgr2=0x7f6ffc0c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.961+0000 7f70043ff700 1 --2- 172.21.15.119:0/4158377885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc1569f0 0x7f6ffc0c87f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0098e0 tx=0x7f6fec0091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.961+0000 7f70043ff700 1 -- 172.21.15.119:0/4158377885 shutdown_connections 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.961+0000 7f70043ff700 1 --2- 172.21.15.119:0/4158377885 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc1569f0 0x7f6ffc0c87f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.961+0000 7f70043ff700 1 -- 172.21.15.119:0/4158377885 >> 172.21.15.119:0/4158377885 conn(0x7f6ffc0cc540 msgr2=0x7f6ffc0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:20.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.961+0000 7f70043ff700 1 -- 172.21.15.119:0/4158377885 shutdown_connections 2023-10-08T01:14:20.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.961+0000 7f70043ff700 1 -- 172.21.15.119:0/4158377885 wait complete. 2023-10-08T01:14:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.962+0000 7f70043ff700 1 Processor -- start 2023-10-08T01:14:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.962+0000 7f70043ff700 1 -- start start 2023-10-08T01:14:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.963+0000 7f70043ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc068390 0x7f6ffc143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.963+0000 7f70043ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc156620 0x7f6ffc0bff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.963+0000 7f70043ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ffc1569f0 0x7f6ffc0c04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.963+0000 7f70043ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc16aeb0 con 0x7f6ffc068390 2023-10-08T01:14:20.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.963+0000 7f70043ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc0c29f0 con 0x7f6ffc156620 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.963+0000 7f70043ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc0c1e40 con 0x7f6ffc1569f0 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc068390 0x7f6ffc143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc068390 0x7f6ffc143230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48818/0 (socket says 172.21.15.119:48818) 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 -- 172.21.15.119:0/3396684650 learned_addr learned my addr 172.21.15.119:0/3396684650 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700299c700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ffc1569f0 0x7f6ffc0c04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700199a700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc156620 0x7f6ffc0bff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 -- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ffc1569f0 msgr2=0x7f6ffc0c04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ffc1569f0 0x7f6ffc0c04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 -- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc156620 msgr2=0x7f6ffc0bff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc156620 0x7f6ffc0bff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:20.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 -- 172.21.15.119:0/3396684650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec009420 con 0x7f6ffc068390 2023-10-08T01:14:20.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700199a700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc156620 0x7f6ffc0bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:20.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.964+0000 7f700219b700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc068390 0x7f6ffc143230 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800c510 tx=0x7f6ff800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:20.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.965+0000 7f6ff27fc700 1 -- 172.21.15.119:0/3396684650 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011590 con 0x7f6ffc068390 2023-10-08T01:14:20.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.965+0000 7f6ff27fc700 1 -- 172.21.15.119:0/3396684650 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8011710 con 0x7f6ffc068390 2023-10-08T01:14:20.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.965+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc0c2e80 con 0x7f6ffc068390 2023-10-08T01:14:20.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.965+0000 7f6ff27fc700 1 -- 172.21.15.119:0/3396684650 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011ad0 con 0x7f6ffc068390 2023-10-08T01:14:20.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.965+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc0c23a0 con 0x7f6ffc068390 2023-10-08T01:14:20.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.966+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc0cd020 con 0x7f6ffc068390 2023-10-08T01:14:20.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.967+0000 7f6ff27fc700 1 -- 172.21.15.119:0/3396684650 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ff801b3b0 con 0x7f6ffc068390 2023-10-08T01:14:20.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.968+0000 7f6ff27fc700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fe0066f30 0x7f6fe00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:20.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.968+0000 7f6ff27fc700 1 -- 172.21.15.119:0/3396684650 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5615+0+0 (secure 0 0 0) 0x7f6ff8083b30 con 0x7f6ffc068390 2023-10-08T01:14:20.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.968+0000 7f700199a700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fe0066f30 0x7f6fe00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:20.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.968+0000 7f700199a700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fe0066f30 0x7f6fe00693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0093f0 tx=0x7f6fec017000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:20.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:20.972+0000 7f6ff27fc700 1 -- 172.21.15.119:0/3396684650 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ff8050d60 con 0x7f6ffc068390 2023-10-08T01:14:21.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.164+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6ffc0c2cd0 con 0x7f6ffc068390 2023-10-08T01:14:21.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.748+0000 7f6ff27fc700 1 -- 172.21.15.119:0/3396684650 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6ff8017070 con 0x7f6ffc068390 2023-10-08T01:14:21.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.762+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fe0066f30 msgr2=0x7f6fe00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:21.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.762+0000 7f70043ff700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fe0066f30 0x7f6fe00693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6fec0093f0 tx=0x7f6fec017000 comp rx=0 tx=0).stop 2023-10-08T01:14:21.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.762+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc068390 msgr2=0x7f6ffc143230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:21.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.762+0000 7f70043ff700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc068390 0x7f6ffc143230 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800c510 tx=0x7f6ff800c8d0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.765+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 shutdown_connections 2023-10-08T01:14:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.765+0000 7f70043ff700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fe0066f30 0x7f6fe00693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.765+0000 7f70043ff700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ffc1569f0 0x7f6ffc0c04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.765+0000 7f70043ff700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ffc068390 0x7f6ffc143230 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.765+0000 7f70043ff700 1 --2- 172.21.15.119:0/3396684650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ffc156620 0x7f6ffc0bff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.765+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 >> 172.21.15.119:0/3396684650 conn(0x7f6ffc0cc540 msgr2=0x7f6ffc0cc910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:21.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.770+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 shutdown_connections 2023-10-08T01:14:21.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.770+0000 7f70043ff700 1 -- 172.21.15.119:0/3396684650 wait complete. 2023-10-08T01:14:21.772 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T01:14:21.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T01:14:21.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.972+0000 7fd9b61ff700 1 Processor -- start 2023-10-08T01:14:21.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.972+0000 7fd9b61ff700 1 -- start start 2023-10-08T01:14:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9b61ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b00c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9b61ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd9b00ccee0 0x7fd9b01561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9b61ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd9b00c87c0 0x7fd9b0156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9b61ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b0156c50 con 0x7fd9b00c87c0 2023-10-08T01:14:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9b61ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b015b0d0 con 0x7fd9b00ccee0 2023-10-08T01:14:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9b61ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b00c7a60 con 0x7fd9b00c83f0 2023-10-08T01:14:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9affff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd9b00c87c0 0x7fd9b0156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9affff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd9b00c87c0 0x7fd9b0156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48824/0 (socket says 172.21.15.119:48824) 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9affff700 1 -- 172.21.15.119:0/728263634 learned_addr learned my addr 172.21.15.119:0/728263634 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9af7fe700 1 --2- 172.21.15.119:0/728263634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b00c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9aeffd700 1 --2- 172.21.15.119:0/728263634 >> v2:172.21.15.184:3300/0 conn(0x7fd9b00ccee0 0x7fd9b01561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9affff700 1 -- 172.21.15.119:0/728263634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 msgr2=0x7fd9b00c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9affff700 1 --2- 172.21.15.119:0/728263634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b00c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9affff700 1 -- 172.21.15.119:0/728263634 >> v2:172.21.15.184:3300/0 conn(0x7fd9b00ccee0 msgr2=0x7fd9b01561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9affff700 1 --2- 172.21.15.119:0/728263634 >> v2:172.21.15.184:3300/0 conn(0x7fd9b00ccee0 0x7fd9b01561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9affff700 1 -- 172.21.15.119:0/728263634 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b015a4c0 con 0x7fd9b00c87c0 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.974+0000 7fd9aeffd700 1 --2- 172.21.15.119:0/728263634 >> v2:172.21.15.184:3300/0 conn(0x7fd9b00ccee0 0x7fd9b01561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.975+0000 7fd9affff700 1 --2- 172.21.15.119:0/728263634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0156710 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd9b006fe40 tx=0x7fd9a800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95e7ca1cf1a6cb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.975+0000 7fd9ae7fc700 1 -- 172.21.15.119:0/728263634 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8010070 con 0x7fd9b00c87c0 2023-10-08T01:14:21.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.975+0000 7fd9ae7fc700 1 -- 172.21.15.119:0/728263634 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a800de90 con 0x7fd9b00c87c0 2023-10-08T01:14:21.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.975+0000 7fd9ae7fc700 1 -- 172.21.15.119:0/728263634 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a800b320 con 0x7fd9b00c87c0 2023-10-08T01:14:21.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.975+0000 7fd9b61ff700 1 -- 172.21.15.119:0/728263634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 msgr2=0x7fd9b0156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:21.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.975+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/728263634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0156710 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fd9b006fe40 tx=0x7fd9a800ba50 comp rx=0 tx=0).stop 2023-10-08T01:14:21.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.976+0000 7fd9b61ff700 1 -- 172.21.15.119:0/728263634 shutdown_connections 2023-10-08T01:14:21.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.976+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/728263634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0156710 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.976+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/728263634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b00c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.976+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/728263634 >> v2:172.21.15.184:3300/0 conn(0x7fd9b00ccee0 0x7fd9b01561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.976+0000 7fd9b61ff700 1 -- 172.21.15.119:0/728263634 >> 172.21.15.119:0/728263634 conn(0x7fd9b0068e10 msgr2=0x7fd9b00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.976+0000 7fd9b61ff700 1 -- 172.21.15.119:0/728263634 shutdown_connections 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.976+0000 7fd9b61ff700 1 -- 172.21.15.119:0/728263634 wait complete. 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.977+0000 7fd9b61ff700 1 Processor -- start 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.977+0000 7fd9b61ff700 1 -- start start 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9b61ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b015e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9b61ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0169f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9b61ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b00ccee0 0x7fd9b0155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:21.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9b61ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0154620 con 0x7fd9b00c87c0 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9b61ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b0154bc0 con 0x7fd9b00ccee0 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9b61ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b016af10 con 0x7fd9b00c83f0 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9aeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0169f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9aeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0169f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48830/0 (socket says 172.21.15.119:48830) 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.978+0000 7fd9aeffd700 1 -- 172.21.15.119:0/548558664 learned_addr learned my addr 172.21.15.119:0/548558664 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9affff700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b00ccee0 0x7fd9b0155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9af7fe700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b015e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9aeffd700 1 -- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 msgr2=0x7fd9b015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9aeffd700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b015e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9aeffd700 1 -- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b00ccee0 msgr2=0x7fd9b0155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9aeffd700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b00ccee0 0x7fd9b0155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:21.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9aeffd700 1 -- 172.21.15.119:0/548558664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a800bcf0 con 0x7fd9b00c87c0 2023-10-08T01:14:21.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9affff700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b00ccee0 0x7fd9b0155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:21.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9af7fe700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b015e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:21.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9aeffd700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0169f30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400d860 tx=0x7fd9a4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:21.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9977fe700 1 -- 172.21.15.119:0/548558664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a40125f0 con 0x7fd9b00c87c0 2023-10-08T01:14:21.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9977fe700 1 -- 172.21.15.119:0/548558664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a4012750 con 0x7fd9b00c87c0 2023-10-08T01:14:21.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b016a950 con 0x7fd9b00c87c0 2023-10-08T01:14:21.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9977fe700 1 -- 172.21.15.119:0/548558664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4012ad0 con 0x7fd9b00c87c0 2023-10-08T01:14:21.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.979+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b0154850 con 0x7fd9b00c87c0 2023-10-08T01:14:21.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.980+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b0154d30 con 0x7fd9b00c87c0 2023-10-08T01:14:21.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.982+0000 7fd9977fe700 1 -- 172.21.15.119:0/548558664 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd9a4014430 con 0x7fd9b00c87c0 2023-10-08T01:14:21.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.982+0000 7fd9977fe700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd99c066f30 0x7fd99c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:21.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.982+0000 7fd9977fe700 1 -- 172.21.15.119:0/548558664 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5997+0+0 (secure 0 0 0) 0x7fd9a40855e0 con 0x7fd9b00c87c0 2023-10-08T01:14:21.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.983+0000 7fd9af7fe700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd99c066f30 0x7fd99c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:21.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.983+0000 7fd9af7fe700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd99c066f30 0x7fd99c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:21.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:21.986+0000 7fd9977fe700 1 -- 172.21.15.119:0/548558664 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd9a40561a0 con 0x7fd9b00c87c0 2023-10-08T01:14:22.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.218+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd9b004c3c0 con 0x7fd9b00c87c0 2023-10-08T01:14:22.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.763+0000 7fd9977fe700 1 -- 172.21.15.119:0/548558664 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd9a4055d30 con 0x7fd9b00c87c0 2023-10-08T01:14:22.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.770+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd99c066f30 msgr2=0x7fd99c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:22.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.770+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd99c066f30 0x7fd99c0693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).stop 2023-10-08T01:14:22.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.770+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 msgr2=0x7fd9b0169f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:22.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.770+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0169f30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400d860 tx=0x7fd9a4008040 comp rx=0 tx=0).stop 2023-10-08T01:14:22.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.773+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 shutdown_connections 2023-10-08T01:14:22.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.773+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd99c066f30 0x7fd99c0693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:22.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.773+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd9b00c83f0 0x7fd9b015e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:22.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.773+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd9b00ccee0 0x7fd9b0155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:22.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.773+0000 7fd9b61ff700 1 --2- 172.21.15.119:0/548558664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b00c87c0 0x7fd9b0169f30 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:22.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.774+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 >> 172.21.15.119:0/548558664 conn(0x7fd9b0068e10 msgr2=0x7fd9b006eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:22.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.782+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 shutdown_connections 2023-10-08T01:14:22.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.783+0000 7fd9b61ff700 1 -- 172.21.15.119:0/548558664 wait complete. 2023-10-08T01:14:22.788 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 14 and data pool 15 2023-10-08T01:14:22.812 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:14:22.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.993+0000 7f31049ff700 1 Processor -- start 2023-10-08T01:14:22.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.993+0000 7f31049ff700 1 -- start start 2023-10-08T01:14:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.994+0000 7f31049ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f31000cbf30 0x7f31000ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.994+0000 7f31049ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f31000b1420 0x7f31000c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.994+0000 7f31049ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31000cc300 con 0x7f31000ccf90 2023-10-08T01:14:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.994+0000 7f31049ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3100156bf0 con 0x7f31000cbf30 2023-10-08T01:14:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f31049ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31000ae950 con 0x7f31000b1420 2023-10-08T01:14:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fe59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f31000cbf30 0x7f31000ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fe59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f31000cbf30 0x7f31000ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48836/0 (socket says 172.21.15.119:48836) 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fe59c700 1 -- 172.21.15.119:0/3444894231 learned_addr learned my addr 172.21.15.119:0/3444894231 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30f7fff700 1 --1- 172.21.15.119:0/3444894231 >> v1:172.21.15.119:6789/0 conn(0x7f31000ccf90 0x7f31000b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54790/0 (socket says 172.21.15.119:54790) 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fed9d700 1 --2- 172.21.15.119:0/3444894231 >> v2:172.21.15.184:3300/0 conn(0x7f31000b1420 0x7f31000c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fdd9b700 1 -- 172.21.15.119:0/3444894231 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757336500 0 0) 0x7f31000cc300 con 0x7f31000ccf90 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fdd9b700 1 -- 172.21.15.119:0/3444894231 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30ec003570 con 0x7f31000ccf90 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fe59c700 1 -- 172.21.15.119:0/3444894231 >> v2:172.21.15.184:3300/0 conn(0x7f31000b1420 msgr2=0x7f31000c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fe59c700 1 --2- 172.21.15.119:0/3444894231 >> v2:172.21.15.184:3300/0 conn(0x7f31000b1420 0x7f31000c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fe59c700 1 -- 172.21.15.119:0/3444894231 >> v1:172.21.15.119:6789/0 conn(0x7f31000ccf90 legacy=0x7f31000b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fe59c700 1 -- 172.21.15.119:0/3444894231 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31000ad370 con 0x7f31000cbf30 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.995+0000 7f30fed9d700 1 --2- 172.21.15.119:0/3444894231 >> v2:172.21.15.184:3300/0 conn(0x7f31000b1420 0x7f31000c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f30fe59c700 1 --2- 172.21.15.119:0/3444894231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000ae410 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f31000b19e0 tx=0x7f30f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c24ec0507b0f44 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f30fdd9b700 1 -- 172.21.15.119:0/3444894231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f0009ea0 con 0x7f31000cbf30 2023-10-08T01:14:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f30fdd9b700 1 -- 172.21.15.119:0/3444894231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f000b820 con 0x7f31000cbf30 2023-10-08T01:14:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f30fdd9b700 1 -- 172.21.15.119:0/3444894231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f000bbc0 con 0x7f31000cbf30 2023-10-08T01:14:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f31049ff700 1 -- 172.21.15.119:0/3444894231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 msgr2=0x7f31000ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f31049ff700 1 --2- 172.21.15.119:0/3444894231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000ae410 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f31000b19e0 tx=0x7f30f0009250 comp rx=0 tx=0).stop 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f31049ff700 1 -- 172.21.15.119:0/3444894231 shutdown_connections 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f31049ff700 1 --2- 172.21.15.119:0/3444894231 >> v2:172.21.15.184:3300/0 conn(0x7f31000b1420 0x7f31000c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f31049ff700 1 --2- 172.21.15.119:0/3444894231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000ae410 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f31049ff700 1 -- 172.21.15.119:0/3444894231 >> 172.21.15.119:0/3444894231 conn(0x7f310006a320 msgr2=0x7f31000cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.996+0000 7f31049ff700 1 -- 172.21.15.119:0/3444894231 shutdown_connections 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.997+0000 7f31049ff700 1 -- 172.21.15.119:0/3444894231 wait complete. 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.998+0000 7f31049ff700 1 Processor -- start 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.998+0000 7f31049ff700 1 -- start start 2023-10-08T01:14:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f31049ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31000b1420 0x7f310016a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f31049ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f31049ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31000ccf90 0x7f31000a7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f31049ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310016ae50 con 0x7f31000cbf30 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f31049ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310016b960 con 0x7f31000b1420 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f31049ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f310016b030 con 0x7f31000ccf90 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30f7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30f7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000a6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48850/0 (socket says 172.21.15.119:48850) 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30f7fff700 1 -- 172.21.15.119:0/162672447 learned_addr learned my addr 172.21.15.119:0/162672447 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30fed9d700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31000ccf90 0x7f31000a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30fe59c700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31000b1420 0x7f310016a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30f7fff700 1 -- 172.21.15.119:0/162672447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31000ccf90 msgr2=0x7f31000a7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30f7fff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31000ccf90 0x7f31000a7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30f7fff700 1 -- 172.21.15.119:0/162672447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31000b1420 msgr2=0x7f310016a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:22.999+0000 7f30f7fff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31000b1420 0x7f310016a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.000+0000 7f30f7fff700 1 -- 172.21.15.119:0/162672447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f00094f0 con 0x7f31000cbf30 2023-10-08T01:14:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.000+0000 7f30f7fff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000a6b10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c590 tx=0x7f30e8007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.000+0000 7f30f67fc700 1 -- 172.21.15.119:0/162672447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8005a70 con 0x7f31000cbf30 2023-10-08T01:14:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.000+0000 7f30f67fc700 1 -- 172.21.15.119:0/162672447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30e8005bf0 con 0x7f31000cbf30 2023-10-08T01:14:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.000+0000 7f30f67fc700 1 -- 172.21.15.119:0/162672447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e80070d0 con 0x7f31000cbf30 2023-10-08T01:14:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.000+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310016b310 con 0x7f31000cbf30 2023-10-08T01:14:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.000+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310016a970 con 0x7f31000cbf30 2023-10-08T01:14:23.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.001+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f310004c3c0 con 0x7f31000cbf30 2023-10-08T01:14:23.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.002+0000 7f30f67fc700 1 -- 172.21.15.119:0/162672447 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f30e8007480 con 0x7f31000cbf30 2023-10-08T01:14:23.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.003+0000 7f30f67fc700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f30e0066f30 0x7f30e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.003+0000 7f30f67fc700 1 -- 172.21.15.119:0/162672447 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f30e8083d30 con 0x7f31000cbf30 2023-10-08T01:14:23.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.003+0000 7f30fe59c700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f30e0066f30 0x7f30e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.004+0000 7f30fe59c700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f30e0066f30 0x7f30e00693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31000b19e0 tx=0x7f30f0017000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:23.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.007+0000 7f30f67fc700 1 -- 172.21.15.119:0/162672447 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f30e8050d70 con 0x7f31000cbf30 2023-10-08T01:14:23.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.199+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31000ad650 con 0x7f31000cbf30 2023-10-08T01:14:23.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.200+0000 7f30f67fc700 1 -- 172.21.15.119:0/162672447 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16022 (secure 0 0 0) 0x7f30e8017070 con 0x7f31000cbf30 2023-10-08T01:14:23.200 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:23.201 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":119,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:22.219605+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:14:20.330219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T01:14:21.202759+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:14:23.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f30e0066f30 msgr2=0x7f30e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f30e0066f30 0x7f30e00693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f31000b19e0 tx=0x7f30f0017000 comp rx=0 tx=0).stop 2023-10-08T01:14:23.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 msgr2=0x7f31000a6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000a6b10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f30e800c590 tx=0x7f30e8007e00 comp rx=0 tx=0).stop 2023-10-08T01:14:23.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 shutdown_connections 2023-10-08T01:14:23.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f30e0066f30 0x7f30e00693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31000ccf90 0x7f31000a7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31000b1420 0x7f310016a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 --2- 172.21.15.119:0/162672447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31000cbf30 0x7f31000a6b10 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 >> 172.21.15.119:0/162672447 conn(0x7f310006a320 msgr2=0x7f31000caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:23.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 shutdown_connections 2023-10-08T01:14:23.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.203+0000 7f31049ff700 1 -- 172.21.15.119:0/162672447 wait complete. 2023-10-08T01:14:23.224 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:14:23.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.403+0000 7efccb19d700 1 Processor -- start 2023-10-08T01:14:23.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.403+0000 7efccb19d700 1 -- start start 2023-10-08T01:14:23.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efccb19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150180 0x7efcc4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efccb19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcc4150550 0x7efcc40c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efccb19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcc406e2a0 con 0x7efcc40c7240 2023-10-08T01:14:23.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efccb19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcc40c76e0 con 0x7efcc4150550 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efccb19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc406b2e0 con 0x7efcc4150180 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efcc999a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcc4150550 0x7efcc40c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efcc999a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcc4150550 0x7efcc40c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48860/0 (socket says 172.21.15.119:48860) 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efcc999a700 1 -- 172.21.15.119:0/2301118208 learned_addr learned my addr 172.21.15.119:0/2301118208 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.405+0000 7efcca19b700 1 --2- 172.21.15.119:0/2301118208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150180 0x7efcc4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.406+0000 7efcc999a700 1 -- 172.21.15.119:0/2301118208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150180 msgr2=0x7efcc4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.406+0000 7efcc999a700 1 --2- 172.21.15.119:0/2301118208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150180 0x7efcc4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.406+0000 7efcc999a700 1 -- 172.21.15.119:0/2301118208 >> v1:172.21.15.184:6789/0 conn(0x7efcc40c7240 legacy=0x7efcc4069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.406+0000 7efcc999a700 1 -- 172.21.15.119:0/2301118208 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc40c78e0 con 0x7efcc4150550 2023-10-08T01:14:23.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.406+0000 7efcc999a700 1 --2- 172.21.15.119:0/2301118208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150550 0x7efcc40c6d00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7efcb40099b0 tx=0x7efcb4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2464d2781d047f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:23.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.406+0000 7efcc9199700 1 -- 172.21.15.119:0/2301118208 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4009ea0 con 0x7efcc4150550 2023-10-08T01:14:23.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.406+0000 7efcc9199700 1 -- 172.21.15.119:0/2301118208 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb400b860 con 0x7efcc4150550 2023-10-08T01:14:23.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.406+0000 7efcc9199700 1 -- 172.21.15.119:0/2301118208 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400bc00 con 0x7efcc4150550 2023-10-08T01:14:23.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.407+0000 7efccb19d700 1 -- 172.21.15.119:0/2301118208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150550 msgr2=0x7efcc40c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.407+0000 7efccb19d700 1 --2- 172.21.15.119:0/2301118208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150550 0x7efcc40c6d00 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7efcb40099b0 tx=0x7efcb4009250 comp rx=0 tx=0).stop 2023-10-08T01:14:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.407+0000 7efccb19d700 1 -- 172.21.15.119:0/2301118208 shutdown_connections 2023-10-08T01:14:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.407+0000 7efccb19d700 1 --2- 172.21.15.119:0/2301118208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150180 0x7efcc4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.407+0000 7efccb19d700 1 --2- 172.21.15.119:0/2301118208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150550 0x7efcc40c6d00 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.407+0000 7efccb19d700 1 -- 172.21.15.119:0/2301118208 >> 172.21.15.119:0/2301118208 conn(0x7efcc40cc1f0 msgr2=0x7efcc40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.407+0000 7efccb19d700 1 -- 172.21.15.119:0/2301118208 shutdown_connections 2023-10-08T01:14:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.407+0000 7efccb19d700 1 -- 172.21.15.119:0/2301118208 wait complete. 2023-10-08T01:14:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.408+0000 7efccb19d700 1 Processor -- start 2023-10-08T01:14:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.408+0000 7efccb19d700 1 -- start start 2023-10-08T01:14:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efccb19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc40c7240 0x7efcc40aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efccb19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150180 0x7efcc414d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efccb19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150550 0x7efcc40af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efccb19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc40c5740 con 0x7efcc4150180 2023-10-08T01:14:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efccb19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcc40c5a20 con 0x7efcc40c7240 2023-10-08T01:14:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efccb19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcc40c5d00 con 0x7efcc4150550 2023-10-08T01:14:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efcc999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150180 0x7efcc414d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efcc999a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150180 0x7efcc414d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48874/0 (socket says 172.21.15.119:48874) 2023-10-08T01:14:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.409+0000 7efcc999a700 1 -- 172.21.15.119:0/3549874110 learned_addr learned my addr 172.21.15.119:0/3549874110 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcca19b700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc40c7240 0x7efcc40aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcca99c700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150550 0x7efcc40af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcc999a700 1 -- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150550 msgr2=0x7efcc40af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcc999a700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150550 0x7efcc40af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcc999a700 1 -- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc40c7240 msgr2=0x7efcc40aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcc999a700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc40c7240 0x7efcc40aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcc999a700 1 -- 172.21.15.119:0/3549874110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40094f0 con 0x7efcc4150180 2023-10-08T01:14:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcca99c700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150550 0x7efcc40af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcc999a700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150180 0x7efcc414d1f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7efcc406fe40 tx=0x7efcb4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcba7fc700 1 -- 172.21.15.119:0/3549874110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb400bbe0 con 0x7efcc4150180 2023-10-08T01:14:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcba7fc700 1 -- 172.21.15.119:0/3549874110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb4011420 con 0x7efcc4150180 2023-10-08T01:14:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcc40c5fe0 con 0x7efcc4150180 2023-10-08T01:14:23.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efcba7fc700 1 -- 172.21.15.119:0/3549874110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40117c0 con 0x7efcc4150180 2023-10-08T01:14:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.410+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcc40af690 con 0x7efcc4150180 2023-10-08T01:14:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.411+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcc404c3c0 con 0x7efcc4150180 2023-10-08T01:14:23.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.412+0000 7efcba7fc700 1 -- 172.21.15.119:0/3549874110 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcb4011bf0 con 0x7efcc4150180 2023-10-08T01:14:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.414+0000 7efcba7fc700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efca8066f30 0x7efca80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.414+0000 7efcba7fc700 1 -- 172.21.15.119:0/3549874110 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6097+0+0 (secure 0 0 0) 0x7efcb4085ff0 con 0x7efcc4150180 2023-10-08T01:14:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.414+0000 7efcca19b700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efca8066f30 0x7efca80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.414+0000 7efcca19b700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efca8066f30 0x7efca80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efcc000adc0 tx=0x7efcc0008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.418+0000 7efcba7fc700 1 -- 172.21.15.119:0/3549874110 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcb40b75a0 con 0x7efcc4150180 2023-10-08T01:14:23.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.608+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcc406ada0 con 0x7efcc4150180 2023-10-08T01:14:23.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.609+0000 7efcba7fc700 1 -- 172.21.15.119:0/3549874110 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+16022 (secure 0 0 0) 0x7efcb4053050 con 0x7efcc4150180 2023-10-08T01:14:23.609 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:23.609 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":119,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:22.219605+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:14:20.330219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T01:14:21.202759+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:14:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efca8066f30 msgr2=0x7efca80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efca8066f30 0x7efca80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efcc000adc0 tx=0x7efcc0008040 comp rx=0 tx=0).stop 2023-10-08T01:14:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150180 msgr2=0x7efcc414d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150180 0x7efcc414d1f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7efcc406fe40 tx=0x7efcb4017000 comp rx=0 tx=0).stop 2023-10-08T01:14:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 shutdown_connections 2023-10-08T01:14:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efca8066f30 0x7efca80693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcc4150550 0x7efcc40af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcc40c7240 0x7efcc40aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 --2- 172.21.15.119:0/3549874110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcc4150180 0x7efcc414d1f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 >> 172.21.15.119:0/3549874110 conn(0x7efcc40cc1f0 msgr2=0x7efcc414f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 shutdown_connections 2023-10-08T01:14:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.611+0000 7efccb19d700 1 -- 172.21.15.119:0/3549874110 wait complete. 2023-10-08T01:14:23.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T01:14:23.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.815+0000 7eff44b9d700 1 Processor -- start 2023-10-08T01:14:23.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.816+0000 7eff44b9d700 1 -- start start 2023-10-08T01:14:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff44b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff4006f060 0x7eff400ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff44b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f4b0 0x7eff400ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff44b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4006ebd0 con 0x7eff400cb1a0 2023-10-08T01:14:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff44b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff4015b2e0 con 0x7eff4006f060 2023-10-08T01:14:23.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff44b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4015aa70 con 0x7eff4006f4b0 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff3effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff4006f060 0x7eff400ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff3effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7eff4006f060 0x7eff400ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48884/0 (socket says 172.21.15.119:48884) 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff3effd700 1 -- 172.21.15.119:0/4196181749 learned_addr learned my addr 172.21.15.119:0/4196181749 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.817+0000 7eff3ffff700 1 --2- 172.21.15.119:0/4196181749 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f4b0 0x7eff400ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.818+0000 7eff3effd700 1 -- 172.21.15.119:0/4196181749 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f4b0 msgr2=0x7eff400ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.818+0000 7eff3effd700 1 --2- 172.21.15.119:0/4196181749 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f4b0 0x7eff400ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.818+0000 7eff3effd700 1 -- 172.21.15.119:0/4196181749 >> v1:172.21.15.184:6789/0 conn(0x7eff400cb1a0 legacy=0x7eff4015a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.818+0000 7eff3effd700 1 -- 172.21.15.119:0/4196181749 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4015ad20 con 0x7eff4006f060 2023-10-08T01:14:23.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.818+0000 7eff3effd700 1 --2- 172.21.15.119:0/4196181749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f060 0x7eff400ae0d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7eff340099b0 tx=0x7eff34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d23932380cdb4d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.818+0000 7eff3e7fc700 1 -- 172.21.15.119:0/4196181749 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400b8f0 con 0x7eff4006f060 2023-10-08T01:14:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.818+0000 7eff3e7fc700 1 -- 172.21.15.119:0/4196181749 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff3400ba50 con 0x7eff4006f060 2023-10-08T01:14:23.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.818+0000 7eff3e7fc700 1 -- 172.21.15.119:0/4196181749 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3400bdf0 con 0x7eff4006f060 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.819+0000 7eff44b9d700 1 -- 172.21.15.119:0/4196181749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f060 msgr2=0x7eff400ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.819+0000 7eff44b9d700 1 --2- 172.21.15.119:0/4196181749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f060 0x7eff400ae0d0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7eff340099b0 tx=0x7eff34009250 comp rx=0 tx=0).stop 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.819+0000 7eff44b9d700 1 -- 172.21.15.119:0/4196181749 shutdown_connections 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.819+0000 7eff44b9d700 1 --2- 172.21.15.119:0/4196181749 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f4b0 0x7eff400ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.819+0000 7eff44b9d700 1 --2- 172.21.15.119:0/4196181749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f060 0x7eff400ae0d0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.819+0000 7eff44b9d700 1 -- 172.21.15.119:0/4196181749 >> 172.21.15.119:0/4196181749 conn(0x7eff400cc1e0 msgr2=0x7eff400cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.819+0000 7eff44b9d700 1 -- 172.21.15.119:0/4196181749 shutdown_connections 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.819+0000 7eff44b9d700 1 -- 172.21.15.119:0/4196181749 wait complete. 2023-10-08T01:14:23.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.820+0000 7eff44b9d700 1 Processor -- start 2023-10-08T01:14:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.820+0000 7eff44b9d700 1 -- start start 2023-10-08T01:14:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.821+0000 7eff44b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f060 0x7eff4016a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.821+0000 7eff44b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f4b0 0x7eff40153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.821+0000 7eff44b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff400cb1a0 0x7eff400c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.821+0000 7eff44b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff40153760 con 0x7eff4006f4b0 2023-10-08T01:14:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.821+0000 7eff44b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff40154200 con 0x7eff400cb1a0 2023-10-08T01:14:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.821+0000 7eff44b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4016d260 con 0x7eff4006f060 2023-10-08T01:14:23.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f4b0 0x7eff40153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f4b0 0x7eff40153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48896/0 (socket says 172.21.15.119:48896) 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 -- 172.21.15.119:0/2427525198 learned_addr learned my addr 172.21.15.119:0/2427525198 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 -- 172.21.15.119:0/2427525198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f060 msgr2=0x7eff4016a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f060 0x7eff4016a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 -- 172.21.15.119:0/2427525198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff400cb1a0 msgr2=0x7eff400c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff400cb1a0 0x7eff400c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 -- 172.21.15.119:0/2427525198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340094f0 con 0x7eff4006f4b0 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff3effd700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f4b0 0x7eff40153220 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7eff4006ce80 tx=0x7eff34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff277fe700 1 -- 172.21.15.119:0/2427525198 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34024440 con 0x7eff4006f4b0 2023-10-08T01:14:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff277fe700 1 -- 172.21.15.119:0/2427525198 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff340245c0 con 0x7eff4006f4b0 2023-10-08T01:14:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff277fe700 1 -- 172.21.15.119:0/2427525198 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34024980 con 0x7eff4006f4b0 2023-10-08T01:14:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.822+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff4016d820 con 0x7eff4006f4b0 2023-10-08T01:14:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.823+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff40158f40 con 0x7eff4006f4b0 2023-10-08T01:14:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.823+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff4006c450 con 0x7eff4006f4b0 2023-10-08T01:14:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.824+0000 7eff277fe700 1 -- 172.21.15.119:0/2427525198 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff3401a420 con 0x7eff4006f4b0 2023-10-08T01:14:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.825+0000 7eff277fe700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff2c066f30 0x7eff2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.825+0000 7eff277fe700 1 -- 172.21.15.119:0/2427525198 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7eff340863f0 con 0x7eff4006f4b0 2023-10-08T01:14:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.825+0000 7eff3f7fe700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff2c066f30 0x7eff2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:23.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.826+0000 7eff3f7fe700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff2c066f30 0x7eff2c0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff30001690 tx=0x7eff30007b40 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:23.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:23.829+0000 7eff277fe700 1 -- 172.21.15.119:0/2427525198 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff34053430 con 0x7eff4006f4b0 2023-10-08T01:14:24.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.062+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7eff4015b000 con 0x7eff4006f4b0 2023-10-08T01:14:24.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.789+0000 7eff277fe700 1 -- 172.21.15.119:0/2427525198 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7eff34056a50 con 0x7eff4006f4b0 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.791+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff2c066f30 msgr2=0x7eff2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.791+0000 7eff44b9d700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff2c066f30 0x7eff2c0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7eff30001690 tx=0x7eff30007b40 comp rx=0 tx=0).stop 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.791+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f4b0 msgr2=0x7eff40153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.791+0000 7eff44b9d700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f4b0 0x7eff40153220 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7eff4006ce80 tx=0x7eff34017000 comp rx=0 tx=0).stop 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.792+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 shutdown_connections 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.792+0000 7eff44b9d700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff2c066f30 0x7eff2c0693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.792+0000 7eff44b9d700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff4006f060 0x7eff4016a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.792+0000 7eff44b9d700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff400cb1a0 0x7eff400c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:24.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.792+0000 7eff44b9d700 1 --2- 172.21.15.119:0/2427525198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff4006f4b0 0x7eff40153220 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:24.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.792+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 >> 172.21.15.119:0/2427525198 conn(0x7eff400cc1e0 msgr2=0x7eff400c6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:24.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.792+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 shutdown_connections 2023-10-08T01:14:24.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.792+0000 7eff44b9d700 1 -- 172.21.15.119:0/2427525198 wait complete. 2023-10-08T01:14:24.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T01:14:24.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.997+0000 7fbd83dff700 1 Processor -- start 2023-10-08T01:14:24.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.997+0000 7fbd83dff700 1 -- start start 2023-10-08T01:14:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.998+0000 7fbd83dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 0x7fbd7c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.998+0000 7fbd83dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c15a160 con 0x7fbd7c159c60 2023-10-08T01:14:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.998+0000 7fbd83dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7c0d13a0 con 0x7fbd7c069560 2023-10-08T01:14:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.998+0000 7fbd83dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c15e140 con 0x7fbd7c159890 2023-10-08T01:14:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd8139a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbd7c159c60 0x7fbd7c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59700/0 (socket says 172.21.15.119:59700) 2023-10-08T01:14:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd8139a700 1 -- 172.21.15.119:0/3788592780 learned_addr learned my addr 172.21.15.119:0/3788592780 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd81b9b700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 0x7fbd7c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624663618 0 0) 0x7fbd7c15a160 con 0x7fbd7c159c60 2023-10-08T01:14:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd68003570 con 0x7fbd7c159c60 2023-10-08T01:14:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886415880 0 0) 0x7fbd7c0d13a0 con 0x7fbd7c069560 2023-10-08T01:14:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd7c15a160 con 0x7fbd7c069560 2023-10-08T01:14:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2801449846 0 0) 0x7fbd68003570 con 0x7fbd7c159c60 2023-10-08T01:14:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 msgr2=0x7fbd7c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 0x7fbd7c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 >> v1:172.21.15.184:6789/0 conn(0x7fbd7c069560 legacy=0x7fbd7c0ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c15a3d0 con 0x7fbd7c159c60 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fbd6c001470 con 0x7fbd7c159c60 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 >> v1:172.21.15.119:6789/0 conn(0x7fbd7c159c60 legacy=0x7fbd7c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd68003c20 0x7fbd68004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 0x7fbd68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68004d50 0x7fbd680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c15e140 con 0x7fbd68003c20 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6800bbf0 con 0x7fbd68004d50 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:24.999+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6800bd60 con 0x7fbd7c159890 2023-10-08T01:14:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd81b9b700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd68003c20 0x7fbd68004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd8139a700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 0x7fbd68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd8239c700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68004d50 0x7fbd680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd81b9b700 1 -- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 msgr2=0x7fbd68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd81b9b700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 0x7fbd68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd81b9b700 1 -- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68004d50 msgr2=0x7fbd680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd81b9b700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68004d50 0x7fbd680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd81b9b700 1 -- 172.21.15.119:0/3788592780 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd68003570 con 0x7fbd68003c20 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd8139a700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c159890 0x7fbd68004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd81b9b700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd68003c20 0x7fbd68004010 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbd78006f00 tx=0x7fbd780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b158268e5f38d3cb server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.000+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800e710 con 0x7fbd68003c20 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7800e870 con 0x7fbd68003c20 2023-10-08T01:14:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd80b99700 1 -- 172.21.15.119:0/3788592780 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7800ec30 con 0x7fbd68003c20 2023-10-08T01:14:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd83dff700 1 -- 172.21.15.119:0/3788592780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd68003c20 msgr2=0x7fbd68004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd83dff700 1 --2- 172.21.15.119:0/3788592780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd68003c20 0x7fbd68004010 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fbd78006f00 tx=0x7fbd780091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd8239c700 1 -- 172.21.15.119:0/3788592780 reap_dead start 2023-10-08T01:14:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd83dff700 1 -- 172.21.15.119:0/3788592780 shutdown_connections 2023-10-08T01:14:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd83dff700 1 -- 172.21.15.119:0/3788592780 >> 172.21.15.119:0/3788592780 conn(0x7fbd7c068e20 msgr2=0x7fbd7c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd83dff700 1 -- 172.21.15.119:0/3788592780 shutdown_connections 2023-10-08T01:14:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.001+0000 7fbd83dff700 1 -- 172.21.15.119:0/3788592780 wait complete. 2023-10-08T01:14:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.003+0000 7fbd83dff700 1 Processor -- start 2023-10-08T01:14:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.003+0000 7fbd83dff700 1 -- start start 2023-10-08T01:14:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd83dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68003c20 0x7fbd7c1492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd83dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7c0b59d0 0x7fbd7c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd83dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c16a230 0x7fbd7c16a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd83dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c16b170 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd83dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0b3590 con 0x7fbd68003c20 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd83dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7c0b43f0 con 0x7fbd7c16a230 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7c0b59d0 0x7fbd7c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7c0b59d0 0x7fbd7c0b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40632/0 (socket says 172.21.15.119:40632) 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8139a700 1 -- 172.21.15.119:0/4002043676 learned_addr learned my addr 172.21.15.119:0/4002043676 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd81b9b700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68003c20 0x7fbd7c1492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8239c700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c16a230 0x7fbd7c16a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8139a700 1 -- 172.21.15.119:0/4002043676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c16a230 msgr2=0x7fbd7c16a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8139a700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c16a230 0x7fbd7c16a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8139a700 1 -- 172.21.15.119:0/4002043676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68003c20 msgr2=0x7fbd7c1492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8139a700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68003c20 0x7fbd7c1492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.004+0000 7fbd8139a700 1 -- 172.21.15.119:0/4002043676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0b4880 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.005+0000 7fbd8139a700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7c0b59d0 0x7fbd7c0b5e40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00eb90 tx=0x7fbd6c001d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.005+0000 7fbd71ffb700 1 -- 172.21.15.119:0/4002043676 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c005e90 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.005+0000 7fbd71ffb700 1 -- 172.21.15.119:0/4002043676 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c00c5d0 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.005+0000 7fbd71ffb700 1 -- 172.21.15.119:0/4002043676 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c00c970 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.005+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7c0b37e0 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.005+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7c0b3da0 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.006+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7c0b45a0 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.007+0000 7fbd71ffb700 1 -- 172.21.15.119:0/4002043676 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbd6c00cce0 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.008+0000 7fbd71ffb700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbd60066f30 0x7fbd600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.008+0000 7fbd71ffb700 1 -- 172.21.15.119:0/4002043676 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fbd6c0845c0 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.008+0000 7fbd81b9b700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbd60066f30 0x7fbd600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.009+0000 7fbd81b9b700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbd60066f30 0x7fbd600693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd780091e0 tx=0x7fbd7801a000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:25.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.012+0000 7fbd71ffb700 1 -- 172.21.15.119:0/4002043676 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd7c0b45a0 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.247+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbd7c0b4560 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.792+0000 7fbd71ffb700 1 -- 172.21.15.119:0/4002043676 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbd6c051600 con 0x7fbd7c0b59d0 2023-10-08T01:14:25.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.794+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbd60066f30 msgr2=0x7fbd600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.794+0000 7fbd83dff700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbd60066f30 0x7fbd600693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbd780091e0 tx=0x7fbd7801a000 comp rx=0 tx=0).stop 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.794+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7c0b59d0 msgr2=0x7fbd7c0b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.794+0000 7fbd83dff700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7c0b59d0 0x7fbd7c0b5e40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00eb90 tx=0x7fbd6c001d10 comp rx=0 tx=0).stop 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.794+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 shutdown_connections 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.795+0000 7fbd83dff700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbd60066f30 0x7fbd600693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.795+0000 7fbd83dff700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd7c16a230 0x7fbd7c16a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.795+0000 7fbd83dff700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd68003c20 0x7fbd7c1492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.795+0000 7fbd83dff700 1 --2- 172.21.15.119:0/4002043676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd7c0b59d0 0x7fbd7c0b5e40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.795+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 >> 172.21.15.119:0/4002043676 conn(0x7fbd7c068e20 msgr2=0x7fbd7c15ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.795+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 shutdown_connections 2023-10-08T01:14:25.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.795+0000 7fbd83dff700 1 -- 172.21.15.119:0/4002043676 wait complete. 2023-10-08T01:14:25.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:14:25.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.993+0000 7ff7577ff700 1 Processor -- start 2023-10-08T01:14:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.993+0000 7ff7577ff700 1 -- start start 2023-10-08T01:14:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff7577ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff750155c10 0x7ff750155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff7577ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff7500ac6d0 0x7ff7500b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff7577ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7500acac0 con 0x7ff7500ac230 2023-10-08T01:14:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff7577ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500ad930 con 0x7ff7500ac6d0 2023-10-08T01:14:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff7577ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500acd20 con 0x7ff750155c10 2023-10-08T01:14:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff754d9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff7500ac230 0x7ff7501568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59716/0 (socket says 172.21.15.119:59716) 2023-10-08T01:14:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff754d9a700 1 -- 172.21.15.119:0/535912994 learned_addr learned my addr 172.21.15.119:0/535912994 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff755d9c700 1 --2- 172.21.15.119:0/535912994 >> v2:172.21.15.119:3300/0 conn(0x7ff7500ac6d0 0x7ff7500b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff75559b700 1 --2- 172.21.15.119:0/535912994 >> v2:172.21.15.184:3300/0 conn(0x7ff750155c10 0x7ff750155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff755d9c700 1 -- 172.21.15.119:0/535912994 >> v2:172.21.15.184:3300/0 conn(0x7ff750155c10 msgr2=0x7ff750155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff747fff700 1 -- 172.21.15.119:0/535912994 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3505369869 0 0) 0x7ff7500acac0 con 0x7ff7500ac230 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff755d9c700 1 --2- 172.21.15.119:0/535912994 >> v2:172.21.15.184:3300/0 conn(0x7ff750155c10 0x7ff750155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff755d9c700 1 -- 172.21.15.119:0/535912994 >> v1:172.21.15.119:6789/0 conn(0x7ff7500ac230 legacy=0x7ff7501568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.995+0000 7ff755d9c700 1 -- 172.21.15.119:0/535912994 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500ad000 con 0x7ff7500ac6d0 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff755d9c700 1 --2- 172.21.15.119:0/535912994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff7500b0300 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff750063d40 tx=0x7ff748009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323ff3eb404ebb6b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff747fff700 1 -- 172.21.15.119:0/535912994 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800e070 con 0x7ff7500ac6d0 2023-10-08T01:14:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff747fff700 1 -- 172.21.15.119:0/535912994 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74800bc30 con 0x7ff7500ac6d0 2023-10-08T01:14:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff747fff700 1 -- 172.21.15.119:0/535912994 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748012430 con 0x7ff7500ac6d0 2023-10-08T01:14:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff7577ff700 1 -- 172.21.15.119:0/535912994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 msgr2=0x7ff7500b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff7577ff700 1 --2- 172.21.15.119:0/535912994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff7500b0300 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7ff750063d40 tx=0x7ff748009780 comp rx=0 tx=0).stop 2023-10-08T01:14:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff7577ff700 1 -- 172.21.15.119:0/535912994 shutdown_connections 2023-10-08T01:14:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff7577ff700 1 --2- 172.21.15.119:0/535912994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff7500b0300 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff7577ff700 1 --2- 172.21.15.119:0/535912994 >> v2:172.21.15.184:3300/0 conn(0x7ff750155c10 0x7ff750155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.996+0000 7ff7577ff700 1 -- 172.21.15.119:0/535912994 >> 172.21.15.119:0/535912994 conn(0x7ff75006a320 msgr2=0x7ff7500cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:26.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.997+0000 7ff7577ff700 1 -- 172.21.15.119:0/535912994 shutdown_connections 2023-10-08T01:14:26.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.997+0000 7ff7577ff700 1 -- 172.21.15.119:0/535912994 wait complete. 2023-10-08T01:14:26.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.998+0000 7ff7577ff700 1 Processor -- start 2023-10-08T01:14:26.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.998+0000 7ff7577ff700 1 -- start start 2023-10-08T01:14:26.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff7577ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7500ac230 0x7ff75016a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff7577ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff750146100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff7577ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff750155c10 0x7ff75016a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff7577ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750146640 con 0x7ff7500ac6d0 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff7577ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75016bc40 con 0x7ff750155c10 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff7577ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75016a760 con 0x7ff7500ac230 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff754d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff750146100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff754d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff750146100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40646/0 (socket says 172.21.15.119:40646) 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:25.999+0000 7ff754d9a700 1 -- 172.21.15.119:0/2102369414 learned_addr learned my addr 172.21.15.119:0/2102369414 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff755d9c700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff750155c10 0x7ff75016a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff75559b700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7500ac230 0x7ff75016a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff754d9a700 1 -- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7500ac230 msgr2=0x7ff75016a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff754d9a700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7500ac230 0x7ff75016a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff754d9a700 1 -- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff750155c10 msgr2=0x7ff75016a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff754d9a700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff750155c10 0x7ff75016a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff754d9a700 1 -- 172.21.15.119:0/2102369414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748009a20 con 0x7ff7500ac6d0 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff755d9c700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff750155c10 0x7ff75016a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff75559b700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7500ac230 0x7ff75016a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff754d9a700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff750146100 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff74000c810 tx=0x7ff740007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff7457fa700 1 -- 172.21.15.119:0/2102369414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740005b30 con 0x7ff7500ac6d0 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff7457fa700 1 -- 172.21.15.119:0/2102369414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff740005c90 con 0x7ff7500ac6d0 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff7457fa700 1 -- 172.21.15.119:0/2102369414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740007300 con 0x7ff7500ac6d0 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75016b030 con 0x7ff7500ac6d0 2023-10-08T01:14:26.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.000+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75016b5f0 con 0x7ff7500ac6d0 2023-10-08T01:14:26.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.001+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75016bdb0 con 0x7ff7500ac6d0 2023-10-08T01:14:26.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.002+0000 7ff7457fa700 1 -- 172.21.15.119:0/2102369414 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff74001b400 con 0x7ff7500ac6d0 2023-10-08T01:14:26.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.003+0000 7ff7457fa700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff734066f30 0x7ff7340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.003+0000 7ff7457fa700 1 -- 172.21.15.119:0/2102369414 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ff740084280 con 0x7ff7500ac6d0 2023-10-08T01:14:26.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.003+0000 7ff75559b700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff734066f30 0x7ff7340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.004+0000 7ff75559b700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff734066f30 0x7ff7340693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0075f0 tx=0x7ff74c009040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:26.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.007+0000 7ff7457fa700 1 -- 172.21.15.119:0/2102369414 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff740051340 con 0x7ff7500ac6d0 2023-10-08T01:14:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.240+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff75004c3c0 con 0x7ff7500ac6d0 2023-10-08T01:14:26.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.240+0000 7ff7457fa700 1 -- 172.21.15.119:0/2102369414 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4608 (secure 0 0 0) 0x7ff740054960 con 0x7ff7500ac6d0 2023-10-08T01:14:26.241 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:26.241 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7141,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3118853373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3118853373},{"type":"v1","addr":"172.21.15.184:6834","nonce":3118853373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7144,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/737543181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":737543181},{"type":"v1","addr":"172.21.15.184:6835","nonce":737543181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/2667608886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2667608886},{"type":"v1","addr":"172.21.15.119:6834","nonce":2667608886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:14:22.219941+0000","modified":"2023-10-08T01:14:25.788077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7680},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7680":{"gid":7680,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1713175169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1713175169},{"type":"v1","addr":"172.21.15.119:6835","nonce":1713175169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T01:14:26.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff734066f30 msgr2=0x7ff7340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff734066f30 0x7ff7340693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0075f0 tx=0x7ff74c009040 comp rx=0 tx=0).stop 2023-10-08T01:14:26.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 msgr2=0x7ff750146100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff750146100 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7ff74000c810 tx=0x7ff740007df0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 shutdown_connections 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff734066f30 0x7ff7340693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7500ac230 0x7ff75016a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff750155c10 0x7ff75016a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 --2- 172.21.15.119:0/2102369414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7500ac6d0 0x7ff750146100 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 >> 172.21.15.119:0/2102369414 conn(0x7ff75006a320 msgr2=0x7ff75006e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 shutdown_connections 2023-10-08T01:14:26.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.243+0000 7ff7577ff700 1 -- 172.21.15.119:0/2102369414 wait complete. 2023-10-08T01:14:26.245 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 55 2023-10-08T01:14:26.261 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:14:26.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.441+0000 7fbc4e99d700 1 Processor -- start 2023-10-08T01:14:26.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.441+0000 7fbc4e99d700 1 -- start start 2023-10-08T01:14:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.442+0000 7fbc4e99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc48150550 0x7fbc48157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.443+0000 7fbc4e99d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4806e2a0 con 0x7fbc48150180 2023-10-08T01:14:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.443+0000 7fbc4e99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc4806b2e0 con 0x7fbc480c7410 2023-10-08T01:14:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.443+0000 7fbc4e99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc480c78e0 con 0x7fbc48150550 2023-10-08T01:14:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.443+0000 7fbc4d99b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbc48150180 0x7fbc480c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59728/0 (socket says 172.21.15.119:59728) 2023-10-08T01:14:26.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.443+0000 7fbc4d99b700 1 -- 172.21.15.119:0/1845739303 learned_addr learned my addr 172.21.15.119:0/1845739303 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.443+0000 7fbc4d19a700 1 --2- 172.21.15.119:0/1845739303 >> v2:172.21.15.184:3300/0 conn(0x7fbc48150550 0x7fbc48157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.443+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4087417582 0 0) 0x7fbc4806e2a0 con 0x7fbc48150180 2023-10-08T01:14:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.443+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc34003570 con 0x7fbc48150180 2023-10-08T01:14:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647330079 0 0) 0x7fbc4806b2e0 con 0x7fbc480c7410 2023-10-08T01:14:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc4806e2a0 con 0x7fbc480c7410 2023-10-08T01:14:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1574511875 0 0) 0x7fbc34003570 con 0x7fbc48150180 2023-10-08T01:14:26.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 >> v2:172.21.15.184:3300/0 conn(0x7fbc48150550 msgr2=0x7fbc48157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 --2- 172.21.15.119:0/1845739303 >> v2:172.21.15.184:3300/0 conn(0x7fbc48150550 0x7fbc48157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 >> v1:172.21.15.184:6789/0 conn(0x7fbc480c7410 legacy=0x7fbc48069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc480c7bc0 con 0x7fbc48150180 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fbc44001470 con 0x7fbc48150180 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 >> v1:172.21.15.119:6789/0 conn(0x7fbc48150180 legacy=0x7fbc480c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc34003bf0 0x7fbc34003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34004810 0x7fbc34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc340051c0 0x7fbc3400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc480c78e0 con 0x7fbc34003bf0 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3400c0b0 con 0x7fbc340051c0 2023-10-08T01:14:26.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3400c270 con 0x7fbc34004810 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4d99b700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc34003bf0 0x7fbc34003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4d19a700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34004810 0x7fbc34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4e19c700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc340051c0 0x7fbc3400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4d99b700 1 -- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34004810 msgr2=0x7fbc34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4d99b700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34004810 0x7fbc34004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4d99b700 1 -- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc340051c0 msgr2=0x7fbc3400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4d99b700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc340051c0 0x7fbc3400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4d99b700 1 -- 172.21.15.119:0/1845739303 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc34003570 con 0x7fbc34003bf0 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4e19c700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc340051c0 0x7fbc3400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:26.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4e19c700 1 -- 172.21.15.119:0/1845739303 reap_dead start 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.444+0000 7fbc4d19a700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34004810 0x7fbc34004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4d99b700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc34003bf0 0x7fbc34003fe0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400c2d0 tx=0x7fbc44001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165a45d7a7d61105 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4400cba0 con 0x7fbc34003bf0 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4400cd00 con 0x7fbc34003bf0 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4c999700 1 -- 172.21.15.119:0/1845739303 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc44005b70 con 0x7fbc34003bf0 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4e99d700 1 -- 172.21.15.119:0/1845739303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc34003bf0 msgr2=0x7fbc34003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4e99d700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc34003bf0 0x7fbc34003fe0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400c2d0 tx=0x7fbc44001e10 comp rx=0 tx=0).stop 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4e99d700 1 -- 172.21.15.119:0/1845739303 shutdown_connections 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4e99d700 1 --2- 172.21.15.119:0/1845739303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc34003bf0 0x7fbc34003fe0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.445+0000 7fbc4e99d700 1 -- 172.21.15.119:0/1845739303 >> 172.21.15.119:0/1845739303 conn(0x7fbc480cc1f0 msgr2=0x7fbc480cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:26.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.446+0000 7fbc4e99d700 1 -- 172.21.15.119:0/1845739303 shutdown_connections 2023-10-08T01:14:26.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.446+0000 7fbc4e99d700 1 -- 172.21.15.119:0/1845739303 wait complete. 2023-10-08T01:14:26.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.447+0000 7fbc4e99d700 1 Processor -- start 2023-10-08T01:14:26.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.447+0000 7fbc4e99d700 1 -- start start 2023-10-08T01:14:26.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34003bf0 0x7fbc4813a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc48158b80 0x7fbc48158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480b14b0 0x7fbc480b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc480b1e60 con 0x7fbc480b14b0 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc481586f0 con 0x7fbc48158b80 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc48157ae0 con 0x7fbc34003bf0 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480b14b0 0x7fbc480b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480b14b0 0x7fbc480b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40658/0 (socket says 172.21.15.119:40658) 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.448+0000 7fbc4e19c700 1 -- 172.21.15.119:0/3000617111 learned_addr learned my addr 172.21.15.119:0/3000617111 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4d99b700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34003bf0 0x7fbc4813a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4d19a700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc48158b80 0x7fbc48158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4e19c700 1 -- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34003bf0 msgr2=0x7fbc4813a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4e19c700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34003bf0 0x7fbc4813a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4e19c700 1 -- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc48158b80 msgr2=0x7fbc48158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4e19c700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc48158b80 0x7fbc48158ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4e19c700 1 -- 172.21.15.119:0/3000617111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc48157dc0 con 0x7fbc480b14b0 2023-10-08T01:14:26.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4d99b700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34003bf0 0x7fbc4813a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:26.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4e19c700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480b14b0 0x7fbc480b1920 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fbc480c8140 tx=0x7fbc40009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc3dffb700 1 -- 172.21.15.119:0/3000617111 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40017070 con 0x7fbc480b14b0 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc3dffb700 1 -- 172.21.15.119:0/3000617111 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc400032e0 con 0x7fbc480b14b0 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc3dffb700 1 -- 172.21.15.119:0/3000617111 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40006ba0 con 0x7fbc480b14b0 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc481580a0 con 0x7fbc480b14b0 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.449+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc48157770 con 0x7fbc480b14b0 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.450+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc48158540 con 0x7fbc480b14b0 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.451+0000 7fbc3dffb700 1 -- 172.21.15.119:0/3000617111 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc400016e0 con 0x7fbc480b14b0 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.452+0000 7fbc3dffb700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc2c066f30 0x7fbc2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.452+0000 7fbc3dffb700 1 -- 172.21.15.119:0/3000617111 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fbc400842c0 con 0x7fbc480b14b0 2023-10-08T01:14:26.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.452+0000 7fbc4d99b700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc2c066f30 0x7fbc2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.453+0000 7fbc4d99b700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc2c066f30 0x7fbc2c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc44001e50 tx=0x7fbc44015000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:26.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.456+0000 7fbc3dffb700 1 -- 172.21.15.119:0/3000617111 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc40051300 con 0x7fbc480b14b0 2023-10-08T01:14:26.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.647+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc48158860 con 0x7fbc480b14b0 2023-10-08T01:14:26.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.648+0000 7fbc3dffb700 1 -- 172.21.15.119:0/3000617111 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+16009 (secure 0 0 0) 0x7fbc40054920 con 0x7fbc480b14b0 2023-10-08T01:14:26.648 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:26.648 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":120,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:23.768617+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:14:20.330219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T01:14:21.202759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc2c066f30 msgr2=0x7fbc2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc2c066f30 0x7fbc2c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc44001e50 tx=0x7fbc44015000 comp rx=0 tx=0).stop 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480b14b0 msgr2=0x7fbc480b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480b14b0 0x7fbc480b1920 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fbc480c8140 tx=0x7fbc40009040 comp rx=0 tx=0).stop 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 shutdown_connections 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc2c066f30 0x7fbc2c0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc34003bf0 0x7fbc4813a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480b14b0 0x7fbc480b1920 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 --2- 172.21.15.119:0/3000617111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc48158b80 0x7fbc48158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.650+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 >> 172.21.15.119:0/3000617111 conn(0x7fbc480cc1f0 msgr2=0x7fbc480cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.651+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 shutdown_connections 2023-10-08T01:14:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.651+0000 7fbc4e99d700 1 -- 172.21.15.119:0/3000617111 wait complete. 2023-10-08T01:14:26.668 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T01:14:26.668 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:14:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.851+0000 7f1b5dd9d700 1 Processor -- start 2023-10-08T01:14:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.851+0000 7f1b5dd9d700 1 -- start start 2023-10-08T01:14:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.852+0000 7f1b5dd9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b58150550 0x7f1b58157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.852+0000 7f1b5dd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b580c7410 0x7f1b58069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.852+0000 7f1b5dd9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5806e2a0 con 0x7f1b58150180 2023-10-08T01:14:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.852+0000 7f1b5dd9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5806b2e0 con 0x7f1b58150550 2023-10-08T01:14:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.852+0000 7f1b5dd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c78e0 con 0x7f1b580c7410 2023-10-08T01:14:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b57fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b58150550 0x7f1b58157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b57fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1b58150550 0x7f1b58157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40664/0 (socket says 172.21.15.119:40664) 2023-10-08T01:14:26.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b57fff700 1 -- 172.21.15.119:0/1060847115 learned_addr learned my addr 172.21.15.119:0/1060847115 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:26.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b5d59c700 1 --2- 172.21.15.119:0/1060847115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b580c7410 0x7f1b58069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b57fff700 1 -- 172.21.15.119:0/1060847115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b580c7410 msgr2=0x7f1b58069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:26.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b57fff700 1 --2- 172.21.15.119:0/1060847115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b580c7410 0x7f1b58069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b57fff700 1 -- 172.21.15.119:0/1060847115 >> v1:172.21.15.184:6789/0 conn(0x7f1b58150180 legacy=0x7f1b580c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b57fff700 1 -- 172.21.15.119:0/1060847115 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b580c7bc0 con 0x7f1b58150550 2023-10-08T01:14:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.853+0000 7f1b57fff700 1 --2- 172.21.15.119:0/1060847115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150550 0x7f1b58157770 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1b440099b0 tx=0x7f1b44009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51448d72123392e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b577fe700 1 -- 172.21.15.119:0/1060847115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400b860 con 0x7f1b58150550 2023-10-08T01:14:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b577fe700 1 -- 172.21.15.119:0/1060847115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4400b9c0 con 0x7f1b58150550 2023-10-08T01:14:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b577fe700 1 -- 172.21.15.119:0/1060847115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4400bd60 con 0x7f1b58150550 2023-10-08T01:14:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/1060847115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150550 msgr2=0x7f1b58157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/1060847115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150550 0x7f1b58157770 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f1b440099b0 tx=0x7f1b44009250 comp rx=0 tx=0).stop 2023-10-08T01:14:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/1060847115 shutdown_connections 2023-10-08T01:14:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/1060847115 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b580c7410 0x7f1b58069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/1060847115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150550 0x7f1b58157770 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/1060847115 >> 172.21.15.119:0/1060847115 conn(0x7f1b580cc1f0 msgr2=0x7f1b580cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/1060847115 shutdown_connections 2023-10-08T01:14:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.854+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/1060847115 wait complete. 2023-10-08T01:14:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.856+0000 7f1b5dd9d700 1 Processor -- start 2023-10-08T01:14:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.856+0000 7f1b5dd9d700 1 -- start start 2023-10-08T01:14:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b5dd9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b580c7410 0x7f1b58165e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b5dd9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150180 0x7f1b58168190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b5dd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b58150550 0x7f1b58166520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b5dd9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b58166380 con 0x7f1b58150180 2023-10-08T01:14:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b5dd9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580c3930 con 0x7f1b580c7410 2023-10-08T01:14:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b5dd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b580ad0a0 con 0x7f1b58150550 2023-10-08T01:14:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150180 0x7f1b58168190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b57fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150180 0x7f1b58168190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40666/0 (socket says 172.21.15.119:40666) 2023-10-08T01:14:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b57fff700 1 -- 172.21.15.119:0/2949871901 learned_addr learned my addr 172.21.15.119:0/2949871901 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b5d59c700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b58150550 0x7f1b58166520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.857+0000 7f1b5cd9b700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b580c7410 0x7f1b58165e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b57fff700 1 -- 172.21.15.119:0/2949871901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b58150550 msgr2=0x7f1b58166520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b57fff700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b58150550 0x7f1b58166520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b57fff700 1 -- 172.21.15.119:0/2949871901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b580c7410 msgr2=0x7f1b58165e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b57fff700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b580c7410 0x7f1b58165e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b57fff700 1 -- 172.21.15.119:0/2949871901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440094f0 con 0x7f1b58150180 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b57fff700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150180 0x7f1b58168190 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1b5806fe40 tx=0x7f1b44017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b54ff9700 1 -- 172.21.15.119:0/2949871901 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44024440 con 0x7f1b58150180 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b54ff9700 1 -- 172.21.15.119:0/2949871901 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b440245a0 con 0x7f1b58150180 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b580ad380 con 0x7f1b58150180 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b54ff9700 1 -- 172.21.15.119:0/2949871901 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b44024960 con 0x7f1b58150180 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.858+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b580ad940 con 0x7f1b58150180 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.859+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b580adb30 con 0x7f1b58150180 2023-10-08T01:14:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.860+0000 7f1b54ff9700 1 -- 172.21.15.119:0/2949871901 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b4401a420 con 0x7f1b58150180 2023-10-08T01:14:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.861+0000 7f1b54ff9700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b3c066f30 0x7f1b3c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.861+0000 7f1b54ff9700 1 -- 172.21.15.119:0/2949871901 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f1b440865d0 con 0x7f1b58150180 2023-10-08T01:14:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.861+0000 7f1b5cd9b700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b3c066f30 0x7f1b3c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.862+0000 7f1b5cd9b700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b3c066f30 0x7f1b3c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b580cc010 tx=0x7f1b4c009040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:26.865+0000 7f1b54ff9700 1 -- 172.21.15.119:0/2949871901 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b44053610 con 0x7f1b58150180 2023-10-08T01:14:27.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.054+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b5804c3c0 con 0x7f1b3c066f30 2023-10-08T01:14:27.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.064+0000 7f1b54ff9700 1 -- 172.21.15.119:0/2949871901 <== mgr.7488 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284704 (secure 0 0 0) 0x7f1b5804c3c0 con 0x7f1b3c066f30 2023-10-08T01:14:27.070 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b3c066f30 msgr2=0x7f1b3c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b3c066f30 0x7f1b3c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1b580cc010 tx=0x7f1b4c009040 comp rx=0 tx=0).stop 2023-10-08T01:14:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150180 msgr2=0x7f1b58168190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150180 0x7f1b58168190 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1b5806fe40 tx=0x7f1b44017000 comp rx=0 tx=0).stop 2023-10-08T01:14:27.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 shutdown_connections 2023-10-08T01:14:27.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1b3c066f30 0x7f1b3c0693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1b58150550 0x7f1b58166520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1b580c7410 0x7f1b58165e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 --2- 172.21.15.119:0/2949871901 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1b58150180 0x7f1b58168190 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 >> 172.21.15.119:0/2949871901 conn(0x7f1b580cc1f0 msgr2=0x7f1b580c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 shutdown_connections 2023-10-08T01:14:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.068+0000 7f1b5dd9d700 1 -- 172.21.15.119:0/2949871901 wait complete. 2023-10-08T01:14:27.077 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:14:27.085 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:14:26.707054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592492,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":394,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":138,"acting":138,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444168,"kb_used_data":12616,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752530552,"statfs":{"total":773094113280,"available":770591285248,"internally_reserved":0,"allocated":12918784,"data_stored":6897355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2124,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.965050"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798738+0000","last_change":"2023-10-08T01:14:22.797353+0000","last_active":"2023-10-08T01:14:22.798738+0000","last_peered":"2023-10-08T01:14:22.798738+0000","last_clean":"2023-10-08T01:14:22.798738+0000","last_became_active":"2023-10-08T01:14:22.795282+0000","last_became_peered":"2023-10-08T01:14:22.795282+0000","last_unstale":"2023-10-08T01:14:22.798738+0000","last_undegraded":"2023-10-08T01:14:22.798738+0000","last_fullsized":"2023-10-08T01:14:22.798738+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:19.889654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789827+0000","last_change":"2023-10-08T01:14:22.782483+0000","last_active":"2023-10-08T01:14:22.789827+0000","last_peered":"2023-10-08T01:14:22.789827+0000","last_clean":"2023-10-08T01:14:22.789827+0000","last_became_active":"2023-10-08T01:14:22.781426+0000","last_became_peered":"2023-10-08T01:14:22.781426+0000","last_unstale":"2023-10-08T01:14:22.789827+0000","last_undegraded":"2023-10-08T01:14:22.789827+0000","last_fullsized":"2023-10-08T01:14:22.789827+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:59.712158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794183+0000","last_change":"2023-10-08T01:14:22.786530+0000","last_active":"2023-10-08T01:14:22.794183+0000","last_peered":"2023-10-08T01:14:22.794183+0000","last_clean":"2023-10-08T01:14:22.794183+0000","last_became_active":"2023-10-08T01:14:22.786260+0000","last_became_peered":"2023-10-08T01:14:22.786260+0000","last_unstale":"2023-10-08T01:14:22.794183+0000","last_undegraded":"2023-10-08T01:14:22.794183+0000","last_fullsized":"2023-10-08T01:14:22.794183+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:25.960964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788496+0000","last_change":"2023-10-08T01:14:22.783965+0000","last_active":"2023-10-08T01:14:22.788496+0000","last_peered":"2023-10-08T01:14:22.788496+0000","last_clean":"2023-10-08T01:14:22.788496+0000","last_became_active":"2023-10-08T01:14:22.783699+0000","last_became_peered":"2023-10-08T01:14:22.783699+0000","last_unstale":"2023-10-08T01:14:22.788496+0000","last_undegraded":"2023-10-08T01:14:22.788496+0000","last_fullsized":"2023-10-08T01:14:22.788496+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:06.030874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792319+0000","last_change":"2023-10-08T01:14:22.788581+0000","last_active":"2023-10-08T01:14:22.792319+0000","last_peered":"2023-10-08T01:14:22.792319+0000","last_clean":"2023-10-08T01:14:22.792319+0000","last_became_active":"2023-10-08T01:14:22.788210+0000","last_became_peered":"2023-10-08T01:14:22.788210+0000","last_unstale":"2023-10-08T01:14:22.792319+0000","last_undegraded":"2023-10-08T01:14:22.792319+0000","last_fullsized":"2023-10-08T01:14:22.792319+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792484+0000","last_change":"2023-10-08T01:14:22.790486+0000","last_active":"2023-10-08T01:14:22.792484+0000","last_peered":"2023-10-08T01:14:22.792484+0000","last_clean":"2023-10-08T01:14:22.792484+0000","last_became_active":"2023-10-08T01:14:22.790175+0000","last_became_peered":"2023-10-08T01:14:22.790175+0000","last_unstale":"2023-10-08T01:14:22.792484+0000","last_undegraded":"2023-10-08T01:14:22.792484+0000","last_fullsized":"2023-10-08T01:14:22.792484+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.779024+0000","last_change":"2023-10-08T01:14:22.777745+0000","last_active":"2023-10-08T01:14:22.779024+0000","last_peered":"2023-10-08T01:14:22.779024+0000","last_clean":"2023-10-08T01:14:22.779024+0000","last_became_active":"2023-10-08T01:14:22.777473+0000","last_became_peered":"2023-10-08T01:14:22.777473+0000","last_unstale":"2023-10-08T01:14:22.779024+0000","last_undegraded":"2023-10-08T01:14:22.779024+0000","last_fullsized":"2023-10-08T01:14:22.779024+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:57.315090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788635+0000","last_change":"2023-10-08T01:14:22.782436+0000","last_active":"2023-10-08T01:14:22.788635+0000","last_peered":"2023-10-08T01:14:22.788635+0000","last_clean":"2023-10-08T01:14:22.788635+0000","last_became_active":"2023-10-08T01:14:22.782125+0000","last_became_peered":"2023-10-08T01:14:22.782125+0000","last_unstale":"2023-10-08T01:14:22.788635+0000","last_undegraded":"2023-10-08T01:14:22.788635+0000","last_fullsized":"2023-10-08T01:14:22.788635+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:26.317877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792306+0000","last_change":"2023-10-08T01:14:22.789825+0000","last_active":"2023-10-08T01:14:22.792306+0000","last_peered":"2023-10-08T01:14:22.792306+0000","last_clean":"2023-10-08T01:14:22.792306+0000","last_became_active":"2023-10-08T01:14:22.789560+0000","last_became_peered":"2023-10-08T01:14:22.789560+0000","last_unstale":"2023-10-08T01:14:22.792306+0000","last_undegraded":"2023-10-08T01:14:22.792306+0000","last_fullsized":"2023-10-08T01:14:22.792306+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792179+0000","last_change":"2023-10-08T01:14:22.788302+0000","last_active":"2023-10-08T01:14:22.792179+0000","last_peered":"2023-10-08T01:14:22.792179+0000","last_clean":"2023-10-08T01:14:22.792179+0000","last_became_active":"2023-10-08T01:14:22.787799+0000","last_became_peered":"2023-10-08T01:14:22.787799+0000","last_unstale":"2023-10-08T01:14:22.792179+0000","last_undegraded":"2023-10-08T01:14:22.792179+0000","last_fullsized":"2023-10-08T01:14:22.792179+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783436+0000","last_change":"2023-10-08T01:14:22.779249+0000","last_active":"2023-10-08T01:14:22.783436+0000","last_peered":"2023-10-08T01:14:22.783436+0000","last_clean":"2023-10-08T01:14:22.783436+0000","last_became_active":"2023-10-08T01:14:22.779000+0000","last_became_peered":"2023-10-08T01:14:22.779000+0000","last_unstale":"2023-10-08T01:14:22.783436+0000","last_undegraded":"2023-10-08T01:14:22.783436+0000","last_fullsized":"2023-10-08T01:14:22.783436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:44:52.899739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788599+0000","last_change":"2023-10-08T01:14:22.782182+0000","last_active":"2023-10-08T01:14:22.788599+0000","last_peered":"2023-10-08T01:14:22.788599+0000","last_clean":"2023-10-08T01:14:22.788599+0000","last_became_active":"2023-10-08T01:14:22.781939+0000","last_became_peered":"2023-10-08T01:14:22.781939+0000","last_unstale":"2023-10-08T01:14:22.788599+0000","last_undegraded":"2023-10-08T01:14:22.788599+0000","last_fullsized":"2023-10-08T01:14:22.788599+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:19.690095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787156+0000","last_change":"2023-10-08T01:14:22.780281+0000","last_active":"2023-10-08T01:14:22.787156+0000","last_peered":"2023-10-08T01:14:22.787156+0000","last_clean":"2023-10-08T01:14:22.787156+0000","last_became_active":"2023-10-08T01:14:22.780033+0000","last_became_peered":"2023-10-08T01:14:22.780033+0000","last_unstale":"2023-10-08T01:14:22.787156+0000","last_undegraded":"2023-10-08T01:14:22.787156+0000","last_fullsized":"2023-10-08T01:14:22.787156+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:41.656567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799940+0000","last_change":"2023-10-08T01:14:22.799140+0000","last_active":"2023-10-08T01:14:22.799940+0000","last_peered":"2023-10-08T01:14:22.799940+0000","last_clean":"2023-10-08T01:14:22.799940+0000","last_became_active":"2023-10-08T01:14:22.798851+0000","last_became_peered":"2023-10-08T01:14:22.798851+0000","last_unstale":"2023-10-08T01:14:22.799940+0000","last_undegraded":"2023-10-08T01:14:22.799940+0000","last_fullsized":"2023-10-08T01:14:22.799940+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:30.232699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789675+0000","last_change":"2023-10-08T01:14:22.788999+0000","last_active":"2023-10-08T01:14:22.789675+0000","last_peered":"2023-10-08T01:14:22.789675+0000","last_clean":"2023-10-08T01:14:22.789675+0000","last_became_active":"2023-10-08T01:14:22.787618+0000","last_became_peered":"2023-10-08T01:14:22.787618+0000","last_unstale":"2023-10-08T01:14:22.789675+0000","last_undegraded":"2023-10-08T01:14:22.789675+0000","last_fullsized":"2023-10-08T01:14:22.789675+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:17.572620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791070+0000","last_change":"2023-10-08T01:14:22.788072+0000","last_active":"2023-10-08T01:14:22.791070+0000","last_peered":"2023-10-08T01:14:22.791070+0000","last_clean":"2023-10-08T01:14:22.791070+0000","last_became_active":"2023-10-08T01:14:22.787677+0000","last_became_peered":"2023-10-08T01:14:22.787677+0000","last_unstale":"2023-10-08T01:14:22.791070+0000","last_undegraded":"2023-10-08T01:14:22.791070+0000","last_fullsized":"2023-10-08T01:14:22.791070+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:19:12.933240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798256+0000","last_change":"2023-10-08T01:14:22.794783+0000","last_active":"2023-10-08T01:14:22.798256+0000","last_peered":"2023-10-08T01:14:22.798256+0000","last_clean":"2023-10-08T01:14:22.798256+0000","last_became_active":"2023-10-08T01:14:22.794304+0000","last_became_peered":"2023-10-08T01:14:22.794304+0000","last_unstale":"2023-10-08T01:14:22.798256+0000","last_undegraded":"2023-10-08T01:14:22.798256+0000","last_fullsized":"2023-10-08T01:14:22.798256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:59.697433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783465+0000","last_change":"2023-10-08T01:14:22.779446+0000","last_active":"2023-10-08T01:14:22.783465+0000","last_peered":"2023-10-08T01:14:22.783465+0000","last_clean":"2023-10-08T01:14:22.783465+0000","last_became_active":"2023-10-08T01:14:22.778675+0000","last_became_peered":"2023-10-08T01:14:22.778675+0000","last_unstale":"2023-10-08T01:14:22.783465+0000","last_undegraded":"2023-10-08T01:14:22.783465+0000","last_fullsized":"2023-10-08T01:14:22.783465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:34.335870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795814+0000","last_change":"2023-10-08T01:14:21.755008+0000","last_active":"2023-10-08T01:14:22.795814+0000","last_peered":"2023-10-08T01:14:22.795814+0000","last_clean":"2023-10-08T01:14:22.795814+0000","last_became_active":"2023-10-08T01:14:21.754712+0000","last_became_peered":"2023-10-08T01:14:21.754712+0000","last_unstale":"2023-10-08T01:14:22.795814+0000","last_undegraded":"2023-10-08T01:14:22.795814+0000","last_fullsized":"2023-10-08T01:14:22.795814+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:24.330581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748043+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799279+0000","last_change":"2023-10-08T01:14:22.798147+0000","last_active":"2023-10-08T01:14:22.799279+0000","last_peered":"2023-10-08T01:14:22.799279+0000","last_clean":"2023-10-08T01:14:22.799279+0000","last_became_active":"2023-10-08T01:14:22.797856+0000","last_became_peered":"2023-10-08T01:14:22.797856+0000","last_unstale":"2023-10-08T01:14:22.799279+0000","last_undegraded":"2023-10-08T01:14:22.799279+0000","last_fullsized":"2023-10-08T01:14:22.799279+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:32.952368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793353+0000","last_change":"2023-10-08T01:14:21.758907+0000","last_active":"2023-10-08T01:14:22.793353+0000","last_peered":"2023-10-08T01:14:22.793353+0000","last_clean":"2023-10-08T01:14:22.793353+0000","last_became_active":"2023-10-08T01:14:21.758669+0000","last_became_peered":"2023-10-08T01:14:21.758669+0000","last_unstale":"2023-10-08T01:14:22.793353+0000","last_undegraded":"2023-10-08T01:14:22.793353+0000","last_fullsized":"2023-10-08T01:14:22.793353+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:33.791532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782110+0000","last_change":"2023-10-08T01:14:22.777591+0000","last_active":"2023-10-08T01:14:22.782110+0000","last_peered":"2023-10-08T01:14:22.782110+0000","last_clean":"2023-10-08T01:14:22.782110+0000","last_became_active":"2023-10-08T01:14:22.777315+0000","last_became_peered":"2023-10-08T01:14:22.777315+0000","last_unstale":"2023-10-08T01:14:22.782110+0000","last_undegraded":"2023-10-08T01:14:22.782110+0000","last_fullsized":"2023-10-08T01:14:22.782110+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793240+0000","last_change":"2023-10-08T01:14:21.770043+0000","last_active":"2023-10-08T01:14:22.793240+0000","last_peered":"2023-10-08T01:14:22.793240+0000","last_clean":"2023-10-08T01:14:22.793240+0000","last_became_active":"2023-10-08T01:14:21.769799+0000","last_became_peered":"2023-10-08T01:14:21.769799+0000","last_unstale":"2023-10-08T01:14:22.793240+0000","last_undegraded":"2023-10-08T01:14:22.793240+0000","last_fullsized":"2023-10-08T01:14:22.793240+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:29.304154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.786306+0000","last_change":"2023-10-08T01:14:21.767177+0000","last_active":"2023-10-08T01:14:22.786306+0000","last_peered":"2023-10-08T01:14:22.786306+0000","last_clean":"2023-10-08T01:14:22.786306+0000","last_became_active":"2023-10-08T01:14:21.766891+0000","last_became_peered":"2023-10-08T01:14:21.766891+0000","last_unstale":"2023-10-08T01:14:22.786306+0000","last_undegraded":"2023-10-08T01:14:22.786306+0000","last_fullsized":"2023-10-08T01:14:22.786306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:48.667974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798893+0000","last_change":"2023-10-08T01:14:21.763020+0000","last_active":"2023-10-08T01:14:22.798893+0000","last_peered":"2023-10-08T01:14:22.798893+0000","last_clean":"2023-10-08T01:14:22.798893+0000","last_became_active":"2023-10-08T01:14:21.762782+0000","last_became_peered":"2023-10-08T01:14:21.762782+0000","last_unstale":"2023-10-08T01:14:22.798893+0000","last_undegraded":"2023-10-08T01:14:22.798893+0000","last_fullsized":"2023-10-08T01:14:22.798893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:59.904749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745429+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:59.004815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789574+0000","last_change":"2023-10-08T01:14:22.783171+0000","last_active":"2023-10-08T01:14:22.789574+0000","last_peered":"2023-10-08T01:14:22.789574+0000","last_clean":"2023-10-08T01:14:22.789574+0000","last_became_active":"2023-10-08T01:14:22.782872+0000","last_became_peered":"2023-10-08T01:14:22.782872+0000","last_unstale":"2023-10-08T01:14:22.789574+0000","last_undegraded":"2023-10-08T01:14:22.789574+0000","last_fullsized":"2023-10-08T01:14:22.789574+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.780252+0000","last_change":"2023-10-08T01:14:21.766845+0000","last_active":"2023-10-08T01:14:22.780252+0000","last_peered":"2023-10-08T01:14:22.780252+0000","last_clean":"2023-10-08T01:14:22.780252+0000","last_became_active":"2023-10-08T01:14:21.765270+0000","last_became_peered":"2023-10-08T01:14:21.765270+0000","last_unstale":"2023-10-08T01:14:22.780252+0000","last_undegraded":"2023-10-08T01:14:22.780252+0000","last_fullsized":"2023-10-08T01:14:22.780252+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:33.330389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789845+0000","last_change":"2023-10-08T01:14:22.786181+0000","last_active":"2023-10-08T01:14:22.789845+0000","last_peered":"2023-10-08T01:14:22.789845+0000","last_clean":"2023-10-08T01:14:22.789845+0000","last_became_active":"2023-10-08T01:14:22.785912+0000","last_became_peered":"2023-10-08T01:14:22.785912+0000","last_unstale":"2023-10-08T01:14:22.789845+0000","last_undegraded":"2023-10-08T01:14:22.789845+0000","last_fullsized":"2023-10-08T01:14:22.789845+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:33.304186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.781565+0000","last_change":"2023-10-08T01:14:21.763195+0000","last_active":"2023-10-08T01:14:22.781565+0000","last_peered":"2023-10-08T01:14:22.781565+0000","last_clean":"2023-10-08T01:14:22.781565+0000","last_became_active":"2023-10-08T01:14:21.762953+0000","last_became_peered":"2023-10-08T01:14:21.762953+0000","last_unstale":"2023-10-08T01:14:22.781565+0000","last_undegraded":"2023-10-08T01:14:22.781565+0000","last_fullsized":"2023-10-08T01:14:22.781565+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:56.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789148+0000","last_change":"2023-10-08T01:14:22.782447+0000","last_active":"2023-10-08T01:14:22.789148+0000","last_peered":"2023-10-08T01:14:22.789148+0000","last_clean":"2023-10-08T01:14:22.789148+0000","last_became_active":"2023-10-08T01:14:22.782195+0000","last_became_peered":"2023-10-08T01:14:22.782195+0000","last_unstale":"2023-10-08T01:14:22.789148+0000","last_undegraded":"2023-10-08T01:14:22.789148+0000","last_fullsized":"2023-10-08T01:14:22.789148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745607+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748365+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.800832+0000","last_change":"2023-10-08T01:14:21.764918+0000","last_active":"2023-10-08T01:14:22.800832+0000","last_peered":"2023-10-08T01:14:22.800832+0000","last_clean":"2023-10-08T01:14:22.800832+0000","last_became_active":"2023-10-08T01:14:21.763653+0000","last_became_peered":"2023-10-08T01:14:21.763653+0000","last_unstale":"2023-10-08T01:14:22.800832+0000","last_undegraded":"2023-10-08T01:14:22.800832+0000","last_fullsized":"2023-10-08T01:14:22.800832+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793156+0000","last_change":"2023-10-08T01:14:21.769762+0000","last_active":"2023-10-08T01:14:22.793156+0000","last_peered":"2023-10-08T01:14:22.793156+0000","last_clean":"2023-10-08T01:14:22.793156+0000","last_became_active":"2023-10-08T01:14:21.769522+0000","last_became_peered":"2023-10-08T01:14:21.769522+0000","last_unstale":"2023-10-08T01:14:22.793156+0000","last_undegraded":"2023-10-08T01:14:22.793156+0000","last_fullsized":"2023-10-08T01:14:22.793156+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:27.726884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797374+0000","last_change":"2023-10-08T01:14:22.791650+0000","last_active":"2023-10-08T01:14:22.797374+0000","last_peered":"2023-10-08T01:14:22.797374+0000","last_clean":"2023-10-08T01:14:22.797374+0000","last_became_active":"2023-10-08T01:14:22.791109+0000","last_became_peered":"2023-10-08T01:14:22.791109+0000","last_unstale":"2023-10-08T01:14:22.797374+0000","last_undegraded":"2023-10-08T01:14:22.797374+0000","last_fullsized":"2023-10-08T01:14:22.797374+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765608+0000","last_change":"2023-10-08T01:14:21.768459+0000","last_active":"2023-10-08T01:14:22.765608+0000","last_peered":"2023-10-08T01:14:22.765608+0000","last_clean":"2023-10-08T01:14:22.765608+0000","last_became_active":"2023-10-08T01:14:21.768189+0000","last_became_peered":"2023-10-08T01:14:21.768189+0000","last_unstale":"2023-10-08T01:14:22.765608+0000","last_undegraded":"2023-10-08T01:14:22.765608+0000","last_fullsized":"2023-10-08T01:14:22.765608+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:54.324955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789515+0000","last_change":"2023-10-08T01:14:22.781160+0000","last_active":"2023-10-08T01:14:22.789515+0000","last_peered":"2023-10-08T01:14:22.789515+0000","last_clean":"2023-10-08T01:14:22.789515+0000","last_became_active":"2023-10-08T01:14:22.780893+0000","last_became_peered":"2023-10-08T01:14:22.780893+0000","last_unstale":"2023-10-08T01:14:22.789515+0000","last_undegraded":"2023-10-08T01:14:22.789515+0000","last_fullsized":"2023-10-08T01:14:22.789515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:30.309636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768642+0000","last_change":"2023-10-08T01:14:21.760294+0000","last_active":"2023-10-08T01:14:22.768642+0000","last_peered":"2023-10-08T01:14:22.768642+0000","last_clean":"2023-10-08T01:14:22.768642+0000","last_became_active":"2023-10-08T01:14:21.758928+0000","last_became_peered":"2023-10-08T01:14:21.758928+0000","last_unstale":"2023-10-08T01:14:22.768642+0000","last_undegraded":"2023-10-08T01:14:22.768642+0000","last_fullsized":"2023-10-08T01:14:22.768642+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788009+0000","last_change":"2023-10-08T01:14:22.784690+0000","last_active":"2023-10-08T01:14:22.788009+0000","last_peered":"2023-10-08T01:14:22.788009+0000","last_clean":"2023-10-08T01:14:22.788009+0000","last_became_active":"2023-10-08T01:14:22.784408+0000","last_became_peered":"2023-10-08T01:14:22.784408+0000","last_unstale":"2023-10-08T01:14:22.788009+0000","last_undegraded":"2023-10-08T01:14:22.788009+0000","last_fullsized":"2023-10-08T01:14:22.788009+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:55.180105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795020+0000","last_change":"2023-10-08T01:14:21.765533+0000","last_active":"2023-10-08T01:14:22.795020+0000","last_peered":"2023-10-08T01:14:22.795020+0000","last_clean":"2023-10-08T01:14:22.795020+0000","last_became_active":"2023-10-08T01:14:21.764914+0000","last_became_peered":"2023-10-08T01:14:21.764914+0000","last_unstale":"2023-10-08T01:14:22.795020+0000","last_undegraded":"2023-10-08T01:14:22.795020+0000","last_fullsized":"2023-10-08T01:14:22.795020+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:00.128756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797139+0000","last_change":"2023-10-08T01:14:21.767123+0000","last_active":"2023-10-08T01:14:22.797139+0000","last_peered":"2023-10-08T01:14:22.797139+0000","last_clean":"2023-10-08T01:14:22.797139+0000","last_became_active":"2023-10-08T01:14:21.766826+0000","last_became_peered":"2023-10-08T01:14:21.766826+0000","last_unstale":"2023-10-08T01:14:22.797139+0000","last_undegraded":"2023-10-08T01:14:22.797139+0000","last_fullsized":"2023-10-08T01:14:22.797139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745225+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:13.819182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.767122+0000","last_change":"2023-10-08T01:14:21.769975+0000","last_active":"2023-10-08T01:14:22.767122+0000","last_peered":"2023-10-08T01:14:22.767122+0000","last_clean":"2023-10-08T01:14:22.767122+0000","last_became_active":"2023-10-08T01:14:21.769477+0000","last_became_peered":"2023-10-08T01:14:21.769477+0000","last_unstale":"2023-10-08T01:14:22.767122+0000","last_undegraded":"2023-10-08T01:14:22.767122+0000","last_fullsized":"2023-10-08T01:14:22.767122+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:34.177414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787894+0000","last_change":"2023-10-08T01:14:22.781041+0000","last_active":"2023-10-08T01:14:22.787894+0000","last_peered":"2023-10-08T01:14:22.787894+0000","last_clean":"2023-10-08T01:14:22.787894+0000","last_became_active":"2023-10-08T01:14:22.780765+0000","last_became_peered":"2023-10-08T01:14:22.780765+0000","last_unstale":"2023-10-08T01:14:22.787894+0000","last_undegraded":"2023-10-08T01:14:22.787894+0000","last_fullsized":"2023-10-08T01:14:22.787894+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:00.278431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792713+0000","last_change":"2023-10-08T01:14:21.768612+0000","last_active":"2023-10-08T01:14:22.792713+0000","last_peered":"2023-10-08T01:14:22.792713+0000","last_clean":"2023-10-08T01:14:22.792713+0000","last_became_active":"2023-10-08T01:14:21.767991+0000","last_became_peered":"2023-10-08T01:14:21.767991+0000","last_unstale":"2023-10-08T01:14:22.792713+0000","last_undegraded":"2023-10-08T01:14:22.792713+0000","last_fullsized":"2023-10-08T01:14:22.792713+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:50.772143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788094+0000","last_change":"2023-10-08T01:14:22.783198+0000","last_active":"2023-10-08T01:14:22.788094+0000","last_peered":"2023-10-08T01:14:22.788094+0000","last_clean":"2023-10-08T01:14:22.788094+0000","last_became_active":"2023-10-08T01:14:22.782950+0000","last_became_peered":"2023-10-08T01:14:22.782950+0000","last_unstale":"2023-10-08T01:14:22.788094+0000","last_undegraded":"2023-10-08T01:14:22.788094+0000","last_fullsized":"2023-10-08T01:14:22.788094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:30:30.559893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749667+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:31.946887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788041+0000","last_change":"2023-10-08T01:14:22.782578+0000","last_active":"2023-10-08T01:14:22.788041+0000","last_peered":"2023-10-08T01:14:22.788041+0000","last_clean":"2023-10-08T01:14:22.788041+0000","last_became_active":"2023-10-08T01:14:22.781703+0000","last_became_peered":"2023-10-08T01:14:22.781703+0000","last_unstale":"2023-10-08T01:14:22.788041+0000","last_undegraded":"2023-10-08T01:14:22.788041+0000","last_fullsized":"2023-10-08T01:14:22.788041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:38.983859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749808+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:13.819182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.763121+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.762883+0000","last_became_peered":"2023-10-08T01:14:21.762883+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:03.018597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789807+0000","last_change":"2023-10-08T01:14:22.782302+0000","last_active":"2023-10-08T01:14:22.789807+0000","last_peered":"2023-10-08T01:14:22.789807+0000","last_clean":"2023-10-08T01:14:22.789807+0000","last_became_active":"2023-10-08T01:14:22.780497+0000","last_became_peered":"2023-10-08T01:14:22.780497+0000","last_unstale":"2023-10-08T01:14:22.789807+0000","last_undegraded":"2023-10-08T01:14:22.789807+0000","last_fullsized":"2023-10-08T01:14:22.789807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:59.391189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749695+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:10.276377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798148+0000","last_change":"2023-10-08T01:14:22.793832+0000","last_active":"2023-10-08T01:14:22.798148+0000","last_peered":"2023-10-08T01:14:22.798148+0000","last_clean":"2023-10-08T01:14:22.798148+0000","last_became_active":"2023-10-08T01:14:22.792633+0000","last_became_peered":"2023-10-08T01:14:22.792633+0000","last_unstale":"2023-10-08T01:14:22.798148+0000","last_undegraded":"2023-10-08T01:14:22.798148+0000","last_fullsized":"2023-10-08T01:14:22.798148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:11.281859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768046+0000","last_change":"2023-10-08T01:14:21.763932+0000","last_active":"2023-10-08T01:14:22.768046+0000","last_peered":"2023-10-08T01:14:22.768046+0000","last_clean":"2023-10-08T01:14:22.768046+0000","last_became_active":"2023-10-08T01:14:21.763673+0000","last_became_peered":"2023-10-08T01:14:21.763673+0000","last_unstale":"2023-10-08T01:14:22.768046+0000","last_undegraded":"2023-10-08T01:14:22.768046+0000","last_fullsized":"2023-10-08T01:14:22.768046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:07.616557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797649+0000","last_change":"2023-10-08T01:14:22.791844+0000","last_active":"2023-10-08T01:14:22.797649+0000","last_peered":"2023-10-08T01:14:22.797649+0000","last_clean":"2023-10-08T01:14:22.797649+0000","last_became_active":"2023-10-08T01:14:22.791583+0000","last_became_peered":"2023-10-08T01:14:22.791583+0000","last_unstale":"2023-10-08T01:14:22.797649+0000","last_undegraded":"2023-10-08T01:14:22.797649+0000","last_fullsized":"2023-10-08T01:14:22.797649+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.770984+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.770705+0000","last_became_peered":"2023-10-08T01:14:21.770705+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:48:20.485203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745127+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:10.276377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788178+0000","last_change":"2023-10-08T01:14:22.784755+0000","last_active":"2023-10-08T01:14:22.788178+0000","last_peered":"2023-10-08T01:14:22.788178+0000","last_clean":"2023-10-08T01:14:22.788178+0000","last_became_active":"2023-10-08T01:14:22.784506+0000","last_became_peered":"2023-10-08T01:14:22.784506+0000","last_unstale":"2023-10-08T01:14:22.788178+0000","last_undegraded":"2023-10-08T01:14:22.788178+0000","last_fullsized":"2023-10-08T01:14:22.788178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:12.608356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766909+0000","last_change":"2023-10-08T01:14:21.754925+0000","last_active":"2023-10-08T01:14:22.766909+0000","last_peered":"2023-10-08T01:14:22.766909+0000","last_clean":"2023-10-08T01:14:22.766909+0000","last_became_active":"2023-10-08T01:14:21.754606+0000","last_became_peered":"2023-10-08T01:14:21.754606+0000","last_unstale":"2023-10-08T01:14:22.766909+0000","last_undegraded":"2023-10-08T01:14:22.766909+0000","last_fullsized":"2023-10-08T01:14:22.766909+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:07.890275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.790593+0000","last_change":"2023-10-08T01:14:22.787439+0000","last_active":"2023-10-08T01:14:22.790593+0000","last_peered":"2023-10-08T01:14:22.790593+0000","last_clean":"2023-10-08T01:14:22.790593+0000","last_became_active":"2023-10-08T01:14:22.787105+0000","last_became_peered":"2023-10-08T01:14:22.787105+0000","last_unstale":"2023-10-08T01:14:22.790593+0000","last_undegraded":"2023-10-08T01:14:22.790593+0000","last_fullsized":"2023-10-08T01:14:22.790593+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745107+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:31.946887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789364+0000","last_change":"2023-10-08T01:14:22.785451+0000","last_active":"2023-10-08T01:14:22.789364+0000","last_peered":"2023-10-08T01:14:22.789364+0000","last_clean":"2023-10-08T01:14:22.789364+0000","last_became_active":"2023-10-08T01:14:22.785188+0000","last_became_peered":"2023-10-08T01:14:22.785188+0000","last_unstale":"2023-10-08T01:14:22.789364+0000","last_undegraded":"2023-10-08T01:14:22.789364+0000","last_fullsized":"2023-10-08T01:14:22.789364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745527+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766989+0000","last_change":"2023-10-08T01:14:21.770863+0000","last_active":"2023-10-08T01:14:22.766989+0000","last_peered":"2023-10-08T01:14:22.766989+0000","last_clean":"2023-10-08T01:14:22.766989+0000","last_became_active":"2023-10-08T01:14:21.770532+0000","last_became_peered":"2023-10-08T01:14:21.770532+0000","last_unstale":"2023-10-08T01:14:22.766989+0000","last_undegraded":"2023-10-08T01:14:22.766989+0000","last_fullsized":"2023-10-08T01:14:22.766989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:32.319886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787982+0000","last_change":"2023-10-08T01:14:22.781044+0000","last_active":"2023-10-08T01:14:22.787982+0000","last_peered":"2023-10-08T01:14:22.787982+0000","last_clean":"2023-10-08T01:14:22.787982+0000","last_became_active":"2023-10-08T01:14:22.780772+0000","last_became_peered":"2023-10-08T01:14:22.780772+0000","last_unstale":"2023-10-08T01:14:22.787982+0000","last_undegraded":"2023-10-08T01:14:22.787982+0000","last_fullsized":"2023-10-08T01:14:22.787982+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748071+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"109'227","reported_seq":634,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-08T01:14:18.452767+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:14:18.452767+0000","last_peered":"2023-10-08T01:14:18.452767+0000","last_clean":"2023-10-08T01:14:18.452767+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:14:18.452767+0000","last_undegraded":"2023-10-08T01:14:18.452767+0000","last_fullsized":"2023-10-08T01:14:18.452767+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2124,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":72,"acting":72,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303376,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068464,"statfs":{"total":96636764160,"available":96326107136,"internally_reserved":0,"allocated":2039808,"data_stored":1295194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":1.123,"15min":1.123},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.97899999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96999999999999997,"5min":1.133,"15min":1.133},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.72299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":1.167,"15min":1.167},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67100000000000004,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.91700000000000004,"5min":1.079,"15min":1.079},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.58299999999999996},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.88,"15min":0.88300000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.495},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":1.004,"15min":1.004},"last":0.64000000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72599999999999998,"15min":0.75},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302784,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069056,"statfs":{"total":96636764160,"available":96326713344,"internally_reserved":0,"allocated":1433600,"data_stored":698629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68300000000000005,"15min":0.69999999999999996},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.69499999999999995},"min":{"1min":0.54500000000000004,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.48399999999999999,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.984},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.59099999999999997,"5min":0.45300000000000001,"15min":0.38100000000000001},"max":{"1min":0.97099999999999997,"5min":1.018,"15min":3.0190000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.64100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.70299999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71999999999999997,"15min":0.745},"min":{"1min":0.55400000000000005,"5min":0.47599999999999998,"15min":0.47099999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.94399999999999995,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":1.087},"last":0.73099999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":0.84999999999999998,"5min":1.0189999999999999,"15min":1.395},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94999999999999996,"5min":1.0489999999999999,"15min":1.21},"last":0.54200000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":1.095,"15min":2.9489999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":725316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.46700000000000003},"min":{"1min":0.32600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47799999999999998,"15min":0.503},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.745,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.377,"5min":0.317,"15min":0.26700000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.77300000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.53500000000000003},"min":{"1min":0.45100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.82099999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.53700000000000003},"min":{"1min":0.42399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":1.296,"5min":1.349,"15min":1.452},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.76200000000000001},"min":{"1min":0.55700000000000005,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.137,"5min":1.1499999999999999,"15min":1.1839999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73199999999999998,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":1.119,"5min":1.274,"15min":1.3959999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.5209999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73799999999999999,"15min":0.78500000000000003},"min":{"1min":0.56799999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.069,"5min":1.3839999999999999,"15min":1.6020000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74199999999999999,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1499999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.621}]},{"osd":7,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.82099999999999995},"min":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.236,"5min":1.3109999999999999,"15min":1.621},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78500000000000003,"15min":0.82899999999999996},"min":{"1min":0.66000000000000003,"5min":0.58099999999999996,"15min":0.51300000000000001},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.4890000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":1316238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.625,"5min":0.95999999999999996,"15min":0.997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67700000000000005,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46800000000000003,"15min":0.48299999999999998},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.53000000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.70599999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.59899999999999998},"min":{"1min":0.504,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.754},"min":{"1min":0.53500000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94099999999999995,"5min":1.341,"15min":1.617},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.61299999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92100000000000004,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99199999999999999,"5min":2.085,"15min":2.085},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79100000000000004,"15min":0.81699999999999995},"min":{"1min":0.56699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.381,"15min":1.5289999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77100000000000002,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98299999999999998,"5min":1.361,"15min":1.3779999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99399999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77900000000000003,"15min":0.81299999999999994},"min":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.077,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.871},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81399999999999995,"15min":0.83899999999999997},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.028,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.89400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311028,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060812,"statfs":{"total":96636764160,"available":96318271488,"internally_reserved":0,"allocated":1486848,"data_stored":725350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.41099999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.46000000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.88800000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.90400000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306892,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064948,"statfs":{"total":96636764160,"available":96322506752,"internally_reserved":0,"allocated":1445888,"data_stored":704826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.46200000000000002},"min":{"1min":0.32000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54400000000000004,"5min":0.627,"15min":0.80100000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":0.81200000000000006},"last":0.48299999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.28299999999999997},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.495,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.311,"15min":0.31},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.89700000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.5,"5min":0.438,"15min":0.39600000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.752},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.41099999999999998},"max":{"1min":1.028,"5min":1.095,"15min":1.3029999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.046,"5min":1.0580000000000001,"15min":1.413},"last":0.88400000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.77300000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51000000000000001},"max":{"1min":0.995,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.67500000000000004,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.089},"last":0.98799999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.57199999999999995,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.94799999999999995,"5min":1.101,"15min":1.379},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72899999999999998,"15min":0.77900000000000003},"min":{"1min":0.59499999999999997,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.486},"last":0.85999999999999999}]},{"osd":7,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.755,"15min":0.79300000000000004},"min":{"1min":0.63200000000000001,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97799999999999998,"5min":1.0840000000000001,"15min":1.389},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.79500000000000004},"min":{"1min":0.63300000000000001,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.96699999999999997,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311044,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060796,"statfs":{"total":96636764160,"available":96318255104,"internally_reserved":0,"allocated":1503232,"data_stored":726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.84299999999999997}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.875}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.70999999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.92400000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.80100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302796,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069044,"statfs":{"total":96636764160,"available":96326701056,"internally_reserved":0,"allocated":1445888,"data_stored":704826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.71699999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.65000000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":1.026,"15min":1.026},"last":0.754},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.53300000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.38700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58499999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71999999999999997,"15min":0.73299999999999998},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.98699999999999999,"5min":1.034,"15min":1.034},"last":0.68700000000000006}]},{"osd":4,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.874,"15min":0.91900000000000004},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.90300000000000002,"15min":1.0680000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.90200000000000002,"15min":0.95699999999999996},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":0.94399999999999995,"15min":1.095},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47399999999999998},"max":{"1min":0.90200000000000002,"5min":1.05,"15min":1.05},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:14:27.095 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:14:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.306+0000 7f2bbfdff700 1 Processor -- start 2023-10-08T01:14:27.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.306+0000 7f2bbfdff700 1 -- start start 2023-10-08T01:14:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.307+0000 7f2bbfdff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2bb81578b0 0x7f2bb8155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.307+0000 7f2bbfdff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2bb80af2f0 0x7f2bb80af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.307+0000 7f2bbfdff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb80ad930 con 0x7f2bb8155c10 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.307+0000 7f2bbfdff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb80b0c80 con 0x7f2bb81578b0 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.307+0000 7f2bbfdff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb80acd20 con 0x7f2bb80af2f0 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbd39a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2bb81578b0 0x7f2bb8155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbd39a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2bb81578b0 0x7f2bb8155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40680/0 (socket says 172.21.15.119:40680) 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbd39a700 1 -- 172.21.15.119:0/1331230256 learned_addr learned my addr 172.21.15.119:0/1331230256 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbdb9b700 1 --1- 172.21.15.119:0/1331230256 >> v1:172.21.15.119:6789/0 conn(0x7f2bb8155c10 0x7f2bb80ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59732/0 (socket says 172.21.15.119:59732) 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbe39c700 1 --2- 172.21.15.119:0/1331230256 >> v2:172.21.15.184:3300/0 conn(0x7f2bb80af2f0 0x7f2bb80af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbd39a700 1 -- 172.21.15.119:0/1331230256 >> v2:172.21.15.184:3300/0 conn(0x7f2bb80af2f0 msgr2=0x7f2bb80af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbd39a700 1 --2- 172.21.15.119:0/1331230256 >> v2:172.21.15.184:3300/0 conn(0x7f2bb80af2f0 0x7f2bb80af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbd39a700 1 -- 172.21.15.119:0/1331230256 >> v1:172.21.15.119:6789/0 conn(0x7f2bb8155c10 legacy=0x7f2bb80ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbd39a700 1 -- 172.21.15.119:0/1331230256 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80ad000 con 0x7f2bb81578b0 2023-10-08T01:14:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbcb99700 1 -- 172.21.15.119:0/1331230256 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 969993342 0 0) 0x7f2bb80ad930 con 0x7f2bb8155c10 2023-10-08T01:14:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.308+0000 7f2bbd39a700 1 --2- 172.21.15.119:0/1331230256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb8155fe0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8065be0 tx=0x7f2ba8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92488693e5d70460 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbcb99700 1 -- 172.21.15.119:0/1331230256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800b860 con 0x7f2bb81578b0 2023-10-08T01:14:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbcb99700 1 -- 172.21.15.119:0/1331230256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba800b9c0 con 0x7f2bb81578b0 2023-10-08T01:14:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbcb99700 1 -- 172.21.15.119:0/1331230256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800bd60 con 0x7f2bb81578b0 2023-10-08T01:14:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbfdff700 1 -- 172.21.15.119:0/1331230256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 msgr2=0x7f2bb8155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/1331230256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb8155fe0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8065be0 tx=0x7f2ba8009250 comp rx=0 tx=0).stop 2023-10-08T01:14:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbfdff700 1 -- 172.21.15.119:0/1331230256 shutdown_connections 2023-10-08T01:14:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/1331230256 >> v2:172.21.15.184:3300/0 conn(0x7f2bb80af2f0 0x7f2bb80af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/1331230256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb8155fe0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbfdff700 1 -- 172.21.15.119:0/1331230256 >> 172.21.15.119:0/1331230256 conn(0x7f2bb806a320 msgr2=0x7f2bb80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:27.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbfdff700 1 -- 172.21.15.119:0/1331230256 shutdown_connections 2023-10-08T01:14:27.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.309+0000 7f2bbfdff700 1 -- 172.21.15.119:0/1331230256 wait complete. 2023-10-08T01:14:27.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.311+0000 7f2bbfdff700 1 Processor -- start 2023-10-08T01:14:27.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.311+0000 7f2bbfdff700 1 -- start start 2023-10-08T01:14:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbfdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb80af2f0 0x7f2bb8151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbfdff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb8155c10 0x7f2bb8151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbfdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb814e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbfdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8158340 con 0x7f2bb81578b0 2023-10-08T01:14:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbfdff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb81515b0 con 0x7f2bb8155c10 2023-10-08T01:14:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbfdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb814e620 con 0x7f2bb80af2f0 2023-10-08T01:14:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbe39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb814e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbe39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb814e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40682/0 (socket says 172.21.15.119:40682) 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbe39c700 1 -- 172.21.15.119:0/2124818945 learned_addr learned my addr 172.21.15.119:0/2124818945 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbdb9b700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb80af2f0 0x7f2bb8151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbd39a700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb8155c10 0x7f2bb8151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbe39c700 1 -- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb80af2f0 msgr2=0x7f2bb8151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbe39c700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb80af2f0 0x7f2bb8151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbe39c700 1 -- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb8155c10 msgr2=0x7f2bb8151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbe39c700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb8155c10 0x7f2bb8151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbe39c700 1 -- 172.21.15.119:0/2124818945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba80094f0 con 0x7f2bb81578b0 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbd39a700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb8155c10 0x7f2bb8151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.312+0000 7f2bbdb9b700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb80af2f0 0x7f2bb8151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.313+0000 7f2bbe39c700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb814e0e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009fa0 tx=0x7f2bb0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.313+0000 7f2badffb700 1 -- 172.21.15.119:0/2124818945 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb000ed20 con 0x7f2bb81578b0 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.313+0000 7f2badffb700 1 -- 172.21.15.119:0/2124818945 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb000ee80 con 0x7f2bb81578b0 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.313+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb8152340 con 0x7f2bb81578b0 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.313+0000 7f2badffb700 1 -- 172.21.15.119:0/2124818945 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0014670 con 0x7f2bb81578b0 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.313+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb8157cf0 con 0x7f2bb81578b0 2023-10-08T01:14:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.314+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb81584b0 con 0x7f2bb81578b0 2023-10-08T01:14:27.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.315+0000 7f2badffb700 1 -- 172.21.15.119:0/2124818945 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2bb00149e0 con 0x7f2bb81578b0 2023-10-08T01:14:27.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.316+0000 7f2badffb700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b9c066f30 0x7f2b9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.316+0000 7f2badffb700 1 -- 172.21.15.119:0/2124818945 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f2bb0084f50 con 0x7f2bb81578b0 2023-10-08T01:14:27.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.316+0000 7f2bbdb9b700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b9c066f30 0x7f2b9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.317+0000 7f2bbdb9b700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b9c066f30 0x7f2b9c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40018d0 tx=0x7f2bb4009040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:27.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.320+0000 7f2badffb700 1 -- 172.21.15.119:0/2124818945 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bb0051f90 con 0x7f2bb81578b0 2023-10-08T01:14:27.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.507+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2bb804c430 con 0x7f2b9c066f30 2023-10-08T01:14:27.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.515+0000 7f2badffb700 1 -- 172.21.15.119:0/2124818945 <== mgr.7488 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284704 (secure 0 0 0) 0x7f2bb804c430 con 0x7f2b9c066f30 2023-10-08T01:14:27.520 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:27.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.519+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b9c066f30 msgr2=0x7f2b9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.519+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b9c066f30 0x7f2b9c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40018d0 tx=0x7f2bb4009040 comp rx=0 tx=0).stop 2023-10-08T01:14:27.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.519+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 msgr2=0x7f2bb814e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.519+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb814e0e0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009fa0 tx=0x7f2bb0009900 comp rx=0 tx=0).stop 2023-10-08T01:14:27.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.520+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 shutdown_connections 2023-10-08T01:14:27.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.520+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b9c066f30 0x7f2b9c0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.520+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bb80af2f0 0x7f2bb8151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.520+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb81578b0 0x7f2bb814e0e0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.520+0000 7f2bbfdff700 1 --2- 172.21.15.119:0/2124818945 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bb8155c10 0x7f2bb8151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.520+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 >> 172.21.15.119:0/2124818945 conn(0x7f2bb806a320 msgr2=0x7f2bb80abe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:27.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.520+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 shutdown_connections 2023-10-08T01:14:27.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.520+0000 7f2bbfdff700 1 -- 172.21.15.119:0/2124818945 wait complete. 2023-10-08T01:14:27.527 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:14:27.537 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:14:26.707054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592492,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":394,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":138,"acting":138,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444168,"kb_used_data":12616,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752530552,"statfs":{"total":773094113280,"available":770591285248,"internally_reserved":0,"allocated":12918784,"data_stored":6897355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2124,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.965050"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798738+0000","last_change":"2023-10-08T01:14:22.797353+0000","last_active":"2023-10-08T01:14:22.798738+0000","last_peered":"2023-10-08T01:14:22.798738+0000","last_clean":"2023-10-08T01:14:22.798738+0000","last_became_active":"2023-10-08T01:14:22.795282+0000","last_became_peered":"2023-10-08T01:14:22.795282+0000","last_unstale":"2023-10-08T01:14:22.798738+0000","last_undegraded":"2023-10-08T01:14:22.798738+0000","last_fullsized":"2023-10-08T01:14:22.798738+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:19.889654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789827+0000","last_change":"2023-10-08T01:14:22.782483+0000","last_active":"2023-10-08T01:14:22.789827+0000","last_peered":"2023-10-08T01:14:22.789827+0000","last_clean":"2023-10-08T01:14:22.789827+0000","last_became_active":"2023-10-08T01:14:22.781426+0000","last_became_peered":"2023-10-08T01:14:22.781426+0000","last_unstale":"2023-10-08T01:14:22.789827+0000","last_undegraded":"2023-10-08T01:14:22.789827+0000","last_fullsized":"2023-10-08T01:14:22.789827+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:59.712158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794183+0000","last_change":"2023-10-08T01:14:22.786530+0000","last_active":"2023-10-08T01:14:22.794183+0000","last_peered":"2023-10-08T01:14:22.794183+0000","last_clean":"2023-10-08T01:14:22.794183+0000","last_became_active":"2023-10-08T01:14:22.786260+0000","last_became_peered":"2023-10-08T01:14:22.786260+0000","last_unstale":"2023-10-08T01:14:22.794183+0000","last_undegraded":"2023-10-08T01:14:22.794183+0000","last_fullsized":"2023-10-08T01:14:22.794183+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:25.960964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788496+0000","last_change":"2023-10-08T01:14:22.783965+0000","last_active":"2023-10-08T01:14:22.788496+0000","last_peered":"2023-10-08T01:14:22.788496+0000","last_clean":"2023-10-08T01:14:22.788496+0000","last_became_active":"2023-10-08T01:14:22.783699+0000","last_became_peered":"2023-10-08T01:14:22.783699+0000","last_unstale":"2023-10-08T01:14:22.788496+0000","last_undegraded":"2023-10-08T01:14:22.788496+0000","last_fullsized":"2023-10-08T01:14:22.788496+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:06.030874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792319+0000","last_change":"2023-10-08T01:14:22.788581+0000","last_active":"2023-10-08T01:14:22.792319+0000","last_peered":"2023-10-08T01:14:22.792319+0000","last_clean":"2023-10-08T01:14:22.792319+0000","last_became_active":"2023-10-08T01:14:22.788210+0000","last_became_peered":"2023-10-08T01:14:22.788210+0000","last_unstale":"2023-10-08T01:14:22.792319+0000","last_undegraded":"2023-10-08T01:14:22.792319+0000","last_fullsized":"2023-10-08T01:14:22.792319+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792484+0000","last_change":"2023-10-08T01:14:22.790486+0000","last_active":"2023-10-08T01:14:22.792484+0000","last_peered":"2023-10-08T01:14:22.792484+0000","last_clean":"2023-10-08T01:14:22.792484+0000","last_became_active":"2023-10-08T01:14:22.790175+0000","last_became_peered":"2023-10-08T01:14:22.790175+0000","last_unstale":"2023-10-08T01:14:22.792484+0000","last_undegraded":"2023-10-08T01:14:22.792484+0000","last_fullsized":"2023-10-08T01:14:22.792484+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.779024+0000","last_change":"2023-10-08T01:14:22.777745+0000","last_active":"2023-10-08T01:14:22.779024+0000","last_peered":"2023-10-08T01:14:22.779024+0000","last_clean":"2023-10-08T01:14:22.779024+0000","last_became_active":"2023-10-08T01:14:22.777473+0000","last_became_peered":"2023-10-08T01:14:22.777473+0000","last_unstale":"2023-10-08T01:14:22.779024+0000","last_undegraded":"2023-10-08T01:14:22.779024+0000","last_fullsized":"2023-10-08T01:14:22.779024+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:57.315090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788635+0000","last_change":"2023-10-08T01:14:22.782436+0000","last_active":"2023-10-08T01:14:22.788635+0000","last_peered":"2023-10-08T01:14:22.788635+0000","last_clean":"2023-10-08T01:14:22.788635+0000","last_became_active":"2023-10-08T01:14:22.782125+0000","last_became_peered":"2023-10-08T01:14:22.782125+0000","last_unstale":"2023-10-08T01:14:22.788635+0000","last_undegraded":"2023-10-08T01:14:22.788635+0000","last_fullsized":"2023-10-08T01:14:22.788635+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:26.317877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792306+0000","last_change":"2023-10-08T01:14:22.789825+0000","last_active":"2023-10-08T01:14:22.792306+0000","last_peered":"2023-10-08T01:14:22.792306+0000","last_clean":"2023-10-08T01:14:22.792306+0000","last_became_active":"2023-10-08T01:14:22.789560+0000","last_became_peered":"2023-10-08T01:14:22.789560+0000","last_unstale":"2023-10-08T01:14:22.792306+0000","last_undegraded":"2023-10-08T01:14:22.792306+0000","last_fullsized":"2023-10-08T01:14:22.792306+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792179+0000","last_change":"2023-10-08T01:14:22.788302+0000","last_active":"2023-10-08T01:14:22.792179+0000","last_peered":"2023-10-08T01:14:22.792179+0000","last_clean":"2023-10-08T01:14:22.792179+0000","last_became_active":"2023-10-08T01:14:22.787799+0000","last_became_peered":"2023-10-08T01:14:22.787799+0000","last_unstale":"2023-10-08T01:14:22.792179+0000","last_undegraded":"2023-10-08T01:14:22.792179+0000","last_fullsized":"2023-10-08T01:14:22.792179+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783436+0000","last_change":"2023-10-08T01:14:22.779249+0000","last_active":"2023-10-08T01:14:22.783436+0000","last_peered":"2023-10-08T01:14:22.783436+0000","last_clean":"2023-10-08T01:14:22.783436+0000","last_became_active":"2023-10-08T01:14:22.779000+0000","last_became_peered":"2023-10-08T01:14:22.779000+0000","last_unstale":"2023-10-08T01:14:22.783436+0000","last_undegraded":"2023-10-08T01:14:22.783436+0000","last_fullsized":"2023-10-08T01:14:22.783436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:44:52.899739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788599+0000","last_change":"2023-10-08T01:14:22.782182+0000","last_active":"2023-10-08T01:14:22.788599+0000","last_peered":"2023-10-08T01:14:22.788599+0000","last_clean":"2023-10-08T01:14:22.788599+0000","last_became_active":"2023-10-08T01:14:22.781939+0000","last_became_peered":"2023-10-08T01:14:22.781939+0000","last_unstale":"2023-10-08T01:14:22.788599+0000","last_undegraded":"2023-10-08T01:14:22.788599+0000","last_fullsized":"2023-10-08T01:14:22.788599+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:19.690095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787156+0000","last_change":"2023-10-08T01:14:22.780281+0000","last_active":"2023-10-08T01:14:22.787156+0000","last_peered":"2023-10-08T01:14:22.787156+0000","last_clean":"2023-10-08T01:14:22.787156+0000","last_became_active":"2023-10-08T01:14:22.780033+0000","last_became_peered":"2023-10-08T01:14:22.780033+0000","last_unstale":"2023-10-08T01:14:22.787156+0000","last_undegraded":"2023-10-08T01:14:22.787156+0000","last_fullsized":"2023-10-08T01:14:22.787156+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:41.656567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799940+0000","last_change":"2023-10-08T01:14:22.799140+0000","last_active":"2023-10-08T01:14:22.799940+0000","last_peered":"2023-10-08T01:14:22.799940+0000","last_clean":"2023-10-08T01:14:22.799940+0000","last_became_active":"2023-10-08T01:14:22.798851+0000","last_became_peered":"2023-10-08T01:14:22.798851+0000","last_unstale":"2023-10-08T01:14:22.799940+0000","last_undegraded":"2023-10-08T01:14:22.799940+0000","last_fullsized":"2023-10-08T01:14:22.799940+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:30.232699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789675+0000","last_change":"2023-10-08T01:14:22.788999+0000","last_active":"2023-10-08T01:14:22.789675+0000","last_peered":"2023-10-08T01:14:22.789675+0000","last_clean":"2023-10-08T01:14:22.789675+0000","last_became_active":"2023-10-08T01:14:22.787618+0000","last_became_peered":"2023-10-08T01:14:22.787618+0000","last_unstale":"2023-10-08T01:14:22.789675+0000","last_undegraded":"2023-10-08T01:14:22.789675+0000","last_fullsized":"2023-10-08T01:14:22.789675+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:17.572620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791070+0000","last_change":"2023-10-08T01:14:22.788072+0000","last_active":"2023-10-08T01:14:22.791070+0000","last_peered":"2023-10-08T01:14:22.791070+0000","last_clean":"2023-10-08T01:14:22.791070+0000","last_became_active":"2023-10-08T01:14:22.787677+0000","last_became_peered":"2023-10-08T01:14:22.787677+0000","last_unstale":"2023-10-08T01:14:22.791070+0000","last_undegraded":"2023-10-08T01:14:22.791070+0000","last_fullsized":"2023-10-08T01:14:22.791070+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:19:12.933240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798256+0000","last_change":"2023-10-08T01:14:22.794783+0000","last_active":"2023-10-08T01:14:22.798256+0000","last_peered":"2023-10-08T01:14:22.798256+0000","last_clean":"2023-10-08T01:14:22.798256+0000","last_became_active":"2023-10-08T01:14:22.794304+0000","last_became_peered":"2023-10-08T01:14:22.794304+0000","last_unstale":"2023-10-08T01:14:22.798256+0000","last_undegraded":"2023-10-08T01:14:22.798256+0000","last_fullsized":"2023-10-08T01:14:22.798256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:59.697433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783465+0000","last_change":"2023-10-08T01:14:22.779446+0000","last_active":"2023-10-08T01:14:22.783465+0000","last_peered":"2023-10-08T01:14:22.783465+0000","last_clean":"2023-10-08T01:14:22.783465+0000","last_became_active":"2023-10-08T01:14:22.778675+0000","last_became_peered":"2023-10-08T01:14:22.778675+0000","last_unstale":"2023-10-08T01:14:22.783465+0000","last_undegraded":"2023-10-08T01:14:22.783465+0000","last_fullsized":"2023-10-08T01:14:22.783465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:34.335870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795814+0000","last_change":"2023-10-08T01:14:21.755008+0000","last_active":"2023-10-08T01:14:22.795814+0000","last_peered":"2023-10-08T01:14:22.795814+0000","last_clean":"2023-10-08T01:14:22.795814+0000","last_became_active":"2023-10-08T01:14:21.754712+0000","last_became_peered":"2023-10-08T01:14:21.754712+0000","last_unstale":"2023-10-08T01:14:22.795814+0000","last_undegraded":"2023-10-08T01:14:22.795814+0000","last_fullsized":"2023-10-08T01:14:22.795814+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:24.330581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748043+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799279+0000","last_change":"2023-10-08T01:14:22.798147+0000","last_active":"2023-10-08T01:14:22.799279+0000","last_peered":"2023-10-08T01:14:22.799279+0000","last_clean":"2023-10-08T01:14:22.799279+0000","last_became_active":"2023-10-08T01:14:22.797856+0000","last_became_peered":"2023-10-08T01:14:22.797856+0000","last_unstale":"2023-10-08T01:14:22.799279+0000","last_undegraded":"2023-10-08T01:14:22.799279+0000","last_fullsized":"2023-10-08T01:14:22.799279+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:32.952368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793353+0000","last_change":"2023-10-08T01:14:21.758907+0000","last_active":"2023-10-08T01:14:22.793353+0000","last_peered":"2023-10-08T01:14:22.793353+0000","last_clean":"2023-10-08T01:14:22.793353+0000","last_became_active":"2023-10-08T01:14:21.758669+0000","last_became_peered":"2023-10-08T01:14:21.758669+0000","last_unstale":"2023-10-08T01:14:22.793353+0000","last_undegraded":"2023-10-08T01:14:22.793353+0000","last_fullsized":"2023-10-08T01:14:22.793353+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:33.791532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782110+0000","last_change":"2023-10-08T01:14:22.777591+0000","last_active":"2023-10-08T01:14:22.782110+0000","last_peered":"2023-10-08T01:14:22.782110+0000","last_clean":"2023-10-08T01:14:22.782110+0000","last_became_active":"2023-10-08T01:14:22.777315+0000","last_became_peered":"2023-10-08T01:14:22.777315+0000","last_unstale":"2023-10-08T01:14:22.782110+0000","last_undegraded":"2023-10-08T01:14:22.782110+0000","last_fullsized":"2023-10-08T01:14:22.782110+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793240+0000","last_change":"2023-10-08T01:14:21.770043+0000","last_active":"2023-10-08T01:14:22.793240+0000","last_peered":"2023-10-08T01:14:22.793240+0000","last_clean":"2023-10-08T01:14:22.793240+0000","last_became_active":"2023-10-08T01:14:21.769799+0000","last_became_peered":"2023-10-08T01:14:21.769799+0000","last_unstale":"2023-10-08T01:14:22.793240+0000","last_undegraded":"2023-10-08T01:14:22.793240+0000","last_fullsized":"2023-10-08T01:14:22.793240+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:29.304154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.786306+0000","last_change":"2023-10-08T01:14:21.767177+0000","last_active":"2023-10-08T01:14:22.786306+0000","last_peered":"2023-10-08T01:14:22.786306+0000","last_clean":"2023-10-08T01:14:22.786306+0000","last_became_active":"2023-10-08T01:14:21.766891+0000","last_became_peered":"2023-10-08T01:14:21.766891+0000","last_unstale":"2023-10-08T01:14:22.786306+0000","last_undegraded":"2023-10-08T01:14:22.786306+0000","last_fullsized":"2023-10-08T01:14:22.786306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:48.667974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798893+0000","last_change":"2023-10-08T01:14:21.763020+0000","last_active":"2023-10-08T01:14:22.798893+0000","last_peered":"2023-10-08T01:14:22.798893+0000","last_clean":"2023-10-08T01:14:22.798893+0000","last_became_active":"2023-10-08T01:14:21.762782+0000","last_became_peered":"2023-10-08T01:14:21.762782+0000","last_unstale":"2023-10-08T01:14:22.798893+0000","last_undegraded":"2023-10-08T01:14:22.798893+0000","last_fullsized":"2023-10-08T01:14:22.798893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:59.904749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745429+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:59.004815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789574+0000","last_change":"2023-10-08T01:14:22.783171+0000","last_active":"2023-10-08T01:14:22.789574+0000","last_peered":"2023-10-08T01:14:22.789574+0000","last_clean":"2023-10-08T01:14:22.789574+0000","last_became_active":"2023-10-08T01:14:22.782872+0000","last_became_peered":"2023-10-08T01:14:22.782872+0000","last_unstale":"2023-10-08T01:14:22.789574+0000","last_undegraded":"2023-10-08T01:14:22.789574+0000","last_fullsized":"2023-10-08T01:14:22.789574+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.780252+0000","last_change":"2023-10-08T01:14:21.766845+0000","last_active":"2023-10-08T01:14:22.780252+0000","last_peered":"2023-10-08T01:14:22.780252+0000","last_clean":"2023-10-08T01:14:22.780252+0000","last_became_active":"2023-10-08T01:14:21.765270+0000","last_became_peered":"2023-10-08T01:14:21.765270+0000","last_unstale":"2023-10-08T01:14:22.780252+0000","last_undegraded":"2023-10-08T01:14:22.780252+0000","last_fullsized":"2023-10-08T01:14:22.780252+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:33.330389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789845+0000","last_change":"2023-10-08T01:14:22.786181+0000","last_active":"2023-10-08T01:14:22.789845+0000","last_peered":"2023-10-08T01:14:22.789845+0000","last_clean":"2023-10-08T01:14:22.789845+0000","last_became_active":"2023-10-08T01:14:22.785912+0000","last_became_peered":"2023-10-08T01:14:22.785912+0000","last_unstale":"2023-10-08T01:14:22.789845+0000","last_undegraded":"2023-10-08T01:14:22.789845+0000","last_fullsized":"2023-10-08T01:14:22.789845+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:33.304186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.781565+0000","last_change":"2023-10-08T01:14:21.763195+0000","last_active":"2023-10-08T01:14:22.781565+0000","last_peered":"2023-10-08T01:14:22.781565+0000","last_clean":"2023-10-08T01:14:22.781565+0000","last_became_active":"2023-10-08T01:14:21.762953+0000","last_became_peered":"2023-10-08T01:14:21.762953+0000","last_unstale":"2023-10-08T01:14:22.781565+0000","last_undegraded":"2023-10-08T01:14:22.781565+0000","last_fullsized":"2023-10-08T01:14:22.781565+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:56.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789148+0000","last_change":"2023-10-08T01:14:22.782447+0000","last_active":"2023-10-08T01:14:22.789148+0000","last_peered":"2023-10-08T01:14:22.789148+0000","last_clean":"2023-10-08T01:14:22.789148+0000","last_became_active":"2023-10-08T01:14:22.782195+0000","last_became_peered":"2023-10-08T01:14:22.782195+0000","last_unstale":"2023-10-08T01:14:22.789148+0000","last_undegraded":"2023-10-08T01:14:22.789148+0000","last_fullsized":"2023-10-08T01:14:22.789148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745607+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748365+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.800832+0000","last_change":"2023-10-08T01:14:21.764918+0000","last_active":"2023-10-08T01:14:22.800832+0000","last_peered":"2023-10-08T01:14:22.800832+0000","last_clean":"2023-10-08T01:14:22.800832+0000","last_became_active":"2023-10-08T01:14:21.763653+0000","last_became_peered":"2023-10-08T01:14:21.763653+0000","last_unstale":"2023-10-08T01:14:22.800832+0000","last_undegraded":"2023-10-08T01:14:22.800832+0000","last_fullsized":"2023-10-08T01:14:22.800832+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793156+0000","last_change":"2023-10-08T01:14:21.769762+0000","last_active":"2023-10-08T01:14:22.793156+0000","last_peered":"2023-10-08T01:14:22.793156+0000","last_clean":"2023-10-08T01:14:22.793156+0000","last_became_active":"2023-10-08T01:14:21.769522+0000","last_became_peered":"2023-10-08T01:14:21.769522+0000","last_unstale":"2023-10-08T01:14:22.793156+0000","last_undegraded":"2023-10-08T01:14:22.793156+0000","last_fullsized":"2023-10-08T01:14:22.793156+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:27.726884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797374+0000","last_change":"2023-10-08T01:14:22.791650+0000","last_active":"2023-10-08T01:14:22.797374+0000","last_peered":"2023-10-08T01:14:22.797374+0000","last_clean":"2023-10-08T01:14:22.797374+0000","last_became_active":"2023-10-08T01:14:22.791109+0000","last_became_peered":"2023-10-08T01:14:22.791109+0000","last_unstale":"2023-10-08T01:14:22.797374+0000","last_undegraded":"2023-10-08T01:14:22.797374+0000","last_fullsized":"2023-10-08T01:14:22.797374+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765608+0000","last_change":"2023-10-08T01:14:21.768459+0000","last_active":"2023-10-08T01:14:22.765608+0000","last_peered":"2023-10-08T01:14:22.765608+0000","last_clean":"2023-10-08T01:14:22.765608+0000","last_became_active":"2023-10-08T01:14:21.768189+0000","last_became_peered":"2023-10-08T01:14:21.768189+0000","last_unstale":"2023-10-08T01:14:22.765608+0000","last_undegraded":"2023-10-08T01:14:22.765608+0000","last_fullsized":"2023-10-08T01:14:22.765608+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:54.324955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789515+0000","last_change":"2023-10-08T01:14:22.781160+0000","last_active":"2023-10-08T01:14:22.789515+0000","last_peered":"2023-10-08T01:14:22.789515+0000","last_clean":"2023-10-08T01:14:22.789515+0000","last_became_active":"2023-10-08T01:14:22.780893+0000","last_became_peered":"2023-10-08T01:14:22.780893+0000","last_unstale":"2023-10-08T01:14:22.789515+0000","last_undegraded":"2023-10-08T01:14:22.789515+0000","last_fullsized":"2023-10-08T01:14:22.789515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:30.309636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768642+0000","last_change":"2023-10-08T01:14:21.760294+0000","last_active":"2023-10-08T01:14:22.768642+0000","last_peered":"2023-10-08T01:14:22.768642+0000","last_clean":"2023-10-08T01:14:22.768642+0000","last_became_active":"2023-10-08T01:14:21.758928+0000","last_became_peered":"2023-10-08T01:14:21.758928+0000","last_unstale":"2023-10-08T01:14:22.768642+0000","last_undegraded":"2023-10-08T01:14:22.768642+0000","last_fullsized":"2023-10-08T01:14:22.768642+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788009+0000","last_change":"2023-10-08T01:14:22.784690+0000","last_active":"2023-10-08T01:14:22.788009+0000","last_peered":"2023-10-08T01:14:22.788009+0000","last_clean":"2023-10-08T01:14:22.788009+0000","last_became_active":"2023-10-08T01:14:22.784408+0000","last_became_peered":"2023-10-08T01:14:22.784408+0000","last_unstale":"2023-10-08T01:14:22.788009+0000","last_undegraded":"2023-10-08T01:14:22.788009+0000","last_fullsized":"2023-10-08T01:14:22.788009+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:55.180105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795020+0000","last_change":"2023-10-08T01:14:21.765533+0000","last_active":"2023-10-08T01:14:22.795020+0000","last_peered":"2023-10-08T01:14:22.795020+0000","last_clean":"2023-10-08T01:14:22.795020+0000","last_became_active":"2023-10-08T01:14:21.764914+0000","last_became_peered":"2023-10-08T01:14:21.764914+0000","last_unstale":"2023-10-08T01:14:22.795020+0000","last_undegraded":"2023-10-08T01:14:22.795020+0000","last_fullsized":"2023-10-08T01:14:22.795020+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:00.128756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797139+0000","last_change":"2023-10-08T01:14:21.767123+0000","last_active":"2023-10-08T01:14:22.797139+0000","last_peered":"2023-10-08T01:14:22.797139+0000","last_clean":"2023-10-08T01:14:22.797139+0000","last_became_active":"2023-10-08T01:14:21.766826+0000","last_became_peered":"2023-10-08T01:14:21.766826+0000","last_unstale":"2023-10-08T01:14:22.797139+0000","last_undegraded":"2023-10-08T01:14:22.797139+0000","last_fullsized":"2023-10-08T01:14:22.797139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745225+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:13.819182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.767122+0000","last_change":"2023-10-08T01:14:21.769975+0000","last_active":"2023-10-08T01:14:22.767122+0000","last_peered":"2023-10-08T01:14:22.767122+0000","last_clean":"2023-10-08T01:14:22.767122+0000","last_became_active":"2023-10-08T01:14:21.769477+0000","last_became_peered":"2023-10-08T01:14:21.769477+0000","last_unstale":"2023-10-08T01:14:22.767122+0000","last_undegraded":"2023-10-08T01:14:22.767122+0000","last_fullsized":"2023-10-08T01:14:22.767122+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:34.177414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787894+0000","last_change":"2023-10-08T01:14:22.781041+0000","last_active":"2023-10-08T01:14:22.787894+0000","last_peered":"2023-10-08T01:14:22.787894+0000","last_clean":"2023-10-08T01:14:22.787894+0000","last_became_active":"2023-10-08T01:14:22.780765+0000","last_became_peered":"2023-10-08T01:14:22.780765+0000","last_unstale":"2023-10-08T01:14:22.787894+0000","last_undegraded":"2023-10-08T01:14:22.787894+0000","last_fullsized":"2023-10-08T01:14:22.787894+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:00.278431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792713+0000","last_change":"2023-10-08T01:14:21.768612+0000","last_active":"2023-10-08T01:14:22.792713+0000","last_peered":"2023-10-08T01:14:22.792713+0000","last_clean":"2023-10-08T01:14:22.792713+0000","last_became_active":"2023-10-08T01:14:21.767991+0000","last_became_peered":"2023-10-08T01:14:21.767991+0000","last_unstale":"2023-10-08T01:14:22.792713+0000","last_undegraded":"2023-10-08T01:14:22.792713+0000","last_fullsized":"2023-10-08T01:14:22.792713+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:50.772143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788094+0000","last_change":"2023-10-08T01:14:22.783198+0000","last_active":"2023-10-08T01:14:22.788094+0000","last_peered":"2023-10-08T01:14:22.788094+0000","last_clean":"2023-10-08T01:14:22.788094+0000","last_became_active":"2023-10-08T01:14:22.782950+0000","last_became_peered":"2023-10-08T01:14:22.782950+0000","last_unstale":"2023-10-08T01:14:22.788094+0000","last_undegraded":"2023-10-08T01:14:22.788094+0000","last_fullsized":"2023-10-08T01:14:22.788094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:30:30.559893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749667+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:31.946887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788041+0000","last_change":"2023-10-08T01:14:22.782578+0000","last_active":"2023-10-08T01:14:22.788041+0000","last_peered":"2023-10-08T01:14:22.788041+0000","last_clean":"2023-10-08T01:14:22.788041+0000","last_became_active":"2023-10-08T01:14:22.781703+0000","last_became_peered":"2023-10-08T01:14:22.781703+0000","last_unstale":"2023-10-08T01:14:22.788041+0000","last_undegraded":"2023-10-08T01:14:22.788041+0000","last_fullsized":"2023-10-08T01:14:22.788041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:38.983859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749808+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:13.819182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.763121+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.762883+0000","last_became_peered":"2023-10-08T01:14:21.762883+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:03.018597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789807+0000","last_change":"2023-10-08T01:14:22.782302+0000","last_active":"2023-10-08T01:14:22.789807+0000","last_peered":"2023-10-08T01:14:22.789807+0000","last_clean":"2023-10-08T01:14:22.789807+0000","last_became_active":"2023-10-08T01:14:22.780497+0000","last_became_peered":"2023-10-08T01:14:22.780497+0000","last_unstale":"2023-10-08T01:14:22.789807+0000","last_undegraded":"2023-10-08T01:14:22.789807+0000","last_fullsized":"2023-10-08T01:14:22.789807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:59.391189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749695+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:10.276377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798148+0000","last_change":"2023-10-08T01:14:22.793832+0000","last_active":"2023-10-08T01:14:22.798148+0000","last_peered":"2023-10-08T01:14:22.798148+0000","last_clean":"2023-10-08T01:14:22.798148+0000","last_became_active":"2023-10-08T01:14:22.792633+0000","last_became_peered":"2023-10-08T01:14:22.792633+0000","last_unstale":"2023-10-08T01:14:22.798148+0000","last_undegraded":"2023-10-08T01:14:22.798148+0000","last_fullsized":"2023-10-08T01:14:22.798148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:11.281859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768046+0000","last_change":"2023-10-08T01:14:21.763932+0000","last_active":"2023-10-08T01:14:22.768046+0000","last_peered":"2023-10-08T01:14:22.768046+0000","last_clean":"2023-10-08T01:14:22.768046+0000","last_became_active":"2023-10-08T01:14:21.763673+0000","last_became_peered":"2023-10-08T01:14:21.763673+0000","last_unstale":"2023-10-08T01:14:22.768046+0000","last_undegraded":"2023-10-08T01:14:22.768046+0000","last_fullsized":"2023-10-08T01:14:22.768046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:07.616557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797649+0000","last_change":"2023-10-08T01:14:22.791844+0000","last_active":"2023-10-08T01:14:22.797649+0000","last_peered":"2023-10-08T01:14:22.797649+0000","last_clean":"2023-10-08T01:14:22.797649+0000","last_became_active":"2023-10-08T01:14:22.791583+0000","last_became_peered":"2023-10-08T01:14:22.791583+0000","last_unstale":"2023-10-08T01:14:22.797649+0000","last_undegraded":"2023-10-08T01:14:22.797649+0000","last_fullsized":"2023-10-08T01:14:22.797649+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.770984+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.770705+0000","last_became_peered":"2023-10-08T01:14:21.770705+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:48:20.485203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745127+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:10.276377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788178+0000","last_change":"2023-10-08T01:14:22.784755+0000","last_active":"2023-10-08T01:14:22.788178+0000","last_peered":"2023-10-08T01:14:22.788178+0000","last_clean":"2023-10-08T01:14:22.788178+0000","last_became_active":"2023-10-08T01:14:22.784506+0000","last_became_peered":"2023-10-08T01:14:22.784506+0000","last_unstale":"2023-10-08T01:14:22.788178+0000","last_undegraded":"2023-10-08T01:14:22.788178+0000","last_fullsized":"2023-10-08T01:14:22.788178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:12.608356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766909+0000","last_change":"2023-10-08T01:14:21.754925+0000","last_active":"2023-10-08T01:14:22.766909+0000","last_peered":"2023-10-08T01:14:22.766909+0000","last_clean":"2023-10-08T01:14:22.766909+0000","last_became_active":"2023-10-08T01:14:21.754606+0000","last_became_peered":"2023-10-08T01:14:21.754606+0000","last_unstale":"2023-10-08T01:14:22.766909+0000","last_undegraded":"2023-10-08T01:14:22.766909+0000","last_fullsized":"2023-10-08T01:14:22.766909+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:07.890275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.790593+0000","last_change":"2023-10-08T01:14:22.787439+0000","last_active":"2023-10-08T01:14:22.790593+0000","last_peered":"2023-10-08T01:14:22.790593+0000","last_clean":"2023-10-08T01:14:22.790593+0000","last_became_active":"2023-10-08T01:14:22.787105+0000","last_became_peered":"2023-10-08T01:14:22.787105+0000","last_unstale":"2023-10-08T01:14:22.790593+0000","last_undegraded":"2023-10-08T01:14:22.790593+0000","last_fullsized":"2023-10-08T01:14:22.790593+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745107+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:31.946887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789364+0000","last_change":"2023-10-08T01:14:22.785451+0000","last_active":"2023-10-08T01:14:22.789364+0000","last_peered":"2023-10-08T01:14:22.789364+0000","last_clean":"2023-10-08T01:14:22.789364+0000","last_became_active":"2023-10-08T01:14:22.785188+0000","last_became_peered":"2023-10-08T01:14:22.785188+0000","last_unstale":"2023-10-08T01:14:22.789364+0000","last_undegraded":"2023-10-08T01:14:22.789364+0000","last_fullsized":"2023-10-08T01:14:22.789364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745527+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766989+0000","last_change":"2023-10-08T01:14:21.770863+0000","last_active":"2023-10-08T01:14:22.766989+0000","last_peered":"2023-10-08T01:14:22.766989+0000","last_clean":"2023-10-08T01:14:22.766989+0000","last_became_active":"2023-10-08T01:14:21.770532+0000","last_became_peered":"2023-10-08T01:14:21.770532+0000","last_unstale":"2023-10-08T01:14:22.766989+0000","last_undegraded":"2023-10-08T01:14:22.766989+0000","last_fullsized":"2023-10-08T01:14:22.766989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:32.319886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787982+0000","last_change":"2023-10-08T01:14:22.781044+0000","last_active":"2023-10-08T01:14:22.787982+0000","last_peered":"2023-10-08T01:14:22.787982+0000","last_clean":"2023-10-08T01:14:22.787982+0000","last_became_active":"2023-10-08T01:14:22.780772+0000","last_became_peered":"2023-10-08T01:14:22.780772+0000","last_unstale":"2023-10-08T01:14:22.787982+0000","last_undegraded":"2023-10-08T01:14:22.787982+0000","last_fullsized":"2023-10-08T01:14:22.787982+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748071+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"109'227","reported_seq":634,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-08T01:14:18.452767+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:14:18.452767+0000","last_peered":"2023-10-08T01:14:18.452767+0000","last_clean":"2023-10-08T01:14:18.452767+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:14:18.452767+0000","last_undegraded":"2023-10-08T01:14:18.452767+0000","last_fullsized":"2023-10-08T01:14:18.452767+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2124,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":72,"acting":72,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303376,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068464,"statfs":{"total":96636764160,"available":96326107136,"internally_reserved":0,"allocated":2039808,"data_stored":1295194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":1.123,"15min":1.123},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.97899999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96999999999999997,"5min":1.133,"15min":1.133},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.72299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":1.167,"15min":1.167},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67100000000000004,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.91700000000000004,"5min":1.079,"15min":1.079},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.58299999999999996},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.88,"15min":0.88300000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.495},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":1.004,"15min":1.004},"last":0.64000000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72599999999999998,"15min":0.75},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302784,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069056,"statfs":{"total":96636764160,"available":96326713344,"internally_reserved":0,"allocated":1433600,"data_stored":698629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68300000000000005,"15min":0.69999999999999996},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.69499999999999995},"min":{"1min":0.54500000000000004,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.48399999999999999,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.984},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.59099999999999997,"5min":0.45300000000000001,"15min":0.38100000000000001},"max":{"1min":0.97099999999999997,"5min":1.018,"15min":3.0190000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.64100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.70299999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71999999999999997,"15min":0.745},"min":{"1min":0.55400000000000005,"5min":0.47599999999999998,"15min":0.47099999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.94399999999999995,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":1.087},"last":0.73099999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":0.84999999999999998,"5min":1.0189999999999999,"15min":1.395},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94999999999999996,"5min":1.0489999999999999,"15min":1.21},"last":0.54200000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":1.095,"15min":2.9489999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":725316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.46700000000000003},"min":{"1min":0.32600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47799999999999998,"15min":0.503},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.745,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.377,"5min":0.317,"15min":0.26700000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.77300000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.53500000000000003},"min":{"1min":0.45100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.82099999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.53700000000000003},"min":{"1min":0.42399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":1.296,"5min":1.349,"15min":1.452},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.76200000000000001},"min":{"1min":0.55700000000000005,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.137,"5min":1.1499999999999999,"15min":1.1839999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73199999999999998,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":1.119,"5min":1.274,"15min":1.3959999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.5209999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73799999999999999,"15min":0.78500000000000003},"min":{"1min":0.56799999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.069,"5min":1.3839999999999999,"15min":1.6020000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74199999999999999,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1499999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.621}]},{"osd":7,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.82099999999999995},"min":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.236,"5min":1.3109999999999999,"15min":1.621},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78500000000000003,"15min":0.82899999999999996},"min":{"1min":0.66000000000000003,"5min":0.58099999999999996,"15min":0.51300000000000001},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.4890000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":1316238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.625,"5min":0.95999999999999996,"15min":0.997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67700000000000005,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46800000000000003,"15min":0.48299999999999998},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.53000000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.70599999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.59899999999999998},"min":{"1min":0.504,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.754},"min":{"1min":0.53500000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94099999999999995,"5min":1.341,"15min":1.617},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.61299999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92100000000000004,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99199999999999999,"5min":2.085,"15min":2.085},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79100000000000004,"15min":0.81699999999999995},"min":{"1min":0.56699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.381,"15min":1.5289999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77100000000000002,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98299999999999998,"5min":1.361,"15min":1.3779999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99399999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77900000000000003,"15min":0.81299999999999994},"min":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.077,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.871},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81399999999999995,"15min":0.83899999999999997},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.028,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.89400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311028,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060812,"statfs":{"total":96636764160,"available":96318271488,"internally_reserved":0,"allocated":1486848,"data_stored":725350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.41099999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.46000000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.88800000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.90400000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306892,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064948,"statfs":{"total":96636764160,"available":96322506752,"internally_reserved":0,"allocated":1445888,"data_stored":704826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.46200000000000002},"min":{"1min":0.32000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54400000000000004,"5min":0.627,"15min":0.80100000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":0.81200000000000006},"last":0.48299999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.28299999999999997},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.495,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.311,"15min":0.31},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.89700000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.5,"5min":0.438,"15min":0.39600000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.752},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.41099999999999998},"max":{"1min":1.028,"5min":1.095,"15min":1.3029999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.046,"5min":1.0580000000000001,"15min":1.413},"last":0.88400000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.77300000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51000000000000001},"max":{"1min":0.995,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.67500000000000004,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.089},"last":0.98799999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.57199999999999995,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.94799999999999995,"5min":1.101,"15min":1.379},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72899999999999998,"15min":0.77900000000000003},"min":{"1min":0.59499999999999997,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.486},"last":0.85999999999999999}]},{"osd":7,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.755,"15min":0.79300000000000004},"min":{"1min":0.63200000000000001,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97799999999999998,"5min":1.0840000000000001,"15min":1.389},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.79500000000000004},"min":{"1min":0.63300000000000001,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.96699999999999997,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311044,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060796,"statfs":{"total":96636764160,"available":96318255104,"internally_reserved":0,"allocated":1503232,"data_stored":726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.84299999999999997}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.875}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.70999999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.92400000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.80100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302796,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069044,"statfs":{"total":96636764160,"available":96326701056,"internally_reserved":0,"allocated":1445888,"data_stored":704826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.71699999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.65000000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":1.026,"15min":1.026},"last":0.754},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.53300000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.38700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58499999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71999999999999997,"15min":0.73299999999999998},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.98699999999999999,"5min":1.034,"15min":1.034},"last":0.68700000000000006}]},{"osd":4,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.874,"15min":0.91900000000000004},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.90300000000000002,"15min":1.0680000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.90200000000000002,"15min":0.95699999999999996},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":0.94399999999999995,"15min":1.095},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47399999999999998},"max":{"1min":0.90200000000000002,"5min":1.05,"15min":1.05},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:14:27.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:14:27.735 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-10-08T01:14:27.735+0000 7ff71c3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:14:27.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.759+0000 7f7071fff700 1 Processor -- start 2023-10-08T01:14:27.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.760+0000 7f7071fff700 1 -- start start 2023-10-08T01:14:27.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f7071fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f706c0cbf30 0x7f706c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f7071fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f706c0b1420 0x7f706c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f7071fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f706c0cc300 con 0x7f706c0ccf90 2023-10-08T01:14:27.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f7071fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706c156bf0 con 0x7f706c0cbf30 2023-10-08T01:14:27.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f7071fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f706c0ae950 con 0x7f706c0b1420 2023-10-08T01:14:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f706b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f706c0cbf30 0x7f706c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f706b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f706c0cbf30 0x7f706c0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40690/0 (socket says 172.21.15.119:40690) 2023-10-08T01:14:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f706b7fe700 1 -- 172.21.15.119:0/3613560603 learned_addr learned my addr 172.21.15.119:0/3613560603 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f706bfff700 1 --2- 172.21.15.119:0/3613560603 >> v2:172.21.15.184:3300/0 conn(0x7f706c0b1420 0x7f706c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.761+0000 7f706b7fe700 1 -- 172.21.15.119:0/3613560603 >> v2:172.21.15.184:3300/0 conn(0x7f706c0b1420 msgr2=0x7f706c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:27.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.762+0000 7f706b7fe700 1 --2- 172.21.15.119:0/3613560603 >> v2:172.21.15.184:3300/0 conn(0x7f706c0b1420 0x7f706c0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.762+0000 7f706b7fe700 1 -- 172.21.15.119:0/3613560603 >> v1:172.21.15.184:6789/0 conn(0x7f706c0ccf90 legacy=0x7f706c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:27.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.762+0000 7f706b7fe700 1 -- 172.21.15.119:0/3613560603 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706c0ad370 con 0x7f706c0cbf30 2023-10-08T01:14:27.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.762+0000 7f706bfff700 1 --2- 172.21.15.119:0/3613560603 >> v2:172.21.15.184:3300/0 conn(0x7f706c0b1420 0x7f706c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:27.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.762+0000 7f706b7fe700 1 --2- 172.21.15.119:0/3613560603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0ae410 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f706c0b19e0 tx=0x7f705c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb4fa20125210be server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.762+0000 7f706affd700 1 -- 172.21.15.119:0/3613560603 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c00b7e0 con 0x7f706c0cbf30 2023-10-08T01:14:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.762+0000 7f706affd700 1 -- 172.21.15.119:0/3613560603 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c00b940 con 0x7f706c0cbf30 2023-10-08T01:14:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.762+0000 7f706affd700 1 -- 172.21.15.119:0/3613560603 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c00bce0 con 0x7f706c0cbf30 2023-10-08T01:14:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.763+0000 7f7071fff700 1 -- 172.21.15.119:0/3613560603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 msgr2=0x7f706c0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.763+0000 7f7071fff700 1 --2- 172.21.15.119:0/3613560603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0ae410 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f706c0b19e0 tx=0x7f705c009250 comp rx=0 tx=0).stop 2023-10-08T01:14:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.763+0000 7f7071fff700 1 -- 172.21.15.119:0/3613560603 shutdown_connections 2023-10-08T01:14:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.763+0000 7f7071fff700 1 --2- 172.21.15.119:0/3613560603 >> v2:172.21.15.184:3300/0 conn(0x7f706c0b1420 0x7f706c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.763+0000 7f7071fff700 1 --2- 172.21.15.119:0/3613560603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0ae410 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.763+0000 7f7071fff700 1 -- 172.21.15.119:0/3613560603 >> 172.21.15.119:0/3613560603 conn(0x7f706c06a320 msgr2=0x7f706c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.763+0000 7f7071fff700 1 -- 172.21.15.119:0/3613560603 shutdown_connections 2023-10-08T01:14:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.763+0000 7f7071fff700 1 -- 172.21.15.119:0/3613560603 wait complete. 2023-10-08T01:14:27.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.764+0000 7f7071fff700 1 Processor -- start 2023-10-08T01:14:27.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.764+0000 7f7071fff700 1 -- start start 2023-10-08T01:14:27.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.765+0000 7f7071fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f706c0b1420 0x7f706c16ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.765+0000 7f7071fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.765+0000 7f7071fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f706c0ccf90 0x7f706c15e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.765+0000 7f7071fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c16c600 con 0x7f706c0cbf30 2023-10-08T01:14:27.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.765+0000 7f7071fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c16b710 con 0x7f706c0b1420 2023-10-08T01:14:27.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.765+0000 7f7071fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706c16b9f0 con 0x7f706c0ccf90 2023-10-08T01:14:27.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.765+0000 7f7063fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7063fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0c12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40700/0 (socket says 172.21.15.119:40700) 2023-10-08T01:14:27.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7063fff700 1 -- 172.21.15.119:0/3911260844 learned_addr learned my addr 172.21.15.119:0/3911260844 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:27.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f706b7fe700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f706c0b1420 0x7f706c16ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f706bfff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f706c0ccf90 0x7f706c15e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7063fff700 1 -- 172.21.15.119:0/3911260844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f706c0ccf90 msgr2=0x7f706c15e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7063fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f706c0ccf90 0x7f706c15e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7063fff700 1 -- 172.21.15.119:0/3911260844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f706c0b1420 msgr2=0x7f706c16ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7063fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f706c0b1420 0x7f706c16ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7063fff700 1 -- 172.21.15.119:0/3911260844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c0094f0 con 0x7f706c0cbf30 2023-10-08T01:14:27.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7063fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0c12d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f705400b5e0 tx=0x7f7054009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:27.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f70637fe700 1 -- 172.21.15.119:0/3911260844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400f510 con 0x7f706c0cbf30 2023-10-08T01:14:27.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f70637fe700 1 -- 172.21.15.119:0/3911260844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400f670 con 0x7f706c0cbf30 2023-10-08T01:14:27.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f70637fe700 1 -- 172.21.15.119:0/3911260844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400fa30 con 0x7f706c0cbf30 2023-10-08T01:14:27.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.766+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c16bcd0 con 0x7f706c0cbf30 2023-10-08T01:14:27.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.767+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c16c290 con 0x7f706c0cbf30 2023-10-08T01:14:27.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.768+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706c06d170 con 0x7f706c0cbf30 2023-10-08T01:14:27.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.768+0000 7f70637fe700 1 -- 172.21.15.119:0/3911260844 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f705401a400 con 0x7f706c0cbf30 2023-10-08T01:14:27.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.769+0000 7f70637fe700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f704c066f30 0x7f704c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:27.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.769+0000 7f70637fe700 1 -- 172.21.15.119:0/3911260844 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f7054084290 con 0x7f706c0cbf30 2023-10-08T01:14:27.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.769+0000 7f706b7fe700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f704c066f30 0x7f704c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:27.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.770+0000 7f706b7fe700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f704c066f30 0x7f704c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f706c0b19e0 tx=0x7f705c016000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:27.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.773+0000 7f70637fe700 1 -- 172.21.15.119:0/3911260844 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70540513f0 con 0x7f706c0cbf30 2023-10-08T01:14:27.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.959+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f706c04c3c0 con 0x7f704c066f30 2023-10-08T01:14:27.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.968+0000 7f70637fe700 1 -- 172.21.15.119:0/3911260844 <== mgr.7488 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284704 (secure 0 0 0) 0x7f706c04c3c0 con 0x7f704c066f30 2023-10-08T01:14:27.972 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:27.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f704c066f30 msgr2=0x7f704c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f704c066f30 0x7f704c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f706c0b19e0 tx=0x7f705c016000 comp rx=0 tx=0).stop 2023-10-08T01:14:27.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 msgr2=0x7f706c0c12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:27.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0c12d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f705400b5e0 tx=0x7f7054009120 comp rx=0 tx=0).stop 2023-10-08T01:14:27.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 shutdown_connections 2023-10-08T01:14:27.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f704c066f30 0x7f704c0693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f706c0ccf90 0x7f706c15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f706c0b1420 0x7f706c16ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 --2- 172.21.15.119:0/3911260844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f706c0cbf30 0x7f706c0c12d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:27.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 >> 172.21.15.119:0/3911260844 conn(0x7f706c06a320 msgr2=0x7f706c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:27.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 shutdown_connections 2023-10-08T01:14:27.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:27.972+0000 7f7071fff700 1 -- 172.21.15.119:0/3911260844 wait complete. 2023-10-08T01:14:27.980 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:14:27.990 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:14:26.707054+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592492,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":394,"num_write_kb":2426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":257,"ondisk_log_size":257,"up":138,"acting":138,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2444168,"kb_used_data":12616,"kb_used_omap":0,"kb_used_meta":2431488,"kb_avail":752530552,"statfs":{"total":773094113280,"available":770591285248,"internally_reserved":0,"allocated":12918784,"data_stored":6897355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2489843712},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2124,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.965050"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798738+0000","last_change":"2023-10-08T01:14:22.797353+0000","last_active":"2023-10-08T01:14:22.798738+0000","last_peered":"2023-10-08T01:14:22.798738+0000","last_clean":"2023-10-08T01:14:22.798738+0000","last_became_active":"2023-10-08T01:14:22.795282+0000","last_became_peered":"2023-10-08T01:14:22.795282+0000","last_unstale":"2023-10-08T01:14:22.798738+0000","last_undegraded":"2023-10-08T01:14:22.798738+0000","last_fullsized":"2023-10-08T01:14:22.798738+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:19.889654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789827+0000","last_change":"2023-10-08T01:14:22.782483+0000","last_active":"2023-10-08T01:14:22.789827+0000","last_peered":"2023-10-08T01:14:22.789827+0000","last_clean":"2023-10-08T01:14:22.789827+0000","last_became_active":"2023-10-08T01:14:22.781426+0000","last_became_peered":"2023-10-08T01:14:22.781426+0000","last_unstale":"2023-10-08T01:14:22.789827+0000","last_undegraded":"2023-10-08T01:14:22.789827+0000","last_fullsized":"2023-10-08T01:14:22.789827+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:59.712158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794183+0000","last_change":"2023-10-08T01:14:22.786530+0000","last_active":"2023-10-08T01:14:22.794183+0000","last_peered":"2023-10-08T01:14:22.794183+0000","last_clean":"2023-10-08T01:14:22.794183+0000","last_became_active":"2023-10-08T01:14:22.786260+0000","last_became_peered":"2023-10-08T01:14:22.786260+0000","last_unstale":"2023-10-08T01:14:22.794183+0000","last_undegraded":"2023-10-08T01:14:22.794183+0000","last_fullsized":"2023-10-08T01:14:22.794183+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:25.960964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788496+0000","last_change":"2023-10-08T01:14:22.783965+0000","last_active":"2023-10-08T01:14:22.788496+0000","last_peered":"2023-10-08T01:14:22.788496+0000","last_clean":"2023-10-08T01:14:22.788496+0000","last_became_active":"2023-10-08T01:14:22.783699+0000","last_became_peered":"2023-10-08T01:14:22.783699+0000","last_unstale":"2023-10-08T01:14:22.788496+0000","last_undegraded":"2023-10-08T01:14:22.788496+0000","last_fullsized":"2023-10-08T01:14:22.788496+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:06.030874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792319+0000","last_change":"2023-10-08T01:14:22.788581+0000","last_active":"2023-10-08T01:14:22.792319+0000","last_peered":"2023-10-08T01:14:22.792319+0000","last_clean":"2023-10-08T01:14:22.792319+0000","last_became_active":"2023-10-08T01:14:22.788210+0000","last_became_peered":"2023-10-08T01:14:22.788210+0000","last_unstale":"2023-10-08T01:14:22.792319+0000","last_undegraded":"2023-10-08T01:14:22.792319+0000","last_fullsized":"2023-10-08T01:14:22.792319+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792484+0000","last_change":"2023-10-08T01:14:22.790486+0000","last_active":"2023-10-08T01:14:22.792484+0000","last_peered":"2023-10-08T01:14:22.792484+0000","last_clean":"2023-10-08T01:14:22.792484+0000","last_became_active":"2023-10-08T01:14:22.790175+0000","last_became_peered":"2023-10-08T01:14:22.790175+0000","last_unstale":"2023-10-08T01:14:22.792484+0000","last_undegraded":"2023-10-08T01:14:22.792484+0000","last_fullsized":"2023-10-08T01:14:22.792484+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.779024+0000","last_change":"2023-10-08T01:14:22.777745+0000","last_active":"2023-10-08T01:14:22.779024+0000","last_peered":"2023-10-08T01:14:22.779024+0000","last_clean":"2023-10-08T01:14:22.779024+0000","last_became_active":"2023-10-08T01:14:22.777473+0000","last_became_peered":"2023-10-08T01:14:22.777473+0000","last_unstale":"2023-10-08T01:14:22.779024+0000","last_undegraded":"2023-10-08T01:14:22.779024+0000","last_fullsized":"2023-10-08T01:14:22.779024+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:57.315090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788635+0000","last_change":"2023-10-08T01:14:22.782436+0000","last_active":"2023-10-08T01:14:22.788635+0000","last_peered":"2023-10-08T01:14:22.788635+0000","last_clean":"2023-10-08T01:14:22.788635+0000","last_became_active":"2023-10-08T01:14:22.782125+0000","last_became_peered":"2023-10-08T01:14:22.782125+0000","last_unstale":"2023-10-08T01:14:22.788635+0000","last_undegraded":"2023-10-08T01:14:22.788635+0000","last_fullsized":"2023-10-08T01:14:22.788635+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:26.317877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792306+0000","last_change":"2023-10-08T01:14:22.789825+0000","last_active":"2023-10-08T01:14:22.792306+0000","last_peered":"2023-10-08T01:14:22.792306+0000","last_clean":"2023-10-08T01:14:22.792306+0000","last_became_active":"2023-10-08T01:14:22.789560+0000","last_became_peered":"2023-10-08T01:14:22.789560+0000","last_unstale":"2023-10-08T01:14:22.792306+0000","last_undegraded":"2023-10-08T01:14:22.792306+0000","last_fullsized":"2023-10-08T01:14:22.792306+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792179+0000","last_change":"2023-10-08T01:14:22.788302+0000","last_active":"2023-10-08T01:14:22.792179+0000","last_peered":"2023-10-08T01:14:22.792179+0000","last_clean":"2023-10-08T01:14:22.792179+0000","last_became_active":"2023-10-08T01:14:22.787799+0000","last_became_peered":"2023-10-08T01:14:22.787799+0000","last_unstale":"2023-10-08T01:14:22.792179+0000","last_undegraded":"2023-10-08T01:14:22.792179+0000","last_fullsized":"2023-10-08T01:14:22.792179+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783436+0000","last_change":"2023-10-08T01:14:22.779249+0000","last_active":"2023-10-08T01:14:22.783436+0000","last_peered":"2023-10-08T01:14:22.783436+0000","last_clean":"2023-10-08T01:14:22.783436+0000","last_became_active":"2023-10-08T01:14:22.779000+0000","last_became_peered":"2023-10-08T01:14:22.779000+0000","last_unstale":"2023-10-08T01:14:22.783436+0000","last_undegraded":"2023-10-08T01:14:22.783436+0000","last_fullsized":"2023-10-08T01:14:22.783436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:44:52.899739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788599+0000","last_change":"2023-10-08T01:14:22.782182+0000","last_active":"2023-10-08T01:14:22.788599+0000","last_peered":"2023-10-08T01:14:22.788599+0000","last_clean":"2023-10-08T01:14:22.788599+0000","last_became_active":"2023-10-08T01:14:22.781939+0000","last_became_peered":"2023-10-08T01:14:22.781939+0000","last_unstale":"2023-10-08T01:14:22.788599+0000","last_undegraded":"2023-10-08T01:14:22.788599+0000","last_fullsized":"2023-10-08T01:14:22.788599+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:19.690095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787156+0000","last_change":"2023-10-08T01:14:22.780281+0000","last_active":"2023-10-08T01:14:22.787156+0000","last_peered":"2023-10-08T01:14:22.787156+0000","last_clean":"2023-10-08T01:14:22.787156+0000","last_became_active":"2023-10-08T01:14:22.780033+0000","last_became_peered":"2023-10-08T01:14:22.780033+0000","last_unstale":"2023-10-08T01:14:22.787156+0000","last_undegraded":"2023-10-08T01:14:22.787156+0000","last_fullsized":"2023-10-08T01:14:22.787156+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:41.656567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799940+0000","last_change":"2023-10-08T01:14:22.799140+0000","last_active":"2023-10-08T01:14:22.799940+0000","last_peered":"2023-10-08T01:14:22.799940+0000","last_clean":"2023-10-08T01:14:22.799940+0000","last_became_active":"2023-10-08T01:14:22.798851+0000","last_became_peered":"2023-10-08T01:14:22.798851+0000","last_unstale":"2023-10-08T01:14:22.799940+0000","last_undegraded":"2023-10-08T01:14:22.799940+0000","last_fullsized":"2023-10-08T01:14:22.799940+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:30.232699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789675+0000","last_change":"2023-10-08T01:14:22.788999+0000","last_active":"2023-10-08T01:14:22.789675+0000","last_peered":"2023-10-08T01:14:22.789675+0000","last_clean":"2023-10-08T01:14:22.789675+0000","last_became_active":"2023-10-08T01:14:22.787618+0000","last_became_peered":"2023-10-08T01:14:22.787618+0000","last_unstale":"2023-10-08T01:14:22.789675+0000","last_undegraded":"2023-10-08T01:14:22.789675+0000","last_fullsized":"2023-10-08T01:14:22.789675+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:17.572620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791070+0000","last_change":"2023-10-08T01:14:22.788072+0000","last_active":"2023-10-08T01:14:22.791070+0000","last_peered":"2023-10-08T01:14:22.791070+0000","last_clean":"2023-10-08T01:14:22.791070+0000","last_became_active":"2023-10-08T01:14:22.787677+0000","last_became_peered":"2023-10-08T01:14:22.787677+0000","last_unstale":"2023-10-08T01:14:22.791070+0000","last_undegraded":"2023-10-08T01:14:22.791070+0000","last_fullsized":"2023-10-08T01:14:22.791070+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:19:12.933240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798256+0000","last_change":"2023-10-08T01:14:22.794783+0000","last_active":"2023-10-08T01:14:22.798256+0000","last_peered":"2023-10-08T01:14:22.798256+0000","last_clean":"2023-10-08T01:14:22.798256+0000","last_became_active":"2023-10-08T01:14:22.794304+0000","last_became_peered":"2023-10-08T01:14:22.794304+0000","last_unstale":"2023-10-08T01:14:22.798256+0000","last_undegraded":"2023-10-08T01:14:22.798256+0000","last_fullsized":"2023-10-08T01:14:22.798256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:59.697433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783465+0000","last_change":"2023-10-08T01:14:22.779446+0000","last_active":"2023-10-08T01:14:22.783465+0000","last_peered":"2023-10-08T01:14:22.783465+0000","last_clean":"2023-10-08T01:14:22.783465+0000","last_became_active":"2023-10-08T01:14:22.778675+0000","last_became_peered":"2023-10-08T01:14:22.778675+0000","last_unstale":"2023-10-08T01:14:22.783465+0000","last_undegraded":"2023-10-08T01:14:22.783465+0000","last_fullsized":"2023-10-08T01:14:22.783465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:34.335870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795814+0000","last_change":"2023-10-08T01:14:21.755008+0000","last_active":"2023-10-08T01:14:22.795814+0000","last_peered":"2023-10-08T01:14:22.795814+0000","last_clean":"2023-10-08T01:14:22.795814+0000","last_became_active":"2023-10-08T01:14:21.754712+0000","last_became_peered":"2023-10-08T01:14:21.754712+0000","last_unstale":"2023-10-08T01:14:22.795814+0000","last_undegraded":"2023-10-08T01:14:22.795814+0000","last_fullsized":"2023-10-08T01:14:22.795814+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:24.330581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748043+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799279+0000","last_change":"2023-10-08T01:14:22.798147+0000","last_active":"2023-10-08T01:14:22.799279+0000","last_peered":"2023-10-08T01:14:22.799279+0000","last_clean":"2023-10-08T01:14:22.799279+0000","last_became_active":"2023-10-08T01:14:22.797856+0000","last_became_peered":"2023-10-08T01:14:22.797856+0000","last_unstale":"2023-10-08T01:14:22.799279+0000","last_undegraded":"2023-10-08T01:14:22.799279+0000","last_fullsized":"2023-10-08T01:14:22.799279+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:32.952368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793353+0000","last_change":"2023-10-08T01:14:21.758907+0000","last_active":"2023-10-08T01:14:22.793353+0000","last_peered":"2023-10-08T01:14:22.793353+0000","last_clean":"2023-10-08T01:14:22.793353+0000","last_became_active":"2023-10-08T01:14:21.758669+0000","last_became_peered":"2023-10-08T01:14:21.758669+0000","last_unstale":"2023-10-08T01:14:22.793353+0000","last_undegraded":"2023-10-08T01:14:22.793353+0000","last_fullsized":"2023-10-08T01:14:22.793353+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:33.791532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782110+0000","last_change":"2023-10-08T01:14:22.777591+0000","last_active":"2023-10-08T01:14:22.782110+0000","last_peered":"2023-10-08T01:14:22.782110+0000","last_clean":"2023-10-08T01:14:22.782110+0000","last_became_active":"2023-10-08T01:14:22.777315+0000","last_became_peered":"2023-10-08T01:14:22.777315+0000","last_unstale":"2023-10-08T01:14:22.782110+0000","last_undegraded":"2023-10-08T01:14:22.782110+0000","last_fullsized":"2023-10-08T01:14:22.782110+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793240+0000","last_change":"2023-10-08T01:14:21.770043+0000","last_active":"2023-10-08T01:14:22.793240+0000","last_peered":"2023-10-08T01:14:22.793240+0000","last_clean":"2023-10-08T01:14:22.793240+0000","last_became_active":"2023-10-08T01:14:21.769799+0000","last_became_peered":"2023-10-08T01:14:21.769799+0000","last_unstale":"2023-10-08T01:14:22.793240+0000","last_undegraded":"2023-10-08T01:14:22.793240+0000","last_fullsized":"2023-10-08T01:14:22.793240+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:29.304154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.786306+0000","last_change":"2023-10-08T01:14:21.767177+0000","last_active":"2023-10-08T01:14:22.786306+0000","last_peered":"2023-10-08T01:14:22.786306+0000","last_clean":"2023-10-08T01:14:22.786306+0000","last_became_active":"2023-10-08T01:14:21.766891+0000","last_became_peered":"2023-10-08T01:14:21.766891+0000","last_unstale":"2023-10-08T01:14:22.786306+0000","last_undegraded":"2023-10-08T01:14:22.786306+0000","last_fullsized":"2023-10-08T01:14:22.786306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:48.667974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798893+0000","last_change":"2023-10-08T01:14:21.763020+0000","last_active":"2023-10-08T01:14:22.798893+0000","last_peered":"2023-10-08T01:14:22.798893+0000","last_clean":"2023-10-08T01:14:22.798893+0000","last_became_active":"2023-10-08T01:14:21.762782+0000","last_became_peered":"2023-10-08T01:14:21.762782+0000","last_unstale":"2023-10-08T01:14:22.798893+0000","last_undegraded":"2023-10-08T01:14:22.798893+0000","last_fullsized":"2023-10-08T01:14:22.798893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:59.904749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745429+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:59.004815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789574+0000","last_change":"2023-10-08T01:14:22.783171+0000","last_active":"2023-10-08T01:14:22.789574+0000","last_peered":"2023-10-08T01:14:22.789574+0000","last_clean":"2023-10-08T01:14:22.789574+0000","last_became_active":"2023-10-08T01:14:22.782872+0000","last_became_peered":"2023-10-08T01:14:22.782872+0000","last_unstale":"2023-10-08T01:14:22.789574+0000","last_undegraded":"2023-10-08T01:14:22.789574+0000","last_fullsized":"2023-10-08T01:14:22.789574+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.780252+0000","last_change":"2023-10-08T01:14:21.766845+0000","last_active":"2023-10-08T01:14:22.780252+0000","last_peered":"2023-10-08T01:14:22.780252+0000","last_clean":"2023-10-08T01:14:22.780252+0000","last_became_active":"2023-10-08T01:14:21.765270+0000","last_became_peered":"2023-10-08T01:14:21.765270+0000","last_unstale":"2023-10-08T01:14:22.780252+0000","last_undegraded":"2023-10-08T01:14:22.780252+0000","last_fullsized":"2023-10-08T01:14:22.780252+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:33.330389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789845+0000","last_change":"2023-10-08T01:14:22.786181+0000","last_active":"2023-10-08T01:14:22.789845+0000","last_peered":"2023-10-08T01:14:22.789845+0000","last_clean":"2023-10-08T01:14:22.789845+0000","last_became_active":"2023-10-08T01:14:22.785912+0000","last_became_peered":"2023-10-08T01:14:22.785912+0000","last_unstale":"2023-10-08T01:14:22.789845+0000","last_undegraded":"2023-10-08T01:14:22.789845+0000","last_fullsized":"2023-10-08T01:14:22.789845+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:33.304186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.781565+0000","last_change":"2023-10-08T01:14:21.763195+0000","last_active":"2023-10-08T01:14:22.781565+0000","last_peered":"2023-10-08T01:14:22.781565+0000","last_clean":"2023-10-08T01:14:22.781565+0000","last_became_active":"2023-10-08T01:14:21.762953+0000","last_became_peered":"2023-10-08T01:14:21.762953+0000","last_unstale":"2023-10-08T01:14:22.781565+0000","last_undegraded":"2023-10-08T01:14:22.781565+0000","last_fullsized":"2023-10-08T01:14:22.781565+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:56.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789148+0000","last_change":"2023-10-08T01:14:22.782447+0000","last_active":"2023-10-08T01:14:22.789148+0000","last_peered":"2023-10-08T01:14:22.789148+0000","last_clean":"2023-10-08T01:14:22.789148+0000","last_became_active":"2023-10-08T01:14:22.782195+0000","last_became_peered":"2023-10-08T01:14:22.782195+0000","last_unstale":"2023-10-08T01:14:22.789148+0000","last_undegraded":"2023-10-08T01:14:22.789148+0000","last_fullsized":"2023-10-08T01:14:22.789148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745607+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748365+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.800832+0000","last_change":"2023-10-08T01:14:21.764918+0000","last_active":"2023-10-08T01:14:22.800832+0000","last_peered":"2023-10-08T01:14:22.800832+0000","last_clean":"2023-10-08T01:14:22.800832+0000","last_became_active":"2023-10-08T01:14:21.763653+0000","last_became_peered":"2023-10-08T01:14:21.763653+0000","last_unstale":"2023-10-08T01:14:22.800832+0000","last_undegraded":"2023-10-08T01:14:22.800832+0000","last_fullsized":"2023-10-08T01:14:22.800832+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793156+0000","last_change":"2023-10-08T01:14:21.769762+0000","last_active":"2023-10-08T01:14:22.793156+0000","last_peered":"2023-10-08T01:14:22.793156+0000","last_clean":"2023-10-08T01:14:22.793156+0000","last_became_active":"2023-10-08T01:14:21.769522+0000","last_became_peered":"2023-10-08T01:14:21.769522+0000","last_unstale":"2023-10-08T01:14:22.793156+0000","last_undegraded":"2023-10-08T01:14:22.793156+0000","last_fullsized":"2023-10-08T01:14:22.793156+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:27.726884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797374+0000","last_change":"2023-10-08T01:14:22.791650+0000","last_active":"2023-10-08T01:14:22.797374+0000","last_peered":"2023-10-08T01:14:22.797374+0000","last_clean":"2023-10-08T01:14:22.797374+0000","last_became_active":"2023-10-08T01:14:22.791109+0000","last_became_peered":"2023-10-08T01:14:22.791109+0000","last_unstale":"2023-10-08T01:14:22.797374+0000","last_undegraded":"2023-10-08T01:14:22.797374+0000","last_fullsized":"2023-10-08T01:14:22.797374+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765608+0000","last_change":"2023-10-08T01:14:21.768459+0000","last_active":"2023-10-08T01:14:22.765608+0000","last_peered":"2023-10-08T01:14:22.765608+0000","last_clean":"2023-10-08T01:14:22.765608+0000","last_became_active":"2023-10-08T01:14:21.768189+0000","last_became_peered":"2023-10-08T01:14:21.768189+0000","last_unstale":"2023-10-08T01:14:22.765608+0000","last_undegraded":"2023-10-08T01:14:22.765608+0000","last_fullsized":"2023-10-08T01:14:22.765608+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:54.324955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789515+0000","last_change":"2023-10-08T01:14:22.781160+0000","last_active":"2023-10-08T01:14:22.789515+0000","last_peered":"2023-10-08T01:14:22.789515+0000","last_clean":"2023-10-08T01:14:22.789515+0000","last_became_active":"2023-10-08T01:14:22.780893+0000","last_became_peered":"2023-10-08T01:14:22.780893+0000","last_unstale":"2023-10-08T01:14:22.789515+0000","last_undegraded":"2023-10-08T01:14:22.789515+0000","last_fullsized":"2023-10-08T01:14:22.789515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:30.309636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768642+0000","last_change":"2023-10-08T01:14:21.760294+0000","last_active":"2023-10-08T01:14:22.768642+0000","last_peered":"2023-10-08T01:14:22.768642+0000","last_clean":"2023-10-08T01:14:22.768642+0000","last_became_active":"2023-10-08T01:14:21.758928+0000","last_became_peered":"2023-10-08T01:14:21.758928+0000","last_unstale":"2023-10-08T01:14:22.768642+0000","last_undegraded":"2023-10-08T01:14:22.768642+0000","last_fullsized":"2023-10-08T01:14:22.768642+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788009+0000","last_change":"2023-10-08T01:14:22.784690+0000","last_active":"2023-10-08T01:14:22.788009+0000","last_peered":"2023-10-08T01:14:22.788009+0000","last_clean":"2023-10-08T01:14:22.788009+0000","last_became_active":"2023-10-08T01:14:22.784408+0000","last_became_peered":"2023-10-08T01:14:22.784408+0000","last_unstale":"2023-10-08T01:14:22.788009+0000","last_undegraded":"2023-10-08T01:14:22.788009+0000","last_fullsized":"2023-10-08T01:14:22.788009+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:55.180105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795020+0000","last_change":"2023-10-08T01:14:21.765533+0000","last_active":"2023-10-08T01:14:22.795020+0000","last_peered":"2023-10-08T01:14:22.795020+0000","last_clean":"2023-10-08T01:14:22.795020+0000","last_became_active":"2023-10-08T01:14:21.764914+0000","last_became_peered":"2023-10-08T01:14:21.764914+0000","last_unstale":"2023-10-08T01:14:22.795020+0000","last_undegraded":"2023-10-08T01:14:22.795020+0000","last_fullsized":"2023-10-08T01:14:22.795020+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:00.128756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797139+0000","last_change":"2023-10-08T01:14:21.767123+0000","last_active":"2023-10-08T01:14:22.797139+0000","last_peered":"2023-10-08T01:14:22.797139+0000","last_clean":"2023-10-08T01:14:22.797139+0000","last_became_active":"2023-10-08T01:14:21.766826+0000","last_became_peered":"2023-10-08T01:14:21.766826+0000","last_unstale":"2023-10-08T01:14:22.797139+0000","last_undegraded":"2023-10-08T01:14:22.797139+0000","last_fullsized":"2023-10-08T01:14:22.797139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745225+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:13.819182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.767122+0000","last_change":"2023-10-08T01:14:21.769975+0000","last_active":"2023-10-08T01:14:22.767122+0000","last_peered":"2023-10-08T01:14:22.767122+0000","last_clean":"2023-10-08T01:14:22.767122+0000","last_became_active":"2023-10-08T01:14:21.769477+0000","last_became_peered":"2023-10-08T01:14:21.769477+0000","last_unstale":"2023-10-08T01:14:22.767122+0000","last_undegraded":"2023-10-08T01:14:22.767122+0000","last_fullsized":"2023-10-08T01:14:22.767122+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:34.177414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787894+0000","last_change":"2023-10-08T01:14:22.781041+0000","last_active":"2023-10-08T01:14:22.787894+0000","last_peered":"2023-10-08T01:14:22.787894+0000","last_clean":"2023-10-08T01:14:22.787894+0000","last_became_active":"2023-10-08T01:14:22.780765+0000","last_became_peered":"2023-10-08T01:14:22.780765+0000","last_unstale":"2023-10-08T01:14:22.787894+0000","last_undegraded":"2023-10-08T01:14:22.787894+0000","last_fullsized":"2023-10-08T01:14:22.787894+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:00.278431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792713+0000","last_change":"2023-10-08T01:14:21.768612+0000","last_active":"2023-10-08T01:14:22.792713+0000","last_peered":"2023-10-08T01:14:22.792713+0000","last_clean":"2023-10-08T01:14:22.792713+0000","last_became_active":"2023-10-08T01:14:21.767991+0000","last_became_peered":"2023-10-08T01:14:21.767991+0000","last_unstale":"2023-10-08T01:14:22.792713+0000","last_undegraded":"2023-10-08T01:14:22.792713+0000","last_fullsized":"2023-10-08T01:14:22.792713+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:50.772143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788094+0000","last_change":"2023-10-08T01:14:22.783198+0000","last_active":"2023-10-08T01:14:22.788094+0000","last_peered":"2023-10-08T01:14:22.788094+0000","last_clean":"2023-10-08T01:14:22.788094+0000","last_became_active":"2023-10-08T01:14:22.782950+0000","last_became_peered":"2023-10-08T01:14:22.782950+0000","last_unstale":"2023-10-08T01:14:22.788094+0000","last_undegraded":"2023-10-08T01:14:22.788094+0000","last_fullsized":"2023-10-08T01:14:22.788094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:30:30.559893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749667+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:31.946887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788041+0000","last_change":"2023-10-08T01:14:22.782578+0000","last_active":"2023-10-08T01:14:22.788041+0000","last_peered":"2023-10-08T01:14:22.788041+0000","last_clean":"2023-10-08T01:14:22.788041+0000","last_became_active":"2023-10-08T01:14:22.781703+0000","last_became_peered":"2023-10-08T01:14:22.781703+0000","last_unstale":"2023-10-08T01:14:22.788041+0000","last_undegraded":"2023-10-08T01:14:22.788041+0000","last_fullsized":"2023-10-08T01:14:22.788041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:38.983859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749808+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:13.819182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.763121+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.762883+0000","last_became_peered":"2023-10-08T01:14:21.762883+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:03.018597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789807+0000","last_change":"2023-10-08T01:14:22.782302+0000","last_active":"2023-10-08T01:14:22.789807+0000","last_peered":"2023-10-08T01:14:22.789807+0000","last_clean":"2023-10-08T01:14:22.789807+0000","last_became_active":"2023-10-08T01:14:22.780497+0000","last_became_peered":"2023-10-08T01:14:22.780497+0000","last_unstale":"2023-10-08T01:14:22.789807+0000","last_undegraded":"2023-10-08T01:14:22.789807+0000","last_fullsized":"2023-10-08T01:14:22.789807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:59.391189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.749695+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:10.276377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798148+0000","last_change":"2023-10-08T01:14:22.793832+0000","last_active":"2023-10-08T01:14:22.798148+0000","last_peered":"2023-10-08T01:14:22.798148+0000","last_clean":"2023-10-08T01:14:22.798148+0000","last_became_active":"2023-10-08T01:14:22.792633+0000","last_became_peered":"2023-10-08T01:14:22.792633+0000","last_unstale":"2023-10-08T01:14:22.798148+0000","last_undegraded":"2023-10-08T01:14:22.798148+0000","last_fullsized":"2023-10-08T01:14:22.798148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:11.281859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768046+0000","last_change":"2023-10-08T01:14:21.763932+0000","last_active":"2023-10-08T01:14:22.768046+0000","last_peered":"2023-10-08T01:14:22.768046+0000","last_clean":"2023-10-08T01:14:22.768046+0000","last_became_active":"2023-10-08T01:14:21.763673+0000","last_became_peered":"2023-10-08T01:14:21.763673+0000","last_unstale":"2023-10-08T01:14:22.768046+0000","last_undegraded":"2023-10-08T01:14:22.768046+0000","last_fullsized":"2023-10-08T01:14:22.768046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:07.616557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797649+0000","last_change":"2023-10-08T01:14:22.791844+0000","last_active":"2023-10-08T01:14:22.797649+0000","last_peered":"2023-10-08T01:14:22.797649+0000","last_clean":"2023-10-08T01:14:22.797649+0000","last_became_active":"2023-10-08T01:14:22.791583+0000","last_became_peered":"2023-10-08T01:14:22.791583+0000","last_unstale":"2023-10-08T01:14:22.797649+0000","last_undegraded":"2023-10-08T01:14:22.797649+0000","last_fullsized":"2023-10-08T01:14:22.797649+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.770984+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.770705+0000","last_became_peered":"2023-10-08T01:14:21.770705+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:48:20.485203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745127+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:10.276377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788178+0000","last_change":"2023-10-08T01:14:22.784755+0000","last_active":"2023-10-08T01:14:22.788178+0000","last_peered":"2023-10-08T01:14:22.788178+0000","last_clean":"2023-10-08T01:14:22.788178+0000","last_became_active":"2023-10-08T01:14:22.784506+0000","last_became_peered":"2023-10-08T01:14:22.784506+0000","last_unstale":"2023-10-08T01:14:22.788178+0000","last_undegraded":"2023-10-08T01:14:22.788178+0000","last_fullsized":"2023-10-08T01:14:22.788178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:12.608356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766909+0000","last_change":"2023-10-08T01:14:21.754925+0000","last_active":"2023-10-08T01:14:22.766909+0000","last_peered":"2023-10-08T01:14:22.766909+0000","last_clean":"2023-10-08T01:14:22.766909+0000","last_became_active":"2023-10-08T01:14:21.754606+0000","last_became_peered":"2023-10-08T01:14:21.754606+0000","last_unstale":"2023-10-08T01:14:22.766909+0000","last_undegraded":"2023-10-08T01:14:22.766909+0000","last_fullsized":"2023-10-08T01:14:22.766909+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:07.890275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.790593+0000","last_change":"2023-10-08T01:14:22.787439+0000","last_active":"2023-10-08T01:14:22.790593+0000","last_peered":"2023-10-08T01:14:22.790593+0000","last_clean":"2023-10-08T01:14:22.790593+0000","last_became_active":"2023-10-08T01:14:22.787105+0000","last_became_peered":"2023-10-08T01:14:22.787105+0000","last_unstale":"2023-10-08T01:14:22.790593+0000","last_undegraded":"2023-10-08T01:14:22.790593+0000","last_fullsized":"2023-10-08T01:14:22.790593+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745107+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:31.946887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789364+0000","last_change":"2023-10-08T01:14:22.785451+0000","last_active":"2023-10-08T01:14:22.789364+0000","last_peered":"2023-10-08T01:14:22.789364+0000","last_clean":"2023-10-08T01:14:22.789364+0000","last_became_active":"2023-10-08T01:14:22.785188+0000","last_became_peered":"2023-10-08T01:14:22.785188+0000","last_unstale":"2023-10-08T01:14:22.789364+0000","last_undegraded":"2023-10-08T01:14:22.789364+0000","last_fullsized":"2023-10-08T01:14:22.789364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.745527+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:14:21.740942+0000","last_change":"2023-10-08T01:14:21.740942+0000","last_active":"2023-10-08T01:14:21.740942+0000","last_peered":"2023-10-08T01:14:21.740942+0000","last_clean":"2023-10-08T01:14:21.740942+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:21.740942+0000","last_undegraded":"2023-10-08T01:14:21.740942+0000","last_fullsized":"2023-10-08T01:14:21.740942+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766989+0000","last_change":"2023-10-08T01:14:21.770863+0000","last_active":"2023-10-08T01:14:22.766989+0000","last_peered":"2023-10-08T01:14:22.766989+0000","last_clean":"2023-10-08T01:14:22.766989+0000","last_became_active":"2023-10-08T01:14:21.770532+0000","last_became_peered":"2023-10-08T01:14:21.770532+0000","last_unstale":"2023-10-08T01:14:22.766989+0000","last_undegraded":"2023-10-08T01:14:22.766989+0000","last_fullsized":"2023-10-08T01:14:22.766989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:32.319886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787982+0000","last_change":"2023-10-08T01:14:22.781044+0000","last_active":"2023-10-08T01:14:22.787982+0000","last_peered":"2023-10-08T01:14:22.787982+0000","last_clean":"2023-10-08T01:14:22.787982+0000","last_became_active":"2023-10-08T01:14:22.780772+0000","last_became_peered":"2023-10-08T01:14:22.780772+0000","last_unstale":"2023-10-08T01:14:22.787982+0000","last_undegraded":"2023-10-08T01:14:22.787982+0000","last_fullsized":"2023-10-08T01:14:22.787982+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:14:20.735460+0000","last_change":"2023-10-08T01:14:20.748071+0000","last_active":"2023-10-08T01:14:20.735460+0000","last_peered":"2023-10-08T01:14:20.735460+0000","last_clean":"2023-10-08T01:14:20.735460+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:14:20.735460+0000","last_undegraded":"2023-10-08T01:14:20.735460+0000","last_fullsized":"2023-10-08T01:14:20.735460+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"109'227","reported_seq":634,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-10-08T01:14:18.452767+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:14:18.452767+0000","last_peered":"2023-10-08T01:14:18.452767+0000","last_clean":"2023-10-08T01:14:18.452767+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:14:18.452767+0000","last_undegraded":"2023-10-08T01:14:18.452767+0000","last_fullsized":"2023-10-08T01:14:18.452767+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2124,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":30,"ondisk_log_size":30,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":72,"acting":72,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303376,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068464,"statfs":{"total":96636764160,"available":96326107136,"internally_reserved":0,"allocated":2039808,"data_stored":1295194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":1.123,"15min":1.123},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.97899999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67900000000000005}]},{"osd":1,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96999999999999997,"5min":1.133,"15min":1.133},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.72299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":1.167,"15min":1.167},"last":0.60099999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67100000000000004,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.91700000000000004,"5min":1.079,"15min":1.079},"last":0.58599999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.58299999999999996},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.88,"15min":0.88300000000000001},"last":0.61299999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.495},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":1.004,"15min":1.004},"last":0.64000000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72599999999999998,"15min":0.75},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302784,"kb_used_data":1400,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069056,"statfs":{"total":96636764160,"available":96326713344,"internally_reserved":0,"allocated":1433600,"data_stored":698629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68300000000000005,"15min":0.69999999999999996},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.69499999999999995},"min":{"1min":0.54500000000000004,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.48999999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.48399999999999999,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.984},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.59099999999999997,"5min":0.45300000000000001,"15min":0.38100000000000001},"max":{"1min":0.97099999999999997,"5min":1.018,"15min":3.0190000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.64100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.70299999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71999999999999997,"15min":0.745},"min":{"1min":0.55400000000000005,"5min":0.47599999999999998,"15min":0.47099999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.94399999999999995,"5min":1.048,"15min":1.048},"last":0.68200000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":1.087},"last":0.73099999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":0.84999999999999998,"5min":1.0189999999999999,"15min":1.395},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94999999999999996,"5min":1.0489999999999999,"15min":1.21},"last":0.54200000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":1.095,"15min":2.9489999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.747}]}]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":725316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.46700000000000003},"min":{"1min":0.32600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47799999999999998,"15min":0.503},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.745,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.377,"5min":0.317,"15min":0.26700000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.77300000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.53500000000000003},"min":{"1min":0.45100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.82099999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.53700000000000003},"min":{"1min":0.42399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":1.296,"5min":1.349,"15min":1.452},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.76200000000000001},"min":{"1min":0.55700000000000005,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.137,"5min":1.1499999999999999,"15min":1.1839999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73199999999999998,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":1.119,"5min":1.274,"15min":1.3959999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.5209999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73799999999999999,"15min":0.78500000000000003},"min":{"1min":0.56799999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.069,"5min":1.3839999999999999,"15min":1.6020000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74199999999999999,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1499999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.621}]},{"osd":7,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.82099999999999995},"min":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.236,"5min":1.3109999999999999,"15min":1.621},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78500000000000003,"15min":0.82899999999999996},"min":{"1min":0.66000000000000003,"5min":0.58099999999999996,"15min":0.51300000000000001},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.4890000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":1316238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.625,"5min":0.95999999999999996,"15min":0.997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67700000000000005,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46800000000000003,"15min":0.48299999999999998},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.53000000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.70599999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.59899999999999998},"min":{"1min":0.504,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.754},"min":{"1min":0.53500000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94099999999999995,"5min":1.341,"15min":1.617},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.61299999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92100000000000004,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99199999999999999,"5min":2.085,"15min":2.085},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79100000000000004,"15min":0.81699999999999995},"min":{"1min":0.56699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.381,"15min":1.5289999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77100000000000002,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98299999999999998,"5min":1.361,"15min":1.3779999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99399999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77900000000000003,"15min":0.81299999999999994},"min":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.077,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.871},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81399999999999995,"15min":0.83899999999999997},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.028,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.89400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311028,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060812,"statfs":{"total":96636764160,"available":96318271488,"internally_reserved":0,"allocated":1486848,"data_stored":725350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.41099999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.46000000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.88800000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.90400000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575068,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306892,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064948,"statfs":{"total":96636764160,"available":96322506752,"internally_reserved":0,"allocated":1445888,"data_stored":704826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.46200000000000002},"min":{"1min":0.32000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54400000000000004,"5min":0.627,"15min":0.80100000000000005},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":0.81200000000000006},"last":0.48299999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.28299999999999997},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.495,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.311,"15min":0.31},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.46100000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.89700000000000002},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.5,"5min":0.438,"15min":0.39600000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.57499999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.752},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.41099999999999998},"max":{"1min":1.028,"5min":1.095,"15min":1.3029999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.046,"5min":1.0580000000000001,"15min":1.413},"last":0.88400000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.77300000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51000000000000001},"max":{"1min":0.995,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.67500000000000004,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.089},"last":0.98799999999999999}]},{"osd":6,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.57199999999999995,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.94799999999999995,"5min":1.101,"15min":1.379},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72899999999999998,"15min":0.77900000000000003},"min":{"1min":0.59499999999999997,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.486},"last":0.85999999999999999}]},{"osd":7,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.755,"15min":0.79300000000000004},"min":{"1min":0.63200000000000001,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97799999999999998,"5min":1.0840000000000001,"15min":1.389},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.79500000000000004},"min":{"1min":0.63300000000000001,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.96699999999999997,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.749}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311044,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060796,"statfs":{"total":96636764160,"available":96318255104,"internally_reserved":0,"allocated":1503232,"data_stored":726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.84299999999999997}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.875}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.70999999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.92400000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.80100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575068,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302796,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069044,"statfs":{"total":96636764160,"available":96326701056,"internally_reserved":0,"allocated":1445888,"data_stored":704826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.71699999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.65000000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":1.026,"15min":1.026},"last":0.754},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.53300000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.38700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.58499999999999996}]},{"osd":3,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71999999999999997,"15min":0.73299999999999998},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.98699999999999999,"5min":1.034,"15min":1.034},"last":0.68700000000000006}]},{"osd":4,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.874,"15min":0.91900000000000004},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.90300000000000002,"15min":1.0680000000000001},"last":0.59999999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.90200000000000002,"15min":0.95699999999999996},"last":0.71499999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":0.94399999999999995,"15min":1.095},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47399999999999998},"max":{"1min":0.90200000000000002,"5min":1.05,"15min":1.05},"last":0.79400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:14:28.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-10-08T01:14:28.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.001 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-10-08T01:14:28.001 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-10-08T01:14:28.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-10-08T01:14:28.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-10-08T01:14:28.002 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.002 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-10-08T01:14:28.003 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-10-08T01:14:28.004 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-10-08T01:14:28.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2023-10-08T01:14:28.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.748043+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:14:28.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-10-08T01:14:28.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.005 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-10-08T01:14:28.005 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.006 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-10-08T01:14:28.006 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2023-10-08T01:14:28.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.745429+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:13:59.004815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:14:28.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2023-10-08T01:14:28.007 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.745607+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:14:28.007 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.748365+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-10-08T01:14:28.008 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.745225+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:13:13.819182+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.749667+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:28:31.946887+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:14:28.009 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.749808+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:13:13.819182+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.749695+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:58:10.276377+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2023-10-08T01:14:28.010 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.745127+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:58:10.276377+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:14:28.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2023-10-08T01:14:28.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.745107+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:28:31.946887+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:14:28.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2023-10-08T01:14:28.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.745527+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:14:28.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-10-08T01:14:28.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:14:21.740942+0000', 'last_change': '2023-10-08T01:14:21.740942+0000', 'last_active': '2023-10-08T01:14:21.740942+0000', 'last_peered': '2023-10-08T01:14:21.740942+0000', 'last_clean': '2023-10-08T01:14:21.740942+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:21.740942+0000', 'last_undegraded': '2023-10-08T01:14:21.740942+0000', 'last_fullsized': '2023-10-08T01:14:21.740942+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:21.740942+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:14:28.011 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2023-10-08T01:14:28.011 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:14:20.735460+0000', 'last_change': '2023-10-08T01:14:20.748071+0000', 'last_active': '2023-10-08T01:14:20.735460+0000', 'last_peered': '2023-10-08T01:14:20.735460+0000', 'last_clean': '2023-10-08T01:14:20.735460+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:14:20.735460+0000', 'last_undegraded': '2023-10-08T01:14:20.735460+0000', 'last_fullsized': '2023-10-08T01:14:20.735460+0000', 'mapping_epoch': 117, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 117, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'last_clean_scrub_stamp': '2023-10-08T01:14:20.735460+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:14:28.012 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T01:14:28.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.209+0000 7f59e21ff700 1 Processor -- start 2023-10-08T01:14:28.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.209+0000 7f59e21ff700 1 -- start start 2023-10-08T01:14:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59e21ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59dc156770 0x7f59dc0b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59e21ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc0b0300 0x7f59dc0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59e21ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc0c9810 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59e21ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc0c9680 con 0x7f59dc156770 2023-10-08T01:14:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59e21ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0adf80 con 0x7f59dc0b0300 2023-10-08T01:14:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59db7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59dc156770 0x7f59dc0b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59daffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f59dc0ad9b0 0x7f59dc0c9ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59734/0 (socket says 172.21.15.119:59734) 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59daffd700 1 -- 172.21.15.119:0/2446520958 learned_addr learned my addr 172.21.15.119:0/2446520958 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.211+0000 7f59dbfff700 1 --2- 172.21.15.119:0/2446520958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc0b0300 0x7f59dc0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59db7fe700 1 -- 172.21.15.119:0/2446520958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc0b0300 msgr2=0x7f59dc0b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59db7fe700 1 --2- 172.21.15.119:0/2446520958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc0b0300 0x7f59dc0b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59db7fe700 1 -- 172.21.15.119:0/2446520958 >> v1:172.21.15.119:6789/0 conn(0x7f59dc0ad9b0 legacy=0x7f59dc0c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59db7fe700 1 -- 172.21.15.119:0/2446520958 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc0ae260 con 0x7f59dc156770 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59da7fc700 1 -- 172.21.15.119:0/2446520958 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965414277 0 0) 0x7f59dc0c9810 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59dbfff700 1 --2- 172.21.15.119:0/2446520958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc0b0300 0x7f59dc0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59db7fe700 1 --2- 172.21.15.119:0/2446520958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc156770 0x7f59dc0b0bd0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098e0 tx=0x7f59cc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18ecf56e7616d8a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59da7fc700 1 -- 172.21.15.119:0/2446520958 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc009e80 con 0x7f59dc156770 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59da7fc700 1 -- 172.21.15.119:0/2446520958 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc00b860 con 0x7f59dc156770 2023-10-08T01:14:28.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.212+0000 7f59da7fc700 1 -- 172.21.15.119:0/2446520958 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00bbe0 con 0x7f59dc156770 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.213+0000 7f59e21ff700 1 -- 172.21.15.119:0/2446520958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc156770 msgr2=0x7f59dc0b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.213+0000 7f59e21ff700 1 --2- 172.21.15.119:0/2446520958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc156770 0x7f59dc0b0bd0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098e0 tx=0x7f59cc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.213+0000 7f59e21ff700 1 -- 172.21.15.119:0/2446520958 shutdown_connections 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.213+0000 7f59e21ff700 1 --2- 172.21.15.119:0/2446520958 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc0b0300 0x7f59dc0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.213+0000 7f59e21ff700 1 --2- 172.21.15.119:0/2446520958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc156770 0x7f59dc0b0bd0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.213+0000 7f59e21ff700 1 -- 172.21.15.119:0/2446520958 >> 172.21.15.119:0/2446520958 conn(0x7f59dc06a310 msgr2=0x7f59dc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.213+0000 7f59e21ff700 1 -- 172.21.15.119:0/2446520958 shutdown_connections 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.213+0000 7f59e21ff700 1 -- 172.21.15.119:0/2446520958 wait complete. 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.214+0000 7f59e21ff700 1 Processor -- start 2023-10-08T01:14:28.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.214+0000 7f59e21ff700 1 -- start start 2023-10-08T01:14:28.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.215+0000 7f59e21ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc0ad9b0 0x7f59dc0a8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.215+0000 7f59e21ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59dc0b0300 0x7f59dc168710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.215+0000 7f59e21ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc156770 0x7f59dc168fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.215+0000 7f59e21ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc144db0 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.215+0000 7f59e21ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0a7460 con 0x7f59dc0b0300 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.215+0000 7f59e21ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0a7740 con 0x7f59dc156770 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc0ad9b0 0x7f59dc0a8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc0ad9b0 0x7f59dc0a8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40716/0 (socket says 172.21.15.119:40716) 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 -- 172.21.15.119:0/890846676 learned_addr learned my addr 172.21.15.119:0/890846676 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59daffd700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59dc0b0300 0x7f59dc168710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59dbfff700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc156770 0x7f59dc168fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 -- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc156770 msgr2=0x7f59dc168fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc156770 0x7f59dc168fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 -- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59dc0b0300 msgr2=0x7f59dc168710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59dc0b0300 0x7f59dc168710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 -- 172.21.15.119:0/890846676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc009420 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59daffd700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59dc0b0300 0x7f59dc168710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.216+0000 7f59db7fe700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc0ad9b0 0x7f59dc0a8520 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f59dc06d540 tx=0x7f59cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.217+0000 7f59c37fe700 1 -- 172.21.15.119:0/890846676 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc00bbe0 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.217+0000 7f59c37fe700 1 -- 172.21.15.119:0/890846676 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc000cc0 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.217+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc0a7a20 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.217+0000 7f59c37fe700 1 -- 172.21.15.119:0/890846676 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc011670 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.217+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc0a8a60 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.218+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc04c3c0 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.219+0000 7f59c37fe700 1 -- 172.21.15.119:0/890846676 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f59cc011a20 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.219+0000 7f59c37fe700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f59c8066f30 0x7f59c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.219+0000 7f59c37fe700 1 -- 172.21.15.119:0/890846676 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f59cc085b10 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.220+0000 7f59daffd700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f59c8066f30 0x7f59c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.220+0000 7f59daffd700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f59c8066f30 0x7f59c80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59dc06f250 tx=0x7f59d0009040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:28.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.224+0000 7f59c37fe700 1 -- 172.21.15.119:0/890846676 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f59cc052bf0 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.489+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f59dc0ae540 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.490+0000 7f59c37fe700 1 -- 172.21.15.119:0/890846676 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1179 (secure 0 0 0) 0x7f59cc056210 con 0x7f59dc0ad9b0 2023-10-08T01:14:28.490 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:28.490 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1145,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1696726535,"num_in_osds":8,"osd_in_since":1696726526,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":57},{"state_name":"unknown","count":28},{"state_name":"creating+peering","count":12}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592492,"bytes_used":2502828032,"bytes_avail":770591285248,"bytes_total":773094113280,"unknown_pgs_ratio":0.28865978121757507,"inactive_pgs_ratio":0.12371134012937546,"write_bytes_sec":1716,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":55,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7680}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-10-08T01:14:02.449356+0000","services":{}},"progress_events":{}} 2023-10-08T01:14:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f59c8066f30 msgr2=0x7f59c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f59c8066f30 0x7f59c80693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59dc06f250 tx=0x7f59d0009040 comp rx=0 tx=0).stop 2023-10-08T01:14:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc0ad9b0 msgr2=0x7f59dc0a8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc0ad9b0 0x7f59dc0a8520 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f59dc06d540 tx=0x7f59cc017000 comp rx=0 tx=0).stop 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 shutdown_connections 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f59c8066f30 0x7f59c80693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f59dc156770 0x7f59dc168fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59dc0ad9b0 0x7f59dc0a8520 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 --2- 172.21.15.119:0/890846676 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f59dc0b0300 0x7f59dc168710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 >> 172.21.15.119:0/890846676 conn(0x7f59dc06a310 msgr2=0x7f59dc0c8940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 shutdown_connections 2023-10-08T01:14:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.492+0000 7f59e21ff700 1 -- 172.21.15.119:0/890846676 wait complete. 2023-10-08T01:14:28.510 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T01:14:28.511 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:14:28.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.690+0000 7feffd3ff700 1 Processor -- start 2023-10-08T01:14:28.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.691+0000 7feffd3ff700 1 -- start start 2023-10-08T01:14:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.692+0000 7feffd3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80ccf90 0x7feff80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.692+0000 7feffd3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80cc300 con 0x7feff80b1420 2023-10-08T01:14:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.692+0000 7feffd3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff80ad370 con 0x7feff80cbf30 2023-10-08T01:14:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.692+0000 7feffd3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff8156bf0 con 0x7feff80ccf90 2023-10-08T01:14:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.692+0000 7feff67fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7feff80b1420 0x7feff80c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59740/0 (socket says 172.21.15.119:59740) 2023-10-08T01:14:28.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.692+0000 7feff67fc700 1 -- 172.21.15.119:0/484387491 learned_addr learned my addr 172.21.15.119:0/484387491 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:28.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.692+0000 7feff77fe700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80ccf90 0x7feff80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.692+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2005799316 0 0) 0x7feff80cc300 con 0x7feff80b1420 2023-10-08T01:14:28.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefe0003570 con 0x7feff80b1420 2023-10-08T01:14:28.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 145667000 0 0) 0x7fefe0003570 con 0x7feff80b1420 2023-10-08T01:14:28.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80ccf90 msgr2=0x7feff80ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:28.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80ccf90 0x7feff80ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 >> v1:172.21.15.184:6789/0 conn(0x7feff80cbf30 legacy=0x7feff80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80ad650 con 0x7feff80b1420 2023-10-08T01:14:28.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fefec001470 con 0x7feff80b1420 2023-10-08T01:14:28.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 >> v1:172.21.15.119:6789/0 conn(0x7feff80b1420 legacy=0x7feff80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80ccf90 0x7fefe0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0003fd0 0x7fefe0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe0004d50 0x7fefe00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff8156bf0 con 0x7fefe0004d50 2023-10-08T01:14:28.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe000bb40 con 0x7fefe0003fd0 2023-10-08T01:14:28.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe000df10 con 0x7feff80ccf90 2023-10-08T01:14:28.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff77fe700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe0004d50 0x7fefe00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff67fc700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0003fd0 0x7fefe0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.693+0000 7feff6ffd700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80ccf90 0x7fefe0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff77fe700 1 -- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80ccf90 msgr2=0x7fefe0003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:28.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff77fe700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80ccf90 0x7fefe0003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff77fe700 1 -- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0003fd0 msgr2=0x7fefe0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:28.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff77fe700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0003fd0 0x7fefe0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff77fe700 1 -- 172.21.15.119:0/484387491 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe0003570 con 0x7fefe0004d50 2023-10-08T01:14:28.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff67fc700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0003fd0 0x7fefe0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:28.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff77fe700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe0004d50 0x7fefe00051c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7feff000e210 tx=0x7feff000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ced76a4a05c8316 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:28.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000eac0 con 0x7fefe0004d50 2023-10-08T01:14:28.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff000f040 con 0x7fefe0004d50 2023-10-08T01:14:28.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.694+0000 7feff5ffb700 1 -- 172.21.15.119:0/484387491 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000ee70 con 0x7fefe0004d50 2023-10-08T01:14:28.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.695+0000 7feffd3ff700 1 -- 172.21.15.119:0/484387491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe0004d50 msgr2=0x7fefe00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.695+0000 7feffd3ff700 1 --2- 172.21.15.119:0/484387491 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefe0004d50 0x7fefe00051c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7feff000e210 tx=0x7feff000b040 comp rx=0 tx=0).stop 2023-10-08T01:14:28.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.695+0000 7feff77fe700 1 -- 172.21.15.119:0/484387491 reap_dead start 2023-10-08T01:14:28.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.695+0000 7feffd3ff700 1 -- 172.21.15.119:0/484387491 shutdown_connections 2023-10-08T01:14:28.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.695+0000 7feffd3ff700 1 -- 172.21.15.119:0/484387491 >> 172.21.15.119:0/484387491 conn(0x7feff806a320 msgr2=0x7feff80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:28.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.695+0000 7feffd3ff700 1 -- 172.21.15.119:0/484387491 shutdown_connections 2023-10-08T01:14:28.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.695+0000 7feffd3ff700 1 -- 172.21.15.119:0/484387491 wait complete. 2023-10-08T01:14:28.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.696+0000 7feffd3ff700 1 Processor -- start 2023-10-08T01:14:28.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.697+0000 7feffd3ff700 1 -- start start 2023-10-08T01:14:28.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.697+0000 7feffd3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0004d50 0x7feff80aba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.697+0000 7feffd3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80c7d80 0x7feff8155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.697+0000 7feffd3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80a8c60 0x7feff80a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.697+0000 7feffd3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80a95f0 con 0x7feff80a8c60 2023-10-08T01:14:28.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.697+0000 7feffd3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff80ac2d0 con 0x7fefe0004d50 2023-10-08T01:14:28.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.697+0000 7feffd3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff80ac5b0 con 0x7feff80c7d80 2023-10-08T01:14:28.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80a8c60 0x7feff80a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80a8c60 0x7feff80a90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40728/0 (socket says 172.21.15.119:40728) 2023-10-08T01:14:28.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 -- 172.21.15.119:0/4112563270 learned_addr learned my addr 172.21.15.119:0/4112563270 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:28.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff6ffd700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0004d50 0x7feff80aba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff67fc700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80c7d80 0x7feff8155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 -- 172.21.15.119:0/4112563270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80c7d80 msgr2=0x7feff8155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80c7d80 0x7feff8155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 -- 172.21.15.119:0/4112563270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0004d50 msgr2=0x7feff80aba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0004d50 0x7feff80aba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 -- 172.21.15.119:0/4112563270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff80ac890 con 0x7feff80a8c60 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.698+0000 7feff77fe700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80a8c60 0x7feff80a90b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7feff000afd0 tx=0x7feff001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.699+0000 7fefdeffd700 1 -- 172.21.15.119:0/4112563270 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff001d630 con 0x7feff80a8c60 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.699+0000 7fefdeffd700 1 -- 172.21.15.119:0/4112563270 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff001d790 con 0x7feff80a8c60 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.699+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff80acb70 con 0x7feff80a8c60 2023-10-08T01:14:28.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.699+0000 7fefdeffd700 1 -- 172.21.15.119:0/4112563270 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff001db90 con 0x7feff80a8c60 2023-10-08T01:14:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.699+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff8156510 con 0x7feff80a8c60 2023-10-08T01:14:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.700+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff804c3c0 con 0x7feff80a8c60 2023-10-08T01:14:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.701+0000 7fefdeffd700 1 -- 172.21.15.119:0/4112563270 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feff0003040 con 0x7feff80a8c60 2023-10-08T01:14:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.701+0000 7fefdeffd700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fefe4066f30 0x7fefe40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.701+0000 7fefdeffd700 1 -- 172.21.15.119:0/4112563270 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7feff0085ac0 con 0x7feff80a8c60 2023-10-08T01:14:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.702+0000 7feff6ffd700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fefe4066f30 0x7fefe40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.702+0000 7feff6ffd700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fefe4066f30 0x7fefe40693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fefe8007e40 tx=0x7fefe8007d30 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.706+0000 7fefdeffd700 1 -- 172.21.15.119:0/4112563270 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feff0052c60 con 0x7feff80a8c60 2023-10-08T01:14:28.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.893+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feff80ad930 con 0x7fefe4066f30 2023-10-08T01:14:28.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.902+0000 7fefdeffd700 1 -- 172.21.15.119:0/4112563270 <== mgr.7488 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289491 (secure 0 0 0) 0x7feff80ad930 con 0x7fefe4066f30 2023-10-08T01:14:28.906 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:28.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fefe4066f30 msgr2=0x7fefe40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fefe4066f30 0x7fefe40693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fefe8007e40 tx=0x7fefe8007d30 comp rx=0 tx=0).stop 2023-10-08T01:14:28.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80a8c60 msgr2=0x7feff80a90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:28.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80a8c60 0x7feff80a90b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7feff000afd0 tx=0x7feff001a000 comp rx=0 tx=0).stop 2023-10-08T01:14:28.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 shutdown_connections 2023-10-08T01:14:28.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fefe4066f30 0x7fefe40693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feff80a8c60 0x7feff80a90b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fefe0004d50 0x7feff80aba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 --2- 172.21.15.119:0/4112563270 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feff80c7d80 0x7feff8155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:28.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 >> 172.21.15.119:0/4112563270 conn(0x7feff806a320 msgr2=0x7feff80c9950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:28.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 shutdown_connections 2023-10-08T01:14:28.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:28.906+0000 7feffd3ff700 1 -- 172.21.15.119:0/4112563270 wait complete. 2023-10-08T01:14:28.910 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:14:28.925 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T01:14:28.707533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2448456,"kb_used_data":12808,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752526264,"statfs":{"total":773094113280,"available":770586894336,"internally_reserved":0,"allocated":13115392,"data_stored":6987876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.963023"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798738+0000","last_change":"2023-10-08T01:14:22.797353+0000","last_active":"2023-10-08T01:14:22.798738+0000","last_peered":"2023-10-08T01:14:22.798738+0000","last_clean":"2023-10-08T01:14:22.798738+0000","last_became_active":"2023-10-08T01:14:22.795282+0000","last_became_peered":"2023-10-08T01:14:22.795282+0000","last_unstale":"2023-10-08T01:14:22.798738+0000","last_undegraded":"2023-10-08T01:14:22.798738+0000","last_fullsized":"2023-10-08T01:14:22.798738+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:19.889654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789827+0000","last_change":"2023-10-08T01:14:22.782483+0000","last_active":"2023-10-08T01:14:22.789827+0000","last_peered":"2023-10-08T01:14:22.789827+0000","last_clean":"2023-10-08T01:14:22.789827+0000","last_became_active":"2023-10-08T01:14:22.781426+0000","last_became_peered":"2023-10-08T01:14:22.781426+0000","last_unstale":"2023-10-08T01:14:22.789827+0000","last_undegraded":"2023-10-08T01:14:22.789827+0000","last_fullsized":"2023-10-08T01:14:22.789827+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:59.712158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794183+0000","last_change":"2023-10-08T01:14:22.786530+0000","last_active":"2023-10-08T01:14:22.794183+0000","last_peered":"2023-10-08T01:14:22.794183+0000","last_clean":"2023-10-08T01:14:22.794183+0000","last_became_active":"2023-10-08T01:14:22.786260+0000","last_became_peered":"2023-10-08T01:14:22.786260+0000","last_unstale":"2023-10-08T01:14:22.794183+0000","last_undegraded":"2023-10-08T01:14:22.794183+0000","last_fullsized":"2023-10-08T01:14:22.794183+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:25.960964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787217+0000","last_change":"2023-10-08T01:14:22.778060+0000","last_active":"2023-10-08T01:14:22.787217+0000","last_peered":"2023-10-08T01:14:22.787217+0000","last_clean":"2023-10-08T01:14:22.787217+0000","last_became_active":"2023-10-08T01:14:22.777818+0000","last_became_peered":"2023-10-08T01:14:22.777818+0000","last_unstale":"2023-10-08T01:14:22.787217+0000","last_undegraded":"2023-10-08T01:14:22.787217+0000","last_fullsized":"2023-10-08T01:14:22.787217+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787408+0000","last_change":"2023-10-08T01:14:22.783478+0000","last_active":"2023-10-08T01:14:22.787408+0000","last_peered":"2023-10-08T01:14:22.787408+0000","last_clean":"2023-10-08T01:14:22.787408+0000","last_became_active":"2023-10-08T01:14:22.783208+0000","last_became_peered":"2023-10-08T01:14:22.783208+0000","last_unstale":"2023-10-08T01:14:22.787408+0000","last_undegraded":"2023-10-08T01:14:22.787408+0000","last_fullsized":"2023-10-08T01:14:22.787408+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.785834+0000","last_change":"2023-10-08T01:14:22.780525+0000","last_active":"2023-10-08T01:14:22.785834+0000","last_peered":"2023-10-08T01:14:22.785834+0000","last_clean":"2023-10-08T01:14:22.785834+0000","last_became_active":"2023-10-08T01:14:22.780240+0000","last_became_peered":"2023-10-08T01:14:22.780240+0000","last_unstale":"2023-10-08T01:14:22.785834+0000","last_undegraded":"2023-10-08T01:14:22.785834+0000","last_fullsized":"2023-10-08T01:14:22.785834+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783228+0000","last_change":"2023-10-08T01:14:22.780046+0000","last_active":"2023-10-08T01:14:22.783228+0000","last_peered":"2023-10-08T01:14:22.783228+0000","last_clean":"2023-10-08T01:14:22.783228+0000","last_became_active":"2023-10-08T01:14:22.779731+0000","last_became_peered":"2023-10-08T01:14:22.779731+0000","last_unstale":"2023-10-08T01:14:22.783228+0000","last_undegraded":"2023-10-08T01:14:22.783228+0000","last_fullsized":"2023-10-08T01:14:22.783228+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788496+0000","last_change":"2023-10-08T01:14:22.783965+0000","last_active":"2023-10-08T01:14:22.788496+0000","last_peered":"2023-10-08T01:14:22.788496+0000","last_clean":"2023-10-08T01:14:22.788496+0000","last_became_active":"2023-10-08T01:14:22.783699+0000","last_became_peered":"2023-10-08T01:14:22.783699+0000","last_unstale":"2023-10-08T01:14:22.788496+0000","last_undegraded":"2023-10-08T01:14:22.788496+0000","last_fullsized":"2023-10-08T01:14:22.788496+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:06.030874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792319+0000","last_change":"2023-10-08T01:14:22.788581+0000","last_active":"2023-10-08T01:14:22.792319+0000","last_peered":"2023-10-08T01:14:22.792319+0000","last_clean":"2023-10-08T01:14:22.792319+0000","last_became_active":"2023-10-08T01:14:22.788210+0000","last_became_peered":"2023-10-08T01:14:22.788210+0000","last_unstale":"2023-10-08T01:14:22.792319+0000","last_undegraded":"2023-10-08T01:14:22.792319+0000","last_fullsized":"2023-10-08T01:14:22.792319+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795777+0000","last_change":"2023-10-08T01:14:22.794165+0000","last_active":"2023-10-08T01:14:22.795777+0000","last_peered":"2023-10-08T01:14:22.795777+0000","last_clean":"2023-10-08T01:14:22.795777+0000","last_became_active":"2023-10-08T01:14:22.793828+0000","last_became_peered":"2023-10-08T01:14:22.793828+0000","last_unstale":"2023-10-08T01:14:22.795777+0000","last_undegraded":"2023-10-08T01:14:22.795777+0000","last_fullsized":"2023-10-08T01:14:22.795777+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:30.309636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782072+0000","last_change":"2023-10-08T01:14:22.778678+0000","last_active":"2023-10-08T01:14:22.782072+0000","last_peered":"2023-10-08T01:14:22.782072+0000","last_clean":"2023-10-08T01:14:22.782072+0000","last_became_active":"2023-10-08T01:14:22.777690+0000","last_became_peered":"2023-10-08T01:14:22.777690+0000","last_unstale":"2023-10-08T01:14:22.782072+0000","last_undegraded":"2023-10-08T01:14:22.782072+0000","last_fullsized":"2023-10-08T01:14:22.782072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:30.209334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792484+0000","last_change":"2023-10-08T01:14:22.790486+0000","last_active":"2023-10-08T01:14:22.792484+0000","last_peered":"2023-10-08T01:14:22.792484+0000","last_clean":"2023-10-08T01:14:22.792484+0000","last_became_active":"2023-10-08T01:14:22.790175+0000","last_became_peered":"2023-10-08T01:14:22.790175+0000","last_unstale":"2023-10-08T01:14:22.792484+0000","last_undegraded":"2023-10-08T01:14:22.792484+0000","last_fullsized":"2023-10-08T01:14:22.792484+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794520+0000","last_change":"2023-10-08T01:14:22.792182+0000","last_active":"2023-10-08T01:14:22.794520+0000","last_peered":"2023-10-08T01:14:22.794520+0000","last_clean":"2023-10-08T01:14:22.794520+0000","last_became_active":"2023-10-08T01:14:22.791942+0000","last_became_peered":"2023-10-08T01:14:22.791942+0000","last_unstale":"2023-10-08T01:14:22.794520+0000","last_undegraded":"2023-10-08T01:14:22.794520+0000","last_fullsized":"2023-10-08T01:14:22.794520+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.779024+0000","last_change":"2023-10-08T01:14:22.777745+0000","last_active":"2023-10-08T01:14:22.779024+0000","last_peered":"2023-10-08T01:14:22.779024+0000","last_clean":"2023-10-08T01:14:22.779024+0000","last_became_active":"2023-10-08T01:14:22.777473+0000","last_became_peered":"2023-10-08T01:14:22.777473+0000","last_unstale":"2023-10-08T01:14:22.779024+0000","last_undegraded":"2023-10-08T01:14:22.779024+0000","last_fullsized":"2023-10-08T01:14:22.779024+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:57.315090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793463+0000","last_change":"2023-10-08T01:14:22.790518+0000","last_active":"2023-10-08T01:14:22.793463+0000","last_peered":"2023-10-08T01:14:22.793463+0000","last_clean":"2023-10-08T01:14:22.793463+0000","last_became_active":"2023-10-08T01:14:22.790204+0000","last_became_peered":"2023-10-08T01:14:22.790204+0000","last_unstale":"2023-10-08T01:14:22.793463+0000","last_undegraded":"2023-10-08T01:14:22.793463+0000","last_fullsized":"2023-10-08T01:14:22.793463+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789520+0000","last_change":"2023-10-08T01:14:22.785475+0000","last_active":"2023-10-08T01:14:22.789520+0000","last_peered":"2023-10-08T01:14:22.789520+0000","last_clean":"2023-10-08T01:14:22.789520+0000","last_became_active":"2023-10-08T01:14:22.785148+0000","last_became_peered":"2023-10-08T01:14:22.785148+0000","last_unstale":"2023-10-08T01:14:22.789520+0000","last_undegraded":"2023-10-08T01:14:22.789520+0000","last_fullsized":"2023-10-08T01:14:22.789520+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:42:48.572865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788635+0000","last_change":"2023-10-08T01:14:22.782436+0000","last_active":"2023-10-08T01:14:22.788635+0000","last_peered":"2023-10-08T01:14:22.788635+0000","last_clean":"2023-10-08T01:14:22.788635+0000","last_became_active":"2023-10-08T01:14:22.782125+0000","last_became_peered":"2023-10-08T01:14:22.782125+0000","last_unstale":"2023-10-08T01:14:22.788635+0000","last_undegraded":"2023-10-08T01:14:22.788635+0000","last_fullsized":"2023-10-08T01:14:22.788635+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:26.317877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787247+0000","last_change":"2023-10-08T01:14:22.779107+0000","last_active":"2023-10-08T01:14:22.787247+0000","last_peered":"2023-10-08T01:14:22.787247+0000","last_clean":"2023-10-08T01:14:22.787247+0000","last_became_active":"2023-10-08T01:14:22.778834+0000","last_became_peered":"2023-10-08T01:14:22.778834+0000","last_unstale":"2023-10-08T01:14:22.787247+0000","last_undegraded":"2023-10-08T01:14:22.787247+0000","last_fullsized":"2023-10-08T01:14:22.787247+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792306+0000","last_change":"2023-10-08T01:14:22.789825+0000","last_active":"2023-10-08T01:14:22.792306+0000","last_peered":"2023-10-08T01:14:22.792306+0000","last_clean":"2023-10-08T01:14:22.792306+0000","last_became_active":"2023-10-08T01:14:22.789560+0000","last_became_peered":"2023-10-08T01:14:22.789560+0000","last_unstale":"2023-10-08T01:14:22.792306+0000","last_undegraded":"2023-10-08T01:14:22.792306+0000","last_fullsized":"2023-10-08T01:14:22.792306+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792179+0000","last_change":"2023-10-08T01:14:22.788302+0000","last_active":"2023-10-08T01:14:22.792179+0000","last_peered":"2023-10-08T01:14:22.792179+0000","last_clean":"2023-10-08T01:14:22.792179+0000","last_became_active":"2023-10-08T01:14:22.787799+0000","last_became_peered":"2023-10-08T01:14:22.787799+0000","last_unstale":"2023-10-08T01:14:22.792179+0000","last_undegraded":"2023-10-08T01:14:22.792179+0000","last_fullsized":"2023-10-08T01:14:22.792179+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783436+0000","last_change":"2023-10-08T01:14:22.779249+0000","last_active":"2023-10-08T01:14:22.783436+0000","last_peered":"2023-10-08T01:14:22.783436+0000","last_clean":"2023-10-08T01:14:22.783436+0000","last_became_active":"2023-10-08T01:14:22.779000+0000","last_became_peered":"2023-10-08T01:14:22.779000+0000","last_unstale":"2023-10-08T01:14:22.783436+0000","last_undegraded":"2023-10-08T01:14:22.783436+0000","last_fullsized":"2023-10-08T01:14:22.783436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:44:52.899739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788599+0000","last_change":"2023-10-08T01:14:22.782182+0000","last_active":"2023-10-08T01:14:22.788599+0000","last_peered":"2023-10-08T01:14:22.788599+0000","last_clean":"2023-10-08T01:14:22.788599+0000","last_became_active":"2023-10-08T01:14:22.781939+0000","last_became_peered":"2023-10-08T01:14:22.781939+0000","last_unstale":"2023-10-08T01:14:22.788599+0000","last_undegraded":"2023-10-08T01:14:22.788599+0000","last_fullsized":"2023-10-08T01:14:22.788599+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:19.690095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787156+0000","last_change":"2023-10-08T01:14:22.780281+0000","last_active":"2023-10-08T01:14:22.787156+0000","last_peered":"2023-10-08T01:14:22.787156+0000","last_clean":"2023-10-08T01:14:22.787156+0000","last_became_active":"2023-10-08T01:14:22.780033+0000","last_became_peered":"2023-10-08T01:14:22.780033+0000","last_unstale":"2023-10-08T01:14:22.787156+0000","last_undegraded":"2023-10-08T01:14:22.787156+0000","last_fullsized":"2023-10-08T01:14:22.787156+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:41.656567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793213+0000","last_change":"2023-10-08T01:14:22.791550+0000","last_active":"2023-10-08T01:14:22.793213+0000","last_peered":"2023-10-08T01:14:22.793213+0000","last_clean":"2023-10-08T01:14:22.793213+0000","last_became_active":"2023-10-08T01:14:22.791030+0000","last_became_peered":"2023-10-08T01:14:22.791030+0000","last_unstale":"2023-10-08T01:14:22.793213+0000","last_undegraded":"2023-10-08T01:14:22.793213+0000","last_fullsized":"2023-10-08T01:14:22.793213+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:41.420761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791956+0000","last_change":"2023-10-08T01:14:22.786510+0000","last_active":"2023-10-08T01:14:22.791956+0000","last_peered":"2023-10-08T01:14:22.791956+0000","last_clean":"2023-10-08T01:14:22.791956+0000","last_became_active":"2023-10-08T01:14:22.785116+0000","last_became_peered":"2023-10-08T01:14:22.785116+0000","last_unstale":"2023-10-08T01:14:22.791956+0000","last_undegraded":"2023-10-08T01:14:22.791956+0000","last_fullsized":"2023-10-08T01:14:22.791956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799940+0000","last_change":"2023-10-08T01:14:22.799140+0000","last_active":"2023-10-08T01:14:22.799940+0000","last_peered":"2023-10-08T01:14:22.799940+0000","last_clean":"2023-10-08T01:14:22.799940+0000","last_became_active":"2023-10-08T01:14:22.798851+0000","last_became_peered":"2023-10-08T01:14:22.798851+0000","last_unstale":"2023-10-08T01:14:22.799940+0000","last_undegraded":"2023-10-08T01:14:22.799940+0000","last_fullsized":"2023-10-08T01:14:22.799940+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:30.232699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.775002+0000","last_change":"2023-10-08T01:14:22.770556+0000","last_active":"2023-10-08T01:14:22.775002+0000","last_peered":"2023-10-08T01:14:22.775002+0000","last_clean":"2023-10-08T01:14:22.775002+0000","last_became_active":"2023-10-08T01:14:22.770239+0000","last_became_peered":"2023-10-08T01:14:22.770239+0000","last_unstale":"2023-10-08T01:14:22.775002+0000","last_undegraded":"2023-10-08T01:14:22.775002+0000","last_fullsized":"2023-10-08T01:14:22.775002+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787921+0000","last_change":"2023-10-08T01:14:22.782733+0000","last_active":"2023-10-08T01:14:22.787921+0000","last_peered":"2023-10-08T01:14:22.787921+0000","last_clean":"2023-10-08T01:14:22.787921+0000","last_became_active":"2023-10-08T01:14:22.782427+0000","last_became_peered":"2023-10-08T01:14:22.782427+0000","last_unstale":"2023-10-08T01:14:22.787921+0000","last_undegraded":"2023-10-08T01:14:22.787921+0000","last_fullsized":"2023-10-08T01:14:22.787921+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789675+0000","last_change":"2023-10-08T01:14:22.788999+0000","last_active":"2023-10-08T01:14:22.789675+0000","last_peered":"2023-10-08T01:14:22.789675+0000","last_clean":"2023-10-08T01:14:22.789675+0000","last_became_active":"2023-10-08T01:14:22.787618+0000","last_became_peered":"2023-10-08T01:14:22.787618+0000","last_unstale":"2023-10-08T01:14:22.789675+0000","last_undegraded":"2023-10-08T01:14:22.789675+0000","last_fullsized":"2023-10-08T01:14:22.789675+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:17.572620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791070+0000","last_change":"2023-10-08T01:14:22.788072+0000","last_active":"2023-10-08T01:14:22.791070+0000","last_peered":"2023-10-08T01:14:22.791070+0000","last_clean":"2023-10-08T01:14:22.791070+0000","last_became_active":"2023-10-08T01:14:22.787677+0000","last_became_peered":"2023-10-08T01:14:22.787677+0000","last_unstale":"2023-10-08T01:14:22.791070+0000","last_undegraded":"2023-10-08T01:14:22.791070+0000","last_fullsized":"2023-10-08T01:14:22.791070+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:19:12.933240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788371+0000","last_change":"2023-10-08T01:14:22.783785+0000","last_active":"2023-10-08T01:14:22.788371+0000","last_peered":"2023-10-08T01:14:22.788371+0000","last_clean":"2023-10-08T01:14:22.788371+0000","last_became_active":"2023-10-08T01:14:22.783501+0000","last_became_peered":"2023-10-08T01:14:22.783501+0000","last_unstale":"2023-10-08T01:14:22.788371+0000","last_undegraded":"2023-10-08T01:14:22.788371+0000","last_fullsized":"2023-10-08T01:14:22.788371+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:51.777625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798256+0000","last_change":"2023-10-08T01:14:22.794783+0000","last_active":"2023-10-08T01:14:22.798256+0000","last_peered":"2023-10-08T01:14:22.798256+0000","last_clean":"2023-10-08T01:14:22.798256+0000","last_became_active":"2023-10-08T01:14:22.794304+0000","last_became_peered":"2023-10-08T01:14:22.794304+0000","last_unstale":"2023-10-08T01:14:22.798256+0000","last_undegraded":"2023-10-08T01:14:22.798256+0000","last_fullsized":"2023-10-08T01:14:22.798256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:59.697433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783465+0000","last_change":"2023-10-08T01:14:22.779446+0000","last_active":"2023-10-08T01:14:22.783465+0000","last_peered":"2023-10-08T01:14:22.783465+0000","last_clean":"2023-10-08T01:14:22.783465+0000","last_became_active":"2023-10-08T01:14:22.778675+0000","last_became_peered":"2023-10-08T01:14:22.778675+0000","last_unstale":"2023-10-08T01:14:22.783465+0000","last_undegraded":"2023-10-08T01:14:22.783465+0000","last_fullsized":"2023-10-08T01:14:22.783465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:34.335870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795814+0000","last_change":"2023-10-08T01:14:21.755008+0000","last_active":"2023-10-08T01:14:22.795814+0000","last_peered":"2023-10-08T01:14:22.795814+0000","last_clean":"2023-10-08T01:14:22.795814+0000","last_became_active":"2023-10-08T01:14:21.754712+0000","last_became_peered":"2023-10-08T01:14:21.754712+0000","last_unstale":"2023-10-08T01:14:22.795814+0000","last_undegraded":"2023-10-08T01:14:22.795814+0000","last_fullsized":"2023-10-08T01:14:22.795814+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:24.330581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788340+0000","last_change":"2023-10-08T01:14:22.782815+0000","last_active":"2023-10-08T01:14:22.788340+0000","last_peered":"2023-10-08T01:14:22.788340+0000","last_clean":"2023-10-08T01:14:22.788340+0000","last_became_active":"2023-10-08T01:14:22.782563+0000","last_became_peered":"2023-10-08T01:14:22.782563+0000","last_unstale":"2023-10-08T01:14:22.788340+0000","last_undegraded":"2023-10-08T01:14:22.788340+0000","last_fullsized":"2023-10-08T01:14:22.788340+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794676+0000","last_change":"2023-10-08T01:14:21.767509+0000","last_active":"2023-10-08T01:14:22.794676+0000","last_peered":"2023-10-08T01:14:22.794676+0000","last_clean":"2023-10-08T01:14:22.794676+0000","last_became_active":"2023-10-08T01:14:21.767219+0000","last_became_peered":"2023-10-08T01:14:21.767219+0000","last_unstale":"2023-10-08T01:14:22.794676+0000","last_undegraded":"2023-10-08T01:14:22.794676+0000","last_fullsized":"2023-10-08T01:14:22.794676+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799279+0000","last_change":"2023-10-08T01:14:22.798147+0000","last_active":"2023-10-08T01:14:22.799279+0000","last_peered":"2023-10-08T01:14:22.799279+0000","last_clean":"2023-10-08T01:14:22.799279+0000","last_became_active":"2023-10-08T01:14:22.797856+0000","last_became_peered":"2023-10-08T01:14:22.797856+0000","last_unstale":"2023-10-08T01:14:22.799279+0000","last_undegraded":"2023-10-08T01:14:22.799279+0000","last_fullsized":"2023-10-08T01:14:22.799279+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:32.952368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793353+0000","last_change":"2023-10-08T01:14:21.758907+0000","last_active":"2023-10-08T01:14:22.793353+0000","last_peered":"2023-10-08T01:14:22.793353+0000","last_clean":"2023-10-08T01:14:22.793353+0000","last_became_active":"2023-10-08T01:14:21.758669+0000","last_became_peered":"2023-10-08T01:14:21.758669+0000","last_unstale":"2023-10-08T01:14:22.793353+0000","last_undegraded":"2023-10-08T01:14:22.793353+0000","last_fullsized":"2023-10-08T01:14:22.793353+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:33.791532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782110+0000","last_change":"2023-10-08T01:14:22.777591+0000","last_active":"2023-10-08T01:14:22.782110+0000","last_peered":"2023-10-08T01:14:22.782110+0000","last_clean":"2023-10-08T01:14:22.782110+0000","last_became_active":"2023-10-08T01:14:22.777315+0000","last_became_peered":"2023-10-08T01:14:22.777315+0000","last_unstale":"2023-10-08T01:14:22.782110+0000","last_undegraded":"2023-10-08T01:14:22.782110+0000","last_fullsized":"2023-10-08T01:14:22.782110+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793240+0000","last_change":"2023-10-08T01:14:21.770043+0000","last_active":"2023-10-08T01:14:22.793240+0000","last_peered":"2023-10-08T01:14:22.793240+0000","last_clean":"2023-10-08T01:14:22.793240+0000","last_became_active":"2023-10-08T01:14:21.769799+0000","last_became_peered":"2023-10-08T01:14:21.769799+0000","last_unstale":"2023-10-08T01:14:22.793240+0000","last_undegraded":"2023-10-08T01:14:22.793240+0000","last_fullsized":"2023-10-08T01:14:22.793240+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:29.304154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.786431+0000","last_change":"2023-10-08T01:14:22.785288+0000","last_active":"2023-10-08T01:14:22.786431+0000","last_peered":"2023-10-08T01:14:22.786431+0000","last_clean":"2023-10-08T01:14:22.786431+0000","last_became_active":"2023-10-08T01:14:22.784973+0000","last_became_peered":"2023-10-08T01:14:22.784973+0000","last_unstale":"2023-10-08T01:14:22.786431+0000","last_undegraded":"2023-10-08T01:14:22.786431+0000","last_fullsized":"2023-10-08T01:14:22.786431+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:45.942767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.786306+0000","last_change":"2023-10-08T01:14:21.767177+0000","last_active":"2023-10-08T01:14:22.786306+0000","last_peered":"2023-10-08T01:14:22.786306+0000","last_clean":"2023-10-08T01:14:22.786306+0000","last_became_active":"2023-10-08T01:14:21.766891+0000","last_became_peered":"2023-10-08T01:14:21.766891+0000","last_unstale":"2023-10-08T01:14:22.786306+0000","last_undegraded":"2023-10-08T01:14:22.786306+0000","last_fullsized":"2023-10-08T01:14:22.786306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:48.667974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.777234+0000","last_change":"2023-10-08T01:14:22.772907+0000","last_active":"2023-10-08T01:14:22.777234+0000","last_peered":"2023-10-08T01:14:22.777234+0000","last_clean":"2023-10-08T01:14:22.777234+0000","last_became_active":"2023-10-08T01:14:22.772625+0000","last_became_peered":"2023-10-08T01:14:22.772625+0000","last_unstale":"2023-10-08T01:14:22.777234+0000","last_undegraded":"2023-10-08T01:14:22.777234+0000","last_fullsized":"2023-10-08T01:14:22.777234+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:13.379147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798893+0000","last_change":"2023-10-08T01:14:21.763020+0000","last_active":"2023-10-08T01:14:22.798893+0000","last_peered":"2023-10-08T01:14:22.798893+0000","last_clean":"2023-10-08T01:14:22.798893+0000","last_became_active":"2023-10-08T01:14:21.762782+0000","last_became_peered":"2023-10-08T01:14:21.762782+0000","last_unstale":"2023-10-08T01:14:22.798893+0000","last_undegraded":"2023-10-08T01:14:22.798893+0000","last_fullsized":"2023-10-08T01:14:22.798893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:59.904749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793777+0000","last_change":"2023-10-08T01:14:22.791043+0000","last_active":"2023-10-08T01:14:22.793777+0000","last_peered":"2023-10-08T01:14:22.793777+0000","last_clean":"2023-10-08T01:14:22.793777+0000","last_became_active":"2023-10-08T01:14:22.790796+0000","last_became_peered":"2023-10-08T01:14:22.790796+0000","last_unstale":"2023-10-08T01:14:22.793777+0000","last_undegraded":"2023-10-08T01:14:22.793777+0000","last_fullsized":"2023-10-08T01:14:22.793777+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.763917+0000","last_change":"2023-10-08T01:14:21.760681+0000","last_active":"2023-10-08T01:14:22.763917+0000","last_peered":"2023-10-08T01:14:22.763917+0000","last_clean":"2023-10-08T01:14:22.763917+0000","last_became_active":"2023-10-08T01:14:21.759392+0000","last_became_peered":"2023-10-08T01:14:21.759392+0000","last_unstale":"2023-10-08T01:14:22.763917+0000","last_undegraded":"2023-10-08T01:14:22.763917+0000","last_fullsized":"2023-10-08T01:14:22.763917+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:00.436555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789574+0000","last_change":"2023-10-08T01:14:22.783171+0000","last_active":"2023-10-08T01:14:22.789574+0000","last_peered":"2023-10-08T01:14:22.789574+0000","last_clean":"2023-10-08T01:14:22.789574+0000","last_became_active":"2023-10-08T01:14:22.782872+0000","last_became_peered":"2023-10-08T01:14:22.782872+0000","last_unstale":"2023-10-08T01:14:22.789574+0000","last_undegraded":"2023-10-08T01:14:22.789574+0000","last_fullsized":"2023-10-08T01:14:22.789574+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.780252+0000","last_change":"2023-10-08T01:14:21.766845+0000","last_active":"2023-10-08T01:14:22.780252+0000","last_peered":"2023-10-08T01:14:22.780252+0000","last_clean":"2023-10-08T01:14:22.780252+0000","last_became_active":"2023-10-08T01:14:21.765270+0000","last_became_peered":"2023-10-08T01:14:21.765270+0000","last_unstale":"2023-10-08T01:14:22.780252+0000","last_undegraded":"2023-10-08T01:14:22.780252+0000","last_fullsized":"2023-10-08T01:14:22.780252+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:33.330389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789845+0000","last_change":"2023-10-08T01:14:22.786181+0000","last_active":"2023-10-08T01:14:22.789845+0000","last_peered":"2023-10-08T01:14:22.789845+0000","last_clean":"2023-10-08T01:14:22.789845+0000","last_became_active":"2023-10-08T01:14:22.785912+0000","last_became_peered":"2023-10-08T01:14:22.785912+0000","last_unstale":"2023-10-08T01:14:22.789845+0000","last_undegraded":"2023-10-08T01:14:22.789845+0000","last_fullsized":"2023-10-08T01:14:22.789845+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:33.304186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.781565+0000","last_change":"2023-10-08T01:14:21.763195+0000","last_active":"2023-10-08T01:14:22.781565+0000","last_peered":"2023-10-08T01:14:22.781565+0000","last_clean":"2023-10-08T01:14:22.781565+0000","last_became_active":"2023-10-08T01:14:21.762953+0000","last_became_peered":"2023-10-08T01:14:21.762953+0000","last_unstale":"2023-10-08T01:14:22.781565+0000","last_undegraded":"2023-10-08T01:14:22.781565+0000","last_fullsized":"2023-10-08T01:14:22.781565+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:56.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789148+0000","last_change":"2023-10-08T01:14:22.782447+0000","last_active":"2023-10-08T01:14:22.789148+0000","last_peered":"2023-10-08T01:14:22.789148+0000","last_clean":"2023-10-08T01:14:22.789148+0000","last_became_active":"2023-10-08T01:14:22.782195+0000","last_became_peered":"2023-10-08T01:14:22.782195+0000","last_unstale":"2023-10-08T01:14:22.789148+0000","last_undegraded":"2023-10-08T01:14:22.789148+0000","last_fullsized":"2023-10-08T01:14:22.789148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.775594+0000","last_change":"2023-10-08T01:14:21.756007+0000","last_active":"2023-10-08T01:14:22.775594+0000","last_peered":"2023-10-08T01:14:22.775594+0000","last_clean":"2023-10-08T01:14:22.775594+0000","last_became_active":"2023-10-08T01:14:21.755725+0000","last_became_peered":"2023-10-08T01:14:21.755725+0000","last_unstale":"2023-10-08T01:14:22.775594+0000","last_undegraded":"2023-10-08T01:14:22.775594+0000","last_fullsized":"2023-10-08T01:14:22.775594+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792488+0000","last_change":"2023-10-08T01:14:22.787821+0000","last_active":"2023-10-08T01:14:22.792488+0000","last_peered":"2023-10-08T01:14:22.792488+0000","last_clean":"2023-10-08T01:14:22.792488+0000","last_became_active":"2023-10-08T01:14:22.784125+0000","last_became_peered":"2023-10-08T01:14:22.784125+0000","last_unstale":"2023-10-08T01:14:22.792488+0000","last_undegraded":"2023-10-08T01:14:22.792488+0000","last_fullsized":"2023-10-08T01:14:22.792488+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"119'3","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792752+0000","last_change":"2023-10-08T01:14:21.768343+0000","last_active":"2023-10-08T01:14:22.792752+0000","last_peered":"2023-10-08T01:14:22.792752+0000","last_clean":"2023-10-08T01:14:22.792752+0000","last_became_active":"2023-10-08T01:14:21.767996+0000","last_became_peered":"2023-10-08T01:14:21.767996+0000","last_unstale":"2023-10-08T01:14:22.792752+0000","last_undegraded":"2023-10-08T01:14:22.792752+0000","last_fullsized":"2023-10-08T01:14:22.792752+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:44.937285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.777118+0000","last_change":"2023-10-08T01:14:22.773623+0000","last_active":"2023-10-08T01:14:22.777118+0000","last_peered":"2023-10-08T01:14:22.777118+0000","last_clean":"2023-10-08T01:14:22.777118+0000","last_became_active":"2023-10-08T01:14:22.773338+0000","last_became_peered":"2023-10-08T01:14:22.773338+0000","last_unstale":"2023-10-08T01:14:22.777118+0000","last_undegraded":"2023-10-08T01:14:22.777118+0000","last_fullsized":"2023-10-08T01:14:22.777118+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:58:23.838250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.800832+0000","last_change":"2023-10-08T01:14:21.764918+0000","last_active":"2023-10-08T01:14:22.800832+0000","last_peered":"2023-10-08T01:14:22.800832+0000","last_clean":"2023-10-08T01:14:22.800832+0000","last_became_active":"2023-10-08T01:14:21.763653+0000","last_became_peered":"2023-10-08T01:14:21.763653+0000","last_unstale":"2023-10-08T01:14:22.800832+0000","last_undegraded":"2023-10-08T01:14:22.800832+0000","last_fullsized":"2023-10-08T01:14:22.800832+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787341+0000","last_change":"2023-10-08T01:14:22.778889+0000","last_active":"2023-10-08T01:14:22.787341+0000","last_peered":"2023-10-08T01:14:22.787341+0000","last_clean":"2023-10-08T01:14:22.787341+0000","last_became_active":"2023-10-08T01:14:22.778592+0000","last_became_peered":"2023-10-08T01:14:22.778592+0000","last_unstale":"2023-10-08T01:14:22.787341+0000","last_undegraded":"2023-10-08T01:14:22.787341+0000","last_fullsized":"2023-10-08T01:14:22.787341+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793156+0000","last_change":"2023-10-08T01:14:21.769762+0000","last_active":"2023-10-08T01:14:22.793156+0000","last_peered":"2023-10-08T01:14:22.793156+0000","last_clean":"2023-10-08T01:14:22.793156+0000","last_became_active":"2023-10-08T01:14:21.769522+0000","last_became_peered":"2023-10-08T01:14:21.769522+0000","last_unstale":"2023-10-08T01:14:22.793156+0000","last_undegraded":"2023-10-08T01:14:22.793156+0000","last_fullsized":"2023-10-08T01:14:22.793156+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:27.726884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797374+0000","last_change":"2023-10-08T01:14:22.791650+0000","last_active":"2023-10-08T01:14:22.797374+0000","last_peered":"2023-10-08T01:14:22.797374+0000","last_clean":"2023-10-08T01:14:22.797374+0000","last_became_active":"2023-10-08T01:14:22.791109+0000","last_became_peered":"2023-10-08T01:14:22.791109+0000","last_unstale":"2023-10-08T01:14:22.797374+0000","last_undegraded":"2023-10-08T01:14:22.797374+0000","last_fullsized":"2023-10-08T01:14:22.797374+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765608+0000","last_change":"2023-10-08T01:14:21.768459+0000","last_active":"2023-10-08T01:14:22.765608+0000","last_peered":"2023-10-08T01:14:22.765608+0000","last_clean":"2023-10-08T01:14:22.765608+0000","last_became_active":"2023-10-08T01:14:21.768189+0000","last_became_peered":"2023-10-08T01:14:21.768189+0000","last_unstale":"2023-10-08T01:14:22.765608+0000","last_undegraded":"2023-10-08T01:14:22.765608+0000","last_fullsized":"2023-10-08T01:14:22.765608+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:54.324955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789515+0000","last_change":"2023-10-08T01:14:22.781160+0000","last_active":"2023-10-08T01:14:22.789515+0000","last_peered":"2023-10-08T01:14:22.789515+0000","last_clean":"2023-10-08T01:14:22.789515+0000","last_became_active":"2023-10-08T01:14:22.780893+0000","last_became_peered":"2023-10-08T01:14:22.780893+0000","last_unstale":"2023-10-08T01:14:22.789515+0000","last_undegraded":"2023-10-08T01:14:22.789515+0000","last_fullsized":"2023-10-08T01:14:22.789515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:30.309636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768642+0000","last_change":"2023-10-08T01:14:21.760294+0000","last_active":"2023-10-08T01:14:22.768642+0000","last_peered":"2023-10-08T01:14:22.768642+0000","last_clean":"2023-10-08T01:14:22.768642+0000","last_became_active":"2023-10-08T01:14:21.758928+0000","last_became_peered":"2023-10-08T01:14:21.758928+0000","last_unstale":"2023-10-08T01:14:22.768642+0000","last_undegraded":"2023-10-08T01:14:22.768642+0000","last_fullsized":"2023-10-08T01:14:22.768642+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788009+0000","last_change":"2023-10-08T01:14:22.784690+0000","last_active":"2023-10-08T01:14:22.788009+0000","last_peered":"2023-10-08T01:14:22.788009+0000","last_clean":"2023-10-08T01:14:22.788009+0000","last_became_active":"2023-10-08T01:14:22.784408+0000","last_became_peered":"2023-10-08T01:14:22.784408+0000","last_unstale":"2023-10-08T01:14:22.788009+0000","last_undegraded":"2023-10-08T01:14:22.788009+0000","last_fullsized":"2023-10-08T01:14:22.788009+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:55.180105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795020+0000","last_change":"2023-10-08T01:14:21.765533+0000","last_active":"2023-10-08T01:14:22.795020+0000","last_peered":"2023-10-08T01:14:22.795020+0000","last_clean":"2023-10-08T01:14:22.795020+0000","last_became_active":"2023-10-08T01:14:21.764914+0000","last_became_peered":"2023-10-08T01:14:21.764914+0000","last_unstale":"2023-10-08T01:14:22.795020+0000","last_undegraded":"2023-10-08T01:14:22.795020+0000","last_fullsized":"2023-10-08T01:14:22.795020+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:00.128756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791772+0000","last_change":"2023-10-08T01:14:22.784334+0000","last_active":"2023-10-08T01:14:22.791772+0000","last_peered":"2023-10-08T01:14:22.791772+0000","last_clean":"2023-10-08T01:14:22.791772+0000","last_became_active":"2023-10-08T01:14:22.784048+0000","last_became_peered":"2023-10-08T01:14:22.784048+0000","last_unstale":"2023-10-08T01:14:22.791772+0000","last_undegraded":"2023-10-08T01:14:22.791772+0000","last_fullsized":"2023-10-08T01:14:22.791772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797139+0000","last_change":"2023-10-08T01:14:21.767123+0000","last_active":"2023-10-08T01:14:22.797139+0000","last_peered":"2023-10-08T01:14:22.797139+0000","last_clean":"2023-10-08T01:14:22.797139+0000","last_became_active":"2023-10-08T01:14:21.766826+0000","last_became_peered":"2023-10-08T01:14:21.766826+0000","last_unstale":"2023-10-08T01:14:22.797139+0000","last_undegraded":"2023-10-08T01:14:22.797139+0000","last_fullsized":"2023-10-08T01:14:22.797139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.779071+0000","last_change":"2023-10-08T01:14:22.775187+0000","last_active":"2023-10-08T01:14:22.779071+0000","last_peered":"2023-10-08T01:14:22.779071+0000","last_clean":"2023-10-08T01:14:22.779071+0000","last_became_active":"2023-10-08T01:14:22.774922+0000","last_became_peered":"2023-10-08T01:14:22.774922+0000","last_unstale":"2023-10-08T01:14:22.779071+0000","last_undegraded":"2023-10-08T01:14:22.779071+0000","last_fullsized":"2023-10-08T01:14:22.779071+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:36.091585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793251+0000","last_change":"2023-10-08T01:14:21.761079+0000","last_active":"2023-10-08T01:14:22.793251+0000","last_peered":"2023-10-08T01:14:22.793251+0000","last_clean":"2023-10-08T01:14:22.793251+0000","last_became_active":"2023-10-08T01:14:21.760705+0000","last_became_peered":"2023-10-08T01:14:21.760705+0000","last_unstale":"2023-10-08T01:14:22.793251+0000","last_undegraded":"2023-10-08T01:14:22.793251+0000","last_fullsized":"2023-10-08T01:14:22.793251+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788803+0000","last_change":"2023-10-08T01:14:22.785209+0000","last_active":"2023-10-08T01:14:22.788803+0000","last_peered":"2023-10-08T01:14:22.788803+0000","last_clean":"2023-10-08T01:14:22.788803+0000","last_became_active":"2023-10-08T01:14:22.784735+0000","last_became_peered":"2023-10-08T01:14:22.784735+0000","last_unstale":"2023-10-08T01:14:22.788803+0000","last_undegraded":"2023-10-08T01:14:22.788803+0000","last_fullsized":"2023-10-08T01:14:22.788803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.767122+0000","last_change":"2023-10-08T01:14:21.769975+0000","last_active":"2023-10-08T01:14:22.767122+0000","last_peered":"2023-10-08T01:14:22.767122+0000","last_clean":"2023-10-08T01:14:22.767122+0000","last_became_active":"2023-10-08T01:14:21.769477+0000","last_became_peered":"2023-10-08T01:14:21.769477+0000","last_unstale":"2023-10-08T01:14:22.767122+0000","last_undegraded":"2023-10-08T01:14:22.767122+0000","last_fullsized":"2023-10-08T01:14:22.767122+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:34.177414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787894+0000","last_change":"2023-10-08T01:14:22.781041+0000","last_active":"2023-10-08T01:14:22.787894+0000","last_peered":"2023-10-08T01:14:22.787894+0000","last_clean":"2023-10-08T01:14:22.787894+0000","last_became_active":"2023-10-08T01:14:22.780765+0000","last_became_peered":"2023-10-08T01:14:22.780765+0000","last_unstale":"2023-10-08T01:14:22.787894+0000","last_undegraded":"2023-10-08T01:14:22.787894+0000","last_fullsized":"2023-10-08T01:14:22.787894+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:00.278431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792713+0000","last_change":"2023-10-08T01:14:21.768612+0000","last_active":"2023-10-08T01:14:22.792713+0000","last_peered":"2023-10-08T01:14:22.792713+0000","last_clean":"2023-10-08T01:14:22.792713+0000","last_became_active":"2023-10-08T01:14:21.767991+0000","last_became_peered":"2023-10-08T01:14:21.767991+0000","last_unstale":"2023-10-08T01:14:22.792713+0000","last_undegraded":"2023-10-08T01:14:22.792713+0000","last_fullsized":"2023-10-08T01:14:22.792713+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:50.772143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788094+0000","last_change":"2023-10-08T01:14:22.783198+0000","last_active":"2023-10-08T01:14:22.788094+0000","last_peered":"2023-10-08T01:14:22.788094+0000","last_clean":"2023-10-08T01:14:22.788094+0000","last_became_active":"2023-10-08T01:14:22.782950+0000","last_became_peered":"2023-10-08T01:14:22.782950+0000","last_unstale":"2023-10-08T01:14:22.788094+0000","last_undegraded":"2023-10-08T01:14:22.788094+0000","last_fullsized":"2023-10-08T01:14:22.788094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:30:30.559893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.767998+0000","last_change":"2023-10-08T01:14:21.762370+0000","last_active":"2023-10-08T01:14:22.767998+0000","last_peered":"2023-10-08T01:14:22.767998+0000","last_clean":"2023-10-08T01:14:22.767998+0000","last_became_active":"2023-10-08T01:14:21.761985+0000","last_became_peered":"2023-10-08T01:14:21.761985+0000","last_unstale":"2023-10-08T01:14:22.767998+0000","last_undegraded":"2023-10-08T01:14:22.767998+0000","last_fullsized":"2023-10-08T01:14:22.767998+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:12.383080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788041+0000","last_change":"2023-10-08T01:14:22.782578+0000","last_active":"2023-10-08T01:14:22.788041+0000","last_peered":"2023-10-08T01:14:22.788041+0000","last_clean":"2023-10-08T01:14:22.788041+0000","last_became_active":"2023-10-08T01:14:22.781703+0000","last_became_peered":"2023-10-08T01:14:22.781703+0000","last_unstale":"2023-10-08T01:14:22.788041+0000","last_undegraded":"2023-10-08T01:14:22.788041+0000","last_fullsized":"2023-10-08T01:14:22.788041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:38.983859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768007+0000","last_change":"2023-10-08T01:14:21.753473+0000","last_active":"2023-10-08T01:14:22.768007+0000","last_peered":"2023-10-08T01:14:22.768007+0000","last_clean":"2023-10-08T01:14:22.768007+0000","last_became_active":"2023-10-08T01:14:21.753194+0000","last_became_peered":"2023-10-08T01:14:21.753194+0000","last_unstale":"2023-10-08T01:14:22.768007+0000","last_undegraded":"2023-10-08T01:14:22.768007+0000","last_fullsized":"2023-10-08T01:14:22.768007+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:14:53.171270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792546+0000","last_change":"2023-10-08T01:14:22.788676+0000","last_active":"2023-10-08T01:14:22.792546+0000","last_peered":"2023-10-08T01:14:22.792546+0000","last_clean":"2023-10-08T01:14:22.792546+0000","last_became_active":"2023-10-08T01:14:22.788378+0000","last_became_peered":"2023-10-08T01:14:22.788378+0000","last_unstale":"2023-10-08T01:14:22.792546+0000","last_undegraded":"2023-10-08T01:14:22.792546+0000","last_fullsized":"2023-10-08T01:14:22.792546+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:51.777625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.763121+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.762883+0000","last_became_peered":"2023-10-08T01:14:21.762883+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:03.018597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789807+0000","last_change":"2023-10-08T01:14:22.782302+0000","last_active":"2023-10-08T01:14:22.789807+0000","last_peered":"2023-10-08T01:14:22.789807+0000","last_clean":"2023-10-08T01:14:22.789807+0000","last_became_active":"2023-10-08T01:14:22.780497+0000","last_became_peered":"2023-10-08T01:14:22.780497+0000","last_unstale":"2023-10-08T01:14:22.789807+0000","last_undegraded":"2023-10-08T01:14:22.789807+0000","last_fullsized":"2023-10-08T01:14:22.789807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:59.391189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.790321+0000","last_change":"2023-10-08T01:14:21.760699+0000","last_active":"2023-10-08T01:14:22.790321+0000","last_peered":"2023-10-08T01:14:22.790321+0000","last_clean":"2023-10-08T01:14:22.790321+0000","last_became_active":"2023-10-08T01:14:21.760424+0000","last_became_peered":"2023-10-08T01:14:21.760424+0000","last_unstale":"2023-10-08T01:14:22.790321+0000","last_undegraded":"2023-10-08T01:14:22.790321+0000","last_fullsized":"2023-10-08T01:14:22.790321+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:59.004815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798148+0000","last_change":"2023-10-08T01:14:22.793832+0000","last_active":"2023-10-08T01:14:22.798148+0000","last_peered":"2023-10-08T01:14:22.798148+0000","last_clean":"2023-10-08T01:14:22.798148+0000","last_became_active":"2023-10-08T01:14:22.792633+0000","last_became_peered":"2023-10-08T01:14:22.792633+0000","last_unstale":"2023-10-08T01:14:22.798148+0000","last_undegraded":"2023-10-08T01:14:22.798148+0000","last_fullsized":"2023-10-08T01:14:22.798148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:11.281859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768046+0000","last_change":"2023-10-08T01:14:21.763932+0000","last_active":"2023-10-08T01:14:22.768046+0000","last_peered":"2023-10-08T01:14:22.768046+0000","last_clean":"2023-10-08T01:14:22.768046+0000","last_became_active":"2023-10-08T01:14:21.763673+0000","last_became_peered":"2023-10-08T01:14:21.763673+0000","last_unstale":"2023-10-08T01:14:22.768046+0000","last_undegraded":"2023-10-08T01:14:22.768046+0000","last_fullsized":"2023-10-08T01:14:22.768046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:07.616557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797649+0000","last_change":"2023-10-08T01:14:22.791844+0000","last_active":"2023-10-08T01:14:22.797649+0000","last_peered":"2023-10-08T01:14:22.797649+0000","last_clean":"2023-10-08T01:14:22.797649+0000","last_became_active":"2023-10-08T01:14:22.791583+0000","last_became_peered":"2023-10-08T01:14:22.791583+0000","last_unstale":"2023-10-08T01:14:22.797649+0000","last_undegraded":"2023-10-08T01:14:22.797649+0000","last_fullsized":"2023-10-08T01:14:22.797649+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.770984+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.770705+0000","last_became_peered":"2023-10-08T01:14:21.770705+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:48:20.485203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782896+0000","last_change":"2023-10-08T01:14:22.781159+0000","last_active":"2023-10-08T01:14:22.782896+0000","last_peered":"2023-10-08T01:14:22.782896+0000","last_clean":"2023-10-08T01:14:22.782896+0000","last_became_active":"2023-10-08T01:14:22.780879+0000","last_became_peered":"2023-10-08T01:14:22.780879+0000","last_unstale":"2023-10-08T01:14:22.782896+0000","last_undegraded":"2023-10-08T01:14:22.782896+0000","last_fullsized":"2023-10-08T01:14:22.782896+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.764576+0000","last_change":"2023-10-08T01:14:21.760347+0000","last_active":"2023-10-08T01:14:22.764576+0000","last_peered":"2023-10-08T01:14:22.764576+0000","last_clean":"2023-10-08T01:14:22.764576+0000","last_became_active":"2023-10-08T01:14:21.759959+0000","last_became_peered":"2023-10-08T01:14:21.759959+0000","last_unstale":"2023-10-08T01:14:22.764576+0000","last_undegraded":"2023-10-08T01:14:22.764576+0000","last_fullsized":"2023-10-08T01:14:22.764576+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:14:53.171270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788178+0000","last_change":"2023-10-08T01:14:22.784755+0000","last_active":"2023-10-08T01:14:22.788178+0000","last_peered":"2023-10-08T01:14:22.788178+0000","last_clean":"2023-10-08T01:14:22.788178+0000","last_became_active":"2023-10-08T01:14:22.784506+0000","last_became_peered":"2023-10-08T01:14:22.784506+0000","last_unstale":"2023-10-08T01:14:22.788178+0000","last_undegraded":"2023-10-08T01:14:22.788178+0000","last_fullsized":"2023-10-08T01:14:22.788178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:12.608356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766909+0000","last_change":"2023-10-08T01:14:21.754925+0000","last_active":"2023-10-08T01:14:22.766909+0000","last_peered":"2023-10-08T01:14:22.766909+0000","last_clean":"2023-10-08T01:14:22.766909+0000","last_became_active":"2023-10-08T01:14:21.754606+0000","last_became_peered":"2023-10-08T01:14:21.754606+0000","last_unstale":"2023-10-08T01:14:22.766909+0000","last_undegraded":"2023-10-08T01:14:22.766909+0000","last_fullsized":"2023-10-08T01:14:22.766909+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:07.890275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.790593+0000","last_change":"2023-10-08T01:14:22.787439+0000","last_active":"2023-10-08T01:14:22.790593+0000","last_peered":"2023-10-08T01:14:22.790593+0000","last_clean":"2023-10-08T01:14:22.790593+0000","last_became_active":"2023-10-08T01:14:22.787105+0000","last_became_peered":"2023-10-08T01:14:22.787105+0000","last_unstale":"2023-10-08T01:14:22.790593+0000","last_undegraded":"2023-10-08T01:14:22.790593+0000","last_fullsized":"2023-10-08T01:14:22.790593+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.762908+0000","last_change":"2023-10-08T01:14:21.767557+0000","last_active":"2023-10-08T01:14:22.762908+0000","last_peered":"2023-10-08T01:14:22.762908+0000","last_clean":"2023-10-08T01:14:22.762908+0000","last_became_active":"2023-10-08T01:14:21.767293+0000","last_became_peered":"2023-10-08T01:14:21.767293+0000","last_unstale":"2023-10-08T01:14:22.762908+0000","last_undegraded":"2023-10-08T01:14:22.762908+0000","last_fullsized":"2023-10-08T01:14:22.762908+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:12.383080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789364+0000","last_change":"2023-10-08T01:14:22.785451+0000","last_active":"2023-10-08T01:14:22.789364+0000","last_peered":"2023-10-08T01:14:22.789364+0000","last_clean":"2023-10-08T01:14:22.789364+0000","last_became_active":"2023-10-08T01:14:22.785188+0000","last_became_peered":"2023-10-08T01:14:22.785188+0000","last_unstale":"2023-10-08T01:14:22.789364+0000","last_undegraded":"2023-10-08T01:14:22.789364+0000","last_fullsized":"2023-10-08T01:14:22.789364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.773008+0000","last_change":"2023-10-08T01:14:21.764982+0000","last_active":"2023-10-08T01:14:22.773008+0000","last_peered":"2023-10-08T01:14:22.773008+0000","last_clean":"2023-10-08T01:14:22.773008+0000","last_became_active":"2023-10-08T01:14:21.764710+0000","last_became_peered":"2023-10-08T01:14:21.764710+0000","last_unstale":"2023-10-08T01:14:22.773008+0000","last_undegraded":"2023-10-08T01:14:22.773008+0000","last_fullsized":"2023-10-08T01:14:22.773008+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782858+0000","last_change":"2023-10-08T01:14:22.780210+0000","last_active":"2023-10-08T01:14:22.782858+0000","last_peered":"2023-10-08T01:14:22.782858+0000","last_clean":"2023-10-08T01:14:22.782858+0000","last_became_active":"2023-10-08T01:14:22.779948+0000","last_became_peered":"2023-10-08T01:14:22.779948+0000","last_unstale":"2023-10-08T01:14:22.782858+0000","last_undegraded":"2023-10-08T01:14:22.782858+0000","last_fullsized":"2023-10-08T01:14:22.782858+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:35.182896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766989+0000","last_change":"2023-10-08T01:14:21.770863+0000","last_active":"2023-10-08T01:14:22.766989+0000","last_peered":"2023-10-08T01:14:22.766989+0000","last_clean":"2023-10-08T01:14:22.766989+0000","last_became_active":"2023-10-08T01:14:21.770532+0000","last_became_peered":"2023-10-08T01:14:21.770532+0000","last_unstale":"2023-10-08T01:14:22.766989+0000","last_undegraded":"2023-10-08T01:14:22.766989+0000","last_fullsized":"2023-10-08T01:14:22.766989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:32.319886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787982+0000","last_change":"2023-10-08T01:14:22.781044+0000","last_active":"2023-10-08T01:14:22.787982+0000","last_peered":"2023-10-08T01:14:22.787982+0000","last_clean":"2023-10-08T01:14:22.787982+0000","last_became_active":"2023-10-08T01:14:22.780772+0000","last_became_peered":"2023-10-08T01:14:22.780772+0000","last_unstale":"2023-10-08T01:14:22.787982+0000","last_undegraded":"2023-10-08T01:14:22.787982+0000","last_fullsized":"2023-10-08T01:14:22.787982+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794163+0000","last_change":"2023-10-08T01:14:21.765701+0000","last_active":"2023-10-08T01:14:22.794163+0000","last_peered":"2023-10-08T01:14:22.794163+0000","last_clean":"2023-10-08T01:14:22.794163+0000","last_became_active":"2023-10-08T01:14:21.765371+0000","last_became_peered":"2023-10-08T01:14:21.765371+0000","last_unstale":"2023-10-08T01:14:22.794163+0000","last_undegraded":"2023-10-08T01:14:22.794163+0000","last_fullsized":"2023-10-08T01:14:22.794163+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:35.086104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"109'227","reported_seq":637,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765396+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:14:22.765396+0000","last_peered":"2023-10-08T01:14:22.765396+0000","last_clean":"2023-10-08T01:14:22.765396+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:14:22.765396+0000","last_undegraded":"2023-10-08T01:14:22.765396+0000","last_fullsized":"2023-10-08T01:14:22.765396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575070,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307516,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064324,"statfs":{"total":96636764160,"available":96321867776,"internally_reserved":0,"allocated":2084864,"data_stored":1315796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":1.123,"15min":1.123},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.97899999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96999999999999997,"5min":1.133,"15min":1.133},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.72299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":1.167,"15min":1.167},"last":0.53200000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67100000000000004,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.91700000000000004,"5min":1.079,"15min":1.079},"last":0.59699999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.58299999999999996},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.88,"15min":0.88300000000000001},"last":0.40799999999999997}]},{"osd":5,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":1.004,"15min":1.004},"last":0.56999999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72599999999999998,"15min":0.75},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302840,"kb_used_data":1456,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069000,"statfs":{"total":96636764160,"available":96326656000,"internally_reserved":0,"allocated":1490944,"data_stored":725940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68300000000000005,"15min":0.69999999999999996},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.625},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.69499999999999995},"min":{"1min":0.54500000000000004,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.875}]},{"osd":1,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.48399999999999999,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.984},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.59099999999999997,"5min":0.45300000000000001,"15min":0.38100000000000001},"max":{"1min":0.97099999999999997,"5min":1.018,"15min":3.0190000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.64100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71699999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71999999999999997,"15min":0.745},"min":{"1min":0.55400000000000005,"5min":0.47599999999999998,"15min":0.47099999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.94399999999999995,"5min":1.048,"15min":1.048},"last":0.64200000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":1.087},"last":0.79000000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":0.84999999999999998,"5min":1.0189999999999999,"15min":1.395},"last":0.75},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94999999999999996,"5min":1.0489999999999999,"15min":1.21},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":1.095,"15min":2.9489999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.80400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":725316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.46700000000000003},"min":{"1min":0.32600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47799999999999998,"15min":0.503},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.745,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.377,"5min":0.317,"15min":0.26700000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.77300000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.53500000000000003},"min":{"1min":0.45100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.82099999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.53700000000000003},"min":{"1min":0.42399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":1.296,"5min":1.349,"15min":1.452},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.76200000000000001},"min":{"1min":0.55700000000000005,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.137,"5min":1.1499999999999999,"15min":1.1839999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73199999999999998,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":1.119,"5min":1.274,"15min":1.3959999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.5209999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73799999999999999,"15min":0.78500000000000003},"min":{"1min":0.56799999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.069,"5min":1.3839999999999999,"15min":1.6020000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74199999999999999,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1499999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.621}]},{"osd":7,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.82099999999999995},"min":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.236,"5min":1.3109999999999999,"15min":1.621},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78500000000000003,"15min":0.82899999999999996},"min":{"1min":0.66000000000000003,"5min":0.58099999999999996,"15min":0.51300000000000001},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.4890000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":1316238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.625,"5min":0.95999999999999996,"15min":0.997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67700000000000005,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46800000000000003,"15min":0.48299999999999998},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.53000000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.70599999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.59899999999999998},"min":{"1min":0.504,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.754},"min":{"1min":0.53500000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94099999999999995,"5min":1.341,"15min":1.617},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.61299999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92100000000000004,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99199999999999999,"5min":2.085,"15min":2.085},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79100000000000004,"15min":0.81699999999999995},"min":{"1min":0.56699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.381,"15min":1.5289999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77100000000000002,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98299999999999998,"5min":1.361,"15min":1.3779999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99399999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77900000000000003,"15min":0.81299999999999994},"min":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.077,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.871},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81399999999999995,"15min":0.83899999999999997},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.028,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.89400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311028,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060812,"statfs":{"total":96636764160,"available":96318271488,"internally_reserved":0,"allocated":1486848,"data_stored":725350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.41099999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.46000000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.88800000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.90400000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306944,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064896,"statfs":{"total":96636764160,"available":96322453504,"internally_reserved":0,"allocated":1499136,"data_stored":726874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.46200000000000002},"min":{"1min":0.32000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54400000000000004,"5min":0.627,"15min":0.80100000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":0.81200000000000006},"last":0.50900000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.28299999999999997},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.371},{"interface":"front","average":{"1min":0.504,"5min":0.495,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.311,"15min":0.31},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.39100000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.89700000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.5,"5min":0.438,"15min":0.39600000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.752},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.41099999999999998},"max":{"1min":1.028,"5min":1.095,"15min":1.3029999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.046,"5min":1.0580000000000001,"15min":1.413},"last":0.74299999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.77300000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51000000000000001},"max":{"1min":0.995,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.67500000000000004,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.089},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.57199999999999995,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.94799999999999995,"5min":1.101,"15min":1.379},"last":0.877},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72899999999999998,"15min":0.77900000000000003},"min":{"1min":0.59499999999999997,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.486},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.755,"15min":0.79300000000000004},"min":{"1min":0.63200000000000001,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97799999999999998,"5min":1.0840000000000001,"15min":1.389},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.79500000000000004},"min":{"1min":0.63300000000000001,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.96699999999999997,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.78300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311044,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060796,"statfs":{"total":96636764160,"available":96318255104,"internally_reserved":0,"allocated":1503232,"data_stored":726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.84299999999999997}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.875}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.70999999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.92400000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.80100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":725386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.629},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.71699999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":1.026,"15min":1.026},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.53300000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.38700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64800000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71999999999999997,"15min":0.73299999999999998},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.98699999999999999,"5min":1.034,"15min":1.034},"last":0.56100000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.874,"15min":0.91900000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.90300000000000002,"15min":1.0680000000000001},"last":0.753}]},{"osd":6,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.90200000000000002,"15min":0.95699999999999996},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":0.94399999999999995,"15min":1.095},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47399999999999998},"max":{"1min":0.90200000000000002,"5min":1.05,"15min":1.05},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:14:31.932 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:14:32.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.123+0000 7efcfa5ff700 1 Processor -- start 2023-10-08T01:14:32.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.123+0000 7efcfa5ff700 1 -- start start 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcfa5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf4155c10 0x7efcf4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcfa5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcf40ac230 0x7efcf40ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcfa5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efcf41568a0 0x7efcf4156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcfa5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf40ad930 con 0x7efcf40ac230 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcfa5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcf40acd20 con 0x7efcf41568a0 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcfa5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40ad000 con 0x7efcf4155c10 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcebfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcf40ac230 0x7efcf40ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcebfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcf40ac230 0x7efcf40ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40744/0 (socket says 172.21.15.119:40744) 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcebfff700 1 -- 172.21.15.119:0/4185837534 learned_addr learned my addr 172.21.15.119:0/4185837534 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.125+0000 7efcf3fff700 1 --2- 172.21.15.119:0/4185837534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf4155c10 0x7efcf4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcf8b9c700 1 --2- 172.21.15.119:0/4185837534 >> v2:172.21.15.184:3300/0 conn(0x7efcf41568a0 0x7efcf4156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcebfff700 1 -- 172.21.15.119:0/4185837534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf4155c10 msgr2=0x7efcf4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcebfff700 1 --2- 172.21.15.119:0/4185837534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf4155c10 0x7efcf4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcebfff700 1 -- 172.21.15.119:0/4185837534 >> v2:172.21.15.184:3300/0 conn(0x7efcf41568a0 msgr2=0x7efcf4156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcebfff700 1 --2- 172.21.15.119:0/4185837534 >> v2:172.21.15.184:3300/0 conn(0x7efcf41568a0 0x7efcf4156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcebfff700 1 -- 172.21.15.119:0/4185837534 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf40ad2e0 con 0x7efcf40ac230 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcebfff700 1 --2- 172.21.15.119:0/4185837534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40ac230 0x7efcf40ac6a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7efcf40c8c60 tx=0x7efcdc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09398f2b1f1a39f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcf37fe700 1 -- 172.21.15.119:0/4185837534 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00dc50 con 0x7efcf40ac230 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcf37fe700 1 -- 172.21.15.119:0/4185837534 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc00ddb0 con 0x7efcf40ac230 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.126+0000 7efcf37fe700 1 -- 172.21.15.119:0/4185837534 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc0196d0 con 0x7efcf40ac230 2023-10-08T01:14:32.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 -- 172.21.15.119:0/4185837534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40ac230 msgr2=0x7efcf40ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/4185837534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40ac230 0x7efcf40ac6a0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7efcf40c8c60 tx=0x7efcdc00b4f0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 -- 172.21.15.119:0/4185837534 shutdown_connections 2023-10-08T01:14:32.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/4185837534 >> v2:172.21.15.184:3300/0 conn(0x7efcf41568a0 0x7efcf4156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/4185837534 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf4155c10 0x7efcf4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/4185837534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf40ac230 0x7efcf40ac6a0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 -- 172.21.15.119:0/4185837534 >> 172.21.15.119:0/4185837534 conn(0x7efcf406a320 msgr2=0x7efcf40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:32.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 -- 172.21.15.119:0/4185837534 shutdown_connections 2023-10-08T01:14:32.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.127+0000 7efcfa5ff700 1 -- 172.21.15.119:0/4185837534 wait complete. 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.128+0000 7efcfa5ff700 1 Processor -- start 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.128+0000 7efcfa5ff700 1 -- start start 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.129+0000 7efcfa5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf40ac230 0x7efcf414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.129+0000 7efcfa5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4155c10 0x7efcf415a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.129+0000 7efcfa5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf41568a0 0x7efcf40aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.129+0000 7efcfa5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf414c600 con 0x7efcf4155c10 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.129+0000 7efcfa5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcf415ab10 con 0x7efcf40ac230 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.129+0000 7efcfa5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcf40aa7d0 con 0x7efcf41568a0 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4155c10 0x7efcf415a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4155c10 0x7efcf415a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40748/0 (socket says 172.21.15.119:40748) 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 -- 172.21.15.119:0/545113802 learned_addr learned my addr 172.21.15.119:0/545113802 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcf3fff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf40ac230 0x7efcf414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcf8b9c700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf41568a0 0x7efcf40aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 -- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf41568a0 msgr2=0x7efcf40aa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf41568a0 0x7efcf40aa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 -- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf40ac230 msgr2=0x7efcf414c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf40ac230 0x7efcf414c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 -- 172.21.15.119:0/545113802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc00b790 con 0x7efcf4155c10 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcf8b9c700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf41568a0 0x7efcf40aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.130+0000 7efcebfff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4155c10 0x7efcf415a5d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7efcdc006fd0 tx=0x7efcdc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:32.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.131+0000 7efcf0ff9700 1 -- 172.21.15.119:0/545113802 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00dc50 con 0x7efcf4155c10 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.131+0000 7efcf0ff9700 1 -- 172.21.15.119:0/545113802 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc010680 con 0x7efcf4155c10 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.131+0000 7efcf0ff9700 1 -- 172.21.15.119:0/545113802 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc010a40 con 0x7efcf4155c10 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.131+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf40aaa50 con 0x7efcf4155c10 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.131+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf40ab010 con 0x7efcf4155c10 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.132+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcf40ab230 con 0x7efcf4155c10 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.133+0000 7efcf0ff9700 1 -- 172.21.15.119:0/545113802 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcdc0233f0 con 0x7efcf4155c10 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.133+0000 7efcf0ff9700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcd4066f30 0x7efcd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.134+0000 7efcf0ff9700 1 -- 172.21.15.119:0/545113802 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7efcdc014070 con 0x7efcf4155c10 2023-10-08T01:14:32.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.134+0000 7efcf3fff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcd4066f30 0x7efcd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.134+0000 7efcf3fff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcd4066f30 0x7efcd40693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efce4006fd0 tx=0x7efce4008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:32.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.138+0000 7efcf0ff9700 1 -- 172.21.15.119:0/545113802 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcdc053790 con 0x7efcf4155c10 2023-10-08T01:14:32.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.325+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcf404c3c0 con 0x7efcd4066f30 2023-10-08T01:14:32.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.334+0000 7efcf0ff9700 1 -- 172.21.15.119:0/545113802 <== mgr.7488 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289491 (secure 0 0 0) 0x7efcf404c3c0 con 0x7efcd4066f30 2023-10-08T01:14:32.342 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:32.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcd4066f30 msgr2=0x7efcd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcd4066f30 0x7efcd40693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7efce4006fd0 tx=0x7efce4008040 comp rx=0 tx=0).stop 2023-10-08T01:14:32.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4155c10 msgr2=0x7efcf415a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4155c10 0x7efcf415a5d0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7efcdc006fd0 tx=0x7efcdc01b000 comp rx=0 tx=0).stop 2023-10-08T01:14:32.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 shutdown_connections 2023-10-08T01:14:32.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efcd4066f30 0x7efcd40693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efcf41568a0 0x7efcf40aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efcf40ac230 0x7efcf414c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 --2- 172.21.15.119:0/545113802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcf4155c10 0x7efcf415a5d0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 >> 172.21.15.119:0/545113802 conn(0x7efcf406a320 msgr2=0x7efcf4153810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:32.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 shutdown_connections 2023-10-08T01:14:32.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.338+0000 7efcfa5ff700 1 -- 172.21.15.119:0/545113802 wait complete. 2023-10-08T01:14:32.351 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:14:32.358 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-08T01:14:30.707956+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":403,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":265,"ondisk_log_size":265,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":191,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2448456,"kb_used_data":12808,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752526264,"statfs":{"total":773094113280,"available":770586894336,"internally_reserved":0,"allocated":13115392,"data_stored":6987876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001748"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798738+0000","last_change":"2023-10-08T01:14:22.797353+0000","last_active":"2023-10-08T01:14:22.798738+0000","last_peered":"2023-10-08T01:14:22.798738+0000","last_clean":"2023-10-08T01:14:22.798738+0000","last_became_active":"2023-10-08T01:14:22.795282+0000","last_became_peered":"2023-10-08T01:14:22.795282+0000","last_unstale":"2023-10-08T01:14:22.798738+0000","last_undegraded":"2023-10-08T01:14:22.798738+0000","last_fullsized":"2023-10-08T01:14:22.798738+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:59:19.889654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789827+0000","last_change":"2023-10-08T01:14:22.782483+0000","last_active":"2023-10-08T01:14:22.789827+0000","last_peered":"2023-10-08T01:14:22.789827+0000","last_clean":"2023-10-08T01:14:22.789827+0000","last_became_active":"2023-10-08T01:14:22.781426+0000","last_became_peered":"2023-10-08T01:14:22.781426+0000","last_unstale":"2023-10-08T01:14:22.789827+0000","last_undegraded":"2023-10-08T01:14:22.789827+0000","last_fullsized":"2023-10-08T01:14:22.789827+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:44:59.712158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794183+0000","last_change":"2023-10-08T01:14:22.786530+0000","last_active":"2023-10-08T01:14:22.794183+0000","last_peered":"2023-10-08T01:14:22.794183+0000","last_clean":"2023-10-08T01:14:22.794183+0000","last_became_active":"2023-10-08T01:14:22.786260+0000","last_became_peered":"2023-10-08T01:14:22.786260+0000","last_unstale":"2023-10-08T01:14:22.794183+0000","last_undegraded":"2023-10-08T01:14:22.794183+0000","last_fullsized":"2023-10-08T01:14:22.794183+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:45:25.960964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787217+0000","last_change":"2023-10-08T01:14:22.778060+0000","last_active":"2023-10-08T01:14:22.787217+0000","last_peered":"2023-10-08T01:14:22.787217+0000","last_clean":"2023-10-08T01:14:22.787217+0000","last_became_active":"2023-10-08T01:14:22.777818+0000","last_became_peered":"2023-10-08T01:14:22.777818+0000","last_unstale":"2023-10-08T01:14:22.787217+0000","last_undegraded":"2023-10-08T01:14:22.787217+0000","last_fullsized":"2023-10-08T01:14:22.787217+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787408+0000","last_change":"2023-10-08T01:14:22.783478+0000","last_active":"2023-10-08T01:14:22.787408+0000","last_peered":"2023-10-08T01:14:22.787408+0000","last_clean":"2023-10-08T01:14:22.787408+0000","last_became_active":"2023-10-08T01:14:22.783208+0000","last_became_peered":"2023-10-08T01:14:22.783208+0000","last_unstale":"2023-10-08T01:14:22.787408+0000","last_undegraded":"2023-10-08T01:14:22.787408+0000","last_fullsized":"2023-10-08T01:14:22.787408+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.785834+0000","last_change":"2023-10-08T01:14:22.780525+0000","last_active":"2023-10-08T01:14:22.785834+0000","last_peered":"2023-10-08T01:14:22.785834+0000","last_clean":"2023-10-08T01:14:22.785834+0000","last_became_active":"2023-10-08T01:14:22.780240+0000","last_became_peered":"2023-10-08T01:14:22.780240+0000","last_unstale":"2023-10-08T01:14:22.785834+0000","last_undegraded":"2023-10-08T01:14:22.785834+0000","last_fullsized":"2023-10-08T01:14:22.785834+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783228+0000","last_change":"2023-10-08T01:14:22.780046+0000","last_active":"2023-10-08T01:14:22.783228+0000","last_peered":"2023-10-08T01:14:22.783228+0000","last_clean":"2023-10-08T01:14:22.783228+0000","last_became_active":"2023-10-08T01:14:22.779731+0000","last_became_peered":"2023-10-08T01:14:22.779731+0000","last_unstale":"2023-10-08T01:14:22.783228+0000","last_undegraded":"2023-10-08T01:14:22.783228+0000","last_fullsized":"2023-10-08T01:14:22.783228+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788496+0000","last_change":"2023-10-08T01:14:22.783965+0000","last_active":"2023-10-08T01:14:22.788496+0000","last_peered":"2023-10-08T01:14:22.788496+0000","last_clean":"2023-10-08T01:14:22.788496+0000","last_became_active":"2023-10-08T01:14:22.783699+0000","last_became_peered":"2023-10-08T01:14:22.783699+0000","last_unstale":"2023-10-08T01:14:22.788496+0000","last_undegraded":"2023-10-08T01:14:22.788496+0000","last_fullsized":"2023-10-08T01:14:22.788496+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:06.030874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792319+0000","last_change":"2023-10-08T01:14:22.788581+0000","last_active":"2023-10-08T01:14:22.792319+0000","last_peered":"2023-10-08T01:14:22.792319+0000","last_clean":"2023-10-08T01:14:22.792319+0000","last_became_active":"2023-10-08T01:14:22.788210+0000","last_became_peered":"2023-10-08T01:14:22.788210+0000","last_unstale":"2023-10-08T01:14:22.792319+0000","last_undegraded":"2023-10-08T01:14:22.792319+0000","last_fullsized":"2023-10-08T01:14:22.792319+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795777+0000","last_change":"2023-10-08T01:14:22.794165+0000","last_active":"2023-10-08T01:14:22.795777+0000","last_peered":"2023-10-08T01:14:22.795777+0000","last_clean":"2023-10-08T01:14:22.795777+0000","last_became_active":"2023-10-08T01:14:22.793828+0000","last_became_peered":"2023-10-08T01:14:22.793828+0000","last_unstale":"2023-10-08T01:14:22.795777+0000","last_undegraded":"2023-10-08T01:14:22.795777+0000","last_fullsized":"2023-10-08T01:14:22.795777+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:30.309636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782072+0000","last_change":"2023-10-08T01:14:22.778678+0000","last_active":"2023-10-08T01:14:22.782072+0000","last_peered":"2023-10-08T01:14:22.782072+0000","last_clean":"2023-10-08T01:14:22.782072+0000","last_became_active":"2023-10-08T01:14:22.777690+0000","last_became_peered":"2023-10-08T01:14:22.777690+0000","last_unstale":"2023-10-08T01:14:22.782072+0000","last_undegraded":"2023-10-08T01:14:22.782072+0000","last_fullsized":"2023-10-08T01:14:22.782072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:07:30.209334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792484+0000","last_change":"2023-10-08T01:14:22.790486+0000","last_active":"2023-10-08T01:14:22.792484+0000","last_peered":"2023-10-08T01:14:22.792484+0000","last_clean":"2023-10-08T01:14:22.792484+0000","last_became_active":"2023-10-08T01:14:22.790175+0000","last_became_peered":"2023-10-08T01:14:22.790175+0000","last_unstale":"2023-10-08T01:14:22.792484+0000","last_undegraded":"2023-10-08T01:14:22.792484+0000","last_fullsized":"2023-10-08T01:14:22.792484+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794520+0000","last_change":"2023-10-08T01:14:22.792182+0000","last_active":"2023-10-08T01:14:22.794520+0000","last_peered":"2023-10-08T01:14:22.794520+0000","last_clean":"2023-10-08T01:14:22.794520+0000","last_became_active":"2023-10-08T01:14:22.791942+0000","last_became_peered":"2023-10-08T01:14:22.791942+0000","last_unstale":"2023-10-08T01:14:22.794520+0000","last_undegraded":"2023-10-08T01:14:22.794520+0000","last_fullsized":"2023-10-08T01:14:22.794520+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.779024+0000","last_change":"2023-10-08T01:14:22.777745+0000","last_active":"2023-10-08T01:14:22.779024+0000","last_peered":"2023-10-08T01:14:22.779024+0000","last_clean":"2023-10-08T01:14:22.779024+0000","last_became_active":"2023-10-08T01:14:22.777473+0000","last_became_peered":"2023-10-08T01:14:22.777473+0000","last_unstale":"2023-10-08T01:14:22.779024+0000","last_undegraded":"2023-10-08T01:14:22.779024+0000","last_fullsized":"2023-10-08T01:14:22.779024+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:57.315090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793463+0000","last_change":"2023-10-08T01:14:22.790518+0000","last_active":"2023-10-08T01:14:22.793463+0000","last_peered":"2023-10-08T01:14:22.793463+0000","last_clean":"2023-10-08T01:14:22.793463+0000","last_became_active":"2023-10-08T01:14:22.790204+0000","last_became_peered":"2023-10-08T01:14:22.790204+0000","last_unstale":"2023-10-08T01:14:22.793463+0000","last_undegraded":"2023-10-08T01:14:22.793463+0000","last_fullsized":"2023-10-08T01:14:22.793463+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789520+0000","last_change":"2023-10-08T01:14:22.785475+0000","last_active":"2023-10-08T01:14:22.789520+0000","last_peered":"2023-10-08T01:14:22.789520+0000","last_clean":"2023-10-08T01:14:22.789520+0000","last_became_active":"2023-10-08T01:14:22.785148+0000","last_became_peered":"2023-10-08T01:14:22.785148+0000","last_unstale":"2023-10-08T01:14:22.789520+0000","last_undegraded":"2023-10-08T01:14:22.789520+0000","last_fullsized":"2023-10-08T01:14:22.789520+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:42:48.572865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788635+0000","last_change":"2023-10-08T01:14:22.782436+0000","last_active":"2023-10-08T01:14:22.788635+0000","last_peered":"2023-10-08T01:14:22.788635+0000","last_clean":"2023-10-08T01:14:22.788635+0000","last_became_active":"2023-10-08T01:14:22.782125+0000","last_became_peered":"2023-10-08T01:14:22.782125+0000","last_unstale":"2023-10-08T01:14:22.788635+0000","last_undegraded":"2023-10-08T01:14:22.788635+0000","last_fullsized":"2023-10-08T01:14:22.788635+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:26.317877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787247+0000","last_change":"2023-10-08T01:14:22.779107+0000","last_active":"2023-10-08T01:14:22.787247+0000","last_peered":"2023-10-08T01:14:22.787247+0000","last_clean":"2023-10-08T01:14:22.787247+0000","last_became_active":"2023-10-08T01:14:22.778834+0000","last_became_peered":"2023-10-08T01:14:22.778834+0000","last_unstale":"2023-10-08T01:14:22.787247+0000","last_undegraded":"2023-10-08T01:14:22.787247+0000","last_fullsized":"2023-10-08T01:14:22.787247+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792306+0000","last_change":"2023-10-08T01:14:22.789825+0000","last_active":"2023-10-08T01:14:22.792306+0000","last_peered":"2023-10-08T01:14:22.792306+0000","last_clean":"2023-10-08T01:14:22.792306+0000","last_became_active":"2023-10-08T01:14:22.789560+0000","last_became_peered":"2023-10-08T01:14:22.789560+0000","last_unstale":"2023-10-08T01:14:22.792306+0000","last_undegraded":"2023-10-08T01:14:22.792306+0000","last_fullsized":"2023-10-08T01:14:22.792306+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792179+0000","last_change":"2023-10-08T01:14:22.788302+0000","last_active":"2023-10-08T01:14:22.792179+0000","last_peered":"2023-10-08T01:14:22.792179+0000","last_clean":"2023-10-08T01:14:22.792179+0000","last_became_active":"2023-10-08T01:14:22.787799+0000","last_became_peered":"2023-10-08T01:14:22.787799+0000","last_unstale":"2023-10-08T01:14:22.792179+0000","last_undegraded":"2023-10-08T01:14:22.792179+0000","last_fullsized":"2023-10-08T01:14:22.792179+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783436+0000","last_change":"2023-10-08T01:14:22.779249+0000","last_active":"2023-10-08T01:14:22.783436+0000","last_peered":"2023-10-08T01:14:22.783436+0000","last_clean":"2023-10-08T01:14:22.783436+0000","last_became_active":"2023-10-08T01:14:22.779000+0000","last_became_peered":"2023-10-08T01:14:22.779000+0000","last_unstale":"2023-10-08T01:14:22.783436+0000","last_undegraded":"2023-10-08T01:14:22.783436+0000","last_fullsized":"2023-10-08T01:14:22.783436+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:44:52.899739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788599+0000","last_change":"2023-10-08T01:14:22.782182+0000","last_active":"2023-10-08T01:14:22.788599+0000","last_peered":"2023-10-08T01:14:22.788599+0000","last_clean":"2023-10-08T01:14:22.788599+0000","last_became_active":"2023-10-08T01:14:22.781939+0000","last_became_peered":"2023-10-08T01:14:22.781939+0000","last_unstale":"2023-10-08T01:14:22.788599+0000","last_undegraded":"2023-10-08T01:14:22.788599+0000","last_fullsized":"2023-10-08T01:14:22.788599+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:19.690095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787156+0000","last_change":"2023-10-08T01:14:22.780281+0000","last_active":"2023-10-08T01:14:22.787156+0000","last_peered":"2023-10-08T01:14:22.787156+0000","last_clean":"2023-10-08T01:14:22.787156+0000","last_became_active":"2023-10-08T01:14:22.780033+0000","last_became_peered":"2023-10-08T01:14:22.780033+0000","last_unstale":"2023-10-08T01:14:22.787156+0000","last_undegraded":"2023-10-08T01:14:22.787156+0000","last_fullsized":"2023-10-08T01:14:22.787156+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:41.656567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793213+0000","last_change":"2023-10-08T01:14:22.791550+0000","last_active":"2023-10-08T01:14:22.793213+0000","last_peered":"2023-10-08T01:14:22.793213+0000","last_clean":"2023-10-08T01:14:22.793213+0000","last_became_active":"2023-10-08T01:14:22.791030+0000","last_became_peered":"2023-10-08T01:14:22.791030+0000","last_unstale":"2023-10-08T01:14:22.793213+0000","last_undegraded":"2023-10-08T01:14:22.793213+0000","last_fullsized":"2023-10-08T01:14:22.793213+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:21:41.420761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791956+0000","last_change":"2023-10-08T01:14:22.786510+0000","last_active":"2023-10-08T01:14:22.791956+0000","last_peered":"2023-10-08T01:14:22.791956+0000","last_clean":"2023-10-08T01:14:22.791956+0000","last_became_active":"2023-10-08T01:14:22.785116+0000","last_became_peered":"2023-10-08T01:14:22.785116+0000","last_unstale":"2023-10-08T01:14:22.791956+0000","last_undegraded":"2023-10-08T01:14:22.791956+0000","last_fullsized":"2023-10-08T01:14:22.791956+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799940+0000","last_change":"2023-10-08T01:14:22.799140+0000","last_active":"2023-10-08T01:14:22.799940+0000","last_peered":"2023-10-08T01:14:22.799940+0000","last_clean":"2023-10-08T01:14:22.799940+0000","last_became_active":"2023-10-08T01:14:22.798851+0000","last_became_peered":"2023-10-08T01:14:22.798851+0000","last_unstale":"2023-10-08T01:14:22.799940+0000","last_undegraded":"2023-10-08T01:14:22.799940+0000","last_fullsized":"2023-10-08T01:14:22.799940+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:36:30.232699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.775002+0000","last_change":"2023-10-08T01:14:22.770556+0000","last_active":"2023-10-08T01:14:22.775002+0000","last_peered":"2023-10-08T01:14:22.775002+0000","last_clean":"2023-10-08T01:14:22.775002+0000","last_became_active":"2023-10-08T01:14:22.770239+0000","last_became_peered":"2023-10-08T01:14:22.770239+0000","last_unstale":"2023-10-08T01:14:22.775002+0000","last_undegraded":"2023-10-08T01:14:22.775002+0000","last_fullsized":"2023-10-08T01:14:22.775002+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787921+0000","last_change":"2023-10-08T01:14:22.782733+0000","last_active":"2023-10-08T01:14:22.787921+0000","last_peered":"2023-10-08T01:14:22.787921+0000","last_clean":"2023-10-08T01:14:22.787921+0000","last_became_active":"2023-10-08T01:14:22.782427+0000","last_became_peered":"2023-10-08T01:14:22.782427+0000","last_unstale":"2023-10-08T01:14:22.787921+0000","last_undegraded":"2023-10-08T01:14:22.787921+0000","last_fullsized":"2023-10-08T01:14:22.787921+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789675+0000","last_change":"2023-10-08T01:14:22.788999+0000","last_active":"2023-10-08T01:14:22.789675+0000","last_peered":"2023-10-08T01:14:22.789675+0000","last_clean":"2023-10-08T01:14:22.789675+0000","last_became_active":"2023-10-08T01:14:22.787618+0000","last_became_peered":"2023-10-08T01:14:22.787618+0000","last_unstale":"2023-10-08T01:14:22.789675+0000","last_undegraded":"2023-10-08T01:14:22.789675+0000","last_fullsized":"2023-10-08T01:14:22.789675+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:17.572620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791070+0000","last_change":"2023-10-08T01:14:22.788072+0000","last_active":"2023-10-08T01:14:22.791070+0000","last_peered":"2023-10-08T01:14:22.791070+0000","last_clean":"2023-10-08T01:14:22.791070+0000","last_became_active":"2023-10-08T01:14:22.787677+0000","last_became_peered":"2023-10-08T01:14:22.787677+0000","last_unstale":"2023-10-08T01:14:22.791070+0000","last_undegraded":"2023-10-08T01:14:22.791070+0000","last_fullsized":"2023-10-08T01:14:22.791070+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:19:12.933240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788371+0000","last_change":"2023-10-08T01:14:22.783785+0000","last_active":"2023-10-08T01:14:22.788371+0000","last_peered":"2023-10-08T01:14:22.788371+0000","last_clean":"2023-10-08T01:14:22.788371+0000","last_became_active":"2023-10-08T01:14:22.783501+0000","last_became_peered":"2023-10-08T01:14:22.783501+0000","last_unstale":"2023-10-08T01:14:22.788371+0000","last_undegraded":"2023-10-08T01:14:22.788371+0000","last_fullsized":"2023-10-08T01:14:22.788371+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:51.777625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798256+0000","last_change":"2023-10-08T01:14:22.794783+0000","last_active":"2023-10-08T01:14:22.798256+0000","last_peered":"2023-10-08T01:14:22.798256+0000","last_clean":"2023-10-08T01:14:22.798256+0000","last_became_active":"2023-10-08T01:14:22.794304+0000","last_became_peered":"2023-10-08T01:14:22.794304+0000","last_unstale":"2023-10-08T01:14:22.798256+0000","last_undegraded":"2023-10-08T01:14:22.798256+0000","last_fullsized":"2023-10-08T01:14:22.798256+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:59.697433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.783465+0000","last_change":"2023-10-08T01:14:22.779446+0000","last_active":"2023-10-08T01:14:22.783465+0000","last_peered":"2023-10-08T01:14:22.783465+0000","last_clean":"2023-10-08T01:14:22.783465+0000","last_became_active":"2023-10-08T01:14:22.778675+0000","last_became_peered":"2023-10-08T01:14:22.778675+0000","last_unstale":"2023-10-08T01:14:22.783465+0000","last_undegraded":"2023-10-08T01:14:22.783465+0000","last_fullsized":"2023-10-08T01:14:22.783465+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:34.335870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"119'4","reported_seq":14,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795814+0000","last_change":"2023-10-08T01:14:21.755008+0000","last_active":"2023-10-08T01:14:22.795814+0000","last_peered":"2023-10-08T01:14:22.795814+0000","last_clean":"2023-10-08T01:14:22.795814+0000","last_became_active":"2023-10-08T01:14:21.754712+0000","last_became_peered":"2023-10-08T01:14:21.754712+0000","last_unstale":"2023-10-08T01:14:22.795814+0000","last_undegraded":"2023-10-08T01:14:22.795814+0000","last_fullsized":"2023-10-08T01:14:22.795814+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:24.330581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788340+0000","last_change":"2023-10-08T01:14:22.782815+0000","last_active":"2023-10-08T01:14:22.788340+0000","last_peered":"2023-10-08T01:14:22.788340+0000","last_clean":"2023-10-08T01:14:22.788340+0000","last_became_active":"2023-10-08T01:14:22.782563+0000","last_became_peered":"2023-10-08T01:14:22.782563+0000","last_unstale":"2023-10-08T01:14:22.788340+0000","last_undegraded":"2023-10-08T01:14:22.788340+0000","last_fullsized":"2023-10-08T01:14:22.788340+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794676+0000","last_change":"2023-10-08T01:14:21.767509+0000","last_active":"2023-10-08T01:14:22.794676+0000","last_peered":"2023-10-08T01:14:22.794676+0000","last_clean":"2023-10-08T01:14:22.794676+0000","last_became_active":"2023-10-08T01:14:21.767219+0000","last_became_peered":"2023-10-08T01:14:21.767219+0000","last_unstale":"2023-10-08T01:14:22.794676+0000","last_undegraded":"2023-10-08T01:14:22.794676+0000","last_fullsized":"2023-10-08T01:14:22.794676+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.799279+0000","last_change":"2023-10-08T01:14:22.798147+0000","last_active":"2023-10-08T01:14:22.799279+0000","last_peered":"2023-10-08T01:14:22.799279+0000","last_clean":"2023-10-08T01:14:22.799279+0000","last_became_active":"2023-10-08T01:14:22.797856+0000","last_became_peered":"2023-10-08T01:14:22.797856+0000","last_unstale":"2023-10-08T01:14:22.799279+0000","last_undegraded":"2023-10-08T01:14:22.799279+0000","last_fullsized":"2023-10-08T01:14:22.799279+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:32.952368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793353+0000","last_change":"2023-10-08T01:14:21.758907+0000","last_active":"2023-10-08T01:14:22.793353+0000","last_peered":"2023-10-08T01:14:22.793353+0000","last_clean":"2023-10-08T01:14:22.793353+0000","last_became_active":"2023-10-08T01:14:21.758669+0000","last_became_peered":"2023-10-08T01:14:21.758669+0000","last_unstale":"2023-10-08T01:14:22.793353+0000","last_undegraded":"2023-10-08T01:14:22.793353+0000","last_fullsized":"2023-10-08T01:14:22.793353+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:33.791532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782110+0000","last_change":"2023-10-08T01:14:22.777591+0000","last_active":"2023-10-08T01:14:22.782110+0000","last_peered":"2023-10-08T01:14:22.782110+0000","last_clean":"2023-10-08T01:14:22.782110+0000","last_became_active":"2023-10-08T01:14:22.777315+0000","last_became_peered":"2023-10-08T01:14:22.777315+0000","last_unstale":"2023-10-08T01:14:22.782110+0000","last_undegraded":"2023-10-08T01:14:22.782110+0000","last_fullsized":"2023-10-08T01:14:22.782110+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793240+0000","last_change":"2023-10-08T01:14:21.770043+0000","last_active":"2023-10-08T01:14:22.793240+0000","last_peered":"2023-10-08T01:14:22.793240+0000","last_clean":"2023-10-08T01:14:22.793240+0000","last_became_active":"2023-10-08T01:14:21.769799+0000","last_became_peered":"2023-10-08T01:14:21.769799+0000","last_unstale":"2023-10-08T01:14:22.793240+0000","last_undegraded":"2023-10-08T01:14:22.793240+0000","last_fullsized":"2023-10-08T01:14:22.793240+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:29.304154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.786431+0000","last_change":"2023-10-08T01:14:22.785288+0000","last_active":"2023-10-08T01:14:22.786431+0000","last_peered":"2023-10-08T01:14:22.786431+0000","last_clean":"2023-10-08T01:14:22.786431+0000","last_became_active":"2023-10-08T01:14:22.784973+0000","last_became_peered":"2023-10-08T01:14:22.784973+0000","last_unstale":"2023-10-08T01:14:22.786431+0000","last_undegraded":"2023-10-08T01:14:22.786431+0000","last_fullsized":"2023-10-08T01:14:22.786431+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:45.942767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.786306+0000","last_change":"2023-10-08T01:14:21.767177+0000","last_active":"2023-10-08T01:14:22.786306+0000","last_peered":"2023-10-08T01:14:22.786306+0000","last_clean":"2023-10-08T01:14:22.786306+0000","last_became_active":"2023-10-08T01:14:21.766891+0000","last_became_peered":"2023-10-08T01:14:21.766891+0000","last_unstale":"2023-10-08T01:14:22.786306+0000","last_undegraded":"2023-10-08T01:14:22.786306+0000","last_fullsized":"2023-10-08T01:14:22.786306+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:48.667974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.777234+0000","last_change":"2023-10-08T01:14:22.772907+0000","last_active":"2023-10-08T01:14:22.777234+0000","last_peered":"2023-10-08T01:14:22.777234+0000","last_clean":"2023-10-08T01:14:22.777234+0000","last_became_active":"2023-10-08T01:14:22.772625+0000","last_became_peered":"2023-10-08T01:14:22.772625+0000","last_unstale":"2023-10-08T01:14:22.777234+0000","last_undegraded":"2023-10-08T01:14:22.777234+0000","last_fullsized":"2023-10-08T01:14:22.777234+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:13.379147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798893+0000","last_change":"2023-10-08T01:14:21.763020+0000","last_active":"2023-10-08T01:14:22.798893+0000","last_peered":"2023-10-08T01:14:22.798893+0000","last_clean":"2023-10-08T01:14:22.798893+0000","last_became_active":"2023-10-08T01:14:21.762782+0000","last_became_peered":"2023-10-08T01:14:21.762782+0000","last_unstale":"2023-10-08T01:14:22.798893+0000","last_undegraded":"2023-10-08T01:14:22.798893+0000","last_fullsized":"2023-10-08T01:14:22.798893+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:59.904749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793777+0000","last_change":"2023-10-08T01:14:22.791043+0000","last_active":"2023-10-08T01:14:22.793777+0000","last_peered":"2023-10-08T01:14:22.793777+0000","last_clean":"2023-10-08T01:14:22.793777+0000","last_became_active":"2023-10-08T01:14:22.790796+0000","last_became_peered":"2023-10-08T01:14:22.790796+0000","last_unstale":"2023-10-08T01:14:22.793777+0000","last_undegraded":"2023-10-08T01:14:22.793777+0000","last_fullsized":"2023-10-08T01:14:22.793777+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:36:25.989312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.763917+0000","last_change":"2023-10-08T01:14:21.760681+0000","last_active":"2023-10-08T01:14:22.763917+0000","last_peered":"2023-10-08T01:14:22.763917+0000","last_clean":"2023-10-08T01:14:22.763917+0000","last_became_active":"2023-10-08T01:14:21.759392+0000","last_became_peered":"2023-10-08T01:14:21.759392+0000","last_unstale":"2023-10-08T01:14:22.763917+0000","last_undegraded":"2023-10-08T01:14:22.763917+0000","last_fullsized":"2023-10-08T01:14:22.763917+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:00.436555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789574+0000","last_change":"2023-10-08T01:14:22.783171+0000","last_active":"2023-10-08T01:14:22.789574+0000","last_peered":"2023-10-08T01:14:22.789574+0000","last_clean":"2023-10-08T01:14:22.789574+0000","last_became_active":"2023-10-08T01:14:22.782872+0000","last_became_peered":"2023-10-08T01:14:22.782872+0000","last_unstale":"2023-10-08T01:14:22.789574+0000","last_undegraded":"2023-10-08T01:14:22.789574+0000","last_fullsized":"2023-10-08T01:14:22.789574+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.780252+0000","last_change":"2023-10-08T01:14:21.766845+0000","last_active":"2023-10-08T01:14:22.780252+0000","last_peered":"2023-10-08T01:14:22.780252+0000","last_clean":"2023-10-08T01:14:22.780252+0000","last_became_active":"2023-10-08T01:14:21.765270+0000","last_became_peered":"2023-10-08T01:14:21.765270+0000","last_unstale":"2023-10-08T01:14:22.780252+0000","last_undegraded":"2023-10-08T01:14:22.780252+0000","last_fullsized":"2023-10-08T01:14:22.780252+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:36:33.330389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789845+0000","last_change":"2023-10-08T01:14:22.786181+0000","last_active":"2023-10-08T01:14:22.789845+0000","last_peered":"2023-10-08T01:14:22.789845+0000","last_clean":"2023-10-08T01:14:22.789845+0000","last_became_active":"2023-10-08T01:14:22.785912+0000","last_became_peered":"2023-10-08T01:14:22.785912+0000","last_unstale":"2023-10-08T01:14:22.789845+0000","last_undegraded":"2023-10-08T01:14:22.789845+0000","last_fullsized":"2023-10-08T01:14:22.789845+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:33.304186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.781565+0000","last_change":"2023-10-08T01:14:21.763195+0000","last_active":"2023-10-08T01:14:22.781565+0000","last_peered":"2023-10-08T01:14:22.781565+0000","last_clean":"2023-10-08T01:14:22.781565+0000","last_became_active":"2023-10-08T01:14:21.762953+0000","last_became_peered":"2023-10-08T01:14:21.762953+0000","last_unstale":"2023-10-08T01:14:22.781565+0000","last_undegraded":"2023-10-08T01:14:22.781565+0000","last_fullsized":"2023-10-08T01:14:22.781565+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:16:56.309608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789148+0000","last_change":"2023-10-08T01:14:22.782447+0000","last_active":"2023-10-08T01:14:22.789148+0000","last_peered":"2023-10-08T01:14:22.789148+0000","last_clean":"2023-10-08T01:14:22.789148+0000","last_became_active":"2023-10-08T01:14:22.782195+0000","last_became_peered":"2023-10-08T01:14:22.782195+0000","last_unstale":"2023-10-08T01:14:22.789148+0000","last_undegraded":"2023-10-08T01:14:22.789148+0000","last_fullsized":"2023-10-08T01:14:22.789148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.775594+0000","last_change":"2023-10-08T01:14:21.756007+0000","last_active":"2023-10-08T01:14:22.775594+0000","last_peered":"2023-10-08T01:14:22.775594+0000","last_clean":"2023-10-08T01:14:22.775594+0000","last_became_active":"2023-10-08T01:14:21.755725+0000","last_became_peered":"2023-10-08T01:14:21.755725+0000","last_unstale":"2023-10-08T01:14:22.775594+0000","last_undegraded":"2023-10-08T01:14:22.775594+0000","last_fullsized":"2023-10-08T01:14:22.775594+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792488+0000","last_change":"2023-10-08T01:14:22.787821+0000","last_active":"2023-10-08T01:14:22.792488+0000","last_peered":"2023-10-08T01:14:22.792488+0000","last_clean":"2023-10-08T01:14:22.792488+0000","last_became_active":"2023-10-08T01:14:22.784125+0000","last_became_peered":"2023-10-08T01:14:22.784125+0000","last_unstale":"2023-10-08T01:14:22.792488+0000","last_undegraded":"2023-10-08T01:14:22.792488+0000","last_fullsized":"2023-10-08T01:14:22.792488+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:30.315094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"119'3","reported_seq":13,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792752+0000","last_change":"2023-10-08T01:14:21.768343+0000","last_active":"2023-10-08T01:14:22.792752+0000","last_peered":"2023-10-08T01:14:22.792752+0000","last_clean":"2023-10-08T01:14:22.792752+0000","last_became_active":"2023-10-08T01:14:21.767996+0000","last_became_peered":"2023-10-08T01:14:21.767996+0000","last_unstale":"2023-10-08T01:14:22.792752+0000","last_undegraded":"2023-10-08T01:14:22.792752+0000","last_fullsized":"2023-10-08T01:14:22.792752+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:44.937285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.777118+0000","last_change":"2023-10-08T01:14:22.773623+0000","last_active":"2023-10-08T01:14:22.777118+0000","last_peered":"2023-10-08T01:14:22.777118+0000","last_clean":"2023-10-08T01:14:22.777118+0000","last_became_active":"2023-10-08T01:14:22.773338+0000","last_became_peered":"2023-10-08T01:14:22.773338+0000","last_unstale":"2023-10-08T01:14:22.777118+0000","last_undegraded":"2023-10-08T01:14:22.777118+0000","last_fullsized":"2023-10-08T01:14:22.777118+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:58:23.838250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"119'6","reported_seq":16,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.800832+0000","last_change":"2023-10-08T01:14:21.764918+0000","last_active":"2023-10-08T01:14:22.800832+0000","last_peered":"2023-10-08T01:14:22.800832+0000","last_clean":"2023-10-08T01:14:22.800832+0000","last_became_active":"2023-10-08T01:14:21.763653+0000","last_became_peered":"2023-10-08T01:14:21.763653+0000","last_unstale":"2023-10-08T01:14:22.800832+0000","last_undegraded":"2023-10-08T01:14:22.800832+0000","last_fullsized":"2023-10-08T01:14:22.800832+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:25:53.462148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787341+0000","last_change":"2023-10-08T01:14:22.778889+0000","last_active":"2023-10-08T01:14:22.787341+0000","last_peered":"2023-10-08T01:14:22.787341+0000","last_clean":"2023-10-08T01:14:22.787341+0000","last_became_active":"2023-10-08T01:14:22.778592+0000","last_became_peered":"2023-10-08T01:14:22.778592+0000","last_unstale":"2023-10-08T01:14:22.787341+0000","last_undegraded":"2023-10-08T01:14:22.787341+0000","last_fullsized":"2023-10-08T01:14:22.787341+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:45.881758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793156+0000","last_change":"2023-10-08T01:14:21.769762+0000","last_active":"2023-10-08T01:14:22.793156+0000","last_peered":"2023-10-08T01:14:22.793156+0000","last_clean":"2023-10-08T01:14:22.793156+0000","last_became_active":"2023-10-08T01:14:21.769522+0000","last_became_peered":"2023-10-08T01:14:21.769522+0000","last_unstale":"2023-10-08T01:14:22.793156+0000","last_undegraded":"2023-10-08T01:14:22.793156+0000","last_fullsized":"2023-10-08T01:14:22.793156+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:27.726884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797374+0000","last_change":"2023-10-08T01:14:22.791650+0000","last_active":"2023-10-08T01:14:22.797374+0000","last_peered":"2023-10-08T01:14:22.797374+0000","last_clean":"2023-10-08T01:14:22.797374+0000","last_became_active":"2023-10-08T01:14:22.791109+0000","last_became_peered":"2023-10-08T01:14:22.791109+0000","last_unstale":"2023-10-08T01:14:22.797374+0000","last_undegraded":"2023-10-08T01:14:22.797374+0000","last_fullsized":"2023-10-08T01:14:22.797374+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765608+0000","last_change":"2023-10-08T01:14:21.768459+0000","last_active":"2023-10-08T01:14:22.765608+0000","last_peered":"2023-10-08T01:14:22.765608+0000","last_clean":"2023-10-08T01:14:22.765608+0000","last_became_active":"2023-10-08T01:14:21.768189+0000","last_became_peered":"2023-10-08T01:14:21.768189+0000","last_unstale":"2023-10-08T01:14:22.765608+0000","last_undegraded":"2023-10-08T01:14:22.765608+0000","last_fullsized":"2023-10-08T01:14:22.765608+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:54.324955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789515+0000","last_change":"2023-10-08T01:14:22.781160+0000","last_active":"2023-10-08T01:14:22.789515+0000","last_peered":"2023-10-08T01:14:22.789515+0000","last_clean":"2023-10-08T01:14:22.789515+0000","last_became_active":"2023-10-08T01:14:22.780893+0000","last_became_peered":"2023-10-08T01:14:22.780893+0000","last_unstale":"2023-10-08T01:14:22.789515+0000","last_undegraded":"2023-10-08T01:14:22.789515+0000","last_fullsized":"2023-10-08T01:14:22.789515+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:04:30.309636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768642+0000","last_change":"2023-10-08T01:14:21.760294+0000","last_active":"2023-10-08T01:14:22.768642+0000","last_peered":"2023-10-08T01:14:22.768642+0000","last_clean":"2023-10-08T01:14:22.768642+0000","last_became_active":"2023-10-08T01:14:21.758928+0000","last_became_peered":"2023-10-08T01:14:21.758928+0000","last_unstale":"2023-10-08T01:14:22.768642+0000","last_undegraded":"2023-10-08T01:14:22.768642+0000","last_fullsized":"2023-10-08T01:14:22.768642+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788009+0000","last_change":"2023-10-08T01:14:22.784690+0000","last_active":"2023-10-08T01:14:22.788009+0000","last_peered":"2023-10-08T01:14:22.788009+0000","last_clean":"2023-10-08T01:14:22.788009+0000","last_became_active":"2023-10-08T01:14:22.784408+0000","last_became_peered":"2023-10-08T01:14:22.784408+0000","last_unstale":"2023-10-08T01:14:22.788009+0000","last_undegraded":"2023-10-08T01:14:22.788009+0000","last_fullsized":"2023-10-08T01:14:22.788009+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:55.180105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"119'8","reported_seq":18,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.795020+0000","last_change":"2023-10-08T01:14:21.765533+0000","last_active":"2023-10-08T01:14:22.795020+0000","last_peered":"2023-10-08T01:14:22.795020+0000","last_clean":"2023-10-08T01:14:22.795020+0000","last_became_active":"2023-10-08T01:14:21.764914+0000","last_became_peered":"2023-10-08T01:14:21.764914+0000","last_unstale":"2023-10-08T01:14:22.795020+0000","last_undegraded":"2023-10-08T01:14:22.795020+0000","last_fullsized":"2023-10-08T01:14:22.795020+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:47:00.128756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.791772+0000","last_change":"2023-10-08T01:14:22.784334+0000","last_active":"2023-10-08T01:14:22.791772+0000","last_peered":"2023-10-08T01:14:22.791772+0000","last_clean":"2023-10-08T01:14:22.791772+0000","last_became_active":"2023-10-08T01:14:22.784048+0000","last_became_peered":"2023-10-08T01:14:22.784048+0000","last_unstale":"2023-10-08T01:14:22.791772+0000","last_undegraded":"2023-10-08T01:14:22.791772+0000","last_fullsized":"2023-10-08T01:14:22.791772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:31:34.797013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797139+0000","last_change":"2023-10-08T01:14:21.767123+0000","last_active":"2023-10-08T01:14:22.797139+0000","last_peered":"2023-10-08T01:14:22.797139+0000","last_clean":"2023-10-08T01:14:22.797139+0000","last_became_active":"2023-10-08T01:14:21.766826+0000","last_became_peered":"2023-10-08T01:14:21.766826+0000","last_unstale":"2023-10-08T01:14:22.797139+0000","last_undegraded":"2023-10-08T01:14:22.797139+0000","last_fullsized":"2023-10-08T01:14:22.797139+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:50:38.595943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.779071+0000","last_change":"2023-10-08T01:14:22.775187+0000","last_active":"2023-10-08T01:14:22.779071+0000","last_peered":"2023-10-08T01:14:22.779071+0000","last_clean":"2023-10-08T01:14:22.779071+0000","last_became_active":"2023-10-08T01:14:22.774922+0000","last_became_peered":"2023-10-08T01:14:22.774922+0000","last_unstale":"2023-10-08T01:14:22.779071+0000","last_undegraded":"2023-10-08T01:14:22.779071+0000","last_fullsized":"2023-10-08T01:14:22.779071+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:36.091585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.793251+0000","last_change":"2023-10-08T01:14:21.761079+0000","last_active":"2023-10-08T01:14:22.793251+0000","last_peered":"2023-10-08T01:14:22.793251+0000","last_clean":"2023-10-08T01:14:22.793251+0000","last_became_active":"2023-10-08T01:14:21.760705+0000","last_became_peered":"2023-10-08T01:14:21.760705+0000","last_unstale":"2023-10-08T01:14:22.793251+0000","last_undegraded":"2023-10-08T01:14:22.793251+0000","last_fullsized":"2023-10-08T01:14:22.793251+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:16:34.632642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788803+0000","last_change":"2023-10-08T01:14:22.785209+0000","last_active":"2023-10-08T01:14:22.788803+0000","last_peered":"2023-10-08T01:14:22.788803+0000","last_clean":"2023-10-08T01:14:22.788803+0000","last_became_active":"2023-10-08T01:14:22.784735+0000","last_became_peered":"2023-10-08T01:14:22.784735+0000","last_unstale":"2023-10-08T01:14:22.788803+0000","last_undegraded":"2023-10-08T01:14:22.788803+0000","last_fullsized":"2023-10-08T01:14:22.788803+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:57:50.424173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.767122+0000","last_change":"2023-10-08T01:14:21.769975+0000","last_active":"2023-10-08T01:14:22.767122+0000","last_peered":"2023-10-08T01:14:22.767122+0000","last_clean":"2023-10-08T01:14:22.767122+0000","last_became_active":"2023-10-08T01:14:21.769477+0000","last_became_peered":"2023-10-08T01:14:21.769477+0000","last_unstale":"2023-10-08T01:14:22.767122+0000","last_undegraded":"2023-10-08T01:14:22.767122+0000","last_fullsized":"2023-10-08T01:14:22.767122+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:34.177414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787894+0000","last_change":"2023-10-08T01:14:22.781041+0000","last_active":"2023-10-08T01:14:22.787894+0000","last_peered":"2023-10-08T01:14:22.787894+0000","last_clean":"2023-10-08T01:14:22.787894+0000","last_became_active":"2023-10-08T01:14:22.780765+0000","last_became_peered":"2023-10-08T01:14:22.780765+0000","last_unstale":"2023-10-08T01:14:22.787894+0000","last_undegraded":"2023-10-08T01:14:22.787894+0000","last_fullsized":"2023-10-08T01:14:22.787894+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:25:00.278431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792713+0000","last_change":"2023-10-08T01:14:21.768612+0000","last_active":"2023-10-08T01:14:22.792713+0000","last_peered":"2023-10-08T01:14:22.792713+0000","last_clean":"2023-10-08T01:14:22.792713+0000","last_became_active":"2023-10-08T01:14:21.767991+0000","last_became_peered":"2023-10-08T01:14:21.767991+0000","last_unstale":"2023-10-08T01:14:22.792713+0000","last_undegraded":"2023-10-08T01:14:22.792713+0000","last_fullsized":"2023-10-08T01:14:22.792713+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:50.772143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788094+0000","last_change":"2023-10-08T01:14:22.783198+0000","last_active":"2023-10-08T01:14:22.788094+0000","last_peered":"2023-10-08T01:14:22.788094+0000","last_clean":"2023-10-08T01:14:22.788094+0000","last_became_active":"2023-10-08T01:14:22.782950+0000","last_became_peered":"2023-10-08T01:14:22.782950+0000","last_unstale":"2023-10-08T01:14:22.788094+0000","last_undegraded":"2023-10-08T01:14:22.788094+0000","last_fullsized":"2023-10-08T01:14:22.788094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:30:30.559893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.767998+0000","last_change":"2023-10-08T01:14:21.762370+0000","last_active":"2023-10-08T01:14:22.767998+0000","last_peered":"2023-10-08T01:14:22.767998+0000","last_clean":"2023-10-08T01:14:22.767998+0000","last_became_active":"2023-10-08T01:14:21.761985+0000","last_became_peered":"2023-10-08T01:14:21.761985+0000","last_unstale":"2023-10-08T01:14:22.767998+0000","last_undegraded":"2023-10-08T01:14:22.767998+0000","last_fullsized":"2023-10-08T01:14:22.767998+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:12.383080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788041+0000","last_change":"2023-10-08T01:14:22.782578+0000","last_active":"2023-10-08T01:14:22.788041+0000","last_peered":"2023-10-08T01:14:22.788041+0000","last_clean":"2023-10-08T01:14:22.788041+0000","last_became_active":"2023-10-08T01:14:22.781703+0000","last_became_peered":"2023-10-08T01:14:22.781703+0000","last_unstale":"2023-10-08T01:14:22.788041+0000","last_undegraded":"2023-10-08T01:14:22.788041+0000","last_fullsized":"2023-10-08T01:14:22.788041+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:42:38.983859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768007+0000","last_change":"2023-10-08T01:14:21.753473+0000","last_active":"2023-10-08T01:14:22.768007+0000","last_peered":"2023-10-08T01:14:22.768007+0000","last_clean":"2023-10-08T01:14:22.768007+0000","last_became_active":"2023-10-08T01:14:21.753194+0000","last_became_peered":"2023-10-08T01:14:21.753194+0000","last_unstale":"2023-10-08T01:14:22.768007+0000","last_undegraded":"2023-10-08T01:14:22.768007+0000","last_fullsized":"2023-10-08T01:14:22.768007+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:14:53.171270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.792546+0000","last_change":"2023-10-08T01:14:22.788676+0000","last_active":"2023-10-08T01:14:22.792546+0000","last_peered":"2023-10-08T01:14:22.792546+0000","last_clean":"2023-10-08T01:14:22.792546+0000","last_became_active":"2023-10-08T01:14:22.788378+0000","last_became_peered":"2023-10-08T01:14:22.788378+0000","last_unstale":"2023-10-08T01:14:22.792546+0000","last_undegraded":"2023-10-08T01:14:22.792546+0000","last_fullsized":"2023-10-08T01:14:22.792546+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:45:51.777625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.763121+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.762883+0000","last_became_peered":"2023-10-08T01:14:21.762883+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:52:03.018597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789807+0000","last_change":"2023-10-08T01:14:22.782302+0000","last_active":"2023-10-08T01:14:22.789807+0000","last_peered":"2023-10-08T01:14:22.789807+0000","last_clean":"2023-10-08T01:14:22.789807+0000","last_became_active":"2023-10-08T01:14:22.780497+0000","last_became_peered":"2023-10-08T01:14:22.780497+0000","last_unstale":"2023-10-08T01:14:22.789807+0000","last_undegraded":"2023-10-08T01:14:22.789807+0000","last_fullsized":"2023-10-08T01:14:22.789807+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:59.391189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"119'2","reported_seq":12,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.790321+0000","last_change":"2023-10-08T01:14:21.760699+0000","last_active":"2023-10-08T01:14:22.790321+0000","last_peered":"2023-10-08T01:14:22.790321+0000","last_clean":"2023-10-08T01:14:22.790321+0000","last_became_active":"2023-10-08T01:14:21.760424+0000","last_became_peered":"2023-10-08T01:14:21.760424+0000","last_unstale":"2023-10-08T01:14:22.790321+0000","last_undegraded":"2023-10-08T01:14:22.790321+0000","last_fullsized":"2023-10-08T01:14:22.790321+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:13:59.004815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.798148+0000","last_change":"2023-10-08T01:14:22.793832+0000","last_active":"2023-10-08T01:14:22.798148+0000","last_peered":"2023-10-08T01:14:22.798148+0000","last_clean":"2023-10-08T01:14:22.798148+0000","last_became_active":"2023-10-08T01:14:22.792633+0000","last_became_peered":"2023-10-08T01:14:22.792633+0000","last_unstale":"2023-10-08T01:14:22.798148+0000","last_undegraded":"2023-10-08T01:14:22.798148+0000","last_fullsized":"2023-10-08T01:14:22.798148+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:58:11.281859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.768046+0000","last_change":"2023-10-08T01:14:21.763932+0000","last_active":"2023-10-08T01:14:22.768046+0000","last_peered":"2023-10-08T01:14:22.768046+0000","last_clean":"2023-10-08T01:14:22.768046+0000","last_became_active":"2023-10-08T01:14:21.763673+0000","last_became_peered":"2023-10-08T01:14:21.763673+0000","last_unstale":"2023-10-08T01:14:22.768046+0000","last_undegraded":"2023-10-08T01:14:22.768046+0000","last_fullsized":"2023-10-08T01:14:22.768046+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:07.616557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.797649+0000","last_change":"2023-10-08T01:14:22.791844+0000","last_active":"2023-10-08T01:14:22.797649+0000","last_peered":"2023-10-08T01:14:22.797649+0000","last_clean":"2023-10-08T01:14:22.797649+0000","last_became_active":"2023-10-08T01:14:22.791583+0000","last_became_peered":"2023-10-08T01:14:22.791583+0000","last_unstale":"2023-10-08T01:14:22.797649+0000","last_undegraded":"2023-10-08T01:14:22.797649+0000","last_fullsized":"2023-10-08T01:14:22.797649+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:33:37.307586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765828+0000","last_change":"2023-10-08T01:14:21.770984+0000","last_active":"2023-10-08T01:14:22.765828+0000","last_peered":"2023-10-08T01:14:22.765828+0000","last_clean":"2023-10-08T01:14:22.765828+0000","last_became_active":"2023-10-08T01:14:21.770705+0000","last_became_peered":"2023-10-08T01:14:21.770705+0000","last_unstale":"2023-10-08T01:14:22.765828+0000","last_undegraded":"2023-10-08T01:14:22.765828+0000","last_fullsized":"2023-10-08T01:14:22.765828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:48:20.485203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782896+0000","last_change":"2023-10-08T01:14:22.781159+0000","last_active":"2023-10-08T01:14:22.782896+0000","last_peered":"2023-10-08T01:14:22.782896+0000","last_clean":"2023-10-08T01:14:22.782896+0000","last_became_active":"2023-10-08T01:14:22.780879+0000","last_became_peered":"2023-10-08T01:14:22.780879+0000","last_unstale":"2023-10-08T01:14:22.782896+0000","last_undegraded":"2023-10-08T01:14:22.782896+0000","last_fullsized":"2023-10-08T01:14:22.782896+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:25.864976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.764576+0000","last_change":"2023-10-08T01:14:21.760347+0000","last_active":"2023-10-08T01:14:22.764576+0000","last_peered":"2023-10-08T01:14:22.764576+0000","last_clean":"2023-10-08T01:14:22.764576+0000","last_became_active":"2023-10-08T01:14:21.759959+0000","last_became_peered":"2023-10-08T01:14:21.759959+0000","last_unstale":"2023-10-08T01:14:22.764576+0000","last_undegraded":"2023-10-08T01:14:22.764576+0000","last_fullsized":"2023-10-08T01:14:22.764576+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:14:53.171270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.788178+0000","last_change":"2023-10-08T01:14:22.784755+0000","last_active":"2023-10-08T01:14:22.788178+0000","last_peered":"2023-10-08T01:14:22.788178+0000","last_clean":"2023-10-08T01:14:22.788178+0000","last_became_active":"2023-10-08T01:14:22.784506+0000","last_became_peered":"2023-10-08T01:14:22.784506+0000","last_unstale":"2023-10-08T01:14:22.788178+0000","last_undegraded":"2023-10-08T01:14:22.788178+0000","last_fullsized":"2023-10-08T01:14:22.788178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:36:12.608356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766909+0000","last_change":"2023-10-08T01:14:21.754925+0000","last_active":"2023-10-08T01:14:22.766909+0000","last_peered":"2023-10-08T01:14:22.766909+0000","last_clean":"2023-10-08T01:14:22.766909+0000","last_became_active":"2023-10-08T01:14:21.754606+0000","last_became_peered":"2023-10-08T01:14:21.754606+0000","last_unstale":"2023-10-08T01:14:22.766909+0000","last_undegraded":"2023-10-08T01:14:22.766909+0000","last_fullsized":"2023-10-08T01:14:22.766909+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:26:07.890275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.790593+0000","last_change":"2023-10-08T01:14:22.787439+0000","last_active":"2023-10-08T01:14:22.790593+0000","last_peered":"2023-10-08T01:14:22.790593+0000","last_clean":"2023-10-08T01:14:22.790593+0000","last_became_active":"2023-10-08T01:14:22.787105+0000","last_became_peered":"2023-10-08T01:14:22.787105+0000","last_unstale":"2023-10-08T01:14:22.790593+0000","last_undegraded":"2023-10-08T01:14:22.790593+0000","last_fullsized":"2023-10-08T01:14:22.790593+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:08.622039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.762908+0000","last_change":"2023-10-08T01:14:21.767557+0000","last_active":"2023-10-08T01:14:22.762908+0000","last_peered":"2023-10-08T01:14:22.762908+0000","last_clean":"2023-10-08T01:14:22.762908+0000","last_became_active":"2023-10-08T01:14:21.767293+0000","last_became_peered":"2023-10-08T01:14:21.767293+0000","last_unstale":"2023-10-08T01:14:22.762908+0000","last_undegraded":"2023-10-08T01:14:22.762908+0000","last_fullsized":"2023-10-08T01:14:22.762908+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:12.383080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.789364+0000","last_change":"2023-10-08T01:14:22.785451+0000","last_active":"2023-10-08T01:14:22.789364+0000","last_peered":"2023-10-08T01:14:22.789364+0000","last_clean":"2023-10-08T01:14:22.789364+0000","last_became_active":"2023-10-08T01:14:22.785188+0000","last_became_peered":"2023-10-08T01:14:22.785188+0000","last_unstale":"2023-10-08T01:14:22.789364+0000","last_undegraded":"2023-10-08T01:14:22.789364+0000","last_fullsized":"2023-10-08T01:14:22.789364+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.228339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.773008+0000","last_change":"2023-10-08T01:14:21.764982+0000","last_active":"2023-10-08T01:14:22.773008+0000","last_peered":"2023-10-08T01:14:22.773008+0000","last_clean":"2023-10-08T01:14:22.773008+0000","last_became_active":"2023-10-08T01:14:21.764710+0000","last_became_peered":"2023-10-08T01:14:21.764710+0000","last_unstale":"2023-10-08T01:14:22.773008+0000","last_undegraded":"2023-10-08T01:14:22.773008+0000","last_fullsized":"2023-10-08T01:14:22.773008+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:18:47.662167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.782858+0000","last_change":"2023-10-08T01:14:22.780210+0000","last_active":"2023-10-08T01:14:22.782858+0000","last_peered":"2023-10-08T01:14:22.782858+0000","last_clean":"2023-10-08T01:14:22.782858+0000","last_became_active":"2023-10-08T01:14:22.779948+0000","last_became_peered":"2023-10-08T01:14:22.779948+0000","last_unstale":"2023-10-08T01:14:22.782858+0000","last_undegraded":"2023-10-08T01:14:22.782858+0000","last_fullsized":"2023-10-08T01:14:22.782858+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:35.182896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.766989+0000","last_change":"2023-10-08T01:14:21.770863+0000","last_active":"2023-10-08T01:14:22.766989+0000","last_peered":"2023-10-08T01:14:22.766989+0000","last_clean":"2023-10-08T01:14:22.766989+0000","last_became_active":"2023-10-08T01:14:21.770532+0000","last_became_peered":"2023-10-08T01:14:21.770532+0000","last_unstale":"2023-10-08T01:14:22.766989+0000","last_undegraded":"2023-10-08T01:14:22.766989+0000","last_fullsized":"2023-10-08T01:14:22.766989+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:32.319886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.787982+0000","last_change":"2023-10-08T01:14:22.781044+0000","last_active":"2023-10-08T01:14:22.787982+0000","last_peered":"2023-10-08T01:14:22.787982+0000","last_clean":"2023-10-08T01:14:22.787982+0000","last_became_active":"2023-10-08T01:14:22.780772+0000","last_became_peered":"2023-10-08T01:14:22.780772+0000","last_unstale":"2023-10-08T01:14:22.787982+0000","last_undegraded":"2023-10-08T01:14:22.787982+0000","last_fullsized":"2023-10-08T01:14:22.787982+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:21.740942+0000","last_clean_scrub_stamp":"2023-10-08T01:14:21.740942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:33:46.300139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"119'1","reported_seq":11,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.794163+0000","last_change":"2023-10-08T01:14:21.765701+0000","last_active":"2023-10-08T01:14:22.794163+0000","last_peered":"2023-10-08T01:14:22.794163+0000","last_clean":"2023-10-08T01:14:22.794163+0000","last_became_active":"2023-10-08T01:14:21.765371+0000","last_became_peered":"2023-10-08T01:14:21.765371+0000","last_unstale":"2023-10-08T01:14:22.794163+0000","last_undegraded":"2023-10-08T01:14:22.794163+0000","last_fullsized":"2023-10-08T01:14:22.794163+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:14:20.735460+0000","last_clean_scrub_stamp":"2023-10-08T01:14:20.735460+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:35.086104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"109'227","reported_seq":637,"reported_epoch":119,"state":"active+clean","last_fresh":"2023-10-08T01:14:22.765396+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:14:22.765396+0000","last_peered":"2023-10-08T01:14:22.765396+0000","last_clean":"2023-10-08T01:14:22.765396+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:14:22.765396+0000","last_undegraded":"2023-10-08T01:14:22.765396+0000","last_fullsized":"2023-10-08T01:14:22.765396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1076,"num_read_kb":2160,"num_write":359,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575070,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307516,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064324,"statfs":{"total":96636764160,"available":96321867776,"internally_reserved":0,"allocated":2084864,"data_stored":1315796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.70399999999999996},"min":{"1min":0.45600000000000002,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.872,"5min":1.123,"15min":1.123},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67100000000000004,"15min":0.70399999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.97899999999999998,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61299999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.66800000000000004,"15min":0.69399999999999995},"min":{"1min":0.51100000000000001,"5min":0.42999999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.96699999999999997,"15min":0.97699999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69399999999999995,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.39200000000000002},"max":{"1min":0.83999999999999997,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.54300000000000004}]},{"osd":2,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.70399999999999996,"15min":0.73099999999999998},"min":{"1min":0.52200000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96999999999999997,"5min":1.133,"15min":1.133},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.72299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.88300000000000001,"5min":1.167,"15min":1.167},"last":0.53200000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67100000000000004,"15min":0.70699999999999996},"min":{"1min":0.52800000000000002,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89400000000000002,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72499999999999998,"15min":0.751},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":0.91700000000000004,"5min":1.079,"15min":1.079},"last":0.59699999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56200000000000006,"15min":0.58299999999999996},"min":{"1min":0.40899999999999997,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.44500000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.77500000000000002,"5min":0.88,"15min":0.88300000000000001},"last":0.40799999999999997}]},{"osd":5,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.41899999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.80600000000000005,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.65200000000000002},"min":{"1min":0.439,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93500000000000005,"5min":1.004,"15min":1.004},"last":0.56999999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:16 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72599999999999998,"15min":0.75},"min":{"1min":0.54800000000000004,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.97599999999999998,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69599999999999995,"15min":0.69799999999999995},"min":{"1min":0.46100000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58699999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302840,"kb_used_data":1456,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069000,"statfs":{"total":96636764160,"available":96326656000,"internally_reserved":0,"allocated":1490944,"data_stored":725940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.68300000000000005,"15min":0.69999999999999996},"min":{"1min":0.54700000000000004,"5min":0.40600000000000003,"15min":0.36699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.625},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.68600000000000005,"15min":0.69499999999999995},"min":{"1min":0.54500000000000004,"5min":0.433,"15min":0.38800000000000001},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.875}]},{"osd":1,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69999999999999996,"15min":0.72599999999999998},"min":{"1min":0.48399999999999999,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":2.984},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.72299999999999998},"min":{"1min":0.59099999999999997,"5min":0.45300000000000001,"15min":0.38100000000000001},"max":{"1min":0.97099999999999997,"5min":1.018,"15min":3.0190000000000001},"last":0.65200000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67400000000000004,"15min":0.69199999999999995},"min":{"1min":0.53700000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.92400000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.64100000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71699999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.71999999999999997,"15min":0.745},"min":{"1min":0.55400000000000005,"5min":0.47599999999999998,"15min":0.47099999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.72799999999999998,"15min":0.73799999999999999},"min":{"1min":0.60899999999999999,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.94399999999999995,"5min":1.048,"15min":1.048},"last":0.64200000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.60799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.38400000000000001,"15min":0.34599999999999997},"max":{"1min":0.91100000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61799999999999999,"15min":0.621},"min":{"1min":0.46899999999999997,"5min":0.43099999999999999,"15min":0.41899999999999998},"max":{"1min":0.86199999999999999,"5min":0.89800000000000002,"15min":1.087},"last":0.79000000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.621,"15min":0.623},"min":{"1min":0.42399999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":0.84999999999999998,"5min":1.0189999999999999,"15min":1.395},"last":0.75},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.435},"max":{"1min":0.94999999999999996,"5min":1.0489999999999999,"15min":1.21},"last":0.82099999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:07 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70799999999999996,"15min":0.71099999999999997},"min":{"1min":0.51000000000000001,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93400000000000005,"5min":1.095,"15min":2.9489999999999998},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.73799999999999999,"15min":0.73499999999999999},"min":{"1min":0.52700000000000002,"5min":0.50600000000000001,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.80400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302832,"kb_used_data":1448,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069008,"statfs":{"total":96636764160,"available":96326664192,"internally_reserved":0,"allocated":1482752,"data_stored":725316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.45900000000000002,"15min":0.46700000000000003},"min":{"1min":0.32600000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.47799999999999998,"15min":0.503},"min":{"1min":0.372,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.745,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.52300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.46700000000000003,"15min":0.47999999999999998},"min":{"1min":0.377,"5min":0.317,"15min":0.26700000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.77300000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.51900000000000002,"15min":0.53500000000000003},"min":{"1min":0.45100000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.753,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.55800000000000005,"15min":0.57299999999999995},"min":{"1min":0.41999999999999998,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.76700000000000002,"5min":0.78000000000000003,"15min":0.82099999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.50600000000000001,"15min":0.53700000000000003},"min":{"1min":0.42399999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.68100000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73499999999999999,"15min":0.76300000000000001},"min":{"1min":0.56100000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":1.296,"5min":1.349,"15min":1.452},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72699999999999998,"15min":0.76200000000000001},"min":{"1min":0.55700000000000005,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.137,"5min":1.1499999999999999,"15min":1.1839999999999999},"last":0.46300000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73199999999999998,"15min":0.76400000000000001},"min":{"1min":0.59899999999999998,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":1.119,"5min":1.274,"15min":1.3959999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.73599999999999999,"15min":0.78000000000000003},"min":{"1min":0.57099999999999995,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.169,"5min":1.169,"15min":1.5209999999999999},"last":0.56599999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73799999999999999,"15min":0.78500000000000003},"min":{"1min":0.56799999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.069,"5min":1.3839999999999999,"15min":1.6020000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.74199999999999999,"15min":0.78200000000000003},"min":{"1min":0.55700000000000005,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.1499999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.621}]},{"osd":7,"last update":"Sun Oct 8 01:14:12 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77700000000000002,"15min":0.82099999999999995},"min":{"1min":0.59999999999999998,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.236,"5min":1.3109999999999999,"15min":1.621},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.78500000000000003,"15min":0.82899999999999996},"min":{"1min":0.66000000000000003,"5min":0.58099999999999996,"15min":0.51300000000000001},"max":{"1min":1.21,"5min":1.2370000000000001,"15min":1.4890000000000001},"last":0.78300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303416,"kb_used_data":2032,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068424,"statfs":{"total":96636764160,"available":96326066176,"internally_reserved":0,"allocated":2080768,"data_stored":1316238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.34300000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.625,"5min":0.95999999999999996,"15min":0.997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48699999999999999,"15min":0.496},"min":{"1min":0.33600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67700000000000005,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.33300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.46800000000000003,"15min":0.48299999999999998},"min":{"1min":0.377,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.64800000000000002,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.312},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.50900000000000001,"15min":0.53000000000000003},"min":{"1min":0.39500000000000002,"5min":0.35299999999999998,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.40899999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.53400000000000003,"15min":0.54900000000000004},"min":{"1min":0.40600000000000003,"5min":0.317,"15min":0.317},"max":{"1min":0.70599999999999996,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.59899999999999998},"min":{"1min":0.504,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72699999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.754},"min":{"1min":0.53500000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":0.94099999999999995,"5min":1.341,"15min":1.617},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.61299999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92100000000000004,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.61699999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77300000000000002,"15min":0.79100000000000004},"min":{"1min":0.59599999999999997,"5min":0.498,"15min":0.498},"max":{"1min":0.99199999999999999,"5min":2.085,"15min":2.085},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.79100000000000004,"15min":0.81699999999999995},"min":{"1min":0.56699999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0129999999999999,"5min":1.381,"15min":1.5289999999999999},"last":0.70499999999999996}]},{"osd":6,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.77100000000000002,"15min":0.80000000000000004},"min":{"1min":0.63500000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.98299999999999998,"5min":1.361,"15min":1.3779999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76100000000000001,"15min":0.78500000000000003},"min":{"1min":0.60099999999999998,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.99399999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.68899999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:08 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.77900000000000003,"15min":0.81299999999999994},"min":{"1min":0.61199999999999999,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.077,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.871},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.81399999999999995,"15min":0.83899999999999997},"min":{"1min":0.71899999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.028,"5min":1.2170000000000001,"15min":1.3160000000000001},"last":0.89400000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311028,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060812,"statfs":{"total":96636764160,"available":96318271488,"internally_reserved":0,"allocated":1486848,"data_stored":725350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.41099999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.36299999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.46000000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.88800000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.81399999999999995}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.90400000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.875}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306944,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064896,"statfs":{"total":96636764160,"available":96322453504,"internally_reserved":0,"allocated":1499136,"data_stored":726874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42799999999999999,"15min":0.46200000000000002},"min":{"1min":0.32000000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.54400000000000004,"5min":0.627,"15min":0.80100000000000005},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.47699999999999998,"15min":0.48099999999999998},"min":{"1min":0.38100000000000001,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.69099999999999995,"5min":0.77100000000000002,"15min":0.81200000000000006},"last":0.50900000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47599999999999998,"15min":0.47899999999999998},"min":{"1min":0.33600000000000002,"5min":0.32600000000000001,"15min":0.28299999999999997},"max":{"1min":0.79900000000000004,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.371},{"interface":"front","average":{"1min":0.504,"5min":0.495,"15min":0.51600000000000001},"min":{"1min":0.371,"5min":0.311,"15min":0.31},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.39100000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.58199999999999996,"15min":0.58999999999999997},"min":{"1min":0.47299999999999998,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.877,"5min":0.877,"15min":0.89700000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.622},"min":{"1min":0.5,"5min":0.438,"15min":0.39600000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71999999999999997,"15min":0.752},"min":{"1min":0.48999999999999999,"5min":0.47799999999999998,"15min":0.41099999999999998},"max":{"1min":1.028,"5min":1.095,"15min":1.3029999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.73499999999999999,"15min":0.77700000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.046,"5min":1.0580000000000001,"15min":1.413},"last":0.74299999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.73399999999999999,"15min":0.77300000000000002},"min":{"1min":0.57399999999999995,"5min":0.52500000000000002,"15min":0.51000000000000001},"max":{"1min":0.995,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.73299999999999998,"15min":0.77100000000000002},"min":{"1min":0.67500000000000004,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.089},"last":0.71599999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.751,"15min":0.79400000000000004},"min":{"1min":0.57199999999999995,"5min":0.48699999999999999,"15min":0.47999999999999998},"max":{"1min":0.94799999999999995,"5min":1.101,"15min":1.379},"last":0.877},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.72899999999999998,"15min":0.77900000000000003},"min":{"1min":0.59499999999999997,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.486},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:14:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.755,"15min":0.79300000000000004},"min":{"1min":0.63200000000000001,"5min":0.53200000000000003,"15min":0.46500000000000002},"max":{"1min":0.97799999999999998,"5min":1.0840000000000001,"15min":1.389},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.753,"15min":0.79500000000000004},"min":{"1min":0.63300000000000001,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.96699999999999997,"5min":1.0760000000000001,"15min":1.4730000000000001},"last":0.78300000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311044,"kb_used_data":1468,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060796,"statfs":{"total":96636764160,"available":96318255104,"internally_reserved":0,"allocated":1503232,"data_stored":726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.84299999999999997}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.621},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.875}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":0.67000000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.70999999999999996}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.92400000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.80100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575069,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302836,"kb_used_data":1452,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94069004,"statfs":{"total":96636764160,"available":96326660096,"internally_reserved":0,"allocated":1486848,"data_stored":725386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.68899999999999995},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.33700000000000002},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.629},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70599999999999996,"15min":0.71699999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38},"max":{"1min":0.94999999999999996,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.60199999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.69399999999999995,"15min":0.71299999999999997},"min":{"1min":0.44700000000000001,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":0.95299999999999996,"5min":1.026,"15min":1.026},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.68899999999999995,"15min":0.69699999999999995},"min":{"1min":0.53300000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.47999999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.46999999999999997,"5min":0.44900000000000001,"15min":0.38700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.73699999999999999},"min":{"1min":0.53500000000000003,"5min":0.48599999999999999,"15min":0.47399999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.64800000000000002}]},{"osd":3,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.71999999999999997,"15min":0.73299999999999998},"min":{"1min":0.58399999999999996,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.75},"min":{"1min":0.55300000000000005,"5min":0.46300000000000002,"15min":0.46200000000000002},"max":{"1min":0.98699999999999999,"5min":1.034,"15min":1.034},"last":0.56100000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58699999999999997,"15min":0.59499999999999997},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.70899999999999996,"5min":0.874,"15min":0.91900000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.65200000000000002},"min":{"1min":0.51400000000000001,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.86599999999999999,"5min":0.90300000000000002,"15min":1.0680000000000001},"last":0.753}]},{"osd":6,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.86799999999999999,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66900000000000004,"15min":0.67100000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":0.86299999999999999,"5min":0.90200000000000002,"15min":0.95699999999999996},"last":0.59099999999999997}]},{"osd":7,"last update":"Sun Oct 8 01:14:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68100000000000005,"15min":0.68700000000000006},"min":{"1min":0.49299999999999999,"5min":0.432,"15min":0.432},"max":{"1min":0.88300000000000001,"5min":0.94399999999999995,"15min":1.095},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71299999999999997,"15min":0.71799999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47399999999999998},"max":{"1min":0.90200000000000002,"5min":1.05,"15min":1.05},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:14:32.365 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T01:14:32.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T01:14:32.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.585+0000 7f99203ff700 1 Processor -- start 2023-10-08T01:14:32.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.585+0000 7f99203ff700 1 -- start start 2023-10-08T01:14:32.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.586+0000 7f99203ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f99180cca50 0x7f99180caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.586+0000 7f99203ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 0x7f991815a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.586+0000 7f99203ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f991815df40 0x7f991815e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.586+0000 7f99203ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9918157c50 con 0x7f991815df40 2023-10-08T01:14:32.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.586+0000 7f99203ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991815d7d0 con 0x7f99180cca50 2023-10-08T01:14:32.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.586+0000 7f99203ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991815aa20 con 0x7f99180cce20 2023-10-08T01:14:32.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f991815df40 0x7f991815e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f991815df40 0x7f991815e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40758/0 (socket says 172.21.15.119:40758) 2023-10-08T01:14:32.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 -- 172.21.15.119:0/1075850539 learned_addr learned my addr 172.21.15.119:0/1075850539 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:32.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e19b700 1 --2- 172.21.15.119:0/1075850539 >> v2:172.21.15.184:3300/0 conn(0x7f99180cca50 0x7f99180caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991d99a700 1 --2- 172.21.15.119:0/1075850539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 0x7f991815a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 -- 172.21.15.119:0/1075850539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 msgr2=0x7f991815a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 --2- 172.21.15.119:0/1075850539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 0x7f991815a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 -- 172.21.15.119:0/1075850539 >> v2:172.21.15.184:3300/0 conn(0x7f99180cca50 msgr2=0x7f99180caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 --2- 172.21.15.119:0/1075850539 >> v2:172.21.15.184:3300/0 conn(0x7f99180cca50 0x7f99180caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 -- 172.21.15.119:0/1075850539 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991815ad00 con 0x7f991815df40 2023-10-08T01:14:32.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.587+0000 7f991e99c700 1 --2- 172.21.15.119:0/1075850539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f991815e380 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f99180691a0 tx=0x7f9910009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2932d047694578ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:32.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f991d199700 1 -- 172.21.15.119:0/1075850539 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991000b860 con 0x7f991815df40 2023-10-08T01:14:32.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f991d199700 1 -- 172.21.15.119:0/1075850539 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991000b9c0 con 0x7f991815df40 2023-10-08T01:14:32.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f991d199700 1 -- 172.21.15.119:0/1075850539 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991000bd60 con 0x7f991815df40 2023-10-08T01:14:32.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 -- 172.21.15.119:0/1075850539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 msgr2=0x7f991815e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 --2- 172.21.15.119:0/1075850539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f991815e380 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f99180691a0 tx=0x7f9910009250 comp rx=0 tx=0).stop 2023-10-08T01:14:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 -- 172.21.15.119:0/1075850539 shutdown_connections 2023-10-08T01:14:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 --2- 172.21.15.119:0/1075850539 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f991815e380 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 --2- 172.21.15.119:0/1075850539 >> v2:172.21.15.184:3300/0 conn(0x7f99180cca50 0x7f99180caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 --2- 172.21.15.119:0/1075850539 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 0x7f991815a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 -- 172.21.15.119:0/1075850539 >> 172.21.15.119:0/1075850539 conn(0x7f99180ce5c0 msgr2=0x7f99180ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:32.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 -- 172.21.15.119:0/1075850539 shutdown_connections 2023-10-08T01:14:32.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.588+0000 7f99203ff700 1 -- 172.21.15.119:0/1075850539 wait complete. 2023-10-08T01:14:32.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.590+0000 7f99203ff700 1 Processor -- start 2023-10-08T01:14:32.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.590+0000 7f99203ff700 1 -- start start 2023-10-08T01:14:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f99203ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99180cca50 0x7f991816abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f99203ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 0x7f9918169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f99203ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f99181543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f99203ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99180c7aa0 con 0x7f991815df40 2023-10-08T01:14:32.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f99203ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991815a350 con 0x7f99180cca50 2023-10-08T01:14:32.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f99203ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991816a0a0 con 0x7f99180cce20 2023-10-08T01:14:32.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f99181543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f99181543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40774/0 (socket says 172.21.15.119:40774) 2023-10-08T01:14:32.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e99c700 1 -- 172.21.15.119:0/1237639181 learned_addr learned my addr 172.21.15.119:0/1237639181 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:32.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e19b700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99180cca50 0x7f991816abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991d99a700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 0x7f9918169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e99c700 1 -- 172.21.15.119:0/1237639181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 msgr2=0x7f9918169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e99c700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 0x7f9918169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e99c700 1 -- 172.21.15.119:0/1237639181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99180cca50 msgr2=0x7f991816abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e99c700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99180cca50 0x7f991816abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.591+0000 7f991e99c700 1 -- 172.21.15.119:0/1237639181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99100094f0 con 0x7f991815df40 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.592+0000 7f991e99c700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f99181543c0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9918069220 tx=0x7f9910017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.592+0000 7f990e7fc700 1 -- 172.21.15.119:0/1237639181 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910011ea0 con 0x7f991815df40 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.592+0000 7f990e7fc700 1 -- 172.21.15.119:0/1237639181 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991001a630 con 0x7f991815df40 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.592+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991816a380 con 0x7f991815df40 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.592+0000 7f990e7fc700 1 -- 172.21.15.119:0/1237639181 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991001aa10 con 0x7f991815df40 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.592+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991816a940 con 0x7f991815df40 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.593+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99180c7900 con 0x7f991815df40 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.594+0000 7f990e7fc700 1 -- 172.21.15.119:0/1237639181 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f991002d440 con 0x7f991815df40 2023-10-08T01:14:32.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.595+0000 7f990e7fc700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98fc066f30 0x7f98fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:32.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.595+0000 7f990e7fc700 1 -- 172.21.15.119:0/1237639181 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f9910086310 con 0x7f991815df40 2023-10-08T01:14:32.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.595+0000 7f991e19b700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98fc066f30 0x7f98fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:32.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.596+0000 7f991e19b700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98fc066f30 0x7f98fc0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99180cc650 tx=0x7f99140097e0 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:32.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.599+0000 7f990e7fc700 1 -- 172.21.15.119:0/1237639181 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f99180c7900 con 0x7f991815df40 2023-10-08T01:14:32.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.869+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f99180c7900 con 0x7f991815df40 2023-10-08T01:14:32.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.873+0000 7f990e7fc700 1 -- 172.21.15.119:0/1237639181 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7f99180c7900 con 0x7f991815df40 2023-10-08T01:14:32.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98fc066f30 msgr2=0x7f98fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98fc066f30 0x7f98fc0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f99180cc650 tx=0x7f99140097e0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 msgr2=0x7f99181543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:32.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f99181543c0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9918069220 tx=0x7f9910017000 comp rx=0 tx=0).stop 2023-10-08T01:14:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 shutdown_connections 2023-10-08T01:14:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f98fc066f30 0x7f98fc0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f991815df40 0x7f99181543c0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f99180cca50 0x7f991816abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 --2- 172.21.15.119:0/1237639181 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f99180cce20 0x7f9918169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 >> 172.21.15.119:0/1237639181 conn(0x7f99180ce5c0 msgr2=0x7f9918155000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 shutdown_connections 2023-10-08T01:14:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:32.876+0000 7f99203ff700 1 -- 172.21.15.119:0/1237639181 wait complete. 2023-10-08T01:14:32.879 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T01:14:32.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:14:33.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.078+0000 7fe214fff700 1 Processor -- start 2023-10-08T01:14:33.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.079+0000 7fe214fff700 1 -- start start 2023-10-08T01:14:33.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe214fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2100b1850 0x7fe2100b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe214fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2100ae680 0x7fe2101572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe214fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2100cb300 con 0x7fe2100aead0 2023-10-08T01:14:33.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe214fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21006fc30 con 0x7fe2100b1850 2023-10-08T01:14:33.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe214fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100cc260 con 0x7fe2100ae680 2023-10-08T01:14:33.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe20ed9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2100aead0 0x7fe21015b3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59744/0 (socket says 172.21.15.119:59744) 2023-10-08T01:14:33.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe20ed9d700 1 -- 172.21.15.119:0/3072129839 learned_addr learned my addr 172.21.15.119:0/3072129839 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:33.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe20e59c700 1 --2- 172.21.15.119:0/3072129839 >> v2:172.21.15.184:3300/0 conn(0x7fe2100b1850 0x7fe2100b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe207fff700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2100ae680 0x7fe2101572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777006238 0 0) 0x7fe2100cb300 con 0x7fe2100aead0 2023-10-08T01:14:33.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.080+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1fc003570 con 0x7fe2100aead0 2023-10-08T01:14:33.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536111940 0 0) 0x7fe1fc003570 con 0x7fe2100aead0 2023-10-08T01:14:33.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2100ae680 msgr2=0x7fe2101572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2100ae680 0x7fe2101572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 >> v2:172.21.15.184:3300/0 conn(0x7fe2100b1850 msgr2=0x7fe2100b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 --2- 172.21.15.119:0/3072129839 >> v2:172.21.15.184:3300/0 conn(0x7fe2100b1850 0x7fe2100b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100cb5e0 con 0x7fe2100aead0 2023-10-08T01:14:33.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fe2080017f0 con 0x7fe2100aead0 2023-10-08T01:14:33.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 >> v1:172.21.15.119:6789/0 conn(0x7fe2100aead0 legacy=0x7fe21015b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc003ae0 0x7fe1fc003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1fc004810 0x7fe1fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2100ae680 0x7fe1fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21006fc30 con 0x7fe1fc004810 2023-10-08T01:14:33.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100cc260 con 0x7fe1fc003ae0 2023-10-08T01:14:33.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc00bb10 con 0x7fe2100ae680 2023-10-08T01:14:33.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe207fff700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1fc004810 0x7fe1fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20e59c700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc003ae0 0x7fe1fc003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.081+0000 7fe20ed9d700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2100ae680 0x7fe1fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe207fff700 1 -- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2100ae680 msgr2=0x7fe1fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe207fff700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2100ae680 0x7fe1fc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe207fff700 1 -- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc003ae0 msgr2=0x7fe1fc003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe207fff700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc003ae0 0x7fe1fc003ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe207fff700 1 -- 172.21.15.119:0/3072129839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc003570 con 0x7fe1fc004810 2023-10-08T01:14:33.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe207fff700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1fc004810 0x7fe1fc004c80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009c00 tx=0x7fe1f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd0278958672974 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800e710 con 0x7fe1fc004810 2023-10-08T01:14:33.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f800e870 con 0x7fe1fc004810 2023-10-08T01:14:33.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe20dd9b700 1 -- 172.21.15.119:0/3072129839 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f800ec30 con 0x7fe1fc004810 2023-10-08T01:14:33.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe214fff700 1 -- 172.21.15.119:0/3072129839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1fc004810 msgr2=0x7fe1fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.082+0000 7fe214fff700 1 --2- 172.21.15.119:0/3072129839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe1fc004810 0x7fe1fc004c80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8009c00 tx=0x7fe1f80091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.083+0000 7fe20ed9d700 1 -- 172.21.15.119:0/3072129839 reap_dead start 2023-10-08T01:14:33.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.083+0000 7fe214fff700 1 -- 172.21.15.119:0/3072129839 shutdown_connections 2023-10-08T01:14:33.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.083+0000 7fe214fff700 1 -- 172.21.15.119:0/3072129839 >> 172.21.15.119:0/3072129839 conn(0x7fe21006a320 msgr2=0x7fe2100cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:33.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.083+0000 7fe214fff700 1 -- 172.21.15.119:0/3072129839 shutdown_connections 2023-10-08T01:14:33.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.083+0000 7fe214fff700 1 -- 172.21.15.119:0/3072129839 wait complete. 2023-10-08T01:14:33.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.084+0000 7fe214fff700 1 Processor -- start 2023-10-08T01:14:33.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.084+0000 7fe214fff700 1 -- start start 2023-10-08T01:14:33.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe214fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc004810 0x7fe210151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe214fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2100c3550 0x7fe2100c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe214fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe210154640 0x7fe210154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe214fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe210151550 con 0x7fe2100c3550 2023-10-08T01:14:33.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe214fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21013e840 con 0x7fe1fc004810 2023-10-08T01:14:33.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe214fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2100c3f00 con 0x7fe210154640 2023-10-08T01:14:33.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe207fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2100c3550 0x7fe2100c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe207fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2100c3550 0x7fe2100c39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40796/0 (socket says 172.21.15.119:40796) 2023-10-08T01:14:33.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.085+0000 7fe207fff700 1 -- 172.21.15.119:0/3292808189 learned_addr learned my addr 172.21.15.119:0/3292808189 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:33.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe20e59c700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc004810 0x7fe210151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe207fff700 1 -- 172.21.15.119:0/3292808189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe210154640 msgr2=0x7fe210154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe207fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe210154640 0x7fe210154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe207fff700 1 -- 172.21.15.119:0/3292808189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc004810 msgr2=0x7fe210151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe207fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc004810 0x7fe210151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe207fff700 1 -- 172.21.15.119:0/3292808189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2101556e0 con 0x7fe2100c3550 2023-10-08T01:14:33.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe20e59c700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc004810 0x7fe210151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe207fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2100c3550 0x7fe2100c39c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8013f20 tx=0x7fe1f801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe2067fc700 1 -- 172.21.15.119:0/3292808189 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8022070 con 0x7fe2100c3550 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe2067fc700 1 -- 172.21.15.119:0/3292808189 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f800e9e0 con 0x7fe2100c3550 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe2067fc700 1 -- 172.21.15.119:0/3292808189 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f8027470 con 0x7fe2100c3550 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2100c1a50 con 0x7fe2100c3550 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.086+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2100c1400 con 0x7fe2100c3550 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.087+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2100c1be0 con 0x7fe2100c3550 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.088+0000 7fe2067fc700 1 -- 172.21.15.119:0/3292808189 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe1f801c450 con 0x7fe2100c3550 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.089+0000 7fe2067fc700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe1f0066f30 0x7fe1f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.089+0000 7fe2067fc700 1 -- 172.21.15.119:0/3292808189 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe1f80861b0 con 0x7fe2100c3550 2023-10-08T01:14:33.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.089+0000 7fe20e59c700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe1f0066f30 0x7fe1f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.090+0000 7fe20e59c700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe1f0066f30 0x7fe1f00693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2000093f0 tx=0x7fe200008040 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.093+0000 7fe2067fc700 1 -- 172.21.15.119:0/3292808189 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe1f8053290 con 0x7fe2100c3550 2023-10-08T01:14:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.331+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe21004c3c0 con 0x7fe2100c3550 2023-10-08T01:14:33.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.332+0000 7fe2067fc700 1 -- 172.21.15.119:0/3292808189 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4608 (secure 0 0 0) 0x7fe1f80568b0 con 0x7fe2100c3550 2023-10-08T01:14:33.333 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:33.333 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7141,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3118853373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3118853373},{"type":"v1","addr":"172.21.15.184:6834","nonce":3118853373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7144,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/737543181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":737543181},{"type":"v1","addr":"172.21.15.184:6835","nonce":737543181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/2667608886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2667608886},{"type":"v1","addr":"172.21.15.119:6834","nonce":2667608886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:14:22.219941+0000","modified":"2023-10-08T01:14:25.788077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7680},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7680":{"gid":7680,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1713175169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1713175169},{"type":"v1","addr":"172.21.15.119:6835","nonce":1713175169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T01:14:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe1f0066f30 msgr2=0x7fe1f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe1f0066f30 0x7fe1f00693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe2000093f0 tx=0x7fe200008040 comp rx=0 tx=0).stop 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2100c3550 msgr2=0x7fe2100c39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2100c3550 0x7fe2100c39c0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8013f20 tx=0x7fe1f801a000 comp rx=0 tx=0).stop 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 shutdown_connections 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe1f0066f30 0x7fe1f00693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe210154640 0x7fe210154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe1fc004810 0x7fe210151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 --2- 172.21.15.119:0/3292808189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2100c3550 0x7fe2100c39c0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 >> 172.21.15.119:0/3292808189 conn(0x7fe21006a320 msgr2=0x7fe2100ade70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.334+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 shutdown_connections 2023-10-08T01:14:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.335+0000 7fe214fff700 1 -- 172.21.15.119:0/3292808189 wait complete. 2023-10-08T01:14:33.337 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 55 2023-10-08T01:14:33.755 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T01:14:22.219941+0000', 'modified': '2023-10-08T01:14:25.788077+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7680}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7680': {'gid': 7680, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6835/1713175169', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6833', 'nonce': 1713175169}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 1713175169}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T01:14:33.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T01:14:33.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T01:14:33.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.938+0000 7f0cf4f9d700 1 Processor -- start 2023-10-08T01:14:33.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.938+0000 7f0cf4f9d700 1 -- start start 2023-10-08T01:14:33.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0cf4f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0159420 0x7f0cf00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0cf4f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0cf01597f0 0x7f0cf00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0cf4f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0cf015d230 0x7f0cf015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0cf4f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf015dbd0 con 0x7f0cf015d230 2023-10-08T01:14:33.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0cf4f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf00b11d0 con 0x7f0cf01597f0 2023-10-08T01:14:33.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0cf4f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf00b14b0 con 0x7f0cf0159420 2023-10-08T01:14:33.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0ceffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0cf015d230 0x7f0cf015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0ceffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0cf015d230 0x7f0cf015d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40798/0 (socket says 172.21.15.119:40798) 2023-10-08T01:14:33.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.939+0000 7f0ceffff700 1 -- 172.21.15.119:0/1021444719 learned_addr learned my addr 172.21.15.119:0/1021444719 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:33.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/1021444719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0159420 0x7f0cf00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/1021444719 >> v2:172.21.15.184:3300/0 conn(0x7f0cf01597f0 0x7f0cf00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0ceffff700 1 -- 172.21.15.119:0/1021444719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0159420 msgr2=0x7f0cf00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0ceffff700 1 --2- 172.21.15.119:0/1021444719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0159420 0x7f0cf00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0ceffff700 1 -- 172.21.15.119:0/1021444719 >> v2:172.21.15.184:3300/0 conn(0x7f0cf01597f0 msgr2=0x7f0cf00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0ceffff700 1 --2- 172.21.15.119:0/1021444719 >> v2:172.21.15.184:3300/0 conn(0x7f0cf01597f0 0x7f0cf00b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0ceffff700 1 -- 172.21.15.119:0/1021444719 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf00b1790 con 0x7f0cf015d230 2023-10-08T01:14:33.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/1021444719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0159420 0x7f0cf00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:33.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0ceffff700 1 --2- 172.21.15.119:0/1021444719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf015d230 0x7f0cf015d690 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00d1770 tx=0x7f0ce8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8133087df61afcd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0cee7fc700 1 -- 172.21.15.119:0/1021444719 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce800e070 con 0x7f0cf015d230 2023-10-08T01:14:33.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0cee7fc700 1 -- 172.21.15.119:0/1021444719 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce800ba50 con 0x7f0cf015d230 2023-10-08T01:14:33.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.940+0000 7f0cee7fc700 1 -- 172.21.15.119:0/1021444719 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80092e0 con 0x7f0cf015d230 2023-10-08T01:14:33.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/1021444719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf015d230 msgr2=0x7f0cf015d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/1021444719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf015d230 0x7f0cf015d690 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00d1770 tx=0x7f0ce8009840 comp rx=0 tx=0).stop 2023-10-08T01:14:33.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/1021444719 shutdown_connections 2023-10-08T01:14:33.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/1021444719 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf015d230 0x7f0cf015d690 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/1021444719 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0159420 0x7f0cf00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/1021444719 >> v2:172.21.15.184:3300/0 conn(0x7f0cf01597f0 0x7f0cf00b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/1021444719 >> 172.21.15.119:0/1021444719 conn(0x7f0cf00d0660 msgr2=0x7f0cf00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:33.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/1021444719 shutdown_connections 2023-10-08T01:14:33.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.941+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/1021444719 wait complete. 2023-10-08T01:14:33.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.942+0000 7f0cf4f9d700 1 Processor -- start 2023-10-08T01:14:33.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.943+0000 7f0cf4f9d700 1 -- start start 2023-10-08T01:14:33.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.943+0000 7f0cf4f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf0159420 0x7f0cf00c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.943+0000 7f0cf4f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf01597f0 0x7f0cf00c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.943+0000 7f0cf4f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf015d230 0x7f0cf01544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.943+0000 7f0cf4f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf016a940 con 0x7f0cf01597f0 2023-10-08T01:14:33.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.943+0000 7f0cf4f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf00c8730 con 0x7f0cf0159420 2023-10-08T01:14:33.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.943+0000 7f0cf4f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf015c000 con 0x7f0cf015d230 2023-10-08T01:14:33.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf01597f0 0x7f0cf00c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf01597f0 0x7f0cf00c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40806/0 (socket says 172.21.15.119:40806) 2023-10-08T01:14:33.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 -- 172.21.15.119:0/3635156455 learned_addr learned my addr 172.21.15.119:0/3635156455 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:33.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceffff700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf015d230 0x7f0cf01544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf0159420 0x7f0cf00c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 -- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf015d230 msgr2=0x7f0cf01544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf015d230 0x7f0cf01544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 -- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf0159420 msgr2=0x7f0cf00c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf0159420 0x7f0cf00c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 -- 172.21.15.119:0/3635156455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce8009ae0 con 0x7f0cf01597f0 2023-10-08T01:14:33.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceffff700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf015d230 0x7f0cf01544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:33.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.944+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf01597f0 0x7f0cf00c81f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400d890 tx=0x7f0ce4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.945+0000 7f0cd77fe700 1 -- 172.21.15.119:0/3635156455 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400b600 con 0x7f0cf01597f0 2023-10-08T01:14:33.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.945+0000 7f0cd77fe700 1 -- 172.21.15.119:0/3635156455 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce400b760 con 0x7f0cf01597f0 2023-10-08T01:14:33.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.945+0000 7f0cd77fe700 1 -- 172.21.15.119:0/3635156455 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce400bae0 con 0x7f0cf01597f0 2023-10-08T01:14:33.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.945+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/3635156455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf015c2e0 con 0x7f0cf01597f0 2023-10-08T01:14:33.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.945+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/3635156455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf016ab90 con 0x7f0cf01597f0 2023-10-08T01:14:33.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.948+0000 7f0cd77fe700 1 -- 172.21.15.119:0/3635156455 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ce401e460 con 0x7f0cf01597f0 2023-10-08T01:14:33.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.948+0000 7f0cd77fe700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.948+0000 7f0cd77fe700 1 -- 172.21.15.119:0/3635156455 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f0ce4084e10 con 0x7f0cf01597f0 2023-10-08T01:14:33.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.949+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.949+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000ac00 tx=0x7f0ce0006f90 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.950+0000 7f0d04970b80 1 Processor -- start 2023-10-08T01:14:33.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.950+0000 7f0d04970b80 1 -- start start 2023-10-08T01:14:33.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x56256297b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x56256299c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562951ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0d04970b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625628a8a70 con 0x562562951b80 2023-10-08T01:14:33.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0d04970b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625628a6270 con 0x56256299c480 2023-10-08T01:14:33.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0d04970b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625628a2060 con 0x56256294f5f0 2023-10-08T01:14:33.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0ceffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562951ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0ceffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562951ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40820/0 (socket says 172.21.15.119:40820) 2023-10-08T01:14:33.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.951+0000 7f0ceffff700 1 -- 172.21.15.119:0/3998494663 learned_addr learned my addr 172.21.15.119:0/3998494663 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:33.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x56256299c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x56256297b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0ceffff700 1 -- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 msgr2=0x56256299c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0ceffff700 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x56256299c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0ceffff700 1 -- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 msgr2=0x56256297b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0ceffff700 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x56256297b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0ceffff700 1 -- 172.21.15.119:0/3998494663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56256273c270 con 0x562562951b80 2023-10-08T01:14:33.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x56256297b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:33.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0ceffff700 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562951ff0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009fd0 tx=0x7f0ce8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106c93dccf0d2762 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0cd4ff9700 1 -- 172.21.15.119:0/3998494663 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce80072a0 con 0x562562951b80 2023-10-08T01:14:33.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0cd4ff9700 1 -- 172.21.15.119:0/3998494663 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce800bbc0 con 0x562562951b80 2023-10-08T01:14:33.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.952+0000 7f0cd4ff9700 1 -- 172.21.15.119:0/3998494663 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8007610 con 0x562562951b80 2023-10-08T01:14:33.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 -- 172.21.15.119:0/3998494663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 msgr2=0x562562951ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562951ff0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009fd0 tx=0x7f0ce8018000 comp rx=0 tx=0).stop 2023-10-08T01:14:33.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 -- 172.21.15.119:0/3998494663 shutdown_connections 2023-10-08T01:14:33.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562951ff0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x56256297b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3998494663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x56256299c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 -- 172.21.15.119:0/3998494663 >> 172.21.15.119:0/3998494663 conn(0x5625629530c0 msgr2=0x5625629538f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:33.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 -- 172.21.15.119:0/3998494663 shutdown_connections 2023-10-08T01:14:33.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.953+0000 7f0d04970b80 1 -- 172.21.15.119:0/3998494663 wait complete. 2023-10-08T01:14:33.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.954+0000 7f0d04970b80 1 -- start start 2023-10-08T01:14:33.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.954+0000 7f0d04970b80 1 Processor -- start 2023-10-08T01:14:33.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x56256297cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562562951b80 0x5625629814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256299c480 0x562562981a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0d04970b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256284d890 con 0x56256299c480 2023-10-08T01:14:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0d04970b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562868ea0 con 0x562562951b80 2023-10-08T01:14:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0d04970b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x562562724e30 con 0x56256294f5f0 2023-10-08T01:14:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0d04970b80 10 client.0 _ll_register_callbacks cb 0x5625629152c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:14:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0ceffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256299c480 0x562562981a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0ceffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256299c480 0x562562981a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40826/0 (socket says 172.21.15.119:40826) 2023-10-08T01:14:33.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.955+0000 7f0ceffff700 1 -- 172.21.15.119:0/3955661768 learned_addr learned my addr 172.21.15.119:0/3955661768 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:33.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ceffff700 1 -- 172.21.15.119:0/3955661768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 msgr2=0x56256297cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ceffff700 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x56256297cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ceffff700 1 -- 172.21.15.119:0/3955661768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562562951b80 msgr2=0x5625629814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ceffff700 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562562951b80 0x5625629814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ceffff700 1 -- 172.21.15.119:0/3955661768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf00b1790 con 0x56256299c480 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ceffff700 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256299c480 0x562562981a10 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8007530 tx=0x7f0ce801f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf87c5885756c49c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ccdffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3955661768 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8007bd0 con 0x56256299c480 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0d04970b80 1 -- 172.21.15.119:0/3955661768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562562946cd0 con 0x56256299c480 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.956+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3955661768 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f0ce8007d30 con 0x56256299c480 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3955661768 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce8025630 con 0x56256299c480 2023-10-08T01:14:33.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0d04970b80 10 client.7851 fetch_fsmap learned FSMap version 55 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0d04970b80 1 -- 172.21.15.119:0/3955661768 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x5625626defe0 con 0x56256299c480 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3955661768 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562562946cd0 con 0x56256299c480 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3955661768 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 55) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0ce800b050 con 0x56256299c480 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0d04970b80 10 client.7851 fetch_fsmap finished waiting for FSMap version 55 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0d04970b80 10 client.7851 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0d04970b80 20 client.7851 populate_metadata read hostname 'smithi119' 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] conn(0x56256297bca0 0x56256297c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0d04970b80 4 client.7851 mds_command: new command op to 7680 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T01:14:33.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0d04970b80 1 -- 172.21.15.119:0/3955661768 --> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562562923aa0 con 0x56256297bca0 2023-10-08T01:14:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.957+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] conn(0x56256297bca0 0x56256297c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.958+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] conn(0x56256297bca0 0x56256297c070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cdd82cfd1f9f80a server_cookie=b038c2d27c1cca4c in_seq=0 out_seq=0 2023-10-08T01:14:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.958+0000 7f0ccdffb700 10 client.7851 ms_handle_connect on v2:172.21.15.119:6833/1713175169 2023-10-08T01:14:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.958+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3955661768 <== mds.0 v2:172.21.15.119:6833/1713175169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x562562923aa0 con 0x56256297bca0 2023-10-08T01:14:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.958+0000 7f0ccdffb700 10 client.7851 handle_command_reply: tid=0 2023-10-08T01:14:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 client.7851 shutdown 2023-10-08T01:14:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 -- 172.21.15.119:0/3955661768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256299c480 msgr2=0x562562981a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256299c480 0x562562981a10 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8007530 tx=0x7f0ce801f070 comp rx=0 tx=0).stop 2023-10-08T01:14:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 -- 172.21.15.119:0/3955661768 shutdown_connections 2023-10-08T01:14:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] conn(0x56256297bca0 0x56256297c070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0ccdffb700 0 client.7851 ms_handle_reset on v2:172.21.15.119:6833/1713175169 2023-10-08T01:14:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x56256297cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256299c480 0x562562981a10 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3955661768 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562562951b80 0x5625629814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 -- 172.21.15.119:0/3955661768 >> 172.21.15.119:0/3955661768 conn(0x5625629530c0 msgr2=0x562562951f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 -- 172.21.15.119:0/3955661768 shutdown_connections 2023-10-08T01:14:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 1 -- 172.21.15.119:0/3955661768 wait complete. 2023-10-08T01:14:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.959+0000 7f0d04970b80 20 client.7851 trim_cache size 0 max 16384 2023-10-08T01:14:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.976+0000 7f0d04970b80 1 Processor -- start 2023-10-08T01:14:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.976+0000 7f0d04970b80 1 -- start start 2023-10-08T01:14:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x562562981800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562947f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x562562953660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0d04970b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562562975b20 con 0x562562951b80 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0d04970b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56256299bae0 con 0x56256299c480 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0d04970b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x562562951f50 con 0x56256294f5f0 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0ceeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562947f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0ceeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562947f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40842/0 (socket says 172.21.15.119:40842) 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0ceeffd700 1 -- 172.21.15.119:0/891157453 learned_addr learned my addr 172.21.15.119:0/891157453 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0ceffff700 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x562562953660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.977+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x562562981800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0ceeffd700 1 -- 172.21.15.119:0/891157453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 msgr2=0x562562981800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x562562981800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0ceeffd700 1 -- 172.21.15.119:0/891157453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 msgr2=0x562562953660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x562562953660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0ceeffd700 1 -- 172.21.15.119:0/891157453 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562562870780 con 0x562562951b80 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562947f40 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4083190 tx=0x7f0ce4020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e41b08edc8c714 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0cd57fa700 1 -- 172.21.15.119:0/891157453 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce406d030 con 0x562562951b80 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0cd57fa700 1 -- 172.21.15.119:0/891157453 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce4055470 con 0x562562951b80 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0cd57fa700 1 -- 172.21.15.119:0/891157453 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce4058d60 con 0x562562951b80 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0d04970b80 1 -- 172.21.15.119:0/891157453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 msgr2=0x562562947f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.978+0000 7f0d04970b80 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562947f40 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4083190 tx=0x7f0ce4020040 comp rx=0 tx=0).stop 2023-10-08T01:14:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 -- 172.21.15.119:0/891157453 shutdown_connections 2023-10-08T01:14:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256294f5f0 0x562562981800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56256299c480 0x562562953660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 --2- 172.21.15.119:0/891157453 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562562951b80 0x562562947f40 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 -- 172.21.15.119:0/891157453 >> 172.21.15.119:0/891157453 conn(0x5625629530c0 msgr2=0x56256297fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 -- 172.21.15.119:0/891157453 shutdown_connections 2023-10-08T01:14:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 -- 172.21.15.119:0/891157453 wait complete. 2023-10-08T01:14:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 -- start start 2023-10-08T01:14:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.979+0000 7f0d04970b80 1 Processor -- start 2023-10-08T01:14:33.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.980+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256294f5f0 0x562562a00b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.980+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562562951b80 0x56256294de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.980+0000 7f0d04970b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256299c480 0x5625629b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.980+0000 7f0d04970b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625629fdb00 con 0x56256294f5f0 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.980+0000 7f0d04970b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625629ff0c0 con 0x562562951b80 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.980+0000 7f0d04970b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5625629fdd00 con 0x56256299c480 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.980+0000 7f0d04970b80 10 client.0 _ll_register_callbacks cb 0x5625629152c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256294f5f0 0x562562a00b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256294f5f0 0x562562a00b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40850/0 (socket says 172.21.15.119:40850) 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 -- 172.21.15.119:0/3761283149 learned_addr learned my addr 172.21.15.119:0/3761283149 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 -- 172.21.15.119:0/3761283149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256299c480 msgr2=0x5625629b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256299c480 0x5625629b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 -- 172.21.15.119:0/3761283149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562562951b80 msgr2=0x56256294de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562562951b80 0x56256294de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 -- 172.21.15.119:0/3761283149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf016ab90 con 0x56256294f5f0 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256294f5f0 0x562562a00b10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x562562982060 tx=0x7f0ce000a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c71266abf71039 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0ccdffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3761283149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce0026e90 con 0x56256294f5f0 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3761283149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f0ce0014460 con 0x56256294f5f0 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.981+0000 7f0d04970b80 1 -- 172.21.15.119:0/3761283149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cf015b970 con 0x56256294f5f0 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3761283149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce0014830 con 0x56256294f5f0 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0d04970b80 10 client.7857 fetch_fsmap learned FSMap version 55 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0d04970b80 1 -- 172.21.15.119:0/3761283149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) v3 -- 0x562562857470 con 0x56256294f5f0 2023-10-08T01:14:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3761283149 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf015b970 con 0x56256294f5f0 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3761283149 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 55) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f0ce0014be0 con 0x56256294f5f0 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0d04970b80 10 client.7857 fetch_fsmap finished waiting for FSMap version 55 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0d04970b80 10 client.7857 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0d04970b80 20 client.7857 populate_metadata read hostname 'smithi119' 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] conn(0x56256297a420 0x56256297a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0d04970b80 4 client.7857 mds_command: new command op to 7680 tid=0[{"prefix": "status"}] 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0d04970b80 1 -- 172.21.15.119:0/3761283149 --> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f0ce800bbc0 con 0x56256297a420 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.982+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] conn(0x56256297a420 0x56256297a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] conn(0x56256297a420 0x56256297a810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d11627eec6f2dd5 server_cookie=bf82370c7dd220be in_seq=0 out_seq=0 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0ccdffb700 10 client.7857 ms_handle_connect on v2:172.21.15.119:6833/1713175169 2023-10-08T01:14:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0ccdffb700 1 -- 172.21.15.119:0/3761283149 <== mds.0 v2:172.21.15.119:6833/1713175169 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f0ce800bbc0 con 0x56256297a420 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0ccdffb700 10 client.7857 handle_command_reply: tid=0 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 client.7857 shutdown 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 -- 172.21.15.119:0/3761283149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256294f5f0 msgr2=0x562562a00b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256294f5f0 0x562562a00b10 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x562562982060 tx=0x7f0ce000a040 comp rx=0 tx=0).stop 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 -- 172.21.15.119:0/3761283149 shutdown_connections 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.119:6833/1713175169,v1:172.21.15.119:6835/1713175169] conn(0x56256297a420 0x56256297a810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56256299c480 0x5625629b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56256294f5f0 0x562562a00b10 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 --2- 172.21.15.119:0/3761283149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x562562951b80 0x56256294de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0ccdffb700 0 client.7857 ms_handle_reset on v2:172.21.15.119:6833/1713175169 2023-10-08T01:14:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 -- 172.21.15.119:0/3761283149 >> 172.21.15.119:0/3761283149 conn(0x5625629530c0 msgr2=0x562562947f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:33.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 -- 172.21.15.119:0/3761283149 shutdown_connections 2023-10-08T01:14:33.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 1 -- 172.21.15.119:0/3761283149 wait complete. 2023-10-08T01:14:33.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.983+0000 7f0d04970b80 20 client.7857 trim_cache size 0 max 16384 2023-10-08T01:14:33.991 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:14:33.991 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T01:14:33.991 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T01:14:33.991 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7680, 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 11.224748557, 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 55, 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 120, 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 120, 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 16.999535869999999 2023-10-08T01:14:33.992 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:14:33.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 msgr2=0x7f0cdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0ce000ac00 tx=0x7f0ce0006f90 comp rx=0 tx=0).stop 2023-10-08T01:14:33.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf01597f0 msgr2=0x7f0cf00c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:33.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf01597f0 0x7f0cf00c81f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f0ce400d890 tx=0x7f0ce4008040 comp rx=0 tx=0).stop 2023-10-08T01:14:33.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/3635156455 shutdown_connections 2023-10-08T01:14:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf015d230 0x7f0cf01544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf0159420 0x7f0cf00c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 --2- 172.21.15.119:0/3635156455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf01597f0 0x7f0cf00c81f0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/3635156455 >> 172.21.15.119:0/3635156455 conn(0x7f0cf00d0660 msgr2=0x7f0cf0069050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/3635156455 shutdown_connections 2023-10-08T01:14:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:33.984+0000 7f0cf4f9d700 1 -- 172.21.15.119:0/3635156455 wait complete. 2023-10-08T01:14:34.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T01:14:34.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.182+0000 7fe84099d700 1 Processor -- start 2023-10-08T01:14:34.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.183+0000 7fe84099d700 1 -- start start 2023-10-08T01:14:34.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe84099d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe83c06d950 0x7fe83c0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:34.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe84099d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06a0a0 0x7fe83c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:34.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe84099d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c06ae70 con 0x7fe83c06dd20 2023-10-08T01:14:34.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe84099d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c06a540 con 0x7fe83c06d950 2023-10-08T01:14:34.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe84099d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c06a820 con 0x7fe83c06a0a0 2023-10-08T01:14:34.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe83bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe83c06dd20 0x7fe83c0c7150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59756/0 (socket says 172.21.15.119:59756) 2023-10-08T01:14:34.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe83bfff700 1 -- 172.21.15.119:0/783632530 learned_addr learned my addr 172.21.15.119:0/783632530 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:34.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe83b7fe700 1 --2- 172.21.15.119:0/783632530 >> v2:172.21.15.184:3300/0 conn(0x7fe83c06d950 0x7fe83c0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.184+0000 7fe833fff700 1 --2- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06a0a0 0x7fe83c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.185+0000 7fe83affd700 1 -- 172.21.15.119:0/783632530 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152474821 0 0) 0x7fe83c06ae70 con 0x7fe83c06dd20 2023-10-08T01:14:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.185+0000 7fe83affd700 1 -- 172.21.15.119:0/783632530 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe828003570 con 0x7fe83c06dd20 2023-10-08T01:14:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.185+0000 7fe83b7fe700 1 -- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06a0a0 msgr2=0x7fe83c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.185+0000 7fe83b7fe700 1 --2- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06a0a0 0x7fe83c0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.185+0000 7fe83b7fe700 1 -- 172.21.15.119:0/783632530 >> v1:172.21.15.119:6789/0 conn(0x7fe83c06dd20 legacy=0x7fe83c0c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.185+0000 7fe83b7fe700 1 -- 172.21.15.119:0/783632530 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c06ab00 con 0x7fe83c06d950 2023-10-08T01:14:34.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.185+0000 7fe833fff700 1 --2- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06a0a0 0x7fe83c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:34.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.185+0000 7fe83b7fe700 1 --2- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c06d950 0x7fe83c0c8350 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0098e0 tx=0x7fe82c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fd1bc1625e15b4d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:34.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe83affd700 1 -- 172.21.15.119:0/783632530 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c011630 con 0x7fe83c06d950 2023-10-08T01:14:34.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe83affd700 1 -- 172.21.15.119:0/783632530 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c011790 con 0x7fe83c06d950 2023-10-08T01:14:34.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe83affd700 1 -- 172.21.15.119:0/783632530 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c011b10 con 0x7fe83c06d950 2023-10-08T01:14:34.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe84099d700 1 -- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c06d950 msgr2=0x7fe83c0c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:34.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe84099d700 1 --2- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c06d950 0x7fe83c0c8350 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0098e0 tx=0x7fe82c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe84099d700 1 -- 172.21.15.119:0/783632530 shutdown_connections 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe84099d700 1 --2- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c06d950 0x7fe83c0c8350 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe84099d700 1 --2- 172.21.15.119:0/783632530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06a0a0 0x7fe83c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe84099d700 1 -- 172.21.15.119:0/783632530 >> 172.21.15.119:0/783632530 conn(0x7fe83c0cc1f0 msgr2=0x7fe83c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe84099d700 1 -- 172.21.15.119:0/783632530 shutdown_connections 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.186+0000 7fe84099d700 1 -- 172.21.15.119:0/783632530 wait complete. 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.188+0000 7fe84099d700 1 Processor -- start 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.188+0000 7fe84099d700 1 -- start start 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe84099d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe83c06a0a0 0x7fe83c144db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe84099d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06d950 0x7fe83c150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe84099d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c06dd20 0x7fe83c152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:34.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe84099d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c151c20 con 0x7fe83c06a0a0 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe84099d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c152eb0 con 0x7fe83c06dd20 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe84099d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c151380 con 0x7fe83c06d950 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe83c06a0a0 0x7fe83c144db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe83c06a0a0 0x7fe83c144db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40866/0 (socket says 172.21.15.119:40866) 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 -- 172.21.15.119:0/3482448792 learned_addr learned my addr 172.21.15.119:0/3482448792 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 -- 172.21.15.119:0/3482448792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06d950 msgr2=0x7fe83c150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06d950 0x7fe83c150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 -- 172.21.15.119:0/3482448792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c06dd20 msgr2=0x7fe83c152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c06dd20 0x7fe83c152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 -- 172.21.15.119:0/3482448792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe824009420 con 0x7fe83c06a0a0 2023-10-08T01:14:34.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.189+0000 7fe83b7fe700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe83c06a0a0 0x7fe83c144db0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe83c06fe40 tx=0x7fe82c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:34.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.190+0000 7fe8337fe700 1 -- 172.21.15.119:0/3482448792 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c011b10 con 0x7fe83c06a0a0 2023-10-08T01:14:34.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.190+0000 7fe8337fe700 1 -- 172.21.15.119:0/3482448792 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c000cc0 con 0x7fe83c06a0a0 2023-10-08T01:14:34.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.190+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c009420 con 0x7fe83c06a0a0 2023-10-08T01:14:34.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.190+0000 7fe8337fe700 1 -- 172.21.15.119:0/3482448792 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c01a680 con 0x7fe83c06a0a0 2023-10-08T01:14:34.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.190+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c151940 con 0x7fe83c06a0a0 2023-10-08T01:14:34.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.191+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c04c3c0 con 0x7fe83c06a0a0 2023-10-08T01:14:34.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.192+0000 7fe8337fe700 1 -- 172.21.15.119:0/3482448792 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe82c01aa70 con 0x7fe83c06a0a0 2023-10-08T01:14:34.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.192+0000 7fe8337fe700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe81c066f30 0x7fe81c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:34.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.193+0000 7fe8337fe700 1 -- 172.21.15.119:0/3482448792 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe82c0863e0 con 0x7fe83c06a0a0 2023-10-08T01:14:34.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.193+0000 7fe833fff700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe81c066f30 0x7fe81c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:34.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.193+0000 7fe833fff700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe81c066f30 0x7fe81c0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0b1da0 tx=0x7fe8240091a0 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:34.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.197+0000 7fe8337fe700 1 -- 172.21.15.119:0/3482448792 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe82c0534c0 con 0x7fe83c06a0a0 2023-10-08T01:14:34.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.431+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe83c069e80 con 0x7fe83c06a0a0 2023-10-08T01:14:34.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.432+0000 7fe8337fe700 1 -- 172.21.15.119:0/3482448792 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe82c056ae0 con 0x7fe83c06a0a0 2023-10-08T01:14:34.433 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe81c066f30 msgr2=0x7fe81c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe81c066f30 0x7fe81c0693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0b1da0 tx=0x7fe8240091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe83c06a0a0 msgr2=0x7fe83c144db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe83c06a0a0 0x7fe83c144db0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fe83c06fe40 tx=0x7fe82c017000 comp rx=0 tx=0).stop 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 shutdown_connections 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe81c066f30 0x7fe81c0693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe83c06dd20 0x7fe83c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe83c06a0a0 0x7fe83c144db0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 --2- 172.21.15.119:0/3482448792 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe83c06d950 0x7fe83c150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:34.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 >> 172.21.15.119:0/3482448792 conn(0x7fe83c0cc1f0 msgr2=0x7fe83c0c2960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:34.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.434+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 shutdown_connections 2023-10-08T01:14:34.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:34.435+0000 7fe84099d700 1 -- 172.21.15.119:0/3482448792 wait complete. 2023-10-08T01:14:34.453 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T01:14:34.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T01:14:34.454 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T01:14:34.454 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T01:14:34.454 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T01:14:34.454 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T01:14:34.454 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T01:14:34.454 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T01:14:34.454 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:14:34.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:14:34.455 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T01:14:34.491 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T01:14:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T01:14:34.491 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T01:14:34.492 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41177sec preferred_lft 41177sec 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:14:34.493 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T01:14:34.494 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:14:34.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:14:34.495 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T01:14:34.590 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T01:14:34.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:14:34.593 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T01:14:34.631 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T01:14:34.631 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T01:14:34.631 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T01:14:34.632 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T01:14:34.632 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T01:14:34.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:14:34.633 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T01:14:34.633 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T01:14:34.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T01:14:34.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T01:14:34.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T01:14:34.633 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T01:14:34.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:14:34.828 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T01:14:34.864 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T01:14:34.864 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T01:14:34.864 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:14:34.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:14:34.864 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T01:14:34.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:14:34.967 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T01:14:35.003 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:14:35.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:14:35.004 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:14:35.078 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T01:14:35.079 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T01:14:35.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:14:35.208 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:14:35.209 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:14:35.237 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:14:35.237 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:14:35.238 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T01:14:35.238 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T01:14:35.238 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T01:14:35.243 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T01:14:35.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:14:35.244 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:14:35.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T01:14:35.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.486+0000 7f2b76dff700 1 Processor -- start 2023-10-08T01:14:35.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.486+0000 7f2b76dff700 1 -- start start 2023-10-08T01:14:35.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b76dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b70155c10 0x7f2b70155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b76dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2b700ac6a0 0x7f2b700b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b76dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b700acac0 con 0x7f2b700ac230 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b76dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b700ad930 con 0x7f2b700ac6a0 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b76dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b700acd20 con 0x7f2b70155c10 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b6ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2b700ac230 0x7f2b701568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34600/0 (socket says 172.21.15.119:34600) 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b6ffff700 1 -- 172.21.15.119:0/436831584 learned_addr learned my addr 172.21.15.119:0/436831584 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b74b9b700 1 --2- 172.21.15.119:0/436831584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b70155c10 0x7f2b70155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b7539c700 1 --2- 172.21.15.119:0/436831584 >> v2:172.21.15.184:3300/0 conn(0x7f2b700ac6a0 0x7f2b700b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.488+0000 7f2b6f7fe700 1 -- 172.21.15.119:0/436831584 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310522280 0 0) 0x7f2b700acac0 con 0x7f2b700ac230 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b6f7fe700 1 -- 172.21.15.119:0/436831584 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b60003570 con 0x7f2b700ac230 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b74b9b700 1 -- 172.21.15.119:0/436831584 >> v2:172.21.15.184:3300/0 conn(0x7f2b700ac6a0 msgr2=0x7f2b700b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b74b9b700 1 --2- 172.21.15.119:0/436831584 >> v2:172.21.15.184:3300/0 conn(0x7f2b700ac6a0 0x7f2b700b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b74b9b700 1 -- 172.21.15.119:0/436831584 >> v1:172.21.15.119:6789/0 conn(0x7f2b700ac230 legacy=0x7f2b701568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b74b9b700 1 -- 172.21.15.119:0/436831584 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b700ad000 con 0x7f2b70155c10 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b74b9b700 1 --2- 172.21.15.119:0/436831584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b70155c10 0x7f2b70155fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b640098e0 tx=0x7f2b640091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728e6cca7ed8cfc9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b6f7fe700 1 -- 172.21.15.119:0/436831584 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64011630 con 0x7f2b70155c10 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b6f7fe700 1 -- 172.21.15.119:0/436831584 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b64011790 con 0x7f2b70155c10 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.489+0000 7f2b6f7fe700 1 -- 172.21.15.119:0/436831584 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64011b30 con 0x7f2b70155c10 2023-10-08T01:14:35.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.490+0000 7f2b76dff700 1 -- 172.21.15.119:0/436831584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b70155c10 msgr2=0x7f2b70155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.490+0000 7f2b76dff700 1 --2- 172.21.15.119:0/436831584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b70155c10 0x7f2b70155fe0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2b640098e0 tx=0x7f2b640091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.490+0000 7f2b76dff700 1 -- 172.21.15.119:0/436831584 shutdown_connections 2023-10-08T01:14:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.490+0000 7f2b76dff700 1 --2- 172.21.15.119:0/436831584 >> v2:172.21.15.184:3300/0 conn(0x7f2b700ac6a0 0x7f2b700b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.490+0000 7f2b76dff700 1 --2- 172.21.15.119:0/436831584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b70155c10 0x7f2b70155fe0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.490+0000 7f2b76dff700 1 -- 172.21.15.119:0/436831584 >> 172.21.15.119:0/436831584 conn(0x7f2b7006a320 msgr2=0x7f2b700cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.490+0000 7f2b76dff700 1 -- 172.21.15.119:0/436831584 shutdown_connections 2023-10-08T01:14:35.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.490+0000 7f2b76dff700 1 -- 172.21.15.119:0/436831584 wait complete. 2023-10-08T01:14:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.491+0000 7f2b76dff700 1 Processor -- start 2023-10-08T01:14:35.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.492+0000 7f2b76dff700 1 -- start start 2023-10-08T01:14:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.492+0000 7f2b76dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b700ac230 0x7f2b701583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.492+0000 7f2b76dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700ac6a0 0x7f2b7015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.492+0000 7f2b76dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b70155c10 0x7f2b7015da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b76dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7015c980 con 0x7f2b70155c10 2023-10-08T01:14:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b76dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7015cc60 con 0x7f2b700ac230 2023-10-08T01:14:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b76dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7015cf40 con 0x7f2b700ac6a0 2023-10-08T01:14:35.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b70155c10 0x7f2b7015da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b70155c10 0x7f2b7015da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41042/0 (socket says 172.21.15.119:41042) 2023-10-08T01:14:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 -- 172.21.15.119:0/991338964 learned_addr learned my addr 172.21.15.119:0/991338964 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 -- 172.21.15.119:0/991338964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700ac6a0 msgr2=0x7f2b7015afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700ac6a0 0x7f2b7015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 -- 172.21.15.119:0/991338964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b700ac230 msgr2=0x7f2b701583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b700ac230 0x7f2b701583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 -- 172.21.15.119:0/991338964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b64009420 con 0x7f2b70155c10 2023-10-08T01:14:35.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b7539c700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b70155c10 0x7f2b7015da20 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009f10 tx=0x7f2b6800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b6cff9700 1 -- 172.21.15.119:0/991338964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68007cb0 con 0x7f2b70155c10 2023-10-08T01:14:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.493+0000 7f2b6cff9700 1 -- 172.21.15.119:0/991338964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b68010040 con 0x7f2b70155c10 2023-10-08T01:14:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.494+0000 7f2b6cff9700 1 -- 172.21.15.119:0/991338964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b68014640 con 0x7f2b70155c10 2023-10-08T01:14:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.494+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7015d220 con 0x7f2b70155c10 2023-10-08T01:14:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.494+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7015efc0 con 0x7f2b70155c10 2023-10-08T01:14:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.495+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7015d740 con 0x7f2b70155c10 2023-10-08T01:14:35.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.496+0000 7f2b6cff9700 1 -- 172.21.15.119:0/991338964 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2b68014a30 con 0x7f2b70155c10 2023-10-08T01:14:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.497+0000 7f2b6cff9700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b54066f30 0x7f2b540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.497+0000 7f2b6cff9700 1 -- 172.21.15.119:0/991338964 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f2b68084f40 con 0x7f2b70155c10 2023-10-08T01:14:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.497+0000 7f2b74b9b700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b54066f30 0x7f2b540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.497+0000 7f2b74b9b700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b54066f30 0x7f2b540693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2b640091e0 tx=0x7f2b64017000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:35.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.500+0000 7f2b6cff9700 1 -- 172.21.15.119:0/991338964 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2b7015d740 con 0x7f2b70155c10 2023-10-08T01:14:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.716+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b7015d740 con 0x7f2b54066f30 2023-10-08T01:14:35.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.718+0000 7f2b6cff9700 1 -- 172.21.15.119:0/991338964 <== mgr.7488 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2b7015d740 con 0x7f2b54066f30 2023-10-08T01:14:35.719 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:14:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:14:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T01:14:35.720 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:14:35.720 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b54066f30 msgr2=0x7f2b540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b54066f30 0x7f2b540693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2b640091e0 tx=0x7f2b64017000 comp rx=0 tx=0).stop 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b70155c10 msgr2=0x7f2b7015da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b70155c10 0x7f2b7015da20 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009f10 tx=0x7f2b6800a890 comp rx=0 tx=0).stop 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 shutdown_connections 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b54066f30 0x7f2b540693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b70155c10 0x7f2b7015da20 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b700ac230 0x7f2b701583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 --2- 172.21.15.119:0/991338964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b700ac6a0 0x7f2b7015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.720+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 >> 172.21.15.119:0/991338964 conn(0x7f2b7006a320 msgr2=0x7f2b70154810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.721+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 shutdown_connections 2023-10-08T01:14:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.721+0000 7f2b76dff700 1 -- 172.21.15.119:0/991338964 wait complete. 2023-10-08T01:14:35.739 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T01:14:35.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.920+0000 7f2fed5ff700 1 Processor -- start 2023-10-08T01:14:35.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.920+0000 7f2fed5ff700 1 -- start start 2023-10-08T01:14:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.921+0000 7f2fed5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2fe8069e80 0x7f2fe80c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.921+0000 7f2fed5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe80af2f0 con 0x7f2fe80b0a70 2023-10-08T01:14:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fed5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe80af5d0 con 0x7f2fe815ac80 2023-10-08T01:14:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fed5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe80c8660 con 0x7f2fe8069e80 2023-10-08T01:14:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fe67fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2fe8069e80 0x7f2fe80c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fe77fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2fe80b0a70 0x7f2fe80c70b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34604/0 (socket says 172.21.15.119:34604) 2023-10-08T01:14:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fe77fe700 1 -- 172.21.15.119:0/991606120 learned_addr learned my addr 172.21.15.119:0/991606120 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fe5ffb700 1 -- 172.21.15.119:0/991606120 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684614750 0 0) 0x7f2fe80af2f0 con 0x7f2fe80b0a70 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fe67fc700 1 -- 172.21.15.119:0/991606120 >> v1:172.21.15.184:6789/0 conn(0x7f2fe815ac80 legacy=0x7f2fe8069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fe67fc700 1 -- 172.21.15.119:0/991606120 >> v1:172.21.15.119:6789/0 conn(0x7f2fe80b0a70 legacy=0x7f2fe80c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.922+0000 7f2fe67fc700 1 -- 172.21.15.119:0/991606120 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe80c8940 con 0x7f2fe8069e80 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fe67fc700 1 --2- 172.21.15.119:0/991606120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80c6b70 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0098e0 tx=0x7f2fdc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8074aa5954e71808 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fe5ffb700 1 -- 172.21.15.119:0/991606120 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc009e80 con 0x7f2fe8069e80 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fe5ffb700 1 -- 172.21.15.119:0/991606120 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fdc00b860 con 0x7f2fe8069e80 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fe5ffb700 1 -- 172.21.15.119:0/991606120 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fdc00bbe0 con 0x7f2fe8069e80 2023-10-08T01:14:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fed5ff700 1 -- 172.21.15.119:0/991606120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 msgr2=0x7f2fe80c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fed5ff700 1 --2- 172.21.15.119:0/991606120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80c6b70 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0098e0 tx=0x7f2fdc0091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fed5ff700 1 -- 172.21.15.119:0/991606120 shutdown_connections 2023-10-08T01:14:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fed5ff700 1 --2- 172.21.15.119:0/991606120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80c6b70 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fed5ff700 1 -- 172.21.15.119:0/991606120 >> 172.21.15.119:0/991606120 conn(0x7f2fe8068e10 msgr2=0x7f2fe80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.923+0000 7f2fed5ff700 1 -- 172.21.15.119:0/991606120 shutdown_connections 2023-10-08T01:14:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.924+0000 7f2fed5ff700 1 -- 172.21.15.119:0/991606120 wait complete. 2023-10-08T01:14:35.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.925+0000 7f2fed5ff700 1 Processor -- start 2023-10-08T01:14:35.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.925+0000 7f2fed5ff700 1 -- start start 2023-10-08T01:14:35.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fed5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80aaf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fed5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fe80b0a70 0x7f2fe815b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fed5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fe815ac80 0x7f2fe8155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fed5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe815b730 con 0x7f2fe8069e80 2023-10-08T01:14:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fed5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe816e180 con 0x7f2fe80b0a70 2023-10-08T01:14:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fed5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe816dcf0 con 0x7f2fe815ac80 2023-10-08T01:14:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fe6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80aaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fe6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80aaf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41066/0 (socket says 172.21.15.119:41066) 2023-10-08T01:14:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fe6ffd700 1 -- 172.21.15.119:0/509197501 learned_addr learned my addr 172.21.15.119:0/509197501 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fe67fc700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fe80b0a70 0x7f2fe815b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.926+0000 7f2fe77fe700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fe815ac80 0x7f2fe8155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fe6ffd700 1 -- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fe815ac80 msgr2=0x7f2fe8155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fe6ffd700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fe815ac80 0x7f2fe8155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fe6ffd700 1 -- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fe80b0a70 msgr2=0x7f2fe815b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fe6ffd700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fe80b0a70 0x7f2fe815b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fe6ffd700 1 -- 172.21.15.119:0/509197501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc009420 con 0x7f2fe8069e80 2023-10-08T01:14:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fe77fe700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fe815ac80 0x7f2fe8155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fe67fc700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fe80b0a70 0x7f2fe815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:14:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fe6ffd700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80aaf20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c4e0 tx=0x7f2fd800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fceffd700 1 -- 172.21.15.119:0/509197501 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8011590 con 0x7f2fe8069e80 2023-10-08T01:14:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fceffd700 1 -- 172.21.15.119:0/509197501 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd8011710 con 0x7f2fe8069e80 2023-10-08T01:14:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe816d3c0 con 0x7f2fe8069e80 2023-10-08T01:14:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fceffd700 1 -- 172.21.15.119:0/509197501 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd8011ad0 con 0x7f2fe8069e80 2023-10-08T01:14:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.927+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe816d980 con 0x7f2fe8069e80 2023-10-08T01:14:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.928+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe816dfd0 con 0x7f2fe8069e80 2023-10-08T01:14:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.929+0000 7f2fceffd700 1 -- 172.21.15.119:0/509197501 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 58) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2fd801b3b0 con 0x7f2fe8069e80 2023-10-08T01:14:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.930+0000 7f2fceffd700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fd4066f30 0x7f2fd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.930+0000 7f2fceffd700 1 -- 172.21.15.119:0/509197501 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f2fd8083be0 con 0x7f2fe8069e80 2023-10-08T01:14:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.930+0000 7f2fe67fc700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fd4066f30 0x7f2fd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.931+0000 7f2fe67fc700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fd4066f30 0x7f2fd40693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0098e0 tx=0x7f2fdc017000 comp rx=0 tx=0).ready entity=mgr.7488 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:35.934+0000 7f2fceffd700 1 -- 172.21.15.119:0/509197501 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2fd8050c20 con 0x7f2fe8069e80 2023-10-08T01:14:36.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.174+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f2fe804c3c0 con 0x7f2fe8069e80 2023-10-08T01:14:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.249+0000 7f2fceffd700 1 -- 172.21.15.119:0/509197501 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 59) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f2fd8050ff0 con 0x7f2fe8069e80 2023-10-08T01:14:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.251+0000 7f2fceffd700 1 -- 172.21.15.119:0/509197501 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v59) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2fd8011e40 con 0x7f2fe8069e80 2023-10-08T01:14:36.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fd4066f30 msgr2=0x7f2fd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fd4066f30 0x7f2fd40693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0098e0 tx=0x7f2fdc017000 comp rx=0 tx=0).stop 2023-10-08T01:14:36.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 msgr2=0x7f2fe80aaf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80aaf20 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2fd800c4e0 tx=0x7f2fd800c8a0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 shutdown_connections 2023-10-08T01:14:36.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2fd4066f30 0x7f2fd40693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2fe815ac80 0x7f2fe8155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fe8069e80 0x7f2fe80aaf20 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 --2- 172.21.15.119:0/509197501 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2fe80b0a70 0x7f2fe815b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 >> 172.21.15.119:0/509197501 conn(0x7f2fe8068e10 msgr2=0x7f2fe8159f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:36.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 shutdown_connections 2023-10-08T01:14:36.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.253+0000 7f2fed5ff700 1 -- 172.21.15.119:0/509197501 wait complete. 2023-10-08T01:14:36.274 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T01:14:36.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.477+0000 7ffaf999d700 1 Processor -- start 2023-10-08T01:14:36.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.477+0000 7ffaf999d700 1 -- start start 2023-10-08T01:14:36.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.478+0000 7ffaf999d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffaf415a3b0 0x7ffaf415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.478+0000 7ffaf999d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffaf406e470 0x7ffaf406e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.478+0000 7ffaf999d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf40aea70 con 0x7ffaf40aef00 2023-10-08T01:14:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.478+0000 7ffaf999d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf406ee30 con 0x7ffaf415a3b0 2023-10-08T01:14:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.478+0000 7ffaf999d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf415dad0 con 0x7ffaf406e470 2023-10-08T01:14:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf899b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffaf415a3b0 0x7ffaf415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffaf40aef00 0x7ffaf415d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34630/0 (socket says 172.21.15.119:34630) 2023-10-08T01:14:36.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf3fff700 1 -- 172.21.15.119:0/1950470519 learned_addr learned my addr 172.21.15.119:0/1950470519 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf919c700 1 --2- 172.21.15.119:0/1950470519 >> v2:172.21.15.184:3300/0 conn(0x7ffaf406e470 0x7ffaf406e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf37fe700 1 -- 172.21.15.119:0/1950470519 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 664931668 0 0) 0x7ffaf40aea70 con 0x7ffaf40aef00 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf37fe700 1 -- 172.21.15.119:0/1950470519 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae4003570 con 0x7ffaf40aef00 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf899b700 1 -- 172.21.15.119:0/1950470519 >> v2:172.21.15.184:3300/0 conn(0x7ffaf406e470 msgr2=0x7ffaf406e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf899b700 1 --2- 172.21.15.119:0/1950470519 >> v2:172.21.15.184:3300/0 conn(0x7ffaf406e470 0x7ffaf406e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf899b700 1 -- 172.21.15.119:0/1950470519 >> v1:172.21.15.119:6789/0 conn(0x7ffaf40aef00 legacy=0x7ffaf415d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.479+0000 7ffaf899b700 1 -- 172.21.15.119:0/1950470519 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf40ae480 con 0x7ffaf415a3b0 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf899b700 1 --2- 172.21.15.119:0/1950470519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf415a780 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40c8660 tx=0x7ffae80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eef6868f3a457ec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:36.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf37fe700 1 -- 172.21.15.119:0/1950470519 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011630 con 0x7ffaf415a3b0 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf37fe700 1 -- 172.21.15.119:0/1950470519 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae80117b0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf37fe700 1 -- 172.21.15.119:0/1950470519 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8011b30 con 0x7ffaf415a3b0 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf999d700 1 -- 172.21.15.119:0/1950470519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 msgr2=0x7ffaf415a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf999d700 1 --2- 172.21.15.119:0/1950470519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf415a780 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ffaf40c8660 tx=0x7ffae80091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf999d700 1 -- 172.21.15.119:0/1950470519 shutdown_connections 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf999d700 1 --2- 172.21.15.119:0/1950470519 >> v2:172.21.15.184:3300/0 conn(0x7ffaf406e470 0x7ffaf406e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf999d700 1 --2- 172.21.15.119:0/1950470519 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf415a780 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.480+0000 7ffaf999d700 1 -- 172.21.15.119:0/1950470519 >> 172.21.15.119:0/1950470519 conn(0x7ffaf40c9f70 msgr2=0x7ffaf406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.481+0000 7ffaf999d700 1 -- 172.21.15.119:0/1950470519 shutdown_connections 2023-10-08T01:14:36.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.481+0000 7ffaf999d700 1 -- 172.21.15.119:0/1950470519 wait complete. 2023-10-08T01:14:36.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.482+0000 7ffaf999d700 1 Processor -- start 2023-10-08T01:14:36.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.482+0000 7ffaf999d700 1 -- start start 2023-10-08T01:14:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf999d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf406e470 0x7ffaf4151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf999d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf40aef00 0x7ffaf41594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf999d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf40c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf999d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf40b13e0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf999d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf40b16c0 con 0x7ffaf40aef00 2023-10-08T01:14:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf999d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf40b19a0 con 0x7ffaf406e470 2023-10-08T01:14:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf919c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf40c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf919c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf40c5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41098/0 (socket says 172.21.15.119:41098) 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf919c700 1 -- 172.21.15.119:0/2973954448 learned_addr learned my addr 172.21.15.119:0/2973954448 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf3fff700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf40aef00 0x7ffaf41594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.483+0000 7ffaf899b700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf406e470 0x7ffaf4151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf919c700 1 -- 172.21.15.119:0/2973954448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf406e470 msgr2=0x7ffaf4151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf919c700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf406e470 0x7ffaf4151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf919c700 1 -- 172.21.15.119:0/2973954448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf40aef00 msgr2=0x7ffaf41594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf919c700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf40aef00 0x7ffaf41594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf919c700 1 -- 172.21.15.119:0/2973954448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae8009420 con 0x7ffaf415a3b0 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf919c700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf40c5c30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009df0 tx=0x7ffaec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf0ff9700 1 -- 172.21.15.119:0/2973954448 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00caa0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf0ff9700 1 -- 172.21.15.119:0/2973954448 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaec00f040 con 0x7ffaf415a3b0 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf0ff9700 1 -- 172.21.15.119:0/2973954448 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaec00ce70 con 0x7ffaf415a3b0 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf40b1c80 con 0x7ffaf415a3b0 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.484+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf41521e0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.485+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf40b21a0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.486+0000 7ffaf0ff9700 1 -- 172.21.15.119:0/2973954448 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 59) v1 ==== 84757+0+0 (secure 0 0 0) 0x7ffaec014690 con 0x7ffaf415a3b0 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.487+0000 7ffaf0ff9700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffad8066f80 0x7ffad8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.488+0000 7ffaf899b700 1 -- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffad8066f80 msgr2=0x7ffad8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.488+0000 7ffaf0ff9700 1 -- 172.21.15.119:0/2973954448 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7ffaec0852e0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.488+0000 7ffaf899b700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffad8066f80 0x7ffad8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.492+0000 7ffaf0ff9700 1 -- 172.21.15.119:0/2973954448 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffaf40b21a0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.689+0000 7ffaf899b700 1 -- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffad8066f80 msgr2=0x7ffad8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:36.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.689+0000 7ffaf899b700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffad8066f80 0x7ffad8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.691+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ffaf40b21a0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.696+0000 7ffaf0ff9700 1 -- 172.21.15.119:0/2973954448 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7ffaf40b21a0 con 0x7ffaf415a3b0 2023-10-08T01:14:36.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffad8066f80 msgr2=0x7ffad8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:36.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffad8066f80 0x7ffad8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 msgr2=0x7ffaf40c5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf40c5c30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ffaec009df0 tx=0x7ffaec00b040 comp rx=0 tx=0).stop 2023-10-08T01:14:36.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 shutdown_connections 2023-10-08T01:14:36.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ffad8066f80 0x7ffad8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffaf406e470 0x7ffaf4151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaf415a3b0 0x7ffaf40c5c30 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 --2- 172.21.15.119:0/2973954448 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffaf40aef00 0x7ffaf41594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 >> 172.21.15.119:0/2973954448 conn(0x7ffaf40c9f70 msgr2=0x7ffaf40c8840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:36.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 shutdown_connections 2023-10-08T01:14:36.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.698+0000 7ffaf999d700 1 -- 172.21.15.119:0/2973954448 wait complete. 2023-10-08T01:14:36.719 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T01:14:36.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.908+0000 7fb63519d700 1 Processor -- start 2023-10-08T01:14:36.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.908+0000 7fb63519d700 1 -- start start 2023-10-08T01:14:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.909+0000 7fb63519d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb6300aef00 0x7fb63015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.909+0000 7fb63519d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb63006e470 0x7fb63006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.909+0000 7fb63519d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6300aea70 con 0x7fb63015a3b0 2023-10-08T01:14:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.909+0000 7fb63519d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63006ee30 con 0x7fb6300aef00 2023-10-08T01:14:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.909+0000 7fb63519d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6300ae480 con 0x7fb63006e470 2023-10-08T01:14:36.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb6300aef00 0x7fb63015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb63015a3b0 0x7fb63015a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34642/0 (socket says 172.21.15.119:34642) 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62ffff700 1 -- 172.21.15.119:0/2959298424 learned_addr learned my addr 172.21.15.119:0/2959298424 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb63499c700 1 --2- 172.21.15.119:0/2959298424 >> v2:172.21.15.184:3300/0 conn(0x7fb63006e470 0x7fb63006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62effd700 1 -- 172.21.15.119:0/2959298424 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183585745 0 0) 0x7fb6300aea70 con 0x7fb63015a3b0 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62f7fe700 1 -- 172.21.15.119:0/2959298424 >> v2:172.21.15.184:3300/0 conn(0x7fb63006e470 msgr2=0x7fb63006e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62f7fe700 1 --2- 172.21.15.119:0/2959298424 >> v2:172.21.15.184:3300/0 conn(0x7fb63006e470 0x7fb63006e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62f7fe700 1 -- 172.21.15.119:0/2959298424 >> v1:172.21.15.119:6789/0 conn(0x7fb63015a3b0 legacy=0x7fb63015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62f7fe700 1 -- 172.21.15.119:0/2959298424 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63015dd30 con 0x7fb6300aef00 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb63499c700 1 --2- 172.21.15.119:0/2959298424 >> v2:172.21.15.184:3300/0 conn(0x7fb63006e470 0x7fb63006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.910+0000 7fb62f7fe700 1 --2- 172.21.15.119:0/2959298424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63015d3c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb6300c8660 tx=0x7fb6240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ccde3f9af2d3377 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:36.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb62effd700 1 -- 172.21.15.119:0/2959298424 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62400b6b0 con 0x7fb6300aef00 2023-10-08T01:14:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb62effd700 1 -- 172.21.15.119:0/2959298424 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62400b810 con 0x7fb6300aef00 2023-10-08T01:14:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb62effd700 1 -- 172.21.15.119:0/2959298424 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62400bbb0 con 0x7fb6300aef00 2023-10-08T01:14:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb63519d700 1 -- 172.21.15.119:0/2959298424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 msgr2=0x7fb63015d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb63519d700 1 --2- 172.21.15.119:0/2959298424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63015d3c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fb6300c8660 tx=0x7fb6240091a0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb63519d700 1 -- 172.21.15.119:0/2959298424 shutdown_connections 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb63519d700 1 --2- 172.21.15.119:0/2959298424 >> v2:172.21.15.184:3300/0 conn(0x7fb63006e470 0x7fb63006e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb63519d700 1 --2- 172.21.15.119:0/2959298424 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63015d3c0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb63519d700 1 -- 172.21.15.119:0/2959298424 >> 172.21.15.119:0/2959298424 conn(0x7fb6300c9f70 msgr2=0x7fb63006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb63519d700 1 -- 172.21.15.119:0/2959298424 shutdown_connections 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.911+0000 7fb63519d700 1 -- 172.21.15.119:0/2959298424 wait complete. 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.913+0000 7fb63519d700 1 Processor -- start 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.913+0000 7fb63519d700 1 -- start start 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.914+0000 7fb63519d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb63006e470 0x7fb6300b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.914+0000 7fb63519d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63016da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.914+0000 7fb63519d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb63015a3b0 0x7fb630158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.914+0000 7fb63519d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63016df40 con 0x7fb6300aef00 2023-10-08T01:14:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.914+0000 7fb63519d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb630156020 con 0x7fb63015a3b0 2023-10-08T01:14:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.914+0000 7fb63519d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6300b1b50 con 0x7fb63006e470 2023-10-08T01:14:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63016da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb63006e470 0x7fb6300b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63016da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41112/0 (socket says 172.21.15.119:41112) 2023-10-08T01:14:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 -- 172.21.15.119:0/3921553190 learned_addr learned my addr 172.21.15.119:0/3921553190 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb63499c700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb63015a3b0 0x7fb630158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 -- 172.21.15.119:0/3921553190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb63006e470 msgr2=0x7fb6300b0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb63006e470 0x7fb6300b0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 -- 172.21.15.119:0/3921553190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb63015a3b0 msgr2=0x7fb630158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb63015a3b0 0x7fb630158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 -- 172.21.15.119:0/3921553190 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb624009420 con 0x7fb6300aef00 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb62f7fe700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63016da00 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb630064280 tx=0x7fb624017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.915+0000 7fb617fff700 1 -- 172.21.15.119:0/3921553190 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624000d10 con 0x7fb6300aef00 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.916+0000 7fb617fff700 1 -- 172.21.15.119:0/3921553190 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62400bb90 con 0x7fb6300aef00 2023-10-08T01:14:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.916+0000 7fb617fff700 1 -- 172.21.15.119:0/3921553190 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624011650 con 0x7fb6300aef00 2023-10-08T01:14:36.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.916+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6300b0f40 con 0x7fb6300aef00 2023-10-08T01:14:36.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.916+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6300b1500 con 0x7fb6300aef00 2023-10-08T01:14:36.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.917+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6300b1cc0 con 0x7fb6300aef00 2023-10-08T01:14:36.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.918+0000 7fb617fff700 1 -- 172.21.15.119:0/3921553190 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 59) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fb624011a00 con 0x7fb6300aef00 2023-10-08T01:14:36.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.918+0000 7fb617fff700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb61c066f80 0x7fb61c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.918+0000 7fb617fff700 1 -- 172.21.15.119:0/3921553190 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fb624085ed0 con 0x7fb6300aef00 2023-10-08T01:14:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.918+0000 7fb62ffff700 1 -- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb61c066f80 msgr2=0x7fb61c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.919+0000 7fb62ffff700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb61c066f80 0x7fb61c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:36.923+0000 7fb617fff700 1 -- 172.21.15.119:0/3921553190 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb624052fb0 con 0x7fb6300aef00 2023-10-08T01:14:37.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.120+0000 7fb62ffff700 1 -- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb61c066f80 msgr2=0x7fb61c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:37.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.120+0000 7fb62ffff700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb61c066f80 0x7fb61c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:37.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.123+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fb63015e080 con 0x7fb6300aef00 2023-10-08T01:14:37.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.128+0000 7fb617fff700 1 -- 172.21.15.119:0/3921553190 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb624011cb0 con 0x7fb6300aef00 2023-10-08T01:14:37.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb61c066f80 msgr2=0x7fb61c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:37.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb61c066f80 0x7fb61c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 msgr2=0x7fb63016da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63016da00 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fb630064280 tx=0x7fb624017000 comp rx=0 tx=0).stop 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 shutdown_connections 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb61c066f80 0x7fb61c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb63006e470 0x7fb6300b0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb63015a3b0 0x7fb630158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 --2- 172.21.15.119:0/3921553190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6300aef00 0x7fb63016da00 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 >> 172.21.15.119:0/3921553190 conn(0x7fb6300c9f70 msgr2=0x7fb630152620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 shutdown_connections 2023-10-08T01:14:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.131+0000 7fb63519d700 1 -- 172.21.15.119:0/3921553190 wait complete. 2023-10-08T01:14:37.151 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:37.151 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T01:14:37.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-08T01:14:37.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.422+0000 7f302299d700 1 Processor -- start 2023-10-08T01:14:37.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.423+0000 7f302299d700 1 -- start start 2023-10-08T01:14:37.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302299d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b03c0 0x7f301c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:37.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302299d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f301c1591e0 0x7f301c159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:37.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302299d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f301c0cfa00 con 0x7f301c0b0790 2023-10-08T01:14:37.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302299d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301c0cb580 con 0x7f301c1591e0 2023-10-08T01:14:37.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302299d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0cf160 con 0x7f301c0b03c0 2023-10-08T01:14:37.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302219c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f301c1591e0 0x7f301c159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:37.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302199b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b03c0 0x7f301c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:37.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302199b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b03c0 0x7f301c0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:33180/0 (socket says 172.21.15.119:33180) 2023-10-08T01:14:37.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302219c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f301c1591e0 0x7f301c159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:48250/0 (socket says 172.21.15.119:48250) 2023-10-08T01:14:37.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302199b700 1 -- 172.21.15.119:0/2971946932 learned_addr learned my addr 172.21.15.119:0/2971946932 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:37.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.424+0000 7f302119a700 1 --1- 172.21.15.119:0/2971946932 >> v1:172.21.15.184:6789/0 conn(0x7f301c0b0790 0x7f301c0b28a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:36740/0 (socket says 172.21.15.119:36740) 2023-10-08T01:14:37.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f302199b700 1 -- 172.21.15.119:0/2971946932 >> v2:172.21.15.184:3300/0 conn(0x7f301c1591e0 msgr2=0x7f301c159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f302199b700 1 --2- 172.21.15.119:0/2971946932 >> v2:172.21.15.184:3300/0 conn(0x7f301c1591e0 0x7f301c159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f302199b700 1 -- 172.21.15.119:0/2971946932 >> v1:172.21.15.184:6789/0 conn(0x7f301c0b0790 legacy=0x7f301c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f302199b700 1 -- 172.21.15.119:0/2971946932 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f301c0cf440 con 0x7f301c0b03c0 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f302219c700 1 --2- 172.21.15.119:0/2971946932 >> v2:172.21.15.184:3300/0 conn(0x7f301c1591e0 0x7f301c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f302199b700 1 --2- 172.21.15.119:0/2971946932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b03c0 0x7f301c0cea70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30180099b0 tx=0x7f3018009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36f8fdeb4de9a443 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f3020999700 1 -- 172.21.15.119:0/2971946932 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018009ea0 con 0x7f301c0b03c0 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f3020999700 1 -- 172.21.15.119:0/2971946932 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301800b860 con 0x7f301c0b03c0 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.425+0000 7f3020999700 1 -- 172.21.15.119:0/2971946932 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301800bc00 con 0x7f301c0b03c0 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.426+0000 7f302299d700 1 -- 172.21.15.119:0/2971946932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b03c0 msgr2=0x7f301c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:37.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.426+0000 7f302299d700 1 --2- 172.21.15.119:0/2971946932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b03c0 0x7f301c0cea70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30180099b0 tx=0x7f3018009250 comp rx=0 tx=0).stop 2023-10-08T01:14:37.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.426+0000 7f302299d700 1 -- 172.21.15.119:0/2971946932 shutdown_connections 2023-10-08T01:14:37.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.426+0000 7f302299d700 1 --2- 172.21.15.119:0/2971946932 >> v2:172.21.15.184:3300/0 conn(0x7f301c1591e0 0x7f301c159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.426+0000 7f302299d700 1 --2- 172.21.15.119:0/2971946932 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b03c0 0x7f301c0cea70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.426+0000 7f302299d700 1 -- 172.21.15.119:0/2971946932 >> 172.21.15.119:0/2971946932 conn(0x7f301c063230 msgr2=0x7f301c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:37.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.426+0000 7f302299d700 1 -- 172.21.15.119:0/2971946932 shutdown_connections 2023-10-08T01:14:37.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.426+0000 7f302299d700 1 -- 172.21.15.119:0/2971946932 wait complete. 2023-10-08T01:14:37.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.427+0000 7f302299d700 1 Processor -- start 2023-10-08T01:14:37.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.427+0000 7f302299d700 1 -- start start 2023-10-08T01:14:37.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.428+0000 7f302299d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0b03c0 0x7f301c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:37.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.428+0000 7f302299d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b0790 0x7f301c068ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:37.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.428+0000 7f302299d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f301c1591e0 0x7f301c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:37.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.428+0000 7f302299d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c169f80 con 0x7f301c0b03c0 2023-10-08T01:14:37.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.428+0000 7f302299d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301c15d460 con 0x7f301c1591e0 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.428+0000 7f302299d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301c0c96e0 con 0x7f301c0b0790 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302199b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0b03c0 0x7f301c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f301c1591e0 0x7f301c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302119a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b0790 0x7f301c068ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f301c1591e0 0x7f301c069530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:48266/0 (socket says 172.21.15.119:48266) 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302199b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0b03c0 0x7f301c157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41126/0 (socket says 172.21.15.119:41126) 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 -- 172.21.15.119:0/3937592823 learned_addr learned my addr 172.21.15.119:0/3937592823 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b0790 msgr2=0x7f301c068ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b0790 0x7f301c068ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0b03c0 msgr2=0x7f301c157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0b03c0 0x7f301c157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 -- 172.21.15.119:0/3937592823 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3014009a20 con 0x7f301c1591e0 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302199b700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0b03c0 0x7f301c157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.429+0000 7f302219c700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f301c1591e0 0x7f301c069530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3014009fc0 tx=0x7f301400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.430+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301400cbd0 con 0x7f301c1591e0 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.430+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301400f040 con 0x7f301c1591e0 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.430+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014014480 con 0x7f301c1591e0 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.430+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30180094f0 con 0x7f301c1591e0 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.430+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f301c169c10 con 0x7f301c1591e0 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.431+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301c04c3c0 con 0x7f301c1591e0 2023-10-08T01:14:37.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.432+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 59) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f3014003020 con 0x7f301c1591e0 2023-10-08T01:14:37.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.433+0000 7f3011ffb700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f3000069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:37.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.433+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 6097+0+0 (secure 0 0 0) 0x7f30140850e0 con 0x7f301c1591e0 2023-10-08T01:14:37.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.433+0000 7f302199b700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 msgr2=0x7f3000069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:37.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.433+0000 7f302199b700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f3000069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:14:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.437+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3014052120 con 0x7f301c1591e0 2023-10-08T01:14:37.534 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:37.533+0000 7f0fbf995180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:14:37.592 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:37.591+0000 7f3e625aa180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:14:37.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.634+0000 7f302199b700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 msgr2=0x7f3000069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:37.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.634+0000 7f302199b700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f3000069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:14:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:37.665+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f301c0cad50 con 0x7f3000066f80 2023-10-08T01:14:37.697 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:37.697+0000 7f0fbf995180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:14:37.760 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:37.759+0000 7f3e625aa180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:14:37.786 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:37.785+0000 7f0fbf995180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:14:37.848 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:37.846+0000 7f3e625aa180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:14:38.008 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:38.008+0000 7f0fbf995180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:14:38.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:38.035+0000 7f302199b700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 msgr2=0x7f3000069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:38.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:38.035+0000 7f302199b700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f3000069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:14:38.070 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:38.069+0000 7f3e625aa180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:14:38.183 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:38.182+0000 7f0fbf995180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:14:38.247 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:38.246+0000 7f3e625aa180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:14:38.274 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:38.274+0000 7f0fbf995180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:14:38.338 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:38.337+0000 7f3e625aa180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:14:38.435 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:38.435+0000 7f0fbf995180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:14:38.499 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:38.498+0000 7f3e625aa180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:14:38.534 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:38.534+0000 7f0fbf995180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:14:38.599 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:38.598+0000 7f3e625aa180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:14:38.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:38.836+0000 7f302199b700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 msgr2=0x7f3000069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:38.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:38.836+0000 7f302199b700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f3000069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T01:14:39.015 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:39.015+0000 7f0fbf995180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:14:39.087 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:39.086+0000 7f3e625aa180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:14:39.107 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:39.107+0000 7f0fbf995180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:14:39.182 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:39.180+0000 7f3e625aa180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:14:39.195 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:39.195+0000 7f0fbf995180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:14:39.271 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:39.270+0000 7f3e625aa180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:14:39.812 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:39.812+0000 7f0fbf995180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:14:39.887 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:39.886+0000 7f3e625aa180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:14:39.983 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:39.983+0000 7f0fbf995180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:14:40.059 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:40.058+0000 7f3e625aa180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:14:40.079 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:40.079+0000 7f0fbf995180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:14:40.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:40.153+0000 7f3e625aa180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:14:40.376 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:40.375+0000 7f0fbf995180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:14:40.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:40.438+0000 7f302199b700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 msgr2=0x7f3000069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:14:40.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:40.438+0000 7f302199b700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f3000069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T01:14:40.446 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:40.445+0000 7f3e625aa180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:14:40.471 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:40.471+0000 7f0fbf995180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:14:40.542 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:40.541+0000 7f3e625aa180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:14:41.169 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:41.169+0000 7f0fbf995180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:14:41.242 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:41.241+0000 7f3e625aa180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:14:41.454 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:41.453+0000 7f0fbf995180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:14:41.525 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:41.525+0000 7f3e625aa180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:14:41.546 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:41.546+0000 7f0fbf995180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:14:41.619 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:41.618+0000 7f3e625aa180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:14:41.701 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:41.701+0000 7f0fbf995180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:14:41.774 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:41.773+0000 7f3e625aa180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:14:41.806 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:41.806+0000 7f0fbf995180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:14:41.877 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:41.876+0000 7f3e625aa180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:14:41.995 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:41.995+0000 7f0fbf995180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:14:42.065 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.065+0000 7f3e625aa180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:14:42.088 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:42.088+0000 7f0fbf995180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:14:42.157 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.156+0000 7f3e625aa180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:14:42.179 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:42.179+0000 7f0fbf995180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:14:42.249 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.248+0000 7f3e625aa180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:14:42.272 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:14:42.272+0000 7f0fbf995180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:14:42.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:42.332+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mgrmap(e 60) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f30140524f0 con 0x7f301c1591e0 2023-10-08T01:14:42.345 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.345+0000 7f3e625aa180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:14:42.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:42.359+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 8 ==== mgrmap(e 61) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f30140552d0 con 0x7f301c1591e0 2023-10-08T01:14:42.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:42.359+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 msgr2=0x7f3000069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:14:42.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:42.359+0000 7f3011ffb700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f3000069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:42.481 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.479+0000 7f3dd7937700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.481 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.479+0000 7f3dd7937700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.481 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.479+0000 7f3dd7937700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.481 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.479+0000 7f3dd7937700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.481 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.479+0000 7f3dd7937700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.482 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.480+0000 7f3dd4931700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.482 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.480+0000 7f3dd4931700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.482 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.480+0000 7f3dd4931700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.482 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.480+0000 7f3dd4931700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:42.482 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:14:42.480+0000 7f3dd4931700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:14:43.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.363+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mon.1 v2:172.21.15.184:3300/0 9 ==== mgrmap(e 62) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f3014052dc0 con 0x7f301c1591e0 2023-10-08T01:14:43.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.364+0000 7f3011ffb700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f30000693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:43.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.364+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f301c0cad50 con 0x7f3000066f80 2023-10-08T01:14:43.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.365+0000 7f302199b700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f30000693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:43.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.368+0000 7f302199b700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f30000693b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f30180099b0 tx=0x7f3018017000 comp rx=0 tx=0).ready entity=mgr.7890 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:43.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.428+0000 7f3011ffb700 1 -- 172.21.15.119:0/3937592823 <== mgr.7890 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f301c0cad50 con 0x7f3000066f80 2023-10-08T01:14:43.429 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path snap_test_dir1 2023-10-08T01:14:43.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 msgr2=0x7f30000693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:43.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f30000693b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f30180099b0 tx=0x7f3018017000 comp rx=0 tx=0).stop 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f301c1591e0 msgr2=0x7f301c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f301c1591e0 0x7f301c069530 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3014009fc0 tx=0x7f301400b040 comp rx=0 tx=0).stop 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 shutdown_connections 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3000066f80 0x7f30000693b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f301c1591e0 0x7f301c069530 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f301c0b03c0 0x7f301c157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 --2- 172.21.15.119:0/3937592823 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f301c0b0790 0x7f301c068ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 >> 172.21.15.119:0/3937592823 conn(0x7f301c063230 msgr2=0x7f301c1587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 shutdown_connections 2023-10-08T01:14:43.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.431+0000 7f302299d700 1 -- 172.21.15.119:0/3937592823 wait complete. 2023-10-08T01:14:43.452 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T01:14:43.452 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-01_15_00 in ~17s... 2023-10-08T01:14:43.452 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-10-08T01:14:43.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-08T01:14:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.636+0000 7f3a9179d700 1 Processor -- start 2023-10-08T01:14:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.636+0000 7f3a9179d700 1 -- start start 2023-10-08T01:14:43.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.637+0000 7f3a9179d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a8c15b050 0x7f3a8c0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.637+0000 7f3a9179d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3a8c0cc8d0 0x7f3a8c15b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.637+0000 7f3a9179d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c15b960 0x7f3a8c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.637+0000 7f3a9179d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c157a50 con 0x7f3a8c15b050 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.637+0000 7f3a9179d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c1591b0 con 0x7f3a8c0cc8d0 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.637+0000 7f3a9179d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c0ca9e0 con 0x7f3a8c15b960 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a8c15b050 0x7f3a8c0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a8c15b050 0x7f3a8c0ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41280/0 (socket says 172.21.15.119:41280) 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 -- 172.21.15.119:0/3369796909 learned_addr learned my addr 172.21.15.119:0/3369796909 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 -- 172.21.15.119:0/3369796909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c15b960 msgr2=0x7f3a8c157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 --2- 172.21.15.119:0/3369796909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c15b960 0x7f3a8c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 -- 172.21.15.119:0/3369796909 >> v2:172.21.15.184:3300/0 conn(0x7f3a8c0cc8d0 msgr2=0x7f3a8c15b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:14:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 --2- 172.21.15.119:0/3369796909 >> v2:172.21.15.184:3300/0 conn(0x7f3a8c0cc8d0 0x7f3a8c15b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 -- 172.21.15.119:0/3369796909 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0cacc0 con 0x7f3a8c15b050 2023-10-08T01:14:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8bfff700 1 --2- 172.21.15.119:0/3369796909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c0ca100 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c06fdd0 tx=0x7f3a7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb1173dfe17e03e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8affd700 1 -- 172.21.15.119:0/3369796909 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00b860 con 0x7f3a8c15b050 2023-10-08T01:14:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8affd700 1 -- 172.21.15.119:0/3369796909 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c00b9c0 con 0x7f3a8c15b050 2023-10-08T01:14:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.638+0000 7f3a8affd700 1 -- 172.21.15.119:0/3369796909 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c00bd60 con 0x7f3a8c15b050 2023-10-08T01:14:43.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 -- 172.21.15.119:0/3369796909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 msgr2=0x7f3a8c0ca100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 --2- 172.21.15.119:0/3369796909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c0ca100 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c06fdd0 tx=0x7f3a7c009250 comp rx=0 tx=0).stop 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 -- 172.21.15.119:0/3369796909 shutdown_connections 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 --2- 172.21.15.119:0/3369796909 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c15b960 0x7f3a8c157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 --2- 172.21.15.119:0/3369796909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c0ca100 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 --2- 172.21.15.119:0/3369796909 >> v2:172.21.15.184:3300/0 conn(0x7f3a8c0cc8d0 0x7f3a8c15b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 -- 172.21.15.119:0/3369796909 >> 172.21.15.119:0/3369796909 conn(0x7f3a8c068e20 msgr2=0x7f3a8c0cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 -- 172.21.15.119:0/3369796909 shutdown_connections 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.639+0000 7f3a9179d700 1 -- 172.21.15.119:0/3369796909 wait complete. 2023-10-08T01:14:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.640+0000 7f3a9179d700 1 Processor -- start 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.641+0000 7f3a9179d700 1 -- start start 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.641+0000 7f3a9179d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c0cc8d0 0x7f3a8c153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.641+0000 7f3a9179d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c1448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a9179d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a8c15b960 0x7f3a8c144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a9179d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c145370 con 0x7f3a8c15b050 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a9179d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c1512e0 con 0x7f3a8c15b960 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a9179d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c15ef20 con 0x7f3a8c0cc8d0 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c1448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c1448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41288/0 (socket says 172.21.15.119:41288) 2023-10-08T01:14:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 -- 172.21.15.119:0/240366752 learned_addr learned my addr 172.21.15.119:0/240366752 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8bfff700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c0cc8d0 0x7f3a8c153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a90f9c700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a8c15b960 0x7f3a8c144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 -- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c0cc8d0 msgr2=0x7f3a8c153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c0cc8d0 0x7f3a8c153a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 -- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a8c15b960 msgr2=0x7f3a8c144e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a8c15b960 0x7f3a8c144e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 -- 172.21.15.119:0/240366752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c0094f0 con 0x7f3a8c15b050 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a90f9c700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a8c15b960 0x7f3a8c144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.642+0000 7f3a8b7fe700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c1448f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b810 tx=0x7f3a800091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:43.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.643+0000 7f3a73fff700 1 -- 172.21.15.119:0/240366752 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000f510 con 0x7f3a8c15b050 2023-10-08T01:14:43.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.643+0000 7f3a73fff700 1 -- 172.21.15.119:0/240366752 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000f690 con 0x7f3a8c15b050 2023-10-08T01:14:43.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.643+0000 7f3a73fff700 1 -- 172.21.15.119:0/240366752 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000fa50 con 0x7f3a8c15b050 2023-10-08T01:14:43.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.643+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c15cb10 con 0x7f3a8c15b050 2023-10-08T01:14:43.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.643+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c15d350 con 0x7f3a8c15b050 2023-10-08T01:14:43.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.644+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c06d4f0 con 0x7f3a8c15b050 2023-10-08T01:14:43.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.645+0000 7f3a73fff700 1 -- 172.21.15.119:0/240366752 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 62) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f3a8001a400 con 0x7f3a8c15b050 2023-10-08T01:14:43.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.646+0000 7f3a73fff700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3a78066dd0 0x7f3a78069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:14:43.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.646+0000 7f3a73fff700 1 -- 172.21.15.119:0/240366752 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f3a80084100 con 0x7f3a8c15b050 2023-10-08T01:14:43.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.646+0000 7f3a8bfff700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3a78066dd0 0x7f3a78069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:14:43.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.647+0000 7f3a8bfff700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3a78066dd0 0x7f3a78069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c06fdd0 tx=0x7f3a7c017000 comp rx=0 tx=0).ready entity=mgr.7890 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:14:43.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.650+0000 7f3a73fff700 1 -- 172.21.15.119:0/240366752 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a80051340 con 0x7f3a8c15b050 2023-10-08T01:14:43.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.864+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a8c15d570 con 0x7f3a78066dd0 2023-10-08T01:14:43.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.871+0000 7f3a73fff700 1 -- 172.21.15.119:0/240366752 <== mgr.7890 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f3a8c15d570 con 0x7f3a78066dd0 2023-10-08T01:14:43.872 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:14:43.872 INFO:teuthology.orchestra.run.smithi119.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-10-08T01:14:43.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3a78066dd0 msgr2=0x7f3a78069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:43.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3a78066dd0 0x7f3a78069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c06fdd0 tx=0x7f3a7c017000 comp rx=0 tx=0).stop 2023-10-08T01:14:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 msgr2=0x7f3a8c1448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:14:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c1448f0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b810 tx=0x7f3a800091d0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 shutdown_connections 2023-10-08T01:14:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3a78066dd0 0x7f3a78069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a8c0cc8d0 0x7f3a8c153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a8c15b960 0x7f3a8c144e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 --2- 172.21.15.119:0/240366752 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a8c15b050 0x7f3a8c1448f0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:14:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 >> 172.21.15.119:0/240366752 conn(0x7f3a8c068e20 msgr2=0x7f3a8c1597c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:14:43.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.873+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 shutdown_connections 2023-10-08T01:14:43.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:14:43.874+0000 7f3a9179d700 1 -- 172.21.15.119:0/240366752 wait complete. 2023-10-08T01:14:43.893 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-10-08T01:14:43.893 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-10-08T01:14:43.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:43.894 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:43.932 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:43.932 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:43.932 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:44.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:44.933 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:44.975 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:44.976 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:44.976 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:45.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:45.977 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:46.020 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:46.020 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:46.021 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:47.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:47.022 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:47.066 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:47.067 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:47.067 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:48.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:14:48.016 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:14:48.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:48.068 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:48.110 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:48.110 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:48.110 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:49.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:49.111 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:49.153 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:49.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:49.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:50.154 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:50.155 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:50.197 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:50.198 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:50.198 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:51.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:51.199 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:51.242 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:51.242 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:51.243 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:52.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:52.244 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:52.287 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:52.287 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:52.288 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:53.288 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:53.288 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:53.331 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:53.331 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:53.332 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:54.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:54.333 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:54.376 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:54.376 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:54.376 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:55.377 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:55.377 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:55.420 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:55.421 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:55.421 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:56.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:56.422 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:56.465 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:56.465 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:56.465 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:57.466 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:57.466 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:57.509 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:57.509 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:57.509 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:58.510 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:58.511 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:58.553 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:58.553 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:58.554 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:14:59.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:14:59.555 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:14:59.597 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T01:14:59.597 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T01:14:59.598 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:15:00.441 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:00.441+0000 7f3dc7116700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:00.441 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:00.441+0000 7f3dc7116700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:00.442 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:00.441+0000 7f3dc7116700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:00.442 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:00.441+0000 7f3dc7116700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:00.442 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:00.441+0000 7f3dc7116700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:00.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:15:00.598 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:15:00.637 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_15_00_UTC 2023-10-08T01:15:00.638 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-01_15_00_UTC'} 2023-10-08T01:15:00.639 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-01_15_00_UTC'} 2023-10-08T01:15:00.639 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T01:15:00.639 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-08-01_15_00_UTC'] 2023-10-08T01:15:00.639 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696727683.452593, now = 1696727700.6394439, timo = 17 2023-10-08T01:15:01.640 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-08T01:15:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.838+0000 7f2d6dfff700 1 Processor -- start 2023-10-08T01:15:01.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.838+0000 7f2d6dfff700 1 -- start start 2023-10-08T01:15:01.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.839+0000 7f2d6dfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2d68069a50 0x7f2d680c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:01.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.839+0000 7f2d6dfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d68069e20 0x7f2d681569b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:01.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.839+0000 7f2d6dfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d680ac030 con 0x7f2d680c8490 2023-10-08T01:15:01.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.839+0000 7f2d6dfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d680ab820 con 0x7f2d68069e20 2023-10-08T01:15:01.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.839+0000 7f2d6dfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6806dbb0 con 0x7f2d68069a50 2023-10-08T01:15:01.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d67fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d68069e20 0x7f2d681569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:01.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d67fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d68069e20 0x7f2d681569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37552/0 (socket says 172.21.15.119:37552) 2023-10-08T01:15:01.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d67fff700 1 -- 172.21.15.119:0/1707949073 learned_addr learned my addr 172.21.15.119:0/1707949073 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:01.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d677fe700 1 --2- 172.21.15.119:0/1707949073 >> v2:172.21.15.184:3300/0 conn(0x7f2d68069a50 0x7f2d680c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:01.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d67fff700 1 -- 172.21.15.119:0/1707949073 >> v2:172.21.15.184:3300/0 conn(0x7f2d68069a50 msgr2=0x7f2d680c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:01.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d67fff700 1 --2- 172.21.15.119:0/1707949073 >> v2:172.21.15.184:3300/0 conn(0x7f2d68069a50 0x7f2d680c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:01.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d67fff700 1 -- 172.21.15.119:0/1707949073 >> v1:172.21.15.184:6789/0 conn(0x7f2d680c8490 legacy=0x7f2d6815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:01.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d67fff700 1 -- 172.21.15.119:0/1707949073 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680aba70 con 0x7f2d68069e20 2023-10-08T01:15:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d677fe700 1 --2- 172.21.15.119:0/1707949073 >> v2:172.21.15.184:3300/0 conn(0x7f2d68069a50 0x7f2d680c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.840+0000 7f2d67fff700 1 --2- 172.21.15.119:0/1707949073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d68069e20 0x7f2d681569b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2d6806b970 tx=0x7f2d6000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a9a982cac95fb7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d667fc700 1 -- 172.21.15.119:0/1707949073 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60010070 con 0x7f2d68069e20 2023-10-08T01:15:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d667fc700 1 -- 172.21.15.119:0/1707949073 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6000dc40 con 0x7f2d68069e20 2023-10-08T01:15:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d667fc700 1 -- 172.21.15.119:0/1707949073 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d60016440 con 0x7f2d68069e20 2023-10-08T01:15:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1707949073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d68069e20 msgr2=0x7f2d681569b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1707949073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d68069e20 0x7f2d681569b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f2d6806b970 tx=0x7f2d6000ba50 comp rx=0 tx=0).stop 2023-10-08T01:15:01.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1707949073 shutdown_connections 2023-10-08T01:15:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1707949073 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d68069e20 0x7f2d681569b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1707949073 >> v2:172.21.15.184:3300/0 conn(0x7f2d68069a50 0x7f2d680c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1707949073 >> 172.21.15.119:0/1707949073 conn(0x7f2d680c8f90 msgr2=0x7f2d680c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1707949073 shutdown_connections 2023-10-08T01:15:01.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.841+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1707949073 wait complete. 2023-10-08T01:15:01.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.843+0000 7f2d6dfff700 1 Processor -- start 2023-10-08T01:15:01.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.843+0000 7f2d6dfff700 1 -- start start 2023-10-08T01:15:01.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d6dfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68069a50 0x7f2d680aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:01.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d6dfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d68069e20 0x7f2d681559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:01.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d6dfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d680c8490 0x7f2d68152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:01.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d6dfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680bfe20 con 0x7f2d680c8490 2023-10-08T01:15:01.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d6dfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680aaa20 con 0x7f2d68069a50 2023-10-08T01:15:01.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d6dfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680aacd0 con 0x7f2d68069e20 2023-10-08T01:15:01.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d680c8490 0x7f2d68152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:01.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d680c8490 0x7f2d68152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37558/0 (socket says 172.21.15.119:37558) 2023-10-08T01:15:01.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d67fff700 1 -- 172.21.15.119:0/1902238262 learned_addr learned my addr 172.21.15.119:0/1902238262 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:01.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d677fe700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68069a50 0x7f2d680aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:01.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.844+0000 7f2d66ffd700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d68069e20 0x7f2d681559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d67fff700 1 -- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d68069e20 msgr2=0x7f2d681559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d67fff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d68069e20 0x7f2d681559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d67fff700 1 -- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68069a50 msgr2=0x7f2d680aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d67fff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68069a50 0x7f2d680aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d67fff700 1 -- 172.21.15.119:0/1902238262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6000bcf0 con 0x7f2d680c8490 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d66ffd700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d68069e20 0x7f2d681559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d677fe700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68069a50 0x7f2d680aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d67fff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d680c8490 0x7f2d68152d30 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2d680cc0b0 tx=0x7f2d6001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d4f7fe700 1 -- 172.21.15.119:0/1902238262 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d600164c0 con 0x7f2d680c8490 2023-10-08T01:15:01.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d4f7fe700 1 -- 172.21.15.119:0/1902238262 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d60016e80 con 0x7f2d680c8490 2023-10-08T01:15:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d680aafb0 con 0x7f2d680c8490 2023-10-08T01:15:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d4f7fe700 1 -- 172.21.15.119:0/1902238262 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d600208c0 con 0x7f2d680c8490 2023-10-08T01:15:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.845+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d680ab570 con 0x7f2d680c8490 2023-10-08T01:15:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.846+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d680718b0 con 0x7f2d680c8490 2023-10-08T01:15:01.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.847+0000 7f2d4f7fe700 1 -- 172.21.15.119:0/1902238262 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 63) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f2d60020cf0 con 0x7f2d680c8490 2023-10-08T01:15:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.848+0000 7f2d4f7fe700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d54067130 0x7f2d540695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.848+0000 7f2d4f7fe700 1 -- 172.21.15.119:0/1902238262 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f2d60086f20 con 0x7f2d680c8490 2023-10-08T01:15:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.848+0000 7f2d677fe700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d54067130 0x7f2d540695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:01.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.849+0000 7f2d677fe700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d54067130 0x7f2d540695e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d58006fd0 tx=0x7f2d58008040 comp rx=0 tx=0).ready entity=mgr.7890 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:01.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:01.852+0000 7f2d4f7fe700 1 -- 172.21.15.119:0/1902238262 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d60053ad0 con 0x7f2d680c8490 2023-10-08T01:15:02.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.070+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d680ac030 con 0x7f2d54067130 2023-10-08T01:15:02.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.130+0000 7f2d4f7fe700 1 -- 172.21.15.119:0/1902238262 <== mgr.7890 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f2d680ac030 con 0x7f2d54067130 2023-10-08T01:15:02.131 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path snap_test_dir1 2023-10-08T01:15:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.132+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d54067130 msgr2=0x7f2d540695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.132+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d54067130 0x7f2d540695e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d58006fd0 tx=0x7f2d58008040 comp rx=0 tx=0).stop 2023-10-08T01:15:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.132+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d680c8490 msgr2=0x7f2d68152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.132+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d680c8490 0x7f2d68152d30 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f2d680cc0b0 tx=0x7f2d6001a000 comp rx=0 tx=0).stop 2023-10-08T01:15:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.133+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 shutdown_connections 2023-10-08T01:15:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.133+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d54067130 0x7f2d540695e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.133+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d680c8490 0x7f2d68152d30 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.133+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68069a50 0x7f2d680aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.133+0000 7f2d6dfff700 1 --2- 172.21.15.119:0/1902238262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d68069e20 0x7f2d681559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.133+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 >> 172.21.15.119:0/1902238262 conn(0x7f2d680c8f90 msgr2=0x7f2d680ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:02.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.133+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 shutdown_connections 2023-10-08T01:15:02.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.133+0000 7f2d6dfff700 1 -- 172.21.15.119:0/1902238262 wait complete. 2023-10-08T01:15:02.150 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T01:15:02.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:15:02.185 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T01:15:02.222 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_15_00_UTC 2023-10-08T01:15:02.223 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_15_00_UTC 2023-10-08T01:15:02.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:15:02.223 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_15_00_UTC) 2023-10-08T01:15:02.305 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:15:02.305 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T01:15:02.384 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T01:15:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.608+0000 7f0f7599d700 1 Processor -- start 2023-10-08T01:15:02.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.608+0000 7f0f7599d700 1 -- start start 2023-10-08T01:15:02.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.609+0000 7f0f7599d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f70159a70 0x7f0f700c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:02.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.609+0000 7f0f7599d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f7006bfa0 0x7f0f700c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:02.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.609+0000 7f0f7599d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f70068810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:02.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.609+0000 7f0f7599d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f70068d50 con 0x7f0f7006bfa0 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.609+0000 7f0f7599d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f700ca920 con 0x7f0f70159a70 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.609+0000 7f0f7599d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7006af00 con 0x7f0f70068390 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.609+0000 7f0f6ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f7006bfa0 0x7f0f700c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.609+0000 7f0f6ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f7006bfa0 0x7f0f700c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37572/0 (socket says 172.21.15.119:37572) 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6ffff700 1 -- 172.21.15.119:0/1036210711 learned_addr learned my addr 172.21.15.119:0/1036210711 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f7519c700 1 --2- 172.21.15.119:0/1036210711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f70068810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f7499b700 1 --2- 172.21.15.119:0/1036210711 >> v2:172.21.15.184:3300/0 conn(0x7f0f70159a70 0x7f0f700c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6ffff700 1 -- 172.21.15.119:0/1036210711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 msgr2=0x7f0f70068810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6ffff700 1 --2- 172.21.15.119:0/1036210711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f70068810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6ffff700 1 -- 172.21.15.119:0/1036210711 >> v2:172.21.15.184:3300/0 conn(0x7f0f70159a70 msgr2=0x7f0f700c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6ffff700 1 --2- 172.21.15.119:0/1036210711 >> v2:172.21.15.184:3300/0 conn(0x7f0f70159a70 0x7f0f700c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6ffff700 1 -- 172.21.15.119:0/1036210711 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7006b1e0 con 0x7f0f7006bfa0 2023-10-08T01:15:02.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6ffff700 1 --2- 172.21.15.119:0/1036210711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7006bfa0 0x7f0f700c8830 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0f700640b0 tx=0x7f0f5c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c06e79022ba831 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:02.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6f7fe700 1 -- 172.21.15.119:0/1036210711 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c005c10 con 0x7f0f7006bfa0 2023-10-08T01:15:02.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6f7fe700 1 -- 172.21.15.119:0/1036210711 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f5c005d70 con 0x7f0f7006bfa0 2023-10-08T01:15:02.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.610+0000 7f0f6f7fe700 1 -- 172.21.15.119:0/1036210711 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5c00b880 con 0x7f0f7006bfa0 2023-10-08T01:15:02.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 -- 172.21.15.119:0/1036210711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7006bfa0 msgr2=0x7f0f700c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:02.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 --2- 172.21.15.119:0/1036210711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7006bfa0 0x7f0f700c8830 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f0f700640b0 tx=0x7f0f5c009220 comp rx=0 tx=0).stop 2023-10-08T01:15:02.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 -- 172.21.15.119:0/1036210711 shutdown_connections 2023-10-08T01:15:02.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 --2- 172.21.15.119:0/1036210711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f70068810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 --2- 172.21.15.119:0/1036210711 >> v2:172.21.15.184:3300/0 conn(0x7f0f70159a70 0x7f0f700c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 --2- 172.21.15.119:0/1036210711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f7006bfa0 0x7f0f700c8830 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 -- 172.21.15.119:0/1036210711 >> 172.21.15.119:0/1036210711 conn(0x7f0f700cddc0 msgr2=0x7f0f700c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 -- 172.21.15.119:0/1036210711 shutdown_connections 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.611+0000 7f0f7599d700 1 -- 172.21.15.119:0/1036210711 wait complete. 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.612+0000 7f0f7599d700 1 Processor -- start 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.613+0000 7f0f7599d700 1 -- start start 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7599d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f700b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7599d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f7006bfa0 0x7f0f700c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7599d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f70159a70 0x7f0f7016aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7599d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700b1170 con 0x7f0f70159a70 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7599d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700b0530 con 0x7f0f7006bfa0 2023-10-08T01:15:02.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7599d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f700b1ad0 con 0x7f0f70068390 2023-10-08T01:15:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7519c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f70159a70 0x7f0f7016aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7519c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f70159a70 0x7f0f7016aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37584/0 (socket says 172.21.15.119:37584) 2023-10-08T01:15:02.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7519c700 1 -- 172.21.15.119:0/3941579360 learned_addr learned my addr 172.21.15.119:0/3941579360 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f6ffff700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f7006bfa0 0x7f0f700c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7499b700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f700b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7519c700 1 -- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 msgr2=0x7f0f700b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7519c700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f700b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7519c700 1 -- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f7006bfa0 msgr2=0x7f0f700c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7519c700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f7006bfa0 0x7f0f700c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:02.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7519c700 1 -- 172.21.15.119:0/3941579360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c009480 con 0x7f0f70159a70 2023-10-08T01:15:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f7499b700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f700b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.614+0000 7f0f6ffff700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f7006bfa0 0x7f0f700c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.615+0000 7f0f7519c700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f70159a70 0x7f0f7016aa40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0f68007f20 tx=0x7f0f68009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.615+0000 7f0f6cff9700 1 -- 172.21.15.119:0/3941579360 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6800eca0 con 0x7f0f70159a70 2023-10-08T01:15:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.615+0000 7f0f6cff9700 1 -- 172.21.15.119:0/3941579360 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6800ee20 con 0x7f0f70159a70 2023-10-08T01:15:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.615+0000 7f0f6cff9700 1 -- 172.21.15.119:0/3941579360 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68014670 con 0x7f0f70159a70 2023-10-08T01:15:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.615+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f700b07d0 con 0x7f0f70159a70 2023-10-08T01:15:02.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.615+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f700b0d90 con 0x7f0f70159a70 2023-10-08T01:15:02.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.616+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f700b12b0 con 0x7f0f70159a70 2023-10-08T01:15:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.617+0000 7f0f6cff9700 1 -- 172.21.15.119:0/3941579360 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 63) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f0f680149e0 con 0x7f0f70159a70 2023-10-08T01:15:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.618+0000 7f0f6cff9700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f54067130 0x7f0f540695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.618+0000 7f0f6cff9700 1 -- 172.21.15.119:0/3941579360 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f0f680852d0 con 0x7f0f70159a70 2023-10-08T01:15:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.618+0000 7f0f7499b700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f54067130 0x7f0f540695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.619+0000 7f0f7499b700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f54067130 0x7f0f540695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0f7014bd50 tx=0x7f0f64008040 comp rx=0 tx=0).ready entity=mgr.7890 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.622+0000 7f0f6cff9700 1 -- 172.21.15.119:0/3941579360 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f68051f90 con 0x7f0f70159a70 2023-10-08T01:15:02.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:02.871+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f0f7006b4c0 con 0x7f0f70159a70 2023-10-08T01:15:03.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.255+0000 7f0f6cff9700 1 -- 172.21.15.119:0/3941579360 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f0f68012da0 con 0x7f0f70159a70 2023-10-08T01:15:03.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.256+0000 7f0f6cff9700 1 -- 172.21.15.119:0/3941579360 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v64) v1 ==== 93+0+0 (secure 0 0 0) 0x7f0f68055a20 con 0x7f0f70159a70 2023-10-08T01:15:03.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f54067130 msgr2=0x7f0f540695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:03.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f54067130 0x7f0f540695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0f7014bd50 tx=0x7f0f64008040 comp rx=0 tx=0).stop 2023-10-08T01:15:03.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f70159a70 msgr2=0x7f0f7016aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:03.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f70159a70 0x7f0f7016aa40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0f68007f20 tx=0x7f0f68009860 comp rx=0 tx=0).stop 2023-10-08T01:15:03.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 shutdown_connections 2023-10-08T01:15:03.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f54067130 0x7f0f540695e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f70068390 0x7f0f700b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f70159a70 0x7f0f7016aa40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 --2- 172.21.15.119:0/3941579360 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f7006bfa0 0x7f0f700c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 >> 172.21.15.119:0/3941579360 conn(0x7f0f700cddc0 msgr2=0x7f0f701577a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:03.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 shutdown_connections 2023-10-08T01:15:03.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.259+0000 7f0f7599d700 1 -- 172.21.15.119:0/3941579360 wait complete. 2023-10-08T01:15:03.277 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:15:03.277 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:15:03.363 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:15:03.454 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:15:03.454 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T01:15:03.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:15:03.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T01:15:03.539 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T01:15:03.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:15:03.539 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T01:15:03.576 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:15:03.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.802+0000 7f680e99d700 1 Processor -- start 2023-10-08T01:15:03.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.802+0000 7f680e99d700 1 -- start start 2023-10-08T01:15:03.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680e99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f68081597f0 0x7f68080b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:03.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680e99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f680815d230 0x7f680815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:03.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680e99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f680815dbb0 con 0x7f6808159420 2023-10-08T01:15:03.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680e99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68080b11d0 con 0x7f68081597f0 2023-10-08T01:15:03.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680e99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68080b14b0 con 0x7f680815d230 2023-10-08T01:15:03.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680e19c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f680815d230 0x7f680815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:03.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680d19a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f68081597f0 0x7f68080b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:03.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680e19c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f680815d230 0x7f680815d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:43782/0 (socket says 172.21.15.119:43782) 2023-10-08T01:15:03.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680e19c700 1 -- 172.21.15.119:0/1165065047 learned_addr learned my addr 172.21.15.119:0/1165065047 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:03.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680d99b700 1 --1- 172.21.15.119:0/1165065047 >> v1:172.21.15.184:6789/0 conn(0x7f6808159420 0x7f68080b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:35934/0 (socket says 172.21.15.119:35934) 2023-10-08T01:15:03.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680d19a700 1 -- 172.21.15.119:0/1165065047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f680815d230 msgr2=0x7f680815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.804+0000 7f680d19a700 1 --2- 172.21.15.119:0/1165065047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f680815d230 0x7f680815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.805+0000 7f680d19a700 1 -- 172.21.15.119:0/1165065047 >> v1:172.21.15.184:6789/0 conn(0x7f6808159420 legacy=0x7f68080b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.805+0000 7f680d19a700 1 -- 172.21.15.119:0/1165065047 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68080b1790 con 0x7f68081597f0 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.805+0000 7f680c999700 1 -- 172.21.15.119:0/1165065047 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316523496 0 0) 0x7f680815dbb0 con 0x7f6808159420 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.805+0000 7f680e19c700 1 --2- 172.21.15.119:0/1165065047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f680815d230 0x7f680815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.805+0000 7f680d19a700 1 --2- 172.21.15.119:0/1165065047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68081597f0 0x7f68080b2270 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098e0 tx=0x7f67f80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46f0d5183eaa304d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.805+0000 7f680c999700 1 -- 172.21.15.119:0/1165065047 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800b640 con 0x7f68081597f0 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.805+0000 7f680c999700 1 -- 172.21.15.119:0/1165065047 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f800b7a0 con 0x7f68081597f0 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.805+0000 7f680c999700 1 -- 172.21.15.119:0/1165065047 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800bb40 con 0x7f68081597f0 2023-10-08T01:15:03.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.806+0000 7f680e99d700 1 -- 172.21.15.119:0/1165065047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68081597f0 msgr2=0x7f68080b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.806+0000 7f680e99d700 1 --2- 172.21.15.119:0/1165065047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68081597f0 0x7f68080b2270 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098e0 tx=0x7f67f80091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.806+0000 7f680e99d700 1 -- 172.21.15.119:0/1165065047 shutdown_connections 2023-10-08T01:15:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.806+0000 7f680e99d700 1 --2- 172.21.15.119:0/1165065047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f680815d230 0x7f680815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.806+0000 7f680e99d700 1 --2- 172.21.15.119:0/1165065047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f68081597f0 0x7f68080b2270 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.806+0000 7f680e99d700 1 -- 172.21.15.119:0/1165065047 >> 172.21.15.119:0/1165065047 conn(0x7f68080d0660 msgr2=0x7f68080d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.806+0000 7f680e99d700 1 -- 172.21.15.119:0/1165065047 shutdown_connections 2023-10-08T01:15:03.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.806+0000 7f680e99d700 1 -- 172.21.15.119:0/1165065047 wait complete. 2023-10-08T01:15:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.807+0000 7f680e99d700 1 Processor -- start 2023-10-08T01:15:03.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.807+0000 7f680e99d700 1 -- start start 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.808+0000 7f680e99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6808159420 0x7f6808154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.808+0000 7f680e99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68081597f0 0x7f680815bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.808+0000 7f680e99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680815d230 0x7f68080c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.808+0000 7f680e99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081697c0 con 0x7f68081597f0 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.808+0000 7f680e99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6808154790 con 0x7f680815d230 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.808+0000 7f680e99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f680815c470 con 0x7f6808159420 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680d99b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6808159420 0x7f6808154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680e19c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680815d230 0x7f68080c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680d99b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6808159420 0x7f6808154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:43784/0 (socket says 172.21.15.119:43784) 2023-10-08T01:15:03.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680e19c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680815d230 0x7f68080c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:40334/0 (socket says 172.21.15.119:40334) 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680d99b700 1 -- 172.21.15.119:0/989992569 learned_addr learned my addr 172.21.15.119:0/989992569 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680d19a700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68081597f0 0x7f680815bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680e19c700 1 -- 172.21.15.119:0/989992569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6808159420 msgr2=0x7f6808154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680e19c700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6808159420 0x7f6808154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680e19c700 1 -- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68081597f0 msgr2=0x7f680815bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680e19c700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68081597f0 0x7f680815bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680e19c700 1 -- 172.21.15.119:0/989992569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6800009ae0 con 0x7f680815d230 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.809+0000 7f680e19c700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680815d230 0x7f68080c5dc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6800009e70 tx=0x7f680000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.810+0000 7f67fdffb700 1 -- 172.21.15.119:0/989992569 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800018070 con 0x7f680815d230 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.810+0000 7f67fdffb700 1 -- 172.21.15.119:0/989992569 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680000f040 con 0x7f680815d230 2023-10-08T01:15:03.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.810+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f8009420 con 0x7f680815d230 2023-10-08T01:15:03.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.810+0000 7f67fdffb700 1 -- 172.21.15.119:0/989992569 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680000cb00 con 0x7f680815d230 2023-10-08T01:15:03.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.810+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680815ca30 con 0x7f680815d230 2023-10-08T01:15:03.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.811+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6808169930 con 0x7f680815d230 2023-10-08T01:15:03.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.812+0000 7f67fdffb700 1 -- 172.21.15.119:0/989992569 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f6800003020 con 0x7f680815d230 2023-10-08T01:15:03.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.812+0000 7f67fdffb700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67ec067170 0x7f67ec069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:03.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.813+0000 7f680d99b700 1 -- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67ec067170 msgr2=0x7f67ec069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:03.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.813+0000 7f67fdffb700 1 -- 172.21.15.119:0/989992569 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f6800084a90 con 0x7f680815d230 2023-10-08T01:15:03.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.813+0000 7f680d99b700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67ec067170 0x7f67ec069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:03.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:03.817+0000 7f67fdffb700 1 -- 172.21.15.119:0/989992569 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f680004d5a0 con 0x7f680815d230 2023-10-08T01:15:04.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.014+0000 7f680d99b700 1 -- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67ec067170 msgr2=0x7f67ec069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:04.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.014+0000 7f680d99b700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67ec067170 0x7f67ec069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:04.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.073+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f680804c3c0 con 0x7f680815d230 2023-10-08T01:15:04.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.074+0000 7f67fdffb700 1 -- 172.21.15.119:0/989992569 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f6800051630 con 0x7f680815d230 2023-10-08T01:15:04.074 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:04.074 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7141,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3118853373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3118853373},{"type":"v1","addr":"172.21.15.184:6834","nonce":3118853373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7144,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/737543181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":737543181},{"type":"v1","addr":"172.21.15.184:6835","nonce":737543181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/2667608886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2667608886},{"type":"v1","addr":"172.21.15.119:6834","nonce":2667608886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:14:22.219941+0000","modified":"2023-10-08T01:14:25.788077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7680},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7680":{"gid":7680,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1713175169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1713175169},{"type":"v1","addr":"172.21.15.119:6835","nonce":1713175169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T01:15:04.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.076+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67ec067170 msgr2=0x7f67ec069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:04.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.076+0000 7f680e99d700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67ec067170 0x7f67ec069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.076+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680815d230 msgr2=0x7f68080c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.076+0000 7f680e99d700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680815d230 0x7f68080c5dc0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f6800009e70 tx=0x7f680000b040 comp rx=0 tx=0).stop 2023-10-08T01:15:04.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.077+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 shutdown_connections 2023-10-08T01:15:04.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.077+0000 7f680e99d700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67ec067170 0x7f67ec069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.077+0000 7f680e99d700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6808159420 0x7f6808154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.077+0000 7f680e99d700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f680815d230 0x7f68080c5dc0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.077+0000 7f680e99d700 1 --2- 172.21.15.119:0/989992569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68081597f0 0x7f680815bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.077+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 >> 172.21.15.119:0/989992569 conn(0x7f68080d0660 msgr2=0x7f68080699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:04.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.077+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 shutdown_connections 2023-10-08T01:15:04.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.077+0000 7f680e99d700 1 -- 172.21.15.119:0/989992569 wait complete. 2023-10-08T01:15:04.078 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 55 2023-10-08T01:15:04.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:15:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.282+0000 7fe358b9d700 1 Processor -- start 2023-10-08T01:15:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.282+0000 7fe358b9d700 1 -- start start 2023-10-08T01:15:04.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.283+0000 7fe358b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe35415d230 0x7fe35415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.283+0000 7fe358b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3540b2980 con 0x7fe354159420 2023-10-08T01:15:04.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.283+0000 7fe358b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35415db70 con 0x7fe3541597f0 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.283+0000 7fe358b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3540b11d0 con 0x7fe35415d230 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe3537fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe354159420 0x7fe3540b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46348/0 (socket says 172.21.15.119:46348) 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe3537fe700 1 -- 172.21.15.119:0/836616576 learned_addr learned my addr 172.21.15.119:0/836616576 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe353fff700 1 --2- 172.21.15.119:0/836616576 >> v2:172.21.15.184:3300/0 conn(0x7fe35415d230 0x7fe35415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe3527fc700 1 -- 172.21.15.119:0/836616576 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052926931 0 0) 0x7fe3540b2980 con 0x7fe354159420 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe3527fc700 1 -- 172.21.15.119:0/836616576 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe33c003570 con 0x7fe354159420 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe3527fc700 1 -- 172.21.15.119:0/836616576 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370938692 0 0) 0x7fe35415db70 con 0x7fe3541597f0 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe3527fc700 1 -- 172.21.15.119:0/836616576 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3540b2980 con 0x7fe3541597f0 2023-10-08T01:15:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe353fff700 1 -- 172.21.15.119:0/836616576 >> v1:172.21.15.184:6789/0 conn(0x7fe3541597f0 legacy=0x7fe3540b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe353fff700 1 -- 172.21.15.119:0/836616576 >> v1:172.21.15.119:6789/0 conn(0x7fe354159420 legacy=0x7fe3540b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.284+0000 7fe353fff700 1 -- 172.21.15.119:0/836616576 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3540b14b0 con 0x7fe35415d230 2023-10-08T01:15:04.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.285+0000 7fe353fff700 1 --2- 172.21.15.119:0/836616576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe35415d230 0x7fe35415d630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009e60 tx=0x7fe34c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a72c52a2051929 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:04.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.285+0000 7fe3527fc700 1 -- 172.21.15.119:0/836616576 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00e070 con 0x7fe35415d230 2023-10-08T01:15:04.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.285+0000 7fe3527fc700 1 -- 172.21.15.119:0/836616576 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c00bab0 con 0x7fe35415d230 2023-10-08T01:15:04.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.285+0000 7fe3527fc700 1 -- 172.21.15.119:0/836616576 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00be70 con 0x7fe35415d230 2023-10-08T01:15:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.285+0000 7fe358b9d700 1 -- 172.21.15.119:0/836616576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe35415d230 msgr2=0x7fe35415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.285+0000 7fe358b9d700 1 --2- 172.21.15.119:0/836616576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe35415d230 0x7fe35415d630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009e60 tx=0x7fe34c009780 comp rx=0 tx=0).stop 2023-10-08T01:15:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.286+0000 7fe358b9d700 1 -- 172.21.15.119:0/836616576 shutdown_connections 2023-10-08T01:15:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.286+0000 7fe358b9d700 1 --2- 172.21.15.119:0/836616576 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe35415d230 0x7fe35415d630 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.286+0000 7fe358b9d700 1 -- 172.21.15.119:0/836616576 >> 172.21.15.119:0/836616576 conn(0x7fe3540d0660 msgr2=0x7fe3540d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.286+0000 7fe358b9d700 1 -- 172.21.15.119:0/836616576 shutdown_connections 2023-10-08T01:15:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.286+0000 7fe358b9d700 1 -- 172.21.15.119:0/836616576 wait complete. 2023-10-08T01:15:04.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.287+0000 7fe358b9d700 1 Processor -- start 2023-10-08T01:15:04.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.287+0000 7fe358b9d700 1 -- start start 2023-10-08T01:15:04.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe358b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe354159420 0x7fe3540c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe358b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3541597f0 0x7fe3540b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe358b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe35415d230 0x7fe3540b4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe358b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540b4b50 con 0x7fe35415d230 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe358b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540b4d50 con 0x7fe3541597f0 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe358b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3540b5030 con 0x7fe354159420 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe353fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe35415d230 0x7fe3540b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe352ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3541597f0 0x7fe3540b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe3537fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe354159420 0x7fe3540c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.288+0000 7fe353fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe35415d230 0x7fe3540b4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37624/0 (socket says 172.21.15.119:37624) 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe353fff700 1 -- 172.21.15.119:0/1945904269 learned_addr learned my addr 172.21.15.119:0/1945904269 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe352ffd700 1 -- 172.21.15.119:0/1945904269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe354159420 msgr2=0x7fe3540c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe352ffd700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe354159420 0x7fe3540c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe352ffd700 1 -- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe35415d230 msgr2=0x7fe3540b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe352ffd700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe35415d230 0x7fe3540b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe352ffd700 1 -- 172.21.15.119:0/1945904269 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c009a20 con 0x7fe3541597f0 2023-10-08T01:15:04.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe353fff700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe35415d230 0x7fe3540b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:04.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe352ffd700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3541597f0 0x7fe3540b40d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe354159c70 tx=0x7fe348007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:04.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe33b7fe700 1 -- 172.21.15.119:0/1945904269 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348011590 con 0x7fe3541597f0 2023-10-08T01:15:04.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe33b7fe700 1 -- 172.21.15.119:0/1945904269 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3480116f0 con 0x7fe3541597f0 2023-10-08T01:15:04.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.289+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3540b5310 con 0x7fe3541597f0 2023-10-08T01:15:04.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.290+0000 7fe33b7fe700 1 -- 172.21.15.119:0/1945904269 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348011ab0 con 0x7fe3541597f0 2023-10-08T01:15:04.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.290+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3540b58d0 con 0x7fe3541597f0 2023-10-08T01:15:04.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.291+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3540b5af0 con 0x7fe3541597f0 2023-10-08T01:15:04.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.292+0000 7fe33b7fe700 1 -- 172.21.15.119:0/1945904269 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fe34801c460 con 0x7fe3541597f0 2023-10-08T01:15:04.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.292+0000 7fe33b7fe700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe340067170 0x7fe340069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.293+0000 7fe3537fe700 1 -- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe340067170 msgr2=0x7fe340069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:04.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.293+0000 7fe33b7fe700 1 -- 172.21.15.119:0/1945904269 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7fe348084770 con 0x7fe3541597f0 2023-10-08T01:15:04.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.293+0000 7fe3537fe700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe340067170 0x7fe340069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:04.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.301+0000 7fe33b7fe700 1 -- 172.21.15.119:0/1945904269 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe348051420 con 0x7fe3541597f0 2023-10-08T01:15:04.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.494+0000 7fe3537fe700 1 -- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe340067170 msgr2=0x7fe340069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:04.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.494+0000 7fe3537fe700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe340067170 0x7fe340069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:04.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.518+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe35404c3c0 con 0x7fe3541597f0 2023-10-08T01:15:04.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.520+0000 7fe33b7fe700 1 -- 172.21.15.119:0/1945904269 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16380 (secure 0 0 0) 0x7fe348054a40 con 0x7fe3541597f0 2023-10-08T01:15:04.520 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:04.520 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":121,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:42.347943+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:14:20.330219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T01:14:21.202759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1219956712":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/2381954001":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/546958670":"2023-10-09T01:14:42.347907+0000","172.21.15.119:6837/85511":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/309487327":"2023-10-09T01:14:42.347907+0000","172.21.15.119:6836/85511":"2023-10-09T01:14:42.347907+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:15:04.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe340067170 msgr2=0x7fe340069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:04.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe340067170 0x7fe340069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3541597f0 msgr2=0x7fe3540b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3541597f0 0x7fe3540b40d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe354159c70 tx=0x7fe348007d50 comp rx=0 tx=0).stop 2023-10-08T01:15:04.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 shutdown_connections 2023-10-08T01:15:04.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe340067170 0x7fe340069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe354159420 0x7fe3540c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe35415d230 0x7fe3540b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 --2- 172.21.15.119:0/1945904269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3541597f0 0x7fe3540b40d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 >> 172.21.15.119:0/1945904269 conn(0x7fe3540d0660 msgr2=0x7fe3540afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:04.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 shutdown_connections 2023-10-08T01:15:04.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.522+0000 7fe358b9d700 1 -- 172.21.15.119:0/1945904269 wait complete. 2023-10-08T01:15:04.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T01:14:20.330219+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '119', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T01:14:21.202759+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '120', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T01:15:04.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:15:04.552 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:04.552+0000 7f9fb3ebf180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:15:04.613 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:04.613+0000 7f9416b94180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:15:04.715 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:04.715+0000 7f9fb3ebf180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:15:04.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.729+0000 7f1c54b9d700 1 Processor -- start 2023-10-08T01:15:04.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.729+0000 7f1c54b9d700 1 -- start start 2023-10-08T01:15:04.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c54b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1c500b4e70 0x7f1c500ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c54b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c500b1b40 0x7f1c5015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c54b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c500cfb00 con 0x7f1c500b16d0 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c54b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c500b5c40 con 0x7f1c500b1b40 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c54b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c500b5f20 con 0x7f1c500b4e70 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c4ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c500b1b40 0x7f1c5015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c4f7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1c500b4e70 0x7f1c500ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c4ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c500b1b40 0x7f1c5015d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37634/0 (socket says 172.21.15.119:37634) 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c4ffff700 1 -- 172.21.15.119:0/2348898368 learned_addr learned my addr 172.21.15.119:0/2348898368 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c4ffff700 1 -- 172.21.15.119:0/2348898368 >> v2:172.21.15.184:3300/0 conn(0x7f1c500b4e70 msgr2=0x7f1c500ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c4ffff700 1 --2- 172.21.15.119:0/2348898368 >> v2:172.21.15.184:3300/0 conn(0x7f1c500b4e70 0x7f1c500ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c4ffff700 1 -- 172.21.15.119:0/2348898368 >> v1:172.21.15.184:6789/0 conn(0x7f1c500b16d0 legacy=0x7f1c5015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.731+0000 7f1c4ffff700 1 -- 172.21.15.119:0/2348898368 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c500b6200 con 0x7f1c500b1b40 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.732+0000 7f1c4ffff700 1 --2- 172.21.15.119:0/2348898368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015d950 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1c50064910 tx=0x7f1c48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30045b489bf93102 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.732+0000 7f1c4e7fc700 1 -- 172.21.15.119:0/2348898368 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4800e070 con 0x7f1c500b1b40 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.732+0000 7f1c4e7fc700 1 -- 172.21.15.119:0/2348898368 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4800ba20 con 0x7f1c500b1b40 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.732+0000 7f1c4e7fc700 1 -- 172.21.15.119:0/2348898368 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c480092e0 con 0x7f1c500b1b40 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.732+0000 7f1c54b9d700 1 -- 172.21.15.119:0/2348898368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 msgr2=0x7f1c5015d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.732+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/2348898368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015d950 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1c50064910 tx=0x7f1c48009840 comp rx=0 tx=0).stop 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.733+0000 7f1c54b9d700 1 -- 172.21.15.119:0/2348898368 shutdown_connections 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.733+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/2348898368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015d950 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.733+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/2348898368 >> v2:172.21.15.184:3300/0 conn(0x7f1c500b4e70 0x7f1c500ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.733+0000 7f1c54b9d700 1 -- 172.21.15.119:0/2348898368 >> 172.21.15.119:0/2348898368 conn(0x7f1c500d10c0 msgr2=0x7f1c500affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.733+0000 7f1c54b9d700 1 -- 172.21.15.119:0/2348898368 shutdown_connections 2023-10-08T01:15:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.733+0000 7f1c54b9d700 1 -- 172.21.15.119:0/2348898368 wait complete. 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.734+0000 7f1c54b9d700 1 Processor -- start 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.734+0000 7f1c54b9d700 1 -- start start 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c54b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c500b16d0 0x7f1c5014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c54b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c54b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c500b4e70 0x7f1c5015e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c54b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c501589c0 con 0x7f1c500b1b40 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c54b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500b36e0 con 0x7f1c500b16d0 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c54b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c500b39c0 con 0x7f1c500b4e70 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c4effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c4f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c500b16d0 0x7f1c5014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c4f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c500b16d0 0x7f1c5014af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:40374/0 (socket says 172.21.15.119:40374) 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/4057051361 learned_addr learned my addr 172.21.15.119:0/4057051361 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c4ffff700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c500b4e70 0x7f1c5015e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.735+0000 7f1c4effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37648/0 (socket says 172.21.15.119:37648) 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/4057051361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c500b4e70 msgr2=0x7f1c5015e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c4f7fe700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c500b4e70 0x7f1c5015e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 msgr2=0x7f1c5015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c4f7fe700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c4f7fe700 1 -- 172.21.15.119:0/4057051361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c48009ae0 con 0x7f1c500b16d0 2023-10-08T01:15:04.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c4effd700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:04.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c4f7fe700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c500b16d0 0x7f1c5014af90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000e930 tx=0x7f1c400091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:04.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c377fe700 1 -- 172.21.15.119:0/4057051361 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40013680 con 0x7f1c500b16d0 2023-10-08T01:15:04.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c377fe700 1 -- 172.21.15.119:0/4057051361 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c400137e0 con 0x7f1c500b16d0 2023-10-08T01:15:04.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c377fe700 1 -- 172.21.15.119:0/4057051361 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40013b80 con 0x7f1c500b16d0 2023-10-08T01:15:04.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.736+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c500b3ca0 con 0x7f1c500b16d0 2023-10-08T01:15:04.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.737+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c500c6400 con 0x7f1c500b16d0 2023-10-08T01:15:04.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.737+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c500c6620 con 0x7f1c500b16d0 2023-10-08T01:15:04.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.739+0000 7f1c377fe700 1 -- 172.21.15.119:0/4057051361 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f1c40005a00 con 0x7f1c500b16d0 2023-10-08T01:15:04.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.739+0000 7f1c377fe700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1c3c067170 0x7f1c3c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:04.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.740+0000 7f1c377fe700 1 -- 172.21.15.119:0/4057051361 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f1c40084da0 con 0x7f1c500b16d0 2023-10-08T01:15:04.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.740+0000 7f1c4effd700 1 -- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1c3c067170 msgr2=0x7f1c3c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:04.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.740+0000 7f1c4effd700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1c3c067170 0x7f1c3c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:04.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.747+0000 7f1c377fe700 1 -- 172.21.15.119:0/4057051361 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c40051a50 con 0x7f1c500b16d0 2023-10-08T01:15:04.783 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:04.783+0000 7f9416b94180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:15:04.806 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:04.805+0000 7f9fb3ebf180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:15:04.869 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:04.869+0000 7f9416b94180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:15:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.941+0000 7f1c4effd700 1 -- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1c3c067170 msgr2=0x7f1c3c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:04.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.941+0000 7f1c4effd700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1c3c067170 0x7f1c3c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:04.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.944+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c5004bb80 con 0x7f1c500b16d0 2023-10-08T01:15:04.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.945+0000 7f1c377fe700 1 -- 172.21.15.119:0/4057051361 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c40021020 con 0x7f1c500b16d0 2023-10-08T01:15:04.945 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.947+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1c3c067170 msgr2=0x7f1c3c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.947+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1c3c067170 0x7f1c3c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.947+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c500b16d0 msgr2=0x7f1c5014af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.947+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c500b16d0 0x7f1c5014af90 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000e930 tx=0x7f1c400091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.947+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 shutdown_connections 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.947+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1c3c067170 0x7f1c3c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.947+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1c500b4e70 0x7f1c5015e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.948+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1c500b16d0 0x7f1c5014af90 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.948+0000 7f1c54b9d700 1 --2- 172.21.15.119:0/4057051361 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c500b1b40 0x7f1c5015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:04.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.948+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 >> 172.21.15.119:0/4057051361 conn(0x7f1c500d10c0 msgr2=0x7f1c50069c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:04.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.948+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 shutdown_connections 2023-10-08T01:15:04.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:04.948+0000 7f1c54b9d700 1 -- 172.21.15.119:0/4057051361 wait complete. 2023-10-08T01:15:04.966 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T01:15:05.030 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:05.030+0000 7f9fb3ebf180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:15:05.103 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:05.103+0000 7f9416b94180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:15:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.154+0000 7f37fb59e700 1 Processor -- start 2023-10-08T01:15:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.154+0000 7f37fb59e700 1 -- start start 2023-10-08T01:15:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.155+0000 7f37fb59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37fc06f060 0x7f37fc0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.155+0000 7f37fb59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc06f4b0 0x7f37fc0ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.155+0000 7f37fb59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37fc06ebd0 con 0x7f37fc0cb1a0 2023-10-08T01:15:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.155+0000 7f37fb59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37fc15b2e0 con 0x7f37fc06f060 2023-10-08T01:15:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.155+0000 7f37fb59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc15aa70 con 0x7f37fc06f4b0 2023-10-08T01:15:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.155+0000 7f37f9d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37fc06f060 0x7f37fc0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.155+0000 7f37f9d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f37fc06f060 0x7f37fc0ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43608/0 (socket says 172.21.15.119:43608) 2023-10-08T01:15:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.155+0000 7f37f9d9b700 1 -- 172.21.15.119:0/1797022087 learned_addr learned my addr 172.21.15.119:0/1797022087 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37fad9d700 1 --2- 172.21.15.119:0/1797022087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc06f4b0 0x7f37fc0ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37f9d9b700 1 -- 172.21.15.119:0/1797022087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc06f4b0 msgr2=0x7f37fc0ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37f9d9b700 1 --2- 172.21.15.119:0/1797022087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc06f4b0 0x7f37fc0ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37f9d9b700 1 -- 172.21.15.119:0/1797022087 >> v1:172.21.15.184:6789/0 conn(0x7f37fc0cb1a0 legacy=0x7f37fc15a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37f9d9b700 1 -- 172.21.15.119:0/1797022087 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37fc15ad20 con 0x7f37fc06f060 2023-10-08T01:15:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37f9d9b700 1 --2- 172.21.15.119:0/1797022087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f060 0x7f37fc0ae0d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f37f00099b0 tx=0x7f37f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d6dd9cf183c9b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:05.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37f959a700 1 -- 172.21.15.119:0/1797022087 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0009ea0 con 0x7f37fc06f060 2023-10-08T01:15:05.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37f959a700 1 -- 172.21.15.119:0/1797022087 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000b8f0 con 0x7f37fc06f060 2023-10-08T01:15:05.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.156+0000 7f37f959a700 1 -- 172.21.15.119:0/1797022087 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000bc90 con 0x7f37fc06f060 2023-10-08T01:15:05.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.157+0000 7f37fb59e700 1 -- 172.21.15.119:0/1797022087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f060 msgr2=0x7f37fc0ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.157+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1797022087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f060 0x7f37fc0ae0d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f37f00099b0 tx=0x7f37f0009250 comp rx=0 tx=0).stop 2023-10-08T01:15:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.157+0000 7f37fb59e700 1 -- 172.21.15.119:0/1797022087 shutdown_connections 2023-10-08T01:15:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.157+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1797022087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc06f4b0 0x7f37fc0ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.157+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1797022087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f060 0x7f37fc0ae0d0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.157+0000 7f37fb59e700 1 -- 172.21.15.119:0/1797022087 >> 172.21.15.119:0/1797022087 conn(0x7f37fc0cc1e0 msgr2=0x7f37fc0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:05.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.157+0000 7f37fb59e700 1 -- 172.21.15.119:0/1797022087 shutdown_connections 2023-10-08T01:15:05.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.157+0000 7f37fb59e700 1 -- 172.21.15.119:0/1797022087 wait complete. 2023-10-08T01:15:05.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.158+0000 7f37fb59e700 1 Processor -- start 2023-10-08T01:15:05.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.159+0000 7f37fb59e700 1 -- start start 2023-10-08T01:15:05.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.159+0000 7f37fb59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37fc06f060 0x7f37fc147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37fb59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f4b0 0x7f37fc0c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37fb59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc0cb1a0 0x7f37fc0c4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37fb59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0aff00 con 0x7f37fc06f4b0 2023-10-08T01:15:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37fb59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc15cc50 con 0x7f37fc06f060 2023-10-08T01:15:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37fb59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37fc0b01a0 con 0x7f37fc0cb1a0 2023-10-08T01:15:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37f9d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f4b0 0x7f37fc0c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37f9d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f4b0 0x7f37fc0c3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43614/0 (socket says 172.21.15.119:43614) 2023-10-08T01:15:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37f9d9b700 1 -- 172.21.15.119:0/1142393532 learned_addr learned my addr 172.21.15.119:0/1142393532 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37fa59c700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37fc06f060 0x7f37fc147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37fad9d700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc0cb1a0 0x7f37fc0c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37f9d9b700 1 -- 172.21.15.119:0/1142393532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc0cb1a0 msgr2=0x7f37fc0c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37f9d9b700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc0cb1a0 0x7f37fc0c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37f9d9b700 1 -- 172.21.15.119:0/1142393532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37fc06f060 msgr2=0x7f37fc147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37f9d9b700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37fc06f060 0x7f37fc147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.160+0000 7f37f9d9b700 1 -- 172.21.15.119:0/1142393532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f00094f0 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.161+0000 7f37f9d9b700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f4b0 0x7f37fc0c3b30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f37fc06ce80 tx=0x7f37f0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.161+0000 7f37ea7fc700 1 -- 172.21.15.119:0/1142393532 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f000ba60 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.161+0000 7f37ea7fc700 1 -- 172.21.15.119:0/1142393532 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f000bbe0 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.161+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37fc0b0480 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.161+0000 7f37ea7fc700 1 -- 172.21.15.119:0/1142393532 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0026680 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.161+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37fc1482e0 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.162+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37fc0b06c0 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.163+0000 7f37ea7fc700 1 -- 172.21.15.119:0/1142393532 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f37f0023030 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.164+0000 7f37ea7fc700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f37dc067170 0x7f37dc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.164+0000 7f37ea7fc700 1 -- 172.21.15.119:0/1142393532 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f37f0085ed0 con 0x7f37fc06f4b0 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.164+0000 7f37fa59c700 1 -- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f37dc067170 msgr2=0x7f37dc069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.164+0000 7f37fa59c700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f37dc067170 0x7f37dc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:05.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.172+0000 7f37ea7fc700 1 -- 172.21.15.119:0/1142393532 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37fc0b06c0 con 0x7f37fc06f4b0 2023-10-08T01:15:05.207 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:05.206+0000 7f9fb3ebf180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:15:05.292 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:05.292+0000 7f9416b94180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:15:05.299 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:05.299+0000 7f9fb3ebf180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:15:05.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.365+0000 7f37fa59c700 1 -- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f37dc067170 msgr2=0x7f37dc069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:05.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.365+0000 7f37fa59c700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f37dc067170 0x7f37dc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.367+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f37fc0b06c0 con 0x7f37fc06f4b0 2023-10-08T01:15:05.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.368+0000 7f37ea7fc700 1 -- 172.21.15.119:0/1142393532 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v121) v1 ==== 104+0+11 (secure 0 0 0) 0x7f37fc0b06c0 con 0x7f37fc06f4b0 2023-10-08T01:15:05.368 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T01:15:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.370+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f37dc067170 msgr2=0x7f37dc069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.370+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f37dc067170 0x7f37dc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.370+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f4b0 msgr2=0x7f37fc0c3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.370+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f4b0 0x7f37fc0c3b30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f37fc06ce80 tx=0x7f37f0017000 comp rx=0 tx=0).stop 2023-10-08T01:15:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.370+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 shutdown_connections 2023-10-08T01:15:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.371+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f37dc067170 0x7f37dc069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.371+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37fc0cb1a0 0x7f37fc0c4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.371+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37fc06f060 0x7f37fc147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.371+0000 7f37fb59e700 1 --2- 172.21.15.119:0/1142393532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37fc06f4b0 0x7f37fc0c3b30 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.371+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 >> 172.21.15.119:0/1142393532 conn(0x7f37fc0cc1e0 msgr2=0x7f37fc155df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:05.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.371+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 shutdown_connections 2023-10-08T01:15:05.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.371+0000 7f37fb59e700 1 -- 172.21.15.119:0/1142393532 wait complete. 2023-10-08T01:15:05.387 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:05.387+0000 7f9416b94180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:15:05.388 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T01:15:05.460 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:05.460+0000 7f9fb3ebf180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:15:05.561 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:05.561+0000 7f9fb3ebf180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:15:05.562 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:05.561+0000 7f9416b94180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:15:05.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.584+0000 7f9a7db9d700 1 Processor -- start 2023-10-08T01:15:05.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.585+0000 7f9a7db9d700 1 -- start start 2023-10-08T01:15:05.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a7db9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a780b4e70 0x7f9a780ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a7db9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9a780b16d0 0x7f9a780b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a7db9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a780b2080 con 0x7f9a7815d240 2023-10-08T01:15:05.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a7db9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a780cfb00 con 0x7f9a780b16d0 2023-10-08T01:15:05.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a7db9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780b5c40 con 0x7f9a780b4e70 2023-10-08T01:15:05.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a7cb9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a780b4e70 0x7f9a780ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a7cb9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a780b4e70 0x7f9a780ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:43564/0 (socket says 172.21.15.119:43564) 2023-10-08T01:15:05.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a7cb9b700 1 -- 172.21.15.119:0/2773166322 learned_addr learned my addr 172.21.15.119:0/2773166322 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:05.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.586+0000 7f9a77fff700 1 --2- 172.21.15.119:0/2773166322 >> v2:172.21.15.184:3300/0 conn(0x7f9a780b16d0 0x7f9a780b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.587+0000 7f9a7cb9b700 1 -- 172.21.15.119:0/2773166322 >> v2:172.21.15.184:3300/0 conn(0x7f9a780b16d0 msgr2=0x7f9a780b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:05.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.587+0000 7f9a7cb9b700 1 --2- 172.21.15.119:0/2773166322 >> v2:172.21.15.184:3300/0 conn(0x7f9a780b16d0 0x7f9a780b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.587+0000 7f9a7cb9b700 1 -- 172.21.15.119:0/2773166322 >> v1:172.21.15.184:6789/0 conn(0x7f9a7815d240 legacy=0x7f9a7815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.587+0000 7f9a7cb9b700 1 -- 172.21.15.119:0/2773166322 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780b5f20 con 0x7f9a780b4e70 2023-10-08T01:15:05.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.587+0000 7f9a7cb9b700 1 --2- 172.21.15.119:0/2773166322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a780b4e70 0x7f9a780ccff0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0099b0 tx=0x7f9a6c009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=924cb4368a665cdd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:05.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.587+0000 7f9a777fe700 1 -- 172.21.15.119:0/2773166322 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c009ea0 con 0x7f9a780b4e70 2023-10-08T01:15:05.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.587+0000 7f9a777fe700 1 -- 172.21.15.119:0/2773166322 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00b860 con 0x7f9a780b4e70 2023-10-08T01:15:05.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.587+0000 7f9a777fe700 1 -- 172.21.15.119:0/2773166322 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00bc00 con 0x7f9a780b4e70 2023-10-08T01:15:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.588+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2773166322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a780b4e70 msgr2=0x7f9a780ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.588+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2773166322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a780b4e70 0x7f9a780ccff0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c0099b0 tx=0x7f9a6c009250 comp rx=0 tx=0).stop 2023-10-08T01:15:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.588+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2773166322 shutdown_connections 2023-10-08T01:15:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.588+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2773166322 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a780b4e70 0x7f9a780ccff0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.588+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2773166322 >> v2:172.21.15.184:3300/0 conn(0x7f9a780b16d0 0x7f9a780b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.588+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2773166322 >> 172.21.15.119:0/2773166322 conn(0x7f9a780d10c0 msgr2=0x7f9a780affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.588+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2773166322 shutdown_connections 2023-10-08T01:15:05.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.588+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2773166322 wait complete. 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.589+0000 7f9a7db9d700 1 Processor -- start 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.590+0000 7f9a7db9d700 1 -- start start 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.590+0000 7f9a7db9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a780b16d0 0x7f9a78153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.590+0000 7f9a7db9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a780b4e70 0x7f9a781536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.590+0000 7f9a7db9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a7815d240 0x7f9a7816d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.590+0000 7f9a7db9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78155fb0 con 0x7f9a780b4e70 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.590+0000 7f9a7db9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a781512d0 con 0x7f9a780b16d0 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.590+0000 7f9a7db9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a78155cd0 con 0x7f9a7815d240 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a780b4e70 0x7f9a781536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a780b4e70 0x7f9a781536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43616/0 (socket says 172.21.15.119:43616) 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 -- 172.21.15.119:0/2031493775 learned_addr learned my addr 172.21.15.119:0/2031493775 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:05.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a7cb9b700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a780b16d0 0x7f9a78153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a7d39c700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a7815d240 0x7f9a7816d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 -- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a7815d240 msgr2=0x7f9a7816d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a7815d240 0x7f9a7816d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 -- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a780b16d0 msgr2=0x7f9a78153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a780b16d0 0x7f9a78153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 -- 172.21.15.119:0/2031493775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c0094f0 con 0x7f9a780b4e70 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a7cb9b700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a780b16d0 0x7f9a78153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.591+0000 7f9a77fff700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a780b4e70 0x7f9a781536b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400ec80 tx=0x7f9a640091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.592+0000 7f9a74ff9700 1 -- 172.21.15.119:0/2031493775 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400c850 con 0x7f9a780b4e70 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.592+0000 7f9a74ff9700 1 -- 172.21.15.119:0/2031493775 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6400c9b0 con 0x7f9a780b4e70 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.592+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a78150c80 con 0x7f9a780b4e70 2023-10-08T01:15:05.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.592+0000 7f9a74ff9700 1 -- 172.21.15.119:0/2031493775 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400cd50 con 0x7f9a780b4e70 2023-10-08T01:15:05.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.592+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7816daa0 con 0x7f9a780b4e70 2023-10-08T01:15:05.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.593+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a78150ae0 con 0x7f9a780b4e70 2023-10-08T01:15:05.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.594+0000 7f9a74ff9700 1 -- 172.21.15.119:0/2031493775 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f9a64013680 con 0x7f9a780b4e70 2023-10-08T01:15:05.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.595+0000 7f9a74ff9700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9a5c067170 0x7f9a5c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:05.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.595+0000 7f9a74ff9700 1 -- 172.21.15.119:0/2031493775 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f9a64084d10 con 0x7f9a780b4e70 2023-10-08T01:15:05.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.595+0000 7f9a7cb9b700 1 -- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9a5c067170 msgr2=0x7f9a5c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:05.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.595+0000 7f9a7cb9b700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9a5c067170 0x7f9a5c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:05.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.598+0000 7f9a74ff9700 1 -- 172.21.15.119:0/2031493775 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a78150ae0 con 0x7f9a780b4e70 2023-10-08T01:15:05.668 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:05.668+0000 7f9416b94180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:15:05.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.796+0000 7f9a7cb9b700 1 -- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9a5c067170 msgr2=0x7f9a5c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:05.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.796+0000 7f9a7cb9b700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9a5c067170 0x7f9a5c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:05.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.797+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a78150ae0 con 0x7f9a780b4e70 2023-10-08T01:15:05.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.797+0000 7f9a74ff9700 1 -- 172.21.15.119:0/2031493775 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v121) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9a78150ae0 con 0x7f9a780b4e70 2023-10-08T01:15:05.797 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T01:15:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.799+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9a5c067170 msgr2=0x7f9a5c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.799+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9a5c067170 0x7f9a5c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.799+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a780b4e70 msgr2=0x7f9a781536b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.799+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a780b4e70 0x7f9a781536b0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9a6400ec80 tx=0x7f9a640091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.800+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 shutdown_connections 2023-10-08T01:15:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.800+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9a5c067170 0x7f9a5c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.800+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a7815d240 0x7f9a7816d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.800+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a780b16d0 0x7f9a78153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.800+0000 7f9a7db9d700 1 --2- 172.21.15.119:0/2031493775 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a780b4e70 0x7f9a781536b0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:05.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.800+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 >> 172.21.15.119:0/2031493775 conn(0x7f9a780d10c0 msgr2=0x7f9a78069ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:05.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.800+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 shutdown_connections 2023-10-08T01:15:05.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:05.800+0000 7f9a7db9d700 1 -- 172.21.15.119:0/2031493775 wait complete. 2023-10-08T01:15:05.819 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:15:06.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.007+0000 7f91feb9d700 1 Processor -- start 2023-10-08T01:15:06.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.007+0000 7f91feb9d700 1 -- start start 2023-10-08T01:15:06.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.008+0000 7f91feb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f8159420 0x7f91f80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.008+0000 7f91feb9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f91f81597f0 0x7f91f80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.008+0000 7f91feb9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91f80b11d0 con 0x7f91f815d230 2023-10-08T01:15:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.008+0000 7f91feb9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f80b14b0 con 0x7f91f81597f0 2023-10-08T01:15:06.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.008+0000 7f91feb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f80b1790 con 0x7f91f8159420 2023-10-08T01:15:06.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fd39a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f91f81597f0 0x7f91f80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fdb9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f8159420 0x7f91f80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fdb9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f8159420 0x7f91f80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:43578/0 (socket says 172.21.15.119:43578) 2023-10-08T01:15:06.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fd39a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f91f81597f0 0x7f91f80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:48452/0 (socket says 172.21.15.119:48452) 2023-10-08T01:15:06.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fdb9b700 1 -- 172.21.15.119:0/3962645214 learned_addr learned my addr 172.21.15.119:0/3962645214 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:06.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fe39c700 1 --1- 172.21.15.119:0/3962645214 >> v1:172.21.15.184:6789/0 conn(0x7f91f815d230 0x7f91f80b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:36050/0 (socket says 172.21.15.119:36050) 2023-10-08T01:15:06.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fdb9b700 1 -- 172.21.15.119:0/3962645214 >> v2:172.21.15.184:3300/0 conn(0x7f91f81597f0 msgr2=0x7f91f80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:06.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fdb9b700 1 --2- 172.21.15.119:0/3962645214 >> v2:172.21.15.184:3300/0 conn(0x7f91f81597f0 0x7f91f80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fdb9b700 1 -- 172.21.15.119:0/3962645214 >> v1:172.21.15.184:6789/0 conn(0x7f91f815d230 legacy=0x7f91f80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fdb9b700 1 -- 172.21.15.119:0/3962645214 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f80b1a70 con 0x7f91f8159420 2023-10-08T01:15:06.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.009+0000 7f91fd39a700 1 --2- 172.21.15.119:0/3962645214 >> v2:172.21.15.184:3300/0 conn(0x7f91f81597f0 0x7f91f80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:06.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.010+0000 7f91fdb9b700 1 --2- 172.21.15.119:0/3962645214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f8159420 0x7f91f80b5f40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91f40099b0 tx=0x7f91f4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639e92d4f0fadd5a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:06.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.010+0000 7f91fcb99700 1 -- 172.21.15.119:0/3962645214 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f4009ea0 con 0x7f91f8159420 2023-10-08T01:15:06.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.010+0000 7f91fcb99700 1 -- 172.21.15.119:0/3962645214 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f400b860 con 0x7f91f8159420 2023-10-08T01:15:06.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.010+0000 7f91fcb99700 1 -- 172.21.15.119:0/3962645214 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f400bc00 con 0x7f91f8159420 2023-10-08T01:15:06.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.010+0000 7f91feb9d700 1 -- 172.21.15.119:0/3962645214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f8159420 msgr2=0x7f91f80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.010+0000 7f91feb9d700 1 --2- 172.21.15.119:0/3962645214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f8159420 0x7f91f80b5f40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f91f40099b0 tx=0x7f91f4009250 comp rx=0 tx=0).stop 2023-10-08T01:15:06.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.011+0000 7f91feb9d700 1 -- 172.21.15.119:0/3962645214 shutdown_connections 2023-10-08T01:15:06.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.011+0000 7f91feb9d700 1 --2- 172.21.15.119:0/3962645214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f8159420 0x7f91f80b5f40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.011+0000 7f91feb9d700 1 --2- 172.21.15.119:0/3962645214 >> v2:172.21.15.184:3300/0 conn(0x7f91f81597f0 0x7f91f80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.011+0000 7f91feb9d700 1 -- 172.21.15.119:0/3962645214 >> 172.21.15.119:0/3962645214 conn(0x7f91f80d0660 msgr2=0x7f91f80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:06.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.011+0000 7f91feb9d700 1 -- 172.21.15.119:0/3962645214 shutdown_connections 2023-10-08T01:15:06.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.011+0000 7f91feb9d700 1 -- 172.21.15.119:0/3962645214 wait complete. 2023-10-08T01:15:06.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.012+0000 7f91feb9d700 1 Processor -- start 2023-10-08T01:15:06.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.012+0000 7f91feb9d700 1 -- start start 2023-10-08T01:15:06.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91feb9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8159420 0x7f91f814d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91feb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f81597f0 0x7f91f814d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91feb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f815d230 0x7f91f814de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91feb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f815cfb0 con 0x7f91f815d230 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91feb9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8152ba0 con 0x7f91f8159420 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91feb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f8152d30 con 0x7f91f81597f0 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91fe39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f815d230 0x7f91f814de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91fe39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f815d230 0x7f91f814de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43622/0 (socket says 172.21.15.119:43622) 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91fe39c700 1 -- 172.21.15.119:0/863847696 learned_addr learned my addr 172.21.15.119:0/863847696 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91fdb9b700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8159420 0x7f91f814d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.013+0000 7f91fd39a700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f81597f0 0x7f91f814d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91fe39c700 1 -- 172.21.15.119:0/863847696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f81597f0 msgr2=0x7f91f814d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91fe39c700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f81597f0 0x7f91f814d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91fe39c700 1 -- 172.21.15.119:0/863847696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8159420 msgr2=0x7f91f814d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91fe39c700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8159420 0x7f91f814d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91fe39c700 1 -- 172.21.15.119:0/863847696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e8009420 con 0x7f91f815d230 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91fe39c700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f815d230 0x7f91f814de60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f91f000cc70 tx=0x7f91f0001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91edffb700 1 -- 172.21.15.119:0/863847696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0005df0 con 0x7f91f815d230 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91edffb700 1 -- 172.21.15.119:0/863847696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f000a5b0 con 0x7f91f815d230 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91edffb700 1 -- 172.21.15.119:0/863847696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f000a930 con 0x7f91f815d230 2023-10-08T01:15:06.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f40094f0 con 0x7f91f815d230 2023-10-08T01:15:06.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.014+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f814f880 con 0x7f91f815d230 2023-10-08T01:15:06.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.016+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f804c3c0 con 0x7f91f815d230 2023-10-08T01:15:06.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.016+0000 7f91edffb700 1 -- 172.21.15.119:0/863847696 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f91f000ace0 con 0x7f91f815d230 2023-10-08T01:15:06.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.017+0000 7f91edffb700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91dc067170 0x7f91dc069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.017+0000 7f91edffb700 1 -- 172.21.15.119:0/863847696 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f91f0084ab0 con 0x7f91f815d230 2023-10-08T01:15:06.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.017+0000 7f91fdb9b700 1 -- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91dc067170 msgr2=0x7f91dc069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:06.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.017+0000 7f91fdb9b700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91dc067170 0x7f91dc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:06.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.021+0000 7f91edffb700 1 -- 172.21.15.119:0/863847696 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91f0051880 con 0x7f91f815d230 2023-10-08T01:15:06.045 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:06.045+0000 7f9fb3ebf180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:15:06.137 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:06.137+0000 7f9fb3ebf180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:15:06.175 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:06.175+0000 7f9416b94180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:15:06.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.218+0000 7f91fdb9b700 1 -- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91dc067170 msgr2=0x7f91dc069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:06.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.218+0000 7f91fdb9b700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91dc067170 0x7f91dc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:06.227 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:06.227+0000 7f9fb3ebf180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:15:06.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.264+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91f8152e70 con 0x7f91f815d230 2023-10-08T01:15:06.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.265+0000 7f91edffb700 1 -- 172.21.15.119:0/863847696 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f91f0054ea0 con 0x7f91f815d230 2023-10-08T01:15:06.265 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:06.265 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7141,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3118853373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3118853373},{"type":"v1","addr":"172.21.15.184:6834","nonce":3118853373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7144,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/737543181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":737543181},{"type":"v1","addr":"172.21.15.184:6835","nonce":737543181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/2667608886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2667608886},{"type":"v1","addr":"172.21.15.119:6834","nonce":2667608886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:14:22.219941+0000","modified":"2023-10-08T01:14:25.788077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7680},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7680":{"gid":7680,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1713175169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1713175169},{"type":"v1","addr":"172.21.15.119:6835","nonce":1713175169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.267+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91dc067170 msgr2=0x7f91dc069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.267+0000 7f91feb9d700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91dc067170 0x7f91dc069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.267+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f815d230 msgr2=0x7f91f814de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.267+0000 7f91feb9d700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f815d230 0x7f91f814de60 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f91f000cc70 tx=0x7f91f0001a70 comp rx=0 tx=0).stop 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.267+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 shutdown_connections 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.268+0000 7f91feb9d700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91dc067170 0x7f91dc069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.268+0000 7f91feb9d700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f815d230 0x7f91f814de60 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.268+0000 7f91feb9d700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91f8159420 0x7f91f814d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.268+0000 7f91feb9d700 1 --2- 172.21.15.119:0/863847696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91f81597f0 0x7f91f814d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.268+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 >> 172.21.15.119:0/863847696 conn(0x7f91f80d0660 msgr2=0x7f91f80b5d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:06.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.268+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 shutdown_connections 2023-10-08T01:15:06.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.268+0000 7f91feb9d700 1 -- 172.21.15.119:0/863847696 wait complete. 2023-10-08T01:15:06.269 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 55 2023-10-08T01:15:06.274 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:06.274+0000 7f9416b94180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:15:06.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:15:06.371 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:06.371+0000 7f9416b94180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:15:06.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.473+0000 7f0edc9fe700 1 Processor -- start 2023-10-08T01:15:06.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.473+0000 7f0edc9fe700 1 -- start start 2023-10-08T01:15:06.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0edc9fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ed806c0f0 0x7f0ed81642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0edc9fe700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed80b40f0 con 0x7f0ed80ad870 2023-10-08T01:15:06.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0edc9fe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ed8069640 con 0x7f0ed8142ea0 2023-10-08T01:15:06.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0edc9fe700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ed8069920 con 0x7f0ed806c0f0 2023-10-08T01:15:06.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0ed759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ed806c0f0 0x7f0ed81642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0ed659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0ed80ad870 0x7f0ed806b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34226/0 (socket says 172.21.15.119:34226) 2023-10-08T01:15:06.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0ed659c700 1 -- 172.21.15.119:0/2831064647 learned_addr learned my addr 172.21.15.119:0/2831064647 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0ed759e700 1 -- 172.21.15.119:0/2831064647 >> v1:172.21.15.184:6789/0 conn(0x7f0ed8142ea0 legacy=0x7f0ed80b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0ed759e700 1 -- 172.21.15.119:0/2831064647 >> v1:172.21.15.119:6789/0 conn(0x7f0ed80ad870 legacy=0x7f0ed806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.476+0000 7f0ed759e700 1 -- 172.21.15.119:0/2831064647 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed8069c00 con 0x7f0ed806c0f0 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.475+0000 7f0ed5d9b700 1 -- 172.21.15.119:0/2831064647 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266662811 0 0) 0x7f0ed8069640 con 0x7f0ed8142ea0 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.476+0000 7f0ed759e700 1 --2- 172.21.15.119:0/2831064647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed81642e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009e60 tx=0x7f0ed0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84b7fabe82ab60 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.476+0000 7f0ed5d9b700 1 -- 172.21.15.119:0/2831064647 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed000e070 con 0x7f0ed806c0f0 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.476+0000 7f0ed5d9b700 1 -- 172.21.15.119:0/2831064647 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed000bb30 con 0x7f0ed806c0f0 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.476+0000 7f0ed5d9b700 1 -- 172.21.15.119:0/2831064647 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed0012430 con 0x7f0ed806c0f0 2023-10-08T01:15:06.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.476+0000 7f0edc9fe700 1 -- 172.21.15.119:0/2831064647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 msgr2=0x7f0ed81642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.477+0000 7f0edc9fe700 1 --2- 172.21.15.119:0/2831064647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed81642e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f0ed0009e60 tx=0x7f0ed0009780 comp rx=0 tx=0).stop 2023-10-08T01:15:06.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.477+0000 7f0edc9fe700 1 -- 172.21.15.119:0/2831064647 shutdown_connections 2023-10-08T01:15:06.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.477+0000 7f0edc9fe700 1 --2- 172.21.15.119:0/2831064647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed81642e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.477+0000 7f0edc9fe700 1 -- 172.21.15.119:0/2831064647 >> 172.21.15.119:0/2831064647 conn(0x7f0ed80ce430 msgr2=0x7f0ed80ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:06.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.477+0000 7f0edc9fe700 1 -- 172.21.15.119:0/2831064647 shutdown_connections 2023-10-08T01:15:06.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.477+0000 7f0edc9fe700 1 -- 172.21.15.119:0/2831064647 wait complete. 2023-10-08T01:15:06.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.478+0000 7f0edc9fe700 1 Processor -- start 2023-10-08T01:15:06.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.478+0000 7f0edc9fe700 1 -- start start 2023-10-08T01:15:06.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0edc9fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed80c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0edc9fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed8142ea0 0x7f0ed815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0edc9fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed8169f10 0x7f0ed80c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0edc9fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed816d510 con 0x7f0ed806c0f0 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0edc9fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed816a400 con 0x7f0ed8142ea0 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0edc9fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ed816a6e0 con 0x7f0ed8169f10 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0ed6d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed80c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0ed6d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed80c7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43636/0 (socket says 172.21.15.119:43636) 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.479+0000 7f0ed6d9d700 1 -- 172.21.15.119:0/3554903183 learned_addr learned my addr 172.21.15.119:0/3554903183 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed659c700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed8142ea0 0x7f0ed815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed759e700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed8169f10 0x7f0ed80c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed6d9d700 1 -- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed8169f10 msgr2=0x7f0ed80c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed6d9d700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed8169f10 0x7f0ed80c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed6d9d700 1 -- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed8142ea0 msgr2=0x7f0ed815e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed6d9d700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed8142ea0 0x7f0ed815e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed6d9d700 1 -- 172.21.15.119:0/3554903183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed0009a20 con 0x7f0ed806c0f0 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed759e700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed8169f10 0x7f0ed80c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ed6d9d700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed80c7ff0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c530 tx=0x7f0ec800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ebeffd700 1 -- 172.21.15.119:0/3554903183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8011590 con 0x7f0ed806c0f0 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ebeffd700 1 -- 172.21.15.119:0/3554903183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec80116f0 con 0x7f0ed806c0f0 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ed816a9c0 con 0x7f0ed806c0f0 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0ebeffd700 1 -- 172.21.15.119:0/3554903183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8011ab0 con 0x7f0ed806c0f0 2023-10-08T01:15:06.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.480+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ed815af90 con 0x7f0ed806c0f0 2023-10-08T01:15:06.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.481+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ed806bc60 con 0x7f0ed806c0f0 2023-10-08T01:15:06.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.482+0000 7f0ebeffd700 1 -- 172.21.15.119:0/3554903183 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f0ec801b3b0 con 0x7f0ed806c0f0 2023-10-08T01:15:06.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.483+0000 7f0ebeffd700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0ec4067170 0x7f0ec4069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.483+0000 7f0ebeffd700 1 -- 172.21.15.119:0/3554903183 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7f0ec8084090 con 0x7f0ed806c0f0 2023-10-08T01:15:06.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.483+0000 7f0ed659c700 1 -- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0ec4067170 msgr2=0x7f0ec4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:06.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.483+0000 7f0ed659c700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0ec4067170 0x7f0ec4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:06.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.487+0000 7f0ebeffd700 1 -- 172.21.15.119:0/3554903183 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ec8050de0 con 0x7f0ed806c0f0 2023-10-08T01:15:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.684+0000 7f0ed659c700 1 -- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0ec4067170 msgr2=0x7f0ec4069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:06.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.684+0000 7f0ed659c700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0ec4067170 0x7f0ec4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:06.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.688+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ed804c3c0 con 0x7f0ed806c0f0 2023-10-08T01:15:06.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.689+0000 7f0ebeffd700 1 -- 172.21.15.119:0/3554903183 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+16380 (secure 0 0 0) 0x7f0ec8017090 con 0x7f0ed806c0f0 2023-10-08T01:15:06.689 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:06.689 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":121,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:14:42.347943+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:14:20.330219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T01:14:21.202759+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1219956712":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/2381954001":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/546958670":"2023-10-09T01:14:42.347907+0000","172.21.15.119:6837/85511":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/309487327":"2023-10-09T01:14:42.347907+0000","172.21.15.119:6836/85511":"2023-10-09T01:14:42.347907+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:15:06.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0ec4067170 msgr2=0x7f0ec4069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0ec4067170 0x7f0ec4069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 msgr2=0x7f0ed80c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed80c7ff0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f0ec800c530 tx=0x7f0ec800c8f0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 shutdown_connections 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0ec4067170 0x7f0ec4069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ed8169f10 0x7f0ed80c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ed806c0f0 0x7f0ed80c7ff0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 --2- 172.21.15.119:0/3554903183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ed8142ea0 0x7f0ed815e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 >> 172.21.15.119:0/3554903183 conn(0x7f0ed80ce430 msgr2=0x7f0ed80b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 shutdown_connections 2023-10-08T01:15:06.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.691+0000 7f0edc9fe700 1 -- 172.21.15.119:0/3554903183 wait complete. 2023-10-08T01:15:06.710 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T01:15:06.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:15:06.849 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:06.848+0000 7f9fb3ebf180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:15:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.904+0000 7ff46499d700 1 Processor -- start 2023-10-08T01:15:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.905+0000 7ff46499d700 1 -- start start 2023-10-08T01:15:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff46499d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460166530 0x7ff4600c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff46499d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff4600caad0 0x7ff4600872f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff46499d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff460069cd0 con 0x7ff460087830 2023-10-08T01:15:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff46499d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff460166900 con 0x7ff4600caad0 2023-10-08T01:15:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff46499d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff460087c70 con 0x7ff460166530 2023-10-08T01:15:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff45f7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460166530 0x7ff4600c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff45effd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff4600caad0 0x7ff4600872f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff45effd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff4600caad0 0x7ff4600872f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:48488/0 (socket says 172.21.15.119:48488) 2023-10-08T01:15:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff45f7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460166530 0x7ff4600c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:43594/0 (socket says 172.21.15.119:43594) 2023-10-08T01:15:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.907+0000 7ff45effd700 1 -- 172.21.15.119:0/3226341083 learned_addr learned my addr 172.21.15.119:0/3226341083 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.908+0000 7ff45effd700 1 -- 172.21.15.119:0/3226341083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460166530 msgr2=0x7ff4600c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.908+0000 7ff45effd700 1 --2- 172.21.15.119:0/3226341083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460166530 0x7ff4600c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.908+0000 7ff45effd700 1 -- 172.21.15.119:0/3226341083 >> v1:172.21.15.119:6789/0 conn(0x7ff460087830 legacy=0x7ff46006a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.908+0000 7ff45effd700 1 -- 172.21.15.119:0/3226341083 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4600696c0 con 0x7ff4600caad0 2023-10-08T01:15:06.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.908+0000 7ff45effd700 1 --2- 172.21.15.119:0/3226341083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4600caad0 0x7ff4600872f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff454006260 tx=0x7ff45400b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c800f5724de429f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.908+0000 7ff45e7fc700 1 -- 172.21.15.119:0/3226341083 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff454010260 con 0x7ff4600caad0 2023-10-08T01:15:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.908+0000 7ff45e7fc700 1 -- 172.21.15.119:0/3226341083 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff454014020 con 0x7ff4600caad0 2023-10-08T01:15:06.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.908+0000 7ff45e7fc700 1 -- 172.21.15.119:0/3226341083 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4540105d0 con 0x7ff4600caad0 2023-10-08T01:15:06.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.909+0000 7ff46499d700 1 -- 172.21.15.119:0/3226341083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4600caad0 msgr2=0x7ff4600872f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.909+0000 7ff46499d700 1 --2- 172.21.15.119:0/3226341083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4600caad0 0x7ff4600872f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff454006260 tx=0x7ff45400b640 comp rx=0 tx=0).stop 2023-10-08T01:15:06.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.909+0000 7ff46499d700 1 -- 172.21.15.119:0/3226341083 shutdown_connections 2023-10-08T01:15:06.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.909+0000 7ff46499d700 1 --2- 172.21.15.119:0/3226341083 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460166530 0x7ff4600c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.909+0000 7ff46499d700 1 --2- 172.21.15.119:0/3226341083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4600caad0 0x7ff4600872f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.909+0000 7ff46499d700 1 -- 172.21.15.119:0/3226341083 >> 172.21.15.119:0/3226341083 conn(0x7ff4600cc1f0 msgr2=0x7ff4600cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:06.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.909+0000 7ff46499d700 1 -- 172.21.15.119:0/3226341083 shutdown_connections 2023-10-08T01:15:06.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.910+0000 7ff46499d700 1 -- 172.21.15.119:0/3226341083 wait complete. 2023-10-08T01:15:06.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.913+0000 7ff46499d700 1 Processor -- start 2023-10-08T01:15:06.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.913+0000 7ff46499d700 1 -- start start 2023-10-08T01:15:06.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.914+0000 7ff46499d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460087830 0x7ff460086de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.914+0000 7ff46499d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4600caad0 0x7ff46015fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.914+0000 7ff46499d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff460166530 0x7ff460179bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.914+0000 7ff46499d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4600af300 con 0x7ff460166530 2023-10-08T01:15:06.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.914+0000 7ff46499d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46017a0f0 con 0x7ff4600caad0 2023-10-08T01:15:06.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.914+0000 7ff46499d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff46017a370 con 0x7ff460087830 2023-10-08T01:15:06.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45f7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460087830 0x7ff460086de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45f7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460087830 0x7ff460086de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:43604/0 (socket says 172.21.15.119:43604) 2023-10-08T01:15:06.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45f7fe700 1 -- 172.21.15.119:0/2667282473 learned_addr learned my addr 172.21.15.119:0/2667282473 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:06.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45ffff700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff460166530 0x7ff460179bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:06.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45f7fe700 1 -- 172.21.15.119:0/2667282473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4600caad0 msgr2=0x7ff46015fc80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T01:15:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45f7fe700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4600caad0 0x7ff46015fc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45f7fe700 1 -- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff460166530 msgr2=0x7ff460179bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45f7fe700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff460166530 0x7ff460179bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45f7fe700 1 -- 172.21.15.119:0/2667282473 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff454006030 con 0x7ff460087830 2023-10-08T01:15:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.915+0000 7ff45ffff700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff460166530 0x7ff460179bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.916+0000 7ff45f7fe700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460087830 0x7ff460086de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff45000d540 tx=0x7ff450008040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.916+0000 7ff4477fe700 1 -- 172.21.15.119:0/2667282473 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff45000dea0 con 0x7ff460087830 2023-10-08T01:15:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.916+0000 7ff4477fe700 1 -- 172.21.15.119:0/2667282473 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff450017070 con 0x7ff460087830 2023-10-08T01:15:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.916+0000 7ff4477fe700 1 -- 172.21.15.119:0/2667282473 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff450013580 con 0x7ff460087830 2023-10-08T01:15:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.916+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46017a650 con 0x7ff460087830 2023-10-08T01:15:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.916+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4600aef90 con 0x7ff460087830 2023-10-08T01:15:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.918+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46004c3c0 con 0x7ff460087830 2023-10-08T01:15:06.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.920+0000 7ff4477fe700 1 -- 172.21.15.119:0/2667282473 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7ff4500123e0 con 0x7ff460087830 2023-10-08T01:15:06.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.921+0000 7ff4477fe700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff44c0671f0 0x7ff44c0696a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:06.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.922+0000 7ff4477fe700 1 -- 172.21.15.119:0/2667282473 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 6355+0+0 (secure 0 0 0) 0x7ff450085200 con 0x7ff460087830 2023-10-08T01:15:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.923+0000 7ff45effd700 1 -- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff44c0671f0 msgr2=0x7ff44c0696a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:06.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.923+0000 7ff45effd700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff44c0671f0 0x7ff44c0696a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:06.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:06.930+0000 7ff4477fe700 1 -- 172.21.15.119:0/2667282473 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff450055940 con 0x7ff460087830 2023-10-08T01:15:07.024 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:07.024+0000 7f9416b94180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:15:07.027 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:07.026+0000 7f9fb3ebf180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:15:07.122 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:07.121+0000 7f9fb3ebf180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:15:07.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.124+0000 7ff45effd700 1 -- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff44c0671f0 msgr2=0x7ff44c0696a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:07.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.124+0000 7ff45effd700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff44c0671f0 0x7ff44c0696a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:07.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.194+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4600c7b20 con 0x7ff460087830 2023-10-08T01:15:07.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.195+0000 7ff4477fe700 1 -- 172.21.15.119:0/2667282473 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4608 (secure 0 0 0) 0x7ff4500554d0 con 0x7ff460087830 2023-10-08T01:15:07.195 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:07.195 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7141,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3118853373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3118853373},{"type":"v1","addr":"172.21.15.184:6834","nonce":3118853373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7144,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/737543181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":737543181},{"type":"v1","addr":"172.21.15.184:6835","nonce":737543181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/2667608886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2667608886},{"type":"v1","addr":"172.21.15.119:6834","nonce":2667608886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:14:22.219941+0000","modified":"2023-10-08T01:14:25.788077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7680},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7680":{"gid":7680,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1713175169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1713175169},{"type":"v1","addr":"172.21.15.119:6835","nonce":1713175169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T01:15:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff44c0671f0 msgr2=0x7ff44c0696a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff44c0671f0 0x7ff44c0696a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460087830 msgr2=0x7ff460086de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460087830 0x7ff460086de0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff45000d540 tx=0x7ff450008040 comp rx=0 tx=0).stop 2023-10-08T01:15:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 shutdown_connections 2023-10-08T01:15:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff44c0671f0 0x7ff44c0696a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff460087830 0x7ff460086de0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff460166530 0x7ff460179bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 --2- 172.21.15.119:0/2667282473 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff4600caad0 0x7ff46015fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.197+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 >> 172.21.15.119:0/2667282473 conn(0x7ff4600cc1f0 msgr2=0x7ff4600cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:07.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.198+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 shutdown_connections 2023-10-08T01:15:07.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.198+0000 7ff46499d700 1 -- 172.21.15.119:0/2667282473 wait complete. 2023-10-08T01:15:07.199 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 55 2023-10-08T01:15:07.202 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:07.202+0000 7f9416b94180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:15:07.216 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:15:07.308 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:07.308+0000 7f9416b94180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:15:07.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.407+0000 7f6b76d9d700 1 Processor -- start 2023-10-08T01:15:07.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.407+0000 7f6b76d9d700 1 -- start start 2023-10-08T01:15:07.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.408+0000 7f6b76d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6b70159420 0x7f6b700b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b76d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b7015d230 0x7f6b7015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b76d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b700b2980 con 0x7f6b701597f0 2023-10-08T01:15:07.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b76d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7015dbb0 con 0x7f6b70159420 2023-10-08T01:15:07.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b76d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700b11d0 con 0x7f6b7015d230 2023-10-08T01:15:07.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b75d9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6b70159420 0x7f6b700b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b7659c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b7015d230 0x7f6b7015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b7659c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b7015d230 0x7f6b7015d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:43606/0 (socket says 172.21.15.119:43606) 2023-10-08T01:15:07.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b7559a700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f6b701597f0 0x7f6b700b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:36072/0 (socket says 172.21.15.119:36072) 2023-10-08T01:15:07.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b7659c700 1 -- 172.21.15.119:0/1179555973 learned_addr learned my addr 172.21.15.119:0/1179555973 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:07.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b75d9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6b70159420 0x7f6b700b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:48506/0 (socket says 172.21.15.119:48506) 2023-10-08T01:15:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b74d99700 1 -- 172.21.15.119:0/1179555973 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186981759 0 0) 0x7f6b700b2980 con 0x7f6b701597f0 2023-10-08T01:15:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b74d99700 1 -- 172.21.15.119:0/1179555973 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b5c003570 con 0x7f6b701597f0 2023-10-08T01:15:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b7659c700 1 -- 172.21.15.119:0/1179555973 >> v2:172.21.15.184:3300/0 conn(0x7f6b70159420 msgr2=0x7f6b700b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b7659c700 1 --2- 172.21.15.119:0/1179555973 >> v2:172.21.15.184:3300/0 conn(0x7f6b70159420 0x7f6b700b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.409+0000 7f6b7659c700 1 -- 172.21.15.119:0/1179555973 >> v1:172.21.15.184:6789/0 conn(0x7f6b701597f0 legacy=0x7f6b700b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.410+0000 7f6b7659c700 1 -- 172.21.15.119:0/1179555973 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b700b14b0 con 0x7f6b7015d230 2023-10-08T01:15:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.410+0000 7f6b75d9b700 1 --2- 172.21.15.119:0/1179555973 >> v2:172.21.15.184:3300/0 conn(0x7f6b70159420 0x7f6b700b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.410+0000 7f6b7659c700 1 --2- 172.21.15.119:0/1179555973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b7015d230 0x7f6b7015d670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009f20 tx=0x7f6b68009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a59c8316f657f14 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.410+0000 7f6b74d99700 1 -- 172.21.15.119:0/1179555973 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6800e070 con 0x7f6b7015d230 2023-10-08T01:15:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.410+0000 7f6b74d99700 1 -- 172.21.15.119:0/1179555973 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6800b970 con 0x7f6b7015d230 2023-10-08T01:15:07.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.410+0000 7f6b74d99700 1 -- 172.21.15.119:0/1179555973 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680092e0 con 0x7f6b7015d230 2023-10-08T01:15:07.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.411+0000 7f6b76d9d700 1 -- 172.21.15.119:0/1179555973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b7015d230 msgr2=0x7f6b7015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.411+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/1179555973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b7015d230 0x7f6b7015d670 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009f20 tx=0x7f6b68009840 comp rx=0 tx=0).stop 2023-10-08T01:15:07.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.411+0000 7f6b76d9d700 1 -- 172.21.15.119:0/1179555973 shutdown_connections 2023-10-08T01:15:07.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.411+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/1179555973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b7015d230 0x7f6b7015d670 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.411+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/1179555973 >> v2:172.21.15.184:3300/0 conn(0x7f6b70159420 0x7f6b700b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.411+0000 7f6b76d9d700 1 -- 172.21.15.119:0/1179555973 >> 172.21.15.119:0/1179555973 conn(0x7f6b700d0660 msgr2=0x7f6b700d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:07.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.411+0000 7f6b76d9d700 1 -- 172.21.15.119:0/1179555973 shutdown_connections 2023-10-08T01:15:07.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.411+0000 7f6b76d9d700 1 -- 172.21.15.119:0/1179555973 wait complete. 2023-10-08T01:15:07.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.412+0000 7f6b76d9d700 1 Processor -- start 2023-10-08T01:15:07.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.413+0000 7f6b76d9d700 1 -- start start 2023-10-08T01:15:07.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.413+0000 7f6b76d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b70159420 0x7f6b70154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.413+0000 7f6b76d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b701597f0 0x7f6b70154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b76d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b7015d230 0x7f6b700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b76d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7015a7e0 con 0x7f6b7015d230 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b76d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b701545d0 con 0x7f6b70159420 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b76d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b700c5bb0 con 0x7f6b701597f0 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b75d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b70159420 0x7f6b70154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b7659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b7015d230 0x7f6b700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b7559a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b701597f0 0x7f6b70154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b75d9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b70159420 0x7f6b70154800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:48518/0 (socket says 172.21.15.119:48518) 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b7659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b7015d230 0x7f6b700b4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43658/0 (socket says 172.21.15.119:43658) 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b75d9b700 1 -- 172.21.15.119:0/3777262577 learned_addr learned my addr 172.21.15.119:0/3777262577 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b75d9b700 1 -- 172.21.15.119:0/3777262577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b701597f0 msgr2=0x7f6b70154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b75d9b700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b701597f0 0x7f6b70154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b75d9b700 1 -- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b7015d230 msgr2=0x7f6b700b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b75d9b700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b7015d230 0x7f6b700b4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.414+0000 7f6b75d9b700 1 -- 172.21.15.119:0/3777262577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6c009420 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.415+0000 7f6b75d9b700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b70159420 0x7f6b70154800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00ec80 tx=0x7f6b6c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.415+0000 7f6b65ffb700 1 -- 172.21.15.119:0/3777262577 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00c920 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.415+0000 7f6b65ffb700 1 -- 172.21.15.119:0/3777262577 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c00ca80 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.415+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b68009ae0 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.415+0000 7f6b65ffb700 1 -- 172.21.15.119:0/3777262577 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00ce00 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.415+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b700c6170 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.416+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7004c3c0 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.419+0000 7f6b65ffb700 1 -- 172.21.15.119:0/3777262577 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f6b6c013680 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.420+0000 7f6b65ffb700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6b54067170 0x7f6b54069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.420+0000 7f6b65ffb700 1 -- 172.21.15.119:0/3777262577 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f6b6c085730 con 0x7f6b70159420 2023-10-08T01:15:07.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.420+0000 7f6b7559a700 1 -- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6b54067170 msgr2=0x7f6b54069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:07.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.420+0000 7f6b7559a700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6b54067170 0x7f6b54069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:07.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.422+0000 7f6b65ffb700 1 -- 172.21.15.119:0/3777262577 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b6c052430 con 0x7f6b70159420 2023-10-08T01:15:07.427 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:07.426+0000 7f9fb3ebf180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:15:07.522 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:07.521+0000 7f9fb3ebf180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:15:07.619 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:07.619+0000 7f9416b94180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:15:07.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.621+0000 7f6b7559a700 1 -- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6b54067170 msgr2=0x7f6b54069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:07.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.621+0000 7f6b7559a700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6b54067170 0x7f6b54069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:07.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.669+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b700b1790 con 0x7f6b70159420 2023-10-08T01:15:07.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.670+0000 7f6b65ffb700 1 -- 172.21.15.119:0/3777262577 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4608 (secure 0 0 0) 0x7f6b6c055a50 con 0x7f6b70159420 2023-10-08T01:15:07.673 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:07.673 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7141,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3118853373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3118853373},{"type":"v1","addr":"172.21.15.184:6834","nonce":3118853373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7144,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/737543181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":737543181},{"type":"v1","addr":"172.21.15.184:6835","nonce":737543181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/2667608886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2667608886},{"type":"v1","addr":"172.21.15.119:6834","nonce":2667608886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:14:22.219941+0000","modified":"2023-10-08T01:14:25.788077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7680},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7680":{"gid":7680,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1713175169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1713175169},{"type":"v1","addr":"172.21.15.119:6835","nonce":1713175169}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-10-08T01:15:07.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6b54067170 msgr2=0x7f6b54069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6b54067170 0x7f6b54069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b70159420 msgr2=0x7f6b70154800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b70159420 0x7f6b70154800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00ec80 tx=0x7f6b6c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 shutdown_connections 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6b54067170 0x7f6b54069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6b7015d230 0x7f6b700b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6b70159420 0x7f6b70154800 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 --2- 172.21.15.119:0/3777262577 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6b701597f0 0x7f6b70154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 >> 172.21.15.119:0/3777262577 conn(0x7f6b700d0660 msgr2=0x7f6b70069c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 shutdown_connections 2023-10-08T01:15:07.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.672+0000 7f6b76d9d700 1 -- 172.21.15.119:0/3777262577 wait complete. 2023-10-08T01:15:07.676 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 55 2023-10-08T01:15:07.691 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:15:07.719 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:07.719+0000 7f9416b94180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:15:07.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.877+0000 7fb884f9d700 1 Processor -- start 2023-10-08T01:15:07.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.877+0000 7fb884f9d700 1 -- start start 2023-10-08T01:15:07.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.878+0000 7fb884f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8801597f0 0x7fb8800b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.878+0000 7fb884f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb88015d230 0x7fb88015d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb884f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88015dbb0 con 0x7fb880159420 2023-10-08T01:15:07.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb884f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8800b11d0 con 0x7fb8801597f0 2023-10-08T01:15:07.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb884f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800b14b0 con 0x7fb88015d230 2023-10-08T01:15:07.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87f7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb880159420 0x7fb8800b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34236/0 (socket says 172.21.15.119:34236) 2023-10-08T01:15:07.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87f7fe700 1 -- 172.21.15.119:0/2815267859 learned_addr learned my addr 172.21.15.119:0/2815267859 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:07.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87ffff700 1 --2- 172.21.15.119:0/2815267859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb88015d230 0x7fb88015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87effd700 1 --2- 172.21.15.119:0/2815267859 >> v2:172.21.15.184:3300/0 conn(0x7fb8801597f0 0x7fb8800b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87e7fc700 1 -- 172.21.15.119:0/2815267859 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646968538 0 0) 0x7fb88015dbb0 con 0x7fb880159420 2023-10-08T01:15:07.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87e7fc700 1 -- 172.21.15.119:0/2815267859 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb868003570 con 0x7fb880159420 2023-10-08T01:15:07.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87effd700 1 -- 172.21.15.119:0/2815267859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb88015d230 msgr2=0x7fb88015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:07.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87effd700 1 --2- 172.21.15.119:0/2815267859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb88015d230 0x7fb88015d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87effd700 1 -- 172.21.15.119:0/2815267859 >> v1:172.21.15.119:6789/0 conn(0x7fb880159420 legacy=0x7fb8800b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.879+0000 7fb87effd700 1 -- 172.21.15.119:0/2815267859 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8800b1790 con 0x7fb8801597f0 2023-10-08T01:15:07.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.880+0000 7fb87effd700 1 --2- 172.21.15.119:0/2815267859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8801597f0 0x7fb8800b2270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb8740098e0 tx=0x7fb8740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1eb2ca825d4fb25d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:07.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.880+0000 7fb87e7fc700 1 -- 172.21.15.119:0/2815267859 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874011630 con 0x7fb8801597f0 2023-10-08T01:15:07.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.880+0000 7fb87e7fc700 1 -- 172.21.15.119:0/2815267859 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874011790 con 0x7fb8801597f0 2023-10-08T01:15:07.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.880+0000 7fb87e7fc700 1 -- 172.21.15.119:0/2815267859 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874011b30 con 0x7fb8801597f0 2023-10-08T01:15:07.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.881+0000 7fb884f9d700 1 -- 172.21.15.119:0/2815267859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8801597f0 msgr2=0x7fb8800b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.881+0000 7fb884f9d700 1 --2- 172.21.15.119:0/2815267859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8801597f0 0x7fb8800b2270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb8740098e0 tx=0x7fb8740091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.881+0000 7fb884f9d700 1 -- 172.21.15.119:0/2815267859 shutdown_connections 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.881+0000 7fb884f9d700 1 --2- 172.21.15.119:0/2815267859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb88015d230 0x7fb88015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.881+0000 7fb884f9d700 1 --2- 172.21.15.119:0/2815267859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8801597f0 0x7fb8800b2270 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.881+0000 7fb884f9d700 1 -- 172.21.15.119:0/2815267859 >> 172.21.15.119:0/2815267859 conn(0x7fb8800d0660 msgr2=0x7fb8800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.881+0000 7fb884f9d700 1 -- 172.21.15.119:0/2815267859 shutdown_connections 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.881+0000 7fb884f9d700 1 -- 172.21.15.119:0/2815267859 wait complete. 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.882+0000 7fb884f9d700 1 Processor -- start 2023-10-08T01:15:07.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.882+0000 7fb884f9d700 1 -- start start 2023-10-08T01:15:07.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb884f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb880159420 0x7fb8800cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb884f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8801597f0 0x7fb88015b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb884f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb88015d230 0x7fb880154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb884f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88015bb90 con 0x7fb8801597f0 2023-10-08T01:15:07.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb884f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88015c670 con 0x7fb88015d230 2023-10-08T01:15:07.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb884f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88015bdd0 con 0x7fb880159420 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb87effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8801597f0 0x7fb88015b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb87effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8801597f0 0x7fb88015b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43674/0 (socket says 172.21.15.119:43674) 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.883+0000 7fb87effd700 1 -- 172.21.15.119:0/3915278842 learned_addr learned my addr 172.21.15.119:0/3915278842 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87f7fe700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb880159420 0x7fb8800cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87ffff700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb88015d230 0x7fb880154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87effd700 1 -- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb880159420 msgr2=0x7fb8800cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87effd700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb880159420 0x7fb8800cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87effd700 1 -- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb88015d230 msgr2=0x7fb880154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87effd700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb88015d230 0x7fb880154b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87effd700 1 -- 172.21.15.119:0/3915278842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb874009420 con 0x7fb8801597f0 2023-10-08T01:15:07.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87f7fe700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb880159420 0x7fb8800cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:07.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb87effd700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8801597f0 0x7fb88015b650 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb874013000 tx=0x7fb874017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:07.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb8677fe700 1 -- 172.21.15.119:0/3915278842 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874000d10 con 0x7fb8801597f0 2023-10-08T01:15:07.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88015c0b0 con 0x7fb8801597f0 2023-10-08T01:15:07.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.884+0000 7fb8677fe700 1 -- 172.21.15.119:0/3915278842 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb874011b10 con 0x7fb8801597f0 2023-10-08T01:15:07.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.885+0000 7fb8677fe700 1 -- 172.21.15.119:0/3915278842 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb874025660 con 0x7fb8801597f0 2023-10-08T01:15:07.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.885+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8800c57b0 con 0x7fb8801597f0 2023-10-08T01:15:07.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.886+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8800c59a0 con 0x7fb8801597f0 2023-10-08T01:15:07.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.887+0000 7fb8677fe700 1 -- 172.21.15.119:0/3915278842 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fb874025a50 con 0x7fb8801597f0 2023-10-08T01:15:07.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.887+0000 7fb8677fe700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb86c067170 0x7fb86c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:07.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.887+0000 7fb8677fe700 1 -- 172.21.15.119:0/3915278842 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6411+0+0 (secure 0 0 0) 0x7fb874086410 con 0x7fb8801597f0 2023-10-08T01:15:07.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.888+0000 7fb87f7fe700 1 -- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb86c067170 msgr2=0x7fb86c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:07.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.888+0000 7fb87f7fe700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb86c067170 0x7fb86c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:07.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:07.895+0000 7fb8677fe700 1 -- 172.21.15.119:0/3915278842 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb874053110 con 0x7fb8801597f0 2023-10-08T01:15:08.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.089+0000 7fb87f7fe700 1 -- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb86c067170 msgr2=0x7fb86c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:08.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.089+0000 7fb87f7fe700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb86c067170 0x7fb86c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:08.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.095+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb88004c3c0 con 0x7fb8801597f0 2023-10-08T01:15:08.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.097+0000 7fb8677fe700 1 -- 172.21.15.119:0/3915278842 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16492 (secure 0 0 0) 0x7fb874056730 con 0x7fb8801597f0 2023-10-08T01:15:08.097 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:08.097 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":122,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:15:07.254695+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:14:20.330219+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-10-08T01:14:21.202759+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":122,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1219956712":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/2381954001":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/546958670":"2023-10-09T01:14:42.347907+0000","172.21.15.119:6837/85511":"2023-10-09T01:14:42.347907+0000","172.21.15.119:0/309487327":"2023-10-09T01:14:42.347907+0000","172.21.15.119:6836/85511":"2023-10-09T01:14:42.347907+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:15:08.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.099+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb86c067170 msgr2=0x7fb86c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:08.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.099+0000 7fb884f9d700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb86c067170 0x7fb86c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.099+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8801597f0 msgr2=0x7fb88015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.099+0000 7fb884f9d700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8801597f0 0x7fb88015b650 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fb874013000 tx=0x7fb874017000 comp rx=0 tx=0).stop 2023-10-08T01:15:08.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.099+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 shutdown_connections 2023-10-08T01:15:08.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.100+0000 7fb884f9d700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb86c067170 0x7fb86c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.100+0000 7fb884f9d700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb880159420 0x7fb8800cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.100+0000 7fb884f9d700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb88015d230 0x7fb880154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.100+0000 7fb884f9d700 1 --2- 172.21.15.119:0/3915278842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8801597f0 0x7fb88015b650 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.100+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 >> 172.21.15.119:0/3915278842 conn(0x7fb8800d0660 msgr2=0x7fb8801561a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:08.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.100+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 shutdown_connections 2023-10-08T01:15:08.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.100+0000 7fb884f9d700 1 -- 172.21.15.119:0/3915278842 wait complete. 2023-10-08T01:15:08.118 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T01:15:08.227 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:08.226+0000 7f9fb3ebf180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:15:08.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.349+0000 7fc43ed9d700 1 Processor -- start 2023-10-08T01:15:08.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.350+0000 7fc43ed9d700 1 -- start start 2023-10-08T01:15:08.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.351+0000 7fc43ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc43806c0f0 0x7fc4381642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.351+0000 7fc43ed9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4380b40f0 con 0x7fc438142ea0 2023-10-08T01:15:08.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.351+0000 7fc43ed9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc438069640 con 0x7fc4380ad870 2023-10-08T01:15:08.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.351+0000 7fc43ed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc438069920 con 0x7fc43806c0f0 2023-10-08T01:15:08.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.351+0000 7fc43dd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc438142ea0 0x7fc4380b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34242/0 (socket says 172.21.15.119:34242) 2023-10-08T01:15:08.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.351+0000 7fc43dd9b700 1 -- 172.21.15.119:0/3913271088 learned_addr learned my addr 172.21.15.119:0/3913271088 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:08.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.351+0000 7fc43e59c700 1 --2- 172.21.15.119:0/3913271088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc43806c0f0 0x7fc4381642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43cd99700 1 -- 172.21.15.119:0/3913271088 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464746118 0 0) 0x7fc4380b40f0 con 0x7fc438142ea0 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43cd99700 1 -- 172.21.15.119:0/3913271088 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc424003570 con 0x7fc438142ea0 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43cd99700 1 -- 172.21.15.119:0/3913271088 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756126926 0 0) 0x7fc438069640 con 0x7fc4380ad870 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43cd99700 1 -- 172.21.15.119:0/3913271088 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4380b40f0 con 0x7fc4380ad870 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43cd99700 1 -- 172.21.15.119:0/3913271088 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2327769757 0 0) 0x7fc424003570 con 0x7fc438142ea0 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43e59c700 1 -- 172.21.15.119:0/3913271088 >> v1:172.21.15.184:6789/0 conn(0x7fc4380ad870 legacy=0x7fc43806b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43e59c700 1 -- 172.21.15.119:0/3913271088 >> v1:172.21.15.119:6789/0 conn(0x7fc438142ea0 legacy=0x7fc4380b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43e59c700 1 -- 172.21.15.119:0/3913271088 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc438069c00 con 0x7fc43806c0f0 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43e59c700 1 --2- 172.21.15.119:0/3913271088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc43806c0f0 0x7fc4381642e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc430009e90 tx=0x7fc430009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef4280b4313925b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.352+0000 7fc43cd99700 1 -- 172.21.15.119:0/3913271088 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000e070 con 0x7fc43806c0f0 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43cd99700 1 -- 172.21.15.119:0/3913271088 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43000b870 con 0x7fc43806c0f0 2023-10-08T01:15:08.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43cd99700 1 -- 172.21.15.119:0/3913271088 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc434001470 con 0x7fc43806c0f0 2023-10-08T01:15:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43ed9d700 1 -- 172.21.15.119:0/3913271088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc43806c0f0 msgr2=0x7fc4381642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43ed9d700 1 --2- 172.21.15.119:0/3913271088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc43806c0f0 0x7fc4381642e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc430009e90 tx=0x7fc430009780 comp rx=0 tx=0).stop 2023-10-08T01:15:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43ed9d700 1 -- 172.21.15.119:0/3913271088 shutdown_connections 2023-10-08T01:15:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43ed9d700 1 --2- 172.21.15.119:0/3913271088 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc43806c0f0 0x7fc4381642e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43ed9d700 1 -- 172.21.15.119:0/3913271088 >> 172.21.15.119:0/3913271088 conn(0x7fc4380ce430 msgr2=0x7fc4380ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43ed9d700 1 -- 172.21.15.119:0/3913271088 shutdown_connections 2023-10-08T01:15:08.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.353+0000 7fc43ed9d700 1 -- 172.21.15.119:0/3913271088 wait complete. 2023-10-08T01:15:08.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.355+0000 7fc43ed9d700 1 Processor -- start 2023-10-08T01:15:08.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.355+0000 7fc43ed9d700 1 -- start start 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc43806c0f0 0x7fc43815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc438142ea0 0x7fc4380ac900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4380ace40 0x7fc4380c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43ed9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380ca0b0 con 0x7fc4380ace40 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43ed9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380ad2e0 con 0x7fc43806c0f0 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43ed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4380c9dd0 con 0x7fc438142ea0 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4380ace40 0x7fc4380c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4380ace40 0x7fc4380c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43688/0 (socket says 172.21.15.119:43688) 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43e59c700 1 -- 172.21.15.119:0/4189028484 learned_addr learned my addr 172.21.15.119:0/4189028484 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43dd9b700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc43806c0f0 0x7fc43815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.356+0000 7fc43d59a700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc438142ea0 0x7fc4380ac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc43e59c700 1 -- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc438142ea0 msgr2=0x7fc4380ac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc43e59c700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc438142ea0 0x7fc4380ac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc43e59c700 1 -- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc43806c0f0 msgr2=0x7fc43815c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc43e59c700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc43806c0f0 0x7fc43815c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc43e59c700 1 -- 172.21.15.119:0/4189028484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc430009a20 con 0x7fc4380ace40 2023-10-08T01:15:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc43dd9b700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc43806c0f0 0x7fc43815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc43e59c700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4380ace40 0x7fc4380c9070 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc430009de0 tx=0x7fc430017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:08.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc42dffb700 1 -- 172.21.15.119:0/4189028484 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc430012c20 con 0x7fc4380ace40 2023-10-08T01:15:08.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc42dffb700 1 -- 172.21.15.119:0/4189028484 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc430012d80 con 0x7fc4380ace40 2023-10-08T01:15:08.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc42dffb700 1 -- 172.21.15.119:0/4189028484 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43001e6a0 con 0x7fc4380ace40 2023-10-08T01:15:08.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.357+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc438159e40 con 0x7fc4380ace40 2023-10-08T01:15:08.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.358+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc43815a400 con 0x7fc4380ace40 2023-10-08T01:15:08.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.359+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43815a620 con 0x7fc4380ace40 2023-10-08T01:15:08.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.359+0000 7fc42dffb700 1 -- 172.21.15.119:0/4189028484 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fc43001ead0 con 0x7fc4380ace40 2023-10-08T01:15:08.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.361+0000 7fc42dffb700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc41c067170 0x7fc41c069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.361+0000 7fc43dd9b700 1 -- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc41c067170 msgr2=0x7fc41c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:08.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.361+0000 7fc42dffb700 1 -- 172.21.15.119:0/4189028484 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6411+0+0 (secure 0 0 0) 0x7fc430086a50 con 0x7fc4380ace40 2023-10-08T01:15:08.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.361+0000 7fc43dd9b700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc41c067170 0x7fc41c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:08.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.365+0000 7fc42dffb700 1 -- 172.21.15.119:0/4189028484 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc430053660 con 0x7fc4380ace40 2023-10-08T01:15:08.517 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:08.516+0000 7f9fb3ebf180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:15:08.520 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:08.520+0000 7f9416b94180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:15:08.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.562+0000 7fc43dd9b700 1 -- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc41c067170 msgr2=0x7fc41c069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:08.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.562+0000 7fc43dd9b700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc41c067170 0x7fc41c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:08.609 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:08.607+0000 7f9fb3ebf180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:15:08.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.663+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc43804c3c0 con 0x7fc4380ace40 2023-10-08T01:15:08.663 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T01:15:08.663+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T01:15:08.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.668+0000 7fc42dffb700 1 -- 172.21.15.119:0/4189028484 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v56) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc430016090 con 0x7fc4380ace40 2023-10-08T01:15:08.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.670+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc41c067170 msgr2=0x7fc41c069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:08.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.670+0000 7fc43ed9d700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc41c067170 0x7fc41c069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4380ace40 msgr2=0x7fc4380c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4380ace40 0x7fc4380c9070 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fc430009de0 tx=0x7fc430017000 comp rx=0 tx=0).stop 2023-10-08T01:15:08.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 shutdown_connections 2023-10-08T01:15:08.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc41c067170 0x7fc41c069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc4380ace40 0x7fc4380c9070 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc43806c0f0 0x7fc43815c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 --2- 172.21.15.119:0/4189028484 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc438142ea0 0x7fc4380ac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 >> 172.21.15.119:0/4189028484 conn(0x7fc4380ce430 msgr2=0x7fc438141d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:08.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 shutdown_connections 2023-10-08T01:15:08.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.671+0000 7fc43ed9d700 1 -- 172.21.15.119:0/4189028484 wait complete. 2023-10-08T01:15:08.672 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T01:15:08.682 INFO:tasks.ceph.mds.a.smithi119.stderr: -5136> 2023-10-08T01:14:27.735+0000 7ff71c3f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:15:08.690 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T01:15:08.736 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:15:08.769 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:08.767+0000 7f9fb3ebf180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:15:08.822 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:08.822+0000 7f9416b94180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:15:08.872 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:08.871+0000 7f9fb3ebf180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:15:08.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.881+0000 7fccaf59e700 1 Processor -- start 2023-10-08T01:15:08.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.881+0000 7fccaf59e700 1 -- start start 2023-10-08T01:15:08.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccaf59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fccb00b16d0 0x7fccb00b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccaf59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b33b0 0x7fccb00b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccaf59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccb00b2080 con 0x7fccb00b4e70 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccaf59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccb00cfb00 con 0x7fccb00b16d0 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccaf59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00b5c40 con 0x7fccb00b33b0 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccae59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fccb00b4e70 0x7fccb00ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34252/0 (socket says 172.21.15.119:34252) 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccae59c700 1 -- 172.21.15.119:0/999771880 learned_addr learned my addr 172.21.15.119:0/999771880 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccaed9d700 1 --2- 172.21.15.119:0/999771880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b33b0 0x7fccb00b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccadd9b700 1 --2- 172.21.15.119:0/999771880 >> v2:172.21.15.184:3300/0 conn(0x7fccb00b16d0 0x7fccb00b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.883+0000 7fccad59a700 1 -- 172.21.15.119:0/999771880 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325874104 0 0) 0x7fccb00b2080 con 0x7fccb00b4e70 2023-10-08T01:15:08.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccad59a700 1 -- 172.21.15.119:0/999771880 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc98003570 con 0x7fccb00b4e70 2023-10-08T01:15:08.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccaed9d700 1 -- 172.21.15.119:0/999771880 >> v2:172.21.15.184:3300/0 conn(0x7fccb00b16d0 msgr2=0x7fccb00b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:08.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccaed9d700 1 --2- 172.21.15.119:0/999771880 >> v2:172.21.15.184:3300/0 conn(0x7fccb00b16d0 0x7fccb00b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccaed9d700 1 -- 172.21.15.119:0/999771880 >> v1:172.21.15.119:6789/0 conn(0x7fccb00b4e70 legacy=0x7fccb00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccaed9d700 1 -- 172.21.15.119:0/999771880 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00b5f20 con 0x7fccb00b33b0 2023-10-08T01:15:08.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccaed9d700 1 --2- 172.21.15.119:0/999771880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b33b0 0x7fccb00b3810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009e60 tx=0x7fcca8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e6f84babd292476 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:08.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccad59a700 1 -- 172.21.15.119:0/999771880 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca800e070 con 0x7fccb00b33b0 2023-10-08T01:15:08.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccad59a700 1 -- 172.21.15.119:0/999771880 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca800bab0 con 0x7fccb00b33b0 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.884+0000 7fccad59a700 1 -- 172.21.15.119:0/999771880 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca800be50 con 0x7fccb00b33b0 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.885+0000 7fccaf59e700 1 -- 172.21.15.119:0/999771880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b33b0 msgr2=0x7fccb00b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.885+0000 7fccaf59e700 1 --2- 172.21.15.119:0/999771880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b33b0 0x7fccb00b3810 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009e60 tx=0x7fcca8009780 comp rx=0 tx=0).stop 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.885+0000 7fccaf59e700 1 -- 172.21.15.119:0/999771880 shutdown_connections 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.885+0000 7fccaf59e700 1 --2- 172.21.15.119:0/999771880 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b33b0 0x7fccb00b3810 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.885+0000 7fccaf59e700 1 --2- 172.21.15.119:0/999771880 >> v2:172.21.15.184:3300/0 conn(0x7fccb00b16d0 0x7fccb00b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.885+0000 7fccaf59e700 1 -- 172.21.15.119:0/999771880 >> 172.21.15.119:0/999771880 conn(0x7fccb00d10c0 msgr2=0x7fccb00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.885+0000 7fccaf59e700 1 -- 172.21.15.119:0/999771880 shutdown_connections 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.885+0000 7fccaf59e700 1 -- 172.21.15.119:0/999771880 wait complete. 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.887+0000 7fccaf59e700 1 Processor -- start 2023-10-08T01:15:08.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.887+0000 7fccaf59e700 1 -- start start 2023-10-08T01:15:08.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.887+0000 7fccaf59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b16d0 0x7fccb0149130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccaf59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb00b33b0 0x7fccb00c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccaf59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb00b4e70 0x7fccb00ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccaf59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb0149670 con 0x7fccb00b33b0 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccaf59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00c67d0 con 0x7fccb00b4e70 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccaf59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb00c5bc0 con 0x7fccb00b16d0 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb00b33b0 0x7fccb00c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb00b33b0 0x7fccb00c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43704/0 (socket says 172.21.15.119:43704) 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 -- 172.21.15.119:0/3765244874 learned_addr learned my addr 172.21.15.119:0/3765244874 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccae59c700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b16d0 0x7fccb0149130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccaed9d700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb00b4e70 0x7fccb00ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 -- 172.21.15.119:0/3765244874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b16d0 msgr2=0x7fccb0149130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b16d0 0x7fccb0149130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 -- 172.21.15.119:0/3765244874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb00b4e70 msgr2=0x7fccb00ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb00b4e70 0x7fccb00ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 -- 172.21.15.119:0/3765244874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcca8009a20 con 0x7fccb00b33b0 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.888+0000 7fccadd9b700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb00b33b0 0x7fccb00c5390 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcca400e930 tx=0x7fcca40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.889+0000 7fcc9e7fc700 1 -- 172.21.15.119:0/3765244874 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4013680 con 0x7fccb00b33b0 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.889+0000 7fcc9e7fc700 1 -- 172.21.15.119:0/3765244874 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca40137e0 con 0x7fccb00b33b0 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.889+0000 7fcc9e7fc700 1 -- 172.21.15.119:0/3765244874 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4013b80 con 0x7fccb00b33b0 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.889+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccb00c5ea0 con 0x7fccb00b33b0 2023-10-08T01:15:08.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.889+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccb00c6460 con 0x7fccb00b33b0 2023-10-08T01:15:08.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.890+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccb00c6980 con 0x7fccb00b33b0 2023-10-08T01:15:08.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.891+0000 7fcc9e7fc700 1 -- 172.21.15.119:0/3765244874 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 64) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fcca401c450 con 0x7fccb00b33b0 2023-10-08T01:15:08.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.892+0000 7fcc9e7fc700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 0x7fcc90069620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:08.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.892+0000 7fcc9e7fc700 1 -- 172.21.15.119:0/3765244874 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6469+0+0 (secure 0 0 0) 0x7fcca4084c90 con 0x7fccb00b33b0 2023-10-08T01:15:08.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.892+0000 7fccae59c700 1 -- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 msgr2=0x7fcc90069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:08.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.892+0000 7fccae59c700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 0x7fcc90069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:08.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:08.898+0000 7fcc9e7fc700 1 -- 172.21.15.119:0/3765244874 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fccb00c6980 con 0x7fccb00b33b0 2023-10-08T01:15:08.922 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:08.922+0000 7f9416b94180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:15:09.061 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:09.060+0000 7f9fb3ebf180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:15:09.082 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:09.082+0000 7f9416b94180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:15:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.093+0000 7fccae59c700 1 -- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 msgr2=0x7fcc90069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.093+0000 7fccae59c700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 0x7fcc90069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:09.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.142+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fccb00c6940 con 0x7fccb00b33b0 2023-10-08T01:15:09.153 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:09.152+0000 7f9fb3ebf180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:15:09.190 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:09.190+0000 7f9416b94180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:15:09.244 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:09.243+0000 7f9fb3ebf180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:15:09.341 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:09.340+0000 7f9fb3ebf180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:15:09.383 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:09.383+0000 7f9416b94180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:15:09.477 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:09.477+0000 7f9416b94180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:15:09.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.494+0000 7fccae59c700 1 -- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 msgr2=0x7fcc90069620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:09.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.494+0000 7fccae59c700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 0x7fcc90069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:15:09.571 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:09.571+0000 7f9416b94180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:15:09.666 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:09.666+0000 7f9416b94180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:15:09.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.669+0000 7fcc9e7fc700 1 -- 172.21.15.119:0/3765244874 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcca40518d0 con 0x7fccb00b33b0 2023-10-08T01:15:09.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 msgr2=0x7fcc90069620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:09.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 0x7fcc90069620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb00b33b0 msgr2=0x7fccb00c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb00b33b0 0x7fccb00c5390 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fcca400e930 tx=0x7fcca40091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 shutdown_connections 2023-10-08T01:15:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc90067170 0x7fcc90069620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fccb00b16d0 0x7fccb0149130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fccb00b4e70 0x7fccb00ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.671+0000 7fccaf59e700 1 --2- 172.21.15.119:0/3765244874 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fccb00b33b0 0x7fccb00c5390 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.672+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 >> 172.21.15.119:0/3765244874 conn(0x7fccb00d10c0 msgr2=0x7fccb0158ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.672+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 shutdown_connections 2023-10-08T01:15:09.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.672+0000 7fccaf59e700 1 -- 172.21.15.119:0/3765244874 wait complete. 2023-10-08T01:15:09.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T01:15:09.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.891+0000 7fa4273ff700 1 Processor -- start 2023-10-08T01:15:09.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.892+0000 7fa4273ff700 1 -- start start 2023-10-08T01:15:09.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.893+0000 7fa4273ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa4200c8490 0x7fa4200c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:09.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.893+0000 7fa4273ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200ac030 con 0x7fa420069a50 2023-10-08T01:15:09.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.893+0000 7fa4273ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4200ab820 con 0x7fa420069e20 2023-10-08T01:15:09.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.893+0000 7fa4273ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42006dbb0 con 0x7fa4200c8490 2023-10-08T01:15:09.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.893+0000 7fa42519b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa420069a50 0x7fa42015a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34260/0 (socket says 172.21.15.119:34260) 2023-10-08T01:15:09.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.893+0000 7fa42519b700 1 -- 172.21.15.119:0/678669763 learned_addr learned my addr 172.21.15.119:0/678669763 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:09.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.893+0000 7fa42499a700 1 --2- 172.21.15.119:0/678669763 >> v2:172.21.15.184:3300/0 conn(0x7fa4200c8490 0x7fa4200c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:09.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.893+0000 7fa417fff700 1 -- 172.21.15.119:0/678669763 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145831989 0 0) 0x7fa41c001110 con 0x7fa420069a50 2023-10-08T01:15:09.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa417fff700 1 -- 172.21.15.119:0/678669763 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa40c003570 con 0x7fa420069a50 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa417fff700 1 -- 172.21.15.119:0/678669763 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303365140 0 0) 0x7fa4200ab820 con 0x7fa420069e20 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa417fff700 1 -- 172.21.15.119:0/678669763 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41c001110 con 0x7fa420069e20 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa42499a700 1 -- 172.21.15.119:0/678669763 >> v1:172.21.15.184:6789/0 conn(0x7fa420069e20 legacy=0x7fa4201569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa42499a700 1 -- 172.21.15.119:0/678669763 >> v1:172.21.15.119:6789/0 conn(0x7fa420069a50 legacy=0x7fa42015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa42499a700 1 -- 172.21.15.119:0/678669763 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4200aba70 con 0x7fa4200c8490 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa42499a700 1 --2- 172.21.15.119:0/678669763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 0x7fa4200c6ac0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa41000aa90 tx=0x7fa41000a330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c456657b76eae215 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa417fff700 1 -- 172.21.15.119:0/678669763 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41000c970 con 0x7fa4200c8490 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa417fff700 1 -- 172.21.15.119:0/678669763 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41000cad0 con 0x7fa4200c8490 2023-10-08T01:15:09.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.894+0000 7fa417fff700 1 -- 172.21.15.119:0/678669763 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41000ce70 con 0x7fa4200c8490 2023-10-08T01:15:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.895+0000 7fa4273ff700 1 -- 172.21.15.119:0/678669763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 msgr2=0x7fa4200c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.895+0000 7fa4273ff700 1 --2- 172.21.15.119:0/678669763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 0x7fa4200c6ac0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa41000aa90 tx=0x7fa41000a330 comp rx=0 tx=0).stop 2023-10-08T01:15:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.895+0000 7fa4273ff700 1 -- 172.21.15.119:0/678669763 shutdown_connections 2023-10-08T01:15:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.895+0000 7fa4273ff700 1 --2- 172.21.15.119:0/678669763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 0x7fa4200c6ac0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.895+0000 7fa4273ff700 1 -- 172.21.15.119:0/678669763 >> 172.21.15.119:0/678669763 conn(0x7fa4200c8f90 msgr2=0x7fa4200c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.895+0000 7fa4273ff700 1 -- 172.21.15.119:0/678669763 shutdown_connections 2023-10-08T01:15:09.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.895+0000 7fa4273ff700 1 -- 172.21.15.119:0/678669763 wait complete. 2023-10-08T01:15:09.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.896+0000 7fa4273ff700 1 Processor -- start 2023-10-08T01:15:09.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.896+0000 7fa4273ff700 1 -- start start 2023-10-08T01:15:09.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.897+0000 7fa4273ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa420069a50 0x7fa4200c0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.897+0000 7fa4273ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa420069e20 0x7fa420154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.897+0000 7fa4273ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 0x7fa4200c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.897+0000 7fa4273ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200b0450 con 0x7fa420069a50 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.897+0000 7fa4273ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200af840 con 0x7fa4200c8490 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.897+0000 7fa4273ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4200afb20 con 0x7fa420069e20 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa420069a50 0x7fa4200c0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa420069a50 0x7fa4200c0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43824/0 (socket says 172.21.15.119:43824) 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 -- 172.21.15.119:0/1907485198 learned_addr learned my addr 172.21.15.119:0/1907485198 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:09.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42499a700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa420069e20 0x7fa420154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42599c700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 0x7fa4200c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 -- 172.21.15.119:0/1907485198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa420069e20 msgr2=0x7fa420154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa420069e20 0x7fa420154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 -- 172.21.15.119:0/1907485198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 msgr2=0x7fa4200c3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 0x7fa4200c3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 -- 172.21.15.119:0/1907485198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41000a5d0 con 0x7fa420069a50 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa42519b700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa420069a50 0x7fa4200c0490 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c790 tx=0x7fa41c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa4157fa700 1 -- 172.21.15.119:0/1907485198 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c005b30 con 0x7fa420069a50 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.898+0000 7fa4157fa700 1 -- 172.21.15.119:0/1907485198 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa41c005c90 con 0x7fa420069a50 2023-10-08T01:15:09.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.899+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4200afe00 con 0x7fa420069a50 2023-10-08T01:15:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.899+0000 7fa4157fa700 1 -- 172.21.15.119:0/1907485198 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa41c007300 con 0x7fa420069a50 2023-10-08T01:15:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.899+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4200c09d0 con 0x7fa420069a50 2023-10-08T01:15:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.899+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4200718b0 con 0x7fa420069a50 2023-10-08T01:15:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.900+0000 7fa4157fa700 1 -- 172.21.15.119:0/1907485198 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 65) v1 ==== 84451+0+0 (secure 0 0 0) 0x7fa41c01b3b0 con 0x7fa420069a50 2023-10-08T01:15:09.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.901+0000 7fa4157fa700 1 -- 172.21.15.119:0/1907485198 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa41c083890 con 0x7fa420069a50 2023-10-08T01:15:09.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:09.905+0000 7fa4157fa700 1 -- 172.21.15.119:0/1907485198 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa41c050a70 con 0x7fa420069a50 2023-10-08T01:15:10.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.094+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa4200ac030 con 0x7fa420069a50 2023-10-08T01:15:10.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.679+0000 7fa4157fa700 1 -- 172.21.15.119:0/1907485198 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa4200ac030 con 0x7fa420069a50 2023-10-08T01:15:10.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.684+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa420069a50 msgr2=0x7fa4200c0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:10.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.684+0000 7fa4273ff700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa420069a50 0x7fa4200c0490 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fa41c00c790 tx=0x7fa41c007ea0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.684+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 shutdown_connections 2023-10-08T01:15:10.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.684+0000 7fa4273ff700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4200c8490 0x7fa4200c3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.684+0000 7fa4273ff700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa420069a50 0x7fa4200c0490 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.685+0000 7fa4273ff700 1 --2- 172.21.15.119:0/1907485198 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa420069e20 0x7fa420154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.685+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 >> 172.21.15.119:0/1907485198 conn(0x7fa4200c8f90 msgr2=0x7fa4200cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:10.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.685+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 shutdown_connections 2023-10-08T01:15:10.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.685+0000 7fa4273ff700 1 -- 172.21.15.119:0/1907485198 wait complete. 2023-10-08T01:15:10.688 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T01:15:10.715 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T01:15:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.902+0000 7fdb4c5ff700 1 Processor -- start 2023-10-08T01:15:10.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.902+0000 7fdb4c5ff700 1 -- start start 2023-10-08T01:15:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.903+0000 7fdb4c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 0x7fdb440c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.903+0000 7fdb4c5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb440ac030 con 0x7fdb44069a50 2023-10-08T01:15:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.903+0000 7fdb4c5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb440ab820 con 0x7fdb440c8490 2023-10-08T01:15:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.903+0000 7fdb4c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4406dbb0 con 0x7fdb441570c0 2023-10-08T01:15:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.903+0000 7fdb4a39b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdb44069a50 0x7fdb4415a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34276/0 (socket says 172.21.15.119:34276) 2023-10-08T01:15:10.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.903+0000 7fdb4a39b700 1 -- 172.21.15.119:0/829310078 learned_addr learned my addr 172.21.15.119:0/829310078 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:10.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb4ab9c700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 0x7fdb440c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:10.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304368515 0 0) 0x7fdb40001490 con 0x7fdb44069a50 2023-10-08T01:15:10.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb30004680 con 0x7fdb44069a50 2023-10-08T01:15:10.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1866336664 0 0) 0x7fdb440ab820 con 0x7fdb440c8490 2023-10-08T01:15:10.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb40001490 con 0x7fdb440c8490 2023-10-08T01:15:10.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1564381930 0 0) 0x7fdb30004680 con 0x7fdb44069a50 2023-10-08T01:15:10.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 msgr2=0x7fdb440c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:10.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 0x7fdb440c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 >> v1:172.21.15.184:6789/0 conn(0x7fdb440c8490 legacy=0x7fdb441569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:10.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440aba70 con 0x7fdb44069a50 2023-10-08T01:15:10.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fdb40001980 con 0x7fdb44069a50 2023-10-08T01:15:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 >> v1:172.21.15.119:6789/0 conn(0x7fdb44069a50 legacy=0x7fdb4415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 0x7fdb30004d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30005290 0x7fdb30005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb30005e60 0x7fdb300062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4406dbb0 con 0x7fdb30005290 2023-10-08T01:15:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3000ccb0 con 0x7fdb30005e60 2023-10-08T01:15:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3000ce70 con 0x7fdb441570c0 2023-10-08T01:15:10.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.904+0000 7fdb49b9a700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30005290 0x7fdb30005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:10.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb4a39b700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 0x7fdb30004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:10.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb4ab9c700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb30005e60 0x7fdb300062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:10.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb4a39b700 1 -- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb30005e60 msgr2=0x7fdb300062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:10.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb4a39b700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb30005e60 0x7fdb300062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb4a39b700 1 -- 172.21.15.119:0/829310078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30005290 msgr2=0x7fdb30005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb4a39b700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30005290 0x7fdb30005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb4a39b700 1 -- 172.21.15.119:0/829310078 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30004680 con 0x7fdb441570c0 2023-10-08T01:15:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb49b9a700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb30005290 0x7fdb30005920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb4a39b700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 0x7fdb30004d30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000aa60 tx=0x7fdb4406dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2372ba9174a30a38 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000a3f0 con 0x7fdb441570c0 2023-10-08T01:15:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4000c5d0 con 0x7fdb441570c0 2023-10-08T01:15:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.905+0000 7fdb49399700 1 -- 172.21.15.119:0/829310078 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4000c990 con 0x7fdb441570c0 2023-10-08T01:15:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.906+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 msgr2=0x7fdb30004d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.906+0000 7fdb4c5ff700 1 --2- 172.21.15.119:0/829310078 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb441570c0 0x7fdb30004d30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000aa60 tx=0x7fdb4406dcf0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.906+0000 7fdb4ab9c700 1 -- 172.21.15.119:0/829310078 reap_dead start 2023-10-08T01:15:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.906+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/829310078 shutdown_connections 2023-10-08T01:15:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.906+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/829310078 >> 172.21.15.119:0/829310078 conn(0x7fdb440c8f90 msgr2=0x7fdb440c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.906+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/829310078 shutdown_connections 2023-10-08T01:15:10.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.906+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/829310078 wait complete. 2023-10-08T01:15:10.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.907+0000 7fdb4c5ff700 1 Processor -- start 2023-10-08T01:15:10.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.908+0000 7fdb4c5ff700 1 -- start start 2023-10-08T01:15:10.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.908+0000 7fdb4c5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb441570c0 0x7fdb440a98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:10.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.908+0000 7fdb4c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb440c6eb0 0x7fdb4416d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:10.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.908+0000 7fdb4c5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb4416da80 0x7fdb440b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.908+0000 7fdb4c5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440c0c30 con 0x7fdb4416da80 2023-10-08T01:15:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.908+0000 7fdb4c5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440c0600 con 0x7fdb441570c0 2023-10-08T01:15:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.908+0000 7fdb4c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4416e200 con 0x7fdb440c6eb0 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb4416da80 0x7fdb440b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb4416da80 0x7fdb440b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43838/0 (socket says 172.21.15.119:43838) 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 -- 172.21.15.119:0/2524511973 learned_addr learned my addr 172.21.15.119:0/2524511973 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4a39b700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb441570c0 0x7fdb440a98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb49b9a700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb440c6eb0 0x7fdb4416d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 -- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb440c6eb0 msgr2=0x7fdb4416d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb440c6eb0 0x7fdb4416d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 -- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb441570c0 msgr2=0x7fdb440a98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb441570c0 0x7fdb440a98e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 -- 172.21.15.119:0/2524511973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440c08c0 con 0x7fdb4416da80 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb49b9a700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb440c6eb0 0x7fdb4416d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.909+0000 7fdb4ab9c700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb4416da80 0x7fdb440b1640 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdb4415ac10 tx=0x7fdb3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.910+0000 7fdb3a7fc700 1 -- 172.21.15.119:0/2524511973 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c011ee0 con 0x7fdb4416da80 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.910+0000 7fdb3a7fc700 1 -- 172.21.15.119:0/2524511973 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c00f910 con 0x7fdb4416da80 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.910+0000 7fdb3a7fc700 1 -- 172.21.15.119:0/2524511973 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c00fcb0 con 0x7fdb4416da80 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.910+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb440a9e20 con 0x7fdb4416da80 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.910+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb440b1b80 con 0x7fdb4416da80 2023-10-08T01:15:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.911+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb440c0de0 con 0x7fdb4416da80 2023-10-08T01:15:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.912+0000 7fdb3a7fc700 1 -- 172.21.15.119:0/2524511973 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 66) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fdb3c014650 con 0x7fdb4416da80 2023-10-08T01:15:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.912+0000 7fdb3a7fc700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb28066db0 0x7fdb28069260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.913+0000 7fdb3a7fc700 1 -- 172.21.15.119:0/2524511973 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6375+0+0 (secure 0 0 0) 0x7fdb3c084ba0 con 0x7fdb4416da80 2023-10-08T01:15:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.913+0000 7fdb4a39b700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb28066db0 0x7fdb28069260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.913+0000 7fdb4a39b700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb28066db0 0x7fdb28069260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000a0b0 tx=0x7fdb40016000 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:10.917+0000 7fdb3a7fc700 1 -- 172.21.15.119:0/2524511973 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdb440c0de0 con 0x7fdb4416da80 2023-10-08T01:15:11.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.106+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdb440c0a50 con 0x7fdb4416da80 2023-10-08T01:15:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.713+0000 7fdb3a7fc700 1 -- 172.21.15.119:0/2524511973 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdb3c0521a0 con 0x7fdb4416da80 2023-10-08T01:15:11.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.714+0000 7fdb3a7fc700 1 -- 172.21.15.119:0/2524511973 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdb3c051dd0 con 0x7fdb4416da80 2023-10-08T01:15:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb28066db0 msgr2=0x7fdb28069260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb28066db0 0x7fdb28069260 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000a0b0 tx=0x7fdb40016000 comp rx=0 tx=0).stop 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb4416da80 msgr2=0x7fdb440b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb4416da80 0x7fdb440b1640 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fdb4415ac10 tx=0x7fdb3c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 shutdown_connections 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdb28066db0 0x7fdb28069260 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdb4416da80 0x7fdb440b1640 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdb441570c0 0x7fdb440a98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 --2- 172.21.15.119:0/2524511973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdb440c6eb0 0x7fdb4416d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 >> 172.21.15.119:0/2524511973 conn(0x7fdb440c8f90 msgr2=0x7fdb440c77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 shutdown_connections 2023-10-08T01:15:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.717+0000 7fdb4c5ff700 1 -- 172.21.15.119:0/2524511973 wait complete. 2023-10-08T01:15:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T01:15:11.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T01:15:11.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.920+0000 7ff803bff700 1 Processor -- start 2023-10-08T01:15:11.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.920+0000 7ff803bff700 1 -- start start 2023-10-08T01:15:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.921+0000 7ff803bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc15ac80 0x7ff7fc069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.921+0000 7ff803bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff7fc069cb0 0x7ff7fc0c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.921+0000 7ff803bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7fc0af2f0 con 0x7ff7fc15b050 2023-10-08T01:15:11.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.921+0000 7ff803bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc0af5d0 con 0x7ff7fc069cb0 2023-10-08T01:15:11.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.921+0000 7ff803bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc06a0f0 con 0x7ff7fc15ac80 2023-10-08T01:15:11.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80219c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff7fc069cb0 0x7ff7fc0c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80119a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff7fc15b050 0x7ff7fc0afa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34280/0 (socket says 172.21.15.119:34280) 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80119a700 1 -- 172.21.15.119:0/3439665910 learned_addr learned my addr 172.21.15.119:0/3439665910 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80199b700 1 --2- 172.21.15.119:0/3439665910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc15ac80 0x7ff7fc069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80219c700 1 -- 172.21.15.119:0/3439665910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc15ac80 msgr2=0x7ff7fc069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80219c700 1 --2- 172.21.15.119:0/3439665910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc15ac80 0x7ff7fc069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80219c700 1 -- 172.21.15.119:0/3439665910 >> v1:172.21.15.119:6789/0 conn(0x7ff7fc15b050 legacy=0x7ff7fc0afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80219c700 1 -- 172.21.15.119:0/3439665910 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc0c8660 con 0x7ff7fc069cb0 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff800999700 1 -- 172.21.15.119:0/3439665910 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3778856086 0 0) 0x7ff7fc0af2f0 con 0x7ff7fc15b050 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.922+0000 7ff80219c700 1 --2- 172.21.15.119:0/3439665910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc069cb0 0x7ff7fc0c7b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009e60 tx=0x7ff7f4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d999bf3493625e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff800999700 1 -- 172.21.15.119:0/3439665910 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f400e070 con 0x7ff7fc069cb0 2023-10-08T01:15:11.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff800999700 1 -- 172.21.15.119:0/3439665910 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f400bc30 con 0x7ff7fc069cb0 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff800999700 1 -- 172.21.15.119:0/3439665910 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4012430 con 0x7ff7fc069cb0 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff803bff700 1 -- 172.21.15.119:0/3439665910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc069cb0 msgr2=0x7ff7fc0c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff803bff700 1 --2- 172.21.15.119:0/3439665910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc069cb0 0x7ff7fc0c7b80 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4009e60 tx=0x7ff7f4009780 comp rx=0 tx=0).stop 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff803bff700 1 -- 172.21.15.119:0/3439665910 shutdown_connections 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff803bff700 1 --2- 172.21.15.119:0/3439665910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc069cb0 0x7ff7fc0c7b80 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff803bff700 1 --2- 172.21.15.119:0/3439665910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc15ac80 0x7ff7fc069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff803bff700 1 -- 172.21.15.119:0/3439665910 >> 172.21.15.119:0/3439665910 conn(0x7ff7fc068e10 msgr2=0x7ff7fc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff803bff700 1 -- 172.21.15.119:0/3439665910 shutdown_connections 2023-10-08T01:15:11.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.923+0000 7ff803bff700 1 -- 172.21.15.119:0/3439665910 wait complete. 2023-10-08T01:15:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.925+0000 7ff803bff700 1 Processor -- start 2023-10-08T01:15:11.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.925+0000 7ff803bff700 1 -- start start 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff803bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc069cb0 0x7ff7fc0ab7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff803bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc15ac80 0x7ff7fc0ad940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff803bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7fc15b050 0x7ff7fc0aaca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff803bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0abdb0 con 0x7ff7fc15ac80 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff803bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0ac090 con 0x7ff7fc15b050 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff803bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0ac370 con 0x7ff7fc069cb0 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80119a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc15ac80 0x7ff7fc0ad940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80119a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc15ac80 0x7ff7fc0ad940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43850/0 (socket says 172.21.15.119:43850) 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80119a700 1 -- 172.21.15.119:0/1840744860 learned_addr learned my addr 172.21.15.119:0/1840744860 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80199b700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc069cb0 0x7ff7fc0ab7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:11.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80219c700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7fc15b050 0x7ff7fc0aaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80119a700 1 -- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc069cb0 msgr2=0x7ff7fc0ab7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80119a700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc069cb0 0x7ff7fc0ab7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80119a700 1 -- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7fc15b050 msgr2=0x7ff7fc0aaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80119a700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7fc15b050 0x7ff7fc0aaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80119a700 1 -- 172.21.15.119:0/1840744860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4009a20 con 0x7ff7fc15ac80 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.926+0000 7ff80199b700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc069cb0 0x7ff7fc0ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.927+0000 7ff80119a700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc15ac80 0x7ff7fc0ad940 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c5b0 tx=0x7ff7ec001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.927+0000 7ff7f1ffb700 1 -- 172.21.15.119:0/1840744860 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec005b30 con 0x7ff7fc15ac80 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.927+0000 7ff7f1ffb700 1 -- 172.21.15.119:0/1840744860 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec005c90 con 0x7ff7fc15ac80 2023-10-08T01:15:11.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.927+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc0ac650 con 0x7ff7fc15ac80 2023-10-08T01:15:11.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.927+0000 7ff7f1ffb700 1 -- 172.21.15.119:0/1840744860 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec007300 con 0x7ff7fc15ac80 2023-10-08T01:15:11.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.927+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc0ab1e0 con 0x7ff7fc15ac80 2023-10-08T01:15:11.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.928+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7fc0ac890 con 0x7ff7fc15ac80 2023-10-08T01:15:11.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.929+0000 7ff7f1ffb700 1 -- 172.21.15.119:0/1840744860 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7ec01b3b0 con 0x7ff7fc15ac80 2023-10-08T01:15:11.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.930+0000 7ff7f1ffb700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7e0066f30 0x7ff7e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:11.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.930+0000 7ff7f1ffb700 1 -- 172.21.15.119:0/1840744860 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5961+0+0 (secure 0 0 0) 0x7ff7ec083c00 con 0x7ff7fc15ac80 2023-10-08T01:15:11.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.930+0000 7ff80199b700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7e0066f30 0x7ff7e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:11.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.930+0000 7ff80199b700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7e0066f30 0x7ff7e00693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8006fd0 tx=0x7ff7f8008040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:11.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:11.934+0000 7ff7f1ffb700 1 -- 172.21.15.119:0/1840744860 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7fc0ac890 con 0x7ff7fc15ac80 2023-10-08T01:15:12.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.124+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff7fc0ac890 con 0x7ff7fc15ac80 2023-10-08T01:15:12.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.130+0000 7ff7f1ffb700 1 -- 172.21.15.119:0/1840744860 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v43) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff7fc0ac890 con 0x7ff7fc15ac80 2023-10-08T01:15:12.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7e0066f30 msgr2=0x7ff7e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7e0066f30 0x7ff7e00693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8006fd0 tx=0x7ff7f8008040 comp rx=0 tx=0).stop 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc15ac80 msgr2=0x7ff7fc0ad940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc15ac80 0x7ff7fc0ad940 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00c5b0 tx=0x7ff7ec001070 comp rx=0 tx=0).stop 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 shutdown_connections 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff7e0066f30 0x7ff7e00693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff7fc069cb0 0x7ff7fc0ab7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff7fc15b050 0x7ff7fc0aaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 --2- 172.21.15.119:0/1840744860 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff7fc15ac80 0x7ff7fc0ad940 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 >> 172.21.15.119:0/1840744860 conn(0x7ff7fc068e10 msgr2=0x7ff7fc157f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:12.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 shutdown_connections 2023-10-08T01:15:12.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.132+0000 7ff803bff700 1 -- 172.21.15.119:0/1840744860 wait complete. 2023-10-08T01:15:12.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T01:15:12.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.331+0000 7fecca79d700 1 Processor -- start 2023-10-08T01:15:12.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.332+0000 7fecca79d700 1 -- start start 2023-10-08T01:15:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.333+0000 7fecca79d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fecc406f060 0x7fecc40ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.333+0000 7fecca79d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc406ebd0 con 0x7fecc40cb1a0 2023-10-08T01:15:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.333+0000 7fecca79d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecc415aa70 con 0x7fecc406f4b0 2023-10-08T01:15:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.333+0000 7fecca79d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc415b2e0 con 0x7fecc406f060 2023-10-08T01:15:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.333+0000 7fecc979b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fecc40cb1a0 0x7fecc415a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34290/0 (socket says 172.21.15.119:34290) 2023-10-08T01:15:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.333+0000 7fecc979b700 1 -- 172.21.15.119:0/4079523777 learned_addr learned my addr 172.21.15.119:0/4079523777 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.333+0000 7fecc8f9a700 1 --2- 172.21.15.119:0/4079523777 >> v2:172.21.15.119:3300/0 conn(0x7fecc406f060 0x7fecc40ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.333+0000 7fecbbfff700 1 -- 172.21.15.119:0/4079523777 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852370271 0 0) 0x7fecc406ebd0 con 0x7fecc40cb1a0 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.334+0000 7fecbbfff700 1 -- 172.21.15.119:0/4079523777 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecb0003570 con 0x7fecc40cb1a0 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.334+0000 7fecc8f9a700 1 -- 172.21.15.119:0/4079523777 >> v1:172.21.15.184:6789/0 conn(0x7fecc406f4b0 legacy=0x7fecc40c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.334+0000 7fecc8f9a700 1 -- 172.21.15.119:0/4079523777 >> v1:172.21.15.119:6789/0 conn(0x7fecc40cb1a0 legacy=0x7fecc415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.334+0000 7fecc8f9a700 1 -- 172.21.15.119:0/4079523777 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc415ad20 con 0x7fecc406f060 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.334+0000 7fecc8f9a700 1 --2- 172.21.15.119:0/4079523777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc406f060 0x7fecc40ae0d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fecb40098e0 tx=0x7fecb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b6875c420775f22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.334+0000 7fecbbfff700 1 -- 172.21.15.119:0/4079523777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb400b6d0 con 0x7fecc406f060 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.334+0000 7fecbbfff700 1 -- 172.21.15.119:0/4079523777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb400b830 con 0x7fecc406f060 2023-10-08T01:15:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.334+0000 7fecbbfff700 1 -- 172.21.15.119:0/4079523777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb400bbd0 con 0x7fecc406f060 2023-10-08T01:15:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.335+0000 7fecca79d700 1 -- 172.21.15.119:0/4079523777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc406f060 msgr2=0x7fecc40ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.335+0000 7fecca79d700 1 --2- 172.21.15.119:0/4079523777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc406f060 0x7fecc40ae0d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fecb40098e0 tx=0x7fecb40091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.335+0000 7fecca79d700 1 -- 172.21.15.119:0/4079523777 shutdown_connections 2023-10-08T01:15:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.335+0000 7fecca79d700 1 --2- 172.21.15.119:0/4079523777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc406f060 0x7fecc40ae0d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.335+0000 7fecca79d700 1 -- 172.21.15.119:0/4079523777 >> 172.21.15.119:0/4079523777 conn(0x7fecc40cc1e0 msgr2=0x7fecc40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.335+0000 7fecca79d700 1 -- 172.21.15.119:0/4079523777 shutdown_connections 2023-10-08T01:15:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.335+0000 7fecca79d700 1 -- 172.21.15.119:0/4079523777 wait complete. 2023-10-08T01:15:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.336+0000 7fecca79d700 1 Processor -- start 2023-10-08T01:15:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.336+0000 7fecca79d700 1 -- start start 2023-10-08T01:15:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.337+0000 7fecca79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc406f060 0x7fecc40c0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.337+0000 7fecca79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc406f4b0 0x7fecc4156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.337+0000 7fecca79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40cb1a0 0x7fecc41572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.337+0000 7fecca79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40c2170 con 0x7fecc40cb1a0 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.337+0000 7fecca79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40c1560 con 0x7fecc406f060 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.337+0000 7fecca79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc40c1840 con 0x7fecc406f4b0 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40cb1a0 0x7fecc41572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40cb1a0 0x7fecc41572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43870/0 (socket says 172.21.15.119:43870) 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 -- 172.21.15.119:0/4031989258 learned_addr learned my addr 172.21.15.119:0/4031989258 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc979b700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc406f060 0x7fecc40c0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc8f9a700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc406f4b0 0x7fecc4156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 -- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc406f4b0 msgr2=0x7fecc4156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc406f4b0 0x7fecc4156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 -- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc406f060 msgr2=0x7fecc40c0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc406f060 0x7fecc40c0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 -- 172.21.15.119:0/4031989258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb4009420 con 0x7fecc40cb1a0 2023-10-08T01:15:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc8f9a700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc406f4b0 0x7fecc4156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.338+0000 7fecc9f9c700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40cb1a0 0x7fecc41572d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c8e0 tx=0x7fecbc001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.339+0000 7fecb97fa700 1 -- 172.21.15.119:0/4031989258 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc012630 con 0x7fecc40cb1a0 2023-10-08T01:15:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.339+0000 7fecb97fa700 1 -- 172.21.15.119:0/4031989258 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecbc0127b0 con 0x7fecc40cb1a0 2023-10-08T01:15:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.339+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc40c1b20 con 0x7fecc40cb1a0 2023-10-08T01:15:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.339+0000 7fecb97fa700 1 -- 172.21.15.119:0/4031989258 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecbc012b70 con 0x7fecc40cb1a0 2023-10-08T01:15:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.339+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc40c2600 con 0x7fecc40cb1a0 2023-10-08T01:15:12.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.340+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc40c28e0 con 0x7fecc40cb1a0 2023-10-08T01:15:12.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.341+0000 7fecb97fa700 1 -- 172.21.15.119:0/4031989258 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fecbc003030 con 0x7fecc40cb1a0 2023-10-08T01:15:12.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.342+0000 7fecb97fa700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feca8066f30 0x7feca80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.342+0000 7fecb97fa700 1 -- 172.21.15.119:0/4031989258 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5961+0+0 (secure 0 0 0) 0x7fecbc084690 con 0x7fecc40cb1a0 2023-10-08T01:15:12.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.342+0000 7fecc979b700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feca8066f30 0x7feca80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.343+0000 7fecc979b700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feca8066f30 0x7feca80693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fecc0007fd0 tx=0x7fecc0009040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:12.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.348+0000 7fecb97fa700 1 -- 172.21.15.119:0/4031989258 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fecbc051760 con 0x7fecc40cb1a0 2023-10-08T01:15:12.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.542+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fecc415b000 con 0x7fecc40cb1a0 2023-10-08T01:15:12.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.546+0000 7fecb97fa700 1 -- 172.21.15.119:0/4031989258 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v44) v1 ==== 115+0+0 (secure 0 0 0) 0x7fecbc054d80 con 0x7fecc40cb1a0 2023-10-08T01:15:12.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feca8066f30 msgr2=0x7feca80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feca8066f30 0x7feca80693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fecc0007fd0 tx=0x7fecc0009040 comp rx=0 tx=0).stop 2023-10-08T01:15:12.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40cb1a0 msgr2=0x7fecc41572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40cb1a0 0x7fecc41572d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fecbc00c8e0 tx=0x7fecbc001710 comp rx=0 tx=0).stop 2023-10-08T01:15:12.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 shutdown_connections 2023-10-08T01:15:12.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feca8066f30 0x7feca80693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecc40cb1a0 0x7fecc41572d0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fecc406f060 0x7fecc40c0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 --2- 172.21.15.119:0/4031989258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fecc406f4b0 0x7fecc4156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 >> 172.21.15.119:0/4031989258 conn(0x7fecc40cc1e0 msgr2=0x7fecc4154880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:12.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 shutdown_connections 2023-10-08T01:15:12.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.549+0000 7fecca79d700 1 -- 172.21.15.119:0/4031989258 wait complete. 2023-10-08T01:15:12.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T01:15:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.761+0000 7fd43759e700 1 Processor -- start 2023-10-08T01:15:12.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.761+0000 7fd43759e700 1 -- start start 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.762+0000 7fd43759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd438140bb0 0x7fd43806d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.762+0000 7fd43759e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd43806da30 0x7fd4380c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.762+0000 7fd43759e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4380c6820 con 0x7fd4381407e0 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.762+0000 7fd43759e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4380c6de0 con 0x7fd43806da30 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.762+0000 7fd43759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43806de60 con 0x7fd438140bb0 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.762+0000 7fd43659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd4381407e0 0x7fd43806a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34292/0 (socket says 172.21.15.119:34292) 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.762+0000 7fd43659c700 1 -- 172.21.15.119:0/1114698464 learned_addr learned my addr 172.21.15.119:0/1114698464 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd435d9b700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd438140bb0 0x7fd43806d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd436d9d700 1 --2- 172.21.15.119:0/1114698464 >> v2:172.21.15.184:3300/0 conn(0x7fd43806da30 0x7fd4380c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101245459 0 0) 0x7fd4380c6820 con 0x7fd4381407e0 2023-10-08T01:15:12.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd420003570 con 0x7fd4381407e0 2023-10-08T01:15:12.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2769480073 0 0) 0x7fd420003570 con 0x7fd4381407e0 2023-10-08T01:15:12.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd438140bb0 msgr2=0x7fd43806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:12.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd438140bb0 0x7fd43806d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 >> v2:172.21.15.184:3300/0 conn(0x7fd43806da30 msgr2=0x7fd4380c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 --2- 172.21.15.119:0/1114698464 >> v2:172.21.15.184:3300/0 conn(0x7fd43806da30 0x7fd4380c7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd438141270 con 0x7fd4381407e0 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fd428001470 con 0x7fd4381407e0 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 >> v1:172.21.15.119:6789/0 conn(0x7fd4381407e0 legacy=0x7fd43806a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd420003a60 0x7fd420003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd420004810 0x7fd420004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd438140bb0 0x7fd4200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380c6de0 con 0x7fd420003a60 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43806de60 con 0x7fd420004810 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.763+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42000ba60 con 0x7fd438140bb0 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43659c700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd420003a60 0x7fd420003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd435d9b700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd420004810 0x7fd420004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd436d9d700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd438140bb0 0x7fd4200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43659c700 1 -- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd438140bb0 msgr2=0x7fd4200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43659c700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd438140bb0 0x7fd4200051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43659c700 1 -- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd420004810 msgr2=0x7fd420004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43659c700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd420004810 0x7fd420004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43659c700 1 -- 172.21.15.119:0/1114698464 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd420003570 con 0x7fd420003a60 2023-10-08T01:15:12.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd435d9b700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd420004810 0x7fd420004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43659c700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd420003a60 0x7fd420003e70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd42800c3f0 tx=0x7fd428001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87088a57e7455b7e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42800cc80 con 0x7fd420003a60 2023-10-08T01:15:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42800ce20 con 0x7fd420003a60 2023-10-08T01:15:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.764+0000 7fd43559a700 1 -- 172.21.15.119:0/1114698464 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd428005b70 con 0x7fd420003a60 2023-10-08T01:15:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.765+0000 7fd43759e700 1 -- 172.21.15.119:0/1114698464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd420003a60 msgr2=0x7fd420003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.765+0000 7fd43759e700 1 --2- 172.21.15.119:0/1114698464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd420003a60 0x7fd420003e70 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fd42800c3f0 tx=0x7fd428001e90 comp rx=0 tx=0).stop 2023-10-08T01:15:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.765+0000 7fd436d9d700 1 -- 172.21.15.119:0/1114698464 reap_dead start 2023-10-08T01:15:12.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.765+0000 7fd43759e700 1 -- 172.21.15.119:0/1114698464 shutdown_connections 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.765+0000 7fd43759e700 1 -- 172.21.15.119:0/1114698464 >> 172.21.15.119:0/1114698464 conn(0x7fd4380cc1e0 msgr2=0x7fd4380cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.765+0000 7fd43759e700 1 -- 172.21.15.119:0/1114698464 shutdown_connections 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.765+0000 7fd43759e700 1 -- 172.21.15.119:0/1114698464 wait complete. 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.766+0000 7fd43759e700 1 Processor -- start 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.767+0000 7fd43759e700 1 -- start start 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.767+0000 7fd43759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd420003a60 0x7fd4381696c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.767+0000 7fd43759e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4380c40a0 0x7fd4380c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.767+0000 7fd43759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4380c23a0 0x7fd4380c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.767+0000 7fd43759e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380c2d50 con 0x7fd4380c23a0 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd43759e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380c4640 con 0x7fd4380c40a0 2023-10-08T01:15:12.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd43759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4380c4920 con 0x7fd420003a60 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4380c23a0 0x7fd4380c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4380c23a0 0x7fd4380c2810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43886/0 (socket says 172.21.15.119:43886) 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 -- 172.21.15.119:0/564792572 learned_addr learned my addr 172.21.15.119:0/564792572 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd435d9b700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4380c40a0 0x7fd4380c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 -- 172.21.15.119:0/564792572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd420003a60 msgr2=0x7fd4381696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd420003a60 0x7fd4381696c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 -- 172.21.15.119:0/564792572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4380c40a0 msgr2=0x7fd4380c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4380c40a0 0x7fd4380c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 -- 172.21.15.119:0/564792572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4380c4c00 con 0x7fd4380c23a0 2023-10-08T01:15:12.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.768+0000 7fd436d9d700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4380c23a0 0x7fd4380c2810 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd43806cfa0 tx=0x7fd43000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.769+0000 7fd4267fc700 1 -- 172.21.15.119:0/564792572 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43000e070 con 0x7fd4380c23a0 2023-10-08T01:15:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.769+0000 7fd4267fc700 1 -- 172.21.15.119:0/564792572 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430011d00 con 0x7fd4380c23a0 2023-10-08T01:15:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.769+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4380c4ee0 con 0x7fd4380c23a0 2023-10-08T01:15:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.769+0000 7fd4267fc700 1 -- 172.21.15.119:0/564792572 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4300166e0 con 0x7fd4380c23a0 2023-10-08T01:15:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.769+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd438169c00 con 0x7fd4380c23a0 2023-10-08T01:15:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.770+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4380c5400 con 0x7fd4380c23a0 2023-10-08T01:15:12.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.770+0000 7fd4267fc700 1 -- 172.21.15.119:0/564792572 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd430016a90 con 0x7fd4380c23a0 2023-10-08T01:15:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.771+0000 7fd4267fc700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd418066f30 0x7fd4180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.771+0000 7fd4267fc700 1 -- 172.21.15.119:0/564792572 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5961+0+0 (secure 0 0 0) 0x7fd430085130 con 0x7fd4380c23a0 2023-10-08T01:15:12.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.772+0000 7fd43659c700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd418066f30 0x7fd4180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.772+0000 7fd43659c700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd418066f30 0x7fd4180693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd42800c3f0 tx=0x7fd428015000 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:12.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.775+0000 7fd4267fc700 1 -- 172.21.15.119:0/564792572 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4380c5400 con 0x7fd4380c23a0 2023-10-08T01:15:12.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.971+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fd4380c5400 con 0x7fd4380c23a0 2023-10-08T01:15:12.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.975+0000 7fd4267fc700 1 -- 172.21.15.119:0/564792572 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v45) v1 ==== 120+0+0 (secure 0 0 0) 0x7fd4380c5400 con 0x7fd4380c23a0 2023-10-08T01:15:12.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd418066f30 msgr2=0x7fd4180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd418066f30 0x7fd4180693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd42800c3f0 tx=0x7fd428015000 comp rx=0 tx=0).stop 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4380c23a0 msgr2=0x7fd4380c2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4380c23a0 0x7fd4380c2810 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fd43806cfa0 tx=0x7fd43000b040 comp rx=0 tx=0).stop 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 shutdown_connections 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd418066f30 0x7fd4180693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd420003a60 0x7fd4381696c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4380c23a0 0x7fd4380c2810 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 --2- 172.21.15.119:0/564792572 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd4380c40a0 0x7fd4380c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 >> 172.21.15.119:0/564792572 conn(0x7fd4380cc1e0 msgr2=0x7fd4380c7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 shutdown_connections 2023-10-08T01:15:12.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:12.978+0000 7fd43759e700 1 -- 172.21.15.119:0/564792572 wait complete. 2023-10-08T01:15:12.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-10-08T01:15:13.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.179+0000 7f0cf499d700 1 Processor -- start 2023-10-08T01:15:13.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.179+0000 7f0cf499d700 1 -- start start 2023-10-08T01:15:13.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.180+0000 7f0cf499d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0cf0157770 0x7f0cf0150180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.180+0000 7f0cf499d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf006e2a0 con 0x7f0cf00c6d00 2023-10-08T01:15:13.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.180+0000 7f0cf499d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf0157b40 con 0x7f0cf00c7150 2023-10-08T01:15:13.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.180+0000 7f0cf499d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf006b2e0 con 0x7f0cf0157770 2023-10-08T01:15:13.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.180+0000 7f0cef7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0cf0157770 0x7f0cf0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cef7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0cf0157770 0x7f0cf0150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43900/0 (socket says 172.21.15.119:43900) 2023-10-08T01:15:13.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cef7fe700 1 -- 172.21.15.119:0/1624281465 learned_addr learned my addr 172.21.15.119:0/1624281465 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:13.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0ceeffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0cf00c6d00 0x7f0cf006e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34298/0 (socket says 172.21.15.119:34298) 2023-10-08T01:15:13.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cef7fe700 1 -- 172.21.15.119:0/1624281465 >> v1:172.21.15.184:6789/0 conn(0x7f0cf00c7150 legacy=0x7f0cf006ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:13.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cef7fe700 1 -- 172.21.15.119:0/1624281465 >> v1:172.21.15.119:6789/0 conn(0x7f0cf00c6d00 legacy=0x7f0cf006e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cef7fe700 1 -- 172.21.15.119:0/1624281465 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf00c78e0 con 0x7f0cf0157770 2023-10-08T01:15:13.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cee7fc700 1 -- 172.21.15.119:0/1624281465 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009134390 0 0) 0x7f0cf006e2a0 con 0x7f0cf00c6d00 2023-10-08T01:15:13.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/1624281465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf0157770 0x7f0cf0150180 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00099b0 tx=0x7f0ce0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c48e2df7966d6d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:13.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cee7fc700 1 -- 172.21.15.119:0/1624281465 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000b820 con 0x7f0cf0157770 2023-10-08T01:15:13.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cee7fc700 1 -- 172.21.15.119:0/1624281465 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce000b980 con 0x7f0cf0157770 2023-10-08T01:15:13.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.181+0000 7f0cee7fc700 1 -- 172.21.15.119:0/1624281465 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000bd20 con 0x7f0cf0157770 2023-10-08T01:15:13.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.182+0000 7f0cf499d700 1 -- 172.21.15.119:0/1624281465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf0157770 msgr2=0x7f0cf0150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.182+0000 7f0cf499d700 1 --2- 172.21.15.119:0/1624281465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf0157770 0x7f0cf0150180 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00099b0 tx=0x7f0ce0009250 comp rx=0 tx=0).stop 2023-10-08T01:15:13.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.182+0000 7f0cf499d700 1 -- 172.21.15.119:0/1624281465 shutdown_connections 2023-10-08T01:15:13.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.182+0000 7f0cf499d700 1 --2- 172.21.15.119:0/1624281465 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf0157770 0x7f0cf0150180 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.182+0000 7f0cf499d700 1 -- 172.21.15.119:0/1624281465 >> 172.21.15.119:0/1624281465 conn(0x7f0cf00cc1f0 msgr2=0x7f0cf00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:13.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.182+0000 7f0cf499d700 1 -- 172.21.15.119:0/1624281465 shutdown_connections 2023-10-08T01:15:13.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.182+0000 7f0cf499d700 1 -- 172.21.15.119:0/1624281465 wait complete. 2023-10-08T01:15:13.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.183+0000 7f0cf499d700 1 Processor -- start 2023-10-08T01:15:13.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.184+0000 7f0cf499d700 1 -- start start 2023-10-08T01:15:13.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.184+0000 7f0cf499d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf00c6d00 0x7f0cf01448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.184+0000 7f0cf499d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf00c7150 0x7f0cf013ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.184+0000 7f0cf499d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0157770 0x7f0cf00c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.184+0000 7f0cf499d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0144e20 con 0x7f0cf00c6d00 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.184+0000 7f0cf499d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0143840 con 0x7f0cf00c7150 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.184+0000 7f0cf499d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0143b20 con 0x7f0cf0157770 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf00c6d00 0x7f0cf01448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf00c6d00 0x7f0cf01448e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43902/0 (socket says 172.21.15.119:43902) 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 -- 172.21.15.119:0/2497767559 learned_addr learned my addr 172.21.15.119:0/2497767559 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf00c7150 0x7f0cf013ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0ceffff700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0157770 0x7f0cf00c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 -- 172.21.15.119:0/2497767559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0157770 msgr2=0x7f0cf00c4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0157770 0x7f0cf00c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 -- 172.21.15.119:0/2497767559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf00c7150 msgr2=0x7f0cf013ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf00c7150 0x7f0cf013ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 -- 172.21.15.119:0/2497767559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce00094f0 con 0x7f0cf00c6d00 2023-10-08T01:15:13.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.185+0000 7f0cef7fe700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf00c6d00 0x7f0cf01448e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009290 tx=0x7f0ce0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:13.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.186+0000 7f0cd77fe700 1 -- 172.21.15.119:0/2497767559 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce000bc40 con 0x7f0cf00c6d00 2023-10-08T01:15:13.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.186+0000 7f0cd77fe700 1 -- 172.21.15.119:0/2497767559 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce0020070 con 0x7f0cf00c6d00 2023-10-08T01:15:13.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.186+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf0143e00 con 0x7f0cf00c6d00 2023-10-08T01:15:13.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.186+0000 7f0cd77fe700 1 -- 172.21.15.119:0/2497767559 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce001b6a0 con 0x7f0cf00c6d00 2023-10-08T01:15:13.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.186+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf01443c0 con 0x7f0cf00c6d00 2023-10-08T01:15:13.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.187+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf00ab7a0 con 0x7f0cf00c6d00 2023-10-08T01:15:13.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.188+0000 7f0cd77fe700 1 -- 172.21.15.119:0/2497767559 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ce0022030 con 0x7f0cf00c6d00 2023-10-08T01:15:13.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.188+0000 7f0cd77fe700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.189+0000 7f0cd77fe700 1 -- 172.21.15.119:0/2497767559 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f0ce0086680 con 0x7f0cf00c6d00 2023-10-08T01:15:13.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.189+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.189+0000 7f0ceeffd700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4007fd0 tx=0x7f0ce4007e10 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:13.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.193+0000 7f0cd77fe700 1 -- 172.21.15.119:0/2497767559 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ce0053750 con 0x7f0cf00c6d00 2023-10-08T01:15:13.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.460+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f0cf00ab7a0 con 0x7f0cf00c6d00 2023-10-08T01:15:13.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.716+0000 7f0cd77fe700 1 -- 172.21.15.119:0/2497767559 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v732) v1 ==== 148+0+0 (secure 0 0 0) 0x7f0ce0056d70 con 0x7f0cf00c6d00 2023-10-08T01:15:13.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 msgr2=0x7f0cdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4007fd0 tx=0x7f0ce4007e10 comp rx=0 tx=0).stop 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf00c6d00 msgr2=0x7f0cf01448e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf00c6d00 0x7f0cf01448e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f0ce0009290 tx=0x7f0ce0016000 comp rx=0 tx=0).stop 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 shutdown_connections 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0cdc066f30 0x7f0cdc0693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0cf0157770 0x7f0cf00c4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0cf00c6d00 0x7f0cf01448e0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 --2- 172.21.15.119:0/2497767559 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0cf00c7150 0x7f0cf013ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 >> 172.21.15.119:0/2497767559 conn(0x7f0cf00cc1f0 msgr2=0x7f0cf00c2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:13.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.718+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 shutdown_connections 2023-10-08T01:15:13.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.719+0000 7f0cf499d700 1 -- 172.21.15.119:0/2497767559 wait complete. 2023-10-08T01:15:13.737 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-10-08T01:15:13.737 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-10-08T01:15:13.737 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T01:15:13.738 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-08T01:15:13.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.918+0000 7f7b477ff700 1 Processor -- start 2023-10-08T01:15:13.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.919+0000 7f7b477ff700 1 -- start start 2023-10-08T01:15:13.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b477ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b400cbf30 0x7f7b400ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b477ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b400ccf90 0x7f7b400b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b477ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b4015a270 0x7f7b400b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b477ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b40156bf0 con 0x7f7b400ccf90 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b477ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b400ae950 con 0x7f7b400cbf30 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b477ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b400cc300 con 0x7f7b4015a270 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b44d9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b400ccf90 0x7f7b400b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b44d9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b400ccf90 0x7f7b400b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43918/0 (socket says 172.21.15.119:43918) 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b44d9a700 1 -- 172.21.15.119:0/1135125665 learned_addr learned my addr 172.21.15.119:0/1135125665 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.920+0000 7f7b4559b700 1 --2- 172.21.15.119:0/1135125665 >> v2:172.21.15.184:3300/0 conn(0x7f7b400cbf30 0x7f7b400ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b45d9c700 1 --2- 172.21.15.119:0/1135125665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b4015a270 0x7f7b400b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b44d9a700 1 -- 172.21.15.119:0/1135125665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b4015a270 msgr2=0x7f7b400b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b44d9a700 1 --2- 172.21.15.119:0/1135125665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b4015a270 0x7f7b400b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b44d9a700 1 -- 172.21.15.119:0/1135125665 >> v2:172.21.15.184:3300/0 conn(0x7f7b400cbf30 msgr2=0x7f7b400ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:13.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b44d9a700 1 --2- 172.21.15.119:0/1135125665 >> v2:172.21.15.184:3300/0 conn(0x7f7b400cbf30 0x7f7b400ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b44d9a700 1 -- 172.21.15.119:0/1135125665 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b400ad370 con 0x7f7b400ccf90 2023-10-08T01:15:13.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b4559b700 1 --2- 172.21.15.119:0/1135125665 >> v2:172.21.15.184:3300/0 conn(0x7f7b400cbf30 0x7f7b400ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:13.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b44d9a700 1 --2- 172.21.15.119:0/1135125665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b400ccf90 0x7f7b400b0d10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f7b40063d40 tx=0x7f7b3000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d882663150004d62 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:13.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b37fff700 1 -- 172.21.15.119:0/1135125665 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3000be60 con 0x7f7b400ccf90 2023-10-08T01:15:13.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b37fff700 1 -- 172.21.15.119:0/1135125665 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3000ddc0 con 0x7f7b400ccf90 2023-10-08T01:15:13.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.921+0000 7f7b37fff700 1 -- 172.21.15.119:0/1135125665 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b300196b0 con 0x7f7b400ccf90 2023-10-08T01:15:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 -- 172.21.15.119:0/1135125665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b400ccf90 msgr2=0x7f7b400b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 --2- 172.21.15.119:0/1135125665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b400ccf90 0x7f7b400b0d10 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f7b40063d40 tx=0x7f7b3000b4f0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 -- 172.21.15.119:0/1135125665 shutdown_connections 2023-10-08T01:15:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 --2- 172.21.15.119:0/1135125665 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b4015a270 0x7f7b400b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 --2- 172.21.15.119:0/1135125665 >> v2:172.21.15.184:3300/0 conn(0x7f7b400cbf30 0x7f7b400ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 --2- 172.21.15.119:0/1135125665 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b400ccf90 0x7f7b400b0d10 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 -- 172.21.15.119:0/1135125665 >> 172.21.15.119:0/1135125665 conn(0x7f7b4006a320 msgr2=0x7f7b400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 -- 172.21.15.119:0/1135125665 shutdown_connections 2023-10-08T01:15:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.922+0000 7f7b477ff700 1 -- 172.21.15.119:0/1135125665 wait complete. 2023-10-08T01:15:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.923+0000 7f7b477ff700 1 Processor -- start 2023-10-08T01:15:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.924+0000 7f7b477ff700 1 -- start start 2023-10-08T01:15:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.924+0000 7f7b477ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400cbf30 0x7f7b401658b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.924+0000 7f7b477ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400ccf90 0x7f7b400abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.924+0000 7f7b477ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4015a270 0x7f7b40154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b477ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b400aaf90 con 0x7f7b4015a270 2023-10-08T01:15:13.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b477ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4016a5b0 con 0x7f7b400ccf90 2023-10-08T01:15:13.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b477ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b400ab1c0 con 0x7f7b400cbf30 2023-10-08T01:15:13.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4015a270 0x7f7b40154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4015a270 0x7f7b40154fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43930/0 (socket says 172.21.15.119:43930) 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 -- 172.21.15.119:0/2744748527 learned_addr learned my addr 172.21.15.119:0/2744748527 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b44d9a700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400ccf90 0x7f7b400abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b4559b700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400cbf30 0x7f7b401658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 -- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400cbf30 msgr2=0x7f7b401658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400cbf30 0x7f7b401658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 -- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400ccf90 msgr2=0x7f7b400abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400ccf90 0x7f7b400abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 -- 172.21.15.119:0/2744748527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3000b790 con 0x7f7b4015a270 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b44d9a700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400ccf90 0x7f7b400abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.925+0000 7f7b45d9c700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4015a270 0x7f7b40154fe0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009fa0 tx=0x7f7b38009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.926+0000 7f7b357fa700 1 -- 172.21.15.119:0/2744748527 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3800ed20 con 0x7f7b4015a270 2023-10-08T01:15:13.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.926+0000 7f7b357fa700 1 -- 172.21.15.119:0/2744748527 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3800ee80 con 0x7f7b4015a270 2023-10-08T01:15:13.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.926+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b400ab4a0 con 0x7f7b4015a270 2023-10-08T01:15:13.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.926+0000 7f7b357fa700 1 -- 172.21.15.119:0/2744748527 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b38014670 con 0x7f7b4015a270 2023-10-08T01:15:13.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.926+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b400abae0 con 0x7f7b4015a270 2023-10-08T01:15:13.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.927+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b40165eb0 con 0x7f7b4015a270 2023-10-08T01:15:13.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.928+0000 7f7b357fa700 1 -- 172.21.15.119:0/2744748527 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7b380149e0 con 0x7f7b4015a270 2023-10-08T01:15:13.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.928+0000 7f7b357fa700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7b24066f00 0x7f7b240693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:13.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.928+0000 7f7b357fa700 1 -- 172.21.15.119:0/2744748527 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f7b38084e80 con 0x7f7b4015a270 2023-10-08T01:15:13.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.929+0000 7f7b4559b700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7b24066f00 0x7f7b240693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:13.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.929+0000 7f7b4559b700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7b24066f00 0x7f7b240693b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c006f00 tx=0x7f7b3c008040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:13.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:13.933+0000 7f7b357fa700 1 -- 172.21.15.119:0/2744748527 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b38051f50 con 0x7f7b4015a270 2023-10-08T01:15:14.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.200+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f7b4004c3c0 con 0x7f7b4015a270 2023-10-08T01:15:14.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.717+0000 7f7b357fa700 1 -- 172.21.15.119:0/2744748527 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v733) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7b38055570 con 0x7f7b4015a270 2023-10-08T01:15:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7b24066f00 msgr2=0x7f7b240693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7b24066f00 0x7f7b240693b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c006f00 tx=0x7f7b3c008040 comp rx=0 tx=0).stop 2023-10-08T01:15:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4015a270 msgr2=0x7f7b40154fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4015a270 0x7f7b40154fe0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009fa0 tx=0x7f7b38009900 comp rx=0 tx=0).stop 2023-10-08T01:15:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 shutdown_connections 2023-10-08T01:15:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7b24066f00 0x7f7b240693b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b400cbf30 0x7f7b401658b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b4015a270 0x7f7b40154fe0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 --2- 172.21.15.119:0/2744748527 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b400ccf90 0x7f7b400abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 >> 172.21.15.119:0/2744748527 conn(0x7f7b4006a320 msgr2=0x7f7b40068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 shutdown_connections 2023-10-08T01:15:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.720+0000 7f7b477ff700 1 -- 172.21.15.119:0/2744748527 wait complete. 2023-10-08T01:15:14.740 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T01:15:14.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.924+0000 7ff8a2bff700 1 Processor -- start 2023-10-08T01:15:14.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.924+0000 7ff8a2bff700 1 -- start start 2023-10-08T01:15:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a2bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff89c159890 0x7ff89c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a2bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 0x7ff89c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a2bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff89c15a160 con 0x7ff89c15db10 2023-10-08T01:15:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a2bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff89c0d13a0 con 0x7ff89c159890 2023-10-08T01:15:14.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a2bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c15e140 con 0x7ff89c159c60 2023-10-08T01:15:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a099b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff89c159890 0x7ff89c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a099b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff89c159890 0x7ff89c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49578/0 (socket says 172.21.15.119:49578) 2023-10-08T01:15:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a119c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff89c15db10 0x7ff89c069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50924/0 (socket says 172.21.15.119:50924) 2023-10-08T01:15:14.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.926+0000 7ff8a099b700 1 -- 172.21.15.119:0/3099126849 learned_addr learned my addr 172.21.15.119:0/3099126849 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff89bfff700 1 --2- 172.21.15.119:0/3099126849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 0x7ff89c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff8a099b700 1 -- 172.21.15.119:0/3099126849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 msgr2=0x7ff89c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff8a099b700 1 --2- 172.21.15.119:0/3099126849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 0x7ff89c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff8a099b700 1 -- 172.21.15.119:0/3099126849 >> v1:172.21.15.119:6789/0 conn(0x7ff89c15db10 legacy=0x7ff89c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:14.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff8a099b700 1 -- 172.21.15.119:0/3099126849 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff89c15a3d0 con 0x7ff89c159890 2023-10-08T01:15:14.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff89b7fe700 1 -- 172.21.15.119:0/3099126849 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930102830 0 0) 0x7ff89c15a160 con 0x7ff89c15db10 2023-10-08T01:15:14.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff8a099b700 1 --2- 172.21.15.119:0/3099126849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c15abd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff8900099b0 tx=0x7ff890009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e7c35259f1b98f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:14.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff89b7fe700 1 -- 172.21.15.119:0/3099126849 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff890009ea0 con 0x7ff89c159890 2023-10-08T01:15:14.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff89b7fe700 1 -- 172.21.15.119:0/3099126849 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89000b860 con 0x7ff89c159890 2023-10-08T01:15:14.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.927+0000 7ff89b7fe700 1 -- 172.21.15.119:0/3099126849 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89000bc00 con 0x7ff89c159890 2023-10-08T01:15:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.928+0000 7ff8a2bff700 1 -- 172.21.15.119:0/3099126849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 msgr2=0x7ff89c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.928+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/3099126849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c15abd0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff8900099b0 tx=0x7ff890009250 comp rx=0 tx=0).stop 2023-10-08T01:15:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.928+0000 7ff8a2bff700 1 -- 172.21.15.119:0/3099126849 shutdown_connections 2023-10-08T01:15:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.928+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/3099126849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c15abd0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.928+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/3099126849 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 0x7ff89c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.928+0000 7ff8a2bff700 1 -- 172.21.15.119:0/3099126849 >> 172.21.15.119:0/3099126849 conn(0x7ff89c068e20 msgr2=0x7ff89c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.928+0000 7ff8a2bff700 1 -- 172.21.15.119:0/3099126849 shutdown_connections 2023-10-08T01:15:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.928+0000 7ff8a2bff700 1 -- 172.21.15.119:0/3099126849 wait complete. 2023-10-08T01:15:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.929+0000 7ff8a2bff700 1 Processor -- start 2023-10-08T01:15:14.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.930+0000 7ff8a2bff700 1 -- start start 2023-10-08T01:15:14.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.930+0000 7ff8a2bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c14b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:14.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.930+0000 7ff8a2bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 0x7ff89c0c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:14.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.930+0000 7ff8a2bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c15db10 0x7ff89c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:14.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.930+0000 7ff8a2bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0c6bf0 con 0x7ff89c159890 2023-10-08T01:15:14.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.930+0000 7ff8a2bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0c5fe0 con 0x7ff89c15db10 2023-10-08T01:15:14.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.930+0000 7ff8a2bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff89c0c62c0 con 0x7ff89c159c60 2023-10-08T01:15:14.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c14b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c14b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49586/0 (socket says 172.21.15.119:49586) 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 -- 172.21.15.119:0/1467694654 learned_addr learned my addr 172.21.15.119:0/1467694654 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff89bfff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 0x7ff89c0c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a119c700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c15db10 0x7ff89c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 -- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 msgr2=0x7ff89c0c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 0x7ff89c0c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 -- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c15db10 msgr2=0x7ff89c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c15db10 0x7ff89c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 -- 172.21.15.119:0/1467694654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8900094f0 con 0x7ff89c159890 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a119c700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c15db10 0x7ff89c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:14.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.931+0000 7ff8a099b700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c14b6f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0899c0 tx=0x7ff890017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.932+0000 7ff898ff9700 1 -- 172.21.15.119:0/1467694654 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89000bbe0 con 0x7ff89c159890 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.932+0000 7ff898ff9700 1 -- 172.21.15.119:0/1467694654 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff890011420 con 0x7ff89c159890 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.932+0000 7ff898ff9700 1 -- 172.21.15.119:0/1467694654 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8900117e0 con 0x7ff89c159890 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.932+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff89c0c65a0 con 0x7ff89c159890 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.932+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff89c15c740 con 0x7ff89c159890 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.933+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89c04c3c0 con 0x7ff89c159890 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.934+0000 7ff898ff9700 1 -- 172.21.15.119:0/1467694654 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff890011c10 con 0x7ff89c159890 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.934+0000 7ff898ff9700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff880066f30 0x7ff8800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.935+0000 7ff898ff9700 1 -- 172.21.15.119:0/1467694654 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5961+0+0 (secure 0 0 0) 0x7ff890085f80 con 0x7ff89c159890 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.935+0000 7ff89bfff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff880066f30 0x7ff8800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.935+0000 7ff89bfff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff880066f30 0x7ff8800693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff89c151240 tx=0x7ff888008040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:14.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:14.938+0000 7ff898ff9700 1 -- 172.21.15.119:0/1467694654 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8900b75a0 con 0x7ff89c159890 2023-10-08T01:15:15.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.129+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff89c06a5b0 con 0x7ff89c159890 2023-10-08T01:15:15.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.133+0000 7ff898ff9700 1 -- 172.21.15.119:0/1467694654 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8900530f0 con 0x7ff89c159890 2023-10-08T01:15:15.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.135+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff880066f30 msgr2=0x7ff8800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:15.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.135+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff880066f30 0x7ff8800693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff89c151240 tx=0x7ff888008040 comp rx=0 tx=0).stop 2023-10-08T01:15:15.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.135+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 msgr2=0x7ff89c14b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:15.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c14b6f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff89c0899c0 tx=0x7ff890017000 comp rx=0 tx=0).stop 2023-10-08T01:15:15.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 shutdown_connections 2023-10-08T01:15:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff880066f30 0x7ff8800693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff89c15db10 0x7ff89c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff89c159890 0x7ff89c14b6f0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 --2- 172.21.15.119:0/1467694654 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff89c159c60 0x7ff89c0c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 >> 172.21.15.119:0/1467694654 conn(0x7ff89c068e20 msgr2=0x7ff89c14d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 shutdown_connections 2023-10-08T01:15:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.136+0000 7ff8a2bff700 1 -- 172.21.15.119:0/1467694654 wait complete. 2023-10-08T01:15:15.155 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T01:15:15.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:15:15.236 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T01:15:15.323 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T01:15:15.323 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T01:15:15.323 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:15:15.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.504+0000 7f548ffff700 1 Processor -- start 2023-10-08T01:15:15.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.504+0000 7f548ffff700 1 -- start start 2023-10-08T01:15:15.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54901566e0 0x7f54900ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:15.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54900cb900 0x7f54900ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:15.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548ffff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900ca860 con 0x7f5490156310 2023-10-08T01:15:15.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548ffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54900cab40 con 0x7f54900cb900 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900cae20 con 0x7f54901566e0 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54900cb900 0x7f54900ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54900cb900 0x7f54900ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49594/0 (socket says 172.21.15.119:49594) 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548f7fe700 1 -- 172.21.15.119:0/1303005190 learned_addr learned my addr 172.21.15.119:0/1303005190 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.505+0000 7f548e7fc700 1 --2- 172.21.15.119:0/1303005190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54901566e0 0x7f54900ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.506+0000 7f548f7fe700 1 -- 172.21.15.119:0/1303005190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54901566e0 msgr2=0x7f54900ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.506+0000 7f548f7fe700 1 --2- 172.21.15.119:0/1303005190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54901566e0 0x7f54900ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.506+0000 7f548f7fe700 1 -- 172.21.15.119:0/1303005190 >> v1:172.21.15.184:6789/0 conn(0x7f5490156310 legacy=0x7f54900c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.506+0000 7f548f7fe700 1 -- 172.21.15.119:0/1303005190 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900cb100 con 0x7f54900cb900 2023-10-08T01:15:15.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.506+0000 7f548f7fe700 1 --2- 172.21.15.119:0/1303005190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f54900ae540 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5488009f20 tx=0x7f5488009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e6bc501510042c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:15.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.506+0000 7f548dffb700 1 -- 172.21.15.119:0/1303005190 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800e070 con 0x7f54900cb900 2023-10-08T01:15:15.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.506+0000 7f548dffb700 1 -- 172.21.15.119:0/1303005190 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548800b9d0 con 0x7f54900cb900 2023-10-08T01:15:15.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.506+0000 7f548dffb700 1 -- 172.21.15.119:0/1303005190 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54880092e0 con 0x7f54900cb900 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.507+0000 7f548ffff700 1 -- 172.21.15.119:0/1303005190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 msgr2=0x7f54900ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.507+0000 7f548ffff700 1 --2- 172.21.15.119:0/1303005190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f54900ae540 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f5488009f20 tx=0x7f5488009840 comp rx=0 tx=0).stop 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.507+0000 7f548ffff700 1 -- 172.21.15.119:0/1303005190 shutdown_connections 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.507+0000 7f548ffff700 1 --2- 172.21.15.119:0/1303005190 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f54900ae540 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.507+0000 7f548ffff700 1 --2- 172.21.15.119:0/1303005190 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54901566e0 0x7f54900ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.507+0000 7f548ffff700 1 -- 172.21.15.119:0/1303005190 >> 172.21.15.119:0/1303005190 conn(0x7f54900cc1f0 msgr2=0x7f54900cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.507+0000 7f548ffff700 1 -- 172.21.15.119:0/1303005190 shutdown_connections 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.507+0000 7f548ffff700 1 -- 172.21.15.119:0/1303005190 wait complete. 2023-10-08T01:15:15.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.508+0000 7f548ffff700 1 Processor -- start 2023-10-08T01:15:15.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.509+0000 7f548ffff700 1 -- start start 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.509+0000 7f548ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f549013ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.509+0000 7f548ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490156310 0x7f54900afba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54901566e0 0x7f54900b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549015e000 con 0x7f54900cb900 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549015e2e0 con 0x7f54901566e0 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549015e5c0 con 0x7f5490156310 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f549013ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f549013ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49608/0 (socket says 172.21.15.119:49608) 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 -- 172.21.15.119:0/176837320 learned_addr learned my addr 172.21.15.119:0/176837320 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548e7fc700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490156310 0x7f54900afba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:15.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548f7fe700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54901566e0 0x7f54900b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:15.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 -- 172.21.15.119:0/176837320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490156310 msgr2=0x7f54900afba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:15.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490156310 0x7f54900afba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 -- 172.21.15.119:0/176837320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54901566e0 msgr2=0x7f54900b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:15.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54901566e0 0x7f54900b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 -- 172.21.15.119:0/176837320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009ae0 con 0x7f54900cb900 2023-10-08T01:15:15.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.510+0000 7f548effd700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f549013ab70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f548000c590 tx=0x7f548000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:15.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.511+0000 7f5476ffd700 1 -- 172.21.15.119:0/176837320 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480011590 con 0x7f54900cb900 2023-10-08T01:15:15.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.511+0000 7f5476ffd700 1 -- 172.21.15.119:0/176837320 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480011710 con 0x7f54900cb900 2023-10-08T01:15:15.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.511+0000 7f5476ffd700 1 -- 172.21.15.119:0/176837320 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480011ad0 con 0x7f54900cb900 2023-10-08T01:15:15.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.511+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549015e8a0 con 0x7f54900cb900 2023-10-08T01:15:15.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.511+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549015d290 con 0x7f54900cb900 2023-10-08T01:15:15.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.512+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549015ebf0 con 0x7f54900cb900 2023-10-08T01:15:15.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.513+0000 7f5476ffd700 1 -- 172.21.15.119:0/176837320 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f548001b3b0 con 0x7f54900cb900 2023-10-08T01:15:15.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.514+0000 7f5476ffd700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:15.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.514+0000 7f5476ffd700 1 -- 172.21.15.119:0/176837320 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f5480083b40 con 0x7f54900cb900 2023-10-08T01:15:15.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.514+0000 7f548e7fc700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:15.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.515+0000 7f548e7fc700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f547c066f30 0x7f547c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5490152620 tx=0x7f54840091a0 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:15.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.517+0000 7f5476ffd700 1 -- 172.21.15.119:0/176837320 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5480050c10 con 0x7f54900cb900 2023-10-08T01:15:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.754+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f549004c3c0 con 0x7f54900cb900 2023-10-08T01:15:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.755+0000 7f5476ffd700 1 -- 172.21.15.119:0/176837320 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f5480017070 con 0x7f54900cb900 2023-10-08T01:15:15.756 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:15.756 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":57,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7141,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3118853373","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3118853373},{"type":"v1","addr":"172.21.15.184:6834","nonce":3118853373}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7144,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/737543181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":737543181},{"type":"v1","addr":"172.21.15.184:6835","nonce":737543181}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":7677,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/2667608886","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2667608886},{"type":"v1","addr":"172.21.15.119:6834","nonce":2667608886}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":51},{"gid":8055,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1489989005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":1489989005},{"type":"v1","addr":"172.21.15.119:6835","nonce":1489989005}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[]} 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f547c066f30 msgr2=0x7f547c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f547c066f30 0x7f547c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5490152620 tx=0x7f54840091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 msgr2=0x7f549013ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f549013ab70 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f548000c590 tx=0x7f548000c8a0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 shutdown_connections 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f547c066f30 0x7f547c0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54901566e0 0x7f54900b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb900 0x7f549013ab70 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 --2- 172.21.15.119:0/176837320 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490156310 0x7f54900afba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 >> 172.21.15.119:0/176837320 conn(0x7f54900cc1f0 msgr2=0x7f54900c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 shutdown_connections 2023-10-08T01:15:15.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:15.757+0000 7f548ffff700 1 -- 172.21.15.119:0/176837320 wait complete. 2023-10-08T01:15:15.760 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 57 2023-10-08T01:15:15.776 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T01:15:15.776 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T01:15:15.776 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T01:15:15.776 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:15:15.777 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T01:15:15.777 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T01:15:15.777 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T01:15:15.777 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:15:15.777 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T01:15:15.777 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T01:15:15.777 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T01:15:15.778 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:15:15.778 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T01:15:15.778 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T01:15:15.778 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T01:15:15.778 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T01:15:15.793 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T01:15:15.793 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T01:15:15.794 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T01:15:15.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T01:15:15.795 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T01:15:15.795 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T01:15:15.796 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T01:15:15.796 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T01:15:15.837 INFO:tasks.ceph.mds.a:Started 2023-10-08T01:15:15.838 DEBUG:teuthology.parallel:result is None 2023-10-08T01:15:15.840 INFO:tasks.ceph.mds.c:Started 2023-10-08T01:15:15.840 DEBUG:teuthology.parallel:result is None 2023-10-08T01:15:15.841 INFO:tasks.ceph.mds.b:Started 2023-10-08T01:15:15.841 DEBUG:teuthology.parallel:result is None 2023-10-08T01:15:15.843 INFO:tasks.ceph.mds.d:Started 2023-10-08T01:15:15.843 DEBUG:teuthology.parallel:result is None 2023-10-08T01:15:15.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T01:15:15.973 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T01:15:15.973 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T01:15:15.989 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T01:15:15.989 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T01:15:16.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.037+0000 7f0302dff700 1 Processor -- start 2023-10-08T01:15:16.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.037+0000 7f0302dff700 1 -- start start 2023-10-08T01:15:16.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.038+0000 7f0302dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:16.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.038+0000 7f0302dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f02fc159c60 0x7f02fc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:16.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.038+0000 7f0302dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02fc15a160 con 0x7f02fc15db10 2023-10-08T01:15:16.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.038+0000 7f0302dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02fc0d13a0 con 0x7f02fc159c60 2023-10-08T01:15:16.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.038+0000 7f0302dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc15e140 con 0x7f02fc159890 2023-10-08T01:15:16.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fbfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f02fc159c60 0x7f02fc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fbfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f02fc159c60 0x7f02fc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44658/0 (socket says 172.21.15.119:44658) 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fbfff700 1 -- 172.21.15.119:0/3733832919 learned_addr learned my addr 172.21.15.119:0/3733832919 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f030139c700 1 --1- 172.21.15.119:0/3733832919 >> v1:172.21.15.184:6789/0 conn(0x7f02fc15db10 0x7f02fc069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:39394/0 (socket says 172.21.15.119:39394) 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f0300b9b700 1 --2- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fb7fe700 1 -- 172.21.15.119:0/3733832919 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540467191 0 0) 0x7f02fc15a160 con 0x7f02fc15db10 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fb7fe700 1 -- 172.21.15.119:0/3733832919 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ec003570 con 0x7f02fc15db10 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fbfff700 1 -- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 msgr2=0x7f02fc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fbfff700 1 --2- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:16.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fbfff700 1 -- 172.21.15.119:0/3733832919 >> v1:172.21.15.184:6789/0 conn(0x7f02fc15db10 legacy=0x7f02fc069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f02fbfff700 1 -- 172.21.15.119:0/3733832919 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc15a3d0 con 0x7f02fc159c60 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.039+0000 7f0300b9b700 1 --2- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.040+0000 7f02fbfff700 1 --2- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc159c60 0x7f02fc15d5d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f02e80099b0 tx=0x7f02e8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63934948a714f6d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.040+0000 7f02fb7fe700 1 -- 172.21.15.119:0/3733832919 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800b680 con 0x7f02fc159c60 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.040+0000 7f02fb7fe700 1 -- 172.21.15.119:0/3733832919 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e800b7e0 con 0x7f02fc159c60 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.040+0000 7f02fb7fe700 1 -- 172.21.15.119:0/3733832919 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800bb80 con 0x7f02fc159c60 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.040+0000 7f0302dff700 1 -- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc159c60 msgr2=0x7f02fc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.040+0000 7f0302dff700 1 --2- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc159c60 0x7f02fc15d5d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f02e80099b0 tx=0x7f02e8009250 comp rx=0 tx=0).stop 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.041+0000 7f0302dff700 1 -- 172.21.15.119:0/3733832919 shutdown_connections 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.041+0000 7f0302dff700 1 --2- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.041+0000 7f0302dff700 1 --2- 172.21.15.119:0/3733832919 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc159c60 0x7f02fc15d5d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:16.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.041+0000 7f0302dff700 1 -- 172.21.15.119:0/3733832919 >> 172.21.15.119:0/3733832919 conn(0x7f02fc068e20 msgr2=0x7f02fc0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.041+0000 7f0302dff700 1 -- 172.21.15.119:0/3733832919 shutdown_connections 2023-10-08T01:15:16.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.041+0000 7f0302dff700 1 -- 172.21.15.119:0/3733832919 wait complete. 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.042+0000 7f0302dff700 1 Processor -- start 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.042+0000 7f0302dff700 1 -- start start 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f0302dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc0b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f0302dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc159c60 0x7f02fc15caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f0302dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc15db10 0x7f02fc0b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f0302dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0b0b10 con 0x7f02fc159c60 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f0302dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0b0d40 con 0x7f02fc15db10 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f0302dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc0b1020 con 0x7f02fc159890 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f02fbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc159c60 0x7f02fc15caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:16.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f02fbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc159c60 0x7f02fc15caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49672/0 (socket says 172.21.15.119:49672) 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.043+0000 7f02fbfff700 1 -- 172.21.15.119:0/1586075220 learned_addr learned my addr 172.21.15.119:0/1586075220 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.044+0000 7f0300b9b700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc0b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.044+0000 7f030139c700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc15db10 0x7f02fc0b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.044+0000 7f02fbfff700 1 -- 172.21.15.119:0/1586075220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 msgr2=0x7f02fc0b1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.044+0000 7f02fbfff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc0b1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.044+0000 7f02fbfff700 1 -- 172.21.15.119:0/1586075220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc15db10 msgr2=0x7f02fc0b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.044+0000 7f02fbfff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc15db10 0x7f02fc0b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.044+0000 7f02fbfff700 1 -- 172.21.15.119:0/1586075220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009420 con 0x7f02fc159c60 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.044+0000 7f02fbfff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc159c60 0x7f02fc15caf0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009290 tx=0x7f02e8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.045+0000 7f02f8ff9700 1 -- 172.21.15.119:0/1586075220 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8002f90 con 0x7f02fc159c60 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.045+0000 7f02f8ff9700 1 -- 172.21.15.119:0/1586075220 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e800b950 con 0x7f02fc159c60 2023-10-08T01:15:16.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.045+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e80094f0 con 0x7f02fc159c60 2023-10-08T01:15:16.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.045+0000 7f02f8ff9700 1 -- 172.21.15.119:0/1586075220 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e801a430 con 0x7f02fc159c60 2023-10-08T01:15:16.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.045+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc0b15e0 con 0x7f02fc159c60 2023-10-08T01:15:16.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.046+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02fc04c3c0 con 0x7f02fc159c60 2023-10-08T01:15:16.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.047+0000 7f02f8ff9700 1 -- 172.21.15.119:0/1586075220 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f02e801a7a0 con 0x7f02fc159c60 2023-10-08T01:15:16.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.047+0000 7f02f8ff9700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f02e0066f30 0x7f02e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:16.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.047+0000 7f02f8ff9700 1 -- 172.21.15.119:0/1586075220 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5961+0+0 (secure 0 0 0) 0x7f02e8086210 con 0x7f02fc159c60 2023-10-08T01:15:16.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.048+0000 7f0300b9b700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f02e0066f30 0x7f02e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:16.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.048+0000 7f0300b9b700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f02e0066f30 0x7f02e00693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009f40 tx=0x7f02f00091a0 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:16.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.051+0000 7f02f8ff9700 1 -- 172.21.15.119:0/1586075220 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02e8053380 con 0x7f02fc159c60 2023-10-08T01:15:16.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:16.262+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f02fc0657f0 con 0x7f02fc159c60 2023-10-08T01:15:17.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.156+0000 7f02f8ff9700 1 -- 172.21.15.119:0/1586075220 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v131) v1 ==== 97+0+0 (secure 0 0 0) 0x7f02e80569a0 con 0x7f02fc159c60 2023-10-08T01:15:17.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f02e0066f30 msgr2=0x7f02e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f02e0066f30 0x7f02e00693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009f40 tx=0x7f02f00091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc159c60 msgr2=0x7f02fc15caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc159c60 0x7f02fc15caf0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f02e8009290 tx=0x7f02e8017000 comp rx=0 tx=0).stop 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 shutdown_connections 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f02e0066f30 0x7f02e00693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02fc159890 0x7f02fc0b1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02fc15db10 0x7f02fc0b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 --2- 172.21.15.119:0/1586075220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02fc159c60 0x7f02fc15caf0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 >> 172.21.15.119:0/1586075220 conn(0x7f02fc068e20 msgr2=0x7f02fc14d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 shutdown_connections 2023-10-08T01:15:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.159+0000 7f0302dff700 1 -- 172.21.15.119:0/1586075220 wait complete. 2023-10-08T01:15:17.162 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T01:15:17.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T01:15:17.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.362+0000 7f2b0e5ff700 1 Processor -- start 2023-10-08T01:15:17.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.362+0000 7f2b0e5ff700 1 -- start start 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b0e5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2b080b1420 0x7f2b080ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b0e5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2b080ccf90 0x7f2b080c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b0e5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b080cc300 con 0x7f2b080cbf30 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b0e5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b08156bf0 con 0x7f2b080b1420 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b0e5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b080ae950 con 0x7f2b080ccf90 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b077fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2b080b1420 0x7f2b080ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b077fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2b080b1420 0x7f2b080ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49682/0 (socket says 172.21.15.119:49682) 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b077fe700 1 -- 172.21.15.119:0/3933730724 learned_addr learned my addr 172.21.15.119:0/3933730724 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:17.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b0cb9c700 1 --2- 172.21.15.119:0/3933730724 >> v2:172.21.15.184:3300/0 conn(0x7f2b080ccf90 0x7f2b080c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b077fe700 1 -- 172.21.15.119:0/3933730724 >> v2:172.21.15.184:3300/0 conn(0x7f2b080ccf90 msgr2=0x7f2b080c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b077fe700 1 --2- 172.21.15.119:0/3933730724 >> v2:172.21.15.184:3300/0 conn(0x7f2b080ccf90 0x7f2b080c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b077fe700 1 -- 172.21.15.119:0/3933730724 >> v1:172.21.15.184:6789/0 conn(0x7f2b080cbf30 legacy=0x7f2b080b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.364+0000 7f2b077fe700 1 -- 172.21.15.119:0/3933730724 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b080ad370 con 0x7f2b080b1420 2023-10-08T01:15:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.365+0000 7f2b0cb9c700 1 --2- 172.21.15.119:0/3933730724 >> v2:172.21.15.184:3300/0 conn(0x7f2b080ccf90 0x7f2b080c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.365+0000 7f2b077fe700 1 --2- 172.21.15.119:0/3933730724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080b1420 0x7f2b080ae410 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2b080c8fd0 tx=0x7f2afc00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd83c775d73f677 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.365+0000 7f2b06ffd700 1 -- 172.21.15.119:0/3933730724 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00be60 con 0x7f2b080b1420 2023-10-08T01:15:17.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.365+0000 7f2b06ffd700 1 -- 172.21.15.119:0/3933730724 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc00ddc0 con 0x7f2b080b1420 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.365+0000 7f2b06ffd700 1 -- 172.21.15.119:0/3933730724 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0196b0 con 0x7f2b080b1420 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.365+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/3933730724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080b1420 msgr2=0x7f2b080ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.365+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/3933730724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080b1420 0x7f2b080ae410 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f2b080c8fd0 tx=0x7f2afc00b4f0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.366+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/3933730724 shutdown_connections 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.366+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/3933730724 >> v2:172.21.15.184:3300/0 conn(0x7f2b080ccf90 0x7f2b080c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.366+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/3933730724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080b1420 0x7f2b080ae410 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.366+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/3933730724 >> 172.21.15.119:0/3933730724 conn(0x7f2b0806a320 msgr2=0x7f2b080cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.366+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/3933730724 shutdown_connections 2023-10-08T01:15:17.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.366+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/3933730724 wait complete. 2023-10-08T01:15:17.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.367+0000 7f2b0e5ff700 1 Processor -- start 2023-10-08T01:15:17.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.367+0000 7f2b0e5ff700 1 -- start start 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b0e5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b080b1420 0x7f2b08158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b0e5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080cbf30 0x7f2b080c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b0e5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b080ccf90 0x7f2b0806e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b0e5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08159a90 con 0x7f2b080cbf30 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b0e5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08158e80 con 0x7f2b080b1420 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b0e5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b08159160 con 0x7f2b080ccf90 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b077fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080cbf30 0x7f2b080c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b077fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080cbf30 0x7f2b080c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49688/0 (socket says 172.21.15.119:49688) 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.368+0000 7f2b077fe700 1 -- 172.21.15.119:0/1867678047 learned_addr learned my addr 172.21.15.119:0/1867678047 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2b077fe700 1 -- 172.21.15.119:0/1867678047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b080ccf90 msgr2=0x7f2b0806e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2b077fe700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b080ccf90 0x7f2b0806e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2b077fe700 1 -- 172.21.15.119:0/1867678047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b080b1420 msgr2=0x7f2b08158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2b077fe700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b080b1420 0x7f2b08158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2b077fe700 1 -- 172.21.15.119:0/1867678047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc00b790 con 0x7f2b080cbf30 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2b077fe700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080cbf30 0x7f2b080c2140 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b760 tx=0x7f2afc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2aeffff700 1 -- 172.21.15.119:0/1867678047 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0107d0 con 0x7f2b080cbf30 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2aeffff700 1 -- 172.21.15.119:0/1867678047 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2afc010930 con 0x7f2b080cbf30 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b08159440 con 0x7f2b080cbf30 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2aeffff700 1 -- 172.21.15.119:0/1867678047 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc010cd0 con 0x7f2b080cbf30 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.369+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b08158b70 con 0x7f2b080cbf30 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.370+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b0804c3c0 con 0x7f2b080cbf30 2023-10-08T01:15:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.371+0000 7f2aeffff700 1 -- 172.21.15.119:0/1867678047 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2afc02b440 con 0x7f2b080cbf30 2023-10-08T01:15:17.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.372+0000 7f2aeffff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2af4066f30 0x7f2af40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.372+0000 7f2aeffff700 1 -- 172.21.15.119:0/1867678047 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5273+0+0 (secure 0 0 0) 0x7f2afc014070 con 0x7f2b080cbf30 2023-10-08T01:15:17.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.372+0000 7f2b07fff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2af4066f30 0x7f2af40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.373+0000 7f2b07fff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2af4066f30 0x7f2af40693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2af8001730 tx=0x7f2af8007b40 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:17.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.376+0000 7f2aeffff700 1 -- 172.21.15.119:0/1867678047 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2afc053610 con 0x7f2b080cbf30 2023-10-08T01:15:17.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.639+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2b080ad650 con 0x7f2b080cbf30 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.640+0000 7f2aeffff700 1 -- 172.21.15.119:0/1867678047 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2afc056c30 con 0x7f2b080cbf30 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:15:17.641 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:15:17.642 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.643 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.644 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.645 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.646 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.647 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.648 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T01:15:17.650 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:15:17.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:15:17.652 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.653 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T01:15:17.654 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T01:15:17.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T01:15:17.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.656 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:17.656 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T01:15:17.656 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:15:17.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2af4066f30 msgr2=0x7f2af40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2af4066f30 0x7f2af40693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2af8001730 tx=0x7f2af8007b40 comp rx=0 tx=0).stop 2023-10-08T01:15:17.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080cbf30 msgr2=0x7f2b080c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080cbf30 0x7f2b080c2140 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b760 tx=0x7f2afc01b000 comp rx=0 tx=0).stop 2023-10-08T01:15:17.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 shutdown_connections 2023-10-08T01:15:17.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2af4066f30 0x7f2af40693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2b080ccf90 0x7f2b0806e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2b080b1420 0x7f2b08158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 --2- 172.21.15.119:0/1867678047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b080cbf30 0x7f2b080c2140 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 >> 172.21.15.119:0/1867678047 conn(0x7f2b0806a320 msgr2=0x7f2b080c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:17.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 shutdown_connections 2023-10-08T01:15:17.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.642+0000 7f2b0e5ff700 1 -- 172.21.15.119:0/1867678047 wait complete. 2023-10-08T01:15:17.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:15:17.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.841+0000 7efbfbfff700 1 Processor -- start 2023-10-08T01:15:17.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.841+0000 7efbfbfff700 1 -- start start 2023-10-08T01:15:17.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.842+0000 7efbfbfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 0x7efbf4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.842+0000 7efbfbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efbf40ac6a0 0x7efbf40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.842+0000 7efbfbfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf40acac0 con 0x7efbf40ac230 2023-10-08T01:15:17.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.842+0000 7efbfbfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf40ad930 con 0x7efbf40ac6a0 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.842+0000 7efbfbfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf40acd20 con 0x7efbf4155c10 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbfa59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efbf40ac6a0 0x7efbf40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbfa59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efbf40ac6a0 0x7efbf40b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49698/0 (socket says 172.21.15.119:49698) 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbfa59c700 1 -- 172.21.15.119:0/3190696425 learned_addr learned my addr 172.21.15.119:0/3190696425 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbf9d9b700 1 --2- 172.21.15.119:0/3190696425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 0x7efbf4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbfa59c700 1 -- 172.21.15.119:0/3190696425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 msgr2=0x7efbf4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbfa59c700 1 --2- 172.21.15.119:0/3190696425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 0x7efbf4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbfa59c700 1 -- 172.21.15.119:0/3190696425 >> v1:172.21.15.184:6789/0 conn(0x7efbf40ac230 legacy=0x7efbf41568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbfa59c700 1 -- 172.21.15.119:0/3190696425 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf40ad000 con 0x7efbf40ac6a0 2023-10-08T01:15:17.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.843+0000 7efbfa59c700 1 --2- 172.21.15.119:0/3190696425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf40b0300 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7efbec009f20 tx=0x7efbec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f349f9cf31c886 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbf8d99700 1 -- 172.21.15.119:0/3190696425 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec00e070 con 0x7efbf40ac6a0 2023-10-08T01:15:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbf8d99700 1 -- 172.21.15.119:0/3190696425 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbec00ba80 con 0x7efbf40ac6a0 2023-10-08T01:15:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbf8d99700 1 -- 172.21.15.119:0/3190696425 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbec0092e0 con 0x7efbf40ac6a0 2023-10-08T01:15:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbfbfff700 1 -- 172.21.15.119:0/3190696425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 msgr2=0x7efbf40b0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbfbfff700 1 --2- 172.21.15.119:0/3190696425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf40b0300 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7efbec009f20 tx=0x7efbec009840 comp rx=0 tx=0).stop 2023-10-08T01:15:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbfbfff700 1 -- 172.21.15.119:0/3190696425 shutdown_connections 2023-10-08T01:15:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbfbfff700 1 --2- 172.21.15.119:0/3190696425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf40b0300 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbfbfff700 1 --2- 172.21.15.119:0/3190696425 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 0x7efbf4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbfbfff700 1 -- 172.21.15.119:0/3190696425 >> 172.21.15.119:0/3190696425 conn(0x7efbf406a320 msgr2=0x7efbf40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:17.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbfbfff700 1 -- 172.21.15.119:0/3190696425 shutdown_connections 2023-10-08T01:15:17.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.844+0000 7efbfbfff700 1 -- 172.21.15.119:0/3190696425 wait complete. 2023-10-08T01:15:17.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.846+0000 7efbfbfff700 1 Processor -- start 2023-10-08T01:15:17.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.846+0000 7efbfbfff700 1 -- start start 2023-10-08T01:15:17.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbfbfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf40ac230 0x7efbf41513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbfbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbfbfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 0x7efbf4158080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbfbfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf415ad30 con 0x7efbf40ac6a0 2023-10-08T01:15:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbfbfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf415a120 con 0x7efbf40ac230 2023-10-08T01:15:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbfbfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf415a400 con 0x7efbf4155c10 2023-10-08T01:15:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbf959a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbf959a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49710/0 (socket says 172.21.15.119:49710) 2023-10-08T01:15:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbf959a700 1 -- 172.21.15.119:0/2038881999 learned_addr learned my addr 172.21.15.119:0/2038881999 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbf9d9b700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf40ac230 0x7efbf41513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.847+0000 7efbfa59c700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 0x7efbf4158080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbf959a700 1 -- 172.21.15.119:0/2038881999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 msgr2=0x7efbf4158080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbf959a700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 0x7efbf4158080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbf959a700 1 -- 172.21.15.119:0/2038881999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf40ac230 msgr2=0x7efbf41513e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbf959a700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf40ac230 0x7efbf41513e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbf959a700 1 -- 172.21.15.119:0/2038881999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbec009ae0 con 0x7efbf40ac6a0 2023-10-08T01:15:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbf959a700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf415b1c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7efbe400c590 tx=0x7efbe400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:17.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbe9ffb700 1 -- 172.21.15.119:0/2038881999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4011590 con 0x7efbf40ac6a0 2023-10-08T01:15:17.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbe9ffb700 1 -- 172.21.15.119:0/2038881999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe4011710 con 0x7efbf40ac6a0 2023-10-08T01:15:17.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf415a6e0 con 0x7efbf40ac6a0 2023-10-08T01:15:17.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbe9ffb700 1 -- 172.21.15.119:0/2038881999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4011ad0 con 0x7efbf40ac6a0 2023-10-08T01:15:17.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.848+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf41585c0 con 0x7efbf40ac6a0 2023-10-08T01:15:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.849+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf415aee0 con 0x7efbf40ac6a0 2023-10-08T01:15:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.850+0000 7efbe9ffb700 1 -- 172.21.15.119:0/2038881999 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efbe401b3b0 con 0x7efbf40ac6a0 2023-10-08T01:15:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.851+0000 7efbe9ffb700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efbd8066f30 0x7efbd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.851+0000 7efbe9ffb700 1 -- 172.21.15.119:0/2038881999 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5273+0+0 (secure 0 0 0) 0x7efbe40837b0 con 0x7efbf40ac6a0 2023-10-08T01:15:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.851+0000 7efbf9d9b700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efbd8066f30 0x7efbd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.851+0000 7efbf9d9b700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efbd8066f30 0x7efbd80693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efbf40aea60 tx=0x7efbf0008040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:17.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:17.855+0000 7efbe9ffb700 1 -- 172.21.15.119:0/2038881999 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efbf415aee0 con 0x7efbf40ac6a0 2023-10-08T01:15:18.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.045+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efbf404c3c0 con 0x7efbf40ac6a0 2023-10-08T01:15:18.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.046+0000 7efbe9ffb700 1 -- 172.21.15.119:0/2038881999 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+12834 (secure 0 0 0) 0x7efbe4017020 con 0x7efbf40ac6a0 2023-10-08T01:15:18.047 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:18.047 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":131,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:15:17.153457+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efbd8066f30 msgr2=0x7efbd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efbd8066f30 0x7efbd80693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efbf40aea60 tx=0x7efbf0008040 comp rx=0 tx=0).stop 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 msgr2=0x7efbf415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf415b1c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7efbe400c590 tx=0x7efbe400c8a0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 shutdown_connections 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efbd8066f30 0x7efbd80693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efbf4155c10 0x7efbf4158080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efbf40ac230 0x7efbf41513e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 --2- 172.21.15.119:0/2038881999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efbf40ac6a0 0x7efbf415b1c0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.048+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 >> 172.21.15.119:0/2038881999 conn(0x7efbf406a320 msgr2=0x7efbf406e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:18.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.049+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 shutdown_connections 2023-10-08T01:15:18.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.049+0000 7efbfbfff700 1 -- 172.21.15.119:0/2038881999 wait complete. 2023-10-08T01:15:18.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T01:15:18.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T01:15:18.181 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:15:18.186 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:15:18.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.254+0000 7f1ab359e700 1 Processor -- start 2023-10-08T01:15:18.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.254+0000 7f1ab359e700 1 -- start start 2023-10-08T01:15:18.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab359e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1ab40cb1a0 0x7f1ab415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab415a8a0 0x7f1ab406f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab359e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab406ebd0 con 0x7f1ab406f5a0 2023-10-08T01:15:18.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab359e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab415b2e0 con 0x7f1ab40cb1a0 2023-10-08T01:15:18.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab415ad20 con 0x7f1ab415a8a0 2023-10-08T01:15:18.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab2d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1ab406f5a0 0x7f1ab40ae0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50936/0 (socket says 172.21.15.119:50936) 2023-10-08T01:15:18.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab2d9d700 1 -- 172.21.15.119:0/908362182 learned_addr learned my addr 172.21.15.119:0/908362182 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:18.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab259c700 1 --2- 172.21.15.119:0/908362182 >> v2:172.21.15.184:3300/0 conn(0x7f1ab40cb1a0 0x7f1ab415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab1d9b700 1 --2- 172.21.15.119:0/908362182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab415a8a0 0x7f1ab406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.256+0000 7f1ab159a700 1 -- 172.21.15.119:0/908362182 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693115729 0 0) 0x7f1ab406ebd0 con 0x7f1ab406f5a0 2023-10-08T01:15:18.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab159a700 1 -- 172.21.15.119:0/908362182 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a9c003570 con 0x7f1ab406f5a0 2023-10-08T01:15:18.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab1d9b700 1 -- 172.21.15.119:0/908362182 >> v2:172.21.15.184:3300/0 conn(0x7f1ab40cb1a0 msgr2=0x7f1ab415a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:18.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab1d9b700 1 --2- 172.21.15.119:0/908362182 >> v2:172.21.15.184:3300/0 conn(0x7f1ab40cb1a0 0x7f1ab415a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab1d9b700 1 -- 172.21.15.119:0/908362182 >> v1:172.21.15.119:6789/0 conn(0x7f1ab406f5a0 legacy=0x7f1ab40ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab1d9b700 1 -- 172.21.15.119:0/908362182 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab415b000 con 0x7f1ab415a8a0 2023-10-08T01:15:18.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab259c700 1 --2- 172.21.15.119:0/908362182 >> v2:172.21.15.184:3300/0 conn(0x7f1ab40cb1a0 0x7f1ab415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:18.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab1d9b700 1 --2- 172.21.15.119:0/908362182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab415a8a0 0x7f1ab406f060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80098e0 tx=0x7f1aa80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aca02617d9de7ff server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:18.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab159a700 1 -- 172.21.15.119:0/908362182 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8011630 con 0x7f1ab415a8a0 2023-10-08T01:15:18.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab159a700 1 -- 172.21.15.119:0/908362182 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa8011790 con 0x7f1ab415a8a0 2023-10-08T01:15:18.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.257+0000 7f1ab159a700 1 -- 172.21.15.119:0/908362182 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8011b30 con 0x7f1ab415a8a0 2023-10-08T01:15:18.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.258+0000 7f1ab359e700 1 -- 172.21.15.119:0/908362182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab415a8a0 msgr2=0x7f1ab406f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.258+0000 7f1ab359e700 1 --2- 172.21.15.119:0/908362182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab415a8a0 0x7f1ab406f060 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80098e0 tx=0x7f1aa80091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.258+0000 7f1ab359e700 1 -- 172.21.15.119:0/908362182 shutdown_connections 2023-10-08T01:15:18.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.258+0000 7f1ab359e700 1 --2- 172.21.15.119:0/908362182 >> v2:172.21.15.184:3300/0 conn(0x7f1ab40cb1a0 0x7f1ab415a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.258+0000 7f1ab359e700 1 --2- 172.21.15.119:0/908362182 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab415a8a0 0x7f1ab406f060 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.258+0000 7f1ab359e700 1 -- 172.21.15.119:0/908362182 >> 172.21.15.119:0/908362182 conn(0x7f1ab40cc1e0 msgr2=0x7f1ab40cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:18.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.258+0000 7f1ab359e700 1 -- 172.21.15.119:0/908362182 shutdown_connections 2023-10-08T01:15:18.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.258+0000 7f1ab359e700 1 -- 172.21.15.119:0/908362182 wait complete. 2023-10-08T01:15:18.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.259+0000 7f1ab359e700 1 Processor -- start 2023-10-08T01:15:18.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.259+0000 7f1ab359e700 1 -- start start 2023-10-08T01:15:18.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.260+0000 7f1ab359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ab406f5a0 0x7f1ab4169bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.260+0000 7f1ab359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab40cb1a0 0x7f1ab416c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.260+0000 7f1ab359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40c77e0 0x7f1ab40c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.260+0000 7f1ab359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab416df80 con 0x7f1ab40c77e0 2023-10-08T01:15:18.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.260+0000 7f1ab359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4151830 con 0x7f1ab406f5a0 2023-10-08T01:15:18.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.260+0000 7f1ab359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40c3290 con 0x7f1ab40cb1a0 2023-10-08T01:15:18.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40c77e0 0x7f1ab40c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40c77e0 0x7f1ab40c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49716/0 (socket says 172.21.15.119:49716) 2023-10-08T01:15:18.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 -- 172.21.15.119:0/1907469859 learned_addr learned my addr 172.21.15.119:0/1907469859 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:18.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab259c700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ab406f5a0 0x7f1ab4169bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab1d9b700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab40cb1a0 0x7f1ab416c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 -- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab40cb1a0 msgr2=0x7f1ab416c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab40cb1a0 0x7f1ab416c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 -- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ab406f5a0 msgr2=0x7f1ab4169bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ab406f5a0 0x7f1ab4169bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 -- 172.21.15.119:0/1907469859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4009420 con 0x7f1ab40c77e0 2023-10-08T01:15:18.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab1d9b700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab40cb1a0 0x7f1ab416c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:18.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.261+0000 7f1ab2d9d700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40c77e0 0x7f1ab40c7c80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f1aac001460 tx=0x7f1aac001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:18.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.262+0000 7f1aa27fc700 1 -- 172.21.15.119:0/1907469859 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00a8e0 con 0x7f1ab40c77e0 2023-10-08T01:15:18.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.262+0000 7f1aa27fc700 1 -- 172.21.15.119:0/1907469859 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac00aa40 con 0x7f1ab40c77e0 2023-10-08T01:15:18.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.262+0000 7f1aa27fc700 1 -- 172.21.15.119:0/1907469859 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00adc0 con 0x7f1ab40c77e0 2023-10-08T01:15:18.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.262+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa8009420 con 0x7f1ab40c77e0 2023-10-08T01:15:18.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.262+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab40c8350 con 0x7f1ab40c77e0 2023-10-08T01:15:18.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.263+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab406c450 con 0x7f1ab40c77e0 2023-10-08T01:15:18.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.264+0000 7f1aa27fc700 1 -- 172.21.15.119:0/1907469859 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1aac007300 con 0x7f1ab40c77e0 2023-10-08T01:15:18.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.264+0000 7f1aa27fc700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a94066f30 0x7f1a940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.265+0000 7f1aa27fc700 1 -- 172.21.15.119:0/1907469859 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5273+0+0 (secure 0 0 0) 0x7f1aac084880 con 0x7f1ab40c77e0 2023-10-08T01:15:18.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.265+0000 7f1ab259c700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a94066f30 0x7f1a940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.265+0000 7f1ab259c700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a94066f30 0x7f1a940693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009f40 tx=0x7f1aa40091a0 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:18.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.269+0000 7f1aa27fc700 1 -- 172.21.15.119:0/1907469859 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1aac051bf0 con 0x7f1ab40c77e0 2023-10-08T01:15:18.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.460+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ab406e860 con 0x7f1ab40c77e0 2023-10-08T01:15:18.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.460+0000 7f1aa27fc700 1 -- 172.21.15.119:0/1907469859 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1aac055210 con 0x7f1ab40c77e0 2023-10-08T01:15:18.462 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T01:15:18.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.462+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a94066f30 msgr2=0x7f1a940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.462+0000 7f1ab359e700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a94066f30 0x7f1a940693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1aa4009f40 tx=0x7f1aa40091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40c77e0 msgr2=0x7f1ab40c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40c77e0 0x7f1ab40c7c80 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f1aac001460 tx=0x7f1aac001a80 comp rx=0 tx=0).stop 2023-10-08T01:15:18.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 shutdown_connections 2023-10-08T01:15:18.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1a94066f30 0x7f1a940693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ab40c77e0 0x7f1ab40c7c80 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1ab406f5a0 0x7f1ab4169bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 --2- 172.21.15.119:0/1907469859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1ab40cb1a0 0x7f1ab416c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 >> 172.21.15.119:0/1907469859 conn(0x7f1ab40cc1e0 msgr2=0x7f1ab40af010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:18.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 shutdown_connections 2023-10-08T01:15:18.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.463+0000 7f1ab359e700 1 -- 172.21.15.119:0/1907469859 wait complete. 2023-10-08T01:15:18.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T01:15:18.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.660+0000 7fcdc9bff700 1 Processor -- start 2023-10-08T01:15:18.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.661+0000 7fcdc9bff700 1 -- start start 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc9bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcdc4159c60 0x7fcdc415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc9bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcdc415db10 0x7fcdc4069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc9bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc415a160 con 0x7fcdc4159890 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc9bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc415df50 con 0x7fcdc4159c60 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc9bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc40d13a0 con 0x7fcdc415db10 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc37fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcdc4159890 0x7fcdc415abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50942/0 (socket says 172.21.15.119:50942) 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc37fe700 1 -- 172.21.15.119:0/1255791900 learned_addr learned my addr 172.21.15.119:0/1255791900 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc2ffd700 1 --2- 172.21.15.119:0/1255791900 >> v2:172.21.15.119:3300/0 conn(0x7fcdc4159c60 0x7fcdc415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc3fff700 1 --2- 172.21.15.119:0/1255791900 >> v2:172.21.15.184:3300/0 conn(0x7fcdc415db10 0x7fcdc4069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc27fc700 1 -- 172.21.15.119:0/1255791900 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154139094 0 0) 0x7fcdc415a160 con 0x7fcdc4159890 2023-10-08T01:15:18.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.662+0000 7fcdc27fc700 1 -- 172.21.15.119:0/1255791900 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdac003570 con 0x7fcdc4159890 2023-10-08T01:15:18.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc2ffd700 1 -- 172.21.15.119:0/1255791900 >> v2:172.21.15.184:3300/0 conn(0x7fcdc415db10 msgr2=0x7fcdc4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:18.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc2ffd700 1 --2- 172.21.15.119:0/1255791900 >> v2:172.21.15.184:3300/0 conn(0x7fcdc415db10 0x7fcdc4069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc2ffd700 1 -- 172.21.15.119:0/1255791900 >> v1:172.21.15.119:6789/0 conn(0x7fcdc4159890 legacy=0x7fcdc415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc2ffd700 1 -- 172.21.15.119:0/1255791900 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc415e140 con 0x7fcdc4159c60 2023-10-08T01:15:18.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc2ffd700 1 --2- 172.21.15.119:0/1255791900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159c60 0x7fcdc415d5d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fcdc406a890 tx=0x7fcdb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a754f424577fb73c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:18.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc27fc700 1 -- 172.21.15.119:0/1255791900 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8011630 con 0x7fcdc4159c60 2023-10-08T01:15:18.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc27fc700 1 -- 172.21.15.119:0/1255791900 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb8011790 con 0x7fcdc4159c60 2023-10-08T01:15:18.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc27fc700 1 -- 172.21.15.119:0/1255791900 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8011b30 con 0x7fcdc4159c60 2023-10-08T01:15:18.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc9bff700 1 -- 172.21.15.119:0/1255791900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159c60 msgr2=0x7fcdc415d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.663+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/1255791900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159c60 0x7fcdc415d5d0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fcdc406a890 tx=0x7fcdb80091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.664+0000 7fcdc9bff700 1 -- 172.21.15.119:0/1255791900 shutdown_connections 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.664+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/1255791900 >> v2:172.21.15.184:3300/0 conn(0x7fcdc415db10 0x7fcdc4069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.664+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/1255791900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159c60 0x7fcdc415d5d0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.664+0000 7fcdc9bff700 1 -- 172.21.15.119:0/1255791900 >> 172.21.15.119:0/1255791900 conn(0x7fcdc4068e20 msgr2=0x7fcdc40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.664+0000 7fcdc9bff700 1 -- 172.21.15.119:0/1255791900 shutdown_connections 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.664+0000 7fcdc9bff700 1 -- 172.21.15.119:0/1255791900 wait complete. 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.665+0000 7fcdc9bff700 1 Processor -- start 2023-10-08T01:15:18.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.665+0000 7fcdc9bff700 1 -- start start 2023-10-08T01:15:18.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc9bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159890 0x7fcdc414b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc9bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc4159c60 0x7fcdc40b0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc9bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc415db10 0x7fcdc41515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc9bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b1b00 con 0x7fcdc4159890 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc9bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc414bd60 con 0x7fcdc415db10 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc9bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc40b0ef0 con 0x7fcdc4159c60 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159890 0x7fcdc414b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159890 0x7fcdc414b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49724/0 (socket says 172.21.15.119:49724) 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc37fe700 1 -- 172.21.15.119:0/3550562988 learned_addr learned my addr 172.21.15.119:0/3550562988 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc3fff700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc415db10 0x7fcdc41515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.666+0000 7fcdc2ffd700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc4159c60 0x7fcdc40b0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc37fe700 1 -- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc4159c60 msgr2=0x7fcdc40b0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc37fe700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc4159c60 0x7fcdc40b0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc37fe700 1 -- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc415db10 msgr2=0x7fcdc41515a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc37fe700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc415db10 0x7fcdc41515a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc37fe700 1 -- 172.21.15.119:0/3550562988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8009420 con 0x7fcdc4159890 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc2ffd700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc4159c60 0x7fcdc40b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc3fff700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc415db10 0x7fcdc41515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc37fe700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159890 0x7fcdc414b820 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c790 tx=0x7fcdb4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdab7fe700 1 -- 172.21.15.119:0/3550562988 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4005b30 con 0x7fcdc4159890 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdab7fe700 1 -- 172.21.15.119:0/3550562988 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb4005c90 con 0x7fcdc4159890 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdab7fe700 1 -- 172.21.15.119:0/3550562988 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb4007320 con 0x7fcdc4159890 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc40b11d0 con 0x7fcdc4159890 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.667+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc40b1790 con 0x7fcdc4159890 2023-10-08T01:15:18.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.668+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc40ce450 con 0x7fcdc4159890 2023-10-08T01:15:18.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.669+0000 7fcdab7fe700 1 -- 172.21.15.119:0/3550562988 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdb4007690 con 0x7fcdc4159890 2023-10-08T01:15:18.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.670+0000 7fcdab7fe700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcdb0066f30 0x7fcdb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:18.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.670+0000 7fcdab7fe700 1 -- 172.21.15.119:0/3550562988 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5273+0+0 (secure 0 0 0) 0x7fcdb4083f70 con 0x7fcdc4159890 2023-10-08T01:15:18.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.670+0000 7fcdc2ffd700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcdb0066f30 0x7fcdb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:18.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.671+0000 7fcdc2ffd700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcdb0066f30 0x7fcdb00693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80093f0 tx=0x7fcdb8017000 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:18.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.674+0000 7fcdab7fe700 1 -- 172.21.15.119:0/3550562988 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdb4054e00 con 0x7fcdc4159890 2023-10-08T01:15:18.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.870+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcdc404c3c0 con 0x7fcdc4159890 2023-10-08T01:15:18.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.870+0000 7fcdab7fe700 1 -- 172.21.15.119:0/3550562988 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v131) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcdb4054990 con 0x7fcdc4159890 2023-10-08T01:15:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:18.872 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:15:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:18.872 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T01:15:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T01:15:18.873 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:18.873 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:15:18.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcdb0066f30 msgr2=0x7fcdb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcdb0066f30 0x7fcdb00693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80093f0 tx=0x7fcdb8017000 comp rx=0 tx=0).stop 2023-10-08T01:15:18.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159890 msgr2=0x7fcdc414b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:18.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159890 0x7fcdc414b820 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400c790 tx=0x7fcdb4007df0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 shutdown_connections 2023-10-08T01:15:18.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcdb0066f30 0x7fcdb00693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc415db10 0x7fcdc41515a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc4159890 0x7fcdc414b820 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 --2- 172.21.15.119:0/3550562988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdc4159c60 0x7fcdc40b0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:18.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 >> 172.21.15.119:0/3550562988 conn(0x7fcdc4068e20 msgr2=0x7fcdc414d660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:18.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 shutdown_connections 2023-10-08T01:15:18.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:18.873+0000 7fcdc9bff700 1 -- 172.21.15.119:0/3550562988 wait complete. 2023-10-08T01:15:18.892 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T01:15:18.892 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T01:15:19.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.075+0000 7f8a6b3ff700 1 Processor -- start 2023-10-08T01:15:19.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.075+0000 7f8a6b3ff700 1 -- start start 2023-10-08T01:15:19.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.076+0000 7f8a6b3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8a64159ec0 0x7f8a640b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.076+0000 7f8a6b3ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a640cf350 0x7f8a640cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.076+0000 7f8a6b3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a640b5b00 con 0x7f8a640b53d0 2023-10-08T01:15:19.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.076+0000 7f8a6b3ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a640b5850 con 0x7f8a640cf350 2023-10-08T01:15:19.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.076+0000 7f8a6b3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a640b5da0 con 0x7f8a64159ec0 2023-10-08T01:15:19.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.076+0000 7f8a6999c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a640cf350 0x7f8a640cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.076+0000 7f8a6899a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8a640b53d0 0x7f8a640cc900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50956/0 (socket says 172.21.15.119:50956) 2023-10-08T01:15:19.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.076+0000 7f8a6899a700 1 -- 172.21.15.119:0/4218891006 learned_addr learned my addr 172.21.15.119:0/4218891006 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:19.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a6919b700 1 --2- 172.21.15.119:0/4218891006 >> v2:172.21.15.184:3300/0 conn(0x7f8a64159ec0 0x7f8a640b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a5bfff700 1 -- 172.21.15.119:0/4218891006 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801940937 0 0) 0x7f8a640b5b00 con 0x7f8a640b53d0 2023-10-08T01:15:19.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a6999c700 1 -- 172.21.15.119:0/4218891006 >> v2:172.21.15.184:3300/0 conn(0x7f8a64159ec0 msgr2=0x7f8a640b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:19.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a6999c700 1 --2- 172.21.15.119:0/4218891006 >> v2:172.21.15.184:3300/0 conn(0x7f8a64159ec0 0x7f8a640b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a6999c700 1 -- 172.21.15.119:0/4218891006 >> v1:172.21.15.119:6789/0 conn(0x7f8a640b53d0 legacy=0x7f8a640cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a6999c700 1 -- 172.21.15.119:0/4218891006 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a640b6080 con 0x7f8a640cf350 2023-10-08T01:15:19.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a6919b700 1 --2- 172.21.15.119:0/4218891006 >> v2:172.21.15.184:3300/0 conn(0x7f8a64159ec0 0x7f8a640b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:19.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a6999c700 1 --2- 172.21.15.119:0/4218891006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640cf350 0x7f8a640cd010 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f8a64089120 tx=0x7f8a5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=709cbc33419e7d25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:19.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a5bfff700 1 -- 172.21.15.119:0/4218891006 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00e070 con 0x7f8a640cf350 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.077+0000 7f8a5bfff700 1 -- 172.21.15.119:0/4218891006 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00bc30 con 0x7f8a640cf350 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a5bfff700 1 -- 172.21.15.119:0/4218891006 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c012430 con 0x7f8a640cf350 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/4218891006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640cf350 msgr2=0x7f8a640cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/4218891006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640cf350 0x7f8a640cd010 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f8a64089120 tx=0x7f8a5c009780 comp rx=0 tx=0).stop 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/4218891006 shutdown_connections 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/4218891006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640cf350 0x7f8a640cd010 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/4218891006 >> v2:172.21.15.184:3300/0 conn(0x7f8a64159ec0 0x7f8a640b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/4218891006 >> 172.21.15.119:0/4218891006 conn(0x7f8a640999f0 msgr2=0x7f8a640b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/4218891006 shutdown_connections 2023-10-08T01:15:19.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.078+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/4218891006 wait complete. 2023-10-08T01:15:19.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.080+0000 7f8a6b3ff700 1 Processor -- start 2023-10-08T01:15:19.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.080+0000 7f8a6b3ff700 1 -- start start 2023-10-08T01:15:19.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.080+0000 7f8a6b3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640b53d0 0x7f8a6406adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.080+0000 7f8a6b3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a640cf350 0x7f8a6416d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6b3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a64159ec0 0x7f8a640c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6b3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6416ddd0 con 0x7f8a640b53d0 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6b3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a64069400 con 0x7f8a64159ec0 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6b3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640696e0 con 0x7f8a640cf350 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640b53d0 0x7f8a6406adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640b53d0 0x7f8a6406adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49748/0 (socket says 172.21.15.119:49748) 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 -- 172.21.15.119:0/2897444700 learned_addr learned my addr 172.21.15.119:0/2897444700 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 -- 172.21.15.119:0/2897444700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a640cf350 msgr2=0x7f8a6416d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a640cf350 0x7f8a6416d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 -- 172.21.15.119:0/2897444700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a64159ec0 msgr2=0x7f8a640c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a64159ec0 0x7f8a640c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 -- 172.21.15.119:0/2897444700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c009a20 con 0x7f8a640b53d0 2023-10-08T01:15:19.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.081+0000 7f8a6919b700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640b53d0 0x7f8a6406adf0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000e920 tx=0x7f8a60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:19.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.082+0000 7f8a597fa700 1 -- 172.21.15.119:0/2897444700 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60005a00 con 0x7f8a640b53d0 2023-10-08T01:15:19.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.082+0000 7f8a597fa700 1 -- 172.21.15.119:0/2897444700 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a60005b60 con 0x7f8a640b53d0 2023-10-08T01:15:19.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.082+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a640699c0 con 0x7f8a640b53d0 2023-10-08T01:15:19.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.082+0000 7f8a597fa700 1 -- 172.21.15.119:0/2897444700 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a60006020 con 0x7f8a640b53d0 2023-10-08T01:15:19.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.082+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a640c6e70 con 0x7f8a640b53d0 2023-10-08T01:15:19.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.083+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a640c70f0 con 0x7f8a640b53d0 2023-10-08T01:15:19.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.084+0000 7f8a597fa700 1 -- 172.21.15.119:0/2897444700 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a60006410 con 0x7f8a640b53d0 2023-10-08T01:15:19.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.084+0000 7f8a597fa700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8a48066f30 0x7f8a480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.085+0000 7f8a597fa700 1 -- 172.21.15.119:0/2897444700 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5273+0+0 (secure 0 0 0) 0x7f8a60084550 con 0x7f8a640b53d0 2023-10-08T01:15:19.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.085+0000 7f8a6899a700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8a48066f30 0x7f8a480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.085+0000 7f8a6899a700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8a48066f30 0x7f8a480693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a640c9ce0 tx=0x7f8a54007bd0 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:19.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.089+0000 7f8a597fa700 1 -- 172.21.15.119:0/2897444700 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a600518d0 con 0x7f8a640b53d0 2023-10-08T01:15:19.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.282+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8a64069e60 con 0x7f8a640b53d0 2023-10-08T01:15:19.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.738+0000 7f8a597fa700 1 -- 172.21.15.119:0/2897444700 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v132) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8a60054ef0 con 0x7f8a640b53d0 2023-10-08T01:15:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8a48066f30 msgr2=0x7f8a480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8a48066f30 0x7f8a480693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a640c9ce0 tx=0x7f8a54007bd0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640b53d0 msgr2=0x7f8a6406adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640b53d0 0x7f8a6406adf0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000e920 tx=0x7f8a60008040 comp rx=0 tx=0).stop 2023-10-08T01:15:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 shutdown_connections 2023-10-08T01:15:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8a48066f30 0x7f8a480693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8a64159ec0 0x7f8a640c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a640b53d0 0x7f8a6406adf0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 --2- 172.21.15.119:0/2897444700 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8a640cf350 0x7f8a6416d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 >> 172.21.15.119:0/2897444700 conn(0x7f8a640999f0 msgr2=0x7f8a640c55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:19.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.744+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 shutdown_connections 2023-10-08T01:15:19.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.745+0000 7f8a6b3ff700 1 -- 172.21.15.119:0/2897444700 wait complete. 2023-10-08T01:15:19.748 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T01:15:19.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T01:15:19.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.952+0000 7fcc63bff700 1 Processor -- start 2023-10-08T01:15:19.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.952+0000 7fcc63bff700 1 -- start start 2023-10-08T01:15:19.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc63bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcc5c0c85c0 0x7fcc5c06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc63bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc5c156ee0 0x7fcc5c069890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc63bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 0x7fcc5c159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc63bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c156340 con 0x7fcc5c156ee0 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc63bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc5c06a1f0 con 0x7fcc5c0c85c0 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc63bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0c8990 con 0x7fcc5c069dd0 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6119a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc5c156ee0 0x7fcc5c069890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6119a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc5c156ee0 0x7fcc5c069890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49760/0 (socket says 172.21.15.119:49760) 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6119a700 1 -- 172.21.15.119:0/3544824808 learned_addr learned my addr 172.21.15.119:0/3544824808 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6199b700 1 --2- 172.21.15.119:0/3544824808 >> v2:172.21.15.184:3300/0 conn(0x7fcc5c0c85c0 0x7fcc5c06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6219c700 1 --2- 172.21.15.119:0/3544824808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 0x7fcc5c159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6119a700 1 -- 172.21.15.119:0/3544824808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 msgr2=0x7fcc5c159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:19.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6119a700 1 --2- 172.21.15.119:0/3544824808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 0x7fcc5c159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6119a700 1 -- 172.21.15.119:0/3544824808 >> v2:172.21.15.184:3300/0 conn(0x7fcc5c0c85c0 msgr2=0x7fcc5c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:19.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6119a700 1 --2- 172.21.15.119:0/3544824808 >> v2:172.21.15.184:3300/0 conn(0x7fcc5c0c85c0 0x7fcc5c06d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.954+0000 7fcc6119a700 1 -- 172.21.15.119:0/3544824808 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc5c06ee20 con 0x7fcc5c156ee0 2023-10-08T01:15:19.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.955+0000 7fcc6119a700 1 --2- 172.21.15.119:0/3544824808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c069890 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06b9d0 tx=0x7fcc4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4666ec7c04c51ea8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.955+0000 7fcc60999700 1 -- 172.21.15.119:0/3544824808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c00b640 con 0x7fcc5c156ee0 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.955+0000 7fcc60999700 1 -- 172.21.15.119:0/3544824808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc4c00b7a0 con 0x7fcc5c156ee0 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.955+0000 7fcc60999700 1 -- 172.21.15.119:0/3544824808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c00bb60 con 0x7fcc5c156ee0 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 -- 172.21.15.119:0/3544824808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 msgr2=0x7fcc5c069890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3544824808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c069890 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06b9d0 tx=0x7fcc4c009250 comp rx=0 tx=0).stop 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 -- 172.21.15.119:0/3544824808 shutdown_connections 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3544824808 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 0x7fcc5c159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3544824808 >> v2:172.21.15.184:3300/0 conn(0x7fcc5c0c85c0 0x7fcc5c06d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3544824808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c069890 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 -- 172.21.15.119:0/3544824808 >> 172.21.15.119:0/3544824808 conn(0x7fcc5c0cc540 msgr2=0x7fcc5c0cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:19.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 -- 172.21.15.119:0/3544824808 shutdown_connections 2023-10-08T01:15:19.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.956+0000 7fcc63bff700 1 -- 172.21.15.119:0/3544824808 wait complete. 2023-10-08T01:15:19.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.957+0000 7fcc63bff700 1 Processor -- start 2023-10-08T01:15:19.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.957+0000 7fcc63bff700 1 -- start start 2023-10-08T01:15:19.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc63bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 0x7fcc5c159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc63bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c0c85c0 0x7fcc5c15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc63bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c0b01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc63bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c0af620 con 0x7fcc5c156ee0 2023-10-08T01:15:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc63bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c15a000 con 0x7fcc5c0c85c0 2023-10-08T01:15:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc63bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c15a2e0 con 0x7fcc5c069dd0 2023-10-08T01:15:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc6219c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c0b01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc6219c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c0b01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49766/0 (socket says 172.21.15.119:49766) 2023-10-08T01:15:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.958+0000 7fcc6219c700 1 -- 172.21.15.119:0/3522205763 learned_addr learned my addr 172.21.15.119:0/3522205763 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc6219c700 1 -- 172.21.15.119:0/3522205763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 msgr2=0x7fcc5c159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc6219c700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 0x7fcc5c159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc6219c700 1 -- 172.21.15.119:0/3522205763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c0c85c0 msgr2=0x7fcc5c15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc6219c700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c0c85c0 0x7fcc5c15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc6219c700 1 -- 172.21.15.119:0/3522205763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0094f0 con 0x7fcc5c156ee0 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc6219c700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c0b01d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009df0 tx=0x7fcc5400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc51ffb700 1 -- 172.21.15.119:0/3522205763 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400cb50 con 0x7fcc5c156ee0 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc51ffb700 1 -- 172.21.15.119:0/3522205763 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5400f040 con 0x7fcc5c156ee0 2023-10-08T01:15:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c15a5c0 con 0x7fcc5c156ee0 2023-10-08T01:15:19.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc51ffb700 1 -- 172.21.15.119:0/3522205763 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54014480 con 0x7fcc5c156ee0 2023-10-08T01:15:19.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.959+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c15ab80 con 0x7fcc5c156ee0 2023-10-08T01:15:19.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.960+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c0b1430 con 0x7fcc5c156ee0 2023-10-08T01:15:19.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.961+0000 7fcc51ffb700 1 -- 172.21.15.119:0/3522205763 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc54014830 con 0x7fcc5c156ee0 2023-10-08T01:15:19.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.963+0000 7fcc51ffb700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:19.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.963+0000 7fcc51ffb700 1 -- 172.21.15.119:0/3522205763 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5643+0+0 (secure 0 0 0) 0x7fcc54084ec0 con 0x7fcc5c156ee0 2023-10-08T01:15:19.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.963+0000 7fcc6199b700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:19.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.964+0000 7fcc6199b700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc40066f30 0x7fcc400693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009b20 tx=0x7fcc580091a0 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:19.967+0000 7fcc51ffb700 1 -- 172.21.15.119:0/3522205763 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc540520d0 con 0x7fcc5c156ee0 2023-10-08T01:15:20.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.161+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcc5c15ad70 con 0x7fcc5c156ee0 2023-10-08T01:15:20.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.748+0000 7fcc51ffb700 1 -- 172.21.15.119:0/3522205763 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v133) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcc540556f0 con 0x7fcc5c156ee0 2023-10-08T01:15:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.754+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc40066f30 msgr2=0x7fcc400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.754+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc40066f30 0x7fcc400693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009b20 tx=0x7fcc580091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.754+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 msgr2=0x7fcc5c0b01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:20.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.754+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c0b01d0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009df0 tx=0x7fcc5400b040 comp rx=0 tx=0).stop 2023-10-08T01:15:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.755+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 shutdown_connections 2023-10-08T01:15:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.755+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcc40066f30 0x7fcc400693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.755+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc5c069dd0 0x7fcc5c159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.755+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc5c156ee0 0x7fcc5c0b01d0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.755+0000 7fcc63bff700 1 --2- 172.21.15.119:0/3522205763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc5c0c85c0 0x7fcc5c15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:20.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.755+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 >> 172.21.15.119:0/3522205763 conn(0x7fcc5c0cc540 msgr2=0x7fcc5c0ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:20.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.757+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 shutdown_connections 2023-10-08T01:15:20.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.758+0000 7fcc63bff700 1 -- 172.21.15.119:0/3522205763 wait complete. 2023-10-08T01:15:20.764 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T01:15:20.785 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T01:15:20.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.969+0000 7f094cf9d700 1 Processor -- start 2023-10-08T01:15:20.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.969+0000 7f094cf9d700 1 -- start start 2023-10-08T01:15:20.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.970+0000 7f094cf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f094806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:20.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.970+0000 7f094cf9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09480aea70 con 0x7f09480aef00 2023-10-08T01:15:20.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.970+0000 7f094cf9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094815dad0 con 0x7f094815a3b0 2023-10-08T01:15:20.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.970+0000 7f094cf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094806ee30 con 0x7f094806e470 2023-10-08T01:15:20.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f0946ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f09480aef00 0x7f094815d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50962/0 (socket says 172.21.15.119:50962) 2023-10-08T01:15:20.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f0946ffd700 1 -- 172.21.15.119:0/2763092771 learned_addr learned my addr 172.21.15.119:0/2763092771 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:20.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f0947fff700 1 --2- 172.21.15.119:0/2763092771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f094806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f09467fc700 1 -- 172.21.15.119:0/2763092771 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008694035 0 0) 0x7f09480aea70 con 0x7f09480aef00 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f09467fc700 1 -- 172.21.15.119:0/2763092771 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0930003570 con 0x7f09480aef00 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f09467fc700 1 -- 172.21.15.119:0/2763092771 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 527252473 0 0) 0x7f094815dad0 con 0x7f094815a3b0 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f09467fc700 1 -- 172.21.15.119:0/2763092771 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09480aea70 con 0x7f094815a3b0 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f0947fff700 1 -- 172.21.15.119:0/2763092771 >> v1:172.21.15.184:6789/0 conn(0x7f094815a3b0 legacy=0x7f094815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f0947fff700 1 -- 172.21.15.119:0/2763092771 >> v1:172.21.15.119:6789/0 conn(0x7f09480aef00 legacy=0x7f094815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.971+0000 7f0947fff700 1 -- 172.21.15.119:0/2763092771 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480ae480 con 0x7f094806e470 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.972+0000 7f0947fff700 1 --2- 172.21.15.119:0/2763092771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f094806e8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f094815dc60 tx=0x7f0940009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d2b014a9036733 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:20.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.972+0000 7f09467fc700 1 -- 172.21.15.119:0/2763092771 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940016070 con 0x7f094806e470 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.972+0000 7f09467fc700 1 -- 172.21.15.119:0/2763092771 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094000bab0 con 0x7f094806e470 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.972+0000 7f09467fc700 1 -- 172.21.15.119:0/2763092771 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094000e070 con 0x7f094806e470 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.972+0000 7f094cf9d700 1 -- 172.21.15.119:0/2763092771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 msgr2=0x7f094806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.972+0000 7f094cf9d700 1 --2- 172.21.15.119:0/2763092771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f094806e8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f094815dc60 tx=0x7f0940009780 comp rx=0 tx=0).stop 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.973+0000 7f094cf9d700 1 -- 172.21.15.119:0/2763092771 shutdown_connections 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.973+0000 7f094cf9d700 1 --2- 172.21.15.119:0/2763092771 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f094806e8f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.973+0000 7f094cf9d700 1 -- 172.21.15.119:0/2763092771 >> 172.21.15.119:0/2763092771 conn(0x7f09480c9f70 msgr2=0x7f094806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.973+0000 7f094cf9d700 1 -- 172.21.15.119:0/2763092771 shutdown_connections 2023-10-08T01:15:20.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.973+0000 7f094cf9d700 1 -- 172.21.15.119:0/2763092771 wait complete. 2023-10-08T01:15:20.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.974+0000 7f094cf9d700 1 Processor -- start 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.974+0000 7f094cf9d700 1 -- start start 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f094cf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f09480af870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f094cf9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09480aef00 0x7f0948152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f094cf9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f094815a3b0 0x7f09480c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f094cf9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0948159bf0 con 0x7f094815a3b0 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f094cf9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09480b0b80 con 0x7f09480aef00 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f094cf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480aff70 con 0x7f094806e470 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f0947fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f094815a3b0 0x7f09480c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f0947fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f094815a3b0 0x7f09480c2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49772/0 (socket says 172.21.15.119:49772) 2023-10-08T01:15:20.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f0947fff700 1 -- 172.21.15.119:0/326723174 learned_addr learned my addr 172.21.15.119:0/326723174 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f09477fe700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f09480af870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.975+0000 7f0946ffd700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09480aef00 0x7f0948152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f0947fff700 1 -- 172.21.15.119:0/326723174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 msgr2=0x7f09480af870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f0947fff700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f09480af870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f0947fff700 1 -- 172.21.15.119:0/326723174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09480aef00 msgr2=0x7f0948152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f0947fff700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09480aef00 0x7f0948152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f0947fff700 1 -- 172.21.15.119:0/326723174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940009a20 con 0x7f094815a3b0 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f0947fff700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f094815a3b0 0x7f09480c2800 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f09480714a0 tx=0x7f0940018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f092f7fe700 1 -- 172.21.15.119:0/326723174 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940027470 con 0x7f094815a3b0 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f092f7fe700 1 -- 172.21.15.119:0/326723174 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09400275d0 con 0x7f094815a3b0 2023-10-08T01:15:20.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09480b0250 con 0x7f094815a3b0 2023-10-08T01:15:20.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f092f7fe700 1 -- 172.21.15.119:0/326723174 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940027990 con 0x7f094815a3b0 2023-10-08T01:15:20.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.976+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09480b0810 con 0x7f094815a3b0 2023-10-08T01:15:20.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.977+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094806cfa0 con 0x7f094815a3b0 2023-10-08T01:15:20.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.979+0000 7f092f7fe700 1 -- 172.21.15.119:0/326723174 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0940024070 con 0x7f094815a3b0 2023-10-08T01:15:20.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.980+0000 7f092f7fe700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0934066f30 0x7f09340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:20.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.980+0000 7f092f7fe700 1 -- 172.21.15.119:0/326723174 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f09400870f0 con 0x7f094815a3b0 2023-10-08T01:15:20.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.980+0000 7f09477fe700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0934066f30 0x7f09340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:20.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.981+0000 7f09477fe700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0934066f30 0x7f09340693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0938007fd0 tx=0x7f0938007d50 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:20.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:20.984+0000 7f092f7fe700 1 -- 172.21.15.119:0/326723174 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f09400542a0 con 0x7f094815a3b0 2023-10-08T01:15:21.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.215+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f09480b0cc0 con 0x7f094815a3b0 2023-10-08T01:15:21.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.774+0000 7f092f7fe700 1 -- 172.21.15.119:0/326723174 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v59) v1 ==== 174+0+0 (secure 0 0 0) 0x7f09480b0cc0 con 0x7f094815a3b0 2023-10-08T01:15:21.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.784+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0934066f30 msgr2=0x7f09340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:21.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.784+0000 7f094cf9d700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0934066f30 0x7f09340693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0938007fd0 tx=0x7f0938007d50 comp rx=0 tx=0).stop 2023-10-08T01:15:21.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.784+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f094815a3b0 msgr2=0x7f09480c2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:21.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.784+0000 7f094cf9d700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f094815a3b0 0x7f09480c2800 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f09480714a0 tx=0x7f0940018000 comp rx=0 tx=0).stop 2023-10-08T01:15:21.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.787+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 shutdown_connections 2023-10-08T01:15:21.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.787+0000 7f094cf9d700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0934066f30 0x7f09340693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:21.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.787+0000 7f094cf9d700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f094806e470 0x7f09480af870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:21.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.787+0000 7f094cf9d700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f094815a3b0 0x7f09480c2800 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:21.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.787+0000 7f094cf9d700 1 --2- 172.21.15.119:0/326723174 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09480aef00 0x7f0948152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:21.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.788+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 >> 172.21.15.119:0/326723174 conn(0x7f09480c9f70 msgr2=0x7f094815aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:21.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.792+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 shutdown_connections 2023-10-08T01:15:21.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.792+0000 7f094cf9d700 1 -- 172.21.15.119:0/326723174 wait complete. 2023-10-08T01:15:21.798 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 16 and data pool 17 2023-10-08T01:15:21.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:15:21.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.997+0000 7f4a1b5ff700 1 Processor -- start 2023-10-08T01:15:21.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.997+0000 7f4a1b5ff700 1 -- start start 2023-10-08T01:15:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.998+0000 7f4a1b5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4a140b0a10 0x7f4a14155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:21.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.998+0000 7f4a1b5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a140ad930 con 0x7f4a141578b0 2023-10-08T01:15:22.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.998+0000 7f4a1b5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a140acd20 con 0x7f4a14155c10 2023-10-08T01:15:22.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.998+0000 7f4a1b5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a140ad000 con 0x7f4a140b0a10 2023-10-08T01:15:22.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a18b9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4a141578b0 0x7f4a140af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50978/0 (socket says 172.21.15.119:50978) 2023-10-08T01:15:22.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a18b9a700 1 -- 172.21.15.119:0/2782811868 learned_addr learned my addr 172.21.15.119:0/2782811868 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a19b9c700 1 --2- 172.21.15.119:0/2782811868 >> v2:172.21.15.184:3300/0 conn(0x7f4a140b0a10 0x7f4a14155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365946029 0 0) 0x7f4a140ad930 con 0x7f4a141578b0 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a00003570 con 0x7f4a141578b0 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502732434 0 0) 0x7f4a140acd20 con 0x7f4a14155c10 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a140ad930 con 0x7f4a14155c10 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1625865409 0 0) 0x7f4a00003570 con 0x7f4a141578b0 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 >> v2:172.21.15.184:3300/0 conn(0x7f4a140b0a10 msgr2=0x7f4a14155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 --2- 172.21.15.119:0/2782811868 >> v2:172.21.15.184:3300/0 conn(0x7f4a140b0a10 0x7f4a14155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 >> v1:172.21.15.184:6789/0 conn(0x7f4a14155c10 legacy=0x7f4a140ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140ad2e0 con 0x7f4a141578b0 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f4a04001470 con 0x7f4a141578b0 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:21.999+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 >> v1:172.21.15.119:6789/0 conn(0x7f4a141578b0 legacy=0x7f4a140af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a0bfff700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a00003bf0 0x7f4a00003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a0bfff700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a00004810 0x7f4a00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a0bfff700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a000051c0 0x7f4a0000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140ad000 con 0x7f4a00004810 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0000e2c0 con 0x7f4a00003bf0 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0000e480 con 0x7f4a000051c0 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a18b9a700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a00004810 0x7f4a00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a1939b700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a00003bf0 0x7f4a00003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a19b9c700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a000051c0 0x7f4a0000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a18b9a700 1 -- 172.21.15.119:0/2782811868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a000051c0 msgr2=0x7f4a0000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a18b9a700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a000051c0 0x7f4a0000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a18b9a700 1 -- 172.21.15.119:0/2782811868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a00003bf0 msgr2=0x7f4a00003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a18b9a700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a00003bf0 0x7f4a00003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a18b9a700 1 -- 172.21.15.119:0/2782811868 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00003570 con 0x7f4a00004810 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a19b9c700 1 -- 172.21.15.119:0/2782811868 reap_dead start 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.000+0000 7f4a18b9a700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a00004810 0x7f4a00004c80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c2d0 tx=0x7f4a04001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ad602894533b76 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a0400cbc0 con 0x7f4a00004810 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a0400cd20 con 0x7f4a00004810 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a0bfff700 1 -- 172.21.15.119:0/2782811868 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04005b70 con 0x7f4a00004810 2023-10-08T01:15:22.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/2782811868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a00004810 msgr2=0x7f4a00004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a1b5ff700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a00004810 0x7f4a00004c80 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400c2d0 tx=0x7f4a04001e10 comp rx=0 tx=0).stop 2023-10-08T01:15:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/2782811868 shutdown_connections 2023-10-08T01:15:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a1b5ff700 1 --2- 172.21.15.119:0/2782811868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a00004810 0x7f4a00004c80 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/2782811868 >> 172.21.15.119:0/2782811868 conn(0x7f4a1406a320 msgr2=0x7f4a140cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/2782811868 shutdown_connections 2023-10-08T01:15:22.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.001+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/2782811868 wait complete. 2023-10-08T01:15:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.003+0000 7f4a1b5ff700 1 Processor -- start 2023-10-08T01:15:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.003+0000 7f4a1b5ff700 1 -- start start 2023-10-08T01:15:22.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a1b5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a00004810 0x7f4a140c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a1b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a140ca130 0x7f4a14154000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a1b5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a140cbbc0 0x7f4a14154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a1b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140cb730 con 0x7f4a140ca130 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a1b5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140cab20 con 0x7f4a140cbbc0 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a1b5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a140cae00 con 0x7f4a00004810 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a140ca130 0x7f4a14154000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a140ca130 0x7f4a14154000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49798/0 (socket says 172.21.15.119:49798) 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 -- 172.21.15.119:0/1325194189 learned_addr learned my addr 172.21.15.119:0/1325194189 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a19b9c700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a140cbbc0 0x7f4a14154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a1939b700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a00004810 0x7f4a140c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 -- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a00004810 msgr2=0x7f4a140c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a00004810 0x7f4a140c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 -- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a140cbbc0 msgr2=0x7f4a14154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a140cbbc0 0x7f4a14154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 -- 172.21.15.119:0/1325194189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140cb0e0 con 0x7f4a140ca130 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a19b9c700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a140cbbc0 0x7f4a14154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a1939b700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a00004810 0x7f4a140c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.004+0000 7f4a18b9a700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a140ca130 0x7f4a14154000 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000e290 tx=0x7f4a04015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.005+0000 7f4a097fa700 1 -- 172.21.15.119:0/1325194189 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04022070 con 0x7f4a140ca130 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.005+0000 7f4a097fa700 1 -- 172.21.15.119:0/1325194189 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a04000fe0 con 0x7f4a140ca130 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.005+0000 7f4a097fa700 1 -- 172.21.15.119:0/1325194189 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a04007510 con 0x7f4a140ca130 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.005+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a140cb3c0 con 0x7f4a140ca130 2023-10-08T01:15:22.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.005+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1414f6e0 con 0x7f4a140ca130 2023-10-08T01:15:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.006+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a140cb8a0 con 0x7f4a140ca130 2023-10-08T01:15:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.007+0000 7f4a097fa700 1 -- 172.21.15.119:0/1325194189 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a0401e430 con 0x7f4a140ca130 2023-10-08T01:15:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.008+0000 7f4a097fa700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f49f8066f30 0x7f49f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.008+0000 7f4a097fa700 1 -- 172.21.15.119:0/1325194189 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f4a04085940 con 0x7f4a140ca130 2023-10-08T01:15:22.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.008+0000 7f4a1939b700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f49f8066f30 0x7f49f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.008+0000 7f4a1939b700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f49f8066f30 0x7f49f80693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a10000c00 tx=0x7f4a10009040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.012+0000 7f4a097fa700 1 -- 172.21.15.119:0/1325194189 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a04052a20 con 0x7f4a140ca130 2023-10-08T01:15:22.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.203+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a1404c3c0 con 0x7f4a140ca130 2023-10-08T01:15:22.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.204+0000 7f4a097fa700 1 -- 172.21.15.119:0/1325194189 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f4a04017020 con 0x7f4a140ca130 2023-10-08T01:15:22.205 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:22.205 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":134,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:15:21.217806+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:15:19.321580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T01:15:20.200200+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:15:22.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f49f8066f30 msgr2=0x7f49f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f49f8066f30 0x7f49f80693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a10000c00 tx=0x7f4a10009040 comp rx=0 tx=0).stop 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a140ca130 msgr2=0x7f4a14154000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a140ca130 0x7f4a14154000 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4a0000e290 tx=0x7f4a04015000 comp rx=0 tx=0).stop 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 shutdown_connections 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f49f8066f30 0x7f49f80693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a00004810 0x7f4a140c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a140cbbc0 0x7f4a14154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 --2- 172.21.15.119:0/1325194189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a140ca130 0x7f4a14154000 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 >> 172.21.15.119:0/1325194189 conn(0x7f4a1406a320 msgr2=0x7f4a1406e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.206+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 shutdown_connections 2023-10-08T01:15:22.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.207+0000 7f4a1b5ff700 1 -- 172.21.15.119:0/1325194189 wait complete. 2023-10-08T01:15:22.226 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:15:22.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.406+0000 7f17915ff700 1 Processor -- start 2023-10-08T01:15:22.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.406+0000 7f17915ff700 1 -- start start 2023-10-08T01:15:22.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.408+0000 7f17915ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c1578b0 0x7f178c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.408+0000 7f17915ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c0ad930 con 0x7f178c0af2f0 2023-10-08T01:15:22.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.408+0000 7f17915ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f178c0acd20 con 0x7f178c155c10 2023-10-08T01:15:22.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.408+0000 7f17915ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0ad000 con 0x7f178c1578b0 2023-10-08T01:15:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.408+0000 7f178b7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f178c0af2f0 0x7f178c0af710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50986/0 (socket says 172.21.15.119:50986) 2023-10-08T01:15:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.408+0000 7f178b7fe700 1 -- 172.21.15.119:0/2194745321 learned_addr learned my addr 172.21.15.119:0/2194745321 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.408+0000 7f1783fff700 1 --2- 172.21.15.119:0/2194745321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c1578b0 0x7f178c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.408+0000 7f178a7fc700 1 -- 172.21.15.119:0/2194745321 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162077084 0 0) 0x7f178c0ad930 con 0x7f178c0af2f0 2023-10-08T01:15:22.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f178a7fc700 1 -- 172.21.15.119:0/2194745321 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1778004680 con 0x7f178c0af2f0 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f178a7fc700 1 -- 172.21.15.119:0/2194745321 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842666821 0 0) 0x7f178c0acd20 con 0x7f178c155c10 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f1783fff700 1 -- 172.21.15.119:0/2194745321 >> v1:172.21.15.184:6789/0 conn(0x7f178c155c10 legacy=0x7f178c0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f1783fff700 1 -- 172.21.15.119:0/2194745321 >> v1:172.21.15.119:6789/0 conn(0x7f178c0af2f0 legacy=0x7f178c0af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f1783fff700 1 -- 172.21.15.119:0/2194745321 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c0ad2e0 con 0x7f178c1578b0 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f1783fff700 1 --2- 172.21.15.119:0/2194745321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c1578b0 0x7f178c155fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1774009910 tx=0x7f17740091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72676db77f0c5b23 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f178a7fc700 1 -- 172.21.15.119:0/2194745321 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177400b680 con 0x7f178c1578b0 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f178a7fc700 1 -- 172.21.15.119:0/2194745321 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177400b7e0 con 0x7f178c1578b0 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.409+0000 7f178a7fc700 1 -- 172.21.15.119:0/2194745321 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177400bba0 con 0x7f178c1578b0 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.410+0000 7f17915ff700 1 -- 172.21.15.119:0/2194745321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c1578b0 msgr2=0x7f178c155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.410+0000 7f17915ff700 1 --2- 172.21.15.119:0/2194745321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c1578b0 0x7f178c155fe0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1774009910 tx=0x7f17740091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.410+0000 7f17915ff700 1 -- 172.21.15.119:0/2194745321 shutdown_connections 2023-10-08T01:15:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.410+0000 7f17915ff700 1 --2- 172.21.15.119:0/2194745321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c1578b0 0x7f178c155fe0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.410+0000 7f17915ff700 1 -- 172.21.15.119:0/2194745321 >> 172.21.15.119:0/2194745321 conn(0x7f178c06a320 msgr2=0x7f178c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.410+0000 7f17915ff700 1 -- 172.21.15.119:0/2194745321 shutdown_connections 2023-10-08T01:15:22.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.410+0000 7f17915ff700 1 -- 172.21.15.119:0/2194745321 wait complete. 2023-10-08T01:15:22.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.411+0000 7f17915ff700 1 Processor -- start 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.412+0000 7f17915ff700 1 -- start start 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.412+0000 7f17915ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f178c0af2f0 0x7f178c15ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.412+0000 7f17915ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c155c10 0x7f178c0a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.412+0000 7f17915ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f178c1578b0 0x7f178c145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.412+0000 7f17915ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c9960 con 0x7f178c1578b0 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.412+0000 7f17915ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c9c40 con 0x7f178c0af2f0 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f17915ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c0c9f20 con 0x7f178c155c10 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f178c1578b0 0x7f178c145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f178c1578b0 0x7f178c145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49814/0 (socket says 172.21.15.119:49814) 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 -- 172.21.15.119:0/4029999365 learned_addr learned my addr 172.21.15.119:0/4029999365 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f1783fff700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c155c10 0x7f178c0a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178affd700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f178c0af2f0 0x7f178c15ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 -- 172.21.15.119:0/4029999365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c155c10 msgr2=0x7f178c0a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c155c10 0x7f178c0a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 -- 172.21.15.119:0/4029999365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f178c0af2f0 msgr2=0x7f178c15ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f178c0af2f0 0x7f178c15ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 -- 172.21.15.119:0/4029999365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1774009420 con 0x7f178c1578b0 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.413+0000 7f178b7fe700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f178c1578b0 0x7f178c145b70 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f1784001460 tx=0x7f1784001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.414+0000 7f1782ffd700 1 -- 172.21.15.119:0/4029999365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f178400ce80 con 0x7f178c1578b0 2023-10-08T01:15:22.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.414+0000 7f1782ffd700 1 -- 172.21.15.119:0/4029999365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f178400ab70 con 0x7f178c1578b0 2023-10-08T01:15:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.414+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f178c0ca200 con 0x7f178c1578b0 2023-10-08T01:15:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.414+0000 7f1782ffd700 1 -- 172.21.15.119:0/4029999365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17840070f0 con 0x7f178c1578b0 2023-10-08T01:15:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.414+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f178c0ca7c0 con 0x7f178c1578b0 2023-10-08T01:15:22.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.415+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178c04c3c0 con 0x7f178c1578b0 2023-10-08T01:15:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.416+0000 7f1782ffd700 1 -- 172.21.15.119:0/4029999365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1784007520 con 0x7f178c1578b0 2023-10-08T01:15:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.416+0000 7f1782ffd700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f176c066f30 0x7f176c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.416+0000 7f1782ffd700 1 -- 172.21.15.119:0/4029999365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f1784084bf0 con 0x7f178c1578b0 2023-10-08T01:15:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.417+0000 7f178affd700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f176c066f30 0x7f176c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.417+0000 7f178affd700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f176c066f30 0x7f176c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f178c15d0e0 tx=0x7f177c009040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.421+0000 7f1782ffd700 1 -- 172.21.15.119:0/4029999365 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1784051cb0 con 0x7f178c1578b0 2023-10-08T01:15:22.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.610+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f178c0ad5c0 con 0x7f178c1578b0 2023-10-08T01:15:22.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.611+0000 7f1782ffd700 1 -- 172.21.15.119:0/4029999365 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f17840552d0 con 0x7f178c1578b0 2023-10-08T01:15:22.613 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:22.613 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":134,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:15:21.217806+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:15:19.321580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T01:15:20.200200+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:15:22.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f176c066f30 msgr2=0x7f176c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f176c066f30 0x7f176c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f178c15d0e0 tx=0x7f177c009040 comp rx=0 tx=0).stop 2023-10-08T01:15:22.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f178c1578b0 msgr2=0x7f178c145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f178c1578b0 0x7f178c145b70 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f1784001460 tx=0x7f1784001ae0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 shutdown_connections 2023-10-08T01:15:22.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f176c066f30 0x7f176c0693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f178c1578b0 0x7f178c145b70 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f178c0af2f0 0x7f178c15ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 --2- 172.21.15.119:0/4029999365 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f178c155c10 0x7f178c0a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 >> 172.21.15.119:0/4029999365 conn(0x7f178c06a320 msgr2=0x7f178c0b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 shutdown_connections 2023-10-08T01:15:22.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.614+0000 7f17915ff700 1 -- 172.21.15.119:0/4029999365 wait complete. 2023-10-08T01:15:22.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T01:15:22.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.817+0000 7fee0f5ff700 1 Processor -- start 2023-10-08T01:15:22.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.817+0000 7fee0f5ff700 1 -- start start 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.818+0000 7fee0f5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee08159890 0x7fee0815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.818+0000 7fee0f5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08069560 0x7fee080699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0f5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee08069ee0 con 0x7fee08159c60 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0f5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0815a160 con 0x7fee08159890 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0f5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee080d13a0 con 0x7fee08069560 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0d39b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fee08159890 0x7fee0815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0cb9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fee08159c60 0x7fee0815d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50996/0 (socket says 172.21.15.119:50996) 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0cb9a700 1 -- 172.21.15.119:0/2772231403 learned_addr learned my addr 172.21.15.119:0/2772231403 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0db9c700 1 --2- 172.21.15.119:0/2772231403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08069560 0x7fee080699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0d39b700 1 -- 172.21.15.119:0/2772231403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08069560 msgr2=0x7fee080699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:22.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0d39b700 1 --2- 172.21.15.119:0/2772231403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08069560 0x7fee080699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0d39b700 1 -- 172.21.15.119:0/2772231403 >> v1:172.21.15.119:6789/0 conn(0x7fee08159c60 legacy=0x7fee0815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fee0d39b700 1 -- 172.21.15.119:0/2772231403 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0815e140 con 0x7fee08159890 2023-10-08T01:15:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.819+0000 7fedfffff700 1 -- 172.21.15.119:0/2772231403 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152579395 0 0) 0x7fee08069ee0 con 0x7fee08159c60 2023-10-08T01:15:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fee0d39b700 1 --2- 172.21.15.119:0/2772231403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08159890 0x7fee0815abd0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fee040098e0 tx=0x7fee040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bb5161eacd679f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fedfffff700 1 -- 172.21.15.119:0/2772231403 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0400b640 con 0x7fee08159890 2023-10-08T01:15:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fedfffff700 1 -- 172.21.15.119:0/2772231403 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0400b7a0 con 0x7fee08159890 2023-10-08T01:15:22.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fedfffff700 1 -- 172.21.15.119:0/2772231403 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee0400bb40 con 0x7fee08159890 2023-10-08T01:15:22.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fee0f5ff700 1 -- 172.21.15.119:0/2772231403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08159890 msgr2=0x7fee0815abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/2772231403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08159890 0x7fee0815abd0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fee040098e0 tx=0x7fee040091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fee0f5ff700 1 -- 172.21.15.119:0/2772231403 shutdown_connections 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/2772231403 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08069560 0x7fee080699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/2772231403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08159890 0x7fee0815abd0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.820+0000 7fee0f5ff700 1 -- 172.21.15.119:0/2772231403 >> 172.21.15.119:0/2772231403 conn(0x7fee08068e20 msgr2=0x7fee080d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.821+0000 7fee0f5ff700 1 -- 172.21.15.119:0/2772231403 shutdown_connections 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.821+0000 7fee0f5ff700 1 -- 172.21.15.119:0/2772231403 wait complete. 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.822+0000 7fee0f5ff700 1 Processor -- start 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.822+0000 7fee0f5ff700 1 -- start start 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0f5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08069560 0x7fee08169db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0f5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08159890 0x7fee0816af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0f5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee08159c60 0x7fee080b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0f5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee080b4850 con 0x7fee08069560 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0f5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0816a600 con 0x7fee08159c60 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0f5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0816a8e0 con 0x7fee08159890 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0d39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08069560 0x7fee08169db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0d39b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08069560 0x7fee08169db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49830/0 (socket says 172.21.15.119:49830) 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0d39b700 1 -- 172.21.15.119:0/4191444332 learned_addr learned my addr 172.21.15.119:0/4191444332 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0cb9a700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08159890 0x7fee0816af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.823+0000 7fee0db9c700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee08159c60 0x7fee080b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0d39b700 1 -- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08159890 msgr2=0x7fee0816af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0d39b700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08159890 0x7fee0816af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0d39b700 1 -- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee08159c60 msgr2=0x7fee080b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:22.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0d39b700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee08159c60 0x7fee080b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:22.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0d39b700 1 -- 172.21.15.119:0/4191444332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee04009420 con 0x7fee08069560 2023-10-08T01:15:22.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0db9c700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee08159c60 0x7fee080b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:22.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0d39b700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08069560 0x7fee08169db0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fee080899c0 tx=0x7fee04017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fedfd7fa700 1 -- 172.21.15.119:0/4191444332 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04000d10 con 0x7fee08069560 2023-10-08T01:15:22.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fedfd7fa700 1 -- 172.21.15.119:0/4191444332 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee0400bb20 con 0x7fee08069560 2023-10-08T01:15:22.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fedfd7fa700 1 -- 172.21.15.119:0/4191444332 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee04011670 con 0x7fee08069560 2023-10-08T01:15:22.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee0816abc0 con 0x7fee08069560 2023-10-08T01:15:22.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.824+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0816a2f0 con 0x7fee08069560 2023-10-08T01:15:22.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.825+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0804c3c0 con 0x7fee08069560 2023-10-08T01:15:22.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.826+0000 7fedfd7fa700 1 -- 172.21.15.119:0/4191444332 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fee04011a20 con 0x7fee08069560 2023-10-08T01:15:22.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.827+0000 7fedfd7fa700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fedec066f30 0x7fedec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:22.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.827+0000 7fedfd7fa700 1 -- 172.21.15.119:0/4191444332 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fee04085cb0 con 0x7fee08069560 2023-10-08T01:15:22.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.827+0000 7fee0cb9a700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fedec066f30 0x7fedec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:22.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.828+0000 7fee0cb9a700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fedec066f30 0x7fedec0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee080ce7c0 tx=0x7fedf8007d30 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:22.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:22.831+0000 7fedfd7fa700 1 -- 172.21.15.119:0/4191444332 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fee04052d80 con 0x7fee08069560 2023-10-08T01:15:23.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.065+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fee0806a5b0 con 0x7fee08069560 2023-10-08T01:15:23.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.786+0000 7fedfd7fa700 1 -- 172.21.15.119:0/4191444332 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) v1 ==== 118+0+0 (secure 0 0 0) 0x7fee040563a0 con 0x7fee08069560 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fedec066f30 msgr2=0x7fedec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fedec066f30 0x7fedec0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fee080ce7c0 tx=0x7fedf8007d30 comp rx=0 tx=0).stop 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08069560 msgr2=0x7fee08169db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08069560 0x7fee08169db0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fee080899c0 tx=0x7fee04017000 comp rx=0 tx=0).stop 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 shutdown_connections 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fedec066f30 0x7fedec0693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fee08159c60 0x7fee080b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee08069560 0x7fee08169db0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 --2- 172.21.15.119:0/4191444332 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fee08159890 0x7fee0816af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:23.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.788+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 >> 172.21.15.119:0/4191444332 conn(0x7fee08068e20 msgr2=0x7fee0815d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:23.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.789+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 shutdown_connections 2023-10-08T01:15:23.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.789+0000 7fee0f5ff700 1 -- 172.21.15.119:0/4191444332 wait complete. 2023-10-08T01:15:23.808 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T01:15:23.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.992+0000 7f4ceb7ff700 1 Processor -- start 2023-10-08T01:15:23.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.992+0000 7f4ceb7ff700 1 -- start start 2023-10-08T01:15:23.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ceb7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4ce40c83f0 0x7f4ce40c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:23.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ceb7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4ce41568e0 0x7f4ce40ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:23.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ceb7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ce415b0d0 con 0x7f4ce40ccee0 2023-10-08T01:15:23.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ceb7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce40c7a60 con 0x7f4ce40c83f0 2023-10-08T01:15:23.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ceb7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ce415a4c0 con 0x7f4ce41568e0 2023-10-08T01:15:23.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ce959b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4ce40c83f0 0x7f4ce40c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:23.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ce8d9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4ce40ccee0 0x7f4ce41561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51008/0 (socket says 172.21.15.119:51008) 2023-10-08T01:15:23.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ce8d9a700 1 -- 172.21.15.119:0/1241242378 learned_addr learned my addr 172.21.15.119:0/1241242378 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ce9d9c700 1 --2- 172.21.15.119:0/1241242378 >> v2:172.21.15.184:3300/0 conn(0x7f4ce41568e0 0x7f4ce40ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ce959b700 1 -- 172.21.15.119:0/1241242378 >> v2:172.21.15.184:3300/0 conn(0x7f4ce41568e0 msgr2=0x7f4ce40ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ce959b700 1 --2- 172.21.15.119:0/1241242378 >> v2:172.21.15.184:3300/0 conn(0x7f4ce41568e0 0x7f4ce40ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ce959b700 1 -- 172.21.15.119:0/1241242378 >> v1:172.21.15.119:6789/0 conn(0x7f4ce40ccee0 legacy=0x7f4ce41561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4ce959b700 1 -- 172.21.15.119:0/1241242378 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce415a7a0 con 0x7f4ce40c83f0 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.994+0000 7f4cdbfff700 1 -- 172.21.15.119:0/1241242378 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2638782263 0 0) 0x7f4ce415b0d0 con 0x7f4ce40ccee0 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4ce959b700 1 --2- 172.21.15.119:0/1241242378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c7520 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4ce406d240 tx=0x7f4ce00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b368565a406d396 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4cdbfff700 1 -- 172.21.15.119:0/1241242378 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000b6b0 con 0x7f4ce40c83f0 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4cdbfff700 1 -- 172.21.15.119:0/1241242378 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce000b810 con 0x7f4ce40c83f0 2023-10-08T01:15:23.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4cdbfff700 1 -- 172.21.15.119:0/1241242378 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce000bbb0 con 0x7f4ce40c83f0 2023-10-08T01:15:24.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/1241242378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 msgr2=0x7f4ce40c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/1241242378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c7520 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4ce406d240 tx=0x7f4ce00091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/1241242378 shutdown_connections 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/1241242378 >> v2:172.21.15.184:3300/0 conn(0x7f4ce41568e0 0x7f4ce40ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/1241242378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c7520 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.995+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/1241242378 >> 172.21.15.119:0/1241242378 conn(0x7f4ce4068e10 msgr2=0x7f4ce40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.996+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/1241242378 shutdown_connections 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.996+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/1241242378 wait complete. 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.997+0000 7f4ceb7ff700 1 Processor -- start 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.997+0000 7f4ceb7ff700 1 -- start start 2023-10-08T01:15:24.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ceb7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ceb7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce40ccee0 0x7f4ce416da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ceb7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce41568e0 0x7f4ce40c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ceb7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40c6650 con 0x7f4ce40c83f0 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ceb7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40c5350 con 0x7f4ce41568e0 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ceb7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce40c5630 con 0x7f4ce40ccee0 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ce959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ce959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49850/0 (socket says 172.21.15.119:49850) 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.998+0000 7f4ce959b700 1 -- 172.21.15.119:0/3568220110 learned_addr learned my addr 172.21.15.119:0/3568220110 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ce9d9c700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce41568e0 0x7f4ce40c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ce8d9a700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce40ccee0 0x7f4ce416da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ce959b700 1 -- 172.21.15.119:0/3568220110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce40ccee0 msgr2=0x7f4ce416da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ce959b700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce40ccee0 0x7f4ce416da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ce959b700 1 -- 172.21.15.119:0/3568220110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce41568e0 msgr2=0x7f4ce40c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ce959b700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce41568e0 0x7f4ce40c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ce959b700 1 -- 172.21.15.119:0/3568220110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce0009420 con 0x7f4ce40c83f0 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ce959b700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c35e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4063d40 tx=0x7f4ce0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4cd97fa700 1 -- 172.21.15.119:0/3568220110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011420 con 0x7f4ce40c83f0 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4cd97fa700 1 -- 172.21.15.119:0/3568220110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce0011580 con 0x7f4ce40c83f0 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4cd97fa700 1 -- 172.21.15.119:0/3568220110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0011940 con 0x7f4ce40c83f0 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce40c5910 con 0x7f4ce40c83f0 2023-10-08T01:15:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:23.999+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce40c4fd0 con 0x7f4ce40c83f0 2023-10-08T01:15:24.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:24.000+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce404c3c0 con 0x7f4ce40c83f0 2023-10-08T01:15:24.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:24.001+0000 7f4cd97fa700 1 -- 172.21.15.119:0/3568220110 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ce0011d30 con 0x7f4ce40c83f0 2023-10-08T01:15:24.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:24.002+0000 7f4cd97fa700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4cc8066f30 0x7f4cc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:24.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:24.002+0000 7f4cd97fa700 1 -- 172.21.15.119:0/3568220110 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f4ce0085cc0 con 0x7f4ce40c83f0 2023-10-08T01:15:24.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:24.002+0000 7f4ce8d9a700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4cc8066f30 0x7f4cc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:24.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:24.003+0000 7f4ce8d9a700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4cc8066f30 0x7f4cc80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4150fe0 tx=0x7f4cd4009040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:24.006+0000 7f4cd97fa700 1 -- 172.21.15.119:0/3568220110 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ce008c080 con 0x7f4ce40c83f0 2023-10-08T01:15:24.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:24.241+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4ce415aa80 con 0x7f4ce40c83f0 2023-10-08T01:15:25.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.233+0000 7f4cd97fa700 1 -- 172.21.15.119:0/3568220110 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4ce00150c0 con 0x7f4ce40c83f0 2023-10-08T01:15:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4cc8066f30 msgr2=0x7f4cc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4cc8066f30 0x7f4cc80693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4150fe0 tx=0x7f4cd4009040 comp rx=0 tx=0).stop 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 msgr2=0x7f4ce40c35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c35e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f4ce4063d40 tx=0x7f4ce0017000 comp rx=0 tx=0).stop 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 shutdown_connections 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4cc8066f30 0x7f4cc80693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ce41568e0 0x7f4ce40c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ce40c83f0 0x7f4ce40c35e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 --2- 172.21.15.119:0/3568220110 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4ce40ccee0 0x7f4ce416da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 >> 172.21.15.119:0/3568220110 conn(0x7f4ce4068e10 msgr2=0x7f4ce40ca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 shutdown_connections 2023-10-08T01:15:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.236+0000 7f4ceb7ff700 1 -- 172.21.15.119:0/3568220110 wait complete. 2023-10-08T01:15:25.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:15:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.439+0000 7f8daf59e700 1 Processor -- start 2023-10-08T01:15:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.439+0000 7f8daf59e700 1 -- start start 2023-10-08T01:15:25.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.440+0000 7f8daf59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db0159420 0x7f8db00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.440+0000 7f8daf59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8db01597f0 0x7f8db00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.440+0000 7f8daf59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8db015d230 0x7f8db015d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.440+0000 7f8daf59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db015dbd0 con 0x7f8db01597f0 2023-10-08T01:15:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.440+0000 7f8daf59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db00b11d0 con 0x7f8db015d230 2023-10-08T01:15:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.440+0000 7f8daf59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00b14b0 con 0x7f8db0159420 2023-10-08T01:15:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8db01597f0 0x7f8db00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8db01597f0 0x7f8db00b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41962/0 (socket says 172.21.15.119:41962) 2023-10-08T01:15:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 -- 172.21.15.119:0/3160303038 learned_addr learned my addr 172.21.15.119:0/3160303038 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8daed9d700 1 --2- 172.21.15.119:0/3160303038 >> v2:172.21.15.184:3300/0 conn(0x7f8db015d230 0x7f8db015d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dae59c700 1 --2- 172.21.15.119:0/3160303038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db0159420 0x7f8db00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 -- 172.21.15.119:0/3160303038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db0159420 msgr2=0x7f8db00b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 --2- 172.21.15.119:0/3160303038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db0159420 0x7f8db00b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 -- 172.21.15.119:0/3160303038 >> v2:172.21.15.184:3300/0 conn(0x7f8db015d230 msgr2=0x7f8db015d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 --2- 172.21.15.119:0/3160303038 >> v2:172.21.15.184:3300/0 conn(0x7f8db015d230 0x7f8db015d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 -- 172.21.15.119:0/3160303038 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00b1790 con 0x7f8db01597f0 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.441+0000 7f8dadd9b700 1 --2- 172.21.15.119:0/3160303038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db00b2270 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8db00d1720 tx=0x7f8da4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d584da66b7422 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8dad59a700 1 -- 172.21.15.119:0/3160303038 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400b640 con 0x7f8db01597f0 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8dad59a700 1 -- 172.21.15.119:0/3160303038 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400b7a0 con 0x7f8db01597f0 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8dad59a700 1 -- 172.21.15.119:0/3160303038 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400bb60 con 0x7f8db01597f0 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 -- 172.21.15.119:0/3160303038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 msgr2=0x7f8db00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 --2- 172.21.15.119:0/3160303038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db00b2270 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8db00d1720 tx=0x7f8da4009250 comp rx=0 tx=0).stop 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 -- 172.21.15.119:0/3160303038 shutdown_connections 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 --2- 172.21.15.119:0/3160303038 >> v2:172.21.15.184:3300/0 conn(0x7f8db015d230 0x7f8db015d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 --2- 172.21.15.119:0/3160303038 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db0159420 0x7f8db00b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 --2- 172.21.15.119:0/3160303038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db00b2270 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 -- 172.21.15.119:0/3160303038 >> 172.21.15.119:0/3160303038 conn(0x7f8db00d0660 msgr2=0x7f8db00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 -- 172.21.15.119:0/3160303038 shutdown_connections 2023-10-08T01:15:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.442+0000 7f8daf59e700 1 -- 172.21.15.119:0/3160303038 wait complete. 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.444+0000 7f8daf59e700 1 Processor -- start 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.444+0000 7f8daf59e700 1 -- start start 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8daf59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db0159420 0x7f8db015c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8daf59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db014cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8daf59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db015d230 0x7f8db016d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8daf59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db015b4d0 con 0x7f8db01597f0 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8daf59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db015a610 con 0x7f8db0159420 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8daf59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db015a8c0 con 0x7f8db015d230 2023-10-08T01:15:25.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8dadd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db014cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8dadd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db014cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41972/0 (socket says 172.21.15.119:41972) 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8dadd9b700 1 -- 172.21.15.119:0/418047362 learned_addr learned my addr 172.21.15.119:0/418047362 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8daed9d700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db015d230 0x7f8db016d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.445+0000 7f8dae59c700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db0159420 0x7f8db015c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8dadd9b700 1 -- 172.21.15.119:0/418047362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db015d230 msgr2=0x7f8db016d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8dadd9b700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db015d230 0x7f8db016d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8dadd9b700 1 -- 172.21.15.119:0/418047362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db0159420 msgr2=0x7f8db015c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8dadd9b700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db0159420 0x7f8db015c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8dadd9b700 1 -- 172.21.15.119:0/418047362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da40094f0 con 0x7f8db01597f0 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8dadd9b700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db014cd90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8db0089340 tx=0x7f8da4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8d9e7fc700 1 -- 172.21.15.119:0/418047362 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400b910 con 0x7f8db01597f0 2023-10-08T01:15:25.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8d9e7fc700 1 -- 172.21.15.119:0/418047362 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400ba70 con 0x7f8db01597f0 2023-10-08T01:15:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db015aba0 con 0x7f8db01597f0 2023-10-08T01:15:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8d9e7fc700 1 -- 172.21.15.119:0/418047362 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4011630 con 0x7f8db01597f0 2023-10-08T01:15:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.446+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db015b160 con 0x7f8db01597f0 2023-10-08T01:15:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.447+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db015b640 con 0x7f8db01597f0 2023-10-08T01:15:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.448+0000 7f8d9e7fc700 1 -- 172.21.15.119:0/418047362 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8da40119a0 con 0x7f8db01597f0 2023-10-08T01:15:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.449+0000 7f8d9e7fc700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8d90066f30 0x7f8d900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.449+0000 7f8d9e7fc700 1 -- 172.21.15.119:0/418047362 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f8da4085e40 con 0x7f8db01597f0 2023-10-08T01:15:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.449+0000 7f8dae59c700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8d90066f30 0x7f8d900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.450+0000 7f8dae59c700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8d90066f30 0x7f8d900693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009db0 tx=0x7f8da00091a0 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.453+0000 7f8d9e7fc700 1 -- 172.21.15.119:0/418047362 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8da4052e70 con 0x7f8db01597f0 2023-10-08T01:15:25.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.686+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8db004c3c0 con 0x7f8db01597f0 2023-10-08T01:15:25.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.686+0000 7f8d9e7fc700 1 -- 172.21.15.119:0/418047362 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f8da4056490 con 0x7f8db01597f0 2023-10-08T01:15:25.687 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:25.688 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7549,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/26538394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":26538394},{"type":"v1","addr":"172.21.15.184:6834","nonce":26538394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3494509212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3494509212},{"type":"v1","addr":"172.21.15.119:6834","nonce":3494509212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2792091862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2792091862},{"type":"v1","addr":"172.21.15.119:6835","nonce":2792091862}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:15:21.218141+0000","modified":"2023-10-08T01:15:25.231364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1485819493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1485819493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1485819493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T01:15:25.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.688+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8d90066f30 msgr2=0x7f8d900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.688+0000 7f8daf59e700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8d90066f30 0x7f8d900693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8da0009db0 tx=0x7f8da00091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.688+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 msgr2=0x7f8db014cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.688+0000 7f8daf59e700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db014cd90 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f8db0089340 tx=0x7f8da4017000 comp rx=0 tx=0).stop 2023-10-08T01:15:25.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.689+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 shutdown_connections 2023-10-08T01:15:25.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.689+0000 7f8daf59e700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8d90066f30 0x7f8d900693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.689+0000 7f8daf59e700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8db015d230 0x7f8db016d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.689+0000 7f8daf59e700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8db0159420 0x7f8db015c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.689+0000 7f8daf59e700 1 --2- 172.21.15.119:0/418047362 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db01597f0 0x7f8db014cd90 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.689+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 >> 172.21.15.119:0/418047362 conn(0x7f8db00d0660 msgr2=0x7f8db00ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:25.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.689+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 shutdown_connections 2023-10-08T01:15:25.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.689+0000 7f8daf59e700 1 -- 172.21.15.119:0/418047362 wait complete. 2023-10-08T01:15:25.691 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 63 2023-10-08T01:15:25.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T01:15:25.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.890+0000 7f0965fff700 1 Processor -- start 2023-10-08T01:15:25.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.890+0000 7f0965fff700 1 -- start start 2023-10-08T01:15:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.891+0000 7f0965fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f096006bcd0 0x7f09600caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.891+0000 7f0965fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f096015df50 0x7f096015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.891+0000 7f0965fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096015d7e0 con 0x7f096006c0a0 2023-10-08T01:15:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.891+0000 7f0965fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096015dac0 con 0x7f096015df50 2023-10-08T01:15:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.891+0000 7f0965fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096015aa30 con 0x7f096006bcd0 2023-10-08T01:15:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f096015df50 0x7f096015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f096015df50 0x7f096015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41974/0 (socket says 172.21.15.119:41974) 2023-10-08T01:15:25.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095ffff700 1 -- 172.21.15.119:0/2617641383 learned_addr learned my addr 172.21.15.119:0/2617641383 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:25.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095f7fe700 1 --2- 172.21.15.119:0/2617641383 >> v2:172.21.15.184:3300/0 conn(0x7f096006bcd0 0x7f09600caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095ffff700 1 -- 172.21.15.119:0/2617641383 >> v2:172.21.15.184:3300/0 conn(0x7f096006bcd0 msgr2=0x7f09600caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095ffff700 1 --2- 172.21.15.119:0/2617641383 >> v2:172.21.15.184:3300/0 conn(0x7f096006bcd0 0x7f09600caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095ffff700 1 -- 172.21.15.119:0/2617641383 >> v1:172.21.15.184:6789/0 conn(0x7f096006c0a0 legacy=0x7f096015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095ffff700 1 -- 172.21.15.119:0/2617641383 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096015ad10 con 0x7f096015df50 2023-10-08T01:15:25.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.892+0000 7f095f7fe700 1 --2- 172.21.15.119:0/2617641383 >> v2:172.21.15.184:3300/0 conn(0x7f096006bcd0 0x7f09600caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:25.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f095ffff700 1 --2- 172.21.15.119:0/2617641383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096015df50 0x7f096015e3b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f09600691a0 tx=0x7f0958009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3a29f1f847a491 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:25.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f095e7fc700 1 -- 172.21.15.119:0/2617641383 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800e070 con 0x7f096015df50 2023-10-08T01:15:25.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f095e7fc700 1 -- 172.21.15.119:0/2617641383 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095800ba20 con 0x7f096015df50 2023-10-08T01:15:25.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f095e7fc700 1 -- 172.21.15.119:0/2617641383 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09580092e0 con 0x7f096015df50 2023-10-08T01:15:25.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f0965fff700 1 -- 172.21.15.119:0/2617641383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096015df50 msgr2=0x7f096015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f0965fff700 1 --2- 172.21.15.119:0/2617641383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096015df50 0x7f096015e3b0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f09600691a0 tx=0x7f0958009840 comp rx=0 tx=0).stop 2023-10-08T01:15:25.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f0965fff700 1 -- 172.21.15.119:0/2617641383 shutdown_connections 2023-10-08T01:15:25.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f0965fff700 1 --2- 172.21.15.119:0/2617641383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096015df50 0x7f096015e3b0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f0965fff700 1 --2- 172.21.15.119:0/2617641383 >> v2:172.21.15.184:3300/0 conn(0x7f096006bcd0 0x7f09600caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f0965fff700 1 -- 172.21.15.119:0/2617641383 >> 172.21.15.119:0/2617641383 conn(0x7f09600ce600 msgr2=0x7f09600cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:25.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.893+0000 7f0965fff700 1 -- 172.21.15.119:0/2617641383 shutdown_connections 2023-10-08T01:15:25.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.894+0000 7f0965fff700 1 -- 172.21.15.119:0/2617641383 wait complete. 2023-10-08T01:15:25.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.895+0000 7f0965fff700 1 Processor -- start 2023-10-08T01:15:25.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.895+0000 7f0965fff700 1 -- start start 2023-10-08T01:15:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f0965fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096006bcd0 0x7f09601548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f0965fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f096006c0a0 0x7f096016de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f0965fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f096015df50 0x7f0960153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f0965fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0960169e60 con 0x7f096006bcd0 2023-10-08T01:15:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f0965fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096016a6c0 con 0x7f096006c0a0 2023-10-08T01:15:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f0965fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0960154e90 con 0x7f096015df50 2023-10-08T01:15:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096006bcd0 0x7f09601548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096006bcd0 0x7f09601548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41982/0 (socket says 172.21.15.119:41982) 2023-10-08T01:15:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095f7fe700 1 -- 172.21.15.119:0/2751160261 learned_addr learned my addr 172.21.15.119:0/2751160261 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095effd700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f096006c0a0 0x7f096016de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095ffff700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f096015df50 0x7f0960153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095f7fe700 1 -- 172.21.15.119:0/2751160261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f096015df50 msgr2=0x7f0960153150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095f7fe700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f096015df50 0x7f0960153150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095f7fe700 1 -- 172.21.15.119:0/2751160261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f096006c0a0 msgr2=0x7f096016de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095f7fe700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f096006c0a0 0x7f096016de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.896+0000 7f095f7fe700 1 -- 172.21.15.119:0/2751160261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950009420 con 0x7f096006bcd0 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.897+0000 7f095f7fe700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096006bcd0 0x7f09601548b0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f095000ec80 tx=0x7f09500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.897+0000 7f09477fe700 1 -- 172.21.15.119:0/2751160261 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000c9d0 con 0x7f096006bcd0 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.897+0000 7f09477fe700 1 -- 172.21.15.119:0/2751160261 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095000cb30 con 0x7f096006bcd0 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.897+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0958009ae0 con 0x7f096006bcd0 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.897+0000 7f09477fe700 1 -- 172.21.15.119:0/2751160261 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095000ceb0 con 0x7f096006bcd0 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.897+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096016a350 con 0x7f096006bcd0 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.898+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096016a9a0 con 0x7f096006bcd0 2023-10-08T01:15:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.899+0000 7f09477fe700 1 -- 172.21.15.119:0/2751160261 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0950013680 con 0x7f096006bcd0 2023-10-08T01:15:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.900+0000 7f09477fe700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f094c066f30 0x7f094c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.900+0000 7f09477fe700 1 -- 172.21.15.119:0/2751160261 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f0950084a70 con 0x7f096006bcd0 2023-10-08T01:15:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.900+0000 7f095effd700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f094c066f30 0x7f094c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.901+0000 7f095effd700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f094c066f30 0x7f094c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0960153f40 tx=0x7f0954009040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:25.905+0000 7f09477fe700 1 -- 172.21.15.119:0/2751160261 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0950051b40 con 0x7f096006bcd0 2023-10-08T01:15:26.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.097+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f096004c3c0 con 0x7f096006bcd0 2023-10-08T01:15:26.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.098+0000 7f09477fe700 1 -- 172.21.15.119:0/2751160261 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f0950055160 con 0x7f096006bcd0 2023-10-08T01:15:26.099 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:26.099 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":135,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T01:15:22.771760+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:15:19.321580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T01:15:20.200200+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T01:15:26.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f094c066f30 msgr2=0x7f094c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f094c066f30 0x7f094c0693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0960153f40 tx=0x7f0954009040 comp rx=0 tx=0).stop 2023-10-08T01:15:26.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096006bcd0 msgr2=0x7f09601548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096006bcd0 0x7f09601548b0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f095000ec80 tx=0x7f09500091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 shutdown_connections 2023-10-08T01:15:26.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f094c066f30 0x7f094c0693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f096015df50 0x7f0960153150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f096006bcd0 0x7f09601548b0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 --2- 172.21.15.119:0/2751160261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f096006c0a0 0x7f096016de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 >> 172.21.15.119:0/2751160261 conn(0x7f09600ce600 msgr2=0x7f09600c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:26.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 shutdown_connections 2023-10-08T01:15:26.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.100+0000 7f0965fff700 1 -- 172.21.15.119:0/2751160261 wait complete. 2023-10-08T01:15:26.119 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T01:15:26.119 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:15:26.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.300+0000 7f180e59d700 1 Processor -- start 2023-10-08T01:15:26.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.300+0000 7f180e59d700 1 -- start start 2023-10-08T01:15:26.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180e59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f1808156310 0x7f18080c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180e59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18081566e0 0x7f18080ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180e59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f18080ca300 0x7f18080ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180e59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18080ca860 con 0x7f18080ca300 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180e59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18080cab40 con 0x7f1808156310 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180e59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18080cae20 con 0x7f18081566e0 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180dd9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f18080ca300 0x7f18080ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180dd9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f18080ca300 0x7f18080ae540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41992/0 (socket says 172.21.15.119:41992) 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180dd9c700 1 -- 172.21.15.119:0/545293688 learned_addr learned my addr 172.21.15.119:0/545293688 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180d59b700 1 --2- 172.21.15.119:0/545293688 >> v2:172.21.15.184:3300/0 conn(0x7f1808156310 0x7f18080c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180cd9a700 1 --2- 172.21.15.119:0/545293688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18081566e0 0x7f18080ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180dd9c700 1 -- 172.21.15.119:0/545293688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18081566e0 msgr2=0x7f18080ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:26.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180dd9c700 1 --2- 172.21.15.119:0/545293688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18081566e0 0x7f18080ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180dd9c700 1 -- 172.21.15.119:0/545293688 >> v2:172.21.15.184:3300/0 conn(0x7f1808156310 msgr2=0x7f18080c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:26.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180dd9c700 1 --2- 172.21.15.119:0/545293688 >> v2:172.21.15.184:3300/0 conn(0x7f1808156310 0x7f18080c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180dd9c700 1 -- 172.21.15.119:0/545293688 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18080cb100 con 0x7f18080ca300 2023-10-08T01:15:26.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.302+0000 7f180d59b700 1 --2- 172.21.15.119:0/545293688 >> v2:172.21.15.184:3300/0 conn(0x7f1808156310 0x7f18080c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:26.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.303+0000 7f180dd9c700 1 --2- 172.21.15.119:0/545293688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f18080ae540 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f18080cce20 tx=0x7f1800009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c1cd66be238e72 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:26.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.303+0000 7f17fffff700 1 -- 172.21.15.119:0/545293688 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180000e070 con 0x7f18080ca300 2023-10-08T01:15:26.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.303+0000 7f17fffff700 1 -- 172.21.15.119:0/545293688 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180000ba50 con 0x7f18080ca300 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.303+0000 7f17fffff700 1 -- 172.21.15.119:0/545293688 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18000092e0 con 0x7f18080ca300 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.303+0000 7f180e59d700 1 -- 172.21.15.119:0/545293688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 msgr2=0x7f18080ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.303+0000 7f180e59d700 1 --2- 172.21.15.119:0/545293688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f18080ae540 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f18080cce20 tx=0x7f1800009840 comp rx=0 tx=0).stop 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.304+0000 7f180e59d700 1 -- 172.21.15.119:0/545293688 shutdown_connections 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.304+0000 7f180e59d700 1 --2- 172.21.15.119:0/545293688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f18080ae540 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.304+0000 7f180e59d700 1 --2- 172.21.15.119:0/545293688 >> v2:172.21.15.184:3300/0 conn(0x7f1808156310 0x7f18080c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.304+0000 7f180e59d700 1 --2- 172.21.15.119:0/545293688 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18081566e0 0x7f18080ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.304+0000 7f180e59d700 1 -- 172.21.15.119:0/545293688 >> 172.21.15.119:0/545293688 conn(0x7f18080cc1f0 msgr2=0x7f18080cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.304+0000 7f180e59d700 1 -- 172.21.15.119:0/545293688 shutdown_connections 2023-10-08T01:15:26.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.304+0000 7f180e59d700 1 -- 172.21.15.119:0/545293688 wait complete. 2023-10-08T01:15:26.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.305+0000 7f180e59d700 1 Processor -- start 2023-10-08T01:15:26.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.305+0000 7f180e59d700 1 -- start start 2023-10-08T01:15:26.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180e59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f1808150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180e59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1808156310 0x7f18080c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180e59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18081566e0 0x7f1808169930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180e59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18080afcd0 con 0x7f18080ca300 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180e59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18080c21f0 con 0x7f18081566e0 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180e59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18080af0c0 con 0x7f1808156310 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180d59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f1808150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180d59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f1808150f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42000/0 (socket says 172.21.15.119:42000) 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.306+0000 7f180d59b700 1 -- 172.21.15.119:0/432258607 learned_addr learned my addr 172.21.15.119:0/432258607 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180cd9a700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1808156310 0x7f18080c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180dd9c700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18081566e0 0x7f1808169930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180d59b700 1 -- 172.21.15.119:0/432258607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1808156310 msgr2=0x7f18080c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180d59b700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1808156310 0x7f18080c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180d59b700 1 -- 172.21.15.119:0/432258607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18081566e0 msgr2=0x7f1808169930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180d59b700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18081566e0 0x7f1808169930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180d59b700 1 -- 172.21.15.119:0/432258607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1800009ae0 con 0x7f18080ca300 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180d59b700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f1808150f90 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f180400e8d0 tx=0x7f1804008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f17fd7fa700 1 -- 172.21.15.119:0/432258607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180400ee70 con 0x7f18080ca300 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f17fd7fa700 1 -- 172.21.15.119:0/432258607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180400c5e0 con 0x7f18080ca300 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18080af3a0 con 0x7f18080ca300 2023-10-08T01:15:26.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f17fd7fa700 1 -- 172.21.15.119:0/432258607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180400c980 con 0x7f18080ca300 2023-10-08T01:15:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.307+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18080af960 con 0x7f18080ca300 2023-10-08T01:15:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.308+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180804c3c0 con 0x7f18080ca300 2023-10-08T01:15:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.309+0000 7f17fd7fa700 1 -- 172.21.15.119:0/432258607 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1804013410 con 0x7f18080ca300 2023-10-08T01:15:26.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.310+0000 7f17fd7fa700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17ec066f30 0x7f17ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.310+0000 7f17fd7fa700 1 -- 172.21.15.119:0/432258607 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f18040845d0 con 0x7f18080ca300 2023-10-08T01:15:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.310+0000 7f180cd9a700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17ec066f30 0x7f17ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.311+0000 7f180cd9a700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17ec066f30 0x7f17ec0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f18080c25e0 tx=0x7f17f8008040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:26.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.314+0000 7f17fd7fa700 1 -- 172.21.15.119:0/432258607 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1804051600 con 0x7f18080ca300 2023-10-08T01:15:26.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.501+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f18080afe10 con 0x7f17ec066f30 2023-10-08T01:15:26.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.511+0000 7f17fd7fa700 1 -- 172.21.15.119:0/432258607 <== mgr.7989 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278253 (secure 0 0 0) 0x7f17f800f6e0 con 0x7f17ec066f30 2023-10-08T01:15:26.515 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:26.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17ec066f30 msgr2=0x7f17ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17ec066f30 0x7f17ec0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f18080c25e0 tx=0x7f17f8008040 comp rx=0 tx=0).stop 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 msgr2=0x7f1808150f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f1808150f90 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f180400e8d0 tx=0x7f1804008040 comp rx=0 tx=0).stop 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 shutdown_connections 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17ec066f30 0x7f17ec0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f18081566e0 0x7f1808169930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18080ca300 0x7f1808150f90 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 --2- 172.21.15.119:0/432258607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1808156310 0x7f18080c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 >> 172.21.15.119:0/432258607 conn(0x7f18080cc1f0 msgr2=0x7f1808151d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 shutdown_connections 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.515+0000 7f180e59d700 1 -- 172.21.15.119:0/432258607 wait complete. 2023-10-08T01:15:26.518 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:15:26.535 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:15:25.693568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449596,"kb_used_data":13948,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752525124,"statfs":{"total":773094113280,"available":770585726976,"internally_reserved":0,"allocated":14282752,"data_stored":7604711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.955351"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745570+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:54.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747746+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:37.841133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747552+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:57.251999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747225+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:55.593104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.746581+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:51:03.891824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741037+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:46.116333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741530+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:09.044142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745461+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:17.975581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748200+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:31.074576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741127+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:06:36.574221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744161+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:52.124108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743824+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:10.573223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744443+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:11.456448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741651+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:04.750260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748170+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:56.803819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743801+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:42.670628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748288+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:48.508166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.742969+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:30.277372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744476+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:52.427565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743998+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:48.508166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747232+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:45:48.338360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743580+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:31.074576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744634+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:08.365749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741072+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:27:44.699900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741519+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:33.182686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745045+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:33.222269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741044+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:07.797723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745954+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:29.185128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741519+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:30.277372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747177+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:54:45.165121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744088+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:37.703517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744444+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:29.780863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"124'249","reported_seq":724,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-08T01:15:17.234905+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:15:17.234905+0000","last_peered":"2023-10-08T01:15:17.234905+0000","last_clean":"2023-10-08T01:15:17.234905+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:15:17.234905+0000","last_undegraded":"2023-10-08T01:15:17.234905+0000","last_fullsized":"2023-10-08T01:15:17.234905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307660,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064180,"statfs":{"total":96636764160,"available":96321720320,"internally_reserved":0,"allocated":2232320,"data_stored":1394143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.123},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82999999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.68799999999999994},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.85399999999999998,"5min":0.95499999999999996,"15min":0.97699999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39200000000000002},"max":{"1min":0.92200000000000004,"5min":0.93600000000000005,"15min":1.0269999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.87,"5min":1.133,"15min":1.133},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.66500000000000004,"15min":0.71399999999999997},"min":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.95099999999999996,"15min":1.167},"last":0.79100000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.69899999999999995},"min":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.93200000000000005,"15min":0.94699999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.501,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.052,"15min":1.079},"last":0.77400000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.5},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72499999999999998,"5min":0.875,"15min":0.88300000000000001},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.62},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84899999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82299999999999995,"5min":1.004,"15min":1.004},"last":0.55500000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86099999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.47099999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575078,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.93200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67800000000000005,"15min":0.69099999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91200000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70599999999999996,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":2.984},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38100000000000001},"max":{"1min":0.86799999999999999,"5min":1.004,"15min":3.0190000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.95999999999999996,"5min":0.998,"15min":0.998},"last":0.47699999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72499999999999998,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71699999999999997,"15min":0.73299999999999998},"min":{"1min":0.56100000000000005,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.92400000000000004,"5min":1.048,"15min":1.048},"last":0.73899999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.88500000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.623},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79000000000000004,"5min":0.89800000000000002,"15min":1.087},"last":0.75900000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.395},"last":0.627},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.21},"last":0.61599999999999999}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":2.9489999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45200000000000001,"15min":0.46500000000000002},"min":{"1min":0.29999999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45900000000000002,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.61099999999999999,"5min":0.70599999999999996,"15min":0.72499999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.499}]},{"osd":3,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.81899999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":1.296,"15min":1.452},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.099,"5min":1.137,"15min":1.1839999999999999},"last":0.98599999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":1.119,"15min":1.3959999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.77700000000000002},"min":{"1min":0.53200000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.5209999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.56399999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.6020000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.77700000000000002},"min":{"1min":0.60899999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.046,"5min":1.1499999999999999,"15min":1.3660000000000001},"last":1.032}]},{"osd":7,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77400000000000002,"15min":0.81799999999999995},"min":{"1min":0.63500000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96699999999999997,"5min":1.236,"15min":1.621},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.51300000000000001},"max":{"1min":1.0389999999999999,"5min":1.21,"15min":1.4890000000000001},"last":0.91500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575077,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303564,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068276,"statfs":{"total":96636764160,"available":96325914624,"internally_reserved":0,"allocated":2232320,"data_stored":1394143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45500000000000002,"15min":0.46200000000000002},"min":{"1min":0.28199999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56499999999999995,"5min":0.95999999999999996,"15min":0.997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.28599999999999998},"max":{"1min":0.69699999999999995,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.373}]},{"osd":2,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59099999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.31900000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68200000000000005,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.497}]},{"osd":4,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.53600000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":1.006,"5min":1.341,"15min":1.617},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.55800000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88500000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.623}]},{"osd":5,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":2.085,"15min":2.085},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.014,"5min":1.381,"15min":1.5289999999999999},"last":0.80100000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.50900000000000001},"max":{"1min":0.877,"5min":1.361,"15min":1.3660000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94999999999999996,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.80700000000000005},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.054,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83199999999999996},"min":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93700000000000006,"5min":1.2170000000000001,"15min":1.298},"last":0.73699999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575077,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311176,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060664,"statfs":{"total":96636764160,"available":96318119936,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.433}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.85299999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.88},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.81299999999999994}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.94099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307080,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064760,"statfs":{"total":96636764160,"available":96322314240,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":0.56100000000000005,"5min":0.627,"15min":0.80100000000000005},"last":0.308},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46200000000000002,"15min":0.47899999999999998},"min":{"1min":0.307,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.81200000000000006},"last":0.35999999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":0.70099999999999996,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.376},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48599999999999999,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.31},"max":{"1min":0.57899999999999996,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.877,"15min":0.89700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.46000000000000002,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70599999999999996,"15min":0.74399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.98899999999999999,"5min":1.095,"15min":1.3029999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.413},"last":0.64700000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.76400000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.032,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.089},"last":0.60799999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.47999999999999998},"max":{"1min":1.0680000000000001,"5min":1.101,"15min":1.379},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.57799999999999996,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.486},"last":0.69299999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.78400000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.46500000000000002},"max":{"1min":1.0549999999999999,"5min":1.0840000000000001,"15min":1.389},"last":0.623},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.78500000000000003},"min":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.94899999999999995,"5min":1.054,"15min":1.4730000000000001},"last":0.72199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311164,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060676,"statfs":{"total":96636764160,"available":96318132224,"internally_reserved":0,"allocated":1626112,"data_stored":797550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.628},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.75},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.55800000000000005}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":0.80900000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.90500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.77200000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.93000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.45000000000000001,"15min":0.34699999999999998},"max":{"1min":0.89700000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.622}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.71099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":1.018,"5min":1.026,"15min":1.026},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81699999999999995,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68799999999999994,"15min":0.70999999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71099999999999997,"15min":0.73299999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.96599999999999997,"5min":1.026,"15min":1.026},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94199999999999995,"5min":1.004,"15min":1.004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92200000000000004,"5min":1.034,"15min":1.034},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.90900000000000003},"last":0.38},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0680000000000001},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.52800000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.58699999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.095},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.71599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92000000000000004,"5min":1.05,"15min":1.05},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:15:26.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:15:26.729 INFO:tasks.ceph.mds.b.smithi184.stderr:2023-10-08T01:15:26.729+0000 7fee6a5f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T01:15:26.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.755+0000 7ff82b59e700 1 Processor -- start 2023-10-08T01:15:26.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.755+0000 7ff82b59e700 1 -- start start 2023-10-08T01:15:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82b59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff82c156310 0x7ff82c0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff82c0ae710 0x7ff82c0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82b59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff82c0ca860 con 0x7ff82c1566e0 2023-10-08T01:15:26.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82b59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0cab40 con 0x7ff82c0ae710 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82b59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82c0cae20 con 0x7ff82c156310 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff82c0ae710 0x7ff82c0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff82c0ae710 0x7ff82c0c6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42002/0 (socket says 172.21.15.119:42002) 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82ad9d700 1 -- 172.21.15.119:0/390804542 learned_addr learned my addr 172.21.15.119:0/390804542 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff829d9b700 1 --1- 172.21.15.119:0/390804542 >> v1:172.21.15.119:6789/0 conn(0x7ff82c1566e0 0x7ff82c0ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42582/0 (socket says 172.21.15.119:42582) 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.756+0000 7ff82a59c700 1 --2- 172.21.15.119:0/390804542 >> v2:172.21.15.184:3300/0 conn(0x7ff82c156310 0x7ff82c0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82ad9d700 1 -- 172.21.15.119:0/390804542 >> v2:172.21.15.184:3300/0 conn(0x7ff82c156310 msgr2=0x7ff82c0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82ad9d700 1 --2- 172.21.15.119:0/390804542 >> v2:172.21.15.184:3300/0 conn(0x7ff82c156310 0x7ff82c0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82ad9d700 1 -- 172.21.15.119:0/390804542 >> v1:172.21.15.119:6789/0 conn(0x7ff82c1566e0 legacy=0x7ff82c0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82ad9d700 1 -- 172.21.15.119:0/390804542 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82c0cb100 con 0x7ff82c0ae710 2023-10-08T01:15:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82959a700 1 -- 172.21.15.119:0/390804542 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352821055 0 0) 0x7ff82c0ca860 con 0x7ff82c1566e0 2023-10-08T01:15:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82ad9d700 1 --2- 172.21.15.119:0/390804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c0c6830 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0cce20 tx=0x7ff82400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f410741cf844762 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82959a700 1 -- 172.21.15.119:0/390804542 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824010070 con 0x7ff82c0ae710 2023-10-08T01:15:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82959a700 1 -- 172.21.15.119:0/390804542 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82400dde0 con 0x7ff82c0ae710 2023-10-08T01:15:26.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.757+0000 7ff82959a700 1 -- 172.21.15.119:0/390804542 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82400b320 con 0x7ff82c0ae710 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.758+0000 7ff82b59e700 1 -- 172.21.15.119:0/390804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 msgr2=0x7ff82c0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.758+0000 7ff82b59e700 1 --2- 172.21.15.119:0/390804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c0c6830 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0cce20 tx=0x7ff82400ba50 comp rx=0 tx=0).stop 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.758+0000 7ff82b59e700 1 -- 172.21.15.119:0/390804542 shutdown_connections 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.758+0000 7ff82b59e700 1 --2- 172.21.15.119:0/390804542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c0c6830 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.758+0000 7ff82b59e700 1 --2- 172.21.15.119:0/390804542 >> v2:172.21.15.184:3300/0 conn(0x7ff82c156310 0x7ff82c0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.758+0000 7ff82b59e700 1 -- 172.21.15.119:0/390804542 >> 172.21.15.119:0/390804542 conn(0x7ff82c0cc1f0 msgr2=0x7ff82c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.758+0000 7ff82b59e700 1 -- 172.21.15.119:0/390804542 shutdown_connections 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.758+0000 7ff82b59e700 1 -- 172.21.15.119:0/390804542 wait complete. 2023-10-08T01:15:26.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.759+0000 7ff82b59e700 1 Processor -- start 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.760+0000 7ff82b59e700 1 -- start start 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.760+0000 7ff82b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c14c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c156310 0x7ff82c147ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82b59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c14d330 con 0x7ff82c0ae710 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82b59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c1496b0 con 0x7ff82c1566e0 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff82c0af300 con 0x7ff82c156310 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c14c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c14c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42016/0 (socket says 172.21.15.119:42016) 2023-10-08T01:15:26.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 -- 172.21.15.119:0/632901838 learned_addr learned my addr 172.21.15.119:0/632901838 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 -- 172.21.15.119:0/632901838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c156310 msgr2=0x7ff82c147ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c156310 0x7ff82c147ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 -- 172.21.15.119:0/632901838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c1566e0 msgr2=0x7ff82c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 -- 172.21.15.119:0/632901838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82400bcf0 con 0x7ff82c0ae710 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.761+0000 7ff82a59c700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c14c3a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00e930 tx=0x7ff81c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.762+0000 7ff81a7fc700 1 -- 172.21.15.119:0/632901838 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c00eed0 con 0x7ff82c0ae710 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.762+0000 7ff81a7fc700 1 -- 172.21.15.119:0/632901838 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c00c850 con 0x7ff82c0ae710 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.762+0000 7ff81a7fc700 1 -- 172.21.15.119:0/632901838 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c00cbf0 con 0x7ff82c0ae710 2023-10-08T01:15:26.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.762+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82c0af5e0 con 0x7ff82c0ae710 2023-10-08T01:15:26.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.762+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82c0afba0 con 0x7ff82c0ae710 2023-10-08T01:15:26.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.763+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff82c04c3c0 con 0x7ff82c0ae710 2023-10-08T01:15:26.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.763+0000 7ff81a7fc700 1 -- 172.21.15.119:0/632901838 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff81c013470 con 0x7ff82c0ae710 2023-10-08T01:15:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.765+0000 7ff81a7fc700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff80c066f30 0x7ff80c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.765+0000 7ff81a7fc700 1 -- 172.21.15.119:0/632901838 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7ff81c084900 con 0x7ff82c0ae710 2023-10-08T01:15:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.765+0000 7ff829d9b700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff80c066f30 0x7ff80c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:26.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.765+0000 7ff829d9b700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff80c066f30 0x7ff80c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff8200011e0 tx=0x7ff8200079b0 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:26.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.769+0000 7ff81a7fc700 1 -- 172.21.15.119:0/632901838 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff81c0519d0 con 0x7ff82c0ae710 2023-10-08T01:15:26.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.956+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff82c14d470 con 0x7ff80c066f30 2023-10-08T01:15:26.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.964+0000 7ff81a7fc700 1 -- 172.21.15.119:0/632901838 <== mgr.7989 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278253 (secure 0 0 0) 0x7ff8200070d0 con 0x7ff80c066f30 2023-10-08T01:15:26.968 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:26.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff80c066f30 msgr2=0x7ff80c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff80c066f30 0x7ff80c0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff8200011e0 tx=0x7ff8200079b0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 msgr2=0x7ff82c14c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:26.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c14c3a0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00e930 tx=0x7ff81c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 shutdown_connections 2023-10-08T01:15:26.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff80c066f30 0x7ff80c0693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff82c1566e0 0x7ff82c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff82c0ae710 0x7ff82c14c3a0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 --2- 172.21.15.119:0/632901838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff82c156310 0x7ff82c147ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:26.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.967+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 >> 172.21.15.119:0/632901838 conn(0x7ff82c0cc1f0 msgr2=0x7ff82c069870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:26.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.968+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 shutdown_connections 2023-10-08T01:15:26.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:26.968+0000 7ff82b59e700 1 -- 172.21.15.119:0/632901838 wait complete. 2023-10-08T01:15:26.972 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:15:26.986 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:15:25.693568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449596,"kb_used_data":13948,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752525124,"statfs":{"total":773094113280,"available":770585726976,"internally_reserved":0,"allocated":14282752,"data_stored":7604711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.955351"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745570+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:54.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747746+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:37.841133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747552+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:57.251999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747225+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:55.593104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.746581+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:51:03.891824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741037+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:46.116333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741530+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:09.044142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745461+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:17.975581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748200+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:31.074576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741127+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:06:36.574221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744161+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:52.124108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743824+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:10.573223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744443+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:11.456448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741651+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:04.750260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748170+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:56.803819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743801+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:42.670628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748288+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:48.508166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.742969+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:30.277372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744476+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:52.427565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743998+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:48.508166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747232+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:45:48.338360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743580+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:31.074576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744634+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:08.365749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741072+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:27:44.699900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741519+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:33.182686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745045+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:33.222269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741044+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:07.797723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745954+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:29.185128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741519+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:30.277372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747177+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:54:45.165121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744088+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:37.703517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744444+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:29.780863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"124'249","reported_seq":724,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-08T01:15:17.234905+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:15:17.234905+0000","last_peered":"2023-10-08T01:15:17.234905+0000","last_clean":"2023-10-08T01:15:17.234905+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:15:17.234905+0000","last_undegraded":"2023-10-08T01:15:17.234905+0000","last_fullsized":"2023-10-08T01:15:17.234905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307660,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064180,"statfs":{"total":96636764160,"available":96321720320,"internally_reserved":0,"allocated":2232320,"data_stored":1394143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.123},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82999999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.68799999999999994},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.85399999999999998,"5min":0.95499999999999996,"15min":0.97699999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39200000000000002},"max":{"1min":0.92200000000000004,"5min":0.93600000000000005,"15min":1.0269999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.87,"5min":1.133,"15min":1.133},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.66500000000000004,"15min":0.71399999999999997},"min":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.95099999999999996,"15min":1.167},"last":0.79100000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.69899999999999995},"min":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.93200000000000005,"15min":0.94699999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.501,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.052,"15min":1.079},"last":0.77400000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.5},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72499999999999998,"5min":0.875,"15min":0.88300000000000001},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.62},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84899999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82299999999999995,"5min":1.004,"15min":1.004},"last":0.55500000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86099999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.47099999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575078,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.93200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67800000000000005,"15min":0.69099999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91200000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70599999999999996,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":2.984},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38100000000000001},"max":{"1min":0.86799999999999999,"5min":1.004,"15min":3.0190000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.95999999999999996,"5min":0.998,"15min":0.998},"last":0.47699999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72499999999999998,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71699999999999997,"15min":0.73299999999999998},"min":{"1min":0.56100000000000005,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.92400000000000004,"5min":1.048,"15min":1.048},"last":0.73899999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.88500000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.623},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79000000000000004,"5min":0.89800000000000002,"15min":1.087},"last":0.75900000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.395},"last":0.627},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.21},"last":0.61599999999999999}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":2.9489999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45200000000000001,"15min":0.46500000000000002},"min":{"1min":0.29999999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45900000000000002,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.61099999999999999,"5min":0.70599999999999996,"15min":0.72499999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.499}]},{"osd":3,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.81899999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":1.296,"15min":1.452},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.099,"5min":1.137,"15min":1.1839999999999999},"last":0.98599999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":1.119,"15min":1.3959999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.77700000000000002},"min":{"1min":0.53200000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.5209999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.56399999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.6020000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.77700000000000002},"min":{"1min":0.60899999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.046,"5min":1.1499999999999999,"15min":1.3660000000000001},"last":1.032}]},{"osd":7,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77400000000000002,"15min":0.81799999999999995},"min":{"1min":0.63500000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96699999999999997,"5min":1.236,"15min":1.621},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.51300000000000001},"max":{"1min":1.0389999999999999,"5min":1.21,"15min":1.4890000000000001},"last":0.91500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575077,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303564,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068276,"statfs":{"total":96636764160,"available":96325914624,"internally_reserved":0,"allocated":2232320,"data_stored":1394143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45500000000000002,"15min":0.46200000000000002},"min":{"1min":0.28199999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56499999999999995,"5min":0.95999999999999996,"15min":0.997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.28599999999999998},"max":{"1min":0.69699999999999995,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.373}]},{"osd":2,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59099999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.31900000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68200000000000005,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.497}]},{"osd":4,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.53600000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":1.006,"5min":1.341,"15min":1.617},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.55800000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88500000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.623}]},{"osd":5,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":2.085,"15min":2.085},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.014,"5min":1.381,"15min":1.5289999999999999},"last":0.80100000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.50900000000000001},"max":{"1min":0.877,"5min":1.361,"15min":1.3660000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94999999999999996,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.80700000000000005},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.054,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83199999999999996},"min":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93700000000000006,"5min":1.2170000000000001,"15min":1.298},"last":0.73699999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575077,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311176,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060664,"statfs":{"total":96636764160,"available":96318119936,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.433}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.85299999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.88},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.81299999999999994}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.94099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307080,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064760,"statfs":{"total":96636764160,"available":96322314240,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":0.56100000000000005,"5min":0.627,"15min":0.80100000000000005},"last":0.308},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46200000000000002,"15min":0.47899999999999998},"min":{"1min":0.307,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.81200000000000006},"last":0.35999999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":0.70099999999999996,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.376},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48599999999999999,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.31},"max":{"1min":0.57899999999999996,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.877,"15min":0.89700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.46000000000000002,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70599999999999996,"15min":0.74399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.98899999999999999,"5min":1.095,"15min":1.3029999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.413},"last":0.64700000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.76400000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.032,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.089},"last":0.60799999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.47999999999999998},"max":{"1min":1.0680000000000001,"5min":1.101,"15min":1.379},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.57799999999999996,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.486},"last":0.69299999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.78400000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.46500000000000002},"max":{"1min":1.0549999999999999,"5min":1.0840000000000001,"15min":1.389},"last":0.623},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.78500000000000003},"min":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.94899999999999995,"5min":1.054,"15min":1.4730000000000001},"last":0.72199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311164,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060676,"statfs":{"total":96636764160,"available":96318132224,"internally_reserved":0,"allocated":1626112,"data_stored":797550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.628},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.75},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.55800000000000005}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":0.80900000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.90500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.77200000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.93000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.45000000000000001,"15min":0.34699999999999998},"max":{"1min":0.89700000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.622}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.71099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":1.018,"5min":1.026,"15min":1.026},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81699999999999995,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68799999999999994,"15min":0.70999999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71099999999999997,"15min":0.73299999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.96599999999999997,"5min":1.026,"15min":1.026},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94199999999999995,"5min":1.004,"15min":1.004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92200000000000004,"5min":1.034,"15min":1.034},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.90900000000000003},"last":0.38},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0680000000000001},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.52800000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.58699999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.095},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.71599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92000000000000004,"5min":1.05,"15min":1.05},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:15:26.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:15:27.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.207+0000 7f5f45b9d700 1 Processor -- start 2023-10-08T01:15:27.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.207+0000 7f5f45b9d700 1 -- start start 2023-10-08T01:15:27.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.208+0000 7f5f45b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f40142ea0 0x7f5f400b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.208+0000 7f5f45b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5f4006c0f0 0x7f5f4006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.208+0000 7f5f45b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f400b40f0 con 0x7f5f400ad870 2023-10-08T01:15:27.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.208+0000 7f5f45b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f40069640 con 0x7f5f4006c0f0 2023-10-08T01:15:27.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.208+0000 7f5f45b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40069920 con 0x7f5f40142ea0 2023-10-08T01:15:27.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f3ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5f400ad870 0x7f5f4006b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42594/0 (socket says 172.21.15.119:42594) 2023-10-08T01:15:27.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f3ffff700 1 -- 172.21.15.119:0/1758308441 learned_addr learned my addr 172.21.15.119:0/1758308441 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:27.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f4539c700 1 --2- 172.21.15.119:0/1758308441 >> v2:172.21.15.184:3300/0 conn(0x7f5f4006c0f0 0x7f5f4006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f44b9b700 1 --2- 172.21.15.119:0/1758308441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f40142ea0 0x7f5f400b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f3f7fe700 1 -- 172.21.15.119:0/1758308441 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3731615771 0 0) 0x7f5f400b40f0 con 0x7f5f400ad870 2023-10-08T01:15:27.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f3f7fe700 1 -- 172.21.15.119:0/1758308441 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f30003570 con 0x7f5f400ad870 2023-10-08T01:15:27.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f4539c700 1 -- 172.21.15.119:0/1758308441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f40142ea0 msgr2=0x7f5f400b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:27.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f4539c700 1 --2- 172.21.15.119:0/1758308441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f40142ea0 0x7f5f400b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f4539c700 1 -- 172.21.15.119:0/1758308441 >> v1:172.21.15.119:6789/0 conn(0x7f5f400ad870 legacy=0x7f5f4006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.209+0000 7f5f4539c700 1 -- 172.21.15.119:0/1758308441 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f40069c00 con 0x7f5f4006c0f0 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.210+0000 7f5f4539c700 1 --2- 172.21.15.119:0/1758308441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f4006c0f0 0x7f5f4006a120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009e60 tx=0x7f5f38009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d246a00cec5452f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.210+0000 7f5f3f7fe700 1 -- 172.21.15.119:0/1758308441 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800e070 con 0x7f5f4006c0f0 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.210+0000 7f5f3f7fe700 1 -- 172.21.15.119:0/1758308441 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3800bab0 con 0x7f5f4006c0f0 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.210+0000 7f5f3f7fe700 1 -- 172.21.15.119:0/1758308441 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3800be70 con 0x7f5f4006c0f0 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.211+0000 7f5f45b9d700 1 -- 172.21.15.119:0/1758308441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f4006c0f0 msgr2=0x7f5f4006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.211+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/1758308441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f4006c0f0 0x7f5f4006a120 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009e60 tx=0x7f5f38009780 comp rx=0 tx=0).stop 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.211+0000 7f5f45b9d700 1 -- 172.21.15.119:0/1758308441 shutdown_connections 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.211+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/1758308441 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f4006c0f0 0x7f5f4006a120 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.211+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/1758308441 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f40142ea0 0x7f5f400b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.211+0000 7f5f45b9d700 1 -- 172.21.15.119:0/1758308441 >> 172.21.15.119:0/1758308441 conn(0x7f5f400ce430 msgr2=0x7f5f400ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:27.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.211+0000 7f5f45b9d700 1 -- 172.21.15.119:0/1758308441 shutdown_connections 2023-10-08T01:15:27.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.211+0000 7f5f45b9d700 1 -- 172.21.15.119:0/1758308441 wait complete. 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.212+0000 7f5f45b9d700 1 Processor -- start 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.212+0000 7f5f45b9d700 1 -- start start 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.213+0000 7f5f45b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f4006c0f0 0x7f5f4016aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.213+0000 7f5f45b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f40142ea0 0x7f5f4016a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.213+0000 7f5f45b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f4015eac0 0x7f5f40158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.213+0000 7f5f45b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400c85b0 con 0x7f5f4015eac0 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.213+0000 7f5f45b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400c9130 con 0x7f5f40142ea0 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.213+0000 7f5f45b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4016b1e0 con 0x7f5f4006c0f0 2023-10-08T01:15:27.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.213+0000 7f5f4539c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f4015eac0 0x7f5f40158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f4539c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f4015eac0 0x7f5f40158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42032/0 (socket says 172.21.15.119:42032) 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f4539c700 1 -- 172.21.15.119:0/3224612191 learned_addr learned my addr 172.21.15.119:0/3224612191 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f44b9b700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f4006c0f0 0x7f5f4016aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f3ffff700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f40142ea0 0x7f5f4016a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f4539c700 1 -- 172.21.15.119:0/3224612191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f4006c0f0 msgr2=0x7f5f4016aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f4539c700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f4006c0f0 0x7f5f4016aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f4539c700 1 -- 172.21.15.119:0/3224612191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f40142ea0 msgr2=0x7f5f4016a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f4539c700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f40142ea0 0x7f5f4016a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f4539c700 1 -- 172.21.15.119:0/3224612191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f38009a20 con 0x7f5f4015eac0 2023-10-08T01:15:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f4539c700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f4015eac0 0x7f5f40158f10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5f38003440 tx=0x7f5f38019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f3cff9700 1 -- 172.21.15.119:0/3224612191 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38027470 con 0x7f5f4015eac0 2023-10-08T01:15:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f3cff9700 1 -- 172.21.15.119:0/3224612191 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f380275f0 con 0x7f5f4015eac0 2023-10-08T01:15:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f3cff9700 1 -- 172.21.15.119:0/3224612191 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f380279b0 con 0x7f5f4015eac0 2023-10-08T01:15:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.214+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f400c8800 con 0x7f5f4015eac0 2023-10-08T01:15:27.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.215+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f400c8dc0 con 0x7f5f4015eac0 2023-10-08T01:15:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.216+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f400c92c0 con 0x7f5f4015eac0 2023-10-08T01:15:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.217+0000 7f5f3cff9700 1 -- 172.21.15.119:0/3224612191 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f38024070 con 0x7f5f4015eac0 2023-10-08T01:15:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.217+0000 7f5f3cff9700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5f24066f30 0x7f5f240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.218+0000 7f5f3cff9700 1 -- 172.21.15.119:0/3224612191 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f5f38087020 con 0x7f5f4015eac0 2023-10-08T01:15:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.218+0000 7f5f44b9b700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5f24066f30 0x7f5f240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.218+0000 7f5f44b9b700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5f24066f30 0x7f5f240693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009ca0 tx=0x7f5f34008040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:27.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.222+0000 7f5f3cff9700 1 -- 172.21.15.119:0/3224612191 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f380541a0 con 0x7f5f4015eac0 2023-10-08T01:15:27.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.411+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f4004c3c0 con 0x7f5f24066f30 2023-10-08T01:15:27.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.419+0000 7f5f3cff9700 1 -- 172.21.15.119:0/3224612191 <== mgr.7989 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278253 (secure 0 0 0) 0x7f5f4004c3c0 con 0x7f5f24066f30 2023-10-08T01:15:27.424 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:27.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5f24066f30 msgr2=0x7f5f240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5f24066f30 0x7f5f240693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5f34009ca0 tx=0x7f5f34008040 comp rx=0 tx=0).stop 2023-10-08T01:15:27.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f4015eac0 msgr2=0x7f5f40158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f4015eac0 0x7f5f40158f10 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f5f38003440 tx=0x7f5f38019000 comp rx=0 tx=0).stop 2023-10-08T01:15:27.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 shutdown_connections 2023-10-08T01:15:27.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5f24066f30 0x7f5f240693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f4006c0f0 0x7f5f4016aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f4015eac0 0x7f5f40158f10 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 --2- 172.21.15.119:0/3224612191 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f40142ea0 0x7f5f4016a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.423+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 >> 172.21.15.119:0/3224612191 conn(0x7f5f400ce430 msgr2=0x7f5f40141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:27.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.424+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 shutdown_connections 2023-10-08T01:15:27.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.424+0000 7f5f45b9d700 1 -- 172.21.15.119:0/3224612191 wait complete. 2023-10-08T01:15:27.429 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:15:27.442 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T01:15:25.693568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449596,"kb_used_data":13948,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752525124,"statfs":{"total":773094113280,"available":770585726976,"internally_reserved":0,"allocated":14282752,"data_stored":7604711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.955351"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745570+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:54.161881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747746+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:37.841133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747552+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:23:57.251999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747225+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:42:55.593104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.746581+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:51:03.891824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741037+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:46.116333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741530+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:09.044142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745461+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:17.975581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748200+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:31.074576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741127+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:06:36.574221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744161+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:52.124108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743824+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:10.573223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744443+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:11.456448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741651+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:04.750260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748170+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:28:56.803819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743801+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:42.670628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.748288+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:48.508166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.742969+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:30.277372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744476+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:52.427565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743998+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:48.508166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747232+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:45:48.338360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.743580+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:38:31.074576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744634+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:08.365749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741072+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:27:44.699900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741519+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:02:33.182686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745045+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:33.222269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741044+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:07.797723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.745954+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:29.185128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.741519+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:10:30.277372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.747177+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:54:45.165121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744088+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:37.703517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T01:15:20.737836+0000","last_change":"2023-10-08T01:15:20.737836+0000","last_active":"2023-10-08T01:15:20.737836+0000","last_peered":"2023-10-08T01:15:20.737836+0000","last_clean":"2023-10-08T01:15:20.737836+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:20.737836+0000","last_undegraded":"2023-10-08T01:15:20.737836+0000","last_fullsized":"2023-10-08T01:15:20.737836+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T01:15:19.731648+0000","last_change":"2023-10-08T01:15:19.744444+0000","last_active":"2023-10-08T01:15:19.731648+0000","last_peered":"2023-10-08T01:15:19.731648+0000","last_clean":"2023-10-08T01:15:19.731648+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T01:15:19.731648+0000","last_undegraded":"2023-10-08T01:15:19.731648+0000","last_fullsized":"2023-10-08T01:15:19.731648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:29.780863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"124'249","reported_seq":724,"reported_epoch":131,"state":"active+clean","last_fresh":"2023-10-08T01:15:17.234905+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:15:17.234905+0000","last_peered":"2023-10-08T01:15:17.234905+0000","last_clean":"2023-10-08T01:15:17.234905+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:15:17.234905+0000","last_undegraded":"2023-10-08T01:15:17.234905+0000","last_fullsized":"2023-10-08T01:15:17.234905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307660,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064180,"statfs":{"total":96636764160,"available":96321720320,"internally_reserved":0,"allocated":2232320,"data_stored":1394143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.123},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82999999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.67400000000000004}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.68799999999999994},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.85399999999999998,"5min":0.95499999999999996,"15min":0.97699999999999998},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39200000000000002},"max":{"1min":0.92200000000000004,"5min":0.93600000000000005,"15min":1.0269999999999999},"last":0.63300000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.87,"5min":1.133,"15min":1.133},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.66500000000000004,"15min":0.71399999999999997},"min":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.95099999999999996,"15min":1.167},"last":0.79100000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.69899999999999995},"min":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.93200000000000005,"15min":0.94699999999999995},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.501,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.052,"15min":1.079},"last":0.77400000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.5},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72499999999999998,"5min":0.875,"15min":0.88300000000000001},"last":0.51500000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.62},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84899999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82299999999999995,"5min":1.004,"15min":1.004},"last":0.55500000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86099999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.47099999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.75600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575078,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.93200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67800000000000005,"15min":0.69099999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91200000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70599999999999996,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":2.984},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38100000000000001},"max":{"1min":0.86799999999999999,"5min":1.004,"15min":3.0190000000000001},"last":0.65300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.95999999999999996,"5min":0.998,"15min":0.998},"last":0.47699999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72499999999999998,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71699999999999997,"15min":0.73299999999999998},"min":{"1min":0.56100000000000005,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.92400000000000004,"5min":1.048,"15min":1.048},"last":0.73899999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.88500000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.623},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79000000000000004,"5min":0.89800000000000002,"15min":1.087},"last":0.75900000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.395},"last":0.627},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.21},"last":0.61599999999999999}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":2.9489999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45200000000000001,"15min":0.46500000000000002},"min":{"1min":0.29999999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.56399999999999995}]},{"osd":2,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45900000000000002,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.61099999999999999,"5min":0.70599999999999996,"15min":0.72499999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.499}]},{"osd":3,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.81899999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":1.296,"15min":1.452},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.099,"5min":1.137,"15min":1.1839999999999999},"last":0.98599999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":1.119,"15min":1.3959999999999999},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.77700000000000002},"min":{"1min":0.53200000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.5209999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.56399999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.6020000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.77700000000000002},"min":{"1min":0.60899999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.046,"5min":1.1499999999999999,"15min":1.3660000000000001},"last":1.032}]},{"osd":7,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77400000000000002,"15min":0.81799999999999995},"min":{"1min":0.63500000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96699999999999997,"5min":1.236,"15min":1.621},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.51300000000000001},"max":{"1min":1.0389999999999999,"5min":1.21,"15min":1.4890000000000001},"last":0.91500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575077,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303564,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068276,"statfs":{"total":96636764160,"available":96325914624,"internally_reserved":0,"allocated":2232320,"data_stored":1394143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45500000000000002,"15min":0.46200000000000002},"min":{"1min":0.28199999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56499999999999995,"5min":0.95999999999999996,"15min":0.997},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.28599999999999998},"max":{"1min":0.69699999999999995,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.373}]},{"osd":2,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59099999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.31900000000000001}]},{"osd":3,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68200000000000005,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.497}]},{"osd":4,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.53600000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":1.006,"5min":1.341,"15min":1.617},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.55800000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88500000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.623}]},{"osd":5,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":2.085,"15min":2.085},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.014,"5min":1.381,"15min":1.5289999999999999},"last":0.80100000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.50900000000000001},"max":{"1min":0.877,"5min":1.361,"15min":1.3660000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94999999999999996,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.66500000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.80700000000000005},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.054,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83199999999999996},"min":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93700000000000006,"5min":1.2170000000000001,"15min":1.298},"last":0.73699999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575077,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311176,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060664,"statfs":{"total":96636764160,"available":96318119936,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.37}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.433}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.53100000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.85299999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.88},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.81299999999999994}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.94099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575077,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307080,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064760,"statfs":{"total":96636764160,"available":96322314240,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":0.56100000000000005,"5min":0.627,"15min":0.80100000000000005},"last":0.308},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46200000000000002,"15min":0.47899999999999998},"min":{"1min":0.307,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.81200000000000006},"last":0.35999999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":0.70099999999999996,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.376},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48599999999999999,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.31},"max":{"1min":0.57899999999999996,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.877,"15min":0.89700000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.46000000000000002,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.50700000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70599999999999996,"15min":0.74399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.98899999999999999,"5min":1.095,"15min":1.3029999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.413},"last":0.64700000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.76400000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.032,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.089},"last":0.60799999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.47999999999999998},"max":{"1min":1.0680000000000001,"5min":1.101,"15min":1.379},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.57799999999999996,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.486},"last":0.69299999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.78400000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.46500000000000002},"max":{"1min":1.0549999999999999,"5min":1.0840000000000001,"15min":1.389},"last":0.623},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.78500000000000003},"min":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.94899999999999995,"5min":1.054,"15min":1.4730000000000001},"last":0.72199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575077,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311164,"kb_used_data":1588,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060676,"statfs":{"total":96636764160,"available":96318132224,"internally_reserved":0,"allocated":1626112,"data_stored":797550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.628},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.75},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.55800000000000005}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":0.80900000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.90500000000000003}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.77200000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.85399999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.93000000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575077,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1600,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1638400,"data_stored":803775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.45000000000000001,"15min":0.34699999999999998},"max":{"1min":0.89700000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.622}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.71099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":1.018,"5min":1.026,"15min":1.026},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81699999999999995,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68799999999999994,"15min":0.70999999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71099999999999997,"15min":0.73299999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.96599999999999997,"5min":1.026,"15min":1.026},"last":0.59499999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94199999999999995,"5min":1.004,"15min":1.004},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92200000000000004,"5min":1.034,"15min":1.034},"last":0.56499999999999995}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.90900000000000003},"last":0.38},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0680000000000001},"last":0.41299999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.52800000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.58699999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.095},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.71599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92000000000000004,"5min":1.05,"15min":1.05},"last":0.68500000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:15:27.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2023-10-08T01:15:27.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2023-10-08T01:15:27.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2023-10-08T01:15:27.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2023-10-08T01:15:27.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2023-10-08T01:15:27.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2023-10-08T01:15:27.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2023-10-08T01:15:27.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2023-10-08T01:15:27.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2023-10-08T01:15:27.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2023-10-08T01:15:27.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2023-10-08T01:15:27.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2023-10-08T01:15:27.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2023-10-08T01:15:27.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2023-10-08T01:15:27.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2023-10-08T01:15:27.455 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2023-10-08T01:15:27.456 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.457 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2023-10-08T01:15:27.458 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.745570+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:50:54.161881+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.747746+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:51:37.841133+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.459 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.747552+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:23:57.251999+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.747225+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:42:55.593104+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-10-08T01:15:27.460 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.746581+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:51:03.891824+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.741037+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:43:46.116333+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.741530+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:30:09.044142+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:15:27.461 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.745461+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:31:17.975581+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.748200+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:38:31.074576+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2023-10-08T01:15:27.462 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.741127+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:06:36.574221+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.744161+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:57:52.124108+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.463 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.743824+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:01:10.573223+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.744443+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T01:23:11.456448+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2023-10-08T01:15:27.464 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.741651+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:34:04.750260+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.748170+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:28:56.803819+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.743801+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:51:42.670628+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:15:27.465 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.748288+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:02:48.508166+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.742969+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:10:30.277372+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2023-10-08T01:15:27.466 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-10-08T01:15:27.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.744476+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:58:52.427565+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:15:27.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2023-10-08T01:15:27.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2023-10-08T01:15:27.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.743998+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:02:48.508166+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:15:27.467 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-10-08T01:15:27.467 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.747232+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:45:48.338360+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.743580+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:38:31.074576+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.468 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.744634+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:29:08.365749+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.741072+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:27:44.699900+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2023-10-08T01:15:27.469 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.741519+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:02:33.182686+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.745045+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:15:33.222269+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.741044+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:19:07.797723+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T01:15:27.470 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.745954+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:21:29.185128+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.741519+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:10:30.277372+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2023-10-08T01:15:27.471 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.747177+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:54:45.165121+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.744088+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:46:37.703517+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T01:15:20.737836+0000', 'last_change': '2023-10-08T01:15:20.737836+0000', 'last_active': '2023-10-08T01:15:20.737836+0000', 'last_peered': '2023-10-08T01:15:20.737836+0000', 'last_clean': '2023-10-08T01:15:20.737836+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:20.737836+0000', 'last_undegraded': '2023-10-08T01:15:20.737836+0000', 'last_fullsized': '2023-10-08T01:15:20.737836+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:20.737836+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T01:15:27.472 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-10-08T01:15:27.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T01:15:19.731648+0000', 'last_change': '2023-10-08T01:15:19.744444+0000', 'last_active': '2023-10-08T01:15:19.731648+0000', 'last_peered': '2023-10-08T01:15:19.731648+0000', 'last_clean': '2023-10-08T01:15:19.731648+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T01:15:19.731648+0000', 'last_undegraded': '2023-10-08T01:15:19.731648+0000', 'last_fullsized': '2023-10-08T01:15:19.731648+0000', 'mapping_epoch': 132, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 132, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'last_clean_scrub_stamp': '2023-10-08T01:15:19.731648+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:37:29.780863+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T01:15:27.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T01:15:27.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.662+0000 7fc6667ff700 1 Processor -- start 2023-10-08T01:15:27.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.662+0000 7fc6667ff700 1 -- start start 2023-10-08T01:15:27.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.663+0000 7fc6667ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc660156510 0x7fc6600ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.663+0000 7fc6667ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6600b0ca0 0x7fc66015fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.663+0000 7fc6667ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6600ad1e0 con 0x7fc6600acd40 2023-10-08T01:15:27.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.663+0000 7fc6667ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6600ad9d0 con 0x7fc6600b0ca0 2023-10-08T01:15:27.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.663+0000 7fc6667ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6600aed00 con 0x7fc660156510 2023-10-08T01:15:27.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc664d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6600b0ca0 0x7fc66015fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc664d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6600b0ca0 0x7fc66015fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42046/0 (socket says 172.21.15.119:42046) 2023-10-08T01:15:27.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc664d9c700 1 -- 172.21.15.119:0/3910615404 learned_addr learned my addr 172.21.15.119:0/3910615404 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:27.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc65f7fe700 1 --1- 172.21.15.119:0/3910615404 >> v1:172.21.15.119:6789/0 conn(0x7fc6600acd40 0x7fc6601568e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42598/0 (socket says 172.21.15.119:42598) 2023-10-08T01:15:27.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc65ffff700 1 --2- 172.21.15.119:0/3910615404 >> v2:172.21.15.184:3300/0 conn(0x7fc660156510 0x7fc6600ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc664d9c700 1 -- 172.21.15.119:0/3910615404 >> v2:172.21.15.184:3300/0 conn(0x7fc660156510 msgr2=0x7fc6600ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:27.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc664d9c700 1 --2- 172.21.15.119:0/3910615404 >> v2:172.21.15.184:3300/0 conn(0x7fc660156510 0x7fc6600ac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc664d9c700 1 -- 172.21.15.119:0/3910615404 >> v1:172.21.15.119:6789/0 conn(0x7fc6600acd40 legacy=0x7fc6601568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc664d9c700 1 -- 172.21.15.119:0/3910615404 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660153f10 con 0x7fc6600b0ca0 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc664d9c700 1 --2- 172.21.15.119:0/3910615404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600b0ca0 0x7fc66015fad0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc6600cd640 tx=0x7fc658009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f8d4c5fcd29c8c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.664+0000 7fc65effd700 1 -- 172.21.15.119:0/3910615404 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6580092e0 con 0x7fc6600b0ca0 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc65effd700 1 -- 172.21.15.119:0/3910615404 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65800ba50 con 0x7fc6600b0ca0 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc65effd700 1 -- 172.21.15.119:0/3910615404 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65800be10 con 0x7fc6600b0ca0 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc6667ff700 1 -- 172.21.15.119:0/3910615404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600b0ca0 msgr2=0x7fc66015fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc6667ff700 1 --2- 172.21.15.119:0/3910615404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600b0ca0 0x7fc66015fad0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc6600cd640 tx=0x7fc658009840 comp rx=0 tx=0).stop 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc6667ff700 1 -- 172.21.15.119:0/3910615404 shutdown_connections 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc6667ff700 1 --2- 172.21.15.119:0/3910615404 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600b0ca0 0x7fc66015fad0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc6667ff700 1 --2- 172.21.15.119:0/3910615404 >> v2:172.21.15.184:3300/0 conn(0x7fc660156510 0x7fc6600ac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc6667ff700 1 -- 172.21.15.119:0/3910615404 >> 172.21.15.119:0/3910615404 conn(0x7fc66006a310 msgr2=0x7fc6600cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:27.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc6667ff700 1 -- 172.21.15.119:0/3910615404 shutdown_connections 2023-10-08T01:15:27.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.665+0000 7fc6667ff700 1 -- 172.21.15.119:0/3910615404 wait complete. 2023-10-08T01:15:27.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.667+0000 7fc6667ff700 1 Processor -- start 2023-10-08T01:15:27.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.667+0000 7fc6667ff700 1 -- start start 2023-10-08T01:15:27.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc6667ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600acd40 0x7fc6600ca760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc6667ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b0ca0 0x7fc66006ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc6667ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc660156510 0x7fc6600affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc6667ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6601458d0 con 0x7fc6600acd40 2023-10-08T01:15:27.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc6667ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600cba80 con 0x7fc660156510 2023-10-08T01:15:27.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc6667ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6600cae70 con 0x7fc6600b0ca0 2023-10-08T01:15:27.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600acd40 0x7fc6600ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600acd40 0x7fc6600ca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42056/0 (socket says 172.21.15.119:42056) 2023-10-08T01:15:27.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65ffff700 1 -- 172.21.15.119:0/514290340 learned_addr learned my addr 172.21.15.119:0/514290340 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:27.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc664d9c700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc660156510 0x7fc6600affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65f7fe700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b0ca0 0x7fc66006ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65ffff700 1 -- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b0ca0 msgr2=0x7fc66006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65ffff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b0ca0 0x7fc66006ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65ffff700 1 -- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc660156510 msgr2=0x7fc6600affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65ffff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc660156510 0x7fc6600affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc65ffff700 1 -- 172.21.15.119:0/514290340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc658009ae0 con 0x7fc6600acd40 2023-10-08T01:15:27.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.668+0000 7fc664d9c700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc660156510 0x7fc6600affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:27.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.669+0000 7fc65ffff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600acd40 0x7fc6600ca760 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d890 tx=0x7fc650008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:27.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.669+0000 7fc647fff700 1 -- 172.21.15.119:0/514290340 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000b600 con 0x7fc6600acd40 2023-10-08T01:15:27.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.669+0000 7fc647fff700 1 -- 172.21.15.119:0/514290340 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65000b760 con 0x7fc6600acd40 2023-10-08T01:15:27.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.669+0000 7fc647fff700 1 -- 172.21.15.119:0/514290340 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65000bae0 con 0x7fc6600acd40 2023-10-08T01:15:27.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.669+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6600cb150 con 0x7fc6600acd40 2023-10-08T01:15:27.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.669+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6600cb710 con 0x7fc6600acd40 2023-10-08T01:15:27.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.670+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66004c3c0 con 0x7fc6600acd40 2023-10-08T01:15:27.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.671+0000 7fc647fff700 1 -- 172.21.15.119:0/514290340 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc65001e460 con 0x7fc6600acd40 2023-10-08T01:15:27.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.672+0000 7fc647fff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc64c066f30 0x7fc64c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:27.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.672+0000 7fc647fff700 1 -- 172.21.15.119:0/514290340 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fc650084e20 con 0x7fc6600acd40 2023-10-08T01:15:27.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.672+0000 7fc65f7fe700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc64c066f30 0x7fc64c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:27.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.673+0000 7fc65f7fe700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc64c066f30 0x7fc64c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc654007fd0 tx=0x7fc654007d70 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:27.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.676+0000 7fc647fff700 1 -- 172.21.15.119:0/514290340 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc650051e50 con 0x7fc6600acd40 2023-10-08T01:15:27.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.940+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc6600c8c90 con 0x7fc6600acd40 2023-10-08T01:15:27.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.940+0000 7fc647fff700 1 -- 172.21.15.119:0/514290340 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1113 (secure 0 0 0) 0x7fc650055470 con 0x7fc6600acd40 2023-10-08T01:15:27.941 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:27.942 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1204,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":135,"num_osds":8,"num_up_osds":8,"osd_up_since":1696726535,"num_in_osds":8,"osd_in_since":1696726526,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2508386304,"bytes_avail":770585726976,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":63,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":8178}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-10-08T01:14:02.449356+0000","services":{}},"progress_events":{}} 2023-10-08T01:15:27.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc64c066f30 msgr2=0x7fc64c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc64c066f30 0x7fc64c0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc654007fd0 tx=0x7fc654007d70 comp rx=0 tx=0).stop 2023-10-08T01:15:27.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600acd40 msgr2=0x7fc6600ca760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:27.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600acd40 0x7fc6600ca760 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc65000d890 tx=0x7fc650008040 comp rx=0 tx=0).stop 2023-10-08T01:15:27.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 shutdown_connections 2023-10-08T01:15:27.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc64c066f30 0x7fc64c0693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc660156510 0x7fc6600affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6600acd40 0x7fc6600ca760 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 --2- 172.21.15.119:0/514290340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6600b0ca0 0x7fc66006ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:27.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 >> 172.21.15.119:0/514290340 conn(0x7fc66006a310 msgr2=0x7fc6601582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:27.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 shutdown_connections 2023-10-08T01:15:27.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:27.942+0000 7fc6667ff700 1 -- 172.21.15.119:0/514290340 wait complete. 2023-10-08T01:15:27.961 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T01:15:27.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:15:28.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.142+0000 7fd6b83ff700 1 Processor -- start 2023-10-08T01:15:28.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.142+0000 7fd6b83ff700 1 -- start start 2023-10-08T01:15:28.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.143+0000 7fd6b83ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd6b006c0a0 0x7fd6b00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.143+0000 7fd6b83ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b015d7e0 con 0x7fd6b006bcd0 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b83ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6b015dac0 con 0x7fd6b015df50 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b83ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6b015aa30 con 0x7fd6b006c0a0 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b599a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd6b006c0a0 0x7fd6b00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b619b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd6b006bcd0 0x7fd6b015a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42614/0 (socket says 172.21.15.119:42614) 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b599a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd6b006c0a0 0x7fd6b00caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42062/0 (socket says 172.21.15.119:42062) 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b619b700 1 -- 172.21.15.119:0/1214025301 learned_addr learned my addr 172.21.15.119:0/1214025301 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b599a700 1 -- 172.21.15.119:0/1214025301 >> v1:172.21.15.184:6789/0 conn(0x7fd6b015df50 legacy=0x7fd6b00c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b599a700 1 -- 172.21.15.119:0/1214025301 >> v1:172.21.15.119:6789/0 conn(0x7fd6b006bcd0 legacy=0x7fd6b015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:28.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b599a700 1 -- 172.21.15.119:0/1214025301 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b015ad10 con 0x7fd6b006c0a0 2023-10-08T01:15:28.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.144+0000 7fd6b5199700 1 -- 172.21.15.119:0/1214025301 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421684761 0 0) 0x7fd6b015d7e0 con 0x7fd6b006bcd0 2023-10-08T01:15:28.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b599a700 1 --2- 172.21.15.119:0/1214025301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b006c0a0 0x7fd6b00caa90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00098e0 tx=0x7fd6a00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31e856365173e2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:28.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b5199700 1 -- 172.21.15.119:0/1214025301 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0009e80 con 0x7fd6b006c0a0 2023-10-08T01:15:28.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b5199700 1 -- 172.21.15.119:0/1214025301 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a000b860 con 0x7fd6b006c0a0 2023-10-08T01:15:28.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b5199700 1 -- 172.21.15.119:0/1214025301 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a000bbe0 con 0x7fd6b006c0a0 2023-10-08T01:15:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b83ff700 1 -- 172.21.15.119:0/1214025301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b006c0a0 msgr2=0x7fd6b00caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b83ff700 1 --2- 172.21.15.119:0/1214025301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b006c0a0 0x7fd6b00caa90 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fd6a00098e0 tx=0x7fd6a00091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b83ff700 1 -- 172.21.15.119:0/1214025301 shutdown_connections 2023-10-08T01:15:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b83ff700 1 --2- 172.21.15.119:0/1214025301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b006c0a0 0x7fd6b00caa90 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b83ff700 1 -- 172.21.15.119:0/1214025301 >> 172.21.15.119:0/1214025301 conn(0x7fd6b00ce600 msgr2=0x7fd6b00cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b83ff700 1 -- 172.21.15.119:0/1214025301 shutdown_connections 2023-10-08T01:15:28.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.145+0000 7fd6b83ff700 1 -- 172.21.15.119:0/1214025301 wait complete. 2023-10-08T01:15:28.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.147+0000 7fd6b83ff700 1 Processor -- start 2023-10-08T01:15:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.147+0000 7fd6b83ff700 1 -- start start 2023-10-08T01:15:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b83ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b006bcd0 0x7fd6b0153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b83ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6b006c0a0 0x7fd6b0157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b83ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b015df50 0x7fd6b00b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b83ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00b3d70 con 0x7fd6b015df50 2023-10-08T01:15:28.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b83ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b01584e0 con 0x7fd6b006c0a0 2023-10-08T01:15:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b83ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b00b2650 con 0x7fd6b006bcd0 2023-10-08T01:15:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b015df50 0x7fd6b00b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b015df50 0x7fd6b00b2110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42064/0 (socket says 172.21.15.119:42064) 2023-10-08T01:15:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b699c700 1 -- 172.21.15.119:0/352070266 learned_addr learned my addr 172.21.15.119:0/352070266 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b599a700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6b006c0a0 0x7fd6b0157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:28.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b619b700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b006bcd0 0x7fd6b0153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b699c700 1 -- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b006bcd0 msgr2=0x7fd6b0153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b699c700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b006bcd0 0x7fd6b0153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b699c700 1 -- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6b006c0a0 msgr2=0x7fd6b0157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b699c700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6b006c0a0 0x7fd6b0157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b699c700 1 -- 172.21.15.119:0/352070266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a0009420 con 0x7fd6b015df50 2023-10-08T01:15:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b599a700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6b006c0a0 0x7fd6b0157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.148+0000 7fd6b619b700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b006bcd0 0x7fd6b0153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:28.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.149+0000 7fd6b699c700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b015df50 0x7fd6b00b2110 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c840 tx=0x7fd6a8001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:28.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.149+0000 7fd6a67fc700 1 -- 172.21.15.119:0/352070266 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8012630 con 0x7fd6b015df50 2023-10-08T01:15:28.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.149+0000 7fd6a67fc700 1 -- 172.21.15.119:0/352070266 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a80127b0 con 0x7fd6b015df50 2023-10-08T01:15:28.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.149+0000 7fd6a67fc700 1 -- 172.21.15.119:0/352070266 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8012b70 con 0x7fd6b015df50 2023-10-08T01:15:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.149+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b0154020 con 0x7fd6b015df50 2023-10-08T01:15:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.149+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b00b3990 con 0x7fd6b015df50 2023-10-08T01:15:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.150+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b00b3eb0 con 0x7fd6b015df50 2023-10-08T01:15:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.151+0000 7fd6a67fc700 1 -- 172.21.15.119:0/352070266 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6a8003030 con 0x7fd6b015df50 2023-10-08T01:15:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.152+0000 7fd6a67fc700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd694066f30 0x7fd6940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.152+0000 7fd6a67fc700 1 -- 172.21.15.119:0/352070266 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fd6a8084640 con 0x7fd6b015df50 2023-10-08T01:15:28.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.152+0000 7fd6b619b700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd694066f30 0x7fd6940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.152+0000 7fd6b619b700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd694066f30 0x7fd6940693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000c00 tx=0x7fd6ac009040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:28.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.155+0000 7fd6a67fc700 1 -- 172.21.15.119:0/352070266 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6b00b3eb0 con 0x7fd6b015df50 2023-10-08T01:15:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.344+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6b004c430 con 0x7fd694066f30 2023-10-08T01:15:28.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.353+0000 7fd6a67fc700 1 -- 172.21.15.119:0/352070266 <== mgr.7989 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289377 (secure 0 0 0) 0x7fd6b004c430 con 0x7fd694066f30 2023-10-08T01:15:28.357 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:28.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd694066f30 msgr2=0x7fd6940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:28.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd694066f30 0x7fd6940693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac000c00 tx=0x7fd6ac009040 comp rx=0 tx=0).stop 2023-10-08T01:15:28.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b015df50 msgr2=0x7fd6b00b2110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:28.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b015df50 0x7fd6b00b2110 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c840 tx=0x7fd6a8001670 comp rx=0 tx=0).stop 2023-10-08T01:15:28.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 shutdown_connections 2023-10-08T01:15:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd694066f30 0x7fd6940693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd6b006bcd0 0x7fd6b0153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6b015df50 0x7fd6b00b2110 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 --2- 172.21.15.119:0/352070266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd6b006c0a0 0x7fd6b0157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 >> 172.21.15.119:0/352070266 conn(0x7fd6b00ce600 msgr2=0x7fd6b015a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 shutdown_connections 2023-10-08T01:15:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:28.357+0000 7fd6b83ff700 1 -- 172.21.15.119:0/352070266 wait complete. 2023-10-08T01:15:28.362 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:15:28.376 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T01:15:27.693897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":443,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":287,"ondisk_log_size":287,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449960,"kb_used_data":14312,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524760,"statfs":{"total":773094113280,"available":770585354240,"internally_reserved":0,"allocated":14655488,"data_stored":7780140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.950300"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.790793+0000","last_change":"2023-10-08T01:15:21.782424+0000","last_active":"2023-10-08T01:15:21.790793+0000","last_peered":"2023-10-08T01:15:21.790793+0000","last_clean":"2023-10-08T01:15:21.790793+0000","last_became_active":"2023-10-08T01:15:21.781981+0000","last_became_peered":"2023-10-08T01:15:21.781981+0000","last_unstale":"2023-10-08T01:15:21.790793+0000","last_undegraded":"2023-10-08T01:15:21.790793+0000","last_fullsized":"2023-10-08T01:15:21.790793+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:11:46.360472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789195+0000","last_change":"2023-10-08T01:15:21.782924+0000","last_active":"2023-10-08T01:15:21.789195+0000","last_peered":"2023-10-08T01:15:21.789195+0000","last_clean":"2023-10-08T01:15:21.789195+0000","last_became_active":"2023-10-08T01:15:21.782617+0000","last_became_peered":"2023-10-08T01:15:21.782617+0000","last_unstale":"2023-10-08T01:15:21.789195+0000","last_undegraded":"2023-10-08T01:15:21.789195+0000","last_fullsized":"2023-10-08T01:15:21.789195+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:37.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.794644+0000","last_change":"2023-10-08T01:15:21.791173+0000","last_active":"2023-10-08T01:15:21.794644+0000","last_peered":"2023-10-08T01:15:21.794644+0000","last_clean":"2023-10-08T01:15:21.794644+0000","last_became_active":"2023-10-08T01:15:21.790924+0000","last_became_peered":"2023-10-08T01:15:21.790924+0000","last_unstale":"2023-10-08T01:15:21.794644+0000","last_undegraded":"2023-10-08T01:15:21.794644+0000","last_fullsized":"2023-10-08T01:15:21.794644+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:18.380162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.794955+0000","last_change":"2023-10-08T01:15:21.788960+0000","last_active":"2023-10-08T01:15:21.794955+0000","last_peered":"2023-10-08T01:15:21.794955+0000","last_clean":"2023-10-08T01:15:21.794955+0000","last_became_active":"2023-10-08T01:15:21.787967+0000","last_became_peered":"2023-10-08T01:15:21.787967+0000","last_unstale":"2023-10-08T01:15:21.794955+0000","last_undegraded":"2023-10-08T01:15:21.794955+0000","last_fullsized":"2023-10-08T01:15:21.794955+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:26.259319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.786910+0000","last_change":"2023-10-08T01:15:21.786282+0000","last_active":"2023-10-08T01:15:21.786910+0000","last_peered":"2023-10-08T01:15:21.786910+0000","last_clean":"2023-10-08T01:15:21.786910+0000","last_became_active":"2023-10-08T01:15:21.786015+0000","last_became_peered":"2023-10-08T01:15:21.786015+0000","last_unstale":"2023-10-08T01:15:21.786910+0000","last_undegraded":"2023-10-08T01:15:21.786910+0000","last_fullsized":"2023-10-08T01:15:21.786910+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:30.787052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.783095+0000","last_change":"2023-10-08T01:15:21.780783+0000","last_active":"2023-10-08T01:15:21.783095+0000","last_peered":"2023-10-08T01:15:21.783095+0000","last_clean":"2023-10-08T01:15:21.783095+0000","last_became_active":"2023-10-08T01:15:21.780126+0000","last_became_peered":"2023-10-08T01:15:21.780126+0000","last_unstale":"2023-10-08T01:15:21.783095+0000","last_undegraded":"2023-10-08T01:15:21.783095+0000","last_fullsized":"2023-10-08T01:15:21.783095+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:16.256609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797806+0000","last_change":"2023-10-08T01:15:21.789868+0000","last_active":"2023-10-08T01:15:21.797806+0000","last_peered":"2023-10-08T01:15:21.797806+0000","last_clean":"2023-10-08T01:15:21.797806+0000","last_became_active":"2023-10-08T01:15:21.789605+0000","last_became_peered":"2023-10-08T01:15:21.789605+0000","last_unstale":"2023-10-08T01:15:21.797806+0000","last_undegraded":"2023-10-08T01:15:21.797806+0000","last_fullsized":"2023-10-08T01:15:21.797806+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:47.122521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.784403+0000","last_change":"2023-10-08T01:15:21.782274+0000","last_active":"2023-10-08T01:15:21.784403+0000","last_peered":"2023-10-08T01:15:21.784403+0000","last_clean":"2023-10-08T01:15:21.784403+0000","last_became_active":"2023-10-08T01:15:21.782014+0000","last_became_peered":"2023-10-08T01:15:21.782014+0000","last_unstale":"2023-10-08T01:15:21.784403+0000","last_undegraded":"2023-10-08T01:15:21.784403+0000","last_fullsized":"2023-10-08T01:15:21.784403+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:58.057009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.790774+0000","last_change":"2023-10-08T01:15:21.780528+0000","last_active":"2023-10-08T01:15:21.790774+0000","last_peered":"2023-10-08T01:15:21.790774+0000","last_clean":"2023-10-08T01:15:21.790774+0000","last_became_active":"2023-10-08T01:15:21.780257+0000","last_became_peered":"2023-10-08T01:15:21.780257+0000","last_unstale":"2023-10-08T01:15:21.790774+0000","last_undegraded":"2023-10-08T01:15:21.790774+0000","last_fullsized":"2023-10-08T01:15:21.790774+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:56.993744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.787497+0000","last_change":"2023-10-08T01:15:21.775199+0000","last_active":"2023-10-08T01:15:21.787497+0000","last_peered":"2023-10-08T01:15:21.787497+0000","last_clean":"2023-10-08T01:15:21.787497+0000","last_became_active":"2023-10-08T01:15:21.774975+0000","last_became_peered":"2023-10-08T01:15:21.774975+0000","last_unstale":"2023-10-08T01:15:21.787497+0000","last_undegraded":"2023-10-08T01:15:21.787497+0000","last_fullsized":"2023-10-08T01:15:21.787497+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:44.231188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796706+0000","last_change":"2023-10-08T01:15:21.790414+0000","last_active":"2023-10-08T01:15:21.796706+0000","last_peered":"2023-10-08T01:15:21.796706+0000","last_clean":"2023-10-08T01:15:21.796706+0000","last_became_active":"2023-10-08T01:15:21.790139+0000","last_became_peered":"2023-10-08T01:15:21.790139+0000","last_unstale":"2023-10-08T01:15:21.796706+0000","last_undegraded":"2023-10-08T01:15:21.796706+0000","last_fullsized":"2023-10-08T01:15:21.796706+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:38.709705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.784656+0000","last_change":"2023-10-08T01:15:21.774368+0000","last_active":"2023-10-08T01:15:21.784656+0000","last_peered":"2023-10-08T01:15:21.784656+0000","last_clean":"2023-10-08T01:15:21.784656+0000","last_became_active":"2023-10-08T01:15:21.774093+0000","last_became_peered":"2023-10-08T01:15:21.774093+0000","last_unstale":"2023-10-08T01:15:21.784656+0000","last_undegraded":"2023-10-08T01:15:21.784656+0000","last_fullsized":"2023-10-08T01:15:21.784656+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:57.415376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.795304+0000","last_change":"2023-10-08T01:15:21.790265+0000","last_active":"2023-10-08T01:15:21.795304+0000","last_peered":"2023-10-08T01:15:21.795304+0000","last_clean":"2023-10-08T01:15:21.795304+0000","last_became_active":"2023-10-08T01:15:21.790000+0000","last_became_peered":"2023-10-08T01:15:21.790000+0000","last_unstale":"2023-10-08T01:15:21.795304+0000","last_undegraded":"2023-10-08T01:15:21.795304+0000","last_fullsized":"2023-10-08T01:15:21.795304+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:18.981769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792176+0000","last_change":"2023-10-08T01:15:21.788126+0000","last_active":"2023-10-08T01:15:21.792176+0000","last_peered":"2023-10-08T01:15:21.792176+0000","last_clean":"2023-10-08T01:15:21.792176+0000","last_became_active":"2023-10-08T01:15:21.787794+0000","last_became_peered":"2023-10-08T01:15:21.787794+0000","last_unstale":"2023-10-08T01:15:21.792176+0000","last_undegraded":"2023-10-08T01:15:21.792176+0000","last_fullsized":"2023-10-08T01:15:21.792176+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:33.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.782492+0000","last_change":"2023-10-08T01:15:21.779837+0000","last_active":"2023-10-08T01:15:21.782492+0000","last_peered":"2023-10-08T01:15:21.782492+0000","last_clean":"2023-10-08T01:15:21.782492+0000","last_became_active":"2023-10-08T01:15:21.779599+0000","last_became_peered":"2023-10-08T01:15:21.779599+0000","last_unstale":"2023-10-08T01:15:21.782492+0000","last_undegraded":"2023-10-08T01:15:21.782492+0000","last_fullsized":"2023-10-08T01:15:21.782492+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:30.191316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789531+0000","last_change":"2023-10-08T01:15:21.782077+0000","last_active":"2023-10-08T01:15:21.789531+0000","last_peered":"2023-10-08T01:15:21.789531+0000","last_clean":"2023-10-08T01:15:21.789531+0000","last_became_active":"2023-10-08T01:15:21.781800+0000","last_became_peered":"2023-10-08T01:15:21.781800+0000","last_unstale":"2023-10-08T01:15:21.789531+0000","last_undegraded":"2023-10-08T01:15:21.789531+0000","last_fullsized":"2023-10-08T01:15:21.789531+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:08.359663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797541+0000","last_change":"2023-10-08T01:15:21.789809+0000","last_active":"2023-10-08T01:15:21.797541+0000","last_peered":"2023-10-08T01:15:21.797541+0000","last_clean":"2023-10-08T01:15:21.797541+0000","last_became_active":"2023-10-08T01:15:21.789454+0000","last_became_peered":"2023-10-08T01:15:21.789454+0000","last_unstale":"2023-10-08T01:15:21.797541+0000","last_undegraded":"2023-10-08T01:15:21.797541+0000","last_fullsized":"2023-10-08T01:15:21.797541+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.786486+0000","last_change":"2023-10-08T01:15:21.784309+0000","last_active":"2023-10-08T01:15:21.786486+0000","last_peered":"2023-10-08T01:15:21.786486+0000","last_clean":"2023-10-08T01:15:21.786486+0000","last_became_active":"2023-10-08T01:15:21.784045+0000","last_became_peered":"2023-10-08T01:15:21.784045+0000","last_unstale":"2023-10-08T01:15:21.786486+0000","last_undegraded":"2023-10-08T01:15:21.786486+0000","last_fullsized":"2023-10-08T01:15:21.786486+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:55.362748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792101+0000","last_change":"2023-10-08T01:15:21.787542+0000","last_active":"2023-10-08T01:15:21.792101+0000","last_peered":"2023-10-08T01:15:21.792101+0000","last_clean":"2023-10-08T01:15:21.792101+0000","last_became_active":"2023-10-08T01:15:21.786844+0000","last_became_peered":"2023-10-08T01:15:21.786844+0000","last_unstale":"2023-10-08T01:15:21.792101+0000","last_undegraded":"2023-10-08T01:15:21.792101+0000","last_fullsized":"2023-10-08T01:15:21.792101+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:03.398793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796769+0000","last_change":"2023-10-08T01:15:21.788914+0000","last_active":"2023-10-08T01:15:21.796769+0000","last_peered":"2023-10-08T01:15:21.796769+0000","last_clean":"2023-10-08T01:15:21.796769+0000","last_became_active":"2023-10-08T01:15:21.788624+0000","last_became_peered":"2023-10-08T01:15:21.788624+0000","last_unstale":"2023-10-08T01:15:21.796769+0000","last_undegraded":"2023-10-08T01:15:21.796769+0000","last_fullsized":"2023-10-08T01:15:21.796769+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:33.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789221+0000","last_change":"2023-10-08T01:15:21.783160+0000","last_active":"2023-10-08T01:15:21.789221+0000","last_peered":"2023-10-08T01:15:21.789221+0000","last_clean":"2023-10-08T01:15:21.789221+0000","last_became_active":"2023-10-08T01:15:21.782407+0000","last_became_peered":"2023-10-08T01:15:21.782407+0000","last_unstale":"2023-10-08T01:15:21.789221+0000","last_undegraded":"2023-10-08T01:15:21.789221+0000","last_fullsized":"2023-10-08T01:15:21.789221+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:34.228457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796873+0000","last_change":"2023-10-08T01:15:21.795149+0000","last_active":"2023-10-08T01:15:21.796873+0000","last_peered":"2023-10-08T01:15:21.796873+0000","last_clean":"2023-10-08T01:15:21.796873+0000","last_became_active":"2023-10-08T01:15:21.794884+0000","last_became_peered":"2023-10-08T01:15:21.794884+0000","last_unstale":"2023-10-08T01:15:21.796873+0000","last_undegraded":"2023-10-08T01:15:21.796873+0000","last_fullsized":"2023-10-08T01:15:21.796873+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:02.399034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789147+0000","last_change":"2023-10-08T01:15:21.782996+0000","last_active":"2023-10-08T01:15:21.789147+0000","last_peered":"2023-10-08T01:15:21.789147+0000","last_clean":"2023-10-08T01:15:21.789147+0000","last_became_active":"2023-10-08T01:15:21.782728+0000","last_became_peered":"2023-10-08T01:15:21.782728+0000","last_unstale":"2023-10-08T01:15:21.789147+0000","last_undegraded":"2023-10-08T01:15:21.789147+0000","last_fullsized":"2023-10-08T01:15:21.789147+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:43.676817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792512+0000","last_change":"2023-10-08T01:15:21.785571+0000","last_active":"2023-10-08T01:15:21.792512+0000","last_peered":"2023-10-08T01:15:21.792512+0000","last_clean":"2023-10-08T01:15:21.792512+0000","last_became_active":"2023-10-08T01:15:21.785133+0000","last_became_peered":"2023-10-08T01:15:21.785133+0000","last_unstale":"2023-10-08T01:15:21.792512+0000","last_undegraded":"2023-10-08T01:15:21.792512+0000","last_fullsized":"2023-10-08T01:15:21.792512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:47.258669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797516+0000","last_change":"2023-10-08T01:15:21.787246+0000","last_active":"2023-10-08T01:15:21.797516+0000","last_peered":"2023-10-08T01:15:21.797516+0000","last_clean":"2023-10-08T01:15:21.797516+0000","last_became_active":"2023-10-08T01:15:21.786907+0000","last_became_peered":"2023-10-08T01:15:21.786907+0000","last_unstale":"2023-10-08T01:15:21.797516+0000","last_undegraded":"2023-10-08T01:15:21.797516+0000","last_fullsized":"2023-10-08T01:15:21.797516+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:11.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789677+0000","last_change":"2023-10-08T01:15:21.781323+0000","last_active":"2023-10-08T01:15:21.789677+0000","last_peered":"2023-10-08T01:15:21.789677+0000","last_clean":"2023-10-08T01:15:21.789677+0000","last_became_active":"2023-10-08T01:15:21.780917+0000","last_became_peered":"2023-10-08T01:15:21.780917+0000","last_unstale":"2023-10-08T01:15:21.789677+0000","last_undegraded":"2023-10-08T01:15:21.789677+0000","last_fullsized":"2023-10-08T01:15:21.789677+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:54:39.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.790237+0000","last_change":"2023-10-08T01:15:21.786972+0000","last_active":"2023-10-08T01:15:21.790237+0000","last_peered":"2023-10-08T01:15:21.790237+0000","last_clean":"2023-10-08T01:15:21.790237+0000","last_became_active":"2023-10-08T01:15:21.786481+0000","last_became_peered":"2023-10-08T01:15:21.786481+0000","last_unstale":"2023-10-08T01:15:21.790237+0000","last_undegraded":"2023-10-08T01:15:21.790237+0000","last_fullsized":"2023-10-08T01:15:21.790237+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:30.347688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.795879+0000","last_change":"2023-10-08T01:15:21.790335+0000","last_active":"2023-10-08T01:15:21.795879+0000","last_peered":"2023-10-08T01:15:21.795879+0000","last_clean":"2023-10-08T01:15:21.795879+0000","last_became_active":"2023-10-08T01:15:21.790073+0000","last_became_peered":"2023-10-08T01:15:21.790073+0000","last_unstale":"2023-10-08T01:15:21.795879+0000","last_undegraded":"2023-10-08T01:15:21.795879+0000","last_fullsized":"2023-10-08T01:15:21.795879+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:33.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.793099+0000","last_change":"2023-10-08T01:15:21.788246+0000","last_active":"2023-10-08T01:15:21.793099+0000","last_peered":"2023-10-08T01:15:21.793099+0000","last_clean":"2023-10-08T01:15:21.793099+0000","last_became_active":"2023-10-08T01:15:21.787987+0000","last_became_peered":"2023-10-08T01:15:21.787987+0000","last_unstale":"2023-10-08T01:15:21.793099+0000","last_undegraded":"2023-10-08T01:15:21.793099+0000","last_fullsized":"2023-10-08T01:15:21.793099+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:02.399034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.794373+0000","last_change":"2023-10-08T01:15:21.781441+0000","last_active":"2023-10-08T01:15:21.794373+0000","last_peered":"2023-10-08T01:15:21.794373+0000","last_clean":"2023-10-08T01:15:21.794373+0000","last_became_active":"2023-10-08T01:15:21.781187+0000","last_became_peered":"2023-10-08T01:15:21.781187+0000","last_unstale":"2023-10-08T01:15:21.794373+0000","last_undegraded":"2023-10-08T01:15:21.794373+0000","last_fullsized":"2023-10-08T01:15:21.794373+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:06:37.580409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789503+0000","last_change":"2023-10-08T01:15:21.781599+0000","last_active":"2023-10-08T01:15:21.789503+0000","last_peered":"2023-10-08T01:15:21.789503+0000","last_clean":"2023-10-08T01:15:21.789503+0000","last_became_active":"2023-10-08T01:15:21.781316+0000","last_became_peered":"2023-10-08T01:15:21.781316+0000","last_unstale":"2023-10-08T01:15:21.789503+0000","last_undegraded":"2023-10-08T01:15:21.789503+0000","last_fullsized":"2023-10-08T01:15:21.789503+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:55.237677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.782546+0000","last_change":"2023-10-08T01:15:21.779332+0000","last_active":"2023-10-08T01:15:21.782546+0000","last_peered":"2023-10-08T01:15:21.782546+0000","last_clean":"2023-10-08T01:15:21.782546+0000","last_became_active":"2023-10-08T01:15:21.779082+0000","last_became_peered":"2023-10-08T01:15:21.779082+0000","last_unstale":"2023-10-08T01:15:21.782546+0000","last_undegraded":"2023-10-08T01:15:21.782546+0000","last_fullsized":"2023-10-08T01:15:21.782546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:43:18.530363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.788532+0000","last_change":"2023-10-08T01:15:21.783048+0000","last_active":"2023-10-08T01:15:21.788532+0000","last_peered":"2023-10-08T01:15:21.788532+0000","last_clean":"2023-10-08T01:15:21.788532+0000","last_became_active":"2023-10-08T01:15:21.782740+0000","last_became_peered":"2023-10-08T01:15:21.782740+0000","last_unstale":"2023-10-08T01:15:21.788532+0000","last_undegraded":"2023-10-08T01:15:21.788532+0000","last_fullsized":"2023-10-08T01:15:21.788532+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:55.168069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"134'4","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.798197+0000","last_change":"2023-10-08T01:15:20.759292+0000","last_active":"2023-10-08T01:15:21.798197+0000","last_peered":"2023-10-08T01:15:21.798197+0000","last_clean":"2023-10-08T01:15:21.798197+0000","last_became_active":"2023-10-08T01:15:20.759026+0000","last_became_peered":"2023-10-08T01:15:20.759026+0000","last_unstale":"2023-10-08T01:15:21.798197+0000","last_undegraded":"2023-10-08T01:15:21.798197+0000","last_fullsized":"2023-10-08T01:15:21.798197+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:16.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796130+0000","last_change":"2023-10-08T01:15:21.789046+0000","last_active":"2023-10-08T01:15:21.796130+0000","last_peered":"2023-10-08T01:15:21.796130+0000","last_clean":"2023-10-08T01:15:21.796130+0000","last_became_active":"2023-10-08T01:15:21.788707+0000","last_became_peered":"2023-10-08T01:15:21.788707+0000","last_unstale":"2023-10-08T01:15:21.796130+0000","last_undegraded":"2023-10-08T01:15:21.796130+0000","last_fullsized":"2023-10-08T01:15:21.796130+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:08:10.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799550+0000","last_change":"2023-10-08T01:15:20.760884+0000","last_active":"2023-10-08T01:15:21.799550+0000","last_peered":"2023-10-08T01:15:21.799550+0000","last_clean":"2023-10-08T01:15:21.799550+0000","last_became_active":"2023-10-08T01:15:20.760631+0000","last_became_peered":"2023-10-08T01:15:20.760631+0000","last_unstale":"2023-10-08T01:15:21.799550+0000","last_undegraded":"2023-10-08T01:15:21.799550+0000","last_fullsized":"2023-10-08T01:15:21.799550+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:47.347739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.787025+0000","last_change":"2023-10-08T01:15:21.782815+0000","last_active":"2023-10-08T01:15:21.787025+0000","last_peered":"2023-10-08T01:15:21.787025+0000","last_clean":"2023-10-08T01:15:21.787025+0000","last_became_active":"2023-10-08T01:15:21.782500+0000","last_became_peered":"2023-10-08T01:15:21.782500+0000","last_unstale":"2023-10-08T01:15:21.787025+0000","last_undegraded":"2023-10-08T01:15:21.787025+0000","last_fullsized":"2023-10-08T01:15:21.787025+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:51:04.898012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797869+0000","last_change":"2023-10-08T01:15:20.765142+0000","last_active":"2023-10-08T01:15:21.797869+0000","last_peered":"2023-10-08T01:15:21.797869+0000","last_clean":"2023-10-08T01:15:21.797869+0000","last_became_active":"2023-10-08T01:15:20.764874+0000","last_became_peered":"2023-10-08T01:15:20.764874+0000","last_unstale":"2023-10-08T01:15:21.797869+0000","last_undegraded":"2023-10-08T01:15:21.797869+0000","last_fullsized":"2023-10-08T01:15:21.797869+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:21:26.740469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792120+0000","last_change":"2023-10-08T01:15:21.788245+0000","last_active":"2023-10-08T01:15:21.792120+0000","last_peered":"2023-10-08T01:15:21.792120+0000","last_clean":"2023-10-08T01:15:21.792120+0000","last_became_active":"2023-10-08T01:15:21.787939+0000","last_became_peered":"2023-10-08T01:15:21.787939+0000","last_unstale":"2023-10-08T01:15:21.792120+0000","last_undegraded":"2023-10-08T01:15:21.792120+0000","last_fullsized":"2023-10-08T01:15:21.792120+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:29:47.511265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796917+0000","last_change":"2023-10-08T01:15:20.766881+0000","last_active":"2023-10-08T01:15:21.796917+0000","last_peered":"2023-10-08T01:15:21.796917+0000","last_clean":"2023-10-08T01:15:21.796917+0000","last_became_active":"2023-10-08T01:15:20.766569+0000","last_became_peered":"2023-10-08T01:15:20.766569+0000","last_unstale":"2023-10-08T01:15:21.796917+0000","last_undegraded":"2023-10-08T01:15:21.796917+0000","last_fullsized":"2023-10-08T01:15:21.796917+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:22.379487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792453+0000","last_change":"2023-10-08T01:15:21.781168+0000","last_active":"2023-10-08T01:15:21.792453+0000","last_peered":"2023-10-08T01:15:21.792453+0000","last_clean":"2023-10-08T01:15:21.792453+0000","last_became_active":"2023-10-08T01:15:21.780884+0000","last_became_peered":"2023-10-08T01:15:21.780884+0000","last_unstale":"2023-10-08T01:15:21.792453+0000","last_undegraded":"2023-10-08T01:15:21.792453+0000","last_fullsized":"2023-10-08T01:15:21.792453+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:41:44.233312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797819+0000","last_change":"2023-10-08T01:15:20.766640+0000","last_active":"2023-10-08T01:15:21.797819+0000","last_peered":"2023-10-08T01:15:21.797819+0000","last_clean":"2023-10-08T01:15:21.797819+0000","last_became_active":"2023-10-08T01:15:20.766421+0000","last_became_peered":"2023-10-08T01:15:20.766421+0000","last_unstale":"2023-10-08T01:15:21.797819+0000","last_undegraded":"2023-10-08T01:15:21.797819+0000","last_fullsized":"2023-10-08T01:15:21.797819+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:47.902204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789239+0000","last_change":"2023-10-08T01:15:21.783360+0000","last_active":"2023-10-08T01:15:21.789239+0000","last_peered":"2023-10-08T01:15:21.789239+0000","last_clean":"2023-10-08T01:15:21.789239+0000","last_became_active":"2023-10-08T01:15:21.782826+0000","last_became_peered":"2023-10-08T01:15:21.782826+0000","last_unstale":"2023-10-08T01:15:21.789239+0000","last_undegraded":"2023-10-08T01:15:21.789239+0000","last_fullsized":"2023-10-08T01:15:21.789239+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:18.380162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797702+0000","last_change":"2023-10-08T01:15:20.758281+0000","last_active":"2023-10-08T01:15:21.797702+0000","last_peered":"2023-10-08T01:15:21.797702+0000","last_clean":"2023-10-08T01:15:21.797702+0000","last_became_active":"2023-10-08T01:15:20.757981+0000","last_became_peered":"2023-10-08T01:15:20.757981+0000","last_unstale":"2023-10-08T01:15:21.797702+0000","last_undegraded":"2023-10-08T01:15:21.797702+0000","last_fullsized":"2023-10-08T01:15:21.797702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:47.347739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.787062+0000","last_change":"2023-10-08T01:15:21.776121+0000","last_active":"2023-10-08T01:15:21.787062+0000","last_peered":"2023-10-08T01:15:21.787062+0000","last_clean":"2023-10-08T01:15:21.787062+0000","last_became_active":"2023-10-08T01:15:21.775812+0000","last_became_peered":"2023-10-08T01:15:21.775812+0000","last_unstale":"2023-10-08T01:15:21.787062+0000","last_undegraded":"2023-10-08T01:15:21.787062+0000","last_fullsized":"2023-10-08T01:15:21.787062+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:06.156436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.770109+0000","last_change":"2023-10-08T01:15:20.754017+0000","last_active":"2023-10-08T01:15:21.770109+0000","last_peered":"2023-10-08T01:15:21.770109+0000","last_clean":"2023-10-08T01:15:21.770109+0000","last_became_active":"2023-10-08T01:15:20.753760+0000","last_became_peered":"2023-10-08T01:15:20.753760+0000","last_unstale":"2023-10-08T01:15:21.770109+0000","last_undegraded":"2023-10-08T01:15:21.770109+0000","last_fullsized":"2023-10-08T01:15:21.770109+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:33.356972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797781+0000","last_change":"2023-10-08T01:15:21.789223+0000","last_active":"2023-10-08T01:15:21.797781+0000","last_peered":"2023-10-08T01:15:21.797781+0000","last_clean":"2023-10-08T01:15:21.797781+0000","last_became_active":"2023-10-08T01:15:21.788927+0000","last_became_peered":"2023-10-08T01:15:21.788927+0000","last_unstale":"2023-10-08T01:15:21.797781+0000","last_undegraded":"2023-10-08T01:15:21.797781+0000","last_fullsized":"2023-10-08T01:15:21.797781+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:08.803911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.768805+0000","last_change":"2023-10-08T01:15:20.755658+0000","last_active":"2023-10-08T01:15:21.768805+0000","last_peered":"2023-10-08T01:15:21.768805+0000","last_clean":"2023-10-08T01:15:21.768805+0000","last_became_active":"2023-10-08T01:15:20.755399+0000","last_became_peered":"2023-10-08T01:15:20.755399+0000","last_unstale":"2023-10-08T01:15:21.768805+0000","last_undegraded":"2023-10-08T01:15:21.768805+0000","last_fullsized":"2023-10-08T01:15:21.768805+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:17.373973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797948+0000","last_change":"2023-10-08T01:15:21.790482+0000","last_active":"2023-10-08T01:15:21.797948+0000","last_peered":"2023-10-08T01:15:21.797948+0000","last_clean":"2023-10-08T01:15:21.797948+0000","last_became_active":"2023-10-08T01:15:21.790236+0000","last_became_peered":"2023-10-08T01:15:21.790236+0000","last_unstale":"2023-10-08T01:15:21.797948+0000","last_undegraded":"2023-10-08T01:15:21.797948+0000","last_fullsized":"2023-10-08T01:15:21.797948+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:38.847322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767615+0000","last_change":"2023-10-08T01:15:20.767305+0000","last_active":"2023-10-08T01:15:21.767615+0000","last_peered":"2023-10-08T01:15:21.767615+0000","last_clean":"2023-10-08T01:15:21.767615+0000","last_became_active":"2023-10-08T01:15:20.767063+0000","last_became_peered":"2023-10-08T01:15:20.767063+0000","last_unstale":"2023-10-08T01:15:21.767615+0000","last_undegraded":"2023-10-08T01:15:21.767615+0000","last_fullsized":"2023-10-08T01:15:21.767615+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:27:18.647954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.781572+0000","last_change":"2023-10-08T01:15:21.777861+0000","last_active":"2023-10-08T01:15:21.781572+0000","last_peered":"2023-10-08T01:15:21.781572+0000","last_clean":"2023-10-08T01:15:21.781572+0000","last_became_active":"2023-10-08T01:15:21.777456+0000","last_became_peered":"2023-10-08T01:15:21.777456+0000","last_unstale":"2023-10-08T01:15:21.781572+0000","last_undegraded":"2023-10-08T01:15:21.781572+0000","last_fullsized":"2023-10-08T01:15:21.781572+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:31.022194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.776505+0000","last_change":"2023-10-08T01:15:20.757509+0000","last_active":"2023-10-08T01:15:21.776505+0000","last_peered":"2023-10-08T01:15:21.776505+0000","last_clean":"2023-10-08T01:15:21.776505+0000","last_became_active":"2023-10-08T01:15:20.757211+0000","last_became_peered":"2023-10-08T01:15:20.757211+0000","last_unstale":"2023-10-08T01:15:21.776505+0000","last_undegraded":"2023-10-08T01:15:21.776505+0000","last_fullsized":"2023-10-08T01:15:21.776505+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:37.841133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.798068+0000","last_change":"2023-10-08T01:15:21.796549+0000","last_active":"2023-10-08T01:15:21.798068+0000","last_peered":"2023-10-08T01:15:21.798068+0000","last_clean":"2023-10-08T01:15:21.798068+0000","last_became_active":"2023-10-08T01:15:21.796287+0000","last_became_peered":"2023-10-08T01:15:21.796287+0000","last_unstale":"2023-10-08T01:15:21.798068+0000","last_undegraded":"2023-10-08T01:15:21.798068+0000","last_fullsized":"2023-10-08T01:15:21.798068+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:34.228457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"134'3","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.801217+0000","last_change":"2023-10-08T01:15:20.768112+0000","last_active":"2023-10-08T01:15:21.801217+0000","last_peered":"2023-10-08T01:15:21.801217+0000","last_clean":"2023-10-08T01:15:21.801217+0000","last_became_active":"2023-10-08T01:15:20.767845+0000","last_became_peered":"2023-10-08T01:15:20.767845+0000","last_unstale":"2023-10-08T01:15:21.801217+0000","last_undegraded":"2023-10-08T01:15:21.801217+0000","last_fullsized":"2023-10-08T01:15:21.801217+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:01.392845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.798002+0000","last_change":"2023-10-08T01:15:21.791677+0000","last_active":"2023-10-08T01:15:21.798002+0000","last_peered":"2023-10-08T01:15:21.798002+0000","last_clean":"2023-10-08T01:15:21.798002+0000","last_became_active":"2023-10-08T01:15:21.791212+0000","last_became_peered":"2023-10-08T01:15:21.791212+0000","last_unstale":"2023-10-08T01:15:21.798002+0000","last_undegraded":"2023-10-08T01:15:21.798002+0000","last_fullsized":"2023-10-08T01:15:21.798002+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:27:45.706088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"134'6","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.803708+0000","last_change":"2023-10-08T01:15:20.756790+0000","last_active":"2023-10-08T01:15:21.803708+0000","last_peered":"2023-10-08T01:15:21.803708+0000","last_clean":"2023-10-08T01:15:21.803708+0000","last_became_active":"2023-10-08T01:15:20.756491+0000","last_became_peered":"2023-10-08T01:15:20.756491+0000","last_unstale":"2023-10-08T01:15:21.803708+0000","last_undegraded":"2023-10-08T01:15:21.803708+0000","last_fullsized":"2023-10-08T01:15:21.803708+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792468+0000","last_change":"2023-10-08T01:15:21.787149+0000","last_active":"2023-10-08T01:15:21.792468+0000","last_peered":"2023-10-08T01:15:21.792468+0000","last_clean":"2023-10-08T01:15:21.792468+0000","last_became_active":"2023-10-08T01:15:21.786902+0000","last_became_peered":"2023-10-08T01:15:21.786902+0000","last_unstale":"2023-10-08T01:15:21.792468+0000","last_undegraded":"2023-10-08T01:15:21.792468+0000","last_fullsized":"2023-10-08T01:15:21.792468+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:03.398793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799603+0000","last_change":"2023-10-08T01:15:20.765298+0000","last_active":"2023-10-08T01:15:21.799603+0000","last_peered":"2023-10-08T01:15:21.799603+0000","last_clean":"2023-10-08T01:15:21.799603+0000","last_became_active":"2023-10-08T01:15:20.764456+0000","last_became_peered":"2023-10-08T01:15:20.764456+0000","last_unstale":"2023-10-08T01:15:21.799603+0000","last_undegraded":"2023-10-08T01:15:21.799603+0000","last_fullsized":"2023-10-08T01:15:21.799603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:09.800976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789936+0000","last_change":"2023-10-08T01:15:21.785515+0000","last_active":"2023-10-08T01:15:21.789936+0000","last_peered":"2023-10-08T01:15:21.789936+0000","last_clean":"2023-10-08T01:15:21.789936+0000","last_became_active":"2023-10-08T01:15:21.784893+0000","last_became_peered":"2023-10-08T01:15:21.784893+0000","last_unstale":"2023-10-08T01:15:21.789936+0000","last_undegraded":"2023-10-08T01:15:21.789936+0000","last_fullsized":"2023-10-08T01:15:21.789936+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:18.380162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767473+0000","last_change":"2023-10-08T01:15:20.762200+0000","last_active":"2023-10-08T01:15:21.767473+0000","last_peered":"2023-10-08T01:15:21.767473+0000","last_clean":"2023-10-08T01:15:21.767473+0000","last_became_active":"2023-10-08T01:15:20.762001+0000","last_became_peered":"2023-10-08T01:15:20.762001+0000","last_unstale":"2023-10-08T01:15:21.767473+0000","last_undegraded":"2023-10-08T01:15:21.767473+0000","last_fullsized":"2023-10-08T01:15:21.767473+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:33.356972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.798866+0000","last_change":"2023-10-08T01:15:21.796655+0000","last_active":"2023-10-08T01:15:21.798866+0000","last_peered":"2023-10-08T01:15:21.798866+0000","last_clean":"2023-10-08T01:15:21.798866+0000","last_became_active":"2023-10-08T01:15:21.796402+0000","last_became_peered":"2023-10-08T01:15:21.796402+0000","last_unstale":"2023-10-08T01:15:21.798866+0000","last_undegraded":"2023-10-08T01:15:21.798866+0000","last_fullsized":"2023-10-08T01:15:21.798866+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:37.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"134'8","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.801692+0000","last_change":"2023-10-08T01:15:20.762398+0000","last_active":"2023-10-08T01:15:21.801692+0000","last_peered":"2023-10-08T01:15:21.801692+0000","last_clean":"2023-10-08T01:15:21.801692+0000","last_became_active":"2023-10-08T01:15:20.762132+0000","last_became_peered":"2023-10-08T01:15:20.762132+0000","last_unstale":"2023-10-08T01:15:21.801692+0000","last_undegraded":"2023-10-08T01:15:21.801692+0000","last_fullsized":"2023-10-08T01:15:21.801692+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:39.667978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.785286+0000","last_change":"2023-10-08T01:15:21.784507+0000","last_active":"2023-10-08T01:15:21.785286+0000","last_peered":"2023-10-08T01:15:21.785286+0000","last_clean":"2023-10-08T01:15:21.785286+0000","last_became_active":"2023-10-08T01:15:21.783943+0000","last_became_peered":"2023-10-08T01:15:21.783943+0000","last_unstale":"2023-10-08T01:15:21.785286+0000","last_undegraded":"2023-10-08T01:15:21.785286+0000","last_fullsized":"2023-10-08T01:15:21.785286+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:53.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.802073+0000","last_change":"2023-10-08T01:15:20.769238+0000","last_active":"2023-10-08T01:15:21.802073+0000","last_peered":"2023-10-08T01:15:21.802073+0000","last_clean":"2023-10-08T01:15:21.802073+0000","last_became_active":"2023-10-08T01:15:20.768131+0000","last_became_peered":"2023-10-08T01:15:20.768131+0000","last_unstale":"2023-10-08T01:15:21.802073+0000","last_undegraded":"2023-10-08T01:15:21.802073+0000","last_fullsized":"2023-10-08T01:15:21.802073+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:02.392605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789900+0000","last_change":"2023-10-08T01:15:21.787162+0000","last_active":"2023-10-08T01:15:21.789900+0000","last_peered":"2023-10-08T01:15:21.789900+0000","last_clean":"2023-10-08T01:15:21.789900+0000","last_became_active":"2023-10-08T01:15:21.786914+0000","last_became_peered":"2023-10-08T01:15:21.786914+0000","last_unstale":"2023-10-08T01:15:21.789900+0000","last_undegraded":"2023-10-08T01:15:21.789900+0000","last_fullsized":"2023-10-08T01:15:21.789900+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:48.908392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.802301+0000","last_change":"2023-10-08T01:15:20.755132+0000","last_active":"2023-10-08T01:15:21.802301+0000","last_peered":"2023-10-08T01:15:21.802301+0000","last_clean":"2023-10-08T01:15:21.802301+0000","last_became_active":"2023-10-08T01:15:20.754845+0000","last_became_peered":"2023-10-08T01:15:20.754845+0000","last_unstale":"2023-10-08T01:15:21.802301+0000","last_undegraded":"2023-10-08T01:15:21.802301+0000","last_fullsized":"2023-10-08T01:15:21.802301+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:09.800976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792021+0000","last_change":"2023-10-08T01:15:21.787527+0000","last_active":"2023-10-08T01:15:21.792021+0000","last_peered":"2023-10-08T01:15:21.792021+0000","last_clean":"2023-10-08T01:15:21.792021+0000","last_became_active":"2023-10-08T01:15:21.787232+0000","last_became_peered":"2023-10-08T01:15:21.787232+0000","last_unstale":"2023-10-08T01:15:21.792021+0000","last_undegraded":"2023-10-08T01:15:21.792021+0000","last_fullsized":"2023-10-08T01:15:21.792021+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:09.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799073+0000","last_change":"2023-10-08T01:15:20.761636+0000","last_active":"2023-10-08T01:15:21.799073+0000","last_peered":"2023-10-08T01:15:21.799073+0000","last_clean":"2023-10-08T01:15:21.799073+0000","last_became_active":"2023-10-08T01:15:20.761003+0000","last_became_peered":"2023-10-08T01:15:20.761003+0000","last_unstale":"2023-10-08T01:15:21.799073+0000","last_undegraded":"2023-10-08T01:15:21.799073+0000","last_fullsized":"2023-10-08T01:15:21.799073+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:09.044142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.787669+0000","last_change":"2023-10-08T01:15:21.785065+0000","last_active":"2023-10-08T01:15:21.787669+0000","last_peered":"2023-10-08T01:15:21.787669+0000","last_clean":"2023-10-08T01:15:21.787669+0000","last_became_active":"2023-10-08T01:15:21.784791+0000","last_became_peered":"2023-10-08T01:15:21.784791+0000","last_unstale":"2023-10-08T01:15:21.787669+0000","last_undegraded":"2023-10-08T01:15:21.787669+0000","last_fullsized":"2023-10-08T01:15:21.787669+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:37.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.773432+0000","last_change":"2023-10-08T01:15:20.765279+0000","last_active":"2023-10-08T01:15:21.773432+0000","last_peered":"2023-10-08T01:15:21.773432+0000","last_clean":"2023-10-08T01:15:21.773432+0000","last_became_active":"2023-10-08T01:15:20.765031+0000","last_became_peered":"2023-10-08T01:15:20.765031+0000","last_unstale":"2023-10-08T01:15:21.773432+0000","last_undegraded":"2023-10-08T01:15:21.773432+0000","last_fullsized":"2023-10-08T01:15:21.773432+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:15.250420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.795132+0000","last_change":"2023-10-08T01:15:21.790228+0000","last_active":"2023-10-08T01:15:21.795132+0000","last_peered":"2023-10-08T01:15:21.795132+0000","last_clean":"2023-10-08T01:15:21.795132+0000","last_became_active":"2023-10-08T01:15:21.789949+0000","last_became_peered":"2023-10-08T01:15:21.789949+0000","last_unstale":"2023-10-08T01:15:21.795132+0000","last_undegraded":"2023-10-08T01:15:21.795132+0000","last_fullsized":"2023-10-08T01:15:21.795132+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:02.399034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.791981+0000","last_change":"2023-10-08T01:15:20.764782+0000","last_active":"2023-10-08T01:15:21.791981+0000","last_peered":"2023-10-08T01:15:21.791981+0000","last_clean":"2023-10-08T01:15:21.791981+0000","last_became_active":"2023-10-08T01:15:20.764156+0000","last_became_peered":"2023-10-08T01:15:20.764156+0000","last_unstale":"2023-10-08T01:15:21.791981+0000","last_undegraded":"2023-10-08T01:15:21.791981+0000","last_fullsized":"2023-10-08T01:15:21.791981+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:22.379487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789541+0000","last_change":"2023-10-08T01:15:21.781595+0000","last_active":"2023-10-08T01:15:21.789541+0000","last_peered":"2023-10-08T01:15:21.789541+0000","last_clean":"2023-10-08T01:15:21.789541+0000","last_became_active":"2023-10-08T01:15:21.781058+0000","last_became_peered":"2023-10-08T01:15:21.781058+0000","last_unstale":"2023-10-08T01:15:21.789541+0000","last_undegraded":"2023-10-08T01:15:21.789541+0000","last_fullsized":"2023-10-08T01:15:21.789541+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:55:57.721830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767374+0000","last_change":"2023-10-08T01:15:20.766456+0000","last_active":"2023-10-08T01:15:21.767374+0000","last_peered":"2023-10-08T01:15:21.767374+0000","last_clean":"2023-10-08T01:15:21.767374+0000","last_became_active":"2023-10-08T01:15:20.766166+0000","last_became_peered":"2023-10-08T01:15:20.766166+0000","last_unstale":"2023-10-08T01:15:21.767374+0000","last_undegraded":"2023-10-08T01:15:21.767374+0000","last_fullsized":"2023-10-08T01:15:21.767374+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:12.990214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792127+0000","last_change":"2023-10-08T01:15:21.786289+0000","last_active":"2023-10-08T01:15:21.792127+0000","last_peered":"2023-10-08T01:15:21.792127+0000","last_clean":"2023-10-08T01:15:21.792127+0000","last_became_active":"2023-10-08T01:15:21.786033+0000","last_became_peered":"2023-10-08T01:15:21.786033+0000","last_unstale":"2023-10-08T01:15:21.792127+0000","last_undegraded":"2023-10-08T01:15:21.792127+0000","last_fullsized":"2023-10-08T01:15:21.792127+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:47.258669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.774200+0000","last_change":"2023-10-08T01:15:20.761046+0000","last_active":"2023-10-08T01:15:21.774200+0000","last_peered":"2023-10-08T01:15:21.774200+0000","last_clean":"2023-10-08T01:15:21.774200+0000","last_became_active":"2023-10-08T01:15:20.760786+0000","last_became_peered":"2023-10-08T01:15:20.760786+0000","last_unstale":"2023-10-08T01:15:21.774200+0000","last_undegraded":"2023-10-08T01:15:21.774200+0000","last_fullsized":"2023-10-08T01:15:21.774200+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:12.990214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.794366+0000","last_change":"2023-10-08T01:15:21.788597+0000","last_active":"2023-10-08T01:15:21.794366+0000","last_peered":"2023-10-08T01:15:21.794366+0000","last_clean":"2023-10-08T01:15:21.794366+0000","last_became_active":"2023-10-08T01:15:21.788354+0000","last_became_peered":"2023-10-08T01:15:21.788354+0000","last_unstale":"2023-10-08T01:15:21.794366+0000","last_undegraded":"2023-10-08T01:15:21.794366+0000","last_fullsized":"2023-10-08T01:15:21.794366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:18.981769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.768898+0000","last_change":"2023-10-08T01:15:20.760877+0000","last_active":"2023-10-08T01:15:21.768898+0000","last_peered":"2023-10-08T01:15:21.768898+0000","last_clean":"2023-10-08T01:15:21.768898+0000","last_became_active":"2023-10-08T01:15:20.760150+0000","last_became_peered":"2023-10-08T01:15:20.760150+0000","last_unstale":"2023-10-08T01:15:21.768898+0000","last_undegraded":"2023-10-08T01:15:21.768898+0000","last_fullsized":"2023-10-08T01:15:21.768898+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:29:46.505077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.790999+0000","last_change":"2023-10-08T01:15:21.788077+0000","last_active":"2023-10-08T01:15:21.790999+0000","last_peered":"2023-10-08T01:15:21.790999+0000","last_clean":"2023-10-08T01:15:21.790999+0000","last_became_active":"2023-10-08T01:15:21.787828+0000","last_became_peered":"2023-10-08T01:15:21.787828+0000","last_unstale":"2023-10-08T01:15:21.790999+0000","last_undegraded":"2023-10-08T01:15:21.790999+0000","last_fullsized":"2023-10-08T01:15:21.790999+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:54:47.586067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.802044+0000","last_change":"2023-10-08T01:15:20.753870+0000","last_active":"2023-10-08T01:15:21.802044+0000","last_peered":"2023-10-08T01:15:21.802044+0000","last_clean":"2023-10-08T01:15:21.802044+0000","last_became_active":"2023-10-08T01:15:20.753614+0000","last_became_peered":"2023-10-08T01:15:20.753614+0000","last_unstale":"2023-10-08T01:15:21.802044+0000","last_undegraded":"2023-10-08T01:15:21.802044+0000","last_fullsized":"2023-10-08T01:15:21.802044+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:26.643549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799000+0000","last_change":"2023-10-08T01:15:21.791040+0000","last_active":"2023-10-08T01:15:21.799000+0000","last_peered":"2023-10-08T01:15:21.799000+0000","last_clean":"2023-10-08T01:15:21.799000+0000","last_became_active":"2023-10-08T01:15:21.790605+0000","last_became_peered":"2023-10-08T01:15:21.790605+0000","last_unstale":"2023-10-08T01:15:21.799000+0000","last_undegraded":"2023-10-08T01:15:21.799000+0000","last_fullsized":"2023-10-08T01:15:21.799000+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:21.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767882+0000","last_change":"2023-10-08T01:15:20.753955+0000","last_active":"2023-10-08T01:15:21.767882+0000","last_peered":"2023-10-08T01:15:21.767882+0000","last_clean":"2023-10-08T01:15:21.767882+0000","last_became_active":"2023-10-08T01:15:20.753682+0000","last_became_peered":"2023-10-08T01:15:20.753682+0000","last_unstale":"2023-10-08T01:15:21.767882+0000","last_undegraded":"2023-10-08T01:15:21.767882+0000","last_fullsized":"2023-10-08T01:15:21.767882+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:07.797723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789592+0000","last_change":"2023-10-08T01:15:21.784259+0000","last_active":"2023-10-08T01:15:21.789592+0000","last_peered":"2023-10-08T01:15:21.789592+0000","last_clean":"2023-10-08T01:15:21.789592+0000","last_became_active":"2023-10-08T01:15:21.784008+0000","last_became_peered":"2023-10-08T01:15:21.784008+0000","last_unstale":"2023-10-08T01:15:21.789592+0000","last_undegraded":"2023-10-08T01:15:21.789592+0000","last_fullsized":"2023-10-08T01:15:21.789592+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:50:52.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.768465+0000","last_change":"2023-10-08T01:15:20.755101+0000","last_active":"2023-10-08T01:15:21.768465+0000","last_peered":"2023-10-08T01:15:21.768465+0000","last_clean":"2023-10-08T01:15:21.768465+0000","last_became_active":"2023-10-08T01:15:20.754832+0000","last_became_peered":"2023-10-08T01:15:20.754832+0000","last_unstale":"2023-10-08T01:15:21.768465+0000","last_undegraded":"2023-10-08T01:15:21.768465+0000","last_fullsized":"2023-10-08T01:15:21.768465+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:46.252481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.785152+0000","last_change":"2023-10-08T01:15:21.777061+0000","last_active":"2023-10-08T01:15:21.785152+0000","last_peered":"2023-10-08T01:15:21.785152+0000","last_clean":"2023-10-08T01:15:21.785152+0000","last_became_active":"2023-10-08T01:15:21.776708+0000","last_became_peered":"2023-10-08T01:15:21.776708+0000","last_unstale":"2023-10-08T01:15:21.785152+0000","last_undegraded":"2023-10-08T01:15:21.785152+0000","last_fullsized":"2023-10-08T01:15:21.785152+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:01.530176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.777731+0000","last_change":"2023-10-08T01:15:20.764641+0000","last_active":"2023-10-08T01:15:21.777731+0000","last_peered":"2023-10-08T01:15:21.777731+0000","last_clean":"2023-10-08T01:15:21.777731+0000","last_became_active":"2023-10-08T01:15:20.764364+0000","last_became_peered":"2023-10-08T01:15:20.764364+0000","last_unstale":"2023-10-08T01:15:21.777731+0000","last_undegraded":"2023-10-08T01:15:21.777731+0000","last_fullsized":"2023-10-08T01:15:21.777731+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:31.099680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.784968+0000","last_change":"2023-10-08T01:15:21.778266+0000","last_active":"2023-10-08T01:15:21.784968+0000","last_peered":"2023-10-08T01:15:21.784968+0000","last_clean":"2023-10-08T01:15:21.784968+0000","last_became_active":"2023-10-08T01:15:21.777903+0000","last_became_peered":"2023-10-08T01:15:21.777903+0000","last_unstale":"2023-10-08T01:15:21.784968+0000","last_undegraded":"2023-10-08T01:15:21.784968+0000","last_fullsized":"2023-10-08T01:15:21.784968+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:18.981769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.773698+0000","last_change":"2023-10-08T01:15:20.758963+0000","last_active":"2023-10-08T01:15:21.773698+0000","last_peered":"2023-10-08T01:15:21.773698+0000","last_clean":"2023-10-08T01:15:21.773698+0000","last_became_active":"2023-10-08T01:15:20.757575+0000","last_became_peered":"2023-10-08T01:15:20.757575+0000","last_unstale":"2023-10-08T01:15:21.773698+0000","last_undegraded":"2023-10-08T01:15:21.773698+0000","last_fullsized":"2023-10-08T01:15:21.773698+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:29.341500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.785051+0000","last_change":"2023-10-08T01:15:21.778917+0000","last_active":"2023-10-08T01:15:21.785051+0000","last_peered":"2023-10-08T01:15:21.785051+0000","last_clean":"2023-10-08T01:15:21.785051+0000","last_became_active":"2023-10-08T01:15:21.778629+0000","last_became_peered":"2023-10-08T01:15:21.778629+0000","last_unstale":"2023-10-08T01:15:21.785051+0000","last_undegraded":"2023-10-08T01:15:21.785051+0000","last_fullsized":"2023-10-08T01:15:21.785051+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:09.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.769452+0000","last_change":"2023-10-08T01:15:20.764672+0000","last_active":"2023-10-08T01:15:21.769452+0000","last_peered":"2023-10-08T01:15:21.769452+0000","last_clean":"2023-10-08T01:15:21.769452+0000","last_became_active":"2023-10-08T01:15:20.764379+0000","last_became_peered":"2023-10-08T01:15:20.764379+0000","last_unstale":"2023-10-08T01:15:21.769452+0000","last_undegraded":"2023-10-08T01:15:21.769452+0000","last_fullsized":"2023-10-08T01:15:21.769452+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:07.797723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799423+0000","last_change":"2023-10-08T01:15:21.795609+0000","last_active":"2023-10-08T01:15:21.799423+0000","last_peered":"2023-10-08T01:15:21.799423+0000","last_clean":"2023-10-08T01:15:21.799423+0000","last_became_active":"2023-10-08T01:15:21.795246+0000","last_became_peered":"2023-10-08T01:15:21.795246+0000","last_unstale":"2023-10-08T01:15:21.799423+0000","last_undegraded":"2023-10-08T01:15:21.799423+0000","last_fullsized":"2023-10-08T01:15:21.799423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:34.228457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767342+0000","last_change":"2023-10-08T01:15:20.764516+0000","last_active":"2023-10-08T01:15:21.767342+0000","last_peered":"2023-10-08T01:15:21.767342+0000","last_clean":"2023-10-08T01:15:21.767342+0000","last_became_active":"2023-10-08T01:15:20.764252+0000","last_became_peered":"2023-10-08T01:15:20.764252+0000","last_unstale":"2023-10-08T01:15:21.767342+0000","last_undegraded":"2023-10-08T01:15:21.767342+0000","last_fullsized":"2023-10-08T01:15:21.767342+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:11.456448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.793569+0000","last_change":"2023-10-08T01:15:21.784243+0000","last_active":"2023-10-08T01:15:21.793569+0000","last_peered":"2023-10-08T01:15:21.793569+0000","last_clean":"2023-10-08T01:15:21.793569+0000","last_became_active":"2023-10-08T01:15:21.783987+0000","last_became_peered":"2023-10-08T01:15:21.783987+0000","last_unstale":"2023-10-08T01:15:21.793569+0000","last_undegraded":"2023-10-08T01:15:21.793569+0000","last_fullsized":"2023-10-08T01:15:21.793569+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:57:59.430267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.769085+0000","last_change":"2023-10-08T01:15:20.760896+0000","last_active":"2023-10-08T01:15:21.769085+0000","last_peered":"2023-10-08T01:15:21.769085+0000","last_clean":"2023-10-08T01:15:21.769085+0000","last_became_active":"2023-10-08T01:15:20.760585+0000","last_became_peered":"2023-10-08T01:15:20.760585+0000","last_unstale":"2023-10-08T01:15:21.769085+0000","last_undegraded":"2023-10-08T01:15:21.769085+0000","last_fullsized":"2023-10-08T01:15:21.769085+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:32.801443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792491+0000","last_change":"2023-10-08T01:15:21.784922+0000","last_active":"2023-10-08T01:15:21.792491+0000","last_peered":"2023-10-08T01:15:21.792491+0000","last_clean":"2023-10-08T01:15:21.792491+0000","last_became_active":"2023-10-08T01:15:21.784546+0000","last_became_peered":"2023-10-08T01:15:21.784546+0000","last_unstale":"2023-10-08T01:15:21.792491+0000","last_undegraded":"2023-10-08T01:15:21.792491+0000","last_fullsized":"2023-10-08T01:15:21.792491+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:29:47.511265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.775328+0000","last_change":"2023-10-08T01:15:20.764695+0000","last_active":"2023-10-08T01:15:21.775328+0000","last_peered":"2023-10-08T01:15:21.775328+0000","last_clean":"2023-10-08T01:15:21.775328+0000","last_became_active":"2023-10-08T01:15:20.764421+0000","last_became_peered":"2023-10-08T01:15:20.764421+0000","last_unstale":"2023-10-08T01:15:21.775328+0000","last_undegraded":"2023-10-08T01:15:21.775328+0000","last_fullsized":"2023-10-08T01:15:21.775328+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:43:17.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"124'249","reported_seq":727,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.772088+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:15:21.772088+0000","last_peered":"2023-10-08T01:15:21.772088+0000","last_clean":"2023-10-08T01:15:21.772088+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:15:21.772088+0000","last_undegraded":"2023-10-08T01:15:21.772088+0000","last_fullsized":"2023-10-08T01:15:21.772088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575079,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307704,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064136,"statfs":{"total":96636764160,"available":96321675264,"internally_reserved":0,"allocated":2277376,"data_stored":1415305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.123},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82999999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.60799999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.68799999999999994},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.85399999999999998,"5min":0.95499999999999996,"15min":0.97699999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39200000000000002},"max":{"1min":0.92200000000000004,"5min":0.93600000000000005,"15min":1.0269999999999999},"last":0.626}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.87,"5min":1.133,"15min":1.133},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.66500000000000004,"15min":0.71399999999999997},"min":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.95099999999999996,"15min":1.167},"last":0.54300000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.69899999999999995},"min":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.93200000000000005,"15min":0.94699999999999995},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.501,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.052,"15min":1.079},"last":0.498}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72499999999999998,"5min":0.875,"15min":0.88300000000000001},"last":0.64900000000000002}]},{"osd":5,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.62},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84899999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82299999999999995,"5min":1.004,"15min":1.004},"last":0.47499999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86099999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.47099999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.69899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575079,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303024,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068816,"statfs":{"total":96636764160,"available":96326467584,"internally_reserved":0,"allocated":1679360,"data_stored":824419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.93200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67800000000000005,"15min":0.69099999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91200000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.63900000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70599999999999996,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":2.984},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38100000000000001},"max":{"1min":0.86799999999999999,"5min":1.004,"15min":3.0190000000000001},"last":0.69999999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.95999999999999996,"5min":0.998,"15min":0.998},"last":0.61499999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72499999999999998,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71699999999999997,"15min":0.73299999999999998},"min":{"1min":0.56100000000000005,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.92400000000000004,"5min":1.048,"15min":1.048},"last":0.65900000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.88500000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.623},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79000000000000004,"5min":0.89800000000000002,"15min":1.087},"last":0.80500000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.395},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.21},"last":0.75800000000000001}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":2.9489999999999998},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.81799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575079,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":824473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45200000000000001,"15min":0.46500000000000002},"min":{"1min":0.29999999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.45400000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45900000000000002,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.61099999999999999,"5min":0.70599999999999996,"15min":0.72499999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.54800000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.81899999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.58399999999999996}]},{"osd":4,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":1.296,"15min":1.452},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.099,"5min":1.137,"15min":1.1839999999999999},"last":0.92400000000000004}]},{"osd":5,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":1.119,"15min":1.3959999999999999},"last":1.135},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.77700000000000002},"min":{"1min":0.53200000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.5209999999999999},"last":0.93700000000000006}]},{"osd":6,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.56399999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.6020000000000001},"last":0.998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.77700000000000002},"min":{"1min":0.60899999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.046,"5min":1.1499999999999999,"15min":1.3660000000000001},"last":1.0089999999999999}]},{"osd":7,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77400000000000002,"15min":0.81799999999999995},"min":{"1min":0.63500000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96699999999999997,"5min":1.236,"15min":1.621},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.51300000000000001},"max":{"1min":1.0389999999999999,"5min":1.21,"15min":1.4890000000000001},"last":1.0389999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575078,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303620,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068220,"statfs":{"total":96636764160,"available":96325857280,"internally_reserved":0,"allocated":2289664,"data_stored":1416365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45500000000000002,"15min":0.46200000000000002},"min":{"1min":0.28199999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56499999999999995,"5min":0.95999999999999996,"15min":0.997},"last":0.443},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.28599999999999998},"max":{"1min":0.69699999999999995,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59099999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.433}]},{"osd":3,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68200000000000005,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.51400000000000001}]},{"osd":4,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.53600000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":1.006,"5min":1.341,"15min":1.617},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.55800000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88500000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":2.085,"15min":2.085},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.014,"5min":1.381,"15min":1.5289999999999999},"last":0.90400000000000003}]},{"osd":6,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.50900000000000001},"max":{"1min":0.877,"5min":1.361,"15min":1.3660000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94999999999999996,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.79600000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.80700000000000005},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.054,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83199999999999996},"min":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93700000000000006,"5min":1.2170000000000001,"15min":1.298},"last":0.97499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311212,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060628,"statfs":{"total":96636764160,"available":96318083072,"internally_reserved":0,"allocated":1675264,"data_stored":824349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.443,"15min":0.45700000000000002},"min":{"1min":0.34000000000000002,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":0.752,"15min":0.752},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.501,"5min":0.495,"15min":0.50900000000000001},"min":{"1min":0.316,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70499999999999996,"5min":0.85699999999999998,"15min":2.2930000000000001},"last":0.63500000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46500000000000002,"15min":0.48299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":2.786},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53200000000000003,"15min":0.54500000000000004},"min":{"1min":0.36299999999999999,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.751,"5min":0.80500000000000005,"15min":3.3180000000000001},"last":0.60299999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.57899999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.76600000000000001,"5min":0.93600000000000005,"15min":0.98399999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.34999999999999998},"max":{"1min":0.71299999999999997,"5min":0.83799999999999997,"15min":2.2170000000000001},"last":0.55200000000000005}]},{"osd":4,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.74199999999999999,"15min":0.76300000000000001},"min":{"1min":0.52500000000000002,"5min":0.503,"15min":0.503},"max":{"1min":1.079,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.76700000000000002,"15min":0.79600000000000004},"min":{"1min":0.65300000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.056,"5min":1.23,"15min":2.3559999999999999},"last":0.872}]},{"osd":5,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79500000000000004,"15min":0.82899999999999996},"min":{"1min":0.58299999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.1000000000000001,"5min":1.21,"15min":2.488},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.79800000000000004,"15min":0.82099999999999995},"min":{"1min":0.58799999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.167,"5min":1.284,"15min":2.7080000000000002},"last":0.72099999999999997}]},{"osd":6,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77300000000000002,"15min":0.79800000000000004},"min":{"1min":0.621,"5min":0.501,"15min":0.48499999999999999},"max":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":2.653},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.78500000000000003,"15min":0.79900000000000004},"min":{"1min":0.56000000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:14:24 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.77800000000000002,"15min":0.79900000000000004},"min":{"1min":0.57099999999999995,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.0289999999999999,"5min":1.157,"15min":2.5550000000000002},"last":0.95299999999999996},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.79200000000000004,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":1.204,"5min":1.2270000000000001,"15min":2.7490000000000001},"last":0.69299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575078,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":1691648,"data_stored":825955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":0.56100000000000005,"5min":0.627,"15min":0.80100000000000005},"last":0.308},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46200000000000002,"15min":0.47899999999999998},"min":{"1min":0.307,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.81200000000000006},"last":0.36399999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":0.70099999999999996,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48599999999999999,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.31},"max":{"1min":0.57899999999999996,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.51800000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.877,"15min":0.89700000000000002},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.46000000000000002,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.45900000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70599999999999996,"15min":0.74399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.98899999999999999,"5min":1.095,"15min":1.3029999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.413},"last":0.68899999999999995}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.76400000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.032,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.089},"last":0.80600000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.47999999999999998},"max":{"1min":1.0680000000000001,"5min":1.101,"15min":1.379},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.57799999999999996,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.486},"last":0.66700000000000004}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.78400000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.46500000000000002},"max":{"1min":1.0549999999999999,"5min":1.0840000000000001,"15min":1.389},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.78500000000000003},"min":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.94899999999999995,"5min":1.054,"15min":1.4730000000000001},"last":0.73499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575078,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311212,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060628,"statfs":{"total":96636764160,"available":96318083072,"internally_reserved":0,"allocated":1675264,"data_stored":824349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.65900000000000003,"15min":0.68000000000000005},"min":{"1min":0.40400000000000003,"5min":0.32500000000000001,"15min":0.30099999999999999},"max":{"1min":0.86499999999999999,"5min":0.999,"15min":0.999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.67500000000000004,"15min":0.69899999999999995},"min":{"1min":0.48899999999999999,"5min":0.40400000000000003,"15min":0.36599999999999999},"max":{"1min":0.878,"5min":0.97999999999999998,"15min":1.0049999999999999},"last":0.49199999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.68600000000000005,"15min":0.70299999999999996},"min":{"1min":0.46999999999999997,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89600000000000002,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.875,"5min":1.04,"15min":1.04},"last":0.61299999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69499999999999995,"15min":0.73299999999999998},"min":{"1min":0.52700000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89200000000000002,"5min":0.97199999999999998,"15min":1.002},"last":1.1859999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.72099999999999997,"15min":0.76300000000000001},"min":{"1min":0.63100000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92000000000000004,"5min":1.004,"15min":1.323},"last":1.1419999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.70999999999999996,"15min":0.753},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90100000000000002,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.749,"15min":0.78800000000000003},"min":{"1min":0.57599999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.879,"5min":1.0369999999999999,"15min":1.454},"last":0.75600000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60499999999999998,"15min":0.60899999999999999},"min":{"1min":0.39200000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.91200000000000003,"5min":0.98399999999999999,"15min":1.0209999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.58299999999999996,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.93000000000000005,"5min":0.94499999999999995,"15min":0.998},"last":0.72899999999999998}]},{"osd":6,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.46999999999999997,"5min":0.42599999999999999,"15min":0.40799999999999997},"max":{"1min":0.92300000000000004,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.45300000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.65400000000000003}]},{"osd":7,"last update":"Sun Oct 8 01:14:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.66700000000000004,"15min":0.67000000000000004},"min":{"1min":0.501,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.7490000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69899999999999995,"15min":0.71199999999999997},"min":{"1min":0.51700000000000002,"5min":0.503,"15min":0.45300000000000001},"max":{"1min":0.93100000000000005,"5min":0.96799999999999997,"15min":1.2450000000000001},"last":0.80900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":824925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.45000000000000001,"15min":0.34699999999999998},"max":{"1min":0.89700000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.61599999999999999}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.71099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":1.018,"5min":1.026,"15min":1.026},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81699999999999995,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.64300000000000002}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68799999999999994,"15min":0.70999999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71099999999999997,"15min":0.73299999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.96599999999999997,"5min":1.026,"15min":1.026},"last":0.84699999999999998}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94199999999999995,"5min":1.004,"15min":1.004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92200000000000004,"5min":1.034,"15min":1.034},"last":0.74299999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.90900000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0680000000000001},"last":0.76200000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.77600000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.58699999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.095},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.71599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92000000000000004,"5min":1.05,"15min":1.05},"last":0.79200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:15:31.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T01:15:31.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.582+0000 7f60b81ff700 1 Processor -- start 2023-10-08T01:15:31.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.582+0000 7f60b81ff700 1 -- start start 2023-10-08T01:15:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b81ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f60b00cbf30 0x7f60b00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b81ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00ccf90 0x7f60b00b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b81ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b00cc300 con 0x7f60b015a270 2023-10-08T01:15:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b81ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b0156bf0 con 0x7f60b00cbf30 2023-10-08T01:15:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b81ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b00ae950 con 0x7f60b00ccf90 2023-10-08T01:15:31.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b579a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00ccf90 0x7f60b00b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:31.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b5f9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f60b00cbf30 0x7f60b00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:31.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b679c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f60b015a270 0x7f60b00c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42620/0 (socket says 172.21.15.119:42620) 2023-10-08T01:15:31.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.584+0000 7f60b679c700 1 -- 172.21.15.119:0/1363326092 learned_addr learned my addr 172.21.15.119:0/1363326092 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:31.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b4f99700 1 -- 172.21.15.119:0/1363326092 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1394607209 0 0) 0x7f60b00cc300 con 0x7f60b015a270 2023-10-08T01:15:31.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b579a700 1 -- 172.21.15.119:0/1363326092 >> v2:172.21.15.184:3300/0 conn(0x7f60b00cbf30 msgr2=0x7f60b00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:31.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b579a700 1 --2- 172.21.15.119:0/1363326092 >> v2:172.21.15.184:3300/0 conn(0x7f60b00cbf30 0x7f60b00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b579a700 1 -- 172.21.15.119:0/1363326092 >> v1:172.21.15.119:6789/0 conn(0x7f60b015a270 legacy=0x7f60b00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:31.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b579a700 1 -- 172.21.15.119:0/1363326092 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b00ad370 con 0x7f60b00ccf90 2023-10-08T01:15:31.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b5f9b700 1 --2- 172.21.15.119:0/1363326092 >> v2:172.21.15.184:3300/0 conn(0x7f60b00cbf30 0x7f60b00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:31.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b579a700 1 --2- 172.21.15.119:0/1363326092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00ccf90 0x7f60b00b0d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60a00098e0 tx=0x7f60a00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdbc5fa0bd66ddf2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:31.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b4f99700 1 -- 172.21.15.119:0/1363326092 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a000f6b0 con 0x7f60b00ccf90 2023-10-08T01:15:31.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b4f99700 1 -- 172.21.15.119:0/1363326092 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a000f810 con 0x7f60b00ccf90 2023-10-08T01:15:31.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.585+0000 7f60b4f99700 1 -- 172.21.15.119:0/1363326092 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a000fbb0 con 0x7f60b00ccf90 2023-10-08T01:15:31.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.586+0000 7f60b81ff700 1 -- 172.21.15.119:0/1363326092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00ccf90 msgr2=0x7f60b00b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:31.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.586+0000 7f60b81ff700 1 --2- 172.21.15.119:0/1363326092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00ccf90 0x7f60b00b0d10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60a00098e0 tx=0x7f60a00091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.586+0000 7f60b81ff700 1 -- 172.21.15.119:0/1363326092 shutdown_connections 2023-10-08T01:15:31.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.586+0000 7f60b81ff700 1 --2- 172.21.15.119:0/1363326092 >> v2:172.21.15.184:3300/0 conn(0x7f60b00cbf30 0x7f60b00ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.586+0000 7f60b81ff700 1 --2- 172.21.15.119:0/1363326092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b00ccf90 0x7f60b00b0d10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.586+0000 7f60b81ff700 1 -- 172.21.15.119:0/1363326092 >> 172.21.15.119:0/1363326092 conn(0x7f60b006a320 msgr2=0x7f60b00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:31.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.586+0000 7f60b81ff700 1 -- 172.21.15.119:0/1363326092 shutdown_connections 2023-10-08T01:15:31.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.586+0000 7f60b81ff700 1 -- 172.21.15.119:0/1363326092 wait complete. 2023-10-08T01:15:31.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.587+0000 7f60b81ff700 1 Processor -- start 2023-10-08T01:15:31.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.587+0000 7f60b81ff700 1 -- start start 2023-10-08T01:15:31.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.588+0000 7f60b81ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b00cbf30 0x7f60b016caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:31.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.588+0000 7f60b81ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00ccf90 0x7f60b01576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:31.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.588+0000 7f60b81ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b015a270 0x7f60b016d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:31.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.588+0000 7f60b81ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0158c30 con 0x7f60b00cbf30 2023-10-08T01:15:31.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.588+0000 7f60b81ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0158020 con 0x7f60b00ccf90 2023-10-08T01:15:31.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.588+0000 7f60b81ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b0158300 con 0x7f60b015a270 2023-10-08T01:15:31.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.588+0000 7f60b5f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b00cbf30 0x7f60b016caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:31.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.588+0000 7f60b5f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b00cbf30 0x7f60b016caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42068/0 (socket says 172.21.15.119:42068) 2023-10-08T01:15:31.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b5f9b700 1 -- 172.21.15.119:0/910707028 learned_addr learned my addr 172.21.15.119:0/910707028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:31.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b579a700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00ccf90 0x7f60b01576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:31.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b679c700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b015a270 0x7f60b016d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:31.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b5f9b700 1 -- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b015a270 msgr2=0x7f60b016d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:31.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b5f9b700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b015a270 0x7f60b016d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b5f9b700 1 -- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00ccf90 msgr2=0x7f60b01576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:31.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b5f9b700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00ccf90 0x7f60b01576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b5f9b700 1 -- 172.21.15.119:0/910707028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ac009420 con 0x7f60b00cbf30 2023-10-08T01:15:31.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b579a700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00ccf90 0x7f60b01576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:31.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b679c700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b015a270 0x7f60b016d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:31.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60b5f9b700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b00cbf30 0x7f60b016caf0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00ec80 tx=0x7f60ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60a5ffb700 1 -- 172.21.15.119:0/910707028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00c9d0 con 0x7f60b00cbf30 2023-10-08T01:15:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60a5ffb700 1 -- 172.21.15.119:0/910707028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac00cb30 con 0x7f60b00cbf30 2023-10-08T01:15:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.589+0000 7f60a5ffb700 1 -- 172.21.15.119:0/910707028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00ceb0 con 0x7f60b00cbf30 2023-10-08T01:15:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.590+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a0009420 con 0x7f60b00cbf30 2023-10-08T01:15:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.590+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b01588c0 con 0x7f60b00cbf30 2023-10-08T01:15:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.591+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b016af50 con 0x7f60b00cbf30 2023-10-08T01:15:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.592+0000 7f60a5ffb700 1 -- 172.21.15.119:0/910707028 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f60ac013680 con 0x7f60b00cbf30 2023-10-08T01:15:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.592+0000 7f60a5ffb700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6094066f30 0x7f60940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:31.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.593+0000 7f60a5ffb700 1 -- 172.21.15.119:0/910707028 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f60ac084b20 con 0x7f60b00cbf30 2023-10-08T01:15:31.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.593+0000 7f60b579a700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6094066f30 0x7f60940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:31.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.593+0000 7f60b579a700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6094066f30 0x7f60940693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60a000a010 tx=0x7f60a0019000 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:31.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.597+0000 7f60a5ffb700 1 -- 172.21.15.119:0/910707028 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f60ac051bf0 con 0x7f60b00cbf30 2023-10-08T01:15:31.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.784+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60b004c3c0 con 0x7f6094066f30 2023-10-08T01:15:31.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.792+0000 7f60a5ffb700 1 -- 172.21.15.119:0/910707028 <== mgr.7989 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289250 (secure 0 0 0) 0x7f60b004c3c0 con 0x7f6094066f30 2023-10-08T01:15:31.800 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:31.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.796+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6094066f30 msgr2=0x7f60940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:31.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.796+0000 7f60b81ff700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6094066f30 0x7f60940693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f60a000a010 tx=0x7f60a0019000 comp rx=0 tx=0).stop 2023-10-08T01:15:31.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.796+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b00cbf30 msgr2=0x7f60b016caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:31.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.796+0000 7f60b81ff700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b00cbf30 0x7f60b016caf0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f60ac00ec80 tx=0x7f60ac0091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.797+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 shutdown_connections 2023-10-08T01:15:31.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.797+0000 7f60b81ff700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6094066f30 0x7f60940693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.797+0000 7f60b81ff700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f60b015a270 0x7f60b016d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.797+0000 7f60b81ff700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60b00cbf30 0x7f60b016caf0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.797+0000 7f60b81ff700 1 --2- 172.21.15.119:0/910707028 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f60b00ccf90 0x7f60b01576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:31.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.797+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 >> 172.21.15.119:0/910707028 conn(0x7f60b006a320 msgr2=0x7f60b0155d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:31.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.797+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 shutdown_connections 2023-10-08T01:15:31.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:31.797+0000 7f60b81ff700 1 -- 172.21.15.119:0/910707028 wait complete. 2023-10-08T01:15:31.806 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T01:15:31.816 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T01:15:31.694693+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":444,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":288,"ondisk_log_size":288,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2449960,"kb_used_data":14312,"kb_used_omap":0,"kb_used_meta":2435584,"kb_avail":752524760,"statfs":{"total":773094113280,"available":770585354240,"internally_reserved":0,"allocated":14655488,"data_stored":7780140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2494038016},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.931062"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.790793+0000","last_change":"2023-10-08T01:15:21.782424+0000","last_active":"2023-10-08T01:15:21.790793+0000","last_peered":"2023-10-08T01:15:21.790793+0000","last_clean":"2023-10-08T01:15:21.790793+0000","last_became_active":"2023-10-08T01:15:21.781981+0000","last_became_peered":"2023-10-08T01:15:21.781981+0000","last_unstale":"2023-10-08T01:15:21.790793+0000","last_undegraded":"2023-10-08T01:15:21.790793+0000","last_fullsized":"2023-10-08T01:15:21.790793+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:11:46.360472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789195+0000","last_change":"2023-10-08T01:15:21.782924+0000","last_active":"2023-10-08T01:15:21.789195+0000","last_peered":"2023-10-08T01:15:21.789195+0000","last_clean":"2023-10-08T01:15:21.789195+0000","last_became_active":"2023-10-08T01:15:21.782617+0000","last_became_peered":"2023-10-08T01:15:21.782617+0000","last_unstale":"2023-10-08T01:15:21.789195+0000","last_undegraded":"2023-10-08T01:15:21.789195+0000","last_fullsized":"2023-10-08T01:15:21.789195+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:37.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.794644+0000","last_change":"2023-10-08T01:15:21.791173+0000","last_active":"2023-10-08T01:15:21.794644+0000","last_peered":"2023-10-08T01:15:21.794644+0000","last_clean":"2023-10-08T01:15:21.794644+0000","last_became_active":"2023-10-08T01:15:21.790924+0000","last_became_peered":"2023-10-08T01:15:21.790924+0000","last_unstale":"2023-10-08T01:15:21.794644+0000","last_undegraded":"2023-10-08T01:15:21.794644+0000","last_fullsized":"2023-10-08T01:15:21.794644+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:18.380162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.794955+0000","last_change":"2023-10-08T01:15:21.788960+0000","last_active":"2023-10-08T01:15:21.794955+0000","last_peered":"2023-10-08T01:15:21.794955+0000","last_clean":"2023-10-08T01:15:21.794955+0000","last_became_active":"2023-10-08T01:15:21.787967+0000","last_became_peered":"2023-10-08T01:15:21.787967+0000","last_unstale":"2023-10-08T01:15:21.794955+0000","last_undegraded":"2023-10-08T01:15:21.794955+0000","last_fullsized":"2023-10-08T01:15:21.794955+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:24:26.259319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.786910+0000","last_change":"2023-10-08T01:15:21.786282+0000","last_active":"2023-10-08T01:15:21.786910+0000","last_peered":"2023-10-08T01:15:21.786910+0000","last_clean":"2023-10-08T01:15:21.786910+0000","last_became_active":"2023-10-08T01:15:21.786015+0000","last_became_peered":"2023-10-08T01:15:21.786015+0000","last_unstale":"2023-10-08T01:15:21.786910+0000","last_undegraded":"2023-10-08T01:15:21.786910+0000","last_fullsized":"2023-10-08T01:15:21.786910+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:30.787052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.783095+0000","last_change":"2023-10-08T01:15:21.780783+0000","last_active":"2023-10-08T01:15:21.783095+0000","last_peered":"2023-10-08T01:15:21.783095+0000","last_clean":"2023-10-08T01:15:21.783095+0000","last_became_active":"2023-10-08T01:15:21.780126+0000","last_became_peered":"2023-10-08T01:15:21.780126+0000","last_unstale":"2023-10-08T01:15:21.783095+0000","last_undegraded":"2023-10-08T01:15:21.783095+0000","last_fullsized":"2023-10-08T01:15:21.783095+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:16.256609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797806+0000","last_change":"2023-10-08T01:15:21.789868+0000","last_active":"2023-10-08T01:15:21.797806+0000","last_peered":"2023-10-08T01:15:21.797806+0000","last_clean":"2023-10-08T01:15:21.797806+0000","last_became_active":"2023-10-08T01:15:21.789605+0000","last_became_peered":"2023-10-08T01:15:21.789605+0000","last_unstale":"2023-10-08T01:15:21.797806+0000","last_undegraded":"2023-10-08T01:15:21.797806+0000","last_fullsized":"2023-10-08T01:15:21.797806+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:43:47.122521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.784403+0000","last_change":"2023-10-08T01:15:21.782274+0000","last_active":"2023-10-08T01:15:21.784403+0000","last_peered":"2023-10-08T01:15:21.784403+0000","last_clean":"2023-10-08T01:15:21.784403+0000","last_became_active":"2023-10-08T01:15:21.782014+0000","last_became_peered":"2023-10-08T01:15:21.782014+0000","last_unstale":"2023-10-08T01:15:21.784403+0000","last_undegraded":"2023-10-08T01:15:21.784403+0000","last_fullsized":"2023-10-08T01:15:21.784403+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:24:58.057009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.790774+0000","last_change":"2023-10-08T01:15:21.780528+0000","last_active":"2023-10-08T01:15:21.790774+0000","last_peered":"2023-10-08T01:15:21.790774+0000","last_clean":"2023-10-08T01:15:21.790774+0000","last_became_active":"2023-10-08T01:15:21.780257+0000","last_became_peered":"2023-10-08T01:15:21.780257+0000","last_unstale":"2023-10-08T01:15:21.790774+0000","last_undegraded":"2023-10-08T01:15:21.790774+0000","last_fullsized":"2023-10-08T01:15:21.790774+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:11:56.993744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.787497+0000","last_change":"2023-10-08T01:15:21.775199+0000","last_active":"2023-10-08T01:15:21.787497+0000","last_peered":"2023-10-08T01:15:21.787497+0000","last_clean":"2023-10-08T01:15:21.787497+0000","last_became_active":"2023-10-08T01:15:21.774975+0000","last_became_peered":"2023-10-08T01:15:21.774975+0000","last_unstale":"2023-10-08T01:15:21.787497+0000","last_undegraded":"2023-10-08T01:15:21.787497+0000","last_fullsized":"2023-10-08T01:15:21.787497+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:44.231188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796706+0000","last_change":"2023-10-08T01:15:21.790414+0000","last_active":"2023-10-08T01:15:21.796706+0000","last_peered":"2023-10-08T01:15:21.796706+0000","last_clean":"2023-10-08T01:15:21.796706+0000","last_became_active":"2023-10-08T01:15:21.790139+0000","last_became_peered":"2023-10-08T01:15:21.790139+0000","last_unstale":"2023-10-08T01:15:21.796706+0000","last_undegraded":"2023-10-08T01:15:21.796706+0000","last_fullsized":"2023-10-08T01:15:21.796706+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:46:38.709705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.784656+0000","last_change":"2023-10-08T01:15:21.774368+0000","last_active":"2023-10-08T01:15:21.784656+0000","last_peered":"2023-10-08T01:15:21.784656+0000","last_clean":"2023-10-08T01:15:21.784656+0000","last_became_active":"2023-10-08T01:15:21.774093+0000","last_became_peered":"2023-10-08T01:15:21.774093+0000","last_unstale":"2023-10-08T01:15:21.784656+0000","last_undegraded":"2023-10-08T01:15:21.784656+0000","last_fullsized":"2023-10-08T01:15:21.784656+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:10:57.415376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.795304+0000","last_change":"2023-10-08T01:15:21.790265+0000","last_active":"2023-10-08T01:15:21.795304+0000","last_peered":"2023-10-08T01:15:21.795304+0000","last_clean":"2023-10-08T01:15:21.795304+0000","last_became_active":"2023-10-08T01:15:21.790000+0000","last_became_peered":"2023-10-08T01:15:21.790000+0000","last_unstale":"2023-10-08T01:15:21.795304+0000","last_undegraded":"2023-10-08T01:15:21.795304+0000","last_fullsized":"2023-10-08T01:15:21.795304+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:18.981769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792176+0000","last_change":"2023-10-08T01:15:21.788126+0000","last_active":"2023-10-08T01:15:21.792176+0000","last_peered":"2023-10-08T01:15:21.792176+0000","last_clean":"2023-10-08T01:15:21.792176+0000","last_became_active":"2023-10-08T01:15:21.787794+0000","last_became_peered":"2023-10-08T01:15:21.787794+0000","last_unstale":"2023-10-08T01:15:21.792176+0000","last_undegraded":"2023-10-08T01:15:21.792176+0000","last_fullsized":"2023-10-08T01:15:21.792176+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:33.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.782492+0000","last_change":"2023-10-08T01:15:21.779837+0000","last_active":"2023-10-08T01:15:21.782492+0000","last_peered":"2023-10-08T01:15:21.782492+0000","last_clean":"2023-10-08T01:15:21.782492+0000","last_became_active":"2023-10-08T01:15:21.779599+0000","last_became_peered":"2023-10-08T01:15:21.779599+0000","last_unstale":"2023-10-08T01:15:21.782492+0000","last_undegraded":"2023-10-08T01:15:21.782492+0000","last_fullsized":"2023-10-08T01:15:21.782492+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:21:30.191316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789531+0000","last_change":"2023-10-08T01:15:21.782077+0000","last_active":"2023-10-08T01:15:21.789531+0000","last_peered":"2023-10-08T01:15:21.789531+0000","last_clean":"2023-10-08T01:15:21.789531+0000","last_became_active":"2023-10-08T01:15:21.781800+0000","last_became_peered":"2023-10-08T01:15:21.781800+0000","last_unstale":"2023-10-08T01:15:21.789531+0000","last_undegraded":"2023-10-08T01:15:21.789531+0000","last_fullsized":"2023-10-08T01:15:21.789531+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:58:08.359663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797541+0000","last_change":"2023-10-08T01:15:21.789809+0000","last_active":"2023-10-08T01:15:21.797541+0000","last_peered":"2023-10-08T01:15:21.797541+0000","last_clean":"2023-10-08T01:15:21.797541+0000","last_became_active":"2023-10-08T01:15:21.789454+0000","last_became_peered":"2023-10-08T01:15:21.789454+0000","last_unstale":"2023-10-08T01:15:21.797541+0000","last_undegraded":"2023-10-08T01:15:21.797541+0000","last_fullsized":"2023-10-08T01:15:21.797541+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:57:53.130296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.786486+0000","last_change":"2023-10-08T01:15:21.784309+0000","last_active":"2023-10-08T01:15:21.786486+0000","last_peered":"2023-10-08T01:15:21.786486+0000","last_clean":"2023-10-08T01:15:21.786486+0000","last_became_active":"2023-10-08T01:15:21.784045+0000","last_became_peered":"2023-10-08T01:15:21.784045+0000","last_unstale":"2023-10-08T01:15:21.786486+0000","last_undegraded":"2023-10-08T01:15:21.786486+0000","last_fullsized":"2023-10-08T01:15:21.786486+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:39:55.362748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792101+0000","last_change":"2023-10-08T01:15:21.787542+0000","last_active":"2023-10-08T01:15:21.792101+0000","last_peered":"2023-10-08T01:15:21.792101+0000","last_clean":"2023-10-08T01:15:21.792101+0000","last_became_active":"2023-10-08T01:15:21.786844+0000","last_became_peered":"2023-10-08T01:15:21.786844+0000","last_unstale":"2023-10-08T01:15:21.792101+0000","last_undegraded":"2023-10-08T01:15:21.792101+0000","last_fullsized":"2023-10-08T01:15:21.792101+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:03.398793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796769+0000","last_change":"2023-10-08T01:15:21.788914+0000","last_active":"2023-10-08T01:15:21.796769+0000","last_peered":"2023-10-08T01:15:21.796769+0000","last_clean":"2023-10-08T01:15:21.796769+0000","last_became_active":"2023-10-08T01:15:21.788624+0000","last_became_peered":"2023-10-08T01:15:21.788624+0000","last_unstale":"2023-10-08T01:15:21.796769+0000","last_undegraded":"2023-10-08T01:15:21.796769+0000","last_fullsized":"2023-10-08T01:15:21.796769+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:33.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789221+0000","last_change":"2023-10-08T01:15:21.783160+0000","last_active":"2023-10-08T01:15:21.789221+0000","last_peered":"2023-10-08T01:15:21.789221+0000","last_clean":"2023-10-08T01:15:21.789221+0000","last_became_active":"2023-10-08T01:15:21.782407+0000","last_became_peered":"2023-10-08T01:15:21.782407+0000","last_unstale":"2023-10-08T01:15:21.789221+0000","last_undegraded":"2023-10-08T01:15:21.789221+0000","last_fullsized":"2023-10-08T01:15:21.789221+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:34.228457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796873+0000","last_change":"2023-10-08T01:15:21.795149+0000","last_active":"2023-10-08T01:15:21.796873+0000","last_peered":"2023-10-08T01:15:21.796873+0000","last_clean":"2023-10-08T01:15:21.796873+0000","last_became_active":"2023-10-08T01:15:21.794884+0000","last_became_peered":"2023-10-08T01:15:21.794884+0000","last_unstale":"2023-10-08T01:15:21.796873+0000","last_undegraded":"2023-10-08T01:15:21.796873+0000","last_fullsized":"2023-10-08T01:15:21.796873+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:02.399034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789147+0000","last_change":"2023-10-08T01:15:21.782996+0000","last_active":"2023-10-08T01:15:21.789147+0000","last_peered":"2023-10-08T01:15:21.789147+0000","last_clean":"2023-10-08T01:15:21.789147+0000","last_became_active":"2023-10-08T01:15:21.782728+0000","last_became_peered":"2023-10-08T01:15:21.782728+0000","last_unstale":"2023-10-08T01:15:21.789147+0000","last_undegraded":"2023-10-08T01:15:21.789147+0000","last_fullsized":"2023-10-08T01:15:21.789147+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:51:43.676817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792512+0000","last_change":"2023-10-08T01:15:21.785571+0000","last_active":"2023-10-08T01:15:21.792512+0000","last_peered":"2023-10-08T01:15:21.792512+0000","last_clean":"2023-10-08T01:15:21.792512+0000","last_became_active":"2023-10-08T01:15:21.785133+0000","last_became_peered":"2023-10-08T01:15:21.785133+0000","last_unstale":"2023-10-08T01:15:21.792512+0000","last_undegraded":"2023-10-08T01:15:21.792512+0000","last_fullsized":"2023-10-08T01:15:21.792512+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:47.258669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797516+0000","last_change":"2023-10-08T01:15:21.787246+0000","last_active":"2023-10-08T01:15:21.797516+0000","last_peered":"2023-10-08T01:15:21.797516+0000","last_clean":"2023-10-08T01:15:21.797516+0000","last_became_active":"2023-10-08T01:15:21.786907+0000","last_became_peered":"2023-10-08T01:15:21.786907+0000","last_unstale":"2023-10-08T01:15:21.797516+0000","last_undegraded":"2023-10-08T01:15:21.797516+0000","last_fullsized":"2023-10-08T01:15:21.797516+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:11.579411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789677+0000","last_change":"2023-10-08T01:15:21.781323+0000","last_active":"2023-10-08T01:15:21.789677+0000","last_peered":"2023-10-08T01:15:21.789677+0000","last_clean":"2023-10-08T01:15:21.789677+0000","last_became_active":"2023-10-08T01:15:21.780917+0000","last_became_peered":"2023-10-08T01:15:21.780917+0000","last_unstale":"2023-10-08T01:15:21.789677+0000","last_undegraded":"2023-10-08T01:15:21.789677+0000","last_fullsized":"2023-10-08T01:15:21.789677+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:54:39.383007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.790237+0000","last_change":"2023-10-08T01:15:21.786972+0000","last_active":"2023-10-08T01:15:21.790237+0000","last_peered":"2023-10-08T01:15:21.790237+0000","last_clean":"2023-10-08T01:15:21.790237+0000","last_became_active":"2023-10-08T01:15:21.786481+0000","last_became_peered":"2023-10-08T01:15:21.786481+0000","last_unstale":"2023-10-08T01:15:21.790237+0000","last_undegraded":"2023-10-08T01:15:21.790237+0000","last_fullsized":"2023-10-08T01:15:21.790237+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:30.347688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.795879+0000","last_change":"2023-10-08T01:15:21.790335+0000","last_active":"2023-10-08T01:15:21.795879+0000","last_peered":"2023-10-08T01:15:21.795879+0000","last_clean":"2023-10-08T01:15:21.795879+0000","last_became_active":"2023-10-08T01:15:21.790073+0000","last_became_peered":"2023-10-08T01:15:21.790073+0000","last_unstale":"2023-10-08T01:15:21.795879+0000","last_undegraded":"2023-10-08T01:15:21.795879+0000","last_fullsized":"2023-10-08T01:15:21.795879+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:33.807632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.793099+0000","last_change":"2023-10-08T01:15:21.788246+0000","last_active":"2023-10-08T01:15:21.793099+0000","last_peered":"2023-10-08T01:15:21.793099+0000","last_clean":"2023-10-08T01:15:21.793099+0000","last_became_active":"2023-10-08T01:15:21.787987+0000","last_became_peered":"2023-10-08T01:15:21.787987+0000","last_unstale":"2023-10-08T01:15:21.793099+0000","last_undegraded":"2023-10-08T01:15:21.793099+0000","last_fullsized":"2023-10-08T01:15:21.793099+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:02.399034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.794373+0000","last_change":"2023-10-08T01:15:21.781441+0000","last_active":"2023-10-08T01:15:21.794373+0000","last_peered":"2023-10-08T01:15:21.794373+0000","last_clean":"2023-10-08T01:15:21.794373+0000","last_became_active":"2023-10-08T01:15:21.781187+0000","last_became_peered":"2023-10-08T01:15:21.781187+0000","last_unstale":"2023-10-08T01:15:21.794373+0000","last_undegraded":"2023-10-08T01:15:21.794373+0000","last_fullsized":"2023-10-08T01:15:21.794373+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:06:37.580409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789503+0000","last_change":"2023-10-08T01:15:21.781599+0000","last_active":"2023-10-08T01:15:21.789503+0000","last_peered":"2023-10-08T01:15:21.789503+0000","last_clean":"2023-10-08T01:15:21.789503+0000","last_became_active":"2023-10-08T01:15:21.781316+0000","last_became_peered":"2023-10-08T01:15:21.781316+0000","last_unstale":"2023-10-08T01:15:21.789503+0000","last_undegraded":"2023-10-08T01:15:21.789503+0000","last_fullsized":"2023-10-08T01:15:21.789503+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:27:55.237677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.782546+0000","last_change":"2023-10-08T01:15:21.779332+0000","last_active":"2023-10-08T01:15:21.782546+0000","last_peered":"2023-10-08T01:15:21.782546+0000","last_clean":"2023-10-08T01:15:21.782546+0000","last_became_active":"2023-10-08T01:15:21.779082+0000","last_became_peered":"2023-10-08T01:15:21.779082+0000","last_unstale":"2023-10-08T01:15:21.782546+0000","last_undegraded":"2023-10-08T01:15:21.782546+0000","last_fullsized":"2023-10-08T01:15:21.782546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:43:18.530363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.788532+0000","last_change":"2023-10-08T01:15:21.783048+0000","last_active":"2023-10-08T01:15:21.788532+0000","last_peered":"2023-10-08T01:15:21.788532+0000","last_clean":"2023-10-08T01:15:21.788532+0000","last_became_active":"2023-10-08T01:15:21.782740+0000","last_became_peered":"2023-10-08T01:15:21.782740+0000","last_unstale":"2023-10-08T01:15:21.788532+0000","last_undegraded":"2023-10-08T01:15:21.788532+0000","last_fullsized":"2023-10-08T01:15:21.788532+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:50:55.168069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"134'4","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.798197+0000","last_change":"2023-10-08T01:15:20.759292+0000","last_active":"2023-10-08T01:15:21.798197+0000","last_peered":"2023-10-08T01:15:21.798197+0000","last_clean":"2023-10-08T01:15:21.798197+0000","last_became_active":"2023-10-08T01:15:20.759026+0000","last_became_peered":"2023-10-08T01:15:20.759026+0000","last_unstale":"2023-10-08T01:15:21.798197+0000","last_undegraded":"2023-10-08T01:15:21.798197+0000","last_fullsized":"2023-10-08T01:15:21.798197+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:43:16.248787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796130+0000","last_change":"2023-10-08T01:15:21.789046+0000","last_active":"2023-10-08T01:15:21.796130+0000","last_peered":"2023-10-08T01:15:21.796130+0000","last_clean":"2023-10-08T01:15:21.796130+0000","last_became_active":"2023-10-08T01:15:21.788707+0000","last_became_peered":"2023-10-08T01:15:21.788707+0000","last_unstale":"2023-10-08T01:15:21.796130+0000","last_undegraded":"2023-10-08T01:15:21.796130+0000","last_fullsized":"2023-10-08T01:15:21.796130+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:08:10.980900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799550+0000","last_change":"2023-10-08T01:15:20.760884+0000","last_active":"2023-10-08T01:15:21.799550+0000","last_peered":"2023-10-08T01:15:21.799550+0000","last_clean":"2023-10-08T01:15:21.799550+0000","last_became_active":"2023-10-08T01:15:20.760631+0000","last_became_peered":"2023-10-08T01:15:20.760631+0000","last_unstale":"2023-10-08T01:15:21.799550+0000","last_undegraded":"2023-10-08T01:15:21.799550+0000","last_fullsized":"2023-10-08T01:15:21.799550+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:47.347739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.787025+0000","last_change":"2023-10-08T01:15:21.782815+0000","last_active":"2023-10-08T01:15:21.787025+0000","last_peered":"2023-10-08T01:15:21.787025+0000","last_clean":"2023-10-08T01:15:21.787025+0000","last_became_active":"2023-10-08T01:15:21.782500+0000","last_became_peered":"2023-10-08T01:15:21.782500+0000","last_unstale":"2023-10-08T01:15:21.787025+0000","last_undegraded":"2023-10-08T01:15:21.787025+0000","last_fullsized":"2023-10-08T01:15:21.787025+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:51:04.898012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797869+0000","last_change":"2023-10-08T01:15:20.765142+0000","last_active":"2023-10-08T01:15:21.797869+0000","last_peered":"2023-10-08T01:15:21.797869+0000","last_clean":"2023-10-08T01:15:21.797869+0000","last_became_active":"2023-10-08T01:15:20.764874+0000","last_became_peered":"2023-10-08T01:15:20.764874+0000","last_unstale":"2023-10-08T01:15:21.797869+0000","last_undegraded":"2023-10-08T01:15:21.797869+0000","last_fullsized":"2023-10-08T01:15:21.797869+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:21:26.740469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792120+0000","last_change":"2023-10-08T01:15:21.788245+0000","last_active":"2023-10-08T01:15:21.792120+0000","last_peered":"2023-10-08T01:15:21.792120+0000","last_clean":"2023-10-08T01:15:21.792120+0000","last_became_active":"2023-10-08T01:15:21.787939+0000","last_became_peered":"2023-10-08T01:15:21.787939+0000","last_unstale":"2023-10-08T01:15:21.792120+0000","last_undegraded":"2023-10-08T01:15:21.792120+0000","last_fullsized":"2023-10-08T01:15:21.792120+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:29:47.511265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.796917+0000","last_change":"2023-10-08T01:15:20.766881+0000","last_active":"2023-10-08T01:15:21.796917+0000","last_peered":"2023-10-08T01:15:21.796917+0000","last_clean":"2023-10-08T01:15:21.796917+0000","last_became_active":"2023-10-08T01:15:20.766569+0000","last_became_peered":"2023-10-08T01:15:20.766569+0000","last_unstale":"2023-10-08T01:15:21.796917+0000","last_undegraded":"2023-10-08T01:15:21.796917+0000","last_fullsized":"2023-10-08T01:15:21.796917+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:22.379487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792453+0000","last_change":"2023-10-08T01:15:21.781168+0000","last_active":"2023-10-08T01:15:21.792453+0000","last_peered":"2023-10-08T01:15:21.792453+0000","last_clean":"2023-10-08T01:15:21.792453+0000","last_became_active":"2023-10-08T01:15:21.780884+0000","last_became_peered":"2023-10-08T01:15:21.780884+0000","last_unstale":"2023-10-08T01:15:21.792453+0000","last_undegraded":"2023-10-08T01:15:21.792453+0000","last_fullsized":"2023-10-08T01:15:21.792453+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:41:44.233312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797819+0000","last_change":"2023-10-08T01:15:20.766640+0000","last_active":"2023-10-08T01:15:21.797819+0000","last_peered":"2023-10-08T01:15:21.797819+0000","last_clean":"2023-10-08T01:15:21.797819+0000","last_became_active":"2023-10-08T01:15:20.766421+0000","last_became_peered":"2023-10-08T01:15:20.766421+0000","last_unstale":"2023-10-08T01:15:21.797819+0000","last_undegraded":"2023-10-08T01:15:21.797819+0000","last_fullsized":"2023-10-08T01:15:21.797819+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:47.902204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789239+0000","last_change":"2023-10-08T01:15:21.783360+0000","last_active":"2023-10-08T01:15:21.789239+0000","last_peered":"2023-10-08T01:15:21.789239+0000","last_clean":"2023-10-08T01:15:21.789239+0000","last_became_active":"2023-10-08T01:15:21.782826+0000","last_became_peered":"2023-10-08T01:15:21.782826+0000","last_unstale":"2023-10-08T01:15:21.789239+0000","last_undegraded":"2023-10-08T01:15:21.789239+0000","last_fullsized":"2023-10-08T01:15:21.789239+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:18.380162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797702+0000","last_change":"2023-10-08T01:15:20.758281+0000","last_active":"2023-10-08T01:15:21.797702+0000","last_peered":"2023-10-08T01:15:21.797702+0000","last_clean":"2023-10-08T01:15:21.797702+0000","last_became_active":"2023-10-08T01:15:20.757981+0000","last_became_peered":"2023-10-08T01:15:20.757981+0000","last_unstale":"2023-10-08T01:15:21.797702+0000","last_undegraded":"2023-10-08T01:15:21.797702+0000","last_fullsized":"2023-10-08T01:15:21.797702+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:33:47.347739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.787062+0000","last_change":"2023-10-08T01:15:21.776121+0000","last_active":"2023-10-08T01:15:21.787062+0000","last_peered":"2023-10-08T01:15:21.787062+0000","last_clean":"2023-10-08T01:15:21.787062+0000","last_became_active":"2023-10-08T01:15:21.775812+0000","last_became_peered":"2023-10-08T01:15:21.775812+0000","last_unstale":"2023-10-08T01:15:21.787062+0000","last_undegraded":"2023-10-08T01:15:21.787062+0000","last_fullsized":"2023-10-08T01:15:21.787062+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:09:06.156436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.770109+0000","last_change":"2023-10-08T01:15:20.754017+0000","last_active":"2023-10-08T01:15:21.770109+0000","last_peered":"2023-10-08T01:15:21.770109+0000","last_clean":"2023-10-08T01:15:21.770109+0000","last_became_active":"2023-10-08T01:15:20.753760+0000","last_became_peered":"2023-10-08T01:15:20.753760+0000","last_unstale":"2023-10-08T01:15:21.770109+0000","last_undegraded":"2023-10-08T01:15:21.770109+0000","last_fullsized":"2023-10-08T01:15:21.770109+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:33.356972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797781+0000","last_change":"2023-10-08T01:15:21.789223+0000","last_active":"2023-10-08T01:15:21.797781+0000","last_peered":"2023-10-08T01:15:21.797781+0000","last_clean":"2023-10-08T01:15:21.797781+0000","last_became_active":"2023-10-08T01:15:21.788927+0000","last_became_peered":"2023-10-08T01:15:21.788927+0000","last_unstale":"2023-10-08T01:15:21.797781+0000","last_undegraded":"2023-10-08T01:15:21.797781+0000","last_fullsized":"2023-10-08T01:15:21.797781+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:08.803911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.768805+0000","last_change":"2023-10-08T01:15:20.755658+0000","last_active":"2023-10-08T01:15:21.768805+0000","last_peered":"2023-10-08T01:15:21.768805+0000","last_clean":"2023-10-08T01:15:21.768805+0000","last_became_active":"2023-10-08T01:15:20.755399+0000","last_became_peered":"2023-10-08T01:15:20.755399+0000","last_unstale":"2023-10-08T01:15:21.768805+0000","last_undegraded":"2023-10-08T01:15:21.768805+0000","last_fullsized":"2023-10-08T01:15:21.768805+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:17.373973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.797948+0000","last_change":"2023-10-08T01:15:21.790482+0000","last_active":"2023-10-08T01:15:21.797948+0000","last_peered":"2023-10-08T01:15:21.797948+0000","last_clean":"2023-10-08T01:15:21.797948+0000","last_became_active":"2023-10-08T01:15:21.790236+0000","last_became_peered":"2023-10-08T01:15:21.790236+0000","last_unstale":"2023-10-08T01:15:21.797948+0000","last_undegraded":"2023-10-08T01:15:21.797948+0000","last_fullsized":"2023-10-08T01:15:21.797948+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:38.847322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767615+0000","last_change":"2023-10-08T01:15:20.767305+0000","last_active":"2023-10-08T01:15:21.767615+0000","last_peered":"2023-10-08T01:15:21.767615+0000","last_clean":"2023-10-08T01:15:21.767615+0000","last_became_active":"2023-10-08T01:15:20.767063+0000","last_became_peered":"2023-10-08T01:15:20.767063+0000","last_unstale":"2023-10-08T01:15:21.767615+0000","last_undegraded":"2023-10-08T01:15:21.767615+0000","last_fullsized":"2023-10-08T01:15:21.767615+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:27:18.647954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.781572+0000","last_change":"2023-10-08T01:15:21.777861+0000","last_active":"2023-10-08T01:15:21.781572+0000","last_peered":"2023-10-08T01:15:21.781572+0000","last_clean":"2023-10-08T01:15:21.781572+0000","last_became_active":"2023-10-08T01:15:21.777456+0000","last_became_peered":"2023-10-08T01:15:21.777456+0000","last_unstale":"2023-10-08T01:15:21.781572+0000","last_undegraded":"2023-10-08T01:15:21.781572+0000","last_fullsized":"2023-10-08T01:15:21.781572+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:31.022194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.776505+0000","last_change":"2023-10-08T01:15:20.757509+0000","last_active":"2023-10-08T01:15:21.776505+0000","last_peered":"2023-10-08T01:15:21.776505+0000","last_clean":"2023-10-08T01:15:21.776505+0000","last_became_active":"2023-10-08T01:15:20.757211+0000","last_became_peered":"2023-10-08T01:15:20.757211+0000","last_unstale":"2023-10-08T01:15:21.776505+0000","last_undegraded":"2023-10-08T01:15:21.776505+0000","last_fullsized":"2023-10-08T01:15:21.776505+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:51:37.841133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.798068+0000","last_change":"2023-10-08T01:15:21.796549+0000","last_active":"2023-10-08T01:15:21.798068+0000","last_peered":"2023-10-08T01:15:21.798068+0000","last_clean":"2023-10-08T01:15:21.798068+0000","last_became_active":"2023-10-08T01:15:21.796287+0000","last_became_peered":"2023-10-08T01:15:21.796287+0000","last_unstale":"2023-10-08T01:15:21.798068+0000","last_undegraded":"2023-10-08T01:15:21.798068+0000","last_fullsized":"2023-10-08T01:15:21.798068+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:34.228457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"134'3","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.801217+0000","last_change":"2023-10-08T01:15:20.768112+0000","last_active":"2023-10-08T01:15:21.801217+0000","last_peered":"2023-10-08T01:15:21.801217+0000","last_clean":"2023-10-08T01:15:21.801217+0000","last_became_active":"2023-10-08T01:15:20.767845+0000","last_became_peered":"2023-10-08T01:15:20.767845+0000","last_unstale":"2023-10-08T01:15:21.801217+0000","last_undegraded":"2023-10-08T01:15:21.801217+0000","last_fullsized":"2023-10-08T01:15:21.801217+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:01.392845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.798002+0000","last_change":"2023-10-08T01:15:21.791677+0000","last_active":"2023-10-08T01:15:21.798002+0000","last_peered":"2023-10-08T01:15:21.798002+0000","last_clean":"2023-10-08T01:15:21.798002+0000","last_became_active":"2023-10-08T01:15:21.791212+0000","last_became_peered":"2023-10-08T01:15:21.791212+0000","last_unstale":"2023-10-08T01:15:21.798002+0000","last_undegraded":"2023-10-08T01:15:21.798002+0000","last_fullsized":"2023-10-08T01:15:21.798002+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:27:45.706088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"134'6","reported_seq":16,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.803708+0000","last_change":"2023-10-08T01:15:20.756790+0000","last_active":"2023-10-08T01:15:21.803708+0000","last_peered":"2023-10-08T01:15:21.803708+0000","last_clean":"2023-10-08T01:15:21.803708+0000","last_became_active":"2023-10-08T01:15:20.756491+0000","last_became_peered":"2023-10-08T01:15:20.756491+0000","last_unstale":"2023-10-08T01:15:21.803708+0000","last_undegraded":"2023-10-08T01:15:21.803708+0000","last_fullsized":"2023-10-08T01:15:21.803708+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.284304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792468+0000","last_change":"2023-10-08T01:15:21.787149+0000","last_active":"2023-10-08T01:15:21.792468+0000","last_peered":"2023-10-08T01:15:21.792468+0000","last_clean":"2023-10-08T01:15:21.792468+0000","last_became_active":"2023-10-08T01:15:21.786902+0000","last_became_peered":"2023-10-08T01:15:21.786902+0000","last_unstale":"2023-10-08T01:15:21.792468+0000","last_undegraded":"2023-10-08T01:15:21.792468+0000","last_fullsized":"2023-10-08T01:15:21.792468+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:03.398793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799603+0000","last_change":"2023-10-08T01:15:20.765298+0000","last_active":"2023-10-08T01:15:21.799603+0000","last_peered":"2023-10-08T01:15:21.799603+0000","last_clean":"2023-10-08T01:15:21.799603+0000","last_became_active":"2023-10-08T01:15:20.764456+0000","last_became_peered":"2023-10-08T01:15:20.764456+0000","last_unstale":"2023-10-08T01:15:21.799603+0000","last_undegraded":"2023-10-08T01:15:21.799603+0000","last_fullsized":"2023-10-08T01:15:21.799603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:09.800976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789936+0000","last_change":"2023-10-08T01:15:21.785515+0000","last_active":"2023-10-08T01:15:21.789936+0000","last_peered":"2023-10-08T01:15:21.789936+0000","last_clean":"2023-10-08T01:15:21.789936+0000","last_became_active":"2023-10-08T01:15:21.784893+0000","last_became_peered":"2023-10-08T01:15:21.784893+0000","last_unstale":"2023-10-08T01:15:21.789936+0000","last_undegraded":"2023-10-08T01:15:21.789936+0000","last_fullsized":"2023-10-08T01:15:21.789936+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:18.380162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767473+0000","last_change":"2023-10-08T01:15:20.762200+0000","last_active":"2023-10-08T01:15:21.767473+0000","last_peered":"2023-10-08T01:15:21.767473+0000","last_clean":"2023-10-08T01:15:21.767473+0000","last_became_active":"2023-10-08T01:15:20.762001+0000","last_became_peered":"2023-10-08T01:15:20.762001+0000","last_unstale":"2023-10-08T01:15:21.767473+0000","last_undegraded":"2023-10-08T01:15:21.767473+0000","last_fullsized":"2023-10-08T01:15:21.767473+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:04:33.356972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.798866+0000","last_change":"2023-10-08T01:15:21.796655+0000","last_active":"2023-10-08T01:15:21.798866+0000","last_peered":"2023-10-08T01:15:21.798866+0000","last_clean":"2023-10-08T01:15:21.798866+0000","last_became_active":"2023-10-08T01:15:21.796402+0000","last_became_peered":"2023-10-08T01:15:21.796402+0000","last_unstale":"2023-10-08T01:15:21.798866+0000","last_undegraded":"2023-10-08T01:15:21.798866+0000","last_fullsized":"2023-10-08T01:15:21.798866+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:37.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"134'8","reported_seq":18,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.801692+0000","last_change":"2023-10-08T01:15:20.762398+0000","last_active":"2023-10-08T01:15:21.801692+0000","last_peered":"2023-10-08T01:15:21.801692+0000","last_clean":"2023-10-08T01:15:21.801692+0000","last_became_active":"2023-10-08T01:15:20.762132+0000","last_became_peered":"2023-10-08T01:15:20.762132+0000","last_unstale":"2023-10-08T01:15:21.801692+0000","last_undegraded":"2023-10-08T01:15:21.801692+0000","last_fullsized":"2023-10-08T01:15:21.801692+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:05:39.667978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.785286+0000","last_change":"2023-10-08T01:15:21.784507+0000","last_active":"2023-10-08T01:15:21.785286+0000","last_peered":"2023-10-08T01:15:21.785286+0000","last_clean":"2023-10-08T01:15:21.785286+0000","last_became_active":"2023-10-08T01:15:21.783943+0000","last_became_peered":"2023-10-08T01:15:21.783943+0000","last_unstale":"2023-10-08T01:15:21.785286+0000","last_undegraded":"2023-10-08T01:15:21.785286+0000","last_fullsized":"2023-10-08T01:15:21.785286+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:58:53.433753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.802073+0000","last_change":"2023-10-08T01:15:20.769238+0000","last_active":"2023-10-08T01:15:21.802073+0000","last_peered":"2023-10-08T01:15:21.802073+0000","last_clean":"2023-10-08T01:15:21.802073+0000","last_became_active":"2023-10-08T01:15:20.768131+0000","last_became_peered":"2023-10-08T01:15:20.768131+0000","last_unstale":"2023-10-08T01:15:21.802073+0000","last_undegraded":"2023-10-08T01:15:21.802073+0000","last_fullsized":"2023-10-08T01:15:21.802073+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:02.392605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789900+0000","last_change":"2023-10-08T01:15:21.787162+0000","last_active":"2023-10-08T01:15:21.789900+0000","last_peered":"2023-10-08T01:15:21.789900+0000","last_clean":"2023-10-08T01:15:21.789900+0000","last_became_active":"2023-10-08T01:15:21.786914+0000","last_became_peered":"2023-10-08T01:15:21.786914+0000","last_unstale":"2023-10-08T01:15:21.789900+0000","last_undegraded":"2023-10-08T01:15:21.789900+0000","last_fullsized":"2023-10-08T01:15:21.789900+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:13:48.908392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.802301+0000","last_change":"2023-10-08T01:15:20.755132+0000","last_active":"2023-10-08T01:15:21.802301+0000","last_peered":"2023-10-08T01:15:21.802301+0000","last_clean":"2023-10-08T01:15:21.802301+0000","last_became_active":"2023-10-08T01:15:20.754845+0000","last_became_peered":"2023-10-08T01:15:20.754845+0000","last_unstale":"2023-10-08T01:15:21.802301+0000","last_undegraded":"2023-10-08T01:15:21.802301+0000","last_fullsized":"2023-10-08T01:15:21.802301+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:13:09.800976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792021+0000","last_change":"2023-10-08T01:15:21.787527+0000","last_active":"2023-10-08T01:15:21.792021+0000","last_peered":"2023-10-08T01:15:21.792021+0000","last_clean":"2023-10-08T01:15:21.792021+0000","last_became_active":"2023-10-08T01:15:21.787232+0000","last_became_peered":"2023-10-08T01:15:21.787232+0000","last_unstale":"2023-10-08T01:15:21.792021+0000","last_undegraded":"2023-10-08T01:15:21.792021+0000","last_fullsized":"2023-10-08T01:15:21.792021+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:09.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"134'1","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799073+0000","last_change":"2023-10-08T01:15:20.761636+0000","last_active":"2023-10-08T01:15:21.799073+0000","last_peered":"2023-10-08T01:15:21.799073+0000","last_clean":"2023-10-08T01:15:21.799073+0000","last_became_active":"2023-10-08T01:15:20.761003+0000","last_became_peered":"2023-10-08T01:15:20.761003+0000","last_unstale":"2023-10-08T01:15:21.799073+0000","last_undegraded":"2023-10-08T01:15:21.799073+0000","last_fullsized":"2023-10-08T01:15:21.799073+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:30:09.044142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.787669+0000","last_change":"2023-10-08T01:15:21.785065+0000","last_active":"2023-10-08T01:15:21.787669+0000","last_peered":"2023-10-08T01:15:21.787669+0000","last_clean":"2023-10-08T01:15:21.787669+0000","last_became_active":"2023-10-08T01:15:21.784791+0000","last_became_peered":"2023-10-08T01:15:21.784791+0000","last_unstale":"2023-10-08T01:15:21.787669+0000","last_undegraded":"2023-10-08T01:15:21.787669+0000","last_fullsized":"2023-10-08T01:15:21.787669+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:37.290492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.773432+0000","last_change":"2023-10-08T01:15:20.765279+0000","last_active":"2023-10-08T01:15:21.773432+0000","last_peered":"2023-10-08T01:15:21.773432+0000","last_clean":"2023-10-08T01:15:21.773432+0000","last_became_active":"2023-10-08T01:15:20.765031+0000","last_became_peered":"2023-10-08T01:15:20.765031+0000","last_unstale":"2023-10-08T01:15:21.773432+0000","last_undegraded":"2023-10-08T01:15:21.773432+0000","last_fullsized":"2023-10-08T01:15:21.773432+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:15.250420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.795132+0000","last_change":"2023-10-08T01:15:21.790228+0000","last_active":"2023-10-08T01:15:21.795132+0000","last_peered":"2023-10-08T01:15:21.795132+0000","last_clean":"2023-10-08T01:15:21.795132+0000","last_became_active":"2023-10-08T01:15:21.789949+0000","last_became_peered":"2023-10-08T01:15:21.789949+0000","last_unstale":"2023-10-08T01:15:21.795132+0000","last_undegraded":"2023-10-08T01:15:21.795132+0000","last_fullsized":"2023-10-08T01:15:21.795132+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:02.399034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.791981+0000","last_change":"2023-10-08T01:15:20.764782+0000","last_active":"2023-10-08T01:15:21.791981+0000","last_peered":"2023-10-08T01:15:21.791981+0000","last_clean":"2023-10-08T01:15:21.791981+0000","last_became_active":"2023-10-08T01:15:20.764156+0000","last_became_peered":"2023-10-08T01:15:20.764156+0000","last_unstale":"2023-10-08T01:15:21.791981+0000","last_undegraded":"2023-10-08T01:15:21.791981+0000","last_fullsized":"2023-10-08T01:15:21.791981+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:22.379487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789541+0000","last_change":"2023-10-08T01:15:21.781595+0000","last_active":"2023-10-08T01:15:21.789541+0000","last_peered":"2023-10-08T01:15:21.789541+0000","last_clean":"2023-10-08T01:15:21.789541+0000","last_became_active":"2023-10-08T01:15:21.781058+0000","last_became_peered":"2023-10-08T01:15:21.781058+0000","last_unstale":"2023-10-08T01:15:21.789541+0000","last_undegraded":"2023-10-08T01:15:21.789541+0000","last_fullsized":"2023-10-08T01:15:21.789541+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:55:57.721830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767374+0000","last_change":"2023-10-08T01:15:20.766456+0000","last_active":"2023-10-08T01:15:21.767374+0000","last_peered":"2023-10-08T01:15:21.767374+0000","last_clean":"2023-10-08T01:15:21.767374+0000","last_became_active":"2023-10-08T01:15:20.766166+0000","last_became_peered":"2023-10-08T01:15:20.766166+0000","last_unstale":"2023-10-08T01:15:21.767374+0000","last_undegraded":"2023-10-08T01:15:21.767374+0000","last_fullsized":"2023-10-08T01:15:21.767374+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:12.990214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792127+0000","last_change":"2023-10-08T01:15:21.786289+0000","last_active":"2023-10-08T01:15:21.792127+0000","last_peered":"2023-10-08T01:15:21.792127+0000","last_clean":"2023-10-08T01:15:21.792127+0000","last_became_active":"2023-10-08T01:15:21.786033+0000","last_became_peered":"2023-10-08T01:15:21.786033+0000","last_unstale":"2023-10-08T01:15:21.792127+0000","last_undegraded":"2023-10-08T01:15:21.792127+0000","last_fullsized":"2023-10-08T01:15:21.792127+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:47.258669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.774200+0000","last_change":"2023-10-08T01:15:20.761046+0000","last_active":"2023-10-08T01:15:21.774200+0000","last_peered":"2023-10-08T01:15:21.774200+0000","last_clean":"2023-10-08T01:15:21.774200+0000","last_became_active":"2023-10-08T01:15:20.760786+0000","last_became_peered":"2023-10-08T01:15:20.760786+0000","last_unstale":"2023-10-08T01:15:21.774200+0000","last_undegraded":"2023-10-08T01:15:21.774200+0000","last_fullsized":"2023-10-08T01:15:21.774200+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:13:12.990214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.794366+0000","last_change":"2023-10-08T01:15:21.788597+0000","last_active":"2023-10-08T01:15:21.794366+0000","last_peered":"2023-10-08T01:15:21.794366+0000","last_clean":"2023-10-08T01:15:21.794366+0000","last_became_active":"2023-10-08T01:15:21.788354+0000","last_became_peered":"2023-10-08T01:15:21.788354+0000","last_unstale":"2023-10-08T01:15:21.794366+0000","last_undegraded":"2023-10-08T01:15:21.794366+0000","last_fullsized":"2023-10-08T01:15:21.794366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:18.981769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.768898+0000","last_change":"2023-10-08T01:15:20.760877+0000","last_active":"2023-10-08T01:15:21.768898+0000","last_peered":"2023-10-08T01:15:21.768898+0000","last_clean":"2023-10-08T01:15:21.768898+0000","last_became_active":"2023-10-08T01:15:20.760150+0000","last_became_peered":"2023-10-08T01:15:20.760150+0000","last_unstale":"2023-10-08T01:15:21.768898+0000","last_undegraded":"2023-10-08T01:15:21.768898+0000","last_fullsized":"2023-10-08T01:15:21.768898+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:29:46.505077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.790999+0000","last_change":"2023-10-08T01:15:21.788077+0000","last_active":"2023-10-08T01:15:21.790999+0000","last_peered":"2023-10-08T01:15:21.790999+0000","last_clean":"2023-10-08T01:15:21.790999+0000","last_became_active":"2023-10-08T01:15:21.787828+0000","last_became_peered":"2023-10-08T01:15:21.787828+0000","last_unstale":"2023-10-08T01:15:21.790999+0000","last_undegraded":"2023-10-08T01:15:21.790999+0000","last_fullsized":"2023-10-08T01:15:21.790999+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:54:47.586067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"134'2","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.802044+0000","last_change":"2023-10-08T01:15:20.753870+0000","last_active":"2023-10-08T01:15:21.802044+0000","last_peered":"2023-10-08T01:15:21.802044+0000","last_clean":"2023-10-08T01:15:21.802044+0000","last_became_active":"2023-10-08T01:15:20.753614+0000","last_became_peered":"2023-10-08T01:15:20.753614+0000","last_unstale":"2023-10-08T01:15:21.802044+0000","last_undegraded":"2023-10-08T01:15:21.802044+0000","last_fullsized":"2023-10-08T01:15:21.802044+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:04:26.643549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799000+0000","last_change":"2023-10-08T01:15:21.791040+0000","last_active":"2023-10-08T01:15:21.799000+0000","last_peered":"2023-10-08T01:15:21.799000+0000","last_clean":"2023-10-08T01:15:21.799000+0000","last_became_active":"2023-10-08T01:15:21.790605+0000","last_became_peered":"2023-10-08T01:15:21.790605+0000","last_unstale":"2023-10-08T01:15:21.799000+0000","last_undegraded":"2023-10-08T01:15:21.799000+0000","last_fullsized":"2023-10-08T01:15:21.799000+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:08:21.065314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"135'1","reported_seq":11,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-10-08T01:15:22.783291+0000","last_change":"2023-10-08T01:15:20.753955+0000","last_active":"2023-10-08T01:15:22.783291+0000","last_peered":"2023-10-08T01:15:22.783291+0000","last_clean":"2023-10-08T01:15:22.783291+0000","last_became_active":"2023-10-08T01:15:20.753682+0000","last_became_peered":"2023-10-08T01:15:20.753682+0000","last_unstale":"2023-10-08T01:15:22.783291+0000","last_undegraded":"2023-10-08T01:15:22.783291+0000","last_fullsized":"2023-10-08T01:15:22.783291+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:07.797723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.789592+0000","last_change":"2023-10-08T01:15:21.784259+0000","last_active":"2023-10-08T01:15:21.789592+0000","last_peered":"2023-10-08T01:15:21.789592+0000","last_clean":"2023-10-08T01:15:21.789592+0000","last_became_active":"2023-10-08T01:15:21.784008+0000","last_became_peered":"2023-10-08T01:15:21.784008+0000","last_unstale":"2023-10-08T01:15:21.789592+0000","last_undegraded":"2023-10-08T01:15:21.789592+0000","last_fullsized":"2023-10-08T01:15:21.789592+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:50:52.880022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.768465+0000","last_change":"2023-10-08T01:15:20.755101+0000","last_active":"2023-10-08T01:15:21.768465+0000","last_peered":"2023-10-08T01:15:21.768465+0000","last_clean":"2023-10-08T01:15:21.768465+0000","last_became_active":"2023-10-08T01:15:20.754832+0000","last_became_peered":"2023-10-08T01:15:20.754832+0000","last_unstale":"2023-10-08T01:15:21.768465+0000","last_undegraded":"2023-10-08T01:15:21.768465+0000","last_fullsized":"2023-10-08T01:15:21.768465+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:48:46.252481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.785152+0000","last_change":"2023-10-08T01:15:21.777061+0000","last_active":"2023-10-08T01:15:21.785152+0000","last_peered":"2023-10-08T01:15:21.785152+0000","last_clean":"2023-10-08T01:15:21.785152+0000","last_became_active":"2023-10-08T01:15:21.776708+0000","last_became_peered":"2023-10-08T01:15:21.776708+0000","last_unstale":"2023-10-08T01:15:21.785152+0000","last_undegraded":"2023-10-08T01:15:21.785152+0000","last_fullsized":"2023-10-08T01:15:21.785152+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:39:01.530176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.777731+0000","last_change":"2023-10-08T01:15:20.764641+0000","last_active":"2023-10-08T01:15:21.777731+0000","last_peered":"2023-10-08T01:15:21.777731+0000","last_clean":"2023-10-08T01:15:21.777731+0000","last_became_active":"2023-10-08T01:15:20.764364+0000","last_became_peered":"2023-10-08T01:15:20.764364+0000","last_unstale":"2023-10-08T01:15:21.777731+0000","last_undegraded":"2023-10-08T01:15:21.777731+0000","last_fullsized":"2023-10-08T01:15:21.777731+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:49:31.099680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.784968+0000","last_change":"2023-10-08T01:15:21.778266+0000","last_active":"2023-10-08T01:15:21.784968+0000","last_peered":"2023-10-08T01:15:21.784968+0000","last_clean":"2023-10-08T01:15:21.784968+0000","last_became_active":"2023-10-08T01:15:21.777903+0000","last_became_peered":"2023-10-08T01:15:21.777903+0000","last_unstale":"2023-10-08T01:15:21.784968+0000","last_undegraded":"2023-10-08T01:15:21.784968+0000","last_fullsized":"2023-10-08T01:15:21.784968+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:31:18.981769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.773698+0000","last_change":"2023-10-08T01:15:20.758963+0000","last_active":"2023-10-08T01:15:21.773698+0000","last_peered":"2023-10-08T01:15:21.773698+0000","last_clean":"2023-10-08T01:15:21.773698+0000","last_became_active":"2023-10-08T01:15:20.757575+0000","last_became_peered":"2023-10-08T01:15:20.757575+0000","last_unstale":"2023-10-08T01:15:21.773698+0000","last_undegraded":"2023-10-08T01:15:21.773698+0000","last_fullsized":"2023-10-08T01:15:21.773698+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:29.341500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.785051+0000","last_change":"2023-10-08T01:15:21.778917+0000","last_active":"2023-10-08T01:15:21.785051+0000","last_peered":"2023-10-08T01:15:21.785051+0000","last_clean":"2023-10-08T01:15:21.785051+0000","last_became_active":"2023-10-08T01:15:21.778629+0000","last_became_peered":"2023-10-08T01:15:21.778629+0000","last_unstale":"2023-10-08T01:15:21.785051+0000","last_undegraded":"2023-10-08T01:15:21.785051+0000","last_fullsized":"2023-10-08T01:15:21.785051+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:29:09.371938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.769452+0000","last_change":"2023-10-08T01:15:20.764672+0000","last_active":"2023-10-08T01:15:21.769452+0000","last_peered":"2023-10-08T01:15:21.769452+0000","last_clean":"2023-10-08T01:15:21.769452+0000","last_became_active":"2023-10-08T01:15:20.764379+0000","last_became_peered":"2023-10-08T01:15:20.764379+0000","last_unstale":"2023-10-08T01:15:21.769452+0000","last_undegraded":"2023-10-08T01:15:21.769452+0000","last_fullsized":"2023-10-08T01:15:21.769452+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:19:07.797723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.799423+0000","last_change":"2023-10-08T01:15:21.795609+0000","last_active":"2023-10-08T01:15:21.799423+0000","last_peered":"2023-10-08T01:15:21.799423+0000","last_clean":"2023-10-08T01:15:21.799423+0000","last_became_active":"2023-10-08T01:15:21.795246+0000","last_became_peered":"2023-10-08T01:15:21.795246+0000","last_unstale":"2023-10-08T01:15:21.799423+0000","last_undegraded":"2023-10-08T01:15:21.799423+0000","last_fullsized":"2023-10-08T01:15:21.799423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:15:34.228457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.767342+0000","last_change":"2023-10-08T01:15:20.764516+0000","last_active":"2023-10-08T01:15:21.767342+0000","last_peered":"2023-10-08T01:15:21.767342+0000","last_clean":"2023-10-08T01:15:21.767342+0000","last_became_active":"2023-10-08T01:15:20.764252+0000","last_became_peered":"2023-10-08T01:15:20.764252+0000","last_unstale":"2023-10-08T01:15:21.767342+0000","last_undegraded":"2023-10-08T01:15:21.767342+0000","last_fullsized":"2023-10-08T01:15:21.767342+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T01:23:11.456448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.793569+0000","last_change":"2023-10-08T01:15:21.784243+0000","last_active":"2023-10-08T01:15:21.793569+0000","last_peered":"2023-10-08T01:15:21.793569+0000","last_clean":"2023-10-08T01:15:21.793569+0000","last_became_active":"2023-10-08T01:15:21.783987+0000","last_became_peered":"2023-10-08T01:15:21.783987+0000","last_unstale":"2023-10-08T01:15:21.793569+0000","last_undegraded":"2023-10-08T01:15:21.793569+0000","last_fullsized":"2023-10-08T01:15:21.793569+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:57:59.430267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.769085+0000","last_change":"2023-10-08T01:15:20.760896+0000","last_active":"2023-10-08T01:15:21.769085+0000","last_peered":"2023-10-08T01:15:21.769085+0000","last_clean":"2023-10-08T01:15:21.769085+0000","last_became_active":"2023-10-08T01:15:20.760585+0000","last_became_peered":"2023-10-08T01:15:20.760585+0000","last_unstale":"2023-10-08T01:15:21.769085+0000","last_undegraded":"2023-10-08T01:15:21.769085+0000","last_fullsized":"2023-10-08T01:15:21.769085+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:32.801443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.792491+0000","last_change":"2023-10-08T01:15:21.784922+0000","last_active":"2023-10-08T01:15:21.792491+0000","last_peered":"2023-10-08T01:15:21.792491+0000","last_clean":"2023-10-08T01:15:21.792491+0000","last_became_active":"2023-10-08T01:15:21.784546+0000","last_became_peered":"2023-10-08T01:15:21.784546+0000","last_unstale":"2023-10-08T01:15:21.792491+0000","last_undegraded":"2023-10-08T01:15:21.792491+0000","last_fullsized":"2023-10-08T01:15:21.792491+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:20.737836+0000","last_clean_scrub_stamp":"2023-10-08T01:15:20.737836+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:29:47.511265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.775328+0000","last_change":"2023-10-08T01:15:20.764695+0000","last_active":"2023-10-08T01:15:21.775328+0000","last_peered":"2023-10-08T01:15:21.775328+0000","last_clean":"2023-10-08T01:15:21.775328+0000","last_became_active":"2023-10-08T01:15:20.764421+0000","last_became_peered":"2023-10-08T01:15:20.764421+0000","last_unstale":"2023-10-08T01:15:21.775328+0000","last_undegraded":"2023-10-08T01:15:21.775328+0000","last_fullsized":"2023-10-08T01:15:21.775328+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T01:15:19.731648+0000","last_clean_scrub_stamp":"2023-10-08T01:15:19.731648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:43:17.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"124'249","reported_seq":727,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-10-08T01:15:21.772088+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T01:15:21.772088+0000","last_peered":"2023-10-08T01:15:21.772088+0000","last_clean":"2023-10-08T01:15:21.772088+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T01:15:21.772088+0000","last_undegraded":"2023-10-08T01:15:21.772088+0000","last_fullsized":"2023-10-08T01:15:21.772088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":249,"log_dups_size":0,"ondisk_log_size":249,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1268,"num_read_kb":2572,"num_write":399,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":249,"ondisk_log_size":249,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575080,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307704,"kb_used_data":2224,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064136,"statfs":{"total":96636764160,"available":96321675264,"internally_reserved":0,"allocated":2277376,"data_stored":1415305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.69599999999999995},"min":{"1min":0.443,"5min":0.38600000000000001,"15min":0.36399999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.123},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.69799999999999995},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82999999999999996,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.88600000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.68799999999999994},"min":{"1min":0.50600000000000001,"5min":0.45200000000000001,"15min":0.42899999999999999},"max":{"1min":0.85399999999999998,"5min":0.95499999999999996,"15min":0.97699999999999998},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.68500000000000005,"15min":0.71399999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.39200000000000002},"max":{"1min":0.92200000000000004,"5min":0.93600000000000005,"15min":1.0269999999999999},"last":0.748}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.52600000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.87,"5min":1.133,"15min":1.133},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.66500000000000004,"15min":0.71399999999999997},"min":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.80600000000000005,"5min":0.95099999999999996,"15min":1.167},"last":0.73499999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.66100000000000003,"15min":0.69899999999999995},"min":{"1min":0.50800000000000001,"5min":0.43099999999999999,"15min":0.41499999999999998},"max":{"1min":0.89300000000000002,"5min":0.93200000000000005,"15min":0.94699999999999995},"last":0.872},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.501,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":1.052,"15min":1.079},"last":0.77400000000000002}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.57999999999999996},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79700000000000004,"5min":0.83299999999999996,"15min":1.0069999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.40799999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72499999999999998,"5min":0.875,"15min":0.88300000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.62},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84899999999999998,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.63200000000000001,"15min":0.65000000000000002},"min":{"1min":0.46000000000000002,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.82299999999999995,"5min":1.004,"15min":1.004},"last":0.628}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71799999999999997,"15min":0.746},"min":{"1min":0.47699999999999998,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.86099999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.68500000000000005,"15min":0.69399999999999995},"min":{"1min":0.47099999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86499999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575080,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303024,"kb_used_data":1640,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068816,"statfs":{"total":96636764160,"available":96326467584,"internally_reserved":0,"allocated":1679360,"data_stored":824419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67800000000000005,"15min":0.69399999999999995},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.36699999999999999},"max":{"1min":0.93200000000000005,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.67800000000000005,"15min":0.69099999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.91200000000000003,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.875}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70599999999999996,"15min":0.72399999999999998},"min":{"1min":0.45300000000000001,"5min":0.39900000000000002,"15min":0.39000000000000001},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":2.984},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.68300000000000005,"15min":0.71699999999999997},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.38100000000000001},"max":{"1min":0.86799999999999999,"5min":1.004,"15min":3.0190000000000001},"last":0.83299999999999996}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67700000000000005,"15min":0.68899999999999995},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.71799999999999997,"15min":0.72499999999999998},"min":{"1min":0.53300000000000003,"5min":0.44,"15min":0.44},"max":{"1min":0.95999999999999996,"5min":0.998,"15min":0.998},"last":0.91800000000000004}]},{"osd":3,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72499999999999998,"15min":0.73899999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.47099999999999997},"max":{"1min":0.94599999999999995,"5min":0.96699999999999997,"15min":0.99199999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71699999999999997,"15min":0.73299999999999998},"min":{"1min":0.56100000000000005,"5min":0.501,"15min":0.45900000000000002},"max":{"1min":0.92400000000000004,"5min":1.048,"15min":1.048},"last":0.93799999999999994}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.62,"15min":0.60199999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.88500000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.75},{"interface":"front","average":{"1min":0.623,"5min":0.622,"15min":0.623},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.79000000000000004,"5min":0.89800000000000002,"15min":1.087},"last":0.72899999999999998}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63700000000000001,"15min":0.629},"min":{"1min":0.42899999999999999,"5min":0.42299999999999999,"15min":0.38},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.395},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.101,"5min":1.101,"15min":1.21},"last":0.81100000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71699999999999997,"15min":0.71399999999999997},"min":{"1min":0.45800000000000002,"5min":0.44500000000000001,"15min":0.44},"max":{"1min":0.93700000000000006,"5min":1.095,"15min":2.9489999999999998},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.746,"15min":0.73699999999999999},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.45200000000000001},"max":{"1min":0.92900000000000005,"5min":1.075,"15min":1.329},"last":0.77300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575080,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":824473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.45200000000000001,"15min":0.46500000000000002},"min":{"1min":0.29999999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56100000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.35699999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.61199999999999999,"5min":0.76600000000000001,"15min":0.98799999999999999},"last":0.32800000000000001}]},{"osd":2,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.45900000000000002,"15min":0.47599999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.26700000000000002},"max":{"1min":0.61099999999999999,"5min":0.70599999999999996,"15min":0.72499999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.53200000000000003},"min":{"1min":0.33000000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.84799999999999998},"last":0.40500000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55300000000000005,"15min":0.57199999999999995},"min":{"1min":0.438,"5min":0.38300000000000001,"15min":0.35399999999999998},"max":{"1min":0.67800000000000005,"5min":0.78000000000000003,"15min":0.81899999999999995},"last":0.442},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.51300000000000001,"15min":0.53900000000000003},"min":{"1min":0.36099999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72399999999999998,"5min":0.78400000000000003,"15min":1.5449999999999999},"last":0.42099999999999999}]},{"osd":4,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.55500000000000005,"5min":0.495,"15min":0.41299999999999998},"max":{"1min":0.997,"5min":1.296,"15min":1.452},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72599999999999998,"15min":0.75600000000000001},"min":{"1min":0.46300000000000002,"5min":0.46200000000000002,"15min":0.434},"max":{"1min":1.099,"5min":1.137,"15min":1.1839999999999999},"last":0.61099999999999999}]},{"osd":5,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.71699999999999997,"15min":0.75900000000000001},"min":{"1min":0.50900000000000001,"5min":0.5,"15min":0.48799999999999999},"max":{"1min":0.92300000000000004,"5min":1.119,"15min":1.3959999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.77700000000000002},"min":{"1min":0.53200000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0169999999999999,"5min":1.169,"15min":1.5209999999999999},"last":0.622}]},{"osd":6,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74299999999999999,"15min":0.78300000000000003},"min":{"1min":0.56399999999999995,"5min":0.54600000000000004,"15min":0.48399999999999999},"max":{"1min":1.077,"5min":1.077,"15min":1.6020000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73199999999999998,"15min":0.77700000000000002},"min":{"1min":0.60899999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.046,"5min":1.1499999999999999,"15min":1.3660000000000001},"last":0.64800000000000002}]},{"osd":7,"last update":"Sun Oct 8 01:15:13 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.77400000000000002,"15min":0.81799999999999995},"min":{"1min":0.63500000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.96699999999999997,"5min":1.236,"15min":1.621},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.60599999999999998,"5min":0.59099999999999997,"15min":0.51300000000000001},"max":{"1min":1.0389999999999999,"5min":1.21,"15min":1.4890000000000001},"last":0.66100000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575079,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303620,"kb_used_data":2236,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068220,"statfs":{"total":96636764160,"available":96325857280,"internally_reserved":0,"allocated":2289664,"data_stored":1416365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44900000000000001,"5min":0.45500000000000002,"15min":0.46200000000000002},"min":{"1min":0.28199999999999997,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.56499999999999995,"5min":0.95999999999999996,"15min":0.997},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48799999999999999,"15min":0.49399999999999999},"min":{"1min":0.33300000000000002,"5min":0.315,"15min":0.28599999999999998},"max":{"1min":0.69699999999999995,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46999999999999997,"15min":0.47899999999999998},"min":{"1min":0.312,"5min":0.28399999999999997,"15min":0.28100000000000003},"max":{"1min":0.63500000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.51200000000000001,"15min":0.52700000000000002},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.59099999999999997,"5min":0.83699999999999997,"15min":1.7370000000000001},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53500000000000003,"15min":0.54600000000000004},"min":{"1min":0.36299999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57599999999999996,"15min":0.59399999999999997},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.68200000000000005,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.66100000000000003}]},{"osd":4,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.53600000000000003,"5min":0.51100000000000001,"15min":0.503},"max":{"1min":1.006,"5min":1.341,"15min":1.617},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.77100000000000002},"min":{"1min":0.55800000000000005,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.88500000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.755}]},{"osd":5,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77000000000000002,"15min":0.78600000000000003},"min":{"1min":0.53800000000000003,"5min":0.498,"15min":0.498},"max":{"1min":0.95999999999999996,"5min":2.085,"15min":2.085},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.79000000000000004,"15min":0.81100000000000005},"min":{"1min":0.57599999999999996,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.014,"5min":1.381,"15min":1.5289999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.77100000000000002,"15min":0.79500000000000004},"min":{"1min":0.52800000000000002,"5min":0.51400000000000001,"15min":0.50900000000000001},"max":{"1min":0.877,"5min":1.361,"15min":1.3660000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.76200000000000001,"15min":0.78000000000000003},"min":{"1min":0.56000000000000005,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94999999999999996,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.84799999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:15:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.78400000000000003,"15min":0.80700000000000005},"min":{"1min":0.56699999999999995,"5min":0.53300000000000003,"15min":0.51700000000000002},"max":{"1min":1.054,"5min":1.3260000000000001,"15min":1.5740000000000001},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.80800000000000005,"15min":0.83199999999999996},"min":{"1min":0.58399999999999996,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.93700000000000006,"5min":1.2170000000000001,"15min":1.298},"last":0.82899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575079,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311212,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060628,"statfs":{"total":96636764160,"available":96318083072,"internally_reserved":0,"allocated":1675264,"data_stored":824349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.42999999999999999,"15min":0.45200000000000001},"min":{"1min":0.29199999999999998,"5min":0.27200000000000002,"15min":0.252},"max":{"1min":0.53700000000000003,"5min":0.72299999999999998,"15min":0.752},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.48199999999999998,"15min":0.50600000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.63500000000000001,"5min":0.752,"15min":2.2930000000000001},"last":0.56299999999999994}]},{"osd":1,"last update":"Sun Oct 8 01:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46100000000000002,"15min":0.48099999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28799999999999998},"max":{"1min":0.56999999999999995,"5min":0.78100000000000003,"15min":2.786},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52300000000000002,"15min":0.54500000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.753,"5min":0.79600000000000004,"15min":3.3180000000000001},"last":0.65800000000000003}]},{"osd":3,"last update":"Sun Oct 8 01:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.59299999999999997},"min":{"1min":0.375,"5min":0.37,"15min":0.37},"max":{"1min":0.65200000000000002,"5min":0.84099999999999997,"15min":0.98399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53200000000000003,"15min":0.55300000000000005},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.34999999999999998},"max":{"1min":0.69399999999999995,"5min":0.78700000000000003,"15min":2.2170000000000001},"last":0.59399999999999997}]},{"osd":4,"last update":"Sun Oct 8 01:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.73699999999999999,"15min":0.75800000000000001},"min":{"1min":0.54500000000000004,"5min":0.503,"15min":0.503},"max":{"1min":1.0089999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.76800000000000002,"15min":0.79000000000000004},"min":{"1min":0.59899999999999998,"5min":0.46100000000000002,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":1.23,"15min":2.3559999999999999},"last":0.92600000000000005}]},{"osd":5,"last update":"Sun Oct 8 01:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78600000000000003,"15min":0.82099999999999995},"min":{"1min":0.61799999999999999,"5min":0.499,"15min":0.499},"max":{"1min":1.0189999999999999,"5min":1.1379999999999999,"15min":2.488},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78800000000000003,"15min":0.81399999999999995},"min":{"1min":0.63600000000000001,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96099999999999997,"5min":1.284,"15min":2.7080000000000002},"last":0.88600000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.76400000000000001,"15min":0.79200000000000004},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.48499999999999999},"max":{"1min":0.96999999999999997,"5min":1.2290000000000001,"15min":2.653},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.754,"5min":0.77400000000000002,"15min":0.79200000000000004},"min":{"1min":0.59999999999999998,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.96599999999999997,"5min":1.2569999999999999,"15min":2.1379999999999999},"last":0.81999999999999995}]},{"osd":7,"last update":"Sun Oct 8 01:15:29 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.76900000000000002,"15min":0.79200000000000004},"min":{"1min":0.55000000000000004,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.97799999999999998,"5min":1.0289999999999999,"15min":2.5550000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.78200000000000003,"15min":0.80900000000000005},"min":{"1min":0.53500000000000003,"5min":0.51800000000000002,"15min":0.504},"max":{"1min":0.99399999999999999,"5min":1.204,"15min":2.7490000000000001},"last":0.77300000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575079,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307132,"kb_used_data":1652,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064708,"statfs":{"total":96636764160,"available":96322260992,"internally_reserved":0,"allocated":1691648,"data_stored":825955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.42399999999999999,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.27200000000000002},"max":{"1min":0.56100000000000005,"5min":0.627,"15min":0.80100000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.46200000000000002,"15min":0.47899999999999998},"min":{"1min":0.307,"5min":0.28699999999999998,"15min":0.27900000000000003},"max":{"1min":0.60999999999999999,"5min":0.69099999999999995,"15min":0.81200000000000006},"last":0.40999999999999998}]},{"osd":1,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.28299999999999997},"max":{"1min":0.70099999999999996,"5min":0.81799999999999995,"15min":0.89700000000000002},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48599999999999999,"15min":0.51300000000000001},"min":{"1min":0.38600000000000001,"5min":0.371,"15min":0.31},"max":{"1min":0.57899999999999996,"5min":0.86599999999999999,"15min":0.88100000000000001},"last":0.58899999999999997}]},{"osd":2,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57699999999999996,"15min":0.58599999999999997},"min":{"1min":0.39000000000000001,"5min":0.38500000000000001,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.877,"15min":0.89700000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60099999999999998,"15min":0.62},"min":{"1min":0.46000000000000002,"5min":0.44400000000000001,"15min":0.39600000000000002},"max":{"1min":0.77000000000000002,"5min":0.94099999999999995,"15min":1.1060000000000001},"last":0.629}]},{"osd":4,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70599999999999996,"15min":0.74399999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.41099999999999998},"max":{"1min":0.98899999999999999,"5min":1.095,"15min":1.3029999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72799999999999998,"15min":0.76700000000000002},"min":{"1min":0.55500000000000005,"5min":0.49399999999999999,"15min":0.46000000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.413},"last":0.76200000000000001}]},{"osd":5,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.72399999999999998,"15min":0.76400000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.032,"5min":1.0920000000000001,"15min":1.3580000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.53300000000000003,"5min":0.47899999999999998,"15min":0.45900000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.089},"last":0.77500000000000002}]},{"osd":6,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.748,"15min":0.78600000000000003},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.47999999999999998},"max":{"1min":1.0680000000000001,"5min":1.101,"15min":1.379},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.57799999999999996,"5min":0.48199999999999998,"15min":0.44500000000000001},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.486},"last":0.73299999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:15:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.746,"15min":0.78400000000000003},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.46500000000000002},"max":{"1min":1.0549999999999999,"5min":1.0840000000000001,"15min":1.389},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.74099999999999999,"15min":0.78500000000000003},"min":{"1min":0.58999999999999997,"5min":0.55300000000000005,"15min":0.50700000000000001},"max":{"1min":0.94899999999999995,"5min":1.054,"15min":1.4730000000000001},"last":0.80600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575079,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311212,"kb_used_data":1636,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060628,"statfs":{"total":96636764160,"available":96318083072,"internally_reserved":0,"allocated":1675264,"data_stored":824349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.65300000000000002,"15min":0.67600000000000005},"min":{"1min":0.42699999999999999,"5min":0.40400000000000003,"15min":0.30099999999999999},"max":{"1min":0.85499999999999998,"5min":0.999,"15min":0.999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.63,"5min":0.66900000000000004,"15min":0.69499999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.872,"5min":0.96599999999999997,"15min":1.0049999999999999},"last":0.63300000000000001}]},{"osd":1,"last update":"Sun Oct 8 01:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.45500000000000002,"5min":0.40100000000000002,"15min":0.39300000000000002},"max":{"1min":0.878,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.67500000000000004,"15min":0.70299999999999996},"min":{"1min":0.54200000000000004,"5min":0.46899999999999997,"15min":0.42099999999999999},"max":{"1min":0.84799999999999998,"5min":1.04,"15min":1.04},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Oct 8 01:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.69899999999999995,"15min":0.73199999999999998},"min":{"1min":0.51800000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.73099999999999998,"15min":0.76000000000000001},"min":{"1min":0.53300000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.323},"last":0.71999999999999997}]},{"osd":3,"last update":"Sun Oct 8 01:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.71099999999999997,"15min":0.751},"min":{"1min":0.56599999999999995,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93200000000000005,"5min":1.0760000000000001,"15min":1.4770000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.75,"15min":0.78400000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.454},"last":0.753}]},{"osd":5,"last update":"Sun Oct 8 01:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60899999999999999,"15min":0.60799999999999998},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.98399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72799999999999998,"15min":0.73099999999999998},"min":{"1min":0.55900000000000005,"5min":0.502,"15min":0.42399999999999999},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.998},"last":0.76500000000000001}]},{"osd":6,"last update":"Sun Oct 8 01:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65200000000000002,"15min":0.64400000000000002},"min":{"1min":0.439,"5min":0.439,"15min":0.40799999999999997},"max":{"1min":0.88300000000000001,"5min":0.93799999999999994,"15min":1.3620000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66400000000000003,"15min":0.65400000000000003},"min":{"1min":0.438,"5min":0.41299999999999998,"15min":0.40300000000000002},"max":{"1min":0.89300000000000002,"5min":0.95399999999999996,"15min":1.4039999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Sun Oct 8 01:15:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.67500000000000004,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.44800000000000001,"15min":0.443},"max":{"1min":0.997,"5min":0.997,"15min":1.7490000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.70899999999999996,"15min":0.71199999999999997},"min":{"1min":0.50800000000000001,"5min":0.503,"15min":0.48299999999999998},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.2450000000000001},"last":0.77800000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575079,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303028,"kb_used_data":1644,"kb_used_omap":0,"kb_used_meta":301376,"kb_avail":94068812,"statfs":{"total":96636764160,"available":96326463488,"internally_reserved":0,"allocated":1683456,"data_stored":824925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308609024},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.66900000000000004,"15min":0.68500000000000005},"min":{"1min":0.46700000000000003,"5min":0.45000000000000001,"15min":0.34699999999999998},"max":{"1min":0.89700000000000002,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.70699999999999996,"15min":0.71799999999999997},"min":{"1min":0.56499999999999995,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90200000000000002,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.754}]},{"osd":1,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69599999999999995,"15min":0.71099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.41699999999999998},"max":{"1min":1.018,"5min":1.026,"15min":1.026},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.67700000000000005,"15min":0.69399999999999995},"min":{"1min":0.47999999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81699999999999995,"5min":0.92000000000000004,"15min":0.99099999999999999},"last":0.66700000000000004}]},{"osd":2,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.68799999999999994,"15min":0.70999999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.92000000000000004,"15min":0.93500000000000005},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.71099999999999997,"15min":0.73299999999999998},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.96599999999999997,"5min":1.026,"15min":1.026},"last":0.68100000000000005}]},{"osd":3,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.71199999999999997,"15min":0.73099999999999998},"min":{"1min":0.46300000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94199999999999995,"5min":1.004,"15min":1.004},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.74299999999999999},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92200000000000004,"5min":1.034,"15min":1.034},"last":0.72299999999999998}]},{"osd":4,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57899999999999996,"15min":0.59299999999999997},"min":{"1min":0.42599999999999999,"5min":0.40899999999999997,"15min":0.35599999999999998},"max":{"1min":0.83999999999999997,"5min":0.874,"15min":0.90900000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.44,"5min":0.43099999999999999,"15min":0.42199999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":1.0680000000000001},"last":0.505}]},{"osd":6,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":0.97499999999999998,"15min":1.0669999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66200000000000003,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.44700000000000001,"15min":0.41699999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47999999999999998}]},{"osd":7,"last update":"Sun Oct 8 01:15:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.68200000000000005,"15min":0.69099999999999995},"min":{"1min":0.58699999999999997,"5min":0.432,"15min":0.432},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.095},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70499999999999996,"15min":0.71599999999999997},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.92000000000000004,"5min":1.05,"15min":1.05},"last":0.77000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T01:15:31.825 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T01:15:31.825 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T01:15:32.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.042+0000 7f02979ff700 1 Processor -- start 2023-10-08T01:15:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.042+0000 7f02979ff700 1 -- start start 2023-10-08T01:15:32.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f02979ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02901569f0 0x7f02900c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f02979ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029015a6f0 con 0x7f0290156620 2023-10-08T01:15:32.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f02979ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029015a9d0 con 0x7f0290068390 2023-10-08T01:15:32.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f02979ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900c8d30 con 0x7f02901569f0 2023-10-08T01:15:32.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f029579b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0290156620 0x7f02900ace70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42634/0 (socket says 172.21.15.119:42634) 2023-10-08T01:15:32.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f029579b700 1 -- 172.21.15.119:0/2332543255 learned_addr learned my addr 172.21.15.119:0/2332543255 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:32.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f0294f9a700 1 --2- 172.21.15.119:0/2332543255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02901569f0 0x7f02900c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f0287fff700 1 -- 172.21.15.119:0/2332543255 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305584838 0 0) 0x7f029015a6f0 con 0x7f0290156620 2023-10-08T01:15:32.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.044+0000 7f0287fff700 1 -- 172.21.15.119:0/2332543255 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f027c003570 con 0x7f0290156620 2023-10-08T01:15:32.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0287fff700 1 -- 172.21.15.119:0/2332543255 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 77958665 0 0) 0x7f029015a9d0 con 0x7f0290068390 2023-10-08T01:15:32.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0287fff700 1 -- 172.21.15.119:0/2332543255 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f029015a6f0 con 0x7f0290068390 2023-10-08T01:15:32.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0287fff700 1 -- 172.21.15.119:0/2332543255 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 764815445 0 0) 0x7f027c003570 con 0x7f0290156620 2023-10-08T01:15:32.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0294f9a700 1 -- 172.21.15.119:0/2332543255 >> v1:172.21.15.184:6789/0 conn(0x7f0290068390 legacy=0x7f029006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0294f9a700 1 -- 172.21.15.119:0/2332543255 >> v1:172.21.15.119:6789/0 conn(0x7f0290156620 legacy=0x7f02900ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0294f9a700 1 -- 172.21.15.119:0/2332543255 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029015acb0 con 0x7f02901569f0 2023-10-08T01:15:32.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0294f9a700 1 --2- 172.21.15.119:0/2332543255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02901569f0 0x7f02900c87f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0280009910 tx=0x7f02800091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4e8b888c1b6ede5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:32.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0287fff700 1 -- 172.21.15.119:0/2332543255 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c001470 con 0x7f02901569f0 2023-10-08T01:15:32.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0287fff700 1 -- 172.21.15.119:0/2332543255 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0280011420 con 0x7f02901569f0 2023-10-08T01:15:32.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.045+0000 7f0287fff700 1 -- 172.21.15.119:0/2332543255 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800117c0 con 0x7f02901569f0 2023-10-08T01:15:32.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.046+0000 7f02979ff700 1 -- 172.21.15.119:0/2332543255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02901569f0 msgr2=0x7f02900c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.046+0000 7f02979ff700 1 --2- 172.21.15.119:0/2332543255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02901569f0 0x7f02900c87f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0280009910 tx=0x7f02800091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.046+0000 7f02979ff700 1 -- 172.21.15.119:0/2332543255 shutdown_connections 2023-10-08T01:15:32.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.046+0000 7f02979ff700 1 --2- 172.21.15.119:0/2332543255 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02901569f0 0x7f02900c87f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.046+0000 7f02979ff700 1 -- 172.21.15.119:0/2332543255 >> 172.21.15.119:0/2332543255 conn(0x7f02900cc540 msgr2=0x7f02900cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:32.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.046+0000 7f02979ff700 1 -- 172.21.15.119:0/2332543255 shutdown_connections 2023-10-08T01:15:32.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.046+0000 7f02979ff700 1 -- 172.21.15.119:0/2332543255 wait complete. 2023-10-08T01:15:32.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.047+0000 7f02979ff700 1 Processor -- start 2023-10-08T01:15:32.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.047+0000 7f02979ff700 1 -- start start 2023-10-08T01:15:32.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.048+0000 7f02979ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0290068390 0x7f029016da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.048+0000 7f02979ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0290156620 0x7f0290154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.048+0000 7f02979ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02901569f0 0x7f0290148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.048+0000 7f02979ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900b02c0 con 0x7f02901569f0 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.048+0000 7f02979ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02900b04d0 con 0x7f0290068390 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.048+0000 7f02979ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900b07b0 con 0x7f0290156620 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02901569f0 0x7f0290148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02901569f0 0x7f0290148930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42080/0 (socket says 172.21.15.119:42080) 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 -- 172.21.15.119:0/1681622721 learned_addr learned my addr 172.21.15.119:0/1681622721 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f029579b700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0290068390 0x7f029016da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0294f9a700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0290156620 0x7f0290154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 -- 172.21.15.119:0/1681622721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0290156620 msgr2=0x7f0290154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0290156620 0x7f0290154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 -- 172.21.15.119:0/1681622721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0290068390 msgr2=0x7f029016da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0290068390 0x7f029016da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 -- 172.21.15.119:0/1681622721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280009420 con 0x7f02901569f0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f0295f9c700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02901569f0 0x7f0290148930 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f028800c860 tx=0x7f0288002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f02857fa700 1 -- 172.21.15.119:0/1681622721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288002030 con 0x7f02901569f0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f02857fa700 1 -- 172.21.15.119:0/1681622721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880021b0 con 0x7f02901569f0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f02857fa700 1 -- 172.21.15.119:0/1681622721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288012630 con 0x7f02901569f0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.049+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02900b0a90 con 0x7f02901569f0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.050+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0290148e70 con 0x7f02901569f0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.051+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02900718b0 con 0x7f02901569f0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.052+0000 7f02857fa700 1 -- 172.21.15.119:0/1681622721 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0288012a20 con 0x7f02901569f0 2023-10-08T01:15:32.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.052+0000 7f02857fa700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0274066f30 0x7f02740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.052+0000 7f02857fa700 1 -- 172.21.15.119:0/1681622721 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f02880845e0 con 0x7f02901569f0 2023-10-08T01:15:32.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.053+0000 7f029579b700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0274066f30 0x7f02740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.053+0000 7f029579b700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0274066f30 0x7f02740693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f028c001080 tx=0x7f028c001930 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:32.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.056+0000 7f02857fa700 1 -- 172.21.15.119:0/1681622721 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f02880516b0 con 0x7f02901569f0 2023-10-08T01:15:32.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.321+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f02900c7bc0 con 0x7f02901569f0 2023-10-08T01:15:32.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.325+0000 7f02857fa700 1 -- 172.21.15.119:0/1681622721 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0288054cd0 con 0x7f02901569f0 2023-10-08T01:15:32.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0274066f30 msgr2=0x7f02740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0274066f30 0x7f02740693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f028c001080 tx=0x7f028c001930 comp rx=0 tx=0).stop 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02901569f0 msgr2=0x7f0290148930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02901569f0 0x7f0290148930 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f028800c860 tx=0x7f0288002700 comp rx=0 tx=0).stop 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 shutdown_connections 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0274066f30 0x7f02740693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02901569f0 0x7f0290148930 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0290068390 0x7f029016da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 --2- 172.21.15.119:0/1681622721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0290156620 0x7f0290154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 >> 172.21.15.119:0/1681622721 conn(0x7f02900cc540 msgr2=0x7f02900ad5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:32.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 shutdown_connections 2023-10-08T01:15:32.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.327+0000 7f02979ff700 1 -- 172.21.15.119:0/1681622721 wait complete. 2023-10-08T01:15:32.330 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T01:15:32.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T01:15:32.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.525+0000 7fbc523ff700 1 Processor -- start 2023-10-08T01:15:32.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.525+0000 7fbc523ff700 1 -- start start 2023-10-08T01:15:32.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.526+0000 7fbc523ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc4c0cbf30 0x7fbc4c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.526+0000 7fbc523ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc4c0ccf90 0x7fbc4c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.526+0000 7fbc523ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c15a270 0x7fbc4c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.526+0000 7fbc523ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c156bf0 con 0x7fbc4c0ccf90 2023-10-08T01:15:32.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.526+0000 7fbc523ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4c0ae950 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.526+0000 7fbc523ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c0cc300 con 0x7fbc4c15a270 2023-10-08T01:15:32.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc4c0ccf90 0x7fbc4c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc4c0ccf90 0x7fbc4c0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42082/0 (socket says 172.21.15.119:42082) 2023-10-08T01:15:32.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 -- 172.21.15.119:0/1198753028 learned_addr learned my addr 172.21.15.119:0/1198753028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:32.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4bfff700 1 --2- 172.21.15.119:0/1198753028 >> v2:172.21.15.184:3300/0 conn(0x7fbc4c0cbf30 0x7fbc4c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc5099c700 1 --2- 172.21.15.119:0/1198753028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c15a270 0x7fbc4c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 -- 172.21.15.119:0/1198753028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c15a270 msgr2=0x7fbc4c0b1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:32.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 --2- 172.21.15.119:0/1198753028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c15a270 0x7fbc4c0b1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 -- 172.21.15.119:0/1198753028 >> v2:172.21.15.184:3300/0 conn(0x7fbc4c0cbf30 msgr2=0x7fbc4c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:32.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 --2- 172.21.15.119:0/1198753028 >> v2:172.21.15.184:3300/0 conn(0x7fbc4c0cbf30 0x7fbc4c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 -- 172.21.15.119:0/1198753028 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4c0ad370 con 0x7fbc4c0ccf90 2023-10-08T01:15:32.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.527+0000 7fbc4b7fe700 1 --2- 172.21.15.119:0/1198753028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0ccf90 0x7fbc4c0b0d10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c063d40 tx=0x7fbc40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9c71e3df8146ea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:32.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc4affd700 1 -- 172.21.15.119:0/1198753028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000f6b0 con 0x7fbc4c0ccf90 2023-10-08T01:15:32.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc4affd700 1 -- 172.21.15.119:0/1198753028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4000f810 con 0x7fbc4c0ccf90 2023-10-08T01:15:32.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc4affd700 1 -- 172.21.15.119:0/1198753028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000fbb0 con 0x7fbc4c0ccf90 2023-10-08T01:15:32.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 -- 172.21.15.119:0/1198753028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0ccf90 msgr2=0x7fbc4c0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 --2- 172.21.15.119:0/1198753028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0ccf90 0x7fbc4c0b0d10 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c063d40 tx=0x7fbc40009250 comp rx=0 tx=0).stop 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 -- 172.21.15.119:0/1198753028 shutdown_connections 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 --2- 172.21.15.119:0/1198753028 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c15a270 0x7fbc4c0b1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 --2- 172.21.15.119:0/1198753028 >> v2:172.21.15.184:3300/0 conn(0x7fbc4c0cbf30 0x7fbc4c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 --2- 172.21.15.119:0/1198753028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0ccf90 0x7fbc4c0b0d10 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 -- 172.21.15.119:0/1198753028 >> 172.21.15.119:0/1198753028 conn(0x7fbc4c06a320 msgr2=0x7fbc4c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 -- 172.21.15.119:0/1198753028 shutdown_connections 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.528+0000 7fbc523ff700 1 -- 172.21.15.119:0/1198753028 wait complete. 2023-10-08T01:15:32.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.530+0000 7fbc523ff700 1 Processor -- start 2023-10-08T01:15:32.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.530+0000 7fbc523ff700 1 -- start start 2023-10-08T01:15:32.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc523ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0cbf30 0x7fbc4c0688e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc523ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c0ccf90 0x7fbc4c1688c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc523ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc4c15a270 0x7fbc4c0a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc523ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c159ba0 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc523ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c168e00 con 0x7fbc4c15a270 2023-10-08T01:15:32.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc523ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4c165af0 con 0x7fbc4c0ccf90 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0cbf30 0x7fbc4c0688e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0cbf30 0x7fbc4c0688e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42088/0 (socket says 172.21.15.119:42088) 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4bfff700 1 -- 172.21.15.119:0/3479206668 learned_addr learned my addr 172.21.15.119:0/3479206668 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4b7fe700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c0ccf90 0x7fbc4c1688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc5099c700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc4c15a270 0x7fbc4c0a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4bfff700 1 -- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c0ccf90 msgr2=0x7fbc4c1688c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4bfff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c0ccf90 0x7fbc4c1688c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4bfff700 1 -- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc4c15a270 msgr2=0x7fbc4c0a7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4bfff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc4c15a270 0x7fbc4c0a7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc4bfff700 1 -- 172.21.15.119:0/3479206668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc400094f0 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.531+0000 7fbc5099c700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc4c15a270 0x7fbc4c0a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:32.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.532+0000 7fbc4bfff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0cbf30 0x7fbc4c0688e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00e930 tx=0x7fbc3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:32.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.532+0000 7fbc33fff700 1 -- 172.21.15.119:0/3479206668 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c013680 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.532+0000 7fbc33fff700 1 -- 172.21.15.119:0/3479206668 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3c0137e0 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.532+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4c165db0 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.532+0000 7fbc33fff700 1 -- 172.21.15.119:0/3479206668 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3c013b80 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.532+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4c166370 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.533+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4c04c3c0 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.534+0000 7fbc33fff700 1 -- 172.21.15.119:0/3479206668 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc3c00c460 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.535+0000 7fbc33fff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc38066f30 0x7fbc380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.535+0000 7fbc33fff700 1 -- 172.21.15.119:0/3479206668 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fbc3c084840 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.536+0000 7fbc4b7fe700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc38066f30 0x7fbc380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:32.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.536+0000 7fbc4b7fe700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc38066f30 0x7fbc380693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c063d40 tx=0x7fbc40019000 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:32.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.539+0000 7fbc33fff700 1 -- 172.21.15.119:0/3479206668 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc3c051910 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.770+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc4c06f8a0 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.771+0000 7fbc33fff700 1 -- 172.21.15.119:0/3479206668 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fbc3c054f30 con 0x7fbc4c0cbf30 2023-10-08T01:15:32.772 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:32.772 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7549,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/26538394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":26538394},{"type":"v1","addr":"172.21.15.184:6834","nonce":26538394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3494509212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3494509212},{"type":"v1","addr":"172.21.15.119:6834","nonce":3494509212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2792091862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2792091862},{"type":"v1","addr":"172.21.15.119:6835","nonce":2792091862}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:15:21.218141+0000","modified":"2023-10-08T01:15:25.231364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1485819493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1485819493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1485819493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T01:15:32.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc38066f30 msgr2=0x7fbc380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc38066f30 0x7fbc380693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c063d40 tx=0x7fbc40019000 comp rx=0 tx=0).stop 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0cbf30 msgr2=0x7fbc4c0688e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0cbf30 0x7fbc4c0688e0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbc3c00e930 tx=0x7fbc3c0091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 shutdown_connections 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbc38066f30 0x7fbc380693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc4c15a270 0x7fbc4c0a7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc4c0cbf30 0x7fbc4c0688e0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 --2- 172.21.15.119:0/3479206668 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc4c0ccf90 0x7fbc4c1688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 >> 172.21.15.119:0/3479206668 conn(0x7fbc4c06a320 msgr2=0x7fbc4c0acbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:32.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 shutdown_connections 2023-10-08T01:15:32.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:32.773+0000 7fbc523ff700 1 -- 172.21.15.119:0/3479206668 wait complete. 2023-10-08T01:15:32.776 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 63 2023-10-08T01:15:33.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T01:15:21.218141+0000', 'modified': '2023-10-08T01:15:25.231364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8178}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8178': {'gid': 8178, 'name': 'b', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6835/1485819493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6833', 'nonce': 1485819493}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 1485819493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T01:15:33.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T01:15:33.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-10-08T01:15:33.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.376+0000 7fb829fff700 1 Processor -- start 2023-10-08T01:15:33.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.376+0000 7fb829fff700 1 -- start start 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb829fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb8240cbf30 0x7fb8240ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb829fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8240b1420 0x7fb8240c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb829fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8240cc300 con 0x7fb8240ccf90 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb829fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb824156bf0 con 0x7fb8240b1420 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb829fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8240ae950 con 0x7fb8240cbf30 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb823fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8240b1420 0x7fb8240c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb823fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8240b1420 0x7fb8240c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42102/0 (socket says 172.21.15.119:42102) 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb823fff700 1 -- 172.21.15.119:0/2621000904 learned_addr learned my addr 172.21.15.119:0/2621000904 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:33.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2621000904 >> v2:172.21.15.184:3300/0 conn(0x7fb8240cbf30 0x7fb8240ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb823fff700 1 -- 172.21.15.119:0/2621000904 >> v2:172.21.15.184:3300/0 conn(0x7fb8240cbf30 msgr2=0x7fb8240ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb823fff700 1 --2- 172.21.15.119:0/2621000904 >> v2:172.21.15.184:3300/0 conn(0x7fb8240cbf30 0x7fb8240ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb823fff700 1 -- 172.21.15.119:0/2621000904 >> v1:172.21.15.184:6789/0 conn(0x7fb8240ccf90 legacy=0x7fb8240b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.378+0000 7fb823fff700 1 -- 172.21.15.119:0/2621000904 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8240ad370 con 0x7fb8240b1420 2023-10-08T01:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.379+0000 7fb823fff700 1 --2- 172.21.15.119:0/2621000904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240b1420 0x7fb8240c9740 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb824063d40 tx=0x7fb81c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5b10514badd164 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.379+0000 7fb8227fc700 1 -- 172.21.15.119:0/2621000904 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c00e070 con 0x7fb8240b1420 2023-10-08T01:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.379+0000 7fb8227fc700 1 -- 172.21.15.119:0/2621000904 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c00ba00 con 0x7fb8240b1420 2023-10-08T01:15:33.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.379+0000 7fb8227fc700 1 -- 172.21.15.119:0/2621000904 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c0092e0 con 0x7fb8240b1420 2023-10-08T01:15:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.379+0000 7fb829fff700 1 -- 172.21.15.119:0/2621000904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240b1420 msgr2=0x7fb8240c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.379+0000 7fb829fff700 1 --2- 172.21.15.119:0/2621000904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240b1420 0x7fb8240c9740 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb824063d40 tx=0x7fb81c009840 comp rx=0 tx=0).stop 2023-10-08T01:15:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.380+0000 7fb829fff700 1 -- 172.21.15.119:0/2621000904 shutdown_connections 2023-10-08T01:15:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.380+0000 7fb829fff700 1 --2- 172.21.15.119:0/2621000904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240b1420 0x7fb8240c9740 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.380+0000 7fb829fff700 1 --2- 172.21.15.119:0/2621000904 >> v2:172.21.15.184:3300/0 conn(0x7fb8240cbf30 0x7fb8240ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.380+0000 7fb829fff700 1 -- 172.21.15.119:0/2621000904 >> 172.21.15.119:0/2621000904 conn(0x7fb82406a320 msgr2=0x7fb8240cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.380+0000 7fb829fff700 1 -- 172.21.15.119:0/2621000904 shutdown_connections 2023-10-08T01:15:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.380+0000 7fb829fff700 1 -- 172.21.15.119:0/2621000904 wait complete. 2023-10-08T01:15:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.381+0000 7fb829fff700 1 Processor -- start 2023-10-08T01:15:33.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.381+0000 7fb829fff700 1 -- start start 2023-10-08T01:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb829fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8240b1420 0x7fb8240acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb829fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240cbf30 0x7fb824158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb829fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8240ccf90 0x7fb8241593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb829fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82416a500 con 0x7fb8240cbf30 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb829fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8241698f0 con 0x7fb8240ccf90 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb829fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb824169bd0 con 0x7fb8240b1420 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb822ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240cbf30 0x7fb824158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb822ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240cbf30 0x7fb824158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42114/0 (socket says 172.21.15.119:42114) 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb822ffd700 1 -- 172.21.15.119:0/2590538571 learned_addr learned my addr 172.21.15.119:0/2590538571 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8240b1420 0x7fb8240acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.382+0000 7fb823fff700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8240ccf90 0x7fb8241593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb822ffd700 1 -- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8240b1420 msgr2=0x7fb8240acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb822ffd700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8240b1420 0x7fb8240acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb822ffd700 1 -- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8240ccf90 msgr2=0x7fb8241593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb822ffd700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8240ccf90 0x7fb8241593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb822ffd700 1 -- 172.21.15.119:0/2590538571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c009ae0 con 0x7fb8240cbf30 2023-10-08T01:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8240b1420 0x7fb8240acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:33.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb822ffd700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240cbf30 0x7fb824158e80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b5e0 tx=0x7fb818009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb80b7fe700 1 -- 172.21.15.119:0/2590538571 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81800f510 con 0x7fb8240cbf30 2023-10-08T01:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb80b7fe700 1 -- 172.21.15.119:0/2590538571 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81800f670 con 0x7fb8240cbf30 2023-10-08T01:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb829fff700 1 -- 172.21.15.119:0/2590538571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb824169eb0 con 0x7fb8240cbf30 2023-10-08T01:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb80b7fe700 1 -- 172.21.15.119:0/2590538571 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81800fa30 con 0x7fb8240cbf30 2023-10-08T01:15:33.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.383+0000 7fb829fff700 1 -- 172.21.15.119:0/2590538571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8240c66e0 con 0x7fb8240cbf30 2023-10-08T01:15:33.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.385+0000 7fb80b7fe700 1 -- 172.21.15.119:0/2590538571 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb81801a400 con 0x7fb8240cbf30 2023-10-08T01:15:33.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.386+0000 7fb80b7fe700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb810066f30 0x7fb8100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.386+0000 7fb80b7fe700 1 -- 172.21.15.119:0/2590538571 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fb818084210 con 0x7fb8240cbf30 2023-10-08T01:15:33.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.387+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb810066f30 0x7fb8100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.387+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb810066f30 0x7fb8100693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb814006fd0 tx=0x7fb814008040 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.388+0000 7fb8386f1b80 1 Processor -- start 2023-10-08T01:15:33.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.388+0000 7fb8386f1b80 1 -- start start 2023-10-08T01:15:33.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 0x5593fc3cfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc3f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fc326a70 con 0x5593fc419d00 2023-10-08T01:15:33.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fc324270 con 0x5593fc3f8f60 2023-10-08T01:15:33.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593fc320060 con 0x5593fc3cfa50 2023-10-08T01:15:33.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb822ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc3f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb822ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc3f8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42120/0 (socket says 172.21.15.119:42120) 2023-10-08T01:15:33.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb822ffd700 1 -- 172.21.15.119:0/1693239224 learned_addr learned my addr 172.21.15.119:0/1693239224 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:33.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb8237fe700 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 0x5593fc3cfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb823fff700 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb822ffd700 1 -- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 msgr2=0x5593fc3cfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb822ffd700 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 0x5593fc3cfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb822ffd700 1 -- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 msgr2=0x5593fc3cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb822ffd700 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb822ffd700 1 -- 172.21.15.119:0/1693239224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fc1ba270 con 0x5593fc419d00 2023-10-08T01:15:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.389+0000 7fb823fff700 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb822ffd700 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc3f8a20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb8180824c0 tx=0x7fb818082450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1f7d7dea55931f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb808ff9700 1 -- 172.21.15.119:0/1693239224 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81805bf20 con 0x5593fc419d00 2023-10-08T01:15:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb808ff9700 1 -- 172.21.15.119:0/1693239224 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8180530c0 con 0x5593fc419d00 2023-10-08T01:15:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb808ff9700 1 -- 172.21.15.119:0/1693239224 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818053320 con 0x5593fc419d00 2023-10-08T01:15:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1693239224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 msgr2=0x5593fc3f8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc3f8a20 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb8180824c0 tx=0x7fb818082450 comp rx=0 tx=0).stop 2023-10-08T01:15:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1693239224 shutdown_connections 2023-10-08T01:15:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 0x5593fc3cfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1693239224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc3f8a20 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.390+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1693239224 >> 172.21.15.119:0/1693239224 conn(0x5593fc3d1370 msgr2=0x5593fc3d1740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.391+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1693239224 shutdown_connections 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.391+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1693239224 wait complete. 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.391+0000 7fb8386f1b80 1 -- start start 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.391+0000 7fb8386f1b80 1 Processor -- start 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.392+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc3f5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.392+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3f8f60 0x5593fc4188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.392+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc419d00 0x5593fc416a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.392+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fc2cb890 con 0x5593fc3cfa50 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.392+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fc2e6ea0 con 0x5593fc419d00 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.392+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593fc1a2e30 con 0x5593fc3f8f60 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.392+0000 7fb8386f1b80 10 client.0 _ll_register_callbacks cb 0x5593fc393110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc3f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc3f5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42122/0 (socket says 172.21.15.119:42122) 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 -- 172.21.15.119:0/1410674760 learned_addr learned my addr 172.21.15.119:0/1410674760 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb823fff700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc419d00 0x5593fc416a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb822ffd700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3f8f60 0x5593fc4188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 -- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3f8f60 msgr2=0x5593fc4188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3f8f60 0x5593fc4188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 -- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc419d00 msgr2=0x5593fc416a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc419d00 0x5593fc416a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 -- 172.21.15.119:0/1410674760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8240c66e0 con 0x5593fc3cfa50 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb823fff700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc419d00 0x5593fc416a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb822ffd700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3f8f60 0x5593fc4188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb8237fe700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc3f5060 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb814014810 tx=0x7fb814018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa4da5f4168cfe0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb801ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.393+0000 7fb801ffb700 1 -- 172.21.15.119:0/1410674760 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8140157a0 con 0x5593fc3cfa50 2023-10-08T01:15:33.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1410674760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593fc3c4b70 con 0x5593fc3cfa50 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb801ffb700 1 -- 172.21.15.119:0/1410674760 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fb814015900 con 0x5593fc3cfa50 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb801ffb700 1 -- 172.21.15.119:0/1410674760 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb814015cd0 con 0x5593fc3cfa50 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 10 client.8340 fetch_fsmap learned FSMap version 63 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1410674760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5593fc15cfe0 con 0x5593fc3cfa50 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb801ffb700 1 -- 172.21.15.119:0/1410674760 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5593fc3c4b70 con 0x5593fc3cfa50 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb801ffb700 1 -- 172.21.15.119:0/1410674760 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 63) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb814021420 con 0x5593fc3cfa50 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 10 client.8340 fetch_fsmap finished waiting for FSMap version 63 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 10 client.8340 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 20 client.8340 populate_metadata read hostname 'smithi119' 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] conn(0x5593fc3caca0 0x5593fc3cb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 4 client.8340 mds_command: new command op to 8178 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.394+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1410674760 --> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593fc3a1aa0 con 0x5593fc3caca0 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.395+0000 7fb822ffd700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] conn(0x5593fc3caca0 0x5593fc3cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.396+0000 7fb801ffb700 10 client.8340 ms_handle_connect on v2:172.21.15.184:6833/1485819493 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.396+0000 7fb822ffd700 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] conn(0x5593fc3caca0 0x5593fc3cb070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b304c75e31697ffd server_cookie=5586cbfcb35acc76 in_seq=0 out_seq=0 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb801ffb700 1 -- 172.21.15.119:0/1410674760 <== mds.0 v2:172.21.15.184:6833/1485819493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x5593fc3a1aa0 con 0x5593fc3caca0 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb801ffb700 10 client.8340 handle_command_reply: tid=0 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 client.8340 shutdown 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1410674760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 msgr2=0x5593fc3f5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc3f5060 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb814014810 tx=0x7fb814018040 comp rx=0 tx=0).stop 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1410674760 shutdown_connections 2023-10-08T01:15:33.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] conn(0x5593fc3caca0 0x5593fc3cb070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc419d00 0x5593fc416a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc3f5060 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/1410674760 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3f8f60 0x5593fc4188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1410674760 >> 172.21.15.119:0/1410674760 conn(0x5593fc3d1370 msgr2=0x5593fc3d2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb801ffb700 0 client.8340 ms_handle_reset on v2:172.21.15.184:6833/1485819493 2023-10-08T01:15:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1410674760 shutdown_connections 2023-10-08T01:15:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.397+0000 7fb8386f1b80 1 -- 172.21.15.119:0/1410674760 wait complete. 2023-10-08T01:15:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.398+0000 7fb8386f1b80 20 client.8340 trim_cache size 0 max 16384 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.415+0000 7fb8386f1b80 1 Processor -- start 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.415+0000 7fb8386f1b80 1 -- start start 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.415+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 0x5593fc416a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.415+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3f7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.415+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc417660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.415+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fc415130 con 0x5593fc419d00 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.415+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fc3cc060 con 0x5593fc3f8f60 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.415+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593fc3a8410 con 0x5593fc3cfa50 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc417660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc417660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42136/0 (socket says 172.21.15.119:42136) 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 -- 172.21.15.119:0/2594420831 learned_addr learned my addr 172.21.15.119:0/2594420831 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb822ffd700 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 0x5593fc416a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 -- 172.21.15.119:0/2594420831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 msgr2=0x5593fc416a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 0x5593fc416a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 -- 172.21.15.119:0/2594420831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 msgr2=0x5593fc3f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 -- 172.21.15.119:0/2594420831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593fc2ee780 con 0x5593fc419d00 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb823fff700 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc417660 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009ea0 tx=0x7fb81c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b706ddb5eb73018a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.416+0000 7fb8097fa700 1 -- 172.21.15.119:0/2594420831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c024070 con 0x5593fc419d00 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8097fa700 1 -- 172.21.15.119:0/2594420831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb81c00bd20 con 0x5593fc419d00 2023-10-08T01:15:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8097fa700 1 -- 172.21.15.119:0/2594420831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81c012890 con 0x5593fc419d00 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2594420831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 msgr2=0x5593fc417660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc417660 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009ea0 tx=0x7fb81c017000 comp rx=0 tx=0).stop 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2594420831 shutdown_connections 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc3cfa50 0x5593fc416a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc419d00 0x5593fc417660 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2594420831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc3f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2594420831 >> 172.21.15.119:0/2594420831 conn(0x5593fc3d1370 msgr2=0x5593fc415550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2594420831 shutdown_connections 2023-10-08T01:15:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.417+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2594420831 wait complete. 2023-10-08T01:15:33.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.418+0000 7fb8386f1b80 1 -- start start 2023-10-08T01:15:33.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.418+0000 7fb8386f1b80 1 Processor -- start 2023-10-08T01:15:33.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc479d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc40cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8386f1b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc419d00 0x5593fc42f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fc47a590 con 0x5593fc3cfa50 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593fc479690 con 0x5593fc3f8f60 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8386f1b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x5593fc40d4b0 con 0x5593fc419d00 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8386f1b80 10 client.0 _ll_register_callbacks cb 0x5593fc393110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8237fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc479d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8237fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc479d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42152/0 (socket says 172.21.15.119:42152) 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb8237fe700 1 -- 172.21.15.119:0/2875021783 learned_addr learned my addr 172.21.15.119:0/2875021783 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb823fff700 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc419d00 0x5593fc42f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.419+0000 7fb822ffd700 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc40cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8237fe700 1 -- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc419d00 msgr2=0x5593fc42f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc419d00 0x5593fc42f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8237fe700 1 -- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 msgr2=0x5593fc40cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc40cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8237fe700 1 -- 172.21.15.119:0/2875021783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8240ad370 con 0x5593fc3cfa50 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8237fe700 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc479d00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb81400f6d0 tx=0x7fb81400ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50eabcbf82753cf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb801ffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb801ffb700 1 -- 172.21.15.119:0/2875021783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb814007800 con 0x5593fc3cfa50 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb801ffb700 1 -- 172.21.15.119:0/2875021783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fb8140157a0 con 0x5593fc3cfa50 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2875021783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb824156d30 con 0x5593fc3cfa50 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb801ffb700 1 -- 172.21.15.119:0/2875021783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb814015960 con 0x5593fc3cfa50 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8386f1b80 10 client.8346 fetch_fsmap learned FSMap version 63 2023-10-08T01:15:33.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2875021783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) v3 -- 0x5593fc2d5470 con 0x5593fc3cfa50 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.420+0000 7fb801ffb700 1 -- 172.21.15.119:0/2875021783 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb824156d30 con 0x5593fc3cfa50 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb801ffb700 1 -- 172.21.15.119:0/2875021783 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 63) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb814021bc0 con 0x5593fc3cfa50 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb8386f1b80 10 client.8346 fetch_fsmap finished waiting for FSMap version 63 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb8386f1b80 10 client.8346 resolve_mds: resolved name 'b' to daemon mds.b 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb8386f1b80 20 client.8346 populate_metadata read hostname 'smithi119' 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] conn(0x5593fc477be0 0x5593fc477fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb8386f1b80 4 client.8346 mds_command: new command op to 8178 tid=0[{"prefix": "status"}] 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2875021783 --> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fb8180530c0 con 0x5593fc477be0 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb822ffd700 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] conn(0x5593fc477be0 0x5593fc477fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb801ffb700 10 client.8346 ms_handle_connect on v2:172.21.15.184:6833/1485819493 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.421+0000 7fb822ffd700 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] conn(0x5593fc477be0 0x5593fc477fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b903da5d73ddc27f server_cookie=443b1259f378a89a in_seq=0 out_seq=0 2023-10-08T01:15:33.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb801ffb700 1 -- 172.21.15.119:0/2875021783 <== mds.0 v2:172.21.15.184:6833/1485819493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fb8180530c0 con 0x5593fc477be0 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb801ffb700 10 client.8346 handle_command_reply: tid=0 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 client.8346 shutdown 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2875021783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 msgr2=0x5593fc479d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc479d00 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fb81400f6d0 tx=0x7fb81400ffd0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2875021783 shutdown_connections 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:6833/1485819493,v1:172.21.15.184:6835/1485819493] conn(0x5593fc477be0 0x5593fc477fb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x5593fc419d00 0x5593fc42f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5593fc3cfa50 0x5593fc479d00 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb801ffb700 0 client.8346 ms_handle_reset on v2:172.21.15.184:6833/1485819493 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 --2- 172.21.15.119:0/2875021783 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x5593fc3f8f60 0x5593fc40cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2875021783 >> 172.21.15.119:0/2875021783 conn(0x5593fc3d1370 msgr2=0x5593fc3f7420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2875021783 shutdown_connections 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 1 -- 172.21.15.119:0/2875021783 wait complete. 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.422+0000 7fb8386f1b80 20 client.8346 trim_cache size 0 max 16384 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 8178, 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T01:15:33.430 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T01:15:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T01:15:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 11.650707869, 2023-10-08T01:15:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 63, 2023-10-08T01:15:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 135, 2023-10-08T01:15:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 135, 2023-10-08T01:15:33.431 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 17.429845142000001 2023-10-08T01:15:33.431 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T01:15:33.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 -- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb810066f30 msgr2=0x7fb8100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb810066f30 0x7fb8100693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb814006fd0 tx=0x7fb814008040 comp rx=0 tx=0).stop 2023-10-08T01:15:33.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 -- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240cbf30 msgr2=0x7fb824158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240cbf30 0x7fb824158e80 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb81800b5e0 tx=0x7fb818009120 comp rx=0 tx=0).stop 2023-10-08T01:15:33.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 -- 172.21.15.119:0/2590538571 shutdown_connections 2023-10-08T01:15:33.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb810066f30 0x7fb8100693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb8240b1420 0x7fb8240acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb8240ccf90 0x7fb8241593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 --2- 172.21.15.119:0/2590538571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8240cbf30 0x7fb824158e80 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 -- 172.21.15.119:0/2590538571 >> 172.21.15.119:0/2590538571 conn(0x7fb82406a320 msgr2=0x7fb82415a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:33.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 -- 172.21.15.119:0/2590538571 shutdown_connections 2023-10-08T01:15:33.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.423+0000 7fb829fff700 1 -- 172.21.15.119:0/2590538571 wait complete. 2023-10-08T01:15:33.440 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T01:15:33.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.621+0000 7f2c5a7ff700 1 Processor -- start 2023-10-08T01:15:33.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.621+0000 7f2c5a7ff700 1 -- start start 2023-10-08T01:15:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c5a7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2c540cbf30 0x7f2c540ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c5a7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c5a7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c540cc300 con 0x7f2c5415a270 2023-10-08T01:15:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c5a7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c54156bf0 con 0x7f2c540cbf30 2023-10-08T01:15:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c5a7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c540ae950 con 0x7f2c540ccf90 2023-10-08T01:15:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c53fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2c540cbf30 0x7f2c540ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c53fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2c540cbf30 0x7f2c540ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42158/0 (socket says 172.21.15.119:42158) 2023-10-08T01:15:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c53fff700 1 -- 172.21.15.119:0/2929678788 learned_addr learned my addr 172.21.15.119:0/2929678788 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.623+0000 7f2c4bfff700 1 --2- 172.21.15.119:0/2929678788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c53fff700 1 -- 172.21.15.119:0/2929678788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 msgr2=0x7f2c540b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c53fff700 1 --2- 172.21.15.119:0/2929678788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c53fff700 1 -- 172.21.15.119:0/2929678788 >> v1:172.21.15.184:6789/0 conn(0x7f2c5415a270 legacy=0x7f2c540c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c53fff700 1 -- 172.21.15.119:0/2929678788 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c540ad370 con 0x7f2c540cbf30 2023-10-08T01:15:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c4bfff700 1 --2- 172.21.15.119:0/2929678788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c53fff700 1 --2- 172.21.15.119:0/2929678788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c540cbf30 0x7f2c540ae410 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400bc80 tx=0x7f2c4400b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab2e60f23728726 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c537fe700 1 -- 172.21.15.119:0/2929678788 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400be90 con 0x7f2c540cbf30 2023-10-08T01:15:33.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c537fe700 1 -- 172.21.15.119:0/2929678788 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4400ddc0 con 0x7f2c540cbf30 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.624+0000 7f2c537fe700 1 -- 172.21.15.119:0/2929678788 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c440196b0 con 0x7f2c540cbf30 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.625+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/2929678788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c540cbf30 msgr2=0x7f2c540ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.625+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/2929678788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c540cbf30 0x7f2c540ae410 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400bc80 tx=0x7f2c4400b4f0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.625+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/2929678788 shutdown_connections 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.625+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/2929678788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c540cbf30 0x7f2c540ae410 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.625+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/2929678788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.625+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/2929678788 >> 172.21.15.119:0/2929678788 conn(0x7f2c5406a320 msgr2=0x7f2c540cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.625+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/2929678788 shutdown_connections 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.625+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/2929678788 wait complete. 2023-10-08T01:15:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.626+0000 7f2c5a7ff700 1 Processor -- start 2023-10-08T01:15:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.626+0000 7f2c5a7ff700 1 -- start start 2023-10-08T01:15:33.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.627+0000 7f2c5a7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c540cbf30 0x7f2c5416ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.627+0000 7f2c5a7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.627+0000 7f2c5a7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5415a270 0x7f2c540c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.627+0000 7f2c5a7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c540c5330 con 0x7f2c5415a270 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.627+0000 7f2c5a7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c540c67f0 con 0x7f2c540cbf30 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.627+0000 7f2c5a7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c540c5050 con 0x7f2c540ccf90 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5415a270 0x7f2c540c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5415a270 0x7f2c540c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42166/0 (socket says 172.21.15.119:42166) 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 -- 172.21.15.119:0/4200950333 learned_addr learned my addr 172.21.15.119:0/4200950333 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c53fff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c540cbf30 0x7f2c5416ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c4bfff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 -- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 msgr2=0x7f2c540abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 -- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c540cbf30 msgr2=0x7f2c5416ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c540cbf30 0x7f2c5416ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 -- 172.21.15.119:0/4200950333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4400b790 con 0x7f2c5415a270 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c4bfff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c58d9c700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5415a270 0x7f2c540c45d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00c840 tx=0x7f2c4c001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c50ff9700 1 -- 172.21.15.119:0/4200950333 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c012630 con 0x7f2c5415a270 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c50ff9700 1 -- 172.21.15.119:0/4200950333 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0127b0 con 0x7f2c5415a270 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c540c6a70 con 0x7f2c5415a270 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.628+0000 7f2c50ff9700 1 -- 172.21.15.119:0/4200950333 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c012b70 con 0x7f2c5415a270 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.629+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c540c6e00 con 0x7f2c5415a270 2023-10-08T01:15:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.630+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5404c3c0 con 0x7f2c5415a270 2023-10-08T01:15:33.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.630+0000 7f2c50ff9700 1 -- 172.21.15.119:0/4200950333 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c4c003030 con 0x7f2c5415a270 2023-10-08T01:15:33.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.631+0000 7f2c50ff9700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2c34066f60 0x7f2c34069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:33.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.632+0000 7f2c50ff9700 1 -- 172.21.15.119:0/4200950333 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f2c4c0845d0 con 0x7f2c5415a270 2023-10-08T01:15:33.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.632+0000 7f2c53fff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2c34066f60 0x7f2c34069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:33.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.632+0000 7f2c53fff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2c34066f60 0x7f2c34069410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400bc80 tx=0x7f2c4401b000 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:33.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.636+0000 7f2c50ff9700 1 -- 172.21.15.119:0/4200950333 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c4c055130 con 0x7f2c5415a270 2023-10-08T01:15:33.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.870+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2c5406f8a0 con 0x7f2c5415a270 2023-10-08T01:15:33.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.871+0000 7f2c50ff9700 1 -- 172.21.15.119:0/4200950333 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2c4c054cc0 con 0x7f2c5415a270 2023-10-08T01:15:33.872 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T01:15:33.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2c34066f60 msgr2=0x7f2c34069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2c34066f60 0x7f2c34069410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2c4400bc80 tx=0x7f2c4401b000 comp rx=0 tx=0).stop 2023-10-08T01:15:33.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5415a270 msgr2=0x7f2c540c45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5415a270 0x7f2c540c45d0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00c840 tx=0x7f2c4c001670 comp rx=0 tx=0).stop 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 shutdown_connections 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2c34066f60 0x7f2c34069410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c5415a270 0x7f2c540c45d0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2c540cbf30 0x7f2c5416ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 --2- 172.21.15.119:0/4200950333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2c540ccf90 0x7f2c540abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 >> 172.21.15.119:0/4200950333 conn(0x7f2c5406a320 msgr2=0x7f2c54159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 shutdown_connections 2023-10-08T01:15:33.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:33.873+0000 7f2c5a7ff700 1 -- 172.21.15.119:0/4200950333 wait complete. 2023-10-08T01:15:33.891 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T01:15:33.891 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T01:15:33.891 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T01:15:33.891 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T01:15:33.892 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T01:15:33.892 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T01:15:33.892 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T01:15:33.892 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T01:15:33.892 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:15:33.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:15:33.892 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T01:15:33.929 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T01:15:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T01:15:33.929 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T01:15:33.930 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 41118sec preferred_lft 41118sec 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T01:15:33.931 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T01:15:33.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:15:33.933 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T01:15:34.029 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T01:15:34.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:15:34.032 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T01:15:34.070 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T01:15:34.070 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T01:15:34.070 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T01:15:34.070 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T01:15:34.070 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T01:15:34.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:15:34.071 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T01:15:34.071 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T01:15:34.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T01:15:34.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T01:15:34.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T01:15:34.071 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T01:15:34.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:15:34.267 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T01:15:34.304 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T01:15:34.304 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T01:15:34.304 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:15:34.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:15:34.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T01:15:34.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:15:34.407 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T01:15:34.447 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T01:15:34.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T01:15:34.448 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:15:34.524 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T01:15:34.524 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T01:15:34.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:15:34.654 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:15:34.654 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:15:34.683 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T01:15:34.683 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T01:15:34.683 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T01:15:34.683 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T01:15:34.683 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T01:15:34.687 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T01:15:34.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T01:15:34.687 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T01:15:34.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T01:15:34.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.929+0000 7f40c95ff700 1 Processor -- start 2023-10-08T01:15:34.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.929+0000 7f40c95ff700 1 -- start start 2023-10-08T01:15:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.931+0000 7f40c95ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c4155c10 0x7f40c4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.931+0000 7f40c95ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f40c40ac6a0 0x7f40c40b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.931+0000 7f40c95ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40c40acac0 con 0x7f40c40ac230 2023-10-08T01:15:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.931+0000 7f40c95ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40c40ad930 con 0x7f40c40ac6a0 2023-10-08T01:15:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.931+0000 7f40c95ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40acd20 con 0x7f40c4155c10 2023-10-08T01:15:34.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.931+0000 7f40c2ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c4155c10 0x7f40c4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.931+0000 7f40c37fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f40c40ac6a0 0x7f40c40b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.931+0000 7f40bbfff700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f40c40ac230 0x7f40c41568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:48256/0 (socket says 172.21.15.119:48256) 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.932+0000 7f40bbfff700 1 -- 172.21.15.119:0/3982217997 learned_addr learned my addr 172.21.15.119:0/3982217997 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.932+0000 7f40c27fc700 1 -- 172.21.15.119:0/3982217997 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743160569 0 0) 0x7f40c40acac0 con 0x7f40c40ac230 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.932+0000 7f40c27fc700 1 -- 172.21.15.119:0/3982217997 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40b0003570 con 0x7f40c40ac230 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.932+0000 7f40c2ffd700 1 -- 172.21.15.119:0/3982217997 >> v2:172.21.15.184:3300/0 conn(0x7f40c40ac6a0 msgr2=0x7f40c40b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.932+0000 7f40c2ffd700 1 --2- 172.21.15.119:0/3982217997 >> v2:172.21.15.184:3300/0 conn(0x7f40c40ac6a0 0x7f40c40b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.932+0000 7f40c2ffd700 1 -- 172.21.15.119:0/3982217997 >> v1:172.21.15.184:6789/0 conn(0x7f40c40ac230 legacy=0x7f40c41568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.932+0000 7f40c2ffd700 1 -- 172.21.15.119:0/3982217997 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c40ad000 con 0x7f40c4155c10 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.932+0000 7f40c2ffd700 1 --2- 172.21.15.119:0/3982217997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c4155c10 0x7f40c4155fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40b400bb90 tx=0x7f40b400b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12fd58dca1b4da5e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c27fc700 1 -- 172.21.15.119:0/3982217997 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b400da70 con 0x7f40c4155c10 2023-10-08T01:15:34.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c27fc700 1 -- 172.21.15.119:0/3982217997 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b400dbd0 con 0x7f40c4155c10 2023-10-08T01:15:34.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c27fc700 1 -- 172.21.15.119:0/3982217997 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b4019480 con 0x7f40c4155c10 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c95ff700 1 -- 172.21.15.119:0/3982217997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c4155c10 msgr2=0x7f40c4155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3982217997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c4155c10 0x7f40c4155fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f40b400bb90 tx=0x7f40b400b430 comp rx=0 tx=0).stop 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c95ff700 1 -- 172.21.15.119:0/3982217997 shutdown_connections 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3982217997 >> v2:172.21.15.184:3300/0 conn(0x7f40c40ac6a0 0x7f40c40b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3982217997 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c4155c10 0x7f40c4155fe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c95ff700 1 -- 172.21.15.119:0/3982217997 >> 172.21.15.119:0/3982217997 conn(0x7f40c406a320 msgr2=0x7f40c40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c95ff700 1 -- 172.21.15.119:0/3982217997 shutdown_connections 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.933+0000 7f40c95ff700 1 -- 172.21.15.119:0/3982217997 wait complete. 2023-10-08T01:15:34.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.935+0000 7f40c95ff700 1 Processor -- start 2023-10-08T01:15:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.935+0000 7f40c95ff700 1 -- start start 2023-10-08T01:15:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40c95ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40ac230 0x7f40c40aaf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40c95ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40c40ac6a0 0x7f40c4168f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40c95ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4155c10 0x7f40c41694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:34.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40c95ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c40c9af0 con 0x7f40c40ac6a0 2023-10-08T01:15:34.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40c95ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c414efd0 con 0x7f40c4155c10 2023-10-08T01:15:34.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40c95ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40c414f2b0 con 0x7f40c40ac230 2023-10-08T01:15:34.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40c40ac6a0 0x7f40c4168f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:34.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40bbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40c40ac6a0 0x7f40c4168f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42528/0 (socket says 172.21.15.119:42528) 2023-10-08T01:15:34.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40bbfff700 1 -- 172.21.15.119:0/3402386562 learned_addr learned my addr 172.21.15.119:0/3402386562 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:34.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40c37fe700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4155c10 0x7f40c41694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:34.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40c2ffd700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40ac230 0x7f40c40aaf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40bbfff700 1 -- 172.21.15.119:0/3402386562 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40ac230 msgr2=0x7f40c40aaf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40bbfff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40ac230 0x7f40c40aaf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40bbfff700 1 -- 172.21.15.119:0/3402386562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4155c10 msgr2=0x7f40c41694c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40bbfff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4155c10 0x7f40c41694c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.936+0000 7f40bbfff700 1 -- 172.21.15.119:0/3402386562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b400b6d0 con 0x7f40c40ac6a0 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.937+0000 7f40bbfff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40c40ac6a0 0x7f40c4168f80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00c7c0 tx=0x7f40ac007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.937+0000 7f40baffd700 1 -- 172.21.15.119:0/3402386562 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac005cd0 con 0x7f40c40ac6a0 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.937+0000 7f40baffd700 1 -- 172.21.15.119:0/3402386562 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac005e30 con 0x7f40c40ac6a0 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.937+0000 7f40baffd700 1 -- 172.21.15.119:0/3402386562 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac007320 con 0x7f40c40ac6a0 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.937+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c414f590 con 0x7f40c40ac6a0 2023-10-08T01:15:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.937+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c414e280 con 0x7f40c40ac6a0 2023-10-08T01:15:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.938+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c40c9ca0 con 0x7f40c40ac6a0 2023-10-08T01:15:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.939+0000 7f40baffd700 1 -- 172.21.15.119:0/3402386562 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40ac01b3b0 con 0x7f40c40ac6a0 2023-10-08T01:15:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.940+0000 7f40baffd700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f40a4066f30 0x7f40a40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.940+0000 7f40baffd700 1 -- 172.21.15.119:0/3402386562 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f40ac083e10 con 0x7f40c40ac6a0 2023-10-08T01:15:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.940+0000 7f40c2ffd700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f40a4066f30 0x7f40a40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.941+0000 7f40c2ffd700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f40a4066f30 0x7f40a40693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40b400b470 tx=0x7f40b401b000 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:34.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:34.945+0000 7f40baffd700 1 -- 172.21.15.119:0/3402386562 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40c40c9ca0 con 0x7f40c40ac6a0 2023-10-08T01:15:35.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.159+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f40c40c9ca0 con 0x7f40a4066f30 2023-10-08T01:15:35.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.161+0000 7f40baffd700 1 -- 172.21.15.119:0/3402386562 <== mgr.7989 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f40c40c9ca0 con 0x7f40a4066f30 2023-10-08T01:15:35.162 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T01:15:35.162 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T01:15:35.162 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T01:15:35.162 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T01:15:35.162 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T01:15:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f40a4066f30 msgr2=0x7f40a40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:35.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f40a4066f30 0x7f40a40693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f40b400b470 tx=0x7f40b401b000 comp rx=0 tx=0).stop 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40c40ac6a0 msgr2=0x7f40c4168f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40c40ac6a0 0x7f40c4168f80 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00c7c0 tx=0x7f40ac007e00 comp rx=0 tx=0).stop 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 shutdown_connections 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f40a4066f30 0x7f40a40693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f40c40ac230 0x7f40c40aaf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f40c4155c10 0x7f40c41694c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 --2- 172.21.15.119:0/3402386562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f40c40ac6a0 0x7f40c4168f80 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 >> 172.21.15.119:0/3402386562 conn(0x7f40c406a320 msgr2=0x7f40c41548a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 shutdown_connections 2023-10-08T01:15:35.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.163+0000 7f40c95ff700 1 -- 172.21.15.119:0/3402386562 wait complete. 2023-10-08T01:15:35.182 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T01:15:35.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.364+0000 7f8c5a39d700 1 Processor -- start 2023-10-08T01:15:35.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.364+0000 7f8c5a39d700 1 -- start start 2023-10-08T01:15:35.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c5a39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54159a60 0x7f8c540c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:35.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c5a39d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c540c9cb0 con 0x7f8c540ccc00 2023-10-08T01:15:35.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c5a39d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c540c9f30 con 0x7f8c540cb230 2023-10-08T01:15:35.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c5a39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5406dfc0 con 0x7f8c54159a60 2023-10-08T01:15:35.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c58b9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8c540ccc00 0x7f8c54069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43572/0 (socket says 172.21.15.119:43572) 2023-10-08T01:15:35.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c58b9a700 1 -- 172.21.15.119:0/1526215176 learned_addr learned my addr 172.21.15.119:0/1526215176 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:35.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c5939b700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54159a60 0x7f8c540c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:35.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2321180443 0 0) 0x7f8c540c9cb0 con 0x7f8c540ccc00 2023-10-08T01:15:35.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c40003570 con 0x7f8c540ccc00 2023-10-08T01:15:35.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297187223 0 0) 0x7f8c540c9f30 con 0x7f8c540cb230 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c540c9cb0 con 0x7f8c540cb230 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3465284055 0 0) 0x7f8c40003570 con 0x7f8c540ccc00 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54159a60 msgr2=0x7f8c540c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54159a60 0x7f8c540c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 >> v1:172.21.15.184:6789/0 conn(0x7f8c540cb230 legacy=0x7f8c5406d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540ca470 con 0x7f8c540ccc00 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f8c44001470 con 0x7f8c540ccc00 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 >> v1:172.21.15.119:6789/0 conn(0x7f8c540ccc00 legacy=0x7f8c54069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40003c20 0x7f8c40004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54159a60 0x7f8c40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40004d50 0x7f8c400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:35.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5406dfc0 con 0x7f8c40003c20 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4000bb40 con 0x7f8c40004d50 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4000bd00 con 0x7f8c54159a60 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.365+0000 7f8c5939b700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40003c20 0x7f8c40004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c59b9c700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40004d50 0x7f8c400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c58b9a700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54159a60 0x7f8c40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c5939b700 1 -- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54159a60 msgr2=0x7f8c40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c5939b700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54159a60 0x7f8c40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c5939b700 1 -- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40004d50 msgr2=0x7f8c400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c5939b700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40004d50 0x7f8c400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c5939b700 1 -- 172.21.15.119:0/1526215176 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c40003570 con 0x7f8c40003c20 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c59b9c700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40004d50 0x7f8c400051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T01:15:35.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c5939b700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40003c20 0x7f8c40004010 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009c00 tx=0x7f8c500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c81383a5d208f4d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5000e710 con 0x7f8c40003c20 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5000e870 con 0x7f8c40003c20 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.366+0000 7f8c4bfff700 1 -- 172.21.15.119:0/1526215176 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5000ec30 con 0x7f8c40003c20 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.367+0000 7f8c5a39d700 1 -- 172.21.15.119:0/1526215176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40003c20 msgr2=0x7f8c40004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.367+0000 7f8c5a39d700 1 --2- 172.21.15.119:0/1526215176 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c40003c20 0x7f8c40004010 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f8c50009c00 tx=0x7f8c500091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.367+0000 7f8c59b9c700 1 -- 172.21.15.119:0/1526215176 reap_dead start 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.367+0000 7f8c5a39d700 1 -- 172.21.15.119:0/1526215176 shutdown_connections 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.367+0000 7f8c5a39d700 1 -- 172.21.15.119:0/1526215176 >> 172.21.15.119:0/1526215176 conn(0x7f8c5406baa0 msgr2=0x7f8c5406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.367+0000 7f8c5a39d700 1 -- 172.21.15.119:0/1526215176 shutdown_connections 2023-10-08T01:15:35.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.367+0000 7f8c5a39d700 1 -- 172.21.15.119:0/1526215176 wait complete. 2023-10-08T01:15:35.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.368+0000 7f8c5a39d700 1 Processor -- start 2023-10-08T01:15:35.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.368+0000 7f8c5a39d700 1 -- start start 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.369+0000 7f8c5a39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40003c20 0x7f8c540b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.369+0000 7f8c5a39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c540c7ff0 0x7f8c54169910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.369+0000 7f8c5a39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54169e50 0x7f8c54157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.369+0000 7f8c5a39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b1a10 con 0x7f8c540c7ff0 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.369+0000 7f8c5a39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b1cf0 con 0x7f8c40003c20 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.369+0000 7f8c5a39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c540b1fd0 con 0x7f8c54169e50 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.369+0000 7f8c58b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c540c7ff0 0x7f8c54169910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c58b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c540c7ff0 0x7f8c54169910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42540/0 (socket says 172.21.15.119:42540) 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c58b9a700 1 -- 172.21.15.119:0/2624128031 learned_addr learned my addr 172.21.15.119:0/2624128031 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c5939b700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40003c20 0x7f8c540b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:35.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c58b9a700 1 -- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54169e50 msgr2=0x7f8c54157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c59b9c700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54169e50 0x7f8c54157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c58b9a700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54169e50 0x7f8c54157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c58b9a700 1 -- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40003c20 msgr2=0x7f8c540b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c58b9a700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40003c20 0x7f8c540b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c58b9a700 1 -- 172.21.15.119:0/2624128031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c540b22b0 con 0x7f8c540c7ff0 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c5939b700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40003c20 0x7f8c540b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c59b9c700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54169e50 0x7f8c54157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c58b9a700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c540c7ff0 0x7f8c54169910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8c44000fd0 tx=0x7f8c44000d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c497fa700 1 -- 172.21.15.119:0/2624128031 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400c7e0 con 0x7f8c540c7ff0 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c497fa700 1 -- 172.21.15.119:0/2624128031 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4400c940 con 0x7f8c540c7ff0 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c540c8460 con 0x7f8c540c7ff0 2023-10-08T01:15:35.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.370+0000 7f8c497fa700 1 -- 172.21.15.119:0/2624128031 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c4400cd00 con 0x7f8c540c7ff0 2023-10-08T01:15:35.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.371+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c54157960 con 0x7f8c540c7ff0 2023-10-08T01:15:35.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.372+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5416a290 con 0x7f8c540c7ff0 2023-10-08T01:15:35.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.372+0000 7f8c497fa700 1 -- 172.21.15.119:0/2624128031 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 67) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c44013630 con 0x7f8c540c7ff0 2023-10-08T01:15:35.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.373+0000 7f8c497fa700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8c38066f30 0x7f8c380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:35.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.373+0000 7f8c497fa700 1 -- 172.21.15.119:0/2624128031 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f8c44084710 con 0x7f8c540c7ff0 2023-10-08T01:15:35.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.374+0000 7f8c5939b700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8c38066f30 0x7f8c380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:35.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.374+0000 7f8c5939b700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8c38066f30 0x7f8c380693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c50006ed0 tx=0x7f8c5001a000 comp rx=0 tx=0).ready entity=mgr.7989 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:35.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.378+0000 7f8c497fa700 1 -- 172.21.15.119:0/2624128031 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c5416a290 con 0x7f8c540c7ff0 2023-10-08T01:15:35.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:35.624+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f8c5416a220 con 0x7f8c540c7ff0 2023-10-08T01:15:36.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.275+0000 7f8c497fa700 1 -- 172.21.15.119:0/2624128031 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 68) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f8c44051bb0 con 0x7f8c540c7ff0 2023-10-08T01:15:36.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.277+0000 7f8c497fa700 1 -- 172.21.15.119:0/2624128031 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v68) v1 ==== 92+0+0 (secure 0 0 0) 0x7f8c440517e0 con 0x7f8c540c7ff0 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8c38066f30 msgr2=0x7f8c380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8c38066f30 0x7f8c380693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8c50006ed0 tx=0x7f8c5001a000 comp rx=0 tx=0).stop 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c540c7ff0 msgr2=0x7f8c54169910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c540c7ff0 0x7f8c54169910 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f8c44000fd0 tx=0x7f8c44000d90 comp rx=0 tx=0).stop 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 shutdown_connections 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8c38066f30 0x7f8c380693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8c54169e50 0x7f8c54157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8c40003c20 0x7f8c540b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 --2- 172.21.15.119:0/2624128031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c540c7ff0 0x7f8c54169910 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 >> 172.21.15.119:0/2624128031 conn(0x7f8c5406baa0 msgr2=0x7f8c5406a320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:36.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.279+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 shutdown_connections 2023-10-08T01:15:36.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.280+0000 7f8c5a39d700 1 -- 172.21.15.119:0/2624128031 wait complete. 2023-10-08T01:15:36.298 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T01:15:36.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.489+0000 7f19d3fff700 1 Processor -- start 2023-10-08T01:15:36.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.489+0000 7f19d3fff700 1 -- start start 2023-10-08T01:15:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.490+0000 7f19d3fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f19d40b4e70 0x7f19d40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.490+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b16d0 0x7f19d40b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.490+0000 7f19d3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f19d415d240 0x7f19d415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.490+0000 7f19d3fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40b2080 con 0x7f19d415d240 2023-10-08T01:15:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.490+0000 7f19d3fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40cfb00 con 0x7f19d40b4e70 2023-10-08T01:15:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.490+0000 7f19d3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40b5c40 con 0x7f19d40b16d0 2023-10-08T01:15:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f19d415d240 0x7f19d415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d27fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b16d0 0x7f19d40b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f19d415d240 0x7f19d415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42560/0 (socket says 172.21.15.119:42560) 2023-10-08T01:15:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 -- 172.21.15.119:0/3869173373 learned_addr learned my addr 172.21.15.119:0/3869173373 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d2ffd700 1 --2- 172.21.15.119:0/3869173373 >> v2:172.21.15.184:3300/0 conn(0x7f19d40b4e70 0x7f19d40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 -- 172.21.15.119:0/3869173373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b16d0 msgr2=0x7f19d40b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 --2- 172.21.15.119:0/3869173373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b16d0 0x7f19d40b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 -- 172.21.15.119:0/3869173373 >> v2:172.21.15.184:3300/0 conn(0x7f19d40b4e70 msgr2=0x7f19d40ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 --2- 172.21.15.119:0/3869173373 >> v2:172.21.15.184:3300/0 conn(0x7f19d40b4e70 0x7f19d40ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 -- 172.21.15.119:0/3869173373 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d40b5f20 con 0x7f19d415d240 2023-10-08T01:15:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.491+0000 7f19d37fe700 1 --2- 172.21.15.119:0/3869173373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415d6c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f19d4064910 tx=0x7f19cc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2019642a45c2a17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d1ffb700 1 -- 172.21.15.119:0/3869173373 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00e070 con 0x7f19d415d240 2023-10-08T01:15:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d1ffb700 1 -- 172.21.15.119:0/3869173373 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc00b970 con 0x7f19d415d240 2023-10-08T01:15:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d1ffb700 1 -- 172.21.15.119:0/3869173373 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0092e0 con 0x7f19d415d240 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d3fff700 1 -- 172.21.15.119:0/3869173373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 msgr2=0x7f19d415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3869173373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415d6c0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f19d4064910 tx=0x7f19cc009840 comp rx=0 tx=0).stop 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d3fff700 1 -- 172.21.15.119:0/3869173373 shutdown_connections 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3869173373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415d6c0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3869173373 >> v2:172.21.15.184:3300/0 conn(0x7f19d40b4e70 0x7f19d40ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3869173373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b16d0 0x7f19d40b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.492+0000 7f19d3fff700 1 -- 172.21.15.119:0/3869173373 >> 172.21.15.119:0/3869173373 conn(0x7f19d40d10c0 msgr2=0x7f19d40affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.493+0000 7f19d3fff700 1 -- 172.21.15.119:0/3869173373 shutdown_connections 2023-10-08T01:15:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.493+0000 7f19d3fff700 1 -- 172.21.15.119:0/3869173373 wait complete. 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.494+0000 7f19d3fff700 1 Processor -- start 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.494+0000 7f19d3fff700 1 -- start start 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d40b16d0 0x7f19d41471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b4e70 0x7f19d415b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40cae90 con 0x7f19d415d240 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d3fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40ca280 con 0x7f19d40b16d0 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40ca560 con 0x7f19d40b4e70 2023-10-08T01:15:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42572/0 (socket says 172.21.15.119:42572) 2023-10-08T01:15:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d37fe700 1 -- 172.21.15.119:0/3961830780 learned_addr learned my addr 172.21.15.119:0/3961830780 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d2ffd700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d40b16d0 0x7f19d41471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.495+0000 7f19d27fc700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b4e70 0x7f19d415b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d2ffd700 1 -- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b4e70 msgr2=0x7f19d415b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d2ffd700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b4e70 0x7f19d415b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d2ffd700 1 -- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 msgr2=0x7f19d415be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d2ffd700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d2ffd700 1 -- 172.21.15.119:0/3961830780 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19cc009ae0 con 0x7f19d40b16d0 2023-10-08T01:15:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d37fe700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d2ffd700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d40b16d0 0x7f19d41471a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19c400ea50 tx=0x7f19c40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19baffd700 1 -- 172.21.15.119:0/3961830780 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400ee90 con 0x7f19d40b16d0 2023-10-08T01:15:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19baffd700 1 -- 172.21.15.119:0/3961830780 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c400c7f0 con 0x7f19d40b16d0 2023-10-08T01:15:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19baffd700 1 -- 172.21.15.119:0/3961830780 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c400cb90 con 0x7f19d40b16d0 2023-10-08T01:15:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d40ca840 con 0x7f19d40b16d0 2023-10-08T01:15:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.496+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d4147b80 con 0x7f19d40b16d0 2023-10-08T01:15:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.497+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d4088990 con 0x7f19d40b16d0 2023-10-08T01:15:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.498+0000 7f19baffd700 1 -- 172.21.15.119:0/3961830780 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 68) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f19c4013470 con 0x7f19d40b16d0 2023-10-08T01:15:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.499+0000 7f19baffd700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f19c0066f80 0x7f19c0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.499+0000 7f19baffd700 1 -- 172.21.15.119:0/3961830780 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f19c4084960 con 0x7f19d40b16d0 2023-10-08T01:15:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.499+0000 7f19d27fc700 1 -- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f19c0066f80 msgr2=0x7f19c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.499+0000 7f19d27fc700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f19c0066f80 0x7f19c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.504+0000 7f19baffd700 1 -- 172.21.15.119:0/3961830780 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19c4051990 con 0x7f19d40b16d0 2023-10-08T01:15:36.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.700+0000 7f19d27fc700 1 -- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f19c0066f80 msgr2=0x7f19c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:36.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.700+0000 7f19d27fc700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f19c0066f80 0x7f19c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:36.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.704+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f19d404bb80 con 0x7f19d40b16d0 2023-10-08T01:15:36.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.711+0000 7f19baffd700 1 -- 172.21.15.119:0/3961830780 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f19c4054fb0 con 0x7f19d40b16d0 2023-10-08T01:15:36.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f19c0066f80 msgr2=0x7f19c0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:36.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f19c0066f80 0x7f19c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d40b16d0 msgr2=0x7f19d41471a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d40b16d0 0x7f19d41471a0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f19c400ea50 tx=0x7f19c40091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 shutdown_connections 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f19c0066f80 0x7f19c0069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19d415d240 0x7f19d415be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f19d40b16d0 0x7f19d41471a0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 --2- 172.21.15.119:0/3961830780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f19d40b4e70 0x7f19d415b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 >> 172.21.15.119:0/3961830780 conn(0x7f19d40d10c0 msgr2=0x7f19d4159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 shutdown_connections 2023-10-08T01:15:36.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.714+0000 7f19d3fff700 1 -- 172.21.15.119:0/3961830780 wait complete. 2023-10-08T01:15:36.732 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T01:15:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.921+0000 7fb2c259d700 1 Processor -- start 2023-10-08T01:15:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.921+0000 7fb2c259d700 1 -- start start 2023-10-08T01:15:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c259d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb2bc0b4e70 0x7fb2bc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c259d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 0x7fb2bc15d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c259d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0cfb00 con 0x7fb2bc0b16d0 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c259d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b5c40 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c259d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b5f20 con 0x7fb2bc0b1b40 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c159b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb2bc0b4e70 0x7fb2bc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c159b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb2bc0b4e70 0x7fb2bc0ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42580/0 (socket says 172.21.15.119:42580) 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c159b700 1 -- 172.21.15.119:0/3942306356 learned_addr learned my addr 172.21.15.119:0/3942306356 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c1d9c700 1 --2- 172.21.15.119:0/3942306356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 0x7fb2bc15d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c159b700 1 -- 172.21.15.119:0/3942306356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 msgr2=0x7fb2bc15d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c159b700 1 --2- 172.21.15.119:0/3942306356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 0x7fb2bc15d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.923+0000 7fb2c159b700 1 -- 172.21.15.119:0/3942306356 >> v1:172.21.15.119:6789/0 conn(0x7fb2bc0b16d0 legacy=0x7fb2bc15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.924+0000 7fb2c159b700 1 -- 172.21.15.119:0/3942306356 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0b6200 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.924+0000 7fb2b3fff700 1 -- 172.21.15.119:0/3942306356 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275056217 0 0) 0x7fb2bc0cfb00 con 0x7fb2bc0b16d0 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.924+0000 7fb2c159b700 1 --2- 172.21.15.119:0/3942306356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0ccff0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80099b0 tx=0x7fb2b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4372a1e427fbc3b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.924+0000 7fb2b3fff700 1 -- 172.21.15.119:0/3942306356 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b8009ea0 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.924+0000 7fb2b3fff700 1 -- 172.21.15.119:0/3942306356 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b800b860 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.924+0000 7fb2b3fff700 1 -- 172.21.15.119:0/3942306356 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b800bc00 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.925+0000 7fb2c259d700 1 -- 172.21.15.119:0/3942306356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 msgr2=0x7fb2bc0ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.925+0000 7fb2c259d700 1 --2- 172.21.15.119:0/3942306356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0ccff0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fb2b80099b0 tx=0x7fb2b8009250 comp rx=0 tx=0).stop 2023-10-08T01:15:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.925+0000 7fb2c259d700 1 -- 172.21.15.119:0/3942306356 shutdown_connections 2023-10-08T01:15:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.925+0000 7fb2c259d700 1 --2- 172.21.15.119:0/3942306356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 0x7fb2bc15d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.925+0000 7fb2c259d700 1 --2- 172.21.15.119:0/3942306356 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0ccff0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.925+0000 7fb2c259d700 1 -- 172.21.15.119:0/3942306356 >> 172.21.15.119:0/3942306356 conn(0x7fb2bc0d10c0 msgr2=0x7fb2bc0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.925+0000 7fb2c259d700 1 -- 172.21.15.119:0/3942306356 shutdown_connections 2023-10-08T01:15:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.925+0000 7fb2c259d700 1 -- 172.21.15.119:0/3942306356 wait complete. 2023-10-08T01:15:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.926+0000 7fb2c259d700 1 Processor -- start 2023-10-08T01:15:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.926+0000 7fb2c259d700 1 -- start start 2023-10-08T01:15:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.927+0000 7fb2c259d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc0b16d0 0x7fb2bc0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.927+0000 7fb2c259d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 0x7fb2bc0c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.927+0000 7fb2c259d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.927+0000 7fb2c259d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0c9440 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.927+0000 7fb2c259d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0c73f0 con 0x7fb2bc0b16d0 2023-10-08T01:15:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.927+0000 7fb2c259d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0c76d0 con 0x7fb2bc0b1b40 2023-10-08T01:15:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42584/0 (socket says 172.21.15.119:42584) 2023-10-08T01:15:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 -- 172.21.15.119:0/702328411 learned_addr learned my addr 172.21.15.119:0/702328411 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c0d9a700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 0x7fb2bc0c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c159b700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc0b16d0 0x7fb2bc0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 -- 172.21.15.119:0/702328411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 msgr2=0x7fb2bc0c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 0x7fb2bc0c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 -- 172.21.15.119:0/702328411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc0b16d0 msgr2=0x7fb2bc0c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc0b16d0 0x7fb2bc0c6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:36.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 -- 172.21.15.119:0/702328411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80094f0 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c1d9c700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0b3e80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009e90 tx=0x7fb2b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2b17fa700 1 -- 172.21.15.119:0/702328411 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b400cb70 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2b17fa700 1 -- 172.21.15.119:0/702328411 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b400f040 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2b17fa700 1 -- 172.21.15.119:0/702328411 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4014480 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.928+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc0c79b0 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.929+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc0c7f70 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.930+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc088990 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.930+0000 7fb2b17fa700 1 -- 172.21.15.119:0/702328411 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 68) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fb2b4014830 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.931+0000 7fb2b17fa700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2a0066f80 0x7fb2a0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:36.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.931+0000 7fb2b17fa700 1 -- 172.21.15.119:0/702328411 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fb2b40851a0 con 0x7fb2bc0b4e70 2023-10-08T01:15:36.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.931+0000 7fb2c159b700 1 -- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2a0066f80 msgr2=0x7fb2a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:36.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.932+0000 7fb2c159b700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2a0066f80 0x7fb2a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:36.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:36.936+0000 7fb2b17fa700 1 -- 172.21.15.119:0/702328411 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb2b40521d0 con 0x7fb2bc0b4e70 2023-10-08T01:15:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.133+0000 7fb2c159b700 1 -- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2a0066f80 msgr2=0x7fb2a0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.133+0000 7fb2c159b700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2a0066f80 0x7fb2a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:37.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.137+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fb2bc0c5640 con 0x7fb2bc0b4e70 2023-10-08T01:15:37.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.141+0000 7fb2b17fa700 1 -- 172.21.15.119:0/702328411 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb2b40557f0 con 0x7fb2bc0b4e70 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2a0066f80 msgr2=0x7fb2a0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2a0066f80 0x7fb2a0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 msgr2=0x7fb2bc0b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0b3e80 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009e90 tx=0x7fb2b400b040 comp rx=0 tx=0).stop 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 shutdown_connections 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb2a0066f80 0x7fb2a0069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2bc0b4e70 0x7fb2bc0b3e80 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2bc0b16d0 0x7fb2bc0c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 --2- 172.21.15.119:0/702328411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2bc0b1b40 0x7fb2bc0c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.143+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 >> 172.21.15.119:0/702328411 conn(0x7fb2bc0d10c0 msgr2=0x7fb2bc158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.144+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 shutdown_connections 2023-10-08T01:15:37.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.144+0000 7fb2c259d700 1 -- 172.21.15.119:0/702328411 wait complete. 2023-10-08T01:15:37.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T01:15:37.163 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T01:15:37.204 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1 1M 2023-10-08T01:15:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.434+0000 7f2bb219d700 1 Processor -- start 2023-10-08T01:15:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.434+0000 7f2bb219d700 1 -- start start 2023-10-08T01:15:37.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.435+0000 7f2bb219d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac0b03c0 0x7f2bac0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.435+0000 7f2bb219d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2bac0b0790 0x7f2bac0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.435+0000 7f2bb219d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2bac1591e0 0x7f2bac159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.435+0000 7f2bb219d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bac0cfa00 con 0x7f2bac0b0790 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.435+0000 7f2bb219d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bac0cb580 con 0x7f2bac1591e0 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.435+0000 7f2bb219d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac0b2de0 con 0x7f2bac0b03c0 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb099a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2bac0b0790 0x7f2bac0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb099a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2bac0b0790 0x7f2bac0b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42590/0 (socket says 172.21.15.119:42590) 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb099a700 1 -- 172.21.15.119:0/4275497730 learned_addr learned my addr 172.21.15.119:0/4275497730 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:37.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb199c700 1 --2- 172.21.15.119:0/4275497730 >> v2:172.21.15.184:3300/0 conn(0x7f2bac1591e0 0x7f2bac159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb119b700 1 --2- 172.21.15.119:0/4275497730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac0b03c0 0x7f2bac0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb099a700 1 -- 172.21.15.119:0/4275497730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac0b03c0 msgr2=0x7f2bac0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb099a700 1 --2- 172.21.15.119:0/4275497730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac0b03c0 0x7f2bac0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb099a700 1 -- 172.21.15.119:0/4275497730 >> v2:172.21.15.184:3300/0 conn(0x7f2bac1591e0 msgr2=0x7f2bac159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb099a700 1 --2- 172.21.15.119:0/4275497730 >> v2:172.21.15.184:3300/0 conn(0x7f2bac1591e0 0x7f2bac159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.436+0000 7f2bb099a700 1 -- 172.21.15.119:0/4275497730 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac0cf160 con 0x7f2bac0b0790 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2bb099a700 1 --2- 172.21.15.119:0/4275497730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac0b28a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0cb4f0 tx=0x7f2b9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d068021f7b248 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2ba3fff700 1 -- 172.21.15.119:0/4275497730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c00b640 con 0x7f2bac0b0790 2023-10-08T01:15:37.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2ba3fff700 1 -- 172.21.15.119:0/4275497730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c00b7a0 con 0x7f2bac0b0790 2023-10-08T01:15:37.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2ba3fff700 1 -- 172.21.15.119:0/4275497730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c00bb60 con 0x7f2bac0b0790 2023-10-08T01:15:37.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2bb219d700 1 -- 172.21.15.119:0/4275497730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 msgr2=0x7f2bac0b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:37.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2bb219d700 1 --2- 172.21.15.119:0/4275497730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac0b28a0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0cb4f0 tx=0x7f2b9c009250 comp rx=0 tx=0).stop 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2bb219d700 1 -- 172.21.15.119:0/4275497730 shutdown_connections 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2bb219d700 1 --2- 172.21.15.119:0/4275497730 >> v2:172.21.15.184:3300/0 conn(0x7f2bac1591e0 0x7f2bac159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2bb219d700 1 --2- 172.21.15.119:0/4275497730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac0b03c0 0x7f2bac0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2bb219d700 1 --2- 172.21.15.119:0/4275497730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac0b28a0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.437+0000 7f2bb219d700 1 -- 172.21.15.119:0/4275497730 >> 172.21.15.119:0/4275497730 conn(0x7f2bac063230 msgr2=0x7f2bac063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.438+0000 7f2bb219d700 1 -- 172.21.15.119:0/4275497730 shutdown_connections 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.438+0000 7f2bb219d700 1 -- 172.21.15.119:0/4275497730 wait complete. 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.439+0000 7f2bb219d700 1 Processor -- start 2023-10-08T01:15:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.439+0000 7f2bb219d700 1 -- start start 2023-10-08T01:15:37.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb219d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac0b03c0 0x7f2bac16d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:37.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb219d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac16db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:37.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb219d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac1591e0 0x7f2bac0c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:37.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb219d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac16bae0 con 0x7f2bac0b0790 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb219d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac0c5850 con 0x7f2bac0b03c0 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb219d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac16b1b0 con 0x7f2bac1591e0 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb099a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac16db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb099a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac16db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42606/0 (socket says 172.21.15.119:42606) 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb099a700 1 -- 172.21.15.119:0/3717504714 learned_addr learned my addr 172.21.15.119:0/3717504714 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac0b03c0 0x7f2bac16d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.440+0000 7f2bb199c700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac1591e0 0x7f2bac0c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb099a700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac1591e0 msgr2=0x7f2bac0c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb099a700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac1591e0 0x7f2bac0c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb099a700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac0b03c0 msgr2=0x7f2bac16d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb099a700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac0b03c0 0x7f2bac16d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb099a700 1 -- 172.21.15.119:0/3717504714 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0094f0 con 0x7f2bac0b0790 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac0b03c0 0x7f2bac16d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb199c700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac1591e0 0x7f2bac0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb099a700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac16db20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0cfe40 tx=0x7f2b9c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c00b910 con 0x7f2bac0b0790 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c00ba70 con 0x7f2bac0b0790 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bac16b490 con 0x7f2bac0b0790 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.441+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c011630 con 0x7f2bac0b0790 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.442+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bac15bf20 con 0x7f2bac0b0790 2023-10-08T01:15:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.443+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bac16bc90 con 0x7f2bac0b0790 2023-10-08T01:15:37.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.443+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 68) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f2b9c0119a0 con 0x7f2bac0b0790 2023-10-08T01:15:37.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.444+0000 7f2ba17fa700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b90069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:37.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.444+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6125+0+0 (secure 0 0 0) 0x7f2b9c085920 con 0x7f2bac0b0790 2023-10-08T01:15:37.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.444+0000 7f2bb119b700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 msgr2=0x7f2b90069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:37.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.444+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b90069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T01:15:37.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.448+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2bac16bc90 con 0x7f2bac0b0790 2023-10-08T01:15:37.557 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:37.557+0000 7f8b6f1aa180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:15:37.612 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:37.612+0000 7fbceba6d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T01:15:37.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.645+0000 7f2bb119b700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 msgr2=0x7f2b90069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:37.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.645+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b90069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T01:15:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:37.673+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bac0cca20 con 0x7f2b90066fb0 2023-10-08T01:15:37.720 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:37.719+0000 7f8b6f1aa180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:15:37.779 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:37.779+0000 7fbceba6d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T01:15:37.809 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:37.809+0000 7f8b6f1aa180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:15:37.867 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:37.867+0000 7fbceba6d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T01:15:38.033 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:38.033+0000 7f8b6f1aa180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:15:38.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:38.046+0000 7f2bb119b700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 msgr2=0x7f2b90069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:38.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:38.046+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b90069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T01:15:38.092 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:38.092+0000 7fbceba6d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:15:38.209 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:38.209+0000 7f8b6f1aa180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:15:38.272 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:38.272+0000 7fbceba6d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T01:15:38.307 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:38.307+0000 7f8b6f1aa180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:15:38.365 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:38.365+0000 7fbceba6d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T01:15:38.469 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:38.469+0000 7f8b6f1aa180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:15:38.527 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:38.527+0000 7fbceba6d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T01:15:38.572 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:38.572+0000 7f8b6f1aa180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:15:38.627 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:38.627+0000 7fbceba6d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T01:15:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:38.847+0000 7f2bb119b700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 msgr2=0x7f2b90069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:38.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:38.847+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b90069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T01:15:39.056 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:39.055+0000 7f8b6f1aa180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:15:39.110 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:39.110+0000 7fbceba6d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T01:15:39.149 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:39.149+0000 7f8b6f1aa180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:15:39.205 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:39.205+0000 7fbceba6d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T01:15:39.239 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:39.239+0000 7f8b6f1aa180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:15:39.296 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:39.296+0000 7fbceba6d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T01:15:39.861 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:39.861+0000 7f8b6f1aa180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:15:39.913 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:39.913+0000 7fbceba6d180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T01:15:40.034 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:40.034+0000 7f8b6f1aa180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:15:40.086 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:40.086+0000 7fbceba6d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T01:15:40.130 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:40.130+0000 7f8b6f1aa180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:15:40.181 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:40.181+0000 7fbceba6d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T01:15:40.425 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:40.424+0000 7f8b6f1aa180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:15:40.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:40.448+0000 7f2bb119b700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 msgr2=0x7f2b90069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T01:15:40.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:40.448+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b90069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T01:15:40.474 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:40.474+0000 7fbceba6d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T01:15:40.521 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:40.521+0000 7f8b6f1aa180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:15:40.569 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:40.569+0000 7fbceba6d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T01:15:41.218 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:41.217+0000 7f8b6f1aa180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:15:41.268 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:41.268+0000 7fbceba6d180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T01:15:41.504 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:41.504+0000 7f8b6f1aa180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:15:41.553 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:41.553+0000 7fbceba6d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T01:15:41.597 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:41.596+0000 7f8b6f1aa180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:15:41.645 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:41.645+0000 7fbceba6d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T01:15:41.750 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:41.749+0000 7f8b6f1aa180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:15:41.799 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:41.799+0000 7fbceba6d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T01:15:41.855 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:41.854+0000 7f8b6f1aa180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:15:41.905 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:41.905+0000 7fbceba6d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T01:15:42.041 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:42.041+0000 7f8b6f1aa180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:15:42.094 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.094+0000 7fbceba6d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T01:15:42.132 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:42.131+0000 7f8b6f1aa180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:15:42.188 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.188+0000 7fbceba6d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T01:15:42.222 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:42.222+0000 7f8b6f1aa180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:15:42.280 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.280+0000 7fbceba6d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T01:15:42.315 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T01:15:42.314+0000 7f8b6f1aa180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:15:42.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:42.374+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 69) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f2b9c052e50 con 0x7f2bac0b0790 2023-10-08T01:15:42.375 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.375+0000 7fbceba6d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T01:15:42.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:42.387+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 70) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f2b9c01fcd0 con 0x7f2bac0b0790 2023-10-08T01:15:42.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:42.387+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 msgr2=0x7f2b90069460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T01:15:42.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:42.387+0000 7f2ba17fa700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b90069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:42.508 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc5ddb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.508 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc5ddb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.508 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc5ddb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.508 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc5ddb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.508 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc5ddb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.509 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc60db7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.509 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc60db7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.509 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc60db7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.509 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc60db7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:42.509 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:15:42.507+0000 7fbc60db7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:15:43.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.392+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mon.0 v2:172.21.15.119:3300/0 9 ==== mgrmap(e 71) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f2b9c053720 con 0x7f2bac0b0790 2023-10-08T01:15:43.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.392+0000 7f2ba17fa700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:43.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.392+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bac0cca20 con 0x7f2b90066fb0 2023-10-08T01:15:43.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.394+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.398+0000 7f2bb119b700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b900693e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009a30 tx=0x7f2ba80091a0 comp rx=0 tx=0).ready entity=mgr.8376 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:43.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.464+0000 7f2ba17fa700 1 -- 172.21.15.119:0/3717504714 <== mgr.8376 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f2bac0cca20 con 0x7f2b90066fb0 2023-10-08T01:15:43.465 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1 2023-10-08T01:15:43.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 msgr2=0x7f2b900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b900693e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009a30 tx=0x7f2ba80091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 msgr2=0x7f2bac16db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac16db20 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f2bac0cfe40 tx=0x7f2b9c017000 comp rx=0 tx=0).stop 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 shutdown_connections 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2b90066fb0 0x7f2b900693e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2bac1591e0 0x7f2bac0c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2bac0b03c0 0x7f2bac16d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 --2- 172.21.15.119:0/3717504714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bac0b0790 0x7f2bac16db20 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 >> 172.21.15.119:0/3717504714 conn(0x7f2bac063230 msgr2=0x7f2bac1589d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 shutdown_connections 2023-10-08T01:15:43.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.467+0000 7f2bb219d700 1 -- 172.21.15.119:0/3717504714 wait complete. 2023-10-08T01:15:43.487 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-10-08T01:15:43.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs /snap_test_dir1 55n 2023-10-08T01:15:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.676+0000 7f8d13fff700 1 Processor -- start 2023-10-08T01:15:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.676+0000 7f8d13fff700 1 -- start start 2023-10-08T01:15:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.677+0000 7f8d13fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cce20 0x7f8d0c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:43.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.677+0000 7f8d13fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8d0c15df40 0x7f8d0c15e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:43.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.677+0000 7f8d13fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0c157c50 con 0x7f8d0c0cca50 2023-10-08T01:15:43.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.677+0000 7f8d13fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d0c15d7d0 con 0x7f8d0c15df40 2023-10-08T01:15:43.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.677+0000 7f8d13fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c15aa20 con 0x7f8d0c0cce20 2023-10-08T01:15:43.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.677+0000 7f8d11d9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8d0c0cca50 0x7f8d0c15a1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43594/0 (socket says 172.21.15.119:43594) 2023-10-08T01:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.677+0000 7f8d11d9b700 1 -- 172.21.15.119:0/2813796938 learned_addr learned my addr 172.21.15.119:0/2813796938 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:43.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.677+0000 7f8d1259c700 1 --2- 172.21.15.119:0/2813796938 >> v2:172.21.15.119:3300/0 conn(0x7f8d0c15df40 0x7f8d0c15e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d1159a700 1 --2- 172.21.15.119:0/2813796938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cce20 0x7f8d0c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d1259c700 1 -- 172.21.15.119:0/2813796938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cce20 msgr2=0x7f8d0c0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d1259c700 1 --2- 172.21.15.119:0/2813796938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cce20 0x7f8d0c0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d1259c700 1 -- 172.21.15.119:0/2813796938 >> v1:172.21.15.119:6789/0 conn(0x7f8d0c0cca50 legacy=0x7f8d0c15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:43.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d1259c700 1 -- 172.21.15.119:0/2813796938 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c15ad00 con 0x7f8d0c15df40 2023-10-08T01:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d1159a700 1 --2- 172.21.15.119:0/2813796938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cce20 0x7f8d0c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T01:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d1259c700 1 --2- 172.21.15.119:0/2813796938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c15e380 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e60 tx=0x7f8d04009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff37ae034427ef70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d10d99700 1 -- 172.21.15.119:0/2813796938 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400e070 con 0x7f8d0c15df40 2023-10-08T01:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d10d99700 1 -- 172.21.15.119:0/2813796938 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0400bc30 con 0x7f8d0c15df40 2023-10-08T01:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.678+0000 7f8d10d99700 1 -- 172.21.15.119:0/2813796938 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04012430 con 0x7f8d0c15df40 2023-10-08T01:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.679+0000 7f8d13fff700 1 -- 172.21.15.119:0/2813796938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 msgr2=0x7f8d0c15e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.679+0000 7f8d13fff700 1 --2- 172.21.15.119:0/2813796938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c15e380 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009e60 tx=0x7f8d04009780 comp rx=0 tx=0).stop 2023-10-08T01:15:43.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.679+0000 7f8d13fff700 1 -- 172.21.15.119:0/2813796938 shutdown_connections 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.679+0000 7f8d13fff700 1 --2- 172.21.15.119:0/2813796938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c15e380 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.679+0000 7f8d13fff700 1 --2- 172.21.15.119:0/2813796938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cce20 0x7f8d0c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.679+0000 7f8d13fff700 1 -- 172.21.15.119:0/2813796938 >> 172.21.15.119:0/2813796938 conn(0x7f8d0c0ce5c0 msgr2=0x7f8d0c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.679+0000 7f8d13fff700 1 -- 172.21.15.119:0/2813796938 shutdown_connections 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.679+0000 7f8d13fff700 1 -- 172.21.15.119:0/2813796938 wait complete. 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.680+0000 7f8d13fff700 1 Processor -- start 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.680+0000 7f8d13fff700 1 -- start start 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.681+0000 7f8d13fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cca50 0x7f8d0c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:43.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.681+0000 7f8d13fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d0c0cce20 0x7f8d0c0b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.681+0000 7f8d13fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c0c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.681+0000 7f8d13fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c15a520 con 0x7f8d0c15df40 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.681+0000 7f8d13fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0c8110 con 0x7f8d0c0cce20 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.681+0000 7f8d13fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c154ac0 con 0x7f8d0c0cca50 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c0c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c0c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42792/0 (socket says 172.21.15.119:42792) 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 -- 172.21.15.119:0/752507108 learned_addr learned my addr 172.21.15.119:0/752507108 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d11d9b700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cca50 0x7f8d0c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1159a700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d0c0cce20 0x7f8d0c0b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 -- 172.21.15.119:0/752507108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cca50 msgr2=0x7f8d0c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cca50 0x7f8d0c154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 -- 172.21.15.119:0/752507108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d0c0cce20 msgr2=0x7f8d0c0b2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:43.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d0c0cce20 0x7f8d0c0b2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 -- 172.21.15.119:0/752507108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04009a20 con 0x7f8d0c15df40 2023-10-08T01:15:43.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.682+0000 7f8d1259c700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c0c85a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f8d04013000 tx=0x7f8d04019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:43.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.683+0000 7f8d01ffb700 1 -- 172.21.15.119:0/752507108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d04012e90 con 0x7f8d0c15df40 2023-10-08T01:15:43.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.683+0000 7f8d01ffb700 1 -- 172.21.15.119:0/752507108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0401e630 con 0x7f8d0c15df40 2023-10-08T01:15:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.683+0000 7f8d01ffb700 1 -- 172.21.15.119:0/752507108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0401e9d0 con 0x7f8d0c15df40 2023-10-08T01:15:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.683+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c0b1f40 con 0x7f8d0c15df40 2023-10-08T01:15:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.683+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c0c7da0 con 0x7f8d0c15df40 2023-10-08T01:15:43.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.684+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c0c82c0 con 0x7f8d0c15df40 2023-10-08T01:15:43.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.685+0000 7f8d01ffb700 1 -- 172.21.15.119:0/752507108 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 71) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f8d04024070 con 0x7f8d0c15df40 2023-10-08T01:15:43.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.685+0000 7f8d01ffb700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8cf0066dd0 0x7f8cf0069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:43.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.685+0000 7f8d01ffb700 1 -- 172.21.15.119:0/752507108 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6383+0+0 (secure 0 0 0) 0x7f8d04086d80 con 0x7f8d0c15df40 2023-10-08T01:15:43.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.686+0000 7f8d11d9b700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8cf0066dd0 0x7f8cf0069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:43.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.686+0000 7f8d11d9b700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8cf0066dd0 0x7f8cf0069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d08001950 tx=0x7f8d08009040 comp rx=0 tx=0).ready entity=mgr.8376 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:43.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.690+0000 7f8d01ffb700 1 -- 172.21.15.119:0/752507108 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8d04053f30 con 0x7f8d0c15df40 2023-10-08T01:15:43.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.903+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d0c0c9870 con 0x7f8cf0066dd0 2023-10-08T01:15:43.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.964+0000 7f8d01ffb700 1 -- 172.21.15.119:0/752507108 <== mgr.8376 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f8d080015e0 con 0x7f8cf0066dd0 2023-10-08T01:15:43.965 INFO:teuthology.orchestra.run.smithi119.stdout:Retention added to path /snap_test_dir1 2023-10-08T01:15:43.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8cf0066dd0 msgr2=0x7f8cf0069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:43.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8cf0066dd0 0x7f8cf0069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8d08001950 tx=0x7f8d08009040 comp rx=0 tx=0).stop 2023-10-08T01:15:43.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 msgr2=0x7f8d0c0c85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:43.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c0c85a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f8d04013000 tx=0x7f8d04019000 comp rx=0 tx=0).stop 2023-10-08T01:15:43.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 shutdown_connections 2023-10-08T01:15:43.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8cf0066dd0 0x7f8cf0069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8d0c0cca50 0x7f8d0c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8d0c15df40 0x7f8d0c0c85a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 --2- 172.21.15.119:0/752507108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8d0c0cce20 0x7f8d0c0b2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:43.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 >> 172.21.15.119:0/752507108 conn(0x7f8d0c0ce5c0 msgr2=0x7f8d0c0ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:43.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 shutdown_connections 2023-10-08T01:15:43.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:43.967+0000 7f8d13fff700 1 -- 172.21.15.119:0/752507108 wait complete. 2023-10-08T01:15:43.985 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-10-08T01:15:43.985 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-10-08T01:15:43.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json /snap_test_dir1 2023-10-08T01:15:44.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.174+0000 7f2d6ebff700 1 Processor -- start 2023-10-08T01:15:44.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.174+0000 7f2d6ebff700 1 -- start start 2023-10-08T01:15:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.175+0000 7f2d6ebff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2d680b2100 0x7f2d68068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.175+0000 7f2d6ebff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d680ae6d0 con 0x7f2d680688d0 2023-10-08T01:15:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.175+0000 7f2d6ebff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d68068d30 con 0x7f2d68157740 2023-10-08T01:15:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.175+0000 7f2d6ebff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d680acc00 con 0x7f2d680b2100 2023-10-08T01:15:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.175+0000 7f2d6d19c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2d680688d0 0x7f2d6806e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43596/0 (socket says 172.21.15.119:43596) 2023-10-08T01:15:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.175+0000 7f2d6d19c700 1 -- 172.21.15.119:0/3497702429 learned_addr learned my addr 172.21.15.119:0/3497702429 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T01:15:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d67fff700 1 --2- 172.21.15.119:0/3497702429 >> v2:172.21.15.184:3300/0 conn(0x7f2d680b2100 0x7f2d68068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168122737 0 0) 0x7f2d60001490 con 0x7f2d680688d0 2023-10-08T01:15:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d58003570 con 0x7f2d680688d0 2023-10-08T01:15:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1836363351 0 0) 0x7f2d58003570 con 0x7f2d680688d0 2023-10-08T01:15:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 >> v2:172.21.15.184:3300/0 conn(0x7f2d680b2100 msgr2=0x7f2d68068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 --2- 172.21.15.119:0/3497702429 >> v2:172.21.15.184:3300/0 conn(0x7f2d680b2100 0x7f2d68068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 >> v1:172.21.15.184:6789/0 conn(0x7f2d68157740 legacy=0x7f2d680adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:44.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6806dc20 con 0x7f2d680688d0 2023-10-08T01:15:44.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f2d60001980 con 0x7f2d680688d0 2023-10-08T01:15:44.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 >> v1:172.21.15.119:6789/0 conn(0x7f2d680688d0 legacy=0x7f2d6806e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:44.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58003ab0 0x7f2d58003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:44.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:44.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d58005210 0x7f2d5800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d680acc00 con 0x7f2d58004810 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800c130 con 0x7f2d58005210 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.176+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5800c2f0 con 0x7f2d58003ab0 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d67fff700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d6d19c700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d58005210 0x7f2d5800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d6c99b700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58003ab0 0x7f2d58003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d67fff700 1 -- 172.21.15.119:0/3497702429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58003ab0 msgr2=0x7f2d58003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d67fff700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58003ab0 0x7f2d58003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d67fff700 1 -- 172.21.15.119:0/3497702429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d58005210 msgr2=0x7f2d5800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d67fff700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d58005210 0x7f2d5800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d67fff700 1 -- 172.21.15.119:0/3497702429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d58003570 con 0x7f2d58004810 2023-10-08T01:15:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d6d19c700 1 -- 172.21.15.119:0/3497702429 reap_dead start 2023-10-08T01:15:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d67fff700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d58004810 0x7f2d58004c80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009c00 tx=0x7f2d540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de8939928c39dcc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400e710 con 0x7f2d58004810 2023-10-08T01:15:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5400e870 con 0x7f2d58004810 2023-10-08T01:15:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.177+0000 7f2d677fe700 1 -- 172.21.15.119:0/3497702429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5400ec30 con 0x7f2d58004810 2023-10-08T01:15:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.178+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3497702429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d58004810 msgr2=0x7f2d58004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.178+0000 7f2d6ebff700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d58004810 0x7f2d58004c80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f2d54009c00 tx=0x7f2d540091a0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.178+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3497702429 shutdown_connections 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.178+0000 7f2d6ebff700 1 --2- 172.21.15.119:0/3497702429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d58004810 0x7f2d58004c80 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.178+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3497702429 >> 172.21.15.119:0/3497702429 conn(0x7f2d680c8f90 msgr2=0x7f2d680c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.178+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3497702429 shutdown_connections 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.178+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3497702429 wait complete. 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.179+0000 7f2d6ebff700 1 Processor -- start 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.180+0000 7f2d6ebff700 1 -- start start 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.180+0000 7f2d6ebff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58004810 0x7f2d680c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.180+0000 7f2d6ebff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681501f0 0x7f2d6815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.180+0000 7f2d6ebff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68153260 0x7f2d6815b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.180+0000 7f2d6ebff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681521c0 con 0x7f2d681501f0 2023-10-08T01:15:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.180+0000 7f2d6ebff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d681524a0 con 0x7f2d68153260 2023-10-08T01:15:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.180+0000 7f2d6ebff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d68152780 con 0x7f2d58004810 2023-10-08T01:15:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681501f0 0x7f2d6815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681501f0 0x7f2d6815aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42814/0 (socket says 172.21.15.119:42814) 2023-10-08T01:15:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 -- 172.21.15.119:0/3783979482 learned_addr learned my addr 172.21.15.119:0/3783979482 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T01:15:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d6c99b700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58004810 0x7f2d680c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d6d19c700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68153260 0x7f2d6815b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 -- 172.21.15.119:0/3783979482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58004810 msgr2=0x7f2d680c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58004810 0x7f2d680c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 -- 172.21.15.119:0/3783979482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68153260 msgr2=0x7f2d6815b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68153260 0x7f2d6815b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 -- 172.21.15.119:0/3783979482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d68152a60 con 0x7f2d681501f0 2023-10-08T01:15:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d67fff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681501f0 0x7f2d6815aef0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2d580057d0 tx=0x7f2d5401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.181+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3783979482 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d54022070 con 0x7f2d681501f0 2023-10-08T01:15:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.182+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3783979482 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5400e9e0 con 0x7f2d681501f0 2023-10-08T01:15:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.182+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3783979482 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5401c450 con 0x7f2d681501f0 2023-10-08T01:15:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.182+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d68152d40 con 0x7f2d681501f0 2023-10-08T01:15:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.182+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6815b970 con 0x7f2d681501f0 2023-10-08T01:15:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.183+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d68152ed0 con 0x7f2d681501f0 2023-10-08T01:15:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.183+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3783979482 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 71) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f2d5401c880 con 0x7f2d681501f0 2023-10-08T01:15:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.184+0000 7f2d64ff9700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d4c066dd0 0x7f2d4c069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T01:15:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.184+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3783979482 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6383+0+0 (secure 0 0 0) 0x7f2d54085e20 con 0x7f2d681501f0 2023-10-08T01:15:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.184+0000 7f2d6c99b700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d4c066dd0 0x7f2d4c069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T01:15:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.185+0000 7f2d6c99b700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d4c066dd0 0x7f2d4c069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c000c00 tx=0x7f2d5c009040 comp rx=0 tx=0).ready entity=mgr.8376 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T01:15:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.188+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3783979482 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d54053080 con 0x7f2d681501f0 2023-10-08T01:15:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.394+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3783979482 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 72) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f2d5408b020 con 0x7f2d681501f0 2023-10-08T01:15:44.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.407+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d68151fa0 con 0x7f2d4c066dd0 2023-10-08T01:15:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.416+0000 7f2d64ff9700 1 -- 172.21.15.119:0/3783979482 <== mgr.8376 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f2d5c0038a0 con 0x7f2d4c066dd0 2023-10-08T01:15:44.416 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T01:15:44.416 INFO:teuthology.orchestra.run.smithi119.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-10-08T01:15:44.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d4c066dd0 msgr2=0x7f2d4c069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d4c066dd0 0x7f2d4c069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c000c00 tx=0x7f2d5c009040 comp rx=0 tx=0).stop 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681501f0 msgr2=0x7f2d6815aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681501f0 0x7f2d6815aef0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f2d580057d0 tx=0x7f2d5401a000 comp rx=0 tx=0).stop 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 shutdown_connections 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2d4c066dd0 0x7f2d4c069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d58004810 0x7f2d680c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d68153260 0x7f2d6815b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 --2- 172.21.15.119:0/3783979482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d681501f0 0x7f2d6815aef0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 >> 172.21.15.119:0/3783979482 conn(0x7f2d680c8f90 msgr2=0x7f2d680b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 shutdown_connections 2023-10-08T01:15:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T01:15:44.418+0000 7f2d6ebff700 1 -- 172.21.15.119:0/3783979482 wait complete. 2023-10-08T01:15:44.436 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-10-08T01:15:44.437 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-10-08T01:15:48.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:15:48.309 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:16:00.977 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:16:00.976+0000 7fbc4fd95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:16:00.977 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:16:00.976+0000 7fbc4fd95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:16:00.977 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:16:00.976+0000 7fbc4fd95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:16:00.977 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:16:00.976+0000 7fbc4fd95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:16:00.978 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T01:16:00.976+0000 7fbc4fd95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T01:16:18.452 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:16:18.458 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:16:48.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:16:48.568 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:17:18.676 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:17:18.682 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:17:48.788 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:17:48.794 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:18:18.893 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:18:18.899 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:18:49.006 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:18:49.012 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:19:19.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:19:19.150 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:19:49.259 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:19:49.264 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:20:19.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:20:19.401 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:20:49.509 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:20:49.514 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:21:19.648 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:21:19.653 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:21:49.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:21:49.784 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:22:19.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:22:19.947 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:22:50.083 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:22:50.089 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:23:20.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:23:20.191 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:23:50.297 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:23:50.302 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:24:20.403 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:24:20.409 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:24:50.557 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:24:50.567 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:25:20.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:25:20.689 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:25:50.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:25:50.864 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:26:21.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:26:21.035 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:26:51.200 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:26:51.206 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:27:21.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:27:21.403 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:27:51.560 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:27:51.566 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:28:21.723 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:28:21.729 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:28:51.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:28:51.886 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:29:22.058 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:29:22.064 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:29:52.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:29:52.238 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:30:22.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:30:22.430 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:30:52.597 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:30:52.602 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:31:22.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:31:22.765 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:31:52.906 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:31:52.912 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:32:23.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:32:23.083 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:32:53.248 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:32:53.254 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:33:23.419 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:33:23.424 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:33:53.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:33:53.597 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:34:23.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:34:23.785 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:34:53.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:34:53.973 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:35:24.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:35:24.203 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:35:54.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:35:54.361 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:36:24.543 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:36:24.798 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:36:54.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:36:54.902 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:37:25.094 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:37:25.100 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:37:55.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:37:55.293 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:38:25.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:38:25.482 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:38:55.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:38:55.666 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:39:25.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:39:25.779 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:39:55.920 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:39:55.926 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:40:26.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:40:26.122 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:40:56.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:40:56.320 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:41:26.503 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:41:26.509 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:41:56.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:41:56.681 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:42:26.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:42:26.868 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:42:57.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:42:57.048 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:43:27.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:43:27.219 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:43:57.377 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:43:57.383 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:44:27.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:44:27.554 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:44:57.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:44:57.741 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:45:27.907 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:45:27.912 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:45:58.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:45:58.076 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:46:28.241 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:46:28.247 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:46:58.346 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:46:58.352 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:47:28.459 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:47:28.465 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:47:58.648 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:47:58.654 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:48:28.820 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:48:28.826 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:48:58.958 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:48:58.964 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:49:29.121 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:49:29.127 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:49:59.287 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:49:59.293 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:50:29.484 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:50:29.490 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:50:59.654 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:50:59.660 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:51:29.761 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:51:29.767 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:51:59.933 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:51:59.938 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:52:30.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:52:30.074 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:53:00.232 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:53:00.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:53:30.405 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:53:30.425 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:54:00.568 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:54:00.574 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:54:30.763 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:54:30.769 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:55:00.961 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:55:00.966 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:55:31.091 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:55:31.096 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:56:01.262 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:56:01.267 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:56:31.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:56:31.447 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:57:01.631 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:57:01.636 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:57:31.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:57:31.808 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:58:01.991 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:58:02.000 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:58:32.140 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:58:32.149 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:59:02.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:59:02.315 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:59:32.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T01:59:32.510 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:00:02.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:00:02.707 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:00:32.915 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:00:32.965 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:01:03.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:01:03.066 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:01:33.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:01:33.249 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:02:03.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:02:03.428 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:02:33.543 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:02:33.634 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:03:03.740 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:03:03.745 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:03:33.928 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:03:33.933 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:04:04.065 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:04:04.070 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:04:34.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:04:34.217 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:05:04.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:05:04.397 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:05:34.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:05:34.611 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:06:04.769 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:06:04.775 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:06:34.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:06:34.946 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:07:05.104 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:07:05.111 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:07:35.234 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:07:35.239 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:08:05.420 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:08:05.426 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:08:35.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:08:35.614 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:09:05.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:09:05.786 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:09:35.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:09:35.942 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:10:06.106 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:10:06.112 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:10:36.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:10:36.310 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:11:06.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:11:06.481 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:11:16.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:16.438 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:11:16.486 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_17_00_UTC 2023-10-08T02:11:16.486 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_18_00_UTC 2023-10-08T02:11:16.486 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_19_00_UTC 2023-10-08T02:11:16.486 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_20_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_21_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_22_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_23_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_24_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_25_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_26_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_27_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_28_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_29_00_UTC 2023-10-08T02:11:16.487 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_30_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_31_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_32_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_33_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_34_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_35_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_36_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_37_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_38_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_39_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_40_00_UTC 2023-10-08T02:11:16.488 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_41_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_42_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_43_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_44_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_45_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_46_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_47_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_48_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_49_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_50_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_51_00_UTC 2023-10-08T02:11:16.489 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_52_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_53_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_54_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_55_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_56_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_57_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_58_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_59_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_00_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_01_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_02_00_UTC 2023-10-08T02:11:16.490 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_03_00_UTC 2023-10-08T02:11:16.491 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_04_00_UTC 2023-10-08T02:11:16.491 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_05_00_UTC 2023-10-08T02:11:16.491 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_06_00_UTC 2023-10-08T02:11:16.491 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_07_00_UTC 2023-10-08T02:11:16.491 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_08_00_UTC 2023-10-08T02:11:16.491 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_09_00_UTC 2023-10-08T02:11:16.491 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_10_00_UTC 2023-10-08T02:11:16.491 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_11_00_UTC 2023-10-08T02:11:16.492 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-01_17_00_UTC', 'scheduled-2023-10-08-01_18_00_UTC', 'scheduled-2023-10-08-01_19_00_UTC', 'scheduled-2023-10-08-01_20_00_UTC', 'scheduled-2023-10-08-01_21_00_UTC', 'scheduled-2023-10-08-01_22_00_UTC', 'scheduled-2023-10-08-01_23_00_UTC', 'scheduled-2023-10-08-01_24_00_UTC', 'scheduled-2023-10-08-01_25_00_UTC', 'scheduled-2023-10-08-01_26_00_UTC', 'scheduled-2023-10-08-01_27_00_UTC', 'scheduled-2023-10-08-01_28_00_UTC', 'scheduled-2023-10-08-01_29_00_UTC', 'scheduled-2023-10-08-01_30_00_UTC', 'scheduled-2023-10-08-01_31_00_UTC', 'scheduled-2023-10-08-01_32_00_UTC', 'scheduled-2023-10-08-01_33_00_UTC', 'scheduled-2023-10-08-01_34_00_UTC', 'scheduled-2023-10-08-01_35_00_UTC', 'scheduled-2023-10-08-01_36_00_UTC', 'scheduled-2023-10-08-01_37_00_UTC', 'scheduled-2023-10-08-01_38_00_UTC', 'scheduled-2023-10-08-01_39_00_UTC', 'scheduled-2023-10-08-01_40_00_UTC', 'scheduled-2023-10-08-01_41_00_UTC', 'scheduled-2023-10-08-01_42_00_UTC', 'scheduled-2023-10-08-01_43_00_UTC', 'scheduled-2023-10-08-01_44_00_UTC', 'scheduled-2023-10-08-01_45_00_UTC', 'scheduled-2023-10-08-01_46_00_UTC', 'scheduled-2023-10-08-01_47_00_UTC', 'scheduled-2023-10-08-01_48_00_UTC', 'scheduled-2023-10-08-01_49_00_UTC', 'scheduled-2023-10-08-01_50_00_UTC', 'scheduled-2023-10-08-01_51_00_UTC', 'scheduled-2023-10-08-01_52_00_UTC', 'scheduled-2023-10-08-01_53_00_UTC', 'scheduled-2023-10-08-01_54_00_UTC', 'scheduled-2023-10-08-01_55_00_UTC', 'scheduled-2023-10-08-01_56_00_UTC', 'scheduled-2023-10-08-01_57_00_UTC', 'scheduled-2023-10-08-01_58_00_UTC', 'scheduled-2023-10-08-01_59_00_UTC', 'scheduled-2023-10-08-02_00_00_UTC', 'scheduled-2023-10-08-02_01_00_UTC', 'scheduled-2023-10-08-02_02_00_UTC', 'scheduled-2023-10-08-02_03_00_UTC', 'scheduled-2023-10-08-02_04_00_UTC', 'scheduled-2023-10-08-02_05_00_UTC', 'scheduled-2023-10-08-02_06_00_UTC', 'scheduled-2023-10-08-02_07_00_UTC', 'scheduled-2023-10-08-02_08_00_UTC', 'scheduled-2023-10-08-02_09_00_UTC', 'scheduled-2023-10-08-02_10_00_UTC', 'scheduled-2023-10-08-02_11_00_UTC'] 2023-10-08T02:11:16.492 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1 2023-10-08T02:11:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.681+0000 7fe6d53ff700 1 Processor -- start 2023-10-08T02:11:16.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.682+0000 7fe6d53ff700 1 -- start start 2023-10-08T02:11:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6d53ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe6d01570c0 0x7fe6d00c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6d53ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d00ac030 con 0x7fe6d00c8490 2023-10-08T02:11:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6d53ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d00ab820 con 0x7fe6d0069a50 2023-10-08T02:11:16.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6d53ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d006dbb0 con 0x7fe6d01570c0 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6ce7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe6d00c8490 0x7fe6d01569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35684/0 (socket says 172.21.15.119:35684) 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6ce7fc700 1 -- 172.21.15.119:0/495112283 learned_addr learned my addr 172.21.15.119:0/495112283 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6cf7fe700 1 --2- 172.21.15.119:0/495112283 >> v2:172.21.15.184:3300/0 conn(0x7fe6d01570c0 0x7fe6d00c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6cdffb700 1 -- 172.21.15.119:0/495112283 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431473371 0 0) 0x7fe6c4001110 con 0x7fe6d00c8490 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.683+0000 7fe6cdffb700 1 -- 172.21.15.119:0/495112283 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6b8003570 con 0x7fe6d00c8490 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.684+0000 7fe6cf7fe700 1 -- 172.21.15.119:0/495112283 >> v1:172.21.15.184:6789/0 conn(0x7fe6d0069a50 legacy=0x7fe6d015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.684+0000 7fe6cf7fe700 1 -- 172.21.15.119:0/495112283 >> v1:172.21.15.119:6789/0 conn(0x7fe6d00c8490 legacy=0x7fe6d01569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.684+0000 7fe6cf7fe700 1 -- 172.21.15.119:0/495112283 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d00aba70 con 0x7fe6d01570c0 2023-10-08T02:11:16.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.684+0000 7fe6cf7fe700 1 --2- 172.21.15.119:0/495112283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d01570c0 0x7fe6d00c6ac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009e60 tx=0x7fe6c8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=588e56186f34ce04 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:16.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.684+0000 7fe6cdffb700 1 -- 172.21.15.119:0/495112283 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c800bb60 con 0x7fe6d01570c0 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.684+0000 7fe6cdffb700 1 -- 172.21.15.119:0/495112283 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c800bcc0 con 0x7fe6d01570c0 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.684+0000 7fe6cdffb700 1 -- 172.21.15.119:0/495112283 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c80116b0 con 0x7fe6d01570c0 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.685+0000 7fe6d53ff700 1 -- 172.21.15.119:0/495112283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d01570c0 msgr2=0x7fe6d00c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.685+0000 7fe6d53ff700 1 --2- 172.21.15.119:0/495112283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d01570c0 0x7fe6d00c6ac0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8009e60 tx=0x7fe6c8009780 comp rx=0 tx=0).stop 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.685+0000 7fe6d53ff700 1 -- 172.21.15.119:0/495112283 shutdown_connections 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.685+0000 7fe6d53ff700 1 --2- 172.21.15.119:0/495112283 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d01570c0 0x7fe6d00c6ac0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.685+0000 7fe6d53ff700 1 -- 172.21.15.119:0/495112283 >> 172.21.15.119:0/495112283 conn(0x7fe6d00c8f90 msgr2=0x7fe6d00c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.685+0000 7fe6d53ff700 1 -- 172.21.15.119:0/495112283 shutdown_connections 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.685+0000 7fe6d53ff700 1 -- 172.21.15.119:0/495112283 wait complete. 2023-10-08T02:11:16.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.686+0000 7fe6d53ff700 1 Processor -- start 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.686+0000 7fe6d53ff700 1 -- start start 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.687+0000 7fe6d53ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d0069a50 0x7fe6d01697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.687+0000 7fe6d53ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d00c8490 0x7fe6d00cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.687+0000 7fe6d53ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6d01570c0 0x7fe6d01547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.687+0000 7fe6d53ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d0169d30 con 0x7fe6d0069a50 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.687+0000 7fe6d53ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00bfe40 con 0x7fe6d00c8490 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.687+0000 7fe6d53ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00c0120 con 0x7fe6d01570c0 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.687+0000 7fe6ceffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d0069a50 0x7fe6d01697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:16.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ceffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d0069a50 0x7fe6d01697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36916/0 (socket says 172.21.15.119:36916) 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ceffd700 1 -- 172.21.15.119:0/1256925495 learned_addr learned my addr 172.21.15.119:0/1256925495 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ce7fc700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d00c8490 0x7fe6d00cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6cf7fe700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6d01570c0 0x7fe6d01547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ceffd700 1 -- 172.21.15.119:0/1256925495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6d01570c0 msgr2=0x7fe6d01547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ceffd700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6d01570c0 0x7fe6d01547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ceffd700 1 -- 172.21.15.119:0/1256925495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d00c8490 msgr2=0x7fe6d00cb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ceffd700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d00c8490 0x7fe6d00cb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ceffd700 1 -- 172.21.15.119:0/1256925495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c8009a20 con 0x7fe6d0069a50 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6ceffd700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d0069a50 0x7fe6d01697f0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000c590 tx=0x7fe6c000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6b6ffd700 1 -- 172.21.15.119:0/1256925495 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0011590 con 0x7fe6d0069a50 2023-10-08T02:11:16.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6b6ffd700 1 -- 172.21.15.119:0/1256925495 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c00116f0 con 0x7fe6d0069a50 2023-10-08T02:11:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d00c0400 con 0x7fe6d0069a50 2023-10-08T02:11:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.688+0000 7fe6b6ffd700 1 -- 172.21.15.119:0/1256925495 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0011ab0 con 0x7fe6d0069a50 2023-10-08T02:11:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.689+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d00c09c0 con 0x7fe6d0069a50 2023-10-08T02:11:16.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.689+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d00718b0 con 0x7fe6d0069a50 2023-10-08T02:11:16.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.690+0000 7fe6b6ffd700 1 -- 172.21.15.119:0/1256925495 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 74) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fe6c001b3b0 con 0x7fe6d0069a50 2023-10-08T02:11:16.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.691+0000 7fe6b6ffd700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe6bc0670a0 0x7fe6bc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:16.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.691+0000 7fe6b6ffd700 1 -- 172.21.15.119:0/1256925495 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6485+0+0 (secure 0 0 0) 0x7fe6c0084070 con 0x7fe6d0069a50 2023-10-08T02:11:16.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.691+0000 7fe6ce7fc700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe6bc0670a0 0x7fe6bc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:16.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.692+0000 7fe6ce7fc700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe6bc0670a0 0x7fe6bc069550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00b23c0 tx=0x7fe6c4009040 comp rx=0 tx=0).ready entity=mgr.8376 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:16.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.695+0000 7fe6b6ffd700 1 -- 172.21.15.119:0/1256925495 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6c0050e20 con 0x7fe6d0069a50 2023-10-08T02:11:16.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.908+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6d0068e00 con 0x7fe6bc0670a0 2023-10-08T02:11:16.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.918+0000 7fe6b6ffd700 1 -- 172.21.15.119:0/1256925495 <== mgr.8376 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fe6d0068e00 con 0x7fe6bc0670a0 2023-10-08T02:11:16.918 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:16.919 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T01:15:43", "first": "2023-10-08T01:16:00", "last": "2023-10-08T02:11:00", "last_pruned": "2023-10-08T02:11:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-08T02:11:16.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe6bc0670a0 msgr2=0x7fe6bc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:16.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe6bc0670a0 0x7fe6bc069550 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00b23c0 tx=0x7fe6c4009040 comp rx=0 tx=0).stop 2023-10-08T02:11:16.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d0069a50 msgr2=0x7fe6d01697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d0069a50 0x7fe6d01697f0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe6c000c590 tx=0x7fe6c000c950 comp rx=0 tx=0).stop 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 shutdown_connections 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe6bc0670a0 0x7fe6bc069550 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6d01570c0 0x7fe6d01547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d0069a50 0x7fe6d01697f0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 --2- 172.21.15.119:0/1256925495 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe6d00c8490 0x7fe6d00cb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 >> 172.21.15.119:0/1256925495 conn(0x7fe6d00c8f90 msgr2=0x7fe6d0159620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.920+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 shutdown_connections 2023-10-08T02:11:16.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:16.921+0000 7fe6d53ff700 1 -- 172.21.15.119:0/1256925495 wait complete. 2023-10-08T02:11:16.938 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T01:15:43", "first": "2023-10-08T01:16:00", "last": "2023-10-08T02:11:00", "last_pruned": "2023-10-08T02:11:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-10-08T02:11:16.941 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T01:15:43', 'first': '2023-10-08T01:16:00', 'last': '2023-10-08T02:11:00', 'last_pruned': '2023-10-08T02:11:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-10-08T02:11:16.941 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-10-08T02:11:16.941 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-10-08T02:11:16.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1 2023-10-08T02:11:17.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.126+0000 7f181f3ff700 1 Processor -- start 2023-10-08T02:11:17.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.126+0000 7f181f3ff700 1 -- start start 2023-10-08T02:11:17.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.127+0000 7f181f3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18181563b0 0x7f1818156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:17.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.127+0000 7f181f3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f181815a440 0x7f18180acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:17.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.127+0000 7f181f3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181815a9b0 con 0x7f1818068890 2023-10-08T02:11:17.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.127+0000 7f181f3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f181815ac90 con 0x7f181815a440 2023-10-08T02:11:17.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.127+0000 7f181f3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181815af70 con 0x7f18181563b0 2023-10-08T02:11:17.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f181c99a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f181815a440 0x7f18180acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:17.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f181d19b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18181563b0 0x7f1818156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:17.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f181c99a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f181815a440 0x7f18180acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:59292/0 (socket says 172.21.15.119:59292) 2023-10-08T02:11:17.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f181c99a700 1 -- 172.21.15.119:0/3462447964 learned_addr learned my addr 172.21.15.119:0/3462447964 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:17.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f181c99a700 1 -- 172.21.15.119:0/3462447964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18181563b0 msgr2=0x7f1818156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f181c99a700 1 --2- 172.21.15.119:0/3462447964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18181563b0 0x7f1818156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f181c99a700 1 -- 172.21.15.119:0/3462447964 >> v1:172.21.15.184:6789/0 conn(0x7f1818068890 legacy=0x7f18180c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f181c99a700 1 -- 172.21.15.119:0/3462447964 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181815b250 con 0x7f181815a440 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.128+0000 7f180ffff700 1 -- 172.21.15.119:0/3462447964 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026986890 0 0) 0x7f181815a9b0 con 0x7f1818068890 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.129+0000 7f181c99a700 1 --2- 172.21.15.119:0/3462447964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f181815a440 0x7f18180acf30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f18080099b0 tx=0x7f1808009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e23a22cb2521ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.129+0000 7f180ffff700 1 -- 172.21.15.119:0/3462447964 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808009ea0 con 0x7f181815a440 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.129+0000 7f180ffff700 1 -- 172.21.15.119:0/3462447964 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180800b860 con 0x7f181815a440 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.129+0000 7f180ffff700 1 -- 172.21.15.119:0/3462447964 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180800bc00 con 0x7f181815a440 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.129+0000 7f181f3ff700 1 -- 172.21.15.119:0/3462447964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f181815a440 msgr2=0x7f18180acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:17.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.129+0000 7f181f3ff700 1 --2- 172.21.15.119:0/3462447964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f181815a440 0x7f18180acf30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f18080099b0 tx=0x7f1808009250 comp rx=0 tx=0).stop 2023-10-08T02:11:17.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.130+0000 7f181f3ff700 1 -- 172.21.15.119:0/3462447964 shutdown_connections 2023-10-08T02:11:17.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.130+0000 7f181f3ff700 1 --2- 172.21.15.119:0/3462447964 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f18181563b0 0x7f1818156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.130+0000 7f181f3ff700 1 --2- 172.21.15.119:0/3462447964 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f181815a440 0x7f18180acf30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.130+0000 7f181f3ff700 1 -- 172.21.15.119:0/3462447964 >> 172.21.15.119:0/3462447964 conn(0x7f18180c8f90 msgr2=0x7f18180c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:17.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.130+0000 7f181f3ff700 1 -- 172.21.15.119:0/3462447964 shutdown_connections 2023-10-08T02:11:17.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.130+0000 7f181f3ff700 1 -- 172.21.15.119:0/3462447964 wait complete. 2023-10-08T02:11:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.131+0000 7f181f3ff700 1 Processor -- start 2023-10-08T02:11:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.131+0000 7f181f3ff700 1 -- start start 2023-10-08T02:11:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181f3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1818068890 0x7f18180aa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181f3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18181563b0 0x7f1818146960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:17.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181f3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f181815a440 0x7f18180b1450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181f3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1818146ea0 con 0x7f18181563b0 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181f3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18180a9980 con 0x7f1818068890 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181f3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18180b1ec0 con 0x7f181815a440 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18181563b0 0x7f1818146960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18181563b0 0x7f1818146960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36928/0 (socket says 172.21.15.119:36928) 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181c99a700 1 -- 172.21.15.119:0/236434224 learned_addr learned my addr 172.21.15.119:0/236434224 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181d19b700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1818068890 0x7f18180aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.132+0000 7f181d99c700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f181815a440 0x7f18180b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:17.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f181c99a700 1 -- 172.21.15.119:0/236434224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f181815a440 msgr2=0x7f18180b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f181c99a700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f181815a440 0x7f18180b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f181c99a700 1 -- 172.21.15.119:0/236434224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1818068890 msgr2=0x7f18180aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f181c99a700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1818068890 0x7f18180aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f181c99a700 1 -- 172.21.15.119:0/236434224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18080094f0 con 0x7f18181563b0 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f181c99a700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18181563b0 0x7f1818146960 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f181806bbb0 tx=0x7f1808017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f180d7fa700 1 -- 172.21.15.119:0/236434224 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808000d10 con 0x7f18181563b0 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f180d7fa700 1 -- 172.21.15.119:0/236434224 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180800bbe0 con 0x7f18181563b0 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18180a9c20 con 0x7f18181563b0 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f180d7fa700 1 -- 172.21.15.119:0/236434224 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1808011690 con 0x7f18181563b0 2023-10-08T02:11:17.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.133+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18180b21a0 con 0x7f18181563b0 2023-10-08T02:11:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.134+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18180aa140 con 0x7f18181563b0 2023-10-08T02:11:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.135+0000 7f180d7fa700 1 -- 172.21.15.119:0/236434224 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 74) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f1808011ac0 con 0x7f18181563b0 2023-10-08T02:11:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.136+0000 7f180d7fa700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17fc0670a0 0x7f17fc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:17.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.137+0000 7f180d7fa700 1 -- 172.21.15.119:0/236434224 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6485+0+0 (secure 0 0 0) 0x7f1808086270 con 0x7f18181563b0 2023-10-08T02:11:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.137+0000 7f181d19b700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17fc0670a0 0x7f17fc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:17.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.137+0000 7f181d19b700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17fc0670a0 0x7f17fc069550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1814009db0 tx=0x7f18140091a0 comp rx=0 tx=0).ready entity=mgr.8376 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:17.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.141+0000 7f180d7fa700 1 -- 172.21.15.119:0/236434224 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f18180aa140 con 0x7f18181563b0 2023-10-08T02:11:17.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.359+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f18180aa140 con 0x7f17fc0670a0 2023-10-08T02:11:17.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.439+0000 7f180d7fa700 1 -- 172.21.15.119:0/236434224 <== mgr.8376 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f18180aa140 con 0x7f17fc0670a0 2023-10-08T02:11:17.440 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1 2023-10-08T02:11:17.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17fc0670a0 msgr2=0x7f17fc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:17.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17fc0670a0 0x7f17fc069550 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1814009db0 tx=0x7f18140091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18181563b0 msgr2=0x7f1818146960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:17.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18181563b0 0x7f1818146960 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f181806bbb0 tx=0x7f1808017000 comp rx=0 tx=0).stop 2023-10-08T02:11:17.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 shutdown_connections 2023-10-08T02:11:17.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f17fc0670a0 0x7f17fc069550 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f181815a440 0x7f18180b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1818068890 0x7f18180aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 --2- 172.21.15.119:0/236434224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18181563b0 0x7f1818146960 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 >> 172.21.15.119:0/236434224 conn(0x7f18180c8f90 msgr2=0x7f1818068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 shutdown_connections 2023-10-08T02:11:17.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:17.442+0000 7f181f3ff700 1 -- 172.21.15.119:0/236434224 wait complete. 2023-10-08T02:11:17.460 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-10-08T02:11:17.460 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:17.460 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:11:17.500 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_17_00_UTC 2023-10-08T02:11:17.500 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_18_00_UTC 2023-10-08T02:11:17.500 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_19_00_UTC 2023-10-08T02:11:17.500 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_20_00_UTC 2023-10-08T02:11:17.500 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_21_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_22_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_23_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_24_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_25_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_26_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_27_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_28_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_29_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_30_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_31_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_32_00_UTC 2023-10-08T02:11:17.501 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_33_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_34_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_35_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_36_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_37_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_38_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_39_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_40_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_41_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_42_00_UTC 2023-10-08T02:11:17.502 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_43_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_44_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_45_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_46_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_47_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_48_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_49_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_50_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_51_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_52_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_53_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_54_00_UTC 2023-10-08T02:11:17.503 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_55_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_56_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_57_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_58_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-01_59_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_00_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_01_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_02_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_03_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_04_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_05_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_06_00_UTC 2023-10-08T02:11:17.504 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_07_00_UTC 2023-10-08T02:11:17.505 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_08_00_UTC 2023-10-08T02:11:17.505 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_09_00_UTC 2023-10-08T02:11:17.505 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_10_00_UTC 2023-10-08T02:11:17.505 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_11_00_UTC 2023-10-08T02:11:17.506 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_17_00_UTC 2023-10-08T02:11:17.506 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:17.506 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_17_00_UTC) 2023-10-08T02:11:17.589 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_18_00_UTC 2023-10-08T02:11:17.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:17.590 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_18_00_UTC) 2023-10-08T02:11:17.673 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_19_00_UTC 2023-10-08T02:11:17.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:17.673 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_19_00_UTC) 2023-10-08T02:11:17.757 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_20_00_UTC 2023-10-08T02:11:17.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:17.757 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_20_00_UTC) 2023-10-08T02:11:17.840 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_21_00_UTC 2023-10-08T02:11:17.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:17.841 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_21_00_UTC) 2023-10-08T02:11:17.925 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_22_00_UTC 2023-10-08T02:11:17.925 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:17.925 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_22_00_UTC) 2023-10-08T02:11:18.010 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_23_00_UTC 2023-10-08T02:11:18.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.010 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_23_00_UTC) 2023-10-08T02:11:18.094 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_24_00_UTC 2023-10-08T02:11:18.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.095 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_24_00_UTC) 2023-10-08T02:11:18.178 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_25_00_UTC 2023-10-08T02:11:18.178 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.179 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_25_00_UTC) 2023-10-08T02:11:18.262 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_26_00_UTC 2023-10-08T02:11:18.262 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.262 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_26_00_UTC) 2023-10-08T02:11:18.345 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_27_00_UTC 2023-10-08T02:11:18.345 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.346 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_27_00_UTC) 2023-10-08T02:11:18.429 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_28_00_UTC 2023-10-08T02:11:18.429 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.429 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_28_00_UTC) 2023-10-08T02:11:18.513 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_29_00_UTC 2023-10-08T02:11:18.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.514 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_29_00_UTC) 2023-10-08T02:11:18.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_30_00_UTC 2023-10-08T02:11:18.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.598 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_30_00_UTC) 2023-10-08T02:11:18.681 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_31_00_UTC 2023-10-08T02:11:18.681 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.681 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_31_00_UTC) 2023-10-08T02:11:18.764 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_32_00_UTC 2023-10-08T02:11:18.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.765 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_32_00_UTC) 2023-10-08T02:11:18.848 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_33_00_UTC 2023-10-08T02:11:18.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.848 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_33_00_UTC) 2023-10-08T02:11:18.933 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_34_00_UTC 2023-10-08T02:11:18.933 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:18.933 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_34_00_UTC) 2023-10-08T02:11:19.017 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_35_00_UTC 2023-10-08T02:11:19.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.017 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_35_00_UTC) 2023-10-08T02:11:19.101 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_36_00_UTC 2023-10-08T02:11:19.101 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.102 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_36_00_UTC) 2023-10-08T02:11:19.184 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_37_00_UTC 2023-10-08T02:11:19.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.185 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_37_00_UTC) 2023-10-08T02:11:19.267 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_38_00_UTC 2023-10-08T02:11:19.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.268 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_38_00_UTC) 2023-10-08T02:11:19.351 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_39_00_UTC 2023-10-08T02:11:19.351 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.351 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_39_00_UTC) 2023-10-08T02:11:19.434 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_40_00_UTC 2023-10-08T02:11:19.435 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.435 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_40_00_UTC) 2023-10-08T02:11:19.518 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_41_00_UTC 2023-10-08T02:11:19.518 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.518 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_41_00_UTC) 2023-10-08T02:11:19.602 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_42_00_UTC 2023-10-08T02:11:19.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.602 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_42_00_UTC) 2023-10-08T02:11:19.686 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_43_00_UTC 2023-10-08T02:11:19.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.686 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_43_00_UTC) 2023-10-08T02:11:19.769 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_44_00_UTC 2023-10-08T02:11:19.769 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.769 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_44_00_UTC) 2023-10-08T02:11:19.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_45_00_UTC 2023-10-08T02:11:19.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.853 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_45_00_UTC) 2023-10-08T02:11:19.935 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_46_00_UTC 2023-10-08T02:11:19.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:19.935 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_46_00_UTC) 2023-10-08T02:11:20.019 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_47_00_UTC 2023-10-08T02:11:20.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.019 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_47_00_UTC) 2023-10-08T02:11:20.104 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_48_00_UTC 2023-10-08T02:11:20.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.104 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_48_00_UTC) 2023-10-08T02:11:20.188 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_49_00_UTC 2023-10-08T02:11:20.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.188 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_49_00_UTC) 2023-10-08T02:11:20.273 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_50_00_UTC 2023-10-08T02:11:20.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.273 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_50_00_UTC) 2023-10-08T02:11:20.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_51_00_UTC 2023-10-08T02:11:20.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.357 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_51_00_UTC) 2023-10-08T02:11:20.440 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_52_00_UTC 2023-10-08T02:11:20.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.440 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_52_00_UTC) 2023-10-08T02:11:20.524 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_53_00_UTC 2023-10-08T02:11:20.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.524 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_53_00_UTC) 2023-10-08T02:11:20.608 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_54_00_UTC 2023-10-08T02:11:20.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.608 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_54_00_UTC) 2023-10-08T02:11:20.692 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_55_00_UTC 2023-10-08T02:11:20.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.692 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_55_00_UTC) 2023-10-08T02:11:20.776 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_56_00_UTC 2023-10-08T02:11:20.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.776 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_56_00_UTC) 2023-10-08T02:11:20.860 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_57_00_UTC 2023-10-08T02:11:20.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.860 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_57_00_UTC) 2023-10-08T02:11:20.944 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_58_00_UTC 2023-10-08T02:11:20.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:20.945 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_58_00_UTC) 2023-10-08T02:11:21.027 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-01_59_00_UTC 2023-10-08T02:11:21.028 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.029 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-01_59_00_UTC) 2023-10-08T02:11:21.111 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_00_00_UTC 2023-10-08T02:11:21.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.111 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_00_00_UTC) 2023-10-08T02:11:21.244 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_01_00_UTC 2023-10-08T02:11:21.244 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.244 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_01_00_UTC) 2023-10-08T02:11:21.307 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_02_00_UTC 2023-10-08T02:11:21.307 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.307 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_02_00_UTC) 2023-10-08T02:11:21.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_03_00_UTC 2023-10-08T02:11:21.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.357 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_03_00_UTC) 2023-10-08T02:11:21.399 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_04_00_UTC 2023-10-08T02:11:21.399 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.399 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_04_00_UTC) 2023-10-08T02:11:21.511 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_05_00_UTC 2023-10-08T02:11:21.511 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.511 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_05_00_UTC) 2023-10-08T02:11:21.587 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_06_00_UTC 2023-10-08T02:11:21.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.588 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_06_00_UTC) 2023-10-08T02:11:21.652 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_07_00_UTC 2023-10-08T02:11:21.652 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.652 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_07_00_UTC) 2023-10-08T02:11:21.713 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_08_00_UTC 2023-10-08T02:11:21.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.714 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_08_00_UTC) 2023-10-08T02:11:21.764 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_09_00_UTC 2023-10-08T02:11:21.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.765 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_09_00_UTC) 2023-10-08T02:11:21.813 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_10_00_UTC 2023-10-08T02:11:21.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.814 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_10_00_UTC) 2023-10-08T02:11:21.865 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_11_00_UTC 2023-10-08T02:11:21.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.865 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_11_00_UTC) 2023-10-08T02:11:21.911 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:21.911 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T02:11:21.992 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T02:11:22.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.215+0000 7fcd46dff700 1 Processor -- start 2023-10-08T02:11:22.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.216+0000 7fcd46dff700 1 -- start start 2023-10-08T02:11:22.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd46dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd40155eb0 0x7fcd4006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd46dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40069e80 0x7fcd400ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd46dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4006eb90 con 0x7fcd400c70b0 2023-10-08T02:11:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd46dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd4006ee70 con 0x7fcd40155eb0 2023-10-08T02:11:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd46dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4006f150 con 0x7fcd40069e80 2023-10-08T02:11:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd44b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd40155eb0 0x7fcd4006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd44b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd40155eb0 0x7fcd4006e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36932/0 (socket says 172.21.15.119:36932) 2023-10-08T02:11:22.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd44b9b700 1 -- 172.21.15.119:0/2878512894 learned_addr learned my addr 172.21.15.119:0/2878512894 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.217+0000 7fcd4539c700 1 --2- 172.21.15.119:0/2878512894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40069e80 0x7fcd400ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.218+0000 7fcd44b9b700 1 -- 172.21.15.119:0/2878512894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40069e80 msgr2=0x7fcd400ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.218+0000 7fcd44b9b700 1 --2- 172.21.15.119:0/2878512894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40069e80 0x7fcd400ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.218+0000 7fcd44b9b700 1 -- 172.21.15.119:0/2878512894 >> v1:172.21.15.184:6789/0 conn(0x7fcd400c70b0 legacy=0x7fcd40069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.218+0000 7fcd44b9b700 1 -- 172.21.15.119:0/2878512894 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd4006f430 con 0x7fcd40155eb0 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.218+0000 7fcd44b9b700 1 --2- 172.21.15.119:0/2878512894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40155eb0 0x7fcd4006e240 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcd340099b0 tx=0x7fcd34009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfba47157e3c02b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.218+0000 7fcd3f7fe700 1 -- 172.21.15.119:0/2878512894 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34009ea0 con 0x7fcd40155eb0 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.218+0000 7fcd3f7fe700 1 -- 172.21.15.119:0/2878512894 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3400b860 con 0x7fcd40155eb0 2023-10-08T02:11:22.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.218+0000 7fcd3f7fe700 1 -- 172.21.15.119:0/2878512894 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400bc00 con 0x7fcd40155eb0 2023-10-08T02:11:22.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.219+0000 7fcd46dff700 1 -- 172.21.15.119:0/2878512894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40155eb0 msgr2=0x7fcd4006e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:22.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.219+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2878512894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40155eb0 0x7fcd4006e240 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcd340099b0 tx=0x7fcd34009250 comp rx=0 tx=0).stop 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.219+0000 7fcd46dff700 1 -- 172.21.15.119:0/2878512894 shutdown_connections 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.219+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2878512894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40069e80 0x7fcd400ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.219+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2878512894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40155eb0 0x7fcd4006e240 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.219+0000 7fcd46dff700 1 -- 172.21.15.119:0/2878512894 >> 172.21.15.119:0/2878512894 conn(0x7fcd40068e10 msgr2=0x7fcd400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.219+0000 7fcd46dff700 1 -- 172.21.15.119:0/2878512894 shutdown_connections 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.219+0000 7fcd46dff700 1 -- 172.21.15.119:0/2878512894 wait complete. 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.220+0000 7fcd46dff700 1 Processor -- start 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.220+0000 7fcd46dff700 1 -- start start 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.221+0000 7fcd46dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40069e80 0x7fcd400aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.221+0000 7fcd46dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd400c70b0 0x7fcd400abc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:22.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.221+0000 7fcd46dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40155eb0 0x7fcd400acae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.221+0000 7fcd46dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400c18e0 con 0x7fcd40069e80 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.221+0000 7fcd46dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400ac140 con 0x7fcd400c70b0 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.221+0000 7fcd46dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400aa8a0 con 0x7fcd40155eb0 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.221+0000 7fcd44b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40069e80 0x7fcd400aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd44b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40069e80 0x7fcd400aa290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36946/0 (socket says 172.21.15.119:36946) 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd44b9b700 1 -- 172.21.15.119:0/2485442396 learned_addr learned my addr 172.21.15.119:0/2485442396 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd44b9b700 1 -- 172.21.15.119:0/2485442396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40155eb0 msgr2=0x7fcd400acae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd44b9b700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40155eb0 0x7fcd400acae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd44b9b700 1 -- 172.21.15.119:0/2485442396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd400c70b0 msgr2=0x7fcd400abc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd44b9b700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd400c70b0 0x7fcd400abc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:22.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd44b9b700 1 -- 172.21.15.119:0/2485442396 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd340094f0 con 0x7fcd40069e80 2023-10-08T02:11:22.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd44b9b700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40069e80 0x7fcd400aa290 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcd4006fe40 tx=0x7fcd34017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd3cff9700 1 -- 172.21.15.119:0/2485442396 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3400bbe0 con 0x7fcd40069e80 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd3cff9700 1 -- 172.21.15.119:0/2485442396 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd34011420 con 0x7fcd40069e80 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd400aab80 con 0x7fcd40069e80 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.222+0000 7fcd3cff9700 1 -- 172.21.15.119:0/2485442396 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340117c0 con 0x7fcd40069e80 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.223+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd400ab140 con 0x7fcd40069e80 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.223+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd4004c3c0 con 0x7fcd40069e80 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.224+0000 7fcd3cff9700 1 -- 172.21.15.119:0/2485442396 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 74) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fcd34011bf0 con 0x7fcd40069e80 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.225+0000 7fcd3cff9700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcd240670a0 0x7fcd24069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:22.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.225+0000 7fcd3cff9700 1 -- 172.21.15.119:0/2485442396 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fcd34086420 con 0x7fcd40069e80 2023-10-08T02:11:22.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.225+0000 7fcd3ffff700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcd240670a0 0x7fcd24069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:22.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.226+0000 7fcd3ffff700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcd240670a0 0x7fcd24069550 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd400aa7d0 tx=0x7fcd2c007b40 comp rx=0 tx=0).ready entity=mgr.8376 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:22.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.230+0000 7fcd3cff9700 1 -- 172.21.15.119:0/2485442396 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcd340531a0 con 0x7fcd40069e80 2023-10-08T02:11:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:22.480+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fcd40156280 con 0x7fcd40069e80 2023-10-08T02:11:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.163+0000 7fcd3cff9700 1 -- 172.21.15.119:0/2485442396 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fcd34053570 con 0x7fcd40069e80 2023-10-08T02:11:23.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.163+0000 7fcd3cff9700 1 -- 172.21.15.119:0/2485442396 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v75) v1 ==== 93+0+0 (secure 0 0 0) 0x7fcd34056c30 con 0x7fcd40069e80 2023-10-08T02:11:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.168+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcd240670a0 msgr2=0x7fcd24069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.168+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcd240670a0 0x7fcd24069550 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcd400aa7d0 tx=0x7fcd2c007b40 comp rx=0 tx=0).stop 2023-10-08T02:11:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.168+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40069e80 msgr2=0x7fcd400aa290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:23.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.168+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40069e80 0x7fcd400aa290 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcd4006fe40 tx=0x7fcd34017000 comp rx=0 tx=0).stop 2023-10-08T02:11:23.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.169+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 shutdown_connections 2023-10-08T02:11:23.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.169+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcd240670a0 0x7fcd24069550 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.169+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcd40155eb0 0x7fcd400acae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.169+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd40069e80 0x7fcd400aa290 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.169+0000 7fcd46dff700 1 --2- 172.21.15.119:0/2485442396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcd400c70b0 0x7fcd400abc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.169+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 >> 172.21.15.119:0/2485442396 conn(0x7fcd40068e10 msgr2=0x7fcd40154b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:23.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.169+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 shutdown_connections 2023-10-08T02:11:23.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.169+0000 7fcd46dff700 1 -- 172.21.15.119:0/2485442396 wait complete. 2023-10-08T02:11:23.194 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:11:23.194 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:11:23.276 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:11:23.364 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:11:23.365 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T02:11:23.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:23.365 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T02:11:23.469 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:11:23.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:11:23.470 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T02:11:23.529 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:11:23.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.727+0000 7fd53c9fe700 1 Processor -- start 2023-10-08T02:11:23.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.727+0000 7fd53c9fe700 1 -- start start 2023-10-08T02:11:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd53c9fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd53c9fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5380ae710 0x7fd5380c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd53c9fe700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5380ca860 con 0x7fd5381566e0 2023-10-08T02:11:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd53c9fe700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5380cab40 con 0x7fd5380ae710 2023-10-08T02:11:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd53c9fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380cae20 con 0x7fd538156310 2023-10-08T02:11:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd53659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd5381566e0 0x7fd5380ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35688/0 (socket says 172.21.15.119:35688) 2023-10-08T02:11:23.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd53659c700 1 -- 172.21.15.119:0/1329479359 learned_addr learned my addr 172.21.15.119:0/1329479359 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd536d9d700 1 --2- 172.21.15.119:0/1329479359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd53759e700 1 --2- 172.21.15.119:0/1329479359 >> v2:172.21.15.184:3300/0 conn(0x7fd5380ae710 0x7fd5380c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd535d9b700 1 -- 172.21.15.119:0/1329479359 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2729836584 0 0) 0x7fd5380ca860 con 0x7fd5381566e0 2023-10-08T02:11:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.729+0000 7fd535d9b700 1 -- 172.21.15.119:0/1329479359 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd520003570 con 0x7fd5381566e0 2023-10-08T02:11:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.730+0000 7fd536d9d700 1 -- 172.21.15.119:0/1329479359 >> v2:172.21.15.184:3300/0 conn(0x7fd5380ae710 msgr2=0x7fd5380c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.730+0000 7fd536d9d700 1 --2- 172.21.15.119:0/1329479359 >> v2:172.21.15.184:3300/0 conn(0x7fd5380ae710 0x7fd5380c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.730+0000 7fd536d9d700 1 -- 172.21.15.119:0/1329479359 >> v1:172.21.15.119:6789/0 conn(0x7fd5381566e0 legacy=0x7fd5380ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:23.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.730+0000 7fd536d9d700 1 -- 172.21.15.119:0/1329479359 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380cb100 con 0x7fd538156310 2023-10-08T02:11:23.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.730+0000 7fd536d9d700 1 --2- 172.21.15.119:0/1329479359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c9dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5280098e0 tx=0x7fd5280091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88d352543039202c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:23.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.730+0000 7fd535d9b700 1 -- 172.21.15.119:0/1329479359 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528011630 con 0x7fd538156310 2023-10-08T02:11:23.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.730+0000 7fd535d9b700 1 -- 172.21.15.119:0/1329479359 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd528011790 con 0x7fd538156310 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.730+0000 7fd535d9b700 1 -- 172.21.15.119:0/1329479359 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528011b30 con 0x7fd538156310 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.731+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1329479359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 msgr2=0x7fd5380c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.731+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1329479359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c9dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd5280098e0 tx=0x7fd5280091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.731+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1329479359 shutdown_connections 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.731+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1329479359 >> v2:172.21.15.184:3300/0 conn(0x7fd5380ae710 0x7fd5380c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.731+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1329479359 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c9dc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.731+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1329479359 >> 172.21.15.119:0/1329479359 conn(0x7fd5380cc1f0 msgr2=0x7fd5380cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.731+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1329479359 shutdown_connections 2023-10-08T02:11:23.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.731+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1329479359 wait complete. 2023-10-08T02:11:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.732+0000 7fd53c9fe700 1 Processor -- start 2023-10-08T02:11:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.733+0000 7fd53c9fe700 1 -- start start 2023-10-08T02:11:23.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.733+0000 7fd53c9fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5380ae710 0x7fd538148b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd53c9fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd53c9fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5381566e0 0x7fd5380c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd53c9fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c1f30 con 0x7fd5380ae710 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd53c9fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380a5960 con 0x7fd5381566e0 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd53c9fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5380c1970 con 0x7fd538156310 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd536d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5380ae710 0x7fd538148b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd536d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5380ae710 0x7fd538148b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36962/0 (socket says 172.21.15.119:36962) 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd536d9d700 1 -- 172.21.15.119:0/1879497693 learned_addr learned my addr 172.21.15.119:0/1879497693 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd53659c700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd53759e700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5381566e0 0x7fd5380c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd536d9d700 1 -- 172.21.15.119:0/1879497693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 msgr2=0x7fd5380c0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:23.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd536d9d700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd536d9d700 1 -- 172.21.15.119:0/1879497693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5381566e0 msgr2=0x7fd5380c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd536d9d700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5381566e0 0x7fd5380c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.734+0000 7fd536d9d700 1 -- 172.21.15.119:0/1879497693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd528009420 con 0x7fd5380ae710 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.735+0000 7fd536d9d700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5380ae710 0x7fd538148b40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd5380caf60 tx=0x7fd528017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.735+0000 7fd51effd700 1 -- 172.21.15.119:0/1879497693 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528011900 con 0x7fd5380ae710 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.735+0000 7fd51effd700 1 -- 172.21.15.119:0/1879497693 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd528011a60 con 0x7fd5380ae710 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.735+0000 7fd51effd700 1 -- 172.21.15.119:0/1879497693 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd528023620 con 0x7fd5380ae710 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.735+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5380c1c50 con 0x7fd5380ae710 2023-10-08T02:11:23.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.735+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5380c26e0 con 0x7fd5380ae710 2023-10-08T02:11:23.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.736+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53804c3c0 con 0x7fd5380ae710 2023-10-08T02:11:23.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.738+0000 7fd51effd700 1 -- 172.21.15.119:0/1879497693 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fd52802c3b0 con 0x7fd5380ae710 2023-10-08T02:11:23.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.739+0000 7fd51effd700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd524067050 0x7fd524069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:23.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.739+0000 7fd51effd700 1 -- 172.21.15.119:0/1879497693 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fd528085e10 con 0x7fd5380ae710 2023-10-08T02:11:23.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.739+0000 7fd53659c700 1 -- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd524067050 msgr2=0x7fd524069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:23.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.739+0000 7fd53659c700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd524067050 0x7fd524069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:23.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.743+0000 7fd51effd700 1 -- 172.21.15.119:0/1879497693 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd528089030 con 0x7fd5380ae710 2023-10-08T02:11:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.940+0000 7fd53659c700 1 -- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd524067050 msgr2=0x7fd524069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:23.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.940+0000 7fd53659c700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd524067050 0x7fd524069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:23.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.984+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5380cd0b0 con 0x7fd5380ae710 2023-10-08T02:11:23.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.984+0000 7fd51effd700 1 -- 172.21.15.119:0/1879497693 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fd528015090 con 0x7fd5380ae710 2023-10-08T02:11:23.985 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:23.985 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7549,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/26538394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":26538394},{"type":"v1","addr":"172.21.15.184:6834","nonce":26538394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3494509212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3494509212},{"type":"v1","addr":"172.21.15.119:6834","nonce":3494509212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2792091862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2792091862},{"type":"v1","addr":"172.21.15.119:6835","nonce":2792091862}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:15:21.218141+0000","modified":"2023-10-08T01:15:25.231364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1485819493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1485819493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1485819493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd524067050 msgr2=0x7fd524069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd524067050 0x7fd524069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5380ae710 msgr2=0x7fd538148b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5380ae710 0x7fd538148b40 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd5380caf60 tx=0x7fd528017000 comp rx=0 tx=0).stop 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 shutdown_connections 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd524067050 0x7fd524069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5381566e0 0x7fd5380c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5380ae710 0x7fd538148b40 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 --2- 172.21.15.119:0/1879497693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd538156310 0x7fd5380c0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:23.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 >> 172.21.15.119:0/1879497693 conn(0x7fd5380cc1f0 msgr2=0x7fd538155370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 shutdown_connections 2023-10-08T02:11:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:23.987+0000 7fd53c9fe700 1 -- 172.21.15.119:0/1879497693 wait complete. 2023-10-08T02:11:23.989 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 63 2023-10-08T02:11:24.006 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:11:24.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.194+0000 7eff9dd9d700 1 Processor -- start 2023-10-08T02:11:24.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.194+0000 7eff9dd9d700 1 -- start start 2023-10-08T02:11:24.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff9dd9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7eff981595a0 0x7eff980b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff9dd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff980c6c60 0x7eff980c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff9dd9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff980b1410 con 0x7eff980b1de0 2023-10-08T02:11:24.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff9dd9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff980b2b90 con 0x7eff981595a0 2023-10-08T02:11:24.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff9dd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff98063230 con 0x7eff980c6c60 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff9d59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7eff980b1de0 0x7eff9815d460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35698/0 (socket says 172.21.15.119:35698) 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff9d59c700 1 -- 172.21.15.119:0/614933552 learned_addr learned my addr 172.21.15.119:0/614933552 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff97fff700 1 --2- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff980c6c60 0x7eff980c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff9cd9b700 1 --2- 172.21.15.119:0/614933552 >> v2:172.21.15.184:3300/0 conn(0x7eff981595a0 0x7eff980b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff977fe700 1 -- 172.21.15.119:0/614933552 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700394531 0 0) 0x7eff980b1410 con 0x7eff980b1de0 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.196+0000 7eff977fe700 1 -- 172.21.15.119:0/614933552 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff88003570 con 0x7eff980b1de0 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff9cd9b700 1 -- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff980c6c60 msgr2=0x7eff980c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff9cd9b700 1 --2- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff980c6c60 0x7eff980c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff9cd9b700 1 -- 172.21.15.119:0/614933552 >> v1:172.21.15.119:6789/0 conn(0x7eff980b1de0 legacy=0x7eff9815d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff9cd9b700 1 -- 172.21.15.119:0/614933552 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980634f0 con 0x7eff981595a0 2023-10-08T02:11:24.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff97fff700 1 --2- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff980c6c60 0x7eff980c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:11:24.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff9cd9b700 1 --2- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff981595a0 0x7eff980b18a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0098e0 tx=0x7eff8c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a235a68257b95b27 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:24.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff977fe700 1 -- 172.21.15.119:0/614933552 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c011630 con 0x7eff981595a0 2023-10-08T02:11:24.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff977fe700 1 -- 172.21.15.119:0/614933552 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c011790 con 0x7eff981595a0 2023-10-08T02:11:24.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.197+0000 7eff977fe700 1 -- 172.21.15.119:0/614933552 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c011b30 con 0x7eff981595a0 2023-10-08T02:11:24.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.198+0000 7eff9dd9d700 1 -- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff981595a0 msgr2=0x7eff980b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.198+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff981595a0 0x7eff980b18a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0098e0 tx=0x7eff8c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.198+0000 7eff9dd9d700 1 -- 172.21.15.119:0/614933552 shutdown_connections 2023-10-08T02:11:24.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.198+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff981595a0 0x7eff980b18a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.198+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/614933552 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff980c6c60 0x7eff980c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.198+0000 7eff9dd9d700 1 -- 172.21.15.119:0/614933552 >> 172.21.15.119:0/614933552 conn(0x7eff9806ddb0 msgr2=0x7eff9806e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:24.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.198+0000 7eff9dd9d700 1 -- 172.21.15.119:0/614933552 shutdown_connections 2023-10-08T02:11:24.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.198+0000 7eff9dd9d700 1 -- 172.21.15.119:0/614933552 wait complete. 2023-10-08T02:11:24.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.199+0000 7eff9dd9d700 1 Processor -- start 2023-10-08T02:11:24.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.199+0000 7eff9dd9d700 1 -- start start 2023-10-08T02:11:24.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.200+0000 7eff9dd9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff980b1de0 0x7eff9815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.200+0000 7eff9dd9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff980c6c60 0x7eff9815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.200+0000 7eff9dd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff981595a0 0x7eff98152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.200+0000 7eff9dd9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9815b950 con 0x7eff980c6c60 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.200+0000 7eff9dd9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9815a980 con 0x7eff980b1de0 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.200+0000 7eff9dd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9814fc00 con 0x7eff981595a0 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff97fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff980c6c60 0x7eff9815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff97fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff980c6c60 0x7eff9815b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36976/0 (socket says 172.21.15.119:36976) 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff97fff700 1 -- 172.21.15.119:0/2429755336 learned_addr learned my addr 172.21.15.119:0/2429755336 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff9cd9b700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff980b1de0 0x7eff9815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff9d59c700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff981595a0 0x7eff98152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff97fff700 1 -- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff981595a0 msgr2=0x7eff98152ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff97fff700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff981595a0 0x7eff98152ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff97fff700 1 -- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff980b1de0 msgr2=0x7eff9815ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff97fff700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff980b1de0 0x7eff9815ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff97fff700 1 -- 172.21.15.119:0/2429755336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff84009420 con 0x7eff980c6c60 2023-10-08T02:11:24.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.201+0000 7eff9cd9b700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff980b1de0 0x7eff9815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:11:24.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.202+0000 7eff97fff700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff980c6c60 0x7eff9815b2b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7eff8400ecb0 tx=0x7eff840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:24.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.202+0000 7eff94ff9700 1 -- 172.21.15.119:0/2429755336 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400c920 con 0x7eff980c6c60 2023-10-08T02:11:24.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.202+0000 7eff94ff9700 1 -- 172.21.15.119:0/2429755336 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8400ca80 con 0x7eff980c6c60 2023-10-08T02:11:24.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.202+0000 7eff94ff9700 1 -- 172.21.15.119:0/2429755336 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8400ce00 con 0x7eff980c6c60 2023-10-08T02:11:24.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.202+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8c009420 con 0x7eff980c6c60 2023-10-08T02:11:24.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.202+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9814f310 con 0x7eff980c6c60 2023-10-08T02:11:24.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.203+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9804c3c0 con 0x7eff980c6c60 2023-10-08T02:11:24.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.205+0000 7eff94ff9700 1 -- 172.21.15.119:0/2429755336 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7eff84013680 con 0x7eff980c6c60 2023-10-08T02:11:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.206+0000 7eff94ff9700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff7c067050 0x7eff7c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.206+0000 7eff94ff9700 1 -- 172.21.15.119:0/2429755336 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6529+0+0 (secure 0 0 0) 0x7eff84084d90 con 0x7eff980c6c60 2023-10-08T02:11:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.206+0000 7eff9cd9b700 1 -- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff7c067050 msgr2=0x7eff7c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:24.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.206+0000 7eff9cd9b700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff7c067050 0x7eff7c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.211+0000 7eff94ff9700 1 -- 172.21.15.119:0/2429755336 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff84051b10 con 0x7eff980c6c60 2023-10-08T02:11:24.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.407+0000 7eff9cd9b700 1 -- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff7c067050 msgr2=0x7eff7c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:24.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.407+0000 7eff9cd9b700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff7c067050 0x7eff7c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:24.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.411+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff9814fd40 con 0x7eff980c6c60 2023-10-08T02:11:24.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.412+0000 7eff94ff9700 1 -- 172.21.15.119:0/2429755336 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16693 (secure 0 0 0) 0x7eff84055130 con 0x7eff980c6c60 2023-10-08T02:11:24.413 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:24.413 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":143,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:11:23.136852+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:15:19.321580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T01:15:20.200200+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":143,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":2}]},{"pgid":"17.3c","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1681557219":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/1844314331":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/3405191039":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/3624517726":"2023-10-09T01:15:42.377210+0000","172.21.15.119:6837/85511":"2023-10-09T01:15:42.377210+0000","172.21.15.119:6836/85511":"2023-10-09T01:15:42.377210+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":54},{"begin":59,"length":54}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":45,"length":12},{"begin":101,"length":12}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:11:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff7c067050 msgr2=0x7eff7c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff7c067050 0x7eff7c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff980c6c60 msgr2=0x7eff9815b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff980c6c60 0x7eff9815b2b0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7eff8400ecb0 tx=0x7eff840091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 shutdown_connections 2023-10-08T02:11:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7eff7c067050 0x7eff7c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7eff981595a0 0x7eff98152ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7eff980b1de0 0x7eff9815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 --2- 172.21.15.119:0/2429755336 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff980c6c60 0x7eff9815b2b0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 >> 172.21.15.119:0/2429755336 conn(0x7eff9806ddb0 msgr2=0x7eff9806e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 shutdown_connections 2023-10-08T02:11:24.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.414+0000 7eff9dd9d700 1 -- 172.21.15.119:0/2429755336 wait complete. 2023-10-08T02:11:24.434 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T01:15:19.321580+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T01:15:20.200200+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 143, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T02:11:24.435 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:11:24.510 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:24.509+0000 7f482f797180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:11:24.564 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:24.563+0000 7f356cb43180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:11:24.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.623+0000 7f5177fff700 1 Processor -- start 2023-10-08T02:11:24.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.623+0000 7f5177fff700 1 -- start start 2023-10-08T02:11:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.624+0000 7f5177fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f51780b4e70 0x7f51780ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.624+0000 7f5177fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51780b16d0 0x7f51780b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.624+0000 7f5177fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51780b2080 con 0x7f517815d240 2023-10-08T02:11:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.624+0000 7f5177fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51780cfb00 con 0x7f51780b16d0 2023-10-08T02:11:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.624+0000 7f5177fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51780b5c40 con 0x7f51780b4e70 2023-10-08T02:11:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f51767fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f51780b16d0 0x7f51780b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f51777fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f517815d240 0x7f517815d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35714/0 (socket says 172.21.15.119:35714) 2023-10-08T02:11:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f51777fe700 1 -- 172.21.15.119:0/3397443168 learned_addr learned my addr 172.21.15.119:0/3397443168 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f5176ffd700 1 --2- 172.21.15.119:0/3397443168 >> v2:172.21.15.184:3300/0 conn(0x7f51780b4e70 0x7f51780ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f51767fc700 1 -- 172.21.15.119:0/3397443168 >> v2:172.21.15.184:3300/0 conn(0x7f51780b4e70 msgr2=0x7f51780ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f51767fc700 1 --2- 172.21.15.119:0/3397443168 >> v2:172.21.15.184:3300/0 conn(0x7f51780b4e70 0x7f51780ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f51767fc700 1 -- 172.21.15.119:0/3397443168 >> v1:172.21.15.119:6789/0 conn(0x7f517815d240 legacy=0x7f517815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f51767fc700 1 -- 172.21.15.119:0/3397443168 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51780b5f20 con 0x7f51780b16d0 2023-10-08T02:11:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.625+0000 7f5175ffb700 1 -- 172.21.15.119:0/3397443168 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272619171 0 0) 0x7f51780b2080 con 0x7f517815d240 2023-10-08T02:11:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.626+0000 7f51767fc700 1 --2- 172.21.15.119:0/3397443168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b16d0 0x7f51780b1b40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f5178064910 tx=0x7f516c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9587ce31b7ff3b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:24.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.626+0000 7f5175ffb700 1 -- 172.21.15.119:0/3397443168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c011630 con 0x7f51780b16d0 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.626+0000 7f5175ffb700 1 -- 172.21.15.119:0/3397443168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c011790 con 0x7f51780b16d0 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.626+0000 7f5175ffb700 1 -- 172.21.15.119:0/3397443168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c011b30 con 0x7f51780b16d0 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.626+0000 7f5177fff700 1 -- 172.21.15.119:0/3397443168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b16d0 msgr2=0x7f51780b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.626+0000 7f5177fff700 1 --2- 172.21.15.119:0/3397443168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b16d0 0x7f51780b1b40 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f5178064910 tx=0x7f516c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.627+0000 7f5177fff700 1 -- 172.21.15.119:0/3397443168 shutdown_connections 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.627+0000 7f5177fff700 1 --2- 172.21.15.119:0/3397443168 >> v2:172.21.15.184:3300/0 conn(0x7f51780b4e70 0x7f51780ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.627+0000 7f5177fff700 1 --2- 172.21.15.119:0/3397443168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b16d0 0x7f51780b1b40 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.627+0000 7f5177fff700 1 -- 172.21.15.119:0/3397443168 >> 172.21.15.119:0/3397443168 conn(0x7f51780d10c0 msgr2=0x7f51780affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.627+0000 7f5177fff700 1 -- 172.21.15.119:0/3397443168 shutdown_connections 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.627+0000 7f5177fff700 1 -- 172.21.15.119:0/3397443168 wait complete. 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.628+0000 7f5177fff700 1 Processor -- start 2023-10-08T02:11:24.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.628+0000 7f5177fff700 1 -- start start 2023-10-08T02:11:24.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.629+0000 7f5177fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51780b16d0 0x7f51780c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f5177fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b4e70 0x7f5178169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f5177fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517815d240 0x7f517815e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f5177fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517816ac80 con 0x7f51780b4e70 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f5177fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517816d750 con 0x7f517815d240 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f5177fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f517816a6c0 con 0x7f51780b16d0 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b4e70 0x7f5178169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b4e70 0x7f5178169ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36982/0 (socket says 172.21.15.119:36982) 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 -- 172.21.15.119:0/706819408 learned_addr learned my addr 172.21.15.119:0/706819408 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51777fe700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517815d240 0x7f517815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f5176ffd700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51780b16d0 0x7f51780c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 -- 172.21.15.119:0/706819408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51780b16d0 msgr2=0x7f51780c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51780b16d0 0x7f51780c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 -- 172.21.15.119:0/706819408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517815d240 msgr2=0x7f517815e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517815d240 0x7f517815e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 -- 172.21.15.119:0/706819408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c009420 con 0x7f51780b4e70 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.630+0000 7f51767fc700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b4e70 0x7f5178169ca0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f5178070dd0 tx=0x7f516c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.631+0000 7f515effd700 1 -- 172.21.15.119:0/706819408 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c011b10 con 0x7f51780b4e70 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.631+0000 7f515effd700 1 -- 172.21.15.119:0/706819408 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f516c0253f0 con 0x7f51780b4e70 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.631+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517816a9a0 con 0x7f51780b4e70 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.631+0000 7f515effd700 1 -- 172.21.15.119:0/706819408 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516c025790 con 0x7f51780b4e70 2023-10-08T02:11:24.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.631+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51780c86f0 con 0x7f51780b4e70 2023-10-08T02:11:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.632+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517816d240 con 0x7f51780b4e70 2023-10-08T02:11:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.633+0000 7f515effd700 1 -- 172.21.15.119:0/706819408 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f516c025b80 con 0x7f51780b4e70 2023-10-08T02:11:24.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.634+0000 7f515effd700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5164067050 0x7f5164069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.634+0000 7f515effd700 1 -- 172.21.15.119:0/706819408 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f516c086370 con 0x7f51780b4e70 2023-10-08T02:11:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.634+0000 7f5176ffd700 1 -- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5164067050 msgr2=0x7f5164069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:24.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.634+0000 7f5176ffd700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5164067050 0x7f5164069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:24.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.638+0000 7f515effd700 1 -- 172.21.15.119:0/706819408 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f517816d240 con 0x7f51780b4e70 2023-10-08T02:11:24.673 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:24.672+0000 7f482f797180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:11:24.732 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:24.732+0000 7f356cb43180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:11:24.757 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:24.756+0000 7f482f797180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:11:24.817 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:24.817+0000 7f356cb43180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:11:24.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.833+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f517816d240 con 0x7f51780b4e70 2023-10-08T02:11:24.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.833+0000 7f515effd700 1 -- 172.21.15.119:0/706819408 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) v1 ==== 93+0+10 (secure 0 0 0) 0x7f517816d240 con 0x7f51780b4e70 2023-10-08T02:11:24.834 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:11:24.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.835+0000 7f5176ffd700 1 -- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5164067050 msgr2=0x7f5164069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:24.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.835+0000 7f5176ffd700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5164067050 0x7f5164069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5164067050 msgr2=0x7f5164069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5164067050 0x7f5164069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b4e70 msgr2=0x7f5178169ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b4e70 0x7f5178169ca0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f5178070dd0 tx=0x7f516c017000 comp rx=0 tx=0).stop 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 shutdown_connections 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5164067050 0x7f5164069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f51780b16d0 0x7f51780c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f517815d240 0x7f517815e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 --2- 172.21.15.119:0/706819408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51780b4e70 0x7f5178169ca0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 >> 172.21.15.119:0/706819408 conn(0x7f51780d10c0 msgr2=0x7f51780b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:24.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 shutdown_connections 2023-10-08T02:11:24.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:24.836+0000 7f5177fff700 1 -- 172.21.15.119:0/706819408 wait complete. 2023-10-08T02:11:24.855 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T02:11:24.984 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:24.983+0000 7f482f797180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:11:25.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.046+0000 7f7f3359e700 1 Processor -- start 2023-10-08T02:11:25.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.046+0000 7f7f3359e700 1 -- start start 2023-10-08T02:11:25.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.047+0000 7f7f3359e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7f340b4e70 0x7f7f340ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.047+0000 7f7f3359e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7f340b16d0 0x7f7f340b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.047+0000 7f7f3359e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f340b2080 con 0x7f7f3415d240 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.047+0000 7f7f3359e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f340cfb00 con 0x7f7f340b16d0 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.047+0000 7f7f3359e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f340b5c40 con 0x7f7f340b4e70 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.047+0000 7f7f31d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7f340b16d0 0x7f7f340b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.047+0000 7f7f32d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7f3415d240 0x7f7f3415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47350/0 (socket says 172.21.15.119:47350) 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.047+0000 7f7f32d9d700 1 -- 172.21.15.119:0/2480325609 learned_addr learned my addr 172.21.15.119:0/2480325609 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.048+0000 7f7f3259c700 1 --2- 172.21.15.119:0/2480325609 >> v2:172.21.15.184:3300/0 conn(0x7f7f340b4e70 0x7f7f340ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.048+0000 7f7f31d9b700 1 -- 172.21.15.119:0/2480325609 >> v2:172.21.15.184:3300/0 conn(0x7f7f340b4e70 msgr2=0x7f7f340ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.048+0000 7f7f31d9b700 1 --2- 172.21.15.119:0/2480325609 >> v2:172.21.15.184:3300/0 conn(0x7f7f340b4e70 0x7f7f340ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.048+0000 7f7f31d9b700 1 -- 172.21.15.119:0/2480325609 >> v1:172.21.15.119:6789/0 conn(0x7f7f3415d240 legacy=0x7f7f3415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.048+0000 7f7f31d9b700 1 -- 172.21.15.119:0/2480325609 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340b5f20 con 0x7f7f340b16d0 2023-10-08T02:11:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.048+0000 7f7f3159a700 1 -- 172.21.15.119:0/2480325609 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265394170 0 0) 0x7f7f340b2080 con 0x7f7f3415d240 2023-10-08T02:11:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.048+0000 7f7f31d9b700 1 --2- 172.21.15.119:0/2480325609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b16d0 0x7f7f340b1b40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7f34064910 tx=0x7f7f280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b423d0c86a1da2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.048+0000 7f7f3159a700 1 -- 172.21.15.119:0/2480325609 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28011630 con 0x7f7f340b16d0 2023-10-08T02:11:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3159a700 1 -- 172.21.15.119:0/2480325609 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f28011790 con 0x7f7f340b16d0 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3159a700 1 -- 172.21.15.119:0/2480325609 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28011b30 con 0x7f7f340b16d0 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3359e700 1 -- 172.21.15.119:0/2480325609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b16d0 msgr2=0x7f7f340b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3359e700 1 --2- 172.21.15.119:0/2480325609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b16d0 0x7f7f340b1b40 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f7f34064910 tx=0x7f7f280091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3359e700 1 -- 172.21.15.119:0/2480325609 shutdown_connections 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3359e700 1 --2- 172.21.15.119:0/2480325609 >> v2:172.21.15.184:3300/0 conn(0x7f7f340b4e70 0x7f7f340ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3359e700 1 --2- 172.21.15.119:0/2480325609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b16d0 0x7f7f340b1b40 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3359e700 1 -- 172.21.15.119:0/2480325609 >> 172.21.15.119:0/2480325609 conn(0x7f7f340d10c0 msgr2=0x7f7f340affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3359e700 1 -- 172.21.15.119:0/2480325609 shutdown_connections 2023-10-08T02:11:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.049+0000 7f7f3359e700 1 -- 172.21.15.119:0/2480325609 wait complete. 2023-10-08T02:11:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.051+0000 7f7f3359e700 1 Processor -- start 2023-10-08T02:11:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.051+0000 7f7f3359e700 1 -- start start 2023-10-08T02:11:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f3359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f340b16d0 0x7f7f340c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f3359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b4e70 0x7f7f34169ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.054 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f356cb43180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:11:25.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f3359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f3415d240 0x7f7f3415e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f3359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3416ac80 con 0x7f7f340b4e70 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f3359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3416d750 con 0x7f7f3415d240 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f3359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3416a6c0 con 0x7f7f340b16d0 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f31d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b4e70 0x7f7f34169ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f31d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b4e70 0x7f7f34169ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52750/0 (socket says 172.21.15.119:52750) 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f31d9b700 1 -- 172.21.15.119:0/770007108 learned_addr learned my addr 172.21.15.119:0/770007108 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f3259c700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f340b16d0 0x7f7f340c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f32d9d700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f3415d240 0x7f7f3415e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f31d9b700 1 -- 172.21.15.119:0/770007108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f340b16d0 msgr2=0x7f7f340c81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f31d9b700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f340b16d0 0x7f7f340c81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f31d9b700 1 -- 172.21.15.119:0/770007108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f3415d240 msgr2=0x7f7f3415e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f31d9b700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f3415d240 0x7f7f3415e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.052+0000 7f7f31d9b700 1 -- 172.21.15.119:0/770007108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f28009420 con 0x7f7f340b4e70 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.053+0000 7f7f31d9b700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b4e70 0x7f7f34169ca0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f7f34070dd0 tx=0x7f7f28017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.053+0000 7f7f227fc700 1 -- 172.21.15.119:0/770007108 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28000d10 con 0x7f7f340b4e70 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.053+0000 7f7f227fc700 1 -- 172.21.15.119:0/770007108 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f28011b10 con 0x7f7f340b4e70 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.053+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3416a9a0 con 0x7f7f340b4e70 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.053+0000 7f7f227fc700 1 -- 172.21.15.119:0/770007108 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f28025640 con 0x7f7f340b4e70 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.053+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f340c86f0 con 0x7f7f340b4e70 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.054+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3416d240 con 0x7f7f340b4e70 2023-10-08T02:11:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.055+0000 7f7f227fc700 1 -- 172.21.15.119:0/770007108 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f7f28025a30 con 0x7f7f340b4e70 2023-10-08T02:11:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.056+0000 7f7f227fc700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7f14067050 0x7f7f14069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.056+0000 7f7f227fc700 1 -- 172.21.15.119:0/770007108 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f7f280861a0 con 0x7f7f340b4e70 2023-10-08T02:11:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.056+0000 7f7f3259c700 1 -- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7f14067050 msgr2=0x7f7f14069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.056+0000 7f7f3259c700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7f14067050 0x7f7f14069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.060+0000 7f7f227fc700 1 -- 172.21.15.119:0/770007108 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7f3416d240 con 0x7f7f340b4e70 2023-10-08T02:11:25.162 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:25.161+0000 7f482f797180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:11:25.237 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:25.236+0000 7f356cb43180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:11:25.254 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:25.253+0000 7f482f797180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:11:25.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.257+0000 7f7f3259c700 1 -- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7f14067050 msgr2=0x7f7f14069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:25.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.257+0000 7f7f3259c700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7f14067050 0x7f7f14069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:25.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.261+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f3416d240 con 0x7f7f340b4e70 2023-10-08T02:11:25.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.262+0000 7f7f227fc700 1 -- 172.21.15.119:0/770007108 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v143) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f3416d240 con 0x7f7f340b4e70 2023-10-08T02:11:25.262 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T02:11:25.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.264+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7f14067050 msgr2=0x7f7f14069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:25.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.264+0000 7f7f3359e700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7f14067050 0x7f7f14069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.264+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b4e70 msgr2=0x7f7f34169ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.264+0000 7f7f3359e700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b4e70 0x7f7f34169ca0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f7f34070dd0 tx=0x7f7f28017000 comp rx=0 tx=0).stop 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.264+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 shutdown_connections 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.264+0000 7f7f3359e700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7f14067050 0x7f7f14069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.264+0000 7f7f3359e700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7f340b16d0 0x7f7f340c81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.264+0000 7f7f3359e700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7f3415d240 0x7f7f3415e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.265+0000 7f7f3359e700 1 --2- 172.21.15.119:0/770007108 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f340b4e70 0x7f7f34169ca0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.265+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 >> 172.21.15.119:0/770007108 conn(0x7f7f340d10c0 msgr2=0x7f7f340b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.265+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 shutdown_connections 2023-10-08T02:11:25.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.265+0000 7f7f3359e700 1 -- 172.21.15.119:0/770007108 wait complete. 2023-10-08T02:11:25.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T02:11:25.334 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:25.333+0000 7f356cb43180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:11:25.417 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:25.416+0000 7f482f797180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:11:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.473+0000 7f4763fff700 1 Processor -- start 2023-10-08T02:11:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.474+0000 7f4763fff700 1 -- start start 2023-10-08T02:11:25.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f4763fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f47640c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f4763fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f47640cb230 0x7f47640cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f4763fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47640c9cb0 con 0x7f47640ccc00 2023-10-08T02:11:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f4763fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47640cbbd0 con 0x7f47640cb230 2023-10-08T02:11:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f4763fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476406dfc0 con 0x7f4764159a60 2023-10-08T02:11:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f47637fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f47640cb230 0x7f47640cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f47640c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f47640c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:60782/0 (socket says 172.21.15.119:60782) 2023-10-08T02:11:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f47637fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f47640cb230 0x7f47640cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:55484/0 (socket says 172.21.15.119:55484) 2023-10-08T02:11:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f4762ffd700 1 -- 172.21.15.119:0/981924567 learned_addr learned my addr 172.21.15.119:0/981924567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.475+0000 7f47627fc700 1 --1- 172.21.15.119:0/981924567 >> v1:172.21.15.184:6789/0 conn(0x7f47640ccc00 0x7f4764069ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:49674/0 (socket says 172.21.15.119:49674) 2023-10-08T02:11:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f4761ffb700 1 -- 172.21.15.119:0/981924567 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4186573681 0 0) 0x7f47640c9cb0 con 0x7f47640ccc00 2023-10-08T02:11:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f4761ffb700 1 -- 172.21.15.119:0/981924567 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f474c003570 con 0x7f47640ccc00 2023-10-08T02:11:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f47637fe700 1 -- 172.21.15.119:0/981924567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 msgr2=0x7f47640c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f47637fe700 1 --2- 172.21.15.119:0/981924567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f47640c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f47637fe700 1 -- 172.21.15.119:0/981924567 >> v1:172.21.15.184:6789/0 conn(0x7f47640ccc00 legacy=0x7f4764069ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f47637fe700 1 -- 172.21.15.119:0/981924567 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47640c9f30 con 0x7f47640cb230 2023-10-08T02:11:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f4762ffd700 1 --2- 172.21.15.119:0/981924567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f47640c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:11:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f47637fe700 1 --2- 172.21.15.119:0/981924567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640cb230 0x7f47640cb690 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f475c009e90 tx=0x7f475c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d62cb054caa64d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f4761ffb700 1 -- 172.21.15.119:0/981924567 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c00e070 con 0x7f47640cb230 2023-10-08T02:11:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f4761ffb700 1 -- 172.21.15.119:0/981924567 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475c00bbd0 con 0x7f47640cb230 2023-10-08T02:11:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.476+0000 7f4761ffb700 1 -- 172.21.15.119:0/981924567 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475c012430 con 0x7f47640cb230 2023-10-08T02:11:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.477+0000 7f4763fff700 1 -- 172.21.15.119:0/981924567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640cb230 msgr2=0x7f47640cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.477+0000 7f4763fff700 1 --2- 172.21.15.119:0/981924567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640cb230 0x7f47640cb690 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f475c009e90 tx=0x7f475c009780 comp rx=0 tx=0).stop 2023-10-08T02:11:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.477+0000 7f4763fff700 1 -- 172.21.15.119:0/981924567 shutdown_connections 2023-10-08T02:11:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.477+0000 7f4763fff700 1 --2- 172.21.15.119:0/981924567 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640cb230 0x7f47640cb690 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.477+0000 7f4763fff700 1 --2- 172.21.15.119:0/981924567 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f47640c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.477+0000 7f4763fff700 1 -- 172.21.15.119:0/981924567 >> 172.21.15.119:0/981924567 conn(0x7f476406baa0 msgr2=0x7f476406bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.477+0000 7f4763fff700 1 -- 172.21.15.119:0/981924567 shutdown_connections 2023-10-08T02:11:25.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.477+0000 7f4763fff700 1 -- 172.21.15.119:0/981924567 wait complete. 2023-10-08T02:11:25.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.478+0000 7f4763fff700 1 Processor -- start 2023-10-08T02:11:25.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.479+0000 7f4763fff700 1 -- start start 2023-10-08T02:11:25.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.479+0000 7f4763fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640cb230 0x7f47640c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4763fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640ccc00 0x7f476416ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4763fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f4764154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4763fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f476416b060 con 0x7f47640cb230 2023-10-08T02:11:25.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4763fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4764155f40 con 0x7f47640ccc00 2023-10-08T02:11:25.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4763fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4764156220 con 0x7f4764159a60 2023-10-08T02:11:25.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640cb230 0x7f47640c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4762ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640cb230 0x7f47640c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52766/0 (socket says 172.21.15.119:52766) 2023-10-08T02:11:25.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4762ffd700 1 -- 172.21.15.119:0/2284167052 learned_addr learned my addr 172.21.15.119:0/2284167052 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:25.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f47627fc700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640ccc00 0x7f476416ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f47637fe700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f4764154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4762ffd700 1 -- 172.21.15.119:0/2284167052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 msgr2=0x7f4764154ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4762ffd700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f4764154ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4762ffd700 1 -- 172.21.15.119:0/2284167052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640ccc00 msgr2=0x7f476416ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4762ffd700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640ccc00 0x7f476416ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.480+0000 7f4762ffd700 1 -- 172.21.15.119:0/2284167052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47540094f0 con 0x7f47640cb230 2023-10-08T02:11:25.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.481+0000 7f4762ffd700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640cb230 0x7f47640c7c10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f475400ec50 tx=0x7f4754009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:25.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.481+0000 7f474affd700 1 -- 172.21.15.119:0/2284167052 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400c850 con 0x7f47640cb230 2023-10-08T02:11:25.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.481+0000 7f474affd700 1 -- 172.21.15.119:0/2284167052 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475400c9b0 con 0x7f47640cb230 2023-10-08T02:11:25.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.481+0000 7f474affd700 1 -- 172.21.15.119:0/2284167052 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475400cd30 con 0x7f47640cb230 2023-10-08T02:11:25.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.481+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475c009a20 con 0x7f47640cb230 2023-10-08T02:11:25.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.481+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47641567e0 con 0x7f47640cb230 2023-10-08T02:11:25.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.482+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4764156b50 con 0x7f47640cb230 2023-10-08T02:11:25.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.483+0000 7f474affd700 1 -- 172.21.15.119:0/2284167052 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f4754013680 con 0x7f47640cb230 2023-10-08T02:11:25.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.484+0000 7f474affd700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4750067050 0x7f4750069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.484+0000 7f474affd700 1 -- 172.21.15.119:0/2284167052 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f4754084d20 con 0x7f47640cb230 2023-10-08T02:11:25.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.484+0000 7f47627fc700 1 -- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4750067050 msgr2=0x7f4750069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:25.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.484+0000 7f47627fc700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4750067050 0x7f4750069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:25.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.488+0000 7f474affd700 1 -- 172.21.15.119:0/2284167052 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4754051aa0 con 0x7f47640cb230 2023-10-08T02:11:25.505 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:25.504+0000 7f356cb43180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:11:25.517 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:25.517+0000 7f482f797180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:11:25.609 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:25.608+0000 7f356cb43180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:11:25.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.685+0000 7f47627fc700 1 -- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4750067050 msgr2=0x7f4750069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:25.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.685+0000 7f47627fc700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4750067050 0x7f4750069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:25.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.687+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47640c91a0 con 0x7f47640cb230 2023-10-08T02:11:25.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.688+0000 7f474affd700 1 -- 172.21.15.119:0/2284167052 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v143) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47540550c0 con 0x7f47640cb230 2023-10-08T02:11:25.688 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.690+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4750067050 msgr2=0x7f4750069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.690+0000 7f4763fff700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4750067050 0x7f4750069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.690+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640cb230 msgr2=0x7f47640c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.690+0000 7f4763fff700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640cb230 0x7f47640c7c10 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f475400ec50 tx=0x7f4754009250 comp rx=0 tx=0).stop 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.691+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 shutdown_connections 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.691+0000 7f4763fff700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4750067050 0x7f4750069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.691+0000 7f4763fff700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4764159a60 0x7f4764154ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.691+0000 7f4763fff700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47640cb230 0x7f47640c7c10 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.691+0000 7f4763fff700 1 --2- 172.21.15.119:0/2284167052 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47640ccc00 0x7f476416ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.691+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 >> 172.21.15.119:0/2284167052 conn(0x7f476406baa0 msgr2=0x7f4764153fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:25.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.691+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 shutdown_connections 2023-10-08T02:11:25.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.691+0000 7f4763fff700 1 -- 172.21.15.119:0/2284167052 wait complete. 2023-10-08T02:11:25.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:11:25.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.897+0000 7fe9effff700 1 Processor -- start 2023-10-08T02:11:25.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.897+0000 7fe9effff700 1 -- start start 2023-10-08T02:11:25.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9effff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe9f01593b0 0x7fe9f0159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9effff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9f00ceaa0 0x7fe9f00b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9effff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f00b6570 con 0x7fe9f00b1740 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9effff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00cfe20 con 0x7fe9f00ceaa0 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9effff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00cf210 con 0x7fe9f01593b0 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9ee7fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9f00ceaa0 0x7fe9f00b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9ef7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe9f00b1740 0x7fe9f00b1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47366/0 (socket says 172.21.15.119:47366) 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9ef7fe700 1 -- 172.21.15.119:0/2009577859 learned_addr learned my addr 172.21.15.119:0/2009577859 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9eeffd700 1 --2- 172.21.15.119:0/2009577859 >> v2:172.21.15.184:3300/0 conn(0x7fe9f01593b0 0x7fe9f0159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9ee7fc700 1 -- 172.21.15.119:0/2009577859 >> v2:172.21.15.184:3300/0 conn(0x7fe9f01593b0 msgr2=0x7fe9f0159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9ee7fc700 1 --2- 172.21.15.119:0/2009577859 >> v2:172.21.15.184:3300/0 conn(0x7fe9f01593b0 0x7fe9f0159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9ee7fc700 1 -- 172.21.15.119:0/2009577859 >> v1:172.21.15.119:6789/0 conn(0x7fe9f00b1740 legacy=0x7fe9f00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.899+0000 7fe9edffb700 1 -- 172.21.15.119:0/2009577859 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589730220 0 0) 0x7fe9e8001490 con 0x7fe9f00b1740 2023-10-08T02:11:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.900+0000 7fe9ee7fc700 1 -- 172.21.15.119:0/2009577859 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f00cf4f0 con 0x7fe9f00ceaa0 2023-10-08T02:11:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.900+0000 7fe9ee7fc700 1 --2- 172.21.15.119:0/2009577859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f00ceaa0 0x7fe9f00b6030 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0070d60 tx=0x7fe9e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed997e6f3a151ef0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.900+0000 7fe9edffb700 1 -- 172.21.15.119:0/2009577859 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4011630 con 0x7fe9f00ceaa0 2023-10-08T02:11:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.900+0000 7fe9edffb700 1 -- 172.21.15.119:0/2009577859 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4011790 con 0x7fe9f00ceaa0 2023-10-08T02:11:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.900+0000 7fe9edffb700 1 -- 172.21.15.119:0/2009577859 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4011b30 con 0x7fe9f00ceaa0 2023-10-08T02:11:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.901+0000 7fe9effff700 1 -- 172.21.15.119:0/2009577859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f00ceaa0 msgr2=0x7fe9f00b6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.901+0000 7fe9effff700 1 --2- 172.21.15.119:0/2009577859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f00ceaa0 0x7fe9f00b6030 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0070d60 tx=0x7fe9e40091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.901+0000 7fe9effff700 1 -- 172.21.15.119:0/2009577859 shutdown_connections 2023-10-08T02:11:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.901+0000 7fe9effff700 1 --2- 172.21.15.119:0/2009577859 >> v2:172.21.15.184:3300/0 conn(0x7fe9f01593b0 0x7fe9f0159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.901+0000 7fe9effff700 1 --2- 172.21.15.119:0/2009577859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f00ceaa0 0x7fe9f00b6030 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.901+0000 7fe9effff700 1 -- 172.21.15.119:0/2009577859 >> 172.21.15.119:0/2009577859 conn(0x7fe9f00d1130 msgr2=0x7fe9f00b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.901+0000 7fe9effff700 1 -- 172.21.15.119:0/2009577859 shutdown_connections 2023-10-08T02:11:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.901+0000 7fe9effff700 1 -- 172.21.15.119:0/2009577859 wait complete. 2023-10-08T02:11:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.902+0000 7fe9effff700 1 Processor -- start 2023-10-08T02:11:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.902+0000 7fe9effff700 1 -- start start 2023-10-08T02:11:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.903+0000 7fe9effff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9f00b1740 0x7fe9f00c5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.903+0000 7fe9effff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f00ceaa0 0x7fe9f0155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.903+0000 7fe9effff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f01593b0 0x7fe9f00cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.903+0000 7fe9effff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f0155940 con 0x7fe9f01593b0 2023-10-08T02:11:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.903+0000 7fe9effff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00cc290 con 0x7fe9f00ceaa0 2023-10-08T02:11:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.903+0000 7fe9effff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00c5a90 con 0x7fe9f00b1740 2023-10-08T02:11:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f01593b0 0x7fe9f00cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f01593b0 0x7fe9f00cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52774/0 (socket says 172.21.15.119:52774) 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 -- 172.21.15.119:0/722548369 learned_addr learned my addr 172.21.15.119:0/722548369 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ee7fc700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f00ceaa0 0x7fe9f0155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9eeffd700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9f00b1740 0x7fe9f00c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 -- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9f00b1740 msgr2=0x7fe9f00c5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9f00b1740 0x7fe9f00c5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 -- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f00ceaa0 msgr2=0x7fe9f0155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f00ceaa0 0x7fe9f0155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 -- 172.21.15.119:0/722548369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4009420 con 0x7fe9f01593b0 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9eeffd700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9f00b1740 0x7fe9f00c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9ef7fe700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f01593b0 0x7fe9f00cbd50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800cc90 tx=0x7fe9e8001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9d6ffd700 1 -- 172.21.15.119:0/722548369 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8004bb0 con 0x7fe9f01593b0 2023-10-08T02:11:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9d6ffd700 1 -- 172.21.15.119:0/722548369 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e8004d10 con 0x7fe9f01593b0 2023-10-08T02:11:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.904+0000 7fe9d6ffd700 1 -- 172.21.15.119:0/722548369 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800abb0 con 0x7fe9f01593b0 2023-10-08T02:11:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.905+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f00c5d70 con 0x7fe9f01593b0 2023-10-08T02:11:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.905+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f00c6330 con 0x7fe9f01593b0 2023-10-08T02:11:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.906+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f004c3c0 con 0x7fe9f01593b0 2023-10-08T02:11:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.907+0000 7fe9d6ffd700 1 -- 172.21.15.119:0/722548369 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fe9e80070f0 con 0x7fe9f01593b0 2023-10-08T02:11:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.908+0000 7fe9d6ffd700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9dc067050 0x7fe9dc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.908+0000 7fe9d6ffd700 1 -- 172.21.15.119:0/722548369 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fe9e8084670 con 0x7fe9f01593b0 2023-10-08T02:11:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.908+0000 7fe9eeffd700 1 -- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9dc067050 msgr2=0x7fe9dc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.908+0000 7fe9eeffd700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9dc067050 0x7fe9dc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:25.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:25.911+0000 7fe9d6ffd700 1 -- 172.21.15.119:0/722548369 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9e8051510 con 0x7fe9f01593b0 2023-10-08T02:11:26.000 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:26.000+0000 7f482f797180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:11:26.094 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:26.093+0000 7f482f797180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:11:26.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.109+0000 7fe9eeffd700 1 -- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9dc067050 msgr2=0x7fe9dc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:26.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.109+0000 7fe9eeffd700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9dc067050 0x7fe9dc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:26.113 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:26.112+0000 7f356cb43180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:11:26.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.154+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9f00b6570 con 0x7fe9f01593b0 2023-10-08T02:11:26.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.155+0000 7fe9d6ffd700 1 -- 172.21.15.119:0/722548369 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fe9f00b6570 con 0x7fe9f01593b0 2023-10-08T02:11:26.156 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:26.156 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7549,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/26538394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":26538394},{"type":"v1","addr":"172.21.15.184:6834","nonce":26538394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3494509212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3494509212},{"type":"v1","addr":"172.21.15.119:6834","nonce":3494509212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2792091862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2792091862},{"type":"v1","addr":"172.21.15.119:6835","nonce":2792091862}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:15:21.218141+0000","modified":"2023-10-08T01:15:25.231364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1485819493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1485819493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1485819493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T02:11:26.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9dc067050 msgr2=0x7fe9dc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:26.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9dc067050 0x7fe9dc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f01593b0 msgr2=0x7fe9f00cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f01593b0 0x7fe9f00cbd50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800cc90 tx=0x7fe9e8001a50 comp rx=0 tx=0).stop 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 shutdown_connections 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9dc067050 0x7fe9dc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9f00b1740 0x7fe9f00c5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9f01593b0 0x7fe9f00cbd50 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 --2- 172.21.15.119:0/722548369 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9f00ceaa0 0x7fe9f0155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.157+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 >> 172.21.15.119:0/722548369 conn(0x7fe9f00d1130 msgr2=0x7fe9f0157b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.158+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 shutdown_connections 2023-10-08T02:11:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.158+0000 7fe9effff700 1 -- 172.21.15.119:0/722548369 wait complete. 2023-10-08T02:11:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 63 2023-10-08T02:11:26.180 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:11:26.187 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:26.187+0000 7f482f797180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:11:26.215 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:26.215+0000 7f356cb43180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:11:26.310 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:26.310+0000 7f356cb43180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:11:26.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.365+0000 7f91d7fff700 1 Processor -- start 2023-10-08T02:11:26.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.365+0000 7f91d7fff700 1 -- start start 2023-10-08T02:11:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.366+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d8159420 0x7f91d80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91d81597f0 0x7f91d80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d7fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d80b11d0 con 0x7f91d815d230 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d7fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d80b14b0 con 0x7f91d81597f0 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d7fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80b1790 con 0x7f91d8159420 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d67fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91d81597f0 0x7f91d80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d67fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f91d81597f0 0x7f91d80b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52784/0 (socket says 172.21.15.119:52784) 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d67fc700 1 -- 172.21.15.119:0/886721777 learned_addr learned my addr 172.21.15.119:0/886721777 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d6ffd700 1 --2- 172.21.15.119:0/886721777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d8159420 0x7f91d80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d67fc700 1 -- 172.21.15.119:0/886721777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d8159420 msgr2=0x7f91d80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d67fc700 1 --2- 172.21.15.119:0/886721777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d8159420 0x7f91d80b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d67fc700 1 -- 172.21.15.119:0/886721777 >> v1:172.21.15.184:6789/0 conn(0x7f91d815d230 legacy=0x7f91d80b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d67fc700 1 -- 172.21.15.119:0/886721777 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80b1a70 con 0x7f91d81597f0 2023-10-08T02:11:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.367+0000 7f91d5ffb700 1 -- 172.21.15.119:0/886721777 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719649225 0 0) 0x7f91d80b11d0 con 0x7f91d815d230 2023-10-08T02:11:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.368+0000 7f91d67fc700 1 --2- 172.21.15.119:0/886721777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d81597f0 0x7f91d80b2270 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0099b0 tx=0x7f91cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7d58cf80af9202 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.368+0000 7f91d5ffb700 1 -- 172.21.15.119:0/886721777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc009ea0 con 0x7f91d81597f0 2023-10-08T02:11:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.368+0000 7f91d5ffb700 1 -- 172.21.15.119:0/886721777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc00b860 con 0x7f91d81597f0 2023-10-08T02:11:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.368+0000 7f91d5ffb700 1 -- 172.21.15.119:0/886721777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc00bc00 con 0x7f91d81597f0 2023-10-08T02:11:26.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.368+0000 7f91d7fff700 1 -- 172.21.15.119:0/886721777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d81597f0 msgr2=0x7f91d80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.368+0000 7f91d7fff700 1 --2- 172.21.15.119:0/886721777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d81597f0 0x7f91d80b2270 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0099b0 tx=0x7f91cc009250 comp rx=0 tx=0).stop 2023-10-08T02:11:26.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.369+0000 7f91d7fff700 1 -- 172.21.15.119:0/886721777 shutdown_connections 2023-10-08T02:11:26.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.369+0000 7f91d7fff700 1 --2- 172.21.15.119:0/886721777 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d8159420 0x7f91d80b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.369+0000 7f91d7fff700 1 --2- 172.21.15.119:0/886721777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d81597f0 0x7f91d80b2270 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.369+0000 7f91d7fff700 1 -- 172.21.15.119:0/886721777 >> 172.21.15.119:0/886721777 conn(0x7f91d80d0660 msgr2=0x7f91d80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.369+0000 7f91d7fff700 1 -- 172.21.15.119:0/886721777 shutdown_connections 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.369+0000 7f91d7fff700 1 -- 172.21.15.119:0/886721777 wait complete. 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.370+0000 7f91d7fff700 1 Processor -- start 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.370+0000 7f91d7fff700 1 -- start start 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d8159420 0x7f91d816c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d81597f0 0x7f91d816a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d815d230 0x7f91d81571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d7fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8151280 con 0x7f91d8159420 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d7fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80b5ac0 con 0x7f91d81597f0 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d7fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80b5da0 con 0x7f91d815d230 2023-10-08T02:11:26.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d8159420 0x7f91d816c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d8159420 0x7f91d816c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52788/0 (socket says 172.21.15.119:52788) 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d6ffd700 1 -- 172.21.15.119:0/972314183 learned_addr learned my addr 172.21.15.119:0/972314183 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d77fe700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d815d230 0x7f91d81571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.371+0000 7f91d67fc700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d81597f0 0x7f91d816a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d6ffd700 1 -- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d815d230 msgr2=0x7f91d81571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d6ffd700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d815d230 0x7f91d81571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d6ffd700 1 -- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d81597f0 msgr2=0x7f91d816a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d6ffd700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d81597f0 0x7f91d816a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d6ffd700 1 -- 172.21.15.119:0/972314183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc0094f0 con 0x7f91d8159420 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d77fe700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d815d230 0x7f91d81571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d6ffd700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d8159420 0x7f91d816c6e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f91c800ec80 tx=0x7f91c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:26.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91beffd700 1 -- 172.21.15.119:0/972314183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c800c9d0 con 0x7f91d8159420 2023-10-08T02:11:26.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91beffd700 1 -- 172.21.15.119:0/972314183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c800cb30 con 0x7f91d8159420 2023-10-08T02:11:26.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d8151520 con 0x7f91d8159420 2023-10-08T02:11:26.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91beffd700 1 -- 172.21.15.119:0/972314183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8013470 con 0x7f91d8159420 2023-10-08T02:11:26.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.372+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d8157730 con 0x7f91d8159420 2023-10-08T02:11:26.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.373+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d804c3c0 con 0x7f91d8159420 2023-10-08T02:11:26.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.374+0000 7f91beffd700 1 -- 172.21.15.119:0/972314183 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f91c8013820 con 0x7f91d8159420 2023-10-08T02:11:26.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.375+0000 7f91beffd700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91c4067050 0x7f91c4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.375+0000 7f91beffd700 1 -- 172.21.15.119:0/972314183 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f91c8084e00 con 0x7f91d8159420 2023-10-08T02:11:26.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.375+0000 7f91d67fc700 1 -- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91c4067050 msgr2=0x7f91c4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:26.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.375+0000 7f91d67fc700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91c4067050 0x7f91c4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:26.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.381+0000 7f91beffd700 1 -- 172.21.15.119:0/972314183 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f91c8051c20 con 0x7f91d8159420 2023-10-08T02:11:26.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.576+0000 7f91d67fc700 1 -- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91c4067050 msgr2=0x7f91c4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:26.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.576+0000 7f91d67fc700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91c4067050 0x7f91c4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:26.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.584+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91d80b1d50 con 0x7f91d8159420 2023-10-08T02:11:26.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.586+0000 7f91beffd700 1 -- 172.21.15.119:0/972314183 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16691 (secure 0 0 0) 0x7f91c8017090 con 0x7f91d8159420 2023-10-08T02:11:26.587 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:26.587 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":144,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:11:26.160075+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:15:19.321580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T01:15:20.200200+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":2}]},{"pgid":"17.3c","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1681557219":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/1844314331":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/3405191039":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/3624517726":"2023-10-09T01:15:42.377210+0000","172.21.15.119:6837/85511":"2023-10-09T01:15:42.377210+0000","172.21.15.119:6836/85511":"2023-10-09T01:15:42.377210+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:11:26.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.588+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91c4067050 msgr2=0x7f91c4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:26.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.588+0000 7f91d7fff700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91c4067050 0x7f91c4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.588+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d8159420 msgr2=0x7f91d816c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.588+0000 7f91d7fff700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d8159420 0x7f91d816c6e0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f91c800ec80 tx=0x7f91c80091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.588+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 shutdown_connections 2023-10-08T02:11:26.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.588+0000 7f91d7fff700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f91c4067050 0x7f91c4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.588+0000 7f91d7fff700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f91d815d230 0x7f91d81571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.589+0000 7f91d7fff700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91d8159420 0x7f91d816c6e0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.589+0000 7f91d7fff700 1 --2- 172.21.15.119:0/972314183 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f91d81597f0 0x7f91d816a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.589+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 >> 172.21.15.119:0/972314183 conn(0x7f91d80d0660 msgr2=0x7f91d8155100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:26.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.589+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 shutdown_connections 2023-10-08T02:11:26.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.590+0000 7f91d7fff700 1 -- 172.21.15.119:0/972314183 wait complete. 2023-10-08T02:11:26.611 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T02:11:26.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:11:26.810 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:26.809+0000 7f482f797180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:11:26.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.842+0000 7f700eb9d700 1 Processor -- start 2023-10-08T02:11:26.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.842+0000 7f700eb9d700 1 -- start start 2023-10-08T02:11:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.844+0000 7f700eb9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f700815d230 0x7f700815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.844+0000 7f700eb9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70080b2980 con 0x7f70081597f0 2023-10-08T02:11:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700eb9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700815db70 con 0x7f7008159420 2023-10-08T02:11:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700eb9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70080b11d0 con 0x7f700815d230 2023-10-08T02:11:26.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700e39c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f700815d230 0x7f700815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:26.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700e39c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f700815d230 0x7f700815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:55528/0 (socket says 172.21.15.119:55528) 2023-10-08T02:11:26.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700e39c700 1 -- 172.21.15.119:0/1453419605 learned_addr learned my addr 172.21.15.119:0/1453419605 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:26.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700db9b700 1 --1- 172.21.15.119:0/1453419605 >> v1:172.21.15.184:6789/0 conn(0x7f7008159420 0x7f70080b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:49688/0 (socket says 172.21.15.119:49688) 2023-10-08T02:11:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700cb99700 1 -- 172.21.15.119:0/1453419605 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539694661 0 0) 0x7f700815db70 con 0x7f7008159420 2023-10-08T02:11:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700cb99700 1 -- 172.21.15.119:0/1453419605 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ff4003570 con 0x7f7008159420 2023-10-08T02:11:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700e39c700 1 -- 172.21.15.119:0/1453419605 >> v1:172.21.15.184:6789/0 conn(0x7f7008159420 legacy=0x7f70080b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700e39c700 1 -- 172.21.15.119:0/1453419605 >> v1:172.21.15.119:6789/0 conn(0x7f70081597f0 legacy=0x7f70080b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.845+0000 7f700e39c700 1 -- 172.21.15.119:0/1453419605 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70080b14b0 con 0x7f700815d230 2023-10-08T02:11:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.846+0000 7f700e39c700 1 --2- 172.21.15.119:0/1453419605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f700815d230 0x7f700815d630 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7000009f20 tx=0x7f7000009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e537b554c3e8652e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.846+0000 7f700cb99700 1 -- 172.21.15.119:0/1453419605 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700000e070 con 0x7f700815d230 2023-10-08T02:11:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.846+0000 7f700cb99700 1 -- 172.21.15.119:0/1453419605 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700000b9d0 con 0x7f700815d230 2023-10-08T02:11:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.846+0000 7f700cb99700 1 -- 172.21.15.119:0/1453419605 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70000092e0 con 0x7f700815d230 2023-10-08T02:11:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.847+0000 7f700eb9d700 1 -- 172.21.15.119:0/1453419605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f700815d230 msgr2=0x7f700815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.847+0000 7f700eb9d700 1 --2- 172.21.15.119:0/1453419605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f700815d230 0x7f700815d630 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7000009f20 tx=0x7f7000009840 comp rx=0 tx=0).stop 2023-10-08T02:11:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.848+0000 7f700eb9d700 1 -- 172.21.15.119:0/1453419605 shutdown_connections 2023-10-08T02:11:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.848+0000 7f700eb9d700 1 --2- 172.21.15.119:0/1453419605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f700815d230 0x7f700815d630 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.848+0000 7f700eb9d700 1 -- 172.21.15.119:0/1453419605 >> 172.21.15.119:0/1453419605 conn(0x7f70080d0660 msgr2=0x7f70080d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:26.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.849+0000 7f700eb9d700 1 -- 172.21.15.119:0/1453419605 shutdown_connections 2023-10-08T02:11:26.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.849+0000 7f700eb9d700 1 -- 172.21.15.119:0/1453419605 wait complete. 2023-10-08T02:11:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.851+0000 7f700eb9d700 1 Processor -- start 2023-10-08T02:11:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.851+0000 7f700eb9d700 1 -- start start 2023-10-08T02:11:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.852+0000 7f700eb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7008159420 0x7f70080c5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.852+0000 7f700eb9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70081597f0 0x7f70081550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.852+0000 7f700eb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f700815d230 0x7f700816cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.853+0000 7f700eb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70080ca800 con 0x7f7008159420 2023-10-08T02:11:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.853+0000 7f700eb9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70080c5950 con 0x7f70081597f0 2023-10-08T02:11:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.853+0000 7f700eb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700816d880 con 0x7f700815d230 2023-10-08T02:11:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.853+0000 7f700db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7008159420 0x7f70080c5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.853+0000 7f700db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7008159420 0x7f70080c5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52796/0 (socket says 172.21.15.119:52796) 2023-10-08T02:11:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.853+0000 7f700db9b700 1 -- 172.21.15.119:0/2487289660 learned_addr learned my addr 172.21.15.119:0/2487289660 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.853+0000 7f700d39a700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70081597f0 0x7f70081550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.854+0000 7f700d39a700 1 -- 172.21.15.119:0/2487289660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f700815d230 msgr2=0x7f700816cd10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.854+0000 7f700d39a700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f700815d230 0x7f700816cd10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.854+0000 7f700d39a700 1 -- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7008159420 msgr2=0x7f70080c5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.854+0000 7f700d39a700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7008159420 0x7f70080c5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.854+0000 7f700d39a700 1 -- 172.21.15.119:0/2487289660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000009ae0 con 0x7f70081597f0 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.854+0000 7f700db9b700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7008159420 0x7f70080c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.854+0000 7f700d39a700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70081597f0 0x7f70081550a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800c560 tx=0x7f6ff8007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.855+0000 7f6ffdffb700 1 -- 172.21.15.119:0/2487289660 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011590 con 0x7f70081597f0 2023-10-08T02:11:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.855+0000 7f6ffdffb700 1 -- 172.21.15.119:0/2487289660 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8011710 con 0x7f70081597f0 2023-10-08T02:11:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.855+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700816db60 con 0x7f70081597f0 2023-10-08T02:11:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.856+0000 7f6ffdffb700 1 -- 172.21.15.119:0/2487289660 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011ad0 con 0x7f70081597f0 2023-10-08T02:11:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.856+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70080c6340 con 0x7f70081597f0 2023-10-08T02:11:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.857+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70080ca9b0 con 0x7f70081597f0 2023-10-08T02:11:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.858+0000 7f6ffdffb700 1 -- 172.21.15.119:0/2487289660 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f6ff8012420 con 0x7f70081597f0 2023-10-08T02:11:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.859+0000 7f6ffdffb700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fec067020 0x7f6fec0694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.860+0000 7f6ffdffb700 1 -- 172.21.15.119:0/2487289660 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f6ff8083d70 con 0x7f70081597f0 2023-10-08T02:11:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.860+0000 7f700db9b700 1 -- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fec067020 msgr2=0x7f6fec0694d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.860+0000 7f700db9b700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fec067020 0x7f6fec0694d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:26.864+0000 7f6ffdffb700 1 -- 172.21.15.119:0/2487289660 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70080ca9b0 con 0x7f70081597f0 2023-10-08T02:11:26.972 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:26.971+0000 7f356cb43180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:11:26.987 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:26.987+0000 7f482f797180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:11:27.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.061+0000 7f700db9b700 1 -- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fec067020 msgr2=0x7f6fec0694d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:27.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.061+0000 7f700db9b700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fec067020 0x7f6fec0694d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:27.082 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:27.082+0000 7f482f797180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:11:27.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.144+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70080ca970 con 0x7f70081597f0 2023-10-08T02:11:27.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.144+0000 7f6ffdffb700 1 -- 172.21.15.119:0/2487289660 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f6ff8017020 con 0x7f70081597f0 2023-10-08T02:11:27.145 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:27.145 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7549,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/26538394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":26538394},{"type":"v1","addr":"172.21.15.184:6834","nonce":26538394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3494509212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3494509212},{"type":"v1","addr":"172.21.15.119:6834","nonce":3494509212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2792091862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2792091862},{"type":"v1","addr":"172.21.15.119:6835","nonce":2792091862}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:15:21.218141+0000","modified":"2023-10-08T01:15:25.231364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1485819493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1485819493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1485819493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T02:11:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fec067020 msgr2=0x7f6fec0694d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fec067020 0x7f6fec0694d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70081597f0 msgr2=0x7f70081550a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70081597f0 0x7f70081550a0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800c560 tx=0x7f6ff8007ec0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 shutdown_connections 2023-10-08T02:11:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6fec067020 0x7f6fec0694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f700815d230 0x7f700816cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7008159420 0x7f70080c5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 --2- 172.21.15.119:0/2487289660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70081597f0 0x7f70081550a0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 >> 172.21.15.119:0/2487289660 conn(0x7f70080d0660 msgr2=0x7f70080b5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:27.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 shutdown_connections 2023-10-08T02:11:27.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.148+0000 7f700eb9d700 1 -- 172.21.15.119:0/2487289660 wait complete. 2023-10-08T02:11:27.150 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 63 2023-10-08T02:11:27.158 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:27.157+0000 7f356cb43180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:11:27.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:11:27.265 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:27.264+0000 7f356cb43180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:11:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.361+0000 7f874bfff700 1 Processor -- start 2023-10-08T02:11:27.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.361+0000 7f874bfff700 1 -- start start 2023-10-08T02:11:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f874c0ad870 0x7f874c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f874c06bf20 0x7f874c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874bfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f874c0b40f0 con 0x7f874c142ea0 2023-10-08T02:11:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874bfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c06c3a0 con 0x7f874c0ad870 2023-10-08T02:11:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874bfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874c069640 con 0x7f874c06bf20 2023-10-08T02:11:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874a7fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f874c0ad870 0x7f874c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874a7fc700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f874c0ad870 0x7f874c06b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52804/0 (socket says 172.21.15.119:52804) 2023-10-08T02:11:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874a7fc700 1 -- 172.21.15.119:0/2734086274 learned_addr learned my addr 172.21.15.119:0/2734086274 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874b7fe700 1 --2- 172.21.15.119:0/2734086274 >> v2:172.21.15.184:3300/0 conn(0x7f874c06bf20 0x7f874c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874a7fc700 1 -- 172.21.15.119:0/2734086274 >> v2:172.21.15.184:3300/0 conn(0x7f874c06bf20 msgr2=0x7f874c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874a7fc700 1 --2- 172.21.15.119:0/2734086274 >> v2:172.21.15.184:3300/0 conn(0x7f874c06bf20 0x7f874c06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874a7fc700 1 -- 172.21.15.119:0/2734086274 >> v1:172.21.15.184:6789/0 conn(0x7f874c142ea0 legacy=0x7f874c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.363+0000 7f874a7fc700 1 -- 172.21.15.119:0/2734086274 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c069920 con 0x7f874c0ad870 2023-10-08T02:11:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.364+0000 7f874a7fc700 1 --2- 172.21.15.119:0/2734086274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c0ad870 0x7f874c06b9e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f874c071330 tx=0x7f8740009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1634f6be0592e55a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.364+0000 7f8749ffb700 1 -- 172.21.15.119:0/2734086274 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740009e70 con 0x7f874c0ad870 2023-10-08T02:11:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.364+0000 7f8749ffb700 1 -- 172.21.15.119:0/2734086274 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874000b860 con 0x7f874c0ad870 2023-10-08T02:11:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.364+0000 7f8749ffb700 1 -- 172.21.15.119:0/2734086274 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874000bc00 con 0x7f874c0ad870 2023-10-08T02:11:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.365+0000 7f874bfff700 1 -- 172.21.15.119:0/2734086274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c0ad870 msgr2=0x7f874c06b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.365+0000 7f874bfff700 1 --2- 172.21.15.119:0/2734086274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c0ad870 0x7f874c06b9e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f874c071330 tx=0x7f8740009250 comp rx=0 tx=0).stop 2023-10-08T02:11:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.365+0000 7f874bfff700 1 -- 172.21.15.119:0/2734086274 shutdown_connections 2023-10-08T02:11:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.365+0000 7f874bfff700 1 --2- 172.21.15.119:0/2734086274 >> v2:172.21.15.184:3300/0 conn(0x7f874c06bf20 0x7f874c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.365+0000 7f874bfff700 1 --2- 172.21.15.119:0/2734086274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c0ad870 0x7f874c06b9e0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.365+0000 7f874bfff700 1 -- 172.21.15.119:0/2734086274 >> 172.21.15.119:0/2734086274 conn(0x7f874c0ce430 msgr2=0x7f874c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.365+0000 7f874bfff700 1 -- 172.21.15.119:0/2734086274 shutdown_connections 2023-10-08T02:11:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.365+0000 7f874bfff700 1 -- 172.21.15.119:0/2734086274 wait complete. 2023-10-08T02:11:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.366+0000 7f874bfff700 1 Processor -- start 2023-10-08T02:11:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.366+0000 7f874bfff700 1 -- start start 2023-10-08T02:11:27.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.367+0000 7f874bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f874c06bf20 0x7f874c131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.367+0000 7f874bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c142ea0 0x7f874c15daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.367+0000 7f874bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f874c15dfe0 0x7f874c140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.367+0000 7f874bfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c13f650 con 0x7f874c142ea0 2023-10-08T02:11:27.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.367+0000 7f874bfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0ae1d0 con 0x7f874c15dfe0 2023-10-08T02:11:27.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.367+0000 7f874bfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874c0ae4b0 con 0x7f874c06bf20 2023-10-08T02:11:27.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c142ea0 0x7f874c15daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c142ea0 0x7f874c15daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52808/0 (socket says 172.21.15.119:52808) 2023-10-08T02:11:27.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874a7fc700 1 -- 172.21.15.119:0/3667841797 learned_addr learned my addr 172.21.15.119:0/3667841797 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:27.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874b7fe700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f874c15dfe0 0x7f874c140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874affd700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f874c06bf20 0x7f874c131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874b7fe700 1 -- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f874c06bf20 msgr2=0x7f874c131d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874b7fe700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f874c06bf20 0x7f874c131d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874b7fe700 1 -- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c142ea0 msgr2=0x7f874c15daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874b7fe700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c142ea0 0x7f874c15daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874b7fe700 1 -- 172.21.15.119:0/3667841797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400094f0 con 0x7f874c15dfe0 2023-10-08T02:11:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874a7fc700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c142ea0 0x7f874c15daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:11:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.368+0000 7f874b7fe700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f874c15dfe0 0x7f874c140d00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8744009e50 tx=0x7f874400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.369+0000 7f8732ffd700 1 -- 172.21.15.119:0/3667841797 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400caa0 con 0x7f874c15dfe0 2023-10-08T02:11:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.369+0000 7f8732ffd700 1 -- 172.21.15.119:0/3667841797 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874400f040 con 0x7f874c15dfe0 2023-10-08T02:11:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.369+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c0ae790 con 0x7f874c15dfe0 2023-10-08T02:11:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.369+0000 7f8732ffd700 1 -- 172.21.15.119:0/3667841797 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400ce50 con 0x7f874c15dfe0 2023-10-08T02:11:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.369+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c0aed50 con 0x7f874c15dfe0 2023-10-08T02:11:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.370+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c04c3c0 con 0x7f874c15dfe0 2023-10-08T02:11:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.371+0000 7f8732ffd700 1 -- 172.21.15.119:0/3667841797 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f8744014690 con 0x7f874c15dfe0 2023-10-08T02:11:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.372+0000 7f8732ffd700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8738067050 0x7f8738069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.372+0000 7f8732ffd700 1 -- 172.21.15.119:0/3667841797 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f8744085720 con 0x7f874c15dfe0 2023-10-08T02:11:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.372+0000 7f874affd700 1 -- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8738067050 msgr2=0x7f8738069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.372+0000 7f874affd700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8738067050 0x7f8738069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:27.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.376+0000 7f8732ffd700 1 -- 172.21.15.119:0/3667841797 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8744055e20 con 0x7f874c15dfe0 2023-10-08T02:11:27.380 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:27.380+0000 7f482f797180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:11:27.475 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:27.476+0000 7f482f797180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:11:27.570 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:27.569+0000 7f356cb43180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:11:27.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.573+0000 7f874affd700 1 -- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8738067050 msgr2=0x7f8738069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:27.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.573+0000 7f874affd700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8738067050 0x7f8738069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:27.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.621+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f874c069c00 con 0x7f874c15dfe0 2023-10-08T02:11:27.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.622+0000 7f8732ffd700 1 -- 172.21.15.119:0/3667841797 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f87440559b0 con 0x7f874c15dfe0 2023-10-08T02:11:27.623 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:27.623 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":63,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7549,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/26538394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":26538394},{"type":"v1","addr":"172.21.15.184:6834","nonce":26538394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3494509212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3494509212},{"type":"v1","addr":"172.21.15.119:6834","nonce":3494509212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2792091862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2792091862},{"type":"v1","addr":"172.21.15.119:6835","nonce":2792091862}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T01:15:21.218141+0000","modified":"2023-10-08T01:15:25.231364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8178},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8178":{"gid":8178,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/1485819493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1485819493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1485819493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-10-08T02:11:27.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8738067050 msgr2=0x7f8738069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:27.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8738067050 0x7f8738069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f874c15dfe0 msgr2=0x7f874c140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f874c15dfe0 0x7f874c140d00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8744009e50 tx=0x7f874400b040 comp rx=0 tx=0).stop 2023-10-08T02:11:27.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 shutdown_connections 2023-10-08T02:11:27.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8738067050 0x7f8738069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f874c06bf20 0x7f874c131d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f874c15dfe0 0x7f874c140d00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 --2- 172.21.15.119:0/3667841797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f874c142ea0 0x7f874c15daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 >> 172.21.15.119:0/3667841797 conn(0x7f874c0ce430 msgr2=0x7f874c15f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:27.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 shutdown_connections 2023-10-08T02:11:27.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.624+0000 7f874bfff700 1 -- 172.21.15.119:0/3667841797 wait complete. 2023-10-08T02:11:27.626 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 63 2023-10-08T02:11:27.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:11:27.670 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:27.670+0000 7f356cb43180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:11:27.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.828+0000 7fdd4f59e700 1 Processor -- start 2023-10-08T02:11:27.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.828+0000 7fdd4f59e700 1 -- start start 2023-10-08T02:11:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.829+0000 7fdd4f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd500ad870 0x7fdd5006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.829+0000 7fdd4f59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd5006bf20 0x7fdd5006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.829+0000 7fdd4f59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5006c350 con 0x7fdd50142ea0 2023-10-08T02:11:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.829+0000 7fdd4f59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd500b40f0 con 0x7fdd5006bf20 2023-10-08T02:11:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.829+0000 7fdd4f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50069640 con 0x7fdd500ad870 2023-10-08T02:11:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4ed9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd5006bf20 0x7fdd5006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4e59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd500ad870 0x7fdd5006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4e59c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd500ad870 0x7fdd5006b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:60858/0 (socket says 172.21.15.119:60858) 2023-10-08T02:11:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4e59c700 1 -- 172.21.15.119:0/4148321897 learned_addr learned my addr 172.21.15.119:0/4148321897 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:27.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4659c700 1 --1- 172.21.15.119:0/4148321897 >> v1:172.21.15.184:6789/0 conn(0x7fdd50142ea0 0x7fdd500b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:49700/0 (socket says 172.21.15.119:49700) 2023-10-08T02:11:27.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4ed9d700 1 -- 172.21.15.119:0/4148321897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd500ad870 msgr2=0x7fdd5006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4ed9d700 1 --2- 172.21.15.119:0/4148321897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd500ad870 0x7fdd5006b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4ed9d700 1 -- 172.21.15.119:0/4148321897 >> v1:172.21.15.184:6789/0 conn(0x7fdd50142ea0 legacy=0x7fdd500b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4ed9d700 1 -- 172.21.15.119:0/4148321897 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50069920 con 0x7fdd5006bf20 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.830+0000 7fdd4dd9b700 1 -- 172.21.15.119:0/4148321897 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852111374 0 0) 0x7fdd5006c350 con 0x7fdd50142ea0 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.831+0000 7fdd4ed9d700 1 --2- 172.21.15.119:0/4148321897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5006bf20 0x7fdd5006a120 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009e60 tx=0x7fdd48009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb9aea6880c0db8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.831+0000 7fdd4dd9b700 1 -- 172.21.15.119:0/4148321897 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4800e070 con 0x7fdd5006bf20 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.831+0000 7fdd4dd9b700 1 -- 172.21.15.119:0/4148321897 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4800bb60 con 0x7fdd5006bf20 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.831+0000 7fdd4dd9b700 1 -- 172.21.15.119:0/4148321897 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd48012430 con 0x7fdd5006bf20 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.831+0000 7fdd4f59e700 1 -- 172.21.15.119:0/4148321897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5006bf20 msgr2=0x7fdd5006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.831+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/4148321897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5006bf20 0x7fdd5006a120 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdd48009e60 tx=0x7fdd48009780 comp rx=0 tx=0).stop 2023-10-08T02:11:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.831+0000 7fdd4f59e700 1 -- 172.21.15.119:0/4148321897 shutdown_connections 2023-10-08T02:11:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.831+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/4148321897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5006bf20 0x7fdd5006a120 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.832+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/4148321897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd500ad870 0x7fdd5006b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.832+0000 7fdd4f59e700 1 -- 172.21.15.119:0/4148321897 >> 172.21.15.119:0/4148321897 conn(0x7fdd500ce430 msgr2=0x7fdd500ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.832+0000 7fdd4f59e700 1 -- 172.21.15.119:0/4148321897 shutdown_connections 2023-10-08T02:11:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.832+0000 7fdd4f59e700 1 -- 172.21.15.119:0/4148321897 wait complete. 2023-10-08T02:11:27.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.833+0000 7fdd4f59e700 1 Processor -- start 2023-10-08T02:11:27.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.833+0000 7fdd4f59e700 1 -- start start 2023-10-08T02:11:27.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5006bf20 0x7fdd500c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd50142ea0 0x7fdd5016dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd5015e5c0 0x7fdd500acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4f59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5015ed20 con 0x7fdd50142ea0 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5016e010 con 0x7fdd5006bf20 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5015a0c0 con 0x7fdd5015e5c0 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd50142ea0 0x7fdd5016dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd50142ea0 0x7fdd5016dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52826/0 (socket says 172.21.15.119:52826) 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4e59c700 1 -- 172.21.15.119:0/991517439 learned_addr learned my addr 172.21.15.119:0/991517439 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.834+0000 7fdd4659c700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5006bf20 0x7fdd500c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd4ed9d700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd5015e5c0 0x7fdd500acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd4e59c700 1 -- 172.21.15.119:0/991517439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd5015e5c0 msgr2=0x7fdd500acd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd4e59c700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd5015e5c0 0x7fdd500acd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd4e59c700 1 -- 172.21.15.119:0/991517439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5006bf20 msgr2=0x7fdd500c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd4e59c700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5006bf20 0x7fdd500c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd4e59c700 1 -- 172.21.15.119:0/991517439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009a20 con 0x7fdd50142ea0 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd4e59c700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd50142ea0 0x7fdd5016dad0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800ea50 tx=0x7fdd38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd46ffd700 1 -- 172.21.15.119:0/991517439 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800c640 con 0x7fdd50142ea0 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd46ffd700 1 -- 172.21.15.119:0/991517439 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3800c7a0 con 0x7fdd50142ea0 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5015ea40 con 0x7fdd50142ea0 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.835+0000 7fdd46ffd700 1 -- 172.21.15.119:0/991517439 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800cb20 con 0x7fdd50142ea0 2023-10-08T02:11:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.836+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5015a9f0 con 0x7fdd50142ea0 2023-10-08T02:11:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.837+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5004c3c0 con 0x7fdd50142ea0 2023-10-08T02:11:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.837+0000 7fdd46ffd700 1 -- 172.21.15.119:0/991517439 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fdd38013470 con 0x7fdd50142ea0 2023-10-08T02:11:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.839+0000 7fdd46ffd700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd30067050 0x7fdd30069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.839+0000 7fdd4659c700 1 -- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd30067050 msgr2=0x7fdd30069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.839+0000 7fdd46ffd700 1 -- 172.21.15.119:0/991517439 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6529+0+0 (secure 0 0 0) 0x7fdd38084c50 con 0x7fdd50142ea0 2023-10-08T02:11:27.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.839+0000 7fdd4659c700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd30067050 0x7fdd30069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:27.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:27.843+0000 7fdd46ffd700 1 -- 172.21.15.119:0/991517439 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdd380519d0 con 0x7fdd50142ea0 2023-10-08T02:11:28.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.040+0000 7fdd4659c700 1 -- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd30067050 msgr2=0x7fdd30069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:28.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.040+0000 7fdd4659c700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd30067050 0x7fdd30069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:28.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.043+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd50069c00 con 0x7fdd50142ea0 2023-10-08T02:11:28.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.044+0000 7fdd46ffd700 1 -- 172.21.15.119:0/991517439 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+16691 (secure 0 0 0) 0x7fdd38054ff0 con 0x7fdd50142ea0 2023-10-08T02:11:28.045 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:28.045 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":144,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:11:26.160075+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-10-08T01:15:19.321580+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-10-08T01:15:20.200200+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":144,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1a","mappings":[{"from":0,"to":2}]},{"pgid":"17.3c","mappings":[{"from":0,"to":5}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1681557219":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/1844314331":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/3405191039":"2023-10-09T01:15:42.377210+0000","172.21.15.119:0/3624517726":"2023-10-09T01:15:42.377210+0000","172.21.15.119:6837/85511":"2023-10-09T01:15:42.377210+0000","172.21.15.119:6836/85511":"2023-10-09T01:15:42.377210+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:11:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd30067050 msgr2=0x7fdd30069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd30067050 0x7fdd30069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd50142ea0 msgr2=0x7fdd5016dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd50142ea0 0x7fdd5016dad0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800ea50 tx=0x7fdd38009250 comp rx=0 tx=0).stop 2023-10-08T02:11:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 shutdown_connections 2023-10-08T02:11:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdd30067050 0x7fdd30069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdd5015e5c0 0x7fdd500acd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdd5006bf20 0x7fdd500c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 --2- 172.21.15.119:0/991517439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd50142ea0 0x7fdd5016dad0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 >> 172.21.15.119:0/991517439 conn(0x7fdd500ce430 msgr2=0x7fdd500b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 shutdown_connections 2023-10-08T02:11:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.046+0000 7fdd4f59e700 1 -- 172.21.15.119:0/991517439 wait complete. 2023-10-08T02:11:28.066 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T02:11:28.177 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:28.177+0000 7f482f797180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:11:28.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.294+0000 7f861199d700 1 Processor -- start 2023-10-08T02:11:28.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.295+0000 7f861199d700 1 -- start start 2023-10-08T02:11:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f861199d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f861199d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860c0b2980 con 0x7f860c1597f0 2023-10-08T02:11:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f861199d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f860c15dbb0 con 0x7f860c159420 2023-10-08T02:11:28.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f861199d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0b11d0 con 0x7f860c15d230 2023-10-08T02:11:28.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f860bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f860c1597f0 0x7f860c0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47390/0 (socket says 172.21.15.119:47390) 2023-10-08T02:11:28.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f860bfff700 1 -- 172.21.15.119:0/861757356 learned_addr learned my addr 172.21.15.119:0/861757356 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:28.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f861119c700 1 --2- 172.21.15.119:0/861757356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f860b7fe700 1 -- 172.21.15.119:0/861757356 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723071040 0 0) 0x7f860c0b2980 con 0x7f860c1597f0 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f860b7fe700 1 -- 172.21.15.119:0/861757356 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85fc003570 con 0x7f860c1597f0 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f860b7fe700 1 -- 172.21.15.119:0/861757356 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 971744515 0 0) 0x7f860c15dbb0 con 0x7f860c159420 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f860b7fe700 1 -- 172.21.15.119:0/861757356 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f860c0b2980 con 0x7f860c159420 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f861119c700 1 -- 172.21.15.119:0/861757356 >> v1:172.21.15.184:6789/0 conn(0x7f860c159420 legacy=0x7f860c0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.297+0000 7f861119c700 1 -- 172.21.15.119:0/861757356 >> v1:172.21.15.119:6789/0 conn(0x7f860c1597f0 legacy=0x7f860c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.298+0000 7f861119c700 1 -- 172.21.15.119:0/861757356 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f860c0b14b0 con 0x7f860c15d230 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.298+0000 7f861119c700 1 --2- 172.21.15.119:0/861757356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c15d670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8604009e90 tx=0x7f8604009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=179ed9439d18b1e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.298+0000 7f860b7fe700 1 -- 172.21.15.119:0/861757356 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f860400e070 con 0x7f860c15d230 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.298+0000 7f860b7fe700 1 -- 172.21.15.119:0/861757356 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f860400bb80 con 0x7f860c15d230 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.298+0000 7f860b7fe700 1 -- 172.21.15.119:0/861757356 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8604012430 con 0x7f860c15d230 2023-10-08T02:11:28.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.299+0000 7f861199d700 1 -- 172.21.15.119:0/861757356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 msgr2=0x7f860c15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.299+0000 7f861199d700 1 --2- 172.21.15.119:0/861757356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c15d670 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8604009e90 tx=0x7f8604009780 comp rx=0 tx=0).stop 2023-10-08T02:11:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.299+0000 7f861199d700 1 -- 172.21.15.119:0/861757356 shutdown_connections 2023-10-08T02:11:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.299+0000 7f861199d700 1 --2- 172.21.15.119:0/861757356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c15d670 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.299+0000 7f861199d700 1 -- 172.21.15.119:0/861757356 >> 172.21.15.119:0/861757356 conn(0x7f860c0d0660 msgr2=0x7f860c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.299+0000 7f861199d700 1 -- 172.21.15.119:0/861757356 shutdown_connections 2023-10-08T02:11:28.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.299+0000 7f861199d700 1 -- 172.21.15.119:0/861757356 wait complete. 2023-10-08T02:11:28.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.300+0000 7f861199d700 1 Processor -- start 2023-10-08T02:11:28.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.300+0000 7f861199d700 1 -- start start 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.301+0000 7f861199d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f860c159420 0x7f860c14f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.301+0000 7f861199d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f860c1597f0 0x7f860c0b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.301+0000 7f861199d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.301+0000 7f861199d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0b5770 con 0x7f860c1597f0 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.301+0000 7f861199d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f860c0b4db0 con 0x7f860c159420 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.301+0000 7f861199d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860c150250 con 0x7f860c15d230 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f860c1597f0 0x7f860c0b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f860c1597f0 0x7f860c0b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52838/0 (socket says 172.21.15.119:52838) 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 -- 172.21.15.119:0/2267410826 learned_addr learned my addr 172.21.15.119:0/2267410826 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f861119c700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f861099b700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f860c159420 0x7f860c14f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 -- 172.21.15.119:0/2267410826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 msgr2=0x7f860c154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 -- 172.21.15.119:0/2267410826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f860c159420 msgr2=0x7f860c14f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f860c159420 0x7f860c14f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 -- 172.21.15.119:0/2267410826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8604009a20 con 0x7f860c1597f0 2023-10-08T02:11:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.302+0000 7f860bfff700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f860c1597f0 0x7f860c0b5bb0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c790 tx=0x7f85f8007e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.303+0000 7f8608ff9700 1 -- 172.21.15.119:0/2267410826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8005b30 con 0x7f860c1597f0 2023-10-08T02:11:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.303+0000 7f8608ff9700 1 -- 172.21.15.119:0/2267410826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f8005c90 con 0x7f860c1597f0 2023-10-08T02:11:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.303+0000 7f8608ff9700 1 -- 172.21.15.119:0/2267410826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8007320 con 0x7f860c1597f0 2023-10-08T02:11:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.303+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f860c0b4230 con 0x7f860c1597f0 2023-10-08T02:11:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.303+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860c0b47f0 con 0x7f860c1597f0 2023-10-08T02:11:28.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.304+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f860c04c3c0 con 0x7f860c1597f0 2023-10-08T02:11:28.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.305+0000 7f8608ff9700 1 -- 172.21.15.119:0/2267410826 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f85f801b3b0 con 0x7f860c1597f0 2023-10-08T02:11:28.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.306+0000 7f8608ff9700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f0067050 0x7f85f0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.306+0000 7f8608ff9700 1 -- 172.21.15.119:0/2267410826 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6529+0+0 (secure 0 0 0) 0x7f85f8083f70 con 0x7f860c1597f0 2023-10-08T02:11:28.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.306+0000 7f861099b700 1 -- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f0067050 msgr2=0x7f85f0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:28.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.306+0000 7f861099b700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f0067050 0x7f85f0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:28.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.310+0000 7f8608ff9700 1 -- 172.21.15.119:0/2267410826 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85f8050cf0 con 0x7f860c1597f0 2023-10-08T02:11:28.416 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:28.415+0000 7f356cb43180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:11:28.465 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:28.465+0000 7f482f797180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:11:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.507+0000 7f861099b700 1 -- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f0067050 msgr2=0x7f85f0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:28.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.507+0000 7f861099b700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f0067050 0x7f85f0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:28.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.552+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f860c0b6470 con 0x7f860c1597f0 2023-10-08T02:11:28.553 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T02:11:28.552+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T02:11:28.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.557+0000 7f8608ff9700 1 -- 172.21.15.119:0/2267410826 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) v1 ==== 161+0+0 (secure 0 0 0) 0x7f85f8017020 con 0x7f860c1597f0 2023-10-08T02:11:28.559 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:28.559+0000 7f482f797180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f0067050 msgr2=0x7f85f0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f0067050 0x7f85f0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f860c1597f0 msgr2=0x7f860c0b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f860c1597f0 0x7f860c0b5bb0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f85f800c790 tx=0x7f85f8007e20 comp rx=0 tx=0).stop 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 shutdown_connections 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85f0067050 0x7f85f0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f860c15d230 0x7f860c154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f860c159420 0x7f860c14f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 --2- 172.21.15.119:0/2267410826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f860c1597f0 0x7f860c0b5bb0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 >> 172.21.15.119:0/2267410826 conn(0x7f860c0d0660 msgr2=0x7f860c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:28.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 shutdown_connections 2023-10-08T02:11:28.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.560+0000 7f861199d700 1 -- 172.21.15.119:0/2267410826 wait complete. 2023-10-08T02:11:28.562 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T02:11:28.579 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T02:11:28.630 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T02:11:28.712 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:28.711+0000 7f356cb43180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:11:28.728 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:28.728+0000 7f482f797180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:11:28.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.768+0000 7f076af9d700 1 Processor -- start 2023-10-08T02:11:28.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.768+0000 7f076af9d700 1 -- start start 2023-10-08T02:11:28.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.769+0000 7f076af9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f07640b4e70 0x7f07640ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.769+0000 7f076af9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b1b40 0x7f076415d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.769+0000 7f076af9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07640cfb00 con 0x7f07640b16d0 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.769+0000 7f076af9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640b5c40 con 0x7f07640b4e70 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.769+0000 7f076af9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07640b5f20 con 0x7f07640b1b40 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f0769f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f07640b4e70 0x7f07640ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f076979a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f07640b16d0 0x7f076415d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47398/0 (socket says 172.21.15.119:47398) 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f076979a700 1 -- 172.21.15.119:0/1476013333 learned_addr learned my addr 172.21.15.119:0/1476013333 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f076a79c700 1 --2- 172.21.15.119:0/1476013333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b1b40 0x7f076415d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f0769f9b700 1 -- 172.21.15.119:0/1476013333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b1b40 msgr2=0x7f076415d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f0769f9b700 1 --2- 172.21.15.119:0/1476013333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b1b40 0x7f076415d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f0769f9b700 1 -- 172.21.15.119:0/1476013333 >> v1:172.21.15.119:6789/0 conn(0x7f07640b16d0 legacy=0x7f076415d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f0769f9b700 1 -- 172.21.15.119:0/1476013333 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07640b6200 con 0x7f07640b4e70 2023-10-08T02:11:28.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f0768f99700 1 -- 172.21.15.119:0/1476013333 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577778524 0 0) 0x7f07640cfb00 con 0x7f07640b16d0 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.770+0000 7f0769f9b700 1 --2- 172.21.15.119:0/1476013333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b4e70 0x7f07640ccff0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f07600098e0 tx=0x7f07600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572d3f2bda6926bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f0768f99700 1 -- 172.21.15.119:0/1476013333 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000b640 con 0x7f07640b4e70 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f0768f99700 1 -- 172.21.15.119:0/1476013333 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076000b7a0 con 0x7f07640b4e70 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f0768f99700 1 -- 172.21.15.119:0/1476013333 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000bb40 con 0x7f07640b4e70 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f076af9d700 1 -- 172.21.15.119:0/1476013333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b4e70 msgr2=0x7f07640ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f076af9d700 1 --2- 172.21.15.119:0/1476013333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b4e70 0x7f07640ccff0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f07600098e0 tx=0x7f07600091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f076af9d700 1 -- 172.21.15.119:0/1476013333 shutdown_connections 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f076af9d700 1 --2- 172.21.15.119:0/1476013333 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b1b40 0x7f076415d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f076af9d700 1 --2- 172.21.15.119:0/1476013333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b4e70 0x7f07640ccff0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f076af9d700 1 -- 172.21.15.119:0/1476013333 >> 172.21.15.119:0/1476013333 conn(0x7f07640d10c0 msgr2=0x7f07640affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f076af9d700 1 -- 172.21.15.119:0/1476013333 shutdown_connections 2023-10-08T02:11:28.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.771+0000 7f076af9d700 1 -- 172.21.15.119:0/1476013333 wait complete. 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.773+0000 7f076af9d700 1 Processor -- start 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.773+0000 7f076af9d700 1 -- start start 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f076af9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b16d0 0x7f076414b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f076af9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640b1b40 0x7f07641699d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f076af9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b4e70 0x7f07640c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f076af9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640c9380 con 0x7f07640b16d0 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f076af9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640cacd0 con 0x7f07640b1b40 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f076af9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07640c8dc0 con 0x7f07640b4e70 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b16d0 0x7f076414b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f0769f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b16d0 0x7f076414b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52850/0 (socket says 172.21.15.119:52850) 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f0769f9b700 1 -- 172.21.15.119:0/1672490135 learned_addr learned my addr 172.21.15.119:0/1672490135 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f076979a700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640b1b40 0x7f07641699d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.774+0000 7f076a79c700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b4e70 0x7f07640c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0769f9b700 1 -- 172.21.15.119:0/1672490135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b4e70 msgr2=0x7f07640c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0769f9b700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b4e70 0x7f07640c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0769f9b700 1 -- 172.21.15.119:0/1672490135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640b1b40 msgr2=0x7f07641699d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0769f9b700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640b1b40 0x7f07641699d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0769f9b700 1 -- 172.21.15.119:0/1672490135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0760009420 con 0x7f07640b16d0 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0769f9b700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b16d0 0x7f076414b8f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f0764070dd0 tx=0x7f0760017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0759ffb700 1 -- 172.21.15.119:0/1672490135 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000bb20 con 0x7f07640b16d0 2023-10-08T02:11:28.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0759ffb700 1 -- 172.21.15.119:0/1672490135 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0760000c90 con 0x7f07640b16d0 2023-10-08T02:11:28.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07640c90a0 con 0x7f07640b16d0 2023-10-08T02:11:28.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f0759ffb700 1 -- 172.21.15.119:0/1672490135 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760011630 con 0x7f07640b16d0 2023-10-08T02:11:28.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.775+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07640ca150 con 0x7f07640b16d0 2023-10-08T02:11:28.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.776+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0764088990 con 0x7f07640b16d0 2023-10-08T02:11:28.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.778+0000 7f0759ffb700 1 -- 172.21.15.119:0/1672490135 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 75) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f07600119a0 con 0x7f07640b16d0 2023-10-08T02:11:28.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.779+0000 7f0759ffb700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 0x7f0748069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:28.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.779+0000 7f0759ffb700 1 -- 172.21.15.119:0/1672490135 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6571+0+0 (secure 0 0 0) 0x7f0760085dd0 con 0x7f07640b16d0 2023-10-08T02:11:28.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.779+0000 7f076979a700 1 -- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 msgr2=0x7f0748069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:28.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.779+0000 7f076979a700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 0x7f0748069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:28.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.783+0000 7f0759ffb700 1 -- 172.21.15.119:0/1672490135 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0760052b30 con 0x7f07640b16d0 2023-10-08T02:11:28.815 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:28.814+0000 7f356cb43180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:11:28.831 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:28.832+0000 7f482f797180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:11:28.975 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:28.974+0000 7f356cb43180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:11:28.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.980+0000 7f076979a700 1 -- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 msgr2=0x7f0748069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:28.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:28.980+0000 7f076979a700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 0x7f0748069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:29.020 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:29.020+0000 7f482f797180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:11:29.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.035+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f07640cab20 con 0x7f07640b16d0 2023-10-08T02:11:29.085 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:29.084+0000 7f356cb43180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:11:29.115 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:29.115+0000 7f482f797180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:11:29.209 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:29.209+0000 7f482f797180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:11:29.277 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:29.277+0000 7f356cb43180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:11:29.303 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:29.303+0000 7f482f797180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:11:29.371 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:29.370+0000 7f356cb43180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:11:29.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.381+0000 7f076979a700 1 -- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 msgr2=0x7f0748069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:29.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.381+0000 7f076979a700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 0x7f0748069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:11:29.466 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:29.465+0000 7f356cb43180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:11:29.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.559+0000 7f0759ffb700 1 -- 172.21.15.119:0/1672490135 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v65) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0760056150 con 0x7f07640b16d0 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 msgr2=0x7f0748069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 0x7f0748069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b16d0 msgr2=0x7f076414b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b16d0 0x7f076414b8f0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f0764070dd0 tx=0x7f0760017000 comp rx=0 tx=0).stop 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 shutdown_connections 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0748067050 0x7f0748069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f07640b4e70 0x7f07640c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f07640b16d0 0x7f076414b8f0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 --2- 172.21.15.119:0/1672490135 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f07640b1b40 0x7f07641699d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 >> 172.21.15.119:0/1672490135 conn(0x7f07640d10c0 msgr2=0x7f07640b3390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 shutdown_connections 2023-10-08T02:11:29.563 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f356cb43180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:11:29.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.561+0000 7f076af9d700 1 -- 172.21.15.119:0/1672490135 wait complete. 2023-10-08T02:11:29.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T02:11:29.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.778+0000 7f48069ff700 1 Processor -- start 2023-10-08T02:11:29.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.778+0000 7f48069ff700 1 -- start start 2023-10-08T02:11:29.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f48069ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f48000c6ac0 0x7f4800068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:29.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f48069ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4800069a50 0x7f4800156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:29.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f48069ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48000ac030 con 0x7f48000c8490 2023-10-08T02:11:29.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f48069ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48000ab820 con 0x7f4800069a50 2023-10-08T02:11:29.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f48069ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480006dbb0 con 0x7f48000c6ac0 2023-10-08T02:11:29.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f4804f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4800069a50 0x7f4800156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:29.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f47fffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f48000c8490 0x7f480015a490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47404/0 (socket says 172.21.15.119:47404) 2023-10-08T02:11:29.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f47fffff700 1 -- 172.21.15.119:0/3012068880 learned_addr learned my addr 172.21.15.119:0/3012068880 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:29.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f47ff7fe700 1 --2- 172.21.15.119:0/3012068880 >> v2:172.21.15.184:3300/0 conn(0x7f48000c6ac0 0x7f4800068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:29.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f4804f9c700 1 -- 172.21.15.119:0/3012068880 >> v2:172.21.15.184:3300/0 conn(0x7f48000c6ac0 msgr2=0x7f4800068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:29.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f4804f9c700 1 --2- 172.21.15.119:0/3012068880 >> v2:172.21.15.184:3300/0 conn(0x7f48000c6ac0 0x7f4800068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f4804f9c700 1 -- 172.21.15.119:0/3012068880 >> v1:172.21.15.119:6789/0 conn(0x7f48000c8490 legacy=0x7f480015a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:29.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f4804f9c700 1 -- 172.21.15.119:0/3012068880 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48000c6f10 con 0x7f4800069a50 2023-10-08T02:11:29.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.780+0000 7f47feffd700 1 -- 172.21.15.119:0/3012068880 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649389624 0 0) 0x7f47f0001110 con 0x7f48000c8490 2023-10-08T02:11:29.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.781+0000 7f4804f9c700 1 --2- 172.21.15.119:0/3012068880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4800069a50 0x7f4800156ab0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f480006b920 tx=0x7f47f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4e811802c0ce977 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:29.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.781+0000 7f47feffd700 1 -- 172.21.15.119:0/3012068880 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f800e070 con 0x7f4800069a50 2023-10-08T02:11:29.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.781+0000 7f47feffd700 1 -- 172.21.15.119:0/3012068880 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f800bb60 con 0x7f4800069a50 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.781+0000 7f47feffd700 1 -- 172.21.15.119:0/3012068880 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8012430 con 0x7f4800069a50 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.781+0000 7f48069ff700 1 -- 172.21.15.119:0/3012068880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4800069a50 msgr2=0x7f4800156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.781+0000 7f48069ff700 1 --2- 172.21.15.119:0/3012068880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4800069a50 0x7f4800156ab0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f480006b920 tx=0x7f47f8009780 comp rx=0 tx=0).stop 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.782+0000 7f48069ff700 1 -- 172.21.15.119:0/3012068880 shutdown_connections 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.782+0000 7f48069ff700 1 --2- 172.21.15.119:0/3012068880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4800069a50 0x7f4800156ab0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.782+0000 7f48069ff700 1 --2- 172.21.15.119:0/3012068880 >> v2:172.21.15.184:3300/0 conn(0x7f48000c6ac0 0x7f4800068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.782+0000 7f48069ff700 1 -- 172.21.15.119:0/3012068880 >> 172.21.15.119:0/3012068880 conn(0x7f48000c8f90 msgr2=0x7f48000c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.782+0000 7f48069ff700 1 -- 172.21.15.119:0/3012068880 shutdown_connections 2023-10-08T02:11:29.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.782+0000 7f48069ff700 1 -- 172.21.15.119:0/3012068880 wait complete. 2023-10-08T02:11:29.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.783+0000 7f48069ff700 1 Processor -- start 2023-10-08T02:11:29.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.783+0000 7f48069ff700 1 -- start start 2023-10-08T02:11:29.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f48069ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4800069a50 0x7f48000aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:29.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f48069ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48000c6ac0 0x7f48000ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:29.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f48069ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48000c8490 0x7f4800168f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:29.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f48069ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48000b02a0 con 0x7f48000c8490 2023-10-08T02:11:29.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f48069ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48000b0860 con 0x7f48000c6ac0 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f48069ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48000aade0 con 0x7f4800069a50 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f4804f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48000c8490 0x7f4800168f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f4804f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48000c8490 0x7f4800168f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52948/0 (socket says 172.21.15.119:52948) 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f4804f9c700 1 -- 172.21.15.119:0/551464394 learned_addr learned my addr 172.21.15.119:0/551464394 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f47ff7fe700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48000c6ac0 0x7f48000ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.784+0000 7f47fffff700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4800069a50 0x7f48000aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f4804f9c700 1 -- 172.21.15.119:0/551464394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4800069a50 msgr2=0x7f48000aa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f4804f9c700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4800069a50 0x7f48000aa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f4804f9c700 1 -- 172.21.15.119:0/551464394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48000c6ac0 msgr2=0x7f48000ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f4804f9c700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48000c6ac0 0x7f48000ac840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:29.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f4804f9c700 1 -- 172.21.15.119:0/551464394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f8009a20 con 0x7f48000c8490 2023-10-08T02:11:29.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f4804f9c700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48000c8490 0x7f4800168f50 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f47f80099f0 tx=0x7f47f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:29.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f47e7fff700 1 -- 172.21.15.119:0/551464394 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8027470 con 0x7f48000c8490 2023-10-08T02:11:29.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f47e7fff700 1 -- 172.21.15.119:0/551464394 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f8027610 con 0x7f48000c8490 2023-10-08T02:11:29.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f47e7fff700 1 -- 172.21.15.119:0/551464394 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f80279d0 con 0x7f48000c8490 2023-10-08T02:11:29.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48000ab0c0 con 0x7f48000c8490 2023-10-08T02:11:29.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.785+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480006ea80 con 0x7f48000c8490 2023-10-08T02:11:29.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.786+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48000718b0 con 0x7f48000c8490 2023-10-08T02:11:29.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.787+0000 7f47e7fff700 1 -- 172.21.15.119:0/551464394 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 76) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f47f8024070 con 0x7f48000c8490 2023-10-08T02:11:29.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.789+0000 7f47e7fff700 1 -- 172.21.15.119:0/551464394 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6829+0+0 (secure 0 0 0) 0x7f47f8086b90 con 0x7f48000c8490 2023-10-08T02:11:29.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.793+0000 7f47e7fff700 1 -- 172.21.15.119:0/551464394 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47f8053c70 con 0x7f48000c8490 2023-10-08T02:11:29.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:29.985+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f48000aba70 con 0x7f48000c8490 2023-10-08T02:11:30.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.572+0000 7f47e7fff700 1 -- 172.21.15.119:0/551464394 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) v1 ==== 181+0+0 (secure 0 0 0) 0x7f47f8057290 con 0x7f48000c8490 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48000c8490 msgr2=0x7f4800168f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48000c8490 0x7f4800168f50 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f47f80099f0 tx=0x7f47f8019000 comp rx=0 tx=0).stop 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 shutdown_connections 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4800069a50 0x7f48000aa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48000c8490 0x7f4800168f50 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 --2- 172.21.15.119:0/551464394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f48000c6ac0 0x7f48000ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 >> 172.21.15.119:0/551464394 conn(0x7f48000c8f90 msgr2=0x7f4800159620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 shutdown_connections 2023-10-08T02:11:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.577+0000 7f48069ff700 1 -- 172.21.15.119:0/551464394 wait complete. 2023-10-08T02:11:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T02:11:30.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T02:11:30.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.791+0000 7f01bd99d700 1 Processor -- start 2023-10-08T02:11:30.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.791+0000 7f01bd99d700 1 -- start start 2023-10-08T02:11:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.792+0000 7f01bd99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f01b80ccd70 0x7f01b8161a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.792+0000 7f01bd99d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b815a480 con 0x7f01b80ae750 2023-10-08T02:11:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.792+0000 7f01bd99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b80afb40 con 0x7f01b80cc8d0 2023-10-08T02:11:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.792+0000 7f01bd99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b8153b90 con 0x7f01b80ccd70 2023-10-08T02:11:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01bc99b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f01b80ae750 0x7f01b8153480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47414/0 (socket says 172.21.15.119:47414) 2023-10-08T02:11:30.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01bc99b700 1 -- 172.21.15.119:0/1907113765 learned_addr learned my addr 172.21.15.119:0/1907113765 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:30.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01bd19c700 1 --2- 172.21.15.119:0/1907113765 >> v2:172.21.15.184:3300/0 conn(0x7f01b80ccd70 0x7f01b8161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:30.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961818428 0 0) 0x7f01b815a480 con 0x7f01b80ae750 2023-10-08T02:11:30.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01a8003570 con 0x7f01b80ae750 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495819084 0 0) 0x7f01b80afb40 con 0x7f01b80cc8d0 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b815a480 con 0x7f01b80cc8d0 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3328040581 0 0) 0x7f01a8003570 con 0x7f01b80ae750 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 >> v2:172.21.15.184:3300/0 conn(0x7f01b80ccd70 msgr2=0x7f01b8161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 --2- 172.21.15.119:0/1907113765 >> v2:172.21.15.184:3300/0 conn(0x7f01b80ccd70 0x7f01b8161a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 >> v1:172.21.15.184:6789/0 conn(0x7f01b80cc8d0 legacy=0x7f01b81596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b80afe20 con 0x7f01b80ae750 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f01ac001470 con 0x7f01b80ae750 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 >> v1:172.21.15.119:6789/0 conn(0x7f01b80ae750 legacy=0x7f01b8153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8003c40 0x7f01a8004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a8004810 0x7f01a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:30.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a8005210 0x7f01a800dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b8153b90 con 0x7f01a8005210 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a800e340 con 0x7f01a8004810 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.793+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01a800e4d0 con 0x7f01a8003c40 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bd19c700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a8005210 0x7f01a800dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bc99b700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8003c40 0x7f01a8004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01b7fff700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a8004810 0x7f01a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bd19c700 1 -- 172.21.15.119:0/1907113765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8003c40 msgr2=0x7f01a8004030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bd19c700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8003c40 0x7f01a8004030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bd19c700 1 -- 172.21.15.119:0/1907113765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a8004810 msgr2=0x7f01a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bd19c700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01a8004810 0x7f01a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bd19c700 1 -- 172.21.15.119:0/1907113765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a8003570 con 0x7f01a8005210 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bd19c700 1 -- 172.21.15.119:0/1907113765 reap_dead start 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.794+0000 7f01bd19c700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a8005210 0x7f01a800dc70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f01b000e240 tx=0x7f01b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0838c8bf6f558c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0009ac0 con 0x7f01a8005210 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b000f040 con 0x7f01a8005210 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01b77fe700 1 -- 172.21.15.119:0/1907113765 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b000ed40 con 0x7f01a8005210 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01bd99d700 1 -- 172.21.15.119:0/1907113765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a8005210 msgr2=0x7f01a800dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01bd99d700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a8005210 0x7f01a800dc70 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f01b000e240 tx=0x7f01b000b040 comp rx=0 tx=0).stop 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01bd99d700 1 -- 172.21.15.119:0/1907113765 shutdown_connections 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01bd99d700 1 --2- 172.21.15.119:0/1907113765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01a8005210 0x7f01a800dc70 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01bd99d700 1 -- 172.21.15.119:0/1907113765 >> 172.21.15.119:0/1907113765 conn(0x7f01b8068e20 msgr2=0x7f01b80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:30.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01bd99d700 1 -- 172.21.15.119:0/1907113765 shutdown_connections 2023-10-08T02:11:30.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.795+0000 7f01bd99d700 1 -- 172.21.15.119:0/1907113765 wait complete. 2023-10-08T02:11:30.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.797+0000 7f01bd99d700 1 Processor -- start 2023-10-08T02:11:30.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.797+0000 7f01bd99d700 1 -- start start 2023-10-08T02:11:30.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bd99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8005210 0x7f01b8153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:30.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bd99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b816cfb0 0x7f01b81561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:30.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bd99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b8169a30 0x7f01b8169eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:30.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bd99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b8154dd0 con 0x7f01b816cfb0 2023-10-08T02:11:30.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bd99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b81535c0 con 0x7f01b8169a30 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bd99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b8156700 con 0x7f01a8005210 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b816cfb0 0x7f01b81561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b816cfb0 0x7f01b81561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52968/0 (socket says 172.21.15.119:52968) 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 -- 172.21.15.119:0/712483430 learned_addr learned my addr 172.21.15.119:0/712483430 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bc99b700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8005210 0x7f01b8153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bd19c700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b8169a30 0x7f01b8169eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 -- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8005210 msgr2=0x7f01b8153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8005210 0x7f01b8153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 -- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b8169a30 msgr2=0x7f01b8169eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b8169a30 0x7f01b8169eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 -- 172.21.15.119:0/712483430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b816d570 con 0x7f01b816cfb0 2023-10-08T02:11:30.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01bd19c700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b8169a30 0x7f01b8169eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.798+0000 7f01b7fff700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b816cfb0 0x7f01b81561c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f01a4002b30 tx=0x7f01a4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.799+0000 7f01b4ff9700 1 -- 172.21.15.119:0/712483430 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400f940 con 0x7f01b816cfb0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.799+0000 7f01b4ff9700 1 -- 172.21.15.119:0/712483430 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a400faa0 con 0x7f01b816cfb0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.799+0000 7f01b4ff9700 1 -- 172.21.15.119:0/712483430 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a400fe60 con 0x7f01b816cfb0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.799+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b816d850 con 0x7f01b816cfb0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.799+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b8155220 con 0x7f01b816cfb0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.800+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b8154f40 con 0x7f01b816cfb0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.801+0000 7f01b4ff9700 1 -- 172.21.15.119:0/712483430 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 77) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f01a4006a30 con 0x7f01b816cfb0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.801+0000 7f01b4ff9700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f019c066d80 0x7f019c069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.801+0000 7f01b4ff9700 1 -- 172.21.15.119:0/712483430 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6391+0+0 (secure 0 0 0) 0x7f01a4084880 con 0x7f01b816cfb0 2023-10-08T02:11:30.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.802+0000 7f01bc99b700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f019c066d80 0x7f019c069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:30.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.802+0000 7f01bc99b700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f019c066d80 0x7f019c069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f01b80c8710 tx=0x7f01b815a610 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:30.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.806+0000 7f01b4ff9700 1 -- 172.21.15.119:0/712483430 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01b8154f40 con 0x7f01b816cfb0 2023-10-08T02:11:30.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:30.996+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01b8154f10 con 0x7f01b816cfb0 2023-10-08T02:11:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.598+0000 7f01b4ff9700 1 -- 172.21.15.119:0/712483430 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01a4086ce0 con 0x7f01b816cfb0 2023-10-08T02:11:31.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.599+0000 7f01b4ff9700 1 -- 172.21.15.119:0/712483430 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) v1 ==== 169+0+0 (secure 0 0 0) 0x7f01a404b8d0 con 0x7f01b816cfb0 2023-10-08T02:11:31.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f019c066d80 msgr2=0x7f019c069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:31.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f019c066d80 0x7f019c069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f01b80c8710 tx=0x7f01b815a610 comp rx=0 tx=0).stop 2023-10-08T02:11:31.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b816cfb0 msgr2=0x7f01b81561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:31.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b816cfb0 0x7f01b81561c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f01a4002b30 tx=0x7f01a4007d30 comp rx=0 tx=0).stop 2023-10-08T02:11:31.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 shutdown_connections 2023-10-08T02:11:31.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f019c066d80 0x7f019c069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f01a8005210 0x7f01b8153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f01b8169a30 0x7f01b8169eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 --2- 172.21.15.119:0/712483430 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b816cfb0 0x7f01b81561c0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 >> 172.21.15.119:0/712483430 conn(0x7f01b8068e20 msgr2=0x7f01b80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:31.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 shutdown_connections 2023-10-08T02:11:31.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.602+0000 7f01bd99d700 1 -- 172.21.15.119:0/712483430 wait complete. 2023-10-08T02:11:31.604 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T02:11:31.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T02:11:31.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.810+0000 7f365e5ff700 1 Processor -- start 2023-10-08T02:11:31.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.811+0000 7f365e5ff700 1 -- start start 2023-10-08T02:11:31.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.812+0000 7f365e5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f36580c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:31.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.812+0000 7f365e5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f36580ccee0 0x7f36581561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:31.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.812+0000 7f365e5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f365815b0d0 con 0x7f36580c87c0 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.812+0000 7f365e5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36580c7a60 con 0x7f36580ccee0 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.812+0000 7f365e5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365815a4c0 con 0x7f36580c83f0 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.812+0000 7f365cb9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f36580c87c0 0x7f365806eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47420/0 (socket says 172.21.15.119:47420) 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.812+0000 7f365cb9c700 1 -- 172.21.15.119:0/2922173087 learned_addr learned my addr 172.21.15.119:0/2922173087 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.812+0000 7f3657fff700 1 --2- 172.21.15.119:0/2922173087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f36580c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.813+0000 7f364ffff700 1 --2- 172.21.15.119:0/2922173087 >> v2:172.21.15.184:3300/0 conn(0x7f36580ccee0 0x7f36581561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.813+0000 7f36577fe700 1 -- 172.21.15.119:0/2922173087 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829561923 0 0) 0x7f365815b0d0 con 0x7f36580c87c0 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.813+0000 7f36577fe700 1 -- 172.21.15.119:0/2922173087 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3644003570 con 0x7f36580c87c0 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.813+0000 7f3657fff700 1 -- 172.21.15.119:0/2922173087 >> v2:172.21.15.184:3300/0 conn(0x7f36580ccee0 msgr2=0x7f36581561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.813+0000 7f3657fff700 1 --2- 172.21.15.119:0/2922173087 >> v2:172.21.15.184:3300/0 conn(0x7f36580ccee0 0x7f36581561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.813+0000 7f3657fff700 1 -- 172.21.15.119:0/2922173087 >> v1:172.21.15.119:6789/0 conn(0x7f36580c87c0 legacy=0x7f365806eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:31.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.813+0000 7f3657fff700 1 -- 172.21.15.119:0/2922173087 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365815a7a0 con 0x7f36580c83f0 2023-10-08T02:11:31.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.813+0000 7f3657fff700 1 --2- 172.21.15.119:0/2922173087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f36580c7520 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f36480098e0 tx=0x7f36480091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d732ad92237b9114 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:31.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f36577fe700 1 -- 172.21.15.119:0/2922173087 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648011630 con 0x7f36580c83f0 2023-10-08T02:11:31.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f36577fe700 1 -- 172.21.15.119:0/2922173087 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648011790 con 0x7f36580c83f0 2023-10-08T02:11:31.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f36577fe700 1 -- 172.21.15.119:0/2922173087 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648011b10 con 0x7f36580c83f0 2023-10-08T02:11:31.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f365e5ff700 1 -- 172.21.15.119:0/2922173087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 msgr2=0x7f36580c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:31.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f365e5ff700 1 --2- 172.21.15.119:0/2922173087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f36580c7520 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f36480098e0 tx=0x7f36480091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f365e5ff700 1 -- 172.21.15.119:0/2922173087 shutdown_connections 2023-10-08T02:11:31.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f365e5ff700 1 --2- 172.21.15.119:0/2922173087 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f36580c7520 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f365e5ff700 1 --2- 172.21.15.119:0/2922173087 >> v2:172.21.15.184:3300/0 conn(0x7f36580ccee0 0x7f36581561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f365e5ff700 1 -- 172.21.15.119:0/2922173087 >> 172.21.15.119:0/2922173087 conn(0x7f3658068e10 msgr2=0x7f36580cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:31.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f365e5ff700 1 -- 172.21.15.119:0/2922173087 shutdown_connections 2023-10-08T02:11:31.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.814+0000 7f365e5ff700 1 -- 172.21.15.119:0/2922173087 wait complete. 2023-10-08T02:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.816+0000 7f365e5ff700 1 Processor -- start 2023-10-08T02:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.816+0000 7f365e5ff700 1 -- start start 2023-10-08T02:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f365e5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f365813e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f365e5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36580c87c0 0x7f36580cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f365e5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36580ccee0 0x7f36580cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f365e5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36580b05d0 con 0x7f36580c87c0 2023-10-08T02:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f365e5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36580b08b0 con 0x7f36580ccee0 2023-10-08T02:11:31.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f365e5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36580b0b90 con 0x7f36580c83f0 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f364ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36580c87c0 0x7f36580cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f3657fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f365813e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f364ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36580c87c0 0x7f36580cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52984/0 (socket says 172.21.15.119:52984) 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f364ffff700 1 -- 172.21.15.119:0/716863020 learned_addr learned my addr 172.21.15.119:0/716863020 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f365cb9c700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36580ccee0 0x7f36580cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f364ffff700 1 -- 172.21.15.119:0/716863020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 msgr2=0x7f365813e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f364ffff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f365813e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f364ffff700 1 -- 172.21.15.119:0/716863020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36580ccee0 msgr2=0x7f36580cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.817+0000 7f364ffff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36580ccee0 0x7f36580cb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.818+0000 7f364ffff700 1 -- 172.21.15.119:0/716863020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3648009420 con 0x7f36580c87c0 2023-10-08T02:11:31.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.818+0000 7f3657fff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f365813e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.818+0000 7f364ffff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36580c87c0 0x7f36580cb240 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f364000e930 tx=0x7f36400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.818+0000 7f3654ff9700 1 -- 172.21.15.119:0/716863020 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640013680 con 0x7f36580c87c0 2023-10-08T02:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.818+0000 7f3654ff9700 1 -- 172.21.15.119:0/716863020 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36400137e0 con 0x7f36580c87c0 2023-10-08T02:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.818+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36580b0e70 con 0x7f36580c87c0 2023-10-08T02:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.818+0000 7f3654ff9700 1 -- 172.21.15.119:0/716863020 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3640013b80 con 0x7f36580c87c0 2023-10-08T02:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.818+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36580b02a0 con 0x7f36580c87c0 2023-10-08T02:11:31.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.819+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f365804c3c0 con 0x7f36580c87c0 2023-10-08T02:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.820+0000 7f3654ff9700 1 -- 172.21.15.119:0/716863020 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f364000c3e0 con 0x7f36580c87c0 2023-10-08T02:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.821+0000 7f3654ff9700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3638066f30 0x7f36380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.821+0000 7f3654ff9700 1 -- 172.21.15.119:0/716863020 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f3640084720 con 0x7f36580c87c0 2023-10-08T02:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.821+0000 7f3657fff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3638066f30 0x7f36380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:31.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.822+0000 7f3657fff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3638066f30 0x7f36380693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f36480091e0 tx=0x7f3648017000 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:31.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:31.825+0000 7f3654ff9700 1 -- 172.21.15.119:0/716863020 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3640051820 con 0x7f36580c87c0 2023-10-08T02:11:32.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.013+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f365815aa80 con 0x7f36580c87c0 2023-10-08T02:11:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.017+0000 7f3654ff9700 1 -- 172.21.15.119:0/716863020 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3640054e40 con 0x7f36580c87c0 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3638066f30 msgr2=0x7f36380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3638066f30 0x7f36380693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f36480091e0 tx=0x7f3648017000 comp rx=0 tx=0).stop 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36580c87c0 msgr2=0x7f36580cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36580c87c0 0x7f36580cb240 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f364000e930 tx=0x7f36400091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 shutdown_connections 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3638066f30 0x7f36380693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36580c83f0 0x7f365813e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36580ccee0 0x7f36580cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 --2- 172.21.15.119:0/716863020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36580c87c0 0x7f36580cb240 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 >> 172.21.15.119:0/716863020 conn(0x7f3658068e10 msgr2=0x7f36581553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:32.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 shutdown_connections 2023-10-08T02:11:32.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.020+0000 7f365e5ff700 1 -- 172.21.15.119:0/716863020 wait complete. 2023-10-08T02:11:32.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T02:11:32.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.221+0000 7faac79ff700 1 Processor -- start 2023-10-08T02:11:32.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.221+0000 7faac79ff700 1 -- start start 2023-10-08T02:11:32.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.222+0000 7faac79ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faac0155eb0 0x7faac006e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.222+0000 7faac79ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7faac00c70b0 0x7faac0069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.222+0000 7faac79ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac0069cb0 0x7faac0159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.222+0000 7faac79ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac006a130 con 0x7faac0155eb0 2023-10-08T02:11:32.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.222+0000 7faac79ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac006eb90 con 0x7faac00c70b0 2023-10-08T02:11:32.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.222+0000 7faac79ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac006ee70 con 0x7faac0069cb0 2023-10-08T02:11:32.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faac0155eb0 0x7faac006e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faac0155eb0 0x7faac006e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52994/0 (socket says 172.21.15.119:52994) 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 -- 172.21.15.119:0/653717438 learned_addr learned my addr 172.21.15.119:0/653717438 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac5f9c700 1 --2- 172.21.15.119:0/653717438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac0069cb0 0x7faac0159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac4f9a700 1 --2- 172.21.15.119:0/653717438 >> v2:172.21.15.184:3300/0 conn(0x7faac00c70b0 0x7faac0069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 -- 172.21.15.119:0/653717438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac0069cb0 msgr2=0x7faac0159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 --2- 172.21.15.119:0/653717438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac0069cb0 0x7faac0159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 -- 172.21.15.119:0/653717438 >> v2:172.21.15.184:3300/0 conn(0x7faac00c70b0 msgr2=0x7faac0069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 --2- 172.21.15.119:0/653717438 >> v2:172.21.15.184:3300/0 conn(0x7faac00c70b0 0x7faac0069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 -- 172.21.15.119:0/653717438 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac006f150 con 0x7faac0155eb0 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac4f9a700 1 --2- 172.21.15.119:0/653717438 >> v2:172.21.15.184:3300/0 conn(0x7faac00c70b0 0x7faac0069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faac579b700 1 --2- 172.21.15.119:0/653717438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0155eb0 0x7faac006e240 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7faac0065130 tx=0x7faabc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c156566f11352d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:32.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.223+0000 7faab7fff700 1 -- 172.21.15.119:0/653717438 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc00b820 con 0x7faac0155eb0 2023-10-08T02:11:32.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faab7fff700 1 -- 172.21.15.119:0/653717438 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc00b980 con 0x7faac0155eb0 2023-10-08T02:11:32.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faab7fff700 1 -- 172.21.15.119:0/653717438 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc00bd20 con 0x7faac0155eb0 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faac79ff700 1 -- 172.21.15.119:0/653717438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0155eb0 msgr2=0x7faac006e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faac79ff700 1 --2- 172.21.15.119:0/653717438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0155eb0 0x7faac006e240 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7faac0065130 tx=0x7faabc009250 comp rx=0 tx=0).stop 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faac79ff700 1 -- 172.21.15.119:0/653717438 shutdown_connections 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faac79ff700 1 --2- 172.21.15.119:0/653717438 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac0069cb0 0x7faac0159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faac79ff700 1 --2- 172.21.15.119:0/653717438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0155eb0 0x7faac006e240 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faac79ff700 1 --2- 172.21.15.119:0/653717438 >> v2:172.21.15.184:3300/0 conn(0x7faac00c70b0 0x7faac0069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.224+0000 7faac79ff700 1 -- 172.21.15.119:0/653717438 >> 172.21.15.119:0/653717438 conn(0x7faac0068e10 msgr2=0x7faac00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.225+0000 7faac79ff700 1 -- 172.21.15.119:0/653717438 shutdown_connections 2023-10-08T02:11:32.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.225+0000 7faac79ff700 1 -- 172.21.15.119:0/653717438 wait complete. 2023-10-08T02:11:32.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.226+0000 7faac79ff700 1 Processor -- start 2023-10-08T02:11:32.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.226+0000 7faac79ff700 1 -- start start 2023-10-08T02:11:32.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac79ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0069cb0 0x7faac00ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac79ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac00c70b0 0x7faac0154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac79ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac0155eb0 0x7faac00abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac79ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00c43c0 con 0x7faac0069cb0 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac79ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00c46a0 con 0x7faac0155eb0 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac79ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac0154760 con 0x7faac00c70b0 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0069cb0 0x7faac00ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0069cb0 0x7faac00ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53006/0 (socket says 172.21.15.119:53006) 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac579b700 1 -- 172.21.15.119:0/633115955 learned_addr learned my addr 172.21.15.119:0/633115955 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac5f9c700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac0155eb0 0x7faac00abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac4f9a700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac00c70b0 0x7faac0154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.227+0000 7faac579b700 1 -- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac00c70b0 msgr2=0x7faac0154220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faac579b700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac00c70b0 0x7faac0154220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faac579b700 1 -- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac0155eb0 msgr2=0x7faac00abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faac579b700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac0155eb0 0x7faac00abb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faac579b700 1 -- 172.21.15.119:0/633115955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0094f0 con 0x7faac0069cb0 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faac5f9c700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac0155eb0 0x7faac00abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faac579b700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0069cb0 0x7faac00ae760 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7faac006fe40 tx=0x7faabc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faab57fa700 1 -- 172.21.15.119:0/633115955 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc011e70 con 0x7faac0069cb0 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faab57fa700 1 -- 172.21.15.119:0/633115955 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faabc01a420 con 0x7faac0069cb0 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac0154a30 con 0x7faac0069cb0 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faab57fa700 1 -- 172.21.15.119:0/633115955 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faabc01a800 con 0x7faac0069cb0 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.228+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac0154ff0 con 0x7faac0069cb0 2023-10-08T02:11:32.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.229+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac004c3c0 con 0x7faac0069cb0 2023-10-08T02:11:32.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.230+0000 7faab57fa700 1 -- 172.21.15.119:0/633115955 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7faabc01abf0 con 0x7faac0069cb0 2023-10-08T02:11:32.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.231+0000 7faab57fa700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faaa4066f30 0x7faaa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.231+0000 7faab57fa700 1 -- 172.21.15.119:0/633115955 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5919+0+0 (secure 0 0 0) 0x7faabc085f90 con 0x7faac0069cb0 2023-10-08T02:11:32.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.231+0000 7faac4f9a700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faaa4066f30 0x7faaa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.232+0000 7faac4f9a700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faaa4066f30 0x7faaa40693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faac00adec0 tx=0x7faab0006f90 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:32.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.235+0000 7faab57fa700 1 -- 172.21.15.119:0/633115955 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7faabc053090 con 0x7faac0069cb0 2023-10-08T02:11:32.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.428+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7faac00cc230 con 0x7faac0069cb0 2023-10-08T02:11:32.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.434+0000 7faab57fa700 1 -- 172.21.15.119:0/633115955 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v50) v1 ==== 115+0+0 (secure 0 0 0) 0x7faabc0150b0 con 0x7faac0069cb0 2023-10-08T02:11:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faaa4066f30 msgr2=0x7faaa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faaa4066f30 0x7faaa40693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7faac00adec0 tx=0x7faab0006f90 comp rx=0 tx=0).stop 2023-10-08T02:11:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0069cb0 msgr2=0x7faac00ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0069cb0 0x7faac00ae760 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7faac006fe40 tx=0x7faabc017000 comp rx=0 tx=0).stop 2023-10-08T02:11:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 shutdown_connections 2023-10-08T02:11:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7faaa4066f30 0x7faaa40693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7faac0155eb0 0x7faac00abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faac0069cb0 0x7faac00ae760 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 --2- 172.21.15.119:0/633115955 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7faac00c70b0 0x7faac0154220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 >> 172.21.15.119:0/633115955 conn(0x7faac0068e10 msgr2=0x7faac00c2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:32.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 shutdown_connections 2023-10-08T02:11:32.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.437+0000 7faac79ff700 1 -- 172.21.15.119:0/633115955 wait complete. 2023-10-08T02:11:32.455 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T02:11:32.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.640+0000 7f169fbff700 1 Processor -- start 2023-10-08T02:11:32.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.640+0000 7f169fbff700 1 -- start start 2023-10-08T02:11:32.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.641+0000 7f169fbff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f16980ccee0 0x7f16981561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.641+0000 7f169fbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c87c0 0x7f1698156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.641+0000 7f169fbff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169815b0d0 con 0x7f16980c83f0 2023-10-08T02:11:32.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.641+0000 7f169fbff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1698156c50 con 0x7f16980ccee0 2023-10-08T02:11:32.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.641+0000 7f169fbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169815a4c0 con 0x7f16980c87c0 2023-10-08T02:11:32.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169d99b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f16980c83f0 0x7f16980c7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47434/0 (socket says 172.21.15.119:47434) 2023-10-08T02:11:32.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169d99b700 1 -- 172.21.15.119:0/2003047358 learned_addr learned my addr 172.21.15.119:0/2003047358 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:32.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356830693 0 0) 0x7f169815b0d0 con 0x7f16980c83f0 2023-10-08T02:11:32.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169e19c700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c87c0 0x7f1698156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169d19a700 1 --2- 172.21.15.119:0/2003047358 >> v2:172.21.15.184:3300/0 conn(0x7f16980ccee0 0x7f16981561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1684003570 con 0x7f16980c83f0 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3257659175 0 0) 0x7f1684003570 con 0x7f16980c83f0 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c87c0 msgr2=0x7f1698156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c87c0 0x7f1698156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 >> v2:172.21.15.184:3300/0 conn(0x7f16980ccee0 msgr2=0x7f16981561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 --2- 172.21.15.119:0/2003047358 >> v2:172.21.15.184:3300/0 conn(0x7f16980ccee0 0x7f16981561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169815a7a0 con 0x7f16980c83f0 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f1694001470 con 0x7f16980c83f0 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.642+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 >> v1:172.21.15.119:6789/0 conn(0x7f16980c83f0 legacy=0x7f16980c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169c999700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684003a60 0x7f1684003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169c999700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c87c0 0x7f1684004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169c999700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1684004d50 0x7f16840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1698156c50 con 0x7f1684004d50 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169815a4c0 con 0x7f1684003a60 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168400dc70 con 0x7f16980c87c0 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169e19c700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1684004d50 0x7f16840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169d99b700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684003a60 0x7f1684003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169d19a700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c87c0 0x7f1684004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169e19c700 1 -- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c87c0 msgr2=0x7f1684004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169e19c700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c87c0 0x7f1684004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169e19c700 1 -- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684003a60 msgr2=0x7f1684003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169e19c700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684003a60 0x7f1684003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169e19c700 1 -- 172.21.15.119:0/2003047358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1684003570 con 0x7f1684004d50 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.643+0000 7f169e19c700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1684004d50 0x7f16840051c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f169000e210 tx=0x7f169000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e330333f5644250 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000ea60 con 0x7f1684004d50 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169000f040 con 0x7f1684004d50 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169c999700 1 -- 172.21.15.119:0/2003047358 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169000ee10 con 0x7f1684004d50 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169fbff700 1 -- 172.21.15.119:0/2003047358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1684004d50 msgr2=0x7f16840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169fbff700 1 --2- 172.21.15.119:0/2003047358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1684004d50 0x7f16840051c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f169000e210 tx=0x7f169000b040 comp rx=0 tx=0).stop 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169e19c700 1 -- 172.21.15.119:0/2003047358 reap_dead start 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169fbff700 1 -- 172.21.15.119:0/2003047358 shutdown_connections 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169fbff700 1 -- 172.21.15.119:0/2003047358 >> 172.21.15.119:0/2003047358 conn(0x7f1698068e10 msgr2=0x7f16980cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169fbff700 1 -- 172.21.15.119:0/2003047358 shutdown_connections 2023-10-08T02:11:32.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.644+0000 7f169fbff700 1 -- 172.21.15.119:0/2003047358 wait complete. 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.646+0000 7f169fbff700 1 Processor -- start 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.646+0000 7f169fbff700 1 -- start start 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169fbff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684004d50 0x7f169814cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169fbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1698158b50 0x7f1698149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169fbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c12d0 0x7f16980c1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169fbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16980c1c90 con 0x7f1698158b50 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169fbff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169814d150 con 0x7f1684004d50 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169fbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169814d430 con 0x7f16980c12d0 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169d19a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1698158b50 0x7f1698149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169d19a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1698158b50 0x7f1698149000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53016/0 (socket says 172.21.15.119:53016) 2023-10-08T02:11:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169d19a700 1 -- 172.21.15.119:0/280892774 learned_addr learned my addr 172.21.15.119:0/280892774 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:32.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169d99b700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684004d50 0x7f169814cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.647+0000 7f169e19c700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c12d0 0x7f16980c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f169d19a700 1 -- 172.21.15.119:0/280892774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c12d0 msgr2=0x7f16980c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f169d19a700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c12d0 0x7f16980c1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f169d19a700 1 -- 172.21.15.119:0/280892774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684004d50 msgr2=0x7f169814cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f169d19a700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684004d50 0x7f169814cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f169d19a700 1 -- 172.21.15.119:0/280892774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169814da20 con 0x7f1698158b50 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f169d19a700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1698158b50 0x7f1698149000 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f1688006ac0 tx=0x7f16880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f168dffb700 1 -- 172.21.15.119:0/280892774 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688011ec0 con 0x7f1698158b50 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f168dffb700 1 -- 172.21.15.119:0/280892774 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168800f940 con 0x7f1698158b50 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f168dffb700 1 -- 172.21.15.119:0/280892774 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168800fcc0 con 0x7f1698158b50 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169814dd00 con 0x7f1698158b50 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.648+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1698158fc0 con 0x7f1698158b50 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.649+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169814d880 con 0x7f1698158b50 2023-10-08T02:11:32.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.650+0000 7f168dffb700 1 -- 172.21.15.119:0/280892774 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1688014650 con 0x7f1698158b50 2023-10-08T02:11:32.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.651+0000 7f168dffb700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f167c066f30 0x7f167c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:32.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.651+0000 7f168dffb700 1 -- 172.21.15.119:0/280892774 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f1688018030 con 0x7f1698158b50 2023-10-08T02:11:32.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.651+0000 7f169d99b700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f167c066f30 0x7f167c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:32.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.652+0000 7f169d99b700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f167c066f30 0x7f167c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1694001050 tx=0x7f1694001e90 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:32.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.655+0000 7f168dffb700 1 -- 172.21.15.119:0/280892774 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f169814d880 con 0x7f1698158b50 2023-10-08T02:11:32.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.845+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f169814d880 con 0x7f1698158b50 2023-10-08T02:11:32.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.849+0000 7f168dffb700 1 -- 172.21.15.119:0/280892774 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v51) v1 ==== 120+0+0 (secure 0 0 0) 0x7f169814d880 con 0x7f1698158b50 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f167c066f30 msgr2=0x7f167c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f167c066f30 0x7f167c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1694001050 tx=0x7f1694001e90 comp rx=0 tx=0).stop 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1698158b50 msgr2=0x7f1698149000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1698158b50 0x7f1698149000 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f1688006ac0 tx=0x7f16880091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 shutdown_connections 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f167c066f30 0x7f167c0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16980c12d0 0x7f16980c1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1684004d50 0x7f169814cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 --2- 172.21.15.119:0/280892774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1698158b50 0x7f1698149000 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:32.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 >> 172.21.15.119:0/280892774 conn(0x7f1698068e10 msgr2=0x7f16980b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:32.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 shutdown_connections 2023-10-08T02:11:32.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:32.852+0000 7f169fbff700 1 -- 172.21.15.119:0/280892774 wait complete. 2023-10-08T02:11:32.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-10-08T02:11:33.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.050+0000 7f6d12f9d700 1 Processor -- start 2023-10-08T02:11:33.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.050+0000 7f6d12f9d700 1 -- start start 2023-10-08T02:11:33.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.051+0000 7f6d12f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d0c0ad870 0x7f6d0c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.051+0000 7f6d12f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c06bf20 0x7f6d0c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.051+0000 7f6d12f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d0c0b40f0 con 0x7f6d0c142ea0 2023-10-08T02:11:33.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d12f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d0c06c3a0 con 0x7f6d0c0ad870 2023-10-08T02:11:33.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d12f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c069640 con 0x7f6d0c06bf20 2023-10-08T02:11:33.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d11f9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6d0c142ea0 0x7f6d0c0b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47448/0 (socket says 172.21.15.119:47448) 2023-10-08T02:11:33.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d11f9b700 1 -- 172.21.15.119:0/2835983669 learned_addr learned my addr 172.21.15.119:0/2835983669 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:33.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d1179a700 1 --2- 172.21.15.119:0/2835983669 >> v2:172.21.15.184:3300/0 conn(0x7f6d0c0ad870 0x7f6d0c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d1279c700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c06bf20 0x7f6d0c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859269543 0 0) 0x7f6d0c0b40f0 con 0x7f6d0c142ea0 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cf8003570 con 0x7f6d0c142ea0 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.052+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4261045532 0 0) 0x7f6cf8003570 con 0x7f6d0c142ea0 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c06bf20 msgr2=0x7f6d0c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c06bf20 0x7f6d0c06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 >> v2:172.21.15.184:3300/0 conn(0x7f6d0c0ad870 msgr2=0x7f6d0c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 --2- 172.21.15.119:0/2835983669 >> v2:172.21.15.184:3300/0 conn(0x7f6d0c0ad870 0x7f6d0c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c069920 con 0x7f6d0c142ea0 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f6d08001470 con 0x7f6d0c142ea0 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 >> v1:172.21.15.119:6789/0 conn(0x7f6d0c142ea0 legacy=0x7f6d0c0b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c06bf20 0x7f6cf8003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf80040b0 0x7f6cf8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6cf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c06c3a0 con 0x7f6cf8004d50 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c069640 con 0x7f6cf80040b0 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf800bb60 con 0x7f6d0c06bf20 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1279c700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6cf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1179a700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf80040b0 0x7f6cf8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d11f9b700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c06bf20 0x7f6cf8003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1279c700 1 -- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c06bf20 msgr2=0x7f6cf8003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1279c700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c06bf20 0x7f6cf8003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1279c700 1 -- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf80040b0 msgr2=0x7f6cf8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:33.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1279c700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf80040b0 0x7f6cf8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1279c700 1 -- 172.21.15.119:0/2835983669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf8003570 con 0x7f6cf8004d50 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1179a700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cf80040b0 0x7f6cf8004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.053+0000 7f6d1279c700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6cf80051c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6d040094d0 tx=0x7f6d0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132593e7c38cce5b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400ea90 con 0x7f6cf8004d50 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0400f040 con 0x7f6cf8004d50 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d10f99700 1 -- 172.21.15.119:0/2835983669 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0400ee20 con 0x7f6cf8004d50 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d12f9d700 1 -- 172.21.15.119:0/2835983669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 msgr2=0x7f6cf80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d12f9d700 1 --2- 172.21.15.119:0/2835983669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6cf80051c0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f6d040094d0 tx=0x7f6d0400b040 comp rx=0 tx=0).stop 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d1279c700 1 -- 172.21.15.119:0/2835983669 reap_dead start 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d12f9d700 1 -- 172.21.15.119:0/2835983669 shutdown_connections 2023-10-08T02:11:33.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d12f9d700 1 -- 172.21.15.119:0/2835983669 >> 172.21.15.119:0/2835983669 conn(0x7f6d0c0ce430 msgr2=0x7f6d0c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:33.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d12f9d700 1 -- 172.21.15.119:0/2835983669 shutdown_connections 2023-10-08T02:11:33.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.054+0000 7f6d12f9d700 1 -- 172.21.15.119:0/2835983669 wait complete. 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.056+0000 7f6d12f9d700 1 Processor -- start 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.056+0000 7f6d12f9d700 1 -- start start 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d12f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6d0c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d12f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c169ae0 0x7f6d0c15e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d12f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c169f50 0x7f6d0c141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d12f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0ad370 con 0x7f6cf8004d50 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d12f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c140310 con 0x7f6d0c169f50 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d12f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c0c94e0 con 0x7f6d0c169ae0 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6d0c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6d0c0ace30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53032/0 (socket says 172.21.15.119:53032) 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 -- 172.21.15.119:0/1380700829 learned_addr learned my addr 172.21.15.119:0/1380700829 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 -- 172.21.15.119:0/1380700829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c169ae0 msgr2=0x7f6d0c15e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c169ae0 0x7f6d0c15e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 -- 172.21.15.119:0/1380700829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c169f50 msgr2=0x7f6d0c141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c169f50 0x7f6d0c141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 -- 172.21.15.119:0/1380700829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0c0c88d0 con 0x7f6cf8004d50 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.057+0000 7f6d11f9b700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6d0c0ace30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800e840 tx=0x7f6d08000d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.058+0000 7f6d01ffb700 1 -- 172.21.15.119:0/1380700829 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d08012500 con 0x7f6cf8004d50 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.058+0000 7f6d01ffb700 1 -- 172.21.15.119:0/1380700829 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d08010040 con 0x7f6cf8004d50 2023-10-08T02:11:33.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.058+0000 7f6d01ffb700 1 -- 172.21.15.119:0/1380700829 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080128d0 con 0x7f6cf8004d50 2023-10-08T02:11:33.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.058+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c0c8bb0 con 0x7f6cf8004d50 2023-10-08T02:11:33.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.058+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0c0c9170 con 0x7f6cf8004d50 2023-10-08T02:11:33.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.059+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0c0cf250 con 0x7f6cf8004d50 2023-10-08T02:11:33.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.060+0000 7f6d01ffb700 1 -- 172.21.15.119:0/1380700829 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d08012c40 con 0x7f6cf8004d50 2023-10-08T02:11:33.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.060+0000 7f6d01ffb700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6cf0066f30 0x7f6cf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.060+0000 7f6d01ffb700 1 -- 172.21.15.119:0/1380700829 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f6d08083af0 con 0x7f6cf8004d50 2023-10-08T02:11:33.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.061+0000 7f6d1179a700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6cf0066f30 0x7f6cf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.061+0000 7f6d1179a700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6cf0066f30 0x7f6cf00693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c15e590 tx=0x7f6cfc008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:33.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.064+0000 7f6d01ffb700 1 -- 172.21.15.119:0/1380700829 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d08050bf0 con 0x7f6cf8004d50 2023-10-08T02:11:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.330+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f6d0c04c3c0 con 0x7f6cf8004d50 2023-10-08T02:11:33.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.594+0000 7f6d01ffb700 1 -- 172.21.15.119:0/1380700829 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2441) v1 ==== 178+0+0 (secure 0 0 0) 0x7f6d08019020 con 0x7f6cf8004d50 2023-10-08T02:11:33.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6cf0066f30 msgr2=0x7f6cf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6cf0066f30 0x7f6cf00693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c15e590 tx=0x7f6cfc008040 comp rx=0 tx=0).stop 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 msgr2=0x7f6d0c0ace30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6d0c0ace30 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800e840 tx=0x7f6d08000d90 comp rx=0 tx=0).stop 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 shutdown_connections 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6cf0066f30 0x7f6cf00693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d0c169f50 0x7f6d0c141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cf8004d50 0x7f6d0c0ace30 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 --2- 172.21.15.119:0/1380700829 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d0c169ae0 0x7f6d0c15e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 >> 172.21.15.119:0/1380700829 conn(0x7f6d0c0ce430 msgr2=0x7f6d0c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 shutdown_connections 2023-10-08T02:11:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.597+0000 7f6d12f9d700 1 -- 172.21.15.119:0/1380700829 wait complete. 2023-10-08T02:11:33.615 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-10-08T02:11:33.615 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-10-08T02:11:33.615 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T02:11:33.616 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-08T02:11:33.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.796+0000 7fcac4dff700 1 Processor -- start 2023-10-08T02:11:33.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.796+0000 7fcac4dff700 1 -- start start 2023-10-08T02:11:33.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcac4dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcac00b1420 0x7fcac00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcac4dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcac00ccf90 0x7fcac00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcac4dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac00cc300 con 0x7fcac00cbf30 2023-10-08T02:11:33.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcac4dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac0156bf0 con 0x7fcac00b1420 2023-10-08T02:11:33.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcac4dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00ae950 con 0x7fcac00ccf90 2023-10-08T02:11:33.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcabe59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcac00cbf30 0x7fcac00b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47464/0 (socket says 172.21.15.119:47464) 2023-10-08T02:11:33.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcabe59c700 1 -- 172.21.15.119:0/3950272620 learned_addr learned my addr 172.21.15.119:0/3950272620 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556339919 0 0) 0x7fcac00cc300 con 0x7fcac00cbf30 2023-10-08T02:11:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaac003570 con 0x7fcac00cbf30 2023-10-08T02:11:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcabed9d700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcac00ccf90 0x7fcac00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.798+0000 7fcab7fff700 1 --2- 172.21.15.119:0/3950272620 >> v2:172.21.15.184:3300/0 conn(0x7fcac00b1420 0x7fcac00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4259822385 0 0) 0x7fcaac003570 con 0x7fcac00cbf30 2023-10-08T02:11:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcac00ccf90 msgr2=0x7fcac00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcac00ccf90 0x7fcac00c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 >> v2:172.21.15.184:3300/0 conn(0x7fcac00b1420 msgr2=0x7fcac00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 --2- 172.21.15.119:0/3950272620 >> v2:172.21.15.184:3300/0 conn(0x7fcac00b1420 0x7fcac00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac00ad370 con 0x7fcac00cbf30 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fcab0001470 con 0x7fcac00cbf30 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 >> v1:172.21.15.119:6789/0 conn(0x7fcac00cbf30 legacy=0x7fcac00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaac003a90 0x7fcaac003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcac00ccf90 0x7fcaac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac004d50 0x7fcaac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0156bf0 con 0x7fcaac004d50 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00ae950 con 0x7fcaac003a90 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaac00dc70 con 0x7fcac00ccf90 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabed9d700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac004d50 0x7fcaac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabe59c700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaac003a90 0x7fcaac003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcab7fff700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcac00ccf90 0x7fcaac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabed9d700 1 -- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcac00ccf90 msgr2=0x7fcaac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabed9d700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcac00ccf90 0x7fcaac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabed9d700 1 -- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaac003a90 msgr2=0x7fcaac003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabed9d700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcaac003a90 0x7fcaac003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.799+0000 7fcabed9d700 1 -- 172.21.15.119:0/3950272620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac003570 con 0x7fcaac004d50 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcabed9d700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac004d50 0x7fcaac0051c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcab800e210 tx=0x7fcab800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c703a540e4a1646b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800ea60 con 0x7fcaac004d50 2023-10-08T02:11:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab800f040 con 0x7fcaac004d50 2023-10-08T02:11:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcabdd9b700 1 -- 172.21.15.119:0/3950272620 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800ee10 con 0x7fcaac004d50 2023-10-08T02:11:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcac4dff700 1 -- 172.21.15.119:0/3950272620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac004d50 msgr2=0x7fcaac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcac4dff700 1 --2- 172.21.15.119:0/3950272620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcaac004d50 0x7fcaac0051c0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcab800e210 tx=0x7fcab800b040 comp rx=0 tx=0).stop 2023-10-08T02:11:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcabed9d700 1 -- 172.21.15.119:0/3950272620 reap_dead start 2023-10-08T02:11:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcac4dff700 1 -- 172.21.15.119:0/3950272620 shutdown_connections 2023-10-08T02:11:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.800+0000 7fcac4dff700 1 -- 172.21.15.119:0/3950272620 >> 172.21.15.119:0/3950272620 conn(0x7fcac006a320 msgr2=0x7fcac00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.801+0000 7fcac4dff700 1 -- 172.21.15.119:0/3950272620 shutdown_connections 2023-10-08T02:11:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.801+0000 7fcac4dff700 1 -- 172.21.15.119:0/3950272620 wait complete. 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.802+0000 7fcac4dff700 1 Processor -- start 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.802+0000 7fcac4dff700 1 -- start start 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcac4dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaac004d50 0x7fcac016a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcac4dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac015ba20 0x7fcac0156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcac4dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac00c7110 0x7fcac00c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcac4dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00c7ac0 con 0x7fcac00c7110 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcac4dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac015eee0 con 0x7fcac015ba20 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcac4dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac015f370 con 0x7fcaac004d50 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcabed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac00c7110 0x7fcac00c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcabed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac00c7110 0x7fcac00c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53046/0 (socket says 172.21.15.119:53046) 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcabed9d700 1 -- 172.21.15.119:0/3181839586 learned_addr learned my addr 172.21.15.119:0/3181839586 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcab7fff700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac015ba20 0x7fcac0156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.803+0000 7fcabe59c700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaac004d50 0x7fcac016a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcabed9d700 1 -- 172.21.15.119:0/3181839586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaac004d50 msgr2=0x7fcac016a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcabed9d700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaac004d50 0x7fcac016a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcabed9d700 1 -- 172.21.15.119:0/3181839586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac015ba20 msgr2=0x7fcac0156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcabed9d700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac015ba20 0x7fcac0156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcabed9d700 1 -- 172.21.15.119:0/3181839586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac015e2d0 con 0x7fcac00c7110 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcabed9d700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac00c7110 0x7fcac00c7580 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcab800afd0 tx=0x7fcab801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcab67fc700 1 -- 172.21.15.119:0/3181839586 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab800eee0 con 0x7fcac00c7110 2023-10-08T02:11:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcab67fc700 1 -- 172.21.15.119:0/3181839586 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab801d630 con 0x7fcac00c7110 2023-10-08T02:11:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcab67fc700 1 -- 172.21.15.119:0/3181839586 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab801da30 con 0x7fcac00c7110 2023-10-08T02:11:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac015e5b0 con 0x7fcac00c7110 2023-10-08T02:11:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.804+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac015eb70 con 0x7fcac00c7110 2023-10-08T02:11:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.805+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac004c3c0 con 0x7fcac00c7110 2023-10-08T02:11:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.806+0000 7fcab67fc700 1 -- 172.21.15.119:0/3181839586 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcab8003040 con 0x7fcac00c7110 2023-10-08T02:11:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.807+0000 7fcab67fc700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcaa0066f30 0x7fcaa00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.807+0000 7fcab67fc700 1 -- 172.21.15.119:0/3181839586 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5919+0+0 (secure 0 0 0) 0x7fcab8085a80 con 0x7fcac00c7110 2023-10-08T02:11:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.807+0000 7fcabe59c700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcaa0066f30 0x7fcaa00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:33.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.808+0000 7fcabe59c700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcaa0066f30 0x7fcaa00693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000c00 tx=0x7fcac00c8fd0 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:33.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:33.811+0000 7fcab67fc700 1 -- 172.21.15.119:0/3181839586 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcab8052b80 con 0x7fcac00c7110 2023-10-08T02:11:34.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.077+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fcac015f020 con 0x7fcac00c7110 2023-10-08T02:11:34.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.595+0000 7fcab67fc700 1 -- 172.21.15.119:0/3181839586 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2442) v1 ==== 177+0+0 (secure 0 0 0) 0x7fcab80180f0 con 0x7fcac00c7110 2023-10-08T02:11:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcaa0066f30 msgr2=0x7fcaa00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcaa0066f30 0x7fcaa00693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fcab0000c00 tx=0x7fcac00c8fd0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac00c7110 msgr2=0x7fcac00c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac00c7110 0x7fcac00c7580 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fcab800afd0 tx=0x7fcab801a000 comp rx=0 tx=0).stop 2023-10-08T02:11:34.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 shutdown_connections 2023-10-08T02:11:34.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fcaa0066f30 0x7fcaa00693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcaac004d50 0x7fcac016a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcac00c7110 0x7fcac00c7580 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 --2- 172.21.15.119:0/3181839586 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcac015ba20 0x7fcac0156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 >> 172.21.15.119:0/3181839586 conn(0x7fcac006a320 msgr2=0x7fcac00c9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:34.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 shutdown_connections 2023-10-08T02:11:34.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.598+0000 7fcac4dff700 1 -- 172.21.15.119:0/3181839586 wait complete. 2023-10-08T02:11:34.618 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T02:11:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.803+0000 7f57ae19d700 1 Processor -- start 2023-10-08T02:11:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.803+0000 7f57ae19d700 1 -- start start 2023-10-08T02:11:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.804+0000 7f57ae19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57a8153900 0x7f57a80c6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.804+0000 7f57ae19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f57a806a710 0x7f57a806e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.804+0000 7f57ae19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a806e150 con 0x7f57a80c9db0 2023-10-08T02:11:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.804+0000 7f57ae19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a80c7800 con 0x7f57a8153900 2023-10-08T02:11:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.804+0000 7f57ae19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a80c7ae0 con 0x7f57a806a710 2023-10-08T02:11:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.804+0000 7f57ad19b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57a8153900 0x7f57a80c6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.804+0000 7f57ad19b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57a8153900 0x7f57a80c6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53060/0 (socket says 172.21.15.119:53060) 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.804+0000 7f57ad19b700 1 -- 172.21.15.119:0/978828946 learned_addr learned my addr 172.21.15.119:0/978828946 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f57ad19b700 1 -- 172.21.15.119:0/978828946 >> v2:172.21.15.184:3300/0 conn(0x7f57a806a710 msgr2=0x7f57a806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f57ad99c700 1 --2- 172.21.15.119:0/978828946 >> v2:172.21.15.184:3300/0 conn(0x7f57a806a710 0x7f57a806e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f57ad19b700 1 --2- 172.21.15.119:0/978828946 >> v2:172.21.15.184:3300/0 conn(0x7f57a806a710 0x7f57a806e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f57ad19b700 1 -- 172.21.15.119:0/978828946 >> v1:172.21.15.184:6789/0 conn(0x7f57a80c9db0 legacy=0x7f57a806a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f57ad19b700 1 -- 172.21.15.119:0/978828946 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80c7dc0 con 0x7f57a8153900 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f57ad99c700 1 --2- 172.21.15.119:0/978828946 >> v2:172.21.15.184:3300/0 conn(0x7f57a806a710 0x7f57a806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f57ad19b700 1 --2- 172.21.15.119:0/978828946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a8153900 0x7f57a80c6f00 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f57a80cd2d0 tx=0x7f57a4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b99c31a094a07a9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f579ffff700 1 -- 172.21.15.119:0/978828946 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a400b820 con 0x7f57a8153900 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f579ffff700 1 -- 172.21.15.119:0/978828946 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a400b980 con 0x7f57a8153900 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.805+0000 7f579ffff700 1 -- 172.21.15.119:0/978828946 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a400bd20 con 0x7f57a8153900 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.806+0000 7f57ae19d700 1 -- 172.21.15.119:0/978828946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a8153900 msgr2=0x7f57a80c6f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.806+0000 7f57ae19d700 1 --2- 172.21.15.119:0/978828946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a8153900 0x7f57a80c6f00 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f57a80cd2d0 tx=0x7f57a4009250 comp rx=0 tx=0).stop 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.806+0000 7f57ae19d700 1 -- 172.21.15.119:0/978828946 shutdown_connections 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.806+0000 7f57ae19d700 1 --2- 172.21.15.119:0/978828946 >> v2:172.21.15.184:3300/0 conn(0x7f57a806a710 0x7f57a806e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.806+0000 7f57ae19d700 1 --2- 172.21.15.119:0/978828946 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a8153900 0x7f57a80c6f00 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.806+0000 7f57ae19d700 1 -- 172.21.15.119:0/978828946 >> 172.21.15.119:0/978828946 conn(0x7f57a80cc1e0 msgr2=0x7f57a80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.806+0000 7f57ae19d700 1 -- 172.21.15.119:0/978828946 shutdown_connections 2023-10-08T02:11:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.806+0000 7f57ae19d700 1 -- 172.21.15.119:0/978828946 wait complete. 2023-10-08T02:11:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.807+0000 7f57ae19d700 1 Processor -- start 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.808+0000 7f57ae19d700 1 -- start start 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.808+0000 7f57ae19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a806a710 0x7f57a8153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.808+0000 7f57ae19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80c9db0 0x7f57a80ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.808+0000 7f57ae19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8153900 0x7f57a80b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.808+0000 7f57ae19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a80ae980 con 0x7f57a80c9db0 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.808+0000 7f57ae19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a8154f40 con 0x7f57a806a710 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.808+0000 7f57ae19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a814a840 con 0x7f57a8153900 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80c9db0 0x7f57a80ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80c9db0 0x7f57a80ae440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53076/0 (socket says 172.21.15.119:53076) 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 -- 172.21.15.119:0/564534685 learned_addr learned my addr 172.21.15.119:0/564534685 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ad19b700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a806a710 0x7f57a8153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ad99c700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8153900 0x7f57a80b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 -- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8153900 msgr2=0x7f57a80b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8153900 0x7f57a80b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 -- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a806a710 msgr2=0x7f57a8153d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a806a710 0x7f57a8153d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 -- 172.21.15.119:0/564534685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a40094f0 con 0x7f57a80c9db0 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ad19b700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a806a710 0x7f57a8153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f57ac99a700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80c9db0 0x7f57a80ae440 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f579800c590 tx=0x7f5798007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.809+0000 7f579d7fa700 1 -- 172.21.15.119:0/564534685 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798011590 con 0x7f57a80c9db0 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.810+0000 7f579d7fa700 1 -- 172.21.15.119:0/564534685 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798011730 con 0x7f57a80c9db0 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.810+0000 7f579d7fa700 1 -- 172.21.15.119:0/564534685 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798011af0 con 0x7f57a80c9db0 2023-10-08T02:11:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.810+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a8154330 con 0x7f57a80c9db0 2023-10-08T02:11:34.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.810+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a81548f0 con 0x7f57a80c9db0 2023-10-08T02:11:34.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.811+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a81550f0 con 0x7f57a80c9db0 2023-10-08T02:11:34.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.811+0000 7f579d7fa700 1 -- 172.21.15.119:0/564534685 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f579801c460 con 0x7f57a80c9db0 2023-10-08T02:11:34.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.812+0000 7f579d7fa700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f578c066f30 0x7f578c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:34.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.812+0000 7f579d7fa700 1 -- 172.21.15.119:0/564534685 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f5798084430 con 0x7f57a80c9db0 2023-10-08T02:11:34.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.812+0000 7f57ad19b700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f578c066f30 0x7f578c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:34.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.813+0000 7f57ad19b700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f578c066f30 0x7f578c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f57a80cd2d0 tx=0x7f57a4016000 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:34.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:34.817+0000 7f579d7fa700 1 -- 172.21.15.119:0/564534685 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f57a81550f0 con 0x7f57a80c9db0 2023-10-08T02:11:35.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.008+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f57a806c4f0 con 0x7f57a80c9db0 2023-10-08T02:11:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.013+0000 7f579d7fa700 1 -- 172.21.15.119:0/564534685 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7f57a806c4f0 con 0x7f57a80c9db0 2023-10-08T02:11:35.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f578c066f30 msgr2=0x7f578c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f578c066f30 0x7f578c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f57a80cd2d0 tx=0x7f57a4016000 comp rx=0 tx=0).stop 2023-10-08T02:11:35.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80c9db0 msgr2=0x7f57a80ae440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80c9db0 0x7f57a80ae440 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f579800c590 tx=0x7f5798007f30 comp rx=0 tx=0).stop 2023-10-08T02:11:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 shutdown_connections 2023-10-08T02:11:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f578c066f30 0x7f578c0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57a8153900 0x7f57a80b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57a806a710 0x7f57a8153d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 --2- 172.21.15.119:0/564534685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57a80c9db0 0x7f57a80ae440 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 >> 172.21.15.119:0/564534685 conn(0x7f57a80cc1e0 msgr2=0x7f57a814e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 shutdown_connections 2023-10-08T02:11:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.015+0000 7f57ae19d700 1 -- 172.21.15.119:0/564534685 wait complete. 2023-10-08T02:11:35.033 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:11:35.033 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:11:35.114 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:11:35.203 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:11:35.203 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T02:11:35.204 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:11:35.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.386+0000 7f82b57ff700 1 Processor -- start 2023-10-08T02:11:35.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.386+0000 7f82b57ff700 1 -- start start 2023-10-08T02:11:35.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.387+0000 7f82b57ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f82b00b1c20 0x7f82b01572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.387+0000 7f82b57ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82b0157820 0x7f82b00abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82b57ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b0157c60 con 0x7f82b00b1850 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82b57ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b006fc30 con 0x7f82b0157820 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82b57ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b00cc260 con 0x7f82b00b1c20 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82aeffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f82b00b1850 0x7f82b00ae680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38506/0 (socket says 172.21.15.119:38506) 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82aeffd700 1 -- 172.21.15.119:0/2291889409 learned_addr learned my addr 172.21.15.119:0/2291889409 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82af7fe700 1 --2- 172.21.15.119:0/2291889409 >> v2:172.21.15.119:3300/0 conn(0x7f82b0157820 0x7f82b00abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82ae7fc700 1 --2- 172.21.15.119:0/2291889409 >> v2:172.21.15.184:3300/0 conn(0x7f82b00b1c20 0x7f82b01572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82adffb700 1 -- 172.21.15.119:0/2291889409 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974653963 0 0) 0x7f82b0157c60 con 0x7f82b00b1850 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82adffb700 1 -- 172.21.15.119:0/2291889409 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8298003570 con 0x7f82b00b1850 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82af7fe700 1 -- 172.21.15.119:0/2291889409 >> v2:172.21.15.184:3300/0 conn(0x7f82b00b1c20 msgr2=0x7f82b01572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:35.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82af7fe700 1 --2- 172.21.15.119:0/2291889409 >> v2:172.21.15.184:3300/0 conn(0x7f82b00b1c20 0x7f82b01572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82af7fe700 1 -- 172.21.15.119:0/2291889409 >> v1:172.21.15.119:6789/0 conn(0x7f82b00b1850 legacy=0x7f82b00ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.388+0000 7f82af7fe700 1 -- 172.21.15.119:0/2291889409 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00cb300 con 0x7f82b0157820 2023-10-08T02:11:35.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82af7fe700 1 --2- 172.21.15.119:0/2291889409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b0157820 0x7f82b00abf60 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f82b006d340 tx=0x7f82a8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304345dc6032bc0b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:35.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82adffb700 1 -- 172.21.15.119:0/2291889409 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800e070 con 0x7f82b0157820 2023-10-08T02:11:35.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82adffb700 1 -- 172.21.15.119:0/2291889409 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800bab0 con 0x7f82b0157820 2023-10-08T02:11:35.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82adffb700 1 -- 172.21.15.119:0/2291889409 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800be70 con 0x7f82b0157820 2023-10-08T02:11:35.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82b57ff700 1 -- 172.21.15.119:0/2291889409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b0157820 msgr2=0x7f82b00abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82b57ff700 1 --2- 172.21.15.119:0/2291889409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b0157820 0x7f82b00abf60 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f82b006d340 tx=0x7f82a8009780 comp rx=0 tx=0).stop 2023-10-08T02:11:35.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82b57ff700 1 -- 172.21.15.119:0/2291889409 shutdown_connections 2023-10-08T02:11:35.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82b57ff700 1 --2- 172.21.15.119:0/2291889409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b0157820 0x7f82b00abf60 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82b57ff700 1 --2- 172.21.15.119:0/2291889409 >> v2:172.21.15.184:3300/0 conn(0x7f82b00b1c20 0x7f82b01572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.389+0000 7f82b57ff700 1 -- 172.21.15.119:0/2291889409 >> 172.21.15.119:0/2291889409 conn(0x7f82b006a320 msgr2=0x7f82b00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:35.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.390+0000 7f82b57ff700 1 -- 172.21.15.119:0/2291889409 shutdown_connections 2023-10-08T02:11:35.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.390+0000 7f82b57ff700 1 -- 172.21.15.119:0/2291889409 wait complete. 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.391+0000 7f82b57ff700 1 Processor -- start 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.391+0000 7f82b57ff700 1 -- start start 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82b57ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b00b1850 0x7f82b0140f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82b57ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82b00b1c20 0x7f82b014d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82b57ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82b0157820 0x7f82b014dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82b57ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00af4b0 con 0x7f82b00b1850 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82b57ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01695e0 con 0x7f82b00b1c20 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82b57ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b014e330 con 0x7f82b0157820 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b00b1850 0x7f82b0140f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b00b1850 0x7f82b0140f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56924/0 (socket says 172.21.15.119:56924) 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.392+0000 7f82aeffd700 1 -- 172.21.15.119:0/1646639072 learned_addr learned my addr 172.21.15.119:0/1646639072 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:35.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f82aeffd700 1 -- 172.21.15.119:0/1646639072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82b0157820 msgr2=0x7f82b014dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f82aeffd700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82b0157820 0x7f82b014dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f82aeffd700 1 -- 172.21.15.119:0/1646639072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82b00b1c20 msgr2=0x7f82b014d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f82aeffd700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82b00b1c20 0x7f82b014d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f82aeffd700 1 -- 172.21.15.119:0/1646639072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8009a20 con 0x7f82b00b1850 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f82aeffd700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b00b1850 0x7f82b0140f10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f82a00011e0 tx=0x7f82a0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f8296ffd700 1 -- 172.21.15.119:0/1646639072 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0005b30 con 0x7f82b00b1850 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f8296ffd700 1 -- 172.21.15.119:0/1646639072 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a0005c90 con 0x7f82b00b1850 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b014e610 con 0x7f82b00b1850 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f8296ffd700 1 -- 172.21.15.119:0/1646639072 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a00072e0 con 0x7f82b00b1850 2023-10-08T02:11:35.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.393+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b0141450 con 0x7f82b00b1850 2023-10-08T02:11:35.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.394+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b004c3c0 con 0x7f82b00b1850 2023-10-08T02:11:35.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.395+0000 7f8296ffd700 1 -- 172.21.15.119:0/1646639072 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f82a0007650 con 0x7f82b00b1850 2023-10-08T02:11:35.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.396+0000 7f8296ffd700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f829c066f30 0x7f829c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.396+0000 7f8296ffd700 1 -- 172.21.15.119:0/1646639072 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f82a0083ab0 con 0x7f82b00b1850 2023-10-08T02:11:35.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.396+0000 7f82ae7fc700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f829c066f30 0x7f829c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.397+0000 7f82ae7fc700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f829c066f30 0x7f829c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f82b0068ee0 tx=0x7f82a40091a0 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:35.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.400+0000 7f8296ffd700 1 -- 172.21.15.119:0/1646639072 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82a004b5a0 con 0x7f82b00b1850 2023-10-08T02:11:35.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.637+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82b00cb5e0 con 0x7f82b00b1850 2023-10-08T02:11:35.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.637+0000 7f8296ffd700 1 -- 172.21.15.119:0/1646639072 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f82a0017020 con 0x7f82b00b1850 2023-10-08T02:11:35.638 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:35.638 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":65,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7549,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/26538394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":26538394},{"type":"v1","addr":"172.21.15.184:6834","nonce":26538394}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":7906,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2925410966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2925410966},{"type":"v1","addr":"172.21.15.184:6835","nonce":2925410966}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3494509212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3494509212},{"type":"v1","addr":"172.21.15.119:6834","nonce":3494509212}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58},{"gid":8166,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2792091862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":2792091862},{"type":"v1","addr":"172.21.15.119:6835","nonce":2792091862}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":58}],"filesystems":[]} 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f829c066f30 msgr2=0x7f829c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f829c066f30 0x7f829c0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f82b0068ee0 tx=0x7f82a40091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b00b1850 msgr2=0x7f82b0140f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b00b1850 0x7f82b0140f10 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f82a00011e0 tx=0x7f82a0007df0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 shutdown_connections 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f829c066f30 0x7f829c0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82b0157820 0x7f82b014dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82b00b1850 0x7f82b0140f10 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 --2- 172.21.15.119:0/1646639072 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82b00b1c20 0x7f82b014d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 >> 172.21.15.119:0/1646639072 conn(0x7f82b006a320 msgr2=0x7f82b0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:35.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 shutdown_connections 2023-10-08T02:11:35.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.640+0000 7f82b57ff700 1 -- 172.21.15.119:0/1646639072 wait complete. 2023-10-08T02:11:35.642 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 65 2023-10-08T02:11:35.658 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T02:11:35.658 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T02:11:35.658 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T02:11:35.659 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:11:35.659 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T02:11:35.659 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T02:11:35.659 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T02:11:35.659 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:11:35.659 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T02:11:35.659 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T02:11:35.660 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T02:11:35.660 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:11:35.660 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T02:11:35.660 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T02:11:35.660 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T02:11:35.660 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:11:35.673 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T02:11:35.673 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T02:11:35.674 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T02:11:35.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T02:11:35.676 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T02:11:35.677 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T02:11:35.678 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T02:11:35.679 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T02:11:35.718 INFO:tasks.ceph.mds.a:Started 2023-10-08T02:11:35.718 DEBUG:teuthology.parallel:result is None 2023-10-08T02:11:35.720 INFO:tasks.ceph.mds.c:Started 2023-10-08T02:11:35.721 DEBUG:teuthology.parallel:result is None 2023-10-08T02:11:35.723 INFO:tasks.ceph.mds.b:Started 2023-10-08T02:11:35.723 DEBUG:teuthology.parallel:result is None 2023-10-08T02:11:35.726 INFO:tasks.ceph.mds.d:Started 2023-10-08T02:11:35.726 DEBUG:teuthology.parallel:result is None 2023-10-08T02:11:35.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T02:11:35.853 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T02:11:35.853 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T02:11:35.864 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T02:11:35.864 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T02:11:35.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.921+0000 7f28371ff700 1 Processor -- start 2023-10-08T02:11:35.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.922+0000 7f28371ff700 1 -- start start 2023-10-08T02:11:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f28371ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2830159c60 0x7f283015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f28371ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283015a160 con 0x7f283015db10 2023-10-08T02:11:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f28371ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28300d13a0 con 0x7f2830159890 2023-10-08T02:11:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f28371ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f283015df50 con 0x7f2830159c60 2023-10-08T02:11:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f283579c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f283015db10 0x7f2830069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38514/0 (socket says 172.21.15.119:38514) 2023-10-08T02:11:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f283579c700 1 -- 172.21.15.119:0/3257065070 learned_addr learned my addr 172.21.15.119:0/3257065070 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f282ffff700 1 --2- 172.21.15.119:0/3257065070 >> v2:172.21.15.184:3300/0 conn(0x7f2830159c60 0x7f283015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f282f7fe700 1 -- 172.21.15.119:0/3257065070 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264896930 0 0) 0x7f283015a160 con 0x7f283015db10 2023-10-08T02:11:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.923+0000 7f282f7fe700 1 -- 172.21.15.119:0/3257065070 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2820003570 con 0x7f283015db10 2023-10-08T02:11:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.924+0000 7f282f7fe700 1 -- 172.21.15.119:0/3257065070 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856952159 0 0) 0x7f2820003570 con 0x7f283015db10 2023-10-08T02:11:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.924+0000 7f282ffff700 1 -- 172.21.15.119:0/3257065070 >> v1:172.21.15.184:6789/0 conn(0x7f2830159890 legacy=0x7f283015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.924+0000 7f282ffff700 1 -- 172.21.15.119:0/3257065070 >> v1:172.21.15.119:6789/0 conn(0x7f283015db10 legacy=0x7f2830069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.924+0000 7f282ffff700 1 -- 172.21.15.119:0/3257065070 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283015e140 con 0x7f2830159c60 2023-10-08T02:11:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.924+0000 7f282ffff700 1 --2- 172.21.15.119:0/3257065070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 0x7f283015d5d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f281c0098e0 tx=0x7f281c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3351ca93304e7f6e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.924+0000 7f282f7fe700 1 -- 172.21.15.119:0/3257065070 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28280017f0 con 0x7f2830159c60 2023-10-08T02:11:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.924+0000 7f282f7fe700 1 -- 172.21.15.119:0/3257065070 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281c011420 con 0x7f2830159c60 2023-10-08T02:11:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.924+0000 7f282f7fe700 1 -- 172.21.15.119:0/3257065070 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281c0117c0 con 0x7f2830159c60 2023-10-08T02:11:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.925+0000 7f28371ff700 1 -- 172.21.15.119:0/3257065070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 msgr2=0x7f283015d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.925+0000 7f28371ff700 1 --2- 172.21.15.119:0/3257065070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 0x7f283015d5d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f281c0098e0 tx=0x7f281c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.925+0000 7f28371ff700 1 -- 172.21.15.119:0/3257065070 shutdown_connections 2023-10-08T02:11:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.925+0000 7f28371ff700 1 --2- 172.21.15.119:0/3257065070 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 0x7f283015d5d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.925+0000 7f28371ff700 1 -- 172.21.15.119:0/3257065070 >> 172.21.15.119:0/3257065070 conn(0x7f2830068e20 msgr2=0x7f28300d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.925+0000 7f28371ff700 1 -- 172.21.15.119:0/3257065070 shutdown_connections 2023-10-08T02:11:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.925+0000 7f28371ff700 1 -- 172.21.15.119:0/3257065070 wait complete. 2023-10-08T02:11:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.926+0000 7f28371ff700 1 Processor -- start 2023-10-08T02:11:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.927+0000 7f28371ff700 1 -- start start 2023-10-08T02:11:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.927+0000 7f28371ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2830159890 0x7f28300cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.927+0000 7f28371ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 0x7f28300b5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.927+0000 7f28371ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f283015db10 0x7f283016a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.927+0000 7f28371ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300ce100 con 0x7f283015db10 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f28371ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28300cedf0 con 0x7f2830159c60 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f28371ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283016abb0 con 0x7f2830159890 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f283579c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f283015db10 0x7f283016a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f283579c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f283015db10 0x7f283016a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56966/0 (socket says 172.21.15.119:56966) 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f283579c700 1 -- 172.21.15.119:0/250877547 learned_addr learned my addr 172.21.15.119:0/250877547 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f282ffff700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 0x7f28300b5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f2834f9b700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2830159890 0x7f28300cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f283579c700 1 -- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2830159890 msgr2=0x7f28300cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f283579c700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2830159890 0x7f28300cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f283579c700 1 -- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 msgr2=0x7f28300b5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f283579c700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 0x7f28300b5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f283579c700 1 -- 172.21.15.119:0/250877547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c009420 con 0x7f283015db10 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.928+0000 7f282ffff700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 0x7f28300b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.929+0000 7f283579c700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f283015db10 0x7f283016a000 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f282800cae0 tx=0x7f2828007ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.929+0000 7f282cff9700 1 -- 172.21.15.119:0/250877547 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2828004dc0 con 0x7f283015db10 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.929+0000 7f282cff9700 1 -- 172.21.15.119:0/250877547 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282800a5d0 con 0x7f283015db10 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.929+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283016ae90 con 0x7f283015db10 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.929+0000 7f282cff9700 1 -- 172.21.15.119:0/250877547 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800a990 con 0x7f283015db10 2023-10-08T02:11:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.929+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28300cd6c0 con 0x7f283015db10 2023-10-08T02:11:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.930+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28300cda30 con 0x7f283015db10 2023-10-08T02:11:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.931+0000 7f282cff9700 1 -- 172.21.15.119:0/250877547 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2828007150 con 0x7f283015db10 2023-10-08T02:11:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.931+0000 7f282cff9700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.932+0000 7f282cff9700 1 -- 172.21.15.119:0/250877547 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 5919+0+0 (secure 0 0 0) 0x7f2828083ff0 con 0x7f283015db10 2023-10-08T02:11:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.932+0000 7f2834f9b700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.932+0000 7f2834f9b700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2814066f30 0x7f28140693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2824003a80 tx=0x7f2824007ec0 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:35.936+0000 7f282cff9700 1 -- 172.21.15.119:0/250877547 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28280511b0 con 0x7f283015db10 2023-10-08T02:11:36.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:36.146+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f283004c3c0 con 0x7f283015db10 2023-10-08T02:11:36.663 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:11:36.667 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:11:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.030+0000 7f282cff9700 1 -- 172.21.15.119:0/250877547 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v153) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2828018020 con 0x7f283015db10 2023-10-08T02:11:37.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2814066f30 msgr2=0x7f28140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2814066f30 0x7f28140693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2824003a80 tx=0x7f2824007ec0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f283015db10 msgr2=0x7f283016a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f283015db10 0x7f283016a000 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f282800cae0 tx=0x7f2828007ca0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 shutdown_connections 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2814066f30 0x7f28140693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2830159890 0x7f28300cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f283015db10 0x7f283016a000 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 --2- 172.21.15.119:0/250877547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2830159c60 0x7f28300b5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 >> 172.21.15.119:0/250877547 conn(0x7f2830068e20 msgr2=0x7f28300cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 shutdown_connections 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.033+0000 7f28371ff700 1 -- 172.21.15.119:0/250877547 wait complete. 2023-10-08T02:11:37.035 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T02:11:37.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T02:11:37.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.231+0000 7f4fa0fff700 1 Processor -- start 2023-10-08T02:11:37.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.231+0000 7f4fa0fff700 1 -- start start 2023-10-08T02:11:37.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.232+0000 7f4fa0fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f9c159890 0x7f4f9c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.232+0000 7f4fa0fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c069560 0x7f4f9c0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.232+0000 7f4fa0fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f9c069ee0 con 0x7f4f9c159c60 2023-10-08T02:11:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.232+0000 7f4fa0fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9c15a160 con 0x7f4f9c159890 2023-10-08T02:11:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4fa0fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c0d13a0 con 0x7f4f9c069560 2023-10-08T02:11:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9a59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f9c159890 0x7f4f9c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9a59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f9c159890 0x7f4f9c15abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56978/0 (socket says 172.21.15.119:56978) 2023-10-08T02:11:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9a59c700 1 -- 172.21.15.119:0/2427943916 learned_addr learned my addr 172.21.15.119:0/2427943916 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9ad9d700 1 --2- 172.21.15.119:0/2427943916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c069560 0x7f4f9c0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9a59c700 1 -- 172.21.15.119:0/2427943916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c069560 msgr2=0x7f4f9c0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:37.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9a59c700 1 --2- 172.21.15.119:0/2427943916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c069560 0x7f4f9c0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9a59c700 1 -- 172.21.15.119:0/2427943916 >> v1:172.21.15.119:6789/0 conn(0x7f4f9c159c60 legacy=0x7f4f9c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9a59c700 1 -- 172.21.15.119:0/2427943916 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c15e140 con 0x7f4f9c159890 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9959a700 1 -- 172.21.15.119:0/2427943916 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1268543253 0 0) 0x7f4f9c069ee0 con 0x7f4f9c159c60 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.233+0000 7f4f9a59c700 1 --2- 172.21.15.119:0/2427943916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c159890 0x7f4f9c15abd0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0099b0 tx=0x7f4f8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ddb31a5abe6758 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4f9959a700 1 -- 172.21.15.119:0/2427943916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00b860 con 0x7f4f9c159890 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4f9959a700 1 -- 172.21.15.119:0/2427943916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c00b9c0 con 0x7f4f9c159890 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4f9959a700 1 -- 172.21.15.119:0/2427943916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00bd60 con 0x7f4f9c159890 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4fa0fff700 1 -- 172.21.15.119:0/2427943916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c159890 msgr2=0x7f4f9c15abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/2427943916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c159890 0x7f4f9c15abd0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c0099b0 tx=0x7f4f8c009250 comp rx=0 tx=0).stop 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4fa0fff700 1 -- 172.21.15.119:0/2427943916 shutdown_connections 2023-10-08T02:11:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/2427943916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c069560 0x7f4f9c0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/2427943916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c159890 0x7f4f9c15abd0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4fa0fff700 1 -- 172.21.15.119:0/2427943916 >> 172.21.15.119:0/2427943916 conn(0x7f4f9c068e20 msgr2=0x7f4f9c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:37.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4fa0fff700 1 -- 172.21.15.119:0/2427943916 shutdown_connections 2023-10-08T02:11:37.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.234+0000 7f4fa0fff700 1 -- 172.21.15.119:0/2427943916 wait complete. 2023-10-08T02:11:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.236+0000 7f4fa0fff700 1 Processor -- start 2023-10-08T02:11:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.236+0000 7f4fa0fff700 1 -- start start 2023-10-08T02:11:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4fa0fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c069560 0x7f4f9c16d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4fa0fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c159890 0x7f4f9c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4fa0fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9c159c60 0x7f4f9c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4fa0fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c1537e0 con 0x7f4f9c069560 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4fa0fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c16ddd0 con 0x7f4f9c159c60 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4fa0fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9c153a50 con 0x7f4f9c159890 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c069560 0x7f4f9c16d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c069560 0x7f4f9c16d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56992/0 (socket says 172.21.15.119:56992) 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9a59c700 1 -- 172.21.15.119:0/1307800682 learned_addr learned my addr 172.21.15.119:0/1307800682 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9ad9d700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9c159c60 0x7f4f9c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f99d9b700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c159890 0x7f4f9c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9a59c700 1 -- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c159890 msgr2=0x7f4f9c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9a59c700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c159890 0x7f4f9c0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9a59c700 1 -- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9c159c60 msgr2=0x7f4f9c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9a59c700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9c159c60 0x7f4f9c0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9a59c700 1 -- 172.21.15.119:0/1307800682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f8c0094f0 con 0x7f4f9c069560 2023-10-08T02:11:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.237+0000 7f4f9ad9d700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9c159c60 0x7f4f9c0cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.238+0000 7f4f9a59c700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c069560 0x7f4f9c16d890 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006fd0 tx=0x7f4f8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.238+0000 7f4f8a7fc700 1 -- 172.21.15.119:0/1307800682 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c011e90 con 0x7f4f9c069560 2023-10-08T02:11:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.238+0000 7f4f8a7fc700 1 -- 172.21.15.119:0/1307800682 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f8c024440 con 0x7f4f9c069560 2023-10-08T02:11:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.238+0000 7f4f8a7fc700 1 -- 172.21.15.119:0/1307800682 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c024820 con 0x7f4f9c069560 2023-10-08T02:11:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.238+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f9c16e000 con 0x7f4f9c069560 2023-10-08T02:11:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.238+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9c16e5c0 con 0x7f4f9c069560 2023-10-08T02:11:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.239+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f9c04c3c0 con 0x7f4f9c069560 2023-10-08T02:11:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.240+0000 7f4f8a7fc700 1 -- 172.21.15.119:0/1307800682 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f8c01a420 con 0x7f4f9c069560 2023-10-08T02:11:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.241+0000 7f4f8a7fc700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4f7c066f30 0x7f4f7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.241+0000 7f4f8a7fc700 1 -- 172.21.15.119:0/1307800682 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f4f8c086480 con 0x7f4f9c069560 2023-10-08T02:11:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.241+0000 7f4f99d9b700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4f7c066f30 0x7f4f7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.242+0000 7f4f99d9b700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4f7c066f30 0x7f4f7c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4f900011e0 tx=0x7f4f90009040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.245+0000 7f4f8a7fc700 1 -- 172.21.15.119:0/1307800682 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f8c0537d0 con 0x7f4f9c069560 2023-10-08T02:11:37.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.508+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4f9c16e780 con 0x7f4f9c069560 2023-10-08T02:11:37.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.508+0000 7f4f8a7fc700 1 -- 172.21.15.119:0/1307800682 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v42) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4f8c056df0 con 0x7f4f9c069560 2023-10-08T02:11:37.509 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:37.509 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:11:37.509 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T02:11:37.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.509 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T02:11:37.510 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.511 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:11:37.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T02:11:37.514 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:11:37.515 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.516 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.517 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:11:37.518 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.519 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T02:11:37.520 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:11:37.521 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:37.523 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T02:11:37.524 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:11:37.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4f7c066f30 msgr2=0x7f4f7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4f7c066f30 0x7f4f7c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4f900011e0 tx=0x7f4f90009040 comp rx=0 tx=0).stop 2023-10-08T02:11:37.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c069560 msgr2=0x7f4f9c16d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c069560 0x7f4f9c16d890 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c006fd0 tx=0x7f4f8c017000 comp rx=0 tx=0).stop 2023-10-08T02:11:37.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 shutdown_connections 2023-10-08T02:11:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4f7c066f30 0x7f4f7c0693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f9c159c60 0x7f4f9c0cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f9c069560 0x7f4f9c16d890 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 --2- 172.21.15.119:0/1307800682 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f9c159890 0x7f4f9c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 >> 172.21.15.119:0/1307800682 conn(0x7f4f9c068e20 msgr2=0x7f4f9c152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 shutdown_connections 2023-10-08T02:11:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.511+0000 7f4fa0fff700 1 -- 172.21.15.119:0/1307800682 wait complete. 2023-10-08T02:11:37.529 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:11:37.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.712+0000 7f8b7d9ff700 1 Processor -- start 2023-10-08T02:11:37.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.712+0000 7f8b7d9ff700 1 -- start start 2023-10-08T02:11:37.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b7d9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b78155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b7d9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b780acac0 con 0x7f8b780ac230 2023-10-08T02:11:37.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b7d9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b780ad930 con 0x7f8b780ac6a0 2023-10-08T02:11:37.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b7d9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780acd20 con 0x7f8b78155c10 2023-10-08T02:11:37.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b6ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8b780ac230 0x7f8b781568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38538/0 (socket says 172.21.15.119:38538) 2023-10-08T02:11:37.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b6ffff700 1 -- 172.21.15.119:0/2281031187 learned_addr learned my addr 172.21.15.119:0/2281031187 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:37.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b76ffd700 1 --2- 172.21.15.119:0/2281031187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b78155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b767fc700 1 -- 172.21.15.119:0/2281031187 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521426643 0 0) 0x7f8b780acac0 con 0x7f8b780ac230 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.714+0000 7f8b767fc700 1 -- 172.21.15.119:0/2281031187 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b64003570 con 0x7f8b780ac230 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b767fc700 1 -- 172.21.15.119:0/2281031187 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3705282714 0 0) 0x7f8b780ad930 con 0x7f8b780ac6a0 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b767fc700 1 -- 172.21.15.119:0/2281031187 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b780acac0 con 0x7f8b780ac6a0 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b767fc700 1 -- 172.21.15.119:0/2281031187 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3269653375 0 0) 0x7f8b64003570 con 0x7f8b780ac230 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b76ffd700 1 -- 172.21.15.119:0/2281031187 >> v1:172.21.15.184:6789/0 conn(0x7f8b780ac6a0 legacy=0x7f8b780b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b76ffd700 1 -- 172.21.15.119:0/2281031187 >> v1:172.21.15.119:6789/0 conn(0x7f8b780ac230 legacy=0x7f8b781568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b76ffd700 1 -- 172.21.15.119:0/2281031187 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780ad000 con 0x7f8b78155c10 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b76ffd700 1 --2- 172.21.15.119:0/2281031187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b78155fe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009910 tx=0x7f8b680091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d806a1a3d6ca843 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b767fc700 1 -- 172.21.15.119:0/2281031187 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60001470 con 0x7f8b78155c10 2023-10-08T02:11:37.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b767fc700 1 -- 172.21.15.119:0/2281031187 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b68011420 con 0x7f8b78155c10 2023-10-08T02:11:37.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.715+0000 7f8b767fc700 1 -- 172.21.15.119:0/2281031187 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b680117c0 con 0x7f8b78155c10 2023-10-08T02:11:37.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.716+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/2281031187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 msgr2=0x7f8b78155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.716+0000 7f8b7d9ff700 1 --2- 172.21.15.119:0/2281031187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b78155fe0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009910 tx=0x7f8b680091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.716+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/2281031187 shutdown_connections 2023-10-08T02:11:37.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.716+0000 7f8b7d9ff700 1 --2- 172.21.15.119:0/2281031187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b78155fe0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.716+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/2281031187 >> 172.21.15.119:0/2281031187 conn(0x7f8b7806a320 msgr2=0x7f8b780cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:37.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.716+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/2281031187 shutdown_connections 2023-10-08T02:11:37.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.716+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/2281031187 wait complete. 2023-10-08T02:11:37.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.717+0000 7f8b7d9ff700 1 Processor -- start 2023-10-08T02:11:37.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.718+0000 7f8b7d9ff700 1 -- start start 2023-10-08T02:11:37.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.718+0000 7f8b7d9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b780ac230 0x7f8b7806f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.718+0000 7f8b7d9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b780ac6a0 0x7f8b780a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b7d9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b780a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b7d9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7806f870 con 0x7f8b780ac230 2023-10-08T02:11:37.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b7d9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780aa4f0 con 0x7f8b780ac6a0 2023-10-08T02:11:37.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b7d9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780aa7d0 con 0x7f8b78155c10 2023-10-08T02:11:37.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b780ac230 0x7f8b7806f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b780ac230 0x7f8b7806f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56996/0 (socket says 172.21.15.119:56996) 2023-10-08T02:11:37.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 -- 172.21.15.119:0/932009763 learned_addr learned my addr 172.21.15.119:0/932009763 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:37.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b777fe700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b780a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b6ffff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b780ac6a0 0x7f8b780a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 -- 172.21.15.119:0/932009763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 msgr2=0x7f8b780a9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b780a9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 -- 172.21.15.119:0/932009763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b780ac6a0 msgr2=0x7f8b780a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b780ac6a0 0x7f8b780a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 -- 172.21.15.119:0/932009763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b68009420 con 0x7f8b780ac230 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.719+0000 7f8b76ffd700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b780ac230 0x7f8b7806f330 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f8b680091e0 tx=0x7f8b68016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.720+0000 7f8b6effd700 1 -- 172.21.15.119:0/932009763 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6801cc90 con 0x7f8b780ac230 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.720+0000 7f8b6effd700 1 -- 172.21.15.119:0/932009763 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6801cdf0 con 0x7f8b780ac230 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.720+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b780aaab0 con 0x7f8b780ac230 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.720+0000 7f8b6effd700 1 -- 172.21.15.119:0/932009763 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6801a6a0 con 0x7f8b780ac230 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.720+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b780ab070 con 0x7f8b780ac230 2023-10-08T02:11:37.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.721+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7804c3c0 con 0x7f8b780ac230 2023-10-08T02:11:37.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.722+0000 7f8b6effd700 1 -- 172.21.15.119:0/932009763 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b6801aa90 con 0x7f8b780ac230 2023-10-08T02:11:37.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.723+0000 7f8b6effd700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8b58066f30 0x7f8b580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:37.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.723+0000 7f8b6effd700 1 -- 172.21.15.119:0/932009763 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f8b680862b0 con 0x7f8b780ac230 2023-10-08T02:11:37.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.723+0000 7f8b6ffff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8b58066f30 0x7f8b580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:37.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.724+0000 7f8b6ffff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8b58066f30 0x7f8b580693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8b78167420 tx=0x7f8b600015c0 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:37.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.727+0000 7f8b6effd700 1 -- 172.21.15.119:0/932009763 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b68053600 con 0x7f8b780ac230 2023-10-08T02:11:37.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.921+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b780ad2e0 con 0x7f8b780ac230 2023-10-08T02:11:37.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.922+0000 7f8b6effd700 1 -- 172.21.15.119:0/932009763 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) v1 ==== 74+0+12905 (secure 0 0 0) 0x7f8b68056c20 con 0x7f8b780ac230 2023-10-08T02:11:37.922 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:37.922 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":153,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:11:37.027859+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:11:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8b58066f30 msgr2=0x7f8b580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8b58066f30 0x7f8b580693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8b78167420 tx=0x7f8b600015c0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b780ac230 msgr2=0x7f8b7806f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:37.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b780ac230 0x7f8b7806f330 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f8b680091e0 tx=0x7f8b68016000 comp rx=0 tx=0).stop 2023-10-08T02:11:37.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 shutdown_connections 2023-10-08T02:11:37.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8b58066f30 0x7f8b580693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b78155c10 0x7f8b780a9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b780ac230 0x7f8b7806f330 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 --2- 172.21.15.119:0/932009763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b780ac6a0 0x7f8b780a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:37.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 >> 172.21.15.119:0/932009763 conn(0x7f8b7806a320 msgr2=0x7f8b780af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:37.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 shutdown_connections 2023-10-08T02:11:37.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:37.924+0000 7f8b7d9ff700 1 -- 172.21.15.119:0/932009763 wait complete. 2023-10-08T02:11:37.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T02:11:37.943 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:11:38.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.126+0000 7f04d89ff700 1 Processor -- start 2023-10-08T02:11:38.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.127+0000 7f04d89ff700 1 -- start start 2023-10-08T02:11:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d89ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 0x7f04d406e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d89ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f04d40c70b0 0x7f04d4069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d89ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f04d4069cb0 0x7f04d4159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d89ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d406a130 con 0x7f04d40c70b0 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d89ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d406eb90 con 0x7f04d4069cb0 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d89ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d406ee70 con 0x7f04d4155eb0 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d1d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f04d40c70b0 0x7f04d4069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d1d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f04d40c70b0 0x7f04d4069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57006/0 (socket says 172.21.15.119:57006) 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d1d9b700 1 -- 172.21.15.119:0/2397786308 learned_addr learned my addr 172.21.15.119:0/2397786308 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d259c700 1 --2- 172.21.15.119:0/2397786308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 0x7f04d406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d2d9d700 1 --2- 172.21.15.119:0/2397786308 >> v2:172.21.15.184:3300/0 conn(0x7f04d4069cb0 0x7f04d4159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d1d9b700 1 -- 172.21.15.119:0/2397786308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 msgr2=0x7f04d406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.128+0000 7f04d1d9b700 1 --2- 172.21.15.119:0/2397786308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 0x7f04d406e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d1d9b700 1 -- 172.21.15.119:0/2397786308 >> v2:172.21.15.184:3300/0 conn(0x7f04d4069cb0 msgr2=0x7f04d4159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d1d9b700 1 --2- 172.21.15.119:0/2397786308 >> v2:172.21.15.184:3300/0 conn(0x7f04d4069cb0 0x7f04d4159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d1d9b700 1 -- 172.21.15.119:0/2397786308 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d406f150 con 0x7f04d40c70b0 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d1d9b700 1 --2- 172.21.15.119:0/2397786308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d4069770 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f04d40c7740 tx=0x7f04c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c388607ececff91f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d159a700 1 -- 172.21.15.119:0/2397786308 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c8009ea0 con 0x7f04d40c70b0 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d159a700 1 -- 172.21.15.119:0/2397786308 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c800b820 con 0x7f04d40c70b0 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d159a700 1 -- 172.21.15.119:0/2397786308 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800bbc0 con 0x7f04d40c70b0 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d89ff700 1 -- 172.21.15.119:0/2397786308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 msgr2=0x7f04d4069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.129+0000 7f04d89ff700 1 --2- 172.21.15.119:0/2397786308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d4069770 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f04d40c7740 tx=0x7f04c8009250 comp rx=0 tx=0).stop 2023-10-08T02:11:38.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.130+0000 7f04d89ff700 1 -- 172.21.15.119:0/2397786308 shutdown_connections 2023-10-08T02:11:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.130+0000 7f04d89ff700 1 --2- 172.21.15.119:0/2397786308 >> v2:172.21.15.184:3300/0 conn(0x7f04d4069cb0 0x7f04d4159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.130+0000 7f04d89ff700 1 --2- 172.21.15.119:0/2397786308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 0x7f04d406e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.130+0000 7f04d89ff700 1 --2- 172.21.15.119:0/2397786308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d4069770 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.130+0000 7f04d89ff700 1 -- 172.21.15.119:0/2397786308 >> 172.21.15.119:0/2397786308 conn(0x7f04d4068e10 msgr2=0x7f04d40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.130+0000 7f04d89ff700 1 -- 172.21.15.119:0/2397786308 shutdown_connections 2023-10-08T02:11:38.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.130+0000 7f04d89ff700 1 -- 172.21.15.119:0/2397786308 wait complete. 2023-10-08T02:11:38.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.131+0000 7f04d89ff700 1 Processor -- start 2023-10-08T02:11:38.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.131+0000 7f04d89ff700 1 -- start start 2023-10-08T02:11:38.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d89ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d4069cb0 0x7f04d41550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d89ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d40c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d89ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 0x7f04d4169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d89ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40c5550 con 0x7f04d40c70b0 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d89ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40c50c0 con 0x7f04d4069cb0 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d89ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40c44b0 con 0x7f04d4155eb0 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d40c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d1d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d40c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57014/0 (socket says 172.21.15.119:57014) 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.132+0000 7f04d1d9b700 1 -- 172.21.15.119:0/547878824 learned_addr learned my addr 172.21.15.119:0/547878824 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04d1d9b700 1 -- 172.21.15.119:0/547878824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 msgr2=0x7f04d4169b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04d1d9b700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 0x7f04d4169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04d1d9b700 1 -- 172.21.15.119:0/547878824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d4069cb0 msgr2=0x7f04d41550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04d1d9b700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d4069cb0 0x7f04d41550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04d1d9b700 1 -- 172.21.15.119:0/547878824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c80094f0 con 0x7f04d40c70b0 2023-10-08T02:11:38.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04d1d9b700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d40c3a60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f04c80094c0 tx=0x7f04c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04c27fc700 1 -- 172.21.15.119:0/547878824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c800bba0 con 0x7f04d40c70b0 2023-10-08T02:11:38.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04c27fc700 1 -- 172.21.15.119:0/547878824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c8011420 con 0x7f04d40c70b0 2023-10-08T02:11:38.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04c27fc700 1 -- 172.21.15.119:0/547878824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80117c0 con 0x7f04d40c70b0 2023-10-08T02:11:38.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d40c4790 con 0x7f04d40c70b0 2023-10-08T02:11:38.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.133+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d40c4d50 con 0x7f04d40c70b0 2023-10-08T02:11:38.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.134+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d404c3c0 con 0x7f04d40c70b0 2023-10-08T02:11:38.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.135+0000 7f04c27fc700 1 -- 172.21.15.119:0/547878824 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f04c8011bf0 con 0x7f04d40c70b0 2023-10-08T02:11:38.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.136+0000 7f04c27fc700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f04b4066f30 0x7f04b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.136+0000 7f04c27fc700 1 -- 172.21.15.119:0/547878824 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f04c8085720 con 0x7f04d40c70b0 2023-10-08T02:11:38.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.137+0000 7f04d259c700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f04b4066f30 0x7f04b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.137+0000 7f04d259c700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f04b4066f30 0x7f04b40693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009b10 tx=0x7f04c40091a0 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.140+0000 7f04c27fc700 1 -- 172.21.15.119:0/547878824 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04c8052a70 con 0x7f04d40c70b0 2023-10-08T02:11:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.330+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04d40c5230 con 0x7f04d40c70b0 2023-10-08T02:11:38.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.331+0000 7f04c27fc700 1 -- 172.21.15.119:0/547878824 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v153) v1 ==== 93+0+10 (secure 0 0 0) 0x7f04c80150a0 con 0x7f04d40c70b0 2023-10-08T02:11:38.332 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:11:38.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f04b4066f30 msgr2=0x7f04b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f04b4066f30 0x7f04b40693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f04c4009b10 tx=0x7f04c40091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 msgr2=0x7f04d40c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d40c3a60 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f04c80094c0 tx=0x7f04c8017000 comp rx=0 tx=0).stop 2023-10-08T02:11:38.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 shutdown_connections 2023-10-08T02:11:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f04b4066f30 0x7f04b40693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04d4155eb0 0x7f04d4169b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04d4069cb0 0x7f04d41550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 --2- 172.21.15.119:0/547878824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04d40c70b0 0x7f04d40c3a60 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 >> 172.21.15.119:0/547878824 conn(0x7f04d4068e10 msgr2=0x7f04d40ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 shutdown_connections 2023-10-08T02:11:38.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.333+0000 7f04d89ff700 1 -- 172.21.15.119:0/547878824 wait complete. 2023-10-08T02:11:38.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T02:11:38.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.532+0000 7fdbe99ff700 1 Processor -- start 2023-10-08T02:11:38.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.532+0000 7fdbe99ff700 1 -- start start 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.533+0000 7fdbe99ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 0x7fdbe4155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.533+0000 7fdbe99ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdbe4156150 0x7fdbe40ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbe99ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbe40ad930 con 0x7fdbe40cd010 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbe99ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbe40acd20 con 0x7fdbe4156150 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbe99ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe40ad000 con 0x7fdbe40ccc40 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbdbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdbe4156150 0x7fdbe40ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbe37fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdbe40cd010 0x7fdbe40af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38554/0 (socket says 172.21.15.119:38554) 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbe37fe700 1 -- 172.21.15.119:0/366016387 learned_addr learned my addr 172.21.15.119:0/366016387 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbe2ffd700 1 --2- 172.21.15.119:0/366016387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 0x7fdbe4155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbdbfff700 1 -- 172.21.15.119:0/366016387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 msgr2=0x7fdbe4155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbdbfff700 1 --2- 172.21.15.119:0/366016387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 0x7fdbe4155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbdbfff700 1 -- 172.21.15.119:0/366016387 >> v1:172.21.15.119:6789/0 conn(0x7fdbe40cd010 legacy=0x7fdbe40af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbdbfff700 1 -- 172.21.15.119:0/366016387 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe40ad2e0 con 0x7fdbe4156150 2023-10-08T02:11:38.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.534+0000 7fdbdbfff700 1 --2- 172.21.15.119:0/366016387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe4156150 0x7fdbe40ac230 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098e0 tx=0x7fdbcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5838f57fa618d818 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe27fc700 1 -- 172.21.15.119:0/366016387 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc009e80 con 0x7fdbe4156150 2023-10-08T02:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe27fc700 1 -- 172.21.15.119:0/366016387 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc00b860 con 0x7fdbe4156150 2023-10-08T02:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe27fc700 1 -- 172.21.15.119:0/366016387 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00bbe0 con 0x7fdbe4156150 2023-10-08T02:11:38.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe99ff700 1 -- 172.21.15.119:0/366016387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe4156150 msgr2=0x7fdbe40ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/366016387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe4156150 0x7fdbe40ac230 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc0098e0 tx=0x7fdbcc0091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe99ff700 1 -- 172.21.15.119:0/366016387 shutdown_connections 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/366016387 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 0x7fdbe4155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/366016387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe4156150 0x7fdbe40ac230 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe99ff700 1 -- 172.21.15.119:0/366016387 >> 172.21.15.119:0/366016387 conn(0x7fdbe406a320 msgr2=0x7fdbe40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe99ff700 1 -- 172.21.15.119:0/366016387 shutdown_connections 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.535+0000 7fdbe99ff700 1 -- 172.21.15.119:0/366016387 wait complete. 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.537+0000 7fdbe99ff700 1 Processor -- start 2023-10-08T02:11:38.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.537+0000 7fdbe99ff700 1 -- start start 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe99ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 0x7fdbe415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe99ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40cd010 0x7fdbe4154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe99ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe4156150 0x7fdbe41579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe99ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe415ec80 con 0x7fdbe40cd010 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe99ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe415d1c0 con 0x7fdbe4156150 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe99ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe415f1f0 con 0x7fdbe40ccc40 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbdbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40cd010 0x7fdbe4154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbdbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40cd010 0x7fdbe4154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57030/0 (socket says 172.21.15.119:57030) 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbdbfff700 1 -- 172.21.15.119:0/3208695000 learned_addr learned my addr 172.21.15.119:0/3208695000 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe2ffd700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 0x7fdbe415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe37fe700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe4156150 0x7fdbe41579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbdbfff700 1 -- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 msgr2=0x7fdbe415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbdbfff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 0x7fdbe415d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbdbfff700 1 -- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe4156150 msgr2=0x7fdbe41579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbdbfff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe4156150 0x7fdbe41579c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbdbfff700 1 -- 172.21.15.119:0/3208695000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbcc009420 con 0x7fdbe40cd010 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.538+0000 7fdbe37fe700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe4156150 0x7fdbe41579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.539+0000 7fdbdbfff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40cd010 0x7fdbe4154510 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4064290 tx=0x7fdbcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.539+0000 7fdbdaffd700 1 -- 172.21.15.119:0/3208695000 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00bbe0 con 0x7fdbe40cd010 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.539+0000 7fdbdaffd700 1 -- 172.21.15.119:0/3208695000 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc011420 con 0x7fdbe40cd010 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.539+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe415e9a0 con 0x7fdbe40cd010 2023-10-08T02:11:38.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.539+0000 7fdbdaffd700 1 -- 172.21.15.119:0/3208695000 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc0117e0 con 0x7fdbe40cd010 2023-10-08T02:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.539+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe415ce50 con 0x7fdbe40cd010 2023-10-08T02:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.540+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe415c9e0 con 0x7fdbe40cd010 2023-10-08T02:11:38.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.541+0000 7fdbdaffd700 1 -- 172.21.15.119:0/3208695000 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdbcc011bd0 con 0x7fdbe40cd010 2023-10-08T02:11:38.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.542+0000 7fdbdaffd700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdbc4066f30 0x7fdbc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.542+0000 7fdbdaffd700 1 -- 172.21.15.119:0/3208695000 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5317+0+0 (secure 0 0 0) 0x7fdbcc085d60 con 0x7fdbe40cd010 2023-10-08T02:11:38.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.542+0000 7fdbe2ffd700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdbc4066f30 0x7fdbc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.543+0000 7fdbe2ffd700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdbc4066f30 0x7fdbc40693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4006fd0 tx=0x7fdbd4008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.546+0000 7fdbdaffd700 1 -- 172.21.15.119:0/3208695000 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdbcc053150 con 0x7fdbe40cd010 2023-10-08T02:11:38.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.741+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdbe415d350 con 0x7fdbe40cd010 2023-10-08T02:11:38.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.741+0000 7fdbdaffd700 1 -- 172.21.15.119:0/3208695000 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v153) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdbcc056770 con 0x7fdbe40cd010 2023-10-08T02:11:38.742 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:38.742 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T02:11:38.742 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:38.742 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T02:11:38.743 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T02:11:38.743 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:38.743 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdbc4066f30 msgr2=0x7fdbc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdbc4066f30 0x7fdbc40693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdbd4006fd0 tx=0x7fdbd4008040 comp rx=0 tx=0).stop 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40cd010 msgr2=0x7fdbe4154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40cd010 0x7fdbe4154510 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4064290 tx=0x7fdbcc017000 comp rx=0 tx=0).stop 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 shutdown_connections 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdbc4066f30 0x7fdbc40693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdbe40ccc40 0x7fdbe415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdbe4156150 0x7fdbe41579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 --2- 172.21.15.119:0/3208695000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdbe40cd010 0x7fdbe4154510 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 >> 172.21.15.119:0/3208695000 conn(0x7fdbe406a320 msgr2=0x7fdbe40af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:38.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 shutdown_connections 2023-10-08T02:11:38.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.744+0000 7fdbe99ff700 1 -- 172.21.15.119:0/3208695000 wait complete. 2023-10-08T02:11:38.762 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T02:11:38.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T02:11:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.948+0000 7fc71ffff700 1 Processor -- start 2023-10-08T02:11:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.948+0000 7fc71ffff700 1 -- start start 2023-10-08T02:11:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.948+0000 7fc71ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc720156350 0x7fc720157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.948+0000 7fc71ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7200b0c00 0x7fc72015a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.948+0000 7fc71ffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7200b0570 con 0x7fc7200b0760 2023-10-08T02:11:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.948+0000 7fc71ffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7201578b0 con 0x7fc7200b0c00 2023-10-08T02:11:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.948+0000 7fc71ffff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7200af4d0 con 0x7fc720156350 2023-10-08T02:11:38.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71e7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc7200b0760 0x7fc720159f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38562/0 (socket says 172.21.15.119:38562) 2023-10-08T02:11:38.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71e7fc700 1 -- 172.21.15.119:0/905442850 learned_addr learned my addr 172.21.15.119:0/905442850 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:38.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71f7fe700 1 --2- 172.21.15.119:0/905442850 >> v2:172.21.15.119:3300/0 conn(0x7fc7200b0c00 0x7fc72015a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71effd700 1 --2- 172.21.15.119:0/905442850 >> v2:172.21.15.184:3300/0 conn(0x7fc720156350 0x7fc720157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71f7fe700 1 -- 172.21.15.119:0/905442850 >> v2:172.21.15.184:3300/0 conn(0x7fc720156350 msgr2=0x7fc720157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:38.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71f7fe700 1 --2- 172.21.15.119:0/905442850 >> v2:172.21.15.184:3300/0 conn(0x7fc720156350 0x7fc720157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71f7fe700 1 -- 172.21.15.119:0/905442850 >> v1:172.21.15.119:6789/0 conn(0x7fc7200b0760 legacy=0x7fc720159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71f7fe700 1 -- 172.21.15.119:0/905442850 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7200af7b0 con 0x7fc7200b0c00 2023-10-08T02:11:38.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71dffb700 1 -- 172.21.15.119:0/905442850 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890833134 0 0) 0x7fc7200b0570 con 0x7fc7200b0760 2023-10-08T02:11:38.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.949+0000 7fc71f7fe700 1 --2- 172.21.15.119:0/905442850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc72015a630 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc720065950 tx=0x7fc718009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff4af775a850c21d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71dffb700 1 -- 172.21.15.119:0/905442850 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71800e070 con 0x7fc7200b0c00 2023-10-08T02:11:38.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71dffb700 1 -- 172.21.15.119:0/905442850 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71800bc30 con 0x7fc7200b0c00 2023-10-08T02:11:38.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71dffb700 1 -- 172.21.15.119:0/905442850 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc718012430 con 0x7fc7200b0c00 2023-10-08T02:11:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71ffff700 1 -- 172.21.15.119:0/905442850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 msgr2=0x7fc72015a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71ffff700 1 --2- 172.21.15.119:0/905442850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc72015a630 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc720065950 tx=0x7fc718009780 comp rx=0 tx=0).stop 2023-10-08T02:11:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71ffff700 1 -- 172.21.15.119:0/905442850 shutdown_connections 2023-10-08T02:11:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71ffff700 1 --2- 172.21.15.119:0/905442850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc72015a630 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71ffff700 1 --2- 172.21.15.119:0/905442850 >> v2:172.21.15.184:3300/0 conn(0x7fc720156350 0x7fc720157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71ffff700 1 -- 172.21.15.119:0/905442850 >> 172.21.15.119:0/905442850 conn(0x7fc720068e20 msgr2=0x7fc7200cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71ffff700 1 -- 172.21.15.119:0/905442850 shutdown_connections 2023-10-08T02:11:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.950+0000 7fc71ffff700 1 -- 172.21.15.119:0/905442850 wait complete. 2023-10-08T02:11:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.952+0000 7fc71ffff700 1 Processor -- start 2023-10-08T02:11:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.952+0000 7fc71ffff700 1 -- start start 2023-10-08T02:11:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7200b0760 0x7fc7200c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc7201699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc720156350 0x7fc7200ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200aac40 con 0x7fc7200b0c00 2023-10-08T02:11:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200aa030 con 0x7fc720156350 2023-10-08T02:11:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7200aa310 con 0x7fc7200b0760 2023-10-08T02:11:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc7201699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc7201699a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57040/0 (socket says 172.21.15.119:57040) 2023-10-08T02:11:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71e7fc700 1 -- 172.21.15.119:0/1867896756 learned_addr learned my addr 172.21.15.119:0/1867896756 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71f7fe700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc720156350 0x7fc7200ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71effd700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7200b0760 0x7fc7200c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71e7fc700 1 -- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7200b0760 msgr2=0x7fc7200c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71e7fc700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7200b0760 0x7fc7200c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71e7fc700 1 -- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc720156350 msgr2=0x7fc7200ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71e7fc700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc720156350 0x7fc7200ab0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71e7fc700 1 -- 172.21.15.119:0/1867896756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc718009a20 con 0x7fc7200b0c00 2023-10-08T02:11:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71f7fe700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc720156350 0x7fc7200ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.953+0000 7fc71effd700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7200b0760 0x7fc7200c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.954+0000 7fc71e7fc700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc7201699a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c5b0 tx=0x7fc714001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.954+0000 7fc706ffd700 1 -- 172.21.15.119:0/1867896756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714005b30 con 0x7fc7200b0c00 2023-10-08T02:11:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.954+0000 7fc706ffd700 1 -- 172.21.15.119:0/1867896756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc714005c90 con 0x7fc7200b0c00 2023-10-08T02:11:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.954+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7200aa5f0 con 0x7fc7200b0c00 2023-10-08T02:11:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.954+0000 7fc706ffd700 1 -- 172.21.15.119:0/1867896756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714007300 con 0x7fc7200b0c00 2023-10-08T02:11:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.954+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7200c21c0 con 0x7fc7200b0c00 2023-10-08T02:11:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.955+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7200aadf0 con 0x7fc7200b0c00 2023-10-08T02:11:38.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.956+0000 7fc706ffd700 1 -- 172.21.15.119:0/1867896756 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc71401b3b0 con 0x7fc7200b0c00 2023-10-08T02:11:38.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.957+0000 7fc706ffd700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc70c066f30 0x7fc70c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:38.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.957+0000 7fc706ffd700 1 -- 172.21.15.119:0/1867896756 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5317+0+0 (secure 0 0 0) 0x7fc7140838f0 con 0x7fc7200b0c00 2023-10-08T02:11:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.957+0000 7fc71effd700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc70c066f30 0x7fc70c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.958+0000 7fc71effd700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc70c066f30 0x7fc70c0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc710006fd0 tx=0x7fc710008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:38.961+0000 7fc706ffd700 1 -- 172.21.15.119:0/1867896756 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7200aadf0 con 0x7fc7200b0c00 2023-10-08T02:11:39.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:39.151+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc72004c3c0 con 0x7fc7200b0c00 2023-10-08T02:11:40.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.065+0000 7fc706ffd700 1 -- 172.21.15.119:0/1867896756 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v154) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc7140170b0 con 0x7fc7200b0c00 2023-10-08T02:11:40.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.069+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc70c066f30 msgr2=0x7fc70c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:40.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.069+0000 7fc71ffff700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc70c066f30 0x7fc70c0693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc710006fd0 tx=0x7fc710008040 comp rx=0 tx=0).stop 2023-10-08T02:11:40.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.069+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 msgr2=0x7fc7201699a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:40.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.069+0000 7fc71ffff700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc7201699a0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc71400c5b0 tx=0x7fc714001070 comp rx=0 tx=0).stop 2023-10-08T02:11:40.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.070+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 shutdown_connections 2023-10-08T02:11:40.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.070+0000 7fc71ffff700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc70c066f30 0x7fc70c0693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:40.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.070+0000 7fc71ffff700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7200b0760 0x7fc7200c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:40.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.070+0000 7fc71ffff700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc720156350 0x7fc7200ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:40.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.070+0000 7fc71ffff700 1 --2- 172.21.15.119:0/1867896756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7200b0c00 0x7fc7201699a0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:40.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.070+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 >> 172.21.15.119:0/1867896756 conn(0x7fc720068e20 msgr2=0x7fc7200add60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:40.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.071+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 shutdown_connections 2023-10-08T02:11:40.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.071+0000 7fc71ffff700 1 -- 172.21.15.119:0/1867896756 wait complete. 2023-10-08T02:11:40.074 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T02:11:40.094 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T02:11:40.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.286+0000 7f3e455ff700 1 Processor -- start 2023-10-08T02:11:40.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.286+0000 7f3e455ff700 1 -- start start 2023-10-08T02:11:40.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.287+0000 7f3e455ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e40159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.287+0000 7f3e455ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4015e880 con 0x7f3e4006a570 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.287+0000 7f3e455ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e4015e3f0 con 0x7f3e40069780 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.287+0000 7f3e455ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e400cba40 con 0x7f3e400c8590 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.287+0000 7f3e3effd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3e4006a570 0x7f3e400cb1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38570/0 (socket says 172.21.15.119:38570) 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.287+0000 7f3e3effd700 1 -- 172.21.15.119:0/1441012845 learned_addr learned my addr 172.21.15.119:0/1441012845 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.288+0000 7f3e3f7fe700 1 --2- 172.21.15.119:0/1441012845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e40159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.288+0000 7f3e3e7fc700 1 -- 172.21.15.119:0/1441012845 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420080402 0 0) 0x7f3e30001110 con 0x7f3e4006a570 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.288+0000 7f3e3e7fc700 1 -- 172.21.15.119:0/1441012845 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e2c003570 con 0x7f3e4006a570 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.288+0000 7f3e3e7fc700 1 -- 172.21.15.119:0/1441012845 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472689126 0 0) 0x7f3e4015e3f0 con 0x7f3e40069780 2023-10-08T02:11:40.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.288+0000 7f3e3e7fc700 1 -- 172.21.15.119:0/1441012845 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e30001110 con 0x7f3e40069780 2023-10-08T02:11:40.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.288+0000 7f3e3f7fe700 1 -- 172.21.15.119:0/1441012845 >> v1:172.21.15.184:6789/0 conn(0x7f3e40069780 legacy=0x7f3e401597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:40.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.288+0000 7f3e3f7fe700 1 -- 172.21.15.119:0/1441012845 >> v1:172.21.15.119:6789/0 conn(0x7f3e4006a570 legacy=0x7f3e400cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:40.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.288+0000 7f3e3f7fe700 1 -- 172.21.15.119:0/1441012845 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e4015dac0 con 0x7f3e400c8590 2023-10-08T02:11:40.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e3f7fe700 1 --2- 172.21.15.119:0/1441012845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e40159eb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009ec0 tx=0x7f3e38009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d99a804e7654b1e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:40.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e3e7fc700 1 -- 172.21.15.119:0/1441012845 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3800e070 con 0x7f3e400c8590 2023-10-08T02:11:40.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e3e7fc700 1 -- 172.21.15.119:0/1441012845 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3800bb40 con 0x7f3e400c8590 2023-10-08T02:11:40.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e3e7fc700 1 -- 172.21.15.119:0/1441012845 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e38012430 con 0x7f3e400c8590 2023-10-08T02:11:40.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e455ff700 1 -- 172.21.15.119:0/1441012845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 msgr2=0x7f3e40159eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:40.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e455ff700 1 --2- 172.21.15.119:0/1441012845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e40159eb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3e38009ec0 tx=0x7f3e38009780 comp rx=0 tx=0).stop 2023-10-08T02:11:40.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e455ff700 1 -- 172.21.15.119:0/1441012845 shutdown_connections 2023-10-08T02:11:40.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e455ff700 1 --2- 172.21.15.119:0/1441012845 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e40159eb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:40.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.289+0000 7f3e455ff700 1 -- 172.21.15.119:0/1441012845 >> 172.21.15.119:0/1441012845 conn(0x7f3e400c9870 msgr2=0x7f3e400c9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:40.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.290+0000 7f3e455ff700 1 -- 172.21.15.119:0/1441012845 shutdown_connections 2023-10-08T02:11:40.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.290+0000 7f3e455ff700 1 -- 172.21.15.119:0/1441012845 wait complete. 2023-10-08T02:11:40.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.291+0000 7f3e455ff700 1 Processor -- start 2023-10-08T02:11:40.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.291+0000 7f3e455ff700 1 -- start start 2023-10-08T02:11:40.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e455ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e40069780 0x7f3e40159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:40.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e455ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e4006a570 0x7f3e401426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:40.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e455ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e401559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:40.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e455ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e40155550 con 0x7f3e40069780 2023-10-08T02:11:40.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e455ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e40154940 con 0x7f3e4006a570 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e455ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e40154c20 con 0x7f3e400c8590 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e3effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e40069780 0x7f3e40159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e3effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e40069780 0x7f3e40159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57056/0 (socket says 172.21.15.119:57056) 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e3effd700 1 -- 172.21.15.119:0/3745574881 learned_addr learned my addr 172.21.15.119:0/3745574881 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.292+0000 7f3e3f7fe700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e401559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e37fff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e4006a570 0x7f3e401426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e3effd700 1 -- 172.21.15.119:0/3745574881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 msgr2=0x7f3e401559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e3effd700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e401559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e3effd700 1 -- 172.21.15.119:0/3745574881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e4006a570 msgr2=0x7f3e401426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e3effd700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e4006a570 0x7f3e401426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:40.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e3effd700 1 -- 172.21.15.119:0/3745574881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38009a20 con 0x7f3e40069780 2023-10-08T02:11:40.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e3effd700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e40069780 0x7f3e40159570 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f3e300058d0 tx=0x7f3e30007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:40.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e36ffd700 1 -- 172.21.15.119:0/3745574881 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30005b30 con 0x7f3e40069780 2023-10-08T02:11:40.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e36ffd700 1 -- 172.21.15.119:0/3745574881 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e30005c90 con 0x7f3e40069780 2023-10-08T02:11:40.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e40154f00 con 0x7f3e40069780 2023-10-08T02:11:40.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.293+0000 7f3e36ffd700 1 -- 172.21.15.119:0/3745574881 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e30007300 con 0x7f3e40069780 2023-10-08T02:11:40.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.294+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e40159ab0 con 0x7f3e40069780 2023-10-08T02:11:40.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.294+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e400718b0 con 0x7f3e40069780 2023-10-08T02:11:40.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.296+0000 7f3e36ffd700 1 -- 172.21.15.119:0/3745574881 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3e3001b400 con 0x7f3e40069780 2023-10-08T02:11:40.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.296+0000 7f3e36ffd700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3e20066f30 0x7f3e200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:40.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.296+0000 7f3e36ffd700 1 -- 172.21.15.119:0/3745574881 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f3e30083fa0 con 0x7f3e40069780 2023-10-08T02:11:40.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.297+0000 7f3e37fff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3e20066f30 0x7f3e200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:40.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.297+0000 7f3e37fff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3e20066f30 0x7f3e200693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3e400afcb0 tx=0x7f3e28007d30 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:40.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.301+0000 7f3e36ffd700 1 -- 172.21.15.119:0/3745574881 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e30051200 con 0x7f3e40069780 2023-10-08T02:11:40.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:40.491+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3e4015e880 con 0x7f3e40069780 2023-10-08T02:11:41.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.072+0000 7f3e36ffd700 1 -- 172.21.15.119:0/3745574881 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v155) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3e4015e880 con 0x7f3e40069780 2023-10-08T02:11:41.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.082+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3e20066f30 msgr2=0x7f3e200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:41.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.082+0000 7f3e455ff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3e20066f30 0x7f3e200693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3e400afcb0 tx=0x7f3e28007d30 comp rx=0 tx=0).stop 2023-10-08T02:11:41.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.082+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e40069780 msgr2=0x7f3e40159570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:41.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.082+0000 7f3e455ff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e40069780 0x7f3e40159570 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f3e300058d0 tx=0x7f3e30007ea0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.084+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 shutdown_connections 2023-10-08T02:11:41.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.084+0000 7f3e455ff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3e20066f30 0x7f3e200693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.084+0000 7f3e455ff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e400c8590 0x7f3e401559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.084+0000 7f3e455ff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e40069780 0x7f3e40159570 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.084+0000 7f3e455ff700 1 --2- 172.21.15.119:0/3745574881 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e4006a570 0x7f3e401426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.084+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 >> 172.21.15.119:0/3745574881 conn(0x7f3e400c9870 msgr2=0x7f3e400c30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:41.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.085+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 shutdown_connections 2023-10-08T02:11:41.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.086+0000 7f3e455ff700 1 -- 172.21.15.119:0/3745574881 wait complete. 2023-10-08T02:11:41.095 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T02:11:41.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T02:11:41.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.296+0000 7f85dcdff700 1 Processor -- start 2023-10-08T02:11:41.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.296+0000 7f85dcdff700 1 -- start start 2023-10-08T02:11:41.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.297+0000 7f85dcdff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f85d8155eb0 0x7f85d806e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:41.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85dcdff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85d8069e80 0x7f85d80ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85dcdff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d806eb90 con 0x7f85d80c70b0 2023-10-08T02:11:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85dcdff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d806ee70 con 0x7f85d8069e80 2023-10-08T02:11:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85dcdff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d806f150 con 0x7f85d8155eb0 2023-10-08T02:11:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85d6d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85d8069e80 0x7f85d80ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85d6d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85d8069e80 0x7f85d80ca4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57068/0 (socket says 172.21.15.119:57068) 2023-10-08T02:11:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85d6d9d700 1 -- 172.21.15.119:0/2466599106 learned_addr learned my addr 172.21.15.119:0/2466599106 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85d659c700 1 --2- 172.21.15.119:0/2466599106 >> v2:172.21.15.184:3300/0 conn(0x7f85d8155eb0 0x7f85d806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:41.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85d6d9d700 1 -- 172.21.15.119:0/2466599106 >> v2:172.21.15.184:3300/0 conn(0x7f85d8155eb0 msgr2=0x7f85d806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:41.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85d6d9d700 1 --2- 172.21.15.119:0/2466599106 >> v2:172.21.15.184:3300/0 conn(0x7f85d8155eb0 0x7f85d806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85d6d9d700 1 -- 172.21.15.119:0/2466599106 >> v1:172.21.15.184:6789/0 conn(0x7f85d80c70b0 legacy=0x7f85d8069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:41.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.298+0000 7f85d6d9d700 1 -- 172.21.15.119:0/2466599106 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d806f430 con 0x7f85d8069e80 2023-10-08T02:11:41.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.299+0000 7f85d6d9d700 1 --2- 172.21.15.119:0/2466599106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8069e80 0x7f85d80ca4c0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f85d8065160 tx=0x7f85d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f221fa68e0f9e7db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.299+0000 7f85d559a700 1 -- 172.21.15.119:0/2466599106 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d000e070 con 0x7f85d8069e80 2023-10-08T02:11:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.299+0000 7f85d559a700 1 -- 172.21.15.119:0/2466599106 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d000ba20 con 0x7f85d8069e80 2023-10-08T02:11:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.299+0000 7f85d559a700 1 -- 172.21.15.119:0/2466599106 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d00092e0 con 0x7f85d8069e80 2023-10-08T02:11:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.299+0000 7f85dcdff700 1 -- 172.21.15.119:0/2466599106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8069e80 msgr2=0x7f85d80ca4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:41.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.299+0000 7f85dcdff700 1 --2- 172.21.15.119:0/2466599106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8069e80 0x7f85d80ca4c0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f85d8065160 tx=0x7f85d0009840 comp rx=0 tx=0).stop 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.300+0000 7f85dcdff700 1 -- 172.21.15.119:0/2466599106 shutdown_connections 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.300+0000 7f85dcdff700 1 --2- 172.21.15.119:0/2466599106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8069e80 0x7f85d80ca4c0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.300+0000 7f85dcdff700 1 --2- 172.21.15.119:0/2466599106 >> v2:172.21.15.184:3300/0 conn(0x7f85d8155eb0 0x7f85d806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.300+0000 7f85dcdff700 1 -- 172.21.15.119:0/2466599106 >> 172.21.15.119:0/2466599106 conn(0x7f85d8068e10 msgr2=0x7f85d80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.300+0000 7f85dcdff700 1 -- 172.21.15.119:0/2466599106 shutdown_connections 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.300+0000 7f85dcdff700 1 -- 172.21.15.119:0/2466599106 wait complete. 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.301+0000 7f85dcdff700 1 Processor -- start 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.301+0000 7f85dcdff700 1 -- start start 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85dcdff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85d8069e80 0x7f85d81544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:41.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85dcdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85d80c70b0 0x7f85d8169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85dcdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8155eb0 0x7f85d80cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85dcdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d816b360 con 0x7f85d8155eb0 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85dcdff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d816a6e0 con 0x7f85d8069e80 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85dcdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d816a9c0 con 0x7f85d80c70b0 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85d6d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8155eb0 0x7f85d80cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85d6d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8155eb0 0x7f85d80cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57076/0 (socket says 172.21.15.119:57076) 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85d6d9d700 1 -- 172.21.15.119:0/1415315627 learned_addr learned my addr 172.21.15.119:0/1415315627 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85d659c700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85d8069e80 0x7f85d81544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.302+0000 7f85d5d9b700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85d80c70b0 0x7f85d8169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:41.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85d6d9d700 1 -- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85d80c70b0 msgr2=0x7f85d8169e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85d6d9d700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85d80c70b0 0x7f85d8169e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85d6d9d700 1 -- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85d8069e80 msgr2=0x7f85d81544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85d6d9d700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85d8069e80 0x7f85d81544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85d6d9d700 1 -- 172.21.15.119:0/1415315627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d0009ae0 con 0x7f85d8155eb0 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85d5d9b700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85d80c70b0 0x7f85d8169e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85d6d9d700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8155eb0 0x7f85d80cc950 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f85d00072f0 tx=0x7f85d0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85c67fc700 1 -- 172.21.15.119:0/1415315627 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0008bb0 con 0x7f85d8155eb0 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85c67fc700 1 -- 172.21.15.119:0/1415315627 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d0008d30 con 0x7f85d8155eb0 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85c67fc700 1 -- 172.21.15.119:0/1415315627 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0008ea0 con 0x7f85d8155eb0 2023-10-08T02:11:41.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d816aca0 con 0x7f85d8155eb0 2023-10-08T02:11:41.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.303+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d8154a10 con 0x7f85d8155eb0 2023-10-08T02:11:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.304+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d804c3c0 con 0x7f85d8155eb0 2023-10-08T02:11:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.306+0000 7f85c67fc700 1 -- 172.21.15.119:0/1415315627 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f85d0003040 con 0x7f85d8155eb0 2023-10-08T02:11:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.307+0000 7f85c67fc700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85b8066f30 0x7f85b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.307+0000 7f85c67fc700 1 -- 172.21.15.119:0/1415315627 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f85d0086380 con 0x7f85d8155eb0 2023-10-08T02:11:41.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.307+0000 7f85d659c700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85b8066f30 0x7f85b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:41.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.307+0000 7f85d659c700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85b8066f30 0x7f85b80693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85c800ae30 tx=0x7f85c800a330 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:41.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.310+0000 7f85c67fc700 1 -- 172.21.15.119:0/1415315627 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f85d0053510 con 0x7f85d8155eb0 2023-10-08T02:11:41.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:41.544+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f85d816b4a0 con 0x7f85d8155eb0 2023-10-08T02:11:42.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.097+0000 7f85c67fc700 1 -- 172.21.15.119:0/1415315627 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7f85d0056b30 con 0x7f85d8155eb0 2023-10-08T02:11:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85b8066f30 msgr2=0x7f85b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85b8066f30 0x7f85b80693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f85c800ae30 tx=0x7f85c800a330 comp rx=0 tx=0).stop 2023-10-08T02:11:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8155eb0 msgr2=0x7f85d80cc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8155eb0 0x7f85d80cc950 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f85d00072f0 tx=0x7f85d0018000 comp rx=0 tx=0).stop 2023-10-08T02:11:42.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 shutdown_connections 2023-10-08T02:11:42.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f85b8066f30 0x7f85b80693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85d8155eb0 0x7f85d80cc950 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f85d8069e80 0x7f85d81544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 --2- 172.21.15.119:0/1415315627 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85d80c70b0 0x7f85d8169e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.108+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 >> 172.21.15.119:0/1415315627 conn(0x7f85d8068e10 msgr2=0x7f85d80b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:42.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.109+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 shutdown_connections 2023-10-08T02:11:42.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.109+0000 7f85dcdff700 1 -- 172.21.15.119:0/1415315627 wait complete. 2023-10-08T02:11:42.114 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 18 and data pool 19 2023-10-08T02:11:42.134 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:11:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.317+0000 7fd579dff700 1 Processor -- start 2023-10-08T02:11:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.317+0000 7fd579dff700 1 -- start start 2023-10-08T02:11:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.318+0000 7fd579dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5741578b0 0x7fd574155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.318+0000 7fd579dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5740ad930 con 0x7fd5740af2f0 2023-10-08T02:11:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.318+0000 7fd579dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5740acd20 con 0x7fd574155c10 2023-10-08T02:11:42.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.318+0000 7fd579dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5740ad000 con 0x7fd5741578b0 2023-10-08T02:11:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.319+0000 7fd572ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5741578b0 0x7fd574155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.319+0000 7fd573fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd5740af2f0 0x7fd5740af730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38574/0 (socket says 172.21.15.119:38574) 2023-10-08T02:11:42.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.319+0000 7fd573fff700 1 -- 172.21.15.119:0/1561116210 learned_addr learned my addr 172.21.15.119:0/1561116210 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.319+0000 7fd5727fc700 1 -- 172.21.15.119:0/1561116210 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001925601 0 0) 0x7fd5740ad930 con 0x7fd5740af2f0 2023-10-08T02:11:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.319+0000 7fd572ffd700 1 -- 172.21.15.119:0/1561116210 >> v1:172.21.15.184:6789/0 conn(0x7fd574155c10 legacy=0x7fd5740ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.319+0000 7fd572ffd700 1 -- 172.21.15.119:0/1561116210 >> v1:172.21.15.119:6789/0 conn(0x7fd5740af2f0 legacy=0x7fd5740af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.319+0000 7fd572ffd700 1 -- 172.21.15.119:0/1561116210 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740ad2e0 con 0x7fd5741578b0 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.319+0000 7fd572ffd700 1 --2- 172.21.15.119:0/1561116210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5741578b0 0x7fd574155fe0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098e0 tx=0x7fd5680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e904cb9ecf98b878 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd5727fc700 1 -- 172.21.15.119:0/1561116210 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568009e80 con 0x7fd5741578b0 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd5727fc700 1 -- 172.21.15.119:0/1561116210 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd56800b860 con 0x7fd5741578b0 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd5727fc700 1 -- 172.21.15.119:0/1561116210 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd56800bbe0 con 0x7fd5741578b0 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd579dff700 1 -- 172.21.15.119:0/1561116210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5741578b0 msgr2=0x7fd574155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd579dff700 1 --2- 172.21.15.119:0/1561116210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5741578b0 0x7fd574155fe0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fd5680098e0 tx=0x7fd5680091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd579dff700 1 -- 172.21.15.119:0/1561116210 shutdown_connections 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd579dff700 1 --2- 172.21.15.119:0/1561116210 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5741578b0 0x7fd574155fe0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd579dff700 1 -- 172.21.15.119:0/1561116210 >> 172.21.15.119:0/1561116210 conn(0x7fd57406a320 msgr2=0x7fd5740cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd579dff700 1 -- 172.21.15.119:0/1561116210 shutdown_connections 2023-10-08T02:11:42.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.320+0000 7fd579dff700 1 -- 172.21.15.119:0/1561116210 wait complete. 2023-10-08T02:11:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.322+0000 7fd579dff700 1 Processor -- start 2023-10-08T02:11:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.322+0000 7fd579dff700 1 -- start start 2023-10-08T02:11:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd579dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5740af2f0 0x7fd574167120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd579dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd574155c10 0x7fd5740c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd579dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5741578b0 0x7fd5740cc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd579dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740cb290 con 0x7fd574155c10 2023-10-08T02:11:42.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd579dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740ca680 con 0x7fd5741578b0 2023-10-08T02:11:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd579dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5740ca960 con 0x7fd5740af2f0 2023-10-08T02:11:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd574155c10 0x7fd5740c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd572ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd574155c10 0x7fd5740c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57082/0 (socket says 172.21.15.119:57082) 2023-10-08T02:11:42.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd572ffd700 1 -- 172.21.15.119:0/329388446 learned_addr learned my addr 172.21.15.119:0/329388446 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd573fff700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5741578b0 0x7fd5740cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd5737fe700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5740af2f0 0x7fd574167120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd572ffd700 1 -- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5740af2f0 msgr2=0x7fd574167120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd572ffd700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5740af2f0 0x7fd574167120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd572ffd700 1 -- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5741578b0 msgr2=0x7fd5740cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd572ffd700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5741578b0 0x7fd5740cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd572ffd700 1 -- 172.21.15.119:0/329388446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd568009420 con 0x7fd574155c10 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.323+0000 7fd5737fe700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5740af2f0 0x7fd574167120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.324+0000 7fd572ffd700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd574155c10 0x7fd5740c9d30 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd574064290 tx=0x7fd568017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.324+0000 7fd55b7fe700 1 -- 172.21.15.119:0/329388446 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568011420 con 0x7fd574155c10 2023-10-08T02:11:42.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.324+0000 7fd55b7fe700 1 -- 172.21.15.119:0/329388446 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd568011580 con 0x7fd574155c10 2023-10-08T02:11:42.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.324+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5740cac40 con 0x7fd574155c10 2023-10-08T02:11:42.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.324+0000 7fd55b7fe700 1 -- 172.21.15.119:0/329388446 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd568011920 con 0x7fd574155c10 2023-10-08T02:11:42.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.324+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5740ca270 con 0x7fd574155c10 2023-10-08T02:11:42.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.325+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5740cb400 con 0x7fd574155c10 2023-10-08T02:11:42.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.326+0000 7fd55b7fe700 1 -- 172.21.15.119:0/329388446 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd568011c90 con 0x7fd574155c10 2023-10-08T02:11:42.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.327+0000 7fd55b7fe700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd560066f30 0x7fd5600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.327+0000 7fd55b7fe700 1 -- 172.21.15.119:0/329388446 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd5680861a0 con 0x7fd574155c10 2023-10-08T02:11:42.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.327+0000 7fd5737fe700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd560066f30 0x7fd5600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.328+0000 7fd5737fe700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd560066f30 0x7fd5600693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd564001840 tx=0x7fd564009040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:42.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.331+0000 7fd55b7fe700 1 -- 172.21.15.119:0/329388446 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd568053220 con 0x7fd574155c10 2023-10-08T02:11:42.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.521+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd57404c3c0 con 0x7fd574155c10 2023-10-08T02:11:42.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.522+0000 7fd55b7fe700 1 -- 172.21.15.119:0/329388446 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+16138 (secure 0 0 0) 0x7fd568056840 con 0x7fd574155c10 2023-10-08T02:11:42.522 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:42.522 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":156,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:11:41.544995+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:11:39.192970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T02:11:40.531612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:11:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd560066f30 msgr2=0x7fd5600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd560066f30 0x7fd5600693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd564001840 tx=0x7fd564009040 comp rx=0 tx=0).stop 2023-10-08T02:11:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd574155c10 msgr2=0x7fd5740c9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd574155c10 0x7fd5740c9d30 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fd574064290 tx=0x7fd568017000 comp rx=0 tx=0).stop 2023-10-08T02:11:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 shutdown_connections 2023-10-08T02:11:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd560066f30 0x7fd5600693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5740af2f0 0x7fd574167120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5741578b0 0x7fd5740cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 --2- 172.21.15.119:0/329388446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd574155c10 0x7fd5740c9d30 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 >> 172.21.15.119:0/329388446 conn(0x7fd57406a320 msgr2=0x7fd57406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:42.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 shutdown_connections 2023-10-08T02:11:42.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.524+0000 7fd579dff700 1 -- 172.21.15.119:0/329388446 wait complete. 2023-10-08T02:11:42.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:11:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.722+0000 7fc7071ff700 1 Processor -- start 2023-10-08T02:11:42.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.722+0000 7fc7071ff700 1 -- start start 2023-10-08T02:11:42.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc7071ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7000ccc40 0x7fc700155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc7071ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc700156150 0x7fc7000ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc7071ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7000cd010 0x7fc7000af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc7071ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7000ad930 con 0x7fc7000cd010 2023-10-08T02:11:42.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc7071ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7000acd20 con 0x7fc700156150 2023-10-08T02:11:42.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc7071ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000ad000 con 0x7fc7000ccc40 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc70579c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7000cd010 0x7fc7000af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc70579c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7000cd010 0x7fc7000af2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57094/0 (socket says 172.21.15.119:57094) 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc70579c700 1 -- 172.21.15.119:0/2660122669 learned_addr learned my addr 172.21.15.119:0/2660122669 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc6fffff700 1 --2- 172.21.15.119:0/2660122669 >> v2:172.21.15.184:3300/0 conn(0x7fc700156150 0x7fc7000ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc704f9b700 1 --2- 172.21.15.119:0/2660122669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7000ccc40 0x7fc700155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc70579c700 1 -- 172.21.15.119:0/2660122669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7000ccc40 msgr2=0x7fc700155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc70579c700 1 --2- 172.21.15.119:0/2660122669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7000ccc40 0x7fc700155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc70579c700 1 -- 172.21.15.119:0/2660122669 >> v2:172.21.15.184:3300/0 conn(0x7fc700156150 msgr2=0x7fc7000ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc70579c700 1 --2- 172.21.15.119:0/2660122669 >> v2:172.21.15.184:3300/0 conn(0x7fc700156150 0x7fc7000ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.724+0000 7fc70579c700 1 -- 172.21.15.119:0/2660122669 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7000ad2e0 con 0x7fc7000cd010 2023-10-08T02:11:42.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.725+0000 7fc70579c700 1 --2- 172.21.15.119:0/2660122669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc7000af2f0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc700064210 tx=0x7fc6f8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2dd18c0b7b93da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:42.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.725+0000 7fc6ff7fe700 1 -- 172.21.15.119:0/2660122669 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f800e070 con 0x7fc7000cd010 2023-10-08T02:11:42.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.725+0000 7fc6ff7fe700 1 -- 172.21.15.119:0/2660122669 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f800ba50 con 0x7fc7000cd010 2023-10-08T02:11:42.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.725+0000 7fc6ff7fe700 1 -- 172.21.15.119:0/2660122669 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f80092e0 con 0x7fc7000cd010 2023-10-08T02:11:42.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.725+0000 7fc7071ff700 1 -- 172.21.15.119:0/2660122669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 msgr2=0x7fc7000af2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.725+0000 7fc7071ff700 1 --2- 172.21.15.119:0/2660122669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc7000af2f0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fc700064210 tx=0x7fc6f8009840 comp rx=0 tx=0).stop 2023-10-08T02:11:42.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.726+0000 7fc7071ff700 1 -- 172.21.15.119:0/2660122669 shutdown_connections 2023-10-08T02:11:42.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.726+0000 7fc7071ff700 1 --2- 172.21.15.119:0/2660122669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc7000af2f0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.726+0000 7fc7071ff700 1 --2- 172.21.15.119:0/2660122669 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7000ccc40 0x7fc700155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.726+0000 7fc7071ff700 1 --2- 172.21.15.119:0/2660122669 >> v2:172.21.15.184:3300/0 conn(0x7fc700156150 0x7fc7000ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.726+0000 7fc7071ff700 1 -- 172.21.15.119:0/2660122669 >> 172.21.15.119:0/2660122669 conn(0x7fc70006a320 msgr2=0x7fc7000cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:42.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.726+0000 7fc7071ff700 1 -- 172.21.15.119:0/2660122669 shutdown_connections 2023-10-08T02:11:42.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.726+0000 7fc7071ff700 1 -- 172.21.15.119:0/2660122669 wait complete. 2023-10-08T02:11:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.727+0000 7fc7071ff700 1 Processor -- start 2023-10-08T02:11:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.727+0000 7fc7071ff700 1 -- start start 2023-10-08T02:11:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc7071ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7000ccc40 0x7fc700154140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc7071ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc700159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc7071ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc700156150 0x7fc7000c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc7071ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000ca0b0 con 0x7fc7000cd010 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc7071ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7000caec0 con 0x7fc7000ccc40 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc7071ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc700156940 con 0x7fc700156150 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc6fffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc700159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc6fffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc700159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57096/0 (socket says 172.21.15.119:57096) 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc6fffff700 1 -- 172.21.15.119:0/1834863274 learned_addr learned my addr 172.21.15.119:0/1834863274 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.728+0000 7fc704f9b700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7000ccc40 0x7fc700154140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc70579c700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc700156150 0x7fc7000c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fffff700 1 -- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc700156150 msgr2=0x7fc7000c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fffff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc700156150 0x7fc7000c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fffff700 1 -- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7000ccc40 msgr2=0x7fc700154140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fffff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7000ccc40 0x7fc700154140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fffff700 1 -- 172.21.15.119:0/1834863274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f8009ae0 con 0x7fc7000cd010 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc704f9b700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7000ccc40 0x7fc700154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fffff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc700159d90 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00d860 tx=0x7fc6ec008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fcff9700 1 -- 172.21.15.119:0/1834863274 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec009610 con 0x7fc7000cd010 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fcff9700 1 -- 172.21.15.119:0/1834863274 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ec009770 con 0x7fc7000cd010 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc6fcff9700 1 -- 172.21.15.119:0/1834863274 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec009b30 con 0x7fc7000cd010 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7000c9af0 con 0x7fc7000cd010 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.729+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc700156be0 con 0x7fc7000cd010 2023-10-08T02:11:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.730+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7001567a0 con 0x7fc7000cd010 2023-10-08T02:11:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.731+0000 7fc6fcff9700 1 -- 172.21.15.119:0/1834863274 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6ec01e460 con 0x7fc7000cd010 2023-10-08T02:11:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.732+0000 7fc6fcff9700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6e4066f30 0x7fc6e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.733+0000 7fc6fcff9700 1 -- 172.21.15.119:0/1834863274 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fc6ec084dd0 con 0x7fc7000cd010 2023-10-08T02:11:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.733+0000 7fc704f9b700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6e4066f30 0x7fc6e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.733+0000 7fc704f9b700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6e4066f30 0x7fc6e40693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4006fd0 tx=0x7fc6f4008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.737+0000 7fc6fcff9700 1 -- 172.21.15.119:0/1834863274 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7001567a0 con 0x7fc7000cd010 2023-10-08T02:11:42.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.927+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7001567a0 con 0x7fc7000cd010 2023-10-08T02:11:42.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.928+0000 7fc6fcff9700 1 -- 172.21.15.119:0/1834863274 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+16138 (secure 0 0 0) 0x7fc7001567a0 con 0x7fc7000cd010 2023-10-08T02:11:42.928 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:42.928 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":156,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:11:41.544995+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:11:39.192970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T02:11:40.531612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:11:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6e4066f30 msgr2=0x7fc6e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6e4066f30 0x7fc6e40693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4006fd0 tx=0x7fc6f4008040 comp rx=0 tx=0).stop 2023-10-08T02:11:42.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 msgr2=0x7fc700159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc700159d90 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec00d860 tx=0x7fc6ec008040 comp rx=0 tx=0).stop 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 shutdown_connections 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6e4066f30 0x7fc6e40693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc700156150 0x7fc7000c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7000ccc40 0x7fc700154140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 --2- 172.21.15.119:0/1834863274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7000cd010 0x7fc700159d90 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.930+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 >> 172.21.15.119:0/1834863274 conn(0x7fc70006a320 msgr2=0x7fc70015fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.931+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 shutdown_connections 2023-10-08T02:11:42.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:42.931+0000 7fc7071ff700 1 -- 172.21.15.119:0/1834863274 wait complete. 2023-10-08T02:11:42.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T02:11:43.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.135+0000 7fac627ff700 1 Processor -- start 2023-10-08T02:11:43.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.135+0000 7fac627ff700 1 -- start start 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.136+0000 7fac627ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.136+0000 7fac627ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5c15d7e0 con 0x7fac5c06bcd0 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.136+0000 7fac627ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5c15dac0 con 0x7fac5c15df50 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.136+0000 7fac627ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c15aa30 con 0x7fac5c06c0a0 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.136+0000 7fac5bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fac5c06bcd0 0x7fac5c15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:38580/0 (socket says 172.21.15.119:38580) 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5bfff700 1 -- 172.21.15.119:0/1573373885 learned_addr learned my addr 172.21.15.119:0/1573373885 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5b7fe700 1 --2- 172.21.15.119:0/1573373885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5affd700 1 -- 172.21.15.119:0/1573373885 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216272427 0 0) 0x7fac5c15d7e0 con 0x7fac5c06bcd0 2023-10-08T02:11:43.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5affd700 1 -- 172.21.15.119:0/1573373885 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac44003570 con 0x7fac5c06bcd0 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5affd700 1 -- 172.21.15.119:0/1573373885 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376877809 0 0) 0x7fac5c15dac0 con 0x7fac5c15df50 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5affd700 1 -- 172.21.15.119:0/1573373885 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac5c15d7e0 con 0x7fac5c15df50 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5b7fe700 1 -- 172.21.15.119:0/1573373885 >> v1:172.21.15.184:6789/0 conn(0x7fac5c15df50 legacy=0x7fac5c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5b7fe700 1 -- 172.21.15.119:0/1573373885 >> v1:172.21.15.119:6789/0 conn(0x7fac5c06bcd0 legacy=0x7fac5c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5b7fe700 1 -- 172.21.15.119:0/1573373885 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c15ad10 con 0x7fac5c06c0a0 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.137+0000 7fac5b7fe700 1 --2- 172.21.15.119:0/1573373885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0caa90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac50009910 tx=0x7fac500091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ac772cd0d99190 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac5affd700 1 -- 172.21.15.119:0/1573373885 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50011630 con 0x7fac5c06c0a0 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac5affd700 1 -- 172.21.15.119:0/1573373885 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac50011790 con 0x7fac5c06c0a0 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac5affd700 1 -- 172.21.15.119:0/1573373885 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50011b30 con 0x7fac5c06c0a0 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac627ff700 1 -- 172.21.15.119:0/1573373885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 msgr2=0x7fac5c0caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac627ff700 1 --2- 172.21.15.119:0/1573373885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0caa90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fac50009910 tx=0x7fac500091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:43.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac627ff700 1 -- 172.21.15.119:0/1573373885 shutdown_connections 2023-10-08T02:11:43.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac627ff700 1 --2- 172.21.15.119:0/1573373885 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0caa90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:43.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac627ff700 1 -- 172.21.15.119:0/1573373885 >> 172.21.15.119:0/1573373885 conn(0x7fac5c0ce600 msgr2=0x7fac5c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:43.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac627ff700 1 -- 172.21.15.119:0/1573373885 shutdown_connections 2023-10-08T02:11:43.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.138+0000 7fac627ff700 1 -- 172.21.15.119:0/1573373885 wait complete. 2023-10-08T02:11:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.140+0000 7fac627ff700 1 Processor -- start 2023-10-08T02:11:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.140+0000 7fac627ff700 1 -- start start 2023-10-08T02:11:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac627ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac5c06bcd0 0x7fac5c0c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:43.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac627ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac627ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac5c15df50 0x7fac5c0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac627ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0c6910 con 0x7fac5c06bcd0 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac627ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0b1440 con 0x7fac5c15df50 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac627ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5c0b1720 con 0x7fac5c06c0a0 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac5c06bcd0 0x7fac5c0c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac5c06bcd0 0x7fac5c0c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57108/0 (socket says 172.21.15.119:57108) 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5bfff700 1 -- 172.21.15.119:0/3238895533 learned_addr learned my addr 172.21.15.119:0/3238895533 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5b7fe700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac60d9c700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac5c15df50 0x7fac5c0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5bfff700 1 -- 172.21.15.119:0/3238895533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 msgr2=0x7fac5c0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5bfff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0c7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:43.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5bfff700 1 -- 172.21.15.119:0/3238895533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac5c15df50 msgr2=0x7fac5c0b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5bfff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac5c15df50 0x7fac5c0b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.141+0000 7fac5bfff700 1 -- 172.21.15.119:0/3238895533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac50009420 con 0x7fac5c06bcd0 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.142+0000 7fac5bfff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac5c06bcd0 0x7fac5c0c6090 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00c790 tx=0x7fac4c007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.142+0000 7fac43fff700 1 -- 172.21.15.119:0/3238895533 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c005b30 con 0x7fac5c06bcd0 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.142+0000 7fac43fff700 1 -- 172.21.15.119:0/3238895533 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac4c005c90 con 0x7fac5c06bcd0 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.142+0000 7fac43fff700 1 -- 172.21.15.119:0/3238895533 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac4c007300 con 0x7fac5c06bcd0 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.142+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5c0b1a00 con 0x7fac5c06bcd0 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.142+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5c0c65d0 con 0x7fac5c06bcd0 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.143+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5c04c3c0 con 0x7fac5c06bcd0 2023-10-08T02:11:43.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.144+0000 7fac43fff700 1 -- 172.21.15.119:0/3238895533 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac4c01b3b0 con 0x7fac5c06bcd0 2023-10-08T02:11:43.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.145+0000 7fac43fff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fac48066f30 0x7fac480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:43.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.145+0000 7fac43fff700 1 -- 172.21.15.119:0/3238895533 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fac4c083bd0 con 0x7fac5c06bcd0 2023-10-08T02:11:43.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.145+0000 7fac5b7fe700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fac48066f30 0x7fac480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:43.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.146+0000 7fac5b7fe700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fac48066f30 0x7fac480693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac50009860 tx=0x7fac50017000 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:43.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.149+0000 7fac43fff700 1 -- 172.21.15.119:0/3238895533 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fac4c050bd0 con 0x7fac5c06bcd0 2023-10-08T02:11:43.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:43.383+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fac5c15aff0 con 0x7fac5c06bcd0 2023-10-08T02:11:44.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.112+0000 7fac43fff700 1 -- 172.21.15.119:0/3238895533 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7fac4c017020 con 0x7fac5c06bcd0 2023-10-08T02:11:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fac48066f30 msgr2=0x7fac480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fac48066f30 0x7fac480693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fac50009860 tx=0x7fac50017000 comp rx=0 tx=0).stop 2023-10-08T02:11:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac5c06bcd0 msgr2=0x7fac5c0c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac5c06bcd0 0x7fac5c0c6090 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00c790 tx=0x7fac4c007ea0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 shutdown_connections 2023-10-08T02:11:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fac48066f30 0x7fac480693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac5c15df50 0x7fac5c0b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac5c06bcd0 0x7fac5c0c6090 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 --2- 172.21.15.119:0/3238895533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac5c06c0a0 0x7fac5c0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 >> 172.21.15.119:0/3238895533 conn(0x7fac5c0ce600 msgr2=0x7fac5c15b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:44.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 shutdown_connections 2023-10-08T02:11:44.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.115+0000 7fac627ff700 1 -- 172.21.15.119:0/3238895533 wait complete. 2023-10-08T02:11:44.135 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T02:11:44.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.323+0000 7f9f419ff700 1 Processor -- start 2023-10-08T02:11:44.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.323+0000 7f9f419ff700 1 -- start start 2023-10-08T02:11:44.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f419ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f3c069e80 0x7f9f3c0c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:44.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f419ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c0b0a70 0x7f9f3c0c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:44.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f419ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3c0af2f0 con 0x7f9f3c15ac80 2023-10-08T02:11:44.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f419ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3c0af5d0 con 0x7f9f3c069e80 2023-10-08T02:11:44.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f419ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0c8660 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f33fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f3c069e80 0x7f9f3c0c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:44.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f33fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9f3c069e80 0x7f9f3c0c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57122/0 (socket says 172.21.15.119:57122) 2023-10-08T02:11:44.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f33fff700 1 -- 172.21.15.119:0/3093853258 learned_addr learned my addr 172.21.15.119:0/3093853258 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:44.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f3b7fe700 1 --2- 172.21.15.119:0/3093853258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c0b0a70 0x7f9f3c0c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:44.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f33fff700 1 -- 172.21.15.119:0/3093853258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c0b0a70 msgr2=0x7f9f3c0c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:44.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f33fff700 1 --2- 172.21.15.119:0/3093853258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c0b0a70 0x7f9f3c0c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f33fff700 1 -- 172.21.15.119:0/3093853258 >> v1:172.21.15.184:6789/0 conn(0x7f9f3c15ac80 legacy=0x7f9f3c069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:44.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.325+0000 7f9f33fff700 1 -- 172.21.15.119:0/3093853258 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c0c8940 con 0x7f9f3c069e80 2023-10-08T02:11:44.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.326+0000 7f9f33fff700 1 --2- 172.21.15.119:0/3093853258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c069e80 0x7f9f3c0c6b70 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f9f240099b0 tx=0x7f9f24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191e8abd63d36eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:44.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.326+0000 7f9f3a7fc700 1 -- 172.21.15.119:0/3093853258 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400b860 con 0x7f9f3c069e80 2023-10-08T02:11:44.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.326+0000 7f9f3a7fc700 1 -- 172.21.15.119:0/3093853258 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2400b9c0 con 0x7f9f3c069e80 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.326+0000 7f9f3a7fc700 1 -- 172.21.15.119:0/3093853258 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400bd60 con 0x7f9f3c069e80 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.326+0000 7f9f419ff700 1 -- 172.21.15.119:0/3093853258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c069e80 msgr2=0x7f9f3c0c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.326+0000 7f9f419ff700 1 --2- 172.21.15.119:0/3093853258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c069e80 0x7f9f3c0c6b70 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f9f240099b0 tx=0x7f9f24009250 comp rx=0 tx=0).stop 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.327+0000 7f9f419ff700 1 -- 172.21.15.119:0/3093853258 shutdown_connections 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.327+0000 7f9f419ff700 1 --2- 172.21.15.119:0/3093853258 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c0b0a70 0x7f9f3c0c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.327+0000 7f9f419ff700 1 --2- 172.21.15.119:0/3093853258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c069e80 0x7f9f3c0c6b70 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.327+0000 7f9f419ff700 1 -- 172.21.15.119:0/3093853258 >> 172.21.15.119:0/3093853258 conn(0x7f9f3c068e10 msgr2=0x7f9f3c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.327+0000 7f9f419ff700 1 -- 172.21.15.119:0/3093853258 shutdown_connections 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.327+0000 7f9f419ff700 1 -- 172.21.15.119:0/3093853258 wait complete. 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.328+0000 7f9f419ff700 1 Processor -- start 2023-10-08T02:11:44.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.328+0000 7f9f419ff700 1 -- start start 2023-10-08T02:11:44.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f419ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c069e80 0x7f9f3c143ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:44.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f419ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c0b0a70 0x7f9f3c158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:44.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f419ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f3c15ac80 0x7f9f3c0ac590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:44.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f419ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0ac100 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f419ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c16ac30 con 0x7f9f3c15ac80 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f419ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0ab7d0 con 0x7f9f3c069e80 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c0b0a70 0x7f9f3c158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c0b0a70 0x7f9f3c158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57136/0 (socket says 172.21.15.119:57136) 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.329+0000 7f9f33fff700 1 -- 172.21.15.119:0/2064078247 learned_addr learned my addr 172.21.15.119:0/2064078247 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f3b7fe700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f3c15ac80 0x7f9f3c0ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f3affd700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c069e80 0x7f9f3c143ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f33fff700 1 -- 172.21.15.119:0/2064078247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c069e80 msgr2=0x7f9f3c143ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f33fff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c069e80 0x7f9f3c143ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f33fff700 1 -- 172.21.15.119:0/2064078247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f3c15ac80 msgr2=0x7f9f3c0ac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f33fff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f3c15ac80 0x7f9f3c0ac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f33fff700 1 -- 172.21.15.119:0/2064078247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f240094f0 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f33fff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c0b0a70 0x7f9f3c158060 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009290 tx=0x7f9f24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:44.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f32ffd700 1 -- 172.21.15.119:0/2064078247 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24024440 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f32ffd700 1 -- 172.21.15.119:0/2064078247 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f240245c0 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3c0abab0 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f32ffd700 1 -- 172.21.15.119:0/2064078247 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24024980 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.330+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3c1443e0 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.331+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3c0ac2b0 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.332+0000 7f9f32ffd700 1 -- 172.21.15.119:0/2064078247 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9f2401a420 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.333+0000 7f9f32ffd700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f1c066f30 0x7f9f1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:44.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.333+0000 7f9f32ffd700 1 -- 172.21.15.119:0/2064078247 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f9f24086580 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.333+0000 7f9f3affd700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f1c066f30 0x7f9f1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:44.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.334+0000 7f9f3affd700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f1c066f30 0x7f9f1c0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0037c0 tx=0x7f9f2c009040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:44.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.337+0000 7f9f32ffd700 1 -- 172.21.15.119:0/2064078247 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9f3c0ac2b0 con 0x7f9f3c0b0a70 2023-10-08T02:11:44.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:44.571+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9f3c04c3c0 con 0x7f9f3c0b0a70 2023-10-08T02:11:45.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.115+0000 7f9f32ffd700 1 -- 172.21.15.119:0/2064078247 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9f24053580 con 0x7f9f3c0b0a70 2023-10-08T02:11:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f1c066f30 msgr2=0x7f9f1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f1c066f30 0x7f9f1c0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0037c0 tx=0x7f9f2c009040 comp rx=0 tx=0).stop 2023-10-08T02:11:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c0b0a70 msgr2=0x7f9f3c158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c0b0a70 0x7f9f3c158060 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9f24009290 tx=0x7f9f24017000 comp rx=0 tx=0).stop 2023-10-08T02:11:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 shutdown_connections 2023-10-08T02:11:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9f1c066f30 0x7f9f1c0693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9f3c069e80 0x7f9f3c143ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9f3c15ac80 0x7f9f3c0ac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 --2- 172.21.15.119:0/2064078247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f3c0b0a70 0x7f9f3c158060 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 >> 172.21.15.119:0/2064078247 conn(0x7f9f3c068e10 msgr2=0x7f9f3c1575a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:45.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 shutdown_connections 2023-10-08T02:11:45.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.118+0000 7f9f419ff700 1 -- 172.21.15.119:0/2064078247 wait complete. 2023-10-08T02:11:45.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:11:45.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.321+0000 7f60715ff700 1 Processor -- start 2023-10-08T02:11:45.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.321+0000 7f60715ff700 1 -- start start 2023-10-08T02:11:45.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.322+0000 7f60715ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c0caa90 0x7f606c0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.322+0000 7f60715ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606c15d7e0 con 0x7f606c06bcd0 2023-10-08T02:11:45.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.322+0000 7f60715ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f606c15dac0 con 0x7f606c06c0a0 2023-10-08T02:11:45.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.322+0000 7f60715ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0caf30 con 0x7f606c0caa90 2023-10-08T02:11:45.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.322+0000 7f606affd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f606c06bcd0 0x7f606c15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55692/0 (socket says 172.21.15.119:55692) 2023-10-08T02:11:45.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.322+0000 7f606affd700 1 -- 172.21.15.119:0/2249435644 learned_addr learned my addr 172.21.15.119:0/2249435644 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f606b7fe700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c0caa90 0x7f606c0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750547864 0 0) 0x7f606c15d7e0 con 0x7f606c06bcd0 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6054003570 con 0x7f606c06bcd0 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2566337959 0 0) 0x7f6054003570 con 0x7f606c06bcd0 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c0caa90 msgr2=0x7f606c0c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c0caa90 0x7f606c0c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 >> v1:172.21.15.184:6789/0 conn(0x7f606c06c0a0 legacy=0x7f606c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c15aa30 con 0x7f606c06bcd0 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f605c001470 con 0x7f606c06bcd0 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 >> v1:172.21.15.119:6789/0 conn(0x7f606c06bcd0 legacy=0x7f606c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c0caa90 0x7f6054003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6054004060 0x7f6054004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f60540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0caf30 con 0x7f6054004d50 2023-10-08T02:11:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605400bcc0 con 0x7f6054004060 2023-10-08T02:11:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.323+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605400be80 con 0x7f606c0caa90 2023-10-08T02:11:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606b7fe700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f60540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606affd700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c0caa90 0x7f6054003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606a7fc700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6054004060 0x7f6054004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606b7fe700 1 -- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c0caa90 msgr2=0x7f6054003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606b7fe700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c0caa90 0x7f6054003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606b7fe700 1 -- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6054004060 msgr2=0x7f6054004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606b7fe700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6054004060 0x7f6054004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606b7fe700 1 -- 172.21.15.119:0/2249435644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6054003570 con 0x7f6054004d50 2023-10-08T02:11:45.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f606b7fe700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f60540051c0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f606400e210 tx=0x7f606400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae7db060c1c1e3b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.324+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400eac0 con 0x7f6054004d50 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400f040 con 0x7f6054004d50 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f6069ffb700 1 -- 172.21.15.119:0/2249435644 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400ee70 con 0x7f6054004d50 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f60715ff700 1 -- 172.21.15.119:0/2249435644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 msgr2=0x7f60540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f60715ff700 1 --2- 172.21.15.119:0/2249435644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f60540051c0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f606400e210 tx=0x7f606400b040 comp rx=0 tx=0).stop 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f606b7fe700 1 -- 172.21.15.119:0/2249435644 reap_dead start 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f60715ff700 1 -- 172.21.15.119:0/2249435644 shutdown_connections 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f60715ff700 1 -- 172.21.15.119:0/2249435644 >> 172.21.15.119:0/2249435644 conn(0x7f606c0ce600 msgr2=0x7f606c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f60715ff700 1 -- 172.21.15.119:0/2249435644 shutdown_connections 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.325+0000 7f60715ff700 1 -- 172.21.15.119:0/2249435644 wait complete. 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.326+0000 7f60715ff700 1 Processor -- start 2023-10-08T02:11:45.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.327+0000 7f60715ff700 1 -- start start 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.327+0000 7f60715ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f606c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f60715ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f606c0b13a0 0x7f606c1593b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f60715ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c16a490 0x7f606c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f60715ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0b2440 con 0x7f6054004d50 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f60715ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c159ee0 con 0x7f606c0b13a0 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f60715ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606c0b1830 con 0x7f606c16a490 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f606c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f606c0b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50786/0 (socket says 172.21.15.119:50786) 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 -- 172.21.15.119:0/2068322146 learned_addr learned my addr 172.21.15.119:0/2068322146 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 -- 172.21.15.119:0/2068322146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c16a490 msgr2=0x7f606c0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c16a490 0x7f606c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 -- 172.21.15.119:0/2068322146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f606c0b13a0 msgr2=0x7f606c1593b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f606c0b13a0 0x7f606c1593b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 -- 172.21.15.119:0/2068322146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f606c0b1b10 con 0x7f6054004d50 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f606affd700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f606c0b28d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f605c0011e0 tx=0x7f605c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.328+0000 7f6052ffd700 1 -- 172.21.15.119:0/2068322146 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c018030 con 0x7f6054004d50 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.329+0000 7f6052ffd700 1 -- 172.21.15.119:0/2068322146 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605c00f8a0 con 0x7f6054004d50 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.329+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606c0b1df0 con 0x7f6054004d50 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.329+0000 7f6052ffd700 1 -- 172.21.15.119:0/2068322146 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c00fc60 con 0x7f6054004d50 2023-10-08T02:11:45.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.329+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c16a910 con 0x7f6054004d50 2023-10-08T02:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.330+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606c0b2290 con 0x7f6054004d50 2023-10-08T02:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.331+0000 7f6052ffd700 1 -- 172.21.15.119:0/2068322146 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f605c01b070 con 0x7f6054004d50 2023-10-08T02:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.331+0000 7f6052ffd700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.332+0000 7f6052ffd700 1 -- 172.21.15.119:0/2068322146 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f605c019070 con 0x7f6054004d50 2023-10-08T02:11:45.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.332+0000 7f606a7fc700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.332+0000 7f606a7fc700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6058066f30 0x7f60580693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6060007e40 tx=0x7f6060007d30 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:45.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.335+0000 7f6052ffd700 1 -- 172.21.15.119:0/2068322146 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f605c051760 con 0x7f6054004d50 2023-10-08T02:11:45.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.568+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f606c0b2720 con 0x7f6054004d50 2023-10-08T02:11:45.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.569+0000 7f6052ffd700 1 -- 172.21.15.119:0/2068322146 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f605c054d80 con 0x7f6054004d50 2023-10-08T02:11:45.569 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:45.569 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/603902131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":603902131},{"type":"v1","addr":"172.21.15.184:6834","nonce":603902131}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3930162267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3930162267},{"type":"v1","addr":"172.21.15.119:6834","nonce":3930162267}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/237096421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":237096421},{"type":"v1","addr":"172.21.15.119:6835","nonce":237096421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:11:41.545410+0000","modified":"2023-10-08T02:11:45.111313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8667":{"gid":8667,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3810992025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3810992025},{"type":"v1","addr":"172.21.15.184:6835","nonce":3810992025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T02:11:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6058066f30 msgr2=0x7f60580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6058066f30 0x7f60580693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6060007e40 tx=0x7f6060007d30 comp rx=0 tx=0).stop 2023-10-08T02:11:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 msgr2=0x7f606c0b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f606c0b28d0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f605c0011e0 tx=0x7f605c001d80 comp rx=0 tx=0).stop 2023-10-08T02:11:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 shutdown_connections 2023-10-08T02:11:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6058066f30 0x7f60580693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f606c16a490 0x7f606c0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6054004d50 0x7f606c0b28d0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 --2- 172.21.15.119:0/2068322146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f606c0b13a0 0x7f606c1593b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 >> 172.21.15.119:0/2068322146 conn(0x7f606c0ce600 msgr2=0x7f606c155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:45.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 shutdown_connections 2023-10-08T02:11:45.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.571+0000 7f60715ff700 1 -- 172.21.15.119:0/2068322146 wait complete. 2023-10-08T02:11:45.572 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 71 2023-10-08T02:11:45.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:11:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.770+0000 7fbfb319d700 1 Processor -- start 2023-10-08T02:11:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.770+0000 7fbfb319d700 1 -- start start 2023-10-08T02:11:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.771+0000 7fbfb319d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbfac0b02a0 0x7fbfac15bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.771+0000 7fbfb319d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfac06ae70 con 0x7fbfac06d950 2023-10-08T02:11:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.771+0000 7fbfb319d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfac0c8a60 con 0x7fbfac06a0a0 2023-10-08T02:11:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.771+0000 7fbfb319d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfac06a540 con 0x7fbfac0b02a0 2023-10-08T02:11:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.771+0000 7fbfb219b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbfac06d950 0x7fbfac0c8350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55708/0 (socket says 172.21.15.119:55708) 2023-10-08T02:11:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.771+0000 7fbfb219b700 1 -- 172.21.15.119:0/639288736 learned_addr learned my addr 172.21.15.119:0/639288736 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550922892 0 0) 0x7fbfac06ae70 con 0x7fbfac06d950 2023-10-08T02:11:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf98003570 con 0x7fbfac06d950 2023-10-08T02:11:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb299c700 1 --2- 172.21.15.119:0/639288736 >> v2:172.21.15.184:3300/0 conn(0x7fbfac0b02a0 0x7fbfac15bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1363176722 0 0) 0x7fbf98003570 con 0x7fbfac06d950 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 >> v2:172.21.15.184:3300/0 conn(0x7fbfac0b02a0 msgr2=0x7fbfac15bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 --2- 172.21.15.119:0/639288736 >> v2:172.21.15.184:3300/0 conn(0x7fbfac0b02a0 0x7fbfac15bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 >> v1:172.21.15.184:6789/0 conn(0x7fbfac06a0a0 legacy=0x7fbfac15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac06a820 con 0x7fbfac06d950 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fbfa8001470 con 0x7fbfac06d950 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 >> v1:172.21.15.119:6789/0 conn(0x7fbfac06d950 legacy=0x7fbfac0c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98003a60 0x7fbf98003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf98004810 0x7fbf98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf980052e0 0x7fbf9800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac06a540 con 0x7fbf98003a60 2023-10-08T02:11:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9800c190 con 0x7fbf98004810 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9800c350 con 0x7fbf980052e0 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.772+0000 7fbfb299c700 1 --2- 172.21.15.119:0/639288736 >> v2:172.21.15.184:3300/0 conn(0x7fbfac0b02a0 0x7fbfac15bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb219b700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98003a60 0x7fbf98003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb199a700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf98004810 0x7fbf98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb299c700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf980052e0 0x7fbf9800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb219b700 1 -- 172.21.15.119:0/639288736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf980052e0 msgr2=0x7fbf9800ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb219b700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf980052e0 0x7fbf9800ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb219b700 1 -- 172.21.15.119:0/639288736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf98004810 msgr2=0x7fbf98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb219b700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf98004810 0x7fbf98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb219b700 1 -- 172.21.15.119:0/639288736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf98003570 con 0x7fbf98003a60 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb299c700 1 -- 172.21.15.119:0/639288736 reap_dead start 2023-10-08T02:11:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb219b700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98003a60 0x7fbf98003e70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800c3c0 tx=0x7fbfa8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55435a489cac9083 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800cc90 con 0x7fbf98003a60 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfa800cdf0 con 0x7fbf98003a60 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.773+0000 7fbfb1199700 1 -- 172.21.15.119:0/639288736 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8005b50 con 0x7fbf98003a60 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.774+0000 7fbfb319d700 1 -- 172.21.15.119:0/639288736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98003a60 msgr2=0x7fbf98003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.774+0000 7fbfb319d700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98003a60 0x7fbf98003e70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800c3c0 tx=0x7fbfa8001e50 comp rx=0 tx=0).stop 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.774+0000 7fbfb319d700 1 -- 172.21.15.119:0/639288736 shutdown_connections 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.774+0000 7fbfb319d700 1 --2- 172.21.15.119:0/639288736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98003a60 0x7fbf98003e70 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.774+0000 7fbfb319d700 1 -- 172.21.15.119:0/639288736 >> 172.21.15.119:0/639288736 conn(0x7fbfac0cc1f0 msgr2=0x7fbfac0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.774+0000 7fbfb319d700 1 -- 172.21.15.119:0/639288736 shutdown_connections 2023-10-08T02:11:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.774+0000 7fbfb319d700 1 -- 172.21.15.119:0/639288736 wait complete. 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.775+0000 7fbfb319d700 1 Processor -- start 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.776+0000 7fbfb319d700 1 -- start start 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.776+0000 7fbfb319d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf98003a60 0x7fbfac0c5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.776+0000 7fbfb319d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfac15a6f0 0x7fbfac151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.776+0000 7fbfb319d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfac0c4400 0x7fbfac153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.776+0000 7fbfb319d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac15ab60 con 0x7fbfac15a6f0 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.776+0000 7fbfb319d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac0c4bc0 con 0x7fbfac0c4400 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.776+0000 7fbfb319d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfac0c4ea0 con 0x7fbf98003a60 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfac15a6f0 0x7fbfac151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfac15a6f0 0x7fbfac151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50792/0 (socket says 172.21.15.119:50792) 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 -- 172.21.15.119:0/3929304294 learned_addr learned my addr 172.21.15.119:0/3929304294 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 -- 172.21.15.119:0/3929304294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf98003a60 msgr2=0x7fbfac0c5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf98003a60 0x7fbfac0c5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 -- 172.21.15.119:0/3929304294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfac0c4400 msgr2=0x7fbfac153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfac0c4400 0x7fbfac153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 -- 172.21.15.119:0/3929304294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac0c5180 con 0x7fbfac15a6f0 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb199a700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfac15a6f0 0x7fbfac151980 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c002950 tx=0x7fbf9c007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfa27fc700 1 -- 172.21.15.119:0/3929304294 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00f8d0 con 0x7fbfac15a6f0 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfa27fc700 1 -- 172.21.15.119:0/3929304294 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9c00fa30 con 0x7fbfac15a6f0 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfac0c5460 con 0x7fbfac15a6f0 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.777+0000 7fbfa27fc700 1 -- 172.21.15.119:0/3929304294 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00fdd0 con 0x7fbfac15a6f0 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.778+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfac0c48a0 con 0x7fbfac15a6f0 2023-10-08T02:11:45.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.779+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfac0c5ab0 con 0x7fbfac15a6f0 2023-10-08T02:11:45.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.779+0000 7fbfa27fc700 1 -- 172.21.15.119:0/3929304294 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf9c013600 con 0x7fbfac15a6f0 2023-10-08T02:11:45.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.780+0000 7fbfa27fc700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf90066f30 0x7fbf900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:45.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.780+0000 7fbfa27fc700 1 -- 172.21.15.119:0/3929304294 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fbf9c084cf0 con 0x7fbfac15a6f0 2023-10-08T02:11:45.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.780+0000 7fbfb219b700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf90066f30 0x7fbf900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:45.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.781+0000 7fbfb219b700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf90066f30 0x7fbf900693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8001ed0 tx=0x7fbfa8015000 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:45.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.784+0000 7fbfa27fc700 1 -- 172.21.15.119:0/3929304294 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf9c051cf0 con 0x7fbfac15a6f0 2023-10-08T02:11:45.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.974+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfac04c3c0 con 0x7fbfac15a6f0 2023-10-08T02:11:45.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.975+0000 7fbfa27fc700 1 -- 172.21.15.119:0/3929304294 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v157) v1 ==== 74+0+16125 (secure 0 0 0) 0x7fbf9c055310 con 0x7fbfac15a6f0 2023-10-08T02:11:45.975 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:45.975 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":157,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:11:43.093190+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:11:39.192970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T02:11:40.531612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:11:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf90066f30 msgr2=0x7fbf900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf90066f30 0x7fbf900693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8001ed0 tx=0x7fbfa8015000 comp rx=0 tx=0).stop 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfac15a6f0 msgr2=0x7fbfac151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfac15a6f0 0x7fbfac151980 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c002950 tx=0x7fbf9c007d30 comp rx=0 tx=0).stop 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 shutdown_connections 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf90066f30 0x7fbf900693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf98003a60 0x7fbfac0c5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbfac0c4400 0x7fbfac153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.977+0000 7fbfb319d700 1 --2- 172.21.15.119:0/3929304294 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbfac15a6f0 0x7fbfac151980 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.978+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 >> 172.21.15.119:0/3929304294 conn(0x7fbfac0cc1f0 msgr2=0x7fbfac14cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.978+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 shutdown_connections 2023-10-08T02:11:45.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:45.978+0000 7fbfb319d700 1 -- 172.21.15.119:0/3929304294 wait complete. 2023-10-08T02:11:45.995 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T02:11:45.995 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:11:46.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.180+0000 7f6ec0bff700 1 Processor -- start 2023-10-08T02:11:46.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.180+0000 7f6ec0bff700 1 -- start start 2023-10-08T02:11:46.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.181+0000 7f6ec0bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6ebc0b1420 0x7f6ebc0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.181+0000 7f6ec0bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6ebc0ccf90 0x7f6ebc0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.181+0000 7f6ec0bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc0cc300 con 0x7f6ebc0cbf30 2023-10-08T02:11:46.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.181+0000 7f6ec0bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc156bf0 con 0x7f6ebc0b1420 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.181+0000 7f6ec0bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc0ae950 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eba59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6ebc0cbf30 0x7f6ebc0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55714/0 (socket says 172.21.15.119:55714) 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eba59c700 1 -- 172.21.15.119:0/552716903 learned_addr learned my addr 172.21.15.119:0/552716903 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eb9d9b700 1 --2- 172.21.15.119:0/552716903 >> v2:172.21.15.119:3300/0 conn(0x7f6ebc0b1420 0x7f6ebc0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6ebad9d700 1 --2- 172.21.15.119:0/552716903 >> v2:172.21.15.184:3300/0 conn(0x7f6ebc0ccf90 0x7f6ebc0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eb959a700 1 -- 172.21.15.119:0/552716903 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507469594 0 0) 0x7f6ebc0cc300 con 0x7f6ebc0cbf30 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eb959a700 1 -- 172.21.15.119:0/552716903 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea4003570 con 0x7f6ebc0cbf30 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eb9d9b700 1 -- 172.21.15.119:0/552716903 >> v2:172.21.15.184:3300/0 conn(0x7f6ebc0ccf90 msgr2=0x7f6ebc0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eb9d9b700 1 --2- 172.21.15.119:0/552716903 >> v2:172.21.15.184:3300/0 conn(0x7f6ebc0ccf90 0x7f6ebc0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eb9d9b700 1 -- 172.21.15.119:0/552716903 >> v1:172.21.15.119:6789/0 conn(0x7f6ebc0cbf30 legacy=0x7f6ebc0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eb9d9b700 1 -- 172.21.15.119:0/552716903 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0ad370 con 0x7f6ebc0b1420 2023-10-08T02:11:46.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6ebad9d700 1 --2- 172.21.15.119:0/552716903 >> v2:172.21.15.184:3300/0 conn(0x7f6ebc0ccf90 0x7f6ebc0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.182+0000 7f6eb9d9b700 1 --2- 172.21.15.119:0/552716903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0b1420 0x7f6ebc0ae410 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0c8fd0 tx=0x7f6eb00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad9f87fed2c0441 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6eb959a700 1 -- 172.21.15.119:0/552716903 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0011630 con 0x7f6ebc0b1420 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6eb959a700 1 -- 172.21.15.119:0/552716903 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb0011790 con 0x7f6ebc0b1420 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6eb959a700 1 -- 172.21.15.119:0/552716903 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0011b30 con 0x7f6ebc0b1420 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6ec0bff700 1 -- 172.21.15.119:0/552716903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0b1420 msgr2=0x7f6ebc0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/552716903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0b1420 0x7f6ebc0ae410 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0c8fd0 tx=0x7f6eb00091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6ec0bff700 1 -- 172.21.15.119:0/552716903 shutdown_connections 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/552716903 >> v2:172.21.15.184:3300/0 conn(0x7f6ebc0ccf90 0x7f6ebc0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/552716903 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0b1420 0x7f6ebc0ae410 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6ec0bff700 1 -- 172.21.15.119:0/552716903 >> 172.21.15.119:0/552716903 conn(0x7f6ebc06a320 msgr2=0x7f6ebc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6ec0bff700 1 -- 172.21.15.119:0/552716903 shutdown_connections 2023-10-08T02:11:46.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.183+0000 7f6ec0bff700 1 -- 172.21.15.119:0/552716903 wait complete. 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.185+0000 7f6ec0bff700 1 Processor -- start 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.185+0000 7f6ec0bff700 1 -- start start 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ec0bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ebc0b1420 0x7f6ebc155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ec0bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc0cbf30 0x7f6ebc15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ec0bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0ccf90 0x7f6ebc144eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ec0bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc155dc0 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ec0bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc15b890 con 0x7f6ebc0cbf30 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ec0bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc15bb70 con 0x7f6ebc0b1420 2023-10-08T02:11:46.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ebad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0ccf90 0x7f6ebc144eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ebad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0ccf90 0x7f6ebc144eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50800/0 (socket says 172.21.15.119:50800) 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ebad9d700 1 -- 172.21.15.119:0/3808579499 learned_addr learned my addr 172.21.15.119:0/3808579499 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6eba59c700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ebc0b1420 0x7f6ebc155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6eb9d9b700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc0cbf30 0x7f6ebc15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ebad9d700 1 -- 172.21.15.119:0/3808579499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ebc0b1420 msgr2=0x7f6ebc155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ebad9d700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ebc0b1420 0x7f6ebc155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ebad9d700 1 -- 172.21.15.119:0/3808579499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc0cbf30 msgr2=0x7f6ebc15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ebad9d700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc0cbf30 0x7f6ebc15b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.186+0000 7f6ebad9d700 1 -- 172.21.15.119:0/3808579499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb0009420 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.187+0000 7f6ebad9d700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0ccf90 0x7f6ebc144eb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009fc0 tx=0x7f6eb40099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.187+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3808579499 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb400dd40 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.187+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3808579499 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb400f040 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.187+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3808579499 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4015690 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.187+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc15be50 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.187+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc15c410 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.188+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc15c630 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.189+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3808579499 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6eb401e3b0 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.190+0000 7f6eaa7fc700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e9c066f30 0x7f6e9c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.190+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3808579499 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f6eb4084cf0 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.190+0000 7f6eba59c700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e9c066f30 0x7f6e9c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.191+0000 7f6eba59c700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e9c066f30 0x7f6e9c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0058d0 tx=0x7f6eac009040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:46.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.194+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3808579499 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6eb4051cf0 con 0x7f6ebc0ccf90 2023-10-08T02:11:46.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.381+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ebc04c3c0 con 0x7f6e9c066f30 2023-10-08T02:11:46.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.391+0000 7f6eaa7fc700 1 -- 172.21.15.119:0/3808579499 <== mgr.7822 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279887 (secure 0 0 0) 0x7f6ebc04c3c0 con 0x7f6e9c066f30 2023-10-08T02:11:46.394 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:46.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e9c066f30 msgr2=0x7f6e9c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e9c066f30 0x7f6e9c0693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6eac0058d0 tx=0x7f6eac009040 comp rx=0 tx=0).stop 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0ccf90 msgr2=0x7f6ebc144eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0ccf90 0x7f6ebc144eb0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f6eb4009fc0 tx=0x7f6eb40099b0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 shutdown_connections 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6e9c066f30 0x7f6e9c0693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6ebc0b1420 0x7f6ebc155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ebc0ccf90 0x7f6ebc144eb0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 --2- 172.21.15.119:0/3808579499 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6ebc0cbf30 0x7f6ebc15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 >> 172.21.15.119:0/3808579499 conn(0x7f6ebc06a320 msgr2=0x7f6ebc0cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 shutdown_connections 2023-10-08T02:11:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.395+0000 7f6ec0bff700 1 -- 172.21.15.119:0/3808579499 wait complete. 2023-10-08T02:11:46.398 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:11:46.413 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:11:45.590224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":442,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2488656,"kb_used_data":16144,"kb_used_omap":0,"kb_used_meta":2472448,"kb_avail":752486064,"statfs":{"total":773094113280,"available":770545729536,"internally_reserved":0,"allocated":16531456,"data_stored":8795583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2531786752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.526010"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114682+0000","last_change":"2023-10-08T02:11:42.113823+0000","last_active":"2023-10-08T02:11:42.114682+0000","last_peered":"2023-10-08T02:11:42.114682+0000","last_clean":"2023-10-08T02:11:42.114682+0000","last_became_active":"2023-10-08T02:11:42.113493+0000","last_became_peered":"2023-10-08T02:11:42.113493+0000","last_unstale":"2023-10-08T02:11:42.114682+0000","last_undegraded":"2023-10-08T02:11:42.114682+0000","last_fullsized":"2023-10-08T02:11:42.114682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.103531+0000","last_change":"2023-10-08T02:11:42.098697+0000","last_active":"2023-10-08T02:11:42.103531+0000","last_peered":"2023-10-08T02:11:42.103531+0000","last_clean":"2023-10-08T02:11:42.103531+0000","last_became_active":"2023-10-08T02:11:42.098447+0000","last_became_peered":"2023-10-08T02:11:42.098447+0000","last_unstale":"2023-10-08T02:11:42.103531+0000","last_undegraded":"2023-10-08T02:11:42.103531+0000","last_fullsized":"2023-10-08T02:11:42.103531+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113481+0000","last_change":"2023-10-08T02:11:42.111978+0000","last_active":"2023-10-08T02:11:42.113481+0000","last_peered":"2023-10-08T02:11:42.113481+0000","last_clean":"2023-10-08T02:11:42.113481+0000","last_became_active":"2023-10-08T02:11:42.110828+0000","last_became_peered":"2023-10-08T02:11:42.110828+0000","last_unstale":"2023-10-08T02:11:42.113481+0000","last_undegraded":"2023-10-08T02:11:42.113481+0000","last_fullsized":"2023-10-08T02:11:42.113481+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:11.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107993+0000","last_change":"2023-10-08T02:11:42.105477+0000","last_active":"2023-10-08T02:11:42.107993+0000","last_peered":"2023-10-08T02:11:42.107993+0000","last_clean":"2023-10-08T02:11:42.107993+0000","last_became_active":"2023-10-08T02:11:42.105203+0000","last_became_peered":"2023-10-08T02:11:42.105203+0000","last_unstale":"2023-10-08T02:11:42.107993+0000","last_undegraded":"2023-10-08T02:11:42.107993+0000","last_fullsized":"2023-10-08T02:11:42.107993+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:34.769593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.111811+0000","last_change":"2023-10-08T02:11:42.110297+0000","last_active":"2023-10-08T02:11:42.111811+0000","last_peered":"2023-10-08T02:11:42.111811+0000","last_clean":"2023-10-08T02:11:42.111811+0000","last_became_active":"2023-10-08T02:11:42.110051+0000","last_became_peered":"2023-10-08T02:11:42.110051+0000","last_unstale":"2023-10-08T02:11:42.111811+0000","last_undegraded":"2023-10-08T02:11:42.111811+0000","last_fullsized":"2023-10-08T02:11:42.111811+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:02.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106088+0000","last_change":"2023-10-08T02:11:42.105002+0000","last_active":"2023-10-08T02:11:42.106088+0000","last_peered":"2023-10-08T02:11:42.106088+0000","last_clean":"2023-10-08T02:11:42.106088+0000","last_became_active":"2023-10-08T02:11:42.104707+0000","last_became_peered":"2023-10-08T02:11:42.104707+0000","last_unstale":"2023-10-08T02:11:42.106088+0000","last_undegraded":"2023-10-08T02:11:42.106088+0000","last_fullsized":"2023-10-08T02:11:42.106088+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106069+0000","last_change":"2023-10-08T02:11:42.104624+0000","last_active":"2023-10-08T02:11:42.106069+0000","last_peered":"2023-10-08T02:11:42.106069+0000","last_clean":"2023-10-08T02:11:42.106069+0000","last_became_active":"2023-10-08T02:11:42.104319+0000","last_became_peered":"2023-10-08T02:11:42.104319+0000","last_unstale":"2023-10-08T02:11:42.106069+0000","last_undegraded":"2023-10-08T02:11:42.106069+0000","last_fullsized":"2023-10-08T02:11:42.106069+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:56.323343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067690+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:10.718365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072538+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:22.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070853+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:40.446588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075526+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:30.613532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.076459+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:50.165805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070924+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:17:37.906952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067817+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:00.041784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075337+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:20.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100491+0000","last_change":"2023-10-08T02:11:42.095233+0000","last_active":"2023-10-08T02:11:42.100491+0000","last_peered":"2023-10-08T02:11:42.100491+0000","last_clean":"2023-10-08T02:11:42.100491+0000","last_became_active":"2023-10-08T02:11:42.094991+0000","last_became_peered":"2023-10-08T02:11:42.094991+0000","last_unstale":"2023-10-08T02:11:42.100491+0000","last_undegraded":"2023-10-08T02:11:42.100491+0000","last_fullsized":"2023-10-08T02:11:42.100491+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:58.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.073213+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:23:52.815532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.069899+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:36.101961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072441+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:02.277276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.066903+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:50.650114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100695+0000","last_change":"2023-10-08T02:11:42.096482+0000","last_active":"2023-10-08T02:11:42.100695+0000","last_peered":"2023-10-08T02:11:42.100695+0000","last_clean":"2023-10-08T02:11:42.100695+0000","last_became_active":"2023-10-08T02:11:42.094165+0000","last_became_peered":"2023-10-08T02:11:42.094165+0000","last_unstale":"2023-10-08T02:11:42.100695+0000","last_undegraded":"2023-10-08T02:11:42.100695+0000","last_fullsized":"2023-10-08T02:11:42.100695+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:23.801545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086789+0000","last_change":"2023-10-08T02:11:41.084280+0000","last_active":"2023-10-08T02:11:42.086789+0000","last_peered":"2023-10-08T02:11:42.086789+0000","last_clean":"2023-10-08T02:11:42.086789+0000","last_became_active":"2023-10-08T02:11:41.083971+0000","last_became_peered":"2023-10-08T02:11:41.083971+0000","last_unstale":"2023-10-08T02:11:42.086789+0000","last_undegraded":"2023-10-08T02:11:42.086789+0000","last_fullsized":"2023-10-08T02:11:42.086789+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075287+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:22.374785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067379+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072535+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:10.498785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.071357+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:12:35.260528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067542+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:28.886973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072478+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:57.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113445+0000","last_change":"2023-10-08T02:11:42.111899+0000","last_active":"2023-10-08T02:11:42.113445+0000","last_peered":"2023-10-08T02:11:42.113445+0000","last_clean":"2023-10-08T02:11:42.113445+0000","last_became_active":"2023-10-08T02:11:42.111582+0000","last_became_peered":"2023-10-08T02:11:42.111582+0000","last_unstale":"2023-10-08T02:11:42.113445+0000","last_undegraded":"2023-10-08T02:11:42.113445+0000","last_fullsized":"2023-10-08T02:11:42.113445+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:11.724000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072605+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:25.497645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070329+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:31.493936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070821+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:42.593819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072123+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:06.207304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125349+0000","last_change":"2023-10-08T02:11:41.087828+0000","last_active":"2023-10-08T02:11:42.125349+0000","last_peered":"2023-10-08T02:11:42.125349+0000","last_clean":"2023-10-08T02:11:42.125349+0000","last_became_active":"2023-10-08T02:11:41.087533+0000","last_became_peered":"2023-10-08T02:11:41.087533+0000","last_unstale":"2023-10-08T02:11:42.125349+0000","last_undegraded":"2023-10-08T02:11:42.125349+0000","last_fullsized":"2023-10-08T02:11:42.125349+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.101659+0000","last_change":"2023-10-08T02:11:42.098444+0000","last_active":"2023-10-08T02:11:42.101659+0000","last_peered":"2023-10-08T02:11:42.101659+0000","last_clean":"2023-10-08T02:11:42.101659+0000","last_became_active":"2023-10-08T02:11:42.098168+0000","last_became_peered":"2023-10-08T02:11:42.098168+0000","last_unstale":"2023-10-08T02:11:42.101659+0000","last_undegraded":"2023-10-08T02:11:42.101659+0000","last_fullsized":"2023-10-08T02:11:42.101659+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:12.133057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067619+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:55.317708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075585+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:56.324149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072482+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:24.569836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075994+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:50.650114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067082+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:11.127422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070305+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:18.323015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.069817+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:28.409914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067076+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.160399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"146'271","reported_seq":820,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T02:11:37.032549+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:11:37.032549+0000","last_peered":"2023-10-08T02:11:37.032549+0000","last_clean":"2023-10-08T02:11:37.032549+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:11:37.032549+0000","last_undegraded":"2023-10-08T02:11:37.032549+0000","last_fullsized":"2023-10-08T02:11:37.032549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":1},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307920,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063920,"statfs":{"total":96636764160,"available":96321454080,"internally_reserved":0,"allocated":2498560,"data_stored":1535644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44400000000000001},"max":{"1min":0.96499999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.755}]},{"osd":1,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.60899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.74199999999999999},"min":{"1min":0.621,"5min":0.53100000000000003,"15min":0.48899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":1.0580000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":1.0600000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.48999999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.626},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82799999999999996,"15min":0.80100000000000005},"min":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.57399999999999995},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.80900000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.752,"5min":0.94999999999999996,"15min":0.98399999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.67800000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.71099999999999997}]},{"osd":6,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.57299999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.92400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315544,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056296,"statfs":{"total":96636764160,"available":96313647104,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76500000000000001,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77200000000000002,"15min":0.748},"min":{"1min":0.54900000000000004,"5min":0.501,"15min":0.46400000000000002},"max":{"1min":1.4139999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.48599999999999999,"15min":0.47599999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.875},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77500000000000002,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.48899999999999999},"max":{"1min":1.464,"5min":1.472,"15min":1.472},"last":0.624}]},{"osd":2,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.73499999999999999},"min":{"1min":0.60099999999999998,"5min":0.53500000000000003,"15min":0.47699999999999998},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.79700000000000004,"15min":0.76600000000000001},"min":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.436},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.55600000000000005}]},{"osd":3,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.60099999999999998,"5min":0.57999999999999996,"15min":0.52800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.78500000000000003},"min":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"max":{"1min":1.048,"5min":1.169,"15min":1.169},"last":0.71799999999999997}]},{"osd":4,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":1.0129999999999999,"5min":1.133,"15min":1.133},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63800000000000001,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.90200000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82999999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54800000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95599999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.90300000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.752,"15min":0.72099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0289999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98099999999999998,"5min":1.232,"15min":1.232},"last":0.85299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307340,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064500,"statfs":{"total":96636764160,"available":96322048000,"internally_reserved":0,"allocated":1904640,"data_stored":945276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.36299999999999999,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.69899999999999995,"5min":1.024,"15min":1.024},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66200000000000003,"5min":1.603,"15min":1.603},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.505},"min":{"1min":0.40200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64900000000000002,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77700000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.872,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.71599999999999997,"5min":3.698,"15min":3.698},"last":0.45800000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.88800000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.999,"5min":1.734,"15min":1.734},"last":0.70299999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.80100000000000005},"min":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.008,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.58999999999999997,"15min":0.56100000000000005},"max":{"1min":1.0640000000000001,"5min":1.403,"15min":1.403},"last":0.55100000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.66300000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0269999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.68100000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.04,"5min":1.573,"15min":1.573},"last":0.65900000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1120000000000001,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.58999999999999997},"max":{"1min":1.028,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.80800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2510848,"data_stored":1541913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48499999999999999,"15min":0.46999999999999997},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.254},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.443}]},{"osd":2,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.317},"max":{"1min":0.59699999999999998,"5min":0.70699999999999996,"15min":0.746},"last":0.378},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.37},"max":{"1min":0.69099999999999995,"5min":0.88300000000000001,"15min":1.325},"last":0.49099999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.44600000000000001,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.873,"5min":0.875,"15min":0.875},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.54300000000000004,"5min":0.47399999999999998,"15min":0.45900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.70899999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.24},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80200000000000005,"15min":0.79500000000000004},"min":{"1min":0.621,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3759999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.94799999999999995,"5min":1.327,"15min":1.327},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89700000000000002,"15min":0.88200000000000001},"min":{"1min":0.754,"5min":0.60999999999999999,"15min":0.56100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.871}]},{"osd":6,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85399999999999998,"15min":0.83899999999999997},"min":{"1min":0.66100000000000003,"5min":0.55000000000000004,"15min":0.53400000000000003},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.55500000000000005},"max":{"1min":0.90900000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.875,"15min":0.84999999999999998},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.55800000000000005},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.5269999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.85799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311492,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060348,"statfs":{"total":96636764160,"available":96317796352,"internally_reserved":0,"allocated":1961984,"data_stored":972839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48699999999999999,"15min":0.47999999999999998},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":1.228,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66800000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.79500000000000004,"15min":0.872},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.41799999999999998,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.54000000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.46800000000000003,"5min":0.45500000000000002,"15min":0.39400000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.83299999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.92300000000000004,"15min":0.871},"min":{"1min":0.749,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":1.046},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89400000000000002,"15min":0.86399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6220000000000001,"5min":1.847,"15min":1.847},"last":1.0860000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.878,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.87},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0580000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.90300000000000002,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.64100000000000001,"5min":0.622,"15min":0.58899999999999997},"max":{"1min":1.7909999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.998}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315544,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056296,"statfs":{"total":96636764160,"available":96313647104,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.46500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.72199999999999998,"15min":0.87},"last":0.436},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.32600000000000001},"max":{"1min":0.627,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.42399999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.56000000000000005,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68300000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.5}]},{"osd":2,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.42299999999999999,"15min":0.41999999999999998},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.629},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45600000000000002},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54300000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.214},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.65400000000000003,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.1599999999999999},"last":0.751}]},{"osd":5,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79900000000000004,"15min":0.80600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"max":{"1min":1.0209999999999999,"5min":1.0589999999999999,"15min":1.125},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82299999999999995,"15min":0.82499999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56899999999999995},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.67800000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85199999999999998,"15min":0.83799999999999997},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58499999999999996},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.85499999999999998,"15min":0.84499999999999997},"min":{"1min":0.67800000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.77300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311448,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060392,"statfs":{"total":96636764160,"available":96317841408,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40400000000000003},"max":{"1min":0.90100000000000002,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0009999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.86099999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.52},"last":0.80100000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.748},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60999999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90800000000000003,"5min":1.1579999999999999,"15min":1.403},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.208,"15min":1.5069999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":1.05,"15min":1.05},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93400000000000005,"5min":1.038,"15min":1.1559999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.016},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.0680000000000001,"15min":1.129},"last":0.628},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.48799999999999999},"max":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311436,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060404,"statfs":{"total":96636764160,"available":96317853696,"internally_reserved":0,"allocated":1904640,"data_stored":945276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.73899999999999999},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.38900000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.75800000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.438},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.78000000000000003},"min":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.42699999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.46200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80600000000000005,"15min":0.78100000000000003},"min":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.499},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.621},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.76200000000000001},"min":{"1min":0.61599999999999999,"5min":0.57499999999999996,"15min":0.50900000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.56100000000000005,"15min":0.497},"max":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.0529999999999999,"15min":1.091},"last":0.78000000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.627,"15min":0.62},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.0620000000000001,"15min":1.115},"last":0.81299999999999994}]},{"osd":6,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":0.93999999999999995,"5min":1.006,"15min":1.0629999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52000000000000002,"5min":0.436,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":0.95299999999999996,"15min":1.077},"last":0.79300000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":1.0249999999999999,"15min":1.161},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.88100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:11:46.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:11:46.609 INFO:tasks.ceph.mds.d.smithi184.stderr:2023-10-08T02:11:46.609+0000 7f1afb1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T02:11:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.635+0000 7f00b91ff700 1 Processor -- start 2023-10-08T02:11:46.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.635+0000 7f00b91ff700 1 -- start start 2023-10-08T02:11:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b91ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b41578b0 0x7f00b4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b91ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00b40af2f0 0x7f00b40af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b91ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b40ad930 con 0x7f00b4155c10 2023-10-08T02:11:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b91ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b40acd20 con 0x7f00b40af2f0 2023-10-08T02:11:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b91ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b40ad000 con 0x7f00b41578b0 2023-10-08T02:11:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b359e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00b40af2f0 0x7f00b40af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b2d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f00b4155c10 0x7f00b40ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55720/0 (socket says 172.21.15.119:55720) 2023-10-08T02:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b2d9d700 1 -- 172.21.15.119:0/4261022844 learned_addr learned my addr 172.21.15.119:0/4261022844 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b259c700 1 --2- 172.21.15.119:0/4261022844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b41578b0 0x7f00b4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b359e700 1 -- 172.21.15.119:0/4261022844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b41578b0 msgr2=0x7f00b4155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b359e700 1 --2- 172.21.15.119:0/4261022844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b41578b0 0x7f00b4155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b359e700 1 -- 172.21.15.119:0/4261022844 >> v1:172.21.15.119:6789/0 conn(0x7f00b4155c10 legacy=0x7f00b40ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b359e700 1 -- 172.21.15.119:0/4261022844 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b40ad2e0 con 0x7f00b40af2f0 2023-10-08T02:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.637+0000 7f00b1d9b700 1 -- 172.21.15.119:0/4261022844 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2252274048 0 0) 0x7f00b40ad930 con 0x7f00b4155c10 2023-10-08T02:11:46.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.638+0000 7f00b359e700 1 --2- 172.21.15.119:0/4261022844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b40af710 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009e60 tx=0x7f00ac009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21c7f42215ba9dc7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.638+0000 7f00b1d9b700 1 -- 172.21.15.119:0/4261022844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac00e070 con 0x7f00b40af2f0 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.638+0000 7f00b1d9b700 1 -- 172.21.15.119:0/4261022844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac00bc30 con 0x7f00b40af2f0 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.638+0000 7f00b1d9b700 1 -- 172.21.15.119:0/4261022844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac012430 con 0x7f00b40af2f0 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.638+0000 7f00b91ff700 1 -- 172.21.15.119:0/4261022844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 msgr2=0x7f00b40af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.638+0000 7f00b91ff700 1 --2- 172.21.15.119:0/4261022844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b40af710 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009e60 tx=0x7f00ac009780 comp rx=0 tx=0).stop 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.639+0000 7f00b91ff700 1 -- 172.21.15.119:0/4261022844 shutdown_connections 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.639+0000 7f00b91ff700 1 --2- 172.21.15.119:0/4261022844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b40af710 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.639+0000 7f00b91ff700 1 --2- 172.21.15.119:0/4261022844 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b41578b0 0x7f00b4155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.639+0000 7f00b91ff700 1 -- 172.21.15.119:0/4261022844 >> 172.21.15.119:0/4261022844 conn(0x7f00b406a320 msgr2=0x7f00b40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.639+0000 7f00b91ff700 1 -- 172.21.15.119:0/4261022844 shutdown_connections 2023-10-08T02:11:46.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.639+0000 7f00b91ff700 1 -- 172.21.15.119:0/4261022844 wait complete. 2023-10-08T02:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.640+0000 7f00b91ff700 1 Processor -- start 2023-10-08T02:11:46.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.640+0000 7f00b91ff700 1 -- start start 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b91ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b413ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b91ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b4155c10 0x7f00b4165c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b91ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b41578b0 0x7f00b4153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b91ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b41668c0 con 0x7f00b40af2f0 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b91ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b4166ba0 con 0x7f00b41578b0 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b91ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b4166e80 con 0x7f00b4155c10 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b413ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b413ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50818/0 (socket says 172.21.15.119:50818) 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b2d9d700 1 -- 172.21.15.119:0/3932754784 learned_addr learned my addr 172.21.15.119:0/3932754784 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:46.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b259c700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b4155c10 0x7f00b4165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.641+0000 7f00b359e700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b41578b0 0x7f00b4153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b2d9d700 1 -- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b4155c10 msgr2=0x7f00b4165c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b2d9d700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b4155c10 0x7f00b4165c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b2d9d700 1 -- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b41578b0 msgr2=0x7f00b4153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b2d9d700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b41578b0 0x7f00b4153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b2d9d700 1 -- 172.21.15.119:0/3932754784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac009a20 con 0x7f00b40af2f0 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b359e700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b41578b0 0x7f00b4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b259c700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b4155c10 0x7f00b4165c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b2d9d700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b413ed20 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c7e0 tx=0x7f00a4007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f009affd700 1 -- 172.21.15.119:0/3932754784 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4005b30 con 0x7f00b40af2f0 2023-10-08T02:11:46.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f009affd700 1 -- 172.21.15.119:0/3932754784 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a4005c90 con 0x7f00b40af2f0 2023-10-08T02:11:46.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f009affd700 1 -- 172.21.15.119:0/3932754784 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4007300 con 0x7f00b40af2f0 2023-10-08T02:11:46.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b4167160 con 0x7f00b40af2f0 2023-10-08T02:11:46.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.642+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b41515a0 con 0x7f00b40af2f0 2023-10-08T02:11:46.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.644+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b4167680 con 0x7f00b40af2f0 2023-10-08T02:11:46.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.644+0000 7f009affd700 1 -- 172.21.15.119:0/3932754784 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f00a401b400 con 0x7f00b40af2f0 2023-10-08T02:11:46.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.645+0000 7f009affd700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f00a0066f30 0x7f00a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:46.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.646+0000 7f009affd700 1 -- 172.21.15.119:0/3932754784 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f00a4084270 con 0x7f00b40af2f0 2023-10-08T02:11:46.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.646+0000 7f00b259c700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f00a0066f30 0x7f00a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:46.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.646+0000 7f00b259c700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f00a0066f30 0x7f00a00693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f00b4154e50 tx=0x7f00a8008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:46.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.649+0000 7f009affd700 1 -- 172.21.15.119:0/3932754784 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f00b4167680 con 0x7f00b40af2f0 2023-10-08T02:11:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.837+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f00b4166d10 con 0x7f00a0066f30 2023-10-08T02:11:46.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.846+0000 7f009affd700 1 -- 172.21.15.119:0/3932754784 <== mgr.7822 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279887 (secure 0 0 0) 0x7f00b4166d10 con 0x7f00a0066f30 2023-10-08T02:11:46.851 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:46.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f00a0066f30 msgr2=0x7f00a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f00a0066f30 0x7f00a00693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f00b4154e50 tx=0x7f00a8008040 comp rx=0 tx=0).stop 2023-10-08T02:11:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 msgr2=0x7f00b413ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b413ed20 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c7e0 tx=0x7f00a4007df0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 shutdown_connections 2023-10-08T02:11:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f00a0066f30 0x7f00a00693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f00b41578b0 0x7f00b4153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00b40af2f0 0x7f00b413ed20 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 --2- 172.21.15.119:0/3932754784 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f00b4155c10 0x7f00b4165c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 >> 172.21.15.119:0/3932754784 conn(0x7f00b406a320 msgr2=0x7f00b40ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:46.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 shutdown_connections 2023-10-08T02:11:46.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:46.850+0000 7f00b91ff700 1 -- 172.21.15.119:0/3932754784 wait complete. 2023-10-08T02:11:46.857 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:11:46.867 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:11:45.590224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":442,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2488656,"kb_used_data":16144,"kb_used_omap":0,"kb_used_meta":2472448,"kb_avail":752486064,"statfs":{"total":773094113280,"available":770545729536,"internally_reserved":0,"allocated":16531456,"data_stored":8795583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2531786752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.526010"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114682+0000","last_change":"2023-10-08T02:11:42.113823+0000","last_active":"2023-10-08T02:11:42.114682+0000","last_peered":"2023-10-08T02:11:42.114682+0000","last_clean":"2023-10-08T02:11:42.114682+0000","last_became_active":"2023-10-08T02:11:42.113493+0000","last_became_peered":"2023-10-08T02:11:42.113493+0000","last_unstale":"2023-10-08T02:11:42.114682+0000","last_undegraded":"2023-10-08T02:11:42.114682+0000","last_fullsized":"2023-10-08T02:11:42.114682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.103531+0000","last_change":"2023-10-08T02:11:42.098697+0000","last_active":"2023-10-08T02:11:42.103531+0000","last_peered":"2023-10-08T02:11:42.103531+0000","last_clean":"2023-10-08T02:11:42.103531+0000","last_became_active":"2023-10-08T02:11:42.098447+0000","last_became_peered":"2023-10-08T02:11:42.098447+0000","last_unstale":"2023-10-08T02:11:42.103531+0000","last_undegraded":"2023-10-08T02:11:42.103531+0000","last_fullsized":"2023-10-08T02:11:42.103531+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113481+0000","last_change":"2023-10-08T02:11:42.111978+0000","last_active":"2023-10-08T02:11:42.113481+0000","last_peered":"2023-10-08T02:11:42.113481+0000","last_clean":"2023-10-08T02:11:42.113481+0000","last_became_active":"2023-10-08T02:11:42.110828+0000","last_became_peered":"2023-10-08T02:11:42.110828+0000","last_unstale":"2023-10-08T02:11:42.113481+0000","last_undegraded":"2023-10-08T02:11:42.113481+0000","last_fullsized":"2023-10-08T02:11:42.113481+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:11.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107993+0000","last_change":"2023-10-08T02:11:42.105477+0000","last_active":"2023-10-08T02:11:42.107993+0000","last_peered":"2023-10-08T02:11:42.107993+0000","last_clean":"2023-10-08T02:11:42.107993+0000","last_became_active":"2023-10-08T02:11:42.105203+0000","last_became_peered":"2023-10-08T02:11:42.105203+0000","last_unstale":"2023-10-08T02:11:42.107993+0000","last_undegraded":"2023-10-08T02:11:42.107993+0000","last_fullsized":"2023-10-08T02:11:42.107993+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:34.769593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.111811+0000","last_change":"2023-10-08T02:11:42.110297+0000","last_active":"2023-10-08T02:11:42.111811+0000","last_peered":"2023-10-08T02:11:42.111811+0000","last_clean":"2023-10-08T02:11:42.111811+0000","last_became_active":"2023-10-08T02:11:42.110051+0000","last_became_peered":"2023-10-08T02:11:42.110051+0000","last_unstale":"2023-10-08T02:11:42.111811+0000","last_undegraded":"2023-10-08T02:11:42.111811+0000","last_fullsized":"2023-10-08T02:11:42.111811+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:02.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106088+0000","last_change":"2023-10-08T02:11:42.105002+0000","last_active":"2023-10-08T02:11:42.106088+0000","last_peered":"2023-10-08T02:11:42.106088+0000","last_clean":"2023-10-08T02:11:42.106088+0000","last_became_active":"2023-10-08T02:11:42.104707+0000","last_became_peered":"2023-10-08T02:11:42.104707+0000","last_unstale":"2023-10-08T02:11:42.106088+0000","last_undegraded":"2023-10-08T02:11:42.106088+0000","last_fullsized":"2023-10-08T02:11:42.106088+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106069+0000","last_change":"2023-10-08T02:11:42.104624+0000","last_active":"2023-10-08T02:11:42.106069+0000","last_peered":"2023-10-08T02:11:42.106069+0000","last_clean":"2023-10-08T02:11:42.106069+0000","last_became_active":"2023-10-08T02:11:42.104319+0000","last_became_peered":"2023-10-08T02:11:42.104319+0000","last_unstale":"2023-10-08T02:11:42.106069+0000","last_undegraded":"2023-10-08T02:11:42.106069+0000","last_fullsized":"2023-10-08T02:11:42.106069+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:56.323343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067690+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:10.718365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072538+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:22.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070853+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:40.446588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075526+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:30.613532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.076459+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:50.165805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070924+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:17:37.906952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067817+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:00.041784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075337+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:20.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100491+0000","last_change":"2023-10-08T02:11:42.095233+0000","last_active":"2023-10-08T02:11:42.100491+0000","last_peered":"2023-10-08T02:11:42.100491+0000","last_clean":"2023-10-08T02:11:42.100491+0000","last_became_active":"2023-10-08T02:11:42.094991+0000","last_became_peered":"2023-10-08T02:11:42.094991+0000","last_unstale":"2023-10-08T02:11:42.100491+0000","last_undegraded":"2023-10-08T02:11:42.100491+0000","last_fullsized":"2023-10-08T02:11:42.100491+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:58.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.073213+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:23:52.815532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.069899+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:36.101961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072441+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:02.277276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.066903+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:50.650114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100695+0000","last_change":"2023-10-08T02:11:42.096482+0000","last_active":"2023-10-08T02:11:42.100695+0000","last_peered":"2023-10-08T02:11:42.100695+0000","last_clean":"2023-10-08T02:11:42.100695+0000","last_became_active":"2023-10-08T02:11:42.094165+0000","last_became_peered":"2023-10-08T02:11:42.094165+0000","last_unstale":"2023-10-08T02:11:42.100695+0000","last_undegraded":"2023-10-08T02:11:42.100695+0000","last_fullsized":"2023-10-08T02:11:42.100695+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:23.801545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086789+0000","last_change":"2023-10-08T02:11:41.084280+0000","last_active":"2023-10-08T02:11:42.086789+0000","last_peered":"2023-10-08T02:11:42.086789+0000","last_clean":"2023-10-08T02:11:42.086789+0000","last_became_active":"2023-10-08T02:11:41.083971+0000","last_became_peered":"2023-10-08T02:11:41.083971+0000","last_unstale":"2023-10-08T02:11:42.086789+0000","last_undegraded":"2023-10-08T02:11:42.086789+0000","last_fullsized":"2023-10-08T02:11:42.086789+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075287+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:22.374785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067379+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072535+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:10.498785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.071357+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:12:35.260528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067542+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:28.886973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072478+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:57.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113445+0000","last_change":"2023-10-08T02:11:42.111899+0000","last_active":"2023-10-08T02:11:42.113445+0000","last_peered":"2023-10-08T02:11:42.113445+0000","last_clean":"2023-10-08T02:11:42.113445+0000","last_became_active":"2023-10-08T02:11:42.111582+0000","last_became_peered":"2023-10-08T02:11:42.111582+0000","last_unstale":"2023-10-08T02:11:42.113445+0000","last_undegraded":"2023-10-08T02:11:42.113445+0000","last_fullsized":"2023-10-08T02:11:42.113445+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:11.724000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072605+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:25.497645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070329+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:31.493936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070821+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:42.593819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072123+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:06.207304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125349+0000","last_change":"2023-10-08T02:11:41.087828+0000","last_active":"2023-10-08T02:11:42.125349+0000","last_peered":"2023-10-08T02:11:42.125349+0000","last_clean":"2023-10-08T02:11:42.125349+0000","last_became_active":"2023-10-08T02:11:41.087533+0000","last_became_peered":"2023-10-08T02:11:41.087533+0000","last_unstale":"2023-10-08T02:11:42.125349+0000","last_undegraded":"2023-10-08T02:11:42.125349+0000","last_fullsized":"2023-10-08T02:11:42.125349+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.101659+0000","last_change":"2023-10-08T02:11:42.098444+0000","last_active":"2023-10-08T02:11:42.101659+0000","last_peered":"2023-10-08T02:11:42.101659+0000","last_clean":"2023-10-08T02:11:42.101659+0000","last_became_active":"2023-10-08T02:11:42.098168+0000","last_became_peered":"2023-10-08T02:11:42.098168+0000","last_unstale":"2023-10-08T02:11:42.101659+0000","last_undegraded":"2023-10-08T02:11:42.101659+0000","last_fullsized":"2023-10-08T02:11:42.101659+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:12.133057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067619+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:55.317708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075585+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:56.324149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072482+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:24.569836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075994+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:50.650114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067082+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:11.127422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070305+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:18.323015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.069817+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:28.409914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067076+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.160399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"146'271","reported_seq":820,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T02:11:37.032549+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:11:37.032549+0000","last_peered":"2023-10-08T02:11:37.032549+0000","last_clean":"2023-10-08T02:11:37.032549+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:11:37.032549+0000","last_undegraded":"2023-10-08T02:11:37.032549+0000","last_fullsized":"2023-10-08T02:11:37.032549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":1},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307920,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063920,"statfs":{"total":96636764160,"available":96321454080,"internally_reserved":0,"allocated":2498560,"data_stored":1535644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44400000000000001},"max":{"1min":0.96499999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.755}]},{"osd":1,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.60899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.74199999999999999},"min":{"1min":0.621,"5min":0.53100000000000003,"15min":0.48899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":1.0580000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":1.0600000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.48999999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.626},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82799999999999996,"15min":0.80100000000000005},"min":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.57399999999999995},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.80900000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.752,"5min":0.94999999999999996,"15min":0.98399999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.67800000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.71099999999999997}]},{"osd":6,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.57299999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.92400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315544,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056296,"statfs":{"total":96636764160,"available":96313647104,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76500000000000001,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77200000000000002,"15min":0.748},"min":{"1min":0.54900000000000004,"5min":0.501,"15min":0.46400000000000002},"max":{"1min":1.4139999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.48599999999999999,"15min":0.47599999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.875},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77500000000000002,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.48899999999999999},"max":{"1min":1.464,"5min":1.472,"15min":1.472},"last":0.624}]},{"osd":2,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.73499999999999999},"min":{"1min":0.60099999999999998,"5min":0.53500000000000003,"15min":0.47699999999999998},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.79700000000000004,"15min":0.76600000000000001},"min":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.436},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.55600000000000005}]},{"osd":3,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.60099999999999998,"5min":0.57999999999999996,"15min":0.52800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.78500000000000003},"min":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"max":{"1min":1.048,"5min":1.169,"15min":1.169},"last":0.71799999999999997}]},{"osd":4,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":1.0129999999999999,"5min":1.133,"15min":1.133},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63800000000000001,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.90200000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82999999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54800000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95599999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.90300000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.752,"15min":0.72099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0289999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98099999999999998,"5min":1.232,"15min":1.232},"last":0.85299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307340,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064500,"statfs":{"total":96636764160,"available":96322048000,"internally_reserved":0,"allocated":1904640,"data_stored":945276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.36299999999999999,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.69899999999999995,"5min":1.024,"15min":1.024},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66200000000000003,"5min":1.603,"15min":1.603},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.505},"min":{"1min":0.40200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64900000000000002,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77700000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.872,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.71599999999999997,"5min":3.698,"15min":3.698},"last":0.45800000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.88800000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.999,"5min":1.734,"15min":1.734},"last":0.70299999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.80100000000000005},"min":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.008,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.58999999999999997,"15min":0.56100000000000005},"max":{"1min":1.0640000000000001,"5min":1.403,"15min":1.403},"last":0.55100000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.66300000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0269999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.68100000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.04,"5min":1.573,"15min":1.573},"last":0.65900000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1120000000000001,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.58999999999999997},"max":{"1min":1.028,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.80800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2510848,"data_stored":1541913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48499999999999999,"15min":0.46999999999999997},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.254},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.443}]},{"osd":2,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.317},"max":{"1min":0.59699999999999998,"5min":0.70699999999999996,"15min":0.746},"last":0.378},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.37},"max":{"1min":0.69099999999999995,"5min":0.88300000000000001,"15min":1.325},"last":0.49099999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.44600000000000001,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.873,"5min":0.875,"15min":0.875},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.54300000000000004,"5min":0.47399999999999998,"15min":0.45900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.70899999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.24},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80200000000000005,"15min":0.79500000000000004},"min":{"1min":0.621,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3759999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.94799999999999995,"5min":1.327,"15min":1.327},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89700000000000002,"15min":0.88200000000000001},"min":{"1min":0.754,"5min":0.60999999999999999,"15min":0.56100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.871}]},{"osd":6,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85399999999999998,"15min":0.83899999999999997},"min":{"1min":0.66100000000000003,"5min":0.55000000000000004,"15min":0.53400000000000003},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.55500000000000005},"max":{"1min":0.90900000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.875,"15min":0.84999999999999998},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.55800000000000005},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.5269999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.85799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311492,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060348,"statfs":{"total":96636764160,"available":96317796352,"internally_reserved":0,"allocated":1961984,"data_stored":972839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48699999999999999,"15min":0.47999999999999998},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":1.228,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66800000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.79500000000000004,"15min":0.872},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.41799999999999998,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.54000000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.46800000000000003,"5min":0.45500000000000002,"15min":0.39400000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.83299999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.92300000000000004,"15min":0.871},"min":{"1min":0.749,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":1.046},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89400000000000002,"15min":0.86399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6220000000000001,"5min":1.847,"15min":1.847},"last":1.0860000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.878,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.87},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0580000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.90300000000000002,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.64100000000000001,"5min":0.622,"15min":0.58899999999999997},"max":{"1min":1.7909999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.998}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315544,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056296,"statfs":{"total":96636764160,"available":96313647104,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.46500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.72199999999999998,"15min":0.87},"last":0.436},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.32600000000000001},"max":{"1min":0.627,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.42399999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.56000000000000005,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68300000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.5}]},{"osd":2,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.42299999999999999,"15min":0.41999999999999998},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.629},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45600000000000002},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54300000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.214},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.65400000000000003,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.1599999999999999},"last":0.751}]},{"osd":5,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79900000000000004,"15min":0.80600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"max":{"1min":1.0209999999999999,"5min":1.0589999999999999,"15min":1.125},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82299999999999995,"15min":0.82499999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56899999999999995},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.67800000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85199999999999998,"15min":0.83799999999999997},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58499999999999996},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.85499999999999998,"15min":0.84499999999999997},"min":{"1min":0.67800000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.77300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311448,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060392,"statfs":{"total":96636764160,"available":96317841408,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40400000000000003},"max":{"1min":0.90100000000000002,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0009999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.86099999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.52},"last":0.80100000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.748},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60999999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90800000000000003,"5min":1.1579999999999999,"15min":1.403},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.208,"15min":1.5069999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":1.05,"15min":1.05},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93400000000000005,"5min":1.038,"15min":1.1559999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.016},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.0680000000000001,"15min":1.129},"last":0.628},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.48799999999999999},"max":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311436,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060404,"statfs":{"total":96636764160,"available":96317853696,"internally_reserved":0,"allocated":1904640,"data_stored":945276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.73899999999999999},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.38900000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.75800000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.438},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.78000000000000003},"min":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.42699999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.46200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80600000000000005,"15min":0.78100000000000003},"min":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.499},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.621},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.76200000000000001},"min":{"1min":0.61599999999999999,"5min":0.57499999999999996,"15min":0.50900000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.56100000000000005,"15min":0.497},"max":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.0529999999999999,"15min":1.091},"last":0.78000000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.627,"15min":0.62},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.0620000000000001,"15min":1.115},"last":0.81299999999999994}]},{"osd":6,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":0.93999999999999995,"5min":1.006,"15min":1.0629999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52000000000000002,"5min":0.436,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":0.95299999999999996,"15min":1.077},"last":0.79300000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":1.0249999999999999,"15min":1.161},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.88100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:11:46.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:11:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.087+0000 7fc6e8dff700 1 Processor -- start 2023-10-08T02:11:47.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.087+0000 7fc6e8dff700 1 -- start start 2023-10-08T02:11:47.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e8dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6e40ccf90 0x7fc6e40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:47.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e8dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e40cc300 con 0x7fc6e40b1420 2023-10-08T02:11:47.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e8dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e40ad370 con 0x7fc6e40cbf30 2023-10-08T02:11:47.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e8dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e4156bf0 con 0x7fc6e40ccf90 2023-10-08T02:11:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e2d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6e40ccf90 0x7fc6e40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e2d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc6e40ccf90 0x7fc6e40ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50820/0 (socket says 172.21.15.119:50820) 2023-10-08T02:11:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e2d9d700 1 -- 172.21.15.119:0/1917844667 learned_addr learned my addr 172.21.15.119:0/1917844667 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6dbfff700 1 --1- 172.21.15.119:0/1917844667 >> v1:172.21.15.119:6789/0 conn(0x7fc6e40b1420 0x7fc6e40c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55734/0 (socket says 172.21.15.119:55734) 2023-10-08T02:11:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e2d9d700 1 -- 172.21.15.119:0/1917844667 >> v1:172.21.15.184:6789/0 conn(0x7fc6e40cbf30 legacy=0x7fc6e40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e2d9d700 1 -- 172.21.15.119:0/1917844667 >> v1:172.21.15.119:6789/0 conn(0x7fc6e40b1420 legacy=0x7fc6e40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:47.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e2d9d700 1 -- 172.21.15.119:0/1917844667 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e40ad650 con 0x7fc6e40ccf90 2023-10-08T02:11:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.089+0000 7fc6e1d9b700 1 -- 172.21.15.119:0/1917844667 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838091013 0 0) 0x7fc6e40cc300 con 0x7fc6e40b1420 2023-10-08T02:11:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.090+0000 7fc6e2d9d700 1 --2- 172.21.15.119:0/1917844667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40ccf90 0x7fc6e40ae410 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009f20 tx=0x7fc6dc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b59e688a8f68560 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.090+0000 7fc6e1d9b700 1 -- 172.21.15.119:0/1917844667 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc00e070 con 0x7fc6e40ccf90 2023-10-08T02:11:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.090+0000 7fc6e1d9b700 1 -- 172.21.15.119:0/1917844667 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc00ba80 con 0x7fc6e40ccf90 2023-10-08T02:11:47.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.090+0000 7fc6e1d9b700 1 -- 172.21.15.119:0/1917844667 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0092e0 con 0x7fc6e40ccf90 2023-10-08T02:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.090+0000 7fc6e8dff700 1 -- 172.21.15.119:0/1917844667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40ccf90 msgr2=0x7fc6e40ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.090+0000 7fc6e8dff700 1 --2- 172.21.15.119:0/1917844667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40ccf90 0x7fc6e40ae410 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc009f20 tx=0x7fc6dc009840 comp rx=0 tx=0).stop 2023-10-08T02:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.091+0000 7fc6e8dff700 1 -- 172.21.15.119:0/1917844667 shutdown_connections 2023-10-08T02:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.091+0000 7fc6e8dff700 1 --2- 172.21.15.119:0/1917844667 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40ccf90 0x7fc6e40ae410 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.091+0000 7fc6e8dff700 1 -- 172.21.15.119:0/1917844667 >> 172.21.15.119:0/1917844667 conn(0x7fc6e406a320 msgr2=0x7fc6e40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.091+0000 7fc6e8dff700 1 -- 172.21.15.119:0/1917844667 shutdown_connections 2023-10-08T02:11:47.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.091+0000 7fc6e8dff700 1 -- 172.21.15.119:0/1917844667 wait complete. 2023-10-08T02:11:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.092+0000 7fc6e8dff700 1 Processor -- start 2023-10-08T02:11:47.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.092+0000 7fc6e8dff700 1 -- start start 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6e8dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6e40b1420 0x7fc6e40c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6e8dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40cbf30 0x7fc6e414e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6e8dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6e40ccf90 0x7fc6e4159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6e8dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e4155710 con 0x7fc6e40cbf30 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6e8dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e40ca450 con 0x7fc6e40b1420 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6e8dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e4154650 con 0x7fc6e40ccf90 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6dbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40cbf30 0x7fc6e414e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6dbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40cbf30 0x7fc6e414e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50822/0 (socket says 172.21.15.119:50822) 2023-10-08T02:11:47.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6dbfff700 1 -- 172.21.15.119:0/2184743702 learned_addr learned my addr 172.21.15.119:0/2184743702 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6e2d9d700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6e40ccf90 0x7fc6e4159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.093+0000 7fc6e259c700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6e40b1420 0x7fc6e40c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6dbfff700 1 -- 172.21.15.119:0/2184743702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6e40ccf90 msgr2=0x7fc6e4159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6dbfff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6e40ccf90 0x7fc6e4159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6dbfff700 1 -- 172.21.15.119:0/2184743702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6e40b1420 msgr2=0x7fc6e40c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6dbfff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6e40b1420 0x7fc6e40c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6dbfff700 1 -- 172.21.15.119:0/2184743702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6dc009ae0 con 0x7fc6e40cbf30 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6dbfff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40cbf30 0x7fc6e414e7a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c5b0 tx=0x7fc6cc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6da7fc700 1 -- 172.21.15.119:0/2184743702 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc005a70 con 0x7fc6e40cbf30 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6da7fc700 1 -- 172.21.15.119:0/2184743702 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6cc005bf0 con 0x7fc6e40cbf30 2023-10-08T02:11:47.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e4153d90 con 0x7fc6e40cbf30 2023-10-08T02:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6da7fc700 1 -- 172.21.15.119:0/2184743702 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc0070d0 con 0x7fc6e40cbf30 2023-10-08T02:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.094+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e41540b0 con 0x7fc6e40cbf30 2023-10-08T02:11:47.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.095+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e41547e0 con 0x7fc6e40cbf30 2023-10-08T02:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.096+0000 7fc6da7fc700 1 -- 172.21.15.119:0/2184743702 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6cc007440 con 0x7fc6e40cbf30 2023-10-08T02:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.097+0000 7fc6da7fc700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6c4066f30 0x7fc6c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.097+0000 7fc6da7fc700 1 -- 172.21.15.119:0/2184743702 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fc6cc083cb0 con 0x7fc6e40cbf30 2023-10-08T02:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.097+0000 7fc6e259c700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6c4066f30 0x7fc6c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:47.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.098+0000 7fc6e259c700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6c4066f30 0x7fc6c40693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4003860 tx=0x7fc6d4009040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:47.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.101+0000 7fc6da7fc700 1 -- 172.21.15.119:0/2184743702 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6cc050cb0 con 0x7fc6e40cbf30 2023-10-08T02:11:47.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.290+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6e414eda0 con 0x7fc6c4066f30 2023-10-08T02:11:47.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.298+0000 7fc6da7fc700 1 -- 172.21.15.119:0/2184743702 <== mgr.7822 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+279887 (secure 0 0 0) 0x7fc6e414eda0 con 0x7fc6c4066f30 2023-10-08T02:11:47.301 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:47.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6c4066f30 msgr2=0x7fc6c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6c4066f30 0x7fc6c40693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4003860 tx=0x7fc6d4009040 comp rx=0 tx=0).stop 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40cbf30 msgr2=0x7fc6e414e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40cbf30 0x7fc6e414e7a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc00c5b0 tx=0x7fc6cc007d30 comp rx=0 tx=0).stop 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 shutdown_connections 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc6c4066f30 0x7fc6c40693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc6e40ccf90 0x7fc6e4159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc6e40b1420 0x7fc6e40c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 --2- 172.21.15.119:0/2184743702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc6e40cbf30 0x7fc6e414e7a0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 >> 172.21.15.119:0/2184743702 conn(0x7fc6e406a320 msgr2=0x7fc6e406e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 shutdown_connections 2023-10-08T02:11:47.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:47.302+0000 7fc6e8dff700 1 -- 172.21.15.119:0/2184743702 wait complete. 2023-10-08T02:11:47.305 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:11:47.319 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:11:45.590224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":442,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":88,"acting":88,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2488656,"kb_used_data":16144,"kb_used_omap":0,"kb_used_meta":2472448,"kb_avail":752486064,"statfs":{"total":773094113280,"available":770545729536,"internally_reserved":0,"allocated":16531456,"data_stored":8795583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2531786752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.526010"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114682+0000","last_change":"2023-10-08T02:11:42.113823+0000","last_active":"2023-10-08T02:11:42.114682+0000","last_peered":"2023-10-08T02:11:42.114682+0000","last_clean":"2023-10-08T02:11:42.114682+0000","last_became_active":"2023-10-08T02:11:42.113493+0000","last_became_peered":"2023-10-08T02:11:42.113493+0000","last_unstale":"2023-10-08T02:11:42.114682+0000","last_undegraded":"2023-10-08T02:11:42.114682+0000","last_fullsized":"2023-10-08T02:11:42.114682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.103531+0000","last_change":"2023-10-08T02:11:42.098697+0000","last_active":"2023-10-08T02:11:42.103531+0000","last_peered":"2023-10-08T02:11:42.103531+0000","last_clean":"2023-10-08T02:11:42.103531+0000","last_became_active":"2023-10-08T02:11:42.098447+0000","last_became_peered":"2023-10-08T02:11:42.098447+0000","last_unstale":"2023-10-08T02:11:42.103531+0000","last_undegraded":"2023-10-08T02:11:42.103531+0000","last_fullsized":"2023-10-08T02:11:42.103531+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113481+0000","last_change":"2023-10-08T02:11:42.111978+0000","last_active":"2023-10-08T02:11:42.113481+0000","last_peered":"2023-10-08T02:11:42.113481+0000","last_clean":"2023-10-08T02:11:42.113481+0000","last_became_active":"2023-10-08T02:11:42.110828+0000","last_became_peered":"2023-10-08T02:11:42.110828+0000","last_unstale":"2023-10-08T02:11:42.113481+0000","last_undegraded":"2023-10-08T02:11:42.113481+0000","last_fullsized":"2023-10-08T02:11:42.113481+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:11.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107993+0000","last_change":"2023-10-08T02:11:42.105477+0000","last_active":"2023-10-08T02:11:42.107993+0000","last_peered":"2023-10-08T02:11:42.107993+0000","last_clean":"2023-10-08T02:11:42.107993+0000","last_became_active":"2023-10-08T02:11:42.105203+0000","last_became_peered":"2023-10-08T02:11:42.105203+0000","last_unstale":"2023-10-08T02:11:42.107993+0000","last_undegraded":"2023-10-08T02:11:42.107993+0000","last_fullsized":"2023-10-08T02:11:42.107993+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:34.769593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.111811+0000","last_change":"2023-10-08T02:11:42.110297+0000","last_active":"2023-10-08T02:11:42.111811+0000","last_peered":"2023-10-08T02:11:42.111811+0000","last_clean":"2023-10-08T02:11:42.111811+0000","last_became_active":"2023-10-08T02:11:42.110051+0000","last_became_peered":"2023-10-08T02:11:42.110051+0000","last_unstale":"2023-10-08T02:11:42.111811+0000","last_undegraded":"2023-10-08T02:11:42.111811+0000","last_fullsized":"2023-10-08T02:11:42.111811+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:02.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106088+0000","last_change":"2023-10-08T02:11:42.105002+0000","last_active":"2023-10-08T02:11:42.106088+0000","last_peered":"2023-10-08T02:11:42.106088+0000","last_clean":"2023-10-08T02:11:42.106088+0000","last_became_active":"2023-10-08T02:11:42.104707+0000","last_became_peered":"2023-10-08T02:11:42.104707+0000","last_unstale":"2023-10-08T02:11:42.106088+0000","last_undegraded":"2023-10-08T02:11:42.106088+0000","last_fullsized":"2023-10-08T02:11:42.106088+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106069+0000","last_change":"2023-10-08T02:11:42.104624+0000","last_active":"2023-10-08T02:11:42.106069+0000","last_peered":"2023-10-08T02:11:42.106069+0000","last_clean":"2023-10-08T02:11:42.106069+0000","last_became_active":"2023-10-08T02:11:42.104319+0000","last_became_peered":"2023-10-08T02:11:42.104319+0000","last_unstale":"2023-10-08T02:11:42.106069+0000","last_undegraded":"2023-10-08T02:11:42.106069+0000","last_fullsized":"2023-10-08T02:11:42.106069+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:56.323343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067690+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:10.718365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072538+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:22.795910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070853+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:40.446588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075526+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:07:30.613532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.076459+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:50.165805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070924+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:17:37.906952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067817+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:00.041784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075337+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:14:20.425942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100491+0000","last_change":"2023-10-08T02:11:42.095233+0000","last_active":"2023-10-08T02:11:42.100491+0000","last_peered":"2023-10-08T02:11:42.100491+0000","last_clean":"2023-10-08T02:11:42.100491+0000","last_became_active":"2023-10-08T02:11:42.094991+0000","last_became_peered":"2023-10-08T02:11:42.094991+0000","last_unstale":"2023-10-08T02:11:42.100491+0000","last_undegraded":"2023-10-08T02:11:42.100491+0000","last_fullsized":"2023-10-08T02:11:42.100491+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:58.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.073213+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:23:52.815532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.069899+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:36.101961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072441+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:02.277276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.066903+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:50.650114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100695+0000","last_change":"2023-10-08T02:11:42.096482+0000","last_active":"2023-10-08T02:11:42.100695+0000","last_peered":"2023-10-08T02:11:42.100695+0000","last_clean":"2023-10-08T02:11:42.100695+0000","last_became_active":"2023-10-08T02:11:42.094165+0000","last_became_peered":"2023-10-08T02:11:42.094165+0000","last_unstale":"2023-10-08T02:11:42.100695+0000","last_undegraded":"2023-10-08T02:11:42.100695+0000","last_fullsized":"2023-10-08T02:11:42.100695+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:23.801545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086789+0000","last_change":"2023-10-08T02:11:41.084280+0000","last_active":"2023-10-08T02:11:42.086789+0000","last_peered":"2023-10-08T02:11:42.086789+0000","last_clean":"2023-10-08T02:11:42.086789+0000","last_became_active":"2023-10-08T02:11:41.083971+0000","last_became_peered":"2023-10-08T02:11:41.083971+0000","last_unstale":"2023-10-08T02:11:42.086789+0000","last_undegraded":"2023-10-08T02:11:42.086789+0000","last_fullsized":"2023-10-08T02:11:42.086789+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075287+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:07:22.374785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067379+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072535+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:10.498785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.071357+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:12:35.260528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067542+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:28.886973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072478+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:57.358384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113445+0000","last_change":"2023-10-08T02:11:42.111899+0000","last_active":"2023-10-08T02:11:42.113445+0000","last_peered":"2023-10-08T02:11:42.113445+0000","last_clean":"2023-10-08T02:11:42.113445+0000","last_became_active":"2023-10-08T02:11:42.111582+0000","last_became_peered":"2023-10-08T02:11:42.111582+0000","last_unstale":"2023-10-08T02:11:42.113445+0000","last_undegraded":"2023-10-08T02:11:42.113445+0000","last_fullsized":"2023-10-08T02:11:42.113445+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:11.724000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072605+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:25.497645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070329+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:31.493936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070821+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:24:42.593819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072123+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:23:06.207304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.7","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125349+0000","last_change":"2023-10-08T02:11:41.087828+0000","last_active":"2023-10-08T02:11:42.125349+0000","last_peered":"2023-10-08T02:11:42.125349+0000","last_clean":"2023-10-08T02:11:42.125349+0000","last_became_active":"2023-10-08T02:11:41.087533+0000","last_became_peered":"2023-10-08T02:11:41.087533+0000","last_unstale":"2023-10-08T02:11:42.125349+0000","last_undegraded":"2023-10-08T02:11:42.125349+0000","last_fullsized":"2023-10-08T02:11:42.125349+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.101659+0000","last_change":"2023-10-08T02:11:42.098444+0000","last_active":"2023-10-08T02:11:42.101659+0000","last_peered":"2023-10-08T02:11:42.101659+0000","last_clean":"2023-10-08T02:11:42.101659+0000","last_became_active":"2023-10-08T02:11:42.098168+0000","last_became_peered":"2023-10-08T02:11:42.098168+0000","last_unstale":"2023-10-08T02:11:42.101659+0000","last_undegraded":"2023-10-08T02:11:42.101659+0000","last_fullsized":"2023-10-08T02:11:42.101659+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:12.133057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067619+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:55.317708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075585+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:07:56.324149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.072482+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:30:24.569836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.075994+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:50.650114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067082+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:11.127422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.070305+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:06:18.323015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.069817+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:28.409914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:11:41.063653+0000","last_change":"2023-10-08T02:11:41.063653+0000","last_active":"2023-10-08T02:11:41.063653+0000","last_peered":"2023-10-08T02:11:41.063653+0000","last_clean":"2023-10-08T02:11:41.063653+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:41.063653+0000","last_undegraded":"2023-10-08T02:11:41.063653+0000","last_fullsized":"2023-10-08T02:11:41.063653+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:11:40.058018+0000","last_change":"2023-10-08T02:11:40.067076+0000","last_active":"2023-10-08T02:11:40.058018+0000","last_peered":"2023-10-08T02:11:40.058018+0000","last_clean":"2023-10-08T02:11:40.058018+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:11:40.058018+0000","last_undegraded":"2023-10-08T02:11:40.058018+0000","last_fullsized":"2023-10-08T02:11:40.058018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.160399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"146'271","reported_seq":820,"reported_epoch":153,"state":"active+clean","last_fresh":"2023-10-08T02:11:37.032549+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:11:37.032549+0000","last_peered":"2023-10-08T02:11:37.032549+0000","last_clean":"2023-10-08T02:11:37.032549+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:11:37.032549+0000","last_undegraded":"2023-10-08T02:11:37.032549+0000","last_fullsized":"2023-10-08T02:11:37.032549+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":22,"acting":22,"num_store_stats":1},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307920,"kb_used_data":2440,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063920,"statfs":{"total":96636764160,"available":96321454080,"internally_reserved":0,"allocated":2498560,"data_stored":1535644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44400000000000001},"max":{"1min":0.96499999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.755}]},{"osd":1,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.60899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.74199999999999999},"min":{"1min":0.621,"5min":0.53100000000000003,"15min":0.48899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":1.0580000000000001},"last":0.47899999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":1.0600000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.48999999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.58399999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.626},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82799999999999996,"15min":0.80100000000000005},"min":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.57399999999999995},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.80900000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.752,"5min":0.94999999999999996,"15min":0.98399999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.67800000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.71099999999999997}]},{"osd":6,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.57299999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.92400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.73199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315544,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056296,"statfs":{"total":96636764160,"available":96313647104,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76500000000000001,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77200000000000002,"15min":0.748},"min":{"1min":0.54900000000000004,"5min":0.501,"15min":0.46400000000000002},"max":{"1min":1.4139999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.48599999999999999,"15min":0.47599999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.875},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77500000000000002,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.48899999999999999},"max":{"1min":1.464,"5min":1.472,"15min":1.472},"last":0.624}]},{"osd":2,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.73499999999999999},"min":{"1min":0.60099999999999998,"5min":0.53500000000000003,"15min":0.47699999999999998},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.79700000000000004,"15min":0.76600000000000001},"min":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.436},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.55600000000000005}]},{"osd":3,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.60099999999999998,"5min":0.57999999999999996,"15min":0.52800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.78500000000000003},"min":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"max":{"1min":1.048,"5min":1.169,"15min":1.169},"last":0.71799999999999997}]},{"osd":4,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":1.0129999999999999,"5min":1.133,"15min":1.133},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63800000000000001,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.90200000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82999999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54800000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95599999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.90300000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.752,"15min":0.72099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0289999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98099999999999998,"5min":1.232,"15min":1.232},"last":0.85299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575752,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307340,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064500,"statfs":{"total":96636764160,"available":96322048000,"internally_reserved":0,"allocated":1904640,"data_stored":945276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.36299999999999999,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.69899999999999995,"5min":1.024,"15min":1.024},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66200000000000003,"5min":1.603,"15min":1.603},"last":0.41399999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.505},"min":{"1min":0.40200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64900000000000002,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77700000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.42699999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.872,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.71599999999999997,"5min":3.698,"15min":3.698},"last":0.45800000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.88800000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.999,"5min":1.734,"15min":1.734},"last":0.70299999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.80100000000000005},"min":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.008,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.58999999999999997,"15min":0.56100000000000005},"max":{"1min":1.0640000000000001,"5min":1.403,"15min":1.403},"last":0.55100000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.66300000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0269999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.68100000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.04,"5min":1.573,"15min":1.573},"last":0.65900000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1120000000000001,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.58999999999999997},"max":{"1min":1.028,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.80800000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2452,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2510848,"data_stored":1541913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48499999999999999,"15min":0.46999999999999997},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.254},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.443}]},{"osd":2,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.317},"max":{"1min":0.59699999999999998,"5min":0.70699999999999996,"15min":0.746},"last":0.378},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.37},"max":{"1min":0.69099999999999995,"5min":0.88300000000000001,"15min":1.325},"last":0.49099999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.44600000000000001,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.873,"5min":0.875,"15min":0.875},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.54300000000000004,"5min":0.47399999999999998,"15min":0.45900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48199999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.70899999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.24},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80200000000000005,"15min":0.79500000000000004},"min":{"1min":0.621,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3759999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.73599999999999999}]},{"osd":5,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.94799999999999995,"5min":1.327,"15min":1.327},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89700000000000002,"15min":0.88200000000000001},"min":{"1min":0.754,"5min":0.60999999999999999,"15min":0.56100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.871}]},{"osd":6,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85399999999999998,"15min":0.83899999999999997},"min":{"1min":0.66100000000000003,"5min":0.55000000000000004,"15min":0.53400000000000003},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.55500000000000005},"max":{"1min":0.90900000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.875,"15min":0.84999999999999998},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.55800000000000005},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.5269999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.85799999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311492,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060348,"statfs":{"total":96636764160,"available":96317796352,"internally_reserved":0,"allocated":1961984,"data_stored":972839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48699999999999999,"15min":0.47999999999999998},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":1.228,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66800000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.79500000000000004,"15min":0.872},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.41799999999999998,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.54000000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.46800000000000003,"5min":0.45500000000000002,"15min":0.39400000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.83299999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.92300000000000004,"15min":0.871},"min":{"1min":0.749,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":1.046},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89400000000000002,"15min":0.86399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6220000000000001,"5min":1.847,"15min":1.847},"last":1.0860000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.878,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.87},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0580000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.90300000000000002,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.64100000000000001,"5min":0.622,"15min":0.58899999999999997},"max":{"1min":1.7909999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.998}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315544,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056296,"statfs":{"total":96636764160,"available":96313647104,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.46500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.72199999999999998,"15min":0.87},"last":0.436},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.32600000000000001},"max":{"1min":0.627,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.42399999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.56000000000000005,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68300000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.5}]},{"osd":2,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.42299999999999999,"15min":0.41999999999999998},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.629},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45600000000000002},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.54300000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.214},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.65400000000000003,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.1599999999999999},"last":0.751}]},{"osd":5,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79900000000000004,"15min":0.80600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"max":{"1min":1.0209999999999999,"5min":1.0589999999999999,"15min":1.125},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82299999999999995,"15min":0.82499999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56899999999999995},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.67800000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85199999999999998,"15min":0.83799999999999997},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58499999999999996},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.85499999999999998,"15min":0.84499999999999997},"min":{"1min":0.67800000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.77300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311448,"kb_used_data":1872,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060392,"statfs":{"total":96636764160,"available":96317841408,"internally_reserved":0,"allocated":1916928,"data_stored":951545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40400000000000003},"max":{"1min":0.90100000000000002,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0009999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.86099999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.52},"last":0.80100000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.748},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60999999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90800000000000003,"5min":1.1579999999999999,"15min":1.403},"last":0.73399999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.208,"15min":1.5069999999999999},"last":0.82899999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":1.05,"15min":1.05},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93400000000000005,"5min":1.038,"15min":1.1559999999999999},"last":0.69799999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.016},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.0680000000000001,"15min":1.129},"last":0.628},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.48799999999999999},"max":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.68100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311436,"kb_used_data":1860,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060404,"statfs":{"total":96636764160,"available":96317853696,"internally_reserved":0,"allocated":1904640,"data_stored":945276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.73899999999999999},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.38900000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.75800000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.438},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70699999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.78000000000000003},"min":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.42699999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.46200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80600000000000005,"15min":0.78100000000000003},"min":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.499},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.621},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.76200000000000001},"min":{"1min":0.61599999999999999,"5min":0.57499999999999996,"15min":0.50900000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.63500000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.56100000000000005,"15min":0.497},"max":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.0529999999999999,"15min":1.091},"last":0.78000000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.627,"15min":0.62},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.0620000000000001,"15min":1.115},"last":0.81299999999999994}]},{"osd":6,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":0.93999999999999995,"5min":1.006,"15min":1.0629999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52000000000000002,"5min":0.436,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":0.95299999999999996,"15min":1.077},"last":0.79300000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":1.0249999999999999,"15min":1.161},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.88100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:11:47.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3e is not active+clean 2023-10-08T02:11:47.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3d is not active+clean 2023-10-08T02:11:47.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3b is not active+clean 2023-10-08T02:11:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3a is not active+clean 2023-10-08T02:11:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.39 is not active+clean 2023-10-08T02:11:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.38 is not active+clean 2023-10-08T02:11:47.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.37 is not active+clean 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.36 is not active+clean 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.35 is not active+clean 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.34 is not active+clean 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.32 is not active+clean 2023-10-08T02:11:47.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.30 is not active+clean 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2f is not active+clean 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2e is not active+clean 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2d is not active+clean 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2c is not active+clean 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2b is not active+clean 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2a is not active+clean 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.29 is not active+clean 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.28 is not active+clean 2023-10-08T02:11:47.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.27 is not active+clean 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.26 is not active+clean 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.24 is not active+clean 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.21 is not active+clean 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.20 is not active+clean 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1e is not active+clean 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.067690+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:23:10.718365+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1f is not active+clean 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1e is not active+clean 2023-10-08T02:11:47.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.072538+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:14:22.795910+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1c is not active+clean 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.070853+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:59:40.446588+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1d is not active+clean 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.075526+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:07:30.613532+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:11:47.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1a is not active+clean 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.076459+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:10:50.165805+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1b is not active+clean 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1a is not active+clean 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.070924+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:17:37.906952+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.18 is not active+clean 2023-10-08T02:11:47.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.19 is not active+clean 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.067817+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:11:00.041784+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.19 is not active+clean 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.18 is not active+clean 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.075337+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:14:20.425942+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.17 is not active+clean 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.073213+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T13:23:52.815532+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:11:47.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.17 is not active+clean 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.16 is not active+clean 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.069899+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:34:36.101961+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.14 is not active+clean 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.072441+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T13:25:02.277276+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.15 is not active+clean 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2023-10-08T02:11:47.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.066903+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:14:50.650114+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:11:47.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.10 is not active+clean 2023-10-08T02:11:47.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2023-10-08T02:11:47.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.075287+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T14:07:22.374785+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:11:47.336 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.11 is not active+clean 2023-10-08T02:11:47.336 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.10 is not active+clean 2023-10-08T02:11:47.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.067379+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:11:47.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.e is not active+clean 2023-10-08T02:11:47.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2023-10-08T02:11:47.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.072535+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T13:28:10.498785+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:11:47.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.f is not active+clean 2023-10-08T02:11:47.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.e is not active+clean 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.071357+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:12:35.260528+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.c is not active+clean 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.067542+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:34:28.886973+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.d is not active+clean 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2023-10-08T02:11:47.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.072478+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:40:57.358384+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.b is not active+clean 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.072605+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:19:25.497645+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.b is not active+clean 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.a is not active+clean 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.070329+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T14:10:31.493936+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.8 is not active+clean 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:47.654 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.9 is not active+clean 2023-10-08T02:11:47.655 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.070821+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:24:42.593819+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:11:48.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.9 is not active+clean 2023-10-08T02:11:48.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2023-10-08T02:11:48.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.072123+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:23:06.207304+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:11:48.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.6 is not active+clean 2023-10-08T02:11:48.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2023-10-08T02:11:48.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.067619+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:48:55.317708+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:11:48.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.4 is not active+clean 2023-10-08T02:11:48.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.5 is not active+clean 2023-10-08T02:11:48.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.075585+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:07:56.324149+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:11:48.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.5 is not active+clean 2023-10-08T02:11:48.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.072482+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:30:24.569836+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.2 is not active+clean 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.075994+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:14:50.650114+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.3 is not active+clean 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2023-10-08T02:11:48.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.067082+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:01:11.127422+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:11:48.124 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.0 is not active+clean 2023-10-08T02:11:48.124 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1 is not active+clean 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.070305+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:06:18.323015+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.1 is not active+clean 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.069817+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:42:28.409914+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 19.12 is not active+clean 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:11:41.063653+0000', 'last_change': '2023-10-08T02:11:41.063653+0000', 'last_active': '2023-10-08T02:11:41.063653+0000', 'last_peered': '2023-10-08T02:11:41.063653+0000', 'last_clean': '2023-10-08T02:11:41.063653+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:41.063653+0000', 'last_undegraded': '2023-10-08T02:11:41.063653+0000', 'last_fullsized': '2023-10-08T02:11:41.063653+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:41.063653+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:11:48.125 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2023-10-08T02:11:48.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:11:40.058018+0000', 'last_change': '2023-10-08T02:11:40.067076+0000', 'last_active': '2023-10-08T02:11:40.058018+0000', 'last_peered': '2023-10-08T02:11:40.058018+0000', 'last_clean': '2023-10-08T02:11:40.058018+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:11:40.058018+0000', 'last_undegraded': '2023-10-08T02:11:40.058018+0000', 'last_fullsized': '2023-10-08T02:11:40.058018+0000', 'mapping_epoch': 154, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 154, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'last_clean_scrub_stamp': '2023-10-08T02:11:40.058018+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:21:52.160399+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:11:48.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T02:11:48.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.310+0000 7f379219d700 1 Processor -- start 2023-10-08T02:11:48.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.310+0000 7f379219d700 1 -- start start 2023-10-08T02:11:48.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.311+0000 7f379219d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15c9a0 0x7f378c0cfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.311+0000 7f379219d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c0cfa60 con 0x7f378c15cd70 2023-10-08T02:11:48.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.311+0000 7f379219d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378c0cee50 con 0x7f378c0ce900 2023-10-08T02:11:48.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.311+0000 7f379219d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0cf130 con 0x7f378c15c9a0 2023-10-08T02:11:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f379199c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f378c15cd70 0x7f378c15d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55736/0 (socket says 172.21.15.119:55736) 2023-10-08T02:11:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f379199c700 1 -- 172.21.15.119:0/868058340 learned_addr learned my addr 172.21.15.119:0/868058340 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f379119b700 1 --2- 172.21.15.119:0/868058340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15c9a0 0x7f378c0cfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f3783fff700 1 -- 172.21.15.119:0/868058340 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409134150 0 0) 0x7f378c0cfa60 con 0x7f378c15cd70 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f3783fff700 1 -- 172.21.15.119:0/868058340 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3778003570 con 0x7f378c15cd70 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f3783fff700 1 -- 172.21.15.119:0/868058340 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243726204 0 0) 0x7f378c0cee50 con 0x7f378c0ce900 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f3783fff700 1 -- 172.21.15.119:0/868058340 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f378c0cfa60 con 0x7f378c0ce900 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f379119b700 1 -- 172.21.15.119:0/868058340 >> v1:172.21.15.184:6789/0 conn(0x7f378c0ce900 legacy=0x7f378c0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f379119b700 1 -- 172.21.15.119:0/868058340 >> v1:172.21.15.119:6789/0 conn(0x7f378c15cd70 legacy=0x7f378c15d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.312+0000 7f379119b700 1 -- 172.21.15.119:0/868058340 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c0cf410 con 0x7f378c15c9a0 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f379119b700 1 --2- 172.21.15.119:0/868058340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15c9a0 0x7f378c0cfef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3788009910 tx=0x7f37880091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e17883afd83c98a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f3783fff700 1 -- 172.21.15.119:0/868058340 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788011630 con 0x7f378c15c9a0 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f3783fff700 1 -- 172.21.15.119:0/868058340 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3788011790 con 0x7f378c15c9a0 2023-10-08T02:11:48.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f3783fff700 1 -- 172.21.15.119:0/868058340 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788011b10 con 0x7f378c15c9a0 2023-10-08T02:11:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f379219d700 1 -- 172.21.15.119:0/868058340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15c9a0 msgr2=0x7f378c0cfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f379219d700 1 --2- 172.21.15.119:0/868058340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15c9a0 0x7f378c0cfef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3788009910 tx=0x7f37880091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f379219d700 1 -- 172.21.15.119:0/868058340 shutdown_connections 2023-10-08T02:11:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f379219d700 1 --2- 172.21.15.119:0/868058340 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c15c9a0 0x7f378c0cfef0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f379219d700 1 -- 172.21.15.119:0/868058340 >> 172.21.15.119:0/868058340 conn(0x7f378c0d05c0 msgr2=0x7f378c0d09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.313+0000 7f379219d700 1 -- 172.21.15.119:0/868058340 shutdown_connections 2023-10-08T02:11:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.314+0000 7f379219d700 1 -- 172.21.15.119:0/868058340 wait complete. 2023-10-08T02:11:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.315+0000 7f379219d700 1 Processor -- start 2023-10-08T02:11:48.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.315+0000 7f379219d700 1 -- start start 2023-10-08T02:11:48.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379219d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c0ce900 0x7f378c15b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379219d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c15c9a0 0x7f378c153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379219d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f378c15cd70 0x7f378c0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379219d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0b5ff0 con 0x7f378c15cd70 2023-10-08T02:11:48.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379219d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0b4430 con 0x7f378c15c9a0 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379219d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c0b3820 con 0x7f378c0ce900 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379199c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f378c15cd70 0x7f378c0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379199c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f378c15cd70 0x7f378c0c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50838/0 (socket says 172.21.15.119:50838) 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379199c700 1 -- 172.21.15.119:0/886845753 learned_addr learned my addr 172.21.15.119:0/886845753 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379119b700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c0ce900 0x7f378c15b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.316+0000 7f379099a700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c15c9a0 0x7f378c153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f379199c700 1 -- 172.21.15.119:0/886845753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c0ce900 msgr2=0x7f378c15b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f379199c700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c0ce900 0x7f378c15b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f379199c700 1 -- 172.21.15.119:0/886845753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c15c9a0 msgr2=0x7f378c153210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f379199c700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c15c9a0 0x7f378c153210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f379199c700 1 -- 172.21.15.119:0/886845753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3788009420 con 0x7f378c15cd70 2023-10-08T02:11:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f379199c700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f378c15cd70 0x7f378c0c64e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f378400cc20 tx=0x7f3784001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f37817fa700 1 -- 172.21.15.119:0/886845753 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400a850 con 0x7f378c15cd70 2023-10-08T02:11:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f37817fa700 1 -- 172.21.15.119:0/886845753 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378400a9d0 con 0x7f378c15cd70 2023-10-08T02:11:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378c0b3b00 con 0x7f378c15cd70 2023-10-08T02:11:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f37817fa700 1 -- 172.21.15.119:0/886845753 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378400ad90 con 0x7f378c15cd70 2023-10-08T02:11:48.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.317+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c0b40c0 con 0x7f378c15cd70 2023-10-08T02:11:48.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.318+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f378c088ea0 con 0x7f378c15cd70 2023-10-08T02:11:48.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.319+0000 7f37817fa700 1 -- 172.21.15.119:0/886845753 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3784007300 con 0x7f378c15cd70 2023-10-08T02:11:48.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.320+0000 7f37817fa700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3770066f30 0x7f37700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.320+0000 7f37817fa700 1 -- 172.21.15.119:0/886845753 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f3784084a40 con 0x7f378c15cd70 2023-10-08T02:11:48.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.320+0000 7f379119b700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3770066f30 0x7f37700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.321+0000 7f379119b700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3770066f30 0x7f37700693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3788009fd0 tx=0x7f3788017000 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:48.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.324+0000 7f37817fa700 1 -- 172.21.15.119:0/886845753 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3784051ac0 con 0x7f378c15cd70 2023-10-08T02:11:48.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.590+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f378c0b4570 con 0x7f378c15cd70 2023-10-08T02:11:48.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.590+0000 7f37817fa700 1 -- 172.21.15.119:0/886845753 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1018 (secure 0 0 0) 0x7f37840550e0 con 0x7f378c15cd70 2023-10-08T02:11:48.590 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:48.590 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4585,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":157,"num_osds":8,"num_up_osds":8,"osd_up_since":1696726535,"num_in_osds":8,"osd_in_since":1696726526,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592674,"bytes_used":2548736000,"bytes_avail":770545377280,"bytes_total":773094113280,"write_bytes_sec":2041,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":71,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"d","status":"up:active","gid":8667}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-10-08T02:01:45.128305+0000","services":{}},"progress_events":{}} 2023-10-08T02:11:48.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3770066f30 msgr2=0x7f37700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3770066f30 0x7f37700693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3788009fd0 tx=0x7f3788017000 comp rx=0 tx=0).stop 2023-10-08T02:11:48.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f378c15cd70 msgr2=0x7f378c0c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f378c15cd70 0x7f378c0c64e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f378400cc20 tx=0x7f3784001a40 comp rx=0 tx=0).stop 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 shutdown_connections 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3770066f30 0x7f37700693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f378c0ce900 0x7f378c15b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f378c15cd70 0x7f378c0c64e0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 --2- 172.21.15.119:0/886845753 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f378c15c9a0 0x7f378c153210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.592+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 >> 172.21.15.119:0/886845753 conn(0x7f378c0d05c0 msgr2=0x7f378c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.593+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 shutdown_connections 2023-10-08T02:11:48.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.593+0000 7f379219d700 1 -- 172.21.15.119:0/886845753 wait complete. 2023-10-08T02:11:48.610 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T02:11:48.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:11:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.791+0000 7efe645ff700 1 Processor -- start 2023-10-08T02:11:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.791+0000 7efe645ff700 1 -- start start 2023-10-08T02:11:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.792+0000 7efe645ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0cbf30 0x7efe5c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe645ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe5c0b1420 0x7efe5c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe645ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5c0cc300 con 0x7efe5c0ccf90 2023-10-08T02:11:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe645ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5c156bf0 con 0x7efe5c0b1420 2023-10-08T02:11:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe645ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0ae950 con 0x7efe5c0cbf30 2023-10-08T02:11:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe62b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe5c0b1420 0x7efe5c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe62b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe5c0b1420 0x7efe5c0c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50854/0 (socket says 172.21.15.119:50854) 2023-10-08T02:11:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe62b9c700 1 -- 172.21.15.119:0/4065748848 learned_addr learned my addr 172.21.15.119:0/4065748848 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe6239b700 1 --2- 172.21.15.119:0/4065748848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0cbf30 0x7efe5c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe62b9c700 1 -- 172.21.15.119:0/4065748848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0cbf30 msgr2=0x7efe5c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe62b9c700 1 --2- 172.21.15.119:0/4065748848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0cbf30 0x7efe5c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe62b9c700 1 -- 172.21.15.119:0/4065748848 >> v1:172.21.15.184:6789/0 conn(0x7efe5c0ccf90 legacy=0x7efe5c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.793+0000 7efe62b9c700 1 -- 172.21.15.119:0/4065748848 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0ad370 con 0x7efe5c0b1420 2023-10-08T02:11:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.794+0000 7efe62b9c700 1 --2- 172.21.15.119:0/4065748848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0b1420 0x7efe5c0c9740 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7efe54009f50 tx=0x7efe54009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ff56ca853b1b70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.794+0000 7efe61399700 1 -- 172.21.15.119:0/4065748848 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe540092e0 con 0x7efe5c0b1420 2023-10-08T02:11:48.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.794+0000 7efe61399700 1 -- 172.21.15.119:0/4065748848 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe5400b9c0 con 0x7efe5c0b1420 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.794+0000 7efe61399700 1 -- 172.21.15.119:0/4065748848 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe5400bd80 con 0x7efe5c0b1420 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.794+0000 7efe645ff700 1 -- 172.21.15.119:0/4065748848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0b1420 msgr2=0x7efe5c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.794+0000 7efe645ff700 1 --2- 172.21.15.119:0/4065748848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0b1420 0x7efe5c0c9740 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7efe54009f50 tx=0x7efe54009840 comp rx=0 tx=0).stop 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.795+0000 7efe645ff700 1 -- 172.21.15.119:0/4065748848 shutdown_connections 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.795+0000 7efe645ff700 1 --2- 172.21.15.119:0/4065748848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0b1420 0x7efe5c0c9740 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.795+0000 7efe645ff700 1 --2- 172.21.15.119:0/4065748848 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0cbf30 0x7efe5c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.795+0000 7efe645ff700 1 -- 172.21.15.119:0/4065748848 >> 172.21.15.119:0/4065748848 conn(0x7efe5c06a320 msgr2=0x7efe5c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.795+0000 7efe645ff700 1 -- 172.21.15.119:0/4065748848 shutdown_connections 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.795+0000 7efe645ff700 1 -- 172.21.15.119:0/4065748848 wait complete. 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.796+0000 7efe645ff700 1 Processor -- start 2023-10-08T02:11:48.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.796+0000 7efe645ff700 1 -- start start 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe645ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe5c0b1420 0x7efe5c1535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe645ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0cbf30 0x7efe5c0a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe645ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0ccf90 0x7efe5c158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe645ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0ab330 con 0x7efe5c0cbf30 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe645ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0aa780 con 0x7efe5c0b1420 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe645ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c153af0 con 0x7efe5c0ccf90 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe61b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0cbf30 0x7efe5c0a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe61b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0cbf30 0x7efe5c0a8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50870/0 (socket says 172.21.15.119:50870) 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe61b9a700 1 -- 172.21.15.119:0/3163200136 learned_addr learned my addr 172.21.15.119:0/3163200136 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:48.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.797+0000 7efe6239b700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe5c0b1420 0x7efe5c1535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe62b9c700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0ccf90 0x7efe5c158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe61b9a700 1 -- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0ccf90 msgr2=0x7efe5c158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe61b9a700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0ccf90 0x7efe5c158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe61b9a700 1 -- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe5c0b1420 msgr2=0x7efe5c1535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe61b9a700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe5c0b1420 0x7efe5c1535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe61b9a700 1 -- 172.21.15.119:0/3163200136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe54009ae0 con 0x7efe5c0cbf30 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe6239b700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe5c0b1420 0x7efe5c1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe61b9a700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0cbf30 0x7efe5c0a8a70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c510 tx=0x7efe4c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe527fc700 1 -- 172.21.15.119:0/3163200136 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011590 con 0x7efe5c0cbf30 2023-10-08T02:11:48.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe527fc700 1 -- 172.21.15.119:0/3163200136 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c011710 con 0x7efe5c0cbf30 2023-10-08T02:11:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c0aaa00 con 0x7efe5c0cbf30 2023-10-08T02:11:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe527fc700 1 -- 172.21.15.119:0/3163200136 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011ad0 con 0x7efe5c0cbf30 2023-10-08T02:11:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.798+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c0aafc0 con 0x7efe5c0cbf30 2023-10-08T02:11:48.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.799+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5c04c3c0 con 0x7efe5c0cbf30 2023-10-08T02:11:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.800+0000 7efe527fc700 1 -- 172.21.15.119:0/3163200136 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe4c01b3b0 con 0x7efe5c0cbf30 2023-10-08T02:11:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.801+0000 7efe527fc700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efe40066f30 0x7efe400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.801+0000 7efe527fc700 1 -- 172.21.15.119:0/3163200136 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7efe4c083c70 con 0x7efe5c0cbf30 2023-10-08T02:11:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.801+0000 7efe6239b700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efe40066f30 0x7efe400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:48.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.802+0000 7efe6239b700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efe40066f30 0x7efe400693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe58009d80 tx=0x7efe58008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:48.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.805+0000 7efe527fc700 1 -- 172.21.15.119:0/3163200136 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe4c050c70 con 0x7efe5c0cbf30 2023-10-08T02:11:48.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:48.992+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe5c0b0af0 con 0x7efe40066f30 2023-10-08T02:11:48.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.000+0000 7efe527fc700 1 -- 172.21.15.119:0/3163200136 <== mgr.7822 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289328 (secure 0 0 0) 0x7efe5c0b0af0 con 0x7efe40066f30 2023-10-08T02:11:49.003 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efe40066f30 msgr2=0x7efe400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efe40066f30 0x7efe400693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe58009d80 tx=0x7efe58008040 comp rx=0 tx=0).stop 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0cbf30 msgr2=0x7efe5c0a8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0cbf30 0x7efe5c0a8a70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c510 tx=0x7efe4c00c8d0 comp rx=0 tx=0).stop 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 shutdown_connections 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7efe40066f30 0x7efe400693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efe5c0ccf90 0x7efe5c158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efe5c0b1420 0x7efe5c1535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 --2- 172.21.15.119:0/3163200136 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5c0cbf30 0x7efe5c0a8a70 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 >> 172.21.15.119:0/3163200136 conn(0x7efe5c06a320 msgr2=0x7efe5c0afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:49.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 shutdown_connections 2023-10-08T02:11:49.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:49.004+0000 7efe645ff700 1 -- 172.21.15.119:0/3163200136 wait complete. 2023-10-08T02:11:49.007 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:11:49.021 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T02:11:47.590724+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":483,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2489000,"kb_used_data":16488,"kb_used_omap":0,"kb_used_meta":2472448,"kb_avail":752485720,"statfs":{"total":773094113280,"available":770545377280,"internally_reserved":0,"allocated":16883712,"data_stored":8963356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2531786752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.521032"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114682+0000","last_change":"2023-10-08T02:11:42.113823+0000","last_active":"2023-10-08T02:11:42.114682+0000","last_peered":"2023-10-08T02:11:42.114682+0000","last_clean":"2023-10-08T02:11:42.114682+0000","last_became_active":"2023-10-08T02:11:42.113493+0000","last_became_peered":"2023-10-08T02:11:42.113493+0000","last_unstale":"2023-10-08T02:11:42.114682+0000","last_undegraded":"2023-10-08T02:11:42.114682+0000","last_fullsized":"2023-10-08T02:11:42.114682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114624+0000","last_change":"2023-10-08T02:11:42.113537+0000","last_active":"2023-10-08T02:11:42.114624+0000","last_peered":"2023-10-08T02:11:42.114624+0000","last_clean":"2023-10-08T02:11:42.114624+0000","last_became_active":"2023-10-08T02:11:42.113170+0000","last_became_peered":"2023-10-08T02:11:42.113170+0000","last_unstale":"2023-10-08T02:11:42.114624+0000","last_undegraded":"2023-10-08T02:11:42.114624+0000","last_fullsized":"2023-10-08T02:11:42.114624+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:23:53.821167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108736+0000","last_change":"2023-10-08T02:11:42.098817+0000","last_active":"2023-10-08T02:11:42.108736+0000","last_peered":"2023-10-08T02:11:42.108736+0000","last_clean":"2023-10-08T02:11:42.108736+0000","last_became_active":"2023-10-08T02:11:42.098478+0000","last_became_peered":"2023-10-08T02:11:42.098478+0000","last_unstale":"2023-10-08T02:11:42.108736+0000","last_undegraded":"2023-10-08T02:11:42.108736+0000","last_fullsized":"2023-10-08T02:11:42.108736+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:02.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.103531+0000","last_change":"2023-10-08T02:11:42.098697+0000","last_active":"2023-10-08T02:11:42.103531+0000","last_peered":"2023-10-08T02:11:42.103531+0000","last_clean":"2023-10-08T02:11:42.103531+0000","last_became_active":"2023-10-08T02:11:42.098447+0000","last_became_peered":"2023-10-08T02:11:42.098447+0000","last_unstale":"2023-10-08T02:11:42.103531+0000","last_undegraded":"2023-10-08T02:11:42.103531+0000","last_fullsized":"2023-10-08T02:11:42.103531+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114887+0000","last_change":"2023-10-08T02:11:42.112291+0000","last_active":"2023-10-08T02:11:42.114887+0000","last_peered":"2023-10-08T02:11:42.114887+0000","last_clean":"2023-10-08T02:11:42.114887+0000","last_became_active":"2023-10-08T02:11:42.111998+0000","last_became_peered":"2023-10-08T02:11:42.111998+0000","last_unstale":"2023-10-08T02:11:42.114887+0000","last_undegraded":"2023-10-08T02:11:42.114887+0000","last_fullsized":"2023-10-08T02:11:42.114887+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:19:24.869313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110796+0000","last_change":"2023-10-08T02:11:42.097979+0000","last_active":"2023-10-08T02:11:42.110796+0000","last_peered":"2023-10-08T02:11:42.110796+0000","last_clean":"2023-10-08T02:11:42.110796+0000","last_became_active":"2023-10-08T02:11:42.097748+0000","last_became_peered":"2023-10-08T02:11:42.097748+0000","last_unstale":"2023-10-08T02:11:42.110796+0000","last_undegraded":"2023-10-08T02:11:42.110796+0000","last_fullsized":"2023-10-08T02:11:42.110796+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:01.940169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110968+0000","last_change":"2023-10-08T02:11:42.104178+0000","last_active":"2023-10-08T02:11:42.110968+0000","last_peered":"2023-10-08T02:11:42.110968+0000","last_clean":"2023-10-08T02:11:42.110968+0000","last_became_active":"2023-10-08T02:11:42.103901+0000","last_became_peered":"2023-10-08T02:11:42.103901+0000","last_unstale":"2023-10-08T02:11:42.110968+0000","last_undegraded":"2023-10-08T02:11:42.110968+0000","last_fullsized":"2023-10-08T02:11:42.110968+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:55:56.724622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108817+0000","last_change":"2023-10-08T02:11:42.100208+0000","last_active":"2023-10-08T02:11:42.108817+0000","last_peered":"2023-10-08T02:11:42.108817+0000","last_clean":"2023-10-08T02:11:42.108817+0000","last_became_active":"2023-10-08T02:11:42.099895+0000","last_became_peered":"2023-10-08T02:11:42.099895+0000","last_unstale":"2023-10-08T02:11:42.108817+0000","last_undegraded":"2023-10-08T02:11:42.108817+0000","last_fullsized":"2023-10-08T02:11:42.108817+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.111468+0000","last_change":"2023-10-08T02:11:42.109029+0000","last_active":"2023-10-08T02:11:42.111468+0000","last_peered":"2023-10-08T02:11:42.111468+0000","last_clean":"2023-10-08T02:11:42.111468+0000","last_became_active":"2023-10-08T02:11:42.108769+0000","last_became_peered":"2023-10-08T02:11:42.108769+0000","last_unstale":"2023-10-08T02:11:42.111468+0000","last_undegraded":"2023-10-08T02:11:42.111468+0000","last_fullsized":"2023-10-08T02:11:42.111468+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:34.769593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114643+0000","last_change":"2023-10-08T02:11:42.113624+0000","last_active":"2023-10-08T02:11:42.114643+0000","last_peered":"2023-10-08T02:11:42.114643+0000","last_clean":"2023-10-08T02:11:42.114643+0000","last_became_active":"2023-10-08T02:11:42.113152+0000","last_became_peered":"2023-10-08T02:11:42.113152+0000","last_unstale":"2023-10-08T02:11:42.114643+0000","last_undegraded":"2023-10-08T02:11:42.114643+0000","last_fullsized":"2023-10-08T02:11:42.114643+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:11.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.103419+0000","last_change":"2023-10-08T02:11:42.100894+0000","last_active":"2023-10-08T02:11:42.103419+0000","last_peered":"2023-10-08T02:11:42.103419+0000","last_clean":"2023-10-08T02:11:42.103419+0000","last_became_active":"2023-10-08T02:11:42.100588+0000","last_became_peered":"2023-10-08T02:11:42.100588+0000","last_unstale":"2023-10-08T02:11:42.103419+0000","last_undegraded":"2023-10-08T02:11:42.103419+0000","last_fullsized":"2023-10-08T02:11:42.103419+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:51.531656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110205+0000","last_change":"2023-10-08T02:11:42.104602+0000","last_active":"2023-10-08T02:11:42.110205+0000","last_peered":"2023-10-08T02:11:42.110205+0000","last_clean":"2023-10-08T02:11:42.110205+0000","last_became_active":"2023-10-08T02:11:42.104357+0000","last_became_peered":"2023-10-08T02:11:42.104357+0000","last_unstale":"2023-10-08T02:11:42.110205+0000","last_undegraded":"2023-10-08T02:11:42.110205+0000","last_fullsized":"2023-10-08T02:11:42.110205+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:11.724000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113481+0000","last_change":"2023-10-08T02:11:42.111978+0000","last_active":"2023-10-08T02:11:42.113481+0000","last_peered":"2023-10-08T02:11:42.113481+0000","last_clean":"2023-10-08T02:11:42.113481+0000","last_became_active":"2023-10-08T02:11:42.110828+0000","last_became_peered":"2023-10-08T02:11:42.110828+0000","last_unstale":"2023-10-08T02:11:42.113481+0000","last_undegraded":"2023-10-08T02:11:42.113481+0000","last_fullsized":"2023-10-08T02:11:42.113481+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:11.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.109820+0000","last_change":"2023-10-08T02:11:42.106863+0000","last_active":"2023-10-08T02:11:42.109820+0000","last_peered":"2023-10-08T02:11:42.109820+0000","last_clean":"2023-10-08T02:11:42.109820+0000","last_became_active":"2023-10-08T02:11:42.106606+0000","last_became_peered":"2023-10-08T02:11:42.106606+0000","last_unstale":"2023-10-08T02:11:42.109820+0000","last_undegraded":"2023-10-08T02:11:42.109820+0000","last_fullsized":"2023-10-08T02:11:42.109820+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:58.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107993+0000","last_change":"2023-10-08T02:11:42.105477+0000","last_active":"2023-10-08T02:11:42.107993+0000","last_peered":"2023-10-08T02:11:42.107993+0000","last_clean":"2023-10-08T02:11:42.107993+0000","last_became_active":"2023-10-08T02:11:42.105203+0000","last_became_peered":"2023-10-08T02:11:42.105203+0000","last_unstale":"2023-10-08T02:11:42.107993+0000","last_undegraded":"2023-10-08T02:11:42.107993+0000","last_fullsized":"2023-10-08T02:11:42.107993+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:34.769593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100272+0000","last_change":"2023-10-08T02:11:42.096789+0000","last_active":"2023-10-08T02:11:42.100272+0000","last_peered":"2023-10-08T02:11:42.100272+0000","last_clean":"2023-10-08T02:11:42.100272+0000","last_became_active":"2023-10-08T02:11:42.096402+0000","last_became_peered":"2023-10-08T02:11:42.096402+0000","last_unstale":"2023-10-08T02:11:42.100272+0000","last_undegraded":"2023-10-08T02:11:42.100272+0000","last_fullsized":"2023-10-08T02:11:42.100272+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:10.087278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113401+0000","last_change":"2023-10-08T02:11:42.108742+0000","last_active":"2023-10-08T02:11:42.113401+0000","last_peered":"2023-10-08T02:11:42.113401+0000","last_clean":"2023-10-08T02:11:42.113401+0000","last_became_active":"2023-10-08T02:11:42.108463+0000","last_became_peered":"2023-10-08T02:11:42.108463+0000","last_unstale":"2023-10-08T02:11:42.113401+0000","last_undegraded":"2023-10-08T02:11:42.113401+0000","last_fullsized":"2023-10-08T02:11:42.113401+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:42:34.505384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112365+0000","last_change":"2023-10-08T02:11:42.109976+0000","last_active":"2023-10-08T02:11:42.112365+0000","last_peered":"2023-10-08T02:11:42.112365+0000","last_clean":"2023-10-08T02:11:42.112365+0000","last_became_active":"2023-10-08T02:11:42.109735+0000","last_became_peered":"2023-10-08T02:11:42.109735+0000","last_unstale":"2023-10-08T02:11:42.112365+0000","last_undegraded":"2023-10-08T02:11:42.112365+0000","last_fullsized":"2023-10-08T02:11:42.112365+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:18.861043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113932+0000","last_change":"2023-10-08T02:11:42.112275+0000","last_active":"2023-10-08T02:11:42.113932+0000","last_peered":"2023-10-08T02:11:42.113932+0000","last_clean":"2023-10-08T02:11:42.113932+0000","last_became_active":"2023-10-08T02:11:42.112031+0000","last_became_peered":"2023-10-08T02:11:42.112031+0000","last_unstale":"2023-10-08T02:11:42.113932+0000","last_undegraded":"2023-10-08T02:11:42.113932+0000","last_fullsized":"2023-10-08T02:11:42.113932+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:06.332369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107231+0000","last_change":"2023-10-08T02:11:42.103237+0000","last_active":"2023-10-08T02:11:42.107231+0000","last_peered":"2023-10-08T02:11:42.107231+0000","last_clean":"2023-10-08T02:11:42.107231+0000","last_became_active":"2023-10-08T02:11:42.102996+0000","last_became_peered":"2023-10-08T02:11:42.102996+0000","last_unstale":"2023-10-08T02:11:42.107231+0000","last_undegraded":"2023-10-08T02:11:42.107231+0000","last_fullsized":"2023-10-08T02:11:42.107231+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:25.186901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113380+0000","last_change":"2023-10-08T02:11:42.111018+0000","last_active":"2023-10-08T02:11:42.113380+0000","last_peered":"2023-10-08T02:11:42.113380+0000","last_clean":"2023-10-08T02:11:42.113380+0000","last_became_active":"2023-10-08T02:11:42.110766+0000","last_became_peered":"2023-10-08T02:11:42.110766+0000","last_unstale":"2023-10-08T02:11:42.113380+0000","last_undegraded":"2023-10-08T02:11:42.113380+0000","last_fullsized":"2023-10-08T02:11:42.113380+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:25.186901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107088+0000","last_change":"2023-10-08T02:11:42.106099+0000","last_active":"2023-10-08T02:11:42.107088+0000","last_peered":"2023-10-08T02:11:42.107088+0000","last_clean":"2023-10-08T02:11:42.107088+0000","last_became_active":"2023-10-08T02:11:42.105837+0000","last_became_peered":"2023-10-08T02:11:42.105837+0000","last_unstale":"2023-10-08T02:11:42.107088+0000","last_undegraded":"2023-10-08T02:11:42.107088+0000","last_fullsized":"2023-10-08T02:11:42.107088+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:20:22.999277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112022+0000","last_change":"2023-10-08T02:11:42.105982+0000","last_active":"2023-10-08T02:11:42.112022+0000","last_peered":"2023-10-08T02:11:42.112022+0000","last_clean":"2023-10-08T02:11:42.112022+0000","last_became_active":"2023-10-08T02:11:42.105508+0000","last_became_peered":"2023-10-08T02:11:42.105508+0000","last_unstale":"2023-10-08T02:11:42.112022+0000","last_undegraded":"2023-10-08T02:11:42.112022+0000","last_fullsized":"2023-10-08T02:11:42.112022+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:56.323343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108198+0000","last_change":"2023-10-08T02:11:42.104513+0000","last_active":"2023-10-08T02:11:42.108198+0000","last_peered":"2023-10-08T02:11:42.108198+0000","last_clean":"2023-10-08T02:11:42.108198+0000","last_became_active":"2023-10-08T02:11:42.104256+0000","last_became_peered":"2023-10-08T02:11:42.104256+0000","last_unstale":"2023-10-08T02:11:42.108198+0000","last_undegraded":"2023-10-08T02:11:42.108198+0000","last_fullsized":"2023-10-08T02:11:42.108198+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:19.029967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112851+0000","last_change":"2023-10-08T02:11:42.110658+0000","last_active":"2023-10-08T02:11:42.112851+0000","last_peered":"2023-10-08T02:11:42.112851+0000","last_clean":"2023-10-08T02:11:42.112851+0000","last_became_active":"2023-10-08T02:11:42.110397+0000","last_became_peered":"2023-10-08T02:11:42.110397+0000","last_unstale":"2023-10-08T02:11:42.112851+0000","last_undegraded":"2023-10-08T02:11:42.112851+0000","last_fullsized":"2023-10-08T02:11:42.112851+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:04.169449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100237+0000","last_change":"2023-10-08T02:11:42.097346+0000","last_active":"2023-10-08T02:11:42.100237+0000","last_peered":"2023-10-08T02:11:42.100237+0000","last_clean":"2023-10-08T02:11:42.100237+0000","last_became_active":"2023-10-08T02:11:42.097114+0000","last_became_peered":"2023-10-08T02:11:42.097114+0000","last_unstale":"2023-10-08T02:11:42.100237+0000","last_undegraded":"2023-10-08T02:11:42.100237+0000","last_fullsized":"2023-10-08T02:11:42.100237+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:17.058773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.111811+0000","last_change":"2023-10-08T02:11:42.110297+0000","last_active":"2023-10-08T02:11:42.111811+0000","last_peered":"2023-10-08T02:11:42.111811+0000","last_clean":"2023-10-08T02:11:42.111811+0000","last_became_active":"2023-10-08T02:11:42.110051+0000","last_became_peered":"2023-10-08T02:11:42.110051+0000","last_unstale":"2023-10-08T02:11:42.111811+0000","last_undegraded":"2023-10-08T02:11:42.111811+0000","last_fullsized":"2023-10-08T02:11:42.111811+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:02.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.104854+0000","last_change":"2023-10-08T02:11:42.103016+0000","last_active":"2023-10-08T02:11:42.104854+0000","last_peered":"2023-10-08T02:11:42.104854+0000","last_clean":"2023-10-08T02:11:42.104854+0000","last_became_active":"2023-10-08T02:11:42.102673+0000","last_became_peered":"2023-10-08T02:11:42.102673+0000","last_unstale":"2023-10-08T02:11:42.104854+0000","last_undegraded":"2023-10-08T02:11:42.104854+0000","last_fullsized":"2023-10-08T02:11:42.104854+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:39:19.177440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106088+0000","last_change":"2023-10-08T02:11:42.105002+0000","last_active":"2023-10-08T02:11:42.106088+0000","last_peered":"2023-10-08T02:11:42.106088+0000","last_clean":"2023-10-08T02:11:42.106088+0000","last_became_active":"2023-10-08T02:11:42.104707+0000","last_became_peered":"2023-10-08T02:11:42.104707+0000","last_unstale":"2023-10-08T02:11:42.106088+0000","last_undegraded":"2023-10-08T02:11:42.106088+0000","last_fullsized":"2023-10-08T02:11:42.106088+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106069+0000","last_change":"2023-10-08T02:11:42.104624+0000","last_active":"2023-10-08T02:11:42.106069+0000","last_peered":"2023-10-08T02:11:42.106069+0000","last_clean":"2023-10-08T02:11:42.106069+0000","last_became_active":"2023-10-08T02:11:42.104319+0000","last_became_peered":"2023-10-08T02:11:42.104319+0000","last_unstale":"2023-10-08T02:11:42.106069+0000","last_undegraded":"2023-10-08T02:11:42.106069+0000","last_fullsized":"2023-10-08T02:11:42.106069+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:56.323343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107233+0000","last_change":"2023-10-08T02:11:42.100377+0000","last_active":"2023-10-08T02:11:42.107233+0000","last_peered":"2023-10-08T02:11:42.107233+0000","last_clean":"2023-10-08T02:11:42.107233+0000","last_became_active":"2023-10-08T02:11:42.100065+0000","last_became_peered":"2023-10-08T02:11:42.100065+0000","last_unstale":"2023-10-08T02:11:42.107233+0000","last_undegraded":"2023-10-08T02:11:42.107233+0000","last_fullsized":"2023-10-08T02:11:42.107233+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:02.949267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110762+0000","last_change":"2023-10-08T02:11:42.103389+0000","last_active":"2023-10-08T02:11:42.110762+0000","last_peered":"2023-10-08T02:11:42.110762+0000","last_clean":"2023-10-08T02:11:42.110762+0000","last_became_active":"2023-10-08T02:11:42.103115+0000","last_became_peered":"2023-10-08T02:11:42.103115+0000","last_unstale":"2023-10-08T02:11:42.110762+0000","last_undegraded":"2023-10-08T02:11:42.110762+0000","last_fullsized":"2023-10-08T02:11:42.110762+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:59.671124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107216+0000","last_change":"2023-10-08T02:11:42.104463+0000","last_active":"2023-10-08T02:11:42.107216+0000","last_peered":"2023-10-08T02:11:42.107216+0000","last_clean":"2023-10-08T02:11:42.107216+0000","last_became_active":"2023-10-08T02:11:42.104103+0000","last_became_peered":"2023-10-08T02:11:42.104103+0000","last_unstale":"2023-10-08T02:11:42.107216+0000","last_undegraded":"2023-10-08T02:11:42.107216+0000","last_fullsized":"2023-10-08T02:11:42.107216+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:26.503280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"156'4","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.123290+0000","last_change":"2023-10-08T02:11:41.091637+0000","last_active":"2023-10-08T02:11:42.123290+0000","last_peered":"2023-10-08T02:11:42.123290+0000","last_clean":"2023-10-08T02:11:42.123290+0000","last_became_active":"2023-10-08T02:11:41.091309+0000","last_became_peered":"2023-10-08T02:11:41.091309+0000","last_unstale":"2023-10-08T02:11:42.123290+0000","last_undegraded":"2023-10-08T02:11:42.123290+0000","last_fullsized":"2023-10-08T02:11:42.123290+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:28.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.099197+0000","last_change":"2023-10-08T02:11:42.097072+0000","last_active":"2023-10-08T02:11:42.099197+0000","last_peered":"2023-10-08T02:11:42.099197+0000","last_clean":"2023-10-08T02:11:42.099197+0000","last_became_active":"2023-10-08T02:11:42.095258+0000","last_became_peered":"2023-10-08T02:11:42.095258+0000","last_unstale":"2023-10-08T02:11:42.099197+0000","last_undegraded":"2023-10-08T02:11:42.099197+0000","last_fullsized":"2023-10-08T02:11:42.099197+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:53.166034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122732+0000","last_change":"2023-10-08T02:11:41.086355+0000","last_active":"2023-10-08T02:11:42.122732+0000","last_peered":"2023-10-08T02:11:42.122732+0000","last_clean":"2023-10-08T02:11:42.122732+0000","last_became_active":"2023-10-08T02:11:41.084532+0000","last_became_peered":"2023-10-08T02:11:41.084532+0000","last_unstale":"2023-10-08T02:11:42.122732+0000","last_undegraded":"2023-10-08T02:11:42.122732+0000","last_fullsized":"2023-10-08T02:11:42.122732+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:17.855408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114801+0000","last_change":"2023-10-08T02:11:42.112854+0000","last_active":"2023-10-08T02:11:42.114801+0000","last_peered":"2023-10-08T02:11:42.114801+0000","last_clean":"2023-10-08T02:11:42.114801+0000","last_became_active":"2023-10-08T02:11:42.112549+0000","last_became_peered":"2023-10-08T02:11:42.112549+0000","last_unstale":"2023-10-08T02:11:42.114801+0000","last_undegraded":"2023-10-08T02:11:42.114801+0000","last_fullsized":"2023-10-08T02:11:42.114801+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:29.892608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.121532+0000","last_change":"2023-10-08T02:11:41.095273+0000","last_active":"2023-10-08T02:11:42.121532+0000","last_peered":"2023-10-08T02:11:42.121532+0000","last_clean":"2023-10-08T02:11:42.121532+0000","last_became_active":"2023-10-08T02:11:41.095001+0000","last_became_peered":"2023-10-08T02:11:41.095001+0000","last_unstale":"2023-10-08T02:11:42.121532+0000","last_undegraded":"2023-10-08T02:11:42.121532+0000","last_fullsized":"2023-10-08T02:11:42.121532+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:19.066208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108482+0000","last_change":"2023-10-08T02:11:42.101145+0000","last_active":"2023-10-08T02:11:42.108482+0000","last_peered":"2023-10-08T02:11:42.108482+0000","last_clean":"2023-10-08T02:11:42.108482+0000","last_became_active":"2023-10-08T02:11:42.100850+0000","last_became_peered":"2023-10-08T02:11:42.100850+0000","last_unstale":"2023-10-08T02:11:42.108482+0000","last_undegraded":"2023-10-08T02:11:42.108482+0000","last_fullsized":"2023-10-08T02:11:42.108482+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:59.627184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.120117+0000","last_change":"2023-10-08T02:11:41.094608+0000","last_active":"2023-10-08T02:11:42.120117+0000","last_peered":"2023-10-08T02:11:42.120117+0000","last_clean":"2023-10-08T02:11:42.120117+0000","last_became_active":"2023-10-08T02:11:41.094299+0000","last_became_peered":"2023-10-08T02:11:41.094299+0000","last_unstale":"2023-10-08T02:11:42.120117+0000","last_undegraded":"2023-10-08T02:11:42.120117+0000","last_fullsized":"2023-10-08T02:11:42.120117+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:44.763746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113544+0000","last_change":"2023-10-08T02:11:42.111527+0000","last_active":"2023-10-08T02:11:42.113544+0000","last_peered":"2023-10-08T02:11:42.113544+0000","last_clean":"2023-10-08T02:11:42.113544+0000","last_became_active":"2023-10-08T02:11:42.111206+0000","last_became_peered":"2023-10-08T02:11:42.111206+0000","last_unstale":"2023-10-08T02:11:42.113544+0000","last_undegraded":"2023-10-08T02:11:42.113544+0000","last_fullsized":"2023-10-08T02:11:42.113544+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:14.761546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.119642+0000","last_change":"2023-10-08T02:11:41.094567+0000","last_active":"2023-10-08T02:11:42.119642+0000","last_peered":"2023-10-08T02:11:42.119642+0000","last_clean":"2023-10-08T02:11:42.119642+0000","last_became_active":"2023-10-08T02:11:41.094277+0000","last_became_peered":"2023-10-08T02:11:41.094277+0000","last_unstale":"2023-10-08T02:11:42.119642+0000","last_undegraded":"2023-10-08T02:11:42.119642+0000","last_fullsized":"2023-10-08T02:11:42.119642+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:36.101961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114403+0000","last_change":"2023-10-08T02:11:42.109492+0000","last_active":"2023-10-08T02:11:42.114403+0000","last_peered":"2023-10-08T02:11:42.114403+0000","last_clean":"2023-10-08T02:11:42.114403+0000","last_became_active":"2023-10-08T02:11:42.109130+0000","last_became_peered":"2023-10-08T02:11:42.109130+0000","last_unstale":"2023-10-08T02:11:42.114403+0000","last_undegraded":"2023-10-08T02:11:42.114403+0000","last_fullsized":"2023-10-08T02:11:42.114403+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:50.680455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122553+0000","last_change":"2023-10-08T02:11:41.094113+0000","last_active":"2023-10-08T02:11:42.122553+0000","last_peered":"2023-10-08T02:11:42.122553+0000","last_clean":"2023-10-08T02:11:42.122553+0000","last_became_active":"2023-10-08T02:11:41.093857+0000","last_became_peered":"2023-10-08T02:11:41.093857+0000","last_unstale":"2023-10-08T02:11:42.122553+0000","last_undegraded":"2023-10-08T02:11:42.122553+0000","last_fullsized":"2023-10-08T02:11:42.122553+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:55.317708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108797+0000","last_change":"2023-10-08T02:11:42.099629+0000","last_active":"2023-10-08T02:11:42.108797+0000","last_peered":"2023-10-08T02:11:42.108797+0000","last_clean":"2023-10-08T02:11:42.108797+0000","last_became_active":"2023-10-08T02:11:42.099303+0000","last_became_peered":"2023-10-08T02:11:42.099303+0000","last_unstale":"2023-10-08T02:11:42.108797+0000","last_undegraded":"2023-10-08T02:11:42.108797+0000","last_fullsized":"2023-10-08T02:11:42.108797+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.096277+0000","last_change":"2023-10-08T02:11:41.086476+0000","last_active":"2023-10-08T02:11:42.096277+0000","last_peered":"2023-10-08T02:11:42.096277+0000","last_clean":"2023-10-08T02:11:42.096277+0000","last_became_active":"2023-10-08T02:11:41.085954+0000","last_became_peered":"2023-10-08T02:11:41.085954+0000","last_unstale":"2023-10-08T02:11:42.096277+0000","last_undegraded":"2023-10-08T02:11:42.096277+0000","last_fullsized":"2023-10-08T02:11:42.096277+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:23.678635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110289+0000","last_change":"2023-10-08T02:11:42.107521+0000","last_active":"2023-10-08T02:11:42.110289+0000","last_peered":"2023-10-08T02:11:42.110289+0000","last_clean":"2023-10-08T02:11:42.110289+0000","last_became_active":"2023-10-08T02:11:42.107255+0000","last_became_peered":"2023-10-08T02:11:42.107255+0000","last_unstale":"2023-10-08T02:11:42.110289+0000","last_undegraded":"2023-10-08T02:11:42.110289+0000","last_fullsized":"2023-10-08T02:11:42.110289+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.096045+0000","last_change":"2023-10-08T02:11:41.091321+0000","last_active":"2023-10-08T02:11:42.096045+0000","last_peered":"2023-10-08T02:11:42.096045+0000","last_clean":"2023-10-08T02:11:42.096045+0000","last_became_active":"2023-10-08T02:11:41.091092+0000","last_became_peered":"2023-10-08T02:11:41.091092+0000","last_unstale":"2023-10-08T02:11:42.096045+0000","last_undegraded":"2023-10-08T02:11:42.096045+0000","last_fullsized":"2023-10-08T02:11:42.096045+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100491+0000","last_change":"2023-10-08T02:11:42.095233+0000","last_active":"2023-10-08T02:11:42.100491+0000","last_peered":"2023-10-08T02:11:42.100491+0000","last_clean":"2023-10-08T02:11:42.100491+0000","last_became_active":"2023-10-08T02:11:42.094991+0000","last_became_peered":"2023-10-08T02:11:42.094991+0000","last_unstale":"2023-10-08T02:11:42.100491+0000","last_undegraded":"2023-10-08T02:11:42.100491+0000","last_fullsized":"2023-10-08T02:11:42.100491+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:58.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.095962+0000","last_change":"2023-10-08T02:11:41.090834+0000","last_active":"2023-10-08T02:11:42.095962+0000","last_peered":"2023-10-08T02:11:42.095962+0000","last_clean":"2023-10-08T02:11:42.095962+0000","last_became_active":"2023-10-08T02:11:41.090569+0000","last_became_peered":"2023-10-08T02:11:41.090569+0000","last_unstale":"2023-10-08T02:11:42.095962+0000","last_undegraded":"2023-10-08T02:11:42.095962+0000","last_fullsized":"2023-10-08T02:11:42.095962+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:24.181266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112899+0000","last_change":"2023-10-08T02:11:42.107968+0000","last_active":"2023-10-08T02:11:42.112899+0000","last_peered":"2023-10-08T02:11:42.112899+0000","last_clean":"2023-10-08T02:11:42.112899+0000","last_became_active":"2023-10-08T02:11:42.107634+0000","last_became_peered":"2023-10-08T02:11:42.107634+0000","last_unstale":"2023-10-08T02:11:42.112899+0000","last_undegraded":"2023-10-08T02:11:42.112899+0000","last_fullsized":"2023-10-08T02:11:42.112899+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:59.627184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.092141+0000","last_change":"2023-10-08T02:11:41.092290+0000","last_active":"2023-10-08T02:11:42.092141+0000","last_peered":"2023-10-08T02:11:42.092141+0000","last_clean":"2023-10-08T02:11:42.092141+0000","last_became_active":"2023-10-08T02:11:41.091969+0000","last_became_peered":"2023-10-08T02:11:41.091969+0000","last_unstale":"2023-10-08T02:11:42.092141+0000","last_undegraded":"2023-10-08T02:11:42.092141+0000","last_fullsized":"2023-10-08T02:11:42.092141+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:53.014859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107241+0000","last_change":"2023-10-08T02:11:42.100490+0000","last_active":"2023-10-08T02:11:42.107241+0000","last_peered":"2023-10-08T02:11:42.107241+0000","last_clean":"2023-10-08T02:11:42.107241+0000","last_became_active":"2023-10-08T02:11:42.100156+0000","last_became_peered":"2023-10-08T02:11:42.100156+0000","last_unstale":"2023-10-08T02:11:42.107241+0000","last_undegraded":"2023-10-08T02:11:42.107241+0000","last_fullsized":"2023-10-08T02:11:42.107241+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:39:19.177440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"156'3","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122321+0000","last_change":"2023-10-08T02:11:41.094352+0000","last_active":"2023-10-08T02:11:42.122321+0000","last_peered":"2023-10-08T02:11:42.122321+0000","last_clean":"2023-10-08T02:11:42.122321+0000","last_became_active":"2023-10-08T02:11:41.094048+0000","last_became_peered":"2023-10-08T02:11:41.094048+0000","last_unstale":"2023-10-08T02:11:42.122321+0000","last_undegraded":"2023-10-08T02:11:42.122321+0000","last_fullsized":"2023-10-08T02:11:42.122321+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:03.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108463+0000","last_change":"2023-10-08T02:11:42.100509+0000","last_active":"2023-10-08T02:11:42.108463+0000","last_peered":"2023-10-08T02:11:42.108463+0000","last_clean":"2023-10-08T02:11:42.108463+0000","last_became_active":"2023-10-08T02:11:42.100202+0000","last_became_peered":"2023-10-08T02:11:42.100202+0000","last_unstale":"2023-10-08T02:11:42.108463+0000","last_undegraded":"2023-10-08T02:11:42.108463+0000","last_fullsized":"2023-10-08T02:11:42.108463+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:01.047419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"156'6","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125472+0000","last_change":"2023-10-08T02:11:41.080631+0000","last_active":"2023-10-08T02:11:42.125472+0000","last_peered":"2023-10-08T02:11:42.125472+0000","last_clean":"2023-10-08T02:11:42.125472+0000","last_became_active":"2023-10-08T02:11:41.080365+0000","last_became_peered":"2023-10-08T02:11:41.080365+0000","last_unstale":"2023-10-08T02:11:42.125472+0000","last_undegraded":"2023-10-08T02:11:42.125472+0000","last_fullsized":"2023-10-08T02:11:42.125472+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100695+0000","last_change":"2023-10-08T02:11:42.096482+0000","last_active":"2023-10-08T02:11:42.100695+0000","last_peered":"2023-10-08T02:11:42.100695+0000","last_clean":"2023-10-08T02:11:42.100695+0000","last_became_active":"2023-10-08T02:11:42.094165+0000","last_became_peered":"2023-10-08T02:11:42.094165+0000","last_unstale":"2023-10-08T02:11:42.100695+0000","last_undegraded":"2023-10-08T02:11:42.100695+0000","last_fullsized":"2023-10-08T02:11:42.100695+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:23.801545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086789+0000","last_change":"2023-10-08T02:11:41.084280+0000","last_active":"2023-10-08T02:11:42.086789+0000","last_peered":"2023-10-08T02:11:42.086789+0000","last_clean":"2023-10-08T02:11:42.086789+0000","last_became_active":"2023-10-08T02:11:41.083971+0000","last_became_peered":"2023-10-08T02:11:41.083971+0000","last_unstale":"2023-10-08T02:11:42.086789+0000","last_undegraded":"2023-10-08T02:11:42.086789+0000","last_fullsized":"2023-10-08T02:11:42.086789+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112430+0000","last_change":"2023-10-08T02:11:42.110754+0000","last_active":"2023-10-08T02:11:42.112430+0000","last_peered":"2023-10-08T02:11:42.112430+0000","last_clean":"2023-10-08T02:11:42.112430+0000","last_became_active":"2023-10-08T02:11:42.110475+0000","last_became_peered":"2023-10-08T02:11:42.110475+0000","last_unstale":"2023-10-08T02:11:42.112430+0000","last_undegraded":"2023-10-08T02:11:42.112430+0000","last_fullsized":"2023-10-08T02:11:42.112430+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:17.058773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.098318+0000","last_change":"2023-10-08T02:11:41.084543+0000","last_active":"2023-10-08T02:11:42.098318+0000","last_peered":"2023-10-08T02:11:42.098318+0000","last_clean":"2023-10-08T02:11:42.098318+0000","last_became_active":"2023-10-08T02:11:41.084263+0000","last_became_peered":"2023-10-08T02:11:41.084263+0000","last_unstale":"2023-10-08T02:11:42.098318+0000","last_undegraded":"2023-10-08T02:11:42.098318+0000","last_fullsized":"2023-10-08T02:11:42.098318+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112003+0000","last_change":"2023-10-08T02:11:42.105772+0000","last_active":"2023-10-08T02:11:42.112003+0000","last_peered":"2023-10-08T02:11:42.112003+0000","last_clean":"2023-10-08T02:11:42.112003+0000","last_became_active":"2023-10-08T02:11:42.105165+0000","last_became_peered":"2023-10-08T02:11:42.105165+0000","last_unstale":"2023-10-08T02:11:42.112003+0000","last_undegraded":"2023-10-08T02:11:42.112003+0000","last_fullsized":"2023-10-08T02:11:42.112003+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:12.133057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"156'8","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.123411+0000","last_change":"2023-10-08T02:11:41.089825+0000","last_active":"2023-10-08T02:11:42.123411+0000","last_peered":"2023-10-08T02:11:42.123411+0000","last_clean":"2023-10-08T02:11:42.123411+0000","last_became_active":"2023-10-08T02:11:41.089299+0000","last_became_peered":"2023-10-08T02:11:41.089299+0000","last_unstale":"2023-10-08T02:11:42.123411+0000","last_undegraded":"2023-10-08T02:11:42.123411+0000","last_fullsized":"2023-10-08T02:11:42.123411+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:50.165805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.105986+0000","last_change":"2023-10-08T02:11:42.098176+0000","last_active":"2023-10-08T02:11:42.105986+0000","last_peered":"2023-10-08T02:11:42.105986+0000","last_clean":"2023-10-08T02:11:42.105986+0000","last_became_active":"2023-10-08T02:11:42.097581+0000","last_became_peered":"2023-10-08T02:11:42.097581+0000","last_unstale":"2023-10-08T02:11:42.105986+0000","last_undegraded":"2023-10-08T02:11:42.105986+0000","last_fullsized":"2023-10-08T02:11:42.105986+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122176+0000","last_change":"2023-10-08T02:11:41.081179+0000","last_active":"2023-10-08T02:11:42.122176+0000","last_peered":"2023-10-08T02:11:42.122176+0000","last_clean":"2023-10-08T02:11:42.122176+0000","last_became_active":"2023-10-08T02:11:41.080899+0000","last_became_peered":"2023-10-08T02:11:41.080899+0000","last_unstale":"2023-10-08T02:11:42.122176+0000","last_undegraded":"2023-10-08T02:11:42.122176+0000","last_fullsized":"2023-10-08T02:11:42.122176+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:09.081644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108072+0000","last_change":"2023-10-08T02:11:42.105520+0000","last_active":"2023-10-08T02:11:42.108072+0000","last_peered":"2023-10-08T02:11:42.108072+0000","last_clean":"2023-10-08T02:11:42.108072+0000","last_became_active":"2023-10-08T02:11:42.105200+0000","last_became_peered":"2023-10-08T02:11:42.105200+0000","last_unstale":"2023-10-08T02:11:42.108072+0000","last_undegraded":"2023-10-08T02:11:42.108072+0000","last_fullsized":"2023-10-08T02:11:42.108072+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:21.328506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125324+0000","last_change":"2023-10-08T02:11:41.089638+0000","last_active":"2023-10-08T02:11:42.125324+0000","last_peered":"2023-10-08T02:11:42.125324+0000","last_clean":"2023-10-08T02:11:42.125324+0000","last_became_active":"2023-10-08T02:11:41.089339+0000","last_became_peered":"2023-10-08T02:11:41.089339+0000","last_unstale":"2023-10-08T02:11:42.125324+0000","last_undegraded":"2023-10-08T02:11:42.125324+0000","last_fullsized":"2023-10-08T02:11:42.125324+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:26.191071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107213+0000","last_change":"2023-10-08T02:11:42.100803+0000","last_active":"2023-10-08T02:11:42.107213+0000","last_peered":"2023-10-08T02:11:42.107213+0000","last_clean":"2023-10-08T02:11:42.107213+0000","last_became_active":"2023-10-08T02:11:42.100558+0000","last_became_peered":"2023-10-08T02:11:42.100558+0000","last_unstale":"2023-10-08T02:11:42.107213+0000","last_undegraded":"2023-10-08T02:11:42.107213+0000","last_fullsized":"2023-10-08T02:11:42.107213+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:04.169449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.124741+0000","last_change":"2023-10-08T02:11:41.096436+0000","last_active":"2023-10-08T02:11:42.124741+0000","last_peered":"2023-10-08T02:11:42.124741+0000","last_clean":"2023-10-08T02:11:42.124741+0000","last_became_active":"2023-10-08T02:11:41.096182+0000","last_became_peered":"2023-10-08T02:11:41.096182+0000","last_unstale":"2023-10-08T02:11:42.124741+0000","last_undegraded":"2023-10-08T02:11:42.124741+0000","last_fullsized":"2023-10-08T02:11:42.124741+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:56.085728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.105522+0000","last_change":"2023-10-08T02:11:42.103049+0000","last_active":"2023-10-08T02:11:42.105522+0000","last_peered":"2023-10-08T02:11:42.105522+0000","last_clean":"2023-10-08T02:11:42.105522+0000","last_became_active":"2023-10-08T02:11:42.102617+0000","last_became_peered":"2023-10-08T02:11:42.102617+0000","last_unstale":"2023-10-08T02:11:42.105522+0000","last_undegraded":"2023-10-08T02:11:42.105522+0000","last_fullsized":"2023-10-08T02:11:42.105522+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:23.801545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.095197+0000","last_change":"2023-10-08T02:11:41.083978+0000","last_active":"2023-10-08T02:11:42.095197+0000","last_peered":"2023-10-08T02:11:42.095197+0000","last_clean":"2023-10-08T02:11:42.095197+0000","last_became_active":"2023-10-08T02:11:41.083711+0000","last_became_peered":"2023-10-08T02:11:41.083711+0000","last_unstale":"2023-10-08T02:11:42.095197+0000","last_undegraded":"2023-10-08T02:11:42.095197+0000","last_fullsized":"2023-10-08T02:11:42.095197+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:13.755911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113445+0000","last_change":"2023-10-08T02:11:42.111899+0000","last_active":"2023-10-08T02:11:42.113445+0000","last_peered":"2023-10-08T02:11:42.113445+0000","last_clean":"2023-10-08T02:11:42.113445+0000","last_became_active":"2023-10-08T02:11:42.111582+0000","last_became_peered":"2023-10-08T02:11:42.111582+0000","last_unstale":"2023-10-08T02:11:42.113445+0000","last_undegraded":"2023-10-08T02:11:42.113445+0000","last_fullsized":"2023-10-08T02:11:42.113445+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:11.724000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122478+0000","last_change":"2023-10-08T02:11:41.091273+0000","last_active":"2023-10-08T02:11:42.122478+0000","last_peered":"2023-10-08T02:11:42.122478+0000","last_clean":"2023-10-08T02:11:42.122478+0000","last_became_active":"2023-10-08T02:11:41.091018+0000","last_became_peered":"2023-10-08T02:11:41.091018+0000","last_unstale":"2023-10-08T02:11:42.122478+0000","last_undegraded":"2023-10-08T02:11:42.122478+0000","last_fullsized":"2023-10-08T02:11:42.122478+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:16.053138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110910+0000","last_change":"2023-10-08T02:11:42.103858+0000","last_active":"2023-10-08T02:11:42.110910+0000","last_peered":"2023-10-08T02:11:42.110910+0000","last_clean":"2023-10-08T02:11:42.110910+0000","last_became_active":"2023-10-08T02:11:42.103492+0000","last_became_peered":"2023-10-08T02:11:42.103492+0000","last_unstale":"2023-10-08T02:11:42.110910+0000","last_undegraded":"2023-10-08T02:11:42.110910+0000","last_fullsized":"2023-10-08T02:11:42.110910+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:29.230376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.091549+0000","last_change":"2023-10-08T02:11:41.091649+0000","last_active":"2023-10-08T02:11:42.091549+0000","last_peered":"2023-10-08T02:11:42.091549+0000","last_clean":"2023-10-08T02:11:42.091549+0000","last_became_active":"2023-10-08T02:11:41.091394+0000","last_became_peered":"2023-10-08T02:11:41.091394+0000","last_unstale":"2023-10-08T02:11:42.091549+0000","last_undegraded":"2023-10-08T02:11:42.091549+0000","last_fullsized":"2023-10-08T02:11:42.091549+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:11.127422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.105430+0000","last_change":"2023-10-08T02:11:42.101708+0000","last_active":"2023-10-08T02:11:42.105430+0000","last_peered":"2023-10-08T02:11:42.105430+0000","last_clean":"2023-10-08T02:11:42.105430+0000","last_became_active":"2023-10-08T02:11:42.101426+0000","last_became_peered":"2023-10-08T02:11:42.101426+0000","last_unstale":"2023-10-08T02:11:42.105430+0000","last_undegraded":"2023-10-08T02:11:42.105430+0000","last_fullsized":"2023-10-08T02:11:42.105430+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.090646+0000","last_change":"2023-10-08T02:11:41.088767+0000","last_active":"2023-10-08T02:11:42.090646+0000","last_peered":"2023-10-08T02:11:42.090646+0000","last_clean":"2023-10-08T02:11:42.090646+0000","last_became_active":"2023-10-08T02:11:41.088418+0000","last_became_peered":"2023-10-08T02:11:41.088418+0000","last_unstale":"2023-10-08T02:11:42.090646+0000","last_undegraded":"2023-10-08T02:11:42.090646+0000","last_fullsized":"2023-10-08T02:11:42.090646+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.160399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110003+0000","last_change":"2023-10-08T02:11:42.104907+0000","last_active":"2023-10-08T02:11:42.110003+0000","last_peered":"2023-10-08T02:11:42.110003+0000","last_clean":"2023-10-08T02:11:42.110003+0000","last_became_active":"2023-10-08T02:11:42.102578+0000","last_became_peered":"2023-10-08T02:11:42.102578+0000","last_unstale":"2023-10-08T02:11:42.110003+0000","last_undegraded":"2023-10-08T02:11:42.110003+0000","last_fullsized":"2023-10-08T02:11:42.110003+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:57.091363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086116+0000","last_change":"2023-10-08T02:11:41.089652+0000","last_active":"2023-10-08T02:11:42.086116+0000","last_peered":"2023-10-08T02:11:42.086116+0000","last_clean":"2023-10-08T02:11:42.086116+0000","last_became_active":"2023-10-08T02:11:41.089362+0000","last_became_peered":"2023-10-08T02:11:41.089362+0000","last_unstale":"2023-10-08T02:11:42.086116+0000","last_undegraded":"2023-10-08T02:11:42.086116+0000","last_fullsized":"2023-10-08T02:11:42.086116+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:02.686826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114760+0000","last_change":"2023-10-08T02:11:42.109782+0000","last_active":"2023-10-08T02:11:42.114760+0000","last_peered":"2023-10-08T02:11:42.114760+0000","last_clean":"2023-10-08T02:11:42.114760+0000","last_became_active":"2023-10-08T02:11:42.109478+0000","last_became_peered":"2023-10-08T02:11:42.109478+0000","last_unstale":"2023-10-08T02:11:42.114760+0000","last_undegraded":"2023-10-08T02:11:42.114760+0000","last_fullsized":"2023-10-08T02:11:42.114760+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:32:40.681432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125349+0000","last_change":"2023-10-08T02:11:41.087828+0000","last_active":"2023-10-08T02:11:42.125349+0000","last_peered":"2023-10-08T02:11:42.125349+0000","last_clean":"2023-10-08T02:11:42.125349+0000","last_became_active":"2023-10-08T02:11:41.087533+0000","last_became_peered":"2023-10-08T02:11:41.087533+0000","last_unstale":"2023-10-08T02:11:42.125349+0000","last_undegraded":"2023-10-08T02:11:42.125349+0000","last_fullsized":"2023-10-08T02:11:42.125349+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.101659+0000","last_change":"2023-10-08T02:11:42.098444+0000","last_active":"2023-10-08T02:11:42.101659+0000","last_peered":"2023-10-08T02:11:42.101659+0000","last_clean":"2023-10-08T02:11:42.101659+0000","last_became_active":"2023-10-08T02:11:42.098168+0000","last_became_peered":"2023-10-08T02:11:42.098168+0000","last_unstale":"2023-10-08T02:11:42.101659+0000","last_undegraded":"2023-10-08T02:11:42.101659+0000","last_fullsized":"2023-10-08T02:11:42.101659+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:12.133057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.088939+0000","last_change":"2023-10-08T02:11:41.083886+0000","last_active":"2023-10-08T02:11:42.088939+0000","last_peered":"2023-10-08T02:11:42.088939+0000","last_clean":"2023-10-08T02:11:42.088939+0000","last_became_active":"2023-10-08T02:11:41.083613+0000","last_became_peered":"2023-10-08T02:11:41.083613+0000","last_unstale":"2023-10-08T02:11:42.088939+0000","last_undegraded":"2023-10-08T02:11:42.088939+0000","last_fullsized":"2023-10-08T02:11:42.088939+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:33.763959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112978+0000","last_change":"2023-10-08T02:11:42.111324+0000","last_active":"2023-10-08T02:11:42.112978+0000","last_peered":"2023-10-08T02:11:42.112978+0000","last_clean":"2023-10-08T02:11:42.112978+0000","last_became_active":"2023-10-08T02:11:42.111081+0000","last_became_peered":"2023-10-08T02:11:42.111081+0000","last_unstale":"2023-10-08T02:11:42.112978+0000","last_undegraded":"2023-10-08T02:11:42.112978+0000","last_fullsized":"2023-10-08T02:11:42.112978+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:26.503280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.099015+0000","last_change":"2023-10-08T02:11:41.090739+0000","last_active":"2023-10-08T02:11:42.099015+0000","last_peered":"2023-10-08T02:11:42.099015+0000","last_clean":"2023-10-08T02:11:42.099015+0000","last_became_active":"2023-10-08T02:11:41.090496+0000","last_became_peered":"2023-10-08T02:11:41.090496+0000","last_unstale":"2023-10-08T02:11:42.099015+0000","last_undegraded":"2023-10-08T02:11:42.099015+0000","last_fullsized":"2023-10-08T02:11:42.099015+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:22.244415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114322+0000","last_change":"2023-10-08T02:11:42.109238+0000","last_active":"2023-10-08T02:11:42.114322+0000","last_peered":"2023-10-08T02:11:42.114322+0000","last_clean":"2023-10-08T02:11:42.114322+0000","last_became_active":"2023-10-08T02:11:42.108826+0000","last_became_peered":"2023-10-08T02:11:42.108826+0000","last_unstale":"2023-10-08T02:11:42.114322+0000","last_undegraded":"2023-10-08T02:11:42.114322+0000","last_fullsized":"2023-10-08T02:11:42.114322+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:07.980822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086700+0000","last_change":"2023-10-08T02:11:41.091434+0000","last_active":"2023-10-08T02:11:42.086700+0000","last_peered":"2023-10-08T02:11:42.086700+0000","last_clean":"2023-10-08T02:11:42.086700+0000","last_became_active":"2023-10-08T02:11:41.091153+0000","last_became_peered":"2023-10-08T02:11:41.091153+0000","last_unstale":"2023-10-08T02:11:42.086700+0000","last_undegraded":"2023-10-08T02:11:42.086700+0000","last_fullsized":"2023-10-08T02:11:42.086700+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:42.218801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.116417+0000","last_change":"2023-10-08T02:11:42.113410+0000","last_active":"2023-10-08T02:11:42.116417+0000","last_peered":"2023-10-08T02:11:42.116417+0000","last_clean":"2023-10-08T02:11:42.116417+0000","last_became_active":"2023-10-08T02:11:42.113122+0000","last_became_peered":"2023-10-08T02:11:42.113122+0000","last_unstale":"2023-10-08T02:11:42.116417+0000","last_undegraded":"2023-10-08T02:11:42.116417+0000","last_fullsized":"2023-10-08T02:11:42.116417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:24.684270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.098192+0000","last_change":"2023-10-08T02:11:41.089067+0000","last_active":"2023-10-08T02:11:42.098192+0000","last_peered":"2023-10-08T02:11:42.098192+0000","last_clean":"2023-10-08T02:11:42.098192+0000","last_became_active":"2023-10-08T02:11:41.088808+0000","last_became_peered":"2023-10-08T02:11:41.088808+0000","last_unstale":"2023-10-08T02:11:42.098192+0000","last_undegraded":"2023-10-08T02:11:42.098192+0000","last_fullsized":"2023-10-08T02:11:42.098192+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:48:39.872955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112267+0000","last_change":"2023-10-08T02:11:42.107010+0000","last_active":"2023-10-08T02:11:42.112267+0000","last_peered":"2023-10-08T02:11:42.112267+0000","last_clean":"2023-10-08T02:11:42.112267+0000","last_became_active":"2023-10-08T02:11:42.106730+0000","last_became_peered":"2023-10-08T02:11:42.106730+0000","last_unstale":"2023-10-08T02:11:42.112267+0000","last_undegraded":"2023-10-08T02:11:42.112267+0000","last_fullsized":"2023-10-08T02:11:42.112267+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:02.949267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.090599+0000","last_change":"2023-10-08T02:11:41.085060+0000","last_active":"2023-10-08T02:11:42.090599+0000","last_peered":"2023-10-08T02:11:42.090599+0000","last_clean":"2023-10-08T02:11:42.090599+0000","last_became_active":"2023-10-08T02:11:41.084795+0000","last_became_peered":"2023-10-08T02:11:41.084795+0000","last_unstale":"2023-10-08T02:11:42.090599+0000","last_undegraded":"2023-10-08T02:11:42.090599+0000","last_fullsized":"2023-10-08T02:11:42.090599+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:00.998019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107239+0000","last_change":"2023-10-08T02:11:42.102214+0000","last_active":"2023-10-08T02:11:42.107239+0000","last_peered":"2023-10-08T02:11:42.107239+0000","last_clean":"2023-10-08T02:11:42.107239+0000","last_became_active":"2023-10-08T02:11:42.101961+0000","last_became_peered":"2023-10-08T02:11:42.101961+0000","last_unstale":"2023-10-08T02:11:42.107239+0000","last_undegraded":"2023-10-08T02:11:42.107239+0000","last_fullsized":"2023-10-08T02:11:42.107239+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:23:53.821167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.091432+0000","last_change":"2023-10-08T02:11:41.090616+0000","last_active":"2023-10-08T02:11:42.091432+0000","last_peered":"2023-10-08T02:11:42.091432+0000","last_clean":"2023-10-08T02:11:42.091432+0000","last_became_active":"2023-10-08T02:11:41.090325+0000","last_became_peered":"2023-10-08T02:11:41.090325+0000","last_unstale":"2023-10-08T02:11:42.091432+0000","last_undegraded":"2023-10-08T02:11:42.091432+0000","last_fullsized":"2023-10-08T02:11:42.091432+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:38.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113008+0000","last_change":"2023-10-08T02:11:42.111921+0000","last_active":"2023-10-08T02:11:42.113008+0000","last_peered":"2023-10-08T02:11:42.113008+0000","last_clean":"2023-10-08T02:11:42.113008+0000","last_became_active":"2023-10-08T02:11:42.111678+0000","last_became_peered":"2023-10-08T02:11:42.111678+0000","last_unstale":"2023-10-08T02:11:42.113008+0000","last_undegraded":"2023-10-08T02:11:42.113008+0000","last_fullsized":"2023-10-08T02:11:42.113008+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086209+0000","last_change":"2023-10-08T02:11:41.084308+0000","last_active":"2023-10-08T02:11:42.086209+0000","last_peered":"2023-10-08T02:11:42.086209+0000","last_clean":"2023-10-08T02:11:42.086209+0000","last_became_active":"2023-10-08T02:11:41.084031+0000","last_became_peered":"2023-10-08T02:11:41.084031+0000","last_unstale":"2023-10-08T02:11:42.086209+0000","last_undegraded":"2023-10-08T02:11:42.086209+0000","last_fullsized":"2023-10-08T02:11:42.086209+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:13.398792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.109876+0000","last_change":"2023-10-08T02:11:42.102399+0000","last_active":"2023-10-08T02:11:42.109876+0000","last_peered":"2023-10-08T02:11:42.109876+0000","last_clean":"2023-10-08T02:11:42.109876+0000","last_became_active":"2023-10-08T02:11:42.102130+0000","last_became_peered":"2023-10-08T02:11:42.102130+0000","last_unstale":"2023-10-08T02:11:42.109876+0000","last_undegraded":"2023-10-08T02:11:42.109876+0000","last_fullsized":"2023-10-08T02:11:42.109876+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:21.328506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122295+0000","last_change":"2023-10-08T02:11:41.094140+0000","last_active":"2023-10-08T02:11:42.122295+0000","last_peered":"2023-10-08T02:11:42.122295+0000","last_clean":"2023-10-08T02:11:42.122295+0000","last_became_active":"2023-10-08T02:11:41.093877+0000","last_became_peered":"2023-10-08T02:11:41.093877+0000","last_unstale":"2023-10-08T02:11:42.122295+0000","last_undegraded":"2023-10-08T02:11:42.122295+0000","last_fullsized":"2023-10-08T02:11:42.122295+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:28.606962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"146'271","reported_seq":823,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.090620+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:11:42.090620+0000","last_peered":"2023-10-08T02:11:42.090620+0000","last_clean":"2023-10-08T02:11:42.090620+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:11:42.090620+0000","last_undegraded":"2023-10-08T02:11:42.090620+0000","last_fullsized":"2023-10-08T02:11:42.090620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575753,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307992,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063848,"statfs":{"total":96636764160,"available":96321380352,"internally_reserved":0,"allocated":2572288,"data_stored":1564765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44400000000000001},"max":{"1min":0.96499999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.751}]},{"osd":1,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.60899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.74199999999999999},"min":{"1min":0.621,"5min":0.53100000000000003,"15min":0.48899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":1.0580000000000001},"last":0.57599999999999996}]},{"osd":2,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":1.0600000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.48999999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.61599999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82799999999999996,"15min":0.80100000000000005},"min":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.57399999999999995},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.66800000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.752,"5min":0.94999999999999996,"15min":0.98399999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.64700000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.57299999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.92400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.76600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575753,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315592,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056248,"statfs":{"total":96636764160,"available":96313597952,"internally_reserved":0,"allocated":1966080,"data_stored":973787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76500000000000001,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77200000000000002,"15min":0.748},"min":{"1min":0.54900000000000004,"5min":0.501,"15min":0.46400000000000002},"max":{"1min":1.4139999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.94199999999999995}]},{"osd":1,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.48599999999999999,"15min":0.47599999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77500000000000002,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.48899999999999999},"max":{"1min":1.464,"5min":1.472,"15min":1.472},"last":0.89900000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.73499999999999999},"min":{"1min":0.60099999999999998,"5min":0.53500000000000003,"15min":0.47699999999999998},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.79700000000000004,"15min":0.76600000000000001},"min":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.436},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.84399999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.60099999999999998,"5min":0.57999999999999996,"15min":0.52800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.78500000000000003},"min":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"max":{"1min":1.048,"5min":1.169,"15min":1.169},"last":0.79200000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":1.0129999999999999,"5min":1.133,"15min":1.133},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63800000000000001,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.90200000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.97199999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54800000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95599999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.89100000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.752,"15min":0.72099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0289999999999999,"5min":1.151,"15min":1.1559999999999999},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98099999999999998,"5min":1.232,"15min":1.232},"last":0.86499999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575753,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":1953792,"data_stored":972251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.48999999999999999,"15min":0.49099999999999999},"min":{"1min":0.36299999999999999,"5min":0.33100000000000002,"15min":0.30599999999999999},"max":{"1min":0.69899999999999995,"5min":1.024,"15min":1.024},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.38100000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.66200000000000003,"5min":1.603,"15min":1.603},"last":0.53400000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.505},"min":{"1min":0.40200000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.64900000000000002,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.495},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.42599999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77700000000000002,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.55600000000000005}]},{"osd":3,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61699999999999999,"15min":0.60299999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.872,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.58799999999999997,"15min":0.56499999999999995},"min":{"1min":0.436,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.71599999999999997,"5min":3.698,"15min":3.698},"last":0.47599999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.81499999999999995,"15min":0.81000000000000005},"min":{"1min":0.56000000000000005,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":0.88800000000000001,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81100000000000005,"15min":0.82499999999999996},"min":{"1min":0.56599999999999995,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.999,"5min":1.734,"15min":1.734},"last":0.81499999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.81000000000000005,"15min":0.80100000000000005},"min":{"1min":0.59899999999999998,"5min":0.54300000000000004,"15min":0.53600000000000003},"max":{"1min":1.008,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.62,"5min":0.58999999999999997,"15min":0.56100000000000005},"max":{"1min":1.0640000000000001,"5min":1.403,"15min":1.403},"last":0.76900000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84199999999999997,"15min":0.83099999999999996},"min":{"1min":0.66300000000000003,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0269999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.68100000000000005,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.04,"5min":1.573,"15min":1.573},"last":0.85999999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.875,"15min":0.86799999999999999},"min":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.1120000000000001,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.747},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.69099999999999995,"5min":0.65100000000000002,"15min":0.58999999999999997},"max":{"1min":1.028,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.76100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575752,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1562619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48499999999999999,"15min":0.46999999999999997},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.443},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.46500000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.317},"max":{"1min":0.59699999999999998,"5min":0.70699999999999996,"15min":0.746},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.37},"max":{"1min":0.69099999999999995,"5min":0.88300000000000001,"15min":1.325},"last":0.51700000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.44600000000000001,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.873,"5min":0.875,"15min":0.875},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.54300000000000004,"5min":0.47399999999999998,"15min":0.45900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53100000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.70899999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.24},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80200000000000005,"15min":0.79500000000000004},"min":{"1min":0.621,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3759999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.94799999999999995,"5min":1.327,"15min":1.327},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89700000000000002,"15min":0.88200000000000001},"min":{"1min":0.754,"5min":0.60999999999999999,"15min":0.56100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.83899999999999997}]},{"osd":6,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85399999999999998,"15min":0.83899999999999997},"min":{"1min":0.66100000000000003,"5min":0.55000000000000004,"15min":0.53400000000000003},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.55500000000000005},"max":{"1min":0.90900000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.875,"15min":0.84999999999999998},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.55800000000000005},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.5269999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311492,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060348,"statfs":{"total":96636764160,"available":96317796352,"internally_reserved":0,"allocated":1961984,"data_stored":972839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48699999999999999,"15min":0.47999999999999998},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":1.228,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.66800000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.79500000000000004,"15min":0.872},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.41799999999999998,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.54000000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.46800000000000003,"5min":0.45500000000000002,"15min":0.39400000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.61599999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.83299999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.92300000000000004,"15min":0.871},"min":{"1min":0.749,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":1.046},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89400000000000002,"15min":0.86399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6220000000000001,"5min":1.847,"15min":1.847},"last":1.0860000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.878,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.87},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.0580000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.90300000000000002,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.64100000000000001,"5min":0.622,"15min":0.58899999999999997},"max":{"1min":1.7909999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.998}]}]},{"osd":3,"up_from":13,"seq":55834575752,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315588,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056252,"statfs":{"total":96636764160,"available":96313602048,"internally_reserved":0,"allocated":1961984,"data_stored":972411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.46500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.72199999999999998,"15min":0.87},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.32600000000000001},"max":{"1min":0.627,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.40000000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.56000000000000005,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68300000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.55200000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.42299999999999999,"15min":0.41999999999999998},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45600000000000002},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.57199999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.214},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.65400000000000003,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.1599999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79900000000000004,"15min":0.80600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"max":{"1min":1.0209999999999999,"5min":1.0589999999999999,"15min":1.125},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82299999999999995,"15min":0.82499999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56899999999999995},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.62}]},{"osd":7,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85199999999999998,"15min":0.83799999999999997},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58499999999999996},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.85499999999999998,"15min":0.84499999999999997},"min":{"1min":0.67800000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.71799999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575752,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311488,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060352,"statfs":{"total":96636764160,"available":96317800448,"internally_reserved":0,"allocated":1957888,"data_stored":972321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40400000000000003},"max":{"1min":0.90100000000000002,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0009999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.51500000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.52},"last":0.624}]},{"osd":2,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60999999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90800000000000003,"5min":1.1579999999999999,"15min":1.403},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.208,"15min":1.5069999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":1.05,"15min":1.05},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93400000000000005,"5min":1.038,"15min":1.1559999999999999},"last":0.74199999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.016},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.67000000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.0680000000000001,"15min":1.129},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.48799999999999999},"max":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70699999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575752,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311492,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060348,"statfs":{"total":96636764160,"available":96317796352,"internally_reserved":0,"allocated":1961984,"data_stored":972363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.73899999999999999},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.38900000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.75800000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.438},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.54700000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.78000000000000003},"min":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.42699999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.46200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80600000000000005,"15min":0.78100000000000003},"min":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.499},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.76200000000000001},"min":{"1min":0.61599999999999999,"5min":0.57499999999999996,"15min":0.50900000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81000000000000005}]},{"osd":3,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.56100000000000005,"15min":0.497},"max":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.0529999999999999,"15min":1.091},"last":0.68300000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.627,"15min":0.62},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.0620000000000001,"15min":1.115},"last":0.77800000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":0.93999999999999995,"5min":1.006,"15min":1.0629999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52000000000000002,"5min":0.436,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":0.95299999999999996,"15min":1.077},"last":0.88200000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":1.0249999999999999,"15min":1.161},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":1.0189999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:11:52.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:11:52.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.221+0000 7f573cbff700 1 Processor -- start 2023-10-08T02:11:52.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.221+0000 7f573cbff700 1 -- start start 2023-10-08T02:11:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.222+0000 7f573cbff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f57380cbf30 0x7f57380ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.222+0000 7f573cbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380ccf90 0x7f57380b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.222+0000 7f573cbff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57380cc300 con 0x7f573815a270 2023-10-08T02:11:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.222+0000 7f573cbff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5738156bf0 con 0x7f57380cbf30 2023-10-08T02:11:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.222+0000 7f573cbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57380ae950 con 0x7f57380ccf90 2023-10-08T02:11:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f5736d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f573815a270 0x7f57380c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55746/0 (socket says 172.21.15.119:55746) 2023-10-08T02:11:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f5736d9d700 1 -- 172.21.15.119:0/576969647 learned_addr learned my addr 172.21.15.119:0/576969647 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:52.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f5735d9b700 1 --2- 172.21.15.119:0/576969647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380ccf90 0x7f57380b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f573659c700 1 --2- 172.21.15.119:0/576969647 >> v2:172.21.15.184:3300/0 conn(0x7f57380cbf30 0x7f57380ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f573559a700 1 -- 172.21.15.119:0/576969647 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651651509 0 0) 0x7f57380cc300 con 0x7f573815a270 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f573559a700 1 -- 172.21.15.119:0/576969647 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5720003570 con 0x7f573815a270 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f5735d9b700 1 -- 172.21.15.119:0/576969647 >> v2:172.21.15.184:3300/0 conn(0x7f57380cbf30 msgr2=0x7f57380ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f5735d9b700 1 --2- 172.21.15.119:0/576969647 >> v2:172.21.15.184:3300/0 conn(0x7f57380cbf30 0x7f57380ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f5735d9b700 1 -- 172.21.15.119:0/576969647 >> v1:172.21.15.119:6789/0 conn(0x7f573815a270 legacy=0x7f57380c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f5735d9b700 1 -- 172.21.15.119:0/576969647 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57380ad370 con 0x7f57380ccf90 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.223+0000 7f573659c700 1 --2- 172.21.15.119:0/576969647 >> v2:172.21.15.184:3300/0 conn(0x7f57380cbf30 0x7f57380ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:11:52.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.224+0000 7f5735d9b700 1 --2- 172.21.15.119:0/576969647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380ccf90 0x7f57380b0d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f572c0098e0 tx=0x7f572c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8cb3b5c8a22dcb1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:52.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.224+0000 7f573559a700 1 -- 172.21.15.119:0/576969647 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c011630 con 0x7f57380ccf90 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.224+0000 7f573559a700 1 -- 172.21.15.119:0/576969647 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c011790 con 0x7f57380ccf90 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.224+0000 7f573559a700 1 -- 172.21.15.119:0/576969647 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c011b30 con 0x7f57380ccf90 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.224+0000 7f573cbff700 1 -- 172.21.15.119:0/576969647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380ccf90 msgr2=0x7f57380b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.224+0000 7f573cbff700 1 --2- 172.21.15.119:0/576969647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380ccf90 0x7f57380b0d10 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f572c0098e0 tx=0x7f572c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.225+0000 7f573cbff700 1 -- 172.21.15.119:0/576969647 shutdown_connections 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.225+0000 7f573cbff700 1 --2- 172.21.15.119:0/576969647 >> v2:172.21.15.184:3300/0 conn(0x7f57380cbf30 0x7f57380ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.225+0000 7f573cbff700 1 --2- 172.21.15.119:0/576969647 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57380ccf90 0x7f57380b0d10 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.225+0000 7f573cbff700 1 -- 172.21.15.119:0/576969647 >> 172.21.15.119:0/576969647 conn(0x7f573806a320 msgr2=0x7f57380cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.225+0000 7f573cbff700 1 -- 172.21.15.119:0/576969647 shutdown_connections 2023-10-08T02:11:52.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.225+0000 7f573cbff700 1 -- 172.21.15.119:0/576969647 wait complete. 2023-10-08T02:11:52.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.226+0000 7f573cbff700 1 Processor -- start 2023-10-08T02:11:52.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.226+0000 7f573cbff700 1 -- start start 2023-10-08T02:11:52.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f573cbff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57380cbf30 0x7f5738151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f573cbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57380ccf90 0x7f57380c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f573cbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815a270 0x7f57380c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f573cbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57380c1620 con 0x7f57380ccf90 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f573cbff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5738166730 con 0x7f57380cbf30 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f573cbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5738166a10 con 0x7f573815a270 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f5735d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57380ccf90 0x7f57380c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f5735d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57380ccf90 0x7f57380c78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50876/0 (socket says 172.21.15.119:50876) 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.227+0000 7f5735d9b700 1 -- 172.21.15.119:0/467596721 learned_addr learned my addr 172.21.15.119:0/467596721 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f5735d9b700 1 -- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815a270 msgr2=0x7f57380c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f573659c700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57380cbf30 0x7f5738151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f5736d9d700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815a270 0x7f57380c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f5735d9b700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815a270 0x7f57380c7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f5735d9b700 1 -- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57380cbf30 msgr2=0x7f5738151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f5735d9b700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57380cbf30 0x7f5738151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f5735d9b700 1 -- 172.21.15.119:0/467596721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728009420 con 0x7f57380ccf90 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f573659c700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57380cbf30 0x7f5738151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f5736d9d700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815a270 0x7f57380c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f5735d9b700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57380ccf90 0x7f57380c78c0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f572c009ed0 tx=0x7f572c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f57267fc700 1 -- 172.21.15.119:0/467596721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c011b10 con 0x7f57380ccf90 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f57267fc700 1 -- 172.21.15.119:0/467596721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c0233f0 con 0x7f57380ccf90 2023-10-08T02:11:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f572c009420 con 0x7f57380ccf90 2023-10-08T02:11:52.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f57267fc700 1 -- 172.21.15.119:0/467596721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c023770 con 0x7f57380ccf90 2023-10-08T02:11:52.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.228+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5738166fd0 con 0x7f57380ccf90 2023-10-08T02:11:52.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.230+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573804c3c0 con 0x7f57380ccf90 2023-10-08T02:11:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.230+0000 7f57267fc700 1 -- 172.21.15.119:0/467596721 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f572c02c3b0 con 0x7f57380ccf90 2023-10-08T02:11:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.231+0000 7f57267fc700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5718066f30 0x7f57180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.231+0000 7f57267fc700 1 -- 172.21.15.119:0/467596721 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f572c085cc0 con 0x7f57380ccf90 2023-10-08T02:11:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.232+0000 7f573659c700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5718066f30 0x7f57180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.232+0000 7f573659c700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5718066f30 0x7f57180693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f57280093f0 tx=0x7f5728008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.235+0000 7f57267fc700 1 -- 172.21.15.119:0/467596721 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f572c08c080 con 0x7f57380ccf90 2023-10-08T02:11:52.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.420+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f573806f8a0 con 0x7f5718066f30 2023-10-08T02:11:52.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.428+0000 7f57267fc700 1 -- 172.21.15.119:0/467596721 <== mgr.7822 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289256 (secure 0 0 0) 0x7f573806f8a0 con 0x7f5718066f30 2023-10-08T02:11:52.432 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5718066f30 msgr2=0x7f57180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5718066f30 0x7f57180693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f57280093f0 tx=0x7f5728008040 comp rx=0 tx=0).stop 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57380ccf90 msgr2=0x7f57380c78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57380ccf90 0x7f57380c78c0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f572c009ed0 tx=0x7f572c017000 comp rx=0 tx=0).stop 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 shutdown_connections 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f5718066f30 0x7f57180693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f573815a270 0x7f57380c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f57380cbf30 0x7f5738151110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 --2- 172.21.15.119:0/467596721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57380ccf90 0x7f57380c78c0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 >> 172.21.15.119:0/467596721 conn(0x7f573806a320 msgr2=0x7f5738152f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:52.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 shutdown_connections 2023-10-08T02:11:52.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.433+0000 7f573cbff700 1 -- 172.21.15.119:0/467596721 wait complete. 2023-10-08T02:11:52.436 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:11:52.451 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T02:11:51.591639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":484,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":310,"ondisk_log_size":310,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2489000,"kb_used_data":16488,"kb_used_omap":0,"kb_used_meta":2472448,"kb_avail":752485720,"statfs":{"total":773094113280,"available":770545377280,"internally_reserved":0,"allocated":16883712,"data_stored":8963356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2531786752},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.509383"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114682+0000","last_change":"2023-10-08T02:11:42.113823+0000","last_active":"2023-10-08T02:11:42.114682+0000","last_peered":"2023-10-08T02:11:42.114682+0000","last_clean":"2023-10-08T02:11:42.114682+0000","last_became_active":"2023-10-08T02:11:42.113493+0000","last_became_peered":"2023-10-08T02:11:42.113493+0000","last_unstale":"2023-10-08T02:11:42.114682+0000","last_undegraded":"2023-10-08T02:11:42.114682+0000","last_fullsized":"2023-10-08T02:11:42.114682+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114624+0000","last_change":"2023-10-08T02:11:42.113537+0000","last_active":"2023-10-08T02:11:42.114624+0000","last_peered":"2023-10-08T02:11:42.114624+0000","last_clean":"2023-10-08T02:11:42.114624+0000","last_became_active":"2023-10-08T02:11:42.113170+0000","last_became_peered":"2023-10-08T02:11:42.113170+0000","last_unstale":"2023-10-08T02:11:42.114624+0000","last_undegraded":"2023-10-08T02:11:42.114624+0000","last_fullsized":"2023-10-08T02:11:42.114624+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:23:53.821167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108736+0000","last_change":"2023-10-08T02:11:42.098817+0000","last_active":"2023-10-08T02:11:42.108736+0000","last_peered":"2023-10-08T02:11:42.108736+0000","last_clean":"2023-10-08T02:11:42.108736+0000","last_became_active":"2023-10-08T02:11:42.098478+0000","last_became_peered":"2023-10-08T02:11:42.098478+0000","last_unstale":"2023-10-08T02:11:42.108736+0000","last_undegraded":"2023-10-08T02:11:42.108736+0000","last_fullsized":"2023-10-08T02:11:42.108736+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:02.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.103531+0000","last_change":"2023-10-08T02:11:42.098697+0000","last_active":"2023-10-08T02:11:42.103531+0000","last_peered":"2023-10-08T02:11:42.103531+0000","last_clean":"2023-10-08T02:11:42.103531+0000","last_became_active":"2023-10-08T02:11:42.098447+0000","last_became_peered":"2023-10-08T02:11:42.098447+0000","last_unstale":"2023-10-08T02:11:42.103531+0000","last_undegraded":"2023-10-08T02:11:42.103531+0000","last_fullsized":"2023-10-08T02:11:42.103531+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114887+0000","last_change":"2023-10-08T02:11:42.112291+0000","last_active":"2023-10-08T02:11:42.114887+0000","last_peered":"2023-10-08T02:11:42.114887+0000","last_clean":"2023-10-08T02:11:42.114887+0000","last_became_active":"2023-10-08T02:11:42.111998+0000","last_became_peered":"2023-10-08T02:11:42.111998+0000","last_unstale":"2023-10-08T02:11:42.114887+0000","last_undegraded":"2023-10-08T02:11:42.114887+0000","last_fullsized":"2023-10-08T02:11:42.114887+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:19:24.869313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110796+0000","last_change":"2023-10-08T02:11:42.097979+0000","last_active":"2023-10-08T02:11:42.110796+0000","last_peered":"2023-10-08T02:11:42.110796+0000","last_clean":"2023-10-08T02:11:42.110796+0000","last_became_active":"2023-10-08T02:11:42.097748+0000","last_became_peered":"2023-10-08T02:11:42.097748+0000","last_unstale":"2023-10-08T02:11:42.110796+0000","last_undegraded":"2023-10-08T02:11:42.110796+0000","last_fullsized":"2023-10-08T02:11:42.110796+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:01:01.940169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110968+0000","last_change":"2023-10-08T02:11:42.104178+0000","last_active":"2023-10-08T02:11:42.110968+0000","last_peered":"2023-10-08T02:11:42.110968+0000","last_clean":"2023-10-08T02:11:42.110968+0000","last_became_active":"2023-10-08T02:11:42.103901+0000","last_became_peered":"2023-10-08T02:11:42.103901+0000","last_unstale":"2023-10-08T02:11:42.110968+0000","last_undegraded":"2023-10-08T02:11:42.110968+0000","last_fullsized":"2023-10-08T02:11:42.110968+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:55:56.724622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108817+0000","last_change":"2023-10-08T02:11:42.100208+0000","last_active":"2023-10-08T02:11:42.108817+0000","last_peered":"2023-10-08T02:11:42.108817+0000","last_clean":"2023-10-08T02:11:42.108817+0000","last_became_active":"2023-10-08T02:11:42.099895+0000","last_became_peered":"2023-10-08T02:11:42.099895+0000","last_unstale":"2023-10-08T02:11:42.108817+0000","last_undegraded":"2023-10-08T02:11:42.108817+0000","last_fullsized":"2023-10-08T02:11:42.108817+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.111468+0000","last_change":"2023-10-08T02:11:42.109029+0000","last_active":"2023-10-08T02:11:42.111468+0000","last_peered":"2023-10-08T02:11:42.111468+0000","last_clean":"2023-10-08T02:11:42.111468+0000","last_became_active":"2023-10-08T02:11:42.108769+0000","last_became_peered":"2023-10-08T02:11:42.108769+0000","last_unstale":"2023-10-08T02:11:42.111468+0000","last_undegraded":"2023-10-08T02:11:42.111468+0000","last_fullsized":"2023-10-08T02:11:42.111468+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:34.769593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114643+0000","last_change":"2023-10-08T02:11:42.113624+0000","last_active":"2023-10-08T02:11:42.114643+0000","last_peered":"2023-10-08T02:11:42.114643+0000","last_clean":"2023-10-08T02:11:42.114643+0000","last_became_active":"2023-10-08T02:11:42.113152+0000","last_became_peered":"2023-10-08T02:11:42.113152+0000","last_unstale":"2023-10-08T02:11:42.114643+0000","last_undegraded":"2023-10-08T02:11:42.114643+0000","last_fullsized":"2023-10-08T02:11:42.114643+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:11.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.103419+0000","last_change":"2023-10-08T02:11:42.100894+0000","last_active":"2023-10-08T02:11:42.103419+0000","last_peered":"2023-10-08T02:11:42.103419+0000","last_clean":"2023-10-08T02:11:42.103419+0000","last_became_active":"2023-10-08T02:11:42.100588+0000","last_became_peered":"2023-10-08T02:11:42.100588+0000","last_unstale":"2023-10-08T02:11:42.103419+0000","last_undegraded":"2023-10-08T02:11:42.103419+0000","last_fullsized":"2023-10-08T02:11:42.103419+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:51.531656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110205+0000","last_change":"2023-10-08T02:11:42.104602+0000","last_active":"2023-10-08T02:11:42.110205+0000","last_peered":"2023-10-08T02:11:42.110205+0000","last_clean":"2023-10-08T02:11:42.110205+0000","last_became_active":"2023-10-08T02:11:42.104357+0000","last_became_peered":"2023-10-08T02:11:42.104357+0000","last_unstale":"2023-10-08T02:11:42.110205+0000","last_undegraded":"2023-10-08T02:11:42.110205+0000","last_fullsized":"2023-10-08T02:11:42.110205+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:11.724000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113481+0000","last_change":"2023-10-08T02:11:42.111978+0000","last_active":"2023-10-08T02:11:42.113481+0000","last_peered":"2023-10-08T02:11:42.113481+0000","last_clean":"2023-10-08T02:11:42.113481+0000","last_became_active":"2023-10-08T02:11:42.110828+0000","last_became_peered":"2023-10-08T02:11:42.110828+0000","last_unstale":"2023-10-08T02:11:42.113481+0000","last_undegraded":"2023-10-08T02:11:42.113481+0000","last_fullsized":"2023-10-08T02:11:42.113481+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:28:11.504420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.109820+0000","last_change":"2023-10-08T02:11:42.106863+0000","last_active":"2023-10-08T02:11:42.109820+0000","last_peered":"2023-10-08T02:11:42.109820+0000","last_clean":"2023-10-08T02:11:42.109820+0000","last_became_active":"2023-10-08T02:11:42.106606+0000","last_became_peered":"2023-10-08T02:11:42.106606+0000","last_unstale":"2023-10-08T02:11:42.109820+0000","last_undegraded":"2023-10-08T02:11:42.109820+0000","last_fullsized":"2023-10-08T02:11:42.109820+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:58.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107993+0000","last_change":"2023-10-08T02:11:42.105477+0000","last_active":"2023-10-08T02:11:42.107993+0000","last_peered":"2023-10-08T02:11:42.107993+0000","last_clean":"2023-10-08T02:11:42.107993+0000","last_became_active":"2023-10-08T02:11:42.105203+0000","last_became_peered":"2023-10-08T02:11:42.105203+0000","last_unstale":"2023-10-08T02:11:42.107993+0000","last_undegraded":"2023-10-08T02:11:42.107993+0000","last_fullsized":"2023-10-08T02:11:42.107993+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:34.769593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100272+0000","last_change":"2023-10-08T02:11:42.096789+0000","last_active":"2023-10-08T02:11:42.100272+0000","last_peered":"2023-10-08T02:11:42.100272+0000","last_clean":"2023-10-08T02:11:42.100272+0000","last_became_active":"2023-10-08T02:11:42.096402+0000","last_became_peered":"2023-10-08T02:11:42.096402+0000","last_unstale":"2023-10-08T02:11:42.100272+0000","last_undegraded":"2023-10-08T02:11:42.100272+0000","last_fullsized":"2023-10-08T02:11:42.100272+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:10.087278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113401+0000","last_change":"2023-10-08T02:11:42.108742+0000","last_active":"2023-10-08T02:11:42.113401+0000","last_peered":"2023-10-08T02:11:42.113401+0000","last_clean":"2023-10-08T02:11:42.113401+0000","last_became_active":"2023-10-08T02:11:42.108463+0000","last_became_peered":"2023-10-08T02:11:42.108463+0000","last_unstale":"2023-10-08T02:11:42.113401+0000","last_undegraded":"2023-10-08T02:11:42.113401+0000","last_fullsized":"2023-10-08T02:11:42.113401+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:42:34.505384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112365+0000","last_change":"2023-10-08T02:11:42.109976+0000","last_active":"2023-10-08T02:11:42.112365+0000","last_peered":"2023-10-08T02:11:42.112365+0000","last_clean":"2023-10-08T02:11:42.112365+0000","last_became_active":"2023-10-08T02:11:42.109735+0000","last_became_peered":"2023-10-08T02:11:42.109735+0000","last_unstale":"2023-10-08T02:11:42.112365+0000","last_undegraded":"2023-10-08T02:11:42.112365+0000","last_fullsized":"2023-10-08T02:11:42.112365+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:18.861043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113932+0000","last_change":"2023-10-08T02:11:42.112275+0000","last_active":"2023-10-08T02:11:42.113932+0000","last_peered":"2023-10-08T02:11:42.113932+0000","last_clean":"2023-10-08T02:11:42.113932+0000","last_became_active":"2023-10-08T02:11:42.112031+0000","last_became_peered":"2023-10-08T02:11:42.112031+0000","last_unstale":"2023-10-08T02:11:42.113932+0000","last_undegraded":"2023-10-08T02:11:42.113932+0000","last_fullsized":"2023-10-08T02:11:42.113932+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:06.332369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107231+0000","last_change":"2023-10-08T02:11:42.103237+0000","last_active":"2023-10-08T02:11:42.107231+0000","last_peered":"2023-10-08T02:11:42.107231+0000","last_clean":"2023-10-08T02:11:42.107231+0000","last_became_active":"2023-10-08T02:11:42.102996+0000","last_became_peered":"2023-10-08T02:11:42.102996+0000","last_unstale":"2023-10-08T02:11:42.107231+0000","last_undegraded":"2023-10-08T02:11:42.107231+0000","last_fullsized":"2023-10-08T02:11:42.107231+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:25.186901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113380+0000","last_change":"2023-10-08T02:11:42.111018+0000","last_active":"2023-10-08T02:11:42.113380+0000","last_peered":"2023-10-08T02:11:42.113380+0000","last_clean":"2023-10-08T02:11:42.113380+0000","last_became_active":"2023-10-08T02:11:42.110766+0000","last_became_peered":"2023-10-08T02:11:42.110766+0000","last_unstale":"2023-10-08T02:11:42.113380+0000","last_undegraded":"2023-10-08T02:11:42.113380+0000","last_fullsized":"2023-10-08T02:11:42.113380+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:25.186901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107088+0000","last_change":"2023-10-08T02:11:42.106099+0000","last_active":"2023-10-08T02:11:42.107088+0000","last_peered":"2023-10-08T02:11:42.107088+0000","last_clean":"2023-10-08T02:11:42.107088+0000","last_became_active":"2023-10-08T02:11:42.105837+0000","last_became_peered":"2023-10-08T02:11:42.105837+0000","last_unstale":"2023-10-08T02:11:42.107088+0000","last_undegraded":"2023-10-08T02:11:42.107088+0000","last_fullsized":"2023-10-08T02:11:42.107088+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:20:22.999277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112022+0000","last_change":"2023-10-08T02:11:42.105982+0000","last_active":"2023-10-08T02:11:42.112022+0000","last_peered":"2023-10-08T02:11:42.112022+0000","last_clean":"2023-10-08T02:11:42.112022+0000","last_became_active":"2023-10-08T02:11:42.105508+0000","last_became_peered":"2023-10-08T02:11:42.105508+0000","last_unstale":"2023-10-08T02:11:42.112022+0000","last_undegraded":"2023-10-08T02:11:42.112022+0000","last_fullsized":"2023-10-08T02:11:42.112022+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:56.323343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108198+0000","last_change":"2023-10-08T02:11:42.104513+0000","last_active":"2023-10-08T02:11:42.108198+0000","last_peered":"2023-10-08T02:11:42.108198+0000","last_clean":"2023-10-08T02:11:42.108198+0000","last_became_active":"2023-10-08T02:11:42.104256+0000","last_became_peered":"2023-10-08T02:11:42.104256+0000","last_unstale":"2023-10-08T02:11:42.108198+0000","last_undegraded":"2023-10-08T02:11:42.108198+0000","last_fullsized":"2023-10-08T02:11:42.108198+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:26:19.029967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112851+0000","last_change":"2023-10-08T02:11:42.110658+0000","last_active":"2023-10-08T02:11:42.112851+0000","last_peered":"2023-10-08T02:11:42.112851+0000","last_clean":"2023-10-08T02:11:42.112851+0000","last_became_active":"2023-10-08T02:11:42.110397+0000","last_became_peered":"2023-10-08T02:11:42.110397+0000","last_unstale":"2023-10-08T02:11:42.112851+0000","last_undegraded":"2023-10-08T02:11:42.112851+0000","last_fullsized":"2023-10-08T02:11:42.112851+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:04.169449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100237+0000","last_change":"2023-10-08T02:11:42.097346+0000","last_active":"2023-10-08T02:11:42.100237+0000","last_peered":"2023-10-08T02:11:42.100237+0000","last_clean":"2023-10-08T02:11:42.100237+0000","last_became_active":"2023-10-08T02:11:42.097114+0000","last_became_peered":"2023-10-08T02:11:42.097114+0000","last_unstale":"2023-10-08T02:11:42.100237+0000","last_undegraded":"2023-10-08T02:11:42.100237+0000","last_fullsized":"2023-10-08T02:11:42.100237+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:17.058773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.111811+0000","last_change":"2023-10-08T02:11:42.110297+0000","last_active":"2023-10-08T02:11:42.111811+0000","last_peered":"2023-10-08T02:11:42.111811+0000","last_clean":"2023-10-08T02:11:42.111811+0000","last_became_active":"2023-10-08T02:11:42.110051+0000","last_became_peered":"2023-10-08T02:11:42.110051+0000","last_unstale":"2023-10-08T02:11:42.111811+0000","last_undegraded":"2023-10-08T02:11:42.111811+0000","last_fullsized":"2023-10-08T02:11:42.111811+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:02.003654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.104854+0000","last_change":"2023-10-08T02:11:42.103016+0000","last_active":"2023-10-08T02:11:42.104854+0000","last_peered":"2023-10-08T02:11:42.104854+0000","last_clean":"2023-10-08T02:11:42.104854+0000","last_became_active":"2023-10-08T02:11:42.102673+0000","last_became_peered":"2023-10-08T02:11:42.102673+0000","last_unstale":"2023-10-08T02:11:42.104854+0000","last_undegraded":"2023-10-08T02:11:42.104854+0000","last_fullsized":"2023-10-08T02:11:42.104854+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:39:19.177440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106088+0000","last_change":"2023-10-08T02:11:42.105002+0000","last_active":"2023-10-08T02:11:42.106088+0000","last_peered":"2023-10-08T02:11:42.106088+0000","last_clean":"2023-10-08T02:11:42.106088+0000","last_became_active":"2023-10-08T02:11:42.104707+0000","last_became_peered":"2023-10-08T02:11:42.104707+0000","last_unstale":"2023-10-08T02:11:42.106088+0000","last_undegraded":"2023-10-08T02:11:42.106088+0000","last_fullsized":"2023-10-08T02:11:42.106088+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.106069+0000","last_change":"2023-10-08T02:11:42.104624+0000","last_active":"2023-10-08T02:11:42.106069+0000","last_peered":"2023-10-08T02:11:42.106069+0000","last_clean":"2023-10-08T02:11:42.106069+0000","last_became_active":"2023-10-08T02:11:42.104319+0000","last_became_peered":"2023-10-08T02:11:42.104319+0000","last_unstale":"2023-10-08T02:11:42.106069+0000","last_undegraded":"2023-10-08T02:11:42.106069+0000","last_fullsized":"2023-10-08T02:11:42.106069+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:56.323343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107233+0000","last_change":"2023-10-08T02:11:42.100377+0000","last_active":"2023-10-08T02:11:42.107233+0000","last_peered":"2023-10-08T02:11:42.107233+0000","last_clean":"2023-10-08T02:11:42.107233+0000","last_became_active":"2023-10-08T02:11:42.100065+0000","last_became_peered":"2023-10-08T02:11:42.100065+0000","last_unstale":"2023-10-08T02:11:42.107233+0000","last_undegraded":"2023-10-08T02:11:42.107233+0000","last_fullsized":"2023-10-08T02:11:42.107233+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:02.949267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110762+0000","last_change":"2023-10-08T02:11:42.103389+0000","last_active":"2023-10-08T02:11:42.110762+0000","last_peered":"2023-10-08T02:11:42.110762+0000","last_clean":"2023-10-08T02:11:42.110762+0000","last_became_active":"2023-10-08T02:11:42.103115+0000","last_became_peered":"2023-10-08T02:11:42.103115+0000","last_unstale":"2023-10-08T02:11:42.110762+0000","last_undegraded":"2023-10-08T02:11:42.110762+0000","last_fullsized":"2023-10-08T02:11:42.110762+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:38:59.671124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107216+0000","last_change":"2023-10-08T02:11:42.104463+0000","last_active":"2023-10-08T02:11:42.107216+0000","last_peered":"2023-10-08T02:11:42.107216+0000","last_clean":"2023-10-08T02:11:42.107216+0000","last_became_active":"2023-10-08T02:11:42.104103+0000","last_became_peered":"2023-10-08T02:11:42.104103+0000","last_unstale":"2023-10-08T02:11:42.107216+0000","last_undegraded":"2023-10-08T02:11:42.107216+0000","last_fullsized":"2023-10-08T02:11:42.107216+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:26.503280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"156'4","reported_seq":14,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.123290+0000","last_change":"2023-10-08T02:11:41.091637+0000","last_active":"2023-10-08T02:11:42.123290+0000","last_peered":"2023-10-08T02:11:42.123290+0000","last_clean":"2023-10-08T02:11:42.123290+0000","last_became_active":"2023-10-08T02:11:41.091309+0000","last_became_peered":"2023-10-08T02:11:41.091309+0000","last_unstale":"2023-10-08T02:11:42.123290+0000","last_undegraded":"2023-10-08T02:11:42.123290+0000","last_fullsized":"2023-10-08T02:11:42.123290+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:28.798304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.099197+0000","last_change":"2023-10-08T02:11:42.097072+0000","last_active":"2023-10-08T02:11:42.099197+0000","last_peered":"2023-10-08T02:11:42.099197+0000","last_clean":"2023-10-08T02:11:42.099197+0000","last_became_active":"2023-10-08T02:11:42.095258+0000","last_became_peered":"2023-10-08T02:11:42.095258+0000","last_unstale":"2023-10-08T02:11:42.099197+0000","last_undegraded":"2023-10-08T02:11:42.099197+0000","last_fullsized":"2023-10-08T02:11:42.099197+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:53.166034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122732+0000","last_change":"2023-10-08T02:11:41.086355+0000","last_active":"2023-10-08T02:11:42.122732+0000","last_peered":"2023-10-08T02:11:42.122732+0000","last_clean":"2023-10-08T02:11:42.122732+0000","last_became_active":"2023-10-08T02:11:41.084532+0000","last_became_peered":"2023-10-08T02:11:41.084532+0000","last_unstale":"2023-10-08T02:11:42.122732+0000","last_undegraded":"2023-10-08T02:11:42.122732+0000","last_fullsized":"2023-10-08T02:11:42.122732+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:52:17.855408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114801+0000","last_change":"2023-10-08T02:11:42.112854+0000","last_active":"2023-10-08T02:11:42.114801+0000","last_peered":"2023-10-08T02:11:42.114801+0000","last_clean":"2023-10-08T02:11:42.114801+0000","last_became_active":"2023-10-08T02:11:42.112549+0000","last_became_peered":"2023-10-08T02:11:42.112549+0000","last_unstale":"2023-10-08T02:11:42.114801+0000","last_undegraded":"2023-10-08T02:11:42.114801+0000","last_fullsized":"2023-10-08T02:11:42.114801+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:29.892608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.121532+0000","last_change":"2023-10-08T02:11:41.095273+0000","last_active":"2023-10-08T02:11:42.121532+0000","last_peered":"2023-10-08T02:11:42.121532+0000","last_clean":"2023-10-08T02:11:42.121532+0000","last_became_active":"2023-10-08T02:11:41.095001+0000","last_became_peered":"2023-10-08T02:11:41.095001+0000","last_unstale":"2023-10-08T02:11:42.121532+0000","last_undegraded":"2023-10-08T02:11:42.121532+0000","last_fullsized":"2023-10-08T02:11:42.121532+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:19.066208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108482+0000","last_change":"2023-10-08T02:11:42.101145+0000","last_active":"2023-10-08T02:11:42.108482+0000","last_peered":"2023-10-08T02:11:42.108482+0000","last_clean":"2023-10-08T02:11:42.108482+0000","last_became_active":"2023-10-08T02:11:42.100850+0000","last_became_peered":"2023-10-08T02:11:42.100850+0000","last_unstale":"2023-10-08T02:11:42.108482+0000","last_undegraded":"2023-10-08T02:11:42.108482+0000","last_fullsized":"2023-10-08T02:11:42.108482+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:59.627184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.120117+0000","last_change":"2023-10-08T02:11:41.094608+0000","last_active":"2023-10-08T02:11:42.120117+0000","last_peered":"2023-10-08T02:11:42.120117+0000","last_clean":"2023-10-08T02:11:42.120117+0000","last_became_active":"2023-10-08T02:11:41.094299+0000","last_became_peered":"2023-10-08T02:11:41.094299+0000","last_unstale":"2023-10-08T02:11:42.120117+0000","last_undegraded":"2023-10-08T02:11:42.120117+0000","last_fullsized":"2023-10-08T02:11:42.120117+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:03:44.763746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113544+0000","last_change":"2023-10-08T02:11:42.111527+0000","last_active":"2023-10-08T02:11:42.113544+0000","last_peered":"2023-10-08T02:11:42.113544+0000","last_clean":"2023-10-08T02:11:42.113544+0000","last_became_active":"2023-10-08T02:11:42.111206+0000","last_became_peered":"2023-10-08T02:11:42.111206+0000","last_unstale":"2023-10-08T02:11:42.113544+0000","last_undegraded":"2023-10-08T02:11:42.113544+0000","last_fullsized":"2023-10-08T02:11:42.113544+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:14.761546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.119642+0000","last_change":"2023-10-08T02:11:41.094567+0000","last_active":"2023-10-08T02:11:42.119642+0000","last_peered":"2023-10-08T02:11:42.119642+0000","last_clean":"2023-10-08T02:11:42.119642+0000","last_became_active":"2023-10-08T02:11:41.094277+0000","last_became_peered":"2023-10-08T02:11:41.094277+0000","last_unstale":"2023-10-08T02:11:42.119642+0000","last_undegraded":"2023-10-08T02:11:42.119642+0000","last_fullsized":"2023-10-08T02:11:42.119642+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:36.101961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114403+0000","last_change":"2023-10-08T02:11:42.109492+0000","last_active":"2023-10-08T02:11:42.114403+0000","last_peered":"2023-10-08T02:11:42.114403+0000","last_clean":"2023-10-08T02:11:42.114403+0000","last_became_active":"2023-10-08T02:11:42.109130+0000","last_became_peered":"2023-10-08T02:11:42.109130+0000","last_unstale":"2023-10-08T02:11:42.114403+0000","last_undegraded":"2023-10-08T02:11:42.114403+0000","last_fullsized":"2023-10-08T02:11:42.114403+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:01:50.680455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122553+0000","last_change":"2023-10-08T02:11:41.094113+0000","last_active":"2023-10-08T02:11:42.122553+0000","last_peered":"2023-10-08T02:11:42.122553+0000","last_clean":"2023-10-08T02:11:42.122553+0000","last_became_active":"2023-10-08T02:11:41.093857+0000","last_became_peered":"2023-10-08T02:11:41.093857+0000","last_unstale":"2023-10-08T02:11:42.122553+0000","last_undegraded":"2023-10-08T02:11:42.122553+0000","last_fullsized":"2023-10-08T02:11:42.122553+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:55.317708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108797+0000","last_change":"2023-10-08T02:11:42.099629+0000","last_active":"2023-10-08T02:11:42.108797+0000","last_peered":"2023-10-08T02:11:42.108797+0000","last_clean":"2023-10-08T02:11:42.108797+0000","last_became_active":"2023-10-08T02:11:42.099303+0000","last_became_peered":"2023-10-08T02:11:42.099303+0000","last_unstale":"2023-10-08T02:11:42.108797+0000","last_undegraded":"2023-10-08T02:11:42.108797+0000","last_fullsized":"2023-10-08T02:11:42.108797+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.096277+0000","last_change":"2023-10-08T02:11:41.086476+0000","last_active":"2023-10-08T02:11:42.096277+0000","last_peered":"2023-10-08T02:11:42.096277+0000","last_clean":"2023-10-08T02:11:42.096277+0000","last_became_active":"2023-10-08T02:11:41.085954+0000","last_became_peered":"2023-10-08T02:11:41.085954+0000","last_unstale":"2023-10-08T02:11:42.096277+0000","last_undegraded":"2023-10-08T02:11:42.096277+0000","last_fullsized":"2023-10-08T02:11:42.096277+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:23.678635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110289+0000","last_change":"2023-10-08T02:11:42.107521+0000","last_active":"2023-10-08T02:11:42.110289+0000","last_peered":"2023-10-08T02:11:42.110289+0000","last_clean":"2023-10-08T02:11:42.110289+0000","last_became_active":"2023-10-08T02:11:42.107255+0000","last_became_peered":"2023-10-08T02:11:42.107255+0000","last_unstale":"2023-10-08T02:11:42.110289+0000","last_undegraded":"2023-10-08T02:11:42.110289+0000","last_fullsized":"2023-10-08T02:11:42.110289+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.096045+0000","last_change":"2023-10-08T02:11:41.091321+0000","last_active":"2023-10-08T02:11:42.096045+0000","last_peered":"2023-10-08T02:11:42.096045+0000","last_clean":"2023-10-08T02:11:42.096045+0000","last_became_active":"2023-10-08T02:11:41.091092+0000","last_became_peered":"2023-10-08T02:11:41.091092+0000","last_unstale":"2023-10-08T02:11:42.096045+0000","last_undegraded":"2023-10-08T02:11:42.096045+0000","last_fullsized":"2023-10-08T02:11:42.096045+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100491+0000","last_change":"2023-10-08T02:11:42.095233+0000","last_active":"2023-10-08T02:11:42.100491+0000","last_peered":"2023-10-08T02:11:42.100491+0000","last_clean":"2023-10-08T02:11:42.100491+0000","last_became_active":"2023-10-08T02:11:42.094991+0000","last_became_peered":"2023-10-08T02:11:42.094991+0000","last_unstale":"2023-10-08T02:11:42.100491+0000","last_undegraded":"2023-10-08T02:11:42.100491+0000","last_fullsized":"2023-10-08T02:11:42.100491+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:40:58.364019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.095962+0000","last_change":"2023-10-08T02:11:41.090834+0000","last_active":"2023-10-08T02:11:42.095962+0000","last_peered":"2023-10-08T02:11:42.095962+0000","last_clean":"2023-10-08T02:11:42.095962+0000","last_became_active":"2023-10-08T02:11:41.090569+0000","last_became_peered":"2023-10-08T02:11:41.090569+0000","last_unstale":"2023-10-08T02:11:42.095962+0000","last_undegraded":"2023-10-08T02:11:42.095962+0000","last_fullsized":"2023-10-08T02:11:42.095962+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:42:24.181266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112899+0000","last_change":"2023-10-08T02:11:42.107968+0000","last_active":"2023-10-08T02:11:42.112899+0000","last_peered":"2023-10-08T02:11:42.112899+0000","last_clean":"2023-10-08T02:11:42.112899+0000","last_became_active":"2023-10-08T02:11:42.107634+0000","last_became_peered":"2023-10-08T02:11:42.107634+0000","last_unstale":"2023-10-08T02:11:42.112899+0000","last_undegraded":"2023-10-08T02:11:42.112899+0000","last_fullsized":"2023-10-08T02:11:42.112899+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:59.627184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.092141+0000","last_change":"2023-10-08T02:11:41.092290+0000","last_active":"2023-10-08T02:11:42.092141+0000","last_peered":"2023-10-08T02:11:42.092141+0000","last_clean":"2023-10-08T02:11:42.092141+0000","last_became_active":"2023-10-08T02:11:41.091969+0000","last_became_peered":"2023-10-08T02:11:41.091969+0000","last_unstale":"2023-10-08T02:11:42.092141+0000","last_undegraded":"2023-10-08T02:11:42.092141+0000","last_fullsized":"2023-10-08T02:11:42.092141+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:15:53.014859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107241+0000","last_change":"2023-10-08T02:11:42.100490+0000","last_active":"2023-10-08T02:11:42.107241+0000","last_peered":"2023-10-08T02:11:42.107241+0000","last_clean":"2023-10-08T02:11:42.107241+0000","last_became_active":"2023-10-08T02:11:42.100156+0000","last_became_peered":"2023-10-08T02:11:42.100156+0000","last_unstale":"2023-10-08T02:11:42.107241+0000","last_undegraded":"2023-10-08T02:11:42.107241+0000","last_fullsized":"2023-10-08T02:11:42.107241+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:39:19.177440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"156'3","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122321+0000","last_change":"2023-10-08T02:11:41.094352+0000","last_active":"2023-10-08T02:11:42.122321+0000","last_peered":"2023-10-08T02:11:42.122321+0000","last_clean":"2023-10-08T02:11:42.122321+0000","last_became_active":"2023-10-08T02:11:41.094048+0000","last_became_peered":"2023-10-08T02:11:41.094048+0000","last_unstale":"2023-10-08T02:11:42.122321+0000","last_undegraded":"2023-10-08T02:11:42.122321+0000","last_fullsized":"2023-10-08T02:11:42.122321+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:03.163814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108463+0000","last_change":"2023-10-08T02:11:42.100509+0000","last_active":"2023-10-08T02:11:42.108463+0000","last_peered":"2023-10-08T02:11:42.108463+0000","last_clean":"2023-10-08T02:11:42.108463+0000","last_became_active":"2023-10-08T02:11:42.100202+0000","last_became_peered":"2023-10-08T02:11:42.100202+0000","last_unstale":"2023-10-08T02:11:42.108463+0000","last_undegraded":"2023-10-08T02:11:42.108463+0000","last_fullsized":"2023-10-08T02:11:42.108463+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:01.047419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"156'6","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125472+0000","last_change":"2023-10-08T02:11:41.080631+0000","last_active":"2023-10-08T02:11:42.125472+0000","last_peered":"2023-10-08T02:11:42.125472+0000","last_clean":"2023-10-08T02:11:42.125472+0000","last_became_active":"2023-10-08T02:11:41.080365+0000","last_became_peered":"2023-10-08T02:11:41.080365+0000","last_unstale":"2023-10-08T02:11:42.125472+0000","last_undegraded":"2023-10-08T02:11:42.125472+0000","last_fullsized":"2023-10-08T02:11:42.125472+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.100695+0000","last_change":"2023-10-08T02:11:42.096482+0000","last_active":"2023-10-08T02:11:42.100695+0000","last_peered":"2023-10-08T02:11:42.100695+0000","last_clean":"2023-10-08T02:11:42.100695+0000","last_became_active":"2023-10-08T02:11:42.094165+0000","last_became_peered":"2023-10-08T02:11:42.094165+0000","last_unstale":"2023-10-08T02:11:42.100695+0000","last_undegraded":"2023-10-08T02:11:42.100695+0000","last_fullsized":"2023-10-08T02:11:42.100695+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:23.801545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086789+0000","last_change":"2023-10-08T02:11:41.084280+0000","last_active":"2023-10-08T02:11:42.086789+0000","last_peered":"2023-10-08T02:11:42.086789+0000","last_clean":"2023-10-08T02:11:42.086789+0000","last_became_active":"2023-10-08T02:11:41.083971+0000","last_became_peered":"2023-10-08T02:11:41.083971+0000","last_unstale":"2023-10-08T02:11:42.086789+0000","last_undegraded":"2023-10-08T02:11:42.086789+0000","last_fullsized":"2023-10-08T02:11:42.086789+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112430+0000","last_change":"2023-10-08T02:11:42.110754+0000","last_active":"2023-10-08T02:11:42.112430+0000","last_peered":"2023-10-08T02:11:42.112430+0000","last_clean":"2023-10-08T02:11:42.112430+0000","last_became_active":"2023-10-08T02:11:42.110475+0000","last_became_peered":"2023-10-08T02:11:42.110475+0000","last_unstale":"2023-10-08T02:11:42.112430+0000","last_undegraded":"2023-10-08T02:11:42.112430+0000","last_fullsized":"2023-10-08T02:11:42.112430+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:17.058773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.098318+0000","last_change":"2023-10-08T02:11:41.084543+0000","last_active":"2023-10-08T02:11:42.098318+0000","last_peered":"2023-10-08T02:11:42.098318+0000","last_clean":"2023-10-08T02:11:42.098318+0000","last_became_active":"2023-10-08T02:11:41.084263+0000","last_became_peered":"2023-10-08T02:11:41.084263+0000","last_unstale":"2023-10-08T02:11:42.098318+0000","last_undegraded":"2023-10-08T02:11:42.098318+0000","last_fullsized":"2023-10-08T02:11:42.098318+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:48:41.860287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112003+0000","last_change":"2023-10-08T02:11:42.105772+0000","last_active":"2023-10-08T02:11:42.112003+0000","last_peered":"2023-10-08T02:11:42.112003+0000","last_clean":"2023-10-08T02:11:42.112003+0000","last_became_active":"2023-10-08T02:11:42.105165+0000","last_became_peered":"2023-10-08T02:11:42.105165+0000","last_unstale":"2023-10-08T02:11:42.112003+0000","last_undegraded":"2023-10-08T02:11:42.112003+0000","last_fullsized":"2023-10-08T02:11:42.112003+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:12.133057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"156'8","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.123411+0000","last_change":"2023-10-08T02:11:41.089825+0000","last_active":"2023-10-08T02:11:42.123411+0000","last_peered":"2023-10-08T02:11:42.123411+0000","last_clean":"2023-10-08T02:11:42.123411+0000","last_became_active":"2023-10-08T02:11:41.089299+0000","last_became_peered":"2023-10-08T02:11:41.089299+0000","last_unstale":"2023-10-08T02:11:42.123411+0000","last_undegraded":"2023-10-08T02:11:42.123411+0000","last_fullsized":"2023-10-08T02:11:42.123411+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:10:50.165805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.105986+0000","last_change":"2023-10-08T02:11:42.098176+0000","last_active":"2023-10-08T02:11:42.105986+0000","last_peered":"2023-10-08T02:11:42.105986+0000","last_clean":"2023-10-08T02:11:42.105986+0000","last_became_active":"2023-10-08T02:11:42.097581+0000","last_became_peered":"2023-10-08T02:11:42.097581+0000","last_unstale":"2023-10-08T02:11:42.105986+0000","last_undegraded":"2023-10-08T02:11:42.105986+0000","last_fullsized":"2023-10-08T02:11:42.105986+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:12:29.803939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122176+0000","last_change":"2023-10-08T02:11:41.081179+0000","last_active":"2023-10-08T02:11:42.122176+0000","last_peered":"2023-10-08T02:11:42.122176+0000","last_clean":"2023-10-08T02:11:42.122176+0000","last_became_active":"2023-10-08T02:11:41.080899+0000","last_became_peered":"2023-10-08T02:11:41.080899+0000","last_unstale":"2023-10-08T02:11:42.122176+0000","last_undegraded":"2023-10-08T02:11:42.122176+0000","last_fullsized":"2023-10-08T02:11:42.122176+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:20:09.081644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.108072+0000","last_change":"2023-10-08T02:11:42.105520+0000","last_active":"2023-10-08T02:11:42.108072+0000","last_peered":"2023-10-08T02:11:42.108072+0000","last_clean":"2023-10-08T02:11:42.108072+0000","last_became_active":"2023-10-08T02:11:42.105200+0000","last_became_peered":"2023-10-08T02:11:42.105200+0000","last_unstale":"2023-10-08T02:11:42.108072+0000","last_undegraded":"2023-10-08T02:11:42.108072+0000","last_fullsized":"2023-10-08T02:11:42.108072+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:21.328506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125324+0000","last_change":"2023-10-08T02:11:41.089638+0000","last_active":"2023-10-08T02:11:42.125324+0000","last_peered":"2023-10-08T02:11:42.125324+0000","last_clean":"2023-10-08T02:11:42.125324+0000","last_became_active":"2023-10-08T02:11:41.089339+0000","last_became_peered":"2023-10-08T02:11:41.089339+0000","last_unstale":"2023-10-08T02:11:42.125324+0000","last_undegraded":"2023-10-08T02:11:42.125324+0000","last_fullsized":"2023-10-08T02:11:42.125324+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:22:26.191071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107213+0000","last_change":"2023-10-08T02:11:42.100803+0000","last_active":"2023-10-08T02:11:42.107213+0000","last_peered":"2023-10-08T02:11:42.107213+0000","last_clean":"2023-10-08T02:11:42.107213+0000","last_became_active":"2023-10-08T02:11:42.100558+0000","last_became_peered":"2023-10-08T02:11:42.100558+0000","last_unstale":"2023-10-08T02:11:42.107213+0000","last_undegraded":"2023-10-08T02:11:42.107213+0000","last_fullsized":"2023-10-08T02:11:42.107213+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:04.169449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"156'1","reported_seq":11,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.124741+0000","last_change":"2023-10-08T02:11:41.096436+0000","last_active":"2023-10-08T02:11:42.124741+0000","last_peered":"2023-10-08T02:11:42.124741+0000","last_clean":"2023-10-08T02:11:42.124741+0000","last_became_active":"2023-10-08T02:11:41.096182+0000","last_became_peered":"2023-10-08T02:11:41.096182+0000","last_unstale":"2023-10-08T02:11:42.124741+0000","last_undegraded":"2023-10-08T02:11:42.124741+0000","last_fullsized":"2023-10-08T02:11:42.124741+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:56.085728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.105522+0000","last_change":"2023-10-08T02:11:42.103049+0000","last_active":"2023-10-08T02:11:42.105522+0000","last_peered":"2023-10-08T02:11:42.105522+0000","last_clean":"2023-10-08T02:11:42.105522+0000","last_became_active":"2023-10-08T02:11:42.102617+0000","last_became_peered":"2023-10-08T02:11:42.102617+0000","last_unstale":"2023-10-08T02:11:42.105522+0000","last_undegraded":"2023-10-08T02:11:42.105522+0000","last_fullsized":"2023-10-08T02:11:42.105522+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:23.801545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.095197+0000","last_change":"2023-10-08T02:11:41.083978+0000","last_active":"2023-10-08T02:11:42.095197+0000","last_peered":"2023-10-08T02:11:42.095197+0000","last_clean":"2023-10-08T02:11:42.095197+0000","last_became_active":"2023-10-08T02:11:41.083711+0000","last_became_peered":"2023-10-08T02:11:41.083711+0000","last_unstale":"2023-10-08T02:11:42.095197+0000","last_undegraded":"2023-10-08T02:11:42.095197+0000","last_fullsized":"2023-10-08T02:11:42.095197+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:20:13.755911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113445+0000","last_change":"2023-10-08T02:11:42.111899+0000","last_active":"2023-10-08T02:11:42.113445+0000","last_peered":"2023-10-08T02:11:42.113445+0000","last_clean":"2023-10-08T02:11:42.113445+0000","last_became_active":"2023-10-08T02:11:42.111582+0000","last_became_peered":"2023-10-08T02:11:42.111582+0000","last_unstale":"2023-10-08T02:11:42.113445+0000","last_undegraded":"2023-10-08T02:11:42.113445+0000","last_fullsized":"2023-10-08T02:11:42.113445+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:23:11.724000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122478+0000","last_change":"2023-10-08T02:11:41.091273+0000","last_active":"2023-10-08T02:11:42.122478+0000","last_peered":"2023-10-08T02:11:42.122478+0000","last_clean":"2023-10-08T02:11:42.122478+0000","last_became_active":"2023-10-08T02:11:41.091018+0000","last_became_peered":"2023-10-08T02:11:41.091018+0000","last_unstale":"2023-10-08T02:11:42.122478+0000","last_undegraded":"2023-10-08T02:11:42.122478+0000","last_fullsized":"2023-10-08T02:11:42.122478+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:15:16.053138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110910+0000","last_change":"2023-10-08T02:11:42.103858+0000","last_active":"2023-10-08T02:11:42.110910+0000","last_peered":"2023-10-08T02:11:42.110910+0000","last_clean":"2023-10-08T02:11:42.110910+0000","last_became_active":"2023-10-08T02:11:42.103492+0000","last_became_peered":"2023-10-08T02:11:42.103492+0000","last_unstale":"2023-10-08T02:11:42.110910+0000","last_undegraded":"2023-10-08T02:11:42.110910+0000","last_fullsized":"2023-10-08T02:11:42.110910+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:22:29.230376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.091549+0000","last_change":"2023-10-08T02:11:41.091649+0000","last_active":"2023-10-08T02:11:42.091549+0000","last_peered":"2023-10-08T02:11:42.091549+0000","last_clean":"2023-10-08T02:11:42.091549+0000","last_became_active":"2023-10-08T02:11:41.091394+0000","last_became_peered":"2023-10-08T02:11:41.091394+0000","last_unstale":"2023-10-08T02:11:42.091549+0000","last_undegraded":"2023-10-08T02:11:42.091549+0000","last_fullsized":"2023-10-08T02:11:42.091549+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:11.127422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.105430+0000","last_change":"2023-10-08T02:11:42.101708+0000","last_active":"2023-10-08T02:11:42.105430+0000","last_peered":"2023-10-08T02:11:42.105430+0000","last_clean":"2023-10-08T02:11:42.105430+0000","last_became_active":"2023-10-08T02:11:42.101426+0000","last_became_peered":"2023-10-08T02:11:42.101426+0000","last_unstale":"2023-10-08T02:11:42.105430+0000","last_undegraded":"2023-10-08T02:11:42.105430+0000","last_fullsized":"2023-10-08T02:11:42.105430+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:29.612597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.090646+0000","last_change":"2023-10-08T02:11:41.088767+0000","last_active":"2023-10-08T02:11:42.090646+0000","last_peered":"2023-10-08T02:11:42.090646+0000","last_clean":"2023-10-08T02:11:42.090646+0000","last_became_active":"2023-10-08T02:11:41.088418+0000","last_became_peered":"2023-10-08T02:11:41.088418+0000","last_unstale":"2023-10-08T02:11:42.090646+0000","last_undegraded":"2023-10-08T02:11:42.090646+0000","last_fullsized":"2023-10-08T02:11:42.090646+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:21:52.160399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.110003+0000","last_change":"2023-10-08T02:11:42.104907+0000","last_active":"2023-10-08T02:11:42.110003+0000","last_peered":"2023-10-08T02:11:42.110003+0000","last_clean":"2023-10-08T02:11:42.110003+0000","last_became_active":"2023-10-08T02:11:42.102578+0000","last_became_peered":"2023-10-08T02:11:42.102578+0000","last_unstale":"2023-10-08T02:11:42.110003+0000","last_undegraded":"2023-10-08T02:11:42.110003+0000","last_fullsized":"2023-10-08T02:11:42.110003+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:33:57.091363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086116+0000","last_change":"2023-10-08T02:11:41.089652+0000","last_active":"2023-10-08T02:11:42.086116+0000","last_peered":"2023-10-08T02:11:42.086116+0000","last_clean":"2023-10-08T02:11:42.086116+0000","last_became_active":"2023-10-08T02:11:41.089362+0000","last_became_peered":"2023-10-08T02:11:41.089362+0000","last_unstale":"2023-10-08T02:11:42.086116+0000","last_undegraded":"2023-10-08T02:11:42.086116+0000","last_fullsized":"2023-10-08T02:11:42.086116+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:02.686826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114760+0000","last_change":"2023-10-08T02:11:42.109782+0000","last_active":"2023-10-08T02:11:42.114760+0000","last_peered":"2023-10-08T02:11:42.114760+0000","last_clean":"2023-10-08T02:11:42.114760+0000","last_became_active":"2023-10-08T02:11:42.109478+0000","last_became_peered":"2023-10-08T02:11:42.109478+0000","last_unstale":"2023-10-08T02:11:42.114760+0000","last_undegraded":"2023-10-08T02:11:42.114760+0000","last_fullsized":"2023-10-08T02:11:42.114760+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:32:40.681432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.125349+0000","last_change":"2023-10-08T02:11:41.087828+0000","last_active":"2023-10-08T02:11:42.125349+0000","last_peered":"2023-10-08T02:11:42.125349+0000","last_clean":"2023-10-08T02:11:42.125349+0000","last_became_active":"2023-10-08T02:11:41.087533+0000","last_became_peered":"2023-10-08T02:11:41.087533+0000","last_unstale":"2023-10-08T02:11:42.125349+0000","last_undegraded":"2023-10-08T02:11:42.125349+0000","last_fullsized":"2023-10-08T02:11:42.125349+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:33:55.157660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.101659+0000","last_change":"2023-10-08T02:11:42.098444+0000","last_active":"2023-10-08T02:11:42.101659+0000","last_peered":"2023-10-08T02:11:42.101659+0000","last_clean":"2023-10-08T02:11:42.101659+0000","last_became_active":"2023-10-08T02:11:42.098168+0000","last_became_peered":"2023-10-08T02:11:42.098168+0000","last_unstale":"2023-10-08T02:11:42.101659+0000","last_undegraded":"2023-10-08T02:11:42.101659+0000","last_fullsized":"2023-10-08T02:11:42.101659+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:01:12.133057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"157'1","reported_seq":11,"reported_epoch":157,"state":"active+clean","last_fresh":"2023-10-08T02:11:44.221880+0000","last_change":"2023-10-08T02:11:41.083886+0000","last_active":"2023-10-08T02:11:44.221880+0000","last_peered":"2023-10-08T02:11:44.221880+0000","last_clean":"2023-10-08T02:11:44.221880+0000","last_became_active":"2023-10-08T02:11:41.083613+0000","last_became_peered":"2023-10-08T02:11:41.083613+0000","last_unstale":"2023-10-08T02:11:44.221880+0000","last_undegraded":"2023-10-08T02:11:44.221880+0000","last_fullsized":"2023-10-08T02:11:44.221880+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:59:33.763959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112978+0000","last_change":"2023-10-08T02:11:42.111324+0000","last_active":"2023-10-08T02:11:42.112978+0000","last_peered":"2023-10-08T02:11:42.112978+0000","last_clean":"2023-10-08T02:11:42.112978+0000","last_became_active":"2023-10-08T02:11:42.111081+0000","last_became_peered":"2023-10-08T02:11:42.111081+0000","last_unstale":"2023-10-08T02:11:42.112978+0000","last_undegraded":"2023-10-08T02:11:42.112978+0000","last_fullsized":"2023-10-08T02:11:42.112978+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:19:26.503280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.099015+0000","last_change":"2023-10-08T02:11:41.090739+0000","last_active":"2023-10-08T02:11:42.099015+0000","last_peered":"2023-10-08T02:11:42.099015+0000","last_clean":"2023-10-08T02:11:42.099015+0000","last_became_active":"2023-10-08T02:11:41.090496+0000","last_became_peered":"2023-10-08T02:11:41.090496+0000","last_unstale":"2023-10-08T02:11:42.099015+0000","last_undegraded":"2023-10-08T02:11:42.099015+0000","last_fullsized":"2023-10-08T02:11:42.099015+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:22.244415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.114322+0000","last_change":"2023-10-08T02:11:42.109238+0000","last_active":"2023-10-08T02:11:42.114322+0000","last_peered":"2023-10-08T02:11:42.114322+0000","last_clean":"2023-10-08T02:11:42.114322+0000","last_became_active":"2023-10-08T02:11:42.108826+0000","last_became_peered":"2023-10-08T02:11:42.108826+0000","last_unstale":"2023-10-08T02:11:42.114322+0000","last_undegraded":"2023-10-08T02:11:42.114322+0000","last_fullsized":"2023-10-08T02:11:42.114322+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:07.980822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086700+0000","last_change":"2023-10-08T02:11:41.091434+0000","last_active":"2023-10-08T02:11:42.086700+0000","last_peered":"2023-10-08T02:11:42.086700+0000","last_clean":"2023-10-08T02:11:42.086700+0000","last_became_active":"2023-10-08T02:11:41.091153+0000","last_became_peered":"2023-10-08T02:11:41.091153+0000","last_unstale":"2023-10-08T02:11:42.086700+0000","last_undegraded":"2023-10-08T02:11:42.086700+0000","last_fullsized":"2023-10-08T02:11:42.086700+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:44:42.218801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.116417+0000","last_change":"2023-10-08T02:11:42.113410+0000","last_active":"2023-10-08T02:11:42.116417+0000","last_peered":"2023-10-08T02:11:42.116417+0000","last_clean":"2023-10-08T02:11:42.116417+0000","last_became_active":"2023-10-08T02:11:42.113122+0000","last_became_peered":"2023-10-08T02:11:42.113122+0000","last_unstale":"2023-10-08T02:11:42.116417+0000","last_undegraded":"2023-10-08T02:11:42.116417+0000","last_fullsized":"2023-10-08T02:11:42.116417+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:24.684270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.098192+0000","last_change":"2023-10-08T02:11:41.089067+0000","last_active":"2023-10-08T02:11:42.098192+0000","last_peered":"2023-10-08T02:11:42.098192+0000","last_clean":"2023-10-08T02:11:42.098192+0000","last_became_active":"2023-10-08T02:11:41.088808+0000","last_became_peered":"2023-10-08T02:11:41.088808+0000","last_unstale":"2023-10-08T02:11:42.098192+0000","last_undegraded":"2023-10-08T02:11:42.098192+0000","last_fullsized":"2023-10-08T02:11:42.098192+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:48:39.872955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.112267+0000","last_change":"2023-10-08T02:11:42.107010+0000","last_active":"2023-10-08T02:11:42.112267+0000","last_peered":"2023-10-08T02:11:42.112267+0000","last_clean":"2023-10-08T02:11:42.112267+0000","last_became_active":"2023-10-08T02:11:42.106730+0000","last_became_peered":"2023-10-08T02:11:42.106730+0000","last_unstale":"2023-10-08T02:11:42.112267+0000","last_undegraded":"2023-10-08T02:11:42.112267+0000","last_fullsized":"2023-10-08T02:11:42.112267+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:51:02.949267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.090599+0000","last_change":"2023-10-08T02:11:41.085060+0000","last_active":"2023-10-08T02:11:42.090599+0000","last_peered":"2023-10-08T02:11:42.090599+0000","last_clean":"2023-10-08T02:11:42.090599+0000","last_became_active":"2023-10-08T02:11:41.084795+0000","last_became_peered":"2023-10-08T02:11:41.084795+0000","last_unstale":"2023-10-08T02:11:42.090599+0000","last_undegraded":"2023-10-08T02:11:42.090599+0000","last_fullsized":"2023-10-08T02:11:42.090599+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:59:00.998019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.107239+0000","last_change":"2023-10-08T02:11:42.102214+0000","last_active":"2023-10-08T02:11:42.107239+0000","last_peered":"2023-10-08T02:11:42.107239+0000","last_clean":"2023-10-08T02:11:42.107239+0000","last_became_active":"2023-10-08T02:11:42.101961+0000","last_became_peered":"2023-10-08T02:11:42.101961+0000","last_unstale":"2023-10-08T02:11:42.107239+0000","last_undegraded":"2023-10-08T02:11:42.107239+0000","last_fullsized":"2023-10-08T02:11:42.107239+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:23:53.821167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.091432+0000","last_change":"2023-10-08T02:11:41.090616+0000","last_active":"2023-10-08T02:11:42.091432+0000","last_peered":"2023-10-08T02:11:42.091432+0000","last_clean":"2023-10-08T02:11:42.091432+0000","last_became_active":"2023-10-08T02:11:41.090325+0000","last_became_peered":"2023-10-08T02:11:41.090325+0000","last_unstale":"2023-10-08T02:11:42.091432+0000","last_undegraded":"2023-10-08T02:11:42.091432+0000","last_fullsized":"2023-10-08T02:11:42.091432+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:42:38.736526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.113008+0000","last_change":"2023-10-08T02:11:42.111921+0000","last_active":"2023-10-08T02:11:42.113008+0000","last_peered":"2023-10-08T02:11:42.113008+0000","last_clean":"2023-10-08T02:11:42.113008+0000","last_became_active":"2023-10-08T02:11:42.111678+0000","last_became_peered":"2023-10-08T02:11:42.111678+0000","last_unstale":"2023-10-08T02:11:42.113008+0000","last_undegraded":"2023-10-08T02:11:42.113008+0000","last_fullsized":"2023-10-08T02:11:42.113008+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:25:03.282910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.086209+0000","last_change":"2023-10-08T02:11:41.084308+0000","last_active":"2023-10-08T02:11:42.086209+0000","last_peered":"2023-10-08T02:11:42.086209+0000","last_clean":"2023-10-08T02:11:42.086209+0000","last_became_active":"2023-10-08T02:11:41.084031+0000","last_became_peered":"2023-10-08T02:11:41.084031+0000","last_unstale":"2023-10-08T02:11:42.086209+0000","last_undegraded":"2023-10-08T02:11:42.086209+0000","last_fullsized":"2023-10-08T02:11:42.086209+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:53:13.398792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.109876+0000","last_change":"2023-10-08T02:11:42.102399+0000","last_active":"2023-10-08T02:11:42.109876+0000","last_peered":"2023-10-08T02:11:42.109876+0000","last_clean":"2023-10-08T02:11:42.109876+0000","last_became_active":"2023-10-08T02:11:42.102130+0000","last_became_peered":"2023-10-08T02:11:42.102130+0000","last_unstale":"2023-10-08T02:11:42.109876+0000","last_undegraded":"2023-10-08T02:11:42.109876+0000","last_fullsized":"2023-10-08T02:11:42.109876+0000","mapping_epoch":155,"log_start":"0'0","ondisk_log_start":"0'0","created":155,"last_epoch_clean":156,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:41.063653+0000","last_clean_scrub_stamp":"2023-10-08T02:11:41.063653+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:33:21.328506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"156'2","reported_seq":12,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.122295+0000","last_change":"2023-10-08T02:11:41.094140+0000","last_active":"2023-10-08T02:11:42.122295+0000","last_peered":"2023-10-08T02:11:42.122295+0000","last_clean":"2023-10-08T02:11:42.122295+0000","last_became_active":"2023-10-08T02:11:41.093877+0000","last_became_peered":"2023-10-08T02:11:41.093877+0000","last_unstale":"2023-10-08T02:11:42.122295+0000","last_undegraded":"2023-10-08T02:11:42.122295+0000","last_fullsized":"2023-10-08T02:11:42.122295+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:11:40.058018+0000","last_clean_scrub_stamp":"2023-10-08T02:11:40.058018+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:01:28.606962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"146'271","reported_seq":823,"reported_epoch":156,"state":"active+clean","last_fresh":"2023-10-08T02:11:42.090620+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:11:42.090620+0000","last_peered":"2023-10-08T02:11:42.090620+0000","last_clean":"2023-10-08T02:11:42.090620+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:11:42.090620+0000","last_undegraded":"2023-10-08T02:11:42.090620+0000","last_fullsized":"2023-10-08T02:11:42.090620+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1460,"num_read_kb":2984,"num_write":439,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":271,"ondisk_log_size":271,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575754,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307992,"kb_used_data":2512,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063848,"statfs":{"total":96636764160,"available":96321380352,"internally_reserved":0,"allocated":2572288,"data_stored":1564765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73399999999999999,"15min":0.73899999999999999},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.95099999999999996,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.76600000000000001,"15min":0.754},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.44400000000000001},"max":{"1min":0.96499999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.54300000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.60899999999999999,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.435},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.755,"15min":0.74199999999999999},"min":{"1min":0.621,"5min":0.53100000000000003,"15min":0.48899999999999999},"max":{"1min":0.999,"5min":0.999,"15min":1.0580000000000001},"last":0.41099999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.76600000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.505},"max":{"1min":0.93899999999999995,"5min":0.98999999999999999,"15min":1.0600000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78400000000000003,"15min":0.76400000000000001},"min":{"1min":0.55900000000000005,"5min":0.53800000000000003,"15min":0.48999999999999999},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.67500000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.747,"15min":0.73499999999999999},"min":{"1min":0.59299999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.85699999999999998,"5min":0.82799999999999996,"15min":0.80100000000000005},"min":{"1min":0.66400000000000003,"5min":0.59999999999999998,"15min":0.57399999999999995},"max":{"1min":1.502,"5min":1.502,"15min":1.502},"last":0.56799999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.60099999999999998,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.752,"5min":0.94999999999999996,"15min":0.98399999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.627},"min":{"1min":0.43099999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.496}]},{"osd":5,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64500000000000002,"15min":0.63500000000000001},"min":{"1min":0.44600000000000001,"5min":0.42999999999999999,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.82599999999999996,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.72599999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76800000000000002,"15min":0.76900000000000002},"min":{"1min":0.57299999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.92400000000000004,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.48999999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.76200000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575754,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315592,"kb_used_data":1920,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056248,"statfs":{"total":96636764160,"available":96313597952,"internally_reserved":0,"allocated":1966080,"data_stored":973787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.76500000000000001,"15min":0.74399999999999999},"min":{"1min":0.53900000000000003,"5min":0.52900000000000003,"15min":0.41199999999999998},"max":{"1min":1.569,"5min":1.569,"15min":1.569},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77200000000000002,"15min":0.748},"min":{"1min":0.54900000000000004,"5min":0.501,"15min":0.46400000000000002},"max":{"1min":1.4139999999999999,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.73799999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.79100000000000004,"15min":0.76000000000000001},"min":{"1min":0.51200000000000001,"5min":0.48599999999999999,"15min":0.47599999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.77500000000000002,"15min":0.75600000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.48899999999999999},"max":{"1min":1.464,"5min":1.472,"15min":1.472},"last":0.61699999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.73499999999999999},"min":{"1min":0.60099999999999998,"5min":0.53500000000000003,"15min":0.47699999999999998},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.79700000000000004,"15min":0.76600000000000001},"min":{"1min":0.56000000000000005,"5min":0.51200000000000001,"15min":0.436},"max":{"1min":1.603,"5min":1.603,"15min":1.603},"last":0.77600000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81599999999999995,"15min":0.79800000000000004},"min":{"1min":0.60099999999999998,"5min":0.57999999999999996,"15min":0.52800000000000002},"max":{"1min":1.1299999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.80100000000000005,"15min":0.78500000000000003},"min":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.56299999999999994},"max":{"1min":1.048,"5min":1.169,"15min":1.169},"last":0.871}]},{"osd":4,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.60299999999999998},"min":{"1min":0.439,"5min":0.38700000000000001,"15min":0.38400000000000001},"max":{"1min":1.0129999999999999,"5min":1.133,"15min":1.133},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63800000000000001,"15min":0.62},"min":{"1min":0.48299999999999998,"5min":0.44700000000000001,"15min":0.40500000000000003},"max":{"1min":0.90200000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.78700000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.85099999999999998,"5min":1.0680000000000001,"15min":1.0680000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.70399999999999996,"15min":0.68200000000000005},"min":{"1min":0.54800000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.95599999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.81100000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:10:54 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.752,"15min":0.72099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0289999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.56799999999999995,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.98099999999999998,"5min":1.232,"15min":1.232},"last":0.90200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575754,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307388,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064452,"statfs":{"total":96636764160,"available":96321998848,"internally_reserved":0,"allocated":1953792,"data_stored":972251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.603,"15min":1.603},"last":0.28799999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.497,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.60199999999999998,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.36699999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.751,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.70799999999999996,"5min":3.698,"15min":3.698},"last":0.35899999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86399999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95999999999999996,"5min":1.734,"15min":1.734},"last":0.69199999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.53600000000000003},"max":{"1min":1.0489999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91600000000000004,"5min":1.403,"15min":1.403},"last":0.65400000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.61299999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0149999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.573,"15min":1.573},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.875,"15min":0.86499999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.58999999999999997},"max":{"1min":0.99399999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.76700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575753,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307968,"kb_used_data":2488,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063872,"statfs":{"total":96636764160,"available":96321404928,"internally_reserved":0,"allocated":2547712,"data_stored":1562619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.48499999999999999,"15min":0.46999999999999997},"min":{"1min":0.36199999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.82399999999999995,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.51800000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.35699999999999998,"15min":0.308},"max":{"1min":0.66800000000000004,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.28799999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.496,"15min":0.48699999999999999},"min":{"1min":0.38,"5min":0.34300000000000003,"15min":0.317},"max":{"1min":0.59699999999999998,"5min":0.70699999999999996,"15min":0.746},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57099999999999995,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.37},"max":{"1min":0.69099999999999995,"5min":0.88300000000000001,"15min":1.325},"last":0.36199999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58199999999999996,"15min":0.56599999999999995},"min":{"1min":0.44600000000000001,"5min":0.40200000000000002,"15min":0.371},"max":{"1min":0.873,"5min":0.875,"15min":0.875},"last":0.502},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61699999999999999,"15min":0.61299999999999999},"min":{"1min":0.54300000000000004,"5min":0.47399999999999998,"15min":0.45900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.42299999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.79000000000000004,"15min":0.78700000000000003},"min":{"1min":0.70899999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.24},"last":0.629},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.80200000000000005,"15min":0.79500000000000004},"min":{"1min":0.621,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.3759999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.879}]},{"osd":5,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.94799999999999995,"5min":1.327,"15min":1.327},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.89700000000000002,"15min":0.88200000000000001},"min":{"1min":0.754,"5min":0.60999999999999999,"15min":0.56100000000000005},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":0.91800000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.85399999999999998,"15min":0.83899999999999997},"min":{"1min":0.66100000000000003,"5min":0.55000000000000004,"15min":0.53400000000000003},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.81299999999999994,"15min":0.80500000000000005},"min":{"1min":0.61099999999999999,"5min":0.60099999999999998,"15min":0.55500000000000005},"max":{"1min":0.90900000000000003,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:11:08 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.875,"15min":0.84999999999999998},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.55800000000000005},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.90000000000000002,"15min":0.88300000000000001},"min":{"1min":0.70699999999999996,"5min":0.63400000000000001,"15min":0.61099999999999999},"max":{"1min":1.5269999999999999,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.90600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575753,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311492,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060348,"statfs":{"total":96636764160,"available":96317796352,"internally_reserved":0,"allocated":1961984,"data_stored":972839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48699999999999999,"15min":0.47999999999999998},"min":{"1min":0.33800000000000002,"5min":0.32300000000000001,"15min":0.30199999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.40400000000000003,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":1.228,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.499}]},{"osd":1,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.501,"15min":0.498},"min":{"1min":0.36199999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.68000000000000005,"5min":0.79500000000000004,"15min":0.872},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.41799999999999998,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63400000000000001,"15min":0.629},"min":{"1min":0.46800000000000003,"5min":0.45500000000000002,"15min":0.39400000000000002},"max":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.59699999999999998,"15min":0.58899999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":1.4179999999999999,"5min":1.4179999999999999,"15min":1.4179999999999999},"last":0.46899999999999997}]},{"osd":4,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.82699999999999996,"15min":0.81100000000000005},"min":{"1min":0.58999999999999997,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.8560000000000001,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.84999999999999998,"15min":0.82099999999999995},"min":{"1min":0.58599999999999997,"5min":0.57699999999999996,"15min":0.53400000000000003},"max":{"1min":3.004,"5min":3.004,"15min":3.004},"last":0.65700000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.92300000000000004,"15min":0.871},"min":{"1min":0.749,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.89400000000000002,"15min":0.86399999999999999},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6220000000000001,"5min":1.847,"15min":1.847},"last":0.67300000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.878,"15min":0.83899999999999997},"min":{"1min":0.58499999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.7,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.86399999999999999,"15min":0.84499999999999997},"min":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.77400000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:11:13 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":0.90300000000000002,"15min":0.86199999999999999},"min":{"1min":0.61299999999999999,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":2.899,"5min":2.899,"15min":2.899},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.90700000000000003,"15min":0.86599999999999999},"min":{"1min":0.64100000000000001,"5min":0.622,"15min":0.58899999999999997},"max":{"1min":1.7909999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.63400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575753,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315588,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056252,"statfs":{"total":96636764160,"available":96313602048,"internally_reserved":0,"allocated":1961984,"data_stored":972411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.46500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33100000000000002,"15min":0.31},"max":{"1min":0.69999999999999996,"5min":0.72199999999999998,"15min":0.87},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51600000000000001,"15min":0.51500000000000001},"min":{"1min":0.38800000000000001,"5min":0.36499999999999999,"15min":0.32600000000000001},"max":{"1min":0.627,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.38100000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48599999999999999,"15min":0.48499999999999999},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.56000000000000005,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.55300000000000005,"15min":0.54100000000000004},"min":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.68300000000000005,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.49299999999999999,"5min":0.42299999999999999,"15min":0.41999999999999998},"max":{"1min":0.70999999999999996,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64400000000000002,"15min":0.64300000000000002},"min":{"1min":0.53000000000000003,"5min":0.46600000000000003,"15min":0.45600000000000002},"max":{"1min":0.75800000000000001,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.497}]},{"osd":4,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":0.79300000000000004},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.214},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.80900000000000005,"15min":0.81100000000000005},"min":{"1min":0.65400000000000003,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":1.0589999999999999,"5min":1.0680000000000001,"15min":1.1599999999999999},"last":0.58099999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.79900000000000004,"15min":0.80600000000000005},"min":{"1min":0.59799999999999998,"5min":0.58699999999999997,"15min":0.54500000000000004},"max":{"1min":1.0209999999999999,"5min":1.0589999999999999,"15min":1.125},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.81299999999999994},"min":{"1min":0.627,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.3560000000000001,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.68300000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.82299999999999995,"15min":0.82499999999999996},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.56899999999999995},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.745},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.83699999999999997,"15min":0.82699999999999996},"min":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.60599999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:11:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.85199999999999998,"15min":0.83799999999999997},"min":{"1min":0.59299999999999997,"5min":0.59099999999999997,"15min":0.58499999999999996},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.85499999999999998,"15min":0.84499999999999997},"min":{"1min":0.67800000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.55900000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575753,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311488,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060352,"statfs":{"total":96636764160,"available":96317800448,"internally_reserved":0,"allocated":1957888,"data_stored":972321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40400000000000003},"max":{"1min":0.90100000000000002,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0009999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.73299999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.52},"last":0.71899999999999997}]},{"osd":2,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.629},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60999999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90800000000000003,"5min":1.1579999999999999,"15min":1.403},"last":0.85399999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.208,"15min":1.5069999999999999},"last":0.755}]},{"osd":5,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":1.05,"15min":1.05},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93400000000000005,"5min":1.038,"15min":1.1559999999999999},"last":0.78900000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.016},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.65000000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.0680000000000001,"15min":1.129},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.48799999999999999},"max":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834575753,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311492,"kb_used_data":1916,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060348,"statfs":{"total":96636764160,"available":96317796352,"internally_reserved":0,"allocated":1961984,"data_stored":972363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.752,"15min":0.73899999999999999},"min":{"1min":0.44,"5min":0.41999999999999998,"15min":0.38900000000000001},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.75800000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.438},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.84299999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.80400000000000005,"15min":0.78000000000000003},"min":{"1min":0.63200000000000001,"5min":0.54200000000000004,"15min":0.42699999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.753,"15min":0.74099999999999999},"min":{"1min":0.57899999999999996,"5min":0.53000000000000003,"15min":0.46200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.94799999999999995}]},{"osd":2,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.80600000000000005,"15min":0.78100000000000003},"min":{"1min":0.60399999999999998,"5min":0.55600000000000005,"15min":0.499},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.77700000000000002,"15min":0.76200000000000001},"min":{"1min":0.61599999999999999,"5min":0.57499999999999996,"15min":0.50900000000000001},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.97699999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81699999999999995,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.56100000000000005,"15min":0.497},"max":{"1min":0.96099999999999997,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.98699999999999999,"5min":1.0529999999999999,"15min":1.091},"last":0.85699999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.627,"15min":0.62},"min":{"1min":0.39100000000000001,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.91600000000000004,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.67500000000000004,"15min":0.67100000000000004},"min":{"1min":0.47699999999999998,"5min":0.45100000000000001,"15min":0.441},"max":{"1min":0.95399999999999996,"5min":1.0620000000000001,"15min":1.115},"last":0.66600000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.67700000000000005,"15min":0.68200000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":0.93999999999999995,"5min":1.006,"15min":1.0629999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52000000000000002,"5min":0.436,"15min":0.41399999999999998},"max":{"1min":0.88,"5min":0.95299999999999996,"15min":1.077},"last":0.82799999999999996}]},{"osd":7,"last update":"Sun Oct 8 02:11:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.54300000000000004,"5min":0.51800000000000002,"15min":0.47999999999999998},"max":{"1min":0.98899999999999999,"5min":1.0249999999999999,"15min":1.161},"last":0.747},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.76300000000000001,"15min":0.76500000000000001},"min":{"1min":0.53300000000000003,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.997,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.76200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:11:52.457 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T02:11:52.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T02:11:52.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.679+0000 7f0f3a7ff700 1 Processor -- start 2023-10-08T02:11:52.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.679+0000 7f0f3a7ff700 1 -- start start 2023-10-08T02:11:52.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f3a7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157630 0x7f0f34158a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f3a7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f34157a00 0x7f0f340abd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f3a7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f340ac290 0x7f0f34068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f3a7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f34158590 con 0x7f0f34157a00 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f3a7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f34157f80 con 0x7f0f340ac290 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f3a7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f340ac7a0 con 0x7f0f34157630 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f2bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f34157a00 0x7f0f340abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f2bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f34157a00 0x7f0f340abd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50884/0 (socket says 172.21.15.119:50884) 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f2bfff700 1 -- 172.21.15.119:0/1992031656 learned_addr learned my addr 172.21.15.119:0/1992031656 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f33fff700 1 --2- 172.21.15.119:0/1992031656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157630 0x7f0f34158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f38d9c700 1 --2- 172.21.15.119:0/1992031656 >> v2:172.21.15.184:3300/0 conn(0x7f0f340ac290 0x7f0f34068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.681+0000 7f0f2bfff700 1 -- 172.21.15.119:0/1992031656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157630 msgr2=0x7f0f34158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f2bfff700 1 --2- 172.21.15.119:0/1992031656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157630 0x7f0f34158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f2bfff700 1 -- 172.21.15.119:0/1992031656 >> v2:172.21.15.184:3300/0 conn(0x7f0f340ac290 msgr2=0x7f0f34068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f2bfff700 1 --2- 172.21.15.119:0/1992031656 >> v2:172.21.15.184:3300/0 conn(0x7f0f340ac290 0x7f0f34068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f2bfff700 1 -- 172.21.15.119:0/1992031656 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f34158220 con 0x7f0f34157a00 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f38d9c700 1 --2- 172.21.15.119:0/1992031656 >> v2:172.21.15.184:3300/0 conn(0x7f0f340ac290 0x7f0f34068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f2bfff700 1 --2- 172.21.15.119:0/1992031656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f34157a00 0x7f0f340abd50 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0f340c3500 tx=0x7f0f1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32d357fe571f6280 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f337fe700 1 -- 172.21.15.119:0/1992031656 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c009e70 con 0x7f0f34157a00 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f337fe700 1 -- 172.21.15.119:0/1992031656 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c00b860 con 0x7f0f34157a00 2023-10-08T02:11:52.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.682+0000 7f0f337fe700 1 -- 172.21.15.119:0/1992031656 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c00bc00 con 0x7f0f34157a00 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/1992031656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f34157a00 msgr2=0x7f0f340abd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/1992031656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f34157a00 0x7f0f340abd50 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f0f340c3500 tx=0x7f0f1c009250 comp rx=0 tx=0).stop 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/1992031656 shutdown_connections 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/1992031656 >> v2:172.21.15.184:3300/0 conn(0x7f0f340ac290 0x7f0f34068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/1992031656 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157630 0x7f0f34158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/1992031656 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f34157a00 0x7f0f340abd50 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/1992031656 >> 172.21.15.119:0/1992031656 conn(0x7f0f340cc540 msgr2=0x7f0f340cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/1992031656 shutdown_connections 2023-10-08T02:11:52.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.683+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/1992031656 wait complete. 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.684+0000 7f0f3a7ff700 1 Processor -- start 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.684+0000 7f0f3a7ff700 1 -- start start 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f3a7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340ac290 0x7f0f341538c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f3a7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34157630 0x7f0f3406e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f3a7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157a00 0x7f0f3406ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f3a7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34153e00 con 0x7f0f340ac290 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f3a7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34154220 con 0x7f0f34157630 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f3a7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f34154500 con 0x7f0f34157a00 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340ac290 0x7f0f341538c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f33fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340ac290 0x7f0f341538c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50898/0 (socket says 172.21.15.119:50898) 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.685+0000 7f0f33fff700 1 -- 172.21.15.119:0/2121219134 learned_addr learned my addr 172.21.15.119:0/2121219134 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f33fff700 1 -- 172.21.15.119:0/2121219134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157a00 msgr2=0x7f0f3406ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f33fff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157a00 0x7f0f3406ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f33fff700 1 -- 172.21.15.119:0/2121219134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34157630 msgr2=0x7f0f3406e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f33fff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34157630 0x7f0f3406e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f33fff700 1 -- 172.21.15.119:0/2121219134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c0094f0 con 0x7f0f340ac290 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f33fff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340ac290 0x7f0f341538c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400ec80 tx=0x7f0f240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f30ff9700 1 -- 172.21.15.119:0/2121219134 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400c9d0 con 0x7f0f340ac290 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f30ff9700 1 -- 172.21.15.119:0/2121219134 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2400cb30 con 0x7f0f340ac290 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f30ff9700 1 -- 172.21.15.119:0/2121219134 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f24013470 con 0x7f0f340ac290 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f341547e0 con 0x7f0f340ac290 2023-10-08T02:11:52.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.686+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f34154da0 con 0x7f0f340ac290 2023-10-08T02:11:52.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.687+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f340718b0 con 0x7f0f340ac290 2023-10-08T02:11:52.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.688+0000 7f0f30ff9700 1 -- 172.21.15.119:0/2121219134 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f240138a0 con 0x7f0f340ac290 2023-10-08T02:11:52.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.689+0000 7f0f30ff9700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f14066f30 0x7f0f140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:52.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.689+0000 7f0f30ff9700 1 -- 172.21.15.119:0/2121219134 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f0f24084ae0 con 0x7f0f340ac290 2023-10-08T02:11:52.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.689+0000 7f0f2bfff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f14066f30 0x7f0f140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:52.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.690+0000 7f0f2bfff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f14066f30 0x7f0f140693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f340c3500 tx=0x7f0f1c017000 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:52.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.693+0000 7f0f30ff9700 1 -- 172.21.15.119:0/2121219134 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f24051ae0 con 0x7f0f340ac290 2023-10-08T02:11:52.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.958+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0f340aca80 con 0x7f0f340ac290 2023-10-08T02:11:52.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.963+0000 7f0f30ff9700 1 -- 172.21.15.119:0/2121219134 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v43) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0f24055100 con 0x7f0f340ac290 2023-10-08T02:11:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f14066f30 msgr2=0x7f0f140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f14066f30 0x7f0f140693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0f340c3500 tx=0x7f0f1c017000 comp rx=0 tx=0).stop 2023-10-08T02:11:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340ac290 msgr2=0x7f0f341538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340ac290 0x7f0f341538c0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f0f2400ec80 tx=0x7f0f240091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 shutdown_connections 2023-10-08T02:11:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f14066f30 0x7f0f140693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f34157a00 0x7f0f3406ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f340ac290 0x7f0f341538c0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 --2- 172.21.15.119:0/2121219134 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f34157630 0x7f0f3406e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:52.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 >> 172.21.15.119:0/2121219134 conn(0x7f0f340cc540 msgr2=0x7f0f341568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 shutdown_connections 2023-10-08T02:11:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:52.965+0000 7f0f3a7ff700 1 -- 172.21.15.119:0/2121219134 wait complete. 2023-10-08T02:11:52.967 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T02:11:52.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:11:53.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.164+0000 7f15b4f9d700 1 Processor -- start 2023-10-08T02:11:53.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.164+0000 7f15b4f9d700 1 -- start start 2023-10-08T02:11:53.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15b4f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:53.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15b4f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f15b00cb900 0x7f15b00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:53.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15b4f9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b00ca860 con 0x7f15b0156310 2023-10-08T02:11:53.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15b4f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b00cab40 con 0x7f15b00cb900 2023-10-08T02:11:53.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15b4f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00cae20 con 0x7f15b01566e0 2023-10-08T02:11:53.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:53.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15affff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f15b00cb900 0x7f15b00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:53.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:47092/0 (socket says 172.21.15.119:47092) 2023-10-08T02:11:53.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15a7fff700 1 -- 172.21.15.119:0/2602918952 learned_addr learned my addr 172.21.15.119:0/2602918952 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:53.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.166+0000 7f15af7fe700 1 --1- 172.21.15.119:0/2602918952 >> v1:172.21.15.184:6789/0 conn(0x7f15b0156310 0x7f15b00c9dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:42556/0 (socket says 172.21.15.119:42556) 2023-10-08T02:11:53.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15aeffd700 1 -- 172.21.15.119:0/2602918952 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871624143 0 0) 0x7f15b00ca860 con 0x7f15b0156310 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15aeffd700 1 -- 172.21.15.119:0/2602918952 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f159c003570 con 0x7f15b0156310 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15affff700 1 -- 172.21.15.119:0/2602918952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 msgr2=0x7f15b00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15affff700 1 --2- 172.21.15.119:0/2602918952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15affff700 1 -- 172.21.15.119:0/2602918952 >> v1:172.21.15.184:6789/0 conn(0x7f15b0156310 legacy=0x7f15b00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15affff700 1 -- 172.21.15.119:0/2602918952 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b00cb100 con 0x7f15b00cb900 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15affff700 1 --2- 172.21.15.119:0/2602918952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 0x7f15b00ae540 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f15a800d0d0 tx=0x7f15a800b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65ccbbb8b13f5d7f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15aeffd700 1 -- 172.21.15.119:0/2602918952 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8010070 con 0x7f15b00cb900 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15aeffd700 1 -- 172.21.15.119:0/2602918952 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a800b2e0 con 0x7f15b00cb900 2023-10-08T02:11:53.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.167+0000 7f15aeffd700 1 -- 172.21.15.119:0/2602918952 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800dcd0 con 0x7f15b00cb900 2023-10-08T02:11:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.168+0000 7f15b4f9d700 1 -- 172.21.15.119:0/2602918952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 msgr2=0x7f15b00ae540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:53.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.168+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/2602918952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 0x7f15b00ae540 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f15a800d0d0 tx=0x7f15a800b990 comp rx=0 tx=0).stop 2023-10-08T02:11:53.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.168+0000 7f15b4f9d700 1 -- 172.21.15.119:0/2602918952 shutdown_connections 2023-10-08T02:11:53.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.168+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/2602918952 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 0x7f15b00ae540 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.168+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/2602918952 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.168+0000 7f15b4f9d700 1 -- 172.21.15.119:0/2602918952 >> 172.21.15.119:0/2602918952 conn(0x7f15b00cc1f0 msgr2=0x7f15b00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:53.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.168+0000 7f15b4f9d700 1 -- 172.21.15.119:0/2602918952 shutdown_connections 2023-10-08T02:11:53.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.168+0000 7f15b4f9d700 1 -- 172.21.15.119:0/2602918952 wait complete. 2023-10-08T02:11:53.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.169+0000 7f15b4f9d700 1 Processor -- start 2023-10-08T02:11:53.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.170+0000 7f15b4f9d700 1 -- start start 2023-10-08T02:11:53.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.170+0000 7f15b4f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 0x7f15b00c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:53.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.170+0000 7f15b4f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0156310 0x7f15b0143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.170+0000 7f15b4f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b0154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15b4f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0144240 con 0x7f15b0156310 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15b4f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00c8b50 con 0x7f15b00cb900 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15b4f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00c8e30 con 0x7f15b01566e0 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0156310 0x7f15b0143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0156310 0x7f15b0143d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50900/0 (socket says 172.21.15.119:50900) 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 -- 172.21.15.119:0/132842672 learned_addr learned my addr 172.21.15.119:0/132842672 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15affff700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b0154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15af7fe700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 0x7f15b00c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 -- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 msgr2=0x7f15b0154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:53.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b0154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 -- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 msgr2=0x7f15b00c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 0x7f15b00c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 -- 172.21.15.119:0/132842672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a800bc30 con 0x7f15b0156310 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15affff700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b0154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15af7fe700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 0x7f15b00c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.171+0000 7f15a7fff700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0156310 0x7f15b0143d00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f159800e900 tx=0x7f1598009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.172+0000 7f15a77fe700 1 -- 172.21.15.119:0/132842672 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800eea0 con 0x7f15b0156310 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.172+0000 7f15a77fe700 1 -- 172.21.15.119:0/132842672 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800c850 con 0x7f15b0156310 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.172+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b00c9110 con 0x7f15b0156310 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.172+0000 7f15a77fe700 1 -- 172.21.15.119:0/132842672 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159800cbf0 con 0x7f15b0156310 2023-10-08T02:11:53.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.172+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b00c96d0 con 0x7f15b0156310 2023-10-08T02:11:53.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.173+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b004c3c0 con 0x7f15b0156310 2023-10-08T02:11:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.174+0000 7f15a77fe700 1 -- 172.21.15.119:0/132842672 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1598013470 con 0x7f15b0156310 2023-10-08T02:11:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.175+0000 7f15a77fe700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1590066f30 0x7f15900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.175+0000 7f15a77fe700 1 -- 172.21.15.119:0/132842672 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f15980849a0 con 0x7f15b0156310 2023-10-08T02:11:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.175+0000 7f15af7fe700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1590066f30 0x7f15900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:53.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.175+0000 7f15af7fe700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1590066f30 0x7f15900693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15b0152750 tx=0x7f15a0009040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:53.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.179+0000 7f15a77fe700 1 -- 172.21.15.119:0/132842672 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f15980519a0 con 0x7f15b0156310 2023-10-08T02:11:53.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.412+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f15b00cd0b0 con 0x7f15b0156310 2023-10-08T02:11:53.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.412+0000 7f15a77fe700 1 -- 172.21.15.119:0/132842672 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f1598017090 con 0x7f15b0156310 2023-10-08T02:11:53.412 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:11:53.412 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/603902131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":603902131},{"type":"v1","addr":"172.21.15.184:6834","nonce":603902131}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3930162267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3930162267},{"type":"v1","addr":"172.21.15.119:6834","nonce":3930162267}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/237096421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":237096421},{"type":"v1","addr":"172.21.15.119:6835","nonce":237096421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:11:41.545410+0000","modified":"2023-10-08T02:11:45.111313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8667":{"gid":8667,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3810992025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3810992025},{"type":"v1","addr":"172.21.15.184:6835","nonce":3810992025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T02:11:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1590066f30 msgr2=0x7f15900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1590066f30 0x7f15900693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f15b0152750 tx=0x7f15a0009040 comp rx=0 tx=0).stop 2023-10-08T02:11:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0156310 msgr2=0x7f15b0143d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0156310 0x7f15b0143d00 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f159800e900 tx=0x7f1598009250 comp rx=0 tx=0).stop 2023-10-08T02:11:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 shutdown_connections 2023-10-08T02:11:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1590066f30 0x7f15900693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f15b01566e0 0x7f15b0154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f15b00cb900 0x7f15b00c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 --2- 172.21.15.119:0/132842672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15b0156310 0x7f15b0143d00 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 >> 172.21.15.119:0/132842672 conn(0x7f15b00cc1f0 msgr2=0x7f15b00b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 shutdown_connections 2023-10-08T02:11:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:53.415+0000 7f15b4f9d700 1 -- 172.21.15.119:0/132842672 wait complete. 2023-10-08T02:11:53.416 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 71 2023-10-08T02:11:53.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T02:11:41.545410+0000', 'modified': '2023-10-08T02:11:45.111313+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8667}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8667': {'gid': 8667, 'name': 'd', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6835/3810992025', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6833', 'nonce': 3810992025}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 3810992025}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T02:11:53.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T02:11:53.835 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T02:11:54.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.017+0000 7fd5f57ff700 1 Processor -- start 2023-10-08T02:11:54.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.017+0000 7fd5f57ff700 1 -- start start 2023-10-08T02:11:54.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.018+0000 7fd5f57ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5f0159890 0x7fd5f015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.018+0000 7fd5f57ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd5f0159c60 0x7fd5f015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.018+0000 7fd5f57ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5f015a160 con 0x7fd5f015db10 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.018+0000 7fd5f57ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f015df50 con 0x7fd5f0159890 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.018+0000 7fd5f57ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f00d13a0 con 0x7fd5f0159c60 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.018+0000 7fd5eeffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5f0159890 0x7fd5f015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.018+0000 7fd5ef7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd5f015db10 0x7fd5f0069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55760/0 (socket says 172.21.15.119:55760) 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.018+0000 7fd5ef7fe700 1 -- 172.21.15.119:0/861115129 learned_addr learned my addr 172.21.15.119:0/861115129 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.019+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/861115129 >> v2:172.21.15.184:3300/0 conn(0x7fd5f0159c60 0x7fd5f015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.019+0000 7fd5eeffd700 1 -- 172.21.15.119:0/861115129 >> v2:172.21.15.184:3300/0 conn(0x7fd5f0159c60 msgr2=0x7fd5f015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.019+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/861115129 >> v2:172.21.15.184:3300/0 conn(0x7fd5f0159c60 0x7fd5f015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.019+0000 7fd5eeffd700 1 -- 172.21.15.119:0/861115129 >> v1:172.21.15.119:6789/0 conn(0x7fd5f015db10 legacy=0x7fd5f0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.019+0000 7fd5edffb700 1 -- 172.21.15.119:0/861115129 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930235509 0 0) 0x7fd5f015a160 con 0x7fd5f015db10 2023-10-08T02:11:54.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.019+0000 7fd5eeffd700 1 -- 172.21.15.119:0/861115129 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f015e140 con 0x7fd5f0159890 2023-10-08T02:11:54.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.019+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/861115129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f015abd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00b67f0 tx=0x7fd5e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4122ac8b12524a97 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.019+0000 7fd5edffb700 1 -- 172.21.15.119:0/861115129 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0011630 con 0x7fd5f0159890 2023-10-08T02:11:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5edffb700 1 -- 172.21.15.119:0/861115129 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0011790 con 0x7fd5f0159890 2023-10-08T02:11:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5edffb700 1 -- 172.21.15.119:0/861115129 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0011b30 con 0x7fd5f0159890 2023-10-08T02:11:54.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5f57ff700 1 -- 172.21.15.119:0/861115129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 msgr2=0x7fd5f015abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/861115129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f015abd0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00b67f0 tx=0x7fd5e00091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5f57ff700 1 -- 172.21.15.119:0/861115129 shutdown_connections 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/861115129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f015abd0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/861115129 >> v2:172.21.15.184:3300/0 conn(0x7fd5f0159c60 0x7fd5f015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5f57ff700 1 -- 172.21.15.119:0/861115129 >> 172.21.15.119:0/861115129 conn(0x7fd5f0068e20 msgr2=0x7fd5f00d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5f57ff700 1 -- 172.21.15.119:0/861115129 shutdown_connections 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.020+0000 7fd5f57ff700 1 -- 172.21.15.119:0/861115129 wait complete. 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.022+0000 7fd5f57ff700 1 Processor -- start 2023-10-08T02:11:54.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.022+0000 7fd5f57ff700 1 -- start start 2023-10-08T02:11:54.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.022+0000 7fd5f57ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f00ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5f57ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0159c60 0x7fd5f016a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5f57ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f015db10 0x7fd5f0152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5f57ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00cf3c0 con 0x7fd5f0159890 2023-10-08T02:11:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5f57ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00cf6a0 con 0x7fd5f015db10 2023-10-08T02:11:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5f57ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f00cf980 con 0x7fd5f0159c60 2023-10-08T02:11:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f00ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f00ceac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50922/0 (socket says 172.21.15.119:50922) 2023-10-08T02:11:54.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 -- 172.21.15.119:0/2016507295 learned_addr learned my addr 172.21.15.119:0/2016507295 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5ef7fe700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f015db10 0x7fd5f0152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0159c60 0x7fd5f016a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 -- 172.21.15.119:0/2016507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0159c60 msgr2=0x7fd5f016a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0159c60 0x7fd5f016a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 -- 172.21.15.119:0/2016507295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f015db10 msgr2=0x7fd5f0152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f015db10 0x7fd5f0152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 -- 172.21.15.119:0/2016507295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e0009420 con 0x7fd5f0159890 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.023+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f00ceac0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0070e20 tx=0x7fd5e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.024+0000 7fd5d6ffd700 1 -- 172.21.15.119:0/2016507295 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0011b10 con 0x7fd5f0159890 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.024+0000 7fd5d6ffd700 1 -- 172.21.15.119:0/2016507295 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e00253f0 con 0x7fd5f0159890 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.024+0000 7fd5f57ff700 1 -- 172.21.15.119:0/2016507295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f00cfc60 con 0x7fd5f0159890 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.024+0000 7fd5d6ffd700 1 -- 172.21.15.119:0/2016507295 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0025790 con 0x7fd5f0159890 2023-10-08T02:11:54.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.024+0000 7fd5f57ff700 1 -- 172.21.15.119:0/2016507295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f016a710 con 0x7fd5f0159890 2023-10-08T02:11:54.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.026+0000 7fd5d6ffd700 1 -- 172.21.15.119:0/2016507295 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd5e0025b80 con 0x7fd5f0159890 2023-10-08T02:11:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.027+0000 7fd5d6ffd700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5dc066f30 0x7fd5dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.027+0000 7fd5d6ffd700 1 -- 172.21.15.119:0/2016507295 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd5e00860b0 con 0x7fd5f0159890 2023-10-08T02:11:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.027+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5dc066f30 0x7fd5dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.027+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5dc066f30 0x7fd5dc0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd5f01513b0 tx=0x7fd5e4008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.029+0000 7fd603f91b80 1 Processor -- start 2023-10-08T02:11:54.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.029+0000 7fd603f91b80 1 -- start start 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2cae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d2f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d2c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd603f91b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419d221a70 con 0x56419d314d00 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd603f91b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419d21f270 con 0x56419d2f3f60 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd603f91b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56419d21b060 con 0x56419d2caa50 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ee7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d2f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ee7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d2f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50926/0 (socket says 172.21.15.119:50926) 2023-10-08T02:11:54.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ee7fc700 1 -- 172.21.15.119:0/468047146 learned_addr learned my addr 172.21.15.119:0/468047146 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ef7fe700 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d2c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ee7fc700 1 -- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 msgr2=0x56419d2cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ee7fc700 1 -- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 msgr2=0x56419d2c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d2c6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ee7fc700 1 -- 172.21.15.119:0/468047146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56419d0b5270 con 0x56419d314d00 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.030+0000 7fd5ef7fe700 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d2c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:54.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.031+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d2f3a20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4011d80 tx=0x7fd5e4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d2b4413f27639d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.031+0000 7fd5cffff700 1 -- 172.21.15.119:0/468047146 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4011e60 con 0x56419d314d00 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.031+0000 7fd5cffff700 1 -- 172.21.15.119:0/468047146 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e4015c60 con 0x56419d314d00 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.031+0000 7fd5cffff700 1 -- 172.21.15.119:0/468047146 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4021440 con 0x56419d314d00 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.031+0000 7fd603f91b80 1 -- 172.21.15.119:0/468047146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 msgr2=0x56419d2f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.031+0000 7fd603f91b80 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d2f3a20 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4011d80 tx=0x7fd5e4018000 comp rx=0 tx=0).stop 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.032+0000 7fd603f91b80 1 -- 172.21.15.119:0/468047146 shutdown_connections 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.032+0000 7fd603f91b80 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.032+0000 7fd603f91b80 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d2c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.032+0000 7fd603f91b80 1 --2- 172.21.15.119:0/468047146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d2f3a20 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.032+0000 7fd603f91b80 1 -- 172.21.15.119:0/468047146 >> 172.21.15.119:0/468047146 conn(0x56419d2cc370 msgr2=0x56419d2cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:54.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.032+0000 7fd603f91b80 1 -- 172.21.15.119:0/468047146 shutdown_connections 2023-10-08T02:11:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.032+0000 7fd603f91b80 1 -- 172.21.15.119:0/468047146 wait complete. 2023-10-08T02:11:54.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.032+0000 7fd603f91b80 1 -- start start 2023-10-08T02:11:54.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.033+0000 7fd603f91b80 1 Processor -- start 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2eff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d3110f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d30c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd603f91b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419d1c6890 con 0x56419d314d00 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd603f91b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419d1e1ea0 con 0x56419d2f3f60 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd603f91b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56419d09de30 con 0x56419d2caa50 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd603f91b80 10 client.0 _ll_register_callbacks cb 0x56419d28e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d30c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ef7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d30c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50934/0 (socket says 172.21.15.119:50934) 2023-10-08T02:11:54.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ef7fe700 1 -- 172.21.15.119:0/1594024600 learned_addr learned my addr 172.21.15.119:0/1594024600 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d3110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2eff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ef7fe700 1 -- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 msgr2=0x56419d2eff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ef7fe700 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2eff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ef7fe700 1 -- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 msgr2=0x56419d3110f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ef7fe700 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d3110f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.034+0000 7fd5ef7fe700 1 -- 172.21.15.119:0/1594024600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e40119c0 con 0x56419d314d00 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd5ef7fe700 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d30c1d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80073d0 tx=0x7fd5e8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569a73fa1201e0f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd5cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/1594024600 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e8013640 con 0x56419d314d00 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/1594024600 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd5e80137a0 con 0x56419d314d00 2023-10-08T02:11:54.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd603f91b80 1 -- 172.21.15.119:0/1594024600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56419d2bfb70 con 0x56419d314d00 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/1594024600 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e8013b70 con 0x56419d314d00 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd603f91b80 10 client.8835 fetch_fsmap learned FSMap version 71 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd603f91b80 1 -- 172.21.15.119:0/1594024600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x56419d057fe0 con 0x56419d314d00 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/1594024600 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56419d2bfb70 con 0x56419d314d00 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.035+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/1594024600 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 71) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd5e8013cd0 con 0x56419d314d00 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.036+0000 7fd603f91b80 10 client.8835 fetch_fsmap finished waiting for FSMap version 71 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.036+0000 7fd603f91b80 10 client.8835 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.036+0000 7fd603f91b80 20 client.8835 populate_metadata read hostname 'smithi119' 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.036+0000 7fd603f91b80 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] conn(0x56419d311630 0x56419d30aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.036+0000 7fd603f91b80 4 client.8835 mds_command: new command op to 8667 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.036+0000 7fd603f91b80 1 -- 172.21.15.119:0/1594024600 --> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56419d29caa0 con 0x56419d311630 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.036+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] conn(0x56419d311630 0x56419d30aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.037+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] conn(0x56419d311630 0x56419d30aef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56715a9ffe257184 server_cookie=78d8200b06e0a861 in_seq=0 out_seq=0 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.037+0000 7fd5cd7fa700 10 client.8835 ms_handle_connect on v2:172.21.15.184:6833/3810992025 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.038+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/1594024600 <== mds.0 v2:172.21.15.184:6833/3810992025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x56419d29caa0 con 0x56419d311630 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.038+0000 7fd5cd7fa700 10 client.8835 handle_command_reply: tid=0 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.038+0000 7fd603f91b80 1 client.8835 shutdown 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 -- 172.21.15.119:0/1594024600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 msgr2=0x56419d30c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d30c1d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd5e80073d0 tx=0x7fd5e8009040 comp rx=0 tx=0).stop 2023-10-08T02:11:54.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 -- 172.21.15.119:0/1594024600 shutdown_connections 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] conn(0x56419d311630 0x56419d30aef0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d2eff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d314d00 0x56419d30c1d0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd5cd7fa700 0 client.8835 ms_handle_reset on v2:172.21.15.184:6833/3810992025 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 --2- 172.21.15.119:0/1594024600 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d2f3f60 0x56419d3110f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 -- 172.21.15.119:0/1594024600 >> 172.21.15.119:0/1594024600 conn(0x56419d2cc370 msgr2=0x56419d2c7850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 -- 172.21.15.119:0/1594024600 shutdown_connections 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 1 -- 172.21.15.119:0/1594024600 wait complete. 2023-10-08T02:11:54.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.039+0000 7fd603f91b80 20 client.8835 trim_cache size 0 max 16384 2023-10-08T02:11:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.056+0000 7fd603f91b80 1 Processor -- start 2023-10-08T02:11:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.056+0000 7fd603f91b80 1 -- start start 2023-10-08T02:11:54.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.057+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2caa50 0x56419d311080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.057+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2f3f60 0x56419d2c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.057+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 0x56419d312080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.057+0000 7fd603f91b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419d30f4e0 con 0x56419d2caa50 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.057+0000 7fd603f91b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419d312990 con 0x56419d314d00 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.057+0000 7fd603f91b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56419d2a3410 con 0x56419d2f3f60 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.057+0000 7fd5eeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2caa50 0x56419d311080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5eeffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2caa50 0x56419d311080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50938/0 (socket says 172.21.15.119:50938) 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5eeffd700 1 -- 172.21.15.119:0/387774796 learned_addr learned my addr 172.21.15.119:0/387774796 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2f3f60 0x56419d2c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5ef7fe700 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 0x56419d312080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5eeffd700 1 -- 172.21.15.119:0/387774796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2f3f60 msgr2=0x56419d2c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2f3f60 0x56419d2c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5eeffd700 1 -- 172.21.15.119:0/387774796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 msgr2=0x56419d312080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 0x56419d312080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5eeffd700 1 -- 172.21.15.119:0/387774796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56419d1e9780 con 0x56419d2caa50 2023-10-08T02:11:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.058+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2caa50 0x56419d311080 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00091e0 tx=0x7fd5e0030040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4075275297e028 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd5d4ff9700 1 -- 172.21.15.119:0/387774796 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e006d1d0 con 0x56419d2caa50 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd5d4ff9700 1 -- 172.21.15.119:0/387774796 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5e0056d70 con 0x56419d2caa50 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd5d4ff9700 1 -- 172.21.15.119:0/387774796 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0059fc0 con 0x56419d2caa50 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd603f91b80 1 -- 172.21.15.119:0/387774796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2caa50 msgr2=0x56419d311080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd603f91b80 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2caa50 0x56419d311080 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd5e00091e0 tx=0x7fd5e0030040 comp rx=0 tx=0).stop 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd603f91b80 1 -- 172.21.15.119:0/387774796 shutdown_connections 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd603f91b80 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 0x56419d312080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd603f91b80 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2caa50 0x56419d311080 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd603f91b80 1 --2- 172.21.15.119:0/387774796 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2f3f60 0x56419d2c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd603f91b80 1 -- 172.21.15.119:0/387774796 >> 172.21.15.119:0/387774796 conn(0x56419d2cc370 msgr2=0x56419d2c34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.059+0000 7fd603f91b80 1 -- 172.21.15.119:0/387774796 shutdown_connections 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.060+0000 7fd603f91b80 1 -- 172.21.15.119:0/387774796 wait complete. 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.060+0000 7fd603f91b80 1 -- start start 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.060+0000 7fd603f91b80 1 Processor -- start 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.061+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d35fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.061+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2f3f60 0x56419d32a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.061+0000 7fd603f91b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 0x56419d32ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.061+0000 7fd603f91b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419d32b3d0 con 0x56419d2f3f60 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.061+0000 7fd603f91b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x56419d307cd0 con 0x56419d314d00 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.061+0000 7fd603f91b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x56419d31b680 con 0x56419d2caa50 2023-10-08T02:11:54.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.061+0000 7fd603f91b80 10 client.0 _ll_register_callbacks cb 0x56419d28e110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T02:11:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2f3f60 0x56419d32a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2f3f60 0x56419d32a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50946/0 (socket says 172.21.15.119:50946) 2023-10-08T02:11:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 -- 172.21.15.119:0/17356730 learned_addr learned my addr 172.21.15.119:0/17356730 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d35fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ef7fe700 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 0x56419d32ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 -- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 msgr2=0x56419d35fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d35fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 -- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 msgr2=0x56419d32ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 0x56419d32ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 -- 172.21.15.119:0/17356730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f016a710 con 0x56419d2f3f60 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5ee7fc700 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2f3f60 0x56419d32a950 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4008010 tx=0x7fd5e401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d01b50e622679c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.062+0000 7fd5cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/17356730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4011e60 con 0x56419d2f3f60 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/17356730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd5e4015dd0 con 0x56419d2f3f60 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 1 -- 172.21.15.119:0/17356730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd5f00cddd0 con 0x56419d2f3f60 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/17356730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5e402b610 con 0x56419d2f3f60 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 10 client.8841 fetch_fsmap learned FSMap version 71 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 1 -- 172.21.15.119:0/17356730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x56419d1d0470 con 0x56419d2f3f60 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/17356730 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f00cddd0 con 0x56419d2f3f60 2023-10-08T02:11:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/17356730 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 71) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fd5e4036070 con 0x56419d2f3f60 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 10 client.8841 fetch_fsmap finished waiting for FSMap version 71 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 10 client.8841 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 20 client.8841 populate_metadata read hostname 'smithi119' 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] conn(0x56419d2fa6e0 0x56419d2faab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 4 client.8841 mds_command: new command op to 8667 tid=0[{"prefix": "status"}] 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.063+0000 7fd603f91b80 1 -- 172.21.15.119:0/17356730 --> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fd5e4015c60 con 0x56419d2fa6e0 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.064+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] conn(0x56419d2fa6e0 0x56419d2faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.064+0000 7fd5cd7fa700 10 client.8841 ms_handle_connect on v2:172.21.15.184:6833/3810992025 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.064+0000 7fd5eeffd700 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] conn(0x56419d2fa6e0 0x56419d2faab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdac81d59a51e034 server_cookie=f9b23da1e3138917 in_seq=0 out_seq=0 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd5cd7fa700 1 -- 172.21.15.119:0/17356730 <== mds.0 v2:172.21.15.184:6833/3810992025 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fd5e4015c60 con 0x56419d2fa6e0 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd5cd7fa700 10 client.8841 handle_command_reply: tid=0 2023-10-08T02:11:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 client.8841 shutdown 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 -- 172.21.15.119:0/17356730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2f3f60 msgr2=0x56419d32a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2f3f60 0x56419d32a950 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4008010 tx=0x7fd5e401a000 comp rx=0 tx=0).stop 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 -- 172.21.15.119:0/17356730 shutdown_connections 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:6833/3810992025,v1:172.21.15.184:6835/3810992025] conn(0x56419d2fa6e0 0x56419d2faab0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x56419d2caa50 0x56419d35fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x56419d314d00 0x56419d32ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd5cd7fa700 0 client.8841 ms_handle_reset on v2:172.21.15.184:6833/3810992025 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 --2- 172.21.15.119:0/17356730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x56419d2f3f60 0x56419d32a950 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 -- 172.21.15.119:0/17356730 >> 172.21.15.119:0/17356730 conn(0x56419d2cc370 msgr2=0x56419d30ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 -- 172.21.15.119:0/17356730 shutdown_connections 2023-10-08T02:11:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 1 -- 172.21.15.119:0/17356730 wait complete. 2023-10-08T02:11:54.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.065+0000 7fd603f91b80 20 client.8841 trim_cache size 0 max 16384 2023-10-08T02:11:54.071 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:11:54.071 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T02:11:54.071 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T02:11:54.071 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 8667, 2023-10-08T02:11:54.071 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T02:11:54.071 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T02:11:54.071 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T02:11:54.072 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 11.970370054, 2023-10-08T02:11:54.072 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 71, 2023-10-08T02:11:54.072 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 157, 2023-10-08T02:11:54.072 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 157, 2023-10-08T02:11:54.072 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 18.196289586999999 2023-10-08T02:11:54.072 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 -- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5dc066f30 msgr2=0x7fd5dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5dc066f30 0x7fd5dc0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd5f01513b0 tx=0x7fd5e4008040 comp rx=0 tx=0).stop 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 -- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 msgr2=0x7fd5f00ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f00ceac0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd5f0070e20 tx=0x7fd5e0017000 comp rx=0 tx=0).stop 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 -- 172.21.15.119:0/2016507295 shutdown_connections 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd5dc066f30 0x7fd5dc0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd5f015db10 0x7fd5f0152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5f0159890 0x7fd5f00ceac0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 --2- 172.21.15.119:0/2016507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd5f0159c60 0x7fd5f016a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 -- 172.21.15.119:0/2016507295 >> 172.21.15.119:0/2016507295 conn(0x7fd5f0068e20 msgr2=0x7fd5f01584a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:54.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 -- 172.21.15.119:0/2016507295 shutdown_connections 2023-10-08T02:11:54.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.066+0000 7fd5f57ff700 1 -- 172.21.15.119:0/2016507295 wait complete. 2023-10-08T02:11:54.082 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T02:11:54.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.264+0000 7fd8b37ff700 1 Processor -- start 2023-10-08T02:11:54.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.264+0000 7fd8b37ff700 1 -- start start 2023-10-08T02:11:54.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.265+0000 7fd8b37ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8ac0cbf30 0x7fd8ac0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.265+0000 7fd8b37ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ac0cc300 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.265+0000 7fd8b37ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ac0ad370 con 0x7fd8ac0b1420 2023-10-08T02:11:54.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.265+0000 7fd8b37ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ac156bf0 con 0x7fd8ac0cbf30 2023-10-08T02:11:54.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.266+0000 7fd8b159b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8ac0cbf30 0x7fd8ac0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.266+0000 7fd8b0d9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd8ac0ccf90 0x7fd8ac0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:55776/0 (socket says 172.21.15.119:55776) 2023-10-08T02:11:54.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.266+0000 7fd8b0d9a700 1 -- 172.21.15.119:0/2625792252 learned_addr learned my addr 172.21.15.119:0/2625792252 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:54.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.266+0000 7fd8b159b700 1 -- 172.21.15.119:0/2625792252 >> v1:172.21.15.184:6789/0 conn(0x7fd8ac0b1420 legacy=0x7fd8ac0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.266+0000 7fd8b159b700 1 -- 172.21.15.119:0/2625792252 >> v1:172.21.15.119:6789/0 conn(0x7fd8ac0ccf90 legacy=0x7fd8ac0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.266+0000 7fd8b159b700 1 -- 172.21.15.119:0/2625792252 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac0ad650 con 0x7fd8ac0cbf30 2023-10-08T02:11:54.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.266+0000 7fd8a3fff700 1 -- 172.21.15.119:0/2625792252 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825365464 0 0) 0x7fd8ac0cc300 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.266+0000 7fd8b159b700 1 --2- 172.21.15.119:0/2625792252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0cbf30 0x7fd8ac0ae410 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80098e0 tx=0x7fd8a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6315d428d3e1f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8a3fff700 1 -- 172.21.15.119:0/2625792252 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8009e80 con 0x7fd8ac0cbf30 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8a3fff700 1 -- 172.21.15.119:0/2625792252 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a800b860 con 0x7fd8ac0cbf30 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8a3fff700 1 -- 172.21.15.119:0/2625792252 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800bbe0 con 0x7fd8ac0cbf30 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2625792252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0cbf30 msgr2=0x7fd8ac0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8b37ff700 1 --2- 172.21.15.119:0/2625792252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0cbf30 0x7fd8ac0ae410 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80098e0 tx=0x7fd8a80091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2625792252 shutdown_connections 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8b37ff700 1 --2- 172.21.15.119:0/2625792252 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0cbf30 0x7fd8ac0ae410 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2625792252 >> 172.21.15.119:0/2625792252 conn(0x7fd8ac06a320 msgr2=0x7fd8ac0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2625792252 shutdown_connections 2023-10-08T02:11:54.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.267+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2625792252 wait complete. 2023-10-08T02:11:54.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.269+0000 7fd8b37ff700 1 Processor -- start 2023-10-08T02:11:54.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.269+0000 7fd8b37ff700 1 -- start start 2023-10-08T02:11:54.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b37ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ac0b1420 0x7fd8ac169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b37ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ac0cbf30 0x7fd8ac158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b37ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0ccf90 0x7fd8ac159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b37ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac16b270 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b37ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac16b580 con 0x7fd8ac0b1420 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b37ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac16b860 con 0x7fd8ac0cbf30 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b1d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0ccf90 0x7fd8ac159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b1d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0ccf90 0x7fd8ac159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50970/0 (socket says 172.21.15.119:50970) 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b1d9c700 1 -- 172.21.15.119:0/2219700648 learned_addr learned my addr 172.21.15.119:0/2219700648 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b159b700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ac0b1420 0x7fd8ac169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b0d9a700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ac0cbf30 0x7fd8ac158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b1d9c700 1 -- 172.21.15.119:0/2219700648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ac0cbf30 msgr2=0x7fd8ac158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b1d9c700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ac0cbf30 0x7fd8ac158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b1d9c700 1 -- 172.21.15.119:0/2219700648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ac0b1420 msgr2=0x7fd8ac169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b1d9c700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ac0b1420 0x7fd8ac169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.270+0000 7fd8b1d9c700 1 -- 172.21.15.119:0/2219700648 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a8009420 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.271+0000 7fd8b1d9c700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0ccf90 0x7fd8ac159280 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c8e0 tx=0x7fd8a4001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.271+0000 7fd8a17fa700 1 -- 172.21.15.119:0/2219700648 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4012630 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.271+0000 7fd8a17fa700 1 -- 172.21.15.119:0/2219700648 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a4012790 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.271+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac16bb40 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.271+0000 7fd8a17fa700 1 -- 172.21.15.119:0/2219700648 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a4012b50 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.271+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac169f50 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.272+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac16c060 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.273+0000 7fd8a17fa700 1 -- 172.21.15.119:0/2219700648 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8a401a3a0 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.274+0000 7fd8a17fa700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd890066f30 0x7fd8900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:54.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.274+0000 7fd8a17fa700 1 -- 172.21.15.119:0/2219700648 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd8a40848e0 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.274+0000 7fd8b159b700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd890066f30 0x7fd8900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:54.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.274+0000 7fd8b159b700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd890066f30 0x7fd8900693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80093f0 tx=0x7fd8a8017000 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:54.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.278+0000 7fd8a17fa700 1 -- 172.21.15.119:0/2219700648 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8ac16c060 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.509+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd8ac16c060 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.510+0000 7fd8a17fa700 1 -- 172.21.15.119:0/2219700648 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd8ac16c060 con 0x7fd8ac0ccf90 2023-10-08T02:11:54.510 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T02:11:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd890066f30 msgr2=0x7fd8900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd890066f30 0x7fd8900693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80093f0 tx=0x7fd8a8017000 comp rx=0 tx=0).stop 2023-10-08T02:11:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0ccf90 msgr2=0x7fd8ac159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0ccf90 0x7fd8ac159280 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400c8e0 tx=0x7fd8a4001710 comp rx=0 tx=0).stop 2023-10-08T02:11:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 shutdown_connections 2023-10-08T02:11:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd890066f30 0x7fd8900693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8ac0ccf90 0x7fd8ac159280 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8ac0b1420 0x7fd8ac169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 --2- 172.21.15.119:0/2219700648 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8ac0cbf30 0x7fd8ac158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 >> 172.21.15.119:0/2219700648 conn(0x7fd8ac06a320 msgr2=0x7fd8ac0ca3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 shutdown_connections 2023-10-08T02:11:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:54.512+0000 7fd8b37ff700 1 -- 172.21.15.119:0/2219700648 wait complete. 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T02:11:54.530 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:11:54.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:11:54.531 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T02:11:54.567 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T02:11:54.567 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T02:11:54.568 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 37737sec preferred_lft 37737sec 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:11:54.569 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T02:11:54.570 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:11:54.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:11:54.571 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T02:11:54.666 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T02:11:54.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:11:54.669 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T02:11:54.707 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T02:11:54.707 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T02:11:54.707 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T02:11:54.707 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T02:11:54.708 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T02:11:54.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:11:54.709 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T02:11:54.709 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T02:11:54.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T02:11:54.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T02:11:54.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T02:11:54.709 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T02:11:54.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:11:54.902 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T02:11:54.938 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T02:11:54.938 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T02:11:54.938 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:11:54.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:11:54.939 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T02:11:55.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T02:11:55.040 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T02:11:55.076 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:11:55.076 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T02:11:55.076 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T02:11:55.152 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T02:11:55.152 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T02:11:55.152 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:11:55.283 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:11:55.283 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T02:11:55.312 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:11:55.313 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T02:11:55.313 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T02:11:55.313 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T02:11:55.313 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T02:11:55.317 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T02:11:55.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:11:55.317 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T02:11:55.379 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T02:11:55.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.560+0000 7f3329f9d700 1 Processor -- start 2023-10-08T02:11:55.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.560+0000 7f3329f9d700 1 -- start start 2023-10-08T02:11:55.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.561+0000 7f3329f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f33241593b0 0x7f3324159780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:55.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.561+0000 7f3329f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33240cfe20 con 0x7f33240b1e50 2023-10-08T02:11:55.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.561+0000 7f3329f9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33240cf210 con 0x7f33240ceaa0 2023-10-08T02:11:55.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.561+0000 7f3329f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33240cf4f0 con 0x7f33241593b0 2023-10-08T02:11:55.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.561+0000 7f332979c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f33240b1e50 0x7f33240b3420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42210/0 (socket says 172.21.15.119:42210) 2023-10-08T02:11:55.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.561+0000 7f332979c700 1 -- 172.21.15.119:0/592884250 learned_addr learned my addr 172.21.15.119:0/592884250 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3307353371 0 0) 0x7f33240cfe20 con 0x7f33240b1e50 2023-10-08T02:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f3328f9b700 1 --2- 172.21.15.119:0/592884250 >> v2:172.21.15.184:3300/0 conn(0x7f33241593b0 0x7f3324159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3314003570 con 0x7f33240b1e50 2023-10-08T02:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 884490546 0 0) 0x7f3314003570 con 0x7f33240b1e50 2023-10-08T02:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 >> v2:172.21.15.184:3300/0 conn(0x7f33241593b0 msgr2=0x7f3324159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 --2- 172.21.15.119:0/592884250 >> v2:172.21.15.184:3300/0 conn(0x7f33241593b0 0x7f3324159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 >> v1:172.21.15.184:6789/0 conn(0x7f33240ceaa0 legacy=0x7f33240b1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33240cf7d0 con 0x7f33240b1e50 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f331c0017f0 con 0x7f33240b1e50 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f3328f9b700 1 --2- 172.21.15.119:0/592884250 >> v2:172.21.15.184:3300/0 conn(0x7f33241593b0 0x7f3324159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 >> v1:172.21.15.119:6789/0 conn(0x7f33240b1e50 legacy=0x7f33240b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3314003a60 0x7f3314003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3314004810 0x7f3314004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140052e0 0x7f331400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33240cf4f0 con 0x7f33140052e0 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331400c230 con 0x7f3314004810 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.562+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331400c3f0 con 0x7f3314003a60 2023-10-08T02:11:55.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f332979c700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140052e0 0x7f331400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f3328f9b700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3314003a60 0x7f3314003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f3323fff700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3314004810 0x7f3314004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f332979c700 1 -- 172.21.15.119:0/592884250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3314003a60 msgr2=0x7f3314003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f332979c700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3314003a60 0x7f3314003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f332979c700 1 -- 172.21.15.119:0/592884250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3314004810 msgr2=0x7f3314004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f332979c700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3314004810 0x7f3314004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f332979c700 1 -- 172.21.15.119:0/592884250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314003570 con 0x7f33140052e0 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f332979c700 1 -- 172.21.15.119:0/592884250 reap_dead start 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f332979c700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140052e0 0x7f331400ba60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f331c00a900 tx=0x7f33240cf3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432b82e02e8dc03f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.563+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c0070d0 con 0x7f33140052e0 2023-10-08T02:11:55.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331c007230 con 0x7f33140052e0 2023-10-08T02:11:55.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f33237fe700 1 -- 172.21.15.119:0/592884250 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331c0075f0 con 0x7f33140052e0 2023-10-08T02:11:55.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f3329f9d700 1 -- 172.21.15.119:0/592884250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140052e0 msgr2=0x7f331400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:55.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f3329f9d700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140052e0 0x7f331400ba60 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f331c00a900 tx=0x7f33240cf3e0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f3329f9d700 1 -- 172.21.15.119:0/592884250 shutdown_connections 2023-10-08T02:11:55.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f3329f9d700 1 --2- 172.21.15.119:0/592884250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33140052e0 0x7f331400ba60 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f3329f9d700 1 -- 172.21.15.119:0/592884250 >> 172.21.15.119:0/592884250 conn(0x7f33240d1130 msgr2=0x7f33240b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:55.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f3329f9d700 1 -- 172.21.15.119:0/592884250 shutdown_connections 2023-10-08T02:11:55.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.564+0000 7f3329f9d700 1 -- 172.21.15.119:0/592884250 wait complete. 2023-10-08T02:11:55.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.566+0000 7f3329f9d700 1 Processor -- start 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.566+0000 7f3329f9d700 1 -- start start 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3329f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33140052e0 0x7f33240cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3329f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240c7280 0x7f3324155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3329f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f332415b790 0x7f332415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3329f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332415c150 con 0x7f33240c7280 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3329f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332416a720 con 0x7f332415b790 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3329f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3324169ba0 con 0x7f33140052e0 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240c7280 0x7f3324155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240c7280 0x7f3324155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56536/0 (socket says 172.21.15.119:56536) 2023-10-08T02:11:55.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 -- 172.21.15.119:0/2126020212 learned_addr learned my addr 172.21.15.119:0/2126020212 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 -- 172.21.15.119:0/2126020212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33140052e0 msgr2=0x7f33240cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33140052e0 0x7f33240cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f332979c700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f332415b790 0x7f332415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 -- 172.21.15.119:0/2126020212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f332415b790 msgr2=0x7f332415bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f332415b790 0x7f332415bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 -- 172.21.15.119:0/2126020212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3324169e80 con 0x7f33240c7280 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f332979c700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f332415b790 0x7f332415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.567+0000 7f3323fff700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240c7280 0x7f3324155d00 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3310002950 tx=0x7f3310007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.568+0000 7f3320ff9700 1 -- 172.21.15.119:0/2126020212 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310002e90 con 0x7f33240c7280 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.568+0000 7f3320ff9700 1 -- 172.21.15.119:0/2126020212 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331000f8d0 con 0x7f33240c7280 2023-10-08T02:11:55.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.568+0000 7f3320ff9700 1 -- 172.21.15.119:0/2126020212 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331000fc90 con 0x7f33240c7280 2023-10-08T02:11:55.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.568+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332416a160 con 0x7f33240c7280 2023-10-08T02:11:55.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.568+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3324156240 con 0x7f33240c7280 2023-10-08T02:11:55.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.569+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f332404c3c0 con 0x7f33240c7280 2023-10-08T02:11:55.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.570+0000 7f3320ff9700 1 -- 172.21.15.119:0/2126020212 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3310006820 con 0x7f33240c7280 2023-10-08T02:11:55.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.571+0000 7f3320ff9700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3308066f30 0x7f33080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:55.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.571+0000 7f3320ff9700 1 -- 172.21.15.119:0/2126020212 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f3310084900 con 0x7f33240c7280 2023-10-08T02:11:55.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.571+0000 7f3328f9b700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3308066f30 0x7f33080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:55.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.572+0000 7f3328f9b700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3308066f30 0x7f33080693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3318006fa0 tx=0x7f3318008040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:55.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.575+0000 7f3320ff9700 1 -- 172.21.15.119:0/2126020212 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3310051900 con 0x7f33240c7280 2023-10-08T02:11:55.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.795+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f33240cfab0 con 0x7f3308066f30 2023-10-08T02:11:55.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.796+0000 7f3320ff9700 1 -- 172.21.15.119:0/2126020212 <== mgr.7822 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f33240cfab0 con 0x7f3308066f30 2023-10-08T02:11:55.797 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T02:11:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:11:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T02:11:55.797 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:11:55.797 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T02:11:55.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.798+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3308066f30 msgr2=0x7f33080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:55.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.798+0000 7f3329f9d700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3308066f30 0x7f33080693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3318006fa0 tx=0x7f3318008040 comp rx=0 tx=0).stop 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.798+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240c7280 msgr2=0x7f3324155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.798+0000 7f3329f9d700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240c7280 0x7f3324155d00 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f3310002950 tx=0x7f3310007d30 comp rx=0 tx=0).stop 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.799+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 shutdown_connections 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.799+0000 7f3329f9d700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3308066f30 0x7f33080693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.799+0000 7f3329f9d700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f33140052e0 0x7f33240cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.799+0000 7f3329f9d700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f332415b790 0x7f332415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.799+0000 7f3329f9d700 1 --2- 172.21.15.119:0/2126020212 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240c7280 0x7f3324155d00 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.799+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 >> 172.21.15.119:0/2126020212 conn(0x7f33240d1130 msgr2=0x7f33240af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.799+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 shutdown_connections 2023-10-08T02:11:55.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.799+0000 7f3329f9d700 1 -- 172.21.15.119:0/2126020212 wait complete. 2023-10-08T02:11:55.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T02:11:56.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.999+0000 7f47e2fff700 1 Processor -- start 2023-10-08T02:11:56.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:55.999+0000 7f47e2fff700 1 -- start start 2023-10-08T02:11:56.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.000+0000 7f47e2fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f47dc0b0a70 0x7f47dc1625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:56.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.000+0000 7f47e2fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc0af2f0 con 0x7f47dc15ac80 2023-10-08T02:11:56.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47e2fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc0af5d0 con 0x7f47dc069e80 2023-10-08T02:11:56.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47e2fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc0c8660 con 0x7f47dc0b0a70 2023-10-08T02:11:56.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47e0d9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f47dc15ac80 0x7f47dc069770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42220/0 (socket says 172.21.15.119:42220) 2023-10-08T02:11:56.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47e0d9b700 1 -- 172.21.15.119:0/3543354100 learned_addr learned my addr 172.21.15.119:0/3543354100 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:56.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47e159c700 1 --2- 172.21.15.119:0/3543354100 >> v2:172.21.15.119:3300/0 conn(0x7f47dc0b0a70 0x7f47dc1625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:56.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47db7fe700 1 -- 172.21.15.119:0/3543354100 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190301941 0 0) 0x7f47dc0af2f0 con 0x7f47dc15ac80 2023-10-08T02:11:56.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47e159c700 1 -- 172.21.15.119:0/3543354100 >> v1:172.21.15.184:6789/0 conn(0x7f47dc069e80 legacy=0x7f47dc0c6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:56.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47e159c700 1 -- 172.21.15.119:0/3543354100 >> v1:172.21.15.119:6789/0 conn(0x7f47dc15ac80 legacy=0x7f47dc069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:56.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.001+0000 7f47e159c700 1 -- 172.21.15.119:0/3543354100 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc0c8940 con 0x7f47dc0b0a70 2023-10-08T02:11:56.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47e159c700 1 --2- 172.21.15.119:0/3543354100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc0b0a70 0x7f47dc1625e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009e60 tx=0x7f47d4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2734ae7f4a861939 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47db7fe700 1 -- 172.21.15.119:0/3543354100 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d400e070 con 0x7f47dc0b0a70 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47db7fe700 1 -- 172.21.15.119:0/3543354100 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d400bc30 con 0x7f47dc0b0a70 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47db7fe700 1 -- 172.21.15.119:0/3543354100 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4012430 con 0x7f47dc0b0a70 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47e2fff700 1 -- 172.21.15.119:0/3543354100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc0b0a70 msgr2=0x7f47dc1625e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47e2fff700 1 --2- 172.21.15.119:0/3543354100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc0b0a70 0x7f47dc1625e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009e60 tx=0x7f47d4009780 comp rx=0 tx=0).stop 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47e2fff700 1 -- 172.21.15.119:0/3543354100 shutdown_connections 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47e2fff700 1 --2- 172.21.15.119:0/3543354100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc0b0a70 0x7f47dc1625e0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47e2fff700 1 -- 172.21.15.119:0/3543354100 >> 172.21.15.119:0/3543354100 conn(0x7f47dc068e10 msgr2=0x7f47dc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47e2fff700 1 -- 172.21.15.119:0/3543354100 shutdown_connections 2023-10-08T02:11:56.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.002+0000 7f47e2fff700 1 -- 172.21.15.119:0/3543354100 wait complete. 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.004+0000 7f47e2fff700 1 Processor -- start 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.004+0000 7f47e2fff700 1 -- start start 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e2fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc069e80 0x7f47dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e2fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47dc0b0a70 0x7f47dc0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e2fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc15ac80 0x7f47dc0b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e2fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc153f30 con 0x7f47dc15ac80 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e2fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc1552f0 con 0x7f47dc069e80 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e2fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc169cd0 con 0x7f47dc0b0a70 2023-10-08T02:11:56.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e159c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc15ac80 0x7f47dc0b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e159c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc15ac80 0x7f47dc0b1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56550/0 (socket says 172.21.15.119:56550) 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e159c700 1 -- 172.21.15.119:0/198330286 learned_addr learned my addr 172.21.15.119:0/198330286 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e0d9b700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc069e80 0x7f47dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47dbfff700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47dc0b0a70 0x7f47dc0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e159c700 1 -- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47dc0b0a70 msgr2=0x7f47dc0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e159c700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47dc0b0a70 0x7f47dc0b0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e159c700 1 -- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc069e80 msgr2=0x7f47dc0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e159c700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc069e80 0x7f47dc0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.005+0000 7f47e159c700 1 -- 172.21.15.119:0/198330286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4009a20 con 0x7f47dc15ac80 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.006+0000 7f47dbfff700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47dc0b0a70 0x7f47dc0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:11:56.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.006+0000 7f47e159c700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc15ac80 0x7f47dc0b1430 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f47dc063d40 tx=0x7f47d4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:56.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.006+0000 7f47d8ff9700 1 -- 172.21.15.119:0/198330286 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d401d630 con 0x7f47dc15ac80 2023-10-08T02:11:56.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.006+0000 7f47d8ff9700 1 -- 172.21.15.119:0/198330286 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d401d7d0 con 0x7f47dc15ac80 2023-10-08T02:11:56.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.006+0000 7f47d8ff9700 1 -- 172.21.15.119:0/198330286 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d401db90 con 0x7f47dc15ac80 2023-10-08T02:11:56.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.006+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc169fb0 con 0x7f47dc15ac80 2023-10-08T02:11:56.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.006+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc16a570 con 0x7f47dc15ac80 2023-10-08T02:11:56.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.007+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc04c3c0 con 0x7f47dc15ac80 2023-10-08T02:11:56.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.008+0000 7f47d8ff9700 1 -- 172.21.15.119:0/198330286 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 78) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f47d4024070 con 0x7f47dc15ac80 2023-10-08T02:11:56.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.009+0000 7f47d8ff9700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f47c0066f30 0x7f47c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:56.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.009+0000 7f47d8ff9700 1 -- 172.21.15.119:0/198330286 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f47d4087240 con 0x7f47dc15ac80 2023-10-08T02:11:56.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.009+0000 7f47e0d9b700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f47c0066f30 0x7f47c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:56.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.010+0000 7f47e0d9b700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f47c0066f30 0x7f47c00693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47d000c8f0 tx=0x7f47d0009040 comp rx=0 tx=0).ready entity=mgr.7822 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:56.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.013+0000 7f47d8ff9700 1 -- 172.21.15.119:0/198330286 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f47d4054240 con 0x7f47dc15ac80 2023-10-08T02:11:56.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:56.259+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f47dc0c8c20 con 0x7f47dc15ac80 2023-10-08T02:11:57.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.171+0000 7f47d8ff9700 1 -- 172.21.15.119:0/198330286 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 79) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f47d4003040 con 0x7f47dc15ac80 2023-10-08T02:11:57.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.172+0000 7f47d8ff9700 1 -- 172.21.15.119:0/198330286 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v79) v1 ==== 92+0+0 (secure 0 0 0) 0x7f47d407fc30 con 0x7f47dc15ac80 2023-10-08T02:11:57.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.175+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f47c0066f30 msgr2=0x7f47c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.175+0000 7f47e2fff700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f47c0066f30 0x7f47c00693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47d000c8f0 tx=0x7f47d0009040 comp rx=0 tx=0).stop 2023-10-08T02:11:57.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.175+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc15ac80 msgr2=0x7f47dc0b1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.175+0000 7f47e2fff700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc15ac80 0x7f47dc0b1430 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f47dc063d40 tx=0x7f47d4019000 comp rx=0 tx=0).stop 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.175+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 shutdown_connections 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.175+0000 7f47e2fff700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f47c0066f30 0x7f47c00693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.175+0000 7f47e2fff700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc15ac80 0x7f47dc0b1430 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.175+0000 7f47e2fff700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f47dc069e80 0x7f47dc0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.176+0000 7f47e2fff700 1 --2- 172.21.15.119:0/198330286 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f47dc0b0a70 0x7f47dc0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.176+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 >> 172.21.15.119:0/198330286 conn(0x7f47dc068e10 msgr2=0x7f47dc15f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.176+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 shutdown_connections 2023-10-08T02:11:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.176+0000 7f47e2fff700 1 -- 172.21.15.119:0/198330286 wait complete. 2023-10-08T02:11:57.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T02:11:57.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.402+0000 7f14b6b9d700 1 Processor -- start 2023-10-08T02:11:57.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.402+0000 7f14b6b9d700 1 -- start start 2023-10-08T02:11:57.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b6b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14b00b4e70 0x7f14b00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b6b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b1b40 0x7f14b015d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b6b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b00cfb00 con 0x7f14b00b16d0 2023-10-08T02:11:57.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b6b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b00b5c40 con 0x7f14b00b4e70 2023-10-08T02:11:57.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b6b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00b5f20 con 0x7f14b00b1b40 2023-10-08T02:11:57.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b5b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14b00b4e70 0x7f14b00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b5b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f14b00b4e70 0x7f14b00ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56574/0 (socket says 172.21.15.119:56574) 2023-10-08T02:11:57.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b5b9b700 1 -- 172.21.15.119:0/2517702000 learned_addr learned my addr 172.21.15.119:0/2517702000 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:57.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b639c700 1 --2- 172.21.15.119:0/2517702000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b1b40 0x7f14b015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b5b9b700 1 -- 172.21.15.119:0/2517702000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b1b40 msgr2=0x7f14b015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:57.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b5b9b700 1 --2- 172.21.15.119:0/2517702000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b1b40 0x7f14b015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.404+0000 7f14b5b9b700 1 -- 172.21.15.119:0/2517702000 >> v1:172.21.15.184:6789/0 conn(0x7f14b00b16d0 legacy=0x7f14b015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.405+0000 7f14b5b9b700 1 -- 172.21.15.119:0/2517702000 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00b6200 con 0x7f14b00b4e70 2023-10-08T02:11:57.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.405+0000 7f14b5b9b700 1 --2- 172.21.15.119:0/2517702000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b4e70 0x7f14b00ccff0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0099b0 tx=0x7f14ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba081fa7eea1615 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:57.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.405+0000 7f14b4b99700 1 -- 172.21.15.119:0/2517702000 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00b860 con 0x7f14b00b4e70 2023-10-08T02:11:57.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.405+0000 7f14b4b99700 1 -- 172.21.15.119:0/2517702000 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac00b9c0 con 0x7f14b00b4e70 2023-10-08T02:11:57.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.405+0000 7f14b4b99700 1 -- 172.21.15.119:0/2517702000 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00bd60 con 0x7f14b00b4e70 2023-10-08T02:11:57.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.406+0000 7f14b6b9d700 1 -- 172.21.15.119:0/2517702000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b4e70 msgr2=0x7f14b00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.406+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/2517702000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b4e70 0x7f14b00ccff0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0099b0 tx=0x7f14ac009250 comp rx=0 tx=0).stop 2023-10-08T02:11:57.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.406+0000 7f14b6b9d700 1 -- 172.21.15.119:0/2517702000 shutdown_connections 2023-10-08T02:11:57.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.406+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/2517702000 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b1b40 0x7f14b015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.406+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/2517702000 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b4e70 0x7f14b00ccff0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.406+0000 7f14b6b9d700 1 -- 172.21.15.119:0/2517702000 >> 172.21.15.119:0/2517702000 conn(0x7f14b00d10c0 msgr2=0x7f14b00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:57.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.406+0000 7f14b6b9d700 1 -- 172.21.15.119:0/2517702000 shutdown_connections 2023-10-08T02:11:57.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.406+0000 7f14b6b9d700 1 -- 172.21.15.119:0/2517702000 wait complete. 2023-10-08T02:11:57.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.407+0000 7f14b6b9d700 1 Processor -- start 2023-10-08T02:11:57.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.407+0000 7f14b6b9d700 1 -- start start 2023-10-08T02:11:57.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.408+0000 7f14b6b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b16d0 0x7f14b0157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.408+0000 7f14b6b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b00b1b40 0x7f14b00c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.408+0000 7f14b6b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b4e70 0x7f14b0068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.408+0000 7f14b6b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b0157c40 con 0x7f14b00b16d0 2023-10-08T02:11:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.408+0000 7f14b6b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b016a240 con 0x7f14b00b1b40 2023-10-08T02:11:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.408+0000 7f14b6b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00ca620 con 0x7f14b00b4e70 2023-10-08T02:11:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b16d0 0x7f14b0157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b16d0 0x7f14b0157700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56578/0 (socket says 172.21.15.119:56578) 2023-10-08T02:11:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 -- 172.21.15.119:0/1313921316 learned_addr learned my addr 172.21.15.119:0/1313921316 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b539a700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b00b1b40 0x7f14b00c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b639c700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b4e70 0x7f14b0068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 -- 172.21.15.119:0/1313921316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b4e70 msgr2=0x7f14b0068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b4e70 0x7f14b0068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 -- 172.21.15.119:0/1313921316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b00b1b40 msgr2=0x7f14b00c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b00b1b40 0x7f14b00c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 -- 172.21.15.119:0/1313921316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0094f0 con 0x7f14b00b16d0 2023-10-08T02:11:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14b5b9b700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b16d0 0x7f14b0157700 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f14b0070dd0 tx=0x7f14ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.409+0000 7f14a5ffb700 1 -- 172.21.15.119:0/1313921316 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac011e40 con 0x7f14b00b16d0 2023-10-08T02:11:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.410+0000 7f14a5ffb700 1 -- 172.21.15.119:0/1313921316 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac024440 con 0x7f14b00b16d0 2023-10-08T02:11:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.410+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b00ca900 con 0x7f14b00b16d0 2023-10-08T02:11:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.410+0000 7f14a5ffb700 1 -- 172.21.15.119:0/1313921316 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac024800 con 0x7f14b00b16d0 2023-10-08T02:11:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.410+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b0169ed0 con 0x7f14b00b16d0 2023-10-08T02:11:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.411+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b0088990 con 0x7f14b00b16d0 2023-10-08T02:11:57.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.412+0000 7f14a5ffb700 1 -- 172.21.15.119:0/1313921316 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 79) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f14ac01a420 con 0x7f14b00b16d0 2023-10-08T02:11:57.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.413+0000 7f14a5ffb700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1494066f80 0x7f1494069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.413+0000 7f14b539a700 1 -- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1494066f80 msgr2=0x7f1494069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:57.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.413+0000 7f14a5ffb700 1 -- 172.21.15.119:0/1313921316 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f14ac0866e0 con 0x7f14b00b16d0 2023-10-08T02:11:57.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.413+0000 7f14b539a700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1494066f80 0x7f1494069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:57.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.417+0000 7f14a5ffb700 1 -- 172.21.15.119:0/1313921316 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14ac0536e0 con 0x7f14b00b16d0 2023-10-08T02:11:57.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.614+0000 7f14b539a700 1 -- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1494066f80 msgr2=0x7f1494069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:57.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.614+0000 7f14b539a700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1494066f80 0x7f1494069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.618+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f14b004bb80 con 0x7f14b00b16d0 2023-10-08T02:11:57.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.625+0000 7f14a5ffb700 1 -- 172.21.15.119:0/1313921316 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7f14ac056d00 con 0x7f14b00b16d0 2023-10-08T02:11:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1494066f80 msgr2=0x7f1494069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1494066f80 0x7f1494069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b16d0 msgr2=0x7f14b0157700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b16d0 0x7f14b0157700 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f14b0070dd0 tx=0x7f14ac017000 comp rx=0 tx=0).stop 2023-10-08T02:11:57.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 shutdown_connections 2023-10-08T02:11:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1494066f80 0x7f1494069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14b00b4e70 0x7f14b0068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14b00b16d0 0x7f14b0157700 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 --2- 172.21.15.119:0/1313921316 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f14b00b1b40 0x7f14b00c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 >> 172.21.15.119:0/1313921316 conn(0x7f14b00d10c0 msgr2=0x7f14b015ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 shutdown_connections 2023-10-08T02:11:57.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.627+0000 7f14b6b9d700 1 -- 172.21.15.119:0/1313921316 wait complete. 2023-10-08T02:11:57.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T02:11:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.835+0000 7fedbcf9d700 1 Processor -- start 2023-10-08T02:11:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.835+0000 7fedbcf9d700 1 -- start start 2023-10-08T02:11:57.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.836+0000 7fedbcf9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fedb80ccc00 0x7fedb8069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.836+0000 7fedbcf9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb80c9f30 con 0x7fedb806a020 2023-10-08T02:11:57.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.836+0000 7fedbcf9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb80ca470 con 0x7fedb8159a60 2023-10-08T02:11:57.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.836+0000 7fedbcf9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb806dfc0 con 0x7fedb80ccc00 2023-10-08T02:11:57.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb7fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fedb806a020 0x7fedb80cb230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42228/0 (socket says 172.21.15.119:42228) 2023-10-08T02:11:57.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb7fff700 1 -- 172.21.15.119:0/3015142923 learned_addr learned my addr 172.21.15.119:0/3015142923 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:11:57.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb6ffd700 1 --2- 172.21.15.119:0/3015142923 >> v2:172.21.15.184:3300/0 conn(0x7fedb80ccc00 0x7fedb8069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb67fc700 1 -- 172.21.15.119:0/3015142923 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27720940 0 0) 0x7fedb80c9f30 con 0x7fedb806a020 2023-10-08T02:11:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb67fc700 1 -- 172.21.15.119:0/3015142923 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feda0003570 con 0x7fedb806a020 2023-10-08T02:11:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb6ffd700 1 -- 172.21.15.119:0/3015142923 >> v1:172.21.15.184:6789/0 conn(0x7fedb8159a60 legacy=0x7fedb80c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb67fc700 1 -- 172.21.15.119:0/3015142923 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180010510 0 0) 0x7fedb80ca470 con 0x7fedb8159a60 2023-10-08T02:11:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb6ffd700 1 -- 172.21.15.119:0/3015142923 >> v1:172.21.15.119:6789/0 conn(0x7fedb806a020 legacy=0x7fedb80cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.837+0000 7fedb6ffd700 1 -- 172.21.15.119:0/3015142923 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb80ca750 con 0x7fedb80ccc00 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.838+0000 7fedb6ffd700 1 --2- 172.21.15.119:0/3015142923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb80ccc00 0x7fedb8069ae0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fedac0098e0 tx=0x7fedac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b6083c0ad52eb0f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.838+0000 7fedb67fc700 1 -- 172.21.15.119:0/3015142923 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac011630 con 0x7fedb80ccc00 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.838+0000 7fedb67fc700 1 -- 172.21.15.119:0/3015142923 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac011790 con 0x7fedb80ccc00 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.838+0000 7fedb67fc700 1 -- 172.21.15.119:0/3015142923 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac011b30 con 0x7fedb80ccc00 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.839+0000 7fedbcf9d700 1 -- 172.21.15.119:0/3015142923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb80ccc00 msgr2=0x7fedb8069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.839+0000 7fedbcf9d700 1 --2- 172.21.15.119:0/3015142923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb80ccc00 0x7fedb8069ae0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fedac0098e0 tx=0x7fedac0091a0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.839+0000 7fedbcf9d700 1 -- 172.21.15.119:0/3015142923 shutdown_connections 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.839+0000 7fedbcf9d700 1 --2- 172.21.15.119:0/3015142923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb80ccc00 0x7fedb8069ae0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.839+0000 7fedbcf9d700 1 -- 172.21.15.119:0/3015142923 >> 172.21.15.119:0/3015142923 conn(0x7fedb806baa0 msgr2=0x7fedb806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:57.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.839+0000 7fedbcf9d700 1 -- 172.21.15.119:0/3015142923 shutdown_connections 2023-10-08T02:11:57.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.839+0000 7fedbcf9d700 1 -- 172.21.15.119:0/3015142923 wait complete. 2023-10-08T02:11:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.841+0000 7fedbcf9d700 1 Processor -- start 2023-10-08T02:11:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.841+0000 7fedbcf9d700 1 -- start start 2023-10-08T02:11:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedbcf9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb806a020 0x7fedb81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedbcf9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedb80ccc00 0x7fedb815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedbcf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fedb8159a60 0x7fedb80c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedbcf9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb815f1d0 con 0x7fedb80ccc00 2023-10-08T02:11:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedbcf9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80c6cc0 con 0x7fedb806a020 2023-10-08T02:11:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedbcf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb80c6f10 con 0x7fedb8159a60 2023-10-08T02:11:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedb80ccc00 0x7fedb815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedb80ccc00 0x7fedb815ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56590/0 (socket says 172.21.15.119:56590) 2023-10-08T02:11:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb6ffd700 1 -- 172.21.15.119:0/4061452401 learned_addr learned my addr 172.21.15.119:0/4061452401 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb77fe700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb806a020 0x7fedb81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb7fff700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fedb8159a60 0x7fedb80c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb6ffd700 1 -- 172.21.15.119:0/4061452401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fedb8159a60 msgr2=0x7fedb80c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb6ffd700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fedb8159a60 0x7fedb80c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb6ffd700 1 -- 172.21.15.119:0/4061452401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb806a020 msgr2=0x7fedb81551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb6ffd700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb806a020 0x7fedb81551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.842+0000 7fedb6ffd700 1 -- 172.21.15.119:0/4061452401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac009420 con 0x7fedb80ccc00 2023-10-08T02:11:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.843+0000 7fedb6ffd700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedb80ccc00 0x7fedb815ec90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fedac0030c0 tx=0x7fedac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.843+0000 7fed9f7fe700 1 -- 172.21.15.119:0/4061452401 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac011900 con 0x7fedb80ccc00 2023-10-08T02:11:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.843+0000 7fed9f7fe700 1 -- 172.21.15.119:0/4061452401 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedac011a80 con 0x7fedb80ccc00 2023-10-08T02:11:57.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.843+0000 7fed9f7fe700 1 -- 172.21.15.119:0/4061452401 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedac01a6a0 con 0x7fedb80ccc00 2023-10-08T02:11:57.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.843+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb80c71f0 con 0x7fedb80ccc00 2023-10-08T02:11:57.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.843+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb80c77b0 con 0x7fedb80ccc00 2023-10-08T02:11:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.844+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb806eba0 con 0x7fedb80ccc00 2023-10-08T02:11:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.846+0000 7fed9f7fe700 1 -- 172.21.15.119:0/4061452401 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 79) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fedac01aa50 con 0x7fedb80ccc00 2023-10-08T02:11:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.847+0000 7fed9f7fe700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feda4066f80 0x7feda4069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.847+0000 7fed9f7fe700 1 -- 172.21.15.119:0/4061452401 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fedac086010 con 0x7fedb80ccc00 2023-10-08T02:11:57.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.847+0000 7fedb77fe700 1 -- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feda4066f80 msgr2=0x7feda4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:57.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.847+0000 7fedb77fe700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feda4066f80 0x7feda4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:57.852+0000 7fed9f7fe700 1 -- 172.21.15.119:0/4061452401 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedac0530b0 con 0x7fedb80ccc00 2023-10-08T02:11:58.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.048+0000 7fedb77fe700 1 -- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feda4066f80 msgr2=0x7feda4069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:58.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.048+0000 7fedb77fe700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feda4066f80 0x7feda4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:58.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.054+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fedb80b3210 con 0x7fedb80ccc00 2023-10-08T02:11:58.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.058+0000 7fed9f7fe700 1 -- 172.21.15.119:0/4061452401 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7fedac0566d0 con 0x7fedb80ccc00 2023-10-08T02:11:58.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feda4066f80 msgr2=0x7feda4069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feda4066f80 0x7feda4069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedb80ccc00 msgr2=0x7fedb815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedb80ccc00 0x7fedb815ec90 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fedac0030c0 tx=0x7fedac016000 comp rx=0 tx=0).stop 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 shutdown_connections 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feda4066f80 0x7feda4069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fedb8159a60 0x7fedb80c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fedb806a020 0x7fedb81551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 --2- 172.21.15.119:0/4061452401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fedb80ccc00 0x7fedb815ec90 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 >> 172.21.15.119:0/4061452401 conn(0x7fedb806baa0 msgr2=0x7fedb806a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:58.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 shutdown_connections 2023-10-08T02:11:58.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.061+0000 7fedbcf9d700 1 -- 172.21.15.119:0/4061452401 wait complete. 2023-10-08T02:11:58.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:11:58.080 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T02:11:58.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-10-08T02:11:58.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.355+0000 7fe92a19d700 1 Processor -- start 2023-10-08T02:11:58.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.355+0000 7fe92a19d700 1 -- start start 2023-10-08T02:11:58.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.356+0000 7fe92a19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9240b03c0 0x7fe9240cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:58.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.356+0000 7fe92a19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9240b0790 0x7fe9240b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:58.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.356+0000 7fe92a19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe9241591e0 0x7fe924159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:58.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.356+0000 7fe92a19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9240cfa00 con 0x7fe9240b03c0 2023-10-08T02:11:58.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.356+0000 7fe92a19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9240cb580 con 0x7fe9241591e0 2023-10-08T02:11:58.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.356+0000 7fe92a19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9240b2de0 con 0x7fe9240b0790 2023-10-08T02:11:58.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9240b03c0 0x7fe9240cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9240b03c0 0x7fe9240cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56594/0 (socket says 172.21.15.119:56594) 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 -- 172.21.15.119:0/3900812288 learned_addr learned my addr 172.21.15.119:0/3900812288 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92999c700 1 --2- 172.21.15.119:0/3900812288 >> v2:172.21.15.184:3300/0 conn(0x7fe9241591e0 0x7fe924159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92899a700 1 --2- 172.21.15.119:0/3900812288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9240b0790 0x7fe9240b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 -- 172.21.15.119:0/3900812288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9240b0790 msgr2=0x7fe9240b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 --2- 172.21.15.119:0/3900812288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9240b0790 0x7fe9240b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 -- 172.21.15.119:0/3900812288 >> v2:172.21.15.184:3300/0 conn(0x7fe9241591e0 msgr2=0x7fe924159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 --2- 172.21.15.119:0/3900812288 >> v2:172.21.15.184:3300/0 conn(0x7fe9241591e0 0x7fe924159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 -- 172.21.15.119:0/3900812288 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9240cf160 con 0x7fe9240b03c0 2023-10-08T02:11:58.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92899a700 1 --2- 172.21.15.119:0/3900812288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9240b0790 0x7fe9240b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92999c700 1 --2- 172.21.15.119:0/3900812288 >> v2:172.21.15.184:3300/0 conn(0x7fe9241591e0 0x7fe924159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.357+0000 7fe92919b700 1 --2- 172.21.15.119:0/3900812288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe9240cea70 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe9240cb4f0 tx=0x7fe920009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46bf1ae43e3a2bc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe91bfff700 1 -- 172.21.15.119:0/3900812288 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92000b860 con 0x7fe9240b03c0 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe91bfff700 1 -- 172.21.15.119:0/3900812288 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92000b9c0 con 0x7fe9240b03c0 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe91bfff700 1 -- 172.21.15.119:0/3900812288 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92000bd60 con 0x7fe9240b03c0 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe92a19d700 1 -- 172.21.15.119:0/3900812288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 msgr2=0x7fe9240cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe92a19d700 1 --2- 172.21.15.119:0/3900812288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe9240cea70 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fe9240cb4f0 tx=0x7fe920009250 comp rx=0 tx=0).stop 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe92a19d700 1 -- 172.21.15.119:0/3900812288 shutdown_connections 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe92a19d700 1 --2- 172.21.15.119:0/3900812288 >> v2:172.21.15.184:3300/0 conn(0x7fe9241591e0 0x7fe924159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe92a19d700 1 --2- 172.21.15.119:0/3900812288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe9240cea70 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe92a19d700 1 --2- 172.21.15.119:0/3900812288 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9240b0790 0x7fe9240b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe92a19d700 1 -- 172.21.15.119:0/3900812288 >> 172.21.15.119:0/3900812288 conn(0x7fe924063230 msgr2=0x7fe924063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:11:58.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.358+0000 7fe92a19d700 1 -- 172.21.15.119:0/3900812288 shutdown_connections 2023-10-08T02:11:58.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.359+0000 7fe92a19d700 1 -- 172.21.15.119:0/3900812288 wait complete. 2023-10-08T02:11:58.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.360+0000 7fe92a19d700 1 Processor -- start 2023-10-08T02:11:58.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.360+0000 7fe92a19d700 1 -- start start 2023-10-08T02:11:58.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92a19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe924157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92a19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9240b0790 0x7fe92415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92a19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9241591e0 0x7fe9240cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92a19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92416a430 con 0x7fe9240b03c0 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92a19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe92416a710 con 0x7fe9240b0790 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92a19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe92416a9f0 con 0x7fe9241591e0 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92919b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe924157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92919b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe924157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56610/0 (socket says 172.21.15.119:56610) 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92919b700 1 -- 172.21.15.119:0/2898446787 learned_addr learned my addr 172.21.15.119:0/2898446787 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.361+0000 7fe92899a700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9240b0790 0x7fe92415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe92999c700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9241591e0 0x7fe9240cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:11:58.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe92919b700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9241591e0 msgr2=0x7fe9240cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe92919b700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9241591e0 0x7fe9240cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe92919b700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9240b0790 msgr2=0x7fe92415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe92919b700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9240b0790 0x7fe92415aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe92919b700 1 -- 172.21.15.119:0/2898446787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9200094f0 con 0x7fe9240b03c0 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe92919b700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe924157aa0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe920009290 tx=0x7fe920017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe920011e40 con 0x7fe9240b03c0 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe92001f070 con 0x7fe9240b03c0 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200266d0 con 0x7fe9240b03c0 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.362+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe92416acd0 con 0x7fe9240b03c0 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.363+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92416b170 con 0x7fe9240b03c0 2023-10-08T02:11:58.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.364+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe92404c3c0 con 0x7fe9240b03c0 2023-10-08T02:11:58.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.364+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 79) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fe920026a40 con 0x7fe9240b03c0 2023-10-08T02:11:58.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.365+0000 7fe9197fa700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe908069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:11:58.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.365+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fe920021070 con 0x7fe9240b03c0 2023-10-08T02:11:58.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.365+0000 7fe92899a700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 msgr2=0x7fe908069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:58.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.365+0000 7fe92899a700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe908069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:11:58.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.369+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9200534b0 con 0x7fe9240b03c0 2023-10-08T02:11:58.453 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:58.453+0000 7f7ef66db180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:11:58.543 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:58.542+0000 7ff621566180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:11:58.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.566+0000 7fe92899a700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 msgr2=0x7fe908069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:58.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.566+0000 7fe92899a700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe908069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:11:58.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.609+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fe924159620 con 0x7fe908066f80 2023-10-08T02:11:58.615 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:58.615+0000 7f7ef66db180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:11:58.700 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:58.700+0000 7f7ef66db180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:11:58.711 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:58.710+0000 7ff621566180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:11:58.796 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:58.796+0000 7ff621566180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:11:58.925 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:58.923+0000 7f7ef66db180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:11:58.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.967+0000 7fe92899a700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 msgr2=0x7fe908069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:58.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:58.967+0000 7fe92899a700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe908069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:11:59.021 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:59.021+0000 7ff621566180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:11:59.100 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:59.100+0000 7f7ef66db180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:11:59.192 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:59.192+0000 7f7ef66db180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:11:59.199 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:59.198+0000 7ff621566180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:11:59.291 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:59.291+0000 7ff621566180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:11:59.354 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:59.353+0000 7f7ef66db180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:11:59.453 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:59.453+0000 7ff621566180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:11:59.454 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:59.453+0000 7f7ef66db180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:11:59.555 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:11:59.555+0000 7ff621566180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:11:59.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:59.768+0000 7fe92899a700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 msgr2=0x7fe908069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:11:59.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:11:59.768+0000 7fe92899a700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe908069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T02:11:59.940 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:11:59.939+0000 7f7ef66db180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:12:00.035 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:00.035+0000 7f7ef66db180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:12:00.039 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:00.039+0000 7ff621566180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:12:00.126 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:00.125+0000 7f7ef66db180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:12:00.133 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:00.133+0000 7ff621566180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:12:00.223 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:00.222+0000 7ff621566180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:12:00.746 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:00.745+0000 7f7ef66db180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:12:00.838 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:00.838+0000 7ff621566180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:12:00.917 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:00.916+0000 7f7ef66db180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:01.010 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:01.010+0000 7ff621566180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:01.014 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:01.013+0000 7f7ef66db180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:12:01.106 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:01.106+0000 7ff621566180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:12:01.309 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:01.308+0000 7f7ef66db180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:12:01.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:01.370+0000 7fe92899a700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 msgr2=0x7fe908069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:01.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:01.370+0000 7fe92899a700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe908069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T02:12:01.397 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:01.397+0000 7ff621566180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:12:01.406 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:01.405+0000 7f7ef66db180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:12:01.493 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:01.493+0000 7ff621566180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:12:02.106 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:02.105+0000 7f7ef66db180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:12:02.194 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:02.194+0000 7ff621566180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:12:02.397 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:02.394+0000 7f7ef66db180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:12:02.477 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:02.476+0000 7ff621566180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:12:02.488 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:02.487+0000 7f7ef66db180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:12:02.570 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:02.569+0000 7ff621566180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:12:02.642 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:02.641+0000 7f7ef66db180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:12:02.721 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:02.721+0000 7ff621566180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:12:02.747 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:02.746+0000 7f7ef66db180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:12:02.825 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:02.824+0000 7ff621566180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:12:02.938 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:02.937+0000 7f7ef66db180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:12:03.014 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.013+0000 7ff621566180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:12:03.033 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:03.030+0000 7f7ef66db180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:12:03.105 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.105+0000 7ff621566180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:12:03.124 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:03.123+0000 7f7ef66db180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:12:03.197 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.196+0000 7ff621566180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:12:03.219 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:03.218+0000 7f7ef66db180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:12:03.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:03.279+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 80) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fe920026cf0 con 0x7fe9240b03c0 2023-10-08T02:12:03.294 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.293+0000 7ff621566180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:12:03.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:03.308+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 81) v1 ==== 84468+0+0 (secure 0 0 0) 0x7fe920053880 con 0x7fe9240b03c0 2023-10-08T02:12:03.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:03.308+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 msgr2=0x7fe908069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:03.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:03.308+0000 7fe9197fa700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe908069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:03.432 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.431+0000 7ff5968b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.432 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.431+0000 7ff5968b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.432 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.431+0000 7ff5968b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.432 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.431+0000 7ff5968b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.433 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.431+0000 7ff5968b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.433 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.432+0000 7ff5928af700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.434 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.432+0000 7ff5928af700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.434 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.432+0000 7ff5928af700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.434 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.432+0000 7ff5928af700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:03.434 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:03.432+0000 7ff5928af700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:04.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.310+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mon.0 v2:172.21.15.119:3300/0 9 ==== mgrmap(e 82) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fe920054150 con 0x7fe9240b03c0 2023-10-08T02:12:04.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.310+0000 7fe9197fa700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe9080693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:04.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.310+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fe924159620 con 0x7fe908066f80 2023-10-08T02:12:04.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.312+0000 7fe92899a700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe9080693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:04.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.316+0000 7fe92899a700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe9080693b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe9240b0110 tx=0x7fe914008040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:04.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.388+0000 7fe9197fa700 1 -- 172.21.15.119:0/2898446787 <== mgr.8877 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe924159620 con 0x7fe908066f80 2023-10-08T02:12:04.389 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path snap_test_dir1 2023-10-08T02:12:04.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 msgr2=0x7fe9080693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:04.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe9080693b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe9240b0110 tx=0x7fe914008040 comp rx=0 tx=0).stop 2023-10-08T02:12:04.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 msgr2=0x7fe924157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:04.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe924157aa0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fe920009290 tx=0x7fe920017000 comp rx=0 tx=0).stop 2023-10-08T02:12:04.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 shutdown_connections 2023-10-08T02:12:04.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe908066f80 0x7fe9080693b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9241591e0 0x7fe9240cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9240b03c0 0x7fe924157aa0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 --2- 172.21.15.119:0/2898446787 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9240b0790 0x7fe92415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 >> 172.21.15.119:0/2898446787 conn(0x7fe924063230 msgr2=0x7fe9241588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 shutdown_connections 2023-10-08T02:12:04.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.391+0000 7fe92a19d700 1 -- 172.21.15.119:0/2898446787 wait complete. 2023-10-08T02:12:04.410 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T02:12:04.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-08T02:12:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.598+0000 7fbd0fdff700 1 Processor -- start 2023-10-08T02:12:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.599+0000 7fbd0fdff700 1 -- start start 2023-10-08T02:12:04.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0fdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 0x7fbd080acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:04.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0fdff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbd08068890 0x7fbd080c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:04.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0fdff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0815a9b0 con 0x7fbd081563b0 2023-10-08T02:12:04.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0fdff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0815ac90 con 0x7fbd08068890 2023-10-08T02:12:04.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0fdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0815af70 con 0x7fbd0815a440 2023-10-08T02:12:04.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0e39c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbd08068890 0x7fbd080c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:04.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0e39c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbd08068890 0x7fbd080c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56730/0 (socket says 172.21.15.119:56730) 2023-10-08T02:12:04.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0e39c700 1 -- 172.21.15.119:0/1458930672 learned_addr learned my addr 172.21.15.119:0/1458930672 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:04.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0d39a700 1 --2- 172.21.15.119:0/1458930672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 0x7fbd080acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:04.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.600+0000 7fbd0e39c700 1 -- 172.21.15.119:0/1458930672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 msgr2=0x7fbd080acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:04.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.601+0000 7fbd0e39c700 1 --2- 172.21.15.119:0/1458930672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 0x7fbd080acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.601+0000 7fbd0e39c700 1 -- 172.21.15.119:0/1458930672 >> v1:172.21.15.184:6789/0 conn(0x7fbd081563b0 legacy=0x7fbd08156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:04.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.601+0000 7fbd0e39c700 1 -- 172.21.15.119:0/1458930672 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0815b250 con 0x7fbd08068890 2023-10-08T02:12:04.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.601+0000 7fbd0e39c700 1 --2- 172.21.15.119:0/1458930672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd080c8230 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009f20 tx=0x7fbd00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afe96c0fcbc82e42 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:04.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.601+0000 7fbd0cb99700 1 -- 172.21.15.119:0/1458930672 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0000e070 con 0x7fbd08068890 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.601+0000 7fbd0cb99700 1 -- 172.21.15.119:0/1458930672 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0000b9d0 con 0x7fbd08068890 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.601+0000 7fbd0cb99700 1 -- 172.21.15.119:0/1458930672 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd000092e0 con 0x7fbd08068890 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.602+0000 7fbd0fdff700 1 -- 172.21.15.119:0/1458930672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 msgr2=0x7fbd080c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.602+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/1458930672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd080c8230 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fbd00009f20 tx=0x7fbd00009840 comp rx=0 tx=0).stop 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.602+0000 7fbd0fdff700 1 -- 172.21.15.119:0/1458930672 shutdown_connections 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.602+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/1458930672 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd080c8230 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.602+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/1458930672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 0x7fbd080acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.602+0000 7fbd0fdff700 1 -- 172.21.15.119:0/1458930672 >> 172.21.15.119:0/1458930672 conn(0x7fbd080c8f90 msgr2=0x7fbd080c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.602+0000 7fbd0fdff700 1 -- 172.21.15.119:0/1458930672 shutdown_connections 2023-10-08T02:12:04.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.602+0000 7fbd0fdff700 1 -- 172.21.15.119:0/1458930672 wait complete. 2023-10-08T02:12:04.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.603+0000 7fbd0fdff700 1 Processor -- start 2023-10-08T02:12:04.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.603+0000 7fbd0fdff700 1 -- start start 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0fdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd081651e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0fdff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd081563b0 0x7fbd080b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0fdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 0x7fbd08159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0fdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd080af2d0 con 0x7fbd08068890 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0fdff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd081599d0 con 0x7fbd081563b0 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0fdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08165850 con 0x7fbd0815a440 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd081651e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd081651e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56732/0 (socket says 172.21.15.119:56732) 2023-10-08T02:12:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.604+0000 7fbd0db9b700 1 -- 172.21.15.119:0/4221020194 learned_addr learned my addr 172.21.15.119:0/4221020194 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0d39a700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd081563b0 0x7fbd080b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0e39c700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 0x7fbd08159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0db9b700 1 -- 172.21.15.119:0/4221020194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 msgr2=0x7fbd08159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0db9b700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 0x7fbd08159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0db9b700 1 -- 172.21.15.119:0/4221020194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd081563b0 msgr2=0x7fbd080b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0db9b700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd081563b0 0x7fbd080b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0db9b700 1 -- 172.21.15.119:0/4221020194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd00009ae0 con 0x7fbd08068890 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0db9b700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd081651e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400c590 tx=0x7fbd0400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbcfdffb700 1 -- 172.21.15.119:0/4221020194 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04011590 con 0x7fbd08068890 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbcfdffb700 1 -- 172.21.15.119:0/4221020194 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd040116f0 con 0x7fbd08068890 2023-10-08T02:12:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbcfdffb700 1 -- 172.21.15.119:0/4221020194 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04011ab0 con 0x7fbd08068890 2023-10-08T02:12:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd08165b30 con 0x7fbd08068890 2023-10-08T02:12:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.605+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd081660f0 con 0x7fbd08068890 2023-10-08T02:12:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.606+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd08159b40 con 0x7fbd08068890 2023-10-08T02:12:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.607+0000 7fbcfdffb700 1 -- 172.21.15.119:0/4221020194 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 82) v1 ==== 84595+0+0 (secure 0 0 0) 0x7fbd0401b3b0 con 0x7fbd08068890 2023-10-08T02:12:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.608+0000 7fbcfdffb700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbcec066dd0 0x7fbcec069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.608+0000 7fbcfdffb700 1 -- 172.21.15.119:0/4221020194 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fbd04083a90 con 0x7fbd08068890 2023-10-08T02:12:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.608+0000 7fbd0d39a700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbcec066dd0 0x7fbcec069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.609+0000 7fbd0d39a700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbcec066dd0 0x7fbcec069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbd08159f30 tx=0x7fbcf8008040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:04.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.613+0000 7fbcfdffb700 1 -- 172.21.15.119:0/4221020194 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbd04050c10 con 0x7fbd08068890 2023-10-08T02:12:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.826+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd080b0f70 con 0x7fbcec066dd0 2023-10-08T02:12:04.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.898+0000 7fbcfdffb700 1 -- 172.21.15.119:0/4221020194 <== mgr.8877 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbd080b0f70 con 0x7fbcec066dd0 2023-10-08T02:12:04.899 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path snap_test_dir1 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbcec066dd0 msgr2=0x7fbcec069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbcec066dd0 0x7fbcec069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbd08159f30 tx=0x7fbcf8008040 comp rx=0 tx=0).stop 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 msgr2=0x7fbd081651e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd081651e0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fbd0400c590 tx=0x7fbd0400c950 comp rx=0 tx=0).stop 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 shutdown_connections 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbcec066dd0 0x7fbcec069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbd0815a440 0x7fbd08159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd08068890 0x7fbd081651e0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 --2- 172.21.15.119:0/4221020194 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbd081563b0 0x7fbd080b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:04.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 >> 172.21.15.119:0/4221020194 conn(0x7fbd080c8f90 msgr2=0x7fbd080add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:04.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 shutdown_connections 2023-10-08T02:12:04.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:04.901+0000 7fbd0fdff700 1 -- 172.21.15.119:0/4221020194 wait complete. 2023-10-08T02:12:04.919 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T02:12:04.920 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-10-08T02:12:05.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.105+0000 7f0eba99d700 1 Processor -- start 2023-10-08T02:12:05.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.105+0000 7f0eba99d700 1 -- start start 2023-10-08T02:12:05.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.106+0000 7f0eba99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0eb40afa40 0x7f0eb41647e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.106+0000 7f0eba99d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb40afe60 con 0x7f0eb41597d0 2023-10-08T02:12:05.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.106+0000 7f0eba99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb406e510 con 0x7f0eb40ac0e0 2023-10-08T02:12:05.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.106+0000 7f0eba99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb415e450 con 0x7f0eb40afa40 2023-10-08T02:12:05.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.106+0000 7f0eb999b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0eb41597d0 0x7f0eb406e9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59104/0 (socket says 172.21.15.119:59104) 2023-10-08T02:12:05.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.106+0000 7f0eb999b700 1 -- 172.21.15.119:0/3944959002 learned_addr learned my addr 172.21.15.119:0/3944959002 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:05.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 168634520 0 0) 0x7f0eb40afe60 con 0x7f0eb41597d0 2023-10-08T02:12:05.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea0003570 con 0x7f0eb41597d0 2023-10-08T02:12:05.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eba19c700 1 --2- 172.21.15.119:0/3944959002 >> v2:172.21.15.184:3300/0 conn(0x7f0eb40afa40 0x7f0eb41647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 135594670 0 0) 0x7f0ea0003570 con 0x7f0eb41597d0 2023-10-08T02:12:05.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 >> v2:172.21.15.184:3300/0 conn(0x7f0eb40afa40 msgr2=0x7f0eb41647e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:05.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 --2- 172.21.15.119:0/3944959002 >> v2:172.21.15.184:3300/0 conn(0x7f0eb40afa40 0x7f0eb41647e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 >> v1:172.21.15.184:6789/0 conn(0x7f0eb40ac0e0 legacy=0x7f0eb406f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb4069180 con 0x7f0eb41597d0 2023-10-08T02:12:05.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f0eb00017f0 con 0x7f0eb41597d0 2023-10-08T02:12:05.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 >> v1:172.21.15.119:6789/0 conn(0x7f0eb41597d0 legacy=0x7f0eb406e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ea0003a60 0x7f0ea0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.107+0000 7f0eb8999700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea0004400 0x7f0ea00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eb8999700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea00050b0 0x7f0ea000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea000bfa0 con 0x7f0ea00050b0 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea000c110 con 0x7f0ea0004400 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea000c2d0 con 0x7f0ea0003a60 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eba19c700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea00050b0 0x7f0ea000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eb919a700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea0004400 0x7f0ea00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eb999b700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ea0003a60 0x7f0ea0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eba19c700 1 -- 172.21.15.119:0/3944959002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ea0003a60 msgr2=0x7f0ea0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:05.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eba19c700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ea0003a60 0x7f0ea0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eba19c700 1 -- 172.21.15.119:0/3944959002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea0004400 msgr2=0x7f0ea00048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eba19c700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea0004400 0x7f0ea00048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eba19c700 1 -- 172.21.15.119:0/3944959002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea0003570 con 0x7f0ea00050b0 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eba19c700 1 -- 172.21.15.119:0/3944959002 reap_dead start 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.108+0000 7f0eba19c700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea00050b0 0x7f0ea000ba60 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009c00 tx=0x7f0eac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24df29385a9a3b4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00e710 con 0x7f0ea00050b0 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac00e890 con 0x7f0ea00050b0 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eb8999700 1 -- 172.21.15.119:0/3944959002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00ec50 con 0x7f0ea00050b0 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eba99d700 1 -- 172.21.15.119:0/3944959002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea00050b0 msgr2=0x7f0ea000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eba99d700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea00050b0 0x7f0ea000ba60 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f0eac009c00 tx=0x7f0eac0091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eba99d700 1 -- 172.21.15.119:0/3944959002 shutdown_connections 2023-10-08T02:12:05.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eba99d700 1 --2- 172.21.15.119:0/3944959002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea00050b0 0x7f0ea000ba60 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eba99d700 1 -- 172.21.15.119:0/3944959002 >> 172.21.15.119:0/3944959002 conn(0x7f0eb406adf0 msgr2=0x7f0eb40ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:05.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eba99d700 1 -- 172.21.15.119:0/3944959002 shutdown_connections 2023-10-08T02:12:05.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.109+0000 7f0eba99d700 1 -- 172.21.15.119:0/3944959002 wait complete. 2023-10-08T02:12:05.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.111+0000 7f0eba99d700 1 Processor -- start 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.111+0000 7f0eba99d700 1 -- start start 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea00050b0 0x7f0eb4156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0eb41556e0 0x7f0eb41579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb4157f30 0x7f0eb40b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40b08e0 con 0x7f0eb4157f30 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb41583d0 con 0x7f0ea00050b0 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4157000 con 0x7f0eb41556e0 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb4157f30 0x7f0eb40b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb4157f30 0x7f0eb40b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35352/0 (socket says 172.21.15.119:35352) 2023-10-08T02:12:05.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba19c700 1 -- 172.21.15.119:0/1042798228 learned_addr learned my addr 172.21.15.119:0/1042798228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:05.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eb999b700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea00050b0 0x7f0eb4156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eb919a700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0eb41556e0 0x7f0eb41579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba19c700 1 -- 172.21.15.119:0/1042798228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0eb41556e0 msgr2=0x7f0eb41579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba19c700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0eb41556e0 0x7f0eb41579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba19c700 1 -- 172.21.15.119:0/1042798228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea00050b0 msgr2=0x7f0eb4156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba19c700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea00050b0 0x7f0eb4156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eba19c700 1 -- 172.21.15.119:0/1042798228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40aff40 con 0x7f0eb4157f30 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eb999b700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ea00050b0 0x7f0eb4156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.112+0000 7f0eb919a700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0eb41556e0 0x7f0eb41579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.113+0000 7f0eba19c700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0eb4157f30 0x7f0eb40b0d00 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f0eb406ddb0 tx=0x7f0eac019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.113+0000 7f0ea9ffb700 1 -- 172.21.15.119:0/1042798228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00ec10 con 0x7f0eb4157f30 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.113+0000 7f0ea9ffb700 1 -- 172.21.15.119:0/1042798228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac01c450 con 0x7f0eb4157f30 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.113+0000 7f0eba99d700 1 -- 172.21.15.119:0/1042798228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb40b0220 con 0x7f0eb4157f30 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.113+0000 7f0ea9ffb700 1 -- 172.21.15.119:0/1042798228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac01c7f0 con 0x7f0eb4157f30 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.113+0000 7f0eba99d700 1 -- 172.21.15.119:0/1042798228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb4158760 con 0x7f0eb4157f30 2023-10-08T02:12:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.114+0000 7f0eba99d700 1 -- 172.21.15.119:0/1042798228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb40b0a20 con 0x7f0eb4157f30 2023-10-08T02:12:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.115+0000 7f0ea9ffb700 1 -- 172.21.15.119:0/1042798228 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 82) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f0eac01cc20 con 0x7f0eb4157f30 2023-10-08T02:12:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.115+0000 7f0ea9ffb700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0e98066dd0 0x7f0e98069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.116+0000 7f0ea9ffb700 1 -- 172.21.15.119:0/1042798228 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f0eac086350 con 0x7f0eb4157f30 2023-10-08T02:12:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.116+0000 7f0eb999b700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0e98066dd0 0x7f0e98069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.116+0000 7f0eb999b700 1 --2- 172.21.15.119:0/1042798228 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0e98066dd0 0x7f0e98069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0001390 tx=0x7f0eb000a040 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.120+0000 7f0ea9ffb700 1 -- 172.21.15.119:0/1042798228 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0eb40b0a20 con 0x7f0eb4157f30 2023-10-08T02:12:05.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.318+0000 7f0ea9ffb700 1 -- 172.21.15.119:0/1042798228 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 83) v1 ==== 84916+0+0 (secure 0 0 0) 0x7f0eac053970 con 0x7f0eb4157f30 2023-10-08T02:12:05.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.340+0000 7f0eba99d700 1 -- 172.21.15.119:0/1042798228 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eb40b0a20 con 0x7f0e98066dd0 2023-10-08T02:12:05.348 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:05.347+0000 7ff59d8c5700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-10-08T02:12:05.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.348+0000 7f0ea9ffb700 1 -- 172.21.15.119:0/1042798228 <== mgr.8877 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f0eb40b0a20 con 0x7f0e98066dd0 2023-10-08T02:12:05.350 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-10-08T02:12:05.357 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-10-08T02:12:05.358 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:12:05.358 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T02:12:05.397 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T02:12:05.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.619+0000 7fe9d5dff700 1 Processor -- start 2023-10-08T02:12:05.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.619+0000 7fe9d5dff700 1 -- start start 2023-10-08T02:12:05.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.620+0000 7fe9d5dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d00c83f0 0x7fe9d00c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.620+0000 7fe9d5dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d015b0d0 con 0x7fe9d00ccee0 2023-10-08T02:12:05.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.620+0000 7fe9d5dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d00c7a60 con 0x7fe9d01568e0 2023-10-08T02:12:05.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.620+0000 7fe9d5dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d015a4c0 con 0x7fe9d00c83f0 2023-10-08T02:12:05.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9ceffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe9d00ccee0 0x7fe9d01561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59118/0 (socket says 172.21.15.119:59118) 2023-10-08T02:12:05.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9ceffd700 1 -- 172.21.15.119:0/149258417 learned_addr learned my addr 172.21.15.119:0/149258417 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:05.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9cf7fe700 1 --2- 172.21.15.119:0/149258417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d00c83f0 0x7fe9d00c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9ce7fc700 1 -- 172.21.15.119:0/149258417 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690072073 0 0) 0x7fe9d015b0d0 con 0x7fe9d00ccee0 2023-10-08T02:12:05.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9ce7fc700 1 -- 172.21.15.119:0/149258417 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9b8003570 con 0x7fe9d00ccee0 2023-10-08T02:12:05.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9ce7fc700 1 -- 172.21.15.119:0/149258417 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157311614 0 0) 0x7fe9d00c7a60 con 0x7fe9d01568e0 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9ce7fc700 1 -- 172.21.15.119:0/149258417 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9d015b0d0 con 0x7fe9d01568e0 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9cf7fe700 1 -- 172.21.15.119:0/149258417 >> v1:172.21.15.184:6789/0 conn(0x7fe9d01568e0 legacy=0x7fe9d00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9cf7fe700 1 -- 172.21.15.119:0/149258417 >> v1:172.21.15.119:6789/0 conn(0x7fe9d00ccee0 legacy=0x7fe9d01561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.621+0000 7fe9cf7fe700 1 -- 172.21.15.119:0/149258417 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d015a7a0 con 0x7fe9d00c83f0 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.622+0000 7fe9cf7fe700 1 --2- 172.21.15.119:0/149258417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d00c83f0 0x7fe9d00c7520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009910 tx=0x7fe9c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa74b1587c7e491e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.622+0000 7fe9ce7fc700 1 -- 172.21.15.119:0/149258417 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0011630 con 0x7fe9d00c83f0 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.622+0000 7fe9ce7fc700 1 -- 172.21.15.119:0/149258417 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c0011790 con 0x7fe9d00c83f0 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.622+0000 7fe9ce7fc700 1 -- 172.21.15.119:0/149258417 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0011b30 con 0x7fe9d00c83f0 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.622+0000 7fe9d5dff700 1 -- 172.21.15.119:0/149258417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d00c83f0 msgr2=0x7fe9d00c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.622+0000 7fe9d5dff700 1 --2- 172.21.15.119:0/149258417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d00c83f0 0x7fe9d00c7520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009910 tx=0x7fe9c00091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.623+0000 7fe9d5dff700 1 -- 172.21.15.119:0/149258417 shutdown_connections 2023-10-08T02:12:05.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.623+0000 7fe9d5dff700 1 --2- 172.21.15.119:0/149258417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d00c83f0 0x7fe9d00c7520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.623+0000 7fe9d5dff700 1 -- 172.21.15.119:0/149258417 >> 172.21.15.119:0/149258417 conn(0x7fe9d0068e10 msgr2=0x7fe9d00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:05.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.623+0000 7fe9d5dff700 1 -- 172.21.15.119:0/149258417 shutdown_connections 2023-10-08T02:12:05.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.623+0000 7fe9d5dff700 1 -- 172.21.15.119:0/149258417 wait complete. 2023-10-08T02:12:05.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.624+0000 7fe9d5dff700 1 Processor -- start 2023-10-08T02:12:05.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.624+0000 7fe9d5dff700 1 -- start start 2023-10-08T02:12:05.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9d5dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d00c83f0 0x7fe9d00c6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9d5dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d00ccee0 0x7fe9d00b0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9d5dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d01568e0 0x7fe9d00b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9d5dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d016b930 con 0x7fe9d00ccee0 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9d5dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0168870 con 0x7fe9d00c83f0 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9d5dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d016b2e0 con 0x7fe9d01568e0 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9ceffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d00ccee0 0x7fe9d00b0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9ceffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d00ccee0 0x7fe9d00b0cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35366/0 (socket says 172.21.15.119:35366) 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.625+0000 7fe9ceffd700 1 -- 172.21.15.119:0/1528047162 learned_addr learned my addr 172.21.15.119:0/1528047162 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9cf7fe700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d00c83f0 0x7fe9d00c6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9ceffd700 1 -- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d01568e0 msgr2=0x7fe9d00b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9cffff700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d01568e0 0x7fe9d00b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9ceffd700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d01568e0 0x7fe9d00b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9ceffd700 1 -- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d00c83f0 msgr2=0x7fe9d00c6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9ceffd700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d00c83f0 0x7fe9d00c6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9ceffd700 1 -- 172.21.15.119:0/1528047162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c0009420 con 0x7fe9d00ccee0 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9cf7fe700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d00c83f0 0x7fe9d00c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9cffff700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d01568e0 0x7fe9d00b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9ceffd700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d00ccee0 0x7fe9d00b0cf0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c790 tx=0x7fe9c4007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9b77fe700 1 -- 172.21.15.119:0/1528047162 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4005b30 con 0x7fe9d00ccee0 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9b77fe700 1 -- 172.21.15.119:0/1528047162 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c4005c90 con 0x7fe9d00ccee0 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9b77fe700 1 -- 172.21.15.119:0/1528047162 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4007300 con 0x7fe9d00ccee0 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.626+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d016b5c0 con 0x7fe9d00ccee0 2023-10-08T02:12:05.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.627+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d0167b00 con 0x7fe9d00ccee0 2023-10-08T02:12:05.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.627+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d016b140 con 0x7fe9d00ccee0 2023-10-08T02:12:05.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.628+0000 7fe9b77fe700 1 -- 172.21.15.119:0/1528047162 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 83) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fe9c401b400 con 0x7fe9d00ccee0 2023-10-08T02:12:05.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.629+0000 7fe9b77fe700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9bc067130 0x7fe9bc0695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:05.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.629+0000 7fe9b77fe700 1 -- 172.21.15.119:0/1528047162 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fe9c4084760 con 0x7fe9d00ccee0 2023-10-08T02:12:05.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.629+0000 7fe9cf7fe700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9bc067130 0x7fe9bc0695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:05.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.630+0000 7fe9cf7fe700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9bc067130 0x7fe9bc0695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0151ad0 tx=0x7fe9c0017000 comp rx=0 tx=0).ready entity=mgr.8877 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:05.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.633+0000 7fe9b77fe700 1 -- 172.21.15.119:0/1528047162 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9d016b140 con 0x7fe9d00ccee0 2023-10-08T02:12:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:05.875+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fe9d016b750 con 0x7fe9d00ccee0 2023-10-08T02:12:06.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.323+0000 7fe9b77fe700 1 -- 172.21.15.119:0/1528047162 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fe9c40518e0 con 0x7fe9d00ccee0 2023-10-08T02:12:06.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.324+0000 7fe9b77fe700 1 -- 172.21.15.119:0/1528047162 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v84) v1 ==== 93+0+0 (secure 0 0 0) 0x7fe9c4054fa0 con 0x7fe9d00ccee0 2023-10-08T02:12:06.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9bc067130 msgr2=0x7fe9bc0695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:06.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9bc067130 0x7fe9bc0695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0151ad0 tx=0x7fe9c0017000 comp rx=0 tx=0).stop 2023-10-08T02:12:06.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d00ccee0 msgr2=0x7fe9d00b0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:06.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d00ccee0 0x7fe9d00b0cf0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c790 tx=0x7fe9c4007ea0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 shutdown_connections 2023-10-08T02:12:06.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9bc067130 0x7fe9bc0695e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d01568e0 0x7fe9d00b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d00c83f0 0x7fe9d00c6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 --2- 172.21.15.119:0/1528047162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d00ccee0 0x7fe9d00b0cf0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 >> 172.21.15.119:0/1528047162 conn(0x7fe9d0068e10 msgr2=0x7fe9d00c78b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:06.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 shutdown_connections 2023-10-08T02:12:06.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.328+0000 7fe9d5dff700 1 -- 172.21.15.119:0/1528047162 wait complete. 2023-10-08T02:12:06.347 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:12:06.348 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:12:06.434 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:12:06.515 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:12:06.516 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T02:12:06.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:12:06.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T02:12:06.600 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:12:06.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:12:06.600 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T02:12:06.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:12:06.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:12:06.843 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:12:06.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.865+0000 7fc7caf9d700 1 Processor -- start 2023-10-08T02:12:06.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.865+0000 7fc7caf9d700 1 -- start start 2023-10-08T02:12:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.866+0000 7fc7caf9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc7c40ad870 0x7fc7c406b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.866+0000 7fc7caf9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c40b40f0 con 0x7fc7c4142ea0 2023-10-08T02:12:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.866+0000 7fc7caf9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c4069640 con 0x7fc7c406bf20 2023-10-08T02:12:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.866+0000 7fc7caf9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c406c3a0 con 0x7fc7c40ad870 2023-10-08T02:12:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.866+0000 7fc7c9f9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc7c4142ea0 0x7fc7c40b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59126/0 (socket says 172.21.15.119:59126) 2023-10-08T02:12:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.866+0000 7fc7c9f9b700 1 -- 172.21.15.119:0/89916627 learned_addr learned my addr 172.21.15.119:0/89916627 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.866+0000 7fc7c979a700 1 --2- 172.21.15.119:0/89916627 >> v2:172.21.15.184:3300/0 conn(0x7fc7c40ad870 0x7fc7c406b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.867+0000 7fc7c8f99700 1 -- 172.21.15.119:0/89916627 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488200595 0 0) 0x7fc7c40b40f0 con 0x7fc7c4142ea0 2023-10-08T02:12:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.867+0000 7fc7c8f99700 1 -- 172.21.15.119:0/89916627 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7b0003570 con 0x7fc7c4142ea0 2023-10-08T02:12:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.867+0000 7fc7c8f99700 1 -- 172.21.15.119:0/89916627 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3677248039 0 0) 0x7fc7c4069640 con 0x7fc7c406bf20 2023-10-08T02:12:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.867+0000 7fc7c8f99700 1 -- 172.21.15.119:0/89916627 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c40b40f0 con 0x7fc7c406bf20 2023-10-08T02:12:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.867+0000 7fc7c979a700 1 -- 172.21.15.119:0/89916627 >> v1:172.21.15.184:6789/0 conn(0x7fc7c406bf20 legacy=0x7fc7c406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.867+0000 7fc7c979a700 1 -- 172.21.15.119:0/89916627 >> v1:172.21.15.119:6789/0 conn(0x7fc7c4142ea0 legacy=0x7fc7c40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.867+0000 7fc7c979a700 1 -- 172.21.15.119:0/89916627 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c4069920 con 0x7fc7c40ad870 2023-10-08T02:12:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.867+0000 7fc7c979a700 1 --2- 172.21.15.119:0/89916627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c40ad870 0x7fc7c406b9e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40098e0 tx=0x7fc7b40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77e33e4c7092ead5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7c8f99700 1 -- 172.21.15.119:0/89916627 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4011630 con 0x7fc7c40ad870 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7c8f99700 1 -- 172.21.15.119:0/89916627 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b4011790 con 0x7fc7c40ad870 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7c8f99700 1 -- 172.21.15.119:0/89916627 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4011b30 con 0x7fc7c40ad870 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7caf9d700 1 -- 172.21.15.119:0/89916627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c40ad870 msgr2=0x7fc7c406b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7caf9d700 1 --2- 172.21.15.119:0/89916627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c40ad870 0x7fc7c406b9e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc7b40098e0 tx=0x7fc7b40091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7caf9d700 1 -- 172.21.15.119:0/89916627 shutdown_connections 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7caf9d700 1 --2- 172.21.15.119:0/89916627 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c40ad870 0x7fc7c406b9e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7caf9d700 1 -- 172.21.15.119:0/89916627 >> 172.21.15.119:0/89916627 conn(0x7fc7c40ce430 msgr2=0x7fc7c40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7caf9d700 1 -- 172.21.15.119:0/89916627 shutdown_connections 2023-10-08T02:12:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.868+0000 7fc7caf9d700 1 -- 172.21.15.119:0/89916627 wait complete. 2023-10-08T02:12:06.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.870+0000 7fc7caf9d700 1 Processor -- start 2023-10-08T02:12:06.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.870+0000 7fc7caf9d700 1 -- start start 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7caf9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7c406bf20 0x7fc7c413f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7caf9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c4142ea0 0x7fc7c413e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7caf9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7c415d520 0x7fc7c40cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7caf9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c413e630 con 0x7fc7c406bf20 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7caf9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c415dc90 con 0x7fc7c4142ea0 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7caf9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c415df70 con 0x7fc7c415d520 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7c9f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7c406bf20 0x7fc7c413f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7c979a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c4142ea0 0x7fc7c413e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7c9f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7c406bf20 0x7fc7c413f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35390/0 (socket says 172.21.15.119:35390) 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7c9f9b700 1 -- 172.21.15.119:0/197904440 learned_addr learned my addr 172.21.15.119:0/197904440 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.871+0000 7fc7ca79c700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7c415d520 0x7fc7c40cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7c9f9b700 1 -- 172.21.15.119:0/197904440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7c415d520 msgr2=0x7fc7c40cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7c9f9b700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7c415d520 0x7fc7c40cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7c9f9b700 1 -- 172.21.15.119:0/197904440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c4142ea0 msgr2=0x7fc7c413e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7c9f9b700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c4142ea0 0x7fc7c413e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7c9f9b700 1 -- 172.21.15.119:0/197904440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b4009420 con 0x7fc7c406bf20 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7c9f9b700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7c406bf20 0x7fc7c413f1d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c790 tx=0x7fc7c0007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7b9ffb700 1 -- 172.21.15.119:0/197904440 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0005b30 con 0x7fc7c406bf20 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7b9ffb700 1 -- 172.21.15.119:0/197904440 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0005c90 con 0x7fc7c406bf20 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c415e250 con 0x7fc7c406bf20 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7b9ffb700 1 -- 172.21.15.119:0/197904440 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00072e0 con 0x7fc7c406bf20 2023-10-08T02:12:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.872+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c415d9a0 con 0x7fc7c406bf20 2023-10-08T02:12:06.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.873+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c404c3c0 con 0x7fc7c406bf20 2023-10-08T02:12:06.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.875+0000 7fc7b9ffb700 1 -- 172.21.15.119:0/197904440 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fc7c001b3b0 con 0x7fc7c406bf20 2023-10-08T02:12:06.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.876+0000 7fc7b9ffb700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc7a80670e0 0x7fc7a8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:06.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.876+0000 7fc7b9ffb700 1 -- 172.21.15.119:0/197904440 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fc7c0083e10 con 0x7fc7c406bf20 2023-10-08T02:12:06.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.876+0000 7fc7c979a700 1 -- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc7a80670e0 msgr2=0x7fc7a8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:06.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.876+0000 7fc7c979a700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc7a80670e0 0x7fc7a8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:06.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:06.880+0000 7fc7b9ffb700 1 -- 172.21.15.119:0/197904440 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7c004b5a0 con 0x7fc7c406bf20 2023-10-08T02:12:07.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.077+0000 7fc7c979a700 1 -- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc7a80670e0 msgr2=0x7fc7a8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:07.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.077+0000 7fc7c979a700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc7a80670e0 0x7fc7a8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:07.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.132+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7c40c8580 con 0x7fc7c406bf20 2023-10-08T02:12:07.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.133+0000 7fc7b9ffb700 1 -- 172.21.15.119:0/197904440 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fc7c0017020 con 0x7fc7c406bf20 2023-10-08T02:12:07.134 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:07.135 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/603902131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":603902131},{"type":"v1","addr":"172.21.15.184:6834","nonce":603902131}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3930162267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3930162267},{"type":"v1","addr":"172.21.15.119:6834","nonce":3930162267}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/237096421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":237096421},{"type":"v1","addr":"172.21.15.119:6835","nonce":237096421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:11:41.545410+0000","modified":"2023-10-08T02:11:45.111313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8667":{"gid":8667,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3810992025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3810992025},{"type":"v1","addr":"172.21.15.184:6835","nonce":3810992025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T02:12:07.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc7a80670e0 msgr2=0x7fc7a8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:07.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc7a80670e0 0x7fc7a8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7c406bf20 msgr2=0x7fc7c413f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7c406bf20 0x7fc7c413f1d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c790 tx=0x7fc7c0007ea0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 shutdown_connections 2023-10-08T02:12:07.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc7a80670e0 0x7fc7a8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc7c415d520 0x7fc7c40cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7c406bf20 0x7fc7c413f1d0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.135+0000 7fc7caf9d700 1 --2- 172.21.15.119:0/197904440 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc7c4142ea0 0x7fc7c413e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.136+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 >> 172.21.15.119:0/197904440 conn(0x7fc7c40ce430 msgr2=0x7fc7c40b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:07.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.136+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 shutdown_connections 2023-10-08T02:12:07.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.136+0000 7fc7caf9d700 1 -- 172.21.15.119:0/197904440 wait complete. 2023-10-08T02:12:07.138 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 71 2023-10-08T02:12:07.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:12:07.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.342+0000 7f67d539d700 1 Processor -- start 2023-10-08T02:12:07.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.342+0000 7f67d539d700 1 -- start start 2023-10-08T02:12:07.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.343+0000 7f67d539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.343+0000 7f67d539d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f67d006a0a0 0x7f67d00c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.343+0000 7f67d539d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f67d006dd20 0x7f67d00c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.343+0000 7f67d539d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d006ae70 con 0x7f67d006dd20 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.343+0000 7f67d539d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d006a540 con 0x7f67d006a0a0 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.343+0000 7f67d539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d006a820 con 0x7f67d006d950 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67d4b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f67d006dd20 0x7f67d00c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cf7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f67d006a0a0 0x7f67d00c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cf7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f67d006a0a0 0x7f67d00c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54276/0 (socket says 172.21.15.119:54276) 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cf7fe700 1 -- 172.21.15.119:0/2114863842 learned_addr learned my addr 172.21.15.119:0/2114863842 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:07.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cffff700 1 -- 172.21.15.119:0/2114863842 >> v2:172.21.15.184:3300/0 conn(0x7f67d006a0a0 msgr2=0x7f67d00c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cffff700 1 --2- 172.21.15.119:0/2114863842 >> v2:172.21.15.184:3300/0 conn(0x7f67d006a0a0 0x7f67d00c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cffff700 1 -- 172.21.15.119:0/2114863842 >> v2:172.21.15.119:3300/0 conn(0x7f67d006dd20 msgr2=0x7f67d00c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cffff700 1 --2- 172.21.15.119:0/2114863842 >> v2:172.21.15.119:3300/0 conn(0x7f67d006dd20 0x7f67d00c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cffff700 1 -- 172.21.15.119:0/2114863842 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d006ab00 con 0x7f67d006d950 2023-10-08T02:12:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.344+0000 7f67cffff700 1 --2- 172.21.15.119:0/2114863842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00c8350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67d00cc150 tx=0x7f67c00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59d4546a9e948aac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67ceffd700 1 -- 172.21.15.119:0/2114863842 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0011630 con 0x7f67d006d950 2023-10-08T02:12:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67ceffd700 1 -- 172.21.15.119:0/2114863842 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c00117b0 con 0x7f67d006d950 2023-10-08T02:12:07.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67ceffd700 1 -- 172.21.15.119:0/2114863842 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c0011b70 con 0x7f67d006d950 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 -- 172.21.15.119:0/2114863842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 msgr2=0x7f67d00c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 --2- 172.21.15.119:0/2114863842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00c8350 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f67d00cc150 tx=0x7f67c00091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 -- 172.21.15.119:0/2114863842 shutdown_connections 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 --2- 172.21.15.119:0/2114863842 >> v2:172.21.15.119:3300/0 conn(0x7f67d006dd20 0x7f67d00c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 --2- 172.21.15.119:0/2114863842 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00c8350 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 --2- 172.21.15.119:0/2114863842 >> v2:172.21.15.184:3300/0 conn(0x7f67d006a0a0 0x7f67d00c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 -- 172.21.15.119:0/2114863842 >> 172.21.15.119:0/2114863842 conn(0x7f67d00cc1f0 msgr2=0x7f67d00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 -- 172.21.15.119:0/2114863842 shutdown_connections 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.345+0000 7f67d539d700 1 -- 172.21.15.119:0/2114863842 wait complete. 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.347+0000 7f67d539d700 1 Processor -- start 2023-10-08T02:12:07.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.347+0000 7f67d539d700 1 -- start start 2023-10-08T02:12:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67d539d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d006a0a0 0x7f67d014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67d539d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67d539d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d006dd20 0x7f67d0169910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67d539d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00cacb0 con 0x7f67d006dd20 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67d539d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d014f0f0 con 0x7f67d006a0a0 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67d539d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d00af290 con 0x7f67d006d950 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67cffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d006a0a0 0x7f67d014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67cf7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67cffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d006a0a0 0x7f67d014ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54278/0 (socket says 172.21.15.119:54278) 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67cffff700 1 -- 172.21.15.119:0/380067292 learned_addr learned my addr 172.21.15.119:0/380067292 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.348+0000 7f67d4b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d006dd20 0x7f67d0169910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67cf7fe700 1 -- 172.21.15.119:0/380067292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d006a0a0 msgr2=0x7f67d014ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67cf7fe700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d006a0a0 0x7f67d014ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67cf7fe700 1 -- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d006dd20 msgr2=0x7f67d0169910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67cf7fe700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d006dd20 0x7f67d0169910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67cf7fe700 1 -- 172.21.15.119:0/380067292 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c0009420 con 0x7f67d006d950 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67cffff700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d006a0a0 0x7f67d014ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67cf7fe700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00ca770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f67c4006d40 tx=0x7f67c4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67b7fff700 1 -- 172.21.15.119:0/380067292 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4013680 con 0x7f67d006d950 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67b7fff700 1 -- 172.21.15.119:0/380067292 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c40137e0 con 0x7f67d006d950 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d00af570 con 0x7f67d006d950 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67b7fff700 1 -- 172.21.15.119:0/380067292 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4013b60 con 0x7f67d006d950 2023-10-08T02:12:07.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.349+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d00afb30 con 0x7f67d006d950 2023-10-08T02:12:07.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.350+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d004c3c0 con 0x7f67d006d950 2023-10-08T02:12:07.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.351+0000 7f67b7fff700 1 -- 172.21.15.119:0/380067292 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f67c400c3e0 con 0x7f67d006d950 2023-10-08T02:12:07.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.352+0000 7f67b7fff700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67bc0670e0 0x7f67bc069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.352+0000 7f67cffff700 1 -- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67bc0670e0 msgr2=0x7f67bc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:07.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.352+0000 7f67b7fff700 1 -- 172.21.15.119:0/380067292 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f67c4084bd0 con 0x7f67d006d950 2023-10-08T02:12:07.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.352+0000 7f67cffff700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67bc0670e0 0x7f67bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:07.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.356+0000 7f67b7fff700 1 -- 172.21.15.119:0/380067292 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f67c40518f0 con 0x7f67d006d950 2023-10-08T02:12:07.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.553+0000 7f67cffff700 1 -- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67bc0670e0 msgr2=0x7f67bc069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:07.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.553+0000 7f67cffff700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67bc0670e0 0x7f67bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:07.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.559+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d0153b20 con 0x7f67d006d950 2023-10-08T02:12:07.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.560+0000 7f67b7fff700 1 -- 172.21.15.119:0/380067292 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f67c4054f10 con 0x7f67d006d950 2023-10-08T02:12:07.561 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:07.561 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":158,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:12:03.296789+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:11:39.192970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T02:11:40.531612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2185260495":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/3673331094":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/1337159806":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/294111303":"2023-10-09T02:12:03.296742+0000","172.21.15.119:6837/85511":"2023-10-09T02:12:03.296742+0000","172.21.15.119:6836/85511":"2023-10-09T02:12:03.296742+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:12:07.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67bc0670e0 msgr2=0x7f67bc069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:07.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67bc0670e0 0x7f67bc069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 msgr2=0x7f67d00ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00ca770 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f67c4006d40 tx=0x7f67c4009250 comp rx=0 tx=0).stop 2023-10-08T02:12:07.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 shutdown_connections 2023-10-08T02:12:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f67bc0670e0 0x7f67bc069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67d006dd20 0x7f67d0169910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f67d006a0a0 0x7f67d014ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 --2- 172.21.15.119:0/380067292 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f67d006d950 0x7f67d00ca770 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 >> 172.21.15.119:0/380067292 conn(0x7f67d00cc1f0 msgr2=0x7f67d00c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 shutdown_connections 2023-10-08T02:12:07.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.562+0000 7f67d539d700 1 -- 172.21.15.119:0/380067292 wait complete. 2023-10-08T02:12:07.582 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T02:11:39.192970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '156', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T02:11:40.531612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '157', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T02:12:07.582 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:12:07.620 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:07.619+0000 7f30e1e76180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:12:07.670 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:07.669+0000 7fabedb09180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:12:07.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.771+0000 7f9d07fff700 1 Processor -- start 2023-10-08T02:12:07.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.771+0000 7f9d07fff700 1 -- start start 2023-10-08T02:12:07.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.773+0000 7f9d07fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d080cc9d0 0x7f9d0815d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.773+0000 7f9d07fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d08150b50 con 0x7f9d080cc600 2023-10-08T02:12:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.773+0000 7f9d07fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d080cede0 con 0x7f9d080cd0f0 2023-10-08T02:12:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.773+0000 7f9d07fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d080b2260 con 0x7f9d080cc9d0 2023-10-08T02:12:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.773+0000 7f9d06ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9d080cc600 0x7f9d080b1a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59134/0 (socket says 172.21.15.119:59134) 2023-10-08T02:12:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.773+0000 7f9d06ffd700 1 -- 172.21.15.119:0/790507703 learned_addr learned my addr 172.21.15.119:0/790507703 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:07.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.773+0000 7f9d067fc700 1 --2- 172.21.15.119:0/790507703 >> v2:172.21.15.119:3300/0 conn(0x7f9d080cc9d0 0x7f9d0815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.773+0000 7f9d05ffb700 1 -- 172.21.15.119:0/790507703 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1466684592 0 0) 0x7f9d08150b50 con 0x7f9d080cc600 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d05ffb700 1 -- 172.21.15.119:0/790507703 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cf0003570 con 0x7f9d080cc600 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d067fc700 1 -- 172.21.15.119:0/790507703 >> v1:172.21.15.184:6789/0 conn(0x7f9d080cd0f0 legacy=0x7f9d080cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d067fc700 1 -- 172.21.15.119:0/790507703 >> v1:172.21.15.119:6789/0 conn(0x7f9d080cc600 legacy=0x7f9d080b1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d05ffb700 1 -- 172.21.15.119:0/790507703 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518733244 0 0) 0x7f9d080cede0 con 0x7f9d080cd0f0 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d067fc700 1 -- 172.21.15.119:0/790507703 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d080b2540 con 0x7f9d080cc9d0 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d067fc700 1 --2- 172.21.15.119:0/790507703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d0815d280 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098e0 tx=0x7f9cfc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e99c03965f47cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d05ffb700 1 -- 172.21.15.119:0/790507703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00b640 con 0x7f9d080cc9d0 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d05ffb700 1 -- 172.21.15.119:0/790507703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc00b7a0 con 0x7f9d080cc9d0 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.774+0000 7f9d05ffb700 1 -- 172.21.15.119:0/790507703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc00bb40 con 0x7f9d080cc9d0 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.775+0000 7f9d07fff700 1 -- 172.21.15.119:0/790507703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 msgr2=0x7f9d0815d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.775+0000 7f9d07fff700 1 --2- 172.21.15.119:0/790507703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d0815d280 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0098e0 tx=0x7f9cfc0091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.775+0000 7f9d07fff700 1 -- 172.21.15.119:0/790507703 shutdown_connections 2023-10-08T02:12:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.775+0000 7f9d07fff700 1 --2- 172.21.15.119:0/790507703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d0815d280 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.775+0000 7f9d07fff700 1 -- 172.21.15.119:0/790507703 >> 172.21.15.119:0/790507703 conn(0x7f9d080d0630 msgr2=0x7f9d080d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.775+0000 7f9d07fff700 1 -- 172.21.15.119:0/790507703 shutdown_connections 2023-10-08T02:12:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.775+0000 7f9d07fff700 1 -- 172.21.15.119:0/790507703 wait complete. 2023-10-08T02:12:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.777+0000 7f9d07fff700 1 Processor -- start 2023-10-08T02:12:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.777+0000 7f9d07fff700 1 -- start start 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d07fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d080cc600 0x7f9d0814e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d07fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d08154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d07fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d080cd0f0 0x7f9d08068b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d07fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08155180 con 0x7f9d080cc9d0 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d07fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08151a20 con 0x7f9d080cc600 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d07fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d08154bc0 con 0x7f9d080cd0f0 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d08154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d08154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35432/0 (socket says 172.21.15.119:35432) 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 -- 172.21.15.119:0/668537633 learned_addr learned my addr 172.21.15.119:0/668537633 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d06ffd700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d080cc600 0x7f9d0814e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d077fe700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d080cd0f0 0x7f9d08068b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 -- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d080cd0f0 msgr2=0x7f9d08068b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d080cd0f0 0x7f9d08068b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 -- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d080cc600 msgr2=0x7f9d0814e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d080cc600 0x7f9d0814e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 -- 172.21.15.119:0/668537633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009420 con 0x7f9d080cc9d0 2023-10-08T02:12:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d06ffd700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d080cc600 0x7f9d0814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:07.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.778+0000 7f9d067fc700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d08154290 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009f80 tx=0x7f9cfc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:07.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.779+0000 7f9ceeffd700 1 -- 172.21.15.119:0/668537633 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc01f070 con 0x7f9d080cc9d0 2023-10-08T02:12:07.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.779+0000 7f9ceeffd700 1 -- 172.21.15.119:0/668537633 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc000bc0 con 0x7f9d080cc9d0 2023-10-08T02:12:07.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.779+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d08154ea0 con 0x7f9d080cc9d0 2023-10-08T02:12:07.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.779+0000 7f9ceeffd700 1 -- 172.21.15.119:0/668537633 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc025420 con 0x7f9d080cc9d0 2023-10-08T02:12:07.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.779+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0814e810 con 0x7f9d080cc9d0 2023-10-08T02:12:07.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.780+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d08151510 con 0x7f9d080cc9d0 2023-10-08T02:12:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.782+0000 7f9ceeffd700 1 -- 172.21.15.119:0/668537633 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f9cfc023070 con 0x7f9d080cc9d0 2023-10-08T02:12:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.783+0000 7f9ceeffd700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9cf40670e0 0x7f9cf4069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.783+0000 7f9ceeffd700 1 -- 172.21.15.119:0/668537633 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f9cfc086010 con 0x7f9d080cc9d0 2023-10-08T02:12:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.783+0000 7f9d06ffd700 1 -- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9cf40670e0 msgr2=0x7f9cf4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.783+0000 7f9d06ffd700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9cf40670e0 0x7f9cf4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:07.784 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:07.784+0000 7f30e1e76180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:12:07.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.787+0000 7f9ceeffd700 1 -- 172.21.15.119:0/668537633 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9d08151510 con 0x7f9d080cc9d0 2023-10-08T02:12:07.843 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:07.843+0000 7fabedb09180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:12:07.871 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:07.870+0000 7f30e1e76180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:12:07.932 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:07.931+0000 7fabedb09180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:12:07.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.984+0000 7f9d06ffd700 1 -- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9cf40670e0 msgr2=0x7f9cf4069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:07.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.984+0000 7f9d06ffd700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9cf40670e0 0x7f9cf4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:07.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.986+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d08151510 con 0x7f9d080cc9d0 2023-10-08T02:12:07.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.987+0000 7f9ceeffd700 1 -- 172.21.15.119:0/668537633 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d08151510 con 0x7f9d080cc9d0 2023-10-08T02:12:07.987 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:12:07.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9cf40670e0 msgr2=0x7f9cf4069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:07.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9cf40670e0 0x7f9cf4069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 msgr2=0x7f9d08154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d08154290 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009f80 tx=0x7f9cfc017000 comp rx=0 tx=0).stop 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 shutdown_connections 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9cf40670e0 0x7f9cf4069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9d080cd0f0 0x7f9d08068b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9d080cc600 0x7f9d0814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 --2- 172.21.15.119:0/668537633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d080cc9d0 0x7f9d08154290 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 >> 172.21.15.119:0/668537633 conn(0x7f9d080d0630 msgr2=0x7f9d080b5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 shutdown_connections 2023-10-08T02:12:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:07.989+0000 7f9d07fff700 1 -- 172.21.15.119:0/668537633 wait complete. 2023-10-08T02:12:08.010 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T02:12:08.097 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:08.096+0000 7f30e1e76180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:08.165 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:08.165+0000 7fabedb09180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:08.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.197+0000 7f9e34f9d700 1 Processor -- start 2023-10-08T02:12:08.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.197+0000 7f9e34f9d700 1 -- start start 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.198+0000 7f9e34f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e300b4e70 0x7f9e300ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.198+0000 7f9e34f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e300b16d0 0x7f9e300b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.198+0000 7f9e34f9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e300b2080 con 0x7f9e3015d240 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.198+0000 7f9e34f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e300cfb00 con 0x7f9e300b16d0 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.198+0000 7f9e34f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300b5c40 con 0x7f9e300b4e70 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e300b16d0 0x7f9e300b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e300b16d0 0x7f9e300b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35440/0 (socket says 172.21.15.119:35440) 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2effd700 1 -- 172.21.15.119:0/4138028052 learned_addr learned my addr 172.21.15.119:0/4138028052 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:08.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2f7fe700 1 --2- 172.21.15.119:0/4138028052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e300b4e70 0x7f9e300ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2effd700 1 -- 172.21.15.119:0/4138028052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e300b4e70 msgr2=0x7f9e300ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:08.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2effd700 1 --2- 172.21.15.119:0/4138028052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e300b4e70 0x7f9e300ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2effd700 1 -- 172.21.15.119:0/4138028052 >> v1:172.21.15.184:6789/0 conn(0x7f9e3015d240 legacy=0x7f9e3015d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2effd700 1 -- 172.21.15.119:0/4138028052 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e300b5f20 con 0x7f9e300b16d0 2023-10-08T02:12:08.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.199+0000 7f9e2effd700 1 --2- 172.21.15.119:0/4138028052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e300b1b40 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f9e240099b0 tx=0x7f9e24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d24ad149153d5c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:08.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e2e7fc700 1 -- 172.21.15.119:0/4138028052 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e24009ea0 con 0x7f9e300b16d0 2023-10-08T02:12:08.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e2e7fc700 1 -- 172.21.15.119:0/4138028052 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2400b860 con 0x7f9e300b16d0 2023-10-08T02:12:08.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e2e7fc700 1 -- 172.21.15.119:0/4138028052 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400bc00 con 0x7f9e300b16d0 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e34f9d700 1 -- 172.21.15.119:0/4138028052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 msgr2=0x7f9e300b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/4138028052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e300b1b40 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f9e240099b0 tx=0x7f9e24009250 comp rx=0 tx=0).stop 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e34f9d700 1 -- 172.21.15.119:0/4138028052 shutdown_connections 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/4138028052 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e300b4e70 0x7f9e300ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/4138028052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e300b1b40 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e34f9d700 1 -- 172.21.15.119:0/4138028052 >> 172.21.15.119:0/4138028052 conn(0x7f9e300d10c0 msgr2=0x7f9e300affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e34f9d700 1 -- 172.21.15.119:0/4138028052 shutdown_connections 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.200+0000 7f9e34f9d700 1 -- 172.21.15.119:0/4138028052 wait complete. 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.202+0000 7f9e34f9d700 1 Processor -- start 2023-10-08T02:12:08.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.202+0000 7f9e34f9d700 1 -- start start 2023-10-08T02:12:08.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e34f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e3014c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e34f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e300b4e70 0x7f9e300c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e34f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e3015d240 0x7f9e300ca000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e34f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300c6db0 con 0x7f9e300b16d0 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e34f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e300c7cc0 con 0x7f9e300b4e70 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e34f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3014c5f0 con 0x7f9e3015d240 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e3014c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e3014c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35448/0 (socket says 172.21.15.119:35448) 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2f7fe700 1 -- 172.21.15.119:0/1865582308 learned_addr learned my addr 172.21.15.119:0/1865582308 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2effd700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e300b4e70 0x7f9e300c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2ffff700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e3015d240 0x7f9e300ca000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2f7fe700 1 -- 172.21.15.119:0/1865582308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e3015d240 msgr2=0x7f9e300ca000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2f7fe700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e3015d240 0x7f9e300ca000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2f7fe700 1 -- 172.21.15.119:0/1865582308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e300b4e70 msgr2=0x7f9e300c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2f7fe700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e300b4e70 0x7f9e300c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.203+0000 7f9e2f7fe700 1 -- 172.21.15.119:0/1865582308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240094f0 con 0x7f9e300b16d0 2023-10-08T02:12:08.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.204+0000 7f9e2f7fe700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e3014c0b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000ec80 tx=0x7f9e200091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:08.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.204+0000 7f9e177fe700 1 -- 172.21.15.119:0/1865582308 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e20009690 con 0x7f9e300b16d0 2023-10-08T02:12:08.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.204+0000 7f9e177fe700 1 -- 172.21.15.119:0/1865582308 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2000c850 con 0x7f9e300b16d0 2023-10-08T02:12:08.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.204+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e300c67f0 con 0x7f9e300b16d0 2023-10-08T02:12:08.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.204+0000 7f9e177fe700 1 -- 172.21.15.119:0/1865582308 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2000cc10 con 0x7f9e300b16d0 2023-10-08T02:12:08.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.204+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e300c7950 con 0x7f9e300b16d0 2023-10-08T02:12:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.205+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e30088990 con 0x7f9e300b16d0 2023-10-08T02:12:08.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.207+0000 7f9e177fe700 1 -- 172.21.15.119:0/1865582308 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f9e20013470 con 0x7f9e300b16d0 2023-10-08T02:12:08.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.208+0000 7f9e177fe700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9e1c0670e0 0x7f9e1c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.208+0000 7f9e177fe700 1 -- 172.21.15.119:0/1865582308 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f9e20084c80 con 0x7f9e300b16d0 2023-10-08T02:12:08.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.208+0000 7f9e2effd700 1 -- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9e1c0670e0 msgr2=0x7f9e1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:08.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.208+0000 7f9e2effd700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9e1c0670e0 0x7f9e1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:08.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.212+0000 7f9e177fe700 1 -- 172.21.15.119:0/1865582308 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9e20055430 con 0x7f9e300b16d0 2023-10-08T02:12:08.277 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:08.276+0000 7f30e1e76180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:12:08.351 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:08.350+0000 7fabedb09180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:12:08.370 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:08.369+0000 7f30e1e76180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:12:08.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.409+0000 7f9e2effd700 1 -- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9e1c0670e0 msgr2=0x7f9e1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:08.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.409+0000 7f9e2effd700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9e1c0670e0 0x7f9e1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:08.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.411+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e3004bb80 con 0x7f9e300b16d0 2023-10-08T02:12:08.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.411+0000 7f9e177fe700 1 -- 172.21.15.119:0/1865582308 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e20054fc0 con 0x7f9e300b16d0 2023-10-08T02:12:08.412 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T02:12:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9e1c0670e0 msgr2=0x7f9e1c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9e1c0670e0 0x7f9e1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 msgr2=0x7f9e3014c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e3014c0b0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f9e2000ec80 tx=0x7f9e200091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 shutdown_connections 2023-10-08T02:12:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9e1c0670e0 0x7f9e1c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9e3015d240 0x7f9e300ca000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e300b16d0 0x7f9e3014c0b0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 --2- 172.21.15.119:0/1865582308 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9e300b4e70 0x7f9e300c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 >> 172.21.15.119:0/1865582308 conn(0x7f9e300d10c0 msgr2=0x7f9e3015c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 shutdown_connections 2023-10-08T02:12:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.414+0000 7f9e34f9d700 1 -- 172.21.15.119:0/1865582308 wait complete. 2023-10-08T02:12:08.433 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T02:12:08.449 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:08.448+0000 7fabedb09180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:12:08.534 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:08.533+0000 7f30e1e76180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:12:08.618 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:08.618+0000 7fabedb09180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:12:08.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.623+0000 7f0c6619d700 1 Processor -- start 2023-10-08T02:12:08.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.623+0000 7f0c6619d700 1 -- start start 2023-10-08T02:12:08.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.624+0000 7f0c6619d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0c600b1b40 0x7f0c600b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.624+0000 7f0c6619d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c600cfb00 con 0x7f0c600b4e70 2023-10-08T02:12:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.624+0000 7f0c6619d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c600b5c40 con 0x7f0c600b16d0 2023-10-08T02:12:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.624+0000 7f0c6619d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c600b5f20 con 0x7f0c600b1b40 2023-10-08T02:12:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.624+0000 7f0c6519b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0c600b4e70 0x7f0c600ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59142/0 (socket says 172.21.15.119:59142) 2023-10-08T02:12:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.624+0000 7f0c6519b700 1 -- 172.21.15.119:0/2926092435 learned_addr learned my addr 172.21.15.119:0/2926092435 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.624+0000 7f0c6599c700 1 --2- 172.21.15.119:0/2926092435 >> v2:172.21.15.184:3300/0 conn(0x7f0c600b1b40 0x7f0c600b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.625+0000 7f0c57fff700 1 -- 172.21.15.119:0/2926092435 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1822931942 0 0) 0x7f0c600cfb00 con 0x7f0c600b4e70 2023-10-08T02:12:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.625+0000 7f0c57fff700 1 -- 172.21.15.119:0/2926092435 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c4c003570 con 0x7f0c600b4e70 2023-10-08T02:12:08.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.625+0000 7f0c6599c700 1 -- 172.21.15.119:0/2926092435 >> v1:172.21.15.184:6789/0 conn(0x7f0c600b16d0 legacy=0x7f0c600b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.625+0000 7f0c6599c700 1 -- 172.21.15.119:0/2926092435 >> v1:172.21.15.119:6789/0 conn(0x7f0c600b4e70 legacy=0x7f0c600ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.625+0000 7f0c6599c700 1 -- 172.21.15.119:0/2926092435 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c600b6200 con 0x7f0c600b1b40 2023-10-08T02:12:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.625+0000 7f0c6599c700 1 --2- 172.21.15.119:0/2926092435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 0x7f0c600b3ac0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009e60 tx=0x7f0c58009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64976bbc9014e84 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.625+0000 7f0c57fff700 1 -- 172.21.15.119:0/2926092435 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800e070 con 0x7f0c600b1b40 2023-10-08T02:12:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c57fff700 1 -- 172.21.15.119:0/2926092435 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5800bb60 con 0x7f0c600b1b40 2023-10-08T02:12:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c57fff700 1 -- 172.21.15.119:0/2926092435 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58012430 con 0x7f0c600b1b40 2023-10-08T02:12:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c6619d700 1 -- 172.21.15.119:0/2926092435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 msgr2=0x7f0c600b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c6619d700 1 --2- 172.21.15.119:0/2926092435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 0x7f0c600b3ac0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009e60 tx=0x7f0c58009780 comp rx=0 tx=0).stop 2023-10-08T02:12:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c6619d700 1 -- 172.21.15.119:0/2926092435 shutdown_connections 2023-10-08T02:12:08.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c6619d700 1 --2- 172.21.15.119:0/2926092435 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 0x7f0c600b3ac0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c6619d700 1 -- 172.21.15.119:0/2926092435 >> 172.21.15.119:0/2926092435 conn(0x7f0c600d10c0 msgr2=0x7f0c600affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:08.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c6619d700 1 -- 172.21.15.119:0/2926092435 shutdown_connections 2023-10-08T02:12:08.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.626+0000 7f0c6619d700 1 -- 172.21.15.119:0/2926092435 wait complete. 2023-10-08T02:12:08.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.628+0000 7f0c6619d700 1 Processor -- start 2023-10-08T02:12:08.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.628+0000 7f0c6619d700 1 -- start start 2023-10-08T02:12:08.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6619d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c600b16d0 0x7f0c600c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6619d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 0x7f0c600cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6619d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c600b4e70 0x7f0c60169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6619d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600cb230 con 0x7f0c600b4e70 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6619d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6016ac40 con 0x7f0c600b1b40 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6619d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600c9da0 con 0x7f0c600b16d0 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c600b4e70 0x7f0c60169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6599c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c600b4e70 0x7f0c60169c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35452/0 (socket says 172.21.15.119:35452) 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6599c700 1 -- 172.21.15.119:0/3261162149 learned_addr learned my addr 172.21.15.119:0/3261162149 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6519b700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c600b16d0 0x7f0c600c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6499a700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 0x7f0c600cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6599c700 1 -- 172.21.15.119:0/3261162149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c600b16d0 msgr2=0x7f0c600c9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6599c700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c600b16d0 0x7f0c600c9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6599c700 1 -- 172.21.15.119:0/3261162149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 msgr2=0x7f0c600cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.629+0000 7f0c6599c700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 0x7f0c600cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.630+0000 7f0c6599c700 1 -- 172.21.15.119:0/3261162149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c58009a20 con 0x7f0c600b4e70 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.630+0000 7f0c6599c700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c600b4e70 0x7f0c60169c40 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0c60070dd0 tx=0x7f0c58019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.630+0000 7f0c557fa700 1 -- 172.21.15.119:0/3261162149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580032b0 con 0x7f0c600b4e70 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.630+0000 7f0c557fa700 1 -- 172.21.15.119:0/3261162149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c580263f0 con 0x7f0c600b4e70 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.630+0000 7f0c557fa700 1 -- 172.21.15.119:0/3261162149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580267b0 con 0x7f0c600b4e70 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.630+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6016a6e0 con 0x7f0c600b4e70 2023-10-08T02:12:08.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.630+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c600cb740 con 0x7f0c600b4e70 2023-10-08T02:12:08.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.631+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6016aa80 con 0x7f0c600b4e70 2023-10-08T02:12:08.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.632+0000 7f0c557fa700 1 -- 172.21.15.119:0/3261162149 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f0c58026b60 con 0x7f0c600b4e70 2023-10-08T02:12:08.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.633+0000 7f0c557fa700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c440670e0 0x7f0c44069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:08.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.633+0000 7f0c6519b700 1 -- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c440670e0 msgr2=0x7f0c44069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:08.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.633+0000 7f0c557fa700 1 -- 172.21.15.119:0/3261162149 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f0c58086b50 con 0x7f0c600b4e70 2023-10-08T02:12:08.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.633+0000 7f0c6519b700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c440670e0 0x7f0c44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:08.635 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:08.634+0000 7f30e1e76180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:12:08.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.637+0000 7f0c557fa700 1 -- 172.21.15.119:0/3261162149 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0c580536c0 con 0x7f0c600b4e70 2023-10-08T02:12:08.724 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:08.723+0000 7fabedb09180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:12:08.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.834+0000 7f0c6519b700 1 -- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c440670e0 msgr2=0x7f0c44069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:08.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.834+0000 7f0c6519b700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c440670e0 0x7f0c44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:08.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.836+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c600cb960 con 0x7f0c600b4e70 2023-10-08T02:12:08.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.836+0000 7f0c557fa700 1 -- 172.21.15.119:0/3261162149 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0c58056ce0 con 0x7f0c600b4e70 2023-10-08T02:12:08.837 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c440670e0 msgr2=0x7f0c44069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c440670e0 0x7f0c44069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c600b4e70 msgr2=0x7f0c60169c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c600b4e70 0x7f0c60169c40 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0c60070dd0 tx=0x7f0c58019000 comp rx=0 tx=0).stop 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 shutdown_connections 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0c440670e0 0x7f0c44069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0c600b16d0 0x7f0c600c9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0c600b4e70 0x7f0c60169c40 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 --2- 172.21.15.119:0/3261162149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0c600b1b40 0x7f0c600cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:08.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 >> 172.21.15.119:0/3261162149 conn(0x7f0c600d10c0 msgr2=0x7f0c601590e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 shutdown_connections 2023-10-08T02:12:08.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:08.839+0000 7f0c6619d700 1 -- 172.21.15.119:0/3261162149 wait complete. 2023-10-08T02:12:08.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:12:09.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.085+0000 7fb57359d700 1 Processor -- start 2023-10-08T02:12:09.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.085+0000 7fb57359d700 1 -- start start 2023-10-08T02:12:09.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb57359d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c1595a0 0x7fb56c0b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb57359d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb56c0c8830 0x7fb56c159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb57359d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb56c0b1410 con 0x7fb56c0c6c60 2023-10-08T02:12:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb57359d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56c0b2b90 con 0x7fb56c0c8830 2023-10-08T02:12:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb57359d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c063230 con 0x7fb56c1595a0 2023-10-08T02:12:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb571d9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb56c0c6c60 0x7fb56c0c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59154/0 (socket says 172.21.15.119:59154) 2023-10-08T02:12:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb572d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb56c0c8830 0x7fb56c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb571d9a700 1 -- 172.21.15.119:0/589394219 learned_addr learned my addr 172.21.15.119:0/589394219 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:09.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb57259b700 1 --2- 172.21.15.119:0/589394219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c1595a0 0x7fb56c0b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb572d9c700 1 -- 172.21.15.119:0/589394219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c1595a0 msgr2=0x7fb56c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.087+0000 7fb572d9c700 1 --2- 172.21.15.119:0/589394219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c1595a0 0x7fb56c0b18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.088+0000 7fb572d9c700 1 -- 172.21.15.119:0/589394219 >> v1:172.21.15.119:6789/0 conn(0x7fb56c0c6c60 legacy=0x7fb56c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.088+0000 7fb572d9c700 1 -- 172.21.15.119:0/589394219 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c0634f0 con 0x7fb56c0c8830 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.088+0000 7fb571599700 1 -- 172.21.15.119:0/589394219 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923452917 0 0) 0x7fb56c0b1410 con 0x7fb56c0c6c60 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.088+0000 7fb572d9c700 1 --2- 172.21.15.119:0/589394219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c0c8830 0x7fb56c159970 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb564009e60 tx=0x7fb564009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a801f491936443 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.088+0000 7fb571599700 1 -- 172.21.15.119:0/589394219 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400e070 con 0x7fb56c0c8830 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.088+0000 7fb571599700 1 -- 172.21.15.119:0/589394219 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56400bb60 con 0x7fb56c0c8830 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.088+0000 7fb571599700 1 -- 172.21.15.119:0/589394219 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564012430 con 0x7fb56c0c8830 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.089+0000 7fb57359d700 1 -- 172.21.15.119:0/589394219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c0c8830 msgr2=0x7fb56c159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.089+0000 7fb57359d700 1 --2- 172.21.15.119:0/589394219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c0c8830 0x7fb56c159970 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fb564009e60 tx=0x7fb564009780 comp rx=0 tx=0).stop 2023-10-08T02:12:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.089+0000 7fb57359d700 1 -- 172.21.15.119:0/589394219 shutdown_connections 2023-10-08T02:12:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.089+0000 7fb57359d700 1 --2- 172.21.15.119:0/589394219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c0c8830 0x7fb56c159970 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.089+0000 7fb57359d700 1 --2- 172.21.15.119:0/589394219 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c1595a0 0x7fb56c0b18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.089+0000 7fb57359d700 1 -- 172.21.15.119:0/589394219 >> 172.21.15.119:0/589394219 conn(0x7fb56c06ddb0 msgr2=0x7fb56c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.089+0000 7fb57359d700 1 -- 172.21.15.119:0/589394219 shutdown_connections 2023-10-08T02:12:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.089+0000 7fb57359d700 1 -- 172.21.15.119:0/589394219 wait complete. 2023-10-08T02:12:09.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.090+0000 7fb57359d700 1 Processor -- start 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.090+0000 7fb57359d700 1 -- start start 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb57359d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c0c6c60 0x7fb56c14d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb57359d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0c8830 0x7fb56c13c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb57359d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c1595a0 0x7fb56c13ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb57359d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c14de30 con 0x7fb56c1595a0 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb57359d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c14c030 con 0x7fb56c0c8830 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb57359d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c14d020 con 0x7fb56c0c6c60 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb572d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c1595a0 0x7fb56c13ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb572d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c1595a0 0x7fb56c13ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35468/0 (socket says 172.21.15.119:35468) 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb572d9c700 1 -- 172.21.15.119:0/4229164884 learned_addr learned my addr 172.21.15.119:0/4229164884 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.091+0000 7fb57259b700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c0c6c60 0x7fb56c14d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.092+0000 7fb571d9a700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0c8830 0x7fb56c13c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.092+0000 7fb572d9c700 1 -- 172.21.15.119:0/4229164884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c0c6c60 msgr2=0x7fb56c14d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.092+0000 7fb572d9c700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c0c6c60 0x7fb56c14d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.092+0000 7fb572d9c700 1 -- 172.21.15.119:0/4229164884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0c8830 msgr2=0x7fb56c13c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.092+0000 7fb572d9c700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0c8830 0x7fb56c13c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.092+0000 7fb572d9c700 1 -- 172.21.15.119:0/4229164884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb564009a20 con 0x7fb56c1595a0 2023-10-08T02:12:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.092+0000 7fb572d9c700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c1595a0 0x7fb56c13ccd0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb564013000 tx=0x7fb564019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.093+0000 7fb5627fc700 1 -- 172.21.15.119:0/4229164884 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564012ea0 con 0x7fb56c1595a0 2023-10-08T02:12:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.093+0000 7fb5627fc700 1 -- 172.21.15.119:0/4229164884 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb564027470 con 0x7fb56c1595a0 2023-10-08T02:12:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.093+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c14c280 con 0x7fb56c1595a0 2023-10-08T02:12:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.093+0000 7fb5627fc700 1 -- 172.21.15.119:0/4229164884 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb564027830 con 0x7fb56c1595a0 2023-10-08T02:12:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.093+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c14c840 con 0x7fb56c1595a0 2023-10-08T02:12:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.094+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c04c3c0 con 0x7fb56c1595a0 2023-10-08T02:12:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.095+0000 7fb5627fc700 1 -- 172.21.15.119:0/4229164884 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fb564024070 con 0x7fb56c1595a0 2023-10-08T02:12:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.095+0000 7fb5627fc700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb5500670e0 0x7fb550069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.095+0000 7fb5627fc700 1 -- 172.21.15.119:0/4229164884 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fb5640871c0 con 0x7fb56c1595a0 2023-10-08T02:12:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.096+0000 7fb57259b700 1 -- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb5500670e0 msgr2=0x7fb550069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.096+0000 7fb57259b700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb5500670e0 0x7fb550069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:09.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.100+0000 7fb5627fc700 1 -- 172.21.15.119:0/4229164884 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb564053e40 con 0x7fb56c1595a0 2023-10-08T02:12:09.120 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:09.119+0000 7f30e1e76180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:12:09.215 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:09.214+0000 7f30e1e76180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:12:09.230 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:09.229+0000 7fabedb09180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:12:09.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.297+0000 7fb57259b700 1 -- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb5500670e0 msgr2=0x7fb550069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:09.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.297+0000 7fb57259b700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb5500670e0 0x7fb550069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:09.305 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:09.304+0000 7f30e1e76180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:12:09.325 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:09.325+0000 7fabedb09180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:12:09.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.340+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb56c0637d0 con 0x7fb56c1595a0 2023-10-08T02:12:09.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.341+0000 7fb5627fc700 1 -- 172.21.15.119:0/4229164884 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fb564057460 con 0x7fb56c1595a0 2023-10-08T02:12:09.341 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:09.341 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/603902131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":603902131},{"type":"v1","addr":"172.21.15.184:6834","nonce":603902131}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3930162267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3930162267},{"type":"v1","addr":"172.21.15.119:6834","nonce":3930162267}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/237096421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":237096421},{"type":"v1","addr":"172.21.15.119:6835","nonce":237096421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:11:41.545410+0000","modified":"2023-10-08T02:11:45.111313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8667":{"gid":8667,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3810992025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3810992025},{"type":"v1","addr":"172.21.15.184:6835","nonce":3810992025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb5500670e0 msgr2=0x7fb550069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb5500670e0 0x7fb550069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c1595a0 msgr2=0x7fb56c13ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c1595a0 0x7fb56c13ccd0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb564013000 tx=0x7fb564019000 comp rx=0 tx=0).stop 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 shutdown_connections 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb5500670e0 0x7fb550069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb56c0c6c60 0x7fb56c14d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb56c1595a0 0x7fb56c13ccd0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 --2- 172.21.15.119:0/4229164884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb56c0c8830 0x7fb56c13c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 >> 172.21.15.119:0/4229164884 conn(0x7fb56c06ddb0 msgr2=0x7fb56c157630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:09.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 shutdown_connections 2023-10-08T02:12:09.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.343+0000 7fb57359d700 1 -- 172.21.15.119:0/4229164884 wait complete. 2023-10-08T02:12:09.345 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 71 2023-10-08T02:12:09.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:12:09.433 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:09.433+0000 7fabedb09180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:12:09.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.558+0000 7f7273fff700 1 Processor -- start 2023-10-08T02:12:09.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.558+0000 7f7273fff700 1 -- start start 2023-10-08T02:12:09.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.559+0000 7f7273fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72741595a0 0x7f72740b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.559+0000 7f7273fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c8830 0x7f7274159970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.559+0000 7f7273fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72740b1410 con 0x7f72740c6c60 2023-10-08T02:12:09.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.559+0000 7f7273fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72740b2b90 con 0x7f72741595a0 2023-10-08T02:12:09.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.559+0000 7f7273fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7274063230 con 0x7f72740c8830 2023-10-08T02:12:09.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7272ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72741595a0 0x7f72740b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7272ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f72741595a0 0x7f72740b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35482/0 (socket says 172.21.15.119:35482) 2023-10-08T02:12:09.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7272ffd700 1 -- 172.21.15.119:0/1330686064 learned_addr learned my addr 172.21.15.119:0/1330686064 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:09.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f72737fe700 1 --2- 172.21.15.119:0/1330686064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c8830 0x7f7274159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7272ffd700 1 -- 172.21.15.119:0/1330686064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c8830 msgr2=0x7f7274159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:09.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7272ffd700 1 --2- 172.21.15.119:0/1330686064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c8830 0x7f7274159970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7272ffd700 1 -- 172.21.15.119:0/1330686064 >> v1:172.21.15.184:6789/0 conn(0x7f72740c6c60 legacy=0x7f72740c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7272ffd700 1 -- 172.21.15.119:0/1330686064 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72740634f0 con 0x7f72741595a0 2023-10-08T02:12:09.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7271ffb700 1 -- 172.21.15.119:0/1330686064 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 769694428 0 0) 0x7f72740b1410 con 0x7f72740c6c60 2023-10-08T02:12:09.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.560+0000 7f7272ffd700 1 --2- 172.21.15.119:0/1330686064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72741595a0 0x7f72740b18a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f72640099b0 tx=0x7f7264009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7719b9111724f453 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:09.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7271ffb700 1 -- 172.21.15.119:0/1330686064 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264009ea0 con 0x7f72741595a0 2023-10-08T02:12:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7271ffb700 1 -- 172.21.15.119:0/1330686064 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726400b860 con 0x7f72741595a0 2023-10-08T02:12:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7271ffb700 1 -- 172.21.15.119:0/1330686064 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726400bc00 con 0x7f72741595a0 2023-10-08T02:12:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7273fff700 1 -- 172.21.15.119:0/1330686064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72741595a0 msgr2=0x7f72740b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7273fff700 1 --2- 172.21.15.119:0/1330686064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72741595a0 0x7f72740b18a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f72640099b0 tx=0x7f7264009250 comp rx=0 tx=0).stop 2023-10-08T02:12:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7273fff700 1 -- 172.21.15.119:0/1330686064 shutdown_connections 2023-10-08T02:12:09.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7273fff700 1 --2- 172.21.15.119:0/1330686064 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c8830 0x7f7274159970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7273fff700 1 --2- 172.21.15.119:0/1330686064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72741595a0 0x7f72740b18a0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7273fff700 1 -- 172.21.15.119:0/1330686064 >> 172.21.15.119:0/1330686064 conn(0x7f727406ddb0 msgr2=0x7f727406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:09.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7273fff700 1 -- 172.21.15.119:0/1330686064 shutdown_connections 2023-10-08T02:12:09.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.561+0000 7f7273fff700 1 -- 172.21.15.119:0/1330686064 wait complete. 2023-10-08T02:12:09.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.563+0000 7f7273fff700 1 Processor -- start 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.563+0000 7f7273fff700 1 -- start start 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7273fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c6c60 0x7f7274142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7273fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72740c8830 0x7f727415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7273fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72741595a0 0x7f7274155b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7273fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727415a9d0 con 0x7f72740c8830 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7273fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72740c5380 con 0x7f72741595a0 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7273fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7274159e50 con 0x7f72740c6c60 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7272ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c6c60 0x7f7274142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f72727fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72740c8830 0x7f727415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7272ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c6c60 0x7f7274142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:38920/0 (socket says 172.21.15.119:38920) 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f7272ffd700 1 -- 172.21.15.119:0/3426371746 learned_addr learned my addr 172.21.15.119:0/3426371746 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.564+0000 7f72737fe700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72741595a0 0x7f7274155b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f72727fc700 1 -- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c6c60 msgr2=0x7f7274142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f72727fc700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c6c60 0x7f7274142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f72727fc700 1 -- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72741595a0 msgr2=0x7f7274155b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f72727fc700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72741595a0 0x7f7274155b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f72727fc700 1 -- 172.21.15.119:0/3426371746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72640094f0 con 0x7f72740c8830 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f7272ffd700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c6c60 0x7f7274142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f72737fe700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72741595a0 0x7f7274155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f72727fc700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72740c8830 0x7f727415b4c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f72680011e0 tx=0x7f7268007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:09.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f725affd700 1 -- 172.21.15.119:0/3426371746 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268011590 con 0x7f72740c8830 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f725affd700 1 -- 172.21.15.119:0/3426371746 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7268011710 con 0x7f72740c8830 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f725affd700 1 -- 172.21.15.119:0/3426371746 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268011ad0 con 0x7f72740c8830 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727415a130 con 0x7f72740c8830 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.565+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727415a6f0 con 0x7f72740c8830 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.566+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727404c3c0 con 0x7f72740c8830 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.568+0000 7f725affd700 1 -- 172.21.15.119:0/3426371746 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f726801b3b0 con 0x7f72740c8830 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.569+0000 7f725affd700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72600670e0 0x7f7260069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.569+0000 7f725affd700 1 -- 172.21.15.119:0/3426371746 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f7268083fb0 con 0x7f72740c8830 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.569+0000 7f7272ffd700 1 -- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72600670e0 msgr2=0x7f7260069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:09.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.569+0000 7f7272ffd700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72600670e0 0x7f7260069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:09.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.573+0000 7f725affd700 1 -- 172.21.15.119:0/3426371746 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7268050c30 con 0x7f72740c8830 2023-10-08T02:12:09.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.769+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72740637d0 con 0x7f72740c8830 2023-10-08T02:12:09.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.770+0000 7f725affd700 1 -- 172.21.15.119:0/3426371746 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f7268017070 con 0x7f72740c8830 2023-10-08T02:12:09.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.770+0000 7f7272ffd700 1 -- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72600670e0 msgr2=0x7f7260069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:09.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.770+0000 7f7272ffd700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72600670e0 0x7f7260069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:09.771 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:09.771 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":158,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:12:03.296789+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:11:39.192970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T02:11:40.531612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2185260495":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/3673331094":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/1337159806":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/294111303":"2023-10-09T02:12:03.296742+0000","172.21.15.119:6837/85511":"2023-10-09T02:12:03.296742+0000","172.21.15.119:6836/85511":"2023-10-09T02:12:03.296742+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:12:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72600670e0 msgr2=0x7f7260069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72600670e0 0x7f7260069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72740c8830 msgr2=0x7f727415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:09.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72740c8830 0x7f727415b4c0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f72680011e0 tx=0x7f7268007d30 comp rx=0 tx=0).stop 2023-10-08T02:12:09.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 shutdown_connections 2023-10-08T02:12:09.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f72600670e0 0x7f7260069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f72740c6c60 0x7f7274142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f72741595a0 0x7f7274155b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 --2- 172.21.15.119:0/3426371746 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72740c8830 0x7f727415b4c0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:09.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.772+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 >> 172.21.15.119:0/3426371746 conn(0x7f727406ddb0 msgr2=0x7f72740cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:09.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.773+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 shutdown_connections 2023-10-08T02:12:09.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.773+0000 7f7273fff700 1 -- 172.21.15.119:0/3426371746 wait complete. 2023-10-08T02:12:09.791 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T02:12:09.791 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:12:09.933 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:09.932+0000 7f30e1e76180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:12:10.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.998+0000 7f256499d700 1 Processor -- start 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:09.998+0000 7f256499d700 1 -- start start 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.000+0000 7f256499d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f256015de70 0x7f256006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.000+0000 7f256499d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f25600696c0 0x7f256006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.000+0000 7f256499d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f256006e8c0 con 0x7f256015daa0 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.000+0000 7f256499d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25600ca740 con 0x7f25600696c0 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.000+0000 7f256499d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256006e5e0 con 0x7f256015de70 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.001+0000 7f255effd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f256015de70 0x7f256006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.001+0000 7f255effd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f256015de70 0x7f256006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54340/0 (socket says 172.21.15.119:54340) 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.001+0000 7f255effd700 1 -- 172.21.15.119:0/1322116862 learned_addr learned my addr 172.21.15.119:0/1322116862 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:10.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.001+0000 7f255ffff700 1 --2- 172.21.15.119:0/1322116862 >> v2:172.21.15.119:3300/0 conn(0x7f25600696c0 0x7f256006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.001+0000 7f255effd700 1 -- 172.21.15.119:0/1322116862 >> v2:172.21.15.119:3300/0 conn(0x7f25600696c0 msgr2=0x7f256006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.001+0000 7f255effd700 1 --2- 172.21.15.119:0/1322116862 >> v2:172.21.15.119:3300/0 conn(0x7f25600696c0 0x7f256006ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.001+0000 7f255effd700 1 -- 172.21.15.119:0/1322116862 >> v1:172.21.15.184:6789/0 conn(0x7f256015daa0 legacy=0x7f25600c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:10.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.001+0000 7f255effd700 1 -- 172.21.15.119:0/1322116862 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25600ca0f0 con 0x7f256015de70 2023-10-08T02:12:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.002+0000 7f255effd700 1 --2- 172.21.15.119:0/1322116862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f256015de70 0x7f256006e000 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2560064030 tx=0x7f2548009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b84312c195ff3aa1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.002+0000 7f255e7fc700 1 -- 172.21.15.119:0/1322116862 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254800e070 con 0x7f256015de70 2023-10-08T02:12:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.002+0000 7f255e7fc700 1 -- 172.21.15.119:0/1322116862 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254800ba10 con 0x7f256015de70 2023-10-08T02:12:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.002+0000 7f255e7fc700 1 -- 172.21.15.119:0/1322116862 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25480092e0 con 0x7f256015de70 2023-10-08T02:12:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.003+0000 7f256499d700 1 -- 172.21.15.119:0/1322116862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f256015de70 msgr2=0x7f256006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.003+0000 7f256499d700 1 --2- 172.21.15.119:0/1322116862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f256015de70 0x7f256006e000 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2560064030 tx=0x7f2548009840 comp rx=0 tx=0).stop 2023-10-08T02:12:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.003+0000 7f256499d700 1 -- 172.21.15.119:0/1322116862 shutdown_connections 2023-10-08T02:12:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.003+0000 7f256499d700 1 --2- 172.21.15.119:0/1322116862 >> v2:172.21.15.119:3300/0 conn(0x7f25600696c0 0x7f256006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.003+0000 7f256499d700 1 --2- 172.21.15.119:0/1322116862 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f256015de70 0x7f256006e000 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.003+0000 7f256499d700 1 -- 172.21.15.119:0/1322116862 >> 172.21.15.119:0/1322116862 conn(0x7f25600cc1f0 msgr2=0x7f25600cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:10.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.006+0000 7f256499d700 1 -- 172.21.15.119:0/1322116862 shutdown_connections 2023-10-08T02:12:10.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.010+0000 7f256499d700 1 -- 172.21.15.119:0/1322116862 wait complete. 2023-10-08T02:12:10.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.012+0000 7f256499d700 1 Processor -- start 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.012+0000 7f256499d700 1 -- start start 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.013+0000 7f256499d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25600696c0 0x7f2560146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.013+0000 7f256499d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f256015daa0 0x7f25601471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.013+0000 7f256499d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256015de70 0x7f256014b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.013+0000 7f256499d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601423b0 con 0x7f256015de70 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.013+0000 7f256499d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b1870 con 0x7f25600696c0 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.013+0000 7f256499d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25600b0f80 con 0x7f256015daa0 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255effd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f256015daa0 0x7f25601471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255effd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f256015daa0 0x7f25601471b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:38930/0 (socket says 172.21.15.119:38930) 2023-10-08T02:12:10.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255effd700 1 -- 172.21.15.119:0/433274214 learned_addr learned my addr 172.21.15.119:0/433274214 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255ffff700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256015de70 0x7f256014b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255effd700 1 -- 172.21.15.119:0/433274214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25600696c0 msgr2=0x7f2560146c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255effd700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25600696c0 0x7f2560146c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255effd700 1 -- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256015de70 msgr2=0x7f256014b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255effd700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256015de70 0x7f256014b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.014+0000 7f255effd700 1 -- 172.21.15.119:0/433274214 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548009ae0 con 0x7f256015daa0 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.015+0000 7f255effd700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f256015daa0 0x7f25601471b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f25480093e0 tx=0x7f2548017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.015+0000 7f25577fe700 1 -- 172.21.15.119:0/433274214 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548012970 con 0x7f256015daa0 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.015+0000 7f25577fe700 1 -- 172.21.15.119:0/433274214 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2548012af0 con 0x7f256015daa0 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.015+0000 7f25577fe700 1 -- 172.21.15.119:0/433274214 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254801d420 con 0x7f256015daa0 2023-10-08T02:12:10.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.015+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25601420d0 con 0x7f256015daa0 2023-10-08T02:12:10.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.015+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25600b1500 con 0x7f256015daa0 2023-10-08T02:12:10.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.017+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256004c3c0 con 0x7f256015daa0 2023-10-08T02:12:10.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.018+0000 7f25577fe700 1 -- 172.21.15.119:0/433274214 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f254801d790 con 0x7f256015daa0 2023-10-08T02:12:10.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.019+0000 7f25577fe700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25400670e0 0x7f2540069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.019+0000 7f25577fe700 1 -- 172.21.15.119:0/433274214 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f25480861e0 con 0x7f256015daa0 2023-10-08T02:12:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.019+0000 7f255f7fe700 1 -- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25400670e0 msgr2=0x7f2540069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:10.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.019+0000 7f255f7fe700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25400670e0 0x7f2540069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:10.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.023+0000 7f25577fe700 1 -- 172.21.15.119:0/433274214 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2548052e10 con 0x7f256015daa0 2023-10-08T02:12:10.080 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:10.079+0000 7fabedb09180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:12:10.106 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:10.105+0000 7f30e1e76180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:10.203 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:10.202+0000 7f30e1e76180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:12:10.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.220+0000 7f255f7fe700 1 -- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25400670e0 msgr2=0x7f2540069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:10.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.220+0000 7f255f7fe700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25400670e0 0x7f2540069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:10.257 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:10.257+0000 7fabedb09180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:10.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.263+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25600ca3d0 con 0x7f256015daa0 2023-10-08T02:12:10.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.264+0000 7f25577fe700 1 -- 172.21.15.119:0/433274214 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7f2548056430 con 0x7f256015daa0 2023-10-08T02:12:10.264 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:10.265 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/603902131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":603902131},{"type":"v1","addr":"172.21.15.184:6834","nonce":603902131}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3930162267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3930162267},{"type":"v1","addr":"172.21.15.119:6834","nonce":3930162267}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/237096421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":237096421},{"type":"v1","addr":"172.21.15.119:6835","nonce":237096421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:11:41.545410+0000","modified":"2023-10-08T02:11:45.111313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8667":{"gid":8667,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3810992025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3810992025},{"type":"v1","addr":"172.21.15.184:6835","nonce":3810992025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T02:12:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25400670e0 msgr2=0x7f2540069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25400670e0 0x7f2540069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f256015daa0 msgr2=0x7f25601471b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f256015daa0 0x7f25601471b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f25480093e0 tx=0x7f2548017000 comp rx=0 tx=0).stop 2023-10-08T02:12:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 shutdown_connections 2023-10-08T02:12:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f25400670e0 0x7f2540069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f256015de70 0x7f256014b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f25600696c0 0x7f2560146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 --2- 172.21.15.119:0/433274214 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f256015daa0 0x7f25601471b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 >> 172.21.15.119:0/433274214 conn(0x7f25600cc1f0 msgr2=0x7f25600c1270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 shutdown_connections 2023-10-08T02:12:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.266+0000 7f256499d700 1 -- 172.21.15.119:0/433274214 wait complete. 2023-10-08T02:12:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 71 2023-10-08T02:12:10.285 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:12:10.360 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:10.360+0000 7fabedb09180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:12:10.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.472+0000 7fde655ff700 1 Processor -- start 2023-10-08T02:12:10.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.472+0000 7fde655ff700 1 -- start start 2023-10-08T02:12:10.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde655ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fde6006bcd0 0x7fde600caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde655ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde6015df50 0x7fde6015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde655ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde6015d7e0 con 0x7fde6006c0a0 2023-10-08T02:12:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde655ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6015dac0 con 0x7fde6015df50 2023-10-08T02:12:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde655ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6015aa30 con 0x7fde6006bcd0 2023-10-08T02:12:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde5f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde6015df50 0x7fde6015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde5f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde6015df50 0x7fde6015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35524/0 (socket says 172.21.15.119:35524) 2023-10-08T02:12:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde5f7fe700 1 -- 172.21.15.119:0/2533570847 learned_addr learned my addr 172.21.15.119:0/2533570847 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.474+0000 7fde5effd700 1 --2- 172.21.15.119:0/2533570847 >> v2:172.21.15.184:3300/0 conn(0x7fde6006bcd0 0x7fde600caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.475+0000 7fde5f7fe700 1 -- 172.21.15.119:0/2533570847 >> v2:172.21.15.184:3300/0 conn(0x7fde6006bcd0 msgr2=0x7fde600caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.475+0000 7fde5f7fe700 1 --2- 172.21.15.119:0/2533570847 >> v2:172.21.15.184:3300/0 conn(0x7fde6006bcd0 0x7fde600caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.475+0000 7fde5f7fe700 1 -- 172.21.15.119:0/2533570847 >> v1:172.21.15.184:6789/0 conn(0x7fde6006c0a0 legacy=0x7fde6015a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.475+0000 7fde5f7fe700 1 -- 172.21.15.119:0/2533570847 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6015ad10 con 0x7fde6015df50 2023-10-08T02:12:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.475+0000 7fde5f7fe700 1 --2- 172.21.15.119:0/2533570847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde6015e3b0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fde600691a0 tx=0x7fde58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4106c6fb4699d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.475+0000 7fde5dffb700 1 -- 172.21.15.119:0/2533570847 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5800e070 con 0x7fde6015df50 2023-10-08T02:12:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.475+0000 7fde5dffb700 1 -- 172.21.15.119:0/2533570847 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5800ba20 con 0x7fde6015df50 2023-10-08T02:12:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.475+0000 7fde5dffb700 1 -- 172.21.15.119:0/2533570847 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580092e0 con 0x7fde6015df50 2023-10-08T02:12:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.476+0000 7fde655ff700 1 -- 172.21.15.119:0/2533570847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 msgr2=0x7fde6015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.476+0000 7fde655ff700 1 --2- 172.21.15.119:0/2533570847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde6015e3b0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fde600691a0 tx=0x7fde58009840 comp rx=0 tx=0).stop 2023-10-08T02:12:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.476+0000 7fde655ff700 1 -- 172.21.15.119:0/2533570847 shutdown_connections 2023-10-08T02:12:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.476+0000 7fde655ff700 1 --2- 172.21.15.119:0/2533570847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde6015e3b0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.476+0000 7fde655ff700 1 --2- 172.21.15.119:0/2533570847 >> v2:172.21.15.184:3300/0 conn(0x7fde6006bcd0 0x7fde600caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.476+0000 7fde655ff700 1 -- 172.21.15.119:0/2533570847 >> 172.21.15.119:0/2533570847 conn(0x7fde600ce600 msgr2=0x7fde600cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.476+0000 7fde655ff700 1 -- 172.21.15.119:0/2533570847 shutdown_connections 2023-10-08T02:12:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.476+0000 7fde655ff700 1 -- 172.21.15.119:0/2533570847 wait complete. 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.477+0000 7fde655ff700 1 Processor -- start 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.478+0000 7fde655ff700 1 -- start start 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.478+0000 7fde655ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde6006bcd0 0x7fde601548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.478+0000 7fde655ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6006c0a0 0x7fde6016de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde655ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde60153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde655ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde60169e60 con 0x7fde6015df50 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde655ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6016a6c0 con 0x7fde6006c0a0 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde655ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde60154e90 con 0x7fde6006bcd0 2023-10-08T02:12:10.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde60153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde60153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35538/0 (socket says 172.21.15.119:35538) 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5f7fe700 1 -- 172.21.15.119:0/3950767961 learned_addr learned my addr 172.21.15.119:0/3950767961 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5e7fc700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6006c0a0 0x7fde6016de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5effd700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde6006bcd0 0x7fde601548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5f7fe700 1 -- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde6006bcd0 msgr2=0x7fde601548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5f7fe700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde6006bcd0 0x7fde601548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5f7fe700 1 -- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6006c0a0 msgr2=0x7fde6016de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5f7fe700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6006c0a0 0x7fde6016de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5f7fe700 1 -- 172.21.15.119:0/3950767961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde58009ae0 con 0x7fde6015df50 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5effd700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde6006bcd0 0x7fde601548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:10.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.479+0000 7fde5e7fc700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6006c0a0 0x7fde6016de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.480+0000 7fde5f7fe700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde60153150 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fde5800b6c0 tx=0x7fde58018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.480+0000 7fde46ffd700 1 -- 172.21.15.119:0/3950767961 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5800be50 con 0x7fde6015df50 2023-10-08T02:12:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.480+0000 7fde46ffd700 1 -- 172.21.15.119:0/3950767961 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58007040 con 0x7fde6015df50 2023-10-08T02:12:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.480+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6016a070 con 0x7fde6015df50 2023-10-08T02:12:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.480+0000 7fde46ffd700 1 -- 172.21.15.119:0/3950767961 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5801c410 con 0x7fde6015df50 2023-10-08T02:12:10.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.480+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde60153690 con 0x7fde6015df50 2023-10-08T02:12:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.481+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6004c3c0 con 0x7fde6015df50 2023-10-08T02:12:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.482+0000 7fde46ffd700 1 -- 172.21.15.119:0/3950767961 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fde58003050 con 0x7fde6015df50 2023-10-08T02:12:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.483+0000 7fde46ffd700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fde4c0670e0 0x7fde4c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.483+0000 7fde46ffd700 1 -- 172.21.15.119:0/3950767961 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7fde58086500 con 0x7fde6015df50 2023-10-08T02:12:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.483+0000 7fde5effd700 1 -- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fde4c0670e0 msgr2=0x7fde4c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:10.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.483+0000 7fde5effd700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fde4c0670e0 0x7fde4c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:10.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.487+0000 7fde46ffd700 1 -- 172.21.15.119:0/3950767961 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde58053110 con 0x7fde6015df50 2023-10-08T02:12:10.502 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:10.501+0000 7f30e1e76180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:12:10.598 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:10.597+0000 7f30e1e76180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:12:10.665 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:10.665+0000 7fabedb09180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:12:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.684+0000 7fde5effd700 1 -- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fde4c0670e0 msgr2=0x7fde4c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:10.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.684+0000 7fde5effd700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fde4c0670e0 0x7fde4c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:10.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.733+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde6015aff0 con 0x7fde6015df50 2023-10-08T02:12:10.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.734+0000 7fde46ffd700 1 -- 172.21.15.119:0/3950767961 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4605 (secure 0 0 0) 0x7fde58056730 con 0x7fde6015df50 2023-10-08T02:12:10.735 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:10.735 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/603902131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":603902131},{"type":"v1","addr":"172.21.15.184:6834","nonce":603902131}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3930162267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3930162267},{"type":"v1","addr":"172.21.15.119:6834","nonce":3930162267}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/237096421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":237096421},{"type":"v1","addr":"172.21.15.119:6835","nonce":237096421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:11:41.545410+0000","modified":"2023-10-08T02:11:45.111313+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8667},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8667":{"gid":8667,"name":"d","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3810992025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3810992025},{"type":"v1","addr":"172.21.15.184:6835","nonce":3810992025}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-10-08T02:12:10.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fde4c0670e0 msgr2=0x7fde4c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:10.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fde4c0670e0 0x7fde4c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 msgr2=0x7fde60153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde60153150 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fde5800b6c0 tx=0x7fde58018000 comp rx=0 tx=0).stop 2023-10-08T02:12:10.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 shutdown_connections 2023-10-08T02:12:10.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fde4c0670e0 0x7fde4c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fde6006bcd0 0x7fde601548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6015df50 0x7fde60153150 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 --2- 172.21.15.119:0/3950767961 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fde6006c0a0 0x7fde6016de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 >> 172.21.15.119:0/3950767961 conn(0x7fde600ce600 msgr2=0x7fde600c8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:10.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 shutdown_connections 2023-10-08T02:12:10.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.737+0000 7fde655ff700 1 -- 172.21.15.119:0/3950767961 wait complete. 2023-10-08T02:12:10.741 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 71 2023-10-08T02:12:10.755 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:12:10.766 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:10.766+0000 7fabedb09180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:12:10.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.941+0000 7f86519ff700 1 Processor -- start 2023-10-08T02:12:10.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.941+0000 7f86519ff700 1 -- start start 2023-10-08T02:12:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.942+0000 7f86519ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f864c0cbf30 0x7f864c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.942+0000 7f86519ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0ccf90 0x7f864c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.942+0000 7f86519ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f864c15a270 0x7f864c0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.942+0000 7f86519ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c156bf0 con 0x7f864c15a270 2023-10-08T02:12:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.942+0000 7f86519ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864c0ae950 con 0x7f864c0cbf30 2023-10-08T02:12:10.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.942+0000 7f86519ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c0cc300 con 0x7f864c0ccf90 2023-10-08T02:12:10.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f864c15a270 0x7f864c0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f864c15a270 0x7f864c0b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35548/0 (socket says 172.21.15.119:35548) 2023-10-08T02:12:10.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 -- 172.21.15.119:0/513918145 learned_addr learned my addr 172.21.15.119:0/513918145 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:10.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864a7fc700 1 --2- 172.21.15.119:0/513918145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0ccf90 0x7f864c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864affd700 1 --2- 172.21.15.119:0/513918145 >> v2:172.21.15.184:3300/0 conn(0x7f864c0cbf30 0x7f864c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 -- 172.21.15.119:0/513918145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0ccf90 msgr2=0x7f864c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 --2- 172.21.15.119:0/513918145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0ccf90 0x7f864c0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 -- 172.21.15.119:0/513918145 >> v2:172.21.15.184:3300/0 conn(0x7f864c0cbf30 msgr2=0x7f864c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 --2- 172.21.15.119:0/513918145 >> v2:172.21.15.184:3300/0 conn(0x7f864c0cbf30 0x7f864c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 -- 172.21.15.119:0/513918145 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c0ad370 con 0x7f864c15a270 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.943+0000 7f864b7fe700 1 --2- 172.21.15.119:0/513918145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c15a270 0x7f864c0b1250 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f864c063d40 tx=0x7f8644009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec786e21fb66a64 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f8649ffb700 1 -- 172.21.15.119:0/513918145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864400e070 con 0x7f864c15a270 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f8649ffb700 1 -- 172.21.15.119:0/513918145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864400b930 con 0x7f864c15a270 2023-10-08T02:12:10.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f8649ffb700 1 -- 172.21.15.119:0/513918145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86440092e0 con 0x7f864c15a270 2023-10-08T02:12:10.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f86519ff700 1 -- 172.21.15.119:0/513918145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c15a270 msgr2=0x7f864c0b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f86519ff700 1 --2- 172.21.15.119:0/513918145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c15a270 0x7f864c0b1250 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f864c063d40 tx=0x7f8644009840 comp rx=0 tx=0).stop 2023-10-08T02:12:10.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f86519ff700 1 -- 172.21.15.119:0/513918145 shutdown_connections 2023-10-08T02:12:10.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f86519ff700 1 --2- 172.21.15.119:0/513918145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c15a270 0x7f864c0b1250 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f86519ff700 1 --2- 172.21.15.119:0/513918145 >> v2:172.21.15.184:3300/0 conn(0x7f864c0cbf30 0x7f864c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f86519ff700 1 --2- 172.21.15.119:0/513918145 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0ccf90 0x7f864c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f86519ff700 1 -- 172.21.15.119:0/513918145 >> 172.21.15.119:0/513918145 conn(0x7f864c06a320 msgr2=0x7f864c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:10.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.944+0000 7f86519ff700 1 -- 172.21.15.119:0/513918145 shutdown_connections 2023-10-08T02:12:10.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.945+0000 7f86519ff700 1 -- 172.21.15.119:0/513918145 wait complete. 2023-10-08T02:12:10.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.946+0000 7f86519ff700 1 Processor -- start 2023-10-08T02:12:10.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.946+0000 7f86519ff700 1 -- start start 2023-10-08T02:12:10.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.947+0000 7f86519ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0cbf30 0x7f864c155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.947+0000 7f86519ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c0ccf90 0x7f864c16ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.947+0000 7f86519ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864c15a270 0x7f864c16b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.947+0000 7f86519ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c16c980 con 0x7f864c0ccf90 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.947+0000 7f86519ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864c1558a0 con 0x7f864c15a270 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.947+0000 7f86519ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864c16bd70 con 0x7f864c0cbf30 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c0ccf90 0x7f864c16ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864a7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c0ccf90 0x7f864c16ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35562/0 (socket says 172.21.15.119:35562) 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864affd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0cbf30 0x7f864c155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864a7fc700 1 -- 172.21.15.119:0/3820359413 learned_addr learned my addr 172.21.15.119:0/3820359413 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864b7fe700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864c15a270 0x7f864c16b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864b7fe700 1 -- 172.21.15.119:0/3820359413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0cbf30 msgr2=0x7f864c155360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864b7fe700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0cbf30 0x7f864c155360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864b7fe700 1 -- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c0ccf90 msgr2=0x7f864c16ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864b7fe700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c0ccf90 0x7f864c16ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864b7fe700 1 -- 172.21.15.119:0/3820359413 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8644009ae0 con 0x7f864c15a270 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864a7fc700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c0ccf90 0x7f864c16ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.948+0000 7f864b7fe700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864c15a270 0x7f864c16b320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8644015000 tx=0x7f8644018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.949+0000 7f8632ffd700 1 -- 172.21.15.119:0/3820359413 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644008ad0 con 0x7f864c15a270 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.949+0000 7f8632ffd700 1 -- 172.21.15.119:0/3820359413 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8644008c50 con 0x7f864c15a270 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.949+0000 7f8632ffd700 1 -- 172.21.15.119:0/3820359413 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8644026460 con 0x7f864c15a270 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.949+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864c16c050 con 0x7f864c15a270 2023-10-08T02:12:10.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.949+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864c16c610 con 0x7f864c15a270 2023-10-08T02:12:10.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.950+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f864c0cb600 con 0x7f864c15a270 2023-10-08T02:12:10.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.951+0000 7f8632ffd700 1 -- 172.21.15.119:0/3820359413 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f8644003040 con 0x7f864c15a270 2023-10-08T02:12:10.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.952+0000 7f8632ffd700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86380670e0 0x7f8638069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:10.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.952+0000 7f8632ffd700 1 -- 172.21.15.119:0/3820359413 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f86440869b0 con 0x7f864c15a270 2023-10-08T02:12:10.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.952+0000 7f864affd700 1 -- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86380670e0 msgr2=0x7f8638069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:10.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.952+0000 7f864affd700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86380670e0 0x7f8638069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:10.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:10.956+0000 7f8632ffd700 1 -- 172.21.15.119:0/3820359413 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8644053520 con 0x7f864c15a270 2023-10-08T02:12:11.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.153+0000 7f864affd700 1 -- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86380670e0 msgr2=0x7f8638069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:11.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.153+0000 7f864affd700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86380670e0 0x7f8638069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:11.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.161+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f864c0cc590 con 0x7f864c15a270 2023-10-08T02:12:11.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.162+0000 7f8632ffd700 1 -- 172.21.15.119:0/3820359413 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16497 (secure 0 0 0) 0x7f864405c070 con 0x7f864c15a270 2023-10-08T02:12:11.164 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:11.164 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":158,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:12:03.296789+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:11:39.192970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-10-08T02:11:40.531612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"157","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/2185260495":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/3673331094":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/1337159806":"2023-10-09T02:12:03.296742+0000","172.21.15.119:0/294111303":"2023-10-09T02:12:03.296742+0000","172.21.15.119:6837/85511":"2023-10-09T02:12:03.296742+0000","172.21.15.119:6836/85511":"2023-10-09T02:12:03.296742+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:12:11.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86380670e0 msgr2=0x7f8638069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86380670e0 0x7f8638069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864c15a270 msgr2=0x7f864c16b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864c15a270 0x7f864c16b320 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f8644015000 tx=0x7f8644018000 comp rx=0 tx=0).stop 2023-10-08T02:12:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 shutdown_connections 2023-10-08T02:12:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86380670e0 0x7f8638069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f864c0cbf30 0x7f864c155360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f864c15a270 0x7f864c16b320 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 --2- 172.21.15.119:0/3820359413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f864c0ccf90 0x7f864c16ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 >> 172.21.15.119:0/3820359413 conn(0x7f864c06a320 msgr2=0x7f864c155e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 shutdown_connections 2023-10-08T02:12:11.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.164+0000 7f86519ff700 1 -- 172.21.15.119:0/3820359413 wait complete. 2023-10-08T02:12:11.183 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T02:12:11.309 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:11.308+0000 7f30e1e76180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:12:11.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.411+0000 7f937159d700 1 Processor -- start 2023-10-08T02:12:11.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.411+0000 7f937159d700 1 -- start start 2023-10-08T02:12:11.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f937159d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c159cc0 0x7f936c0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f937159d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f936c0b2920 0x7f936c15a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f937159d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f936c15a5d0 0x7f936c0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f937159d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c0ce260 con 0x7f936c15a5d0 2023-10-08T02:12:11.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f937159d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f936c0b5b40 con 0x7f936c0b2920 2023-10-08T02:12:11.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f937159d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0b5e20 con 0x7f936c159cc0 2023-10-08T02:12:11.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f9370d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f936c15a5d0 0x7f936c0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f936b7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f936c0b2920 0x7f936c15a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f936bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c159cc0 0x7f936c0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f936b7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f936c0b2920 0x7f936c15a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54390/0 (socket says 172.21.15.119:54390) 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f936b7fe700 1 -- 172.21.15.119:0/1978508568 learned_addr learned my addr 172.21.15.119:0/1978508568 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f9370d9c700 1 -- 172.21.15.119:0/1978508568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c159cc0 msgr2=0x7f936c0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f9370d9c700 1 --2- 172.21.15.119:0/1978508568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c159cc0 0x7f936c0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f9370d9c700 1 -- 172.21.15.119:0/1978508568 >> v2:172.21.15.184:3300/0 conn(0x7f936c0b2920 msgr2=0x7f936c15a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.413+0000 7f9370d9c700 1 --2- 172.21.15.119:0/1978508568 >> v2:172.21.15.184:3300/0 conn(0x7f936c0b2920 0x7f936c15a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.414+0000 7f9370d9c700 1 -- 172.21.15.119:0/1978508568 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936c0b6100 con 0x7f936c15a5d0 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.414+0000 7f936b7fe700 1 --2- 172.21.15.119:0/1978508568 >> v2:172.21.15.184:3300/0 conn(0x7f936c0b2920 0x7f936c15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:11.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.414+0000 7f9370d9c700 1 --2- 172.21.15.119:0/1978508568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c0ce6b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f936c0679b0 tx=0x7f9364009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a811c1112af03b80 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:11.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.414+0000 7f936affd700 1 -- 172.21.15.119:0/1978508568 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400e070 con 0x7f936c15a5d0 2023-10-08T02:12:11.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.414+0000 7f936affd700 1 -- 172.21.15.119:0/1978508568 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936400bab0 con 0x7f936c15a5d0 2023-10-08T02:12:11.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.414+0000 7f936affd700 1 -- 172.21.15.119:0/1978508568 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936400be70 con 0x7f936c15a5d0 2023-10-08T02:12:11.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 -- 172.21.15.119:0/1978508568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 msgr2=0x7f936c0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 --2- 172.21.15.119:0/1978508568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c0ce6b0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f936c0679b0 tx=0x7f9364009780 comp rx=0 tx=0).stop 2023-10-08T02:12:11.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 -- 172.21.15.119:0/1978508568 shutdown_connections 2023-10-08T02:12:11.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 --2- 172.21.15.119:0/1978508568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c0ce6b0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 --2- 172.21.15.119:0/1978508568 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c159cc0 0x7f936c0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 --2- 172.21.15.119:0/1978508568 >> v2:172.21.15.184:3300/0 conn(0x7f936c0b2920 0x7f936c15a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 -- 172.21.15.119:0/1978508568 >> 172.21.15.119:0/1978508568 conn(0x7f936c0d0560 msgr2=0x7f936c0d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 -- 172.21.15.119:0/1978508568 shutdown_connections 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.415+0000 7f937159d700 1 -- 172.21.15.119:0/1978508568 wait complete. 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.416+0000 7f937159d700 1 Processor -- start 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.416+0000 7f937159d700 1 -- start start 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.417+0000 7f937159d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c0b2920 0x7f936c1512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.417+0000 7f937159d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f936c159cc0 0x7f936c151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.417+0000 7f937159d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.417+0000 7f937159d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c0cc750 con 0x7f936c15a5d0 2023-10-08T02:12:11.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.417+0000 7f937159d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936c1553b0 con 0x7f936c159cc0 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.417+0000 7f937159d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936c154480 con 0x7f936c0b2920 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35586/0 (socket says 172.21.15.119:35586) 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 -- 172.21.15.119:0/1455167711 learned_addr learned my addr 172.21.15.119:0/1455167711 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f936b7fe700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f936c159cc0 0x7f936c151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f936bfff700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c0b2920 0x7f936c1512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 -- 172.21.15.119:0/1455167711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c0b2920 msgr2=0x7f936c1512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c0b2920 0x7f936c1512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 -- 172.21.15.119:0/1455167711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f936c159cc0 msgr2=0x7f936c151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f936c159cc0 0x7f936c151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 -- 172.21.15.119:0/1455167711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600094f0 con 0x7f936c15a5d0 2023-10-08T02:12:11.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.418+0000 7f9370d9c700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c156850 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9364009fd0 tx=0x7f9364019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:11.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.419+0000 7f9353fff700 1 -- 172.21.15.119:0/1455167711 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364012dd0 con 0x7f936c15a5d0 2023-10-08T02:12:11.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.419+0000 7f9353fff700 1 -- 172.21.15.119:0/1455167711 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9364027470 con 0x7f936c15a5d0 2023-10-08T02:12:11.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.419+0000 7f9353fff700 1 -- 172.21.15.119:0/1455167711 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9364027830 con 0x7f936c15a5d0 2023-10-08T02:12:11.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.419+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9364009a20 con 0x7f936c15a5d0 2023-10-08T02:12:11.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.419+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936c155040 con 0x7f936c15a5d0 2023-10-08T02:12:11.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.420+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936c04c3c0 con 0x7f936c15a5d0 2023-10-08T02:12:11.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.421+0000 7f9353fff700 1 -- 172.21.15.119:0/1455167711 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f9364024070 con 0x7f936c15a5d0 2023-10-08T02:12:11.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.422+0000 7f9353fff700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f93580670e0 0x7f9358069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.422+0000 7f9353fff700 1 -- 172.21.15.119:0/1455167711 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f93640870b0 con 0x7f936c15a5d0 2023-10-08T02:12:11.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.422+0000 7f936bfff700 1 -- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f93580670e0 msgr2=0x7f9358069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:11.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.422+0000 7f936bfff700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f93580670e0 0x7f9358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:11.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.427+0000 7f9353fff700 1 -- 172.21.15.119:0/1455167711 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9364053d30 con 0x7f936c15a5d0 2023-10-08T02:12:11.512 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:11.511+0000 7fabedb09180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:12:11.599 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:11.598+0000 7f30e1e76180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:12:11.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.623+0000 7f936bfff700 1 -- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f93580670e0 msgr2=0x7f9358069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:11.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.623+0000 7f936bfff700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f93580670e0 0x7f9358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:11.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.672+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f936c0b63e0 con 0x7f936c15a5d0 2023-10-08T02:12:11.673 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T02:12:11.673+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T02:12:11.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.677+0000 7f9353fff700 1 -- 172.21.15.119:0/1455167711 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9364057350 con 0x7f936c15a5d0 2023-10-08T02:12:11.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f93580670e0 msgr2=0x7f9358069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:11.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f93580670e0 0x7f9358069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 msgr2=0x7f936c156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c156850 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f9364009fd0 tx=0x7f9364019000 comp rx=0 tx=0).stop 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 shutdown_connections 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f93580670e0 0x7f9358069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f936c0b2920 0x7f936c1512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f936c15a5d0 0x7f936c156850 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 --2- 172.21.15.119:0/1455167711 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f936c159cc0 0x7f936c151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.680+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 >> 172.21.15.119:0/1455167711 conn(0x7f936c0d0560 msgr2=0x7f936c0cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.681+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 shutdown_connections 2023-10-08T02:12:11.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.681+0000 7f937159d700 1 -- 172.21.15.119:0/1455167711 wait complete. 2023-10-08T02:12:11.683 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T02:12:11.683 INFO:tasks.ceph.mds.d.smithi184.stderr: -2395> 2023-10-08T02:11:46.609+0000 7f1afb1f1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T02:12:11.694 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:11.694+0000 7f30e1e76180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:12:11.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T02:12:11.724 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T02:12:11.810 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:11.810+0000 7fabedb09180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:12:11.857 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:11.856+0000 7f30e1e76180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:12:11.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.890+0000 7f71b2d9d700 1 Processor -- start 2023-10-08T02:12:11.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.890+0000 7f71b2d9d700 1 -- start start 2023-10-08T02:12:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b2d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71ac0b16d0 0x7f71ac0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b2d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b33b0 0x7f71ac0b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b2d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71ac0b2080 con 0x7f71ac0b4e70 2023-10-08T02:12:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b2d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71ac0cfb00 con 0x7f71ac0b16d0 2023-10-08T02:12:11.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b2d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac0b5c40 con 0x7f71ac0b33b0 2023-10-08T02:12:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b159a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71ac0b16d0 0x7f71ac0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b259c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b33b0 0x7f71ac0b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b159a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71ac0b16d0 0x7f71ac0b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54406/0 (socket says 172.21.15.119:54406) 2023-10-08T02:12:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.892+0000 7f71b159a700 1 -- 172.21.15.119:0/801301394 learned_addr learned my addr 172.21.15.119:0/801301394 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b159a700 1 -- 172.21.15.119:0/801301394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b33b0 msgr2=0x7f71ac0b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b159a700 1 --2- 172.21.15.119:0/801301394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b33b0 0x7f71ac0b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b159a700 1 -- 172.21.15.119:0/801301394 >> v1:172.21.15.184:6789/0 conn(0x7f71ac0b4e70 legacy=0x7f71ac0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b159a700 1 -- 172.21.15.119:0/801301394 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac0b5f20 con 0x7f71ac0b16d0 2023-10-08T02:12:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b0d99700 1 -- 172.21.15.119:0/801301394 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100604229 0 0) 0x7f71ac0b2080 con 0x7f71ac0b4e70 2023-10-08T02:12:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b159a700 1 --2- 172.21.15.119:0/801301394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b16d0 0x7f71ac0b1b40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f719c0099b0 tx=0x7f719c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=350e749dbbdb4965 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b0d99700 1 -- 172.21.15.119:0/801301394 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c009ea0 con 0x7f71ac0b16d0 2023-10-08T02:12:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b0d99700 1 -- 172.21.15.119:0/801301394 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c00b860 con 0x7f71ac0b16d0 2023-10-08T02:12:11.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.893+0000 7f71b0d99700 1 -- 172.21.15.119:0/801301394 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00bc20 con 0x7f71ac0b16d0 2023-10-08T02:12:11.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.894+0000 7f71b2d9d700 1 -- 172.21.15.119:0/801301394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b16d0 msgr2=0x7f71ac0b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.894+0000 7f71b2d9d700 1 --2- 172.21.15.119:0/801301394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b16d0 0x7f71ac0b1b40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f719c0099b0 tx=0x7f719c009250 comp rx=0 tx=0).stop 2023-10-08T02:12:11.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.894+0000 7f71b2d9d700 1 -- 172.21.15.119:0/801301394 shutdown_connections 2023-10-08T02:12:11.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.894+0000 7f71b2d9d700 1 --2- 172.21.15.119:0/801301394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b33b0 0x7f71ac0b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.894+0000 7f71b2d9d700 1 --2- 172.21.15.119:0/801301394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b16d0 0x7f71ac0b1b40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.894+0000 7f71b2d9d700 1 -- 172.21.15.119:0/801301394 >> 172.21.15.119:0/801301394 conn(0x7f71ac0d10c0 msgr2=0x7f71ac0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:11.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.894+0000 7f71b2d9d700 1 -- 172.21.15.119:0/801301394 shutdown_connections 2023-10-08T02:12:11.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.894+0000 7f71b2d9d700 1 -- 172.21.15.119:0/801301394 wait complete. 2023-10-08T02:12:11.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.896+0000 7f71b2d9d700 1 Processor -- start 2023-10-08T02:12:11.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.897+0000 7f71b2d9d700 1 -- start start 2023-10-08T02:12:11.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b2d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b16d0 0x7f71ac14a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b2d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b33b0 0x7f71ac0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ac0b4e70 0x7f71ac169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b2d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac16cec0 con 0x7f71ac0b4e70 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b2d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac16a100 con 0x7f71ac0b33b0 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b2d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71ac16a3e0 con 0x7f71ac0b16d0 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ac0b4e70 0x7f71ac169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b1d9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b16d0 0x7f71ac14a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b159a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b33b0 0x7f71ac0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ac0b4e70 0x7f71ac169870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35590/0 (socket says 172.21.15.119:35590) 2023-10-08T02:12:11.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.898+0000 7f71b259c700 1 -- 172.21.15.119:0/3028434608 learned_addr learned my addr 172.21.15.119:0/3028434608 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:11.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71b259c700 1 -- 172.21.15.119:0/3028434608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b16d0 msgr2=0x7f71ac14a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71b259c700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b16d0 0x7f71ac14a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71b259c700 1 -- 172.21.15.119:0/3028434608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b33b0 msgr2=0x7f71ac0c8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:11.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71b259c700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b33b0 0x7f71ac0c8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:11.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71b259c700 1 -- 172.21.15.119:0/3028434608 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c0094f0 con 0x7f71ac0b4e70 2023-10-08T02:12:11.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71b259c700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ac0b4e70 0x7f71ac169870 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e90 tx=0x7f71a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:11.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a400cb70 con 0x7f71ac0b4e70 2023-10-08T02:12:11.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a400f040 con 0x7f71ac0b4e70 2023-10-08T02:12:11.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ac16a6c0 con 0x7f71ac0b4e70 2023-10-08T02:12:11.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.899+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a4014480 con 0x7f71ac0b4e70 2023-10-08T02:12:11.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.900+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ac169db0 con 0x7f71ac0b4e70 2023-10-08T02:12:11.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.901+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ac16d310 con 0x7f71ac0b4e70 2023-10-08T02:12:11.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.901+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 84) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f71a4014830 con 0x7f71ac0b4e70 2023-10-08T02:12:11.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.902+0000 7f71a1ffb700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 0x7f7190069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:11.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.902+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6513+0+0 (secure 0 0 0) 0x7f71a40856e0 con 0x7f71ac0b4e70 2023-10-08T02:12:11.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.902+0000 7f71b1d9b700 1 -- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 msgr2=0x7f7190069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:11.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.902+0000 7f71b1d9b700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 0x7f7190069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:11.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:11.910+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71a40521f0 con 0x7f71ac0b4e70 2023-10-08T02:12:11.916 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:11.916+0000 7fabedb09180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:12:11.963 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:11.962+0000 7f30e1e76180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:12:12.076 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:12.075+0000 7fabedb09180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:12:12.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.103+0000 7f71b1d9b700 1 -- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 msgr2=0x7f7190069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:12.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.103+0000 7f71b1d9b700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 0x7f7190069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:12.154 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:12.153+0000 7f30e1e76180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:12:12.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.155+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f71ac16a850 con 0x7f71ac0b4e70 2023-10-08T02:12:12.185 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:12.184+0000 7fabedb09180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:12:12.247 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:12.246+0000 7f30e1e76180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:12:12.341 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:12.340+0000 7f30e1e76180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:12:12.378 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:12.377+0000 7fabedb09180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:12:12.434 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:12.434+0000 7f30e1e76180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:12:12.472 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:12.471+0000 7fabedb09180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:12:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.504+0000 7f71b1d9b700 1 -- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 msgr2=0x7f7190069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.504+0000 7f71b1d9b700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 0x7f7190069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:12:12.566 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:12.565+0000 7fabedb09180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:12:12.662 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:12.661+0000 7fabedb09180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:12:12.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.670+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 85) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f71a40525c0 con 0x7f71ac0b4e70 2023-10-08T02:12:12.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.670+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 msgr2=0x7f7190069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.670+0000 7f71a1ffb700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 0x7f7190069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.678+0000 7f71a1ffb700 1 -- 172.21.15.119:0/3028434608 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f71a4070740 con 0x7f71ac0b4e70 2023-10-08T02:12:12.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ac0b4e70 msgr2=0x7f71ac169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ac0b4e70 0x7f71ac169870 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f71a4009e90 tx=0x7f71a400b040 comp rx=0 tx=0).stop 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 shutdown_connections 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f71900670e0 0x7f7190069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71ac0b16d0 0x7f71ac14a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71ac0b4e70 0x7f71ac169870 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 --2- 172.21.15.119:0/3028434608 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71ac0b33b0 0x7f71ac0c8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 >> 172.21.15.119:0/3028434608 conn(0x7f71ac0d10c0 msgr2=0x7f71ac15c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 shutdown_connections 2023-10-08T02:12:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.682+0000 7f71b2d9d700 1 -- 172.21.15.119:0/3028434608 wait complete. 2023-10-08T02:12:12.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T02:12:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.898+0000 7f527c7ff700 1 Processor -- start 2023-10-08T02:12:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.898+0000 7f527c7ff700 1 -- start start 2023-10-08T02:12:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.899+0000 7f527c7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f527415a440 0x7f52740acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.899+0000 7f527c7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5274068890 0x7f52740c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.899+0000 7f527c7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527415a9b0 con 0x7f52741563b0 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.899+0000 7f527c7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527415ac90 con 0x7f5274068890 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.899+0000 7f527c7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f527415af70 con 0x7f527415a440 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f527ad9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5274068890 0x7f52740c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f527a59b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f52741563b0 0x7f5274156780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59158/0 (socket says 172.21.15.119:59158) 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f527a59b700 1 -- 172.21.15.119:0/1817131256 learned_addr learned my addr 172.21.15.119:0/1817131256 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f5279d9a700 1 --2- 172.21.15.119:0/1817131256 >> v2:172.21.15.184:3300/0 conn(0x7f527415a440 0x7f52740acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f5279599700 1 -- 172.21.15.119:0/1817131256 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655881916 0 0) 0x7f527415a9b0 con 0x7f52741563b0 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f527ad9c700 1 -- 172.21.15.119:0/1817131256 >> v2:172.21.15.184:3300/0 conn(0x7f527415a440 msgr2=0x7f52740acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f527ad9c700 1 --2- 172.21.15.119:0/1817131256 >> v2:172.21.15.184:3300/0 conn(0x7f527415a440 0x7f52740acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f527ad9c700 1 -- 172.21.15.119:0/1817131256 >> v1:172.21.15.119:6789/0 conn(0x7f52741563b0 legacy=0x7f5274156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f527ad9c700 1 -- 172.21.15.119:0/1817131256 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527415b250 con 0x7f5274068890 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.900+0000 7f527ad9c700 1 --2- 172.21.15.119:0/1817131256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5274068890 0x7f52740c8230 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f527406fb10 tx=0x7f526c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f834493284351a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f5279599700 1 -- 172.21.15.119:0/1817131256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c00e070 con 0x7f5274068890 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f5279599700 1 -- 172.21.15.119:0/1817131256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526c00bb60 con 0x7f5274068890 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f5279599700 1 -- 172.21.15.119:0/1817131256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526c012430 con 0x7f5274068890 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f527c7ff700 1 -- 172.21.15.119:0/1817131256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5274068890 msgr2=0x7f52740c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f527c7ff700 1 --2- 172.21.15.119:0/1817131256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5274068890 0x7f52740c8230 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f527406fb10 tx=0x7f526c009780 comp rx=0 tx=0).stop 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f527c7ff700 1 -- 172.21.15.119:0/1817131256 shutdown_connections 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f527c7ff700 1 --2- 172.21.15.119:0/1817131256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5274068890 0x7f52740c8230 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f527c7ff700 1 --2- 172.21.15.119:0/1817131256 >> v2:172.21.15.184:3300/0 conn(0x7f527415a440 0x7f52740acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f527c7ff700 1 -- 172.21.15.119:0/1817131256 >> 172.21.15.119:0/1817131256 conn(0x7f52740c8f90 msgr2=0x7f52740c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f527c7ff700 1 -- 172.21.15.119:0/1817131256 shutdown_connections 2023-10-08T02:12:12.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.901+0000 7f527c7ff700 1 -- 172.21.15.119:0/1817131256 wait complete. 2023-10-08T02:12:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.903+0000 7f527c7ff700 1 Processor -- start 2023-10-08T02:12:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.903+0000 7f527c7ff700 1 -- start start 2023-10-08T02:12:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f527c7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5274068890 0x7f52740b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f527c7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52741563b0 0x7f5274155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f527c7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527415a440 0x7f5274169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f527c7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740bfe90 con 0x7f52741563b0 2023-10-08T02:12:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f527c7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52740c0170 con 0x7f527415a440 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f527c7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5274155570 con 0x7f5274068890 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f5279d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52741563b0 0x7f5274155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f5279d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52741563b0 0x7f5274155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35688/0 (socket says 172.21.15.119:35688) 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f5279d9a700 1 -- 172.21.15.119:0/1298137092 learned_addr learned my addr 172.21.15.119:0/1298137092 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f527a59b700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5274068890 0x7f52740b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.904+0000 7f527ad9c700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527415a440 0x7f5274169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f527a59b700 1 -- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527415a440 msgr2=0x7f5274169e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f527a59b700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527415a440 0x7f5274169e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f527a59b700 1 -- 172.21.15.119:0/1298137092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52741563b0 msgr2=0x7f5274155030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f527a59b700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52741563b0 0x7f5274155030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f527a59b700 1 -- 172.21.15.119:0/1298137092 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f526c009a20 con 0x7f5274068890 2023-10-08T02:12:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f5279d9a700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52741563b0 0x7f5274155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f527a59b700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5274068890 0x7f52740b1dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f527000c5d0 tx=0x7f5270007d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f526a7fc700 1 -- 172.21.15.119:0/1298137092 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270005b30 con 0x7f5274068890 2023-10-08T02:12:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f526a7fc700 1 -- 172.21.15.119:0/1298137092 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5270005c90 con 0x7f5274068890 2023-10-08T02:12:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f526a7fc700 1 -- 172.21.15.119:0/1298137092 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5270007320 con 0x7f5274068890 2023-10-08T02:12:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f527416a6b0 con 0x7f5274068890 2023-10-08T02:12:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.905+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f527416ac70 con 0x7f5274068890 2023-10-08T02:12:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.906+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52740718b0 con 0x7f5274068890 2023-10-08T02:12:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.907+0000 7f526a7fc700 1 -- 172.21.15.119:0/1298137092 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 85) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f527001b3b0 con 0x7f5274068890 2023-10-08T02:12:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.908+0000 7f526a7fc700 1 -- 172.21.15.119:0/1298137092 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6814+0+0 (secure 0 0 0) 0x7f5270083930 con 0x7f5274068890 2023-10-08T02:12:12.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:12.912+0000 7f526a7fc700 1 -- 172.21.15.119:0/1298137092 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5270050a20 con 0x7f5274068890 2023-10-08T02:12:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.105+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f527406d6a0 con 0x7f5274068890 2023-10-08T02:12:13.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.684+0000 7f526a7fc700 1 -- 172.21.15.119:0/1298137092 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v161) v1 ==== 181+0+0 (secure 0 0 0) 0x7f52700170d0 con 0x7f5274068890 2023-10-08T02:12:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.686+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5274068890 msgr2=0x7f52740b1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.686+0000 7f527c7ff700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5274068890 0x7f52740b1dd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f527000c5d0 tx=0x7f5270007d30 comp rx=0 tx=0).stop 2023-10-08T02:12:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.686+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 shutdown_connections 2023-10-08T02:12:13.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.686+0000 7f527c7ff700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5274068890 0x7f52740b1dd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.687+0000 7f527c7ff700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f527415a440 0x7f5274169e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.687+0000 7f527c7ff700 1 --2- 172.21.15.119:0/1298137092 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f52741563b0 0x7f5274155030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.687+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 >> 172.21.15.119:0/1298137092 conn(0x7f52740c8f90 msgr2=0x7f52740adea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.687+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 shutdown_connections 2023-10-08T02:12:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.687+0000 7f527c7ff700 1 -- 172.21.15.119:0/1298137092 wait complete. 2023-10-08T02:12:13.688 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T02:12:13.715 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T02:12:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.902+0000 7fbf2afff700 1 Processor -- start 2023-10-08T02:12:13.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.902+0000 7fbf2afff700 1 -- start start 2023-10-08T02:12:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.903+0000 7fbf2afff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf24069a50 0x7fbf240c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.903+0000 7fbf2afff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240ac030 con 0x7fbf24069e20 2023-10-08T02:12:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.903+0000 7fbf2afff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240ab820 con 0x7fbf240c8490 2023-10-08T02:12:13.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.903+0000 7fbf2afff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf2406dbb0 con 0x7fbf24069a50 2023-10-08T02:12:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf2959c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbf24069e20 0x7fbf241569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59162/0 (socket says 172.21.15.119:59162) 2023-10-08T02:12:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf2959c700 1 -- 172.21.15.119:0/3234073486 learned_addr learned my addr 172.21.15.119:0/3234073486 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf28d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf24069a50 0x7fbf240c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf28d9b700 1 -- 172.21.15.119:0/3234073486 >> v1:172.21.15.184:6789/0 conn(0x7fbf240c8490 legacy=0x7fbf2415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf28d9b700 1 -- 172.21.15.119:0/3234073486 >> v1:172.21.15.119:6789/0 conn(0x7fbf24069e20 legacy=0x7fbf241569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf28d9b700 1 -- 172.21.15.119:0/3234073486 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf240aba70 con 0x7fbf24069a50 2023-10-08T02:12:13.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf28d9b700 1 --2- 172.21.15.119:0/3234073486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069a50 0x7fbf240c6ac0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fbf180098e0 tx=0x7fbf180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e532a6712fa80b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:13.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf237fe700 1 -- 172.21.15.119:0/3234073486 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf18009e80 con 0x7fbf24069a50 2023-10-08T02:12:13.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.904+0000 7fbf237fe700 1 -- 172.21.15.119:0/3234073486 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1800b860 con 0x7fbf24069a50 2023-10-08T02:12:13.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.905+0000 7fbf237fe700 1 -- 172.21.15.119:0/3234073486 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1800bbe0 con 0x7fbf24069a50 2023-10-08T02:12:13.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.905+0000 7fbf2afff700 1 -- 172.21.15.119:0/3234073486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069a50 msgr2=0x7fbf240c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:13.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.905+0000 7fbf2afff700 1 --2- 172.21.15.119:0/3234073486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069a50 0x7fbf240c6ac0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fbf180098e0 tx=0x7fbf180091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:13.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.905+0000 7fbf2afff700 1 -- 172.21.15.119:0/3234073486 shutdown_connections 2023-10-08T02:12:13.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.905+0000 7fbf2afff700 1 --2- 172.21.15.119:0/3234073486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069a50 0x7fbf240c6ac0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:13.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.905+0000 7fbf2afff700 1 -- 172.21.15.119:0/3234073486 >> 172.21.15.119:0/3234073486 conn(0x7fbf240c8f90 msgr2=0x7fbf240c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:13.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.905+0000 7fbf2afff700 1 -- 172.21.15.119:0/3234073486 shutdown_connections 2023-10-08T02:12:13.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.905+0000 7fbf2afff700 1 -- 172.21.15.119:0/3234073486 wait complete. 2023-10-08T02:12:13.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.907+0000 7fbf2afff700 1 Processor -- start 2023-10-08T02:12:13.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.907+0000 7fbf2afff700 1 -- start start 2023-10-08T02:12:13.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.907+0000 7fbf2afff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24069a50 0x7fbf2416cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:13.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.907+0000 7fbf2afff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069e20 0x7fbf24155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.907+0000 7fbf2afff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240c8490 0x7fbf24155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf2afff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240afb80 con 0x7fbf24069e20 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf2afff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241595b0 con 0x7fbf240c8490 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf2afff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240afdd0 con 0x7fbf24069a50 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069e20 0x7fbf24155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069e20 0x7fbf24155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35702/0 (socket says 172.21.15.119:35702) 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 -- 172.21.15.119:0/4037770990 learned_addr learned my addr 172.21.15.119:0/4037770990 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf28d9b700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24069a50 0x7fbf2416cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:13.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf2959c700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240c8490 0x7fbf24155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:13.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 -- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24069a50 msgr2=0x7fbf2416cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24069a50 0x7fbf2416cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 -- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240c8490 msgr2=0x7fbf24155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240c8490 0x7fbf24155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 -- 172.21.15.119:0/4037770990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf18009420 con 0x7fbf24069e20 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf2959c700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240c8490 0x7fbf24155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.908+0000 7fbf23fff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069e20 0x7fbf24155770 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c590 tx=0x7fbf1000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.909+0000 7fbf20ff9700 1 -- 172.21.15.119:0/4037770990 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10011590 con 0x7fbf24069e20 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.909+0000 7fbf20ff9700 1 -- 172.21.15.119:0/4037770990 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf100116f0 con 0x7fbf24069e20 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.909+0000 7fbf20ff9700 1 -- 172.21.15.119:0/4037770990 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf10011ab0 con 0x7fbf24069e20 2023-10-08T02:12:13.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.909+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf240b00b0 con 0x7fbf24069e20 2023-10-08T02:12:13.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.909+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf2416d2a0 con 0x7fbf24069e20 2023-10-08T02:12:13.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.910+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf240b05d0 con 0x7fbf24069e20 2023-10-08T02:12:13.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.911+0000 7fbf20ff9700 1 -- 172.21.15.119:0/4037770990 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 86) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fbf1001b3b0 con 0x7fbf24069e20 2023-10-08T02:12:13.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.912+0000 7fbf20ff9700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf08066d80 0x7fbf08069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:13.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.912+0000 7fbf20ff9700 1 -- 172.21.15.119:0/4037770990 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6376+0+0 (secure 0 0 0) 0x7fbf10083ab0 con 0x7fbf24069e20 2023-10-08T02:12:13.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.912+0000 7fbf28d9b700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf08066d80 0x7fbf08069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:13.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.912+0000 7fbf28d9b700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf08066d80 0x7fbf08069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf180093f0 tx=0x7fbf18017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:13.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:13.916+0000 7fbf20ff9700 1 -- 172.21.15.119:0/4037770990 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf240b05d0 con 0x7fbf24069e20 2023-10-08T02:12:14.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:14.111+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbf240b05d0 con 0x7fbf24069e20 2023-10-08T02:12:15.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.068+0000 7fbf20ff9700 1 -- 172.21.15.119:0/4037770990 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v162) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbf240b05d0 con 0x7fbf24069e20 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf08066d80 msgr2=0x7fbf08069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf08066d80 0x7fbf08069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbf180093f0 tx=0x7fbf18017000 comp rx=0 tx=0).stop 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069e20 msgr2=0x7fbf24155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069e20 0x7fbf24155770 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fbf1000c590 tx=0x7fbf1000c950 comp rx=0 tx=0).stop 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 shutdown_connections 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fbf08066d80 0x7fbf08069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbf24069a50 0x7fbf2416cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbf240c8490 0x7fbf24155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 --2- 172.21.15.119:0/4037770990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf24069e20 0x7fbf24155770 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 >> 172.21.15.119:0/4037770990 conn(0x7fbf240c8f90 msgr2=0x7fbf240ca050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 shutdown_connections 2023-10-08T02:12:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.071+0000 7fbf2afff700 1 -- 172.21.15.119:0/4037770990 wait complete. 2023-10-08T02:12:15.074 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T02:12:15.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T02:12:15.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.281+0000 7fa9231ff700 1 Processor -- start 2023-10-08T02:12:15.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.281+0000 7fa9231ff700 1 -- start start 2023-10-08T02:12:15.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.282+0000 7fa9231ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0c70b0 0x7fa91c155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.282+0000 7fa9231ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c06e950 con 0x7fa91c0ccc40 2023-10-08T02:12:15.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.282+0000 7fa9231ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa91c06eb90 con 0x7fa91c0cd0b0 2023-10-08T02:12:15.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.282+0000 7fa9231ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c1563f0 con 0x7fa91c0c70b0 2023-10-08T02:12:15.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa91bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa91c0ccc40 0x7fa91c06e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47754/0 (socket says 172.21.15.119:47754) 2023-10-08T02:12:15.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa91bfff700 1 -- 172.21.15.119:0/4101873407 learned_addr learned my addr 172.21.15.119:0/4101873407 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:15.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa920f9b700 1 --2- 172.21.15.119:0/4101873407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0c70b0 0x7fa91c155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa91b7fe700 1 -- 172.21.15.119:0/4101873407 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139355941 0 0) 0x7fa908001110 con 0x7fa91c0ccc40 2023-10-08T02:12:15.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa91b7fe700 1 -- 172.21.15.119:0/4101873407 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa90c003570 con 0x7fa91c0ccc40 2023-10-08T02:12:15.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa91b7fe700 1 -- 172.21.15.119:0/4101873407 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561575434 0 0) 0x7fa91c06eb90 con 0x7fa91c0cd0b0 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa91b7fe700 1 -- 172.21.15.119:0/4101873407 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa908001110 con 0x7fa91c0cd0b0 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa920f9b700 1 -- 172.21.15.119:0/4101873407 >> v1:172.21.15.184:6789/0 conn(0x7fa91c0cd0b0 legacy=0x7fa91c069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa920f9b700 1 -- 172.21.15.119:0/4101873407 >> v1:172.21.15.119:6789/0 conn(0x7fa91c0ccc40 legacy=0x7fa91c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.283+0000 7fa920f9b700 1 -- 172.21.15.119:0/4101873407 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa91c06ee70 con 0x7fa91c0c70b0 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa920f9b700 1 --2- 172.21.15.119:0/4101873407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0c70b0 0x7fa91c155eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa910009910 tx=0x7fa9100091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e757be06c5b17e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa91b7fe700 1 -- 172.21.15.119:0/4101873407 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011630 con 0x7fa91c0c70b0 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa91b7fe700 1 -- 172.21.15.119:0/4101873407 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa910011790 con 0x7fa91c0c70b0 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa91b7fe700 1 -- 172.21.15.119:0/4101873407 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa910011b30 con 0x7fa91c0c70b0 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa9231ff700 1 -- 172.21.15.119:0/4101873407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0c70b0 msgr2=0x7fa91c155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa9231ff700 1 --2- 172.21.15.119:0/4101873407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0c70b0 0x7fa91c155eb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa910009910 tx=0x7fa9100091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa9231ff700 1 -- 172.21.15.119:0/4101873407 shutdown_connections 2023-10-08T02:12:15.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa9231ff700 1 --2- 172.21.15.119:0/4101873407 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0c70b0 0x7fa91c155eb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.284+0000 7fa9231ff700 1 -- 172.21.15.119:0/4101873407 >> 172.21.15.119:0/4101873407 conn(0x7fa91c068e10 msgr2=0x7fa91c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:15.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.285+0000 7fa9231ff700 1 -- 172.21.15.119:0/4101873407 shutdown_connections 2023-10-08T02:12:15.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.285+0000 7fa9231ff700 1 -- 172.21.15.119:0/4101873407 wait complete. 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.286+0000 7fa9231ff700 1 Processor -- start 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.286+0000 7fa9231ff700 1 -- start start 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa9231ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0c70b0 0x7fa91c0ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa9231ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa91c0ccc40 0x7fa91c0ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa9231ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0cd0b0 0x7fa91c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa9231ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0aea80 con 0x7fa91c0ccc40 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa9231ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0c3710 con 0x7fa91c0c70b0 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa9231ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa91c0c39f0 con 0x7fa91c0cd0b0 2023-10-08T02:12:15.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa91c0ccc40 0x7fa91c0ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa91c0ccc40 0x7fa91c0ae1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60936/0 (socket says 172.21.15.119:60936) 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa91bfff700 1 -- 172.21.15.119:0/1424921355 learned_addr learned my addr 172.21.15.119:0/1424921355 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa920f9b700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0c70b0 0x7fa91c0ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.287+0000 7fa92179c700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0cd0b0 0x7fa91c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa91bfff700 1 -- 172.21.15.119:0/1424921355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0cd0b0 msgr2=0x7fa91c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa91bfff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0cd0b0 0x7fa91c0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa91bfff700 1 -- 172.21.15.119:0/1424921355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0c70b0 msgr2=0x7fa91c0ac860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa91bfff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0c70b0 0x7fa91c0ac860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa91bfff700 1 -- 172.21.15.119:0/1424921355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910009420 con 0x7fa91c0ccc40 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa91bfff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa91c0ccc40 0x7fa91c0ae1a0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fa9080058d0 tx=0x7fa908007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:15.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa918ff9700 1 -- 172.21.15.119:0/1424921355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908005b30 con 0x7fa91c0ccc40 2023-10-08T02:12:15.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa918ff9700 1 -- 172.21.15.119:0/1424921355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa908005c90 con 0x7fa91c0ccc40 2023-10-08T02:12:15.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa918ff9700 1 -- 172.21.15.119:0/1424921355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa908007300 con 0x7fa91c0ccc40 2023-10-08T02:12:15.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa91c0c3cd0 con 0x7fa91c0ccc40 2023-10-08T02:12:15.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.288+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa91c0ae6e0 con 0x7fa91c0ccc40 2023-10-08T02:12:15.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.289+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91c0c41f0 con 0x7fa91c0ccc40 2023-10-08T02:12:15.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.290+0000 7fa918ff9700 1 -- 172.21.15.119:0/1424921355 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa90801b400 con 0x7fa91c0ccc40 2023-10-08T02:12:15.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.291+0000 7fa918ff9700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa900066f30 0x7fa9000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.291+0000 7fa918ff9700 1 -- 172.21.15.119:0/1424921355 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fa908083fe0 con 0x7fa91c0ccc40 2023-10-08T02:12:15.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.291+0000 7fa920f9b700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa900066f30 0x7fa9000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.292+0000 7fa920f9b700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa900066f30 0x7fa9000693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa910009860 tx=0x7fa910017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:15.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.296+0000 7fa918ff9700 1 -- 172.21.15.119:0/1424921355 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa91c0c41f0 con 0x7fa91c0ccc40 2023-10-08T02:12:15.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.496+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa91c06e950 con 0x7fa91c0ccc40 2023-10-08T02:12:15.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.500+0000 7fa918ff9700 1 -- 172.21.15.119:0/1424921355 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v55) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa91c06e950 con 0x7fa91c0ccc40 2023-10-08T02:12:15.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa900066f30 msgr2=0x7fa9000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa900066f30 0x7fa9000693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa910009860 tx=0x7fa910017000 comp rx=0 tx=0).stop 2023-10-08T02:12:15.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa91c0ccc40 msgr2=0x7fa91c0ae1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa91c0ccc40 0x7fa91c0ae1a0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fa9080058d0 tx=0x7fa908007ea0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 shutdown_connections 2023-10-08T02:12:15.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa900066f30 0x7fa9000693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa91c0cd0b0 0x7fa91c0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa91c0c70b0 0x7fa91c0ac860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 --2- 172.21.15.119:0/1424921355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa91c0ccc40 0x7fa91c0ae1a0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 >> 172.21.15.119:0/1424921355 conn(0x7fa91c068e10 msgr2=0x7fa91c0c2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:15.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 shutdown_connections 2023-10-08T02:12:15.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.502+0000 7fa9231ff700 1 -- 172.21.15.119:0/1424921355 wait complete. 2023-10-08T02:12:15.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T02:12:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.701+0000 7f295d5ff700 1 Processor -- start 2023-10-08T02:12:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.702+0000 7f295d5ff700 1 -- start start 2023-10-08T02:12:15.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f295d5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29580c83f0 0x7f29580c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f295d5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 0x7f29580ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f295d5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295815b0d0 con 0x7f29580ccee0 2023-10-08T02:12:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f295d5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29580c7a60 con 0x7f29580c83f0 2023-10-08T02:12:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f295d5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295815a4c0 con 0x7f29581568e0 2023-10-08T02:12:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f2956ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29580c83f0 0x7f29580c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f2956ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29580c83f0 0x7f29580c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60946/0 (socket says 172.21.15.119:60946) 2023-10-08T02:12:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f2956ffd700 1 -- 172.21.15.119:0/550167374 learned_addr learned my addr 172.21.15.119:0/550167374 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.703+0000 7f29577fe700 1 --2- 172.21.15.119:0/550167374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 0x7f29580ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f2956ffd700 1 -- 172.21.15.119:0/550167374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 msgr2=0x7f29580ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f2956ffd700 1 --2- 172.21.15.119:0/550167374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 0x7f29580ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f2956ffd700 1 -- 172.21.15.119:0/550167374 >> v1:172.21.15.184:6789/0 conn(0x7f29580ccee0 legacy=0x7f29581561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f2956ffd700 1 -- 172.21.15.119:0/550167374 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295815a7a0 con 0x7f29580c83f0 2023-10-08T02:12:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f29577fe700 1 --2- 172.21.15.119:0/550167374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 0x7f29580ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f2956ffd700 1 --2- 172.21.15.119:0/550167374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f29580c7520 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2948009f20 tx=0x7f2948009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8b574e41f5e02e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f2955ffb700 1 -- 172.21.15.119:0/550167374 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800e070 con 0x7f29580c83f0 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f2955ffb700 1 -- 172.21.15.119:0/550167374 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294800ba80 con 0x7f29580c83f0 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.704+0000 7f2955ffb700 1 -- 172.21.15.119:0/550167374 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480092e0 con 0x7f29580c83f0 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.705+0000 7f295d5ff700 1 -- 172.21.15.119:0/550167374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 msgr2=0x7f29580c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.705+0000 7f295d5ff700 1 --2- 172.21.15.119:0/550167374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f29580c7520 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2948009f20 tx=0x7f2948009840 comp rx=0 tx=0).stop 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.705+0000 7f295d5ff700 1 -- 172.21.15.119:0/550167374 shutdown_connections 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.705+0000 7f295d5ff700 1 --2- 172.21.15.119:0/550167374 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 0x7f29580ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.705+0000 7f295d5ff700 1 --2- 172.21.15.119:0/550167374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f29580c7520 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.705+0000 7f295d5ff700 1 -- 172.21.15.119:0/550167374 >> 172.21.15.119:0/550167374 conn(0x7f2958068e10 msgr2=0x7f29580cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.705+0000 7f295d5ff700 1 -- 172.21.15.119:0/550167374 shutdown_connections 2023-10-08T02:12:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.705+0000 7f295d5ff700 1 -- 172.21.15.119:0/550167374 wait complete. 2023-10-08T02:12:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.706+0000 7f295d5ff700 1 Processor -- start 2023-10-08T02:12:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.706+0000 7f295d5ff700 1 -- start start 2023-10-08T02:12:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f295d5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f295813f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f295d5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29580ccee0 0x7f29581584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f295d5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 0x7f29580c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f295d5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2958159bd0 con 0x7f29580c83f0 2023-10-08T02:12:15.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f295d5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2958158fc0 con 0x7f29580ccee0 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f295d5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29581592a0 con 0x7f29581568e0 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f2956ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f295813f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f2956ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f295813f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60960/0 (socket says 172.21.15.119:60960) 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.707+0000 7f2956ffd700 1 -- 172.21.15.119:0/722439545 learned_addr learned my addr 172.21.15.119:0/722439545 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f2956ffd700 1 -- 172.21.15.119:0/722439545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 msgr2=0x7f29580c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f2956ffd700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 0x7f29580c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f2956ffd700 1 -- 172.21.15.119:0/722439545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29580ccee0 msgr2=0x7f29581584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f2956ffd700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29580ccee0 0x7f29581584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f2956ffd700 1 -- 172.21.15.119:0/722439545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948009ae0 con 0x7f29580c83f0 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f2956ffd700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f295813f8e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f294800b6c0 tx=0x7f2948018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:15.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f293effd700 1 -- 172.21.15.119:0/722439545 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948008a80 con 0x7f29580c83f0 2023-10-08T02:12:15.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f293effd700 1 -- 172.21.15.119:0/722439545 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948008c00 con 0x7f29580c83f0 2023-10-08T02:12:15.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2958159580 con 0x7f29580c83f0 2023-10-08T02:12:15.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.708+0000 7f293effd700 1 -- 172.21.15.119:0/722439545 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948026460 con 0x7f29580c83f0 2023-10-08T02:12:15.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.709+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29580b1120 con 0x7f29580c83f0 2023-10-08T02:12:15.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.709+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295804c3c0 con 0x7f29580c83f0 2023-10-08T02:12:15.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.710+0000 7f293effd700 1 -- 172.21.15.119:0/722439545 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2948003040 con 0x7f29580c83f0 2023-10-08T02:12:15.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.711+0000 7f293effd700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2944066f30 0x7f29440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:15.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.711+0000 7f293effd700 1 -- 172.21.15.119:0/722439545 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f2948086500 con 0x7f29580c83f0 2023-10-08T02:12:15.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.711+0000 7f29567fc700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2944066f30 0x7f29440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:15.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.712+0000 7f29567fc700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2944066f30 0x7f29440693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f295816b250 tx=0x7f294c007b40 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:15.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.715+0000 7f293effd700 1 -- 172.21.15.119:0/722439545 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2948053670 con 0x7f29580c83f0 2023-10-08T02:12:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.906+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f295815aa80 con 0x7f29580c83f0 2023-10-08T02:12:15.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.910+0000 7f293effd700 1 -- 172.21.15.119:0/722439545 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v56) v1 ==== 115+0+0 (secure 0 0 0) 0x7f2948056c90 con 0x7f29580c83f0 2023-10-08T02:12:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2944066f30 msgr2=0x7f29440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2944066f30 0x7f29440693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f295816b250 tx=0x7f294c007b40 comp rx=0 tx=0).stop 2023-10-08T02:12:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 msgr2=0x7f295813f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f295813f8e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f294800b6c0 tx=0x7f2948018000 comp rx=0 tx=0).stop 2023-10-08T02:12:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 shutdown_connections 2023-10-08T02:12:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f2944066f30 0x7f29440693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f29581568e0 0x7f29580c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580c83f0 0x7f295813f8e0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 --2- 172.21.15.119:0/722439545 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f29580ccee0 0x7f29581584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 >> 172.21.15.119:0/722439545 conn(0x7f2958068e10 msgr2=0x7f2958148050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 shutdown_connections 2023-10-08T02:12:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:15.913+0000 7f295d5ff700 1 -- 172.21.15.119:0/722439545 wait complete. 2023-10-08T02:12:15.932 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T02:12:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.114+0000 7f0f3eb9d700 1 Processor -- start 2023-10-08T02:12:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.115+0000 7f0f3eb9d700 1 -- start start 2023-10-08T02:12:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3eb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f3806f060 0x7f0f380ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3eb9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0f3806f4b0 0x7f0f380ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3eb9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f3806ebd0 con 0x7f0f380cb1a0 2023-10-08T02:12:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3eb9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3815b2e0 con 0x7f0f3806f4b0 2023-10-08T02:12:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3eb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3815aa70 con 0x7f0f3806f060 2023-10-08T02:12:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3db9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0f380cb1a0 0x7f0f3815a360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47768/0 (socket says 172.21.15.119:47768) 2023-10-08T02:12:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3db9b700 1 -- 172.21.15.119:0/2940541601 learned_addr learned my addr 172.21.15.119:0/2940541601 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3d39a700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f3806f060 0x7f0f380ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3e39c700 1 --2- 172.21.15.119:0/2940541601 >> v2:172.21.15.184:3300/0 conn(0x7f0f3806f4b0 0x7f0f380ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.116+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770271242 0 0) 0x7f0f3806ebd0 con 0x7f0f380cb1a0 2023-10-08T02:12:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f24003570 con 0x7f0f380cb1a0 2023-10-08T02:12:16.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1568905892 0 0) 0x7f0f24003570 con 0x7f0f380cb1a0 2023-10-08T02:12:16.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f3806f060 msgr2=0x7f0f380ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f3806f060 0x7f0f380ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 >> v2:172.21.15.184:3300/0 conn(0x7f0f3806f4b0 msgr2=0x7f0f380ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 --2- 172.21.15.119:0/2940541601 >> v2:172.21.15.184:3300/0 conn(0x7f0f3806f4b0 0x7f0f380ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3815ad20 con 0x7f0f380cb1a0 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f0f34001470 con 0x7f0f380cb1a0 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3e39c700 1 --2- 172.21.15.119:0/2940541601 >> v2:172.21.15.184:3300/0 conn(0x7f0f3806f4b0 0x7f0f380ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 >> v1:172.21.15.119:6789/0 conn(0x7f0f380cb1a0 legacy=0x7f0f3815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f3806f060 0x7f0f24003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f240040b0 0x7f0f24004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f24004d50 0x7f0f240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3815b2e0 con 0x7f0f24004d50 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3815aa70 con 0x7f0f240040b0 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2400dd70 con 0x7f0f3806f060 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3e39c700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f24004d50 0x7f0f240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3db9b700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f3806f060 0x7f0f24003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.117+0000 7f0f3d39a700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f240040b0 0x7f0f24004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3e39c700 1 -- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f3806f060 msgr2=0x7f0f24003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3e39c700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f3806f060 0x7f0f24003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3e39c700 1 -- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f240040b0 msgr2=0x7f0f24004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3e39c700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f240040b0 0x7f0f24004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3e39c700 1 -- 172.21.15.119:0/2940541601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f24003570 con 0x7f0f24004d50 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3e39c700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f24004d50 0x7f0f240051c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f0f300094d0 tx=0x7f0f3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd09062442d88c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3000ec40 con 0x7f0f24004d50 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3000f040 con 0x7f0f24004d50 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.118+0000 7f0f3cb99700 1 -- 172.21.15.119:0/2940541601 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f30014480 con 0x7f0f24004d50 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.119+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/2940541601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f24004d50 msgr2=0x7f0f240051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.119+0000 7f0f3eb9d700 1 --2- 172.21.15.119:0/2940541601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f24004d50 0x7f0f240051c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f0f300094d0 tx=0x7f0f3000b040 comp rx=0 tx=0).stop 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.119+0000 7f0f3e39c700 1 -- 172.21.15.119:0/2940541601 reap_dead start 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.119+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/2940541601 shutdown_connections 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.119+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/2940541601 >> 172.21.15.119:0/2940541601 conn(0x7f0f380cc1e0 msgr2=0x7f0f380cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.119+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/2940541601 shutdown_connections 2023-10-08T02:12:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.119+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/2940541601 wait complete. 2023-10-08T02:12:16.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.120+0000 7f0f3eb9d700 1 Processor -- start 2023-10-08T02:12:16.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.120+0000 7f0f3eb9d700 1 -- start start 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.121+0000 7f0f3eb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f24004d50 0x7f0f38159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.121+0000 7f0f3eb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f3816aca0 0x7f0f3815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.121+0000 7f0f3eb9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f3815d2a0 0x7f0f380c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.121+0000 7f0f3eb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f38159cb0 con 0x7f0f3816aca0 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.121+0000 7f0f3eb9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3815e800 con 0x7f0f3815d2a0 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.121+0000 7f0f3eb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3815dbf0 con 0x7f0f24004d50 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f3816aca0 0x7f0f3815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f3816aca0 0x7f0f3815ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60974/0 (socket says 172.21.15.119:60974) 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 -- 172.21.15.119:0/3927924872 learned_addr learned my addr 172.21.15.119:0/3927924872 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 -- 172.21.15.119:0/3927924872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f24004d50 msgr2=0x7f0f38159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f24004d50 0x7f0f38159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 -- 172.21.15.119:0/3927924872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f3815d2a0 msgr2=0x7f0f380c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f3815d2a0 0x7f0f380c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 -- 172.21.15.119:0/3927924872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3815ded0 con 0x7f0f3816aca0 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3d39a700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f3816aca0 0x7f0f3815ec90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0f28006ac0 tx=0x7f0f280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f2dffb700 1 -- 172.21.15.119:0/3927924872 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2800fa10 con 0x7f0f3816aca0 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f2dffb700 1 -- 172.21.15.119:0/3927924872 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2800fb70 con 0x7f0f3816aca0 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f2dffb700 1 -- 172.21.15.119:0/3927924872 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f28014440 con 0x7f0f3816aca0 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.122+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3815e1b0 con 0x7f0f3816aca0 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.123+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3815f1d0 con 0x7f0f3816aca0 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.124+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3806c450 con 0x7f0f3816aca0 2023-10-08T02:12:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.124+0000 7f0f2dffb700 1 -- 172.21.15.119:0/3927924872 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f28014830 con 0x7f0f3816aca0 2023-10-08T02:12:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.126+0000 7f0f2dffb700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f1c066f30 0x7f0f1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.126+0000 7f0f2dffb700 1 -- 172.21.15.119:0/3927924872 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f0f28018030 con 0x7f0f3816aca0 2023-10-08T02:12:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.126+0000 7f0f3db9b700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f1c066f30 0x7f0f1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.127+0000 7f0f3db9b700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f1c066f30 0x7f0f1c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f34000e10 tx=0x7f0f34001dc0 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:16.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.130+0000 7f0f2dffb700 1 -- 172.21.15.119:0/3927924872 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0f28052240 con 0x7f0f3816aca0 2023-10-08T02:12:16.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.329+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f0f3815b000 con 0x7f0f3816aca0 2023-10-08T02:12:16.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.333+0000 7f0f2dffb700 1 -- 172.21.15.119:0/3927924872 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v57) v1 ==== 120+0+0 (secure 0 0 0) 0x7f0f28055860 con 0x7f0f3816aca0 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f1c066f30 msgr2=0x7f0f1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f1c066f30 0x7f0f1c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0f34000e10 tx=0x7f0f34001dc0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f3816aca0 msgr2=0x7f0f3815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f3816aca0 0x7f0f3815ec90 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f0f28006ac0 tx=0x7f0f280091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 shutdown_connections 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0f1c066f30 0x7f0f1c0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f24004d50 0x7f0f38159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f3815d2a0 0x7f0f380c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 --2- 172.21.15.119:0/3927924872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f3816aca0 0x7f0f3815ec90 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 >> 172.21.15.119:0/3927924872 conn(0x7f0f380cc1e0 msgr2=0x7f0f3806a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:16.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 shutdown_connections 2023-10-08T02:12:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.336+0000 7f0f3eb9d700 1 -- 172.21.15.119:0/3927924872 wait complete. 2023-10-08T02:12:16.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-10-08T02:12:16.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.533+0000 7fe9d8fff700 1 Processor -- start 2023-10-08T02:12:16.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.534+0000 7fe9d8fff700 1 -- start start 2023-10-08T02:12:16.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9d8fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9d41578b0 0x7fe9d4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9d8fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe9d40af2f0 0x7fe9d40af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9d8fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d40ad930 con 0x7fe9d4155c10 2023-10-08T02:12:16.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9d8fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d40b0c80 con 0x7fe9d41578b0 2023-10-08T02:12:16.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9d8fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d40acd20 con 0x7fe9d40af2f0 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9cbfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9d41578b0 0x7fe9d4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9d259c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe9d4155c10 0x7fe9d40ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47782/0 (socket says 172.21.15.119:47782) 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9d259c700 1 -- 172.21.15.119:0/3686520914 learned_addr learned my addr 172.21.15.119:0/3686520914 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.535+0000 7fe9d2d9d700 1 --2- 172.21.15.119:0/3686520914 >> v2:172.21.15.184:3300/0 conn(0x7fe9d40af2f0 0x7fe9d40af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9d1d9b700 1 -- 172.21.15.119:0/3686520914 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629620380 0 0) 0x7fe9d40ad930 con 0x7fe9d4155c10 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9d1d9b700 1 -- 172.21.15.119:0/3686520914 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9c0003570 con 0x7fe9d4155c10 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9cbfff700 1 -- 172.21.15.119:0/3686520914 >> v2:172.21.15.184:3300/0 conn(0x7fe9d40af2f0 msgr2=0x7fe9d40af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9cbfff700 1 --2- 172.21.15.119:0/3686520914 >> v2:172.21.15.184:3300/0 conn(0x7fe9d40af2f0 0x7fe9d40af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9cbfff700 1 -- 172.21.15.119:0/3686520914 >> v1:172.21.15.119:6789/0 conn(0x7fe9d4155c10 legacy=0x7fe9d40ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9cbfff700 1 -- 172.21.15.119:0/3686520914 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d40ad000 con 0x7fe9d41578b0 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9cbfff700 1 --2- 172.21.15.119:0/3686520914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d41578b0 0x7fe9d4155fe0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4065be0 tx=0x7fe9bc00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f149e0750d3a8a75 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9d1d9b700 1 -- 172.21.15.119:0/3686520914 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00aea0 con 0x7fe9d41578b0 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9d1d9b700 1 -- 172.21.15.119:0/3686520914 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc00c970 con 0x7fe9d41578b0 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.536+0000 7fe9d1d9b700 1 -- 172.21.15.119:0/3686520914 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00cd10 con 0x7fe9d41578b0 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.537+0000 7fe9d8fff700 1 -- 172.21.15.119:0/3686520914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d41578b0 msgr2=0x7fe9d4155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.537+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/3686520914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d41578b0 0x7fe9d4155fe0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fe9d4065be0 tx=0x7fe9bc00a330 comp rx=0 tx=0).stop 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.537+0000 7fe9d8fff700 1 -- 172.21.15.119:0/3686520914 shutdown_connections 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.537+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/3686520914 >> v2:172.21.15.184:3300/0 conn(0x7fe9d40af2f0 0x7fe9d40af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.537+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/3686520914 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d41578b0 0x7fe9d4155fe0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.537+0000 7fe9d8fff700 1 -- 172.21.15.119:0/3686520914 >> 172.21.15.119:0/3686520914 conn(0x7fe9d406a320 msgr2=0x7fe9d40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.537+0000 7fe9d8fff700 1 -- 172.21.15.119:0/3686520914 shutdown_connections 2023-10-08T02:12:16.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.537+0000 7fe9d8fff700 1 -- 172.21.15.119:0/3686520914 wait complete. 2023-10-08T02:12:16.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.538+0000 7fe9d8fff700 1 Processor -- start 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.538+0000 7fe9d8fff700 1 -- start start 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.539+0000 7fe9d8fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d40af2f0 0x7fe9d4158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.539+0000 7fe9d8fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d4155c10 0x7fe9d416a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.539+0000 7fe9d8fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d41578b0 0x7fe9d414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.539+0000 7fe9d8fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d414fa20 con 0x7fe9d4155c10 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.539+0000 7fe9d8fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d414f4a0 con 0x7fe9d41578b0 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.539+0000 7fe9d8fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d41592a0 con 0x7fe9d40af2f0 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d4155c10 0x7fe9d416a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d4155c10 0x7fe9d416a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60986/0 (socket says 172.21.15.119:60986) 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 -- 172.21.15.119:0/4154166485 learned_addr learned my addr 172.21.15.119:0/4154166485 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9d259c700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d40af2f0 0x7fe9d4158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9d2d9d700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d41578b0 0x7fe9d414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 -- 172.21.15.119:0/4154166485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d40af2f0 msgr2=0x7fe9d4158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d40af2f0 0x7fe9d4158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 -- 172.21.15.119:0/4154166485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d41578b0 msgr2=0x7fe9d414ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d41578b0 0x7fe9d414ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 -- 172.21.15.119:0/4154166485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc00a5d0 con 0x7fe9d4155c10 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9cbfff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d4155c10 0x7fe9d416a530 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00a370 tx=0x7fe9bc01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9ca7fc700 1 -- 172.21.15.119:0/4154166485 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc021430 con 0x7fe9d4155c10 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9ca7fc700 1 -- 172.21.15.119:0/4154166485 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc021590 con 0x7fe9d4155c10 2023-10-08T02:12:16.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9ca7fc700 1 -- 172.21.15.119:0/4154166485 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc021950 con 0x7fe9d4155c10 2023-10-08T02:12:16.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.540+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d40c9b50 con 0x7fe9d4155c10 2023-10-08T02:12:16.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.541+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d40c9df0 con 0x7fe9d4155c10 2023-10-08T02:12:16.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.542+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d404c3c0 con 0x7fe9d4155c10 2023-10-08T02:12:16.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.542+0000 7fe9ca7fc700 1 -- 172.21.15.119:0/4154166485 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe9bc0103f0 con 0x7fe9d4155c10 2023-10-08T02:12:16.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.544+0000 7fe9ca7fc700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9b4066f30 0x7fe9b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:16.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.544+0000 7fe9ca7fc700 1 -- 172.21.15.119:0/4154166485 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fe9bc085dc0 con 0x7fe9d4155c10 2023-10-08T02:12:16.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.544+0000 7fe9d259c700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9b4066f30 0x7fe9b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:16.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.545+0000 7fe9d259c700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9b4066f30 0x7fe9b40693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40058d0 tx=0x7fe9c4009040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:16.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.548+0000 7fe9ca7fc700 1 -- 172.21.15.119:0/4154166485 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe9bc052f30 con 0x7fe9d4155c10 2023-10-08T02:12:16.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:16.816+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fe9d40ad2e0 con 0x7fe9d4155c10 2023-10-08T02:12:17.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.071+0000 7fe9ca7fc700 1 -- 172.21.15.119:0/4154166485 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2477) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe9bc056550 con 0x7fe9d4155c10 2023-10-08T02:12:17.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9b4066f30 msgr2=0x7fe9b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:17.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9b4066f30 0x7fe9b40693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe9c40058d0 tx=0x7fe9c4009040 comp rx=0 tx=0).stop 2023-10-08T02:12:17.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d4155c10 msgr2=0x7fe9d416a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:17.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d4155c10 0x7fe9d416a530 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00a370 tx=0x7fe9bc01a000 comp rx=0 tx=0).stop 2023-10-08T02:12:17.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 shutdown_connections 2023-10-08T02:12:17.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe9b4066f30 0x7fe9b40693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9d40af2f0 0x7fe9d4158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9d41578b0 0x7fe9d414ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 --2- 172.21.15.119:0/4154166485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9d4155c10 0x7fe9d416a530 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 >> 172.21.15.119:0/4154166485 conn(0x7fe9d406a320 msgr2=0x7fe9d41512a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 shutdown_connections 2023-10-08T02:12:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.074+0000 7fe9d8fff700 1 -- 172.21.15.119:0/4154166485 wait complete. 2023-10-08T02:12:17.092 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-10-08T02:12:17.092 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-10-08T02:12:17.092 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T02:12:17.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-08T02:12:17.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.273+0000 7fc2db39d700 1 Processor -- start 2023-10-08T02:12:17.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.273+0000 7fc2db39d700 1 -- start start 2023-10-08T02:12:17.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.274+0000 7fc2db39d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc2d4157770 0x7fc2d4150180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:17.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2db39d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d406e2a0 con 0x7fc2d40c6d00 2023-10-08T02:12:17.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2db39d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d4157b40 con 0x7fc2d40c7150 2023-10-08T02:12:17.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2db39d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d406b2e0 con 0x7fc2d4157770 2023-10-08T02:12:17.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9b9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc2d40c6d00 0x7fc2d406e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47790/0 (socket says 172.21.15.119:47790) 2023-10-08T02:12:17.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9b9a700 1 -- 172.21.15.119:0/2621665431 learned_addr learned my addr 172.21.15.119:0/2621665431 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:17.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2da39b700 1 --2- 172.21.15.119:0/2621665431 >> v2:172.21.15.184:3300/0 conn(0x7fc2d4157770 0x7fc2d4150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:17.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2055112417 0 0) 0x7fc2d406e2a0 con 0x7fc2d40c6d00 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c0003570 con 0x7fc2d40c6d00 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 284101571 0 0) 0x7fc2c0003570 con 0x7fc2d40c6d00 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 >> v2:172.21.15.184:3300/0 conn(0x7fc2d4157770 msgr2=0x7fc2d4150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9399700 1 --2- 172.21.15.119:0/2621665431 >> v2:172.21.15.184:3300/0 conn(0x7fc2d4157770 0x7fc2d4150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 >> v1:172.21.15.184:6789/0 conn(0x7fc2d40c7150 legacy=0x7fc2d406ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40c78e0 con 0x7fc2d40c6d00 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.275+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fc2c4001470 con 0x7fc2d40c6d00 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 >> v1:172.21.15.119:6789/0 conn(0x7fc2d40c6d00 legacy=0x7fc2d406e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9399700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c0003a60 0x7fc2c0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9399700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c0004810 0x7fc2c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:17.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9399700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c00051c0 0x7fc2c000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d406b2e0 con 0x7fc2c0003a60 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c000c0d0 con 0x7fc2c00051c0 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c000e4a0 con 0x7fc2c0004810 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2da39b700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c0003a60 0x7fc2c0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9b9a700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c0004810 0x7fc2c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2dab9c700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c00051c0 0x7fc2c000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2da39b700 1 -- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c0004810 msgr2=0x7fc2c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2da39b700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c0004810 0x7fc2c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2da39b700 1 -- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c00051c0 msgr2=0x7fc2c000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2da39b700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c00051c0 0x7fc2c000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2da39b700 1 -- 172.21.15.119:0/2621665431 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c0003570 con 0x7fc2c0003a60 2023-10-08T02:12:17.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2dab9c700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c00051c0 0x7fc2c000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2dab9c700 1 -- 172.21.15.119:0/2621665431 reap_dead start 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.276+0000 7fc2d9b9a700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2c0004810 0x7fc2c0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2da39b700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c0003a60 0x7fc2c0003e70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009c00 tx=0x7fc2d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dd75e8455c8463 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000e710 con 0x7fc2c0003a60 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d000e870 con 0x7fc2c0003a60 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2d9399700 1 -- 172.21.15.119:0/2621665431 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000ec30 con 0x7fc2c0003a60 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2db39d700 1 -- 172.21.15.119:0/2621665431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c0003a60 msgr2=0x7fc2c0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2db39d700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c0003a60 0x7fc2c0003e70 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009c00 tx=0x7fc2d00091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2db39d700 1 -- 172.21.15.119:0/2621665431 shutdown_connections 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2db39d700 1 --2- 172.21.15.119:0/2621665431 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2c0003a60 0x7fc2c0003e70 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2db39d700 1 -- 172.21.15.119:0/2621665431 >> 172.21.15.119:0/2621665431 conn(0x7fc2d40cc1f0 msgr2=0x7fc2d40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:17.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2db39d700 1 -- 172.21.15.119:0/2621665431 shutdown_connections 2023-10-08T02:12:17.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.277+0000 7fc2db39d700 1 -- 172.21.15.119:0/2621665431 wait complete. 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.279+0000 7fc2db39d700 1 Processor -- start 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.279+0000 7fc2db39d700 1 -- start start 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2db39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c0003a60 0x7fc2d416c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2db39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d416ab10 0x7fc2d40abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2db39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2d40abff0 0x7fc2d4169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2db39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d41495d0 con 0x7fc2d416ab10 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2db39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d40ac470 con 0x7fc2c0003a60 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2db39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d414b070 con 0x7fc2d40abff0 2023-10-08T02:12:17.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2d9b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d416ab10 0x7fc2d40abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2d9b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d416ab10 0x7fc2d40abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32770/0 (socket says 172.21.15.119:32770) 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2d9b9a700 1 -- 172.21.15.119:0/4072166992 learned_addr learned my addr 172.21.15.119:0/4072166992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.280+0000 7fc2dab9c700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2d40abff0 0x7fc2d4169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2da39b700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c0003a60 0x7fc2d416c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2d9b9a700 1 -- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2d40abff0 msgr2=0x7fc2d4169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2d9b9a700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2d40abff0 0x7fc2d4169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2d9b9a700 1 -- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c0003a60 msgr2=0x7fc2d416c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2d9b9a700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c0003a60 0x7fc2d416c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2d9b9a700 1 -- 172.21.15.119:0/4072166992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d40a9970 con 0x7fc2d416ab10 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2da39b700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c0003a60 0x7fc2d416c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:17.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2d9b9a700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d416ab10 0x7fc2d40abab0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400eb90 tx=0x7fc2c4001e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2ca7fc700 1 -- 172.21.15.119:0/4072166992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400c7e0 con 0x7fc2d416ab10 2023-10-08T02:12:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2ca7fc700 1 -- 172.21.15.119:0/4072166992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c400c940 con 0x7fc2d416ab10 2023-10-08T02:12:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2ca7fc700 1 -- 172.21.15.119:0/4072166992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c400cce0 con 0x7fc2d416ab10 2023-10-08T02:12:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d416a590 con 0x7fc2d416ab10 2023-10-08T02:12:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.281+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d40a9ef0 con 0x7fc2d416ab10 2023-10-08T02:12:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.282+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d416a930 con 0x7fc2d416ab10 2023-10-08T02:12:17.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.283+0000 7fc2ca7fc700 1 -- 172.21.15.119:0/4072166992 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc2c4013630 con 0x7fc2d416ab10 2023-10-08T02:12:17.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.284+0000 7fc2ca7fc700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b8066f30 0x7fc2b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:17.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.285+0000 7fc2ca7fc700 1 -- 172.21.15.119:0/4072166992 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fc2c4084600 con 0x7fc2d416ab10 2023-10-08T02:12:17.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.285+0000 7fc2da39b700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b8066f30 0x7fc2b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:17.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.285+0000 7fc2da39b700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b8066f30 0x7fc2b80693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00091e0 tx=0x7fc2d001a000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:17.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.289+0000 7fc2ca7fc700 1 -- 172.21.15.119:0/4072166992 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2c4051790 con 0x7fc2d416ab10 2023-10-08T02:12:17.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:17.553+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fc2d40aa110 con 0x7fc2d416ab10 2023-10-08T02:12:18.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.073+0000 7fc2ca7fc700 1 -- 172.21.15.119:0/4072166992 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2478) v1 ==== 168+0+0 (secure 0 0 0) 0x7fc2c4054db0 con 0x7fc2d416ab10 2023-10-08T02:12:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b8066f30 msgr2=0x7fc2b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b8066f30 0x7fc2b80693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00091e0 tx=0x7fc2d001a000 comp rx=0 tx=0).stop 2023-10-08T02:12:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d416ab10 msgr2=0x7fc2d40abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d416ab10 0x7fc2d40abab0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fc2c400eb90 tx=0x7fc2c4001e20 comp rx=0 tx=0).stop 2023-10-08T02:12:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 shutdown_connections 2023-10-08T02:12:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fc2b8066f30 0x7fc2b80693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2d40abff0 0x7fc2d4169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc2c0003a60 0x7fc2d416c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 --2- 172.21.15.119:0/4072166992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2d416ab10 0x7fc2d40abab0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 >> 172.21.15.119:0/4072166992 conn(0x7fc2d40cc1f0 msgr2=0x7fc2d4157cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 shutdown_connections 2023-10-08T02:12:18.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.076+0000 7fc2db39d700 1 -- 172.21.15.119:0/4072166992 wait complete. 2023-10-08T02:12:18.095 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T02:12:18.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.280+0000 7f023d79d700 1 Processor -- start 2023-10-08T02:12:18.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.280+0000 7f023d79d700 1 -- start start 2023-10-08T02:12:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.281+0000 7f023d79d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f02380b16d0 0x7f02380b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.281+0000 7f023d79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b33b0 0x7f02380b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.281+0000 7f023d79d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02380b2080 con 0x7f02380b4e70 2023-10-08T02:12:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.281+0000 7f023d79d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02380cfb00 con 0x7f02380b16d0 2023-10-08T02:12:18.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.281+0000 7f023d79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02380b5c40 con 0x7f02380b33b0 2023-10-08T02:12:18.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f02377fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f02380b16d0 0x7f02380b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f023cf9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b33b0 0x7f02380b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f0237fff700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f02380b4e70 0x7f02380ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:38162/0 (socket says 172.21.15.119:38162) 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f0237fff700 1 -- 172.21.15.119:0/3316696419 learned_addr learned my addr 172.21.15.119:0/3316696419 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f0236ffd700 1 -- 172.21.15.119:0/3316696419 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754420655 0 0) 0x7f02380b2080 con 0x7f02380b4e70 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f0236ffd700 1 -- 172.21.15.119:0/3316696419 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0220003570 con 0x7f02380b4e70 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f02377fe700 1 -- 172.21.15.119:0/3316696419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b33b0 msgr2=0x7f02380b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f02377fe700 1 --2- 172.21.15.119:0/3316696419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b33b0 0x7f02380b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f02377fe700 1 -- 172.21.15.119:0/3316696419 >> v1:172.21.15.184:6789/0 conn(0x7f02380b4e70 legacy=0x7f02380ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f02377fe700 1 -- 172.21.15.119:0/3316696419 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02380b5f20 con 0x7f02380b16d0 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.282+0000 7f023cf9c700 1 --2- 172.21.15.119:0/3316696419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b33b0 0x7f02380b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.283+0000 7f02377fe700 1 --2- 172.21.15.119:0/3316696419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 0x7f02380b1b40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f022c0098e0 tx=0x7f022c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d15a7d9dd61985 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.283+0000 7f0236ffd700 1 -- 172.21.15.119:0/3316696419 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c011630 con 0x7f02380b16d0 2023-10-08T02:12:18.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.283+0000 7f0236ffd700 1 -- 172.21.15.119:0/3316696419 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c011790 con 0x7f02380b16d0 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.283+0000 7f0236ffd700 1 -- 172.21.15.119:0/3316696419 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c011b30 con 0x7f02380b16d0 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.283+0000 7f023d79d700 1 -- 172.21.15.119:0/3316696419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 msgr2=0x7f02380b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.283+0000 7f023d79d700 1 --2- 172.21.15.119:0/3316696419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 0x7f02380b1b40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f022c0098e0 tx=0x7f022c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.284+0000 7f023d79d700 1 -- 172.21.15.119:0/3316696419 shutdown_connections 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.284+0000 7f023d79d700 1 --2- 172.21.15.119:0/3316696419 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b33b0 0x7f02380b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.284+0000 7f023d79d700 1 --2- 172.21.15.119:0/3316696419 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 0x7f02380b1b40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.284+0000 7f023d79d700 1 -- 172.21.15.119:0/3316696419 >> 172.21.15.119:0/3316696419 conn(0x7f02380d10c0 msgr2=0x7f02380affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.284+0000 7f023d79d700 1 -- 172.21.15.119:0/3316696419 shutdown_connections 2023-10-08T02:12:18.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.284+0000 7f023d79d700 1 -- 172.21.15.119:0/3316696419 wait complete. 2023-10-08T02:12:18.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.285+0000 7f023d79d700 1 Processor -- start 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.285+0000 7f023d79d700 1 -- start start 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f023d79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 0x7f023815c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f023d79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02380b33b0 0x7f0238157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f023d79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b4e70 0x7f0238158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f023d79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023815b8d0 con 0x7f02380b33b0 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f023d79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023815b5f0 con 0x7f02380b16d0 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f023d79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0238153e00 con 0x7f02380b4e70 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f02377fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02380b33b0 0x7f0238157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f02377fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02380b33b0 0x7f0238157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32772/0 (socket says 172.21.15.119:32772) 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.286+0000 7f02377fe700 1 -- 172.21.15.119:0/4225234408 learned_addr learned my addr 172.21.15.119:0/4225234408 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f0237fff700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 0x7f023815c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f023cf9c700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b4e70 0x7f0238158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f02377fe700 1 -- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b4e70 msgr2=0x7f0238158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f02377fe700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b4e70 0x7f0238158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f02377fe700 1 -- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 msgr2=0x7f023815c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f02377fe700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 0x7f023815c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f02377fe700 1 -- 172.21.15.119:0/4225234408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0230009a20 con 0x7f02380b33b0 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f023cf9c700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b4e70 0x7f0238158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f02377fe700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02380b33b0 0x7f0238157d30 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f022c009ed0 tx=0x7f022c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:18.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f021ffff700 1 -- 172.21.15.119:0/4225234408 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c011b10 con 0x7f02380b33b0 2023-10-08T02:12:18.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f021ffff700 1 -- 172.21.15.119:0/4225234408 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022c000c90 con 0x7f02380b33b0 2023-10-08T02:12:18.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f021ffff700 1 -- 172.21.15.119:0/4225234408 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022c01a680 con 0x7f02380b33b0 2023-10-08T02:12:18.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.287+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c009420 con 0x7f02380b33b0 2023-10-08T02:12:18.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.288+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02381543c0 con 0x7f02380b33b0 2023-10-08T02:12:18.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.288+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0238153c60 con 0x7f02380b33b0 2023-10-08T02:12:18.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.289+0000 7f021ffff700 1 -- 172.21.15.119:0/4225234408 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f022c01aa70 con 0x7f02380b33b0 2023-10-08T02:12:18.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.290+0000 7f021ffff700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0224066f30 0x7f02240693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.290+0000 7f021ffff700 1 -- 172.21.15.119:0/4225234408 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f022c0861d0 con 0x7f02380b33b0 2023-10-08T02:12:18.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.290+0000 7f0237fff700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0224066f30 0x7f02240693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.291+0000 7f0237fff700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0224066f30 0x7f02240693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0228007fd0 tx=0x7f0228009040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:18.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.294+0000 7f021ffff700 1 -- 172.21.15.119:0/4225234408 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0238153c60 con 0x7f02380b33b0 2023-10-08T02:12:18.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.486+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0238153c60 con 0x7f02380b33b0 2023-10-08T02:12:18.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.491+0000 7f021ffff700 1 -- 172.21.15.119:0/4225234408 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0238153c60 con 0x7f02380b33b0 2023-10-08T02:12:18.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0224066f30 msgr2=0x7f02240693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0224066f30 0x7f02240693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0228007fd0 tx=0x7f0228009040 comp rx=0 tx=0).stop 2023-10-08T02:12:18.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02380b33b0 msgr2=0x7f0238157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02380b33b0 0x7f0238157d30 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f022c009ed0 tx=0x7f022c017000 comp rx=0 tx=0).stop 2023-10-08T02:12:18.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 shutdown_connections 2023-10-08T02:12:18.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0224066f30 0x7f02240693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f02380b4e70 0x7f0238158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f02380b16d0 0x7f023815c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 --2- 172.21.15.119:0/4225234408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02380b33b0 0x7f0238157d30 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 >> 172.21.15.119:0/4225234408 conn(0x7f02380d10c0 msgr2=0x7f0238159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:18.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 shutdown_connections 2023-10-08T02:12:18.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.493+0000 7f023d79d700 1 -- 172.21.15.119:0/4225234408 wait complete. 2023-10-08T02:12:18.511 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:12:18.511 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:12:18.593 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:12:18.677 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:12:18.677 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T02:12:18.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:12:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.856+0000 7fd8f53ff700 1 Processor -- start 2023-10-08T02:12:18.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.856+0000 7fd8f53ff700 1 -- start start 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.857+0000 7fd8f53ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8f0155c10 0x7fd8f0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.857+0000 7fd8f53ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f00ac230 0x7fd8f00ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.857+0000 7fd8f53ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f00ad930 con 0x7fd8f01568a0 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.857+0000 7fd8f53ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f00acd20 con 0x7fd8f0155c10 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.857+0000 7fd8f53ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00ad000 con 0x7fd8f00ac230 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8eeffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd8f0155c10 0x7fd8f0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8ef7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd8f01568a0 0x7fd8f0156cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47804/0 (socket says 172.21.15.119:47804) 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8ef7fe700 1 -- 172.21.15.119:0/1771776802 learned_addr learned my addr 172.21.15.119:0/1771776802 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:18.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8ee7fc700 1 --2- 172.21.15.119:0/1771776802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f00ac230 0x7fd8f00ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8eeffd700 1 -- 172.21.15.119:0/1771776802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f00ac230 msgr2=0x7fd8f00ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:18.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8eeffd700 1 --2- 172.21.15.119:0/1771776802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f00ac230 0x7fd8f00ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8eeffd700 1 -- 172.21.15.119:0/1771776802 >> v1:172.21.15.119:6789/0 conn(0x7fd8f01568a0 legacy=0x7fd8f0156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8edffb700 1 -- 172.21.15.119:0/1771776802 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850883233 0 0) 0x7fd8f00ad930 con 0x7fd8f01568a0 2023-10-08T02:12:18.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8eeffd700 1 -- 172.21.15.119:0/1771776802 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f00ad2e0 con 0x7fd8f0155c10 2023-10-08T02:12:18.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.858+0000 7fd8eeffd700 1 --2- 172.21.15.119:0/1771776802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f0155fe0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00098e0 tx=0x7fd8e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7343aa90f212f25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:18.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8edffb700 1 -- 172.21.15.119:0/1771776802 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000b640 con 0x7fd8f0155c10 2023-10-08T02:12:18.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8edffb700 1 -- 172.21.15.119:0/1771776802 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e000b7a0 con 0x7fd8f0155c10 2023-10-08T02:12:18.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8edffb700 1 -- 172.21.15.119:0/1771776802 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000bb40 con 0x7fd8f0155c10 2023-10-08T02:12:18.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8f53ff700 1 -- 172.21.15.119:0/1771776802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 msgr2=0x7fd8f0155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/1771776802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f0155fe0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00098e0 tx=0x7fd8e00091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8f53ff700 1 -- 172.21.15.119:0/1771776802 shutdown_connections 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/1771776802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f0155fe0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/1771776802 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f00ac230 0x7fd8f00ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8f53ff700 1 -- 172.21.15.119:0/1771776802 >> 172.21.15.119:0/1771776802 conn(0x7fd8f006a320 msgr2=0x7fd8f00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8f53ff700 1 -- 172.21.15.119:0/1771776802 shutdown_connections 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.859+0000 7fd8f53ff700 1 -- 172.21.15.119:0/1771776802 wait complete. 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.861+0000 7fd8f53ff700 1 Processor -- start 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.861+0000 7fd8f53ff700 1 -- start start 2023-10-08T02:12:18.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8f53ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f00ac230 0x7fd8f0157780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8f53ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f00a6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8f53ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f01568a0 0x7fd8f014c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8f53ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00a9420 con 0x7fd8f0155c10 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8f53ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00a7ef0 con 0x7fd8f00ac230 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8f53ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f00a73b0 con 0x7fd8f01568a0 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f00a6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ee7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f00a6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32790/0 (socket says 172.21.15.119:32790) 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ee7fc700 1 -- 172.21.15.119:0/541312609 learned_addr learned my addr 172.21.15.119:0/541312609 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8eeffd700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f00ac230 0x7fd8f0157780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ef7fe700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f01568a0 0x7fd8f014c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ee7fc700 1 -- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f01568a0 msgr2=0x7fd8f014c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ee7fc700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f01568a0 0x7fd8f014c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ee7fc700 1 -- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f00ac230 msgr2=0x7fd8f0157780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:18.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ee7fc700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f00ac230 0x7fd8f0157780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:18.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.862+0000 7fd8ee7fc700 1 -- 172.21.15.119:0/541312609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e0009420 con 0x7fd8f0155c10 2023-10-08T02:12:18.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.863+0000 7fd8eeffd700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f00ac230 0x7fd8f0157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:18.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.863+0000 7fd8ef7fe700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f01568a0 0x7fd8f014c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:18.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.863+0000 7fd8ee7fc700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f00a6e70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ec80 tx=0x7fd8e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:18.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.863+0000 7fd8d6ffd700 1 -- 172.21.15.119:0/541312609 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e400c9d0 con 0x7fd8f0155c10 2023-10-08T02:12:18.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.863+0000 7fd8d6ffd700 1 -- 172.21.15.119:0/541312609 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e400cb30 con 0x7fd8f0155c10 2023-10-08T02:12:18.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.863+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f00a76b0 con 0x7fd8f0155c10 2023-10-08T02:12:18.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.863+0000 7fd8d6ffd700 1 -- 172.21.15.119:0/541312609 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4013470 con 0x7fd8f0155c10 2023-10-08T02:12:18.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.863+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f00a9650 con 0x7fd8f0155c10 2023-10-08T02:12:18.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.864+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f00a99c0 con 0x7fd8f0155c10 2023-10-08T02:12:18.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.865+0000 7fd8d6ffd700 1 -- 172.21.15.119:0/541312609 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd8e4013820 con 0x7fd8f0155c10 2023-10-08T02:12:18.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.866+0000 7fd8d6ffd700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd8dc066f30 0x7fd8dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:18.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.866+0000 7fd8d6ffd700 1 -- 172.21.15.119:0/541312609 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fd8e4084a60 con 0x7fd8f0155c10 2023-10-08T02:12:18.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.866+0000 7fd8eeffd700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd8dc066f30 0x7fd8dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:18.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.867+0000 7fd8eeffd700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd8dc066f30 0x7fd8dc0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00091e0 tx=0x7fd8e0017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:18.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:18.870+0000 7fd8d6ffd700 1 -- 172.21.15.119:0/541312609 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd8e4051b30 con 0x7fd8f0155c10 2023-10-08T02:12:19.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.102+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8f004c3c0 con 0x7fd8f0155c10 2023-10-08T02:12:19.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.103+0000 7fd8d6ffd700 1 -- 172.21.15.119:0/541312609 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fd8e4017de0 con 0x7fd8f0155c10 2023-10-08T02:12:19.104 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:19.104 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7331,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/3377468407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3377468407},{"type":"v1","addr":"172.21.15.184:6835","nonce":3377468407}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":8002,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/603902131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":603902131},{"type":"v1","addr":"172.21.15.184:6834","nonce":603902131}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8655,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/3930162267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":3930162267},{"type":"v1","addr":"172.21.15.119:6834","nonce":3930162267}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66},{"gid":8658,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/237096421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":237096421},{"type":"v1","addr":"172.21.15.119:6835","nonce":237096421}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":66}],"filesystems":[]} 2023-10-08T02:12:19.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd8dc066f30 msgr2=0x7fd8dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:19.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd8dc066f30 0x7fd8dc0693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00091e0 tx=0x7fd8e0017000 comp rx=0 tx=0).stop 2023-10-08T02:12:19.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 msgr2=0x7fd8f00a6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f00a6e70 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400ec80 tx=0x7fd8e40091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 shutdown_connections 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd8dc066f30 0x7fd8dc0693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd8f01568a0 0x7fd8f014c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd8f00ac230 0x7fd8f0157780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 --2- 172.21.15.119:0/541312609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8f0155c10 0x7fd8f00a6e70 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.105+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 >> 172.21.15.119:0/541312609 conn(0x7fd8f006a320 msgr2=0x7fd8f00af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.106+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 shutdown_connections 2023-10-08T02:12:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.106+0000 7fd8f53ff700 1 -- 172.21.15.119:0/541312609 wait complete. 2023-10-08T02:12:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 73 2023-10-08T02:12:19.124 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T02:12:19.124 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T02:12:19.124 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T02:12:19.124 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:12:19.124 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T02:12:19.124 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T02:12:19.125 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T02:12:19.125 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:12:19.125 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T02:12:19.125 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T02:12:19.125 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T02:12:19.125 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:12:19.126 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T02:12:19.126 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T02:12:19.126 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T02:12:19.126 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:12:19.139 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T02:12:19.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T02:12:19.140 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T02:12:19.140 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T02:12:19.141 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T02:12:19.142 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T02:12:19.142 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T02:12:19.142 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T02:12:19.184 INFO:tasks.ceph.mds.a:Started 2023-10-08T02:12:19.184 DEBUG:teuthology.parallel:result is None 2023-10-08T02:12:19.186 INFO:tasks.ceph.mds.b:Started 2023-10-08T02:12:19.186 DEBUG:teuthology.parallel:result is None 2023-10-08T02:12:19.187 INFO:tasks.ceph.mds.c:Started 2023-10-08T02:12:19.187 DEBUG:teuthology.parallel:result is None 2023-10-08T02:12:19.189 INFO:tasks.ceph.mds.d:Started 2023-10-08T02:12:19.189 DEBUG:teuthology.parallel:result is None 2023-10-08T02:12:19.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T02:12:19.316 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T02:12:19.321 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T02:12:19.325 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T02:12:19.326 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T02:12:19.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.383+0000 7fdfaf39d700 1 Processor -- start 2023-10-08T02:12:19.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.383+0000 7fdfaf39d700 1 -- start start 2023-10-08T02:12:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.384+0000 7fdfaf39d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfa8150550 0x7fdfa8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.384+0000 7fdfaf39d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa806e2a0 con 0x7fdfa8150180 2023-10-08T02:12:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.384+0000 7fdfaf39d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa806b2e0 con 0x7fdfa80c7410 2023-10-08T02:12:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.384+0000 7fdfaf39d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa80c78e0 con 0x7fdfa8150550 2023-10-08T02:12:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.384+0000 7fdfae39b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdfa8150180 0x7fdfa80c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47828/0 (socket says 172.21.15.119:47828) 2023-10-08T02:12:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.384+0000 7fdfae39b700 1 -- 172.21.15.119:0/1379418911 learned_addr learned my addr 172.21.15.119:0/1379418911 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfadb9a700 1 --2- 172.21.15.119:0/1379418911 >> v2:172.21.15.119:3300/0 conn(0x7fdfa8150550 0x7fdfa8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfad399700 1 -- 172.21.15.119:0/1379418911 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 182870488 0 0) 0x7fdfa806e2a0 con 0x7fdfa8150180 2023-10-08T02:12:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfad399700 1 -- 172.21.15.119:0/1379418911 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf94004680 con 0x7fdfa8150180 2023-10-08T02:12:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfad399700 1 -- 172.21.15.119:0/1379418911 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760199096 0 0) 0x7fdfa806b2e0 con 0x7fdfa80c7410 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfadb9a700 1 -- 172.21.15.119:0/1379418911 >> v1:172.21.15.184:6789/0 conn(0x7fdfa80c7410 legacy=0x7fdfa8069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfadb9a700 1 -- 172.21.15.119:0/1379418911 >> v1:172.21.15.119:6789/0 conn(0x7fdfa8150180 legacy=0x7fdfa80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfadb9a700 1 -- 172.21.15.119:0/1379418911 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa80c7bc0 con 0x7fdfa8150550 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfadb9a700 1 --2- 172.21.15.119:0/1379418911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa8157770 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fdf980098e0 tx=0x7fdf980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c404a5e58a3fbe65 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.385+0000 7fdfad399700 1 -- 172.21.15.119:0/1379418911 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800b640 con 0x7fdfa8150550 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfad399700 1 -- 172.21.15.119:0/1379418911 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9800b7a0 con 0x7fdfa8150550 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfad399700 1 -- 172.21.15.119:0/1379418911 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9800bb40 con 0x7fdfa8150550 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfaf39d700 1 -- 172.21.15.119:0/1379418911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 msgr2=0x7fdfa8157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfaf39d700 1 --2- 172.21.15.119:0/1379418911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa8157770 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7fdf980098e0 tx=0x7fdf980091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfaf39d700 1 -- 172.21.15.119:0/1379418911 shutdown_connections 2023-10-08T02:12:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfaf39d700 1 --2- 172.21.15.119:0/1379418911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa8157770 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfaf39d700 1 -- 172.21.15.119:0/1379418911 >> 172.21.15.119:0/1379418911 conn(0x7fdfa80cc1f0 msgr2=0x7fdfa80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:19.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfaf39d700 1 -- 172.21.15.119:0/1379418911 shutdown_connections 2023-10-08T02:12:19.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.386+0000 7fdfaf39d700 1 -- 172.21.15.119:0/1379418911 wait complete. 2023-10-08T02:12:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.388+0000 7fdfaf39d700 1 Processor -- start 2023-10-08T02:12:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.388+0000 7fdfaf39d700 1 -- start start 2023-10-08T02:12:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaf39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa80c7410 0x7fdfa80adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaf39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8150180 0x7fdfa80b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaf39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa81587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaf39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80b00c0 con 0x7fdfa8150550 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaf39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa816d4a0 con 0x7fdfa80c7410 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaf39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80afb00 con 0x7fdfa8150180 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaeb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa81587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaeb9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa81587c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32840/0 (socket says 172.21.15.119:32840) 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaeb9c700 1 -- 172.21.15.119:0/4005502988 learned_addr learned my addr 172.21.15.119:0/4005502988 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfae39b700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa80c7410 0x7fdfa80adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfadb9a700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8150180 0x7fdfa80b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaeb9c700 1 -- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8150180 msgr2=0x7fdfa80b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaeb9c700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8150180 0x7fdfa80b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaeb9c700 1 -- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa80c7410 msgr2=0x7fdfa80adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaeb9c700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa80c7410 0x7fdfa80adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfaeb9c700 1 -- 172.21.15.119:0/4005502988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf98009420 con 0x7fdfa8150550 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfae39b700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa80c7410 0x7fdfa80adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.389+0000 7fdfadb9a700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8150180 0x7fdfa80b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.390+0000 7fdfaeb9c700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa81587c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0002230 tx=0x7fdfa00016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.390+0000 7fdf9e7fc700 1 -- 172.21.15.119:0/4005502988 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000cea0 con 0x7fdfa8150550 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.390+0000 7fdf9e7fc700 1 -- 172.21.15.119:0/4005502988 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa000a7e0 con 0x7fdfa8150550 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.390+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa80afde0 con 0x7fdfa8150550 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.390+0000 7fdf9e7fc700 1 -- 172.21.15.119:0/4005502988 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000aba0 con 0x7fdfa8150550 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.390+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa80ae040 con 0x7fdfa8150550 2023-10-08T02:12:19.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.391+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa804c3c0 con 0x7fdfa8150550 2023-10-08T02:12:19.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.392+0000 7fdf9e7fc700 1 -- 172.21.15.119:0/4005502988 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfa0012420 con 0x7fdfa8150550 2023-10-08T02:12:19.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.392+0000 7fdf9e7fc700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf8c066f30 0x7fdf8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:19.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.393+0000 7fdf9e7fc700 1 -- 172.21.15.119:0/4005502988 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5962+0+0 (secure 0 0 0) 0x7fdfa00849d0 con 0x7fdfa8150550 2023-10-08T02:12:19.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.393+0000 7fdfae39b700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf8c066f30 0x7fdf8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:19.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.393+0000 7fdfae39b700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf8c066f30 0x7fdf8c0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40058d0 tx=0x7fdfa4009040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:19.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.397+0000 7fdf9e7fc700 1 -- 172.21.15.119:0/4005502988 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfa0051b40 con 0x7fdfa8150550 2023-10-08T02:12:19.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:19.612+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fdfa806ada0 con 0x7fdfa8150550 2023-10-08T02:12:20.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.514+0000 7fdf9e7fc700 1 -- 172.21.15.119:0/4005502988 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v167) v1 ==== 97+0+0 (secure 0 0 0) 0x7fdfa0055160 con 0x7fdfa8150550 2023-10-08T02:12:20.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf8c066f30 msgr2=0x7fdf8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:20.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf8c066f30 0x7fdf8c0693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdfa40058d0 tx=0x7fdfa4009040 comp rx=0 tx=0).stop 2023-10-08T02:12:20.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 msgr2=0x7fdfa81587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:20.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa81587c0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0002230 tx=0x7fdfa00016f0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 shutdown_connections 2023-10-08T02:12:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdf8c066f30 0x7fdf8c0693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfa8150550 0x7fdfa81587c0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfa80c7410 0x7fdfa80adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 --2- 172.21.15.119:0/4005502988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfa8150180 0x7fdfa80b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 >> 172.21.15.119:0/4005502988 conn(0x7fdfa80cc1f0 msgr2=0x7fdfa8156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.517+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 shutdown_connections 2023-10-08T02:12:20.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.518+0000 7fdfaf39d700 1 -- 172.21.15.119:0/4005502988 wait complete. 2023-10-08T02:12:20.519 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T02:12:20.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T02:12:20.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.723+0000 7f6f68d9d700 1 Processor -- start 2023-10-08T02:12:20.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.723+0000 7f6f68d9d700 1 -- start start 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f68d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6f6406d950 0x7f6f640c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f68d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6406ae70 con 0x7f6f6406dd20 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f68d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f640c7a30 con 0x7f6f6406a0a0 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f68d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6406a540 con 0x7f6f6406d950 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f63fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6f6406dd20 0x7f6f640c7320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47842/0 (socket says 172.21.15.119:47842) 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f63fff700 1 -- 172.21.15.119:0/776840266 learned_addr learned my addr 172.21.15.119:0/776840266 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f637fe700 1 --2- 172.21.15.119:0/776840266 >> v2:172.21.15.184:3300/0 conn(0x7f6f6406d950 0x7f6f640c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f627fc700 1 -- 172.21.15.119:0/776840266 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303091746 0 0) 0x7f6f6406ae70 con 0x7f6f6406dd20 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f627fc700 1 -- 172.21.15.119:0/776840266 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f4c003570 con 0x7f6f6406dd20 2023-10-08T02:12:20.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f637fe700 1 -- 172.21.15.119:0/776840266 >> v1:172.21.15.184:6789/0 conn(0x7f6f6406a0a0 legacy=0x7f6f640c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:20.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f637fe700 1 -- 172.21.15.119:0/776840266 >> v1:172.21.15.119:6789/0 conn(0x7f6f6406dd20 legacy=0x7f6f640c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:20.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f637fe700 1 -- 172.21.15.119:0/776840266 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6406a820 con 0x7f6f6406d950 2023-10-08T02:12:20.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.725+0000 7f6f637fe700 1 --2- 172.21.15.119:0/776840266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406d950 0x7f6f640c8350 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f540098e0 tx=0x7f6f540091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=884bd964b62d9be6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:20.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f627fc700 1 -- 172.21.15.119:0/776840266 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400b640 con 0x7f6f6406d950 2023-10-08T02:12:20.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f627fc700 1 -- 172.21.15.119:0/776840266 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5400b7a0 con 0x7f6f6406d950 2023-10-08T02:12:20.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f627fc700 1 -- 172.21.15.119:0/776840266 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5400bb20 con 0x7f6f6406d950 2023-10-08T02:12:20.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f68d9d700 1 -- 172.21.15.119:0/776840266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406d950 msgr2=0x7f6f640c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:20.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f68d9d700 1 --2- 172.21.15.119:0/776840266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406d950 0x7f6f640c8350 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6f540098e0 tx=0x7f6f540091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f68d9d700 1 -- 172.21.15.119:0/776840266 shutdown_connections 2023-10-08T02:12:20.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f68d9d700 1 --2- 172.21.15.119:0/776840266 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406d950 0x7f6f640c8350 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f68d9d700 1 -- 172.21.15.119:0/776840266 >> 172.21.15.119:0/776840266 conn(0x7f6f640cc1f0 msgr2=0x7f6f640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:20.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f68d9d700 1 -- 172.21.15.119:0/776840266 shutdown_connections 2023-10-08T02:12:20.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.726+0000 7f6f68d9d700 1 -- 172.21.15.119:0/776840266 wait complete. 2023-10-08T02:12:20.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.728+0000 7f6f68d9d700 1 Processor -- start 2023-10-08T02:12:20.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.728+0000 7f6f68d9d700 1 -- start start 2023-10-08T02:12:20.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f68d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406a0a0 0x7f6f640ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:20.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f68d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6406d950 0x7f6f640cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:20.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f68d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6406dd20 0x7f6f640c0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f68d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640ca4c0 con 0x7f6f6406dd20 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f68d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640cafd0 con 0x7f6f6406a0a0 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f68d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f640b15b0 con 0x7f6f6406d950 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f63fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6406dd20 0x7f6f640c0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f63fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6406dd20 0x7f6f640c0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32844/0 (socket says 172.21.15.119:32844) 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f63fff700 1 -- 172.21.15.119:0/2043058788 learned_addr learned my addr 172.21.15.119:0/2043058788 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f637fe700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406a0a0 0x7f6f640ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.729+0000 7f6f62ffd700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6406d950 0x7f6f640cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f63fff700 1 -- 172.21.15.119:0/2043058788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6406d950 msgr2=0x7f6f640cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f63fff700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6406d950 0x7f6f640cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f63fff700 1 -- 172.21.15.119:0/2043058788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406a0a0 msgr2=0x7f6f640ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f63fff700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406a0a0 0x7f6f640ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f63fff700 1 -- 172.21.15.119:0/2043058788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f54009420 con 0x7f6f6406dd20 2023-10-08T02:12:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f63fff700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6406dd20 0x7f6f640c0f90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00cca0 tx=0x7f6f5c001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f4b7fe700 1 -- 172.21.15.119:0/2043058788 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00a980 con 0x7f6f6406dd20 2023-10-08T02:12:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f4b7fe700 1 -- 172.21.15.119:0/2043058788 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c00aae0 con 0x7f6f6406dd20 2023-10-08T02:12:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f640ca6a0 con 0x7f6f6406dd20 2023-10-08T02:12:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f4b7fe700 1 -- 172.21.15.119:0/2043058788 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00ae60 con 0x7f6f6406dd20 2023-10-08T02:12:20.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.730+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f640cac60 con 0x7f6f6406dd20 2023-10-08T02:12:20.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.731+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f640cb160 con 0x7f6f6406dd20 2023-10-08T02:12:20.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.732+0000 7f6f4b7fe700 1 -- 172.21.15.119:0/2043058788 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6f5c007300 con 0x7f6f6406dd20 2023-10-08T02:12:20.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.733+0000 7f6f4b7fe700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6f50066f30 0x7f6f500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:20.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.733+0000 7f6f4b7fe700 1 -- 172.21.15.119:0/2043058788 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f6f5c0848f0 con 0x7f6f6406dd20 2023-10-08T02:12:20.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.733+0000 7f6f637fe700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6f50066f30 0x7f6f500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:20.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.734+0000 7f6f637fe700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6f50066f30 0x7f6f500693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6f540093f0 tx=0x7f6f54017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:20.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.737+0000 7f6f4b7fe700 1 -- 172.21.15.119:0/2043058788 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6f5c051c30 con 0x7f6f6406dd20 2023-10-08T02:12:20.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.998+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6f640ae430 con 0x7f6f6406dd20 2023-10-08T02:12:20.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:20.999+0000 7f6f4b7fe700 1 -- 172.21.15.119:0/2043058788 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v43) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6f5c055250 con 0x7f6f6406dd20 2023-10-08T02:12:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:20.999 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:12:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T02:12:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T02:12:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T02:12:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:20.999 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:12:21.000 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:12:21.001 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.002 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.003 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.004 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.005 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.006 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T02:12:21.007 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.008 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.009 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T02:12:21.010 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:12:21.011 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:12:21.012 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.013 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T02:12:21.014 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T02:12:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:12:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:12:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T02:12:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:12:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:21.015 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T02:12:21.015 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:12:21.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6f50066f30 msgr2=0x7f6f500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6f50066f30 0x7f6f500693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6f540093f0 tx=0x7f6f54017000 comp rx=0 tx=0).stop 2023-10-08T02:12:21.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6406dd20 msgr2=0x7f6f640c0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6406dd20 0x7f6f640c0f90 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c00cca0 tx=0x7f6f5c001ae0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 shutdown_connections 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f6f50066f30 0x7f6f500693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f6406dd20 0x7f6f640c0f90 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6f6406a0a0 0x7f6f640ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 --2- 172.21.15.119:0/2043058788 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6f6406d950 0x7f6f640cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 >> 172.21.15.119:0/2043058788 conn(0x7f6f640cc1f0 msgr2=0x7f6f64153150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 shutdown_connections 2023-10-08T02:12:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.001+0000 7f6f68d9d700 1 -- 172.21.15.119:0/2043058788 wait complete. 2023-10-08T02:12:21.018 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:12:21.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.199+0000 7fe693fff700 1 Processor -- start 2023-10-08T02:12:21.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.200+0000 7fe693fff700 1 -- start start 2023-10-08T02:12:21.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe693fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe694142ea0 0x7fe6940b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe693fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69406c0f0 0x7fe69406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe693fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6940b40f0 con 0x7fe6940ad870 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe693fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe694069640 con 0x7fe694142ea0 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe693fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe694069920 con 0x7fe69406c0f0 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe6927fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe6940ad870 0x7fe69406b9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47856/0 (socket says 172.21.15.119:47856) 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe692ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe694142ea0 0x7fe6940b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe6927fc700 1 -- 172.21.15.119:0/3797371089 learned_addr learned my addr 172.21.15.119:0/3797371089 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.201+0000 7fe6937fe700 1 --2- 172.21.15.119:0/3797371089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69406c0f0 0x7fe69406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe692ffd700 1 -- 172.21.15.119:0/3797371089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69406c0f0 msgr2=0x7fe69406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe692ffd700 1 --2- 172.21.15.119:0/3797371089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69406c0f0 0x7fe69406a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe692ffd700 1 -- 172.21.15.119:0/3797371089 >> v1:172.21.15.119:6789/0 conn(0x7fe6940ad870 legacy=0x7fe69406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe691ffb700 1 -- 172.21.15.119:0/3797371089 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260768762 0 0) 0x7fe6940b40f0 con 0x7fe6940ad870 2023-10-08T02:12:21.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe692ffd700 1 -- 172.21.15.119:0/3797371089 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe694069c00 con 0x7fe694142ea0 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe692ffd700 1 --2- 172.21.15.119:0/3797371089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe694142ea0 0x7fe6940b3630 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fe6840098e0 tx=0x7fe6840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601748e33108e56d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe691ffb700 1 -- 172.21.15.119:0/3797371089 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684009e80 con 0x7fe694142ea0 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe691ffb700 1 -- 172.21.15.119:0/3797371089 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400b860 con 0x7fe694142ea0 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.202+0000 7fe691ffb700 1 -- 172.21.15.119:0/3797371089 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400bbe0 con 0x7fe694142ea0 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.203+0000 7fe693fff700 1 -- 172.21.15.119:0/3797371089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe694142ea0 msgr2=0x7fe6940b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.203+0000 7fe693fff700 1 --2- 172.21.15.119:0/3797371089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe694142ea0 0x7fe6940b3630 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fe6840098e0 tx=0x7fe6840091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.203+0000 7fe693fff700 1 -- 172.21.15.119:0/3797371089 shutdown_connections 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.203+0000 7fe693fff700 1 --2- 172.21.15.119:0/3797371089 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe69406c0f0 0x7fe69406a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.203+0000 7fe693fff700 1 --2- 172.21.15.119:0/3797371089 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe694142ea0 0x7fe6940b3630 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.203+0000 7fe693fff700 1 -- 172.21.15.119:0/3797371089 >> 172.21.15.119:0/3797371089 conn(0x7fe6940ce430 msgr2=0x7fe6940ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.203+0000 7fe693fff700 1 -- 172.21.15.119:0/3797371089 shutdown_connections 2023-10-08T02:12:21.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.203+0000 7fe693fff700 1 -- 172.21.15.119:0/3797371089 wait complete. 2023-10-08T02:12:21.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.204+0000 7fe693fff700 1 Processor -- start 2023-10-08T02:12:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.204+0000 7fe693fff700 1 -- start start 2023-10-08T02:12:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe693fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe69406c0f0 0x7fe6940c8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe693fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe694142ea0 0x7fe694158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe693fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6941699a0 0x7fe694169e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe693fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940c75e0 con 0x7fe69406c0f0 2023-10-08T02:12:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe693fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940c7ba0 con 0x7fe694142ea0 2023-10-08T02:12:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe693fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6940c8570 con 0x7fe6941699a0 2023-10-08T02:12:21.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe692ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe69406c0f0 0x7fe6940c8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe692ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe69406c0f0 0x7fe6940c8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32862/0 (socket says 172.21.15.119:32862) 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.205+0000 7fe692ffd700 1 -- 172.21.15.119:0/4147077047 learned_addr learned my addr 172.21.15.119:0/4147077047 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe6927fc700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe694142ea0 0x7fe694158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe6937fe700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6941699a0 0x7fe694169e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe692ffd700 1 -- 172.21.15.119:0/4147077047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6941699a0 msgr2=0x7fe694169e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe692ffd700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6941699a0 0x7fe694169e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe692ffd700 1 -- 172.21.15.119:0/4147077047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe694142ea0 msgr2=0x7fe694158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe692ffd700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe694142ea0 0x7fe694158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe692ffd700 1 -- 172.21.15.119:0/4147077047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684009420 con 0x7fe69406c0f0 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe692ffd700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe69406c0f0 0x7fe6940c8030 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fe6940cd940 tx=0x7fe684017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe67affd700 1 -- 172.21.15.119:0/4147077047 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684011420 con 0x7fe69406c0f0 2023-10-08T02:12:21.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe67affd700 1 -- 172.21.15.119:0/4147077047 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6840115c0 con 0x7fe69406c0f0 2023-10-08T02:12:21.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe67affd700 1 -- 172.21.15.119:0/4147077047 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684011980 con 0x7fe69406c0f0 2023-10-08T02:12:21.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69416a4d0 con 0x7fe69406c0f0 2023-10-08T02:12:21.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.206+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6940c7830 con 0x7fe69406c0f0 2023-10-08T02:12:21.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.207+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6940c7e80 con 0x7fe69406c0f0 2023-10-08T02:12:21.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.208+0000 7fe67affd700 1 -- 172.21.15.119:0/4147077047 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe6840253f0 con 0x7fe69406c0f0 2023-10-08T02:12:21.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.209+0000 7fe67affd700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe680066f30 0x7fe6800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.209+0000 7fe67affd700 1 -- 172.21.15.119:0/4147077047 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5317+0+0 (secure 0 0 0) 0x7fe684085d80 con 0x7fe69406c0f0 2023-10-08T02:12:21.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.209+0000 7fe6927fc700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe680066f30 0x7fe6800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.210+0000 7fe6927fc700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe680066f30 0x7fe6800693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6940b0a30 tx=0x7fe688009040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:21.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.213+0000 7fe67affd700 1 -- 172.21.15.119:0/4147077047 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe684053170 con 0x7fe69406c0f0 2023-10-08T02:12:21.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.405+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe69404c3c0 con 0x7fe69406c0f0 2023-10-08T02:12:21.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.406+0000 7fe67affd700 1 -- 172.21.15.119:0/4147077047 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+12905 (secure 0 0 0) 0x7fe684056790 con 0x7fe69406c0f0 2023-10-08T02:12:21.406 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:21.406 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":167,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:12:20.510986+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:12:21.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe680066f30 msgr2=0x7fe6800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe680066f30 0x7fe6800693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6940b0a30 tx=0x7fe688009040 comp rx=0 tx=0).stop 2023-10-08T02:12:21.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe69406c0f0 msgr2=0x7fe6940c8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe69406c0f0 0x7fe6940c8030 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fe6940cd940 tx=0x7fe684017000 comp rx=0 tx=0).stop 2023-10-08T02:12:21.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 shutdown_connections 2023-10-08T02:12:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fe680066f30 0x7fe6800693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe6941699a0 0x7fe694169e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe69406c0f0 0x7fe6940c8030 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 --2- 172.21.15.119:0/4147077047 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe694142ea0 0x7fe694158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 >> 172.21.15.119:0/4147077047 conn(0x7fe6940ce430 msgr2=0x7fe69406ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 shutdown_connections 2023-10-08T02:12:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.408+0000 7fe693fff700 1 -- 172.21.15.119:0/4147077047 wait complete. 2023-10-08T02:12:21.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T02:12:21.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:12:21.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.612+0000 7f7fd1bff700 1 Processor -- start 2023-10-08T02:12:21.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.612+0000 7f7fd1bff700 1 -- start start 2023-10-08T02:12:21.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.613+0000 7f7fd1bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc0637d0 0x7f7fcc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.613+0000 7f7fd1bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7fcc06a2b0 0x7f7fcc06a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.613+0000 7f7fd1bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fcc0ac450 con 0x7f7fcc06b160 2023-10-08T02:12:21.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.613+0000 7f7fd1bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fcc0693d0 con 0x7f7fcc06a2b0 2023-10-08T02:12:21.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.613+0000 7f7fd1bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc069d00 con 0x7f7fcc0637d0 2023-10-08T02:12:21.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcaffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc0637d0 0x7f7fcc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcbfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7fcc06a2b0 0x7f7fcc06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcaffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc0637d0 0x7f7fcc06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:38038/0 (socket says 172.21.15.119:38038) 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcaffd700 1 -- 172.21.15.119:0/4038338264 learned_addr learned my addr 172.21.15.119:0/4038338264 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcb7fe700 1 --1- 172.21.15.119:0/4038338264 >> v1:172.21.15.184:6789/0 conn(0x7f7fcc06b160 0x7f7fcc068cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:38204/0 (socket says 172.21.15.119:38204) 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcaffd700 1 -- 172.21.15.119:0/4038338264 >> v2:172.21.15.184:3300/0 conn(0x7f7fcc06a2b0 msgr2=0x7f7fcc06a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcaffd700 1 --2- 172.21.15.119:0/4038338264 >> v2:172.21.15.184:3300/0 conn(0x7f7fcc06a2b0 0x7f7fcc06a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcaffd700 1 -- 172.21.15.119:0/4038338264 >> v1:172.21.15.184:6789/0 conn(0x7f7fcc06b160 legacy=0x7f7fcc068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcaffd700 1 -- 172.21.15.119:0/4038338264 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc068830 con 0x7f7fcc0637d0 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.614+0000 7f7fcbfff700 1 --2- 172.21.15.119:0/4038338264 >> v2:172.21.15.184:3300/0 conn(0x7f7fcc06a2b0 0x7f7fcc06a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.615+0000 7f7fcaffd700 1 --2- 172.21.15.119:0/4038338264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc0637d0 0x7f7fcc06b530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00099b0 tx=0x7f7fc0009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb9bbdce2d4b17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.615+0000 7f7fca7fc700 1 -- 172.21.15.119:0/4038338264 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0009ea0 con 0x7f7fcc0637d0 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.615+0000 7f7fca7fc700 1 -- 172.21.15.119:0/4038338264 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc000b860 con 0x7f7fcc0637d0 2023-10-08T02:12:21.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.615+0000 7f7fca7fc700 1 -- 172.21.15.119:0/4038338264 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc000bc20 con 0x7f7fcc0637d0 2023-10-08T02:12:21.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.616+0000 7f7fd1bff700 1 -- 172.21.15.119:0/4038338264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc0637d0 msgr2=0x7f7fcc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.616+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/4038338264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc0637d0 0x7f7fcc06b530 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7fc00099b0 tx=0x7f7fc0009250 comp rx=0 tx=0).stop 2023-10-08T02:12:21.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.616+0000 7f7fd1bff700 1 -- 172.21.15.119:0/4038338264 shutdown_connections 2023-10-08T02:12:21.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.616+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/4038338264 >> v2:172.21.15.184:3300/0 conn(0x7f7fcc06a2b0 0x7f7fcc06a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.616+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/4038338264 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc0637d0 0x7f7fcc06b530 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.616+0000 7f7fd1bff700 1 -- 172.21.15.119:0/4038338264 >> 172.21.15.119:0/4038338264 conn(0x7f7fcc06ded0 msgr2=0x7f7fcc06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:21.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.616+0000 7f7fd1bff700 1 -- 172.21.15.119:0/4038338264 shutdown_connections 2023-10-08T02:12:21.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.616+0000 7f7fd1bff700 1 -- 172.21.15.119:0/4038338264 wait complete. 2023-10-08T02:12:21.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.617+0000 7f7fd1bff700 1 Processor -- start 2023-10-08T02:12:21.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.617+0000 7f7fd1bff700 1 -- start start 2023-10-08T02:12:21.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.618+0000 7f7fd1bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fcc0637d0 0x7f7fcc1712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.618+0000 7f7fd1bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc06a2b0 0x7f7fcc1554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.618+0000 7f7fd1bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fcc06b160 0x7f7fcc159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.618+0000 7f7fd1bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc156920 con 0x7f7fcc06b160 2023-10-08T02:12:21.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.618+0000 7f7fd1bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc155d10 con 0x7f7fcc0637d0 2023-10-08T02:12:21.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.618+0000 7f7fd1bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc155ff0 con 0x7f7fcc06a2b0 2023-10-08T02:12:21.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.618+0000 7f7fcbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fcc06b160 0x7f7fcc159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fcc06b160 0x7f7fcc159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32872/0 (socket says 172.21.15.119:32872) 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcbfff700 1 -- 172.21.15.119:0/1794083402 learned_addr learned my addr 172.21.15.119:0/1794083402 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcb7fe700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fcc0637d0 0x7f7fcc1712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcaffd700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc06a2b0 0x7f7fcc1554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcbfff700 1 -- 172.21.15.119:0/1794083402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc06a2b0 msgr2=0x7f7fcc1554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcbfff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc06a2b0 0x7f7fcc1554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcbfff700 1 -- 172.21.15.119:0/1794083402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fcc0637d0 msgr2=0x7f7fcc1712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcbfff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fcc0637d0 0x7f7fcc1712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcbfff700 1 -- 172.21.15.119:0/1794083402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4009a20 con 0x7f7fcc06b160 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fcbfff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fcc06b160 0x7f7fcc159c30 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009fc0 tx=0x7f7fc400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:21.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fb37fe700 1 -- 172.21.15.119:0/1794083402 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400cbd0 con 0x7f7fcc06b160 2023-10-08T02:12:21.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fb37fe700 1 -- 172.21.15.119:0/1794083402 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc400f040 con 0x7f7fcc06b160 2023-10-08T02:12:21.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc00094f0 con 0x7f7fcc06b160 2023-10-08T02:12:21.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.619+0000 7f7fb37fe700 1 -- 172.21.15.119:0/1794083402 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4014480 con 0x7f7fcc06b160 2023-10-08T02:12:21.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.620+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc1565b0 con 0x7f7fcc06b160 2023-10-08T02:12:21.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.621+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc156a90 con 0x7f7fcc06b160 2023-10-08T02:12:21.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.621+0000 7f7fb37fe700 1 -- 172.21.15.119:0/1794083402 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fc4003020 con 0x7f7fcc06b160 2023-10-08T02:12:21.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.622+0000 7f7fb37fe700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fb8066f30 0x7f7fb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:21.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.622+0000 7f7fb37fe700 1 -- 172.21.15.119:0/1794083402 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f7fc4084d70 con 0x7f7fcc06b160 2023-10-08T02:12:21.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.622+0000 7f7fcb7fe700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fb8066f30 0x7f7fb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:21.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.623+0000 7f7fcb7fe700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fb8066f30 0x7f7fb80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc004fd0 tx=0x7f7fbc007eb0 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:21.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.626+0000 7f7fb37fe700 1 -- 172.21.15.119:0/1794083402 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fc40520c0 con 0x7f7fcc06b160 2023-10-08T02:12:21.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.815+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fcc04c3c0 con 0x7f7fcc06b160 2023-10-08T02:12:21.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.816+0000 7f7fb37fe700 1 -- 172.21.15.119:0/1794083402 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fc4018020 con 0x7f7fcc06b160 2023-10-08T02:12:21.815 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:12:21.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fb8066f30 msgr2=0x7f7fb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fb8066f30 0x7f7fb80693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc004fd0 tx=0x7f7fbc007eb0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fcc06b160 msgr2=0x7f7fcc159c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:21.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fcc06b160 0x7f7fcc159c30 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4009fc0 tx=0x7f7fc400b040 comp rx=0 tx=0).stop 2023-10-08T02:12:21.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 shutdown_connections 2023-10-08T02:12:21.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7fb8066f30 0x7f7fb80693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fcc06b160 0x7f7fcc159c30 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fcc0637d0 0x7f7fcc1712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 --2- 172.21.15.119:0/1794083402 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fcc06a2b0 0x7f7fcc1554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:21.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 >> 172.21.15.119:0/1794083402 conn(0x7f7fcc06ded0 msgr2=0x7f7fcc0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:21.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 shutdown_connections 2023-10-08T02:12:21.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:21.818+0000 7f7fd1bff700 1 -- 172.21.15.119:0/1794083402 wait complete. 2023-10-08T02:12:21.835 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T02:12:22.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.017+0000 7f7aa05ff700 1 Processor -- start 2023-10-08T02:12:22.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.017+0000 7f7aa05ff700 1 -- start start 2023-10-08T02:12:22.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.018+0000 7f7aa05ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980cbf30 0x7f7a980ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.018+0000 7f7aa05ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980cc300 con 0x7f7a980b1420 2023-10-08T02:12:22.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7aa05ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a980ad370 con 0x7f7a980ccf90 2023-10-08T02:12:22.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7aa05ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a98156bf0 con 0x7f7a980cbf30 2023-10-08T02:12:22.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9eb9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7a980b1420 0x7f7a980c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47860/0 (socket says 172.21.15.119:47860) 2023-10-08T02:12:22.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9eb9c700 1 -- 172.21.15.119:0/64145108 learned_addr learned my addr 172.21.15.119:0/64145108 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:22.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9e39b700 1 --2- 172.21.15.119:0/64145108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980cbf30 0x7f7a980ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9d399700 1 -- 172.21.15.119:0/64145108 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252043825 0 0) 0x7f7a980cc300 con 0x7f7a980b1420 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9d399700 1 -- 172.21.15.119:0/64145108 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a84003570 con 0x7f7a980b1420 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9d399700 1 -- 172.21.15.119:0/64145108 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2640753713 0 0) 0x7f7a980ad370 con 0x7f7a980ccf90 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9e39b700 1 -- 172.21.15.119:0/64145108 >> v1:172.21.15.184:6789/0 conn(0x7f7a980ccf90 legacy=0x7f7a980b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9e39b700 1 -- 172.21.15.119:0/64145108 >> v1:172.21.15.119:6789/0 conn(0x7f7a980b1420 legacy=0x7f7a980c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.019+0000 7f7a9e39b700 1 -- 172.21.15.119:0/64145108 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980ad650 con 0x7f7a980cbf30 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.020+0000 7f7a9e39b700 1 --2- 172.21.15.119:0/64145108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980cbf30 0x7f7a980ae410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009910 tx=0x7f7a940091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8029a1de5caae9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.020+0000 7f7a9d399700 1 -- 172.21.15.119:0/64145108 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9400b680 con 0x7f7a980cbf30 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.020+0000 7f7a9d399700 1 -- 172.21.15.119:0/64145108 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9400b7e0 con 0x7f7a980cbf30 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.020+0000 7f7a9d399700 1 -- 172.21.15.119:0/64145108 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9400bba0 con 0x7f7a980cbf30 2023-10-08T02:12:22.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.021+0000 7f7aa05ff700 1 -- 172.21.15.119:0/64145108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980cbf30 msgr2=0x7f7a980ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.021+0000 7f7aa05ff700 1 --2- 172.21.15.119:0/64145108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980cbf30 0x7f7a980ae410 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009910 tx=0x7f7a940091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.021+0000 7f7aa05ff700 1 -- 172.21.15.119:0/64145108 shutdown_connections 2023-10-08T02:12:22.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.021+0000 7f7aa05ff700 1 --2- 172.21.15.119:0/64145108 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980cbf30 0x7f7a980ae410 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.021+0000 7f7aa05ff700 1 -- 172.21.15.119:0/64145108 >> 172.21.15.119:0/64145108 conn(0x7f7a9806a320 msgr2=0x7f7a980cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:22.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.021+0000 7f7aa05ff700 1 -- 172.21.15.119:0/64145108 shutdown_connections 2023-10-08T02:12:22.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.021+0000 7f7aa05ff700 1 -- 172.21.15.119:0/64145108 wait complete. 2023-10-08T02:12:22.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.022+0000 7f7aa05ff700 1 Processor -- start 2023-10-08T02:12:22.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.022+0000 7f7aa05ff700 1 -- start start 2023-10-08T02:12:22.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.023+0000 7f7aa05ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a980b1420 0x7f7a980abcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.023+0000 7f7aa05ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a980cbf30 0x7f7a98155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.023+0000 7f7aa05ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980ccf90 0x7f7a9806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.023+0000 7f7aa05ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980c5810 con 0x7f7a980cbf30 2023-10-08T02:12:22.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.023+0000 7f7aa05ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980ab340 con 0x7f7a980b1420 2023-10-08T02:12:22.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.023+0000 7f7aa05ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a980aad80 con 0x7f7a980ccf90 2023-10-08T02:12:22.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a980cbf30 0x7f7a98155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a980cbf30 0x7f7a98155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32876/0 (socket says 172.21.15.119:32876) 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 -- 172.21.15.119:0/948161250 learned_addr learned my addr 172.21.15.119:0/948161250 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9e39b700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a980b1420 0x7f7a980abcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9eb9c700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980ccf90 0x7f7a9806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 -- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980ccf90 msgr2=0x7f7a9806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980ccf90 0x7f7a9806d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 -- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a980b1420 msgr2=0x7f7a980abcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a980b1420 0x7f7a980abcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 -- 172.21.15.119:0/948161250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94009420 con 0x7f7a980cbf30 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9eb9c700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980ccf90 0x7f7a9806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:22.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a9db9a700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a980cbf30 0x7f7a98155970 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c530 tx=0x7f7a8800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:22.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.024+0000 7f7a8e7fc700 1 -- 172.21.15.119:0/948161250 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011590 con 0x7f7a980cbf30 2023-10-08T02:12:22.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.025+0000 7f7a8e7fc700 1 -- 172.21.15.119:0/948161250 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a88011710 con 0x7f7a980cbf30 2023-10-08T02:12:22.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.025+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a980ab060 con 0x7f7a980cbf30 2023-10-08T02:12:22.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.025+0000 7f7a8e7fc700 1 -- 172.21.15.119:0/948161250 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a88011ad0 con 0x7f7a980cbf30 2023-10-08T02:12:22.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.025+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a980aaa50 con 0x7f7a980cbf30 2023-10-08T02:12:22.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.026+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a98065870 con 0x7f7a980cbf30 2023-10-08T02:12:22.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.027+0000 7f7a8e7fc700 1 -- 172.21.15.119:0/948161250 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a8801b3b0 con 0x7f7a980cbf30 2023-10-08T02:12:22.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.027+0000 7f7a8e7fc700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7a7c066f30 0x7f7a7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.027+0000 7f7a8e7fc700 1 -- 172.21.15.119:0/948161250 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f7a88083930 con 0x7f7a980cbf30 2023-10-08T02:12:22.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.028+0000 7f7a9e39b700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7a7c066f30 0x7f7a7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.028+0000 7f7a9e39b700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7a7c066f30 0x7f7a7c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7a940093f0 tx=0x7f7a94017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:22.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.031+0000 7f7a8e7fc700 1 -- 172.21.15.119:0/948161250 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a88050c80 con 0x7f7a980cbf30 2023-10-08T02:12:22.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.229+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f7a9804c3c0 con 0x7f7a980cbf30 2023-10-08T02:12:22.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.229+0000 7f7a8e7fc700 1 -- 172.21.15.119:0/948161250 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v167) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7a88017070 con 0x7f7a980cbf30 2023-10-08T02:12:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:22.229 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T02:12:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T02:12:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T02:12:22.229 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:22.230 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T02:12:22.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.231+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7a7c066f30 msgr2=0x7f7a7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.231+0000 7f7aa05ff700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7a7c066f30 0x7f7a7c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7a940093f0 tx=0x7f7a94017000 comp rx=0 tx=0).stop 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.231+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a980cbf30 msgr2=0x7f7a98155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.231+0000 7f7aa05ff700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a980cbf30 0x7f7a98155970 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c530 tx=0x7f7a8800c840 comp rx=0 tx=0).stop 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.231+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 shutdown_connections 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.231+0000 7f7aa05ff700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7a7c066f30 0x7f7a7c0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.232+0000 7f7aa05ff700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7a980ccf90 0x7f7a9806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.232+0000 7f7aa05ff700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7a980b1420 0x7f7a980abcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.232+0000 7f7aa05ff700 1 --2- 172.21.15.119:0/948161250 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7a980cbf30 0x7f7a98155970 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.232+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 >> 172.21.15.119:0/948161250 conn(0x7f7a9806a320 msgr2=0x7f7a980cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:22.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.232+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 shutdown_connections 2023-10-08T02:12:22.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.232+0000 7f7aa05ff700 1 -- 172.21.15.119:0/948161250 wait complete. 2023-10-08T02:12:22.248 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T02:12:22.249 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T02:12:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.433+0000 7f76ae9ff700 1 Processor -- start 2023-10-08T02:12:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.433+0000 7f76ae9ff700 1 -- start start 2023-10-08T02:12:22.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.435+0000 7f76ae9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f76a80c8490 0x7f76a80c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.435+0000 7f76ae9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80ac030 con 0x7f76a8069e20 2023-10-08T02:12:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.435+0000 7f76ae9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a80ab820 con 0x7f76a8069a50 2023-10-08T02:12:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.435+0000 7f76ae9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a806dbb0 con 0x7f76a80c8490 2023-10-08T02:12:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.435+0000 7f769ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f76a80c8490 0x7f76a80c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.435+0000 7f76acf9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f76a8069e20 0x7f76a81569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47870/0 (socket says 172.21.15.119:47870) 2023-10-08T02:12:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.435+0000 7f76acf9c700 1 -- 172.21.15.119:0/1057753077 learned_addr learned my addr 172.21.15.119:0/1057753077 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.436+0000 7f769ffff700 1 -- 172.21.15.119:0/1057753077 >> v1:172.21.15.184:6789/0 conn(0x7f76a8069a50 legacy=0x7f76a815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.436+0000 7f769ffff700 1 -- 172.21.15.119:0/1057753077 >> v1:172.21.15.119:6789/0 conn(0x7f76a8069e20 legacy=0x7f76a81569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.436+0000 7f769ffff700 1 -- 172.21.15.119:0/1057753077 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80aba70 con 0x7f76a80c8490 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.436+0000 7f76a77fe700 1 -- 172.21.15.119:0/1057753077 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714729095 0 0) 0x7f76a0001490 con 0x7f76a8069e20 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.436+0000 7f769ffff700 1 --2- 172.21.15.119:0/1057753077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a80c8490 0x7f76a80c6ac0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f76900098e0 tx=0x7f76900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de07a2f9969e7e81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.436+0000 7f76a77fe700 1 -- 172.21.15.119:0/1057753077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690009e80 con 0x7f76a80c8490 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.436+0000 7f76a77fe700 1 -- 172.21.15.119:0/1057753077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769000b860 con 0x7f76a80c8490 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.436+0000 7f76a77fe700 1 -- 172.21.15.119:0/1057753077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769000bbe0 con 0x7f76a80c8490 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.437+0000 7f76ae9ff700 1 -- 172.21.15.119:0/1057753077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a80c8490 msgr2=0x7f76a80c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.437+0000 7f76ae9ff700 1 --2- 172.21.15.119:0/1057753077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a80c8490 0x7f76a80c6ac0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f76900098e0 tx=0x7f76900091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.437+0000 7f76ae9ff700 1 -- 172.21.15.119:0/1057753077 shutdown_connections 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.437+0000 7f76ae9ff700 1 --2- 172.21.15.119:0/1057753077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a80c8490 0x7f76a80c6ac0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.437+0000 7f76ae9ff700 1 -- 172.21.15.119:0/1057753077 >> 172.21.15.119:0/1057753077 conn(0x7f76a80c8f90 msgr2=0x7f76a80c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.437+0000 7f76ae9ff700 1 -- 172.21.15.119:0/1057753077 shutdown_connections 2023-10-08T02:12:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.437+0000 7f76ae9ff700 1 -- 172.21.15.119:0/1057753077 wait complete. 2023-10-08T02:12:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.438+0000 7f76ae9ff700 1 Processor -- start 2023-10-08T02:12:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.438+0000 7f76ae9ff700 1 -- start start 2023-10-08T02:12:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.439+0000 7f76ae9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a8069a50 0x7f76a80bfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.439+0000 7f76ae9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76a8069e20 0x7f76a80c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.439+0000 7f76ae9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a80c8490 0x7f76a80c05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.439+0000 7f76ae9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8158e60 con 0x7f76a8069a50 2023-10-08T02:12:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.439+0000 7f76ae9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8157fd0 con 0x7f76a80c8490 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.439+0000 7f76ae9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a8158250 con 0x7f76a8069e20 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.439+0000 7f76a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a8069a50 0x7f76a80bfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a8069a50 0x7f76a80bfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32904/0 (socket says 172.21.15.119:32904) 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a7fff700 1 -- 172.21.15.119:0/81952585 learned_addr learned my addr 172.21.15.119:0/81952585 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f769ffff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76a8069e20 0x7f76a80c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76acf9c700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a80c8490 0x7f76a80c05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a7fff700 1 -- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76a8069e20 msgr2=0x7f76a80c00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a7fff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76a8069e20 0x7f76a80c00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a7fff700 1 -- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a80c8490 msgr2=0x7f76a80c05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a7fff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a80c8490 0x7f76a80c05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a7fff700 1 -- 172.21.15.119:0/81952585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009420 con 0x7f76a8069a50 2023-10-08T02:12:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f769ffff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76a8069e20 0x7f76a80c00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76acf9c700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a80c8490 0x7f76a80c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a7fff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a8069a50 0x7f76a80bfb70 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f769800c510 tx=0x7f769800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a4ff9700 1 -- 172.21.15.119:0/81952585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698011590 con 0x7f76a8069a50 2023-10-08T02:12:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a4ff9700 1 -- 172.21.15.119:0/81952585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7698011710 con 0x7f76a8069a50 2023-10-08T02:12:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76a4ff9700 1 -- 172.21.15.119:0/81952585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7698011ad0 con 0x7f76a8069a50 2023-10-08T02:12:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.440+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a8158530 con 0x7f76a8069a50 2023-10-08T02:12:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.441+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a8158af0 con 0x7f76a8069a50 2023-10-08T02:12:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.441+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a80718b0 con 0x7f76a8069a50 2023-10-08T02:12:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.443+0000 7f76a4ff9700 1 -- 172.21.15.119:0/81952585 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f769801b3b0 con 0x7f76a8069a50 2023-10-08T02:12:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.444+0000 7f76a4ff9700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7688066f30 0x7f76880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.444+0000 7f76a4ff9700 1 -- 172.21.15.119:0/81952585 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5317+0+0 (secure 0 0 0) 0x7f7698083d40 con 0x7f76a8069a50 2023-10-08T02:12:22.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.444+0000 7f769ffff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7688066f30 0x7f76880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:22.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.444+0000 7f769ffff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7688066f30 0x7f76880693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f76900091e0 tx=0x7f7690017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.446+0000 7f76a4ff9700 1 -- 172.21.15.119:0/81952585 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7698051110 con 0x7f76a8069a50 2023-10-08T02:12:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:22.637+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f76a80c8d00 con 0x7f76a8069a50 2023-10-08T02:12:23.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.090+0000 7f76a4ff9700 1 -- 172.21.15.119:0/81952585 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v168) v1 ==== 138+0+0 (secure 0 0 0) 0x7f76980170b0 con 0x7f76a8069a50 2023-10-08T02:12:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7688066f30 msgr2=0x7f76880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7688066f30 0x7f76880693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f76900091e0 tx=0x7f7690017000 comp rx=0 tx=0).stop 2023-10-08T02:12:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a8069a50 msgr2=0x7f76a80bfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:23.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a8069a50 0x7f76a80bfb70 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f769800c510 tx=0x7f769800c8d0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 shutdown_connections 2023-10-08T02:12:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7688066f30 0x7f76880693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f76a80c8490 0x7f76a80c05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f76a8069a50 0x7f76a80bfb70 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 --2- 172.21.15.119:0/81952585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f76a8069e20 0x7f76a80c00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 >> 172.21.15.119:0/81952585 conn(0x7f76a80c8f90 msgr2=0x7f76a80cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.093+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 shutdown_connections 2023-10-08T02:12:23.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.094+0000 7f76ae9ff700 1 -- 172.21.15.119:0/81952585 wait complete. 2023-10-08T02:12:23.095 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T02:12:23.115 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T02:12:23.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.304+0000 7f4e7eb9d700 1 Processor -- start 2023-10-08T02:12:23.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.304+0000 7f4e7eb9d700 1 -- start start 2023-10-08T02:12:23.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.305+0000 7f4e7eb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e78159920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:23.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.305+0000 7f4e7eb9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4e78155860 0x7f4e78155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.305+0000 7f4e7eb9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e7806e510 con 0x7f4e780c7c10 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.305+0000 7f4e7eb9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e780c7220 con 0x7f4e78155860 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.305+0000 7f4e7eb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7806df20 con 0x7f4e780c7840 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.305+0000 7f4e7e39c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4e78155860 0x7f4e78155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.305+0000 7f4e7e39c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4e78155860 0x7f4e78155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32914/0 (socket says 172.21.15.119:32914) 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.305+0000 7f4e7e39c700 1 -- 172.21.15.119:0/843213188 learned_addr learned my addr 172.21.15.119:0/843213188 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7db9b700 1 --2- 172.21.15.119:0/843213188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e78159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7e39c700 1 -- 172.21.15.119:0/843213188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 msgr2=0x7f4e78159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7e39c700 1 --2- 172.21.15.119:0/843213188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e78159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7e39c700 1 -- 172.21.15.119:0/843213188 >> v1:172.21.15.184:6789/0 conn(0x7f4e780c7c10 legacy=0x7f4e78159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:23.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7e39c700 1 -- 172.21.15.119:0/843213188 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e78157040 con 0x7f4e78155860 2023-10-08T02:12:23.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7db9b700 1 --2- 172.21.15.119:0/843213188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e78159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:23.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7e39c700 1 --2- 172.21.15.119:0/843213188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e78155860 0x7f4e78155ce0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009f20 tx=0x7f4e70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f116d469df4f6e13 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:23.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7cb99700 1 -- 172.21.15.119:0/843213188 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e700092e0 con 0x7f4e78155860 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7cb99700 1 -- 172.21.15.119:0/843213188 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7000bb10 con 0x7f4e78155860 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.306+0000 7f4e7cb99700 1 -- 172.21.15.119:0/843213188 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70011400 con 0x7f4e78155860 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.307+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/843213188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e78155860 msgr2=0x7f4e78155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.307+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/843213188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e78155860 0x7f4e78155ce0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009f20 tx=0x7f4e70009840 comp rx=0 tx=0).stop 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.307+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/843213188 shutdown_connections 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.307+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/843213188 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e78155860 0x7f4e78155ce0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.307+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/843213188 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e78159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.307+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/843213188 >> 172.21.15.119:0/843213188 conn(0x7f4e78068e20 msgr2=0x7f4e780cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.307+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/843213188 shutdown_connections 2023-10-08T02:12:23.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.307+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/843213188 wait complete. 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.308+0000 7f4e7eb9d700 1 Processor -- start 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.309+0000 7f4e7eb9d700 1 -- start start 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.309+0000 7f4e7eb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e780cb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.309+0000 7f4e7eb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e780c7c10 0x7f4e78148be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7eb9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e78155860 0x7f4e780b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7eb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780c97e0 con 0x7f4e780c7c10 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7eb9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780cb9a0 con 0x7f4e78155860 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7eb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e780ade80 con 0x7f4e780c7840 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e780c7c10 0x7f4e78148be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e780c7c10 0x7f4e78148be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32916/0 (socket says 172.21.15.119:32916) 2023-10-08T02:12:23.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 -- 172.21.15.119:0/186037555 learned_addr learned my addr 172.21.15.119:0/186037555 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7db9b700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e780cb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7e39c700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e78155860 0x7f4e780b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 -- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 msgr2=0x7f4e780cb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e780cb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 -- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e78155860 msgr2=0x7f4e780b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e78155860 0x7f4e780b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 -- 172.21.15.119:0/186037555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e70009ae0 con 0x7f4e780c7c10 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7db9b700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e780cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7e39c700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e78155860 0x7f4e780b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.310+0000 7f4e7d39a700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e780c7c10 0x7f4e78148be0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c590 tx=0x7f4e6800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:23.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.311+0000 7f4e6dffb700 1 -- 172.21.15.119:0/186037555 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68011590 con 0x7f4e780c7c10 2023-10-08T02:12:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.311+0000 7f4e6dffb700 1 -- 172.21.15.119:0/186037555 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e680116f0 con 0x7f4e780c7c10 2023-10-08T02:12:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.311+0000 7f4e6dffb700 1 -- 172.21.15.119:0/186037555 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e68011ab0 con 0x7f4e780c7c10 2023-10-08T02:12:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.311+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e780ae160 con 0x7f4e780c7c10 2023-10-08T02:12:23.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.311+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e780c9470 con 0x7f4e780c7c10 2023-10-08T02:12:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.312+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7804c3c0 con 0x7f4e780c7c10 2023-10-08T02:12:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.313+0000 7f4e6dffb700 1 -- 172.21.15.119:0/186037555 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4e6801b3b0 con 0x7f4e780c7c10 2023-10-08T02:12:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.314+0000 7f4e6dffb700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4e5c066f30 0x7f4e5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.314+0000 7f4e6dffb700 1 -- 172.21.15.119:0/186037555 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f4e68083f70 con 0x7f4e780c7c10 2023-10-08T02:12:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.314+0000 7f4e7db9b700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4e5c066f30 0x7f4e5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:23.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.315+0000 7f4e7db9b700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4e5c066f30 0x7f4e5c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e74006f00 tx=0x7f4e74008040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:23.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.317+0000 7f4e6dffb700 1 -- 172.21.15.119:0/186037555 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4e680511d0 con 0x7f4e780c7c10 2023-10-08T02:12:23.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:23.508+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4e780c9920 con 0x7f4e780c7c10 2023-10-08T02:12:24.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.100+0000 7f4e6dffb700 1 -- 172.21.15.119:0/186037555 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v169) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4e680170d0 con 0x7f4e780c7c10 2023-10-08T02:12:24.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.111+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4e5c066f30 msgr2=0x7f4e5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:24.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.111+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4e5c066f30 0x7f4e5c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e74006f00 tx=0x7f4e74008040 comp rx=0 tx=0).stop 2023-10-08T02:12:24.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.111+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e780c7c10 msgr2=0x7f4e78148be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:24.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.111+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e780c7c10 0x7f4e78148be0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c590 tx=0x7f4e6800c950 comp rx=0 tx=0).stop 2023-10-08T02:12:24.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.111+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 shutdown_connections 2023-10-08T02:12:24.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.111+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4e5c066f30 0x7f4e5c0693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.111+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4e780c7840 0x7f4e780cb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.111+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4e78155860 0x7f4e780b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.112+0000 7f4e7eb9d700 1 --2- 172.21.15.119:0/186037555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e780c7c10 0x7f4e78148be0 secure :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800c590 tx=0x7f4e6800c950 comp rx=0 tx=0).stop 2023-10-08T02:12:24.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.112+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 >> 172.21.15.119:0/186037555 conn(0x7f4e78068e20 msgr2=0x7f4e780aa6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:24.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.115+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 shutdown_connections 2023-10-08T02:12:24.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.117+0000 7f4e7eb9d700 1 -- 172.21.15.119:0/186037555 wait complete. 2023-10-08T02:12:24.122 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T02:12:24.143 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T02:12:24.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.329+0000 7f13bb9ff700 1 Processor -- start 2023-10-08T02:12:24.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.329+0000 7f13bb9ff700 1 -- start start 2023-10-08T02:12:24.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13bb9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13b4155eb0 0x7f13b406e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13bb9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f13b40c70b0 0x7f13b4069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13bb9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 0x7f13b4159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13bb9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b406a130 con 0x7f13b4155eb0 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13bb9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b406eb90 con 0x7f13b40c70b0 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13bb9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b406ee70 con 0x7f13b4069cb0 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b979b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13b4155eb0 0x7f13b406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b979b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13b4155eb0 0x7f13b406e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32930/0 (socket says 172.21.15.119:32930) 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b979b700 1 -- 172.21.15.119:0/143629363 learned_addr learned my addr 172.21.15.119:0/143629363 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b9f9c700 1 --2- 172.21.15.119:0/143629363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 0x7f13b4159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b8f9a700 1 --2- 172.21.15.119:0/143629363 >> v2:172.21.15.184:3300/0 conn(0x7f13b40c70b0 0x7f13b4069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:24.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b979b700 1 -- 172.21.15.119:0/143629363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 msgr2=0x7f13b4159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:24.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b979b700 1 --2- 172.21.15.119:0/143629363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 0x7f13b4159a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b979b700 1 -- 172.21.15.119:0/143629363 >> v2:172.21.15.184:3300/0 conn(0x7f13b40c70b0 msgr2=0x7f13b4069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:24.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b979b700 1 --2- 172.21.15.119:0/143629363 >> v2:172.21.15.184:3300/0 conn(0x7f13b40c70b0 0x7f13b4069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.331+0000 7f13b979b700 1 -- 172.21.15.119:0/143629363 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b406f150 con 0x7f13b4155eb0 2023-10-08T02:12:24.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.332+0000 7f13b979b700 1 --2- 172.21.15.119:0/143629363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b4155eb0 0x7f13b406e240 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f13b4065130 tx=0x7f13b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=990078aaf7e4caea server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:24.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.332+0000 7f13abfff700 1 -- 172.21.15.119:0/143629363 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000b640 con 0x7f13b4155eb0 2023-10-08T02:12:24.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.332+0000 7f13abfff700 1 -- 172.21.15.119:0/143629363 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b000b7a0 con 0x7f13b4155eb0 2023-10-08T02:12:24.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.332+0000 7f13abfff700 1 -- 172.21.15.119:0/143629363 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b000bb60 con 0x7f13b4155eb0 2023-10-08T02:12:24.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.332+0000 7f13bb9ff700 1 -- 172.21.15.119:0/143629363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b4155eb0 msgr2=0x7f13b406e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:24.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.332+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/143629363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b4155eb0 0x7f13b406e240 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f13b4065130 tx=0x7f13b0009250 comp rx=0 tx=0).stop 2023-10-08T02:12:24.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.333+0000 7f13bb9ff700 1 -- 172.21.15.119:0/143629363 shutdown_connections 2023-10-08T02:12:24.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.333+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/143629363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 0x7f13b4159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.333+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/143629363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b4155eb0 0x7f13b406e240 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.333+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/143629363 >> v2:172.21.15.184:3300/0 conn(0x7f13b40c70b0 0x7f13b4069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.333+0000 7f13bb9ff700 1 -- 172.21.15.119:0/143629363 >> 172.21.15.119:0/143629363 conn(0x7f13b4068e10 msgr2=0x7f13b40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:24.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.333+0000 7f13bb9ff700 1 -- 172.21.15.119:0/143629363 shutdown_connections 2023-10-08T02:12:24.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.333+0000 7f13bb9ff700 1 -- 172.21.15.119:0/143629363 wait complete. 2023-10-08T02:12:24.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.334+0000 7f13bb9ff700 1 Processor -- start 2023-10-08T02:12:24.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.334+0000 7f13bb9ff700 1 -- start start 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13bb9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 0x7f13b40aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13bb9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b40c70b0 0x7f13b40b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13bb9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4155eb0 0x7f13b415a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13bb9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40c6430 con 0x7f13b40c70b0 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13bb9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40c5bf0 con 0x7f13b4155eb0 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13bb9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40c4170 con 0x7f13b4069cb0 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13b8f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b40c70b0 0x7f13b40b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13b8f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b40c70b0 0x7f13b40b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32940/0 (socket says 172.21.15.119:32940) 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13b8f9a700 1 -- 172.21.15.119:0/3425390480 learned_addr learned my addr 172.21.15.119:0/3425390480 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.335+0000 7f13b979b700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 0x7f13b40aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:24.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13b9f9c700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4155eb0 0x7f13b415a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:24.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13b8f9a700 1 -- 172.21.15.119:0/3425390480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 msgr2=0x7f13b40aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:24.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13b8f9a700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 0x7f13b40aad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13b8f9a700 1 -- 172.21.15.119:0/3425390480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4155eb0 msgr2=0x7f13b415a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:24.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13b8f9a700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4155eb0 0x7f13b415a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:24.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13b8f9a700 1 -- 172.21.15.119:0/3425390480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b00094f0 con 0x7f13b40c70b0 2023-10-08T02:12:24.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13b8f9a700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b40c70b0 0x7f13b40b0cb0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f13a400ec80 tx=0x7f13a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:24.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13a97fa700 1 -- 172.21.15.119:0/3425390480 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400c850 con 0x7f13b40c70b0 2023-10-08T02:12:24.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13a97fa700 1 -- 172.21.15.119:0/3425390480 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a400c9b0 con 0x7f13b40c70b0 2023-10-08T02:12:24.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13a97fa700 1 -- 172.21.15.119:0/3425390480 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400cd50 con 0x7f13b40c70b0 2023-10-08T02:12:24.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.336+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b40c5e70 con 0x7f13b40c70b0 2023-10-08T02:12:24.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.337+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b40c4410 con 0x7f13b40c70b0 2023-10-08T02:12:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.338+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b40c3fb0 con 0x7f13b40c70b0 2023-10-08T02:12:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.339+0000 7f13a97fa700 1 -- 172.21.15.119:0/3425390480 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f13a4013680 con 0x7f13b40c70b0 2023-10-08T02:12:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.340+0000 7f13a97fa700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1398066f30 0x7f13980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.340+0000 7f13a97fa700 1 -- 172.21.15.119:0/3425390480 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6069+0+0 (secure 0 0 0) 0x7f13a40850c0 con 0x7f13b40c70b0 2023-10-08T02:12:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.340+0000 7f13b979b700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1398066f30 0x7f13980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.341+0000 7f13b979b700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1398066f30 0x7f13980693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f13b4065130 tx=0x7f13b0017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:24.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.343+0000 7f13a97fa700 1 -- 172.21.15.119:0/3425390480 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f13a4052240 con 0x7f13b40c70b0 2023-10-08T02:12:24.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:24.578+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f13b404c3c0 con 0x7f13b40c70b0 2023-10-08T02:12:25.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.115+0000 7f13a97fa700 1 -- 172.21.15.119:0/3425390480 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7f13a4055860 con 0x7f13b40c70b0 2023-10-08T02:12:25.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.129+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1398066f30 msgr2=0x7f13980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1398066f30 0x7f13980693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f13b4065130 tx=0x7f13b0017000 comp rx=0 tx=0).stop 2023-10-08T02:12:25.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b40c70b0 msgr2=0x7f13b40b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b40c70b0 0x7f13b40b0cb0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f13a400ec80 tx=0x7f13a40091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 shutdown_connections 2023-10-08T02:12:25.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f1398066f30 0x7f13980693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f13b4069cb0 0x7f13b40aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f13b4155eb0 0x7f13b415a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 --2- 172.21.15.119:0/3425390480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13b40c70b0 0x7f13b40b0cb0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.130+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 >> 172.21.15.119:0/3425390480 conn(0x7f13b4068e10 msgr2=0x7f13b40cb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:25.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.131+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 shutdown_connections 2023-10-08T02:12:25.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.131+0000 7f13bb9ff700 1 -- 172.21.15.119:0/3425390480 wait complete. 2023-10-08T02:12:25.136 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 20 and data pool 21 2023-10-08T02:12:25.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:12:25.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.335+0000 7f86c4bff700 1 Processor -- start 2023-10-08T02:12:25.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.335+0000 7f86c4bff700 1 -- start start 2023-10-08T02:12:25.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86c4bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86c00b1420 0x7f86c00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86c4bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c00cc300 con 0x7f86c00cbf30 2023-10-08T02:12:25.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86c4bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c00ad370 con 0x7f86c00ccf90 2023-10-08T02:12:25.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86c4bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c0156bf0 con 0x7f86c00b1420 2023-10-08T02:12:25.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86bdd9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86c00b1420 0x7f86c00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:25.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86be59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f86c00cbf30 0x7f86c00b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59488/0 (socket says 172.21.15.119:59488) 2023-10-08T02:12:25.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86be59c700 1 -- 172.21.15.119:0/792072916 learned_addr learned my addr 172.21.15.119:0/792072916 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:25.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86bd59a700 1 -- 172.21.15.119:0/792072916 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2575937898 0 0) 0x7f86c00cc300 con 0x7f86c00cbf30 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.337+0000 7f86bd59a700 1 -- 172.21.15.119:0/792072916 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86a8003570 con 0x7f86c00cbf30 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.338+0000 7f86bdd9b700 1 -- 172.21.15.119:0/792072916 >> v1:172.21.15.184:6789/0 conn(0x7f86c00ccf90 legacy=0x7f86c00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.338+0000 7f86bd59a700 1 -- 172.21.15.119:0/792072916 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025025391 0 0) 0x7f86c00ad370 con 0x7f86c00ccf90 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.338+0000 7f86bdd9b700 1 -- 172.21.15.119:0/792072916 >> v1:172.21.15.119:6789/0 conn(0x7f86c00cbf30 legacy=0x7f86c00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.338+0000 7f86bdd9b700 1 -- 172.21.15.119:0/792072916 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c00ad650 con 0x7f86c00b1420 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.338+0000 7f86bdd9b700 1 --2- 172.21.15.119:0/792072916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c00ae410 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f86b40098e0 tx=0x7f86b40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b3328a64abde95 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.338+0000 7f86bd59a700 1 -- 172.21.15.119:0/792072916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b400b640 con 0x7f86c00b1420 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.338+0000 7f86bd59a700 1 -- 172.21.15.119:0/792072916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b400b7a0 con 0x7f86c00b1420 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.338+0000 7f86bd59a700 1 -- 172.21.15.119:0/792072916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b400bb40 con 0x7f86c00b1420 2023-10-08T02:12:25.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.339+0000 7f86c4bff700 1 -- 172.21.15.119:0/792072916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 msgr2=0x7f86c00ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.339+0000 7f86c4bff700 1 --2- 172.21.15.119:0/792072916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c00ae410 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f86b40098e0 tx=0x7f86b40091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.339+0000 7f86c4bff700 1 -- 172.21.15.119:0/792072916 shutdown_connections 2023-10-08T02:12:25.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.339+0000 7f86c4bff700 1 --2- 172.21.15.119:0/792072916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c00ae410 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.339+0000 7f86c4bff700 1 -- 172.21.15.119:0/792072916 >> 172.21.15.119:0/792072916 conn(0x7f86c006a320 msgr2=0x7f86c00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:25.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.339+0000 7f86c4bff700 1 -- 172.21.15.119:0/792072916 shutdown_connections 2023-10-08T02:12:25.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.339+0000 7f86c4bff700 1 -- 172.21.15.119:0/792072916 wait complete. 2023-10-08T02:12:25.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.340+0000 7f86c4bff700 1 Processor -- start 2023-10-08T02:12:25.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.340+0000 7f86c4bff700 1 -- start start 2023-10-08T02:12:25.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.341+0000 7f86c4bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c0155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.341+0000 7f86c4bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c00cbf30 0x7f86c00aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.341+0000 7f86c4bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c00ccf90 0x7f86c00aa970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.341+0000 7f86c4bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00a6740 con 0x7f86c00b1420 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.341+0000 7f86c4bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00a5b30 con 0x7f86c00cbf30 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.341+0000 7f86c4bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c00a5e10 con 0x7f86c00ccf90 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.341+0000 7f86be59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c0155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86be59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c0155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44000/0 (socket says 172.21.15.119:44000) 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86be59c700 1 -- 172.21.15.119:0/1207013925 learned_addr learned my addr 172.21.15.119:0/1207013925 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86be59c700 1 -- 172.21.15.119:0/1207013925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c00ccf90 msgr2=0x7f86c00aa970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86be59c700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c00ccf90 0x7f86c00aa970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86be59c700 1 -- 172.21.15.119:0/1207013925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c00cbf30 msgr2=0x7f86c00aa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86be59c700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c00cbf30 0x7f86c00aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86be59c700 1 -- 172.21.15.119:0/1207013925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4009420 con 0x7f86c00b1420 2023-10-08T02:12:25.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86be59c700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c0155870 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f86b000cd00 tx=0x7f86b0001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:25.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86ae7fc700 1 -- 172.21.15.119:0/1207013925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b000ab10 con 0x7f86c00b1420 2023-10-08T02:12:25.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86ae7fc700 1 -- 172.21.15.119:0/1207013925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b000ac70 con 0x7f86c00b1420 2023-10-08T02:12:25.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c00a60f0 con 0x7f86c00b1420 2023-10-08T02:12:25.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86ae7fc700 1 -- 172.21.15.119:0/1207013925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b00070f0 con 0x7f86c00b1420 2023-10-08T02:12:25.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.342+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c0155db0 con 0x7f86c00b1420 2023-10-08T02:12:25.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.343+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c004c3c0 con 0x7f86c00b1420 2023-10-08T02:12:25.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.344+0000 7f86ae7fc700 1 -- 172.21.15.119:0/1207013925 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86b00074a0 con 0x7f86c00b1420 2023-10-08T02:12:25.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.345+0000 7f86ae7fc700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86a0066f30 0x7f86a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.345+0000 7f86ae7fc700 1 -- 172.21.15.119:0/1207013925 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f86b0084ca0 con 0x7f86c00b1420 2023-10-08T02:12:25.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.346+0000 7f86bdd9b700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86a0066f30 0x7f86a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:25.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.346+0000 7f86bdd9b700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86a0066f30 0x7f86a00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86c00b1820 tx=0x7f86b4017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:25.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.349+0000 7f86ae7fc700 1 -- 172.21.15.119:0/1207013925 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86b0051d20 con 0x7f86c00b1420 2023-10-08T02:12:25.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.540+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86c006f8a0 con 0x7f86c00b1420 2023-10-08T02:12:25.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.541+0000 7f86ae7fc700 1 -- 172.21.15.119:0/1207013925 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16138 (secure 0 0 0) 0x7f86b0055340 con 0x7f86c00b1420 2023-10-08T02:12:25.541 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:25.541 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":170,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:12:24.579414+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:12:22.679007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T02:12:23.548789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:12:25.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86a0066f30 msgr2=0x7f86a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86a0066f30 0x7f86a00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86c00b1820 tx=0x7f86b4017000 comp rx=0 tx=0).stop 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 msgr2=0x7f86c0155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c0155870 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f86b000cd00 tx=0x7f86b0001b00 comp rx=0 tx=0).stop 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 shutdown_connections 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f86a0066f30 0x7f86a00693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86c00ccf90 0x7f86c00aa970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86c00b1420 0x7f86c0155870 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 --2- 172.21.15.119:0/1207013925 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86c00cbf30 0x7f86c00aa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 >> 172.21.15.119:0/1207013925 conn(0x7f86c006a320 msgr2=0x7f86c015a480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 shutdown_connections 2023-10-08T02:12:25.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.544+0000 7f86c4bff700 1 -- 172.21.15.119:0/1207013925 wait complete. 2023-10-08T02:12:25.561 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:12:25.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.742+0000 7fd440fff700 1 Processor -- start 2023-10-08T02:12:25.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.742+0000 7fd440fff700 1 -- start start 2023-10-08T02:12:25.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd440fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd43c0b1850 0x7fd43c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd440fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c0cb300 con 0x7fd43c1579f0 2023-10-08T02:12:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd440fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43c0cb5e0 con 0x7fd43c0ae680 2023-10-08T02:12:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd440fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43c06fc30 con 0x7fd43c0b1850 2023-10-08T02:12:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd43ad9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd43c1579f0 0x7fd43c0abf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59504/0 (socket says 172.21.15.119:59504) 2023-10-08T02:12:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd43ad9d700 1 -- 172.21.15.119:0/1583075897 learned_addr learned my addr 172.21.15.119:0/1583075897 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd43a59c700 1 --2- 172.21.15.119:0/1583075897 >> v2:172.21.15.184:3300/0 conn(0x7fd43c0b1850 0x7fd43c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd439d9b700 1 -- 172.21.15.119:0/1583075897 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012380821 0 0) 0x7fd43c0cb300 con 0x7fd43c1579f0 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.744+0000 7fd439d9b700 1 -- 172.21.15.119:0/1583075897 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd428003570 con 0x7fd43c1579f0 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.745+0000 7fd43a59c700 1 -- 172.21.15.119:0/1583075897 >> v1:172.21.15.184:6789/0 conn(0x7fd43c0ae680 legacy=0x7fd43c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.745+0000 7fd43a59c700 1 -- 172.21.15.119:0/1583075897 >> v1:172.21.15.119:6789/0 conn(0x7fd43c1579f0 legacy=0x7fd43c0abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.745+0000 7fd43a59c700 1 -- 172.21.15.119:0/1583075897 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43c0cb8c0 con 0x7fd43c0b1850 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.745+0000 7fd43a59c700 1 --2- 172.21.15.119:0/1583075897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c0b1850 0x7fd43c0b1c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0098e0 tx=0x7fd42c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8251427d40b67b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.745+0000 7fd439d9b700 1 -- 172.21.15.119:0/1583075897 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00b640 con 0x7fd43c0b1850 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.745+0000 7fd439d9b700 1 -- 172.21.15.119:0/1583075897 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c00b7a0 con 0x7fd43c0b1850 2023-10-08T02:12:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.745+0000 7fd439d9b700 1 -- 172.21.15.119:0/1583075897 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c00bb40 con 0x7fd43c0b1850 2023-10-08T02:12:25.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.746+0000 7fd440fff700 1 -- 172.21.15.119:0/1583075897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c0b1850 msgr2=0x7fd43c0b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.746+0000 7fd440fff700 1 --2- 172.21.15.119:0/1583075897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c0b1850 0x7fd43c0b1c20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0098e0 tx=0x7fd42c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.746+0000 7fd440fff700 1 -- 172.21.15.119:0/1583075897 shutdown_connections 2023-10-08T02:12:25.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.746+0000 7fd440fff700 1 --2- 172.21.15.119:0/1583075897 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c0b1850 0x7fd43c0b1c20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.746+0000 7fd440fff700 1 -- 172.21.15.119:0/1583075897 >> 172.21.15.119:0/1583075897 conn(0x7fd43c06a320 msgr2=0x7fd43c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:25.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.746+0000 7fd440fff700 1 -- 172.21.15.119:0/1583075897 shutdown_connections 2023-10-08T02:12:25.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.746+0000 7fd440fff700 1 -- 172.21.15.119:0/1583075897 wait complete. 2023-10-08T02:12:25.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.747+0000 7fd440fff700 1 Processor -- start 2023-10-08T02:12:25.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.748+0000 7fd440fff700 1 -- start start 2023-10-08T02:12:25.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.748+0000 7fd440fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd43c0ae680 0x7fd43c15c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.748+0000 7fd440fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd43c0b1850 0x7fd43c148110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.748+0000 7fd440fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c1579f0 0x7fd43c15b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.748+0000 7fd440fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c0cc260 con 0x7fd43c0b1850 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.748+0000 7fd440fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c16ad20 con 0x7fd43c1579f0 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.748+0000 7fd440fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd43c15bd60 con 0x7fd43c0ae680 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd43c0b1850 0x7fd43c148110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd43c0b1850 0x7fd43c148110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44008/0 (socket says 172.21.15.119:44008) 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 -- 172.21.15.119:0/1456518549 learned_addr learned my addr 172.21.15.119:0/1456518549 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd43a59c700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd43c0ae680 0x7fd43c15c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd43ad9d700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c1579f0 0x7fd43c15b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 -- 172.21.15.119:0/1456518549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd43c0ae680 msgr2=0x7fd43c15c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd43c0ae680 0x7fd43c15c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 -- 172.21.15.119:0/1456518549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c1579f0 msgr2=0x7fd43c15b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c1579f0 0x7fd43c15b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 -- 172.21.15.119:0/1456518549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42c009420 con 0x7fd43c0b1850 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.749+0000 7fd433fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd43c0b1850 0x7fd43c148110 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fd42400c790 tx=0x7fd424007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.750+0000 7fd4327fc700 1 -- 172.21.15.119:0/1456518549 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424012630 con 0x7fd43c0b1850 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.750+0000 7fd4327fc700 1 -- 172.21.15.119:0/1456518549 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4240127b0 con 0x7fd43c0b1850 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.750+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd43c157ff0 con 0x7fd43c0b1850 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.750+0000 7fd4327fc700 1 -- 172.21.15.119:0/1456518549 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424012b70 con 0x7fd43c0b1850 2023-10-08T02:12:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.750+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43c16a760 con 0x7fd43c0b1850 2023-10-08T02:12:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.751+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43c158180 con 0x7fd43c0b1850 2023-10-08T02:12:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.752+0000 7fd4327fc700 1 -- 172.21.15.119:0/1456518549 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd42401b400 con 0x7fd43c0b1850 2023-10-08T02:12:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.752+0000 7fd4327fc700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd41c066f30 0x7fd41c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.753+0000 7fd4327fc700 1 -- 172.21.15.119:0/1456518549 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd424084290 con 0x7fd43c0b1850 2023-10-08T02:12:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.753+0000 7fd43a59c700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd41c066f30 0x7fd41c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.753+0000 7fd43a59c700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd41c066f30 0x7fd41c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0093f0 tx=0x7fd42c017000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:25.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.757+0000 7fd4327fc700 1 -- 172.21.15.119:0/1456518549 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd424051330 con 0x7fd43c0b1850 2023-10-08T02:12:25.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.949+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd43c04c3c0 con 0x7fd43c0b1850 2023-10-08T02:12:25.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.950+0000 7fd4327fc700 1 -- 172.21.15.119:0/1456518549 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16138 (secure 0 0 0) 0x7fd424054950 con 0x7fd43c0b1850 2023-10-08T02:12:25.950 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:25.950 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":170,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:12:24.579414+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:12:22.679007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T02:12:23.548789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd41c066f30 msgr2=0x7fd41c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd41c066f30 0x7fd41c0693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd42c0093f0 tx=0x7fd42c017000 comp rx=0 tx=0).stop 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd43c0b1850 msgr2=0x7fd43c148110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd43c0b1850 0x7fd43c148110 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fd42400c790 tx=0x7fd424007f60 comp rx=0 tx=0).stop 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 shutdown_connections 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd41c066f30 0x7fd41c0693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd43c0ae680 0x7fd43c15c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd43c1579f0 0x7fd43c15b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 --2- 172.21.15.119:0/1456518549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd43c0b1850 0x7fd43c148110 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 >> 172.21.15.119:0/1456518549 conn(0x7fd43c06a320 msgr2=0x7fd43c068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 shutdown_connections 2023-10-08T02:12:25.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:25.953+0000 7fd440fff700 1 -- 172.21.15.119:0/1456518549 wait complete. 2023-10-08T02:12:25.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T02:12:26.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.159+0000 7ff0d19ff700 1 Processor -- start 2023-10-08T02:12:26.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.159+0000 7ff0d19ff700 1 -- start start 2023-10-08T02:12:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.160+0000 7ff0d19ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0ccee0 0x7ff0cc1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.160+0000 7ff0d19ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0cc0c87c0 0x7ff0cc156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:26.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.160+0000 7ff0d19ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0cc15b0d0 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.160+0000 7ff0d19ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc156c50 con 0x7ff0cc0c87c0 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.160+0000 7ff0d19ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc15a4c0 con 0x7ff0cc0ccee0 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.160+0000 7ff0cb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0cc0c87c0 0x7ff0cc156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0cb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0cc0c87c0 0x7ff0cc156710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44014/0 (socket says 172.21.15.119:44014) 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0cb7fe700 1 -- 172.21.15.119:0/1313724081 learned_addr learned my addr 172.21.15.119:0/1313724081 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0c3fff700 1 --2- 172.21.15.119:0/1313724081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0ccee0 0x7ff0cc1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0cb7fe700 1 -- 172.21.15.119:0/1313724081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0ccee0 msgr2=0x7ff0cc1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0cb7fe700 1 --2- 172.21.15.119:0/1313724081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0ccee0 0x7ff0cc1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0cb7fe700 1 -- 172.21.15.119:0/1313724081 >> v1:172.21.15.184:6789/0 conn(0x7ff0cc0c83f0 legacy=0x7ff0cc0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:26.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0cb7fe700 1 -- 172.21.15.119:0/1313724081 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc15a7a0 con 0x7ff0cc0c87c0 2023-10-08T02:12:26.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0cb7fe700 1 --2- 172.21.15.119:0/1313724081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c87c0 0x7ff0cc156710 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009f20 tx=0x7ff0c4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2f10d3193a8d0f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:26.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0ca7fc700 1 -- 172.21.15.119:0/1313724081 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400e070 con 0x7ff0cc0c87c0 2023-10-08T02:12:26.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0ca7fc700 1 -- 172.21.15.119:0/1313724081 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c400ba80 con 0x7ff0cc0c87c0 2023-10-08T02:12:26.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.161+0000 7ff0ca7fc700 1 -- 172.21.15.119:0/1313724081 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40092e0 con 0x7ff0cc0c87c0 2023-10-08T02:12:26.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.162+0000 7ff0d19ff700 1 -- 172.21.15.119:0/1313724081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c87c0 msgr2=0x7ff0cc156710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:26.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.162+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/1313724081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c87c0 0x7ff0cc156710 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009f20 tx=0x7ff0c4009840 comp rx=0 tx=0).stop 2023-10-08T02:12:26.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.162+0000 7ff0d19ff700 1 -- 172.21.15.119:0/1313724081 shutdown_connections 2023-10-08T02:12:26.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.162+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/1313724081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c87c0 0x7ff0cc156710 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:26.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.162+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/1313724081 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0ccee0 0x7ff0cc1561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:26.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.162+0000 7ff0d19ff700 1 -- 172.21.15.119:0/1313724081 >> 172.21.15.119:0/1313724081 conn(0x7ff0cc068e10 msgr2=0x7ff0cc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:26.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.162+0000 7ff0d19ff700 1 -- 172.21.15.119:0/1313724081 shutdown_connections 2023-10-08T02:12:26.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.162+0000 7ff0d19ff700 1 -- 172.21.15.119:0/1313724081 wait complete. 2023-10-08T02:12:26.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.164+0000 7ff0d19ff700 1 Processor -- start 2023-10-08T02:12:26.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.164+0000 7ff0d19ff700 1 -- start start 2023-10-08T02:12:26.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0d19ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c83f0 0x7ff0cc0b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:26.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0d19ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0c87c0 0x7ff0cc0caa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:26.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0d19ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0cc0ccee0 0x7ff0cc16a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:26.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0d19ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c35e0 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0d19ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0caf60 con 0x7ff0cc0ccee0 2023-10-08T02:12:26.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0d19ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0cb230 con 0x7ff0cc0c87c0 2023-10-08T02:12:26.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0caffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c83f0 0x7ff0cc0b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0caffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c83f0 0x7ff0cc0b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44024/0 (socket says 172.21.15.119:44024) 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0caffd700 1 -- 172.21.15.119:0/2122075247 learned_addr learned my addr 172.21.15.119:0/2122075247 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0cb7fe700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0cc0ccee0 0x7ff0cc16a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0c3fff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0c87c0 0x7ff0cc0caa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0caffd700 1 -- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0c87c0 msgr2=0x7ff0cc0caa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0caffd700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0c87c0 0x7ff0cc0caa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0caffd700 1 -- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0cc0ccee0 msgr2=0x7ff0cc16a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0caffd700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0cc0ccee0 0x7ff0cc16a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0caffd700 1 -- 172.21.15.119:0/2122075247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4009ae0 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0cb7fe700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0cc0ccee0 0x7ff0cc16a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:26.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.165+0000 7ff0c3fff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0c87c0 0x7ff0cc0caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:26.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.166+0000 7ff0caffd700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c83f0 0x7ff0cc0b0130 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00c590 tx=0x7ff0bc00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:26.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.166+0000 7ff0c2ffd700 1 -- 172.21.15.119:0/2122075247 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011590 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.166+0000 7ff0c2ffd700 1 -- 172.21.15.119:0/2122075247 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0116f0 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.166+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc0cb510 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.166+0000 7ff0c2ffd700 1 -- 172.21.15.119:0/2122075247 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc011ab0 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.166+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc0c3830 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.167+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc04c3c0 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.168+0000 7ff0c2ffd700 1 -- 172.21.15.119:0/2122075247 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff0bc01b3b0 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.169+0000 7ff0c2ffd700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0ac066f30 0x7ff0ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:26.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.169+0000 7ff0c2ffd700 1 -- 172.21.15.119:0/2122075247 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7ff0bc083cb0 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.169+0000 7ff0c3fff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0ac066f30 0x7ff0ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:26.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.169+0000 7ff0c3fff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0ac066f30 0x7ff0ac0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc167c00 tx=0x7ff0b4008040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:26.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.173+0000 7ff0c2ffd700 1 -- 172.21.15.119:0/2122075247 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0bc050d50 con 0x7ff0cc0c83f0 2023-10-08T02:12:26.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:26.405+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff0cc15aa80 con 0x7ff0cc0c83f0 2023-10-08T02:12:27.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.135+0000 7ff0c2ffd700 1 -- 172.21.15.119:0/2122075247 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff0bc017070 con 0x7ff0cc0c83f0 2023-10-08T02:12:27.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0ac066f30 msgr2=0x7ff0ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:27.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0ac066f30 0x7ff0ac0693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc167c00 tx=0x7ff0b4008040 comp rx=0 tx=0).stop 2023-10-08T02:12:27.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c83f0 msgr2=0x7ff0cc0b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:27.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c83f0 0x7ff0cc0b0130 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00c590 tx=0x7ff0bc00c950 comp rx=0 tx=0).stop 2023-10-08T02:12:27.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 shutdown_connections 2023-10-08T02:12:27.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0ac066f30 0x7ff0ac0693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:27.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0cc0ccee0 0x7ff0cc16a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:27.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0cc0c83f0 0x7ff0cc0b0130 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:27.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.137+0000 7ff0d19ff700 1 --2- 172.21.15.119:0/2122075247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0cc0c87c0 0x7ff0cc0caa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:27.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.138+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 >> 172.21.15.119:0/2122075247 conn(0x7ff0cc068e10 msgr2=0x7ff0cc0cbfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:27.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.138+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 shutdown_connections 2023-10-08T02:12:27.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.138+0000 7ff0d19ff700 1 -- 172.21.15.119:0/2122075247 wait complete. 2023-10-08T02:12:27.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T02:12:27.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.346+0000 7f895c9ff700 1 Processor -- start 2023-10-08T02:12:27.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.346+0000 7f895c9ff700 1 -- start start 2023-10-08T02:12:27.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.347+0000 7f895c9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89580caa90 0x7f89580c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:27.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.347+0000 7f895c9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895815d7e0 con 0x7f895806bcd0 2023-10-08T02:12:27.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.347+0000 7f895c9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f895815dac0 con 0x7f895806c0a0 2023-10-08T02:12:27.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.347+0000 7f895c9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89580caef0 con 0x7f89580caa90 2023-10-08T02:12:27.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f895659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f895806bcd0 0x7f895815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59514/0 (socket says 172.21.15.119:59514) 2023-10-08T02:12:27.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f895659c700 1 -- 172.21.15.119:0/2032448834 learned_addr learned my addr 172.21.15.119:0/2032448834 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:27.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f8956d9d700 1 --2- 172.21.15.119:0/2032448834 >> v2:172.21.15.119:3300/0 conn(0x7f89580caa90 0x7f89580c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:27.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f895559a700 1 -- 172.21.15.119:0/2032448834 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247434130 0 0) 0x7f895815d7e0 con 0x7f895806bcd0 2023-10-08T02:12:27.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f895559a700 1 -- 172.21.15.119:0/2032448834 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8940003570 con 0x7f895806bcd0 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f8956d9d700 1 -- 172.21.15.119:0/2032448834 >> v1:172.21.15.184:6789/0 conn(0x7f895806c0a0 legacy=0x7f895815df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f8956d9d700 1 -- 172.21.15.119:0/2032448834 >> v1:172.21.15.119:6789/0 conn(0x7f895806bcd0 legacy=0x7f895815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f8956d9d700 1 -- 172.21.15.119:0/2032448834 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895815aa30 con 0x7f89580caa90 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.348+0000 7f8956d9d700 1 --2- 172.21.15.119:0/2032448834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580c8c40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f8950009e60 tx=0x7f8950009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7351b42c24d31c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895559a700 1 -- 172.21.15.119:0/2032448834 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000e070 con 0x7f89580caa90 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895559a700 1 -- 172.21.15.119:0/2032448834 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895000bb60 con 0x7f89580caa90 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895559a700 1 -- 172.21.15.119:0/2032448834 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950012430 con 0x7f89580caa90 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895c9ff700 1 -- 172.21.15.119:0/2032448834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 msgr2=0x7f89580c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895c9ff700 1 --2- 172.21.15.119:0/2032448834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580c8c40 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f8950009e60 tx=0x7f8950009780 comp rx=0 tx=0).stop 2023-10-08T02:12:27.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895c9ff700 1 -- 172.21.15.119:0/2032448834 shutdown_connections 2023-10-08T02:12:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895c9ff700 1 --2- 172.21.15.119:0/2032448834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580c8c40 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895c9ff700 1 -- 172.21.15.119:0/2032448834 >> 172.21.15.119:0/2032448834 conn(0x7f89580ce600 msgr2=0x7f89580cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895c9ff700 1 -- 172.21.15.119:0/2032448834 shutdown_connections 2023-10-08T02:12:27.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.349+0000 7f895c9ff700 1 -- 172.21.15.119:0/2032448834 wait complete. 2023-10-08T02:12:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.351+0000 7f895c9ff700 1 Processor -- start 2023-10-08T02:12:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.351+0000 7f895c9ff700 1 -- start start 2023-10-08T02:12:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f895c9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f895806bcd0 0x7f89580c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f895c9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f895806c0a0 0x7f89580affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f895c9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580b2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f895c9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580b13a0 con 0x7f89580caa90 2023-10-08T02:12:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f895c9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89580b15b0 con 0x7f895806c0a0 2023-10-08T02:12:27.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f895c9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89580b1890 con 0x7f895806bcd0 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580b2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8956d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580b2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44036/0 (socket says 172.21.15.119:44036) 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8956d9d700 1 -- 172.21.15.119:0/3849737323 learned_addr learned my addr 172.21.15.119:0/3849737323 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f895659c700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f895806bcd0 0x7f89580c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8955d9b700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f895806c0a0 0x7f89580affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8956d9d700 1 -- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f895806bcd0 msgr2=0x7f89580c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8956d9d700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f895806bcd0 0x7f89580c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8956d9d700 1 -- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f895806c0a0 msgr2=0x7f89580affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8956d9d700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f895806c0a0 0x7f89580affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8956d9d700 1 -- 172.21.15.119:0/3849737323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009a20 con 0x7f89580caa90 2023-10-08T02:12:27.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f8955d9b700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f895806c0a0 0x7f89580affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.352+0000 7f895659c700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f895806bcd0 0x7f89580c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.353+0000 7f8956d9d700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580b2c80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8958069220 tx=0x7f8950019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.353+0000 7f89467fc700 1 -- 172.21.15.119:0/3849737323 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950012e80 con 0x7f89580caa90 2023-10-08T02:12:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.353+0000 7f89467fc700 1 -- 172.21.15.119:0/3849737323 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89500032b0 con 0x7f89580caa90 2023-10-08T02:12:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.353+0000 7f89467fc700 1 -- 172.21.15.119:0/3849737323 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895001b850 con 0x7f89580caa90 2023-10-08T02:12:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.353+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89580b1b70 con 0x7f89580caa90 2023-10-08T02:12:27.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.353+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89580b2130 con 0x7f89580caa90 2023-10-08T02:12:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.354+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895804c3c0 con 0x7f89580caa90 2023-10-08T02:12:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.355+0000 7f89467fc700 1 -- 172.21.15.119:0/3849737323 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f895001bc40 con 0x7f89580caa90 2023-10-08T02:12:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.356+0000 7f89467fc700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8938066f30 0x7f89380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.356+0000 7f89467fc700 1 -- 172.21.15.119:0/3849737323 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f89500867c0 con 0x7f89580caa90 2023-10-08T02:12:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.356+0000 7f895659c700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8938066f30 0x7f89380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:27.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.357+0000 7f895659c700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8938066f30 0x7f89380693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8948007fd0 tx=0x7f8948009040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:27.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.360+0000 7f89467fc700 1 -- 172.21.15.119:0/3849737323 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89500537c0 con 0x7f89580caa90 2023-10-08T02:12:27.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:27.593+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f895815ad10 con 0x7f89580caa90 2023-10-08T02:12:28.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.138+0000 7f89467fc700 1 -- 172.21.15.119:0/3849737323 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8950056de0 con 0x7f89580caa90 2023-10-08T02:12:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8938066f30 msgr2=0x7f89380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8938066f30 0x7f89380693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8948007fd0 tx=0x7f8948009040 comp rx=0 tx=0).stop 2023-10-08T02:12:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 msgr2=0x7f89580b2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580b2c80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8958069220 tx=0x7f8950019000 comp rx=0 tx=0).stop 2023-10-08T02:12:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 shutdown_connections 2023-10-08T02:12:28.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8938066f30 0x7f89380693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f895806bcd0 0x7f89580c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89580caa90 0x7f89580b2c80 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 --2- 172.21.15.119:0/3849737323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f895806c0a0 0x7f89580affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 >> 172.21.15.119:0/3849737323 conn(0x7f89580ce600 msgr2=0x7f895815c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 shutdown_connections 2023-10-08T02:12:28.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.140+0000 7f895c9ff700 1 -- 172.21.15.119:0/3849737323 wait complete. 2023-10-08T02:12:28.158 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:12:28.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.340+0000 7f61e21ff700 1 Processor -- start 2023-10-08T02:12:28.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.340+0000 7f61e21ff700 1 -- start start 2023-10-08T02:12:28.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61e21ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61dc155c10 0x7f61dc155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61e21ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc0ac230 0x7f61dc0ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61e21ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61dc0ad930 con 0x7f61dc1568a0 2023-10-08T02:12:28.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61e21ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61dc0acd20 con 0x7f61dc155c10 2023-10-08T02:12:28.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61e21ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0ad000 con 0x7f61dc0ac230 2023-10-08T02:12:28.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61db7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61dc155c10 0x7f61dc155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61d3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc0ac230 0x7f61dc0ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61d3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc0ac230 0x7f61dc0ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:59664/0 (socket says 172.21.15.119:59664) 2023-10-08T02:12:28.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61db7fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f61dc155c10 0x7f61dc155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54912/0 (socket says 172.21.15.119:54912) 2023-10-08T02:12:28.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61d3fff700 1 -- 172.21.15.119:0/2140596396 learned_addr learned my addr 172.21.15.119:0/2140596396 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:28.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.342+0000 7f61dbfff700 1 --1- 172.21.15.119:0/2140596396 >> v1:172.21.15.184:6789/0 conn(0x7f61dc1568a0 0x7f61dc156cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:44928/0 (socket says 172.21.15.119:44928) 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61d3fff700 1 -- 172.21.15.119:0/2140596396 >> v2:172.21.15.184:3300/0 conn(0x7f61dc155c10 msgr2=0x7f61dc155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61d3fff700 1 --2- 172.21.15.119:0/2140596396 >> v2:172.21.15.184:3300/0 conn(0x7f61dc155c10 0x7f61dc155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61d3fff700 1 -- 172.21.15.119:0/2140596396 >> v1:172.21.15.184:6789/0 conn(0x7f61dc1568a0 legacy=0x7f61dc156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61d3fff700 1 -- 172.21.15.119:0/2140596396 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc0ad2e0 con 0x7f61dc0ac230 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61db7fe700 1 --2- 172.21.15.119:0/2140596396 >> v2:172.21.15.184:3300/0 conn(0x7f61dc155c10 0x7f61dc155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61daffd700 1 -- 172.21.15.119:0/2140596396 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007164184 0 0) 0x7f61dc0ad930 con 0x7f61dc1568a0 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61d3fff700 1 --2- 172.21.15.119:0/2140596396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc0ac230 0x7f61dc0ac6a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c40099b0 tx=0x7f61c4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad2fa62ef1d2d56c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61daffd700 1 -- 172.21.15.119:0/2140596396 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4009ea0 con 0x7f61dc0ac230 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61daffd700 1 -- 172.21.15.119:0/2140596396 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c400b860 con 0x7f61dc0ac230 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.343+0000 7f61daffd700 1 -- 172.21.15.119:0/2140596396 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c400bc00 con 0x7f61dc0ac230 2023-10-08T02:12:28.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.344+0000 7f61e21ff700 1 -- 172.21.15.119:0/2140596396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc0ac230 msgr2=0x7f61dc0ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.344+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2140596396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc0ac230 0x7f61dc0ac6a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c40099b0 tx=0x7f61c4009250 comp rx=0 tx=0).stop 2023-10-08T02:12:28.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.344+0000 7f61e21ff700 1 -- 172.21.15.119:0/2140596396 shutdown_connections 2023-10-08T02:12:28.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.344+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2140596396 >> v2:172.21.15.184:3300/0 conn(0x7f61dc155c10 0x7f61dc155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.344+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2140596396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc0ac230 0x7f61dc0ac6a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.344+0000 7f61e21ff700 1 -- 172.21.15.119:0/2140596396 >> 172.21.15.119:0/2140596396 conn(0x7f61dc06a320 msgr2=0x7f61dc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:28.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.344+0000 7f61e21ff700 1 -- 172.21.15.119:0/2140596396 shutdown_connections 2023-10-08T02:12:28.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.344+0000 7f61e21ff700 1 -- 172.21.15.119:0/2140596396 wait complete. 2023-10-08T02:12:28.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.345+0000 7f61e21ff700 1 Processor -- start 2023-10-08T02:12:28.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.345+0000 7f61e21ff700 1 -- start start 2023-10-08T02:12:28.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.346+0000 7f61e21ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc0ac230 0x7f61dc0cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.346+0000 7f61e21ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61dc155c10 0x7f61dc16d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.346+0000 7f61e21ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc1568a0 0x7f61dc0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.346+0000 7f61e21ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc157320 con 0x7f61dc155c10 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.346+0000 7f61e21ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0a9d50 con 0x7f61dc0ac230 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.346+0000 7f61e21ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc0ca660 con 0x7f61dc1568a0 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61dc155c10 0x7f61dc16d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61d3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61dc155c10 0x7f61dc16d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44040/0 (socket says 172.21.15.119:44040) 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61d3fff700 1 -- 172.21.15.119:0/2834371852 learned_addr learned my addr 172.21.15.119:0/2834371852 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61dbfff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc1568a0 0x7f61dc0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61db7fe700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc0ac230 0x7f61dc0cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61d3fff700 1 -- 172.21.15.119:0/2834371852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc1568a0 msgr2=0x7f61dc0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61d3fff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc1568a0 0x7f61dc0ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61d3fff700 1 -- 172.21.15.119:0/2834371852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc0ac230 msgr2=0x7f61dc0cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61d3fff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc0ac230 0x7f61dc0cbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.347+0000 7f61d3fff700 1 -- 172.21.15.119:0/2834371852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc009420 con 0x7f61dc155c10 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.348+0000 7f61d3fff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61dc155c10 0x7f61dc16d8f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f61c4006fd0 tx=0x7f61c4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.348+0000 7f61d37fe700 1 -- 172.21.15.119:0/2834371852 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4000d10 con 0x7f61dc155c10 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.348+0000 7f61d37fe700 1 -- 172.21.15.119:0/2834371852 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c400bbe0 con 0x7f61dc155c10 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.348+0000 7f61d37fe700 1 -- 172.21.15.119:0/2834371852 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c4011690 con 0x7f61dc155c10 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.348+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c40094f0 con 0x7f61dc155c10 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.348+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61dc157040 con 0x7f61dc155c10 2023-10-08T02:12:28.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.349+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61dc04c3c0 con 0x7f61dc155c10 2023-10-08T02:12:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.350+0000 7f61d37fe700 1 -- 172.21.15.119:0/2834371852 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f61c4011a80 con 0x7f61dc155c10 2023-10-08T02:12:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.351+0000 7f61d37fe700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f61bc066f30 0x7f61bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.351+0000 7f61d37fe700 1 -- 172.21.15.119:0/2834371852 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f61c4085f30 con 0x7f61dc155c10 2023-10-08T02:12:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.351+0000 7f61db7fe700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f61bc066f30 0x7f61bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.351+0000 7f61db7fe700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f61bc066f30 0x7f61bc0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009f40 tx=0x7f61cc0091a0 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:28.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.355+0000 7f61d37fe700 1 -- 172.21.15.119:0/2834371852 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f61c4052fd0 con 0x7f61dc155c10 2023-10-08T02:12:28.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.590+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61dc0ad5c0 con 0x7f61dc155c10 2023-10-08T02:12:28.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.591+0000 7f61d37fe700 1 -- 172.21.15.119:0/2834371852 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f61c40565f0 con 0x7f61dc155c10 2023-10-08T02:12:28.590 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:28.590 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7409,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3771852147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3771852147},{"type":"v1","addr":"172.21.15.184:6834","nonce":3771852147}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/524755843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":524755843},{"type":"v1","addr":"172.21.15.184:6835","nonce":524755843}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9144,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/295310163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":295310163},{"type":"v1","addr":"172.21.15.119:6833","nonce":295310163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:12:24.579741+0000","modified":"2023-10-08T02:12:28.133805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9150},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9150":{"gid":9150,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2713045427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2713045427},{"type":"v1","addr":"172.21.15.119:6835","nonce":2713045427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T02:12:28.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f61bc066f30 msgr2=0x7f61bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f61bc066f30 0x7f61bc0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61cc009f40 tx=0x7f61cc0091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61dc155c10 msgr2=0x7f61dc16d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61dc155c10 0x7f61dc16d8f0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f61c4006fd0 tx=0x7f61c4017000 comp rx=0 tx=0).stop 2023-10-08T02:12:28.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 shutdown_connections 2023-10-08T02:12:28.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f61bc066f30 0x7f61bc0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f61dc1568a0 0x7f61dc0ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f61dc0ac230 0x7f61dc0cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 --2- 172.21.15.119:0/2834371852 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61dc155c10 0x7f61dc16d8f0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 >> 172.21.15.119:0/2834371852 conn(0x7f61dc06a320 msgr2=0x7f61dc1532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:28.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 shutdown_connections 2023-10-08T02:12:28.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.593+0000 7f61e21ff700 1 -- 172.21.15.119:0/2834371852 wait complete. 2023-10-08T02:12:28.594 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 79 2023-10-08T02:12:28.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:12:28.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.793+0000 7fa47c7ff700 1 Processor -- start 2023-10-08T02:12:28.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.793+0000 7fa47c7ff700 1 -- start start 2023-10-08T02:12:28.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.794+0000 7fa47c7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b1850 0x7fa4740b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.794+0000 7fa47c7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa4741579f0 0x7fa4740abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.794+0000 7fa47c7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4740cb300 con 0x7fa4740ae680 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.794+0000 7fa47c7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa47406fc30 con 0x7fa4741579f0 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.794+0000 7fa47c7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4740cc260 con 0x7fa4740b1850 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47ad9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa4741579f0 0x7fa4740abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47ad9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa4741579f0 0x7fa4740abf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44048/0 (socket says 172.21.15.119:44048) 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47ad9c700 1 -- 172.21.15.119:0/2911381977 learned_addr learned my addr 172.21.15.119:0/2911381977 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47a59b700 1 --2- 172.21.15.119:0/2911381977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b1850 0x7fa4740b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47ad9c700 1 -- 172.21.15.119:0/2911381977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b1850 msgr2=0x7fa4740b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47ad9c700 1 --2- 172.21.15.119:0/2911381977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b1850 0x7fa4740b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47ad9c700 1 -- 172.21.15.119:0/2911381977 >> v1:172.21.15.184:6789/0 conn(0x7fa4740ae680 legacy=0x7fa4741572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:28.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47ad9c700 1 -- 172.21.15.119:0/2911381977 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740cb5e0 con 0x7fa4741579f0 2023-10-08T02:12:28.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.795+0000 7fa47ad9c700 1 --2- 172.21.15.119:0/2911381977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4741579f0 0x7fa4740abf60 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009f20 tx=0x7fa46c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5a538076d9a093 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:28.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa479599700 1 -- 172.21.15.119:0/2911381977 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00e070 con 0x7fa4741579f0 2023-10-08T02:12:28.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa479599700 1 -- 172.21.15.119:0/2911381977 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46c00ba40 con 0x7fa4741579f0 2023-10-08T02:12:28.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa479599700 1 -- 172.21.15.119:0/2911381977 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c0092e0 con 0x7fa4741579f0 2023-10-08T02:12:28.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa47c7ff700 1 -- 172.21.15.119:0/2911381977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4741579f0 msgr2=0x7fa4740abf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/2911381977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4741579f0 0x7fa4740abf60 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009f20 tx=0x7fa46c009840 comp rx=0 tx=0).stop 2023-10-08T02:12:28.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa47c7ff700 1 -- 172.21.15.119:0/2911381977 shutdown_connections 2023-10-08T02:12:28.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/2911381977 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4741579f0 0x7fa4740abf60 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/2911381977 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4740b1850 0x7fa4740b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa47c7ff700 1 -- 172.21.15.119:0/2911381977 >> 172.21.15.119:0/2911381977 conn(0x7fa47406a320 msgr2=0x7fa4740cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:28.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa47c7ff700 1 -- 172.21.15.119:0/2911381977 shutdown_connections 2023-10-08T02:12:28.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.796+0000 7fa47c7ff700 1 -- 172.21.15.119:0/2911381977 wait complete. 2023-10-08T02:12:28.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.798+0000 7fa47c7ff700 1 Processor -- start 2023-10-08T02:12:28.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.798+0000 7fa47c7ff700 1 -- start start 2023-10-08T02:12:28.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47c7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4740ae680 0x7fa4741511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47c7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4740b1850 0x7fa47406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47c7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4741579f0 0x7fa47406f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47c7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741516e0 con 0x7fa4740ae680 2023-10-08T02:12:28.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47c7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa474167560 con 0x7fa4740b1850 2023-10-08T02:12:28.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47c7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa47415bba0 con 0x7fa4741579f0 2023-10-08T02:12:28.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47a59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4740ae680 0x7fa4741511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47a59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4740ae680 0x7fa4741511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44056/0 (socket says 172.21.15.119:44056) 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47a59b700 1 -- 172.21.15.119:0/971176444 learned_addr learned my addr 172.21.15.119:0/971176444 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa479d9a700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4740b1850 0x7fa47406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47ad9c700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4741579f0 0x7fa47406f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47a59b700 1 -- 172.21.15.119:0/971176444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4741579f0 msgr2=0x7fa47406f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47a59b700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4741579f0 0x7fa47406f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47a59b700 1 -- 172.21.15.119:0/971176444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4740b1850 msgr2=0x7fa47406eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47a59b700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4740b1850 0x7fa47406eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.799+0000 7fa47a59b700 1 -- 172.21.15.119:0/971176444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c009ae0 con 0x7fa4740ae680 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.800+0000 7fa47a59b700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4740ae680 0x7fa4741511a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fa47000d890 tx=0x7fa470008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.800+0000 7fa46a7fc700 1 -- 172.21.15.119:0/971176444 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4700125f0 con 0x7fa4740ae680 2023-10-08T02:12:28.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.800+0000 7fa46a7fc700 1 -- 172.21.15.119:0/971176444 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa470012770 con 0x7fa4740ae680 2023-10-08T02:12:28.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.800+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47415be80 con 0x7fa4740ae680 2023-10-08T02:12:28.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.800+0000 7fa46a7fc700 1 -- 172.21.15.119:0/971176444 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470012b30 con 0x7fa4740ae680 2023-10-08T02:12:28.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.800+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa474167820 con 0x7fa4740ae680 2023-10-08T02:12:28.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.801+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa474167a40 con 0x7fa4740ae680 2023-10-08T02:12:28.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.802+0000 7fa46a7fc700 1 -- 172.21.15.119:0/971176444 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa47001e460 con 0x7fa4740ae680 2023-10-08T02:12:28.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.803+0000 7fa46a7fc700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa458066f30 0x7fa4580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:28.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.803+0000 7fa46a7fc700 1 -- 172.21.15.119:0/971176444 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fa470084d70 con 0x7fa4740ae680 2023-10-08T02:12:28.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.803+0000 7fa479d9a700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa458066f30 0x7fa4580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:28.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.803+0000 7fa479d9a700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa458066f30 0x7fa4580693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa474068f10 tx=0x7fa464007ea0 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:28.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.807+0000 7fa46a7fc700 1 -- 172.21.15.119:0/971176444 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa470051d70 con 0x7fa4740ae680 2023-10-08T02:12:28.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.997+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa47404c3c0 con 0x7fa4740ae680 2023-10-08T02:12:28.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:28.998+0000 7fa46a7fc700 1 -- 172.21.15.119:0/971176444 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16125 (secure 0 0 0) 0x7fa470055390 con 0x7fa4740ae680 2023-10-08T02:12:28.998 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:28.998 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":171,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:12:26.110258+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:12:22.679007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T02:12:23.548789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:12:29.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa458066f30 msgr2=0x7fa4580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa458066f30 0x7fa4580693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa474068f10 tx=0x7fa464007ea0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4740ae680 msgr2=0x7fa4741511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4740ae680 0x7fa4741511a0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fa47000d890 tx=0x7fa470008040 comp rx=0 tx=0).stop 2023-10-08T02:12:29.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 shutdown_connections 2023-10-08T02:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fa458066f30 0x7fa4580693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa4741579f0 0x7fa47406f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4740ae680 0x7fa4741511a0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 --2- 172.21.15.119:0/971176444 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa4740b1850 0x7fa47406eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 >> 172.21.15.119:0/971176444 conn(0x7fa47406a320 msgr2=0x7fa4740adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.000+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 shutdown_connections 2023-10-08T02:12:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.001+0000 7fa47c7ff700 1 -- 172.21.15.119:0/971176444 wait complete. 2023-10-08T02:12:29.017 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T02:12:29.018 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:12:29.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.206+0000 7f708adff700 1 Processor -- start 2023-10-08T02:12:29.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.206+0000 7f708adff700 1 -- start start 2023-10-08T02:12:29.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.207+0000 7f708adff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f708406c0a0 0x7f70840caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.207+0000 7f708adff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708415d7e0 con 0x7f708415df50 2023-10-08T02:12:29.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.207+0000 7f708adff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708415dac0 con 0x7f708406bcd0 2023-10-08T02:12:29.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.207+0000 7f708adff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708415aa30 con 0x7f708406c0a0 2023-10-08T02:12:29.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f7083fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f708406c0a0 0x7f70840caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f708939c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f708415df50 0x7f70840c8c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59518/0 (socket says 172.21.15.119:59518) 2023-10-08T02:12:29.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f708939c700 1 -- 172.21.15.119:0/3494628910 learned_addr learned my addr 172.21.15.119:0/3494628910 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:29.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f70837fe700 1 -- 172.21.15.119:0/3494628910 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56417077 0 0) 0x7f708415d7e0 con 0x7f708415df50 2023-10-08T02:12:29.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f70837fe700 1 -- 172.21.15.119:0/3494628910 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7074003570 con 0x7f708415df50 2023-10-08T02:12:29.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f7083fff700 1 -- 172.21.15.119:0/3494628910 >> v1:172.21.15.184:6789/0 conn(0x7f708406bcd0 legacy=0x7f708415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:29.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f7083fff700 1 -- 172.21.15.119:0/3494628910 >> v1:172.21.15.119:6789/0 conn(0x7f708415df50 legacy=0x7f70840c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f7083fff700 1 -- 172.21.15.119:0/3494628910 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708415ad10 con 0x7f708406c0a0 2023-10-08T02:12:29.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f7083fff700 1 --2- 172.21.15.119:0/3494628910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708406c0a0 0x7f70840caa90 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f70700098e0 tx=0x7f70700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b828d46920822d17 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:29.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.208+0000 7f70837fe700 1 -- 172.21.15.119:0/3494628910 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070011630 con 0x7f708406c0a0 2023-10-08T02:12:29.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f70837fe700 1 -- 172.21.15.119:0/3494628910 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070011790 con 0x7f708406c0a0 2023-10-08T02:12:29.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f70837fe700 1 -- 172.21.15.119:0/3494628910 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070011b30 con 0x7f708406c0a0 2023-10-08T02:12:29.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f708adff700 1 -- 172.21.15.119:0/3494628910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708406c0a0 msgr2=0x7f70840caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f708adff700 1 --2- 172.21.15.119:0/3494628910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708406c0a0 0x7f70840caa90 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f70700098e0 tx=0x7f70700091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f708adff700 1 -- 172.21.15.119:0/3494628910 shutdown_connections 2023-10-08T02:12:29.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f708adff700 1 --2- 172.21.15.119:0/3494628910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708406c0a0 0x7f70840caa90 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f708adff700 1 -- 172.21.15.119:0/3494628910 >> 172.21.15.119:0/3494628910 conn(0x7f70840ce600 msgr2=0x7f70840cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:29.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f708adff700 1 -- 172.21.15.119:0/3494628910 shutdown_connections 2023-10-08T02:12:29.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.209+0000 7f708adff700 1 -- 172.21.15.119:0/3494628910 wait complete. 2023-10-08T02:12:29.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.211+0000 7f708adff700 1 Processor -- start 2023-10-08T02:12:29.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.211+0000 7f708adff700 1 -- start start 2023-10-08T02:12:29.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708adff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708406bcd0 0x7f70840c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708adff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f708406c0a0 0x7f708416b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708adff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708415df50 0x7f708416d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708adff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708416a260 con 0x7f708415df50 2023-10-08T02:12:29.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708adff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70840c87c0 con 0x7f708406bcd0 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708adff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840c5950 con 0x7f708406c0a0 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708939c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708415df50 0x7f708416d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708939c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708415df50 0x7f708416d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44084/0 (socket says 172.21.15.119:44084) 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708939c700 1 -- 172.21.15.119:0/865312696 learned_addr learned my addr 172.21.15.119:0/865312696 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f7088b9b700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708406bcd0 0x7f70840c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f7083fff700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f708406c0a0 0x7f708416b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708939c700 1 -- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f708406c0a0 msgr2=0x7f708416b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708939c700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f708406c0a0 0x7f708416b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708939c700 1 -- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708406bcd0 msgr2=0x7f70840c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708939c700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708406bcd0 0x7f70840c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f708939c700 1 -- 172.21.15.119:0/865312696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7070009420 con 0x7f708415df50 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f7088b9b700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708406bcd0 0x7f70840c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.212+0000 7f7083fff700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f708406c0a0 0x7f708416b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.213+0000 7f708939c700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708415df50 0x7f708416d0a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f707c001460 tx=0x7f707c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.213+0000 7f7080ff9700 1 -- 172.21.15.119:0/865312696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c00a950 con 0x7f708415df50 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.213+0000 7f7080ff9700 1 -- 172.21.15.119:0/865312696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c00aab0 con 0x7f708415df50 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.213+0000 7f7080ff9700 1 -- 172.21.15.119:0/865312696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c00ae30 con 0x7f708415df50 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.213+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708416dcf0 con 0x7f708415df50 2023-10-08T02:12:29.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.213+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708416e2b0 con 0x7f708415df50 2023-10-08T02:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.214+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708404c3c0 con 0x7f708415df50 2023-10-08T02:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.215+0000 7f7080ff9700 1 -- 172.21.15.119:0/865312696 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f707c007300 con 0x7f708415df50 2023-10-08T02:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.216+0000 7f7080ff9700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7068066f30 0x7f70680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.216+0000 7f7080ff9700 1 -- 172.21.15.119:0/865312696 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f707c084b70 con 0x7f708415df50 2023-10-08T02:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.216+0000 7f7088b9b700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7068066f30 0x7f70680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.216+0000 7f7088b9b700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7068066f30 0x7f70680693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7084155c40 tx=0x7f7078008040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:29.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.220+0000 7f7080ff9700 1 -- 172.21.15.119:0/865312696 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f707c051c90 con 0x7f708415df50 2023-10-08T02:12:29.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.405+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f708415aff0 con 0x7f7068066f30 2023-10-08T02:12:29.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.415+0000 7f7080ff9700 1 -- 172.21.15.119:0/865312696 <== mgr.8967 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278615 (secure 0 0 0) 0x7f708415aff0 con 0x7f7068066f30 2023-10-08T02:12:29.418 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:29.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7068066f30 msgr2=0x7f70680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7068066f30 0x7f70680693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7084155c40 tx=0x7f7078008040 comp rx=0 tx=0).stop 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708415df50 msgr2=0x7f708416d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708415df50 0x7f708416d0a0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f707c001460 tx=0x7f707c001a40 comp rx=0 tx=0).stop 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 shutdown_connections 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f7068066f30 0x7f70680693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f708415df50 0x7f708416d0a0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f708406bcd0 0x7f70840c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 --2- 172.21.15.119:0/865312696 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f708406c0a0 0x7f708416b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 >> 172.21.15.119:0/865312696 conn(0x7f70840ce600 msgr2=0x7f708415a190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 shutdown_connections 2023-10-08T02:12:29.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.419+0000 7f708adff700 1 -- 172.21.15.119:0/865312696 wait complete. 2023-10-08T02:12:29.422 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:12:29.436 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:12:28.684490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2494092,"kb_used_data":17484,"kb_used_omap":0,"kb_used_meta":2476544,"kb_avail":752480628,"statfs":{"total":773094113280,"available":770540163072,"internally_reserved":0,"allocated":17903616,"data_stored":9499959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2535981056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.594994"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098112+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:03.268132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099113+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:03.137966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099808+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:30.649373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.100563+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:46.851648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102497+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:25.895322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099636+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:44.349959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101831+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.351918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099129+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:53.423324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098367+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:26.108797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097986+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:49.675021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098740+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:18.805561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.096562+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.992916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095862+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:47.969697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101713+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:54.743852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102722+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:47.969697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.100333+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:59.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098397+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:14.786069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102471+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:33.551205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098330+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:41.052726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099253+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:15.100827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098187+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:16.271655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097408+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:43.911275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095703+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.351918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097439+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:38.305361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101273+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:42.653885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101358+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:02.429294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101190+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.136401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098790+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:03.061501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098706+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:31.610625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095890+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:25.895322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098402+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:02.557364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095826+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:33.551205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"160'293","reported_seq":911,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T02:12:20.515876+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:12:20.515876+0000","last_peered":"2023-10-08T02:12:20.515876+0000","last_clean":"2023-10-08T02:12:20.515876+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:12:20.515876+0000","last_undegraded":"2023-10-08T02:12:20.515876+0000","last_fullsized":"2023-10-08T02:12:20.515876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575759,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312192,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059648,"statfs":{"total":96636764160,"available":96317079552,"internally_reserved":0,"allocated":2678784,"data_stored":1627920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.70099999999999996,"15min":0.73099999999999998},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76100000000000001,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.751,"15min":0.75},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.73599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81200000000000006,"5min":0.999,"15min":1.0580000000000001},"last":0.64800000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.755,"15min":0.76400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.873,"5min":0.94899999999999995,"15min":1.0600000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.755,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.71299999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.79500000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.875,"5min":1.502,"15min":1.502},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.46300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":0.94999999999999996,"15min":0.97799999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":1.034,"15min":1.034},"last":0.432}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":0.76800000000000002},"min":{"1min":0.56999999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.998,"15min":1.0149999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":1.109,"15min":1.109},"last":0.75}]}]},{"osd":6,"up_from":13,"seq":55834575759,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056120,"statfs":{"total":96636764160,"available":96313466880,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.76000000000000001,"15min":0.74199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.569,"15min":1.569},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.74399999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94199999999999995,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.503}]},{"osd":1,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89900000000000002,"5min":1.472,"15min":1.472},"last":0.67600000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.81799999999999995,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79600000000000004,"15min":0.76500000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.495},"max":{"1min":0.91700000000000004,"5min":1.603,"15min":1.603},"last":0.625}]},{"osd":3,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.871,"5min":1.169,"15min":1.169},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":1.133,"15min":1.133},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.439,"5min":0.439,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82599999999999996,"5min":0.94299999999999995,"15min":1.0680000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74199999999999999,"15min":0.72099999999999997},"min":{"1min":0.503,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90200000000000002,"5min":1.232,"15min":1.232},"last":0.83799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307516,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064324,"statfs":{"total":96636764160,"available":96321867776,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.497},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.603,"15min":1.603},"last":0.434}]},{"osd":2,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.497,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.60199999999999998,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.751,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.70799999999999996,"5min":3.698,"15min":3.698},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86399999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95999999999999996,"5min":1.734,"15min":1.734},"last":0.77400000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.53600000000000003},"max":{"1min":1.0489999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91600000000000004,"5min":1.403,"15min":1.403},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.61299999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0149999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.573,"15min":1.573},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.875,"15min":0.86499999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.58999999999999997},"max":{"1min":0.99399999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575758,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308096,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063744,"statfs":{"total":96636764160,"available":96321273856,"internally_reserved":0,"allocated":2678784,"data_stored":1627920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.82899999999999996,"15min":0.83999999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.315}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58299999999999996,"5min":0.70699999999999996,"15min":0.746},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68500000000000005,"5min":0.88300000000000001,"15min":1.325},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.42399999999999999,"5min":0.41199999999999998,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.5},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72399999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54700000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.99399999999999999,"5min":1.052,"15min":1.24},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0009999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0489999999999999,"5min":1.327,"15min":1.327},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.878,"15min":0.879},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.653,"15min":1.653},"last":0.94399999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83599999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"max":{"1min":1.0209999999999999,"5min":1.577,"15min":1.577},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58099999999999996},"max":{"1min":0.97699999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.96499999999999997}]},{"osd":7,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.84799999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.129,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.89000000000000001,"15min":0.88200000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0720000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.97999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311624,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060216,"statfs":{"total":96636764160,"available":96317661184,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65500000000000003,"5min":1.27,"15min":1.27},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.45100000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65600000000000003,"5min":0.79500000000000004,"15min":0.81699999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":0.69699999999999995,"5min":1.651,"15min":1.651},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.39400000000000002},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":0.55200000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81399999999999995,"15min":0.81000000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.55100000000000005},"max":{"1min":1.0209999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53400000000000003},"max":{"1min":0.93799999999999994,"5min":3.004,"15min":3.004},"last":0.64700000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.86799999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.046,"5min":3.145,"15min":3.145},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0860000000000001,"5min":1.847,"15min":1.847},"last":0.72399999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.57699999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4950000000000001,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.623,"5min":0.60199999999999998,"15min":0.58599999999999997},"max":{"1min":1.0580000000000001,"5min":1.331,"15min":1.331},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.88700000000000001,"15min":0.85999999999999999},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":2.899,"15min":2.899},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.86599999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"max":{"1min":1.4570000000000001,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.754}]}]},{"osd":3,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056120,"statfs":{"total":96636764160,"available":96313466880,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47999999999999998,"15min":0.48299999999999998},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.54700000000000004,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89500000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.629,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83699999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56699999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.214},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":0.998,"5min":1.0680000000000001,"15min":1.0900000000000001},"last":0.628}]},{"osd":5,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.77100000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.876,"5min":1.0589999999999999,"15min":1.125},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64200000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.01,"5min":1.403,"15min":1.403},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.81399999999999995,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.55600000000000005},"max":{"1min":0.90700000000000003,"5min":1.492,"15min":1.492},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.83799999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93799999999999994,"5min":1.341,"15min":1.341},"last":0.71499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575758,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311612,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060228,"statfs":{"total":96636764160,"available":96317673472,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40400000000000003},"max":{"1min":0.90100000000000002,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0009999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.52},"last":0.622}]},{"osd":2,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60999999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90800000000000003,"5min":1.1579999999999999,"15min":1.403},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.208,"15min":1.5069999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":1.05,"15min":1.05},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93400000000000005,"5min":1.038,"15min":1.1559999999999999},"last":0.63600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.016},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.0680000000000001,"15min":1.129},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.48799999999999999},"max":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311612,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060228,"statfs":{"total":96636764160,"available":96317673472,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.111,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.76700000000000002,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.438},"max":{"1min":1.0009999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.64500000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":0.77800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42699999999999999},"max":{"1min":0.95399999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.77600000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.499},"max":{"1min":0.995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.50900000000000001},"max":{"1min":0.97699999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80700000000000005,"15min":0.77900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96399999999999997,"5min":1.0620000000000001,"15min":1.0669999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0529999999999999,"15min":1.091},"last":0.53300000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.441},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.115},"last":0.872}]},{"osd":6,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0629999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.077},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0189999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.82299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:12:29.443 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:12:29.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.658+0000 7f0bf6dff700 1 Processor -- start 2023-10-08T02:12:29.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.658+0000 7f0bf6dff700 1 -- start start 2023-10-08T02:12:29.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.660+0000 7f0bf6dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0bf00b0a10 0x7f0bf0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.660+0000 7f0bf6dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf00ad930 con 0x7f0bf01578b0 2023-10-08T02:12:29.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.660+0000 7f0bf6dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf00acd20 con 0x7f0bf0155c10 2023-10-08T02:12:29.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.660+0000 7f0bf6dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf00ad000 con 0x7f0bf00b0a10 2023-10-08T02:12:29.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.660+0000 7f0beffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0bf01578b0 0x7f0bf00af2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59534/0 (socket says 172.21.15.119:59534) 2023-10-08T02:12:29.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.660+0000 7f0beffff700 1 -- 172.21.15.119:0/1931540007 learned_addr learned my addr 172.21.15.119:0/1931540007 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:29.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.660+0000 7f0bf539c700 1 --2- 172.21.15.119:0/1931540007 >> v2:172.21.15.119:3300/0 conn(0x7f0bf00b0a10 0x7f0bf0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.660+0000 7f0bef7fe700 1 -- 172.21.15.119:0/1931540007 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283934472 0 0) 0x7f0bf00ad930 con 0x7f0bf01578b0 2023-10-08T02:12:29.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.661+0000 7f0bef7fe700 1 -- 172.21.15.119:0/1931540007 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0be0003570 con 0x7f0bf01578b0 2023-10-08T02:12:29.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.661+0000 7f0bf539c700 1 -- 172.21.15.119:0/1931540007 >> v1:172.21.15.184:6789/0 conn(0x7f0bf0155c10 legacy=0x7f0bf00ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.661+0000 7f0bf539c700 1 -- 172.21.15.119:0/1931540007 >> v1:172.21.15.119:6789/0 conn(0x7f0bf01578b0 legacy=0x7f0bf00af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.661+0000 7f0bf539c700 1 -- 172.21.15.119:0/1931540007 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf00ad2e0 con 0x7f0bf00b0a10 2023-10-08T02:12:29.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.661+0000 7f0bf539c700 1 --2- 172.21.15.119:0/1931540007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf00b0a10 0x7f0bf0155fe0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009e60 tx=0x7f0be8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df18361760b11e9f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:29.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.661+0000 7f0bef7fe700 1 -- 172.21.15.119:0/1931540007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be800e070 con 0x7f0bf00b0a10 2023-10-08T02:12:29.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.661+0000 7f0bef7fe700 1 -- 172.21.15.119:0/1931540007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be800bb60 con 0x7f0bf00b0a10 2023-10-08T02:12:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.661+0000 7f0bef7fe700 1 -- 172.21.15.119:0/1931540007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8012430 con 0x7f0bf00b0a10 2023-10-08T02:12:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.662+0000 7f0bf6dff700 1 -- 172.21.15.119:0/1931540007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf00b0a10 msgr2=0x7f0bf0155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.662+0000 7f0bf6dff700 1 --2- 172.21.15.119:0/1931540007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf00b0a10 0x7f0bf0155fe0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f0be8009e60 tx=0x7f0be8009780 comp rx=0 tx=0).stop 2023-10-08T02:12:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.662+0000 7f0bf6dff700 1 -- 172.21.15.119:0/1931540007 shutdown_connections 2023-10-08T02:12:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.662+0000 7f0bf6dff700 1 --2- 172.21.15.119:0/1931540007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf00b0a10 0x7f0bf0155fe0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.662+0000 7f0bf6dff700 1 -- 172.21.15.119:0/1931540007 >> 172.21.15.119:0/1931540007 conn(0x7f0bf006a320 msgr2=0x7f0bf00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.662+0000 7f0bf6dff700 1 -- 172.21.15.119:0/1931540007 shutdown_connections 2023-10-08T02:12:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.662+0000 7f0bf6dff700 1 -- 172.21.15.119:0/1931540007 wait complete. 2023-10-08T02:12:29.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.663+0000 7f0bf6dff700 1 Processor -- start 2023-10-08T02:12:29.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.663+0000 7f0bf6dff700 1 -- start start 2023-10-08T02:12:29.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.664+0000 7f0bf6dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf00b0a10 0x7f0bf014af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.664+0000 7f0bf6dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf0155c10 0x7f0bf00aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.664+0000 7f0bf6dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf01578b0 0x7f0bf0149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.664+0000 7f0bf6dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf016aa50 con 0x7f0bf0155c10 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.664+0000 7f0bf6dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf014b4a0 con 0x7f0bf00b0a10 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.664+0000 7f0bf6dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0169e40 con 0x7f0bf01578b0 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf0155c10 0x7f0bf00aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf0155c10 0x7f0bf00aa3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44094/0 (socket says 172.21.15.119:44094) 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 -- 172.21.15.119:0/3493035923 learned_addr learned my addr 172.21.15.119:0/3493035923 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0bf4b9b700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf00b0a10 0x7f0bf014af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 -- 172.21.15.119:0/3493035923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf01578b0 msgr2=0x7f0bf0149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf01578b0 0x7f0bf0149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 -- 172.21.15.119:0/3493035923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf00b0a10 msgr2=0x7f0bf014af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf00b0a10 0x7f0bf014af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 -- 172.21.15.119:0/3493035923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be8009a20 con 0x7f0bf0155c10 2023-10-08T02:12:29.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0beffff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf0155c10 0x7f0bf00aa3d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00c590 tx=0x7f0bdc007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:29.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.665+0000 7f0becff9700 1 -- 172.21.15.119:0/3493035923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc005b30 con 0x7f0bf0155c10 2023-10-08T02:12:29.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.666+0000 7f0becff9700 1 -- 172.21.15.119:0/3493035923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc005cb0 con 0x7f0bf0155c10 2023-10-08T02:12:29.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.666+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf016a120 con 0x7f0bf0155c10 2023-10-08T02:12:29.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.666+0000 7f0becff9700 1 -- 172.21.15.119:0/3493035923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc007320 con 0x7f0bf0155c10 2023-10-08T02:12:29.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.666+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf016a6e0 con 0x7f0bf0155c10 2023-10-08T02:12:29.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.667+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf004c3c0 con 0x7f0bf0155c10 2023-10-08T02:12:29.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.667+0000 7f0becff9700 1 -- 172.21.15.119:0/3493035923 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0bdc01b3b0 con 0x7f0bf0155c10 2023-10-08T02:12:29.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.668+0000 7f0becff9700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bd4066f30 0x7f0bd40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:29.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.668+0000 7f0becff9700 1 -- 172.21.15.119:0/3493035923 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f0bdc083ca0 con 0x7f0bf0155c10 2023-10-08T02:12:29.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.669+0000 7f0bf4b9b700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bd4066f30 0x7f0bd40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:29.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.669+0000 7f0bf4b9b700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bd4066f30 0x7f0bd40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0be4003860 tx=0x7f0be4009040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:29.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.672+0000 7f0becff9700 1 -- 172.21.15.119:0/3493035923 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0bdc050d40 con 0x7f0bf0155c10 2023-10-08T02:12:29.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.857+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0bf00ad5c0 con 0x7f0bd4066f30 2023-10-08T02:12:29.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.865+0000 7f0becff9700 1 -- 172.21.15.119:0/3493035923 <== mgr.8967 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278615 (secure 0 0 0) 0x7f0bf00ad5c0 con 0x7f0bd4066f30 2023-10-08T02:12:29.868 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bd4066f30 msgr2=0x7f0bd40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bd4066f30 0x7f0bd40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0be4003860 tx=0x7f0be4009040 comp rx=0 tx=0).stop 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf0155c10 msgr2=0x7f0bf00aa3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf0155c10 0x7f0bf00aa3d0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f0bdc00c590 tx=0x7f0bdc007e00 comp rx=0 tx=0).stop 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 shutdown_connections 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0bd4066f30 0x7f0bd40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0bf01578b0 0x7f0bf0149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0bf00b0a10 0x7f0bf014af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 --2- 172.21.15.119:0/3493035923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0bf0155c10 0x7f0bf00aa3d0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:29.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 >> 172.21.15.119:0/3493035923 conn(0x7f0bf006a320 msgr2=0x7f0bf006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:29.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 shutdown_connections 2023-10-08T02:12:29.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:29.870+0000 7f0bf6dff700 1 -- 172.21.15.119:0/3493035923 wait complete. 2023-10-08T02:12:29.872 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:12:29.887 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:12:28.684490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2494092,"kb_used_data":17484,"kb_used_omap":0,"kb_used_meta":2476544,"kb_avail":752480628,"statfs":{"total":773094113280,"available":770540163072,"internally_reserved":0,"allocated":17903616,"data_stored":9499959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2535981056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.594994"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098112+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:03.268132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099113+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:03.137966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099808+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:30.649373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.100563+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:46.851648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102497+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:25.895322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099636+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:44.349959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101831+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.351918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099129+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:53.423324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098367+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:26.108797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097986+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:49.675021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098740+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:18.805561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.096562+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.992916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095862+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:47.969697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101713+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:54.743852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102722+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:47.969697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.100333+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:59.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098397+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:14.786069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102471+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:33.551205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098330+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:41.052726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099253+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:15.100827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098187+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:16.271655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097408+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:43.911275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095703+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.351918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097439+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:38.305361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101273+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:42.653885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101358+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:02.429294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101190+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.136401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098790+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:03.061501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098706+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:31.610625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095890+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:25.895322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098402+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:02.557364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095826+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:33.551205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"160'293","reported_seq":911,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T02:12:20.515876+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:12:20.515876+0000","last_peered":"2023-10-08T02:12:20.515876+0000","last_clean":"2023-10-08T02:12:20.515876+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:12:20.515876+0000","last_undegraded":"2023-10-08T02:12:20.515876+0000","last_fullsized":"2023-10-08T02:12:20.515876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575759,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312192,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059648,"statfs":{"total":96636764160,"available":96317079552,"internally_reserved":0,"allocated":2678784,"data_stored":1627920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.70099999999999996,"15min":0.73099999999999998},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76100000000000001,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.751,"15min":0.75},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.73599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81200000000000006,"5min":0.999,"15min":1.0580000000000001},"last":0.64800000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.755,"15min":0.76400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.873,"5min":0.94899999999999995,"15min":1.0600000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.755,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.71299999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.79500000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.875,"5min":1.502,"15min":1.502},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.46300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":0.94999999999999996,"15min":0.97799999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":1.034,"15min":1.034},"last":0.432}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":0.76800000000000002},"min":{"1min":0.56999999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.998,"15min":1.0149999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":1.109,"15min":1.109},"last":0.75}]}]},{"osd":6,"up_from":13,"seq":55834575759,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056120,"statfs":{"total":96636764160,"available":96313466880,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.76000000000000001,"15min":0.74199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.569,"15min":1.569},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.74399999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94199999999999995,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.503}]},{"osd":1,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89900000000000002,"5min":1.472,"15min":1.472},"last":0.67600000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.81799999999999995,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79600000000000004,"15min":0.76500000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.495},"max":{"1min":0.91700000000000004,"5min":1.603,"15min":1.603},"last":0.625}]},{"osd":3,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.871,"5min":1.169,"15min":1.169},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":1.133,"15min":1.133},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.439,"5min":0.439,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82599999999999996,"5min":0.94299999999999995,"15min":1.0680000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74199999999999999,"15min":0.72099999999999997},"min":{"1min":0.503,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90200000000000002,"5min":1.232,"15min":1.232},"last":0.83799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307516,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064324,"statfs":{"total":96636764160,"available":96321867776,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.497},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.603,"15min":1.603},"last":0.434}]},{"osd":2,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.497,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.60199999999999998,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.751,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.70799999999999996,"5min":3.698,"15min":3.698},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86399999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95999999999999996,"5min":1.734,"15min":1.734},"last":0.77400000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.53600000000000003},"max":{"1min":1.0489999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91600000000000004,"5min":1.403,"15min":1.403},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.61299999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0149999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.573,"15min":1.573},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.875,"15min":0.86499999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.58999999999999997},"max":{"1min":0.99399999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575758,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308096,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063744,"statfs":{"total":96636764160,"available":96321273856,"internally_reserved":0,"allocated":2678784,"data_stored":1627920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.82899999999999996,"15min":0.83999999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.315}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58299999999999996,"5min":0.70699999999999996,"15min":0.746},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68500000000000005,"5min":0.88300000000000001,"15min":1.325},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.42399999999999999,"5min":0.41199999999999998,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.5},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72399999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54700000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.99399999999999999,"5min":1.052,"15min":1.24},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0009999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0489999999999999,"5min":1.327,"15min":1.327},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.878,"15min":0.879},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.653,"15min":1.653},"last":0.94399999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83599999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"max":{"1min":1.0209999999999999,"5min":1.577,"15min":1.577},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58099999999999996},"max":{"1min":0.97699999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.96499999999999997}]},{"osd":7,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.84799999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.129,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.89000000000000001,"15min":0.88200000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0720000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.97999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311624,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060216,"statfs":{"total":96636764160,"available":96317661184,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65500000000000003,"5min":1.27,"15min":1.27},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.45100000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65600000000000003,"5min":0.79500000000000004,"15min":0.81699999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":0.69699999999999995,"5min":1.651,"15min":1.651},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.39400000000000002},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":0.55200000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81399999999999995,"15min":0.81000000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.55100000000000005},"max":{"1min":1.0209999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53400000000000003},"max":{"1min":0.93799999999999994,"5min":3.004,"15min":3.004},"last":0.64700000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.86799999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.046,"5min":3.145,"15min":3.145},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0860000000000001,"5min":1.847,"15min":1.847},"last":0.72399999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.57699999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4950000000000001,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.623,"5min":0.60199999999999998,"15min":0.58599999999999997},"max":{"1min":1.0580000000000001,"5min":1.331,"15min":1.331},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.88700000000000001,"15min":0.85999999999999999},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":2.899,"15min":2.899},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.86599999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"max":{"1min":1.4570000000000001,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.754}]}]},{"osd":3,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056120,"statfs":{"total":96636764160,"available":96313466880,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47999999999999998,"15min":0.48299999999999998},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.54700000000000004,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89500000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.629,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83699999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56699999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.214},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":0.998,"5min":1.0680000000000001,"15min":1.0900000000000001},"last":0.628}]},{"osd":5,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.77100000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.876,"5min":1.0589999999999999,"15min":1.125},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64200000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.01,"5min":1.403,"15min":1.403},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.81399999999999995,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.55600000000000005},"max":{"1min":0.90700000000000003,"5min":1.492,"15min":1.492},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.83799999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93799999999999994,"5min":1.341,"15min":1.341},"last":0.71499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575758,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311612,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060228,"statfs":{"total":96636764160,"available":96317673472,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40400000000000003},"max":{"1min":0.90100000000000002,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0009999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.52},"last":0.622}]},{"osd":2,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60999999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90800000000000003,"5min":1.1579999999999999,"15min":1.403},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.208,"15min":1.5069999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":1.05,"15min":1.05},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93400000000000005,"5min":1.038,"15min":1.1559999999999999},"last":0.63600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.016},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.0680000000000001,"15min":1.129},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.48799999999999999},"max":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311612,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060228,"statfs":{"total":96636764160,"available":96317673472,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.111,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.76700000000000002,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.438},"max":{"1min":1.0009999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.64500000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":0.77800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42699999999999999},"max":{"1min":0.95399999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.77600000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.499},"max":{"1min":0.995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.50900000000000001},"max":{"1min":0.97699999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80700000000000005,"15min":0.77900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96399999999999997,"5min":1.0620000000000001,"15min":1.0669999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0529999999999999,"15min":1.091},"last":0.53300000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.441},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.115},"last":0.872}]},{"osd":6,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0629999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.077},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0189999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.82299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:12:29.894 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:12:30.085 INFO:tasks.ceph.mds.c.smithi119.stderr:2023-10-08T02:12:30.086+0000 7f0985ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T02:12:30.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.108+0000 7f1720bff700 1 Processor -- start 2023-10-08T02:12:30.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.108+0000 7f1720bff700 1 -- start start 2023-10-08T02:12:30.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.109+0000 7f1720bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f171c0cbf30 0x7f171c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.109+0000 7f1720bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.109+0000 7f1720bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c0cc300 con 0x7f171c15a270 2023-10-08T02:12:30.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.109+0000 7f1720bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c156bf0 con 0x7f171c0cbf30 2023-10-08T02:12:30.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.110+0000 7f1720bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0ae950 con 0x7f171c0ccf90 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.110+0000 7f171a59c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f171c0cbf30 0x7f171c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.110+0000 7f1713fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.110+0000 7f171a59c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f171c0cbf30 0x7f171c0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54958/0 (socket says 172.21.15.119:54958) 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.110+0000 7f171a59c700 1 -- 172.21.15.119:0/1110426935 learned_addr learned my addr 172.21.15.119:0/1110426935 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.111+0000 7f171a59c700 1 -- 172.21.15.119:0/1110426935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 msgr2=0x7f171c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.111+0000 7f171a59c700 1 --2- 172.21.15.119:0/1110426935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.111+0000 7f171a59c700 1 -- 172.21.15.119:0/1110426935 >> v1:172.21.15.184:6789/0 conn(0x7f171c15a270 legacy=0x7f171c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.111+0000 7f171a59c700 1 -- 172.21.15.119:0/1110426935 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0ad370 con 0x7f171c0cbf30 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.111+0000 7f171a59c700 1 --2- 172.21.15.119:0/1110426935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 0x7f171c0ae410 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f170c00bc80 tx=0x7f170c00b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b524df49baee3064 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.111+0000 7f1719d9b700 1 -- 172.21.15.119:0/1110426935 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c00dbb0 con 0x7f171c0cbf30 2023-10-08T02:12:30.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.111+0000 7f1719d9b700 1 -- 172.21.15.119:0/1110426935 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170c00dd10 con 0x7f171c0cbf30 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.111+0000 7f1719d9b700 1 -- 172.21.15.119:0/1110426935 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f170c0196b0 con 0x7f171c0cbf30 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.112+0000 7f1720bff700 1 -- 172.21.15.119:0/1110426935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 msgr2=0x7f171c0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.112+0000 7f1720bff700 1 --2- 172.21.15.119:0/1110426935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 0x7f171c0ae410 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f170c00bc80 tx=0x7f170c00b4f0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.112+0000 7f1720bff700 1 -- 172.21.15.119:0/1110426935 shutdown_connections 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.112+0000 7f1720bff700 1 --2- 172.21.15.119:0/1110426935 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 0x7f171c0ae410 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.112+0000 7f1720bff700 1 --2- 172.21.15.119:0/1110426935 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.112+0000 7f1720bff700 1 -- 172.21.15.119:0/1110426935 >> 172.21.15.119:0/1110426935 conn(0x7f171c06a320 msgr2=0x7f171c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.112+0000 7f1720bff700 1 -- 172.21.15.119:0/1110426935 shutdown_connections 2023-10-08T02:12:30.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.112+0000 7f1720bff700 1 -- 172.21.15.119:0/1110426935 wait complete. 2023-10-08T02:12:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.113+0000 7f1720bff700 1 Processor -- start 2023-10-08T02:12:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.113+0000 7f1720bff700 1 -- start start 2023-10-08T02:12:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.114+0000 7f1720bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 0x7f171c15f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.114+0000 7f1720bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.114+0000 7f1720bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c15a270 0x7f171c158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.114+0000 7f1720bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c15ba40 con 0x7f171c15a270 2023-10-08T02:12:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.114+0000 7f1720bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c15f810 con 0x7f171c0cbf30 2023-10-08T02:12:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.114+0000 7f1720bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c15ae30 con 0x7f171c0ccf90 2023-10-08T02:12:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f1713fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c15a270 0x7f171c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 0x7f171c15f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f1713fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c15c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:59742/0 (socket says 172.21.15.119:59742) 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f1713fff700 1 -- 172.21.15.119:0/1038324619 learned_addr learned my addr 172.21.15.119:0/1038324619 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171ad9d700 1 -- 172.21.15.119:0/1038324619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 msgr2=0x7f171c15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171ad9d700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c15c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171ad9d700 1 -- 172.21.15.119:0/1038324619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 msgr2=0x7f171c15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171ad9d700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 0x7f171c15f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171ad9d700 1 -- 172.21.15.119:0/1038324619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c00b790 con 0x7f171c15a270 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f1713fff700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171a59c700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 0x7f171c15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.115+0000 7f171ad9d700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c15a270 0x7f171c158550 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f171400c790 tx=0x7f17140015f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:30.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.116+0000 7f17127fc700 1 -- 172.21.15.119:0/1038324619 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714012630 con 0x7f171c15a270 2023-10-08T02:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.116+0000 7f17127fc700 1 -- 172.21.15.119:0/1038324619 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17140127b0 con 0x7f171c15a270 2023-10-08T02:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.116+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c15b110 con 0x7f171c15a270 2023-10-08T02:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.116+0000 7f17127fc700 1 -- 172.21.15.119:0/1038324619 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714012b70 con 0x7f171c15a270 2023-10-08T02:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.116+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c15b6d0 con 0x7f171c15a270 2023-10-08T02:12:30.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.117+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c15bbf0 con 0x7f171c15a270 2023-10-08T02:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.118+0000 7f17127fc700 1 -- 172.21.15.119:0/1038324619 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1714003030 con 0x7f171c15a270 2023-10-08T02:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.118+0000 7f17127fc700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f16fc066f30 0x7f16fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.118+0000 7f17127fc700 1 -- 172.21.15.119:0/1038324619 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f17140844d0 con 0x7f171c15a270 2023-10-08T02:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.119+0000 7f171a59c700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f16fc066f30 0x7f16fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.119+0000 7f171a59c700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f16fc066f30 0x7f16fc0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f170c00bc80 tx=0x7f170c019000 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:30.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.122+0000 7f17127fc700 1 -- 172.21.15.119:0/1038324619 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f171c15bbf0 con 0x7f171c15a270 2023-10-08T02:12:30.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.309+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f171c04c3c0 con 0x7f16fc066f30 2023-10-08T02:12:30.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.317+0000 7f17127fc700 1 -- 172.21.15.119:0/1038324619 <== mgr.8967 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278615 (secure 0 0 0) 0x7f171c04c3c0 con 0x7f16fc066f30 2023-10-08T02:12:30.320 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:30.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.321+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f16fc066f30 msgr2=0x7f16fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.321+0000 7f1720bff700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f16fc066f30 0x7f16fc0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f170c00bc80 tx=0x7f170c019000 comp rx=0 tx=0).stop 2023-10-08T02:12:30.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.321+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c15a270 msgr2=0x7f171c158550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.321+0000 7f1720bff700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c15a270 0x7f171c158550 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f171400c790 tx=0x7f17140015f0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.322+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 shutdown_connections 2023-10-08T02:12:30.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.322+0000 7f1720bff700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f16fc066f30 0x7f16fc0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.322+0000 7f1720bff700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c15a270 0x7f171c158550 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.322+0000 7f1720bff700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0cbf30 0x7f171c15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.322+0000 7f1720bff700 1 --2- 172.21.15.119:0/1038324619 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0ccf90 0x7f171c15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.322+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 >> 172.21.15.119:0/1038324619 conn(0x7f171c06a320 msgr2=0x7f171c0c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:30.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.322+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 shutdown_connections 2023-10-08T02:12:30.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.322+0000 7f1720bff700 1 -- 172.21.15.119:0/1038324619 wait complete. 2023-10-08T02:12:30.324 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:12:30.339 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:12:28.684490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2494092,"kb_used_data":17484,"kb_used_omap":0,"kb_used_meta":2476544,"kb_avail":752480628,"statfs":{"total":773094113280,"available":770540163072,"internally_reserved":0,"allocated":17903616,"data_stored":9499959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2535981056},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.594994"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098112+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:03.268132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099113+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:18:03.137966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099808+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:30.649373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.100563+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:46.851648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102497+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:25.895322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099636+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:44.349959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101831+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.351918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099129+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:53.423324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098367+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:26.108797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097986+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:49.675021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098740+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:48:18.805561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.096562+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.992916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095862+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:47.969697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101713+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:54.743852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102722+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:22:47.969697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.100333+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:07:59.396853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098397+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:14.786069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.102471+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:33.551205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098330+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:41.052726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.099253+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:48:15.100827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098187+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:16.271655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097408+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:43.911275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095703+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:01:48.351918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.097439+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:38.305361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101273+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:42.653885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101358+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:02.429294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.101190+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.136401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098790+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:27:03.061501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098706+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:31.610625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095890+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:25.895322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.098402+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:02.557364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:12:24.089566+0000","last_change":"2023-10-08T02:12:24.089566+0000","last_active":"2023-10-08T02:12:24.089566+0000","last_peered":"2023-10-08T02:12:24.089566+0000","last_clean":"2023-10-08T02:12:24.089566+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:24.089566+0000","last_undegraded":"2023-10-08T02:12:24.089566+0000","last_fullsized":"2023-10-08T02:12:24.089566+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:12:23.083202+0000","last_change":"2023-10-08T02:12:23.095826+0000","last_active":"2023-10-08T02:12:23.083202+0000","last_peered":"2023-10-08T02:12:23.083202+0000","last_clean":"2023-10-08T02:12:23.083202+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:12:23.083202+0000","last_undegraded":"2023-10-08T02:12:23.083202+0000","last_fullsized":"2023-10-08T02:12:23.083202+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:10:33.551205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"160'293","reported_seq":911,"reported_epoch":167,"state":"active+clean","last_fresh":"2023-10-08T02:12:20.515876+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:12:20.515876+0000","last_peered":"2023-10-08T02:12:20.515876+0000","last_clean":"2023-10-08T02:12:20.515876+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:12:20.515876+0000","last_undegraded":"2023-10-08T02:12:20.515876+0000","last_fullsized":"2023-10-08T02:12:20.515876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575759,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312192,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059648,"statfs":{"total":96636764160,"available":96317079552,"internally_reserved":0,"allocated":2678784,"data_stored":1627920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.70099999999999996,"15min":0.73099999999999998},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76100000000000001,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.751,"15min":0.75},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.77800000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.73599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81200000000000006,"5min":0.999,"15min":1.0580000000000001},"last":0.64800000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.755,"15min":0.76400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.873,"5min":0.94899999999999995,"15min":1.0600000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.755,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.71299999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.79500000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.875,"5min":1.502,"15min":1.502},"last":0.69799999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.46300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":0.94999999999999996,"15min":0.97799999999999998},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":1.034,"15min":1.034},"last":0.432}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.56799999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":0.76800000000000002},"min":{"1min":0.56999999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.998,"15min":1.0149999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":1.109,"15min":1.109},"last":0.75}]}]},{"osd":6,"up_from":13,"seq":55834575759,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056120,"statfs":{"total":96636764160,"available":96313466880,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.76000000000000001,"15min":0.74199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.569,"15min":1.569},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.74399999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94199999999999995,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.503}]},{"osd":1,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89900000000000002,"5min":1.472,"15min":1.472},"last":0.67600000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.81799999999999995,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79600000000000004,"15min":0.76500000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.495},"max":{"1min":0.91700000000000004,"5min":1.603,"15min":1.603},"last":0.625}]},{"osd":3,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.871,"5min":1.169,"15min":1.169},"last":0.65200000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":1.133,"15min":1.133},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.439,"5min":0.439,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.76900000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82599999999999996,"5min":0.94299999999999995,"15min":1.0680000000000001},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.74099999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74199999999999999,"15min":0.72099999999999997},"min":{"1min":0.503,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90200000000000002,"5min":1.232,"15min":1.232},"last":0.83799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575759,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307516,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064324,"statfs":{"total":96636764160,"available":96321867776,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.497},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.603,"15min":1.603},"last":0.434}]},{"osd":2,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.497,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.60199999999999998,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.53900000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.751,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.70799999999999996,"5min":3.698,"15min":3.698},"last":0.60599999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86399999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95999999999999996,"5min":1.734,"15min":1.734},"last":0.77400000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.53600000000000003},"max":{"1min":1.0489999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91600000000000004,"5min":1.403,"15min":1.403},"last":0.86299999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.61299999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0149999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.573,"15min":1.573},"last":0.72799999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.875,"15min":0.86499999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.58999999999999997},"max":{"1min":0.99399999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.67700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575758,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308096,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063744,"statfs":{"total":96636764160,"available":96321273856,"internally_reserved":0,"allocated":2678784,"data_stored":1627920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.82899999999999996,"15min":0.83999999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.315}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58299999999999996,"5min":0.70699999999999996,"15min":0.746},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68500000000000005,"5min":0.88300000000000001,"15min":1.325},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.42399999999999999,"5min":0.41199999999999998,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.5},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72399999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54700000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.99399999999999999,"5min":1.052,"15min":1.24},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0009999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.92900000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0489999999999999,"5min":1.327,"15min":1.327},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.878,"15min":0.879},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.653,"15min":1.653},"last":0.94399999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83599999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"max":{"1min":1.0209999999999999,"5min":1.577,"15min":1.577},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58099999999999996},"max":{"1min":0.97699999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.96499999999999997}]},{"osd":7,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.84799999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.129,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.89000000000000001,"15min":0.88200000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0720000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.97999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311624,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060216,"statfs":{"total":96636764160,"available":96317661184,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65500000000000003,"5min":1.27,"15min":1.27},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.45100000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65600000000000003,"5min":0.79500000000000004,"15min":0.81699999999999995},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":0.69699999999999995,"5min":1.651,"15min":1.651},"last":0.53600000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.39400000000000002},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":0.55200000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81399999999999995,"15min":0.81000000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.55100000000000005},"max":{"1min":1.0209999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53400000000000003},"max":{"1min":0.93799999999999994,"5min":3.004,"15min":3.004},"last":0.64700000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.86799999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.046,"5min":3.145,"15min":3.145},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0860000000000001,"5min":1.847,"15min":1.847},"last":0.72399999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.57699999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4950000000000001,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.623,"5min":0.60199999999999998,"15min":0.58599999999999997},"max":{"1min":1.0580000000000001,"5min":1.331,"15min":1.331},"last":0.76100000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.88700000000000001,"15min":0.85999999999999999},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":2.899,"15min":2.899},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.86599999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"max":{"1min":1.4570000000000001,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.754}]}]},{"osd":3,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315720,"kb_used_data":2048,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056120,"statfs":{"total":96636764160,"available":96313466880,"internally_reserved":0,"allocated":2097152,"data_stored":1043821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.44800000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47999999999999998,"15min":0.48299999999999998},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.54700000000000004,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.432},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89500000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.48999999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.629,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83699999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56699999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.214},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":0.998,"5min":1.0680000000000001,"15min":1.0900000000000001},"last":0.628}]},{"osd":5,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.77100000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.876,"5min":1.0589999999999999,"15min":1.125},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.64200000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.01,"5min":1.403,"15min":1.403},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.81399999999999995,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.55600000000000005},"max":{"1min":0.90700000000000003,"5min":1.492,"15min":1.492},"last":0.61299999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.83799999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93799999999999994,"5min":1.341,"15min":1.341},"last":0.71499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575758,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311612,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060228,"statfs":{"total":96636764160,"available":96317673472,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.443,"5min":0.443,"15min":0.40400000000000003},"max":{"1min":0.90100000000000002,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.747,"15min":0.73199999999999998},"min":{"1min":0.47899999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0009999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.53000000000000003}]},{"osd":1,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.435},"max":{"1min":0.94099999999999995,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.96499999999999997,"5min":1.1000000000000001,"15min":1.52},"last":0.622}]},{"osd":2,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77400000000000002,"15min":0.77100000000000002},"min":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.96299999999999997,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.60999999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.90800000000000003,"5min":1.1579999999999999,"15min":1.403},"last":0.69599999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.78400000000000003,"15min":0.77700000000000002},"min":{"1min":0.56799999999999995,"5min":0.51400000000000001,"15min":0.50700000000000001},"max":{"1min":0.92500000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80900000000000005,"15min":0.80300000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47999999999999998},"max":{"1min":0.90500000000000003,"5min":1.208,"15min":1.5069999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.437,"5min":0.36399999999999999,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":1.05,"15min":1.05},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58799999999999997,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93400000000000005,"5min":1.038,"15min":1.1559999999999999},"last":0.63600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66100000000000003,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.016},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.66800000000000004,"15min":0.66300000000000003},"min":{"1min":0.46600000000000003,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.51000000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:11:24 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.54600000000000004,"5min":0.48299999999999998,"15min":0.40600000000000003},"max":{"1min":0.874,"5min":1.0680000000000001,"15min":1.129},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72499999999999998,"15min":0.71999999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.48799999999999999},"max":{"1min":0.91800000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575758,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311612,"kb_used_data":2036,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060228,"statfs":{"total":96636764160,"available":96317673472,"internally_reserved":0,"allocated":2084864,"data_stored":1037552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.111,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.76700000000000002,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.438},"max":{"1min":1.0009999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.64500000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":0.77800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42699999999999999},"max":{"1min":0.95399999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55700000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.77600000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.499},"max":{"1min":0.995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.751,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.50900000000000001},"max":{"1min":0.97699999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80700000000000005,"15min":0.77900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96399999999999997,"5min":1.0620000000000001,"15min":1.0669999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0529999999999999,"15min":1.091},"last":0.53300000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.441},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.115},"last":0.872}]},{"osd":6,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0629999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.077},"last":0.66800000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0189999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.82299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:12:30.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3f is not active+clean 2023-10-08T02:12:30.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3e is not active+clean 2023-10-08T02:12:30.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3d is not active+clean 2023-10-08T02:12:30.346 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.346 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3c is not active+clean 2023-10-08T02:12:30.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3b is not active+clean 2023-10-08T02:12:30.347 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.347 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3a is not active+clean 2023-10-08T02:12:30.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.39 is not active+clean 2023-10-08T02:12:30.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.38 is not active+clean 2023-10-08T02:12:30.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.348 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.37 is not active+clean 2023-10-08T02:12:30.348 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.36 is not active+clean 2023-10-08T02:12:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.35 is not active+clean 2023-10-08T02:12:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.34 is not active+clean 2023-10-08T02:12:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.33 is not active+clean 2023-10-08T02:12:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.32 is not active+clean 2023-10-08T02:12:30.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.31 is not active+clean 2023-10-08T02:12:30.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.30 is not active+clean 2023-10-08T02:12:30.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.350 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2f is not active+clean 2023-10-08T02:12:30.350 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2e is not active+clean 2023-10-08T02:12:30.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2d is not active+clean 2023-10-08T02:12:30.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.351 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2c is not active+clean 2023-10-08T02:12:30.351 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2b is not active+clean 2023-10-08T02:12:30.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2a is not active+clean 2023-10-08T02:12:30.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.29 is not active+clean 2023-10-08T02:12:30.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.352 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.28 is not active+clean 2023-10-08T02:12:30.352 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.27 is not active+clean 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.26 is not active+clean 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.25 is not active+clean 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.24 is not active+clean 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.353 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.23 is not active+clean 2023-10-08T02:12:30.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.22 is not active+clean 2023-10-08T02:12:30.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.21 is not active+clean 2023-10-08T02:12:30.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.354 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.20 is not active+clean 2023-10-08T02:12:30.354 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1e is not active+clean 2023-10-08T02:12:30.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1f is not active+clean 2023-10-08T02:12:30.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098112+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:57:03.268132+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:12:30.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2023-10-08T02:12:30.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.355 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1e is not active+clean 2023-10-08T02:12:30.355 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.099113+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:18:03.137966+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:12:30.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2023-10-08T02:12:30.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1d is not active+clean 2023-10-08T02:12:30.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.099808+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:03:30.649373+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:12:30.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1d is not active+clean 2023-10-08T02:12:30.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.356 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1c is not active+clean 2023-10-08T02:12:30.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.100563+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:57:46.851648+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:12:30.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2023-10-08T02:12:30.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1b is not active+clean 2023-10-08T02:12:30.357 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.102497+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:10:25.895322+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T02:12:30.357 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2023-10-08T02:12:30.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1a is not active+clean 2023-10-08T02:12:30.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.099636+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:28:44.349959+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:12:30.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2023-10-08T02:12:30.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.358 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.19 is not active+clean 2023-10-08T02:12:30.358 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.101831+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:01:48.351918+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T02:12:30.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.19 is not active+clean 2023-10-08T02:12:30.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.18 is not active+clean 2023-10-08T02:12:30.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.099129+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:56:53.423324+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:12:30.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.16 is not active+clean 2023-10-08T02:12:30.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.359 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.17 is not active+clean 2023-10-08T02:12:30.359 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098367+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:40:26.108797+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:12:30.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.17 is not active+clean 2023-10-08T02:12:30.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.16 is not active+clean 2023-10-08T02:12:30.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.097986+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:02:49.675021+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:12:30.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2023-10-08T02:12:30.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.360 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.14 is not active+clean 2023-10-08T02:12:30.360 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098740+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:48:18.805561+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:12:30.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.12 is not active+clean 2023-10-08T02:12:30.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.13 is not active+clean 2023-10-08T02:12:30.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.096562+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:41:26.992916+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:12:30.361 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2023-10-08T02:12:30.361 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.12 is not active+clean 2023-10-08T02:12:30.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.095862+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:22:47.969697+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:12:30.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.10 is not active+clean 2023-10-08T02:12:30.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.11 is not active+clean 2023-10-08T02:12:30.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.101713+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:00:54.743852+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:12:30.362 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.11 is not active+clean 2023-10-08T02:12:30.362 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.10 is not active+clean 2023-10-08T02:12:30.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.102722+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:22:47.969697+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T02:12:30.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.e is not active+clean 2023-10-08T02:12:30.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.f is not active+clean 2023-10-08T02:12:30.363 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.100333+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:07:59.396853+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:12:30.363 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2023-10-08T02:12:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.e is not active+clean 2023-10-08T02:12:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098397+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:47:14.786069+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:12:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.c is not active+clean 2023-10-08T02:12:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.d is not active+clean 2023-10-08T02:12:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.102471+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:10:33.551205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T02:12:30.364 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2023-10-08T02:12:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.c is not active+clean 2023-10-08T02:12:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098330+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:12:41.052726+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:12:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2023-10-08T02:12:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.b is not active+clean 2023-10-08T02:12:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.099253+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:48:15.100827+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:12:30.365 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2023-10-08T02:12:30.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.a is not active+clean 2023-10-08T02:12:30.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098187+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:36:16.271655+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:12:30.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2023-10-08T02:12:30.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.366 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.9 is not active+clean 2023-10-08T02:12:30.366 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.097408+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:40:43.911275+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:12:30.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2023-10-08T02:12:30.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.8 is not active+clean 2023-10-08T02:12:30.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.095703+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:01:48.351918+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:12:30.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.6 is not active+clean 2023-10-08T02:12:30.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.367 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.7 is not active+clean 2023-10-08T02:12:30.367 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.097439+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:21:38.305361+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.7 is not active+clean 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.6 is not active+clean 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.101273+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:25:42.653885+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.5 is not active+clean 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.101358+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:20:02.429294+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2023-10-08T02:12:30.368 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.4 is not active+clean 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.101190+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T11:33:36.136401+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.3 is not active+clean 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098790+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:27:03.061501+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3 is not active+clean 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.369 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.2 is not active+clean 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098706+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:11:31.610625+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.0 is not active+clean 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.1 is not active+clean 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.095890+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:10:25.895322+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1 is not active+clean 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.0 is not active+clean 2023-10-08T02:12:30.370 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.098402+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:05:02.557364+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:12:30.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2023-10-08T02:12:30.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:12:24.089566+0000', 'last_change': '2023-10-08T02:12:24.089566+0000', 'last_active': '2023-10-08T02:12:24.089566+0000', 'last_peered': '2023-10-08T02:12:24.089566+0000', 'last_clean': '2023-10-08T02:12:24.089566+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:24.089566+0000', 'last_undegraded': '2023-10-08T02:12:24.089566+0000', 'last_fullsized': '2023-10-08T02:12:24.089566+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:24.089566+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:12:30.371 INFO:tasks.cephfs.filesystem.ceph_manager:PG 20.15 is not active+clean 2023-10-08T02:12:30.371 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '20.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:12:23.083202+0000', 'last_change': '2023-10-08T02:12:23.095826+0000', 'last_active': '2023-10-08T02:12:23.083202+0000', 'last_peered': '2023-10-08T02:12:23.083202+0000', 'last_clean': '2023-10-08T02:12:23.083202+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:12:23.083202+0000', 'last_undegraded': '2023-10-08T02:12:23.083202+0000', 'last_fullsized': '2023-10-08T02:12:23.083202+0000', 'mapping_epoch': 168, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 168, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'last_clean_scrub_stamp': '2023-10-08T02:12:23.083202+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:10:33.551205+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:12:30.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T02:12:30.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.560+0000 7fd1007ff700 1 Processor -- start 2023-10-08T02:12:30.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.560+0000 7fd1007ff700 1 -- start start 2023-10-08T02:12:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd1007ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd0f8156510 0x7fd0f80ac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd1007ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80acd40 0x7fd0f81568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd1007ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0f8153f10 con 0x7fd0f8156e20 2023-10-08T02:12:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd1007ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0f80ad9d0 con 0x7fd0f8156510 2023-10-08T02:12:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd1007ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f80aed00 con 0x7fd0f80acd40 2023-10-08T02:12:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd0fed9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd0f8156e20 0x7fd0f80afc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59538/0 (socket says 172.21.15.119:59538) 2023-10-08T02:12:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd0fed9c700 1 -- 172.21.15.119:0/3856645474 learned_addr learned my addr 172.21.15.119:0/3856645474 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:30.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd0fe59b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd0f8156510 0x7fd0f80ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd0fdd9a700 1 --2- 172.21.15.119:0/3856645474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80acd40 0x7fd0f81568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd0fd599700 1 -- 172.21.15.119:0/3856645474 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68018638 0 0) 0x7fd0f8153f10 con 0x7fd0f8156e20 2023-10-08T02:12:30.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.562+0000 7fd0fd599700 1 -- 172.21.15.119:0/3856645474 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e4003570 con 0x7fd0f8156e20 2023-10-08T02:12:30.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.563+0000 7fd0fe59b700 1 -- 172.21.15.119:0/3856645474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80acd40 msgr2=0x7fd0f81568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:30.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.563+0000 7fd0fe59b700 1 --2- 172.21.15.119:0/3856645474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80acd40 0x7fd0f81568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.563+0000 7fd0fe59b700 1 -- 172.21.15.119:0/3856645474 >> v1:172.21.15.119:6789/0 conn(0x7fd0f8156e20 legacy=0x7fd0f80afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.563+0000 7fd0fe59b700 1 -- 172.21.15.119:0/3856645474 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f80af800 con 0x7fd0f8156510 2023-10-08T02:12:30.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.563+0000 7fd0fe59b700 1 --2- 172.21.15.119:0/3856645474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f8156510 0x7fd0f80ac800 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098e0 tx=0x7fd0f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecf439bf782c059 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:30.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.563+0000 7fd0fd599700 1 -- 172.21.15.119:0/3856645474 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011630 con 0x7fd0f8156510 2023-10-08T02:12:30.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.563+0000 7fd0fd599700 1 -- 172.21.15.119:0/3856645474 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f4011790 con 0x7fd0f8156510 2023-10-08T02:12:30.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.563+0000 7fd0fd599700 1 -- 172.21.15.119:0/3856645474 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011b30 con 0x7fd0f8156510 2023-10-08T02:12:30.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.564+0000 7fd1007ff700 1 -- 172.21.15.119:0/3856645474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f8156510 msgr2=0x7fd0f80ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.564+0000 7fd1007ff700 1 --2- 172.21.15.119:0/3856645474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f8156510 0x7fd0f80ac800 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fd0f40098e0 tx=0x7fd0f40091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.564+0000 7fd1007ff700 1 -- 172.21.15.119:0/3856645474 shutdown_connections 2023-10-08T02:12:30.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.564+0000 7fd1007ff700 1 --2- 172.21.15.119:0/3856645474 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f8156510 0x7fd0f80ac800 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.564+0000 7fd1007ff700 1 --2- 172.21.15.119:0/3856645474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f80acd40 0x7fd0f81568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.564+0000 7fd1007ff700 1 -- 172.21.15.119:0/3856645474 >> 172.21.15.119:0/3856645474 conn(0x7fd0f806a310 msgr2=0x7fd0f80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:30.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.564+0000 7fd1007ff700 1 -- 172.21.15.119:0/3856645474 shutdown_connections 2023-10-08T02:12:30.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.564+0000 7fd1007ff700 1 -- 172.21.15.119:0/3856645474 wait complete. 2023-10-08T02:12:30.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.565+0000 7fd1007ff700 1 Processor -- start 2023-10-08T02:12:30.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.565+0000 7fd1007ff700 1 -- start start 2023-10-08T02:12:30.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.566+0000 7fd1007ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f80acd40 0x7fd0f814b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.566+0000 7fd1007ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f8156510 0x7fd0f80c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.566+0000 7fd1007ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f8156e20 0x7fd0f8153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.566+0000 7fd1007ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f8169c30 con 0x7fd0f80acd40 2023-10-08T02:12:30.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.566+0000 7fd1007ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f806e340 con 0x7fd0f8156510 2023-10-08T02:12:30.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.566+0000 7fd1007ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f806e620 con 0x7fd0f8156e20 2023-10-08T02:12:30.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.566+0000 7fd0fe59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f80acd40 0x7fd0f814b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fe59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f80acd40 0x7fd0f814b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44114/0 (socket says 172.21.15.119:44114) 2023-10-08T02:12:30.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fe59b700 1 -- 172.21.15.119:0/2043041547 learned_addr learned my addr 172.21.15.119:0/2043041547 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:30.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fed9c700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f8156e20 0x7fd0f8153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fdd9a700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f8156510 0x7fd0f80c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fe59b700 1 -- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f8156e20 msgr2=0x7fd0f8153f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fe59b700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f8156e20 0x7fd0f8153f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fe59b700 1 -- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f8156510 msgr2=0x7fd0f80c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fe59b700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f8156510 0x7fd0f80c12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fe59b700 1 -- 172.21.15.119:0/2043041547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009420 con 0x7fd0f80acd40 2023-10-08T02:12:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fed9c700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f8156e20 0x7fd0f8153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0fe59b700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f80acd40 0x7fd0f814b140 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80cd610 tx=0x7fd0f4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0ee7fc700 1 -- 172.21.15.119:0/2043041547 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4011b10 con 0x7fd0f80acd40 2023-10-08T02:12:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0ee7fc700 1 -- 172.21.15.119:0/2043041547 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f40253f0 con 0x7fd0f80acd40 2023-10-08T02:12:30.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd0ee7fc700 1 -- 172.21.15.119:0/2043041547 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f4025790 con 0x7fd0f80acd40 2023-10-08T02:12:30.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.567+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f806e900 con 0x7fd0f80acd40 2023-10-08T02:12:30.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.568+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0f80c1810 con 0x7fd0f80acd40 2023-10-08T02:12:30.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.568+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0f806ea90 con 0x7fd0f80acd40 2023-10-08T02:12:30.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.569+0000 7fd0ee7fc700 1 -- 172.21.15.119:0/2043041547 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd0f4025b80 con 0x7fd0f80acd40 2023-10-08T02:12:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.570+0000 7fd0ee7fc700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.570+0000 7fd0ee7fc700 1 -- 172.21.15.119:0/2043041547 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fd0f4085fe0 con 0x7fd0f80acd40 2023-10-08T02:12:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.570+0000 7fd0fdd9a700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.571+0000 7fd0fdd9a700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd0dc066f30 0x7fd0dc0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8006f00 tx=0x7fd0e8008040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.574+0000 7fd0ee7fc700 1 -- 172.21.15.119:0/2043041547 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0f4053020 con 0x7fd0f80acd40 2023-10-08T02:12:30.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.841+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd0f804c3c0 con 0x7fd0f80acd40 2023-10-08T02:12:30.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.842+0000 7fd0ee7fc700 1 -- 172.21.15.119:0/2043041547 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1116 (secure 0 0 0) 0x7fd0f4056640 con 0x7fd0f80acd40 2023-10-08T02:12:30.842 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:30.842 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4627,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":171,"num_osds":8,"num_up_osds":8,"osd_up_since":1696726535,"num_in_osds":8,"osd_in_since":1696726526,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2553950208,"bytes_avail":770540163072,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":79,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9150}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2023-10-08T02:01:45.128305+0000","services":{}},"progress_events":{}} 2023-10-08T02:12:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd0dc066f30 msgr2=0x7fd0dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd0dc066f30 0x7fd0dc0693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8006f00 tx=0x7fd0e8008040 comp rx=0 tx=0).stop 2023-10-08T02:12:30.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f80acd40 msgr2=0x7fd0f814b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f80acd40 0x7fd0f814b140 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fd0f80cd610 tx=0x7fd0f4017000 comp rx=0 tx=0).stop 2023-10-08T02:12:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 shutdown_connections 2023-10-08T02:12:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fd0dc066f30 0x7fd0dc0693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd0f8156e20 0x7fd0f8153f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0f80acd40 0x7fd0f814b140 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 --2- 172.21.15.119:0/2043041547 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd0f8156510 0x7fd0f80c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 >> 172.21.15.119:0/2043041547 conn(0x7fd0f806a310 msgr2=0x7fd0f80b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:30.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.844+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 shutdown_connections 2023-10-08T02:12:30.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:30.845+0000 7fd1007ff700 1 -- 172.21.15.119:0/2043041547 wait complete. 2023-10-08T02:12:30.862 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T02:12:30.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:12:31.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.043+0000 7f0abe9ff700 1 Processor -- start 2023-10-08T02:12:31.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.043+0000 7f0abe9ff700 1 -- start start 2023-10-08T02:12:31.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.044+0000 7f0abe9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0ab8159720 0x7f0ab8159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:31.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.044+0000 7f0abe9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80ad250 0x7f0ab80ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:31.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.044+0000 7f0abe9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab80adbf0 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.044+0000 7f0abe9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab80ab980 con 0x7f0ab8159720 2023-10-08T02:12:31.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.044+0000 7f0abe9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab80ac020 con 0x7f0ab80ad250 2023-10-08T02:12:31.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0ab7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0ab8159720 0x7f0ab8159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:31.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0abcf9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80ad250 0x7f0ab80ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:31.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0abcf9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80ad250 0x7f0ab80ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:59760/0 (socket says 172.21.15.119:59760) 2023-10-08T02:12:31.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0ab7fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f0ab8159720 0x7f0ab8159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54992/0 (socket says 172.21.15.119:54992) 2023-10-08T02:12:31.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0abcf9c700 1 -- 172.21.15.119:0/2282522947 learned_addr learned my addr 172.21.15.119:0/2282522947 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:31.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0abcf9c700 1 -- 172.21.15.119:0/2282522947 >> v2:172.21.15.184:3300/0 conn(0x7f0ab8159720 msgr2=0x7f0ab8159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:31.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0abcf9c700 1 --2- 172.21.15.119:0/2282522947 >> v2:172.21.15.184:3300/0 conn(0x7f0ab8159720 0x7f0ab8159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0abcf9c700 1 -- 172.21.15.119:0/2282522947 >> v1:172.21.15.184:6789/0 conn(0x7f0ab80ab4e0 legacy=0x7f0ab80a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0abcf9c700 1 -- 172.21.15.119:0/2282522947 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80aa440 con 0x7f0ab80ad250 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.045+0000 7f0ab7fff700 1 --2- 172.21.15.119:0/2282522947 >> v2:172.21.15.184:3300/0 conn(0x7f0ab8159720 0x7f0ab8159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0abcf9c700 1 --2- 172.21.15.119:0/2282522947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80ad250 0x7f0ab80ad6b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009f20 tx=0x7f0ab0009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2f2b57acd08d93e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0ab6ffd700 1 -- 172.21.15.119:0/2282522947 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000e070 con 0x7f0ab80ad250 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0ab6ffd700 1 -- 172.21.15.119:0/2282522947 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab000b9d0 con 0x7f0ab80ad250 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0ab6ffd700 1 -- 172.21.15.119:0/2282522947 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00092e0 con 0x7f0ab80ad250 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2282522947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80ad250 msgr2=0x7f0ab80ad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2282522947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80ad250 0x7f0ab80ad6b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009f20 tx=0x7f0ab0009840 comp rx=0 tx=0).stop 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2282522947 shutdown_connections 2023-10-08T02:12:31.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2282522947 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab80ad250 0x7f0ab80ad6b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2282522947 >> v2:172.21.15.184:3300/0 conn(0x7f0ab8159720 0x7f0ab8159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.046+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2282522947 >> 172.21.15.119:0/2282522947 conn(0x7f0ab806a320 msgr2=0x7f0ab80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:31.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.047+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2282522947 shutdown_connections 2023-10-08T02:12:31.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.047+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2282522947 wait complete. 2023-10-08T02:12:31.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.048+0000 7f0abe9ff700 1 Processor -- start 2023-10-08T02:12:31.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.048+0000 7f0abe9ff700 1 -- start start 2023-10-08T02:12:31.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0abe9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80ab4e0 0x7f0ab80b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0abe9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab80ad250 0x7f0ab80b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0abe9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab8159720 0x7f0ab8156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0abe9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab815a810 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0abe9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8159c00 con 0x7f0ab80ad250 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0abe9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab8159ee0 con 0x7f0ab8159720 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0ab7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80ab4e0 0x7f0ab80b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0ab7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80ab4e0 0x7f0ab80b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44116/0 (socket says 172.21.15.119:44116) 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0ab7fff700 1 -- 172.21.15.119:0/2688348498 learned_addr learned my addr 172.21.15.119:0/2688348498 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0ab77fe700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab80ad250 0x7f0ab80b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.049+0000 7f0abcf9c700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab8159720 0x7f0ab8156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:31.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0ab7fff700 1 -- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab8159720 msgr2=0x7f0ab8156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0ab7fff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab8159720 0x7f0ab8156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0ab7fff700 1 -- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab80ad250 msgr2=0x7f0ab80b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0ab7fff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab80ad250 0x7f0ab80b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0ab7fff700 1 -- 172.21.15.119:0/2688348498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8009420 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0abcf9c700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab8159720 0x7f0ab8156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0ab7fff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80ab4e0 0x7f0ab80b0f60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800ec80 tx=0x7f0aa80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0a9ffff700 1 -- 172.21.15.119:0/2688348498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800c850 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0a9ffff700 1 -- 172.21.15.119:0/2688348498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa800c9b0 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0a9ffff700 1 -- 172.21.15.119:0/2688348498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa800cd30 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab0009ae0 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.050+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab815a4a0 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.051+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab80ae890 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.052+0000 7f0a9ffff700 1 -- 172.21.15.119:0/2688348498 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0aa8013680 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.053+0000 7f0a9ffff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0aa4066f30 0x7f0aa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:31.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.053+0000 7f0a9ffff700 1 -- 172.21.15.119:0/2688348498 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f0aa8084ab0 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.053+0000 7f0ab77fe700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0aa4066f30 0x7f0aa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:31.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.054+0000 7f0ab77fe700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0aa4066f30 0x7f0aa40693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0ab81582f0 tx=0x7f0aac007ec0 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:31.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.057+0000 7f0a9ffff700 1 -- 172.21.15.119:0/2688348498 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0aa8051ab0 con 0x7f0ab80ab4e0 2023-10-08T02:12:31.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.246+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ab815a950 con 0x7f0aa4066f30 2023-10-08T02:12:31.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.254+0000 7f0a9ffff700 1 -- 172.21.15.119:0/2688348498 <== mgr.8967 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289600 (secure 0 0 0) 0x7f0aac002480 con 0x7f0aa4066f30 2023-10-08T02:12:31.257 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0aa4066f30 msgr2=0x7f0aa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0aa4066f30 0x7f0aa40693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0ab81582f0 tx=0x7f0aac007ec0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80ab4e0 msgr2=0x7f0ab80b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80ab4e0 0x7f0ab80b0f60 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800ec80 tx=0x7f0aa80091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 shutdown_connections 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f0aa4066f30 0x7f0aa40693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0ab8159720 0x7f0ab8156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ab80ab4e0 0x7f0ab80b0f60 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 --2- 172.21.15.119:0/2688348498 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0ab80ad250 0x7f0ab80b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:31.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 >> 172.21.15.119:0/2688348498 conn(0x7f0ab806a320 msgr2=0x7f0ab815cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:31.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 shutdown_connections 2023-10-08T02:12:31.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:31.258+0000 7f0abe9ff700 1 -- 172.21.15.119:0/2688348498 wait complete. 2023-10-08T02:12:31.261 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:12:31.276 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T02:12:30.684834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2478120,"kb_used_data":17896,"kb_used_omap":0,"kb_used_meta":2460160,"kb_avail":752496600,"statfs":{"total":773094113280,"available":770556518400,"internally_reserved":0,"allocated":18325504,"data_stored":9701564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2519203840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.585713"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129836+0000","last_change":"2023-10-08T02:12:25.127935+0000","last_active":"2023-10-08T02:12:25.129836+0000","last_peered":"2023-10-08T02:12:25.129836+0000","last_clean":"2023-10-08T02:12:25.129836+0000","last_became_active":"2023-10-08T02:12:25.127666+0000","last_became_peered":"2023-10-08T02:12:25.127666+0000","last_unstale":"2023-10-08T02:12:25.129836+0000","last_undegraded":"2023-10-08T02:12:25.129836+0000","last_fullsized":"2023-10-08T02:12:25.129836+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:54.429689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.117989+0000","last_change":"2023-10-08T02:12:25.114785+0000","last_active":"2023-10-08T02:12:25.117989+0000","last_peered":"2023-10-08T02:12:25.117989+0000","last_clean":"2023-10-08T02:12:25.117989+0000","last_became_active":"2023-10-08T02:12:25.114477+0000","last_became_peered":"2023-10-08T02:12:25.114477+0000","last_unstale":"2023-10-08T02:12:25.117989+0000","last_undegraded":"2023-10-08T02:12:25.117989+0000","last_fullsized":"2023-10-08T02:12:25.117989+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:29.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122727+0000","last_change":"2023-10-08T02:12:25.116047+0000","last_active":"2023-10-08T02:12:25.122727+0000","last_peered":"2023-10-08T02:12:25.122727+0000","last_clean":"2023-10-08T02:12:25.122727+0000","last_became_active":"2023-10-08T02:12:25.115767+0000","last_became_peered":"2023-10-08T02:12:25.115767+0000","last_unstale":"2023-10-08T02:12:25.122727+0000","last_undegraded":"2023-10-08T02:12:25.122727+0000","last_fullsized":"2023-10-08T02:12:25.122727+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:37.774429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138480+0000","last_change":"2023-10-08T02:12:25.136222+0000","last_active":"2023-10-08T02:12:25.138480+0000","last_peered":"2023-10-08T02:12:25.138480+0000","last_clean":"2023-10-08T02:12:25.138480+0000","last_became_active":"2023-10-08T02:12:25.135973+0000","last_became_peered":"2023-10-08T02:12:25.135973+0000","last_unstale":"2023-10-08T02:12:25.138480+0000","last_undegraded":"2023-10-08T02:12:25.138480+0000","last_fullsized":"2023-10-08T02:12:25.138480+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:37.031644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.131634+0000","last_change":"2023-10-08T02:12:25.128861+0000","last_active":"2023-10-08T02:12:25.131634+0000","last_peered":"2023-10-08T02:12:25.131634+0000","last_clean":"2023-10-08T02:12:25.131634+0000","last_became_active":"2023-10-08T02:12:25.128427+0000","last_became_peered":"2023-10-08T02:12:25.128427+0000","last_unstale":"2023-10-08T02:12:25.131634+0000","last_undegraded":"2023-10-08T02:12:25.131634+0000","last_fullsized":"2023-10-08T02:12:25.131634+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:17:09.195946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122258+0000","last_change":"2023-10-08T02:12:25.120777+0000","last_active":"2023-10-08T02:12:25.122258+0000","last_peered":"2023-10-08T02:12:25.122258+0000","last_clean":"2023-10-08T02:12:25.122258+0000","last_became_active":"2023-10-08T02:12:25.120526+0000","last_became_peered":"2023-10-08T02:12:25.120526+0000","last_unstale":"2023-10-08T02:12:25.122258+0000","last_undegraded":"2023-10-08T02:12:25.122258+0000","last_fullsized":"2023-10-08T02:12:25.122258+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:27.746043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.124600+0000","last_change":"2023-10-08T02:12:25.123401+0000","last_active":"2023-10-08T02:12:25.124600+0000","last_peered":"2023-10-08T02:12:25.124600+0000","last_clean":"2023-10-08T02:12:25.124600+0000","last_became_active":"2023-10-08T02:12:25.123126+0000","last_became_peered":"2023-10-08T02:12:25.123126+0000","last_unstale":"2023-10-08T02:12:25.124600+0000","last_undegraded":"2023-10-08T02:12:25.124600+0000","last_fullsized":"2023-10-08T02:12:25.124600+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:15:47.715028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.121556+0000","last_change":"2023-10-08T02:12:25.117340+0000","last_active":"2023-10-08T02:12:25.121556+0000","last_peered":"2023-10-08T02:12:25.121556+0000","last_clean":"2023-10-08T02:12:25.121556+0000","last_became_active":"2023-10-08T02:12:25.117038+0000","last_became_peered":"2023-10-08T02:12:25.117038+0000","last_unstale":"2023-10-08T02:12:25.121556+0000","last_undegraded":"2023-10-08T02:12:25.121556+0000","last_fullsized":"2023-10-08T02:12:25.121556+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:35.429110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.124604+0000","last_change":"2023-10-08T02:12:25.119671+0000","last_active":"2023-10-08T02:12:25.124604+0000","last_peered":"2023-10-08T02:12:25.124604+0000","last_clean":"2023-10-08T02:12:25.124604+0000","last_became_active":"2023-10-08T02:12:25.119381+0000","last_became_peered":"2023-10-08T02:12:25.119381+0000","last_unstale":"2023-10-08T02:12:25.124604+0000","last_undegraded":"2023-10-08T02:12:25.124604+0000","last_fullsized":"2023-10-08T02:12:25.124604+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:31.655738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.137145+0000","last_change":"2023-10-08T02:12:25.135795+0000","last_active":"2023-10-08T02:12:25.137145+0000","last_peered":"2023-10-08T02:12:25.137145+0000","last_clean":"2023-10-08T02:12:25.137145+0000","last_became_active":"2023-10-08T02:12:25.135499+0000","last_became_peered":"2023-10-08T02:12:25.135499+0000","last_unstale":"2023-10-08T02:12:25.137145+0000","last_undegraded":"2023-10-08T02:12:25.137145+0000","last_fullsized":"2023-10-08T02:12:25.137145+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:07.222935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.123810+0000","last_change":"2023-10-08T02:12:25.122103+0000","last_active":"2023-10-08T02:12:25.123810+0000","last_peered":"2023-10-08T02:12:25.123810+0000","last_clean":"2023-10-08T02:12:25.123810+0000","last_became_active":"2023-10-08T02:12:25.121849+0000","last_became_peered":"2023-10-08T02:12:25.121849+0000","last_unstale":"2023-10-08T02:12:25.123810+0000","last_undegraded":"2023-10-08T02:12:25.123810+0000","last_fullsized":"2023-10-08T02:12:25.123810+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:17.278020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138982+0000","last_change":"2023-10-08T02:12:25.134916+0000","last_active":"2023-10-08T02:12:25.138982+0000","last_peered":"2023-10-08T02:12:25.138982+0000","last_clean":"2023-10-08T02:12:25.138982+0000","last_became_active":"2023-10-08T02:12:25.134637+0000","last_became_peered":"2023-10-08T02:12:25.134637+0000","last_unstale":"2023-10-08T02:12:25.138982+0000","last_undegraded":"2023-10-08T02:12:25.138982+0000","last_fullsized":"2023-10-08T02:12:25.138982+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:54.776020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.131614+0000","last_change":"2023-10-08T02:12:25.128693+0000","last_active":"2023-10-08T02:12:25.131614+0000","last_peered":"2023-10-08T02:12:25.131614+0000","last_clean":"2023-10-08T02:12:25.131614+0000","last_became_active":"2023-10-08T02:12:25.128023+0000","last_became_peered":"2023-10-08T02:12:25.128023+0000","last_unstale":"2023-10-08T02:12:25.131614+0000","last_undegraded":"2023-10-08T02:12:25.131614+0000","last_fullsized":"2023-10-08T02:12:25.131614+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:39.143538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.133097+0000","last_change":"2023-10-08T02:12:25.129113+0000","last_active":"2023-10-08T02:12:25.133097+0000","last_peered":"2023-10-08T02:12:25.133097+0000","last_clean":"2023-10-08T02:12:25.133097+0000","last_became_active":"2023-10-08T02:12:25.128200+0000","last_became_peered":"2023-10-08T02:12:25.128200+0000","last_unstale":"2023-10-08T02:12:25.133097+0000","last_undegraded":"2023-10-08T02:12:25.133097+0000","last_fullsized":"2023-10-08T02:12:25.133097+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:07.222935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.136403+0000","last_change":"2023-10-08T02:12:25.132973+0000","last_active":"2023-10-08T02:12:25.136403+0000","last_peered":"2023-10-08T02:12:25.136403+0000","last_clean":"2023-10-08T02:12:25.136403+0000","last_became_active":"2023-10-08T02:12:25.132701+0000","last_became_peered":"2023-10-08T02:12:25.132701+0000","last_unstale":"2023-10-08T02:12:25.136403+0000","last_undegraded":"2023-10-08T02:12:25.136403+0000","last_fullsized":"2023-10-08T02:12:25.136403+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:56.240224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126590+0000","last_change":"2023-10-08T02:12:25.120110+0000","last_active":"2023-10-08T02:12:25.126590+0000","last_peered":"2023-10-08T02:12:25.126590+0000","last_clean":"2023-10-08T02:12:25.126590+0000","last_became_active":"2023-10-08T02:12:25.119450+0000","last_became_peered":"2023-10-08T02:12:25.119450+0000","last_unstale":"2023-10-08T02:12:25.126590+0000","last_undegraded":"2023-10-08T02:12:25.126590+0000","last_fullsized":"2023-10-08T02:12:25.126590+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:16.426669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130294+0000","last_change":"2023-10-08T02:12:25.127106+0000","last_active":"2023-10-08T02:12:25.130294+0000","last_peered":"2023-10-08T02:12:25.130294+0000","last_clean":"2023-10-08T02:12:25.130294+0000","last_became_active":"2023-10-08T02:12:25.126851+0000","last_became_peered":"2023-10-08T02:12:25.126851+0000","last_unstale":"2023-10-08T02:12:25.130294+0000","last_undegraded":"2023-10-08T02:12:25.130294+0000","last_fullsized":"2023-10-08T02:12:25.130294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:37.031644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.127219+0000","last_change":"2023-10-08T02:12:25.116110+0000","last_active":"2023-10-08T02:12:25.127219+0000","last_peered":"2023-10-08T02:12:25.127219+0000","last_clean":"2023-10-08T02:12:25.127219+0000","last_became_active":"2023-10-08T02:12:25.115860+0000","last_became_peered":"2023-10-08T02:12:25.115860+0000","last_unstale":"2023-10-08T02:12:25.127219+0000","last_undegraded":"2023-10-08T02:12:25.127219+0000","last_fullsized":"2023-10-08T02:12:25.127219+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:16.426669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.118675+0000","last_change":"2023-10-08T02:12:25.116341+0000","last_active":"2023-10-08T02:12:25.118675+0000","last_peered":"2023-10-08T02:12:25.118675+0000","last_clean":"2023-10-08T02:12:25.118675+0000","last_became_active":"2023-10-08T02:12:25.116073+0000","last_became_peered":"2023-10-08T02:12:25.116073+0000","last_unstale":"2023-10-08T02:12:25.118675+0000","last_undegraded":"2023-10-08T02:12:25.118675+0000","last_fullsized":"2023-10-08T02:12:25.118675+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:32.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126319+0000","last_change":"2023-10-08T02:12:25.119603+0000","last_active":"2023-10-08T02:12:25.126319+0000","last_peered":"2023-10-08T02:12:25.126319+0000","last_clean":"2023-10-08T02:12:25.126319+0000","last_became_active":"2023-10-08T02:12:25.119327+0000","last_became_peered":"2023-10-08T02:12:25.119327+0000","last_unstale":"2023-10-08T02:12:25.126319+0000","last_undegraded":"2023-10-08T02:12:25.126319+0000","last_fullsized":"2023-10-08T02:12:25.126319+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:29:30.293616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.137870+0000","last_change":"2023-10-08T02:12:25.135574+0000","last_active":"2023-10-08T02:12:25.137870+0000","last_peered":"2023-10-08T02:12:25.137870+0000","last_clean":"2023-10-08T02:12:25.137870+0000","last_became_active":"2023-10-08T02:12:25.135270+0000","last_became_peered":"2023-10-08T02:12:25.135270+0000","last_unstale":"2023-10-08T02:12:25.137870+0000","last_undegraded":"2023-10-08T02:12:25.137870+0000","last_fullsized":"2023-10-08T02:12:25.137870+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:29.648405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126297+0000","last_change":"2023-10-08T02:12:25.119570+0000","last_active":"2023-10-08T02:12:25.126297+0000","last_peered":"2023-10-08T02:12:25.126297+0000","last_clean":"2023-10-08T02:12:25.126297+0000","last_became_active":"2023-10-08T02:12:25.119320+0000","last_became_peered":"2023-10-08T02:12:25.119320+0000","last_unstale":"2023-10-08T02:12:25.126297+0000","last_undegraded":"2023-10-08T02:12:25.126297+0000","last_fullsized":"2023-10-08T02:12:25.126297+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:57.502890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.134690+0000","last_change":"2023-10-08T02:12:25.128862+0000","last_active":"2023-10-08T02:12:25.134690+0000","last_peered":"2023-10-08T02:12:25.134690+0000","last_clean":"2023-10-08T02:12:25.134690+0000","last_became_active":"2023-10-08T02:12:25.127885+0000","last_became_peered":"2023-10-08T02:12:25.127885+0000","last_unstale":"2023-10-08T02:12:25.134690+0000","last_undegraded":"2023-10-08T02:12:25.134690+0000","last_fullsized":"2023-10-08T02:12:25.134690+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:29.648405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138830+0000","last_change":"2023-10-08T02:12:25.135989+0000","last_active":"2023-10-08T02:12:25.138830+0000","last_peered":"2023-10-08T02:12:25.138830+0000","last_clean":"2023-10-08T02:12:25.138830+0000","last_became_active":"2023-10-08T02:12:25.135726+0000","last_became_peered":"2023-10-08T02:12:25.135726+0000","last_unstale":"2023-10-08T02:12:25.138830+0000","last_undegraded":"2023-10-08T02:12:25.138830+0000","last_fullsized":"2023-10-08T02:12:25.138830+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:34.173348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129102+0000","last_change":"2023-10-08T02:12:25.121007+0000","last_active":"2023-10-08T02:12:25.129102+0000","last_peered":"2023-10-08T02:12:25.129102+0000","last_clean":"2023-10-08T02:12:25.129102+0000","last_became_active":"2023-10-08T02:12:25.120726+0000","last_became_peered":"2023-10-08T02:12:25.120726+0000","last_unstale":"2023-10-08T02:12:25.129102+0000","last_undegraded":"2023-10-08T02:12:25.129102+0000","last_fullsized":"2023-10-08T02:12:25.129102+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:18.587452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.118594+0000","last_change":"2023-10-08T02:12:25.115621+0000","last_active":"2023-10-08T02:12:25.118594+0000","last_peered":"2023-10-08T02:12:25.118594+0000","last_clean":"2023-10-08T02:12:25.118594+0000","last_became_active":"2023-10-08T02:12:25.115341+0000","last_became_peered":"2023-10-08T02:12:25.115341+0000","last_unstale":"2023-10-08T02:12:25.118594+0000","last_undegraded":"2023-10-08T02:12:25.118594+0000","last_fullsized":"2023-10-08T02:12:25.118594+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:39.311726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.113749+0000","last_change":"2023-10-08T02:12:25.112605+0000","last_active":"2023-10-08T02:12:25.113749+0000","last_peered":"2023-10-08T02:12:25.113749+0000","last_clean":"2023-10-08T02:12:25.113749+0000","last_became_active":"2023-10-08T02:12:25.112334+0000","last_became_peered":"2023-10-08T02:12:25.112334+0000","last_unstale":"2023-10-08T02:12:25.113749+0000","last_undegraded":"2023-10-08T02:12:25.113749+0000","last_fullsized":"2023-10-08T02:12:25.113749+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:37.774429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.113652+0000","last_change":"2023-10-08T02:12:25.111752+0000","last_active":"2023-10-08T02:12:25.113652+0000","last_peered":"2023-10-08T02:12:25.113652+0000","last_clean":"2023-10-08T02:12:25.113652+0000","last_became_active":"2023-10-08T02:12:25.111507+0000","last_became_peered":"2023-10-08T02:12:25.111507+0000","last_unstale":"2023-10-08T02:12:25.113652+0000","last_undegraded":"2023-10-08T02:12:25.113652+0000","last_fullsized":"2023-10-08T02:12:25.113652+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:06.678907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122234+0000","last_change":"2023-10-08T02:12:25.119382+0000","last_active":"2023-10-08T02:12:25.122234+0000","last_peered":"2023-10-08T02:12:25.122234+0000","last_clean":"2023-10-08T02:12:25.122234+0000","last_became_active":"2023-10-08T02:12:25.119143+0000","last_became_peered":"2023-10-08T02:12:25.119143+0000","last_unstale":"2023-10-08T02:12:25.122234+0000","last_undegraded":"2023-10-08T02:12:25.122234+0000","last_fullsized":"2023-10-08T02:12:25.122234+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:18.587452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129236+0000","last_change":"2023-10-08T02:12:25.123913+0000","last_active":"2023-10-08T02:12:25.129236+0000","last_peered":"2023-10-08T02:12:25.129236+0000","last_clean":"2023-10-08T02:12:25.129236+0000","last_became_active":"2023-10-08T02:12:25.123675+0000","last_became_peered":"2023-10-08T02:12:25.123675+0000","last_unstale":"2023-10-08T02:12:25.129236+0000","last_undegraded":"2023-10-08T02:12:25.129236+0000","last_fullsized":"2023-10-08T02:12:25.129236+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:13.345275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.128323+0000","last_change":"2023-10-08T02:12:25.125205+0000","last_active":"2023-10-08T02:12:25.128323+0000","last_peered":"2023-10-08T02:12:25.128323+0000","last_clean":"2023-10-08T02:12:25.128323+0000","last_became_active":"2023-10-08T02:12:25.124966+0000","last_became_peered":"2023-10-08T02:12:25.124966+0000","last_unstale":"2023-10-08T02:12:25.128323+0000","last_undegraded":"2023-10-08T02:12:25.128323+0000","last_fullsized":"2023-10-08T02:12:25.128323+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:54.429689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126606+0000","last_change":"2023-10-08T02:12:25.120295+0000","last_active":"2023-10-08T02:12:25.126606+0000","last_peered":"2023-10-08T02:12:25.126606+0000","last_clean":"2023-10-08T02:12:25.126606+0000","last_became_active":"2023-10-08T02:12:25.119861+0000","last_became_peered":"2023-10-08T02:12:25.119861+0000","last_unstale":"2023-10-08T02:12:25.126606+0000","last_undegraded":"2023-10-08T02:12:25.126606+0000","last_fullsized":"2023-10-08T02:12:25.126606+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:45.356324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.137096+0000","last_change":"2023-10-08T02:12:25.135040+0000","last_active":"2023-10-08T02:12:25.137096+0000","last_peered":"2023-10-08T02:12:25.137096+0000","last_clean":"2023-10-08T02:12:25.137096+0000","last_became_active":"2023-10-08T02:12:25.134323+0000","last_became_peered":"2023-10-08T02:12:25.134323+0000","last_unstale":"2023-10-08T02:12:25.137096+0000","last_undegraded":"2023-10-08T02:12:25.137096+0000","last_fullsized":"2023-10-08T02:12:25.137096+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:52.000775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"170'4","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.139274+0000","last_change":"2023-10-08T02:12:24.129340+0000","last_active":"2023-10-08T02:12:25.139274+0000","last_peered":"2023-10-08T02:12:25.139274+0000","last_clean":"2023-10-08T02:12:25.139274+0000","last_became_active":"2023-10-08T02:12:24.127369+0000","last_became_peered":"2023-10-08T02:12:24.127369+0000","last_unstale":"2023-10-08T02:12:25.139274+0000","last_undegraded":"2023-10-08T02:12:25.139274+0000","last_fullsized":"2023-10-08T02:12:25.139274+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:30.649373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.117579+0000","last_change":"2023-10-08T02:12:25.114794+0000","last_active":"2023-10-08T02:12:25.117579+0000","last_peered":"2023-10-08T02:12:25.117579+0000","last_clean":"2023-10-08T02:12:25.117579+0000","last_became_active":"2023-10-08T02:12:25.114561+0000","last_became_peered":"2023-10-08T02:12:25.114561+0000","last_unstale":"2023-10-08T02:12:25.117579+0000","last_undegraded":"2023-10-08T02:12:25.117579+0000","last_fullsized":"2023-10-08T02:12:25.117579+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:32.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138103+0000","last_change":"2023-10-08T02:12:24.126839+0000","last_active":"2023-10-08T02:12:25.138103+0000","last_peered":"2023-10-08T02:12:25.138103+0000","last_clean":"2023-10-08T02:12:25.138103+0000","last_became_active":"2023-10-08T02:12:24.126581+0000","last_became_peered":"2023-10-08T02:12:24.126581+0000","last_unstale":"2023-10-08T02:12:25.138103+0000","last_undegraded":"2023-10-08T02:12:25.138103+0000","last_fullsized":"2023-10-08T02:12:25.138103+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:29:29.287252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.128272+0000","last_change":"2023-10-08T02:12:25.126693+0000","last_active":"2023-10-08T02:12:25.128272+0000","last_peered":"2023-10-08T02:12:25.128272+0000","last_clean":"2023-10-08T02:12:25.128272+0000","last_became_active":"2023-10-08T02:12:25.126415+0000","last_became_peered":"2023-10-08T02:12:25.126415+0000","last_unstale":"2023-10-08T02:12:25.128272+0000","last_undegraded":"2023-10-08T02:12:25.128272+0000","last_fullsized":"2023-10-08T02:12:25.128272+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:50.681386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141415+0000","last_change":"2023-10-08T02:12:24.120849+0000","last_active":"2023-10-08T02:12:25.141415+0000","last_peered":"2023-10-08T02:12:25.141415+0000","last_clean":"2023-10-08T02:12:25.141415+0000","last_became_active":"2023-10-08T02:12:24.120518+0000","last_became_peered":"2023-10-08T02:12:24.120518+0000","last_unstale":"2023-10-08T02:12:25.141415+0000","last_undegraded":"2023-10-08T02:12:25.141415+0000","last_fullsized":"2023-10-08T02:12:25.141415+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:06.476543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.137056+0000","last_change":"2023-10-08T02:12:25.135441+0000","last_active":"2023-10-08T02:12:25.137056+0000","last_peered":"2023-10-08T02:12:25.137056+0000","last_clean":"2023-10-08T02:12:25.137056+0000","last_became_active":"2023-10-08T02:12:25.135107+0000","last_became_peered":"2023-10-08T02:12:25.135107+0000","last_unstale":"2023-10-08T02:12:25.137056+0000","last_undegraded":"2023-10-08T02:12:25.137056+0000","last_fullsized":"2023-10-08T02:12:25.137056+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:13.345275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141207+0000","last_change":"2023-10-08T02:12:24.117074+0000","last_active":"2023-10-08T02:12:25.141207+0000","last_peered":"2023-10-08T02:12:25.141207+0000","last_clean":"2023-10-08T02:12:25.141207+0000","last_became_active":"2023-10-08T02:12:24.116819+0000","last_became_peered":"2023-10-08T02:12:24.116819+0000","last_unstale":"2023-10-08T02:12:25.141207+0000","last_undegraded":"2023-10-08T02:12:25.141207+0000","last_fullsized":"2023-10-08T02:12:25.141207+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:26.197740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.127655+0000","last_change":"2023-10-08T02:12:25.126125+0000","last_active":"2023-10-08T02:12:25.127655+0000","last_peered":"2023-10-08T02:12:25.127655+0000","last_clean":"2023-10-08T02:12:25.127655+0000","last_became_active":"2023-10-08T02:12:25.125845+0000","last_became_peered":"2023-10-08T02:12:25.125845+0000","last_unstale":"2023-10-08T02:12:25.127655+0000","last_undegraded":"2023-10-08T02:12:25.127655+0000","last_fullsized":"2023-10-08T02:12:25.127655+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:35.429110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.140873+0000","last_change":"2023-10-08T02:12:24.127689+0000","last_active":"2023-10-08T02:12:25.140873+0000","last_peered":"2023-10-08T02:12:25.140873+0000","last_clean":"2023-10-08T02:12:25.140873+0000","last_became_active":"2023-10-08T02:12:24.127406+0000","last_became_peered":"2023-10-08T02:12:24.127406+0000","last_unstale":"2023-10-08T02:12:25.140873+0000","last_undegraded":"2023-10-08T02:12:25.140873+0000","last_fullsized":"2023-10-08T02:12:25.140873+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:42.653885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138267+0000","last_change":"2023-10-08T02:12:25.130873+0000","last_active":"2023-10-08T02:12:25.138267+0000","last_peered":"2023-10-08T02:12:25.138267+0000","last_clean":"2023-10-08T02:12:25.138267+0000","last_became_active":"2023-10-08T02:12:25.130564+0000","last_became_peered":"2023-10-08T02:12:25.130564+0000","last_unstale":"2023-10-08T02:12:25.138267+0000","last_undegraded":"2023-10-08T02:12:25.138267+0000","last_fullsized":"2023-10-08T02:12:25.138267+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:54.776020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141242+0000","last_change":"2023-10-08T02:12:24.134435+0000","last_active":"2023-10-08T02:12:25.141242+0000","last_peered":"2023-10-08T02:12:25.141242+0000","last_clean":"2023-10-08T02:12:25.141242+0000","last_became_active":"2023-10-08T02:12:24.134172+0000","last_became_peered":"2023-10-08T02:12:24.134172+0000","last_unstale":"2023-10-08T02:12:25.141242+0000","last_undegraded":"2023-10-08T02:12:25.141242+0000","last_fullsized":"2023-10-08T02:12:25.141242+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:17.581087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.118043+0000","last_change":"2023-10-08T02:12:25.117306+0000","last_active":"2023-10-08T02:12:25.118043+0000","last_peered":"2023-10-08T02:12:25.118043+0000","last_clean":"2023-10-08T02:12:25.118043+0000","last_became_active":"2023-10-08T02:12:25.117061+0000","last_became_peered":"2023-10-08T02:12:25.117061+0000","last_unstale":"2023-10-08T02:12:25.118043+0000","last_undegraded":"2023-10-08T02:12:25.118043+0000","last_fullsized":"2023-10-08T02:12:25.118043+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:27.746043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.110586+0000","last_change":"2023-10-08T02:12:24.115928+0000","last_active":"2023-10-08T02:12:25.110586+0000","last_peered":"2023-10-08T02:12:25.110586+0000","last_clean":"2023-10-08T02:12:25.110586+0000","last_became_active":"2023-10-08T02:12:24.115667+0000","last_became_peered":"2023-10-08T02:12:24.115667+0000","last_unstale":"2023-10-08T02:12:25.110586+0000","last_undegraded":"2023-10-08T02:12:25.110586+0000","last_fullsized":"2023-10-08T02:12:25.110586+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.992916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.123376+0000","last_change":"2023-10-08T02:12:25.120217+0000","last_active":"2023-10-08T02:12:25.123376+0000","last_peered":"2023-10-08T02:12:25.123376+0000","last_clean":"2023-10-08T02:12:25.123376+0000","last_became_active":"2023-10-08T02:12:25.119954+0000","last_became_peered":"2023-10-08T02:12:25.119954+0000","last_unstale":"2023-10-08T02:12:25.123376+0000","last_undegraded":"2023-10-08T02:12:25.123376+0000","last_fullsized":"2023-10-08T02:12:25.123376+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:47.858013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.111471+0000","last_change":"2023-10-08T02:12:24.130840+0000","last_active":"2023-10-08T02:12:25.111471+0000","last_peered":"2023-10-08T02:12:25.111471+0000","last_clean":"2023-10-08T02:12:25.111471+0000","last_became_active":"2023-10-08T02:12:24.130532+0000","last_became_peered":"2023-10-08T02:12:24.130532+0000","last_unstale":"2023-10-08T02:12:25.111471+0000","last_undegraded":"2023-10-08T02:12:25.111471+0000","last_fullsized":"2023-10-08T02:12:25.111471+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:50.220057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.134154+0000","last_change":"2023-10-08T02:12:25.130739+0000","last_active":"2023-10-08T02:12:25.134154+0000","last_peered":"2023-10-08T02:12:25.134154+0000","last_clean":"2023-10-08T02:12:25.134154+0000","last_became_active":"2023-10-08T02:12:25.130478+0000","last_became_peered":"2023-10-08T02:12:25.130478+0000","last_unstale":"2023-10-08T02:12:25.134154+0000","last_undegraded":"2023-10-08T02:12:25.134154+0000","last_fullsized":"2023-10-08T02:12:25.134154+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:35.429110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.110397+0000","last_change":"2023-10-08T02:12:24.119252+0000","last_active":"2023-10-08T02:12:25.110397+0000","last_peered":"2023-10-08T02:12:25.110397+0000","last_clean":"2023-10-08T02:12:25.110397+0000","last_became_active":"2023-10-08T02:12:24.119019+0000","last_became_peered":"2023-10-08T02:12:24.119019+0000","last_unstale":"2023-10-08T02:12:25.110397+0000","last_undegraded":"2023-10-08T02:12:25.110397+0000","last_fullsized":"2023-10-08T02:12:25.110397+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:36.768065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130313+0000","last_change":"2023-10-08T02:12:25.128058+0000","last_active":"2023-10-08T02:12:25.130313+0000","last_peered":"2023-10-08T02:12:25.130313+0000","last_clean":"2023-10-08T02:12:25.130313+0000","last_became_active":"2023-10-08T02:12:25.127795+0000","last_became_peered":"2023-10-08T02:12:25.127795+0000","last_unstale":"2023-10-08T02:12:25.130313+0000","last_undegraded":"2023-10-08T02:12:25.130313+0000","last_fullsized":"2023-10-08T02:12:25.130313+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:33.509282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.116751+0000","last_change":"2023-10-08T02:12:24.130902+0000","last_active":"2023-10-08T02:12:25.116751+0000","last_peered":"2023-10-08T02:12:25.116751+0000","last_clean":"2023-10-08T02:12:25.116751+0000","last_became_active":"2023-10-08T02:12:24.130383+0000","last_became_peered":"2023-10-08T02:12:24.130383+0000","last_unstale":"2023-10-08T02:12:25.116751+0000","last_undegraded":"2023-10-08T02:12:25.116751+0000","last_fullsized":"2023-10-08T02:12:25.116751+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:26.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.113768+0000","last_change":"2023-10-08T02:12:25.112840+0000","last_active":"2023-10-08T02:12:25.113768+0000","last_peered":"2023-10-08T02:12:25.113768+0000","last_clean":"2023-10-08T02:12:25.113768+0000","last_became_active":"2023-10-08T02:12:25.112584+0000","last_became_peered":"2023-10-08T02:12:25.112584+0000","last_unstale":"2023-10-08T02:12:25.113768+0000","last_undegraded":"2023-10-08T02:12:25.113768+0000","last_fullsized":"2023-10-08T02:12:25.113768+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:12.015550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"170'6","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.143767+0000","last_change":"2023-10-08T02:12:24.113457+0000","last_active":"2023-10-08T02:12:25.143767+0000","last_peered":"2023-10-08T02:12:25.143767+0000","last_clean":"2023-10-08T02:12:25.143767+0000","last_became_active":"2023-10-08T02:12:24.113148+0000","last_became_peered":"2023-10-08T02:12:24.113148+0000","last_unstale":"2023-10-08T02:12:25.143767+0000","last_undegraded":"2023-10-08T02:12:25.143767+0000","last_fullsized":"2023-10-08T02:12:25.143767+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:38.869273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130139+0000","last_change":"2023-10-08T02:12:25.121235+0000","last_active":"2023-10-08T02:12:25.130139+0000","last_peered":"2023-10-08T02:12:25.130139+0000","last_clean":"2023-10-08T02:12:25.130139+0000","last_became_active":"2023-10-08T02:12:25.121045+0000","last_became_peered":"2023-10-08T02:12:25.121045+0000","last_unstale":"2023-10-08T02:12:25.130139+0000","last_undegraded":"2023-10-08T02:12:25.130139+0000","last_fullsized":"2023-10-08T02:12:25.130139+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:18.587452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141742+0000","last_change":"2023-10-08T02:12:24.116681+0000","last_active":"2023-10-08T02:12:25.141742+0000","last_peered":"2023-10-08T02:12:25.141742+0000","last_clean":"2023-10-08T02:12:25.141742+0000","last_became_active":"2023-10-08T02:12:24.116381+0000","last_became_peered":"2023-10-08T02:12:24.116381+0000","last_unstale":"2023-10-08T02:12:25.141742+0000","last_undegraded":"2023-10-08T02:12:25.141742+0000","last_fullsized":"2023-10-08T02:12:25.141742+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:02.429294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.114824+0000","last_change":"2023-10-08T02:12:25.114200+0000","last_active":"2023-10-08T02:12:25.114824+0000","last_peered":"2023-10-08T02:12:25.114824+0000","last_clean":"2023-10-08T02:12:25.114824+0000","last_became_active":"2023-10-08T02:12:25.113933+0000","last_became_peered":"2023-10-08T02:12:25.113933+0000","last_unstale":"2023-10-08T02:12:25.114824+0000","last_undegraded":"2023-10-08T02:12:25.114824+0000","last_fullsized":"2023-10-08T02:12:25.114824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:17.278020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.106790+0000","last_change":"2023-10-08T02:12:24.120101+0000","last_active":"2023-10-08T02:12:25.106790+0000","last_peered":"2023-10-08T02:12:25.106790+0000","last_clean":"2023-10-08T02:12:25.106790+0000","last_became_active":"2023-10-08T02:12:24.119835+0000","last_became_peered":"2023-10-08T02:12:24.119835+0000","last_unstale":"2023-10-08T02:12:25.106790+0000","last_undegraded":"2023-10-08T02:12:25.106790+0000","last_fullsized":"2023-10-08T02:12:25.106790+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:54.743852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.131677+0000","last_change":"2023-10-08T02:12:25.127127+0000","last_active":"2023-10-08T02:12:25.131677+0000","last_peered":"2023-10-08T02:12:25.131677+0000","last_clean":"2023-10-08T02:12:25.131677+0000","last_became_active":"2023-10-08T02:12:25.126835+0000","last_became_peered":"2023-10-08T02:12:25.126835+0000","last_unstale":"2023-10-08T02:12:25.131677+0000","last_undegraded":"2023-10-08T02:12:25.131677+0000","last_fullsized":"2023-10-08T02:12:25.131677+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:51.226421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.107297+0000","last_change":"2023-10-08T02:12:24.119093+0000","last_active":"2023-10-08T02:12:25.107297+0000","last_peered":"2023-10-08T02:12:25.107297+0000","last_clean":"2023-10-08T02:12:25.107297+0000","last_became_active":"2023-10-08T02:12:24.118840+0000","last_became_peered":"2023-10-08T02:12:24.118840+0000","last_unstale":"2023-10-08T02:12:25.107297+0000","last_undegraded":"2023-10-08T02:12:25.107297+0000","last_fullsized":"2023-10-08T02:12:25.107297+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:14.786069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.127456+0000","last_change":"2023-10-08T02:12:25.119744+0000","last_active":"2023-10-08T02:12:25.127456+0000","last_peered":"2023-10-08T02:12:25.127456+0000","last_clean":"2023-10-08T02:12:25.127456+0000","last_became_active":"2023-10-08T02:12:25.119484+0000","last_became_peered":"2023-10-08T02:12:25.119484+0000","last_unstale":"2023-10-08T02:12:25.127456+0000","last_undegraded":"2023-10-08T02:12:25.127456+0000","last_fullsized":"2023-10-08T02:12:25.127456+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:12.015550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"170'8","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.142279+0000","last_change":"2023-10-08T02:12:24.114740+0000","last_active":"2023-10-08T02:12:25.142279+0000","last_peered":"2023-10-08T02:12:25.142279+0000","last_clean":"2023-10-08T02:12:25.142279+0000","last_became_active":"2023-10-08T02:12:24.114464+0000","last_became_peered":"2023-10-08T02:12:24.114464+0000","last_unstale":"2023-10-08T02:12:25.142279+0000","last_undegraded":"2023-10-08T02:12:25.142279+0000","last_fullsized":"2023-10-08T02:12:25.142279+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:35.512125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130151+0000","last_change":"2023-10-08T02:12:25.126757+0000","last_active":"2023-10-08T02:12:25.130151+0000","last_peered":"2023-10-08T02:12:25.130151+0000","last_clean":"2023-10-08T02:12:25.130151+0000","last_became_active":"2023-10-08T02:12:25.126434+0000","last_became_peered":"2023-10-08T02:12:25.126434+0000","last_unstale":"2023-10-08T02:12:25.130151+0000","last_undegraded":"2023-10-08T02:12:25.130151+0000","last_fullsized":"2023-10-08T02:12:25.130151+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:57.502890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141160+0000","last_change":"2023-10-08T02:12:24.119874+0000","last_active":"2023-10-08T02:12:25.141160+0000","last_peered":"2023-10-08T02:12:25.141160+0000","last_clean":"2023-10-08T02:12:25.141160+0000","last_became_active":"2023-10-08T02:12:24.119567+0000","last_became_peered":"2023-10-08T02:12:24.119567+0000","last_unstale":"2023-10-08T02:12:25.141160+0000","last_undegraded":"2023-10-08T02:12:25.141160+0000","last_fullsized":"2023-10-08T02:12:25.141160+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:31:57.955586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130018+0000","last_change":"2023-10-08T02:12:25.128147+0000","last_active":"2023-10-08T02:12:25.130018+0000","last_peered":"2023-10-08T02:12:25.130018+0000","last_clean":"2023-10-08T02:12:25.130018+0000","last_became_active":"2023-10-08T02:12:25.127835+0000","last_became_peered":"2023-10-08T02:12:25.127835+0000","last_unstale":"2023-10-08T02:12:25.130018+0000","last_undegraded":"2023-10-08T02:12:25.130018+0000","last_fullsized":"2023-10-08T02:12:25.130018+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:31:58.961951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.140222+0000","last_change":"2023-10-08T02:12:24.108816+0000","last_active":"2023-10-08T02:12:25.140222+0000","last_peered":"2023-10-08T02:12:25.140222+0000","last_clean":"2023-10-08T02:12:25.140222+0000","last_became_active":"2023-10-08T02:12:24.108533+0000","last_became_peered":"2023-10-08T02:12:24.108533+0000","last_unstale":"2023-10-08T02:12:25.140222+0000","last_undegraded":"2023-10-08T02:12:25.140222+0000","last_fullsized":"2023-10-08T02:12:25.140222+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:26.497367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.132164+0000","last_change":"2023-10-08T02:12:25.122104+0000","last_active":"2023-10-08T02:12:25.132164+0000","last_peered":"2023-10-08T02:12:25.132164+0000","last_clean":"2023-10-08T02:12:25.132164+0000","last_became_active":"2023-10-08T02:12:25.121839+0000","last_became_peered":"2023-10-08T02:12:25.121839+0000","last_unstale":"2023-10-08T02:12:25.132164+0000","last_undegraded":"2023-10-08T02:12:25.132164+0000","last_fullsized":"2023-10-08T02:12:25.132164+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:45.356324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.139685+0000","last_change":"2023-10-08T02:12:24.126998+0000","last_active":"2023-10-08T02:12:25.139685+0000","last_peered":"2023-10-08T02:12:25.139685+0000","last_clean":"2023-10-08T02:12:25.139685+0000","last_became_active":"2023-10-08T02:12:24.126700+0000","last_became_peered":"2023-10-08T02:12:24.126700+0000","last_unstale":"2023-10-08T02:12:25.139685+0000","last_undegraded":"2023-10-08T02:12:25.139685+0000","last_fullsized":"2023-10-08T02:12:25.139685+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.136401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122183+0000","last_change":"2023-10-08T02:12:25.120029+0000","last_active":"2023-10-08T02:12:25.122183+0000","last_peered":"2023-10-08T02:12:25.122183+0000","last_clean":"2023-10-08T02:12:25.122183+0000","last_became_active":"2023-10-08T02:12:25.119796+0000","last_became_peered":"2023-10-08T02:12:25.119796+0000","last_unstale":"2023-10-08T02:12:25.122183+0000","last_undegraded":"2023-10-08T02:12:25.122183+0000","last_fullsized":"2023-10-08T02:12:25.122183+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:29.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.111159+0000","last_change":"2023-10-08T02:12:24.114716+0000","last_active":"2023-10-08T02:12:25.111159+0000","last_peered":"2023-10-08T02:12:25.111159+0000","last_clean":"2023-10-08T02:12:25.111159+0000","last_became_active":"2023-10-08T02:12:24.114464+0000","last_became_peered":"2023-10-08T02:12:24.114464+0000","last_unstale":"2023-10-08T02:12:25.111159+0000","last_undegraded":"2023-10-08T02:12:25.111159+0000","last_fullsized":"2023-10-08T02:12:25.111159+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:11.009186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129217+0000","last_change":"2023-10-08T02:12:25.123285+0000","last_active":"2023-10-08T02:12:25.129217+0000","last_peered":"2023-10-08T02:12:25.129217+0000","last_clean":"2023-10-08T02:12:25.129217+0000","last_became_active":"2023-10-08T02:12:25.123034+0000","last_became_peered":"2023-10-08T02:12:25.123034+0000","last_unstale":"2023-10-08T02:12:25.129217+0000","last_undegraded":"2023-10-08T02:12:25.129217+0000","last_fullsized":"2023-10-08T02:12:25.129217+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:52.000775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141137+0000","last_change":"2023-10-08T02:12:24.119426+0000","last_active":"2023-10-08T02:12:25.141137+0000","last_peered":"2023-10-08T02:12:25.141137+0000","last_clean":"2023-10-08T02:12:25.141137+0000","last_became_active":"2023-10-08T02:12:24.119135+0000","last_became_peered":"2023-10-08T02:12:24.119135+0000","last_unstale":"2023-10-08T02:12:25.141137+0000","last_undegraded":"2023-10-08T02:12:25.141137+0000","last_fullsized":"2023-10-08T02:12:25.141137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:10.627672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.123808+0000","last_change":"2023-10-08T02:12:25.123130+0000","last_active":"2023-10-08T02:12:25.123808+0000","last_peered":"2023-10-08T02:12:25.123808+0000","last_clean":"2023-10-08T02:12:25.123808+0000","last_became_active":"2023-10-08T02:12:25.122884+0000","last_became_peered":"2023-10-08T02:12:25.122884+0000","last_unstale":"2023-10-08T02:12:25.123808+0000","last_undegraded":"2023-10-08T02:12:25.123808+0000","last_fullsized":"2023-10-08T02:12:25.123808+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:28:12.679734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.114898+0000","last_change":"2023-10-08T02:12:24.129065+0000","last_active":"2023-10-08T02:12:25.114898+0000","last_peered":"2023-10-08T02:12:25.114898+0000","last_clean":"2023-10-08T02:12:25.114898+0000","last_became_active":"2023-10-08T02:12:24.128711+0000","last_became_peered":"2023-10-08T02:12:24.128711+0000","last_unstale":"2023-10-08T02:12:25.114898+0000","last_undegraded":"2023-10-08T02:12:25.114898+0000","last_fullsized":"2023-10-08T02:12:25.114898+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:53.423324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.132538+0000","last_change":"2023-10-08T02:12:25.127609+0000","last_active":"2023-10-08T02:12:25.132538+0000","last_peered":"2023-10-08T02:12:25.132538+0000","last_clean":"2023-10-08T02:12:25.132538+0000","last_became_active":"2023-10-08T02:12:25.126204+0000","last_became_peered":"2023-10-08T02:12:25.126204+0000","last_unstale":"2023-10-08T02:12:25.132538+0000","last_undegraded":"2023-10-08T02:12:25.132538+0000","last_fullsized":"2023-10-08T02:12:25.132538+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:34.173348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.114623+0000","last_change":"2023-10-08T02:12:24.130331+0000","last_active":"2023-10-08T02:12:25.114623+0000","last_peered":"2023-10-08T02:12:25.114623+0000","last_clean":"2023-10-08T02:12:25.114623+0000","last_became_active":"2023-10-08T02:12:24.129961+0000","last_became_peered":"2023-10-08T02:12:24.129961+0000","last_unstale":"2023-10-08T02:12:25.114623+0000","last_undegraded":"2023-10-08T02:12:25.114623+0000","last_fullsized":"2023-10-08T02:12:25.114623+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:46.851648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.123491+0000","last_change":"2023-10-08T02:12:25.119504+0000","last_active":"2023-10-08T02:12:25.123491+0000","last_peered":"2023-10-08T02:12:25.123491+0000","last_clean":"2023-10-08T02:12:25.123491+0000","last_became_active":"2023-10-08T02:12:25.119249+0000","last_became_peered":"2023-10-08T02:12:25.119249+0000","last_unstale":"2023-10-08T02:12:25.123491+0000","last_undegraded":"2023-10-08T02:12:25.123491+0000","last_fullsized":"2023-10-08T02:12:25.123491+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:28.172639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.106560+0000","last_change":"2023-10-08T02:12:24.109467+0000","last_active":"2023-10-08T02:12:25.106560+0000","last_peered":"2023-10-08T02:12:25.106560+0000","last_clean":"2023-10-08T02:12:25.106560+0000","last_became_active":"2023-10-08T02:12:24.108744+0000","last_became_peered":"2023-10-08T02:12:24.108744+0000","last_unstale":"2023-10-08T02:12:25.106560+0000","last_undegraded":"2023-10-08T02:12:25.106560+0000","last_fullsized":"2023-10-08T02:12:25.106560+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:35.512125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.139748+0000","last_change":"2023-10-08T02:12:25.136826+0000","last_active":"2023-10-08T02:12:25.139748+0000","last_peered":"2023-10-08T02:12:25.139748+0000","last_clean":"2023-10-08T02:12:25.139748+0000","last_became_active":"2023-10-08T02:12:25.136531+0000","last_became_peered":"2023-10-08T02:12:25.136531+0000","last_unstale":"2023-10-08T02:12:25.139748+0000","last_undegraded":"2023-10-08T02:12:25.139748+0000","last_fullsized":"2023-10-08T02:12:25.139748+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:27.204105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.140354+0000","last_change":"2023-10-08T02:12:24.129117+0000","last_active":"2023-10-08T02:12:25.140354+0000","last_peered":"2023-10-08T02:12:25.140354+0000","last_clean":"2023-10-08T02:12:25.140354+0000","last_became_active":"2023-10-08T02:12:24.128826+0000","last_became_peered":"2023-10-08T02:12:24.128826+0000","last_unstale":"2023-10-08T02:12:25.140354+0000","last_undegraded":"2023-10-08T02:12:25.140354+0000","last_fullsized":"2023-10-08T02:12:25.140354+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:28.536848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.127615+0000","last_change":"2023-10-08T02:12:25.120691+0000","last_active":"2023-10-08T02:12:25.127615+0000","last_peered":"2023-10-08T02:12:25.127615+0000","last_clean":"2023-10-08T02:12:25.127615+0000","last_became_active":"2023-10-08T02:12:25.120426+0000","last_became_peered":"2023-10-08T02:12:25.120426+0000","last_unstale":"2023-10-08T02:12:25.127615+0000","last_undegraded":"2023-10-08T02:12:25.127615+0000","last_fullsized":"2023-10-08T02:12:25.127615+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:00.403217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.108217+0000","last_change":"2023-10-08T02:12:24.131763+0000","last_active":"2023-10-08T02:12:25.108217+0000","last_peered":"2023-10-08T02:12:25.108217+0000","last_clean":"2023-10-08T02:12:25.108217+0000","last_became_active":"2023-10-08T02:12:24.131500+0000","last_became_peered":"2023-10-08T02:12:24.131500+0000","last_unstale":"2023-10-08T02:12:25.108217+0000","last_undegraded":"2023-10-08T02:12:25.108217+0000","last_fullsized":"2023-10-08T02:12:25.108217+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:41.052726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.125862+0000","last_change":"2023-10-08T02:12:25.120637+0000","last_active":"2023-10-08T02:12:25.125862+0000","last_peered":"2023-10-08T02:12:25.125862+0000","last_clean":"2023-10-08T02:12:25.125862+0000","last_became_active":"2023-10-08T02:12:25.120348+0000","last_became_peered":"2023-10-08T02:12:25.120348+0000","last_unstale":"2023-10-08T02:12:25.125862+0000","last_undegraded":"2023-10-08T02:12:25.125862+0000","last_fullsized":"2023-10-08T02:12:25.125862+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:47.858013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.109433+0000","last_change":"2023-10-08T02:12:24.121322+0000","last_active":"2023-10-08T02:12:25.109433+0000","last_peered":"2023-10-08T02:12:25.109433+0000","last_clean":"2023-10-08T02:12:25.109433+0000","last_became_active":"2023-10-08T02:12:24.121017+0000","last_became_peered":"2023-10-08T02:12:24.121017+0000","last_unstale":"2023-10-08T02:12:25.109433+0000","last_undegraded":"2023-10-08T02:12:25.109433+0000","last_fullsized":"2023-10-08T02:12:25.109433+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:02.557364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129932+0000","last_change":"2023-10-08T02:12:25.125302+0000","last_active":"2023-10-08T02:12:25.129932+0000","last_peered":"2023-10-08T02:12:25.129932+0000","last_clean":"2023-10-08T02:12:25.129932+0000","last_became_active":"2023-10-08T02:12:25.125045+0000","last_became_peered":"2023-10-08T02:12:25.125045+0000","last_unstale":"2023-10-08T02:12:25.129932+0000","last_undegraded":"2023-10-08T02:12:25.129932+0000","last_fullsized":"2023-10-08T02:12:25.129932+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:04.274497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.108875+0000","last_change":"2023-10-08T02:12:24.120609+0000","last_active":"2023-10-08T02:12:25.108875+0000","last_peered":"2023-10-08T02:12:25.108875+0000","last_clean":"2023-10-08T02:12:25.108875+0000","last_became_active":"2023-10-08T02:12:24.120253+0000","last_became_peered":"2023-10-08T02:12:24.120253+0000","last_unstale":"2023-10-08T02:12:25.108875+0000","last_undegraded":"2023-10-08T02:12:25.108875+0000","last_fullsized":"2023-10-08T02:12:25.108875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:26.108797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129915+0000","last_change":"2023-10-08T02:12:25.127167+0000","last_active":"2023-10-08T02:12:25.129915+0000","last_peered":"2023-10-08T02:12:25.129915+0000","last_clean":"2023-10-08T02:12:25.129915+0000","last_became_active":"2023-10-08T02:12:25.126867+0000","last_became_peered":"2023-10-08T02:12:25.126867+0000","last_unstale":"2023-10-08T02:12:25.129915+0000","last_undegraded":"2023-10-08T02:12:25.129915+0000","last_fullsized":"2023-10-08T02:12:25.129915+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:00.403217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.106241+0000","last_change":"2023-10-08T02:12:24.122209+0000","last_active":"2023-10-08T02:12:25.106241+0000","last_peered":"2023-10-08T02:12:25.106241+0000","last_clean":"2023-10-08T02:12:25.106241+0000","last_became_active":"2023-10-08T02:12:24.121963+0000","last_became_peered":"2023-10-08T02:12:24.121963+0000","last_unstale":"2023-10-08T02:12:25.106241+0000","last_undegraded":"2023-10-08T02:12:25.106241+0000","last_fullsized":"2023-10-08T02:12:25.106241+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:32:49.145683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.134184+0000","last_change":"2023-10-08T02:12:25.131333+0000","last_active":"2023-10-08T02:12:25.134184+0000","last_peered":"2023-10-08T02:12:25.134184+0000","last_clean":"2023-10-08T02:12:25.134184+0000","last_became_active":"2023-10-08T02:12:25.131087+0000","last_became_peered":"2023-10-08T02:12:25.131087+0000","last_unstale":"2023-10-08T02:12:25.134184+0000","last_undegraded":"2023-10-08T02:12:25.134184+0000","last_fullsized":"2023-10-08T02:12:25.134184+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:31.655738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.114808+0000","last_change":"2023-10-08T02:12:24.128461+0000","last_active":"2023-10-08T02:12:25.114808+0000","last_peered":"2023-10-08T02:12:25.114808+0000","last_clean":"2023-10-08T02:12:25.114808+0000","last_became_active":"2023-10-08T02:12:24.126911+0000","last_became_peered":"2023-10-08T02:12:24.126911+0000","last_unstale":"2023-10-08T02:12:25.114808+0000","last_undegraded":"2023-10-08T02:12:25.114808+0000","last_fullsized":"2023-10-08T02:12:25.114808+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:34.422745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.136308+0000","last_change":"2023-10-08T02:12:25.132146+0000","last_active":"2023-10-08T02:12:25.136308+0000","last_peered":"2023-10-08T02:12:25.136308+0000","last_clean":"2023-10-08T02:12:25.136308+0000","last_became_active":"2023-10-08T02:12:25.131867+0000","last_became_peered":"2023-10-08T02:12:25.131867+0000","last_unstale":"2023-10-08T02:12:25.136308+0000","last_undegraded":"2023-10-08T02:12:25.136308+0000","last_fullsized":"2023-10-08T02:12:25.136308+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:14:14.358279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.107269+0000","last_change":"2023-10-08T02:12:24.116761+0000","last_active":"2023-10-08T02:12:25.107269+0000","last_peered":"2023-10-08T02:12:25.107269+0000","last_clean":"2023-10-08T02:12:25.107269+0000","last_became_active":"2023-10-08T02:12:24.116469+0000","last_became_peered":"2023-10-08T02:12:24.116469+0000","last_unstale":"2023-10-08T02:12:25.107269+0000","last_undegraded":"2023-10-08T02:12:25.107269+0000","last_fullsized":"2023-10-08T02:12:25.107269+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:42.653885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122451+0000","last_change":"2023-10-08T02:12:25.116799+0000","last_active":"2023-10-08T02:12:25.122451+0000","last_peered":"2023-10-08T02:12:25.122451+0000","last_clean":"2023-10-08T02:12:25.122451+0000","last_became_active":"2023-10-08T02:12:25.116518+0000","last_became_peered":"2023-10-08T02:12:25.116518+0000","last_unstale":"2023-10-08T02:12:25.122451+0000","last_undegraded":"2023-10-08T02:12:25.122451+0000","last_fullsized":"2023-10-08T02:12:25.122451+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:54.429689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.110134+0000","last_change":"2023-10-08T02:12:24.126247+0000","last_active":"2023-10-08T02:12:25.110134+0000","last_peered":"2023-10-08T02:12:25.110134+0000","last_clean":"2023-10-08T02:12:25.110134+0000","last_became_active":"2023-10-08T02:12:24.125977+0000","last_became_peered":"2023-10-08T02:12:24.125977+0000","last_unstale":"2023-10-08T02:12:25.110134+0000","last_undegraded":"2023-10-08T02:12:25.110134+0000","last_fullsized":"2023-10-08T02:12:25.110134+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:05:57.759874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126282+0000","last_change":"2023-10-08T02:12:25.124610+0000","last_active":"2023-10-08T02:12:25.126282+0000","last_peered":"2023-10-08T02:12:25.126282+0000","last_clean":"2023-10-08T02:12:25.126282+0000","last_became_active":"2023-10-08T02:12:25.124354+0000","last_became_peered":"2023-10-08T02:12:25.124354+0000","last_unstale":"2023-10-08T02:12:25.126282+0000","last_undegraded":"2023-10-08T02:12:25.126282+0000","last_fullsized":"2023-10-08T02:12:25.126282+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:24:48.457813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"170'3","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.139967+0000","last_change":"2023-10-08T02:12:24.108579+0000","last_active":"2023-10-08T02:12:25.139967+0000","last_peered":"2023-10-08T02:12:25.139967+0000","last_clean":"2023-10-08T02:12:25.139967+0000","last_became_active":"2023-10-08T02:12:24.108269+0000","last_became_peered":"2023-10-08T02:12:24.108269+0000","last_unstale":"2023-10-08T02:12:25.139967+0000","last_undegraded":"2023-10-08T02:12:25.139967+0000","last_fullsized":"2023-10-08T02:12:25.139967+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.136401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"160'293","reported_seq":914,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.113000+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:12:25.113000+0000","last_peered":"2023-10-08T02:12:25.113000+0000","last_clean":"2023-10-08T02:12:25.113000+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:12:25.113000+0000","last_undegraded":"2023-10-08T02:12:25.113000+0000","last_fullsized":"2023-10-08T02:12:25.113000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575760,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312248,"kb_used_data":2672,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059592,"statfs":{"total":96636764160,"available":96317022208,"internally_reserved":0,"allocated":2736128,"data_stored":1655519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.70099999999999996,"15min":0.73099999999999998},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76100000000000001,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.748},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.751,"15min":0.75},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.69299999999999995}]},{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.73599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81200000000000006,"5min":0.999,"15min":1.0580000000000001},"last":0.64900000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.755,"15min":0.76400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.873,"5min":0.94899999999999995,"15min":1.0600000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.755,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.79500000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.875,"5min":1.502,"15min":1.502},"last":0.71099999999999997}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.46300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":0.94999999999999996,"15min":0.97799999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":1.034,"15min":1.034},"last":0.44700000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.54000000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":0.76800000000000002},"min":{"1min":0.56999999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.998,"15min":1.0149999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":1.109,"15min":1.109},"last":0.67200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575760,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299384,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072456,"statfs":{"total":96636764160,"available":96330194944,"internally_reserved":0,"allocated":2146304,"data_stored":1064673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.76000000000000001,"15min":0.74199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.569,"15min":1.569},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.74399999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94199999999999995,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89900000000000002,"5min":1.472,"15min":1.472},"last":0.64400000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.81799999999999995,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79600000000000004,"15min":0.76500000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.495},"max":{"1min":0.91700000000000004,"5min":1.603,"15min":1.603},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.871,"5min":1.169,"15min":1.169},"last":0.73299999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":1.133,"15min":1.133},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.439,"5min":0.439,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.504}]},{"osd":5,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82599999999999996,"5min":0.94299999999999995,"15min":1.0680000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74199999999999999,"15min":0.72099999999999997},"min":{"1min":0.503,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90200000000000002,"5min":1.232,"15min":1.232},"last":0.77100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575760,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307568,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064272,"statfs":{"total":96636764160,"available":96321814528,"internally_reserved":0,"allocated":2138112,"data_stored":1065081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.315},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.603,"15min":1.603},"last":0.44600000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.497,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.60199999999999998,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.751,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.70799999999999996,"5min":3.698,"15min":3.698},"last":0.36899999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86399999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95999999999999996,"5min":1.734,"15min":1.734},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.53600000000000003},"max":{"1min":1.0489999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91600000000000004,"5min":1.403,"15min":1.403},"last":0.73099999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.61299999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0149999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.573,"15min":1.573},"last":0.74399999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.875,"15min":0.86499999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.58999999999999997},"max":{"1min":0.99399999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575759,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308160,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063680,"statfs":{"total":96636764160,"available":96321208320,"internally_reserved":0,"allocated":2744320,"data_stored":1656465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.82899999999999996,"15min":0.83999999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58299999999999996,"5min":0.70699999999999996,"15min":0.746},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68500000000000005,"5min":0.88300000000000001,"15min":1.325},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.42399999999999999,"5min":0.41199999999999998,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72399999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.99399999999999999,"5min":1.052,"15min":1.24},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0009999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.77000000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0489999999999999,"5min":1.327,"15min":1.327},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.878,"15min":0.879},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.653,"15min":1.653},"last":0.755}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83599999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"max":{"1min":1.0209999999999999,"5min":1.577,"15min":1.577},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58099999999999996},"max":{"1min":0.97699999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.84799999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.129,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.89000000000000001,"15min":0.88200000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0720000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.75}]}]},{"osd":2,"up_from":13,"seq":55834575759,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311676,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060164,"statfs":{"total":96636764160,"available":96317607936,"internally_reserved":0,"allocated":2150400,"data_stored":1066133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65500000000000003,"5min":1.27,"15min":1.27},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.45600000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65600000000000003,"5min":0.79500000000000004,"15min":0.81699999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":0.69699999999999995,"5min":1.651,"15min":1.651},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.39400000000000002},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.495},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":0.52200000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81399999999999995,"15min":0.81000000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.55100000000000005},"max":{"1min":1.0209999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53400000000000003},"max":{"1min":0.93799999999999994,"5min":3.004,"15min":3.004},"last":0.71699999999999997}]},{"osd":5,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.86799999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.046,"5min":3.145,"15min":3.145},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0860000000000001,"5min":1.847,"15min":1.847},"last":0.76900000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.57699999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4950000000000001,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.623,"5min":0.60199999999999998,"15min":0.58599999999999997},"max":{"1min":1.0580000000000001,"5min":1.331,"15min":1.331},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.88700000000000001,"15min":0.85999999999999999},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":2.899,"15min":2.899},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.86599999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"max":{"1min":1.4570000000000001,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.84199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575759,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315760,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056080,"statfs":{"total":96636764160,"available":96313425920,"internally_reserved":0,"allocated":2138112,"data_stored":1064617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.375},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.497}]},{"osd":1,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47999999999999998,"15min":0.48299999999999998},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.54700000000000004,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89500000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.629,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83699999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56699999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.214},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":0.998,"5min":1.0680000000000001,"15min":1.0900000000000001},"last":0.76800000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.77100000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.876,"5min":1.0589999999999999,"15min":1.125},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.749}]},{"osd":6,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.01,"5min":1.403,"15min":1.403},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.81399999999999995,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.55600000000000005},"max":{"1min":0.90700000000000003,"5min":1.492,"15min":1.492},"last":0.90600000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.83799999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93799999999999994,"5min":1.341,"15min":1.341},"last":0.89400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575759,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311660,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060180,"statfs":{"total":96636764160,"available":96317624320,"internally_reserved":0,"allocated":2134016,"data_stored":1064527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":0.92100000000000004,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90900000000000003,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73799999999999999,"15min":0.749},"min":{"1min":0.51000000000000001,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":1.1000000000000001,"15min":1.52},"last":0.56100000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.74299999999999999,"15min":0.76600000000000001},"min":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76400000000000001,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88300000000000001,"5min":1.1579999999999999,"15min":1.403},"last":0.63100000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.76000000000000001,"15min":0.77400000000000002},"min":{"1min":0.56499999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85599999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.872,"5min":1.208,"15min":1.5069999999999999},"last":0.66000000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78900000000000003,"5min":1.05,"15min":1.05},"last":0.498},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95799999999999996,"5min":1.038,"15min":1.1559999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":1.016,"15min":1.016},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.83499999999999996,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.441}]},{"osd":7,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.44600000000000001},"max":{"1min":0.92900000000000005,"5min":1.0680000000000001,"15min":1.129},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79200000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575759,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311664,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060176,"statfs":{"total":96636764160,"available":96317620224,"internally_reserved":0,"allocated":2138112,"data_stored":1064549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.111,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.873},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.76700000000000002,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.438},"max":{"1min":1.0009999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70199999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":0.77800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42699999999999999},"max":{"1min":0.95399999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.77600000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.499},"max":{"1min":0.995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.50900000000000001},"max":{"1min":0.97699999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80700000000000005,"15min":0.77900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96399999999999997,"5min":1.0620000000000001,"15min":1.0669999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0529999999999999,"15min":1.091},"last":0.82199999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.441},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.115},"last":0.53900000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0629999999999999},"last":0.63},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.077},"last":0.73599999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0189999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:12:34.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:12:34.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.475+0000 7f4ed45ff700 1 Processor -- start 2023-10-08T02:12:34.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.476+0000 7f4ed45ff700 1 -- start start 2023-10-08T02:12:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.477+0000 7f4ed45ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0b0a10 0x7f4ecc155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.477+0000 7f4ed45ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0ad930 con 0x7f4ecc155c10 2023-10-08T02:12:34.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.477+0000 7f4ed45ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ecc0acd20 con 0x7f4ecc1578b0 2023-10-08T02:12:34.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.477+0000 7f4ed45ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ecc0ad000 con 0x7f4ecc0b0a10 2023-10-08T02:12:34.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.477+0000 7f4ed239b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4ecc155c10 0x7f4ecc0ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59552/0 (socket says 172.21.15.119:59552) 2023-10-08T02:12:34.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.477+0000 7f4ed239b700 1 -- 172.21.15.119:0/1570585021 learned_addr learned my addr 172.21.15.119:0/1570585021 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:34.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.477+0000 7f4ed2b9c700 1 --2- 172.21.15.119:0/1570585021 >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0b0a10 0x7f4ecc155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.477+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530944410 0 0) 0x7f4ecc0ad930 con 0x7f4ecc155c10 2023-10-08T02:12:34.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb8004680 con 0x7f4ecc155c10 2023-10-08T02:12:34.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1167442682 0 0) 0x7f4eb8004680 con 0x7f4ecc155c10 2023-10-08T02:12:34.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0b0a10 msgr2=0x7f4ecc155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:34.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 --2- 172.21.15.119:0/1570585021 >> v2:172.21.15.184:3300/0 conn(0x7f4ecc0b0a10 0x7f4ecc155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 >> v1:172.21.15.184:6789/0 conn(0x7f4ecc1578b0 legacy=0x7f4ecc0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0ad2e0 con 0x7f4ecc155c10 2023-10-08T02:12:34.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f4ec8001470 con 0x7f4ecc155c10 2023-10-08T02:12:34.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 >> v1:172.21.15.119:6789/0 conn(0x7f4ecc155c10 legacy=0x7f4ecc0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb8004b70 0x7f4eb8004f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb8005920 0x7f4eb8005d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb80062d0 0x7f4eb800cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0ad000 con 0x7f4eb80062d0 2023-10-08T02:12:34.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb800d1e0 con 0x7f4eb8005920 2023-10-08T02:12:34.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb800d3a0 con 0x7f4eb8004b70 2023-10-08T02:12:34.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.478+0000 7f4ed2b9c700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb80062d0 0x7f4eb800cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed1b9a700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb8005920 0x7f4eb8005d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed2b9c700 1 -- 172.21.15.119:0/1570585021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb8004b70 msgr2=0x7f4eb8004f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:34.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed2b9c700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb8004b70 0x7f4eb8004f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed2b9c700 1 -- 172.21.15.119:0/1570585021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb8005920 msgr2=0x7f4eb8005d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:34.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed2b9c700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb8005920 0x7f4eb8005d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed2b9c700 1 -- 172.21.15.119:0/1570585021 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb8004680 con 0x7f4eb80062d0 2023-10-08T02:12:34.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed1b9a700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4eb8005920 0x7f4eb8005d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:34.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed2b9c700 1 -- 172.21.15.119:0/1570585021 reap_dead start 2023-10-08T02:12:34.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed2b9c700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb80062d0 0x7f4eb800cb70 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0b0e10 tx=0x7f4ec400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62a96e153887e44 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:34.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400ea90 con 0x7f4eb80062d0 2023-10-08T02:12:34.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec400f040 con 0x7f4eb80062d0 2023-10-08T02:12:34.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.479+0000 7f4ed1399700 1 -- 172.21.15.119:0/1570585021 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec400ee40 con 0x7f4eb80062d0 2023-10-08T02:12:34.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.480+0000 7f4ed45ff700 1 -- 172.21.15.119:0/1570585021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb80062d0 msgr2=0x7f4eb800cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.480+0000 7f4ed45ff700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb80062d0 0x7f4eb800cb70 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0b0e10 tx=0x7f4ec400b040 comp rx=0 tx=0).stop 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.480+0000 7f4ed45ff700 1 -- 172.21.15.119:0/1570585021 shutdown_connections 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.480+0000 7f4ed45ff700 1 --2- 172.21.15.119:0/1570585021 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4eb80062d0 0x7f4eb800cb70 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.480+0000 7f4ed45ff700 1 -- 172.21.15.119:0/1570585021 >> 172.21.15.119:0/1570585021 conn(0x7f4ecc06a320 msgr2=0x7f4ecc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.480+0000 7f4ed45ff700 1 -- 172.21.15.119:0/1570585021 shutdown_connections 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.480+0000 7f4ed45ff700 1 -- 172.21.15.119:0/1570585021 wait complete. 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.481+0000 7f4ed45ff700 1 Processor -- start 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.481+0000 7f4ed45ff700 1 -- start start 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.482+0000 7f4ed45ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb80062d0 0x7f4ecc14f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.482+0000 7f4ed45ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0ca990 0x7f4ecc1659d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.482+0000 7f4ed45ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc165f10 0x7f4ecc068390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.482+0000 7f4ed45ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc14f970 con 0x7f4ecc0ca990 2023-10-08T02:12:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.482+0000 7f4ed45ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc0c9470 con 0x7f4ecc165f10 2023-10-08T02:12:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.482+0000 7f4ed45ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ecc1663d0 con 0x7f4eb80062d0 2023-10-08T02:12:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0ca990 0x7f4ecc1659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0ca990 0x7f4ecc1659d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44130/0 (socket says 172.21.15.119:44130) 2023-10-08T02:12:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 -- 172.21.15.119:0/2253507970 learned_addr learned my addr 172.21.15.119:0/2253507970 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:34.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed239b700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb80062d0 0x7f4ecc14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed2b9c700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc165f10 0x7f4ecc068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 -- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb80062d0 msgr2=0x7f4ecc14f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb80062d0 0x7f4ecc14f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 -- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc165f10 msgr2=0x7f4ecc068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc165f10 0x7f4ecc068390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 -- 172.21.15.119:0/2253507970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc1666b0 con 0x7f4ecc0ca990 2023-10-08T02:12:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed239b700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb80062d0 0x7f4ecc14f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:34.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed2b9c700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc165f10 0x7f4ecc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ed1b9a700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0ca990 0x7f4ecc1659d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00d5e0 tx=0x7f4ebc009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.483+0000 7f4ec27fc700 1 -- 172.21.15.119:0/2253507970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc006820 con 0x7f4ecc0ca990 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.484+0000 7f4ec27fc700 1 -- 172.21.15.119:0/2253507970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc006980 con 0x7f4ecc0ca990 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.484+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ecc166990 con 0x7f4ecc0ca990 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.484+0000 7f4ec27fc700 1 -- 172.21.15.119:0/2253507970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc006d20 con 0x7f4ecc0ca990 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.484+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ecc151b50 con 0x7f4ecc0ca990 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.485+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ecc0c9630 con 0x7f4ecc0ca990 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.485+0000 7f4ec27fc700 1 -- 172.21.15.119:0/2253507970 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ebc007230 con 0x7f4ecc0ca990 2023-10-08T02:12:34.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.486+0000 7f4ec27fc700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4eb0066f60 0x7f4eb0069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.486+0000 7f4ec27fc700 1 -- 172.21.15.119:0/2253507970 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f4ebc084530 con 0x7f4ecc0ca990 2023-10-08T02:12:34.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.487+0000 7f4ed239b700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4eb0066f60 0x7f4eb0069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.487+0000 7f4ed239b700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4eb0066f60 0x7f4eb0069410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8000e10 tx=0x7f4ec80016a0 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:34.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.491+0000 7f4ec27fc700 1 -- 172.21.15.119:0/2253507970 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ebc0514a0 con 0x7f4ecc0ca990 2023-10-08T02:12:34.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.675+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ecc0c92f0 con 0x7f4eb0066f60 2023-10-08T02:12:34.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.683+0000 7f4ec27fc700 1 -- 172.21.15.119:0/2253507970 <== mgr.8967 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289600 (secure 0 0 0) 0x7f4ec80059f0 con 0x7f4eb0066f60 2023-10-08T02:12:34.687 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:34.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4eb0066f60 msgr2=0x7f4eb0069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4eb0066f60 0x7f4eb0069410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8000e10 tx=0x7f4ec80016a0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0ca990 msgr2=0x7f4ecc1659d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0ca990 0x7f4ecc1659d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00d5e0 tx=0x7f4ebc009040 comp rx=0 tx=0).stop 2023-10-08T02:12:34.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 shutdown_connections 2023-10-08T02:12:34.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f4eb0066f60 0x7f4eb0069410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4eb80062d0 0x7f4ecc14f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4ecc165f10 0x7f4ecc068390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.687+0000 7f4ed45ff700 1 --2- 172.21.15.119:0/2253507970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4ecc0ca990 0x7f4ecc1659d0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.688+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 >> 172.21.15.119:0/2253507970 conn(0x7f4ecc06a320 msgr2=0x7f4ecc0b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:34.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.688+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 shutdown_connections 2023-10-08T02:12:34.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.688+0000 7f4ed45ff700 1 -- 172.21.15.119:0/2253507970 wait complete. 2023-10-08T02:12:34.690 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:12:34.705 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-08T02:12:32.685288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":523,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":331,"ondisk_log_size":331,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2478120,"kb_used_data":17896,"kb_used_omap":0,"kb_used_meta":2460160,"kb_avail":752496600,"statfs":{"total":773094113280,"available":770556518400,"internally_reserved":0,"allocated":18325504,"data_stored":9701564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2519203840},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001495"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129836+0000","last_change":"2023-10-08T02:12:25.127935+0000","last_active":"2023-10-08T02:12:25.129836+0000","last_peered":"2023-10-08T02:12:25.129836+0000","last_clean":"2023-10-08T02:12:25.129836+0000","last_became_active":"2023-10-08T02:12:25.127666+0000","last_became_peered":"2023-10-08T02:12:25.127666+0000","last_unstale":"2023-10-08T02:12:25.129836+0000","last_undegraded":"2023-10-08T02:12:25.129836+0000","last_fullsized":"2023-10-08T02:12:25.129836+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:54.429689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.117989+0000","last_change":"2023-10-08T02:12:25.114785+0000","last_active":"2023-10-08T02:12:25.117989+0000","last_peered":"2023-10-08T02:12:25.117989+0000","last_clean":"2023-10-08T02:12:25.117989+0000","last_became_active":"2023-10-08T02:12:25.114477+0000","last_became_peered":"2023-10-08T02:12:25.114477+0000","last_unstale":"2023-10-08T02:12:25.117989+0000","last_undegraded":"2023-10-08T02:12:25.117989+0000","last_fullsized":"2023-10-08T02:12:25.117989+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:29.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122727+0000","last_change":"2023-10-08T02:12:25.116047+0000","last_active":"2023-10-08T02:12:25.122727+0000","last_peered":"2023-10-08T02:12:25.122727+0000","last_clean":"2023-10-08T02:12:25.122727+0000","last_became_active":"2023-10-08T02:12:25.115767+0000","last_became_peered":"2023-10-08T02:12:25.115767+0000","last_unstale":"2023-10-08T02:12:25.122727+0000","last_undegraded":"2023-10-08T02:12:25.122727+0000","last_fullsized":"2023-10-08T02:12:25.122727+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:37.774429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138480+0000","last_change":"2023-10-08T02:12:25.136222+0000","last_active":"2023-10-08T02:12:25.138480+0000","last_peered":"2023-10-08T02:12:25.138480+0000","last_clean":"2023-10-08T02:12:25.138480+0000","last_became_active":"2023-10-08T02:12:25.135973+0000","last_became_peered":"2023-10-08T02:12:25.135973+0000","last_unstale":"2023-10-08T02:12:25.138480+0000","last_undegraded":"2023-10-08T02:12:25.138480+0000","last_fullsized":"2023-10-08T02:12:25.138480+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:37.031644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.131634+0000","last_change":"2023-10-08T02:12:25.128861+0000","last_active":"2023-10-08T02:12:25.131634+0000","last_peered":"2023-10-08T02:12:25.131634+0000","last_clean":"2023-10-08T02:12:25.131634+0000","last_became_active":"2023-10-08T02:12:25.128427+0000","last_became_peered":"2023-10-08T02:12:25.128427+0000","last_unstale":"2023-10-08T02:12:25.131634+0000","last_undegraded":"2023-10-08T02:12:25.131634+0000","last_fullsized":"2023-10-08T02:12:25.131634+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:17:09.195946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122258+0000","last_change":"2023-10-08T02:12:25.120777+0000","last_active":"2023-10-08T02:12:25.122258+0000","last_peered":"2023-10-08T02:12:25.122258+0000","last_clean":"2023-10-08T02:12:25.122258+0000","last_became_active":"2023-10-08T02:12:25.120526+0000","last_became_peered":"2023-10-08T02:12:25.120526+0000","last_unstale":"2023-10-08T02:12:25.122258+0000","last_undegraded":"2023-10-08T02:12:25.122258+0000","last_fullsized":"2023-10-08T02:12:25.122258+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:27.746043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.124600+0000","last_change":"2023-10-08T02:12:25.123401+0000","last_active":"2023-10-08T02:12:25.124600+0000","last_peered":"2023-10-08T02:12:25.124600+0000","last_clean":"2023-10-08T02:12:25.124600+0000","last_became_active":"2023-10-08T02:12:25.123126+0000","last_became_peered":"2023-10-08T02:12:25.123126+0000","last_unstale":"2023-10-08T02:12:25.124600+0000","last_undegraded":"2023-10-08T02:12:25.124600+0000","last_fullsized":"2023-10-08T02:12:25.124600+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:15:47.715028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.121556+0000","last_change":"2023-10-08T02:12:25.117340+0000","last_active":"2023-10-08T02:12:25.121556+0000","last_peered":"2023-10-08T02:12:25.121556+0000","last_clean":"2023-10-08T02:12:25.121556+0000","last_became_active":"2023-10-08T02:12:25.117038+0000","last_became_peered":"2023-10-08T02:12:25.117038+0000","last_unstale":"2023-10-08T02:12:25.121556+0000","last_undegraded":"2023-10-08T02:12:25.121556+0000","last_fullsized":"2023-10-08T02:12:25.121556+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:35.429110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.124604+0000","last_change":"2023-10-08T02:12:25.119671+0000","last_active":"2023-10-08T02:12:25.124604+0000","last_peered":"2023-10-08T02:12:25.124604+0000","last_clean":"2023-10-08T02:12:25.124604+0000","last_became_active":"2023-10-08T02:12:25.119381+0000","last_became_peered":"2023-10-08T02:12:25.119381+0000","last_unstale":"2023-10-08T02:12:25.124604+0000","last_undegraded":"2023-10-08T02:12:25.124604+0000","last_fullsized":"2023-10-08T02:12:25.124604+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:31.655738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.137145+0000","last_change":"2023-10-08T02:12:25.135795+0000","last_active":"2023-10-08T02:12:25.137145+0000","last_peered":"2023-10-08T02:12:25.137145+0000","last_clean":"2023-10-08T02:12:25.137145+0000","last_became_active":"2023-10-08T02:12:25.135499+0000","last_became_peered":"2023-10-08T02:12:25.135499+0000","last_unstale":"2023-10-08T02:12:25.137145+0000","last_undegraded":"2023-10-08T02:12:25.137145+0000","last_fullsized":"2023-10-08T02:12:25.137145+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:07.222935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.123810+0000","last_change":"2023-10-08T02:12:25.122103+0000","last_active":"2023-10-08T02:12:25.123810+0000","last_peered":"2023-10-08T02:12:25.123810+0000","last_clean":"2023-10-08T02:12:25.123810+0000","last_became_active":"2023-10-08T02:12:25.121849+0000","last_became_peered":"2023-10-08T02:12:25.121849+0000","last_unstale":"2023-10-08T02:12:25.123810+0000","last_undegraded":"2023-10-08T02:12:25.123810+0000","last_fullsized":"2023-10-08T02:12:25.123810+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:17.278020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138982+0000","last_change":"2023-10-08T02:12:25.134916+0000","last_active":"2023-10-08T02:12:25.138982+0000","last_peered":"2023-10-08T02:12:25.138982+0000","last_clean":"2023-10-08T02:12:25.138982+0000","last_became_active":"2023-10-08T02:12:25.134637+0000","last_became_peered":"2023-10-08T02:12:25.134637+0000","last_unstale":"2023-10-08T02:12:25.138982+0000","last_undegraded":"2023-10-08T02:12:25.138982+0000","last_fullsized":"2023-10-08T02:12:25.138982+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:54.776020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.131614+0000","last_change":"2023-10-08T02:12:25.128693+0000","last_active":"2023-10-08T02:12:25.131614+0000","last_peered":"2023-10-08T02:12:25.131614+0000","last_clean":"2023-10-08T02:12:25.131614+0000","last_became_active":"2023-10-08T02:12:25.128023+0000","last_became_peered":"2023-10-08T02:12:25.128023+0000","last_unstale":"2023-10-08T02:12:25.131614+0000","last_undegraded":"2023-10-08T02:12:25.131614+0000","last_fullsized":"2023-10-08T02:12:25.131614+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:06:39.143538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.133097+0000","last_change":"2023-10-08T02:12:25.129113+0000","last_active":"2023-10-08T02:12:25.133097+0000","last_peered":"2023-10-08T02:12:25.133097+0000","last_clean":"2023-10-08T02:12:25.133097+0000","last_became_active":"2023-10-08T02:12:25.128200+0000","last_became_peered":"2023-10-08T02:12:25.128200+0000","last_unstale":"2023-10-08T02:12:25.133097+0000","last_undegraded":"2023-10-08T02:12:25.133097+0000","last_fullsized":"2023-10-08T02:12:25.133097+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:41:07.222935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.136403+0000","last_change":"2023-10-08T02:12:25.132973+0000","last_active":"2023-10-08T02:12:25.136403+0000","last_peered":"2023-10-08T02:12:25.136403+0000","last_clean":"2023-10-08T02:12:25.136403+0000","last_became_active":"2023-10-08T02:12:25.132701+0000","last_became_peered":"2023-10-08T02:12:25.132701+0000","last_unstale":"2023-10-08T02:12:25.136403+0000","last_undegraded":"2023-10-08T02:12:25.136403+0000","last_fullsized":"2023-10-08T02:12:25.136403+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:51:56.240224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126590+0000","last_change":"2023-10-08T02:12:25.120110+0000","last_active":"2023-10-08T02:12:25.126590+0000","last_peered":"2023-10-08T02:12:25.126590+0000","last_clean":"2023-10-08T02:12:25.126590+0000","last_became_active":"2023-10-08T02:12:25.119450+0000","last_became_peered":"2023-10-08T02:12:25.119450+0000","last_unstale":"2023-10-08T02:12:25.126590+0000","last_undegraded":"2023-10-08T02:12:25.126590+0000","last_fullsized":"2023-10-08T02:12:25.126590+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:16.426669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130294+0000","last_change":"2023-10-08T02:12:25.127106+0000","last_active":"2023-10-08T02:12:25.130294+0000","last_peered":"2023-10-08T02:12:25.130294+0000","last_clean":"2023-10-08T02:12:25.130294+0000","last_became_active":"2023-10-08T02:12:25.126851+0000","last_became_peered":"2023-10-08T02:12:25.126851+0000","last_unstale":"2023-10-08T02:12:25.130294+0000","last_undegraded":"2023-10-08T02:12:25.130294+0000","last_fullsized":"2023-10-08T02:12:25.130294+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:05:37.031644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.127219+0000","last_change":"2023-10-08T02:12:25.116110+0000","last_active":"2023-10-08T02:12:25.127219+0000","last_peered":"2023-10-08T02:12:25.127219+0000","last_clean":"2023-10-08T02:12:25.127219+0000","last_became_active":"2023-10-08T02:12:25.115860+0000","last_became_peered":"2023-10-08T02:12:25.115860+0000","last_unstale":"2023-10-08T02:12:25.127219+0000","last_undegraded":"2023-10-08T02:12:25.127219+0000","last_fullsized":"2023-10-08T02:12:25.127219+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:40:16.426669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.118675+0000","last_change":"2023-10-08T02:12:25.116341+0000","last_active":"2023-10-08T02:12:25.118675+0000","last_peered":"2023-10-08T02:12:25.118675+0000","last_clean":"2023-10-08T02:12:25.118675+0000","last_became_active":"2023-10-08T02:12:25.116073+0000","last_became_peered":"2023-10-08T02:12:25.116073+0000","last_unstale":"2023-10-08T02:12:25.118675+0000","last_undegraded":"2023-10-08T02:12:25.118675+0000","last_fullsized":"2023-10-08T02:12:25.118675+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:32.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126319+0000","last_change":"2023-10-08T02:12:25.119603+0000","last_active":"2023-10-08T02:12:25.126319+0000","last_peered":"2023-10-08T02:12:25.126319+0000","last_clean":"2023-10-08T02:12:25.126319+0000","last_became_active":"2023-10-08T02:12:25.119327+0000","last_became_peered":"2023-10-08T02:12:25.119327+0000","last_unstale":"2023-10-08T02:12:25.126319+0000","last_undegraded":"2023-10-08T02:12:25.126319+0000","last_fullsized":"2023-10-08T02:12:25.126319+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:29:30.293616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.137870+0000","last_change":"2023-10-08T02:12:25.135574+0000","last_active":"2023-10-08T02:12:25.137870+0000","last_peered":"2023-10-08T02:12:25.137870+0000","last_clean":"2023-10-08T02:12:25.137870+0000","last_became_active":"2023-10-08T02:12:25.135270+0000","last_became_peered":"2023-10-08T02:12:25.135270+0000","last_unstale":"2023-10-08T02:12:25.137870+0000","last_undegraded":"2023-10-08T02:12:25.137870+0000","last_fullsized":"2023-10-08T02:12:25.137870+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:29.648405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126297+0000","last_change":"2023-10-08T02:12:25.119570+0000","last_active":"2023-10-08T02:12:25.126297+0000","last_peered":"2023-10-08T02:12:25.126297+0000","last_clean":"2023-10-08T02:12:25.126297+0000","last_became_active":"2023-10-08T02:12:25.119320+0000","last_became_peered":"2023-10-08T02:12:25.119320+0000","last_unstale":"2023-10-08T02:12:25.126297+0000","last_undegraded":"2023-10-08T02:12:25.126297+0000","last_fullsized":"2023-10-08T02:12:25.126297+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:57.502890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.134690+0000","last_change":"2023-10-08T02:12:25.128862+0000","last_active":"2023-10-08T02:12:25.134690+0000","last_peered":"2023-10-08T02:12:25.134690+0000","last_clean":"2023-10-08T02:12:25.134690+0000","last_became_active":"2023-10-08T02:12:25.127885+0000","last_became_peered":"2023-10-08T02:12:25.127885+0000","last_unstale":"2023-10-08T02:12:25.134690+0000","last_undegraded":"2023-10-08T02:12:25.134690+0000","last_fullsized":"2023-10-08T02:12:25.134690+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:29.648405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138830+0000","last_change":"2023-10-08T02:12:25.135989+0000","last_active":"2023-10-08T02:12:25.138830+0000","last_peered":"2023-10-08T02:12:25.138830+0000","last_clean":"2023-10-08T02:12:25.138830+0000","last_became_active":"2023-10-08T02:12:25.135726+0000","last_became_peered":"2023-10-08T02:12:25.135726+0000","last_unstale":"2023-10-08T02:12:25.138830+0000","last_undegraded":"2023-10-08T02:12:25.138830+0000","last_fullsized":"2023-10-08T02:12:25.138830+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:34.173348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129102+0000","last_change":"2023-10-08T02:12:25.121007+0000","last_active":"2023-10-08T02:12:25.129102+0000","last_peered":"2023-10-08T02:12:25.129102+0000","last_clean":"2023-10-08T02:12:25.129102+0000","last_became_active":"2023-10-08T02:12:25.120726+0000","last_became_peered":"2023-10-08T02:12:25.120726+0000","last_unstale":"2023-10-08T02:12:25.129102+0000","last_undegraded":"2023-10-08T02:12:25.129102+0000","last_fullsized":"2023-10-08T02:12:25.129102+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:18.587452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.118594+0000","last_change":"2023-10-08T02:12:25.115621+0000","last_active":"2023-10-08T02:12:25.118594+0000","last_peered":"2023-10-08T02:12:25.118594+0000","last_clean":"2023-10-08T02:12:25.118594+0000","last_became_active":"2023-10-08T02:12:25.115341+0000","last_became_peered":"2023-10-08T02:12:25.115341+0000","last_unstale":"2023-10-08T02:12:25.118594+0000","last_undegraded":"2023-10-08T02:12:25.118594+0000","last_fullsized":"2023-10-08T02:12:25.118594+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:39.311726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.113749+0000","last_change":"2023-10-08T02:12:25.112605+0000","last_active":"2023-10-08T02:12:25.113749+0000","last_peered":"2023-10-08T02:12:25.113749+0000","last_clean":"2023-10-08T02:12:25.113749+0000","last_became_active":"2023-10-08T02:12:25.112334+0000","last_became_peered":"2023-10-08T02:12:25.112334+0000","last_unstale":"2023-10-08T02:12:25.113749+0000","last_undegraded":"2023-10-08T02:12:25.113749+0000","last_fullsized":"2023-10-08T02:12:25.113749+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:37.774429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.113652+0000","last_change":"2023-10-08T02:12:25.111752+0000","last_active":"2023-10-08T02:12:25.113652+0000","last_peered":"2023-10-08T02:12:25.113652+0000","last_clean":"2023-10-08T02:12:25.113652+0000","last_became_active":"2023-10-08T02:12:25.111507+0000","last_became_peered":"2023-10-08T02:12:25.111507+0000","last_unstale":"2023-10-08T02:12:25.113652+0000","last_undegraded":"2023-10-08T02:12:25.113652+0000","last_fullsized":"2023-10-08T02:12:25.113652+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:06.678907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122234+0000","last_change":"2023-10-08T02:12:25.119382+0000","last_active":"2023-10-08T02:12:25.122234+0000","last_peered":"2023-10-08T02:12:25.122234+0000","last_clean":"2023-10-08T02:12:25.122234+0000","last_became_active":"2023-10-08T02:12:25.119143+0000","last_became_peered":"2023-10-08T02:12:25.119143+0000","last_unstale":"2023-10-08T02:12:25.122234+0000","last_undegraded":"2023-10-08T02:12:25.122234+0000","last_fullsized":"2023-10-08T02:12:25.122234+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:18.587452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129236+0000","last_change":"2023-10-08T02:12:25.123913+0000","last_active":"2023-10-08T02:12:25.129236+0000","last_peered":"2023-10-08T02:12:25.129236+0000","last_clean":"2023-10-08T02:12:25.129236+0000","last_became_active":"2023-10-08T02:12:25.123675+0000","last_became_peered":"2023-10-08T02:12:25.123675+0000","last_unstale":"2023-10-08T02:12:25.129236+0000","last_undegraded":"2023-10-08T02:12:25.129236+0000","last_fullsized":"2023-10-08T02:12:25.129236+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:13.345275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.128323+0000","last_change":"2023-10-08T02:12:25.125205+0000","last_active":"2023-10-08T02:12:25.128323+0000","last_peered":"2023-10-08T02:12:25.128323+0000","last_clean":"2023-10-08T02:12:25.128323+0000","last_became_active":"2023-10-08T02:12:25.124966+0000","last_became_peered":"2023-10-08T02:12:25.124966+0000","last_unstale":"2023-10-08T02:12:25.128323+0000","last_undegraded":"2023-10-08T02:12:25.128323+0000","last_fullsized":"2023-10-08T02:12:25.128323+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:54.429689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126606+0000","last_change":"2023-10-08T02:12:25.120295+0000","last_active":"2023-10-08T02:12:25.126606+0000","last_peered":"2023-10-08T02:12:25.126606+0000","last_clean":"2023-10-08T02:12:25.126606+0000","last_became_active":"2023-10-08T02:12:25.119861+0000","last_became_peered":"2023-10-08T02:12:25.119861+0000","last_unstale":"2023-10-08T02:12:25.126606+0000","last_undegraded":"2023-10-08T02:12:25.126606+0000","last_fullsized":"2023-10-08T02:12:25.126606+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:45.356324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.137096+0000","last_change":"2023-10-08T02:12:25.135040+0000","last_active":"2023-10-08T02:12:25.137096+0000","last_peered":"2023-10-08T02:12:25.137096+0000","last_clean":"2023-10-08T02:12:25.137096+0000","last_became_active":"2023-10-08T02:12:25.134323+0000","last_became_peered":"2023-10-08T02:12:25.134323+0000","last_unstale":"2023-10-08T02:12:25.137096+0000","last_undegraded":"2023-10-08T02:12:25.137096+0000","last_fullsized":"2023-10-08T02:12:25.137096+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:52.000775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"170'4","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.139274+0000","last_change":"2023-10-08T02:12:24.129340+0000","last_active":"2023-10-08T02:12:25.139274+0000","last_peered":"2023-10-08T02:12:25.139274+0000","last_clean":"2023-10-08T02:12:25.139274+0000","last_became_active":"2023-10-08T02:12:24.127369+0000","last_became_peered":"2023-10-08T02:12:24.127369+0000","last_unstale":"2023-10-08T02:12:25.139274+0000","last_undegraded":"2023-10-08T02:12:25.139274+0000","last_fullsized":"2023-10-08T02:12:25.139274+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:30.649373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.117579+0000","last_change":"2023-10-08T02:12:25.114794+0000","last_active":"2023-10-08T02:12:25.117579+0000","last_peered":"2023-10-08T02:12:25.117579+0000","last_clean":"2023-10-08T02:12:25.117579+0000","last_became_active":"2023-10-08T02:12:25.114561+0000","last_became_peered":"2023-10-08T02:12:25.114561+0000","last_unstale":"2023-10-08T02:12:25.117579+0000","last_undegraded":"2023-10-08T02:12:25.117579+0000","last_fullsized":"2023-10-08T02:12:25.117579+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:11:32.616990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138103+0000","last_change":"2023-10-08T02:12:24.126839+0000","last_active":"2023-10-08T02:12:25.138103+0000","last_peered":"2023-10-08T02:12:25.138103+0000","last_clean":"2023-10-08T02:12:25.138103+0000","last_became_active":"2023-10-08T02:12:24.126581+0000","last_became_peered":"2023-10-08T02:12:24.126581+0000","last_unstale":"2023-10-08T02:12:25.138103+0000","last_undegraded":"2023-10-08T02:12:25.138103+0000","last_fullsized":"2023-10-08T02:12:25.138103+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:29:29.287252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.128272+0000","last_change":"2023-10-08T02:12:25.126693+0000","last_active":"2023-10-08T02:12:25.128272+0000","last_peered":"2023-10-08T02:12:25.128272+0000","last_clean":"2023-10-08T02:12:25.128272+0000","last_became_active":"2023-10-08T02:12:25.126415+0000","last_became_peered":"2023-10-08T02:12:25.126415+0000","last_unstale":"2023-10-08T02:12:25.128272+0000","last_undegraded":"2023-10-08T02:12:25.128272+0000","last_fullsized":"2023-10-08T02:12:25.128272+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:50.681386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141415+0000","last_change":"2023-10-08T02:12:24.120849+0000","last_active":"2023-10-08T02:12:25.141415+0000","last_peered":"2023-10-08T02:12:25.141415+0000","last_clean":"2023-10-08T02:12:25.141415+0000","last_became_active":"2023-10-08T02:12:24.120518+0000","last_became_peered":"2023-10-08T02:12:24.120518+0000","last_unstale":"2023-10-08T02:12:25.141415+0000","last_undegraded":"2023-10-08T02:12:25.141415+0000","last_fullsized":"2023-10-08T02:12:25.141415+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:18:06.476543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.137056+0000","last_change":"2023-10-08T02:12:25.135441+0000","last_active":"2023-10-08T02:12:25.137056+0000","last_peered":"2023-10-08T02:12:25.137056+0000","last_clean":"2023-10-08T02:12:25.137056+0000","last_became_active":"2023-10-08T02:12:25.135107+0000","last_became_peered":"2023-10-08T02:12:25.135107+0000","last_unstale":"2023-10-08T02:12:25.137056+0000","last_undegraded":"2023-10-08T02:12:25.137056+0000","last_fullsized":"2023-10-08T02:12:25.137056+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:13.345275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141207+0000","last_change":"2023-10-08T02:12:24.117074+0000","last_active":"2023-10-08T02:12:25.141207+0000","last_peered":"2023-10-08T02:12:25.141207+0000","last_clean":"2023-10-08T02:12:25.141207+0000","last_became_active":"2023-10-08T02:12:24.116819+0000","last_became_peered":"2023-10-08T02:12:24.116819+0000","last_unstale":"2023-10-08T02:12:25.141207+0000","last_undegraded":"2023-10-08T02:12:25.141207+0000","last_fullsized":"2023-10-08T02:12:25.141207+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:26.197740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.127655+0000","last_change":"2023-10-08T02:12:25.126125+0000","last_active":"2023-10-08T02:12:25.127655+0000","last_peered":"2023-10-08T02:12:25.127655+0000","last_clean":"2023-10-08T02:12:25.127655+0000","last_became_active":"2023-10-08T02:12:25.125845+0000","last_became_peered":"2023-10-08T02:12:25.125845+0000","last_unstale":"2023-10-08T02:12:25.127655+0000","last_undegraded":"2023-10-08T02:12:25.127655+0000","last_fullsized":"2023-10-08T02:12:25.127655+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:35.429110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.140873+0000","last_change":"2023-10-08T02:12:24.127689+0000","last_active":"2023-10-08T02:12:25.140873+0000","last_peered":"2023-10-08T02:12:25.140873+0000","last_clean":"2023-10-08T02:12:25.140873+0000","last_became_active":"2023-10-08T02:12:24.127406+0000","last_became_peered":"2023-10-08T02:12:24.127406+0000","last_unstale":"2023-10-08T02:12:25.140873+0000","last_undegraded":"2023-10-08T02:12:25.140873+0000","last_fullsized":"2023-10-08T02:12:25.140873+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:42.653885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.138267+0000","last_change":"2023-10-08T02:12:25.130873+0000","last_active":"2023-10-08T02:12:25.138267+0000","last_peered":"2023-10-08T02:12:25.138267+0000","last_clean":"2023-10-08T02:12:25.138267+0000","last_became_active":"2023-10-08T02:12:25.130564+0000","last_became_peered":"2023-10-08T02:12:25.130564+0000","last_unstale":"2023-10-08T02:12:25.138267+0000","last_undegraded":"2023-10-08T02:12:25.138267+0000","last_fullsized":"2023-10-08T02:12:25.138267+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:41:54.776020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141242+0000","last_change":"2023-10-08T02:12:24.134435+0000","last_active":"2023-10-08T02:12:25.141242+0000","last_peered":"2023-10-08T02:12:25.141242+0000","last_clean":"2023-10-08T02:12:25.141242+0000","last_became_active":"2023-10-08T02:12:24.134172+0000","last_became_peered":"2023-10-08T02:12:24.134172+0000","last_unstale":"2023-10-08T02:12:25.141242+0000","last_undegraded":"2023-10-08T02:12:25.141242+0000","last_fullsized":"2023-10-08T02:12:25.141242+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:17.581087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.118043+0000","last_change":"2023-10-08T02:12:25.117306+0000","last_active":"2023-10-08T02:12:25.118043+0000","last_peered":"2023-10-08T02:12:25.118043+0000","last_clean":"2023-10-08T02:12:25.118043+0000","last_became_active":"2023-10-08T02:12:25.117061+0000","last_became_peered":"2023-10-08T02:12:25.117061+0000","last_unstale":"2023-10-08T02:12:25.118043+0000","last_undegraded":"2023-10-08T02:12:25.118043+0000","last_fullsized":"2023-10-08T02:12:25.118043+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:27.746043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.110586+0000","last_change":"2023-10-08T02:12:24.115928+0000","last_active":"2023-10-08T02:12:25.110586+0000","last_peered":"2023-10-08T02:12:25.110586+0000","last_clean":"2023-10-08T02:12:25.110586+0000","last_became_active":"2023-10-08T02:12:24.115667+0000","last_became_peered":"2023-10-08T02:12:24.115667+0000","last_unstale":"2023-10-08T02:12:25.110586+0000","last_undegraded":"2023-10-08T02:12:25.110586+0000","last_fullsized":"2023-10-08T02:12:25.110586+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:41:26.992916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.123376+0000","last_change":"2023-10-08T02:12:25.120217+0000","last_active":"2023-10-08T02:12:25.123376+0000","last_peered":"2023-10-08T02:12:25.123376+0000","last_clean":"2023-10-08T02:12:25.123376+0000","last_became_active":"2023-10-08T02:12:25.119954+0000","last_became_peered":"2023-10-08T02:12:25.119954+0000","last_unstale":"2023-10-08T02:12:25.123376+0000","last_undegraded":"2023-10-08T02:12:25.123376+0000","last_fullsized":"2023-10-08T02:12:25.123376+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:47.858013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.111471+0000","last_change":"2023-10-08T02:12:24.130840+0000","last_active":"2023-10-08T02:12:25.111471+0000","last_peered":"2023-10-08T02:12:25.111471+0000","last_clean":"2023-10-08T02:12:25.111471+0000","last_became_active":"2023-10-08T02:12:24.130532+0000","last_became_peered":"2023-10-08T02:12:24.130532+0000","last_unstale":"2023-10-08T02:12:25.111471+0000","last_undegraded":"2023-10-08T02:12:25.111471+0000","last_fullsized":"2023-10-08T02:12:25.111471+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:50.220057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.134154+0000","last_change":"2023-10-08T02:12:25.130739+0000","last_active":"2023-10-08T02:12:25.134154+0000","last_peered":"2023-10-08T02:12:25.134154+0000","last_clean":"2023-10-08T02:12:25.134154+0000","last_became_active":"2023-10-08T02:12:25.130478+0000","last_became_peered":"2023-10-08T02:12:25.130478+0000","last_unstale":"2023-10-08T02:12:25.134154+0000","last_undegraded":"2023-10-08T02:12:25.134154+0000","last_fullsized":"2023-10-08T02:12:25.134154+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:35.429110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.110397+0000","last_change":"2023-10-08T02:12:24.119252+0000","last_active":"2023-10-08T02:12:25.110397+0000","last_peered":"2023-10-08T02:12:25.110397+0000","last_clean":"2023-10-08T02:12:25.110397+0000","last_became_active":"2023-10-08T02:12:24.119019+0000","last_became_peered":"2023-10-08T02:12:24.119019+0000","last_unstale":"2023-10-08T02:12:25.110397+0000","last_undegraded":"2023-10-08T02:12:25.110397+0000","last_fullsized":"2023-10-08T02:12:25.110397+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:14:36.768065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130313+0000","last_change":"2023-10-08T02:12:25.128058+0000","last_active":"2023-10-08T02:12:25.130313+0000","last_peered":"2023-10-08T02:12:25.130313+0000","last_clean":"2023-10-08T02:12:25.130313+0000","last_became_active":"2023-10-08T02:12:25.127795+0000","last_became_peered":"2023-10-08T02:12:25.127795+0000","last_unstale":"2023-10-08T02:12:25.130313+0000","last_undegraded":"2023-10-08T02:12:25.130313+0000","last_fullsized":"2023-10-08T02:12:25.130313+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:25:33.509282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.116751+0000","last_change":"2023-10-08T02:12:24.130902+0000","last_active":"2023-10-08T02:12:25.116751+0000","last_peered":"2023-10-08T02:12:25.116751+0000","last_clean":"2023-10-08T02:12:25.116751+0000","last_became_active":"2023-10-08T02:12:24.130383+0000","last_became_peered":"2023-10-08T02:12:24.130383+0000","last_unstale":"2023-10-08T02:12:25.116751+0000","last_undegraded":"2023-10-08T02:12:25.116751+0000","last_fullsized":"2023-10-08T02:12:25.116751+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:34:26.739678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.113768+0000","last_change":"2023-10-08T02:12:25.112840+0000","last_active":"2023-10-08T02:12:25.113768+0000","last_peered":"2023-10-08T02:12:25.113768+0000","last_clean":"2023-10-08T02:12:25.113768+0000","last_became_active":"2023-10-08T02:12:25.112584+0000","last_became_peered":"2023-10-08T02:12:25.112584+0000","last_unstale":"2023-10-08T02:12:25.113768+0000","last_undegraded":"2023-10-08T02:12:25.113768+0000","last_fullsized":"2023-10-08T02:12:25.113768+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:12.015550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"170'6","reported_seq":16,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.143767+0000","last_change":"2023-10-08T02:12:24.113457+0000","last_active":"2023-10-08T02:12:25.143767+0000","last_peered":"2023-10-08T02:12:25.143767+0000","last_clean":"2023-10-08T02:12:25.143767+0000","last_became_active":"2023-10-08T02:12:24.113148+0000","last_became_peered":"2023-10-08T02:12:24.113148+0000","last_unstale":"2023-10-08T02:12:25.143767+0000","last_undegraded":"2023-10-08T02:12:25.143767+0000","last_fullsized":"2023-10-08T02:12:25.143767+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:33:38.869273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130139+0000","last_change":"2023-10-08T02:12:25.121235+0000","last_active":"2023-10-08T02:12:25.130139+0000","last_peered":"2023-10-08T02:12:25.130139+0000","last_clean":"2023-10-08T02:12:25.130139+0000","last_became_active":"2023-10-08T02:12:25.121045+0000","last_became_peered":"2023-10-08T02:12:25.121045+0000","last_unstale":"2023-10-08T02:12:25.130139+0000","last_undegraded":"2023-10-08T02:12:25.130139+0000","last_fullsized":"2023-10-08T02:12:25.130139+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:13:18.587452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141742+0000","last_change":"2023-10-08T02:12:24.116681+0000","last_active":"2023-10-08T02:12:25.141742+0000","last_peered":"2023-10-08T02:12:25.141742+0000","last_clean":"2023-10-08T02:12:25.141742+0000","last_became_active":"2023-10-08T02:12:24.116381+0000","last_became_peered":"2023-10-08T02:12:24.116381+0000","last_unstale":"2023-10-08T02:12:25.141742+0000","last_undegraded":"2023-10-08T02:12:25.141742+0000","last_fullsized":"2023-10-08T02:12:25.141742+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:20:02.429294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.114824+0000","last_change":"2023-10-08T02:12:25.114200+0000","last_active":"2023-10-08T02:12:25.114824+0000","last_peered":"2023-10-08T02:12:25.114824+0000","last_clean":"2023-10-08T02:12:25.114824+0000","last_became_active":"2023-10-08T02:12:25.113933+0000","last_became_peered":"2023-10-08T02:12:25.113933+0000","last_unstale":"2023-10-08T02:12:25.114824+0000","last_undegraded":"2023-10-08T02:12:25.114824+0000","last_fullsized":"2023-10-08T02:12:25.114824+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:17.278020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.106790+0000","last_change":"2023-10-08T02:12:24.120101+0000","last_active":"2023-10-08T02:12:25.106790+0000","last_peered":"2023-10-08T02:12:25.106790+0000","last_clean":"2023-10-08T02:12:25.106790+0000","last_became_active":"2023-10-08T02:12:24.119835+0000","last_became_peered":"2023-10-08T02:12:24.119835+0000","last_unstale":"2023-10-08T02:12:25.106790+0000","last_undegraded":"2023-10-08T02:12:25.106790+0000","last_fullsized":"2023-10-08T02:12:25.106790+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:00:54.743852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.131677+0000","last_change":"2023-10-08T02:12:25.127127+0000","last_active":"2023-10-08T02:12:25.131677+0000","last_peered":"2023-10-08T02:12:25.131677+0000","last_clean":"2023-10-08T02:12:25.131677+0000","last_became_active":"2023-10-08T02:12:25.126835+0000","last_became_peered":"2023-10-08T02:12:25.126835+0000","last_unstale":"2023-10-08T02:12:25.131677+0000","last_undegraded":"2023-10-08T02:12:25.131677+0000","last_fullsized":"2023-10-08T02:12:25.131677+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:51.226421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.107297+0000","last_change":"2023-10-08T02:12:24.119093+0000","last_active":"2023-10-08T02:12:25.107297+0000","last_peered":"2023-10-08T02:12:25.107297+0000","last_clean":"2023-10-08T02:12:25.107297+0000","last_became_active":"2023-10-08T02:12:24.118840+0000","last_became_peered":"2023-10-08T02:12:24.118840+0000","last_unstale":"2023-10-08T02:12:25.107297+0000","last_undegraded":"2023-10-08T02:12:25.107297+0000","last_fullsized":"2023-10-08T02:12:25.107297+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:47:14.786069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.127456+0000","last_change":"2023-10-08T02:12:25.119744+0000","last_active":"2023-10-08T02:12:25.127456+0000","last_peered":"2023-10-08T02:12:25.127456+0000","last_clean":"2023-10-08T02:12:25.127456+0000","last_became_active":"2023-10-08T02:12:25.119484+0000","last_became_peered":"2023-10-08T02:12:25.119484+0000","last_unstale":"2023-10-08T02:12:25.127456+0000","last_undegraded":"2023-10-08T02:12:25.127456+0000","last_fullsized":"2023-10-08T02:12:25.127456+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:12.015550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"170'8","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.142279+0000","last_change":"2023-10-08T02:12:24.114740+0000","last_active":"2023-10-08T02:12:25.142279+0000","last_peered":"2023-10-08T02:12:25.142279+0000","last_clean":"2023-10-08T02:12:25.142279+0000","last_became_active":"2023-10-08T02:12:24.114464+0000","last_became_peered":"2023-10-08T02:12:24.114464+0000","last_unstale":"2023-10-08T02:12:25.142279+0000","last_undegraded":"2023-10-08T02:12:25.142279+0000","last_fullsized":"2023-10-08T02:12:25.142279+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:35.512125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130151+0000","last_change":"2023-10-08T02:12:25.126757+0000","last_active":"2023-10-08T02:12:25.130151+0000","last_peered":"2023-10-08T02:12:25.130151+0000","last_clean":"2023-10-08T02:12:25.130151+0000","last_became_active":"2023-10-08T02:12:25.126434+0000","last_became_peered":"2023-10-08T02:12:25.126434+0000","last_unstale":"2023-10-08T02:12:25.130151+0000","last_undegraded":"2023-10-08T02:12:25.130151+0000","last_fullsized":"2023-10-08T02:12:25.130151+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:30:57.502890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141160+0000","last_change":"2023-10-08T02:12:24.119874+0000","last_active":"2023-10-08T02:12:25.141160+0000","last_peered":"2023-10-08T02:12:25.141160+0000","last_clean":"2023-10-08T02:12:25.141160+0000","last_became_active":"2023-10-08T02:12:24.119567+0000","last_became_peered":"2023-10-08T02:12:24.119567+0000","last_unstale":"2023-10-08T02:12:25.141160+0000","last_undegraded":"2023-10-08T02:12:25.141160+0000","last_fullsized":"2023-10-08T02:12:25.141160+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:31:57.955586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.130018+0000","last_change":"2023-10-08T02:12:25.128147+0000","last_active":"2023-10-08T02:12:25.130018+0000","last_peered":"2023-10-08T02:12:25.130018+0000","last_clean":"2023-10-08T02:12:25.130018+0000","last_became_active":"2023-10-08T02:12:25.127835+0000","last_became_peered":"2023-10-08T02:12:25.127835+0000","last_unstale":"2023-10-08T02:12:25.130018+0000","last_undegraded":"2023-10-08T02:12:25.130018+0000","last_fullsized":"2023-10-08T02:12:25.130018+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:31:58.961951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.140222+0000","last_change":"2023-10-08T02:12:24.108816+0000","last_active":"2023-10-08T02:12:25.140222+0000","last_peered":"2023-10-08T02:12:25.140222+0000","last_clean":"2023-10-08T02:12:25.140222+0000","last_became_active":"2023-10-08T02:12:24.108533+0000","last_became_peered":"2023-10-08T02:12:24.108533+0000","last_unstale":"2023-10-08T02:12:25.140222+0000","last_undegraded":"2023-10-08T02:12:25.140222+0000","last_fullsized":"2023-10-08T02:12:25.140222+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:28:26.497367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.132164+0000","last_change":"2023-10-08T02:12:25.122104+0000","last_active":"2023-10-08T02:12:25.132164+0000","last_peered":"2023-10-08T02:12:25.132164+0000","last_clean":"2023-10-08T02:12:25.132164+0000","last_became_active":"2023-10-08T02:12:25.121839+0000","last_became_peered":"2023-10-08T02:12:25.121839+0000","last_unstale":"2023-10-08T02:12:25.132164+0000","last_undegraded":"2023-10-08T02:12:25.132164+0000","last_fullsized":"2023-10-08T02:12:25.132164+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:28:45.356324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"170'1","reported_seq":11,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.139685+0000","last_change":"2023-10-08T02:12:24.126998+0000","last_active":"2023-10-08T02:12:25.139685+0000","last_peered":"2023-10-08T02:12:25.139685+0000","last_clean":"2023-10-08T02:12:25.139685+0000","last_became_active":"2023-10-08T02:12:24.126700+0000","last_became_peered":"2023-10-08T02:12:24.126700+0000","last_unstale":"2023-10-08T02:12:25.139685+0000","last_undegraded":"2023-10-08T02:12:25.139685+0000","last_fullsized":"2023-10-08T02:12:25.139685+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.136401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122183+0000","last_change":"2023-10-08T02:12:25.120029+0000","last_active":"2023-10-08T02:12:25.122183+0000","last_peered":"2023-10-08T02:12:25.122183+0000","last_clean":"2023-10-08T02:12:25.122183+0000","last_became_active":"2023-10-08T02:12:25.119796+0000","last_became_peered":"2023-10-08T02:12:25.119796+0000","last_unstale":"2023-10-08T02:12:25.122183+0000","last_undegraded":"2023-10-08T02:12:25.122183+0000","last_fullsized":"2023-10-08T02:12:25.122183+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:29.543213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.111159+0000","last_change":"2023-10-08T02:12:24.114716+0000","last_active":"2023-10-08T02:12:25.111159+0000","last_peered":"2023-10-08T02:12:25.111159+0000","last_clean":"2023-10-08T02:12:25.111159+0000","last_became_active":"2023-10-08T02:12:24.114464+0000","last_became_peered":"2023-10-08T02:12:24.114464+0000","last_unstale":"2023-10-08T02:12:25.111159+0000","last_undegraded":"2023-10-08T02:12:25.111159+0000","last_fullsized":"2023-10-08T02:12:25.111159+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:47:11.009186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129217+0000","last_change":"2023-10-08T02:12:25.123285+0000","last_active":"2023-10-08T02:12:25.129217+0000","last_peered":"2023-10-08T02:12:25.129217+0000","last_clean":"2023-10-08T02:12:25.129217+0000","last_became_active":"2023-10-08T02:12:25.123034+0000","last_became_peered":"2023-10-08T02:12:25.123034+0000","last_unstale":"2023-10-08T02:12:25.129217+0000","last_undegraded":"2023-10-08T02:12:25.129217+0000","last_fullsized":"2023-10-08T02:12:25.129217+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:31:52.000775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.141137+0000","last_change":"2023-10-08T02:12:24.119426+0000","last_active":"2023-10-08T02:12:25.141137+0000","last_peered":"2023-10-08T02:12:25.141137+0000","last_clean":"2023-10-08T02:12:25.141137+0000","last_became_active":"2023-10-08T02:12:24.119135+0000","last_became_peered":"2023-10-08T02:12:24.119135+0000","last_unstale":"2023-10-08T02:12:25.141137+0000","last_undegraded":"2023-10-08T02:12:25.141137+0000","last_fullsized":"2023-10-08T02:12:25.141137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:18:10.627672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.123808+0000","last_change":"2023-10-08T02:12:25.123130+0000","last_active":"2023-10-08T02:12:25.123808+0000","last_peered":"2023-10-08T02:12:25.123808+0000","last_clean":"2023-10-08T02:12:25.123808+0000","last_became_active":"2023-10-08T02:12:25.122884+0000","last_became_peered":"2023-10-08T02:12:25.122884+0000","last_unstale":"2023-10-08T02:12:25.123808+0000","last_undegraded":"2023-10-08T02:12:25.123808+0000","last_fullsized":"2023-10-08T02:12:25.123808+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:28:12.679734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.114898+0000","last_change":"2023-10-08T02:12:24.129065+0000","last_active":"2023-10-08T02:12:25.114898+0000","last_peered":"2023-10-08T02:12:25.114898+0000","last_clean":"2023-10-08T02:12:25.114898+0000","last_became_active":"2023-10-08T02:12:24.128711+0000","last_became_peered":"2023-10-08T02:12:24.128711+0000","last_unstale":"2023-10-08T02:12:25.114898+0000","last_undegraded":"2023-10-08T02:12:25.114898+0000","last_fullsized":"2023-10-08T02:12:25.114898+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:53.423324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.132538+0000","last_change":"2023-10-08T02:12:25.127609+0000","last_active":"2023-10-08T02:12:25.132538+0000","last_peered":"2023-10-08T02:12:25.132538+0000","last_clean":"2023-10-08T02:12:25.132538+0000","last_became_active":"2023-10-08T02:12:25.126204+0000","last_became_peered":"2023-10-08T02:12:25.126204+0000","last_unstale":"2023-10-08T02:12:25.132538+0000","last_undegraded":"2023-10-08T02:12:25.132538+0000","last_fullsized":"2023-10-08T02:12:25.132538+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:34.173348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.114623+0000","last_change":"2023-10-08T02:12:24.130331+0000","last_active":"2023-10-08T02:12:25.114623+0000","last_peered":"2023-10-08T02:12:25.114623+0000","last_clean":"2023-10-08T02:12:25.114623+0000","last_became_active":"2023-10-08T02:12:24.129961+0000","last_became_peered":"2023-10-08T02:12:24.129961+0000","last_unstale":"2023-10-08T02:12:25.114623+0000","last_undegraded":"2023-10-08T02:12:25.114623+0000","last_fullsized":"2023-10-08T02:12:25.114623+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:46.851648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.123491+0000","last_change":"2023-10-08T02:12:25.119504+0000","last_active":"2023-10-08T02:12:25.123491+0000","last_peered":"2023-10-08T02:12:25.123491+0000","last_clean":"2023-10-08T02:12:25.123491+0000","last_became_active":"2023-10-08T02:12:25.119249+0000","last_became_peered":"2023-10-08T02:12:25.119249+0000","last_unstale":"2023-10-08T02:12:25.123491+0000","last_undegraded":"2023-10-08T02:12:25.123491+0000","last_fullsized":"2023-10-08T02:12:25.123491+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:33:28.172639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.106560+0000","last_change":"2023-10-08T02:12:24.109467+0000","last_active":"2023-10-08T02:12:25.106560+0000","last_peered":"2023-10-08T02:12:25.106560+0000","last_clean":"2023-10-08T02:12:25.106560+0000","last_became_active":"2023-10-08T02:12:24.108744+0000","last_became_peered":"2023-10-08T02:12:24.108744+0000","last_unstale":"2023-10-08T02:12:25.106560+0000","last_undegraded":"2023-10-08T02:12:25.106560+0000","last_fullsized":"2023-10-08T02:12:25.106560+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:00:35.512125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.139748+0000","last_change":"2023-10-08T02:12:25.136826+0000","last_active":"2023-10-08T02:12:25.139748+0000","last_peered":"2023-10-08T02:12:25.139748+0000","last_clean":"2023-10-08T02:12:25.139748+0000","last_became_active":"2023-10-08T02:12:25.136531+0000","last_became_peered":"2023-10-08T02:12:25.136531+0000","last_unstale":"2023-10-08T02:12:25.139748+0000","last_undegraded":"2023-10-08T02:12:25.139748+0000","last_fullsized":"2023-10-08T02:12:25.139748+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:27.204105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"170'2","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.140354+0000","last_change":"2023-10-08T02:12:24.129117+0000","last_active":"2023-10-08T02:12:25.140354+0000","last_peered":"2023-10-08T02:12:25.140354+0000","last_clean":"2023-10-08T02:12:25.140354+0000","last_became_active":"2023-10-08T02:12:24.128826+0000","last_became_peered":"2023-10-08T02:12:24.128826+0000","last_unstale":"2023-10-08T02:12:25.140354+0000","last_undegraded":"2023-10-08T02:12:25.140354+0000","last_fullsized":"2023-10-08T02:12:25.140354+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:46:28.536848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.127615+0000","last_change":"2023-10-08T02:12:25.120691+0000","last_active":"2023-10-08T02:12:25.127615+0000","last_peered":"2023-10-08T02:12:25.127615+0000","last_clean":"2023-10-08T02:12:25.127615+0000","last_became_active":"2023-10-08T02:12:25.120426+0000","last_became_peered":"2023-10-08T02:12:25.120426+0000","last_unstale":"2023-10-08T02:12:25.127615+0000","last_undegraded":"2023-10-08T02:12:25.127615+0000","last_fullsized":"2023-10-08T02:12:25.127615+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:00.403217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.108217+0000","last_change":"2023-10-08T02:12:24.131763+0000","last_active":"2023-10-08T02:12:25.108217+0000","last_peered":"2023-10-08T02:12:25.108217+0000","last_clean":"2023-10-08T02:12:25.108217+0000","last_became_active":"2023-10-08T02:12:24.131500+0000","last_became_peered":"2023-10-08T02:12:24.131500+0000","last_unstale":"2023-10-08T02:12:25.108217+0000","last_undegraded":"2023-10-08T02:12:25.108217+0000","last_fullsized":"2023-10-08T02:12:25.108217+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:41.052726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.125862+0000","last_change":"2023-10-08T02:12:25.120637+0000","last_active":"2023-10-08T02:12:25.125862+0000","last_peered":"2023-10-08T02:12:25.125862+0000","last_clean":"2023-10-08T02:12:25.125862+0000","last_became_active":"2023-10-08T02:12:25.120348+0000","last_became_peered":"2023-10-08T02:12:25.120348+0000","last_unstale":"2023-10-08T02:12:25.125862+0000","last_undegraded":"2023-10-08T02:12:25.125862+0000","last_fullsized":"2023-10-08T02:12:25.125862+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:57:47.858013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.109433+0000","last_change":"2023-10-08T02:12:24.121322+0000","last_active":"2023-10-08T02:12:25.109433+0000","last_peered":"2023-10-08T02:12:25.109433+0000","last_clean":"2023-10-08T02:12:25.109433+0000","last_became_active":"2023-10-08T02:12:24.121017+0000","last_became_peered":"2023-10-08T02:12:24.121017+0000","last_unstale":"2023-10-08T02:12:25.109433+0000","last_undegraded":"2023-10-08T02:12:25.109433+0000","last_fullsized":"2023-10-08T02:12:25.109433+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:05:02.557364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129932+0000","last_change":"2023-10-08T02:12:25.125302+0000","last_active":"2023-10-08T02:12:25.129932+0000","last_peered":"2023-10-08T02:12:25.129932+0000","last_clean":"2023-10-08T02:12:25.129932+0000","last_became_active":"2023-10-08T02:12:25.125045+0000","last_became_peered":"2023-10-08T02:12:25.125045+0000","last_unstale":"2023-10-08T02:12:25.129932+0000","last_undegraded":"2023-10-08T02:12:25.129932+0000","last_fullsized":"2023-10-08T02:12:25.129932+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:57:04.274497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.108875+0000","last_change":"2023-10-08T02:12:24.120609+0000","last_active":"2023-10-08T02:12:25.108875+0000","last_peered":"2023-10-08T02:12:25.108875+0000","last_clean":"2023-10-08T02:12:25.108875+0000","last_became_active":"2023-10-08T02:12:24.120253+0000","last_became_peered":"2023-10-08T02:12:24.120253+0000","last_unstale":"2023-10-08T02:12:25.108875+0000","last_undegraded":"2023-10-08T02:12:25.108875+0000","last_fullsized":"2023-10-08T02:12:25.108875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:40:26.108797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.129915+0000","last_change":"2023-10-08T02:12:25.127167+0000","last_active":"2023-10-08T02:12:25.129915+0000","last_peered":"2023-10-08T02:12:25.129915+0000","last_clean":"2023-10-08T02:12:25.129915+0000","last_became_active":"2023-10-08T02:12:25.126867+0000","last_became_peered":"2023-10-08T02:12:25.126867+0000","last_unstale":"2023-10-08T02:12:25.129915+0000","last_undegraded":"2023-10-08T02:12:25.129915+0000","last_fullsized":"2023-10-08T02:12:25.129915+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:08:00.403217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.106241+0000","last_change":"2023-10-08T02:12:24.122209+0000","last_active":"2023-10-08T02:12:25.106241+0000","last_peered":"2023-10-08T02:12:25.106241+0000","last_clean":"2023-10-08T02:12:25.106241+0000","last_became_active":"2023-10-08T02:12:24.121963+0000","last_became_peered":"2023-10-08T02:12:24.121963+0000","last_unstale":"2023-10-08T02:12:25.106241+0000","last_undegraded":"2023-10-08T02:12:25.106241+0000","last_fullsized":"2023-10-08T02:12:25.106241+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:32:49.145683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.134184+0000","last_change":"2023-10-08T02:12:25.131333+0000","last_active":"2023-10-08T02:12:25.134184+0000","last_peered":"2023-10-08T02:12:25.134184+0000","last_clean":"2023-10-08T02:12:25.134184+0000","last_became_active":"2023-10-08T02:12:25.131087+0000","last_became_peered":"2023-10-08T02:12:25.131087+0000","last_unstale":"2023-10-08T02:12:25.134184+0000","last_undegraded":"2023-10-08T02:12:25.134184+0000","last_fullsized":"2023-10-08T02:12:25.134184+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:03:31.655738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.114808+0000","last_change":"2023-10-08T02:12:24.128461+0000","last_active":"2023-10-08T02:12:25.114808+0000","last_peered":"2023-10-08T02:12:25.114808+0000","last_clean":"2023-10-08T02:12:25.114808+0000","last_became_active":"2023-10-08T02:12:24.126911+0000","last_became_peered":"2023-10-08T02:12:24.126911+0000","last_unstale":"2023-10-08T02:12:25.114808+0000","last_undegraded":"2023-10-08T02:12:25.114808+0000","last_fullsized":"2023-10-08T02:12:25.114808+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:09:34.422745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.136308+0000","last_change":"2023-10-08T02:12:25.132146+0000","last_active":"2023-10-08T02:12:25.136308+0000","last_peered":"2023-10-08T02:12:25.136308+0000","last_clean":"2023-10-08T02:12:25.136308+0000","last_became_active":"2023-10-08T02:12:25.131867+0000","last_became_peered":"2023-10-08T02:12:25.131867+0000","last_unstale":"2023-10-08T02:12:25.136308+0000","last_undegraded":"2023-10-08T02:12:25.136308+0000","last_fullsized":"2023-10-08T02:12:25.136308+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:14:14.358279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.107269+0000","last_change":"2023-10-08T02:12:24.116761+0000","last_active":"2023-10-08T02:12:25.107269+0000","last_peered":"2023-10-08T02:12:25.107269+0000","last_clean":"2023-10-08T02:12:25.107269+0000","last_became_active":"2023-10-08T02:12:24.116469+0000","last_became_peered":"2023-10-08T02:12:24.116469+0000","last_unstale":"2023-10-08T02:12:25.107269+0000","last_undegraded":"2023-10-08T02:12:25.107269+0000","last_fullsized":"2023-10-08T02:12:25.107269+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:42.653885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.122451+0000","last_change":"2023-10-08T02:12:25.116799+0000","last_active":"2023-10-08T02:12:25.122451+0000","last_peered":"2023-10-08T02:12:25.122451+0000","last_clean":"2023-10-08T02:12:25.122451+0000","last_became_active":"2023-10-08T02:12:25.116518+0000","last_became_peered":"2023-10-08T02:12:25.116518+0000","last_unstale":"2023-10-08T02:12:25.122451+0000","last_undegraded":"2023-10-08T02:12:25.122451+0000","last_fullsized":"2023-10-08T02:12:25.122451+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:56:54.429689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.110134+0000","last_change":"2023-10-08T02:12:24.126247+0000","last_active":"2023-10-08T02:12:25.110134+0000","last_peered":"2023-10-08T02:12:25.110134+0000","last_clean":"2023-10-08T02:12:25.110134+0000","last_became_active":"2023-10-08T02:12:24.125977+0000","last_became_peered":"2023-10-08T02:12:24.125977+0000","last_unstale":"2023-10-08T02:12:25.110134+0000","last_undegraded":"2023-10-08T02:12:25.110134+0000","last_fullsized":"2023-10-08T02:12:25.110134+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:05:57.759874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.126282+0000","last_change":"2023-10-08T02:12:25.124610+0000","last_active":"2023-10-08T02:12:25.126282+0000","last_peered":"2023-10-08T02:12:25.126282+0000","last_clean":"2023-10-08T02:12:25.126282+0000","last_became_active":"2023-10-08T02:12:25.124354+0000","last_became_peered":"2023-10-08T02:12:25.124354+0000","last_unstale":"2023-10-08T02:12:25.126282+0000","last_undegraded":"2023-10-08T02:12:25.126282+0000","last_fullsized":"2023-10-08T02:12:25.126282+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:24.089566+0000","last_clean_scrub_stamp":"2023-10-08T02:12:24.089566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:24:48.457813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"170'3","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.139967+0000","last_change":"2023-10-08T02:12:24.108579+0000","last_active":"2023-10-08T02:12:25.139967+0000","last_peered":"2023-10-08T02:12:25.139967+0000","last_clean":"2023-10-08T02:12:25.139967+0000","last_became_active":"2023-10-08T02:12:24.108269+0000","last_became_peered":"2023-10-08T02:12:24.108269+0000","last_unstale":"2023-10-08T02:12:25.139967+0000","last_undegraded":"2023-10-08T02:12:25.139967+0000","last_fullsized":"2023-10-08T02:12:25.139967+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:12:23.083202+0000","last_clean_scrub_stamp":"2023-10-08T02:12:23.083202+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:33:36.136401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"160'293","reported_seq":914,"reported_epoch":170,"state":"active+clean","last_fresh":"2023-10-08T02:12:25.113000+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:12:25.113000+0000","last_peered":"2023-10-08T02:12:25.113000+0000","last_clean":"2023-10-08T02:12:25.113000+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:12:25.113000+0000","last_undegraded":"2023-10-08T02:12:25.113000+0000","last_fullsized":"2023-10-08T02:12:25.113000+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1652,"num_read_kb":3396,"num_write":479,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":293,"ondisk_log_size":293,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575760,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312248,"kb_used_data":2672,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059592,"statfs":{"total":96636764160,"available":96317022208,"internally_reserved":0,"allocated":2736128,"data_stored":1655519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.70099999999999996,"15min":0.73099999999999998},"min":{"1min":0.46899999999999997,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76100000000000001,"5min":0.96099999999999997,"15min":1.0209999999999999},"last":0.748},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.751,"15min":0.75},"min":{"1min":0.498,"5min":0.48799999999999999,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":1.024,"15min":1.3540000000000001},"last":0.69299999999999995}]},{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.72899999999999998,"15min":0.748},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.83199999999999996,"5min":1.0149999999999999,"15min":1.0149999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.73399999999999999,"15min":0.73599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81200000000000006,"5min":0.999,"15min":1.0580000000000001},"last":0.64900000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.755,"15min":0.76400000000000001},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.505},"max":{"1min":0.873,"5min":0.94899999999999995,"15min":1.0600000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.755,"15min":0.755},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.75,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.73699999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.72399999999999998,"15min":0.72999999999999998},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.79500000000000004,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.80300000000000005,"15min":0.79900000000000004},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.875,"5min":1.502,"15min":1.502},"last":0.71099999999999997}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59899999999999998,"15min":0.59999999999999998},"min":{"1min":0.46300000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.85499999999999998,"5min":0.94999999999999996,"15min":0.97799999999999998},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63,"15min":0.627},"min":{"1min":0.40600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84199999999999997,"5min":1.034,"15min":1.034},"last":0.44700000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.45000000000000001,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.82799999999999996,"5min":0.95799999999999996,"15min":1.0369999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66800000000000004,"15min":0.67400000000000004},"min":{"1min":0.46700000000000003,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93999999999999995,"5min":0.96099999999999997,"15min":0.97799999999999998},"last":0.54000000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76100000000000001,"15min":0.76800000000000002},"min":{"1min":0.56999999999999995,"5min":0.503,"15min":0.48699999999999999},"max":{"1min":0.97499999999999998,"5min":0.998,"15min":1.0149999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.73099999999999998,"15min":0.72699999999999998},"min":{"1min":0.54300000000000004,"5min":0.47799999999999998,"15min":0.45400000000000001},"max":{"1min":0.91500000000000004,"5min":1.109,"15min":1.109},"last":0.67200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575760,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299384,"kb_used_data":2096,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072456,"statfs":{"total":96636764160,"available":96330194944,"internally_reserved":0,"allocated":2146304,"data_stored":1064673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.76000000000000001,"15min":0.74199999999999999},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.41199999999999998},"max":{"1min":0.96299999999999997,"5min":1.569,"15min":1.569},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76400000000000001,"15min":0.74399999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.94199999999999995,"5min":1.5269999999999999,"15min":1.5269999999999999},"last":0.71599999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.77700000000000002,"15min":0.76000000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.748},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.76200000000000001,"15min":0.75},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.89900000000000002,"5min":1.472,"15min":1.472},"last":0.64400000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.745,"15min":0.72899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.81799999999999995,"5min":1.5069999999999999,"15min":1.5069999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.79600000000000004,"15min":0.76500000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.495},"max":{"1min":0.91700000000000004,"5min":1.603,"15min":1.603},"last":0.66500000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80600000000000005,"15min":0.79500000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.998,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.78400000000000003,"15min":0.78100000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.871,"5min":1.169,"15min":1.169},"last":0.73299999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63100000000000001,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.85599999999999998,"5min":1.133,"15min":1.133},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63700000000000001,"15min":0.624},"min":{"1min":0.439,"5min":0.439,"15min":0.40500000000000003},"max":{"1min":0.97199999999999998,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.504}]},{"osd":5,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.621,"15min":0.61099999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.82599999999999996,"5min":0.94299999999999995,"15min":1.0680000000000001},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.70099999999999996,"15min":0.68200000000000005},"min":{"1min":0.47699999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90300000000000002,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.79000000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:11:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.74199999999999999,"15min":0.72099999999999997},"min":{"1min":0.503,"5min":0.47099999999999997,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.151,"15min":1.1559999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.76100000000000001,"15min":0.753},"min":{"1min":0.55400000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.90200000000000002,"5min":1.232,"15min":1.232},"last":0.77100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575760,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307568,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94064272,"statfs":{"total":96636764160,"available":96321814528,"internally_reserved":0,"allocated":2138112,"data_stored":1065081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48799999999999999,"15min":0.48999999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.84199999999999997,"5min":1.024,"15min":1.024},"last":0.315},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.77900000000000003,"5min":1.603,"15min":1.603},"last":0.44600000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.497,"15min":0.504},"min":{"1min":0.35399999999999998,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.60199999999999998,"5min":0.82899999999999996,"15min":0.83899999999999997},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.57799999999999996,"15min":0.56799999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.73099999999999998,"5min":1.4550000000000001,"15min":1.4550000000000001},"last":0.42299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61199999999999999,"15min":0.60199999999999998},"min":{"1min":0.47699999999999998,"5min":0.441,"15min":0.42799999999999999},"max":{"1min":0.751,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.59099999999999997,"15min":0.56399999999999995},"min":{"1min":0.40799999999999997,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":0.70799999999999996,"5min":3.698,"15min":3.698},"last":0.36899999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86399999999999999,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.80900000000000005,"15min":0.82199999999999995},"min":{"1min":0.57999999999999996,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.95999999999999996,"5min":1.734,"15min":1.734},"last":0.77600000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.80400000000000005,"15min":0.79800000000000004},"min":{"1min":0.55800000000000005,"5min":0.54800000000000004,"15min":0.53600000000000003},"max":{"1min":1.0489999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.81299999999999994,"15min":0.82199999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91600000000000004,"5min":1.403,"15min":1.403},"last":0.73099999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.61299999999999999,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.0149999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.83299999999999996,"15min":0.83199999999999996},"min":{"1min":0.59699999999999998,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.0289999999999999,"5min":1.573,"15min":1.573},"last":0.74399999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.624,"5min":0.624,"15min":0.61899999999999999},"max":{"1min":0.99399999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.875,"15min":0.86499999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.58999999999999997},"max":{"1min":0.99399999999999999,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":0.79000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575759,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308160,"kb_used_data":2680,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063680,"statfs":{"total":96636764160,"available":96321208320,"internally_reserved":0,"allocated":2744320,"data_stored":1656465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.70499999999999996,"5min":0.82899999999999996,"15min":0.83999999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.50700000000000001,"15min":0.51500000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65200000000000002,"5min":0.80500000000000005,"15min":0.81599999999999995},"last":0.53200000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.48699999999999999,"15min":0.48499999999999999},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.58299999999999996,"5min":0.70699999999999996,"15min":0.746},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.68500000000000005,"5min":0.88300000000000001,"15min":1.325},"last":0.57999999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.57099999999999995,"15min":0.56399999999999995},"min":{"1min":0.42399999999999999,"5min":0.41199999999999998,"15min":0.371},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72399999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.51000000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.77300000000000002,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.99399999999999999,"5min":1.052,"15min":1.24},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.0009999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":0.77000000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.64700000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.0489999999999999,"5min":1.327,"15min":1.327},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.878,"15min":0.879},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56100000000000005},"max":{"1min":1.048,"5min":1.653,"15min":1.653},"last":0.755}]},{"osd":6,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.83899999999999997,"15min":0.83599999999999997},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53400000000000003},"max":{"1min":1.0209999999999999,"5min":1.577,"15min":1.577},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.80000000000000004},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58099999999999996},"max":{"1min":0.97699999999999998,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:12:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.85699999999999998,"15min":0.84799999999999998},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.55800000000000005},"max":{"1min":1.129,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.89000000000000001,"15min":0.88200000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.0720000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.75}]}]},{"osd":2,"up_from":13,"seq":55834575759,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311676,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060164,"statfs":{"total":96636764160,"available":96317607936,"internally_reserved":0,"allocated":2150400,"data_stored":1066133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.47499999999999998,"15min":0.47799999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65500000000000003,"5min":1.27,"15min":1.27},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53200000000000003,"15min":0.53600000000000003},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":0.45600000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.48599999999999999,"15min":0.497},"min":{"1min":0.36499999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.65600000000000003,"5min":0.79500000000000004,"15min":0.81699999999999995},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.504,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.40799999999999997,"5min":0.38800000000000001,"15min":0.34200000000000003},"max":{"1min":0.69699999999999995,"5min":1.651,"15min":1.651},"last":0.54200000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.39400000000000002},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.495},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.60699999999999998,"15min":0.59799999999999998},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.36799999999999999},"max":{"1min":2.274,"5min":2.274,"15min":2.274},"last":0.52200000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81399999999999995,"15min":0.81000000000000005},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.55100000000000005},"max":{"1min":1.0209999999999999,"5min":1.8560000000000001,"15min":1.8560000000000001},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.82899999999999996,"15min":0.81899999999999995},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.53400000000000003},"max":{"1min":0.93799999999999994,"5min":3.004,"15min":3.004},"last":0.71699999999999997}]},{"osd":5,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.90300000000000002,"15min":0.86799999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.046,"5min":3.145,"15min":3.145},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.54800000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0860000000000001,"5min":1.847,"15min":1.847},"last":0.76900000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.86399999999999999,"15min":0.83799999999999997},"min":{"1min":0.57699999999999996,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.4950000000000001,"5min":1.7949999999999999,"15min":1.7949999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.84699999999999998,"15min":0.84799999999999998},"min":{"1min":0.623,"5min":0.60199999999999998,"15min":0.58599999999999997},"max":{"1min":1.0580000000000001,"5min":1.331,"15min":1.331},"last":0.74199999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:12:17 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.88700000000000001,"15min":0.85999999999999999},"min":{"1min":0.59299999999999997,"5min":0.57199999999999995,"15min":0.56799999999999995},"max":{"1min":0.90300000000000002,"5min":2.899,"15min":2.899},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.89300000000000002,"15min":0.86599999999999999},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.58899999999999997},"max":{"1min":1.4570000000000001,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":0.84199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575759,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315760,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94056080,"statfs":{"total":96636764160,"available":96313425920,"internally_reserved":0,"allocated":2138112,"data_stored":1064617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.46500000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.375},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.504,"15min":0.50900000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.78800000000000003,"15min":0.79300000000000004},"last":0.497}]},{"osd":1,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.47999999999999998,"15min":0.48299999999999998},"min":{"1min":0.36399999999999999,"5min":0.35999999999999999,"15min":0.32900000000000001},"max":{"1min":0.54700000000000004,"5min":0.71599999999999997,"15min":0.84099999999999997},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.54300000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89500000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.41899999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.59199999999999997,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.629,"5min":0.84999999999999998,"15min":0.89500000000000002},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.64000000000000001,"15min":0.63900000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.45600000000000002},"max":{"1min":0.83699999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.56699999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.98499999999999999,"5min":1.1000000000000001,"15min":1.214},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79200000000000004,"15min":0.80000000000000004},"min":{"1min":0.56899999999999995,"5min":0.56299999999999994,"15min":0.55600000000000005},"max":{"1min":0.998,"5min":1.0680000000000001,"15min":1.0900000000000001},"last":0.76800000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.77100000000000002,"15min":0.79600000000000004},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.876,"5min":1.0589999999999999,"15min":1.125},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.79400000000000004,"15min":0.80000000000000004},"min":{"1min":0.56599999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.93300000000000005,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.749}]},{"osd":6,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.80600000000000005,"15min":0.81299999999999994},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.01,"5min":1.403,"15min":1.403},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.81399999999999995,"15min":0.81200000000000006},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.55600000000000005},"max":{"1min":0.90700000000000003,"5min":1.492,"15min":1.492},"last":0.90600000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:12:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":0.83799999999999997,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.93799999999999994,"5min":1.341,"15min":1.341},"last":0.89400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575759,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311660,"kb_used_data":2084,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060180,"statfs":{"total":96636764160,"available":96317624320,"internally_reserved":0,"allocated":2134016,"data_stored":1064527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70099999999999996,"15min":0.71699999999999997},"min":{"1min":0.497,"5min":0.443,"15min":0.443},"max":{"1min":0.92100000000000004,"5min":1.1359999999999999,"15min":1.3859999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.71399999999999997,"15min":0.72699999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.86099999999999999,"5min":1.127,"15min":1.5609999999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.70999999999999996,"15min":0.71899999999999997},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.90900000000000003,"5min":1.0549999999999999,"15min":1.0960000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.73799999999999999,"15min":0.749},"min":{"1min":0.51000000000000001,"5min":0.48399999999999999,"15min":0.42299999999999999},"max":{"1min":0.89300000000000002,"5min":1.1000000000000001,"15min":1.52},"last":0.56100000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.74299999999999999,"15min":0.76600000000000001},"min":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.47599999999999998},"max":{"1min":0.90700000000000003,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.76400000000000001,"15min":0.78500000000000003},"min":{"1min":0.51300000000000001,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.88300000000000001,"5min":1.1579999999999999,"15min":1.403},"last":0.63100000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.76000000000000001,"15min":0.77400000000000002},"min":{"1min":0.56499999999999995,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.85599999999999998,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.78900000000000003,"15min":0.79900000000000004},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.872,"5min":1.208,"15min":1.5069999999999999},"last":0.66000000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.60899999999999999,"15min":0.61299999999999999},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.78900000000000003,"5min":1.05,"15min":1.05},"last":0.498},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.74099999999999999,"15min":0.74399999999999999},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.95799999999999996,"5min":1.038,"15min":1.1559999999999999},"last":0.72299999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.65600000000000003,"15min":0.64600000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.89500000000000002,"5min":1.016,"15min":1.016},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.44,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.83499999999999996,"5min":0.96499999999999997,"15min":1.1060000000000001},"last":0.441}]},{"osd":7,"last update":"Sun Oct 8 02:12:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.44600000000000001},"max":{"1min":0.92900000000000005,"5min":1.0680000000000001,"15min":1.129},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.71199999999999997,"15min":0.71699999999999997},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.79200000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.70199999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575759,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311664,"kb_used_data":2088,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94060176,"statfs":{"total":96636764160,"available":96317620224,"internally_reserved":0,"allocated":2138112,"data_stored":1064549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.74199999999999999,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.111,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.873},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.76700000000000002,"15min":0.754},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.438},"max":{"1min":1.0009999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70199999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.79700000000000004,"15min":0.77800000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.42699999999999999},"max":{"1min":0.95399999999999996,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.83899999999999997}]},{"osd":2,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.79800000000000004,"15min":0.77600000000000002},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.499},"max":{"1min":0.995,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.751,"5min":0.77400000000000002,"15min":0.76200000000000001},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.50900000000000001},"max":{"1min":0.97699999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.77700000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.80700000000000005,"15min":0.77900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.96399999999999997,"5min":1.0620000000000001,"15min":1.0669999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77800000000000002,"15min":0.78000000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.88400000000000001,"5min":1.0529999999999999,"15min":1.091},"last":0.82199999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63500000000000001,"15min":0.623},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.98499999999999999,"5min":1.0029999999999999,"15min":1.0249999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.69099999999999995,"15min":0.68100000000000005},"min":{"1min":0.55100000000000005,"5min":0.45400000000000001,"15min":0.441},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.115},"last":0.53900000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.49199999999999999,"5min":0.45400000000000001,"15min":0.42699999999999999},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0629999999999999},"last":0.63},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.077},"last":0.73599999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:12:04 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.74399999999999999,"15min":0.73799999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.77400000000000002,"15min":0.76900000000000002},"min":{"1min":0.53600000000000003,"5min":0.52900000000000003,"15min":0.48599999999999999},"max":{"1min":1.0189999999999999,"5min":1.0840000000000001,"15min":1.1259999999999999},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:12:34.715 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T02:12:34.716 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T02:12:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.932+0000 7f32987ff700 1 Processor -- start 2023-10-08T02:12:34.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.932+0000 7f32987ff700 1 -- start start 2023-10-08T02:12:34.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.933+0000 7f32987ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3290156620 0x7f32900c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f32987ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329015a6f0 con 0x7f32901569f0 2023-10-08T02:12:34.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f32987ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329015a9d0 con 0x7f3290068390 2023-10-08T02:12:34.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f32987ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32900c8d30 con 0x7f3290156620 2023-10-08T02:12:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295d9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f32901569f0 0x7f32900ace70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60004/0 (socket says 172.21.15.119:60004) 2023-10-08T02:12:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295d9a700 1 -- 172.21.15.119:0/3155353169 learned_addr learned my addr 172.21.15.119:0/3155353169 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:34.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f329659b700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3290156620 0x7f32900c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644222792 0 0) 0x7f329015a6f0 con 0x7f32901569f0 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f327c003570 con 0x7f32901569f0 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2783003370 0 0) 0x7f329015a9d0 con 0x7f3290068390 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f329015a6f0 con 0x7f3290068390 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 426737535 0 0) 0x7f327c003570 con 0x7f32901569f0 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3290156620 msgr2=0x7f32900c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295599700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3290156620 0x7f32900c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.934+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 >> v1:172.21.15.184:6789/0 conn(0x7f3290068390 legacy=0x7f329006da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329015acb0 con 0x7f32901569f0 2023-10-08T02:12:34.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f3280001470 con 0x7f32901569f0 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 >> v1:172.21.15.119:6789/0 conn(0x7f32901569f0 legacy=0x7f32900ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3290156620 0x7f327c003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c004180 0x7f327c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f327c004d50 0x7f327c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32900c8d30 con 0x7f327c004180 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327c00bba0 con 0x7f327c004d50 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327c00bd60 con 0x7f3290156620 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295d9a700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c004180 0x7f327c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f329659b700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3290156620 0x7f327c003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3296d9c700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f327c004d50 0x7f327c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295d9a700 1 -- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3290156620 msgr2=0x7f327c003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295d9a700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3290156620 0x7f327c003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295d9a700 1 -- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f327c004d50 msgr2=0x7f327c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295d9a700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f327c004d50 0x7f327c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.935+0000 7f3295d9a700 1 -- 172.21.15.119:0/3155353169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327c003570 con 0x7f327c004180 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f3295d9a700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c004180 0x7f327c004810 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f329006d590 tx=0x7f3280001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3f50713079d189 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328000cb60 con 0x7f327c004180 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328000ccc0 con 0x7f327c004180 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f3295599700 1 -- 172.21.15.119:0/3155353169 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280005b70 con 0x7f327c004180 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f32987ff700 1 -- 172.21.15.119:0/3155353169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c004180 msgr2=0x7f327c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f32987ff700 1 --2- 172.21.15.119:0/3155353169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f327c004180 0x7f327c004810 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f329006d590 tx=0x7f3280001e10 comp rx=0 tx=0).stop 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f3296d9c700 1 -- 172.21.15.119:0/3155353169 reap_dead start 2023-10-08T02:12:34.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f32987ff700 1 -- 172.21.15.119:0/3155353169 shutdown_connections 2023-10-08T02:12:34.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.936+0000 7f32987ff700 1 -- 172.21.15.119:0/3155353169 >> 172.21.15.119:0/3155353169 conn(0x7f32900cc540 msgr2=0x7f32900cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:34.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.937+0000 7f32987ff700 1 -- 172.21.15.119:0/3155353169 shutdown_connections 2023-10-08T02:12:34.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.937+0000 7f32987ff700 1 -- 172.21.15.119:0/3155353169 wait complete. 2023-10-08T02:12:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.938+0000 7f32987ff700 1 Processor -- start 2023-10-08T02:12:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.938+0000 7f32987ff700 1 -- start start 2023-10-08T02:12:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f32987ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f327c004180 0x7f329014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f32987ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f329014b5e0 0x7f329014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f32987ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f329014d810 0x7f329014dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f32987ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329014e1c0 con 0x7f329014d810 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f32987ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32900b2150 con 0x7f329014b5e0 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f32987ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32900b15b0 con 0x7f327c004180 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f3296d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f329014d810 0x7f329014dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f3296d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f329014d810 0x7f329014dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46342/0 (socket says 172.21.15.119:46342) 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f3296d9c700 1 -- 172.21.15.119:0/3462016111 learned_addr learned my addr 172.21.15.119:0/3462016111 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f3295d9a700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f329014b5e0 0x7f329014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.939+0000 7f329659b700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f327c004180 0x7f329014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f3296d9c700 1 -- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f327c004180 msgr2=0x7f329014af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f3296d9c700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f327c004180 0x7f329014af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f3296d9c700 1 -- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f329014b5e0 msgr2=0x7f329014ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:34.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f3296d9c700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f329014b5e0 0x7f329014ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f3296d9c700 1 -- 172.21.15.119:0/3462016111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32900b1820 con 0x7f329014d810 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f3295d9a700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f329014b5e0 0x7f329014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f3296d9c700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f329014d810 0x7f329014dc80 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f32900cd580 tx=0x7f32880017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f32867fc700 1 -- 172.21.15.119:0/3462016111 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288017070 con 0x7f329014d810 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f32867fc700 1 -- 172.21.15.119:0/3462016111 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3288006ba0 con 0x7f329014d810 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32900b1b00 con 0x7f329014d810 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f32867fc700 1 -- 172.21.15.119:0/3462016111 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328800c880 con 0x7f329014d810 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.940+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32900c01c0 con 0x7f329014d810 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.941+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32900718b0 con 0x7f329014d810 2023-10-08T02:12:34.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.942+0000 7f32867fc700 1 -- 172.21.15.119:0/3462016111 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f328800cc30 con 0x7f329014d810 2023-10-08T02:12:34.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.943+0000 7f32867fc700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3274066f30 0x7f32740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:34.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.944+0000 7f32867fc700 1 -- 172.21.15.119:0/3462016111 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f3288084210 con 0x7f329014d810 2023-10-08T02:12:34.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.944+0000 7f329659b700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3274066f30 0x7f32740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:34.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.944+0000 7f329659b700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3274066f30 0x7f32740693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f328c006f00 tx=0x7f328c0091a0 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:34.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:34.948+0000 7f32867fc700 1 -- 172.21.15.119:0/3462016111 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3288051330 con 0x7f329014d810 2023-10-08T02:12:35.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.216+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f32900678d0 con 0x7f329014d810 2023-10-08T02:12:35.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.220+0000 7f32867fc700 1 -- 172.21.15.119:0/3462016111 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v45) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3288054950 con 0x7f329014d810 2023-10-08T02:12:35.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.222+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3274066f30 msgr2=0x7f32740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:35.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.222+0000 7f32987ff700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3274066f30 0x7f32740693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f328c006f00 tx=0x7f328c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.222+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f329014d810 msgr2=0x7f329014dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.222+0000 7f32987ff700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f329014d810 0x7f329014dc80 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f32900cd580 tx=0x7f32880017a0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.223+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 shutdown_connections 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.223+0000 7f32987ff700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3274066f30 0x7f32740693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.223+0000 7f32987ff700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f327c004180 0x7f329014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.223+0000 7f32987ff700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f329014d810 0x7f329014dc80 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.223+0000 7f32987ff700 1 --2- 172.21.15.119:0/3462016111 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f329014b5e0 0x7f329014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.223+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 >> 172.21.15.119:0/3462016111 conn(0x7f32900cc540 msgr2=0x7f3290069890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.223+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 shutdown_connections 2023-10-08T02:12:35.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.223+0000 7f32987ff700 1 -- 172.21.15.119:0/3462016111 wait complete. 2023-10-08T02:12:35.224 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T02:12:35.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:12:35.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.421+0000 7fb0a359d700 1 Processor -- start 2023-10-08T02:12:35.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.421+0000 7fb0a359d700 1 -- start start 2023-10-08T02:12:35.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.422+0000 7fb0a359d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb09c150180 0x7fb09c157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:35.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.422+0000 7fb0a359d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150550 0x7fb09c0c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:35.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.422+0000 7fb0a359d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09c06e2a0 con 0x7fb09c0c7240 2023-10-08T02:12:35.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.422+0000 7fb0a359d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb09c0c76c0 con 0x7fb09c150180 2023-10-08T02:12:35.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.422+0000 7fb0a359d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c06b2e0 con 0x7fb09c150550 2023-10-08T02:12:35.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.422+0000 7fb0a2d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb09c0c7240 0x7fb09c069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60014/0 (socket says 172.21.15.119:60014) 2023-10-08T02:12:35.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.422+0000 7fb0a2d9c700 1 -- 172.21.15.119:0/1671451220 learned_addr learned my addr 172.21.15.119:0/1671451220 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:35.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.422+0000 7fb0a259b700 1 --2- 172.21.15.119:0/1671451220 >> v2:172.21.15.119:3300/0 conn(0x7fb09c150180 0x7fb09c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:35.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a1d9a700 1 --2- 172.21.15.119:0/1671451220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150550 0x7fb09c0c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:35.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a1599700 1 -- 172.21.15.119:0/1671451220 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247995476 0 0) 0x7fb09c06e2a0 con 0x7fb09c0c7240 2023-10-08T02:12:35.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a1599700 1 -- 172.21.15.119:0/1671451220 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb088003570 con 0x7fb09c0c7240 2023-10-08T02:12:35.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a259b700 1 -- 172.21.15.119:0/1671451220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150550 msgr2=0x7fb09c0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a259b700 1 --2- 172.21.15.119:0/1671451220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150550 0x7fb09c0c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a259b700 1 -- 172.21.15.119:0/1671451220 >> v1:172.21.15.119:6789/0 conn(0x7fb09c0c7240 legacy=0x7fb09c069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a259b700 1 -- 172.21.15.119:0/1671451220 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c0c78e0 con 0x7fb09c150180 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a1d9a700 1 --2- 172.21.15.119:0/1671451220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150550 0x7fb09c0c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a259b700 1 --2- 172.21.15.119:0/1671451220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c150180 0x7fb09c157770 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb09800aac0 tx=0x7fb09800a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd093397da128108 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a1599700 1 -- 172.21.15.119:0/1671451220 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09800c930 con 0x7fb09c150180 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a1599700 1 -- 172.21.15.119:0/1671451220 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09800ca90 con 0x7fb09c150180 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.423+0000 7fb0a1599700 1 -- 172.21.15.119:0/1671451220 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09800ce30 con 0x7fb09c150180 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.424+0000 7fb0a359d700 1 -- 172.21.15.119:0/1671451220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c150180 msgr2=0x7fb09c157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.424+0000 7fb0a359d700 1 --2- 172.21.15.119:0/1671451220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c150180 0x7fb09c157770 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb09800aac0 tx=0x7fb09800a330 comp rx=0 tx=0).stop 2023-10-08T02:12:35.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.424+0000 7fb0a359d700 1 -- 172.21.15.119:0/1671451220 shutdown_connections 2023-10-08T02:12:35.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.424+0000 7fb0a359d700 1 --2- 172.21.15.119:0/1671451220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c150180 0x7fb09c157770 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.424+0000 7fb0a359d700 1 --2- 172.21.15.119:0/1671451220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150550 0x7fb09c0c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.424+0000 7fb0a359d700 1 -- 172.21.15.119:0/1671451220 >> 172.21.15.119:0/1671451220 conn(0x7fb09c0cc1f0 msgr2=0x7fb09c0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:35.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.424+0000 7fb0a359d700 1 -- 172.21.15.119:0/1671451220 shutdown_connections 2023-10-08T02:12:35.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.424+0000 7fb0a359d700 1 -- 172.21.15.119:0/1671451220 wait complete. 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.425+0000 7fb0a359d700 1 Processor -- start 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.426+0000 7fb0a359d700 1 -- start start 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.426+0000 7fb0a359d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c0c7240 0x7fb09c0ac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.426+0000 7fb0a359d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150180 0x7fb09c0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.426+0000 7fb0a359d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c150550 0x7fb09c0c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.426+0000 7fb0a359d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c0ada30 con 0x7fb09c0c7240 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.426+0000 7fb0a359d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c0ace20 con 0x7fb09c150550 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.426+0000 7fb0a359d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09c0ad100 con 0x7fb09c150180 2023-10-08T02:12:35.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c0c7240 0x7fb09c0ac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c0c7240 0x7fb09c0ac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46354/0 (socket says 172.21.15.119:46354) 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 -- 172.21.15.119:0/2764458993 learned_addr learned my addr 172.21.15.119:0/2764458993 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 -- 172.21.15.119:0/2764458993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150180 msgr2=0x7fb09c0aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150180 0x7fb09c0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 -- 172.21.15.119:0/2764458993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c150550 msgr2=0x7fb09c0c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a2d9c700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c150550 0x7fb09c0c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c150550 0x7fb09c0c37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 -- 172.21.15.119:0/2764458993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09800a5d0 con 0x7fb09c0c7240 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a2d9c700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c150550 0x7fb09c0c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.427+0000 7fb0a259b700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c0c7240 0x7fb09c0ac5d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb09800a370 tx=0x7fb09801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:35.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.428+0000 7fb0927fc700 1 -- 172.21.15.119:0/2764458993 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098010600 con 0x7fb09c0c7240 2023-10-08T02:12:35.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.428+0000 7fb0927fc700 1 -- 172.21.15.119:0/2764458993 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0980107a0 con 0x7fb09c0c7240 2023-10-08T02:12:35.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.428+0000 7fb0927fc700 1 -- 172.21.15.119:0/2764458993 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb098010b60 con 0x7fb09c0c7240 2023-10-08T02:12:35.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.428+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09c0ad3e0 con 0x7fb09c0c7240 2023-10-08T02:12:35.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.428+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09c0acb10 con 0x7fb09c0c7240 2023-10-08T02:12:35.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.429+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09c04c3c0 con 0x7fb09c0c7240 2023-10-08T02:12:35.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.429+0000 7fb0927fc700 1 -- 172.21.15.119:0/2764458993 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb098021430 con 0x7fb09c0c7240 2023-10-08T02:12:35.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.430+0000 7fb0927fc700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb080066f30 0x7fb0800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:35.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.430+0000 7fb0927fc700 1 -- 172.21.15.119:0/2764458993 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb098085da0 con 0x7fb09c0c7240 2023-10-08T02:12:35.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.431+0000 7fb0a1d9a700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb080066f30 0x7fb0800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:35.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.431+0000 7fb0a1d9a700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb080066f30 0x7fb0800693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb08c006fd0 tx=0x7fb08c008040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:35.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.434+0000 7fb0927fc700 1 -- 172.21.15.119:0/2764458993 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb098052ef0 con 0x7fb09c0c7240 2023-10-08T02:12:35.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.666+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb09c0add10 con 0x7fb09c0c7240 2023-10-08T02:12:35.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.667+0000 7fb0927fc700 1 -- 172.21.15.119:0/2764458993 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fb098056510 con 0x7fb09c0c7240 2023-10-08T02:12:35.667 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:12:35.667 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7409,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3771852147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3771852147},{"type":"v1","addr":"172.21.15.184:6834","nonce":3771852147}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/524755843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":524755843},{"type":"v1","addr":"172.21.15.184:6835","nonce":524755843}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9144,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/295310163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":295310163},{"type":"v1","addr":"172.21.15.119:6833","nonce":295310163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:12:24.579741+0000","modified":"2023-10-08T02:12:28.133805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9150},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9150":{"gid":9150,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2713045427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2713045427},{"type":"v1","addr":"172.21.15.119:6835","nonce":2713045427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T02:12:35.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb080066f30 msgr2=0x7fb0800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb080066f30 0x7fb0800693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb08c006fd0 tx=0x7fb08c008040 comp rx=0 tx=0).stop 2023-10-08T02:12:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c0c7240 msgr2=0x7fb09c0ac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c0c7240 0x7fb09c0ac5d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb09800a370 tx=0x7fb09801a000 comp rx=0 tx=0).stop 2023-10-08T02:12:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 shutdown_connections 2023-10-08T02:12:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb080066f30 0x7fb0800693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb09c150550 0x7fb09c0c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb09c0c7240 0x7fb09c0ac5d0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 --2- 172.21.15.119:0/2764458993 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb09c150180 0x7fb09c0aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 >> 172.21.15.119:0/2764458993 conn(0x7fb09c0cc1f0 msgr2=0x7fb09c0c94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 shutdown_connections 2023-10-08T02:12:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:35.669+0000 7fb0a359d700 1 -- 172.21.15.119:0/2764458993 wait complete. 2023-10-08T02:12:35.902 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 79 2023-10-08T02:12:36.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T02:12:24.579741+0000', 'modified': '2023-10-08T02:12:28.133805+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9150}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9150': {'gid': 9150, 'name': 'c', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6835/2713045427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 2713045427}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 2713045427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T02:12:36.305 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T02:12:36.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-08T02:12:36.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.490+0000 7fb62bbff700 1 Processor -- start 2023-10-08T02:12:36.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.490+0000 7fb62bbff700 1 -- start start 2023-10-08T02:12:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.491+0000 7fb62bbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb6240caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.491+0000 7fb62bbff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb62406c0a0 0x7fb62415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.491+0000 7fb62bbff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb62415df50 0x7fb62415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.491+0000 7fb62bbff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62415d7e0 con 0x7fb62415df50 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.491+0000 7fb62bbff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62415dac0 con 0x7fb62406c0a0 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.491+0000 7fb62bbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62415aa30 con 0x7fb62406bcd0 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb62415df50 0x7fb62415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb62415df50 0x7fb62415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46370/0 (socket says 172.21.15.119:46370) 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 -- 172.21.15.119:0/4008865300 learned_addr learned my addr 172.21.15.119:0/4008865300 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62999b700 1 --2- 172.21.15.119:0/4008865300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb6240caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62919a700 1 --2- 172.21.15.119:0/4008865300 >> v2:172.21.15.184:3300/0 conn(0x7fb62406c0a0 0x7fb62415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 -- 172.21.15.119:0/4008865300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 msgr2=0x7fb6240caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 --2- 172.21.15.119:0/4008865300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb6240caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 -- 172.21.15.119:0/4008865300 >> v2:172.21.15.184:3300/0 conn(0x7fb62406c0a0 msgr2=0x7fb62415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 --2- 172.21.15.119:0/4008865300 >> v2:172.21.15.184:3300/0 conn(0x7fb62406c0a0 0x7fb62415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 -- 172.21.15.119:0/4008865300 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62415ad10 con 0x7fb62415df50 2023-10-08T02:12:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62999b700 1 --2- 172.21.15.119:0/4008865300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb6240caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62919a700 1 --2- 172.21.15.119:0/4008865300 >> v2:172.21.15.184:3300/0 conn(0x7fb62406c0a0 0x7fb62415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.492+0000 7fb62a19c700 1 --2- 172.21.15.119:0/4008865300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62415df50 0x7fb62415e3b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb6240c5660 tx=0x7fb61c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97cb7987002e709 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb628999700 1 -- 172.21.15.119:0/4008865300 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00e070 con 0x7fb62415df50 2023-10-08T02:12:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb628999700 1 -- 172.21.15.119:0/4008865300 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c00ba50 con 0x7fb62415df50 2023-10-08T02:12:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb628999700 1 -- 172.21.15.119:0/4008865300 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0092e0 con 0x7fb62415df50 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 -- 172.21.15.119:0/4008865300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62415df50 msgr2=0x7fb62415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 --2- 172.21.15.119:0/4008865300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62415df50 0x7fb62415e3b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7fb6240c5660 tx=0x7fb61c009840 comp rx=0 tx=0).stop 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 -- 172.21.15.119:0/4008865300 shutdown_connections 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 --2- 172.21.15.119:0/4008865300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62415df50 0x7fb62415e3b0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 --2- 172.21.15.119:0/4008865300 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb6240caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 --2- 172.21.15.119:0/4008865300 >> v2:172.21.15.184:3300/0 conn(0x7fb62406c0a0 0x7fb62415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 -- 172.21.15.119:0/4008865300 >> 172.21.15.119:0/4008865300 conn(0x7fb6240ce600 msgr2=0x7fb6240cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 -- 172.21.15.119:0/4008865300 shutdown_connections 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.493+0000 7fb62bbff700 1 -- 172.21.15.119:0/4008865300 wait complete. 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.495+0000 7fb62bbff700 1 Processor -- start 2023-10-08T02:12:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.495+0000 7fb62bbff700 1 -- start start 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62bbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb62414f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62bbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62406c0a0 0x7fb624157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62bbff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb62415df50 0x7fb62406d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62bbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6241587c0 con 0x7fb62406c0a0 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62bbff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb624158aa0 con 0x7fb62415df50 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62bbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb624158d80 con 0x7fb62406bcd0 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62919a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62406c0a0 0x7fb624157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62919a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62406c0a0 0x7fb624157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46372/0 (socket says 172.21.15.119:46372) 2023-10-08T02:12:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62919a700 1 -- 172.21.15.119:0/234841531 learned_addr learned my addr 172.21.15.119:0/234841531 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62999b700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb62414f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62a19c700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb62415df50 0x7fb62406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62919a700 1 -- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 msgr2=0x7fb62414f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62919a700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb62414f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62919a700 1 -- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb62415df50 msgr2=0x7fb62406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62919a700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb62415df50 0x7fb62406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62919a700 1 -- 172.21.15.119:0/234841531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c009ae0 con 0x7fb62406c0a0 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62999b700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb62414f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.496+0000 7fb62a19c700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb62415df50 0x7fb62406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.497+0000 7fb62919a700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62406c0a0 0x7fb624157de0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb61400d860 tx=0x7fb614008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.497+0000 7fb619ffb700 1 -- 172.21.15.119:0/234841531 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009610 con 0x7fb62406c0a0 2023-10-08T02:12:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.497+0000 7fb619ffb700 1 -- 172.21.15.119:0/234841531 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb614009770 con 0x7fb62406c0a0 2023-10-08T02:12:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.497+0000 7fb619ffb700 1 -- 172.21.15.119:0/234841531 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb614009b10 con 0x7fb62406c0a0 2023-10-08T02:12:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.497+0000 7fb62bbff700 1 -- 172.21.15.119:0/234841531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb624159060 con 0x7fb62406c0a0 2023-10-08T02:12:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.497+0000 7fb62bbff700 1 -- 172.21.15.119:0/234841531 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62414f900 con 0x7fb62406c0a0 2023-10-08T02:12:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.499+0000 7fb619ffb700 1 -- 172.21.15.119:0/234841531 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb614014430 con 0x7fb62406c0a0 2023-10-08T02:12:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.500+0000 7fb619ffb700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb608066f30 0x7fb6080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.500+0000 7fb619ffb700 1 -- 172.21.15.119:0/234841531 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb614084b70 con 0x7fb62406c0a0 2023-10-08T02:12:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.500+0000 7fb62999b700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb608066f30 0x7fb6080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.501+0000 7fb62999b700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb608066f30 0x7fb6080693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb620006f00 tx=0x7fb620008040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.502+0000 7fb63a33fb80 1 Processor -- start 2023-10-08T02:12:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.502+0000 7fb63a33fb80 1 -- start start 2023-10-08T02:12:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3b7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e3e0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3e0f60 0x557e6e3b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6e30ea70 con 0x557e6e3e0f60 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6e30c270 con 0x557e6e3b7a50 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e6e308060 con 0x557e6e401d00 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62a19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3e0f60 0x557e6e3b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62a19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3e0f60 0x557e6e3b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46382/0 (socket says 172.21.15.119:46382) 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62a19c700 1 -- 172.21.15.119:0/2061308002 learned_addr learned my addr 172.21.15.119:0/2061308002 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62999b700 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3b7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62919a700 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e3e0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62a19c700 1 -- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 msgr2=0x557e6e3e0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62a19c700 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e3e0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62a19c700 1 -- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 msgr2=0x557e6e3b7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62a19c700 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3b7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62a19c700 1 -- 172.21.15.119:0/2061308002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6e1a2270 con 0x557e6e3e0f60 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.503+0000 7fb62999b700 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb62a19c700 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3e0f60 0x557e6e3b3cf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009390 tx=0x7fb61c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a834ccdba41a856 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb606ffd700 1 -- 172.21.15.119:0/2061308002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c009840 con 0x557e6e3e0f60 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb606ffd700 1 -- 172.21.15.119:0/2061308002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c00bbc0 con 0x557e6e3e0f60 2023-10-08T02:12:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb606ffd700 1 -- 172.21.15.119:0/2061308002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c007610 con 0x557e6e3e0f60 2023-10-08T02:12:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb63a33fb80 1 -- 172.21.15.119:0/2061308002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3e0f60 msgr2=0x557e6e3b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3e0f60 0x557e6e3b3cf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009390 tx=0x7fb61c018000 comp rx=0 tx=0).stop 2023-10-08T02:12:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb63a33fb80 1 -- 172.21.15.119:0/2061308002 shutdown_connections 2023-10-08T02:12:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3e0f60 0x557e6e3b3cf0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3b7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/2061308002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e3e0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.504+0000 7fb63a33fb80 1 -- 172.21.15.119:0/2061308002 >> 172.21.15.119:0/2061308002 conn(0x557e6e3b9370 msgr2=0x557e6e3b9740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.505+0000 7fb63a33fb80 1 -- 172.21.15.119:0/2061308002 shutdown_connections 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.505+0000 7fb63a33fb80 1 -- 172.21.15.119:0/2061308002 wait complete. 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.505+0000 7fb63a33fb80 1 -- start start 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.505+0000 7fb63a33fb80 1 Processor -- start 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.506+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e40f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.506+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3e0f60 0x557e6e401400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.506+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e40e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.506+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6e2b3890 con 0x557e6e3b7a50 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.506+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6e2ceea0 con 0x557e6e3e0f60 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.506+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e6e18ae30 con 0x557e6e401d00 2023-10-08T02:12:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.506+0000 7fb63a33fb80 10 client.0 _ll_register_callbacks cb 0x557e6e37b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e40f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e40f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46394/0 (socket says 172.21.15.119:46394) 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 -- 172.21.15.119:0/3709980358 learned_addr learned my addr 172.21.15.119:0/3709980358 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62a19c700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e40e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62919a700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3e0f60 0x557e6e401400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 -- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 msgr2=0x557e6e40e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e40e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 -- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3e0f60 msgr2=0x557e6e401400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3e0f60 0x557e6e401400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 -- 172.21.15.119:0/3709980358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62415ad10 con 0x557e6e3b7a50 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62919a700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3e0f60 0x557e6e401400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62a19c700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e40e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb62999b700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e40f130 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb620014840 tx=0x7fb620018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d42114163ef39c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb5fffff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.507+0000 7fb5fffff700 1 -- 172.21.15.119:0/3709980358 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200156a0 con 0x557e6e3b7a50 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 1 -- 172.21.15.119:0/3709980358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e6e3acb70 con 0x557e6e3b7a50 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb5fffff700 1 -- 172.21.15.119:0/3709980358 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb620015800 con 0x557e6e3b7a50 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb5fffff700 1 -- 172.21.15.119:0/3709980358 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb620015bf0 con 0x557e6e3b7a50 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 10 client.9315 fetch_fsmap learned FSMap version 79 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 1 -- 172.21.15.119:0/3709980358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x557e6e144fe0 con 0x557e6e3b7a50 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb5fffff700 1 -- 172.21.15.119:0/3709980358 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e6e3acb70 con 0x557e6e3b7a50 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb5fffff700 1 -- 172.21.15.119:0/3709980358 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 79) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb620021420 con 0x557e6e3b7a50 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 10 client.9315 fetch_fsmap finished waiting for FSMap version 79 2023-10-08T02:12:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 10 client.9315 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 20 client.9315 populate_metadata read hostname 'smithi119' 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] conn(0x557e6e3f8060 0x557e6e3f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 4 client.9315 mds_command: new command op to 9150 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb63a33fb80 1 -- 172.21.15.119:0/3709980358 --> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e6e389aa0 con 0x557e6e3f8060 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.508+0000 7fb62919a700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] conn(0x557e6e3f8060 0x557e6e3f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.509+0000 7fb62919a700 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] conn(0x557e6e3f8060 0x557e6e3f8430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58cc35e22acee4c server_cookie=b6bb997351c33e84 in_seq=0 out_seq=0 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.509+0000 7fb5fffff700 10 client.9315 ms_handle_connect on v2:172.21.15.119:6834/2713045427 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.509+0000 7fb5fffff700 1 -- 172.21.15.119:0/3709980358 <== mds.0 v2:172.21.15.119:6834/2713045427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x557e6e389aa0 con 0x557e6e3f8060 2023-10-08T02:12:36.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.509+0000 7fb5fffff700 10 client.9315 handle_command_reply: tid=0 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 client.9315 shutdown 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 -- 172.21.15.119:0/3709980358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 msgr2=0x557e6e40f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e40f130 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fb620014840 tx=0x7fb620018040 comp rx=0 tx=0).stop 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 -- 172.21.15.119:0/3709980358 shutdown_connections 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e401d00 0x557e6e40e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] conn(0x557e6e3f8060 0x557e6e3f8430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e40f130 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/3709980358 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3e0f60 0x557e6e401400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 -- 172.21.15.119:0/3709980358 >> 172.21.15.119:0/3709980358 conn(0x557e6e3b9370 msgr2=0x557e6e3fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb5fffff700 0 client.9315 ms_handle_reset on v2:172.21.15.119:6834/2713045427 2023-10-08T02:12:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 -- 172.21.15.119:0/3709980358 shutdown_connections 2023-10-08T02:12:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 1 -- 172.21.15.119:0/3709980358 wait complete. 2023-10-08T02:12:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.510+0000 7fb63a33fb80 20 client.9315 trim_cache size 0 max 16384 2023-10-08T02:12:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.527+0000 7fb63a33fb80 1 Processor -- start 2023-10-08T02:12:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.527+0000 7fb63a33fb80 1 -- start start 2023-10-08T02:12:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 0x557e6e3b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e401d00 0x557e6e3ba9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6e3e76c0 con 0x557e6e401d00 2023-10-08T02:12:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6e3b5fe0 con 0x557e6e3b7a50 2023-10-08T02:12:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e6e400400 con 0x557e6e3e0f60 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb62a19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e401d00 0x557e6e3ba9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb62a19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e401d00 0x557e6e3ba9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46408/0 (socket says 172.21.15.119:46408) 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.528+0000 7fb62a19c700 1 -- 172.21.15.119:0/499556396 learned_addr learned my addr 172.21.15.119:0/499556396 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb62999b700 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb62a19c700 1 -- 172.21.15.119:0/499556396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 msgr2=0x557e6e3b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb62a19c700 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 0x557e6e3b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb62a19c700 1 -- 172.21.15.119:0/499556396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 msgr2=0x557e6e3f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb62a19c700 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb62a19c700 1 -- 172.21.15.119:0/499556396 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e6e40e410 con 0x557e6e401d00 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb62999b700 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb62a19c700 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e401d00 0x557e6e3ba9c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009360 tx=0x7fb61c020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6279b68ec05277 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb6077fe700 1 -- 172.21.15.119:0/499556396 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00bd30 con 0x557e6e401d00 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb6077fe700 1 -- 172.21.15.119:0/499556396 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c02d040 con 0x557e6e401d00 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.529+0000 7fb6077fe700 1 -- 172.21.15.119:0/499556396 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c0075c0 con 0x557e6e401d00 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 -- 172.21.15.119:0/499556396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e401d00 msgr2=0x557e6e3ba9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e401d00 0x557e6e3ba9c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009360 tx=0x7fb61c020070 comp rx=0 tx=0).stop 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 -- 172.21.15.119:0/499556396 shutdown_connections 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e401d00 0x557e6e3ba9c0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e3b7a50 0x557e6e3f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/499556396 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 0x557e6e3b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 -- 172.21.15.119:0/499556396 >> 172.21.15.119:0/499556396 conn(0x557e6e3b9370 msgr2=0x557e6e3fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 -- 172.21.15.119:0/499556396 shutdown_connections 2023-10-08T02:12:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 -- 172.21.15.119:0/499556396 wait complete. 2023-10-08T02:12:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.530+0000 7fb63a33fb80 1 -- start start 2023-10-08T02:12:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.531+0000 7fb63a33fb80 1 Processor -- start 2023-10-08T02:12:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.531+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e4020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.531+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 0x557e6e402610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.531+0000 7fb63a33fb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e401d00 0x557e6e3fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6e468440 con 0x557e6e3b7a50 2023-10-08T02:12:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e6e41f310 con 0x557e6e401d00 2023-10-08T02:12:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb63a33fb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e6e467f00 con 0x557e6e3e0f60 2023-10-08T02:12:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb63a33fb80 10 client.0 _ll_register_callbacks cb 0x557e6e37b110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T02:12:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e4020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e4020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46412/0 (socket says 172.21.15.119:46412) 2023-10-08T02:12:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 -- 172.21.15.119:0/4215435565 learned_addr learned my addr 172.21.15.119:0/4215435565 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62a19c700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e401d00 0x557e6e3fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 -- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 msgr2=0x557e6e402610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62919a700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 0x557e6e402610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 0x557e6e402610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 -- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e401d00 msgr2=0x557e6e3fe7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e401d00 0x557e6e3fe7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 -- 172.21.15.119:0/4215435565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e6e1a2270 con 0x557e6e3b7a50 2023-10-08T02:12:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62a19c700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e401d00 0x557e6e3fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62919a700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 0x557e6e402610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:12:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb62999b700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e4020d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb620014810 tx=0x7fb620021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd5b2bc5a3d766d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.532+0000 7fb5fffff700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T02:12:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb5fffff700 1 -- 172.21.15.119:0/4215435565 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200156a0 con 0x557e6e3b7a50 2023-10-08T02:12:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 1 -- 172.21.15.119:0/4215435565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb62415c600 con 0x557e6e3b7a50 2023-10-08T02:12:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb5fffff700 1 -- 172.21.15.119:0/4215435565 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fb620015800 con 0x557e6e3b7a50 2023-10-08T02:12:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb5fffff700 1 -- 172.21.15.119:0/4215435565 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb620015e00 con 0x557e6e3b7a50 2023-10-08T02:12:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 10 client.9321 fetch_fsmap learned FSMap version 79 2023-10-08T02:12:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 1 -- 172.21.15.119:0/4215435565 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x557e6e2d6780 con 0x557e6e3b7a50 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb5fffff700 1 -- 172.21.15.119:0/4215435565 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb62415c600 con 0x557e6e3b7a50 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb5fffff700 1 -- 172.21.15.119:0/4215435565 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 79) v1 ==== 3100+0+0 (secure 0 0 0) 0x7fb6200218e0 con 0x557e6e3b7a50 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 10 client.9321 fetch_fsmap finished waiting for FSMap version 79 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 10 client.9321 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 20 client.9321 populate_metadata read hostname 'smithi119' 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] conn(0x557e6e3ff020 0x557e6e463900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 4 client.9321 mds_command: new command op to 9150 tid=0[{"prefix": "status"}] 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb63a33fb80 1 -- 172.21.15.119:0/4215435565 --> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fb61c00bbc0 con 0x557e6e3ff020 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.533+0000 7fb62919a700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] conn(0x557e6e3ff020 0x557e6e463900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb62919a700 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] conn(0x557e6e3ff020 0x557e6e463900 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc5980275e76a405 server_cookie=bea0ca57e208562a in_seq=0 out_seq=0 2023-10-08T02:12:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb5fffff700 10 client.9321 ms_handle_connect on v2:172.21.15.119:6834/2713045427 2023-10-08T02:12:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb5fffff700 1 -- 172.21.15.119:0/4215435565 <== mds.0 v2:172.21.15.119:6834/2713045427 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7fb61c00bbc0 con 0x557e6e3ff020 2023-10-08T02:12:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb5fffff700 10 client.9321 handle_command_reply: tid=0 2023-10-08T02:12:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 client.9321 shutdown 2023-10-08T02:12:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 -- 172.21.15.119:0/4215435565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 msgr2=0x557e6e4020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e4020d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb620014810 tx=0x7fb620021000 comp rx=0 tx=0).stop 2023-10-08T02:12:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 -- 172.21.15.119:0/4215435565 shutdown_connections 2023-10-08T02:12:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.119:6834/2713045427,v1:172.21.15.119:6835/2713045427] conn(0x557e6e3ff020 0x557e6e463900 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x557e6e401d00 0x557e6e3fe7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x557e6e3b7a50 0x557e6e4020d0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb5fffff700 0 client.9321 ms_handle_reset on v2:172.21.15.119:6834/2713045427 2023-10-08T02:12:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 --2- 172.21.15.119:0/4215435565 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x557e6e3e0f60 0x557e6e402610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 -- 172.21.15.119:0/4215435565 >> 172.21.15.119:0/4215435565 conn(0x557e6e3b9370 msgr2=0x557e6e3b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 -- 172.21.15.119:0/4215435565 shutdown_connections 2023-10-08T02:12:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.534+0000 7fb63a33fb80 1 -- 172.21.15.119:0/4215435565 wait complete. 2023-10-08T02:12:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb63a33fb80 20 client.9321 trim_cache size 0 max 16384 2023-10-08T02:12:36.544 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:12:36.544 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T02:12:36.544 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T02:12:36.544 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 9150, 2023-10-08T02:12:36.544 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T02:12:36.544 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T02:12:36.544 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T02:12:36.544 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 11.421920587000001, 2023-10-08T02:12:36.545 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 79, 2023-10-08T02:12:36.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 171, 2023-10-08T02:12:36.545 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 171, 2023-10-08T02:12:36.545 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 17.208309905 2023-10-08T02:12:36.545 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:12:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 -- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb608066f30 msgr2=0x7fb6080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb608066f30 0x7fb6080693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb620006f00 tx=0x7fb620008040 comp rx=0 tx=0).stop 2023-10-08T02:12:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 -- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62406c0a0 msgr2=0x7fb624157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62406c0a0 0x7fb624157de0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fb61400d860 tx=0x7fb614008040 comp rx=0 tx=0).stop 2023-10-08T02:12:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 -- 172.21.15.119:0/234841531 shutdown_connections 2023-10-08T02:12:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fb608066f30 0x7fb6080693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb62406bcd0 0x7fb62414f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb62415df50 0x7fb62406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 --2- 172.21.15.119:0/234841531 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb62406c0a0 0x7fb624157de0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 -- 172.21.15.119:0/234841531 >> 172.21.15.119:0/234841531 conn(0x7fb6240ce600 msgr2=0x7fb624157240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 -- 172.21.15.119:0/234841531 shutdown_connections 2023-10-08T02:12:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.535+0000 7fb62bbff700 1 -- 172.21.15.119:0/234841531 wait complete. 2023-10-08T02:12:36.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T02:12:36.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.730+0000 7f31a83ff700 1 Processor -- start 2023-10-08T02:12:36.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.730+0000 7f31a83ff700 1 -- start start 2023-10-08T02:12:36.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.731+0000 7f31a83ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31a00b1420 0x7f31a00ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.731+0000 7f31a83ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f31a00ccf90 0x7f31a00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.731+0000 7f31a83ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a00cc300 con 0x7f31a00cbf30 2023-10-08T02:12:36.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.731+0000 7f31a83ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a0156bf0 con 0x7f31a00ccf90 2023-10-08T02:12:36.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.731+0000 7f31a83ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a00ae950 con 0x7f31a00b1420 2023-10-08T02:12:36.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a619b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f31a00cbf30 0x7f31a00b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60016/0 (socket says 172.21.15.119:60016) 2023-10-08T02:12:36.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a619b700 1 -- 172.21.15.119:0/1637403865 learned_addr learned my addr 172.21.15.119:0/1637403865 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:36.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a699c700 1 --2- 172.21.15.119:0/1637403865 >> v2:172.21.15.184:3300/0 conn(0x7f31a00ccf90 0x7f31a00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777050387 0 0) 0x7f31a00cc300 con 0x7f31a00cbf30 2023-10-08T02:12:36.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318c003570 con 0x7f31a00cbf30 2023-10-08T02:12:36.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a599a700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31a00b1420 0x7f31a00ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 328719293 0 0) 0x7f318c003570 con 0x7f31a00cbf30 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31a00b1420 msgr2=0x7f31a00ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31a00b1420 0x7f31a00ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 >> v2:172.21.15.184:3300/0 conn(0x7f31a00ccf90 msgr2=0x7f31a00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 --2- 172.21.15.119:0/1637403865 >> v2:172.21.15.184:3300/0 conn(0x7f31a00ccf90 0x7f31a00c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a00ad370 con 0x7f31a00cbf30 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.732+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f319c001470 con 0x7f31a00cbf30 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 >> v1:172.21.15.119:6789/0 conn(0x7f31a00cbf30 legacy=0x7f31a00b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a5199700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31a00b1420 0x7f318c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a5199700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f318c004010 0x7f318c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a5199700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c004d50 0x7f318c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a0156bf0 con 0x7f318c004010 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a00ae950 con 0x7f318c004d50 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f318c00dcc0 con 0x7f31a00b1420 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a599a700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f318c004010 0x7f318c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a619b700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31a00b1420 0x7f318c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a699c700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c004d50 0x7f318c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a599a700 1 -- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31a00b1420 msgr2=0x7f318c003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a599a700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f31a00b1420 0x7f318c003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a599a700 1 -- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c004d50 msgr2=0x7f318c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a599a700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f318c004d50 0x7f318c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.733+0000 7f31a599a700 1 -- 172.21.15.119:0/1637403865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c003570 con 0x7f318c004010 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a599a700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f318c004010 0x7f318c004810 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3190009c00 tx=0x7f31900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab9acc97d18d4e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000e710 con 0x7f318c004010 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319000e870 con 0x7f318c004010 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a5199700 1 -- 172.21.15.119:0/1637403865 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319000ec30 con 0x7f318c004010 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a83ff700 1 -- 172.21.15.119:0/1637403865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f318c004010 msgr2=0x7f318c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a83ff700 1 --2- 172.21.15.119:0/1637403865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f318c004010 0x7f318c004810 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f3190009c00 tx=0x7f31900091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a699c700 1 -- 172.21.15.119:0/1637403865 reap_dead start 2023-10-08T02:12:36.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a83ff700 1 -- 172.21.15.119:0/1637403865 shutdown_connections 2023-10-08T02:12:36.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.734+0000 7f31a83ff700 1 -- 172.21.15.119:0/1637403865 >> 172.21.15.119:0/1637403865 conn(0x7f31a006a320 msgr2=0x7f31a00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:36.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.735+0000 7f31a83ff700 1 -- 172.21.15.119:0/1637403865 shutdown_connections 2023-10-08T02:12:36.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.735+0000 7f31a83ff700 1 -- 172.21.15.119:0/1637403865 wait complete. 2023-10-08T02:12:36.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.736+0000 7f31a83ff700 1 Processor -- start 2023-10-08T02:12:36.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.736+0000 7f31a83ff700 1 -- start start 2023-10-08T02:12:36.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a83ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c004010 0x7f31a0141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a83ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31a0151df0 0x7f31a0143b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a83ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a013e5d0 0x7f31a013ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a83ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a013fb30 con 0x7f31a013e5d0 2023-10-08T02:12:36.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a83ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a013ef20 con 0x7f31a0151df0 2023-10-08T02:12:36.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a83ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a013f200 con 0x7f318c004010 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a013e5d0 0x7f31a013ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a699c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a013e5d0 0x7f31a013ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46430/0 (socket says 172.21.15.119:46430) 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a699c700 1 -- 172.21.15.119:0/951519418 learned_addr learned my addr 172.21.15.119:0/951519418 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a599a700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31a0151df0 0x7f31a0143b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.737+0000 7f31a619b700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c004010 0x7f31a0141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a699c700 1 -- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c004010 msgr2=0x7f31a0141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a699c700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c004010 0x7f31a0141130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a699c700 1 -- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31a0151df0 msgr2=0x7f31a0143b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a699c700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31a0151df0 0x7f31a0143b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a699c700 1 -- 172.21.15.119:0/951519418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a013f4e0 con 0x7f31a013e5d0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a599a700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31a0151df0 0x7f31a0143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a699c700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a013e5d0 0x7f31a013ffc0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f31a00b03d0 tx=0x7f319800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31967fc700 1 -- 172.21.15.119:0/951519418 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319800e040 con 0x7f31a013e5d0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31967fc700 1 -- 172.21.15.119:0/951519418 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319800f040 con 0x7f31a013e5d0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a013f7c0 con 0x7f31a013e5d0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31967fc700 1 -- 172.21.15.119:0/951519418 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319800fe20 con 0x7f31a013e5d0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.738+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a0144060 con 0x7f31a013e5d0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.739+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a004c3c0 con 0x7f31a013e5d0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.740+0000 7f31967fc700 1 -- 172.21.15.119:0/951519418 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3198015640 con 0x7f31a013e5d0 2023-10-08T02:12:36.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.741+0000 7f31967fc700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3184066f30 0x7f31840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:36.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.741+0000 7f31967fc700 1 -- 172.21.15.119:0/951519418 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f31980850a0 con 0x7f31a013e5d0 2023-10-08T02:12:36.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.741+0000 7f31a619b700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3184066f30 0x7f31840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:36.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.742+0000 7f31a619b700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3184066f30 0x7f31840693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f319c001200 tx=0x7f319c009040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:36.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.745+0000 7f31967fc700 1 -- 172.21.15.119:0/951519418 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f31980520a0 con 0x7f31a013e5d0 2023-10-08T02:12:36.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.978+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f31a00ad650 con 0x7f31a013e5d0 2023-10-08T02:12:36.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.978+0000 7f31967fc700 1 -- 172.21.15.119:0/951519418 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7f31980556c0 con 0x7f31a013e5d0 2023-10-08T02:12:36.979 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T02:12:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3184066f30 msgr2=0x7f31840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3184066f30 0x7f31840693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f319c001200 tx=0x7f319c009040 comp rx=0 tx=0).stop 2023-10-08T02:12:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a013e5d0 msgr2=0x7f31a013ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a013e5d0 0x7f31a013ffc0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f31a00b03d0 tx=0x7f319800b040 comp rx=0 tx=0).stop 2023-10-08T02:12:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 shutdown_connections 2023-10-08T02:12:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f3184066f30 0x7f31840693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f318c004010 0x7f31a0141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31a013e5d0 0x7f31a013ffc0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 --2- 172.21.15.119:0/951519418 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f31a0151df0 0x7f31a0143b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 >> 172.21.15.119:0/951519418 conn(0x7f31a006a320 msgr2=0x7f31a00ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 shutdown_connections 2023-10-08T02:12:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:36.981+0000 7f31a83ff700 1 -- 172.21.15.119:0/951519418 wait complete. 2023-10-08T02:12:36.999 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T02:12:36.999 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T02:12:36.999 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T02:12:36.999 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T02:12:36.999 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T02:12:36.999 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T02:12:36.999 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T02:12:37.000 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T02:12:37.000 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:12:37.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:12:37.000 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T02:12:37.006 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:12:37.037 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T02:12:37.037 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:12:37.038 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 37694sec preferred_lft 37694sec 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T02:12:37.039 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:12:37.040 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T02:12:37.041 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T02:12:37.041 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T02:12:37.041 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:12:37.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:12:37.041 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T02:12:37.042 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:12:37.141 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T02:12:37.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:12:37.144 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T02:12:37.223 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T02:12:37.223 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T02:12:37.223 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T02:12:37.223 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T02:12:37.223 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T02:12:37.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:12:37.224 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T02:12:37.225 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T02:12:37.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T02:12:37.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T02:12:37.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T02:12:37.225 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T02:12:37.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:12:37.419 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T02:12:37.455 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T02:12:37.455 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T02:12:37.456 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:12:37.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:12:37.456 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T02:12:37.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T02:12:37.557 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T02:12:37.593 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:12:37.594 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T02:12:37.594 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T02:12:37.668 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T02:12:37.668 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T02:12:37.669 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:12:37.798 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:12:37.799 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T02:12:37.827 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:12:37.827 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T02:12:37.827 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T02:12:37.828 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T02:12:37.828 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T02:12:37.833 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T02:12:37.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:12:37.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T02:12:37.895 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T02:12:38.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.075+0000 7fdfc0f9d700 1 Processor -- start 2023-10-08T02:12:38.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.075+0000 7fdfc0f9d700 1 -- start start 2023-10-08T02:12:38.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.076+0000 7fdfc0f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 0x7fdfbc0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.076+0000 7fdfc0f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfbc1566e0 0x7fdfbc0ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.076+0000 7fdfc0f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdfbc0ca300 0x7fdfbc0ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.076+0000 7fdfc0f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0ca860 con 0x7fdfbc1566e0 2023-10-08T02:12:38.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.076+0000 7fdfc0f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfbc0cab40 con 0x7fdfbc0ca300 2023-10-08T02:12:38.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.076+0000 7fdfc0f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc0cae20 con 0x7fdfbc156310 2023-10-08T02:12:38.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.076+0000 7fdfbaffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfbc1566e0 0x7fdfbc0ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbb7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 0x7fdfbc0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbaffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfbc1566e0 0x7fdfbc0ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46434/0 (socket says 172.21.15.119:46434) 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbaffd700 1 -- 172.21.15.119:0/3693038109 learned_addr learned my addr 172.21.15.119:0/3693038109 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbbfff700 1 --2- 172.21.15.119:0/3693038109 >> v2:172.21.15.184:3300/0 conn(0x7fdfbc0ca300 0x7fdfbc0ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbaffd700 1 -- 172.21.15.119:0/3693038109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 msgr2=0x7fdfbc0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbaffd700 1 --2- 172.21.15.119:0/3693038109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 0x7fdfbc0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbaffd700 1 -- 172.21.15.119:0/3693038109 >> v2:172.21.15.184:3300/0 conn(0x7fdfbc0ca300 msgr2=0x7fdfbc0ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbaffd700 1 --2- 172.21.15.119:0/3693038109 >> v2:172.21.15.184:3300/0 conn(0x7fdfbc0ca300 0x7fdfbc0ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbaffd700 1 -- 172.21.15.119:0/3693038109 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfbc0cb100 con 0x7fdfbc1566e0 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfbaffd700 1 --2- 172.21.15.119:0/3693038109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc0ae000 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc06f970 tx=0x7fdfb000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b0abd74f0354ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:38.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.077+0000 7fdfba7fc700 1 -- 172.21.15.119:0/3693038109 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb000db40 con 0x7fdfbc1566e0 2023-10-08T02:12:38.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfba7fc700 1 -- 172.21.15.119:0/3693038109 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb000dca0 con 0x7fdfbc1566e0 2023-10-08T02:12:38.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfba7fc700 1 -- 172.21.15.119:0/3693038109 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb00196b0 con 0x7fdfbc1566e0 2023-10-08T02:12:38.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/3693038109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 msgr2=0x7fdfbc0ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/3693038109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc0ae000 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc06f970 tx=0x7fdfb000b4f0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/3693038109 shutdown_connections 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/3693038109 >> v2:172.21.15.184:3300/0 conn(0x7fdfbc0ca300 0x7fdfbc0ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/3693038109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 0x7fdfbc0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/3693038109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc0ae000 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/3693038109 >> 172.21.15.119:0/3693038109 conn(0x7fdfbc0cc1f0 msgr2=0x7fdfbc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/3693038109 shutdown_connections 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.078+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/3693038109 wait complete. 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.080+0000 7fdfc0f9d700 1 Processor -- start 2023-10-08T02:12:38.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.080+0000 7fdfc0f9d700 1 -- start start 2023-10-08T02:12:38.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfc0f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0ca300 0x7fdfbc0c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfc0f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 0x7fdfbc15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfc0f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc165bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfc0f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc157b50 con 0x7fdfbc1566e0 2023-10-08T02:12:38.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfc0f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc157da0 con 0x7fdfbc0ca300 2023-10-08T02:12:38.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfc0f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfbc158080 con 0x7fdfbc156310 2023-10-08T02:12:38.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc165bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc165bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46450/0 (socket says 172.21.15.119:46450) 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbbfff700 1 -- 172.21.15.119:0/2019158584 learned_addr learned my addr 172.21.15.119:0/2019158584 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbaffd700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 0x7fdfbc15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbb7fe700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0ca300 0x7fdfbc0c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbbfff700 1 -- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 msgr2=0x7fdfbc15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbbfff700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 0x7fdfbc15a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbbfff700 1 -- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0ca300 msgr2=0x7fdfbc0c34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbbfff700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0ca300 0x7fdfbc0c34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbbfff700 1 -- 172.21.15.119:0/2019158584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb000b790 con 0x7fdfbc1566e0 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.081+0000 7fdfbb7fe700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0ca300 0x7fdfbc0c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.082+0000 7fdfbbfff700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc165bb0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009df0 tx=0x7fdfb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.082+0000 7fdfa37fe700 1 -- 172.21.15.119:0/2019158584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb400cb50 con 0x7fdfbc1566e0 2023-10-08T02:12:38.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.082+0000 7fdfa37fe700 1 -- 172.21.15.119:0/2019158584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb400f040 con 0x7fdfbc1566e0 2023-10-08T02:12:38.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.082+0000 7fdfa37fe700 1 -- 172.21.15.119:0/2019158584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4014480 con 0x7fdfbc1566e0 2023-10-08T02:12:38.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.082+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfbc158360 con 0x7fdfbc1566e0 2023-10-08T02:12:38.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.082+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfbc0c3ab0 con 0x7fdfbc1566e0 2023-10-08T02:12:38.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.083+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfbc158860 con 0x7fdfbc1566e0 2023-10-08T02:12:38.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.084+0000 7fdfa37fe700 1 -- 172.21.15.119:0/2019158584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdfb4014830 con 0x7fdfbc1566e0 2023-10-08T02:12:38.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.085+0000 7fdfa37fe700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.085+0000 7fdfa37fe700 1 -- 172.21.15.119:0/2019158584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7fdfb40850b0 con 0x7fdfbc1566e0 2023-10-08T02:12:38.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.086+0000 7fdfbb7fe700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.086+0000 7fdfbb7fe700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009d80 tx=0x7fdfac008040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:38.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.089+0000 7fdfa37fe700 1 -- 172.21.15.119:0/2019158584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdfb40520b0 con 0x7fdfbc1566e0 2023-10-08T02:12:38.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.306+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfbc157f10 con 0x7fdfa8066f30 2023-10-08T02:12:38.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.307+0000 7fdfa37fe700 1 -- 172.21.15.119:0/2019158584 <== mgr.8967 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fdfbc157f10 con 0x7fdfa8066f30 2023-10-08T02:12:38.308 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T02:12:38.308 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:12:38.308 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T02:12:38.308 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:12:38.309 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T02:12:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdfa8066f30 msgr2=0x7fdfa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009d80 tx=0x7fdfac008040 comp rx=0 tx=0).stop 2023-10-08T02:12:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 msgr2=0x7fdfbc165bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc165bb0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fdfb4009df0 tx=0x7fdfb400b040 comp rx=0 tx=0).stop 2023-10-08T02:12:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 shutdown_connections 2023-10-08T02:12:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7fdfa8066f30 0x7fdfa80693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfbc1566e0 0x7fdfbc165bb0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdfbc0ca300 0x7fdfbc0c34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 --2- 172.21.15.119:0/2019158584 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdfbc156310 0x7fdfbc15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 >> 172.21.15.119:0/2019158584 conn(0x7fdfbc0cc1f0 msgr2=0x7fdfbc0c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 shutdown_connections 2023-10-08T02:12:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.310+0000 7fdfc0f9d700 1 -- 172.21.15.119:0/2019158584 wait complete. 2023-10-08T02:12:38.328 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T02:12:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.511+0000 7feeda9ff700 1 Processor -- start 2023-10-08T02:12:38.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.511+0000 7feeda9ff700 1 -- start start 2023-10-08T02:12:38.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.513+0000 7feeda9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feed40ccee0 0x7feed41561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.513+0000 7feeda9ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed415b0d0 con 0x7feed40c87c0 2023-10-08T02:12:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.513+0000 7feeda9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed415a4c0 con 0x7feed40c83f0 2023-10-08T02:12:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.513+0000 7feeda9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed415a7a0 con 0x7feed40ccee0 2023-10-08T02:12:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.513+0000 7feed8f9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7feed40c87c0 0x7feed40ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60022/0 (socket says 172.21.15.119:60022) 2023-10-08T02:12:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.513+0000 7feed8f9c700 1 -- 172.21.15.119:0/2237691677 learned_addr learned my addr 172.21.15.119:0/2237691677 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:38.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.513+0000 7feed37fe700 1 --2- 172.21.15.119:0/2237691677 >> v2:172.21.15.184:3300/0 conn(0x7feed40ccee0 0x7feed41561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.513+0000 7feed2ffd700 1 -- 172.21.15.119:0/2237691677 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939619405 0 0) 0x7feed415b0d0 con 0x7feed40c87c0 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed2ffd700 1 -- 172.21.15.119:0/2237691677 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feebc003570 con 0x7feed40c87c0 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed2ffd700 1 -- 172.21.15.119:0/2237691677 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585863503 0 0) 0x7feed415a4c0 con 0x7feed40c83f0 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed2ffd700 1 -- 172.21.15.119:0/2237691677 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feed415b0d0 con 0x7feed40c83f0 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed37fe700 1 -- 172.21.15.119:0/2237691677 >> v1:172.21.15.184:6789/0 conn(0x7feed40c83f0 legacy=0x7feed40c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed37fe700 1 -- 172.21.15.119:0/2237691677 >> v1:172.21.15.119:6789/0 conn(0x7feed40c87c0 legacy=0x7feed40ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed2ffd700 1 -- 172.21.15.119:0/2237691677 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4174330540 0 0) 0x7feebc003570 con 0x7feed40c87c0 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed37fe700 1 -- 172.21.15.119:0/2237691677 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed415aa80 con 0x7feed40ccee0 2023-10-08T02:12:38.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed37fe700 1 --2- 172.21.15.119:0/2237691677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40ccee0 0x7feed41561d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7feec80098e0 tx=0x7feec80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9660696ccf3f8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed2ffd700 1 -- 172.21.15.119:0/2237691677 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8011630 con 0x7feed40ccee0 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed2ffd700 1 -- 172.21.15.119:0/2237691677 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec8011790 con 0x7feed40ccee0 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.514+0000 7feed2ffd700 1 -- 172.21.15.119:0/2237691677 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8011b30 con 0x7feed40ccee0 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.515+0000 7feeda9ff700 1 -- 172.21.15.119:0/2237691677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40ccee0 msgr2=0x7feed41561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.515+0000 7feeda9ff700 1 --2- 172.21.15.119:0/2237691677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40ccee0 0x7feed41561d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7feec80098e0 tx=0x7feec80091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.515+0000 7feeda9ff700 1 -- 172.21.15.119:0/2237691677 shutdown_connections 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.515+0000 7feeda9ff700 1 --2- 172.21.15.119:0/2237691677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40ccee0 0x7feed41561d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.515+0000 7feeda9ff700 1 -- 172.21.15.119:0/2237691677 >> 172.21.15.119:0/2237691677 conn(0x7feed4068e10 msgr2=0x7feed40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.515+0000 7feeda9ff700 1 -- 172.21.15.119:0/2237691677 shutdown_connections 2023-10-08T02:12:38.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.515+0000 7feeda9ff700 1 -- 172.21.15.119:0/2237691677 wait complete. 2023-10-08T02:12:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.517+0000 7feeda9ff700 1 Processor -- start 2023-10-08T02:12:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.517+0000 7feeda9ff700 1 -- start start 2023-10-08T02:12:38.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feeda9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40c83f0 0x7feed4144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feeda9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed40c87c0 0x7feed40c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feeda9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed40ccee0 0x7feed40c35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feeda9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed40b0100 con 0x7feed40ccee0 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feeda9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed4144fa0 con 0x7feed40c83f0 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feeda9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed4145280 con 0x7feed40c87c0 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed8f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed40ccee0 0x7feed40c35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed8f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed40ccee0 0x7feed40c35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46460/0 (socket says 172.21.15.119:46460) 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed8f9c700 1 -- 172.21.15.119:0/2279593800 learned_addr learned my addr 172.21.15.119:0/2279593800 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed37fe700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed40c87c0 0x7feed40c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed3fff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40c83f0 0x7feed4144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed8f9c700 1 -- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed40c87c0 msgr2=0x7feed40c48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed8f9c700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed40c87c0 0x7feed40c48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed8f9c700 1 -- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40c83f0 msgr2=0x7feed4144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed8f9c700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40c83f0 0x7feed4144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed8f9c700 1 -- 172.21.15.119:0/2279593800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8009420 con 0x7feed40ccee0 2023-10-08T02:12:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed37fe700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed40c87c0 0x7feed40c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.518+0000 7feed3fff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40c83f0 0x7feed4144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:12:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.519+0000 7feed8f9c700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed40ccee0 0x7feed40c35e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7feecc00c950 tx=0x7feecc007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:38.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.519+0000 7feebbfff700 1 -- 172.21.15.119:0/2279593800 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc004dc0 con 0x7feed40ccee0 2023-10-08T02:12:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.519+0000 7feebbfff700 1 -- 172.21.15.119:0/2279593800 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc00a5d0 con 0x7feed40ccee0 2023-10-08T02:12:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.519+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed4145560 con 0x7feed40ccee0 2023-10-08T02:12:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.519+0000 7feebbfff700 1 -- 172.21.15.119:0/2279593800 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc00a990 con 0x7feed40ccee0 2023-10-08T02:12:38.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.519+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed40c4e00 con 0x7feed40ccee0 2023-10-08T02:12:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.520+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed40b0270 con 0x7feed40ccee0 2023-10-08T02:12:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.521+0000 7feebbfff700 1 -- 172.21.15.119:0/2279593800 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 87) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feecc007150 con 0x7feed40ccee0 2023-10-08T02:12:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.522+0000 7feebbfff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feec0066f30 0x7feec00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.522+0000 7feebbfff700 1 -- 172.21.15.119:0/2279593800 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7feecc084110 con 0x7feed40ccee0 2023-10-08T02:12:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.522+0000 7feed3fff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feec0066f30 0x7feec00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:38.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.523+0000 7feed3fff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feec0066f30 0x7feec00693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feec40038a0 tx=0x7feec4009040 comp rx=0 tx=0).ready entity=mgr.8967 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:38.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.526+0000 7feebbfff700 1 -- 172.21.15.119:0/2279593800 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feecc051190 con 0x7feed40ccee0 2023-10-08T02:12:38.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:38.775+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7feed404c3c0 con 0x7feed40ccee0 2023-10-08T02:12:39.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.192+0000 7feebbfff700 1 -- 172.21.15.119:0/2279593800 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 88) v1 ==== 84757+0+0 (secure 0 0 0) 0x7feecc05a080 con 0x7feed40ccee0 2023-10-08T02:12:39.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.193+0000 7feebbfff700 1 -- 172.21.15.119:0/2279593800 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v88) v1 ==== 92+0+0 (secure 0 0 0) 0x7feecc051190 con 0x7feed40ccee0 2023-10-08T02:12:39.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feec0066f30 msgr2=0x7feec00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feec0066f30 0x7feec00693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7feec40038a0 tx=0x7feec4009040 comp rx=0 tx=0).stop 2023-10-08T02:12:39.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed40ccee0 msgr2=0x7feed40c35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed40ccee0 0x7feed40c35e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7feecc00c950 tx=0x7feecc007ed0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 shutdown_connections 2023-10-08T02:12:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7feec0066f30 0x7feec00693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feed40ccee0 0x7feed40c35e0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feed40c83f0 0x7feed4144910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 --2- 172.21.15.119:0/2279593800 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7feed40c87c0 0x7feed40c48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 >> 172.21.15.119:0/2279593800 conn(0x7feed4068e10 msgr2=0x7feed40cb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 shutdown_connections 2023-10-08T02:12:39.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.196+0000 7feeda9ff700 1 -- 172.21.15.119:0/2279593800 wait complete. 2023-10-08T02:12:39.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T02:12:39.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.407+0000 7ff0e759e700 1 Processor -- start 2023-10-08T02:12:39.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.407+0000 7ff0e759e700 1 -- start start 2023-10-08T02:12:39.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e759e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff0e80aef00 0x7ff0e815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0e806e470 0x7ff0e806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e759e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0e80aea70 con 0x7ff0e815a3b0 2023-10-08T02:12:39.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e759e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e806ee30 con 0x7ff0e806e470 2023-10-08T02:12:39.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e759e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0e80ae480 con 0x7ff0e80aef00 2023-10-08T02:12:39.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e6d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0e806e470 0x7ff0e806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e659c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff0e815a3b0 0x7ff0e815a780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60052/0 (socket says 172.21.15.119:60052) 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e659c700 1 -- 172.21.15.119:0/361749429 learned_addr learned my addr 172.21.15.119:0/361749429 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e6d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0e806e470 0x7ff0e806e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46478/0 (socket says 172.21.15.119:46478) 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e5d9b700 1 --2- 172.21.15.119:0/361749429 >> v2:172.21.15.184:3300/0 conn(0x7ff0e80aef00 0x7ff0e815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e6d9d700 1 -- 172.21.15.119:0/361749429 >> v2:172.21.15.184:3300/0 conn(0x7ff0e80aef00 msgr2=0x7ff0e815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e6d9d700 1 --2- 172.21.15.119:0/361749429 >> v2:172.21.15.184:3300/0 conn(0x7ff0e80aef00 0x7ff0e815d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e6d9d700 1 -- 172.21.15.119:0/361749429 >> v1:172.21.15.119:6789/0 conn(0x7ff0e815a3b0 legacy=0x7ff0e815a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e6d9d700 1 -- 172.21.15.119:0/361749429 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e815dd30 con 0x7ff0e806e470 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.409+0000 7ff0e559a700 1 -- 172.21.15.119:0/361749429 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705926569 0 0) 0x7ff0e80aea70 con 0x7ff0e815a3b0 2023-10-08T02:12:39.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.410+0000 7ff0e6d9d700 1 --2- 172.21.15.119:0/361749429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e806e8f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ff0e806d7e0 tx=0x7ff0e0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c9d938f4835dbe server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.410+0000 7ff0e559a700 1 -- 172.21.15.119:0/361749429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e000e070 con 0x7ff0e806e470 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.410+0000 7ff0e559a700 1 -- 172.21.15.119:0/361749429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e000bb60 con 0x7ff0e806e470 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.410+0000 7ff0e559a700 1 -- 172.21.15.119:0/361749429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e0012430 con 0x7ff0e806e470 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.411+0000 7ff0e759e700 1 -- 172.21.15.119:0/361749429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 msgr2=0x7ff0e806e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.411+0000 7ff0e759e700 1 --2- 172.21.15.119:0/361749429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e806e8f0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7ff0e806d7e0 tx=0x7ff0e0009780 comp rx=0 tx=0).stop 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.411+0000 7ff0e759e700 1 -- 172.21.15.119:0/361749429 shutdown_connections 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.411+0000 7ff0e759e700 1 --2- 172.21.15.119:0/361749429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e806e8f0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.411+0000 7ff0e759e700 1 --2- 172.21.15.119:0/361749429 >> v2:172.21.15.184:3300/0 conn(0x7ff0e80aef00 0x7ff0e815d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.411+0000 7ff0e759e700 1 -- 172.21.15.119:0/361749429 >> 172.21.15.119:0/361749429 conn(0x7ff0e80c9f70 msgr2=0x7ff0e806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.411+0000 7ff0e759e700 1 -- 172.21.15.119:0/361749429 shutdown_connections 2023-10-08T02:12:39.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.411+0000 7ff0e759e700 1 -- 172.21.15.119:0/361749429 wait complete. 2023-10-08T02:12:39.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.412+0000 7ff0e759e700 1 Processor -- start 2023-10-08T02:12:39.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.412+0000 7ff0e759e700 1 -- start start 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.413+0000 7ff0e759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e80c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.413+0000 7ff0e759e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e80aef00 0x7ff0e80c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.413+0000 7ff0e759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e815a3b0 0x7ff0e80b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.413+0000 7ff0e759e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e816e890 con 0x7ff0e806e470 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.413+0000 7ff0e759e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e806c600 con 0x7ff0e80aef00 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.413+0000 7ff0e759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e816dc10 con 0x7ff0e815a3b0 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.413+0000 7ff0e659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e80c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.413+0000 7ff0e659c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e80c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46492/0 (socket says 172.21.15.119:46492) 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e659c700 1 -- 172.21.15.119:0/2786061695 learned_addr learned my addr 172.21.15.119:0/2786061695 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:39.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e5d9b700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e80aef00 0x7ff0e80c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e6d9d700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e815a3b0 0x7ff0e80b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e659c700 1 -- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e815a3b0 msgr2=0x7ff0e80b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e659c700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e815a3b0 0x7ff0e80b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e659c700 1 -- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e80aef00 msgr2=0x7ff0e80c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e659c700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e80aef00 0x7ff0e80c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e659c700 1 -- 172.21.15.119:0/2786061695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e0009a20 con 0x7ff0e806e470 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e5d9b700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e80aef00 0x7ff0e80c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e659c700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e80c5920 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c5e0 tx=0x7ff0d8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0d67fc700 1 -- 172.21.15.119:0/2786061695 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8005b30 con 0x7ff0e806e470 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0d67fc700 1 -- 172.21.15.119:0/2786061695 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0d8005c90 con 0x7ff0e806e470 2023-10-08T02:12:39.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0d67fc700 1 -- 172.21.15.119:0/2786061695 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8007300 con 0x7ff0e806e470 2023-10-08T02:12:39.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.414+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e816def0 con 0x7ff0e806e470 2023-10-08T02:12:39.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.415+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e816e4b0 con 0x7ff0e806e470 2023-10-08T02:12:39.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.416+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e806cfa0 con 0x7ff0e806e470 2023-10-08T02:12:39.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.416+0000 7ff0d67fc700 1 -- 172.21.15.119:0/2786061695 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 88) v1 ==== 84757+0+0 (secure 0 0 0) 0x7ff0d80076b0 con 0x7ff0e806e470 2023-10-08T02:12:39.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.418+0000 7ff0d67fc700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0c8066f80 0x7ff0c8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.418+0000 7ff0e5d9b700 1 -- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0c8066f80 msgr2=0x7ff0c8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:39.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.418+0000 7ff0d67fc700 1 -- 172.21.15.119:0/2786061695 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7ff0d8083b60 con 0x7ff0e806e470 2023-10-08T02:12:39.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.418+0000 7ff0e5d9b700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0c8066f80 0x7ff0c8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:39.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.422+0000 7ff0d67fc700 1 -- 172.21.15.119:0/2786061695 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0d8050b60 con 0x7ff0e806e470 2023-10-08T02:12:39.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.619+0000 7ff0e5d9b700 1 -- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0c8066f80 msgr2=0x7ff0c8069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:39.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.619+0000 7ff0e5d9b700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0c8066f80 0x7ff0c8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:39.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.635+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff0e815e010 con 0x7ff0e806e470 2023-10-08T02:12:39.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.640+0000 7ff0d67fc700 1 -- 172.21.15.119:0/2786061695 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff0d80170d0 con 0x7ff0e806e470 2023-10-08T02:12:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0c8066f80 msgr2=0x7ff0c8069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0c8066f80 0x7ff0c8069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 msgr2=0x7ff0e80c5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e80c5920 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800c5e0 tx=0x7ff0d8001070 comp rx=0 tx=0).stop 2023-10-08T02:12:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 shutdown_connections 2023-10-08T02:12:39.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7ff0c8066f80 0x7ff0c8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0e815a3b0 0x7ff0e80b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0e806e470 0x7ff0e80c5920 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 --2- 172.21.15.119:0/2786061695 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0e80aef00 0x7ff0e80c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 >> 172.21.15.119:0/2786061695 conn(0x7ff0e80c9f70 msgr2=0x7ff0e815d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 shutdown_connections 2023-10-08T02:12:39.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.642+0000 7ff0e759e700 1 -- 172.21.15.119:0/2786061695 wait complete. 2023-10-08T02:12:39.661 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T02:12:39.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.848+0000 7f90df59e700 1 Processor -- start 2023-10-08T02:12:39.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.848+0000 7f90df59e700 1 -- start start 2023-10-08T02:12:39.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.849+0000 7f90df59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f90e00b4e70 0x7f90e00ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.849+0000 7f90df59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e00cfb00 con 0x7f90e00b1b40 2023-10-08T02:12:39.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.849+0000 7f90df59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90e015e060 con 0x7f90e00b16d0 2023-10-08T02:12:39.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.849+0000 7f90df59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90e00b5c40 con 0x7f90e00b4e70 2023-10-08T02:12:39.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.849+0000 7f90ded9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f90e00b1b40 0x7f90e015d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60054/0 (socket says 172.21.15.119:60054) 2023-10-08T02:12:39.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.849+0000 7f90ded9d700 1 -- 172.21.15.119:0/570379930 learned_addr learned my addr 172.21.15.119:0/570379930 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:39.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90de59c700 1 --2- 172.21.15.119:0/570379930 >> v2:172.21.15.184:3300/0 conn(0x7f90e00b4e70 0x7f90e00ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90dd59a700 1 -- 172.21.15.119:0/570379930 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392105215 0 0) 0x7f90e00cfb00 con 0x7f90e00b1b40 2023-10-08T02:12:39.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90dd59a700 1 -- 172.21.15.119:0/570379930 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90c8003570 con 0x7f90e00b1b40 2023-10-08T02:12:39.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90dd59a700 1 -- 172.21.15.119:0/570379930 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229840970 0 0) 0x7f90e015e060 con 0x7f90e00b16d0 2023-10-08T02:12:39.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90dd59a700 1 -- 172.21.15.119:0/570379930 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e00cfb00 con 0x7f90e00b16d0 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90de59c700 1 -- 172.21.15.119:0/570379930 >> v1:172.21.15.184:6789/0 conn(0x7f90e00b16d0 legacy=0x7f90e015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90de59c700 1 -- 172.21.15.119:0/570379930 >> v1:172.21.15.119:6789/0 conn(0x7f90e00b1b40 legacy=0x7f90e015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90de59c700 1 -- 172.21.15.119:0/570379930 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e00b5f20 con 0x7f90e00b4e70 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.850+0000 7f90de59c700 1 --2- 172.21.15.119:0/570379930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b4e70 0x7f90e00ccff0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f90d00098e0 tx=0x7f90d00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f33a3c541bb837e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90dd59a700 1 -- 172.21.15.119:0/570379930 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0011630 con 0x7f90e00b4e70 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90dd59a700 1 -- 172.21.15.119:0/570379930 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d0011790 con 0x7f90e00b4e70 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90dd59a700 1 -- 172.21.15.119:0/570379930 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d0011b30 con 0x7f90e00b4e70 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90df59e700 1 -- 172.21.15.119:0/570379930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b4e70 msgr2=0x7f90e00ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90df59e700 1 --2- 172.21.15.119:0/570379930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b4e70 0x7f90e00ccff0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f90d00098e0 tx=0x7f90d00091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90df59e700 1 -- 172.21.15.119:0/570379930 shutdown_connections 2023-10-08T02:12:39.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90df59e700 1 --2- 172.21.15.119:0/570379930 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b4e70 0x7f90e00ccff0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90df59e700 1 -- 172.21.15.119:0/570379930 >> 172.21.15.119:0/570379930 conn(0x7f90e00d10c0 msgr2=0x7f90e00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:39.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90df59e700 1 -- 172.21.15.119:0/570379930 shutdown_connections 2023-10-08T02:12:39.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.851+0000 7f90df59e700 1 -- 172.21.15.119:0/570379930 wait complete. 2023-10-08T02:12:39.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.853+0000 7f90df59e700 1 Processor -- start 2023-10-08T02:12:39.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.853+0000 7f90df59e700 1 -- start start 2023-10-08T02:12:39.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90df59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b16d0 0x7f90e00af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90df59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e00b1b40 0x7f90e00c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90df59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e00b4e70 0x7f90e00c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90df59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00c7830 con 0x7f90e00b4e70 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90df59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00c7b10 con 0x7f90e00b16d0 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90df59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e00c9140 con 0x7f90e00b1b40 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90ded9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e00b4e70 0x7f90e00c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90de59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b16d0 0x7f90e00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90de59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b16d0 0x7f90e00af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:33230/0 (socket says 172.21.15.119:33230) 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90de59c700 1 -- 172.21.15.119:0/3107732395 learned_addr learned my addr 172.21.15.119:0/3107732395 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.854+0000 7f90ddd9b700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e00b1b40 0x7f90e00c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90ded9d700 1 -- 172.21.15.119:0/3107732395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e00b1b40 msgr2=0x7f90e00c6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90ded9d700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e00b1b40 0x7f90e00c6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90ded9d700 1 -- 172.21.15.119:0/3107732395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b16d0 msgr2=0x7f90e00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:39.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90ded9d700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b16d0 0x7f90e00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:39.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90ded9d700 1 -- 172.21.15.119:0/3107732395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90d0009420 con 0x7f90e00b4e70 2023-10-08T02:12:39.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90ded9d700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e00b4e70 0x7f90e00c7250 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f90d8001460 tx=0x7f90d8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:39.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90ce7fc700 1 -- 172.21.15.119:0/3107732395 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d800a840 con 0x7f90e00b4e70 2023-10-08T02:12:39.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e00c9420 con 0x7f90e00b4e70 2023-10-08T02:12:39.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.855+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90e0157e80 con 0x7f90e00b4e70 2023-10-08T02:12:39.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.856+0000 7f90ce7fc700 1 -- 172.21.15.119:0/3107732395 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90d800a9a0 con 0x7f90e00b4e70 2023-10-08T02:12:39.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.856+0000 7f90ce7fc700 1 -- 172.21.15.119:0/3107732395 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90d800ad60 con 0x7f90e00b4e70 2023-10-08T02:12:39.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.857+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c40052b0 con 0x7f90e00b4e70 2023-10-08T02:12:39.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.858+0000 7f90ce7fc700 1 -- 172.21.15.119:0/3107732395 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 88) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f90d8007300 con 0x7f90e00b4e70 2023-10-08T02:12:39.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.859+0000 7f90ce7fc700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f90c0066f80 0x7f90c0069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:39.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.859+0000 7f90de59c700 1 -- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f90c0066f80 msgr2=0x7f90c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:39.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.859+0000 7f90ce7fc700 1 -- 172.21.15.119:0/3107732395 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f90d80854e0 con 0x7f90e00b4e70 2023-10-08T02:12:39.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.859+0000 7f90de59c700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f90c0066f80 0x7f90c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:39.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:39.863+0000 7f90ce7fc700 1 -- 172.21.15.119:0/3107732395 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f90d80525e0 con 0x7f90e00b4e70 2023-10-08T02:12:40.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.059+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f90c4005830 con 0x7f90e00b4e70 2023-10-08T02:12:40.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.060+0000 7f90de59c700 1 -- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f90c0066f80 msgr2=0x7f90c0069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:40.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.060+0000 7f90de59c700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f90c0066f80 0x7f90c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:40.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.063+0000 7f90ce7fc700 1 -- 172.21.15.119:0/3107732395 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7f90d8055c00 con 0x7f90e00b4e70 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f90c0066f80 msgr2=0x7f90c0069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f90c0066f80 0x7f90c0069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e00b4e70 msgr2=0x7f90e00c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e00b4e70 0x7f90e00c7250 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f90d8001460 tx=0x7f90d8001a40 comp rx=0 tx=0).stop 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 shutdown_connections 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f90c0066f80 0x7f90c0069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f90e00b4e70 0x7f90e00c7250 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f90e00b16d0 0x7f90e00af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 --2- 172.21.15.119:0/3107732395 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f90e00b1b40 0x7f90e00c6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 >> 172.21.15.119:0/3107732395 conn(0x7f90e00d10c0 msgr2=0x7f90e0159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:40.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 shutdown_connections 2023-10-08T02:12:40.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.066+0000 7f90df59e700 1 -- 172.21.15.119:0/3107732395 wait complete. 2023-10-08T02:12:40.084 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:12:40.085 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T02:12:40.127 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:12:40.128 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-10-08T02:12:40.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-08T02:12:40.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.437+0000 7f9dba39d700 1 Processor -- start 2023-10-08T02:12:40.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.437+0000 7f9dba39d700 1 -- start start 2023-10-08T02:12:40.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.438+0000 7f9dba39d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9db40b0790 0x7f9db40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:40.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.438+0000 7f9dba39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db41591e0 0x7f9db4159620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:40.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9dba39d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db40cfa00 con 0x7f9db40b03c0 2023-10-08T02:12:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9dba39d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db40cb580 con 0x7f9db40b0790 2023-10-08T02:12:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9dba39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40cf160 con 0x7f9db41591e0 2023-10-08T02:12:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9db8b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9db40b0790 0x7f9db40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9db8b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9db40b0790 0x7f9db40cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46508/0 (socket says 172.21.15.119:46508) 2023-10-08T02:12:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9db8b9a700 1 -- 172.21.15.119:0/3226790663 learned_addr learned my addr 172.21.15.119:0/3226790663 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9db9b9c700 1 --2- 172.21.15.119:0/3226790663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db41591e0 0x7f9db4159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:40.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9db8b9a700 1 -- 172.21.15.119:0/3226790663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db41591e0 msgr2=0x7f9db4159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:40.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/3226790663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db41591e0 0x7f9db4159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9db8b9a700 1 -- 172.21.15.119:0/3226790663 >> v1:172.21.15.184:6789/0 conn(0x7f9db40b03c0 legacy=0x7f9db40b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9db8b9a700 1 -- 172.21.15.119:0/3226790663 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40cf440 con 0x7f9db40b0790 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.439+0000 7f9dabfff700 1 -- 172.21.15.119:0/3226790663 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042168511 0 0) 0x7f9db40cfa00 con 0x7f9db40b03c0 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.440+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/3226790663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b0790 0x7f9db40cea70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9da40099e0 tx=0x7f9da4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67937c20e9287707 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.440+0000 7f9dabfff700 1 -- 172.21.15.119:0/3226790663 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400b820 con 0x7f9db40b0790 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.440+0000 7f9dabfff700 1 -- 172.21.15.119:0/3226790663 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da400b980 con 0x7f9db40b0790 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.440+0000 7f9dabfff700 1 -- 172.21.15.119:0/3226790663 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400bd20 con 0x7f9db40b0790 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.441+0000 7f9dba39d700 1 -- 172.21.15.119:0/3226790663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b0790 msgr2=0x7f9db40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.441+0000 7f9dba39d700 1 --2- 172.21.15.119:0/3226790663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b0790 0x7f9db40cea70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9da40099e0 tx=0x7f9da4009250 comp rx=0 tx=0).stop 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.441+0000 7f9dba39d700 1 -- 172.21.15.119:0/3226790663 shutdown_connections 2023-10-08T02:12:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.441+0000 7f9dba39d700 1 --2- 172.21.15.119:0/3226790663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db41591e0 0x7f9db4159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.441+0000 7f9dba39d700 1 --2- 172.21.15.119:0/3226790663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b0790 0x7f9db40cea70 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.441+0000 7f9dba39d700 1 -- 172.21.15.119:0/3226790663 >> 172.21.15.119:0/3226790663 conn(0x7f9db4063230 msgr2=0x7f9db4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.441+0000 7f9dba39d700 1 -- 172.21.15.119:0/3226790663 shutdown_connections 2023-10-08T02:12:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.441+0000 7f9dba39d700 1 -- 172.21.15.119:0/3226790663 wait complete. 2023-10-08T02:12:40.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.442+0000 7f9dba39d700 1 Processor -- start 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.442+0000 7f9dba39d700 1 -- start start 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.443+0000 7f9dba39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b03c0 0x7f9db415b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.443+0000 7f9dba39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db40b0790 0x7f9db41597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.443+0000 7f9dba39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9db41591e0 0x7f9db41579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.443+0000 7f9dba39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db415bb10 con 0x7f9db40b03c0 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.443+0000 7f9dba39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db40c45f0 con 0x7f9db41591e0 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.443+0000 7f9dba39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db415a400 con 0x7f9db40b0790 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.443+0000 7f9db939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b03c0 0x7f9db415b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db939b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b03c0 0x7f9db415b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46522/0 (socket says 172.21.15.119:46522) 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db939b700 1 -- 172.21.15.119:0/34331094 learned_addr learned my addr 172.21.15.119:0/34331094 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db9b9c700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9db41591e0 0x7f9db41579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db40b0790 0x7f9db41597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db939b700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db40b0790 msgr2=0x7f9db41597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db939b700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db40b0790 0x7f9db41597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db939b700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9db41591e0 msgr2=0x7f9db41579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db939b700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9db41591e0 0x7f9db41579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db939b700 1 -- 172.21.15.119:0/34331094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40094f0 con 0x7f9db40b03c0 2023-10-08T02:12:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.444+0000 7f9db939b700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b03c0 0x7f9db415b5d0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f9db000c810 tx=0x7f9db0007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.445+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011590 con 0x7f9db40b03c0 2023-10-08T02:12:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.445+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db0011710 con 0x7f9db40b03c0 2023-10-08T02:12:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.445+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0011ad0 con 0x7f9db40b03c0 2023-10-08T02:12:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.445+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db40c4010 con 0x7f9db40b03c0 2023-10-08T02:12:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.445+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db415bdd0 con 0x7f9db40b03c0 2023-10-08T02:12:40.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.446+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db404c3c0 con 0x7f9db40b03c0 2023-10-08T02:12:40.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.447+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 88) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f9db001c460 con 0x7f9db40b03c0 2023-10-08T02:12:40.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.448+0000 7f9da97fa700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:40.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.448+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6169+0+0 (secure 0 0 0) 0x7f9db0084490 con 0x7f9db40b03c0 2023-10-08T02:12:40.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.448+0000 7f9db8b9a700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 msgr2=0x7f9d98069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:40.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.448+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:12:40.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.456+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9db0051530 con 0x7f9db40b03c0 2023-10-08T02:12:40.463 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:40.464+0000 7f6bce97a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:12:40.539 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:40.538+0000 7f1749833180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:12:40.627 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:40.627+0000 7f6bce97a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:12:40.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.649+0000 7f9db8b9a700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 msgr2=0x7f9d98069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:40.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.649+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:12:40.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:40.679+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9db40ce850 con 0x7f9d98066f80 2023-10-08T02:12:40.704 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:40.703+0000 7f1749833180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:12:40.711 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:40.712+0000 7f6bce97a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:12:40.787 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:40.787+0000 7f1749833180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:12:40.937 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:40.937+0000 7f6bce97a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:41.008 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:41.008+0000 7f1749833180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:41.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:41.050+0000 7f9db8b9a700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 msgr2=0x7f9d98069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:41.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:41.050+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:12:41.114 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:41.114+0000 7f6bce97a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:12:41.187 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:41.187+0000 7f1749833180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:12:41.210 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:41.210+0000 7f6bce97a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:12:41.278 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:41.277+0000 7f1749833180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:12:41.374 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:41.374+0000 7f6bce97a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:12:41.438 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:41.437+0000 7f1749833180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:12:41.475 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:41.475+0000 7f6bce97a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:12:41.538 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:41.537+0000 7f1749833180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:12:41.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:41.851+0000 7f9db8b9a700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 msgr2=0x7f9d98069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:41.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:41.851+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T02:12:41.958 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:41.959+0000 7f6bce97a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:12:42.014 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:42.014+0000 7f1749833180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:12:42.052 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:42.052+0000 7f6bce97a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:12:42.107 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:42.106+0000 7f1749833180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:12:42.141 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:42.141+0000 7f6bce97a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:12:42.196 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:42.196+0000 7f1749833180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:12:42.761 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:42.761+0000 7f6bce97a180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:12:42.810 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:42.809+0000 7f1749833180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:12:42.933 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:42.933+0000 7f6bce97a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:42.980 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:42.980+0000 7f1749833180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:12:43.029 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:43.029+0000 7f6bce97a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:12:43.074 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:43.074+0000 7f1749833180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:12:43.323 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:43.324+0000 7f6bce97a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:12:43.365 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:43.365+0000 7f1749833180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:12:43.420 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:43.420+0000 7f6bce97a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:12:43.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:43.453+0000 7f9db8b9a700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 msgr2=0x7f9d98069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/85511 2023-10-08T02:12:43.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:43.453+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T02:12:43.460 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:43.459+0000 7f1749833180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:12:44.123 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:44.123+0000 7f6bce97a180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:12:44.155 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:44.154+0000 7f1749833180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:12:44.412 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:44.412+0000 7f6bce97a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:12:44.439 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:44.439+0000 7f1749833180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:12:44.504 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:44.505+0000 7f6bce97a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:12:44.531 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:44.531+0000 7f1749833180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:12:44.659 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:44.659+0000 7f6bce97a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:12:44.684 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:44.683+0000 7f1749833180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:12:44.764 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:44.764+0000 7f6bce97a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:12:44.788 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:44.788+0000 7f1749833180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:12:44.953 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:44.953+0000 7f6bce97a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:12:44.976 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:44.975+0000 7f1749833180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:12:45.047 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:45.047+0000 7f6bce97a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:12:45.067 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.067+0000 7f1749833180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:12:45.139 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:45.139+0000 7f6bce97a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:12:45.158 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.158+0000 7f1749833180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:12:45.234 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:12:45.234+0000 7f6bce97a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:12:45.252 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.251+0000 7f1749833180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:12:45.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:45.260+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 89) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f9db0051900 con 0x7f9db40b03c0 2023-10-08T02:12:45.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:45.260+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 msgr2=0x7f9d98069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:12:45.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:45.260+0000 7f9da97fa700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:45.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.377+0000 7f16bbbb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.377+0000 7f16bbbb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.377+0000 7f16bbbb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.377+0000 7f16bbbb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.377+0000 7f16bbbb1700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.378+0000 7f16bebb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.379 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.378+0000 7f16bebb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.380 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.378+0000 7f16bebb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.380 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.378+0000 7f16bebb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:45.380 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:12:45.378+0000 7f16bebb7700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:12:46.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.267+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 90) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f9db000f260 con 0x7f9db40b03c0 2023-10-08T02:12:46.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.267+0000 7f9da97fa700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:46.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.267+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9db40ce850 con 0x7f9d98066f80 2023-10-08T02:12:46.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.269+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:46.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.272+0000 7f9db8b9a700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9da40099e0 tx=0x7f9da4017000 comp rx=0 tx=0).ready entity=mgr.9351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:46.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.340+0000 7f9da97fa700 1 -- 172.21.15.119:0/34331094 <== mgr.9351 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f9db40ce850 con 0x7f9d98066f80 2023-10-08T02:12:46.341 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-10-08T02:12:46.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 msgr2=0x7f9d98069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:46.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9da40099e0 tx=0x7f9da4017000 comp rx=0 tx=0).stop 2023-10-08T02:12:46.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b03c0 msgr2=0x7f9db415b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:46.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b03c0 0x7f9db415b5d0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f9db000c810 tx=0x7f9db0007df0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 shutdown_connections 2023-10-08T02:12:46.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f9d98066f80 0x7f9d98069430 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9db41591e0 0x7f9db41579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9db40b03c0 0x7f9db415b5d0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 --2- 172.21.15.119:0/34331094 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9db40b0790 0x7f9db41597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.342+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 >> 172.21.15.119:0/34331094 conn(0x7f9db4063230 msgr2=0x7f9db4068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:46.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.343+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 shutdown_connections 2023-10-08T02:12:46.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.343+0000 7f9dba39d700 1 -- 172.21.15.119:0/34331094 wait complete. 2023-10-08T02:12:46.362 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-10-08T02:12:46.362 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-08T02:12:46.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.549+0000 7f862a7ff700 1 Processor -- start 2023-10-08T02:12:46.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.549+0000 7f862a7ff700 1 -- start start 2023-10-08T02:12:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f862a7ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f862415a440 0x7f86240acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f862a7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8624068890 0x7f86240c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f862a7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862415a9b0 con 0x7f86241563b0 2023-10-08T02:12:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f862a7ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862415ac90 con 0x7f862415a440 2023-10-08T02:12:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f862a7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862415af70 con 0x7f8624068890 2023-10-08T02:12:46.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f8623fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f86241563b0 0x7f8624156780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41674/0 (socket says 172.21.15.119:41674) 2023-10-08T02:12:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f8623fff700 1 -- 172.21.15.119:0/3762560942 learned_addr learned my addr 172.21.15.119:0/3762560942 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:12:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f86237fe700 1 --2- 172.21.15.119:0/3762560942 >> v2:172.21.15.119:3300/0 conn(0x7f862415a440 0x7f86240acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f8628d9c700 1 --2- 172.21.15.119:0/3762560942 >> v2:172.21.15.184:3300/0 conn(0x7f8624068890 0x7f86240c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f8622ffd700 1 -- 172.21.15.119:0/3762560942 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 771170007 0 0) 0x7f862415a9b0 con 0x7f86241563b0 2023-10-08T02:12:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f8622ffd700 1 -- 172.21.15.119:0/3762560942 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f860c003570 con 0x7f86241563b0 2023-10-08T02:12:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f86237fe700 1 -- 172.21.15.119:0/3762560942 >> v2:172.21.15.184:3300/0 conn(0x7f8624068890 msgr2=0x7f86240c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:12:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f86237fe700 1 --2- 172.21.15.119:0/3762560942 >> v2:172.21.15.184:3300/0 conn(0x7f8624068890 0x7f86240c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f86237fe700 1 -- 172.21.15.119:0/3762560942 >> v1:172.21.15.119:6789/0 conn(0x7f86241563b0 legacy=0x7f8624156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:46.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.551+0000 7f86237fe700 1 -- 172.21.15.119:0/3762560942 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862415b250 con 0x7f862415a440 2023-10-08T02:12:46.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.552+0000 7f86237fe700 1 --2- 172.21.15.119:0/3762560942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240acf30 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f86240bff20 tx=0x7f86180091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e090d7a709d690 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:46.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.552+0000 7f8622ffd700 1 -- 172.21.15.119:0/3762560942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618011630 con 0x7f862415a440 2023-10-08T02:12:46.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.552+0000 7f8622ffd700 1 -- 172.21.15.119:0/3762560942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86180117b0 con 0x7f862415a440 2023-10-08T02:12:46.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.552+0000 7f8622ffd700 1 -- 172.21.15.119:0/3762560942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618011b70 con 0x7f862415a440 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.552+0000 7f862a7ff700 1 -- 172.21.15.119:0/3762560942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 msgr2=0x7f86240acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.552+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3762560942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240acf30 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f86240bff20 tx=0x7f86180091a0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.553+0000 7f862a7ff700 1 -- 172.21.15.119:0/3762560942 shutdown_connections 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.553+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3762560942 >> v2:172.21.15.184:3300/0 conn(0x7f8624068890 0x7f86240c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.553+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3762560942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240acf30 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.553+0000 7f862a7ff700 1 -- 172.21.15.119:0/3762560942 >> 172.21.15.119:0/3762560942 conn(0x7f86240c8f90 msgr2=0x7f86240c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.553+0000 7f862a7ff700 1 -- 172.21.15.119:0/3762560942 shutdown_connections 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.553+0000 7f862a7ff700 1 -- 172.21.15.119:0/3762560942 wait complete. 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.554+0000 7f862a7ff700 1 Processor -- start 2023-10-08T02:12:46.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.554+0000 7f862a7ff700 1 -- start start 2023-10-08T02:12:46.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.555+0000 7f862a7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8624068890 0x7f862416bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.555+0000 7f862a7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86241563b0 0x7f8624154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.555+0000 7f862a7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.555+0000 7f862a7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862416c1d0 con 0x7f862415a440 2023-10-08T02:12:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.555+0000 7f862a7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86240c7be0 con 0x7f86241563b0 2023-10-08T02:12:46.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.555+0000 7f862a7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86240c7060 con 0x7f8624068890 2023-10-08T02:12:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.555+0000 7f8628d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8628d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240c20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39636/0 (socket says 172.21.15.119:39636) 2023-10-08T02:12:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8628d9c700 1 -- 172.21.15.119:0/3273361854 learned_addr learned my addr 172.21.15.119:0/3273361854 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:12:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f86237fe700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86241563b0 0x7f8624154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:46.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8623fff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8624068890 0x7f862416bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:46.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8628d9c700 1 -- 172.21.15.119:0/3273361854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8624068890 msgr2=0x7f862416bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:46.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8628d9c700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8624068890 0x7f862416bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8628d9c700 1 -- 172.21.15.119:0/3273361854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86241563b0 msgr2=0x7f8624154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:46.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8628d9c700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86241563b0 0x7f8624154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8628d9c700 1 -- 172.21.15.119:0/3273361854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8618009420 con 0x7f862415a440 2023-10-08T02:12:46.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f8628d9c700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240c20e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f861c009e90 tx=0x7f861c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:46.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f860bfff700 1 -- 172.21.15.119:0/3273361854 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c00cb50 con 0x7f862415a440 2023-10-08T02:12:46.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f860bfff700 1 -- 172.21.15.119:0/3273361854 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c00f040 con 0x7f862415a440 2023-10-08T02:12:46.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86240c7340 con 0x7f862415a440 2023-10-08T02:12:46.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.556+0000 7f860bfff700 1 -- 172.21.15.119:0/3273361854 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c014480 con 0x7f862415a440 2023-10-08T02:12:46.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.557+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86240c7900 con 0x7f862415a440 2023-10-08T02:12:46.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.557+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86240b02f0 con 0x7f862415a440 2023-10-08T02:12:46.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.558+0000 7f860bfff700 1 -- 172.21.15.119:0/3273361854 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 90) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f861c0147f0 con 0x7f862415a440 2023-10-08T02:12:46.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.560+0000 7f860bfff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8610066dd0 0x7f8610069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:12:46.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.560+0000 7f860bfff700 1 -- 172.21.15.119:0/3273361854 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6427+0+0 (secure 0 0 0) 0x7f861c0852a0 con 0x7f862415a440 2023-10-08T02:12:46.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.560+0000 7f8623fff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8610066dd0 0x7f8610069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:12:46.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.561+0000 7f8623fff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8610066dd0 0x7f8610069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f86140058d0 tx=0x7f8614009040 comp rx=0 tx=0).ready entity=mgr.9351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:12:46.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.564+0000 7f860bfff700 1 -- 172.21.15.119:0/3273361854 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f861c052420 con 0x7f862415a440 2023-10-08T02:12:46.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.783+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f86240718b0 con 0x7f8610066dd0 2023-10-08T02:12:46.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.833+0000 7f860bfff700 1 -- 172.21.15.119:0/3273361854 <== mgr.9351 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f86240718b0 con 0x7f8610066dd0 2023-10-08T02:12:46.834 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-10-08T02:12:46.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8610066dd0 msgr2=0x7f8610069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:46.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8610066dd0 0x7f8610069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f86140058d0 tx=0x7f8614009040 comp rx=0 tx=0).stop 2023-10-08T02:12:46.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 msgr2=0x7f86240c20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:12:46.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240c20e0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f861c009e90 tx=0x7f861c00b040 comp rx=0 tx=0).stop 2023-10-08T02:12:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 shutdown_connections 2023-10-08T02:12:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8610066dd0 0x7f8610069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8624068890 0x7f862416bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f862415a440 0x7f86240c20e0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 --2- 172.21.15.119:0/3273361854 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86241563b0 0x7f8624154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:12:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 >> 172.21.15.119:0/3273361854 conn(0x7f86240c8f90 msgr2=0x7f862406dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:12:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 shutdown_connections 2023-10-08T02:12:46.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:12:46.835+0000 7f862a7ff700 1 -- 172.21.15.119:0/3273361854 wait complete. 2023-10-08T02:12:46.854 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-10-08T02:13:00.847 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:13:00.846+0000 7f16adb95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:13:00.847 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:13:00.846+0000 7f16adb95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:13:00.847 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:13:00.846+0000 7f16adb95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:13:00.847 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:13:00.846+0000 7f16adb95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:13:00.847 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:13:00.846+0000 7f16adb95700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:13:07.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:13:07.237 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:13:37.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:13:37.433 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:14:07.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:14:07.619 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:14:37.786 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:14:37.791 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:15:07.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:15:07.975 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:15:38.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:15:38.151 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:16:08.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:16:08.256 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:16:38.454 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:16:38.459 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:17:08.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:17:08.570 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:17:38.762 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:17:38.768 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:18:08.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:18:08.955 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:18:39.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:18:39.130 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:19:09.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:19:09.294 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:19:39.451 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:19:39.456 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:20:09.630 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:20:09.636 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:20:39.823 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:20:39.829 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:21:10.023 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:21:10.028 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:21:40.209 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:21:40.215 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:22:10.375 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:22:10.380 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:22:40.546 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:22:40.552 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:23:10.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:23:10.741 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:23:15.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:23:15.855 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_13_00_UTC 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_14_00_UTC 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_15_00_UTC 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_16_00_UTC 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_17_00_UTC 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_18_00_UTC 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_19_00_UTC 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_20_00_UTC 2023-10-08T02:23:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_21_00_UTC 2023-10-08T02:23:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_22_00_UTC 2023-10-08T02:23:15.900 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_23_00_UTC 2023-10-08T02:23:15.900 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-02_13_00_UTC', 'scheduled-2023-10-08-02_14_00_UTC', 'scheduled-2023-10-08-02_15_00_UTC', 'scheduled-2023-10-08-02_16_00_UTC', 'scheduled-2023-10-08-02_17_00_UTC', 'scheduled-2023-10-08-02_18_00_UTC', 'scheduled-2023-10-08-02_19_00_UTC', 'scheduled-2023-10-08-02_20_00_UTC', 'scheduled-2023-10-08-02_21_00_UTC', 'scheduled-2023-10-08-02_22_00_UTC', 'scheduled-2023-10-08-02_23_00_UTC'] 2023-10-08T02:23:15.901 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-10-08T02:23:16.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.130+0000 7f549639d700 1 Processor -- start 2023-10-08T02:23:16.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.130+0000 7f549639d700 1 -- start start 2023-10-08T02:23:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.131+0000 7f549639d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54900cb560 0x7f54900cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.131+0000 7f549639d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54900cc970 0x7f54900697b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.131+0000 7f549639d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54900ccde0 con 0x7f5490069cf0 2023-10-08T02:23:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.131+0000 7f549639d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549006aa30 con 0x7f54900cb560 2023-10-08T02:23:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.131+0000 7f549639d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5490159db0 con 0x7f54900cc970 2023-10-08T02:23:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f549539b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f54900cb560 0x7f54900cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f5495b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5490069cf0 0x7f549006d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37282/0 (socket says 172.21.15.119:37282) 2023-10-08T02:23:16.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f5495b9c700 1 -- 172.21.15.119:0/806431762 learned_addr learned my addr 172.21.15.119:0/806431762 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:23:16.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f5494b9a700 1 --2- 172.21.15.119:0/806431762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54900cc970 0x7f54900697b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f549539b700 1 -- 172.21.15.119:0/806431762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54900cc970 msgr2=0x7f54900697b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:23:16.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f549539b700 1 --2- 172.21.15.119:0/806431762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54900cc970 0x7f54900697b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f549539b700 1 -- 172.21.15.119:0/806431762 >> v1:172.21.15.119:6789/0 conn(0x7f5490069cf0 legacy=0x7f549006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f549539b700 1 -- 172.21.15.119:0/806431762 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549006a470 con 0x7f54900cb560 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f5487fff700 1 -- 172.21.15.119:0/806431762 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396718840 0 0) 0x7f54900ccde0 con 0x7f5490069cf0 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f5494b9a700 1 --2- 172.21.15.119:0/806431762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54900cc970 0x7f54900697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.132+0000 7f549539b700 1 --2- 172.21.15.119:0/806431762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900cc430 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f548c0098e0 tx=0x7f548c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1abcfdd874712e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f5487fff700 1 -- 172.21.15.119:0/806431762 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c009e80 con 0x7f54900cb560 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f5487fff700 1 -- 172.21.15.119:0/806431762 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c00b860 con 0x7f54900cb560 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f5487fff700 1 -- 172.21.15.119:0/806431762 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c00bbe0 con 0x7f54900cb560 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f549639d700 1 -- 172.21.15.119:0/806431762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 msgr2=0x7f54900cc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f549639d700 1 --2- 172.21.15.119:0/806431762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900cc430 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f548c0098e0 tx=0x7f548c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f549639d700 1 -- 172.21.15.119:0/806431762 shutdown_connections 2023-10-08T02:23:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f549639d700 1 --2- 172.21.15.119:0/806431762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900cc430 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f549639d700 1 --2- 172.21.15.119:0/806431762 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f54900cc970 0x7f54900697b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f549639d700 1 -- 172.21.15.119:0/806431762 >> 172.21.15.119:0/806431762 conn(0x7f549006baf0 msgr2=0x7f549006bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:23:16.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f549639d700 1 -- 172.21.15.119:0/806431762 shutdown_connections 2023-10-08T02:23:16.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.133+0000 7f549639d700 1 -- 172.21.15.119:0/806431762 wait complete. 2023-10-08T02:23:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.135+0000 7f549639d700 1 Processor -- start 2023-10-08T02:23:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.135+0000 7f549639d700 1 -- start start 2023-10-08T02:23:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f549639d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490069cf0 0x7f54900a98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f549639d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f549639d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54900cc970 0x7f5490157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f549639d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54900aad10 con 0x7f54900cb560 2023-10-08T02:23:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f549639d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54900aa100 con 0x7f54900cc970 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f549639d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54900aa3e0 con 0x7f5490069cf0 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f5494b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f5494b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900ab460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43540/0 (socket says 172.21.15.119:43540) 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f5494b9a700 1 -- 172.21.15.119:0/726561923 learned_addr learned my addr 172.21.15.119:0/726561923 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f549539b700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490069cf0 0x7f54900a98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.136+0000 7f5495b9c700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54900cc970 0x7f5490157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f5494b9a700 1 -- 172.21.15.119:0/726561923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490069cf0 msgr2=0x7f54900a98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f5494b9a700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490069cf0 0x7f54900a98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f5494b9a700 1 -- 172.21.15.119:0/726561923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54900cc970 msgr2=0x7f5490157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f5494b9a700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54900cc970 0x7f5490157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f5494b9a700 1 -- 172.21.15.119:0/726561923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c009420 con 0x7f54900cb560 2023-10-08T02:23:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f5494b9a700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900ab460 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f548000d860 tx=0x7f5480008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:23:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f54857fa700 1 -- 172.21.15.119:0/726561923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54800125f0 con 0x7f54900cb560 2023-10-08T02:23:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f54857fa700 1 -- 172.21.15.119:0/726561923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5480012750 con 0x7f54900cb560 2023-10-08T02:23:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f54857fa700 1 -- 172.21.15.119:0/726561923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5480012af0 con 0x7f54900cb560 2023-10-08T02:23:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54900aa6c0 con 0x7f54900cb560 2023-10-08T02:23:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.137+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54900a9e10 con 0x7f54900cb560 2023-10-08T02:23:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.138+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54900aaec0 con 0x7f54900cb560 2023-10-08T02:23:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.139+0000 7f54857fa700 1 -- 172.21.15.119:0/726561923 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 93) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f5480013480 con 0x7f54900cb560 2023-10-08T02:23:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.140+0000 7f54857fa700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f54740670a0 0x7f5474069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.140+0000 7f54857fa700 1 -- 172.21.15.119:0/726561923 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f5480084ba0 con 0x7f54900cb560 2023-10-08T02:23:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.140+0000 7f549539b700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f54740670a0 0x7f5474069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.141+0000 7f549539b700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f54740670a0 0x7f5474069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f548c0091e0 tx=0x7f548c017000 comp rx=0 tx=0).ready entity=mgr.9351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:23:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.144+0000 7f54857fa700 1 -- 172.21.15.119:0/726561923 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f54900aaec0 con 0x7f54900cb560 2023-10-08T02:23:16.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.359+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 --> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f549004c450 con 0x7f54740670a0 2023-10-08T02:23:16.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.367+0000 7f54857fa700 1 -- 172.21.15.119:0/726561923 <== mgr.9351 v2:172.21.15.119:6836/85511 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f548c00b9d0 con 0x7f54740670a0 2023-10-08T02:23:16.367 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:23:16.367 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T02:12:46", "first": "2023-10-08T02:13:00", "last": "2023-10-08T02:23:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-08T02:23:16.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f54740670a0 msgr2=0x7f5474069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f54740670a0 0x7f5474069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f548c0091e0 tx=0x7f548c017000 comp rx=0 tx=0).stop 2023-10-08T02:23:16.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 msgr2=0x7f54900ab460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900ab460 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f548000d860 tx=0x7f5480008040 comp rx=0 tx=0).stop 2023-10-08T02:23:16.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 shutdown_connections 2023-10-08T02:23:16.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f54740670a0 0x7f5474069550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5490069cf0 0x7f54900a98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f54900cc970 0x7f5490157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 --2- 172.21.15.119:0/726561923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f54900cb560 0x7f54900ab460 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 >> 172.21.15.119:0/726561923 conn(0x7f549006baf0 msgr2=0x7f54900c4770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:23:16.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 shutdown_connections 2023-10-08T02:23:16.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.370+0000 7f549639d700 1 -- 172.21.15.119:0/726561923 wait complete. 2023-10-08T02:23:16.388 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T02:12:46", "first": "2023-10-08T02:13:00", "last": "2023-10-08T02:23:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-10-08T02:23:16.388 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T02:12:46', 'first': '2023-10-08T02:13:00', 'last': '2023-10-08T02:23:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-10-08T02:23:16.388 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-10-08T02:23:16.388 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-10-08T02:23:16.388 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-08T02:23:16.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.570+0000 7f80341ff700 1 Processor -- start 2023-10-08T02:23:16.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.571+0000 7f80341ff700 1 -- start start 2023-10-08T02:23:16.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f80341ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c155c10 0x7f802c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f80341ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f802c0ac6a0 0x7f802c0b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f80341ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c0acac0 con 0x7f802c0ac230 2023-10-08T02:23:16.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f80341ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c0ad930 con 0x7f802c0ac6a0 2023-10-08T02:23:16.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f80341ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0acd20 con 0x7f802c155c10 2023-10-08T02:23:16.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f803179a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f802c0ac230 0x7f802c1568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37296/0 (socket says 172.21.15.119:37296) 2023-10-08T02:23:16.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f803179a700 1 -- 172.21.15.119:0/995129305 learned_addr learned my addr 172.21.15.119:0/995129305 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f8031f9b700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c155c10 0x7f802c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f803279c700 1 --2- 172.21.15.119:0/995129305 >> v2:172.21.15.184:3300/0 conn(0x7f802c0ac6a0 0x7f802c0b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.572+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4188093661 0 0) 0x7f802c0acac0 con 0x7f802c0ac230 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8018005780 con 0x7f802c0ac230 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 847482988 0 0) 0x7f8018005780 con 0x7f802c0ac230 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c155c10 msgr2=0x7f802c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c155c10 0x7f802c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 >> v2:172.21.15.184:3300/0 conn(0x7f802c0ac6a0 msgr2=0x7f802c0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:23:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 --2- 172.21.15.119:0/995129305 >> v2:172.21.15.184:3300/0 conn(0x7f802c0ac6a0 0x7f802c0b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0ad000 con 0x7f802c0ac230 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f801c001470 con 0x7f802c0ac230 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 >> v1:172.21.15.119:6789/0 conn(0x7f802c0ac230 legacy=0x7f802c1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8018005c70 0x7f8018006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8018006a20 0x7f8018006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c155c10 0x7f80180073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0ad930 con 0x7f8018005c70 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0acd20 con 0x7f8018006a20 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801800dc70 con 0x7f802c155c10 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f8031f9b700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8018005c70 0x7f8018006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f803179a700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8018006a20 0x7f8018006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.573+0000 7f803279c700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c155c10 0x7f80180073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8031f9b700 1 -- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c155c10 msgr2=0x7f80180073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8031f9b700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f802c155c10 0x7f80180073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8031f9b700 1 -- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8018006a20 msgr2=0x7f8018006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8031f9b700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8018006a20 0x7f8018006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8031f9b700 1 -- 172.21.15.119:0/995129305 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8018005780 con 0x7f8018005c70 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8031f9b700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8018005c70 0x7f8018006080 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8028009c30 tx=0x7f80280091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bb8174bb78ca6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802800e710 con 0x7f8018005c70 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802800e870 con 0x7f8018005c70 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.574+0000 7f8030f99700 1 -- 172.21.15.119:0/995129305 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802800ec30 con 0x7f8018005c70 2023-10-08T02:23:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.575+0000 7f80341ff700 1 -- 172.21.15.119:0/995129305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8018005c70 msgr2=0x7f8018006080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.575+0000 7f80341ff700 1 --2- 172.21.15.119:0/995129305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8018005c70 0x7f8018006080 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8028009c30 tx=0x7f80280091a0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.575+0000 7f803279c700 1 -- 172.21.15.119:0/995129305 reap_dead start 2023-10-08T02:23:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.575+0000 7f80341ff700 1 -- 172.21.15.119:0/995129305 shutdown_connections 2023-10-08T02:23:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.575+0000 7f80341ff700 1 -- 172.21.15.119:0/995129305 >> 172.21.15.119:0/995129305 conn(0x7f802c06a320 msgr2=0x7f802c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:23:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.575+0000 7f80341ff700 1 -- 172.21.15.119:0/995129305 shutdown_connections 2023-10-08T02:23:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.575+0000 7f80341ff700 1 -- 172.21.15.119:0/995129305 wait complete. 2023-10-08T02:23:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.576+0000 7f80341ff700 1 Processor -- start 2023-10-08T02:23:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.576+0000 7f80341ff700 1 -- start start 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f80341ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8018005c70 0x7f802c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f80341ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0a6490 0x7f802c0a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f80341ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c156fa0 0x7f802c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f80341ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c157950 con 0x7f802c156fa0 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f80341ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0a7c10 con 0x7f802c0a6490 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f80341ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c16a3c0 con 0x7f8018005c70 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f803279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c156fa0 0x7f802c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f803279c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c156fa0 0x7f802c157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43560/0 (socket says 172.21.15.119:43560) 2023-10-08T02:23:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.577+0000 7f803279c700 1 -- 172.21.15.119:0/2649996187 learned_addr learned my addr 172.21.15.119:0/2649996187 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f803179a700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0a6490 0x7f802c0a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f8031f9b700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8018005c70 0x7f802c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f803279c700 1 -- 172.21.15.119:0/2649996187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8018005c70 msgr2=0x7f802c152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f803279c700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8018005c70 0x7f802c152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f803279c700 1 -- 172.21.15.119:0/2649996187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0a6490 msgr2=0x7f802c0a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f803279c700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0a6490 0x7f802c0a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f803279c700 1 -- 172.21.15.119:0/2649996187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0a7000 con 0x7f802c156fa0 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f803279c700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c156fa0 0x7f802c157410 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f802c063d40 tx=0x7f802400c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f8021ffb700 1 -- 172.21.15.119:0/2649996187 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802400f040 con 0x7f802c156fa0 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f8021ffb700 1 -- 172.21.15.119:0/2649996187 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802400a360 con 0x7f802c156fa0 2023-10-08T02:23:16.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f8021ffb700 1 -- 172.21.15.119:0/2649996187 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8024015400 con 0x7f802c156fa0 2023-10-08T02:23:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.578+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c0a72e0 con 0x7f802c156fa0 2023-10-08T02:23:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.579+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c0a78a0 con 0x7f802c156fa0 2023-10-08T02:23:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.579+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c0a7d80 con 0x7f802c156fa0 2023-10-08T02:23:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.580+0000 7f8021ffb700 1 -- 172.21.15.119:0/2649996187 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 93) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f8024003040 con 0x7f802c156fa0 2023-10-08T02:23:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.581+0000 7f8021ffb700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8010067120 0x7f80100695d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:23:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.581+0000 7f8021ffb700 1 -- 172.21.15.119:0/2649996187 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f8024084a40 con 0x7f802c156fa0 2023-10-08T02:23:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.581+0000 7f8031f9b700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8010067120 0x7f80100695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:23:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.582+0000 7f8031f9b700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8010067120 0x7f80100695d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80280091e0 tx=0x7f802801a000 comp rx=0 tx=0).ready entity=mgr.9351 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:23:16.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.585+0000 7f8021ffb700 1 -- 172.21.15.119:0/2649996187 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8024051700 con 0x7f802c156fa0 2023-10-08T02:23:16.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.825+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f802c04c3c0 con 0x7f802c156fa0 2023-10-08T02:23:16.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.828+0000 7f8021ffb700 1 -- 172.21.15.119:0/2649996187 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v93) v1 ==== 74+0+163912 (secure 0 0 0) 0x7f802401d020 con 0x7f802c156fa0 2023-10-08T02:23:16.830 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:23:16.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8010067120 msgr2=0x7f80100695d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8010067120 0x7f80100695d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80280091e0 tx=0x7f802801a000 comp rx=0 tx=0).stop 2023-10-08T02:23:16.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c156fa0 msgr2=0x7f802c157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:23:16.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c156fa0 0x7f802c157410 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f802c063d40 tx=0x7f802400c040 comp rx=0 tx=0).stop 2023-10-08T02:23:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 shutdown_connections 2023-10-08T02:23:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:6836/85511,v1:172.21.15.119:6837/85511] conn(0x7f8010067120 0x7f80100695d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8018005c70 0x7f802c152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f802c156fa0 0x7f802c157410 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 --2- 172.21.15.119:0/2649996187 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f802c0a6490 0x7f802c0a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:23:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 >> 172.21.15.119:0/2649996187 conn(0x7f802c06a320 msgr2=0x7f802c0c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:23:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.831+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 shutdown_connections 2023-10-08T02:23:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:23:16.832+0000 7f80341ff700 1 -- 172.21.15.119:0/2649996187 wait complete. 2023-10-08T02:23:16.849 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":93,"active_gid":9351,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":85511},{"type":"v1","addr":"172.21.15.119:6837","nonce":85511}]},"active_addr":"172.21.15.119:6837/85511","active_change":"2023-10-08T02:12:45.254893+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":8599,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":172,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":655693879}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3986903627}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1092901654}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":1047785372}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":506139810}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":932366218}]}]} 2023-10-08T02:23:16.888 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: y 2023-10-08T02:23:16.888 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-08T02:23:16.888 INFO:tasks.ceph.mgr.y:Stopping old one... 2023-10-08T02:23:16.889 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-08T02:23:16.889 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:23:16.956 INFO:tasks.ceph.mgr.y:Stopped 2023-10-08T02:23:16.956 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-08T02:23:16.962 INFO:tasks.ceph.mgr.y:Started 2023-10-08T02:23:18.274 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:18.274+0000 7fd7fab50180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:23:18.437 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:18.437+0000 7fd7fab50180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:23:18.521 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:18.522+0000 7fd7fab50180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:23:18.744 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:18.744+0000 7fd7fab50180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:23:18.919 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:18.920+0000 7fd7fab50180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:23:19.011 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:19.011+0000 7fd7fab50180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:23:19.171 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:19.171+0000 7fd7fab50180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:23:19.273 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:19.273+0000 7fd7fab50180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:23:19.755 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:19.756+0000 7fd7fab50180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:23:19.849 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:19.849+0000 7fd7fab50180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:23:19.937 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:19.938+0000 7fd7fab50180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:23:20.551 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:20.552+0000 7fd7fab50180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:23:20.723 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:20.724+0000 7fd7fab50180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:23:20.817 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:20.818+0000 7fd7fab50180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:23:21.107 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:21.108+0000 7fd7fab50180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:23:21.204 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:21.205+0000 7fd7fab50180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:23:21.898 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:21.899+0000 7fd7fab50180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:23:22.181 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:22.181+0000 7fd7fab50180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:23:22.272 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:22.273+0000 7fd7fab50180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:23:22.423 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:22.423+0000 7fd7fab50180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:23:22.528 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:22.528+0000 7fd7fab50180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:23:22.715 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:22.715+0000 7fd7fab50180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:23:22.807 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:22.808+0000 7fd7fab50180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:23:22.898 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:22.899+0000 7fd7fab50180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:23:22.992 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:22.992+0000 7fd7fab50180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:23:23.153 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.153+0000 7fd76f676700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.153+0000 7fd76f676700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.153+0000 7fd76f676700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.153+0000 7fd76f676700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.153+0000 7fd76f676700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.154+0000 7fd76be6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.154+0000 7fd76be6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.154+0000 7fd76be6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.154+0000 7fd76be6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:23.154 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:23:23.154+0000 7fd76be6f700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:23:40.924 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:23:40.929 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:24:00.150 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:24:00.150+0000 7fd777686700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:24:00.150 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:24:00.150+0000 7fd777686700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:24:00.150 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:24:00.150+0000 7fd777686700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:24:00.150 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:24:00.150+0000 7fd777686700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:24:00.150 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:24:00.150+0000 7fd777686700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:24:11.112 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:24:11.119 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:24:41.275 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:24:41.280 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:25:11.470 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:25:11.476 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:25:41.608 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:25:41.613 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:26:11.771 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:26:11.777 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:26:41.968 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:26:41.974 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:27:12.314 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:27:12.320 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:27:42.494 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:27:42.499 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:28:12.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:28:12.679 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:28:16.963 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-08T02:28:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.158+0000 7ff1085ff700 1 Processor -- start 2023-10-08T02:28:17.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.158+0000 7ff1085ff700 1 -- start start 2023-10-08T02:28:17.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.159+0000 7ff1085ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff100156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.159+0000 7ff1085ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff10015a440 0x7ff1000acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.159+0000 7ff1085ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff100068890 0x7ff1000c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.159+0000 7ff1085ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10015a9b0 con 0x7ff100068890 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.159+0000 7ff1085ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10015ac90 con 0x7ff10015a440 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.159+0000 7ff1085ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10015af70 con 0x7ff1001563b0 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff106b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff100068890 0x7ff1000c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff105b9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff10015a440 0x7ff1000acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff10639b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff100156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff10639b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff100156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:53830/0 (socket says 172.21.15.119:53830) 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff105b9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff10015a440 0x7ff1000acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:57774/0 (socket says 172.21.15.119:57774) 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff10639b700 1 -- 172.21.15.119:0/1165589355 learned_addr learned my addr 172.21.15.119:0/1165589355 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff106b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff100068890 0x7ff1000c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50776/0 (socket says 172.21.15.119:50776) 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff106b9c700 1 -- 172.21.15.119:0/1165589355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 msgr2=0x7ff100156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff106b9c700 1 --2- 172.21.15.119:0/1165589355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff100156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff106b9c700 1 -- 172.21.15.119:0/1165589355 >> v2:172.21.15.184:3300/0 conn(0x7ff10015a440 msgr2=0x7ff1000acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff106b9c700 1 --2- 172.21.15.119:0/1165589355 >> v2:172.21.15.184:3300/0 conn(0x7ff10015a440 0x7ff1000acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff106b9c700 1 -- 172.21.15.119:0/1165589355 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff10015b250 con 0x7ff100068890 2023-10-08T02:28:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff10639b700 1 --2- 172.21.15.119:0/1165589355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff100156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff105b9a700 1 --2- 172.21.15.119:0/1165589355 >> v2:172.21.15.184:3300/0 conn(0x7ff10015a440 0x7ff1000acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.160+0000 7ff106b9c700 1 --2- 172.21.15.119:0/1165589355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100068890 0x7ff1000c8230 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff10006aa20 tx=0x7ff0f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d45340bbfa443ae server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff105399700 1 -- 172.21.15.119:0/1165589355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800e070 con 0x7ff100068890 2023-10-08T02:28:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff105399700 1 -- 172.21.15.119:0/1165589355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f800bab0 con 0x7ff100068890 2023-10-08T02:28:17.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff105399700 1 -- 172.21.15.119:0/1165589355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f800be70 con 0x7ff100068890 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 -- 172.21.15.119:0/1165589355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100068890 msgr2=0x7ff1000c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1165589355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100068890 0x7ff1000c8230 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff10006aa20 tx=0x7ff0f8009780 comp rx=0 tx=0).stop 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 -- 172.21.15.119:0/1165589355 shutdown_connections 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1165589355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff100068890 0x7ff1000c8230 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1165589355 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff100156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1165589355 >> v2:172.21.15.184:3300/0 conn(0x7ff10015a440 0x7ff1000acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 -- 172.21.15.119:0/1165589355 >> 172.21.15.119:0/1165589355 conn(0x7ff1000c8f90 msgr2=0x7ff1000c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 -- 172.21.15.119:0/1165589355 shutdown_connections 2023-10-08T02:28:17.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.161+0000 7ff1085ff700 1 -- 172.21.15.119:0/1165589355 wait complete. 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.163+0000 7ff1085ff700 1 Processor -- start 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.163+0000 7ff1085ff700 1 -- start start 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff1085ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100068890 0x7ff1001507d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff1085ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff1001584b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff1085ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10015a440 0x7ff1000ad370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff1085ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100159b60 con 0x7ff10015a440 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff1085ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff100158f50 con 0x7ff100068890 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff1085ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff100159230 con 0x7ff1001563b0 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff106b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10015a440 0x7ff1000ad370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff106b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10015a440 0x7ff1000ad370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50780/0 (socket says 172.21.15.119:50780) 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff106b9c700 1 -- 172.21.15.119:0/1063059824 learned_addr learned my addr 172.21.15.119:0/1063059824 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:17.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff10639b700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100068890 0x7ff1001507d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.164+0000 7ff105b9a700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff1001584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff106b9c700 1 -- 172.21.15.119:0/1063059824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 msgr2=0x7ff1001584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff106b9c700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff1001584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff106b9c700 1 -- 172.21.15.119:0/1063059824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100068890 msgr2=0x7ff1001507d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff106b9c700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100068890 0x7ff1001507d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff106b9c700 1 -- 172.21.15.119:0/1063059824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0fc0094f0 con 0x7ff10015a440 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff106b9c700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10015a440 0x7ff1000ad370 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009fd0 tx=0x7ff0f8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff0f67fc700 1 -- 172.21.15.119:0/1063059824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8012dd0 con 0x7ff10015a440 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff0f67fc700 1 -- 172.21.15.119:0/1063059824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f8027470 con 0x7ff10015a440 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f0009420 con 0x7ff10015a440 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff0f67fc700 1 -- 172.21.15.119:0/1063059824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8027830 con 0x7ff10015a440 2023-10-08T02:28:17.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.165+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f8009a20 con 0x7ff10015a440 2023-10-08T02:28:17.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.166+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1000718b0 con 0x7ff10015a440 2023-10-08T02:28:17.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.167+0000 7ff0f67fc700 1 -- 172.21.15.119:0/1063059824 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 98) v1 ==== 84871+0+0 (secure 0 0 0) 0x7ff0f8024070 con 0x7ff10015a440 2023-10-08T02:28:17.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.168+0000 7ff0f67fc700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff0e40670a0 0x7ff0e4069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.168+0000 7ff0f67fc700 1 -- 172.21.15.119:0/1063059824 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6714+0+0 (secure 0 0 0) 0x7ff0f80870d0 con 0x7ff10015a440 2023-10-08T02:28:17.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.168+0000 7ff10639b700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff0e40670a0 0x7ff0e4069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.169+0000 7ff10639b700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff0e40670a0 0x7ff0e4069550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc006d20 tx=0x7ff0fc009250 comp rx=0 tx=0).ready entity=mgr.9450 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:17.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.173+0000 7ff0f67fc700 1 -- 172.21.15.119:0/1063059824 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff0f8053e40 con 0x7ff10015a440 2023-10-08T02:28:17.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.388+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff10006d6a0 con 0x7ff0e40670a0 2023-10-08T02:28:17.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.448+0000 7ff0f67fc700 1 -- 172.21.15.119:0/1063059824 <== mgr.9450 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7ff10006d6a0 con 0x7ff0e40670a0 2023-10-08T02:28:17.448 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-08T02:28:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff0e40670a0 msgr2=0x7ff0e4069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff0e40670a0 0x7ff0e4069550 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc006d20 tx=0x7ff0fc009250 comp rx=0 tx=0).stop 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10015a440 msgr2=0x7ff1000ad370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10015a440 0x7ff1000ad370 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8009fd0 tx=0x7ff0f8019000 comp rx=0 tx=0).stop 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 shutdown_connections 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff0e40670a0 0x7ff0e4069550 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff10015a440 0x7ff1000ad370 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff100068890 0x7ff1001507d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 --2- 172.21.15.119:0/1063059824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1001563b0 0x7ff1001584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 >> 172.21.15.119:0/1063059824 conn(0x7ff1000c8f90 msgr2=0x7ff1000af340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:17.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.450+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 shutdown_connections 2023-10-08T02:28:17.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.451+0000 7ff1085ff700 1 -- 172.21.15.119:0/1063059824 wait complete. 2023-10-08T02:28:17.467 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-10-08T02:28:17.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:17.468 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-08T02:28:17.505 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_13_00_UTC 2023-10-08T02:28:17.505 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_14_00_UTC 2023-10-08T02:28:17.505 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_15_00_UTC 2023-10-08T02:28:17.505 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_16_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_17_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_18_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_19_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_20_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_21_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_22_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_23_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_24_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_25_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_26_00_UTC 2023-10-08T02:28:17.506 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_27_00_UTC 2023-10-08T02:28:17.507 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_28_00_UTC 2023-10-08T02:28:17.507 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-10-08-02_13_00_UTC', 'scheduled-2023-10-08-02_14_00_UTC', 'scheduled-2023-10-08-02_15_00_UTC', 'scheduled-2023-10-08-02_16_00_UTC', 'scheduled-2023-10-08-02_17_00_UTC', 'scheduled-2023-10-08-02_18_00_UTC', 'scheduled-2023-10-08-02_19_00_UTC', 'scheduled-2023-10-08-02_20_00_UTC', 'scheduled-2023-10-08-02_21_00_UTC', 'scheduled-2023-10-08-02_22_00_UTC', 'scheduled-2023-10-08-02_23_00_UTC', 'scheduled-2023-10-08-02_24_00_UTC', 'scheduled-2023-10-08-02_25_00_UTC', 'scheduled-2023-10-08-02_26_00_UTC', 'scheduled-2023-10-08-02_27_00_UTC', 'scheduled-2023-10-08-02_28_00_UTC'] 2023-10-08T02:28:17.508 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-10-08T02:28:17.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.733+0000 7fc283fff700 1 Processor -- start 2023-10-08T02:28:17.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.733+0000 7fc283fff700 1 -- start start 2023-10-08T02:28:17.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.734+0000 7fc283fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc284156350 0x7fc284157370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.734+0000 7fc283fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2840b0570 con 0x7fc2840b0c00 2023-10-08T02:28:17.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.734+0000 7fc283fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28415ad40 con 0x7fc2840b0760 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.734+0000 7fc283fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2841578b0 con 0x7fc284156350 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.734+0000 7fc282ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc284156350 0x7fc284157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.734+0000 7fc2837fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc2840b0c00 0x7fc28415a630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42172/0 (socket says 172.21.15.119:42172) 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.734+0000 7fc2837fe700 1 -- 172.21.15.119:0/3013141879 learned_addr learned my addr 172.21.15.119:0/3013141879 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.735+0000 7fc282ffd700 1 -- 172.21.15.119:0/3013141879 >> v1:172.21.15.184:6789/0 conn(0x7fc2840b0760 legacy=0x7fc284159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.735+0000 7fc282ffd700 1 -- 172.21.15.119:0/3013141879 >> v1:172.21.15.119:6789/0 conn(0x7fc2840b0c00 legacy=0x7fc28415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.735+0000 7fc282ffd700 1 -- 172.21.15.119:0/3013141879 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2840af4d0 con 0x7fc284156350 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.735+0000 7fc281ffb700 1 -- 172.21.15.119:0/3013141879 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719748409 0 0) 0x7fc2840b0570 con 0x7fc2840b0c00 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.735+0000 7fc282ffd700 1 --2- 172.21.15.119:0/3013141879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc284156350 0x7fc284157370 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc274009e60 tx=0x7fc274009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bfa856c48cf347 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.735+0000 7fc281ffb700 1 -- 172.21.15.119:0/3013141879 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27400e070 con 0x7fc284156350 2023-10-08T02:28:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.735+0000 7fc281ffb700 1 -- 172.21.15.119:0/3013141879 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27400bbf0 con 0x7fc284156350 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.735+0000 7fc281ffb700 1 -- 172.21.15.119:0/3013141879 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274012430 con 0x7fc284156350 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.736+0000 7fc283fff700 1 -- 172.21.15.119:0/3013141879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc284156350 msgr2=0x7fc284157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.736+0000 7fc283fff700 1 --2- 172.21.15.119:0/3013141879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc284156350 0x7fc284157370 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fc274009e60 tx=0x7fc274009780 comp rx=0 tx=0).stop 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.736+0000 7fc283fff700 1 -- 172.21.15.119:0/3013141879 shutdown_connections 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.736+0000 7fc283fff700 1 --2- 172.21.15.119:0/3013141879 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc284156350 0x7fc284157370 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.736+0000 7fc283fff700 1 -- 172.21.15.119:0/3013141879 >> 172.21.15.119:0/3013141879 conn(0x7fc284068e20 msgr2=0x7fc2840cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.736+0000 7fc283fff700 1 -- 172.21.15.119:0/3013141879 shutdown_connections 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.736+0000 7fc283fff700 1 -- 172.21.15.119:0/3013141879 wait complete. 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.737+0000 7fc283fff700 1 Processor -- start 2023-10-08T02:28:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.738+0000 7fc283fff700 1 -- start start 2023-10-08T02:28:17.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.738+0000 7fc283fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2840b0760 0x7fc2840acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.738+0000 7fc283fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2840b0c00 0x7fc2840cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.738+0000 7fc283fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc284156350 0x7fc2840cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.738+0000 7fc283fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2840acae0 con 0x7fc2840b0c00 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.738+0000 7fc283fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2840abed0 con 0x7fc284156350 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.738+0000 7fc283fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2840ac1b0 con 0x7fc2840b0760 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2840b0c00 0x7fc2840cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2840b0c00 0x7fc2840cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50790/0 (socket says 172.21.15.119:50790) 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 -- 172.21.15.119:0/25567025 learned_addr learned my addr 172.21.15.119:0/25567025 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2837fe700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc284156350 0x7fc2840cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc282ffd700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2840b0760 0x7fc2840acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 -- 172.21.15.119:0/25567025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2840b0760 msgr2=0x7fc2840acf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2840b0760 0x7fc2840acf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 -- 172.21.15.119:0/25567025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc284156350 msgr2=0x7fc2840cbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc284156350 0x7fc2840cbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 -- 172.21.15.119:0/25567025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274009a20 con 0x7fc2840b0c00 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.739+0000 7fc2827fc700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2840b0c00 0x7fc2840cb5a0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fc27800b5e0 tx=0x7fc278009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.740+0000 7fc26affd700 1 -- 172.21.15.119:0/25567025 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27800f510 con 0x7fc2840b0c00 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.740+0000 7fc26affd700 1 -- 172.21.15.119:0/25567025 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27800f670 con 0x7fc2840b0c00 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.740+0000 7fc26affd700 1 -- 172.21.15.119:0/25567025 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27800fa30 con 0x7fc2840b0c00 2023-10-08T02:28:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.740+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2840ac490 con 0x7fc2840b0c00 2023-10-08T02:28:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.740+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc284148270 con 0x7fc2840b0c00 2023-10-08T02:28:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.741+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2840acc50 con 0x7fc2840b0c00 2023-10-08T02:28:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.742+0000 7fc26affd700 1 -- 172.21.15.119:0/25567025 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 98) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fc27801a400 con 0x7fc2840b0c00 2023-10-08T02:28:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.743+0000 7fc26affd700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc2700670a0 0x7fc270069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.743+0000 7fc26affd700 1 -- 172.21.15.119:0/25567025 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6714+0+0 (secure 0 0 0) 0x7fc278084690 con 0x7fc2840b0c00 2023-10-08T02:28:17.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.743+0000 7fc282ffd700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc2700670a0 0x7fc270069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:17.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.744+0000 7fc282ffd700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc2700670a0 0x7fc270069550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc2740099f0 tx=0x7fc274018000 comp rx=0 tx=0).ready entity=mgr.9450 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:17.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.747+0000 7fc26affd700 1 -- 172.21.15.119:0/25567025 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2780513e0 con 0x7fc2840b0c00 2023-10-08T02:28:17.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.959+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc28404c3c0 con 0x7fc2700670a0 2023-10-08T02:28:17.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.966+0000 7fc26affd700 1 -- 172.21.15.119:0/25567025 <== mgr.9450 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7fc28404c3c0 con 0x7fc2700670a0 2023-10-08T02:28:17.966 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:17.966 INFO:teuthology.orchestra.run.smithi119.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T02:12:46", "first": "2023-10-08T02:13:00", "last": "2023-10-08T02:28:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-08T02:28:17.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc2700670a0 msgr2=0x7fc270069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc2700670a0 0x7fc270069550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc2740099f0 tx=0x7fc274018000 comp rx=0 tx=0).stop 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2840b0c00 msgr2=0x7fc2840cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2840b0c00 0x7fc2840cb5a0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fc27800b5e0 tx=0x7fc278009120 comp rx=0 tx=0).stop 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 shutdown_connections 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc2700670a0 0x7fc270069550 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc2840b0760 0x7fc2840acf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc284156350 0x7fc2840cbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 --2- 172.21.15.119:0/25567025 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc2840b0c00 0x7fc2840cb5a0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 >> 172.21.15.119:0/25567025 conn(0x7fc284068e20 msgr2=0x7fc284154710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 shutdown_connections 2023-10-08T02:28:17.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:17.969+0000 7fc283fff700 1 -- 172.21.15.119:0/25567025 wait complete. 2023-10-08T02:28:17.985 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-10-08T00:00:00", "created": "2023-10-08T02:12:46", "first": "2023-10-08T02:13:00", "last": "2023-10-08T02:28:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-10-08T02:28:17.986 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-10-08T00:00:00', 'created': '2023-10-08T02:12:46', 'first': '2023-10-08T02:13:00', 'last': '2023-10-08T02:28:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-10-08T02:28:17.986 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-10-08T02:28:17.986 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-10-08T02:28:17.986 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-10-08T02:28:18.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.174+0000 7fd70c1ff700 1 Processor -- start 2023-10-08T02:28:18.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.174+0000 7fd70c1ff700 1 -- start start 2023-10-08T02:28:18.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70c1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7040b2100 0x7fd704068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:18.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70c1ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7040ae6d0 con 0x7fd7040688d0 2023-10-08T02:28:18.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70c1ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd704068d30 con 0x7fd704157740 2023-10-08T02:28:18.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70c1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7040acc00 con 0x7fd7040b2100 2023-10-08T02:28:18.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70a79c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd7040688d0 0x7fd70406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42188/0 (socket says 172.21.15.119:42188) 2023-10-08T02:28:18.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70a79c700 1 -- 172.21.15.119:0/2313230418 learned_addr learned my addr 172.21.15.119:0/2313230418 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:18.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70979a700 1 --2- 172.21.15.119:0/2313230418 >> v2:172.21.15.119:3300/0 conn(0x7fd7040b2100 0x7fd704068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:18.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd708f99700 1 -- 172.21.15.119:0/2313230418 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509104016 0 0) 0x7fd6fc001490 con 0x7fd7040688d0 2023-10-08T02:28:18.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd708f99700 1 -- 172.21.15.119:0/2313230418 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6f0003570 con 0x7fd7040688d0 2023-10-08T02:28:18.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70979a700 1 -- 172.21.15.119:0/2313230418 >> v1:172.21.15.184:6789/0 conn(0x7fd704157740 legacy=0x7fd7040adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:18.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70979a700 1 -- 172.21.15.119:0/2313230418 >> v1:172.21.15.119:6789/0 conn(0x7fd7040688d0 legacy=0x7fd70406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:18.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.176+0000 7fd70979a700 1 -- 172.21.15.119:0/2313230418 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd70406dc20 con 0x7fd7040b2100 2023-10-08T02:28:18.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.177+0000 7fd70979a700 1 --2- 172.21.15.119:0/2313230418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040b2100 0x7fd704068390 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40098e0 tx=0x7fd6f40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69690137b3fb74e9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:18.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.177+0000 7fd708f99700 1 -- 172.21.15.119:0/2313230418 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400b640 con 0x7fd7040b2100 2023-10-08T02:28:18.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.177+0000 7fd708f99700 1 -- 172.21.15.119:0/2313230418 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400b7a0 con 0x7fd7040b2100 2023-10-08T02:28:18.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.177+0000 7fd708f99700 1 -- 172.21.15.119:0/2313230418 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400bb40 con 0x7fd7040b2100 2023-10-08T02:28:18.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.177+0000 7fd70c1ff700 1 -- 172.21.15.119:0/2313230418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040b2100 msgr2=0x7fd704068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:18.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.177+0000 7fd70c1ff700 1 --2- 172.21.15.119:0/2313230418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040b2100 0x7fd704068390 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40098e0 tx=0x7fd6f40091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:18.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.178+0000 7fd70c1ff700 1 -- 172.21.15.119:0/2313230418 shutdown_connections 2023-10-08T02:28:18.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.178+0000 7fd70c1ff700 1 --2- 172.21.15.119:0/2313230418 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040b2100 0x7fd704068390 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:18.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.178+0000 7fd70c1ff700 1 -- 172.21.15.119:0/2313230418 >> 172.21.15.119:0/2313230418 conn(0x7fd7040c8f90 msgr2=0x7fd7040c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:18.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.178+0000 7fd70c1ff700 1 -- 172.21.15.119:0/2313230418 shutdown_connections 2023-10-08T02:28:18.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.178+0000 7fd70c1ff700 1 -- 172.21.15.119:0/2313230418 wait complete. 2023-10-08T02:28:18.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.179+0000 7fd70c1ff700 1 Processor -- start 2023-10-08T02:28:18.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.179+0000 7fd70c1ff700 1 -- start start 2023-10-08T02:28:18.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.180+0000 7fd70c1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040688d0 0x7fd70415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:18.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.180+0000 7fd70c1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b2100 0x7fd70414f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:18.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.180+0000 7fd70c1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704157740 0x7fd70415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:18.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.180+0000 7fd70c1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70415c5a0 con 0x7fd7040688d0 2023-10-08T02:28:18.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.180+0000 7fd70c1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd70415b510 con 0x7fd704157740 2023-10-08T02:28:18.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.180+0000 7fd70c1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7040c7820 con 0x7fd7040b2100 2023-10-08T02:28:18.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.180+0000 7fd709f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040688d0 0x7fd70415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:18.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.180+0000 7fd709f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040688d0 0x7fd70415caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50814/0 (socket says 172.21.15.119:50814) 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd709f9b700 1 -- 172.21.15.119:0/557054710 learned_addr learned my addr 172.21.15.119:0/557054710 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd70979a700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b2100 0x7fd70414f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd70a79c700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704157740 0x7fd70415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd709f9b700 1 -- 172.21.15.119:0/557054710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b2100 msgr2=0x7fd70414f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd709f9b700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b2100 0x7fd70414f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd709f9b700 1 -- 172.21.15.119:0/557054710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704157740 msgr2=0x7fd70415a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd709f9b700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704157740 0x7fd70415a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd709f9b700 1 -- 172.21.15.119:0/557054710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f4009420 con 0x7fd7040688d0 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd709f9b700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040688d0 0x7fd70415caa0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c5c0 tx=0x7fd70000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd6f9ffb700 1 -- 172.21.15.119:0/557054710 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700011590 con 0x7fd7040688d0 2023-10-08T02:28:18.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd6f9ffb700 1 -- 172.21.15.119:0/557054710 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700011710 con 0x7fd7040688d0 2023-10-08T02:28:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7040c7b00 con 0x7fd7040688d0 2023-10-08T02:28:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.181+0000 7fd6f9ffb700 1 -- 172.21.15.119:0/557054710 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700011ad0 con 0x7fd7040688d0 2023-10-08T02:28:18.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.182+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7040c80c0 con 0x7fd7040688d0 2023-10-08T02:28:18.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.183+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7040718b0 con 0x7fd7040688d0 2023-10-08T02:28:18.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.183+0000 7fd6f9ffb700 1 -- 172.21.15.119:0/557054710 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 98) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fd70001b3b0 con 0x7fd7040688d0 2023-10-08T02:28:18.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.185+0000 7fd6f9ffb700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd6e80670a0 0x7fd6e8069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:18.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.185+0000 7fd6f9ffb700 1 -- 172.21.15.119:0/557054710 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6714+0+0 (secure 0 0 0) 0x7fd700083f20 con 0x7fd7040688d0 2023-10-08T02:28:18.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.185+0000 7fd70979a700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd6e80670a0 0x7fd6e8069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:18.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.186+0000 7fd70979a700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd6e80670a0 0x7fd6e8069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40093f0 tx=0x7fd6f4017000 comp rx=0 tx=0).ready entity=mgr.9450 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:18.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.189+0000 7fd6f9ffb700 1 -- 172.21.15.119:0/557054710 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd700050c90 con 0x7fd7040688d0 2023-10-08T02:28:18.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.404+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd704068d30 con 0x7fd6e80670a0 2023-10-08T02:28:18.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.467+0000 7fd6f9ffb700 1 -- 172.21.15.119:0/557054710 <== mgr.9450 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fd704068d30 con 0x7fd6e80670a0 2023-10-08T02:28:18.467 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-10-08T02:28:18.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.469+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd6e80670a0 msgr2=0x7fd6e8069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.469+0000 7fd70c1ff700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd6e80670a0 0x7fd6e8069550 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40093f0 tx=0x7fd6f4017000 comp rx=0 tx=0).stop 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.469+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040688d0 msgr2=0x7fd70415caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.469+0000 7fd70c1ff700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040688d0 0x7fd70415caa0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c5c0 tx=0x7fd70000c980 comp rx=0 tx=0).stop 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.470+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 shutdown_connections 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.470+0000 7fd70c1ff700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd6e80670a0 0x7fd6e8069550 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.470+0000 7fd70c1ff700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd704157740 0x7fd70415a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.470+0000 7fd70c1ff700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7040688d0 0x7fd70415caa0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.470+0000 7fd70c1ff700 1 --2- 172.21.15.119:0/557054710 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd7040b2100 0x7fd70414f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.470+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 >> 172.21.15.119:0/557054710 conn(0x7fd7040c8f90 msgr2=0x7fd7040ab800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.470+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 shutdown_connections 2023-10-08T02:28:18.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:18.470+0000 7fd70c1ff700 1 -- 172.21.15.119:0/557054710 wait complete. 2023-10-08T02:28:18.487 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-10-08T02:28:18.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:18.487 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-10-08T02:28:18.525 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_13_00_UTC 2023-10-08T02:28:18.525 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_14_00_UTC 2023-10-08T02:28:18.525 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_15_00_UTC 2023-10-08T02:28:18.525 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_16_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_17_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_18_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_19_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_20_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_21_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_22_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_23_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_24_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_25_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_26_00_UTC 2023-10-08T02:28:18.526 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_27_00_UTC 2023-10-08T02:28:18.527 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_28_00_UTC 2023-10-08T02:28:18.527 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_13_00_UTC 2023-10-08T02:28:18.528 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:18.528 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_13_00_UTC) 2023-10-08T02:28:18.609 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_14_00_UTC 2023-10-08T02:28:18.610 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:18.610 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_14_00_UTC) 2023-10-08T02:28:18.691 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_15_00_UTC 2023-10-08T02:28:18.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:18.691 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_15_00_UTC) 2023-10-08T02:28:18.773 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_16_00_UTC 2023-10-08T02:28:18.773 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:18.773 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_16_00_UTC) 2023-10-08T02:28:18.855 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_17_00_UTC 2023-10-08T02:28:18.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:18.855 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_17_00_UTC) 2023-10-08T02:28:18.938 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_18_00_UTC 2023-10-08T02:28:18.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:18.938 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_18_00_UTC) 2023-10-08T02:28:19.021 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_19_00_UTC 2023-10-08T02:28:19.021 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.021 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_19_00_UTC) 2023-10-08T02:28:19.104 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_20_00_UTC 2023-10-08T02:28:19.104 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.105 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_20_00_UTC) 2023-10-08T02:28:19.187 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_21_00_UTC 2023-10-08T02:28:19.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.187 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_21_00_UTC) 2023-10-08T02:28:19.271 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_22_00_UTC 2023-10-08T02:28:19.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.272 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_22_00_UTC) 2023-10-08T02:28:19.353 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_23_00_UTC 2023-10-08T02:28:19.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.354 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_23_00_UTC) 2023-10-08T02:28:19.437 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_24_00_UTC 2023-10-08T02:28:19.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.437 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_24_00_UTC) 2023-10-08T02:28:19.521 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_25_00_UTC 2023-10-08T02:28:19.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.522 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_25_00_UTC) 2023-10-08T02:28:19.603 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_26_00_UTC 2023-10-08T02:28:19.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.603 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_26_00_UTC) 2023-10-08T02:28:19.685 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_27_00_UTC 2023-10-08T02:28:19.686 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.686 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_27_00_UTC) 2023-10-08T02:28:19.767 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_28_00_UTC 2023-10-08T02:28:19.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.767 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-10-08-02_28_00_UTC) 2023-10-08T02:28:19.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:19.849 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-10-08T02:28:19.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T02:28:20.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.152+0000 7fb55adff700 1 Processor -- start 2023-10-08T02:28:20.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.152+0000 7fb55adff700 1 -- start start 2023-10-08T02:28:20.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb55adff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb55406c0a0 0x7fb5540caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb55adff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb55415df50 0x7fb55415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb55adff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb55415d7e0 con 0x7fb55406bcd0 2023-10-08T02:28:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb55adff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55415dac0 con 0x7fb55406c0a0 2023-10-08T02:28:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb55adff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55415aa30 con 0x7fb55415df50 2023-10-08T02:28:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb553fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb55406c0a0 0x7fb5540caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb553fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb55406c0a0 0x7fb5540caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50826/0 (socket says 172.21.15.119:50826) 2023-10-08T02:28:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb553fff700 1 -- 172.21.15.119:0/1964529781 learned_addr learned my addr 172.21.15.119:0/1964529781 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:20.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb55939c700 1 --2- 172.21.15.119:0/1964529781 >> v2:172.21.15.184:3300/0 conn(0x7fb55415df50 0x7fb55415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:20.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb553fff700 1 -- 172.21.15.119:0/1964529781 >> v2:172.21.15.184:3300/0 conn(0x7fb55415df50 msgr2=0x7fb55415e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:20.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb553fff700 1 --2- 172.21.15.119:0/1964529781 >> v2:172.21.15.184:3300/0 conn(0x7fb55415df50 0x7fb55415e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:20.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb553fff700 1 -- 172.21.15.119:0/1964529781 >> v1:172.21.15.184:6789/0 conn(0x7fb55406bcd0 legacy=0x7fb55415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:20.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb553fff700 1 -- 172.21.15.119:0/1964529781 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55415ad10 con 0x7fb55406c0a0 2023-10-08T02:28:20.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.154+0000 7fb55939c700 1 --2- 172.21.15.119:0/1964529781 >> v2:172.21.15.184:3300/0 conn(0x7fb55415df50 0x7fb55415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:20.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.155+0000 7fb553fff700 1 --2- 172.21.15.119:0/1964529781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406c0a0 0x7fb5540caa90 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fb55415d720 tx=0x7fb540009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1992d7be88d93473 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb5537fe700 1 -- 172.21.15.119:0/1964529781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000b860 con 0x7fb55406c0a0 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb5537fe700 1 -- 172.21.15.119:0/1964529781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54000b9c0 con 0x7fb55406c0a0 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb5537fe700 1 -- 172.21.15.119:0/1964529781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000bd80 con 0x7fb55406c0a0 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb55adff700 1 -- 172.21.15.119:0/1964529781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406c0a0 msgr2=0x7fb5540caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb55adff700 1 --2- 172.21.15.119:0/1964529781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406c0a0 0x7fb5540caa90 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fb55415d720 tx=0x7fb540009250 comp rx=0 tx=0).stop 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb55adff700 1 -- 172.21.15.119:0/1964529781 shutdown_connections 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb55adff700 1 --2- 172.21.15.119:0/1964529781 >> v2:172.21.15.184:3300/0 conn(0x7fb55415df50 0x7fb55415e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb55adff700 1 --2- 172.21.15.119:0/1964529781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406c0a0 0x7fb5540caa90 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.157+0000 7fb55adff700 1 -- 172.21.15.119:0/1964529781 >> 172.21.15.119:0/1964529781 conn(0x7fb5540ce600 msgr2=0x7fb5540cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:20.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.158+0000 7fb55adff700 1 -- 172.21.15.119:0/1964529781 shutdown_connections 2023-10-08T02:28:20.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.158+0000 7fb55adff700 1 -- 172.21.15.119:0/1964529781 wait complete. 2023-10-08T02:28:20.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.159+0000 7fb55adff700 1 Processor -- start 2023-10-08T02:28:20.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.159+0000 7fb55adff700 1 -- start start 2023-10-08T02:28:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb55adff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406bcd0 0x7fb554151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb55adff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb55406c0a0 0x7fb5541569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb55adff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb55415df50 0x7fb554156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb55adff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5540b2180 con 0x7fb55406bcd0 2023-10-08T02:28:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb55adff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55416e1e0 con 0x7fb55415df50 2023-10-08T02:28:20.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb55adff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55416e3f0 con 0x7fb55406c0a0 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb558b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406bcd0 0x7fb554151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb558b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406bcd0 0x7fb554151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50836/0 (socket says 172.21.15.119:50836) 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.160+0000 7fb558b9b700 1 -- 172.21.15.119:0/3913970461 learned_addr learned my addr 172.21.15.119:0/3913970461 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb553fff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb55406c0a0 0x7fb5541569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb558b9b700 1 -- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb55406c0a0 msgr2=0x7fb5541569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb558b9b700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb55406c0a0 0x7fb5541569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb55939c700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb55415df50 0x7fb554156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb558b9b700 1 -- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb55415df50 msgr2=0x7fb554156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb558b9b700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb55415df50 0x7fb554156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb558b9b700 1 -- 172.21.15.119:0/3913970461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400094f0 con 0x7fb55406bcd0 2023-10-08T02:28:20.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb553fff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb55406c0a0 0x7fb5541569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb55939c700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb55415df50 0x7fb554156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb558b9b700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406bcd0 0x7fb554151460 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c4e0 tx=0x7fb54800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb550ff9700 1 -- 172.21.15.119:0/3913970461 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548011590 con 0x7fb55406bcd0 2023-10-08T02:28:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb550ff9700 1 -- 172.21.15.119:0/3913970461 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb548011710 con 0x7fb55406bcd0 2023-10-08T02:28:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55416e6d0 con 0x7fb55406bcd0 2023-10-08T02:28:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb550ff9700 1 -- 172.21.15.119:0/3913970461 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548011ad0 con 0x7fb55406bcd0 2023-10-08T02:28:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.161+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb55416ec90 con 0x7fb55406bcd0 2023-10-08T02:28:20.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.163+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55416ee80 con 0x7fb55406bcd0 2023-10-08T02:28:20.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.163+0000 7fb550ff9700 1 -- 172.21.15.119:0/3913970461 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 98) v1 ==== 84871+0+0 (secure 0 0 0) 0x7fb54801b3b0 con 0x7fb55406bcd0 2023-10-08T02:28:20.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.164+0000 7fb550ff9700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb5380670a0 0x7fb538069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:20.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.165+0000 7fb550ff9700 1 -- 172.21.15.119:0/3913970461 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 6714+0+0 (secure 0 0 0) 0x7fb548084040 con 0x7fb55406bcd0 2023-10-08T02:28:20.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.165+0000 7fb553fff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb5380670a0 0x7fb538069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:20.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.165+0000 7fb553fff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb5380670a0 0x7fb538069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb55415d720 tx=0x7fb540017000 comp rx=0 tx=0).ready entity=mgr.9450 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:20.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.169+0000 7fb550ff9700 1 -- 172.21.15.119:0/3913970461 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb548050db0 con 0x7fb55406bcd0 2023-10-08T02:28:20.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:20.453+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb55404c3c0 con 0x7fb55406bcd0 2023-10-08T02:28:21.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.318+0000 7fb550ff9700 1 -- 172.21.15.119:0/3913970461 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fb548051180 con 0x7fb55406bcd0 2023-10-08T02:28:21.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.319+0000 7fb550ff9700 1 -- 172.21.15.119:0/3913970461 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v99) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb5480706d0 con 0x7fb55406bcd0 2023-10-08T02:28:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb5380670a0 msgr2=0x7fb538069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:21.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb5380670a0 0x7fb538069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb55415d720 tx=0x7fb540017000 comp rx=0 tx=0).stop 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406bcd0 msgr2=0x7fb554151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406bcd0 0x7fb554151460 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c4e0 tx=0x7fb54800c8a0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 shutdown_connections 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb5380670a0 0x7fb538069550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb55415df50 0x7fb554156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb55406bcd0 0x7fb554151460 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 --2- 172.21.15.119:0/3913970461 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb55406c0a0 0x7fb5541569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.322+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 >> 172.21.15.119:0/3913970461 conn(0x7fb5540ce600 msgr2=0x7fb554069c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.323+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 shutdown_connections 2023-10-08T02:28:21.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.323+0000 7fb55adff700 1 -- 172.21.15.119:0/3913970461 wait complete. 2023-10-08T02:28:21.350 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:28:21.351 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:28:21.441 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:28:21.518 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:28:21.519 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T02:28:21.519 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:21.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T02:28:21.610 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:28:21.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:28:21.611 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T02:28:21.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:28:21.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.851+0000 7fb4c3fff700 1 Processor -- start 2023-10-08T02:28:21.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.851+0000 7fb4c3fff700 1 -- start start 2023-10-08T02:28:21.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.852+0000 7fb4c3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:21.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.852+0000 7fb4c3fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c40b1410 con 0x7fb4c40c6c60 2023-10-08T02:28:21.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.852+0000 7fb4c3fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c4063230 con 0x7fb4c40c8830 2023-10-08T02:28:21.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.852+0000 7fb4c3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40b2b90 con 0x7fb4c41595a0 2023-10-08T02:28:21.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.853+0000 7fb4c27fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb4c40c6c60 0x7fb4c40c8120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42214/0 (socket says 172.21.15.119:42214) 2023-10-08T02:28:21.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.853+0000 7fb4c27fc700 1 -- 172.21.15.119:0/4227632462 learned_addr learned my addr 172.21.15.119:0/4227632462 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:21.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.853+0000 7fb4c2ffd700 1 --2- 172.21.15.119:0/4227632462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:21.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.853+0000 7fb4c1ffb700 1 -- 172.21.15.119:0/4227632462 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281651765 0 0) 0x7fb4c4063230 con 0x7fb4c40c8830 2023-10-08T02:28:21.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.853+0000 7fb4c2ffd700 1 -- 172.21.15.119:0/4227632462 >> v1:172.21.15.184:6789/0 conn(0x7fb4c40c8830 legacy=0x7fb4c4159970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:21.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.853+0000 7fb4c2ffd700 1 -- 172.21.15.119:0/4227632462 >> v1:172.21.15.119:6789/0 conn(0x7fb4c40c6c60 legacy=0x7fb4c40c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:21.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.853+0000 7fb4c2ffd700 1 -- 172.21.15.119:0/4227632462 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c40634f0 con 0x7fb4c41595a0 2023-10-08T02:28:21.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c2ffd700 1 --2- 172.21.15.119:0/4227632462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40b18a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009910 tx=0x7fb4b40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46280a67c4e746fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:21.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c1ffb700 1 -- 172.21.15.119:0/4227632462 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400b760 con 0x7fb4c41595a0 2023-10-08T02:28:21.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c1ffb700 1 -- 172.21.15.119:0/4227632462 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400b8e0 con 0x7fb4c41595a0 2023-10-08T02:28:21.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c1ffb700 1 -- 172.21.15.119:0/4227632462 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400bca0 con 0x7fb4c41595a0 2023-10-08T02:28:21.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c3fff700 1 -- 172.21.15.119:0/4227632462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 msgr2=0x7fb4c40b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:21.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c3fff700 1 --2- 172.21.15.119:0/4227632462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40b18a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009910 tx=0x7fb4b40091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c3fff700 1 -- 172.21.15.119:0/4227632462 shutdown_connections 2023-10-08T02:28:21.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c3fff700 1 --2- 172.21.15.119:0/4227632462 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40b18a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.854+0000 7fb4c3fff700 1 -- 172.21.15.119:0/4227632462 >> 172.21.15.119:0/4227632462 conn(0x7fb4c406ddb0 msgr2=0x7fb4c406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:21.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.855+0000 7fb4c3fff700 1 -- 172.21.15.119:0/4227632462 shutdown_connections 2023-10-08T02:28:21.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.855+0000 7fb4c3fff700 1 -- 172.21.15.119:0/4227632462 wait complete. 2023-10-08T02:28:21.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.856+0000 7fb4c3fff700 1 Processor -- start 2023-10-08T02:28:21.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.856+0000 7fb4c3fff700 1 -- start start 2023-10-08T02:28:21.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c40c6c60 0x7fb4c4146590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:21.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c40c8830 0x7fb4c40c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:21.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:21.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40c67b0 con 0x7fb4c40c8830 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c3fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c4146ad0 con 0x7fb4c40c6c60 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c40c26f0 con 0x7fb4c41595a0 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c27fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c40c8830 0x7fb4c40c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c40c6c60 0x7fb4c4146590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c27fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c40c8830 0x7fb4c40c5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50866/0 (socket says 172.21.15.119:50866) 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c37fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.857+0000 7fb4c27fc700 1 -- 172.21.15.119:0/1131399597 learned_addr learned my addr 172.21.15.119:0/1131399597 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4c2ffd700 1 -- 172.21.15.119:0/1131399597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 msgr2=0x7fb4c40c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4c2ffd700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40c6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4c2ffd700 1 -- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c40c8830 msgr2=0x7fb4c40c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:21.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4c2ffd700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c40c8830 0x7fb4c40c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:21.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4c2ffd700 1 -- 172.21.15.119:0/1131399597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b4009420 con 0x7fb4c40c6c60 2023-10-08T02:28:21.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4c2ffd700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c40c6c60 0x7fb4c4146590 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4015000 tx=0x7fb4b4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:21.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4aaffd700 1 -- 172.21.15.119:0/1131399597 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4006b30 con 0x7fb4c40c6c60 2023-10-08T02:28:21.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4aaffd700 1 -- 172.21.15.119:0/1131399597 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b400bc60 con 0x7fb4c40c6c60 2023-10-08T02:28:21.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c40c29d0 con 0x7fb4c40c6c60 2023-10-08T02:28:21.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.858+0000 7fb4aaffd700 1 -- 172.21.15.119:0/1131399597 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b401c690 con 0x7fb4c40c6c60 2023-10-08T02:28:21.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.859+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c40c2f90 con 0x7fb4c40c6c60 2023-10-08T02:28:21.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.859+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c404c3c0 con 0x7fb4c40c6c60 2023-10-08T02:28:21.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.861+0000 7fb4aaffd700 1 -- 172.21.15.119:0/1131399597 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fb4b401cac0 con 0x7fb4c40c6c60 2023-10-08T02:28:21.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.861+0000 7fb4aaffd700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb4b0067050 0x7fb4b0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:21.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.862+0000 7fb4aaffd700 1 -- 172.21.15.119:0/1131399597 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6802+0+0 (secure 0 0 0) 0x7fb4b4087360 con 0x7fb4c40c6c60 2023-10-08T02:28:21.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.862+0000 7fb4c27fc700 1 -- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb4b0067050 msgr2=0x7fb4b0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:21.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.862+0000 7fb4c27fc700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb4b0067050 0x7fb4b0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:21.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:21.869+0000 7fb4aaffd700 1 -- 172.21.15.119:0/1131399597 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb4b4054080 con 0x7fb4c40c6c60 2023-10-08T02:28:22.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.063+0000 7fb4c27fc700 1 -- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb4b0067050 msgr2=0x7fb4b0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:22.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.063+0000 7fb4c27fc700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb4b0067050 0x7fb4b0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:22.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.112+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4c4159970 con 0x7fb4c40c6c60 2023-10-08T02:28:22.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.113+0000 7fb4aaffd700 1 -- 172.21.15.119:0/1131399597 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fb4b40576a0 con 0x7fb4c40c6c60 2023-10-08T02:28:22.113 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:22.114 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7409,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3771852147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3771852147},{"type":"v1","addr":"172.21.15.184:6834","nonce":3771852147}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/524755843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":524755843},{"type":"v1","addr":"172.21.15.184:6835","nonce":524755843}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9144,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/295310163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":295310163},{"type":"v1","addr":"172.21.15.119:6833","nonce":295310163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:12:24.579741+0000","modified":"2023-10-08T02:12:28.133805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9150},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9150":{"gid":9150,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2713045427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2713045427},{"type":"v1","addr":"172.21.15.119:6835","nonce":2713045427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T02:28:22.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb4b0067050 msgr2=0x7fb4b0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:22.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb4b0067050 0x7fb4b0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c40c6c60 msgr2=0x7fb4c4146590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c40c6c60 0x7fb4c4146590 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4015000 tx=0x7fb4b4016000 comp rx=0 tx=0).stop 2023-10-08T02:28:22.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 shutdown_connections 2023-10-08T02:28:22.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb4b0067050 0x7fb4b0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb4c41595a0 0x7fb4c40c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb4c40c6c60 0x7fb4c4146590 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 --2- 172.21.15.119:0/1131399597 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb4c40c8830 0x7fb4c40c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 >> 172.21.15.119:0/1131399597 conn(0x7fb4c406ddb0 msgr2=0x7fb4c4158520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:22.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 shutdown_connections 2023-10-08T02:28:22.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.115+0000 7fb4c3fff700 1 -- 172.21.15.119:0/1131399597 wait complete. 2023-10-08T02:28:22.117 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 79 2023-10-08T02:28:22.134 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:28:22.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.320+0000 7f438f3ff700 1 Processor -- start 2023-10-08T02:28:22.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.320+0000 7f438f3ff700 1 -- start start 2023-10-08T02:28:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.321+0000 7f438f3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880b1420 0x7f43880ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.321+0000 7f438f3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880cc300 con 0x7f43880cbf30 2023-10-08T02:28:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.321+0000 7f438f3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43880ad370 con 0x7f43880ccf90 2023-10-08T02:28:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.321+0000 7f438f3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4388156bf0 con 0x7f43880b1420 2023-10-08T02:28:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.321+0000 7f438d19b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f43880cbf30 0x7f43880b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42218/0 (socket says 172.21.15.119:42218) 2023-10-08T02:28:22.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.321+0000 7f438d19b700 1 -- 172.21.15.119:0/1496222862 learned_addr learned my addr 172.21.15.119:0/1496222862 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f438c99a700 1 --2- 172.21.15.119:0/1496222862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880b1420 0x7f43880ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f437ffff700 1 -- 172.21.15.119:0/1496222862 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137181978 0 0) 0x7f43880cc300 con 0x7f43880cbf30 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f437ffff700 1 -- 172.21.15.119:0/1496222862 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4374003570 con 0x7f43880cbf30 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f437ffff700 1 -- 172.21.15.119:0/1496222862 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3143566317 0 0) 0x7f43880ad370 con 0x7f43880ccf90 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f437ffff700 1 -- 172.21.15.119:0/1496222862 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43880cc300 con 0x7f43880ccf90 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f438c99a700 1 -- 172.21.15.119:0/1496222862 >> v1:172.21.15.184:6789/0 conn(0x7f43880ccf90 legacy=0x7f43880c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f438c99a700 1 -- 172.21.15.119:0/1496222862 >> v1:172.21.15.119:6789/0 conn(0x7f43880cbf30 legacy=0x7f43880b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f438c99a700 1 -- 172.21.15.119:0/1496222862 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43880ad650 con 0x7f43880b1420 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.322+0000 7f438c99a700 1 --2- 172.21.15.119:0/1496222862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880b1420 0x7f43880ae410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4378009910 tx=0x7f43780091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee2ec86cca197cf6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f437ffff700 1 -- 172.21.15.119:0/1496222862 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378011630 con 0x7f43880b1420 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f437ffff700 1 -- 172.21.15.119:0/1496222862 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4378011790 con 0x7f43880b1420 2023-10-08T02:28:22.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f437ffff700 1 -- 172.21.15.119:0/1496222862 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4378011b30 con 0x7f43880b1420 2023-10-08T02:28:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f438f3ff700 1 -- 172.21.15.119:0/1496222862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880b1420 msgr2=0x7f43880ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f438f3ff700 1 --2- 172.21.15.119:0/1496222862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880b1420 0x7f43880ae410 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4378009910 tx=0x7f43780091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f438f3ff700 1 -- 172.21.15.119:0/1496222862 shutdown_connections 2023-10-08T02:28:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f438f3ff700 1 --2- 172.21.15.119:0/1496222862 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880b1420 0x7f43880ae410 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f438f3ff700 1 -- 172.21.15.119:0/1496222862 >> 172.21.15.119:0/1496222862 conn(0x7f438806a320 msgr2=0x7f43880cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f438f3ff700 1 -- 172.21.15.119:0/1496222862 shutdown_connections 2023-10-08T02:28:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.323+0000 7f438f3ff700 1 -- 172.21.15.119:0/1496222862 wait complete. 2023-10-08T02:28:22.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.325+0000 7f438f3ff700 1 Processor -- start 2023-10-08T02:28:22.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.325+0000 7f438f3ff700 1 -- start start 2023-10-08T02:28:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438f3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43880b1420 0x7f43880c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438f3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43880cbf30 0x7f4388144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438f3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880ccf90 0x7f43880a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438f3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43880c4c80 con 0x7f43880cbf30 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438f3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43880c47f0 con 0x7f43880b1420 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438f3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43880c3be0 con 0x7f43880ccf90 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43880cbf30 0x7f4388144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438d19b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43880b1420 0x7f43880c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438c99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43880cbf30 0x7f4388144950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50870/0 (socket says 172.21.15.119:50870) 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438d99c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880ccf90 0x7f43880a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.326+0000 7f438c99a700 1 -- 172.21.15.119:0/393961898 learned_addr learned my addr 172.21.15.119:0/393961898 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438d19b700 1 -- 172.21.15.119:0/393961898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880ccf90 msgr2=0x7f43880a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438d19b700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880ccf90 0x7f43880a54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438d19b700 1 -- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43880cbf30 msgr2=0x7f4388144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438d19b700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43880cbf30 0x7f4388144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438d19b700 1 -- 172.21.15.119:0/393961898 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4378009420 con 0x7f43880b1420 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438c99a700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43880cbf30 0x7f4388144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438d19b700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43880b1420 0x7f43880c32e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f438400c5b0 tx=0x7f4384007d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f437d7fa700 1 -- 172.21.15.119:0/393961898 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384005b30 con 0x7f43880b1420 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f437d7fa700 1 -- 172.21.15.119:0/393961898 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4384005c90 con 0x7f43880b1420 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43880c3ec0 con 0x7f43880b1420 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f437d7fa700 1 -- 172.21.15.119:0/393961898 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384007300 con 0x7f43880b1420 2023-10-08T02:28:22.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.327+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43880c4480 con 0x7f43880b1420 2023-10-08T02:28:22.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.328+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4388169e80 con 0x7f43880b1420 2023-10-08T02:28:22.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.329+0000 7f437d7fa700 1 -- 172.21.15.119:0/393961898 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f438401b3b0 con 0x7f43880b1420 2023-10-08T02:28:22.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.330+0000 7f437d7fa700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f436c067050 0x7f436c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.330+0000 7f437d7fa700 1 -- 172.21.15.119:0/393961898 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6802+0+0 (secure 0 0 0) 0x7f4384083fc0 con 0x7f43880b1420 2023-10-08T02:28:22.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.330+0000 7f438c99a700 1 -- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f436c067050 msgr2=0x7f436c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:22.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.330+0000 7f438c99a700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f436c067050 0x7f436c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:22.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.338+0000 7f437d7fa700 1 -- 172.21.15.119:0/393961898 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4384050c30 con 0x7f43880b1420 2023-10-08T02:28:22.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.531+0000 7f438c99a700 1 -- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f436c067050 msgr2=0x7f436c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:22.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.531+0000 7f438c99a700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f436c067050 0x7f436c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:22.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.536+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f438804c3c0 con 0x7f43880b1420 2023-10-08T02:28:22.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.537+0000 7f437d7fa700 1 -- 172.21.15.119:0/393961898 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+17087 (secure 0 0 0) 0x7f4384017020 con 0x7f43880b1420 2023-10-08T02:28:22.538 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:22.538 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":177,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:28:21.306718+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:12:22.679007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T02:12:23.548789+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":177,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.39","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/506139810":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1047785372":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/932366218":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/3986903627":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1092901654":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1277970358":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/1588612751":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/655693879":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/4078947876":"2023-10-09T02:12:45.254733+0000","172.21.15.119:6836/85511":"2023-10-09T02:12:45.254733+0000","172.21.15.119:6837/85511":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/4064389633":"2023-10-09T02:12:45.254733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":11,"length":6},{"begin":27,"length":6}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:28:22.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f436c067050 msgr2=0x7f436c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:22.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f436c067050 0x7f436c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43880b1420 msgr2=0x7f43880c32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43880b1420 0x7f43880c32e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f438400c5b0 tx=0x7f4384007d70 comp rx=0 tx=0).stop 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 shutdown_connections 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f436c067050 0x7f436c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43880ccf90 0x7f43880a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f43880b1420 0x7f43880c32e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 --2- 172.21.15.119:0/393961898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43880cbf30 0x7f4388144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 >> 172.21.15.119:0/393961898 conn(0x7f438806a320 msgr2=0x7f43880c8550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:22.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.539+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 shutdown_connections 2023-10-08T02:28:22.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.540+0000 7f438f3ff700 1 -- 172.21.15.119:0/393961898 wait complete. 2023-10-08T02:28:22.559 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T02:12:22.679007+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T02:12:23.548789+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '177', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 32, 'snap_epoch': 177, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T02:28:22.559 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:28:22.646 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:22.646+0000 7fe381ed3180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:28:22.693 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:22.693+0000 7ff7a7479180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:28:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.748+0000 7f867dd9d700 1 Processor -- start 2023-10-08T02:28:22.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.748+0000 7f867dd9d700 1 -- start start 2023-10-08T02:28:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.749+0000 7f867dd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b4e70 0x7f86780ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.749+0000 7f867dd9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86780b16d0 0x7f86780b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.749+0000 7f867dd9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86780b2080 con 0x7f867815d240 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f867dd9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780cfb00 con 0x7f86780b16d0 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f867dd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780b5c40 con 0x7f86780b4e70 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f8677fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86780b16d0 0x7f86780b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f8677fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f86780b16d0 0x7f86780b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50872/0 (socket says 172.21.15.119:50872) 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f8677fff700 1 -- 172.21.15.119:0/835739204 learned_addr learned my addr 172.21.15.119:0/835739204 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f867d59c700 1 --1- 172.21.15.119:0/835739204 >> v1:172.21.15.119:6789/0 conn(0x7f867815d240 0x7f867815d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42234/0 (socket says 172.21.15.119:42234) 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f867cd9b700 1 --2- 172.21.15.119:0/835739204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b4e70 0x7f86780ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f8677fff700 1 -- 172.21.15.119:0/835739204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b4e70 msgr2=0x7f86780ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f8677fff700 1 --2- 172.21.15.119:0/835739204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b4e70 0x7f86780ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f8677fff700 1 -- 172.21.15.119:0/835739204 >> v1:172.21.15.119:6789/0 conn(0x7f867815d240 legacy=0x7f867815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f8677fff700 1 -- 172.21.15.119:0/835739204 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86780b5f20 con 0x7f86780b16d0 2023-10-08T02:28:22.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.750+0000 7f86777fe700 1 -- 172.21.15.119:0/835739204 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218577103 0 0) 0x7f86780b2080 con 0x7f867815d240 2023-10-08T02:28:22.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.751+0000 7f8677fff700 1 --2- 172.21.15.119:0/835739204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86780b16d0 0x7f86780b1b40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f86640099b0 tx=0x7f8664009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48f25b751b041e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:22.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.751+0000 7f86777fe700 1 -- 172.21.15.119:0/835739204 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664009ea0 con 0x7f86780b16d0 2023-10-08T02:28:22.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.751+0000 7f86777fe700 1 -- 172.21.15.119:0/835739204 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866400b860 con 0x7f86780b16d0 2023-10-08T02:28:22.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.751+0000 7f86777fe700 1 -- 172.21.15.119:0/835739204 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400bc00 con 0x7f86780b16d0 2023-10-08T02:28:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.751+0000 7f867dd9d700 1 -- 172.21.15.119:0/835739204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86780b16d0 msgr2=0x7f86780b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.751+0000 7f867dd9d700 1 --2- 172.21.15.119:0/835739204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86780b16d0 0x7f86780b1b40 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f86640099b0 tx=0x7f8664009250 comp rx=0 tx=0).stop 2023-10-08T02:28:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.752+0000 7f867dd9d700 1 -- 172.21.15.119:0/835739204 shutdown_connections 2023-10-08T02:28:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.752+0000 7f867dd9d700 1 --2- 172.21.15.119:0/835739204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b4e70 0x7f86780ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.752+0000 7f867dd9d700 1 --2- 172.21.15.119:0/835739204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86780b16d0 0x7f86780b1b40 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.752+0000 7f867dd9d700 1 -- 172.21.15.119:0/835739204 >> 172.21.15.119:0/835739204 conn(0x7f86780d10c0 msgr2=0x7f86780affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:22.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.752+0000 7f867dd9d700 1 -- 172.21.15.119:0/835739204 shutdown_connections 2023-10-08T02:28:22.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.752+0000 7f867dd9d700 1 -- 172.21.15.119:0/835739204 wait complete. 2023-10-08T02:28:22.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.754+0000 7f867dd9d700 1 Processor -- start 2023-10-08T02:28:22.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.754+0000 7f867dd9d700 1 -- start start 2023-10-08T02:28:22.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.755+0000 7f867dd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b16d0 0x7f8678145790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.755+0000 7f867dd9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86780b4e70 0x7f867815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.755+0000 7f867dd9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867815d240 0x7f867815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.755+0000 7f867dd9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c8480 con 0x7f867815d240 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.755+0000 7f867dd9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c7370 con 0x7f86780b4e70 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.755+0000 7f867dd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780c7b50 con 0x7f86780b16d0 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867d59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867815d240 0x7f867815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867d59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867815d240 0x7f867815ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50884/0 (socket says 172.21.15.119:50884) 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867cd9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b16d0 0x7f8678145790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867d59c700 1 -- 172.21.15.119:0/2620366741 learned_addr learned my addr 172.21.15.119:0/2620366741 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f8677fff700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86780b4e70 0x7f867815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:22.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867cd9b700 1 -- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86780b4e70 msgr2=0x7f867815b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867cd9b700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86780b4e70 0x7f867815b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867cd9b700 1 -- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867815d240 msgr2=0x7f867815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867cd9b700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867815d240 0x7f867815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867cd9b700 1 -- 172.21.15.119:0/2620366741 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640094f0 con 0x7f86780b16d0 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f867d59c700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867815d240 0x7f867815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.756+0000 7f8677fff700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86780b4e70 0x7f867815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.757+0000 7f867cd9b700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b16d0 0x7f8678145790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f866c00ea50 tx=0x7f866c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.757+0000 7f8674ff9700 1 -- 172.21.15.119:0/2620366741 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c0095d0 con 0x7f86780b16d0 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.757+0000 7f8674ff9700 1 -- 172.21.15.119:0/2620366741 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c00c7f0 con 0x7f86780b16d0 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.757+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86780c7e30 con 0x7f86780b16d0 2023-10-08T02:28:22.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.757+0000 7f8674ff9700 1 -- 172.21.15.119:0/2620366741 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c00cbb0 con 0x7f86780b16d0 2023-10-08T02:28:22.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.757+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8678145cd0 con 0x7f86780b16d0 2023-10-08T02:28:22.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.758+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86780c8760 con 0x7f86780b16d0 2023-10-08T02:28:22.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.759+0000 7f8674ff9700 1 -- 172.21.15.119:0/2620366741 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f866c013470 con 0x7f86780b16d0 2023-10-08T02:28:22.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.760+0000 7f8674ff9700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f865c067050 0x7f865c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:22.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.760+0000 7f8674ff9700 1 -- 172.21.15.119:0/2620366741 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6802+0+0 (secure 0 0 0) 0x7f866c084c40 con 0x7f86780b16d0 2023-10-08T02:28:22.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.760+0000 7f8677fff700 1 -- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f865c067050 msgr2=0x7f865c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:22.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.760+0000 7f8677fff700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f865c067050 0x7f865c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:22.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.764+0000 7f8674ff9700 1 -- 172.21.15.119:0/2620366741 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f866c0518b0 con 0x7f86780b16d0 2023-10-08T02:28:22.812 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:22.813+0000 7fe381ed3180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:28:22.867 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:22.866+0000 7ff7a7479180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:28:22.898 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:22.898+0000 7fe381ed3180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:28:22.954 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:22.954+0000 7ff7a7479180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:28:22.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.961+0000 7f8677fff700 1 -- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f865c067050 msgr2=0x7f865c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:22.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.961+0000 7f8677fff700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f865c067050 0x7f865c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:22.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.966+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f867804bb80 con 0x7f86780b16d0 2023-10-08T02:28:22.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.967+0000 7f8674ff9700 1 -- 172.21.15.119:0/2620366741 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v177) v1 ==== 93+0+10 (secure 0 0 0) 0x7f866c054ed0 con 0x7f86780b16d0 2023-10-08T02:28:22.967 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:28:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f865c067050 msgr2=0x7f865c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f865c067050 0x7f865c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b16d0 msgr2=0x7f8678145790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b16d0 0x7f8678145790 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f866c00ea50 tx=0x7f866c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 shutdown_connections 2023-10-08T02:28:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f865c067050 0x7f865c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86780b16d0 0x7f8678145790 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f867815d240 0x7f867815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 --2- 172.21.15.119:0/2620366741 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86780b4e70 0x7f867815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:22.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 >> 172.21.15.119:0/2620366741 conn(0x7f86780d10c0 msgr2=0x7f86780ceaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:22.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 shutdown_connections 2023-10-08T02:28:22.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:22.969+0000 7f867dd9d700 1 -- 172.21.15.119:0/2620366741 wait complete. 2023-10-08T02:28:22.988 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T02:28:23.126 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:23.126+0000 7fe381ed3180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:28:23.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.179+0000 7f3a4d19d700 1 Processor -- start 2023-10-08T02:28:23.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.179+0000 7f3a4d19d700 1 -- start start 2023-10-08T02:28:23.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.180+0000 7f3a4d19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a480ccc00 0x7f3a48069ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.180+0000 7f3a4d19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3a4806a020 0x7f3a480cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.180+0000 7f3a4d19d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a480c9f30 con 0x7f3a48159a60 2023-10-08T02:28:23.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.180+0000 7f3a4d19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a4806dfc0 con 0x7f3a480ccc00 2023-10-08T02:28:23.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.180+0000 7f3a4d19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a480ca470 con 0x7f3a4806a020 2023-10-08T02:28:23.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.180+0000 7f3a477fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a480ccc00 0x7f3a48069ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.180+0000 7f3a47fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3a48159a60 0x7f3a480c9770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42248/0 (socket says 172.21.15.119:42248) 2023-10-08T02:28:23.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.180+0000 7f3a47fff700 1 -- 172.21.15.119:0/2547919793 learned_addr learned my addr 172.21.15.119:0/2547919793 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:23.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.181+0000 7f3a4c99c700 1 --2- 172.21.15.119:0/2547919793 >> v2:172.21.15.184:3300/0 conn(0x7f3a4806a020 0x7f3a480cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.181+0000 7f3a477fe700 1 -- 172.21.15.119:0/2547919793 >> v2:172.21.15.184:3300/0 conn(0x7f3a4806a020 msgr2=0x7f3a480cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:23.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.181+0000 7f3a477fe700 1 --2- 172.21.15.119:0/2547919793 >> v2:172.21.15.184:3300/0 conn(0x7f3a4806a020 0x7f3a480cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.181+0000 7f3a477fe700 1 -- 172.21.15.119:0/2547919793 >> v1:172.21.15.119:6789/0 conn(0x7f3a48159a60 legacy=0x7f3a480c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.181+0000 7f3a477fe700 1 -- 172.21.15.119:0/2547919793 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a480ca750 con 0x7f3a480ccc00 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.181+0000 7f3a46ffd700 1 -- 172.21.15.119:0/2547919793 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232072153 0 0) 0x7f3a480c9f30 con 0x7f3a48159a60 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.181+0000 7f3a477fe700 1 --2- 172.21.15.119:0/2547919793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a480ccc00 0x7f3a48069ae0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3a480cc410 tx=0x7f3a3c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a54e3bf8d301da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.181+0000 7f3a46ffd700 1 -- 172.21.15.119:0/2547919793 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00b690 con 0x7f3a480ccc00 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a46ffd700 1 -- 172.21.15.119:0/2547919793 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c00b7f0 con 0x7f3a480ccc00 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a46ffd700 1 -- 172.21.15.119:0/2547919793 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00bb90 con 0x7f3a480ccc00 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a4d19d700 1 -- 172.21.15.119:0/2547919793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a480ccc00 msgr2=0x7f3a48069ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/2547919793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a480ccc00 0x7f3a48069ae0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f3a480cc410 tx=0x7f3a3c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a4d19d700 1 -- 172.21.15.119:0/2547919793 shutdown_connections 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/2547919793 >> v2:172.21.15.184:3300/0 conn(0x7f3a4806a020 0x7f3a480cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/2547919793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a480ccc00 0x7f3a48069ae0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a4d19d700 1 -- 172.21.15.119:0/2547919793 >> 172.21.15.119:0/2547919793 conn(0x7f3a4806baa0 msgr2=0x7f3a4806bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:23.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a4d19d700 1 -- 172.21.15.119:0/2547919793 shutdown_connections 2023-10-08T02:28:23.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.182+0000 7f3a4d19d700 1 -- 172.21.15.119:0/2547919793 wait complete. 2023-10-08T02:28:23.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.184+0000 7f3a4d19d700 1 Processor -- start 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.184+0000 7f3a4d19d700 1 -- start start 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a4d19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4806a020 0x7f3a4816d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a4d19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a480ccc00 0x7f3a480c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a4d19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a48159a60 0x7f3a4815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a4d19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4816dc70 con 0x7f3a4806a020 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a4d19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4816df50 con 0x7f3a48159a60 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a4d19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4816e230 con 0x7f3a480ccc00 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a47fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4806a020 0x7f3a4816d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a47fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4806a020 0x7f3a4816d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50912/0 (socket says 172.21.15.119:50912) 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a47fff700 1 -- 172.21.15.119:0/1252488569 learned_addr learned my addr 172.21.15.119:0/1252488569 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a477fe700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a480ccc00 0x7f3a480c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a4c99c700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a48159a60 0x7f3a4815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a47fff700 1 -- 172.21.15.119:0/1252488569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a480ccc00 msgr2=0x7f3a480c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a47fff700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a480ccc00 0x7f3a480c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a47fff700 1 -- 172.21.15.119:0/1252488569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a48159a60 msgr2=0x7f3a4815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a47fff700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a48159a60 0x7f3a4815a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.185+0000 7f3a47fff700 1 -- 172.21.15.119:0/1252488569 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c009420 con 0x7f3a4806a020 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.186+0000 7f3a47fff700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4806a020 0x7f3a4816d440 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c5b0 tx=0x7f3a38001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.186+0000 7f3a2ffff700 1 -- 172.21.15.119:0/1252488569 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38005b30 con 0x7f3a4806a020 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.186+0000 7f3a2ffff700 1 -- 172.21.15.119:0/1252488569 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38005c90 con 0x7f3a4806a020 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.186+0000 7f3a2ffff700 1 -- 172.21.15.119:0/1252488569 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38007300 con 0x7f3a4806a020 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.186+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a480c7630 con 0x7f3a4806a020 2023-10-08T02:28:23.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.186+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a4816d980 con 0x7f3a4806a020 2023-10-08T02:28:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.187+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4806eba0 con 0x7f3a4806a020 2023-10-08T02:28:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.188+0000 7f3a2ffff700 1 -- 172.21.15.119:0/1252488569 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f3a3801b3b0 con 0x7f3a4806a020 2023-10-08T02:28:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.188+0000 7f3a2ffff700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a34067050 0x7f3a34069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.189+0000 7f3a477fe700 1 -- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a34067050 msgr2=0x7f3a34069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.189+0000 7f3a2ffff700 1 -- 172.21.15.119:0/1252488569 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6802+0+0 (secure 0 0 0) 0x7f3a38083fd0 con 0x7f3a4806a020 2023-10-08T02:28:23.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.189+0000 7f3a477fe700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a34067050 0x7f3a34069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:23.191 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:23.190+0000 7ff7a7479180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:28:23.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.193+0000 7f3a2ffff700 1 -- 172.21.15.119:0/1252488569 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a38050c40 con 0x7f3a4806a020 2023-10-08T02:28:23.305 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:23.305+0000 7fe381ed3180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:28:23.373 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:23.372+0000 7ff7a7479180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:28:23.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.390+0000 7f3a477fe700 1 -- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a34067050 msgr2=0x7f3a34069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:23.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.390+0000 7f3a477fe700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a34067050 0x7f3a34069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:23.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.390+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a480caa30 con 0x7f3a4806a020 2023-10-08T02:28:23.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.390+0000 7f3a2ffff700 1 -- 172.21.15.119:0/1252488569 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v177) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a38017020 con 0x7f3a4806a020 2023-10-08T02:28:23.391 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T02:28:23.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a34067050 msgr2=0x7f3a34069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a34067050 0x7f3a34069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4806a020 msgr2=0x7f3a4816d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4806a020 0x7f3a4816d440 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c5b0 tx=0x7f3a38001070 comp rx=0 tx=0).stop 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 shutdown_connections 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a34067050 0x7f3a34069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a48159a60 0x7f3a4815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a4806a020 0x7f3a4816d440 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 --2- 172.21.15.119:0/1252488569 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a480ccc00 0x7f3a480c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 >> 172.21.15.119:0/1252488569 conn(0x7f3a4806baa0 msgr2=0x7f3a48154300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 shutdown_connections 2023-10-08T02:28:23.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.393+0000 7f3a4d19d700 1 -- 172.21.15.119:0/1252488569 wait complete. 2023-10-08T02:28:23.398 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:23.398+0000 7fe381ed3180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:28:23.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T02:28:23.479 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:23.478+0000 7ff7a7479180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:28:23.562 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:23.563+0000 7fe381ed3180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:28:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.611+0000 7f9414b9d700 1 Processor -- start 2023-10-08T02:28:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.611+0000 7f9414b9d700 1 -- start start 2023-10-08T02:28:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.612+0000 7f9414b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f94100b16d0 0x7f94100b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.612+0000 7f9414b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f94100b33b0 0x7f94100b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.612+0000 7f9414b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100b2080 con 0x7f94100b4e70 2023-10-08T02:28:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.612+0000 7f9414b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94100cfb00 con 0x7f94100b16d0 2023-10-08T02:28:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.612+0000 7f9414b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94100b5c40 con 0x7f94100b33b0 2023-10-08T02:28:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.612+0000 7f940effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f94100b16d0 0x7f94100b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f94100b16d0 0x7f94100b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50924/0 (socket says 172.21.15.119:50924) 2023-10-08T02:28:23.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940effd700 1 -- 172.21.15.119:0/2799066893 learned_addr learned my addr 172.21.15.119:0/2799066893 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:23.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940ffff700 1 --2- 172.21.15.119:0/2799066893 >> v2:172.21.15.184:3300/0 conn(0x7f94100b33b0 0x7f94100b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940effd700 1 -- 172.21.15.119:0/2799066893 >> v2:172.21.15.184:3300/0 conn(0x7f94100b33b0 msgr2=0x7f94100b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:23.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940effd700 1 --2- 172.21.15.119:0/2799066893 >> v2:172.21.15.184:3300/0 conn(0x7f94100b33b0 0x7f94100b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940effd700 1 -- 172.21.15.119:0/2799066893 >> v1:172.21.15.184:6789/0 conn(0x7f94100b4e70 legacy=0x7f94100ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940effd700 1 -- 172.21.15.119:0/2799066893 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100b5f20 con 0x7f94100b16d0 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940effd700 1 --2- 172.21.15.119:0/2799066893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100b1b40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9410071380 tx=0x7f9404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9882cbec842b5fd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940e7fc700 1 -- 172.21.15.119:0/2799066893 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404009e70 con 0x7f94100b16d0 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940e7fc700 1 -- 172.21.15.119:0/2799066893 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940400b860 con 0x7f94100b16d0 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.613+0000 7f940e7fc700 1 -- 172.21.15.119:0/2799066893 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940400bc20 con 0x7f94100b16d0 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.614+0000 7f9414b9d700 1 -- 172.21.15.119:0/2799066893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 msgr2=0x7f94100b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.614+0000 7f9414b9d700 1 --2- 172.21.15.119:0/2799066893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100b1b40 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f9410071380 tx=0x7f9404009250 comp rx=0 tx=0).stop 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.614+0000 7f9414b9d700 1 -- 172.21.15.119:0/2799066893 shutdown_connections 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.614+0000 7f9414b9d700 1 --2- 172.21.15.119:0/2799066893 >> v2:172.21.15.184:3300/0 conn(0x7f94100b33b0 0x7f94100b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.614+0000 7f9414b9d700 1 --2- 172.21.15.119:0/2799066893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100b1b40 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.614+0000 7f9414b9d700 1 -- 172.21.15.119:0/2799066893 >> 172.21.15.119:0/2799066893 conn(0x7f94100d10c0 msgr2=0x7f94100affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:23.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.614+0000 7f9414b9d700 1 -- 172.21.15.119:0/2799066893 shutdown_connections 2023-10-08T02:28:23.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.614+0000 7f9414b9d700 1 -- 172.21.15.119:0/2799066893 wait complete. 2023-10-08T02:28:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.616+0000 7f9414b9d700 1 Processor -- start 2023-10-08T02:28:23.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.616+0000 7f9414b9d700 1 -- start start 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f9414b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f9414b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94100b33b0 0x7f9410158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f9414b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94100b4e70 0x7f941015d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f9414b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100cc680 con 0x7f94100b16d0 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f9414b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9410169d70 con 0x7f94100b4e70 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f9414b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9410169fc0 con 0x7f94100b33b0 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50940/0 (socket says 172.21.15.119:50940) 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940effd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94100b33b0 0x7f9410158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94100b4e70 0x7f941015d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:23.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940f7fe700 1 -- 172.21.15.119:0/1091553713 learned_addr learned my addr 172.21.15.119:0/1091553713 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940f7fe700 1 -- 172.21.15.119:0/1091553713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94100b33b0 msgr2=0x7f9410158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940f7fe700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94100b33b0 0x7f9410158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940f7fe700 1 -- 172.21.15.119:0/1091553713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94100b4e70 msgr2=0x7f941015d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940f7fe700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94100b4e70 0x7f941015d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.617+0000 7f940f7fe700 1 -- 172.21.15.119:0/1091553713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94040094f0 con 0x7f94100b16d0 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.618+0000 7f940f7fe700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100cb5d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f940000c590 tx=0x7f940000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.618+0000 7f93f77fe700 1 -- 172.21.15.119:0/1091553713 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400011590 con 0x7f94100b16d0 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.618+0000 7f93f77fe700 1 -- 172.21.15.119:0/1091553713 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9400011710 con 0x7f94100b16d0 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.618+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94100cbb10 con 0x7f94100b16d0 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.618+0000 7f93f77fe700 1 -- 172.21.15.119:0/1091553713 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400011ad0 con 0x7f94100b16d0 2023-10-08T02:28:23.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.618+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94100cc3a0 con 0x7f94100b16d0 2023-10-08T02:28:23.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.619+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9410088990 con 0x7f94100b16d0 2023-10-08T02:28:23.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.620+0000 7f93f77fe700 1 -- 172.21.15.119:0/1091553713 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f940001b3b0 con 0x7f94100b16d0 2023-10-08T02:28:23.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.621+0000 7f93f77fe700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93fc067050 0x7f93fc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:23.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.621+0000 7f93f77fe700 1 -- 172.21.15.119:0/1091553713 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6802+0+0 (secure 0 0 0) 0x7f9400083fb0 con 0x7f94100b16d0 2023-10-08T02:28:23.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.621+0000 7f940effd700 1 -- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93fc067050 msgr2=0x7f93fc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:23.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.621+0000 7f940effd700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93fc067050 0x7f93fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:23.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.625+0000 7f93f77fe700 1 -- 172.21.15.119:0/1091553713 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9400050c20 con 0x7f94100b16d0 2023-10-08T02:28:23.651 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:23.651+0000 7ff7a7479180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:28:23.665 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:23.666+0000 7fe381ed3180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:28:23.755 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:23.754+0000 7ff7a7479180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:28:23.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.821+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f941004bb80 con 0x7f94100b16d0 2023-10-08T02:28:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.822+0000 7f940effd700 1 -- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93fc067050 msgr2=0x7f93fc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.822+0000 7f940effd700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93fc067050 0x7f93fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:23.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.822+0000 7f93f77fe700 1 -- 172.21.15.119:0/1091553713 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v177) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9400017070 con 0x7f94100b16d0 2023-10-08T02:28:23.823 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T02:28:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93fc067050 msgr2=0x7f93fc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93fc067050 0x7f93fc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 msgr2=0x7f94100cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100cb5d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f940000c590 tx=0x7f940000c8a0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 shutdown_connections 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93fc067050 0x7f93fc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f94100b4e70 0x7f941015d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b16d0 0x7f94100cb5d0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 --2- 172.21.15.119:0/1091553713 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f94100b33b0 0x7f9410158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.824+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 >> 172.21.15.119:0/1091553713 conn(0x7f94100d10c0 msgr2=0x7f94100c5420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.825+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 shutdown_connections 2023-10-08T02:28:23.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:23.825+0000 7f9414b9d700 1 -- 172.21.15.119:0/1091553713 wait complete. 2023-10-08T02:28:23.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:28:24.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.033+0000 7fa6547ff700 1 Processor -- start 2023-10-08T02:28:24.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.033+0000 7fa6547ff700 1 -- start start 2023-10-08T02:28:24.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.034+0000 7fa6547ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.034+0000 7fa6547ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa64c15db10 0x7fa64c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.034+0000 7fa6547ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c15a160 con 0x7fa64c159890 2023-10-08T02:28:24.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.034+0000 7fa6547ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0d13a0 con 0x7fa64c15db10 2023-10-08T02:28:24.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.034+0000 7fa6547ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c15e140 con 0x7fa64c159c60 2023-10-08T02:28:24.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.034+0000 7fa65259b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa64c159890 0x7fa64c15abd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42254/0 (socket says 172.21.15.119:42254) 2023-10-08T02:28:24.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.034+0000 7fa65259b700 1 -- 172.21.15.119:0/2804130512 learned_addr learned my addr 172.21.15.119:0/2804130512 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:24.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa652d9c700 1 --2- 172.21.15.119:0/2804130512 >> v2:172.21.15.184:3300/0 conn(0x7fa64c15db10 0x7fa64c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa651d9a700 1 --2- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa651599700 1 -- 172.21.15.119:0/2804130512 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2283121675 0 0) 0x7fa64c15a160 con 0x7fa64c159890 2023-10-08T02:28:24.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa651599700 1 -- 172.21.15.119:0/2804130512 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa638003570 con 0x7fa64c159890 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa652d9c700 1 -- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 msgr2=0x7fa64c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa652d9c700 1 --2- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa652d9c700 1 -- 172.21.15.119:0/2804130512 >> v1:172.21.15.119:6789/0 conn(0x7fa64c159890 legacy=0x7fa64c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa652d9c700 1 -- 172.21.15.119:0/2804130512 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c15a3d0 con 0x7fa64c15db10 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa651d9a700 1 --2- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.035+0000 7fa652d9c700 1 --2- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c15db10 0x7fa64c069560 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa644009e60 tx=0x7fa644009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39e3a303b838e9d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa651599700 1 -- 172.21.15.119:0/2804130512 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400e070 con 0x7fa64c15db10 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa651599700 1 -- 172.21.15.119:0/2804130512 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64400bab0 con 0x7fa64c15db10 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa651599700 1 -- 172.21.15.119:0/2804130512 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64400be70 con 0x7fa64c15db10 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa6547ff700 1 -- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c15db10 msgr2=0x7fa64c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa6547ff700 1 --2- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c15db10 0x7fa64c069560 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa644009e60 tx=0x7fa644009780 comp rx=0 tx=0).stop 2023-10-08T02:28:24.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa6547ff700 1 -- 172.21.15.119:0/2804130512 shutdown_connections 2023-10-08T02:28:24.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa6547ff700 1 --2- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c15db10 0x7fa64c069560 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa6547ff700 1 --2- 172.21.15.119:0/2804130512 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa6547ff700 1 -- 172.21.15.119:0/2804130512 >> 172.21.15.119:0/2804130512 conn(0x7fa64c068e20 msgr2=0x7fa64c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:24.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa6547ff700 1 -- 172.21.15.119:0/2804130512 shutdown_connections 2023-10-08T02:28:24.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.036+0000 7fa6547ff700 1 -- 172.21.15.119:0/2804130512 wait complete. 2023-10-08T02:28:24.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.038+0000 7fa6547ff700 1 Processor -- start 2023-10-08T02:28:24.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.038+0000 7fa6547ff700 1 -- start start 2023-10-08T02:28:24.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa6547ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c159890 0x7fa64c16a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa6547ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c0cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa6547ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c15db10 0x7fa64c0c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa6547ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c16a190 con 0x7fa64c15db10 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa6547ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c169b60 con 0x7fa64c159890 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa6547ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0cd8f0 con 0x7fa64c159c60 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa652d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c15db10 0x7fa64c0c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa651d9a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c0cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa652d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c15db10 0x7fa64c0c6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50948/0 (socket says 172.21.15.119:50948) 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa652d9c700 1 -- 172.21.15.119:0/1042544992 learned_addr learned my addr 172.21.15.119:0/1042544992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.039+0000 7fa65259b700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c159890 0x7fa64c16a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa651d9a700 1 -- 172.21.15.119:0/1042544992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c159890 msgr2=0x7fa64c16a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa651d9a700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c159890 0x7fa64c16a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa651d9a700 1 -- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c15db10 msgr2=0x7fa64c0c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa651d9a700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c15db10 0x7fa64c0c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa651d9a700 1 -- 172.21.15.119:0/1042544992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c009420 con 0x7fa64c159c60 2023-10-08T02:28:24.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa652d9c700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c15db10 0x7fa64c0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:24.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa651d9a700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c0cd0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00ec50 tx=0x7fa63c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:24.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa6427fc700 1 -- 172.21.15.119:0/1042544992 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00c850 con 0x7fa64c159c60 2023-10-08T02:28:24.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa6427fc700 1 -- 172.21.15.119:0/1042544992 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c00c9b0 con 0x7fa64c159c60 2023-10-08T02:28:24.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa6427fc700 1 -- 172.21.15.119:0/1042544992 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00cd50 con 0x7fa64c159c60 2023-10-08T02:28:24.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa644009a20 con 0x7fa64c159c60 2023-10-08T02:28:24.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.040+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c169e20 con 0x7fa64c159c60 2023-10-08T02:28:24.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.041+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa64c16a340 con 0x7fa64c159c60 2023-10-08T02:28:24.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.043+0000 7fa6427fc700 1 -- 172.21.15.119:0/1042544992 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fa63c013680 con 0x7fa64c159c60 2023-10-08T02:28:24.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.044+0000 7fa6427fc700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa630067080 0x7fa630069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.044+0000 7fa6427fc700 1 -- 172.21.15.119:0/1042544992 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6802+0+0 (secure 0 0 0) 0x7fa63c084cc0 con 0x7fa64c159c60 2023-10-08T02:28:24.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.044+0000 7fa65259b700 1 -- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa630067080 msgr2=0x7fa630069530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:24.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.044+0000 7fa65259b700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa630067080 0x7fa630069530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:24.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.049+0000 7fa6427fc700 1 -- 172.21.15.119:0/1042544992 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa64c16a340 con 0x7fa64c159c60 2023-10-08T02:28:24.153 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:24.153+0000 7fe381ed3180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:28:24.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.245+0000 7fa65259b700 1 -- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa630067080 msgr2=0x7fa630069530 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:24.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.245+0000 7fa65259b700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa630067080 0x7fa630069530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:24.249 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:24.250+0000 7fe381ed3180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:28:24.263 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:24.263+0000 7ff7a7479180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:28:24.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.295+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa64c0cde10 con 0x7fa64c159c60 2023-10-08T02:28:24.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.295+0000 7fa6427fc700 1 -- 172.21.15.119:0/1042544992 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4607 (secure 0 0 0) 0x7fa63c0519d0 con 0x7fa64c159c60 2023-10-08T02:28:24.296 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:24.296 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7409,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3771852147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3771852147},{"type":"v1","addr":"172.21.15.184:6834","nonce":3771852147}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/524755843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":524755843},{"type":"v1","addr":"172.21.15.184:6835","nonce":524755843}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9144,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/295310163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":295310163},{"type":"v1","addr":"172.21.15.119:6833","nonce":295310163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:12:24.579741+0000","modified":"2023-10-08T02:12:28.133805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9150},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9150":{"gid":9150,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2713045427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2713045427},{"type":"v1","addr":"172.21.15.119:6835","nonce":2713045427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T02:28:24.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa630067080 msgr2=0x7fa630069530 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:24.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa630067080 0x7fa630069530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 msgr2=0x7fa64c0cd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c0cd0c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00ec50 tx=0x7fa63c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 shutdown_connections 2023-10-08T02:28:24.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa630067080 0x7fa630069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa64c15db10 0x7fa64c0c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa64c159890 0x7fa64c16a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 --2- 172.21.15.119:0/1042544992 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa64c159c60 0x7fa64c0cd0c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 >> 172.21.15.119:0/1042544992 conn(0x7fa64c068e20 msgr2=0x7fa64c1570c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:24.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 shutdown_connections 2023-10-08T02:28:24.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.298+0000 7fa6547ff700 1 -- 172.21.15.119:0/1042544992 wait complete. 2023-10-08T02:28:24.300 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 79 2023-10-08T02:28:24.317 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:28:24.342 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:24.342+0000 7fe381ed3180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:28:24.366 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:24.365+0000 7ff7a7479180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:28:24.462 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:24.461+0000 7ff7a7479180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:28:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.504+0000 7f3d0259d700 1 Processor -- start 2023-10-08T02:28:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.504+0000 7f3d0259d700 1 -- start start 2023-10-08T02:28:24.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.505+0000 7f3d0259d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 0x7f3cfc0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d0259d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3cfc15d230 0x7f3cfc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d0259d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cfc15dbd0 con 0x7f3cfc1597f0 2023-10-08T02:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d0259d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b2980 con 0x7f3cfc15d230 2023-10-08T02:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d0259d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc0b11d0 con 0x7f3cfc159420 2023-10-08T02:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d01d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3cfc15d230 0x7f3cfc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d0159b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 0x7f3cfc0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d00d9a700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f3cfc1597f0 0x7f3cfc0b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:42160/0 (socket says 172.21.15.119:42160) 2023-10-08T02:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d00d9a700 1 -- 172.21.15.119:0/1226481987 learned_addr learned my addr 172.21.15.119:0/1226481987 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d01d9c700 1 -- 172.21.15.119:0/1226481987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 msgr2=0x7f3cfc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d01d9c700 1 --2- 172.21.15.119:0/1226481987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 0x7f3cfc0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d01d9c700 1 -- 172.21.15.119:0/1226481987 >> v1:172.21.15.184:6789/0 conn(0x7f3cfc1597f0 legacy=0x7f3cfc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3cf3fff700 1 -- 172.21.15.119:0/1226481987 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868303023 0 0) 0x7f3cec001110 con 0x7f3cfc1597f0 2023-10-08T02:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.506+0000 7f3d01d9c700 1 -- 172.21.15.119:0/1226481987 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc0b14b0 con 0x7f3cfc15d230 2023-10-08T02:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.507+0000 7f3d01d9c700 1 --2- 172.21.15.119:0/1226481987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc15d690 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009e60 tx=0x7f3cf4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=348c2dbc6bed6730 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.507+0000 7f3cf3fff700 1 -- 172.21.15.119:0/1226481987 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf400e070 con 0x7f3cfc15d230 2023-10-08T02:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.507+0000 7f3cf3fff700 1 -- 172.21.15.119:0/1226481987 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf400bb60 con 0x7f3cfc15d230 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.507+0000 7f3cf3fff700 1 -- 172.21.15.119:0/1226481987 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4012430 con 0x7f3cfc15d230 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.508+0000 7f3d0259d700 1 -- 172.21.15.119:0/1226481987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 msgr2=0x7f3cfc15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.508+0000 7f3d0259d700 1 --2- 172.21.15.119:0/1226481987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc15d690 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009e60 tx=0x7f3cf4009780 comp rx=0 tx=0).stop 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.508+0000 7f3d0259d700 1 -- 172.21.15.119:0/1226481987 shutdown_connections 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.508+0000 7f3d0259d700 1 --2- 172.21.15.119:0/1226481987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc15d690 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.508+0000 7f3d0259d700 1 --2- 172.21.15.119:0/1226481987 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 0x7f3cfc0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.508+0000 7f3d0259d700 1 -- 172.21.15.119:0/1226481987 >> 172.21.15.119:0/1226481987 conn(0x7f3cfc0d0660 msgr2=0x7f3cfc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.508+0000 7f3d0259d700 1 -- 172.21.15.119:0/1226481987 shutdown_connections 2023-10-08T02:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.508+0000 7f3d0259d700 1 -- 172.21.15.119:0/1226481987 wait complete. 2023-10-08T02:28:24.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.509+0000 7f3d0259d700 1 Processor -- start 2023-10-08T02:28:24.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.509+0000 7f3d0259d700 1 -- start start 2023-10-08T02:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.510+0000 7f3d0259d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 0x7f3cfc15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.510+0000 7f3d0259d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cfc1597f0 0x7f3cfc152df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.510+0000 7f3d0259d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.510+0000 7f3d0259d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc15bae0 con 0x7f3cfc15d230 2023-10-08T02:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.510+0000 7f3d0259d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc15ac30 con 0x7f3cfc1597f0 2023-10-08T02:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.510+0000 7f3d0259d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cfc16a850 con 0x7f3cfc159420 2023-10-08T02:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50962/0 (socket says 172.21.15.119:50962) 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 -- 172.21.15.119:0/853595401 learned_addr learned my addr 172.21.15.119:0/853595401 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d00d9a700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cfc1597f0 0x7f3cfc152df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d0159b700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 0x7f3cfc15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 -- 172.21.15.119:0/853595401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 msgr2=0x7f3cfc15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 0x7f3cfc15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 -- 172.21.15.119:0/853595401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cfc1597f0 msgr2=0x7f3cfc152df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cfc1597f0 0x7f3cfc152df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 -- 172.21.15.119:0/853595401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf4009a20 con 0x7f3cfc15d230 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.511+0000 7f3d01d9c700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc158240 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc089340 tx=0x7f3cf4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.512+0000 7f3cf17fa700 1 -- 172.21.15.119:0/853595401 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4012ea0 con 0x7f3cfc15d230 2023-10-08T02:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.512+0000 7f3cf17fa700 1 -- 172.21.15.119:0/853595401 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf4027470 con 0x7f3cfc15d230 2023-10-08T02:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.512+0000 7f3cf17fa700 1 -- 172.21.15.119:0/853595401 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf4027830 con 0x7f3cfc15d230 2023-10-08T02:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.512+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc15aed0 con 0x7f3cfc15d230 2023-10-08T02:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.512+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cfc15b490 con 0x7f3cfc15d230 2023-10-08T02:28:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.513+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cfc15bc50 con 0x7f3cfc15d230 2023-10-08T02:28:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.514+0000 7f3cf17fa700 1 -- 172.21.15.119:0/853595401 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f3cf4024070 con 0x7f3cfc15d230 2023-10-08T02:28:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.515+0000 7f3cf17fa700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3ce0067050 0x7f3ce0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.515+0000 7f3cf17fa700 1 -- 172.21.15.119:0/853595401 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6802+0+0 (secure 0 0 0) 0x7f3cf40872f0 con 0x7f3cfc15d230 2023-10-08T02:28:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.515+0000 7f3d0159b700 1 -- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3ce0067050 msgr2=0x7f3ce0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.515+0000 7f3d0159b700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3ce0067050 0x7f3ce0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:24.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.519+0000 7f3cf17fa700 1 -- 172.21.15.119:0/853595401 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cf4053f60 con 0x7f3cfc15d230 2023-10-08T02:28:24.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.716+0000 7f3d0159b700 1 -- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3ce0067050 msgr2=0x7f3ce0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:24.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.716+0000 7f3d0159b700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3ce0067050 0x7f3ce0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.722+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3cfc04c3c0 con 0x7f3cfc15d230 2023-10-08T02:28:24.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.723+0000 7f3cf17fa700 1 -- 172.21.15.119:0/853595401 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+17087 (secure 0 0 0) 0x7f3cf4057580 con 0x7f3cfc15d230 2023-10-08T02:28:24.724 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:24.724 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":177,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:28:21.306718+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:12:22.679007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T02:12:23.548789+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"177","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":32,"snap_epoch":177,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.39","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/506139810":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1047785372":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/932366218":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/3986903627":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1092901654":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1277970358":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/1588612751":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/655693879":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/4078947876":"2023-10-09T02:12:45.254733+0000","172.21.15.119:6836/85511":"2023-10-09T02:12:45.254733+0000","172.21.15.119:6837/85511":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/4064389633":"2023-10-09T02:12:45.254733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":15},{"begin":18,"length":15}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":11,"length":6},{"begin":27,"length":6}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:28:24.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.725+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3ce0067050 msgr2=0x7f3ce0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:24.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.725+0000 7f3d0259d700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3ce0067050 0x7f3ce0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.725+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 msgr2=0x7f3cfc158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.725+0000 7f3d0259d700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc158240 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc089340 tx=0x7f3cf4019000 comp rx=0 tx=0).stop 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.726+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 shutdown_connections 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.726+0000 7f3d0259d700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3ce0067050 0x7f3ce0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.726+0000 7f3d0259d700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3cfc159420 0x7f3cfc15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.726+0000 7f3d0259d700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3cfc15d230 0x7f3cfc158240 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.726+0000 7f3d0259d700 1 --2- 172.21.15.119:0/853595401 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3cfc1597f0 0x7f3cfc152df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.726+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 >> 172.21.15.119:0/853595401 conn(0x7f3cfc0d0660 msgr2=0x7f3cfc1540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.726+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 shutdown_connections 2023-10-08T02:28:24.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.726+0000 7f3d0259d700 1 -- 172.21.15.119:0/853595401 wait complete. 2023-10-08T02:28:24.745 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T02:28:24.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:28:24.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.933+0000 7f69e759e700 1 Processor -- start 2023-10-08T02:28:24.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.933+0000 7f69e759e700 1 -- start start 2023-10-08T02:28:24.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.934+0000 7f69e759e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69e8159420 0x7f69e80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.934+0000 7f69e759e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f69e81597f0 0x7f69e80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.934+0000 7f69e759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e815d230 0x7f69e815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.934+0000 7f69e759e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e815dbb0 con 0x7f69e8159420 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.934+0000 7f69e759e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e80b11d0 con 0x7f69e81597f0 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.934+0000 7f69e759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80b14b0 con 0x7f69e815d230 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.934+0000 7f69e659c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69e8159420 0x7f69e80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e659c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f69e8159420 0x7f69e80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54992/0 (socket says 172.21.15.119:54992) 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e659c700 1 -- 172.21.15.119:0/3215595331 learned_addr learned my addr 172.21.15.119:0/3215595331 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e6d9d700 1 --2- 172.21.15.119:0/3215595331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e815d230 0x7f69e815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e5d9b700 1 --2- 172.21.15.119:0/3215595331 >> v2:172.21.15.184:3300/0 conn(0x7f69e81597f0 0x7f69e80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e659c700 1 -- 172.21.15.119:0/3215595331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e815d230 msgr2=0x7f69e815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:24.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e659c700 1 --2- 172.21.15.119:0/3215595331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e815d230 0x7f69e815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e659c700 1 -- 172.21.15.119:0/3215595331 >> v2:172.21.15.184:3300/0 conn(0x7f69e81597f0 msgr2=0x7f69e80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:24.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e659c700 1 --2- 172.21.15.119:0/3215595331 >> v2:172.21.15.184:3300/0 conn(0x7f69e81597f0 0x7f69e80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e659c700 1 -- 172.21.15.119:0/3215595331 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e80b1790 con 0x7f69e8159420 2023-10-08T02:28:24.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e659c700 1 --2- 172.21.15.119:0/3215595331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e8159420 0x7f69e80b5f40 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f69e8089750 tx=0x7f69d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162818796c1d9800 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:24.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.935+0000 7f69e559a700 1 -- 172.21.15.119:0/3215595331 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800b640 con 0x7f69e8159420 2023-10-08T02:28:24.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e559a700 1 -- 172.21.15.119:0/3215595331 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d800b7a0 con 0x7f69e8159420 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e559a700 1 -- 172.21.15.119:0/3215595331 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d800bb60 con 0x7f69e8159420 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 -- 172.21.15.119:0/3215595331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e8159420 msgr2=0x7f69e80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 --2- 172.21.15.119:0/3215595331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e8159420 0x7f69e80b5f40 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f69e8089750 tx=0x7f69d8009250 comp rx=0 tx=0).stop 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 -- 172.21.15.119:0/3215595331 shutdown_connections 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 --2- 172.21.15.119:0/3215595331 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e815d230 0x7f69e815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 --2- 172.21.15.119:0/3215595331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e8159420 0x7f69e80b5f40 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 --2- 172.21.15.119:0/3215595331 >> v2:172.21.15.184:3300/0 conn(0x7f69e81597f0 0x7f69e80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 -- 172.21.15.119:0/3215595331 >> 172.21.15.119:0/3215595331 conn(0x7f69e80d0660 msgr2=0x7f69e80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 -- 172.21.15.119:0/3215595331 shutdown_connections 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.936+0000 7f69e759e700 1 -- 172.21.15.119:0/3215595331 wait complete. 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.938+0000 7f69e759e700 1 Processor -- start 2023-10-08T02:28:24.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.938+0000 7f69e759e700 1 -- start start 2023-10-08T02:28:24.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e759e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e8159420 0x7f69e80c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e759e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e81597f0 0x7f69e816c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e759e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e815d230 0x7f69e816d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e759e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e81699b0 con 0x7f69e815d230 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e759e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e816de50 con 0x7f69e81597f0 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e759e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e816a500 con 0x7f69e8159420 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e6d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e815d230 0x7f69e816d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e659c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e8159420 0x7f69e80c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e6d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e815d230 0x7f69e816d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55006/0 (socket says 172.21.15.119:55006) 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e6d9d700 1 -- 172.21.15.119:0/2328424053 learned_addr learned my addr 172.21.15.119:0/2328424053 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:24.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e5d9b700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e81597f0 0x7f69e816c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e6d9d700 1 -- 172.21.15.119:0/2328424053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e8159420 msgr2=0x7f69e80c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e6d9d700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e8159420 0x7f69e80c5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e6d9d700 1 -- 172.21.15.119:0/2328424053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e81597f0 msgr2=0x7f69e816c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e6d9d700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e81597f0 0x7f69e816c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.939+0000 7f69e6d9d700 1 -- 172.21.15.119:0/2328424053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d80094f0 con 0x7f69e815d230 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.940+0000 7f69e6d9d700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e815d230 0x7f69e816d910 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009df0 tx=0x7f69e000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.940+0000 7f69d67fc700 1 -- 172.21.15.119:0/2328424053 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000caa0 con 0x7f69e815d230 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.940+0000 7f69d67fc700 1 -- 172.21.15.119:0/2328424053 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e000f040 con 0x7f69e815d230 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.940+0000 7f69d67fc700 1 -- 172.21.15.119:0/2328424053 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000ce70 con 0x7f69e815d230 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.940+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e80c6480 con 0x7f69e815d230 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.940+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e8169eb0 con 0x7f69e815d230 2023-10-08T02:28:24.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.941+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e816a670 con 0x7f69e815d230 2023-10-08T02:28:24.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.942+0000 7f69d67fc700 1 -- 172.21.15.119:0/2328424053 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f69e0014690 con 0x7f69e815d230 2023-10-08T02:28:24.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.943+0000 7f69d67fc700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f69c8067050 0x7f69c8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:24.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.943+0000 7f69d67fc700 1 -- 172.21.15.119:0/2328424053 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6802+0+0 (secure 0 0 0) 0x7f69e00857b0 con 0x7f69e815d230 2023-10-08T02:28:24.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.943+0000 7f69e659c700 1 -- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f69c8067050 msgr2=0x7f69c8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:24.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.943+0000 7f69e659c700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f69c8067050 0x7f69c8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:24.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:24.947+0000 7f69d67fc700 1 -- 172.21.15.119:0/2328424053 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f69e0055e40 con 0x7f69e815d230 2023-10-08T02:28:24.969 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:24.969+0000 7fe381ed3180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:28:25.136 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:25.135+0000 7ff7a7479180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:28:25.142 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:25.142+0000 7fe381ed3180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:28:25.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.144+0000 7f69e659c700 1 -- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f69c8067050 msgr2=0x7f69c8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:25.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.144+0000 7f69e659c700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f69c8067050 0x7f69c8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:25.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.232+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69e804c3c0 con 0x7f69e815d230 2023-10-08T02:28:25.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.232+0000 7f69d67fc700 1 -- 172.21.15.119:0/2328424053 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4607 (secure 0 0 0) 0x7f69e00559d0 con 0x7f69e815d230 2023-10-08T02:28:25.233 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:25.233 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7409,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3771852147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3771852147},{"type":"v1","addr":"172.21.15.184:6834","nonce":3771852147}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/524755843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":524755843},{"type":"v1","addr":"172.21.15.184:6835","nonce":524755843}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9144,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/295310163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":295310163},{"type":"v1","addr":"172.21.15.119:6833","nonce":295310163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:12:24.579741+0000","modified":"2023-10-08T02:12:28.133805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9150},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9150":{"gid":9150,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2713045427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2713045427},{"type":"v1","addr":"172.21.15.119:6835","nonce":2713045427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T02:28:25.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f69c8067050 msgr2=0x7f69c8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:25.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f69c8067050 0x7f69c8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e815d230 msgr2=0x7f69e816d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e815d230 0x7f69e816d910 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009df0 tx=0x7f69e000b040 comp rx=0 tx=0).stop 2023-10-08T02:28:25.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 shutdown_connections 2023-10-08T02:28:25.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f69c8067050 0x7f69c8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69e8159420 0x7f69e80c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69e815d230 0x7f69e816d910 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 --2- 172.21.15.119:0/2328424053 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f69e81597f0 0x7f69e816c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 >> 172.21.15.119:0/2328424053 conn(0x7f69e80d0660 msgr2=0x7f69e8069be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 shutdown_connections 2023-10-08T02:28:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.235+0000 7f69e759e700 1 -- 172.21.15.119:0/2328424053 wait complete. 2023-10-08T02:28:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 79 2023-10-08T02:28:25.241 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:25.241+0000 7fe381ed3180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:28:25.253 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:28:25.334 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:25.333+0000 7ff7a7479180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:28:25.436 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:25.436+0000 7ff7a7479180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:28:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.457+0000 7ffbaf59d700 1 Processor -- start 2023-10-08T02:28:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.457+0000 7ffbaf59d700 1 -- start start 2023-10-08T02:28:25.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbaf59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba81597f0 0x7ffba80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbaf59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ffba815d230 0x7ffba815d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbaf59d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffba815dbd0 con 0x7ffba8159420 2023-10-08T02:28:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbaf59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffba80b11d0 con 0x7ffba815d230 2023-10-08T02:28:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbaf59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba80b14b0 con 0x7ffba81597f0 2023-10-08T02:28:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbae59b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffba8159420 0x7ffba80b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51088/0 (socket says 172.21.15.119:51088) 2023-10-08T02:28:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbae59b700 1 -- 172.21.15.119:0/3199823652 learned_addr learned my addr 172.21.15.119:0/3199823652 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbadd9a700 1 --2- 172.21.15.119:0/3199823652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba81597f0 0x7ffba80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbaed9c700 1 --2- 172.21.15.119:0/3199823652 >> v2:172.21.15.184:3300/0 conn(0x7ffba815d230 0x7ffba815d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.459+0000 7ffbad599700 1 -- 172.21.15.119:0/3199823652 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 126298009 0 0) 0x7ffba4001110 con 0x7ffba8159420 2023-10-08T02:28:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbad599700 1 -- 172.21.15.119:0/3199823652 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb94003570 con 0x7ffba8159420 2023-10-08T02:28:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbadd9a700 1 -- 172.21.15.119:0/3199823652 >> v2:172.21.15.184:3300/0 conn(0x7ffba815d230 msgr2=0x7ffba815d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbadd9a700 1 --2- 172.21.15.119:0/3199823652 >> v2:172.21.15.184:3300/0 conn(0x7ffba815d230 0x7ffba815d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbadd9a700 1 -- 172.21.15.119:0/3199823652 >> v1:172.21.15.119:6789/0 conn(0x7ffba8159420 legacy=0x7ffba80b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbadd9a700 1 -- 172.21.15.119:0/3199823652 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba80b1790 con 0x7ffba81597f0 2023-10-08T02:28:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbaed9c700 1 --2- 172.21.15.119:0/3199823652 >> v2:172.21.15.184:3300/0 conn(0x7ffba815d230 0x7ffba815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:25.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbadd9a700 1 --2- 172.21.15.119:0/3199823652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba81597f0 0x7ffba80b2270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb980098e0 tx=0x7ffb980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33767bad818a40bd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbad599700 1 -- 172.21.15.119:0/3199823652 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98011630 con 0x7ffba81597f0 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbad599700 1 -- 172.21.15.119:0/3199823652 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb98011790 con 0x7ffba81597f0 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.460+0000 7ffbad599700 1 -- 172.21.15.119:0/3199823652 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb98011b30 con 0x7ffba81597f0 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.461+0000 7ffbaf59d700 1 -- 172.21.15.119:0/3199823652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba81597f0 msgr2=0x7ffba80b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.461+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/3199823652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba81597f0 0x7ffba80b2270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffb980098e0 tx=0x7ffb980091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.461+0000 7ffbaf59d700 1 -- 172.21.15.119:0/3199823652 shutdown_connections 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.461+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/3199823652 >> v2:172.21.15.184:3300/0 conn(0x7ffba815d230 0x7ffba815d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.461+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/3199823652 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba81597f0 0x7ffba80b2270 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.461+0000 7ffbaf59d700 1 -- 172.21.15.119:0/3199823652 >> 172.21.15.119:0/3199823652 conn(0x7ffba80d0660 msgr2=0x7ffba80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.461+0000 7ffbaf59d700 1 -- 172.21.15.119:0/3199823652 shutdown_connections 2023-10-08T02:28:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.461+0000 7ffbaf59d700 1 -- 172.21.15.119:0/3199823652 wait complete. 2023-10-08T02:28:25.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.462+0000 7ffbaf59d700 1 Processor -- start 2023-10-08T02:28:25.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.463+0000 7ffbaf59d700 1 -- start start 2023-10-08T02:28:25.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.463+0000 7ffbaf59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba8159420 0x7ffba815c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.463+0000 7ffbaf59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba81597f0 0x7ffba80cb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.463+0000 7ffbaf59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba815d230 0x7ffba80c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.463+0000 7ffbaf59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba815c740 con 0x7ffba815d230 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.463+0000 7ffbaf59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba815bd70 con 0x7ffba81597f0 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.463+0000 7ffbaf59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba815b160 con 0x7ffba8159420 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba815d230 0x7ffba80c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbae59b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba8159420 0x7ffba815c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbadd9a700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba81597f0 0x7ffba80cb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba815d230 0x7ffba80c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55012/0 (socket says 172.21.15.119:55012) 2023-10-08T02:28:25.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 -- 172.21.15.119:0/579674373 learned_addr learned my addr 172.21.15.119:0/579674373 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 -- 172.21.15.119:0/579674373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba8159420 msgr2=0x7ffba815c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba8159420 0x7ffba815c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 -- 172.21.15.119:0/579674373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba81597f0 msgr2=0x7ffba80cb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba81597f0 0x7ffba80cb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 -- 172.21.15.119:0/579674373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba0009a20 con 0x7ffba815d230 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbae59b700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba8159420 0x7ffba815c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbadd9a700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba81597f0 0x7ffba80cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.464+0000 7ffbaed9c700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba815d230 0x7ffba80c8280 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009f20 tx=0x7ffba000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.465+0000 7ffb9e7fc700 1 -- 172.21.15.119:0/579674373 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000cb70 con 0x7ffba815d230 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.465+0000 7ffb9e7fc700 1 -- 172.21.15.119:0/579674373 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000f040 con 0x7ffba815d230 2023-10-08T02:28:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.465+0000 7ffb9e7fc700 1 -- 172.21.15.119:0/579674373 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba0014480 con 0x7ffba815d230 2023-10-08T02:28:25.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.465+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb98009420 con 0x7ffba815d230 2023-10-08T02:28:25.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.465+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffba815b720 con 0x7ffba815d230 2023-10-08T02:28:25.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.466+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffba815bee0 con 0x7ffba815d230 2023-10-08T02:28:25.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.467+0000 7ffb9e7fc700 1 -- 172.21.15.119:0/579674373 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7ffba0003020 con 0x7ffba815d230 2023-10-08T02:28:25.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.468+0000 7ffb9e7fc700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ffb8c067050 0x7ffb8c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.469+0000 7ffb9e7fc700 1 -- 172.21.15.119:0/579674373 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6786+0+0 (secure 0 0 0) 0x7ffba00854c0 con 0x7ffba815d230 2023-10-08T02:28:25.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.469+0000 7ffbae59b700 1 -- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ffb8c067050 msgr2=0x7ffb8c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:25.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.469+0000 7ffbae59b700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ffb8c067050 0x7ffb8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.473+0000 7ffb9e7fc700 1 -- 172.21.15.119:0/579674373 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffba0052030 con 0x7ffba815d230 2023-10-08T02:28:25.543 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:25.542+0000 7fe381ed3180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:28:25.638 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:25.638+0000 7fe381ed3180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:28:25.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.670+0000 7ffbae59b700 1 -- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ffb8c067050 msgr2=0x7ffb8c069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:25.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.670+0000 7ffbae59b700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ffb8c067050 0x7ffb8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:25.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.716+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffba80b1a70 con 0x7ffba815d230 2023-10-08T02:28:25.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.716+0000 7ffb9e7fc700 1 -- 172.21.15.119:0/579674373 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4607 (secure 0 0 0) 0x7ffba0055650 con 0x7ffba815d230 2023-10-08T02:28:25.717 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:25.717 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7409,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3771852147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3771852147},{"type":"v1","addr":"172.21.15.184:6834","nonce":3771852147}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/524755843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":524755843},{"type":"v1","addr":"172.21.15.184:6835","nonce":524755843}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9144,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/295310163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":295310163},{"type":"v1","addr":"172.21.15.119:6833","nonce":295310163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:12:24.579741+0000","modified":"2023-10-08T02:12:28.133805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9150},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9150":{"gid":9150,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/2713045427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2713045427},{"type":"v1","addr":"172.21.15.119:6835","nonce":2713045427}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-10-08T02:28:25.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ffb8c067050 msgr2=0x7ffb8c069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:25.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ffb8c067050 0x7ffb8c069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba815d230 msgr2=0x7ffba80c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba815d230 0x7ffba80c8280 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ffba0009f20 tx=0x7ffba000b040 comp rx=0 tx=0).stop 2023-10-08T02:28:25.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 shutdown_connections 2023-10-08T02:28:25.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ffb8c067050 0x7ffb8c069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ffba8159420 0x7ffba815c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffba815d230 0x7ffba80c8280 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 --2- 172.21.15.119:0/579674373 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ffba81597f0 0x7ffba80cb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 >> 172.21.15.119:0/579674373 conn(0x7ffba80d0660 msgr2=0x7ffba8156020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:25.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 shutdown_connections 2023-10-08T02:28:25.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.719+0000 7ffbaf59d700 1 -- 172.21.15.119:0/579674373 wait complete. 2023-10-08T02:28:25.721 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 79 2023-10-08T02:28:25.739 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:28:25.744 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:25.743+0000 7ff7a7479180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:28:25.847 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:25.847+0000 7ff7a7479180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:28:25.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.927+0000 7fe691b9d700 1 Processor -- start 2023-10-08T02:28:25.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.927+0000 7fe691b9d700 1 -- start start 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.929+0000 7fe691b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0c8830 0x7fe68c164800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.929+0000 7fe691b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c0b1410 con 0x7fe68c1595a0 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.929+0000 7fe691b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe68c063230 con 0x7fe68c0c6c60 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.929+0000 7fe691b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c0b2b90 con 0x7fe68c0c8830 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.929+0000 7fe690b9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe68c1595a0 0x7fe68c0b18a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51096/0 (socket says 172.21.15.119:51096) 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.929+0000 7fe690b9b700 1 -- 172.21.15.119:0/3470694838 learned_addr learned my addr 172.21.15.119:0/3470694838 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.929+0000 7fe69139c700 1 --2- 172.21.15.119:0/3470694838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0c8830 0x7fe68c164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe68b7fe700 1 -- 172.21.15.119:0/3470694838 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913826163 0 0) 0x7fe68c0b1410 con 0x7fe68c1595a0 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe68b7fe700 1 -- 172.21.15.119:0/3470694838 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe67c003570 con 0x7fe68c1595a0 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe68b7fe700 1 -- 172.21.15.119:0/3470694838 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088967666 0 0) 0x7fe68c063230 con 0x7fe68c0c6c60 2023-10-08T02:28:25.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe68b7fe700 1 -- 172.21.15.119:0/3470694838 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe68c0b1410 con 0x7fe68c0c6c60 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe69139c700 1 -- 172.21.15.119:0/3470694838 >> v1:172.21.15.184:6789/0 conn(0x7fe68c0c6c60 legacy=0x7fe68c0c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe69139c700 1 -- 172.21.15.119:0/3470694838 >> v1:172.21.15.119:6789/0 conn(0x7fe68c1595a0 legacy=0x7fe68c0b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe69139c700 1 -- 172.21.15.119:0/3470694838 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c0634f0 con 0x7fe68c0c8830 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe69139c700 1 --2- 172.21.15.119:0/3470694838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0c8830 0x7fe68c164800 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe684009e90 tx=0x7fe684009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f554778f23a583d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe68b7fe700 1 -- 172.21.15.119:0/3470694838 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400e070 con 0x7fe68c0c8830 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe68b7fe700 1 -- 172.21.15.119:0/3470694838 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400bb80 con 0x7fe68c0c8830 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.930+0000 7fe68b7fe700 1 -- 172.21.15.119:0/3470694838 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe684012430 con 0x7fe68c0c8830 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.931+0000 7fe691b9d700 1 -- 172.21.15.119:0/3470694838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0c8830 msgr2=0x7fe68c164800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.931+0000 7fe691b9d700 1 --2- 172.21.15.119:0/3470694838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0c8830 0x7fe68c164800 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe684009e90 tx=0x7fe684009780 comp rx=0 tx=0).stop 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.931+0000 7fe691b9d700 1 -- 172.21.15.119:0/3470694838 shutdown_connections 2023-10-08T02:28:25.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.931+0000 7fe691b9d700 1 --2- 172.21.15.119:0/3470694838 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c0c8830 0x7fe68c164800 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.931+0000 7fe691b9d700 1 -- 172.21.15.119:0/3470694838 >> 172.21.15.119:0/3470694838 conn(0x7fe68c06ddb0 msgr2=0x7fe68c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:25.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.931+0000 7fe691b9d700 1 -- 172.21.15.119:0/3470694838 shutdown_connections 2023-10-08T02:28:25.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.931+0000 7fe691b9d700 1 -- 172.21.15.119:0/3470694838 wait complete. 2023-10-08T02:28:25.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.932+0000 7fe691b9d700 1 Processor -- start 2023-10-08T02:28:25.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.933+0000 7fe691b9d700 1 -- start start 2023-10-08T02:28:25.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.933+0000 7fe691b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0c6c60 0x7fe68c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe691b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0c8830 0x7fe68c169b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe691b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c1595a0 0x7fe68c0c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe691b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c15c870 con 0x7fe68c0c6c60 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe691b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c16abb0 con 0x7fe68c0c8830 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe691b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c1520e0 con 0x7fe68c1595a0 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0c6c60 0x7fe68c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0c6c60 0x7fe68c151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55026/0 (socket says 172.21.15.119:55026) 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 -- 172.21.15.119:0/612478149 learned_addr learned my addr 172.21.15.119:0/612478149 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe68bfff700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0c8830 0x7fe68c169b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe69139c700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c1595a0 0x7fe68c0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 -- 172.21.15.119:0/612478149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c1595a0 msgr2=0x7fe68c0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c1595a0 0x7fe68c0c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 -- 172.21.15.119:0/612478149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0c8830 msgr2=0x7fe68c169b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0c8830 0x7fe68c169b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 -- 172.21.15.119:0/612478149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe684009a20 con 0x7fe68c0c6c60 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.934+0000 7fe690b9b700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0c6c60 0x7fe68c151ba0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fe68000c790 tx=0x7fe680007ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.935+0000 7fe688ff9700 1 -- 172.21.15.119:0/612478149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680005b30 con 0x7fe68c0c6c60 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.935+0000 7fe688ff9700 1 -- 172.21.15.119:0/612478149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680005c90 con 0x7fe68c0c6c60 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.935+0000 7fe688ff9700 1 -- 172.21.15.119:0/612478149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680007300 con 0x7fe68c0c6c60 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.935+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68c0c5910 con 0x7fe68c0c6c60 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.935+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68c16a310 con 0x7fe68c0c6c60 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.936+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe68c04c3c0 con 0x7fe68c0c6c60 2023-10-08T02:28:25.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.937+0000 7fe688ff9700 1 -- 172.21.15.119:0/612478149 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fe68001b3b0 con 0x7fe68c0c6c60 2023-10-08T02:28:25.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.937+0000 7fe688ff9700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe670067050 0x7fe670069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:25.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.938+0000 7fe688ff9700 1 -- 172.21.15.119:0/612478149 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6786+0+0 (secure 0 0 0) 0x7fe680083fc0 con 0x7fe68c0c6c60 2023-10-08T02:28:25.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.938+0000 7fe68bfff700 1 -- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe670067050 msgr2=0x7fe670069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:25.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.938+0000 7fe68bfff700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe670067050 0x7fe670069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:25.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:25.941+0000 7fe688ff9700 1 -- 172.21.15.119:0/612478149 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6800546d0 con 0x7fe68c0c6c60 2023-10-08T02:28:26.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.139+0000 7fe68bfff700 1 -- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe670067050 msgr2=0x7fe670069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:26.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.139+0000 7fe68bfff700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe670067050 0x7fe670069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:26.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.144+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe68c0637d0 con 0x7fe68c0c6c60 2023-10-08T02:28:26.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.145+0000 7fe688ff9700 1 -- 172.21.15.119:0/612478149 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+17062 (secure 0 0 0) 0x7fe680017020 con 0x7fe68c0c6c60 2023-10-08T02:28:26.146 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:26.146 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":178,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:28:25.318494+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:12:22.679007+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-10-08T02:12:23.548789+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":178,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.39","mappings":[{"from":4,"to":3}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/506139810":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1047785372":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/932366218":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/3986903627":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1092901654":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/1277970358":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/1588612751":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/655693879":"2023-10-09T02:23:22.994564+0000","172.21.15.119:0/4078947876":"2023-10-09T02:12:45.254733+0000","172.21.15.119:6836/85511":"2023-10-09T02:12:45.254733+0000","172.21.15.119:6837/85511":"2023-10-09T02:12:45.254733+0000","172.21.15.119:0/4064389633":"2023-10-09T02:12:45.254733+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:28:26.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe670067050 msgr2=0x7fe670069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:26.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe670067050 0x7fe670069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0c6c60 msgr2=0x7fe68c151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0c6c60 0x7fe68c151ba0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fe68000c790 tx=0x7fe680007ea0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 shutdown_connections 2023-10-08T02:28:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe670067050 0x7fe670069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe68c1595a0 0x7fe68c0c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe68c0c6c60 0x7fe68c151ba0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 --2- 172.21.15.119:0/612478149 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe68c0c8830 0x7fe68c169b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 >> 172.21.15.119:0/612478149 conn(0x7fe68c06ddb0 msgr2=0x7fe68c153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.147+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 shutdown_connections 2023-10-08T02:28:26.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.148+0000 7fe691b9d700 1 -- 172.21.15.119:0/612478149 wait complete. 2023-10-08T02:28:26.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T02:28:26.351 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:26.351+0000 7fe381ed3180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:28:26.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.357+0000 7fb67f59e700 1 Processor -- start 2023-10-08T02:28:26.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.358+0000 7fb67f59e700 1 -- start start 2023-10-08T02:28:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67f59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb680159420 0x7fb6800b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67f59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb68015d230 0x7fb68015d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67f59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6800b2980 con 0x7fb6801597f0 2023-10-08T02:28:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67f59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb68015db70 con 0x7fb68015d230 2023-10-08T02:28:26.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67f59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800b11d0 con 0x7fb680159420 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67ed9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb68015d230 0x7fb68015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67dd9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb6801597f0 0x7fb6800b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51104/0 (socket says 172.21.15.119:51104) 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67dd9b700 1 -- 172.21.15.119:0/46333844 learned_addr learned my addr 172.21.15.119:0/46333844 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67e59c700 1 --2- 172.21.15.119:0/46333844 >> v2:172.21.15.184:3300/0 conn(0x7fb680159420 0x7fb6800b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67ed9d700 1 -- 172.21.15.119:0/46333844 >> v2:172.21.15.184:3300/0 conn(0x7fb680159420 msgr2=0x7fb6800b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.359+0000 7fb67ed9d700 1 --2- 172.21.15.119:0/46333844 >> v2:172.21.15.184:3300/0 conn(0x7fb680159420 0x7fb6800b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.360+0000 7fb67ed9d700 1 -- 172.21.15.119:0/46333844 >> v1:172.21.15.119:6789/0 conn(0x7fb6801597f0 legacy=0x7fb6800b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.360+0000 7fb67ed9d700 1 -- 172.21.15.119:0/46333844 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6800b14b0 con 0x7fb68015d230 2023-10-08T02:28:26.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.360+0000 7fb67ed9d700 1 --2- 172.21.15.119:0/46333844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb68015d230 0x7fb68015d630 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fb6800b2f20 tx=0x7fb678009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6638f066af7ba23a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.360+0000 7fb67d59a700 1 -- 172.21.15.119:0/46333844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800e070 con 0x7fb68015d230 2023-10-08T02:28:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.360+0000 7fb67d59a700 1 -- 172.21.15.119:0/46333844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67800bb60 con 0x7fb68015d230 2023-10-08T02:28:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.360+0000 7fb67d59a700 1 -- 172.21.15.119:0/46333844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678012430 con 0x7fb68015d230 2023-10-08T02:28:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.361+0000 7fb67f59e700 1 -- 172.21.15.119:0/46333844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb68015d230 msgr2=0x7fb68015d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.361+0000 7fb67f59e700 1 --2- 172.21.15.119:0/46333844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb68015d230 0x7fb68015d630 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fb6800b2f20 tx=0x7fb678009780 comp rx=0 tx=0).stop 2023-10-08T02:28:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.361+0000 7fb67f59e700 1 -- 172.21.15.119:0/46333844 shutdown_connections 2023-10-08T02:28:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.361+0000 7fb67f59e700 1 --2- 172.21.15.119:0/46333844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb68015d230 0x7fb68015d630 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.361+0000 7fb67f59e700 1 --2- 172.21.15.119:0/46333844 >> v2:172.21.15.184:3300/0 conn(0x7fb680159420 0x7fb6800b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.361+0000 7fb67f59e700 1 -- 172.21.15.119:0/46333844 >> 172.21.15.119:0/46333844 conn(0x7fb6800d0660 msgr2=0x7fb6800d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.361+0000 7fb67f59e700 1 -- 172.21.15.119:0/46333844 shutdown_connections 2023-10-08T02:28:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.361+0000 7fb67f59e700 1 -- 172.21.15.119:0/46333844 wait complete. 2023-10-08T02:28:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.362+0000 7fb67f59e700 1 Processor -- start 2023-10-08T02:28:26.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.362+0000 7fb67f59e700 1 -- start start 2023-10-08T02:28:26.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.363+0000 7fb67f59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb680159420 0x7fb68015c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.363+0000 7fb67f59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6801597f0 0x7fb6800b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.363+0000 7fb67f59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb68015d230 0x7fb680154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.363+0000 7fb67f59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6800b5ea0 con 0x7fb680159420 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.363+0000 7fb67f59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68015bdc0 con 0x7fb68015d230 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.363+0000 7fb67f59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb68015cb70 con 0x7fb6801597f0 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.363+0000 7fb67e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb680159420 0x7fb68015c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb680159420 0x7fb68015c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55050/0 (socket says 172.21.15.119:55050) 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67e59c700 1 -- 172.21.15.119:0/1696672243 learned_addr learned my addr 172.21.15.119:0/1696672243 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67ed9d700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb68015d230 0x7fb680154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67dd9b700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6801597f0 0x7fb6800b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67e59c700 1 -- 172.21.15.119:0/1696672243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6801597f0 msgr2=0x7fb6800b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67e59c700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6801597f0 0x7fb6800b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67e59c700 1 -- 172.21.15.119:0/1696672243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb68015d230 msgr2=0x7fb680154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67e59c700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb68015d230 0x7fb680154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67e59c700 1 -- 172.21.15.119:0/1696672243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678009a20 con 0x7fb680159420 2023-10-08T02:28:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb67e59c700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb680159420 0x7fb68015c630 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fb67000ecb0 tx=0x7fb6700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.364+0000 7fb66e7fc700 1 -- 172.21.15.119:0/1696672243 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000c850 con 0x7fb680159420 2023-10-08T02:28:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.365+0000 7fb66e7fc700 1 -- 172.21.15.119:0/1696672243 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67000c9b0 con 0x7fb680159420 2023-10-08T02:28:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.365+0000 7fb66e7fc700 1 -- 172.21.15.119:0/1696672243 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000cd50 con 0x7fb680159420 2023-10-08T02:28:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.365+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68015b1b0 con 0x7fb680159420 2023-10-08T02:28:26.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.365+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68015ba50 con 0x7fb680159420 2023-10-08T02:28:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.366+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb68004c3c0 con 0x7fb680159420 2023-10-08T02:28:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.367+0000 7fb66e7fc700 1 -- 172.21.15.119:0/1696672243 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fb670013680 con 0x7fb680159420 2023-10-08T02:28:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.367+0000 7fb66e7fc700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb660067050 0x7fb660069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.368+0000 7fb66e7fc700 1 -- 172.21.15.119:0/1696672243 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6786+0+0 (secure 0 0 0) 0x7fb670084de0 con 0x7fb680159420 2023-10-08T02:28:26.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.368+0000 7fb67dd9b700 1 -- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb660067050 msgr2=0x7fb660069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:26.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.368+0000 7fb67dd9b700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb660067050 0x7fb660069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:26.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.371+0000 7fb66e7fc700 1 -- 172.21.15.119:0/1696672243 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb670051a60 con 0x7fb680159420 2023-10-08T02:28:26.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.569+0000 7fb67dd9b700 1 -- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb660067050 msgr2=0x7fb660069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:26.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.569+0000 7fb67dd9b700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb660067050 0x7fb660069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:26.612 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:26.611+0000 7ff7a7479180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:28:26.638 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:26.638+0000 7fe381ed3180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:28:26.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.643+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb6800b1790 con 0x7fb680159420 2023-10-08T02:28:26.644 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T02:28:26.644+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T02:28:26.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.648+0000 7fb66e7fc700 1 -- 172.21.15.119:0/1696672243 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb670055080 con 0x7fb680159420 2023-10-08T02:28:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb660067050 msgr2=0x7fb660069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb660067050 0x7fb660069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb680159420 msgr2=0x7fb68015c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb680159420 0x7fb68015c630 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fb67000ecb0 tx=0x7fb6700091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 shutdown_connections 2023-10-08T02:28:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb660067050 0x7fb660069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb68015d230 0x7fb680154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb680159420 0x7fb68015c630 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 --2- 172.21.15.119:0/1696672243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb6801597f0 0x7fb6800b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 >> 172.21.15.119:0/1696672243 conn(0x7fb6800d0660 msgr2=0x7fb680156a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:26.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 shutdown_connections 2023-10-08T02:28:26.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.651+0000 7fb67f59e700 1 -- 172.21.15.119:0/1696672243 wait complete. 2023-10-08T02:28:26.653 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T02:28:26.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T02:28:26.709 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T02:28:26.736 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:26.736+0000 7fe381ed3180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:28:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.865+0000 7f89ac99d700 1 Processor -- start 2023-10-08T02:28:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.865+0000 7f89ac99d700 1 -- start start 2023-10-08T02:28:26.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.866+0000 7f89ac99d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89a815a3b0 0x7f89a815a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.866+0000 7f89ac99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a806e470 0x7f89a806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.866+0000 7f89ac99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a80aea70 con 0x7f89a80aef00 2023-10-08T02:28:26.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.866+0000 7f89ac99d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a806ee30 con 0x7f89a815a3b0 2023-10-08T02:28:26.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.866+0000 7f89ac99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a815dad0 con 0x7f89a806e470 2023-10-08T02:28:26.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.866+0000 7f89a77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89a815a3b0 0x7f89a815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89a815a3b0 0x7f89a815a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55058/0 (socket says 172.21.15.119:55058) 2023-10-08T02:28:26.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a77fe700 1 -- 172.21.15.119:0/3589740435 learned_addr learned my addr 172.21.15.119:0/3589740435 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:26.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a7fff700 1 --2- 172.21.15.119:0/3589740435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a806e470 0x7f89a806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a77fe700 1 -- 172.21.15.119:0/3589740435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a806e470 msgr2=0x7f89a806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a77fe700 1 --2- 172.21.15.119:0/3589740435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a806e470 0x7f89a806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a77fe700 1 -- 172.21.15.119:0/3589740435 >> v1:172.21.15.184:6789/0 conn(0x7f89a80aef00 legacy=0x7f89a815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a77fe700 1 -- 172.21.15.119:0/3589740435 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a80ae480 con 0x7f89a815a3b0 2023-10-08T02:28:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a77fe700 1 --2- 172.21.15.119:0/3589740435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a815a3b0 0x7f89a815a780 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f89980099b0 tx=0x7f8998009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d664f98b24e5af95 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.867+0000 7f89a67fc700 1 -- 172.21.15.119:0/3589740435 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899800b860 con 0x7f89a815a3b0 2023-10-08T02:28:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89a67fc700 1 -- 172.21.15.119:0/3589740435 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899800b9c0 con 0x7f89a815a3b0 2023-10-08T02:28:26.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89a67fc700 1 -- 172.21.15.119:0/3589740435 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899800bd60 con 0x7f89a815a3b0 2023-10-08T02:28:26.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89ac99d700 1 -- 172.21.15.119:0/3589740435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a815a3b0 msgr2=0x7f89a815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89ac99d700 1 --2- 172.21.15.119:0/3589740435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a815a3b0 0x7f89a815a780 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f89980099b0 tx=0x7f8998009250 comp rx=0 tx=0).stop 2023-10-08T02:28:26.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89ac99d700 1 -- 172.21.15.119:0/3589740435 shutdown_connections 2023-10-08T02:28:26.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89ac99d700 1 --2- 172.21.15.119:0/3589740435 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a806e470 0x7f89a806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89ac99d700 1 --2- 172.21.15.119:0/3589740435 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a815a3b0 0x7f89a815a780 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89ac99d700 1 -- 172.21.15.119:0/3589740435 >> 172.21.15.119:0/3589740435 conn(0x7f89a80c9f70 msgr2=0x7f89a806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:26.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89ac99d700 1 -- 172.21.15.119:0/3589740435 shutdown_connections 2023-10-08T02:28:26.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.868+0000 7f89ac99d700 1 -- 172.21.15.119:0/3589740435 wait complete. 2023-10-08T02:28:26.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.870+0000 7f89ac99d700 1 Processor -- start 2023-10-08T02:28:26.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.870+0000 7f89ac99d700 1 -- start start 2023-10-08T02:28:26.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89ac99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a806e470 0x7f89a81543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89ac99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a80aef00 0x7f89a81548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89ac99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a815a3b0 0x7f89a80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89ac99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c5400 con 0x7f89a80aef00 2023-10-08T02:28:26.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89ac99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c47f0 con 0x7f89a806e470 2023-10-08T02:28:26.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89ac99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80c4ad0 con 0x7f89a815a3b0 2023-10-08T02:28:26.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a80aef00 0x7f89a81548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a80aef00 0x7f89a81548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55070/0 (socket says 172.21.15.119:55070) 2023-10-08T02:28:26.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a6ffd700 1 -- 172.21.15.119:0/2066507295 learned_addr learned my addr 172.21.15.119:0/2066507295 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:26.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a77fe700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a806e470 0x7f89a81543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a7fff700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a815a3b0 0x7f89a80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:26.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a6ffd700 1 -- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a815a3b0 msgr2=0x7f89a80b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a6ffd700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a815a3b0 0x7f89a80b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a6ffd700 1 -- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a806e470 msgr2=0x7f89a81543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:26.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a6ffd700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a806e470 0x7f89a81543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:26.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a6ffd700 1 -- 172.21.15.119:0/2066507295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980094f0 con 0x7f89a80aef00 2023-10-08T02:28:26.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.871+0000 7f89a7fff700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a815a3b0 0x7f89a80b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:26.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.872+0000 7f89a6ffd700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a80aef00 0x7f89a81548e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c590 tx=0x7f899c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:26.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.872+0000 7f898f7fe700 1 -- 172.21.15.119:0/2066507295 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c011590 con 0x7f89a80aef00 2023-10-08T02:28:26.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.872+0000 7f898f7fe700 1 -- 172.21.15.119:0/2066507295 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c011710 con 0x7f89a80aef00 2023-10-08T02:28:26.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.872+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a80c4db0 con 0x7f89a80aef00 2023-10-08T02:28:26.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.872+0000 7f898f7fe700 1 -- 172.21.15.119:0/2066507295 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c011ad0 con 0x7f89a80aef00 2023-10-08T02:28:26.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.872+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a80b1b00 con 0x7f89a80aef00 2023-10-08T02:28:26.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.873+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a80c55b0 con 0x7f89a80aef00 2023-10-08T02:28:26.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.874+0000 7f898f7fe700 1 -- 172.21.15.119:0/2066507295 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 99) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f899c01b3b0 con 0x7f89a80aef00 2023-10-08T02:28:26.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.875+0000 7f898f7fe700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 0x7f8994069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:26.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.875+0000 7f898f7fe700 1 -- 172.21.15.119:0/2066507295 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6828+0+0 (secure 0 0 0) 0x7f899c0840c0 con 0x7f89a80aef00 2023-10-08T02:28:26.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.875+0000 7f89a77fe700 1 -- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 msgr2=0x7f8994069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:26.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.875+0000 7f89a77fe700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 0x7f8994069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:26.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:26.879+0000 7f898f7fe700 1 -- 172.21.15.119:0/2066507295 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f89a80c55b0 con 0x7f89a80aef00 2023-10-08T02:28:26.890 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:26.891+0000 7fe381ed3180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:28:26.912 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:26.912+0000 7ff7a7479180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:28:26.996 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:26.996+0000 7fe381ed3180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:28:27.007 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:27.006+0000 7ff7a7479180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:28:27.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.076+0000 7f89a77fe700 1 -- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 msgr2=0x7f8994069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:27.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.076+0000 7f89a77fe700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 0x7f8994069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:27.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.119+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f89a80c5540 con 0x7f89a80aef00 2023-10-08T02:28:27.163 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:27.162+0000 7ff7a7479180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:28:27.186 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:27.186+0000 7fe381ed3180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:28:27.266 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:27.266+0000 7ff7a7479180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:28:27.280 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:27.281+0000 7fe381ed3180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:28:27.373 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:27.373+0000 7fe381ed3180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:28:27.458 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:27.458+0000 7ff7a7479180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:28:27.468 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:27.469+0000 7fe381ed3180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:28:27.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.477+0000 7f89a77fe700 1 -- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 msgr2=0x7f8994069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:27.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.477+0000 7f89a77fe700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 0x7f8994069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:28:27.551 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:27.550+0000 7ff7a7479180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:28:27.646 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:27.645+0000 7ff7a7479180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:28:27.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.650+0000 7f898f7fe700 1 -- 172.21.15.119:0/2066507295 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) v1 ==== 104+0+0 (secure 0 0 0) 0x7f899c0170a0 con 0x7f89a80aef00 2023-10-08T02:28:27.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 msgr2=0x7f8994069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:27.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 0x7f8994069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a80aef00 msgr2=0x7f89a81548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:27.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a80aef00 0x7f89a81548e0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c590 tx=0x7f899c00c8a0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 shutdown_connections 2023-10-08T02:28:27.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8994067050 0x7f8994069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f89a815a3b0 0x7f89a80b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f89a806e470 0x7f89a81543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 --2- 172.21.15.119:0/2066507295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a80aef00 0x7f89a81548e0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 >> 172.21.15.119:0/2066507295 conn(0x7f89a80c9f70 msgr2=0x7f89a80c17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:27.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 shutdown_connections 2023-10-08T02:28:27.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.652+0000 7f89ac99d700 1 -- 172.21.15.119:0/2066507295 wait complete. 2023-10-08T02:28:27.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T02:28:27.746 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:27.745+0000 7ff7a7479180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:28:27.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.914+0000 7fa1aa5ff700 1 Processor -- start 2023-10-08T02:28:27.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.914+0000 7fa1aa5ff700 1 -- start start 2023-10-08T02:28:27.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.915+0000 7fa1aa5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1a41563b0 0x7fa1a4156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:27.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1aa5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a415a9b0 con 0x7fa1a4068890 2023-10-08T02:28:27.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1aa5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a415ac90 con 0x7fa1a415a440 2023-10-08T02:28:27.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1aa5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a415af70 con 0x7fa1a41563b0 2023-10-08T02:28:27.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1a3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1a41563b0 0x7fa1a4156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:27.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1a3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1a41563b0 0x7fa1a4156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55158/0 (socket says 172.21.15.119:55158) 2023-10-08T02:28:27.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1a3fff700 1 -- 172.21.15.119:0/3920941911 learned_addr learned my addr 172.21.15.119:0/3920941911 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:27.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1a3fff700 1 -- 172.21.15.119:0/3920941911 >> v1:172.21.15.184:6789/0 conn(0x7fa1a415a440 legacy=0x7fa1a40acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:27.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1a3fff700 1 -- 172.21.15.119:0/3920941911 >> v1:172.21.15.119:6789/0 conn(0x7fa1a4068890 legacy=0x7fa1a40c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:27.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.916+0000 7fa1a3fff700 1 -- 172.21.15.119:0/3920941911 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a415b250 con 0x7fa1a41563b0 2023-10-08T02:28:27.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1a3fff700 1 --2- 172.21.15.119:0/3920941911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a41563b0 0x7fa1a4156780 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa194009f20 tx=0x7fa194009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cb271badbfca2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1a2ffd700 1 -- 172.21.15.119:0/3920941911 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19400e070 con 0x7fa1a41563b0 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1a2ffd700 1 -- 172.21.15.119:0/3920941911 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19400ba80 con 0x7fa1a41563b0 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1a2ffd700 1 -- 172.21.15.119:0/3920941911 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1940092e0 con 0x7fa1a41563b0 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/3920941911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a41563b0 msgr2=0x7fa1a4156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1aa5ff700 1 --2- 172.21.15.119:0/3920941911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a41563b0 0x7fa1a4156780 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fa194009f20 tx=0x7fa194009840 comp rx=0 tx=0).stop 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/3920941911 shutdown_connections 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1aa5ff700 1 --2- 172.21.15.119:0/3920941911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a41563b0 0x7fa1a4156780 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/3920941911 >> 172.21.15.119:0/3920941911 conn(0x7fa1a40c8f90 msgr2=0x7fa1a40c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:27.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/3920941911 shutdown_connections 2023-10-08T02:28:27.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.917+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/3920941911 wait complete. 2023-10-08T02:28:27.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.919+0000 7fa1aa5ff700 1 Processor -- start 2023-10-08T02:28:27.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.919+0000 7fa1aa5ff700 1 -- start start 2023-10-08T02:28:27.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1aa5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4068890 0x7fa1a41554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:27.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1aa5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a41563b0 0x7fa1a4158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:27.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1aa5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a415a440 0x7fa1a414c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:27.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1aa5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40c0630 con 0x7fa1a415a440 2023-10-08T02:28:27.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1aa5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40af6b0 con 0x7fa1a41563b0 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1aa5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40af990 con 0x7fa1a4068890 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1a8b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a415a440 0x7fa1a414c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1a8b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a415a440 0x7fa1a414c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55174/0 (socket says 172.21.15.119:55174) 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1a8b9c700 1 -- 172.21.15.119:0/4084300084 learned_addr learned my addr 172.21.15.119:0/4084300084 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1a3fff700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4068890 0x7fa1a41554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.920+0000 7fa1a37fe700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a41563b0 0x7fa1a4158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1a3fff700 1 -- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a41563b0 msgr2=0x7fa1a4158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1a3fff700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a41563b0 0x7fa1a4158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1a3fff700 1 -- 172.21.15.119:0/4084300084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a415a440 msgr2=0x7fa1a414c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1a3fff700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a415a440 0x7fa1a414c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1a3fff700 1 -- 172.21.15.119:0/4084300084 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194009ae0 con 0x7fa1a4068890 2023-10-08T02:28:27.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1a8b9c700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a415a440 0x7fa1a414c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:27.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1a3fff700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4068890 0x7fa1a41554d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa194015000 tx=0x7fa194018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:27.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa18bfff700 1 -- 172.21.15.119:0/4084300084 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194008a70 con 0x7fa1a4068890 2023-10-08T02:28:27.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa18bfff700 1 -- 172.21.15.119:0/4084300084 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa194008c10 con 0x7fa1a4068890 2023-10-08T02:28:27.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa18bfff700 1 -- 172.21.15.119:0/4084300084 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194026460 con 0x7fa1a4068890 2023-10-08T02:28:27.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a40afc70 con 0x7fa1a4068890 2023-10-08T02:28:27.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.921+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a40af2d0 con 0x7fa1a4068890 2023-10-08T02:28:27.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.922+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a40af460 con 0x7fa1a4068890 2023-10-08T02:28:27.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.923+0000 7fa18bfff700 1 -- 172.21.15.119:0/4084300084 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 100) v1 ==== 84451+0+0 (secure 0 0 0) 0x7fa194003040 con 0x7fa1a4068890 2023-10-08T02:28:27.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.924+0000 7fa18bfff700 1 -- 172.21.15.119:0/4084300084 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 7172+0+0 (secure 0 0 0) 0x7fa194086470 con 0x7fa1a4068890 2023-10-08T02:28:27.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:27.928+0000 7fa18bfff700 1 -- 172.21.15.119:0/4084300084 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa1940533f0 con 0x7fa1a4068890 2023-10-08T02:28:28.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.117+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa1a40718b0 con 0x7fa1a4068890 2023-10-08T02:28:28.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.756+0000 7fa18bfff700 1 -- 172.21.15.119:0/4084300084 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v181) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa194056a10 con 0x7fa1a4068890 2023-10-08T02:28:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.759+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4068890 msgr2=0x7fa1a41554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.759+0000 7fa1aa5ff700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4068890 0x7fa1a41554d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa194015000 tx=0x7fa194018000 comp rx=0 tx=0).stop 2023-10-08T02:28:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.759+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 shutdown_connections 2023-10-08T02:28:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.759+0000 7fa1aa5ff700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1a4068890 0x7fa1a41554d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:28.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.759+0000 7fa1aa5ff700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1a415a440 0x7fa1a414c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.759+0000 7fa1aa5ff700 1 --2- 172.21.15.119:0/4084300084 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa1a41563b0 0x7fa1a4158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.759+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 >> 172.21.15.119:0/4084300084 conn(0x7fa1a40c8f90 msgr2=0x7fa1a414dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.760+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 shutdown_connections 2023-10-08T02:28:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:28.760+0000 7fa1aa5ff700 1 -- 172.21.15.119:0/4084300084 wait complete. 2023-10-08T02:28:28.761 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T02:28:28.785 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T02:28:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.010+0000 7f65687ff700 1 Processor -- start 2023-10-08T02:28:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.011+0000 7f65687ff700 1 -- start start 2023-10-08T02:28:29.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f65687ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6560157740 0x7f65600adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f65687ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6560068aa0 0x7f656006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f65687ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656006dc20 con 0x7f65600b2100 2023-10-08T02:28:29.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f65687ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65600acc00 con 0x7f6560157740 2023-10-08T02:28:29.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f65687ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656006d6a0 con 0x7f6560068aa0 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f6566d9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6560068aa0 0x7f656006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f656659b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6560157740 0x7f65600adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f656659b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6560157740 0x7f65600adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:50204/0 (socket says 172.21.15.119:50204) 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f6566d9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6560068aa0 0x7f656006e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:38774/0 (socket says 172.21.15.119:38774) 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f656659b700 1 -- 172.21.15.119:0/994589765 learned_addr learned my addr 172.21.15.119:0/994589765 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.012+0000 7f6565d9a700 1 --1- 172.21.15.119:0/994589765 >> v1:172.21.15.184:6789/0 conn(0x7f65600b2100 0x7f6560068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:38898/0 (socket says 172.21.15.119:38898) 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f656659b700 1 -- 172.21.15.119:0/994589765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6560068aa0 msgr2=0x7f656006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f656659b700 1 --2- 172.21.15.119:0/994589765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6560068aa0 0x7f656006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f656659b700 1 -- 172.21.15.119:0/994589765 >> v1:172.21.15.184:6789/0 conn(0x7f65600b2100 legacy=0x7f6560068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f656659b700 1 -- 172.21.15.119:0/994589765 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65600ae910 con 0x7f6560157740 2023-10-08T02:28:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f6566d9c700 1 --2- 172.21.15.119:0/994589765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6560068aa0 0x7f656006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f656659b700 1 --2- 172.21.15.119:0/994589765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 0x7f65600adfc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f655c0099b0 tx=0x7f655c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725d9006efc44a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f6565599700 1 -- 172.21.15.119:0/994589765 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c009ea0 con 0x7f6560157740 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f6565599700 1 -- 172.21.15.119:0/994589765 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c00b860 con 0x7f6560157740 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.013+0000 7f6565599700 1 -- 172.21.15.119:0/994589765 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c00bc00 con 0x7f6560157740 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.014+0000 7f65687ff700 1 -- 172.21.15.119:0/994589765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 msgr2=0x7f65600adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.014+0000 7f65687ff700 1 --2- 172.21.15.119:0/994589765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 0x7f65600adfc0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f655c0099b0 tx=0x7f655c009250 comp rx=0 tx=0).stop 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.014+0000 7f65687ff700 1 -- 172.21.15.119:0/994589765 shutdown_connections 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.014+0000 7f65687ff700 1 --2- 172.21.15.119:0/994589765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6560068aa0 0x7f656006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.014+0000 7f65687ff700 1 --2- 172.21.15.119:0/994589765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 0x7f65600adfc0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.014+0000 7f65687ff700 1 -- 172.21.15.119:0/994589765 >> 172.21.15.119:0/994589765 conn(0x7f65600c8f90 msgr2=0x7f65600c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.014+0000 7f65687ff700 1 -- 172.21.15.119:0/994589765 shutdown_connections 2023-10-08T02:28:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.014+0000 7f65687ff700 1 -- 172.21.15.119:0/994589765 wait complete. 2023-10-08T02:28:29.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.015+0000 7f65687ff700 1 Processor -- start 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.016+0000 7f65687ff700 1 -- start start 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.016+0000 7f65687ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6560068aa0 0x7f65600afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.016+0000 7f65687ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65600b2100 0x7f65600cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f65687ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 0x7f656016cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f65687ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600c43e0 con 0x7f6560068aa0 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f65687ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600c3530 con 0x7f6560157740 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f65687ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65600c4af0 con 0x7f65600b2100 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f656659b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6560068aa0 0x7f65600afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f656659b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6560068aa0 0x7f65600afcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55182/0 (socket says 172.21.15.119:55182) 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f656659b700 1 -- 172.21.15.119:0/169870392 learned_addr learned my addr 172.21.15.119:0/169870392 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f6565d9a700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65600b2100 0x7f65600cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f6566d9c700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 0x7f656016cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f656659b700 1 -- 172.21.15.119:0/169870392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65600b2100 msgr2=0x7f65600cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f656659b700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65600b2100 0x7f65600cb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f656659b700 1 -- 172.21.15.119:0/169870392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 msgr2=0x7f656016cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f656659b700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 0x7f656016cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.017+0000 7f656659b700 1 -- 172.21.15.119:0/169870392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6558009a20 con 0x7f6560068aa0 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.018+0000 7f656659b700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6560068aa0 0x7f65600afcb0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f656006b890 tx=0x7f655c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.018+0000 7f65567fc700 1 -- 172.21.15.119:0/169870392 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c000d10 con 0x7f6560068aa0 2023-10-08T02:28:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.018+0000 7f65567fc700 1 -- 172.21.15.119:0/169870392 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c00bbe0 con 0x7f6560068aa0 2023-10-08T02:28:29.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.018+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655c0094f0 con 0x7f6560068aa0 2023-10-08T02:28:29.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.018+0000 7f65567fc700 1 -- 172.21.15.119:0/169870392 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c011670 con 0x7f6560068aa0 2023-10-08T02:28:29.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.018+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65600c3ab0 con 0x7f6560068aa0 2023-10-08T02:28:29.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.019+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65600c46c0 con 0x7f6560068aa0 2023-10-08T02:28:29.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.020+0000 7f65567fc700 1 -- 172.21.15.119:0/169870392 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 101) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f655c011aa0 con 0x7f6560068aa0 2023-10-08T02:28:29.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.020+0000 7f65567fc700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6544066d80 0x7f6544069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.021+0000 7f65567fc700 1 -- 172.21.15.119:0/169870392 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6734+0+0 (secure 0 0 0) 0x7f655c085da0 con 0x7f6560068aa0 2023-10-08T02:28:29.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.021+0000 7f6565d9a700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6544066d80 0x7f6544069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.021+0000 7f6565d9a700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6544066d80 0x7f6544069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6560160030 tx=0x7f6550007eb0 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:29.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.025+0000 7f65567fc700 1 -- 172.21.15.119:0/169870392 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f655c052ec0 con 0x7f6560068aa0 2023-10-08T02:28:29.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.213+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f65600718b0 con 0x7f6560068aa0 2023-10-08T02:28:29.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.772+0000 7f65567fc700 1 -- 172.21.15.119:0/169870392 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v182) v1 ==== 169+0+0 (secure 0 0 0) 0x7f655c0564e0 con 0x7f6560068aa0 2023-10-08T02:28:29.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.775+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6544066d80 msgr2=0x7f6544069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.775+0000 7f65687ff700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6544066d80 0x7f6544069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6560160030 tx=0x7f6550007eb0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.775+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6560068aa0 msgr2=0x7f65600afcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.775+0000 7f65687ff700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6560068aa0 0x7f65600afcb0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f656006b890 tx=0x7f655c017000 comp rx=0 tx=0).stop 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.776+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 shutdown_connections 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.776+0000 7f65687ff700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6544066d80 0x7f6544069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.776+0000 7f65687ff700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6560157740 0x7f656016cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.776+0000 7f65687ff700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6560068aa0 0x7f65600afcb0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.776+0000 7f65687ff700 1 --2- 172.21.15.119:0/169870392 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f65600b2100 0x7f65600cb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.776+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 >> 172.21.15.119:0/169870392 conn(0x7f65600c8f90 msgr2=0x7f65600aad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.776+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 shutdown_connections 2023-10-08T02:28:29.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.776+0000 7f65687ff700 1 -- 172.21.15.119:0/169870392 wait complete. 2023-10-08T02:28:29.778 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T02:28:29.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T02:28:29.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.987+0000 7efc631ff700 1 Processor -- start 2023-10-08T02:28:29.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.987+0000 7efc631ff700 1 -- start start 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.988+0000 7efc631ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc5c15ac80 0x7efc5c069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.988+0000 7efc631ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc5c069cb0 0x7efc5c0c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.988+0000 7efc631ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc5c0af2f0 con 0x7efc5c15b050 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.988+0000 7efc631ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c0af5d0 con 0x7efc5c15ac80 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.988+0000 7efc631ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5c06a0e0 con 0x7efc5c069cb0 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc60f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc5c15ac80 0x7efc5c069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc60f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc5c15ac80 0x7efc5c069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55188/0 (socket says 172.21.15.119:55188) 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc60f9b700 1 -- 172.21.15.119:0/3610395315 learned_addr learned my addr 172.21.15.119:0/3610395315 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc6179c700 1 --2- 172.21.15.119:0/3610395315 >> v2:172.21.15.184:3300/0 conn(0x7efc5c069cb0 0x7efc5c0c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc60f9b700 1 -- 172.21.15.119:0/3610395315 >> v2:172.21.15.184:3300/0 conn(0x7efc5c069cb0 msgr2=0x7efc5c0c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc60f9b700 1 --2- 172.21.15.119:0/3610395315 >> v2:172.21.15.184:3300/0 conn(0x7efc5c069cb0 0x7efc5c0c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc60f9b700 1 -- 172.21.15.119:0/3610395315 >> v1:172.21.15.184:6789/0 conn(0x7efc5c15b050 legacy=0x7efc5c0afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc60f9b700 1 -- 172.21.15.119:0/3610395315 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5c0c8660 con 0x7efc5c15ac80 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.989+0000 7efc60f9b700 1 --2- 172.21.15.119:0/3610395315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15ac80 0x7efc5c069770 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7efc5c06d240 tx=0x7efc50009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161cceaaa215551e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc5b7fe700 1 -- 172.21.15.119:0/3610395315 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5000b820 con 0x7efc5c15ac80 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc5b7fe700 1 -- 172.21.15.119:0/3610395315 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5000b980 con 0x7efc5c15ac80 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc5b7fe700 1 -- 172.21.15.119:0/3610395315 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5000bd20 con 0x7efc5c15ac80 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc631ff700 1 -- 172.21.15.119:0/3610395315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15ac80 msgr2=0x7efc5c069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc631ff700 1 --2- 172.21.15.119:0/3610395315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15ac80 0x7efc5c069770 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7efc5c06d240 tx=0x7efc50009250 comp rx=0 tx=0).stop 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc631ff700 1 -- 172.21.15.119:0/3610395315 shutdown_connections 2023-10-08T02:28:29.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc631ff700 1 --2- 172.21.15.119:0/3610395315 >> v2:172.21.15.184:3300/0 conn(0x7efc5c069cb0 0x7efc5c0c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc631ff700 1 --2- 172.21.15.119:0/3610395315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15ac80 0x7efc5c069770 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc631ff700 1 -- 172.21.15.119:0/3610395315 >> 172.21.15.119:0/3610395315 conn(0x7efc5c068e10 msgr2=0x7efc5c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:29.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc631ff700 1 -- 172.21.15.119:0/3610395315 shutdown_connections 2023-10-08T02:28:29.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.990+0000 7efc631ff700 1 -- 172.21.15.119:0/3610395315 wait complete. 2023-10-08T02:28:29.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.992+0000 7efc631ff700 1 Processor -- start 2023-10-08T02:28:29.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.992+0000 7efc631ff700 1 -- start start 2023-10-08T02:28:29.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc631ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc5c069cb0 0x7efc5c0ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc631ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc5c15ac80 0x7efc5c0adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc631ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15b050 0x7efc5c156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc631ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0c6940 con 0x7efc5c15b050 2023-10-08T02:28:29.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc631ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c0ae020 con 0x7efc5c069cb0 2023-10-08T02:28:29.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc631ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5c156740 con 0x7efc5c15ac80 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15b050 0x7efc5c156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15b050 0x7efc5c156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55196/0 (socket says 172.21.15.119:55196) 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 -- 172.21.15.119:0/2046071375 learned_addr learned my addr 172.21.15.119:0/2046071375 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc60f9b700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc5c069cb0 0x7efc5c0ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc5bfff700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc5c15ac80 0x7efc5c0adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 -- 172.21.15.119:0/2046071375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc5c15ac80 msgr2=0x7efc5c0adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc5c15ac80 0x7efc5c0adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 -- 172.21.15.119:0/2046071375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc5c069cb0 msgr2=0x7efc5c0ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc5c069cb0 0x7efc5c0ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 -- 172.21.15.119:0/2046071375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc500094f0 con 0x7efc5c15b050 2023-10-08T02:28:29.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.993+0000 7efc6179c700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15b050 0x7efc5c156200 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7efc54009fd0 tx=0x7efc54009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:29.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.994+0000 7efc58ff9700 1 -- 172.21.15.119:0/2046071375 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc5400eca0 con 0x7efc5c15b050 2023-10-08T02:28:29.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.994+0000 7efc58ff9700 1 -- 172.21.15.119:0/2046071375 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc5400ee00 con 0x7efc5c15b050 2023-10-08T02:28:29.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.994+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5c0ae2a0 con 0x7efc5c15b050 2023-10-08T02:28:29.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.994+0000 7efc58ff9700 1 -- 172.21.15.119:0/2046071375 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc54014670 con 0x7efc5c15b050 2023-10-08T02:28:29.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.994+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5c0ae860 con 0x7efc5c15b050 2023-10-08T02:28:29.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.995+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5c0aea80 con 0x7efc5c15b050 2023-10-08T02:28:29.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.996+0000 7efc58ff9700 1 -- 172.21.15.119:0/2046071375 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 101) v1 ==== 84578+0+0 (secure 0 0 0) 0x7efc540149e0 con 0x7efc5c15b050 2023-10-08T02:28:29.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.996+0000 7efc58ff9700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc40066d80 0x7efc40069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:29.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.997+0000 7efc58ff9700 1 -- 172.21.15.119:0/2046071375 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6291+0+0 (secure 0 0 0) 0x7efc54084ba0 con 0x7efc5c15b050 2023-10-08T02:28:29.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.997+0000 7efc60f9b700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc40066d80 0x7efc40069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:29.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:29.997+0000 7efc60f9b700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc40066d80 0x7efc40069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efc5c06d240 tx=0x7efc50016000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:30.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.001+0000 7efc58ff9700 1 -- 172.21.15.119:0/2046071375 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efc54051da0 con 0x7efc5c15b050 2023-10-08T02:28:30.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.192+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7efc5c04c3c0 con 0x7efc5c15b050 2023-10-08T02:28:30.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.196+0000 7efc58ff9700 1 -- 172.21.15.119:0/2046071375 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v61) v1 ==== 104+0+0 (secure 0 0 0) 0x7efc540553c0 con 0x7efc5c15b050 2023-10-08T02:28:30.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc40066d80 msgr2=0x7efc40069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc40066d80 0x7efc40069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efc5c06d240 tx=0x7efc50016000 comp rx=0 tx=0).stop 2023-10-08T02:28:30.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15b050 msgr2=0x7efc5c156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15b050 0x7efc5c156200 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7efc54009fd0 tx=0x7efc54009900 comp rx=0 tx=0).stop 2023-10-08T02:28:30.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 shutdown_connections 2023-10-08T02:28:30.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc40066d80 0x7efc40069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc5c15b050 0x7efc5c156200 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc5c069cb0 0x7efc5c0ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 --2- 172.21.15.119:0/2046071375 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc5c15ac80 0x7efc5c0adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 >> 172.21.15.119:0/2046071375 conn(0x7efc5c068e10 msgr2=0x7efc5c0c2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:30.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.199+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 shutdown_connections 2023-10-08T02:28:30.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.200+0000 7efc631ff700 1 -- 172.21.15.119:0/2046071375 wait complete. 2023-10-08T02:28:30.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T02:28:30.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.404+0000 7fe39db9d700 1 Processor -- start 2023-10-08T02:28:30.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.404+0000 7fe39db9d700 1 -- start start 2023-10-08T02:28:30.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.405+0000 7fe39db9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b4e70 0x7fe3980ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.405+0000 7fe39db9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3980b16d0 0x7fe3980b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.405+0000 7fe39db9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe39815d240 0x7fe39815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.405+0000 7fe39db9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3980b2080 con 0x7fe3980b16d0 2023-10-08T02:28:30.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.405+0000 7fe39db9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3980cfb00 con 0x7fe39815d240 2023-10-08T02:28:30.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.405+0000 7fe39db9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980b5c40 con 0x7fe3980b4e70 2023-10-08T02:28:30.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.405+0000 7fe397fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3980b16d0 0x7fe3980b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe397fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3980b16d0 0x7fe3980b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55204/0 (socket says 172.21.15.119:55204) 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe397fff700 1 -- 172.21.15.119:0/3192723614 learned_addr learned my addr 172.21.15.119:0/3192723614 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe39d39c700 1 --2- 172.21.15.119:0/3192723614 >> v2:172.21.15.184:3300/0 conn(0x7fe39815d240 0x7fe39815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe39cb9b700 1 --2- 172.21.15.119:0/3192723614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b4e70 0x7fe3980ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe397fff700 1 -- 172.21.15.119:0/3192723614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b4e70 msgr2=0x7fe3980ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe397fff700 1 --2- 172.21.15.119:0/3192723614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b4e70 0x7fe3980ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe397fff700 1 -- 172.21.15.119:0/3192723614 >> v2:172.21.15.184:3300/0 conn(0x7fe39815d240 msgr2=0x7fe39815d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe397fff700 1 --2- 172.21.15.119:0/3192723614 >> v2:172.21.15.184:3300/0 conn(0x7fe39815d240 0x7fe39815d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe397fff700 1 -- 172.21.15.119:0/3192723614 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3980b5f20 con 0x7fe3980b16d0 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe397fff700 1 --2- 172.21.15.119:0/3192723614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3980b16d0 0x7fe3980b1b40 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe39815d180 tx=0x7fe384009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ea92a23cd8ec47 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:30.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.406+0000 7fe3977fe700 1 -- 172.21.15.119:0/3192723614 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38400b640 con 0x7fe3980b16d0 2023-10-08T02:28:30.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe3977fe700 1 -- 172.21.15.119:0/3192723614 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe38400b7a0 con 0x7fe3980b16d0 2023-10-08T02:28:30.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe3977fe700 1 -- 172.21.15.119:0/3192723614 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe38400bb60 con 0x7fe3980b16d0 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 -- 172.21.15.119:0/3192723614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3980b16d0 msgr2=0x7fe3980b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3192723614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3980b16d0 0x7fe3980b1b40 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fe39815d180 tx=0x7fe384009250 comp rx=0 tx=0).stop 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 -- 172.21.15.119:0/3192723614 shutdown_connections 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3192723614 >> v2:172.21.15.184:3300/0 conn(0x7fe39815d240 0x7fe39815d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3192723614 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b4e70 0x7fe3980ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3192723614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3980b16d0 0x7fe3980b1b40 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 -- 172.21.15.119:0/3192723614 >> 172.21.15.119:0/3192723614 conn(0x7fe3980d10c0 msgr2=0x7fe3980affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 -- 172.21.15.119:0/3192723614 shutdown_connections 2023-10-08T02:28:30.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.407+0000 7fe39db9d700 1 -- 172.21.15.119:0/3192723614 wait complete. 2023-10-08T02:28:30.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.409+0000 7fe39db9d700 1 Processor -- start 2023-10-08T02:28:30.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.409+0000 7fe39db9d700 1 -- start start 2023-10-08T02:28:30.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39db9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b16d0 0x7fe39814b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39db9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3980b4e70 0x7fe3980cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39db9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe39815d240 0x7fe39815de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39db9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39814b7a0 con 0x7fe39815d240 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39db9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980c54c0 con 0x7fe3980b4e70 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39db9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980c5760 con 0x7fe3980b16d0 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39d39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe39815d240 0x7fe39815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39d39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe39815d240 0x7fe39815de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55220/0 (socket says 172.21.15.119:55220) 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39d39c700 1 -- 172.21.15.119:0/3751489959 learned_addr learned my addr 172.21.15.119:0/3751489959 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39cb9b700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b16d0 0x7fe39814b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe397fff700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3980b4e70 0x7fe3980cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39d39c700 1 -- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b16d0 msgr2=0x7fe39814b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39d39c700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b16d0 0x7fe39814b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39d39c700 1 -- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3980b4e70 msgr2=0x7fe3980cbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39d39c700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3980b4e70 0x7fe3980cbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39d39c700 1 -- 172.21.15.119:0/3751489959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840094f0 con 0x7fe39815d240 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe39cb9b700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b16d0 0x7fe39814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.410+0000 7fe397fff700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3980b4e70 0x7fe3980cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.411+0000 7fe39d39c700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe39815d240 0x7fe39815de20 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe390009df0 tx=0x7fe39000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.411+0000 7fe394ff9700 1 -- 172.21.15.119:0/3751489959 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe39000cb50 con 0x7fe39815d240 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.411+0000 7fe394ff9700 1 -- 172.21.15.119:0/3751489959 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe39000f040 con 0x7fe39815d240 2023-10-08T02:28:30.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.411+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3980c5a40 con 0x7fe39815d240 2023-10-08T02:28:30.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.411+0000 7fe394ff9700 1 -- 172.21.15.119:0/3751489959 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390014480 con 0x7fe39815d240 2023-10-08T02:28:30.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.411+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3980c6000 con 0x7fe39815d240 2023-10-08T02:28:30.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.412+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe398088990 con 0x7fe39815d240 2023-10-08T02:28:30.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.413+0000 7fe394ff9700 1 -- 172.21.15.119:0/3751489959 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe390014830 con 0x7fe39815d240 2023-10-08T02:28:30.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.414+0000 7fe394ff9700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe37c066f30 0x7fe37c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.414+0000 7fe394ff9700 1 -- 172.21.15.119:0/3751489959 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6291+0+0 (secure 0 0 0) 0x7fe3900851a0 con 0x7fe39815d240 2023-10-08T02:28:30.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.414+0000 7fe39cb9b700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe37c066f30 0x7fe37c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.414+0000 7fe39cb9b700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe37c066f30 0x7fe37c0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009cf0 tx=0x7fe38c008040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:30.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.418+0000 7fe394ff9700 1 -- 172.21.15.119:0/3751489959 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe390052120 con 0x7fe39815d240 2023-10-08T02:28:30.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.609+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7fe3980cd860 con 0x7fe39815d240 2023-10-08T02:28:30.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.615+0000 7fe394ff9700 1 -- 172.21.15.119:0/3751489959 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v62) v1 ==== 115+0+0 (secure 0 0 0) 0x7fe390055740 con 0x7fe39815d240 2023-10-08T02:28:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe37c066f30 msgr2=0x7fe37c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe37c066f30 0x7fe37c0693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009cf0 tx=0x7fe38c008040 comp rx=0 tx=0).stop 2023-10-08T02:28:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe39815d240 msgr2=0x7fe39815de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe39815d240 0x7fe39815de20 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fe390009df0 tx=0x7fe39000b040 comp rx=0 tx=0).stop 2023-10-08T02:28:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 shutdown_connections 2023-10-08T02:28:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe37c066f30 0x7fe37c0693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe3980b16d0 0x7fe39814b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe39815d240 0x7fe39815de20 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 --2- 172.21.15.119:0/3751489959 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe3980b4e70 0x7fe3980cbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.617+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 >> 172.21.15.119:0/3751489959 conn(0x7fe3980d10c0 msgr2=0x7fe3980b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.618+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 shutdown_connections 2023-10-08T02:28:30.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.618+0000 7fe39db9d700 1 -- 172.21.15.119:0/3751489959 wait complete. 2023-10-08T02:28:30.636 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T02:28:30.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.819+0000 7fdad3bff700 1 Processor -- start 2023-10-08T02:28:30.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.820+0000 7fdad3bff700 1 -- start start 2023-10-08T02:28:30.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad3bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdacc0ccee0 0x7fdacc1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad3bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c87c0 0x7fdacc156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad3bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdacc15b0d0 con 0x7fdacc0c83f0 2023-10-08T02:28:30.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad3bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdacc156c50 con 0x7fdacc0ccee0 2023-10-08T02:28:30.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad3bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc15a4c0 con 0x7fdacc0c87c0 2023-10-08T02:28:30.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad199b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdacc0c83f0 0x7fdacc0c7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51124/0 (socket says 172.21.15.119:51124) 2023-10-08T02:28:30.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad199b700 1 -- 172.21.15.119:0/4287721049 learned_addr learned my addr 172.21.15.119:0/4287721049 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:30.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808929345 0 0) 0x7fdacc15b0d0 con 0x7fdacc0c83f0 2023-10-08T02:28:30.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad119a700 1 --2- 172.21.15.119:0/4287721049 >> v2:172.21.15.184:3300/0 conn(0x7fdacc0ccee0 0x7fdacc1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.821+0000 7fdad219c700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c87c0 0x7fdacc156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab8003570 con 0x7fdacc0c83f0 2023-10-08T02:28:30.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3555472519 0 0) 0x7fdab8003570 con 0x7fdacc0c83f0 2023-10-08T02:28:30.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c87c0 msgr2=0x7fdacc156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:30.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c87c0 0x7fdacc156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 >> v2:172.21.15.184:3300/0 conn(0x7fdacc0ccee0 msgr2=0x7fdacc1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 --2- 172.21.15.119:0/4287721049 >> v2:172.21.15.184:3300/0 conn(0x7fdacc0ccee0 0x7fdacc1561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc15a7a0 con 0x7fdacc0c83f0 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fdac8001470 con 0x7fdacc0c83f0 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 >> v1:172.21.15.119:6789/0 conn(0x7fdacc0c83f0 legacy=0x7fdacc0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdab8003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c87c0 0x7fdab8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8004d50 0x7fdab80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc156c50 con 0x7fdab8003ae0 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc15a4c0 con 0x7fdab8004d50 2023-10-08T02:28:30.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab800bb10 con 0x7fdacc0c87c0 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad199b700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdab8003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad219c700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8004d50 0x7fdab80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.822+0000 7fdad119a700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c87c0 0x7fdab8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad199b700 1 -- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c87c0 msgr2=0x7fdab8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad199b700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c87c0 0x7fdab8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad199b700 1 -- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8004d50 msgr2=0x7fdab80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad199b700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdab8004d50 0x7fdab80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad199b700 1 -- 172.21.15.119:0/4287721049 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab8003570 con 0x7fdab8003ae0 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad199b700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdab8003ef0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fdac800c300 tx=0x7fdac8001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff6a823355789ac server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac800cb90 con 0x7fdab8003ae0 2023-10-08T02:28:30.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac800cd10 con 0x7fdab8003ae0 2023-10-08T02:28:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.823+0000 7fdad0999700 1 -- 172.21.15.119:0/4287721049 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8005b70 con 0x7fdab8003ae0 2023-10-08T02:28:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.824+0000 7fdad3bff700 1 -- 172.21.15.119:0/4287721049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 msgr2=0x7fdab8003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.824+0000 7fdad3bff700 1 --2- 172.21.15.119:0/4287721049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdab8003ef0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fdac800c300 tx=0x7fdac8001dc0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.824+0000 7fdad219c700 1 -- 172.21.15.119:0/4287721049 reap_dead start 2023-10-08T02:28:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.824+0000 7fdad3bff700 1 -- 172.21.15.119:0/4287721049 shutdown_connections 2023-10-08T02:28:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.824+0000 7fdad3bff700 1 -- 172.21.15.119:0/4287721049 >> 172.21.15.119:0/4287721049 conn(0x7fdacc068e10 msgr2=0x7fdacc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.824+0000 7fdad3bff700 1 -- 172.21.15.119:0/4287721049 shutdown_connections 2023-10-08T02:28:30.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.824+0000 7fdad3bff700 1 -- 172.21.15.119:0/4287721049 wait complete. 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.825+0000 7fdad3bff700 1 Processor -- start 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.825+0000 7fdad3bff700 1 -- start start 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.826+0000 7fdad3bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdacc0b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.826+0000 7fdad3bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c1c00 0x7fdacc0c2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.826+0000 7fdad3bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdacc157e20 0x7fdacc1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.826+0000 7fdad3bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc159bf0 con 0x7fdab8003ae0 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.826+0000 7fdad3bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0c25b0 con 0x7fdacc157e20 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.826+0000 7fdad3bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc158b50 con 0x7fdacc0c1c00 2023-10-08T02:28:30.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdacc0b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdacc0b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55230/0 (socket says 172.21.15.119:55230) 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 -- 172.21.15.119:0/3580345458 learned_addr learned my addr 172.21.15.119:0/3580345458 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad219c700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdacc157e20 0x7fdacc1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad119a700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c1c00 0x7fdacc0c2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 -- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c1c00 msgr2=0x7fdacc0c2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c1c00 0x7fdacc0c2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 -- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdacc157e20 msgr2=0x7fdacc1582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdacc157e20 0x7fdacc1582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 -- 172.21.15.119:0/3580345458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc158e30 con 0x7fdab8003ae0 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad219c700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdacc157e20 0x7fdacc1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:30.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad199b700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdacc0b1ce0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fdac800c8f0 tx=0x7fdac8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdac1ffb700 1 -- 172.21.15.119:0/3580345458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80070d0 con 0x7fdab8003ae0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdac1ffb700 1 -- 172.21.15.119:0/3580345458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac8007230 con 0x7fdab8003ae0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.827+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdacc159110 con 0x7fdab8003ae0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.828+0000 7fdac1ffb700 1 -- 172.21.15.119:0/3580345458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac80075f0 con 0x7fdab8003ae0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.828+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdacc1596d0 con 0x7fdab8003ae0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.828+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdacc0b0450 con 0x7fdab8003ae0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.829+0000 7fdac1ffb700 1 -- 172.21.15.119:0/3580345458 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdac8019030 con 0x7fdab8003ae0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.830+0000 7fdac1ffb700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdab0066f30 0x7fdab00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.830+0000 7fdac1ffb700 1 -- 172.21.15.119:0/3580345458 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6291+0+0 (secure 0 0 0) 0x7fdac80860f0 con 0x7fdab8003ae0 2023-10-08T02:28:30.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.831+0000 7fdad119a700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdab0066f30 0x7fdab00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:30.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.831+0000 7fdad119a700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdab0066f30 0x7fdab00693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdabc006dc0 tx=0x7fdabc00f040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:30.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:30.835+0000 7fdac1ffb700 1 -- 172.21.15.119:0/3580345458 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdac8053120 con 0x7fdab8003ae0 2023-10-08T02:28:31.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.024+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fdacc04c3c0 con 0x7fdab8003ae0 2023-10-08T02:28:31.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.029+0000 7fdac1ffb700 1 -- 172.21.15.119:0/3580345458 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v63) v1 ==== 120+0+0 (secure 0 0 0) 0x7fdac8056740 con 0x7fdab8003ae0 2023-10-08T02:28:31.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.031+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdab0066f30 msgr2=0x7fdab00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.031+0000 7fdad3bff700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdab0066f30 0x7fdab00693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fdabc006dc0 tx=0x7fdabc00f040 comp rx=0 tx=0).stop 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.031+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 msgr2=0x7fdacc0b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.031+0000 7fdad3bff700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdacc0b1ce0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fdac800c8f0 tx=0x7fdac8012000 comp rx=0 tx=0).stop 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.032+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 shutdown_connections 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.032+0000 7fdad3bff700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdab0066f30 0x7fdab00693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.032+0000 7fdad3bff700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdacc157e20 0x7fdacc1582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.032+0000 7fdad3bff700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab8003ae0 0x7fdacc0b1ce0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.032+0000 7fdad3bff700 1 --2- 172.21.15.119:0/3580345458 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdacc0c1c00 0x7fdacc0c2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.032+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 >> 172.21.15.119:0/3580345458 conn(0x7fdacc068e10 msgr2=0x7fdacc0c2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:31.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.032+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 shutdown_connections 2023-10-08T02:28:31.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.032+0000 7fdad3bff700 1 -- 172.21.15.119:0/3580345458 wait complete. 2023-10-08T02:28:31.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-10-08T02:28:31.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.230+0000 7f9a75fff700 1 Processor -- start 2023-10-08T02:28:31.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.231+0000 7f9a75fff700 1 -- start start 2023-10-08T02:28:31.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a75fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9a700cbf30 0x7f9a700ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a75fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700b1420 0x7f9a700c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a75fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a700cc300 con 0x7f9a700ccf90 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a75fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a70156bf0 con 0x7f9a700cbf30 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a75fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700ae950 con 0x7f9a700b1420 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a6effd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9a700ccf90 0x7f9a700b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51134/0 (socket says 172.21.15.119:51134) 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a6effd700 1 -- 172.21.15.119:0/1426968518 learned_addr learned my addr 172.21.15.119:0/1426968518 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a6f7fe700 1 --2- 172.21.15.119:0/1426968518 >> v2:172.21.15.184:3300/0 conn(0x7f9a700cbf30 0x7f9a700ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1059528124 0 0) 0x7f9a700cc300 con 0x7f9a700ccf90 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.232+0000 7f9a6ffff700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700b1420 0x7f9a700c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a58003570 con 0x7f9a700ccf90 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4060580332 0 0) 0x7f9a58003570 con 0x7f9a700ccf90 2023-10-08T02:28:31.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700b1420 msgr2=0x7f9a700c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700b1420 0x7f9a700c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 >> v2:172.21.15.184:3300/0 conn(0x7f9a700cbf30 msgr2=0x7f9a700ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 --2- 172.21.15.119:0/1426968518 >> v2:172.21.15.184:3300/0 conn(0x7f9a700cbf30 0x7f9a700ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a700ad370 con 0x7f9a700ccf90 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f9a64001470 con 0x7f9a700ccf90 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 >> v1:172.21.15.119:6789/0 conn(0x7f9a700ccf90 legacy=0x7f9a700b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a58003a90 0x7f9a58003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700b1420 0x7f9a58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58004d50 0x7f9a580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70156bf0 con 0x7f9a58004d50 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700ae950 con 0x7f9a58003a90 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5800ba60 con 0x7f9a700b1420 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6ffff700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58004d50 0x7f9a580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6f7fe700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a58003a90 0x7f9a58003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.233+0000 7f9a6effd700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700b1420 0x7f9a58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6f7fe700 1 -- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700b1420 msgr2=0x7f9a58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6f7fe700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700b1420 0x7f9a58004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6f7fe700 1 -- 172.21.15.119:0/1426968518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58004d50 msgr2=0x7f9a580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6f7fe700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58004d50 0x7f9a580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6f7fe700 1 -- 172.21.15.119:0/1426968518 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58003570 con 0x7f9a58003a90 2023-10-08T02:28:31.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6ffff700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58004d50 0x7f9a580051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6f7fe700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a58003a90 0x7f9a58003ea0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000beb0 tx=0x7f9a6000b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2d4a3a92156dad server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60010c00 con 0x7f9a58003a90 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a60010d60 con 0x7f9a58003a90 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.234+0000 7f9a6e7fc700 1 -- 172.21.15.119:0/1426968518 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60011680 con 0x7f9a58003a90 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.235+0000 7f9a75fff700 1 -- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a58003a90 msgr2=0x7f9a58003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.235+0000 7f9a75fff700 1 --2- 172.21.15.119:0/1426968518 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a58003a90 0x7f9a58003ea0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000beb0 tx=0x7f9a6000b430 comp rx=0 tx=0).stop 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.235+0000 7f9a6ffff700 1 -- 172.21.15.119:0/1426968518 reap_dead start 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.235+0000 7f9a75fff700 1 -- 172.21.15.119:0/1426968518 shutdown_connections 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.235+0000 7f9a75fff700 1 -- 172.21.15.119:0/1426968518 >> 172.21.15.119:0/1426968518 conn(0x7f9a7006a320 msgr2=0x7f9a700cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.235+0000 7f9a75fff700 1 -- 172.21.15.119:0/1426968518 shutdown_connections 2023-10-08T02:28:31.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.235+0000 7f9a75fff700 1 -- 172.21.15.119:0/1426968518 wait complete. 2023-10-08T02:28:31.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.236+0000 7f9a75fff700 1 Processor -- start 2023-10-08T02:28:31.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.237+0000 7f9a75fff700 1 -- start start 2023-10-08T02:28:31.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.237+0000 7f9a75fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58003a90 0x7f9a701523e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.237+0000 7f9a75fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a700c2180 0x7f9a7016a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.237+0000 7f9a75fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700c49b0 0x7f9a700c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.237+0000 7f9a75fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700c2a30 con 0x7f9a58003a90 2023-10-08T02:28:31.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a75fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700c3890 con 0x7f9a700c2180 2023-10-08T02:28:31.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a75fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a700c5370 con 0x7f9a700c49b0 2023-10-08T02:28:31.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58003a90 0x7f9a701523e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58003a90 0x7f9a701523e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55244/0 (socket says 172.21.15.119:55244) 2023-10-08T02:28:31.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 -- 172.21.15.119:0/3208126967 learned_addr learned my addr 172.21.15.119:0/3208126967 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:31.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6effd700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a700c2180 0x7f9a7016a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6ffff700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700c49b0 0x7f9a700c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 -- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700c49b0 msgr2=0x7f9a700c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700c49b0 0x7f9a700c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 -- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a700c2180 msgr2=0x7f9a7016a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a700c2180 0x7f9a7016a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 -- 172.21.15.119:0/3208126967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7016adc0 con 0x7f9a58003a90 2023-10-08T02:28:31.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6ffff700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700c49b0 0x7f9a700c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:31.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a6f7fe700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58003a90 0x7f9a701523e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b9d0 tx=0x7f9a6001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:31.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.238+0000 7f9a577fe700 1 -- 172.21.15.119:0/3208126967 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60019650 con 0x7f9a58003a90 2023-10-08T02:28:31.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.239+0000 7f9a577fe700 1 -- 172.21.15.119:0/3208126967 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a600197b0 con 0x7f9a58003a90 2023-10-08T02:28:31.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.239+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a700c2c80 con 0x7f9a58003a90 2023-10-08T02:28:31.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.239+0000 7f9a577fe700 1 -- 172.21.15.119:0/3208126967 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60019b70 con 0x7f9a58003a90 2023-10-08T02:28:31.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.239+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a700c3240 con 0x7f9a58003a90 2023-10-08T02:28:31.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.240+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a7006c1e0 con 0x7f9a58003a90 2023-10-08T02:28:31.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.240+0000 7f9a577fe700 1 -- 172.21.15.119:0/3208126967 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a60020070 con 0x7f9a58003a90 2023-10-08T02:28:31.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.241+0000 7f9a577fe700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9a5c066f30 0x7f9a5c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.241+0000 7f9a577fe700 1 -- 172.21.15.119:0/3208126967 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6291+0+0 (secure 0 0 0) 0x7f9a60086690 con 0x7f9a58003a90 2023-10-08T02:28:31.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.242+0000 7f9a6effd700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9a5c066f30 0x7f9a5c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.242+0000 7f9a6effd700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9a5c066f30 0x7f9a5c0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a64001c00 tx=0x7f9a64009040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:31.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.245+0000 7f9a577fe700 1 -- 172.21.15.119:0/3208126967 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a60053690 con 0x7f9a58003a90 2023-10-08T02:28:31.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.513+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f9a7004c3c0 con 0x7f9a58003a90 2023-10-08T02:28:31.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.774+0000 7f9a577fe700 1 -- 172.21.15.119:0/3208126967 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2983) v1 ==== 165+0+0 (secure 0 0 0) 0x7f9a60056cb0 con 0x7f9a58003a90 2023-10-08T02:28:31.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9a5c066f30 msgr2=0x7f9a5c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9a5c066f30 0x7f9a5c0693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9a64001c00 tx=0x7f9a64009040 comp rx=0 tx=0).stop 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58003a90 msgr2=0x7f9a701523e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58003a90 0x7f9a701523e0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b9d0 tx=0x7f9a6001c000 comp rx=0 tx=0).stop 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 shutdown_connections 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9a5c066f30 0x7f9a5c0693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9a700c49b0 0x7f9a700c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a58003a90 0x7f9a701523e0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 --2- 172.21.15.119:0/3208126967 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9a700c2180 0x7f9a7016a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 >> 172.21.15.119:0/3208126967 conn(0x7f9a7006a320 msgr2=0x7f9a700ca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:31.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 shutdown_connections 2023-10-08T02:28:31.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.777+0000 7f9a75fff700 1 -- 172.21.15.119:0/3208126967 wait complete. 2023-10-08T02:28:31.795 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-10-08T02:28:31.795 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-10-08T02:28:31.795 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-10-08T02:28:31.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-08T02:28:31.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.976+0000 7efce5bff700 1 Processor -- start 2023-10-08T02:28:31.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.976+0000 7efce5bff700 1 -- start start 2023-10-08T02:28:31.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efce5bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce01578b0 0x7efce0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efce5bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efce00af2f0 0x7efce00af710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efce5bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00ad930 con 0x7efce0155c10 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efce5bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00b0c60 con 0x7efce00af2f0 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efce5bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00acd20 con 0x7efce01578b0 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efcdf7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efce0155c10 0x7efce00ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51148/0 (socket says 172.21.15.119:51148) 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efcdf7fe700 1 -- 172.21.15.119:0/1327920938 learned_addr learned my addr 172.21.15.119:0/1327920938 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efcdffff700 1 --2- 172.21.15.119:0/1327920938 >> v2:172.21.15.184:3300/0 conn(0x7efce00af2f0 0x7efce00af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efcdeffd700 1 --2- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce01578b0 0x7efce0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efcde7fc700 1 -- 172.21.15.119:0/1327920938 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562353689 0 0) 0x7efce00ad930 con 0x7efce0155c10 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.978+0000 7efcde7fc700 1 -- 172.21.15.119:0/1327920938 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcc8003570 con 0x7efce0155c10 2023-10-08T02:28:31.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcdffff700 1 -- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce01578b0 msgr2=0x7efce0155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:31.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcdffff700 1 --2- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce01578b0 0x7efce0155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcdffff700 1 -- 172.21.15.119:0/1327920938 >> v1:172.21.15.119:6789/0 conn(0x7efce0155c10 legacy=0x7efce00ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcde7fc700 1 -- 172.21.15.119:0/1327920938 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 147877698 0 0) 0x7efcc8003570 con 0x7efce0155c10 2023-10-08T02:28:31.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcdffff700 1 -- 172.21.15.119:0/1327920938 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00ad000 con 0x7efce00af2f0 2023-10-08T02:28:31.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcdeffd700 1 --2- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce01578b0 0x7efce0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:31.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcdffff700 1 --2- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efce00af2f0 0x7efce00af710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009e90 tx=0x7efcd8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4998c42372882eda server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:31.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcde7fc700 1 -- 172.21.15.119:0/1327920938 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800e070 con 0x7efce00af2f0 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcde7fc700 1 -- 172.21.15.119:0/1327920938 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd800b870 con 0x7efce00af2f0 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.979+0000 7efcde7fc700 1 -- 172.21.15.119:0/1327920938 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0001470 con 0x7efce00af2f0 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.980+0000 7efce5bff700 1 -- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efce00af2f0 msgr2=0x7efce00af710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.980+0000 7efce5bff700 1 --2- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efce00af2f0 0x7efce00af710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7efcd8009e90 tx=0x7efcd8009780 comp rx=0 tx=0).stop 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.980+0000 7efce5bff700 1 -- 172.21.15.119:0/1327920938 shutdown_connections 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.980+0000 7efce5bff700 1 --2- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efce00af2f0 0x7efce00af710 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.980+0000 7efce5bff700 1 --2- 172.21.15.119:0/1327920938 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce01578b0 0x7efce0155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.980+0000 7efce5bff700 1 -- 172.21.15.119:0/1327920938 >> 172.21.15.119:0/1327920938 conn(0x7efce006a320 msgr2=0x7efce00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.980+0000 7efce5bff700 1 -- 172.21.15.119:0/1327920938 shutdown_connections 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.980+0000 7efce5bff700 1 -- 172.21.15.119:0/1327920938 wait complete. 2023-10-08T02:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.981+0000 7efce5bff700 1 Processor -- start 2023-10-08T02:28:31.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.982+0000 7efce5bff700 1 -- start start 2023-10-08T02:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.982+0000 7efce5bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efce00af2f0 0x7efce0169cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efce5bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce0155c10 0x7efce016ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efce5bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce01578b0 0x7efce01686a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efce5bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce016c510 con 0x7efce01578b0 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efce5bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce016c080 con 0x7efce00af2f0 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efce5bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce016b220 con 0x7efce0155c10 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce01578b0 0x7efce01686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce01578b0 0x7efce01686a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55260/0 (socket says 172.21.15.119:55260) 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 -- 172.21.15.119:0/2019074386 learned_addr learned my addr 172.21.15.119:0/2019074386 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 -- 172.21.15.119:0/2019074386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce0155c10 msgr2=0x7efce016ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce0155c10 0x7efce016ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 -- 172.21.15.119:0/2019074386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efce00af2f0 msgr2=0x7efce0169cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efce00af2f0 0x7efce0169cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 -- 172.21.15.119:0/2019074386 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd4009420 con 0x7efce01578b0 2023-10-08T02:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.983+0000 7efcdffff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce01578b0 0x7efce01686a0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7efcd800b610 tx=0x7efcd8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.984+0000 7efcc77fe700 1 -- 172.21.15.119:0/2019074386 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd8012c80 con 0x7efce01578b0 2023-10-08T02:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.984+0000 7efcc77fe700 1 -- 172.21.15.119:0/2019074386 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd8012e00 con 0x7efce01578b0 2023-10-08T02:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.984+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd8009a20 con 0x7efce01578b0 2023-10-08T02:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.984+0000 7efcc77fe700 1 -- 172.21.15.119:0/2019074386 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd801e6a0 con 0x7efce01578b0 2023-10-08T02:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.984+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce016b750 con 0x7efce01578b0 2023-10-08T02:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.985+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce004c3c0 con 0x7efce01578b0 2023-10-08T02:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.986+0000 7efcc77fe700 1 -- 172.21.15.119:0/2019074386 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efcd801ea10 con 0x7efce01578b0 2023-10-08T02:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.986+0000 7efcc77fe700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efccc066f30 0x7efccc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.987+0000 7efcc77fe700 1 -- 172.21.15.119:0/2019074386 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6291+0+0 (secure 0 0 0) 0x7efcd8086780 con 0x7efce01578b0 2023-10-08T02:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.987+0000 7efcdf7fe700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efccc066f30 0x7efccc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.987+0000 7efcdf7fe700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efccc066f30 0x7efccc0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efcd00015a0 tx=0x7efcd00012c0 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:31.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:31.991+0000 7efcc77fe700 1 -- 172.21.15.119:0/2019074386 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efcd8053700 con 0x7efce01578b0 2023-10-08T02:28:32.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.260+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7efce00678d0 con 0x7efce01578b0 2023-10-08T02:28:32.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.775+0000 7efcc77fe700 1 -- 172.21.15.119:0/2019074386 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2984) v1 ==== 166+0+0 (secure 0 0 0) 0x7efcd8056d20 con 0x7efce01578b0 2023-10-08T02:28:32.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efccc066f30 msgr2=0x7efccc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:32.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efccc066f30 0x7efccc0693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efcd00015a0 tx=0x7efcd00012c0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce01578b0 msgr2=0x7efce01686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:32.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce01578b0 0x7efce01686a0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7efcd800b610 tx=0x7efcd8017000 comp rx=0 tx=0).stop 2023-10-08T02:28:32.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 shutdown_connections 2023-10-08T02:28:32.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efccc066f30 0x7efccc0693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efce01578b0 0x7efce01686a0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efce00af2f0 0x7efce0169cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 --2- 172.21.15.119:0/2019074386 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efce0155c10 0x7efce016ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 >> 172.21.15.119:0/2019074386 conn(0x7efce006a320 msgr2=0x7efce006e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:32.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 shutdown_connections 2023-10-08T02:28:32.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.778+0000 7efce5bff700 1 -- 172.21.15.119:0/2019074386 wait complete. 2023-10-08T02:28:32.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T02:28:32.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.980+0000 7fe4167ff700 1 Processor -- start 2023-10-08T02:28:32.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.980+0000 7fe4167ff700 1 -- start start 2023-10-08T02:28:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.981+0000 7fe4167ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4100ccee0 0x7fe4101561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.981+0000 7fe4167ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100c87c0 0x7fe410156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.981+0000 7fe4167ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe41015b0d0 con 0x7fe4100c83f0 2023-10-08T02:28:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.981+0000 7fe4167ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe410156c50 con 0x7fe4100ccee0 2023-10-08T02:28:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.981+0000 7fe4167ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41015a4c0 con 0x7fe4100c87c0 2023-10-08T02:28:32.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe40f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4100ccee0 0x7fe4101561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe40f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4100ccee0 0x7fe4101561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55262/0 (socket says 172.21.15.119:55262) 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe40f7fe700 1 -- 172.21.15.119:0/3608612812 learned_addr learned my addr 172.21.15.119:0/3608612812 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe40ffff700 1 --1- 172.21.15.119:0/3608612812 >> v1:172.21.15.119:6789/0 conn(0x7fe4100c83f0 0x7fe4100c7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51154/0 (socket says 172.21.15.119:51154) 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe414d9c700 1 --2- 172.21.15.119:0/3608612812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100c87c0 0x7fe410156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe40f7fe700 1 -- 172.21.15.119:0/3608612812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100c87c0 msgr2=0x7fe410156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe40f7fe700 1 --2- 172.21.15.119:0/3608612812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100c87c0 0x7fe410156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe40f7fe700 1 -- 172.21.15.119:0/3608612812 >> v1:172.21.15.119:6789/0 conn(0x7fe4100c83f0 legacy=0x7fe4100c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe40f7fe700 1 -- 172.21.15.119:0/3608612812 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41015a7a0 con 0x7fe4100ccee0 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.982+0000 7fe414d9c700 1 --2- 172.21.15.119:0/3608612812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100c87c0 0x7fe410156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:32.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe40f7fe700 1 --2- 172.21.15.119:0/3608612812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100ccee0 0x7fe4101561d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fe4040099b0 tx=0x7fe404009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295dc653f198fe03 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe40effd700 1 -- 172.21.15.119:0/3608612812 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40400b860 con 0x7fe4100ccee0 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe40effd700 1 -- 172.21.15.119:0/3608612812 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40400b9c0 con 0x7fe4100ccee0 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe40effd700 1 -- 172.21.15.119:0/3608612812 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40400bd60 con 0x7fe4100ccee0 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe4167ff700 1 -- 172.21.15.119:0/3608612812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100ccee0 msgr2=0x7fe4101561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3608612812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100ccee0 0x7fe4101561d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fe4040099b0 tx=0x7fe404009250 comp rx=0 tx=0).stop 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe4167ff700 1 -- 172.21.15.119:0/3608612812 shutdown_connections 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3608612812 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100c87c0 0x7fe410156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3608612812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100ccee0 0x7fe4101561d0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe4167ff700 1 -- 172.21.15.119:0/3608612812 >> 172.21.15.119:0/3608612812 conn(0x7fe410068e10 msgr2=0x7fe4100cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:32.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe4167ff700 1 -- 172.21.15.119:0/3608612812 shutdown_connections 2023-10-08T02:28:32.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.983+0000 7fe4167ff700 1 -- 172.21.15.119:0/3608612812 wait complete. 2023-10-08T02:28:32.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.985+0000 7fe4167ff700 1 Processor -- start 2023-10-08T02:28:32.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.985+0000 7fe4167ff700 1 -- start start 2023-10-08T02:28:32.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe4167ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100c83f0 0x7fe410148c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:32.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe4167ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4100c87c0 0x7fe4100c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:32.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe4167ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100ccee0 0x7fe4100c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe4167ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe410159c50 con 0x7fe4100c83f0 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe4167ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4101597c0 con 0x7fe4100c87c0 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe4167ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4101491a0 con 0x7fe4100ccee0 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100c83f0 0x7fe410148c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100c83f0 0x7fe410148c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55270/0 (socket says 172.21.15.119:55270) 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40ffff700 1 -- 172.21.15.119:0/3099983470 learned_addr learned my addr 172.21.15.119:0/3099983470 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40f7fe700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4100c87c0 0x7fe4100c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe414d9c700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100ccee0 0x7fe4100c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:32.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40ffff700 1 -- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100ccee0 msgr2=0x7fe4100c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:32.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40ffff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100ccee0 0x7fe4100c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40ffff700 1 -- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4100c87c0 msgr2=0x7fe4100c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:32.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40ffff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4100c87c0 0x7fe4100c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:32.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40ffff700 1 -- 172.21.15.119:0/3099983470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4040094f0 con 0x7fe4100c83f0 2023-10-08T02:28:32.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.986+0000 7fe40f7fe700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4100c87c0 0x7fe4100c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:32.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.987+0000 7fe40ffff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100c83f0 0x7fe410148c60 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c7c0 tx=0x7fe400007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:32.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.987+0000 7fe3f7fff700 1 -- 172.21.15.119:0/3099983470 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe400005a70 con 0x7fe4100c83f0 2023-10-08T02:28:32.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.987+0000 7fe3f7fff700 1 -- 172.21.15.119:0/3099983470 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe400005bf0 con 0x7fe4100c83f0 2023-10-08T02:28:32.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.987+0000 7fe3f7fff700 1 -- 172.21.15.119:0/3099983470 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4000070f0 con 0x7fe4100c83f0 2023-10-08T02:28:32.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.987+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe410158bb0 con 0x7fe4100c83f0 2023-10-08T02:28:32.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.987+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe410159170 con 0x7fe4100c83f0 2023-10-08T02:28:32.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.988+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41004c3c0 con 0x7fe4100c83f0 2023-10-08T02:28:32.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.989+0000 7fe3f7fff700 1 -- 172.21.15.119:0/3099983470 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe400007460 con 0x7fe4100c83f0 2023-10-08T02:28:32.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.990+0000 7fe3f7fff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe3fc066f30 0x7fe3fc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:32.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.990+0000 7fe3f7fff700 1 -- 172.21.15.119:0/3099983470 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6291+0+0 (secure 0 0 0) 0x7fe400084280 con 0x7fe4100c83f0 2023-10-08T02:28:32.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.990+0000 7fe40f7fe700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe3fc066f30 0x7fe3fc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:32.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.991+0000 7fe40f7fe700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe3fc066f30 0x7fe3fc0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe4040099b0 tx=0x7fe404017000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:32.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:32.994+0000 7fe3f7fff700 1 -- 172.21.15.119:0/3099983470 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe400051280 con 0x7fe4100c83f0 2023-10-08T02:28:33.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.186+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe41015aa80 con 0x7fe4100c83f0 2023-10-08T02:28:33.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.190+0000 7fe3f7fff700 1 -- 172.21.15.119:0/3099983470 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4000548a0 con 0x7fe4100c83f0 2023-10-08T02:28:33.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe3fc066f30 msgr2=0x7fe3fc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:33.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe3fc066f30 0x7fe3fc0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe4040099b0 tx=0x7fe404017000 comp rx=0 tx=0).stop 2023-10-08T02:28:33.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100c83f0 msgr2=0x7fe410148c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:33.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100c83f0 0x7fe410148c60 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fe40000c7c0 tx=0x7fe400007f70 comp rx=0 tx=0).stop 2023-10-08T02:28:33.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 shutdown_connections 2023-10-08T02:28:33.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe3fc066f30 0x7fe3fc0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4100ccee0 0x7fe4100c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4100c83f0 0x7fe410148c60 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 --2- 172.21.15.119:0/3099983470 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4100c87c0 0x7fe4100c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 >> 172.21.15.119:0/3099983470 conn(0x7fe410068e10 msgr2=0x7fe4100c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:33.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 shutdown_connections 2023-10-08T02:28:33.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.193+0000 7fe4167ff700 1 -- 172.21.15.119:0/3099983470 wait complete. 2023-10-08T02:28:33.211 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:28:33.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:28:33.295 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:28:33.380 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:28:33.381 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T02:28:33.381 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:28:33.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.564+0000 7fe4f1d9d700 1 Processor -- start 2023-10-08T02:28:33.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.564+0000 7fe4f1d9d700 1 -- start start 2023-10-08T02:28:33.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.565+0000 7fe4f1d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec159420 0x7fe4ec0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:33.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.565+0000 7fe4f1d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4ec15d230 0x7fe4ec15d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:33.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.565+0000 7fe4f1d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ec15dbd0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.565+0000 7fe4f1d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4ec0b2980 con 0x7fe4ec15d230 2023-10-08T02:28:33.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.565+0000 7fe4f1d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0b11d0 con 0x7fe4ec159420 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f159c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4ec15d230 0x7fe4ec15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f159c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4ec15d230 0x7fe4ec15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55274/0 (socket says 172.21.15.119:55274) 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f159c700 1 -- 172.21.15.119:0/467130020 learned_addr learned my addr 172.21.15.119:0/467130020 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f0d9b700 1 --2- 172.21.15.119:0/467130020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec159420 0x7fe4ec0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f159c700 1 -- 172.21.15.119:0/467130020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec159420 msgr2=0x7fe4ec0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f159c700 1 --2- 172.21.15.119:0/467130020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec159420 0x7fe4ec0b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f159c700 1 -- 172.21.15.119:0/467130020 >> v1:172.21.15.184:6789/0 conn(0x7fe4ec1597f0 legacy=0x7fe4ec0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f159c700 1 -- 172.21.15.119:0/467130020 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec0b14b0 con 0x7fe4ec15d230 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4f159c700 1 --2- 172.21.15.119:0/467130020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec15d230 0x7fe4ec15d690 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009f20 tx=0x7fe4e4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4102bf7b3585af server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:33.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.566+0000 7fe4eb7fe700 1 -- 172.21.15.119:0/467130020 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e400e070 con 0x7fe4ec15d230 2023-10-08T02:28:33.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4eb7fe700 1 -- 172.21.15.119:0/467130020 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4e400ba80 con 0x7fe4ec15d230 2023-10-08T02:28:33.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4eb7fe700 1 -- 172.21.15.119:0/467130020 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4e40092e0 con 0x7fe4ec15d230 2023-10-08T02:28:33.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/467130020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec15d230 msgr2=0x7fe4ec15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:33.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/467130020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec15d230 0x7fe4ec15d690 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4009f20 tx=0x7fe4e4009840 comp rx=0 tx=0).stop 2023-10-08T02:28:33.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/467130020 shutdown_connections 2023-10-08T02:28:33.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/467130020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec15d230 0x7fe4ec15d690 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/467130020 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec159420 0x7fe4ec0b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/467130020 >> 172.21.15.119:0/467130020 conn(0x7fe4ec0d0660 msgr2=0x7fe4ec0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:33.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/467130020 shutdown_connections 2023-10-08T02:28:33.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.567+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/467130020 wait complete. 2023-10-08T02:28:33.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.569+0000 7fe4f1d9d700 1 Processor -- start 2023-10-08T02:28:33.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.569+0000 7fe4f1d9d700 1 -- start start 2023-10-08T02:28:33.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f1d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ec159420 0x7fe4ec147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f1d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec1597f0 0x7fe4ec0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f1d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec15d230 0x7fe4ec0c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f1d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0cb840 con 0x7fe4ec1597f0 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f1d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0caf10 con 0x7fe4ec159420 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f1d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0cb1f0 con 0x7fe4ec15d230 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4ebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec1597f0 0x7fe4ec0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4ebfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec1597f0 0x7fe4ec0cbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55282/0 (socket says 172.21.15.119:55282) 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4ebfff700 1 -- 172.21.15.119:0/3201996269 learned_addr learned my addr 172.21.15.119:0/3201996269 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:33.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f0d9b700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ec159420 0x7fe4ec147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f159c700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec15d230 0x7fe4ec0c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4ebfff700 1 -- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec15d230 msgr2=0x7fe4ec0c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4ebfff700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec15d230 0x7fe4ec0c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4ebfff700 1 -- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ec159420 msgr2=0x7fe4ec147160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4ebfff700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ec159420 0x7fe4ec147160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4ebfff700 1 -- 172.21.15.119:0/3201996269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e4009ae0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.570+0000 7fe4f159c700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec15d230 0x7fe4ec0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.571+0000 7fe4ebfff700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec1597f0 0x7fe4ec0cbcd0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800c590 tx=0x7fe4d800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.571+0000 7fe4e8ff9700 1 -- 172.21.15.119:0/3201996269 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8011590 con 0x7fe4ec1597f0 2023-10-08T02:28:33.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.571+0000 7fe4e8ff9700 1 -- 172.21.15.119:0/3201996269 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d8011710 con 0x7fe4ec1597f0 2023-10-08T02:28:33.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.571+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ec0cb4d0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.571+0000 7fe4e8ff9700 1 -- 172.21.15.119:0/3201996269 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d8011ad0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.571+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec0cab30 con 0x7fe4ec1597f0 2023-10-08T02:28:33.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.572+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec04c3c0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.573+0000 7fe4e8ff9700 1 -- 172.21.15.119:0/3201996269 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe4d801b3b0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.574+0000 7fe4e8ff9700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe4d0066f30 0x7fe4d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.574+0000 7fe4e8ff9700 1 -- 172.21.15.119:0/3201996269 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6291+0+0 (secure 0 0 0) 0x7fe4d8083b50 con 0x7fe4ec1597f0 2023-10-08T02:28:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.574+0000 7fe4f0d9b700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe4d0066f30 0x7fe4d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:33.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.574+0000 7fe4f0d9b700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe4d0066f30 0x7fe4d00693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0006fd0 tx=0x7fe4e0008040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:33.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.577+0000 7fe4e8ff9700 1 -- 172.21.15.119:0/3201996269 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe4d8050ad0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.816+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4ec15dbd0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.816+0000 7fe4e8ff9700 1 -- 172.21.15.119:0/3201996269 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe4ec15dbd0 con 0x7fe4ec1597f0 2023-10-08T02:28:33.817 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:33.817 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7409,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3771852147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3771852147},{"type":"v1","addr":"172.21.15.184:6834","nonce":3771852147}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/524755843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":524755843},{"type":"v1","addr":"172.21.15.184:6835","nonce":524755843}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":8839,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/907518617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":907518617},{"type":"v1","addr":"172.21.15.119:6835","nonce":907518617}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81},{"gid":9144,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/295310163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":295310163},{"type":"v1","addr":"172.21.15.119:6833","nonce":295310163}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[]} 2023-10-08T02:28:33.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe4d0066f30 msgr2=0x7fe4d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:33.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe4d0066f30 0x7fe4d00693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe4e0006fd0 tx=0x7fe4e0008040 comp rx=0 tx=0).stop 2023-10-08T02:28:33.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec1597f0 msgr2=0x7fe4ec0cbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec1597f0 0x7fe4ec0cbcd0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fe4d800c590 tx=0x7fe4d800c8a0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 shutdown_connections 2023-10-08T02:28:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe4d0066f30 0x7fe4d00693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe4ec15d230 0x7fe4ec0c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe4ec159420 0x7fe4ec147160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 --2- 172.21.15.119:0/3201996269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4ec1597f0 0x7fe4ec0cbcd0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:33.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.818+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 >> 172.21.15.119:0/3201996269 conn(0x7fe4ec0d0660 msgr2=0x7fe4ec0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:33.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.819+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 shutdown_connections 2023-10-08T02:28:33.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:33.819+0000 7fe4f1d9d700 1 -- 172.21.15.119:0/3201996269 wait complete. 2023-10-08T02:28:33.821 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 81 2023-10-08T02:28:33.837 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T02:28:33.837 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T02:28:33.837 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T02:28:33.837 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:28:33.838 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T02:28:33.838 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T02:28:33.838 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T02:28:33.838 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:28:33.838 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T02:28:33.839 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T02:28:33.839 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T02:28:33.839 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:28:33.839 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T02:28:33.840 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T02:28:33.840 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T02:28:33.840 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:28:33.852 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T02:28:33.852 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T02:28:33.853 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T02:28:33.853 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T02:28:33.856 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T02:28:33.857 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T02:28:33.857 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T02:28:33.858 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T02:28:33.896 INFO:tasks.ceph.mds.a:Started 2023-10-08T02:28:33.896 DEBUG:teuthology.parallel:result is None 2023-10-08T02:28:33.899 INFO:tasks.ceph.mds.c:Started 2023-10-08T02:28:33.899 DEBUG:teuthology.parallel:result is None 2023-10-08T02:28:33.901 INFO:tasks.ceph.mds.b:Started 2023-10-08T02:28:33.902 DEBUG:teuthology.parallel:result is None 2023-10-08T02:28:33.904 INFO:tasks.ceph.mds.d:Started 2023-10-08T02:28:33.904 DEBUG:teuthology.parallel:result is None 2023-10-08T02:28:33.905 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T02:28:34.033 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T02:28:34.033 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T02:28:34.040 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T02:28:34.040 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T02:28:34.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.097+0000 7f45b339d700 1 Processor -- start 2023-10-08T02:28:34.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.097+0000 7f45b339d700 1 -- start start 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac156310 0x7f45ac0c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b339d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f45ac0ae710 0x7f45ac0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b339d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac0ca860 con 0x7f45ac1566e0 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b339d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45ac0cab40 con 0x7f45ac0ae710 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0cae20 con 0x7f45ac156310 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b1b9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f45ac1566e0 0x7f45ac0ae000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51170/0 (socket says 172.21.15.119:51170) 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b1b9a700 1 -- 172.21.15.119:0/3842496999 learned_addr learned my addr 172.21.15.119:0/3842496999 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b239b700 1 --2- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac156310 0x7f45ac0c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:34.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.098+0000 7f45b2b9c700 1 --2- 172.21.15.119:0/3842496999 >> v2:172.21.15.184:3300/0 conn(0x7f45ac0ae710 0x7f45ac0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:34.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b1399700 1 -- 172.21.15.119:0/3842496999 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460673954 0 0) 0x7f45ac0ca860 con 0x7f45ac1566e0 2023-10-08T02:28:34.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b1399700 1 -- 172.21.15.119:0/3842496999 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4598003570 con 0x7f45ac1566e0 2023-10-08T02:28:34.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b2b9c700 1 -- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac156310 msgr2=0x7f45ac0c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:34.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b2b9c700 1 --2- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac156310 0x7f45ac0c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:34.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b2b9c700 1 -- 172.21.15.119:0/3842496999 >> v1:172.21.15.119:6789/0 conn(0x7f45ac1566e0 legacy=0x7f45ac0ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b2b9c700 1 -- 172.21.15.119:0/3842496999 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac0cb100 con 0x7f45ac0ae710 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b239b700 1 --2- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac156310 0x7f45ac0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b2b9c700 1 --2- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac0ae710 0x7f45ac0c6830 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009e60 tx=0x7f45a4009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c41ca8135a8300 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b1399700 1 -- 172.21.15.119:0/3842496999 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400e070 con 0x7f45ac0ae710 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b1399700 1 -- 172.21.15.119:0/3842496999 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a400bab0 con 0x7f45ac0ae710 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.099+0000 7f45b1399700 1 -- 172.21.15.119:0/3842496999 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400be70 con 0x7f45ac0ae710 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.100+0000 7f45b339d700 1 -- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac0ae710 msgr2=0x7f45ac0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.100+0000 7f45b339d700 1 --2- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac0ae710 0x7f45ac0c6830 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009e60 tx=0x7f45a4009780 comp rx=0 tx=0).stop 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.100+0000 7f45b339d700 1 -- 172.21.15.119:0/3842496999 shutdown_connections 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.100+0000 7f45b339d700 1 --2- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac0ae710 0x7f45ac0c6830 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:34.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.100+0000 7f45b339d700 1 --2- 172.21.15.119:0/3842496999 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac156310 0x7f45ac0c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:34.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.100+0000 7f45b339d700 1 -- 172.21.15.119:0/3842496999 >> 172.21.15.119:0/3842496999 conn(0x7f45ac0cc1f0 msgr2=0x7f45ac0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:34.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.100+0000 7f45b339d700 1 -- 172.21.15.119:0/3842496999 shutdown_connections 2023-10-08T02:28:34.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.100+0000 7f45b339d700 1 -- 172.21.15.119:0/3842496999 wait complete. 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.101+0000 7f45b339d700 1 Processor -- start 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.102+0000 7f45b339d700 1 -- start start 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.102+0000 7f45b339d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac0ae710 0x7f45ac143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.102+0000 7f45b339d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac156310 0x7f45ac0b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.102+0000 7f45b339d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ac1566e0 0x7f45ac169870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.102+0000 7f45b339d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac15c230 con 0x7f45ac1566e0 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.102+0000 7f45b339d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac1441d0 con 0x7f45ac156310 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.102+0000 7f45b339d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac15b620 con 0x7f45ac0ae710 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ac1566e0 0x7f45ac169870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:34.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ac1566e0 0x7f45ac169870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55322/0 (socket says 172.21.15.119:55322) 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 -- 172.21.15.119:0/1875092776 learned_addr learned my addr 172.21.15.119:0/1875092776 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b1b9a700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac156310 0x7f45ac0b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b239b700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac0ae710 0x7f45ac143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 -- 172.21.15.119:0/1875092776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac0ae710 msgr2=0x7f45ac143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac0ae710 0x7f45ac143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 -- 172.21.15.119:0/1875092776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac156310 msgr2=0x7f45ac0b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac156310 0x7f45ac0b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 -- 172.21.15.119:0/1875092776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a8009420 con 0x7f45ac1566e0 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.103+0000 7f45b2b9c700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ac1566e0 0x7f45ac169870 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f45a4013000 tx=0x7f45a4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.104+0000 7f45a27fc700 1 -- 172.21.15.119:0/1875092776 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4012e10 con 0x7f45ac1566e0 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.104+0000 7f45a27fc700 1 -- 172.21.15.119:0/1875092776 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a4027470 con 0x7f45ac1566e0 2023-10-08T02:28:34.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.104+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a4009a20 con 0x7f45ac1566e0 2023-10-08T02:28:34.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.104+0000 7f45a27fc700 1 -- 172.21.15.119:0/1875092776 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4027830 con 0x7f45ac1566e0 2023-10-08T02:28:34.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.104+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac15bbe0 con 0x7f45ac1566e0 2023-10-08T02:28:34.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.105+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac15c3a0 con 0x7f45ac1566e0 2023-10-08T02:28:34.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.106+0000 7f45a27fc700 1 -- 172.21.15.119:0/1875092776 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45a4024070 con 0x7f45ac1566e0 2023-10-08T02:28:34.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.107+0000 7f45a27fc700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4590066f30 0x7f45900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:34.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.107+0000 7f45a27fc700 1 -- 172.21.15.119:0/1875092776 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6291+0+0 (secure 0 0 0) 0x7f45a40871d0 con 0x7f45ac1566e0 2023-10-08T02:28:34.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.107+0000 7f45b239b700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4590066f30 0x7f45900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:34.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.107+0000 7f45b239b700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4590066f30 0x7f45900693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009f40 tx=0x7f45a80091a0 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:34.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.111+0000 7f45a27fc700 1 -- 172.21.15.119:0/1875092776 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45a40541f0 con 0x7f45ac1566e0 2023-10-08T02:28:34.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:34.320+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f45ac04c3c0 con 0x7f45ac1566e0 2023-10-08T02:28:35.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.307+0000 7f45a27fc700 1 -- 172.21.15.119:0/1875092776 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v187) v1 ==== 97+0+0 (secure 0 0 0) 0x7f45a4057810 con 0x7f45ac1566e0 2023-10-08T02:28:35.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.309+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4590066f30 msgr2=0x7f45900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:35.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.309+0000 7f45b339d700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4590066f30 0x7f45900693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f45a8009f40 tx=0x7f45a80091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.309+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ac1566e0 msgr2=0x7f45ac169870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.309+0000 7f45b339d700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ac1566e0 0x7f45ac169870 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f45a4013000 tx=0x7f45a4019000 comp rx=0 tx=0).stop 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.310+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 shutdown_connections 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.310+0000 7f45b339d700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4590066f30 0x7f45900693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.310+0000 7f45b339d700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45ac0ae710 0x7f45ac143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.310+0000 7f45b339d700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45ac1566e0 0x7f45ac169870 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.310+0000 7f45b339d700 1 --2- 172.21.15.119:0/1875092776 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45ac156310 0x7f45ac0b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.310+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 >> 172.21.15.119:0/1875092776 conn(0x7f45ac0cc1f0 msgr2=0x7f45ac151d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.310+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 shutdown_connections 2023-10-08T02:28:35.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.310+0000 7f45b339d700 1 -- 172.21.15.119:0/1875092776 wait complete. 2023-10-08T02:28:35.313 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T02:28:35.330 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T02:28:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.513+0000 7fe2c1bff700 1 Processor -- start 2023-10-08T02:28:35.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.513+0000 7fe2c1bff700 1 -- start start 2023-10-08T02:28:35.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2c1bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc0b1420 0x7fe2bc0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2c1bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2bc0ccf90 0x7fe2bc0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2c1bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0cc300 con 0x7fe2bc0cbf30 2023-10-08T02:28:35.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2c1bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2bc156bf0 con 0x7fe2bc0ccf90 2023-10-08T02:28:35.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2c1bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0ae950 con 0x7fe2bc0b1420 2023-10-08T02:28:35.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2bb7fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2bc0cbf30 0x7fe2bc0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51610/0 (socket says 172.21.15.119:51610) 2023-10-08T02:28:35.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2bb7fe700 1 -- 172.21.15.119:0/633498408 learned_addr learned my addr 172.21.15.119:0/633498408 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:35.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc0b1420 0x7fe2bc0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2bbfff700 1 --2- 172.21.15.119:0/633498408 >> v2:172.21.15.184:3300/0 conn(0x7fe2bc0ccf90 0x7fe2bc0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186334079 0 0) 0x7fe2bc0cc300 con 0x7fe2bc0cbf30 2023-10-08T02:28:35.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.515+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a8003570 con 0x7fe2bc0cbf30 2023-10-08T02:28:35.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1824213497 0 0) 0x7fe2a8003570 con 0x7fe2bc0cbf30 2023-10-08T02:28:35.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc0b1420 msgr2=0x7fe2bc0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:35.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc0b1420 0x7fe2bc0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 >> v2:172.21.15.184:3300/0 conn(0x7fe2bc0ccf90 msgr2=0x7fe2bc0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:35.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 --2- 172.21.15.119:0/633498408 >> v2:172.21.15.184:3300/0 conn(0x7fe2bc0ccf90 0x7fe2bc0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0ad370 con 0x7fe2bc0cbf30 2023-10-08T02:28:35.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fe2ac001470 con 0x7fe2bc0cbf30 2023-10-08T02:28:35.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 >> v1:172.21.15.119:6789/0 conn(0x7fe2bc0cbf30 legacy=0x7fe2bc0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:35.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc0b1420 0x7fe2a8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8004d50 0x7fe2a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc156bf0 con 0x7fe2a8004010 2023-10-08T02:28:35.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0ae950 con 0x7fe2a8004d50 2023-10-08T02:28:35.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a800dcc0 con 0x7fe2bc0b1420 2023-10-08T02:28:35.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2bb7fe700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc0b1420 0x7fe2a8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2bbfff700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8004d50 0x7fe2a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2b3fff700 1 -- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc0b1420 msgr2=0x7fe2a8003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:35.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.516+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc0b1420 0x7fe2a8003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2b3fff700 1 -- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8004d50 msgr2=0x7fe2a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:35.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2a8004d50 0x7fe2a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2b3fff700 1 -- 172.21.15.119:0/633498408 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8003570 con 0x7fe2a8004010 2023-10-08T02:28:35.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2a8004810 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009c00 tx=0x7fe2a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfb2b07c7a2a96a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:35.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400e710 con 0x7fe2a8004010 2023-10-08T02:28:35.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a400e870 con 0x7fe2a8004010 2023-10-08T02:28:35.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2baffd700 1 -- 172.21.15.119:0/633498408 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a400ec30 con 0x7fe2a8004010 2023-10-08T02:28:35.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2c1bff700 1 -- 172.21.15.119:0/633498408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 msgr2=0x7fe2a8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:35.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2c1bff700 1 --2- 172.21.15.119:0/633498408 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2a8004810 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009c00 tx=0x7fe2a40091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.517+0000 7fe2bbfff700 1 -- 172.21.15.119:0/633498408 reap_dead start 2023-10-08T02:28:35.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.518+0000 7fe2c1bff700 1 -- 172.21.15.119:0/633498408 shutdown_connections 2023-10-08T02:28:35.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.518+0000 7fe2c1bff700 1 -- 172.21.15.119:0/633498408 >> 172.21.15.119:0/633498408 conn(0x7fe2bc06a320 msgr2=0x7fe2bc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:35.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.518+0000 7fe2c1bff700 1 -- 172.21.15.119:0/633498408 shutdown_connections 2023-10-08T02:28:35.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.518+0000 7fe2c1bff700 1 -- 172.21.15.119:0/633498408 wait complete. 2023-10-08T02:28:35.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.519+0000 7fe2c1bff700 1 Processor -- start 2023-10-08T02:28:35.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.519+0000 7fe2c1bff700 1 -- start start 2023-10-08T02:28:35.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2c1bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2bc15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2c1bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc158670 0x7fe2bc158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2c1bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2bc16c710 0x7fe2bc16cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2c1bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc16a080 con 0x7fe2a8004010 2023-10-08T02:28:35.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2c1bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc16d0b0 con 0x7fe2bc16c710 2023-10-08T02:28:35.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2c1bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc1591e0 con 0x7fe2bc158670 2023-10-08T02:28:35.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2bb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2bc15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2bb7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2bc15e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48746/0 (socket says 172.21.15.119:48746) 2023-10-08T02:28:35.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.520+0000 7fe2bb7fe700 1 -- 172.21.15.119:0/4051083410 learned_addr learned my addr 172.21.15.119:0/4051083410 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:35.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2bbfff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2bc16c710 0x7fe2bc16cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc158670 0x7fe2bc158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2bb7fe700 1 -- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc158670 msgr2=0x7fe2bc158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:35.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2bb7fe700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc158670 0x7fe2bc158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2bb7fe700 1 -- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2bc16c710 msgr2=0x7fe2bc16cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:35.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2bb7fe700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2bc16c710 0x7fe2bc16cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2bb7fe700 1 -- 172.21.15.119:0/4051083410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc1594c0 con 0x7fe2a8004010 2023-10-08T02:28:35.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc158670 0x7fe2bc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:35.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2bb7fe700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2bc15e7c0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac007390 tx=0x7fe2bc0c8fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:35.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2b37fe700 1 -- 172.21.15.119:0/4051083410 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac018030 con 0x7fe2a8004010 2023-10-08T02:28:35.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2b37fe700 1 -- 172.21.15.119:0/4051083410 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac007750 con 0x7fe2a8004010 2023-10-08T02:28:35.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc1597a0 con 0x7fe2a8004010 2023-10-08T02:28:35.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2b37fe700 1 -- 172.21.15.119:0/4051083410 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac00fa80 con 0x7fe2a8004010 2023-10-08T02:28:35.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.521+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc159d60 con 0x7fe2a8004010 2023-10-08T02:28:35.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.522+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc159f50 con 0x7fe2a8004010 2023-10-08T02:28:35.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.523+0000 7fe2b37fe700 1 -- 172.21.15.119:0/4051083410 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2ac01b070 con 0x7fe2a8004010 2023-10-08T02:28:35.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.524+0000 7fe2b37fe700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe29c066f30 0x7fe29c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:35.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.524+0000 7fe2b37fe700 1 -- 172.21.15.119:0/4051083410 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5345+0+0 (secure 0 0 0) 0x7fe2ac019070 con 0x7fe2a8004010 2023-10-08T02:28:35.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.524+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe29c066f30 0x7fe29c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:35.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.525+0000 7fe2b3fff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe29c066f30 0x7fe29c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40091e0 tx=0x7fe2a401a000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:35.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.528+0000 7fe2b37fe700 1 -- 172.21.15.119:0/4051083410 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2ac0515c0 con 0x7fe2a8004010 2023-10-08T02:28:35.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.797+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2bc04c3c0 con 0x7fe2a8004010 2023-10-08T02:28:35.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.797+0000 7fe2b37fe700 1 -- 172.21.15.119:0/4051083410 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v50) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe2ac054be0 con 0x7fe2a8004010 2023-10-08T02:28:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:35.798 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:28:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T02:28:35.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.799 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T02:28:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T02:28:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:28:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:28:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:28:35.800 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:28:35.801 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:28:35.802 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.803 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.804 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.805 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.806 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.807 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T02:28:35.808 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T02:28:35.809 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.810 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:28:35.811 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:28:35.812 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:28:35.812 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:28:35.812 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.812 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.812 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.812 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T02:28:35.812 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T02:28:35.812 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T02:28:35.813 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T02:28:35.814 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T02:28:35.815 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:28:35.816 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:28:35.817 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T02:28:35.817 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:28:35.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.817 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:35.817 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T02:28:35.817 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:28:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe29c066f30 msgr2=0x7fe29c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe29c066f30 0x7fe29c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe2a40091e0 tx=0x7fe2a401a000 comp rx=0 tx=0).stop 2023-10-08T02:28:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 msgr2=0x7fe2bc15e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2bc15e7c0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac007390 tx=0x7fe2bc0c8fd0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 shutdown_connections 2023-10-08T02:28:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe29c066f30 0x7fe29c0693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2bc16c710 0x7fe2bc16cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2a8004010 0x7fe2bc15e7c0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 --2- 172.21.15.119:0/4051083410 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2bc158670 0x7fe2bc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:35.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 >> 172.21.15.119:0/4051083410 conn(0x7fe2bc06a320 msgr2=0x7fe2bc0ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:35.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 shutdown_connections 2023-10-08T02:28:35.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:35.800+0000 7fe2c1bff700 1 -- 172.21.15.119:0/4051083410 wait complete. 2023-10-08T02:28:35.820 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:28:36.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.001+0000 7fb9f53ff700 1 Processor -- start 2023-10-08T02:28:36.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.001+0000 7fb9f53ff700 1 -- start start 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.002+0000 7fb9f53ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb9f00b1c20 0x7fb9f01572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.002+0000 7fb9f53ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb9f0157820 0x7fb9f00abf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9f53ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f0157c60 con 0x7fb9f00b1850 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9f53ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f006fc30 con 0x7fb9f00b1c20 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9f53ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f00cc260 con 0x7fb9f0157820 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9e7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb9f00b1c20 0x7fb9f01572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9e7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb9f00b1c20 0x7fb9f01572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48752/0 (socket says 172.21.15.119:48752) 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9e7fff700 1 -- 172.21.15.119:0/516810884 learned_addr learned my addr 172.21.15.119:0/516810884 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9ef7fe700 1 --2- 172.21.15.119:0/516810884 >> v2:172.21.15.184:3300/0 conn(0x7fb9f0157820 0x7fb9f00abf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9e7fff700 1 -- 172.21.15.119:0/516810884 >> v2:172.21.15.184:3300/0 conn(0x7fb9f0157820 msgr2=0x7fb9f00abf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9e7fff700 1 --2- 172.21.15.119:0/516810884 >> v2:172.21.15.184:3300/0 conn(0x7fb9f0157820 0x7fb9f00abf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9e7fff700 1 -- 172.21.15.119:0/516810884 >> v1:172.21.15.184:6789/0 conn(0x7fb9f00b1850 legacy=0x7fb9f00ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.003+0000 7fb9e7fff700 1 -- 172.21.15.119:0/516810884 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00cb300 con 0x7fb9f00b1c20 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.004+0000 7fb9e7fff700 1 --2- 172.21.15.119:0/516810884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f01572e0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00cc160 tx=0x7fb9d8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8306b7965023fa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.004+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/516810884 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8009e70 con 0x7fb9f00b1c20 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.004+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/516810884 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d800b860 con 0x7fb9f00b1c20 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.004+0000 7fb9ee7fc700 1 -- 172.21.15.119:0/516810884 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800bc00 con 0x7fb9f00b1c20 2023-10-08T02:28:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.004+0000 7fb9f53ff700 1 -- 172.21.15.119:0/516810884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 msgr2=0x7fb9f01572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.004+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/516810884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f01572e0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00cc160 tx=0x7fb9d8009250 comp rx=0 tx=0).stop 2023-10-08T02:28:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.005+0000 7fb9f53ff700 1 -- 172.21.15.119:0/516810884 shutdown_connections 2023-10-08T02:28:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.005+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/516810884 >> v2:172.21.15.184:3300/0 conn(0x7fb9f0157820 0x7fb9f00abf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.005+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/516810884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f01572e0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.005+0000 7fb9f53ff700 1 -- 172.21.15.119:0/516810884 >> 172.21.15.119:0/516810884 conn(0x7fb9f006a320 msgr2=0x7fb9f00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.005+0000 7fb9f53ff700 1 -- 172.21.15.119:0/516810884 shutdown_connections 2023-10-08T02:28:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.005+0000 7fb9f53ff700 1 -- 172.21.15.119:0/516810884 wait complete. 2023-10-08T02:28:36.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.006+0000 7fb9f53ff700 1 Processor -- start 2023-10-08T02:28:36.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.006+0000 7fb9f53ff700 1 -- start start 2023-10-08T02:28:36.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9f53ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9f00b1850 0x7fb9f0151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9f53ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9f53ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9f0157820 0x7fb9f00af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9f53ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00aa9a0 con 0x7fb9f00b1c20 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9f53ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00a9e20 con 0x7fb9f0157820 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9f53ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00aa100 con 0x7fb9f00b1850 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9e7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9e7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f015b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48758/0 (socket says 172.21.15.119:48758) 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9e7fff700 1 -- 172.21.15.119:0/1590626828 learned_addr learned my addr 172.21.15.119:0/1590626828 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9ef7fe700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9f0157820 0x7fb9f00af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.007+0000 7fb9eeffd700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9f00b1850 0x7fb9f0151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e7fff700 1 -- 172.21.15.119:0/1590626828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9f00b1850 msgr2=0x7fb9f0151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e7fff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9f00b1850 0x7fb9f0151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e7fff700 1 -- 172.21.15.119:0/1590626828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9f0157820 msgr2=0x7fb9f00af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e7fff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9f0157820 0x7fb9f00af3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e7fff700 1 -- 172.21.15.119:0/1590626828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d80094f0 con 0x7fb9f00b1c20 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e7fff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f015b7b0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0064290 tx=0x7fb9d8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e6ffd700 1 -- 172.21.15.119:0/1590626828 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d800b9d0 con 0x7fb9f00b1c20 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e6ffd700 1 -- 172.21.15.119:0/1590626828 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d800bb30 con 0x7fb9f00b1c20 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f00aa3e0 con 0x7fb9f00b1c20 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9e6ffd700 1 -- 172.21.15.119:0/1590626828 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8025670 con 0x7fb9f00b1c20 2023-10-08T02:28:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.008+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f0151980 con 0x7fb9f00b1c20 2023-10-08T02:28:36.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.009+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f004c3c0 con 0x7fb9f00b1c20 2023-10-08T02:28:36.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.010+0000 7fb9e6ffd700 1 -- 172.21.15.119:0/1590626828 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb9d8023070 con 0x7fb9f00b1c20 2023-10-08T02:28:36.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.011+0000 7fb9e6ffd700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb9d0066fb0 0x7fb9d0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.011+0000 7fb9e6ffd700 1 -- 172.21.15.119:0/1590626828 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5345+0+0 (secure 0 0 0) 0x7fb9d80858f0 con 0x7fb9f00b1c20 2023-10-08T02:28:36.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.011+0000 7fb9eeffd700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb9d0066fb0 0x7fb9d0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.012+0000 7fb9eeffd700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb9d0066fb0 0x7fb9d0069460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0003860 tx=0x7fb9e0009040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.016+0000 7fb9e6ffd700 1 -- 172.21.15.119:0/1590626828 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb9d8052c20 con 0x7fb9f00b1c20 2023-10-08T02:28:36.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.208+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9f00cb5e0 con 0x7fb9f00b1c20 2023-10-08T02:28:36.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.209+0000 7fb9e6ffd700 1 -- 172.21.15.119:0/1590626828 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+12951 (secure 0 0 0) 0x7fb9d8056240 con 0x7fb9f00b1c20 2023-10-08T02:28:36.210 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:36.210 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":187,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:28:35.304071+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb9d0066fb0 msgr2=0x7fb9d0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb9d0066fb0 0x7fb9d0069460 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb9e0003860 tx=0x7fb9e0009040 comp rx=0 tx=0).stop 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 msgr2=0x7fb9f015b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f015b7b0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0064290 tx=0x7fb9d8017000 comp rx=0 tx=0).stop 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 shutdown_connections 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb9d0066fb0 0x7fb9d0069460 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb9f00b1850 0x7fb9f0151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb9f0157820 0x7fb9f00af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 --2- 172.21.15.119:0/1590626828 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb9f00b1c20 0x7fb9f015b7b0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 >> 172.21.15.119:0/1590626828 conn(0x7fb9f006a320 msgr2=0x7fb9f01532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:36.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 shutdown_connections 2023-10-08T02:28:36.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.212+0000 7fb9f53ff700 1 -- 172.21.15.119:0/1590626828 wait complete. 2023-10-08T02:28:36.230 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T02:28:36.230 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:28:36.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.414+0000 7f49a3fff700 1 Processor -- start 2023-10-08T02:28:36.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.414+0000 7f49a3fff700 1 -- start start 2023-10-08T02:28:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c0c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a3fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f499c0ccee0 0x7f499c1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a3fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499c15b0d0 con 0x7f499c0c87c0 2023-10-08T02:28:36.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a3fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499c0c7a60 con 0x7f499c0ccee0 2023-10-08T02:28:36.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c15a4c0 con 0x7f499c0c83f0 2023-10-08T02:28:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a159a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f499c0ccee0 0x7f499c1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a259c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f499c0c87c0 0x7f499c06eae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:35852/0 (socket says 172.21.15.119:35852) 2023-10-08T02:28:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a259c700 1 -- 172.21.15.119:0/2094138027 learned_addr learned my addr 172.21.15.119:0/2094138027 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.416+0000 7f49a1d9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c0c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a0d99700 1 -- 172.21.15.119:0/2094138027 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966469855 0 0) 0x7f499c15b0d0 con 0x7f499c0c87c0 2023-10-08T02:28:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a0d99700 1 -- 172.21.15.119:0/2094138027 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4988003570 con 0x7f499c0c87c0 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a159a700 1 -- 172.21.15.119:0/2094138027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 msgr2=0x7f499c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a159a700 1 --2- 172.21.15.119:0/2094138027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c0c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a159a700 1 -- 172.21.15.119:0/2094138027 >> v1:172.21.15.184:6789/0 conn(0x7f499c0c87c0 legacy=0x7f499c06eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a159a700 1 -- 172.21.15.119:0/2094138027 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c15a7a0 con 0x7f499c0ccee0 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a1d9b700 1 --2- 172.21.15.119:0/2094138027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a159a700 1 --2- 172.21.15.119:0/2094138027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 0x7f499c1561d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f498c0098e0 tx=0x7f498c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b3b0f4a37e6ffa0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a0d99700 1 -- 172.21.15.119:0/2094138027 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c011630 con 0x7f499c0ccee0 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a0d99700 1 -- 172.21.15.119:0/2094138027 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c011790 con 0x7f499c0ccee0 2023-10-08T02:28:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.417+0000 7f49a0d99700 1 -- 172.21.15.119:0/2094138027 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c011b30 con 0x7f499c0ccee0 2023-10-08T02:28:36.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.418+0000 7f49a3fff700 1 -- 172.21.15.119:0/2094138027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 msgr2=0x7f499c1561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.418+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2094138027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 0x7f499c1561d0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f498c0098e0 tx=0x7f498c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.418+0000 7f49a3fff700 1 -- 172.21.15.119:0/2094138027 shutdown_connections 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.418+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2094138027 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c0c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.418+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2094138027 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 0x7f499c1561d0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.418+0000 7f49a3fff700 1 -- 172.21.15.119:0/2094138027 >> 172.21.15.119:0/2094138027 conn(0x7f499c068e10 msgr2=0x7f499c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.418+0000 7f49a3fff700 1 -- 172.21.15.119:0/2094138027 shutdown_connections 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.418+0000 7f49a3fff700 1 -- 172.21.15.119:0/2094138027 wait complete. 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.419+0000 7f49a3fff700 1 Processor -- start 2023-10-08T02:28:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.420+0000 7f49a3fff700 1 -- start start 2023-10-08T02:28:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.420+0000 7f49a3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c13ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.420+0000 7f49a3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f499c0c87c0 0x7f499c0c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 0x7f499c0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0c3910 con 0x7f499c0c87c0 2023-10-08T02:28:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a3fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c0b0a70 con 0x7f499c0ccee0 2023-10-08T02:28:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c13f4c0 con 0x7f499c0c83f0 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f499c0c87c0 0x7f499c0c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a159a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f499c0c87c0 0x7f499c0c3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48774/0 (socket says 172.21.15.119:48774) 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a159a700 1 -- 172.21.15.119:0/2792904816 learned_addr learned my addr 172.21.15.119:0/2792904816 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a259c700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 0x7f499c0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a1d9b700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c13ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a159a700 1 -- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 msgr2=0x7f499c13ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a159a700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c13ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a159a700 1 -- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 msgr2=0x7f499c0c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a159a700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 0x7f499c0c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a159a700 1 -- 172.21.15.119:0/2792904816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4998009420 con 0x7f499c0c87c0 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.421+0000 7f49a259c700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 0x7f499c0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.422+0000 7f49a159a700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f499c0c87c0 0x7f499c0c3cb0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f499c063d40 tx=0x7f498c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.422+0000 7f4991ffb700 1 -- 172.21.15.119:0/2792904816 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c011b10 con 0x7f499c0c87c0 2023-10-08T02:28:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.422+0000 7f4991ffb700 1 -- 172.21.15.119:0/2792904816 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c0253f0 con 0x7f499c0c87c0 2023-10-08T02:28:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.422+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c009420 con 0x7f499c0c87c0 2023-10-08T02:28:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.422+0000 7f4991ffb700 1 -- 172.21.15.119:0/2792904816 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c0257b0 con 0x7f499c0c87c0 2023-10-08T02:28:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.422+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c0b0140 con 0x7f499c0c87c0 2023-10-08T02:28:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.423+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499c04c3c0 con 0x7f499c0c87c0 2023-10-08T02:28:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.424+0000 7f4991ffb700 1 -- 172.21.15.119:0/2792904816 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f498c025ba0 con 0x7f499c0c87c0 2023-10-08T02:28:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.425+0000 7f4991ffb700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4980066f30 0x7f49800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.425+0000 7f4991ffb700 1 -- 172.21.15.119:0/2792904816 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5345+0+0 (secure 0 0 0) 0x7f498c085cd0 con 0x7f499c0c87c0 2023-10-08T02:28:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.425+0000 7f49a1d9b700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4980066f30 0x7f49800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.425+0000 7f49a1d9b700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4980066f30 0x7f49800693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4998009e40 tx=0x7f4998008040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.429+0000 7f4991ffb700 1 -- 172.21.15.119:0/2792904816 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f498c053000 con 0x7f499c0c87c0 2023-10-08T02:28:36.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.618+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f499c0cbd90 con 0x7f499c0c87c0 2023-10-08T02:28:36.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.618+0000 7f4991ffb700 1 -- 172.21.15.119:0/2792904816 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v187) v1 ==== 93+0+10 (secure 0 0 0) 0x7f498c056620 con 0x7f499c0c87c0 2023-10-08T02:28:36.619 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:28:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.620+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4980066f30 msgr2=0x7f49800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.620+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4980066f30 0x7f49800693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4998009e40 tx=0x7f4998008040 comp rx=0 tx=0).stop 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.620+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f499c0c87c0 msgr2=0x7f499c0c3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.620+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f499c0c87c0 0x7f499c0c3cb0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f499c063d40 tx=0x7f498c017000 comp rx=0 tx=0).stop 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.621+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 shutdown_connections 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.621+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4980066f30 0x7f49800693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.621+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f499c0c83f0 0x7f499c13ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.621+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f499c0ccee0 0x7f499c0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.621+0000 7f49a3fff700 1 --2- 172.21.15.119:0/2792904816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f499c0c87c0 0x7f499c0c3cb0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.621+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 >> 172.21.15.119:0/2792904816 conn(0x7f499c068e10 msgr2=0x7f499c0c2830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:36.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.621+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 shutdown_connections 2023-10-08T02:28:36.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.621+0000 7f49a3fff700 1 -- 172.21.15.119:0/2792904816 wait complete. 2023-10-08T02:28:36.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T02:28:36.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.821+0000 7f4693bff700 1 Processor -- start 2023-10-08T02:28:36.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.821+0000 7f4693bff700 1 -- start start 2023-10-08T02:28:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.822+0000 7f4693bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c0b0a10 0x7f468c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.822+0000 7f4693bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c0ad930 con 0x7f468c155c10 2023-10-08T02:28:36.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.822+0000 7f4693bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f468c0acd20 con 0x7f468c1578b0 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.822+0000 7f4693bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0ad000 con 0x7f468c0b0a10 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f469199b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f468c155c10 0x7f468c0ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51614/0 (socket says 172.21.15.119:51614) 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f469199b700 1 -- 172.21.15.119:0/3012741864 learned_addr learned my addr 172.21.15.119:0/3012741864 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f469219c700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c0b0a10 0x7f468c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618407169 0 0) 0x7f468c0ad930 con 0x7f468c155c10 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4678004680 con 0x7f468c155c10 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361073876 0 0) 0x7f468c0acd20 con 0x7f468c1578b0 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f468c0ad930 con 0x7f468c1578b0 2023-10-08T02:28:36.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4077900265 0 0) 0x7f4678004680 con 0x7f468c155c10 2023-10-08T02:28:36.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c0b0a10 msgr2=0x7f468c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:36.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c0b0a10 0x7f468c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 >> v1:172.21.15.184:6789/0 conn(0x7f468c1578b0 legacy=0x7f468c0af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c0ad2e0 con 0x7f468c155c10 2023-10-08T02:28:36.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f4688001470 con 0x7f468c155c10 2023-10-08T02:28:36.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 >> v1:172.21.15.119:6789/0 conn(0x7f468c155c10 legacy=0x7f468c0ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c0b0a10 0x7f4678004d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.823+0000 7f4690999700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780052e0 0x7f4678005920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f4690999700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4678005e60 0x7f46780062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c0ad000 con 0x7f4678005e60 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f467800cd00 con 0x7f46780052e0 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467800cec0 con 0x7f468c0b0a10 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469219c700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4678005e60 0x7f46780062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469199b700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c0b0a10 0x7f4678004d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469119a700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780052e0 0x7f4678005920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469219c700 1 -- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c0b0a10 msgr2=0x7f4678004d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:36.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469219c700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f468c0b0a10 0x7f4678004d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469219c700 1 -- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780052e0 msgr2=0x7f4678005920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469219c700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f46780052e0 0x7f4678005920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469219c700 1 -- 172.21.15.119:0/3012741864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678004680 con 0x7f4678005e60 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.824+0000 7f469219c700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4678005e60 0x7f46780062d0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f468c0b0e10 tx=0x7f468400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66cf5c2354183cd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468400ea90 con 0x7f4678005e60 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468400f040 con 0x7f4678005e60 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4690999700 1 -- 172.21.15.119:0/3012741864 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468400ee40 con 0x7f4678005e60 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4693bff700 1 -- 172.21.15.119:0/3012741864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4678005e60 msgr2=0x7f46780062d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4693bff700 1 --2- 172.21.15.119:0/3012741864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4678005e60 0x7f46780062d0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f468c0b0e10 tx=0x7f468400b040 comp rx=0 tx=0).stop 2023-10-08T02:28:36.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f469219c700 1 -- 172.21.15.119:0/3012741864 reap_dead start 2023-10-08T02:28:36.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4693bff700 1 -- 172.21.15.119:0/3012741864 shutdown_connections 2023-10-08T02:28:36.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4693bff700 1 -- 172.21.15.119:0/3012741864 >> 172.21.15.119:0/3012741864 conn(0x7f468c06a320 msgr2=0x7f468c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:36.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4693bff700 1 -- 172.21.15.119:0/3012741864 shutdown_connections 2023-10-08T02:28:36.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.825+0000 7f4693bff700 1 -- 172.21.15.119:0/3012741864 wait complete. 2023-10-08T02:28:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.827+0000 7f4693bff700 1 Processor -- start 2023-10-08T02:28:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.827+0000 7f4693bff700 1 -- start start 2023-10-08T02:28:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f4693bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4678005e60 0x7f468c143400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f4693bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f468c14ed20 0x7f468c06f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f4693bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f468c14f190 0x7f468c06f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f4693bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c14cc20 con 0x7f468c14f190 2023-10-08T02:28:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f4693bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c14f700 con 0x7f468c14ed20 2023-10-08T02:28:36.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f4693bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468c14f9e0 con 0x7f4678005e60 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469219c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f468c14f190 0x7f468c06f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469219c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f468c14f190 0x7f468c06f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48800/0 (socket says 172.21.15.119:48800) 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469219c700 1 -- 172.21.15.119:0/702172045 learned_addr learned my addr 172.21.15.119:0/702172045 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469119a700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f468c14ed20 0x7f468c06f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469199b700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4678005e60 0x7f468c143400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469219c700 1 -- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4678005e60 msgr2=0x7f468c143400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469219c700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4678005e60 0x7f468c143400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469219c700 1 -- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f468c14ed20 msgr2=0x7f468c06f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:36.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469219c700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f468c14ed20 0x7f468c06f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469219c700 1 -- 172.21.15.119:0/702172045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f468c14fcc0 con 0x7f468c14f190 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469119a700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f468c14ed20 0x7f468c06f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.828+0000 7f469199b700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4678005e60 0x7f468c143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.829+0000 7f469219c700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f468c14f190 0x7f468c06f680 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f468400afd0 tx=0x7f468401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.829+0000 7f4681ffb700 1 -- 172.21.15.119:0/702172045 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468401e640 con 0x7f468c14f190 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.829+0000 7f4681ffb700 1 -- 172.21.15.119:0/702172045 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4684022070 con 0x7f468c14f190 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.829+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c14c8b0 con 0x7f468c14f190 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.829+0000 7f4681ffb700 1 -- 172.21.15.119:0/702172045 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468401e9f0 con 0x7f468c14f190 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.829+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c1514f0 con 0x7f468c14f190 2023-10-08T02:28:36.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.830+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468c14cdd0 con 0x7f468c14f190 2023-10-08T02:28:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.831+0000 7f4681ffb700 1 -- 172.21.15.119:0/702172045 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4684003040 con 0x7f468c14f190 2023-10-08T02:28:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.832+0000 7f4681ffb700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4670066f30 0x7f46700693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.832+0000 7f4681ffb700 1 -- 172.21.15.119:0/702172045 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5345+0+0 (secure 0 0 0) 0x7f4684024070 con 0x7f468c14f190 2023-10-08T02:28:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.832+0000 7f469199b700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4670066f30 0x7f46700693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.832+0000 7f469199b700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4670066f30 0x7f46700693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4688001200 tx=0x7f4688009040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:36.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:36.836+0000 7f4681ffb700 1 -- 172.21.15.119:0/702172045 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f468c14cdd0 con 0x7f468c14f190 2023-10-08T02:28:37.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.027+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f468c1516b0 con 0x7f468c14f190 2023-10-08T02:28:37.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.027+0000 7f4681ffb700 1 -- 172.21.15.119:0/702172045 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v187) v1 ==== 84+0+65 (secure 0 0 0) 0x7f4684053480 con 0x7f468c14f190 2023-10-08T02:28:37.028 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:37.028 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T02:28:37.028 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:37.028 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T02:28:37.028 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T02:28:37.028 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:37.029 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T02:28:37.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4670066f30 msgr2=0x7f46700693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4670066f30 0x7f46700693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4688001200 tx=0x7f4688009040 comp rx=0 tx=0).stop 2023-10-08T02:28:37.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f468c14f190 msgr2=0x7f468c06f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f468c14f190 0x7f468c06f680 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f468400afd0 tx=0x7f468401a000 comp rx=0 tx=0).stop 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 shutdown_connections 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4670066f30 0x7f46700693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4678005e60 0x7f468c143400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f468c14f190 0x7f468c06f680 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 --2- 172.21.15.119:0/702172045 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f468c14ed20 0x7f468c06f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 >> 172.21.15.119:0/702172045 conn(0x7f468c06a320 msgr2=0x7f468c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 shutdown_connections 2023-10-08T02:28:37.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.029+0000 7f4693bff700 1 -- 172.21.15.119:0/702172045 wait complete. 2023-10-08T02:28:37.047 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T02:28:37.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T02:28:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.233+0000 7f9bba5ff700 1 Processor -- start 2023-10-08T02:28:37.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.234+0000 7f9bba5ff700 1 -- start start 2023-10-08T02:28:37.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bba5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9bb4068890 0x7f9bb40c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:37.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bba5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb415a9b0 con 0x7f9bb415a440 2023-10-08T02:28:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bba5ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bb415ac90 con 0x7f9bb41563b0 2023-10-08T02:28:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bba5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bb415af70 con 0x7f9bb4068890 2023-10-08T02:28:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bb37fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9bb415a440 0x7f9bb40acf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51630/0 (socket says 172.21.15.119:51630) 2023-10-08T02:28:37.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bb37fe700 1 -- 172.21.15.119:0/1278952605 learned_addr learned my addr 172.21.15.119:0/1278952605 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bb8b9c700 1 --2- 172.21.15.119:0/1278952605 >> v2:172.21.15.184:3300/0 conn(0x7f9bb4068890 0x7f9bb40c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958219441 0 0) 0x7f9bb415a9b0 con 0x7f9bb415a440 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.235+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b9c003570 con 0x7f9bb415a440 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594794589 0 0) 0x7f9bb415ac90 con 0x7f9bb41563b0 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb415a9b0 con 0x7f9bb41563b0 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3825950397 0 0) 0x7f9b9c003570 con 0x7f9bb415a440 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 >> v2:172.21.15.184:3300/0 conn(0x7f9bb4068890 msgr2=0x7f9bb40c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 --2- 172.21.15.119:0/1278952605 >> v2:172.21.15.184:3300/0 conn(0x7f9bb4068890 0x7f9bb40c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 >> v1:172.21.15.184:6789/0 conn(0x7f9bb41563b0 legacy=0x7f9bb4156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb415b250 con 0x7f9bb415a440 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f9ba8001470 con 0x7f9bb415a440 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 >> v1:172.21.15.119:6789/0 conn(0x7f9bb415a440 legacy=0x7f9bb40acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b9c003c40 0x7f9b9c004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c004810 0x7f9b9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9c005210 0x7f9b9c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb415af70 con 0x7f9b9c003c40 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c00e340 con 0x7f9b9c005210 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c00e4d0 con 0x7f9b9c004810 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb3fff700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b9c003c40 0x7f9b9c004030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb37fe700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c004810 0x7f9b9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.236+0000 7f9bb8b9c700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9c005210 0x7f9b9c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb3fff700 1 -- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c004810 msgr2=0x7f9b9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb3fff700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c004810 0x7f9b9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb3fff700 1 -- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9c005210 msgr2=0x7f9b9c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb3fff700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9c005210 0x7f9b9c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb3fff700 1 -- 172.21.15.119:0/1278952605 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c003570 con 0x7f9b9c003c40 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb37fe700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c004810 0x7f9b9c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb8b9c700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9b9c005210 0x7f9b9c00ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb8b9c700 1 -- 172.21.15.119:0/1278952605 reap_dead start 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb3fff700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b9c003c40 0x7f9b9c004030 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001a40 tx=0x7f9ba4007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daad2cb8d3f50946 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400c590 con 0x7f9b9c003c40 2023-10-08T02:28:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ba400c6f0 con 0x7f9b9c003c40 2023-10-08T02:28:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.237+0000 7f9bb2ffd700 1 -- 172.21.15.119:0/1278952605 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ba400cab0 con 0x7f9b9c003c40 2023-10-08T02:28:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.238+0000 7f9bba5ff700 1 -- 172.21.15.119:0/1278952605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b9c003c40 msgr2=0x7f9b9c004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.238+0000 7f9bba5ff700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b9c003c40 0x7f9b9c004030 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4001a40 tx=0x7f9ba4007d30 comp rx=0 tx=0).stop 2023-10-08T02:28:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.238+0000 7f9bba5ff700 1 -- 172.21.15.119:0/1278952605 shutdown_connections 2023-10-08T02:28:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.238+0000 7f9bba5ff700 1 --2- 172.21.15.119:0/1278952605 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9b9c003c40 0x7f9b9c004030 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.238+0000 7f9bba5ff700 1 -- 172.21.15.119:0/1278952605 >> 172.21.15.119:0/1278952605 conn(0x7f9bb40c8f90 msgr2=0x7f9bb40c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.238+0000 7f9bba5ff700 1 -- 172.21.15.119:0/1278952605 shutdown_connections 2023-10-08T02:28:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.238+0000 7f9bba5ff700 1 -- 172.21.15.119:0/1278952605 wait complete. 2023-10-08T02:28:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.239+0000 7f9bba5ff700 1 Processor -- start 2023-10-08T02:28:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.239+0000 7f9bba5ff700 1 -- start start 2023-10-08T02:28:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bba5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c003c40 0x7f9bb4169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bba5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bb414d4e0 0x7f9bb4150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bba5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb41511f0 0x7f9bb4156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bba5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb4169dc0 con 0x7f9bb41511f0 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bba5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40af2d0 con 0x7f9bb414d4e0 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bba5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bb40af890 con 0x7f9b9c003c40 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bb8b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb41511f0 0x7f9bb4156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bb8b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb41511f0 0x7f9bb4156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48830/0 (socket says 172.21.15.119:48830) 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bb8b9c700 1 -- 172.21.15.119:0/3569099411 learned_addr learned my addr 172.21.15.119:0/3569099411 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.240+0000 7f9bb3fff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c003c40 0x7f9bb4169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bb37fe700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bb414d4e0 0x7f9bb4150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bb8b9c700 1 -- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c003c40 msgr2=0x7f9bb4169880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bb8b9c700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c003c40 0x7f9bb4169880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bb8b9c700 1 -- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bb414d4e0 msgr2=0x7f9bb4150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bb8b9c700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bb414d4e0 0x7f9bb4150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bb8b9c700 1 -- 172.21.15.119:0/3569099411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb4158b40 con 0x7f9bb41511f0 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bb37fe700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bb414d4e0 0x7f9bb4150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bb8b9c700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb41511f0 0x7f9bb4156f30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009fd0 tx=0x7f9bac00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9b9bfff700 1 -- 172.21.15.119:0/3569099411 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac01b070 con 0x7f9bb41511f0 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9b9bfff700 1 -- 172.21.15.119:0/3569099411 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac00f040 con 0x7f9bb41511f0 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bb4151620 con 0x7f9bb41511f0 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9b9bfff700 1 -- 172.21.15.119:0/3569099411 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00e050 con 0x7f9bb41511f0 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.241+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bb4159550 con 0x7f9bb41511f0 2023-10-08T02:28:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.242+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bb40afa20 con 0x7f9bb41511f0 2023-10-08T02:28:37.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.243+0000 7f9b9bfff700 1 -- 172.21.15.119:0/3569099411 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bac003020 con 0x7f9bb41511f0 2023-10-08T02:28:37.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.244+0000 7f9b9bfff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:37.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.244+0000 7f9b9bfff700 1 -- 172.21.15.119:0/3569099411 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5345+0+0 (secure 0 0 0) 0x7f9bac084970 con 0x7f9bb41511f0 2023-10-08T02:28:37.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.244+0000 7f9bb3fff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:37.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.245+0000 7f9bb3fff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ba0066f30 0x7f9ba00693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40019e0 tx=0x7f9ba4015000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:37.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.248+0000 7f9b9bfff700 1 -- 172.21.15.119:0/3569099411 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bac051ca0 con 0x7f9bb41511f0 2023-10-08T02:28:37.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.440+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9bb406d6a0 con 0x7f9bb41511f0 2023-10-08T02:28:37.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.795+0000 7f9b9bfff700 1 -- 172.21.15.119:0/3569099411 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v188) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9bac010ea0 con 0x7f9bb41511f0 2023-10-08T02:28:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ba0066f30 msgr2=0x7f9ba00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ba0066f30 0x7f9ba00693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9ba40019e0 tx=0x7f9ba4015000 comp rx=0 tx=0).stop 2023-10-08T02:28:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb41511f0 msgr2=0x7f9bb4156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb41511f0 0x7f9bb4156f30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f9bac009fd0 tx=0x7f9bac00b040 comp rx=0 tx=0).stop 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 shutdown_connections 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ba0066f30 0x7f9ba00693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9b9c003c40 0x7f9bb4169880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9bb41511f0 0x7f9bb4156f30 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 --2- 172.21.15.119:0/3569099411 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9bb414d4e0 0x7f9bb4150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.799+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 >> 172.21.15.119:0/3569099411 conn(0x7f9bb40c8f90 msgr2=0x7f9bb40ae1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.800+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 shutdown_connections 2023-10-08T02:28:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:37.800+0000 7f9bba5ff700 1 -- 172.21.15.119:0/3569099411 wait complete. 2023-10-08T02:28:37.803 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T02:28:37.821 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T02:28:38.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.010+0000 7f5f6ed9d700 1 Processor -- start 2023-10-08T02:28:38.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.010+0000 7f5f6ed9d700 1 -- start start 2023-10-08T02:28:38.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.011+0000 7f5f6ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f681595d0 0x7f5f680ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:38.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.011+0000 7f5f6ed9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5f680cf480 0x7f5f6815d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:38.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.011+0000 7f5f6ed9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6815dbc0 con 0x7f5f6815d760 2023-10-08T02:28:38.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.011+0000 7f5f6ed9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f680b10f0 con 0x7f5f680cf480 2023-10-08T02:28:38.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.011+0000 7f5f6ed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f680cd530 con 0x7f5f681595d0 2023-10-08T02:28:38.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.011+0000 7f5f6e59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5f6815d760 0x7f5f681599a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51644/0 (socket says 172.21.15.119:51644) 2023-10-08T02:28:38.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.011+0000 7f5f6e59c700 1 -- 172.21.15.119:0/2796615057 learned_addr learned my addr 172.21.15.119:0/2796615057 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:38.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6d59a700 1 --2- 172.21.15.119:0/2796615057 >> v2:172.21.15.184:3300/0 conn(0x7f5f680cf480 0x7f5f6815d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:38.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6dd9b700 1 --2- 172.21.15.119:0/2796615057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f681595d0 0x7f5f680ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:38.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6cd99700 1 -- 172.21.15.119:0/2796615057 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714106081 0 0) 0x7f5f60001110 con 0x7f5f6815d760 2023-10-08T02:28:38.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6cd99700 1 -- 172.21.15.119:0/2796615057 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f54003570 con 0x7f5f6815d760 2023-10-08T02:28:38.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6cd99700 1 -- 172.21.15.119:0/2796615057 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1073930485 0 0) 0x7f5f54003570 con 0x7f5f6815d760 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6d59a700 1 -- 172.21.15.119:0/2796615057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f681595d0 msgr2=0x7f5f680ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6d59a700 1 --2- 172.21.15.119:0/2796615057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f681595d0 0x7f5f680ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6d59a700 1 -- 172.21.15.119:0/2796615057 >> v1:172.21.15.119:6789/0 conn(0x7f5f6815d760 legacy=0x7f5f681599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6d59a700 1 -- 172.21.15.119:0/2796615057 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6815e000 con 0x7f5f680cf480 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.012+0000 7f5f6d59a700 1 --2- 172.21.15.119:0/2796615057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f680cf480 0x7f5f6815d220 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5f580098e0 tx=0x7f5f580091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74bb7718ac23b690 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6cd99700 1 -- 172.21.15.119:0/2796615057 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60001600 con 0x7f5f680cf480 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6cd99700 1 -- 172.21.15.119:0/2796615057 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f58011420 con 0x7f5f680cf480 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6cd99700 1 -- 172.21.15.119:0/2796615057 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f580117c0 con 0x7f5f680cf480 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/2796615057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f680cf480 msgr2=0x7f5f6815d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/2796615057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f680cf480 0x7f5f6815d220 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5f580098e0 tx=0x7f5f580091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/2796615057 shutdown_connections 2023-10-08T02:28:38.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/2796615057 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f681595d0 0x7f5f680ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/2796615057 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f680cf480 0x7f5f6815d220 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/2796615057 >> 172.21.15.119:0/2796615057 conn(0x7f5f680d05f0 msgr2=0x7f5f680d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:38.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/2796615057 shutdown_connections 2023-10-08T02:28:38.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.013+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/2796615057 wait complete. 2023-10-08T02:28:38.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.015+0000 7f5f6ed9d700 1 Processor -- start 2023-10-08T02:28:38.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.015+0000 7f5f6ed9d700 1 -- start start 2023-10-08T02:28:38.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6ed9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f680cf480 0x7f5f6814edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:38.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f681595d0 0x7f5f68150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:38.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6ed9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815d760 0x7f5f6814cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:38.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6ed9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f68158d60 con 0x7f5f681595d0 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6ed9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f680b46e0 con 0x7f5f6815d760 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6ed9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6814bc70 con 0x7f5f680cf480 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6d59a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f681595d0 0x7f5f68150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6d59a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f681595d0 0x7f5f68150b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48844/0 (socket says 172.21.15.119:48844) 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6d59a700 1 -- 172.21.15.119:0/4151408551 learned_addr learned my addr 172.21.15.119:0/4151408551 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6dd9b700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f680cf480 0x7f5f6814edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.016+0000 7f5f6e59c700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815d760 0x7f5f6814cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6d59a700 1 -- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f680cf480 msgr2=0x7f5f6814edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6d59a700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f680cf480 0x7f5f6814edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6d59a700 1 -- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815d760 msgr2=0x7f5f6814cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:38.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6d59a700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815d760 0x7f5f6814cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6d59a700 1 -- 172.21.15.119:0/4151408551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f58009420 con 0x7f5f681595d0 2023-10-08T02:28:38.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6e59c700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815d760 0x7f5f6814cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:38.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6d59a700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f681595d0 0x7f5f68150b60 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f5f580030c0 tx=0x7f5f58016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:38.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f5dffb700 1 -- 172.21.15.119:0/4151408551 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5801cbe0 con 0x7f5f681595d0 2023-10-08T02:28:38.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f5dffb700 1 -- 172.21.15.119:0/4151408551 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5801cd40 con 0x7f5f681595d0 2023-10-08T02:28:38.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f5dffb700 1 -- 172.21.15.119:0/4151408551 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5801a6a0 con 0x7f5f681595d0 2023-10-08T02:28:38.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6814f2f0 con 0x7f5f681595d0 2023-10-08T02:28:38.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.017+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f680b3d40 con 0x7f5f681595d0 2023-10-08T02:28:38.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.018+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6804c3c0 con 0x7f5f681595d0 2023-10-08T02:28:38.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.019+0000 7f5f5dffb700 1 -- 172.21.15.119:0/4151408551 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f5801aad0 con 0x7f5f681595d0 2023-10-08T02:28:38.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.020+0000 7f5f5dffb700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f4c066f30 0x7f5f4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:38.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.020+0000 7f5f5dffb700 1 -- 172.21.15.119:0/4151408551 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5715+0+0 (secure 0 0 0) 0x7f5f58085ee0 con 0x7f5f681595d0 2023-10-08T02:28:38.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.020+0000 7f5f6dd9b700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f4c066f30 0x7f5f4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:38.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.021+0000 7f5f6dd9b700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f4c066f30 0x7f5f4c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400f5b0 tx=0x7f5f6400b040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:38.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.025+0000 7f5f5dffb700 1 -- 172.21.15.119:0/4151408551 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5f580530a0 con 0x7f5f681595d0 2023-10-08T02:28:38.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.216+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5f680b3800 con 0x7f5f681595d0 2023-10-08T02:28:38.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.803+0000 7f5f5dffb700 1 -- 172.21.15.119:0/4151408551 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v189) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5f580290d0 con 0x7f5f681595d0 2023-10-08T02:28:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.808+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f4c066f30 msgr2=0x7f5f4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:38.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.808+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f4c066f30 0x7f5f4c0693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5f6400f5b0 tx=0x7f5f6400b040 comp rx=0 tx=0).stop 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f681595d0 msgr2=0x7f5f68150b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f681595d0 0x7f5f68150b60 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f5f580030c0 tx=0x7f5f58016000 comp rx=0 tx=0).stop 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 shutdown_connections 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f4c066f30 0x7f5f4c0693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f680cf480 0x7f5f6814edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f6815d760 0x7f5f6814cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 --2- 172.21.15.119:0/4151408551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f681595d0 0x7f5f68150b60 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 >> 172.21.15.119:0/4151408551 conn(0x7f5f680d05f0 msgr2=0x7f5f6815a380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:38.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 shutdown_connections 2023-10-08T02:28:38.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:38.809+0000 7f5f6ed9d700 1 -- 172.21.15.119:0/4151408551 wait complete. 2023-10-08T02:28:38.817 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T02:28:38.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T02:28:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.022+0000 7fe94efff700 1 Processor -- start 2023-10-08T02:28:39.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.022+0000 7fe94efff700 1 -- start start 2023-10-08T02:28:39.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe94efff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe948069d60 0x7fe9480c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe94efff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480c81f0 0x7fe9480cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe94efff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9480c8650 con 0x7fe9480aca80 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe94efff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94806bdb0 con 0x7fe948069d60 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe94efff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480cb370 con 0x7fe9480c81f0 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe947fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe948069d60 0x7fe9480c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe947fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe948069d60 0x7fe9480c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48850/0 (socket says 172.21.15.119:48850) 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe947fff700 1 -- 172.21.15.119:0/1247449639 learned_addr learned my addr 172.21.15.119:0/1247449639 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe947fff700 1 -- 172.21.15.119:0/1247449639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480c81f0 msgr2=0x7fe9480cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:39.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe94d59c700 1 --2- 172.21.15.119:0/1247449639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480c81f0 0x7fe9480cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:39.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe947fff700 1 --2- 172.21.15.119:0/1247449639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480c81f0 0x7fe9480cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe947fff700 1 -- 172.21.15.119:0/1247449639 >> v1:172.21.15.184:6789/0 conn(0x7fe9480aca80 legacy=0x7fe94806aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:39.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe947fff700 1 -- 172.21.15.119:0/1247449639 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948159f60 con 0x7fe948069d60 2023-10-08T02:28:39.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.024+0000 7fe94d59c700 1 --2- 172.21.15.119:0/1247449639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480c81f0 0x7fe9480cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:39.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.025+0000 7fe947fff700 1 --2- 172.21.15.119:0/1247449639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe9480c7cb0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe9340099b0 tx=0x7fe934009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d544077f75a6733 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:39.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.025+0000 7fe9477fe700 1 -- 172.21.15.119:0/1247449639 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93400b820 con 0x7fe948069d60 2023-10-08T02:28:39.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.025+0000 7fe9477fe700 1 -- 172.21.15.119:0/1247449639 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93400b980 con 0x7fe948069d60 2023-10-08T02:28:39.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.025+0000 7fe9477fe700 1 -- 172.21.15.119:0/1247449639 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93400bd20 con 0x7fe948069d60 2023-10-08T02:28:39.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.025+0000 7fe94efff700 1 -- 172.21.15.119:0/1247449639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 msgr2=0x7fe9480c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:39.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.025+0000 7fe94efff700 1 --2- 172.21.15.119:0/1247449639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe9480c7cb0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fe9340099b0 tx=0x7fe934009250 comp rx=0 tx=0).stop 2023-10-08T02:28:39.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.026+0000 7fe94efff700 1 -- 172.21.15.119:0/1247449639 shutdown_connections 2023-10-08T02:28:39.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.026+0000 7fe94efff700 1 --2- 172.21.15.119:0/1247449639 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480c81f0 0x7fe9480cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.026+0000 7fe94efff700 1 --2- 172.21.15.119:0/1247449639 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe9480c7cb0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.026+0000 7fe94efff700 1 -- 172.21.15.119:0/1247449639 >> 172.21.15.119:0/1247449639 conn(0x7fe9480cd670 msgr2=0x7fe9480cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.026+0000 7fe94efff700 1 -- 172.21.15.119:0/1247449639 shutdown_connections 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.026+0000 7fe94efff700 1 -- 172.21.15.119:0/1247449639 wait complete. 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.027+0000 7fe94efff700 1 Processor -- start 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.027+0000 7fe94efff700 1 -- start start 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94efff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe94816d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94efff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480aca80 0x7fe94815e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94efff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9480c81f0 0x7fe94815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94efff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe948157dc0 con 0x7fe948069d60 2023-10-08T02:28:39.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94efff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480b04b0 con 0x7fe9480c81f0 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94efff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94816ddc0 con 0x7fe9480aca80 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94cd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe94816d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94cd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe94816d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48860/0 (socket says 172.21.15.119:48860) 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94cd9b700 1 -- 172.21.15.119:0/1965598913 learned_addr learned my addr 172.21.15.119:0/1965598913 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe94d59c700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9480c81f0 0x7fe94815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.028+0000 7fe947fff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480aca80 0x7fe94815e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe94cd9b700 1 -- 172.21.15.119:0/1965598913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480aca80 msgr2=0x7fe94815e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe94cd9b700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480aca80 0x7fe94815e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe94cd9b700 1 -- 172.21.15.119:0/1965598913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9480c81f0 msgr2=0x7fe94815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:39.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe94cd9b700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9480c81f0 0x7fe94815eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe94cd9b700 1 -- 172.21.15.119:0/1965598913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9340094f0 con 0x7fe948069d60 2023-10-08T02:28:39.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe94cd9b700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe94816d570 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c5e0 tx=0x7fe93c0020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:39.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe944ff9700 1 -- 172.21.15.119:0/1965598913 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011590 con 0x7fe948069d60 2023-10-08T02:28:39.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe944ff9700 1 -- 172.21.15.119:0/1965598913 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93c0116f0 con 0x7fe948069d60 2023-10-08T02:28:39.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94816e0a0 con 0x7fe948069d60 2023-10-08T02:28:39.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.029+0000 7fe944ff9700 1 -- 172.21.15.119:0/1965598913 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93c011a90 con 0x7fe948069d60 2023-10-08T02:28:39.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.030+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94816dab0 con 0x7fe948069d60 2023-10-08T02:28:39.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.031+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9480b0960 con 0x7fe948069d60 2023-10-08T02:28:39.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.033+0000 7fe944ff9700 1 -- 172.21.15.119:0/1965598913 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe93c025430 con 0x7fe948069d60 2023-10-08T02:28:39.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.034+0000 7fe944ff9700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe92c066f30 0x7fe92c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:39.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.034+0000 7fe944ff9700 1 -- 172.21.15.119:0/1965598913 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 6097+0+0 (secure 0 0 0) 0x7fe93c084b10 con 0x7fe948069d60 2023-10-08T02:28:39.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.034+0000 7fe947fff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe92c066f30 0x7fe92c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:39.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.034+0000 7fe947fff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe92c066f30 0x7fe92c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9340099b0 tx=0x7fe934016000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:39.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.037+0000 7fe944ff9700 1 -- 172.21.15.119:0/1965598913 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe93c051c80 con 0x7fe948069d60 2023-10-08T02:28:39.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.272+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe9480b0960 con 0x7fe948069d60 2023-10-08T02:28:39.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.804+0000 7fe944ff9700 1 -- 172.21.15.119:0/1965598913 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v83) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe9480b0960 con 0x7fe948069d60 2023-10-08T02:28:39.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe92c066f30 msgr2=0x7fe92c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:39.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe92c066f30 0x7fe92c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe9340099b0 tx=0x7fe934016000 comp rx=0 tx=0).stop 2023-10-08T02:28:39.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 msgr2=0x7fe94816d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:39.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe94816d570 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fe93c00c5e0 tx=0x7fe93c0020e0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 shutdown_connections 2023-10-08T02:28:39.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe92c066f30 0x7fe92c0693e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe9480c81f0 0x7fe94815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe948069d60 0x7fe94816d570 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 --2- 172.21.15.119:0/1965598913 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe9480aca80 0x7fe94815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:39.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 >> 172.21.15.119:0/1965598913 conn(0x7fe9480cd670 msgr2=0x7fe948158210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:39.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 shutdown_connections 2023-10-08T02:28:39.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:39.807+0000 7fe94efff700 1 -- 172.21.15.119:0/1965598913 wait complete. 2023-10-08T02:28:39.809 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 22 and data pool 23 2023-10-08T02:28:39.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:28:40.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.009+0000 7ff00b5ff700 1 Processor -- start 2023-10-08T02:28:40.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.009+0000 7ff00b5ff700 1 -- start start 2023-10-08T02:28:40.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00b5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0040cbf30 0x7ff0040ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00b5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00b5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0040cc300 con 0x7ff00415a270 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00b5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff004156bf0 con 0x7ff0040cbf30 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00b5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040ae950 con 0x7ff0040ccf90 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00939b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0040cbf30 0x7ff0040ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00939b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0040cbf30 0x7ff0040ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48866/0 (socket says 172.21.15.119:48866) 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00939b700 1 -- 172.21.15.119:0/1979369014 learned_addr learned my addr 172.21.15.119:0/1979369014 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff008b9a700 1 --2- 172.21.15.119:0/1979369014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00939b700 1 -- 172.21.15.119:0/1979369014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 msgr2=0x7ff0040b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00939b700 1 --2- 172.21.15.119:0/1979369014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00939b700 1 -- 172.21.15.119:0/1979369014 >> v1:172.21.15.119:6789/0 conn(0x7ff00415a270 legacy=0x7ff0040c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff00939b700 1 -- 172.21.15.119:0/1979369014 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0040ad370 con 0x7ff0040cbf30 2023-10-08T02:28:40.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.011+0000 7ff008b9a700 1 --2- 172.21.15.119:0/1979369014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:40.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.012+0000 7ff00939b700 1 --2- 172.21.15.119:0/1979369014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0040cbf30 0x7ff0040ae410 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ff0000099e0 tx=0x7ff000009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d23d86a645572d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.012+0000 7feffbfff700 1 -- 172.21.15.119:0/1979369014 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000b820 con 0x7ff0040cbf30 2023-10-08T02:28:40.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.012+0000 7feffbfff700 1 -- 172.21.15.119:0/1979369014 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00000b980 con 0x7ff0040cbf30 2023-10-08T02:28:40.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.012+0000 7feffbfff700 1 -- 172.21.15.119:0/1979369014 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000bd20 con 0x7ff0040cbf30 2023-10-08T02:28:40.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.012+0000 7ff00b5ff700 1 -- 172.21.15.119:0/1979369014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0040cbf30 msgr2=0x7ff0040ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.012+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/1979369014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0040cbf30 0x7ff0040ae410 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ff0000099e0 tx=0x7ff000009250 comp rx=0 tx=0).stop 2023-10-08T02:28:40.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.013+0000 7ff00b5ff700 1 -- 172.21.15.119:0/1979369014 shutdown_connections 2023-10-08T02:28:40.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.013+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/1979369014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0040cbf30 0x7ff0040ae410 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.013+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/1979369014 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.013+0000 7ff00b5ff700 1 -- 172.21.15.119:0/1979369014 >> 172.21.15.119:0/1979369014 conn(0x7ff00406a320 msgr2=0x7ff0040cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:40.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.013+0000 7ff00b5ff700 1 -- 172.21.15.119:0/1979369014 shutdown_connections 2023-10-08T02:28:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.013+0000 7ff00b5ff700 1 -- 172.21.15.119:0/1979369014 wait complete. 2023-10-08T02:28:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.014+0000 7ff00b5ff700 1 Processor -- start 2023-10-08T02:28:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.014+0000 7ff00b5ff700 1 -- start start 2023-10-08T02:28:40.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff00b5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0040cbf30 0x7ff0040c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff00b5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff00b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00415a270 0x7ff0040ab730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff00b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff004154bd0 con 0x7ff00415a270 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff00b5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c6b20 con 0x7ff0040cbf30 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff00b5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0040c5f10 con 0x7ff0040ccf90 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff009b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00415a270 0x7ff0040ab730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff009b9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00415a270 0x7ff0040ab730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48870/0 (socket says 172.21.15.119:48870) 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.015+0000 7ff009b9c700 1 -- 172.21.15.119:0/118379166 learned_addr learned my addr 172.21.15.119:0/118379166 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff008b9a700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff00939b700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0040cbf30 0x7ff0040c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff009b9c700 1 -- 172.21.15.119:0/118379166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 msgr2=0x7ff0040c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff009b9c700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff009b9c700 1 -- 172.21.15.119:0/118379166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0040cbf30 msgr2=0x7ff0040c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff009b9c700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0040cbf30 0x7ff0040c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff009b9c700 1 -- 172.21.15.119:0/118379166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000094f0 con 0x7ff00415a270 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff009b9c700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00415a270 0x7ff0040ab730 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7feffc00c840 tx=0x7feffc001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7feff97fa700 1 -- 172.21.15.119:0/118379166 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc012630 con 0x7ff00415a270 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7feff97fa700 1 -- 172.21.15.119:0/118379166 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc0127b0 con 0x7ff00415a270 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7feff97fa700 1 -- 172.21.15.119:0/118379166 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc012b70 con 0x7ff00415a270 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.016+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0040c61f0 con 0x7ff00415a270 2023-10-08T02:28:40.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.017+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0040c67b0 con 0x7ff00415a270 2023-10-08T02:28:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.017+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff00404c3c0 con 0x7ff00415a270 2023-10-08T02:28:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.019+0000 7feff97fa700 1 -- 172.21.15.119:0/118379166 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feffc01a3a0 con 0x7ff00415a270 2023-10-08T02:28:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.019+0000 7feff97fa700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fefe8066f30 0x7fefe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.020+0000 7feff97fa700 1 -- 172.21.15.119:0/118379166 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 6197+0+0 (secure 0 0 0) 0x7feffc084920 con 0x7ff00415a270 2023-10-08T02:28:40.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.020+0000 7ff00939b700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fefe8066f30 0x7fefe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.020+0000 7ff00939b700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fefe8066f30 0x7fefe80693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0000099e0 tx=0x7ff000017000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.024+0000 7feff97fa700 1 -- 172.21.15.119:0/118379166 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feffc051900 con 0x7ff00415a270 2023-10-08T02:28:40.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.216+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0040b0af0 con 0x7ff00415a270 2023-10-08T02:28:40.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.217+0000 7feff97fa700 1 -- 172.21.15.119:0/118379166 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+16184 (secure 0 0 0) 0x7feffc054f20 con 0x7ff00415a270 2023-10-08T02:28:40.218 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:40.218 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":190,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:28:39.274317+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:28:37.482739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T02:28:38.257857+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fefe8066f30 msgr2=0x7fefe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fefe8066f30 0x7fefe80693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff0000099e0 tx=0x7ff000017000 comp rx=0 tx=0).stop 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00415a270 msgr2=0x7ff0040ab730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00415a270 0x7ff0040ab730 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7feffc00c840 tx=0x7feffc001670 comp rx=0 tx=0).stop 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 shutdown_connections 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fefe8066f30 0x7fefe80693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff00415a270 0x7ff0040ab730 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0040cbf30 0x7ff0040c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 --2- 172.21.15.119:0/118379166 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0040ccf90 0x7ff0040c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 >> 172.21.15.119:0/118379166 conn(0x7ff00406a320 msgr2=0x7ff004158f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:40.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 shutdown_connections 2023-10-08T02:28:40.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.220+0000 7ff00b5ff700 1 -- 172.21.15.119:0/118379166 wait complete. 2023-10-08T02:28:40.239 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:28:40.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.421+0000 7f8942bff700 1 Processor -- start 2023-10-08T02:28:40.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.421+0000 7f8942bff700 1 -- start start 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.422+0000 7f8942bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f893c159890 0x7f893c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.422+0000 7f8942bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f893c159c60 0x7f893c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.422+0000 7f8942bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c15a160 con 0x7f893c15db10 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.422+0000 7f8942bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893c15df50 con 0x7f893c159c60 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.422+0000 7f8942bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893c0d13a0 con 0x7f893c159890 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f893bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f893c159c60 0x7f893c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f893bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f893c159c60 0x7f893c15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48880/0 (socket says 172.21.15.119:48880) 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f893bfff700 1 -- 172.21.15.119:0/3237457594 learned_addr learned my addr 172.21.15.119:0/3237457594 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f894099b700 1 --2- 172.21.15.119:0/3237457594 >> v2:172.21.15.184:3300/0 conn(0x7f893c159890 0x7f893c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f893bfff700 1 -- 172.21.15.119:0/3237457594 >> v2:172.21.15.184:3300/0 conn(0x7f893c159890 msgr2=0x7f893c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:40.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f893bfff700 1 --2- 172.21.15.119:0/3237457594 >> v2:172.21.15.184:3300/0 conn(0x7f893c159890 0x7f893c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f893bfff700 1 -- 172.21.15.119:0/3237457594 >> v1:172.21.15.184:6789/0 conn(0x7f893c15db10 legacy=0x7f893c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f893bfff700 1 -- 172.21.15.119:0/3237457594 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c15e140 con 0x7f893c159c60 2023-10-08T02:28:40.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f894099b700 1 --2- 172.21.15.119:0/3237457594 >> v2:172.21.15.184:3300/0 conn(0x7f893c159890 0x7f893c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:40.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.423+0000 7f893bfff700 1 --2- 172.21.15.119:0/3237457594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c159c60 0x7f893c15d5d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f893c0d1090 tx=0x7f8928009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df63901a8588ee70 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f893b7fe700 1 -- 172.21.15.119:0/3237457594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800b860 con 0x7f893c159c60 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f893b7fe700 1 -- 172.21.15.119:0/3237457594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892800b9c0 con 0x7f893c159c60 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f893b7fe700 1 -- 172.21.15.119:0/3237457594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892800bd60 con 0x7f893c159c60 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f8942bff700 1 -- 172.21.15.119:0/3237457594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c159c60 msgr2=0x7f893c15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f8942bff700 1 --2- 172.21.15.119:0/3237457594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c159c60 0x7f893c15d5d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f893c0d1090 tx=0x7f8928009250 comp rx=0 tx=0).stop 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f8942bff700 1 -- 172.21.15.119:0/3237457594 shutdown_connections 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f8942bff700 1 --2- 172.21.15.119:0/3237457594 >> v2:172.21.15.184:3300/0 conn(0x7f893c159890 0x7f893c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f8942bff700 1 --2- 172.21.15.119:0/3237457594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c159c60 0x7f893c15d5d0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f8942bff700 1 -- 172.21.15.119:0/3237457594 >> 172.21.15.119:0/3237457594 conn(0x7f893c068e20 msgr2=0x7f893c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f8942bff700 1 -- 172.21.15.119:0/3237457594 shutdown_connections 2023-10-08T02:28:40.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.424+0000 7f8942bff700 1 -- 172.21.15.119:0/3237457594 wait complete. 2023-10-08T02:28:40.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.426+0000 7f8942bff700 1 Processor -- start 2023-10-08T02:28:40.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.426+0000 7f8942bff700 1 -- start start 2023-10-08T02:28:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f8942bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f893c159890 0x7f893c140dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f8942bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f893c159c60 0x7f893c0b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f8942bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c15db10 0x7f893c0b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f8942bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0b3fc0 con 0x7f893c15db10 2023-10-08T02:28:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f8942bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1528a0 con 0x7f893c159890 2023-10-08T02:28:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f8942bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0b1130 con 0x7f893c159c60 2023-10-08T02:28:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f894119c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c15db10 0x7f893c0b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f894119c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c15db10 0x7f893c0b0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48882/0 (socket says 172.21.15.119:48882) 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f894119c700 1 -- 172.21.15.119:0/156867757 learned_addr learned my addr 172.21.15.119:0/156867757 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f894099b700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f893c159890 0x7f893c140dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.427+0000 7f893bfff700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f893c159c60 0x7f893c0b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f894119c700 1 -- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f893c159c60 msgr2=0x7f893c0b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f894119c700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f893c159c60 0x7f893c0b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f894119c700 1 -- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f893c159890 msgr2=0x7f893c140dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f894119c700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f893c159890 0x7f893c140dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f894119c700 1 -- 172.21.15.119:0/156867757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89280094f0 con 0x7f893c15db10 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f893bfff700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f893c159c60 0x7f893c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f894119c700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c15db10 0x7f893c0b0b00 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f893400c4c0 tx=0x7f8934007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f8938ff9700 1 -- 172.21.15.119:0/156867757 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934011590 con 0x7f893c15db10 2023-10-08T02:28:40.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f8938ff9700 1 -- 172.21.15.119:0/156867757 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8934011710 con 0x7f893c15db10 2023-10-08T02:28:40.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f8938ff9700 1 -- 172.21.15.119:0/156867757 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934011ad0 con 0x7f893c15db10 2023-10-08T02:28:40.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c0b1410 con 0x7f893c15db10 2023-10-08T02:28:40.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.428+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c0b19d0 con 0x7f893c15db10 2023-10-08T02:28:40.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.429+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c04c3c0 con 0x7f893c15db10 2023-10-08T02:28:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.430+0000 7f8938ff9700 1 -- 172.21.15.119:0/156867757 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f893401b3b0 con 0x7f893c15db10 2023-10-08T02:28:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.432+0000 7f8938ff9700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8920066f30 0x7f89200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.432+0000 7f8938ff9700 1 -- 172.21.15.119:0/156867757 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f8934083b70 con 0x7f893c15db10 2023-10-08T02:28:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.432+0000 7f894099b700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8920066f30 0x7f89200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.432+0000 7f894099b700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8920066f30 0x7f89200693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8930007740 tx=0x7f89300099d0 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.437+0000 7f8938ff9700 1 -- 172.21.15.119:0/156867757 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8934050b50 con 0x7f893c15db10 2023-10-08T02:28:40.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.630+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f893c06a5b0 con 0x7f893c15db10 2023-10-08T02:28:40.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.631+0000 7f8938ff9700 1 -- 172.21.15.119:0/156867757 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16171 (secure 0 0 0) 0x7f8934017070 con 0x7f893c15db10 2023-10-08T02:28:40.631 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:40.632 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":191,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:28:40.305967+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:28:37.482739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T02:28:38.257857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:28:40.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8920066f30 msgr2=0x7f89200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8920066f30 0x7f89200693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8930007740 tx=0x7f89300099d0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c15db10 msgr2=0x7f893c0b0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c15db10 0x7f893c0b0b00 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f893400c4c0 tx=0x7f8934007f50 comp rx=0 tx=0).stop 2023-10-08T02:28:40.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 shutdown_connections 2023-10-08T02:28:40.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8920066f30 0x7f89200693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f893c15db10 0x7f893c0b0b00 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f893c159890 0x7f893c140dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 --2- 172.21.15.119:0/156867757 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f893c159c60 0x7f893c0b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 >> 172.21.15.119:0/156867757 conn(0x7f893c068e20 msgr2=0x7f893c153190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:40.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 shutdown_connections 2023-10-08T02:28:40.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.633+0000 7f8942bff700 1 -- 172.21.15.119:0/156867757 wait complete. 2023-10-08T02:28:40.652 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T02:28:40.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.837+0000 7f5e9b7ff700 1 Processor -- start 2023-10-08T02:28:40.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.837+0000 7f5e9b7ff700 1 -- start start 2023-10-08T02:28:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.838+0000 7f5e9b7ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5e940c83f0 0x7f5e940c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e9b7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e9415b0d0 con 0x7f5e940ccee0 2023-10-08T02:28:40.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e9b7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e940c7a60 con 0x7f5e941568e0 2023-10-08T02:28:40.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e9b7ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e9415a4c0 con 0x7f5e940c83f0 2023-10-08T02:28:40.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e98d9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5e940ccee0 0x7f5e941561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51654/0 (socket says 172.21.15.119:51654) 2023-10-08T02:28:40.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e98d9a700 1 -- 172.21.15.119:0/4124230889 learned_addr learned my addr 172.21.15.119:0/4124230889 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:40.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e9959b700 1 --2- 172.21.15.119:0/4124230889 >> v2:172.21.15.184:3300/0 conn(0x7f5e940c83f0 0x7f5e940c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e8bfff700 1 -- 172.21.15.119:0/4124230889 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500364020 0 0) 0x7f5e9415b0d0 con 0x7f5e940ccee0 2023-10-08T02:28:40.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e8bfff700 1 -- 172.21.15.119:0/4124230889 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e80003570 con 0x7f5e940ccee0 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.839+0000 7f5e8bfff700 1 -- 172.21.15.119:0/4124230889 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2869879553 0 0) 0x7f5e940c7a60 con 0x7f5e941568e0 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.840+0000 7f5e8bfff700 1 -- 172.21.15.119:0/4124230889 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e9415b0d0 con 0x7f5e941568e0 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.840+0000 7f5e9959b700 1 -- 172.21.15.119:0/4124230889 >> v1:172.21.15.184:6789/0 conn(0x7f5e941568e0 legacy=0x7f5e940ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.840+0000 7f5e9959b700 1 -- 172.21.15.119:0/4124230889 >> v1:172.21.15.119:6789/0 conn(0x7f5e940ccee0 legacy=0x7f5e941561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.840+0000 7f5e9959b700 1 -- 172.21.15.119:0/4124230889 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9415a7a0 con 0x7f5e940c83f0 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.840+0000 7f5e9959b700 1 --2- 172.21.15.119:0/4124230889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e940c83f0 0x7f5e940c7520 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5e900098e0 tx=0x7f5e900091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8201397834c55595 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.840+0000 7f5e8bfff700 1 -- 172.21.15.119:0/4124230889 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90011630 con 0x7f5e940c83f0 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.840+0000 7f5e8bfff700 1 -- 172.21.15.119:0/4124230889 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90011790 con 0x7f5e940c83f0 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.840+0000 7f5e8bfff700 1 -- 172.21.15.119:0/4124230889 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90011b30 con 0x7f5e940c83f0 2023-10-08T02:28:40.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.841+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/4124230889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e940c83f0 msgr2=0x7f5e940c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.841+0000 7f5e9b7ff700 1 --2- 172.21.15.119:0/4124230889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e940c83f0 0x7f5e940c7520 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5e900098e0 tx=0x7f5e900091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.841+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/4124230889 shutdown_connections 2023-10-08T02:28:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.841+0000 7f5e9b7ff700 1 --2- 172.21.15.119:0/4124230889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e940c83f0 0x7f5e940c7520 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.841+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/4124230889 >> 172.21.15.119:0/4124230889 conn(0x7f5e94068e10 msgr2=0x7f5e940cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.841+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/4124230889 shutdown_connections 2023-10-08T02:28:40.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.841+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/4124230889 wait complete. 2023-10-08T02:28:40.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.842+0000 7f5e9b7ff700 1 Processor -- start 2023-10-08T02:28:40.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.842+0000 7f5e9b7ff700 1 -- start start 2023-10-08T02:28:40.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9b7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e940c83f0 0x7f5e9416ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9b7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e940ccee0 0x7f5e9414a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9b7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e941568e0 0x7f5e941516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9b7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9416b2b0 con 0x7f5e940c83f0 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9b7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e940c5580 con 0x7f5e940ccee0 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9b7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e940c4970 con 0x7f5e941568e0 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e940c83f0 0x7f5e9416ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e940c83f0 0x7f5e9416ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48896/0 (socket says 172.21.15.119:48896) 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.843+0000 7f5e9959b700 1 -- 172.21.15.119:0/1336356436 learned_addr learned my addr 172.21.15.119:0/1336356436 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e9959b700 1 -- 172.21.15.119:0/1336356436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e941568e0 msgr2=0x7f5e941516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e9959b700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e941568e0 0x7f5e941516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e9959b700 1 -- 172.21.15.119:0/1336356436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e940ccee0 msgr2=0x7f5e9414a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e9959b700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e940ccee0 0x7f5e9414a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e9959b700 1 -- 172.21.15.119:0/1336356436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e90009420 con 0x7f5e940c83f0 2023-10-08T02:28:40.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e9959b700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e940c83f0 0x7f5e9416ad70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5e94063d40 tx=0x7f5e90017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e897fa700 1 -- 172.21.15.119:0/1336356436 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9001f070 con 0x7f5e940c83f0 2023-10-08T02:28:40.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e897fa700 1 -- 172.21.15.119:0/1336356436 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90000c90 con 0x7f5e940c83f0 2023-10-08T02:28:40.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e897fa700 1 -- 172.21.15.119:0/1336356436 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90025420 con 0x7f5e940c83f0 2023-10-08T02:28:40.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e940c4c50 con 0x7f5e940c83f0 2023-10-08T02:28:40.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.844+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e940c5210 con 0x7f5e940c83f0 2023-10-08T02:28:40.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.845+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e94153aa0 con 0x7f5e940c83f0 2023-10-08T02:28:40.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.846+0000 7f5e897fa700 1 -- 172.21.15.119:0/1336356436 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5e90023070 con 0x7f5e940c83f0 2023-10-08T02:28:40.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.847+0000 7f5e897fa700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5e78066f30 0x7f5e780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:40.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.847+0000 7f5e897fa700 1 -- 172.21.15.119:0/1336356436 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f5e90085be0 con 0x7f5e940c83f0 2023-10-08T02:28:40.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.847+0000 7f5e98d9a700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5e78066f30 0x7f5e780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:40.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.848+0000 7f5e98d9a700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5e78066f30 0x7f5e780693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5e840058d0 tx=0x7f5e840079f0 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:40.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:40.851+0000 7f5e897fa700 1 -- 172.21.15.119:0/1336356436 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5e90052bc0 con 0x7f5e940c83f0 2023-10-08T02:28:41.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.084+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5e9404c3c0 con 0x7f5e940c83f0 2023-10-08T02:28:41.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.825+0000 7f5e897fa700 1 -- 172.21.15.119:0/1336356436 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5e9002a0b0 con 0x7f5e940c83f0 2023-10-08T02:28:41.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.827+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5e78066f30 msgr2=0x7f5e780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:41.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.827+0000 7f5e9b7ff700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5e78066f30 0x7f5e780693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5e840058d0 tx=0x7f5e840079f0 comp rx=0 tx=0).stop 2023-10-08T02:28:41.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e940c83f0 msgr2=0x7f5e9416ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e940c83f0 0x7f5e9416ad70 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5e94063d40 tx=0x7f5e90017000 comp rx=0 tx=0).stop 2023-10-08T02:28:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 shutdown_connections 2023-10-08T02:28:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5e78066f30 0x7f5e780693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5e941568e0 0x7f5e941516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e940c83f0 0x7f5e9416ad70 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 --2- 172.21.15.119:0/1336356436 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5e940ccee0 0x7f5e9414a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:41.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 >> 172.21.15.119:0/1336356436 conn(0x7f5e94068e10 msgr2=0x7f5e9414d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:41.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 shutdown_connections 2023-10-08T02:28:41.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:41.828+0000 7f5e9b7ff700 1 -- 172.21.15.119:0/1336356436 wait complete. 2023-10-08T02:28:41.848 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T02:28:42.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.035+0000 7f3329f9d700 1 Processor -- start 2023-10-08T02:28:42.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.035+0000 7f3329f9d700 1 -- start start 2023-10-08T02:28:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3329f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f33240cb1a0 0x7f332415a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3329f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f332415a8a0 0x7f332406f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3329f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332406f5a0 0x7f33240ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3329f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332415b2e0 con 0x7f33240cb1a0 2023-10-08T02:28:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3329f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f332406ebd0 con 0x7f332415a8a0 2023-10-08T02:28:42.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3329f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f332415ad20 con 0x7f332406f5a0 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3328f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f33240cb1a0 0x7f332415a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3328f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f33240cb1a0 0x7f332415a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48910/0 (socket says 172.21.15.119:48910) 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3328f9b700 1 -- 172.21.15.119:0/4228692897 learned_addr learned my addr 172.21.15.119:0/4228692897 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f3323fff700 1 --2- 172.21.15.119:0/4228692897 >> v2:172.21.15.184:3300/0 conn(0x7f332415a8a0 0x7f332406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.036+0000 7f332979c700 1 --2- 172.21.15.119:0/4228692897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332406f5a0 0x7f33240ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f3328f9b700 1 -- 172.21.15.119:0/4228692897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332406f5a0 msgr2=0x7f33240ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f3328f9b700 1 --2- 172.21.15.119:0/4228692897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332406f5a0 0x7f33240ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f3328f9b700 1 -- 172.21.15.119:0/4228692897 >> v2:172.21.15.184:3300/0 conn(0x7f332415a8a0 msgr2=0x7f332406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f3328f9b700 1 --2- 172.21.15.119:0/4228692897 >> v2:172.21.15.184:3300/0 conn(0x7f332415a8a0 0x7f332406f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f3328f9b700 1 -- 172.21.15.119:0/4228692897 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332415b000 con 0x7f33240cb1a0 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f3323fff700 1 --2- 172.21.15.119:0/4228692897 >> v2:172.21.15.184:3300/0 conn(0x7f332415a8a0 0x7f332406f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:42.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f3328f9b700 1 --2- 172.21.15.119:0/4228692897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240cb1a0 0x7f332415a360 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f3324070d30 tx=0x7f3318009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e555e3a3409533 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:42.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f33237fe700 1 -- 172.21.15.119:0/4228692897 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800b8f0 con 0x7f33240cb1a0 2023-10-08T02:28:42.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f33237fe700 1 -- 172.21.15.119:0/4228692897 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331800ba50 con 0x7f33240cb1a0 2023-10-08T02:28:42.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.037+0000 7f33237fe700 1 -- 172.21.15.119:0/4228692897 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800bdf0 con 0x7f33240cb1a0 2023-10-08T02:28:42.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 -- 172.21.15.119:0/4228692897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240cb1a0 msgr2=0x7f332415a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:42.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 --2- 172.21.15.119:0/4228692897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240cb1a0 0x7f332415a360 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f3324070d30 tx=0x7f3318009250 comp rx=0 tx=0).stop 2023-10-08T02:28:42.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 -- 172.21.15.119:0/4228692897 shutdown_connections 2023-10-08T02:28:42.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 --2- 172.21.15.119:0/4228692897 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f332406f5a0 0x7f33240ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 --2- 172.21.15.119:0/4228692897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33240cb1a0 0x7f332415a360 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 --2- 172.21.15.119:0/4228692897 >> v2:172.21.15.184:3300/0 conn(0x7f332415a8a0 0x7f332406f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 -- 172.21.15.119:0/4228692897 >> 172.21.15.119:0/4228692897 conn(0x7f33240cc1e0 msgr2=0x7f33240cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 -- 172.21.15.119:0/4228692897 shutdown_connections 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.038+0000 7f3329f9d700 1 -- 172.21.15.119:0/4228692897 wait complete. 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.039+0000 7f3329f9d700 1 Processor -- start 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.039+0000 7f3329f9d700 1 -- start start 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.040+0000 7f3329f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332406f5a0 0x7f3324159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.040+0000 7f3329f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33240cb1a0 0x7f3324151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.040+0000 7f3329f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324142f10 0x7f3324152800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.040+0000 7f3329f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332415a010 con 0x7f332406f5a0 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.040+0000 7f3329f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3324159640 con 0x7f33240cb1a0 2023-10-08T02:28:42.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.040+0000 7f3329f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3324151fa0 con 0x7f3324142f10 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.040+0000 7f3328f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332406f5a0 0x7f3324159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3328f9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332406f5a0 0x7f3324159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48920/0 (socket says 172.21.15.119:48920) 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3328f9b700 1 -- 172.21.15.119:0/3414230385 learned_addr learned my addr 172.21.15.119:0/3414230385 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3323fff700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33240cb1a0 0x7f3324151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f332979c700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324142f10 0x7f3324152800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3328f9b700 1 -- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324142f10 msgr2=0x7f3324152800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3328f9b700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324142f10 0x7f3324152800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3328f9b700 1 -- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33240cb1a0 msgr2=0x7f3324151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3328f9b700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33240cb1a0 0x7f3324151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3328f9b700 1 -- 172.21.15.119:0/3414230385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33180094f0 con 0x7f332406f5a0 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f332979c700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324142f10 0x7f3324152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:42.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3328f9b700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332406f5a0 0x7f3324159ad0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f33180090b0 tx=0x7f3318017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3320ff9700 1 -- 172.21.15.119:0/3414230385 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800bde0 con 0x7f332406f5a0 2023-10-08T02:28:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3320ff9700 1 -- 172.21.15.119:0/3414230385 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331801f070 con 0x7f332406f5a0 2023-10-08T02:28:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3324154c00 con 0x7f332406f5a0 2023-10-08T02:28:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.041+0000 7f3320ff9700 1 -- 172.21.15.119:0/3414230385 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331801a890 con 0x7f332406f5a0 2023-10-08T02:28:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.042+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3324158d10 con 0x7f332406f5a0 2023-10-08T02:28:42.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.042+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3324159920 con 0x7f332406f5a0 2023-10-08T02:28:42.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.043+0000 7f3320ff9700 1 -- 172.21.15.119:0/3414230385 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f331801ac40 con 0x7f332406f5a0 2023-10-08T02:28:42.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.044+0000 7f3320ff9700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3308066f30 0x7f33080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:42.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.044+0000 7f3320ff9700 1 -- 172.21.15.119:0/3414230385 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f3318021070 con 0x7f332406f5a0 2023-10-08T02:28:42.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.044+0000 7f3323fff700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3308066f30 0x7f33080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:42.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.045+0000 7f3323fff700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3308066f30 0x7f33080693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3310006f00 tx=0x7f3310008040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:42.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.048+0000 7f3320ff9700 1 -- 172.21.15.119:0/3414230385 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f33180537f0 con 0x7f332406f5a0 2023-10-08T02:28:42.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.284+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f332406e860 con 0x7f332406f5a0 2023-10-08T02:28:42.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.827+0000 7f3320ff9700 1 -- 172.21.15.119:0/3414230385 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7f33180150c0 con 0x7f332406f5a0 2023-10-08T02:28:42.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3308066f30 msgr2=0x7f33080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3308066f30 0x7f33080693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3310006f00 tx=0x7f3310008040 comp rx=0 tx=0).stop 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332406f5a0 msgr2=0x7f3324159ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332406f5a0 0x7f3324159ad0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f33180090b0 tx=0x7f3318017000 comp rx=0 tx=0).stop 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 shutdown_connections 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3308066f30 0x7f33080693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3324142f10 0x7f3324152800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f332406f5a0 0x7f3324159ad0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 --2- 172.21.15.119:0/3414230385 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f33240cb1a0 0x7f3324151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 >> 172.21.15.119:0/3414230385 conn(0x7f33240cc1e0 msgr2=0x7f3324151250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:42.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 shutdown_connections 2023-10-08T02:28:42.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:42.830+0000 7f3329f9d700 1 -- 172.21.15.119:0/3414230385 wait complete. 2023-10-08T02:28:42.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:28:42.869 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:28:42.899 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:28:43.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.035+0000 7f584c3ff700 1 Processor -- start 2023-10-08T02:28:43.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.035+0000 7f584c3ff700 1 -- start start 2023-10-08T02:28:43.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584c3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f58440ccf90 0x7f58440ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584c3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58440cc300 con 0x7f58440cbf30 2023-10-08T02:28:43.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584c3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58440ad370 con 0x7f58440b1420 2023-10-08T02:28:43.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584c3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5844156bf0 con 0x7f58440ccf90 2023-10-08T02:28:43.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584a19b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f58440cbf30 0x7f58440b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51664/0 (socket says 172.21.15.119:51664) 2023-10-08T02:28:43.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584a19b700 1 -- 172.21.15.119:0/190484399 learned_addr learned my addr 172.21.15.119:0/190484399 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:43.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584a99c700 1 --2- 172.21.15.119:0/190484399 >> v2:172.21.15.184:3300/0 conn(0x7f58440ccf90 0x7f58440ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805435206 0 0) 0x7f58440cc300 con 0x7f58440cbf30 2023-10-08T02:28:43.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5830003570 con 0x7f58440cbf30 2023-10-08T02:28:43.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 696080509 0 0) 0x7f5830003570 con 0x7f58440cbf30 2023-10-08T02:28:43.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 >> v2:172.21.15.184:3300/0 conn(0x7f58440ccf90 msgr2=0x7f58440ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 --2- 172.21.15.119:0/190484399 >> v2:172.21.15.184:3300/0 conn(0x7f58440ccf90 0x7f58440ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 >> v1:172.21.15.184:6789/0 conn(0x7f58440b1420 legacy=0x7f58440c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440ad650 con 0x7f58440cbf30 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f5840001470 con 0x7f58440cbf30 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 >> v1:172.21.15.119:6789/0 conn(0x7f58440cbf30 legacy=0x7f58440b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5830003ab0 0x7f5830003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5830004810 0x7f5830004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830005210 0x7f583000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844156bf0 con 0x7f5830003ab0 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583000e340 con 0x7f5830005210 2023-10-08T02:28:43.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583000e500 con 0x7f5830004810 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584a19b700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5830003ab0 0x7f5830003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584a99c700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830005210 0x7f583000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.037+0000 7f584999a700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5830004810 0x7f5830004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f584a19b700 1 -- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5830004810 msgr2=0x7f5830004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f584a19b700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5830004810 0x7f5830004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f584a19b700 1 -- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830005210 msgr2=0x7f583000dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f584a19b700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830005210 0x7f583000dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f584a19b700 1 -- 172.21.15.119:0/190484399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5830003570 con 0x7f5830003ab0 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f584a99c700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830005210 0x7f583000dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f584a99c700 1 -- 172.21.15.119:0/190484399 reap_dead start 2023-10-08T02:28:43.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f584a19b700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5830003ab0 0x7f5830003ec0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f584000c2d0 tx=0x7f5840001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abce30242c5fe6e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584000cba0 con 0x7f5830003ab0 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584000cd00 con 0x7f5830003ab0 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.038+0000 7f5849199700 1 -- 172.21.15.119:0/190484399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5840005b70 con 0x7f5830003ab0 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.039+0000 7f584c3ff700 1 -- 172.21.15.119:0/190484399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5830003ab0 msgr2=0x7f5830003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.039+0000 7f584c3ff700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5830003ab0 0x7f5830003ec0 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f584000c2d0 tx=0x7f5840001d80 comp rx=0 tx=0).stop 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.039+0000 7f584c3ff700 1 -- 172.21.15.119:0/190484399 shutdown_connections 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.039+0000 7f584c3ff700 1 --2- 172.21.15.119:0/190484399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5830003ab0 0x7f5830003ec0 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.039+0000 7f584c3ff700 1 -- 172.21.15.119:0/190484399 >> 172.21.15.119:0/190484399 conn(0x7f584406a320 msgr2=0x7f58440cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.039+0000 7f584c3ff700 1 -- 172.21.15.119:0/190484399 shutdown_connections 2023-10-08T02:28:43.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.039+0000 7f584c3ff700 1 -- 172.21.15.119:0/190484399 wait complete. 2023-10-08T02:28:43.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.040+0000 7f584c3ff700 1 Processor -- start 2023-10-08T02:28:43.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.040+0000 7f584c3ff700 1 -- start start 2023-10-08T02:28:43.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584c3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830003ab0 0x7f5844156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584c3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440a99d0 0x7f58441664d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584c3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58440aa360 0x7f58440aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584c3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58440aad40 con 0x7f58440aa360 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584c3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5844166a10 con 0x7f5830003ab0 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584c3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58440ab1a0 con 0x7f58440a99d0 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584a99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58440aa360 0x7f58440aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584a99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58440aa360 0x7f58440aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48940/0 (socket says 172.21.15.119:48940) 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.041+0000 7f584a99c700 1 -- 172.21.15.119:0/1887797834 learned_addr learned my addr 172.21.15.119:0/1887797834 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584a19b700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830003ab0 0x7f5844156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584999a700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440a99d0 0x7f58441664d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584a99c700 1 -- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440a99d0 msgr2=0x7f58441664d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584a99c700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440a99d0 0x7f58441664d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584a99c700 1 -- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830003ab0 msgr2=0x7f5844156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584a99c700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830003ab0 0x7f5844156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584a99c700 1 -- 172.21.15.119:0/1887797834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5844169050 con 0x7f58440aa360 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584999a700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440a99d0 0x7f58441664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584a99c700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58440aa360 0x7f58440aa800 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f583c0094d0 tx=0x7f583c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f583a7fc700 1 -- 172.21.15.119:0/1887797834 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c00e040 con 0x7f58440aa360 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f583a7fc700 1 -- 172.21.15.119:0/1887797834 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583c00f040 con 0x7f58440aa360 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f583a7fc700 1 -- 172.21.15.119:0/1887797834 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c00fe90 con 0x7f58440aa360 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.042+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58441698f0 con 0x7f58440aa360 2023-10-08T02:28:43.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.043+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58441692a0 con 0x7f58440aa360 2023-10-08T02:28:43.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.044+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5844169a80 con 0x7f58440aa360 2023-10-08T02:28:43.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.044+0000 7f583a7fc700 1 -- 172.21.15.119:0/1887797834 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f583c015640 con 0x7f58440aa360 2023-10-08T02:28:43.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.045+0000 7f583a7fc700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5828066f30 0x7f58280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.045+0000 7f583a7fc700 1 -- 172.21.15.119:0/1887797834 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f583c0847a0 con 0x7f58440aa360 2023-10-08T02:28:43.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.046+0000 7f584a19b700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5828066f30 0x7f58280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.046+0000 7f584a19b700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5828066f30 0x7f58280693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58440ae230 tx=0x7f5840015000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.050+0000 7f583a7fc700 1 -- 172.21.15.119:0/1887797834 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f583c0552b0 con 0x7f58440aa360 2023-10-08T02:28:43.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.281+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f584404c3c0 con 0x7f58440aa360 2023-10-08T02:28:43.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.282+0000 7f583a7fc700 1 -- 172.21.15.119:0/1887797834 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f583c054e40 con 0x7f58440aa360 2023-10-08T02:28:43.283 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:43.283 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3041824229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3041824229},{"type":"v1","addr":"172.21.15.184:6834","nonce":3041824229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1478841333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1478841333},{"type":"v1","addr":"172.21.15.119:6834","nonce":1478841333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9723,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/483355636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":483355636},{"type":"v1","addr":"172.21.15.119:6835","nonce":483355636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:28:39.274713+0000","modified":"2023-10-08T02:28:42.824801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9726":{"gid":9726,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3185561357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3185561357},{"type":"v1","addr":"172.21.15.184:6835","nonce":3185561357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T02:28:43.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5828066f30 msgr2=0x7f58280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5828066f30 0x7f58280693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f58440ae230 tx=0x7f5840015000 comp rx=0 tx=0).stop 2023-10-08T02:28:43.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58440aa360 msgr2=0x7f58440aa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58440aa360 0x7f58440aa800 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f583c0094d0 tx=0x7f583c00b040 comp rx=0 tx=0).stop 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 shutdown_connections 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5828066f30 0x7f58280693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f58440aa360 0x7f58440aa800 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5830003ab0 0x7f5844156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 --2- 172.21.15.119:0/1887797834 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f58440a99d0 0x7f58441664d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 >> 172.21.15.119:0/1887797834 conn(0x7f584406a320 msgr2=0x7f58440ca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 shutdown_connections 2023-10-08T02:28:43.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.285+0000 7f584c3ff700 1 -- 172.21.15.119:0/1887797834 wait complete. 2023-10-08T02:28:43.287 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 87 2023-10-08T02:28:43.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:28:43.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.524+0000 7f4c0c5ff700 1 Processor -- start 2023-10-08T02:28:43.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.524+0000 7f4c0c5ff700 1 -- start start 2023-10-08T02:28:43.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.525+0000 7f4c0c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406bcd0 0x7f4c040caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.525+0000 7f4c0c5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c0415df50 0x7f4c0415e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.525+0000 7f4c0c5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c0415d7e0 con 0x7f4c0406c0a0 2023-10-08T02:28:43.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.525+0000 7f4c0c5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0415dac0 con 0x7f4c0415df50 2023-10-08T02:28:43.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.525+0000 7f4c0c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0415aa30 con 0x7f4c0406bcd0 2023-10-08T02:28:43.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0ab9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c0415df50 0x7f4c0415e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0ab9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c0415df50 0x7f4c0415e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48950/0 (socket says 172.21.15.119:48950) 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0ab9c700 1 -- 172.21.15.119:0/55182429 learned_addr learned my addr 172.21.15.119:0/55182429 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0a39b700 1 --2- 172.21.15.119:0/55182429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406bcd0 0x7f4c040caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0ab9c700 1 -- 172.21.15.119:0/55182429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406bcd0 msgr2=0x7f4c040caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0ab9c700 1 --2- 172.21.15.119:0/55182429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406bcd0 0x7f4c040caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0ab9c700 1 -- 172.21.15.119:0/55182429 >> v1:172.21.15.119:6789/0 conn(0x7f4c0406c0a0 legacy=0x7f4c0415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0ab9c700 1 -- 172.21.15.119:0/55182429 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0415ad10 con 0x7f4c0415df50 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c0ab9c700 1 --2- 172.21.15.119:0/55182429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c0415e3b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009f20 tx=0x7f4bfc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216375768bdc3d4d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.526+0000 7f4c09399700 1 -- 172.21.15.119:0/55182429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc00e070 con 0x7f4c0415df50 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c09399700 1 -- 172.21.15.119:0/55182429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc00ba80 con 0x7f4c0415df50 2023-10-08T02:28:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c09399700 1 -- 172.21.15.119:0/55182429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0092e0 con 0x7f4c0415df50 2023-10-08T02:28:43.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/55182429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 msgr2=0x7f4c0415e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/55182429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c0415e3b0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc009f20 tx=0x7f4bfc009840 comp rx=0 tx=0).stop 2023-10-08T02:28:43.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/55182429 shutdown_connections 2023-10-08T02:28:43.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/55182429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c0415e3b0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/55182429 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406bcd0 0x7f4c040caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/55182429 >> 172.21.15.119:0/55182429 conn(0x7f4c040ce600 msgr2=0x7f4c040cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/55182429 shutdown_connections 2023-10-08T02:28:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.527+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/55182429 wait complete. 2023-10-08T02:28:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.529+0000 7f4c0c5ff700 1 Processor -- start 2023-10-08T02:28:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.529+0000 7f4c0c5ff700 1 -- start start 2023-10-08T02:28:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0c5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c0406bcd0 0x7f4c041482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0c5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406c0a0 0x7f4c040b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0c5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c040b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0c5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0415a320 con 0x7f4c0415df50 2023-10-08T02:28:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0c5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0415a600 con 0x7f4c0406bcd0 2023-10-08T02:28:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0c5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0415ebd0 con 0x7f4c0406c0a0 2023-10-08T02:28:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0ab9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c040b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0ab9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c040b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48966/0 (socket says 172.21.15.119:48966) 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0ab9c700 1 -- 172.21.15.119:0/1144262836 learned_addr learned my addr 172.21.15.119:0/1144262836 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0a39b700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c0406bcd0 0x7f4c041482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c09b9a700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406c0a0 0x7f4c040b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0ab9c700 1 -- 172.21.15.119:0/1144262836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406c0a0 msgr2=0x7f4c040b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0ab9c700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406c0a0 0x7f4c040b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0ab9c700 1 -- 172.21.15.119:0/1144262836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c0406bcd0 msgr2=0x7f4c041482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0ab9c700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c0406bcd0 0x7f4c041482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.530+0000 7f4c0ab9c700 1 -- 172.21.15.119:0/1144262836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc009ae0 con 0x7f4c0415df50 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.531+0000 7f4c0ab9c700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c040b2660 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00b6c0 tx=0x7f4bfc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.531+0000 7f4bfa7fc700 1 -- 172.21.15.119:0/1144262836 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc0092e0 con 0x7f4c0415df50 2023-10-08T02:28:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.531+0000 7f4bfa7fc700 1 -- 172.21.15.119:0/1144262836 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc007e00 con 0x7f4c0415df50 2023-10-08T02:28:43.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.531+0000 7f4bfa7fc700 1 -- 172.21.15.119:0/1144262836 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc01d650 con 0x7f4c0415df50 2023-10-08T02:28:43.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.531+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0415eeb0 con 0x7f4c0415df50 2023-10-08T02:28:43.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.531+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c041487f0 con 0x7f4c0415df50 2023-10-08T02:28:43.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.532+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c041489e0 con 0x7f4c0415df50 2023-10-08T02:28:43.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.533+0000 7f4bfa7fc700 1 -- 172.21.15.119:0/1144262836 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bfc003050 con 0x7f4c0415df50 2023-10-08T02:28:43.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.534+0000 7f4bfa7fc700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4be8066f30 0x7f4be80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.534+0000 7f4bfa7fc700 1 -- 172.21.15.119:0/1144262836 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f4bfc086510 con 0x7f4c0415df50 2023-10-08T02:28:43.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.534+0000 7f4c0a39b700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4be8066f30 0x7f4be80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.534+0000 7f4c0a39b700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4be8066f30 0x7f4be80693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c00006fd0 tx=0x7f4c00008040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.538+0000 7f4bfa7fc700 1 -- 172.21.15.119:0/1144262836 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bfc0534f0 con 0x7f4c0415df50 2023-10-08T02:28:43.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.731+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c0406b040 con 0x7f4c0415df50 2023-10-08T02:28:43.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.732+0000 7f4bfa7fc700 1 -- 172.21.15.119:0/1144262836 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16171 (secure 0 0 0) 0x7f4bfc056b10 con 0x7f4c0415df50 2023-10-08T02:28:43.733 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:43.733 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":191,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:28:40.305967+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:28:37.482739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T02:28:38.257857+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4be8066f30 msgr2=0x7f4be80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4be8066f30 0x7f4be80693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4c00006fd0 tx=0x7f4c00008040 comp rx=0 tx=0).stop 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 msgr2=0x7f4c040b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c040b2660 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc00b6c0 tx=0x7f4bfc018000 comp rx=0 tx=0).stop 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 shutdown_connections 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4be8066f30 0x7f4be80693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c0415df50 0x7f4c040b2660 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c0406bcd0 0x7f4c041482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 --2- 172.21.15.119:0/1144262836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c0406c0a0 0x7f4c040b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 >> 172.21.15.119:0/1144262836 conn(0x7f4c040ce600 msgr2=0x7f4c0406cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:43.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 shutdown_connections 2023-10-08T02:28:43.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.736+0000 7f4c0c5ff700 1 -- 172.21.15.119:0/1144262836 wait complete. 2023-10-08T02:28:43.753 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T02:28:43.754 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:28:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.937+0000 7f70e35ff700 1 Processor -- start 2023-10-08T02:28:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.937+0000 7f70e35ff700 1 -- start start 2023-10-08T02:28:43.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.938+0000 7f70e35ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f70dc0cbf30 0x7f70dc0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.938+0000 7f70e35ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc0ccf90 0x7f70dc0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.938+0000 7f70e35ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70dc15a270 0x7f70dc0b1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.938+0000 7f70e35ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc156bf0 con 0x7f70dc15a270 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.938+0000 7f70e35ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc0ae950 con 0x7f70dc0cbf30 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.938+0000 7f70e35ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0cc300 con 0x7f70dc0ccf90 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e1b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70dc15a270 0x7f70dc0b1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e1b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70dc15a270 0x7f70dc0b1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48978/0 (socket says 172.21.15.119:48978) 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e1b9c700 1 -- 172.21.15.119:0/2081180262 learned_addr learned my addr 172.21.15.119:0/2081180262 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e139b700 1 --2- 172.21.15.119:0/2081180262 >> v2:172.21.15.184:3300/0 conn(0x7f70dc0cbf30 0x7f70dc0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e0b9a700 1 --2- 172.21.15.119:0/2081180262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc0ccf90 0x7f70dc0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e1b9c700 1 -- 172.21.15.119:0/2081180262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc0ccf90 msgr2=0x7f70dc0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e1b9c700 1 --2- 172.21.15.119:0/2081180262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc0ccf90 0x7f70dc0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e1b9c700 1 -- 172.21.15.119:0/2081180262 >> v2:172.21.15.184:3300/0 conn(0x7f70dc0cbf30 msgr2=0x7f70dc0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e1b9c700 1 --2- 172.21.15.119:0/2081180262 >> v2:172.21.15.184:3300/0 conn(0x7f70dc0cbf30 0x7f70dc0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e1b9c700 1 -- 172.21.15.119:0/2081180262 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0ad370 con 0x7f70dc15a270 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e0b9a700 1 --2- 172.21.15.119:0/2081180262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc0ccf90 0x7f70dc0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.939+0000 7f70e139b700 1 --2- 172.21.15.119:0/2081180262 >> v2:172.21.15.184:3300/0 conn(0x7f70dc0cbf30 0x7f70dc0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70e1b9c700 1 --2- 172.21.15.119:0/2081180262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a270 0x7f70dc0b1250 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f70dc063d40 tx=0x7f70d4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8013c0f122cee2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70d3fff700 1 -- 172.21.15.119:0/2081180262 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d400e070 con 0x7f70dc15a270 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70d3fff700 1 -- 172.21.15.119:0/2081180262 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d400ba40 con 0x7f70dc15a270 2023-10-08T02:28:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70d3fff700 1 -- 172.21.15.119:0/2081180262 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40092e0 con 0x7f70dc15a270 2023-10-08T02:28:43.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70e35ff700 1 -- 172.21.15.119:0/2081180262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a270 msgr2=0x7f70dc0b1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70e35ff700 1 --2- 172.21.15.119:0/2081180262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a270 0x7f70dc0b1250 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f70dc063d40 tx=0x7f70d4009840 comp rx=0 tx=0).stop 2023-10-08T02:28:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70e35ff700 1 -- 172.21.15.119:0/2081180262 shutdown_connections 2023-10-08T02:28:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70e35ff700 1 --2- 172.21.15.119:0/2081180262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc15a270 0x7f70dc0b1250 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70e35ff700 1 --2- 172.21.15.119:0/2081180262 >> v2:172.21.15.184:3300/0 conn(0x7f70dc0cbf30 0x7f70dc0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70e35ff700 1 --2- 172.21.15.119:0/2081180262 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc0ccf90 0x7f70dc0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.940+0000 7f70e35ff700 1 -- 172.21.15.119:0/2081180262 >> 172.21.15.119:0/2081180262 conn(0x7f70dc06a320 msgr2=0x7f70dc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.941+0000 7f70e35ff700 1 -- 172.21.15.119:0/2081180262 shutdown_connections 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.941+0000 7f70e35ff700 1 -- 172.21.15.119:0/2081180262 wait complete. 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.942+0000 7f70e35ff700 1 Processor -- start 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.942+0000 7f70e35ff700 1 -- start start 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e35ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc0cbf30 0x7f70dc148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e35ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0ccf90 0x7f70dc14e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e35ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc15a270 0x7f70dc14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e35ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc157420 con 0x7f70dc0ccf90 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e35ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc155880 con 0x7f70dc0cbf30 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e35ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc1555a0 con 0x7f70dc15a270 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e0b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0ccf90 0x7f70dc14e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e0b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0ccf90 0x7f70dc14e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48980/0 (socket says 172.21.15.119:48980) 2023-10-08T02:28:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e0b9a700 1 -- 172.21.15.119:0/1795251582 learned_addr learned my addr 172.21.15.119:0/1795251582 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e139b700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc0cbf30 0x7f70dc148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.943+0000 7f70e1b9c700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc15a270 0x7f70dc14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70e0b9a700 1 -- 172.21.15.119:0/1795251582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc15a270 msgr2=0x7f70dc14ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70e0b9a700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc15a270 0x7f70dc14ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70e0b9a700 1 -- 172.21.15.119:0/1795251582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc0cbf30 msgr2=0x7f70dc148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70e0b9a700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc0cbf30 0x7f70dc148f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70e0b9a700 1 -- 172.21.15.119:0/1795251582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4009ae0 con 0x7f70dc0ccf90 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70e0b9a700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0ccf90 0x7f70dc14e7a0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00e850 tx=0x7f70cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70d17fa700 1 -- 172.21.15.119:0/1795251582 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc013410 con 0x7f70dc0ccf90 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70d17fa700 1 -- 172.21.15.119:0/1795251582 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc013570 con 0x7f70dc0ccf90 2023-10-08T02:28:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70d17fa700 1 -- 172.21.15.119:0/1795251582 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc013910 con 0x7f70dc0ccf90 2023-10-08T02:28:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc156dd0 con 0x7f70dc0ccf90 2023-10-08T02:28:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.944+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc151470 con 0x7f70dc0ccf90 2023-10-08T02:28:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.945+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc04c3c0 con 0x7f70dc0ccf90 2023-10-08T02:28:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.946+0000 7f70d17fa700 1 -- 172.21.15.119:0/1795251582 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f70cc013d00 con 0x7f70dc0ccf90 2023-10-08T02:28:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.947+0000 7f70d17fa700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f70c0066f30 0x7f70c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.947+0000 7f70d17fa700 1 -- 172.21.15.119:0/1795251582 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f70cc084590 con 0x7f70dc0ccf90 2023-10-08T02:28:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.948+0000 7f70e139b700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f70c0066f30 0x7f70c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:43.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.948+0000 7f70e139b700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f70c0066f30 0x7f70c00693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f70d8006f00 tx=0x7f70d8008040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:43.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:43.951+0000 7f70d17fa700 1 -- 172.21.15.119:0/1795251582 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f70cc051570 con 0x7f70dc0ccf90 2023-10-08T02:28:44.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.139+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f70dc06f8a0 con 0x7f70c0066f30 2023-10-08T02:28:44.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.149+0000 7f70d17fa700 1 -- 172.21.15.119:0/1795251582 <== mgr.9543 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278167 (secure 0 0 0) 0x7f70dc06f8a0 con 0x7f70c0066f30 2023-10-08T02:28:44.152 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:44.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f70c0066f30 msgr2=0x7f70c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f70c0066f30 0x7f70c00693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f70d8006f00 tx=0x7f70d8008040 comp rx=0 tx=0).stop 2023-10-08T02:28:44.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0ccf90 msgr2=0x7f70dc14e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0ccf90 0x7f70dc14e7a0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f70cc00e850 tx=0x7f70cc008040 comp rx=0 tx=0).stop 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 shutdown_connections 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f70c0066f30 0x7f70c00693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f70dc15a270 0x7f70dc14ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f70dc0cbf30 0x7f70dc148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 --2- 172.21.15.119:0/1795251582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70dc0ccf90 0x7f70dc14e7a0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 >> 172.21.15.119:0/1795251582 conn(0x7f70dc06a320 msgr2=0x7f70dc068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 shutdown_connections 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.153+0000 7f70e35ff700 1 -- 172.21.15.119:0/1795251582 wait complete. 2023-10-08T02:28:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:28:44.170 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:28:43.770597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2475656,"kb_used_data":19528,"kb_used_omap":0,"kb_used_meta":2456064,"kb_avail":752499064,"statfs":{"total":773094113280,"available":770559041536,"internally_reserved":0,"allocated":19996672,"data_stored":10635952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2515009536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.975629"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802674+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:39.191107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801423+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:39.191107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803034+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:29.315855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802786+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798035+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803282+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799693+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:28:53.318755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803255+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:24.497478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799922+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799890+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801449+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:47.028965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803661+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797647+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:41.416738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803853+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802027+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:46.875812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798079+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799538+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803223+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:46.875812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802988+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:02.101147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802727+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:41.323520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802672+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:47.028965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802551+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:48.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803503+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:43.348294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797771+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:31.203917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798035+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798921+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:58:46.910335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797915+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:31.497164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801810+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:02.101147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802569+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:57.132332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798137+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:25.612201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.800911+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:33.526899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.800968+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:46.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"180'326","reported_seq":1045,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-08T02:28:36.018517+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:28:36.018517+0000","last_peered":"2023-10-08T02:28:36.018517+0000","last_clean":"2023-10-08T02:28:36.018517+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:28:36.018517+0000","last_undegraded":"2023-10-08T02:28:36.018517+0000","last_fullsized":"2023-10-08T02:28:36.018517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312452,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059388,"statfs":{"total":96636764160,"available":96316813312,"internally_reserved":0,"allocated":2945024,"data_stored":1772270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.94999999999999996,"15min":1.0429999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.48099999999999998},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.052},"last":0.624}]},{"osd":1,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.61499999999999999,"5min":0.47499999999999998,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.61499999999999999,"5min":0.47699999999999998,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.01,"15min":1.0549999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.60299999999999998,"5min":0.503,"15min":0.47199999999999998},"max":{"1min":0.92900000000000005,"5min":1.03,"15min":1.03},"last":0.73899999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46100000000000002},"max":{"1min":0.872,"5min":0.91000000000000003,"15min":0.96399999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.625,"5min":0.60899999999999999,"15min":0.51300000000000001},"max":{"1min":0.90300000000000002,"5min":1.056,"15min":1.056},"last":0.91500000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":1.091,"15min":1.091},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.76900000000000002,"5min":0.92200000000000004,"15min":0.95699999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1399999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.39400000000000002},"max":{"1min":0.88,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.77400000000000002},"min":{"1min":0.60699999999999998,"5min":0.54500000000000004,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":1.151,"15min":1.151},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.004,"15min":1.0329999999999999},"last":0.71599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299584,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072256,"statfs":{"total":96636764160,"available":96329990144,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.41799999999999998},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.61799999999999999,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":0.97799999999999998,"15min":1.21},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.78200000000000003,"15min":0.75800000000000001},"min":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.45700000000000002},"max":{"1min":3.851,"5min":3.851,"15min":3.851},"last":0.74199999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.60399999999999998,"5min":0.498,"15min":0.44400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":1.002,"15min":1.258},"last":0.68999999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":2.6970000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.58499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.376},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.42599999999999999},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.82799999999999996,"5min":0.86299999999999999,"15min":0.92800000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.56799999999999995}]},{"osd":7,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.0980000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.56599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97399999999999998,"15min":1.054},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311872,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059968,"statfs":{"total":96636764160,"available":96317407232,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.70299999999999996,"15min":1.0780000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":1.571},"last":0.45100000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.318},"max":{"1min":0.624,"5min":0.70899999999999996,"15min":1.1100000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":0.71799999999999997,"5min":0.85899999999999999,"15min":1.1299999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40699999999999997},"max":{"1min":0.78600000000000003,"5min":0.875,"15min":1.2190000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.99099999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.61099999999999999,"5min":0.53500000000000003,"15min":0.48199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.40600000000000003},"max":{"1min":1.05,"5min":1.1539999999999999,"15min":1.1950000000000001},"last":0.90600000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0509999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81799999999999995,"15min":0.82099999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.127,"15min":1.27},"last":0.79100000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.141,"15min":1.141},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.60999999999999999,"5min":0.57099999999999995,"15min":0.50900000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.204},"last":0.82699999999999996}]},{"osd":7,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.61199999999999999},"max":{"1min":1.109,"5min":1.1359999999999999,"15min":1.306},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.55500000000000005},"max":{"1min":1.0629999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.77700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575951,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308356,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063484,"statfs":{"total":96636764160,"available":96321007616,"internally_reserved":0,"allocated":2945024,"data_stored":1772270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.35699999999999998,"5min":0.31,"15min":0.23699999999999999},"max":{"1min":0.65000000000000002,"5min":0.73599999999999999,"15min":0.86499999999999999},"last":0.439},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.36499999999999999,"5min":0.33500000000000002,"15min":0.309},"max":{"1min":0.76300000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.27900000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48799999999999999,"15min":0.48699999999999999},"min":{"1min":0.33600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.77700000000000002,"5min":0.77800000000000002,"15min":0.84399999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34599999999999997},"max":{"1min":0.67300000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.69799999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.52900000000000003,"5min":0.46700000000000003,"15min":0.45000000000000001},"max":{"1min":0.81299999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79300000000000004,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.45700000000000002},"max":{"1min":1.071,"5min":1.232,"15min":1.232},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80900000000000005,"15min":0.80200000000000005},"min":{"1min":0.61299999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.147,"5min":1.262,"15min":1.262},"last":0.94099999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.55700000000000005},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.216},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88200000000000001,"15min":0.876},"min":{"1min":0.71899999999999997,"5min":0.60499999999999998,"15min":0.55500000000000005},"max":{"1min":1.0980000000000001,"5min":1.296,"15min":1.296},"last":0.88}]},{"osd":6,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.66800000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.095,"5min":1.125,"15min":1.1890000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80500000000000005,"15min":0.80300000000000005},"min":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.48999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.753}]},{"osd":7,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.85999999999999999,"15min":0.84699999999999998},"min":{"1min":0.68100000000000005,"5min":0.629,"15min":0.52000000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.296},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.55700000000000005},"max":{"1min":1.198,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.82499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315968,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055872,"statfs":{"total":96636764160,"available":96313212928,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.71999999999999997,"15min":0.871},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.83199999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.86299999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.36599999999999999},"max":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81999999999999995},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63400000000000001,"15min":0.622},"min":{"1min":0.505,"5min":0.47699999999999998,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.90000000000000002},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81299999999999994,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.49099999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85999999999999999,"15min":0.84199999999999997},"min":{"1min":0.65600000000000003,"5min":0.56499999999999995,"15min":0.46400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88600000000000001,"15min":0.85899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.417},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83899999999999997,"15min":0.81799999999999995},"min":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.333},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.84499999999999997,"15min":0.82999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.51800000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84899999999999998,"15min":0.82599999999999996},"min":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.4059999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.85199999999999998,"15min":0.84099999999999997},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56899999999999995},"max":{"1min":1.252,"5min":1.252,"15min":1.5720000000000001},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299584,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072256,"statfs":{"total":96636764160,"available":96329990144,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.29999999999999999,"15min":0.26200000000000001},"max":{"1min":0.66800000000000004,"5min":0.84699999999999998,"15min":0.90300000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.499,"15min":0.5},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.311},"max":{"1min":0.70599999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.308},"max":{"1min":0.64200000000000002,"5min":0.65000000000000002,"15min":0.71399999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.34599999999999997},"max":{"1min":0.69199999999999995,"5min":1.012,"15min":1.012},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.439,"5min":0.38900000000000001,"15min":0.38800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":1.282},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.49199999999999999,"15min":0.42999999999999999},"max":{"1min":0.86699999999999999,"5min":0.89300000000000002,"15min":1.512},"last":0.436}]},{"osd":4,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81000000000000005,"15min":0.77400000000000002},"min":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.48199999999999998},"max":{"1min":0.99099999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.879},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83199999999999996,"15min":0.80300000000000005},"min":{"1min":0.67900000000000005,"5min":0.58899999999999997,"15min":0.505},"max":{"1min":1.113,"5min":1.1399999999999999,"15min":1.3180000000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83899999999999997,"15min":0.81000000000000005},"min":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56100000000000005},"max":{"1min":1.0269999999999999,"5min":1.147,"15min":1.2789999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.80700000000000005},"min":{"1min":0.65100000000000002,"5min":0.628,"15min":0.52900000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.24},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.53600000000000003},"max":{"1min":1.0209999999999999,"5min":1.1599999999999999,"15min":1.161},"last":0.748},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84299999999999997,"15min":0.82099999999999995},"min":{"1min":0.60599999999999998,"5min":0.59999999999999998,"15min":0.52100000000000002},"max":{"1min":1.155,"5min":1.2490000000000001,"15min":1.2849999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.83299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.52500000000000002},"max":{"1min":1.081,"5min":1.22,"15min":1.266},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.86499999999999999,"15min":0.83599999999999997},"min":{"1min":0.70399999999999996,"5min":0.61499999999999999,"15min":0.59599999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.282},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575951,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315968,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055872,"statfs":{"total":96636764160,"available":96313212928,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.57899999999999996,"5min":0.48199999999999998,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":1.016,"15min":1.028},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.49099999999999999},"max":{"1min":0.92900000000000005,"5min":1.079,"15min":1.079},"last":0.65000000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.78100000000000003},"min":{"1min":0.55100000000000005,"5min":0.50800000000000001,"15min":0.44600000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.034},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.69399999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":1.097,"15min":1.097},"last":0.79700000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.47199999999999998},"max":{"1min":1.0029999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.872,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76100000000000001,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.48599999999999999,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":1.105,"15min":1.105},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.98299999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":1.03,"15min":1.03},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97099999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.42699999999999999},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311872,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059968,"statfs":{"total":96636764160,"available":96317407232,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.56499999999999995,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":0.996,"15min":0.996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.746},"min":{"1min":0.503,"5min":0.503,"15min":0.44600000000000001},"max":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":1.046},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.751,"15min":0.752},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.499},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.89200000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.76700000000000002},"min":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.48599999999999999},"max":{"1min":0.96099999999999997,"5min":1.028,"15min":1.028},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.76700000000000002},"min":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.51100000000000001},"max":{"1min":1.0009999999999999,"5min":1.006,"15min":1.006},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98899999999999999,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88600000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.503,"5min":0.434,"15min":0.434},"max":{"1min":0.872,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.93300000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":1.01,"15min":1.044},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.745,"15min":0.751},"min":{"1min":0.60999999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.94699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:28:44.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:28:44.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.390+0000 7f44d579d700 1 Processor -- start 2023-10-08T02:28:44.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.390+0000 7f44d579d700 1 -- start start 2023-10-08T02:28:44.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.391+0000 7f44d579d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f44d0150180 0x7f44d0157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.391+0000 7f44d579d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d006e2a0 con 0x7f44d00c7410 2023-10-08T02:28:44.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.391+0000 7f44d579d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d006b2e0 con 0x7f44d0150550 2023-10-08T02:28:44.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.391+0000 7f44d579d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00c78e0 con 0x7f44d0150180 2023-10-08T02:28:44.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.392+0000 7f44d4f9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f44d00c7410 0x7f44d0069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51684/0 (socket says 172.21.15.119:51684) 2023-10-08T02:28:44.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.392+0000 7f44d4f9c700 1 -- 172.21.15.119:0/4036287798 learned_addr learned my addr 172.21.15.119:0/4036287798 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:44.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.392+0000 7f44cffff700 1 --2- 172.21.15.119:0/4036287798 >> v2:172.21.15.184:3300/0 conn(0x7f44d0150180 0x7f44d0157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.392+0000 7f44ceffd700 1 -- 172.21.15.119:0/4036287798 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760461020 0 0) 0x7f44d006e2a0 con 0x7f44d00c7410 2023-10-08T02:28:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.392+0000 7f44ceffd700 1 -- 172.21.15.119:0/4036287798 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44b8003570 con 0x7f44d00c7410 2023-10-08T02:28:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.392+0000 7f44cffff700 1 -- 172.21.15.119:0/4036287798 >> v1:172.21.15.184:6789/0 conn(0x7f44d0150550 legacy=0x7f44d00c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.392+0000 7f44cffff700 1 -- 172.21.15.119:0/4036287798 >> v1:172.21.15.119:6789/0 conn(0x7f44d00c7410 legacy=0x7f44d0069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.392+0000 7f44cffff700 1 -- 172.21.15.119:0/4036287798 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00c7bc0 con 0x7f44d0150180 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.393+0000 7f44cffff700 1 --2- 172.21.15.119:0/4036287798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0150180 0x7f44d0157770 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f44c00098e0 tx=0x7f44c00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3860a2c9bf70f69 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.393+0000 7f44ceffd700 1 -- 172.21.15.119:0/4036287798 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c000b640 con 0x7f44d0150180 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.393+0000 7f44ceffd700 1 -- 172.21.15.119:0/4036287798 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c000b7a0 con 0x7f44d0150180 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.393+0000 7f44ceffd700 1 -- 172.21.15.119:0/4036287798 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c000bb40 con 0x7f44d0150180 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.393+0000 7f44d579d700 1 -- 172.21.15.119:0/4036287798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0150180 msgr2=0x7f44d0157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.393+0000 7f44d579d700 1 --2- 172.21.15.119:0/4036287798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0150180 0x7f44d0157770 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f44c00098e0 tx=0x7f44c00091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.394+0000 7f44d579d700 1 -- 172.21.15.119:0/4036287798 shutdown_connections 2023-10-08T02:28:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.394+0000 7f44d579d700 1 --2- 172.21.15.119:0/4036287798 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d0150180 0x7f44d0157770 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.394+0000 7f44d579d700 1 -- 172.21.15.119:0/4036287798 >> 172.21.15.119:0/4036287798 conn(0x7f44d00cc1f0 msgr2=0x7f44d00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:44.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.394+0000 7f44d579d700 1 -- 172.21.15.119:0/4036287798 shutdown_connections 2023-10-08T02:28:44.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.394+0000 7f44d579d700 1 -- 172.21.15.119:0/4036287798 wait complete. 2023-10-08T02:28:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.395+0000 7f44d579d700 1 Processor -- start 2023-10-08T02:28:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.395+0000 7f44d579d700 1 -- start start 2023-10-08T02:28:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d579d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d00c7410 0x7f44d00ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d579d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0150180 0x7f44d00cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d579d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0150550 0x7f44d00cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d579d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d016de20 con 0x7f44d0150550 2023-10-08T02:28:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d579d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d016afa0 con 0x7f44d00c7410 2023-10-08T02:28:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d579d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d016b280 con 0x7f44d0150180 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d4f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0150550 0x7f44d00cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d4f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0150550 0x7f44d00cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48988/0 (socket says 172.21.15.119:48988) 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.396+0000 7f44d4f9c700 1 -- 172.21.15.119:0/3748628911 learned_addr learned my addr 172.21.15.119:0/3748628911 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44cffff700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d00c7410 0x7f44d00ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44cf7fe700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0150180 0x7f44d00cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44d4f9c700 1 -- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0150180 msgr2=0x7f44d00cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44d4f9c700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0150180 0x7f44d00cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44d4f9c700 1 -- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d00c7410 msgr2=0x7f44d00ac470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44d4f9c700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d00c7410 0x7f44d00ac470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44d4f9c700 1 -- 172.21.15.119:0/3748628911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c0009420 con 0x7f44d0150550 2023-10-08T02:28:44.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44cffff700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d00c7410 0x7f44d00ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44d4f9c700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0150550 0x7f44d00cb6a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f44c800cc40 tx=0x7f44c8001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44b7fff700 1 -- 172.21.15.119:0/3748628911 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800a9c0 con 0x7f44d0150550 2023-10-08T02:28:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44b7fff700 1 -- 172.21.15.119:0/3748628911 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c800ab20 con 0x7f44d0150550 2023-10-08T02:28:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d016b560 con 0x7f44d0150550 2023-10-08T02:28:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44b7fff700 1 -- 172.21.15.119:0/3748628911 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c800aea0 con 0x7f44d0150550 2023-10-08T02:28:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.397+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d00b0250 con 0x7f44d0150550 2023-10-08T02:28:44.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.398+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d016e2b0 con 0x7f44d0150550 2023-10-08T02:28:44.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.399+0000 7f44b7fff700 1 -- 172.21.15.119:0/3748628911 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f44c8007300 con 0x7f44d0150550 2023-10-08T02:28:44.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.400+0000 7f44b7fff700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f44bc066f30 0x7f44bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.400+0000 7f44b7fff700 1 -- 172.21.15.119:0/3748628911 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f44c8084c40 con 0x7f44d0150550 2023-10-08T02:28:44.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.400+0000 7f44cffff700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f44bc066f30 0x7f44bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.401+0000 7f44cffff700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f44bc066f30 0x7f44bc0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f44c00093f0 tx=0x7f44c0017000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:44.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.405+0000 7f44b7fff700 1 -- 172.21.15.119:0/3748628911 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f44d016e2b0 con 0x7f44d0150550 2023-10-08T02:28:44.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.591+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44d016e2b0 con 0x7f44bc066f30 2023-10-08T02:28:44.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.599+0000 7f44b7fff700 1 -- 172.21.15.119:0/3748628911 <== mgr.9543 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278167 (secure 0 0 0) 0x7f44d016e2b0 con 0x7f44bc066f30 2023-10-08T02:28:44.602 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:44.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f44bc066f30 msgr2=0x7f44bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f44bc066f30 0x7f44bc0693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f44c00093f0 tx=0x7f44c0017000 comp rx=0 tx=0).stop 2023-10-08T02:28:44.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0150550 msgr2=0x7f44d00cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0150550 0x7f44d00cb6a0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f44c800cc40 tx=0x7f44c8001aa0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 shutdown_connections 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f44bc066f30 0x7f44bc0693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44d0150550 0x7f44d00cb6a0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f44d00c7410 0x7f44d00ac470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 --2- 172.21.15.119:0/3748628911 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f44d0150180 0x7f44d00cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 >> 172.21.15.119:0/3748628911 conn(0x7f44d00cc1f0 msgr2=0x7f44d00c9540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 shutdown_connections 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.603+0000 7f44d579d700 1 -- 172.21.15.119:0/3748628911 wait complete. 2023-10-08T02:28:44.605 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:28:44.620 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:28:43.770597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2475656,"kb_used_data":19528,"kb_used_omap":0,"kb_used_meta":2456064,"kb_avail":752499064,"statfs":{"total":773094113280,"available":770559041536,"internally_reserved":0,"allocated":19996672,"data_stored":10635952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2515009536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.975629"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802674+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:39.191107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801423+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:39.191107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803034+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:29.315855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802786+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798035+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803282+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799693+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:28:53.318755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803255+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:24.497478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799922+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799890+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801449+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:47.028965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803661+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797647+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:41.416738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803853+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802027+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:46.875812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798079+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799538+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803223+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:46.875812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802988+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:02.101147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802727+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:41.323520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802672+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:47.028965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802551+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:48.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803503+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:43.348294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797771+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:31.203917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798035+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798921+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:58:46.910335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797915+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:31.497164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801810+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:02.101147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802569+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:57.132332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798137+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:25.612201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.800911+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:33.526899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.800968+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:46.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"180'326","reported_seq":1045,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-08T02:28:36.018517+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:28:36.018517+0000","last_peered":"2023-10-08T02:28:36.018517+0000","last_clean":"2023-10-08T02:28:36.018517+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:28:36.018517+0000","last_undegraded":"2023-10-08T02:28:36.018517+0000","last_fullsized":"2023-10-08T02:28:36.018517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312452,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059388,"statfs":{"total":96636764160,"available":96316813312,"internally_reserved":0,"allocated":2945024,"data_stored":1772270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.94999999999999996,"15min":1.0429999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.48099999999999998},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.052},"last":0.624}]},{"osd":1,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.61499999999999999,"5min":0.47499999999999998,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.61499999999999999,"5min":0.47699999999999998,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.01,"15min":1.0549999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.60299999999999998,"5min":0.503,"15min":0.47199999999999998},"max":{"1min":0.92900000000000005,"5min":1.03,"15min":1.03},"last":0.73899999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46100000000000002},"max":{"1min":0.872,"5min":0.91000000000000003,"15min":0.96399999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.625,"5min":0.60899999999999999,"15min":0.51300000000000001},"max":{"1min":0.90300000000000002,"5min":1.056,"15min":1.056},"last":0.91500000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":1.091,"15min":1.091},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.76900000000000002,"5min":0.92200000000000004,"15min":0.95699999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1399999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.39400000000000002},"max":{"1min":0.88,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.77400000000000002},"min":{"1min":0.60699999999999998,"5min":0.54500000000000004,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":1.151,"15min":1.151},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.004,"15min":1.0329999999999999},"last":0.71599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299584,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072256,"statfs":{"total":96636764160,"available":96329990144,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.41799999999999998},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.61799999999999999,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":0.97799999999999998,"15min":1.21},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.78200000000000003,"15min":0.75800000000000001},"min":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.45700000000000002},"max":{"1min":3.851,"5min":3.851,"15min":3.851},"last":0.74199999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.60399999999999998,"5min":0.498,"15min":0.44400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":1.002,"15min":1.258},"last":0.68999999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":2.6970000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.58499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.376},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.42599999999999999},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.82799999999999996,"5min":0.86299999999999999,"15min":0.92800000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.56799999999999995}]},{"osd":7,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.0980000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.56599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97399999999999998,"15min":1.054},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311872,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059968,"statfs":{"total":96636764160,"available":96317407232,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.70299999999999996,"15min":1.0780000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":1.571},"last":0.45100000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.318},"max":{"1min":0.624,"5min":0.70899999999999996,"15min":1.1100000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":0.71799999999999997,"5min":0.85899999999999999,"15min":1.1299999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40699999999999997},"max":{"1min":0.78600000000000003,"5min":0.875,"15min":1.2190000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.99099999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.61099999999999999,"5min":0.53500000000000003,"15min":0.48199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.40600000000000003},"max":{"1min":1.05,"5min":1.1539999999999999,"15min":1.1950000000000001},"last":0.90600000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0509999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81799999999999995,"15min":0.82099999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.127,"15min":1.27},"last":0.79100000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.141,"15min":1.141},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.60999999999999999,"5min":0.57099999999999995,"15min":0.50900000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.204},"last":0.82699999999999996}]},{"osd":7,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.61199999999999999},"max":{"1min":1.109,"5min":1.1359999999999999,"15min":1.306},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.55500000000000005},"max":{"1min":1.0629999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.77700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575951,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308356,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063484,"statfs":{"total":96636764160,"available":96321007616,"internally_reserved":0,"allocated":2945024,"data_stored":1772270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.35699999999999998,"5min":0.31,"15min":0.23699999999999999},"max":{"1min":0.65000000000000002,"5min":0.73599999999999999,"15min":0.86499999999999999},"last":0.439},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.36499999999999999,"5min":0.33500000000000002,"15min":0.309},"max":{"1min":0.76300000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.27900000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48799999999999999,"15min":0.48699999999999999},"min":{"1min":0.33600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.77700000000000002,"5min":0.77800000000000002,"15min":0.84399999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34599999999999997},"max":{"1min":0.67300000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.69799999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.52900000000000003,"5min":0.46700000000000003,"15min":0.45000000000000001},"max":{"1min":0.81299999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79300000000000004,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.45700000000000002},"max":{"1min":1.071,"5min":1.232,"15min":1.232},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80900000000000005,"15min":0.80200000000000005},"min":{"1min":0.61299999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.147,"5min":1.262,"15min":1.262},"last":0.94099999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.55700000000000005},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.216},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88200000000000001,"15min":0.876},"min":{"1min":0.71899999999999997,"5min":0.60499999999999998,"15min":0.55500000000000005},"max":{"1min":1.0980000000000001,"5min":1.296,"15min":1.296},"last":0.88}]},{"osd":6,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.66800000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.095,"5min":1.125,"15min":1.1890000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80500000000000005,"15min":0.80300000000000005},"min":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.48999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.753}]},{"osd":7,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.85999999999999999,"15min":0.84699999999999998},"min":{"1min":0.68100000000000005,"5min":0.629,"15min":0.52000000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.296},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.55700000000000005},"max":{"1min":1.198,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.82499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315968,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055872,"statfs":{"total":96636764160,"available":96313212928,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.71999999999999997,"15min":0.871},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.83199999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.86299999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.36599999999999999},"max":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81999999999999995},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63400000000000001,"15min":0.622},"min":{"1min":0.505,"5min":0.47699999999999998,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.90000000000000002},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81299999999999994,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.49099999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85999999999999999,"15min":0.84199999999999997},"min":{"1min":0.65600000000000003,"5min":0.56499999999999995,"15min":0.46400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88600000000000001,"15min":0.85899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.417},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83899999999999997,"15min":0.81799999999999995},"min":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.333},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.84499999999999997,"15min":0.82999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.51800000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84899999999999998,"15min":0.82599999999999996},"min":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.4059999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.85199999999999998,"15min":0.84099999999999997},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56899999999999995},"max":{"1min":1.252,"5min":1.252,"15min":1.5720000000000001},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299584,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072256,"statfs":{"total":96636764160,"available":96329990144,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.29999999999999999,"15min":0.26200000000000001},"max":{"1min":0.66800000000000004,"5min":0.84699999999999998,"15min":0.90300000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.499,"15min":0.5},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.311},"max":{"1min":0.70599999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.308},"max":{"1min":0.64200000000000002,"5min":0.65000000000000002,"15min":0.71399999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.34599999999999997},"max":{"1min":0.69199999999999995,"5min":1.012,"15min":1.012},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.439,"5min":0.38900000000000001,"15min":0.38800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":1.282},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.49199999999999999,"15min":0.42999999999999999},"max":{"1min":0.86699999999999999,"5min":0.89300000000000002,"15min":1.512},"last":0.436}]},{"osd":4,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81000000000000005,"15min":0.77400000000000002},"min":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.48199999999999998},"max":{"1min":0.99099999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.879},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83199999999999996,"15min":0.80300000000000005},"min":{"1min":0.67900000000000005,"5min":0.58899999999999997,"15min":0.505},"max":{"1min":1.113,"5min":1.1399999999999999,"15min":1.3180000000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83899999999999997,"15min":0.81000000000000005},"min":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56100000000000005},"max":{"1min":1.0269999999999999,"5min":1.147,"15min":1.2789999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.80700000000000005},"min":{"1min":0.65100000000000002,"5min":0.628,"15min":0.52900000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.24},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.53600000000000003},"max":{"1min":1.0209999999999999,"5min":1.1599999999999999,"15min":1.161},"last":0.748},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84299999999999997,"15min":0.82099999999999995},"min":{"1min":0.60599999999999998,"5min":0.59999999999999998,"15min":0.52100000000000002},"max":{"1min":1.155,"5min":1.2490000000000001,"15min":1.2849999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.83299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.52500000000000002},"max":{"1min":1.081,"5min":1.22,"15min":1.266},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.86499999999999999,"15min":0.83599999999999997},"min":{"1min":0.70399999999999996,"5min":0.61499999999999999,"15min":0.59599999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.282},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575951,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315968,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055872,"statfs":{"total":96636764160,"available":96313212928,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.57899999999999996,"5min":0.48199999999999998,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":1.016,"15min":1.028},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.49099999999999999},"max":{"1min":0.92900000000000005,"5min":1.079,"15min":1.079},"last":0.65000000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.78100000000000003},"min":{"1min":0.55100000000000005,"5min":0.50800000000000001,"15min":0.44600000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.034},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.69399999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":1.097,"15min":1.097},"last":0.79700000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.47199999999999998},"max":{"1min":1.0029999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.872,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76100000000000001,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.48599999999999999,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":1.105,"15min":1.105},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.98299999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":1.03,"15min":1.03},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97099999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.42699999999999999},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311872,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059968,"statfs":{"total":96636764160,"available":96317407232,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.56499999999999995,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":0.996,"15min":0.996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.746},"min":{"1min":0.503,"5min":0.503,"15min":0.44600000000000001},"max":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":1.046},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.751,"15min":0.752},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.499},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.89200000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.76700000000000002},"min":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.48599999999999999},"max":{"1min":0.96099999999999997,"5min":1.028,"15min":1.028},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.76700000000000002},"min":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.51100000000000001},"max":{"1min":1.0009999999999999,"5min":1.006,"15min":1.006},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98899999999999999,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88600000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.503,"5min":0.434,"15min":0.434},"max":{"1min":0.872,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.93300000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":1.01,"15min":1.044},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.745,"15min":0.751},"min":{"1min":0.60999999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.94699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:28:44.627 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:28:44.788 INFO:tasks.ceph.mds.d.smithi184.stderr:2023-10-08T02:28:44.787+0000 7fd3e0ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T02:28:44.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.844+0000 7f8b363ff700 1 Processor -- start 2023-10-08T02:28:44.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.844+0000 7f8b363ff700 1 -- start start 2023-10-08T02:28:44.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.845+0000 7f8b363ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8b301578b0 0x7f8b30155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.845+0000 7f8b363ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8b300af2f0 0x7f8b300af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.845+0000 7f8b363ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b300ad930 con 0x7f8b30155c10 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.845+0000 7f8b363ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b300b0c80 con 0x7f8b301578b0 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.845+0000 7f8b363ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b300acd20 con 0x7f8b300af2f0 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8b30155c10 0x7f8b300ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51688/0 (socket says 172.21.15.119:51688) 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2ffff700 1 -- 172.21.15.119:0/75093811 learned_addr learned my addr 172.21.15.119:0/75093811 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2f7fe700 1 --2- 172.21.15.119:0/75093811 >> v2:172.21.15.119:3300/0 conn(0x7f8b301578b0 0x7f8b30155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b3499c700 1 --2- 172.21.15.119:0/75093811 >> v2:172.21.15.184:3300/0 conn(0x7f8b300af2f0 0x7f8b300af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2effd700 1 -- 172.21.15.119:0/75093811 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667101357 0 0) 0x7f8b300ad930 con 0x7f8b30155c10 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2effd700 1 -- 172.21.15.119:0/75093811 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b18003570 con 0x7f8b30155c10 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2f7fe700 1 -- 172.21.15.119:0/75093811 >> v2:172.21.15.184:3300/0 conn(0x7f8b300af2f0 msgr2=0x7f8b300af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2f7fe700 1 --2- 172.21.15.119:0/75093811 >> v2:172.21.15.184:3300/0 conn(0x7f8b300af2f0 0x7f8b300af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2f7fe700 1 -- 172.21.15.119:0/75093811 >> v1:172.21.15.119:6789/0 conn(0x7f8b30155c10 legacy=0x7f8b300ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b2f7fe700 1 -- 172.21.15.119:0/75093811 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b300ad000 con 0x7f8b301578b0 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.846+0000 7f8b3499c700 1 --2- 172.21.15.119:0/75093811 >> v2:172.21.15.184:3300/0 conn(0x7f8b300af2f0 0x7f8b300af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b2f7fe700 1 --2- 172.21.15.119:0/75093811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b30155fe0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f8b30065be0 tx=0x7f8b240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a484a2f96dd236f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b2effd700 1 -- 172.21.15.119:0/75093811 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24010600 con 0x7f8b301578b0 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b2effd700 1 -- 172.21.15.119:0/75093811 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b24010760 con 0x7f8b301578b0 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b2effd700 1 -- 172.21.15.119:0/75093811 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24010b00 con 0x7f8b301578b0 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b363ff700 1 -- 172.21.15.119:0/75093811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 msgr2=0x7f8b30155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b363ff700 1 --2- 172.21.15.119:0/75093811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b30155fe0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f8b30065be0 tx=0x7f8b240091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b363ff700 1 -- 172.21.15.119:0/75093811 shutdown_connections 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b363ff700 1 --2- 172.21.15.119:0/75093811 >> v2:172.21.15.184:3300/0 conn(0x7f8b300af2f0 0x7f8b300af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b363ff700 1 --2- 172.21.15.119:0/75093811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b30155fe0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b363ff700 1 -- 172.21.15.119:0/75093811 >> 172.21.15.119:0/75093811 conn(0x7f8b3006a320 msgr2=0x7f8b300cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b363ff700 1 -- 172.21.15.119:0/75093811 shutdown_connections 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.847+0000 7f8b363ff700 1 -- 172.21.15.119:0/75093811 wait complete. 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.849+0000 7f8b363ff700 1 Processor -- start 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.849+0000 7f8b363ff700 1 -- start start 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b363ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b300af2f0 0x7f8b3014df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b363ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b30155c10 0x7f8b3014a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b363ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b3014c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b363ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3014ae70 con 0x7f8b301578b0 2023-10-08T02:28:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b363ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3014cec0 con 0x7f8b300af2f0 2023-10-08T02:28:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b363ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3014f710 con 0x7f8b30155c10 2023-10-08T02:28:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b3499c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b3014c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b3499c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b3014c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49008/0 (socket says 172.21.15.119:49008) 2023-10-08T02:28:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b3499c700 1 -- 172.21.15.119:0/3076568285 learned_addr learned my addr 172.21.15.119:0/3076568285 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b2ffff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b300af2f0 0x7f8b3014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b2f7fe700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b30155c10 0x7f8b3014a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b3499c700 1 -- 172.21.15.119:0/3076568285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b30155c10 msgr2=0x7f8b3014a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b3499c700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b30155c10 0x7f8b3014a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b3499c700 1 -- 172.21.15.119:0/3076568285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b300af2f0 msgr2=0x7f8b3014df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b3499c700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b300af2f0 0x7f8b3014df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.850+0000 7f8b3499c700 1 -- 172.21.15.119:0/3076568285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b24009420 con 0x7f8b301578b0 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.851+0000 7f8b3499c700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b3014c980 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800d180 tx=0x7f8b28009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.851+0000 7f8b17fff700 1 -- 172.21.15.119:0/3076568285 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2800dcc0 con 0x7f8b301578b0 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.851+0000 7f8b17fff700 1 -- 172.21.15.119:0/3076568285 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2800f040 con 0x7f8b301578b0 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.851+0000 7f8b17fff700 1 -- 172.21.15.119:0/3076568285 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b28015440 con 0x7f8b301578b0 2023-10-08T02:28:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.851+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3014ede0 con 0x7f8b301578b0 2023-10-08T02:28:44.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.851+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3014f3a0 con 0x7f8b301578b0 2023-10-08T02:28:44.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.852+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3014d180 con 0x7f8b301578b0 2023-10-08T02:28:44.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.853+0000 7f8b17fff700 1 -- 172.21.15.119:0/3076568285 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b2801e3b0 con 0x7f8b301578b0 2023-10-08T02:28:44.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.854+0000 7f8b17fff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8b1c066f30 0x7f8b1c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:44.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.854+0000 7f8b17fff700 1 -- 172.21.15.119:0/3076568285 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f8b28084c40 con 0x7f8b301578b0 2023-10-08T02:28:44.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.855+0000 7f8b2ffff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8b1c066f30 0x7f8b1c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:44.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.855+0000 7f8b2ffff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8b1c066f30 0x7f8b1c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b20007fd0 tx=0x7f8b20009040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:44.858+0000 7f8b17fff700 1 -- 172.21.15.119:0/3076568285 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b28051c20 con 0x7f8b301578b0 2023-10-08T02:28:45.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.045+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8b3014f8a0 con 0x7f8b1c066f30 2023-10-08T02:28:45.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.053+0000 7f8b17fff700 1 -- 172.21.15.119:0/3076568285 <== mgr.9543 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278167 (secure 0 0 0) 0x7f8b3014f8a0 con 0x7f8b1c066f30 2023-10-08T02:28:45.057 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:45.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.056+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8b1c066f30 msgr2=0x7f8b1c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.056+0000 7f8b363ff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8b1c066f30 0x7f8b1c0693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8b20007fd0 tx=0x7f8b20009040 comp rx=0 tx=0).stop 2023-10-08T02:28:45.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 msgr2=0x7f8b3014c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b3014c980 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8b2800d180 tx=0x7f8b28009a50 comp rx=0 tx=0).stop 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 shutdown_connections 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8b1c066f30 0x7f8b1c0693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8b301578b0 0x7f8b3014c980 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8b300af2f0 0x7f8b3014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 --2- 172.21.15.119:0/3076568285 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8b30155c10 0x7f8b3014a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 >> 172.21.15.119:0/3076568285 conn(0x7f8b3006a320 msgr2=0x7f8b3006d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 shutdown_connections 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.057+0000 7f8b363ff700 1 -- 172.21.15.119:0/3076568285 wait complete. 2023-10-08T02:28:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:28:45.074 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:28:43.770597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2475656,"kb_used_data":19528,"kb_used_omap":0,"kb_used_meta":2456064,"kb_avail":752499064,"statfs":{"total":773094113280,"available":770559041536,"internally_reserved":0,"allocated":19996672,"data_stored":10635952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2515009536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.975629"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802674+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:39.191107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801423+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:39.191107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803034+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:29.315855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802786+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798035+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803282+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799693+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:28:53.318755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803255+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:39:24.497478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799922+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799890+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801449+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:47.028965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803661+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797647+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:41.416738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803853+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802027+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:46.875812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798079+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.799538+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803223+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:46.875812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802988+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:02.101147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802727+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:21:41.323520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802672+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:47.028965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802551+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:55:48.984273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.803503+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:43.348294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797771+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:31.203917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798035+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798921+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:58:46.910335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.797915+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:31.497164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.801810+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:02.101147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.802569+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:15:57.132332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.798137+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:10:25.612201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.800911+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:33.526899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-08T02:28:38.795054+0000","last_change":"2023-10-08T02:28:38.795054+0000","last_active":"2023-10-08T02:28:38.795054+0000","last_peered":"2023-10-08T02:28:38.795054+0000","last_clean":"2023-10-08T02:28:38.795054+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:38.795054+0000","last_undegraded":"2023-10-08T02:28:38.795054+0000","last_fullsized":"2023-10-08T02:28:38.795054+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-10-08T02:28:37.787122+0000","last_change":"2023-10-08T02:28:37.800968+0000","last_active":"2023-10-08T02:28:37.787122+0000","last_peered":"2023-10-08T02:28:37.787122+0000","last_clean":"2023-10-08T02:28:37.787122+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-08T02:28:37.787122+0000","last_undegraded":"2023-10-08T02:28:37.787122+0000","last_fullsized":"2023-10-08T02:28:37.787122+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:46.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"180'326","reported_seq":1045,"reported_epoch":187,"state":"active+clean","last_fresh":"2023-10-08T02:28:36.018517+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:28:36.018517+0000","last_peered":"2023-10-08T02:28:36.018517+0000","last_clean":"2023-10-08T02:28:36.018517+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:28:36.018517+0000","last_undegraded":"2023-10-08T02:28:36.018517+0000","last_fullsized":"2023-10-08T02:28:36.018517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312452,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059388,"statfs":{"total":96636764160,"available":96316813312,"internally_reserved":0,"allocated":2945024,"data_stored":1772270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.94999999999999996,"15min":1.0429999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.48099999999999998},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.052},"last":0.624}]},{"osd":1,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.61499999999999999,"5min":0.47499999999999998,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.61499999999999999,"5min":0.47699999999999998,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.76500000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.01,"15min":1.0549999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.60299999999999998,"5min":0.503,"15min":0.47199999999999998},"max":{"1min":0.92900000000000005,"5min":1.03,"15min":1.03},"last":0.73899999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46100000000000002},"max":{"1min":0.872,"5min":0.91000000000000003,"15min":0.96399999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.625,"5min":0.60899999999999999,"15min":0.51300000000000001},"max":{"1min":0.90300000000000002,"5min":1.056,"15min":1.056},"last":0.91500000000000004}]},{"osd":4,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":1.091,"15min":1.091},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.76900000000000002,"5min":0.92200000000000004,"15min":0.95699999999999996},"last":0.79400000000000004}]},{"osd":5,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1399999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.39400000000000002},"max":{"1min":0.88,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.90300000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.77400000000000002},"min":{"1min":0.60699999999999998,"5min":0.54500000000000004,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":1.151,"15min":1.151},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.004,"15min":1.0329999999999999},"last":0.71599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299584,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072256,"statfs":{"total":96636764160,"available":96329990144,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.41799999999999998},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.59899999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.61799999999999999,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":0.97799999999999998,"15min":1.21},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.78200000000000003,"15min":0.75800000000000001},"min":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.45700000000000002},"max":{"1min":3.851,"5min":3.851,"15min":3.851},"last":0.74199999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.60399999999999998,"5min":0.498,"15min":0.44400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":1.002,"15min":1.258},"last":0.68999999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":2.6970000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.58499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.376},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.42599999999999999},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.52300000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.82799999999999996,"5min":0.86299999999999999,"15min":0.92800000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.56799999999999995}]},{"osd":7,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.0980000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.56599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97399999999999998,"15min":1.054},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575952,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311872,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059968,"statfs":{"total":96636764160,"available":96317407232,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.70299999999999996,"15min":1.0780000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":1.571},"last":0.45100000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.318},"max":{"1min":0.624,"5min":0.70899999999999996,"15min":1.1100000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":0.71799999999999997,"5min":0.85899999999999999,"15min":1.1299999999999999},"last":0.47799999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40699999999999997},"max":{"1min":0.78600000000000003,"5min":0.875,"15min":1.2190000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.99099999999999999},"last":0.61099999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.61099999999999999,"5min":0.53500000000000003,"15min":0.48199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.92100000000000004},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.40600000000000003},"max":{"1min":1.05,"5min":1.1539999999999999,"15min":1.1950000000000001},"last":0.90600000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0509999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81799999999999995,"15min":0.82099999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.127,"15min":1.27},"last":0.79100000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.141,"15min":1.141},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.60999999999999999,"5min":0.57099999999999995,"15min":0.50900000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.204},"last":0.82699999999999996}]},{"osd":7,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.61199999999999999},"max":{"1min":1.109,"5min":1.1359999999999999,"15min":1.306},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.55500000000000005},"max":{"1min":1.0629999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.77700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575951,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308356,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063484,"statfs":{"total":96636764160,"available":96321007616,"internally_reserved":0,"allocated":2945024,"data_stored":1772270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.48199999999999998,"15min":0.47999999999999998},"min":{"1min":0.35699999999999998,"5min":0.31,"15min":0.23699999999999999},"max":{"1min":0.65000000000000002,"5min":0.73599999999999999,"15min":0.86499999999999999},"last":0.439},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51900000000000002,"15min":0.52300000000000002},"min":{"1min":0.36499999999999999,"5min":0.33500000000000002,"15min":0.309},"max":{"1min":0.76300000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.27900000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.48799999999999999,"15min":0.48699999999999999},"min":{"1min":0.33600000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.77700000000000002,"5min":0.77800000000000002,"15min":0.84399999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55900000000000005,"15min":0.54800000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34599999999999997},"max":{"1min":0.67300000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47099999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56499999999999995,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.69799999999999995,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.624,"15min":0.61199999999999999},"min":{"1min":0.52900000000000003,"5min":0.46700000000000003,"15min":0.45000000000000001},"max":{"1min":0.81299999999999994,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.42899999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.79300000000000004,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.45700000000000002},"max":{"1min":1.071,"5min":1.232,"15min":1.232},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.80900000000000005,"15min":0.80200000000000005},"min":{"1min":0.61299999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.147,"5min":1.262,"15min":1.262},"last":0.94099999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.55700000000000005},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.216},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.90600000000000003,"5min":0.88200000000000001,"15min":0.876},"min":{"1min":0.71899999999999997,"5min":0.60499999999999998,"15min":0.55500000000000005},"max":{"1min":1.0980000000000001,"5min":1.296,"15min":1.296},"last":0.88}]},{"osd":6,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.83099999999999996,"15min":0.83299999999999996},"min":{"1min":0.66800000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.095,"5min":1.125,"15min":1.1890000000000001},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.80500000000000005,"15min":0.80300000000000005},"min":{"1min":0.58099999999999996,"5min":0.54500000000000004,"15min":0.48999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.753}]},{"osd":7,"last update":"Sun Oct 8 02:27:42 2023","interfaces":[{"interface":"back","average":{"1min":0.90400000000000003,"5min":0.85999999999999999,"15min":0.84699999999999998},"min":{"1min":0.68100000000000005,"5min":0.629,"15min":0.52000000000000002},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.296},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86899999999999999,"15min":0.88200000000000001},"min":{"1min":0.69999999999999996,"5min":0.60199999999999998,"15min":0.55700000000000005},"max":{"1min":1.198,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.82499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315968,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055872,"statfs":{"total":96636764160,"available":96313212928,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.71999999999999997,"15min":0.871},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.83199999999999996},"last":0.41499999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.86299999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.36599999999999999},"max":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81999999999999995},"last":0.375}]},{"osd":3,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63400000000000001,"15min":0.622},"min":{"1min":0.505,"5min":0.47699999999999998,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.90000000000000002},"last":0.45200000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81299999999999994,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.49099999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85999999999999999,"15min":0.84199999999999997},"min":{"1min":0.65600000000000003,"5min":0.56499999999999995,"15min":0.46400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88600000000000001,"15min":0.85899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.417},"last":0.78800000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83899999999999997,"15min":0.81799999999999995},"min":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.333},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.872,"5min":0.84499999999999997,"15min":0.82999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.51800000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.69099999999999995}]},{"osd":7,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84899999999999998,"15min":0.82599999999999996},"min":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.4059999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.875,"5min":0.85199999999999998,"15min":0.84099999999999997},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56899999999999995},"max":{"1min":1.252,"5min":1.252,"15min":1.5720000000000001},"last":0.68000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299584,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072256,"statfs":{"total":96636764160,"available":96329990144,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46999999999999997,"15min":0.46000000000000002},"min":{"1min":0.33400000000000002,"5min":0.29999999999999999,"15min":0.26200000000000001},"max":{"1min":0.66800000000000004,"5min":0.84699999999999998,"15min":0.90300000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.499,"15min":0.5},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.311},"max":{"1min":0.70599999999999996,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.45000000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.308},"max":{"1min":0.64200000000000002,"5min":0.65000000000000002,"15min":0.71399999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.53400000000000003},"min":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.34599999999999997},"max":{"1min":0.69199999999999995,"5min":1.012,"15min":1.012},"last":0.39000000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.62,"15min":0.61599999999999999},"min":{"1min":0.439,"5min":0.38900000000000001,"15min":0.38800000000000001},"max":{"1min":0.879,"5min":0.879,"15min":1.282},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.63800000000000001},"min":{"1min":0.497,"5min":0.49199999999999999,"15min":0.42999999999999999},"max":{"1min":0.86699999999999999,"5min":0.89300000000000002,"15min":1.512},"last":0.436}]},{"osd":4,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.81000000000000005,"15min":0.77400000000000002},"min":{"1min":0.65500000000000003,"5min":0.57799999999999996,"15min":0.48199999999999998},"max":{"1min":0.99099999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.879},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83199999999999996,"15min":0.80300000000000005},"min":{"1min":0.67900000000000005,"5min":0.58899999999999997,"15min":0.505},"max":{"1min":1.113,"5min":1.1399999999999999,"15min":1.3180000000000001},"last":0.83499999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.83899999999999997,"15min":0.81000000000000005},"min":{"1min":0.59799999999999998,"5min":0.56899999999999995,"15min":0.56100000000000005},"max":{"1min":1.0269999999999999,"5min":1.147,"15min":1.2789999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.83199999999999996,"15min":0.80700000000000005},"min":{"1min":0.65100000000000002,"5min":0.628,"15min":0.52900000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.24},"last":0.64400000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.83299999999999996,"15min":0.81499999999999995},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.53600000000000003},"max":{"1min":1.0209999999999999,"5min":1.1599999999999999,"15min":1.161},"last":0.748},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.84299999999999997,"15min":0.82099999999999995},"min":{"1min":0.60599999999999998,"5min":0.59999999999999998,"15min":0.52100000000000002},"max":{"1min":1.155,"5min":1.2490000000000001,"15min":1.2849999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Sun Oct 8 02:27:39 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.86199999999999999,"15min":0.83299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.52500000000000002},"max":{"1min":1.081,"5min":1.22,"15min":1.266},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.86499999999999999,"15min":0.83599999999999997},"min":{"1min":0.70399999999999996,"5min":0.61499999999999999,"15min":0.59599999999999997},"max":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.282},"last":0.59399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575951,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315968,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055872,"statfs":{"total":96636764160,"available":96313212928,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.57899999999999996,"5min":0.48199999999999998,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":1.016,"15min":1.028},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.57899999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.49099999999999999},"max":{"1min":0.92900000000000005,"5min":1.079,"15min":1.079},"last":0.65000000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.78100000000000003},"min":{"1min":0.55100000000000005,"5min":0.50800000000000001,"15min":0.44600000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.034},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.69399999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":1.097,"15min":1.097},"last":0.79700000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.47199999999999998},"max":{"1min":1.0029999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.872,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76100000000000001,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.48599999999999999,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":1.105,"15min":1.105},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.98299999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":1.03,"15min":1.03},"last":0.78200000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97099999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.42699999999999999},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.83399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311872,"kb_used_data":2296,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059968,"statfs":{"total":96636764160,"available":96317407232,"internally_reserved":0,"allocated":2351104,"data_stored":1181902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.56499999999999995,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":0.996,"15min":0.996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.746,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.74199999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.746},"min":{"1min":0.503,"5min":0.503,"15min":0.44600000000000001},"max":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":1.046},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.751,"15min":0.752},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.499},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.89200000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.76700000000000002},"min":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.48599999999999999},"max":{"1min":0.96099999999999997,"5min":1.028,"15min":1.028},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.61799999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.76700000000000002},"min":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.51100000000000001},"max":{"1min":1.0009999999999999,"5min":1.006,"15min":1.006},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98899999999999999,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.82999999999999996}]},{"osd":4,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88600000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.503,"5min":0.434,"15min":0.434},"max":{"1min":0.872,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.93300000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":1.01,"15min":1.044},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.745,"15min":0.751},"min":{"1min":0.60999999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.94699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:28:45.083 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3f is not active+clean 2023-10-08T02:28:45.083 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3e is not active+clean 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3d is not active+clean 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3c is not active+clean 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3b is not active+clean 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.084 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3a is not active+clean 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.39 is not active+clean 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.38 is not active+clean 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.37 is not active+clean 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.36 is not active+clean 2023-10-08T02:28:45.085 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.35 is not active+clean 2023-10-08T02:28:45.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.34 is not active+clean 2023-10-08T02:28:45.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.086 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.33 is not active+clean 2023-10-08T02:28:45.086 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.32 is not active+clean 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.31 is not active+clean 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.30 is not active+clean 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2f is not active+clean 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.087 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2e is not active+clean 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2d is not active+clean 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2c is not active+clean 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2b is not active+clean 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2a is not active+clean 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.088 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.29 is not active+clean 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.28 is not active+clean 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.27 is not active+clean 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.26 is not active+clean 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.25 is not active+clean 2023-10-08T02:28:45.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.24 is not active+clean 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.23 is not active+clean 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.22 is not active+clean 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.21 is not active+clean 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.090 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.20 is not active+clean 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1e is not active+clean 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.802674+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:11:39.191107+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1f is not active+clean 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.801423+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:11:39.191107+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:28:45.091 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1c is not active+clean 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.803034+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:13:29.315855+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1d is not active+clean 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.802786+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1a is not active+clean 2023-10-08T02:28:45.092 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.798035+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1b is not active+clean 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.803282+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.18 is not active+clean 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2023-10-08T02:28:45.093 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.799693+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:28:53.318755+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.19 is not active+clean 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.803255+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:39:24.497478+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.17 is not active+clean 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.799922+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.14 is not active+clean 2023-10-08T02:28:45.094 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2023-10-08T02:28:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.799890+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:42:52.770802+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-10-08T02:28:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.15 is not active+clean 2023-10-08T02:28:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2023-10-08T02:28:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.801449+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:10:47.028965+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:28:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.12 is not active+clean 2023-10-08T02:28:45.095 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.803661+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.13 is not active+clean 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.797647+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:08:41.416738+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.10 is not active+clean 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.096 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.803853+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.11 is not active+clean 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.802027+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:55:46.875812+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.e is not active+clean 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2023-10-08T02:28:45.097 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.798079+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:48:03.340387+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:28:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.f is not active+clean 2023-10-08T02:28:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2023-10-08T02:28:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.799538+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:28:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.c is not active+clean 2023-10-08T02:28:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2023-10-08T02:28:45.098 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.803223+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T02:55:46.875812+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:28:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.d is not active+clean 2023-10-08T02:28:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2023-10-08T02:28:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.802988+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:54:02.101147+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:28:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.a is not active+clean 2023-10-08T02:28:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2023-10-08T02:28:45.099 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.802727+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:21:41.323520+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:28:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.b is not active+clean 2023-10-08T02:28:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2023-10-08T02:28:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.802672+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:10:47.028965+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:28:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.8 is not active+clean 2023-10-08T02:28:45.100 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2023-10-08T02:28:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.802551+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:55:48.984273+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:28:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.9 is not active+clean 2023-10-08T02:28:45.101 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2023-10-08T02:28:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.803503+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T09:48:43.348294+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-10-08T02:28:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.6 is not active+clean 2023-10-08T02:28:45.102 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2023-10-08T02:28:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.797771+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:13:31.203917+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:28:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.7 is not active+clean 2023-10-08T02:28:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.103 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2023-10-08T02:28:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.798035+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-10-08T02:28:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.4 is not active+clean 2023-10-08T02:28:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2023-10-08T02:28:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.798921+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T06:58:46.910335+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:28:45.104 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.5 is not active+clean 2023-10-08T02:28:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2023-10-08T02:28:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.797915+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T07:37:31.497164+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-10-08T02:28:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.2 is not active+clean 2023-10-08T02:28:45.105 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2023-10-08T02:28:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.801810+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T12:54:02.101147+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:28:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.3 is not active+clean 2023-10-08T02:28:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2023-10-08T02:28:45.106 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.802569+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T10:15:57.132332+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.0 is not active+clean 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.798137+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T05:10:25.612201+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.1 is not active+clean 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.800911+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T04:43:33.526899+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:28:45.107 INFO:tasks.cephfs.filesystem.ceph_manager:PG 23.16 is not active+clean 2023-10-08T02:28:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '23.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-08T02:28:38.795054+0000', 'last_change': '2023-10-08T02:28:38.795054+0000', 'last_active': '2023-10-08T02:28:38.795054+0000', 'last_peered': '2023-10-08T02:28:38.795054+0000', 'last_clean': '2023-10-08T02:28:38.795054+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:38.795054+0000', 'last_undegraded': '2023-10-08T02:28:38.795054+0000', 'last_fullsized': '2023-10-08T02:28:38.795054+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:38.795054+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-08T02:28:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2023-10-08T02:28:45.108 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-10-08T02:28:37.787122+0000', 'last_change': '2023-10-08T02:28:37.800968+0000', 'last_active': '2023-10-08T02:28:37.787122+0000', 'last_peered': '2023-10-08T02:28:37.787122+0000', 'last_clean': '2023-10-08T02:28:37.787122+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-08T02:28:37.787122+0000', 'last_undegraded': '2023-10-08T02:28:37.787122+0000', 'last_fullsized': '2023-10-08T02:28:37.787122+0000', 'mapping_epoch': 188, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 188, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'last_clean_scrub_stamp': '2023-10-08T02:28:37.787122+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-10-09T08:53:46.569514+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-10-08T02:28:45.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-08T02:28:45.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.294+0000 7fe7235ff700 1 Processor -- start 2023-10-08T02:28:45.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.294+0000 7fe7235ff700 1 -- start start 2023-10-08T02:28:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe7235ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0b0ca0 0x7fe71c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe7235ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe71c0acf10 0x7fe71c156e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe7235ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c153f10 con 0x7fe71c156510 2023-10-08T02:28:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe7235ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c0ad9d0 con 0x7fe71c0acf10 2023-10-08T02:28:45.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe7235ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0aed00 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe721b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe71c0acf10 0x7fe71c156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe721b9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe71c0acf10 0x7fe71c156e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37492/0 (socket says 172.21.15.119:37492) 2023-10-08T02:28:45.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe721b9c700 1 -- 172.21.15.119:0/2326310202 learned_addr learned my addr 172.21.15.119:0/2326310202 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:45.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe720b9a700 1 --2- 172.21.15.119:0/2326310202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0b0ca0 0x7fe71c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe721b9c700 1 -- 172.21.15.119:0/2326310202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0b0ca0 msgr2=0x7fe71c1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe721b9c700 1 --2- 172.21.15.119:0/2326310202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0b0ca0 0x7fe71c1568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe721b9c700 1 -- 172.21.15.119:0/2326310202 >> v1:172.21.15.184:6789/0 conn(0x7fe71c156510 legacy=0x7fe71c0ac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.296+0000 7fe721b9c700 1 -- 172.21.15.119:0/2326310202 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c0af800 con 0x7fe71c0acf10 2023-10-08T02:28:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.297+0000 7fe721b9c700 1 --2- 172.21.15.119:0/2326310202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0acf10 0x7fe71c156e20 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fe714009f20 tx=0x7fe714009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcd85f814f8fca2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.297+0000 7fe713fff700 1 -- 172.21.15.119:0/2326310202 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71400e070 con 0x7fe71c0acf10 2023-10-08T02:28:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.297+0000 7fe713fff700 1 -- 172.21.15.119:0/2326310202 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71400ba80 con 0x7fe71c0acf10 2023-10-08T02:28:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.297+0000 7fe713fff700 1 -- 172.21.15.119:0/2326310202 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7140092e0 con 0x7fe71c0acf10 2023-10-08T02:28:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.297+0000 7fe7235ff700 1 -- 172.21.15.119:0/2326310202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0acf10 msgr2=0x7fe71c156e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.297+0000 7fe7235ff700 1 --2- 172.21.15.119:0/2326310202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0acf10 0x7fe71c156e20 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fe714009f20 tx=0x7fe714009840 comp rx=0 tx=0).stop 2023-10-08T02:28:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.298+0000 7fe7235ff700 1 -- 172.21.15.119:0/2326310202 shutdown_connections 2023-10-08T02:28:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.298+0000 7fe7235ff700 1 --2- 172.21.15.119:0/2326310202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0acf10 0x7fe71c156e20 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.298+0000 7fe7235ff700 1 --2- 172.21.15.119:0/2326310202 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0b0ca0 0x7fe71c1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.298+0000 7fe7235ff700 1 -- 172.21.15.119:0/2326310202 >> 172.21.15.119:0/2326310202 conn(0x7fe71c06a310 msgr2=0x7fe71c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:45.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.298+0000 7fe7235ff700 1 -- 172.21.15.119:0/2326310202 shutdown_connections 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.298+0000 7fe7235ff700 1 -- 172.21.15.119:0/2326310202 wait complete. 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.299+0000 7fe7235ff700 1 Processor -- start 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.299+0000 7fe7235ff700 1 -- start start 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe7235ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0acf10 0x7fe71c1430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe7235ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0b0ca0 0x7fe71c158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe7235ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c156510 0x7fe71c158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe7235ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c169360 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe7235ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0a73c0 con 0x7fe71c156510 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe7235ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c168a30 con 0x7fe71c0acf10 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe720b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0b0ca0 0x7fe71c158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe720b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0b0ca0 0x7fe71c158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37496/0 (socket says 172.21.15.119:37496) 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.300+0000 7fe720b9a700 1 -- 172.21.15.119:0/787409244 learned_addr learned my addr 172.21.15.119:0/787409244 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe72139b700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0acf10 0x7fe71c1430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe721b9c700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c156510 0x7fe71c158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe720b9a700 1 -- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0acf10 msgr2=0x7fe71c1430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe720b9a700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0acf10 0x7fe71c1430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe720b9a700 1 -- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c156510 msgr2=0x7fe71c158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe720b9a700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c156510 0x7fe71c158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe720b9a700 1 -- 172.21.15.119:0/787409244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe714009ae0 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe72139b700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0acf10 0x7fe71c1430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe721b9c700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c156510 0x7fe71c158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe720b9a700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0b0ca0 0x7fe71c158010 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d890 tx=0x7fe70c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe7117fa700 1 -- 172.21.15.119:0/787409244 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c0125f0 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe7117fa700 1 -- 172.21.15.119:0/787409244 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c012770 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe7117fa700 1 -- 172.21.15.119:0/787409244 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c012b30 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c168d10 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.301+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c143610 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.302+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c1694d0 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.303+0000 7fe7117fa700 1 -- 172.21.15.119:0/787409244 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe70c01e460 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.304+0000 7fe7117fa700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.304+0000 7fe7117fa700 1 -- 172.21.15.119:0/787409244 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fe70c084ec0 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.304+0000 7fe72139b700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.305+0000 7fe72139b700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe700066f30 0x7fe7000693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7180017a0 tx=0x7fe718009040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:45.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.308+0000 7fe7117fa700 1 -- 172.21.15.119:0/787409244 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe70c051ea0 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.578+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe71c04c3c0 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.579+0000 7fe7117fa700 1 -- 172.21.15.119:0/787409244 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1116 (secure 0 0 0) 0x7fe70c0554c0 con 0x7fe71c0b0ca0 2023-10-08T02:28:45.579 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:45.579 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":5602,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":191,"num_osds":8,"num_up_osds":8,"osd_up_since":1696726535,"num_in_osds":8,"osd_in_since":1696726526,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2535071744,"bytes_avail":770559041536,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":87,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"d","status":"up:active","gid":9726}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2023-10-08T02:24:25.031083+0000","services":{}},"progress_events":{}} 2023-10-08T02:28:45.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe700066f30 msgr2=0x7fe7000693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe700066f30 0x7fe7000693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe7180017a0 tx=0x7fe718009040 comp rx=0 tx=0).stop 2023-10-08T02:28:45.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0b0ca0 msgr2=0x7fe71c158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0b0ca0 0x7fe71c158010 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fe70c00d890 tx=0x7fe70c008040 comp rx=0 tx=0).stop 2023-10-08T02:28:45.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 shutdown_connections 2023-10-08T02:28:45.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe700066f30 0x7fe7000693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe71c0acf10 0x7fe71c1430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe71c156510 0x7fe71c158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 --2- 172.21.15.119:0/787409244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe71c0b0ca0 0x7fe71c158010 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 >> 172.21.15.119:0/787409244 conn(0x7fe71c06a310 msgr2=0x7fe71c155860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 shutdown_connections 2023-10-08T02:28:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.581+0000 7fe7235ff700 1 -- 172.21.15.119:0/787409244 wait complete. 2023-10-08T02:28:45.599 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-08T02:28:45.599 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:28:45.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.783+0000 7f2e2ffff700 1 Processor -- start 2023-10-08T02:28:45.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.783+0000 7f2e2ffff700 1 -- start start 2023-10-08T02:28:45.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.784+0000 7f2e2ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e30150180 0x7f2e30157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.784+0000 7f2e2ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300c7410 0x7f2e30069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.784+0000 7f2e2ffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3006e2a0 con 0x7f2e30150550 2023-10-08T02:28:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.784+0000 7f2e2ffff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3006b2e0 con 0x7f2e30150180 2023-10-08T02:28:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.784+0000 7f2e2ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300c78e0 con 0x7f2e300c7410 2023-10-08T02:28:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.784+0000 7f2e2effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e30150180 0x7f2e30157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.784+0000 7f2e2effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2e30150180 0x7f2e30157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37498/0 (socket says 172.21.15.119:37498) 2023-10-08T02:28:45.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.784+0000 7f2e2effd700 1 -- 172.21.15.119:0/1808351278 learned_addr learned my addr 172.21.15.119:0/1808351278 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2effd700 1 -- 172.21.15.119:0/1808351278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300c7410 msgr2=0x7f2e30069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2f7fe700 1 --2- 172.21.15.119:0/1808351278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300c7410 0x7f2e30069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2effd700 1 --2- 172.21.15.119:0/1808351278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300c7410 0x7f2e30069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2effd700 1 -- 172.21.15.119:0/1808351278 >> v1:172.21.15.119:6789/0 conn(0x7f2e30150550 legacy=0x7f2e300c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2effd700 1 -- 172.21.15.119:0/1808351278 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300c7bc0 con 0x7f2e30150180 2023-10-08T02:28:45.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2f7fe700 1 --2- 172.21.15.119:0/1808351278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300c7410 0x7f2e30069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2effd700 1 --2- 172.21.15.119:0/1808351278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e30150180 0x7f2e30157770 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2e200099b0 tx=0x7f2e20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445f5ccc900ec558 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2e7fc700 1 -- 172.21.15.119:0/1808351278 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000b860 con 0x7f2e30150180 2023-10-08T02:28:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2e7fc700 1 -- 172.21.15.119:0/1808351278 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2000b9c0 con 0x7f2e30150180 2023-10-08T02:28:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.785+0000 7f2e2e7fc700 1 -- 172.21.15.119:0/1808351278 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000bd60 con 0x7f2e30150180 2023-10-08T02:28:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.786+0000 7f2e2ffff700 1 -- 172.21.15.119:0/1808351278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e30150180 msgr2=0x7f2e30157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.786+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/1808351278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e30150180 0x7f2e30157770 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f2e200099b0 tx=0x7f2e20009250 comp rx=0 tx=0).stop 2023-10-08T02:28:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.786+0000 7f2e2ffff700 1 -- 172.21.15.119:0/1808351278 shutdown_connections 2023-10-08T02:28:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.786+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/1808351278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e300c7410 0x7f2e30069f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.786+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/1808351278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e30150180 0x7f2e30157770 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.786+0000 7f2e2ffff700 1 -- 172.21.15.119:0/1808351278 >> 172.21.15.119:0/1808351278 conn(0x7f2e300cc1f0 msgr2=0x7f2e300cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.786+0000 7f2e2ffff700 1 -- 172.21.15.119:0/1808351278 shutdown_connections 2023-10-08T02:28:45.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.786+0000 7f2e2ffff700 1 -- 172.21.15.119:0/1808351278 wait complete. 2023-10-08T02:28:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.787+0000 7f2e2ffff700 1 Processor -- start 2023-10-08T02:28:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.787+0000 7f2e2ffff700 1 -- start start 2023-10-08T02:28:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.788+0000 7f2e2ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e300c7410 0x7f2e301472b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.788+0000 7f2e2ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30150180 0x7f2e300b1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.788+0000 7f2e2ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30150550 0x7f2e300b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.788+0000 7f2e2ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300ac280 con 0x7f2e300c7410 2023-10-08T02:28:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.788+0000 7f2e2ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300b1bc0 con 0x7f2e30150180 2023-10-08T02:28:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.788+0000 7f2e2ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e300ac4d0 con 0x7f2e30150550 2023-10-08T02:28:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e300c7410 0x7f2e301472b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e300c7410 0x7f2e301472b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37512/0 (socket says 172.21.15.119:37512) 2023-10-08T02:28:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 -- 172.21.15.119:0/3695626671 learned_addr learned my addr 172.21.15.119:0/3695626671 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2f7fe700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30150550 0x7f2e300b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 -- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30150550 msgr2=0x7f2e300b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30150550 0x7f2e300b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 -- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30150180 msgr2=0x7f2e300b1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e27fff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30150180 0x7f2e300b1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30150180 0x7f2e300b1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 -- 172.21.15.119:0/3695626671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e200094f0 con 0x7f2e300c7410 2023-10-08T02:28:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2f7fe700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30150550 0x7f2e300b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e27fff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30150180 0x7f2e300b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.789+0000 7f2e2effd700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e300c7410 0x7f2e301472b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009290 tx=0x7f2e20019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.790+0000 7f2e26ffd700 1 -- 172.21.15.119:0/3695626671 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20010680 con 0x7f2e300c7410 2023-10-08T02:28:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.790+0000 7f2e26ffd700 1 -- 172.21.15.119:0/3695626671 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e200107e0 con 0x7f2e300c7410 2023-10-08T02:28:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.790+0000 7f2e26ffd700 1 -- 172.21.15.119:0/3695626671 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20010ba0 con 0x7f2e300c7410 2023-10-08T02:28:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.790+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e300ac7b0 con 0x7f2e300c7410 2023-10-08T02:28:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.790+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e300acd70 con 0x7f2e300c7410 2023-10-08T02:28:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.791+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3004c3c0 con 0x7f2e300c7410 2023-10-08T02:28:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.792+0000 7f2e26ffd700 1 -- 172.21.15.119:0/3695626671 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e2002c440 con 0x7f2e300c7410 2023-10-08T02:28:45.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.792+0000 7f2e26ffd700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2e10066f30 0x7f2e100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:45.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.793+0000 7f2e26ffd700 1 -- 172.21.15.119:0/3695626671 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f2e20014070 con 0x7f2e300c7410 2023-10-08T02:28:45.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.793+0000 7f2e27fff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2e10066f30 0x7f2e100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:45.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.793+0000 7f2e27fff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2e10066f30 0x7f2e100693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e180017a0 tx=0x7f2e18009040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:45.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.797+0000 7f2e26ffd700 1 -- 172.21.15.119:0/3695626671 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e20053310 con 0x7f2e300c7410 2023-10-08T02:28:45.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.986+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e3006ada0 con 0x7f2e10066f30 2023-10-08T02:28:45.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.994+0000 7f2e26ffd700 1 -- 172.21.15.119:0/3695626671 <== mgr.9543 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288957 (secure 0 0 0) 0x7f2e3006ada0 con 0x7f2e10066f30 2023-10-08T02:28:45.997 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2e10066f30 msgr2=0x7f2e100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2e10066f30 0x7f2e100693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2e180017a0 tx=0x7f2e18009040 comp rx=0 tx=0).stop 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e300c7410 msgr2=0x7f2e301472b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e300c7410 0x7f2e301472b0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009290 tx=0x7f2e20019000 comp rx=0 tx=0).stop 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 shutdown_connections 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2e10066f30 0x7f2e100693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2e30150550 0x7f2e300b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2e300c7410 0x7f2e301472b0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 --2- 172.21.15.119:0/3695626671 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2e30150180 0x7f2e300b1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.998+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 >> 172.21.15.119:0/3695626671 conn(0x7f2e300cc1f0 msgr2=0x7f2e3014ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:46.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.999+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 shutdown_connections 2023-10-08T02:28:46.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:45.999+0000 7f2e2ffff700 1 -- 172.21.15.119:0/3695626671 wait complete. 2023-10-08T02:28:46.001 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:28:46.016 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-08T02:28:45.771050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2476008,"kb_used_data":19880,"kb_used_omap":0,"kb_used_meta":2456064,"kb_avail":752498712,"statfs":{"total":773094113280,"available":770558681088,"internally_reserved":0,"allocated":20357120,"data_stored":10806884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2515009536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.968040"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333047+0000","last_change":"2023-10-08T02:28:39.329482+0000","last_active":"2023-10-08T02:28:39.333047+0000","last_peered":"2023-10-08T02:28:39.333047+0000","last_clean":"2023-10-08T02:28:39.333047+0000","last_became_active":"2023-10-08T02:28:39.329198+0000","last_became_peered":"2023-10-08T02:28:39.329198+0000","last_unstale":"2023-10-08T02:28:39.333047+0000","last_undegraded":"2023-10-08T02:28:39.333047+0000","last_fullsized":"2023-10-08T02:28:39.333047+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:15.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330383+0000","last_change":"2023-10-08T02:28:39.324811+0000","last_active":"2023-10-08T02:28:39.330383+0000","last_peered":"2023-10-08T02:28:39.330383+0000","last_clean":"2023-10-08T02:28:39.330383+0000","last_became_active":"2023-10-08T02:28:39.324408+0000","last_became_peered":"2023-10-08T02:28:39.324408+0000","last_unstale":"2023-10-08T02:28:39.330383+0000","last_undegraded":"2023-10-08T02:28:39.330383+0000","last_fullsized":"2023-10-08T02:28:39.330383+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:55.807470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331675+0000","last_change":"2023-10-08T02:28:39.328136+0000","last_active":"2023-10-08T02:28:39.331675+0000","last_peered":"2023-10-08T02:28:39.331675+0000","last_clean":"2023-10-08T02:28:39.331675+0000","last_became_active":"2023-10-08T02:28:39.327879+0000","last_became_peered":"2023-10-08T02:28:39.327879+0000","last_unstale":"2023-10-08T02:28:39.331675+0000","last_undegraded":"2023-10-08T02:28:39.331675+0000","last_fullsized":"2023-10-08T02:28:39.331675+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:58.451514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333456+0000","last_change":"2023-10-08T02:28:39.329838+0000","last_active":"2023-10-08T02:28:39.333456+0000","last_peered":"2023-10-08T02:28:39.333456+0000","last_clean":"2023-10-08T02:28:39.333456+0000","last_became_active":"2023-10-08T02:28:39.329181+0000","last_became_peered":"2023-10-08T02:28:39.329181+0000","last_unstale":"2023-10-08T02:28:39.333456+0000","last_undegraded":"2023-10-08T02:28:39.333456+0000","last_fullsized":"2023-10-08T02:28:39.333456+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:47.883744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.319796+0000","last_change":"2023-10-08T02:28:39.318284+0000","last_active":"2023-10-08T02:28:39.319796+0000","last_peered":"2023-10-08T02:28:39.319796+0000","last_clean":"2023-10-08T02:28:39.319796+0000","last_became_active":"2023-10-08T02:28:39.317981+0000","last_became_peered":"2023-10-08T02:28:39.317981+0000","last_unstale":"2023-10-08T02:28:39.319796+0000","last_undegraded":"2023-10-08T02:28:39.319796+0000","last_fullsized":"2023-10-08T02:28:39.319796+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:32.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327172+0000","last_change":"2023-10-08T02:28:39.322993+0000","last_active":"2023-10-08T02:28:39.327172+0000","last_peered":"2023-10-08T02:28:39.327172+0000","last_clean":"2023-10-08T02:28:39.327172+0000","last_became_active":"2023-10-08T02:28:39.322746+0000","last_became_peered":"2023-10-08T02:28:39.322746+0000","last_unstale":"2023-10-08T02:28:39.327172+0000","last_undegraded":"2023-10-08T02:28:39.327172+0000","last_fullsized":"2023-10-08T02:28:39.327172+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:31.075827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331401+0000","last_change":"2023-10-08T02:28:39.326667+0000","last_active":"2023-10-08T02:28:39.331401+0000","last_peered":"2023-10-08T02:28:39.331401+0000","last_clean":"2023-10-08T02:28:39.331401+0000","last_became_active":"2023-10-08T02:28:39.326352+0000","last_became_peered":"2023-10-08T02:28:39.326352+0000","last_unstale":"2023-10-08T02:28:39.331401+0000","last_undegraded":"2023-10-08T02:28:39.331401+0000","last_fullsized":"2023-10-08T02:28:39.331401+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:27.367566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330942+0000","last_change":"2023-10-08T02:28:39.327344+0000","last_active":"2023-10-08T02:28:39.330942+0000","last_peered":"2023-10-08T02:28:39.330942+0000","last_clean":"2023-10-08T02:28:39.330942+0000","last_became_active":"2023-10-08T02:28:39.327079+0000","last_became_peered":"2023-10-08T02:28:39.327079+0000","last_unstale":"2023-10-08T02:28:39.330942+0000","last_undegraded":"2023-10-08T02:28:39.330942+0000","last_fullsized":"2023-10-08T02:28:39.330942+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:42.424670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.332273+0000","last_change":"2023-10-08T02:28:39.330254+0000","last_active":"2023-10-08T02:28:39.332273+0000","last_peered":"2023-10-08T02:28:39.332273+0000","last_clean":"2023-10-08T02:28:39.332273+0000","last_became_active":"2023-10-08T02:28:39.329980+0000","last_became_peered":"2023-10-08T02:28:39.329980+0000","last_unstale":"2023-10-08T02:28:39.332273+0000","last_undegraded":"2023-10-08T02:28:39.332273+0000","last_fullsized":"2023-10-08T02:28:39.332273+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:05.392617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330859+0000","last_change":"2023-10-08T02:28:39.327115+0000","last_active":"2023-10-08T02:28:39.330859+0000","last_peered":"2023-10-08T02:28:39.330859+0000","last_clean":"2023-10-08T02:28:39.330859+0000","last_became_active":"2023-10-08T02:28:39.326835+0000","last_became_peered":"2023-10-08T02:28:39.326835+0000","last_unstale":"2023-10-08T02:28:39.330859+0000","last_undegraded":"2023-10-08T02:28:39.330859+0000","last_fullsized":"2023-10-08T02:28:39.330859+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:04.133657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330538+0000","last_change":"2023-10-08T02:28:39.327088+0000","last_active":"2023-10-08T02:28:39.330538+0000","last_peered":"2023-10-08T02:28:39.330538+0000","last_clean":"2023-10-08T02:28:39.330538+0000","last_became_active":"2023-10-08T02:28:39.326786+0000","last_became_peered":"2023-10-08T02:28:39.326786+0000","last_unstale":"2023-10-08T02:28:39.330538+0000","last_undegraded":"2023-10-08T02:28:39.330538+0000","last_fullsized":"2023-10-08T02:28:39.330538+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:32.211850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330511+0000","last_change":"2023-10-08T02:28:39.325570+0000","last_active":"2023-10-08T02:28:39.330511+0000","last_peered":"2023-10-08T02:28:39.330511+0000","last_clean":"2023-10-08T02:28:39.330511+0000","last_became_active":"2023-10-08T02:28:39.325311+0000","last_became_peered":"2023-10-08T02:28:39.325311+0000","last_unstale":"2023-10-08T02:28:39.330511+0000","last_undegraded":"2023-10-08T02:28:39.330511+0000","last_fullsized":"2023-10-08T02:28:39.330511+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:00.479132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330248+0000","last_change":"2023-10-08T02:28:39.324775+0000","last_active":"2023-10-08T02:28:39.330248+0000","last_peered":"2023-10-08T02:28:39.330248+0000","last_clean":"2023-10-08T02:28:39.330248+0000","last_became_active":"2023-10-08T02:28:39.324509+0000","last_became_peered":"2023-10-08T02:28:39.324509+0000","last_unstale":"2023-10-08T02:28:39.330248+0000","last_undegraded":"2023-10-08T02:28:39.330248+0000","last_fullsized":"2023-10-08T02:28:39.330248+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:03.109079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334896+0000","last_change":"2023-10-08T02:28:39.333508+0000","last_active":"2023-10-08T02:28:39.334896+0000","last_peered":"2023-10-08T02:28:39.334896+0000","last_clean":"2023-10-08T02:28:39.334896+0000","last_became_active":"2023-10-08T02:28:39.332581+0000","last_became_peered":"2023-10-08T02:28:39.332581+0000","last_unstale":"2023-10-08T02:28:39.334896+0000","last_undegraded":"2023-10-08T02:28:39.334896+0000","last_fullsized":"2023-10-08T02:28:39.334896+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:16.498144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329187+0000","last_change":"2023-10-08T02:28:39.324628+0000","last_active":"2023-10-08T02:28:39.329187+0000","last_peered":"2023-10-08T02:28:39.329187+0000","last_clean":"2023-10-08T02:28:39.329187+0000","last_became_active":"2023-10-08T02:28:39.324384+0000","last_became_peered":"2023-10-08T02:28:39.324384+0000","last_unstale":"2023-10-08T02:28:39.329187+0000","last_undegraded":"2023-10-08T02:28:39.329187+0000","last_fullsized":"2023-10-08T02:28:39.329187+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:42.051410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.335810+0000","last_change":"2023-10-08T02:28:39.331258+0000","last_active":"2023-10-08T02:28:39.335810+0000","last_peered":"2023-10-08T02:28:39.335810+0000","last_clean":"2023-10-08T02:28:39.335810+0000","last_became_active":"2023-10-08T02:28:39.331012+0000","last_became_peered":"2023-10-08T02:28:39.331012+0000","last_unstale":"2023-10-08T02:28:39.335810+0000","last_undegraded":"2023-10-08T02:28:39.335810+0000","last_fullsized":"2023-10-08T02:28:39.335810+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:11.626570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329498+0000","last_change":"2023-10-08T02:28:39.323511+0000","last_active":"2023-10-08T02:28:39.329498+0000","last_peered":"2023-10-08T02:28:39.329498+0000","last_clean":"2023-10-08T02:28:39.329498+0000","last_became_active":"2023-10-08T02:28:39.323257+0000","last_became_peered":"2023-10-08T02:28:39.323257+0000","last_unstale":"2023-10-08T02:28:39.329498+0000","last_undegraded":"2023-10-08T02:28:39.329498+0000","last_fullsized":"2023-10-08T02:28:39.329498+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:44.356227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.324791+0000","last_change":"2023-10-08T02:28:39.320581+0000","last_active":"2023-10-08T02:28:39.324791+0000","last_peered":"2023-10-08T02:28:39.324791+0000","last_clean":"2023-10-08T02:28:39.324791+0000","last_became_active":"2023-10-08T02:28:39.320313+0000","last_became_peered":"2023-10-08T02:28:39.320313+0000","last_unstale":"2023-10-08T02:28:39.324791+0000","last_undegraded":"2023-10-08T02:28:39.324791+0000","last_fullsized":"2023-10-08T02:28:39.324791+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:09.648666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329874+0000","last_change":"2023-10-08T02:28:39.327005+0000","last_active":"2023-10-08T02:28:39.329874+0000","last_peered":"2023-10-08T02:28:39.329874+0000","last_clean":"2023-10-08T02:28:39.329874+0000","last_became_active":"2023-10-08T02:28:39.326746+0000","last_became_peered":"2023-10-08T02:28:39.326746+0000","last_unstale":"2023-10-08T02:28:39.329874+0000","last_undegraded":"2023-10-08T02:28:39.329874+0000","last_fullsized":"2023-10-08T02:28:39.329874+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:48.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334891+0000","last_change":"2023-10-08T02:28:39.333330+0000","last_active":"2023-10-08T02:28:39.334891+0000","last_peered":"2023-10-08T02:28:39.334891+0000","last_clean":"2023-10-08T02:28:39.334891+0000","last_became_active":"2023-10-08T02:28:39.332988+0000","last_became_peered":"2023-10-08T02:28:39.332988+0000","last_unstale":"2023-10-08T02:28:39.334891+0000","last_undegraded":"2023-10-08T02:28:39.334891+0000","last_fullsized":"2023-10-08T02:28:39.334891+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:01.177379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329470+0000","last_change":"2023-10-08T02:28:39.322657+0000","last_active":"2023-10-08T02:28:39.329470+0000","last_peered":"2023-10-08T02:28:39.329470+0000","last_clean":"2023-10-08T02:28:39.329470+0000","last_became_active":"2023-10-08T02:28:39.322413+0000","last_became_peered":"2023-10-08T02:28:39.322413+0000","last_unstale":"2023-10-08T02:28:39.329470+0000","last_undegraded":"2023-10-08T02:28:39.329470+0000","last_fullsized":"2023-10-08T02:28:39.329470+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:47.883744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327619+0000","last_change":"2023-10-08T02:28:39.322682+0000","last_active":"2023-10-08T02:28:39.327619+0000","last_peered":"2023-10-08T02:28:39.327619+0000","last_clean":"2023-10-08T02:28:39.327619+0000","last_became_active":"2023-10-08T02:28:39.322383+0000","last_became_peered":"2023-10-08T02:28:39.322383+0000","last_unstale":"2023-10-08T02:28:39.327619+0000","last_undegraded":"2023-10-08T02:28:39.327619+0000","last_fullsized":"2023-10-08T02:28:39.327619+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:32.043813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.320377+0000","last_change":"2023-10-08T02:28:39.318910+0000","last_active":"2023-10-08T02:28:39.320377+0000","last_peered":"2023-10-08T02:28:39.320377+0000","last_clean":"2023-10-08T02:28:39.320377+0000","last_became_active":"2023-10-08T02:28:39.318676+0000","last_became_peered":"2023-10-08T02:28:39.318676+0000","last_unstale":"2023-10-08T02:28:39.320377+0000","last_undegraded":"2023-10-08T02:28:39.320377+0000","last_fullsized":"2023-10-08T02:28:39.320377+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:47.577446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330924+0000","last_change":"2023-10-08T02:28:39.325449+0000","last_active":"2023-10-08T02:28:39.330924+0000","last_peered":"2023-10-08T02:28:39.330924+0000","last_clean":"2023-10-08T02:28:39.330924+0000","last_became_active":"2023-10-08T02:28:39.325179+0000","last_became_peered":"2023-10-08T02:28:39.325179+0000","last_unstale":"2023-10-08T02:28:39.330924+0000","last_undegraded":"2023-10-08T02:28:39.330924+0000","last_fullsized":"2023-10-08T02:28:39.330924+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:08.018509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334159+0000","last_change":"2023-10-08T02:28:39.329687+0000","last_active":"2023-10-08T02:28:39.334159+0000","last_peered":"2023-10-08T02:28:39.334159+0000","last_clean":"2023-10-08T02:28:39.334159+0000","last_became_active":"2023-10-08T02:28:39.329403+0000","last_became_peered":"2023-10-08T02:28:39.329403+0000","last_unstale":"2023-10-08T02:28:39.334159+0000","last_undegraded":"2023-10-08T02:28:39.334159+0000","last_fullsized":"2023-10-08T02:28:39.334159+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:02:38.978566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331550+0000","last_change":"2023-10-08T02:28:39.327169+0000","last_active":"2023-10-08T02:28:39.331550+0000","last_peered":"2023-10-08T02:28:39.331550+0000","last_clean":"2023-10-08T02:28:39.331550+0000","last_became_active":"2023-10-08T02:28:39.326937+0000","last_became_peered":"2023-10-08T02:28:39.326937+0000","last_unstale":"2023-10-08T02:28:39.331550+0000","last_undegraded":"2023-10-08T02:28:39.331550+0000","last_fullsized":"2023-10-08T02:28:39.331550+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:42.843940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330219+0000","last_change":"2023-10-08T02:28:39.326643+0000","last_active":"2023-10-08T02:28:39.330219+0000","last_peered":"2023-10-08T02:28:39.330219+0000","last_clean":"2023-10-08T02:28:39.330219+0000","last_became_active":"2023-10-08T02:28:39.326309+0000","last_became_peered":"2023-10-08T02:28:39.326309+0000","last_unstale":"2023-10-08T02:28:39.330219+0000","last_undegraded":"2023-10-08T02:28:39.330219+0000","last_fullsized":"2023-10-08T02:28:39.330219+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:12.961726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.336299+0000","last_change":"2023-10-08T02:28:39.335145+0000","last_active":"2023-10-08T02:28:39.336299+0000","last_peered":"2023-10-08T02:28:39.336299+0000","last_clean":"2023-10-08T02:28:39.336299+0000","last_became_active":"2023-10-08T02:28:39.334957+0000","last_became_peered":"2023-10-08T02:28:39.334957+0000","last_unstale":"2023-10-08T02:28:39.336299+0000","last_undegraded":"2023-10-08T02:28:39.336299+0000","last_fullsized":"2023-10-08T02:28:39.336299+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:58.451514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330266+0000","last_change":"2023-10-08T02:28:39.326324+0000","last_active":"2023-10-08T02:28:39.330266+0000","last_peered":"2023-10-08T02:28:39.330266+0000","last_clean":"2023-10-08T02:28:39.330266+0000","last_became_active":"2023-10-08T02:28:39.326061+0000","last_became_peered":"2023-10-08T02:28:39.326061+0000","last_unstale":"2023-10-08T02:28:39.330266+0000","last_undegraded":"2023-10-08T02:28:39.330266+0000","last_fullsized":"2023-10-08T02:28:39.330266+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:19.633375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334559+0000","last_change":"2023-10-08T02:28:39.331317+0000","last_active":"2023-10-08T02:28:39.334559+0000","last_peered":"2023-10-08T02:28:39.334559+0000","last_clean":"2023-10-08T02:28:39.334559+0000","last_became_active":"2023-10-08T02:28:39.331010+0000","last_became_peered":"2023-10-08T02:28:39.331010+0000","last_unstale":"2023-10-08T02:28:39.334559+0000","last_undegraded":"2023-10-08T02:28:39.334559+0000","last_fullsized":"2023-10-08T02:28:39.334559+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:27.279209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.325920+0000","last_change":"2023-10-08T02:28:39.321428+0000","last_active":"2023-10-08T02:28:39.325920+0000","last_peered":"2023-10-08T02:28:39.325920+0000","last_clean":"2023-10-08T02:28:39.325920+0000","last_became_active":"2023-10-08T02:28:39.321169+0000","last_became_peered":"2023-10-08T02:28:39.321169+0000","last_unstale":"2023-10-08T02:28:39.325920+0000","last_undegraded":"2023-10-08T02:28:39.325920+0000","last_fullsized":"2023-10-08T02:28:39.325920+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:15.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.323681+0000","last_change":"2023-10-08T02:28:39.319024+0000","last_active":"2023-10-08T02:28:39.323681+0000","last_peered":"2023-10-08T02:28:39.323681+0000","last_clean":"2023-10-08T02:28:39.323681+0000","last_became_active":"2023-10-08T02:28:39.318785+0000","last_became_peered":"2023-10-08T02:28:39.318785+0000","last_unstale":"2023-10-08T02:28:39.323681+0000","last_undegraded":"2023-10-08T02:28:39.323681+0000","last_fullsized":"2023-10-08T02:28:39.323681+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:19.633375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331532+0000","last_change":"2023-10-08T02:28:39.326571+0000","last_active":"2023-10-08T02:28:39.331532+0000","last_peered":"2023-10-08T02:28:39.331532+0000","last_clean":"2023-10-08T02:28:39.331532+0000","last_became_active":"2023-10-08T02:28:39.326293+0000","last_became_peered":"2023-10-08T02:28:39.326293+0000","last_unstale":"2023-10-08T02:28:39.331532+0000","last_undegraded":"2023-10-08T02:28:39.331532+0000","last_fullsized":"2023-10-08T02:28:39.331532+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:03.725569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"190'4","reported_seq":14,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.819789+0000","last_change":"2023-10-08T02:28:38.828668+0000","last_active":"2023-10-08T02:28:39.819789+0000","last_peered":"2023-10-08T02:28:39.819789+0000","last_clean":"2023-10-08T02:28:39.819789+0000","last_became_active":"2023-10-08T02:28:38.828377+0000","last_became_peered":"2023-10-08T02:28:38.828377+0000","last_unstale":"2023-10-08T02:28:39.819789+0000","last_undegraded":"2023-10-08T02:28:39.819789+0000","last_fullsized":"2023-10-08T02:28:39.819789+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:57.443582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.320596+0000","last_change":"2023-10-08T02:28:39.314030+0000","last_active":"2023-10-08T02:28:39.320596+0000","last_peered":"2023-10-08T02:28:39.320596+0000","last_clean":"2023-10-08T02:28:39.320596+0000","last_became_active":"2023-10-08T02:28:39.313761+0000","last_became_peered":"2023-10-08T02:28:39.313761+0000","last_unstale":"2023-10-08T02:28:39.320596+0000","last_undegraded":"2023-10-08T02:28:39.320596+0000","last_fullsized":"2023-10-08T02:28:39.320596+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:33.987576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.819246+0000","last_change":"2023-10-08T02:28:38.828417+0000","last_active":"2023-10-08T02:28:39.819246+0000","last_peered":"2023-10-08T02:28:39.819246+0000","last_clean":"2023-10-08T02:28:39.819246+0000","last_became_active":"2023-10-08T02:28:38.828122+0000","last_became_peered":"2023-10-08T02:28:38.828122+0000","last_unstale":"2023-10-08T02:28:39.819246+0000","last_undegraded":"2023-10-08T02:28:39.819246+0000","last_fullsized":"2023-10-08T02:28:39.819246+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:43.348294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333686+0000","last_change":"2023-10-08T02:28:39.332737+0000","last_active":"2023-10-08T02:28:39.333686+0000","last_peered":"2023-10-08T02:28:39.333686+0000","last_clean":"2023-10-08T02:28:39.333686+0000","last_became_active":"2023-10-08T02:28:39.332497+0000","last_became_peered":"2023-10-08T02:28:39.332497+0000","last_unstale":"2023-10-08T02:28:39.333686+0000","last_undegraded":"2023-10-08T02:28:39.333686+0000","last_fullsized":"2023-10-08T02:28:39.333686+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:40.199040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.818094+0000","last_change":"2023-10-08T02:28:38.827826+0000","last_active":"2023-10-08T02:28:39.818094+0000","last_peered":"2023-10-08T02:28:39.818094+0000","last_clean":"2023-10-08T02:28:39.818094+0000","last_became_active":"2023-10-08T02:28:38.827526+0000","last_became_peered":"2023-10-08T02:28:38.827526+0000","last_unstale":"2023-10-08T02:28:39.818094+0000","last_undegraded":"2023-10-08T02:28:39.818094+0000","last_fullsized":"2023-10-08T02:28:39.818094+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:31.203917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334149+0000","last_change":"2023-10-08T02:28:39.331172+0000","last_active":"2023-10-08T02:28:39.334149+0000","last_peered":"2023-10-08T02:28:39.334149+0000","last_clean":"2023-10-08T02:28:39.334149+0000","last_became_active":"2023-10-08T02:28:39.330901+0000","last_became_peered":"2023-10-08T02:28:39.330901+0000","last_unstale":"2023-10-08T02:28:39.334149+0000","last_undegraded":"2023-10-08T02:28:39.334149+0000","last_fullsized":"2023-10-08T02:28:39.334149+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:44.356227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.818699+0000","last_change":"2023-10-08T02:28:38.819324+0000","last_active":"2023-10-08T02:28:39.818699+0000","last_peered":"2023-10-08T02:28:39.818699+0000","last_clean":"2023-10-08T02:28:39.818699+0000","last_became_active":"2023-10-08T02:28:38.818979+0000","last_became_peered":"2023-10-08T02:28:38.818979+0000","last_unstale":"2023-10-08T02:28:39.818699+0000","last_undegraded":"2023-10-08T02:28:39.818699+0000","last_fullsized":"2023-10-08T02:28:39.818699+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:04.384684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331463+0000","last_change":"2023-10-08T02:28:39.327590+0000","last_active":"2023-10-08T02:28:39.331463+0000","last_peered":"2023-10-08T02:28:39.331463+0000","last_clean":"2023-10-08T02:28:39.331463+0000","last_became_active":"2023-10-08T02:28:39.327322+0000","last_became_peered":"2023-10-08T02:28:39.327322+0000","last_unstale":"2023-10-08T02:28:39.331463+0000","last_undegraded":"2023-10-08T02:28:39.331463+0000","last_fullsized":"2023-10-08T02:28:39.331463+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:26:41.547145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.817393+0000","last_change":"2023-10-08T02:28:38.826581+0000","last_active":"2023-10-08T02:28:39.817393+0000","last_peered":"2023-10-08T02:28:39.817393+0000","last_clean":"2023-10-08T02:28:39.817393+0000","last_became_active":"2023-10-08T02:28:38.826110+0000","last_became_peered":"2023-10-08T02:28:38.826110+0000","last_unstale":"2023-10-08T02:28:39.817393+0000","last_undegraded":"2023-10-08T02:28:39.817393+0000","last_fullsized":"2023-10-08T02:28:39.817393+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:03.125725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.336007+0000","last_change":"2023-10-08T02:28:39.331411+0000","last_active":"2023-10-08T02:28:39.336007+0000","last_peered":"2023-10-08T02:28:39.336007+0000","last_clean":"2023-10-08T02:28:39.336007+0000","last_became_active":"2023-10-08T02:28:39.331098+0000","last_became_peered":"2023-10-08T02:28:39.331098+0000","last_unstale":"2023-10-08T02:28:39.336007+0000","last_undegraded":"2023-10-08T02:28:39.336007+0000","last_fullsized":"2023-10-08T02:28:39.336007+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:19.633375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.814417+0000","last_change":"2023-10-08T02:28:38.822591+0000","last_active":"2023-10-08T02:28:39.814417+0000","last_peered":"2023-10-08T02:28:39.814417+0000","last_clean":"2023-10-08T02:28:39.814417+0000","last_became_active":"2023-10-08T02:28:38.822333+0000","last_became_peered":"2023-10-08T02:28:38.822333+0000","last_unstale":"2023-10-08T02:28:39.814417+0000","last_undegraded":"2023-10-08T02:28:39.814417+0000","last_fullsized":"2023-10-08T02:28:39.814417+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:41.416738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333350+0000","last_change":"2023-10-08T02:28:39.329032+0000","last_active":"2023-10-08T02:28:39.333350+0000","last_peered":"2023-10-08T02:28:39.333350+0000","last_clean":"2023-10-08T02:28:39.333350+0000","last_became_active":"2023-10-08T02:28:39.328798+0000","last_became_peered":"2023-10-08T02:28:39.328798+0000","last_unstale":"2023-10-08T02:28:39.333350+0000","last_undegraded":"2023-10-08T02:28:39.333350+0000","last_fullsized":"2023-10-08T02:28:39.333350+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:48.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.311488+0000","last_change":"2023-10-08T02:28:38.820931+0000","last_active":"2023-10-08T02:28:39.311488+0000","last_peered":"2023-10-08T02:28:39.311488+0000","last_clean":"2023-10-08T02:28:39.311488+0000","last_became_active":"2023-10-08T02:28:38.820618+0000","last_became_peered":"2023-10-08T02:28:38.820618+0000","last_unstale":"2023-10-08T02:28:39.311488+0000","last_undegraded":"2023-10-08T02:28:39.311488+0000","last_fullsized":"2023-10-08T02:28:39.311488+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:34.106471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327195+0000","last_change":"2023-10-08T02:28:39.322164+0000","last_active":"2023-10-08T02:28:39.327195+0000","last_peered":"2023-10-08T02:28:39.327195+0000","last_clean":"2023-10-08T02:28:39.327195+0000","last_became_active":"2023-10-08T02:28:39.321885+0000","last_became_peered":"2023-10-08T02:28:39.321885+0000","last_unstale":"2023-10-08T02:28:39.327195+0000","last_undegraded":"2023-10-08T02:28:39.327195+0000","last_fullsized":"2023-10-08T02:28:39.327195+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:45.277252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.316511+0000","last_change":"2023-10-08T02:28:38.813302+0000","last_active":"2023-10-08T02:28:39.316511+0000","last_peered":"2023-10-08T02:28:39.316511+0000","last_clean":"2023-10-08T02:28:39.316511+0000","last_became_active":"2023-10-08T02:28:38.812605+0000","last_became_peered":"2023-10-08T02:28:38.812605+0000","last_unstale":"2023-10-08T02:28:39.316511+0000","last_undegraded":"2023-10-08T02:28:39.316511+0000","last_fullsized":"2023-10-08T02:28:39.316511+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:43.714637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327852+0000","last_change":"2023-10-08T02:28:39.323313+0000","last_active":"2023-10-08T02:28:39.327852+0000","last_peered":"2023-10-08T02:28:39.327852+0000","last_clean":"2023-10-08T02:28:39.327852+0000","last_became_active":"2023-10-08T02:28:39.323024+0000","last_became_peered":"2023-10-08T02:28:39.323024+0000","last_unstale":"2023-10-08T02:28:39.327852+0000","last_undegraded":"2023-10-08T02:28:39.327852+0000","last_fullsized":"2023-10-08T02:28:39.327852+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:16.498144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.318436+0000","last_change":"2023-10-08T02:28:38.832015+0000","last_active":"2023-10-08T02:28:39.318436+0000","last_peered":"2023-10-08T02:28:39.318436+0000","last_clean":"2023-10-08T02:28:39.318436+0000","last_became_active":"2023-10-08T02:28:38.831734+0000","last_became_peered":"2023-10-08T02:28:38.831734+0000","last_unstale":"2023-10-08T02:28:39.318436+0000","last_undegraded":"2023-10-08T02:28:39.318436+0000","last_fullsized":"2023-10-08T02:28:39.318436+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.326751+0000","last_change":"2023-10-08T02:28:39.322958+0000","last_active":"2023-10-08T02:28:39.326751+0000","last_peered":"2023-10-08T02:28:39.326751+0000","last_clean":"2023-10-08T02:28:39.326751+0000","last_became_active":"2023-10-08T02:28:39.322646+0000","last_became_peered":"2023-10-08T02:28:39.322646+0000","last_unstale":"2023-10-08T02:28:39.326751+0000","last_undegraded":"2023-10-08T02:28:39.326751+0000","last_fullsized":"2023-10-08T02:28:39.326751+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:02:38.978566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"190'3","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.820154+0000","last_change":"2023-10-08T02:28:38.815648+0000","last_active":"2023-10-08T02:28:39.820154+0000","last_peered":"2023-10-08T02:28:39.820154+0000","last_clean":"2023-10-08T02:28:39.820154+0000","last_became_active":"2023-10-08T02:28:38.815397+0000","last_became_peered":"2023-10-08T02:28:38.815397+0000","last_unstale":"2023-10-08T02:28:39.820154+0000","last_undegraded":"2023-10-08T02:28:39.820154+0000","last_fullsized":"2023-10-08T02:28:39.820154+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.328467+0000","last_change":"2023-10-08T02:28:39.325322+0000","last_active":"2023-10-08T02:28:39.328467+0000","last_peered":"2023-10-08T02:28:39.328467+0000","last_clean":"2023-10-08T02:28:39.328467+0000","last_became_active":"2023-10-08T02:28:39.325020+0000","last_became_peered":"2023-10-08T02:28:39.325020+0000","last_unstale":"2023-10-08T02:28:39.328467+0000","last_undegraded":"2023-10-08T02:28:39.328467+0000","last_fullsized":"2023-10-08T02:28:39.328467+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:23.649613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"190'6","reported_seq":16,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.824162+0000","last_change":"2023-10-08T02:28:38.828063+0000","last_active":"2023-10-08T02:28:39.824162+0000","last_peered":"2023-10-08T02:28:39.824162+0000","last_clean":"2023-10-08T02:28:39.824162+0000","last_became_active":"2023-10-08T02:28:38.827800+0000","last_became_peered":"2023-10-08T02:28:38.827800+0000","last_unstale":"2023-10-08T02:28:39.824162+0000","last_undegraded":"2023-10-08T02:28:39.824162+0000","last_fullsized":"2023-10-08T02:28:39.824162+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:11.953794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.323763+0000","last_change":"2023-10-08T02:28:39.318203+0000","last_active":"2023-10-08T02:28:39.323763+0000","last_peered":"2023-10-08T02:28:39.323763+0000","last_clean":"2023-10-08T02:28:39.323763+0000","last_became_active":"2023-10-08T02:28:39.317553+0000","last_became_peered":"2023-10-08T02:28:39.317553+0000","last_unstale":"2023-10-08T02:28:39.323763+0000","last_undegraded":"2023-10-08T02:28:39.323763+0000","last_fullsized":"2023-10-08T02:28:39.323763+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:02:54.426138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.816833+0000","last_change":"2023-10-08T02:28:38.819154+0000","last_active":"2023-10-08T02:28:39.816833+0000","last_peered":"2023-10-08T02:28:39.816833+0000","last_clean":"2023-10-08T02:28:39.816833+0000","last_became_active":"2023-10-08T02:28:38.818890+0000","last_became_peered":"2023-10-08T02:28:38.818890+0000","last_unstale":"2023-10-08T02:28:39.816833+0000","last_undegraded":"2023-10-08T02:28:39.816833+0000","last_fullsized":"2023-10-08T02:28:39.816833+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:25.660989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330982+0000","last_change":"2023-10-08T02:28:39.325439+0000","last_active":"2023-10-08T02:28:39.330982+0000","last_peered":"2023-10-08T02:28:39.330982+0000","last_clean":"2023-10-08T02:28:39.330982+0000","last_became_active":"2023-10-08T02:28:39.324973+0000","last_became_peered":"2023-10-08T02:28:39.324973+0000","last_unstale":"2023-10-08T02:28:39.330982+0000","last_undegraded":"2023-10-08T02:28:39.330982+0000","last_fullsized":"2023-10-08T02:28:39.330982+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:37:30.449689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.310513+0000","last_change":"2023-10-08T02:28:38.823806+0000","last_active":"2023-10-08T02:28:39.310513+0000","last_peered":"2023-10-08T02:28:39.310513+0000","last_clean":"2023-10-08T02:28:39.310513+0000","last_became_active":"2023-10-08T02:28:38.823556+0000","last_became_peered":"2023-10-08T02:28:38.823556+0000","last_unstale":"2023-10-08T02:28:39.310513+0000","last_undegraded":"2023-10-08T02:28:39.310513+0000","last_fullsized":"2023-10-08T02:28:39.310513+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:44.269320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333423+0000","last_change":"2023-10-08T02:28:39.329983+0000","last_active":"2023-10-08T02:28:39.333423+0000","last_peered":"2023-10-08T02:28:39.333423+0000","last_clean":"2023-10-08T02:28:39.333423+0000","last_became_active":"2023-10-08T02:28:39.329722+0000","last_became_peered":"2023-10-08T02:28:39.329722+0000","last_unstale":"2023-10-08T02:28:39.333423+0000","last_undegraded":"2023-10-08T02:28:39.333423+0000","last_fullsized":"2023-10-08T02:28:39.333423+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:12.961726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.315187+0000","last_change":"2023-10-08T02:28:38.824251+0000","last_active":"2023-10-08T02:28:39.315187+0000","last_peered":"2023-10-08T02:28:39.315187+0000","last_clean":"2023-10-08T02:28:39.315187+0000","last_became_active":"2023-10-08T02:28:38.823920+0000","last_became_peered":"2023-10-08T02:28:38.823920+0000","last_unstale":"2023-10-08T02:28:39.315187+0000","last_undegraded":"2023-10-08T02:28:39.315187+0000","last_fullsized":"2023-10-08T02:28:39.315187+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:31.497164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329128+0000","last_change":"2023-10-08T02:28:39.323986+0000","last_active":"2023-10-08T02:28:39.329128+0000","last_peered":"2023-10-08T02:28:39.329128+0000","last_clean":"2023-10-08T02:28:39.329128+0000","last_became_active":"2023-10-08T02:28:39.323715+0000","last_became_peered":"2023-10-08T02:28:39.323715+0000","last_unstale":"2023-10-08T02:28:39.329128+0000","last_undegraded":"2023-10-08T02:28:39.329128+0000","last_fullsized":"2023-10-08T02:28:39.329128+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:09.648666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"190'8","reported_seq":18,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.822052+0000","last_change":"2023-10-08T02:28:38.828752+0000","last_active":"2023-10-08T02:28:39.822052+0000","last_peered":"2023-10-08T02:28:39.822052+0000","last_clean":"2023-10-08T02:28:39.822052+0000","last_became_active":"2023-10-08T02:28:38.828482+0000","last_became_peered":"2023-10-08T02:28:38.828482+0000","last_unstale":"2023-10-08T02:28:39.822052+0000","last_undegraded":"2023-10-08T02:28:39.822052+0000","last_fullsized":"2023-10-08T02:28:39.822052+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:08.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.328162+0000","last_change":"2023-10-08T02:28:39.324786+0000","last_active":"2023-10-08T02:28:39.328162+0000","last_peered":"2023-10-08T02:28:39.328162+0000","last_clean":"2023-10-08T02:28:39.328162+0000","last_became_active":"2023-10-08T02:28:39.324533+0000","last_became_peered":"2023-10-08T02:28:39.324533+0000","last_unstale":"2023-10-08T02:28:39.328162+0000","last_undegraded":"2023-10-08T02:28:39.328162+0000","last_fullsized":"2023-10-08T02:28:39.328162+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:44.356227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.818330+0000","last_change":"2023-10-08T02:28:38.829032+0000","last_active":"2023-10-08T02:28:39.818330+0000","last_peered":"2023-10-08T02:28:39.818330+0000","last_clean":"2023-10-08T02:28:39.818330+0000","last_became_active":"2023-10-08T02:28:38.828787+0000","last_became_peered":"2023-10-08T02:28:38.828787+0000","last_unstale":"2023-10-08T02:28:39.818330+0000","last_undegraded":"2023-10-08T02:28:39.818330+0000","last_fullsized":"2023-10-08T02:28:39.818330+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:04.384684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.320862+0000","last_change":"2023-10-08T02:28:39.317037+0000","last_active":"2023-10-08T02:28:39.320862+0000","last_peered":"2023-10-08T02:28:39.320862+0000","last_clean":"2023-10-08T02:28:39.320862+0000","last_became_active":"2023-10-08T02:28:39.316755+0000","last_became_peered":"2023-10-08T02:28:39.316755+0000","last_unstale":"2023-10-08T02:28:39.320862+0000","last_undegraded":"2023-10-08T02:28:39.320862+0000","last_fullsized":"2023-10-08T02:28:39.320862+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:15.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.817487+0000","last_change":"2023-10-08T02:28:38.830576+0000","last_active":"2023-10-08T02:28:39.817487+0000","last_peered":"2023-10-08T02:28:39.817487+0000","last_clean":"2023-10-08T02:28:39.817487+0000","last_became_active":"2023-10-08T02:28:38.829916+0000","last_became_peered":"2023-10-08T02:28:38.829916+0000","last_unstale":"2023-10-08T02:28:39.817487+0000","last_undegraded":"2023-10-08T02:28:39.817487+0000","last_fullsized":"2023-10-08T02:28:39.817487+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:07.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.336015+0000","last_change":"2023-10-08T02:28:39.331448+0000","last_active":"2023-10-08T02:28:39.336015+0000","last_peered":"2023-10-08T02:28:39.336015+0000","last_clean":"2023-10-08T02:28:39.336015+0000","last_became_active":"2023-10-08T02:28:39.331154+0000","last_became_peered":"2023-10-08T02:28:39.331154+0000","last_unstale":"2023-10-08T02:28:39.336015+0000","last_undegraded":"2023-10-08T02:28:39.336015+0000","last_fullsized":"2023-10-08T02:28:39.336015+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:30.962913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.820105+0000","last_change":"2023-10-08T02:28:38.823230+0000","last_active":"2023-10-08T02:28:39.820105+0000","last_peered":"2023-10-08T02:28:39.820105+0000","last_clean":"2023-10-08T02:28:39.820105+0000","last_became_active":"2023-10-08T02:28:38.822934+0000","last_became_peered":"2023-10-08T02:28:38.822934+0000","last_unstale":"2023-10-08T02:28:39.820105+0000","last_undegraded":"2023-10-08T02:28:39.820105+0000","last_fullsized":"2023-10-08T02:28:39.820105+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:08.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330998+0000","last_change":"2023-10-08T02:28:39.325748+0000","last_active":"2023-10-08T02:28:39.330998+0000","last_peered":"2023-10-08T02:28:39.330998+0000","last_clean":"2023-10-08T02:28:39.330998+0000","last_became_active":"2023-10-08T02:28:39.325507+0000","last_became_peered":"2023-10-08T02:28:39.325507+0000","last_unstale":"2023-10-08T02:28:39.330998+0000","last_undegraded":"2023-10-08T02:28:39.330998+0000","last_fullsized":"2023-10-08T02:28:39.330998+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:58.907072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.317442+0000","last_change":"2023-10-08T02:28:38.819566+0000","last_active":"2023-10-08T02:28:39.317442+0000","last_peered":"2023-10-08T02:28:39.317442+0000","last_clean":"2023-10-08T02:28:39.317442+0000","last_became_active":"2023-10-08T02:28:38.819144+0000","last_became_peered":"2023-10-08T02:28:38.819144+0000","last_unstale":"2023-10-08T02:28:39.317442+0000","last_undegraded":"2023-10-08T02:28:39.317442+0000","last_fullsized":"2023-10-08T02:28:39.317442+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:29.954980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329795+0000","last_change":"2023-10-08T02:28:39.325608+0000","last_active":"2023-10-08T02:28:39.329795+0000","last_peered":"2023-10-08T02:28:39.329795+0000","last_clean":"2023-10-08T02:28:39.329795+0000","last_became_active":"2023-10-08T02:28:39.325349+0000","last_became_peered":"2023-10-08T02:28:39.325349+0000","last_unstale":"2023-10-08T02:28:39.329795+0000","last_undegraded":"2023-10-08T02:28:39.329795+0000","last_fullsized":"2023-10-08T02:28:39.329795+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:47.883744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.818207+0000","last_change":"2023-10-08T02:28:38.824280+0000","last_active":"2023-10-08T02:28:39.818207+0000","last_peered":"2023-10-08T02:28:39.818207+0000","last_clean":"2023-10-08T02:28:39.818207+0000","last_became_active":"2023-10-08T02:28:38.823999+0000","last_became_peered":"2023-10-08T02:28:38.823999+0000","last_unstale":"2023-10-08T02:28:39.818207+0000","last_undegraded":"2023-10-08T02:28:39.818207+0000","last_fullsized":"2023-10-08T02:28:39.818207+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:07.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330961+0000","last_change":"2023-10-08T02:28:39.324643+0000","last_active":"2023-10-08T02:28:39.330961+0000","last_peered":"2023-10-08T02:28:39.330961+0000","last_clean":"2023-10-08T02:28:39.330961+0000","last_became_active":"2023-10-08T02:28:39.324393+0000","last_became_peered":"2023-10-08T02:28:39.324393+0000","last_unstale":"2023-10-08T02:28:39.330961+0000","last_undegraded":"2023-10-08T02:28:39.330961+0000","last_fullsized":"2023-10-08T02:28:39.330961+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:38.883411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.318229+0000","last_change":"2023-10-08T02:28:38.824325+0000","last_active":"2023-10-08T02:28:39.318229+0000","last_peered":"2023-10-08T02:28:39.318229+0000","last_clean":"2023-10-08T02:28:39.318229+0000","last_became_active":"2023-10-08T02:28:38.823161+0000","last_became_peered":"2023-10-08T02:28:38.823161+0000","last_unstale":"2023-10-08T02:28:39.318229+0000","last_undegraded":"2023-10-08T02:28:39.318229+0000","last_fullsized":"2023-10-08T02:28:39.318229+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:18.625443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.337547+0000","last_change":"2023-10-08T02:28:39.334542+0000","last_active":"2023-10-08T02:28:39.337547+0000","last_peered":"2023-10-08T02:28:39.337547+0000","last_clean":"2023-10-08T02:28:39.337547+0000","last_became_active":"2023-10-08T02:28:39.334137+0000","last_became_peered":"2023-10-08T02:28:39.334137+0000","last_unstale":"2023-10-08T02:28:39.337547+0000","last_undegraded":"2023-10-08T02:28:39.337547+0000","last_fullsized":"2023-10-08T02:28:39.337547+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:02:54.426138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.315308+0000","last_change":"2023-10-08T02:28:38.829914+0000","last_active":"2023-10-08T02:28:39.315308+0000","last_peered":"2023-10-08T02:28:39.315308+0000","last_clean":"2023-10-08T02:28:39.315308+0000","last_became_active":"2023-10-08T02:28:38.829662+0000","last_became_peered":"2023-10-08T02:28:38.829662+0000","last_unstale":"2023-10-08T02:28:39.315308+0000","last_undegraded":"2023-10-08T02:28:39.315308+0000","last_fullsized":"2023-10-08T02:28:39.315308+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:54.799537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.319175+0000","last_change":"2023-10-08T02:28:39.317508+0000","last_active":"2023-10-08T02:28:39.319175+0000","last_peered":"2023-10-08T02:28:39.319175+0000","last_clean":"2023-10-08T02:28:39.319175+0000","last_became_active":"2023-10-08T02:28:39.317242+0000","last_became_peered":"2023-10-08T02:28:39.317242+0000","last_unstale":"2023-10-08T02:28:39.319175+0000","last_undegraded":"2023-10-08T02:28:39.319175+0000","last_fullsized":"2023-10-08T02:28:39.319175+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:34.534831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.316246+0000","last_change":"2023-10-08T02:28:38.823623+0000","last_active":"2023-10-08T02:28:39.316246+0000","last_peered":"2023-10-08T02:28:39.316246+0000","last_clean":"2023-10-08T02:28:39.316246+0000","last_became_active":"2023-10-08T02:28:38.823343+0000","last_became_peered":"2023-10-08T02:28:38.823343+0000","last_unstale":"2023-10-08T02:28:39.316246+0000","last_undegraded":"2023-10-08T02:28:39.316246+0000","last_fullsized":"2023-10-08T02:28:39.316246+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:11.953794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327961+0000","last_change":"2023-10-08T02:28:39.324640+0000","last_active":"2023-10-08T02:28:39.327961+0000","last_peered":"2023-10-08T02:28:39.327961+0000","last_clean":"2023-10-08T02:28:39.327961+0000","last_became_active":"2023-10-08T02:28:39.324382+0000","last_became_peered":"2023-10-08T02:28:39.324382+0000","last_unstale":"2023-10-08T02:28:39.327961+0000","last_undegraded":"2023-10-08T02:28:39.327961+0000","last_fullsized":"2023-10-08T02:28:39.327961+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:01.177379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.819346+0000","last_change":"2023-10-08T02:28:38.823012+0000","last_active":"2023-10-08T02:28:39.819346+0000","last_peered":"2023-10-08T02:28:39.819346+0000","last_clean":"2023-10-08T02:28:39.819346+0000","last_became_active":"2023-10-08T02:28:38.822745+0000","last_became_peered":"2023-10-08T02:28:38.822745+0000","last_unstale":"2023-10-08T02:28:39.819346+0000","last_undegraded":"2023-10-08T02:28:39.819346+0000","last_fullsized":"2023-10-08T02:28:39.819346+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:25.660989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329737+0000","last_change":"2023-10-08T02:28:39.321822+0000","last_active":"2023-10-08T02:28:39.329737+0000","last_peered":"2023-10-08T02:28:39.329737+0000","last_clean":"2023-10-08T02:28:39.329737+0000","last_became_active":"2023-10-08T02:28:39.321572+0000","last_became_peered":"2023-10-08T02:28:39.321572+0000","last_unstale":"2023-10-08T02:28:39.329737+0000","last_undegraded":"2023-10-08T02:28:39.329737+0000","last_fullsized":"2023-10-08T02:28:39.329737+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:26:17.762885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.318968+0000","last_change":"2023-10-08T02:28:38.830771+0000","last_active":"2023-10-08T02:28:39.318968+0000","last_peered":"2023-10-08T02:28:39.318968+0000","last_clean":"2023-10-08T02:28:39.318968+0000","last_became_active":"2023-10-08T02:28:38.830279+0000","last_became_peered":"2023-10-08T02:28:38.830279+0000","last_unstale":"2023-10-08T02:28:39.318968+0000","last_undegraded":"2023-10-08T02:28:39.318968+0000","last_fullsized":"2023-10-08T02:28:39.318968+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:54.799537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329539+0000","last_change":"2023-10-08T02:28:39.324486+0000","last_active":"2023-10-08T02:28:39.329539+0000","last_peered":"2023-10-08T02:28:39.329539+0000","last_clean":"2023-10-08T02:28:39.329539+0000","last_became_active":"2023-10-08T02:28:39.323525+0000","last_became_peered":"2023-10-08T02:28:39.323525+0000","last_unstale":"2023-10-08T02:28:39.329539+0000","last_undegraded":"2023-10-08T02:28:39.329539+0000","last_fullsized":"2023-10-08T02:28:39.329539+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:03.109079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.312307+0000","last_change":"2023-10-08T02:28:38.818993+0000","last_active":"2023-10-08T02:28:39.312307+0000","last_peered":"2023-10-08T02:28:39.312307+0000","last_clean":"2023-10-08T02:28:39.312307+0000","last_became_active":"2023-10-08T02:28:38.818679+0000","last_became_peered":"2023-10-08T02:28:38.818679+0000","last_unstale":"2023-10-08T02:28:39.312307+0000","last_undegraded":"2023-10-08T02:28:39.312307+0000","last_fullsized":"2023-10-08T02:28:39.312307+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:45.471846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331794+0000","last_change":"2023-10-08T02:28:39.329653+0000","last_active":"2023-10-08T02:28:39.331794+0000","last_peered":"2023-10-08T02:28:39.331794+0000","last_clean":"2023-10-08T02:28:39.331794+0000","last_became_active":"2023-10-08T02:28:39.329353+0000","last_became_peered":"2023-10-08T02:28:39.329353+0000","last_unstale":"2023-10-08T02:28:39.331794+0000","last_undegraded":"2023-10-08T02:28:39.331794+0000","last_fullsized":"2023-10-08T02:28:39.331794+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:42.051410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.309944+0000","last_change":"2023-10-08T02:28:38.823266+0000","last_active":"2023-10-08T02:28:39.309944+0000","last_peered":"2023-10-08T02:28:39.309944+0000","last_clean":"2023-10-08T02:28:39.309944+0000","last_became_active":"2023-10-08T02:28:38.822984+0000","last_became_peered":"2023-10-08T02:28:38.822984+0000","last_unstale":"2023-10-08T02:28:39.309944+0000","last_undegraded":"2023-10-08T02:28:39.309944+0000","last_fullsized":"2023-10-08T02:28:39.309944+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:57:34.443528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.323781+0000","last_change":"2023-10-08T02:28:39.319795+0000","last_active":"2023-10-08T02:28:39.323781+0000","last_peered":"2023-10-08T02:28:39.323781+0000","last_clean":"2023-10-08T02:28:39.323781+0000","last_became_active":"2023-10-08T02:28:39.319551+0000","last_became_peered":"2023-10-08T02:28:39.319551+0000","last_unstale":"2023-10-08T02:28:39.323781+0000","last_undegraded":"2023-10-08T02:28:39.323781+0000","last_fullsized":"2023-10-08T02:28:39.323781+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:12.961726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.310037+0000","last_change":"2023-10-08T02:28:38.824771+0000","last_active":"2023-10-08T02:28:39.310037+0000","last_peered":"2023-10-08T02:28:39.310037+0000","last_clean":"2023-10-08T02:28:39.310037+0000","last_became_active":"2023-10-08T02:28:38.821862+0000","last_became_peered":"2023-10-08T02:28:38.821862+0000","last_unstale":"2023-10-08T02:28:39.310037+0000","last_undegraded":"2023-10-08T02:28:39.310037+0000","last_fullsized":"2023-10-08T02:28:39.310037+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:57.443582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.328719+0000","last_change":"2023-10-08T02:28:39.323286+0000","last_active":"2023-10-08T02:28:39.328719+0000","last_peered":"2023-10-08T02:28:39.328719+0000","last_clean":"2023-10-08T02:28:39.328719+0000","last_became_active":"2023-10-08T02:28:39.323031+0000","last_became_peered":"2023-10-08T02:28:39.323031+0000","last_unstale":"2023-10-08T02:28:39.328719+0000","last_undegraded":"2023-10-08T02:28:39.328719+0000","last_fullsized":"2023-10-08T02:28:39.328719+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:08.648973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.315573+0000","last_change":"2023-10-08T02:28:38.822167+0000","last_active":"2023-10-08T02:28:39.315573+0000","last_peered":"2023-10-08T02:28:39.315573+0000","last_clean":"2023-10-08T02:28:39.315573+0000","last_became_active":"2023-10-08T02:28:38.821888+0000","last_became_peered":"2023-10-08T02:28:38.821888+0000","last_unstale":"2023-10-08T02:28:39.315573+0000","last_undegraded":"2023-10-08T02:28:39.315573+0000","last_fullsized":"2023-10-08T02:28:39.315573+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:03.125725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.335828+0000","last_change":"2023-10-08T02:28:39.330719+0000","last_active":"2023-10-08T02:28:39.335828+0000","last_peered":"2023-10-08T02:28:39.335828+0000","last_clean":"2023-10-08T02:28:39.335828+0000","last_became_active":"2023-10-08T02:28:39.330455+0000","last_became_peered":"2023-10-08T02:28:39.330455+0000","last_unstale":"2023-10-08T02:28:39.335828+0000","last_undegraded":"2023-10-08T02:28:39.335828+0000","last_fullsized":"2023-10-08T02:28:39.335828+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:24:46.354643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.311510+0000","last_change":"2023-10-08T02:28:38.818228+0000","last_active":"2023-10-08T02:28:39.311510+0000","last_peered":"2023-10-08T02:28:39.311510+0000","last_clean":"2023-10-08T02:28:39.311510+0000","last_became_active":"2023-10-08T02:28:38.817959+0000","last_became_peered":"2023-10-08T02:28:38.817959+0000","last_unstale":"2023-10-08T02:28:39.311510+0000","last_undegraded":"2023-10-08T02:28:39.311510+0000","last_fullsized":"2023-10-08T02:28:39.311510+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:55:51.289187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327630+0000","last_change":"2023-10-08T02:28:39.322512+0000","last_active":"2023-10-08T02:28:39.327630+0000","last_peered":"2023-10-08T02:28:39.327630+0000","last_clean":"2023-10-08T02:28:39.327630+0000","last_became_active":"2023-10-08T02:28:39.321857+0000","last_became_peered":"2023-10-08T02:28:39.321857+0000","last_unstale":"2023-10-08T02:28:39.327630+0000","last_undegraded":"2023-10-08T02:28:39.327630+0000","last_fullsized":"2023-10-08T02:28:39.327630+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:30.962913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.310028+0000","last_change":"2023-10-08T02:28:38.820762+0000","last_active":"2023-10-08T02:28:39.310028+0000","last_peered":"2023-10-08T02:28:39.310028+0000","last_clean":"2023-10-08T02:28:39.310028+0000","last_became_active":"2023-10-08T02:28:38.820439+0000","last_became_peered":"2023-10-08T02:28:38.820439+0000","last_unstale":"2023-10-08T02:28:39.310028+0000","last_undegraded":"2023-10-08T02:28:39.310028+0000","last_fullsized":"2023-10-08T02:28:39.310028+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:18.625443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327982+0000","last_change":"2023-10-08T02:28:39.325300+0000","last_active":"2023-10-08T02:28:39.327982+0000","last_peered":"2023-10-08T02:28:39.327982+0000","last_clean":"2023-10-08T02:28:39.327982+0000","last_became_active":"2023-10-08T02:28:39.325069+0000","last_became_peered":"2023-10-08T02:28:39.325069+0000","last_unstale":"2023-10-08T02:28:39.327982+0000","last_undegraded":"2023-10-08T02:28:39.327982+0000","last_fullsized":"2023-10-08T02:28:39.327982+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:53.962266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.310305+0000","last_change":"2023-10-08T02:28:38.826218+0000","last_active":"2023-10-08T02:28:39.310305+0000","last_peered":"2023-10-08T02:28:39.310305+0000","last_clean":"2023-10-08T02:28:39.310305+0000","last_became_active":"2023-10-08T02:28:38.824990+0000","last_became_peered":"2023-10-08T02:28:38.824990+0000","last_unstale":"2023-10-08T02:28:39.310305+0000","last_undegraded":"2023-10-08T02:28:39.310305+0000","last_fullsized":"2023-10-08T02:28:39.310305+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:29.954980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"180'326","reported_seq":1048,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.316333+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:28:39.316333+0000","last_peered":"2023-10-08T02:28:39.316333+0000","last_clean":"2023-10-08T02:28:39.316333+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:28:39.316333+0000","last_undegraded":"2023-10-08T02:28:39.316333+0000","last_fullsized":"2023-10-08T02:28:39.316333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312512,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059328,"statfs":{"total":96636764160,"available":96316751872,"internally_reserved":0,"allocated":3006464,"data_stored":1795310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.94999999999999996,"15min":1.0429999999999999},"last":0.629},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.48099999999999998},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.052},"last":0.53900000000000003}]},{"osd":1,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.61499999999999999,"5min":0.47499999999999998,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.61499999999999999,"5min":0.47699999999999998,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.01,"15min":1.0549999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.60299999999999998,"5min":0.503,"15min":0.47199999999999998},"max":{"1min":0.92900000000000005,"5min":1.03,"15min":1.03},"last":0.78800000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46100000000000002},"max":{"1min":0.872,"5min":0.91000000000000003,"15min":0.96399999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.625,"5min":0.60899999999999999,"15min":0.51300000000000001},"max":{"1min":0.90300000000000002,"5min":1.056,"15min":1.056},"last":0.80300000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":1.091,"15min":1.091},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.76900000000000002,"5min":0.92200000000000004,"15min":0.95699999999999996},"last":0.755}]},{"osd":5,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1399999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.39400000000000002},"max":{"1min":0.88,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.77400000000000002},"min":{"1min":0.60699999999999998,"5min":0.54500000000000004,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":1.151,"15min":1.151},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.004,"15min":1.0329999999999999},"last":0.84099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575953,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299624,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072216,"statfs":{"total":96636764160,"available":96329949184,"internally_reserved":0,"allocated":2392064,"data_stored":1202714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.41799999999999998},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.40799999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.61799999999999999,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":0.97799999999999998,"15min":1.21},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.78200000000000003,"15min":0.75800000000000001},"min":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.45700000000000002},"max":{"1min":3.851,"5min":3.851,"15min":3.851},"last":0.54600000000000004}]},{"osd":2,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.60399999999999998,"5min":0.498,"15min":0.44400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":1.002,"15min":1.258},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":2.6970000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.58499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.376},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.42599999999999999},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.50800000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.82799999999999996,"5min":0.86299999999999999,"15min":0.92800000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.0980000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.56599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97399999999999998,"15min":1.054},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575953,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311912,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059928,"statfs":{"total":96636764160,"available":96317366272,"internally_reserved":0,"allocated":2392064,"data_stored":1202726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.70299999999999996,"15min":1.0780000000000001},"last":0.503},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":1.571},"last":0.38800000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.318},"max":{"1min":0.624,"5min":0.70899999999999996,"15min":1.1100000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":0.71799999999999997,"5min":0.85899999999999999,"15min":1.1299999999999999},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40699999999999997},"max":{"1min":0.78600000000000003,"5min":0.875,"15min":1.2190000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.99099999999999999},"last":0.435}]},{"osd":4,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.61099999999999999,"5min":0.53500000000000003,"15min":0.48199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.40600000000000003},"max":{"1min":1.05,"5min":1.1539999999999999,"15min":1.1950000000000001},"last":0.75}]},{"osd":5,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0509999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81799999999999995,"15min":0.82099999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.127,"15min":1.27},"last":0.59099999999999997}]},{"osd":6,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.141,"15min":1.141},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.60999999999999999,"5min":0.57099999999999995,"15min":0.50900000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.204},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.61199999999999999},"max":{"1min":1.109,"5min":1.1359999999999999,"15min":1.306},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.55500000000000005},"max":{"1min":1.0629999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.626}]}]},{"osd":0,"up_from":13,"seq":55834575952,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308392,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063448,"statfs":{"total":96636764160,"available":96320970752,"internally_reserved":0,"allocated":2981888,"data_stored":1793060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.47599999999999998,"15min":0.47799999999999998},"min":{"1min":0.32200000000000001,"5min":0.31,"15min":0.23699999999999999},"max":{"1min":0.61499999999999999,"5min":0.73599999999999999,"15min":0.80300000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51300000000000001,"15min":0.52200000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70899999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.314}]},{"osd":2,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.75800000000000001,"5min":0.77800000000000002,"15min":0.84399999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.45200000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.93100000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.89000000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.53800000000000003},"max":{"1min":1.0700000000000001,"5min":1.232,"15min":1.232},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80300000000000005,"15min":0.80100000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.52500000000000002},"max":{"1min":0.97199999999999998,"5min":1.262,"15min":1.262},"last":0.97199999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83999999999999997,"15min":0.84299999999999997},"min":{"1min":0.65400000000000003,"5min":0.61499999999999999,"15min":0.55700000000000005},"max":{"1min":1.008,"5min":1.1990000000000001,"15min":1.216},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.873,"15min":0.876},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.58399999999999996},"max":{"1min":1.089,"5min":1.296,"15min":1.296},"last":0.88400000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.83299999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0640000000000001,"5min":1.125,"15min":1.1890000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58699999999999997,"5min":0.58099999999999996,"15min":0.54500000000000004},"max":{"1min":0.94099999999999995,"5min":1.262,"15min":1.262},"last":0.873}]},{"osd":7,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.04,"5min":1.2170000000000001,"15min":1.296},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85799999999999998,"15min":0.88},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"max":{"1min":1.143,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.83099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316008,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055832,"statfs":{"total":96636764160,"available":96313171968,"internally_reserved":0,"allocated":2392064,"data_stored":1202782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.71999999999999997,"15min":0.871},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.83199999999999996},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.86299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.36599999999999999},"max":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81999999999999995},"last":0.56599999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63400000000000001,"15min":0.622},"min":{"1min":0.505,"5min":0.47699999999999998,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.90000000000000002},"last":0.61899999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81299999999999994,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.49099999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.871}]},{"osd":5,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85999999999999999,"15min":0.84199999999999997},"min":{"1min":0.65600000000000003,"5min":0.56499999999999995,"15min":0.46400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88600000000000001,"15min":0.85899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.417},"last":0.95599999999999996}]},{"osd":6,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83899999999999997,"15min":0.81799999999999995},"min":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.333},"last":1.04},{"interface":"front","average":{"1min":0.872,"5min":0.84499999999999997,"15min":0.82999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.51800000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84899999999999998,"15min":0.82599999999999996},"min":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.4059999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.85199999999999998,"15min":0.84099999999999997},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56899999999999995},"max":{"1min":1.252,"5min":1.252,"15min":1.5720000000000001},"last":1.002}]}]},{"osd":3,"up_from":13,"seq":55834575952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299624,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072216,"statfs":{"total":96636764160,"available":96329949184,"internally_reserved":0,"allocated":2392064,"data_stored":1202726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.47099999999999997,"15min":0.46000000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63,"5min":0.84699999999999998,"15min":0.90300000000000002},"last":0.5},{"interface":"front","average":{"1min":0.496,"5min":0.502,"15min":0.501},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.311},"max":{"1min":0.67300000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.34300000000000003}]},{"osd":1,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.33500000000000002,"15min":0.308},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":1.012,"15min":1.012},"last":0.60199999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":0.879,"15min":1.282},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81499999999999995,"5min":0.89200000000000002,"15min":1.512},"last":0.65100000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.77300000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0209999999999999,"5min":1.0509999999999999,"15min":1.1799999999999999},"last":0.442},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.80300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.004,"5min":1.1399999999999999,"15min":1.3180000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.81799999999999995,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.995,"5min":1.147,"15min":1.2789999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82899999999999996,"15min":0.80600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.006,"5min":1.056,"15min":1.24},"last":0.93000000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.53600000000000003},"max":{"1min":0.95499999999999996,"5min":1.109,"15min":1.161},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.82199999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.99099999999999999,"5min":1.155,"15min":1.2849999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.83199999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.129,"15min":1.266},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.85199999999999998,"15min":0.83399999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.94899999999999995,"5min":1.1160000000000001,"15min":1.282},"last":0.94899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316020,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055820,"statfs":{"total":96636764160,"available":96313159680,"internally_reserved":0,"allocated":2404352,"data_stored":1204298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.57899999999999996,"5min":0.48199999999999998,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":1.016,"15min":1.028},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.497}]},{"osd":1,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.49099999999999999},"max":{"1min":0.92900000000000005,"5min":1.079,"15min":1.079},"last":0.78700000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.78100000000000003},"min":{"1min":0.55100000000000005,"5min":0.50800000000000001,"15min":0.44600000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.034},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.69399999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":1.097,"15min":1.097},"last":0.57399999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.47199999999999998},"max":{"1min":1.0029999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.872,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76100000000000001,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.48599999999999999,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":1.105,"15min":1.105},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.98299999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":1.03,"15min":1.03},"last":0.61199999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97099999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.42699999999999999},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311916,"kb_used_data":2340,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059924,"statfs":{"total":96636764160,"available":96317362176,"internally_reserved":0,"allocated":2396160,"data_stored":1203268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.56499999999999995,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":0.996,"15min":0.996},"last":0.753},{"interface":"front","average":{"1min":0.746,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.746},"min":{"1min":0.503,"5min":0.503,"15min":0.44600000000000001},"max":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":1.046},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.751,"15min":0.752},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.499},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69399999999999995}]},{"osd":2,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.76700000000000002},"min":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.48599999999999999},"max":{"1min":0.96099999999999997,"5min":1.028,"15min":1.028},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.76700000000000002},"min":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.51100000000000001},"max":{"1min":1.0009999999999999,"5min":1.006,"15min":1.006},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98899999999999999,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88600000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.503,"5min":0.434,"15min":0.434},"max":{"1min":0.872,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66700000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":1.01,"15min":1.044},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.745,"15min":0.751},"min":{"1min":0.60999999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:28:49.024 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:28:49.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.217+0000 7f538edff700 1 Processor -- start 2023-10-08T02:28:49.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.217+0000 7f538edff700 1 -- start start 2023-10-08T02:28:49.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.218+0000 7f538edff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5388159890 0x7f538815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.218+0000 7f538edff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5388159c60 0x7f538815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.218+0000 7f538edff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538815a160 con 0x7f538815db10 2023-10-08T02:28:49.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.218+0000 7f538edff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538815df50 con 0x7f5388159c60 2023-10-08T02:28:49.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.218+0000 7f538edff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53880d13a0 con 0x7f5388159890 2023-10-08T02:28:49.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.218+0000 7f5387fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5388159c60 0x7f538815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.218+0000 7f5387fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5388159c60 0x7f538815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37526/0 (socket says 172.21.15.119:37526) 2023-10-08T02:28:49.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f5387fff700 1 -- 172.21.15.119:0/2345877551 learned_addr learned my addr 172.21.15.119:0/2345877551 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:49.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f538cb9b700 1 --2- 172.21.15.119:0/2345877551 >> v2:172.21.15.184:3300/0 conn(0x7f5388159890 0x7f538815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f5387fff700 1 -- 172.21.15.119:0/2345877551 >> v2:172.21.15.184:3300/0 conn(0x7f5388159890 msgr2=0x7f538815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:49.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f5387fff700 1 --2- 172.21.15.119:0/2345877551 >> v2:172.21.15.184:3300/0 conn(0x7f5388159890 0x7f538815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f5387fff700 1 -- 172.21.15.119:0/2345877551 >> v1:172.21.15.184:6789/0 conn(0x7f538815db10 legacy=0x7f5388069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f5387fff700 1 -- 172.21.15.119:0/2345877551 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538815e140 con 0x7f5388159c60 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f5387fff700 1 --2- 172.21.15.119:0/2345877551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5388159c60 0x7f538815d5d0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f53880d1090 tx=0x7f5374009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63823570650042e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f53877fe700 1 -- 172.21.15.119:0/2345877551 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400b860 con 0x7f5388159c60 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.219+0000 7f53877fe700 1 -- 172.21.15.119:0/2345877551 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537400b9c0 con 0x7f5388159c60 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f53877fe700 1 -- 172.21.15.119:0/2345877551 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537400bd60 con 0x7f5388159c60 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f538edff700 1 -- 172.21.15.119:0/2345877551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5388159c60 msgr2=0x7f538815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f538edff700 1 --2- 172.21.15.119:0/2345877551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5388159c60 0x7f538815d5d0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f53880d1090 tx=0x7f5374009250 comp rx=0 tx=0).stop 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f538edff700 1 -- 172.21.15.119:0/2345877551 shutdown_connections 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f538edff700 1 --2- 172.21.15.119:0/2345877551 >> v2:172.21.15.184:3300/0 conn(0x7f5388159890 0x7f538815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f538edff700 1 --2- 172.21.15.119:0/2345877551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5388159c60 0x7f538815d5d0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f538edff700 1 -- 172.21.15.119:0/2345877551 >> 172.21.15.119:0/2345877551 conn(0x7f5388068e20 msgr2=0x7f53880d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:49.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f538edff700 1 -- 172.21.15.119:0/2345877551 shutdown_connections 2023-10-08T02:28:49.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.220+0000 7f538edff700 1 -- 172.21.15.119:0/2345877551 wait complete. 2023-10-08T02:28:49.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.222+0000 7f538edff700 1 Processor -- start 2023-10-08T02:28:49.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.222+0000 7f538edff700 1 -- start start 2023-10-08T02:28:49.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.222+0000 7f538edff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5388159890 0x7f538816a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538edff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388159c60 0x7f53880cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538edff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538815db10 0x7f53880b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538edff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53880b5720 con 0x7f538815db10 2023-10-08T02:28:49.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538edff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53880b36c0 con 0x7f5388159c60 2023-10-08T02:28:49.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538edff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53880b39a0 con 0x7f5388159890 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538815db10 0x7f53880b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538815db10 0x7f53880b5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37540/0 (socket says 172.21.15.119:37540) 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 -- 172.21.15.119:0/423907414 learned_addr learned my addr 172.21.15.119:0/423907414 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f5387fff700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388159c60 0x7f53880cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538cb9b700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5388159890 0x7f538816a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 -- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5388159890 msgr2=0x7f538816a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5388159890 0x7f538816a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 -- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388159c60 msgr2=0x7f53880cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388159c60 0x7f53880cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 -- 172.21.15.119:0/423907414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740094f0 con 0x7f538815db10 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538cb9b700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5388159890 0x7f538816a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.223+0000 7f538d39c700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538815db10 0x7f53880b5bb0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f538000c840 tx=0x7f5380001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.224+0000 7f5384ff9700 1 -- 172.21.15.119:0/423907414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380012630 con 0x7f538815db10 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.224+0000 7f5384ff9700 1 -- 172.21.15.119:0/423907414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53800127b0 con 0x7f538815db10 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.224+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53880b3c80 con 0x7f538815db10 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.224+0000 7f5384ff9700 1 -- 172.21.15.119:0/423907414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380012b70 con 0x7f538815db10 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.224+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53880cfe70 con 0x7f538815db10 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.225+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53880b5890 con 0x7f538815db10 2023-10-08T02:28:49.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.226+0000 7f5384ff9700 1 -- 172.21.15.119:0/423907414 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5380003030 con 0x7f538815db10 2023-10-08T02:28:49.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.226+0000 7f5384ff9700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f536c066f30 0x7f536c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.227+0000 7f5384ff9700 1 -- 172.21.15.119:0/423907414 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f53800846c0 con 0x7f538815db10 2023-10-08T02:28:49.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.227+0000 7f538cb9b700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f536c066f30 0x7f536c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.227+0000 7f538cb9b700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f536c066f30 0x7f536c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f537c006fd0 tx=0x7f537c008040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:49.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.230+0000 7f5384ff9700 1 -- 172.21.15.119:0/423907414 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5380055130 con 0x7f538815db10 2023-10-08T02:28:49.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.422+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53880b5230 con 0x7f536c066f30 2023-10-08T02:28:49.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.431+0000 7f5384ff9700 1 -- 172.21.15.119:0/423907414 <== mgr.9543 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288957 (secure 0 0 0) 0x7f537c00f760 con 0x7f536c066f30 2023-10-08T02:28:49.435 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:49.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f536c066f30 msgr2=0x7f536c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f536c066f30 0x7f536c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f537c006fd0 tx=0x7f537c008040 comp rx=0 tx=0).stop 2023-10-08T02:28:49.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538815db10 msgr2=0x7f53880b5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538815db10 0x7f53880b5bb0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f538000c840 tx=0x7f5380001670 comp rx=0 tx=0).stop 2023-10-08T02:28:49.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 shutdown_connections 2023-10-08T02:28:49.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f536c066f30 0x7f536c0693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5388159890 0x7f538816a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f538815db10 0x7f53880b5bb0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 --2- 172.21.15.119:0/423907414 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5388159c60 0x7f53880cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.435+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 >> 172.21.15.119:0/423907414 conn(0x7f5388068e20 msgr2=0x7f53881531f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:49.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.436+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 shutdown_connections 2023-10-08T02:28:49.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.436+0000 7f538edff700 1 -- 172.21.15.119:0/423907414 wait complete. 2023-10-08T02:28:49.439 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:28:49.453 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-10-08T02:28:47.771491+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":583,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":364,"ondisk_log_size":364,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2476008,"kb_used_data":19880,"kb_used_omap":0,"kb_used_meta":2456064,"kb_avail":752498712,"statfs":{"total":773094113280,"available":770558681088,"internally_reserved":0,"allocated":20357120,"data_stored":10806884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2515009536},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.462770"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333047+0000","last_change":"2023-10-08T02:28:39.329482+0000","last_active":"2023-10-08T02:28:39.333047+0000","last_peered":"2023-10-08T02:28:39.333047+0000","last_clean":"2023-10-08T02:28:39.333047+0000","last_became_active":"2023-10-08T02:28:39.329198+0000","last_became_peered":"2023-10-08T02:28:39.329198+0000","last_unstale":"2023-10-08T02:28:39.333047+0000","last_undegraded":"2023-10-08T02:28:39.333047+0000","last_fullsized":"2023-10-08T02:28:39.333047+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:15.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330383+0000","last_change":"2023-10-08T02:28:39.324811+0000","last_active":"2023-10-08T02:28:39.330383+0000","last_peered":"2023-10-08T02:28:39.330383+0000","last_clean":"2023-10-08T02:28:39.330383+0000","last_became_active":"2023-10-08T02:28:39.324408+0000","last_became_peered":"2023-10-08T02:28:39.324408+0000","last_unstale":"2023-10-08T02:28:39.330383+0000","last_undegraded":"2023-10-08T02:28:39.330383+0000","last_fullsized":"2023-10-08T02:28:39.330383+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:55.807470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331675+0000","last_change":"2023-10-08T02:28:39.328136+0000","last_active":"2023-10-08T02:28:39.331675+0000","last_peered":"2023-10-08T02:28:39.331675+0000","last_clean":"2023-10-08T02:28:39.331675+0000","last_became_active":"2023-10-08T02:28:39.327879+0000","last_became_peered":"2023-10-08T02:28:39.327879+0000","last_unstale":"2023-10-08T02:28:39.331675+0000","last_undegraded":"2023-10-08T02:28:39.331675+0000","last_fullsized":"2023-10-08T02:28:39.331675+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:58.451514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333456+0000","last_change":"2023-10-08T02:28:39.329838+0000","last_active":"2023-10-08T02:28:39.333456+0000","last_peered":"2023-10-08T02:28:39.333456+0000","last_clean":"2023-10-08T02:28:39.333456+0000","last_became_active":"2023-10-08T02:28:39.329181+0000","last_became_peered":"2023-10-08T02:28:39.329181+0000","last_unstale":"2023-10-08T02:28:39.333456+0000","last_undegraded":"2023-10-08T02:28:39.333456+0000","last_fullsized":"2023-10-08T02:28:39.333456+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:47.883744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.319796+0000","last_change":"2023-10-08T02:28:39.318284+0000","last_active":"2023-10-08T02:28:39.319796+0000","last_peered":"2023-10-08T02:28:39.319796+0000","last_clean":"2023-10-08T02:28:39.319796+0000","last_became_active":"2023-10-08T02:28:39.317981+0000","last_became_peered":"2023-10-08T02:28:39.317981+0000","last_unstale":"2023-10-08T02:28:39.319796+0000","last_undegraded":"2023-10-08T02:28:39.319796+0000","last_fullsized":"2023-10-08T02:28:39.319796+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:32.505096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327172+0000","last_change":"2023-10-08T02:28:39.322993+0000","last_active":"2023-10-08T02:28:39.327172+0000","last_peered":"2023-10-08T02:28:39.327172+0000","last_clean":"2023-10-08T02:28:39.327172+0000","last_became_active":"2023-10-08T02:28:39.322746+0000","last_became_peered":"2023-10-08T02:28:39.322746+0000","last_unstale":"2023-10-08T02:28:39.327172+0000","last_undegraded":"2023-10-08T02:28:39.327172+0000","last_fullsized":"2023-10-08T02:28:39.327172+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:41:31.075827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331401+0000","last_change":"2023-10-08T02:28:39.326667+0000","last_active":"2023-10-08T02:28:39.331401+0000","last_peered":"2023-10-08T02:28:39.331401+0000","last_clean":"2023-10-08T02:28:39.331401+0000","last_became_active":"2023-10-08T02:28:39.326352+0000","last_became_peered":"2023-10-08T02:28:39.326352+0000","last_unstale":"2023-10-08T02:28:39.331401+0000","last_undegraded":"2023-10-08T02:28:39.331401+0000","last_fullsized":"2023-10-08T02:28:39.331401+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:21:27.367566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330942+0000","last_change":"2023-10-08T02:28:39.327344+0000","last_active":"2023-10-08T02:28:39.330942+0000","last_peered":"2023-10-08T02:28:39.330942+0000","last_clean":"2023-10-08T02:28:39.330942+0000","last_became_active":"2023-10-08T02:28:39.327079+0000","last_became_peered":"2023-10-08T02:28:39.327079+0000","last_unstale":"2023-10-08T02:28:39.330942+0000","last_undegraded":"2023-10-08T02:28:39.330942+0000","last_fullsized":"2023-10-08T02:28:39.330942+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:42.424670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.332273+0000","last_change":"2023-10-08T02:28:39.330254+0000","last_active":"2023-10-08T02:28:39.332273+0000","last_peered":"2023-10-08T02:28:39.332273+0000","last_clean":"2023-10-08T02:28:39.332273+0000","last_became_active":"2023-10-08T02:28:39.329980+0000","last_became_peered":"2023-10-08T02:28:39.329980+0000","last_unstale":"2023-10-08T02:28:39.332273+0000","last_undegraded":"2023-10-08T02:28:39.332273+0000","last_fullsized":"2023-10-08T02:28:39.332273+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:05.392617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330859+0000","last_change":"2023-10-08T02:28:39.327115+0000","last_active":"2023-10-08T02:28:39.330859+0000","last_peered":"2023-10-08T02:28:39.330859+0000","last_clean":"2023-10-08T02:28:39.330859+0000","last_became_active":"2023-10-08T02:28:39.326835+0000","last_became_peered":"2023-10-08T02:28:39.326835+0000","last_unstale":"2023-10-08T02:28:39.330859+0000","last_undegraded":"2023-10-08T02:28:39.330859+0000","last_fullsized":"2023-10-08T02:28:39.330859+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:04.133657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330538+0000","last_change":"2023-10-08T02:28:39.327088+0000","last_active":"2023-10-08T02:28:39.330538+0000","last_peered":"2023-10-08T02:28:39.330538+0000","last_clean":"2023-10-08T02:28:39.330538+0000","last_became_active":"2023-10-08T02:28:39.326786+0000","last_became_peered":"2023-10-08T02:28:39.326786+0000","last_unstale":"2023-10-08T02:28:39.330538+0000","last_undegraded":"2023-10-08T02:28:39.330538+0000","last_fullsized":"2023-10-08T02:28:39.330538+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:32.211850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330511+0000","last_change":"2023-10-08T02:28:39.325570+0000","last_active":"2023-10-08T02:28:39.330511+0000","last_peered":"2023-10-08T02:28:39.330511+0000","last_clean":"2023-10-08T02:28:39.330511+0000","last_became_active":"2023-10-08T02:28:39.325311+0000","last_became_peered":"2023-10-08T02:28:39.325311+0000","last_unstale":"2023-10-08T02:28:39.330511+0000","last_undegraded":"2023-10-08T02:28:39.330511+0000","last_fullsized":"2023-10-08T02:28:39.330511+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:17:00.479132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330248+0000","last_change":"2023-10-08T02:28:39.324775+0000","last_active":"2023-10-08T02:28:39.330248+0000","last_peered":"2023-10-08T02:28:39.330248+0000","last_clean":"2023-10-08T02:28:39.330248+0000","last_became_active":"2023-10-08T02:28:39.324509+0000","last_became_peered":"2023-10-08T02:28:39.324509+0000","last_unstale":"2023-10-08T02:28:39.330248+0000","last_undegraded":"2023-10-08T02:28:39.330248+0000","last_fullsized":"2023-10-08T02:28:39.330248+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:03.109079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334896+0000","last_change":"2023-10-08T02:28:39.333508+0000","last_active":"2023-10-08T02:28:39.334896+0000","last_peered":"2023-10-08T02:28:39.334896+0000","last_clean":"2023-10-08T02:28:39.334896+0000","last_became_active":"2023-10-08T02:28:39.332581+0000","last_became_peered":"2023-10-08T02:28:39.332581+0000","last_unstale":"2023-10-08T02:28:39.334896+0000","last_undegraded":"2023-10-08T02:28:39.334896+0000","last_fullsized":"2023-10-08T02:28:39.334896+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:16.498144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329187+0000","last_change":"2023-10-08T02:28:39.324628+0000","last_active":"2023-10-08T02:28:39.329187+0000","last_peered":"2023-10-08T02:28:39.329187+0000","last_clean":"2023-10-08T02:28:39.329187+0000","last_became_active":"2023-10-08T02:28:39.324384+0000","last_became_peered":"2023-10-08T02:28:39.324384+0000","last_unstale":"2023-10-08T02:28:39.329187+0000","last_undegraded":"2023-10-08T02:28:39.329187+0000","last_fullsized":"2023-10-08T02:28:39.329187+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:42.051410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.335810+0000","last_change":"2023-10-08T02:28:39.331258+0000","last_active":"2023-10-08T02:28:39.335810+0000","last_peered":"2023-10-08T02:28:39.335810+0000","last_clean":"2023-10-08T02:28:39.335810+0000","last_became_active":"2023-10-08T02:28:39.331012+0000","last_became_peered":"2023-10-08T02:28:39.331012+0000","last_unstale":"2023-10-08T02:28:39.335810+0000","last_undegraded":"2023-10-08T02:28:39.335810+0000","last_fullsized":"2023-10-08T02:28:39.335810+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:00:11.626570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329498+0000","last_change":"2023-10-08T02:28:39.323511+0000","last_active":"2023-10-08T02:28:39.329498+0000","last_peered":"2023-10-08T02:28:39.329498+0000","last_clean":"2023-10-08T02:28:39.329498+0000","last_became_active":"2023-10-08T02:28:39.323257+0000","last_became_peered":"2023-10-08T02:28:39.323257+0000","last_unstale":"2023-10-08T02:28:39.329498+0000","last_undegraded":"2023-10-08T02:28:39.329498+0000","last_fullsized":"2023-10-08T02:28:39.329498+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:44.356227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.324791+0000","last_change":"2023-10-08T02:28:39.320581+0000","last_active":"2023-10-08T02:28:39.324791+0000","last_peered":"2023-10-08T02:28:39.324791+0000","last_clean":"2023-10-08T02:28:39.324791+0000","last_became_active":"2023-10-08T02:28:39.320313+0000","last_became_peered":"2023-10-08T02:28:39.320313+0000","last_unstale":"2023-10-08T02:28:39.324791+0000","last_undegraded":"2023-10-08T02:28:39.324791+0000","last_fullsized":"2023-10-08T02:28:39.324791+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:09.648666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329874+0000","last_change":"2023-10-08T02:28:39.327005+0000","last_active":"2023-10-08T02:28:39.329874+0000","last_peered":"2023-10-08T02:28:39.329874+0000","last_clean":"2023-10-08T02:28:39.329874+0000","last_became_active":"2023-10-08T02:28:39.326746+0000","last_became_peered":"2023-10-08T02:28:39.326746+0000","last_unstale":"2023-10-08T02:28:39.329874+0000","last_undegraded":"2023-10-08T02:28:39.329874+0000","last_fullsized":"2023-10-08T02:28:39.329874+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:48.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334891+0000","last_change":"2023-10-08T02:28:39.333330+0000","last_active":"2023-10-08T02:28:39.334891+0000","last_peered":"2023-10-08T02:28:39.334891+0000","last_clean":"2023-10-08T02:28:39.334891+0000","last_became_active":"2023-10-08T02:28:39.332988+0000","last_became_peered":"2023-10-08T02:28:39.332988+0000","last_unstale":"2023-10-08T02:28:39.334891+0000","last_undegraded":"2023-10-08T02:28:39.334891+0000","last_fullsized":"2023-10-08T02:28:39.334891+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:01.177379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329470+0000","last_change":"2023-10-08T02:28:39.322657+0000","last_active":"2023-10-08T02:28:39.329470+0000","last_peered":"2023-10-08T02:28:39.329470+0000","last_clean":"2023-10-08T02:28:39.329470+0000","last_became_active":"2023-10-08T02:28:39.322413+0000","last_became_peered":"2023-10-08T02:28:39.322413+0000","last_unstale":"2023-10-08T02:28:39.329470+0000","last_undegraded":"2023-10-08T02:28:39.329470+0000","last_fullsized":"2023-10-08T02:28:39.329470+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:47.883744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327619+0000","last_change":"2023-10-08T02:28:39.322682+0000","last_active":"2023-10-08T02:28:39.327619+0000","last_peered":"2023-10-08T02:28:39.327619+0000","last_clean":"2023-10-08T02:28:39.327619+0000","last_became_active":"2023-10-08T02:28:39.322383+0000","last_became_peered":"2023-10-08T02:28:39.322383+0000","last_unstale":"2023-10-08T02:28:39.327619+0000","last_undegraded":"2023-10-08T02:28:39.327619+0000","last_fullsized":"2023-10-08T02:28:39.327619+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:32.043813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.320377+0000","last_change":"2023-10-08T02:28:39.318910+0000","last_active":"2023-10-08T02:28:39.320377+0000","last_peered":"2023-10-08T02:28:39.320377+0000","last_clean":"2023-10-08T02:28:39.320377+0000","last_became_active":"2023-10-08T02:28:39.318676+0000","last_became_peered":"2023-10-08T02:28:39.318676+0000","last_unstale":"2023-10-08T02:28:39.320377+0000","last_undegraded":"2023-10-08T02:28:39.320377+0000","last_fullsized":"2023-10-08T02:28:39.320377+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:53:47.577446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330924+0000","last_change":"2023-10-08T02:28:39.325449+0000","last_active":"2023-10-08T02:28:39.330924+0000","last_peered":"2023-10-08T02:28:39.330924+0000","last_clean":"2023-10-08T02:28:39.330924+0000","last_became_active":"2023-10-08T02:28:39.325179+0000","last_became_peered":"2023-10-08T02:28:39.325179+0000","last_unstale":"2023-10-08T02:28:39.330924+0000","last_undegraded":"2023-10-08T02:28:39.330924+0000","last_fullsized":"2023-10-08T02:28:39.330924+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:08.018509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334159+0000","last_change":"2023-10-08T02:28:39.329687+0000","last_active":"2023-10-08T02:28:39.334159+0000","last_peered":"2023-10-08T02:28:39.334159+0000","last_clean":"2023-10-08T02:28:39.334159+0000","last_became_active":"2023-10-08T02:28:39.329403+0000","last_became_peered":"2023-10-08T02:28:39.329403+0000","last_unstale":"2023-10-08T02:28:39.334159+0000","last_undegraded":"2023-10-08T02:28:39.334159+0000","last_fullsized":"2023-10-08T02:28:39.334159+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:02:38.978566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331550+0000","last_change":"2023-10-08T02:28:39.327169+0000","last_active":"2023-10-08T02:28:39.331550+0000","last_peered":"2023-10-08T02:28:39.331550+0000","last_clean":"2023-10-08T02:28:39.331550+0000","last_became_active":"2023-10-08T02:28:39.326937+0000","last_became_peered":"2023-10-08T02:28:39.326937+0000","last_unstale":"2023-10-08T02:28:39.331550+0000","last_undegraded":"2023-10-08T02:28:39.331550+0000","last_fullsized":"2023-10-08T02:28:39.331550+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:06:42.843940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330219+0000","last_change":"2023-10-08T02:28:39.326643+0000","last_active":"2023-10-08T02:28:39.330219+0000","last_peered":"2023-10-08T02:28:39.330219+0000","last_clean":"2023-10-08T02:28:39.330219+0000","last_became_active":"2023-10-08T02:28:39.326309+0000","last_became_peered":"2023-10-08T02:28:39.326309+0000","last_unstale":"2023-10-08T02:28:39.330219+0000","last_undegraded":"2023-10-08T02:28:39.330219+0000","last_fullsized":"2023-10-08T02:28:39.330219+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:12.961726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.336299+0000","last_change":"2023-10-08T02:28:39.335145+0000","last_active":"2023-10-08T02:28:39.336299+0000","last_peered":"2023-10-08T02:28:39.336299+0000","last_clean":"2023-10-08T02:28:39.336299+0000","last_became_active":"2023-10-08T02:28:39.334957+0000","last_became_peered":"2023-10-08T02:28:39.334957+0000","last_unstale":"2023-10-08T02:28:39.336299+0000","last_undegraded":"2023-10-08T02:28:39.336299+0000","last_fullsized":"2023-10-08T02:28:39.336299+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:58.451514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330266+0000","last_change":"2023-10-08T02:28:39.326324+0000","last_active":"2023-10-08T02:28:39.330266+0000","last_peered":"2023-10-08T02:28:39.330266+0000","last_clean":"2023-10-08T02:28:39.330266+0000","last_became_active":"2023-10-08T02:28:39.326061+0000","last_became_peered":"2023-10-08T02:28:39.326061+0000","last_unstale":"2023-10-08T02:28:39.330266+0000","last_undegraded":"2023-10-08T02:28:39.330266+0000","last_fullsized":"2023-10-08T02:28:39.330266+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:19.633375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334559+0000","last_change":"2023-10-08T02:28:39.331317+0000","last_active":"2023-10-08T02:28:39.334559+0000","last_peered":"2023-10-08T02:28:39.334559+0000","last_clean":"2023-10-08T02:28:39.334559+0000","last_became_active":"2023-10-08T02:28:39.331010+0000","last_became_peered":"2023-10-08T02:28:39.331010+0000","last_unstale":"2023-10-08T02:28:39.334559+0000","last_undegraded":"2023-10-08T02:28:39.334559+0000","last_fullsized":"2023-10-08T02:28:39.334559+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:11:27.279209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.325920+0000","last_change":"2023-10-08T02:28:39.321428+0000","last_active":"2023-10-08T02:28:39.325920+0000","last_peered":"2023-10-08T02:28:39.325920+0000","last_clean":"2023-10-08T02:28:39.325920+0000","last_became_active":"2023-10-08T02:28:39.321169+0000","last_became_peered":"2023-10-08T02:28:39.321169+0000","last_unstale":"2023-10-08T02:28:39.325920+0000","last_undegraded":"2023-10-08T02:28:39.325920+0000","last_fullsized":"2023-10-08T02:28:39.325920+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:15.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.323681+0000","last_change":"2023-10-08T02:28:39.319024+0000","last_active":"2023-10-08T02:28:39.323681+0000","last_peered":"2023-10-08T02:28:39.323681+0000","last_clean":"2023-10-08T02:28:39.323681+0000","last_became_active":"2023-10-08T02:28:39.318785+0000","last_became_peered":"2023-10-08T02:28:39.318785+0000","last_unstale":"2023-10-08T02:28:39.323681+0000","last_undegraded":"2023-10-08T02:28:39.323681+0000","last_fullsized":"2023-10-08T02:28:39.323681+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:19.633375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331532+0000","last_change":"2023-10-08T02:28:39.326571+0000","last_active":"2023-10-08T02:28:39.331532+0000","last_peered":"2023-10-08T02:28:39.331532+0000","last_clean":"2023-10-08T02:28:39.331532+0000","last_became_active":"2023-10-08T02:28:39.326293+0000","last_became_peered":"2023-10-08T02:28:39.326293+0000","last_unstale":"2023-10-08T02:28:39.331532+0000","last_undegraded":"2023-10-08T02:28:39.331532+0000","last_fullsized":"2023-10-08T02:28:39.331532+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:35:03.725569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"190'4","reported_seq":14,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.819789+0000","last_change":"2023-10-08T02:28:38.828668+0000","last_active":"2023-10-08T02:28:39.819789+0000","last_peered":"2023-10-08T02:28:39.819789+0000","last_clean":"2023-10-08T02:28:39.819789+0000","last_became_active":"2023-10-08T02:28:38.828377+0000","last_became_peered":"2023-10-08T02:28:38.828377+0000","last_unstale":"2023-10-08T02:28:39.819789+0000","last_undegraded":"2023-10-08T02:28:39.819789+0000","last_fullsized":"2023-10-08T02:28:39.819789+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:57.443582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.320596+0000","last_change":"2023-10-08T02:28:39.314030+0000","last_active":"2023-10-08T02:28:39.320596+0000","last_peered":"2023-10-08T02:28:39.320596+0000","last_clean":"2023-10-08T02:28:39.320596+0000","last_became_active":"2023-10-08T02:28:39.313761+0000","last_became_peered":"2023-10-08T02:28:39.313761+0000","last_unstale":"2023-10-08T02:28:39.320596+0000","last_undegraded":"2023-10-08T02:28:39.320596+0000","last_fullsized":"2023-10-08T02:28:39.320596+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:30:33.987576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.819246+0000","last_change":"2023-10-08T02:28:38.828417+0000","last_active":"2023-10-08T02:28:39.819246+0000","last_peered":"2023-10-08T02:28:39.819246+0000","last_clean":"2023-10-08T02:28:39.819246+0000","last_became_active":"2023-10-08T02:28:38.828122+0000","last_became_peered":"2023-10-08T02:28:38.828122+0000","last_unstale":"2023-10-08T02:28:39.819246+0000","last_undegraded":"2023-10-08T02:28:39.819246+0000","last_fullsized":"2023-10-08T02:28:39.819246+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:43.348294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333686+0000","last_change":"2023-10-08T02:28:39.332737+0000","last_active":"2023-10-08T02:28:39.333686+0000","last_peered":"2023-10-08T02:28:39.333686+0000","last_clean":"2023-10-08T02:28:39.333686+0000","last_became_active":"2023-10-08T02:28:39.332497+0000","last_became_peered":"2023-10-08T02:28:39.332497+0000","last_unstale":"2023-10-08T02:28:39.333686+0000","last_undegraded":"2023-10-08T02:28:39.333686+0000","last_fullsized":"2023-10-08T02:28:39.333686+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:11:40.199040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.818094+0000","last_change":"2023-10-08T02:28:38.827826+0000","last_active":"2023-10-08T02:28:39.818094+0000","last_peered":"2023-10-08T02:28:39.818094+0000","last_clean":"2023-10-08T02:28:39.818094+0000","last_became_active":"2023-10-08T02:28:38.827526+0000","last_became_peered":"2023-10-08T02:28:38.827526+0000","last_unstale":"2023-10-08T02:28:39.818094+0000","last_undegraded":"2023-10-08T02:28:39.818094+0000","last_fullsized":"2023-10-08T02:28:39.818094+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:13:31.203917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.334149+0000","last_change":"2023-10-08T02:28:39.331172+0000","last_active":"2023-10-08T02:28:39.334149+0000","last_peered":"2023-10-08T02:28:39.334149+0000","last_clean":"2023-10-08T02:28:39.334149+0000","last_became_active":"2023-10-08T02:28:39.330901+0000","last_became_peered":"2023-10-08T02:28:39.330901+0000","last_unstale":"2023-10-08T02:28:39.334149+0000","last_undegraded":"2023-10-08T02:28:39.334149+0000","last_fullsized":"2023-10-08T02:28:39.334149+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:44.356227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.818699+0000","last_change":"2023-10-08T02:28:38.819324+0000","last_active":"2023-10-08T02:28:39.818699+0000","last_peered":"2023-10-08T02:28:39.818699+0000","last_clean":"2023-10-08T02:28:39.818699+0000","last_became_active":"2023-10-08T02:28:38.818979+0000","last_became_peered":"2023-10-08T02:28:38.818979+0000","last_unstale":"2023-10-08T02:28:39.818699+0000","last_undegraded":"2023-10-08T02:28:39.818699+0000","last_fullsized":"2023-10-08T02:28:39.818699+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:04.384684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331463+0000","last_change":"2023-10-08T02:28:39.327590+0000","last_active":"2023-10-08T02:28:39.331463+0000","last_peered":"2023-10-08T02:28:39.331463+0000","last_clean":"2023-10-08T02:28:39.331463+0000","last_became_active":"2023-10-08T02:28:39.327322+0000","last_became_peered":"2023-10-08T02:28:39.327322+0000","last_unstale":"2023-10-08T02:28:39.331463+0000","last_undegraded":"2023-10-08T02:28:39.331463+0000","last_fullsized":"2023-10-08T02:28:39.331463+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:26:41.547145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.817393+0000","last_change":"2023-10-08T02:28:38.826581+0000","last_active":"2023-10-08T02:28:39.817393+0000","last_peered":"2023-10-08T02:28:39.817393+0000","last_clean":"2023-10-08T02:28:39.817393+0000","last_became_active":"2023-10-08T02:28:38.826110+0000","last_became_peered":"2023-10-08T02:28:38.826110+0000","last_unstale":"2023-10-08T02:28:39.817393+0000","last_undegraded":"2023-10-08T02:28:39.817393+0000","last_fullsized":"2023-10-08T02:28:39.817393+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:03.125725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.336007+0000","last_change":"2023-10-08T02:28:39.331411+0000","last_active":"2023-10-08T02:28:39.336007+0000","last_peered":"2023-10-08T02:28:39.336007+0000","last_clean":"2023-10-08T02:28:39.336007+0000","last_became_active":"2023-10-08T02:28:39.331098+0000","last_became_peered":"2023-10-08T02:28:39.331098+0000","last_unstale":"2023-10-08T02:28:39.336007+0000","last_undegraded":"2023-10-08T02:28:39.336007+0000","last_fullsized":"2023-10-08T02:28:39.336007+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:19.633375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.814417+0000","last_change":"2023-10-08T02:28:38.822591+0000","last_active":"2023-10-08T02:28:39.814417+0000","last_peered":"2023-10-08T02:28:39.814417+0000","last_clean":"2023-10-08T02:28:39.814417+0000","last_became_active":"2023-10-08T02:28:38.822333+0000","last_became_peered":"2023-10-08T02:28:38.822333+0000","last_unstale":"2023-10-08T02:28:39.814417+0000","last_undegraded":"2023-10-08T02:28:39.814417+0000","last_fullsized":"2023-10-08T02:28:39.814417+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:08:41.416738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333350+0000","last_change":"2023-10-08T02:28:39.329032+0000","last_active":"2023-10-08T02:28:39.333350+0000","last_peered":"2023-10-08T02:28:39.333350+0000","last_clean":"2023-10-08T02:28:39.333350+0000","last_became_active":"2023-10-08T02:28:39.328798+0000","last_became_peered":"2023-10-08T02:28:39.328798+0000","last_unstale":"2023-10-08T02:28:39.333350+0000","last_undegraded":"2023-10-08T02:28:39.333350+0000","last_fullsized":"2023-10-08T02:28:39.333350+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:10:48.036897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.311488+0000","last_change":"2023-10-08T02:28:38.820931+0000","last_active":"2023-10-08T02:28:39.311488+0000","last_peered":"2023-10-08T02:28:39.311488+0000","last_clean":"2023-10-08T02:28:39.311488+0000","last_became_active":"2023-10-08T02:28:38.820618+0000","last_became_peered":"2023-10-08T02:28:38.820618+0000","last_unstale":"2023-10-08T02:28:39.311488+0000","last_undegraded":"2023-10-08T02:28:39.311488+0000","last_fullsized":"2023-10-08T02:28:39.311488+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:06:34.106471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327195+0000","last_change":"2023-10-08T02:28:39.322164+0000","last_active":"2023-10-08T02:28:39.327195+0000","last_peered":"2023-10-08T02:28:39.327195+0000","last_clean":"2023-10-08T02:28:39.327195+0000","last_became_active":"2023-10-08T02:28:39.321885+0000","last_became_peered":"2023-10-08T02:28:39.321885+0000","last_unstale":"2023-10-08T02:28:39.327195+0000","last_undegraded":"2023-10-08T02:28:39.327195+0000","last_fullsized":"2023-10-08T02:28:39.327195+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:45.277252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.316511+0000","last_change":"2023-10-08T02:28:38.813302+0000","last_active":"2023-10-08T02:28:39.316511+0000","last_peered":"2023-10-08T02:28:39.316511+0000","last_clean":"2023-10-08T02:28:39.316511+0000","last_became_active":"2023-10-08T02:28:38.812605+0000","last_became_peered":"2023-10-08T02:28:38.812605+0000","last_unstale":"2023-10-08T02:28:39.316511+0000","last_undegraded":"2023-10-08T02:28:39.316511+0000","last_fullsized":"2023-10-08T02:28:39.316511+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:40:43.714637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327852+0000","last_change":"2023-10-08T02:28:39.323313+0000","last_active":"2023-10-08T02:28:39.327852+0000","last_peered":"2023-10-08T02:28:39.327852+0000","last_clean":"2023-10-08T02:28:39.327852+0000","last_became_active":"2023-10-08T02:28:39.323024+0000","last_became_peered":"2023-10-08T02:28:39.323024+0000","last_unstale":"2023-10-08T02:28:39.327852+0000","last_undegraded":"2023-10-08T02:28:39.327852+0000","last_fullsized":"2023-10-08T02:28:39.327852+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:42:16.498144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.318436+0000","last_change":"2023-10-08T02:28:38.832015+0000","last_active":"2023-10-08T02:28:39.318436+0000","last_peered":"2023-10-08T02:28:39.318436+0000","last_clean":"2023-10-08T02:28:39.318436+0000","last_became_active":"2023-10-08T02:28:38.831734+0000","last_became_peered":"2023-10-08T02:28:38.831734+0000","last_unstale":"2023-10-08T02:28:39.318436+0000","last_undegraded":"2023-10-08T02:28:39.318436+0000","last_fullsized":"2023-10-08T02:28:39.318436+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:36:31.035881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.326751+0000","last_change":"2023-10-08T02:28:39.322958+0000","last_active":"2023-10-08T02:28:39.326751+0000","last_peered":"2023-10-08T02:28:39.326751+0000","last_clean":"2023-10-08T02:28:39.326751+0000","last_became_active":"2023-10-08T02:28:39.322646+0000","last_became_peered":"2023-10-08T02:28:39.322646+0000","last_unstale":"2023-10-08T02:28:39.326751+0000","last_undegraded":"2023-10-08T02:28:39.326751+0000","last_fullsized":"2023-10-08T02:28:39.326751+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:02:38.978566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"190'3","reported_seq":13,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.820154+0000","last_change":"2023-10-08T02:28:38.815648+0000","last_active":"2023-10-08T02:28:39.820154+0000","last_peered":"2023-10-08T02:28:39.820154+0000","last_clean":"2023-10-08T02:28:39.820154+0000","last_became_active":"2023-10-08T02:28:38.815397+0000","last_became_peered":"2023-10-08T02:28:38.815397+0000","last_unstale":"2023-10-08T02:28:39.820154+0000","last_undegraded":"2023-10-08T02:28:39.820154+0000","last_fullsized":"2023-10-08T02:28:39.820154+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:16:59.471200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.328467+0000","last_change":"2023-10-08T02:28:39.325322+0000","last_active":"2023-10-08T02:28:39.328467+0000","last_peered":"2023-10-08T02:28:39.328467+0000","last_clean":"2023-10-08T02:28:39.328467+0000","last_became_active":"2023-10-08T02:28:39.325020+0000","last_became_peered":"2023-10-08T02:28:39.325020+0000","last_unstale":"2023-10-08T02:28:39.328467+0000","last_undegraded":"2023-10-08T02:28:39.328467+0000","last_fullsized":"2023-10-08T02:28:39.328467+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:23.649613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"190'6","reported_seq":16,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.824162+0000","last_change":"2023-10-08T02:28:38.828063+0000","last_active":"2023-10-08T02:28:39.824162+0000","last_peered":"2023-10-08T02:28:39.824162+0000","last_clean":"2023-10-08T02:28:39.824162+0000","last_became_active":"2023-10-08T02:28:38.827800+0000","last_became_peered":"2023-10-08T02:28:38.827800+0000","last_unstale":"2023-10-08T02:28:39.824162+0000","last_undegraded":"2023-10-08T02:28:39.824162+0000","last_fullsized":"2023-10-08T02:28:39.824162+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:11.953794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.323763+0000","last_change":"2023-10-08T02:28:39.318203+0000","last_active":"2023-10-08T02:28:39.323763+0000","last_peered":"2023-10-08T02:28:39.323763+0000","last_clean":"2023-10-08T02:28:39.323763+0000","last_became_active":"2023-10-08T02:28:39.317553+0000","last_became_peered":"2023-10-08T02:28:39.317553+0000","last_unstale":"2023-10-08T02:28:39.323763+0000","last_undegraded":"2023-10-08T02:28:39.323763+0000","last_fullsized":"2023-10-08T02:28:39.323763+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:02:54.426138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.816833+0000","last_change":"2023-10-08T02:28:38.819154+0000","last_active":"2023-10-08T02:28:39.816833+0000","last_peered":"2023-10-08T02:28:39.816833+0000","last_clean":"2023-10-08T02:28:39.816833+0000","last_became_active":"2023-10-08T02:28:38.818890+0000","last_became_peered":"2023-10-08T02:28:38.818890+0000","last_unstale":"2023-10-08T02:28:39.816833+0000","last_undegraded":"2023-10-08T02:28:39.816833+0000","last_fullsized":"2023-10-08T02:28:39.816833+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:25.660989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330982+0000","last_change":"2023-10-08T02:28:39.325439+0000","last_active":"2023-10-08T02:28:39.330982+0000","last_peered":"2023-10-08T02:28:39.330982+0000","last_clean":"2023-10-08T02:28:39.330982+0000","last_became_active":"2023-10-08T02:28:39.324973+0000","last_became_peered":"2023-10-08T02:28:39.324973+0000","last_unstale":"2023-10-08T02:28:39.330982+0000","last_undegraded":"2023-10-08T02:28:39.330982+0000","last_fullsized":"2023-10-08T02:28:39.330982+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:37:30.449689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.310513+0000","last_change":"2023-10-08T02:28:38.823806+0000","last_active":"2023-10-08T02:28:39.310513+0000","last_peered":"2023-10-08T02:28:39.310513+0000","last_clean":"2023-10-08T02:28:39.310513+0000","last_became_active":"2023-10-08T02:28:38.823556+0000","last_became_peered":"2023-10-08T02:28:38.823556+0000","last_unstale":"2023-10-08T02:28:39.310513+0000","last_undegraded":"2023-10-08T02:28:39.310513+0000","last_fullsized":"2023-10-08T02:28:39.310513+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:12:44.269320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.333423+0000","last_change":"2023-10-08T02:28:39.329983+0000","last_active":"2023-10-08T02:28:39.333423+0000","last_peered":"2023-10-08T02:28:39.333423+0000","last_clean":"2023-10-08T02:28:39.333423+0000","last_became_active":"2023-10-08T02:28:39.329722+0000","last_became_peered":"2023-10-08T02:28:39.329722+0000","last_unstale":"2023-10-08T02:28:39.333423+0000","last_undegraded":"2023-10-08T02:28:39.333423+0000","last_fullsized":"2023-10-08T02:28:39.333423+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:12.961726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.315187+0000","last_change":"2023-10-08T02:28:38.824251+0000","last_active":"2023-10-08T02:28:39.315187+0000","last_peered":"2023-10-08T02:28:39.315187+0000","last_clean":"2023-10-08T02:28:39.315187+0000","last_became_active":"2023-10-08T02:28:38.823920+0000","last_became_peered":"2023-10-08T02:28:38.823920+0000","last_unstale":"2023-10-08T02:28:39.315187+0000","last_undegraded":"2023-10-08T02:28:39.315187+0000","last_fullsized":"2023-10-08T02:28:39.315187+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:37:31.497164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329128+0000","last_change":"2023-10-08T02:28:39.323986+0000","last_active":"2023-10-08T02:28:39.329128+0000","last_peered":"2023-10-08T02:28:39.329128+0000","last_clean":"2023-10-08T02:28:39.329128+0000","last_became_active":"2023-10-08T02:28:39.323715+0000","last_became_peered":"2023-10-08T02:28:39.323715+0000","last_unstale":"2023-10-08T02:28:39.329128+0000","last_undegraded":"2023-10-08T02:28:39.329128+0000","last_fullsized":"2023-10-08T02:28:39.329128+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:09.648666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"190'8","reported_seq":18,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.822052+0000","last_change":"2023-10-08T02:28:38.828752+0000","last_active":"2023-10-08T02:28:39.822052+0000","last_peered":"2023-10-08T02:28:39.822052+0000","last_clean":"2023-10-08T02:28:39.822052+0000","last_became_active":"2023-10-08T02:28:38.828482+0000","last_became_peered":"2023-10-08T02:28:38.828482+0000","last_unstale":"2023-10-08T02:28:39.822052+0000","last_undegraded":"2023-10-08T02:28:39.822052+0000","last_fullsized":"2023-10-08T02:28:39.822052+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:08.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.328162+0000","last_change":"2023-10-08T02:28:39.324786+0000","last_active":"2023-10-08T02:28:39.328162+0000","last_peered":"2023-10-08T02:28:39.328162+0000","last_clean":"2023-10-08T02:28:39.328162+0000","last_became_active":"2023-10-08T02:28:39.324533+0000","last_became_peered":"2023-10-08T02:28:39.324533+0000","last_unstale":"2023-10-08T02:28:39.328162+0000","last_undegraded":"2023-10-08T02:28:39.328162+0000","last_fullsized":"2023-10-08T02:28:39.328162+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:48:44.356227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.818330+0000","last_change":"2023-10-08T02:28:38.829032+0000","last_active":"2023-10-08T02:28:39.818330+0000","last_peered":"2023-10-08T02:28:39.818330+0000","last_clean":"2023-10-08T02:28:39.818330+0000","last_became_active":"2023-10-08T02:28:38.828787+0000","last_became_peered":"2023-10-08T02:28:38.828787+0000","last_unstale":"2023-10-08T02:28:39.818330+0000","last_undegraded":"2023-10-08T02:28:39.818330+0000","last_fullsized":"2023-10-08T02:28:39.818330+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:28:04.384684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.320862+0000","last_change":"2023-10-08T02:28:39.317037+0000","last_active":"2023-10-08T02:28:39.320862+0000","last_peered":"2023-10-08T02:28:39.320862+0000","last_clean":"2023-10-08T02:28:39.320862+0000","last_became_active":"2023-10-08T02:28:39.316755+0000","last_became_peered":"2023-10-08T02:28:39.316755+0000","last_unstale":"2023-10-08T02:28:39.320862+0000","last_undegraded":"2023-10-08T02:28:39.320862+0000","last_fullsized":"2023-10-08T02:28:39.320862+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:05:15.646344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.817487+0000","last_change":"2023-10-08T02:28:38.830576+0000","last_active":"2023-10-08T02:28:39.817487+0000","last_peered":"2023-10-08T02:28:39.817487+0000","last_clean":"2023-10-08T02:28:39.817487+0000","last_became_active":"2023-10-08T02:28:38.829916+0000","last_became_peered":"2023-10-08T02:28:38.829916+0000","last_unstale":"2023-10-08T02:28:39.817487+0000","last_undegraded":"2023-10-08T02:28:39.817487+0000","last_fullsized":"2023-10-08T02:28:39.817487+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:07.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.336015+0000","last_change":"2023-10-08T02:28:39.331448+0000","last_active":"2023-10-08T02:28:39.336015+0000","last_peered":"2023-10-08T02:28:39.336015+0000","last_clean":"2023-10-08T02:28:39.336015+0000","last_became_active":"2023-10-08T02:28:39.331154+0000","last_became_peered":"2023-10-08T02:28:39.331154+0000","last_unstale":"2023-10-08T02:28:39.336015+0000","last_undegraded":"2023-10-08T02:28:39.336015+0000","last_fullsized":"2023-10-08T02:28:39.336015+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:30.962913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"190'1","reported_seq":11,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.820105+0000","last_change":"2023-10-08T02:28:38.823230+0000","last_active":"2023-10-08T02:28:39.820105+0000","last_peered":"2023-10-08T02:28:39.820105+0000","last_clean":"2023-10-08T02:28:39.820105+0000","last_became_active":"2023-10-08T02:28:38.822934+0000","last_became_peered":"2023-10-08T02:28:38.822934+0000","last_unstale":"2023-10-08T02:28:39.820105+0000","last_undegraded":"2023-10-08T02:28:39.820105+0000","last_fullsized":"2023-10-08T02:28:39.820105+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:10:08.640733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330998+0000","last_change":"2023-10-08T02:28:39.325748+0000","last_active":"2023-10-08T02:28:39.330998+0000","last_peered":"2023-10-08T02:28:39.330998+0000","last_clean":"2023-10-08T02:28:39.330998+0000","last_became_active":"2023-10-08T02:28:39.325507+0000","last_became_peered":"2023-10-08T02:28:39.325507+0000","last_unstale":"2023-10-08T02:28:39.330998+0000","last_undegraded":"2023-10-08T02:28:39.330998+0000","last_fullsized":"2023-10-08T02:28:39.330998+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:18:58.907072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.317442+0000","last_change":"2023-10-08T02:28:38.819566+0000","last_active":"2023-10-08T02:28:39.317442+0000","last_peered":"2023-10-08T02:28:39.317442+0000","last_clean":"2023-10-08T02:28:39.317442+0000","last_became_active":"2023-10-08T02:28:38.819144+0000","last_became_peered":"2023-10-08T02:28:38.819144+0000","last_unstale":"2023-10-08T02:28:39.317442+0000","last_undegraded":"2023-10-08T02:28:39.317442+0000","last_fullsized":"2023-10-08T02:28:39.317442+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:29.954980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329795+0000","last_change":"2023-10-08T02:28:39.325608+0000","last_active":"2023-10-08T02:28:39.329795+0000","last_peered":"2023-10-08T02:28:39.329795+0000","last_clean":"2023-10-08T02:28:39.329795+0000","last_became_active":"2023-10-08T02:28:39.325349+0000","last_became_peered":"2023-10-08T02:28:39.325349+0000","last_unstale":"2023-10-08T02:28:39.329795+0000","last_undegraded":"2023-10-08T02:28:39.329795+0000","last_fullsized":"2023-10-08T02:28:39.329795+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:47.883744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.818207+0000","last_change":"2023-10-08T02:28:38.824280+0000","last_active":"2023-10-08T02:28:39.818207+0000","last_peered":"2023-10-08T02:28:39.818207+0000","last_clean":"2023-10-08T02:28:39.818207+0000","last_became_active":"2023-10-08T02:28:38.823999+0000","last_became_peered":"2023-10-08T02:28:38.823999+0000","last_unstale":"2023-10-08T02:28:39.818207+0000","last_undegraded":"2023-10-08T02:28:39.818207+0000","last_fullsized":"2023-10-08T02:28:39.818207+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:57:07.010576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.330961+0000","last_change":"2023-10-08T02:28:39.324643+0000","last_active":"2023-10-08T02:28:39.330961+0000","last_peered":"2023-10-08T02:28:39.330961+0000","last_clean":"2023-10-08T02:28:39.330961+0000","last_became_active":"2023-10-08T02:28:39.324393+0000","last_became_peered":"2023-10-08T02:28:39.324393+0000","last_unstale":"2023-10-08T02:28:39.330961+0000","last_undegraded":"2023-10-08T02:28:39.330961+0000","last_fullsized":"2023-10-08T02:28:39.330961+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:38:38.883411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.318229+0000","last_change":"2023-10-08T02:28:38.824325+0000","last_active":"2023-10-08T02:28:39.318229+0000","last_peered":"2023-10-08T02:28:39.318229+0000","last_clean":"2023-10-08T02:28:39.318229+0000","last_became_active":"2023-10-08T02:28:38.823161+0000","last_became_peered":"2023-10-08T02:28:38.823161+0000","last_unstale":"2023-10-08T02:28:39.318229+0000","last_undegraded":"2023-10-08T02:28:39.318229+0000","last_fullsized":"2023-10-08T02:28:39.318229+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:18.625443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.337547+0000","last_change":"2023-10-08T02:28:39.334542+0000","last_active":"2023-10-08T02:28:39.337547+0000","last_peered":"2023-10-08T02:28:39.337547+0000","last_clean":"2023-10-08T02:28:39.337547+0000","last_became_active":"2023-10-08T02:28:39.334137+0000","last_became_peered":"2023-10-08T02:28:39.334137+0000","last_unstale":"2023-10-08T02:28:39.337547+0000","last_undegraded":"2023-10-08T02:28:39.337547+0000","last_fullsized":"2023-10-08T02:28:39.337547+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:02:54.426138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.315308+0000","last_change":"2023-10-08T02:28:38.829914+0000","last_active":"2023-10-08T02:28:39.315308+0000","last_peered":"2023-10-08T02:28:39.315308+0000","last_clean":"2023-10-08T02:28:39.315308+0000","last_became_active":"2023-10-08T02:28:38.829662+0000","last_became_peered":"2023-10-08T02:28:38.829662+0000","last_unstale":"2023-10-08T02:28:39.315308+0000","last_undegraded":"2023-10-08T02:28:39.315308+0000","last_fullsized":"2023-10-08T02:28:39.315308+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:54.799537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.319175+0000","last_change":"2023-10-08T02:28:39.317508+0000","last_active":"2023-10-08T02:28:39.319175+0000","last_peered":"2023-10-08T02:28:39.319175+0000","last_clean":"2023-10-08T02:28:39.319175+0000","last_became_active":"2023-10-08T02:28:39.317242+0000","last_became_peered":"2023-10-08T02:28:39.317242+0000","last_unstale":"2023-10-08T02:28:39.319175+0000","last_undegraded":"2023-10-08T02:28:39.319175+0000","last_fullsized":"2023-10-08T02:28:39.319175+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:34.534831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.316246+0000","last_change":"2023-10-08T02:28:38.823623+0000","last_active":"2023-10-08T02:28:39.316246+0000","last_peered":"2023-10-08T02:28:39.316246+0000","last_clean":"2023-10-08T02:28:39.316246+0000","last_became_active":"2023-10-08T02:28:38.823343+0000","last_became_peered":"2023-10-08T02:28:38.823343+0000","last_unstale":"2023-10-08T02:28:39.316246+0000","last_undegraded":"2023-10-08T02:28:39.316246+0000","last_fullsized":"2023-10-08T02:28:39.316246+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:11.953794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327961+0000","last_change":"2023-10-08T02:28:39.324640+0000","last_active":"2023-10-08T02:28:39.327961+0000","last_peered":"2023-10-08T02:28:39.327961+0000","last_clean":"2023-10-08T02:28:39.327961+0000","last_became_active":"2023-10-08T02:28:39.324382+0000","last_became_peered":"2023-10-08T02:28:39.324382+0000","last_unstale":"2023-10-08T02:28:39.327961+0000","last_undegraded":"2023-10-08T02:28:39.327961+0000","last_fullsized":"2023-10-08T02:28:39.327961+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:01.177379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"190'2","reported_seq":12,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.819346+0000","last_change":"2023-10-08T02:28:38.823012+0000","last_active":"2023-10-08T02:28:39.819346+0000","last_peered":"2023-10-08T02:28:39.819346+0000","last_clean":"2023-10-08T02:28:39.819346+0000","last_became_active":"2023-10-08T02:28:38.822745+0000","last_became_peered":"2023-10-08T02:28:38.822745+0000","last_unstale":"2023-10-08T02:28:39.819346+0000","last_undegraded":"2023-10-08T02:28:39.819346+0000","last_fullsized":"2023-10-08T02:28:39.819346+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:59:25.660989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329737+0000","last_change":"2023-10-08T02:28:39.321822+0000","last_active":"2023-10-08T02:28:39.329737+0000","last_peered":"2023-10-08T02:28:39.329737+0000","last_clean":"2023-10-08T02:28:39.329737+0000","last_became_active":"2023-10-08T02:28:39.321572+0000","last_became_peered":"2023-10-08T02:28:39.321572+0000","last_unstale":"2023-10-08T02:28:39.329737+0000","last_undegraded":"2023-10-08T02:28:39.329737+0000","last_fullsized":"2023-10-08T02:28:39.329737+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:26:17.762885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.318968+0000","last_change":"2023-10-08T02:28:38.830771+0000","last_active":"2023-10-08T02:28:39.318968+0000","last_peered":"2023-10-08T02:28:39.318968+0000","last_clean":"2023-10-08T02:28:39.318968+0000","last_became_active":"2023-10-08T02:28:38.830279+0000","last_became_peered":"2023-10-08T02:28:38.830279+0000","last_unstale":"2023-10-08T02:28:39.318968+0000","last_undegraded":"2023-10-08T02:28:39.318968+0000","last_fullsized":"2023-10-08T02:28:39.318968+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:54.799537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.329539+0000","last_change":"2023-10-08T02:28:39.324486+0000","last_active":"2023-10-08T02:28:39.329539+0000","last_peered":"2023-10-08T02:28:39.329539+0000","last_clean":"2023-10-08T02:28:39.329539+0000","last_became_active":"2023-10-08T02:28:39.323525+0000","last_became_peered":"2023-10-08T02:28:39.323525+0000","last_unstale":"2023-10-08T02:28:39.329539+0000","last_undegraded":"2023-10-08T02:28:39.329539+0000","last_fullsized":"2023-10-08T02:28:39.329539+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:54:03.109079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.312307+0000","last_change":"2023-10-08T02:28:38.818993+0000","last_active":"2023-10-08T02:28:39.312307+0000","last_peered":"2023-10-08T02:28:39.312307+0000","last_clean":"2023-10-08T02:28:39.312307+0000","last_became_active":"2023-10-08T02:28:38.818679+0000","last_became_peered":"2023-10-08T02:28:38.818679+0000","last_unstale":"2023-10-08T02:28:39.312307+0000","last_undegraded":"2023-10-08T02:28:39.312307+0000","last_fullsized":"2023-10-08T02:28:39.312307+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:12:45.471846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.331794+0000","last_change":"2023-10-08T02:28:39.329653+0000","last_active":"2023-10-08T02:28:39.331794+0000","last_peered":"2023-10-08T02:28:39.331794+0000","last_clean":"2023-10-08T02:28:39.331794+0000","last_became_active":"2023-10-08T02:28:39.329353+0000","last_became_peered":"2023-10-08T02:28:39.329353+0000","last_unstale":"2023-10-08T02:28:39.331794+0000","last_undegraded":"2023-10-08T02:28:39.331794+0000","last_fullsized":"2023-10-08T02:28:39.331794+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:01:42.051410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.309944+0000","last_change":"2023-10-08T02:28:38.823266+0000","last_active":"2023-10-08T02:28:39.309944+0000","last_peered":"2023-10-08T02:28:39.309944+0000","last_clean":"2023-10-08T02:28:39.309944+0000","last_became_active":"2023-10-08T02:28:38.822984+0000","last_became_peered":"2023-10-08T02:28:38.822984+0000","last_unstale":"2023-10-08T02:28:39.309944+0000","last_undegraded":"2023-10-08T02:28:39.309944+0000","last_fullsized":"2023-10-08T02:28:39.309944+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:57:34.443528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.323781+0000","last_change":"2023-10-08T02:28:39.319795+0000","last_active":"2023-10-08T02:28:39.323781+0000","last_peered":"2023-10-08T02:28:39.323781+0000","last_clean":"2023-10-08T02:28:39.323781+0000","last_became_active":"2023-10-08T02:28:39.319551+0000","last_became_peered":"2023-10-08T02:28:39.319551+0000","last_unstale":"2023-10-08T02:28:39.323781+0000","last_undegraded":"2023-10-08T02:28:39.323781+0000","last_fullsized":"2023-10-08T02:28:39.323781+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:49:12.961726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.310037+0000","last_change":"2023-10-08T02:28:38.824771+0000","last_active":"2023-10-08T02:28:39.310037+0000","last_peered":"2023-10-08T02:28:39.310037+0000","last_clean":"2023-10-08T02:28:39.310037+0000","last_became_active":"2023-10-08T02:28:38.821862+0000","last_became_peered":"2023-10-08T02:28:38.821862+0000","last_unstale":"2023-10-08T02:28:39.310037+0000","last_undegraded":"2023-10-08T02:28:39.310037+0000","last_fullsized":"2023-10-08T02:28:39.310037+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:42:57.443582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.328719+0000","last_change":"2023-10-08T02:28:39.323286+0000","last_active":"2023-10-08T02:28:39.328719+0000","last_peered":"2023-10-08T02:28:39.328719+0000","last_clean":"2023-10-08T02:28:39.328719+0000","last_became_active":"2023-10-08T02:28:39.323031+0000","last_became_peered":"2023-10-08T02:28:39.323031+0000","last_unstale":"2023-10-08T02:28:39.328719+0000","last_undegraded":"2023-10-08T02:28:39.328719+0000","last_fullsized":"2023-10-08T02:28:39.328719+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:01:08.648973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.315573+0000","last_change":"2023-10-08T02:28:38.822167+0000","last_active":"2023-10-08T02:28:39.315573+0000","last_peered":"2023-10-08T02:28:39.315573+0000","last_clean":"2023-10-08T02:28:39.315573+0000","last_became_active":"2023-10-08T02:28:38.821888+0000","last_became_peered":"2023-10-08T02:28:38.821888+0000","last_unstale":"2023-10-08T02:28:39.315573+0000","last_undegraded":"2023-10-08T02:28:39.315573+0000","last_fullsized":"2023-10-08T02:28:39.315573+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:58:03.125725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.335828+0000","last_change":"2023-10-08T02:28:39.330719+0000","last_active":"2023-10-08T02:28:39.335828+0000","last_peered":"2023-10-08T02:28:39.335828+0000","last_clean":"2023-10-08T02:28:39.335828+0000","last_became_active":"2023-10-08T02:28:39.330455+0000","last_became_peered":"2023-10-08T02:28:39.330455+0000","last_unstale":"2023-10-08T02:28:39.335828+0000","last_undegraded":"2023-10-08T02:28:39.335828+0000","last_fullsized":"2023-10-08T02:28:39.335828+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:24:46.354643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.311510+0000","last_change":"2023-10-08T02:28:38.818228+0000","last_active":"2023-10-08T02:28:39.311510+0000","last_peered":"2023-10-08T02:28:39.311510+0000","last_clean":"2023-10-08T02:28:39.311510+0000","last_became_active":"2023-10-08T02:28:38.817959+0000","last_became_peered":"2023-10-08T02:28:38.817959+0000","last_unstale":"2023-10-08T02:28:39.311510+0000","last_undegraded":"2023-10-08T02:28:39.311510+0000","last_fullsized":"2023-10-08T02:28:39.311510+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:55:51.289187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327630+0000","last_change":"2023-10-08T02:28:39.322512+0000","last_active":"2023-10-08T02:28:39.327630+0000","last_peered":"2023-10-08T02:28:39.327630+0000","last_clean":"2023-10-08T02:28:39.327630+0000","last_became_active":"2023-10-08T02:28:39.321857+0000","last_became_peered":"2023-10-08T02:28:39.321857+0000","last_unstale":"2023-10-08T02:28:39.327630+0000","last_undegraded":"2023-10-08T02:28:39.327630+0000","last_fullsized":"2023-10-08T02:28:39.327630+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:30.962913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.310028+0000","last_change":"2023-10-08T02:28:38.820762+0000","last_active":"2023-10-08T02:28:39.310028+0000","last_peered":"2023-10-08T02:28:39.310028+0000","last_clean":"2023-10-08T02:28:39.310028+0000","last_became_active":"2023-10-08T02:28:38.820439+0000","last_became_peered":"2023-10-08T02:28:38.820439+0000","last_unstale":"2023-10-08T02:28:39.310028+0000","last_undegraded":"2023-10-08T02:28:39.310028+0000","last_fullsized":"2023-10-08T02:28:39.310028+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:15:18.625443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.327982+0000","last_change":"2023-10-08T02:28:39.325300+0000","last_active":"2023-10-08T02:28:39.327982+0000","last_peered":"2023-10-08T02:28:39.327982+0000","last_clean":"2023-10-08T02:28:39.327982+0000","last_became_active":"2023-10-08T02:28:39.325069+0000","last_became_peered":"2023-10-08T02:28:39.325069+0000","last_unstale":"2023-10-08T02:28:39.327982+0000","last_undegraded":"2023-10-08T02:28:39.327982+0000","last_fullsized":"2023-10-08T02:28:39.327982+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:38.795054+0000","last_clean_scrub_stamp":"2023-10-08T02:28:38.795054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:16:53.962266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.310305+0000","last_change":"2023-10-08T02:28:38.826218+0000","last_active":"2023-10-08T02:28:39.310305+0000","last_peered":"2023-10-08T02:28:39.310305+0000","last_clean":"2023-10-08T02:28:39.310305+0000","last_became_active":"2023-10-08T02:28:38.824990+0000","last_became_peered":"2023-10-08T02:28:38.824990+0000","last_unstale":"2023-10-08T02:28:39.310305+0000","last_undegraded":"2023-10-08T02:28:39.310305+0000","last_fullsized":"2023-10-08T02:28:39.310305+0000","mapping_epoch":188,"log_start":"0'0","ondisk_log_start":"0'0","created":188,"last_epoch_clean":189,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:28:37.787122+0000","last_clean_scrub_stamp":"2023-10-08T02:28:37.787122+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:57:29.954980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"180'326","reported_seq":1048,"reported_epoch":190,"state":"active+clean","last_fresh":"2023-10-08T02:28:39.316333+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:28:39.316333+0000","last_peered":"2023-10-08T02:28:39.316333+0000","last_clean":"2023-10-08T02:28:39.316333+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:28:39.316333+0000","last_undegraded":"2023-10-08T02:28:39.316333+0000","last_fullsized":"2023-10-08T02:28:39.316333+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1940,"num_read_kb":4014,"num_write":539,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":326,"ondisk_log_size":326,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312512,"kb_used_data":2936,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059328,"statfs":{"total":96636764160,"available":96316751872,"internally_reserved":0,"allocated":3006464,"data_stored":1795310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.748,"15min":0.73699999999999999},"min":{"1min":0.51500000000000001,"5min":0.46400000000000002,"15min":0.40699999999999997},"max":{"1min":0.91800000000000004,"5min":0.94999999999999996,"15min":1.0429999999999999},"last":0.629},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.74299999999999999},"min":{"1min":0.51700000000000002,"5min":0.51200000000000001,"15min":0.48099999999999998},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.052},"last":0.53900000000000003}]},{"osd":1,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.61499999999999999,"5min":0.47499999999999998,"15min":0.40500000000000003},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.73399999999999999,"15min":0.73799999999999999},"min":{"1min":0.61499999999999999,"5min":0.47699999999999998,"15min":0.42499999999999999},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.0609999999999999},"last":0.68799999999999994}]},{"osd":2,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.755,"15min":0.75700000000000001},"min":{"1min":0.57599999999999996,"5min":0.54500000000000004,"15min":0.495},"max":{"1min":0.91000000000000003,"5min":1.01,"15min":1.0549999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.60299999999999998,"5min":0.503,"15min":0.47199999999999998},"max":{"1min":0.92900000000000005,"5min":1.03,"15min":1.03},"last":0.78800000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71999999999999997,"15min":0.71399999999999997},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.46100000000000002},"max":{"1min":0.872,"5min":0.91000000000000003,"15min":0.96399999999999997},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79600000000000004,"15min":0.78600000000000003},"min":{"1min":0.625,"5min":0.60899999999999999,"15min":0.51300000000000001},"max":{"1min":0.90300000000000002,"5min":1.056,"15min":1.056},"last":0.80300000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.62,"15min":0.60799999999999998},"min":{"1min":0.46899999999999997,"5min":0.40300000000000002,"15min":0.34399999999999997},"max":{"1min":0.85099999999999998,"5min":1.091,"15min":1.091},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.622,"15min":0.61199999999999999},"min":{"1min":0.46400000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.76900000000000002,"5min":0.92200000000000004,"15min":0.95699999999999996},"last":0.755}]},{"osd":5,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.65400000000000003,"15min":0.64600000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.39300000000000002},"max":{"1min":0.77300000000000002,"5min":1.012,"15min":1.1399999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.68200000000000005,"15min":0.66800000000000004},"min":{"1min":0.51200000000000001,"5min":0.47699999999999998,"15min":0.39400000000000002},"max":{"1min":0.88,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Sun Oct 8 02:27:44 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.78700000000000003,"15min":0.77400000000000002},"min":{"1min":0.60699999999999998,"5min":0.54500000000000004,"15min":0.433},"max":{"1min":0.95999999999999996,"5min":1.151,"15min":1.151},"last":0.97399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.72599999999999998,"15min":0.71499999999999997},"min":{"1min":0.52600000000000002,"5min":0.47799999999999998,"15min":0.42599999999999999},"max":{"1min":0.85999999999999999,"5min":1.004,"15min":1.0329999999999999},"last":0.84099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575953,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299624,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072216,"statfs":{"total":96636764160,"available":96329949184,"internally_reserved":0,"allocated":2392064,"data_stored":1202714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.73199999999999998,"15min":0.73399999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.443},"max":{"1min":0.91100000000000003,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.75,"15min":0.74399999999999999},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.41799999999999998},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.40799999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.746,"15min":0.749},"min":{"1min":0.61799999999999999,"5min":0.51100000000000001,"15min":0.45300000000000001},"max":{"1min":0.83799999999999997,"5min":0.97799999999999998,"15min":1.21},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.92800000000000005,"5min":0.78200000000000003,"15min":0.75800000000000001},"min":{"1min":0.60399999999999998,"5min":0.54700000000000004,"15min":0.45700000000000002},"max":{"1min":3.851,"5min":3.851,"15min":3.851},"last":0.54600000000000004}]},{"osd":2,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.60399999999999998,"5min":0.498,"15min":0.44400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.755,"5min":0.76600000000000001,"15min":0.77000000000000002},"min":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.51700000000000002},"max":{"1min":0.87,"5min":1.002,"15min":1.258},"last":0.60899999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.81100000000000005,"15min":0.79600000000000004},"min":{"1min":0.64100000000000001,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":2.6970000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.871,"5min":0.79900000000000004,"15min":0.78700000000000003},"min":{"1min":0.58499999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.46200000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60999999999999999,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.378,"15min":0.376},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64500000000000002,"15min":0.63900000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.42599999999999999},"max":{"1min":1.341,"5min":1.341,"15min":1.341},"last":0.50800000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.625},"min":{"1min":0.45600000000000002,"5min":0.439,"15min":0.439},"max":{"1min":0.82799999999999996,"5min":0.86299999999999999,"15min":0.92800000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.49199999999999999,"5min":0.443,"15min":0.443},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:28:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71499999999999997,"15min":0.72399999999999998},"min":{"1min":0.56299999999999994,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85299999999999998,"5min":1.038,"15min":1.0980000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.755,"5min":0.75700000000000001,"15min":0.751},"min":{"1min":0.56599999999999995,"5min":0.52000000000000002,"15min":0.48599999999999999},"max":{"1min":0.86499999999999999,"5min":0.97399999999999998,"15min":1.054},"last":0.55400000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575953,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311912,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059928,"statfs":{"total":96636764160,"available":96317366272,"internally_reserved":0,"allocated":2392064,"data_stored":1202726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47199999999999998,"15min":0.48599999999999999},"min":{"1min":0.32600000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63600000000000001,"5min":0.70299999999999996,"15min":1.0780000000000001},"last":0.503},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51700000000000002,"15min":0.53500000000000003},"min":{"1min":0.35499999999999998,"5min":0.314,"15min":0.26500000000000001},"max":{"1min":0.70099999999999996,"5min":0.79100000000000004,"15min":1.571},"last":0.38800000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.49199999999999999,"15min":0.501},"min":{"1min":0.36799999999999999,"5min":0.35699999999999998,"15min":0.318},"max":{"1min":0.624,"5min":0.70899999999999996,"15min":1.1100000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.36699999999999999},"max":{"1min":0.71799999999999997,"5min":0.85899999999999999,"15min":1.1299999999999999},"last":0.42099999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59899999999999998,"15min":0.60299999999999998},"min":{"1min":0.45700000000000002,"5min":0.41999999999999998,"15min":0.40699999999999997},"max":{"1min":0.78600000000000003,"5min":0.875,"15min":1.2190000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.56699999999999995,"15min":0.56899999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.34100000000000003},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.99099999999999999},"last":0.435}]},{"osd":4,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.78400000000000003,"15min":0.78500000000000003},"min":{"1min":0.61099999999999999,"5min":0.53500000000000003,"15min":0.48199999999999998},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.40600000000000003},"max":{"1min":1.05,"5min":1.1539999999999999,"15min":1.1950000000000001},"last":0.75}]},{"osd":5,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77700000000000002,"15min":0.79100000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.0509999999999999,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81799999999999995,"15min":0.82099999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.0409999999999999,"5min":1.127,"15min":1.27},"last":0.59099999999999997}]},{"osd":6,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.79700000000000004,"15min":0.81299999999999994},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.002,"5min":1.141,"15min":1.141},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81999999999999995,"15min":0.82199999999999995},"min":{"1min":0.60999999999999999,"5min":0.57099999999999995,"15min":0.50900000000000001},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.204},"last":0.72999999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:28:22 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.85999999999999999,"15min":0.86599999999999999},"min":{"1min":0.64000000000000001,"5min":0.61899999999999999,"15min":0.61199999999999999},"max":{"1min":1.109,"5min":1.1359999999999999,"15min":1.306},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.875,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.66600000000000004,"5min":0.58399999999999996,"15min":0.55500000000000005},"max":{"1min":1.0629999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.626}]}]},{"osd":0,"up_from":13,"seq":55834575952,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308392,"kb_used_data":2912,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063448,"statfs":{"total":96636764160,"available":96320970752,"internally_reserved":0,"allocated":2981888,"data_stored":1793060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.47599999999999998,"15min":0.47799999999999998},"min":{"1min":0.32200000000000001,"5min":0.31,"15min":0.23699999999999999},"max":{"1min":0.61499999999999999,"5min":0.73599999999999999,"15min":0.80300000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.51300000000000001,"15min":0.52200000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.70899999999999996,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.314}]},{"osd":2,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.48199999999999998,"15min":0.48799999999999999},"min":{"1min":0.35399999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.75800000000000001,"5min":0.77800000000000002,"15min":0.84399999999999997},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.501,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.41299999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.69899999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.45200000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55200000000000005,"15min":0.56200000000000006},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36299999999999999},"max":{"1min":0.78400000000000003,"5min":0.78500000000000003,"15min":0.93100000000000005},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80400000000000005,"5min":0.83199999999999996,"15min":0.89000000000000001},"last":0.53900000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.78500000000000003,"15min":0.79300000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.53800000000000003},"max":{"1min":1.0700000000000001,"5min":1.232,"15min":1.232},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.80300000000000005,"15min":0.80100000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.52500000000000002},"max":{"1min":0.97199999999999998,"5min":1.262,"15min":1.262},"last":0.97199999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.83999999999999997,"15min":0.84299999999999997},"min":{"1min":0.65400000000000003,"5min":0.61499999999999999,"15min":0.55700000000000005},"max":{"1min":1.008,"5min":1.1990000000000001,"15min":1.216},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.873,"15min":0.876},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.58399999999999996},"max":{"1min":1.089,"5min":1.296,"15min":1.296},"last":0.88400000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.83299999999999996},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.0640000000000001,"5min":1.125,"15min":1.1890000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.58699999999999997,"5min":0.58099999999999996,"15min":0.54500000000000004},"max":{"1min":0.94099999999999995,"5min":1.262,"15min":1.262},"last":0.873}]},{"osd":7,"last update":"Sun Oct 8 02:28:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79300000000000004,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.04,"5min":1.2170000000000001,"15min":1.296},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.85799999999999998,"15min":0.88},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.55700000000000005},"max":{"1min":1.143,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.83099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316008,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055832,"statfs":{"total":96636764160,"available":96313171968,"internally_reserved":0,"allocated":2392064,"data_stored":1202782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.36499999999999999,"5min":0.309,"15min":0.29099999999999998},"max":{"1min":0.65200000000000002,"5min":0.71999999999999997,"15min":0.871},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.71299999999999997,"5min":0.78300000000000003,"15min":0.83199999999999996},"last":0.52700000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.48699999999999999,"15min":0.48699999999999999},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.86299999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.39600000000000002,"15min":0.36599999999999999},"max":{"1min":0.78900000000000003,"5min":0.80900000000000005,"15min":0.81999999999999995},"last":0.56599999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63400000000000001,"15min":0.622},"min":{"1min":0.505,"5min":0.47699999999999998,"15min":0.42799999999999999},"max":{"1min":0.80400000000000005,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.56999999999999995},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.74199999999999999,"5min":0.80300000000000005,"15min":0.90000000000000002},"last":0.61899999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.80200000000000005,"15min":0.78800000000000003},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.81299999999999994,"15min":0.80100000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.49099999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.871}]},{"osd":5,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.85999999999999999,"15min":0.84199999999999997},"min":{"1min":0.65600000000000003,"5min":0.56499999999999995,"15min":0.46400000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.88600000000000001,"15min":0.85899999999999999},"min":{"1min":0.60199999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.353,"5min":1.353,"15min":1.417},"last":0.95599999999999996}]},{"osd":6,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83899999999999997,"15min":0.81799999999999995},"min":{"1min":0.59299999999999997,"5min":0.58499999999999996,"15min":0.52900000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.333},"last":1.04},{"interface":"front","average":{"1min":0.872,"5min":0.84499999999999997,"15min":0.82999999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.51800000000000002},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.77200000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:27:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84899999999999998,"15min":0.82599999999999996},"min":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.54500000000000004},"max":{"1min":1.387,"5min":1.387,"15min":1.4059999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.875,"5min":0.85199999999999998,"15min":0.84099999999999997},"min":{"1min":0.70799999999999996,"5min":0.58699999999999997,"15min":0.56899999999999995},"max":{"1min":1.252,"5min":1.252,"15min":1.5720000000000001},"last":1.002}]}]},{"osd":3,"up_from":13,"seq":55834575952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299624,"kb_used_data":2336,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072216,"statfs":{"total":96636764160,"available":96329949184,"internally_reserved":0,"allocated":2392064,"data_stored":1202726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.47099999999999997,"15min":0.46000000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.63,"5min":0.84699999999999998,"15min":0.90300000000000002},"last":0.5},{"interface":"front","average":{"1min":0.496,"5min":0.502,"15min":0.501},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.311},"max":{"1min":0.67300000000000004,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.34300000000000003}]},{"osd":1,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.499,"15min":0.49199999999999999},"min":{"1min":0.34000000000000002,"5min":0.33500000000000002,"15min":0.308},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.505,"5min":0.53800000000000003,"15min":0.53400000000000003},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76000000000000001,"5min":1.012,"15min":1.012},"last":0.60199999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86899999999999999,"5min":0.879,"15min":1.282},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.64600000000000002,"15min":0.64100000000000001},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81499999999999995,"5min":0.89200000000000002,"15min":1.512},"last":0.65100000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.77300000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.0209999999999999,"5min":1.0509999999999999,"15min":1.1799999999999999},"last":0.442},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.82499999999999996,"15min":0.80300000000000005},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.004,"5min":1.1399999999999999,"15min":1.3180000000000001},"last":0.68999999999999995}]},{"osd":5,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.81799999999999995,"15min":0.80900000000000005},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.995,"5min":1.147,"15min":1.2789999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.82899999999999996,"15min":0.80600000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.006,"5min":1.056,"15min":1.24},"last":0.93000000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.81899999999999995,"15min":0.81299999999999994},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.53600000000000003},"max":{"1min":0.95499999999999996,"5min":1.109,"15min":1.161},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.83099999999999996,"15min":0.82199999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.99099999999999999,"5min":1.155,"15min":1.2849999999999999},"last":0.52100000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:28:40 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.84399999999999997,"15min":0.83199999999999996},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.52500000000000002},"max":{"1min":0.98799999999999999,"5min":1.129,"15min":1.266},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.85199999999999998,"15min":0.83399999999999996},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.94899999999999995,"5min":1.1160000000000001,"15min":1.282},"last":0.94899999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316020,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055820,"statfs":{"total":96636764160,"available":96313159680,"internally_reserved":0,"allocated":2404352,"data_stored":1204298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.71799999999999997},"min":{"1min":0.57899999999999996,"5min":0.48199999999999998,"15min":0.40300000000000002},"max":{"1min":0.97399999999999998,"5min":1.016,"15min":1.028},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.73199999999999998,"15min":0.73099999999999998},"min":{"1min":0.51800000000000002,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":0.94499999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.497}]},{"osd":1,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.72399999999999998},"min":{"1min":0.49199999999999999,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89900000000000002,"5min":0.998,"15min":1.0409999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.74399999999999999},"min":{"1min":0.58199999999999996,"5min":0.51000000000000001,"15min":0.49099999999999999},"max":{"1min":0.92900000000000005,"5min":1.079,"15min":1.079},"last":0.78700000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79300000000000004,"15min":0.78100000000000003},"min":{"1min":0.55100000000000005,"5min":0.50800000000000001,"15min":0.44600000000000001},"max":{"1min":1.022,"5min":1.022,"15min":1.034},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.78700000000000003,"15min":0.78900000000000003},"min":{"1min":0.69399999999999995,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.91500000000000004,"5min":1.097,"15min":1.097},"last":0.57399999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77600000000000002,"15min":0.77700000000000002},"min":{"1min":0.58399999999999996,"5min":0.51200000000000001,"15min":0.47199999999999998},"max":{"1min":1.0029999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.79800000000000004,"15min":0.80000000000000004},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.52000000000000002},"max":{"1min":0.95799999999999996,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.71499999999999997}]},{"osd":5,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.625,"15min":0.61799999999999999},"min":{"1min":0.44,"5min":0.39500000000000002,"15min":0.36499999999999999},"max":{"1min":0.872,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.76100000000000001,"15min":0.752},"min":{"1min":0.55800000000000005,"5min":0.48599999999999999,"15min":0.48199999999999998},"max":{"1min":0.96199999999999997,"5min":1.105,"15min":1.105},"last":0.69199999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.66100000000000003,"15min":0.64400000000000002},"min":{"1min":0.47399999999999998,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.98299999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.68300000000000005,"15min":0.67000000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.86199999999999999,"5min":1.03,"15min":1.03},"last":0.61199999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.69799999999999995,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.97099999999999997,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.753,"15min":0.72799999999999998},"min":{"1min":0.56599999999999995,"5min":0.48899999999999999,"15min":0.42699999999999999},"max":{"1min":0.97499999999999998,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66100000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575952,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311916,"kb_used_data":2340,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059924,"statfs":{"total":96636764160,"available":96317362176,"internally_reserved":0,"allocated":2396160,"data_stored":1203268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72099999999999997,"15min":0.73199999999999998},"min":{"1min":0.56499999999999995,"5min":0.48399999999999999,"15min":0.40999999999999998},"max":{"1min":0.97999999999999998,"5min":0.996,"15min":0.996},"last":0.753},{"interface":"front","average":{"1min":0.746,"5min":0.74399999999999999,"15min":0.75},"min":{"1min":0.48999999999999999,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63900000000000001}]},{"osd":1,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.745,"15min":0.746},"min":{"1min":0.503,"5min":0.503,"15min":0.44600000000000001},"max":{"1min":0.94399999999999995,"5min":0.99199999999999999,"15min":1.046},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.751,"15min":0.752},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.499},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.69399999999999995}]},{"osd":2,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76600000000000001,"15min":0.76700000000000002},"min":{"1min":0.60799999999999998,"5min":0.60399999999999998,"15min":0.48599999999999999},"max":{"1min":0.96099999999999997,"5min":1.028,"15min":1.028},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.746,"5min":0.76700000000000002,"15min":0.76900000000000002},"min":{"1min":0.52100000000000002,"5min":0.50900000000000001,"15min":0.47099999999999997},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.86299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.76700000000000002},"min":{"1min":0.55100000000000005,"5min":0.53400000000000003,"15min":0.51100000000000001},"max":{"1min":1.0009999999999999,"5min":1.006,"15min":1.006},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78400000000000003},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.98899999999999999,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.83199999999999996}]},{"osd":4,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.45200000000000001,"5min":0.374,"15min":0.374},"max":{"1min":0.88600000000000001,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64400000000000002,"15min":0.66000000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.68500000000000005,"15min":0.67300000000000004},"min":{"1min":0.503,"5min":0.434,"15min":0.434},"max":{"1min":0.872,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68000000000000005,"15min":0.68500000000000005},"min":{"1min":0.44700000000000001,"5min":0.434,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.66700000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:28:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70599999999999996,"15min":0.71599999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.88900000000000001,"5min":1.01,"15min":1.044},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.745,"15min":0.751},"min":{"1min":0.60999999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:28:49.463 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T02:28:49.463 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T02:28:49.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.682+0000 7fc56e3ff700 1 Processor -- start 2023-10-08T02:28:49.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.682+0000 7fc56e3ff700 1 -- start start 2023-10-08T02:28:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.683+0000 7fc56e3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5680cbb90 0x7fc5680af320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.683+0000 7fc56e3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5680af860 con 0x7fc568069c70 2023-10-08T02:28:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.683+0000 7fc56e3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5680cbff0 con 0x7fc568159950 2023-10-08T02:28:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.683+0000 7fc56e3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5680ce380 con 0x7fc5680cbb90 2023-10-08T02:28:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.683+0000 7fc5677fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc568069c70 0x7fc5680cf0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54638/0 (socket says 172.21.15.119:54638) 2023-10-08T02:28:49.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc5677fe700 1 -- 172.21.15.119:0/287121660 learned_addr learned my addr 172.21.15.119:0/287121660 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:49.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc56c99c700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5680cbb90 0x7fc5680af320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294974173 0 0) 0x7fc5680af860 con 0x7fc568069c70 2023-10-08T02:28:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc550003570 con 0x7fc568069c70 2023-10-08T02:28:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113428836 0 0) 0x7fc558001110 con 0x7fc568159950 2023-10-08T02:28:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5680af860 con 0x7fc568159950 2023-10-08T02:28:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3927479222 0 0) 0x7fc550003570 con 0x7fc568069c70 2023-10-08T02:28:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5680cbb90 msgr2=0x7fc5680af320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5680cbb90 0x7fc5680af320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 >> v1:172.21.15.184:6789/0 conn(0x7fc568159950 legacy=0x7fc568069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5680cec20 con 0x7fc568069c70 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fc55c001470 con 0x7fc568069c70 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 >> v1:172.21.15.119:6789/0 conn(0x7fc568069c70 legacy=0x7fc5680cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc550003c20 0x7fc550004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.684+0000 7fc566ffd700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc550004810 0x7fc550004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc566ffd700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc55000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5680ce380 con 0x7fc5500051c0 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55000c0b0 con 0x7fc550004810 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55000c270 con 0x7fc550003c20 2023-10-08T02:28:49.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc56c99c700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc5680cbb90 0x7fc5680af320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc567fff700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc550003c20 0x7fc550004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc5677fe700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc550004810 0x7fc550004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc56c99c700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc55000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc56c99c700 1 -- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc550003c20 msgr2=0x7fc550004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc56c99c700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc550003c20 0x7fc550004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc56c99c700 1 -- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc550004810 msgr2=0x7fc550004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc56c99c700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc550004810 0x7fc550004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc56c99c700 1 -- 172.21.15.119:0/287121660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc550003570 con 0x7fc5500051c0 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.685+0000 7fc56c99c700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc55000ba60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fc5680cbf60 tx=0x7fc56000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bbb15a4a034d0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56000eaa0 con 0x7fc5500051c0 2023-10-08T02:28:49.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56000f040 con 0x7fc5500051c0 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc566ffd700 1 -- 172.21.15.119:0/287121660 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56000ee30 con 0x7fc5500051c0 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc56e3ff700 1 -- 172.21.15.119:0/287121660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 msgr2=0x7fc55000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc56e3ff700 1 --2- 172.21.15.119:0/287121660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc55000ba60 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fc5680cbf60 tx=0x7fc56000b040 comp rx=0 tx=0).stop 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc56c99c700 1 -- 172.21.15.119:0/287121660 reap_dead start 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc56e3ff700 1 -- 172.21.15.119:0/287121660 shutdown_connections 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc56e3ff700 1 -- 172.21.15.119:0/287121660 >> 172.21.15.119:0/287121660 conn(0x7fc568068e00 msgr2=0x7fc5680d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc56e3ff700 1 -- 172.21.15.119:0/287121660 shutdown_connections 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.686+0000 7fc56e3ff700 1 -- 172.21.15.119:0/287121660 wait complete. 2023-10-08T02:28:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.688+0000 7fc56e3ff700 1 Processor -- start 2023-10-08T02:28:49.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.688+0000 7fc56e3ff700 1 -- start start 2023-10-08T02:28:49.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc56e3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc568159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc56e3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc56816dcc0 0x7fc5680aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc56e3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc56815a3c0 0x7fc5680c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc56e3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc568158060 con 0x7fc5500051c0 2023-10-08T02:28:49.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc56e3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5680c8830 con 0x7fc56816dcc0 2023-10-08T02:28:49.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc56e3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56816e130 con 0x7fc56815a3c0 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc567fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc568159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc567fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc568159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37560/0 (socket says 172.21.15.119:37560) 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc567fff700 1 -- 172.21.15.119:0/1793145139 learned_addr learned my addr 172.21.15.119:0/1793145139 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc5677fe700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc56816dcc0 0x7fc5680aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc56c99c700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc56815a3c0 0x7fc5680c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc567fff700 1 -- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc56815a3c0 msgr2=0x7fc5680c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc567fff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc56815a3c0 0x7fc5680c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc567fff700 1 -- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc56816dcc0 msgr2=0x7fc5680aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc567fff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc56816dcc0 0x7fc5680aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc567fff700 1 -- 172.21.15.119:0/1793145139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5680c8a80 con 0x7fc5500051c0 2023-10-08T02:28:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.689+0000 7fc5677fe700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc56816dcc0 0x7fc5680aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.690+0000 7fc567fff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc568159d20 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc55800f880 tx=0x7fc558007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.690+0000 7fc54ffff700 1 -- 172.21.15.119:0/1793145139 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558013600 con 0x7fc5500051c0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.690+0000 7fc54ffff700 1 -- 172.21.15.119:0/1793145139 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc558013760 con 0x7fc5500051c0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.690+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5680c8d60 con 0x7fc5500051c0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.690+0000 7fc54ffff700 1 -- 172.21.15.119:0/1793145139 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558013ae0 con 0x7fc5500051c0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.690+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5680c9320 con 0x7fc5500051c0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.691+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5680c9540 con 0x7fc5500051c0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.692+0000 7fc54ffff700 1 -- 172.21.15.119:0/1793145139 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc55801c400 con 0x7fc5500051c0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.692+0000 7fc54ffff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc554066f30 0x7fc5540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.693+0000 7fc54ffff700 1 -- 172.21.15.119:0/1793145139 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fc558084970 con 0x7fc5500051c0 2023-10-08T02:28:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.693+0000 7fc5677fe700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc554066f30 0x7fc5540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:49.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.693+0000 7fc5677fe700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc554066f30 0x7fc5540693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000e10 tx=0x7fc55c009040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:49.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.696+0000 7fc54ffff700 1 -- 172.21.15.119:0/1793145139 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc558051950 con 0x7fc5500051c0 2023-10-08T02:28:49.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.966+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc5680cbff0 con 0x7fc5500051c0 2023-10-08T02:28:49.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.970+0000 7fc54ffff700 1 -- 172.21.15.119:0/1793145139 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v51) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc5680cbff0 con 0x7fc5500051c0 2023-10-08T02:28:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.972+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc554066f30 msgr2=0x7fc5540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.972+0000 7fc56e3ff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc554066f30 0x7fc5540693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc55c000e10 tx=0x7fc55c009040 comp rx=0 tx=0).stop 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.972+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 msgr2=0x7fc568159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.972+0000 7fc56e3ff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc568159d20 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc55800f880 tx=0x7fc558007d10 comp rx=0 tx=0).stop 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.973+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 shutdown_connections 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.973+0000 7fc56e3ff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc554066f30 0x7fc5540693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.973+0000 7fc56e3ff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc56815a3c0 0x7fc5680c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.973+0000 7fc56e3ff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5500051c0 0x7fc568159d20 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.973+0000 7fc56e3ff700 1 --2- 172.21.15.119:0/1793145139 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc56816dcc0 0x7fc5680aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.973+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 >> 172.21.15.119:0/1793145139 conn(0x7fc568068e00 msgr2=0x7fc5680cadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.973+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 shutdown_connections 2023-10-08T02:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:49.973+0000 7fc56e3ff700 1 -- 172.21.15.119:0/1793145139 wait complete. 2023-10-08T02:28:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T02:28:49.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:28:50.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.174+0000 7f5dcb1ff700 1 Processor -- start 2023-10-08T02:28:50.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.174+0000 7f5dcb1ff700 1 -- start start 2023-10-08T02:28:50.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.175+0000 7f5dcb1ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5dc40b1420 0x7f5dc40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:50.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.175+0000 7f5dcb1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40ccf90 0x7f5dc40c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:50.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.175+0000 7f5dcb1ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc40cc300 con 0x7f5dc40cbf30 2023-10-08T02:28:50.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.175+0000 7f5dcb1ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc4156bf0 con 0x7f5dc40b1420 2023-10-08T02:28:50.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.175+0000 7f5dcb1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40ae950 con 0x7f5dc40ccf90 2023-10-08T02:28:50.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc8f9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5dc40cbf30 0x7f5dc40b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54644/0 (socket says 172.21.15.119:54644) 2023-10-08T02:28:50.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc8f9b700 1 -- 172.21.15.119:0/779509346 learned_addr learned my addr 172.21.15.119:0/779509346 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:50.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc979c700 1 --2- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40ccf90 0x7f5dc40c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc3fff700 1 --2- 172.21.15.119:0/779509346 >> v2:172.21.15.184:3300/0 conn(0x7f5dc40b1420 0x7f5dc40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc37fe700 1 -- 172.21.15.119:0/779509346 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718716005 0 0) 0x7f5dc40cc300 con 0x7f5dc40cbf30 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc37fe700 1 -- 172.21.15.119:0/779509346 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5db4003570 con 0x7f5dc40cbf30 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc3fff700 1 -- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40ccf90 msgr2=0x7f5dc40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc3fff700 1 --2- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40ccf90 0x7f5dc40c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc37fe700 1 -- 172.21.15.119:0/779509346 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2991010078 0 0) 0x7f5db4003570 con 0x7f5dc40cbf30 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc3fff700 1 -- 172.21.15.119:0/779509346 >> v1:172.21.15.119:6789/0 conn(0x7f5dc40cbf30 legacy=0x7f5dc40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc3fff700 1 -- 172.21.15.119:0/779509346 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc40ad370 con 0x7f5dc40b1420 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.176+0000 7f5dc979c700 1 --2- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40ccf90 0x7f5dc40c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:50.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.177+0000 7f5dc3fff700 1 --2- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 0x7f5dc40ae410 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009910 tx=0x7f5db00091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45ccb58d6e153199 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.177+0000 7f5dc37fe700 1 -- 172.21.15.119:0/779509346 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db8001470 con 0x7f5dc40b1420 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.177+0000 7f5dc37fe700 1 -- 172.21.15.119:0/779509346 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0011420 con 0x7f5dc40b1420 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.177+0000 7f5dc37fe700 1 -- 172.21.15.119:0/779509346 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00117c0 con 0x7f5dc40b1420 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.178+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 msgr2=0x7f5dc40ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.178+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 0x7f5dc40ae410 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5db0009910 tx=0x7f5db00091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.178+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/779509346 shutdown_connections 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.178+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40ccf90 0x7f5dc40c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.178+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/779509346 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 0x7f5dc40ae410 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.178+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/779509346 >> 172.21.15.119:0/779509346 conn(0x7f5dc406a320 msgr2=0x7f5dc40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.178+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/779509346 shutdown_connections 2023-10-08T02:28:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.178+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/779509346 wait complete. 2023-10-08T02:28:50.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.179+0000 7f5dcb1ff700 1 Processor -- start 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.179+0000 7f5dcb1ff700 1 -- start start 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dcb1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 0x7f5dc40c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dcb1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40cbf30 0x7f5dc40c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dcb1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc40ccf90 0x7f5dc40c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dcb1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc414f950 con 0x7f5dc40ccf90 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dcb1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c5e80 con 0x7f5dc40b1420 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dcb1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc40c77c0 con 0x7f5dc40cbf30 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dc979c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc40ccf90 0x7f5dc40c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dc979c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc40ccf90 0x7f5dc40c12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37576/0 (socket says 172.21.15.119:37576) 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.180+0000 7f5dc979c700 1 -- 172.21.15.119:0/3828457220 learned_addr learned my addr 172.21.15.119:0/3828457220 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc8f9b700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 0x7f5dc40c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc3fff700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40cbf30 0x7f5dc40c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc979c700 1 -- 172.21.15.119:0/3828457220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40cbf30 msgr2=0x7f5dc40c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc979c700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40cbf30 0x7f5dc40c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc979c700 1 -- 172.21.15.119:0/3828457220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 msgr2=0x7f5dc40c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc979c700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 0x7f5dc40c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc979c700 1 -- 172.21.15.119:0/3828457220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc009a20 con 0x7f5dc40ccf90 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc979c700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc40ccf90 0x7f5dc40c12d0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009f20 tx=0x7f5dbc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc0ff9700 1 -- 172.21.15.119:0/3828457220 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc00cc80 con 0x7f5dc40ccf90 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc0ff9700 1 -- 172.21.15.119:0/3828457220 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dbc00f040 con 0x7f5dc40ccf90 2023-10-08T02:28:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dc0ff9700 1 -- 172.21.15.119:0/3828457220 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dbc014480 con 0x7f5dc40ccf90 2023-10-08T02:28:50.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db0009420 con 0x7f5dc40ccf90 2023-10-08T02:28:50.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.181+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc40c7e00 con 0x7f5dc40ccf90 2023-10-08T02:28:50.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.182+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc404c3c0 con 0x7f5dc40ccf90 2023-10-08T02:28:50.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.183+0000 7f5dc0ff9700 1 -- 172.21.15.119:0/3828457220 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5dbc003020 con 0x7f5dc40ccf90 2023-10-08T02:28:50.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.184+0000 7f5dc0ff9700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5da8066f30 0x7f5da80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:50.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.184+0000 7f5dc0ff9700 1 -- 172.21.15.119:0/3828457220 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f5dbc0851c0 con 0x7f5dc40ccf90 2023-10-08T02:28:50.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.184+0000 7f5dc8f9b700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5da8066f30 0x7f5da80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:50.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.185+0000 7f5dc8f9b700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5da8066f30 0x7f5da80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5db8001ed0 tx=0x7f5db8001520 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:50.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.188+0000 7f5dc0ff9700 1 -- 172.21.15.119:0/3828457220 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5dbc0521a0 con 0x7f5dc40ccf90 2023-10-08T02:28:50.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.422+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5dc406f8a0 con 0x7f5dc40ccf90 2023-10-08T02:28:50.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.423+0000 7f5dc0ff9700 1 -- 172.21.15.119:0/3828457220 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f5dbc0557c0 con 0x7f5dc40ccf90 2023-10-08T02:28:50.423 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:28:50.423 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3041824229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3041824229},{"type":"v1","addr":"172.21.15.184:6834","nonce":3041824229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1478841333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1478841333},{"type":"v1","addr":"172.21.15.119:6834","nonce":1478841333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9723,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/483355636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":483355636},{"type":"v1","addr":"172.21.15.119:6835","nonce":483355636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:28:39.274713+0000","modified":"2023-10-08T02:28:42.824801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9726":{"gid":9726,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3185561357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3185561357},{"type":"v1","addr":"172.21.15.184:6835","nonce":3185561357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T02:28:50.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5da8066f30 msgr2=0x7f5da80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:50.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5da8066f30 0x7f5da80693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5db8001ed0 tx=0x7f5db8001520 comp rx=0 tx=0).stop 2023-10-08T02:28:50.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc40ccf90 msgr2=0x7f5dc40c12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:50.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc40ccf90 0x7f5dc40c12d0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc009f20 tx=0x7f5dbc00b040 comp rx=0 tx=0).stop 2023-10-08T02:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 shutdown_connections 2023-10-08T02:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5da8066f30 0x7f5da80693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5dc40ccf90 0x7f5dc40c12d0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5dc40b1420 0x7f5dc40c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 --2- 172.21.15.119:0/3828457220 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5dc40cbf30 0x7f5dc40c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 >> 172.21.15.119:0/3828457220 conn(0x7f5dc406a320 msgr2=0x7f5dc414fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 shutdown_connections 2023-10-08T02:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:50.425+0000 7f5dcb1ff700 1 -- 172.21.15.119:0/3828457220 wait complete. 2023-10-08T02:28:50.427 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 87 2023-10-08T02:28:50.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T02:28:39.274713+0000', 'modified': '2023-10-08T02:28:42.824801+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9726}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9726': {'gid': 9726, 'name': 'd', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.184:6835/3185561357', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.184:6833', 'nonce': 3185561357}, {'type': 'v1', 'addr': '172.21.15.184:6835', 'nonce': 3185561357}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T02:28:50.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T02:28:50.846 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-08T02:28:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.029+0000 7f8545dff700 1 Processor -- start 2023-10-08T02:28:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.029+0000 7f8545dff700 1 -- start start 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.030+0000 7f8545dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85400cbf30 0x7f85400ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.030+0000 7f8545dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f85400ccf90 0x7f85400b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.030+0000 7f8545dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85400cc300 con 0x7f854015a270 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.030+0000 7f8545dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8540156bf0 con 0x7f85400cbf30 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f8545dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85400ae950 con 0x7f85400ccf90 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f853f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85400cbf30 0x7f85400ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f853f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85400cbf30 0x7f85400ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37580/0 (socket says 172.21.15.119:37580) 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f853f7fe700 1 -- 172.21.15.119:0/3689366623 learned_addr learned my addr 172.21.15.119:0/3689366623 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:51.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f853effd700 1 --2- 172.21.15.119:0/3689366623 >> v2:172.21.15.184:3300/0 conn(0x7f85400ccf90 0x7f85400b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f853f7fe700 1 -- 172.21.15.119:0/3689366623 >> v2:172.21.15.184:3300/0 conn(0x7f85400ccf90 msgr2=0x7f85400b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f853f7fe700 1 --2- 172.21.15.119:0/3689366623 >> v2:172.21.15.184:3300/0 conn(0x7f85400ccf90 0x7f85400b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f853f7fe700 1 -- 172.21.15.119:0/3689366623 >> v1:172.21.15.184:6789/0 conn(0x7f854015a270 legacy=0x7f85400c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.031+0000 7f853f7fe700 1 -- 172.21.15.119:0/3689366623 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85400ad370 con 0x7f85400cbf30 2023-10-08T02:28:51.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f853f7fe700 1 --2- 172.21.15.119:0/3689366623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400cbf30 0x7f85400ae410 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f854006d670 tx=0x7f8530009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d39e9477d1d3bc3f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f853e7fc700 1 -- 172.21.15.119:0/3689366623 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530009ea0 con 0x7f85400cbf30 2023-10-08T02:28:51.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f853e7fc700 1 -- 172.21.15.119:0/3689366623 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853000b820 con 0x7f85400cbf30 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f853e7fc700 1 -- 172.21.15.119:0/3689366623 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853000bbc0 con 0x7f85400cbf30 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f8545dff700 1 -- 172.21.15.119:0/3689366623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400cbf30 msgr2=0x7f85400ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f8545dff700 1 --2- 172.21.15.119:0/3689366623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400cbf30 0x7f85400ae410 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f854006d670 tx=0x7f8530009250 comp rx=0 tx=0).stop 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f8545dff700 1 -- 172.21.15.119:0/3689366623 shutdown_connections 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f8545dff700 1 --2- 172.21.15.119:0/3689366623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400cbf30 0x7f85400ae410 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f8545dff700 1 --2- 172.21.15.119:0/3689366623 >> v2:172.21.15.184:3300/0 conn(0x7f85400ccf90 0x7f85400b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f8545dff700 1 -- 172.21.15.119:0/3689366623 >> 172.21.15.119:0/3689366623 conn(0x7f854006a320 msgr2=0x7f85400cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.032+0000 7f8545dff700 1 -- 172.21.15.119:0/3689366623 shutdown_connections 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.033+0000 7f8545dff700 1 -- 172.21.15.119:0/3689366623 wait complete. 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.034+0000 7f8545dff700 1 Processor -- start 2023-10-08T02:28:51.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.034+0000 7f8545dff700 1 -- start start 2023-10-08T02:28:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f8545dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85400cbf30 0x7f854016dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f8545dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400ccf90 0x7f8540141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f8545dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854015a270 0x7f8540160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f8545dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8540142150 con 0x7f85400ccf90 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f8545dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8540142cd0 con 0x7f854015a270 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f8545dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8540141e60 con 0x7f85400cbf30 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f853effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400ccf90 0x7f8540141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f853effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400ccf90 0x7f8540141920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37590/0 (socket says 172.21.15.119:37590) 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f853effd700 1 -- 172.21.15.119:0/3293030579 learned_addr learned my addr 172.21.15.119:0/3293030579 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f853ffff700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854015a270 0x7f8540160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f853f7fe700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85400cbf30 0x7f854016dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.035+0000 7f853effd700 1 -- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85400cbf30 msgr2=0x7f854016dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f853effd700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85400cbf30 0x7f854016dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f853effd700 1 -- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854015a270 msgr2=0x7f8540160520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f853effd700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854015a270 0x7f8540160520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f853effd700 1 -- 172.21.15.119:0/3293030579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85300094f0 con 0x7f85400ccf90 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f853ffff700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854015a270 0x7f8540160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f853f7fe700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85400cbf30 0x7f854016dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f853effd700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400ccf90 0x7f8540141920 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f853400ecb0 tx=0x7f85340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f85277fe700 1 -- 172.21.15.119:0/3293030579 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853400c9d0 con 0x7f85400ccf90 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f85277fe700 1 -- 172.21.15.119:0/3293030579 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853400cb30 con 0x7f85400ccf90 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f8545dff700 1 -- 172.21.15.119:0/3293030579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85401423a0 con 0x7f85400ccf90 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f85277fe700 1 -- 172.21.15.119:0/3293030579 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853400ced0 con 0x7f85400ccf90 2023-10-08T02:28:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.036+0000 7f8545dff700 1 -- 172.21.15.119:0/3293030579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8540142960 con 0x7f85400ccf90 2023-10-08T02:28:51.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.038+0000 7f85277fe700 1 -- 172.21.15.119:0/3293030579 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8534013680 con 0x7f85400ccf90 2023-10-08T02:28:51.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.039+0000 7f85277fe700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f852c066f30 0x7f852c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.039+0000 7f85277fe700 1 -- 172.21.15.119:0/3293030579 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f8534084b80 con 0x7f85400ccf90 2023-10-08T02:28:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.039+0000 7f853f7fe700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f852c066f30 0x7f852c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.040+0000 7f853f7fe700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f852c066f30 0x7f852c0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f854006d670 tx=0x7f8530017000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.041+0000 7f85545b8b80 1 Processor -- start 2023-10-08T02:28:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.041+0000 7f85545b8b80 1 -- start start 2023-10-08T02:28:51.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 0x55a0f2b3ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2b67a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b3acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f85545b8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2a95a70 con 0x55a0f2b88d00 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f85545b8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2a93270 con 0x55a0f2b67f60 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f85545b8b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2a8f060 con 0x55a0f2b3ea50 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f853effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2b67a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f853effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2b67a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37596/0 (socket says 172.21.15.119:37596) 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f853effd700 1 -- 172.21.15.119:0/750509261 learned_addr learned my addr 172.21.15.119:0/750509261 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f853ffff700 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b3acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.042+0000 7f853f7fe700 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 0x55a0f2b3ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f853effd700 1 -- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 msgr2=0x55a0f2b3ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f853effd700 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 0x55a0f2b3ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f853effd700 1 -- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 msgr2=0x55a0f2b3acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f853effd700 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b3acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f853effd700 1 -- 172.21.15.119:0/750509261 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0f2929270 con 0x55a0f2b88d00 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f853f7fe700 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 0x55a0f2b3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f853effd700 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2b67a20 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f85340557b0 tx=0x7f8534082d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=870dd8799fc915b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f8524ff9700 1 -- 172.21.15.119:0/750509261 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853405c7c0 con 0x55a0f2b88d00 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f8524ff9700 1 -- 172.21.15.119:0/750509261 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8534053960 con 0x55a0f2b88d00 2023-10-08T02:28:51.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f8524ff9700 1 -- 172.21.15.119:0/750509261 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8534055a80 con 0x55a0f2b88d00 2023-10-08T02:28:51.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f85545b8b80 1 -- 172.21.15.119:0/750509261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 msgr2=0x55a0f2b67a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.043+0000 7f85545b8b80 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2b67a20 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f85340557b0 tx=0x7f8534082d70 comp rx=0 tx=0).stop 2023-10-08T02:28:51.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.044+0000 7f85545b8b80 1 -- 172.21.15.119:0/750509261 shutdown_connections 2023-10-08T02:28:51.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.044+0000 7f85545b8b80 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 0x55a0f2b3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.044+0000 7f85545b8b80 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b3acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.044+0000 7f85545b8b80 1 --2- 172.21.15.119:0/750509261 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2b67a20 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.044+0000 7f85545b8b80 1 -- 172.21.15.119:0/750509261 >> 172.21.15.119:0/750509261 conn(0x55a0f2b40370 msgr2=0x55a0f2b40740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.044+0000 7f85545b8b80 1 -- 172.21.15.119:0/750509261 shutdown_connections 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.044+0000 7f85545b8b80 1 -- 172.21.15.119:0/750509261 wait complete. 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.044+0000 7f85545b8b80 1 -- start start 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.045+0000 7f85545b8b80 1 Processor -- start 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.045+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b64420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b85ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b88d00 0x55a0f2b824a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f85545b8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2a3a890 con 0x55a0f2b3ea50 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f85545b8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2a55ea0 con 0x55a0f2b67f60 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f85545b8b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2911e30 con 0x55a0f2b88d00 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f85545b8b80 10 client.0 _ll_register_callbacks cb 0x55a0f2b02110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b64420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b64420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37610/0 (socket says 172.21.15.119:37610) 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853f7fe700 1 -- 172.21.15.119:0/2807934304 learned_addr learned my addr 172.21.15.119:0/2807934304 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853effd700 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b85ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853ffff700 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b88d00 0x55a0f2b824a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853f7fe700 1 -- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b88d00 msgr2=0x55a0f2b824a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853f7fe700 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b88d00 0x55a0f2b824a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853f7fe700 1 -- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 msgr2=0x55a0f2b85ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853f7fe700 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b85ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.046+0000 7f853f7fe700 1 -- 172.21.15.119:0/2807934304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8540142960 con 0x55a0f2b3ea50 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f853f7fe700 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b64420 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f8530009290 tx=0x7f8530020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39a7941f6b6987f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f851dffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f851dffb700 1 -- 172.21.15.119:0/2807934304 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f853002d070 con 0x55a0f2b3ea50 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f851dffb700 1 -- 172.21.15.119:0/2807934304 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f8530032460 con 0x55a0f2b3ea50 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 1 -- 172.21.15.119:0/2807934304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a0f2b33b70 con 0x55a0f2b3ea50 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f851dffb700 1 -- 172.21.15.119:0/2807934304 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8530020070 con 0x55a0f2b3ea50 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 10 client.9879 fetch_fsmap learned FSMap version 87 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 1 -- 172.21.15.119:0/2807934304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55a0f28cbfe0 con 0x55a0f2b3ea50 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f851dffb700 1 -- 172.21.15.119:0/2807934304 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8530039020 con 0x55a0f2b3ea50 2023-10-08T02:28:51.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f851dffb700 1 -- 172.21.15.119:0/2807934304 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 87) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f8530032830 con 0x55a0f2b3ea50 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 10 client.9879 fetch_fsmap finished waiting for FSMap version 87 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 10 client.9879 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 20 client.9879 populate_metadata read hostname 'smithi119' 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] conn(0x55a0f2b7d1a0 0x55a0f2b7d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 4 client.9879 mds_command: new command op to 9726 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.047+0000 7f85545b8b80 1 -- 172.21.15.119:0/2807934304 --> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0f2b10aa0 con 0x55a0f2b7d1a0 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.048+0000 7f853effd700 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] conn(0x55a0f2b7d1a0 0x55a0f2b7d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.048+0000 7f851dffb700 10 client.9879 ms_handle_connect on v2:172.21.15.184:6833/3185561357 2023-10-08T02:28:51.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.048+0000 7f853effd700 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] conn(0x55a0f2b7d1a0 0x55a0f2b7d570 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9abd678fa6fabcd server_cookie=bbda40e07c4c7046 in_seq=0 out_seq=0 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.049+0000 7f851dffb700 1 -- 172.21.15.119:0/2807934304 <== mds.0 v2:172.21.15.184:6833/3185561357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55a0f2b10aa0 con 0x55a0f2b7d1a0 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.049+0000 7f851dffb700 10 client.9879 handle_command_reply: tid=0 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.049+0000 7f85545b8b80 1 client.9879 shutdown 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.049+0000 7f85545b8b80 1 -- 172.21.15.119:0/2807934304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 msgr2=0x55a0f2b64420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.049+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b64420 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f8530009290 tx=0x7f8530020000 comp rx=0 tx=0).stop 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 1 -- 172.21.15.119:0/2807934304 shutdown_connections 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] conn(0x55a0f2b7d1a0 0x55a0f2b7d570 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b88d00 0x55a0f2b824a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b64420 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f851dffb700 0 client.9879 ms_handle_reset on v2:172.21.15.184:6833/3185561357 2023-10-08T02:28:51.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2807934304 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b85ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 1 -- 172.21.15.119:0/2807934304 >> 172.21.15.119:0/2807934304 conn(0x55a0f2b40370 msgr2=0x55a0f2b84550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:51.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 1 -- 172.21.15.119:0/2807934304 shutdown_connections 2023-10-08T02:28:51.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 1 -- 172.21.15.119:0/2807934304 wait complete. 2023-10-08T02:28:51.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.050+0000 7f85545b8b80 20 client.9879 trim_cache size 0 max 16384 2023-10-08T02:28:51.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.067+0000 7f85545b8b80 1 Processor -- start 2023-10-08T02:28:51.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.067+0000 7f85545b8b80 1 -- start start 2023-10-08T02:28:51.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b66b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b67f60 0x55a0f2b86020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b88d00 0x55a0f2b3ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f85545b8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2b395a0 con 0x55a0f2b3ea50 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f85545b8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2b412f0 con 0x55a0f2b88d00 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f85545b8b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2b3dd40 con 0x55a0f2b67f60 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b66b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853f7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b66b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37620/0 (socket says 172.21.15.119:37620) 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853f7fe700 1 -- 172.21.15.119:0/2549426278 learned_addr learned my addr 172.21.15.119:0/2549426278 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853ffff700 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b88d00 0x55a0f2b3ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853effd700 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b67f60 0x55a0f2b86020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853f7fe700 1 -- 172.21.15.119:0/2549426278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b67f60 msgr2=0x55a0f2b86020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853f7fe700 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b67f60 0x55a0f2b86020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853f7fe700 1 -- 172.21.15.119:0/2549426278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b88d00 msgr2=0x55a0f2b3ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853f7fe700 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b88d00 0x55a0f2b3ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.068+0000 7f853f7fe700 1 -- 172.21.15.119:0/2549426278 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0f2a5d780 con 0x55a0f2b3ea50 2023-10-08T02:28:51.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f853f7fe700 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b66b70 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8530000e50 tx=0x7f853002d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ebc36e7d2954af server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85257fa700 1 -- 172.21.15.119:0/2549426278 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530020030 con 0x55a0f2b3ea50 2023-10-08T02:28:51.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85257fa700 1 -- 172.21.15.119:0/2549426278 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8530039040 con 0x55a0f2b3ea50 2023-10-08T02:28:51.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85257fa700 1 -- 172.21.15.119:0/2549426278 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530032d90 con 0x55a0f2b3ea50 2023-10-08T02:28:51.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85545b8b80 1 -- 172.21.15.119:0/2549426278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 msgr2=0x55a0f2b66b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b66b70 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8530000e50 tx=0x7f853002d000 comp rx=0 tx=0).stop 2023-10-08T02:28:51.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85545b8b80 1 -- 172.21.15.119:0/2549426278 shutdown_connections 2023-10-08T02:28:51.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b88d00 0x55a0f2b3ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b3ea50 0x55a0f2b66b70 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2549426278 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b67f60 0x55a0f2b86020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.069+0000 7f85545b8b80 1 -- 172.21.15.119:0/2549426278 >> 172.21.15.119:0/2549426278 conn(0x55a0f2b40370 msgr2=0x55a0f2b41b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.070+0000 7f85545b8b80 1 -- 172.21.15.119:0/2549426278 shutdown_connections 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.070+0000 7f85545b8b80 1 -- 172.21.15.119:0/2549426278 wait complete. 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.070+0000 7f85545b8b80 1 -- start start 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.070+0000 7f85545b8b80 1 Processor -- start 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.071+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 0x55a0f2bef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.071+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b9e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.071+0000 7f85545b8b80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2bea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.071+0000 7f85545b8b80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2b9f230 con 0x55a0f2b88d00 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.071+0000 7f85545b8b80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2beb780 con 0x55a0f2b67f60 2023-10-08T02:28:51.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.071+0000 7f85545b8b80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0f2b41b00 con 0x55a0f2b3ea50 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.071+0000 7f85545b8b80 10 client.0 _ll_register_callbacks cb 0x55a0f2b02110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.071+0000 7f853ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2bea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f853ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2bea6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37636/0 (socket says 172.21.15.119:37636) 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f853ffff700 1 -- 172.21.15.119:0/2005445623 learned_addr learned my addr 172.21.15.119:0/2005445623 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f853ffff700 1 -- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 msgr2=0x55a0f2bef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f853ffff700 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 0x55a0f2bef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f853ffff700 1 -- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 msgr2=0x55a0f2b9e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f853ffff700 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b9e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f853ffff700 1 -- 172.21.15.119:0/2005445623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0f28cbfe0 con 0x55a0f2b88d00 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f853ffff700 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2bea6f0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f85380138e0 tx=0x7f8538009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e34bedcb901f99 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f851dffb700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f851dffb700 1 -- 172.21.15.119:0/2005445623 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538015620 con 0x55a0f2b88d00 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f851dffb700 1 -- 172.21.15.119:0/2005445623 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f8538015780 con 0x55a0f2b88d00 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f85545b8b80 1 -- 172.21.15.119:0/2005445623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8540159d60 con 0x55a0f2b88d00 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f851dffb700 1 -- 172.21.15.119:0/2005445623 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8538015b50 con 0x55a0f2b88d00 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f85545b8b80 10 client.9885 fetch_fsmap learned FSMap version 87 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.072+0000 7f85545b8b80 1 -- 172.21.15.119:0/2005445623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55a0f2a44470 con 0x55a0f2b88d00 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f851dffb700 1 -- 172.21.15.119:0/2005445623 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8540159d60 con 0x55a0f2b88d00 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f851dffb700 1 -- 172.21.15.119:0/2005445623 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 87) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f853801b020 con 0x55a0f2b88d00 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f85545b8b80 10 client.9885 fetch_fsmap finished waiting for FSMap version 87 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f85545b8b80 10 client.9885 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-08T02:28:51.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f85545b8b80 20 client.9885 populate_metadata read hostname 'smithi119' 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] conn(0x55a0f2b7bc20 0x55a0f2b7bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f85545b8b80 4 client.9885 mds_command: new command op to 9726 tid=0[{"prefix": "status"}] 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f85545b8b80 1 -- 172.21.15.119:0/2005445623 --> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f8534053960 con 0x55a0f2b7bc20 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.073+0000 7f853f7fe700 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] conn(0x55a0f2b7bc20 0x55a0f2b7bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f851dffb700 10 client.9885 ms_handle_connect on v2:172.21.15.184:6833/3185561357 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f853f7fe700 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] conn(0x55a0f2b7bc20 0x55a0f2b7bff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9249a5b3f943c0d server_cookie=c0e3480b0f93ca86 in_seq=0 out_seq=0 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f851dffb700 1 -- 172.21.15.119:0/2005445623 <== mds.0 v2:172.21.15.184:6833/3185561357 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x7f8534053960 con 0x55a0f2b7bc20 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f851dffb700 10 client.9885 handle_command_reply: tid=0 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 client.9885 shutdown 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 -- 172.21.15.119:0/2005445623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 msgr2=0x55a0f2bea6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2bea6f0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f85380138e0 tx=0x7f8538009040 comp rx=0 tx=0).stop 2023-10-08T02:28:51.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 -- 172.21.15.119:0/2005445623 shutdown_connections 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:6833/3185561357,v1:172.21.15.184:6835/3185561357] conn(0x55a0f2b7bc20 0x55a0f2b7bff0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x55a0f2b3ea50 0x55a0f2bef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a0f2b88d00 0x55a0f2bea6f0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f851dffb700 0 client.9885 ms_handle_reset on v2:172.21.15.184:6833/3185561357 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 --2- 172.21.15.119:0/2005445623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x55a0f2b67f60 0x55a0f2b9e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 -- 172.21.15.119:0/2005445623 >> 172.21.15.119:0/2005445623 conn(0x55a0f2b40370 msgr2=0x55a0f2b6eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 -- 172.21.15.119:0/2005445623 shutdown_connections 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.074+0000 7f85545b8b80 1 -- 172.21.15.119:0/2005445623 wait complete. 2023-10-08T02:28:51.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f85545b8b80 20 client.9885 trim_cache size 0 max 16384 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 9726, 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 11.27014904, 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 87, 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 191, 2023-10-08T02:28:51.080 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 191, 2023-10-08T02:28:51.081 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 17.030174070000001 2023-10-08T02:28:51.081 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:28:51.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 -- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f852c066f30 msgr2=0x7f852c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f852c066f30 0x7f852c0693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f854006d670 tx=0x7f8530017000 comp rx=0 tx=0).stop 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 -- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400ccf90 msgr2=0x7f8540141920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400ccf90 0x7f8540141920 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f853400ecb0 tx=0x7f85340091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 -- 172.21.15.119:0/3293030579 shutdown_connections 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f852c066f30 0x7f852c0693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f85400cbf30 0x7f854016dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f854015a270 0x7f8540160520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 --2- 172.21.15.119:0/3293030579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85400ccf90 0x7f8540141920 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 -- 172.21.15.119:0/3293030579 >> 172.21.15.119:0/3293030579 conn(0x7f854006a320 msgr2=0x7f854006ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:51.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 -- 172.21.15.119:0/3293030579 shutdown_connections 2023-10-08T02:28:51.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.075+0000 7f8545dff700 1 -- 172.21.15.119:0/3293030579 wait complete. 2023-10-08T02:28:51.091 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T02:28:51.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.272+0000 7f98d57ff700 1 Processor -- start 2023-10-08T02:28:51.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.272+0000 7f98d57ff700 1 -- start start 2023-10-08T02:28:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98d57ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f98d0155c10 0x7f98d0155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98d57ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d00ac6d0 0x7f98d00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98d57ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00acac0 con 0x7f98d00ac230 2023-10-08T02:28:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98d57ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d00ad930 con 0x7f98d0155c10 2023-10-08T02:28:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98d57ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00acd20 con 0x7f98d00ac6d0 2023-10-08T02:28:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98ce7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f98d00ac230 0x7f98d01568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54656/0 (socket says 172.21.15.119:54656) 2023-10-08T02:28:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98ce7fc700 1 -- 172.21.15.119:0/2003833594 learned_addr learned my addr 172.21.15.119:0/2003833594 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737872460 0 0) 0x7f98d00acac0 con 0x7f98d00ac230 2023-10-08T02:28:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98ceffd700 1 --2- 172.21.15.119:0/2003833594 >> v2:172.21.15.184:3300/0 conn(0x7f98d0155c10 0x7f98d0155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98cf7fe700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d00ac6d0 0x7f98d00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.274+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98b8003570 con 0x7f98d00ac230 2023-10-08T02:28:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2180683849 0 0) 0x7f98b8003570 con 0x7f98d00ac230 2023-10-08T02:28:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d00ac6d0 msgr2=0x7f98d00b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d00ac6d0 0x7f98d00b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 >> v2:172.21.15.184:3300/0 conn(0x7f98d0155c10 msgr2=0x7f98d0155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 --2- 172.21.15.119:0/2003833594 >> v2:172.21.15.184:3300/0 conn(0x7f98d0155c10 0x7f98d0155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00ad000 con 0x7f98d00ac230 2023-10-08T02:28:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f98c4001470 con 0x7f98d00ac230 2023-10-08T02:28:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 >> v1:172.21.15.119:6789/0 conn(0x7f98d00ac230 legacy=0x7f98d01568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d00ac6d0 0x7f98b8003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b8004000 0x7f98b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00ad930 con 0x7f98b8004d50 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00acd20 con 0x7f98b8004000 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b800dcc0 con 0x7f98d00ac6d0 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cf7fe700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98ceffd700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d00ac6d0 0x7f98b8003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98ce7fc700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b8004000 0x7f98b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cf7fe700 1 -- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d00ac6d0 msgr2=0x7f98b8003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cf7fe700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d00ac6d0 0x7f98b8003a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cf7fe700 1 -- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b8004000 msgr2=0x7f98b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cf7fe700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98b8004000 0x7f98b8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.275+0000 7f98cf7fe700 1 -- 172.21.15.119:0/2003833594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8003570 con 0x7f98b8004d50 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.276+0000 7f98cf7fe700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98b80051c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e630 tx=0x7f98c800a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de5a04ee56a8d38 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.276+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c800ab80 con 0x7f98b8004d50 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.276+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c800c450 con 0x7f98b8004d50 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.276+0000 7f98cdffb700 1 -- 172.21.15.119:0/2003833594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c800c810 con 0x7f98b8004d50 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.276+0000 7f98d57ff700 1 -- 172.21.15.119:0/2003833594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 msgr2=0x7f98b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.276+0000 7f98d57ff700 1 --2- 172.21.15.119:0/2003833594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98b80051c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f98c800e630 tx=0x7f98c800a6a0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.276+0000 7f98cf7fe700 1 -- 172.21.15.119:0/2003833594 reap_dead start 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.277+0000 7f98d57ff700 1 -- 172.21.15.119:0/2003833594 shutdown_connections 2023-10-08T02:28:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.277+0000 7f98d57ff700 1 -- 172.21.15.119:0/2003833594 >> 172.21.15.119:0/2003833594 conn(0x7f98d006a320 msgr2=0x7f98d00cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.277+0000 7f98d57ff700 1 -- 172.21.15.119:0/2003833594 shutdown_connections 2023-10-08T02:28:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.277+0000 7f98d57ff700 1 -- 172.21.15.119:0/2003833594 wait complete. 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.278+0000 7f98d57ff700 1 Processor -- start 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.278+0000 7f98d57ff700 1 -- start start 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98d57ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98d0145070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98d57ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d0150be0 0x7f98d014e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98d57ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d0151050 0x7f98d014ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98d57ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d014c230 con 0x7f98b8004d50 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98d57ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d014cdb0 con 0x7f98d0150be0 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98d57ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d014c480 con 0x7f98d0151050 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98ceffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98d0145070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98ceffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98d0145070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37664/0 (socket says 172.21.15.119:37664) 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98ceffd700 1 -- 172.21.15.119:0/1854455602 learned_addr learned my addr 172.21.15.119:0/1854455602 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98ce7fc700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d0150be0 0x7f98d014e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.279+0000 7f98cf7fe700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d0151050 0x7f98d014ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98ceffd700 1 -- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d0151050 msgr2=0x7f98d014ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98ceffd700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d0151050 0x7f98d014ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98ceffd700 1 -- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d0150be0 msgr2=0x7f98d014e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98ceffd700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d0150be0 0x7f98d014e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98ceffd700 1 -- 172.21.15.119:0/1854455602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d014c760 con 0x7f98b8004d50 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98ce7fc700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d0150be0 0x7f98d014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98ceffd700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98d0145070 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f98c0011b10 tx=0x7f98c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98b6ffd700 1 -- 172.21.15.119:0/1854455602 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000f990 con 0x7f98b8004d50 2023-10-08T02:28:51.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98b6ffd700 1 -- 172.21.15.119:0/1854455602 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000faf0 con 0x7f98b8004d50 2023-10-08T02:28:51.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d014ca40 con 0x7f98b8004d50 2023-10-08T02:28:51.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98b6ffd700 1 -- 172.21.15.119:0/1854455602 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000fe70 con 0x7f98b8004d50 2023-10-08T02:28:51.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.280+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d014d240 con 0x7f98b8004d50 2023-10-08T02:28:51.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.281+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d004c3c0 con 0x7f98b8004d50 2023-10-08T02:28:51.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.282+0000 7f98b6ffd700 1 -- 172.21.15.119:0/1854455602 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f98c0014650 con 0x7f98b8004d50 2023-10-08T02:28:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.283+0000 7f98b6ffd700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f98bc066f60 0x7f98bc069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.284+0000 7f98b6ffd700 1 -- 172.21.15.119:0/1854455602 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f98c0018030 con 0x7f98b8004d50 2023-10-08T02:28:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.284+0000 7f98ce7fc700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f98bc066f60 0x7f98bc069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.284+0000 7f98ce7fc700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f98bc066f60 0x7f98bc069410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f98c4001200 tx=0x7f98c4009040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.287+0000 7f98b6ffd700 1 -- 172.21.15.119:0/1854455602 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f98c0055c30 con 0x7f98b8004d50 2023-10-08T02:28:51.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.521+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98d00ad2e0 con 0x7f98b8004d50 2023-10-08T02:28:51.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.522+0000 7f98b6ffd700 1 -- 172.21.15.119:0/1854455602 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7f98c00557c0 con 0x7f98b8004d50 2023-10-08T02:28:51.522 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T02:28:51.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f98bc066f60 msgr2=0x7f98bc069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f98bc066f60 0x7f98bc069410 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f98c4001200 tx=0x7f98c4009040 comp rx=0 tx=0).stop 2023-10-08T02:28:51.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 msgr2=0x7f98d0145070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:51.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98d0145070 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f98c0011b10 tx=0x7f98c00091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 shutdown_connections 2023-10-08T02:28:51.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f98bc066f60 0x7f98bc069410 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f98d0151050 0x7f98d014ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f98b8004d50 0x7f98d0145070 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 --2- 172.21.15.119:0/1854455602 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f98d0150be0 0x7f98d014e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:51.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 >> 172.21.15.119:0/1854455602 conn(0x7f98d006a320 msgr2=0x7f98d00ae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:51.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 shutdown_connections 2023-10-08T02:28:51.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:51.524+0000 7f98d57ff700 1 -- 172.21.15.119:0/1854455602 wait complete. 2023-10-08T02:28:51.541 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T02:28:51.541 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T02:28:51.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T02:28:51.541 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T02:28:51.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T02:28:51.541 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T02:28:51.541 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T02:28:51.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T02:28:51.542 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:28:51.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:28:51.542 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T02:28:51.579 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T02:28:51.579 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T02:28:51.579 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T02:28:51.579 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:28:51.579 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T02:28:51.579 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T02:28:51.580 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 36720sec preferred_lft 36720sec 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:28:51.581 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T02:28:51.582 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:28:51.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:28:51.583 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T02:28:51.677 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T02:28:51.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:28:51.681 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T02:28:51.719 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T02:28:51.719 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T02:28:51.719 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T02:28:51.719 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T02:28:51.719 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T02:28:51.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:28:51.721 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T02:28:51.721 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T02:28:51.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T02:28:51.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T02:28:51.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T02:28:51.721 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T02:28:51.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:28:51.915 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T02:28:51.951 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T02:28:51.951 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T02:28:51.952 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:28:51.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:28:51.952 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T02:28:52.052 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T02:28:52.052 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T02:28:52.089 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:28:52.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T02:28:52.089 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T02:28:52.165 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T02:28:52.165 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T02:28:52.165 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:28:52.296 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:28:52.296 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T02:28:52.325 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:28:52.326 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T02:28:52.326 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T02:28:52.326 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T02:28:52.326 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T02:28:52.330 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T02:28:52.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:28:52.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T02:28:52.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T02:28:52.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.578+0000 7f6cf11ff700 1 Processor -- start 2023-10-08T02:28:52.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.578+0000 7f6cf11ff700 1 -- start start 2023-10-08T02:28:52.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.579+0000 7f6cf11ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec06f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.579+0000 7f6cf11ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6cec06f640 0x7f6cec159510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.579+0000 7f6cf11ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec06ec70 con 0x7f6cec068550 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.579+0000 7f6cf11ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec068d90 con 0x7f6cec06f640 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.579+0000 7f6cf11ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b3430 con 0x7f6cec068920 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6ceb59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6cec06f640 0x7f6cec159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6ceb59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6cec06f640 0x7f6cec159510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37666/0 (socket says 172.21.15.119:37666) 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6ceb59e700 1 -- 172.21.15.119:0/4249820949 learned_addr learned my addr 172.21.15.119:0/4249820949 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6cea59c700 1 --2- 172.21.15.119:0/4249820949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec06f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6ceb59e700 1 -- 172.21.15.119:0/4249820949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 msgr2=0x7f6cec06f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6ceb59e700 1 --2- 172.21.15.119:0/4249820949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec06f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6ceb59e700 1 -- 172.21.15.119:0/4249820949 >> v1:172.21.15.184:6789/0 conn(0x7f6cec068550 legacy=0x7f6cec15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:52.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6ceb59e700 1 -- 172.21.15.119:0/4249820949 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0b2e70 con 0x7f6cec06f640 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.580+0000 7f6ceb59e700 1 --2- 172.21.15.119:0/4249820949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec06f640 0x7f6cec159510 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009f20 tx=0x7f6ce4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cedd79d52438e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6ce9d9b700 1 -- 172.21.15.119:0/4249820949 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce400e070 con 0x7f6cec06f640 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6ce9d9b700 1 -- 172.21.15.119:0/4249820949 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce400ba80 con 0x7f6cec06f640 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6ce9d9b700 1 -- 172.21.15.119:0/4249820949 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40092e0 con 0x7f6cec06f640 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6cf11ff700 1 -- 172.21.15.119:0/4249820949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec06f640 msgr2=0x7f6cec159510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/4249820949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec06f640 0x7f6cec159510 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009f20 tx=0x7f6ce4009840 comp rx=0 tx=0).stop 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6cf11ff700 1 -- 172.21.15.119:0/4249820949 shutdown_connections 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/4249820949 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec06f640 0x7f6cec159510 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/4249820949 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec06f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6cf11ff700 1 -- 172.21.15.119:0/4249820949 >> 172.21.15.119:0/4249820949 conn(0x7f6cec0cd480 msgr2=0x7f6cec0cd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:52.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6cf11ff700 1 -- 172.21.15.119:0/4249820949 shutdown_connections 2023-10-08T02:28:52.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.581+0000 7f6cf11ff700 1 -- 172.21.15.119:0/4249820949 wait complete. 2023-10-08T02:28:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.583+0000 7f6cf11ff700 1 Processor -- start 2023-10-08T02:28:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.583+0000 7f6cf11ff700 1 -- start start 2023-10-08T02:28:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cf11ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec068550 0x7f6cec14da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cf11ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec14c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cf11ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cec06f640 0x7f6cec151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cf11ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec151660 con 0x7f6cec068550 2023-10-08T02:28:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cf11ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec150a50 con 0x7f6cec06f640 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cf11ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec150d30 con 0x7f6cec068920 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cead9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec068550 0x7f6cec14da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cead9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec068550 0x7f6cec14da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37682/0 (socket says 172.21.15.119:37682) 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cead9d700 1 -- 172.21.15.119:0/2375690380 learned_addr learned my addr 172.21.15.119:0/2375690380 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6ceb59e700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cec06f640 0x7f6cec151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cea59c700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec14c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cead9d700 1 -- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 msgr2=0x7f6cec14c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cead9d700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec14c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cead9d700 1 -- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cec06f640 msgr2=0x7f6cec151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cead9d700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cec06f640 0x7f6cec151af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cead9d700 1 -- 172.21.15.119:0/2375690380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009ae0 con 0x7f6cec068550 2023-10-08T02:28:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.584+0000 7f6cea59c700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec14c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:28:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.585+0000 7f6cead9d700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec068550 0x7f6cec14da20 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c590 tx=0x7f6cdc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.585+0000 7f6cd2ffd700 1 -- 172.21.15.119:0/2375690380 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc011590 con 0x7f6cec068550 2023-10-08T02:28:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.585+0000 7f6cd2ffd700 1 -- 172.21.15.119:0/2375690380 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc011710 con 0x7f6cec068550 2023-10-08T02:28:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.585+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec151010 con 0x7f6cec068550 2023-10-08T02:28:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.585+0000 7f6cd2ffd700 1 -- 172.21.15.119:0/2375690380 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc011ad0 con 0x7f6cec068550 2023-10-08T02:28:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.585+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec14df60 con 0x7f6cec068550 2023-10-08T02:28:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.586+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec04c3c0 con 0x7f6cec068550 2023-10-08T02:28:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.587+0000 7f6cd2ffd700 1 -- 172.21.15.119:0/2375690380 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cdc01b3b0 con 0x7f6cec068550 2023-10-08T02:28:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.588+0000 7f6cd2ffd700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cd8066f30 0x7f6cd80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.588+0000 7f6cd2ffd700 1 -- 172.21.15.119:0/2375690380 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f6cdc083c50 con 0x7f6cec068550 2023-10-08T02:28:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.588+0000 7f6cea59c700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cd8066f30 0x7f6cd80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.589+0000 7f6cea59c700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cd8066f30 0x7f6cd80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0006fd0 tx=0x7f6ce0008040 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:52.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.592+0000 7f6cd2ffd700 1 -- 172.21.15.119:0/2375690380 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cdc089020 con 0x7f6cec068550 2023-10-08T02:28:52.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.808+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cec1517a0 con 0x7f6cd8066f30 2023-10-08T02:28:52.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.809+0000 7f6cd2ffd700 1 -- 172.21.15.119:0/2375690380 <== mgr.9543 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6ce0007890 con 0x7f6cd8066f30 2023-10-08T02:28:52.809 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T02:28:52.809 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:28:52.809 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T02:28:52.809 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:28:52.809 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T02:28:52.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cd8066f30 msgr2=0x7f6cd80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cd8066f30 0x7f6cd80693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0006fd0 tx=0x7f6ce0008040 comp rx=0 tx=0).stop 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec068550 msgr2=0x7f6cec14da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec068550 0x7f6cec14da20 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c590 tx=0x7f6cdc00c8a0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 shutdown_connections 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cd8066f30 0x7f6cd80693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6cec06f640 0x7f6cec151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6cec068550 0x7f6cec14da20 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 --2- 172.21.15.119:0/2375690380 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6cec068920 0x7f6cec14c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 >> 172.21.15.119:0/2375690380 conn(0x7f6cec0cd480 msgr2=0x7f6cec0aeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:52.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 shutdown_connections 2023-10-08T02:28:52.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:52.811+0000 7f6cf11ff700 1 -- 172.21.15.119:0/2375690380 wait complete. 2023-10-08T02:28:52.829 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T02:28:53.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.013+0000 7fed81d9d700 1 Processor -- start 2023-10-08T02:28:53.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.013+0000 7fed81d9d700 1 -- start start 2023-10-08T02:28:53.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.014+0000 7fed81d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed7c0aac80 0x7fed7c0ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.014+0000 7fed81d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c0c89a0 con 0x7fed7c159de0 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.014+0000 7fed81d9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed7c0c7510 con 0x7fed7c159a10 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.014+0000 7fed81d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed7c0caee0 con 0x7fed7c0aac80 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.014+0000 7fed7bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed7c0aac80 0x7fed7c0ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.014+0000 7fed7bfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed7c0aac80 0x7fed7c0ca9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37698/0 (socket says 172.21.15.119:37698) 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.014+0000 7fed7bfff700 1 -- 172.21.15.119:0/2278363739 learned_addr learned my addr 172.21.15.119:0/2278363739 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.015+0000 7fed7bfff700 1 -- 172.21.15.119:0/2278363739 >> v1:172.21.15.184:6789/0 conn(0x7fed7c159a10 legacy=0x7fed7c0c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.015+0000 7fed7bfff700 1 -- 172.21.15.119:0/2278363739 >> v1:172.21.15.119:6789/0 conn(0x7fed7c159de0 legacy=0x7fed7c15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.015+0000 7fed7bfff700 1 -- 172.21.15.119:0/2278363739 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0c77f0 con 0x7fed7c0aac80 2023-10-08T02:28:53.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.015+0000 7fed7bfff700 1 --2- 172.21.15.119:0/2278363739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c0ca9a0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fed680099b0 tx=0x7fed68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f03a30bd395039 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:53.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.015+0000 7fed7b7fe700 1 -- 172.21.15.119:0/2278363739 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800b860 con 0x7fed7c0aac80 2023-10-08T02:28:53.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.015+0000 7fed7b7fe700 1 -- 172.21.15.119:0/2278363739 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed6800b9c0 con 0x7fed7c0aac80 2023-10-08T02:28:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.015+0000 7fed7b7fe700 1 -- 172.21.15.119:0/2278363739 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6800bd60 con 0x7fed7c0aac80 2023-10-08T02:28:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.016+0000 7fed81d9d700 1 -- 172.21.15.119:0/2278363739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 msgr2=0x7fed7c0ca9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.016+0000 7fed81d9d700 1 --2- 172.21.15.119:0/2278363739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c0ca9a0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fed680099b0 tx=0x7fed68009250 comp rx=0 tx=0).stop 2023-10-08T02:28:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.016+0000 7fed81d9d700 1 -- 172.21.15.119:0/2278363739 shutdown_connections 2023-10-08T02:28:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.016+0000 7fed81d9d700 1 --2- 172.21.15.119:0/2278363739 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c0ca9a0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.016+0000 7fed81d9d700 1 -- 172.21.15.119:0/2278363739 >> 172.21.15.119:0/2278363739 conn(0x7fed7c0cc1e0 msgr2=0x7fed7c0cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.016+0000 7fed81d9d700 1 -- 172.21.15.119:0/2278363739 shutdown_connections 2023-10-08T02:28:53.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.016+0000 7fed81d9d700 1 -- 172.21.15.119:0/2278363739 wait complete. 2023-10-08T02:28:53.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.017+0000 7fed81d9d700 1 Processor -- start 2023-10-08T02:28:53.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.018+0000 7fed81d9d700 1 -- start start 2023-10-08T02:28:53.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.018+0000 7fed81d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c169a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.018+0000 7fed81d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7c159a10 0x7fed7c150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.018+0000 7fed81d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed7c159de0 0x7fed7c150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.018+0000 7fed81d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c14f8a0 con 0x7fed7c0aac80 2023-10-08T02:28:53.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.018+0000 7fed81d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0b1b50 con 0x7fed7c159a10 2023-10-08T02:28:53.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.018+0000 7fed81d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c0b0f40 con 0x7fed7c159de0 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c169a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c169a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37710/0 (socket says 172.21.15.119:37710) 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 -- 172.21.15.119:0/388191732 learned_addr learned my addr 172.21.15.119:0/388191732 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed7bfff700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7c159a10 0x7fed7c150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed8159c700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed7c159de0 0x7fed7c150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 -- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed7c159de0 msgr2=0x7fed7c150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed7c159de0 0x7fed7c150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 -- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7c159a10 msgr2=0x7fed7c150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7c159a10 0x7fed7c150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 -- 172.21.15.119:0/388191732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed680094f0 con 0x7fed7c0aac80 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed8159c700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed7c159de0 0x7fed7c150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:28:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.019+0000 7fed80d9b700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c169a50 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fed7000c590 tx=0x7fed7000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.020+0000 7fed78ff9700 1 -- 172.21.15.119:0/388191732 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70011590 con 0x7fed7c0aac80 2023-10-08T02:28:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.020+0000 7fed78ff9700 1 -- 172.21.15.119:0/388191732 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed700116f0 con 0x7fed7c0aac80 2023-10-08T02:28:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.020+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c0b1220 con 0x7fed7c0aac80 2023-10-08T02:28:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.020+0000 7fed78ff9700 1 -- 172.21.15.119:0/388191732 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed70011ab0 con 0x7fed7c0aac80 2023-10-08T02:28:53.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.020+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c0b17e0 con 0x7fed7c0aac80 2023-10-08T02:28:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.021+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c06c450 con 0x7fed7c0aac80 2023-10-08T02:28:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.022+0000 7fed78ff9700 1 -- 172.21.15.119:0/388191732 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 102) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fed7001b3b0 con 0x7fed7c0aac80 2023-10-08T02:28:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.023+0000 7fed78ff9700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fed60066f30 0x7fed600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.023+0000 7fed78ff9700 1 -- 172.21.15.119:0/388191732 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fed70083cd0 con 0x7fed7c0aac80 2023-10-08T02:28:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.023+0000 7fed7bfff700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fed60066f30 0x7fed600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.024+0000 7fed7bfff700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fed60066f30 0x7fed600693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed680099b0 tx=0x7fed68017000 comp rx=0 tx=0).ready entity=mgr.9543 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.027+0000 7fed78ff9700 1 -- 172.21.15.119:0/388191732 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed70050d50 con 0x7fed7c0aac80 2023-10-08T02:28:53.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.272+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fed7c04bb80 con 0x7fed7c0aac80 2023-10-08T02:28:53.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.346+0000 7fed78ff9700 1 -- 172.21.15.119:0/388191732 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 103) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fed70051120 con 0x7fed7c0aac80 2023-10-08T02:28:53.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.347+0000 7fed78ff9700 1 -- 172.21.15.119:0/388191732 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v103) v1 ==== 92+0+0 (secure 0 0 0) 0x7fed70050d50 con 0x7fed7c0aac80 2023-10-08T02:28:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fed60066f30 msgr2=0x7fed600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fed60066f30 0x7fed600693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed680099b0 tx=0x7fed68017000 comp rx=0 tx=0).stop 2023-10-08T02:28:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 msgr2=0x7fed7c169a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c169a50 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fed7000c590 tx=0x7fed7000c950 comp rx=0 tx=0).stop 2023-10-08T02:28:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 shutdown_connections 2023-10-08T02:28:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fed60066f30 0x7fed600693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed7c159de0 0x7fed7c150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed7c0aac80 0x7fed7c169a50 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 --2- 172.21.15.119:0/388191732 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed7c159a10 0x7fed7c150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 >> 172.21.15.119:0/388191732 conn(0x7fed7c0cc1e0 msgr2=0x7fed7c06e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 shutdown_connections 2023-10-08T02:28:53.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.350+0000 7fed81d9d700 1 -- 172.21.15.119:0/388191732 wait complete. 2023-10-08T02:28:53.368 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T02:28:53.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.561+0000 7fb2aef9d700 1 Processor -- start 2023-10-08T02:28:53.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.561+0000 7fb2aef9d700 1 -- start start 2023-10-08T02:28:53.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.562+0000 7fb2aef9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b16d0 0x7fb2a80b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.562+0000 7fb2aef9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2a80b33b0 0x7fb2a80b3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.562+0000 7fb2aef9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a80b2080 con 0x7fb2a80b4e70 2023-10-08T02:28:53.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.562+0000 7fb2aef9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a80cfb00 con 0x7fb2a80b33b0 2023-10-08T02:28:53.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.562+0000 7fb2aef9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80b5c40 con 0x7fb2a80b16d0 2023-10-08T02:28:53.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.562+0000 7fb2ad79a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b16d0 0x7fb2a80b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.562+0000 7fb2ae79c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2a80b33b0 0x7fb2a80b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ad79a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b16d0 0x7fb2a80b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:40438/0 (socket says 172.21.15.119:40438) 2023-10-08T02:28:53.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ae79c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb2a80b33b0 0x7fb2a80b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:54090/0 (socket says 172.21.15.119:54090) 2023-10-08T02:28:53.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ad79a700 1 -- 172.21.15.119:0/1586211672 learned_addr learned my addr 172.21.15.119:0/1586211672 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:53.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2adf9b700 1 --1- 172.21.15.119:0/1586211672 >> v1:172.21.15.184:6789/0 conn(0x7fb2a80b4e70 0x7fb2a80ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:32768/0 (socket says 172.21.15.119:32768) 2023-10-08T02:28:53.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ae79c700 1 -- 172.21.15.119:0/1586211672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b16d0 msgr2=0x7fb2a80b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:53.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ae79c700 1 --2- 172.21.15.119:0/1586211672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b16d0 0x7fb2a80b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ae79c700 1 -- 172.21.15.119:0/1586211672 >> v1:172.21.15.184:6789/0 conn(0x7fb2a80b4e70 legacy=0x7fb2a80ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ae79c700 1 -- 172.21.15.119:0/1586211672 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80b5f20 con 0x7fb2a80b33b0 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2acf99700 1 -- 172.21.15.119:0/1586211672 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782051761 0 0) 0x7fb2a80b2080 con 0x7fb2a80b4e70 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ad79a700 1 --2- 172.21.15.119:0/1586211672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b16d0 0x7fb2a80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.563+0000 7fb2ae79c700 1 --2- 172.21.15.119:0/1586211672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b33b0 0x7fb2a80b3830 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009e60 tx=0x7fb2a0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1688e45090cea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2acf99700 1 -- 172.21.15.119:0/1586211672 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a000e070 con 0x7fb2a80b33b0 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2acf99700 1 -- 172.21.15.119:0/1586211672 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a000bb60 con 0x7fb2a80b33b0 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2acf99700 1 -- 172.21.15.119:0/1586211672 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0012430 con 0x7fb2a80b33b0 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1586211672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b33b0 msgr2=0x7fb2a80b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1586211672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b33b0 0x7fb2a80b3830 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009e60 tx=0x7fb2a0009780 comp rx=0 tx=0).stop 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1586211672 shutdown_connections 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1586211672 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b33b0 0x7fb2a80b3830 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1586211672 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b16d0 0x7fb2a80b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1586211672 >> 172.21.15.119:0/1586211672 conn(0x7fb2a80d10c0 msgr2=0x7fb2a80affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:53.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1586211672 shutdown_connections 2023-10-08T02:28:53.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.564+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1586211672 wait complete. 2023-10-08T02:28:53.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.566+0000 7fb2aef9d700 1 Processor -- start 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.566+0000 7fb2aef9d700 1 -- start start 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2aef9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b16d0 0x7fb2a80ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2aef9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80b33b0 0x7fb2a8068a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2aef9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b4e70 0x7fb2a8068f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2aef9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80c6660 con 0x7fb2a80b33b0 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2aef9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a816dfa0 con 0x7fb2a80b16d0 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2aef9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80c5d30 con 0x7fb2a80b4e70 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ad79a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80b33b0 0x7fb2a8068a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ad79a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80b33b0 0x7fb2a8068a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37726/0 (socket says 172.21.15.119:37726) 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ad79a700 1 -- 172.21.15.119:0/1959810451 learned_addr learned my addr 172.21.15.119:0/1959810451 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:53.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2adf9b700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b16d0 0x7fb2a80ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ae79c700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b4e70 0x7fb2a8068f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ad79a700 1 -- 172.21.15.119:0/1959810451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b4e70 msgr2=0x7fb2a8068f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ad79a700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b4e70 0x7fb2a8068f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ad79a700 1 -- 172.21.15.119:0/1959810451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b16d0 msgr2=0x7fb2a80ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ad79a700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b16d0 0x7fb2a80ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.567+0000 7fb2ad79a700 1 -- 172.21.15.119:0/1959810451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980094f0 con 0x7fb2a80b33b0 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.568+0000 7fb2ad79a700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80b33b0 0x7fb2a8068a00 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fb29800ec50 tx=0x7fb298009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.568+0000 7fb29dffb700 1 -- 172.21.15.119:0/1959810451 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29800c850 con 0x7fb2a80b33b0 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.568+0000 7fb29dffb700 1 -- 172.21.15.119:0/1959810451 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29800c9b0 con 0x7fb2a80b33b0 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.568+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0009a20 con 0x7fb2a80b33b0 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.568+0000 7fb29dffb700 1 -- 172.21.15.119:0/1959810451 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29800cd50 con 0x7fb2a80b33b0 2023-10-08T02:28:53.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.568+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a80c62f0 con 0x7fb2a80b33b0 2023-10-08T02:28:53.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.569+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a8088990 con 0x7fb2a80b33b0 2023-10-08T02:28:53.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.570+0000 7fb29dffb700 1 -- 172.21.15.119:0/1959810451 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 103) v1 ==== 84757+0+0 (secure 0 0 0) 0x7fb298013680 con 0x7fb2a80b33b0 2023-10-08T02:28:53.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.571+0000 7fb29dffb700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb28c066f80 0x7fb28c069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.571+0000 7fb29dffb700 1 -- 172.21.15.119:0/1959810451 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fb298084ba0 con 0x7fb2a80b33b0 2023-10-08T02:28:53.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.571+0000 7fb2adf9b700 1 -- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb28c066f80 msgr2=0x7fb28c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:53.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.571+0000 7fb2adf9b700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb28c066f80 0x7fb28c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:53.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.575+0000 7fb29dffb700 1 -- 172.21.15.119:0/1959810451 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb298051c20 con 0x7fb2a80b33b0 2023-10-08T02:28:53.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.772+0000 7fb2adf9b700 1 -- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb28c066f80 msgr2=0x7fb28c069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:53.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.772+0000 7fb2adf9b700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb28c066f80 0x7fb28c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:53.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.775+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb2a80c67a0 con 0x7fb2a80b33b0 2023-10-08T02:28:53.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.781+0000 7fb29dffb700 1 -- 172.21.15.119:0/1959810451 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb298055240 con 0x7fb2a80b33b0 2023-10-08T02:28:53.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb28c066f80 msgr2=0x7fb28c069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:53.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb28c066f80 0x7fb28c069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80b33b0 msgr2=0x7fb2a8068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80b33b0 0x7fb2a8068a00 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fb29800ec50 tx=0x7fb298009250 comp rx=0 tx=0).stop 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 shutdown_connections 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb28c066f80 0x7fb28c069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb2a80b4e70 0x7fb2a8068f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb2a80b16d0 0x7fb2a80ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 --2- 172.21.15.119:0/1959810451 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb2a80b33b0 0x7fb2a8068a00 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.783+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 >> 172.21.15.119:0/1959810451 conn(0x7fb2a80d10c0 msgr2=0x7fb2a8158ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.784+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 shutdown_connections 2023-10-08T02:28:53.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.784+0000 7fb2aef9d700 1 -- 172.21.15.119:0/1959810451 wait complete. 2023-10-08T02:28:53.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T02:28:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.990+0000 7f366ab9d700 1 Processor -- start 2023-10-08T02:28:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.990+0000 7f366ab9d700 1 -- start start 2023-10-08T02:28:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366ab9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366ab9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36640b16d0 0x7f36640b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366ab9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f366415d240 0x7f366415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366ab9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36640b2080 con 0x7f36640b16d0 2023-10-08T02:28:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366ab9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36640cfb00 con 0x7f366415d240 2023-10-08T02:28:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366ab9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640b5c40 con 0x7f36640b4e70 2023-10-08T02:28:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366939a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36640b16d0 0x7f36640b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f3669b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366a39c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f366415d240 0x7f366415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366939a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f36640b16d0 0x7f36640b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37730/0 (socket says 172.21.15.119:37730) 2023-10-08T02:28:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f366939a700 1 -- 172.21.15.119:0/3889697723 learned_addr learned my addr 172.21.15.119:0/3889697723 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f3669b9b700 1 -- 172.21.15.119:0/3889697723 >> v2:172.21.15.184:3300/0 conn(0x7f366415d240 msgr2=0x7f366415d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.992+0000 7f3669b9b700 1 --2- 172.21.15.119:0/3889697723 >> v2:172.21.15.184:3300/0 conn(0x7f366415d240 0x7f366415d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.993+0000 7f3669b9b700 1 -- 172.21.15.119:0/3889697723 >> v2:172.21.15.119:3300/0 conn(0x7f36640b16d0 msgr2=0x7f36640b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.993+0000 7f3669b9b700 1 --2- 172.21.15.119:0/3889697723 >> v2:172.21.15.119:3300/0 conn(0x7f36640b16d0 0x7f36640b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.993+0000 7f3669b9b700 1 -- 172.21.15.119:0/3889697723 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36640b5f20 con 0x7f36640b4e70 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.993+0000 7f366939a700 1 --2- 172.21.15.119:0/3889697723 >> v2:172.21.15.119:3300/0 conn(0x7f36640b16d0 0x7f36640b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.993+0000 7f3669b9b700 1 --2- 172.21.15.119:0/3889697723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640ccff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f366415d180 tx=0x7f36600091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d83f1a50cf2e4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.993+0000 7f3668b99700 1 -- 172.21.15.119:0/3889697723 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660011630 con 0x7f36640b4e70 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.993+0000 7f3668b99700 1 -- 172.21.15.119:0/3889697723 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36600117b0 con 0x7f36640b4e70 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.993+0000 7f3668b99700 1 -- 172.21.15.119:0/3889697723 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660011b70 con 0x7f36640b4e70 2023-10-08T02:28:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 -- 172.21.15.119:0/3889697723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 msgr2=0x7f36640ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 --2- 172.21.15.119:0/3889697723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640ccff0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f366415d180 tx=0x7f36600091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 -- 172.21.15.119:0/3889697723 shutdown_connections 2023-10-08T02:28:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 --2- 172.21.15.119:0/3889697723 >> v2:172.21.15.184:3300/0 conn(0x7f366415d240 0x7f366415d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 --2- 172.21.15.119:0/3889697723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640ccff0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 --2- 172.21.15.119:0/3889697723 >> v2:172.21.15.119:3300/0 conn(0x7f36640b16d0 0x7f36640b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 -- 172.21.15.119:0/3889697723 >> 172.21.15.119:0/3889697723 conn(0x7f36640d10c0 msgr2=0x7f36640affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 -- 172.21.15.119:0/3889697723 shutdown_connections 2023-10-08T02:28:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.994+0000 7f366ab9d700 1 -- 172.21.15.119:0/3889697723 wait complete. 2023-10-08T02:28:53.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.995+0000 7f366ab9d700 1 Processor -- start 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.995+0000 7f366ab9d700 1 -- start start 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.996+0000 7f366ab9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36640b16d0 0x7f36640cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.996+0000 7f366ab9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.996+0000 7f366ab9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f366415d240 0x7f36640c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.996+0000 7f366ab9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3664169fd0 con 0x7f366415d240 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.996+0000 7f366ab9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36640cb1a0 con 0x7f36640b16d0 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.996+0000 7f366ab9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640c81f0 con 0x7f36640b4e70 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f366415d240 0x7f36640c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:54.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366939a700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f366415d240 0x7f36640c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37732/0 (socket says 172.21.15.119:37732) 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 -- 172.21.15.119:0/2065634765 learned_addr learned my addr 172.21.15.119:0/2065634765 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f3669b9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36640b16d0 0x7f36640cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 -- 172.21.15.119:0/2065634765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 msgr2=0x7f36640c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 -- 172.21.15.119:0/2065634765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36640b16d0 msgr2=0x7f36640cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36640b16d0 0x7f36640cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 -- 172.21.15.119:0/2065634765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36540094f0 con 0x7f366415d240 2023-10-08T02:28:54.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.997+0000 7f366a39c700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f366415d240 0x7f36640c8b10 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f365c009e20 tx=0x7f365c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.998+0000 7f3659ffb700 1 -- 172.21.15.119:0/2065634765 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c00caa0 con 0x7f366415d240 2023-10-08T02:28:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.998+0000 7f3659ffb700 1 -- 172.21.15.119:0/2065634765 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c00f040 con 0x7f366415d240 2023-10-08T02:28:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.998+0000 7f3659ffb700 1 -- 172.21.15.119:0/2065634765 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c00ce70 con 0x7f366415d240 2023-10-08T02:28:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.998+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3660009420 con 0x7f366415d240 2023-10-08T02:28:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.998+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36640c9420 con 0x7f366415d240 2023-10-08T02:28:54.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:53.999+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366416a180 con 0x7f366415d240 2023-10-08T02:28:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.001+0000 7f3659ffb700 1 -- 172.21.15.119:0/2065634765 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 103) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f365c014690 con 0x7f366415d240 2023-10-08T02:28:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.002+0000 7f3659ffb700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3648066f80 0x7f3648069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.002+0000 7f3669b9b700 1 -- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3648066f80 msgr2=0x7f3648069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.002+0000 7f3659ffb700 1 -- 172.21.15.119:0/2065634765 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f365c084f70 con 0x7f366415d240 2023-10-08T02:28:54.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.002+0000 7f3669b9b700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3648066f80 0x7f3648069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:54.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.006+0000 7f3659ffb700 1 -- 172.21.15.119:0/2065634765 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f365c055a80 con 0x7f366415d240 2023-10-08T02:28:54.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.203+0000 7f3669b9b700 1 -- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3648066f80 msgr2=0x7f3648069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:54.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.203+0000 7f3669b9b700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3648066f80 0x7f3648069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:54.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.207+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f366415f5c0 con 0x7f366415d240 2023-10-08T02:28:54.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.211+0000 7f3659ffb700 1 -- 172.21.15.119:0/2065634765 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7f365c055610 con 0x7f366415d240 2023-10-08T02:28:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3648066f80 msgr2=0x7f3648069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3648066f80 0x7f3648069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f366415d240 msgr2=0x7f36640c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f366415d240 0x7f36640c8b10 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f365c009e20 tx=0x7f365c00b040 comp rx=0 tx=0).stop 2023-10-08T02:28:54.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 shutdown_connections 2023-10-08T02:28:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3648066f80 0x7f3648069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f366415d240 0x7f36640c8b10 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36640b16d0 0x7f36640cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 --2- 172.21.15.119:0/2065634765 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f36640b4e70 0x7f36640c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 >> 172.21.15.119:0/2065634765 conn(0x7f36640d10c0 msgr2=0x7f366415c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 shutdown_connections 2023-10-08T02:28:54.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.213+0000 7f366ab9d700 1 -- 172.21.15.119:0/2065634765 wait complete. 2023-10-08T02:28:54.231 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:28:54.231 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T02:28:54.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-08T02:28:54.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.506+0000 7f71b559d700 1 Processor -- start 2023-10-08T02:28:54.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.506+0000 7f71b559d700 1 -- start start 2023-10-08T02:28:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.507+0000 7f71b559d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f71b00ac0e0 0x7f71b00afa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.507+0000 7f71b559d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b006e510 con 0x7f71b006f0b0 2023-10-08T02:28:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.507+0000 7f71b559d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b0069180 con 0x7f71b01597d0 2023-10-08T02:28:54.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.507+0000 7f71b559d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b006dec0 con 0x7f71b00ac0e0 2023-10-08T02:28:54.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71b4d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f71b006f0b0 0x7f71b015d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54670/0 (socket says 172.21.15.119:54670) 2023-10-08T02:28:54.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71b4d9c700 1 -- 172.21.15.119:0/887138083 learned_addr learned my addr 172.21.15.119:0/887138083 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:28:54.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71af7fe700 1 --2- 172.21.15.119:0/887138083 >> v2:172.21.15.184:3300/0 conn(0x7f71b00ac0e0 0x7f71b00afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71aeffd700 1 -- 172.21.15.119:0/887138083 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 691142323 0 0) 0x7f71b006e510 con 0x7f71b006f0b0 2023-10-08T02:28:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71aeffd700 1 -- 172.21.15.119:0/887138083 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7198003570 con 0x7f71b006f0b0 2023-10-08T02:28:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71af7fe700 1 -- 172.21.15.119:0/887138083 >> v1:172.21.15.184:6789/0 conn(0x7f71b01597d0 legacy=0x7f71b006e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:54.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71aeffd700 1 -- 172.21.15.119:0/887138083 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781624482 0 0) 0x7f71b0069180 con 0x7f71b01597d0 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71af7fe700 1 -- 172.21.15.119:0/887138083 >> v1:172.21.15.119:6789/0 conn(0x7f71b006f0b0 legacy=0x7f71b015d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71af7fe700 1 -- 172.21.15.119:0/887138083 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b006e1a0 con 0x7f71b00ac0e0 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.508+0000 7f71af7fe700 1 --2- 172.21.15.119:0/887138083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b00ac0e0 0x7f71b00afa40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f71a40098e0 tx=0x7f71a40091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e13a3113e1f7810 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71aeffd700 1 -- 172.21.15.119:0/887138083 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a400b640 con 0x7f71b00ac0e0 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71aeffd700 1 -- 172.21.15.119:0/887138083 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a400b7a0 con 0x7f71b00ac0e0 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71aeffd700 1 -- 172.21.15.119:0/887138083 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a400bb40 con 0x7f71b00ac0e0 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71b559d700 1 -- 172.21.15.119:0/887138083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b00ac0e0 msgr2=0x7f71b00afa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71b559d700 1 --2- 172.21.15.119:0/887138083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b00ac0e0 0x7f71b00afa40 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f71a40098e0 tx=0x7f71a40091a0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71b559d700 1 -- 172.21.15.119:0/887138083 shutdown_connections 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71b559d700 1 --2- 172.21.15.119:0/887138083 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b00ac0e0 0x7f71b00afa40 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71b559d700 1 -- 172.21.15.119:0/887138083 >> 172.21.15.119:0/887138083 conn(0x7f71b006adf0 msgr2=0x7f71b00ccb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:28:54.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71b559d700 1 -- 172.21.15.119:0/887138083 shutdown_connections 2023-10-08T02:28:54.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.509+0000 7f71b559d700 1 -- 172.21.15.119:0/887138083 wait complete. 2023-10-08T02:28:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.511+0000 7f71b559d700 1 Processor -- start 2023-10-08T02:28:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.511+0000 7f71b559d700 1 -- start start 2023-10-08T02:28:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b559d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71b006f0b0 0x7f71b0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b559d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b01597d0 0x7f71b015b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b559d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71b01562d0 0x7f71b0156720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b559d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0156d90 con 0x7f71b01562d0 2023-10-08T02:28:54.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b559d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0154000 con 0x7f71b01597d0 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b559d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01542e0 con 0x7f71b006f0b0 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b4d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71b01562d0 0x7f71b0156720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71af7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b01597d0 0x7f71b015b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b4d9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71b01562d0 0x7f71b0156720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37740/0 (socket says 172.21.15.119:37740) 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71b4d9c700 1 -- 172.21.15.119:0/3131513537 learned_addr learned my addr 172.21.15.119:0/3131513537 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.512+0000 7f71affff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71b006f0b0 0x7f71b0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71b4d9c700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71b006f0b0 msgr2=0x7f71b0154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71b4d9c700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71b006f0b0 0x7f71b0154da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71b4d9c700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b01597d0 msgr2=0x7f71b015b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71b4d9c700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b01597d0 0x7f71b015b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71b4d9c700 1 -- 172.21.15.119:0/3131513537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a4009420 con 0x7f71b01562d0 2023-10-08T02:28:54.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71af7fe700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b01597d0 0x7f71b015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:28:54.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71b4d9c700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71b01562d0 0x7f71b0156720 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f71a8001460 tx=0x7f71a8001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:28:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a800a870 con 0x7f71b01562d0 2023-10-08T02:28:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a800a9d0 con 0x7f71b01562d0 2023-10-08T02:28:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a800ad50 con 0x7f71b01562d0 2023-10-08T02:28:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b01545c0 con 0x7f71b01562d0 2023-10-08T02:28:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.513+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b0151bf0 con 0x7f71b01562d0 2023-10-08T02:28:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.514+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b0154c10 con 0x7f71b01562d0 2023-10-08T02:28:54.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.515+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 103) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f71a8007300 con 0x7f71b01562d0 2023-10-08T02:28:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.516+0000 7f7197fff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:28:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.516+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f71a8084aa0 con 0x7f71b01562d0 2023-10-08T02:28:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.516+0000 7f71affff700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 msgr2=0x7f719c069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:54.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.516+0000 7f71affff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:28:54.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.520+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71a8051b20 con 0x7f71b01562d0 2023-10-08T02:28:54.634 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:54.634+0000 7feea3c8f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:28:54.688 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:54.689+0000 7f6aa0850180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:28:54.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.717+0000 7f71affff700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 msgr2=0x7f719c069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:54.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.717+0000 7f71affff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:28:54.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:54.742+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f71b006eaa0 con 0x7f719c066fb0 2023-10-08T02:28:54.799 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:54.799+0000 7feea3c8f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:28:54.856 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:54.857+0000 7f6aa0850180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:28:54.888 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:54.888+0000 7feea3c8f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:28:54.943 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:54.943+0000 7f6aa0850180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:28:55.113 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:55.113+0000 7feea3c8f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:28:55.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:55.118+0000 7f71affff700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 msgr2=0x7f719c069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:55.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:55.118+0000 7f71affff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:28:55.168 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:55.169+0000 7f6aa0850180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:28:55.291 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:55.291+0000 7feea3c8f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:28:55.350 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:55.350+0000 7f6aa0850180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:28:55.387 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:55.387+0000 7feea3c8f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:28:55.442 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:55.443+0000 7f6aa0850180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:28:55.549 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:55.549+0000 7feea3c8f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:28:55.605 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:55.605+0000 7f6aa0850180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:28:55.650 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:55.650+0000 7feea3c8f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:28:55.706 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:55.706+0000 7f6aa0850180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:28:55.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:55.919+0000 7f71affff700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 msgr2=0x7f719c069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:55.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:55.919+0000 7f71affff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T02:28:56.133 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:56.134+0000 7feea3c8f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:28:56.193 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:56.193+0000 7f6aa0850180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:28:56.226 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:56.227+0000 7feea3c8f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:28:56.286 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:56.287+0000 7f6aa0850180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:28:56.315 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:56.316+0000 7feea3c8f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:28:56.377 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:56.378+0000 7f6aa0850180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:28:56.938 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:56.938+0000 7feea3c8f180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:28:56.998 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:56.999+0000 7f6aa0850180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:28:57.110 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:57.110+0000 7feea3c8f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:28:57.172 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:57.172+0000 7f6aa0850180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:28:57.205 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:57.206+0000 7feea3c8f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:28:57.267 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:57.268+0000 7f6aa0850180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:28:57.501 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:57.501+0000 7feea3c8f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:28:57.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:57.521+0000 7f71affff700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 msgr2=0x7f719c069460 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:28:57.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:57.521+0000 7f71affff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T02:28:57.563 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:57.563+0000 7f6aa0850180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:28:57.596 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:57.596+0000 7feea3c8f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:28:57.658 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:57.659+0000 7f6aa0850180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:28:58.298 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:58.298+0000 7feea3c8f180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:28:58.364 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:58.364+0000 7f6aa0850180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:28:58.586 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:58.586+0000 7feea3c8f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:28:58.652 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:58.652+0000 7f6aa0850180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:28:58.679 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:58.679+0000 7feea3c8f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:28:58.747 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:58.745+0000 7f6aa0850180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:28:58.834 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:58.834+0000 7feea3c8f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:28:58.897 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:58.897+0000 7f6aa0850180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:28:58.939 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:58.939+0000 7feea3c8f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:28:59.003 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.003+0000 7f6aa0850180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:28:59.126 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:59.127+0000 7feea3c8f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:28:59.194 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.194+0000 7f6aa0850180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:28:59.218 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:59.219+0000 7feea3c8f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:28:59.288 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.289+0000 7f6aa0850180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:28:59.311 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:59.312+0000 7feea3c8f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:28:59.384 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.384+0000 7f6aa0850180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:28:59.405 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:28:59.406+0000 7feea3c8f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:28:59.480 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.481+0000 7f6aa0850180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:28:59.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:59.490+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 104) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f71a8051ef0 con 0x7f71b01562d0 2023-10-08T02:28:59.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:59.491+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 msgr2=0x7f719c069460 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:28:59.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:28:59.491+0000 7f7197fff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c069460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:28:59.610 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a12b31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.610 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a12b31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.610 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a12b31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.610 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a12b31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.611 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a12b31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.611 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a15b37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.611 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a15b37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.611 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a15b37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.611 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a15b37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:28:59.611 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:28:59.610+0000 7f6a15b37700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:29:00.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.494+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 105) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f71a805c2f0 con 0x7f71b01562d0 2023-10-08T02:29:00.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.494+0000 7f7197fff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.494+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f71b006eaa0 con 0x7f719c066fb0 2023-10-08T02:29:00.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.497+0000 7f71affff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.502+0000 7f71affff700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c0693e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f71a0007fd0 tx=0x7f71a0007ce0 comp rx=0 tx=0).ready entity=mgr.9918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:29:00.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.565+0000 7f7197fff700 1 -- 172.21.15.119:0/3131513537 <== mgr.9918 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f71b006eaa0 con 0x7f719c066fb0 2023-10-08T02:29:00.566 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path snap_test_dir1 2023-10-08T02:29:00.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 msgr2=0x7f719c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:00.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c0693e0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f71a0007fd0 tx=0x7f71a0007ce0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71b01562d0 msgr2=0x7f71b0156720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71b01562d0 0x7f71b0156720 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f71a8001460 tx=0x7f71a8001a40 comp rx=0 tx=0).stop 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 shutdown_connections 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f719c066fb0 0x7f719c0693e0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f71b006f0b0 0x7f71b0154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71b01562d0 0x7f71b0156720 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.567+0000 7f71b559d700 1 --2- 172.21.15.119:0/3131513537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f71b01597d0 0x7f71b015b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.568+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 >> 172.21.15.119:0/3131513537 conn(0x7f71b006adf0 msgr2=0x7f71b00ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.568+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 shutdown_connections 2023-10-08T02:29:00.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.568+0000 7f71b559d700 1 -- 172.21.15.119:0/3131513537 wait complete. 2023-10-08T02:29:00.587 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T02:29:00.587 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-10-08T02:29:00.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.776+0000 7f3e299ff700 1 Processor -- start 2023-10-08T02:29:00.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.776+0000 7f3e299ff700 1 -- start start 2023-10-08T02:29:00.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.777+0000 7f3e299ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3e241569f0 0x7f3e240c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.777+0000 7f3e299ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e24068390 0x7f3e24068830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.777+0000 7f3e299ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2415a6f0 con 0x7f3e24156620 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.777+0000 7f3e299ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e240c8d30 con 0x7f3e241569f0 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.777+0000 7f3e299ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24068d70 con 0x7f3e24068390 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e22ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3e24156620 0x7f3e240ace70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43292/0 (socket says 172.21.15.119:43292) 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e22ffd700 1 -- 172.21.15.119:0/4235680069 learned_addr learned my addr 172.21.15.119:0/4235680069 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e227fc700 1 --2- 172.21.15.119:0/4235680069 >> v2:172.21.15.184:3300/0 conn(0x7f3e241569f0 0x7f3e240c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956428369 0 0) 0x7f3e2415a6f0 con 0x7f3e24156620 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e237fe700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e24068390 0x7f3e24068830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e0c003570 con 0x7f3e24156620 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 446362212 0 0) 0x7f3e0c003570 con 0x7f3e24156620 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e24068390 msgr2=0x7f3e24068830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e24068390 0x7f3e24068830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 >> v2:172.21.15.184:3300/0 conn(0x7f3e241569f0 msgr2=0x7f3e240c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 --2- 172.21.15.119:0/4235680069 >> v2:172.21.15.184:3300/0 conn(0x7f3e241569f0 0x7f3e240c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2415a9d0 con 0x7f3e24156620 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f3e14001470 con 0x7f3e24156620 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.778+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 >> v1:172.21.15.119:6789/0 conn(0x7f3e24156620 legacy=0x7f3e240ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e21ffb700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e24068390 0x7f3e0c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e21ffb700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e0c003fe0 0x7f3e0c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e21ffb700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240c8d30 con 0x7f3e0c004d50 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24068d70 con 0x7f3e0c003fe0 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0c00dca0 con 0x7f3e24068390 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e237fe700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e22ffd700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e24068390 0x7f3e0c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e227fc700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e0c003fe0 0x7f3e0c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e237fe700 1 -- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e24068390 msgr2=0x7f3e0c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e237fe700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e24068390 0x7f3e0c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e237fe700 1 -- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e0c003fe0 msgr2=0x7f3e0c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e237fe700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e0c003fe0 0x7f3e0c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.779+0000 7f3e237fe700 1 -- 172.21.15.119:0/4235680069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c003570 con 0x7f3e0c004d50 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e237fe700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e0c0051c0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00e210 tx=0x7f3e1c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=582c8eb2c9e839c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00ea60 con 0x7f3e0c004d50 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c00f040 con 0x7f3e0c004d50 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e21ffb700 1 -- 172.21.15.119:0/4235680069 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00ee10 con 0x7f3e0c004d50 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e299ff700 1 -- 172.21.15.119:0/4235680069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 msgr2=0x7f3e0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e299ff700 1 --2- 172.21.15.119:0/4235680069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e0c0051c0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00e210 tx=0x7f3e1c00b040 comp rx=0 tx=0).stop 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e237fe700 1 -- 172.21.15.119:0/4235680069 reap_dead start 2023-10-08T02:29:00.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e299ff700 1 -- 172.21.15.119:0/4235680069 shutdown_connections 2023-10-08T02:29:00.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e299ff700 1 -- 172.21.15.119:0/4235680069 >> 172.21.15.119:0/4235680069 conn(0x7f3e240cc540 msgr2=0x7f3e240cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:29:00.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e299ff700 1 -- 172.21.15.119:0/4235680069 shutdown_connections 2023-10-08T02:29:00.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.780+0000 7f3e299ff700 1 -- 172.21.15.119:0/4235680069 wait complete. 2023-10-08T02:29:00.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.782+0000 7f3e299ff700 1 Processor -- start 2023-10-08T02:29:00.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.782+0000 7f3e299ff700 1 -- start start 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e299ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e241523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e299ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24154bd0 0x7f3e240c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e299ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e240c5b10 0x7f3e240c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e299ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e241528f0 con 0x7f3e0c004d50 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e299ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240b02c0 con 0x7f3e24154bd0 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e299ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240b1120 con 0x7f3e240c5b10 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e241523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e241523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41622/0 (socket says 172.21.15.119:41622) 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e22ffd700 1 -- 172.21.15.119:0/697104532 learned_addr learned my addr 172.21.15.119:0/697104532 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:29:00.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e227fc700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24154bd0 0x7f3e240c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e22ffd700 1 -- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e240c5b10 msgr2=0x7f3e240c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e237fe700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e240c5b10 0x7f3e240c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e22ffd700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e240c5b10 0x7f3e240c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e22ffd700 1 -- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24154bd0 msgr2=0x7f3e240c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e22ffd700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24154bd0 0x7f3e240c3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e22ffd700 1 -- 172.21.15.119:0/697104532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240c64f0 con 0x7f3e0c004d50 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e227fc700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24154bd0 0x7f3e240c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.783+0000 7f3e237fe700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e240c5b10 0x7f3e240c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.784+0000 7f3e22ffd700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e241523b0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3e140073c0 tx=0x7f3e14001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.784+0000 7f3e0affd700 1 -- 172.21.15.119:0/697104532 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400f870 con 0x7f3e0c004d50 2023-10-08T02:29:00.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.784+0000 7f3e0affd700 1 -- 172.21.15.119:0/697104532 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1400fa10 con 0x7f3e0c004d50 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.784+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e240b0510 con 0x7f3e0c004d50 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.784+0000 7f3e0affd700 1 -- 172.21.15.119:0/697104532 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1400fdd0 con 0x7f3e0c004d50 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.784+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e240b0ad0 con 0x7f3e0c004d50 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.785+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e240b24b0 con 0x7f3e0c004d50 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.786+0000 7f3e0affd700 1 -- 172.21.15.119:0/697104532 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 105) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f3e14013600 con 0x7f3e0c004d50 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.786+0000 7f3e0affd700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3e10066dd0 0x7f3e10069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.787+0000 7f3e0affd700 1 -- 172.21.15.119:0/697104532 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6455+0+0 (secure 0 0 0) 0x7f3e14084850 con 0x7f3e0c004d50 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.787+0000 7f3e227fc700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3e10066dd0 0x7f3e10069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:00.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.787+0000 7f3e227fc700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3e10066dd0 0x7f3e10069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e240c39d0 tx=0x7f3e18008040 comp rx=0 tx=0).ready entity=mgr.9918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:29:00.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:00.791+0000 7f3e0affd700 1 -- 172.21.15.119:0/697104532 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3e140519b0 con 0x7f3e0c004d50 2023-10-08T02:29:01.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.008+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e240b1260 con 0x7f3e10066dd0 2023-10-08T02:29:01.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.070+0000 7f3e0affd700 1 -- 172.21.15.119:0/697104532 <== mgr.9918 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f3e18009e70 con 0x7f3e10066dd0 2023-10-08T02:29:01.071 INFO:teuthology.orchestra.run.smithi119.stdout:Retention added to path snap_test_dir1 2023-10-08T02:29:01.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.072+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3e10066dd0 msgr2=0x7f3e10069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.072+0000 7f3e299ff700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3e10066dd0 0x7f3e10069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3e240c39d0 tx=0x7f3e18008040 comp rx=0 tx=0).stop 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.072+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 msgr2=0x7f3e241523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.072+0000 7f3e299ff700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e241523b0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f3e140073c0 tx=0x7f3e14001e90 comp rx=0 tx=0).stop 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.073+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 shutdown_connections 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.073+0000 7f3e299ff700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3e10066dd0 0x7f3e10069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.073+0000 7f3e299ff700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3e240c5b10 0x7f3e240c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.073+0000 7f3e299ff700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3e0c004d50 0x7f3e241523b0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.073+0000 7f3e299ff700 1 --2- 172.21.15.119:0/697104532 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3e24154bd0 0x7f3e240c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.073+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 >> 172.21.15.119:0/697104532 conn(0x7f3e240cc540 msgr2=0x7f3e24069f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:29:01.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.073+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 shutdown_connections 2023-10-08T02:29:01.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.073+0000 7f3e299ff700 1 -- 172.21.15.119:0/697104532 wait complete. 2023-10-08T02:29:01.091 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-08T02:29:01.091 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-02_30_00 in ~59s... 2023-10-08T02:29:01.091 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-10-08T02:29:01.091 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-08T02:29:01.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.275+0000 7ff0435ff700 1 Processor -- start 2023-10-08T02:29:01.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.275+0000 7ff0435ff700 1 -- start start 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.276+0000 7ff0435ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff03c1563b0 0x7ff03c156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.276+0000 7ff0435ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff03c068890 0x7ff03c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.276+0000 7ff0435ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff03c15a9b0 con 0x7ff03c15a440 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.276+0000 7ff0435ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03c15ac90 con 0x7ff03c1563b0 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.276+0000 7ff0435ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff03c15af70 con 0x7ff03c068890 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff04139b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff03c1563b0 0x7ff03c156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff04139b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff03c1563b0 0x7ff03c156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41638/0 (socket says 172.21.15.119:41638) 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff04139b700 1 -- 172.21.15.119:0/3392414264 learned_addr learned my addr 172.21.15.119:0/3392414264 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff041b9c700 1 --2- 172.21.15.119:0/3392414264 >> v2:172.21.15.184:3300/0 conn(0x7ff03c068890 0x7ff03c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:01.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff04139b700 1 -- 172.21.15.119:0/3392414264 >> v2:172.21.15.184:3300/0 conn(0x7ff03c068890 msgr2=0x7ff03c0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:29:01.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff04139b700 1 --2- 172.21.15.119:0/3392414264 >> v2:172.21.15.184:3300/0 conn(0x7ff03c068890 0x7ff03c0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff04139b700 1 -- 172.21.15.119:0/3392414264 >> v1:172.21.15.184:6789/0 conn(0x7ff03c15a440 legacy=0x7ff03c0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:01.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff04139b700 1 -- 172.21.15.119:0/3392414264 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c15b250 con 0x7ff03c1563b0 2023-10-08T02:29:01.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.277+0000 7ff04139b700 1 --2- 172.21.15.119:0/3392414264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c1563b0 0x7ff03c156780 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0bff20 tx=0x7ff038009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17388afd25783f40 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:29:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff033fff700 1 -- 172.21.15.119:0/3392414264 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800b860 con 0x7ff03c1563b0 2023-10-08T02:29:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff033fff700 1 -- 172.21.15.119:0/3392414264 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03800b9c0 con 0x7ff03c1563b0 2023-10-08T02:29:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff033fff700 1 -- 172.21.15.119:0/3392414264 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800bd60 con 0x7ff03c1563b0 2023-10-08T02:29:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff0435ff700 1 -- 172.21.15.119:0/3392414264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c1563b0 msgr2=0x7ff03c156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3392414264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c1563b0 0x7ff03c156780 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0bff20 tx=0x7ff038009250 comp rx=0 tx=0).stop 2023-10-08T02:29:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff0435ff700 1 -- 172.21.15.119:0/3392414264 shutdown_connections 2023-10-08T02:29:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3392414264 >> v2:172.21.15.184:3300/0 conn(0x7ff03c068890 0x7ff03c0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3392414264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c1563b0 0x7ff03c156780 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff0435ff700 1 -- 172.21.15.119:0/3392414264 >> 172.21.15.119:0/3392414264 conn(0x7ff03c0c8f90 msgr2=0x7ff03c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:29:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff0435ff700 1 -- 172.21.15.119:0/3392414264 shutdown_connections 2023-10-08T02:29:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.278+0000 7ff0435ff700 1 -- 172.21.15.119:0/3392414264 wait complete. 2023-10-08T02:29:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.280+0000 7ff0435ff700 1 Processor -- start 2023-10-08T02:29:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.280+0000 7ff0435ff700 1 -- start start 2023-10-08T02:29:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff0435ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c068890 0x7ff03c14e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff0435ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff03c1563b0 0x7ff03c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff0435ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff03c15a440 0x7ff03c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff0435ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0b15c0 con 0x7ff03c068890 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff0435ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c0c0880 con 0x7ff03c1563b0 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff0435ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c157c70 con 0x7ff03c15a440 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff04139b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c068890 0x7ff03c14e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff04139b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c068890 0x7ff03c14e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41648/0 (socket says 172.21.15.119:41648) 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff04139b700 1 -- 172.21.15.119:0/3606718666 learned_addr learned my addr 172.21.15.119:0/3606718666 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff041b9c700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff03c15a440 0x7ff03c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.281+0000 7ff040b9a700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff03c1563b0 0x7ff03c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff04139b700 1 -- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff03c15a440 msgr2=0x7ff03c0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff04139b700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff03c15a440 0x7ff03c0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff04139b700 1 -- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff03c1563b0 msgr2=0x7ff03c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff04139b700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff03c1563b0 0x7ff03c0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff04139b700 1 -- 172.21.15.119:0/3606718666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380094f0 con 0x7ff03c068890 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff040b9a700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff03c1563b0 0x7ff03c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff04139b700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c068890 0x7ff03c14e030 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06bbb0 tx=0x7ff038017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff0317fa700 1 -- 172.21.15.119:0/3606718666 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038024440 con 0x7ff03c068890 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff0317fa700 1 -- 172.21.15.119:0/3606718666 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0380245a0 con 0x7ff03c068890 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff0317fa700 1 -- 172.21.15.119:0/3606718666 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038024960 con 0x7ff03c068890 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c1573d0 con 0x7ff03c068890 2023-10-08T02:29:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.282+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c157990 con 0x7ff03c068890 2023-10-08T02:29:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.283+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff03c0c0650 con 0x7ff03c068890 2023-10-08T02:29:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.284+0000 7ff0317fa700 1 -- 172.21.15.119:0/3606718666 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 105) v1 ==== 84595+0+0 (secure 0 0 0) 0x7ff038024d50 con 0x7ff03c068890 2023-10-08T02:29:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.285+0000 7ff0317fa700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff020066dd0 0x7ff020069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:29:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.286+0000 7ff0317fa700 1 -- 172.21.15.119:0/3606718666 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6455+0+0 (secure 0 0 0) 0x7ff0380861e0 con 0x7ff03c068890 2023-10-08T02:29:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.286+0000 7ff040b9a700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff020066dd0 0x7ff020069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:29:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.286+0000 7ff040b9a700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff020066dd0 0x7ff020069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff02c007fd0 tx=0x7ff02c009040 comp rx=0 tx=0).ready entity=mgr.9918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:29:01.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.290+0000 7ff0317fa700 1 -- 172.21.15.119:0/3606718666 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff03c0c0650 con 0x7ff03c068890 2023-10-08T02:29:01.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.503+0000 7ff0317fa700 1 -- 172.21.15.119:0/3606718666 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 106) v1 ==== 84916+0+0 (secure 0 0 0) 0x7ff038053710 con 0x7ff03c068890 2023-10-08T02:29:01.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.510+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff03c0c0650 con 0x7ff020066dd0 2023-10-08T02:29:01.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.519+0000 7ff0317fa700 1 -- 172.21.15.119:0/3606718666 <== mgr.9918 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7ff03c0c0650 con 0x7ff020066dd0 2023-10-08T02:29:01.520 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:29:01.520 INFO:teuthology.orchestra.run.smithi119.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff020066dd0 msgr2=0x7ff020069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff020066dd0 0x7ff020069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff02c007fd0 tx=0x7ff02c009040 comp rx=0 tx=0).stop 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c068890 msgr2=0x7ff03c14e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c068890 0x7ff03c14e030 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff03c06bbb0 tx=0x7ff038017000 comp rx=0 tx=0).stop 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 shutdown_connections 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff020066dd0 0x7ff020069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff03c15a440 0x7ff03c0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff03c068890 0x7ff03c14e030 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 --2- 172.21.15.119:0/3606718666 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff03c1563b0 0x7ff03c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:29:01.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 >> 172.21.15.119:0/3606718666 conn(0x7ff03c0c8f90 msgr2=0x7ff03c06e0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:29:01.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 shutdown_connections 2023-10-08T02:29:01.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:29:01.522+0000 7ff0435ff700 1 -- 172.21.15.119:0/3606718666 wait complete. 2023-10-08T02:29:01.541 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-08T02:29:01.541 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-10-08T02:29:01.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:01.541 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:01.579 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:01.579 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:01.579 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:02.580 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:02.581 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:02.624 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:02.624 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:02.624 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:03.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:03.626 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:03.668 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:03.668 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:03.668 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:04.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:04.670 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:04.712 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:04.712 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:04.712 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:05.713 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:05.714 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:05.757 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:05.757 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:05.757 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:06.758 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:06.759 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:06.802 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:06.802 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:06.802 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:07.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:07.803 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:07.847 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:07.847 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:07.847 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:08.847 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:08.848 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:08.888 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:08.888 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:08.888 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:09.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:09.890 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:09.933 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:09.933 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:09.933 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:10.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:10.935 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:10.978 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:10.978 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:10.978 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:11.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:11.979 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:12.022 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:12.022 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:12.022 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:13.023 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:13.024 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:13.066 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:13.066 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:13.066 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:13.092 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:29:13.111 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:29:14.067 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:14.067 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:14.110 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:14.110 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:14.110 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:15.112 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:15.112 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:15.155 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:15.155 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:15.155 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:16.156 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:16.156 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:16.199 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:16.199 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:16.199 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:17.200 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:17.200 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:17.243 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:17.243 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:17.243 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:18.245 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:18.245 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:18.285 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:18.285 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:18.285 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:19.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:19.286 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:19.332 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:19.333 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:19.333 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:20.333 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:20.333 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:20.377 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:20.377 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:20.377 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:21.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:21.379 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:21.421 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:21.421 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:21.421 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:22.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:22.423 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:22.466 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:22.467 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:22.467 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:23.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:23.468 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:23.511 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:23.512 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:23.512 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:24.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:24.513 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:24.556 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:24.557 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:24.557 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:25.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:25.557 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:25.600 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:25.600 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:25.601 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:26.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:26.602 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:26.645 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:26.645 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:26.645 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:27.646 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:27.646 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:27.690 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:27.690 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:27.690 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:28.691 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:28.691 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:28.733 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:28.733 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:28.733 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:29.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:29.734 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:29.778 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:29.778 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:29.778 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:30.779 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:30.779 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:30.822 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:30.822 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:30.822 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:31.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:31.824 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:31.867 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:31.867 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:31.867 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:32.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:32.868 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:32.911 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:32.912 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:32.912 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:33.912 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:33.912 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:33.955 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:33.955 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:33.956 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:34.957 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:34.957 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:35.000 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:35.000 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:35.001 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:36.001 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:36.001 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:36.044 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:36.044 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:36.044 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:37.046 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:37.046 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:37.089 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:37.089 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:37.089 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:38.090 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:38.090 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:38.133 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:38.133 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:38.133 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:39.134 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:39.134 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:39.180 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:39.180 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:39.180 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:40.181 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:40.182 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:40.226 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:40.227 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:40.227 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:41.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:41.228 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:41.271 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:41.271 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:41.271 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:42.272 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:42.272 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:42.316 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:42.316 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:42.316 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:43.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:29:43.295 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:29:43.317 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:43.317 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:43.378 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:43.378 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:43.378 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:44.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:44.379 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:44.423 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:44.423 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:44.423 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:45.423 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:45.424 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:45.466 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:45.466 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:45.466 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:46.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:46.467 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:46.510 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:46.510 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:46.510 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:47.512 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:47.512 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:47.554 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:47.555 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:47.555 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:48.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:48.556 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:48.602 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:48.602 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:48.602 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:49.603 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:49.603 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:49.647 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:49.647 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:49.647 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:50.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:50.649 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:50.691 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:50.692 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:50.692 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:51.693 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:51.693 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:51.737 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:51.737 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:51.737 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:52.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:52.738 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:52.781 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:52.781 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:52.781 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:53.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:53.782 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:53.824 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:53.824 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:53.824 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:54.824 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:54.825 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:54.868 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:54.868 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:54.868 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:55.869 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:55.869 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:55.912 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:55.912 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:55.912 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:56.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:56.913 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:56.956 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:56.957 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:56.957 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:57.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:57.958 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:58.001 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:58.002 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:58.002 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:29:59.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:29:59.003 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:29:59.047 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:29:59.047 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:29:59.047 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:00.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:00.049 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:00.083 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:30:00.083+0000 7f6a04b15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:30:00.084 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:30:00.083+0000 7f6a04b15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:30:00.084 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:30:00.083+0000 7f6a04b15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:30:00.084 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:30:00.083+0000 7f6a04b15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:30:00.084 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:30:00.083+0000 7f6a04b15700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:30:00.093 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-10-08T02:30:00.093 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:00.093 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:01.094 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:01.094 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:01.136 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:01.137 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:01.137 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:01.137 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:01.138 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-10-08-02_30_00_UTC'] 2023-10-08T02:30:01.138 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696732141.0913177, now = 1696732201.138129, timo = 59 2023-10-08T02:30:02.139 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-10-08-02_30_00 in ~119s... 2023-10-08T02:30:02.139 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:02.139 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:02.181 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:02.182 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:02.182 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:02.182 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:03.183 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:03.183 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:03.227 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:03.228 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:03.228 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:03.228 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:04.229 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:04.229 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:04.271 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:04.272 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:04.272 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:04.273 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:05.273 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:05.274 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:05.316 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:05.317 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:05.317 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:05.317 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:06.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:06.318 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:06.360 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:06.361 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:06.361 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:06.361 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:07.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:07.362 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:07.404 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:07.405 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:07.405 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:07.405 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:08.406 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:08.406 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:08.449 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:08.450 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:08.450 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:08.450 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:09.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:09.452 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:09.493 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:09.495 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:09.495 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:09.495 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:10.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:10.497 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:10.540 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:10.541 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:10.541 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:10.541 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:11.542 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:11.542 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:11.584 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:11.585 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:11.585 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:11.585 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:12.586 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:12.586 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:12.630 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:12.631 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:12.631 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:12.631 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:13.468 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:30:13.473 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:30:13.632 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:13.632 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:13.670 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:13.672 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:13.672 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:13.672 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:14.672 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:14.673 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:14.715 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:14.716 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:14.716 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:14.716 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:15.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:15.717 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:15.759 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:15.759 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:15.760 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:15.760 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:16.760 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:16.760 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:16.802 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:16.803 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:16.804 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:16.804 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:17.804 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:17.805 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:17.847 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:17.848 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:17.848 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:17.848 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:18.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:18.849 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:18.891 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:18.892 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:18.893 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:18.893 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:19.893 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:19.894 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:19.936 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:19.937 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:19.937 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:19.937 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:20.938 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:20.939 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:20.980 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:20.981 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:20.981 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:20.981 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:21.983 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:21.983 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:22.025 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:22.026 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:22.026 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:22.026 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:23.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:23.027 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:23.069 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:23.070 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:23.070 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:23.070 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:24.071 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:24.072 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:24.113 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:24.115 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:24.115 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:24.115 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:25.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:25.116 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:25.158 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:25.159 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:25.159 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:25.159 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:26.160 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:26.160 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:26.202 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:26.203 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:26.203 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:26.203 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:27.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:27.204 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:27.246 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:27.247 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:27.247 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:27.247 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:28.248 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:28.249 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:28.291 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:28.292 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:28.292 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:28.292 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:29.292 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:29.293 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:29.334 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:29.336 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:29.336 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:29.336 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:30.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:30.337 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:30.380 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:30.381 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:30.381 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:30.381 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:31.382 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:31.382 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:31.423 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:31.425 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:31.425 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:31.425 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:32.425 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:32.425 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:32.466 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:32.467 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:32.467 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:32.467 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:33.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:33.469 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:33.511 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:33.512 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:33.512 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:33.512 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:34.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:34.513 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:34.555 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:34.557 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:34.557 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:34.557 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:35.558 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:35.558 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:35.600 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:35.601 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:35.601 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:35.601 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:36.602 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:36.602 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:36.645 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:36.646 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:36.646 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:36.647 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:37.647 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:37.648 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:37.690 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:37.691 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:37.691 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:37.691 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:38.692 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:38.692 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:38.734 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:38.735 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:38.735 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:38.735 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:39.736 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:39.737 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:39.779 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:39.780 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:39.780 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:39.780 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:40.781 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:40.781 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:40.824 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:40.825 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:40.825 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:40.825 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:41.826 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:41.826 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:41.868 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:41.869 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:41.869 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:41.869 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:42.870 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:42.871 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:42.912 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:42.913 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:42.913 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:42.913 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:43.589 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:30:43.595 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:30:43.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:43.915 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:43.956 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:43.957 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:43.957 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:43.957 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:44.958 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:44.959 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:45.001 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:45.002 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:45.002 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:45.002 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:46.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:46.005 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:46.047 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:46.049 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:46.049 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:46.049 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:47.050 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:47.051 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:47.093 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:47.094 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:47.094 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:47.095 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:48.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:48.096 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:48.138 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:48.139 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:48.139 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:48.139 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:49.140 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:49.140 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:49.183 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:49.184 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:49.184 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:49.184 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:50.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:50.185 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:50.227 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:50.228 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:50.228 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:50.229 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:51.230 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:51.230 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:51.273 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:51.274 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:51.274 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:51.274 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:52.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:52.275 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:52.317 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:52.318 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:52.318 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:52.318 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:53.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:53.319 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:53.360 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:53.361 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:53.361 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:53.362 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:54.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:54.363 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:54.404 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:54.405 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:54.405 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:54.405 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:55.407 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:55.407 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:55.449 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:55.450 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:55.450 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:55.450 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:56.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:56.451 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:56.493 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:56.494 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:56.494 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:56.494 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:57.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:57.496 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:57.538 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:57.539 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:57.539 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:57.539 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:58.540 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:58.540 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:58.583 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:58.584 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:58.584 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:58.584 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:30:59.585 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:30:59.586 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:30:59.628 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_30_00_UTC 2023-10-08T02:30:59.629 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:30:59.629 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-10-08T02:30:59.629 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-10-08T02:31:00.630 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:00.631 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:31:00.673 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_31_00_UTC 2023-10-08T02:31:00.674 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-10-08-02_31_00_UTC'} 2023-10-08T02:31:00.674 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-10-08-02_31_00_UTC'} 2023-10-08T02:31:00.674 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-10-08-02_30_00_UTC'} 2023-10-08T02:31:00.674 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-10-08-02_30_00_UTC'] 2023-10-08T02:31:00.674 DEBUG:tasks.cephfs.test_snap_schedules:exec=1696732141.0913177, now = 1696732260.6746447, timo = 119 2023-10-08T02:31:01.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-08T02:31:01.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.871+0000 7f39417ff700 1 Processor -- start 2023-10-08T02:31:01.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.872+0000 7f39417ff700 1 -- start start 2023-10-08T02:31:01.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f39417ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f393c1563b0 0x7f393c156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:01.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f39417ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f393c15a440 0x7f393c0acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f39417ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c068890 0x7f393c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f39417ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393c15a9b0 con 0x7f393c1563b0 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f39417ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f393c15ac90 con 0x7f393c15a440 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f39417ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393c15af70 con 0x7f393c068890 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f393c1563b0 0x7f393c156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f393c1563b0 0x7f393c156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58504/0 (socket says 172.21.15.119:58504) 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393affd700 1 -- 172.21.15.119:0/1580106530 learned_addr learned my addr 172.21.15.119:0/1580106530 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f3933fff700 1 --2- 172.21.15.119:0/1580106530 >> v2:172.21.15.184:3300/0 conn(0x7f393c15a440 0x7f393c0acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393b7fe700 1 --2- 172.21.15.119:0/1580106530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c068890 0x7f393c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:01.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393affd700 1 -- 172.21.15.119:0/1580106530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c068890 msgr2=0x7f393c0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393affd700 1 --2- 172.21.15.119:0/1580106530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c068890 0x7f393c0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393affd700 1 -- 172.21.15.119:0/1580106530 >> v2:172.21.15.184:3300/0 conn(0x7f393c15a440 msgr2=0x7f393c0acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393affd700 1 --2- 172.21.15.119:0/1580106530 >> v2:172.21.15.184:3300/0 conn(0x7f393c15a440 0x7f393c0acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393affd700 1 -- 172.21.15.119:0/1580106530 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393c15b250 con 0x7f393c1563b0 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f393b7fe700 1 --2- 172.21.15.119:0/1580106530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c068890 0x7f393c0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.873+0000 7f3933fff700 1 --2- 172.21.15.119:0/1580106530 >> v2:172.21.15.184:3300/0 conn(0x7f393c15a440 0x7f393c0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.874+0000 7f393affd700 1 --2- 172.21.15.119:0/1580106530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c1563b0 0x7f393c156780 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f393c06fb10 tx=0x7f392c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f882c33d3fb8b953 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.874+0000 7f393a7fc700 1 -- 172.21.15.119:0/1580106530 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c00b860 con 0x7f393c1563b0 2023-10-08T02:31:01.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.874+0000 7f393a7fc700 1 -- 172.21.15.119:0/1580106530 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f392c00b9c0 con 0x7f393c1563b0 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.874+0000 7f393a7fc700 1 -- 172.21.15.119:0/1580106530 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392c00bd60 con 0x7f393c1563b0 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.874+0000 7f39417ff700 1 -- 172.21.15.119:0/1580106530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c1563b0 msgr2=0x7f393c156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.874+0000 7f39417ff700 1 --2- 172.21.15.119:0/1580106530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c1563b0 0x7f393c156780 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f393c06fb10 tx=0x7f392c009250 comp rx=0 tx=0).stop 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.875+0000 7f39417ff700 1 -- 172.21.15.119:0/1580106530 shutdown_connections 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.875+0000 7f39417ff700 1 --2- 172.21.15.119:0/1580106530 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c068890 0x7f393c0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.875+0000 7f39417ff700 1 --2- 172.21.15.119:0/1580106530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c1563b0 0x7f393c156780 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.875+0000 7f39417ff700 1 --2- 172.21.15.119:0/1580106530 >> v2:172.21.15.184:3300/0 conn(0x7f393c15a440 0x7f393c0acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.875+0000 7f39417ff700 1 -- 172.21.15.119:0/1580106530 >> 172.21.15.119:0/1580106530 conn(0x7f393c0c8f90 msgr2=0x7f393c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:01.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.875+0000 7f39417ff700 1 -- 172.21.15.119:0/1580106530 shutdown_connections 2023-10-08T02:31:01.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.875+0000 7f39417ff700 1 -- 172.21.15.119:0/1580106530 wait complete. 2023-10-08T02:31:01.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.876+0000 7f39417ff700 1 Processor -- start 2023-10-08T02:31:01.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.876+0000 7f39417ff700 1 -- start start 2023-10-08T02:31:01.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f39417ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f393c068890 0x7f393c14bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:01.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f39417ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c1563b0 0x7f393c0cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:01.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f39417ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c15a440 0x7f393c0cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:01.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f39417ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0cae30 con 0x7f393c15a440 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f39417ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0cb110 con 0x7f393c068890 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f39417ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393c0cb3f0 con 0x7f393c1563b0 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f393b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c15a440 0x7f393c0cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f393b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c15a440 0x7f393c0cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58506/0 (socket says 172.21.15.119:58506) 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.877+0000 7f393b7fe700 1 -- 172.21.15.119:0/4182268673 learned_addr learned my addr 172.21.15.119:0/4182268673 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f393affd700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f393c068890 0x7f393c14bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f3933fff700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c1563b0 0x7f393c0cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f393b7fe700 1 -- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c1563b0 msgr2=0x7f393c0cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f393b7fe700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c1563b0 0x7f393c0cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f393b7fe700 1 -- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f393c068890 msgr2=0x7f393c14bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f393b7fe700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f393c068890 0x7f393c14bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:01.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f393b7fe700 1 -- 172.21.15.119:0/4182268673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392c0094f0 con 0x7f393c15a440 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f393affd700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f393c068890 0x7f393c14bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f393b7fe700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c15a440 0x7f393c0cc710 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f393400cf60 tx=0x7f3934007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f3932ffd700 1 -- 172.21.15.119:0/4182268673 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934015440 con 0x7f393c15a440 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f3932ffd700 1 -- 172.21.15.119:0/4182268673 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39340155a0 con 0x7f393c15a440 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f3932ffd700 1 -- 172.21.15.119:0/4182268673 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934015960 con 0x7f393c15a440 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c0cb6d0 con 0x7f393c15a440 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.878+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c14c530 con 0x7f393c15a440 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.879+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f393c0718b0 con 0x7f393c15a440 2023-10-08T02:31:01.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.880+0000 7f3932ffd700 1 -- 172.21.15.119:0/4182268673 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 108) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f3934015d50 con 0x7f393c15a440 2023-10-08T02:31:01.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.881+0000 7f3932ffd700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f391c0670a0 0x7f391c069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:01.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.881+0000 7f3932ffd700 1 -- 172.21.15.119:0/4182268673 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6455+0+0 (secure 0 0 0) 0x7f3934085450 con 0x7f393c15a440 2023-10-08T02:31:01.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.881+0000 7f393affd700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f391c0670a0 0x7f391c069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:01.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.882+0000 7f393affd700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f391c0670a0 0x7f391c069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f393c06fb10 tx=0x7f392c017000 comp rx=0 tx=0).ready entity=mgr.9918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:01.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:01.885+0000 7f3932ffd700 1 -- 172.21.15.119:0/4182268673 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3934052220 con 0x7f393c15a440 2023-10-08T02:31:02.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.098+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f393c06d6a0 con 0x7f391c0670a0 2023-10-08T02:31:02.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.177+0000 7f3932ffd700 1 -- 172.21.15.119:0/4182268673 <== mgr.9918 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f393c06d6a0 con 0x7f391c0670a0 2023-10-08T02:31:02.176 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path snap_test_dir1 2023-10-08T02:31:02.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f391c0670a0 msgr2=0x7f391c069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:02.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f391c0670a0 0x7f391c069550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f393c06fb10 tx=0x7f392c017000 comp rx=0 tx=0).stop 2023-10-08T02:31:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c15a440 msgr2=0x7f393c0cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c15a440 0x7f393c0cc710 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f393400cf60 tx=0x7f3934007870 comp rx=0 tx=0).stop 2023-10-08T02:31:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 shutdown_connections 2023-10-08T02:31:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f391c0670a0 0x7f391c069550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f393c15a440 0x7f393c0cc710 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f393c068890 0x7f393c14bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 --2- 172.21.15.119:0/4182268673 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f393c1563b0 0x7f393c0cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 >> 172.21.15.119:0/4182268673 conn(0x7f393c0c8f90 msgr2=0x7f393c0af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 shutdown_connections 2023-10-08T02:31:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.179+0000 7f39417ff700 1 -- 172.21.15.119:0/4182268673 wait complete. 2023-10-08T02:31:02.198 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T02:31:02.198 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:02.198 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:31:02.236 INFO:teuthology.orchestra.run.smithi119.stdout:scheduled-2023-10-08-02_31_00_UTC 2023-10-08T02:31:02.237 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-10-08-02_31_00_UTC 2023-10-08T02:31:02.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:02.237 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-10-08-02_31_00_UTC) 2023-10-08T02:31:02.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:02.319 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T02:31:02.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T02:31:02.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.622+0000 7f82f579d700 1 Processor -- start 2023-10-08T02:31:02.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.622+0000 7f82f579d700 1 -- start start 2023-10-08T02:31:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.623+0000 7f82f579d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f82f00b1b40 0x7f82f00b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.623+0000 7f82f579d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f00cfb00 con 0x7f82f00b4e70 2023-10-08T02:31:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.623+0000 7f82f579d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82f00b5c40 con 0x7f82f00b16d0 2023-10-08T02:31:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.623+0000 7f82f579d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f00b5f20 con 0x7f82f00b1b40 2023-10-08T02:31:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.623+0000 7f82effff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f82f00b4e70 0x7f82f00ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35280/0 (socket says 172.21.15.119:35280) 2023-10-08T02:31:02.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.623+0000 7f82effff700 1 -- 172.21.15.119:0/4125680696 learned_addr learned my addr 172.21.15.119:0/4125680696 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.624+0000 7f82f4f9c700 1 --2- 172.21.15.119:0/4125680696 >> v2:172.21.15.184:3300/0 conn(0x7f82f00b1b40 0x7f82f00b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.624+0000 7f82eeffd700 1 -- 172.21.15.119:0/4125680696 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2545658978 0 0) 0x7f82f00cfb00 con 0x7f82f00b4e70 2023-10-08T02:31:02.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.624+0000 7f82eeffd700 1 -- 172.21.15.119:0/4125680696 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d8003570 con 0x7f82f00b4e70 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.624+0000 7f82f4f9c700 1 -- 172.21.15.119:0/4125680696 >> v1:172.21.15.184:6789/0 conn(0x7f82f00b16d0 legacy=0x7f82f00b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.624+0000 7f82eeffd700 1 -- 172.21.15.119:0/4125680696 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733249644 0 0) 0x7f82f00b5c40 con 0x7f82f00b16d0 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.624+0000 7f82f4f9c700 1 -- 172.21.15.119:0/4125680696 >> v1:172.21.15.119:6789/0 conn(0x7f82f00b4e70 legacy=0x7f82f00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.624+0000 7f82f4f9c700 1 -- 172.21.15.119:0/4125680696 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f00b6200 con 0x7f82f00b1b40 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82f4f9c700 1 --2- 172.21.15.119:0/4125680696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b1b40 0x7f82f00b3ac0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009e60 tx=0x7f82e8009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4575dcda6416b2dc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82eeffd700 1 -- 172.21.15.119:0/4125680696 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800e070 con 0x7f82f00b1b40 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82eeffd700 1 -- 172.21.15.119:0/4125680696 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e800ba60 con 0x7f82f00b1b40 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82eeffd700 1 -- 172.21.15.119:0/4125680696 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e800be20 con 0x7f82f00b1b40 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82f579d700 1 -- 172.21.15.119:0/4125680696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b1b40 msgr2=0x7f82f00b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82f579d700 1 --2- 172.21.15.119:0/4125680696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b1b40 0x7f82f00b3ac0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f82e8009e60 tx=0x7f82e8009780 comp rx=0 tx=0).stop 2023-10-08T02:31:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82f579d700 1 -- 172.21.15.119:0/4125680696 shutdown_connections 2023-10-08T02:31:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82f579d700 1 --2- 172.21.15.119:0/4125680696 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b1b40 0x7f82f00b3ac0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82f579d700 1 -- 172.21.15.119:0/4125680696 >> 172.21.15.119:0/4125680696 conn(0x7f82f00d10c0 msgr2=0x7f82f00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82f579d700 1 -- 172.21.15.119:0/4125680696 shutdown_connections 2023-10-08T02:31:02.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.625+0000 7f82f579d700 1 -- 172.21.15.119:0/4125680696 wait complete. 2023-10-08T02:31:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.627+0000 7f82f579d700 1 Processor -- start 2023-10-08T02:31:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.627+0000 7f82f579d700 1 -- start start 2023-10-08T02:31:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82f579d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82f00b16d0 0x7f82f0155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:02.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82f579d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f00b1b40 0x7f82f0169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82f579d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b4e70 0x7f82f0153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82f579d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0156c80 con 0x7f82f00b16d0 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82f579d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0151c20 con 0x7f82f00b4e70 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82f579d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f0156230 con 0x7f82f00b1b40 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82effff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82f00b16d0 0x7f82f0155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82effff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82f00b16d0 0x7f82f0155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58522/0 (socket says 172.21.15.119:58522) 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82effff700 1 -- 172.21.15.119:0/1138308831 learned_addr learned my addr 172.21.15.119:0/1138308831 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82f4f9c700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b4e70 0x7f82f0153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82ef7fe700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f00b1b40 0x7f82f0169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82effff700 1 -- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f00b1b40 msgr2=0x7f82f0169960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:02.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82effff700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f00b1b40 0x7f82f0169960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82effff700 1 -- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b4e70 msgr2=0x7f82f0153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82effff700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b4e70 0x7f82f0153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82effff700 1 -- 172.21.15.119:0/1138308831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e8009a20 con 0x7f82f00b16d0 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.628+0000 7f82f4f9c700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b4e70 0x7f82f0153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.629+0000 7f82effff700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82f00b16d0 0x7f82f0155cf0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f82e000c810 tx=0x7f82e0007d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.629+0000 7f82d7fff700 1 -- 172.21.15.119:0/1138308831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0007360 con 0x7f82f00b16d0 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.629+0000 7f82d7fff700 1 -- 172.21.15.119:0/1138308831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e00074c0 con 0x7f82f00b16d0 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.629+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f016da40 con 0x7f82f00b16d0 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.629+0000 7f82d7fff700 1 -- 172.21.15.119:0/1138308831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0004830 con 0x7f82f00b16d0 2023-10-08T02:31:02.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.629+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f016e000 con 0x7f82f00b16d0 2023-10-08T02:31:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.630+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f0088990 con 0x7f82f00b16d0 2023-10-08T02:31:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.631+0000 7f82d7fff700 1 -- 172.21.15.119:0/1138308831 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 108) v1 ==== 84871+0+0 (secure 0 0 0) 0x7f82e0004c20 con 0x7f82f00b16d0 2023-10-08T02:31:02.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.632+0000 7f82d7fff700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82dc0670a0 0x7f82dc069550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.632+0000 7f82d7fff700 1 -- 172.21.15.119:0/1138308831 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6455+0+0 (secure 0 0 0) 0x7f82e0083ef0 con 0x7f82f00b16d0 2023-10-08T02:31:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.632+0000 7f82ef7fe700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82dc0670a0 0x7f82dc069550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:02.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.633+0000 7f82ef7fe700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82dc0670a0 0x7f82dc069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82f00b5cd0 tx=0x7f82e4007d30 comp rx=0 tx=0).ready entity=mgr.9918 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:02.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.636+0000 7f82d7fff700 1 -- 172.21.15.119:0/1138308831 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f82e0050de0 con 0x7f82f00b16d0 2023-10-08T02:31:02.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:02.878+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f82f004bb80 con 0x7f82f00b16d0 2023-10-08T02:31:03.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.452+0000 7f82d7fff700 1 -- 172.21.15.119:0/1138308831 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f82e00511b0 con 0x7f82f00b16d0 2023-10-08T02:31:03.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.454+0000 7f82d7fff700 1 -- 172.21.15.119:0/1138308831 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v109) v1 ==== 93+0+0 (secure 0 0 0) 0x7f82e0050de0 con 0x7f82f00b16d0 2023-10-08T02:31:03.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82dc0670a0 msgr2=0x7f82dc069550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:03.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82dc0670a0 0x7f82dc069550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f82f00b5cd0 tx=0x7f82e4007d30 comp rx=0 tx=0).stop 2023-10-08T02:31:03.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82f00b16d0 msgr2=0x7f82f0155cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:03.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82f00b16d0 0x7f82f0155cf0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f82e000c810 tx=0x7f82e0007d20 comp rx=0 tx=0).stop 2023-10-08T02:31:03.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 shutdown_connections 2023-10-08T02:31:03.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82dc0670a0 0x7f82dc069550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:03.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f82f00b4e70 0x7f82f0153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:03.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82f00b16d0 0x7f82f0155cf0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:03.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 --2- 172.21.15.119:0/1138308831 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f82f00b1b40 0x7f82f0169960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:03.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 >> 172.21.15.119:0/1138308831 conn(0x7f82f00d10c0 msgr2=0x7f82f00cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:03.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 shutdown_connections 2023-10-08T02:31:03.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.457+0000 7f82f579d700 1 -- 172.21.15.119:0/1138308831 wait complete. 2023-10-08T02:31:03.476 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:31:03.477 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:31:03.556 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:31:03.635 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:31:03.635 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T02:31:03.636 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:03.636 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T02:31:03.727 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:31:03.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:03.728 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T02:31:03.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:03.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.991+0000 7fed14d9d700 1 Processor -- start 2023-10-08T02:31:03.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.992+0000 7fed14d9d700 1 -- start start 2023-10-08T02:31:03.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed14d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed10159420 0x7fed100b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:03.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed14d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed101597f0 0x7fed100b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:03.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed14d9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed100b11d0 con 0x7fed1015d230 2023-10-08T02:31:03.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed14d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed100b14b0 con 0x7fed10159420 2023-10-08T02:31:03.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed14d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed100b1790 con 0x7fed101597f0 2023-10-08T02:31:03.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed0f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed10159420 0x7fed100b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:03.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed0f7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed10159420 0x7fed100b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58548/0 (socket says 172.21.15.119:58548) 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed0f7fe700 1 -- 172.21.15.119:0/2505000772 learned_addr learned my addr 172.21.15.119:0/2505000772 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed0ffff700 1 --1- 172.21.15.119:0/2505000772 >> v1:172.21.15.119:6789/0 conn(0x7fed1015d230 0x7fed100b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35284/0 (socket says 172.21.15.119:35284) 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.993+0000 7fed0effd700 1 --2- 172.21.15.119:0/2505000772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed101597f0 0x7fed100b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0f7fe700 1 -- 172.21.15.119:0/2505000772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed101597f0 msgr2=0x7fed100b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0f7fe700 1 --2- 172.21.15.119:0/2505000772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed101597f0 0x7fed100b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0f7fe700 1 -- 172.21.15.119:0/2505000772 >> v1:172.21.15.119:6789/0 conn(0x7fed1015d230 legacy=0x7fed100b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0f7fe700 1 -- 172.21.15.119:0/2505000772 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed100b1a70 con 0x7fed10159420 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0effd700 1 --2- 172.21.15.119:0/2505000772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed101597f0 0x7fed100b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0e7fc700 1 -- 172.21.15.119:0/2505000772 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056488190 0 0) 0x7fed100b11d0 con 0x7fed1015d230 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0f7fe700 1 --2- 172.21.15.119:0/2505000772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed10159420 0x7fed100b5f40 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fed000099b0 tx=0x7fed00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604e021841a08df7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:03.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0e7fc700 1 -- 172.21.15.119:0/2505000772 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed00009ea0 con 0x7fed10159420 2023-10-08T02:31:03.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0e7fc700 1 -- 172.21.15.119:0/2505000772 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0000b860 con 0x7fed10159420 2023-10-08T02:31:03.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.994+0000 7fed0e7fc700 1 -- 172.21.15.119:0/2505000772 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0000bc00 con 0x7fed10159420 2023-10-08T02:31:03.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.995+0000 7fed14d9d700 1 -- 172.21.15.119:0/2505000772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed10159420 msgr2=0x7fed100b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.995+0000 7fed14d9d700 1 --2- 172.21.15.119:0/2505000772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed10159420 0x7fed100b5f40 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fed000099b0 tx=0x7fed00009250 comp rx=0 tx=0).stop 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.995+0000 7fed14d9d700 1 -- 172.21.15.119:0/2505000772 shutdown_connections 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.995+0000 7fed14d9d700 1 --2- 172.21.15.119:0/2505000772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed10159420 0x7fed100b5f40 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.995+0000 7fed14d9d700 1 --2- 172.21.15.119:0/2505000772 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed101597f0 0x7fed100b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.995+0000 7fed14d9d700 1 -- 172.21.15.119:0/2505000772 >> 172.21.15.119:0/2505000772 conn(0x7fed100d0660 msgr2=0x7fed100d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.995+0000 7fed14d9d700 1 -- 172.21.15.119:0/2505000772 shutdown_connections 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.995+0000 7fed14d9d700 1 -- 172.21.15.119:0/2505000772 wait complete. 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.997+0000 7fed14d9d700 1 Processor -- start 2023-10-08T02:31:03.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.997+0000 7fed14d9d700 1 -- start start 2023-10-08T02:31:04.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed14d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed10159420 0x7fed10157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed14d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed101597f0 0x7fed100c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed14d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1015d230 0x7fed100c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed14d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed10157a60 con 0x7fed101597f0 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed14d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed100c99d0 con 0x7fed1015d230 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed14d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1015b1a0 con 0x7fed10159420 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed0effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed101597f0 0x7fed100c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed0effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed101597f0 0x7fed100c7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58552/0 (socket says 172.21.15.119:58552) 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed0effd700 1 -- 172.21.15.119:0/245260726 learned_addr learned my addr 172.21.15.119:0/245260726 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed0f7fe700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed10159420 0x7fed10157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.998+0000 7fed0ffff700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1015d230 0x7fed100c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fed0effd700 1 -- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed10159420 msgr2=0x7fed10157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fed0effd700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed10159420 0x7fed10157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fed0effd700 1 -- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1015d230 msgr2=0x7fed100c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fed0effd700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1015d230 0x7fed100c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fed0effd700 1 -- 172.21.15.119:0/245260726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04009420 con 0x7fed101597f0 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fed0f7fe700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed10159420 0x7fed10157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fed0effd700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed101597f0 0x7fed100c7890 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fed0400ec80 tx=0x7fed040091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fecf77fe700 1 -- 172.21.15.119:0/245260726 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0400c920 con 0x7fed101597f0 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fecf77fe700 1 -- 172.21.15.119:0/245260726 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0400ca80 con 0x7fed101597f0 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fecf77fe700 1 -- 172.21.15.119:0/245260726 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0400ce00 con 0x7fed101597f0 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:03.999+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed000094f0 con 0x7fed101597f0 2023-10-08T02:31:04.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.000+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1015b760 con 0x7fed101597f0 2023-10-08T02:31:04.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.000+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed100c9b90 con 0x7fed101597f0 2023-10-08T02:31:04.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.002+0000 7fecf77fe700 1 -- 172.21.15.119:0/245260726 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fed04013680 con 0x7fed101597f0 2023-10-08T02:31:04.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.003+0000 7fecf77fe700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fecfc067050 0x7fecfc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.003+0000 7fed0f7fe700 1 -- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fecfc067050 msgr2=0x7fecfc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:04.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.003+0000 7fecf77fe700 1 -- 172.21.15.119:0/245260726 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6455+0+0 (secure 0 0 0) 0x7fed04084d80 con 0x7fed101597f0 2023-10-08T02:31:04.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.003+0000 7fed0f7fe700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fecfc067050 0x7fecfc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:04.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.007+0000 7fecf77fe700 1 -- 172.21.15.119:0/245260726 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fed04051bf0 con 0x7fed101597f0 2023-10-08T02:31:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.204+0000 7fed0f7fe700 1 -- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fecfc067050 msgr2=0x7fecfc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.204+0000 7fed0f7fe700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fecfc067050 0x7fecfc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:04.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.251+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed1004c3c0 con 0x7fed101597f0 2023-10-08T02:31:04.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.251+0000 7fecf77fe700 1 -- 172.21.15.119:0/245260726 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4610 (secure 0 0 0) 0x7fed04055210 con 0x7fed101597f0 2023-10-08T02:31:04.252 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:04.252 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3041824229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3041824229},{"type":"v1","addr":"172.21.15.184:6834","nonce":3041824229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1478841333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1478841333},{"type":"v1","addr":"172.21.15.119:6834","nonce":1478841333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9723,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/483355636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":483355636},{"type":"v1","addr":"172.21.15.119:6835","nonce":483355636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:28:39.274713+0000","modified":"2023-10-08T02:28:42.824801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9726":{"gid":9726,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3185561357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3185561357},{"type":"v1","addr":"172.21.15.184:6835","nonce":3185561357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T02:31:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.253+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fecfc067050 msgr2=0x7fecfc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:04.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.253+0000 7fed14d9d700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fecfc067050 0x7fecfc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.253+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed101597f0 msgr2=0x7fed100c7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.253+0000 7fed14d9d700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed101597f0 0x7fed100c7890 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fed0400ec80 tx=0x7fed040091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.254+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 shutdown_connections 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.254+0000 7fed14d9d700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fecfc067050 0x7fecfc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.254+0000 7fed14d9d700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fed10159420 0x7fed10157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.254+0000 7fed14d9d700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fed1015d230 0x7fed100c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.254+0000 7fed14d9d700 1 --2- 172.21.15.119:0/245260726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed101597f0 0x7fed100c7890 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.254+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 >> 172.21.15.119:0/245260726 conn(0x7fed100d0660 msgr2=0x7fed100ce980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.254+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 shutdown_connections 2023-10-08T02:31:04.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.254+0000 7fed14d9d700 1 -- 172.21.15.119:0/245260726 wait complete. 2023-10-08T02:31:04.256 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 87 2023-10-08T02:31:04.273 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:04.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.460+0000 7ff1dd39d700 1 Processor -- start 2023-10-08T02:31:04.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.460+0000 7ff1dd39d700 1 -- start start 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.461+0000 7ff1dd39d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff1d8159420 0x7ff1d80b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.461+0000 7ff1dd39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 0x7ff1d815d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.461+0000 7ff1dd39d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80b2980 con 0x7ff1d81597f0 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.461+0000 7ff1dd39d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d815dbb0 con 0x7ff1d8159420 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.461+0000 7ff1dd39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80b11d0 con 0x7ff1d815d230 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff1d8159420 0x7ff1d80b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff1d8159420 0x7ff1d80b5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58554/0 (socket says 172.21.15.119:58554) 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d77fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff1d81597f0 0x7ff1d80b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35290/0 (socket says 172.21.15.119:35290) 2023-10-08T02:31:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d7fff700 1 -- 172.21.15.119:0/4240990004 learned_addr learned my addr 172.21.15.119:0/4240990004 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1dcb9c700 1 --2- 172.21.15.119:0/4240990004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 0x7ff1d815d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d7fff700 1 -- 172.21.15.119:0/4240990004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 msgr2=0x7ff1d815d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d7fff700 1 --2- 172.21.15.119:0/4240990004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 0x7ff1d815d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d7fff700 1 -- 172.21.15.119:0/4240990004 >> v1:172.21.15.119:6789/0 conn(0x7ff1d81597f0 legacy=0x7ff1d80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d7fff700 1 -- 172.21.15.119:0/4240990004 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80b14b0 con 0x7ff1d8159420 2023-10-08T02:31:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d6ffd700 1 -- 172.21.15.119:0/4240990004 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142709230 0 0) 0x7ff1d80b2980 con 0x7ff1d81597f0 2023-10-08T02:31:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.462+0000 7ff1d7fff700 1 --2- 172.21.15.119:0/4240990004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d8159420 0x7ff1d80b5f40 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80099b0 tx=0x7ff1c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e51610ce8301582 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:04.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1d6ffd700 1 -- 172.21.15.119:0/4240990004 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8009ea0 con 0x7ff1d8159420 2023-10-08T02:31:04.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1d6ffd700 1 -- 172.21.15.119:0/4240990004 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c800b860 con 0x7ff1d8159420 2023-10-08T02:31:04.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1d6ffd700 1 -- 172.21.15.119:0/4240990004 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800bc00 con 0x7ff1d8159420 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1dd39d700 1 -- 172.21.15.119:0/4240990004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d8159420 msgr2=0x7ff1d80b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/4240990004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d8159420 0x7ff1d80b5f40 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7ff1c80099b0 tx=0x7ff1c8009250 comp rx=0 tx=0).stop 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1dd39d700 1 -- 172.21.15.119:0/4240990004 shutdown_connections 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/4240990004 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 0x7ff1d815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/4240990004 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d8159420 0x7ff1d80b5f40 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.463+0000 7ff1dd39d700 1 -- 172.21.15.119:0/4240990004 >> 172.21.15.119:0/4240990004 conn(0x7ff1d80d0660 msgr2=0x7ff1d80d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.464+0000 7ff1dd39d700 1 -- 172.21.15.119:0/4240990004 shutdown_connections 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.464+0000 7ff1dd39d700 1 -- 172.21.15.119:0/4240990004 wait complete. 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.465+0000 7ff1dd39d700 1 Processor -- start 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.465+0000 7ff1dd39d700 1 -- start start 2023-10-08T02:31:04.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1dd39d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1d8159420 0x7ff1d816a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1dd39d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d81597f0 0x7ff1d814cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1dd39d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 0x7ff1d80b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1dd39d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d816a9e0 con 0x7ff1d81597f0 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1dd39d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8152590 con 0x7ff1d8159420 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1dd39d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8152870 con 0x7ff1d815d230 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1d77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d81597f0 0x7ff1d814cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1d77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d81597f0 0x7ff1d814cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58568/0 (socket says 172.21.15.119:58568) 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1d77fe700 1 -- 172.21.15.119:0/2285471118 learned_addr learned my addr 172.21.15.119:0/2285471118 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.466+0000 7ff1dcb9c700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 0x7ff1d80b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1d7fff700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1d8159420 0x7ff1d816a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1d77fe700 1 -- 172.21.15.119:0/2285471118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 msgr2=0x7ff1d80b56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1d77fe700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 0x7ff1d80b56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1d77fe700 1 -- 172.21.15.119:0/2285471118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1d8159420 msgr2=0x7ff1d816a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1d77fe700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1d8159420 0x7ff1d816a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1d77fe700 1 -- 172.21.15.119:0/2285471118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c80094f0 con 0x7ff1d81597f0 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1d77fe700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d81597f0 0x7ff1d814cb90 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c7e0 tx=0x7ff1cc007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1bffff700 1 -- 172.21.15.119:0/2285471118 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc005b40 con 0x7ff1d81597f0 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1bffff700 1 -- 172.21.15.119:0/2285471118 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc005cc0 con 0x7ff1d81597f0 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d8152b50 con 0x7ff1d81597f0 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1bffff700 1 -- 172.21.15.119:0/2285471118 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc007320 con 0x7ff1d81597f0 2023-10-08T02:31:04.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.467+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d8153110 con 0x7ff1d81597f0 2023-10-08T02:31:04.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.469+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d8153330 con 0x7ff1d81597f0 2023-10-08T02:31:04.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.469+0000 7ff1bffff700 1 -- 172.21.15.119:0/2285471118 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7ff1cc01b3b0 con 0x7ff1d81597f0 2023-10-08T02:31:04.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.470+0000 7ff1bffff700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff1c4067050 0x7ff1c4069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.470+0000 7ff1bffff700 1 -- 172.21.15.119:0/2285471118 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 6455+0+0 (secure 0 0 0) 0x7ff1cc083f10 con 0x7ff1d81597f0 2023-10-08T02:31:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.470+0000 7ff1d7fff700 1 -- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff1c4067050 msgr2=0x7ff1c4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:04.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.470+0000 7ff1d7fff700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff1c4067050 0x7ff1c4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:04.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.481+0000 7ff1bffff700 1 -- 172.21.15.119:0/2285471118 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1cc050ce0 con 0x7ff1d81597f0 2023-10-08T02:31:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.671+0000 7ff1d7fff700 1 -- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff1c4067050 msgr2=0x7ff1c4069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:04.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.671+0000 7ff1d7fff700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff1c4067050 0x7ff1c4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:04.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.686+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1d804c3c0 con 0x7ff1d81597f0 2023-10-08T02:31:04.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.687+0000 7ff1bffff700 1 -- 172.21.15.119:0/2285471118 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16657 (secure 0 0 0) 0x7ff1cc017020 con 0x7ff1d81597f0 2023-10-08T02:31:04.688 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:04.688 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":193,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:04.457801+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:28:37.482739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T02:28:38.257857+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":193,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/134108":"2023-10-09T02:28:59.483033+0000","172.21.15.119:6836/134108":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/1455500934":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/510634805":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/2072587307":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/3076563225":"2023-10-09T02:28:59.483033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:04.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff1c4067050 msgr2=0x7ff1c4069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:04.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff1c4067050 0x7ff1c4069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d81597f0 msgr2=0x7ff1d814cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d81597f0 0x7ff1d814cb90 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c7e0 tx=0x7ff1cc007de0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 shutdown_connections 2023-10-08T02:31:04.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff1c4067050 0x7ff1c4069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff1d815d230 0x7ff1d80b56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff1d8159420 0x7ff1d816a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 --2- 172.21.15.119:0/2285471118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff1d81597f0 0x7ff1d814cb90 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 >> 172.21.15.119:0/2285471118 conn(0x7ff1d80d0660 msgr2=0x7ff1d80b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:04.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 shutdown_connections 2023-10-08T02:31:04.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.689+0000 7ff1dd39d700 1 -- 172.21.15.119:0/2285471118 wait complete. 2023-10-08T02:31:04.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T02:28:37.482739+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T02:28:38.257857+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '193', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 193, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T02:31:04.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:31:04.754 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:04.753+0000 7fb629532180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:31:04.808 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:04.808+0000 7fe440e7e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:31:04.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.910+0000 7fb617fff700 1 Processor -- start 2023-10-08T02:31:04.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.910+0000 7fb617fff700 1 -- start start 2023-10-08T02:31:04.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb617fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb61815a3b0 0x7fb61815a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb617fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb61806e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb617fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6180aea70 con 0x7fb6180aef00 2023-10-08T02:31:04.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb617fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb61806ee30 con 0x7fb61815a3b0 2023-10-08T02:31:04.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb617fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61815dad0 con 0x7fb61806e470 2023-10-08T02:31:04.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb616ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb61815a3b0 0x7fb61815a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb616ffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb61815a3b0 0x7fb61815a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48604/0 (socket says 172.21.15.119:48604) 2023-10-08T02:31:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb616ffd700 1 -- 172.21.15.119:0/965619447 learned_addr learned my addr 172.21.15.119:0/965619447 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb6167fc700 1 --1- 172.21.15.119:0/965619447 >> v1:172.21.15.119:6789/0 conn(0x7fb6180aef00 0x7fb61815d3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33566/0 (socket says 172.21.15.119:33566) 2023-10-08T02:31:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb6177fe700 1 --2- 172.21.15.119:0/965619447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb61806e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb616ffd700 1 -- 172.21.15.119:0/965619447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 msgr2=0x7fb61806e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb616ffd700 1 --2- 172.21.15.119:0/965619447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb61806e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb616ffd700 1 -- 172.21.15.119:0/965619447 >> v1:172.21.15.119:6789/0 conn(0x7fb6180aef00 legacy=0x7fb61815d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.912+0000 7fb616ffd700 1 -- 172.21.15.119:0/965619447 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6180ae480 con 0x7fb61815a3b0 2023-10-08T02:31:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.913+0000 7fb616ffd700 1 --2- 172.21.15.119:0/965619447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61815a3b0 0x7fb61815a780 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb6080099b0 tx=0x7fb608009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b60ac7822354b22 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:04.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.913+0000 7fb615ffb700 1 -- 172.21.15.119:0/965619447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608009ea0 con 0x7fb61815a3b0 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.913+0000 7fb615ffb700 1 -- 172.21.15.119:0/965619447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60800b860 con 0x7fb61815a3b0 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.913+0000 7fb615ffb700 1 -- 172.21.15.119:0/965619447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60800bc00 con 0x7fb61815a3b0 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.914+0000 7fb617fff700 1 -- 172.21.15.119:0/965619447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61815a3b0 msgr2=0x7fb61815a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.914+0000 7fb617fff700 1 --2- 172.21.15.119:0/965619447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61815a3b0 0x7fb61815a780 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fb6080099b0 tx=0x7fb608009250 comp rx=0 tx=0).stop 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.914+0000 7fb617fff700 1 -- 172.21.15.119:0/965619447 shutdown_connections 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.914+0000 7fb617fff700 1 --2- 172.21.15.119:0/965619447 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb61806e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.914+0000 7fb617fff700 1 --2- 172.21.15.119:0/965619447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb61815a3b0 0x7fb61815a780 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.914+0000 7fb617fff700 1 -- 172.21.15.119:0/965619447 >> 172.21.15.119:0/965619447 conn(0x7fb6180c9f70 msgr2=0x7fb61806dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.914+0000 7fb617fff700 1 -- 172.21.15.119:0/965619447 shutdown_connections 2023-10-08T02:31:04.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.914+0000 7fb617fff700 1 -- 172.21.15.119:0/965619447 wait complete. 2023-10-08T02:31:04.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.915+0000 7fb617fff700 1 Processor -- start 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.915+0000 7fb617fff700 1 -- start start 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.916+0000 7fb617fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb6180b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.916+0000 7fb617fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6180aef00 0x7fb6180b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.916+0000 7fb617fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb61815a3b0 0x7fb61816a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.916+0000 7fb617fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6180b24c0 con 0x7fb6180aef00 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.916+0000 7fb617fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61816bf90 con 0x7fb61815a3b0 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.916+0000 7fb617fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb618142a80 con 0x7fb61806e470 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.916+0000 7fb6167fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6180aef00 0x7fb6180b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb6167fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6180aef00 0x7fb6180b1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48612/0 (socket says 172.21.15.119:48612) 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb6167fc700 1 -- 172.21.15.119:0/1744125209 learned_addr learned my addr 172.21.15.119:0/1744125209 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb616ffd700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb6180b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb6177fe700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb61815a3b0 0x7fb61816a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb616ffd700 1 -- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb61815a3b0 msgr2=0x7fb61816a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb616ffd700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb61815a3b0 0x7fb61816a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb616ffd700 1 -- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6180aef00 msgr2=0x7fb6180b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb616ffd700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6180aef00 0x7fb6180b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb616ffd700 1 -- 172.21.15.119:0/1744125209 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6080094f0 con 0x7fb61806e470 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb6167fc700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6180aef00 0x7fb6180b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.917+0000 7fb616ffd700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb6180b1a40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb61806d7e0 tx=0x7fb608017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.918+0000 7fb5feffd700 1 -- 172.21.15.119:0/1744125209 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608000d10 con 0x7fb61806e470 2023-10-08T02:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.918+0000 7fb5feffd700 1 -- 172.21.15.119:0/1744125209 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60800bbe0 con 0x7fb61806e470 2023-10-08T02:31:04.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.918+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61816b380 con 0x7fb61806e470 2023-10-08T02:31:04.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.918+0000 7fb5feffd700 1 -- 172.21.15.119:0/1744125209 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608011690 con 0x7fb61806e470 2023-10-08T02:31:04.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.918+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb61816b940 con 0x7fb61806e470 2023-10-08T02:31:04.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.919+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb61806cfa0 con 0x7fb61806e470 2023-10-08T02:31:04.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.920+0000 7fb5feffd700 1 -- 172.21.15.119:0/1744125209 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7fb608011ac0 con 0x7fb61806e470 2023-10-08T02:31:04.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.921+0000 7fb5feffd700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb604067050 0x7fb604069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:04.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.921+0000 7fb5feffd700 1 -- 172.21.15.119:0/1744125209 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7fb608086600 con 0x7fb61806e470 2023-10-08T02:31:04.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.921+0000 7fb6167fc700 1 -- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb604067050 msgr2=0x7fb604069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:04.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.921+0000 7fb6167fc700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb604067050 0x7fb604069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:04.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:04.924+0000 7fb5feffd700 1 -- 172.21.15.119:0/1744125209 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb608053420 con 0x7fb61806e470 2023-10-08T02:31:04.927 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:04.926+0000 7fb629532180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:04.981 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:04.981+0000 7fe440e7e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:05.012 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:05.012+0000 7fb629532180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:31:05.070 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:05.069+0000 7fe440e7e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:31:05.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.122+0000 7fb6167fc700 1 -- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb604067050 msgr2=0x7fb604069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:05.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.122+0000 7fb6167fc700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb604067050 0x7fb604069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:05.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.124+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb61804bb80 con 0x7fb61806e470 2023-10-08T02:31:05.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.124+0000 7fb5feffd700 1 -- 172.21.15.119:0/1744125209 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb608056a40 con 0x7fb61806e470 2023-10-08T02:31:05.125 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb604067050 msgr2=0x7fb604069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb604067050 0x7fb604069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 msgr2=0x7fb6180b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb6180b1a40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb61806d7e0 tx=0x7fb608017000 comp rx=0 tx=0).stop 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 shutdown_connections 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb604067050 0x7fb604069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb61806e470 0x7fb6180b1a40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb61815a3b0 0x7fb61816a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 --2- 172.21.15.119:0/1744125209 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6180aef00 0x7fb6180b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 >> 172.21.15.119:0/1744125209 conn(0x7fb6180c9f70 msgr2=0x7fb61815bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:05.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 shutdown_connections 2023-10-08T02:31:05.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.127+0000 7fb617fff700 1 -- 172.21.15.119:0/1744125209 wait complete. 2023-10-08T02:31:05.147 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T02:31:05.239 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:05.238+0000 7fb629532180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:05.305 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:05.304+0000 7fe440e7e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.334+0000 7f149a59d700 1 Processor -- start 2023-10-08T02:31:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.335+0000 7f149a59d700 1 -- start start 2023-10-08T02:31:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149a59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149a59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14940b16d0 0x7f14940b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149a59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f149415d240 0x7f149415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149a59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14940b2080 con 0x7f149415d240 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149a59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14940cfb00 con 0x7f14940b16d0 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149a59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940b5c40 con 0x7f14940b4e70 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f1499d9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f149415d240 0x7f149415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149959b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f1498d9a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f14940b16d0 0x7f14940b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149959b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52698/0 (socket says 172.21.15.119:52698) 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.336+0000 7f149959b700 1 -- 172.21.15.119:0/3246289633 learned_addr learned my addr 172.21.15.119:0/3246289633 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f1499d9c700 1 -- 172.21.15.119:0/3246289633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 msgr2=0x7f14940ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f1499d9c700 1 --2- 172.21.15.119:0/3246289633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f1499d9c700 1 -- 172.21.15.119:0/3246289633 >> v2:172.21.15.184:3300/0 conn(0x7f14940b16d0 msgr2=0x7f14940b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f1499d9c700 1 --2- 172.21.15.119:0/3246289633 >> v2:172.21.15.184:3300/0 conn(0x7f14940b16d0 0x7f14940b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f1499d9c700 1 -- 172.21.15.119:0/3246289633 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940b5f20 con 0x7f149415d240 2023-10-08T02:31:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f149959b700 1 --2- 172.21.15.119:0/3246289633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f1498d9a700 1 --2- 172.21.15.119:0/3246289633 >> v2:172.21.15.184:3300/0 conn(0x7f14940b16d0 0x7f14940b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f1499d9c700 1 --2- 172.21.15.119:0/3246289633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f149415d240 0x7f149415d6c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1494089440 tx=0x7f148c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae285451fad11589 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:05.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f148bfff700 1 -- 172.21.15.119:0/3246289633 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00e070 con 0x7f149415d240 2023-10-08T02:31:05.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f148bfff700 1 -- 172.21.15.119:0/3246289633 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c00bab0 con 0x7f149415d240 2023-10-08T02:31:05.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.337+0000 7f148bfff700 1 -- 172.21.15.119:0/3246289633 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c00be70 con 0x7f149415d240 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 -- 172.21.15.119:0/3246289633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f149415d240 msgr2=0x7f149415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 --2- 172.21.15.119:0/3246289633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f149415d240 0x7f149415d6c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f1494089440 tx=0x7f148c009780 comp rx=0 tx=0).stop 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 -- 172.21.15.119:0/3246289633 shutdown_connections 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 --2- 172.21.15.119:0/3246289633 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f149415d240 0x7f149415d6c0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 --2- 172.21.15.119:0/3246289633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 --2- 172.21.15.119:0/3246289633 >> v2:172.21.15.184:3300/0 conn(0x7f14940b16d0 0x7f14940b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 -- 172.21.15.119:0/3246289633 >> 172.21.15.119:0/3246289633 conn(0x7f14940d10c0 msgr2=0x7f14940affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 -- 172.21.15.119:0/3246289633 shutdown_connections 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.338+0000 7f149a59d700 1 -- 172.21.15.119:0/3246289633 wait complete. 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.339+0000 7f149a59d700 1 Processor -- start 2023-10-08T02:31:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.340+0000 7f149a59d700 1 -- start start 2023-10-08T02:31:05.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.340+0000 7f149a59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14940b16d0 0x7f14940c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149a59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149a59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149415d240 0x7f14941459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149a59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14940cb360 con 0x7f14940b16d0 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149a59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14940c72e0 con 0x7f149415d240 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149a59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940c6a40 con 0x7f14940b4e70 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14940b16d0 0x7f14940c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14940b16d0 0x7f14940c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48630/0 (socket says 172.21.15.119:48630) 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 -- 172.21.15.119:0/3752258105 learned_addr learned my addr 172.21.15.119:0/3752258105 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f1499d9c700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149415d240 0x7f14941459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f1498d9a700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 -- 172.21.15.119:0/3752258105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 msgr2=0x7f14940c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 -- 172.21.15.119:0/3752258105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149415d240 msgr2=0x7f14941459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149415d240 0x7f14941459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 -- 172.21.15.119:0/3752258105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1484009420 con 0x7f14940b16d0 2023-10-08T02:31:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.341+0000 7f149959b700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14940b16d0 0x7f14940c7f90 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f149000ec50 tx=0x7f1490009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.342+0000 7f14897fa700 1 -- 172.21.15.119:0/3752258105 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000c850 con 0x7f14940b16d0 2023-10-08T02:31:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.342+0000 7f14897fa700 1 -- 172.21.15.119:0/3752258105 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f149000c9b0 con 0x7f14940b16d0 2023-10-08T02:31:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.342+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14900094f0 con 0x7f14940b16d0 2023-10-08T02:31:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.342+0000 7f14897fa700 1 -- 172.21.15.119:0/3752258105 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f149000cd50 con 0x7f14940b16d0 2023-10-08T02:31:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.342+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148c009a20 con 0x7f14940b16d0 2023-10-08T02:31:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.343+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14940c7120 con 0x7f14940b16d0 2023-10-08T02:31:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.344+0000 7f14897fa700 1 -- 172.21.15.119:0/3752258105 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f1490013680 con 0x7f14940b16d0 2023-10-08T02:31:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.345+0000 7f14897fa700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1478067050 0x7f1478069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.345+0000 7f14897fa700 1 -- 172.21.15.119:0/3752258105 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f1490084b10 con 0x7f14940b16d0 2023-10-08T02:31:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.345+0000 7f1498d9a700 1 -- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1478067050 msgr2=0x7f1478069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:05.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.345+0000 7f1498d9a700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1478067050 0x7f1478069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:05.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.349+0000 7f14897fa700 1 -- 172.21.15.119:0/3752258105 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f14900518b0 con 0x7f14940b16d0 2023-10-08T02:31:05.418 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:05.417+0000 7fb629532180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:31:05.493 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:05.492+0000 7fe440e7e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:31:05.511 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:05.511+0000 7fb629532180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:31:05.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.546+0000 7f1498d9a700 1 -- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1478067050 msgr2=0x7f1478069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:05.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.546+0000 7f1498d9a700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1478067050 0x7f1478069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:05.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.551+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f149404bb80 con 0x7f14940b16d0 2023-10-08T02:31:05.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.551+0000 7f14897fa700 1 -- 172.21.15.119:0/3752258105 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v193) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1490054ed0 con 0x7f14940b16d0 2023-10-08T02:31:05.552 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T02:31:05.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1478067050 msgr2=0x7f1478069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:05.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1478067050 0x7f1478069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14940b16d0 msgr2=0x7f14940c7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14940b16d0 0x7f14940c7f90 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f149000ec50 tx=0x7f1490009250 comp rx=0 tx=0).stop 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 shutdown_connections 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1478067050 0x7f1478069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f149415d240 0x7f14941459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14940b16d0 0x7f14940c7f90 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 --2- 172.21.15.119:0/3752258105 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f14940b4e70 0x7f14940c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.553+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 >> 172.21.15.119:0/3752258105 conn(0x7f14940d10c0 msgr2=0x7f14940b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.554+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 shutdown_connections 2023-10-08T02:31:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.554+0000 7f149a59d700 1 -- 172.21.15.119:0/3752258105 wait complete. 2023-10-08T02:31:05.573 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T02:31:05.590 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:05.589+0000 7fe440e7e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:31:05.674 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:05.674+0000 7fb629532180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:31:05.759 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:05.759+0000 7fe440e7e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:31:05.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.763+0000 7f7b7319d700 1 Processor -- start 2023-10-08T02:31:05.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.764+0000 7f7b7319d700 1 -- start start 2023-10-08T02:31:05.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.765+0000 7f7b7319d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b6c0b1b40 0x7f7b6c0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.765+0000 7f7b7319d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c0cfb00 con 0x7f7b6c0b4e70 2023-10-08T02:31:05.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.765+0000 7f7b7319d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6c0b5c40 con 0x7f7b6c0b16d0 2023-10-08T02:31:05.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.765+0000 7f7b7319d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6c0b5f20 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.765+0000 7f7b7219b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7b6c0b4e70 0x7f7b6c0ccff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33576/0 (socket says 172.21.15.119:33576) 2023-10-08T02:31:05.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.765+0000 7f7b7219b700 1 -- 172.21.15.119:0/945043392 learned_addr learned my addr 172.21.15.119:0/945043392 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:05.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.765+0000 7f7b7299c700 1 --2- 172.21.15.119:0/945043392 >> v2:172.21.15.184:3300/0 conn(0x7f7b6c0b1b40 0x7f7b6c0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b71199700 1 -- 172.21.15.119:0/945043392 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457335373 0 0) 0x7f7b6c0cfb00 con 0x7f7b6c0b4e70 2023-10-08T02:31:05.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b71199700 1 -- 172.21.15.119:0/945043392 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b58003570 con 0x7f7b6c0b4e70 2023-10-08T02:31:05.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b71199700 1 -- 172.21.15.119:0/945043392 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116497844 0 0) 0x7f7b6c0b5c40 con 0x7f7b6c0b16d0 2023-10-08T02:31:05.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b7299c700 1 -- 172.21.15.119:0/945043392 >> v1:172.21.15.184:6789/0 conn(0x7f7b6c0b16d0 legacy=0x7f7b6c0b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b7299c700 1 -- 172.21.15.119:0/945043392 >> v1:172.21.15.119:6789/0 conn(0x7f7b6c0b4e70 legacy=0x7f7b6c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b7299c700 1 -- 172.21.15.119:0/945043392 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c0b6200 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b7299c700 1 --2- 172.21.15.119:0/945043392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c0b3ac0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009e60 tx=0x7f7b64009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb7636b48e2df156 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:05.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b71199700 1 -- 172.21.15.119:0/945043392 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6400e070 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b71199700 1 -- 172.21.15.119:0/945043392 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6400bb60 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.766+0000 7f7b71199700 1 -- 172.21.15.119:0/945043392 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b64012430 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.767+0000 7f7b7319d700 1 -- 172.21.15.119:0/945043392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b1b40 msgr2=0x7f7b6c0b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.767+0000 7f7b7319d700 1 --2- 172.21.15.119:0/945043392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c0b3ac0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009e60 tx=0x7f7b64009780 comp rx=0 tx=0).stop 2023-10-08T02:31:05.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.767+0000 7f7b7319d700 1 -- 172.21.15.119:0/945043392 shutdown_connections 2023-10-08T02:31:05.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.767+0000 7f7b7319d700 1 --2- 172.21.15.119:0/945043392 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c0b3ac0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.767+0000 7f7b7319d700 1 -- 172.21.15.119:0/945043392 >> 172.21.15.119:0/945043392 conn(0x7f7b6c0d10c0 msgr2=0x7f7b6c0affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:05.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.767+0000 7f7b7319d700 1 -- 172.21.15.119:0/945043392 shutdown_connections 2023-10-08T02:31:05.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.767+0000 7f7b7319d700 1 -- 172.21.15.119:0/945043392 wait complete. 2023-10-08T02:31:05.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.769+0000 7f7b7319d700 1 Processor -- start 2023-10-08T02:31:05.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.769+0000 7f7b7319d700 1 -- start start 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7319d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0b16d0 0x7f7b6c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7319d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c169960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7319d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b4e70 0x7f7b6c153430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7319d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c156c80 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7319d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c151c20 con 0x7f7b6c0b4e70 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7319d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6c156230 con 0x7f7b6c0b16d0 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c169960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7199a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c169960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48640/0 (socket says 172.21.15.119:48640) 2023-10-08T02:31:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7199a700 1 -- 172.21.15.119:0/310083476 learned_addr learned my addr 172.21.15.119:0/310083476 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:05.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7299c700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b4e70 0x7f7b6c153430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7219b700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0b16d0 0x7f7b6c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:05.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7199a700 1 -- 172.21.15.119:0/310083476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0b16d0 msgr2=0x7f7b6c155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7199a700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0b16d0 0x7f7b6c155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7199a700 1 -- 172.21.15.119:0/310083476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b4e70 msgr2=0x7f7b6c153430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7199a700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b4e70 0x7f7b6c153430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.770+0000 7f7b7199a700 1 -- 172.21.15.119:0/310083476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64009a20 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.771+0000 7f7b7199a700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c169960 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c7e0 tx=0x7f7b5c007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.771+0000 7f7b627fc700 1 -- 172.21.15.119:0/310083476 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0105c0 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.771+0000 7f7b627fc700 1 -- 172.21.15.119:0/310083476 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5c010740 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.771+0000 7f7b627fc700 1 -- 172.21.15.119:0/310083476 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c010b00 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.771+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6c16da40 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.771+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6c16e000 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.772+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6c088990 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.774+0000 7f7b627fc700 1 -- 172.21.15.119:0/310083476 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f7b5c01b3b0 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.775 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:05.774+0000 7fb629532180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:05.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.775+0000 7f7b627fc700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b50067050 0x7f7b50069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:05.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.775+0000 7f7b627fc700 1 -- 172.21.15.119:0/310083476 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f7b5c083f80 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.775+0000 7f7b7219b700 1 -- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b50067050 msgr2=0x7f7b50069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:05.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.775+0000 7f7b7219b700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b50067050 0x7f7b50069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:05.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.779+0000 7f7b627fc700 1 -- 172.21.15.119:0/310083476 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b5c050dc0 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.867 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:05.866+0000 7fe440e7e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:05.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.976+0000 7f7b7219b700 1 -- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b50067050 msgr2=0x7f7b50069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:05.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.976+0000 7f7b7219b700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b50067050 0x7f7b50069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:05.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.981+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b6c04bb80 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.981+0000 7f7b627fc700 1 -- 172.21.15.119:0/310083476 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v193) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b5c017070 con 0x7f7b6c0b1b40 2023-10-08T02:31:05.982 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T02:31:05.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.983+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b50067050 msgr2=0x7f7b50069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.983+0000 7f7b7319d700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b50067050 0x7f7b50069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.983+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0b1b40 msgr2=0x7f7b6c169960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.983+0000 7f7b7319d700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c169960 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c7e0 tx=0x7f7b5c007df0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.984+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 shutdown_connections 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.984+0000 7f7b7319d700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b50067050 0x7f7b50069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.984+0000 7f7b7319d700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b6c0b16d0 0x7f7b6c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.984+0000 7f7b7319d700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b6c0b4e70 0x7f7b6c153430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.984+0000 7f7b7319d700 1 --2- 172.21.15.119:0/310083476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b6c0b1b40 0x7f7b6c169960 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.984+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 >> 172.21.15.119:0/310083476 conn(0x7f7b6c0d10c0 msgr2=0x7f7b6c0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.984+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 shutdown_connections 2023-10-08T02:31:05.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:05.984+0000 7f7b7319d700 1 -- 172.21.15.119:0/310083476 wait complete. 2023-10-08T02:31:06.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:06.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.235+0000 7ff33679d700 1 Processor -- start 2023-10-08T02:31:06.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.235+0000 7ff33679d700 1 -- start start 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.236+0000 7ff33679d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff330142ea0 0x7ff3300b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.236+0000 7ff33679d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff33006c0f0 0x7ff33006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.236+0000 7ff33679d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3300b40f0 con 0x7ff3300ad870 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.236+0000 7ff33679d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff330069640 con 0x7ff330142ea0 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.236+0000 7ff33679d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff330069920 con 0x7ff33006c0f0 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.237+0000 7ff33579b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff330142ea0 0x7ff3300b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.237+0000 7ff33579b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff330142ea0 0x7ff3300b3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48646/0 (socket says 172.21.15.119:48646) 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.237+0000 7ff33579b700 1 -- 172.21.15.119:0/2727377492 learned_addr learned my addr 172.21.15.119:0/2727377492 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.237+0000 7ff335f9c700 1 --2- 172.21.15.119:0/2727377492 >> v2:172.21.15.184:3300/0 conn(0x7ff33006c0f0 0x7ff33006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.237+0000 7ff33579b700 1 -- 172.21.15.119:0/2727377492 >> v2:172.21.15.184:3300/0 conn(0x7ff33006c0f0 msgr2=0x7ff33006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:06.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.237+0000 7ff33579b700 1 --2- 172.21.15.119:0/2727377492 >> v2:172.21.15.184:3300/0 conn(0x7ff33006c0f0 0x7ff33006a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.237+0000 7ff33579b700 1 -- 172.21.15.119:0/2727377492 >> v1:172.21.15.184:6789/0 conn(0x7ff3300ad870 legacy=0x7ff33006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.237+0000 7ff33579b700 1 -- 172.21.15.119:0/2727377492 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff330069c00 con 0x7ff330142ea0 2023-10-08T02:31:06.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff33579b700 1 --2- 172.21.15.119:0/2727377492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff330142ea0 0x7ff3300b3630 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7ff330071330 tx=0x7ff32c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70605651c839ad5 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:06.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff327fff700 1 -- 172.21.15.119:0/2727377492 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c009e70 con 0x7ff330142ea0 2023-10-08T02:31:06.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff327fff700 1 -- 172.21.15.119:0/2727377492 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c00b860 con 0x7ff330142ea0 2023-10-08T02:31:06.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff327fff700 1 -- 172.21.15.119:0/2727377492 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00bc00 con 0x7ff330142ea0 2023-10-08T02:31:06.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff33679d700 1 -- 172.21.15.119:0/2727377492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff330142ea0 msgr2=0x7ff3300b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff33679d700 1 --2- 172.21.15.119:0/2727377492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff330142ea0 0x7ff3300b3630 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7ff330071330 tx=0x7ff32c009250 comp rx=0 tx=0).stop 2023-10-08T02:31:06.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff33679d700 1 -- 172.21.15.119:0/2727377492 shutdown_connections 2023-10-08T02:31:06.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff33679d700 1 --2- 172.21.15.119:0/2727377492 >> v2:172.21.15.184:3300/0 conn(0x7ff33006c0f0 0x7ff33006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff33679d700 1 --2- 172.21.15.119:0/2727377492 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff330142ea0 0x7ff3300b3630 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.238+0000 7ff33679d700 1 -- 172.21.15.119:0/2727377492 >> 172.21.15.119:0/2727377492 conn(0x7ff3300ce430 msgr2=0x7ff3300ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:06.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.239+0000 7ff33679d700 1 -- 172.21.15.119:0/2727377492 shutdown_connections 2023-10-08T02:31:06.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.239+0000 7ff33679d700 1 -- 172.21.15.119:0/2727377492 wait complete. 2023-10-08T02:31:06.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.240+0000 7ff33679d700 1 Processor -- start 2023-10-08T02:31:06.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.240+0000 7ff33679d700 1 -- start start 2023-10-08T02:31:06.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33679d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff33006c0f0 0x7ff330159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33679d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff330142ea0 0x7ff33006a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33679d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff33006a9e0 0x7ff3300c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33679d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff330169d10 con 0x7ff33006c0f0 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33679d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff33016ab30 con 0x7ff330142ea0 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33679d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff33006ae50 con 0x7ff33006a9e0 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff33006c0f0 0x7ff330159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff33006c0f0 0x7ff330159100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48648/0 (socket says 172.21.15.119:48648) 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff33579b700 1 -- 172.21.15.119:0/2697372232 learned_addr learned my addr 172.21.15.119:0/2697372232 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff334f9a700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff330142ea0 0x7ff33006a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.241+0000 7ff335f9c700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff33006a9e0 0x7ff3300c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff33579b700 1 -- 172.21.15.119:0/2697372232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff33006a9e0 msgr2=0x7ff3300c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff33579b700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff33006a9e0 0x7ff3300c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff33579b700 1 -- 172.21.15.119:0/2697372232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff330142ea0 msgr2=0x7ff33006a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff33579b700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff330142ea0 0x7ff33006a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff33579b700 1 -- 172.21.15.119:0/2697372232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0094f0 con 0x7ff33006c0f0 2023-10-08T02:31:06.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff33579b700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff33006c0f0 0x7ff330159100 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7ff33006f0d0 tx=0x7ff32c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:06.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff3257fa700 1 -- 172.21.15.119:0/2697372232 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c00bbe0 con 0x7ff33006c0f0 2023-10-08T02:31:06.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff3257fa700 1 -- 172.21.15.119:0/2697372232 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c011420 con 0x7ff33006c0f0 2023-10-08T02:31:06.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff3257fa700 1 -- 172.21.15.119:0/2697372232 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0117e0 con 0x7ff33006c0f0 2023-10-08T02:31:06.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.242+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff330159640 con 0x7ff33006c0f0 2023-10-08T02:31:06.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.243+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33016a200 con 0x7ff33006c0f0 2023-10-08T02:31:06.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.244+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3300adf10 con 0x7ff33006c0f0 2023-10-08T02:31:06.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.244+0000 7ff3257fa700 1 -- 172.21.15.119:0/2697372232 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7ff32c011c10 con 0x7ff33006c0f0 2023-10-08T02:31:06.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.245+0000 7ff3257fa700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff314067050 0x7ff314069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.245+0000 7ff334f9a700 1 -- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff314067050 msgr2=0x7ff314069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:06.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.245+0000 7ff3257fa700 1 -- 172.21.15.119:0/2697372232 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7ff32c086210 con 0x7ff33006c0f0 2023-10-08T02:31:06.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.245+0000 7ff334f9a700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff314067050 0x7ff314069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:06.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.249+0000 7ff3257fa700 1 -- 172.21.15.119:0/2697372232 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff32c052fb0 con 0x7ff33006c0f0 2023-10-08T02:31:06.263 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:06.263+0000 7fb629532180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:31:06.358 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:06.357+0000 7fb629532180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:31:06.377 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:06.376+0000 7fe440e7e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:31:06.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.446+0000 7ff334f9a700 1 -- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff314067050 msgr2=0x7ff314069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:06.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.446+0000 7ff334f9a700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff314067050 0x7ff314069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:06.449 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:06.447+0000 7fb629532180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:31:06.475 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:06.475+0000 7fe440e7e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:31:06.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.498+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff33004c3c0 con 0x7ff33006c0f0 2023-10-08T02:31:06.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.498+0000 7ff3257fa700 1 -- 172.21.15.119:0/2697372232 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4610 (secure 0 0 0) 0x7ff32c0565d0 con 0x7ff33006c0f0 2023-10-08T02:31:06.499 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:06.499 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3041824229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3041824229},{"type":"v1","addr":"172.21.15.184:6834","nonce":3041824229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1478841333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1478841333},{"type":"v1","addr":"172.21.15.119:6834","nonce":1478841333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9723,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/483355636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":483355636},{"type":"v1","addr":"172.21.15.119:6835","nonce":483355636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:28:39.274713+0000","modified":"2023-10-08T02:28:42.824801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9726":{"gid":9726,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3185561357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3185561357},{"type":"v1","addr":"172.21.15.184:6835","nonce":3185561357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T02:31:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff314067050 msgr2=0x7ff314069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff314067050 0x7ff314069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff33006c0f0 msgr2=0x7ff330159100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff33006c0f0 0x7ff330159100 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7ff33006f0d0 tx=0x7ff32c017000 comp rx=0 tx=0).stop 2023-10-08T02:31:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 shutdown_connections 2023-10-08T02:31:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff314067050 0x7ff314069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff33006a9e0 0x7ff3300c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff33006c0f0 0x7ff330159100 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 --2- 172.21.15.119:0/2697372232 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff330142ea0 0x7ff33006a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 >> 172.21.15.119:0/2697372232 conn(0x7ff3300ce430 msgr2=0x7ff3300cadb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 shutdown_connections 2023-10-08T02:31:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.501+0000 7ff33679d700 1 -- 172.21.15.119:0/2697372232 wait complete. 2023-10-08T02:31:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 87 2023-10-08T02:31:06.520 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:06.572 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:06.572+0000 7fe440e7e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:31:06.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.707+0000 7f7b1499d700 1 Processor -- start 2023-10-08T02:31:06.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.707+0000 7f7b1499d700 1 -- start start 2023-10-08T02:31:06.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.708+0000 7f7b1499d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b100ad870 0x7f7b1006b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.708+0000 7f7b1499d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b1006bf20 0x7f7b1006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.708+0000 7f7b1499d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b100b40f0 con 0x7f7b10142ea0 2023-10-08T02:31:06.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.708+0000 7f7b1499d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b1006c3a0 con 0x7f7b1006bf20 2023-10-08T02:31:06.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.708+0000 7f7b1499d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b10069640 con 0x7f7b100ad870 2023-10-08T02:31:06.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b1006bf20 0x7f7b1006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0effd700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7b100ad870 0x7f7b1006b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0ffff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b1006bf20 0x7f7b1006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48656/0 (socket says 172.21.15.119:48656) 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0ffff700 1 -- 172.21.15.119:0/3574415506 learned_addr learned my addr 172.21.15.119:0/3574415506 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0f7fe700 1 --1- 172.21.15.119:0/3574415506 >> v1:172.21.15.184:6789/0 conn(0x7f7b10142ea0 0x7f7b100b3630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:59180/0 (socket says 172.21.15.119:59180) 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0ffff700 1 -- 172.21.15.119:0/3574415506 >> v2:172.21.15.184:3300/0 conn(0x7f7b100ad870 msgr2=0x7f7b1006b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0ffff700 1 --2- 172.21.15.119:0/3574415506 >> v2:172.21.15.184:3300/0 conn(0x7f7b100ad870 0x7f7b1006b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0ffff700 1 -- 172.21.15.119:0/3574415506 >> v1:172.21.15.184:6789/0 conn(0x7f7b10142ea0 legacy=0x7f7b100b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0ffff700 1 -- 172.21.15.119:0/3574415506 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b10069920 con 0x7f7b1006bf20 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.709+0000 7f7b0ffff700 1 --2- 172.21.15.119:0/3574415506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1006bf20 0x7f7b1006a120 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7b100b0e10 tx=0x7f7b08009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9084d3fe3ee2e2da server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b0e7fc700 1 -- 172.21.15.119:0/3574415506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800e070 con 0x7f7b1006bf20 2023-10-08T02:31:06.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b0e7fc700 1 -- 172.21.15.119:0/3574415506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0800ba20 con 0x7f7b1006bf20 2023-10-08T02:31:06.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b0e7fc700 1 -- 172.21.15.119:0/3574415506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b080092e0 con 0x7f7b1006bf20 2023-10-08T02:31:06.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b1499d700 1 -- 172.21.15.119:0/3574415506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1006bf20 msgr2=0x7f7b1006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b1499d700 1 --2- 172.21.15.119:0/3574415506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1006bf20 0x7f7b1006a120 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f7b100b0e10 tx=0x7f7b08009840 comp rx=0 tx=0).stop 2023-10-08T02:31:06.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b1499d700 1 -- 172.21.15.119:0/3574415506 shutdown_connections 2023-10-08T02:31:06.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b1499d700 1 --2- 172.21.15.119:0/3574415506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1006bf20 0x7f7b1006a120 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b1499d700 1 --2- 172.21.15.119:0/3574415506 >> v2:172.21.15.184:3300/0 conn(0x7f7b100ad870 0x7f7b1006b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b1499d700 1 -- 172.21.15.119:0/3574415506 >> 172.21.15.119:0/3574415506 conn(0x7f7b100ce430 msgr2=0x7f7b100ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:06.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b1499d700 1 -- 172.21.15.119:0/3574415506 shutdown_connections 2023-10-08T02:31:06.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.710+0000 7f7b1499d700 1 -- 172.21.15.119:0/3574415506 wait complete. 2023-10-08T02:31:06.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.713+0000 7f7b1499d700 1 Processor -- start 2023-10-08T02:31:06.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.713+0000 7f7b1499d700 1 -- start start 2023-10-08T02:31:06.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.714+0000 7f7b1499d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1006bf20 0x7f7b100acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.714+0000 7f7b1499d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b10142ea0 0x7f7b1015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.714+0000 7f7b1499d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1015a690 0x7f7b10169cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.714+0000 7f7b1499d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1015d7e0 con 0x7f7b1015a690 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.714+0000 7f7b1499d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b100c88b0 con 0x7f7b1006bf20 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.714+0000 7f7b1499d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b100c7f80 con 0x7f7b10142ea0 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.714+0000 7f7b0ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1015a690 0x7f7b10169cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0f7fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1006bf20 0x7f7b100acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1015a690 0x7f7b10169cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48660/0 (socket says 172.21.15.119:48660) 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0ffff700 1 -- 172.21.15.119:0/4252806984 learned_addr learned my addr 172.21.15.119:0/4252806984 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0effd700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b10142ea0 0x7f7b1015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0ffff700 1 -- 172.21.15.119:0/4252806984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b10142ea0 msgr2=0x7f7b1015cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0ffff700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b10142ea0 0x7f7b1015cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0ffff700 1 -- 172.21.15.119:0/4252806984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1006bf20 msgr2=0x7f7b100acd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0ffff700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1006bf20 0x7f7b100acd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0ffff700 1 -- 172.21.15.119:0/4252806984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b08009ae0 con 0x7f7b1015a690 2023-10-08T02:31:06.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7b0ffff700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1015a690 0x7f7b10169cc0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009880 tx=0x7f7b08018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:06.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.715+0000 7f7af77fe700 1 -- 172.21.15.119:0/4252806984 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08008a80 con 0x7f7b1015a690 2023-10-08T02:31:06.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.716+0000 7f7af77fe700 1 -- 172.21.15.119:0/4252806984 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b08008c00 con 0x7f7b1015a690 2023-10-08T02:31:06.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.716+0000 7f7af77fe700 1 -- 172.21.15.119:0/4252806984 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08026460 con 0x7f7b1015a690 2023-10-08T02:31:06.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.716+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b100c8260 con 0x7f7b1015a690 2023-10-08T02:31:06.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.716+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1015d4d0 con 0x7f7b1015a690 2023-10-08T02:31:06.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.717+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b100c8a40 con 0x7f7b1015a690 2023-10-08T02:31:06.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.718+0000 7f7af77fe700 1 -- 172.21.15.119:0/4252806984 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f7b08003040 con 0x7f7b1015a690 2023-10-08T02:31:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.719+0000 7f7af77fe700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7afc067050 0x7f7afc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.719+0000 7f7af77fe700 1 -- 172.21.15.119:0/4252806984 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f7b08086b60 con 0x7f7b1015a690 2023-10-08T02:31:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.719+0000 7f7b0f7fe700 1 -- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7afc067050 msgr2=0x7f7afc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:06.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.719+0000 7f7b0f7fe700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7afc067050 0x7f7afc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:06.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.726+0000 7f7af77fe700 1 -- 172.21.15.119:0/4252806984 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b08053870 con 0x7f7b1015a690 2023-10-08T02:31:06.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.920+0000 7f7b0f7fe700 1 -- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7afc067050 msgr2=0x7f7afc069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.920+0000 7f7b0f7fe700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7afc067050 0x7f7afc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:06.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.925+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b1004c3c0 con 0x7f7b1015a690 2023-10-08T02:31:06.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.927+0000 7f7af77fe700 1 -- 172.21.15.119:0/4252806984 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16657 (secure 0 0 0) 0x7f7b08021090 con 0x7f7b1015a690 2023-10-08T02:31:06.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:06.928 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":193,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:04.457801+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:28:37.482739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T02:28:38.257857+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":193,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/134108":"2023-10-09T02:28:59.483033+0000","172.21.15.119:6836/134108":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/1455500934":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/510634805":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/2072587307":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/3076563225":"2023-10-09T02:28:59.483033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7afc067050 msgr2=0x7f7afc069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7afc067050 0x7f7afc069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1015a690 msgr2=0x7f7b10169cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1015a690 0x7f7b10169cc0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009880 tx=0x7f7b08018000 comp rx=0 tx=0).stop 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 shutdown_connections 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7afc067050 0x7f7afc069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b1015a690 0x7f7b10169cc0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b1006bf20 0x7f7b100acd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 --2- 172.21.15.119:0/4252806984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b10142ea0 0x7f7b1015cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 >> 172.21.15.119:0/4252806984 conn(0x7f7b100ce430 msgr2=0x7f7b100b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 shutdown_connections 2023-10-08T02:31:06.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:06.929+0000 7f7b1499d700 1 -- 172.21.15.119:0/4252806984 wait complete. 2023-10-08T02:31:06.950 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T02:31:06.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:07.077 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:07.076+0000 7fb629532180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:31:07.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.161+0000 7f36789fe700 1 Processor -- start 2023-10-08T02:31:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.161+0000 7f36789fe700 1 -- start start 2023-10-08T02:31:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.164+0000 7f36789fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 0x7f367406ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.164+0000 7f36789fe700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f367406e8c0 con 0x7f367415daa0 2023-10-08T02:31:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.164+0000 7f36789fe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36740ca740 con 0x7f367415de70 2023-10-08T02:31:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.164+0000 7f36789fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3674069dd0 con 0x7f367406e000 2023-10-08T02:31:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.164+0000 7f3672d9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f367415daa0 0x7f36740c7110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33582/0 (socket says 172.21.15.119:33582) 2023-10-08T02:31:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.164+0000 7f3672d9d700 1 -- 172.21.15.119:0/2670173731 learned_addr learned my addr 172.21.15.119:0/2670173731 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.164+0000 7f367359e700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 0x7f367406ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3253024176 0 0) 0x7f36740ca740 con 0x7f367415de70 2023-10-08T02:31:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3660003570 con 0x7f367415de70 2023-10-08T02:31:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252208234 0 0) 0x7f367406e8c0 con 0x7f367415daa0 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36740ca740 con 0x7f367415daa0 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 <== mon.1 v1:172.21.15.184:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1743693517 0 0) 0x7f3660003570 con 0x7f367415de70 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 msgr2=0x7f367406ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 0x7f367406ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 >> v1:172.21.15.119:6789/0 conn(0x7f367415daa0 legacy=0x7f36740c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 --> v1:172.21.15.184:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367406e5e0 con 0x7f367415de70 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 <== mon.1 v1:172.21.15.184:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f365c0017f0 con 0x7f367415de70 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 >> v1:172.21.15.184:6789/0 conn(0x7f367415de70 legacy=0x7f36740696c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3660003bf0 0x7f3660003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 0x7f3660004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3660004d50 0x7f36600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674069dd0 con 0x7f3660004d50 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366000bc20 con 0x7f3660003bf0 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.165+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366000bde0 con 0x7f367406e000 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f3672d9d700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3660003bf0 0x7f3660003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f367259c700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 0x7f3660004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f367359e700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3660004d50 0x7f36600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f367259c700 1 -- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3660003bf0 msgr2=0x7f3660003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f367259c700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3660003bf0 0x7f3660003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f367259c700 1 -- 172.21.15.119:0/2670173731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3660004d50 msgr2=0x7f36600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f367259c700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3660004d50 0x7f36600051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f367259c700 1 -- 172.21.15.119:0/2670173731 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660003570 con 0x7f367406e000 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.166+0000 7f367359e700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3660004d50 0x7f36600051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.167+0000 7f367259c700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 0x7f3660004810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f365c0031a0 tx=0x7f365c002e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89772106b25837ed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.167+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c00ce60 con 0x7f367406e000 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.167+0000 7f3671d9b700 1 -- 172.21.15.119:0/2670173731 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c034430 con 0x7f367406e000 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.168+0000 7f36789fe700 1 -- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 msgr2=0x7f3660004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.168+0000 7f36789fe700 1 --2- 172.21.15.119:0/2670173731 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f367406e000 0x7f3660004810 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f365c0031a0 tx=0x7f365c002e30 comp rx=0 tx=0).stop 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.168+0000 7f367359e700 1 -- 172.21.15.119:0/2670173731 reap_dead start 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.168+0000 7f36789fe700 1 -- 172.21.15.119:0/2670173731 shutdown_connections 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.168+0000 7f36789fe700 1 -- 172.21.15.119:0/2670173731 >> 172.21.15.119:0/2670173731 conn(0x7f36740cc1f0 msgr2=0x7f36740cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.168+0000 7f36789fe700 1 -- 172.21.15.119:0/2670173731 shutdown_connections 2023-10-08T02:31:07.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.168+0000 7f36789fe700 1 -- 172.21.15.119:0/2670173731 wait complete. 2023-10-08T02:31:07.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.171+0000 7f36789fe700 1 Processor -- start 2023-10-08T02:31:07.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.171+0000 7f36789fe700 1 -- start start 2023-10-08T02:31:07.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f36789fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f367406e000 0x7f367415ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f36789fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3674153020 0x7f3674156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f36789fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36741574a0 0x7f36740c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f36789fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674161480 con 0x7f367406e000 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f36789fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3674160870 con 0x7f36741574a0 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f36789fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3674160b50 con 0x7f3674153020 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f367359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36741574a0 0x7f36740c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f3672d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f367406e000 0x7f367415ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f367359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36741574a0 0x7f36740c4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44876/0 (socket says 172.21.15.119:44876) 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f367359e700 1 -- 172.21.15.119:0/3807307613 learned_addr learned my addr 172.21.15.119:0/3807307613 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.172+0000 7f367259c700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3674153020 0x7f3674156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f367359e700 1 -- 172.21.15.119:0/3807307613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3674153020 msgr2=0x7f3674156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f367359e700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3674153020 0x7f3674156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f367359e700 1 -- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f367406e000 msgr2=0x7f367415ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f367359e700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f367406e000 0x7f367415ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f367359e700 1 -- 172.21.15.119:0/3807307613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3674160e30 con 0x7f36741574a0 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f3672d9d700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f367406e000 0x7f367415ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f367359e700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36741574a0 0x7f36740c4420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3664006f00 tx=0x7f36640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f366affd700 1 -- 172.21.15.119:0/3807307613 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400f8e0 con 0x7f36741574a0 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f366affd700 1 -- 172.21.15.119:0/3807307613 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366400fa40 con 0x7f36741574a0 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f366affd700 1 -- 172.21.15.119:0/3807307613 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366400fe00 con 0x7f36741574a0 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.173+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3674161110 con 0x7f36741574a0 2023-10-08T02:31:07.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.174+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3674161910 con 0x7f36741574a0 2023-10-08T02:31:07.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.175+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3674161630 con 0x7f36741574a0 2023-10-08T02:31:07.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.176+0000 7f366affd700 1 -- 172.21.15.119:0/3807307613 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f3664014650 con 0x7f36741574a0 2023-10-08T02:31:07.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.176+0000 7f366affd700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3654067050 0x7f3654069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.176+0000 7f366affd700 1 -- 172.21.15.119:0/3807307613 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f3664084fe0 con 0x7f36741574a0 2023-10-08T02:31:07.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.176+0000 7f3672d9d700 1 -- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3654067050 msgr2=0x7f3654069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:07.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.177+0000 7f3672d9d700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3654067050 0x7f3654069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:07.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.181+0000 7f366affd700 1 -- 172.21.15.119:0/3807307613 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3674161630 con 0x7f36741574a0 2023-10-08T02:31:07.232 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:07.232+0000 7fe440e7e180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:31:07.251 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:07.250+0000 7fb629532180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:07.347 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:07.346+0000 7fb629532180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:31:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.378+0000 7f3672d9d700 1 -- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3654067050 msgr2=0x7f3654069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.378+0000 7f3672d9d700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3654067050 0x7f3654069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:07.411 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:07.410+0000 7fe440e7e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:07.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.423+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3674161630 con 0x7f36741574a0 2023-10-08T02:31:07.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.424+0000 7f366affd700 1 -- 172.21.15.119:0/3807307613 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f3674161630 con 0x7f36741574a0 2023-10-08T02:31:07.424 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:07.424 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3041824229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3041824229},{"type":"v1","addr":"172.21.15.184:6834","nonce":3041824229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1478841333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1478841333},{"type":"v1","addr":"172.21.15.119:6834","nonce":1478841333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9723,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/483355636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":483355636},{"type":"v1","addr":"172.21.15.119:6835","nonce":483355636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:28:39.274713+0000","modified":"2023-10-08T02:28:42.824801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9726":{"gid":9726,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3185561357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3185561357},{"type":"v1","addr":"172.21.15.184:6835","nonce":3185561357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T02:31:07.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3654067050 msgr2=0x7f3654069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:07.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3654067050 0x7f3654069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36741574a0 msgr2=0x7f36740c4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36741574a0 0x7f36740c4420 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3664006f00 tx=0x7f36640091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 shutdown_connections 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3654067050 0x7f3654069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f36741574a0 0x7f36740c4420 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f367406e000 0x7f367415ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 --2- 172.21.15.119:0/3807307613 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3674153020 0x7f3674156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 >> 172.21.15.119:0/3807307613 conn(0x7f36740cc1f0 msgr2=0x7f36740c0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 shutdown_connections 2023-10-08T02:31:07.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.426+0000 7f36789fe700 1 -- 172.21.15.119:0/3807307613 wait complete. 2023-10-08T02:31:07.429 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 87 2023-10-08T02:31:07.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:07.514 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:07.513+0000 7fe440e7e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:31:07.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.633+0000 7f7ec3fff700 1 Processor -- start 2023-10-08T02:31:07.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.633+0000 7f7ec3fff700 1 -- start start 2023-10-08T02:31:07.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.634+0000 7f7ec3fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7ec415d230 0x7f7ec415d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec3fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec40b2980 con 0x7f7ec41597f0 2023-10-08T02:31:07.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec3fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ec415db70 con 0x7f7ec4159420 2023-10-08T02:31:07.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec3fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ec40b11d0 con 0x7f7ec415d230 2023-10-08T02:31:07.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec27fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7ec41597f0 0x7f7ec40b2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33586/0 (socket says 172.21.15.119:33586) 2023-10-08T02:31:07.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec27fc700 1 -- 172.21.15.119:0/429757229 learned_addr learned my addr 172.21.15.119:0/429757229 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:07.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec37fe700 1 --2- 172.21.15.119:0/429757229 >> v2:172.21.15.184:3300/0 conn(0x7f7ec415d230 0x7f7ec415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec1ffb700 1 -- 172.21.15.119:0/429757229 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547231236 0 0) 0x7f7ec40b2980 con 0x7f7ec41597f0 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec1ffb700 1 -- 172.21.15.119:0/429757229 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eac003570 con 0x7f7ec41597f0 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec1ffb700 1 -- 172.21.15.119:0/429757229 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250027225 0 0) 0x7f7ec415db70 con 0x7f7ec4159420 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.635+0000 7f7ec1ffb700 1 -- 172.21.15.119:0/429757229 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec40b2980 con 0x7f7ec4159420 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.636+0000 7f7ec37fe700 1 -- 172.21.15.119:0/429757229 >> v1:172.21.15.184:6789/0 conn(0x7f7ec4159420 legacy=0x7f7ec40b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.636+0000 7f7ec37fe700 1 -- 172.21.15.119:0/429757229 >> v1:172.21.15.119:6789/0 conn(0x7f7ec41597f0 legacy=0x7f7ec40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.636+0000 7f7ec37fe700 1 -- 172.21.15.119:0/429757229 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec40b14b0 con 0x7f7ec415d230 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.636+0000 7f7ec37fe700 1 --2- 172.21.15.119:0/429757229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec415d230 0x7f7ec415d630 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009e60 tx=0x7f7ebc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17cd047e4a906222 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.636+0000 7f7ec1ffb700 1 -- 172.21.15.119:0/429757229 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00e070 con 0x7f7ec415d230 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.636+0000 7f7ec1ffb700 1 -- 172.21.15.119:0/429757229 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc00bab0 con 0x7f7ec415d230 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.636+0000 7f7ec1ffb700 1 -- 172.21.15.119:0/429757229 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc00be70 con 0x7f7ec415d230 2023-10-08T02:31:07.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.637+0000 7f7ec3fff700 1 -- 172.21.15.119:0/429757229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec415d230 msgr2=0x7f7ec415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.637+0000 7f7ec3fff700 1 --2- 172.21.15.119:0/429757229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec415d230 0x7f7ec415d630 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009e60 tx=0x7f7ebc009780 comp rx=0 tx=0).stop 2023-10-08T02:31:07.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.637+0000 7f7ec3fff700 1 -- 172.21.15.119:0/429757229 shutdown_connections 2023-10-08T02:31:07.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.637+0000 7f7ec3fff700 1 --2- 172.21.15.119:0/429757229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec415d230 0x7f7ec415d630 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.637+0000 7f7ec3fff700 1 -- 172.21.15.119:0/429757229 >> 172.21.15.119:0/429757229 conn(0x7f7ec40d0660 msgr2=0x7f7ec40d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:07.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.637+0000 7f7ec3fff700 1 -- 172.21.15.119:0/429757229 shutdown_connections 2023-10-08T02:31:07.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.637+0000 7f7ec3fff700 1 -- 172.21.15.119:0/429757229 wait complete. 2023-10-08T02:31:07.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.639+0000 7f7ec3fff700 1 Processor -- start 2023-10-08T02:31:07.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.639+0000 7f7ec3fff700 1 -- start start 2023-10-08T02:31:07.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec4159420 0x7f7ec4156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ec41597f0 0x7f7ec415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec415d230 0x7f7ec40b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec41568c0 con 0x7f7ec415d230 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec3fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec416d6d0 con 0x7f7ec4159420 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec416cac0 con 0x7f7ec41597f0 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec415d230 0x7f7ec40b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec4159420 0x7f7ec4156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec27fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ec41597f0 0x7f7ec415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec415d230 0x7f7ec40b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48686/0 (socket says 172.21.15.119:48686) 2023-10-08T02:31:07.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec27fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ec41597f0 0x7f7ec415be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52766/0 (socket says 172.21.15.119:52766) 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec2ffd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec4159420 0x7f7ec4156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:44900/0 (socket says 172.21.15.119:44900) 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec37fe700 1 -- 172.21.15.119:0/2089456564 learned_addr learned my addr 172.21.15.119:0/2089456564 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec37fe700 1 -- 172.21.15.119:0/2089456564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ec41597f0 msgr2=0x7f7ec415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec37fe700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ec41597f0 0x7f7ec415be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec37fe700 1 -- 172.21.15.119:0/2089456564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec4159420 msgr2=0x7f7ec4156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.640+0000 7f7ec37fe700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec4159420 0x7f7ec4156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.641+0000 7f7ec37fe700 1 -- 172.21.15.119:0/2089456564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ebc009a20 con 0x7f7ec415d230 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.641+0000 7f7ec37fe700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec415d230 0x7f7ec40b54e0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009460 tx=0x7f7ebc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.641+0000 7f7eaaffd700 1 -- 172.21.15.119:0/2089456564 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc012dd0 con 0x7f7ec415d230 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.641+0000 7f7eaaffd700 1 -- 172.21.15.119:0/2089456564 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ebc026480 con 0x7f7ec415d230 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.641+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ec416cda0 con 0x7f7ec415d230 2023-10-08T02:31:07.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.641+0000 7f7eaaffd700 1 -- 172.21.15.119:0/2089456564 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc026840 con 0x7f7ec415d230 2023-10-08T02:31:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.641+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ec416d360 con 0x7f7ec415d230 2023-10-08T02:31:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.642+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ec404c3c0 con 0x7f7ec415d230 2023-10-08T02:31:07.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.643+0000 7f7eaaffd700 1 -- 172.21.15.119:0/2089456564 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f7ebc01c400 con 0x7f7ec415d230 2023-10-08T02:31:07.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.644+0000 7f7eaaffd700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7eb0067050 0x7f7eb0069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:07.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.644+0000 7f7eaaffd700 1 -- 172.21.15.119:0/2089456564 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f7ebc087320 con 0x7f7ec415d230 2023-10-08T02:31:07.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.644+0000 7f7ec2ffd700 1 -- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7eb0067050 msgr2=0x7f7eb0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:07.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.644+0000 7f7ec2ffd700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7eb0067050 0x7f7eb0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:07.645 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:07.645+0000 7fb629532180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:31:07.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.650+0000 7f7eaaffd700 1 -- 172.21.15.119:0/2089456564 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ebc054030 con 0x7f7ec415d230 2023-10-08T02:31:07.742 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:07.741+0000 7fb629532180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:31:07.823 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:07.822+0000 7fe440e7e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:31:07.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.845+0000 7f7ec2ffd700 1 -- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7eb0067050 msgr2=0x7f7eb0069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:07.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.845+0000 7f7ec2ffd700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7eb0067050 0x7f7eb0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:07.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.892+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ec416d810 con 0x7f7ec415d230 2023-10-08T02:31:07.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.893+0000 7f7eaaffd700 1 -- 172.21.15.119:0/2089456564 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4610 (secure 0 0 0) 0x7f7ec416d810 con 0x7f7ec415d230 2023-10-08T02:31:07.894 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:07.894 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3041824229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3041824229},{"type":"v1","addr":"172.21.15.184:6834","nonce":3041824229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1478841333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1478841333},{"type":"v1","addr":"172.21.15.119:6834","nonce":1478841333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9723,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/483355636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":483355636},{"type":"v1","addr":"172.21.15.119:6835","nonce":483355636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:28:39.274713+0000","modified":"2023-10-08T02:28:42.824801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9726":{"gid":9726,"name":"d","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.184:6835/3185561357","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":3185561357},{"type":"v1","addr":"172.21.15.184:6835","nonce":3185561357}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-10-08T02:31:07.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7eb0067050 msgr2=0x7f7eb0069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:07.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7eb0067050 0x7f7eb0069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec415d230 msgr2=0x7f7ec40b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:07.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec415d230 0x7f7ec40b54e0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc009460 tx=0x7f7ebc019000 comp rx=0 tx=0).stop 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 shutdown_connections 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7eb0067050 0x7f7eb0069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7ec415d230 0x7f7ec40b54e0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7ec4159420 0x7f7ec4156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 --2- 172.21.15.119:0/2089456564 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7ec41597f0 0x7f7ec415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 >> 172.21.15.119:0/2089456564 conn(0x7f7ec40d0660 msgr2=0x7f7ec41570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 shutdown_connections 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:07.895+0000 7f7ec3fff700 1 -- 172.21.15.119:0/2089456564 wait complete. 2023-10-08T02:31:07.897 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 87 2023-10-08T02:31:07.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:07.923 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:07.923+0000 7fe440e7e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:31:08.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.100+0000 7f6523bff700 1 Processor -- start 2023-10-08T02:31:08.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.100+0000 7f6523bff700 1 -- start start 2023-10-08T02:31:08.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.101+0000 7f6523bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f651c0caa90 0x7f651c0c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.101+0000 7f6523bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c15d7e0 con 0x7f651c06bcd0 2023-10-08T02:31:08.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.101+0000 7f6523bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f651c15dac0 con 0x7f651c06c0a0 2023-10-08T02:31:08.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.101+0000 7f6523bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651c0caef0 con 0x7f651c0caa90 2023-10-08T02:31:08.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f652199b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f651c06bcd0 0x7f651c15a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33602/0 (socket says 172.21.15.119:33602) 2023-10-08T02:31:08.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f652199b700 1 -- 172.21.15.119:0/1107495714 learned_addr learned my addr 172.21.15.119:0/1107495714 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:08.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f652219c700 1 --2- 172.21.15.119:0/1107495714 >> v2:172.21.15.184:3300/0 conn(0x7f651c0caa90 0x7f651c0c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f6520999700 1 -- 172.21.15.119:0/1107495714 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031703715 0 0) 0x7f651c15d7e0 con 0x7f651c06bcd0 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f6520999700 1 -- 172.21.15.119:0/1107495714 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6508003570 con 0x7f651c06bcd0 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f6520999700 1 -- 172.21.15.119:0/1107495714 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2129084226 0 0) 0x7f651c15dac0 con 0x7f651c06c0a0 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f6520999700 1 -- 172.21.15.119:0/1107495714 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f651c15d7e0 con 0x7f651c06c0a0 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f652219c700 1 -- 172.21.15.119:0/1107495714 >> v1:172.21.15.184:6789/0 conn(0x7f651c06c0a0 legacy=0x7f651c15df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f652219c700 1 -- 172.21.15.119:0/1107495714 >> v1:172.21.15.119:6789/0 conn(0x7f651c06bcd0 legacy=0x7f651c15a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.102+0000 7f652219c700 1 -- 172.21.15.119:0/1107495714 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c15aa30 con 0x7f651c0caa90 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f652219c700 1 --2- 172.21.15.119:0/1107495714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c0caa90 0x7f651c0c8c40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6514009e60 tx=0x7f6514009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f1fa17b8ec93e8d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f6520999700 1 -- 172.21.15.119:0/1107495714 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400e070 con 0x7f651c0caa90 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f6520999700 1 -- 172.21.15.119:0/1107495714 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651400bab0 con 0x7f651c0caa90 2023-10-08T02:31:08.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f6520999700 1 -- 172.21.15.119:0/1107495714 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651400be50 con 0x7f651c0caa90 2023-10-08T02:31:08.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f6523bff700 1 -- 172.21.15.119:0/1107495714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c0caa90 msgr2=0x7f651c0c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f6523bff700 1 --2- 172.21.15.119:0/1107495714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c0caa90 0x7f651c0c8c40 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6514009e60 tx=0x7f6514009780 comp rx=0 tx=0).stop 2023-10-08T02:31:08.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f6523bff700 1 -- 172.21.15.119:0/1107495714 shutdown_connections 2023-10-08T02:31:08.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f6523bff700 1 --2- 172.21.15.119:0/1107495714 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c0caa90 0x7f651c0c8c40 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.103+0000 7f6523bff700 1 -- 172.21.15.119:0/1107495714 >> 172.21.15.119:0/1107495714 conn(0x7f651c0ce600 msgr2=0x7f651c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:08.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.104+0000 7f6523bff700 1 -- 172.21.15.119:0/1107495714 shutdown_connections 2023-10-08T02:31:08.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.104+0000 7f6523bff700 1 -- 172.21.15.119:0/1107495714 wait complete. 2023-10-08T02:31:08.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.105+0000 7f6523bff700 1 Processor -- start 2023-10-08T02:31:08.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.105+0000 7f6523bff700 1 -- start start 2023-10-08T02:31:08.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f6523bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c06bcd0 0x7f651c0c9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f6523bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c06c0a0 0x7f651c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f6523bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f651c0caa90 0x7f651c1695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f6523bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b2390 con 0x7f651c06c0a0 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f6523bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0b18e0 con 0x7f651c06bcd0 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f6523bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651c0c58e0 con 0x7f651c0caa90 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f652119a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c06c0a0 0x7f651c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f652119a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c06c0a0 0x7f651c0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48698/0 (socket says 172.21.15.119:48698) 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f652119a700 1 -- 172.21.15.119:0/3162188022 learned_addr learned my addr 172.21.15.119:0/3162188022 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f652199b700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c06bcd0 0x7f651c0c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.106+0000 7f652219c700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f651c0caa90 0x7f651c1695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f652119a700 1 -- 172.21.15.119:0/3162188022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f651c0caa90 msgr2=0x7f651c1695f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f652119a700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f651c0caa90 0x7f651c1695f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f652119a700 1 -- 172.21.15.119:0/3162188022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c06bcd0 msgr2=0x7f651c0c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f652119a700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c06bcd0 0x7f651c0c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f652119a700 1 -- 172.21.15.119:0/3162188022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6514009a20 con 0x7f651c06c0a0 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f652119a700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c06c0a0 0x7f651c0b13a0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c580 tx=0x7f650c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f6511ffb700 1 -- 172.21.15.119:0/3162188022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c010510 con 0x7f651c06c0a0 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f6511ffb700 1 -- 172.21.15.119:0/3162188022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c010670 con 0x7f651c06c0a0 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c0b1af0 con 0x7f651c06c0a0 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f6511ffb700 1 -- 172.21.15.119:0/3162188022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c010a30 con 0x7f651c06c0a0 2023-10-08T02:31:08.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.107+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c0b20b0 con 0x7f651c06c0a0 2023-10-08T02:31:08.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.108+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c0c5690 con 0x7f651c06c0a0 2023-10-08T02:31:08.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.110+0000 7f6511ffb700 1 -- 172.21.15.119:0/3162188022 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f650c025430 con 0x7f651c06c0a0 2023-10-08T02:31:08.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.111+0000 7f6511ffb700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6500067050 0x7f6500069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.111+0000 7f652199b700 1 -- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6500067050 msgr2=0x7f6500069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:08.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.111+0000 7f6511ffb700 1 -- 172.21.15.119:0/3162188022 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f650c084ff0 con 0x7f651c06c0a0 2023-10-08T02:31:08.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.111+0000 7f652199b700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6500067050 0x7f6500069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:08.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.113+0000 7f6511ffb700 1 -- 172.21.15.119:0/3162188022 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f650c051f20 con 0x7f651c06c0a0 2023-10-08T02:31:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.312+0000 7f652199b700 1 -- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6500067050 msgr2=0x7f6500069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:08.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.312+0000 7f652199b700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6500067050 0x7f6500069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:08.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.313+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f651c04c3f0 con 0x7f651c06c0a0 2023-10-08T02:31:08.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.314+0000 7f6511ffb700 1 -- 172.21.15.119:0/3162188022 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16657 (secure 0 0 0) 0x7f650c055540 con 0x7f651c06c0a0 2023-10-08T02:31:08.315 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:08.315 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":193,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:04.457801+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:28:37.482739+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-10-08T02:28:38.257857+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"193","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":193,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/134108":"2023-10-09T02:28:59.483033+0000","172.21.15.119:6836/134108":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/1455500934":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/510634805":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/2072587307":"2023-10-09T02:28:59.483033+0000","172.21.15.119:0/3076563225":"2023-10-09T02:28:59.483033+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:08.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6500067050 msgr2=0x7f6500069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:08.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6500067050 0x7f6500069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c06c0a0 msgr2=0x7f651c0b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c06c0a0 0x7f651c0b13a0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c580 tx=0x7f650c007e10 comp rx=0 tx=0).stop 2023-10-08T02:31:08.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 shutdown_connections 2023-10-08T02:31:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6500067050 0x7f6500069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f651c0caa90 0x7f651c1695f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f651c06bcd0 0x7f651c0c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 --2- 172.21.15.119:0/3162188022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f651c06c0a0 0x7f651c0b13a0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 >> 172.21.15.119:0/3162188022 conn(0x7f651c0ce600 msgr2=0x7f651c159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.316+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 shutdown_connections 2023-10-08T02:31:08.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.317+0000 7f6523bff700 1 -- 172.21.15.119:0/3162188022 wait complete. 2023-10-08T02:31:08.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T02:31:08.445 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:08.444+0000 7fb629532180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:31:08.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.566+0000 7ff5da99d700 1 Processor -- start 2023-10-08T02:31:08.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.566+0000 7ff5da99d700 1 -- start start 2023-10-08T02:31:08.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.567+0000 7ff5da99d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5d40c6c60 0x7ff5d40c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.567+0000 7ff5da99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff5d406e8c0 0x7ff5d40c8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.567+0000 7ff5da99d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d40b1410 con 0x7ff5d41595a0 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.567+0000 7ff5da99d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d40b2b90 con 0x7ff5d40c6c60 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.567+0000 7ff5da99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d4063230 con 0x7ff5d406e8c0 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.567+0000 7ff5d919a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5d40c6c60 0x7ff5d40c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.567+0000 7ff5d919a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5d40c6c60 0x7ff5d40c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48712/0 (socket says 172.21.15.119:48712) 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.567+0000 7ff5d919a700 1 -- 172.21.15.119:0/298848726 learned_addr learned my addr 172.21.15.119:0/298848726 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.568+0000 7ff5da19c700 1 --2- 172.21.15.119:0/298848726 >> v2:172.21.15.184:3300/0 conn(0x7ff5d406e8c0 0x7ff5d40c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.568+0000 7ff5d919a700 1 -- 172.21.15.119:0/298848726 >> v2:172.21.15.184:3300/0 conn(0x7ff5d406e8c0 msgr2=0x7ff5d40c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:08.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.568+0000 7ff5d919a700 1 --2- 172.21.15.119:0/298848726 >> v2:172.21.15.184:3300/0 conn(0x7ff5d406e8c0 0x7ff5d40c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.568+0000 7ff5d919a700 1 -- 172.21.15.119:0/298848726 >> v1:172.21.15.184:6789/0 conn(0x7ff5d41595a0 legacy=0x7ff5d40b18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.568+0000 7ff5d919a700 1 -- 172.21.15.119:0/298848726 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d40634f0 con 0x7ff5d40c6c60 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.568+0000 7ff5d919a700 1 --2- 172.21.15.119:0/298848726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40c6c60 0x7ff5d40c8120 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff5d415d390 tx=0x7ff5c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8fc00c7cd410ab1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.568+0000 7ff5d8999700 1 -- 172.21.15.119:0/298848726 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c4009e70 con 0x7ff5d40c6c60 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5d8999700 1 -- 172.21.15.119:0/298848726 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c400b860 con 0x7ff5d40c6c60 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5d8999700 1 -- 172.21.15.119:0/298848726 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c400bc00 con 0x7ff5d40c6c60 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5da99d700 1 -- 172.21.15.119:0/298848726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40c6c60 msgr2=0x7ff5d40c8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5da99d700 1 --2- 172.21.15.119:0/298848726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40c6c60 0x7ff5d40c8120 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff5d415d390 tx=0x7ff5c4009250 comp rx=0 tx=0).stop 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5da99d700 1 -- 172.21.15.119:0/298848726 shutdown_connections 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5da99d700 1 --2- 172.21.15.119:0/298848726 >> v2:172.21.15.184:3300/0 conn(0x7ff5d406e8c0 0x7ff5d40c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5da99d700 1 --2- 172.21.15.119:0/298848726 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d40c6c60 0x7ff5d40c8120 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5da99d700 1 -- 172.21.15.119:0/298848726 >> 172.21.15.119:0/298848726 conn(0x7ff5d406ddb0 msgr2=0x7ff5d406e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:08.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5da99d700 1 -- 172.21.15.119:0/298848726 shutdown_connections 2023-10-08T02:31:08.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.569+0000 7ff5da99d700 1 -- 172.21.15.119:0/298848726 wait complete. 2023-10-08T02:31:08.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.571+0000 7ff5da99d700 1 Processor -- start 2023-10-08T02:31:08.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.571+0000 7ff5da99d700 1 -- start start 2023-10-08T02:31:08.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5d406e8c0 0x7ff5d40c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5d40c6c60 0x7ff5d40c35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d41595a0 0x7ff5d4152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40c3ae0 con 0x7ff5d41595a0 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40c3d60 con 0x7ff5d40c6c60 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d40c4040 con 0x7ff5d406e8c0 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d41595a0 0x7ff5d4152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d41595a0 0x7ff5d4152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48718/0 (socket says 172.21.15.119:48718) 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5da19c700 1 -- 172.21.15.119:0/206925743 learned_addr learned my addr 172.21.15.119:0/206925743 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5d919a700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5d40c6c60 0x7ff5d40c35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.572+0000 7ff5d999b700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5d406e8c0 0x7ff5d40c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5da19c700 1 -- 172.21.15.119:0/206925743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5d406e8c0 msgr2=0x7ff5d40c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5da19c700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5d406e8c0 0x7ff5d40c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5da19c700 1 -- 172.21.15.119:0/206925743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5d40c6c60 msgr2=0x7ff5d40c35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5da19c700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5d40c6c60 0x7ff5d40c35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5da19c700 1 -- 172.21.15.119:0/206925743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c40094f0 con 0x7ff5d41595a0 2023-10-08T02:31:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5da19c700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d41595a0 0x7ff5d4152010 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009e90 tx=0x7ff5cc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5c9ffb700 1 -- 172.21.15.119:0/206925743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc00cb70 con 0x7ff5d41595a0 2023-10-08T02:31:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5c9ffb700 1 -- 172.21.15.119:0/206925743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5cc00f040 con 0x7ff5d41595a0 2023-10-08T02:31:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5c9ffb700 1 -- 172.21.15.119:0/206925743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5cc014480 con 0x7ff5d41595a0 2023-10-08T02:31:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d40c4320 con 0x7ff5d41595a0 2023-10-08T02:31:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.573+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d40c1cd0 con 0x7ff5d41595a0 2023-10-08T02:31:08.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.574+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d404c3c0 con 0x7ff5d41595a0 2023-10-08T02:31:08.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.575+0000 7ff5c9ffb700 1 -- 172.21.15.119:0/206925743 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7ff5cc014830 con 0x7ff5d41595a0 2023-10-08T02:31:08.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.576+0000 7ff5c9ffb700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff5b8067050 0x7ff5b8069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:08.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.576+0000 7ff5c9ffb700 1 -- 172.21.15.119:0/206925743 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6511+0+0 (secure 0 0 0) 0x7ff5cc0855b0 con 0x7ff5d41595a0 2023-10-08T02:31:08.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.576+0000 7ff5d999b700 1 -- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff5b8067050 msgr2=0x7ff5b8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:08.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.576+0000 7ff5d999b700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff5b8067050 0x7ff5b8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:08.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.580+0000 7ff5c9ffb700 1 -- 172.21.15.119:0/206925743 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff5cc052240 con 0x7ff5d41595a0 2023-10-08T02:31:08.668 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:08.667+0000 7fe440e7e180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:31:08.734 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:08.733+0000 7fb629532180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:31:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.777+0000 7ff5d999b700 1 -- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff5b8067050 msgr2=0x7ff5b8069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:08.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.777+0000 7ff5d999b700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff5b8067050 0x7ff5b8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:08.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.823+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff5d40637d0 con 0x7ff5d41595a0 2023-10-08T02:31:08.824 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T02:31:08.824+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T02:31:08.828 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:08.827+0000 7fb629532180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:31:08.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.829+0000 7ff5c9ffb700 1 -- 172.21.15.119:0/206925743 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff5cc055860 con 0x7ff5d41595a0 2023-10-08T02:31:08.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.831+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff5b8067050 msgr2=0x7ff5b8069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.831+0000 7ff5da99d700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff5b8067050 0x7ff5b8069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.831+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d41595a0 msgr2=0x7ff5d4152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.831+0000 7ff5da99d700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d41595a0 0x7ff5d4152010 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc009e90 tx=0x7ff5cc00b040 comp rx=0 tx=0).stop 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.831+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 shutdown_connections 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.831+0000 7ff5da99d700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff5b8067050 0x7ff5b8069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.831+0000 7ff5da99d700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff5d406e8c0 0x7ff5d40c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.831+0000 7ff5da99d700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d41595a0 0x7ff5d4152010 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.832+0000 7ff5da99d700 1 --2- 172.21.15.119:0/206925743 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff5d40c6c60 0x7ff5d40c35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.832+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 >> 172.21.15.119:0/206925743 conn(0x7ff5d406ddb0 msgr2=0x7ff5d406e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:08.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.832+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 shutdown_connections 2023-10-08T02:31:08.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:08.832+0000 7ff5da99d700 1 -- 172.21.15.119:0/206925743 wait complete. 2023-10-08T02:31:08.834 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T02:31:08.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T02:31:08.889 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T02:31:08.967 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:08.966+0000 7fe440e7e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:31:08.990 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:08.990+0000 7fb629532180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:31:09.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.041+0000 7f56b659d700 1 Processor -- start 2023-10-08T02:31:09.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.041+0000 7f56b659d700 1 -- start start 2023-10-08T02:31:09.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.042+0000 7f56b659d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56b00cc600 0x7f56b00b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.042+0000 7f56b659d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b015d990 0x7f56b00cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.042+0000 7f56b659d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b0150b50 con 0x7f56b00cc9d0 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.042+0000 7f56b659d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b00cede0 con 0x7f56b00cc600 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.042+0000 7f56b659d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b00b2260 con 0x7f56b015d990 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b559b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f56b00cc600 0x7f56b00b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b5d9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b015d990 0x7f56b00cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b5d9c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b015d990 0x7f56b00cf270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:52796/0 (socket says 172.21.15.119:52796) 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b4d9a700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f56b00cc9d0 0x7f56b015d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:59238/0 (socket says 172.21.15.119:59238) 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b5d9c700 1 -- 172.21.15.119:0/3983818751 learned_addr learned my addr 172.21.15.119:0/3983818751 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56a7fff700 1 -- 172.21.15.119:0/3983818751 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78542186 0 0) 0x7f56b0150b50 con 0x7f56b00cc9d0 2023-10-08T02:31:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56a7fff700 1 -- 172.21.15.119:0/3983818751 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f569c003570 con 0x7f56b00cc9d0 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b559b700 1 -- 172.21.15.119:0/3983818751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b015d990 msgr2=0x7f56b00cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b559b700 1 --2- 172.21.15.119:0/3983818751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b015d990 0x7f56b00cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b559b700 1 -- 172.21.15.119:0/3983818751 >> v1:172.21.15.184:6789/0 conn(0x7f56b00cc9d0 legacy=0x7f56b015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b559b700 1 -- 172.21.15.119:0/3983818751 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b00b2540 con 0x7f56b00cc600 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b5d9c700 1 --2- 172.21.15.119:0/3983818751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b015d990 0x7f56b00cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.043+0000 7f56b559b700 1 --2- 172.21.15.119:0/3983818751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b00cc600 0x7f56b00b1a10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0098e0 tx=0x7f56ac0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9da7b64ae4c57ff0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.044+0000 7f56a7fff700 1 -- 172.21.15.119:0/3983818751 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac011630 con 0x7f56b00cc600 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.044+0000 7f56a7fff700 1 -- 172.21.15.119:0/3983818751 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56ac011790 con 0x7f56b00cc600 2023-10-08T02:31:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.044+0000 7f56a7fff700 1 -- 172.21.15.119:0/3983818751 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ac011b30 con 0x7f56b00cc600 2023-10-08T02:31:09.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.044+0000 7f56b659d700 1 -- 172.21.15.119:0/3983818751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b00cc600 msgr2=0x7f56b00b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:09.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.044+0000 7f56b659d700 1 --2- 172.21.15.119:0/3983818751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b00cc600 0x7f56b00b1a10 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0098e0 tx=0x7f56ac0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.045+0000 7f56b659d700 1 -- 172.21.15.119:0/3983818751 shutdown_connections 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.045+0000 7f56b659d700 1 --2- 172.21.15.119:0/3983818751 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b015d990 0x7f56b00cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.045+0000 7f56b659d700 1 --2- 172.21.15.119:0/3983818751 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b00cc600 0x7f56b00b1a10 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.045+0000 7f56b659d700 1 -- 172.21.15.119:0/3983818751 >> 172.21.15.119:0/3983818751 conn(0x7f56b00d0630 msgr2=0x7f56b00d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.045+0000 7f56b659d700 1 -- 172.21.15.119:0/3983818751 shutdown_connections 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.045+0000 7f56b659d700 1 -- 172.21.15.119:0/3983818751 wait complete. 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.046+0000 7f56b659d700 1 Processor -- start 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.046+0000 7f56b659d700 1 -- start start 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b659d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b00cc600 0x7f56b0153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b659d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b00cc9d0 0x7f56b015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:09.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b659d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b015d990 0x7f56b014d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b659d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b0155670 con 0x7f56b00cc9d0 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b659d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b014daa0 con 0x7f56b015d990 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b659d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b015a8f0 con 0x7f56b00cc600 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b4d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b00cc9d0 0x7f56b015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b4d9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b00cc9d0 0x7f56b015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48720/0 (socket says 172.21.15.119:48720) 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b4d9a700 1 -- 172.21.15.119:0/1345898003 learned_addr learned my addr 172.21.15.119:0/1345898003 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b559b700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b00cc600 0x7f56b0153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.047+0000 7f56b5d9c700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b015d990 0x7f56b014d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b4d9a700 1 -- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b00cc600 msgr2=0x7f56b0153c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b4d9a700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b00cc600 0x7f56b0153c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b4d9a700 1 -- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b015d990 msgr2=0x7f56b014d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:09.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b4d9a700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b015d990 0x7f56b014d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b4d9a700 1 -- 172.21.15.119:0/1345898003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a8009ae0 con 0x7f56b00cc9d0 2023-10-08T02:31:09.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b559b700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b00cc600 0x7f56b0153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:09.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b4d9a700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b00cc9d0 0x7f56b015a1f0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c7c0 tx=0x7f56a0007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:09.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0005c10 con 0x7f56b00cc9d0 2023-10-08T02:31:09.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56a0005d70 con 0x7f56b00cc9d0 2023-10-08T02:31:09.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac009420 con 0x7f56b00cc9d0 2023-10-08T02:31:09.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0007320 con 0x7f56b00cc9d0 2023-10-08T02:31:09.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.048+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b0155020 con 0x7f56b00cc9d0 2023-10-08T02:31:09.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.049+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b0157800 con 0x7f56b00cc9d0 2023-10-08T02:31:09.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.050+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 109) v1 ==== 84854+0+0 (secure 0 0 0) 0x7f56a001b3b0 con 0x7f56b00cc9d0 2023-10-08T02:31:09.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.051+0000 7f56a57fa700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 0x7f5694069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:09.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.051+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6569+0+0 (secure 0 0 0) 0x7f56a0084130 con 0x7f56b00cc9d0 2023-10-08T02:31:09.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.051+0000 7f56b559b700 1 -- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 msgr2=0x7f5694069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:09.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.051+0000 7f56b559b700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 0x7f5694069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:09.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.055+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f56b0157800 con 0x7f56b00cc9d0 2023-10-08T02:31:09.068 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:09.068+0000 7fe440e7e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:31:09.099 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:09.098+0000 7fb629532180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:31:09.228 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:09.228+0000 7fe440e7e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:31:09.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.252+0000 7f56b559b700 1 -- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 msgr2=0x7f5694069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:09.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.252+0000 7f56b559b700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 0x7f5694069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:09.292 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:09.291+0000 7fb629532180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:31:09.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.303+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f56b0157800 con 0x7f56b00cc9d0 2023-10-08T02:31:09.337 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:09.337+0000 7fe440e7e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:31:09.385 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:09.384+0000 7fb629532180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:31:09.478 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:09.477+0000 7fb629532180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:31:09.532 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:09.531+0000 7fe440e7e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:31:09.574 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:09.573+0000 7fb629532180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:31:09.625 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:09.625+0000 7fe440e7e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:31:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.653+0000 7f56b559b700 1 -- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 msgr2=0x7f5694069500 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:09.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.653+0000 7f56b559b700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 0x7f5694069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:31:09.720 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:09.718+0000 7fe440e7e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:31:09.814 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:09.813+0000 7fe440e7e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:31:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.825+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 110) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f56a0051260 con 0x7f56b00cc9d0 2023-10-08T02:31:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.825+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 msgr2=0x7f5694069500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.825+0000 7f56a57fa700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 0x7f5694069500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.830+0000 7f56a57fa700 1 -- 172.21.15.119:0/1345898003 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7f56a006e2f0 con 0x7f56b00cc9d0 2023-10-08T02:31:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b00cc9d0 msgr2=0x7f56b015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b00cc9d0 0x7f56b015a1f0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c7c0 tx=0x7f56a0007e00 comp rx=0 tx=0).stop 2023-10-08T02:31:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 shutdown_connections 2023-10-08T02:31:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5694067050 0x7f5694069500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f56b00cc600 0x7f56b0153c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f56b015d990 0x7f56b014d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 --2- 172.21.15.119:0/1345898003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56b00cc9d0 0x7f56b015a1f0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 >> 172.21.15.119:0/1345898003 conn(0x7f56b00d0630 msgr2=0x7f56b014ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:09.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 shutdown_connections 2023-10-08T02:31:09.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:09.833+0000 7f56b659d700 1 -- 172.21.15.119:0/1345898003 wait complete. 2023-10-08T02:31:09.853 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T02:31:10.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.045+0000 7f13429ff700 1 Processor -- start 2023-10-08T02:31:10.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.045+0000 7f13429ff700 1 -- start start 2023-10-08T02:31:10.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f13429ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f133c0c8490 0x7f133c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:10.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f13429ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c0ac030 con 0x7f133c069e20 2023-10-08T02:31:10.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f13429ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c0ab820 con 0x7f133c069a50 2023-10-08T02:31:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f13429ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c06dbb0 con 0x7f133c0c8490 2023-10-08T02:31:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f1340f9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f133c069e20 0x7f133c1569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33604/0 (socket says 172.21.15.119:33604) 2023-10-08T02:31:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f1340f9c700 1 -- 172.21.15.119:0/1491579669 learned_addr learned my addr 172.21.15.119:0/1491579669 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:10.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f1333fff700 1 --2- 172.21.15.119:0/1491579669 >> v2:172.21.15.184:3300/0 conn(0x7f133c0c8490 0x7f133c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f133b7fe700 1 -- 172.21.15.119:0/1491579669 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313204050 0 0) 0x7f1334001490 con 0x7f133c069e20 2023-10-08T02:31:10.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f133b7fe700 1 -- 172.21.15.119:0/1491579669 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1328003570 con 0x7f133c069e20 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f133b7fe700 1 -- 172.21.15.119:0/1491579669 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1284712419 0 0) 0x7f133c0ab820 con 0x7f133c069a50 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f133b7fe700 1 -- 172.21.15.119:0/1491579669 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1334001490 con 0x7f133c069a50 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f1333fff700 1 -- 172.21.15.119:0/1491579669 >> v1:172.21.15.184:6789/0 conn(0x7f133c069a50 legacy=0x7f133c15a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.047+0000 7f1333fff700 1 -- 172.21.15.119:0/1491579669 >> v1:172.21.15.119:6789/0 conn(0x7f133c069e20 legacy=0x7f133c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.048+0000 7f1333fff700 1 -- 172.21.15.119:0/1491579669 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0aba70 con 0x7f133c0c8490 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.048+0000 7f1333fff700 1 --2- 172.21.15.119:0/1491579669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c0c8490 0x7f133c0c6ac0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f13240098e0 tx=0x7f13240091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8a229f25a5a022d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.048+0000 7f133b7fe700 1 -- 172.21.15.119:0/1491579669 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324011630 con 0x7f133c0c8490 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.048+0000 7f133b7fe700 1 -- 172.21.15.119:0/1491579669 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1324011790 con 0x7f133c0c8490 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.048+0000 7f133b7fe700 1 -- 172.21.15.119:0/1491579669 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324011b30 con 0x7f133c0c8490 2023-10-08T02:31:10.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.049+0000 7f13429ff700 1 -- 172.21.15.119:0/1491579669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c0c8490 msgr2=0x7f133c0c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.049+0000 7f13429ff700 1 --2- 172.21.15.119:0/1491579669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c0c8490 0x7f133c0c6ac0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f13240098e0 tx=0x7f13240091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.049+0000 7f13429ff700 1 -- 172.21.15.119:0/1491579669 shutdown_connections 2023-10-08T02:31:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.049+0000 7f13429ff700 1 --2- 172.21.15.119:0/1491579669 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c0c8490 0x7f133c0c6ac0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.049+0000 7f13429ff700 1 -- 172.21.15.119:0/1491579669 >> 172.21.15.119:0/1491579669 conn(0x7f133c0c8f90 msgr2=0x7f133c0c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.049+0000 7f13429ff700 1 -- 172.21.15.119:0/1491579669 shutdown_connections 2023-10-08T02:31:10.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.049+0000 7f13429ff700 1 -- 172.21.15.119:0/1491579669 wait complete. 2023-10-08T02:31:10.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.050+0000 7f13429ff700 1 Processor -- start 2023-10-08T02:31:10.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.050+0000 7f13429ff700 1 -- start start 2023-10-08T02:31:10.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.051+0000 7f13429ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c069a50 0x7f133c0c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:10.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.051+0000 7f13429ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c069e20 0x7f133c154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:10.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.051+0000 7f13429ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f133c0c8490 0x7f133c156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.051+0000 7f13429ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0c6780 con 0x7f133c0c8490 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.051+0000 7f13429ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c154b50 con 0x7f133c069a50 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.051+0000 7f13429ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0c5b70 con 0x7f133c069e20 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f133c0c8490 0x7f133c156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1333fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c069e20 0x7f133c154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f133bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c069a50 0x7f133c0c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f133c0c8490 0x7f133c156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48784/0 (socket says 172.21.15.119:48784) 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 -- 172.21.15.119:0/4088630889 learned_addr learned my addr 172.21.15.119:0/4088630889 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 -- 172.21.15.119:0/4088630889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c069e20 msgr2=0x7f133c154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 --2- 172.21.15.119:0/4088630889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c069e20 0x7f133c154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:10.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 -- 172.21.15.119:0/4088630889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c069a50 msgr2=0x7f133c0c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:10.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 --2- 172.21.15.119:0/4088630889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c069a50 0x7f133c0c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:10.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 -- 172.21.15.119:0/4088630889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324009420 con 0x7f133c0c8490 2023-10-08T02:31:10.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.052+0000 7f1340f9c700 1 --2- 172.21.15.119:0/4088630889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f133c0c8490 0x7f133c156580 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1334002250 tx=0x7f1334001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:10.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.053+0000 7f1338ff9700 1 -- 172.21.15.119:0/4088630889 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133400aac0 con 0x7f133c0c8490 2023-10-08T02:31:10.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.053+0000 7f1338ff9700 1 -- 172.21.15.119:0/4088630889 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133400ac20 con 0x7f133c0c8490 2023-10-08T02:31:10.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.053+0000 7f1338ff9700 1 -- 172.21.15.119:0/4088630889 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13340070f0 con 0x7f133c0c8490 2023-10-08T02:31:10.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.053+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c0c5e50 con 0x7f133c0c8490 2023-10-08T02:31:10.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.053+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c0c6410 con 0x7f133c0c8490 2023-10-08T02:31:10.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.054+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c1571c0 con 0x7f133c0c8490 2023-10-08T02:31:10.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.055+0000 7f1338ff9700 1 -- 172.21.15.119:0/4088630889 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 110) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f13340074e0 con 0x7f133c0c8490 2023-10-08T02:31:10.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.056+0000 7f1338ff9700 1 -- 172.21.15.119:0/4088630889 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6827+0+0 (secure 0 0 0) 0x7f1334084740 con 0x7f133c0c8490 2023-10-08T02:31:10.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.060+0000 7f1338ff9700 1 -- 172.21.15.119:0/4088630889 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1334051990 con 0x7f133c0c8490 2023-10-08T02:31:10.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.249+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f133c0ac030 con 0x7f133c0c8490 2023-10-08T02:31:10.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.833+0000 7f1338ff9700 1 -- 172.21.15.119:0/4088630889 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v196) v1 ==== 181+0+0 (secure 0 0 0) 0x7f133c0ac030 con 0x7f133c0c8490 2023-10-08T02:31:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.837+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f133c0c8490 msgr2=0x7f133c156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.837+0000 7f13429ff700 1 --2- 172.21.15.119:0/4088630889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f133c0c8490 0x7f133c156580 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f1334002250 tx=0x7f1334001bb0 comp rx=0 tx=0).stop 2023-10-08T02:31:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.837+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 shutdown_connections 2023-10-08T02:31:10.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.837+0000 7f13429ff700 1 --2- 172.21.15.119:0/4088630889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f133c0c8490 0x7f133c156580 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.837+0000 7f13429ff700 1 --2- 172.21.15.119:0/4088630889 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f133c069a50 0x7f133c0c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.837+0000 7f13429ff700 1 --2- 172.21.15.119:0/4088630889 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f133c069e20 0x7f133c154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.837+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 >> 172.21.15.119:0/4088630889 conn(0x7f133c0c8f90 msgr2=0x7f133c0cc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.838+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 shutdown_connections 2023-10-08T02:31:10.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:10.838+0000 7f13429ff700 1 -- 172.21.15.119:0/4088630889 wait complete. 2023-10-08T02:31:10.840 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T02:31:10.866 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T02:31:11.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.051+0000 7fcc1bdff700 1 Processor -- start 2023-10-08T02:31:11.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.051+0000 7fcc1bdff700 1 -- start start 2023-10-08T02:31:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.052+0000 7fcc1bdff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc14156540 0x7fcc1406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.052+0000 7fcc1bdff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc140693a0 con 0x7fcc14068390 2023-10-08T02:31:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.052+0000 7fcc1bdff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc140afdf0 con 0x7fcc140b1290 2023-10-08T02:31:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.052+0000 7fcc1bdff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc140b2970 con 0x7fcc14156540 2023-10-08T02:31:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc19b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc14156540 0x7fcc1406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:11.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc1939a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcc14068390 0x7fcc14156910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33618/0 (socket says 172.21.15.119:33618) 2023-10-08T02:31:11.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc1939a700 1 -- 172.21.15.119:0/2134142202 learned_addr learned my addr 172.21.15.119:0/2134142202 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:11.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc19b9b700 1 -- 172.21.15.119:0/2134142202 >> v1:172.21.15.184:6789/0 conn(0x7fcc140b1290 legacy=0x7fcc1415fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:11.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc19b9b700 1 -- 172.21.15.119:0/2134142202 >> v1:172.21.15.119:6789/0 conn(0x7fcc14068390 legacy=0x7fcc14156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:11.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc19b9b700 1 -- 172.21.15.119:0/2134142202 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc14068830 con 0x7fcc14156540 2023-10-08T02:31:11.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc18b99700 1 -- 172.21.15.119:0/2134142202 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547436973 0 0) 0x7fcc140693a0 con 0x7fcc14068390 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc19b9b700 1 --2- 172.21.15.119:0/2134142202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14156540 0x7fcc1406ad70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098e0 tx=0x7fcc100091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cdad03a0fcfe8ba server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.053+0000 7fcc18b99700 1 -- 172.21.15.119:0/2134142202 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10009e80 con 0x7fcc14156540 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc18b99700 1 -- 172.21.15.119:0/2134142202 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1000b820 con 0x7fcc14156540 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc18b99700 1 -- 172.21.15.119:0/2134142202 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000bba0 con 0x7fcc14156540 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc1bdff700 1 -- 172.21.15.119:0/2134142202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14156540 msgr2=0x7fcc1406ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc1bdff700 1 --2- 172.21.15.119:0/2134142202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14156540 0x7fcc1406ad70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098e0 tx=0x7fcc100091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc1bdff700 1 -- 172.21.15.119:0/2134142202 shutdown_connections 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc1bdff700 1 --2- 172.21.15.119:0/2134142202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14156540 0x7fcc1406ad70 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc1bdff700 1 -- 172.21.15.119:0/2134142202 >> 172.21.15.119:0/2134142202 conn(0x7fcc140cd880 msgr2=0x7fcc140ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc1bdff700 1 -- 172.21.15.119:0/2134142202 shutdown_connections 2023-10-08T02:31:11.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.054+0000 7fcc1bdff700 1 -- 172.21.15.119:0/2134142202 wait complete. 2023-10-08T02:31:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.056+0000 7fcc1bdff700 1 Processor -- start 2023-10-08T02:31:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.056+0000 7fcc1bdff700 1 -- start start 2023-10-08T02:31:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc1bdff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14068390 0x7fcc140ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc1bdff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc140b1290 0x7fcc140cbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc1bdff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc14156540 0x7fcc1415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc1bdff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1414cfb0 con 0x7fcc14068390 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc1bdff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140caa10 con 0x7fcc14156540 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc1bdff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1414c6c0 con 0x7fcc140b1290 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc19b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14068390 0x7fcc140ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc19b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14068390 0x7fcc140ca4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48804/0 (socket says 172.21.15.119:48804) 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc19b9b700 1 -- 172.21.15.119:0/1901407196 learned_addr learned my addr 172.21.15.119:0/1901407196 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc1a39c700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc14156540 0x7fcc1415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc1939a700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc140b1290 0x7fcc140cbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc19b9b700 1 -- 172.21.15.119:0/1901407196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc140b1290 msgr2=0x7fcc140cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc19b9b700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc140b1290 0x7fcc140cbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc19b9b700 1 -- 172.21.15.119:0/1901407196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc14156540 msgr2=0x7fcc1415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc19b9b700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc14156540 0x7fcc1415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:11.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.057+0000 7fcc19b9b700 1 -- 172.21.15.119:0/1901407196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10009420 con 0x7fcc14068390 2023-10-08T02:31:11.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.058+0000 7fcc19b9b700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14068390 0x7fcc140ca4d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fcc1406d940 tx=0x7fcc10016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:11.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.058+0000 7fcc09ffb700 1 -- 172.21.15.119:0/1901407196 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000b990 con 0x7fcc14068390 2023-10-08T02:31:11.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.058+0000 7fcc09ffb700 1 -- 172.21.15.119:0/1901407196 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10020070 con 0x7fcc14068390 2023-10-08T02:31:11.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.058+0000 7fcc09ffb700 1 -- 172.21.15.119:0/1901407196 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1001b6a0 con 0x7fcc14068390 2023-10-08T02:31:11.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.058+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1414c960 con 0x7fcc14068390 2023-10-08T02:31:11.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.058+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc141667c0 con 0x7fcc14068390 2023-10-08T02:31:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.059+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc140718b0 con 0x7fcc14068390 2023-10-08T02:31:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.060+0000 7fcc09ffb700 1 -- 172.21.15.119:0/1901407196 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 111) v1 ==== 84578+0+0 (secure 0 0 0) 0x7fcc10022030 con 0x7fcc14068390 2023-10-08T02:31:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.061+0000 7fcc09ffb700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbf8066d80 0x7fcbf8069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.061+0000 7fcc09ffb700 1 -- 172.21.15.119:0/1901407196 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fcc100864a0 con 0x7fcc14068390 2023-10-08T02:31:11.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.061+0000 7fcc1939a700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbf8066d80 0x7fcbf8069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:11.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.061+0000 7fcc1939a700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbf8066d80 0x7fcbf8069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc140c9cd0 tx=0x7fcc04009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:11.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.065+0000 7fcc09ffb700 1 -- 172.21.15.119:0/1901407196 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc10053640 con 0x7fcc14068390 2023-10-08T02:31:11.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.254+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcc140718b0 con 0x7fcc14068390 2023-10-08T02:31:11.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.849+0000 7fcc09ffb700 1 -- 172.21.15.119:0/1901407196 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc10053a10 con 0x7fcc14068390 2023-10-08T02:31:11.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.850+0000 7fcc09ffb700 1 -- 172.21.15.119:0/1901407196 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v197) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcc100570d0 con 0x7fcc14068390 2023-10-08T02:31:11.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.854+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbf8066d80 msgr2=0x7fcbf8069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.854+0000 7fcc1bdff700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbf8066d80 0x7fcbf8069230 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcc140c9cd0 tx=0x7fcc04009040 comp rx=0 tx=0).stop 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.854+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14068390 msgr2=0x7fcc140ca4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.854+0000 7fcc1bdff700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14068390 0x7fcc140ca4d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fcc1406d940 tx=0x7fcc10016000 comp rx=0 tx=0).stop 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.854+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 shutdown_connections 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.854+0000 7fcc1bdff700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbf8066d80 0x7fcbf8069230 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.854+0000 7fcc1bdff700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcc14156540 0x7fcc1415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.855+0000 7fcc1bdff700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc14068390 0x7fcc140ca4d0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.855+0000 7fcc1bdff700 1 --2- 172.21.15.119:0/1901407196 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcc140b1290 0x7fcc140cbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.855+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 >> 172.21.15.119:0/1901407196 conn(0x7fcc140cd880 msgr2=0x7fcc1414e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:11.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.855+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 shutdown_connections 2023-10-08T02:31:11.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:11.855+0000 7fcc1bdff700 1 -- 172.21.15.119:0/1901407196 wait complete. 2023-10-08T02:31:11.857 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T02:31:11.874 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T02:31:12.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.055+0000 7f6301dff700 1 Processor -- start 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.055+0000 7f6301dff700 1 -- start start 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f6301dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f62fc15ac80 0x7f62fc069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f6301dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f62fc069cb0 0x7f62fc0c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f6301dff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62fc0af2f0 con 0x7f62fc15b050 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f6301dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62fc0af5d0 con 0x7f62fc15ac80 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f6301dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62fc06a0e0 con 0x7f62fc069cb0 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62fb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f62fc15ac80 0x7f62fc069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62f3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f62fc15b050 0x7f62fc0afa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33626/0 (socket says 172.21.15.119:33626) 2023-10-08T02:31:12.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62fb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f62fc15ac80 0x7f62fc069770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48810/0 (socket says 172.21.15.119:48810) 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62f3fff700 1 -- 172.21.15.119:0/1670106574 learned_addr learned my addr 172.21.15.119:0/1670106574 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62fb7fe700 1 -- 172.21.15.119:0/1670106574 >> v2:172.21.15.184:3300/0 conn(0x7f62fc069cb0 msgr2=0x7f62fc0c7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62fbfff700 1 --2- 172.21.15.119:0/1670106574 >> v2:172.21.15.184:3300/0 conn(0x7f62fc069cb0 0x7f62fc0c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62faffd700 1 -- 172.21.15.119:0/1670106574 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 906446642 0 0) 0x7f62fc0af2f0 con 0x7f62fc15b050 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62fb7fe700 1 --2- 172.21.15.119:0/1670106574 >> v2:172.21.15.184:3300/0 conn(0x7f62fc069cb0 0x7f62fc0c7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62fb7fe700 1 -- 172.21.15.119:0/1670106574 >> v1:172.21.15.119:6789/0 conn(0x7f62fc15b050 legacy=0x7f62fc0afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.057+0000 7f62fb7fe700 1 -- 172.21.15.119:0/1670106574 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62fc0c8660 con 0x7f62fc15ac80 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.058+0000 7f62fbfff700 1 --2- 172.21.15.119:0/1670106574 >> v2:172.21.15.184:3300/0 conn(0x7f62fc069cb0 0x7f62fc0c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.058+0000 7f62fb7fe700 1 --2- 172.21.15.119:0/1670106574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15ac80 0x7f62fc069770 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f62fc06d240 tx=0x7f62ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406f8335b06196a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.058+0000 7f62faffd700 1 -- 172.21.15.119:0/1670106574 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00b6b0 con 0x7f62fc15ac80 2023-10-08T02:31:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.058+0000 7f62faffd700 1 -- 172.21.15.119:0/1670106574 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ec00b810 con 0x7f62fc15ac80 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.058+0000 7f62faffd700 1 -- 172.21.15.119:0/1670106574 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec00bbb0 con 0x7f62fc15ac80 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.058+0000 7f6301dff700 1 -- 172.21.15.119:0/1670106574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15ac80 msgr2=0x7f62fc069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.058+0000 7f6301dff700 1 --2- 172.21.15.119:0/1670106574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15ac80 0x7f62fc069770 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f62fc06d240 tx=0x7f62ec0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.059+0000 7f6301dff700 1 -- 172.21.15.119:0/1670106574 shutdown_connections 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.059+0000 7f6301dff700 1 --2- 172.21.15.119:0/1670106574 >> v2:172.21.15.184:3300/0 conn(0x7f62fc069cb0 0x7f62fc0c7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.059+0000 7f6301dff700 1 --2- 172.21.15.119:0/1670106574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15ac80 0x7f62fc069770 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.059+0000 7f6301dff700 1 -- 172.21.15.119:0/1670106574 >> 172.21.15.119:0/1670106574 conn(0x7f62fc068e10 msgr2=0x7f62fc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.059+0000 7f6301dff700 1 -- 172.21.15.119:0/1670106574 shutdown_connections 2023-10-08T02:31:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.059+0000 7f6301dff700 1 -- 172.21.15.119:0/1670106574 wait complete. 2023-10-08T02:31:12.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.060+0000 7f6301dff700 1 Processor -- start 2023-10-08T02:31:12.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.060+0000 7f6301dff700 1 -- start start 2023-10-08T02:31:12.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f6301dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62fc069cb0 0x7f62fc0cb970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f6301dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62fc15ac80 0x7f62fc0ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f6301dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15b050 0x7f62fc0ac1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f6301dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc14d8c0 con 0x7f62fc15b050 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f6301dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc16d560 con 0x7f62fc069cb0 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f6301dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62fc16d840 con 0x7f62fc15ac80 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f62fbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15b050 0x7f62fc0ac1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f62fbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15b050 0x7f62fc0ac1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48826/0 (socket says 172.21.15.119:48826) 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.061+0000 7f62fbfff700 1 -- 172.21.15.119:0/2987707859 learned_addr learned my addr 172.21.15.119:0/2987707859 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62fb7fe700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62fc069cb0 0x7f62fc0cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62f3fff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62fc15ac80 0x7f62fc0ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62fbfff700 1 -- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62fc15ac80 msgr2=0x7f62fc0ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62fbfff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62fc15ac80 0x7f62fc0ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62fbfff700 1 -- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62fc069cb0 msgr2=0x7f62fc0cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62fbfff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62fc069cb0 0x7f62fc0cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62fbfff700 1 -- 172.21.15.119:0/2987707859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec009420 con 0x7f62fc15b050 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62fb7fe700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62fc069cb0 0x7f62fc0cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62fbfff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15b050 0x7f62fc0ac1b0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f62f400a380 tx=0x7f62f4007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62f37fe700 1 -- 172.21.15.119:0/2987707859 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f400ae10 con 0x7f62fc15b050 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62f37fe700 1 -- 172.21.15.119:0/2987707859 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f400d650 con 0x7f62fc15b050 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f62f37fe700 1 -- 172.21.15.119:0/2987707859 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f400da10 con 0x7f62fc15b050 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62fc16db20 con 0x7f62fc15b050 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.062+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62fc0cbeb0 con 0x7f62fc15b050 2023-10-08T02:31:12.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.063+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62fc14da70 con 0x7f62fc15b050 2023-10-08T02:31:12.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.064+0000 7f62f37fe700 1 -- 172.21.15.119:0/2987707859 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f62f401e3b0 con 0x7f62fc15b050 2023-10-08T02:31:12.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.065+0000 7f62f37fe700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f62dc066f30 0x7f62dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.065+0000 7f62f37fe700 1 -- 172.21.15.119:0/2987707859 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f62f4085130 con 0x7f62fc15b050 2023-10-08T02:31:12.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.065+0000 7f62fb7fe700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f62dc066f30 0x7f62dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.066+0000 7f62fb7fe700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f62dc066f30 0x7f62dc0693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0093f0 tx=0x7f62ec017000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.069+0000 7f62f37fe700 1 -- 172.21.15.119:0/2987707859 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f62fc14da70 con 0x7f62fc15b050 2023-10-08T02:31:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.261+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f62fc14da70 con 0x7f62fc15b050 2023-10-08T02:31:12.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.265+0000 7f62f37fe700 1 -- 172.21.15.119:0/2987707859 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f62fc14da70 con 0x7f62fc15b050 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f62dc066f30 msgr2=0x7f62dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f62dc066f30 0x7f62dc0693e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0093f0 tx=0x7f62ec017000 comp rx=0 tx=0).stop 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15b050 msgr2=0x7f62fc0ac1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15b050 0x7f62fc0ac1b0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f62f400a380 tx=0x7f62f4007900 comp rx=0 tx=0).stop 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 shutdown_connections 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f62dc066f30 0x7f62dc0693e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62fc15b050 0x7f62fc0ac1b0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f62fc069cb0 0x7f62fc0cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 --2- 172.21.15.119:0/2987707859 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f62fc15ac80 0x7f62fc0ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 >> 172.21.15.119:0/2987707859 conn(0x7f62fc068e10 msgr2=0x7f62fc0b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:12.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 shutdown_connections 2023-10-08T02:31:12.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.268+0000 7f6301dff700 1 -- 172.21.15.119:0/2987707859 wait complete. 2023-10-08T02:31:12.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T02:31:12.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.473+0000 7f3a6e1ff700 1 Processor -- start 2023-10-08T02:31:12.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.473+0000 7f3a6e1ff700 1 -- start start 2023-10-08T02:31:12.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.474+0000 7f3a6e1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a680c70b0 0x7f3a68069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a6e1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 0x7f3a680ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a6e1ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a6806e950 con 0x7f3a68155eb0 2023-10-08T02:31:12.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a6e1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a6806a130 con 0x7f3a680c70b0 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a6e1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6806eb90 con 0x7f3a68069cb0 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a677fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3a68155eb0 0x7f3a6806e240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33636/0 (socket says 172.21.15.119:33636) 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a677fe700 1 -- 172.21.15.119:0/564539383 learned_addr learned my addr 172.21.15.119:0/564539383 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a5ffff700 1 --2- 172.21.15.119:0/564539383 >> v2:172.21.15.119:3300/0 conn(0x7f3a680c70b0 0x7f3a68069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a67fff700 1 --2- 172.21.15.119:0/564539383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 0x7f3a680ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a66ffd700 1 -- 172.21.15.119:0/564539383 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007692836 0 0) 0x7f3a58001110 con 0x7f3a68155eb0 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a66ffd700 1 -- 172.21.15.119:0/564539383 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a54003570 con 0x7f3a68155eb0 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a5ffff700 1 -- 172.21.15.119:0/564539383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 msgr2=0x7f3a680ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a5ffff700 1 --2- 172.21.15.119:0/564539383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 0x7f3a680ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a5ffff700 1 -- 172.21.15.119:0/564539383 >> v1:172.21.15.119:6789/0 conn(0x7f3a68155eb0 legacy=0x7f3a6806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.475+0000 7f3a5ffff700 1 -- 172.21.15.119:0/564539383 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6806ee70 con 0x7f3a680c70b0 2023-10-08T02:31:12.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a5ffff700 1 --2- 172.21.15.119:0/564539383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a680c70b0 0x7f3a68069770 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000aa90 tx=0x7f3a5000a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27603e9ea1dde101 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a66ffd700 1 -- 172.21.15.119:0/564539383 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000c970 con 0x7f3a680c70b0 2023-10-08T02:31:12.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a66ffd700 1 -- 172.21.15.119:0/564539383 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5000cad0 con 0x7f3a680c70b0 2023-10-08T02:31:12.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a66ffd700 1 -- 172.21.15.119:0/564539383 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a5000ce70 con 0x7f3a680c70b0 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/564539383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a680c70b0 msgr2=0x7f3a68069770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/564539383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a680c70b0 0x7f3a68069770 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000aa90 tx=0x7f3a5000a330 comp rx=0 tx=0).stop 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/564539383 shutdown_connections 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/564539383 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 0x7f3a680ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/564539383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a680c70b0 0x7f3a68069770 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.476+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/564539383 >> 172.21.15.119:0/564539383 conn(0x7f3a68068e10 msgr2=0x7f3a680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.477+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/564539383 shutdown_connections 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.477+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/564539383 wait complete. 2023-10-08T02:31:12.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.478+0000 7f3a6e1ff700 1 Processor -- start 2023-10-08T02:31:12.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.478+0000 7f3a6e1ff700 1 -- start start 2023-10-08T02:31:12.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a6e1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 0x7f3a68155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a6e1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680c70b0 0x7f3a68156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a6e1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a68155eb0 0x7f3a680c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a6e1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680aca10 con 0x7f3a68155eb0 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a6e1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a681556a0 con 0x7f3a680c70b0 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a6e1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a680abb20 con 0x7f3a68069cb0 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a68155eb0 0x7f3a680c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a67fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a68155eb0 0x7f3a680c55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48830/0 (socket says 172.21.15.119:48830) 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a67fff700 1 -- 172.21.15.119:0/3972562001 learned_addr learned my addr 172.21.15.119:0/3972562001 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a677fe700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 0x7f3a68155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.479+0000 7f3a5ffff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680c70b0 0x7f3a68156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a67fff700 1 -- 172.21.15.119:0/3972562001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 msgr2=0x7f3a68155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a67fff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 0x7f3a68155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a67fff700 1 -- 172.21.15.119:0/3972562001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680c70b0 msgr2=0x7f3a68156c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a67fff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680c70b0 0x7f3a68156c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a67fff700 1 -- 172.21.15.119:0/3972562001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5000a5d0 con 0x7f3a68155eb0 2023-10-08T02:31:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a67fff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a68155eb0 0x7f3a680c55b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009fd0 tx=0x7f3a60009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a5f7fe700 1 -- 172.21.15.119:0/3972562001 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6000ebf0 con 0x7f3a68155eb0 2023-10-08T02:31:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a5f7fe700 1 -- 172.21.15.119:0/3972562001 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6000ed50 con 0x7f3a68155eb0 2023-10-08T02:31:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a680abe00 con 0x7f3a68155eb0 2023-10-08T02:31:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a5f7fe700 1 -- 172.21.15.119:0/3972562001 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a60014670 con 0x7f3a68155eb0 2023-10-08T02:31:12.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.480+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a680ac3c0 con 0x7f3a68155eb0 2023-10-08T02:31:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.481+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a6804c3c0 con 0x7f3a68155eb0 2023-10-08T02:31:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.482+0000 7f3a5f7fe700 1 -- 172.21.15.119:0/3972562001 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a600149e0 con 0x7f3a68155eb0 2023-10-08T02:31:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.483+0000 7f3a5f7fe700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a48066f30 0x7f3a480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.483+0000 7f3a5f7fe700 1 -- 172.21.15.119:0/3972562001 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f3a60084ce0 con 0x7f3a68155eb0 2023-10-08T02:31:12.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.483+0000 7f3a677fe700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a48066f30 0x7f3a480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.484+0000 7f3a677fe700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a48066f30 0x7f3a480693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3a58007fd0 tx=0x7f3a58007bf0 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.487+0000 7f3a5f7fe700 1 -- 172.21.15.119:0/3972562001 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a60051e40 con 0x7f3a68155eb0 2023-10-08T02:31:12.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.685+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f3a680acb50 con 0x7f3a68155eb0 2023-10-08T02:31:12.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.689+0000 7f3a5f7fe700 1 -- 172.21.15.119:0/3972562001 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v68) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3a60055460 con 0x7f3a68155eb0 2023-10-08T02:31:12.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a48066f30 msgr2=0x7f3a480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a48066f30 0x7f3a480693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3a58007fd0 tx=0x7f3a58007bf0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a68155eb0 msgr2=0x7f3a680c55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a68155eb0 0x7f3a680c55b0 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f3a60009fd0 tx=0x7f3a60009900 comp rx=0 tx=0).stop 2023-10-08T02:31:12.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 shutdown_connections 2023-10-08T02:31:12.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3a48066f30 0x7f3a480693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3a68069cb0 0x7f3a68155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a68155eb0 0x7f3a680c55b0 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 --2- 172.21.15.119:0/3972562001 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3a680c70b0 0x7f3a68156c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 >> 172.21.15.119:0/3972562001 conn(0x7f3a68068e10 msgr2=0x7f3a680b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 shutdown_connections 2023-10-08T02:31:12.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.692+0000 7f3a6e1ff700 1 -- 172.21.15.119:0/3972562001 wait complete. 2023-10-08T02:31:12.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T02:31:12.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.893+0000 7fbb5719d700 1 Processor -- start 2023-10-08T02:31:12.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.893+0000 7fbb5719d700 1 -- start start 2023-10-08T02:31:12.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5719d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbb50140bb0 0x7fbb5006d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5719d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbb5006da30 0x7fbb500c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5719d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb500c6820 con 0x7fbb501407e0 2023-10-08T02:31:12.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5719d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5006deb0 con 0x7fbb5006da30 2023-10-08T02:31:12.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5719d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb500c6de0 con 0x7fbb50140bb0 2023-10-08T02:31:12.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5699c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbb5006da30 0x7fbb500c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5699c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbb5006da30 0x7fbb500c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48832/0 (socket says 172.21.15.119:48832) 2023-10-08T02:31:12.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5699c700 1 -- 172.21.15.119:0/4066104714 learned_addr learned my addr 172.21.15.119:0/4066104714 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:12.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5619b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbb501407e0 0x7fbb5006a180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:33640/0 (socket says 172.21.15.119:33640) 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5599a700 1 --2- 172.21.15.119:0/4066104714 >> v2:172.21.15.184:3300/0 conn(0x7fbb50140bb0 0x7fbb5006d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5699c700 1 -- 172.21.15.119:0/4066104714 >> v2:172.21.15.184:3300/0 conn(0x7fbb50140bb0 msgr2=0x7fbb5006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5699c700 1 --2- 172.21.15.119:0/4066104714 >> v2:172.21.15.184:3300/0 conn(0x7fbb50140bb0 0x7fbb5006d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5699c700 1 -- 172.21.15.119:0/4066104714 >> v1:172.21.15.119:6789/0 conn(0x7fbb501407e0 legacy=0x7fbb5006a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5699c700 1 -- 172.21.15.119:0/4066104714 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb50141270 con 0x7fbb5006da30 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb55199700 1 -- 172.21.15.119:0/4066104714 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958595779 0 0) 0x7fbb500c6820 con 0x7fbb501407e0 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.895+0000 7fbb5599a700 1 --2- 172.21.15.119:0/4066104714 >> v2:172.21.15.184:3300/0 conn(0x7fbb50140bb0 0x7fbb5006d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.896+0000 7fbb5699c700 1 --2- 172.21.15.119:0/4066104714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb5006da30 0x7fbb500c7270 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fbb500c3240 tx=0x7fbb48009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec2e6a1205cb6fc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.896+0000 7fbb55199700 1 -- 172.21.15.119:0/4066104714 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800e070 con 0x7fbb5006da30 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.896+0000 7fbb55199700 1 -- 172.21.15.119:0/4066104714 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4800ba50 con 0x7fbb5006da30 2023-10-08T02:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.896+0000 7fbb55199700 1 -- 172.21.15.119:0/4066104714 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480092e0 con 0x7fbb5006da30 2023-10-08T02:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.896+0000 7fbb5719d700 1 -- 172.21.15.119:0/4066104714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb5006da30 msgr2=0x7fbb500c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.896+0000 7fbb5719d700 1 --2- 172.21.15.119:0/4066104714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb5006da30 0x7fbb500c7270 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fbb500c3240 tx=0x7fbb48009840 comp rx=0 tx=0).stop 2023-10-08T02:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.897+0000 7fbb5719d700 1 -- 172.21.15.119:0/4066104714 shutdown_connections 2023-10-08T02:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.897+0000 7fbb5719d700 1 --2- 172.21.15.119:0/4066104714 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb5006da30 0x7fbb500c7270 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.897+0000 7fbb5719d700 1 --2- 172.21.15.119:0/4066104714 >> v2:172.21.15.184:3300/0 conn(0x7fbb50140bb0 0x7fbb5006d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.897+0000 7fbb5719d700 1 -- 172.21.15.119:0/4066104714 >> 172.21.15.119:0/4066104714 conn(0x7fbb500cc1e0 msgr2=0x7fbb500cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.897+0000 7fbb5719d700 1 -- 172.21.15.119:0/4066104714 shutdown_connections 2023-10-08T02:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.897+0000 7fbb5719d700 1 -- 172.21.15.119:0/4066104714 wait complete. 2023-10-08T02:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.898+0000 7fbb5719d700 1 Processor -- start 2023-10-08T02:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.898+0000 7fbb5719d700 1 -- start start 2023-10-08T02:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5719d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb5006da30 0x7fbb5015f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5719d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb501407e0 0x7fbb50168ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5719d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50140bb0 0x7fbb50169220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5719d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500b0b30 con 0x7fbb501407e0 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5719d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5012c940 con 0x7fbb5006da30 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5719d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5012bd30 con 0x7fbb50140bb0 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb501407e0 0x7fbb50168ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5599a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb501407e0 0x7fbb50168ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48840/0 (socket says 172.21.15.119:48840) 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.899+0000 7fbb5599a700 1 -- 172.21.15.119:0/3030160162 learned_addr learned my addr 172.21.15.119:0/3030160162 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5619b700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb5006da30 0x7fbb5015f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5699c700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50140bb0 0x7fbb50169220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5599a700 1 -- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50140bb0 msgr2=0x7fbb50169220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5599a700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50140bb0 0x7fbb50169220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5599a700 1 -- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb5006da30 msgr2=0x7fbb5015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5599a700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb5006da30 0x7fbb5015f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5599a700 1 -- 172.21.15.119:0/3030160162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48009ae0 con 0x7fbb501407e0 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5699c700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50140bb0 0x7fbb50169220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5599a700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb501407e0 0x7fbb50168ce0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000c850 tx=0x7fbb40008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb467fc700 1 -- 172.21.15.119:0/3030160162 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40011600 con 0x7fbb501407e0 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb467fc700 1 -- 172.21.15.119:0/3030160162 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb40011760 con 0x7fbb501407e0 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb467fc700 1 -- 172.21.15.119:0/3030160162 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb40011b20 con 0x7fbb501407e0 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.900+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5012c010 con 0x7fbb501407e0 2023-10-08T02:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.901+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5012c5d0 con 0x7fbb501407e0 2023-10-08T02:31:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.901+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5012caf0 con 0x7fbb501407e0 2023-10-08T02:31:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.902+0000 7fbb467fc700 1 -- 172.21.15.119:0/3030160162 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb40018350 con 0x7fbb501407e0 2023-10-08T02:31:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.903+0000 7fbb467fc700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb34066f30 0x7fbb340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.903+0000 7fbb467fc700 1 -- 172.21.15.119:0/3030160162 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5975+0+0 (secure 0 0 0) 0x7fbb40084560 con 0x7fbb501407e0 2023-10-08T02:31:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.903+0000 7fbb5619b700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb34066f30 0x7fbb340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:12.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.904+0000 7fbb5619b700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb34066f30 0x7fbb340693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c007fd0 tx=0x7fbb4c007d50 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:12.907+0000 7fbb467fc700 1 -- 172.21.15.119:0/3030160162 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb5012caf0 con 0x7fbb501407e0 2023-10-08T02:31:13.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.096+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fbb5012cab0 con 0x7fbb501407e0 2023-10-08T02:31:13.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.102+0000 7fbb467fc700 1 -- 172.21.15.119:0/3030160162 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v69) v1 ==== 120+0+0 (secure 0 0 0) 0x7fbb40054ec0 con 0x7fbb501407e0 2023-10-08T02:31:13.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb34066f30 msgr2=0x7fbb340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb34066f30 0x7fbb340693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c007fd0 tx=0x7fbb4c007d50 comp rx=0 tx=0).stop 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb501407e0 msgr2=0x7fbb50168ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb501407e0 0x7fbb50168ce0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fbb4000c850 tx=0x7fbb40008040 comp rx=0 tx=0).stop 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 shutdown_connections 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb34066f30 0x7fbb340693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb50140bb0 0x7fbb50169220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb5006da30 0x7fbb5015f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 --2- 172.21.15.119:0/3030160162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb501407e0 0x7fbb50168ce0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 >> 172.21.15.119:0/3030160162 conn(0x7fbb500cc1e0 msgr2=0x7fbb500c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 shutdown_connections 2023-10-08T02:31:13.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.105+0000 7fbb5719d700 1 -- 172.21.15.119:0/3030160162 wait complete. 2023-10-08T02:31:13.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-10-08T02:31:13.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.303+0000 7fa2553ff700 1 Processor -- start 2023-10-08T02:31:13.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.304+0000 7fa2553ff700 1 -- start start 2023-10-08T02:31:13.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa2553ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2500cbf30 0x7fa2500ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:13.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa2553ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa2500b1420 0x7fa2500c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:13.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa2553ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2500cc300 con 0x7fa2500ccf90 2023-10-08T02:31:13.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa2553ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa250156bf0 con 0x7fa2500cbf30 2023-10-08T02:31:13.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa2553ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2500ae950 con 0x7fa2500b1420 2023-10-08T02:31:13.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa24effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2500cbf30 0x7fa2500ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:13.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa24effd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa2500cbf30 0x7fa2500ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48846/0 (socket says 172.21.15.119:48846) 2023-10-08T02:31:13.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa24effd700 1 -- 172.21.15.119:0/2531579831 learned_addr learned my addr 172.21.15.119:0/2531579831 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa24f7fe700 1 --2- 172.21.15.119:0/2531579831 >> v2:172.21.15.184:3300/0 conn(0x7fa2500b1420 0x7fa2500c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.305+0000 7fa24effd700 1 -- 172.21.15.119:0/2531579831 >> v2:172.21.15.184:3300/0 conn(0x7fa2500b1420 msgr2=0x7fa2500c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.306+0000 7fa24effd700 1 --2- 172.21.15.119:0/2531579831 >> v2:172.21.15.184:3300/0 conn(0x7fa2500b1420 0x7fa2500c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.306+0000 7fa24effd700 1 -- 172.21.15.119:0/2531579831 >> v1:172.21.15.184:6789/0 conn(0x7fa2500ccf90 legacy=0x7fa2500b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.306+0000 7fa24effd700 1 -- 172.21.15.119:0/2531579831 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2500ad370 con 0x7fa2500cbf30 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.306+0000 7fa24f7fe700 1 --2- 172.21.15.119:0/2531579831 >> v2:172.21.15.184:3300/0 conn(0x7fa2500b1420 0x7fa2500c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.306+0000 7fa24effd700 1 --2- 172.21.15.119:0/2531579831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500cbf30 0x7fa2500ae410 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fa2500b19e0 tx=0x7fa240009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ce8f8009254d4e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.306+0000 7fa24e7fc700 1 -- 172.21.15.119:0/2531579831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000b820 con 0x7fa2500cbf30 2023-10-08T02:31:13.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.306+0000 7fa24e7fc700 1 -- 172.21.15.119:0/2531579831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24000b980 con 0x7fa2500cbf30 2023-10-08T02:31:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.306+0000 7fa24e7fc700 1 -- 172.21.15.119:0/2531579831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000bd20 con 0x7fa2500cbf30 2023-10-08T02:31:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.307+0000 7fa2553ff700 1 -- 172.21.15.119:0/2531579831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500cbf30 msgr2=0x7fa2500ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.307+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2531579831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500cbf30 0x7fa2500ae410 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fa2500b19e0 tx=0x7fa240009250 comp rx=0 tx=0).stop 2023-10-08T02:31:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.307+0000 7fa2553ff700 1 -- 172.21.15.119:0/2531579831 shutdown_connections 2023-10-08T02:31:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.307+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2531579831 >> v2:172.21.15.184:3300/0 conn(0x7fa2500b1420 0x7fa2500c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.307+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2531579831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500cbf30 0x7fa2500ae410 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.307+0000 7fa2553ff700 1 -- 172.21.15.119:0/2531579831 >> 172.21.15.119:0/2531579831 conn(0x7fa25006a320 msgr2=0x7fa2500cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:13.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.307+0000 7fa2553ff700 1 -- 172.21.15.119:0/2531579831 shutdown_connections 2023-10-08T02:31:13.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.307+0000 7fa2553ff700 1 -- 172.21.15.119:0/2531579831 wait complete. 2023-10-08T02:31:13.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.308+0000 7fa2553ff700 1 Processor -- start 2023-10-08T02:31:13.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.308+0000 7fa2553ff700 1 -- start start 2023-10-08T02:31:13.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.309+0000 7fa2553ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500b1420 0x7fa25015f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:13.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.309+0000 7fa2553ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2500cbf30 0x7fa25015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:13.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.309+0000 7fa2553ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2500ccf90 0x7fa250158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.309+0000 7fa2553ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25015dd60 con 0x7fa2500b1420 2023-10-08T02:31:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.309+0000 7fa2553ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25015d150 con 0x7fa2500cbf30 2023-10-08T02:31:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.309+0000 7fa2553ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25015d430 con 0x7fa2500ccf90 2023-10-08T02:31:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.309+0000 7fa24effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500b1420 0x7fa25015f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24effd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500b1420 0x7fa25015f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48854/0 (socket says 172.21.15.119:48854) 2023-10-08T02:31:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24effd700 1 -- 172.21.15.119:0/2688508922 learned_addr learned my addr 172.21.15.119:0/2688508922 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24f7fe700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2500ccf90 0x7fa250158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:13.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa247fff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2500cbf30 0x7fa25015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24effd700 1 -- 172.21.15.119:0/2688508922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2500ccf90 msgr2=0x7fa250158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24effd700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2500ccf90 0x7fa250158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24effd700 1 -- 172.21.15.119:0/2688508922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2500cbf30 msgr2=0x7fa25015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24effd700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2500cbf30 0x7fa25015e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24effd700 1 -- 172.21.15.119:0/2688508922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2400094f0 con 0x7fa2500b1420 2023-10-08T02:31:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa24effd700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500b1420 0x7fa25015f0c0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa2400094c0 tx=0x7fa240017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa246ffd700 1 -- 172.21.15.119:0/2688508922 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24001a420 con 0x7fa2500b1420 2023-10-08T02:31:13.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa246ffd700 1 -- 172.21.15.119:0/2688508922 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24001a5c0 con 0x7fa2500b1420 2023-10-08T02:31:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa246ffd700 1 -- 172.21.15.119:0/2688508922 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24001a980 con 0x7fa2500b1420 2023-10-08T02:31:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.310+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25015d710 con 0x7fa2500b1420 2023-10-08T02:31:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.311+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25015f600 con 0x7fa2500b1420 2023-10-08T02:31:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.312+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25015e040 con 0x7fa2500b1420 2023-10-08T02:31:13.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.312+0000 7fa246ffd700 1 -- 172.21.15.119:0/2688508922 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa24002e450 con 0x7fa2500b1420 2023-10-08T02:31:13.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.314+0000 7fa246ffd700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa230066f30 0x7fa2300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:13.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.314+0000 7fa246ffd700 1 -- 172.21.15.119:0/2688508922 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5975+0+0 (secure 0 0 0) 0x7fa2400860a0 con 0x7fa2500b1420 2023-10-08T02:31:13.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.314+0000 7fa247fff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa230066f30 0x7fa2300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:13.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.314+0000 7fa247fff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa230066f30 0x7fa2300693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa250156860 tx=0x7fa238009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:13.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.318+0000 7fa246ffd700 1 -- 172.21.15.119:0/2688508922 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa240053200 con 0x7fa2500b1420 2023-10-08T02:31:13.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.588+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fa25004c3c0 con 0x7fa2500b1420 2023-10-08T02:31:13.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.849+0000 7fa246ffd700 1 -- 172.21.15.119:0/2688508922 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3079) v1 ==== 163+0+0 (secure 0 0 0) 0x7fa240056820 con 0x7fa2500b1420 2023-10-08T02:31:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa230066f30 msgr2=0x7fa2300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa230066f30 0x7fa2300693e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa250156860 tx=0x7fa238009040 comp rx=0 tx=0).stop 2023-10-08T02:31:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500b1420 msgr2=0x7fa25015f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500b1420 0x7fa25015f0c0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7fa2400094c0 tx=0x7fa240017000 comp rx=0 tx=0).stop 2023-10-08T02:31:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 shutdown_connections 2023-10-08T02:31:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa230066f30 0x7fa2300693e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa2500ccf90 0x7fa250158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa2500b1420 0x7fa25015f0c0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 --2- 172.21.15.119:0/2688508922 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa2500cbf30 0x7fa25015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 >> 172.21.15.119:0/2688508922 conn(0x7fa25006a320 msgr2=0x7fa2500ca7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 shutdown_connections 2023-10-08T02:31:13.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:13.852+0000 7fa2553ff700 1 -- 172.21.15.119:0/2688508922 wait complete. 2023-10-08T02:31:13.870 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-10-08T02:31:13.870 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-10-08T02:31:13.871 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-10-08T02:31:13.871 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-08T02:31:13.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:31:13.962 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:31:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.056+0000 7f8ca359e700 1 Processor -- start 2023-10-08T02:31:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.056+0000 7f8ca359e700 1 -- start start 2023-10-08T02:31:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150550 0x7f8ca4157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca359e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f8ca40c7410 0x7f8ca4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca359e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca406e2a0 con 0x7f8ca4150180 2023-10-08T02:31:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca359e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca406b2e0 con 0x7f8ca40c7410 2023-10-08T02:31:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40c78e0 con 0x7f8ca4150550 2023-10-08T02:31:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca259c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f8ca4150180 0x7f8ca40c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:59288/0 (socket says 172.21.15.119:59288) 2023-10-08T02:31:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca259c700 1 -- 172.21.15.119:0/3000881474 learned_addr learned my addr 172.21.15.119:0/3000881474 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8c9bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150550 0x7f8ca4157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca2d9d700 1 --2- 172.21.15.119:0/3000881474 >> v2:172.21.15.184:3300/0 conn(0x7f8ca40c7410 0x7f8ca4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:14.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.058+0000 7f8ca1d9b700 1 -- 172.21.15.119:0/3000881474 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1719716765 0 0) 0x7f8ca406e2a0 con 0x7f8ca4150180 2023-10-08T02:31:14.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca1d9b700 1 -- 172.21.15.119:0/3000881474 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c90003570 con 0x7f8ca4150180 2023-10-08T02:31:14.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca2d9d700 1 -- 172.21.15.119:0/3000881474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150550 msgr2=0x7f8ca4157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:14.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca2d9d700 1 --2- 172.21.15.119:0/3000881474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150550 0x7f8ca4157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca2d9d700 1 -- 172.21.15.119:0/3000881474 >> v1:172.21.15.184:6789/0 conn(0x7f8ca4150180 legacy=0x7f8ca40c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:14.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca2d9d700 1 -- 172.21.15.119:0/3000881474 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca40c7bc0 con 0x7f8ca40c7410 2023-10-08T02:31:14.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8c9bfff700 1 --2- 172.21.15.119:0/3000881474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150550 0x7f8ca4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca2d9d700 1 --2- 172.21.15.119:0/3000881474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 0x7f8ca4069f50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009e90 tx=0x7f8c9c009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b28e381c68013 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca1d9b700 1 -- 172.21.15.119:0/3000881474 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c00e070 con 0x7f8ca40c7410 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca1d9b700 1 -- 172.21.15.119:0/3000881474 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c00bb40 con 0x7f8ca40c7410 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.059+0000 7f8ca1d9b700 1 -- 172.21.15.119:0/3000881474 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c012430 con 0x7f8ca40c7410 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.060+0000 7f8ca359e700 1 -- 172.21.15.119:0/3000881474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 msgr2=0x7f8ca4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.060+0000 7f8ca359e700 1 --2- 172.21.15.119:0/3000881474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 0x7f8ca4069f50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009e90 tx=0x7f8c9c009780 comp rx=0 tx=0).stop 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.060+0000 7f8ca359e700 1 -- 172.21.15.119:0/3000881474 shutdown_connections 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.060+0000 7f8ca359e700 1 --2- 172.21.15.119:0/3000881474 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 0x7f8ca4069f50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.060+0000 7f8ca359e700 1 --2- 172.21.15.119:0/3000881474 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150550 0x7f8ca4157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.060+0000 7f8ca359e700 1 -- 172.21.15.119:0/3000881474 >> 172.21.15.119:0/3000881474 conn(0x7f8ca40cc1f0 msgr2=0x7f8ca40cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.060+0000 7f8ca359e700 1 -- 172.21.15.119:0/3000881474 shutdown_connections 2023-10-08T02:31:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.060+0000 7f8ca359e700 1 -- 172.21.15.119:0/3000881474 wait complete. 2023-10-08T02:31:14.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.061+0000 7f8ca359e700 1 Processor -- start 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.062+0000 7f8ca359e700 1 -- start start 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.062+0000 7f8ca359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 0x7f8ca40af950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.062+0000 7f8ca359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150180 0x7f8ca4158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.062+0000 7f8ca359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ca4150550 0x7f8ca40abab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.062+0000 7f8ca359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40b0390 con 0x7f8ca4150550 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.062+0000 7f8ca359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca40b0e80 con 0x7f8ca40c7410 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.062+0000 7f8ca359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca4141070 con 0x7f8ca4150180 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ca4150550 0x7f8ca40abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ca4150550 0x7f8ca40abab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:48862/0 (socket says 172.21.15.119:48862) 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 -- 172.21.15.119:0/1067824002 learned_addr learned my addr 172.21.15.119:0/1067824002 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8c9bfff700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150180 0x7f8ca4158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca259c700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 0x7f8ca40af950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 -- 172.21.15.119:0/1067824002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150180 msgr2=0x7f8ca4158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150180 0x7f8ca4158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 -- 172.21.15.119:0/1067824002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 msgr2=0x7f8ca40af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 0x7f8ca40af950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 -- 172.21.15.119:0/1067824002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c009420 con 0x7f8ca4150550 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.063+0000 7f8ca2d9d700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ca4150550 0x7f8ca40abab0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c013000 tx=0x7f8c9c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.064+0000 7f8c9a7fc700 1 -- 172.21.15.119:0/1067824002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c012ce0 con 0x7f8ca4150550 2023-10-08T02:31:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.064+0000 7f8c9a7fc700 1 -- 172.21.15.119:0/1067824002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c021070 con 0x7f8ca4150550 2023-10-08T02:31:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.064+0000 7f8c9a7fc700 1 -- 172.21.15.119:0/1067824002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c026680 con 0x7f8ca4150550 2023-10-08T02:31:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.064+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c009a20 con 0x7f8ca4150550 2023-10-08T02:31:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.064+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca40b0ba0 con 0x7f8ca4150550 2023-10-08T02:31:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.065+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca404c3c0 con 0x7f8ca4150550 2023-10-08T02:31:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.066+0000 7f8c9a7fc700 1 -- 172.21.15.119:0/1067824002 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8c9c023030 con 0x7f8ca4150550 2023-10-08T02:31:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.066+0000 7f8c9a7fc700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8c84066f30 0x7f8c840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.066+0000 7f8c9a7fc700 1 -- 172.21.15.119:0/1067824002 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f8c9c086770 con 0x7f8ca4150550 2023-10-08T02:31:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.067+0000 7f8ca259c700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8c84066f30 0x7f8c840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.067+0000 7f8ca259c700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8c84066f30 0x7f8c840693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007fd0 tx=0x7f8c94009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:14.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.071+0000 7f8c9a7fc700 1 -- 172.21.15.119:0/1067824002 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8c9c053830 con 0x7f8ca4150550 2023-10-08T02:31:14.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.335+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f8ca406ada0 con 0x7f8ca4150550 2023-10-08T02:31:14.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.853+0000 7f8c9a7fc700 1 -- 172.21.15.119:0/1067824002 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3080) v1 ==== 158+0+0 (secure 0 0 0) 0x7f8c9c012e60 con 0x7f8ca4150550 2023-10-08T02:31:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8c84066f30 msgr2=0x7f8c840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8c84066f30 0x7f8c840693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8c94007fd0 tx=0x7f8c94009040 comp rx=0 tx=0).stop 2023-10-08T02:31:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ca4150550 msgr2=0x7f8ca40abab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ca4150550 0x7f8ca40abab0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c013000 tx=0x7f8c9c019000 comp rx=0 tx=0).stop 2023-10-08T02:31:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 shutdown_connections 2023-10-08T02:31:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f8c84066f30 0x7f8c840693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ca4150550 0x7f8ca40abab0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f8ca40c7410 0x7f8ca40af950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 --2- 172.21.15.119:0/1067824002 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f8ca4150180 0x7f8ca4158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:14.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 >> 172.21.15.119:0/1067824002 conn(0x7f8ca40cc1f0 msgr2=0x7f8ca414f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:14.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 shutdown_connections 2023-10-08T02:31:14.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:14.856+0000 7f8ca359e700 1 -- 172.21.15.119:0/1067824002 wait complete. 2023-10-08T02:31:14.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-08T02:31:15.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.100+0000 7f45bdd9d700 1 Processor -- start 2023-10-08T02:31:15.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.101+0000 7f45bdd9d700 1 -- start start 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.102+0000 7f45bdd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b80c9db0 0x7f45b806a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.102+0000 7f45bdd9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b806e150 con 0x7f45b8157130 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.102+0000 7f45bdd9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b80c7800 con 0x7f45b8153900 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.102+0000 7f45bdd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80c7610 con 0x7f45b80c9db0 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.102+0000 7f45bd59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f45b8157130 0x7f45b806e5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40802/0 (socket says 172.21.15.119:40802) 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.102+0000 7f45bd59c700 1 -- 172.21.15.119:0/3497242121 learned_addr learned my addr 172.21.15.119:0/3497242121 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.102+0000 7f45b7fff700 1 --2- 172.21.15.119:0/3497242121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b80c9db0 0x7f45b806a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b77fe700 1 -- 172.21.15.119:0/3497242121 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871138829 0 0) 0x7f45b806e150 con 0x7f45b8157130 2023-10-08T02:31:15.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b77fe700 1 -- 172.21.15.119:0/3497242121 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45a8003570 con 0x7f45b8157130 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b77fe700 1 -- 172.21.15.119:0/3497242121 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170041585 0 0) 0x7f45b80c7800 con 0x7f45b8153900 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b77fe700 1 -- 172.21.15.119:0/3497242121 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45b806e150 con 0x7f45b8153900 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b7fff700 1 -- 172.21.15.119:0/3497242121 >> v1:172.21.15.184:6789/0 conn(0x7f45b8153900 legacy=0x7f45b80c6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b7fff700 1 -- 172.21.15.119:0/3497242121 >> v1:172.21.15.119:6789/0 conn(0x7f45b8157130 legacy=0x7f45b806e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b7fff700 1 -- 172.21.15.119:0/3497242121 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80c7ae0 con 0x7f45b80c9db0 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b7fff700 1 --2- 172.21.15.119:0/3497242121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b80c9db0 0x7f45b806a000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45a400aac0 tx=0x7f45a400a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45041076879ab325 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b77fe700 1 -- 172.21.15.119:0/3497242121 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400c970 con 0x7f45b80c9db0 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b77fe700 1 -- 172.21.15.119:0/3497242121 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a400cad0 con 0x7f45b80c9db0 2023-10-08T02:31:15.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.103+0000 7f45b77fe700 1 -- 172.21.15.119:0/3497242121 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a400ce70 con 0x7f45b80c9db0 2023-10-08T02:31:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.104+0000 7f45bdd9d700 1 -- 172.21.15.119:0/3497242121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b80c9db0 msgr2=0x7f45b806a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.104+0000 7f45bdd9d700 1 --2- 172.21.15.119:0/3497242121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b80c9db0 0x7f45b806a000 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f45a400aac0 tx=0x7f45a400a330 comp rx=0 tx=0).stop 2023-10-08T02:31:15.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.104+0000 7f45bdd9d700 1 -- 172.21.15.119:0/3497242121 shutdown_connections 2023-10-08T02:31:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.104+0000 7f45bdd9d700 1 --2- 172.21.15.119:0/3497242121 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b80c9db0 0x7f45b806a000 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.104+0000 7f45bdd9d700 1 -- 172.21.15.119:0/3497242121 >> 172.21.15.119:0/3497242121 conn(0x7f45b80cc1e0 msgr2=0x7f45b80cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.104+0000 7f45bdd9d700 1 -- 172.21.15.119:0/3497242121 shutdown_connections 2023-10-08T02:31:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.104+0000 7f45bdd9d700 1 -- 172.21.15.119:0/3497242121 wait complete. 2023-10-08T02:31:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.105+0000 7f45bdd9d700 1 Processor -- start 2023-10-08T02:31:15.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.106+0000 7f45bdd9d700 1 -- start start 2023-10-08T02:31:15.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bdd9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b80c9db0 0x7f45b80ad950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bdd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b8153900 0x7f45b80abd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bdd9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b8157130 0x7f45b80aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bdd9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b81512e0 con 0x7f45b8157130 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bdd9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b80ade90 con 0x7f45b80c9db0 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bdd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b816a440 con 0x7f45b8153900 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bd59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b8157130 0x7f45b80aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bd59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b8157130 0x7f45b80aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59516/0 (socket says 172.21.15.119:59516) 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bd59c700 1 -- 172.21.15.119:0/784755840 learned_addr learned my addr 172.21.15.119:0/784755840 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bcd9b700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b80c9db0 0x7f45b80ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bd59c700 1 -- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b8153900 msgr2=0x7f45b80abd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45b7fff700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b8153900 0x7f45b80abd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bd59c700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b8153900 0x7f45b80abd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bd59c700 1 -- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b80c9db0 msgr2=0x7f45b80ad950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bd59c700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b80c9db0 0x7f45b80ad950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bd59c700 1 -- 172.21.15.119:0/784755840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a400a5d0 con 0x7f45b8157130 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45bcd9b700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b80c9db0 0x7f45b80ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.107+0000 7f45b7fff700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b8153900 0x7f45b80abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.108+0000 7f45bd59c700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b8157130 0x7f45b80aab40 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f45b000cc40 tx=0x7f45b0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.108+0000 7f45b4ff9700 1 -- 172.21.15.119:0/784755840 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000a960 con 0x7f45b8157130 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.108+0000 7f45b4ff9700 1 -- 172.21.15.119:0/784755840 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b000aac0 con 0x7f45b8157130 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.108+0000 7f45b4ff9700 1 -- 172.21.15.119:0/784755840 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000ae60 con 0x7f45b8157130 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.108+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b816a780 con 0x7f45b8157130 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.108+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b816afc0 con 0x7f45b8157130 2023-10-08T02:31:15.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.109+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b806c450 con 0x7f45b8157130 2023-10-08T02:31:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.110+0000 7f45b4ff9700 1 -- 172.21.15.119:0/784755840 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f45b0007300 con 0x7f45b8157130 2023-10-08T02:31:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.110+0000 7f45b4ff9700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.111+0000 7f45b4ff9700 1 -- 172.21.15.119:0/784755840 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5975+0+0 (secure 0 0 0) 0x7f45b0084a30 con 0x7f45b8157130 2023-10-08T02:31:15.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.111+0000 7f45bcd9b700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.111+0000 7f45bcd9b700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f459c066f30 0x7f459c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45ac007fd0 tx=0x7f45ac009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:15.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.115+0000 7f45b4ff9700 1 -- 172.21.15.119:0/784755840 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f45b0051c10 con 0x7f45b8157130 2023-10-08T02:31:15.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.307+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f45b80c7dc0 con 0x7f45b8157130 2023-10-08T02:31:15.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.313+0000 7f45b4ff9700 1 -- 172.21.15.119:0/784755840 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f45b0055230 con 0x7f45b8157130 2023-10-08T02:31:15.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f459c066f30 msgr2=0x7f459c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f459c066f30 0x7f459c0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f45ac007fd0 tx=0x7f45ac009040 comp rx=0 tx=0).stop 2023-10-08T02:31:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b8157130 msgr2=0x7f45b80aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b8157130 0x7f45b80aab40 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f45b000cc40 tx=0x7f45b0001a40 comp rx=0 tx=0).stop 2023-10-08T02:31:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 shutdown_connections 2023-10-08T02:31:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f459c066f30 0x7f459c0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f45b8157130 0x7f45b80aab40 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f45b80c9db0 0x7f45b80ad950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 --2- 172.21.15.119:0/784755840 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f45b8153900 0x7f45b80abd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 >> 172.21.15.119:0/784755840 conn(0x7f45b80cc1e0 msgr2=0x7f45b81519c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 shutdown_connections 2023-10-08T02:31:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.315+0000 7f45bdd9d700 1 -- 172.21.15.119:0/784755840 wait complete. 2023-10-08T02:31:15.333 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:31:15.334 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:31:15.421 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:31:15.505 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:31:15.505 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-08T02:31:15.506 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:15.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.687+0000 7fa6a3fff700 1 Processor -- start 2023-10-08T02:31:15.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.687+0000 7fa6a3fff700 1 -- start start 2023-10-08T02:31:15.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.688+0000 7fa6a3fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa69c159890 0x7fa69c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.688+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159c60 0x7fa69c15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.688+0000 7fa6a3fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa69c15db10 0x7fa69c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.688+0000 7fa6a3fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69c15a160 con 0x7fa69c15db10 2023-10-08T02:31:15.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.688+0000 7fa6a3fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa69c0d13a0 con 0x7fa69c159890 2023-10-08T02:31:15.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.688+0000 7fa6a3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c15e140 con 0x7fa69c159c60 2023-10-08T02:31:15.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa69c15db10 0x7fa69c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa69c15db10 0x7fa69c069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59518/0 (socket says 172.21.15.119:59518) 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 -- 172.21.15.119:0/2906947125 learned_addr learned my addr 172.21.15.119:0/2906947125 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a159a700 1 --2- 172.21.15.119:0/2906947125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159c60 0x7fa69c15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a1d9b700 1 --2- 172.21.15.119:0/2906947125 >> v2:172.21.15.184:3300/0 conn(0x7fa69c159890 0x7fa69c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 -- 172.21.15.119:0/2906947125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159c60 msgr2=0x7fa69c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 --2- 172.21.15.119:0/2906947125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159c60 0x7fa69c15d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 -- 172.21.15.119:0/2906947125 >> v2:172.21.15.184:3300/0 conn(0x7fa69c159890 msgr2=0x7fa69c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 --2- 172.21.15.119:0/2906947125 >> v2:172.21.15.184:3300/0 conn(0x7fa69c159890 0x7fa69c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 -- 172.21.15.119:0/2906947125 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa69c15a3d0 con 0x7fa69c15db10 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a259c700 1 --2- 172.21.15.119:0/2906947125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c069560 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0d1090 tx=0x7fa694009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a49b561115568b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.689+0000 7fa6a0d99700 1 -- 172.21.15.119:0/2906947125 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69400e070 con 0x7fa69c15db10 2023-10-08T02:31:15.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a0d99700 1 -- 172.21.15.119:0/2906947125 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69400ba50 con 0x7fa69c15db10 2023-10-08T02:31:15.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a0d99700 1 -- 172.21.15.119:0/2906947125 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6940092e0 con 0x7fa69c15db10 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 -- 172.21.15.119:0/2906947125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 msgr2=0x7fa69c069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/2906947125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c069560 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0d1090 tx=0x7fa694009840 comp rx=0 tx=0).stop 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 -- 172.21.15.119:0/2906947125 shutdown_connections 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/2906947125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c069560 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/2906947125 >> v2:172.21.15.184:3300/0 conn(0x7fa69c159890 0x7fa69c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/2906947125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159c60 0x7fa69c15d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 -- 172.21.15.119:0/2906947125 >> 172.21.15.119:0/2906947125 conn(0x7fa69c068e20 msgr2=0x7fa69c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 -- 172.21.15.119:0/2906947125 shutdown_connections 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.690+0000 7fa6a3fff700 1 -- 172.21.15.119:0/2906947125 wait complete. 2023-10-08T02:31:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.692+0000 7fa6a3fff700 1 Processor -- start 2023-10-08T02:31:15.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.692+0000 7fa6a3fff700 1 -- start start 2023-10-08T02:31:15.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159890 0x7fa69c155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c159c60 0x7fa69c14f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a3fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a3fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c155180 con 0x7fa69c15db10 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a3fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c154570 con 0x7fa69c159c60 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a3fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69c154850 con 0x7fa69c159890 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59530/0 (socket says 172.21.15.119:59530) 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 -- 172.21.15.119:0/3126561382 learned_addr learned my addr 172.21.15.119:0/3126561382 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a1d9b700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159890 0x7fa69c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a159a700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c159c60 0x7fa69c14f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 -- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159890 msgr2=0x7fa69c155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159890 0x7fa69c155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 -- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c159c60 msgr2=0x7fa69c14f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c159c60 0x7fa69c14f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 -- 172.21.15.119:0/3126561382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa694009ae0 con 0x7fa69c15db10 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a159a700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c159c60 0x7fa69c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a1d9b700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159890 0x7fa69c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.693+0000 7fa6a259c700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c153840 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa69c070e20 tx=0x7fa694018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.694+0000 7fa691ffb700 1 -- 172.21.15.119:0/3126561382 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6940032f0 con 0x7fa69c15db10 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.694+0000 7fa691ffb700 1 -- 172.21.15.119:0/3126561382 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa694008e80 con 0x7fa69c15db10 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.694+0000 7fa691ffb700 1 -- 172.21.15.119:0/3126561382 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69401c660 con 0x7fa69c15db10 2023-10-08T02:31:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.694+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa69c154b30 con 0x7fa69c15db10 2023-10-08T02:31:15.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.694+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa69c14fc70 con 0x7fa69c15db10 2023-10-08T02:31:15.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.695+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa69c155330 con 0x7fa69c15db10 2023-10-08T02:31:15.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.696+0000 7fa691ffb700 1 -- 172.21.15.119:0/3126561382 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa694003040 con 0x7fa69c15db10 2023-10-08T02:31:15.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.697+0000 7fa691ffb700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa680066f30 0x7fa6800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:15.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.697+0000 7fa691ffb700 1 -- 172.21.15.119:0/3126561382 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 5975+0+0 (secure 0 0 0) 0x7fa694085fe0 con 0x7fa69c15db10 2023-10-08T02:31:15.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.697+0000 7fa6a1d9b700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa680066f30 0x7fa6800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:15.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.698+0000 7fa6a1d9b700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa680066f30 0x7fa6800693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698008040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.701+0000 7fa691ffb700 1 -- 172.21.15.119:0/3126561382 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa6940530e0 con 0x7fa69c15db10 2023-10-08T02:31:15.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.936+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa69c04c3c0 con 0x7fa69c15db10 2023-10-08T02:31:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.937+0000 7fa691ffb700 1 -- 172.21.15.119:0/3126561382 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3538 (secure 0 0 0) 0x7fa694021df0 con 0x7fa69c15db10 2023-10-08T02:31:15.938 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:15.938 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7814,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/3041824229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":3041824229},{"type":"v1","addr":"172.21.15.184:6834","nonce":3041824229}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8066,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/2792101658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":2792101658},{"type":"v1","addr":"172.21.15.184:6835","nonce":2792101658}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9717,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6834/1478841333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1478841333},{"type":"v1","addr":"172.21.15.119:6834","nonce":1478841333}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9723,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/483355636","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6833","nonce":483355636},{"type":"v1","addr":"172.21.15.119:6835","nonce":483355636}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[]} 2023-10-08T02:31:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa680066f30 msgr2=0x7fa6800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa680066f30 0x7fa6800693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa698006fd0 tx=0x7fa698008040 comp rx=0 tx=0).stop 2023-10-08T02:31:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 msgr2=0x7fa69c153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c153840 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa69c070e20 tx=0x7fa694018000 comp rx=0 tx=0).stop 2023-10-08T02:31:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 shutdown_connections 2023-10-08T02:31:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa680066f30 0x7fa6800693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa69c159890 0x7fa69c155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa69c15db10 0x7fa69c153840 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 --2- 172.21.15.119:0/3126561382 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa69c159c60 0x7fa69c14f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 >> 172.21.15.119:0/3126561382 conn(0x7fa69c068e20 msgr2=0x7fa69c0d05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 shutdown_connections 2023-10-08T02:31:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:15.939+0000 7fa6a3fff700 1 -- 172.21.15.119:0/3126561382 wait complete. 2023-10-08T02:31:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 89 2023-10-08T02:31:15.958 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-08T02:31:15.958 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-08T02:31:15.958 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T02:31:15.959 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:31:15.959 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-08T02:31:15.959 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-08T02:31:15.959 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T02:31:15.959 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:31:15.960 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-08T02:31:15.960 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-08T02:31:15.960 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T02:31:15.960 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:31:15.961 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-08T02:31:15.961 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-08T02:31:15.961 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T02:31:15.961 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:31:15.974 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T02:31:15.975 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-08T02:31:15.977 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T02:31:15.977 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-08T02:31:15.978 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T02:31:15.978 DEBUG:teuthology.orchestra.run.smithi184:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-08T02:31:16.019 INFO:tasks.ceph.mds.c:Started 2023-10-08T02:31:16.019 DEBUG:teuthology.parallel:result is None 2023-10-08T02:31:16.022 INFO:tasks.ceph.mds.b:Started 2023-10-08T02:31:16.023 DEBUG:teuthology.parallel:result is None 2023-10-08T02:31:16.025 INFO:tasks.ceph.mds.d:Started 2023-10-08T02:31:16.025 DEBUG:teuthology.parallel:result is None 2023-10-08T02:31:16.146 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-10-08T02:31:16.159 INFO:tasks.ceph.mds.b.smithi184.stdout:starting mds.b at 2023-10-08T02:31:16.159 INFO:tasks.ceph.mds.d.smithi184.stdout:starting mds.d at 2023-10-08T02:31:20.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-10-08T02:31:21.974 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T02:31:21.974 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-08T02:31:21.979 INFO:tasks.ceph.mds.a:Started 2023-10-08T02:31:21.980 DEBUG:teuthology.parallel:result is None 2023-10-08T02:31:21.980 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-08T02:31:22.119 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T02:31:22.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.173+0000 7fbc8ebff700 1 Processor -- start 2023-10-08T02:31:22.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.173+0000 7fbc8ebff700 1 -- start start 2023-10-08T02:31:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.174+0000 7fbc8ebff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fbc880cbf30 0x7fbc880ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.174+0000 7fbc8ebff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc880ccf90 0x7fbc880b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.174+0000 7fbc8ebff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880cc300 con 0x7fbc8815a270 2023-10-08T02:31:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.174+0000 7fbc8ebff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc88156bf0 con 0x7fbc880cbf30 2023-10-08T02:31:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.174+0000 7fbc8ebff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880ae950 con 0x7fbc880ccf90 2023-10-08T02:31:22.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc8d19c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbc8815a270 0x7fbc880c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40828/0 (socket says 172.21.15.119:40828) 2023-10-08T02:31:22.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc8d19c700 1 -- 172.21.15.119:0/1137678007 learned_addr learned my addr 172.21.15.119:0/1137678007 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:22.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188689549 0 0) 0x7fbc880cc300 con 0x7fbc8815a270 2023-10-08T02:31:22.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc8c99b700 1 --2- 172.21.15.119:0/1137678007 >> v2:172.21.15.184:3300/0 conn(0x7fbc880cbf30 0x7fbc880ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc880ccf90 0x7fbc880b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc78003570 con 0x7fbc8815a270 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2068079118 0 0) 0x7fbc78003570 con 0x7fbc8815a270 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc880ccf90 msgr2=0x7fbc880b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc877fe700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc880ccf90 0x7fbc880b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 >> v2:172.21.15.184:3300/0 conn(0x7fbc880cbf30 msgr2=0x7fbc880ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc877fe700 1 --2- 172.21.15.119:0/1137678007 >> v2:172.21.15.184:3300/0 conn(0x7fbc880cbf30 0x7fbc880ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.175+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880ad370 con 0x7fbc8815a270 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fbc800017f0 con 0x7fbc8815a270 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 >> v1:172.21.15.119:6789/0 conn(0x7fbc8815a270 legacy=0x7fbc880c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc877fe700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc78003b10 0x7fbc78003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc877fe700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc877fe700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc880ccf90 0x7fbc780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88156bf0 con 0x7fbc78004810 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880ae950 con 0x7fbc78003b10 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc7800dda0 con 0x7fbc880ccf90 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc8c99b700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc78003b10 0x7fbc78003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc8d19c700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc880ccf90 0x7fbc780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc87fff700 1 -- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc880ccf90 msgr2=0x7fbc780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc880ccf90 0x7fbc780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc87fff700 1 -- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc78003b10 msgr2=0x7fbc78003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc78003b10 0x7fbc78003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:22.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc87fff700 1 -- 172.21.15.119:0/1137678007 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc78003570 con 0x7fbc78004810 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.176+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc78004c80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009c00 tx=0x7fbc740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50246a4accec737c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400e710 con 0x7fbc78004810 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7400e870 con 0x7fbc78004810 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc877fe700 1 -- 172.21.15.119:0/1137678007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7400ec30 con 0x7fbc78004810 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1137678007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 msgr2=0x7fbc78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc8ebff700 1 --2- 172.21.15.119:0/1137678007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc78004c80 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fbc74009c00 tx=0x7fbc740091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc8d19c700 1 -- 172.21.15.119:0/1137678007 reap_dead start 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1137678007 shutdown_connections 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1137678007 >> 172.21.15.119:0/1137678007 conn(0x7fbc8806a320 msgr2=0x7fbc880cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1137678007 shutdown_connections 2023-10-08T02:31:22.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.177+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1137678007 wait complete. 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.179+0000 7fbc8ebff700 1 Processor -- start 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.179+0000 7fbc8ebff700 1 -- start start 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8ebff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc88155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8ebff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8806ebf0 0x7fbc8806f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8ebff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8815c870 0x7fbc8815cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8ebff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc88155a50 con 0x7fbc78004810 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8ebff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8806f5a0 con 0x7fbc8806ebf0 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8ebff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc8816da50 con 0x7fbc8815c870 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8c99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc88155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8c99b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc88155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59582/0 (socket says 172.21.15.119:59582) 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8c99b700 1 -- 172.21.15.119:0/1025176580 learned_addr learned my addr 172.21.15.119:0/1025176580 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8d19c700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8815c870 0x7fbc8815cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8806ebf0 0x7fbc8806f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8c99b700 1 -- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8815c870 msgr2=0x7fbc8815cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8c99b700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8815c870 0x7fbc8815cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.180+0000 7fbc8c99b700 1 -- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8806ebf0 msgr2=0x7fbc8806f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc8c99b700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8806ebf0 0x7fbc8806f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc8c99b700 1 -- 172.21.15.119:0/1025176580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc8816d120 con 0x7fbc78004810 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8806ebf0 0x7fbc8806f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc8c99b700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc88155510 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c011d40 tx=0x7fbc7c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc84ff9700 1 -- 172.21.15.119:0/1025176580 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c00fac0 con 0x7fbc78004810 2023-10-08T02:31:22.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc84ff9700 1 -- 172.21.15.119:0/1025176580 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7c00fc20 con 0x7fbc78004810 2023-10-08T02:31:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc8816d400 con 0x7fbc78004810 2023-10-08T02:31:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc84ff9700 1 -- 172.21.15.119:0/1025176580 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c014440 con 0x7fbc78004810 2023-10-08T02:31:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.181+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8815d220 con 0x7fbc78004810 2023-10-08T02:31:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.182+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc8816d8a0 con 0x7fbc78004810 2023-10-08T02:31:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.183+0000 7fbc84ff9700 1 -- 172.21.15.119:0/1025176580 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbc7c014a80 con 0x7fbc78004810 2023-10-08T02:31:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.184+0000 7fbc84ff9700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbc6c066f30 0x7fbc6c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.184+0000 7fbc84ff9700 1 -- 172.21.15.119:0/1025176580 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5975+0+0 (secure 0 0 0) 0x7fbc7c018030 con 0x7fbc78004810 2023-10-08T02:31:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.185+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbc6c066f30 0x7fbc6c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.185+0000 7fbc87fff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbc6c066f30 0x7fbc6c0693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbc740091e0 tx=0x7fbc7401a000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:22.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.189+0000 7fbc84ff9700 1 -- 172.21.15.119:0/1025176580 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbc7c0524a0 con 0x7fbc78004810 2023-10-08T02:31:22.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:22.398+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbc8804c3c0 con 0x7fbc78004810 2023-10-08T02:31:23.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.365+0000 7fbc84ff9700 1 -- 172.21.15.119:0/1025176580 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v202) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbc7c055ac0 con 0x7fbc78004810 2023-10-08T02:31:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbc6c066f30 msgr2=0x7fbc6c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbc6c066f30 0x7fbc6c0693e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbc740091e0 tx=0x7fbc7401a000 comp rx=0 tx=0).stop 2023-10-08T02:31:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 msgr2=0x7fbc88155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc88155510 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c011d40 tx=0x7fbc7c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 shutdown_connections 2023-10-08T02:31:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbc6c066f30 0x7fbc6c0693e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbc8815c870 0x7fbc8815cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc78004810 0x7fbc88155510 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 --2- 172.21.15.119:0/1025176580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbc8806ebf0 0x7fbc8806f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 >> 172.21.15.119:0/1025176580 conn(0x7fbc8806a320 msgr2=0x7fbc880ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 shutdown_connections 2023-10-08T02:31:23.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.368+0000 7fbc8ebff700 1 -- 172.21.15.119:0/1025176580 wait complete. 2023-10-08T02:31:23.370 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-10-08T02:31:23.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-08T02:31:23.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.573+0000 7ff0579ff700 1 Processor -- start 2023-10-08T02:31:23.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.573+0000 7ff0579ff700 1 -- start start 2023-10-08T02:31:23.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.574+0000 7ff0579ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 0x7ff0500ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:23.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.574+0000 7ff0579ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff0500ccf90 0x7ff0500c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:23.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff0579ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0500cc300 con 0x7ff0500cbf30 2023-10-08T02:31:23.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff0579ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff050156bf0 con 0x7ff0500ccf90 2023-10-08T02:31:23.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff0579ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500ae950 con 0x7ff0500b1420 2023-10-08T02:31:23.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff05579b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff0500cbf30 0x7ff0500b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40830/0 (socket says 172.21.15.119:40830) 2023-10-08T02:31:23.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff05579b700 1 -- 172.21.15.119:0/2977310984 learned_addr learned my addr 172.21.15.119:0/2977310984 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:23.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff055f9c700 1 --2- 172.21.15.119:0/2977310984 >> v2:172.21.15.119:3300/0 conn(0x7ff0500ccf90 0x7ff0500c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:23.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff054f9a700 1 --2- 172.21.15.119:0/2977310984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 0x7ff0500ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:23.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff047fff700 1 -- 172.21.15.119:0/2977310984 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898584646 0 0) 0x7ff0500cc300 con 0x7ff0500cbf30 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff047fff700 1 -- 172.21.15.119:0/2977310984 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff03c003570 con 0x7ff0500cbf30 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff055f9c700 1 -- 172.21.15.119:0/2977310984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 msgr2=0x7ff0500ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff055f9c700 1 --2- 172.21.15.119:0/2977310984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 0x7ff0500ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff055f9c700 1 -- 172.21.15.119:0/2977310984 >> v1:172.21.15.119:6789/0 conn(0x7ff0500cbf30 legacy=0x7ff0500b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.575+0000 7ff055f9c700 1 -- 172.21.15.119:0/2977310984 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0500ad370 con 0x7ff0500ccf90 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff055f9c700 1 --2- 172.21.15.119:0/2977310984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500ccf90 0x7ff0500c9740 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff048009e90 tx=0x7ff048009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d80b538a41c64d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff047fff700 1 -- 172.21.15.119:0/2977310984 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04800e070 con 0x7ff0500ccf90 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff047fff700 1 -- 172.21.15.119:0/2977310984 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04800bb40 con 0x7ff0500ccf90 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff047fff700 1 -- 172.21.15.119:0/2977310984 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff048012430 con 0x7ff0500ccf90 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff0579ff700 1 -- 172.21.15.119:0/2977310984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500ccf90 msgr2=0x7ff0500c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:23.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff0579ff700 1 --2- 172.21.15.119:0/2977310984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500ccf90 0x7ff0500c9740 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff048009e90 tx=0x7ff048009780 comp rx=0 tx=0).stop 2023-10-08T02:31:23.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff0579ff700 1 -- 172.21.15.119:0/2977310984 shutdown_connections 2023-10-08T02:31:23.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff0579ff700 1 --2- 172.21.15.119:0/2977310984 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500ccf90 0x7ff0500c9740 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff0579ff700 1 --2- 172.21.15.119:0/2977310984 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 0x7ff0500ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.576+0000 7ff0579ff700 1 -- 172.21.15.119:0/2977310984 >> 172.21.15.119:0/2977310984 conn(0x7ff05006a320 msgr2=0x7ff0500cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:23.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.577+0000 7ff0579ff700 1 -- 172.21.15.119:0/2977310984 shutdown_connections 2023-10-08T02:31:23.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.577+0000 7ff0579ff700 1 -- 172.21.15.119:0/2977310984 wait complete. 2023-10-08T02:31:23.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.578+0000 7ff0579ff700 1 Processor -- start 2023-10-08T02:31:23.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.578+0000 7ff0579ff700 1 -- start start 2023-10-08T02:31:23.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff0579ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 0x7ff0500a88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:23.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff0579ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500cbf30 0x7ff0500a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff0579ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0500ccf90 0x7ff050159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff0579ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500aaa30 con 0x7ff0500cbf30 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff0579ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0501544e0 con 0x7ff0500ccf90 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff0579ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0500abf80 con 0x7ff0500b1420 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff054f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500cbf30 0x7ff0500a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff054f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500cbf30 0x7ff0500a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59604/0 (socket says 172.21.15.119:59604) 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff054f9a700 1 -- 172.21.15.119:0/109294394 learned_addr learned my addr 172.21.15.119:0/109294394 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff05579b700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 0x7ff0500a88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.579+0000 7ff055f9c700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0500ccf90 0x7ff050159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff054f9a700 1 -- 172.21.15.119:0/109294394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 msgr2=0x7ff0500a88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:23.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff054f9a700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 0x7ff0500a88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff054f9a700 1 -- 172.21.15.119:0/109294394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0500ccf90 msgr2=0x7ff050159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff054f9a700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0500ccf90 0x7ff050159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff054f9a700 1 -- 172.21.15.119:0/109294394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048009a20 con 0x7ff0500cbf30 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff054f9a700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500cbf30 0x7ff0500a6510 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7ff04000ec80 tx=0x7ff0400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff0457fa700 1 -- 172.21.15.119:0/109294394 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000c850 con 0x7ff0500cbf30 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff0457fa700 1 -- 172.21.15.119:0/109294394 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff04000c9b0 con 0x7ff0500cbf30 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff0457fa700 1 -- 172.21.15.119:0/109294394 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff04000cd50 con 0x7ff0500cbf30 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0500ac410 con 0x7ff0500cbf30 2023-10-08T02:31:23.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.580+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0500ab650 con 0x7ff0500cbf30 2023-10-08T02:31:23.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.581+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05004c3c0 con 0x7ff0500cbf30 2023-10-08T02:31:23.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.582+0000 7ff0457fa700 1 -- 172.21.15.119:0/109294394 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff040013680 con 0x7ff0500cbf30 2023-10-08T02:31:23.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.583+0000 7ff0457fa700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff034066f30 0x7ff0340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:23.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.583+0000 7ff0457fa700 1 -- 172.21.15.119:0/109294394 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5373+0+0 (secure 0 0 0) 0x7ff040084710 con 0x7ff0500cbf30 2023-10-08T02:31:23.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.583+0000 7ff05579b700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff034066f30 0x7ff0340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:23.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.584+0000 7ff05579b700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff034066f30 0x7ff0340693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff04c007fd0 tx=0x7ff04c009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:23.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.587+0000 7ff0457fa700 1 -- 172.21.15.119:0/109294394 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff040051a30 con 0x7ff0500cbf30 2023-10-08T02:31:23.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.852+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff0500ad650 con 0x7ff0500cbf30 2023-10-08T02:31:23.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.852+0000 7ff0457fa700 1 -- 172.21.15.119:0/109294394 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v51) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff040055050 con 0x7ff0500cbf30 2023-10-08T02:31:23.853 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:23.853 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:31:23.853 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-10-08T02:31:23.853 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlxABCJBAAcSfwegC5nL8f0N16/o8/kA==", 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.854 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlypYMORAAhSXCnoBst6up8aEPpiBs/A==", 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlhqefLhAAEI7HDWk8MnSx/uVj+eChLw==", 2023-10-08T02:31:23.855 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlXGPFBxAAbquugbqEY1KV1kd2ROt3sQ==", 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:31:23.856 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDa/SFlLp8yORAA+w+nKxGPup7Vra3Lb1Xwrg==", 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.857 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDd/SFlmuOFOxAAHJ7e5HPh5XxSd3too/rnpw==", 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.858 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDh/SFl8ToLAhAAax2wN7IKjWex9t7yIWmdRw==", 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDk/SFlnMmDBRAAiHdxSCVocJryscNt6rD7CA==", 2023-10-08T02:31:23.859 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDq/SFlKC59FBAAhTwqxPBd8/84QSuJxMBOUw==", 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.860 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDt/SFlmDm+FhAAs5lytmEk8Bz79P9jdxcUhw==", 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.861 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDw/SFlPvXOGBAAGc9d0t9FiOfh9AKKyIjvjA==", 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDz/SFl/sQDHBAAUBj6k55UOsduYY330q3inA==", 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.862 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFl65aXEBAAwrAMe/0+LYkkcyyPpk1TGQ==", 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-10-08T02:31:23.863 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFld4EqHhAAxBp/EliKLHkDLPYF8glVtw==", 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.864 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlZ1RDFxAA0/fmZb/dYYGF41BuiXPkow==", 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-10-08T02:31:23.865 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDX/SFlNUXsJBAAYzpYaudrpKUCnyZYzsnsdw==", 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDV/SFl5RSWLRAAX8upAb0h7hi//+KXrNkcRg==", 2023-10-08T02:31:23.866 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFljj1nHRAASrHrvgO4LmLgbRsf5aC0Uw==", 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:31:23.867 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQDW/SFlld63FRAA3nvX8abxbHATUIMSXuwZvg==", 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.868 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:23.869 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-10-08T02:31:23.869 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:31:23.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.854+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff034066f30 msgr2=0x7ff0340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:23.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.854+0000 7ff0579ff700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff034066f30 0x7ff0340693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff04c007fd0 tx=0x7ff04c009040 comp rx=0 tx=0).stop 2023-10-08T02:31:23.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.854+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500cbf30 msgr2=0x7ff0500a6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:23.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.854+0000 7ff0579ff700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500cbf30 0x7ff0500a6510 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7ff04000ec80 tx=0x7ff0400091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.855+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 shutdown_connections 2023-10-08T02:31:23.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.855+0000 7ff0579ff700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff034066f30 0x7ff0340693e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.855+0000 7ff0579ff700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff0500b1420 0x7ff0500a88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.855+0000 7ff0579ff700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff0500ccf90 0x7ff050159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.855+0000 7ff0579ff700 1 --2- 172.21.15.119:0/109294394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0500cbf30 0x7ff0500a6510 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:23.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.855+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 >> 172.21.15.119:0/109294394 conn(0x7ff05006a320 msgr2=0x7ff0500afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:23.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.855+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 shutdown_connections 2023-10-08T02:31:23.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:23.855+0000 7ff0579ff700 1 -- 172.21.15.119:0/109294394 wait complete. 2023-10-08T02:31:23.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:24.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.054+0000 7f2d4e3ff700 1 Processor -- start 2023-10-08T02:31:24.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.054+0000 7f2d4e3ff700 1 -- start start 2023-10-08T02:31:24.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.055+0000 7f2d4e3ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f2d4806bcd0 0x7f2d480caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.055+0000 7f2d4e3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4815d7e0 con 0x7f2d4806c0a0 2023-10-08T02:31:24.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.055+0000 7f2d4e3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d4815dac0 con 0x7f2d4815df50 2023-10-08T02:31:24.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.055+0000 7f2d4e3ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d4815aa30 con 0x7f2d4806bcd0 2023-10-08T02:31:24.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d477fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2d4806c0a0 0x7f2d4815a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40836/0 (socket says 172.21.15.119:40836) 2023-10-08T02:31:24.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d477fe700 1 -- 172.21.15.119:0/3470760041 learned_addr learned my addr 172.21.15.119:0/3470760041 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:24.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d47fff700 1 --2- 172.21.15.119:0/3470760041 >> v2:172.21.15.184:3300/0 conn(0x7f2d4806bcd0 0x7f2d480caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d46ffd700 1 -- 172.21.15.119:0/3470760041 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563963925 0 0) 0x7f2d4815d7e0 con 0x7f2d4806c0a0 2023-10-08T02:31:24.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d46ffd700 1 -- 172.21.15.119:0/3470760041 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d30003570 con 0x7f2d4806c0a0 2023-10-08T02:31:24.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d47fff700 1 -- 172.21.15.119:0/3470760041 >> v1:172.21.15.184:6789/0 conn(0x7f2d4815df50 legacy=0x7f2d480c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d46ffd700 1 -- 172.21.15.119:0/3470760041 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4118264528 0 0) 0x7f2d30003570 con 0x7f2d4806c0a0 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d47fff700 1 -- 172.21.15.119:0/3470760041 >> v1:172.21.15.119:6789/0 conn(0x7f2d4806c0a0 legacy=0x7f2d4815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.056+0000 7f2d47fff700 1 -- 172.21.15.119:0/3470760041 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4815ad10 con 0x7f2d4806bcd0 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.057+0000 7f2d47fff700 1 --2- 172.21.15.119:0/3470760041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 0x7f2d480caa90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2d380098e0 tx=0x7f2d380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65038a3a02ec18c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.057+0000 7f2d46ffd700 1 -- 172.21.15.119:0/3470760041 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c001470 con 0x7f2d4806bcd0 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.057+0000 7f2d46ffd700 1 -- 172.21.15.119:0/3470760041 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d38011420 con 0x7f2d4806bcd0 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.057+0000 7f2d46ffd700 1 -- 172.21.15.119:0/3470760041 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380117a0 con 0x7f2d4806bcd0 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.057+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3470760041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 msgr2=0x7f2d480caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.057+0000 7f2d4e3ff700 1 --2- 172.21.15.119:0/3470760041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 0x7f2d480caa90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2d380098e0 tx=0x7f2d380091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.058+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3470760041 shutdown_connections 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.058+0000 7f2d4e3ff700 1 --2- 172.21.15.119:0/3470760041 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 0x7f2d480caa90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.058+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3470760041 >> 172.21.15.119:0/3470760041 conn(0x7f2d480ce600 msgr2=0x7f2d480cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:24.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.058+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3470760041 shutdown_connections 2023-10-08T02:31:24.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.058+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3470760041 wait complete. 2023-10-08T02:31:24.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.059+0000 7f2d4e3ff700 1 Processor -- start 2023-10-08T02:31:24.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.059+0000 7f2d4e3ff700 1 -- start start 2023-10-08T02:31:24.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4e3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 0x7f2d480b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4e3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d4806c0a0 0x7f2d48169a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4e3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4815df50 0x7f2d48157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4e3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480b3cc0 con 0x7f2d4815df50 2023-10-08T02:31:24.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4e3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480b2dd0 con 0x7f2d4806bcd0 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4e3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480b30b0 con 0x7f2d4806c0a0 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4c99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4815df50 0x7f2d48157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4c99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4815df50 0x7f2d48157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59606/0 (socket says 172.21.15.119:59606) 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d4c99c700 1 -- 172.21.15.119:0/3714702590 learned_addr learned my addr 172.21.15.119:0/3714702590 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d47fff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 0x7f2d480b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.060+0000 7f2d477fe700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d4806c0a0 0x7f2d48169a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d4c99c700 1 -- 172.21.15.119:0/3714702590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d4806c0a0 msgr2=0x7f2d48169a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d4c99c700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d4806c0a0 0x7f2d48169a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d4c99c700 1 -- 172.21.15.119:0/3714702590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 msgr2=0x7f2d480b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d4c99c700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 0x7f2d480b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d4c99c700 1 -- 172.21.15.119:0/3714702590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d38009420 con 0x7f2d4815df50 2023-10-08T02:31:24.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d4c99c700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4815df50 0x7f2d48157500 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c840 tx=0x7f2d40001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d2ffff700 1 -- 172.21.15.119:0/3714702590 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40012630 con 0x7f2d4815df50 2023-10-08T02:31:24.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d2ffff700 1 -- 172.21.15.119:0/3714702590 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d400127b0 con 0x7f2d4815df50 2023-10-08T02:31:24.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d480b3390 con 0x7f2d4815df50 2023-10-08T02:31:24.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d2ffff700 1 -- 172.21.15.119:0/3714702590 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d40012b70 con 0x7f2d4815df50 2023-10-08T02:31:24.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.061+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d480b3950 con 0x7f2d4815df50 2023-10-08T02:31:24.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.063+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d480c9990 con 0x7f2d4815df50 2023-10-08T02:31:24.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.063+0000 7f2d2ffff700 1 -- 172.21.15.119:0/3714702590 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2d4001a3a0 con 0x7f2d4815df50 2023-10-08T02:31:24.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.064+0000 7f2d2ffff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2d34066f30 0x7f2d340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.064+0000 7f2d2ffff700 1 -- 172.21.15.119:0/3714702590 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f2d400844c0 con 0x7f2d4815df50 2023-10-08T02:31:24.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.064+0000 7f2d47fff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2d34066f30 0x7f2d340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.065+0000 7f2d47fff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2d34066f30 0x7f2d340693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009fd0 tx=0x7f2d38016000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.068+0000 7f2d2ffff700 1 -- 172.21.15.119:0/3714702590 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2d40051880 con 0x7f2d4815df50 2023-10-08T02:31:24.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.259+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d4804c3c0 con 0x7f2d4815df50 2023-10-08T02:31:24.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.260+0000 7f2d2ffff700 1 -- 172.21.15.119:0/3714702590 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+12996 (secure 0 0 0) 0x7f2d40054ea0 con 0x7f2d4815df50 2023-10-08T02:31:24.261 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:24.261 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":202,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:23.363448+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":189,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:24.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2d34066f30 msgr2=0x7f2d340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2d34066f30 0x7f2d340693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2d38009fd0 tx=0x7f2d38016000 comp rx=0 tx=0).stop 2023-10-08T02:31:24.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4815df50 msgr2=0x7f2d48157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4815df50 0x7f2d48157500 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c840 tx=0x7f2d40001670 comp rx=0 tx=0).stop 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 shutdown_connections 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2d34066f30 0x7f2d340693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d4815df50 0x7f2d48157500 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2d4806bcd0 0x7f2d480b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 --2- 172.21.15.119:0/3714702590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2d4806c0a0 0x7f2d48169a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.262+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 >> 172.21.15.119:0/3714702590 conn(0x7f2d480ce600 msgr2=0x7f2d48158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.263+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 shutdown_connections 2023-10-08T02:31:24.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.263+0000 7f2d4e3ff700 1 -- 172.21.15.119:0/3714702590 wait complete. 2023-10-08T02:31:24.281 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2023-10-08T02:31:24.281 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:31:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.466+0000 7f1ae09ff700 1 Processor -- start 2023-10-08T02:31:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.466+0000 7f1ae09ff700 1 -- start start 2023-10-08T02:31:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ae09ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1adc06b160 0x7f1adc06b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ae09ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc06a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ae09ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1adc0ac450 con 0x7f1adc0637d0 2023-10-08T02:31:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ae09ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1adc0693d0 con 0x7f1adc06b160 2023-10-08T02:31:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ae09ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc069d00 con 0x7f1adc06a2b0 2023-10-08T02:31:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ada59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1adc06b160 0x7f1adc06b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ada59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1adc06b160 0x7f1adc06b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59610/0 (socket says 172.21.15.119:59610) 2023-10-08T02:31:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ada59c700 1 -- 172.21.15.119:0/1114486104 learned_addr learned my addr 172.21.15.119:0/1114486104 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1adad9d700 1 --2- 172.21.15.119:0/1114486104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc06a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ada59c700 1 -- 172.21.15.119:0/1114486104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 msgr2=0x7f1adc06a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ada59c700 1 --2- 172.21.15.119:0/1114486104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc06a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ada59c700 1 -- 172.21.15.119:0/1114486104 >> v1:172.21.15.184:6789/0 conn(0x7f1adc0637d0 legacy=0x7f1adc068cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1ada59c700 1 -- 172.21.15.119:0/1114486104 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc068830 con 0x7f1adc06b160 2023-10-08T02:31:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.468+0000 7f1adad9d700 1 --2- 172.21.15.119:0/1114486104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc06a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.469+0000 7f1ada59c700 1 --2- 172.21.15.119:0/1114486104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc06b160 0x7f1adc06b530 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0099b0 tx=0x7f1acc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753037860c5b5f2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.469+0000 7f1ad959a700 1 -- 172.21.15.119:0/1114486104 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00b860 con 0x7f1adc06b160 2023-10-08T02:31:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.469+0000 7f1ad959a700 1 -- 172.21.15.119:0/1114486104 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc00b9c0 con 0x7f1adc06b160 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.469+0000 7f1ad959a700 1 -- 172.21.15.119:0/1114486104 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc00bd80 con 0x7f1adc06b160 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.469+0000 7f1ae09ff700 1 -- 172.21.15.119:0/1114486104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc06b160 msgr2=0x7f1adc06b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.469+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/1114486104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc06b160 0x7f1adc06b530 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0099b0 tx=0x7f1acc009250 comp rx=0 tx=0).stop 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.470+0000 7f1ae09ff700 1 -- 172.21.15.119:0/1114486104 shutdown_connections 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.470+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/1114486104 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc06a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.470+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/1114486104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc06b160 0x7f1adc06b530 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.470+0000 7f1ae09ff700 1 -- 172.21.15.119:0/1114486104 >> 172.21.15.119:0/1114486104 conn(0x7f1adc06ded0 msgr2=0x7f1adc06e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.470+0000 7f1ae09ff700 1 -- 172.21.15.119:0/1114486104 shutdown_connections 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.470+0000 7f1ae09ff700 1 -- 172.21.15.119:0/1114486104 wait complete. 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.471+0000 7f1ae09ff700 1 Processor -- start 2023-10-08T02:31:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.471+0000 7f1ae09ff700 1 -- start start 2023-10-08T02:31:24.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ae09ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc0637d0 0x7f1adc14e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ae09ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ae09ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1adc06b160 0x7f1adc1505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ae09ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc16d640 con 0x7f1adc0637d0 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ae09ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc16dea0 con 0x7f1adc06b160 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ae09ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1adc16adc0 con 0x7f1adc06a2b0 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ada59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc0637d0 0x7f1adc14e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ada59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc0637d0 0x7f1adc14e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59614/0 (socket says 172.21.15.119:59614) 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ada59c700 1 -- 172.21.15.119:0/919104423 learned_addr learned my addr 172.21.15.119:0/919104423 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1ad9d9b700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.472+0000 7f1adad9d700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1adc06b160 0x7f1adc1505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1ada59c700 1 -- 172.21.15.119:0/919104423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 msgr2=0x7f1adc150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1ada59c700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1ada59c700 1 -- 172.21.15.119:0/919104423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1adc06b160 msgr2=0x7f1adc1505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1ada59c700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1adc06b160 0x7f1adc1505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1ada59c700 1 -- 172.21.15.119:0/919104423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1acc0094f0 con 0x7f1adc0637d0 2023-10-08T02:31:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1ada59c700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc0637d0 0x7f1adc14e2f0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00b3c0 tx=0x7f1acc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1aca7fc700 1 -- 172.21.15.119:0/919104423 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc01f070 con 0x7f1adc0637d0 2023-10-08T02:31:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1aca7fc700 1 -- 172.21.15.119:0/919104423 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1acc011d20 con 0x7f1adc0637d0 2023-10-08T02:31:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1adc16d850 con 0x7f1adc0637d0 2023-10-08T02:31:24.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.473+0000 7f1aca7fc700 1 -- 172.21.15.119:0/919104423 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc01a670 con 0x7f1adc0637d0 2023-10-08T02:31:24.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.474+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1adc14e830 con 0x7f1adc0637d0 2023-10-08T02:31:24.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.475+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc04c3c0 con 0x7f1adc0637d0 2023-10-08T02:31:24.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.475+0000 7f1aca7fc700 1 -- 172.21.15.119:0/919104423 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1acc01aa20 con 0x7f1adc0637d0 2023-10-08T02:31:24.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.476+0000 7f1aca7fc700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1abc066f30 0x7f1abc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.477+0000 7f1aca7fc700 1 -- 172.21.15.119:0/919104423 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f1acc0858e0 con 0x7f1adc0637d0 2023-10-08T02:31:24.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.477+0000 7f1ad9d9b700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1abc066f30 0x7f1abc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.477+0000 7f1ad9d9b700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1abc066f30 0x7f1abc0693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1adc157450 tx=0x7f1ad0007ec0 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.481+0000 7f1aca7fc700 1 -- 172.21.15.119:0/919104423 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1acc052d20 con 0x7f1adc0637d0 2023-10-08T02:31:24.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.670+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1adc069740 con 0x7f1adc0637d0 2023-10-08T02:31:24.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.671+0000 7f1aca7fc700 1 -- 172.21.15.119:0/919104423 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v202) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1acc0280a0 con 0x7f1adc0637d0 2023-10-08T02:31:24.672 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:31:24.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1abc066f30 msgr2=0x7f1abc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1abc066f30 0x7f1abc0693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1adc157450 tx=0x7f1ad0007ec0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc0637d0 msgr2=0x7f1adc14e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc0637d0 0x7f1adc14e2f0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f1acc00b3c0 tx=0x7f1acc017000 comp rx=0 tx=0).stop 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 shutdown_connections 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1abc066f30 0x7f1abc0693e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1adc06b160 0x7f1adc1505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1adc0637d0 0x7f1adc14e2f0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 --2- 172.21.15.119:0/919104423 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f1adc06a2b0 0x7f1adc150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.673+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 >> 172.21.15.119:0/919104423 conn(0x7f1adc06ded0 msgr2=0x7f1adc15b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.674+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 shutdown_connections 2023-10-08T02:31:24.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.674+0000 7f1ae09ff700 1 -- 172.21.15.119:0/919104423 wait complete. 2023-10-08T02:31:24.693 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-08T02:31:24.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.874+0000 7fd1bb59e700 1 Processor -- start 2023-10-08T02:31:24.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.874+0000 7fd1bb59e700 1 -- start start 2023-10-08T02:31:24.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.875+0000 7fd1bb59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1bc150180 0x7fd1bc157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.875+0000 7fd1bb59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1bc06e2a0 con 0x7fd1bc0c7410 2023-10-08T02:31:24.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.875+0000 7fd1bb59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1bc06b2e0 con 0x7fd1bc150550 2023-10-08T02:31:24.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.875+0000 7fd1bb59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1bc0c78e0 con 0x7fd1bc150180 2023-10-08T02:31:24.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.876+0000 7fd1ba59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1bc150180 0x7fd1bc157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.876+0000 7fd1bad9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd1bc0c7410 0x7fd1bc069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47028/0 (socket says 172.21.15.119:47028) 2023-10-08T02:31:24.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.876+0000 7fd1bad9d700 1 -- 172.21.15.119:0/1909402885 learned_addr learned my addr 172.21.15.119:0/1909402885 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:24.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.876+0000 7fd1b959a700 1 -- 172.21.15.119:0/1909402885 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941374389 0 0) 0x7fd1bc06e2a0 con 0x7fd1bc0c7410 2023-10-08T02:31:24.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.876+0000 7fd1ba59c700 1 -- 172.21.15.119:0/1909402885 >> v1:172.21.15.184:6789/0 conn(0x7fd1bc150550 legacy=0x7fd1bc0c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.876+0000 7fd1ba59c700 1 -- 172.21.15.119:0/1909402885 >> v1:172.21.15.119:6789/0 conn(0x7fd1bc0c7410 legacy=0x7fd1bc069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.876+0000 7fd1ba59c700 1 -- 172.21.15.119:0/1909402885 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1bc0c7bc0 con 0x7fd1bc150180 2023-10-08T02:31:24.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.876+0000 7fd1ba59c700 1 --2- 172.21.15.119:0/1909402885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150180 0x7fd1bc157770 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00aa90 tx=0x7fd1ac00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caa0ac0cca83dd8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1b959a700 1 -- 172.21.15.119:0/1909402885 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac00ca40 con 0x7fd1bc150180 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1b959a700 1 -- 172.21.15.119:0/1909402885 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ac00cba0 con 0x7fd1bc150180 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1b959a700 1 -- 172.21.15.119:0/1909402885 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ac018400 con 0x7fd1bc150180 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1bb59e700 1 -- 172.21.15.119:0/1909402885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150180 msgr2=0x7fd1bc157770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1bb59e700 1 --2- 172.21.15.119:0/1909402885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150180 0x7fd1bc157770 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00aa90 tx=0x7fd1ac00a330 comp rx=0 tx=0).stop 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1bb59e700 1 -- 172.21.15.119:0/1909402885 shutdown_connections 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1bb59e700 1 --2- 172.21.15.119:0/1909402885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150180 0x7fd1bc157770 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1bb59e700 1 -- 172.21.15.119:0/1909402885 >> 172.21.15.119:0/1909402885 conn(0x7fd1bc0cc1f0 msgr2=0x7fd1bc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1bb59e700 1 -- 172.21.15.119:0/1909402885 shutdown_connections 2023-10-08T02:31:24.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.877+0000 7fd1bb59e700 1 -- 172.21.15.119:0/1909402885 wait complete. 2023-10-08T02:31:24.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.879+0000 7fd1bb59e700 1 Processor -- start 2023-10-08T02:31:24.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.879+0000 7fd1bb59e700 1 -- start start 2023-10-08T02:31:24.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bb59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1bc0c7410 0x7fd1bc154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bb59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1bc150180 0x7fd1bc13a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bb59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150550 0x7fd1bc13ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bb59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc0c3020 con 0x7fd1bc150550 2023-10-08T02:31:24.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bb59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc0c1f80 con 0x7fd1bc150180 2023-10-08T02:31:24.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bb59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1bc0c2260 con 0x7fd1bc0c7410 2023-10-08T02:31:24.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150550 0x7fd1bc13ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150550 0x7fd1bc13ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57432/0 (socket says 172.21.15.119:57432) 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1bad9d700 1 -- 172.21.15.119:0/2043700973 learned_addr learned my addr 172.21.15.119:0/2043700973 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1ba59c700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1bc0c7410 0x7fd1bc154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.880+0000 7fd1b9d9b700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1bc150180 0x7fd1bc13a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1bad9d700 1 -- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1bc0c7410 msgr2=0x7fd1bc154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1bad9d700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1bc0c7410 0x7fd1bc154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1bad9d700 1 -- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1bc150180 msgr2=0x7fd1bc13a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1bad9d700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1bc150180 0x7fd1bc13a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1bad9d700 1 -- 172.21.15.119:0/2043700973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac00a5d0 con 0x7fd1bc150550 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1b9d9b700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1bc150180 0x7fd1bc13a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1bad9d700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150550 0x7fd1bc13ad30 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400ccc0 tx=0x7fd1b4001a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1aa7fc700 1 -- 172.21.15.119:0/2043700973 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400a930 con 0x7fd1bc150550 2023-10-08T02:31:24.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1aa7fc700 1 -- 172.21.15.119:0/2043700973 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b400aa90 con 0x7fd1bc150550 2023-10-08T02:31:24.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1aa7fc700 1 -- 172.21.15.119:0/2043700973 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b400ae30 con 0x7fd1bc150550 2023-10-08T02:31:24.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1bc0c2540 con 0x7fd1bc150550 2023-10-08T02:31:24.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.881+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1bc0c2b00 con 0x7fd1bc150550 2023-10-08T02:31:24.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.882+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1bc0c2d20 con 0x7fd1bc150550 2023-10-08T02:31:24.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.883+0000 7fd1aa7fc700 1 -- 172.21.15.119:0/2043700973 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1b4007300 con 0x7fd1bc150550 2023-10-08T02:31:24.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.884+0000 7fd1aa7fc700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd19c066f30 0x7fd19c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:24.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.884+0000 7fd1aa7fc700 1 -- 172.21.15.119:0/2043700973 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5373+0+0 (secure 0 0 0) 0x7fd1b4084830 con 0x7fd1bc150550 2023-10-08T02:31:24.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.884+0000 7fd1ba59c700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd19c066f30 0x7fd19c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:24.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.885+0000 7fd1ba59c700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd19c066f30 0x7fd19c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00a5a0 tx=0x7fd1ac01a000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:24.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:24.888+0000 7fd1aa7fc700 1 -- 172.21.15.119:0/2043700973 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1b4051bd0 con 0x7fd1bc150550 2023-10-08T02:31:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.080+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd1bc04c3c0 con 0x7fd1bc150550 2023-10-08T02:31:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.081+0000 7fd1aa7fc700 1 -- 172.21.15.119:0/2043700973 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v202) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd1b40551f0 con 0x7fd1bc150550 2023-10-08T02:31:25.082 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:25.082 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T02:31:25.082 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:25.082 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-10-08T02:31:25.082 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-10-08T02:31:25.082 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:25.082 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T02:31:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.083+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd19c066f30 msgr2=0x7fd19c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.083+0000 7fd1bb59e700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd19c066f30 0x7fd19c0693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac00a5a0 tx=0x7fd1ac01a000 comp rx=0 tx=0).stop 2023-10-08T02:31:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.083+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150550 msgr2=0x7fd1bc13ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.083+0000 7fd1bb59e700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150550 0x7fd1bc13ad30 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7fd1b400ccc0 tx=0x7fd1b4001a20 comp rx=0 tx=0).stop 2023-10-08T02:31:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.083+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 shutdown_connections 2023-10-08T02:31:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.083+0000 7fd1bb59e700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd19c066f30 0x7fd19c0693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.084+0000 7fd1bb59e700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1bc0c7410 0x7fd1bc154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.084+0000 7fd1bb59e700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1bc150550 0x7fd1bc13ad30 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.084+0000 7fd1bb59e700 1 --2- 172.21.15.119:0/2043700973 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1bc150180 0x7fd1bc13a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.084+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 >> 172.21.15.119:0/2043700973 conn(0x7fd1bc0cc1f0 msgr2=0x7fd1bc1562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.084+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 shutdown_connections 2023-10-08T02:31:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.084+0000 7fd1bb59e700 1 -- 172.21.15.119:0/2043700973 wait complete. 2023-10-08T02:31:25.102 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-08T02:31:25.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-08T02:31:25.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.287+0000 7f6e4e9ff700 1 Processor -- start 2023-10-08T02:31:25.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.287+0000 7f6e4e9ff700 1 -- start start 2023-10-08T02:31:25.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.288+0000 7f6e4e9ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e481563b0 0x7f6e48156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:25.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.288+0000 7f6e4e9ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6e4815a440 0x7f6e480acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:25.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.288+0000 7f6e4e9ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4815a9b0 con 0x7f6e48068890 2023-10-08T02:31:25.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.288+0000 7f6e4e9ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4815ac90 con 0x7f6e481563b0 2023-10-08T02:31:25.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.288+0000 7f6e4e9ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4815af70 con 0x7f6e4815a440 2023-10-08T02:31:25.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e47fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e481563b0 0x7f6e48156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:25.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e47fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6e481563b0 0x7f6e48156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57440/0 (socket says 172.21.15.119:57440) 2023-10-08T02:31:25.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e47fff700 1 -- 172.21.15.119:0/1556582244 learned_addr learned my addr 172.21.15.119:0/1556582244 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:25.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e477fe700 1 --2- 172.21.15.119:0/1556582244 >> v2:172.21.15.184:3300/0 conn(0x7f6e4815a440 0x7f6e480acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:25.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e47fff700 1 -- 172.21.15.119:0/1556582244 >> v2:172.21.15.184:3300/0 conn(0x7f6e4815a440 msgr2=0x7f6e480acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e47fff700 1 --2- 172.21.15.119:0/1556582244 >> v2:172.21.15.184:3300/0 conn(0x7f6e4815a440 0x7f6e480acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e47fff700 1 -- 172.21.15.119:0/1556582244 >> v1:172.21.15.184:6789/0 conn(0x7f6e48068890 legacy=0x7f6e480c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e47fff700 1 -- 172.21.15.119:0/1556582244 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4815b250 con 0x7f6e481563b0 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e477fe700 1 --2- 172.21.15.119:0/1556582244 >> v2:172.21.15.184:3300/0 conn(0x7f6e4815a440 0x7f6e480acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.289+0000 7f6e47fff700 1 --2- 172.21.15.119:0/1556582244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e481563b0 0x7f6e48156780 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6e4806fb10 tx=0x7f6e38009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7390304a308f5ee server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e46ffd700 1 -- 172.21.15.119:0/1556582244 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800b860 con 0x7f6e481563b0 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e46ffd700 1 -- 172.21.15.119:0/1556582244 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3800b9c0 con 0x7f6e481563b0 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e46ffd700 1 -- 172.21.15.119:0/1556582244 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3800bd60 con 0x7f6e481563b0 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/1556582244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e481563b0 msgr2=0x7f6e48156780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:25.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/1556582244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e481563b0 0x7f6e48156780 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f6e4806fb10 tx=0x7f6e38009250 comp rx=0 tx=0).stop 2023-10-08T02:31:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/1556582244 shutdown_connections 2023-10-08T02:31:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/1556582244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e481563b0 0x7f6e48156780 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/1556582244 >> v2:172.21.15.184:3300/0 conn(0x7f6e4815a440 0x7f6e480acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/1556582244 >> 172.21.15.119:0/1556582244 conn(0x7f6e480c8f90 msgr2=0x7f6e480c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/1556582244 shutdown_connections 2023-10-08T02:31:25.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.290+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/1556582244 wait complete. 2023-10-08T02:31:25.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.292+0000 7f6e4e9ff700 1 Processor -- start 2023-10-08T02:31:25.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.292+0000 7f6e4e9ff700 1 -- start start 2023-10-08T02:31:25.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e4e9ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e48068890 0x7f6e48151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:25.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e4e9ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e481563b0 0x7f6e4814f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e4e9ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e4815a440 0x7f6e4814ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e4e9ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4814f2a0 con 0x7f6e48068890 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e4e9ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4816b050 con 0x7f6e481563b0 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e4e9ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4816dea0 con 0x7f6e4815a440 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e47fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e48068890 0x7f6e48151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e47fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e48068890 0x7f6e48151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57448/0 (socket says 172.21.15.119:57448) 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e47fff700 1 -- 172.21.15.119:0/2460608208 learned_addr learned my addr 172.21.15.119:0/2460608208 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e477fe700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e481563b0 0x7f6e4814f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e4cf9c700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e4815a440 0x7f6e4814ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e47fff700 1 -- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e4815a440 msgr2=0x7f6e4814ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:25.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e47fff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e4815a440 0x7f6e4814ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e47fff700 1 -- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e481563b0 msgr2=0x7f6e4814f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e47fff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e481563b0 0x7f6e4814f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e47fff700 1 -- 172.21.15.119:0/2460608208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e380094f0 con 0x7f6e48068890 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.293+0000 7f6e4cf9c700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e4815a440 0x7f6e4814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.294+0000 7f6e47fff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e48068890 0x7f6e48151c30 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009290 tx=0x7f6e38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.294+0000 7f6e2ffff700 1 -- 172.21.15.119:0/2460608208 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e38011e60 con 0x7f6e48068890 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.294+0000 7f6e2ffff700 1 -- 172.21.15.119:0/2460608208 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3801a660 con 0x7f6e48068890 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.294+0000 7f6e2ffff700 1 -- 172.21.15.119:0/2460608208 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3801aa20 con 0x7f6e48068890 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.294+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e48152170 con 0x7f6e48068890 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.294+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4816d850 con 0x7f6e48068890 2023-10-08T02:31:25.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.295+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4816e180 con 0x7f6e48068890 2023-10-08T02:31:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.296+0000 7f6e2ffff700 1 -- 172.21.15.119:0/2460608208 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e3801b430 con 0x7f6e48068890 2023-10-08T02:31:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.297+0000 7f6e2ffff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6e34066f30 0x7f6e340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:25.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.297+0000 7f6e2ffff700 1 -- 172.21.15.119:0/2460608208 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5373+0+0 (secure 0 0 0) 0x7f6e38085bf0 con 0x7f6e48068890 2023-10-08T02:31:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.297+0000 7f6e477fe700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6e34066f30 0x7f6e340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:25.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.298+0000 7f6e477fe700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6e34066f30 0x7f6e340693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c006f00 tx=0x7f6e3c008040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:25.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.301+0000 7f6e2ffff700 1 -- 172.21.15.119:0/2460608208 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6e38052f10 con 0x7f6e48068890 2023-10-08T02:31:25.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:25.494+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6e480718b0 con 0x7f6e48068890 2023-10-08T02:31:26.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.378+0000 7f6e2ffff700 1 -- 172.21.15.119:0/2460608208 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v203) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6e380150b0 con 0x7f6e48068890 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6e34066f30 msgr2=0x7f6e340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6e34066f30 0x7f6e340693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c006f00 tx=0x7f6e3c008040 comp rx=0 tx=0).stop 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e48068890 msgr2=0x7f6e48151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e48068890 0x7f6e48151c30 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009290 tx=0x7f6e38017000 comp rx=0 tx=0).stop 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 shutdown_connections 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6e34066f30 0x7f6e340693e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6e4815a440 0x7f6e4814ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6e48068890 0x7f6e48151c30 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 --2- 172.21.15.119:0/2460608208 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6e481563b0 0x7f6e4814f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 >> 172.21.15.119:0/2460608208 conn(0x7f6e480c8f90 msgr2=0x7f6e4806ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:26.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.381+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 shutdown_connections 2023-10-08T02:31:26.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.382+0000 7f6e4e9ff700 1 -- 172.21.15.119:0/2460608208 wait complete. 2023-10-08T02:31:26.386 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-10-08T02:31:26.406 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-08T02:31:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.594+0000 7f37b359e700 1 Processor -- start 2023-10-08T02:31:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.595+0000 7f37b359e700 1 -- start start 2023-10-08T02:31:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37b40caee0 0x7f37b40ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b359e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b40cbe50 con 0x7f37b41563b0 2023-10-08T02:31:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b359e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b40cb370 con 0x7f37b4156780 2023-10-08T02:31:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40cb650 con 0x7f37b40caee0 2023-10-08T02:31:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b259c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f37b41563b0 0x7f37b40ca7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47034/0 (socket says 172.21.15.119:47034) 2023-10-08T02:31:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b259c700 1 -- 172.21.15.119:0/2814930554 learned_addr learned my addr 172.21.15.119:0/2814930554 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37abfff700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37b40caee0 0x7f37b40ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781048612 0 0) 0x7f37b40cbe50 con 0x7f37b41563b0 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a0003570 con 0x7f37b41563b0 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075903119 0 0) 0x7f37b40cb370 con 0x7f37b4156780 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37b40cbe50 con 0x7f37b4156780 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 961131031 0 0) 0x7f37a0003570 con 0x7f37b41563b0 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37b40caee0 msgr2=0x7f37b40ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37b40caee0 0x7f37b40ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 >> v1:172.21.15.184:6789/0 conn(0x7f37b4156780 legacy=0x7f37b40c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40c6860 con 0x7f37b41563b0 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f37a4001470 con 0x7f37b41563b0 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 >> v1:172.21.15.119:6789/0 conn(0x7f37b41563b0 legacy=0x7f37b40ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0003c20 0x7f37a0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37b40caee0 0x7f37a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a0004d50 0x7f37a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40cb650 con 0x7f37a0004d50 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a000bbf0 con 0x7f37a0003c20 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a000bdb0 con 0x7f37b40caee0 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b2d9d700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a0004d50 0x7f37a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37b259c700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0003c20 0x7f37a0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.597+0000 7f37abfff700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37b40caee0 0x7f37a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b2d9d700 1 -- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37b40caee0 msgr2=0x7f37a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b2d9d700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37b40caee0 0x7f37a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b2d9d700 1 -- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0003c20 msgr2=0x7f37a0004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b2d9d700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37a0003c20 0x7f37a0004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b2d9d700 1 -- 172.21.15.119:0/2814930554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a0003570 con 0x7f37a0004d50 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b2d9d700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a0004d50 0x7f37a00051c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f37ac002120 tx=0x7f37ac0017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6f0fc90331388d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac001ab0 con 0x7f37a0004d50 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac00c980 con 0x7f37a0004d50 2023-10-08T02:31:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b1d9b700 1 -- 172.21.15.119:0/2814930554 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac00cd20 con 0x7f37a0004d50 2023-10-08T02:31:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b359e700 1 -- 172.21.15.119:0/2814930554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a0004d50 msgr2=0x7f37a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b359e700 1 --2- 172.21.15.119:0/2814930554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37a0004d50 0x7f37a00051c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f37ac002120 tx=0x7f37ac0017a0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.598+0000 7f37b2d9d700 1 -- 172.21.15.119:0/2814930554 reap_dead start 2023-10-08T02:31:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.599+0000 7f37b359e700 1 -- 172.21.15.119:0/2814930554 shutdown_connections 2023-10-08T02:31:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.599+0000 7f37b359e700 1 -- 172.21.15.119:0/2814930554 >> 172.21.15.119:0/2814930554 conn(0x7f37b4068e20 msgr2=0x7f37b40cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.599+0000 7f37b359e700 1 -- 172.21.15.119:0/2814930554 shutdown_connections 2023-10-08T02:31:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.599+0000 7f37b359e700 1 -- 172.21.15.119:0/2814930554 wait complete. 2023-10-08T02:31:26.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.600+0000 7f37b359e700 1 Processor -- start 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.600+0000 7f37b359e700 1 -- start start 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0004d50 0x7f37b4152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b415fc30 0x7f37b40b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37b40ac9e0 0x7f37b40ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b0b00 con 0x7f37b40ac9e0 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4152b70 con 0x7f37b415fc30 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40b0230 con 0x7f37a0004d50 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37b40ac9e0 0x7f37b40ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b2d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37b40ac9e0 0x7f37b40ad0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57470/0 (socket says 172.21.15.119:57470) 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.601+0000 7f37b2d9d700 1 -- 172.21.15.119:0/262268363 learned_addr learned my addr 172.21.15.119:0/262268363 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37abfff700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b415fc30 0x7f37b40b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b259c700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0004d50 0x7f37b4152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b2d9d700 1 -- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0004d50 msgr2=0x7f37b4152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b2d9d700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0004d50 0x7f37b4152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b2d9d700 1 -- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b415fc30 msgr2=0x7f37b40b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b2d9d700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b415fc30 0x7f37b40b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b2d9d700 1 -- 172.21.15.119:0/262268363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40abc20 con 0x7f37b40ac9e0 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37abfff700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b415fc30 0x7f37b40b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b2d9d700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37b40ac9e0 0x7f37b40ad0f0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f37ac012960 tx=0x7f37ac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37aa7fc700 1 -- 172.21.15.119:0/262268363 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac001400 con 0x7f37b40ac9e0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37aa7fc700 1 -- 172.21.15.119:0/262268363 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ac006ba0 con 0x7f37b40ac9e0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37aa7fc700 1 -- 172.21.15.119:0/262268363 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ac01f440 con 0x7f37b40ac9e0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b40b04b0 con 0x7f37b40ac9e0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.602+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b415f8b0 con 0x7f37b40ac9e0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.603+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b40b0cb0 con 0x7f37b40ac9e0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.604+0000 7f37aa7fc700 1 -- 172.21.15.119:0/262268363 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37ac003030 con 0x7f37b40ac9e0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.605+0000 7f37aa7fc700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3794066f30 0x7f37940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.605+0000 7f37aa7fc700 1 -- 172.21.15.119:0/262268363 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5743+0+0 (secure 0 0 0) 0x7f37ac085c60 con 0x7f37b40ac9e0 2023-10-08T02:31:26.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.605+0000 7f37b259c700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3794066f30 0x7f37940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:26.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.606+0000 7f37b259c700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3794066f30 0x7f37940693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f37a4001bb0 tx=0x7f37a4009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:26.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.611+0000 7f37aa7fc700 1 -- 172.21.15.119:0/262268363 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37b40b0cb0 con 0x7f37b40ac9e0 2023-10-08T02:31:26.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:26.801+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f37b404c3c0 con 0x7f37b40ac9e0 2023-10-08T02:31:27.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.390+0000 7f37aa7fc700 1 -- 172.21.15.119:0/262268363 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v204) v1 ==== 170+0+0 (secure 0 0 0) 0x7f37ac0568a0 con 0x7f37b40ac9e0 2023-10-08T02:31:27.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.400+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3794066f30 msgr2=0x7f37940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:27.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3794066f30 0x7f37940693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f37a4001bb0 tx=0x7f37a4009040 comp rx=0 tx=0).stop 2023-10-08T02:31:27.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37b40ac9e0 msgr2=0x7f37b40ad0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:27.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37b40ac9e0 0x7f37b40ad0f0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f37ac012960 tx=0x7f37ac016000 comp rx=0 tx=0).stop 2023-10-08T02:31:27.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 shutdown_connections 2023-10-08T02:31:27.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3794066f30 0x7f37940693e0 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f37a4001bb0 tx=0x7f37a4009040 comp rx=0 tx=0).stop 2023-10-08T02:31:27.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37a0004d50 0x7f37b4152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:27.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37b40ac9e0 0x7f37b40ad0f0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:27.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 --2- 172.21.15.119:0/262268363 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37b415fc30 0x7f37b40b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:27.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.401+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 >> 172.21.15.119:0/262268363 conn(0x7f37b4068e20 msgr2=0x7f37b40c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:27.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.403+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 shutdown_connections 2023-10-08T02:31:27.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.403+0000 7f37b359e700 1 -- 172.21.15.119:0/262268363 wait complete. 2023-10-08T02:31:27.406 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-10-08T02:31:27.424 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-08T02:31:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.609+0000 7fa36f5ff700 1 Processor -- start 2023-10-08T02:31:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.609+0000 7fa36f5ff700 1 -- start start 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36f5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3680c83f0 0x7fa3680c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36f5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa3681568e0 0x7fa3680ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36f5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36815b0d0 con 0x7fa3680ccee0 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36f5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3680c7a60 con 0x7fa3681568e0 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36f5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36815a4c0 con 0x7fa3680c83f0 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36db9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa3681568e0 0x7fa3680ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36cb9a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa3680ccee0 0x7fa3681561d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47050/0 (socket says 172.21.15.119:47050) 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36cb9a700 1 -- 172.21.15.119:0/2166686942 learned_addr learned my addr 172.21.15.119:0/2166686942 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.612+0000 7fa36d39b700 1 --2- 172.21.15.119:0/2166686942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3680c83f0 0x7fa3680c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa36db9c700 1 -- 172.21.15.119:0/2166686942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3680c83f0 msgr2=0x7fa3680c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa36db9c700 1 --2- 172.21.15.119:0/2166686942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3680c83f0 0x7fa3680c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa36db9c700 1 -- 172.21.15.119:0/2166686942 >> v1:172.21.15.119:6789/0 conn(0x7fa3680ccee0 legacy=0x7fa3681561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa36db9c700 1 -- 172.21.15.119:0/2166686942 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36815a7a0 con 0x7fa3681568e0 2023-10-08T02:31:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa35ffff700 1 -- 172.21.15.119:0/2166686942 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910501906 0 0) 0x7fa36815b0d0 con 0x7fa3680ccee0 2023-10-08T02:31:27.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa36db9c700 1 --2- 172.21.15.119:0/2166686942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3681568e0 0x7fa3680ca1e0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fa360009e60 tx=0x7fa360009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0589a0e907559e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa35ffff700 1 -- 172.21.15.119:0/2166686942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000e070 con 0x7fa3681568e0 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa35ffff700 1 -- 172.21.15.119:0/2166686942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000bb60 con 0x7fa3681568e0 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.613+0000 7fa35ffff700 1 -- 172.21.15.119:0/2166686942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360012430 con 0x7fa3681568e0 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 -- 172.21.15.119:0/2166686942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3681568e0 msgr2=0x7fa3680ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/2166686942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3681568e0 0x7fa3680ca1e0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fa360009e60 tx=0x7fa360009780 comp rx=0 tx=0).stop 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 -- 172.21.15.119:0/2166686942 shutdown_connections 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/2166686942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3681568e0 0x7fa3680ca1e0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/2166686942 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3680c83f0 0x7fa3680c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 -- 172.21.15.119:0/2166686942 >> 172.21.15.119:0/2166686942 conn(0x7fa368068e10 msgr2=0x7fa3680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 -- 172.21.15.119:0/2166686942 shutdown_connections 2023-10-08T02:31:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 -- 172.21.15.119:0/2166686942 wait complete. 2023-10-08T02:31:27.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 Processor -- start 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.614+0000 7fa36f5ff700 1 -- start start 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36f5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3680c83f0 0x7fa36814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36f5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3680ccee0 0x7fa36814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36f5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3681568e0 0x7fa36814dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36f5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680ae780 con 0x7fa3680ccee0 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36f5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3681593c0 con 0x7fa3680c83f0 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36f5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3681596a0 con 0x7fa3681568e0 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3680ccee0 0x7fa36814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3680ccee0 0x7fa36814d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57494/0 (socket says 172.21.15.119:57494) 2023-10-08T02:31:27.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 -- 172.21.15.119:0/847119229 learned_addr learned my addr 172.21.15.119:0/847119229 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36db9c700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3681568e0 0x7fa36814dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 -- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3681568e0 msgr2=0x7fa36814dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36d39b700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3680c83f0 0x7fa36814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3681568e0 0x7fa36814dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 -- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3680c83f0 msgr2=0x7fa36814d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3680c83f0 0x7fa36814d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 -- 172.21.15.119:0/847119229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa360009a20 con 0x7fa3680ccee0 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36db9c700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3681568e0 0x7fa36814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36d39b700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3680c83f0 0x7fa36814d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa36cb9a700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3680ccee0 0x7fa36814d810 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fa35800c7e0 tx=0x7fa358007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:27.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.616+0000 7fa35d7fa700 1 -- 172.21.15.119:0/847119229 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358005b30 con 0x7fa3680ccee0 2023-10-08T02:31:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.618+0000 7fa35d7fa700 1 -- 172.21.15.119:0/847119229 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa358005c90 con 0x7fa3680ccee0 2023-10-08T02:31:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.618+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa368159980 con 0x7fa3680ccee0 2023-10-08T02:31:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.618+0000 7fa35d7fa700 1 -- 172.21.15.119:0/847119229 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358007300 con 0x7fa3680ccee0 2023-10-08T02:31:27.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.618+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3680b1ef0 con 0x7fa3680ccee0 2023-10-08T02:31:27.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.618+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3680b2170 con 0x7fa3680ccee0 2023-10-08T02:31:27.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.620+0000 7fa35d7fa700 1 -- 172.21.15.119:0/847119229 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa358025430 con 0x7fa3680ccee0 2023-10-08T02:31:27.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.621+0000 7fa35d7fa700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa34c066f30 0x7fa34c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:27.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.621+0000 7fa35d7fa700 1 -- 172.21.15.119:0/847119229 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 6125+0+0 (secure 0 0 0) 0x7fa358084c10 con 0x7fa3680ccee0 2023-10-08T02:31:27.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.621+0000 7fa36d39b700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa34c066f30 0x7fa34c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:27.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.621+0000 7fa36d39b700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa34c066f30 0x7fa34c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa364006f00 tx=0x7fa364008040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:27.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.623+0000 7fa35d7fa700 1 -- 172.21.15.119:0/847119229 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa358051ef0 con 0x7fa3680ccee0 2023-10-08T02:31:27.648 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-10-08T02:31:27.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:27.854+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa368159b70 con 0x7fa3680ccee0 2023-10-08T02:31:28.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.423+0000 7fa35d7fa700 1 -- 172.21.15.119:0/847119229 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v92) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa358055510 con 0x7fa3680ccee0 2023-10-08T02:31:28.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa34c066f30 msgr2=0x7fa34c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa34c066f30 0x7fa34c0693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa364006f00 tx=0x7fa364008040 comp rx=0 tx=0).stop 2023-10-08T02:31:28.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3680ccee0 msgr2=0x7fa36814d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3680ccee0 0x7fa36814d810 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fa35800c7e0 tx=0x7fa358007df0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 shutdown_connections 2023-10-08T02:31:28.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa34c066f30 0x7fa34c0693e0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa364006f00 tx=0x7fa364008040 comp rx=0 tx=0).stop 2023-10-08T02:31:28.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa3681568e0 0x7fa36814dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa3680c83f0 0x7fa36814d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 --2- 172.21.15.119:0/847119229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3680ccee0 0x7fa36814d810 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.431+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 >> 172.21.15.119:0/847119229 conn(0x7fa368068e10 msgr2=0x7fa3680c12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:28.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.433+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 shutdown_connections 2023-10-08T02:31:28.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.433+0000 7fa36f5ff700 1 -- 172.21.15.119:0/847119229 wait complete. 2023-10-08T02:31:28.436 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 24 and data pool 25 2023-10-08T02:31:28.456 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:28.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.638+0000 7f86e99ff700 1 Processor -- start 2023-10-08T02:31:28.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.638+0000 7f86e99ff700 1 -- start start 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.639+0000 7f86e99ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f86e40b1420 0x7f86e40ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e99ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e40cc300 con 0x7f86e40ccf90 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e99ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e40ad370 con 0x7f86e40cbf30 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e99ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e4156bf0 con 0x7f86e40b1420 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e37fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f86e40ccf90 0x7f86e40c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47056/0 (socket says 172.21.15.119:47056) 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e37fe700 1 -- 172.21.15.119:0/2757403168 learned_addr learned my addr 172.21.15.119:0/2757403168 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959076701 0 0) 0x7f86e40cc300 con 0x7f86e40ccf90 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86cc003570 con 0x7f86e40ccf90 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e27fc700 1 --2- 172.21.15.119:0/2757403168 >> v2:172.21.15.184:3300/0 conn(0x7f86e40b1420 0x7f86e40ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:28.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.640+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1623203557 0 0) 0x7f86cc003570 con 0x7f86e40ccf90 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 >> v2:172.21.15.184:3300/0 conn(0x7f86e40b1420 msgr2=0x7f86e40ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 --2- 172.21.15.119:0/2757403168 >> v2:172.21.15.184:3300/0 conn(0x7f86e40b1420 0x7f86e40ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 >> v1:172.21.15.184:6789/0 conn(0x7f86e40cbf30 legacy=0x7f86e40b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e40ad650 con 0x7f86e40ccf90 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f86dc0017f0 con 0x7f86e40ccf90 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 >> v1:172.21.15.119:6789/0 conn(0x7f86e40ccf90 legacy=0x7f86e40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc003a60 0x7f86cc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc004810 0x7f86cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc0051c0 0x7f86cc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e4156bf0 con 0x7f86cc0051c0 2023-10-08T02:31:28.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc00c0b0 con 0x7f86cc004810 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86cc00c270 con 0x7f86cc003a60 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e37fe700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc0051c0 0x7f86cc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e27fc700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc004810 0x7f86cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e2ffd700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc003a60 0x7f86cc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e37fe700 1 -- 172.21.15.119:0/2757403168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc003a60 msgr2=0x7f86cc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e37fe700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86cc003a60 0x7f86cc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e37fe700 1 -- 172.21.15.119:0/2757403168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc004810 msgr2=0x7f86cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e37fe700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc004810 0x7f86cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.641+0000 7f86e37fe700 1 -- 172.21.15.119:0/2757403168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86cc003570 con 0x7f86cc0051c0 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e37fe700 1 -- 172.21.15.119:0/2757403168 reap_dead start 2023-10-08T02:31:28.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e37fe700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc0051c0 0x7f86cc00ba60 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00a880 tx=0x7f86dc002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d79fdc1f226f9b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc0070f0 con 0x7f86cc0051c0 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc007270 con 0x7f86cc0051c0 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e1ffb700 1 -- 172.21.15.119:0/2757403168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc007630 con 0x7f86cc0051c0 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e99ff700 1 -- 172.21.15.119:0/2757403168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc0051c0 msgr2=0x7f86cc00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e99ff700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc0051c0 0x7f86cc00ba60 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00a880 tx=0x7f86dc002120 comp rx=0 tx=0).stop 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e99ff700 1 -- 172.21.15.119:0/2757403168 shutdown_connections 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e99ff700 1 --2- 172.21.15.119:0/2757403168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86cc0051c0 0x7f86cc00ba60 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.642+0000 7f86e99ff700 1 -- 172.21.15.119:0/2757403168 >> 172.21.15.119:0/2757403168 conn(0x7f86e406a320 msgr2=0x7f86e40cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.643+0000 7f86e99ff700 1 -- 172.21.15.119:0/2757403168 shutdown_connections 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.643+0000 7f86e99ff700 1 -- 172.21.15.119:0/2757403168 wait complete. 2023-10-08T02:31:28.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.644+0000 7f86e99ff700 1 Processor -- start 2023-10-08T02:31:28.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.644+0000 7f86e99ff700 1 -- start start 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e99ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc0051c0 0x7f86e40c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e99ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86e40c2dc0 0x7f86e40c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e99ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e40c1810 0x7f86e41656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e99ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40c2930 con 0x7f86e40c1810 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e99ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40c1d20 con 0x7f86cc0051c0 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e99ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40c2000 con 0x7f86e40c2dc0 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e40c1810 0x7f86e41656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e37fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e40c1810 0x7f86e41656c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57516/0 (socket says 172.21.15.119:57516) 2023-10-08T02:31:28.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e37fe700 1 -- 172.21.15.119:0/3579929742 learned_addr learned my addr 172.21.15.119:0/3579929742 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e27fc700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86e40c2dc0 0x7f86e40c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.645+0000 7f86e2ffd700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc0051c0 0x7f86e40c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86e37fe700 1 -- 172.21.15.119:0/3579929742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86e40c2dc0 msgr2=0x7f86e40c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86e37fe700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86e40c2dc0 0x7f86e40c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86e37fe700 1 -- 172.21.15.119:0/3579929742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc0051c0 msgr2=0x7f86e40c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86e37fe700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc0051c0 0x7f86e40c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86e37fe700 1 -- 172.21.15.119:0/3579929742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e40c22e0 con 0x7f86e40c1810 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86e37fe700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e40c1810 0x7f86e41656c0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f86dc000f80 tx=0x7f86dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86caffd700 1 -- 172.21.15.119:0/3579929742 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc0075f0 con 0x7f86e40c1810 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86caffd700 1 -- 172.21.15.119:0/3579929742 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc01e430 con 0x7f86e40c1810 2023-10-08T02:31:28.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e40c25c0 con 0x7f86e40c1810 2023-10-08T02:31:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86caffd700 1 -- 172.21.15.119:0/3579929742 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc01e7f0 con 0x7f86e40c1810 2023-10-08T02:31:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.646+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e4165c00 con 0x7f86e40c1810 2023-10-08T02:31:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.647+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e40c2ae0 con 0x7f86e40c1810 2023-10-08T02:31:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.648+0000 7f86caffd700 1 -- 172.21.15.119:0/3579929742 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f86dc01eb60 con 0x7f86e40c1810 2023-10-08T02:31:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.649+0000 7f86caffd700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f86d0066f30 0x7f86d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.649+0000 7f86caffd700 1 -- 172.21.15.119:0/3579929742 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f86dc085f90 con 0x7f86e40c1810 2023-10-08T02:31:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.649+0000 7f86e2ffd700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f86d0066f30 0x7f86d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:28.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.650+0000 7f86e2ffd700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f86d0066f30 0x7f86d00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86d40010e0 tx=0x7f86d4007d10 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:28.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.653+0000 7f86caffd700 1 -- 172.21.15.119:0/3579929742 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f86e40c2ae0 con 0x7f86e40c1810 2023-10-08T02:31:28.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.842+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86e404c3c0 con 0x7f86e40c1810 2023-10-08T02:31:28.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.843+0000 7f86caffd700 1 -- 172.21.15.119:0/3579929742 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) v1 ==== 74+0+16229 (secure 0 0 0) 0x7f86dc052ff0 con 0x7f86e40c1810 2023-10-08T02:31:28.843 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:28.843 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":205,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:27.855556+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:31:25.536431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T02:31:26.843560+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:28.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f86d0066f30 msgr2=0x7f86d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f86d0066f30 0x7f86d00693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f86d40010e0 tx=0x7f86d4007d10 comp rx=0 tx=0).stop 2023-10-08T02:31:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e40c1810 msgr2=0x7f86e41656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e40c1810 0x7f86e41656c0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f86dc000f80 tx=0x7f86dc016000 comp rx=0 tx=0).stop 2023-10-08T02:31:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 shutdown_connections 2023-10-08T02:31:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f86d0066f30 0x7f86d00693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f86e40c1810 0x7f86e41656c0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f86cc0051c0 0x7f86e40c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 --2- 172.21.15.119:0/3579929742 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f86e40c2dc0 0x7f86e40c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:28.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 >> 172.21.15.119:0/3579929742 conn(0x7f86e406a320 msgr2=0x7f86e40c85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:28.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 shutdown_connections 2023-10-08T02:31:28.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:28.845+0000 7f86e99ff700 1 -- 172.21.15.119:0/3579929742 wait complete. 2023-10-08T02:31:28.862 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:29.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.043+0000 7f57227ff700 1 Processor -- start 2023-10-08T02:31:29.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.043+0000 7f57227ff700 1 -- start start 2023-10-08T02:31:29.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.044+0000 7f57227ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f571c155c10 0x7f571c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.044+0000 7f57227ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c0acac0 con 0x7f571c0ac6d0 2023-10-08T02:31:29.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.044+0000 7f57227ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c0ad930 con 0x7f571c0ac230 2023-10-08T02:31:29.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.044+0000 7f57227ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571c0acd20 con 0x7f571c155c10 2023-10-08T02:31:29.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f5720d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f571c0ac6d0 0x7f571c0b0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47066/0 (socket says 172.21.15.119:47066) 2023-10-08T02:31:29.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f5720d9c700 1 -- 172.21.15.119:0/4239466894 learned_addr learned my addr 172.21.15.119:0/4239466894 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:29.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571bfff700 1 --2- 172.21.15.119:0/4239466894 >> v2:172.21.15.184:3300/0 conn(0x7f571c155c10 0x7f571c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830548592 0 0) 0x7f571c0acac0 con 0x7f571c0ac6d0 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5704003570 con 0x7f571c0ac6d0 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900699697 0 0) 0x7f571c0ad930 con 0x7f571c0ac230 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f571c0acac0 con 0x7f571c0ac230 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1399999450 0 0) 0x7f5704003570 con 0x7f571c0ac6d0 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 >> v2:172.21.15.184:3300/0 conn(0x7f571c155c10 msgr2=0x7f571c155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 --2- 172.21.15.119:0/4239466894 >> v2:172.21.15.184:3300/0 conn(0x7f571c155c10 0x7f571c155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 >> v1:172.21.15.184:6789/0 conn(0x7f571c0ac230 legacy=0x7f571c1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0ad000 con 0x7f571c0ac6d0 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f57140017f0 con 0x7f571c0ac6d0 2023-10-08T02:31:29.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.045+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 >> v1:172.21.15.119:6789/0 conn(0x7f571c0ac6d0 legacy=0x7f571c0b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571affd700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5704003bf0 0x7f5704003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571affd700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f5704004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571affd700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57040051c0 0x7f570400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0acd20 con 0x7f5704004810 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570400c0b0 con 0x7f5704003bf0 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570400c270 con 0x7f57040051c0 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571b7fe700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f5704004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f5720d9c700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57040051c0 0x7f570400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571bfff700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5704003bf0 0x7f5704003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571b7fe700 1 -- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57040051c0 msgr2=0x7f570400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571b7fe700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57040051c0 0x7f570400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571b7fe700 1 -- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5704003bf0 msgr2=0x7f5704003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571b7fe700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5704003bf0 0x7f5704003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571b7fe700 1 -- 172.21.15.119:0/4239466894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5704003570 con 0x7f5704004810 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571bfff700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5704003bf0 0x7f5704003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f5720d9c700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f57040051c0 0x7f570400ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f5720d9c700 1 -- 172.21.15.119:0/4239466894 reap_dead start 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.046+0000 7f571b7fe700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f5704004c80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f5710001a10 tx=0x7f5710007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e763ccb1749bbf4b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000c5d0 con 0x7f5704004810 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571000c730 con 0x7f5704004810 2023-10-08T02:31:29.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f571affd700 1 -- 172.21.15.119:0/4239466894 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000cad0 con 0x7f5704004810 2023-10-08T02:31:29.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f57227ff700 1 -- 172.21.15.119:0/4239466894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 msgr2=0x7f5704004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f57227ff700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f5704004c80 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f5710001a10 tx=0x7f5710007d30 comp rx=0 tx=0).stop 2023-10-08T02:31:29.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f57227ff700 1 -- 172.21.15.119:0/4239466894 shutdown_connections 2023-10-08T02:31:29.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f57227ff700 1 --2- 172.21.15.119:0/4239466894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f5704004c80 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f57227ff700 1 -- 172.21.15.119:0/4239466894 >> 172.21.15.119:0/4239466894 conn(0x7f571c06a320 msgr2=0x7f571c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:29.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f57227ff700 1 -- 172.21.15.119:0/4239466894 shutdown_connections 2023-10-08T02:31:29.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.047+0000 7f57227ff700 1 -- 172.21.15.119:0/4239466894 wait complete. 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.049+0000 7f57227ff700 1 Processor -- start 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.049+0000 7f57227ff700 1 -- start start 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f57227ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f571c164480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f57227ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c1342c0 0x7f571c132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f57227ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c133300 0x7f571c142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f57227ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c133770 con 0x7f5704004810 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f57227ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c1421d0 con 0x7f571c1342c0 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f57227ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c141bc0 con 0x7f571c133300 2023-10-08T02:31:29.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f571c164480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f571c164480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57524/0 (socket says 172.21.15.119:57524) 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 -- 172.21.15.119:0/138272693 learned_addr learned my addr 172.21.15.119:0/138272693 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f5720d9c700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c133300 0x7f571c142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571b7fe700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c1342c0 0x7f571c132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 -- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c133300 msgr2=0x7f571c142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c133300 0x7f571c142660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 -- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c1342c0 msgr2=0x7f571c132dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c1342c0 0x7f571c132dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 -- 172.21.15.119:0/138272693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c135da0 con 0x7f5704004810 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571b7fe700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c1342c0 0x7f571c132dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.050+0000 7f571bfff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f571c164480 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f570c0118e0 tx=0x7f570c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:29.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.051+0000 7f5703fff700 1 -- 172.21.15.119:0/138272693 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c011e80 con 0x7f5704004810 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.051+0000 7f5703fff700 1 -- 172.21.15.119:0/138272693 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c00f8e0 con 0x7f5704004810 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.051+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c0aaec0 con 0x7f5704004810 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.051+0000 7f5703fff700 1 -- 172.21.15.119:0/138272693 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c00fc80 con 0x7f5704004810 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.051+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c0ab1a0 con 0x7f5704004810 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.052+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c142020 con 0x7f5704004810 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.053+0000 7f5703fff700 1 -- 172.21.15.119:0/138272693 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f570c015460 con 0x7f5704004810 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.053+0000 7f5703fff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5708066f30 0x7f57080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.054+0000 7f5703fff700 1 -- 172.21.15.119:0/138272693 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f570c084f00 con 0x7f5704004810 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.054+0000 7f571b7fe700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5708066f30 0x7f57080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.055+0000 7f571b7fe700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5708066f30 0x7f57080693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f571c06fc40 tx=0x7f5710015000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:29.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.058+0000 7f5703fff700 1 -- 172.21.15.119:0/138272693 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f570c051fc0 con 0x7f5704004810 2023-10-08T02:31:29.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.250+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f571c141a20 con 0x7f5704004810 2023-10-08T02:31:29.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.251+0000 7f5703fff700 1 -- 172.21.15.119:0/138272693 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) v1 ==== 74+0+16229 (secure 0 0 0) 0x7f571c141a20 con 0x7f5704004810 2023-10-08T02:31:29.251 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:29.251 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":205,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:27.855556+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:31:25.536431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T02:31:26.843560+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:29.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5708066f30 msgr2=0x7f57080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5708066f30 0x7f57080693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f571c06fc40 tx=0x7f5710015000 comp rx=0 tx=0).stop 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 msgr2=0x7f571c164480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f571c164480 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f570c0118e0 tx=0x7f570c008040 comp rx=0 tx=0).stop 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 shutdown_connections 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5708066f30 0x7f57080693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f571c133300 0x7f571c142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5704004810 0x7f571c164480 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 --2- 172.21.15.119:0/138272693 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f571c1342c0 0x7f571c132dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 >> 172.21.15.119:0/138272693 conn(0x7f571c06a320 msgr2=0x7f571c146930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:29.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 shutdown_connections 2023-10-08T02:31:29.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.254+0000 7f57227ff700 1 -- 172.21.15.119:0/138272693 wait complete. 2023-10-08T02:31:29.271 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-08T02:31:29.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.458+0000 7fbc025ff700 1 Processor -- start 2023-10-08T02:31:29.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.458+0000 7fbc025ff700 1 -- start start 2023-10-08T02:31:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.459+0000 7fbc025ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbbfc0ccee0 0x7fbbfc1561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.459+0000 7fbc025ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc15b0d0 con 0x7fbbfc0c87c0 2023-10-08T02:31:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.459+0000 7fbc025ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbfc15a4c0 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.459+0000 7fbc025ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbfc15a7a0 con 0x7fbbfc0ccee0 2023-10-08T02:31:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbbfb7fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbbfc0ccee0 0x7fbbfc1561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbc00b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbbfc0c87c0 0x7fbbfc0ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47068/0 (socket says 172.21.15.119:47068) 2023-10-08T02:31:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbc00b9c700 1 -- 172.21.15.119:0/2031563801 learned_addr learned my addr 172.21.15.119:0/2031563801 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:29.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbbfaffd700 1 -- 172.21.15.119:0/2031563801 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971881921 0 0) 0x7fbbfc15b0d0 con 0x7fbbfc0c87c0 2023-10-08T02:31:29.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbbfaffd700 1 -- 172.21.15.119:0/2031563801 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbe4003570 con 0x7fbbfc0c87c0 2023-10-08T02:31:29.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbbfb7fe700 1 -- 172.21.15.119:0/2031563801 >> v1:172.21.15.184:6789/0 conn(0x7fbbfc0c83f0 legacy=0x7fbbfc0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbbfb7fe700 1 -- 172.21.15.119:0/2031563801 >> v1:172.21.15.119:6789/0 conn(0x7fbbfc0c87c0 legacy=0x7fbbfc0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbbfb7fe700 1 -- 172.21.15.119:0/2031563801 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbfc15aa80 con 0x7fbbfc0ccee0 2023-10-08T02:31:29.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.460+0000 7fbbfb7fe700 1 --2- 172.21.15.119:0/2031563801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0ccee0 0x7fbbfc1561d0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098e0 tx=0x7fbbf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656780a7c6c2972d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:29.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbbfaffd700 1 -- 172.21.15.119:0/2031563801 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000b640 con 0x7fbbfc0ccee0 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbbfaffd700 1 -- 172.21.15.119:0/2031563801 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf000b7a0 con 0x7fbbfc0ccee0 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbbfaffd700 1 -- 172.21.15.119:0/2031563801 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf000bb40 con 0x7fbbfc0ccee0 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbc025ff700 1 -- 172.21.15.119:0/2031563801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0ccee0 msgr2=0x7fbbfc1561d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbc025ff700 1 --2- 172.21.15.119:0/2031563801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0ccee0 0x7fbbfc1561d0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00098e0 tx=0x7fbbf00091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbc025ff700 1 -- 172.21.15.119:0/2031563801 shutdown_connections 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbc025ff700 1 --2- 172.21.15.119:0/2031563801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0ccee0 0x7fbbfc1561d0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbc025ff700 1 -- 172.21.15.119:0/2031563801 >> 172.21.15.119:0/2031563801 conn(0x7fbbfc068e10 msgr2=0x7fbbfc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbc025ff700 1 -- 172.21.15.119:0/2031563801 shutdown_connections 2023-10-08T02:31:29.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.461+0000 7fbc025ff700 1 -- 172.21.15.119:0/2031563801 wait complete. 2023-10-08T02:31:29.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.463+0000 7fbc025ff700 1 Processor -- start 2023-10-08T02:31:29.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.463+0000 7fbc025ff700 1 -- start start 2023-10-08T02:31:29.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbc025ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0c83f0 0x7fbbfc16cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbc025ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbfc0c87c0 0x7fbbfc15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbc025ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbfc0ccee0 0x7fbbfc15bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbc025ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0c1800 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbc025ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0c1ae0 con 0x7fbbfc0c87c0 2023-10-08T02:31:29.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbc025ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0c1dc0 con 0x7fbbfc0ccee0 2023-10-08T02:31:29.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0c83f0 0x7fbbfc16cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbbfbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0c83f0 0x7fbbfc16cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57544/0 (socket says 172.21.15.119:57544) 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.464+0000 7fbbfbfff700 1 -- 172.21.15.119:0/2464356466 learned_addr learned my addr 172.21.15.119:0/2464356466 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbc00b9c700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbfc0ccee0 0x7fbbfc15bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbfb7fe700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbfc0c87c0 0x7fbbfc15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbfbfff700 1 -- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbfc0ccee0 msgr2=0x7fbbfc15bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbfbfff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbfc0ccee0 0x7fbbfc15bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbfbfff700 1 -- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbfc0c87c0 msgr2=0x7fbbfc15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbfbfff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbfc0c87c0 0x7fbbfc15b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbfbfff700 1 -- 172.21.15.119:0/2464356466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf0009420 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbfb7fe700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbfc0c87c0 0x7fbbfc15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbc00b9c700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbfc0ccee0 0x7fbbfc15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:29.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbfbfff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0c83f0 0x7fbbfc16cd00 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c510 tx=0x7fbbec00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:29.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbe3fff700 1 -- 172.21.15.119:0/2464356466 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec011590 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbe3fff700 1 -- 172.21.15.119:0/2464356466 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbec011710 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc0c20a0 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbbe3fff700 1 -- 172.21.15.119:0/2464356466 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbec011ad0 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.465+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc16d240 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.466+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc04c3c0 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.467+0000 7fbbe3fff700 1 -- 172.21.15.119:0/2464356466 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbbec01b3b0 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.468+0000 7fbbe3fff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbbe8066f30 0x7fbbe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:29.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.468+0000 7fbbe3fff700 1 -- 172.21.15.119:0/2464356466 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7fbbec083cc0 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.468+0000 7fbbfb7fe700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbbe8066f30 0x7fbbe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:29.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.469+0000 7fbbfb7fe700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbbe8066f30 0x7fbbe80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc15a020 tx=0x7fbbf0017000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:29.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.472+0000 7fbbe3fff700 1 -- 172.21.15.119:0/2464356466 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbbec050d20 con 0x7fbbfc0c83f0 2023-10-08T02:31:29.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:29.705+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbbfc0af2d0 con 0x7fbbfc0c83f0 2023-10-08T02:31:30.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.422+0000 7fbbe3fff700 1 -- 172.21.15.119:0/2464356466 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v94) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbbec017070 con 0x7fbbfc0c83f0 2023-10-08T02:31:30.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbbe8066f30 msgr2=0x7fbbe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:30.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbbe8066f30 0x7fbbe80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc15a020 tx=0x7fbbf0017000 comp rx=0 tx=0).stop 2023-10-08T02:31:30.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0c83f0 msgr2=0x7fbbfc16cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:30.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0c83f0 0x7fbbfc16cd00 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fbbec00c510 tx=0x7fbbec00c8d0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 shutdown_connections 2023-10-08T02:31:30.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbbe8066f30 0x7fbbe80693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbbfc0ccee0 0x7fbbfc15bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbfc0c83f0 0x7fbbfc16cd00 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 --2- 172.21.15.119:0/2464356466 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbbfc0c87c0 0x7fbbfc15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 >> 172.21.15.119:0/2464356466 conn(0x7fbbfc068e10 msgr2=0x7fbbfc0c9200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:30.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 shutdown_connections 2023-10-08T02:31:30.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.424+0000 7fbc025ff700 1 -- 172.21.15.119:0/2464356466 wait complete. 2023-10-08T02:31:30.442 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-08T02:31:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.628+0000 7f2a523ff700 1 Processor -- start 2023-10-08T02:31:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.628+0000 7f2a523ff700 1 -- start start 2023-10-08T02:31:30.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a523ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 0x7f2a4c1616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:30.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a523ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c15b0d0 con 0x7f2a4c0c83f0 2023-10-08T02:31:30.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a523ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4c15a4c0 con 0x7f2a4c0ccee0 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a523ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c15a7a0 con 0x7f2a4c1568e0 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a4bfff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2a4c0c83f0 0x7f2a4c0c7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47072/0 (socket says 172.21.15.119:47072) 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a4bfff700 1 -- 172.21.15.119:0/1597283165 learned_addr learned my addr 172.21.15.119:0/1597283165 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a5099c700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 0x7f2a4c1616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 297615853 0 0) 0x7f2a4c15b0d0 con 0x7f2a4c0c83f0 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a34003570 con 0x7f2a4c0c83f0 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.630+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965587684 0 0) 0x7f2a4c15a4c0 con 0x7f2a4c0ccee0 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a4c15b0d0 con 0x7f2a4c0ccee0 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3977318010 0 0) 0x7f2a34003570 con 0x7f2a4c0c83f0 2023-10-08T02:31:30.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 msgr2=0x7f2a4c1616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 0x7f2a4c1616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 >> v1:172.21.15.184:6789/0 conn(0x7f2a4c0ccee0 legacy=0x7f2a4c1561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c15aa80 con 0x7f2a4c0c83f0 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f2a3c001470 con 0x7f2a4c0c83f0 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a5099c700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 0x7f2a4c1616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 >> v1:172.21.15.119:6789/0 conn(0x7f2a4c0c83f0 legacy=0x7f2a4c0c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 0x7f2a34003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a34004d50 0x7f2a340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c15a7a0 con 0x7f2a34004160 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3400ddb0 con 0x7f2a34004d50 2023-10-08T02:31:30.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a3400df70 con 0x7f2a4c1568e0 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4bfff700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 0x7f2a34003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a5099c700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a34004d50 0x7f2a340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4b7fe700 1 -- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 msgr2=0x7f2a34003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c1568e0 0x7f2a34003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4b7fe700 1 -- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a34004d50 msgr2=0x7f2a340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a34004d50 0x7f2a340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.631+0000 7f2a4b7fe700 1 -- 172.21.15.119:0/1597283165 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a44009a20 con 0x7f2a34004160 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.632+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a34004810 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f2a40001a10 tx=0x7f2a40007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f127ce5f159c5b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.632+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4000c5d0 con 0x7f2a34004160 2023-10-08T02:31:30.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.632+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4000c730 con 0x7f2a34004160 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.632+0000 7f2a4affd700 1 -- 172.21.15.119:0/1597283165 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4000cad0 con 0x7f2a34004160 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.632+0000 7f2a523ff700 1 -- 172.21.15.119:0/1597283165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 msgr2=0x7f2a34004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.632+0000 7f2a523ff700 1 --2- 172.21.15.119:0/1597283165 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a34004810 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f2a40001a10 tx=0x7f2a40007d30 comp rx=0 tx=0).stop 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.632+0000 7f2a5099c700 1 -- 172.21.15.119:0/1597283165 reap_dead start 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.633+0000 7f2a523ff700 1 -- 172.21.15.119:0/1597283165 shutdown_connections 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.633+0000 7f2a523ff700 1 -- 172.21.15.119:0/1597283165 >> 172.21.15.119:0/1597283165 conn(0x7f2a4c068e10 msgr2=0x7f2a4c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.633+0000 7f2a523ff700 1 -- 172.21.15.119:0/1597283165 shutdown_connections 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.633+0000 7f2a523ff700 1 -- 172.21.15.119:0/1597283165 wait complete. 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.634+0000 7f2a523ff700 1 Processor -- start 2023-10-08T02:31:30.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.634+0000 7f2a523ff700 1 -- start start 2023-10-08T02:31:30.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a523ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a4c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:30.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a523ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4c14d740 0x7f2a4c158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:30.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a523ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c158860 0x7f2a4c14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:30.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a523ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c15a0e0 con 0x7f2a34004160 2023-10-08T02:31:30.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a523ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c159040 con 0x7f2a4c14d740 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a523ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c159320 con 0x7f2a4c158860 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a4bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a4c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a4bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a4c14d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57562/0 (socket says 172.21.15.119:57562) 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.635+0000 7f2a4bfff700 1 -- 172.21.15.119:0/557726109 learned_addr learned my addr 172.21.15.119:0/557726109 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a4bfff700 1 -- 172.21.15.119:0/557726109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c158860 msgr2=0x7f2a4c14db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a4bfff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c158860 0x7f2a4c14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a4bfff700 1 -- 172.21.15.119:0/557726109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4c14d740 msgr2=0x7f2a4c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a4bfff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4c14d740 0x7f2a4c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a4bfff700 1 -- 172.21.15.119:0/557726109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c159600 con 0x7f2a34004160 2023-10-08T02:31:30.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a4bfff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a4c14d200 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c000ff0 tx=0x7f2a4c06e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a33fff700 1 -- 172.21.15.119:0/557726109 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c018030 con 0x7f2a34004160 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a33fff700 1 -- 172.21.15.119:0/557726109 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3c0077d0 con 0x7f2a34004160 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c1598e0 con 0x7f2a34004160 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a33fff700 1 -- 172.21.15.119:0/557726109 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00fad0 con 0x7f2a34004160 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.636+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c158ce0 con 0x7f2a34004160 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.637+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c159e00 con 0x7f2a34004160 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.638+0000 7f2a33fff700 1 -- 172.21.15.119:0/557726109 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2a3c01b070 con 0x7f2a34004160 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.639+0000 7f2a33fff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2a38066f30 0x7f2a380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.639+0000 7f2a33fff700 1 -- 172.21.15.119:0/557726109 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f2a3c019070 con 0x7f2a34004160 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.639+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2a38066f30 0x7f2a380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:30.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.640+0000 7f2a4b7fe700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2a38066f30 0x7f2a380693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c14db10 tx=0x7f2a40015000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:30.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.643+0000 7f2a33fff700 1 -- 172.21.15.119:0/557726109 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2a4c159e00 con 0x7f2a34004160 2023-10-08T02:31:30.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:30.878+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2a4c159e00 con 0x7f2a34004160 2023-10-08T02:31:31.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.423+0000 7f2a33fff700 1 -- 172.21.15.119:0/557726109 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v95) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a3c051600 con 0x7f2a34004160 2023-10-08T02:31:31.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2a38066f30 msgr2=0x7f2a380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:31.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2a38066f30 0x7f2a380693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c14db10 tx=0x7f2a40015000 comp rx=0 tx=0).stop 2023-10-08T02:31:31.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 msgr2=0x7f2a4c14d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a4c14d200 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c000ff0 tx=0x7f2a4c06e870 comp rx=0 tx=0).stop 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 shutdown_connections 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2a38066f30 0x7f2a380693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f2a4c158860 0x7f2a4c14db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a34004160 0x7f2a4c14d200 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 --2- 172.21.15.119:0/557726109 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f2a4c14d740 0x7f2a4c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 >> 172.21.15.119:0/557726109 conn(0x7f2a4c068e10 msgr2=0x7f2a4c0c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 shutdown_connections 2023-10-08T02:31:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.426+0000 7f2a523ff700 1 -- 172.21.15.119:0/557726109 wait complete. 2023-10-08T02:31:31.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:31.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.628+0000 7fae13bff700 1 Processor -- start 2023-10-08T02:31:31.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.629+0000 7fae13bff700 1 -- start start 2023-10-08T02:31:31.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae13bff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae0c155c10 0x7fae0c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae13bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fae0c0ac230 0x7fae0c0ac6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae13bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c0ad930 con 0x7fae0c1568a0 2023-10-08T02:31:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae13bff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c0acd20 con 0x7fae0c155c10 2023-10-08T02:31:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae13bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c0ad000 con 0x7fae0c0ac230 2023-10-08T02:31:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae1199b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae0c155c10 0x7fae0c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae1199b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fae0c155c10 0x7fae0c155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57574/0 (socket says 172.21.15.119:57574) 2023-10-08T02:31:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae1199b700 1 -- 172.21.15.119:0/453086902 learned_addr learned my addr 172.21.15.119:0/453086902 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae1199b700 1 -- 172.21.15.119:0/453086902 >> v2:172.21.15.184:3300/0 conn(0x7fae0c0ac230 msgr2=0x7fae0c0ac6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae1199b700 1 --2- 172.21.15.119:0/453086902 >> v2:172.21.15.184:3300/0 conn(0x7fae0c0ac230 0x7fae0c0ac6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae1199b700 1 -- 172.21.15.119:0/453086902 >> v1:172.21.15.184:6789/0 conn(0x7fae0c1568a0 legacy=0x7fae0c156c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.630+0000 7fae1199b700 1 -- 172.21.15.119:0/453086902 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0ad2e0 con 0x7fae0c155c10 2023-10-08T02:31:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.631+0000 7fae1199b700 1 --2- 172.21.15.119:0/453086902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c155c10 0x7fae0c155fe0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fae0c065bb0 tx=0x7fae08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87419b91abcdb0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.631+0000 7fae10999700 1 -- 172.21.15.119:0/453086902 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800b820 con 0x7fae0c155c10 2023-10-08T02:31:31.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.631+0000 7fae10999700 1 -- 172.21.15.119:0/453086902 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0800b980 con 0x7fae0c155c10 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.631+0000 7fae10999700 1 -- 172.21.15.119:0/453086902 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0800bd20 con 0x7fae0c155c10 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.631+0000 7fae13bff700 1 -- 172.21.15.119:0/453086902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c155c10 msgr2=0x7fae0c155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.631+0000 7fae13bff700 1 --2- 172.21.15.119:0/453086902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c155c10 0x7fae0c155fe0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fae0c065bb0 tx=0x7fae08009250 comp rx=0 tx=0).stop 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.632+0000 7fae13bff700 1 -- 172.21.15.119:0/453086902 shutdown_connections 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.632+0000 7fae13bff700 1 --2- 172.21.15.119:0/453086902 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c155c10 0x7fae0c155fe0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.632+0000 7fae13bff700 1 --2- 172.21.15.119:0/453086902 >> v2:172.21.15.184:3300/0 conn(0x7fae0c0ac230 0x7fae0c0ac6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.632+0000 7fae13bff700 1 -- 172.21.15.119:0/453086902 >> 172.21.15.119:0/453086902 conn(0x7fae0c06a320 msgr2=0x7fae0c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.632+0000 7fae13bff700 1 -- 172.21.15.119:0/453086902 shutdown_connections 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.632+0000 7fae13bff700 1 -- 172.21.15.119:0/453086902 wait complete. 2023-10-08T02:31:31.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.633+0000 7fae13bff700 1 Processor -- start 2023-10-08T02:31:31.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.633+0000 7fae13bff700 1 -- start start 2023-10-08T02:31:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae13bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c0ac230 0x7fae0c0c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae13bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae0c155c10 0x7fae0c14b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae13bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c1568a0 0x7fae0c14bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae13bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c13d810 con 0x7fae0c0ac230 2023-10-08T02:31:31.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae13bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c13cf20 con 0x7fae0c1568a0 2023-10-08T02:31:31.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae13bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c13d1c0 con 0x7fae0c155c10 2023-10-08T02:31:31.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae1199b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c0ac230 0x7fae0c0c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:31.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae1199b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c0ac230 0x7fae0c0c8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57580/0 (socket says 172.21.15.119:57580) 2023-10-08T02:31:31.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.634+0000 7fae1199b700 1 -- 172.21.15.119:0/2173266824 learned_addr learned my addr 172.21.15.119:0/2173266824 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:31.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae1199b700 1 -- 172.21.15.119:0/2173266824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae0c155c10 msgr2=0x7fae0c14b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:31.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae1199b700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae0c155c10 0x7fae0c14b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae1199b700 1 -- 172.21.15.119:0/2173266824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c1568a0 msgr2=0x7fae0c14bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae1199b700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c1568a0 0x7fae0c14bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae1199b700 1 -- 172.21.15.119:0/2173266824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae080094f0 con 0x7fae0c0ac230 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae1199b700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c0ac230 0x7fae0c0c8f80 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fae08009290 tx=0x7fae08016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae01ffb700 1 -- 172.21.15.119:0/2173266824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08026480 con 0x7fae0c0ac230 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae01ffb700 1 -- 172.21.15.119:0/2173266824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0801f070 con 0x7fae0c0ac230 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae01ffb700 1 -- 172.21.15.119:0/2173266824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae08026850 con 0x7fae0c0ac230 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c13d4a0 con 0x7fae0c0ac230 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.635+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c142570 con 0x7fae0c0ac230 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.636+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c04c3c0 con 0x7fae0c0ac230 2023-10-08T02:31:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.637+0000 7fae01ffb700 1 -- 172.21.15.119:0/2173266824 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fae0801a440 con 0x7fae0c0ac230 2023-10-08T02:31:31.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.638+0000 7fae01ffb700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fadf0066f30 0x7fadf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:31.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.638+0000 7fae01ffb700 1 -- 172.21.15.119:0/2173266824 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7fae08021070 con 0x7fae0c0ac230 2023-10-08T02:31:31.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.638+0000 7fae1119a700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fadf0066f30 0x7fadf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:31.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.639+0000 7fae1119a700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fadf0066f30 0x7fadf00693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fadfc006fd0 tx=0x7fadfc006db0 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:31.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.642+0000 7fae01ffb700 1 -- 172.21.15.119:0/2173266824 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fae08053a10 con 0x7fae0c0ac230 2023-10-08T02:31:31.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.879+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae0c0ad5c0 con 0x7fae0c0ac230 2023-10-08T02:31:31.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.880+0000 7fae01ffb700 1 -- 172.21.15.119:0/2173266824 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4618 (secure 0 0 0) 0x7fae08057030 con 0x7fae0c0ac230 2023-10-08T02:31:31.880 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:31.880 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1044912493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1044912493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1044912493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10197,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1564057059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1564057059},{"type":"v1","addr":"172.21.15.119:6833","nonce":1564057059}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2545901849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2545901849},{"type":"v1","addr":"172.21.15.184:6834","nonce":2545901849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:31:27.855883+0000","modified":"2023-10-08T02:31:31.420955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10215":{"gid":10215,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1741909431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1741909431},{"type":"v1","addr":"172.21.15.119:6835","nonce":1741909431}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T02:31:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fadf0066f30 msgr2=0x7fadf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fadf0066f30 0x7fadf00693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fadfc006fd0 tx=0x7fadfc006db0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c0ac230 msgr2=0x7fae0c0c8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c0ac230 0x7fae0c0c8f80 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fae08009290 tx=0x7fae08016000 comp rx=0 tx=0).stop 2023-10-08T02:31:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 shutdown_connections 2023-10-08T02:31:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fadf0066f30 0x7fadf00693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fae0c1568a0 0x7fae0c14bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fae0c0ac230 0x7fae0c0c8f80 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 --2- 172.21.15.119:0/2173266824 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fae0c155c10 0x7fae0c14b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 >> 172.21.15.119:0/2173266824 conn(0x7fae0c06a320 msgr2=0x7fae0c06e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 shutdown_connections 2023-10-08T02:31:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:31.882+0000 7fae13bff700 1 -- 172.21.15.119:0/2173266824 wait complete. 2023-10-08T02:31:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-10-08T02:31:31.899 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:32.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.079+0000 7f5a0cfff700 1 Processor -- start 2023-10-08T02:31:32.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.080+0000 7f5a0cfff700 1 -- start start 2023-10-08T02:31:32.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a0cfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a08155c10 0x7f5a08155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a0cfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5a080ac230 0x7f5a080ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a0cfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5a081568a0 0x7f5a08156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a0cfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080ad930 con 0x7f5a081568a0 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a0cfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080acd20 con 0x7f5a080ac230 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a0cfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080ad000 con 0x7f5a08155c10 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a06d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5a081568a0 0x7f5a08156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a06d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5a081568a0 0x7f5a08156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57584/0 (socket says 172.21.15.119:57584) 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a06d9d700 1 -- 172.21.15.119:0/160596998 learned_addr learned my addr 172.21.15.119:0/160596998 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:32.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a0659c700 1 --2- 172.21.15.119:0/160596998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a08155c10 0x7f5a08155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a05d9b700 1 --2- 172.21.15.119:0/160596998 >> v2:172.21.15.184:3300/0 conn(0x7f5a080ac230 0x7f5a080ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a06d9d700 1 -- 172.21.15.119:0/160596998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a08155c10 msgr2=0x7f5a08155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a06d9d700 1 --2- 172.21.15.119:0/160596998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a08155c10 0x7f5a08155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a06d9d700 1 -- 172.21.15.119:0/160596998 >> v2:172.21.15.184:3300/0 conn(0x7f5a080ac230 msgr2=0x7f5a080ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.081+0000 7f5a06d9d700 1 --2- 172.21.15.119:0/160596998 >> v2:172.21.15.184:3300/0 conn(0x7f5a080ac230 0x7f5a080ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.082+0000 7f5a06d9d700 1 -- 172.21.15.119:0/160596998 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080ad2e0 con 0x7f5a081568a0 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.082+0000 7f5a06d9d700 1 --2- 172.21.15.119:0/160596998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08156cc0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f5a08064210 tx=0x7f5a00009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=192696ba04491b37 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.082+0000 7f5a0559a700 1 -- 172.21.15.119:0/160596998 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0000e070 con 0x7f5a081568a0 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.082+0000 7f5a0559a700 1 -- 172.21.15.119:0/160596998 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0000ba50 con 0x7f5a081568a0 2023-10-08T02:31:32.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.082+0000 7f5a0559a700 1 -- 172.21.15.119:0/160596998 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000092e0 con 0x7f5a081568a0 2023-10-08T02:31:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.082+0000 7f5a0cfff700 1 -- 172.21.15.119:0/160596998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 msgr2=0x7f5a08156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.082+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/160596998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08156cc0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f5a08064210 tx=0x7f5a00009840 comp rx=0 tx=0).stop 2023-10-08T02:31:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.083+0000 7f5a0cfff700 1 -- 172.21.15.119:0/160596998 shutdown_connections 2023-10-08T02:31:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.083+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/160596998 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08156cc0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.083+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/160596998 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a08155c10 0x7f5a08155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.083+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/160596998 >> v2:172.21.15.184:3300/0 conn(0x7f5a080ac230 0x7f5a080ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.083+0000 7f5a0cfff700 1 -- 172.21.15.119:0/160596998 >> 172.21.15.119:0/160596998 conn(0x7f5a0806a320 msgr2=0x7f5a080cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:32.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.083+0000 7f5a0cfff700 1 -- 172.21.15.119:0/160596998 shutdown_connections 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.083+0000 7f5a0cfff700 1 -- 172.21.15.119:0/160596998 wait complete. 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.084+0000 7f5a0cfff700 1 Processor -- start 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.084+0000 7f5a0cfff700 1 -- start start 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a0cfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a080ac230 0x7f5a0814b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a0cfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08155c10 0x7f5a081508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a0cfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a0cfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08151fc0 con 0x7f5a081568a0 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a0cfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a081513b0 con 0x7f5a08155c10 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a0cfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08151690 con 0x7f5a080ac230 2023-10-08T02:31:32.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a06d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a06d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08152b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57594/0 (socket says 172.21.15.119:57594) 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a06d9d700 1 -- 172.21.15.119:0/3520409677 learned_addr learned my addr 172.21.15.119:0/3520409677 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a0659c700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a080ac230 0x7f5a0814b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.085+0000 7f5a05d9b700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08155c10 0x7f5a081508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a06d9d700 1 -- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a080ac230 msgr2=0x7f5a0814b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a06d9d700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a080ac230 0x7f5a0814b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a06d9d700 1 -- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08155c10 msgr2=0x7f5a081508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a06d9d700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08155c10 0x7f5a081508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a06d9d700 1 -- 172.21.15.119:0/3520409677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a00009ae0 con 0x7f5a081568a0 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a05d9b700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08155c10 0x7f5a081508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:32.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a06d9d700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08152b10 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5a08064290 tx=0x7f5a00018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f59f67fc700 1 -- 172.21.15.119:0/3520409677 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000032f0 con 0x7f5a081568a0 2023-10-08T02:31:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f59f67fc700 1 -- 172.21.15.119:0/3520409677 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a00008dd0 con 0x7f5a081568a0 2023-10-08T02:31:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f59f67fc700 1 -- 172.21.15.119:0/3520409677 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0001c660 con 0x7f5a081568a0 2023-10-08T02:31:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a08151970 con 0x7f5a081568a0 2023-10-08T02:31:32.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.086+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0814b590 con 0x7f5a081568a0 2023-10-08T02:31:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.087+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0804c3c0 con 0x7f5a081568a0 2023-10-08T02:31:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.088+0000 7f59f67fc700 1 -- 172.21.15.119:0/3520409677 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5a00003040 con 0x7f5a081568a0 2023-10-08T02:31:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.090+0000 7f59f67fc700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f59e8066f60 0x7f59e8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.090+0000 7f59f67fc700 1 -- 172.21.15.119:0/3520409677 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f5a00085f50 con 0x7f5a081568a0 2023-10-08T02:31:32.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.090+0000 7f5a0659c700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f59e8066f60 0x7f59e8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.091+0000 7f5a0659c700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f59e8066f60 0x7f59e8069410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f59f8006fd0 tx=0x7f59f8008040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.094+0000 7f59f67fc700 1 -- 172.21.15.119:0/3520409677 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5a00052fb0 con 0x7f5a081568a0 2023-10-08T02:31:32.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.288+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a080ad5c0 con 0x7f5a081568a0 2023-10-08T02:31:32.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.289+0000 7f59f67fc700 1 -- 172.21.15.119:0/3520409677 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+16216 (secure 0 0 0) 0x7f5a00009940 con 0x7f5a081568a0 2023-10-08T02:31:32.288 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:32.289 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":206,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:29.401244+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:31:25.536431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T02:31:26.843560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:32.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f59e8066f60 msgr2=0x7f59e8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f59e8066f60 0x7f59e8069410 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f59f8006fd0 tx=0x7f59f8008040 comp rx=0 tx=0).stop 2023-10-08T02:31:32.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 msgr2=0x7f5a08152b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08152b10 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7f5a08064290 tx=0x7f5a00018000 comp rx=0 tx=0).stop 2023-10-08T02:31:32.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 shutdown_connections 2023-10-08T02:31:32.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f59e8066f60 0x7f59e8069410 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5a080ac230 0x7f5a0814b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a081568a0 0x7f5a08152b10 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 --2- 172.21.15.119:0/3520409677 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5a08155c10 0x7f5a081508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 >> 172.21.15.119:0/3520409677 conn(0x7f5a0806a320 msgr2=0x7f5a0806f450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:32.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 shutdown_connections 2023-10-08T02:31:32.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.291+0000 7f5a0cfff700 1 -- 172.21.15.119:0/3520409677 wait complete. 2023-10-08T02:31:32.308 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-08T02:31:32.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:31:32.369 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-10-08T02:31:32.369+0000 7fce2cff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T02:31:32.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.493+0000 7fa19ad9d700 1 Processor -- start 2023-10-08T02:31:32.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.494+0000 7fa19ad9d700 1 -- start start 2023-10-08T02:31:32.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa19ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa194150180 0x7fa194157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa19ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa194150550 0x7fa1940c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa19ad9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19406e2a0 con 0x7fa1940c7240 2023-10-08T02:31:32.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa19ad9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1940c76e0 con 0x7fa194150550 2023-10-08T02:31:32.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa19ad9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19406b2e0 con 0x7fa194150180 2023-10-08T02:31:32.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa19959a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa194150550 0x7fa1940c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa19a59c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa1940c7240 0x7fa194069f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47074/0 (socket says 172.21.15.119:47074) 2023-10-08T02:31:32.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa19a59c700 1 -- 172.21.15.119:0/2240448321 learned_addr learned my addr 172.21.15.119:0/2240448321 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.495+0000 7fa199d9b700 1 --2- 172.21.15.119:0/2240448321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa194150180 0x7fa194157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa19959a700 1 -- 172.21.15.119:0/2240448321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa194150180 msgr2=0x7fa194157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa19959a700 1 --2- 172.21.15.119:0/2240448321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa194150180 0x7fa194157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa19959a700 1 -- 172.21.15.119:0/2240448321 >> v1:172.21.15.119:6789/0 conn(0x7fa1940c7240 legacy=0x7fa194069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa19959a700 1 -- 172.21.15.119:0/2240448321 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1940c78e0 con 0x7fa194150550 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa198d99700 1 -- 172.21.15.119:0/2240448321 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1011495737 0 0) 0x7fa19406e2a0 con 0x7fa1940c7240 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa19959a700 1 --2- 172.21.15.119:0/2240448321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa1940c6d00 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fa1840098e0 tx=0x7fa1840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f466719f11e7a06 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa198d99700 1 -- 172.21.15.119:0/2240448321 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa184009e80 con 0x7fa194150550 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa198d99700 1 -- 172.21.15.119:0/2240448321 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18400b860 con 0x7fa194150550 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.496+0000 7fa198d99700 1 -- 172.21.15.119:0/2240448321 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18400bbe0 con 0x7fa194150550 2023-10-08T02:31:32.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.497+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2240448321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 msgr2=0x7fa1940c6d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.497+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2240448321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa1940c6d00 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fa1840098e0 tx=0x7fa1840091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.497+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2240448321 shutdown_connections 2023-10-08T02:31:32.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.497+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2240448321 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa194150180 0x7fa194157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.497+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2240448321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa1940c6d00 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.497+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2240448321 >> 172.21.15.119:0/2240448321 conn(0x7fa1940cc1f0 msgr2=0x7fa1940cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:32.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.497+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2240448321 shutdown_connections 2023-10-08T02:31:32.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.497+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2240448321 wait complete. 2023-10-08T02:31:32.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.498+0000 7fa19ad9d700 1 Processor -- start 2023-10-08T02:31:32.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.499+0000 7fa19ad9d700 1 -- start start 2023-10-08T02:31:32.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.499+0000 7fa19ad9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940c7240 0x7fa194167940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.499+0000 7fa19ad9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194150180 0x7fa194158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.499+0000 7fa19ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa194158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.499+0000 7fa19ad9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1940b1d30 con 0x7fa194150550 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.499+0000 7fa19ad9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1940b1160 con 0x7fa194150180 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.499+0000 7fa19ad9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1940b1400 con 0x7fa1940c7240 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa194158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa194158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57616/0 (socket says 172.21.15.119:57616) 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 -- 172.21.15.119:0/2017164863 learned_addr learned my addr 172.21.15.119:0/2017164863 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa199d9b700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940c7240 0x7fa194167940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19959a700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194150180 0x7fa194158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 -- 172.21.15.119:0/2017164863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940c7240 msgr2=0x7fa194167940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940c7240 0x7fa194167940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 -- 172.21.15.119:0/2017164863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194150180 msgr2=0x7fa194158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194150180 0x7fa194158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 -- 172.21.15.119:0/2017164863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa184009420 con 0x7fa194150550 2023-10-08T02:31:32.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.500+0000 7fa19a59c700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa194158790 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00cc90 tx=0x7fa18c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.501+0000 7fa189ffb700 1 -- 172.21.15.119:0/2017164863 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00a8b0 con 0x7fa194150550 2023-10-08T02:31:32.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.501+0000 7fa189ffb700 1 -- 172.21.15.119:0/2017164863 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa18c00aa10 con 0x7fa194150550 2023-10-08T02:31:32.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.501+0000 7fa189ffb700 1 -- 172.21.15.119:0/2017164863 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa18c00add0 con 0x7fa194150550 2023-10-08T02:31:32.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.501+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1940b16e0 con 0x7fa194150550 2023-10-08T02:31:32.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.501+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1940c4030 con 0x7fa194150550 2023-10-08T02:31:32.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.502+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1940c49f0 con 0x7fa194150550 2023-10-08T02:31:32.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.503+0000 7fa189ffb700 1 -- 172.21.15.119:0/2017164863 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa18c007300 con 0x7fa194150550 2023-10-08T02:31:32.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.503+0000 7fa189ffb700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa178066f30 0x7fa1780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.504+0000 7fa189ffb700 1 -- 172.21.15.119:0/2017164863 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7fa18c084a60 con 0x7fa194150550 2023-10-08T02:31:32.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.504+0000 7fa199d9b700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa178066f30 0x7fa1780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.504+0000 7fa199d9b700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa178066f30 0x7fa1780693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa190006fd0 tx=0x7fa190008040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.508+0000 7fa189ffb700 1 -- 172.21.15.119:0/2017164863 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa18c051b40 con 0x7fa194150550 2023-10-08T02:31:32.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.696+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa19404c3c0 con 0x7fa178066f30 2023-10-08T02:31:32.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.706+0000 7fa189ffb700 1 -- 172.21.15.119:0/2017164863 <== mgr.10020 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289403 (secure 0 0 0) 0x7fa19404c3c0 con 0x7fa178066f30 2023-10-08T02:31:32.710 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:32.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.710+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa178066f30 msgr2=0x7fa1780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.710+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa178066f30 0x7fa1780693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa190006fd0 tx=0x7fa190008040 comp rx=0 tx=0).stop 2023-10-08T02:31:32.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.710+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 msgr2=0x7fa194158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.710+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa194158790 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00cc90 tx=0x7fa18c001a80 comp rx=0 tx=0).stop 2023-10-08T02:31:32.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.711+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 shutdown_connections 2023-10-08T02:31:32.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.711+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa178066f30 0x7fa1780693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.711+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa1940c7240 0x7fa194167940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.711+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa194150550 0x7fa194158790 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.711+0000 7fa19ad9d700 1 --2- 172.21.15.119:0/2017164863 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa194150180 0x7fa194158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.711+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 >> 172.21.15.119:0/2017164863 conn(0x7fa1940cc1f0 msgr2=0x7fa1940cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:32.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.711+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 shutdown_connections 2023-10-08T02:31:32.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.711+0000 7fa19ad9d700 1 -- 172.21.15.119:0/2017164863 wait complete. 2023-10-08T02:31:32.713 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:31:32.728 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T02:31:31.840572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2469320,"kb_used_data":21384,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752505400,"statfs":{"total":773094113280,"available":770565529600,"internally_reserved":0,"allocated":21897216,"data_stored":11605084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.462699"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.421790+0000","last_change":"2023-10-08T02:31:28.418181+0000","last_active":"2023-10-08T02:31:28.421790+0000","last_peered":"2023-10-08T02:31:28.421790+0000","last_clean":"2023-10-08T02:31:28.421790+0000","last_became_active":"2023-10-08T02:31:28.417928+0000","last_became_peered":"2023-10-08T02:31:28.417928+0000","last_unstale":"2023-10-08T02:31:28.421790+0000","last_undegraded":"2023-10-08T02:31:28.421790+0000","last_fullsized":"2023-10-08T02:31:28.421790+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:29.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427684+0000","last_change":"2023-10-08T02:31:28.426159+0000","last_active":"2023-10-08T02:31:28.427684+0000","last_peered":"2023-10-08T02:31:28.427684+0000","last_clean":"2023-10-08T02:31:28.427684+0000","last_became_active":"2023-10-08T02:31:28.425888+0000","last_became_peered":"2023-10-08T02:31:28.425888+0000","last_unstale":"2023-10-08T02:31:28.427684+0000","last_undegraded":"2023-10-08T02:31:28.427684+0000","last_fullsized":"2023-10-08T02:31:28.427684+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:06:57.465492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428267+0000","last_change":"2023-10-08T02:31:28.423479+0000","last_active":"2023-10-08T02:31:28.428267+0000","last_peered":"2023-10-08T02:31:28.428267+0000","last_clean":"2023-10-08T02:31:28.428267+0000","last_became_active":"2023-10-08T02:31:28.423214+0000","last_became_peered":"2023-10-08T02:31:28.423214+0000","last_unstale":"2023-10-08T02:31:28.428267+0000","last_undegraded":"2023-10-08T02:31:28.428267+0000","last_fullsized":"2023-10-08T02:31:28.428267+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:53.643477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.431042+0000","last_change":"2023-10-08T02:31:28.428427+0000","last_active":"2023-10-08T02:31:28.431042+0000","last_peered":"2023-10-08T02:31:28.431042+0000","last_clean":"2023-10-08T02:31:28.431042+0000","last_became_active":"2023-10-08T02:31:28.428135+0000","last_became_peered":"2023-10-08T02:31:28.428135+0000","last_unstale":"2023-10-08T02:31:28.431042+0000","last_undegraded":"2023-10-08T02:31:28.431042+0000","last_fullsized":"2023-10-08T02:31:28.431042+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:41.222324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.421289+0000","last_change":"2023-10-08T02:31:28.419218+0000","last_active":"2023-10-08T02:31:28.421289+0000","last_peered":"2023-10-08T02:31:28.421289+0000","last_clean":"2023-10-08T02:31:28.421289+0000","last_became_active":"2023-10-08T02:31:28.418913+0000","last_became_peered":"2023-10-08T02:31:28.418913+0000","last_unstale":"2023-10-08T02:31:28.421289+0000","last_undegraded":"2023-10-08T02:31:28.421289+0000","last_fullsized":"2023-10-08T02:31:28.421289+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:53.643477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426550+0000","last_change":"2023-10-08T02:31:28.421154+0000","last_active":"2023-10-08T02:31:28.426550+0000","last_peered":"2023-10-08T02:31:28.426550+0000","last_clean":"2023-10-08T02:31:28.426550+0000","last_became_active":"2023-10-08T02:31:28.420881+0000","last_became_peered":"2023-10-08T02:31:28.420881+0000","last_unstale":"2023-10-08T02:31:28.426550+0000","last_undegraded":"2023-10-08T02:31:28.426550+0000","last_fullsized":"2023-10-08T02:31:28.426550+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:36.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432749+0000","last_change":"2023-10-08T02:31:28.430705+0000","last_active":"2023-10-08T02:31:28.432749+0000","last_peered":"2023-10-08T02:31:28.432749+0000","last_clean":"2023-10-08T02:31:28.432749+0000","last_became_active":"2023-10-08T02:31:28.430452+0000","last_became_peered":"2023-10-08T02:31:28.430452+0000","last_unstale":"2023-10-08T02:31:28.432749+0000","last_undegraded":"2023-10-08T02:31:28.432749+0000","last_fullsized":"2023-10-08T02:31:28.432749+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:29.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428294+0000","last_change":"2023-10-08T02:31:28.416797+0000","last_active":"2023-10-08T02:31:28.428294+0000","last_peered":"2023-10-08T02:31:28.428294+0000","last_clean":"2023-10-08T02:31:28.428294+0000","last_became_active":"2023-10-08T02:31:28.416157+0000","last_became_peered":"2023-10-08T02:31:28.416157+0000","last_unstale":"2023-10-08T02:31:28.428294+0000","last_undegraded":"2023-10-08T02:31:28.428294+0000","last_fullsized":"2023-10-08T02:31:28.428294+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:24.799351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.431318+0000","last_change":"2023-10-08T02:31:28.429084+0000","last_active":"2023-10-08T02:31:28.431318+0000","last_peered":"2023-10-08T02:31:28.431318+0000","last_clean":"2023-10-08T02:31:28.431318+0000","last_became_active":"2023-10-08T02:31:28.428379+0000","last_became_peered":"2023-10-08T02:31:28.428379+0000","last_unstale":"2023-10-08T02:31:28.431318+0000","last_undegraded":"2023-10-08T02:31:28.431318+0000","last_fullsized":"2023-10-08T02:31:28.431318+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.421266+0000","last_change":"2023-10-08T02:31:28.415359+0000","last_active":"2023-10-08T02:31:28.421266+0000","last_peered":"2023-10-08T02:31:28.421266+0000","last_clean":"2023-10-08T02:31:28.421266+0000","last_became_active":"2023-10-08T02:31:28.415111+0000","last_became_peered":"2023-10-08T02:31:28.415111+0000","last_unstale":"2023-10-08T02:31:28.421266+0000","last_undegraded":"2023-10-08T02:31:28.421266+0000","last_fullsized":"2023-10-08T02:31:28.421266+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:51.625087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426920+0000","last_change":"2023-10-08T02:31:28.424096+0000","last_active":"2023-10-08T02:31:28.426920+0000","last_peered":"2023-10-08T02:31:28.426920+0000","last_clean":"2023-10-08T02:31:28.426920+0000","last_became_active":"2023-10-08T02:31:28.423755+0000","last_became_peered":"2023-10-08T02:31:28.423755+0000","last_unstale":"2023-10-08T02:31:28.426920+0000","last_undegraded":"2023-10-08T02:31:28.426920+0000","last_fullsized":"2023-10-08T02:31:28.426920+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:27:23.536896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426249+0000","last_change":"2023-10-08T02:31:28.422555+0000","last_active":"2023-10-08T02:31:28.426249+0000","last_peered":"2023-10-08T02:31:28.426249+0000","last_clean":"2023-10-08T02:31:28.426249+0000","last_became_active":"2023-10-08T02:31:28.422262+0000","last_became_peered":"2023-10-08T02:31:28.422262+0000","last_unstale":"2023-10-08T02:31:28.426249+0000","last_undegraded":"2023-10-08T02:31:28.426249+0000","last_fullsized":"2023-10-08T02:31:28.426249+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:14.758340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.431032+0000","last_change":"2023-10-08T02:31:28.427714+0000","last_active":"2023-10-08T02:31:28.431032+0000","last_peered":"2023-10-08T02:31:28.431032+0000","last_clean":"2023-10-08T02:31:28.431032+0000","last_became_active":"2023-10-08T02:31:28.426640+0000","last_became_peered":"2023-10-08T02:31:28.426640+0000","last_unstale":"2023-10-08T02:31:28.431032+0000","last_undegraded":"2023-10-08T02:31:28.431032+0000","last_fullsized":"2023-10-08T02:31:28.431032+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:31.599689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.430695+0000","last_change":"2023-10-08T02:31:28.428325+0000","last_active":"2023-10-08T02:31:28.430695+0000","last_peered":"2023-10-08T02:31:28.430695+0000","last_clean":"2023-10-08T02:31:28.430695+0000","last_became_active":"2023-10-08T02:31:28.428080+0000","last_became_peered":"2023-10-08T02:31:28.428080+0000","last_unstale":"2023-10-08T02:31:28.430695+0000","last_undegraded":"2023-10-08T02:31:28.430695+0000","last_fullsized":"2023-10-08T02:31:28.430695+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:53.643477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.425951+0000","last_change":"2023-10-08T02:31:28.421565+0000","last_active":"2023-10-08T02:31:28.425951+0000","last_peered":"2023-10-08T02:31:28.425951+0000","last_clean":"2023-10-08T02:31:28.425951+0000","last_became_active":"2023-10-08T02:31:28.421301+0000","last_became_peered":"2023-10-08T02:31:28.421301+0000","last_unstale":"2023-10-08T02:31:28.425951+0000","last_undegraded":"2023-10-08T02:31:28.425951+0000","last_fullsized":"2023-10-08T02:31:28.425951+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:00.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428314+0000","last_change":"2023-10-08T02:31:28.416973+0000","last_active":"2023-10-08T02:31:28.428314+0000","last_peered":"2023-10-08T02:31:28.428314+0000","last_clean":"2023-10-08T02:31:28.428314+0000","last_became_active":"2023-10-08T02:31:28.416527+0000","last_became_peered":"2023-10-08T02:31:28.416527+0000","last_unstale":"2023-10-08T02:31:28.428314+0000","last_undegraded":"2023-10-08T02:31:28.428314+0000","last_fullsized":"2023-10-08T02:31:28.428314+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.421615+0000","last_change":"2023-10-08T02:31:28.419778+0000","last_active":"2023-10-08T02:31:28.421615+0000","last_peered":"2023-10-08T02:31:28.421615+0000","last_clean":"2023-10-08T02:31:28.421615+0000","last_became_active":"2023-10-08T02:31:28.419523+0000","last_became_peered":"2023-10-08T02:31:28.419523+0000","last_unstale":"2023-10-08T02:31:28.421615+0000","last_undegraded":"2023-10-08T02:31:28.421615+0000","last_fullsized":"2023-10-08T02:31:28.421615+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:50:06.141377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432687+0000","last_change":"2023-10-08T02:31:28.429415+0000","last_active":"2023-10-08T02:31:28.432687+0000","last_peered":"2023-10-08T02:31:28.432687+0000","last_clean":"2023-10-08T02:31:28.432687+0000","last_became_active":"2023-10-08T02:31:28.429106+0000","last_became_peered":"2023-10-08T02:31:28.429106+0000","last_unstale":"2023-10-08T02:31:28.432687+0000","last_undegraded":"2023-10-08T02:31:28.432687+0000","last_fullsized":"2023-10-08T02:31:28.432687+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.430270+0000","last_change":"2023-10-08T02:31:28.426900+0000","last_active":"2023-10-08T02:31:28.430270+0000","last_peered":"2023-10-08T02:31:28.430270+0000","last_clean":"2023-10-08T02:31:28.430270+0000","last_became_active":"2023-10-08T02:31:28.426596+0000","last_became_peered":"2023-10-08T02:31:28.426596+0000","last_unstale":"2023-10-08T02:31:28.430270+0000","last_undegraded":"2023-10-08T02:31:28.430270+0000","last_fullsized":"2023-10-08T02:31:28.430270+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:31.096884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.418902+0000","last_change":"2023-10-08T02:31:28.415272+0000","last_active":"2023-10-08T02:31:28.418902+0000","last_peered":"2023-10-08T02:31:28.418902+0000","last_clean":"2023-10-08T02:31:28.418902+0000","last_became_active":"2023-10-08T02:31:28.414998+0000","last_became_peered":"2023-10-08T02:31:28.414998+0000","last_unstale":"2023-10-08T02:31:28.418902+0000","last_undegraded":"2023-10-08T02:31:28.418902+0000","last_fullsized":"2023-10-08T02:31:28.418902+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:24.799351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429153+0000","last_change":"2023-10-08T02:31:28.421213+0000","last_active":"2023-10-08T02:31:28.429153+0000","last_peered":"2023-10-08T02:31:28.429153+0000","last_clean":"2023-10-08T02:31:28.429153+0000","last_became_active":"2023-10-08T02:31:28.420955+0000","last_became_peered":"2023-10-08T02:31:28.420955+0000","last_unstale":"2023-10-08T02:31:28.429153+0000","last_undegraded":"2023-10-08T02:31:28.429153+0000","last_fullsized":"2023-10-08T02:31:28.429153+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:28.907447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429649+0000","last_change":"2023-10-08T02:31:28.426612+0000","last_active":"2023-10-08T02:31:28.429649+0000","last_peered":"2023-10-08T02:31:28.429649+0000","last_clean":"2023-10-08T02:31:28.429649+0000","last_became_active":"2023-10-08T02:31:28.426171+0000","last_became_peered":"2023-10-08T02:31:28.426171+0000","last_unstale":"2023-10-08T02:31:28.429649+0000","last_undegraded":"2023-10-08T02:31:28.429649+0000","last_fullsized":"2023-10-08T02:31:28.429649+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:58.587382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.423638+0000","last_change":"2023-10-08T02:31:28.415019+0000","last_active":"2023-10-08T02:31:28.423638+0000","last_peered":"2023-10-08T02:31:28.423638+0000","last_clean":"2023-10-08T02:31:28.423638+0000","last_became_active":"2023-10-08T02:31:28.414570+0000","last_became_peered":"2023-10-08T02:31:28.414570+0000","last_unstale":"2023-10-08T02:31:28.423638+0000","last_undegraded":"2023-10-08T02:31:28.423638+0000","last_fullsized":"2023-10-08T02:31:28.423638+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:50:06.141377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428331+0000","last_change":"2023-10-08T02:31:28.421401+0000","last_active":"2023-10-08T02:31:28.428331+0000","last_peered":"2023-10-08T02:31:28.428331+0000","last_clean":"2023-10-08T02:31:28.428331+0000","last_became_active":"2023-10-08T02:31:28.421098+0000","last_became_peered":"2023-10-08T02:31:28.421098+0000","last_unstale":"2023-10-08T02:31:28.428331+0000","last_undegraded":"2023-10-08T02:31:28.428331+0000","last_fullsized":"2023-10-08T02:31:28.428331+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:29.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.416251+0000","last_change":"2023-10-08T02:31:28.415479+0000","last_active":"2023-10-08T02:31:28.416251+0000","last_peered":"2023-10-08T02:31:28.416251+0000","last_clean":"2023-10-08T02:31:28.416251+0000","last_became_active":"2023-10-08T02:31:28.415211+0000","last_became_peered":"2023-10-08T02:31:28.415211+0000","last_unstale":"2023-10-08T02:31:28.416251+0000","last_undegraded":"2023-10-08T02:31:28.416251+0000","last_fullsized":"2023-10-08T02:31:28.416251+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:03.504965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432774+0000","last_change":"2023-10-08T02:31:28.429954+0000","last_active":"2023-10-08T02:31:28.432774+0000","last_peered":"2023-10-08T02:31:28.432774+0000","last_clean":"2023-10-08T02:31:28.432774+0000","last_became_active":"2023-10-08T02:31:28.429702+0000","last_became_peered":"2023-10-08T02:31:28.429702+0000","last_unstale":"2023-10-08T02:31:28.432774+0000","last_undegraded":"2023-10-08T02:31:28.432774+0000","last_fullsized":"2023-10-08T02:31:28.432774+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:09.691655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426936+0000","last_change":"2023-10-08T02:31:28.421777+0000","last_active":"2023-10-08T02:31:28.426936+0000","last_peered":"2023-10-08T02:31:28.426936+0000","last_clean":"2023-10-08T02:31:28.426936+0000","last_became_active":"2023-10-08T02:31:28.421533+0000","last_became_peered":"2023-10-08T02:31:28.421533+0000","last_unstale":"2023-10-08T02:31:28.426936+0000","last_undegraded":"2023-10-08T02:31:28.426936+0000","last_fullsized":"2023-10-08T02:31:28.426936+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:42.759676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.417368+0000","last_change":"2023-10-08T02:31:28.413558+0000","last_active":"2023-10-08T02:31:28.417368+0000","last_peered":"2023-10-08T02:31:28.417368+0000","last_clean":"2023-10-08T02:31:28.417368+0000","last_became_active":"2023-10-08T02:31:28.412818+0000","last_became_peered":"2023-10-08T02:31:28.412818+0000","last_unstale":"2023-10-08T02:31:28.417368+0000","last_undegraded":"2023-10-08T02:31:28.417368+0000","last_fullsized":"2023-10-08T02:31:28.417368+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:31:38.361725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426220+0000","last_change":"2023-10-08T02:31:28.420178+0000","last_active":"2023-10-08T02:31:28.426220+0000","last_peered":"2023-10-08T02:31:28.426220+0000","last_clean":"2023-10-08T02:31:28.426220+0000","last_became_active":"2023-10-08T02:31:28.419924+0000","last_became_peered":"2023-10-08T02:31:28.419924+0000","last_unstale":"2023-10-08T02:31:28.426220+0000","last_undegraded":"2023-10-08T02:31:28.426220+0000","last_fullsized":"2023-10-08T02:31:28.426220+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:57.286618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427183+0000","last_change":"2023-10-08T02:31:28.420611+0000","last_active":"2023-10-08T02:31:28.427183+0000","last_peered":"2023-10-08T02:31:28.427183+0000","last_clean":"2023-10-08T02:31:28.427183+0000","last_became_active":"2023-10-08T02:31:28.420326+0000","last_became_peered":"2023-10-08T02:31:28.420326+0000","last_unstale":"2023-10-08T02:31:28.427183+0000","last_undegraded":"2023-10-08T02:31:28.427183+0000","last_fullsized":"2023-10-08T02:31:28.427183+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:50:06.141377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428406+0000","last_change":"2023-10-08T02:31:28.423324+0000","last_active":"2023-10-08T02:31:28.428406+0000","last_peered":"2023-10-08T02:31:28.428406+0000","last_clean":"2023-10-08T02:31:28.428406+0000","last_became_active":"2023-10-08T02:31:28.423071+0000","last_became_peered":"2023-10-08T02:31:28.423071+0000","last_unstale":"2023-10-08T02:31:28.428406+0000","last_undegraded":"2023-10-08T02:31:28.428406+0000","last_fullsized":"2023-10-08T02:31:28.428406+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:07.013855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.418443+0000","last_change":"2023-10-08T02:31:28.412836+0000","last_active":"2023-10-08T02:31:28.418443+0000","last_peered":"2023-10-08T02:31:28.418443+0000","last_clean":"2023-10-08T02:31:28.418443+0000","last_became_active":"2023-10-08T02:31:28.412574+0000","last_became_peered":"2023-10-08T02:31:28.412574+0000","last_unstale":"2023-10-08T02:31:28.418443+0000","last_undegraded":"2023-10-08T02:31:28.418443+0000","last_fullsized":"2023-10-08T02:31:28.418443+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:36.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427690+0000","last_change":"2023-10-08T02:31:28.422490+0000","last_active":"2023-10-08T02:31:28.427690+0000","last_peered":"2023-10-08T02:31:28.427690+0000","last_clean":"2023-10-08T02:31:28.427690+0000","last_became_active":"2023-10-08T02:31:28.422232+0000","last_became_peered":"2023-10-08T02:31:28.422232+0000","last_unstale":"2023-10-08T02:31:28.427690+0000","last_undegraded":"2023-10-08T02:31:28.427690+0000","last_fullsized":"2023-10-08T02:31:28.427690+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:09.691655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"205'4","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.437891+0000","last_change":"2023-10-08T02:31:27.391616+0000","last_active":"2023-10-08T02:31:28.437891+0000","last_peered":"2023-10-08T02:31:28.437891+0000","last_clean":"2023-10-08T02:31:28.437891+0000","last_became_active":"2023-10-08T02:31:27.391355+0000","last_became_peered":"2023-10-08T02:31:27.391355+0000","last_unstale":"2023-10-08T02:31:28.437891+0000","last_undegraded":"2023-10-08T02:31:28.437891+0000","last_fullsized":"2023-10-08T02:31:28.437891+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:06:56.459953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.425572+0000","last_change":"2023-10-08T02:31:28.419066+0000","last_active":"2023-10-08T02:31:28.425572+0000","last_peered":"2023-10-08T02:31:28.425572+0000","last_clean":"2023-10-08T02:31:28.425572+0000","last_became_active":"2023-10-08T02:31:28.417549+0000","last_became_peered":"2023-10-08T02:31:28.417549+0000","last_unstale":"2023-10-08T02:31:28.425572+0000","last_undegraded":"2023-10-08T02:31:28.425572+0000","last_fullsized":"2023-10-08T02:31:28.425572+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:07.591450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.437585+0000","last_change":"2023-10-08T02:31:27.396369+0000","last_active":"2023-10-08T02:31:28.437585+0000","last_peered":"2023-10-08T02:31:28.437585+0000","last_clean":"2023-10-08T02:31:28.437585+0000","last_became_active":"2023-10-08T02:31:27.396078+0000","last_became_peered":"2023-10-08T02:31:27.396078+0000","last_unstale":"2023-10-08T02:31:28.437585+0000","last_undegraded":"2023-10-08T02:31:28.437585+0000","last_fullsized":"2023-10-08T02:31:28.437585+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:15:16.813739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427033+0000","last_change":"2023-10-08T02:31:28.419976+0000","last_active":"2023-10-08T02:31:28.427033+0000","last_peered":"2023-10-08T02:31:28.427033+0000","last_clean":"2023-10-08T02:31:28.427033+0000","last_became_active":"2023-10-08T02:31:28.419715+0000","last_became_peered":"2023-10-08T02:31:28.419715+0000","last_unstale":"2023-10-08T02:31:28.427033+0000","last_undegraded":"2023-10-08T02:31:28.427033+0000","last_fullsized":"2023-10-08T02:31:28.427033+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.439979+0000","last_change":"2023-10-08T02:31:27.408586+0000","last_active":"2023-10-08T02:31:28.439979+0000","last_peered":"2023-10-08T02:31:28.439979+0000","last_clean":"2023-10-08T02:31:28.439979+0000","last_became_active":"2023-10-08T02:31:27.407801+0000","last_became_peered":"2023-10-08T02:31:27.407801+0000","last_unstale":"2023-10-08T02:31:28.439979+0000","last_undegraded":"2023-10-08T02:31:28.439979+0000","last_fullsized":"2023-10-08T02:31:28.439979+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:07.107562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429724+0000","last_change":"2023-10-08T02:31:28.426344+0000","last_active":"2023-10-08T02:31:28.429724+0000","last_peered":"2023-10-08T02:31:28.429724+0000","last_clean":"2023-10-08T02:31:28.429724+0000","last_became_active":"2023-10-08T02:31:28.426062+0000","last_became_peered":"2023-10-08T02:31:28.426062+0000","last_unstale":"2023-10-08T02:31:28.429724+0000","last_undegraded":"2023-10-08T02:31:28.429724+0000","last_fullsized":"2023-10-08T02:31:28.429724+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:24.799351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.436757+0000","last_change":"2023-10-08T02:31:27.400816+0000","last_active":"2023-10-08T02:31:28.436757+0000","last_peered":"2023-10-08T02:31:28.436757+0000","last_clean":"2023-10-08T02:31:28.436757+0000","last_became_active":"2023-10-08T02:31:27.399677+0000","last_became_peered":"2023-10-08T02:31:27.399677+0000","last_unstale":"2023-10-08T02:31:28.436757+0000","last_undegraded":"2023-10-08T02:31:28.436757+0000","last_fullsized":"2023-10-08T02:31:28.436757+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:19.404980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429742+0000","last_change":"2023-10-08T02:31:28.427071+0000","last_active":"2023-10-08T02:31:28.429742+0000","last_peered":"2023-10-08T02:31:28.429742+0000","last_clean":"2023-10-08T02:31:28.429742+0000","last_became_active":"2023-10-08T02:31:28.426809+0000","last_became_peered":"2023-10-08T02:31:28.426809+0000","last_unstale":"2023-10-08T02:31:28.429742+0000","last_undegraded":"2023-10-08T02:31:28.429742+0000","last_fullsized":"2023-10-08T02:31:28.429742+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:17.010439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.439809+0000","last_change":"2023-10-08T02:31:27.401520+0000","last_active":"2023-10-08T02:31:28.439809+0000","last_peered":"2023-10-08T02:31:28.439809+0000","last_clean":"2023-10-08T02:31:28.439809+0000","last_became_active":"2023-10-08T02:31:27.401195+0000","last_became_peered":"2023-10-08T02:31:27.401195+0000","last_unstale":"2023-10-08T02:31:28.439809+0000","last_undegraded":"2023-10-08T02:31:28.439809+0000","last_fullsized":"2023-10-08T02:31:28.439809+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:59.104676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427533+0000","last_change":"2023-10-08T02:31:28.416645+0000","last_active":"2023-10-08T02:31:28.427533+0000","last_peered":"2023-10-08T02:31:28.427533+0000","last_clean":"2023-10-08T02:31:28.427533+0000","last_became_active":"2023-10-08T02:31:28.416372+0000","last_became_peered":"2023-10-08T02:31:28.416372+0000","last_unstale":"2023-10-08T02:31:28.427533+0000","last_undegraded":"2023-10-08T02:31:28.427533+0000","last_fullsized":"2023-10-08T02:31:28.427533+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:34:36.050932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.434315+0000","last_change":"2023-10-08T02:31:27.393885+0000","last_active":"2023-10-08T02:31:28.434315+0000","last_peered":"2023-10-08T02:31:28.434315+0000","last_clean":"2023-10-08T02:31:28.434315+0000","last_became_active":"2023-10-08T02:31:27.393626+0000","last_became_peered":"2023-10-08T02:31:27.393626+0000","last_unstale":"2023-10-08T02:31:28.434315+0000","last_undegraded":"2023-10-08T02:31:28.434315+0000","last_fullsized":"2023-10-08T02:31:28.434315+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:01.848399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426910+0000","last_change":"2023-10-08T02:31:28.424853+0000","last_active":"2023-10-08T02:31:28.426910+0000","last_peered":"2023-10-08T02:31:28.426910+0000","last_clean":"2023-10-08T02:31:28.426910+0000","last_became_active":"2023-10-08T02:31:28.424560+0000","last_became_peered":"2023-10-08T02:31:28.424560+0000","last_unstale":"2023-10-08T02:31:28.426910+0000","last_undegraded":"2023-10-08T02:31:28.426910+0000","last_fullsized":"2023-10-08T02:31:28.426910+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:00:04.803814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.409137+0000","last_change":"2023-10-08T02:31:27.400249+0000","last_active":"2023-10-08T02:31:28.409137+0000","last_peered":"2023-10-08T02:31:28.409137+0000","last_clean":"2023-10-08T02:31:28.409137+0000","last_became_active":"2023-10-08T02:31:27.399991+0000","last_became_peered":"2023-10-08T02:31:27.399991+0000","last_unstale":"2023-10-08T02:31:28.409137+0000","last_undegraded":"2023-10-08T02:31:28.409137+0000","last_fullsized":"2023-10-08T02:31:28.409137+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:41.754137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.423218+0000","last_change":"2023-10-08T02:31:28.413776+0000","last_active":"2023-10-08T02:31:28.423218+0000","last_peered":"2023-10-08T02:31:28.423218+0000","last_clean":"2023-10-08T02:31:28.423218+0000","last_became_active":"2023-10-08T02:31:28.413517+0000","last_became_peered":"2023-10-08T02:31:28.413517+0000","last_unstale":"2023-10-08T02:31:28.423218+0000","last_undegraded":"2023-10-08T02:31:28.423218+0000","last_fullsized":"2023-10-08T02:31:28.423218+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:28.907447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.413170+0000","last_change":"2023-10-08T02:31:27.396510+0000","last_active":"2023-10-08T02:31:28.413170+0000","last_peered":"2023-10-08T02:31:28.413170+0000","last_clean":"2023-10-08T02:31:28.413170+0000","last_became_active":"2023-10-08T02:31:27.396156+0000","last_became_peered":"2023-10-08T02:31:27.396156+0000","last_unstale":"2023-10-08T02:31:28.413170+0000","last_undegraded":"2023-10-08T02:31:28.413170+0000","last_fullsized":"2023-10-08T02:31:28.413170+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:47.569228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427610+0000","last_change":"2023-10-08T02:31:28.415875+0000","last_active":"2023-10-08T02:31:28.427610+0000","last_peered":"2023-10-08T02:31:28.427610+0000","last_clean":"2023-10-08T02:31:28.427610+0000","last_became_active":"2023-10-08T02:31:28.414796+0000","last_became_peered":"2023-10-08T02:31:28.414796+0000","last_unstale":"2023-10-08T02:31:28.427610+0000","last_undegraded":"2023-10-08T02:31:28.427610+0000","last_fullsized":"2023-10-08T02:31:28.427610+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:00:04.803814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.404581+0000","last_change":"2023-10-08T02:31:27.400838+0000","last_active":"2023-10-08T02:31:28.404581+0000","last_peered":"2023-10-08T02:31:28.404581+0000","last_clean":"2023-10-08T02:31:28.404581+0000","last_became_active":"2023-10-08T02:31:27.400475+0000","last_became_peered":"2023-10-08T02:31:27.400475+0000","last_unstale":"2023-10-08T02:31:28.404581+0000","last_undegraded":"2023-10-08T02:31:28.404581+0000","last_fullsized":"2023-10-08T02:31:28.404581+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:02.499427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432827+0000","last_change":"2023-10-08T02:31:28.428120+0000","last_active":"2023-10-08T02:31:28.432827+0000","last_peered":"2023-10-08T02:31:28.432827+0000","last_clean":"2023-10-08T02:31:28.432827+0000","last_became_active":"2023-10-08T02:31:28.427855+0000","last_became_peered":"2023-10-08T02:31:28.427855+0000","last_unstale":"2023-10-08T02:31:28.432827+0000","last_undegraded":"2023-10-08T02:31:28.432827+0000","last_fullsized":"2023-10-08T02:31:28.432827+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:20.001128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"205'3","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.442164+0000","last_change":"2023-10-08T02:31:27.393104+0000","last_active":"2023-10-08T02:31:28.442164+0000","last_peered":"2023-10-08T02:31:28.442164+0000","last_clean":"2023-10-08T02:31:28.442164+0000","last_became_active":"2023-10-08T02:31:27.392831+0000","last_became_peered":"2023-10-08T02:31:27.392831+0000","last_unstale":"2023-10-08T02:31:28.442164+0000","last_undegraded":"2023-10-08T02:31:28.442164+0000","last_fullsized":"2023-10-08T02:31:28.442164+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:06.956710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.430290+0000","last_change":"2023-10-08T02:31:28.429302+0000","last_active":"2023-10-08T02:31:28.430290+0000","last_peered":"2023-10-08T02:31:28.430290+0000","last_clean":"2023-10-08T02:31:28.430290+0000","last_became_active":"2023-10-08T02:31:28.427775+0000","last_became_peered":"2023-10-08T02:31:28.427775+0000","last_unstale":"2023-10-08T02:31:28.430290+0000","last_undegraded":"2023-10-08T02:31:28.430290+0000","last_fullsized":"2023-10-08T02:31:28.430290+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:18.788976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"205'6","reported_seq":16,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.443522+0000","last_change":"2023-10-08T02:31:27.401502+0000","last_active":"2023-10-08T02:31:28.443522+0000","last_peered":"2023-10-08T02:31:28.443522+0000","last_clean":"2023-10-08T02:31:28.443522+0000","last_became_active":"2023-10-08T02:31:27.401249+0000","last_became_peered":"2023-10-08T02:31:27.401249+0000","last_unstale":"2023-10-08T02:31:28.443522+0000","last_undegraded":"2023-10-08T02:31:28.443522+0000","last_fullsized":"2023-10-08T02:31:28.443522+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:15:16.813739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.423693+0000","last_change":"2023-10-08T02:31:28.415470+0000","last_active":"2023-10-08T02:31:28.423693+0000","last_peered":"2023-10-08T02:31:28.423693+0000","last_clean":"2023-10-08T02:31:28.423693+0000","last_became_active":"2023-10-08T02:31:28.415219+0000","last_became_peered":"2023-10-08T02:31:28.415219+0000","last_unstale":"2023-10-08T02:31:28.423693+0000","last_undegraded":"2023-10-08T02:31:28.423693+0000","last_fullsized":"2023-10-08T02:31:28.423693+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:31:38.361725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.442228+0000","last_change":"2023-10-08T02:31:27.405251+0000","last_active":"2023-10-08T02:31:28.442228+0000","last_peered":"2023-10-08T02:31:28.442228+0000","last_clean":"2023-10-08T02:31:28.442228+0000","last_became_active":"2023-10-08T02:31:27.404991+0000","last_became_peered":"2023-10-08T02:31:27.404991+0000","last_unstale":"2023-10-08T02:31:28.442228+0000","last_undegraded":"2023-10-08T02:31:28.442228+0000","last_fullsized":"2023-10-08T02:31:28.442228+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:34.316615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.433560+0000","last_change":"2023-10-08T02:31:28.431299+0000","last_active":"2023-10-08T02:31:28.433560+0000","last_peered":"2023-10-08T02:31:28.433560+0000","last_clean":"2023-10-08T02:31:28.433560+0000","last_became_active":"2023-10-08T02:31:28.431054+0000","last_became_peered":"2023-10-08T02:31:28.431054+0000","last_unstale":"2023-10-08T02:31:28.433560+0000","last_undegraded":"2023-10-08T02:31:28.433560+0000","last_fullsized":"2023-10-08T02:31:28.433560+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:28.967294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.408411+0000","last_change":"2023-10-08T02:31:27.398296+0000","last_active":"2023-10-08T02:31:28.408411+0000","last_peered":"2023-10-08T02:31:28.408411+0000","last_clean":"2023-10-08T02:31:28.408411+0000","last_became_active":"2023-10-08T02:31:27.398028+0000","last_became_peered":"2023-10-08T02:31:27.398028+0000","last_unstale":"2023-10-08T02:31:28.408411+0000","last_undegraded":"2023-10-08T02:31:28.408411+0000","last_fullsized":"2023-10-08T02:31:28.408411+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:22.227598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.430543+0000","last_change":"2023-10-08T02:31:28.427575+0000","last_active":"2023-10-08T02:31:28.430543+0000","last_peered":"2023-10-08T02:31:28.430543+0000","last_clean":"2023-10-08T02:31:28.430543+0000","last_became_active":"2023-10-08T02:31:28.427313+0000","last_became_peered":"2023-10-08T02:31:28.427313+0000","last_unstale":"2023-10-08T02:31:28.430543+0000","last_undegraded":"2023-10-08T02:31:28.430543+0000","last_fullsized":"2023-10-08T02:31:28.430543+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:07.013855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.404294+0000","last_change":"2023-10-08T02:31:27.403716+0000","last_active":"2023-10-08T02:31:28.404294+0000","last_peered":"2023-10-08T02:31:28.404294+0000","last_clean":"2023-10-08T02:31:28.404294+0000","last_became_active":"2023-10-08T02:31:27.403469+0000","last_became_peered":"2023-10-08T02:31:27.403469+0000","last_unstale":"2023-10-08T02:31:28.404294+0000","last_undegraded":"2023-10-08T02:31:28.404294+0000","last_fullsized":"2023-10-08T02:31:28.404294+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:47.569228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429608+0000","last_change":"2023-10-08T02:31:28.426114+0000","last_active":"2023-10-08T02:31:28.429608+0000","last_peered":"2023-10-08T02:31:28.429608+0000","last_clean":"2023-10-08T02:31:28.429608+0000","last_became_active":"2023-10-08T02:31:28.425861+0000","last_became_peered":"2023-10-08T02:31:28.425861+0000","last_unstale":"2023-10-08T02:31:28.429608+0000","last_undegraded":"2023-10-08T02:31:28.429608+0000","last_fullsized":"2023-10-08T02:31:28.429608+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:01:04.293850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"205'8","reported_seq":18,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.442747+0000","last_change":"2023-10-08T02:31:27.405230+0000","last_active":"2023-10-08T02:31:28.442747+0000","last_peered":"2023-10-08T02:31:28.442747+0000","last_clean":"2023-10-08T02:31:28.442747+0000","last_became_active":"2023-10-08T02:31:27.404954+0000","last_became_peered":"2023-10-08T02:31:27.404954+0000","last_unstale":"2023-10-08T02:31:28.442747+0000","last_undegraded":"2023-10-08T02:31:28.442747+0000","last_fullsized":"2023-10-08T02:31:28.442747+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:17.212905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.425335+0000","last_change":"2023-10-08T02:31:28.421187+0000","last_active":"2023-10-08T02:31:28.425335+0000","last_peered":"2023-10-08T02:31:28.425335+0000","last_clean":"2023-10-08T02:31:28.425335+0000","last_became_active":"2023-10-08T02:31:28.420897+0000","last_became_peered":"2023-10-08T02:31:28.420897+0000","last_unstale":"2023-10-08T02:31:28.425335+0000","last_undegraded":"2023-10-08T02:31:28.425335+0000","last_fullsized":"2023-10-08T02:31:28.425335+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:31.873060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.439840+0000","last_change":"2023-10-08T02:31:27.401709+0000","last_active":"2023-10-08T02:31:28.439840+0000","last_peered":"2023-10-08T02:31:28.439840+0000","last_clean":"2023-10-08T02:31:28.439840+0000","last_became_active":"2023-10-08T02:31:27.401456+0000","last_became_peered":"2023-10-08T02:31:27.401456+0000","last_unstale":"2023-10-08T02:31:28.439840+0000","last_undegraded":"2023-10-08T02:31:28.439840+0000","last_fullsized":"2023-10-08T02:31:28.439840+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:12.040775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.431336+0000","last_change":"2023-10-08T02:31:28.429289+0000","last_active":"2023-10-08T02:31:28.431336+0000","last_peered":"2023-10-08T02:31:28.431336+0000","last_clean":"2023-10-08T02:31:28.431336+0000","last_became_active":"2023-10-08T02:31:28.428776+0000","last_became_peered":"2023-10-08T02:31:28.428776+0000","last_unstale":"2023-10-08T02:31:28.431336+0000","last_undegraded":"2023-10-08T02:31:28.431336+0000","last_fullsized":"2023-10-08T02:31:28.431336+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:31:38.361725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.436482+0000","last_change":"2023-10-08T02:31:27.389939+0000","last_active":"2023-10-08T02:31:28.436482+0000","last_peered":"2023-10-08T02:31:28.436482+0000","last_clean":"2023-10-08T02:31:28.436482+0000","last_became_active":"2023-10-08T02:31:27.389684+0000","last_became_peered":"2023-10-08T02:31:27.389684+0000","last_unstale":"2023-10-08T02:31:28.436482+0000","last_undegraded":"2023-10-08T02:31:28.436482+0000","last_fullsized":"2023-10-08T02:31:28.436482+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:53:01.613685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427319+0000","last_change":"2023-10-08T02:31:28.424410+0000","last_active":"2023-10-08T02:31:28.427319+0000","last_peered":"2023-10-08T02:31:28.427319+0000","last_clean":"2023-10-08T02:31:28.427319+0000","last_became_active":"2023-10-08T02:31:28.424121+0000","last_became_peered":"2023-10-08T02:31:28.424121+0000","last_unstale":"2023-10-08T02:31:28.427319+0000","last_undegraded":"2023-10-08T02:31:28.427319+0000","last_fullsized":"2023-10-08T02:31:28.427319+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:36.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.442141+0000","last_change":"2023-10-08T02:31:27.405913+0000","last_active":"2023-10-08T02:31:28.442141+0000","last_peered":"2023-10-08T02:31:28.442141+0000","last_clean":"2023-10-08T02:31:28.442141+0000","last_became_active":"2023-10-08T02:31:27.405587+0000","last_became_peered":"2023-10-08T02:31:27.405587+0000","last_unstale":"2023-10-08T02:31:28.442141+0000","last_undegraded":"2023-10-08T02:31:28.442141+0000","last_fullsized":"2023-10-08T02:31:28.442141+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:47.569228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427691+0000","last_change":"2023-10-08T02:31:28.425554+0000","last_active":"2023-10-08T02:31:28.427691+0000","last_peered":"2023-10-08T02:31:28.427691+0000","last_clean":"2023-10-08T02:31:28.427691+0000","last_became_active":"2023-10-08T02:31:28.425292+0000","last_became_peered":"2023-10-08T02:31:28.425292+0000","last_unstale":"2023-10-08T02:31:28.427691+0000","last_undegraded":"2023-10-08T02:31:28.427691+0000","last_fullsized":"2023-10-08T02:31:28.427691+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:18.356274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.411284+0000","last_change":"2023-10-08T02:31:27.402673+0000","last_active":"2023-10-08T02:31:28.411284+0000","last_peered":"2023-10-08T02:31:28.411284+0000","last_clean":"2023-10-08T02:31:28.411284+0000","last_became_active":"2023-10-08T02:31:27.402392+0000","last_became_peered":"2023-10-08T02:31:27.402392+0000","last_unstale":"2023-10-08T02:31:28.411284+0000","last_undegraded":"2023-10-08T02:31:28.411284+0000","last_fullsized":"2023-10-08T02:31:28.411284+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:17.212905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.418246+0000","last_change":"2023-10-08T02:31:28.411868+0000","last_active":"2023-10-08T02:31:28.418246+0000","last_peered":"2023-10-08T02:31:28.418246+0000","last_clean":"2023-10-08T02:31:28.418246+0000","last_became_active":"2023-10-08T02:31:28.411624+0000","last_became_peered":"2023-10-08T02:31:28.411624+0000","last_unstale":"2023-10-08T02:31:28.418246+0000","last_undegraded":"2023-10-08T02:31:28.418246+0000","last_fullsized":"2023-10-08T02:31:28.418246+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:48.957331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.441851+0000","last_change":"2023-10-08T02:31:27.403154+0000","last_active":"2023-10-08T02:31:28.441851+0000","last_peered":"2023-10-08T02:31:28.441851+0000","last_clean":"2023-10-08T02:31:28.441851+0000","last_became_active":"2023-10-08T02:31:27.402875+0000","last_became_peered":"2023-10-08T02:31:27.402875+0000","last_unstale":"2023-10-08T02:31:28.441851+0000","last_undegraded":"2023-10-08T02:31:28.441851+0000","last_fullsized":"2023-10-08T02:31:28.441851+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:59.104676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432785+0000","last_change":"2023-10-08T02:31:28.428476+0000","last_active":"2023-10-08T02:31:28.432785+0000","last_peered":"2023-10-08T02:31:28.432785+0000","last_clean":"2023-10-08T02:31:28.432785+0000","last_became_active":"2023-10-08T02:31:28.428207+0000","last_became_peered":"2023-10-08T02:31:28.428207+0000","last_unstale":"2023-10-08T02:31:28.432785+0000","last_undegraded":"2023-10-08T02:31:28.432785+0000","last_fullsized":"2023-10-08T02:31:28.432785+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:34:36.050932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.403643+0000","last_change":"2023-10-08T02:31:27.402666+0000","last_active":"2023-10-08T02:31:28.403643+0000","last_peered":"2023-10-08T02:31:28.403643+0000","last_clean":"2023-10-08T02:31:28.403643+0000","last_became_active":"2023-10-08T02:31:27.402417+0000","last_became_peered":"2023-10-08T02:31:27.402417+0000","last_unstale":"2023-10-08T02:31:28.403643+0000","last_undegraded":"2023-10-08T02:31:28.403643+0000","last_fullsized":"2023-10-08T02:31:28.403643+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:40.216786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426433+0000","last_change":"2023-10-08T02:31:28.420775+0000","last_active":"2023-10-08T02:31:28.426433+0000","last_peered":"2023-10-08T02:31:28.426433+0000","last_clean":"2023-10-08T02:31:28.426433+0000","last_became_active":"2023-10-08T02:31:28.420511+0000","last_became_peered":"2023-10-08T02:31:28.420511+0000","last_unstale":"2023-10-08T02:31:28.426433+0000","last_undegraded":"2023-10-08T02:31:28.426433+0000","last_fullsized":"2023-10-08T02:31:28.426433+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:01.017826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.410244+0000","last_change":"2023-10-08T02:31:27.400758+0000","last_active":"2023-10-08T02:31:28.410244+0000","last_peered":"2023-10-08T02:31:28.410244+0000","last_clean":"2023-10-08T02:31:28.410244+0000","last_became_active":"2023-10-08T02:31:27.400509+0000","last_became_peered":"2023-10-08T02:31:27.400509+0000","last_unstale":"2023-10-08T02:31:28.410244+0000","last_undegraded":"2023-10-08T02:31:28.410244+0000","last_fullsized":"2023-10-08T02:31:28.410244+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:06.956710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.419928+0000","last_change":"2023-10-08T02:31:28.416027+0000","last_active":"2023-10-08T02:31:28.419928+0000","last_peered":"2023-10-08T02:31:28.419928+0000","last_clean":"2023-10-08T02:31:28.419928+0000","last_became_active":"2023-10-08T02:31:28.415785+0000","last_became_peered":"2023-10-08T02:31:28.415785+0000","last_unstale":"2023-10-08T02:31:28.419928+0000","last_undegraded":"2023-10-08T02:31:28.419928+0000","last_fullsized":"2023-10-08T02:31:28.419928+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:35.322154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.413368+0000","last_change":"2023-10-08T02:31:27.402671+0000","last_active":"2023-10-08T02:31:28.413368+0000","last_peered":"2023-10-08T02:31:28.413368+0000","last_clean":"2023-10-08T02:31:28.413368+0000","last_became_active":"2023-10-08T02:31:27.402398+0000","last_became_peered":"2023-10-08T02:31:27.402398+0000","last_unstale":"2023-10-08T02:31:28.413368+0000","last_undegraded":"2023-10-08T02:31:28.413368+0000","last_fullsized":"2023-10-08T02:31:28.413368+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:12.040775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.424001+0000","last_change":"2023-10-08T02:31:28.416694+0000","last_active":"2023-10-08T02:31:28.424001+0000","last_peered":"2023-10-08T02:31:28.424001+0000","last_clean":"2023-10-08T02:31:28.424001+0000","last_became_active":"2023-10-08T02:31:28.416476+0000","last_became_peered":"2023-10-08T02:31:28.416476+0000","last_unstale":"2023-10-08T02:31:28.424001+0000","last_undegraded":"2023-10-08T02:31:28.424001+0000","last_fullsized":"2023-10-08T02:31:28.424001+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:42.759676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.439927+0000","last_change":"2023-10-08T02:31:27.398449+0000","last_active":"2023-10-08T02:31:28.439927+0000","last_peered":"2023-10-08T02:31:28.439927+0000","last_clean":"2023-10-08T02:31:28.439927+0000","last_became_active":"2023-10-08T02:31:27.398126+0000","last_became_peered":"2023-10-08T02:31:27.398126+0000","last_unstale":"2023-10-08T02:31:28.439927+0000","last_undegraded":"2023-10-08T02:31:28.439927+0000","last_fullsized":"2023-10-08T02:31:28.439927+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:35.763180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429705+0000","last_change":"2023-10-08T02:31:28.426625+0000","last_active":"2023-10-08T02:31:28.429705+0000","last_peered":"2023-10-08T02:31:28.429705+0000","last_clean":"2023-10-08T02:31:28.429705+0000","last_became_active":"2023-10-08T02:31:28.426286+0000","last_became_peered":"2023-10-08T02:31:28.426286+0000","last_unstale":"2023-10-08T02:31:28.429705+0000","last_undegraded":"2023-10-08T02:31:28.429705+0000","last_fullsized":"2023-10-08T02:31:28.429705+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:57.675320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.408210+0000","last_change":"2023-10-08T02:31:27.403669+0000","last_active":"2023-10-08T02:31:28.408210+0000","last_peered":"2023-10-08T02:31:28.408210+0000","last_clean":"2023-10-08T02:31:28.408210+0000","last_became_active":"2023-10-08T02:31:27.403359+0000","last_became_peered":"2023-10-08T02:31:27.403359+0000","last_unstale":"2023-10-08T02:31:28.408210+0000","last_undegraded":"2023-10-08T02:31:28.408210+0000","last_fullsized":"2023-10-08T02:31:28.408210+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:04:22.618276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427626+0000","last_change":"2023-10-08T02:31:28.417570+0000","last_active":"2023-10-08T02:31:28.427626+0000","last_peered":"2023-10-08T02:31:28.427626+0000","last_clean":"2023-10-08T02:31:28.427626+0000","last_became_active":"2023-10-08T02:31:28.417313+0000","last_became_peered":"2023-10-08T02:31:28.417313+0000","last_unstale":"2023-10-08T02:31:28.427626+0000","last_undegraded":"2023-10-08T02:31:28.427626+0000","last_fullsized":"2023-10-08T02:31:28.427626+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:28.967294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.408839+0000","last_change":"2023-10-08T02:31:27.405779+0000","last_active":"2023-10-08T02:31:28.408839+0000","last_peered":"2023-10-08T02:31:28.408839+0000","last_clean":"2023-10-08T02:31:28.408839+0000","last_became_active":"2023-10-08T02:31:27.405500+0000","last_became_peered":"2023-10-08T02:31:27.405500+0000","last_unstale":"2023-10-08T02:31:28.408839+0000","last_undegraded":"2023-10-08T02:31:28.408839+0000","last_fullsized":"2023-10-08T02:31:28.408839+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:11.572027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427572+0000","last_change":"2023-10-08T02:31:28.416038+0000","last_active":"2023-10-08T02:31:28.427572+0000","last_peered":"2023-10-08T02:31:28.427572+0000","last_clean":"2023-10-08T02:31:28.427572+0000","last_became_active":"2023-10-08T02:31:28.415770+0000","last_became_peered":"2023-10-08T02:31:28.415770+0000","last_unstale":"2023-10-08T02:31:28.427572+0000","last_undegraded":"2023-10-08T02:31:28.427572+0000","last_fullsized":"2023-10-08T02:31:28.427572+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:20.001128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.405557+0000","last_change":"2023-10-08T02:31:27.390133+0000","last_active":"2023-10-08T02:31:28.405557+0000","last_peered":"2023-10-08T02:31:28.405557+0000","last_clean":"2023-10-08T02:31:28.405557+0000","last_became_active":"2023-10-08T02:31:27.389866+0000","last_became_peered":"2023-10-08T02:31:27.389866+0000","last_unstale":"2023-10-08T02:31:28.405557+0000","last_undegraded":"2023-10-08T02:31:28.405557+0000","last_fullsized":"2023-10-08T02:31:28.405557+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:54.894763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432719+0000","last_change":"2023-10-08T02:31:28.424856+0000","last_active":"2023-10-08T02:31:28.432719+0000","last_peered":"2023-10-08T02:31:28.432719+0000","last_clean":"2023-10-08T02:31:28.432719+0000","last_became_active":"2023-10-08T02:31:28.424588+0000","last_became_peered":"2023-10-08T02:31:28.424588+0000","last_unstale":"2023-10-08T02:31:28.432719+0000","last_undegraded":"2023-10-08T02:31:28.432719+0000","last_fullsized":"2023-10-08T02:31:28.432719+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:34:36.050932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.414398+0000","last_change":"2023-10-08T02:31:27.401864+0000","last_active":"2023-10-08T02:31:28.414398+0000","last_peered":"2023-10-08T02:31:28.414398+0000","last_clean":"2023-10-08T02:31:28.414398+0000","last_became_active":"2023-10-08T02:31:27.401513+0000","last_became_peered":"2023-10-08T02:31:27.401513+0000","last_unstale":"2023-10-08T02:31:28.414398+0000","last_undegraded":"2023-10-08T02:31:28.414398+0000","last_fullsized":"2023-10-08T02:31:28.414398+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:54.638239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432921+0000","last_change":"2023-10-08T02:31:28.428460+0000","last_active":"2023-10-08T02:31:28.432921+0000","last_peered":"2023-10-08T02:31:28.432921+0000","last_clean":"2023-10-08T02:31:28.432921+0000","last_became_active":"2023-10-08T02:31:28.428200+0000","last_became_peered":"2023-10-08T02:31:28.428200+0000","last_unstale":"2023-10-08T02:31:28.432921+0000","last_undegraded":"2023-10-08T02:31:28.432921+0000","last_fullsized":"2023-10-08T02:31:28.432921+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:24.799351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.413624+0000","last_change":"2023-10-08T02:31:27.398581+0000","last_active":"2023-10-08T02:31:28.413624+0000","last_peered":"2023-10-08T02:31:28.413624+0000","last_clean":"2023-10-08T02:31:28.413624+0000","last_became_active":"2023-10-08T02:31:27.397628+0000","last_became_peered":"2023-10-08T02:31:27.397628+0000","last_unstale":"2023-10-08T02:31:28.413624+0000","last_undegraded":"2023-10-08T02:31:28.413624+0000","last_fullsized":"2023-10-08T02:31:28.413624+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:07.107562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426414+0000","last_change":"2023-10-08T02:31:28.417585+0000","last_active":"2023-10-08T02:31:28.426414+0000","last_peered":"2023-10-08T02:31:28.426414+0000","last_clean":"2023-10-08T02:31:28.426414+0000","last_became_active":"2023-10-08T02:31:28.417336+0000","last_became_peered":"2023-10-08T02:31:28.417336+0000","last_unstale":"2023-10-08T02:31:28.426414+0000","last_undegraded":"2023-10-08T02:31:28.426414+0000","last_fullsized":"2023-10-08T02:31:28.426414+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:03.213603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.411845+0000","last_change":"2023-10-08T02:31:27.399585+0000","last_active":"2023-10-08T02:31:28.411845+0000","last_peered":"2023-10-08T02:31:28.411845+0000","last_clean":"2023-10-08T02:31:28.411845+0000","last_became_active":"2023-10-08T02:31:27.399306+0000","last_became_peered":"2023-10-08T02:31:27.399306+0000","last_unstale":"2023-10-08T02:31:28.411845+0000","last_undegraded":"2023-10-08T02:31:28.411845+0000","last_fullsized":"2023-10-08T02:31:28.411845+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:05:12.327087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429955+0000","last_change":"2023-10-08T02:31:28.424529+0000","last_active":"2023-10-08T02:31:28.429955+0000","last_peered":"2023-10-08T02:31:28.429955+0000","last_clean":"2023-10-08T02:31:28.429955+0000","last_became_active":"2023-10-08T02:31:28.424231+0000","last_became_peered":"2023-10-08T02:31:28.424231+0000","last_unstale":"2023-10-08T02:31:28.429955+0000","last_undegraded":"2023-10-08T02:31:28.429955+0000","last_fullsized":"2023-10-08T02:31:28.429955+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.410374+0000","last_change":"2023-10-08T02:31:27.402362+0000","last_active":"2023-10-08T02:31:28.410374+0000","last_peered":"2023-10-08T02:31:28.410374+0000","last_clean":"2023-10-08T02:31:28.410374+0000","last_became_active":"2023-10-08T02:31:27.402094+0000","last_became_peered":"2023-10-08T02:31:27.402094+0000","last_unstale":"2023-10-08T02:31:28.410374+0000","last_undegraded":"2023-10-08T02:31:28.410374+0000","last_fullsized":"2023-10-08T02:31:28.410374+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:12.040775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.425326+0000","last_change":"2023-10-08T02:31:28.420972+0000","last_active":"2023-10-08T02:31:28.425326+0000","last_peered":"2023-10-08T02:31:28.425326+0000","last_clean":"2023-10-08T02:31:28.425326+0000","last_became_active":"2023-10-08T02:31:28.420616+0000","last_became_peered":"2023-10-08T02:31:28.420616+0000","last_unstale":"2023-10-08T02:31:28.425326+0000","last_undegraded":"2023-10-08T02:31:28.425326+0000","last_fullsized":"2023-10-08T02:31:28.425326+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:28.782396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.408479+0000","last_change":"2023-10-08T02:31:27.404626+0000","last_active":"2023-10-08T02:31:28.408479+0000","last_peered":"2023-10-08T02:31:28.408479+0000","last_clean":"2023-10-08T02:31:28.408479+0000","last_became_active":"2023-10-08T02:31:27.404343+0000","last_became_peered":"2023-10-08T02:31:27.404343+0000","last_unstale":"2023-10-08T02:31:28.408479+0000","last_undegraded":"2023-10-08T02:31:28.408479+0000","last_fullsized":"2023-10-08T02:31:28.408479+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:01.848399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"195'348","reported_seq":1140,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.409303+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:31:28.409303+0000","last_peered":"2023-10-08T02:31:28.409303+0000","last_clean":"2023-10-08T02:31:28.409303+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:31:28.409303+0000","last_undegraded":"2023-10-08T02:31:28.409303+0000","last_fullsized":"2023-10-08T02:31:28.409303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575984,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316780,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055060,"statfs":{"total":96636764160,"available":96312381440,"internally_reserved":0,"allocated":3182592,"data_stored":1892939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.876,"5min":0.96899999999999997,"15min":1.0429999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.503},"max":{"1min":0.93300000000000005,"5min":0.97399999999999998,"15min":1.052},"last":0.58999999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.749,"15min":0.74199999999999999},"min":{"1min":0.64000000000000001,"5min":0.42199999999999999,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91200000000000003,"5min":0.98499999999999999,"15min":1.002},"last":0.66500000000000004}]},{"osd":2,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.61499999999999999,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":0.96699999999999997,"15min":1.0549999999999999},"last":0.623},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.58499999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93000000000000005,"5min":0.98499999999999999,"15min":1.03},"last":0.71799999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.57999999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79100000000000004,"15min":0.78800000000000003},"min":{"1min":0.66600000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.056},"last":0.73699999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":0.89500000000000002,"15min":1.091},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38100000000000001},"max":{"1min":0.79400000000000004,"5min":0.88800000000000001,"15min":0.92200000000000004},"last":0.78300000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.39300000000000002},"max":{"1min":0.91500000000000004,"5min":0.92100000000000004,"15min":1.012},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.44500000000000001},"max":{"1min":0.86099999999999999,"5min":0.92100000000000004,"15min":1.1240000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77600000000000002,"15min":0.77500000000000002},"min":{"1min":0.60699999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96999999999999997,"5min":1.0089999999999999,"15min":1.151},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.52500000000000002,"5min":0.441,"15min":0.42599999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.749}]}]},{"osd":6,"up_from":13,"seq":55834575984,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299808,"kb_used_data":2520,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072032,"statfs":{"total":96636764160,"available":96329760768,"internally_reserved":0,"allocated":2580480,"data_stored":1302467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95999999999999996,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.51100000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96199999999999997,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.57799999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71599999999999997,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":1.21},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.75900000000000001,"15min":0.752},"min":{"1min":0.504,"5min":0.504,"15min":0.45700000000000002},"max":{"1min":0.90000000000000002,"5min":3.851,"15min":3.851},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.501,"5min":0.47599999999999998,"15min":0.44400000000000001},"max":{"1min":0.77900000000000003,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.57599999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.88200000000000001,"5min":0.91000000000000003,"15min":1.258},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.56499999999999995,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":0.93600000000000005,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.57899999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.872,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.89000000000000001,"5min":1.139,"15min":1.139},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.876,"5min":1.341,"15min":1.341},"last":0.64000000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.92800000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93000000000000005,"5min":1.089,"15min":1.089},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":1.038,"15min":1.0980000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.94199999999999995,"5min":0.96999999999999997,"15min":1.054},"last":0.626}]}]},{"osd":1,"up_from":13,"seq":55834575984,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312108,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059732,"statfs":{"total":96636764160,"available":96317165568,"internally_reserved":0,"allocated":2592768,"data_stored":1304003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.0780000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.29199999999999998,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.571},"last":0.31900000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":1.1100000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":1.1299999999999999},"last":0.35499999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.755,"5min":0.80200000000000005,"15min":1.2190000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":1.341,"15min":1.341},"last":0.374}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.48199999999999998},"max":{"1min":1.171,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.629},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.67800000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.97499999999999998,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.127,"15min":1.27},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.81799999999999995},"min":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.915,"15min":1.915},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.50900000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.204},"last":0.78800000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.85799999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.306},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.66800000000000004,"5min":0.57999999999999996,"15min":0.55500000000000005},"max":{"1min":1.1399999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.77500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575983,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308584,"kb_used_data":3104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063256,"statfs":{"total":96636764160,"available":96320774144,"internally_reserved":0,"allocated":3178496,"data_stored":1892857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.32600000000000001,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.66600000000000004,"5min":0.77700000000000002,"15min":0.78400000000000003},"last":0.377},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.38900000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72599999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.71299999999999997,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49199999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.752,"5min":0.81799999999999995,"15min":0.93100000000000005},"last":0.624},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.89000000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.159,"5min":1.232,"15min":1.232},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.66800000000000004}]},{"osd":5,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.55700000000000005},"max":{"1min":1.0960000000000001,"5min":1.1990000000000001,"15min":1.216},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.59099999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":1.296,"15min":1.296},"last":0.68500000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.82799999999999996,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.1579999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80900000000000005,"15min":0.80400000000000005},"min":{"1min":0.71299999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1990000000000001,"5min":1.262,"15min":1.262},"last":0.91200000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.84199999999999997,"15min":0.84499999999999997},"min":{"1min":0.63500000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.171,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.86699999999999999,"15min":0.875},"min":{"1min":0.64400000000000002,"5min":0.58999999999999997,"15min":0.55700000000000005},"max":{"1min":1.2250000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.73399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316204,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055636,"statfs":{"total":96636764160,"available":96312971264,"internally_reserved":0,"allocated":2592768,"data_stored":1303133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.871},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66500000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48799999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.28699999999999998},"max":{"1min":0.66400000000000003,"5min":0.876,"15min":0.876},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.78900000000000003,"15min":0.80900000000000005},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.49299999999999999,"5min":0.45000000000000001,"15min":0.42799999999999999},"max":{"1min":0.86199999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.503},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90000000000000002,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.59099999999999997,"5min":0.503,"15min":0.49099999999999999},"max":{"1min":0.92600000000000005,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.46400000000000002},"max":{"1min":0.97399999999999998,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83799999999999997,"15min":0.84899999999999998},"min":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"max":{"1min":0.99399999999999999,"5min":1.353,"15min":1.417},"last":0.81599999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.81799999999999995},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.034,"5min":1.153,"15min":1.333},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81200000000000006,"15min":0.81799999999999995},"min":{"1min":0.63500000000000001,"5min":0.53000000000000003,"15min":0.51800000000000002},"max":{"1min":0.94799999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.748}]},{"osd":7,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.58499999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96199999999999997,"5min":1.387,"15min":1.4059999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82999999999999996,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0880000000000001,"5min":1.252,"15min":1.5720000000000001},"last":0.85899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575983,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299820,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072020,"statfs":{"total":96636764160,"available":96329748480,"internally_reserved":0,"allocated":2592768,"data_stored":1302661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.35199999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.90300000000000002},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.505,"15min":0.5},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.311},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68600000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.876,"5min":1.012,"15min":1.012},"last":0.378}]},{"osd":2,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78700000000000003,"5min":0.879,"15min":1.282},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.505,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.89200000000000002,"15min":1.512},"last":0.51800000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.012,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.625},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.56000000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.95699999999999996,"5min":1.1399999999999999,"15min":1.3180000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.51000000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.014,"5min":1.1160000000000001,"15min":1.2789999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.69799999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1220000000000001,"15min":1.24},"last":0.78700000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.81599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0269999999999999,"5min":1.0880000000000001,"15min":1.161},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.63700000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.94099999999999995,"5min":1.155,"15min":1.2849999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.83499999999999996},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96899999999999997,"5min":1.129,"15min":1.266},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98699999999999999,"5min":1.1259999999999999,"15min":1.282},"last":0.59299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299824,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072016,"statfs":{"total":96636764160,"available":96329744384,"internally_reserved":0,"allocated":2596864,"data_stored":1304557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.028},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.497,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90800000000000003,"5min":0.998,"15min":1.0409999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.51400000000000001,"5min":0.496,"15min":0.49099999999999999},"max":{"1min":1.01,"5min":1.079,"15min":1.079},"last":0.72799999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.034},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.65400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.022,"5min":1.097,"15min":1.097},"last":0.74099999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98399999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.1259999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.65600000000000003,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":1.004,"5min":1.0269999999999999,"15min":1.105},"last":0.77500000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":0.98299999999999998,"15min":1.0389999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.52900000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92300000000000004,"5min":1.03,"15min":1.03},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.68899999999999995},"min":{"1min":0.55300000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98099999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.42699999999999999},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0669999999999999},"last":0.874}]}]},{"osd":5,"up_from":13,"seq":55834575983,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316192,"kb_used_data":2520,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055648,"statfs":{"total":96636764160,"available":96312983552,"internally_reserved":0,"allocated":2580480,"data_stored":1302467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.53000000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93899999999999995,"5min":0.996,"15min":0.996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.59399999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.55400000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91400000000000003,"5min":0.97399999999999998,"15min":0.99199999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.61399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.0049999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77800000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.97699999999999998,"5min":1.028,"15min":1.028},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.47099999999999997},"max":{"1min":0.96299999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.51900000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.57299999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.86199999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.374},"max":{"1min":0.90300000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.64000000000000001,"15min":0.65700000000000003},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.73099999999999998,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.91000000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.496,"5min":0.46899999999999997,"15min":0.434},"max":{"1min":1.0009999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.044},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.069,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:31:32.734 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:31:32.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.949+0000 7fd2d619d700 1 Processor -- start 2023-10-08T02:31:32.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.950+0000 7fd2d619d700 1 -- start start 2023-10-08T02:31:32.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d619d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2d0156310 0x7fd2d00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d619d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d01566e0 0x7fd2d00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d619d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd2d00ca300 0x7fd2d00ae540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d619d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d00ca860 con 0x7fd2d0156310 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d619d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d00cab40 con 0x7fd2d00ca300 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d619d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d00cae20 con 0x7fd2d01566e0 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d519b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2d0156310 0x7fd2d00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d519b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2d0156310 0x7fd2d00c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57618/0 (socket says 172.21.15.119:57618) 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d519b700 1 -- 172.21.15.119:0/513594695 learned_addr learned my addr 172.21.15.119:0/513594695 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d499a700 1 --2- 172.21.15.119:0/513594695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d01566e0 0x7fd2d00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.951+0000 7fd2d599c700 1 --2- 172.21.15.119:0/513594695 >> v2:172.21.15.184:3300/0 conn(0x7fd2d00ca300 0x7fd2d00ae540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2d519b700 1 -- 172.21.15.119:0/513594695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d01566e0 msgr2=0x7fd2d00ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:32.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2d519b700 1 --2- 172.21.15.119:0/513594695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d01566e0 0x7fd2d00ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2d519b700 1 -- 172.21.15.119:0/513594695 >> v2:172.21.15.184:3300/0 conn(0x7fd2d00ca300 msgr2=0x7fd2d00ae540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:32.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2d519b700 1 --2- 172.21.15.119:0/513594695 >> v2:172.21.15.184:3300/0 conn(0x7fd2d00ca300 0x7fd2d00ae540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2d519b700 1 -- 172.21.15.119:0/513594695 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d00cb100 con 0x7fd2d0156310 2023-10-08T02:31:32.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2d519b700 1 --2- 172.21.15.119:0/513594695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d0156310 0x7fd2d00c9dc0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fd2d006f970 tx=0x7fd2cc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374430195e53afdc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2c7fff700 1 -- 172.21.15.119:0/513594695 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc00b860 con 0x7fd2d0156310 2023-10-08T02:31:32.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2c7fff700 1 -- 172.21.15.119:0/513594695 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc00b9c0 con 0x7fd2d0156310 2023-10-08T02:31:32.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.952+0000 7fd2c7fff700 1 -- 172.21.15.119:0/513594695 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc00bd60 con 0x7fd2d0156310 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 -- 172.21.15.119:0/513594695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d0156310 msgr2=0x7fd2d00c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 --2- 172.21.15.119:0/513594695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d0156310 0x7fd2d00c9dc0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fd2d006f970 tx=0x7fd2cc009250 comp rx=0 tx=0).stop 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 -- 172.21.15.119:0/513594695 shutdown_connections 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 --2- 172.21.15.119:0/513594695 >> v2:172.21.15.184:3300/0 conn(0x7fd2d00ca300 0x7fd2d00ae540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 --2- 172.21.15.119:0/513594695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d0156310 0x7fd2d00c9dc0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 --2- 172.21.15.119:0/513594695 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d01566e0 0x7fd2d00ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 -- 172.21.15.119:0/513594695 >> 172.21.15.119:0/513594695 conn(0x7fd2d00cc1f0 msgr2=0x7fd2d00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 -- 172.21.15.119:0/513594695 shutdown_connections 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.953+0000 7fd2d619d700 1 -- 172.21.15.119:0/513594695 wait complete. 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.954+0000 7fd2d619d700 1 Processor -- start 2023-10-08T02:31:32.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.954+0000 7fd2d619d700 1 -- start start 2023-10-08T02:31:32.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.955+0000 7fd2d619d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d00ca300 0x7fd2d014ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.955+0000 7fd2d619d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d0156310 0x7fd2d014b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.955+0000 7fd2d619d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2d01566e0 0x7fd2d014dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.955+0000 7fd2d619d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d014c190 con 0x7fd2d00ca300 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.955+0000 7fd2d619d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d014c470 con 0x7fd2d01566e0 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.955+0000 7fd2d619d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d014c750 con 0x7fd2d0156310 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.955+0000 7fd2d519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d00ca300 0x7fd2d014ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d519b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d00ca300 0x7fd2d014ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57634/0 (socket says 172.21.15.119:57634) 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d519b700 1 -- 172.21.15.119:0/1034924289 learned_addr learned my addr 172.21.15.119:0/1034924289 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d519b700 1 -- 172.21.15.119:0/1034924289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d0156310 msgr2=0x7fd2d014b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d519b700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d0156310 0x7fd2d014b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d519b700 1 -- 172.21.15.119:0/1034924289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2d01566e0 msgr2=0x7fd2d014dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d519b700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2d01566e0 0x7fd2d014dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d519b700 1 -- 172.21.15.119:0/1034924289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0094f0 con 0x7fd2d00ca300 2023-10-08T02:31:32.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d519b700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d00ca300 0x7fd2d014ad90 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fd2d006fe40 tx=0x7fd2cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2c57fa700 1 -- 172.21.15.119:0/1034924289 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc024440 con 0x7fd2d00ca300 2023-10-08T02:31:32.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2c57fa700 1 -- 172.21.15.119:0/1034924289 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2cc0245a0 con 0x7fd2d00ca300 2023-10-08T02:31:32.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d014ca30 con 0x7fd2d00ca300 2023-10-08T02:31:32.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.956+0000 7fd2c57fa700 1 -- 172.21.15.119:0/1034924289 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc024960 con 0x7fd2d00ca300 2023-10-08T02:31:32.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.957+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d014bc10 con 0x7fd2d00ca300 2023-10-08T02:31:32.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.957+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d014cd60 con 0x7fd2d00ca300 2023-10-08T02:31:32.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.958+0000 7fd2c57fa700 1 -- 172.21.15.119:0/1034924289 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2cc01a420 con 0x7fd2d00ca300 2023-10-08T02:31:32.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.959+0000 7fd2c57fa700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2b4066f30 0x7fd2b40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:32.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.959+0000 7fd2c57fa700 1 -- 172.21.15.119:0/1034924289 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7fd2cc086550 con 0x7fd2d00ca300 2023-10-08T02:31:32.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.959+0000 7fd2d499a700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2b4066f30 0x7fd2b40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:32.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.960+0000 7fd2d499a700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2b4066f30 0x7fd2b40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0151b80 tx=0x7fd2c0008040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:32.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:32.963+0000 7fd2c57fa700 1 -- 172.21.15.119:0/1034924289 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2cc0535b0 con 0x7fd2d00ca300 2023-10-08T02:31:33.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.149+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd2d004c3c0 con 0x7fd2b4066f30 2023-10-08T02:31:33.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.157+0000 7fd2c57fa700 1 -- 172.21.15.119:0/1034924289 <== mgr.10020 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289403 (secure 0 0 0) 0x7fd2d004c3c0 con 0x7fd2b4066f30 2023-10-08T02:31:33.161 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:33.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2b4066f30 msgr2=0x7fd2b40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2b4066f30 0x7fd2b40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0151b80 tx=0x7fd2c0008040 comp rx=0 tx=0).stop 2023-10-08T02:31:33.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d00ca300 msgr2=0x7fd2d014ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d00ca300 0x7fd2d014ad90 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fd2d006fe40 tx=0x7fd2cc017000 comp rx=0 tx=0).stop 2023-10-08T02:31:33.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 shutdown_connections 2023-10-08T02:31:33.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2b4066f30 0x7fd2b40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2d01566e0 0x7fd2d014dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d00ca300 0x7fd2d014ad90 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 --2- 172.21.15.119:0/1034924289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2d0156310 0x7fd2d014b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 >> 172.21.15.119:0/1034924289 conn(0x7fd2d00cc1f0 msgr2=0x7fd2d00b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 shutdown_connections 2023-10-08T02:31:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.162+0000 7fd2d619d700 1 -- 172.21.15.119:0/1034924289 wait complete. 2023-10-08T02:31:33.164 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:31:33.179 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-08T02:31:31.840572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":623,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":386,"ondisk_log_size":386,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2469320,"kb_used_data":21384,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752505400,"statfs":{"total":773094113280,"available":770565529600,"internally_reserved":0,"allocated":21897216,"data_stored":11605084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.462699"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.421790+0000","last_change":"2023-10-08T02:31:28.418181+0000","last_active":"2023-10-08T02:31:28.421790+0000","last_peered":"2023-10-08T02:31:28.421790+0000","last_clean":"2023-10-08T02:31:28.421790+0000","last_became_active":"2023-10-08T02:31:28.417928+0000","last_became_peered":"2023-10-08T02:31:28.417928+0000","last_unstale":"2023-10-08T02:31:28.421790+0000","last_undegraded":"2023-10-08T02:31:28.421790+0000","last_fullsized":"2023-10-08T02:31:28.421790+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:29.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427684+0000","last_change":"2023-10-08T02:31:28.426159+0000","last_active":"2023-10-08T02:31:28.427684+0000","last_peered":"2023-10-08T02:31:28.427684+0000","last_clean":"2023-10-08T02:31:28.427684+0000","last_became_active":"2023-10-08T02:31:28.425888+0000","last_became_peered":"2023-10-08T02:31:28.425888+0000","last_unstale":"2023-10-08T02:31:28.427684+0000","last_undegraded":"2023-10-08T02:31:28.427684+0000","last_fullsized":"2023-10-08T02:31:28.427684+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:06:57.465492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428267+0000","last_change":"2023-10-08T02:31:28.423479+0000","last_active":"2023-10-08T02:31:28.428267+0000","last_peered":"2023-10-08T02:31:28.428267+0000","last_clean":"2023-10-08T02:31:28.428267+0000","last_became_active":"2023-10-08T02:31:28.423214+0000","last_became_peered":"2023-10-08T02:31:28.423214+0000","last_unstale":"2023-10-08T02:31:28.428267+0000","last_undegraded":"2023-10-08T02:31:28.428267+0000","last_fullsized":"2023-10-08T02:31:28.428267+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:53.643477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.431042+0000","last_change":"2023-10-08T02:31:28.428427+0000","last_active":"2023-10-08T02:31:28.431042+0000","last_peered":"2023-10-08T02:31:28.431042+0000","last_clean":"2023-10-08T02:31:28.431042+0000","last_became_active":"2023-10-08T02:31:28.428135+0000","last_became_peered":"2023-10-08T02:31:28.428135+0000","last_unstale":"2023-10-08T02:31:28.431042+0000","last_undegraded":"2023-10-08T02:31:28.431042+0000","last_fullsized":"2023-10-08T02:31:28.431042+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:41.222324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.421289+0000","last_change":"2023-10-08T02:31:28.419218+0000","last_active":"2023-10-08T02:31:28.421289+0000","last_peered":"2023-10-08T02:31:28.421289+0000","last_clean":"2023-10-08T02:31:28.421289+0000","last_became_active":"2023-10-08T02:31:28.418913+0000","last_became_peered":"2023-10-08T02:31:28.418913+0000","last_unstale":"2023-10-08T02:31:28.421289+0000","last_undegraded":"2023-10-08T02:31:28.421289+0000","last_fullsized":"2023-10-08T02:31:28.421289+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:53.643477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426550+0000","last_change":"2023-10-08T02:31:28.421154+0000","last_active":"2023-10-08T02:31:28.426550+0000","last_peered":"2023-10-08T02:31:28.426550+0000","last_clean":"2023-10-08T02:31:28.426550+0000","last_became_active":"2023-10-08T02:31:28.420881+0000","last_became_peered":"2023-10-08T02:31:28.420881+0000","last_unstale":"2023-10-08T02:31:28.426550+0000","last_undegraded":"2023-10-08T02:31:28.426550+0000","last_fullsized":"2023-10-08T02:31:28.426550+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:36.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432749+0000","last_change":"2023-10-08T02:31:28.430705+0000","last_active":"2023-10-08T02:31:28.432749+0000","last_peered":"2023-10-08T02:31:28.432749+0000","last_clean":"2023-10-08T02:31:28.432749+0000","last_became_active":"2023-10-08T02:31:28.430452+0000","last_became_peered":"2023-10-08T02:31:28.430452+0000","last_unstale":"2023-10-08T02:31:28.432749+0000","last_undegraded":"2023-10-08T02:31:28.432749+0000","last_fullsized":"2023-10-08T02:31:28.432749+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:29.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428294+0000","last_change":"2023-10-08T02:31:28.416797+0000","last_active":"2023-10-08T02:31:28.428294+0000","last_peered":"2023-10-08T02:31:28.428294+0000","last_clean":"2023-10-08T02:31:28.428294+0000","last_became_active":"2023-10-08T02:31:28.416157+0000","last_became_peered":"2023-10-08T02:31:28.416157+0000","last_unstale":"2023-10-08T02:31:28.428294+0000","last_undegraded":"2023-10-08T02:31:28.428294+0000","last_fullsized":"2023-10-08T02:31:28.428294+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:24.799351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.431318+0000","last_change":"2023-10-08T02:31:28.429084+0000","last_active":"2023-10-08T02:31:28.431318+0000","last_peered":"2023-10-08T02:31:28.431318+0000","last_clean":"2023-10-08T02:31:28.431318+0000","last_became_active":"2023-10-08T02:31:28.428379+0000","last_became_peered":"2023-10-08T02:31:28.428379+0000","last_unstale":"2023-10-08T02:31:28.431318+0000","last_undegraded":"2023-10-08T02:31:28.431318+0000","last_fullsized":"2023-10-08T02:31:28.431318+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.421266+0000","last_change":"2023-10-08T02:31:28.415359+0000","last_active":"2023-10-08T02:31:28.421266+0000","last_peered":"2023-10-08T02:31:28.421266+0000","last_clean":"2023-10-08T02:31:28.421266+0000","last_became_active":"2023-10-08T02:31:28.415111+0000","last_became_peered":"2023-10-08T02:31:28.415111+0000","last_unstale":"2023-10-08T02:31:28.421266+0000","last_undegraded":"2023-10-08T02:31:28.421266+0000","last_fullsized":"2023-10-08T02:31:28.421266+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:51.625087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426920+0000","last_change":"2023-10-08T02:31:28.424096+0000","last_active":"2023-10-08T02:31:28.426920+0000","last_peered":"2023-10-08T02:31:28.426920+0000","last_clean":"2023-10-08T02:31:28.426920+0000","last_became_active":"2023-10-08T02:31:28.423755+0000","last_became_peered":"2023-10-08T02:31:28.423755+0000","last_unstale":"2023-10-08T02:31:28.426920+0000","last_undegraded":"2023-10-08T02:31:28.426920+0000","last_fullsized":"2023-10-08T02:31:28.426920+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:27:23.536896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426249+0000","last_change":"2023-10-08T02:31:28.422555+0000","last_active":"2023-10-08T02:31:28.426249+0000","last_peered":"2023-10-08T02:31:28.426249+0000","last_clean":"2023-10-08T02:31:28.426249+0000","last_became_active":"2023-10-08T02:31:28.422262+0000","last_became_peered":"2023-10-08T02:31:28.422262+0000","last_unstale":"2023-10-08T02:31:28.426249+0000","last_undegraded":"2023-10-08T02:31:28.426249+0000","last_fullsized":"2023-10-08T02:31:28.426249+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:34:14.758340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.431032+0000","last_change":"2023-10-08T02:31:28.427714+0000","last_active":"2023-10-08T02:31:28.431032+0000","last_peered":"2023-10-08T02:31:28.431032+0000","last_clean":"2023-10-08T02:31:28.431032+0000","last_became_active":"2023-10-08T02:31:28.426640+0000","last_became_peered":"2023-10-08T02:31:28.426640+0000","last_unstale":"2023-10-08T02:31:28.431032+0000","last_undegraded":"2023-10-08T02:31:28.431032+0000","last_fullsized":"2023-10-08T02:31:28.431032+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:02:31.599689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.430695+0000","last_change":"2023-10-08T02:31:28.428325+0000","last_active":"2023-10-08T02:31:28.430695+0000","last_peered":"2023-10-08T02:31:28.430695+0000","last_clean":"2023-10-08T02:31:28.430695+0000","last_became_active":"2023-10-08T02:31:28.428080+0000","last_became_peered":"2023-10-08T02:31:28.428080+0000","last_unstale":"2023-10-08T02:31:28.430695+0000","last_undegraded":"2023-10-08T02:31:28.430695+0000","last_fullsized":"2023-10-08T02:31:28.430695+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:25:53.643477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.425951+0000","last_change":"2023-10-08T02:31:28.421565+0000","last_active":"2023-10-08T02:31:28.425951+0000","last_peered":"2023-10-08T02:31:28.425951+0000","last_clean":"2023-10-08T02:31:28.425951+0000","last_became_active":"2023-10-08T02:31:28.421301+0000","last_became_peered":"2023-10-08T02:31:28.421301+0000","last_unstale":"2023-10-08T02:31:28.425951+0000","last_undegraded":"2023-10-08T02:31:28.425951+0000","last_fullsized":"2023-10-08T02:31:28.425951+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:56:00.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428314+0000","last_change":"2023-10-08T02:31:28.416973+0000","last_active":"2023-10-08T02:31:28.428314+0000","last_peered":"2023-10-08T02:31:28.428314+0000","last_clean":"2023-10-08T02:31:28.428314+0000","last_became_active":"2023-10-08T02:31:28.416527+0000","last_became_peered":"2023-10-08T02:31:28.416527+0000","last_unstale":"2023-10-08T02:31:28.428314+0000","last_undegraded":"2023-10-08T02:31:28.428314+0000","last_fullsized":"2023-10-08T02:31:28.428314+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.421615+0000","last_change":"2023-10-08T02:31:28.419778+0000","last_active":"2023-10-08T02:31:28.421615+0000","last_peered":"2023-10-08T02:31:28.421615+0000","last_clean":"2023-10-08T02:31:28.421615+0000","last_became_active":"2023-10-08T02:31:28.419523+0000","last_became_peered":"2023-10-08T02:31:28.419523+0000","last_unstale":"2023-10-08T02:31:28.421615+0000","last_undegraded":"2023-10-08T02:31:28.421615+0000","last_fullsized":"2023-10-08T02:31:28.421615+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:50:06.141377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432687+0000","last_change":"2023-10-08T02:31:28.429415+0000","last_active":"2023-10-08T02:31:28.432687+0000","last_peered":"2023-10-08T02:31:28.432687+0000","last_clean":"2023-10-08T02:31:28.432687+0000","last_became_active":"2023-10-08T02:31:28.429106+0000","last_became_peered":"2023-10-08T02:31:28.429106+0000","last_unstale":"2023-10-08T02:31:28.432687+0000","last_undegraded":"2023-10-08T02:31:28.432687+0000","last_fullsized":"2023-10-08T02:31:28.432687+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.430270+0000","last_change":"2023-10-08T02:31:28.426900+0000","last_active":"2023-10-08T02:31:28.430270+0000","last_peered":"2023-10-08T02:31:28.430270+0000","last_clean":"2023-10-08T02:31:28.430270+0000","last_became_active":"2023-10-08T02:31:28.426596+0000","last_became_peered":"2023-10-08T02:31:28.426596+0000","last_unstale":"2023-10-08T02:31:28.430270+0000","last_undegraded":"2023-10-08T02:31:28.430270+0000","last_fullsized":"2023-10-08T02:31:28.430270+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:25:31.096884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.418902+0000","last_change":"2023-10-08T02:31:28.415272+0000","last_active":"2023-10-08T02:31:28.418902+0000","last_peered":"2023-10-08T02:31:28.418902+0000","last_clean":"2023-10-08T02:31:28.418902+0000","last_became_active":"2023-10-08T02:31:28.414998+0000","last_became_peered":"2023-10-08T02:31:28.414998+0000","last_unstale":"2023-10-08T02:31:28.418902+0000","last_undegraded":"2023-10-08T02:31:28.418902+0000","last_fullsized":"2023-10-08T02:31:28.418902+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:24.799351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429153+0000","last_change":"2023-10-08T02:31:28.421213+0000","last_active":"2023-10-08T02:31:28.429153+0000","last_peered":"2023-10-08T02:31:28.429153+0000","last_clean":"2023-10-08T02:31:28.429153+0000","last_became_active":"2023-10-08T02:31:28.420955+0000","last_became_peered":"2023-10-08T02:31:28.420955+0000","last_unstale":"2023-10-08T02:31:28.429153+0000","last_undegraded":"2023-10-08T02:31:28.429153+0000","last_fullsized":"2023-10-08T02:31:28.429153+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:28.907447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429649+0000","last_change":"2023-10-08T02:31:28.426612+0000","last_active":"2023-10-08T02:31:28.429649+0000","last_peered":"2023-10-08T02:31:28.429649+0000","last_clean":"2023-10-08T02:31:28.429649+0000","last_became_active":"2023-10-08T02:31:28.426171+0000","last_became_peered":"2023-10-08T02:31:28.426171+0000","last_unstale":"2023-10-08T02:31:28.429649+0000","last_undegraded":"2023-10-08T02:31:28.429649+0000","last_fullsized":"2023-10-08T02:31:28.429649+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:44:58.587382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.423638+0000","last_change":"2023-10-08T02:31:28.415019+0000","last_active":"2023-10-08T02:31:28.423638+0000","last_peered":"2023-10-08T02:31:28.423638+0000","last_clean":"2023-10-08T02:31:28.423638+0000","last_became_active":"2023-10-08T02:31:28.414570+0000","last_became_peered":"2023-10-08T02:31:28.414570+0000","last_unstale":"2023-10-08T02:31:28.423638+0000","last_undegraded":"2023-10-08T02:31:28.423638+0000","last_fullsized":"2023-10-08T02:31:28.423638+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:50:06.141377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428331+0000","last_change":"2023-10-08T02:31:28.421401+0000","last_active":"2023-10-08T02:31:28.428331+0000","last_peered":"2023-10-08T02:31:28.428331+0000","last_clean":"2023-10-08T02:31:28.428331+0000","last_became_active":"2023-10-08T02:31:28.421098+0000","last_became_peered":"2023-10-08T02:31:28.421098+0000","last_unstale":"2023-10-08T02:31:28.428331+0000","last_undegraded":"2023-10-08T02:31:28.428331+0000","last_fullsized":"2023-10-08T02:31:28.428331+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:26:29.021937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.416251+0000","last_change":"2023-10-08T02:31:28.415479+0000","last_active":"2023-10-08T02:31:28.416251+0000","last_peered":"2023-10-08T02:31:28.416251+0000","last_clean":"2023-10-08T02:31:28.416251+0000","last_became_active":"2023-10-08T02:31:28.415211+0000","last_became_peered":"2023-10-08T02:31:28.415211+0000","last_unstale":"2023-10-08T02:31:28.416251+0000","last_undegraded":"2023-10-08T02:31:28.416251+0000","last_fullsized":"2023-10-08T02:31:28.416251+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:03.504965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432774+0000","last_change":"2023-10-08T02:31:28.429954+0000","last_active":"2023-10-08T02:31:28.432774+0000","last_peered":"2023-10-08T02:31:28.432774+0000","last_clean":"2023-10-08T02:31:28.432774+0000","last_became_active":"2023-10-08T02:31:28.429702+0000","last_became_peered":"2023-10-08T02:31:28.429702+0000","last_unstale":"2023-10-08T02:31:28.432774+0000","last_undegraded":"2023-10-08T02:31:28.432774+0000","last_fullsized":"2023-10-08T02:31:28.432774+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:09.691655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426936+0000","last_change":"2023-10-08T02:31:28.421777+0000","last_active":"2023-10-08T02:31:28.426936+0000","last_peered":"2023-10-08T02:31:28.426936+0000","last_clean":"2023-10-08T02:31:28.426936+0000","last_became_active":"2023-10-08T02:31:28.421533+0000","last_became_peered":"2023-10-08T02:31:28.421533+0000","last_unstale":"2023-10-08T02:31:28.426936+0000","last_undegraded":"2023-10-08T02:31:28.426936+0000","last_fullsized":"2023-10-08T02:31:28.426936+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:42.759676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.417368+0000","last_change":"2023-10-08T02:31:28.413558+0000","last_active":"2023-10-08T02:31:28.417368+0000","last_peered":"2023-10-08T02:31:28.417368+0000","last_clean":"2023-10-08T02:31:28.417368+0000","last_became_active":"2023-10-08T02:31:28.412818+0000","last_became_peered":"2023-10-08T02:31:28.412818+0000","last_unstale":"2023-10-08T02:31:28.417368+0000","last_undegraded":"2023-10-08T02:31:28.417368+0000","last_fullsized":"2023-10-08T02:31:28.417368+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:31:38.361725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426220+0000","last_change":"2023-10-08T02:31:28.420178+0000","last_active":"2023-10-08T02:31:28.426220+0000","last_peered":"2023-10-08T02:31:28.426220+0000","last_clean":"2023-10-08T02:31:28.426220+0000","last_became_active":"2023-10-08T02:31:28.419924+0000","last_became_peered":"2023-10-08T02:31:28.419924+0000","last_unstale":"2023-10-08T02:31:28.426220+0000","last_undegraded":"2023-10-08T02:31:28.426220+0000","last_fullsized":"2023-10-08T02:31:28.426220+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:15:57.286618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427183+0000","last_change":"2023-10-08T02:31:28.420611+0000","last_active":"2023-10-08T02:31:28.427183+0000","last_peered":"2023-10-08T02:31:28.427183+0000","last_clean":"2023-10-08T02:31:28.427183+0000","last_became_active":"2023-10-08T02:31:28.420326+0000","last_became_peered":"2023-10-08T02:31:28.420326+0000","last_unstale":"2023-10-08T02:31:28.427183+0000","last_undegraded":"2023-10-08T02:31:28.427183+0000","last_fullsized":"2023-10-08T02:31:28.427183+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:50:06.141377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.428406+0000","last_change":"2023-10-08T02:31:28.423324+0000","last_active":"2023-10-08T02:31:28.428406+0000","last_peered":"2023-10-08T02:31:28.428406+0000","last_clean":"2023-10-08T02:31:28.428406+0000","last_became_active":"2023-10-08T02:31:28.423071+0000","last_became_peered":"2023-10-08T02:31:28.423071+0000","last_unstale":"2023-10-08T02:31:28.428406+0000","last_undegraded":"2023-10-08T02:31:28.428406+0000","last_fullsized":"2023-10-08T02:31:28.428406+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:07.013855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.418443+0000","last_change":"2023-10-08T02:31:28.412836+0000","last_active":"2023-10-08T02:31:28.418443+0000","last_peered":"2023-10-08T02:31:28.418443+0000","last_clean":"2023-10-08T02:31:28.418443+0000","last_became_active":"2023-10-08T02:31:28.412574+0000","last_became_peered":"2023-10-08T02:31:28.412574+0000","last_unstale":"2023-10-08T02:31:28.418443+0000","last_undegraded":"2023-10-08T02:31:28.418443+0000","last_fullsized":"2023-10-08T02:31:28.418443+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:36.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427690+0000","last_change":"2023-10-08T02:31:28.422490+0000","last_active":"2023-10-08T02:31:28.427690+0000","last_peered":"2023-10-08T02:31:28.427690+0000","last_clean":"2023-10-08T02:31:28.427690+0000","last_became_active":"2023-10-08T02:31:28.422232+0000","last_became_peered":"2023-10-08T02:31:28.422232+0000","last_unstale":"2023-10-08T02:31:28.427690+0000","last_undegraded":"2023-10-08T02:31:28.427690+0000","last_fullsized":"2023-10-08T02:31:28.427690+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:12:09.691655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"205'4","reported_seq":14,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.437891+0000","last_change":"2023-10-08T02:31:27.391616+0000","last_active":"2023-10-08T02:31:28.437891+0000","last_peered":"2023-10-08T02:31:28.437891+0000","last_clean":"2023-10-08T02:31:28.437891+0000","last_became_active":"2023-10-08T02:31:27.391355+0000","last_became_peered":"2023-10-08T02:31:27.391355+0000","last_unstale":"2023-10-08T02:31:28.437891+0000","last_undegraded":"2023-10-08T02:31:28.437891+0000","last_fullsized":"2023-10-08T02:31:28.437891+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:06:56.459953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.425572+0000","last_change":"2023-10-08T02:31:28.419066+0000","last_active":"2023-10-08T02:31:28.425572+0000","last_peered":"2023-10-08T02:31:28.425572+0000","last_clean":"2023-10-08T02:31:28.425572+0000","last_became_active":"2023-10-08T02:31:28.417549+0000","last_became_peered":"2023-10-08T02:31:28.417549+0000","last_unstale":"2023-10-08T02:31:28.425572+0000","last_undegraded":"2023-10-08T02:31:28.425572+0000","last_fullsized":"2023-10-08T02:31:28.425572+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:29:07.591450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.437585+0000","last_change":"2023-10-08T02:31:27.396369+0000","last_active":"2023-10-08T02:31:28.437585+0000","last_peered":"2023-10-08T02:31:28.437585+0000","last_clean":"2023-10-08T02:31:28.437585+0000","last_became_active":"2023-10-08T02:31:27.396078+0000","last_became_peered":"2023-10-08T02:31:27.396078+0000","last_unstale":"2023-10-08T02:31:28.437585+0000","last_undegraded":"2023-10-08T02:31:28.437585+0000","last_fullsized":"2023-10-08T02:31:28.437585+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:15:16.813739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427033+0000","last_change":"2023-10-08T02:31:28.419976+0000","last_active":"2023-10-08T02:31:28.427033+0000","last_peered":"2023-10-08T02:31:28.427033+0000","last_clean":"2023-10-08T02:31:28.427033+0000","last_became_active":"2023-10-08T02:31:28.419715+0000","last_became_peered":"2023-10-08T02:31:28.419715+0000","last_unstale":"2023-10-08T02:31:28.427033+0000","last_undegraded":"2023-10-08T02:31:28.427033+0000","last_fullsized":"2023-10-08T02:31:28.427033+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.439979+0000","last_change":"2023-10-08T02:31:27.408586+0000","last_active":"2023-10-08T02:31:28.439979+0000","last_peered":"2023-10-08T02:31:28.439979+0000","last_clean":"2023-10-08T02:31:28.439979+0000","last_became_active":"2023-10-08T02:31:27.407801+0000","last_became_peered":"2023-10-08T02:31:27.407801+0000","last_unstale":"2023-10-08T02:31:28.439979+0000","last_undegraded":"2023-10-08T02:31:28.439979+0000","last_fullsized":"2023-10-08T02:31:28.439979+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:07.107562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429724+0000","last_change":"2023-10-08T02:31:28.426344+0000","last_active":"2023-10-08T02:31:28.429724+0000","last_peered":"2023-10-08T02:31:28.429724+0000","last_clean":"2023-10-08T02:31:28.429724+0000","last_became_active":"2023-10-08T02:31:28.426062+0000","last_became_peered":"2023-10-08T02:31:28.426062+0000","last_unstale":"2023-10-08T02:31:28.429724+0000","last_undegraded":"2023-10-08T02:31:28.429724+0000","last_fullsized":"2023-10-08T02:31:28.429724+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:24.799351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.436757+0000","last_change":"2023-10-08T02:31:27.400816+0000","last_active":"2023-10-08T02:31:28.436757+0000","last_peered":"2023-10-08T02:31:28.436757+0000","last_clean":"2023-10-08T02:31:28.436757+0000","last_became_active":"2023-10-08T02:31:27.399677+0000","last_became_peered":"2023-10-08T02:31:27.399677+0000","last_unstale":"2023-10-08T02:31:28.436757+0000","last_undegraded":"2023-10-08T02:31:28.436757+0000","last_fullsized":"2023-10-08T02:31:28.436757+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:01:19.404980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429742+0000","last_change":"2023-10-08T02:31:28.427071+0000","last_active":"2023-10-08T02:31:28.429742+0000","last_peered":"2023-10-08T02:31:28.429742+0000","last_clean":"2023-10-08T02:31:28.429742+0000","last_became_active":"2023-10-08T02:31:28.426809+0000","last_became_peered":"2023-10-08T02:31:28.426809+0000","last_unstale":"2023-10-08T02:31:28.429742+0000","last_undegraded":"2023-10-08T02:31:28.429742+0000","last_fullsized":"2023-10-08T02:31:28.429742+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:39:17.010439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.439809+0000","last_change":"2023-10-08T02:31:27.401520+0000","last_active":"2023-10-08T02:31:28.439809+0000","last_peered":"2023-10-08T02:31:28.439809+0000","last_clean":"2023-10-08T02:31:28.439809+0000","last_became_active":"2023-10-08T02:31:27.401195+0000","last_became_peered":"2023-10-08T02:31:27.401195+0000","last_unstale":"2023-10-08T02:31:28.439809+0000","last_undegraded":"2023-10-08T02:31:28.439809+0000","last_fullsized":"2023-10-08T02:31:28.439809+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:59.104676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427533+0000","last_change":"2023-10-08T02:31:28.416645+0000","last_active":"2023-10-08T02:31:28.427533+0000","last_peered":"2023-10-08T02:31:28.427533+0000","last_clean":"2023-10-08T02:31:28.427533+0000","last_became_active":"2023-10-08T02:31:28.416372+0000","last_became_peered":"2023-10-08T02:31:28.416372+0000","last_unstale":"2023-10-08T02:31:28.427533+0000","last_undegraded":"2023-10-08T02:31:28.427533+0000","last_fullsized":"2023-10-08T02:31:28.427533+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:34:36.050932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.434315+0000","last_change":"2023-10-08T02:31:27.393885+0000","last_active":"2023-10-08T02:31:28.434315+0000","last_peered":"2023-10-08T02:31:28.434315+0000","last_clean":"2023-10-08T02:31:28.434315+0000","last_became_active":"2023-10-08T02:31:27.393626+0000","last_became_peered":"2023-10-08T02:31:27.393626+0000","last_unstale":"2023-10-08T02:31:28.434315+0000","last_undegraded":"2023-10-08T02:31:28.434315+0000","last_fullsized":"2023-10-08T02:31:28.434315+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:01.848399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426910+0000","last_change":"2023-10-08T02:31:28.424853+0000","last_active":"2023-10-08T02:31:28.426910+0000","last_peered":"2023-10-08T02:31:28.426910+0000","last_clean":"2023-10-08T02:31:28.426910+0000","last_became_active":"2023-10-08T02:31:28.424560+0000","last_became_peered":"2023-10-08T02:31:28.424560+0000","last_unstale":"2023-10-08T02:31:28.426910+0000","last_undegraded":"2023-10-08T02:31:28.426910+0000","last_fullsized":"2023-10-08T02:31:28.426910+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:00:04.803814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.409137+0000","last_change":"2023-10-08T02:31:27.400249+0000","last_active":"2023-10-08T02:31:28.409137+0000","last_peered":"2023-10-08T02:31:28.409137+0000","last_clean":"2023-10-08T02:31:28.409137+0000","last_became_active":"2023-10-08T02:31:27.399991+0000","last_became_peered":"2023-10-08T02:31:27.399991+0000","last_unstale":"2023-10-08T02:31:28.409137+0000","last_undegraded":"2023-10-08T02:31:28.409137+0000","last_fullsized":"2023-10-08T02:31:28.409137+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:41.754137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.423218+0000","last_change":"2023-10-08T02:31:28.413776+0000","last_active":"2023-10-08T02:31:28.423218+0000","last_peered":"2023-10-08T02:31:28.423218+0000","last_clean":"2023-10-08T02:31:28.423218+0000","last_became_active":"2023-10-08T02:31:28.413517+0000","last_became_peered":"2023-10-08T02:31:28.413517+0000","last_unstale":"2023-10-08T02:31:28.423218+0000","last_undegraded":"2023-10-08T02:31:28.423218+0000","last_fullsized":"2023-10-08T02:31:28.423218+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:09:28.907447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.413170+0000","last_change":"2023-10-08T02:31:27.396510+0000","last_active":"2023-10-08T02:31:28.413170+0000","last_peered":"2023-10-08T02:31:28.413170+0000","last_clean":"2023-10-08T02:31:28.413170+0000","last_became_active":"2023-10-08T02:31:27.396156+0000","last_became_peered":"2023-10-08T02:31:27.396156+0000","last_unstale":"2023-10-08T02:31:28.413170+0000","last_undegraded":"2023-10-08T02:31:28.413170+0000","last_fullsized":"2023-10-08T02:31:28.413170+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:47.569228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427610+0000","last_change":"2023-10-08T02:31:28.415875+0000","last_active":"2023-10-08T02:31:28.427610+0000","last_peered":"2023-10-08T02:31:28.427610+0000","last_clean":"2023-10-08T02:31:28.427610+0000","last_became_active":"2023-10-08T02:31:28.414796+0000","last_became_peered":"2023-10-08T02:31:28.414796+0000","last_unstale":"2023-10-08T02:31:28.427610+0000","last_undegraded":"2023-10-08T02:31:28.427610+0000","last_fullsized":"2023-10-08T02:31:28.427610+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:00:04.803814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.404581+0000","last_change":"2023-10-08T02:31:27.400838+0000","last_active":"2023-10-08T02:31:28.404581+0000","last_peered":"2023-10-08T02:31:28.404581+0000","last_clean":"2023-10-08T02:31:28.404581+0000","last_became_active":"2023-10-08T02:31:27.400475+0000","last_became_peered":"2023-10-08T02:31:27.400475+0000","last_unstale":"2023-10-08T02:31:28.404581+0000","last_undegraded":"2023-10-08T02:31:28.404581+0000","last_fullsized":"2023-10-08T02:31:28.404581+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:13:02.499427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432827+0000","last_change":"2023-10-08T02:31:28.428120+0000","last_active":"2023-10-08T02:31:28.432827+0000","last_peered":"2023-10-08T02:31:28.432827+0000","last_clean":"2023-10-08T02:31:28.432827+0000","last_became_active":"2023-10-08T02:31:28.427855+0000","last_became_peered":"2023-10-08T02:31:28.427855+0000","last_unstale":"2023-10-08T02:31:28.432827+0000","last_undegraded":"2023-10-08T02:31:28.432827+0000","last_fullsized":"2023-10-08T02:31:28.432827+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:20.001128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"205'3","reported_seq":13,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.442164+0000","last_change":"2023-10-08T02:31:27.393104+0000","last_active":"2023-10-08T02:31:28.442164+0000","last_peered":"2023-10-08T02:31:28.442164+0000","last_clean":"2023-10-08T02:31:28.442164+0000","last_became_active":"2023-10-08T02:31:27.392831+0000","last_became_peered":"2023-10-08T02:31:27.392831+0000","last_unstale":"2023-10-08T02:31:28.442164+0000","last_undegraded":"2023-10-08T02:31:28.442164+0000","last_fullsized":"2023-10-08T02:31:28.442164+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:06.956710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.430290+0000","last_change":"2023-10-08T02:31:28.429302+0000","last_active":"2023-10-08T02:31:28.430290+0000","last_peered":"2023-10-08T02:31:28.430290+0000","last_clean":"2023-10-08T02:31:28.430290+0000","last_became_active":"2023-10-08T02:31:28.427775+0000","last_became_peered":"2023-10-08T02:31:28.427775+0000","last_unstale":"2023-10-08T02:31:28.430290+0000","last_undegraded":"2023-10-08T02:31:28.430290+0000","last_fullsized":"2023-10-08T02:31:28.430290+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:07:18.788976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"205'6","reported_seq":16,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.443522+0000","last_change":"2023-10-08T02:31:27.401502+0000","last_active":"2023-10-08T02:31:28.443522+0000","last_peered":"2023-10-08T02:31:28.443522+0000","last_clean":"2023-10-08T02:31:28.443522+0000","last_became_active":"2023-10-08T02:31:27.401249+0000","last_became_peered":"2023-10-08T02:31:27.401249+0000","last_unstale":"2023-10-08T02:31:28.443522+0000","last_undegraded":"2023-10-08T02:31:28.443522+0000","last_fullsized":"2023-10-08T02:31:28.443522+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:15:16.813739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.423693+0000","last_change":"2023-10-08T02:31:28.415470+0000","last_active":"2023-10-08T02:31:28.423693+0000","last_peered":"2023-10-08T02:31:28.423693+0000","last_clean":"2023-10-08T02:31:28.423693+0000","last_became_active":"2023-10-08T02:31:28.415219+0000","last_became_peered":"2023-10-08T02:31:28.415219+0000","last_unstale":"2023-10-08T02:31:28.423693+0000","last_undegraded":"2023-10-08T02:31:28.423693+0000","last_fullsized":"2023-10-08T02:31:28.423693+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:31:38.361725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.442228+0000","last_change":"2023-10-08T02:31:27.405251+0000","last_active":"2023-10-08T02:31:28.442228+0000","last_peered":"2023-10-08T02:31:28.442228+0000","last_clean":"2023-10-08T02:31:28.442228+0000","last_became_active":"2023-10-08T02:31:27.404991+0000","last_became_peered":"2023-10-08T02:31:27.404991+0000","last_unstale":"2023-10-08T02:31:28.442228+0000","last_undegraded":"2023-10-08T02:31:28.442228+0000","last_fullsized":"2023-10-08T02:31:28.442228+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:34.316615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.433560+0000","last_change":"2023-10-08T02:31:28.431299+0000","last_active":"2023-10-08T02:31:28.433560+0000","last_peered":"2023-10-08T02:31:28.433560+0000","last_clean":"2023-10-08T02:31:28.433560+0000","last_became_active":"2023-10-08T02:31:28.431054+0000","last_became_peered":"2023-10-08T02:31:28.431054+0000","last_unstale":"2023-10-08T02:31:28.433560+0000","last_undegraded":"2023-10-08T02:31:28.433560+0000","last_fullsized":"2023-10-08T02:31:28.433560+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:28.967294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.408411+0000","last_change":"2023-10-08T02:31:27.398296+0000","last_active":"2023-10-08T02:31:28.408411+0000","last_peered":"2023-10-08T02:31:28.408411+0000","last_clean":"2023-10-08T02:31:28.408411+0000","last_became_active":"2023-10-08T02:31:27.398028+0000","last_became_peered":"2023-10-08T02:31:27.398028+0000","last_unstale":"2023-10-08T02:31:28.408411+0000","last_undegraded":"2023-10-08T02:31:28.408411+0000","last_fullsized":"2023-10-08T02:31:28.408411+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:10:22.227598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.430543+0000","last_change":"2023-10-08T02:31:28.427575+0000","last_active":"2023-10-08T02:31:28.430543+0000","last_peered":"2023-10-08T02:31:28.430543+0000","last_clean":"2023-10-08T02:31:28.430543+0000","last_became_active":"2023-10-08T02:31:28.427313+0000","last_became_peered":"2023-10-08T02:31:28.427313+0000","last_unstale":"2023-10-08T02:31:28.430543+0000","last_undegraded":"2023-10-08T02:31:28.430543+0000","last_fullsized":"2023-10-08T02:31:28.430543+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:37:07.013855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.404294+0000","last_change":"2023-10-08T02:31:27.403716+0000","last_active":"2023-10-08T02:31:28.404294+0000","last_peered":"2023-10-08T02:31:28.404294+0000","last_clean":"2023-10-08T02:31:28.404294+0000","last_became_active":"2023-10-08T02:31:27.403469+0000","last_became_peered":"2023-10-08T02:31:27.403469+0000","last_unstale":"2023-10-08T02:31:28.404294+0000","last_undegraded":"2023-10-08T02:31:28.404294+0000","last_fullsized":"2023-10-08T02:31:28.404294+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:47.569228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429608+0000","last_change":"2023-10-08T02:31:28.426114+0000","last_active":"2023-10-08T02:31:28.429608+0000","last_peered":"2023-10-08T02:31:28.429608+0000","last_clean":"2023-10-08T02:31:28.429608+0000","last_became_active":"2023-10-08T02:31:28.425861+0000","last_became_peered":"2023-10-08T02:31:28.425861+0000","last_unstale":"2023-10-08T02:31:28.429608+0000","last_undegraded":"2023-10-08T02:31:28.429608+0000","last_fullsized":"2023-10-08T02:31:28.429608+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T14:01:04.293850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"205'8","reported_seq":18,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.442747+0000","last_change":"2023-10-08T02:31:27.405230+0000","last_active":"2023-10-08T02:31:28.442747+0000","last_peered":"2023-10-08T02:31:28.442747+0000","last_clean":"2023-10-08T02:31:28.442747+0000","last_became_active":"2023-10-08T02:31:27.404954+0000","last_became_peered":"2023-10-08T02:31:27.404954+0000","last_unstale":"2023-10-08T02:31:28.442747+0000","last_undegraded":"2023-10-08T02:31:28.442747+0000","last_fullsized":"2023-10-08T02:31:28.442747+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:17.212905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.425335+0000","last_change":"2023-10-08T02:31:28.421187+0000","last_active":"2023-10-08T02:31:28.425335+0000","last_peered":"2023-10-08T02:31:28.425335+0000","last_clean":"2023-10-08T02:31:28.425335+0000","last_became_active":"2023-10-08T02:31:28.420897+0000","last_became_peered":"2023-10-08T02:31:28.420897+0000","last_unstale":"2023-10-08T02:31:28.425335+0000","last_undegraded":"2023-10-08T02:31:28.425335+0000","last_fullsized":"2023-10-08T02:31:28.425335+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:15:31.873060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.439840+0000","last_change":"2023-10-08T02:31:27.401709+0000","last_active":"2023-10-08T02:31:28.439840+0000","last_peered":"2023-10-08T02:31:28.439840+0000","last_clean":"2023-10-08T02:31:28.439840+0000","last_became_active":"2023-10-08T02:31:27.401456+0000","last_became_peered":"2023-10-08T02:31:27.401456+0000","last_unstale":"2023-10-08T02:31:28.439840+0000","last_undegraded":"2023-10-08T02:31:28.439840+0000","last_fullsized":"2023-10-08T02:31:28.439840+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:12.040775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.431336+0000","last_change":"2023-10-08T02:31:28.429289+0000","last_active":"2023-10-08T02:31:28.431336+0000","last_peered":"2023-10-08T02:31:28.431336+0000","last_clean":"2023-10-08T02:31:28.431336+0000","last_became_active":"2023-10-08T02:31:28.428776+0000","last_became_peered":"2023-10-08T02:31:28.428776+0000","last_unstale":"2023-10-08T02:31:28.431336+0000","last_undegraded":"2023-10-08T02:31:28.431336+0000","last_fullsized":"2023-10-08T02:31:28.431336+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:31:38.361725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.436482+0000","last_change":"2023-10-08T02:31:27.389939+0000","last_active":"2023-10-08T02:31:28.436482+0000","last_peered":"2023-10-08T02:31:28.436482+0000","last_clean":"2023-10-08T02:31:28.436482+0000","last_became_active":"2023-10-08T02:31:27.389684+0000","last_became_peered":"2023-10-08T02:31:27.389684+0000","last_unstale":"2023-10-08T02:31:28.436482+0000","last_undegraded":"2023-10-08T02:31:28.436482+0000","last_fullsized":"2023-10-08T02:31:28.436482+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:53:01.613685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427319+0000","last_change":"2023-10-08T02:31:28.424410+0000","last_active":"2023-10-08T02:31:28.427319+0000","last_peered":"2023-10-08T02:31:28.427319+0000","last_clean":"2023-10-08T02:31:28.427319+0000","last_became_active":"2023-10-08T02:31:28.424121+0000","last_became_peered":"2023-10-08T02:31:28.424121+0000","last_unstale":"2023-10-08T02:31:28.427319+0000","last_undegraded":"2023-10-08T02:31:28.427319+0000","last_fullsized":"2023-10-08T02:31:28.427319+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:36.768718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"205'1","reported_seq":11,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.442141+0000","last_change":"2023-10-08T02:31:27.405913+0000","last_active":"2023-10-08T02:31:28.442141+0000","last_peered":"2023-10-08T02:31:28.442141+0000","last_clean":"2023-10-08T02:31:28.442141+0000","last_became_active":"2023-10-08T02:31:27.405587+0000","last_became_peered":"2023-10-08T02:31:27.405587+0000","last_unstale":"2023-10-08T02:31:28.442141+0000","last_undegraded":"2023-10-08T02:31:28.442141+0000","last_fullsized":"2023-10-08T02:31:28.442141+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:26:47.569228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427691+0000","last_change":"2023-10-08T02:31:28.425554+0000","last_active":"2023-10-08T02:31:28.427691+0000","last_peered":"2023-10-08T02:31:28.427691+0000","last_clean":"2023-10-08T02:31:28.427691+0000","last_became_active":"2023-10-08T02:31:28.425292+0000","last_became_peered":"2023-10-08T02:31:28.425292+0000","last_unstale":"2023-10-08T02:31:28.427691+0000","last_undegraded":"2023-10-08T02:31:28.427691+0000","last_fullsized":"2023-10-08T02:31:28.427691+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:40:18.356274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.411284+0000","last_change":"2023-10-08T02:31:27.402673+0000","last_active":"2023-10-08T02:31:28.411284+0000","last_peered":"2023-10-08T02:31:28.411284+0000","last_clean":"2023-10-08T02:31:28.411284+0000","last_became_active":"2023-10-08T02:31:27.402392+0000","last_became_peered":"2023-10-08T02:31:27.402392+0000","last_unstale":"2023-10-08T02:31:28.411284+0000","last_undegraded":"2023-10-08T02:31:28.411284+0000","last_fullsized":"2023-10-08T02:31:28.411284+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:53:17.212905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.418246+0000","last_change":"2023-10-08T02:31:28.411868+0000","last_active":"2023-10-08T02:31:28.418246+0000","last_peered":"2023-10-08T02:31:28.418246+0000","last_clean":"2023-10-08T02:31:28.418246+0000","last_became_active":"2023-10-08T02:31:28.411624+0000","last_became_peered":"2023-10-08T02:31:28.411624+0000","last_unstale":"2023-10-08T02:31:28.418246+0000","last_undegraded":"2023-10-08T02:31:28.418246+0000","last_fullsized":"2023-10-08T02:31:28.418246+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:17:48.957331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.441851+0000","last_change":"2023-10-08T02:31:27.403154+0000","last_active":"2023-10-08T02:31:28.441851+0000","last_peered":"2023-10-08T02:31:28.441851+0000","last_clean":"2023-10-08T02:31:28.441851+0000","last_became_active":"2023-10-08T02:31:27.402875+0000","last_became_peered":"2023-10-08T02:31:27.402875+0000","last_unstale":"2023-10-08T02:31:28.441851+0000","last_undegraded":"2023-10-08T02:31:28.441851+0000","last_fullsized":"2023-10-08T02:31:28.441851+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:55:59.104676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432785+0000","last_change":"2023-10-08T02:31:28.428476+0000","last_active":"2023-10-08T02:31:28.432785+0000","last_peered":"2023-10-08T02:31:28.432785+0000","last_clean":"2023-10-08T02:31:28.432785+0000","last_became_active":"2023-10-08T02:31:28.428207+0000","last_became_peered":"2023-10-08T02:31:28.428207+0000","last_unstale":"2023-10-08T02:31:28.432785+0000","last_undegraded":"2023-10-08T02:31:28.432785+0000","last_fullsized":"2023-10-08T02:31:28.432785+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:34:36.050932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.403643+0000","last_change":"2023-10-08T02:31:27.402666+0000","last_active":"2023-10-08T02:31:28.403643+0000","last_peered":"2023-10-08T02:31:28.403643+0000","last_clean":"2023-10-08T02:31:28.403643+0000","last_became_active":"2023-10-08T02:31:27.402417+0000","last_became_peered":"2023-10-08T02:31:27.402417+0000","last_unstale":"2023-10-08T02:31:28.403643+0000","last_undegraded":"2023-10-08T02:31:28.403643+0000","last_fullsized":"2023-10-08T02:31:28.403643+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:46:40.216786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426433+0000","last_change":"2023-10-08T02:31:28.420775+0000","last_active":"2023-10-08T02:31:28.426433+0000","last_peered":"2023-10-08T02:31:28.426433+0000","last_clean":"2023-10-08T02:31:28.426433+0000","last_became_active":"2023-10-08T02:31:28.420511+0000","last_became_peered":"2023-10-08T02:31:28.420511+0000","last_unstale":"2023-10-08T02:31:28.426433+0000","last_undegraded":"2023-10-08T02:31:28.426433+0000","last_fullsized":"2023-10-08T02:31:28.426433+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T10:09:01.017826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.410244+0000","last_change":"2023-10-08T02:31:27.400758+0000","last_active":"2023-10-08T02:31:28.410244+0000","last_peered":"2023-10-08T02:31:28.410244+0000","last_clean":"2023-10-08T02:31:28.410244+0000","last_became_active":"2023-10-08T02:31:27.400509+0000","last_became_peered":"2023-10-08T02:31:27.400509+0000","last_unstale":"2023-10-08T02:31:28.410244+0000","last_undegraded":"2023-10-08T02:31:28.410244+0000","last_fullsized":"2023-10-08T02:31:28.410244+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:06.956710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.419928+0000","last_change":"2023-10-08T02:31:28.416027+0000","last_active":"2023-10-08T02:31:28.419928+0000","last_peered":"2023-10-08T02:31:28.419928+0000","last_clean":"2023-10-08T02:31:28.419928+0000","last_became_active":"2023-10-08T02:31:28.415785+0000","last_became_peered":"2023-10-08T02:31:28.415785+0000","last_unstale":"2023-10-08T02:31:28.419928+0000","last_undegraded":"2023-10-08T02:31:28.419928+0000","last_fullsized":"2023-10-08T02:31:28.419928+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:37:35.322154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.413368+0000","last_change":"2023-10-08T02:31:27.402671+0000","last_active":"2023-10-08T02:31:28.413368+0000","last_peered":"2023-10-08T02:31:28.413368+0000","last_clean":"2023-10-08T02:31:28.413368+0000","last_became_active":"2023-10-08T02:31:27.402398+0000","last_became_peered":"2023-10-08T02:31:27.402398+0000","last_unstale":"2023-10-08T02:31:28.413368+0000","last_undegraded":"2023-10-08T02:31:28.413368+0000","last_fullsized":"2023-10-08T02:31:28.413368+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:12.040775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.424001+0000","last_change":"2023-10-08T02:31:28.416694+0000","last_active":"2023-10-08T02:31:28.424001+0000","last_peered":"2023-10-08T02:31:28.424001+0000","last_clean":"2023-10-08T02:31:28.424001+0000","last_became_active":"2023-10-08T02:31:28.416476+0000","last_became_peered":"2023-10-08T02:31:28.416476+0000","last_unstale":"2023-10-08T02:31:28.424001+0000","last_undegraded":"2023-10-08T02:31:28.424001+0000","last_fullsized":"2023-10-08T02:31:28.424001+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:56:42.759676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"205'2","reported_seq":12,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.439927+0000","last_change":"2023-10-08T02:31:27.398449+0000","last_active":"2023-10-08T02:31:28.439927+0000","last_peered":"2023-10-08T02:31:28.439927+0000","last_clean":"2023-10-08T02:31:28.439927+0000","last_became_active":"2023-10-08T02:31:27.398126+0000","last_became_peered":"2023-10-08T02:31:27.398126+0000","last_unstale":"2023-10-08T02:31:28.439927+0000","last_undegraded":"2023-10-08T02:31:28.439927+0000","last_fullsized":"2023-10-08T02:31:28.439927+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:43:35.763180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429705+0000","last_change":"2023-10-08T02:31:28.426625+0000","last_active":"2023-10-08T02:31:28.429705+0000","last_peered":"2023-10-08T02:31:28.429705+0000","last_clean":"2023-10-08T02:31:28.429705+0000","last_became_active":"2023-10-08T02:31:28.426286+0000","last_became_peered":"2023-10-08T02:31:28.426286+0000","last_unstale":"2023-10-08T02:31:28.429705+0000","last_undegraded":"2023-10-08T02:31:28.429705+0000","last_fullsized":"2023-10-08T02:31:28.429705+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T07:34:57.675320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.408210+0000","last_change":"2023-10-08T02:31:27.403669+0000","last_active":"2023-10-08T02:31:28.408210+0000","last_peered":"2023-10-08T02:31:28.408210+0000","last_clean":"2023-10-08T02:31:28.408210+0000","last_became_active":"2023-10-08T02:31:27.403359+0000","last_became_peered":"2023-10-08T02:31:27.403359+0000","last_unstale":"2023-10-08T02:31:28.408210+0000","last_undegraded":"2023-10-08T02:31:28.408210+0000","last_fullsized":"2023-10-08T02:31:28.408210+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T05:04:22.618276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427626+0000","last_change":"2023-10-08T02:31:28.417570+0000","last_active":"2023-10-08T02:31:28.427626+0000","last_peered":"2023-10-08T02:31:28.427626+0000","last_clean":"2023-10-08T02:31:28.427626+0000","last_became_active":"2023-10-08T02:31:28.417313+0000","last_became_peered":"2023-10-08T02:31:28.417313+0000","last_unstale":"2023-10-08T02:31:28.427626+0000","last_undegraded":"2023-10-08T02:31:28.427626+0000","last_fullsized":"2023-10-08T02:31:28.427626+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:32:28.967294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.408839+0000","last_change":"2023-10-08T02:31:27.405779+0000","last_active":"2023-10-08T02:31:28.408839+0000","last_peered":"2023-10-08T02:31:28.408839+0000","last_clean":"2023-10-08T02:31:28.408839+0000","last_became_active":"2023-10-08T02:31:27.405500+0000","last_became_peered":"2023-10-08T02:31:27.405500+0000","last_unstale":"2023-10-08T02:31:28.408839+0000","last_undegraded":"2023-10-08T02:31:28.408839+0000","last_fullsized":"2023-10-08T02:31:28.408839+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:59:11.572027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.427572+0000","last_change":"2023-10-08T02:31:28.416038+0000","last_active":"2023-10-08T02:31:28.427572+0000","last_peered":"2023-10-08T02:31:28.427572+0000","last_clean":"2023-10-08T02:31:28.427572+0000","last_became_active":"2023-10-08T02:31:28.415770+0000","last_became_peered":"2023-10-08T02:31:28.415770+0000","last_unstale":"2023-10-08T02:31:28.427572+0000","last_undegraded":"2023-10-08T02:31:28.427572+0000","last_fullsized":"2023-10-08T02:31:28.427572+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T02:35:20.001128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.405557+0000","last_change":"2023-10-08T02:31:27.390133+0000","last_active":"2023-10-08T02:31:28.405557+0000","last_peered":"2023-10-08T02:31:28.405557+0000","last_clean":"2023-10-08T02:31:28.405557+0000","last_became_active":"2023-10-08T02:31:27.389866+0000","last_became_peered":"2023-10-08T02:31:27.389866+0000","last_unstale":"2023-10-08T02:31:28.405557+0000","last_undegraded":"2023-10-08T02:31:28.405557+0000","last_fullsized":"2023-10-08T02:31:28.405557+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T12:10:54.894763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432719+0000","last_change":"2023-10-08T02:31:28.424856+0000","last_active":"2023-10-08T02:31:28.432719+0000","last_peered":"2023-10-08T02:31:28.432719+0000","last_clean":"2023-10-08T02:31:28.432719+0000","last_became_active":"2023-10-08T02:31:28.424588+0000","last_became_peered":"2023-10-08T02:31:28.424588+0000","last_unstale":"2023-10-08T02:31:28.432719+0000","last_undegraded":"2023-10-08T02:31:28.432719+0000","last_fullsized":"2023-10-08T02:31:28.432719+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T09:34:36.050932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.414398+0000","last_change":"2023-10-08T02:31:27.401864+0000","last_active":"2023-10-08T02:31:28.414398+0000","last_peered":"2023-10-08T02:31:28.414398+0000","last_clean":"2023-10-08T02:31:28.414398+0000","last_became_active":"2023-10-08T02:31:27.401513+0000","last_became_peered":"2023-10-08T02:31:27.401513+0000","last_unstale":"2023-10-08T02:31:28.414398+0000","last_undegraded":"2023-10-08T02:31:28.414398+0000","last_fullsized":"2023-10-08T02:31:28.414398+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:40:54.638239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.432921+0000","last_change":"2023-10-08T02:31:28.428460+0000","last_active":"2023-10-08T02:31:28.432921+0000","last_peered":"2023-10-08T02:31:28.432921+0000","last_clean":"2023-10-08T02:31:28.432921+0000","last_became_active":"2023-10-08T02:31:28.428200+0000","last_became_peered":"2023-10-08T02:31:28.428200+0000","last_unstale":"2023-10-08T02:31:28.432921+0000","last_undegraded":"2023-10-08T02:31:28.432921+0000","last_fullsized":"2023-10-08T02:31:28.432921+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:55:24.799351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.413624+0000","last_change":"2023-10-08T02:31:27.398581+0000","last_active":"2023-10-08T02:31:28.413624+0000","last_peered":"2023-10-08T02:31:28.413624+0000","last_clean":"2023-10-08T02:31:28.413624+0000","last_became_active":"2023-10-08T02:31:27.397628+0000","last_became_peered":"2023-10-08T02:31:27.397628+0000","last_unstale":"2023-10-08T02:31:28.413624+0000","last_undegraded":"2023-10-08T02:31:28.413624+0000","last_fullsized":"2023-10-08T02:31:28.413624+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:00:07.107562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.426414+0000","last_change":"2023-10-08T02:31:28.417585+0000","last_active":"2023-10-08T02:31:28.426414+0000","last_peered":"2023-10-08T02:31:28.426414+0000","last_clean":"2023-10-08T02:31:28.426414+0000","last_became_active":"2023-10-08T02:31:28.417336+0000","last_became_peered":"2023-10-08T02:31:28.417336+0000","last_unstale":"2023-10-08T02:31:28.426414+0000","last_undegraded":"2023-10-08T02:31:28.426414+0000","last_fullsized":"2023-10-08T02:31:28.426414+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T04:59:03.213603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.411845+0000","last_change":"2023-10-08T02:31:27.399585+0000","last_active":"2023-10-08T02:31:28.411845+0000","last_peered":"2023-10-08T02:31:28.411845+0000","last_clean":"2023-10-08T02:31:28.411845+0000","last_became_active":"2023-10-08T02:31:27.399306+0000","last_became_peered":"2023-10-08T02:31:27.399306+0000","last_unstale":"2023-10-08T02:31:28.411845+0000","last_undegraded":"2023-10-08T02:31:28.411845+0000","last_fullsized":"2023-10-08T02:31:28.411845+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T06:05:12.327087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.429955+0000","last_change":"2023-10-08T02:31:28.424529+0000","last_active":"2023-10-08T02:31:28.429955+0000","last_peered":"2023-10-08T02:31:28.429955+0000","last_clean":"2023-10-08T02:31:28.429955+0000","last_became_active":"2023-10-08T02:31:28.424231+0000","last_became_peered":"2023-10-08T02:31:28.424231+0000","last_unstale":"2023-10-08T02:31:28.429955+0000","last_undegraded":"2023-10-08T02:31:28.429955+0000","last_fullsized":"2023-10-08T02:31:28.429955+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:29:41.586057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.410374+0000","last_change":"2023-10-08T02:31:27.402362+0000","last_active":"2023-10-08T02:31:28.410374+0000","last_peered":"2023-10-08T02:31:28.410374+0000","last_clean":"2023-10-08T02:31:28.410374+0000","last_became_active":"2023-10-08T02:31:27.402094+0000","last_became_peered":"2023-10-08T02:31:27.402094+0000","last_unstale":"2023-10-08T02:31:28.410374+0000","last_undegraded":"2023-10-08T02:31:28.410374+0000","last_fullsized":"2023-10-08T02:31:28.410374+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:31:12.040775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.425326+0000","last_change":"2023-10-08T02:31:28.420972+0000","last_active":"2023-10-08T02:31:28.425326+0000","last_peered":"2023-10-08T02:31:28.425326+0000","last_clean":"2023-10-08T02:31:28.425326+0000","last_became_active":"2023-10-08T02:31:28.420616+0000","last_became_peered":"2023-10-08T02:31:28.420616+0000","last_unstale":"2023-10-08T02:31:28.425326+0000","last_undegraded":"2023-10-08T02:31:28.425326+0000","last_fullsized":"2023-10-08T02:31:28.425326+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":204,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:27.377083+0000","last_clean_scrub_stamp":"2023-10-08T02:31:27.377083+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T08:14:28.782396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.408479+0000","last_change":"2023-10-08T02:31:27.404626+0000","last_active":"2023-10-08T02:31:28.408479+0000","last_peered":"2023-10-08T02:31:28.408479+0000","last_clean":"2023-10-08T02:31:28.408479+0000","last_became_active":"2023-10-08T02:31:27.404343+0000","last_became_peered":"2023-10-08T02:31:27.404343+0000","last_unstale":"2023-10-08T02:31:28.408479+0000","last_undegraded":"2023-10-08T02:31:28.408479+0000","last_fullsized":"2023-10-08T02:31:28.408479+0000","mapping_epoch":203,"log_start":"0'0","ondisk_log_start":"0'0","created":203,"last_epoch_clean":204,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T02:31:26.371545+0000","last_clean_scrub_stamp":"2023-10-08T02:31:26.371545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T11:25:01.848399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"195'348","reported_seq":1140,"reported_epoch":205,"state":"active+clean","last_fresh":"2023-10-08T02:31:28.409303+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:31:28.409303+0000","last_peered":"2023-10-08T02:31:28.409303+0000","last_clean":"2023-10-08T02:31:28.409303+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:31:28.409303+0000","last_undegraded":"2023-10-08T02:31:28.409303+0000","last_fullsized":"2023-10-08T02:31:28.409303+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":348,"log_dups_size":0,"ondisk_log_size":348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2132,"num_read_kb":4426,"num_write":579,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575984,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316780,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055060,"statfs":{"total":96636764160,"available":96312381440,"internally_reserved":0,"allocated":3182592,"data_stored":1892939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73999999999999999,"15min":0.74199999999999999},"min":{"1min":0.54900000000000004,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.876,"5min":0.96899999999999997,"15min":1.0429999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.746},"min":{"1min":0.55900000000000005,"5min":0.51200000000000001,"15min":0.503},"max":{"1min":0.93300000000000005,"5min":0.97399999999999998,"15min":1.052},"last":0.58999999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.749,"15min":0.74199999999999999},"min":{"1min":0.64000000000000001,"5min":0.42199999999999999,"15min":0.40600000000000003},"max":{"1min":0.91900000000000004,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.91200000000000003,"5min":0.98499999999999999,"15min":1.002},"last":0.66500000000000004}]},{"osd":2,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.61499999999999999,"5min":0.55200000000000005,"15min":0.495},"max":{"1min":0.95799999999999996,"5min":0.96699999999999997,"15min":1.0549999999999999},"last":0.623},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.76100000000000001,"15min":0.755},"min":{"1min":0.58499999999999996,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93000000000000005,"5min":0.98499999999999999,"15min":1.03},"last":0.71799999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.72299999999999998,"15min":0.71799999999999997},"min":{"1min":0.57999999999999996,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.79100000000000004,"15min":0.78800000000000003},"min":{"1min":0.66600000000000004,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98999999999999999,"5min":1.0069999999999999,"15min":1.056},"last":0.73699999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.59799999999999998,"15min":0.60299999999999998},"min":{"1min":0.39600000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.754,"5min":0.89500000000000002,"15min":1.091},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.38100000000000001},"max":{"1min":0.79400000000000004,"5min":0.88800000000000001,"15min":0.92200000000000004},"last":0.78300000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.64700000000000002,"15min":0.64200000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.39300000000000002},"max":{"1min":0.91500000000000004,"5min":0.92100000000000004,"15min":1.012},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67000000000000004,"15min":0.66900000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.44500000000000001},"max":{"1min":0.86099999999999999,"5min":0.92100000000000004,"15min":1.1240000000000001},"last":0.76500000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:30:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.77600000000000002,"15min":0.77500000000000002},"min":{"1min":0.60699999999999998,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.96999999999999997,"5min":1.0089999999999999,"15min":1.151},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70999999999999996,"15min":0.71499999999999997},"min":{"1min":0.52500000000000002,"5min":0.441,"15min":0.42599999999999999},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.749}]}]},{"osd":6,"up_from":13,"seq":55834575984,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299808,"kb_used_data":2520,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072032,"statfs":{"total":96636764160,"available":96329760768,"internally_reserved":0,"allocated":2580480,"data_stored":1302467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.54700000000000004,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95999999999999996,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.72599999999999998,"15min":0.73099999999999998},"min":{"1min":0.51100000000000001,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.96199999999999997,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.57799999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.71599999999999997,"15min":0.74199999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":1.21},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.75900000000000001,"15min":0.752},"min":{"1min":0.504,"5min":0.504,"15min":0.45700000000000002},"max":{"1min":0.90000000000000002,"5min":3.851,"15min":3.851},"last":0.70699999999999996}]},{"osd":2,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.68999999999999995,"15min":0.70899999999999996},"min":{"1min":0.501,"5min":0.47599999999999998,"15min":0.44400000000000001},"max":{"1min":0.77900000000000003,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73899999999999999,"15min":0.76100000000000001},"min":{"1min":0.57599999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.88200000000000001,"5min":0.91000000000000003,"15min":1.258},"last":0.69199999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.78200000000000003,"15min":0.78400000000000003},"min":{"1min":0.56499999999999995,"5min":0.54900000000000004,"15min":0.52600000000000002},"max":{"1min":0.93600000000000005,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.76300000000000001,"15min":0.77500000000000002},"min":{"1min":0.57899999999999996,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.872,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.58999999999999997}]},{"osd":4,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.40100000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.89000000000000001,"5min":1.139,"15min":1.139},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.47299999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.876,"5min":1.341,"15min":1.341},"last":0.64000000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.92800000000000005},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.45100000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93000000000000005,"5min":1.089,"15min":1.089},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:30:28 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.54200000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.91100000000000003,"5min":1.038,"15min":1.0980000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.47499999999999998,"5min":0.437,"15min":0.437},"max":{"1min":0.94199999999999995,"5min":0.96999999999999997,"15min":1.054},"last":0.626}]}]},{"osd":1,"up_from":13,"seq":55834575984,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312108,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059732,"statfs":{"total":96636764160,"available":96317165568,"internally_reserved":0,"allocated":2592768,"data_stored":1304003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.0780000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.29199999999999998,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.571},"last":0.31900000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":1.1100000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":1.1299999999999999},"last":0.35499999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.755,"5min":0.80200000000000005,"15min":1.2190000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":1.341,"15min":1.341},"last":0.374}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.48199999999999998},"max":{"1min":1.171,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.629},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.67800000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.97499999999999998,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.127,"15min":1.27},"last":0.75800000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.81799999999999995},"min":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.915,"15min":1.915},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.50900000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.204},"last":0.78800000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.85799999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.306},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.66800000000000004,"5min":0.57999999999999996,"15min":0.55500000000000005},"max":{"1min":1.1399999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.77500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575983,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308584,"kb_used_data":3104,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063256,"statfs":{"total":96636764160,"available":96320774144,"internally_reserved":0,"allocated":3178496,"data_stored":1892857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.32600000000000001,"5min":0.25900000000000001,"15min":0.23699999999999999},"max":{"1min":0.66600000000000004,"5min":0.77700000000000002,"15min":0.78400000000000003},"last":0.377},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.38900000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.72599999999999998,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.57999999999999996}]},{"osd":2,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48899999999999999},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.312},"max":{"1min":0.71299999999999997,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.432,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79400000000000004,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.49199999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.55000000000000004,"15min":0.55800000000000005},"min":{"1min":0.39400000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.752,"5min":0.81799999999999995,"15min":0.93100000000000005},"last":0.624},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.60899999999999999},"min":{"1min":0.498,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.81399999999999995,"5min":0.83199999999999996,"15min":0.89000000000000001},"last":0.60999999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.77800000000000002,"15min":0.78900000000000003},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.159,"5min":1.232,"15min":1.232},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.66800000000000004}]},{"osd":5,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.84099999999999997,"15min":0.84199999999999997},"min":{"1min":0.59499999999999997,"5min":0.57899999999999996,"15min":0.55700000000000005},"max":{"1min":1.0960000000000001,"5min":1.1990000000000001,"15min":1.216},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.86799999999999999,"15min":0.874},"min":{"1min":0.59099999999999997,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.27,"5min":1.296,"15min":1.296},"last":0.68500000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.82799999999999996,"15min":0.83299999999999996},"min":{"1min":0.57999999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.1579999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.80900000000000005,"15min":0.80400000000000005},"min":{"1min":0.71299999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.1990000000000001,"5min":1.262,"15min":1.262},"last":0.91200000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:30:49 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.84199999999999997,"15min":0.84499999999999997},"min":{"1min":0.63500000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.171,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.86699999999999999,"15min":0.875},"min":{"1min":0.64400000000000002,"5min":0.58999999999999997,"15min":0.55700000000000005},"max":{"1min":1.2250000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"last":0.73399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316204,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055636,"statfs":{"total":96636764160,"available":96312971264,"internally_reserved":0,"allocated":2592768,"data_stored":1303133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47699999999999998,"15min":0.47399999999999998},"min":{"1min":0.33500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.871},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.52100000000000002,"15min":0.51300000000000001},"min":{"1min":0.378,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66500000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.48799999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47799999999999998,"15min":0.48199999999999998},"min":{"1min":0.33200000000000002,"5min":0.314,"15min":0.28699999999999998},"max":{"1min":0.66400000000000003,"5min":0.876,"15min":0.876},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.43099999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.78900000000000003,"15min":0.80900000000000005},"last":0.44600000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.628,"15min":0.621},"min":{"1min":0.49299999999999999,"5min":0.45000000000000001,"15min":0.42799999999999999},"max":{"1min":0.86199999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.503},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.57899999999999996,"15min":0.56999999999999995},"min":{"1min":0.42999999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76400000000000001,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.46600000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77200000000000002,"15min":0.78300000000000003},"min":{"1min":0.55200000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.90000000000000002,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.59099999999999997,"5min":0.503,"15min":0.49099999999999999},"max":{"1min":0.92600000000000005,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72199999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.83099999999999996,"15min":0.83399999999999996},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.46400000000000002},"max":{"1min":0.97399999999999998,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.83799999999999997,"15min":0.84899999999999998},"min":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.54900000000000004},"max":{"1min":0.99399999999999999,"5min":1.353,"15min":1.417},"last":0.81599999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.80900000000000005,"15min":0.81799999999999995},"min":{"1min":0.627,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.034,"5min":1.153,"15min":1.333},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.81200000000000006,"15min":0.81799999999999995},"min":{"1min":0.63500000000000001,"5min":0.53000000000000003,"15min":0.51800000000000002},"max":{"1min":0.94799999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.748}]},{"osd":7,"last update":"Sun Oct 8 02:30:48 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.58499999999999996,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.96199999999999997,"5min":1.387,"15min":1.4059999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.82999999999999996,"15min":0.83599999999999997},"min":{"1min":0.61799999999999999,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0880000000000001,"5min":1.252,"15min":1.5720000000000001},"last":0.85899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575983,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299820,"kb_used_data":2532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072020,"statfs":{"total":96636764160,"available":96329748480,"internally_reserved":0,"allocated":2592768,"data_stored":1302661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.35199999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.90300000000000002},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.505,"15min":0.5},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.311},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.35899999999999999}]},{"osd":1,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.495,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.68600000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.42899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.876,"5min":1.012,"15min":1.012},"last":0.378}]},{"osd":2,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.78700000000000003,"5min":0.879,"15min":1.282},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63800000000000001},"min":{"1min":0.505,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.71699999999999997,"5min":0.89200000000000002,"15min":1.512},"last":0.51800000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.77000000000000002,"15min":0.77400000000000002},"min":{"1min":0.52100000000000002,"5min":0.442,"15min":0.442},"max":{"1min":1.012,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.625},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.79900000000000004,"15min":0.80100000000000005},"min":{"1min":0.56000000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.95699999999999996,"5min":1.1399999999999999,"15min":1.3180000000000001},"last":0.64400000000000002}]},{"osd":5,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.51000000000000001,"5min":0.441,"15min":0.441},"max":{"1min":1.014,"5min":1.1160000000000001,"15min":1.2789999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.81299999999999994,"15min":0.81100000000000005},"min":{"1min":0.69799999999999995,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.98199999999999998,"5min":1.1220000000000001,"15min":1.24},"last":0.78700000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.79800000000000004,"15min":0.81599999999999995},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0269999999999999,"5min":1.0880000000000001,"15min":1.161},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80800000000000005,"15min":0.82099999999999995},"min":{"1min":0.63700000000000001,"5min":0.5,"15min":0.5},"max":{"1min":0.94099999999999995,"5min":1.155,"15min":1.2849999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:30:41 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.81699999999999995,"15min":0.83499999999999996},"min":{"1min":0.59499999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.96899999999999997,"5min":1.129,"15min":1.266},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.82399999999999995,"15min":0.83199999999999996},"min":{"1min":0.53700000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.98699999999999999,"5min":1.1259999999999999,"15min":1.282},"last":0.59299999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299824,"kb_used_data":2536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94072016,"statfs":{"total":96636764160,"available":96329744384,"internally_reserved":0,"allocated":2596864,"data_stored":1304557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.028},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.497,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90800000000000003,"5min":0.998,"15min":1.0409999999999999},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.51400000000000001,"5min":0.496,"15min":0.49099999999999999},"max":{"1min":1.01,"5min":1.079,"15min":1.079},"last":0.72799999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.034},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.65400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.022,"5min":1.097,"15min":1.097},"last":0.74099999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98399999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.82599999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.1259999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.65600000000000003,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":1.004,"5min":1.0269999999999999,"15min":1.105},"last":0.77500000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":0.98299999999999998,"15min":1.0389999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.52900000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92300000000000004,"5min":1.03,"15min":1.03},"last":0.68000000000000005}]},{"osd":7,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.68899999999999995},"min":{"1min":0.55300000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98099999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.42699999999999999},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0669999999999999},"last":0.874}]}]},{"osd":5,"up_from":13,"seq":55834575983,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316192,"kb_used_data":2520,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055648,"statfs":{"total":96636764160,"available":96312983552,"internally_reserved":0,"allocated":2580480,"data_stored":1302467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.71599999999999997,"15min":0.72699999999999998},"min":{"1min":0.53000000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93899999999999995,"5min":0.996,"15min":0.996},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.73899999999999999,"15min":0.748},"min":{"1min":0.59399999999999997,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.84199999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72299999999999998,"15min":0.73699999999999999},"min":{"1min":0.55400000000000005,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91400000000000003,"5min":0.97399999999999998,"15min":0.99199999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.61399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.0049999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.77800000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.75600000000000001,"15min":0.75800000000000001},"min":{"1min":0.54100000000000004,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.97699999999999998,"5min":1.028,"15min":1.028},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.748,"15min":0.76100000000000001},"min":{"1min":0.58099999999999996,"5min":0.50900000000000001,"15min":0.47099999999999997},"max":{"1min":0.96299999999999997,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69699999999999995}]},{"osd":3,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.752,"15min":0.76200000000000001},"min":{"1min":0.51900000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.77400000000000002,"15min":0.77800000000000002},"min":{"1min":0.57299999999999995,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0329999999999999,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.86199999999999999}]},{"osd":4,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.61399999999999999},"min":{"1min":0.45400000000000001,"5min":0.38100000000000001,"15min":0.374},"max":{"1min":0.90300000000000002,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.64000000000000001,"15min":0.65700000000000003},"min":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.73099999999999998,"5min":0.93500000000000005,"15min":1.0449999999999999},"last":0.91000000000000003}]},{"osd":6,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.67900000000000005,"15min":0.67800000000000005},"min":{"1min":0.496,"5min":0.46899999999999997,"15min":0.434},"max":{"1min":1.0009999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.67800000000000005,"15min":0.68400000000000005},"min":{"1min":0.47399999999999998,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:30:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69699999999999995,"15min":0.71399999999999997},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.044},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.73599999999999999,"15min":0.753},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.069,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:31:33.186 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-08T02:31:33.189 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-08T02:31:33.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.407+0000 7fb374fff700 1 Processor -- start 2023-10-08T02:31:33.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.408+0000 7fb374fff700 1 -- start start 2023-10-08T02:31:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb374fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb3700c85c0 0x7fb37006d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb374fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb370069fa0 0x7fb3700c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb374fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb370156340 con 0x7fb370156ee0 2023-10-08T02:31:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb374fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3700c8990 con 0x7fb3700c85c0 2023-10-08T02:31:33.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb374fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37006ee20 con 0x7fb370069fa0 2023-10-08T02:31:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36e59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb3700c85c0 0x7fb37006d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36e59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb3700c85c0 0x7fb37006d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57636/0 (socket says 172.21.15.119:57636) 2023-10-08T02:31:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36e59c700 1 -- 172.21.15.119:0/1315522428 learned_addr learned my addr 172.21.15.119:0/1315522428 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb367fff700 1 --1- 172.21.15.119:0/1315522428 >> v1:172.21.15.119:6789/0 conn(0x7fb370156ee0 0x7fb370069890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47080/0 (socket says 172.21.15.119:47080) 2023-10-08T02:31:33.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36ed9d700 1 --2- 172.21.15.119:0/1315522428 >> v2:172.21.15.184:3300/0 conn(0x7fb370069fa0 0x7fb3700c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36e59c700 1 -- 172.21.15.119:0/1315522428 >> v2:172.21.15.184:3300/0 conn(0x7fb370069fa0 msgr2=0x7fb3700c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36e59c700 1 --2- 172.21.15.119:0/1315522428 >> v2:172.21.15.184:3300/0 conn(0x7fb370069fa0 0x7fb3700c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36e59c700 1 -- 172.21.15.119:0/1315522428 >> v1:172.21.15.119:6789/0 conn(0x7fb370156ee0 legacy=0x7fb370069890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36dd9b700 1 -- 172.21.15.119:0/1315522428 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345432930 0 0) 0x7fb370156340 con 0x7fb370156ee0 2023-10-08T02:31:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.409+0000 7fb36e59c700 1 -- 172.21.15.119:0/1315522428 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37006dd80 con 0x7fb3700c85c0 2023-10-08T02:31:33.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.410+0000 7fb36e59c700 1 --2- 172.21.15.119:0/1315522428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb37006d6c0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fb370069470 tx=0x7fb36000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5226e3aa25bf2801 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.410+0000 7fb36dd9b700 1 -- 172.21.15.119:0/1315522428 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36000dc50 con 0x7fb3700c85c0 2023-10-08T02:31:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.410+0000 7fb36dd9b700 1 -- 172.21.15.119:0/1315522428 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36000ddb0 con 0x7fb3700c85c0 2023-10-08T02:31:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.410+0000 7fb36dd9b700 1 -- 172.21.15.119:0/1315522428 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3600196d0 con 0x7fb3700c85c0 2023-10-08T02:31:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.410+0000 7fb374fff700 1 -- 172.21.15.119:0/1315522428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 msgr2=0x7fb37006d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.410+0000 7fb374fff700 1 --2- 172.21.15.119:0/1315522428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb37006d6c0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fb370069470 tx=0x7fb36000b4f0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.411+0000 7fb374fff700 1 -- 172.21.15.119:0/1315522428 shutdown_connections 2023-10-08T02:31:33.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.411+0000 7fb374fff700 1 --2- 172.21.15.119:0/1315522428 >> v2:172.21.15.184:3300/0 conn(0x7fb370069fa0 0x7fb3700c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.411+0000 7fb374fff700 1 --2- 172.21.15.119:0/1315522428 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb37006d6c0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.411+0000 7fb374fff700 1 -- 172.21.15.119:0/1315522428 >> 172.21.15.119:0/1315522428 conn(0x7fb3700cc540 msgr2=0x7fb3700cc940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.411+0000 7fb374fff700 1 -- 172.21.15.119:0/1315522428 shutdown_connections 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.411+0000 7fb374fff700 1 -- 172.21.15.119:0/1315522428 wait complete. 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.412+0000 7fb374fff700 1 Processor -- start 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.412+0000 7fb374fff700 1 -- start start 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb374fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb370069fa0 0x7fb37016d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb374fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb370169800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb374fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb370156ee0 0x7fb37016c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb374fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700c2050 con 0x7fb3700c85c0 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb374fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700af700 con 0x7fb370156ee0 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb374fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3700b1540 con 0x7fb370069fa0 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb367fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb370169800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb367fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb370169800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57644/0 (socket says 172.21.15.119:57644) 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.413+0000 7fb367fff700 1 -- 172.21.15.119:0/1186531445 learned_addr learned my addr 172.21.15.119:0/1186531445 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb36e59c700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb370069fa0 0x7fb37016d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb36ed9d700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb370156ee0 0x7fb37016c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb367fff700 1 -- 172.21.15.119:0/1186531445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb370069fa0 msgr2=0x7fb37016d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb367fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb370069fa0 0x7fb37016d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb367fff700 1 -- 172.21.15.119:0/1186531445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb370156ee0 msgr2=0x7fb37016c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb367fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb370156ee0 0x7fb37016c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb367fff700 1 -- 172.21.15.119:0/1186531445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36000b790 con 0x7fb3700c85c0 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb367fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb370169800 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fb35800c5b0 tx=0x7fb358001040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb3667fc700 1 -- 172.21.15.119:0/1186531445 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb358005a70 con 0x7fb3700c85c0 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb3667fc700 1 -- 172.21.15.119:0/1186531445 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb358005bd0 con 0x7fb3700c85c0 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37016dca0 con 0x7fb3700c85c0 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb3667fc700 1 -- 172.21.15.119:0/1186531445 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3580070d0 con 0x7fb3700c85c0 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.414+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3700b1260 con 0x7fb3700c85c0 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.415+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3700718b0 con 0x7fb3700c85c0 2023-10-08T02:31:33.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.416+0000 7fb3667fc700 1 -- 172.21.15.119:0/1186531445 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb358007440 con 0x7fb3700c85c0 2023-10-08T02:31:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.417+0000 7fb3667fc700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.417+0000 7fb3667fc700 1 -- 172.21.15.119:0/1186531445 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7fb358083c40 con 0x7fb3700c85c0 2023-10-08T02:31:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.417+0000 7fb36e59c700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.418+0000 7fb36e59c700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb370069470 tx=0x7fb36001b000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:33.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.421+0000 7fb3667fc700 1 -- 172.21.15.119:0/1186531445 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb358050ca0 con 0x7fb3700c85c0 2023-10-08T02:31:33.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.687+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb37006e060 con 0x7fb3700c85c0 2023-10-08T02:31:33.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.691+0000 7fb3667fc700 1 -- 172.21.15.119:0/1186531445 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v53) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb358017070 con 0x7fb3700c85c0 2023-10-08T02:31:33.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 msgr2=0x7fb3500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb370069470 tx=0x7fb36001b000 comp rx=0 tx=0).stop 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 msgr2=0x7fb370169800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb370169800 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7fb35800c5b0 tx=0x7fb358001040 comp rx=0 tx=0).stop 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 shutdown_connections 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb370069fa0 0x7fb37016d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb370156ee0 0x7fb37016c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 --2- 172.21.15.119:0/1186531445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3700c85c0 0x7fb370169800 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 >> 172.21.15.119:0/1186531445 conn(0x7fb3700cc540 msgr2=0x7fb370156340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:33.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 shutdown_connections 2023-10-08T02:31:33.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.694+0000 7fb374fff700 1 -- 172.21.15.119:0/1186531445 wait complete. 2023-10-08T02:31:33.695 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-10-08T02:31:33.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:33.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.893+0000 7f5b493ff700 1 Processor -- start 2023-10-08T02:31:33.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.893+0000 7f5b493ff700 1 -- start start 2023-10-08T02:31:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.894+0000 7f5b493ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b44155c10 0x7f5b44155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.894+0000 7f5b493ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5b440ac6a0 0x7f5b440b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.894+0000 7f5b493ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b440acac0 con 0x7f5b440ac230 2023-10-08T02:31:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.894+0000 7f5b493ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b440ad930 con 0x7f5b440ac6a0 2023-10-08T02:31:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.894+0000 7f5b493ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b440acd20 con 0x7f5b44155c10 2023-10-08T02:31:33.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b42ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b44155c10 0x7f5b44155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b42ffd700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b44155c10 0x7f5b44155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:50598/0 (socket says 172.21.15.119:50598) 2023-10-08T02:31:33.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b42ffd700 1 -- 172.21.15.119:0/2629583690 learned_addr learned my addr 172.21.15.119:0/2629583690 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:33.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b437fe700 1 --2- 172.21.15.119:0/2629583690 >> v2:172.21.15.184:3300/0 conn(0x7f5b440ac6a0 0x7f5b440b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b427fc700 1 --1- 172.21.15.119:0/2629583690 >> v1:172.21.15.184:6789/0 conn(0x7f5b440ac230 0x7f5b441568a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:35786/0 (socket says 172.21.15.119:35786) 2023-10-08T02:31:33.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b42ffd700 1 -- 172.21.15.119:0/2629583690 >> v2:172.21.15.184:3300/0 conn(0x7f5b440ac6a0 msgr2=0x7f5b440b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b42ffd700 1 --2- 172.21.15.119:0/2629583690 >> v2:172.21.15.184:3300/0 conn(0x7f5b440ac6a0 0x7f5b440b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b42ffd700 1 -- 172.21.15.119:0/2629583690 >> v1:172.21.15.184:6789/0 conn(0x7f5b440ac230 legacy=0x7f5b441568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.895+0000 7f5b42ffd700 1 -- 172.21.15.119:0/2629583690 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b440ad000 con 0x7f5b44155c10 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.896+0000 7f5b42ffd700 1 --2- 172.21.15.119:0/2629583690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b44155c10 0x7f5b44155fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400bc40 tx=0x7f5b3400b4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c192b2200593c4d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.896+0000 7f5b41ffb700 1 -- 172.21.15.119:0/2629583690 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400da90 con 0x7f5b44155c10 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.896+0000 7f5b41ffb700 1 -- 172.21.15.119:0/2629583690 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3400dbf0 con 0x7f5b44155c10 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.896+0000 7f5b41ffb700 1 -- 172.21.15.119:0/2629583690 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34017410 con 0x7f5b44155c10 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.896+0000 7f5b493ff700 1 -- 172.21.15.119:0/2629583690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b44155c10 msgr2=0x7f5b44155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.896+0000 7f5b493ff700 1 --2- 172.21.15.119:0/2629583690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b44155c10 0x7f5b44155fe0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400bc40 tx=0x7f5b3400b4e0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.897+0000 7f5b493ff700 1 -- 172.21.15.119:0/2629583690 shutdown_connections 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.897+0000 7f5b493ff700 1 --2- 172.21.15.119:0/2629583690 >> v2:172.21.15.184:3300/0 conn(0x7f5b440ac6a0 0x7f5b440b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.897+0000 7f5b493ff700 1 --2- 172.21.15.119:0/2629583690 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b44155c10 0x7f5b44155fe0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.897+0000 7f5b493ff700 1 -- 172.21.15.119:0/2629583690 >> 172.21.15.119:0/2629583690 conn(0x7f5b4406a320 msgr2=0x7f5b440cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:33.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.897+0000 7f5b493ff700 1 -- 172.21.15.119:0/2629583690 shutdown_connections 2023-10-08T02:31:33.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.897+0000 7f5b493ff700 1 -- 172.21.15.119:0/2629583690 wait complete. 2023-10-08T02:31:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.898+0000 7f5b493ff700 1 Processor -- start 2023-10-08T02:31:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.898+0000 7f5b493ff700 1 -- start start 2023-10-08T02:31:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b493ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b440ac230 0x7f5b44153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b493ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b440ac6a0 0x7f5b4415d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b493ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b44155c10 0x7f5b4415d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b493ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44153c60 con 0x7f5b44155c10 2023-10-08T02:31:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b493ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b44153290 con 0x7f5b440ac6a0 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b493ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b441424a0 con 0x7f5b440ac230 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b437fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b44155c10 0x7f5b4415d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b437fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b44155c10 0x7f5b4415d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57656/0 (socket says 172.21.15.119:57656) 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b437fe700 1 -- 172.21.15.119:0/497291118 learned_addr learned my addr 172.21.15.119:0/497291118 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b42ffd700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b440ac230 0x7f5b44153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.899+0000 7f5b427fc700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b440ac6a0 0x7f5b4415d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b437fe700 1 -- 172.21.15.119:0/497291118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b440ac230 msgr2=0x7f5b44153720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b437fe700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b440ac230 0x7f5b44153720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b437fe700 1 -- 172.21.15.119:0/497291118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b440ac6a0 msgr2=0x7f5b4415d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b437fe700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b440ac6a0 0x7f5b4415d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b437fe700 1 -- 172.21.15.119:0/497291118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3400b780 con 0x7f5b44155c10 2023-10-08T02:31:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b437fe700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b44155c10 0x7f5b4415d640 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009e90 tx=0x7f5b3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b2affd700 1 -- 172.21.15.119:0/497291118 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c00cc20 con 0x7f5b44155c10 2023-10-08T02:31:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b2affd700 1 -- 172.21.15.119:0/497291118 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3c00f040 con 0x7f5b44155c10 2023-10-08T02:31:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b44156950 con 0x7f5b44155c10 2023-10-08T02:31:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b2affd700 1 -- 172.21.15.119:0/497291118 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c014480 con 0x7f5b44155c10 2023-10-08T02:31:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.900+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b44156f10 con 0x7f5b44155c10 2023-10-08T02:31:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.901+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b44153440 con 0x7f5b44155c10 2023-10-08T02:31:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.902+0000 7f5b2affd700 1 -- 172.21.15.119:0/497291118 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5b3c0147f0 con 0x7f5b44155c10 2023-10-08T02:31:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.903+0000 7f5b2affd700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5b30066f30 0x7f5b300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.903+0000 7f5b2affd700 1 -- 172.21.15.119:0/497291118 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f5b3c085450 con 0x7f5b44155c10 2023-10-08T02:31:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.903+0000 7f5b42ffd700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5b30066f30 0x7f5b300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.904+0000 7f5b42ffd700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5b30066f30 0x7f5b300693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400b520 tx=0x7f5b34019000 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:33.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:33.907+0000 7f5b2affd700 1 -- 172.21.15.119:0/497291118 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5b44153440 con 0x7f5b44155c10 2023-10-08T02:31:34.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.140+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b44153400 con 0x7f5b44155c10 2023-10-08T02:31:34.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.140+0000 7f5b2affd700 1 -- 172.21.15.119:0/497291118 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f5b3c055f40 con 0x7f5b44155c10 2023-10-08T02:31:34.140 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:34.140 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1044912493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1044912493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1044912493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10197,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1564057059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1564057059},{"type":"v1","addr":"172.21.15.119:6833","nonce":1564057059}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2545901849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2545901849},{"type":"v1","addr":"172.21.15.184:6834","nonce":2545901849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:31:27.855883+0000","modified":"2023-10-08T02:31:31.420955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10215":{"gid":10215,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1741909431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1741909431},{"type":"v1","addr":"172.21.15.119:6835","nonce":1741909431}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T02:31:34.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5b30066f30 msgr2=0x7f5b300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5b30066f30 0x7f5b300693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400b520 tx=0x7f5b34019000 comp rx=0 tx=0).stop 2023-10-08T02:31:34.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b44155c10 msgr2=0x7f5b4415d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b44155c10 0x7f5b4415d640 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c009e90 tx=0x7f5b3c00b040 comp rx=0 tx=0).stop 2023-10-08T02:31:34.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 shutdown_connections 2023-10-08T02:31:34.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5b30066f30 0x7f5b300693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5b440ac230 0x7f5b44153720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5b44155c10 0x7f5b4415d640 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 --2- 172.21.15.119:0/497291118 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5b440ac6a0 0x7f5b4415d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 >> 172.21.15.119:0/497291118 conn(0x7f5b4406a320 msgr2=0x7f5b4415c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 shutdown_connections 2023-10-08T02:31:34.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.143+0000 7f5b493ff700 1 -- 172.21.15.119:0/497291118 wait complete. 2023-10-08T02:31:34.146 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-10-08T02:31:34.562 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 95, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-08T02:31:27.855883+0000', 'modified': '2023-10-08T02:31:31.420955+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10215}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10215': {'gid': 10215, 'name': 'a', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6835/1741909431', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6834', 'nonce': 1741909431}, {'type': 'v1', 'addr': '172.21.15.119:6835', 'nonce': 1741909431}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-08T02:31:34.563 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-08T02:31:34.563 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-08T02:31:34.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.747+0000 7f5fad79d700 1 Processor -- start 2023-10-08T02:31:34.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.748+0000 7f5fad79d700 1 -- start start 2023-10-08T02:31:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.749+0000 7f5fad79d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f5fa80b02a0 0x7f5fa815bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.749+0000 7f5fad79d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa806ae70 con 0x7f5fa806a0a0 2023-10-08T02:31:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.749+0000 7f5fad79d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fa80c8a60 con 0x7f5fa806d950 2023-10-08T02:31:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.749+0000 7f5fad79d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fa806a540 con 0x7f5fa80b02a0 2023-10-08T02:31:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.749+0000 7f5fa77fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5fa806a0a0 0x7f5fa815b410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:47092/0 (socket says 172.21.15.119:47092) 2023-10-08T02:31:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.749+0000 7f5fa77fe700 1 -- 172.21.15.119:0/1811876659 learned_addr learned my addr 172.21.15.119:0/1811876659 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5facf9c700 1 --2- 172.21.15.119:0/1811876659 >> v2:172.21.15.184:3300/0 conn(0x7f5fa80b02a0 0x7f5fa815bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041605764 0 0) 0x7f5fa806ae70 con 0x7f5fa806a0a0 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f90003570 con 0x7f5fa806a0a0 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172841450 0 0) 0x7f5fa80c8a60 con 0x7f5fa806d950 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fa806ae70 con 0x7f5fa806d950 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3718638055 0 0) 0x7f5f90003570 con 0x7f5fa806a0a0 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 >> v2:172.21.15.184:3300/0 conn(0x7f5fa80b02a0 msgr2=0x7f5fa815bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 --2- 172.21.15.119:0/1811876659 >> v2:172.21.15.184:3300/0 conn(0x7f5fa80b02a0 0x7f5fa815bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 >> v1:172.21.15.184:6789/0 conn(0x7f5fa806d950 legacy=0x7f5fa80c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa806a820 con 0x7f5fa806a0a0 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f5f9c001470 con 0x7f5fa806a0a0 2023-10-08T02:31:34.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 >> v1:172.21.15.119:6789/0 conn(0x7f5fa806a0a0 legacy=0x7f5fa815b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f90003bf0 0x7f5f90003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90004810 0x7f5f90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f900051c0 0x7f5f9000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa806a540 con 0x7f5f90004810 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9000c0b0 con 0x7f5f900051c0 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.750+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9000c270 con 0x7f5f90003bf0 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90004810 0x7f5f90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f90003bf0 0x7f5f90003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5facf9c700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f900051c0 0x7f5f9000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa77fe700 1 -- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f90003bf0 msgr2=0x7f5f90003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5f90003bf0 0x7f5f90003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa77fe700 1 -- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f900051c0 msgr2=0x7f5f9000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f900051c0 0x7f5f9000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa77fe700 1 -- 172.21.15.119:0/1811876659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f90003570 con 0x7f5f90004810 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5facf9c700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f900051c0 0x7f5f9000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5facf9c700 1 -- 172.21.15.119:0/1811876659 reap_dead start 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90004810 0x7f5f90004c80 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c2d0 tx=0x7f5f9c001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dde637f2f33c41e server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c00cb60 con 0x7f5f90004810 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c00cce0 con 0x7f5f90004810 2023-10-08T02:31:34.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.751+0000 7f5fa6ffd700 1 -- 172.21.15.119:0/1811876659 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c005b70 con 0x7f5f90004810 2023-10-08T02:31:34.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.752+0000 7f5fad79d700 1 -- 172.21.15.119:0/1811876659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90004810 msgr2=0x7f5f90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.752+0000 7f5fad79d700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90004810 0x7f5f90004c80 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c2d0 tx=0x7f5f9c001e10 comp rx=0 tx=0).stop 2023-10-08T02:31:34.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.752+0000 7f5fad79d700 1 -- 172.21.15.119:0/1811876659 shutdown_connections 2023-10-08T02:31:34.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.752+0000 7f5fad79d700 1 --2- 172.21.15.119:0/1811876659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f90004810 0x7f5f90004c80 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.752+0000 7f5fad79d700 1 -- 172.21.15.119:0/1811876659 >> 172.21.15.119:0/1811876659 conn(0x7f5fa80cc1f0 msgr2=0x7f5fa80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:34.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.752+0000 7f5fad79d700 1 -- 172.21.15.119:0/1811876659 shutdown_connections 2023-10-08T02:31:34.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.752+0000 7f5fad79d700 1 -- 172.21.15.119:0/1811876659 wait complete. 2023-10-08T02:31:34.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.753+0000 7f5fad79d700 1 Processor -- start 2023-10-08T02:31:34.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.754+0000 7f5fad79d700 1 -- start start 2023-10-08T02:31:34.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.754+0000 7f5fad79d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90004810 0x7f5fa8159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.754+0000 7f5fad79d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81563f0 0x7f5fa8156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.754+0000 7f5fad79d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa815a4f0 0x7f5fa815a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.754+0000 7f5fad79d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa815b100 con 0x7f5fa81563f0 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.754+0000 7f5fad79d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81529e0 con 0x7f5f90004810 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.754+0000 7f5fad79d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8156da0 con 0x7f5fa815a4f0 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81563f0 0x7f5fa8156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81563f0 0x7f5fa8156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57672/0 (socket says 172.21.15.119:57672) 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 -- 172.21.15.119:0/263884607 learned_addr learned my addr 172.21.15.119:0/263884607 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90004810 0x7f5fa8159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5facf9c700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa815a4f0 0x7f5fa815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 -- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa815a4f0 msgr2=0x7f5fa815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa815a4f0 0x7f5fa815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 -- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90004810 msgr2=0x7f5fa8159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90004810 0x7f5fa8159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 -- 172.21.15.119:0/263884607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa81598e0 con 0x7f5fa81563f0 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90004810 0x7f5fa8159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5facf9c700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa815a4f0 0x7f5fa815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.755+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81563f0 0x7f5fa8156860 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c8c0 tx=0x7f5f9c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.756+0000 7f5f8ffff700 1 -- 172.21.15.119:0/263884607 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c000c30 con 0x7f5fa81563f0 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.756+0000 7f5f8ffff700 1 -- 172.21.15.119:0/263884607 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c001650 con 0x7f5fa81563f0 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.756+0000 7f5fad79d700 1 -- 172.21.15.119:0/263884607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa815aea0 con 0x7f5fa81563f0 2023-10-08T02:31:34.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.756+0000 7f5f8ffff700 1 -- 172.21.15.119:0/263884607 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c007320 con 0x7f5fa81563f0 2023-10-08T02:31:34.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.756+0000 7f5fad79d700 1 -- 172.21.15.119:0/263884607 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa8158d10 con 0x7f5fa81563f0 2023-10-08T02:31:34.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.758+0000 7f5f8ffff700 1 -- 172.21.15.119:0/263884607 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5f9c028460 con 0x7f5fa81563f0 2023-10-08T02:31:34.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.759+0000 7f5f8ffff700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f94066f30 0x7f5f940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.759+0000 7f5f8ffff700 1 -- 172.21.15.119:0/263884607 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f5f9c085cf0 con 0x7f5fa81563f0 2023-10-08T02:31:34.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.759+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f94066f30 0x7f5f940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.759+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f94066f30 0x7f5f940693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f98003d10 tx=0x7f5f98009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:34.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.760+0000 7f5fbd1fcb80 1 Processor -- start 2023-10-08T02:31:34.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.760+0000 7f5fbd1fcb80 1 -- start start 2023-10-08T02:31:34.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa831a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa833b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa8247a70 con 0x556aa82f0b80 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa8245270 con 0x556aa833b480 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aa8241060 con 0x556aa82ee5f0 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5facf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5facf9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57680/0 (socket says 172.21.15.119:57680) 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.761+0000 7f5facf9c700 1 -- 172.21.15.119:0/1148031916 learned_addr learned my addr 172.21.15.119:0/1148031916 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5facf9c700 1 -- 172.21.15.119:0/1148031916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 msgr2=0x556aa833b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5facf9c700 1 --2- 172.21.15.119:0/1148031916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa833b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5facf9c700 1 -- 172.21.15.119:0/1148031916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 msgr2=0x556aa831a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5facf9c700 1 --2- 172.21.15.119:0/1148031916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa831a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5facf9c700 1 -- 172.21.15.119:0/1148031916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa80db270 con 0x556aa82f0b80 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5facf9c700 1 --2- 172.21.15.119:0/1148031916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0ff0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f5fa815b290 tx=0x7f5fa000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4060c9f8a355d3f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5f8d7fa700 1 -- 172.21.15.119:0/1148031916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0009c70 con 0x556aa82f0b80 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5f8d7fa700 1 -- 172.21.15.119:0/1148031916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa000f070 con 0x556aa82f0b80 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.762+0000 7f5f8d7fa700 1 -- 172.21.15.119:0/1148031916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fa0015620 con 0x556aa82f0b80 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/1148031916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 msgr2=0x556aa82f0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/1148031916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0ff0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f5fa815b290 tx=0x7f5fa000b040 comp rx=0 tx=0).stop 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/1148031916 shutdown_connections 2023-10-08T02:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/1148031916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0ff0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/1148031916 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa831a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/1148031916 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa833b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/1148031916 >> 172.21.15.119:0/1148031916 conn(0x556aa82f20c0 msgr2=0x556aa82f28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/1148031916 shutdown_connections 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.763+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/1148031916 wait complete. 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.764+0000 7f5fbd1fcb80 1 -- start start 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.764+0000 7f5fbd1fcb80 1 Processor -- start 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa83382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa831a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa831acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa81ec890 con 0x556aa82f0b80 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa8207ea0 con 0x556aa833b480 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aa80c3e30 con 0x556aa82ee5f0 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fbd1fcb80 10 client.0 _ll_register_callbacks cb 0x556aa82b42c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa831a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa831a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57692/0 (socket says 172.21.15.119:57692) 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 -- 172.21.15.119:0/431401142 learned_addr learned my addr 172.21.15.119:0/431401142 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa83382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5facf9c700 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa831acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 -- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 msgr2=0x556aa83382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa83382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 -- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 msgr2=0x556aa831acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa831acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 -- 172.21.15.119:0/431401142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa0009dd0 con 0x556aa82f0b80 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5facf9c700 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa831acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa831a7b0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c056900 tx=0x7f5f9c083ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7918045c4b8e46 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.765+0000 7f5f867fc700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5f867fc700 1 -- 172.21.15.119:0/431401142 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0263c0 con 0x556aa82f0b80 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5f867fc700 1 -- 172.21.15.119:0/431401142 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f5f9c057070 con 0x556aa82f0b80 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/431401142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556aa82e5cd0 con 0x556aa82f0b80 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5f867fc700 1 -- 172.21.15.119:0/431401142 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f9c056970 con 0x556aa82f0b80 2023-10-08T02:31:34.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 10 client.10350 fetch_fsmap learned FSMap version 95 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/431401142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x556aa807dfe0 con 0x556aa82f0b80 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5f867fc700 1 -- 172.21.15.119:0/431401142 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c082020 con 0x556aa82f0b80 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5f867fc700 1 -- 172.21.15.119:0/431401142 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 95) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5f9c055e60 con 0x556aa82f0b80 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 10 client.10350 fetch_fsmap finished waiting for FSMap version 95 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 10 client.10350 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 20 client.10350 populate_metadata read hostname 'smithi119' 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] conn(0x556aa8315150 0x556aa8320490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 4 client.10350 mds_command: new command op to 10215 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.766+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/431401142 --> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556aa82c2aa0 con 0x556aa8315150 2023-10-08T02:31:34.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.767+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] conn(0x556aa8315150 0x556aa8320490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.767+0000 7f5f867fc700 10 client.10350 ms_handle_connect on v2:172.21.15.119:6834/1741909431 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.767+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] conn(0x556aa8315150 0x556aa8320490 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac23bc960bd8031f server_cookie=ff50812fff66083 in_seq=0 out_seq=0 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5f867fc700 1 -- 172.21.15.119:0/431401142 <== mds.0 v2:172.21.15.119:6834/1741909431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x556aa82c2aa0 con 0x556aa8315150 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5f867fc700 10 client.10350 handle_command_reply: tid=0 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 client.10350 shutdown 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/431401142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 msgr2=0x556aa831a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa831a7b0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c056900 tx=0x7f5f9c083ec0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/431401142 shutdown_connections 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] conn(0x556aa8315150 0x556aa8320490 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa83382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa831acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/431401142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa831a7b0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5f867fc700 0 client.10350 ms_handle_reset on v2:172.21.15.119:6834/1741909431 2023-10-08T02:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/431401142 >> 172.21.15.119:0/431401142 conn(0x556aa82f20c0 msgr2=0x556aa82f28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/431401142 shutdown_connections 2023-10-08T02:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.768+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/431401142 wait complete. 2023-10-08T02:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.769+0000 7f5fbd1fcb80 20 client.10350 trim_cache size 0 max 16384 2023-10-08T02:31:34.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.785+0000 7f5fbd1fcb80 1 Processor -- start 2023-10-08T02:31:34.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.786+0000 7f5fbd1fcb80 1 -- start start 2023-10-08T02:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.786+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa83970e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.786+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.786+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa82f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.786+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa8316ba0 con 0x556aa82f0b80 2023-10-08T02:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.786+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa83211c0 con 0x556aa833b480 2023-10-08T02:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.786+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aa8338ed0 con 0x556aa82ee5f0 2023-10-08T02:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57698/0 (socket says 172.21.15.119:57698) 2023-10-08T02:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 -- 172.21.15.119:0/2449441241 learned_addr learned my addr 172.21.15.119:0/2449441241 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa83970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5facf9c700 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa82f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 -- 172.21.15.119:0/2449441241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 msgr2=0x556aa83970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa83970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 -- 172.21.15.119:0/2449441241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 msgr2=0x556aa82f14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa82f14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 -- 172.21.15.119:0/2449441241 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556aa820f780 con 0x556aa82f0b80 2023-10-08T02:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0f70 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c083050 tx=0x7f5f9c06cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84203bd92ff3aacc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5f8dffb700 1 -- 172.21.15.119:0/2449441241 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c0263c0 con 0x556aa82f0b80 2023-10-08T02:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5f8dffb700 1 -- 172.21.15.119:0/2449441241 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f9c082070 con 0x556aa82f0b80 2023-10-08T02:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.787+0000 7f5f8dffb700 1 -- 172.21.15.119:0/2449441241 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c059c10 con 0x556aa82f0b80 2023-10-08T02:31:34.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/2449441241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 msgr2=0x556aa82f0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0f70 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c083050 tx=0x7f5f9c06cf90 comp rx=0 tx=0).stop 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/2449441241 shutdown_connections 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa82ee5f0 0x556aa83970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa833b480 0x556aa82f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/2449441241 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82f0b80 0x556aa82f0f70 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/2449441241 >> 172.21.15.119:0/2449441241 conn(0x556aa82f20c0 msgr2=0x556aa831a790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/2449441241 shutdown_connections 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.788+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/2449441241 wait complete. 2023-10-08T02:31:34.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.789+0000 7f5fbd1fcb80 1 -- start start 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.789+0000 7f5fbd1fcb80 1 Processor -- start 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82ee5f0 0x556aa8393e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa82f0b80 0x556aa839d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fbd1fcb80 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa833b480 0x556aa839d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa839e1d0 con 0x556aa82ee5f0 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x556aa82f17f0 con 0x556aa82f0b80 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fbd1fcb80 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x556aa8313f50 con 0x556aa833b480 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fbd1fcb80 10 client.0 _ll_register_callbacks cb 0x556aa82b42c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82ee5f0 0x556aa8393e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82ee5f0 0x556aa8393e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57706/0 (socket says 172.21.15.119:57706) 2023-10-08T02:31:34.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 -- 172.21.15.119:0/3896329437 learned_addr learned my addr 172.21.15.119:0/3896329437 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 -- 172.21.15.119:0/3896329437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa833b480 msgr2=0x556aa839d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa833b480 0x556aa839d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 -- 172.21.15.119:0/3896329437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa82f0b80 msgr2=0x556aa839d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa82f0b80 0x556aa839d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 -- 172.21.15.119:0/3896329437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa0009dd0 con 0x556aa82ee5f0 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5fa7fff700 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82ee5f0 0x556aa8393e90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x556aa8207f00 tx=0x7f5f98029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d38575f87152b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.790+0000 7f5f867fc700 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5f867fc700 1 -- 172.21.15.119:0/3896329437 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f98015aa0 con 0x556aa82ee5f0 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5f867fc700 1 -- 172.21.15.119:0/3896329437 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f5f9801e040 con 0x556aa82ee5f0 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/3896329437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fa81510d0 con 0x556aa82ee5f0 2023-10-08T02:31:34.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5f867fc700 1 -- 172.21.15.119:0/3896329437 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=95) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f98015c00 con 0x556aa82ee5f0 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 10 client.10356 fetch_fsmap learned FSMap version 95 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/3896329437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=95,monmap=2+}) v3 -- 0x556aa81f6470 con 0x556aa82ee5f0 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5f867fc700 1 -- 172.21.15.119:0/3896329437 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fa81510d0 con 0x556aa82ee5f0 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5f867fc700 1 -- 172.21.15.119:0/3896329437 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 95) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f5f98025440 con 0x556aa82ee5f0 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 10 client.10356 fetch_fsmap finished waiting for FSMap version 95 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 10 client.10356 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 20 client.10356 populate_metadata read hostname 'smithi119' 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] conn(0x556aa8316cf0 0x556aa83170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 4 client.10356 mds_command: new command op to 10215 tid=0[{"prefix": "status"}] 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.791+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/3896329437 --> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f5fa000f070 con 0x556aa8316cf0 2023-10-08T02:31:34.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] conn(0x556aa8316cf0 0x556aa83170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fa77fe700 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] conn(0x556aa8316cf0 0x556aa83170e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4600294d512fa5a8 server_cookie=a201e735cde33ed in_seq=0 out_seq=0 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5f867fc700 10 client.10356 ms_handle_connect on v2:172.21.15.119:6834/1741909431 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5f867fc700 1 -- 172.21.15.119:0/3896329437 <== mds.0 v2:172.21.15.119:6834/1741909431 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x7f5fa000f070 con 0x556aa8316cf0 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5f867fc700 10 client.10356 handle_command_reply: tid=0 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 client.10356 shutdown 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/3896329437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82ee5f0 msgr2=0x556aa8393e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82ee5f0 0x556aa8393e90 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x556aa8207f00 tx=0x7f5f98029040 comp rx=0 tx=0).stop 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/3896329437 shutdown_connections 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.119:6834/1741909431,v1:172.21.15.119:6835/1741909431] conn(0x556aa8316cf0 0x556aa83170e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x556aa833b480 0x556aa839d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556aa82ee5f0 0x556aa8393e90 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5f867fc700 0 client.10356 ms_handle_reset on v2:172.21.15.119:6834/1741909431 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 --2- 172.21.15.119:0/3896329437 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x556aa82f0b80 0x556aa839d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.792+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/3896329437 >> 172.21.15.119:0/3896329437 conn(0x556aa82f20c0 msgr2=0x556aa82f0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/3896329437 shutdown_connections 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fbd1fcb80 1 -- 172.21.15.119:0/3896329437 wait complete. 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fbd1fcb80 20 client.10356 trim_cache size 0 max 16384 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "260cffbb-ac40-4cab-945b-d7e76ccdf950", 2023-10-08T02:31:34.799 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 10215, 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.378416562, 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 95, 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 206, 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 206, 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 12.669103185999999 2023-10-08T02:31:34.800 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:31:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 -- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f94066f30 msgr2=0x7f5f940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f94066f30 0x7f5f940693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5f98003d10 tx=0x7f5f98009040 comp rx=0 tx=0).stop 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 -- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81563f0 msgr2=0x7f5fa8156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81563f0 0x7f5fa8156860 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c00c8c0 tx=0x7f5f9c015000 comp rx=0 tx=0).stop 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 -- 172.21.15.119:0/263884607 shutdown_connections 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5f94066f30 0x7f5f940693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5fa815a4f0 0x7f5fa815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5f90004810 0x7f5fa8159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 --2- 172.21.15.119:0/263884607 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fa81563f0 0x7f5fa8156860 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.793+0000 7f5fad79d700 1 -- 172.21.15.119:0/263884607 >> 172.21.15.119:0/263884607 conn(0x7f5fa80cc1f0 msgr2=0x7f5fa814ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.794+0000 7f5fad79d700 1 -- 172.21.15.119:0/263884607 shutdown_connections 2023-10-08T02:31:34.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.794+0000 7f5fad79d700 1 -- 172.21.15.119:0/263884607 wait complete. 2023-10-08T02:31:34.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-08T02:31:34.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.991+0000 7f556d5ff700 1 Processor -- start 2023-10-08T02:31:34.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.991+0000 7f556d5ff700 1 -- start start 2023-10-08T02:31:34.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f556d5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f55681578b0 0x7f5568155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f556d5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55680af2f0 0x7f55680af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:34.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f556d5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55680ad930 con 0x7f5568155c10 2023-10-08T02:31:34.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f556d5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55680b0c80 con 0x7f55681578b0 2023-10-08T02:31:34.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f556d5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55680acd20 con 0x7f55680af2f0 2023-10-08T02:31:34.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f5566ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5568155c10 0x7f55680ac230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43226/0 (socket says 172.21.15.119:43226) 2023-10-08T02:31:34.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f5566ffd700 1 -- 172.21.15.119:0/426623756 learned_addr learned my addr 172.21.15.119:0/426623756 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:34.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f55667fc700 1 --2- 172.21.15.119:0/426623756 >> v2:172.21.15.119:3300/0 conn(0x7f55681578b0 0x7f5568155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f5565ffb700 1 -- 172.21.15.119:0/426623756 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081870900 0 0) 0x7f55680ad930 con 0x7f5568155c10 2023-10-08T02:31:34.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f5565ffb700 1 -- 172.21.15.119:0/426623756 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5550003570 con 0x7f5568155c10 2023-10-08T02:31:34.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f55667fc700 1 -- 172.21.15.119:0/426623756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55680af2f0 msgr2=0x7f55680af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f55677fe700 1 --2- 172.21.15.119:0/426623756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55680af2f0 0x7f55680af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.993+0000 7f55667fc700 1 --2- 172.21.15.119:0/426623756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55680af2f0 0x7f55680af730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f55667fc700 1 -- 172.21.15.119:0/426623756 >> v1:172.21.15.119:6789/0 conn(0x7f5568155c10 legacy=0x7f55680ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f55667fc700 1 -- 172.21.15.119:0/426623756 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680ad000 con 0x7f55681578b0 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f55677fe700 1 --2- 172.21.15.119:0/426623756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55680af2f0 0x7f55680af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f55667fc700 1 --2- 172.21.15.119:0/426623756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55681578b0 0x7f5568155fe0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f555c0098e0 tx=0x7f555c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95430029ee75eeca server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f5565ffb700 1 -- 172.21.15.119:0/426623756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c00b640 con 0x7f55681578b0 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f5565ffb700 1 -- 172.21.15.119:0/426623756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c00b7a0 con 0x7f55681578b0 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f5565ffb700 1 -- 172.21.15.119:0/426623756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c00bb20 con 0x7f55681578b0 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f556d5ff700 1 -- 172.21.15.119:0/426623756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55681578b0 msgr2=0x7f5568155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.994+0000 7f556d5ff700 1 --2- 172.21.15.119:0/426623756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55681578b0 0x7f5568155fe0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f555c0098e0 tx=0x7f555c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.995+0000 7f556d5ff700 1 -- 172.21.15.119:0/426623756 shutdown_connections 2023-10-08T02:31:34.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.995+0000 7f556d5ff700 1 --2- 172.21.15.119:0/426623756 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55680af2f0 0x7f55680af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.995+0000 7f556d5ff700 1 --2- 172.21.15.119:0/426623756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55681578b0 0x7f5568155fe0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:34.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.995+0000 7f556d5ff700 1 -- 172.21.15.119:0/426623756 >> 172.21.15.119:0/426623756 conn(0x7f556806a320 msgr2=0x7f55680cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:34.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.995+0000 7f556d5ff700 1 -- 172.21.15.119:0/426623756 shutdown_connections 2023-10-08T02:31:34.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.995+0000 7f556d5ff700 1 -- 172.21.15.119:0/426623756 wait complete. 2023-10-08T02:31:35.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.996+0000 7f556d5ff700 1 Processor -- start 2023-10-08T02:31:35.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.996+0000 7f556d5ff700 1 -- start start 2023-10-08T02:31:35.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f556d5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55680af2f0 0x7f556814ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:35.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f556d5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5568155c10 0x7f556814c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:35.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f556d5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55681578b0 0x7f5568168d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:35.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f556d5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556814dff0 con 0x7f5568155c10 2023-10-08T02:31:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f556d5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556814d3e0 con 0x7f55680af2f0 2023-10-08T02:31:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f556d5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556814d6c0 con 0x7f55681578b0 2023-10-08T02:31:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f55667fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5568155c10 0x7f556814c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f55667fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5568155c10 0x7f556814c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41488/0 (socket says 172.21.15.119:41488) 2023-10-08T02:31:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.997+0000 7f55667fc700 1 -- 172.21.15.119:0/112489537 learned_addr learned my addr 172.21.15.119:0/112489537 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f55677fe700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55681578b0 0x7f5568168d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f55667fc700 1 -- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55681578b0 msgr2=0x7f5568168d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:35.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f5566ffd700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55680af2f0 0x7f556814ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f55667fc700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55681578b0 0x7f5568168d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f55667fc700 1 -- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55680af2f0 msgr2=0x7f556814ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f55667fc700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55680af2f0 0x7f556814ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f55667fc700 1 -- 172.21.15.119:0/112489537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c009420 con 0x7f5568155c10 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f55677fe700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55681578b0 0x7f5568168d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f55667fc700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5568155c10 0x7f556814c9d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f55680ae0f0 tx=0x7f555c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f554effd700 1 -- 172.21.15.119:0/112489537 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c00b910 con 0x7f5568155c10 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f554effd700 1 -- 172.21.15.119:0/112489537 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c00bab0 con 0x7f5568155c10 2023-10-08T02:31:35.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556814d9a0 con 0x7f5568155c10 2023-10-08T02:31:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f554effd700 1 -- 172.21.15.119:0/112489537 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c025670 con 0x7f5568155c10 2023-10-08T02:31:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.998+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5568150500 con 0x7f5568155c10 2023-10-08T02:31:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:34.999+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556814e160 con 0x7f5568155c10 2023-10-08T02:31:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.000+0000 7f554effd700 1 -- 172.21.15.119:0/112489537 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f555c023070 con 0x7f5568155c10 2023-10-08T02:31:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.001+0000 7f554effd700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5554066f30 0x7f55540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.001+0000 7f554effd700 1 -- 172.21.15.119:0/112489537 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f555c085bf0 con 0x7f5568155c10 2023-10-08T02:31:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.001+0000 7f5566ffd700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5554066f30 0x7f55540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:35.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.002+0000 7f5566ffd700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5554066f30 0x7f55540693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f55580058d0 tx=0x7f5558009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:35.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.005+0000 7f554effd700 1 -- 172.21.15.119:0/112489537 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f555c052d00 con 0x7f5568155c10 2023-10-08T02:31:35.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.240+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f556804c3c0 con 0x7f5568155c10 2023-10-08T02:31:35.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.241+0000 7f554effd700 1 -- 172.21.15.119:0/112489537 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v95) v1 ==== 53+0+73 (secure 0 0 0) 0x7f555c015070 con 0x7f5568155c10 2023-10-08T02:31:35.241 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-08T02:31:35.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5554066f30 msgr2=0x7f55540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:35.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5554066f30 0x7f55540693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f55580058d0 tx=0x7f5558009040 comp rx=0 tx=0).stop 2023-10-08T02:31:35.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5568155c10 msgr2=0x7f556814c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:35.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5568155c10 0x7f556814c9d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f55680ae0f0 tx=0x7f555c017000 comp rx=0 tx=0).stop 2023-10-08T02:31:35.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 shutdown_connections 2023-10-08T02:31:35.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5554066f30 0x7f55540693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:35.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f55681578b0 0x7f5568168d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:35.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f55680af2f0 0x7f556814ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:35.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 --2- 172.21.15.119:0/112489537 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5568155c10 0x7f556814c9d0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:35.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 >> 172.21.15.119:0/112489537 conn(0x7f556806a320 msgr2=0x7f55680b0500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:35.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 shutdown_connections 2023-10-08T02:31:35.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:35.243+0000 7f556d5ff700 1 -- 172.21.15.119:0/112489537 wait complete. 2023-10-08T02:31:35.261 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-10-08T02:31:35.261 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-08T02:31:35.261 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-10-08T02:31:35.261 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-08T02:31:35.261 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-08T02:31:35.262 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-10-08T02:31:35.262 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-10-08T02:31:35.262 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-08T02:31:35.262 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:31:35.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:35.263 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-10-08T02:31:35.300 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-08T02:31:35.341 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-08T02:31:35.341 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-10-08T02:31:35.342 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 36556sec preferred_lft 36556sec 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::40f0:1dff:fe81:d330/64 scope link 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:31:35.343 INFO:teuthology.orchestra.run.smithi119.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T02:31:35.344 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether a6:24:a5:28:ee:b7 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T02:31:35.344 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::a424:a5ff:fe28:eeb7/64 scope link 2023-10-08T02:31:35.344 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:31:35.344 INFO:teuthology.orchestra.run.smithi119.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-08T02:31:35.344 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 3e:14:b9:75:1c:7a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T02:31:35.344 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::3c14:b9ff:fe75:1c7a/64 scope link 2023-10-08T02:31:35.344 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-10-08T02:31:35.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:35.344 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-08T02:31:35.405 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-10-08T02:31:35.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:35.408 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-10-08T02:31:35.447 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-10-08T02:31:35.447 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-08T02:31:35.447 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-08T02:31:35.447 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-08T02:31:35.447 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-08T02:31:35.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:35.448 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-10-08T02:31:35.449 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-10-08T02:31:35.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-08T02:31:35.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-08T02:31:35.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-08T02:31:35.449 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-10-08T02:31:35.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:35.643 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T02:31:35.679 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T02:31:35.680 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T02:31:35.680 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:31:35.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:35.681 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-10-08T02:31:35.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T02:31:35.781 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-08T02:31:35.817 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-08T02:31:35.817 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-08T02:31:35.817 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-08T02:31:35.893 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-10-08T02:31:35.894 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-08T02:31:35.894 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:31:36.023 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:31:36.023 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T02:31:36.051 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-10-08T02:31:36.051 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-10-08T02:31:36.051 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.184:3300,172.21.15.184:3301" 2023-10-08T02:31:36.051 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@260cffbb-ac40-4cab-945b-d7e76ccdf950.cephfs=/ 2023-10-08T02:31:36.052 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.184:3300/172.21.15.184:3301" will pass to kernel 2023-10-08T02:31:36.055 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-10-08T02:31:36.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:36.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-08T02:31:36.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-10-08T02:31:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.300+0000 7f840279d700 1 Processor -- start 2023-10-08T02:31:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.300+0000 7f840279d700 1 -- start start 2023-10-08T02:31:36.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.301+0000 7f840279d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f83fc1597f0 0x7f83fc0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.301+0000 7f840279d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83fc15d230 0x7f83fc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.301+0000 7f840279d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc15dbb0 con 0x7f83fc159420 2023-10-08T02:31:36.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.301+0000 7f840279d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83fc0b11d0 con 0x7f83fc1597f0 2023-10-08T02:31:36.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.301+0000 7f840279d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0b14b0 con 0x7f83fc15d230 2023-10-08T02:31:36.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.301+0000 7f840179b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f83fc159420 0x7f83fc0b5f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43230/0 (socket says 172.21.15.119:43230) 2023-10-08T02:31:36.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.301+0000 7f840179b700 1 -- 172.21.15.119:0/2969220476 learned_addr learned my addr 172.21.15.119:0/2969220476 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:36.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f8401f9c700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83fc15d230 0x7f83fc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f8400f9a700 1 --2- 172.21.15.119:0/2969220476 >> v2:172.21.15.184:3300/0 conn(0x7f83fc1597f0 0x7f83fc0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145856160 0 0) 0x7f83fc15dbb0 con 0x7f83fc159420 2023-10-08T02:31:36.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e8003570 con 0x7f83fc159420 2023-10-08T02:31:36.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3432921119 0 0) 0x7f83e8003570 con 0x7f83fc159420 2023-10-08T02:31:36.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83fc15d230 msgr2=0x7f83fc15d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:36.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83fc15d230 0x7f83fc15d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 >> v2:172.21.15.184:3300/0 conn(0x7f83fc1597f0 msgr2=0x7f83fc0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:36.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 --2- 172.21.15.119:0/2969220476 >> v2:172.21.15.184:3300/0 conn(0x7f83fc1597f0 0x7f83fc0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0b1790 con 0x7f83fc159420 2023-10-08T02:31:36.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f83f8001470 con 0x7f83fc159420 2023-10-08T02:31:36.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 >> v1:172.21.15.119:6789/0 conn(0x7f83fc159420 legacy=0x7f83fc0b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83fc15d230 0x7f83e8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e8003fe0 0x7f83e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e8004d50 0x7f83e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0b11d0 con 0x7f83e8003fe0 2023-10-08T02:31:36.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0b14b0 con 0x7f83e8004d50 2023-10-08T02:31:36.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.302+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e800ba90 con 0x7f83fc15d230 2023-10-08T02:31:36.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f8400f9a700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e8003fe0 0x7f83e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f840179b700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83fc15d230 0x7f83e8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f8401f9c700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e8004d50 0x7f83e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f8400f9a700 1 -- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83fc15d230 msgr2=0x7f83e8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:36.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f8400f9a700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83fc15d230 0x7f83e8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f8400f9a700 1 -- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e8004d50 msgr2=0x7f83e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:36.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f8400f9a700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83e8004d50 0x7f83e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f8400f9a700 1 -- 172.21.15.119:0/2969220476 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e8003570 con 0x7f83e8003fe0 2023-10-08T02:31:36.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f8400f9a700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e8003fe0 0x7f83e8004810 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f83fc159c70 tx=0x7f83ec0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ea318d771021bf server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:36.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00e6b0 con 0x7f83e8003fe0 2023-10-08T02:31:36.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00e810 con 0x7f83e8003fe0 2023-10-08T02:31:36.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.303+0000 7f83f3fff700 1 -- 172.21.15.119:0/2969220476 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00ebd0 con 0x7f83e8003fe0 2023-10-08T02:31:36.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.304+0000 7f840279d700 1 -- 172.21.15.119:0/2969220476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e8003fe0 msgr2=0x7f83e8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.304+0000 7f840279d700 1 --2- 172.21.15.119:0/2969220476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83e8003fe0 0x7f83e8004810 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f83fc159c70 tx=0x7f83ec0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.304+0000 7f8401f9c700 1 -- 172.21.15.119:0/2969220476 reap_dead start 2023-10-08T02:31:36.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.304+0000 7f840279d700 1 -- 172.21.15.119:0/2969220476 shutdown_connections 2023-10-08T02:31:36.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.304+0000 7f840279d700 1 -- 172.21.15.119:0/2969220476 >> 172.21.15.119:0/2969220476 conn(0x7f83fc0d0660 msgr2=0x7f83fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.304+0000 7f840279d700 1 -- 172.21.15.119:0/2969220476 shutdown_connections 2023-10-08T02:31:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.304+0000 7f840279d700 1 -- 172.21.15.119:0/2969220476 wait complete. 2023-10-08T02:31:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.305+0000 7f840279d700 1 Processor -- start 2023-10-08T02:31:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.306+0000 7f840279d700 1 -- start start 2023-10-08T02:31:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.306+0000 7f840279d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e8003fe0 0x7f83fc0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.306+0000 7f840279d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83fc15c340 0x7f83fc068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f840279d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83fc15c7b0 0x7f83fc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f840279d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0c7320 con 0x7f83fc15c340 2023-10-08T02:31:36.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f840279d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0c7cf0 con 0x7f83fc15c7b0 2023-10-08T02:31:36.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f840279d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0c6750 con 0x7f83e8003fe0 2023-10-08T02:31:36.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8400f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83fc15c340 0x7f83fc068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8400f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83fc15c340 0x7f83fc068dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41510/0 (socket says 172.21.15.119:41510) 2023-10-08T02:31:36.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8400f9a700 1 -- 172.21.15.119:0/2939488632 learned_addr learned my addr 172.21.15.119:0/2939488632 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:36.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f840179b700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e8003fe0 0x7f83fc0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8401f9c700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83fc15c7b0 0x7f83fc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8400f9a700 1 -- 172.21.15.119:0/2939488632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e8003fe0 msgr2=0x7f83fc0c77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8400f9a700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e8003fe0 0x7f83fc0c77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8400f9a700 1 -- 172.21.15.119:0/2939488632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83fc15c7b0 msgr2=0x7f83fc069300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8400f9a700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83fc15c7b0 0x7f83fc069300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.307+0000 7f8400f9a700 1 -- 172.21.15.119:0/2939488632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0c69f0 con 0x7f83fc15c340 2023-10-08T02:31:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.308+0000 7f8400f9a700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83fc15c340 0x7f83fc068dc0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f83e8004730 tx=0x7f83ec01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.308+0000 7f83f17fa700 1 -- 172.21.15.119:0/2939488632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec022070 con 0x7f83fc15c340 2023-10-08T02:31:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.308+0000 7f83f17fa700 1 -- 172.21.15.119:0/2939488632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec00e980 con 0x7f83fc15c340 2023-10-08T02:31:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.308+0000 7f83f17fa700 1 -- 172.21.15.119:0/2939488632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec00e6b0 con 0x7f83fc15c340 2023-10-08T02:31:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.308+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc0c6cd0 con 0x7f83fc15c340 2023-10-08T02:31:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.308+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc0b54a0 con 0x7f83fc15c340 2023-10-08T02:31:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.309+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc04c3c0 con 0x7f83fc15c340 2023-10-08T02:31:36.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.310+0000 7f83f17fa700 1 -- 172.21.15.119:0/2939488632 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f83ec027680 con 0x7f83fc15c340 2023-10-08T02:31:36.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.311+0000 7f83f17fa700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f83e0066f30 0x7f83e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.311+0000 7f83f17fa700 1 -- 172.21.15.119:0/2939488632 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f83ec086070 con 0x7f83fc15c340 2023-10-08T02:31:36.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.311+0000 7f840179b700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f83e0066f30 0x7f83e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.312+0000 7f840179b700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f83e0066f30 0x7f83e00693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f83f8001620 tx=0x7f83f8001e90 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.315+0000 7f83f17fa700 1 -- 172.21.15.119:0/2939488632 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f83ec0b75a0 con 0x7f83fc15c340 2023-10-08T02:31:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.541+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f83fc0b1a70 con 0x7f83e0066f30 2023-10-08T02:31:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.543+0000 7f83f17fa700 1 -- 172.21.15.119:0/2939488632 <== mgr.10020 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f83fc0b1a70 con 0x7f83e0066f30 2023-10-08T02:31:36.542 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-10-08T02:31:36.543 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-10-08T02:31:36.543 INFO:teuthology.orchestra.run.smithi119.stdout: "name": "cephfs" 2023-10-08T02:31:36.543 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-10-08T02:31:36.543 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-10-08T02:31:36.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f83e0066f30 msgr2=0x7f83e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f83e0066f30 0x7f83e00693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f83f8001620 tx=0x7f83f8001e90 comp rx=0 tx=0).stop 2023-10-08T02:31:36.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83fc15c340 msgr2=0x7f83fc068dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83fc15c340 0x7f83fc068dc0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f83e8004730 tx=0x7f83ec01a000 comp rx=0 tx=0).stop 2023-10-08T02:31:36.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 shutdown_connections 2023-10-08T02:31:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f83e0066f30 0x7f83e00693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f83e8003fe0 0x7f83fc0c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f83fc15c7b0 0x7f83fc069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 --2- 172.21.15.119:0/2939488632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f83fc15c340 0x7f83fc068dc0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 >> 172.21.15.119:0/2939488632 conn(0x7f83fc0d0660 msgr2=0x7f83fc069a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 shutdown_connections 2023-10-08T02:31:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.545+0000 7f840279d700 1 -- 172.21.15.119:0/2939488632 wait complete. 2023-10-08T02:31:36.562 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-10-08T02:31:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.744+0000 7f247f59d700 1 Processor -- start 2023-10-08T02:31:36.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.745+0000 7f247f59d700 1 -- start start 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247f59d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f247806f060 0x7f24780ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247f59d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f247806f4b0 0x7f24780ae610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247f59d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f247806ebd0 con 0x7f24780cb1a0 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247f59d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f247815b2e0 con 0x7f247806f060 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247f59d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f247815aa70 con 0x7f247806f4b0 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247dd9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f247806f060 0x7f24780ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247dd9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f247806f060 0x7f24780ae0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41526/0 (socket says 172.21.15.119:41526) 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247dd9a700 1 -- 172.21.15.119:0/270638028 learned_addr learned my addr 172.21.15.119:0/270638028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.746+0000 7f247ed9c700 1 --2- 172.21.15.119:0/270638028 >> v2:172.21.15.184:3300/0 conn(0x7f247806f4b0 0x7f24780ae610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.747+0000 7f247dd9a700 1 -- 172.21.15.119:0/270638028 >> v2:172.21.15.184:3300/0 conn(0x7f247806f4b0 msgr2=0x7f24780ae610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:36.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.747+0000 7f247dd9a700 1 --2- 172.21.15.119:0/270638028 >> v2:172.21.15.184:3300/0 conn(0x7f247806f4b0 0x7f24780ae610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.747+0000 7f247dd9a700 1 -- 172.21.15.119:0/270638028 >> v1:172.21.15.184:6789/0 conn(0x7f24780cb1a0 legacy=0x7f247815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.747+0000 7f247dd9a700 1 -- 172.21.15.119:0/270638028 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247815ad20 con 0x7f247806f060 2023-10-08T02:31:36.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.747+0000 7f247dd9a700 1 --2- 172.21.15.119:0/270638028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247806f060 0x7f24780ae0d0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f247806d480 tx=0x7f2468009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7faba788be5b53ce server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:36.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.747+0000 7f247d599700 1 -- 172.21.15.119:0/270638028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246800b8f0 con 0x7f247806f060 2023-10-08T02:31:36.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.747+0000 7f247d599700 1 -- 172.21.15.119:0/270638028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246800ba50 con 0x7f247806f060 2023-10-08T02:31:36.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.747+0000 7f247d599700 1 -- 172.21.15.119:0/270638028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246800bdf0 con 0x7f247806f060 2023-10-08T02:31:36.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.748+0000 7f247f59d700 1 -- 172.21.15.119:0/270638028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247806f060 msgr2=0x7f24780ae0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.748+0000 7f247f59d700 1 --2- 172.21.15.119:0/270638028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247806f060 0x7f24780ae0d0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f247806d480 tx=0x7f2468009250 comp rx=0 tx=0).stop 2023-10-08T02:31:36.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.748+0000 7f247f59d700 1 -- 172.21.15.119:0/270638028 shutdown_connections 2023-10-08T02:31:36.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.748+0000 7f247f59d700 1 --2- 172.21.15.119:0/270638028 >> v2:172.21.15.184:3300/0 conn(0x7f247806f4b0 0x7f24780ae610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.748+0000 7f247f59d700 1 --2- 172.21.15.119:0/270638028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247806f060 0x7f24780ae0d0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.748+0000 7f247f59d700 1 -- 172.21.15.119:0/270638028 >> 172.21.15.119:0/270638028 conn(0x7f24780cc1e0 msgr2=0x7f24780cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:36.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.748+0000 7f247f59d700 1 -- 172.21.15.119:0/270638028 shutdown_connections 2023-10-08T02:31:36.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.748+0000 7f247f59d700 1 -- 172.21.15.119:0/270638028 wait complete. 2023-10-08T02:31:36.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.749+0000 7f247f59d700 1 Processor -- start 2023-10-08T02:31:36.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.749+0000 7f247f59d700 1 -- start start 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.750+0000 7f247f59d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f247806f060 0x7f2478168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.750+0000 7f247f59d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f247806f4b0 0x7f247814b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.750+0000 7f247f59d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24780cb1a0 0x7f247814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.750+0000 7f247f59d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247814a850 con 0x7f24780cb1a0 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.750+0000 7f247f59d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2478167c10 con 0x7f247806f4b0 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.750+0000 7f247f59d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2478168860 con 0x7f247806f060 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.750+0000 7f247ed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24780cb1a0 0x7f247814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247ed9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24780cb1a0 0x7f247814b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41540/0 (socket says 172.21.15.119:41540) 2023-10-08T02:31:36.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247ed9c700 1 -- 172.21.15.119:0/3842095650 learned_addr learned my addr 172.21.15.119:0/3842095650 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247dd9a700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f247806f4b0 0x7f247814b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247e59b700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f247806f060 0x7f2478168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247ed9c700 1 -- 172.21.15.119:0/3842095650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f247806f060 msgr2=0x7f2478168320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247ed9c700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f247806f060 0x7f2478168320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247ed9c700 1 -- 172.21.15.119:0/3842095650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f247806f4b0 msgr2=0x7f247814b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247ed9c700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f247806f4b0 0x7f247814b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247ed9c700 1 -- 172.21.15.119:0/3842095650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24680094f0 con 0x7f24780cb1a0 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247ed9c700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24780cb1a0 0x7f247814b850 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f2470009fd0 tx=0x7f2470009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f246e7fc700 1 -- 172.21.15.119:0/3842095650 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247000ecf0 con 0x7f24780cb1a0 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f246e7fc700 1 -- 172.21.15.119:0/3842095650 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247000ee50 con 0x7f24780cb1a0 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247814fba0 con 0x7f24780cb1a0 2023-10-08T02:31:36.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.751+0000 7f246e7fc700 1 -- 172.21.15.119:0/3842095650 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470014670 con 0x7f24780cb1a0 2023-10-08T02:31:36.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.752+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247814bed0 con 0x7f24780cb1a0 2023-10-08T02:31:36.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.752+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f247814fd30 con 0x7f24780cb1a0 2023-10-08T02:31:36.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.753+0000 7f246e7fc700 1 -- 172.21.15.119:0/3842095650 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 112) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24700149e0 con 0x7f24780cb1a0 2023-10-08T02:31:36.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.754+0000 7f246e7fc700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f245c066f30 0x7f245c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:36.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.754+0000 7f246e7fc700 1 -- 172.21.15.119:0/3842095650 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f2470084f60 con 0x7f24780cb1a0 2023-10-08T02:31:36.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.754+0000 7f247e59b700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f245c066f30 0x7f245c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:36.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.755+0000 7f247e59b700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f245c066f30 0x7f245c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f24740037c0 tx=0x7f2474009040 comp rx=0 tx=0).ready entity=mgr.10020 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:36.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:36.758+0000 7f246e7fc700 1 -- 172.21.15.119:0/3842095650 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2470051f20 con 0x7f24780cb1a0 2023-10-08T02:31:37.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.003+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f247804bb80 con 0x7f24780cb1a0 2023-10-08T02:31:37.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.442+0000 7f246e7fc700 1 -- 172.21.15.119:0/3842095650 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 113) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f24700522f0 con 0x7f24780cb1a0 2023-10-08T02:31:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.443+0000 7f246e7fc700 1 -- 172.21.15.119:0/3842095650 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v113) v1 ==== 92+0+0 (secure 0 0 0) 0x7f24700559b0 con 0x7f24780cb1a0 2023-10-08T02:31:37.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f245c066f30 msgr2=0x7f245c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:37.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f245c066f30 0x7f245c0693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f24740037c0 tx=0x7f2474009040 comp rx=0 tx=0).stop 2023-10-08T02:31:37.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24780cb1a0 msgr2=0x7f247814b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:37.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24780cb1a0 0x7f247814b850 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f2470009fd0 tx=0x7f2470009900 comp rx=0 tx=0).stop 2023-10-08T02:31:37.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 shutdown_connections 2023-10-08T02:31:37.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f245c066f30 0x7f245c0693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f247806f060 0x7f2478168320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24780cb1a0 0x7f247814b850 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 --2- 172.21.15.119:0/3842095650 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f247806f4b0 0x7f247814b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 >> 172.21.15.119:0/3842095650 conn(0x7f24780cc1e0 msgr2=0x7f2478155920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:37.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 shutdown_connections 2023-10-08T02:31:37.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.447+0000 7f247f59d700 1 -- 172.21.15.119:0/3842095650 wait complete. 2023-10-08T02:31:37.473 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-10-08T02:31:37.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.664+0000 7f12a6b9d700 1 Processor -- start 2023-10-08T02:31:37.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.665+0000 7f12a6b9d700 1 -- start start 2023-10-08T02:31:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.666+0000 7f12a6b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f12a00b1b40 0x7f12a00b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.666+0000 7f12a6b9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a00cfb00 con 0x7f12a00b16d0 2023-10-08T02:31:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.666+0000 7f12a6b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12a00b5c40 con 0x7f12a00b4e70 2023-10-08T02:31:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.666+0000 7f12a6b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a00b5f20 con 0x7f12a00b1b40 2023-10-08T02:31:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.666+0000 7f12a539a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f12a00b16d0 0x7f12a00b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43246/0 (socket says 172.21.15.119:43246) 2023-10-08T02:31:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.666+0000 7f12a539a700 1 -- 172.21.15.119:0/1296573277 learned_addr learned my addr 172.21.15.119:0/1296573277 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.666+0000 7f12a639c700 1 --2- 172.21.15.119:0/1296573277 >> v2:172.21.15.184:3300/0 conn(0x7f12a00b1b40 0x7f12a00b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a4b99700 1 -- 172.21.15.119:0/1296573277 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2384684308 0 0) 0x7f12a00cfb00 con 0x7f12a00b16d0 2023-10-08T02:31:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a4b99700 1 -- 172.21.15.119:0/1296573277 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f128c003570 con 0x7f12a00b16d0 2023-10-08T02:31:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a639c700 1 -- 172.21.15.119:0/1296573277 >> v1:172.21.15.184:6789/0 conn(0x7f12a00b4e70 legacy=0x7f12a00ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a4b99700 1 -- 172.21.15.119:0/1296573277 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404773941 0 0) 0x7f12a00b5c40 con 0x7f12a00b4e70 2023-10-08T02:31:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a639c700 1 -- 172.21.15.119:0/1296573277 >> v1:172.21.15.119:6789/0 conn(0x7f12a00b16d0 legacy=0x7f12a00b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a639c700 1 -- 172.21.15.119:0/1296573277 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00b6200 con 0x7f12a00b1b40 2023-10-08T02:31:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a639c700 1 --2- 172.21.15.119:0/1296573277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b1b40 0x7f12a00b3ac0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1298009e60 tx=0x7f1298009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c54577f19284a8a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a4b99700 1 -- 172.21.15.119:0/1296573277 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129800e070 con 0x7f12a00b1b40 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.667+0000 7f12a4b99700 1 -- 172.21.15.119:0/1296573277 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129800bb60 con 0x7f12a00b1b40 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.668+0000 7f12a4b99700 1 -- 172.21.15.119:0/1296573277 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1298012430 con 0x7f12a00b1b40 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.668+0000 7f12a6b9d700 1 -- 172.21.15.119:0/1296573277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b1b40 msgr2=0x7f12a00b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.668+0000 7f12a6b9d700 1 --2- 172.21.15.119:0/1296573277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b1b40 0x7f12a00b3ac0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f1298009e60 tx=0x7f1298009780 comp rx=0 tx=0).stop 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.668+0000 7f12a6b9d700 1 -- 172.21.15.119:0/1296573277 shutdown_connections 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.668+0000 7f12a6b9d700 1 --2- 172.21.15.119:0/1296573277 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b1b40 0x7f12a00b3ac0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.668+0000 7f12a6b9d700 1 -- 172.21.15.119:0/1296573277 >> 172.21.15.119:0/1296573277 conn(0x7f12a00d10c0 msgr2=0x7f12a00affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.668+0000 7f12a6b9d700 1 -- 172.21.15.119:0/1296573277 shutdown_connections 2023-10-08T02:31:37.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.668+0000 7f12a6b9d700 1 -- 172.21.15.119:0/1296573277 wait complete. 2023-10-08T02:31:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.670+0000 7f12a6b9d700 1 Processor -- start 2023-10-08T02:31:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.670+0000 7f12a6b9d700 1 -- start start 2023-10-08T02:31:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.670+0000 7f12a6b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b16d0 0x7f12a0157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a6b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12a00b1b40 0x7f12a00c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a6b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a00b4e70 0x7f12a015b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a6b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00c6760 con 0x7f12a00b1b40 2023-10-08T02:31:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a6b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0158050 con 0x7f12a00b16d0 2023-10-08T02:31:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a6b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00c9460 con 0x7f12a00b4e70 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a539a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12a00b1b40 0x7f12a00c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a539a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12a00b1b40 0x7f12a00c5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41566/0 (socket says 172.21.15.119:41566) 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a539a700 1 -- 172.21.15.119:0/2445083122 learned_addr learned my addr 172.21.15.119:0/2445083122 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a5b9b700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b16d0 0x7f12a0157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a639c700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a00b4e70 0x7f12a015b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a539a700 1 -- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a00b4e70 msgr2=0x7f12a015b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a539a700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a00b4e70 0x7f12a015b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a539a700 1 -- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b16d0 msgr2=0x7f12a0157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a539a700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b16d0 0x7f12a0157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a539a700 1 -- 172.21.15.119:0/2445083122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298009a20 con 0x7f12a00b1b40 2023-10-08T02:31:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.671+0000 7f12a5b9b700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b16d0 0x7f12a0157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.672+0000 7f12a539a700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12a00b1b40 0x7f12a00c5390 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f129000c790 tx=0x7f1290007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.672+0000 7f1295ffb700 1 -- 172.21.15.119:0/2445083122 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290005b30 con 0x7f12a00b1b40 2023-10-08T02:31:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.672+0000 7f1295ffb700 1 -- 172.21.15.119:0/2445083122 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1290005c90 con 0x7f12a00b1b40 2023-10-08T02:31:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.672+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a00c5be0 con 0x7f12a00b1b40 2023-10-08T02:31:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.672+0000 7f1295ffb700 1 -- 172.21.15.119:0/2445083122 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290007320 con 0x7f12a00b1b40 2023-10-08T02:31:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.672+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a00c61a0 con 0x7f12a00b1b40 2023-10-08T02:31:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.673+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a00c95f0 con 0x7f12a00b1b40 2023-10-08T02:31:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.674+0000 7f1295ffb700 1 -- 172.21.15.119:0/2445083122 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 113) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f1290007690 con 0x7f12a00b1b40 2023-10-08T02:31:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.675+0000 7f1295ffb700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1284066f80 0x7f1284069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.675+0000 7f1295ffb700 1 -- 172.21.15.119:0/2445083122 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f1290083bc0 con 0x7f12a00b1b40 2023-10-08T02:31:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.675+0000 7f12a5b9b700 1 -- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1284066f80 msgr2=0x7f1284069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.675+0000 7f12a5b9b700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1284066f80 0x7f1284069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:37.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.679+0000 7f1295ffb700 1 -- 172.21.15.119:0/2445083122 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12a00c95f0 con 0x7f12a00b1b40 2023-10-08T02:31:37.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.876+0000 7f12a5b9b700 1 -- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1284066f80 msgr2=0x7f1284069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:37.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.876+0000 7f12a5b9b700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1284066f80 0x7f1284069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:37.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.877+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f12a00c95a0 con 0x7f12a00b1b40 2023-10-08T02:31:37.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.881+0000 7f1295ffb700 1 -- 172.21.15.119:0/2445083122 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1290017020 con 0x7f12a00b1b40 2023-10-08T02:31:37.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1284066f80 msgr2=0x7f1284069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:37.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1284066f80 0x7f1284069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12a00b1b40 msgr2=0x7f12a00c5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:37.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12a00b1b40 0x7f12a00c5390 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f129000c790 tx=0x7f1290007df0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 shutdown_connections 2023-10-08T02:31:37.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f1284066f80 0x7f1284069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f12a00b4e70 0x7f12a015b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f12a00b16d0 0x7f12a0157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 --2- 172.21.15.119:0/2445083122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f12a00b1b40 0x7f12a00c5390 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:37.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 >> 172.21.15.119:0/2445083122 conn(0x7f12a00d10c0 msgr2=0x7f12a00b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:37.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 shutdown_connections 2023-10-08T02:31:37.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:37.884+0000 7f12a6b9d700 1 -- 172.21.15.119:0/2445083122 wait complete. 2023-10-08T02:31:37.902 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-10-08T02:31:38.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.090+0000 7f0475f9d700 1 Processor -- start 2023-10-08T02:31:38.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.090+0000 7f0475f9d700 1 -- start start 2023-10-08T02:31:38.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.091+0000 7f0475f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f04700b4e70 0x7f04700ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.091+0000 7f0475f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04700cfb00 con 0x7f04700b16d0 2023-10-08T02:31:38.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.091+0000 7f0475f9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047015e060 con 0x7f04700b1b40 2023-10-08T02:31:38.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.091+0000 7f0475f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04700b5c40 con 0x7f04700b4e70 2023-10-08T02:31:38.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.092+0000 7f046ffff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f04700b16d0 0x7f047015d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43256/0 (socket says 172.21.15.119:43256) 2023-10-08T02:31:38.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.092+0000 7f046ffff700 1 -- 172.21.15.119:0/1020882775 learned_addr learned my addr 172.21.15.119:0/1020882775 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:38.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.092+0000 7f0474f9b700 1 --2- 172.21.15.119:0/1020882775 >> v2:172.21.15.184:3300/0 conn(0x7f04700b4e70 0x7f04700ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.092+0000 7f046f7fe700 1 -- 172.21.15.119:0/1020882775 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148759190 0 0) 0x7f04700cfb00 con 0x7f04700b16d0 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.092+0000 7f046f7fe700 1 -- 172.21.15.119:0/1020882775 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0460003570 con 0x7f04700b16d0 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.092+0000 7f0474f9b700 1 -- 172.21.15.119:0/1020882775 >> v1:172.21.15.184:6789/0 conn(0x7f04700b1b40 legacy=0x7f047015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.092+0000 7f0474f9b700 1 -- 172.21.15.119:0/1020882775 >> v1:172.21.15.119:6789/0 conn(0x7f04700b16d0 legacy=0x7f047015d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.092+0000 7f0474f9b700 1 -- 172.21.15.119:0/1020882775 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04700b5f20 con 0x7f04700b4e70 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f0474f9b700 1 --2- 172.21.15.119:0/1020882775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b4e70 0x7f04700ccff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f04640098e0 tx=0x7f04640091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1010803f169c408 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f046f7fe700 1 -- 172.21.15.119:0/1020882775 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046400b640 con 0x7f04700b4e70 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f046f7fe700 1 -- 172.21.15.119:0/1020882775 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046400b7a0 con 0x7f04700b4e70 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f046f7fe700 1 -- 172.21.15.119:0/1020882775 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046400bb40 con 0x7f04700b4e70 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f0475f9d700 1 -- 172.21.15.119:0/1020882775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b4e70 msgr2=0x7f04700ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f0475f9d700 1 --2- 172.21.15.119:0/1020882775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b4e70 0x7f04700ccff0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f04640098e0 tx=0x7f04640091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f0475f9d700 1 -- 172.21.15.119:0/1020882775 shutdown_connections 2023-10-08T02:31:38.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f0475f9d700 1 --2- 172.21.15.119:0/1020882775 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b4e70 0x7f04700ccff0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f0475f9d700 1 -- 172.21.15.119:0/1020882775 >> 172.21.15.119:0/1020882775 conn(0x7f04700d10c0 msgr2=0x7f04700affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:38.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.093+0000 7f0475f9d700 1 -- 172.21.15.119:0/1020882775 shutdown_connections 2023-10-08T02:31:38.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.094+0000 7f0475f9d700 1 -- 172.21.15.119:0/1020882775 wait complete. 2023-10-08T02:31:38.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.095+0000 7f0475f9d700 1 Processor -- start 2023-10-08T02:31:38.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.095+0000 7f0475f9d700 1 -- start start 2023-10-08T02:31:38.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f0475f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b16d0 0x7f04700cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f0475f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04700b1b40 0x7f0470169d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f0475f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04700b4e70 0x7f04701576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f0475f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047016ad40 con 0x7f04700b1b40 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f0475f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f047016a270 con 0x7f04700b16d0 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f0475f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04700c8770 con 0x7f04700b4e70 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f046ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04700b1b40 0x7f0470169d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f0474f9b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b16d0 0x7f04700cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f046ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04700b1b40 0x7f0470169d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41570/0 (socket says 172.21.15.119:41570) 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f046ffff700 1 -- 172.21.15.119:0/798067836 learned_addr learned my addr 172.21.15.119:0/798067836 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.096+0000 7f047579c700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04700b4e70 0x7f04701576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f0474f9b700 1 -- 172.21.15.119:0/798067836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04700b4e70 msgr2=0x7f04701576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f0474f9b700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04700b4e70 0x7f04701576e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f0474f9b700 1 -- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04700b1b40 msgr2=0x7f0470169d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f0474f9b700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04700b1b40 0x7f0470169d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f0474f9b700 1 -- 172.21.15.119:0/798067836 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0464009420 con 0x7f04700b16d0 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f0474f9b700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b16d0 0x7f04700cac00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0470089830 tx=0x7f0464017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f046cff9700 1 -- 172.21.15.119:0/798067836 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046400bb20 con 0x7f04700b16d0 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f046cff9700 1 -- 172.21.15.119:0/798067836 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0464000c90 con 0x7f04700b16d0 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f046cff9700 1 -- 172.21.15.119:0/798067836 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0464011630 con 0x7f04700b16d0 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047016a4a0 con 0x7f04700b16d0 2023-10-08T02:31:38.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.097+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f047016aa60 con 0x7f04700b16d0 2023-10-08T02:31:38.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.098+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0470088990 con 0x7f04700b16d0 2023-10-08T02:31:38.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.099+0000 7f046cff9700 1 -- 172.21.15.119:0/798067836 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 113) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f04640119a0 con 0x7f04700b16d0 2023-10-08T02:31:38.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.100+0000 7f046cff9700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0454066f80 0x7f0454069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.100+0000 7f046cff9700 1 -- 172.21.15.119:0/798067836 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f0464085990 con 0x7f04700b16d0 2023-10-08T02:31:38.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.100+0000 7f046ffff700 1 -- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0454066f80 msgr2=0x7f0454069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:38.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.100+0000 7f046ffff700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0454066f80 0x7f0454069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:38.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.105+0000 7f046cff9700 1 -- 172.21.15.119:0/798067836 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f04640529d0 con 0x7f04700b16d0 2023-10-08T02:31:38.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.301+0000 7f046ffff700 1 -- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0454066f80 msgr2=0x7f0454069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:38.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.301+0000 7f046ffff700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0454066f80 0x7f0454069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:38.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.303+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f047004bb80 con 0x7f04700b16d0 2023-10-08T02:31:38.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.308+0000 7f046cff9700 1 -- 172.21.15.119:0/798067836 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0464015020 con 0x7f04700b16d0 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0454066f80 msgr2=0x7f0454069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0454066f80 0x7f0454069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b16d0 msgr2=0x7f04700cac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b16d0 0x7f04700cac00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0470089830 tx=0x7f0464017000 comp rx=0 tx=0).stop 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 shutdown_connections 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0454066f80 0x7f0454069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f04700b4e70 0x7f04701576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f04700b16d0 0x7f04700cac00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 --2- 172.21.15.119:0/798067836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f04700b1b40 0x7f0470169d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 >> 172.21.15.119:0/798067836 conn(0x7f04700d10c0 msgr2=0x7f04700b5520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 shutdown_connections 2023-10-08T02:31:38.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.311+0000 7f0475f9d700 1 -- 172.21.15.119:0/798067836 wait complete. 2023-10-08T02:31:38.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:38.329 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-10-08T02:31:38.371 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-10-08T02:31:38.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.603+0000 7f171b59e700 1 Processor -- start 2023-10-08T02:31:38.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.603+0000 7f171b59e700 1 -- start start 2023-10-08T02:31:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f171b59e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f171c0b0790 0x7f171c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f171b59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f171c1591e0 0x7f171c159600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f171b59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f171c0cfa00 con 0x7f171c0b03c0 2023-10-08T02:31:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f171b59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c0cb580 con 0x7f171c1591e0 2023-10-08T02:31:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f171b59e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f171c0cf160 con 0x7f171c0b0790 2023-10-08T02:31:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f171ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f171c1591e0 0x7f171c159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f171ad9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f171c1591e0 0x7f171c159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41578/0 (socket says 172.21.15.119:41578) 2023-10-08T02:31:38.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f171ad9d700 1 -- 172.21.15.119:0/3706952243 learned_addr learned my addr 172.21.15.119:0/3706952243 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:38.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.604+0000 7f1719d9b700 1 --2- 172.21.15.119:0/3706952243 >> v2:172.21.15.184:3300/0 conn(0x7f171c0b0790 0x7f171c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.605+0000 7f171ad9d700 1 -- 172.21.15.119:0/3706952243 >> v2:172.21.15.184:3300/0 conn(0x7f171c0b0790 msgr2=0x7f171c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:38.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.605+0000 7f171ad9d700 1 --2- 172.21.15.119:0/3706952243 >> v2:172.21.15.184:3300/0 conn(0x7f171c0b0790 0x7f171c0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.605+0000 7f171ad9d700 1 -- 172.21.15.119:0/3706952243 >> v1:172.21.15.184:6789/0 conn(0x7f171c0b03c0 legacy=0x7f171c0b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.605+0000 7f171ad9d700 1 -- 172.21.15.119:0/3706952243 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c0cf440 con 0x7f171c1591e0 2023-10-08T02:31:38.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.605+0000 7f171ad9d700 1 --2- 172.21.15.119:0/3706952243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c159600 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f1714009f20 tx=0x7f1714009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6cdd9700dfc74a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.605+0000 7f171959a700 1 -- 172.21.15.119:0/3706952243 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171400e070 con 0x7f171c1591e0 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.605+0000 7f171959a700 1 -- 172.21.15.119:0/3706952243 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171400b990 con 0x7f171c1591e0 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171959a700 1 -- 172.21.15.119:0/3706952243 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140092e0 con 0x7f171c1591e0 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171b59e700 1 -- 172.21.15.119:0/3706952243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 msgr2=0x7f171c159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171b59e700 1 --2- 172.21.15.119:0/3706952243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c159600 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f1714009f20 tx=0x7f1714009840 comp rx=0 tx=0).stop 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171b59e700 1 -- 172.21.15.119:0/3706952243 shutdown_connections 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171b59e700 1 --2- 172.21.15.119:0/3706952243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c159600 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171b59e700 1 --2- 172.21.15.119:0/3706952243 >> v2:172.21.15.184:3300/0 conn(0x7f171c0b0790 0x7f171c0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171b59e700 1 -- 172.21.15.119:0/3706952243 >> 172.21.15.119:0/3706952243 conn(0x7f171c063230 msgr2=0x7f171c063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171b59e700 1 -- 172.21.15.119:0/3706952243 shutdown_connections 2023-10-08T02:31:38.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.606+0000 7f171b59e700 1 -- 172.21.15.119:0/3706952243 wait complete. 2023-10-08T02:31:38.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.608+0000 7f171b59e700 1 Processor -- start 2023-10-08T02:31:38.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.608+0000 7f171b59e700 1 -- start start 2023-10-08T02:31:38.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171b59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0b03c0 0x7f171c15a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171b59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0b0790 0x7f171c0c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171b59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171b59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0c5570 con 0x7f171c1591e0 2023-10-08T02:31:38.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171b59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171c0c59a0 con 0x7f171c0b03c0 2023-10-08T02:31:38.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171b59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171c158f60 con 0x7f171c0b0790 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0b03c0 0x7f171c15a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171ad9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41588/0 (socket says 172.21.15.119:41588) 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171ad9d700 1 -- 172.21.15.119:0/1860477267 learned_addr learned my addr 172.21.15.119:0/1860477267 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f171a59c700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0b03c0 0x7f171c15a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:46762/0 (socket says 172.21.15.119:46762) 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.609+0000 7f1719d9b700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0b0790 0x7f171c0c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f171ad9d700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0b0790 msgr2=0x7f171c0c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f171ad9d700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0b0790 0x7f171c0c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f171ad9d700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0b03c0 msgr2=0x7f171c15a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:38.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f171ad9d700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0b03c0 0x7f171c15a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f171ad9d700 1 -- 172.21.15.119:0/1860477267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1714009ae0 con 0x7f171c1591e0 2023-10-08T02:31:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f171ad9d700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c158a20 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f171c0cfe40 tx=0x7f1714018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17140078b0 con 0x7f171c1591e0 2023-10-08T02:31:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1714007a30 con 0x7f171c1591e0 2023-10-08T02:31:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171c15cd30 con 0x7f171c1591e0 2023-10-08T02:31:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714007df0 con 0x7f171c1591e0 2023-10-08T02:31:38.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.610+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f171c15cfd0 con 0x7f171c1591e0 2023-10-08T02:31:38.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.612+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171c04c3c0 con 0x7f171c1591e0 2023-10-08T02:31:38.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.612+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 113) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f1714026670 con 0x7f171c1591e0 2023-10-08T02:31:38.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.613+0000 7f170a7fc700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:38.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.613+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 6225+0+0 (secure 0 0 0) 0x7f1714086560 con 0x7f171c1591e0 2023-10-08T02:31:38.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.613+0000 7f171a59c700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 msgr2=0x7f16fc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:38.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.613+0000 7f171a59c700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:38.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.617+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1714053520 con 0x7f171c1591e0 2023-10-08T02:31:38.719 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:38.719+0000 7fe368ef9180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:31:38.785 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:38.785+0000 7f1ed8701180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:31:38.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.814+0000 7f171a59c700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 msgr2=0x7f16fc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:38.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.814+0000 7f171a59c700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:38.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:38.840+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f171c0c5780 con 0x7f16fc066f80 2023-10-08T02:31:38.884 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:38.884+0000 7fe368ef9180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:38.951 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:38.952+0000 7f1ed8701180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:38.968 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:38.968+0000 7fe368ef9180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:31:39.036 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:39.037+0000 7f1ed8701180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:31:39.193 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:39.193+0000 7fe368ef9180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:39.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:39.215+0000 7f171a59c700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 msgr2=0x7f16fc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:39.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:39.215+0000 7f171a59c700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:31:39.262 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:39.262+0000 7f1ed8701180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:39.371 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:39.371+0000 7fe368ef9180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:31:39.440 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:39.441+0000 7f1ed8701180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:31:39.462 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:39.462+0000 7fe368ef9180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:31:39.532 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:39.532+0000 7f1ed8701180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:31:39.625 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:39.625+0000 7fe368ef9180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:31:39.693 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:39.693+0000 7f1ed8701180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:31:39.726 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:39.726+0000 7fe368ef9180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:39.793 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:39.793+0000 7f1ed8701180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:40.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:40.016+0000 7f171a59c700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 msgr2=0x7f16fc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:40.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:40.016+0000 7f171a59c700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-10-08T02:31:40.208 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:40.208+0000 7fe368ef9180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:31:40.277 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:40.277+0000 7f1ed8701180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:31:40.301 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:40.301+0000 7fe368ef9180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:31:40.371 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:40.371+0000 7f1ed8701180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:31:40.390 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:40.390+0000 7fe368ef9180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:31:40.461 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:40.461+0000 7f1ed8701180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:31:41.011 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:41.011+0000 7fe368ef9180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:31:41.080 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:41.080+0000 7f1ed8701180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:31:41.184 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:41.184+0000 7fe368ef9180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:41.252 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:41.253+0000 7f1ed8701180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:41.279 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:41.279+0000 7fe368ef9180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:31:41.350 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:41.351+0000 7f1ed8701180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:31:41.574 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:41.575+0000 7fe368ef9180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:31:41.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:41.617+0000 7f171a59c700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 msgr2=0x7f16fc069430 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:41.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:41.617+0000 7f171a59c700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-10-08T02:31:41.644 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:41.644+0000 7f1ed8701180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:31:41.669 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:41.669+0000 7fe368ef9180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:31:41.740 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:41.740+0000 7f1ed8701180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:31:42.367 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:42.367+0000 7fe368ef9180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:31:42.446 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:42.446+0000 7f1ed8701180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:31:42.655 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:42.655+0000 7fe368ef9180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:31:42.732 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:42.732+0000 7f1ed8701180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:31:42.747 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:42.747+0000 7fe368ef9180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:31:42.826 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:42.827+0000 7f1ed8701180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:31:42.899 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:42.900+0000 7fe368ef9180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:31:42.981 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:42.982+0000 7f1ed8701180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:31:43.003 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:43.004+0000 7fe368ef9180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:31:43.087 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.088+0000 7f1ed8701180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:31:43.193 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:43.193+0000 7fe368ef9180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:31:43.281 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.281+0000 7f1ed8701180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:31:43.284 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:43.285+0000 7fe368ef9180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:31:43.373 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.373+0000 7f1ed8701180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:31:43.375 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:43.375+0000 7fe368ef9180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:31:43.469 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.469+0000 7f1ed8701180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:31:43.471 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:43.471+0000 7fe368ef9180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:31:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:43.530+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 114) v1 ==== 84757+0+0 (secure 0 0 0) 0x7f17140538f0 con 0x7f171c1591e0 2023-10-08T02:31:43.566 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.566+0000 7f1ed8701180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:31:43.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:43.574+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mgrmap(e 115) v1 ==== 84468+0+0 (secure 0 0 0) 0x7f1714028d20 con 0x7f171c1591e0 2023-10-08T02:31:43.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:43.575+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 msgr2=0x7f16fc069430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:43.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:43.575+0000 7f170a7fc700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc069430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:43.698 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.696+0000 7f1e4aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.698 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.696+0000 7f1e4aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.698 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.696+0000 7f1e4aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.698 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.696+0000 7f1e4aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.698 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.696+0000 7f1e4aa31700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.698 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.697+0000 7f1e4d236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.699 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.697+0000 7f1e4d236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.699 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.697+0000 7f1e4d236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.699 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.697+0000 7f1e4d236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:43.699 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:43.697+0000 7f1e4d236700 -1 client.0 error registering admin socket command: (17) File exists 2023-10-08T02:31:44.058 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:31:44.063 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:31:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.578+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mon.0 v2:172.21.15.119:3300/0 9 ==== mgrmap(e 116) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f17140541c0 con 0x7f171c1591e0 2023-10-08T02:31:44.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.578+0000 7f170a7fc700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc0693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:44.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.578+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f171c0c5780 con 0x7f16fc066f80 2023-10-08T02:31:44.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.581+0000 7f171a59c700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc0693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:44.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.585+0000 7f171a59c700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc0693b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f170c001200 tx=0x7f170c007ec0 comp rx=0 tx=0).ready entity=mgr.10395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:44.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.640+0000 7f170a7fc700 1 -- 172.21.15.119:0/1860477267 <== mgr.10395 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f171c0c5780 con 0x7f16fc066f80 2023-10-08T02:31:44.641 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule set for path snap_test_dir1 2023-10-08T02:31:44.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 msgr2=0x7f16fc0693b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:44.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc0693b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f170c001200 tx=0x7f170c007ec0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 msgr2=0x7f171c158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c158a20 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f171c0cfe40 tx=0x7f1714018000 comp rx=0 tx=0).stop 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 shutdown_connections 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f16fc066f80 0x7f16fc0693b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f171c1591e0 0x7f171c158a20 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f171c0b03c0 0x7f171c15a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 --2- 172.21.15.119:0/1860477267 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f171c0b0790 0x7f171c0c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.642+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 >> 172.21.15.119:0/1860477267 conn(0x7f171c063230 msgr2=0x7f171c0d0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.643+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 shutdown_connections 2023-10-08T02:31:44.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.643+0000 7f171b59e700 1 -- 172.21.15.119:0/1860477267 wait complete. 2023-10-08T02:31:44.661 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-10-08T02:31:44.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-10-08T02:31:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.847+0000 7f160eb9d700 1 Processor -- start 2023-10-08T02:31:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.848+0000 7f160eb9d700 1 -- start start 2023-10-08T02:31:44.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.849+0000 7f160eb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16080caee0 0x7f16080ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:44.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.849+0000 7f160eb9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1608156780 0x7f16080ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:44.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.849+0000 7f160eb9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16080cbe50 con 0x7f16081563b0 2023-10-08T02:31:44.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.849+0000 7f160eb9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16080cb370 con 0x7f1608156780 2023-10-08T02:31:44.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.849+0000 7f160eb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16080cb650 con 0x7f16080caee0 2023-10-08T02:31:44.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.849+0000 7f160e39c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1608156780 0x7f16080ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:44.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.849+0000 7f160db9b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f16081563b0 0x7f16080ca7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:43270/0 (socket says 172.21.15.119:43270) 2023-10-08T02:31:44.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.849+0000 7f160db9b700 1 -- 172.21.15.119:0/2018344763 learned_addr learned my addr 172.21.15.119:0/2018344763 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160d39a700 1 --2- 172.21.15.119:0/2018344763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16080caee0 0x7f16080ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160e39c700 1 -- 172.21.15.119:0/2018344763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16080caee0 msgr2=0x7f16080ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:44.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160e39c700 1 --2- 172.21.15.119:0/2018344763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16080caee0 0x7f16080ae2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160e39c700 1 -- 172.21.15.119:0/2018344763 >> v1:172.21.15.119:6789/0 conn(0x7f16081563b0 legacy=0x7f16080ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160e39c700 1 -- 172.21.15.119:0/2018344763 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16080c6860 con 0x7f1608156780 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160cb99700 1 -- 172.21.15.119:0/2018344763 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226540986 0 0) 0x7f16080cbe50 con 0x7f16081563b0 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160e39c700 1 --2- 172.21.15.119:0/2018344763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1608156780 0x7f16080ae7f0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f1600009e60 tx=0x7f1600009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a06e420ccbc99f4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160cb99700 1 -- 172.21.15.119:0/2018344763 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160000e070 con 0x7f1608156780 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160cb99700 1 -- 172.21.15.119:0/2018344763 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160000bc30 con 0x7f1608156780 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.850+0000 7f160cb99700 1 -- 172.21.15.119:0/2018344763 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1600012430 con 0x7f1608156780 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.851+0000 7f160eb9d700 1 -- 172.21.15.119:0/2018344763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1608156780 msgr2=0x7f16080ae7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.851+0000 7f160eb9d700 1 --2- 172.21.15.119:0/2018344763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1608156780 0x7f16080ae7f0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f1600009e60 tx=0x7f1600009780 comp rx=0 tx=0).stop 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.851+0000 7f160eb9d700 1 -- 172.21.15.119:0/2018344763 shutdown_connections 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.851+0000 7f160eb9d700 1 --2- 172.21.15.119:0/2018344763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1608156780 0x7f16080ae7f0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.851+0000 7f160eb9d700 1 --2- 172.21.15.119:0/2018344763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16080caee0 0x7f16080ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.851+0000 7f160eb9d700 1 -- 172.21.15.119:0/2018344763 >> 172.21.15.119:0/2018344763 conn(0x7f1608068e20 msgr2=0x7f16080cc220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:44.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.851+0000 7f160eb9d700 1 -- 172.21.15.119:0/2018344763 shutdown_connections 2023-10-08T02:31:44.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.851+0000 7f160eb9d700 1 -- 172.21.15.119:0/2018344763 wait complete. 2023-10-08T02:31:44.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.852+0000 7f160eb9d700 1 Processor -- start 2023-10-08T02:31:44.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.853+0000 7f160eb9d700 1 -- start start 2023-10-08T02:31:44.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.853+0000 7f160eb9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16080caee0 0x7f1608149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:44.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.853+0000 7f160eb9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16081563b0 0x7f1608156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:44.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.853+0000 7f160eb9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1608156780 0x7f1608154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:44.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.853+0000 7f160eb9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160814a2d0 con 0x7f16080caee0 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.853+0000 7f160eb9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16080a9e00 con 0x7f1608156780 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.853+0000 7f160eb9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16080ab200 con 0x7f16081563b0 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16080caee0 0x7f1608149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16080caee0 0x7f1608149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41756/0 (socket says 172.21.15.119:41756) 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 -- 172.21.15.119:0/1461270312 learned_addr learned my addr 172.21.15.119:0/1461270312 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160d39a700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16081563b0 0x7f1608156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160e39c700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1608156780 0x7f1608154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 -- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16081563b0 msgr2=0x7f1608156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16081563b0 0x7f1608156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 -- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1608156780 msgr2=0x7f1608154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:44.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1608156780 0x7f1608154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 -- 172.21.15.119:0/1461270312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600009a20 con 0x7f16080caee0 2023-10-08T02:31:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160d39a700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16081563b0 0x7f1608156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160e39c700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1608156780 0x7f1608154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.854+0000 7f160db9b700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16080caee0 0x7f1608149d90 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f160400c7e0 tx=0x7f1604007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.855+0000 7f15fdffb700 1 -- 172.21.15.119:0/1461270312 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604005b30 con 0x7f16080caee0 2023-10-08T02:31:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.855+0000 7f15fdffb700 1 -- 172.21.15.119:0/1461270312 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1604005c90 con 0x7f16080caee0 2023-10-08T02:31:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.855+0000 7f15fdffb700 1 -- 172.21.15.119:0/1461270312 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1604007300 con 0x7f16080caee0 2023-10-08T02:31:44.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.855+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16080ab4e0 con 0x7f16080caee0 2023-10-08T02:31:44.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.855+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16080aaef0 con 0x7f16080caee0 2023-10-08T02:31:44.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.856+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16080a98d0 con 0x7f16080caee0 2023-10-08T02:31:44.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.856+0000 7f15fdffb700 1 -- 172.21.15.119:0/1461270312 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 116) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f160401b400 con 0x7f16080caee0 2023-10-08T02:31:44.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.857+0000 7f15fdffb700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f15ec066dd0 0x7f15ec069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:44.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.857+0000 7f15fdffb700 1 -- 172.21.15.119:0/1461270312 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f1604084190 con 0x7f16080caee0 2023-10-08T02:31:44.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.858+0000 7f160d39a700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f15ec066dd0 0x7f15ec069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:44.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.858+0000 7f160d39a700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f15ec066dd0 0x7f15ec069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16080c82e0 tx=0x7f15f8008040 comp rx=0 tx=0).ready entity=mgr.10395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:44.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:44.861+0000 7f15fdffb700 1 -- 172.21.15.119:0/1461270312 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1604051350 con 0x7f16080caee0 2023-10-08T02:31:45.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.076+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f16080a9f40 con 0x7f15ec066dd0 2023-10-08T02:31:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.130+0000 7f15fdffb700 1 -- 172.21.15.119:0/1461270312 <== mgr.10395 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f15f80097d0 con 0x7f15ec066dd0 2023-10-08T02:31:45.131 INFO:teuthology.orchestra.run.smithi119.stdout:Retention added to path snap_test_dir1 2023-10-08T02:31:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.132+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f15ec066dd0 msgr2=0x7f15ec069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f15ec066dd0 0x7f15ec069280 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f16080c82e0 tx=0x7f15f8008040 comp rx=0 tx=0).stop 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16080caee0 msgr2=0x7f1608149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16080caee0 0x7f1608149d90 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f160400c7e0 tx=0x7f1604007df0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 shutdown_connections 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f15ec066dd0 0x7f15ec069280 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f1608156780 0x7f1608154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16080caee0 0x7f1608149d90 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 --2- 172.21.15.119:0/1461270312 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f16081563b0 0x7f1608156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 >> 172.21.15.119:0/1461270312 conn(0x7f1608068e20 msgr2=0x7f16080c75c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 shutdown_connections 2023-10-08T02:31:45.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.133+0000 7f160eb9d700 1 -- 172.21.15.119:0/1461270312 wait complete. 2023-10-08T02:31:45.151 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-10-08T02:31:45.155 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-10-08-02_32_00 in ~15s... 2023-10-08T02:31:45.155 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-10-08T02:31:45.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-10-08T02:31:45.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.339+0000 7f2339fff700 1 Processor -- start 2023-10-08T02:31:45.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.339+0000 7f2339fff700 1 -- start start 2023-10-08T02:31:45.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f2339fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23341560b0 0x7f23340cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f2339fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233406df80 con 0x7f23340c6890 2023-10-08T02:31:45.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f2339fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340c79a0 con 0x7f2334156500 2023-10-08T02:31:45.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f2339fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23340acc60 con 0x7f23341560b0 2023-10-08T02:31:45.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f23337fe700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f23340c6890 0x7f23340ac550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41584/0 (socket says 172.21.15.119:41584) 2023-10-08T02:31:45.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f23337fe700 1 -- 172.21.15.119:0/3042677101 learned_addr learned my addr 172.21.15.119:0/3042677101 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:45.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614296905 0 0) 0x7f233406df80 con 0x7f23340c6890 2023-10-08T02:31:45.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f2332ffd700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23341560b0 0x7f23340cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.341+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c003570 con 0x7f23340c6890 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3513492467 0 0) 0x7f231c003570 con 0x7f23340c6890 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23341560b0 msgr2=0x7f23340cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23341560b0 0x7f23340cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 >> v1:172.21.15.184:6789/0 conn(0x7f2334156500 legacy=0x7f233406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340c6d90 con 0x7f23340c6890 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f23240015f0 con 0x7f23340c6890 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 >> v1:172.21.15.119:6789/0 conn(0x7f23340c6890 legacy=0x7f23340ac550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231c003a90 0x7f231c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23341560b0 0x7f231c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f231c004d50 0x7f231c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340acc60 con 0x7f231c004d50 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c00dd80 con 0x7f231c003a90 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c00df40 con 0x7f23341560b0 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f2333fff700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f231c004d50 0x7f231c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f2332ffd700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23341560b0 0x7f231c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f23337fe700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231c003a90 0x7f231c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f2333fff700 1 -- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23341560b0 msgr2=0x7f231c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f2333fff700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23341560b0 0x7f231c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f2333fff700 1 -- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231c003a90 msgr2=0x7f231c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f2333fff700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f231c003a90 0x7f231c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.342+0000 7f2333fff700 1 -- 172.21.15.119:0/3042677101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c003570 con 0x7f231c004d50 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f2333fff700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f231c004d50 0x7f231c0051c0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f23340c7950 tx=0x7f232c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2e72ae341dd81f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c001950 con 0x7f231c004d50 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232c004030 con 0x7f231c004d50 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f23327fc700 1 -- 172.21.15.119:0/3042677101 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232c00cae0 con 0x7f231c004d50 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f2339fff700 1 -- 172.21.15.119:0/3042677101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f231c004d50 msgr2=0x7f231c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f2339fff700 1 --2- 172.21.15.119:0/3042677101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f231c004d50 0x7f231c0051c0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f23340c7950 tx=0x7f232c001640 comp rx=0 tx=0).stop 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f2333fff700 1 -- 172.21.15.119:0/3042677101 reap_dead start 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f2339fff700 1 -- 172.21.15.119:0/3042677101 shutdown_connections 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.343+0000 7f2339fff700 1 -- 172.21.15.119:0/3042677101 >> 172.21.15.119:0/3042677101 conn(0x7f2334068390 msgr2=0x7f2334068790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.344+0000 7f2339fff700 1 -- 172.21.15.119:0/3042677101 shutdown_connections 2023-10-08T02:31:45.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.344+0000 7f2339fff700 1 -- 172.21.15.119:0/3042677101 wait complete. 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.345+0000 7f2339fff700 1 Processor -- start 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.345+0000 7f2339fff700 1 -- start start 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f2339fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233414dbc0 0x7f23340b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f2339fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f233414df90 0x7f23340c1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f2339fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23340c3300 0x7f23340c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f2339fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340c14e0 con 0x7f233414dbc0 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f2339fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340c5a60 con 0x7f233414df90 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f2339fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23340c5d20 con 0x7f23340c3300 2023-10-08T02:31:45.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f23337fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233414dbc0 0x7f23340b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f23337fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233414dbc0 0x7f23340b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42734/0 (socket says 172.21.15.119:42734) 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f23337fe700 1 -- 172.21.15.119:0/1461365705 learned_addr learned my addr 172.21.15.119:0/1461365705 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f2332ffd700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f233414df90 0x7f23340c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.346+0000 7f2333fff700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23340c3300 0x7f23340c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f23337fe700 1 -- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23340c3300 msgr2=0x7f23340c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f23337fe700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23340c3300 0x7f23340c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f23337fe700 1 -- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f233414df90 msgr2=0x7f23340c1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f23337fe700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f233414df90 0x7f23340c1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f23337fe700 1 -- 172.21.15.119:0/1461365705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340c6000 con 0x7f233414dbc0 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f2332ffd700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f233414df90 0x7f23340c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f23337fe700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233414dbc0 0x7f23340b00b0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f232400f9d0 tx=0x7f2324009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:45.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f231b7fe700 1 -- 172.21.15.119:0/1461365705 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23240072e0 con 0x7f233414dbc0 2023-10-08T02:31:45.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f231b7fe700 1 -- 172.21.15.119:0/1461365705 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2324007440 con 0x7f233414dbc0 2023-10-08T02:31:45.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f231b7fe700 1 -- 172.21.15.119:0/1461365705 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23240057f0 con 0x7f233414dbc0 2023-10-08T02:31:45.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23340c5690 con 0x7f233414dbc0 2023-10-08T02:31:45.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.347+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23340c23f0 con 0x7f233414dbc0 2023-10-08T02:31:45.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.348+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23340c5820 con 0x7f233414dbc0 2023-10-08T02:31:45.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.349+0000 7f231b7fe700 1 -- 172.21.15.119:0/1461365705 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 116) v1 ==== 84595+0+0 (secure 0 0 0) 0x7f2324005bc0 con 0x7f233414dbc0 2023-10-08T02:31:45.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.350+0000 7f231b7fe700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2320066dd0 0x7f2320069280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.350+0000 7f231b7fe700 1 -- 172.21.15.119:0/1461365705 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f23240847b0 con 0x7f233414dbc0 2023-10-08T02:31:45.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.350+0000 7f2332ffd700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2320066dd0 0x7f2320069280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.351+0000 7f2332ffd700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2320066dd0 0x7f2320069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f23341519e0 tx=0x7f2328008040 comp rx=0 tx=0).ready entity=mgr.10395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:45.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.354+0000 7f231b7fe700 1 -- 172.21.15.119:0/1461365705 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23340c5820 con 0x7f233414dbc0 2023-10-08T02:31:45.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.571+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23340c1620 con 0x7f2320066dd0 2023-10-08T02:31:45.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.578+0000 7f231b7fe700 1 -- 172.21.15.119:0/1461365705 <== mgr.10395 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f2328009aa0 con 0x7f2320066dd0 2023-10-08T02:31:45.579 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:45.579 INFO:teuthology.orchestra.run.smithi119.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.580+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2320066dd0 msgr2=0x7f2320069280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.580+0000 7f2339fff700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2320066dd0 0x7f2320069280 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f23341519e0 tx=0x7f2328008040 comp rx=0 tx=0).stop 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.580+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233414dbc0 msgr2=0x7f23340b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.580+0000 7f2339fff700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233414dbc0 0x7f23340b00b0 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f232400f9d0 tx=0x7f2324009040 comp rx=0 tx=0).stop 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.581+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 shutdown_connections 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.581+0000 7f2339fff700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f2320066dd0 0x7f2320069280 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.581+0000 7f2339fff700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f23340c3300 0x7f23340c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.581+0000 7f2339fff700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f233414dbc0 0x7f23340b00b0 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.581+0000 7f2339fff700 1 --2- 172.21.15.119:0/1461365705 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f233414df90 0x7f23340c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.581+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 >> 172.21.15.119:0/1461365705 conn(0x7f2334068390 msgr2=0x7f233406d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:45.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.581+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 shutdown_connections 2023-10-08T02:31:45.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.581+0000 7f2339fff700 1 -- 172.21.15.119:0/1461365705 wait complete. 2023-10-08T02:31:45.603 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-10-08T02:31:45.604 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-10-08T02:31:45.604 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-10-08T02:31:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.789+0000 7fecacd9d700 1 Processor -- start 2023-10-08T02:31:45.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.789+0000 7fecacd9d700 1 -- start start 2023-10-08T02:31:45.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.790+0000 7fecacd9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7feca80b0760 0x7feca80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.790+0000 7fecacd9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca80b0570 con 0x7feca80ace60 2023-10-08T02:31:45.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.790+0000 7fecacd9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca80af4d0 con 0x7feca8156350 2023-10-08T02:31:45.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.790+0000 7fecacd9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca80b1140 con 0x7feca80b0760 2023-10-08T02:31:45.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.790+0000 7feca7fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7feca80ace60 0x7feca80ad260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41596/0 (socket says 172.21.15.119:41596) 2023-10-08T02:31:45.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.790+0000 7feca7fff700 1 -- 172.21.15.119:0/3760964589 learned_addr learned my addr 172.21.15.119:0/3760964589 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca6ffd700 1 --2- 172.21.15.119:0/3760964589 >> v2:172.21.15.184:3300/0 conn(0x7feca80b0760 0x7feca80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633851616 0 0) 0x7feca80b0570 con 0x7feca80ace60 2023-10-08T02:31:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec90003570 con 0x7feca80ace60 2023-10-08T02:31:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854456987 0 0) 0x7fec90003570 con 0x7feca80ace60 2023-10-08T02:31:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 >> v2:172.21.15.184:3300/0 conn(0x7feca80b0760 msgr2=0x7feca80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:45.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 --2- 172.21.15.119:0/3760964589 >> v2:172.21.15.184:3300/0 conn(0x7feca80b0760 0x7feca80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 >> v1:172.21.15.184:6789/0 conn(0x7feca8156350 legacy=0x7feca8157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80af7b0 con 0x7feca80ace60 2023-10-08T02:31:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7feca00017f0 con 0x7feca80ace60 2023-10-08T02:31:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 >> v1:172.21.15.119:6789/0 conn(0x7feca80ace60 legacy=0x7feca80ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90003ab0 0x7fec90003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec90004810 0x7fec90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec90005210 0x7fec9000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca80b1140 con 0x7fec90004810 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9000c1b0 con 0x7fec90005210 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.791+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9000c370 con 0x7fec90003ab0 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca6ffd700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec90004810 0x7fec90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca7fff700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec90005210 0x7fec9000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca77fe700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90003ab0 0x7fec90003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca6ffd700 1 -- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90003ab0 msgr2=0x7fec90003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca6ffd700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90003ab0 0x7fec90003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca6ffd700 1 -- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec90005210 msgr2=0x7fec9000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:45.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca6ffd700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec90005210 0x7fec9000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca6ffd700 1 -- 172.21.15.119:0/3760964589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec90003570 con 0x7fec90004810 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca7fff700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fec90005210 0x7fec9000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca7fff700 1 -- 172.21.15.119:0/3760964589 reap_dead start 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca6ffd700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec90004810 0x7fec90004c80 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009c00 tx=0x7fec9c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6729a646225dd7b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00e710 con 0x7fec90004810 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c00e870 con 0x7fec90004810 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.792+0000 7feca67fc700 1 -- 172.21.15.119:0/3760964589 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00ec30 con 0x7fec90004810 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.793+0000 7fecacd9d700 1 -- 172.21.15.119:0/3760964589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec90004810 msgr2=0x7fec90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.793+0000 7fecacd9d700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec90004810 0x7fec90004c80 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009c00 tx=0x7fec9c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.793+0000 7fecacd9d700 1 -- 172.21.15.119:0/3760964589 shutdown_connections 2023-10-08T02:31:45.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.793+0000 7fecacd9d700 1 --2- 172.21.15.119:0/3760964589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec90004810 0x7fec90004c80 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.793+0000 7fecacd9d700 1 -- 172.21.15.119:0/3760964589 >> 172.21.15.119:0/3760964589 conn(0x7feca8068e20 msgr2=0x7feca80cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:45.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.793+0000 7fecacd9d700 1 -- 172.21.15.119:0/3760964589 shutdown_connections 2023-10-08T02:31:45.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.793+0000 7fecacd9d700 1 -- 172.21.15.119:0/3760964589 wait complete. 2023-10-08T02:31:45.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.794+0000 7fecacd9d700 1 Processor -- start 2023-10-08T02:31:45.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.795+0000 7fecacd9d700 1 -- start start 2023-10-08T02:31:45.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.795+0000 7fecacd9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90004810 0x7feca80a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.795+0000 7fecacd9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca8166490 0x7feca8165350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7fecacd9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca80a7510 0x7feca80a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7fecacd9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca80a7ec0 con 0x7feca8166490 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7fecacd9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca80a6b80 con 0x7feca80a7510 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7fecacd9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca80a5f70 con 0x7fec90004810 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca8166490 0x7feca8165350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca8166490 0x7feca8165350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42752/0 (socket says 172.21.15.119:42752) 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 -- 172.21.15.119:0/2127325525 learned_addr learned my addr 172.21.15.119:0/2127325525 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca77fe700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90004810 0x7feca80a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca7fff700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca80a7510 0x7feca80a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 -- 172.21.15.119:0/2127325525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90004810 msgr2=0x7feca80a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90004810 0x7feca80a5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 -- 172.21.15.119:0/2127325525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca80a7510 msgr2=0x7feca80a7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:45.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca80a7510 0x7feca80a7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 -- 172.21.15.119:0/2127325525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80a6250 con 0x7feca8166490 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.796+0000 7feca6ffd700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca8166490 0x7feca8165350 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c180 tx=0x7fec9c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.797+0000 7fec8f7fe700 1 -- 172.21.15.119:0/2127325525 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c022070 con 0x7feca8166490 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.797+0000 7fec8f7fe700 1 -- 172.21.15.119:0/2127325525 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c00e9e0 con 0x7feca8166490 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.797+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca80a6530 con 0x7feca8166490 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.797+0000 7fec8f7fe700 1 -- 172.21.15.119:0/2127325525 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c00e710 con 0x7feca8166490 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.797+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca8165890 con 0x7feca8166490 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.798+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca804c3c0 con 0x7feca8166490 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.799+0000 7fec8f7fe700 1 -- 172.21.15.119:0/2127325525 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 117) v1 ==== 84916+0+0 (secure 0 0 0) 0x7fec9c01c660 con 0x7feca8166490 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.799+0000 7fec8f7fe700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fec94067130 0x7fec940695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:45.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.799+0000 7fec8f7fe700 1 -- 172.21.15.119:0/2127325525 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fec9c086370 con 0x7feca8166490 2023-10-08T02:31:45.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.800+0000 7feca77fe700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fec94067130 0x7fec940695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:45.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.800+0000 7feca77fe700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fec94067130 0x7fec940695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feca8157970 tx=0x7fec98007f30 comp rx=0 tx=0).ready entity=mgr.10395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:45.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:45.804+0000 7fec8f7fe700 1 -- 172.21.15.119:0/2127325525 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec9c052ff0 con 0x7feca8166490 2023-10-08T02:31:46.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.019+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7feca80afa90 con 0x7fec94067130 2023-10-08T02:31:46.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.077+0000 7fec8f7fe700 1 -- 172.21.15.119:0/2127325525 <== mgr.10395 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7feca80afa90 con 0x7fec94067130 2023-10-08T02:31:46.078 INFO:teuthology.orchestra.run.smithi119.stdout:Schedule removed for path snap_test_dir1 2023-10-08T02:31:46.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fec94067130 msgr2=0x7fec940695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:46.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fec94067130 0x7fec940695e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7feca8157970 tx=0x7fec98007f30 comp rx=0 tx=0).stop 2023-10-08T02:31:46.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca8166490 msgr2=0x7feca8165350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:46.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca8166490 0x7feca8165350 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fec9000c180 tx=0x7fec9c01a000 comp rx=0 tx=0).stop 2023-10-08T02:31:46.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 shutdown_connections 2023-10-08T02:31:46.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fec94067130 0x7fec940695e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fec90004810 0x7feca80a5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7feca80a7510 0x7feca80a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 --2- 172.21.15.119:0/2127325525 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feca8166490 0x7feca8165350 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 >> 172.21.15.119:0/2127325525 conn(0x7feca8068e20 msgr2=0x7feca80695f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:46.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 shutdown_connections 2023-10-08T02:31:46.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.080+0000 7fecacd9d700 1 -- 172.21.15.119:0/2127325525 wait complete. 2023-10-08T02:31:46.098 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-10-08T02:31:46.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:46.098 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-10-08T02:31:46.137 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:46.137 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-10-08T02:31:46.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-10-08T02:31:46.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.437+0000 7ff8cdfff700 1 Processor -- start 2023-10-08T02:31:46.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.437+0000 7ff8cdfff700 1 -- start start 2023-10-08T02:31:46.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8cdfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c8155eb0 0x7ff8c806e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:46.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8cdfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7ff8c80c70b0 0x7ff8c8069770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:46.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8cdfff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff8c8069cb0 0x7ff8c8159a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8cdfff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c806a130 con 0x7ff8c8069cb0 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8cdfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c806eb90 con 0x7ff8c80c70b0 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8cdfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c806ee70 con 0x7ff8c8155eb0 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff8c8069cb0 0x7ff8c8159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c7fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff8c8069cb0 0x7ff8c8159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42758/0 (socket says 172.21.15.119:42758) 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c7fff700 1 -- 172.21.15.119:0/1393205689 learned_addr learned my addr 172.21.15.119:0/1393205689 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c6ffd700 1 --2- 172.21.15.119:0/1393205689 >> v2:172.21.15.184:3300/0 conn(0x7ff8c80c70b0 0x7ff8c8069770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c77fe700 1 --2- 172.21.15.119:0/1393205689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c8155eb0 0x7ff8c806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c7fff700 1 -- 172.21.15.119:0/1393205689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c8155eb0 msgr2=0x7ff8c806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c7fff700 1 --2- 172.21.15.119:0/1393205689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c8155eb0 0x7ff8c806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c7fff700 1 -- 172.21.15.119:0/1393205689 >> v2:172.21.15.184:3300/0 conn(0x7ff8c80c70b0 msgr2=0x7ff8c8069770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c7fff700 1 --2- 172.21.15.119:0/1393205689 >> v2:172.21.15.184:3300/0 conn(0x7ff8c80c70b0 0x7ff8c8069770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c7fff700 1 -- 172.21.15.119:0/1393205689 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c806f150 con 0x7ff8c8069cb0 2023-10-08T02:31:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.439+0000 7ff8c77fe700 1 --2- 172.21.15.119:0/1393205689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c8155eb0 0x7ff8c806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.440+0000 7ff8c7fff700 1 --2- 172.21.15.119:0/1393205689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c8159a60 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8063d40 tx=0x7ff8c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1772f5f59b18132b server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.440+0000 7ff8c67fc700 1 -- 172.21.15.119:0/1393205689 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c000e070 con 0x7ff8c8069cb0 2023-10-08T02:31:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.440+0000 7ff8c67fc700 1 -- 172.21.15.119:0/1393205689 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c000ba50 con 0x7ff8c8069cb0 2023-10-08T02:31:46.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.440+0000 7ff8c67fc700 1 -- 172.21.15.119:0/1393205689 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c00092e0 con 0x7ff8c8069cb0 2023-10-08T02:31:46.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.440+0000 7ff8cdfff700 1 -- 172.21.15.119:0/1393205689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 msgr2=0x7ff8c8159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:46.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.440+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/1393205689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c8159a60 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8063d40 tx=0x7ff8c0009840 comp rx=0 tx=0).stop 2023-10-08T02:31:46.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.441+0000 7ff8cdfff700 1 -- 172.21.15.119:0/1393205689 shutdown_connections 2023-10-08T02:31:46.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.441+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/1393205689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c8159a60 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.441+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/1393205689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c8155eb0 0x7ff8c806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.441+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/1393205689 >> v2:172.21.15.184:3300/0 conn(0x7ff8c80c70b0 0x7ff8c8069770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.441+0000 7ff8cdfff700 1 -- 172.21.15.119:0/1393205689 >> 172.21.15.119:0/1393205689 conn(0x7ff8c8068e10 msgr2=0x7ff8c80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.441+0000 7ff8cdfff700 1 -- 172.21.15.119:0/1393205689 shutdown_connections 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.441+0000 7ff8cdfff700 1 -- 172.21.15.119:0/1393205689 wait complete. 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.442+0000 7ff8cdfff700 1 Processor -- start 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.442+0000 7ff8cdfff700 1 -- start start 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8cdfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c80c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8cdfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c80c70b0 0x7ff8c813f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8cdfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c8155eb0 0x7ff8c813f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8cdfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c813fb50 con 0x7ff8c8069cb0 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8cdfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8168530 con 0x7ff8c8155eb0 2023-10-08T02:31:46.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8cdfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c81687f0 con 0x7ff8c80c70b0 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8c77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c80c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8c77fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c80c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42770/0 (socket says 172.21.15.119:42770) 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8c77fe700 1 -- 172.21.15.119:0/4171108500 learned_addr learned my addr 172.21.15.119:0/4171108500 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8c6ffd700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c80c70b0 0x7ff8c813f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.443+0000 7ff8c7fff700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c8155eb0 0x7ff8c813f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8c77fe700 1 -- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c80c70b0 msgr2=0x7ff8c813f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8c77fe700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c80c70b0 0x7ff8c813f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8c77fe700 1 -- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c8155eb0 msgr2=0x7ff8c813f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8c77fe700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c8155eb0 0x7ff8c813f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8c77fe700 1 -- 172.21.15.119:0/4171108500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c0009ae0 con 0x7ff8c8069cb0 2023-10-08T02:31:46.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8c7fff700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c8155eb0 0x7ff8c813f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8c77fe700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c80c26a0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800c850 tx=0x7ff8b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8af7fe700 1 -- 172.21.15.119:0/4171108500 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8011600 con 0x7ff8c8069cb0 2023-10-08T02:31:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8af7fe700 1 -- 172.21.15.119:0/4171108500 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b8011760 con 0x7ff8c8069cb0 2023-10-08T02:31:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c8168ad0 con 0x7ff8c8069cb0 2023-10-08T02:31:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8af7fe700 1 -- 172.21.15.119:0/4171108500 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8011b00 con 0x7ff8c8069cb0 2023-10-08T02:31:46.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.444+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c8169090 con 0x7ff8c8069cb0 2023-10-08T02:31:46.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.445+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c804c3c0 con 0x7ff8c8069cb0 2023-10-08T02:31:46.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.446+0000 7ff8af7fe700 1 -- 172.21.15.119:0/4171108500 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 117) v1 ==== 84916+0+0 (secure 0 0 0) 0x7ff8b80073b0 con 0x7ff8c8069cb0 2023-10-08T02:31:46.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.447+0000 7ff8af7fe700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff8b4067130 0x7ff8b40695e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:46.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.447+0000 7ff8af7fe700 1 -- 172.21.15.119:0/4171108500 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7ff8b8084cf0 con 0x7ff8c8069cb0 2023-10-08T02:31:46.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.447+0000 7ff8c6ffd700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff8b4067130 0x7ff8b40695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:46.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.448+0000 7ff8c6ffd700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff8b4067130 0x7ff8b40695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8c815c2b0 tx=0x7ff8bc008040 comp rx=0 tx=0).ready entity=mgr.10395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:46.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.451+0000 7ff8af7fe700 1 -- 172.21.15.119:0/4171108500 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff8b80552b0 con 0x7ff8c8069cb0 2023-10-08T02:31:46.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:46.696+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff8c80c7790 con 0x7ff8c8069cb0 2023-10-08T02:31:47.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.607+0000 7ff8af7fe700 1 -- 172.21.15.119:0/4171108500 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7ff8b8084020 con 0x7ff8c8069cb0 2023-10-08T02:31:47.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.608+0000 7ff8af7fe700 1 -- 172.21.15.119:0/4171108500 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v118) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff8b80552b0 con 0x7ff8c8069cb0 2023-10-08T02:31:47.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff8b4067130 msgr2=0x7ff8b40695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:47.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff8b4067130 0x7ff8b40695e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8c815c2b0 tx=0x7ff8bc008040 comp rx=0 tx=0).stop 2023-10-08T02:31:47.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 msgr2=0x7ff8c80c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:47.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c80c26a0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800c850 tx=0x7ff8b8008040 comp rx=0 tx=0).stop 2023-10-08T02:31:47.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 shutdown_connections 2023-10-08T02:31:47.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7ff8b4067130 0x7ff8b40695e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:47.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7ff8c8155eb0 0x7ff8c813f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:47.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff8c8069cb0 0x7ff8c80c26a0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:47.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 --2- 172.21.15.119:0/4171108500 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7ff8c80c70b0 0x7ff8c813f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:47.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 >> 172.21.15.119:0/4171108500 conn(0x7ff8c8068e10 msgr2=0x7ff8c80c8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:47.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 shutdown_connections 2023-10-08T02:31:47.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:47.611+0000 7ff8cdfff700 1 -- 172.21.15.119:0/4171108500 wait complete. 2023-10-08T02:31:47.632 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-08T02:31:47.632 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:31:47.716 DEBUG:teuthology.orchestra.run.smithi184:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-08T02:31:47.794 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-08T02:31:47.795 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-10-08T02:31:47.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-08T02:31:47.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-10-08T02:31:47.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:31:47.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:47.876 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-08T02:31:47.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:48.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.140+0000 7f097db9d700 1 Processor -- start 2023-10-08T02:31:48.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.140+0000 7f097db9d700 1 -- start start 2023-10-08T02:31:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.141+0000 7f097db9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0978157270 0x7f0978068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.141+0000 7f097db9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f097815d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.141+0000 7f097db9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f097815a980 con 0x7f09780688d0 2023-10-08T02:31:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.141+0000 7f097db9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0978068d50 con 0x7f0978157270 2023-10-08T02:31:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.141+0000 7f097db9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097815a050 con 0x7f09780b0bd0 2023-10-08T02:31:48.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097cb9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0978157270 0x7f0978068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097cb9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0978157270 0x7f0978068390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42792/0 (socket says 172.21.15.119:42792) 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097cb9b700 1 -- 172.21.15.119:0/2954394231 learned_addr learned my addr 172.21.15.119:0/2954394231 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097d39c700 1 --2- 172.21.15.119:0/2954394231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f097815d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097cb9b700 1 -- 172.21.15.119:0/2954394231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 msgr2=0x7f097815d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097cb9b700 1 --2- 172.21.15.119:0/2954394231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f097815d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097cb9b700 1 -- 172.21.15.119:0/2954394231 >> v1:172.21.15.184:6789/0 conn(0x7f09780688d0 legacy=0x7f09780b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097cb9b700 1 -- 172.21.15.119:0/2954394231 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f097815a330 con 0x7f0978157270 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097d39c700 1 --2- 172.21.15.119:0/2954394231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f097815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f09777fe700 1 -- 172.21.15.119:0/2954394231 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 191992929 0 0) 0x7f097815a980 con 0x7f09780688d0 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.142+0000 7f097cb9b700 1 --2- 172.21.15.119:0/2954394231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f0978068390 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f096c0099b0 tx=0x7f096c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa7e0fb4eab9393 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:48.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f09777fe700 1 -- 172.21.15.119:0/2954394231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c00b860 con 0x7f0978157270 2023-10-08T02:31:48.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f09777fe700 1 -- 172.21.15.119:0/2954394231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096c00b9c0 con 0x7f0978157270 2023-10-08T02:31:48.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f09777fe700 1 -- 172.21.15.119:0/2954394231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096c00bd60 con 0x7f0978157270 2023-10-08T02:31:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f097db9d700 1 -- 172.21.15.119:0/2954394231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 msgr2=0x7f0978068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f097db9d700 1 --2- 172.21.15.119:0/2954394231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f0978068390 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f096c0099b0 tx=0x7f096c009250 comp rx=0 tx=0).stop 2023-10-08T02:31:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f097db9d700 1 -- 172.21.15.119:0/2954394231 shutdown_connections 2023-10-08T02:31:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f097db9d700 1 --2- 172.21.15.119:0/2954394231 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f097815d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f097db9d700 1 --2- 172.21.15.119:0/2954394231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f0978068390 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f097db9d700 1 -- 172.21.15.119:0/2954394231 >> 172.21.15.119:0/2954394231 conn(0x7f09780cd310 msgr2=0x7f09780cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:48.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f097db9d700 1 -- 172.21.15.119:0/2954394231 shutdown_connections 2023-10-08T02:31:48.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.143+0000 7f097db9d700 1 -- 172.21.15.119:0/2954394231 wait complete. 2023-10-08T02:31:48.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.145+0000 7f097db9d700 1 Processor -- start 2023-10-08T02:31:48.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.145+0000 7f097db9d700 1 -- start start 2023-10-08T02:31:48.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097db9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09780688d0 0x7f09780aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097db9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f09781553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097db9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f097815aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097db9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097815ba00 con 0x7f0978157270 2023-10-08T02:31:48.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097db9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0978154f60 con 0x7f09780688d0 2023-10-08T02:31:48.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097db9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0978154630 con 0x7f09780b0bd0 2023-10-08T02:31:48.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097d39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f097815aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097d39c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f097815aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42804/0 (socket says 172.21.15.119:42804) 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097d39c700 1 -- 172.21.15.119:0/2684030723 learned_addr learned my addr 172.21.15.119:0/2684030723 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f097cb9b700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09780688d0 0x7f09780aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.146+0000 7f0977fff700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f09781553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f097d39c700 1 -- 172.21.15.119:0/2684030723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 msgr2=0x7f09781553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f097d39c700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f09781553f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f097d39c700 1 -- 172.21.15.119:0/2684030723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09780688d0 msgr2=0x7f09780aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f097d39c700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09780688d0 0x7f09780aef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f097d39c700 1 -- 172.21.15.119:0/2684030723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c0094f0 con 0x7f0978157270 2023-10-08T02:31:48.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f097d39c700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f097815aeb0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f097000d1b0 tx=0x7f0970009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:48.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f0974ff9700 1 -- 172.21.15.119:0/2684030723 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097000dcf0 con 0x7f0978157270 2023-10-08T02:31:48.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f0974ff9700 1 -- 172.21.15.119:0/2684030723 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097000f040 con 0x7f0978157270 2023-10-08T02:31:48.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0978154910 con 0x7f0978157270 2023-10-08T02:31:48.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f0974ff9700 1 -- 172.21.15.119:0/2684030723 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970015690 con 0x7f0978157270 2023-10-08T02:31:48.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.147+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09780af4c0 con 0x7f0978157270 2023-10-08T02:31:48.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.148+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f097804c3c0 con 0x7f0978157270 2023-10-08T02:31:48.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.149+0000 7f0974ff9700 1 -- 172.21.15.119:0/2684030723 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f097001e3b0 con 0x7f0978157270 2023-10-08T02:31:48.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.150+0000 7f0974ff9700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f095c0670e0 0x7f095c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.150+0000 7f0974ff9700 1 -- 172.21.15.119:0/2684030723 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f09700850e0 con 0x7f0978157270 2023-10-08T02:31:48.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.150+0000 7f097cb9b700 1 -- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f095c0670e0 msgr2=0x7f095c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:48.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.150+0000 7f097cb9b700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f095c0670e0 0x7f095c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:48.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.158+0000 7f0974ff9700 1 -- 172.21.15.119:0/2684030723 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0970051cb0 con 0x7f0978157270 2023-10-08T02:31:48.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.351+0000 7f097cb9b700 1 -- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f095c0670e0 msgr2=0x7f095c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.351+0000 7f097cb9b700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f095c0670e0 0x7f095c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:48.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.398+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09781550a0 con 0x7f0978157270 2023-10-08T02:31:48.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.399+0000 7f0974ff9700 1 -- 172.21.15.119:0/2684030723 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f09700552d0 con 0x7f0978157270 2023-10-08T02:31:48.399 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:48.399 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1044912493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1044912493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1044912493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10197,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1564057059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1564057059},{"type":"v1","addr":"172.21.15.119:6833","nonce":1564057059}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2545901849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2545901849},{"type":"v1","addr":"172.21.15.184:6834","nonce":2545901849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:31:27.855883+0000","modified":"2023-10-08T02:31:31.420955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10215":{"gid":10215,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1741909431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1741909431},{"type":"v1","addr":"172.21.15.119:6835","nonce":1741909431}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T02:31:48.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f095c0670e0 msgr2=0x7f095c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:48.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f095c0670e0 0x7f095c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 msgr2=0x7f097815aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f097815aeb0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f097000d1b0 tx=0x7f0970009a50 comp rx=0 tx=0).stop 2023-10-08T02:31:48.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 shutdown_connections 2023-10-08T02:31:48.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f095c0670e0 0x7f095c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0978157270 0x7f097815aeb0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f09780688d0 0x7f09780aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 --2- 172.21.15.119:0/2684030723 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f09780b0bd0 0x7f09781553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 >> 172.21.15.119:0/2684030723 conn(0x7f09780cd310 msgr2=0x7f09780abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:48.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 shutdown_connections 2023-10-08T02:31:48.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.401+0000 7f097db9d700 1 -- 172.21.15.119:0/2684030723 wait complete. 2023-10-08T02:31:48.403 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-10-08T02:31:48.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:48.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.611+0000 7fd82299d700 1 Processor -- start 2023-10-08T02:31:48.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.612+0000 7fd82299d700 1 -- start start 2023-10-08T02:31:48.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82299d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82299d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd81c1597f0 0x7fd81c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82299d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81c0b11d0 con 0x7fd81c15d230 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82299d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c0b14b0 con 0x7fd81c1597f0 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82299d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0b1790 con 0x7fd81c159420 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82219c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd81c15d230 0x7fd81c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41612/0 (socket says 172.21.15.119:41612) 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82219c700 1 -- 172.21.15.119:0/1947030352 learned_addr learned my addr 172.21.15.119:0/1947030352 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82199b700 1 --2- 172.21.15.119:0/1947030352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.613+0000 7fd82119a700 1 --2- 172.21.15.119:0/1947030352 >> v2:172.21.15.184:3300/0 conn(0x7fd81c1597f0 0x7fd81c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd820999700 1 -- 172.21.15.119:0/1947030352 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2952580262 0 0) 0x7fd81c0b11d0 con 0x7fd81c15d230 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd820999700 1 -- 172.21.15.119:0/1947030352 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd808003570 con 0x7fd81c15d230 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd82199b700 1 -- 172.21.15.119:0/1947030352 >> v2:172.21.15.184:3300/0 conn(0x7fd81c1597f0 msgr2=0x7fd81c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:48.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd82199b700 1 --2- 172.21.15.119:0/1947030352 >> v2:172.21.15.184:3300/0 conn(0x7fd81c1597f0 0x7fd81c0b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd82199b700 1 -- 172.21.15.119:0/1947030352 >> v1:172.21.15.119:6789/0 conn(0x7fd81c15d230 legacy=0x7fd81c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd82199b700 1 -- 172.21.15.119:0/1947030352 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0b1a70 con 0x7fd81c159420 2023-10-08T02:31:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd82199b700 1 --2- 172.21.15.119:0/1947030352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0b5f40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd8180098e0 tx=0x7fd8180091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3319d1a2fe67db server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:48.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd820999700 1 -- 172.21.15.119:0/1947030352 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818011630 con 0x7fd81c159420 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd820999700 1 -- 172.21.15.119:0/1947030352 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd818011790 con 0x7fd81c159420 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.614+0000 7fd820999700 1 -- 172.21.15.119:0/1947030352 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd818011b30 con 0x7fd81c159420 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.615+0000 7fd82299d700 1 -- 172.21.15.119:0/1947030352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 msgr2=0x7fd81c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.615+0000 7fd82299d700 1 --2- 172.21.15.119:0/1947030352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0b5f40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd8180098e0 tx=0x7fd8180091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.615+0000 7fd82299d700 1 -- 172.21.15.119:0/1947030352 shutdown_connections 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.615+0000 7fd82299d700 1 --2- 172.21.15.119:0/1947030352 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0b5f40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.615+0000 7fd82299d700 1 --2- 172.21.15.119:0/1947030352 >> v2:172.21.15.184:3300/0 conn(0x7fd81c1597f0 0x7fd81c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.615+0000 7fd82299d700 1 -- 172.21.15.119:0/1947030352 >> 172.21.15.119:0/1947030352 conn(0x7fd81c0d0660 msgr2=0x7fd81c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:48.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.615+0000 7fd82299d700 1 -- 172.21.15.119:0/1947030352 shutdown_connections 2023-10-08T02:31:48.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.615+0000 7fd82299d700 1 -- 172.21.15.119:0/1947030352 wait complete. 2023-10-08T02:31:48.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.617+0000 7fd82299d700 1 Processor -- start 2023-10-08T02:31:48.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.617+0000 7fd82299d700 1 -- start start 2023-10-08T02:31:48.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82299d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82299d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd81c1597f0 0x7fd81c14bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82299d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c15d230 0x7fd81c0cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82299d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0c7710 con 0x7fd81c15d230 2023-10-08T02:31:48.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82299d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c16d070 con 0x7fd81c1597f0 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82299d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c16d350 con 0x7fd81c159420 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82219c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c15d230 0x7fd81c0cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82219c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c15d230 0x7fd81c0cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42818/0 (socket says 172.21.15.119:42818) 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82219c700 1 -- 172.21.15.119:0/998603595 learned_addr learned my addr 172.21.15.119:0/998603595 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82199b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82119a700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd81c1597f0 0x7fd81c14bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82219c700 1 -- 172.21.15.119:0/998603595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 msgr2=0x7fd81c0c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82219c700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82219c700 1 -- 172.21.15.119:0/998603595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd81c1597f0 msgr2=0x7fd81c14bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82219c700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd81c1597f0 0x7fd81c14bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.618+0000 7fd82219c700 1 -- 172.21.15.119:0/998603595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd818009420 con 0x7fd81c15d230 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.619+0000 7fd82219c700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c15d230 0x7fd81c0cbbe0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fd814001460 tx=0x7fd814001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.619+0000 7fd811ffb700 1 -- 172.21.15.119:0/998603595 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400a870 con 0x7fd81c15d230 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.619+0000 7fd811ffb700 1 -- 172.21.15.119:0/998603595 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81400a9d0 con 0x7fd81c15d230 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.619+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c16d630 con 0x7fd81c15d230 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.619+0000 7fd811ffb700 1 -- 172.21.15.119:0/998603595 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81400ad50 con 0x7fd81c15d230 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.619+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c16dbf0 con 0x7fd81c15d230 2023-10-08T02:31:48.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.620+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c169940 con 0x7fd81c15d230 2023-10-08T02:31:48.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.622+0000 7fd811ffb700 1 -- 172.21.15.119:0/998603595 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fd814007300 con 0x7fd81c15d230 2023-10-08T02:31:48.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.623+0000 7fd811ffb700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd8000670e0 0x7fd800069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:48.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.623+0000 7fd811ffb700 1 -- 172.21.15.119:0/998603595 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fd814084e90 con 0x7fd81c15d230 2023-10-08T02:31:48.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.623+0000 7fd82199b700 1 -- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd8000670e0 msgr2=0x7fd800069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:48.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.623+0000 7fd82199b700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd8000670e0 0x7fd800069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:48.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.627+0000 7fd811ffb700 1 -- 172.21.15.119:0/998603595 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd814051b60 con 0x7fd81c15d230 2023-10-08T02:31:48.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.824+0000 7fd82199b700 1 -- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd8000670e0 msgr2=0x7fd800069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:48.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.824+0000 7fd82199b700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd8000670e0 0x7fd800069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:48.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.826+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd81c04c3c0 con 0x7fd81c15d230 2023-10-08T02:31:48.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.827+0000 7fd811ffb700 1 -- 172.21.15.119:0/998603595 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+16589 (secure 0 0 0) 0x7fd814055180 con 0x7fd81c15d230 2023-10-08T02:31:48.828 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:48.828 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":207,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:43.569130+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:31:25.536431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T02:31:26.843560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/842982126":"2023-10-09T02:31:43.569085+0000","172.21.15.119:6837/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:6836/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/1887277487":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/4191951118":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/773312007":"2023-10-09T02:31:43.569085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:48.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd8000670e0 msgr2=0x7fd800069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:48.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd8000670e0 0x7fd800069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c15d230 msgr2=0x7fd81c0cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:48.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c15d230 0x7fd81c0cbbe0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fd814001460 tx=0x7fd814001a40 comp rx=0 tx=0).stop 2023-10-08T02:31:48.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 shutdown_connections 2023-10-08T02:31:48.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd8000670e0 0x7fd800069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd81c159420 0x7fd81c0c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd81c15d230 0x7fd81c0cbbe0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 --2- 172.21.15.119:0/998603595 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd81c1597f0 0x7fd81c14bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:48.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 >> 172.21.15.119:0/998603595 conn(0x7fd81c0d0660 msgr2=0x7fd81c0b5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:48.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 shutdown_connections 2023-10-08T02:31:48.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:48.830+0000 7fd82299d700 1 -- 172.21.15.119:0/998603595 wait complete. 2023-10-08T02:31:48.849 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-08T00:55:35.909253+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-08T02:31:25.536431+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '205', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-10-08T02:31:26.843560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '206', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-08T02:31:48.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-08T02:31:48.890 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:48.889+0000 7f4549a91180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:31:48.953 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:48.952+0000 7f8a1666a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-08T02:31:49.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.040+0000 7f0809b9d700 1 Processor -- start 2023-10-08T02:31:49.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.040+0000 7f0809b9d700 1 -- start start 2023-10-08T02:31:49.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.041+0000 7f0809b9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f080415a3b0 0x7f080415a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.041+0000 7f0809b9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f08040aef00 0x7f080415d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.041+0000 7f0809b9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08040aea70 con 0x7f080406e470 2023-10-08T02:31:49.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.041+0000 7f0809b9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08040ae480 con 0x7f08040aef00 2023-10-08T02:31:49.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.041+0000 7f0809b9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f080415acc0 con 0x7f080415a3b0 2023-10-08T02:31:49.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f0803fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f08040aef00 0x7f080415d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f0808b9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f080415a3b0 0x7f080415a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f080939c700 1 --1- >> v1:172.21.15.184:6789/0 conn(0x7f080406e470 0x7f08040c7280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:59840/0 (socket says 172.21.15.119:59840) 2023-10-08T02:31:49.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f080939c700 1 -- 172.21.15.119:0/2362504981 learned_addr learned my addr 172.21.15.119:0/2362504981 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f08037fe700 1 -- 172.21.15.119:0/2362504981 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2490857438 0 0) 0x7f08040aea70 con 0x7f080406e470 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f08037fe700 1 -- 172.21.15.119:0/2362504981 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07f4003570 con 0x7f080406e470 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f0803fff700 1 -- 172.21.15.119:0/2362504981 >> v2:172.21.15.184:3300/0 conn(0x7f080415a3b0 msgr2=0x7f080415a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f0803fff700 1 --2- 172.21.15.119:0/2362504981 >> v2:172.21.15.184:3300/0 conn(0x7f080415a3b0 0x7f080415a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f0803fff700 1 -- 172.21.15.119:0/2362504981 >> v1:172.21.15.184:6789/0 conn(0x7f080406e470 legacy=0x7f08040c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.042+0000 7f0803fff700 1 -- 172.21.15.119:0/2362504981 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080415dd30 con 0x7f08040aef00 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.043+0000 7f0803fff700 1 --2- 172.21.15.119:0/2362504981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08040aef00 0x7f080415d3c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f0804063350 tx=0x7f07f00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ec49d5e2162eec server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.043+0000 7f08037fe700 1 -- 172.21.15.119:0/2362504981 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0011630 con 0x7f08040aef00 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.043+0000 7f08037fe700 1 -- 172.21.15.119:0/2362504981 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f00117b0 con 0x7f08040aef00 2023-10-08T02:31:49.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.043+0000 7f08037fe700 1 -- 172.21.15.119:0/2362504981 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f0011b70 con 0x7f08040aef00 2023-10-08T02:31:49.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.043+0000 7f0809b9d700 1 -- 172.21.15.119:0/2362504981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08040aef00 msgr2=0x7f080415d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.043+0000 7f0809b9d700 1 --2- 172.21.15.119:0/2362504981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08040aef00 0x7f080415d3c0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f0804063350 tx=0x7f07f00091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.044+0000 7f0809b9d700 1 -- 172.21.15.119:0/2362504981 shutdown_connections 2023-10-08T02:31:49.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.044+0000 7f0809b9d700 1 --2- 172.21.15.119:0/2362504981 >> v2:172.21.15.184:3300/0 conn(0x7f080415a3b0 0x7f080415a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.044+0000 7f0809b9d700 1 --2- 172.21.15.119:0/2362504981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08040aef00 0x7f080415d3c0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.044+0000 7f0809b9d700 1 -- 172.21.15.119:0/2362504981 >> 172.21.15.119:0/2362504981 conn(0x7f08040c9f70 msgr2=0x7f080406dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:49.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.044+0000 7f0809b9d700 1 -- 172.21.15.119:0/2362504981 shutdown_connections 2023-10-08T02:31:49.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.044+0000 7f0809b9d700 1 -- 172.21.15.119:0/2362504981 wait complete. 2023-10-08T02:31:49.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.045+0000 7f0809b9d700 1 Processor -- start 2023-10-08T02:31:49.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.045+0000 7f0809b9d700 1 -- start start 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0809b9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080406e470 0x7f08040b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0809b9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08040aef00 0x7f08040b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0809b9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f080415a3b0 0x7f08040b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0809b9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040b27c0 con 0x7f080406e470 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0809b9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08040c5980 con 0x7f08040aef00 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0809b9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08040c4d70 con 0x7f080415a3b0 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0808b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080406e470 0x7f08040b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0808b9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080406e470 0x7f08040b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42844/0 (socket says 172.21.15.119:42844) 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0808b9b700 1 -- 172.21.15.119:0/191254663 learned_addr learned my addr 172.21.15.119:0/191254663 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f0803fff700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08040aef00 0x7f08040b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.046+0000 7f080939c700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f080415a3b0 0x7f08040b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0808b9b700 1 -- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f080415a3b0 msgr2=0x7f08040b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0808b9b700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f080415a3b0 0x7f08040b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0808b9b700 1 -- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08040aef00 msgr2=0x7f08040b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0808b9b700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08040aef00 0x7f08040b2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0808b9b700 1 -- 172.21.15.119:0/191254663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0009420 con 0x7f080406e470 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f080939c700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f080415a3b0 0x7f08040b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0808b9b700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080406e470 0x7f08040b1d40 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f07f800e930 tx=0x7f07f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0800ff9700 1 -- 172.21.15.119:0/191254663 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8013680 con 0x7f080406e470 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0800ff9700 1 -- 172.21.15.119:0/191254663 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07f80137e0 con 0x7f080406e470 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0800ff9700 1 -- 172.21.15.119:0/191254663 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07f8013b80 con 0x7f080406e470 2023-10-08T02:31:49.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08040c5050 con 0x7f080406e470 2023-10-08T02:31:49.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.047+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08040c5610 con 0x7f080406e470 2023-10-08T02:31:49.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.049+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08040c5c60 con 0x7f080406e470 2023-10-08T02:31:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.049+0000 7f0800ff9700 1 -- 172.21.15.119:0/191254663 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f07f800c460 con 0x7f080406e470 2023-10-08T02:31:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.050+0000 7f0800ff9700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f07e80670e0 0x7f07e8069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.050+0000 7f0800ff9700 1 -- 172.21.15.119:0/191254663 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f07f8084d70 con 0x7f080406e470 2023-10-08T02:31:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.050+0000 7f0803fff700 1 -- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f07e80670e0 msgr2=0x7f07e8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:49.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.050+0000 7f0803fff700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f07e80670e0 0x7f07e8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:49.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.054+0000 7f0800ff9700 1 -- 172.21.15.119:0/191254663 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f07f80519b0 con 0x7f080406e470 2023-10-08T02:31:49.055 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:49.054+0000 7f4549a91180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:49.126 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:49.126+0000 7f8a1666a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:49.141 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:49.140+0000 7f4549a91180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:31:49.215 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:49.214+0000 7f8a1666a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-08T02:31:49.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.251+0000 7f0803fff700 1 -- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f07e80670e0 msgr2=0x7f07e8069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:49.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.251+0000 7f0803fff700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f07e80670e0 0x7f07e8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:49.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.255+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0804063590 con 0x7f080406e470 2023-10-08T02:31:49.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.255+0000 7f0800ff9700 1 -- 172.21.15.119:0/191254663 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v207) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07f8054fd0 con 0x7f080406e470 2023-10-08T02:31:49.256 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-10-08T02:31:49.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.258+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f07e80670e0 msgr2=0x7f07e8069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.258+0000 7f0809b9d700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f07e80670e0 0x7f07e8069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.258+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080406e470 msgr2=0x7f08040b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080406e470 0x7f08040b1d40 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f07f800e930 tx=0x7f07f80091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 shutdown_connections 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f07e80670e0 0x7f07e8069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f080415a3b0 0x7f08040b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f080406e470 0x7f08040b1d40 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 --2- 172.21.15.119:0/191254663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f08040aef00 0x7f08040b2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 >> 172.21.15.119:0/191254663 conn(0x7f08040c9f70 msgr2=0x7f0804152690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:49.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 shutdown_connections 2023-10-08T02:31:49.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.259+0000 7f0809b9d700 1 -- 172.21.15.119:0/191254663 wait complete. 2023-10-08T02:31:49.284 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-08T02:31:49.366 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:49.365+0000 7f4549a91180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:49.458 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:49.457+0000 7f8a1666a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:49.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.474+0000 7fd2c4f9d700 1 Processor -- start 2023-10-08T02:31:49.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.474+0000 7fd2c4f9d700 1 -- start start 2023-10-08T02:31:49.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.475+0000 7fd2c4f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fd2c0159a70 0x7fd2c00c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.475+0000 7fd2c4f9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2c006bfa0 0x7fd2c00c8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.475+0000 7fd2c4f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 0x7fd2c0068810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.475+0000 7fd2c4f9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c0068d50 con 0x7fd2c006bfa0 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.475+0000 7fd2c4f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c00ca920 con 0x7fd2c0159a70 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.475+0000 7fd2c4f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c006af00 con 0x7fd2c0068390 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2c006bfa0 0x7fd2c00c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2c006bfa0 0x7fd2c00c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42854/0 (socket says 172.21.15.119:42854) 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 -- 172.21.15.119:0/1932983743 learned_addr learned my addr 172.21.15.119:0/1932983743 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2bf7fe700 1 --2- 172.21.15.119:0/1932983743 >> v2:172.21.15.184:3300/0 conn(0x7fd2c0159a70 0x7fd2c00c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2bffff700 1 --2- 172.21.15.119:0/1932983743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 0x7fd2c0068810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 -- 172.21.15.119:0/1932983743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 msgr2=0x7fd2c0068810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 --2- 172.21.15.119:0/1932983743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 0x7fd2c0068810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 -- 172.21.15.119:0/1932983743 >> v2:172.21.15.184:3300/0 conn(0x7fd2c0159a70 msgr2=0x7fd2c00c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:49.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 --2- 172.21.15.119:0/1932983743 >> v2:172.21.15.184:3300/0 conn(0x7fd2c0159a70 0x7fd2c00c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 -- 172.21.15.119:0/1932983743 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c006b1e0 con 0x7fd2c006bfa0 2023-10-08T02:31:49.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2bf7fe700 1 --2- 172.21.15.119:0/1932983743 >> v2:172.21.15.184:3300/0 conn(0x7fd2c0159a70 0x7fd2c00c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:49.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.476+0000 7fd2beffd700 1 --2- 172.21.15.119:0/1932983743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c00c8830 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00640b0 tx=0x7fd2b4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbee013c9d726730 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:49.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2be7fc700 1 -- 172.21.15.119:0/1932983743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4005c10 con 0x7fd2c006bfa0 2023-10-08T02:31:49.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2be7fc700 1 -- 172.21.15.119:0/1932983743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b4005d70 con 0x7fd2c006bfa0 2023-10-08T02:31:49.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2be7fc700 1 -- 172.21.15.119:0/1932983743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4006260 con 0x7fd2c006bfa0 2023-10-08T02:31:49.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1932983743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 msgr2=0x7fd2c00c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1932983743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c00c8830 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00640b0 tx=0x7fd2b4009220 comp rx=0 tx=0).stop 2023-10-08T02:31:49.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1932983743 shutdown_connections 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1932983743 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 0x7fd2c0068810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1932983743 >> v2:172.21.15.184:3300/0 conn(0x7fd2c0159a70 0x7fd2c00c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1932983743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c00c8830 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1932983743 >> 172.21.15.119:0/1932983743 conn(0x7fd2c00cddc0 msgr2=0x7fd2c00c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.477+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1932983743 shutdown_connections 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.478+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1932983743 wait complete. 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.479+0000 7fd2c4f9d700 1 Processor -- start 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.479+0000 7fd2c4f9d700 1 -- start start 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2c4f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 0x7fd2c01554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2c4f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c014ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2c4f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c0159a70 0x7fd2c01570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2c4f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00c4fd0 con 0x7fd2c006bfa0 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2c4f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00c5160 con 0x7fd2c0159a70 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2c4f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c00b1e00 con 0x7fd2c0068390 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2beffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c014ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2beffd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c014ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42864/0 (socket says 172.21.15.119:42864) 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2beffd700 1 -- 172.21.15.119:0/1566022243 learned_addr learned my addr 172.21.15.119:0/1566022243 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2bf7fe700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 0x7fd2c01554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.480+0000 7fd2bffff700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c0159a70 0x7fd2c01570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2beffd700 1 -- 172.21.15.119:0/1566022243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 msgr2=0x7fd2c01554c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2beffd700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 0x7fd2c01554c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2beffd700 1 -- 172.21.15.119:0/1566022243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c0159a70 msgr2=0x7fd2c01570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2beffd700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c0159a70 0x7fd2c01570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2beffd700 1 -- 172.21.15.119:0/1566022243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b0009420 con 0x7fd2c006bfa0 2023-10-08T02:31:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2beffd700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c014ffe0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009f50 tx=0x7fd2b4016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:49.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2a77fe700 1 -- 172.21.15.119:0/1566022243 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4011850 con 0x7fd2c006bfa0 2023-10-08T02:31:49.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2a77fe700 1 -- 172.21.15.119:0/1566022243 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b40119b0 con 0x7fd2c006bfa0 2023-10-08T02:31:49.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b4009480 con 0x7fd2c006bfa0 2023-10-08T02:31:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2a77fe700 1 -- 172.21.15.119:0/1566022243 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4011d70 con 0x7fd2c006bfa0 2023-10-08T02:31:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.481+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c00b26a0 con 0x7fd2c006bfa0 2023-10-08T02:31:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.483+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c00b1fc0 con 0x7fd2c006bfa0 2023-10-08T02:31:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.483+0000 7fd2a77fe700 1 -- 172.21.15.119:0/1566022243 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7fd2b401b640 con 0x7fd2c006bfa0 2023-10-08T02:31:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.485+0000 7fd2a77fe700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2ac0670e0 0x7fd2ac069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.485+0000 7fd2bf7fe700 1 -- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2ac0670e0 msgr2=0x7fd2ac069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.485+0000 7fd2a77fe700 1 -- 172.21.15.119:0/1566022243 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fd2b4086340 con 0x7fd2c006bfa0 2023-10-08T02:31:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.485+0000 7fd2bf7fe700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2ac0670e0 0x7fd2ac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:49.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.489+0000 7fd2a77fe700 1 -- 172.21.15.119:0/1566022243 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2b4053020 con 0x7fd2c006bfa0 2023-10-08T02:31:49.545 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:49.544+0000 7f4549a91180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:31:49.639 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:49.638+0000 7f4549a91180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:31:49.645 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:49.644+0000 7f8a1666a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-08T02:31:49.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.686+0000 7fd2bf7fe700 1 -- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2ac0670e0 msgr2=0x7fd2ac069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:49.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.686+0000 7fd2bf7fe700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2ac0670e0 0x7fd2ac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:49.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.690+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd2c0063d10 con 0x7fd2c006bfa0 2023-10-08T02:31:49.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.690+0000 7fd2a77fe700 1 -- 172.21.15.119:0/1566022243 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v207) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd2b4056640 con 0x7fd2c006bfa0 2023-10-08T02:31:49.691 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-10-08T02:31:49.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2ac0670e0 msgr2=0x7fd2ac069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2ac0670e0 0x7fd2ac069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 msgr2=0x7fd2c014ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c014ffe0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009f50 tx=0x7fd2b4016000 comp rx=0 tx=0).stop 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 shutdown_connections 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd2ac0670e0 0x7fd2ac069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd2c0068390 0x7fd2c01554c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd2c0159a70 0x7fd2c01570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 --2- 172.21.15.119:0/1566022243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c006bfa0 0x7fd2c014ffe0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 >> 172.21.15.119:0/1566022243 conn(0x7fd2c00cddc0 msgr2=0x7fd2c0150e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:49.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 shutdown_connections 2023-10-08T02:31:49.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.693+0000 7fd2c4f9d700 1 -- 172.21.15.119:0/1566022243 wait complete. 2023-10-08T02:31:49.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-08T02:31:49.743 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:49.742+0000 7f8a1666a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-08T02:31:49.802 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:49.801+0000 7f4549a91180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:31:49.902 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:49.901+0000 7f4549a91180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:49.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.903+0000 7f7b3df9d700 1 Processor -- start 2023-10-08T02:31:49.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.903+0000 7f7b3df9d700 1 -- start start 2023-10-08T02:31:49.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.905+0000 7f7b3df9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7b380b1b40 0x7f7b380b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.905+0000 7f7b3df9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b380cfb00 con 0x7f7b380b16d0 2023-10-08T02:31:49.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.905+0000 7f7b3df9d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b380b5c40 con 0x7f7b380b4e70 2023-10-08T02:31:49.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.905+0000 7f7b3df9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b380b5f20 con 0x7f7b380b1b40 2023-10-08T02:31:49.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.905+0000 7f7b37fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7b380b16d0 0x7f7b380b33b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41624/0 (socket says 172.21.15.119:41624) 2023-10-08T02:31:49.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.905+0000 7f7b37fff700 1 -- 172.21.15.119:0/538685461 learned_addr learned my addr 172.21.15.119:0/538685461 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:49.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.905+0000 7f7b3d79c700 1 --2- 172.21.15.119:0/538685461 >> v2:172.21.15.119:3300/0 conn(0x7f7b380b1b40 0x7f7b380b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.905+0000 7f7b377fe700 1 -- 172.21.15.119:0/538685461 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961053820 0 0) 0x7f7b380cfb00 con 0x7f7b380b16d0 2023-10-08T02:31:49.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b377fe700 1 -- 172.21.15.119:0/538685461 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b28003570 con 0x7f7b380b16d0 2023-10-08T02:31:49.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b377fe700 1 -- 172.21.15.119:0/538685461 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868439907 0 0) 0x7f7b380b5c40 con 0x7f7b380b4e70 2023-10-08T02:31:49.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b3d79c700 1 -- 172.21.15.119:0/538685461 >> v1:172.21.15.184:6789/0 conn(0x7f7b380b4e70 legacy=0x7f7b380ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b3d79c700 1 -- 172.21.15.119:0/538685461 >> v1:172.21.15.119:6789/0 conn(0x7f7b380b16d0 legacy=0x7f7b380b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b3d79c700 1 -- 172.21.15.119:0/538685461 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b380b6200 con 0x7f7b380b1b40 2023-10-08T02:31:49.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b3d79c700 1 --2- 172.21.15.119:0/538685461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b1b40 0x7f7b380b3ac0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009e60 tx=0x7f7b30009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1ccf3ffe656cd4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:49.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b377fe700 1 -- 172.21.15.119:0/538685461 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3000e070 con 0x7f7b380b1b40 2023-10-08T02:31:49.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b377fe700 1 -- 172.21.15.119:0/538685461 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3000bb60 con 0x7f7b380b1b40 2023-10-08T02:31:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.906+0000 7f7b377fe700 1 -- 172.21.15.119:0/538685461 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30012430 con 0x7f7b380b1b40 2023-10-08T02:31:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.907+0000 7f7b3df9d700 1 -- 172.21.15.119:0/538685461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b1b40 msgr2=0x7f7b380b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.907+0000 7f7b3df9d700 1 --2- 172.21.15.119:0/538685461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b1b40 0x7f7b380b3ac0 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f7b30009e60 tx=0x7f7b30009780 comp rx=0 tx=0).stop 2023-10-08T02:31:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.907+0000 7f7b3df9d700 1 -- 172.21.15.119:0/538685461 shutdown_connections 2023-10-08T02:31:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.907+0000 7f7b3df9d700 1 --2- 172.21.15.119:0/538685461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b1b40 0x7f7b380b3ac0 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.907+0000 7f7b3df9d700 1 -- 172.21.15.119:0/538685461 >> 172.21.15.119:0/538685461 conn(0x7f7b380d10c0 msgr2=0x7f7b380affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.907+0000 7f7b3df9d700 1 -- 172.21.15.119:0/538685461 shutdown_connections 2023-10-08T02:31:49.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.907+0000 7f7b3df9d700 1 -- 172.21.15.119:0/538685461 wait complete. 2023-10-08T02:31:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.908+0000 7f7b3df9d700 1 Processor -- start 2023-10-08T02:31:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.908+0000 7f7b3df9d700 1 -- start start 2023-10-08T02:31:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.909+0000 7f7b3df9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b380b16d0 0x7f7b38157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.909+0000 7f7b3df9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b380b1b40 0x7f7b380c5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.909+0000 7f7b3df9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b4e70 0x7f7b3815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.909+0000 7f7b3df9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b380c6760 con 0x7f7b380b4e70 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.909+0000 7f7b3df9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b38158050 con 0x7f7b380b16d0 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.909+0000 7f7b3df9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b380c9460 con 0x7f7b380b1b40 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b4e70 0x7f7b3815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b4e70 0x7f7b3815b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42890/0 (socket says 172.21.15.119:42890) 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 -- 172.21.15.119:0/1117200689 learned_addr learned my addr 172.21.15.119:0/1117200689 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b37fff700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b380b1b40 0x7f7b380c5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3cf9b700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b380b16d0 0x7f7b38157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 -- 172.21.15.119:0/1117200689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b380b1b40 msgr2=0x7f7b380c5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b380b1b40 0x7f7b380c5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 -- 172.21.15.119:0/1117200689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b380b16d0 msgr2=0x7f7b38157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b380b16d0 0x7f7b38157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 -- 172.21.15.119:0/1117200689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30009a20 con 0x7f7b380b4e70 2023-10-08T02:31:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b3d79c700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b4e70 0x7f7b3815b590 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f7b38070dd0 tx=0x7f7b30019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.910+0000 7f7b34ff9700 1 -- 172.21.15.119:0/1117200689 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30024400 con 0x7f7b380b4e70 2023-10-08T02:31:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.911+0000 7f7b34ff9700 1 -- 172.21.15.119:0/1117200689 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b30024560 con 0x7f7b380b4e70 2023-10-08T02:31:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.911+0000 7f7b34ff9700 1 -- 172.21.15.119:0/1117200689 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30024900 con 0x7f7b380b4e70 2023-10-08T02:31:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.911+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b380c5be0 con 0x7f7b380b4e70 2023-10-08T02:31:49.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.911+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b380c61a0 con 0x7f7b380b4e70 2023-10-08T02:31:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.912+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b38088990 con 0x7f7b380b4e70 2023-10-08T02:31:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.913+0000 7f7b34ff9700 1 -- 172.21.15.119:0/1117200689 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f7b30024cf0 con 0x7f7b380b4e70 2023-10-08T02:31:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.913+0000 7f7b34ff9700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b1c0670e0 0x7f7b1c069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:49.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.914+0000 7f7b34ff9700 1 -- 172.21.15.119:0/1117200689 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f7b300863d0 con 0x7f7b380b4e70 2023-10-08T02:31:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.914+0000 7f7b3cf9b700 1 -- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b1c0670e0 msgr2=0x7f7b1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:49.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.914+0000 7f7b3cf9b700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b1c0670e0 0x7f7b1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:49.915 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:49.914+0000 7f8a1666a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-08T02:31:49.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:49.918+0000 7f7b34ff9700 1 -- 172.21.15.119:0/1117200689 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7b30052fa0 con 0x7f7b380b4e70 2023-10-08T02:31:50.019 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:50.019+0000 7f8a1666a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-08T02:31:50.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.115+0000 7f7b3cf9b700 1 -- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b1c0670e0 msgr2=0x7f7b1c069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:50.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.115+0000 7f7b3cf9b700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b1c0670e0 0x7f7b1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:50.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.116+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7b3804bb80 con 0x7f7b380b4e70 2023-10-08T02:31:50.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.117+0000 7f7b34ff9700 1 -- 172.21.15.119:0/1117200689 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v207) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7b300565c0 con 0x7f7b380b4e70 2023-10-08T02:31:50.118 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-10-08T02:31:50.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b1c0670e0 msgr2=0x7f7b1c069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:50.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b1c0670e0 0x7f7b1c069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b4e70 msgr2=0x7f7b3815b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b4e70 0x7f7b3815b590 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f7b38070dd0 tx=0x7f7b30019000 comp rx=0 tx=0).stop 2023-10-08T02:31:50.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 shutdown_connections 2023-10-08T02:31:50.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7b1c0670e0 0x7f7b1c069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b380b4e70 0x7f7b3815b590 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7b380b16d0 0x7f7b38157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 --2- 172.21.15.119:0/1117200689 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7b380b1b40 0x7f7b380c5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 >> 172.21.15.119:0/1117200689 conn(0x7f7b380d10c0 msgr2=0x7f7b380b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:50.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 shutdown_connections 2023-10-08T02:31:50.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.119+0000 7f7b3df9d700 1 -- 172.21.15.119:0/1117200689 wait complete. 2023-10-08T02:31:50.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.365+0000 7f4c8179d700 1 Processor -- start 2023-10-08T02:31:50.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.365+0000 7f4c8179d700 1 -- start start 2023-10-08T02:31:50.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c8179d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4c7c142ea0 0x7f4c7c0b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c8179d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c0ad870 0x7f4c7c06b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c8179d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c7c0b40f0 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c8179d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7c069640 con 0x7f4c7c142ea0 2023-10-08T02:31:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c8179d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c069920 con 0x7f4c7c0ad870 2023-10-08T02:31:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c80f9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4c7c0b3b70 0x7f4c7c06cf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41626/0 (socket says 172.21.15.119:41626) 2023-10-08T02:31:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c80f9c700 1 -- 172.21.15.119:0/1816271633 learned_addr learned my addr 172.21.15.119:0/1816271633 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c7b7fe700 1 --2- 172.21.15.119:0/1816271633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c0ad870 0x7f4c7c06b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c7bfff700 1 --2- 172.21.15.119:0/1816271633 >> v2:172.21.15.184:3300/0 conn(0x7f4c7c142ea0 0x7f4c7c0b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c7affd700 1 -- 172.21.15.119:0/1816271633 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122495176 0 0) 0x7f4c7c0b40f0 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.367+0000 7f4c7affd700 1 -- 172.21.15.119:0/1816271633 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c64003570 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.368+0000 7f4c7bfff700 1 -- 172.21.15.119:0/1816271633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c0ad870 msgr2=0x7f4c7c06b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.368+0000 7f4c7bfff700 1 --2- 172.21.15.119:0/1816271633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c0ad870 0x7f4c7c06b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.368+0000 7f4c7bfff700 1 -- 172.21.15.119:0/1816271633 >> v1:172.21.15.119:6789/0 conn(0x7f4c7c0b3b70 legacy=0x7f4c7c06cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.368+0000 7f4c7bfff700 1 -- 172.21.15.119:0/1816271633 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7c069c00 con 0x7f4c7c142ea0 2023-10-08T02:31:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.368+0000 7f4c7bfff700 1 --2- 172.21.15.119:0/1816271633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c142ea0 0x7f4c7c0b3630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0098e0 tx=0x7f4c6c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a6d8351addb4877 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:50.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.368+0000 7f4c7affd700 1 -- 172.21.15.119:0/1816271633 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011630 con 0x7f4c7c142ea0 2023-10-08T02:31:50.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.368+0000 7f4c7affd700 1 -- 172.21.15.119:0/1816271633 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c011790 con 0x7f4c7c142ea0 2023-10-08T02:31:50.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.368+0000 7f4c7affd700 1 -- 172.21.15.119:0/1816271633 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011b30 con 0x7f4c7c142ea0 2023-10-08T02:31:50.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.369+0000 7f4c8179d700 1 -- 172.21.15.119:0/1816271633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c142ea0 msgr2=0x7f4c7c0b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.369+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1816271633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c142ea0 0x7f4c7c0b3630 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c0098e0 tx=0x7f4c6c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.369+0000 7f4c8179d700 1 -- 172.21.15.119:0/1816271633 shutdown_connections 2023-10-08T02:31:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.369+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1816271633 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c142ea0 0x7f4c7c0b3630 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.369+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1816271633 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c0ad870 0x7f4c7c06b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.369+0000 7f4c8179d700 1 -- 172.21.15.119:0/1816271633 >> 172.21.15.119:0/1816271633 conn(0x7f4c7c0ce430 msgr2=0x7f4c7c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.369+0000 7f4c8179d700 1 -- 172.21.15.119:0/1816271633 shutdown_connections 2023-10-08T02:31:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.369+0000 7f4c8179d700 1 -- 172.21.15.119:0/1816271633 wait complete. 2023-10-08T02:31:50.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.371+0000 7f4c8179d700 1 Processor -- start 2023-10-08T02:31:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.371+0000 7f4c8179d700 1 -- start start 2023-10-08T02:31:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.371+0000 7f4c8179d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c0b3b70 0x7f4c7c0c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.371+0000 7f4c8179d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c142ea0 0x7f4c7c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c8179d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c15ecc0 0x7f4c7c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c8179d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c159ea0 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c8179d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0c8bb0 con 0x7f4c7c15ecc0 2023-10-08T02:31:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c8179d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7c0c8e90 con 0x7f4c7c142ea0 2023-10-08T02:31:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c0b3b70 0x7f4c7c0c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c0b3b70 0x7f4c7c0c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42894/0 (socket says 172.21.15.119:42894) 2023-10-08T02:31:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 -- 172.21.15.119:0/1912572405 learned_addr learned my addr 172.21.15.119:0/1912572405 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c80f9c700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c15ecc0 0x7f4c7c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7b7fe700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c142ea0 0x7f4c7c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 -- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c142ea0 msgr2=0x7f4c7c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c142ea0 0x7f4c7c158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 -- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c15ecc0 msgr2=0x7f4c7c15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c15ecc0 0x7f4c7c15c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 -- 172.21.15.119:0/1912572405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c009420 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c80f9c700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c15ecc0 0x7f4c7c15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7b7fe700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c142ea0 0x7f4c7c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.372+0000 7f4c7bfff700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c0b3b70 0x7f4c7c0c82b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c06f0d0 tx=0x7f4c6c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.373+0000 7f4c63fff700 1 -- 172.21.15.119:0/1912572405 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011b10 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.373+0000 7f4c63fff700 1 -- 172.21.15.119:0/1912572405 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6c0253f0 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.373+0000 7f4c63fff700 1 -- 172.21.15.119:0/1912572405 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c025790 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.373+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7c0c9170 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.373+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7c0c87f0 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.374+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c7c15a330 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.375+0000 7f4c63fff700 1 -- 172.21.15.119:0/1912572405 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f4c6c025b80 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.376+0000 7f4c63fff700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4c680670e0 0x7f4c68069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.376+0000 7f4c63fff700 1 -- 172.21.15.119:0/1912572405 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f4c6c086400 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.376+0000 7f4c7b7fe700 1 -- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4c680670e0 msgr2=0x7f4c68069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.376+0000 7f4c7b7fe700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4c680670e0 0x7f4c68069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.384+0000 7f4c63fff700 1 -- 172.21.15.119:0/1912572405 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4c7c15a330 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.385 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:50.385+0000 7f4549a91180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:31:50.480 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:50.479+0000 7f4549a91180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:31:50.525 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:50.525+0000 7f8a1666a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-08T02:31:50.569 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:50.568+0000 7f4549a91180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:31:50.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.577+0000 7f4c7b7fe700 1 -- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4c680670e0 msgr2=0x7f4c68069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:50.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.577+0000 7f4c7b7fe700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4c680670e0 0x7f4c68069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:50.622 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:50.622+0000 7f8a1666a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-08T02:31:50.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.626+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c7c15a330 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.627+0000 7f4c63fff700 1 -- 172.21.15.119:0/1912572405 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f4c7c15a330 con 0x7f4c7c0b3b70 2023-10-08T02:31:50.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:50.627 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1044912493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1044912493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1044912493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10197,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1564057059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1564057059},{"type":"v1","addr":"172.21.15.119:6833","nonce":1564057059}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2545901849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2545901849},{"type":"v1","addr":"172.21.15.184:6834","nonce":2545901849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:31:27.855883+0000","modified":"2023-10-08T02:31:31.420955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10215":{"gid":10215,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1741909431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1741909431},{"type":"v1","addr":"172.21.15.119:6835","nonce":1741909431}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T02:31:50.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4c680670e0 msgr2=0x7f4c68069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:50.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4c680670e0 0x7f4c68069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c0b3b70 msgr2=0x7f4c7c0c82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c0b3b70 0x7f4c7c0c82b0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c06f0d0 tx=0x7f4c6c017000 comp rx=0 tx=0).stop 2023-10-08T02:31:50.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 shutdown_connections 2023-10-08T02:31:50.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4c680670e0 0x7f4c68069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4c7c15ecc0 0x7f4c7c15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c7c0b3b70 0x7f4c7c0c82b0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 --2- 172.21.15.119:0/1912572405 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4c7c142ea0 0x7f4c7c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 >> 172.21.15.119:0/1912572405 conn(0x7f4c7c0ce430 msgr2=0x7f4c7c142120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:50.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 shutdown_connections 2023-10-08T02:31:50.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.629+0000 7f4c8179d700 1 -- 172.21.15.119:0/1912572405 wait complete. 2023-10-08T02:31:50.631 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-10-08T02:31:50.648 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:50.725 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:50.725+0000 7f8a1666a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-08T02:31:50.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.838+0000 7f183159d700 1 Processor -- start 2023-10-08T02:31:50.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.838+0000 7f183159d700 1 -- start start 2023-10-08T02:31:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.839+0000 7f183159d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c159420 0x7f182c0b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.839+0000 7f183159d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0b2980 con 0x7f182c15d230 2023-10-08T02:31:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.839+0000 7f183159d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f182c0b11d0 con 0x7f182c1597f0 2023-10-08T02:31:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.839+0000 7f183159d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b14b0 con 0x7f182c159420 2023-10-08T02:31:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f1830d9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f182c15d230 0x7f182c0b3e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41642/0 (socket says 172.21.15.119:41642) 2023-10-08T02:31:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f1830d9c700 1 -- 172.21.15.119:0/984556816 learned_addr learned my addr 172.21.15.119:0/984556816 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:50.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f182bfff700 1 --2- 172.21.15.119:0/984556816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c159420 0x7f182c0b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f182affd700 1 -- 172.21.15.119:0/984556816 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983612833 0 0) 0x7f182c0b2980 con 0x7f182c15d230 2023-10-08T02:31:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f182affd700 1 -- 172.21.15.119:0/984556816 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1814003570 con 0x7f182c15d230 2023-10-08T02:31:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f182affd700 1 -- 172.21.15.119:0/984556816 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482288735 0 0) 0x7f182c0b11d0 con 0x7f182c1597f0 2023-10-08T02:31:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f182affd700 1 -- 172.21.15.119:0/984556816 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f182c0b2980 con 0x7f182c1597f0 2023-10-08T02:31:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f182bfff700 1 -- 172.21.15.119:0/984556816 >> v1:172.21.15.184:6789/0 conn(0x7f182c1597f0 legacy=0x7f182c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f182bfff700 1 -- 172.21.15.119:0/984556816 >> v1:172.21.15.119:6789/0 conn(0x7f182c15d230 legacy=0x7f182c0b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.840+0000 7f182bfff700 1 -- 172.21.15.119:0/984556816 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f182c0b1790 con 0x7f182c159420 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f182bfff700 1 --2- 172.21.15.119:0/984556816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c159420 0x7f182c0b5f40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f181c009910 tx=0x7f181c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b014296337102d4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f182affd700 1 -- 172.21.15.119:0/984556816 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c011630 con 0x7f182c159420 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f182affd700 1 -- 172.21.15.119:0/984556816 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c011790 con 0x7f182c159420 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f182affd700 1 -- 172.21.15.119:0/984556816 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c011b10 con 0x7f182c159420 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f183159d700 1 -- 172.21.15.119:0/984556816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c159420 msgr2=0x7f182c0b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f183159d700 1 --2- 172.21.15.119:0/984556816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c159420 0x7f182c0b5f40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f181c009910 tx=0x7f181c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f183159d700 1 -- 172.21.15.119:0/984556816 shutdown_connections 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f183159d700 1 --2- 172.21.15.119:0/984556816 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c159420 0x7f182c0b5f40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.841+0000 7f183159d700 1 -- 172.21.15.119:0/984556816 >> 172.21.15.119:0/984556816 conn(0x7f182c0d0660 msgr2=0x7f182c0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:50.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.842+0000 7f183159d700 1 -- 172.21.15.119:0/984556816 shutdown_connections 2023-10-08T02:31:50.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.842+0000 7f183159d700 1 -- 172.21.15.119:0/984556816 wait complete. 2023-10-08T02:31:50.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.843+0000 7f183159d700 1 Processor -- start 2023-10-08T02:31:50.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.843+0000 7f183159d700 1 -- start start 2023-10-08T02:31:50.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f183159d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c159420 0x7f182c14d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f183159d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1597f0 0x7f182c14e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f183159d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15d230 0x7f182c0b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f183159d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c152580 con 0x7f182c1597f0 2023-10-08T02:31:50.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f183159d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182c14f8b0 con 0x7f182c159420 2023-10-08T02:31:50.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f183159d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182c0b4ba0 con 0x7f182c15d230 2023-10-08T02:31:50.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f182b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1597f0 0x7f182c14e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f182bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c159420 0x7f182c14d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f182b7fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1597f0 0x7f182c14e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42910/0 (socket says 172.21.15.119:42910) 2023-10-08T02:31:50.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f182bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c159420 0x7f182c14d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:60490/0 (socket says 172.21.15.119:60490) 2023-10-08T02:31:50.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f182b7fe700 1 -- 172.21.15.119:0/1276725279 learned_addr learned my addr 172.21.15.119:0/1276725279 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:50.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.844+0000 7f1830d9c700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15d230 0x7f182c0b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:50.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f182bfff700 1 -- 172.21.15.119:0/1276725279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15d230 msgr2=0x7f182c0b5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f182bfff700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15d230 0x7f182c0b5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f182bfff700 1 -- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1597f0 msgr2=0x7f182c14e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:50.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f182bfff700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1597f0 0x7f182c14e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:50.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f182bfff700 1 -- 172.21.15.119:0/1276725279 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c009420 con 0x7f182c159420 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f182bfff700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c159420 0x7f182c14d260 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f181c016000 tx=0x7f181c017000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f1813fff700 1 -- 172.21.15.119:0/1276725279 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c011b10 con 0x7f182c159420 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f1813fff700 1 -- 172.21.15.119:0/1276725279 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c01be00 con 0x7f182c159420 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182c14f010 con 0x7f182c159420 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.845+0000 7f1813fff700 1 -- 172.21.15.119:0/1276725279 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c025620 con 0x7f182c159420 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.846+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182c14f5d0 con 0x7f182c159420 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.847+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182c0cda20 con 0x7f182c159420 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.847+0000 7f1813fff700 1 -- 172.21.15.119:0/1276725279 <== mon.1 v2:172.21.15.184:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f181c025a50 con 0x7f182c159420 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.848+0000 7f1813fff700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f18180670e0 0x7f1818069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:50.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.848+0000 7f1813fff700 1 -- 172.21.15.119:0/1276725279 <== mon.1 v2:172.21.15.184:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f181c086470 con 0x7f182c159420 2023-10-08T02:31:50.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.848+0000 7f182b7fe700 1 -- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f18180670e0 msgr2=0x7f1818069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:50.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.848+0000 7f182b7fe700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f18180670e0 0x7f1818069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:50.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:50.853+0000 7f1813fff700 1 -- 172.21.15.119:0/1276725279 <== mon.1 v2:172.21.15.184:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f181c0530b0 con 0x7f182c159420 2023-10-08T02:31:51.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.049+0000 7f182b7fe700 1 -- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f18180670e0 msgr2=0x7f1818069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:51.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.049+0000 7f182b7fe700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f18180670e0 0x7f1818069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:51.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.053+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f182c04c3c0 con 0x7f182c159420 2023-10-08T02:31:51.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.055+0000 7f1813fff700 1 -- 172.21.15.119:0/1276725279 <== mon.1 v2:172.21.15.184:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+16589 (secure 0 0 0) 0x7f181c0566d0 con 0x7f182c159420 2023-10-08T02:31:51.056 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:51.056 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":207,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:43.569130+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:31:25.536431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T02:31:26.843560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/842982126":"2023-10-09T02:31:43.569085+0000","172.21.15.119:6837/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:6836/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/1887277487":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/4191951118":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/773312007":"2023-10-09T02:31:43.569085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:51.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f18180670e0 msgr2=0x7f1818069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:51.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f18180670e0 0x7f1818069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c159420 msgr2=0x7f182c14d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c159420 0x7f182c14d260 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f181c016000 tx=0x7f181c017000 comp rx=0 tx=0).stop 2023-10-08T02:31:51.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 shutdown_connections 2023-10-08T02:31:51.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f18180670e0 0x7f1818069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f182c15d230 0x7f182c0b5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f182c159420 0x7f182c14d260 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 --2- 172.21.15.119:0/1276725279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f182c1597f0 0x7f182c14e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 >> 172.21.15.119:0/1276725279 conn(0x7f182c0d0660 msgr2=0x7f182c0d0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:51.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 shutdown_connections 2023-10-08T02:31:51.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.057+0000 7f183159d700 1 -- 172.21.15.119:0/1276725279 wait complete. 2023-10-08T02:31:51.076 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T02:31:51.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:51.195 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:51.194+0000 7f4549a91180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:31:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.281+0000 7f431ffff700 1 Processor -- start 2023-10-08T02:31:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.282+0000 7f431ffff700 1 -- start start 2023-10-08T02:31:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431ffff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4320166530 0x7f43200c7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4320087a00 0x7f4320170040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431ffff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4320069cd0 con 0x7f43200caad0 2023-10-08T02:31:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431ffff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4320166900 con 0x7f4320166530 2023-10-08T02:31:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200696c0 con 0x7f4320087a00 2023-10-08T02:31:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431e7fc700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f43200caad0 0x7f43200872f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41652/0 (socket says 172.21.15.119:41652) 2023-10-08T02:31:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431e7fc700 1 -- 172.21.15.119:0/1022914242 learned_addr learned my addr 172.21.15.119:0/1022914242 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431effd700 1 --2- 172.21.15.119:0/1022914242 >> v2:172.21.15.184:3300/0 conn(0x7f4320166530 0x7f43200c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.284+0000 7f431f7fe700 1 --2- 172.21.15.119:0/1022914242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4320087a00 0x7f4320170040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.285+0000 7f431effd700 1 -- 172.21.15.119:0/1022914242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4320087a00 msgr2=0x7f4320170040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.285+0000 7f431effd700 1 --2- 172.21.15.119:0/1022914242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4320087a00 0x7f4320170040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.285+0000 7f431effd700 1 -- 172.21.15.119:0/1022914242 >> v1:172.21.15.119:6789/0 conn(0x7f43200caad0 legacy=0x7f43200872f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.285+0000 7f431dffb700 1 -- 172.21.15.119:0/1022914242 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866606197 0 0) 0x7f4320069cd0 con 0x7f43200caad0 2023-10-08T02:31:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.285+0000 7f431effd700 1 -- 172.21.15.119:0/1022914242 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432006b170 con 0x7f4320166530 2023-10-08T02:31:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.285+0000 7f431effd700 1 --2- 172.21.15.119:0/1022914242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320166530 0x7f43200c7260 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f431800b990 tx=0x7f431800a0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e173c33f6be7c3 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:51.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.286+0000 7f431dffb700 1 -- 172.21.15.119:0/1022914242 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431800be80 con 0x7f4320166530 2023-10-08T02:31:51.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.286+0000 7f431dffb700 1 -- 172.21.15.119:0/1022914242 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318006af0 con 0x7f4320166530 2023-10-08T02:31:51.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.286+0000 7f431dffb700 1 -- 172.21.15.119:0/1022914242 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318006e70 con 0x7f4320166530 2023-10-08T02:31:51.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.286+0000 7f431ffff700 1 -- 172.21.15.119:0/1022914242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320166530 msgr2=0x7f43200c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.286+0000 7f431ffff700 1 --2- 172.21.15.119:0/1022914242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320166530 0x7f43200c7260 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f431800b990 tx=0x7f431800a0f0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.286+0000 7f431ffff700 1 -- 172.21.15.119:0/1022914242 shutdown_connections 2023-10-08T02:31:51.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.287+0000 7f431ffff700 1 --2- 172.21.15.119:0/1022914242 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4320087a00 0x7f4320170040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.287+0000 7f431ffff700 1 --2- 172.21.15.119:0/1022914242 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320166530 0x7f43200c7260 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.287+0000 7f431ffff700 1 -- 172.21.15.119:0/1022914242 >> 172.21.15.119:0/1022914242 conn(0x7f43200cc1f0 msgr2=0x7f43200cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:51.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.287+0000 7f431ffff700 1 -- 172.21.15.119:0/1022914242 shutdown_connections 2023-10-08T02:31:51.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.287+0000 7f431ffff700 1 -- 172.21.15.119:0/1022914242 wait complete. 2023-10-08T02:31:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.289+0000 7f431ffff700 1 Processor -- start 2023-10-08T02:31:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.290+0000 7f431ffff700 1 -- start start 2023-10-08T02:31:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431ffff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320087a00 0x7f43200c5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431ffff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43200caad0 0x7f4320161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431ffff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320166530 0x7f4320162180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431ffff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200b04a0 con 0x7f4320166530 2023-10-08T02:31:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431ffff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200afe90 con 0x7f4320087a00 2023-10-08T02:31:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431ffff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200c6200 con 0x7f43200caad0 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431e7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43200caad0 0x7f4320161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431effd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320087a00 0x7f43200c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431e7fc700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43200caad0 0x7f4320161c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:44748/0 (socket says 172.21.15.119:44748) 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431effd700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320087a00 0x7f43200c5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:60508/0 (socket says 172.21.15.119:60508) 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.291+0000 7f431e7fc700 1 -- 172.21.15.119:0/2442724483 learned_addr learned my addr 172.21.15.119:0/2442724483 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.292+0000 7f431f7fe700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320166530 0x7f4320162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.292+0000 7f431e7fc700 1 -- 172.21.15.119:0/2442724483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320087a00 msgr2=0x7f43200c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.292+0000 7f431e7fc700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320087a00 0x7f43200c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.292+0000 7f431e7fc700 1 -- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320166530 msgr2=0x7f4320162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.292+0000 7f431e7fc700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320166530 0x7f4320162180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.292+0000 7f431e7fc700 1 -- 172.21.15.119:0/2442724483 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431800b4d0 con 0x7f43200caad0 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.292+0000 7f431effd700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320087a00 0x7f43200c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.292+0000 7f431f7fe700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320166530 0x7f4320162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.293+0000 7f431e7fc700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43200caad0 0x7f4320161c40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f431000cb90 tx=0x7f4310007520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.293+0000 7f4316ffd700 1 -- 172.21.15.119:0/2442724483 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431000d040 con 0x7f43200caad0 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.293+0000 7f4316ffd700 1 -- 172.21.15.119:0/2442724483 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431000a830 con 0x7f43200caad0 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.293+0000 7f4316ffd700 1 -- 172.21.15.119:0/2442724483 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431000abd0 con 0x7f43200caad0 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.293+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43200c5f20 con 0x7f43200caad0 2023-10-08T02:31:51.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.293+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4320087670 con 0x7f43200caad0 2023-10-08T02:31:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.295+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43200b0610 con 0x7f43200caad0 2023-10-08T02:31:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.296+0000 7f4316ffd700 1 -- 172.21.15.119:0/2442724483 <== mon.2 v2:172.21.15.184:3301/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f431001b460 con 0x7f43200caad0 2023-10-08T02:31:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.297+0000 7f4316ffd700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f43000670e0 0x7f4300069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.298+0000 7f4316ffd700 1 -- 172.21.15.119:0/2442724483 <== mon.2 v2:172.21.15.184:3301/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f4310084a40 con 0x7f43200caad0 2023-10-08T02:31:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.298+0000 7f431effd700 1 -- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f43000670e0 msgr2=0x7f4300069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:51.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.298+0000 7f431effd700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f43000670e0 0x7f4300069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.306+0000 7f4316ffd700 1 -- 172.21.15.119:0/2442724483 <== mon.2 v2:172.21.15.184:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4310086e60 con 0x7f43200caad0 2023-10-08T02:31:51.372 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:51.372+0000 7f4549a91180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:51.374 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:51.373+0000 7f8a1666a180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-08T02:31:51.470 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:51.469+0000 7f4549a91180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:31:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.499+0000 7f431effd700 1 -- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f43000670e0 msgr2=0x7f4300069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.499+0000 7f431effd700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f43000670e0 0x7f4300069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:51.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.554+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f432004c3c0 con 0x7f43200caad0 2023-10-08T02:31:51.555 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:51.554+0000 7f8a1666a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-08T02:31:51.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.554+0000 7f4316ffd700 1 -- 172.21.15.119:0/2442724483 <== mon.2 v2:172.21.15.184:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f4310051750 con 0x7f43200caad0 2023-10-08T02:31:51.555 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:51.556 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1044912493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1044912493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1044912493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10197,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1564057059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1564057059},{"type":"v1","addr":"172.21.15.119:6833","nonce":1564057059}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2545901849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2545901849},{"type":"v1","addr":"172.21.15.184:6834","nonce":2545901849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:31:27.855883+0000","modified":"2023-10-08T02:31:31.420955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10215":{"gid":10215,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1741909431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1741909431},{"type":"v1","addr":"172.21.15.119:6835","nonce":1741909431}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f43000670e0 msgr2=0x7f4300069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f43000670e0 0x7f4300069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43200caad0 msgr2=0x7f4320161c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43200caad0 0x7f4320161c40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f431000cb90 tx=0x7f4310007520 comp rx=0 tx=0).stop 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 shutdown_connections 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f43000670e0 0x7f4300069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320166530 0x7f4320162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4320087a00 0x7f43200c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 --2- 172.21.15.119:0/2442724483 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f43200caad0 0x7f4320161c40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 >> 172.21.15.119:0/2442724483 conn(0x7f43200cc1f0 msgr2=0x7f432006dcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:51.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 shutdown_connections 2023-10-08T02:31:51.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.557+0000 7f431ffff700 1 -- 172.21.15.119:0/2442724483 wait complete. 2023-10-08T02:31:51.559 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-10-08T02:31:51.576 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:31:51.658 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:51.657+0000 7f8a1666a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-08T02:31:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.762+0000 7f8223dff700 1 Processor -- start 2023-10-08T02:31:51.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.763+0000 7f8223dff700 1 -- start start 2023-10-08T02:31:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f8223dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c159890 0x7f821c15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f8223dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f821c069560 0x7f821c0699e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f8223dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c15a160 con 0x7f821c159c60 2023-10-08T02:31:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f8223dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c069f20 con 0x7f821c069560 2023-10-08T02:31:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f8223dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0d13a0 con 0x7f821c159890 2023-10-08T02:31:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f822239c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f821c069560 0x7f821c0699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f822239c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f821c069560 0x7f821c0699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42924/0 (socket says 172.21.15.119:42924) 2023-10-08T02:31:51.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f822239c700 1 -- 172.21.15.119:0/296435101 learned_addr learned my addr 172.21.15.119:0/296435101 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f8221b9b700 1 --2- 172.21.15.119:0/296435101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c159890 0x7f821c15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.764+0000 7f822139a700 1 --1- 172.21.15.119:0/296435101 >> v1:172.21.15.184:6789/0 conn(0x7f821c159c60 0x7f821c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:59858/0 (socket says 172.21.15.119:59858) 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f822239c700 1 -- 172.21.15.119:0/296435101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c159890 msgr2=0x7f821c15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f822239c700 1 --2- 172.21.15.119:0/296435101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c159890 0x7f821c15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f822239c700 1 -- 172.21.15.119:0/296435101 >> v1:172.21.15.184:6789/0 conn(0x7f821c159c60 legacy=0x7f821c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f822239c700 1 -- 172.21.15.119:0/296435101 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c15e140 con 0x7f821c069560 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f8220b99700 1 -- 172.21.15.119:0/296435101 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375633987 0 0) 0x7f821c15a160 con 0x7f821c159c60 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f822239c700 1 --2- 172.21.15.119:0/296435101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c069560 0x7f821c0699e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f8214009f20 tx=0x7f8214009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51a03d1897c059a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f8220b99700 1 -- 172.21.15.119:0/296435101 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400e070 con 0x7f821c069560 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f8220b99700 1 -- 172.21.15.119:0/296435101 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821400b9d0 con 0x7f821c069560 2023-10-08T02:31:51.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.765+0000 7f8220b99700 1 -- 172.21.15.119:0/296435101 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140092e0 con 0x7f821c069560 2023-10-08T02:31:51.768 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:51.766+0000 7f4549a91180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:31:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.766+0000 7f8223dff700 1 -- 172.21.15.119:0/296435101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c069560 msgr2=0x7f821c0699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.766+0000 7f8223dff700 1 --2- 172.21.15.119:0/296435101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c069560 0x7f821c0699e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f8214009f20 tx=0x7f8214009840 comp rx=0 tx=0).stop 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.766+0000 7f8223dff700 1 -- 172.21.15.119:0/296435101 shutdown_connections 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.766+0000 7f8223dff700 1 --2- 172.21.15.119:0/296435101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c069560 0x7f821c0699e0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.766+0000 7f8223dff700 1 --2- 172.21.15.119:0/296435101 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c159890 0x7f821c15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.766+0000 7f8223dff700 1 -- 172.21.15.119:0/296435101 >> 172.21.15.119:0/296435101 conn(0x7f821c068e20 msgr2=0x7f821c0d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.766+0000 7f8223dff700 1 -- 172.21.15.119:0/296435101 shutdown_connections 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.766+0000 7f8223dff700 1 -- 172.21.15.119:0/296435101 wait complete. 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.768+0000 7f8223dff700 1 Processor -- start 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.768+0000 7f8223dff700 1 -- start start 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f8223dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c069560 0x7f821c0b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f8223dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c159890 0x7f821c0c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f8223dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f821c159c60 0x7f821c145df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f8223dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0c99c0 con 0x7f821c159890 2023-10-08T02:31:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f8223dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0c9ca0 con 0x7f821c159c60 2023-10-08T02:31:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f8223dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0c9f80 con 0x7f821c069560 2023-10-08T02:31:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c159890 0x7f821c0c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c159890 0x7f821c0c9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42930/0 (socket says 172.21.15.119:42930) 2023-10-08T02:31:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822139a700 1 -- 172.21.15.119:0/527028781 learned_addr learned my addr 172.21.15.119:0/527028781 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f8221b9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c069560 0x7f821c0b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822239c700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f821c159c60 0x7f821c145df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822139a700 1 -- 172.21.15.119:0/527028781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c069560 msgr2=0x7f821c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822139a700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c069560 0x7f821c0b5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822139a700 1 -- 172.21.15.119:0/527028781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f821c159c60 msgr2=0x7f821c145df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822139a700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f821c159c60 0x7f821c145df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.769+0000 7f822139a700 1 -- 172.21.15.119:0/527028781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214009ae0 con 0x7f821c159890 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.770+0000 7f822139a700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c159890 0x7f821c0c9020 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c7e0 tx=0x7f820c007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.770+0000 7f8211ffb700 1 -- 172.21.15.119:0/527028781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c005b40 con 0x7f821c159890 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.770+0000 7f8211ffb700 1 -- 172.21.15.119:0/527028781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f820c005cc0 con 0x7f821c159890 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.770+0000 7f8211ffb700 1 -- 172.21.15.119:0/527028781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820c007320 con 0x7f821c159890 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.770+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c0ca260 con 0x7f821c159890 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.770+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c0caa60 con 0x7f821c159890 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.771+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c0ca780 con 0x7f821c159890 2023-10-08T02:31:51.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.772+0000 7f8211ffb700 1 -- 172.21.15.119:0/527028781 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f820c01b3b0 con 0x7f821c159890 2023-10-08T02:31:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.773+0000 7f8211ffb700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82000670e0 0x7f8200069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.773+0000 7f8211ffb700 1 -- 172.21.15.119:0/527028781 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f820c0840d0 con 0x7f821c159890 2023-10-08T02:31:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.773+0000 7f8221b9b700 1 -- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82000670e0 msgr2=0x7f8200069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:51.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.773+0000 7f8221b9b700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82000670e0 0x7f8200069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:51.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.777+0000 7f8211ffb700 1 -- 172.21.15.119:0/527028781 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f821c0ca780 con 0x7f821c159890 2023-10-08T02:31:51.864 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:51.863+0000 7f4549a91180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:31:51.964 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:51.964+0000 7f8a1666a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-08T02:31:51.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.974+0000 7f8221b9b700 1 -- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82000670e0 msgr2=0x7f8200069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:51.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:51.974+0000 7f8221b9b700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82000670e0 0x7f8200069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:52.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.021+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f821c0ca780 con 0x7f821c159890 2023-10-08T02:31:52.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.022+0000 7f8211ffb700 1 -- 172.21.15.119:0/527028781 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4618 (secure 0 0 0) 0x7f821c0ca780 con 0x7f821c159890 2023-10-08T02:31:52.023 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:52.023 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1044912493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1044912493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1044912493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10197,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1564057059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1564057059},{"type":"v1","addr":"172.21.15.119:6833","nonce":1564057059}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2545901849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2545901849},{"type":"v1","addr":"172.21.15.184:6834","nonce":2545901849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-08T02:31:27.855883+0000","modified":"2023-10-08T02:31:31.420955+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10215},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10215":{"gid":10215,"name":"a","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6835/1741909431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1741909431},{"type":"v1","addr":"172.21.15.119:6835","nonce":1741909431}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-10-08T02:31:52.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82000670e0 msgr2=0x7f8200069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:52.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82000670e0 0x7f8200069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c159890 msgr2=0x7f821c0c9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c159890 0x7f821c0c9020 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f820c00c7e0 tx=0x7f820c007de0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 shutdown_connections 2023-10-08T02:31:52.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f82000670e0 0x7f8200069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f821c069560 0x7f821c0b5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f821c159c60 0x7f821c145df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 --2- 172.21.15.119:0/527028781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f821c159890 0x7f821c0c9020 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 >> 172.21.15.119:0/527028781 conn(0x7f821c068e20 msgr2=0x7f821c1551a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:52.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 shutdown_connections 2023-10-08T02:31:52.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.024+0000 7f8223dff700 1 -- 172.21.15.119:0/527028781 wait complete. 2023-10-08T02:31:52.027 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-10-08T02:31:52.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:52.065 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:52.064+0000 7f8a1666a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-08T02:31:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.229+0000 7f5ecf59e700 1 Processor -- start 2023-10-08T02:31:52.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.230+0000 7f5ecf59e700 1 -- start start 2023-10-08T02:31:52.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.231+0000 7f5ecf59e700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ed0159420 0x7f5ed00b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.231+0000 7f5ecf59e700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed00b2980 con 0x7f5ed015d230 2023-10-08T02:31:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.231+0000 7f5ecf59e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed00b11d0 con 0x7f5ed01597f0 2023-10-08T02:31:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.231+0000 7f5ecf59e700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed00b14b0 con 0x7f5ed0159420 2023-10-08T02:31:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.231+0000 7f5ece59c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ed0159420 0x7f5ed00b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.231+0000 7f5eced9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5ed015d230 0x7f5ed015d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41656/0 (socket says 172.21.15.119:41656) 2023-10-08T02:31:52.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.231+0000 7f5eced9d700 1 -- 172.21.15.119:0/3666758872 learned_addr learned my addr 172.21.15.119:0/3666758872 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ecd59a700 1 -- 172.21.15.119:0/3666758872 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235387508 0 0) 0x7f5ed00b2980 con 0x7f5ed015d230 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ecd59a700 1 -- 172.21.15.119:0/3666758872 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eb8003570 con 0x7f5ed015d230 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ece59c700 1 -- 172.21.15.119:0/3666758872 >> v1:172.21.15.184:6789/0 conn(0x7f5ed01597f0 legacy=0x7f5ed00b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ecd59a700 1 -- 172.21.15.119:0/3666758872 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310178698 0 0) 0x7f5ed00b11d0 con 0x7f5ed01597f0 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ece59c700 1 -- 172.21.15.119:0/3666758872 >> v1:172.21.15.119:6789/0 conn(0x7f5ed015d230 legacy=0x7f5ed015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ece59c700 1 -- 172.21.15.119:0/3666758872 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed00b1790 con 0x7f5ed0159420 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ece59c700 1 --2- 172.21.15.119:0/3666758872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed00b5f40 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098e0 tx=0x7f5ec00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc163a9c3e7816d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ecd59a700 1 -- 172.21.15.119:0/3666758872 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000b640 con 0x7f5ed0159420 2023-10-08T02:31:52.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ecd59a700 1 -- 172.21.15.119:0/3666758872 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000b7a0 con 0x7f5ed0159420 2023-10-08T02:31:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.232+0000 7f5ecd59a700 1 -- 172.21.15.119:0/3666758872 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000bb40 con 0x7f5ed0159420 2023-10-08T02:31:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.233+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3666758872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 msgr2=0x7f5ed00b5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.233+0000 7f5ecf59e700 1 --2- 172.21.15.119:0/3666758872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed00b5f40 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098e0 tx=0x7f5ec00091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.233+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3666758872 shutdown_connections 2023-10-08T02:31:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.233+0000 7f5ecf59e700 1 --2- 172.21.15.119:0/3666758872 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed00b5f40 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.233+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3666758872 >> 172.21.15.119:0/3666758872 conn(0x7f5ed00d0660 msgr2=0x7f5ed00d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.233+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3666758872 shutdown_connections 2023-10-08T02:31:52.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.233+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3666758872 wait complete. 2023-10-08T02:31:52.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.235+0000 7f5ecf59e700 1 Processor -- start 2023-10-08T02:31:52.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.235+0000 7f5ecf59e700 1 -- start start 2023-10-08T02:31:52.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ecf59e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed0151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ecf59e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01597f0 0x7f5ed014dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ecf59e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed015d230 0x7f5ed014e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ecf59e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed015cd40 con 0x7f5ed0159420 2023-10-08T02:31:52.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ecf59e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed0155860 con 0x7f5ed01597f0 2023-10-08T02:31:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ecf59e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed0152400 con 0x7f5ed015d230 2023-10-08T02:31:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ece59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed0151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ece59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed0151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42948/0 (socket says 172.21.15.119:42948) 2023-10-08T02:31:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ece59c700 1 -- 172.21.15.119:0/3387436533 learned_addr learned my addr 172.21.15.119:0/3387436533 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ecdd9b700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01597f0 0x7f5ed014dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5eced9d700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed015d230 0x7f5ed014e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ece59c700 1 -- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed015d230 msgr2=0x7f5ed014e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ece59c700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed015d230 0x7f5ed014e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ece59c700 1 -- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01597f0 msgr2=0x7f5ed014dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ece59c700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01597f0 0x7f5ed014dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5ece59c700 1 -- 172.21.15.119:0/3387436533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009420 con 0x7f5ed0159420 2023-10-08T02:31:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.236+0000 7f5eced9d700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed015d230 0x7f5ed014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.237+0000 7f5ece59c700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed0151ec0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0089340 tx=0x7f5ec0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.237+0000 7f5ebe7fc700 1 -- 172.21.15.119:0/3387436533 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec001f070 con 0x7f5ed0159420 2023-10-08T02:31:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.237+0000 7f5ebe7fc700 1 -- 172.21.15.119:0/3387436533 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec0000bc0 con 0x7f5ed0159420 2023-10-08T02:31:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.237+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed0154a80 con 0x7f5ed0159420 2023-10-08T02:31:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.237+0000 7f5ebe7fc700 1 -- 172.21.15.119:0/3387436533 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0025420 con 0x7f5ed0159420 2023-10-08T02:31:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.237+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed0155040 con 0x7f5ed0159420 2023-10-08T02:31:52.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.238+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed004c3c0 con 0x7f5ed0159420 2023-10-08T02:31:52.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.239+0000 7f5ebe7fc700 1 -- 172.21.15.119:0/3387436533 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f5ec0023070 con 0x7f5ed0159420 2023-10-08T02:31:52.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.240+0000 7f5ebe7fc700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5eb00670e0 0x7f5eb0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.240+0000 7f5ebe7fc700 1 -- 172.21.15.119:0/3387436533 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f5ec0085ff0 con 0x7f5ed0159420 2023-10-08T02:31:52.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.240+0000 7f5ecdd9b700 1 -- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5eb00670e0 msgr2=0x7f5eb0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:52.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.240+0000 7f5ecdd9b700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5eb00670e0 0x7f5eb0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:52.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.244+0000 7f5ebe7fc700 1 -- 172.21.15.119:0/3387436533 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ec0052c30 con 0x7f5ed0159420 2023-10-08T02:31:52.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.441+0000 7f5ecdd9b700 1 -- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5eb00670e0 msgr2=0x7f5eb0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:52.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.441+0000 7f5ecdd9b700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5eb00670e0 0x7f5eb0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:52.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.450+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ed00b1a70 con 0x7f5ed0159420 2023-10-08T02:31:52.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.451+0000 7f5ebe7fc700 1 -- 172.21.15.119:0/3387436533 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+16589 (secure 0 0 0) 0x7f5ec002a090 con 0x7f5ed0159420 2023-10-08T02:31:52.451 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:52.452 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":207,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:43.569130+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-10-08T02:31:25.536431+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"205","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-10-08T02:31:26.843560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/842982126":"2023-10-09T02:31:43.569085+0000","172.21.15.119:6837/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:6836/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/1887277487":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/4191951118":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/773312007":"2023-10-09T02:31:43.569085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:52.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5eb00670e0 msgr2=0x7f5eb0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:52.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5eb00670e0 0x7f5eb0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 msgr2=0x7f5ed0151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed0151ec0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0089340 tx=0x7f5ec0017000 comp rx=0 tx=0).stop 2023-10-08T02:31:52.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 shutdown_connections 2023-10-08T02:31:52.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f5eb00670e0 0x7f5eb0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f5ed015d230 0x7f5ed014e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ed0159420 0x7f5ed0151ec0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 --2- 172.21.15.119:0/3387436533 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f5ed01597f0 0x7f5ed014dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 >> 172.21.15.119:0/3387436533 conn(0x7f5ed00d0660 msgr2=0x7f5ed0068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:52.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 shutdown_connections 2023-10-08T02:31:52.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.453+0000 7f5ecf59e700 1 -- 172.21.15.119:0/3387436533 wait complete. 2023-10-08T02:31:52.472 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-08T02:31:52.571 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:52.570+0000 7f4549a91180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:31:52.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.703+0000 7f7fd979d700 1 Processor -- start 2023-10-08T02:31:52.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.704+0000 7f7fd979d700 1 -- start start 2023-10-08T02:31:52.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd979d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd979d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7fd406c0f0 0x7f7fd406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd979d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd40b40f0 con 0x7f7fd40ad870 2023-10-08T02:31:52.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd979d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd4069640 con 0x7f7fd406c0f0 2023-10-08T02:31:52.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd979d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4069920 con 0x7f7fd4142ea0 2023-10-08T02:31:52.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd8f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7fd406c0f0 0x7f7fd406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd8f9c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7fd406c0f0 0x7f7fd406a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42964/0 (socket says 172.21.15.119:42964) 2023-10-08T02:31:52.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd8f9c700 1 -- 172.21.15.119:0/13376966 learned_addr learned my addr 172.21.15.119:0/13376966 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:52.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.705+0000 7f7fd3fff700 1 --2- 172.21.15.119:0/13376966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd8f9c700 1 -- 172.21.15.119:0/13376966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 msgr2=0x7f7fd40b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:52.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd2ffd700 1 -- 172.21.15.119:0/13376966 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344327437 0 0) 0x7f7fd40b40f0 con 0x7f7fd40ad870 2023-10-08T02:31:52.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd8f9c700 1 --2- 172.21.15.119:0/13376966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd8f9c700 1 -- 172.21.15.119:0/13376966 >> v1:172.21.15.184:6789/0 conn(0x7f7fd40ad870 legacy=0x7f7fd406b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd8f9c700 1 -- 172.21.15.119:0/13376966 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4069c00 con 0x7f7fd406c0f0 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd3fff700 1 --2- 172.21.15.119:0/13376966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd8f9c700 1 --2- 172.21.15.119:0/13376966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd406c0f0 0x7f7fd406a120 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009f20 tx=0x7f7fcc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92d33a96080c1382 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd2ffd700 1 -- 172.21.15.119:0/13376966 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc00e070 con 0x7f7fd406c0f0 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd2ffd700 1 -- 172.21.15.119:0/13376966 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc00b9d0 con 0x7f7fd406c0f0 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.706+0000 7f7fd2ffd700 1 -- 172.21.15.119:0/13376966 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0092e0 con 0x7f7fd406c0f0 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.707+0000 7f7fd979d700 1 -- 172.21.15.119:0/13376966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd406c0f0 msgr2=0x7f7fd406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.707+0000 7f7fd979d700 1 --2- 172.21.15.119:0/13376966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd406c0f0 0x7f7fd406a120 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc009f20 tx=0x7f7fcc009840 comp rx=0 tx=0).stop 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.707+0000 7f7fd979d700 1 -- 172.21.15.119:0/13376966 shutdown_connections 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.707+0000 7f7fd979d700 1 --2- 172.21.15.119:0/13376966 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd406c0f0 0x7f7fd406a120 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.707+0000 7f7fd979d700 1 --2- 172.21.15.119:0/13376966 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.707+0000 7f7fd979d700 1 -- 172.21.15.119:0/13376966 >> 172.21.15.119:0/13376966 conn(0x7f7fd40ce430 msgr2=0x7f7fd40ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:52.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.707+0000 7f7fd979d700 1 -- 172.21.15.119:0/13376966 shutdown_connections 2023-10-08T02:31:52.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.707+0000 7f7fd979d700 1 -- 172.21.15.119:0/13376966 wait complete. 2023-10-08T02:31:52.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.708+0000 7f7fd979d700 1 Processor -- start 2023-10-08T02:31:52.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.709+0000 7f7fd979d700 1 -- start start 2023-10-08T02:31:52.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.709+0000 7f7fd979d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd406c0f0 0x7f7fd41401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.709+0000 7f7fd979d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40acc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.709+0000 7f7fd979d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4159ea0 0x7f7fd40afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.709+0000 7f7fd979d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd41406e0 con 0x7f7fd4159ea0 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.709+0000 7f7fd979d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4169e60 con 0x7f7fd406c0f0 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.709+0000 7f7fd979d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd416a140 con 0x7f7fd4142ea0 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4159ea0 0x7f7fd40afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4159ea0 0x7f7fd40afee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42970/0 (socket says 172.21.15.119:42970) 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 -- 172.21.15.119:0/2780567046 learned_addr learned my addr 172.21.15.119:0/2780567046 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd3fff700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd406c0f0 0x7f7fd41401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd37fe700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40acc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 -- 172.21.15.119:0/2780567046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 msgr2=0x7f7fd40acc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40acc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 -- 172.21.15.119:0/2780567046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd406c0f0 msgr2=0x7f7fd41401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd406c0f0 0x7f7fd41401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 -- 172.21.15.119:0/2780567046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4009420 con 0x7f7fd4159ea0 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fd8f9c700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4159ea0 0x7f7fd40afee0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f7fd406f0d0 tx=0x7f7fcc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.710+0000 7f7fbbfff700 1 -- 172.21.15.119:0/2780567046 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc008bc0 con 0x7f7fd4159ea0 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.711+0000 7f7fbbfff700 1 -- 172.21.15.119:0/2780567046 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc008d40 con 0x7f7fd4159ea0 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.711+0000 7f7fbbfff700 1 -- 172.21.15.119:0/2780567046 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc0266b0 con 0x7f7fd4159ea0 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.711+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc009ae0 con 0x7f7fd4159ea0 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.711+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd416a700 con 0x7f7fd4159ea0 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.712+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd416a940 con 0x7f7fd4159ea0 2023-10-08T02:31:52.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.713+0000 7f7fbbfff700 1 -- 172.21.15.119:0/2780567046 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f7fcc003040 con 0x7f7fd4159ea0 2023-10-08T02:31:52.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.713+0000 7f7fbbfff700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7fc00670e0 0x7f7fc0069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:52.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.713+0000 7f7fd3fff700 1 -- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7fc00670e0 msgr2=0x7f7fc0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:52.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.713+0000 7f7fbbfff700 1 -- 172.21.15.119:0/2780567046 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f7fcc0865b0 con 0x7f7fd4159ea0 2023-10-08T02:31:52.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.714+0000 7f7fd3fff700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7fc00670e0 0x7f7fc0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:52.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.717+0000 7f7fbbfff700 1 -- 172.21.15.119:0/2780567046 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fd416a940 con 0x7f7fd4159ea0 2023-10-08T02:31:52.822 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:52.821+0000 7f8a1666a180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-08T02:31:52.859 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:52.859+0000 7f4549a91180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:31:52.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.915+0000 7f7fd3fff700 1 -- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7fc00670e0 msgr2=0x7f7fc0069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:52.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.915+0000 7f7fd3fff700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7fc00670e0 0x7f7fc0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:52.953 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:52.952+0000 7f4549a91180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:31:52.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.962+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7fd416a940 con 0x7f7fd4159ea0 2023-10-08T02:31:52.963 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-10-08T02:31:52.962+0000 7f27dddf1700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-08T02:31:52.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.968+0000 7f7fbbfff700 1 -- 172.21.15.119:0/2780567046 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v96) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7fd416a940 con 0x7f7fd4159ea0 2023-10-08T02:31:52.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7fc00670e0 msgr2=0x7f7fc0069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:52.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7fc00670e0 0x7f7fc0069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4159ea0 msgr2=0x7f7fd40afee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:52.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4159ea0 0x7f7fd40afee0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f7fd406f0d0 tx=0x7f7fcc018000 comp rx=0 tx=0).stop 2023-10-08T02:31:52.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 shutdown_connections 2023-10-08T02:31:52.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7fc00670e0 0x7f7fc0069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7fd4159ea0 0x7f7fd40afee0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7fd406c0f0 0x7f7fd41401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 --2- 172.21.15.119:0/2780567046 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7fd4142ea0 0x7f7fd40acc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:52.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 >> 172.21.15.119:0/2780567046 conn(0x7f7fd40ce430 msgr2=0x7f7fd406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:52.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 shutdown_connections 2023-10-08T02:31:52.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:52.972+0000 7f7fd979d700 1 -- 172.21.15.119:0/2780567046 wait complete. 2023-10-08T02:31:52.974 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-08T02:31:52.975 INFO:tasks.ceph.mds.a.smithi119.stderr: -2437> 2023-10-08T02:31:32.369+0000 7fce2cff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-08T02:31:53.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-08T02:31:53.027 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-10-08T02:31:53.113 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:53.112+0000 7f4549a91180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:31:53.121 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:53.120+0000 7f8a1666a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-08T02:31:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.192+0000 7f8775d9d700 1 Processor -- start 2023-10-08T02:31:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.192+0000 7f8775d9d700 1 -- start start 2023-10-08T02:31:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.193+0000 7f8775d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877015a3b0 0x7f877015a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.193+0000 7f8775d9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f87700aef00 0x7f877015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f8775d9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f877006e470 0x7f877006e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f8775d9d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877006ee30 con 0x7f877006e470 2023-10-08T02:31:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f8775d9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87700aea70 con 0x7f87700aef00 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f8775d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87700ae480 con 0x7f877015a3b0 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f877559c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f877006e470 0x7f877006e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f877559c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f877006e470 0x7f877006e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42996/0 (socket says 172.21.15.119:42996) 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f877559c700 1 -- 172.21.15.119:0/3049443172 learned_addr learned my addr 172.21.15.119:0/3049443172 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f876ffff700 1 --2- 172.21.15.119:0/3049443172 >> v2:172.21.15.184:3300/0 conn(0x7f87700aef00 0x7f877015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f8774d9b700 1 --2- 172.21.15.119:0/3049443172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877015a3b0 0x7f877015a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f877559c700 1 -- 172.21.15.119:0/3049443172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877015a3b0 msgr2=0x7f877015a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f877559c700 1 --2- 172.21.15.119:0/3049443172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877015a3b0 0x7f877015a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f877559c700 1 -- 172.21.15.119:0/3049443172 >> v2:172.21.15.184:3300/0 conn(0x7f87700aef00 msgr2=0x7f877015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.194+0000 7f877559c700 1 --2- 172.21.15.119:0/3049443172 >> v2:172.21.15.184:3300/0 conn(0x7f87700aef00 0x7f877015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.195+0000 7f877559c700 1 -- 172.21.15.119:0/3049443172 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877015acc0 con 0x7f877006e470 2023-10-08T02:31:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.195+0000 7f876ffff700 1 --2- 172.21.15.119:0/3049443172 >> v2:172.21.15.184:3300/0 conn(0x7f87700aef00 0x7f877015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.195+0000 7f877559c700 1 --2- 172.21.15.119:0/3049443172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877006e470 0x7f877006e8f0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f877006f500 tx=0x7f8768009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b95f59ab608b459 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.195+0000 7f876f7fe700 1 -- 172.21.15.119:0/3049443172 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876800e070 con 0x7f877006e470 2023-10-08T02:31:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.195+0000 7f876f7fe700 1 -- 172.21.15.119:0/3049443172 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876800b970 con 0x7f877006e470 2023-10-08T02:31:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.195+0000 7f876f7fe700 1 -- 172.21.15.119:0/3049443172 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87680092e0 con 0x7f877006e470 2023-10-08T02:31:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 -- 172.21.15.119:0/3049443172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877006e470 msgr2=0x7f877006e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 --2- 172.21.15.119:0/3049443172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877006e470 0x7f877006e8f0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f877006f500 tx=0x7f8768009840 comp rx=0 tx=0).stop 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 -- 172.21.15.119:0/3049443172 shutdown_connections 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 --2- 172.21.15.119:0/3049443172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877006e470 0x7f877006e8f0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 --2- 172.21.15.119:0/3049443172 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877015a3b0 0x7f877015a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 --2- 172.21.15.119:0/3049443172 >> v2:172.21.15.184:3300/0 conn(0x7f87700aef00 0x7f877015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 -- 172.21.15.119:0/3049443172 >> 172.21.15.119:0/3049443172 conn(0x7f87700c9f70 msgr2=0x7f877006dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 -- 172.21.15.119:0/3049443172 shutdown_connections 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.196+0000 7f8775d9d700 1 -- 172.21.15.119:0/3049443172 wait complete. 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.198+0000 7f8775d9d700 1 Processor -- start 2023-10-08T02:31:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.198+0000 7f8775d9d700 1 -- start start 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.199+0000 7f8775d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877006e470 0x7f8770155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.199+0000 7f8775d9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87700aef00 0x7f877016a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.199+0000 7f8775d9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877015a3b0 0x7f8770156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.199+0000 7f8775d9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877016ace0 con 0x7f877015a3b0 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.199+0000 7f8775d9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8770159810 con 0x7f87700aef00 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.199+0000 7f8775d9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87700c6b10 con 0x7f877006e470 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877015a3b0 0x7f8770156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f8774d9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877006e470 0x7f8770155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:53.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877015a3b0 0x7f8770156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43006/0 (socket says 172.21.15.119:43006) 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 -- 172.21.15.119:0/1141178763 learned_addr learned my addr 172.21.15.119:0/1141178763 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f8774d9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877006e470 0x7f8770155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:44832/0 (socket says 172.21.15.119:44832) 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f876ffff700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87700aef00 0x7f877016a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 -- 172.21.15.119:0/1141178763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877006e470 msgr2=0x7f8770155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877006e470 0x7f8770155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 -- 172.21.15.119:0/1141178763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87700aef00 msgr2=0x7f877016a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87700aef00 0x7f877016a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 -- 172.21.15.119:0/1141178763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c009420 con 0x7f877015a3b0 2023-10-08T02:31:53.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.200+0000 7f877559c700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877015a3b0 0x7f8770156580 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f8770064280 tx=0x7f8768018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:53.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.201+0000 7f876cff9700 1 -- 172.21.15.119:0/1141178763 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768008ad0 con 0x7f877015a3b0 2023-10-08T02:31:53.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.201+0000 7f876cff9700 1 -- 172.21.15.119:0/1141178763 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8768008c50 con 0x7f877015a3b0 2023-10-08T02:31:53.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.201+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8768009ae0 con 0x7f877015a3b0 2023-10-08T02:31:53.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.201+0000 7f876cff9700 1 -- 172.21.15.119:0/1141178763 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8768026460 con 0x7f877015a3b0 2023-10-08T02:31:53.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.201+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8770158ee0 con 0x7f877015a3b0 2023-10-08T02:31:53.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.202+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87701599a0 con 0x7f877015a3b0 2023-10-08T02:31:53.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.203+0000 7f876cff9700 1 -- 172.21.15.119:0/1141178763 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 118) v1 ==== 84899+0+0 (secure 0 0 0) 0x7f8768003040 con 0x7f877015a3b0 2023-10-08T02:31:53.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.204+0000 7f876cff9700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 0x7f8754069590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:53.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.204+0000 7f8774d9b700 1 -- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 msgr2=0x7f8754069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:53.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.204+0000 7f876cff9700 1 -- 172.21.15.119:0/1141178763 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 6569+0+0 (secure 0 0 0) 0x7f8768086be0 con 0x7f877015a3b0 2023-10-08T02:31:53.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.204+0000 7f8774d9b700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 0x7f8754069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-08T02:31:53.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.208+0000 7f876cff9700 1 -- 172.21.15.119:0/1141178763 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8768053820 con 0x7f877015a3b0 2023-10-08T02:31:53.217 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:53.216+0000 7f4549a91180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:31:53.221 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:53.221+0000 7f8a1666a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-08T02:31:53.383 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:53.383+0000 7f8a1666a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-08T02:31:53.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.405+0000 7f8774d9b700 1 -- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 msgr2=0x7f8754069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:53.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.405+0000 7f8774d9b700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 0x7f8754069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-08T02:31:53.406 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:53.405+0000 7f4549a91180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:31:53.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.455+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f877015dd90 con 0x7f877015a3b0 2023-10-08T02:31:53.492 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:53.491+0000 7f8a1666a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-08T02:31:53.498 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:53.498+0000 7f4549a91180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:31:53.590 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:53.589+0000 7f4549a91180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:31:53.684 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:53.683+0000 7f8a1666a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-08T02:31:53.687 INFO:tasks.ceph.mgr.x.smithi184.stderr:2023-10-08T02:31:53.686+0000 7f4549a91180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:31:53.776 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:53.775+0000 7f8a1666a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-08T02:31:53.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.806+0000 7f8774d9b700 1 -- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 msgr2=0x7f8754069590 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.119:6836/134108 2023-10-08T02:31:53.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.806+0000 7f8774d9b700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 0x7f8754069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-10-08T02:31:53.869 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:53.868+0000 7f8a1666a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-08T02:31:53.964 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-10-08T02:31:53.964+0000 7f8a1666a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-08T02:31:53.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.968+0000 7f876cff9700 1 -- 172.21.15.119:0/1141178763 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v97) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8768056e40 con 0x7f877015a3b0 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 msgr2=0x7f8754069590 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 0x7f8754069590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877015a3b0 msgr2=0x7f8770156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877015a3b0 0x7f8770156580 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f8770064280 tx=0x7f8768018000 comp rx=0 tx=0).stop 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 shutdown_connections 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f87540670e0 0x7f8754069590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f877006e470 0x7f8770155c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f877015a3b0 0x7f8770156580 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 --2- 172.21.15.119:0/1141178763 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f87700aef00 0x7f877016a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:53.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 >> 172.21.15.119:0/1141178763 conn(0x7f87700c9f70 msgr2=0x7f877015aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:53.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 shutdown_connections 2023-10-08T02:31:53.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:53.971+0000 7f8775d9d700 1 -- 172.21.15.119:0/1141178763 wait complete. 2023-10-08T02:31:53.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-08T02:31:54.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.224+0000 7f93be1ff700 1 Processor -- start 2023-10-08T02:31:54.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.224+0000 7f93be1ff700 1 -- start start 2023-10-08T02:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.225+0000 7f93be1ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93b8069a50 0x7f93b80c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.225+0000 7f93be1ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f93b80c8490 0x7f93b815a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.225+0000 7f93be1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 0x7f93b806e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.225+0000 7f93be1ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b80ac030 con 0x7f93b8069a50 2023-10-08T02:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.225+0000 7f93be1ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b80ab820 con 0x7f93b80c8490 2023-10-08T02:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.225+0000 7f93be1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b806dbb0 con 0x7f93b8069e20 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93b8069a50 0x7f93b80c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93b8069a50 0x7f93b80c6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43096/0 (socket says 172.21.15.119:43096) 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 -- 172.21.15.119:0/2976899143 learned_addr learned my addr 172.21.15.119:0/2976899143 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b7fff700 1 --2- 172.21.15.119:0/2976899143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 0x7f93b806e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b6ffd700 1 --2- 172.21.15.119:0/2976899143 >> v2:172.21.15.184:3300/0 conn(0x7f93b80c8490 0x7f93b815a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 -- 172.21.15.119:0/2976899143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 msgr2=0x7f93b806e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 --2- 172.21.15.119:0/2976899143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 0x7f93b806e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 -- 172.21.15.119:0/2976899143 >> v2:172.21.15.184:3300/0 conn(0x7f93b80c8490 msgr2=0x7f93b815a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 --2- 172.21.15.119:0/2976899143 >> v2:172.21.15.184:3300/0 conn(0x7f93b80c8490 0x7f93b815a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 -- 172.21.15.119:0/2976899143 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b815a9d0 con 0x7f93b8069a50 2023-10-08T02:31:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b77fe700 1 --2- 172.21.15.119:0/2976899143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8069a50 0x7f93b80c6ac0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f93b806b920 tx=0x7f93a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ef38042cd199d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b67fc700 1 -- 172.21.15.119:0/2976899143 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800b640 con 0x7f93b8069a50 2023-10-08T02:31:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.226+0000 7f93b67fc700 1 -- 172.21.15.119:0/2976899143 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a800b7a0 con 0x7f93b8069a50 2023-10-08T02:31:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93b67fc700 1 -- 172.21.15.119:0/2976899143 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800bb40 con 0x7f93b8069a50 2023-10-08T02:31:54.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 -- 172.21.15.119:0/2976899143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8069a50 msgr2=0x7f93b80c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:54.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 --2- 172.21.15.119:0/2976899143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8069a50 0x7f93b80c6ac0 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f93b806b920 tx=0x7f93a8009250 comp rx=0 tx=0).stop 2023-10-08T02:31:54.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 -- 172.21.15.119:0/2976899143 shutdown_connections 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 --2- 172.21.15.119:0/2976899143 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 0x7f93b806e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 --2- 172.21.15.119:0/2976899143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b8069a50 0x7f93b80c6ac0 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 --2- 172.21.15.119:0/2976899143 >> v2:172.21.15.184:3300/0 conn(0x7f93b80c8490 0x7f93b815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 -- 172.21.15.119:0/2976899143 >> 172.21.15.119:0/2976899143 conn(0x7f93b80c8f90 msgr2=0x7f93b80c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 -- 172.21.15.119:0/2976899143 shutdown_connections 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.227+0000 7f93be1ff700 1 -- 172.21.15.119:0/2976899143 wait complete. 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.229+0000 7f93be1ff700 1 Processor -- start 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.229+0000 7f93be1ff700 1 -- start start 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93be1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93b8069a50 0x7f93b8152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:54.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93be1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 0x7f93b80c1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93be1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b80c8490 0x7f93b8159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93be1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b81545d0 con 0x7f93b80c8490 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93be1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b80c2480 con 0x7f93b8069a50 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93be1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b80c2760 con 0x7f93b8069e20 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b80c8490 0x7f93b8159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b7fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b80c8490 0x7f93b8159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43100/0 (socket says 172.21.15.119:43100) 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b7fff700 1 -- 172.21.15.119:0/3017707623 learned_addr learned my addr 172.21.15.119:0/3017707623 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b77fe700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93b8069a50 0x7f93b8152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b6ffd700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 0x7f93b80c1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b7fff700 1 -- 172.21.15.119:0/3017707623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 msgr2=0x7f93b80c1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:54.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b7fff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 0x7f93b80c1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b7fff700 1 -- 172.21.15.119:0/3017707623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93b8069a50 msgr2=0x7f93b8152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b7fff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93b8069a50 0x7f93b8152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.230+0000 7f93b7fff700 1 -- 172.21.15.119:0/3017707623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a80094f0 con 0x7f93b80c8490 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.231+0000 7f93b7fff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b80c8490 0x7f93b8159b80 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009e90 tx=0x7f93b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.231+0000 7f939f7fe700 1 -- 172.21.15.119:0/3017707623 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000cb70 con 0x7f93b80c8490 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.231+0000 7f939f7fe700 1 -- 172.21.15.119:0/3017707623 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b000f040 con 0x7f93b80c8490 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.231+0000 7f939f7fe700 1 -- 172.21.15.119:0/3017707623 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0014480 con 0x7f93b80c8490 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.231+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b8152150 con 0x7f93b80c8490 2023-10-08T02:31:54.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.231+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b8152710 con 0x7f93b80c8490 2023-10-08T02:31:54.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.232+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b80718b0 con 0x7f93b80c8490 2023-10-08T02:31:54.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.233+0000 7f939f7fe700 1 -- 172.21.15.119:0/3017707623 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 119) v1 ==== 84451+0+0 (secure 0 0 0) 0x7f93b0014830 con 0x7f93b80c8490 2023-10-08T02:31:54.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.234+0000 7f939f7fe700 1 -- 172.21.15.119:0/3017707623 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 6870+0+0 (secure 0 0 0) 0x7f93b0084ee0 con 0x7f93b80c8490 2023-10-08T02:31:54.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.237+0000 7f939f7fe700 1 -- 172.21.15.119:0/3017707623 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f93b0052080 con 0x7f93b80c8490 2023-10-08T02:31:54.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.427+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f93b8068e00 con 0x7f93b80c8490 2023-10-08T02:31:54.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.982+0000 7f939f7fe700 1 -- 172.21.15.119:0/3017707623 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 120) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f93b0051b30 con 0x7f93b80c8490 2023-10-08T02:31:54.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.982+0000 7f939f7fe700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93a406cbb0 0x7f93a406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:54.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.984+0000 7f93b77fe700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93a406cbb0 0x7f93a406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:54.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:54.987+0000 7f93b77fe700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93a406cbb0 0x7f93a406ef90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93b806b920 tx=0x7f93a8017000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:55.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.365+0000 7f939f7fe700 1 -- 172.21.15.119:0/3017707623 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v210) v1 ==== 181+0+0 (secure 0 0 0) 0x7f93b007da70 con 0x7f93b80c8490 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93a406cbb0 msgr2=0x7f93a406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93a406cbb0 0x7f93a406ef90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f93b806b920 tx=0x7f93a8017000 comp rx=0 tx=0).stop 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b80c8490 msgr2=0x7f93b8159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b80c8490 0x7f93b8159b80 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009e90 tx=0x7f93b000b040 comp rx=0 tx=0).stop 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 shutdown_connections 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f93a406cbb0 0x7f93a406ef90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93b80c8490 0x7f93b8159b80 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f93b8069a50 0x7f93b8152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 --2- 172.21.15.119:0/3017707623 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f93b8069e20 0x7f93b80c1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 >> 172.21.15.119:0/3017707623 conn(0x7f93b80c8f90 msgr2=0x7f93b80b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:55.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 shutdown_connections 2023-10-08T02:31:55.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.371+0000 7f93be1ff700 1 -- 172.21.15.119:0/3017707623 wait complete. 2023-10-08T02:31:55.373 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-10-08T02:31:55.392 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-08T02:31:55.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.580+0000 7f4a4bfff700 1 Processor -- start 2023-10-08T02:31:55.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.580+0000 7f4a4bfff700 1 -- start start 2023-10-08T02:31:55.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.581+0000 7f4a4bfff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4a441570c0 0x7f4a440c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:55.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.581+0000 7f4a4bfff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a440ac030 con 0x7f4a440c8490 2023-10-08T02:31:55.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.581+0000 7f4a4bfff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a440ab820 con 0x7f4a44069a50 2023-10-08T02:31:55.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.581+0000 7f4a4bfff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4406dbb0 con 0x7f4a441570c0 2023-10-08T02:31:55.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a4959a700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4a440c8490 0x7f4a441569b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52076/0 (socket says 172.21.15.119:52076) 2023-10-08T02:31:55.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a4959a700 1 -- 172.21.15.119:0/169123125 learned_addr learned my addr 172.21.15.119:0/169123125 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554253550 0 0) 0x7f4a34001110 con 0x7f4a440c8490 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a4a59c700 1 --2- 172.21.15.119:0/169123125 >> v2:172.21.15.184:3300/0 conn(0x7f4a441570c0 0x7f4a440c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a30003570 con 0x7f4a440c8490 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3155442501 0 0) 0x7f4a30003570 con 0x7f4a440c8490 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 >> v2:172.21.15.184:3300/0 conn(0x7f4a441570c0 msgr2=0x7f4a440c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a48d99700 1 --2- 172.21.15.119:0/169123125 >> v2:172.21.15.184:3300/0 conn(0x7f4a441570c0 0x7f4a440c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 >> v1:172.21.15.184:6789/0 conn(0x7f4a44069a50 legacy=0x7f4a4415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440aba70 con 0x7f4a440c8490 2023-10-08T02:31:55.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.582+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f4a34001600 con 0x7f4a440c8490 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 >> v1:172.21.15.119:6789/0 conn(0x7f4a440c8490 legacy=0x7f4a441569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a48d99700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a30003ab0 0x7f4a30003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a48d99700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30004810 0x7f4a30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a48d99700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a30005210 0x7f4a3000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4406dbb0 con 0x7f4a30003ab0 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000c1b0 con 0x7f4a30004810 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000c370 con 0x7f4a30005210 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a49d9b700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a30003ab0 0x7f4a30003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a4959a700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30004810 0x7f4a30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a4a59c700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a30005210 0x7f4a3000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a49d9b700 1 -- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a30005210 msgr2=0x7f4a3000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:55.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a49d9b700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a30005210 0x7f4a3000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a49d9b700 1 -- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30004810 msgr2=0x7f4a30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a49d9b700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30004810 0x7f4a30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a49d9b700 1 -- 172.21.15.119:0/169123125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30003570 con 0x7f4a30003ab0 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a4a59c700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a30005210 0x7f4a3000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a4959a700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30004810 0x7f4a30004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a4a59c700 1 -- 172.21.15.119:0/169123125 reap_dead start 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.583+0000 7f4a49d9b700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a30003ab0 0x7f4a30003ec0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f4a400018b0 tx=0x7f4a40007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6428ccc55495aeed server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40012420 con 0x7f4a30003ab0 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a400125a0 con 0x7f4a30003ab0 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a48d99700 1 -- 172.21.15.119:0/169123125 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40012960 con 0x7f4a30003ab0 2023-10-08T02:31:55.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a4bfff700 1 -- 172.21.15.119:0/169123125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a30003ab0 msgr2=0x7f4a30003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:55.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a4bfff700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a30003ab0 0x7f4a30003ec0 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f4a400018b0 tx=0x7f4a40007d30 comp rx=0 tx=0).stop 2023-10-08T02:31:55.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a4bfff700 1 -- 172.21.15.119:0/169123125 shutdown_connections 2023-10-08T02:31:55.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a4bfff700 1 --2- 172.21.15.119:0/169123125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a30003ab0 0x7f4a30003ec0 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a4bfff700 1 -- 172.21.15.119:0/169123125 >> 172.21.15.119:0/169123125 conn(0x7f4a440c8f90 msgr2=0x7f4a440c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:55.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a4bfff700 1 -- 172.21.15.119:0/169123125 shutdown_connections 2023-10-08T02:31:55.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.584+0000 7f4a4bfff700 1 -- 172.21.15.119:0/169123125 wait complete. 2023-10-08T02:31:55.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.586+0000 7f4a4bfff700 1 Processor -- start 2023-10-08T02:31:55.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.586+0000 7f4a4bfff700 1 -- start start 2023-10-08T02:31:55.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4bfff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30003ab0 0x7f4a441478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:55.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4bfff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a440bfad0 0x7f4a440bff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:55.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4bfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a440adb60 0x7f4a440adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:55.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4bfff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440ae510 con 0x7f4a440adb60 2023-10-08T02:31:55.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4bfff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c0570 con 0x7f4a30003ab0 2023-10-08T02:31:55.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4bfff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c0850 con 0x7f4a440bfad0 2023-10-08T02:31:55.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a440adb60 0x7f4a440adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:55.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4a59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a440adb60 0x7f4a440adfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50376/0 (socket says 172.21.15.119:50376) 2023-10-08T02:31:55.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4a59c700 1 -- 172.21.15.119:0/3341072204 learned_addr learned my addr 172.21.15.119:0/3341072204 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4959a700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a440bfad0 0x7f4a440bff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a49d9b700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30003ab0 0x7f4a441478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4a59c700 1 -- 172.21.15.119:0/3341072204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a440bfad0 msgr2=0x7f4a440bff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4a59c700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a440bfad0 0x7f4a440bff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4a59c700 1 -- 172.21.15.119:0/3341072204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30003ab0 msgr2=0x7f4a441478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4a59c700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30003ab0 0x7f4a441478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.587+0000 7f4a4a59c700 1 -- 172.21.15.119:0/3341072204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440c0b30 con 0x7f4a440adb60 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.588+0000 7f4a4a59c700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a440adb60 0x7f4a440adfd0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f4a440c34c0 tx=0x7f4a3c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:55.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.588+0000 7f4a39ffb700 1 -- 172.21.15.119:0/3341072204 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00fbd0 con 0x7f4a440adb60 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.588+0000 7f4a39ffb700 1 -- 172.21.15.119:0/3341072204 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c00fd30 con 0x7f4a440adb60 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.588+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a440c0e10 con 0x7f4a440adb60 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.588+0000 7f4a39ffb700 1 -- 172.21.15.119:0/3341072204 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c016650 con 0x7f4a440adb60 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.588+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a440c2ff0 con 0x7f4a440adb60 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.589+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a440718b0 con 0x7f4a440adb60 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.590+0000 7f4a39ffb700 1 -- 172.21.15.119:0/3341072204 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 120) v1 ==== 84578+0+0 (secure 0 0 0) 0x7f4a3c016a00 con 0x7f4a440adb60 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.591+0000 7f4a39ffb700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4a28066d80 0x7f4a28069230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.591+0000 7f4a39ffb700 1 -- 172.21.15.119:0/3341072204 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6432+0+0 (secure 0 0 0) 0x7f4a3c084e60 con 0x7f4a440adb60 2023-10-08T02:31:55.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.591+0000 7f4a49d9b700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4a28066d80 0x7f4a28069230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.591+0000 7f4a49d9b700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4a28066d80 0x7f4a28069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a40001010 tx=0x7f4a40015000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:55.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.595+0000 7f4a39ffb700 1 -- 172.21.15.119:0/3341072204 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4a3c0520b0 con 0x7f4a440adb60 2023-10-08T02:31:55.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:55.786+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a440abd50 con 0x7f4a440adb60 2023-10-08T02:31:56.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.366+0000 7f4a39ffb700 1 -- 172.21.15.119:0/3341072204 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4a3c088030 con 0x7f4a440adb60 2023-10-08T02:31:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.373+0000 7f4a39ffb700 1 -- 172.21.15.119:0/3341072204 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v211) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4a3c0520b0 con 0x7f4a440adb60 2023-10-08T02:31:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4a28066d80 msgr2=0x7f4a28069230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4a28066d80 0x7f4a28069230 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4a40001010 tx=0x7f4a40015000 comp rx=0 tx=0).stop 2023-10-08T02:31:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a440adb60 msgr2=0x7f4a440adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a440adb60 0x7f4a440adfd0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f4a440c34c0 tx=0x7f4a3c00b040 comp rx=0 tx=0).stop 2023-10-08T02:31:56.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 shutdown_connections 2023-10-08T02:31:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4a28066d80 0x7f4a28069230 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a440adb60 0x7f4a440adfd0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4a30003ab0 0x7f4a441478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 --2- 172.21.15.119:0/3341072204 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4a440bfad0 0x7f4a440bff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.378+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 >> 172.21.15.119:0/3341072204 conn(0x7f4a440c8f90 msgr2=0x7f4a44068390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:56.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.379+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 shutdown_connections 2023-10-08T02:31:56.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.379+0000 7f4a4bfff700 1 -- 172.21.15.119:0/3341072204 wait complete. 2023-10-08T02:31:56.383 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-10-08T02:31:56.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-08T02:31:56.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.583+0000 7f6d089fe700 1 Processor -- start 2023-10-08T02:31:56.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.583+0000 7f6d089fe700 1 -- start start 2023-10-08T02:31:56.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.584+0000 7f6d089fe700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d040b16d0 0x7f6d040b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:56.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.584+0000 7f6d089fe700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6d040b33b0 0x7f6d040b37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:56.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.584+0000 7f6d089fe700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d040b2080 con 0x7f6d040b4e70 2023-10-08T02:31:56.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.584+0000 7f6d089fe700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d040cfb00 con 0x7f6d040b16d0 2023-10-08T02:31:56.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.584+0000 7f6d089fe700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d040b5c40 con 0x7f6d040b33b0 2023-10-08T02:31:56.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.584+0000 7f6d0259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d040b16d0 0x7f6d040b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:56.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.584+0000 7f6d0259c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d040b16d0 0x7f6d040b1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50388/0 (socket says 172.21.15.119:50388) 2023-10-08T02:31:56.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.584+0000 7f6d0259c700 1 -- 172.21.15.119:0/4010179481 learned_addr learned my addr 172.21.15.119:0/4010179481 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d0259c700 1 -- 172.21.15.119:0/4010179481 >> v2:172.21.15.184:3300/0 conn(0x7f6d040b33b0 msgr2=0x7f6d040b37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d0359e700 1 --2- 172.21.15.119:0/4010179481 >> v2:172.21.15.184:3300/0 conn(0x7f6d040b33b0 0x7f6d040b37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d0259c700 1 --2- 172.21.15.119:0/4010179481 >> v2:172.21.15.184:3300/0 conn(0x7f6d040b33b0 0x7f6d040b37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d0259c700 1 -- 172.21.15.119:0/4010179481 >> v1:172.21.15.184:6789/0 conn(0x7f6d040b4e70 legacy=0x7f6d040ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d0259c700 1 -- 172.21.15.119:0/4010179481 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d040b5f20 con 0x7f6d040b16d0 2023-10-08T02:31:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d0359e700 1 --2- 172.21.15.119:0/4010179481 >> v2:172.21.15.184:3300/0 conn(0x7f6d040b33b0 0x7f6d040b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d0259c700 1 --2- 172.21.15.119:0/4010179481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b16d0 0x7f6d040b1b40 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f6d04071380 tx=0x7f6cf8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e9a3ad024da1df server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:56.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d01d9b700 1 -- 172.21.15.119:0/4010179481 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800b820 con 0x7f6d040b16d0 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d01d9b700 1 -- 172.21.15.119:0/4010179481 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf800b980 con 0x7f6d040b16d0 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.585+0000 7f6d01d9b700 1 -- 172.21.15.119:0/4010179481 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800bd40 con 0x7f6d040b16d0 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.586+0000 7f6d089fe700 1 -- 172.21.15.119:0/4010179481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b16d0 msgr2=0x7f6d040b1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.586+0000 7f6d089fe700 1 --2- 172.21.15.119:0/4010179481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b16d0 0x7f6d040b1b40 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f6d04071380 tx=0x7f6cf8009250 comp rx=0 tx=0).stop 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.586+0000 7f6d089fe700 1 -- 172.21.15.119:0/4010179481 shutdown_connections 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.586+0000 7f6d089fe700 1 --2- 172.21.15.119:0/4010179481 >> v2:172.21.15.184:3300/0 conn(0x7f6d040b33b0 0x7f6d040b37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.586+0000 7f6d089fe700 1 --2- 172.21.15.119:0/4010179481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b16d0 0x7f6d040b1b40 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.586+0000 7f6d089fe700 1 -- 172.21.15.119:0/4010179481 >> 172.21.15.119:0/4010179481 conn(0x7f6d040d10c0 msgr2=0x7f6d040affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.586+0000 7f6d089fe700 1 -- 172.21.15.119:0/4010179481 shutdown_connections 2023-10-08T02:31:56.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.586+0000 7f6d089fe700 1 -- 172.21.15.119:0/4010179481 wait complete. 2023-10-08T02:31:56.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.587+0000 7f6d089fe700 1 Processor -- start 2023-10-08T02:31:56.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.587+0000 7f6d089fe700 1 -- start start 2023-10-08T02:31:56.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.588+0000 7f6d089fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d040b16d0 0x7f6d0416b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:56.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.588+0000 7f6d089fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b33b0 0x7f6d040c7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:56.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.588+0000 7f6d089fe700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d040b4e70 0x7f6d04167ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:56.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.588+0000 7f6d089fe700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d040c7fc0 con 0x7f6d040b33b0 2023-10-08T02:31:56.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.588+0000 7f6d089fe700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0416c740 con 0x7f6d040b4e70 2023-10-08T02:31:56.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.588+0000 7f6d089fe700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0416bb30 con 0x7f6d040b16d0 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b33b0 0x7f6d040c7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b33b0 0x7f6d040c7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50390/0 (socket says 172.21.15.119:50390) 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 -- 172.21.15.119:0/1002523323 learned_addr learned my addr 172.21.15.119:0/1002523323 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 -- 172.21.15.119:0/1002523323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d040b16d0 msgr2=0x7f6d0416b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d040b16d0 0x7f6d0416b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 -- 172.21.15.119:0/1002523323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d040b4e70 msgr2=0x7f6d04167ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d040b4e70 0x7f6d04167ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 -- 172.21.15.119:0/1002523323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf80094f0 con 0x7f6d040b33b0 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6d0259c700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b33b0 0x7f6d040c7a80 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f6d04070dd0 tx=0x7f6cf8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.589+0000 7f6ceaffd700 1 -- 172.21.15.119:0/1002523323 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8006ab0 con 0x7f6d040b33b0 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.590+0000 7f6ceaffd700 1 -- 172.21.15.119:0/1002523323 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf8020070 con 0x7f6d040b33b0 2023-10-08T02:31:56.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.590+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0416be10 con 0x7f6d040b33b0 2023-10-08T02:31:56.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.590+0000 7f6ceaffd700 1 -- 172.21.15.119:0/1002523323 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf801b450 con 0x7f6d040b33b0 2023-10-08T02:31:56.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.590+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0416c3d0 con 0x7f6d040b33b0 2023-10-08T02:31:56.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.591+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d04159c40 con 0x7f6d040b33b0 2023-10-08T02:31:56.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.591+0000 7f6ceaffd700 1 -- 172.21.15.119:0/1002523323 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cf8022030 con 0x7f6d040b33b0 2023-10-08T02:31:56.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.592+0000 7f6ceaffd700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cf0066f30 0x7f6cf00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:56.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.592+0000 7f6ceaffd700 1 -- 172.21.15.119:0/1002523323 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f6cf8086520 con 0x7f6d040b33b0 2023-10-08T02:31:56.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.593+0000 7f6d02d9d700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cf0066f30 0x7f6cf00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:56.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.593+0000 7f6d02d9d700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cf0066f30 0x7f6cf00693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4007fd0 tx=0x7f6cf4007b20 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:56.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.596+0000 7f6ceaffd700 1 -- 172.21.15.119:0/1002523323 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6cf80535b0 con 0x7f6d040b33b0 2023-10-08T02:31:56.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.796+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6d04159c40 con 0x7f6d040b33b0 2023-10-08T02:31:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.800+0000 7f6ceaffd700 1 -- 172.21.15.119:0/1002523323 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6cf80092d0 con 0x7f6d040b33b0 2023-10-08T02:31:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cf0066f30 msgr2=0x7f6cf00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cf0066f30 0x7f6cf00693e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4007fd0 tx=0x7f6cf4007b20 comp rx=0 tx=0).stop 2023-10-08T02:31:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b33b0 msgr2=0x7f6d040c7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b33b0 0x7f6d040c7a80 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f6d04070dd0 tx=0x7f6cf8016000 comp rx=0 tx=0).stop 2023-10-08T02:31:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 shutdown_connections 2023-10-08T02:31:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6cf0066f30 0x7f6cf00693e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f6d040b16d0 0x7f6d0416b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6d040b4e70 0x7f6d04167ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 --2- 172.21.15.119:0/1002523323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d040b33b0 0x7f6d040c7a80 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 >> 172.21.15.119:0/1002523323 conn(0x7f6d040d10c0 msgr2=0x7f6d040affb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 shutdown_connections 2023-10-08T02:31:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:56.802+0000 7f6d089fe700 1 -- 172.21.15.119:0/1002523323 wait complete. 2023-10-08T02:31:56.821 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-10-08T02:31:57.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.003+0000 7f378ebff700 1 Processor -- start 2023-10-08T02:31:57.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.003+0000 7f378ebff700 1 -- start start 2023-10-08T02:31:57.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.004+0000 7f378ebff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37880ccee0 0x7f37881561d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.004+0000 7f378ebff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f37880c87c0 0x7f3788156710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.004+0000 7f378ebff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f378815b0d0 con 0x7f37880c83f0 2023-10-08T02:31:57.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.004+0000 7f378ebff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3788156c50 con 0x7f37880c87c0 2023-10-08T02:31:57.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.004+0000 7f378ebff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378815a4c0 con 0x7f37880ccee0 2023-10-08T02:31:57.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f378c99b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f37880c83f0 0x7f37880c7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52090/0 (socket says 172.21.15.119:52090) 2023-10-08T02:31:57.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f378c99b700 1 -- 172.21.15.119:0/3961981114 learned_addr learned my addr 172.21.15.119:0/3961981114 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:57.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f378d19c700 1 --2- 172.21.15.119:0/3961981114 >> v2:172.21.15.184:3300/0 conn(0x7f37880c87c0 0x7f3788156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f3787fff700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37880ccee0 0x7f37881561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899634109 0 0) 0x7f378815b0d0 con 0x7f37880c83f0 2023-10-08T02:31:57.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3778003570 con 0x7f37880c83f0 2023-10-08T02:31:57.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3347992098 0 0) 0x7f3778003570 con 0x7f37880c83f0 2023-10-08T02:31:57.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37880ccee0 msgr2=0x7f37881561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:57.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37880ccee0 0x7f37881561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 >> v2:172.21.15.184:3300/0 conn(0x7f37880c87c0 msgr2=0x7f3788156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:57.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 --2- 172.21.15.119:0/3961981114 >> v2:172.21.15.184:3300/0 conn(0x7f37880c87c0 0x7f3788156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378815a7a0 con 0x7f37880c83f0 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f377c001470 con 0x7f37880c83f0 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.005+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 >> v1:172.21.15.119:6789/0 conn(0x7f37880c83f0 legacy=0x7f37880c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f37877fe700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37880ccee0 0x7f3778003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f37877fe700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3778003fe0 0x7f3778004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f37877fe700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3778004d50 0x7f37780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788156c50 con 0x7f3778004d50 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378815a4c0 con 0x7f3778003fe0 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377800dca0 con 0x7f37880ccee0 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f378d19c700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3778004d50 0x7f37780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f378c99b700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37880ccee0 0x7f3778003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f3787fff700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3778003fe0 0x7f3778004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f378d19c700 1 -- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37880ccee0 msgr2=0x7f3778003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f378d19c700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f37880ccee0 0x7f3778003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f378d19c700 1 -- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3778003fe0 msgr2=0x7f3778004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f378d19c700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3778003fe0 0x7f3778004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f378d19c700 1 -- 172.21.15.119:0/3961981114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3778003570 con 0x7f3778004d50 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f3787fff700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3778003fe0 0x7f3778004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.006+0000 7f378d19c700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3778004d50 0x7f37780051c0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f378000e210 tx=0x7f378000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3543783cedf409 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378000ea60 con 0x7f3778004d50 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378000f040 con 0x7f3778004d50 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f37877fe700 1 -- 172.21.15.119:0/3961981114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378000ee10 con 0x7f3778004d50 2023-10-08T02:31:57.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f378ebff700 1 -- 172.21.15.119:0/3961981114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3778004d50 msgr2=0x7f37780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f378ebff700 1 --2- 172.21.15.119:0/3961981114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3778004d50 0x7f37780051c0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f378000e210 tx=0x7f378000b040 comp rx=0 tx=0).stop 2023-10-08T02:31:57.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f378d19c700 1 -- 172.21.15.119:0/3961981114 reap_dead start 2023-10-08T02:31:57.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f378ebff700 1 -- 172.21.15.119:0/3961981114 shutdown_connections 2023-10-08T02:31:57.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f378ebff700 1 -- 172.21.15.119:0/3961981114 >> 172.21.15.119:0/3961981114 conn(0x7f3788068e10 msgr2=0x7f37880cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:57.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f378ebff700 1 -- 172.21.15.119:0/3961981114 shutdown_connections 2023-10-08T02:31:57.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.007+0000 7f378ebff700 1 -- 172.21.15.119:0/3961981114 wait complete. 2023-10-08T02:31:57.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.009+0000 7f378ebff700 1 Processor -- start 2023-10-08T02:31:57.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.009+0000 7f378ebff700 1 -- start start 2023-10-08T02:31:57.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378ebff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3778004d50 0x7f3788156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378ebff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37880b04f0 0x7f3788153320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378ebff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37880b0960 0x7f3788153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378ebff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788154af0 con 0x7f37880b0960 2023-10-08T02:31:57.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378ebff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3788153ee0 con 0x7f37880b04f0 2023-10-08T02:31:57.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378ebff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37881541c0 con 0x7f3778004d50 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378d19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37880b0960 0x7f3788153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378d19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37880b0960 0x7f3788153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50406/0 (socket says 172.21.15.119:50406) 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378d19c700 1 -- 172.21.15.119:0/326815625 learned_addr learned my addr 172.21.15.119:0/326815625 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378c99b700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3778004d50 0x7f3788156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f3787fff700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37880b04f0 0x7f3788153320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378d19c700 1 -- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3778004d50 msgr2=0x7f3788156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378d19c700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3778004d50 0x7f3788156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378d19c700 1 -- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37880b04f0 msgr2=0x7f3788153320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378d19c700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37880b04f0 0x7f3788153320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378d19c700 1 -- 172.21.15.119:0/326815625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37881544a0 con 0x7f37880b0960 2023-10-08T02:31:57.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f378c99b700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3778004d50 0x7f3788156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.010+0000 7f3787fff700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37880b04f0 0x7f3788153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.011+0000 7f378d19c700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37880b0960 0x7f3788153860 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f378000afd0 tx=0x7f378001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.011+0000 7f3784ff9700 1 -- 172.21.15.119:0/326815625 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378000ee60 con 0x7f37880b0960 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.011+0000 7f3784ff9700 1 -- 172.21.15.119:0/326815625 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3780022070 con 0x7f37880b0960 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.011+0000 7f3784ff9700 1 -- 172.21.15.119:0/326815625 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378001e890 con 0x7f37880b0960 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.011+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3788154780 con 0x7f37880b0960 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.011+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3788139b20 con 0x7f37880b0960 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.012+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3788154ca0 con 0x7f37880b0960 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.013+0000 7f3784ff9700 1 -- 172.21.15.119:0/326815625 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3780003040 con 0x7f37880b0960 2023-10-08T02:31:57.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.014+0000 7f3784ff9700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f376c066f30 0x7f376c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.014+0000 7f3784ff9700 1 -- 172.21.15.119:0/326815625 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f3780024070 con 0x7f37880b0960 2023-10-08T02:31:57.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.014+0000 7f378c99b700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f376c066f30 0x7f376c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.015+0000 7f378c99b700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f376c066f30 0x7f376c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f377c001c40 tx=0x7f377c009040 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.018+0000 7f3784ff9700 1 -- 172.21.15.119:0/326815625 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3788154ca0 con 0x7f37880b0960 2023-10-08T02:31:57.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.209+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f37880b2160 con 0x7f37880b0960 2023-10-08T02:31:57.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.215+0000 7f3784ff9700 1 -- 172.21.15.119:0/326815625 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v74) v1 ==== 115+0+0 (secure 0 0 0) 0x7f37800533a0 con 0x7f37880b0960 2023-10-08T02:31:57.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f376c066f30 msgr2=0x7f376c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f376c066f30 0x7f376c0693e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f377c001c40 tx=0x7f377c009040 comp rx=0 tx=0).stop 2023-10-08T02:31:57.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37880b0960 msgr2=0x7f3788153860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37880b0960 0x7f3788153860 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f378000afd0 tx=0x7f378001a000 comp rx=0 tx=0).stop 2023-10-08T02:31:57.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 shutdown_connections 2023-10-08T02:31:57.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f376c066f30 0x7f376c0693e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3778004d50 0x7f3788156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f37880b0960 0x7f3788153860 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 --2- 172.21.15.119:0/326815625 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f37880b04f0 0x7f3788153320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 >> 172.21.15.119:0/326815625 conn(0x7f3788068e10 msgr2=0x7f37880b1720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:57.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 shutdown_connections 2023-10-08T02:31:57.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.217+0000 7f378ebff700 1 -- 172.21.15.119:0/326815625 wait complete. 2023-10-08T02:31:57.237 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-10-08T02:31:57.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.421+0000 7fc3895ff700 1 Processor -- start 2023-10-08T02:31:57.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.421+0000 7fc3895ff700 1 -- start start 2023-10-08T02:31:57.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc3895ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc384155eb0 0x7fc38406e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc3895ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc384156320 con 0x7fc3840c70b0 2023-10-08T02:31:57.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc3895ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38406eb90 con 0x7fc384069770 2023-10-08T02:31:57.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc3895ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3840cd350 con 0x7fc384155eb0 2023-10-08T02:31:57.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc382ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc3840c70b0 0x7fc3840ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52100/0 (socket says 172.21.15.119:52100) 2023-10-08T02:31:57.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc382ffd700 1 -- 172.21.15.119:0/50316170 learned_addr learned my addr 172.21.15.119:0/50316170 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:57.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc37bfff700 1 --2- 172.21.15.119:0/50316170 >> v2:172.21.15.184:3300/0 conn(0x7fc384155eb0 0x7fc38406e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471534148 0 0) 0x7fc384156320 con 0x7fc3840c70b0 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.423+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc370003570 con 0x7fc3840c70b0 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2111947907 0 0) 0x7fc370003570 con 0x7fc3840c70b0 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 >> v2:172.21.15.184:3300/0 conn(0x7fc384155eb0 msgr2=0x7fc38406e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 --2- 172.21.15.119:0/50316170 >> v2:172.21.15.184:3300/0 conn(0x7fc384155eb0 0x7fc38406e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 >> v1:172.21.15.184:6789/0 conn(0x7fc384069770 legacy=0x7fc384160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38406ee70 con 0x7fc3840c70b0 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fc374001470 con 0x7fc3840c70b0 2023-10-08T02:31:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc37bfff700 1 --2- 172.21.15.119:0/50316170 >> v2:172.21.15.184:3300/0 conn(0x7fc384155eb0 0x7fc38406e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 >> v1:172.21.15.119:6789/0 conn(0x7fc3840c70b0 legacy=0x7fc3840ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc370003a60 0x7fc370003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370004810 0x7fc370004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3700051c0 0x7fc37000dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3840cd350 con 0x7fc370004810 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000e2e0 con 0x7fc370003a60 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000e4a0 con 0x7fc3700051c0 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc37bfff700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370004810 0x7fc370004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc382ffd700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc370003a60 0x7fc370003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3837fe700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3700051c0 0x7fc37000dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc37bfff700 1 -- 172.21.15.119:0/50316170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3700051c0 msgr2=0x7fc37000dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc37bfff700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc3700051c0 0x7fc37000dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc37bfff700 1 -- 172.21.15.119:0/50316170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc370003a60 msgr2=0x7fc370003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc37bfff700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc370003a60 0x7fc370003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc37bfff700 1 -- 172.21.15.119:0/50316170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc370003570 con 0x7fc370004810 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.424+0000 7fc3837fe700 1 -- 172.21.15.119:0/50316170 reap_dead start 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.425+0000 7fc37bfff700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370004810 0x7fc370004c80 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009df0 tx=0x7fc36c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619e97b1df795a2f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.425+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c009740 con 0x7fc370004810 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.425+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36c00e920 con 0x7fc370004810 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.425+0000 7fc3827fc700 1 -- 172.21.15.119:0/50316170 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36c00ecc0 con 0x7fc370004810 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.425+0000 7fc3895ff700 1 -- 172.21.15.119:0/50316170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370004810 msgr2=0x7fc370004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.425+0000 7fc3895ff700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370004810 0x7fc370004c80 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009df0 tx=0x7fc36c0091a0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.426+0000 7fc3895ff700 1 -- 172.21.15.119:0/50316170 shutdown_connections 2023-10-08T02:31:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.426+0000 7fc3895ff700 1 --2- 172.21.15.119:0/50316170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370004810 0x7fc370004c80 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.426+0000 7fc3895ff700 1 -- 172.21.15.119:0/50316170 >> 172.21.15.119:0/50316170 conn(0x7fc384068e10 msgr2=0x7fc3840cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.426+0000 7fc3895ff700 1 -- 172.21.15.119:0/50316170 shutdown_connections 2023-10-08T02:31:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.426+0000 7fc3895ff700 1 -- 172.21.15.119:0/50316170 wait complete. 2023-10-08T02:31:57.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.427+0000 7fc3895ff700 1 Processor -- start 2023-10-08T02:31:57.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.427+0000 7fc3895ff700 1 -- start start 2023-10-08T02:31:57.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3895ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc370004810 0x7fc3840b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3895ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc384159b40 0x7fc384153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3895ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc384153ee0 0x7fc3841538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3895ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38415a0b0 con 0x7fc384153ee0 2023-10-08T02:31:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3895ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38415a390 con 0x7fc384159b40 2023-10-08T02:31:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3895ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc384157630 con 0x7fc370004810 2023-10-08T02:31:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3837fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc384153ee0 0x7fc3841538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3837fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc384153ee0 0x7fc3841538b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50430/0 (socket says 172.21.15.119:50430) 2023-10-08T02:31:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.428+0000 7fc3837fe700 1 -- 172.21.15.119:0/2863152806 learned_addr learned my addr 172.21.15.119:0/2863152806 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc37bfff700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc384159b40 0x7fc384153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc3837fe700 1 -- 172.21.15.119:0/2863152806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc370004810 msgr2=0x7fc3840b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc3837fe700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc370004810 0x7fc3840b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc3837fe700 1 -- 172.21.15.119:0/2863152806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc384159b40 msgr2=0x7fc384153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc3837fe700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc384159b40 0x7fc384153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc3837fe700 1 -- 172.21.15.119:0/2863152806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc384157910 con 0x7fc384153ee0 2023-10-08T02:31:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc37bfff700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc384159b40 0x7fc384153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:31:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc3837fe700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc384153ee0 0x7fc3841538b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fc3840c8f70 tx=0x7fc37c001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc37affd700 1 -- 172.21.15.119:0/2863152806 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c006e80 con 0x7fc384153ee0 2023-10-08T02:31:57.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc37affd700 1 -- 172.21.15.119:0/2863152806 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37c00c5a0 con 0x7fc384153ee0 2023-10-08T02:31:57.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38415a670 con 0x7fc384153ee0 2023-10-08T02:31:57.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc37affd700 1 -- 172.21.15.119:0/2863152806 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00c940 con 0x7fc384153ee0 2023-10-08T02:31:57.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.429+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc384154600 con 0x7fc384153ee0 2023-10-08T02:31:57.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.430+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38404c3c0 con 0x7fc384153ee0 2023-10-08T02:31:57.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.431+0000 7fc37affd700 1 -- 172.21.15.119:0/2863152806 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc37c00ccf0 con 0x7fc384153ee0 2023-10-08T02:31:57.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.432+0000 7fc37affd700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc364066f60 0x7fc364069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.432+0000 7fc37affd700 1 -- 172.21.15.119:0/2863152806 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fc37c084590 con 0x7fc384153ee0 2023-10-08T02:31:57.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.432+0000 7fc382ffd700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc364066f60 0x7fc364069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.433+0000 7fc382ffd700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc364066f60 0x7fc364069410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc374001050 tx=0x7fc38406f710 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.436+0000 7fc37affd700 1 -- 172.21.15.119:0/2863152806 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc37c0516c0 con 0x7fc384153ee0 2023-10-08T02:31:57.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.626+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fc38404c3c0 con 0x7fc384153ee0 2023-10-08T02:31:57.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.632+0000 7fc37affd700 1 -- 172.21.15.119:0/2863152806 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v75) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc37c054ce0 con 0x7fc384153ee0 2023-10-08T02:31:57.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.634+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc364066f60 msgr2=0x7fc364069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc364066f60 0x7fc364069410 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc374001050 tx=0x7fc38406f710 comp rx=0 tx=0).stop 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc384153ee0 msgr2=0x7fc3841538b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc384153ee0 0x7fc3841538b0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7fc3840c8f70 tx=0x7fc37c001640 comp rx=0 tx=0).stop 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 shutdown_connections 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc364066f60 0x7fc364069410 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc370004810 0x7fc3840b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc384153ee0 0x7fc3841538b0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 --2- 172.21.15.119:0/2863152806 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc384159b40 0x7fc384153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 >> 172.21.15.119:0/2863152806 conn(0x7fc384068e10 msgr2=0x7fc384069b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 shutdown_connections 2023-10-08T02:31:57.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.635+0000 7fc3895ff700 1 -- 172.21.15.119:0/2863152806 wait complete. 2023-10-08T02:31:57.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-10-08T02:31:57.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.833+0000 7f3d84dff700 1 Processor -- start 2023-10-08T02:31:57.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.833+0000 7f3d84dff700 1 -- start start 2023-10-08T02:31:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.834+0000 7f3d84dff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3d800ccc40 0x7f3d80155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d84dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d80156150 0x7f3d800ac230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d84dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d800ad930 con 0x7f3d800cd010 2023-10-08T02:31:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d84dff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d800acd20 con 0x7f3d800ccc40 2023-10-08T02:31:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d84dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d800ad000 con 0x7f3d80156150 2023-10-08T02:31:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d7e59c700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f3d800ccc40 0x7f3d80155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d77fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d80156150 0x7f3d800ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d77fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d80156150 0x7f3d800ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:41546/0 (socket says 172.21.15.119:41546) 2023-10-08T02:31:57.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d77fff700 1 -- 172.21.15.119:0/1290745356 learned_addr learned my addr 172.21.15.119:0/1290745356 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:57.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.835+0000 7f3d7dd9b700 1 -- 172.21.15.119:0/1290745356 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671757925 0 0) 0x7f3d800ad930 con 0x7f3d800cd010 2023-10-08T02:31:57.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d7e59c700 1 -- 172.21.15.119:0/1290745356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d80156150 msgr2=0x7f3d800ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:57.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d7e59c700 1 --2- 172.21.15.119:0/1290745356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d80156150 0x7f3d800ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d7e59c700 1 -- 172.21.15.119:0/1290745356 >> v1:172.21.15.184:6789/0 conn(0x7f3d800cd010 legacy=0x7f3d800af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d7e59c700 1 -- 172.21.15.119:0/1290745356 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d800ad2e0 con 0x7f3d800ccc40 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d77fff700 1 --2- 172.21.15.119:0/1290745356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d80156150 0x7f3d800ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d7e59c700 1 --2- 172.21.15.119:0/1290745356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 0x7f3d80155c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000bb90 tx=0x7f3d7000b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd7c7f5cbb47870 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d7dd9b700 1 -- 172.21.15.119:0/1290745356 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7000db40 con 0x7f3d800ccc40 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d7dd9b700 1 -- 172.21.15.119:0/1290745356 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7000dca0 con 0x7f3d800ccc40 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.836+0000 7f3d7dd9b700 1 -- 172.21.15.119:0/1290745356 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d700196b0 con 0x7f3d800ccc40 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.837+0000 7f3d84dff700 1 -- 172.21.15.119:0/1290745356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 msgr2=0x7f3d80155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.837+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1290745356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 0x7f3d80155c10 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000bb90 tx=0x7f3d7000b430 comp rx=0 tx=0).stop 2023-10-08T02:31:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.837+0000 7f3d84dff700 1 -- 172.21.15.119:0/1290745356 shutdown_connections 2023-10-08T02:31:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.837+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1290745356 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 0x7f3d80155c10 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.837+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1290745356 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d80156150 0x7f3d800ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.837+0000 7f3d84dff700 1 -- 172.21.15.119:0/1290745356 >> 172.21.15.119:0/1290745356 conn(0x7f3d8006a320 msgr2=0x7f3d800cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.837+0000 7f3d84dff700 1 -- 172.21.15.119:0/1290745356 shutdown_connections 2023-10-08T02:31:57.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.837+0000 7f3d84dff700 1 -- 172.21.15.119:0/1290745356 wait complete. 2023-10-08T02:31:57.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.838+0000 7f3d84dff700 1 Processor -- start 2023-10-08T02:31:57.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.838+0000 7f3d84dff700 1 -- start start 2023-10-08T02:31:57.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.839+0000 7f3d84dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 0x7f3d800aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.839+0000 7f3d84dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d800cd010 0x7f3d80169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.839+0000 7f3d84dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80156150 0x7f3d800c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.839+0000 7f3d84dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d801509c0 con 0x7f3d80156150 2023-10-08T02:31:57.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.839+0000 7f3d84dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8016a4b0 con 0x7f3d800ccc40 2023-10-08T02:31:57.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.839+0000 7f3d84dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d80150c10 con 0x7f3d800cd010 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80156150 0x7f3d800c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80156150 0x7f3d800c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50436/0 (socket says 172.21.15.119:50436) 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 -- 172.21.15.119:0/1651959694 learned_addr learned my addr 172.21.15.119:0/1651959694 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7e59c700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 0x7f3d800aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d77fff700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d800cd010 0x7f3d80169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 -- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d800cd010 msgr2=0x7f3d80169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d800cd010 0x7f3d80169f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 -- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 msgr2=0x7f3d800aae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 0x7f3d800aae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:57.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 -- 172.21.15.119:0/1651959694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d680094f0 con 0x7f3d80156150 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7e59c700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 0x7f3d800aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d77fff700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d800cd010 0x7f3d80169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.840+0000 7f3d7ed9d700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80156150 0x7f3d800c9030 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f3d78001560 tx=0x7f3d78001770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.841+0000 7f3d767fc700 1 -- 172.21.15.119:0/1651959694 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800a780 con 0x7f3d80156150 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.841+0000 7f3d767fc700 1 -- 172.21.15.119:0/1651959694 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d7800a8e0 con 0x7f3d80156150 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.841+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7000b6d0 con 0x7f3d80156150 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.841+0000 7f3d767fc700 1 -- 172.21.15.119:0/1651959694 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7800aca0 con 0x7f3d80156150 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.841+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d801511d0 con 0x7f3d80156150 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.842+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8004c3c0 con 0x7f3d80156150 2023-10-08T02:31:57.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.843+0000 7f3d767fc700 1 -- 172.21.15.119:0/1651959694 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d78012630 con 0x7f3d80156150 2023-10-08T02:31:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.843+0000 7f3d767fc700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3d60066f30 0x7f3d600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.843+0000 7f3d767fc700 1 -- 172.21.15.119:0/1651959694 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f3d78084ac0 con 0x7f3d80156150 2023-10-08T02:31:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.844+0000 7f3d7e59c700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3d60066f30 0x7f3d600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:57.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.844+0000 7f3d7e59c700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3d60066f30 0x7f3d600693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000b6a0 tx=0x7f3d7001b000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:57.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:57.848+0000 7f3d767fc700 1 -- 172.21.15.119:0/1651959694 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3d78051b50 con 0x7f3d80156150 2023-10-08T02:31:58.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.113+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f3d800678d0 con 0x7f3d80156150 2023-10-08T02:31:58.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.368+0000 7f3d767fc700 1 -- 172.21.15.119:0/1651959694 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3116) v1 ==== 155+0+0 (secure 0 0 0) 0x7f3d78055170 con 0x7f3d80156150 2023-10-08T02:31:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3d60066f30 msgr2=0x7f3d600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3d60066f30 0x7f3d600693e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000b6a0 tx=0x7f3d7001b000 comp rx=0 tx=0).stop 2023-10-08T02:31:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80156150 msgr2=0x7f3d800c9030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:58.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80156150 0x7f3d800c9030 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f3d78001560 tx=0x7f3d78001770 comp rx=0 tx=0).stop 2023-10-08T02:31:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 shutdown_connections 2023-10-08T02:31:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f3d60066f30 0x7f3d600693e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3d80156150 0x7f3d800c9030 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f3d800ccc40 0x7f3d800aae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 --2- 172.21.15.119:0/1651959694 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f3d800cd010 0x7f3d80169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 >> 172.21.15.119:0/1651959694 conn(0x7f3d8006a320 msgr2=0x7f3d800ab730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 shutdown_connections 2023-10-08T02:31:58.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:58.371+0000 7f3d84dff700 1 -- 172.21.15.119:0/1651959694 wait complete. 2023-10-08T02:31:58.390 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-10-08T02:31:58.390 INFO:tasks.cephfs_test_runner: 2023-10-08T02:31:58.390 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-08T02:31:58.390 INFO:tasks.cephfs_test_runner:Ran 11 tests in 5736.907s 2023-10-08T02:31:58.390 INFO:tasks.cephfs_test_runner: 2023-10-08T02:31:58.390 INFO:tasks.cephfs_test_runner:OK 2023-10-08T02:31:58.391 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-08T02:31:58.422 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-10-08T02:31:58.446 INFO:tasks.kclient:Unmounting kernel clients... 2023-10-08T02:31:58.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:58.447 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T02:31:58.484 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-08T02:31:58.484 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-08T02:31:58.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:58.485 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-08T02:31:58.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:58.586 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-08T02:31:58.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:58.661 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-10-08T02:31:58.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:58.783 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-10-08T02:31:58.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:58.823 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-10-08T02:31:58.927 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2023-10-08T02:31:58.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T02:31:58.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:58.931 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-08T02:31:58.971 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-08T02:31:58.971 INFO:teuthology.orchestra.run.smithi184.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-08T02:31:58.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:58.972 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-08T02:31:59.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:59.077 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-08T02:31:59.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:59.154 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-10-08T02:31:59.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:59.280 DEBUG:teuthology.orchestra.run.smithi184:> ip netns list 2023-10-08T02:31:59.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-08T02:31:59.319 DEBUG:teuthology.orchestra.run.smithi184:> sudo ip link delete ceph-brx 2023-10-08T02:31:59.426 INFO:teuthology.orchestra.run.smithi184.stderr:Cannot find device "ceph-brx" 2023-10-08T02:31:59.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T02:31:59.429 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-08T02:31:59.441 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-08T02:31:59.441 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:31:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.622+0000 7fcddd99d700 1 Processor -- start 2023-10-08T02:31:59.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.622+0000 7fcddd99d700 1 -- start start 2023-10-08T02:31:59.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcddd99d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcdd8150550 0x7fcdd8157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcddd99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c7410 0x7fcdd8069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcddd99d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd806e2a0 con 0x7fcdd8150180 2023-10-08T02:31:59.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcddd99d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdd806b2e0 con 0x7fcdd8150550 2023-10-08T02:31:59.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcddd99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80c78e0 con 0x7fcdd80c7410 2023-10-08T02:31:59.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcddc99b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcdd8150180 0x7fcdd80c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52116/0 (socket says 172.21.15.119:52116) 2023-10-08T02:31:59.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcddc99b700 1 -- 172.21.15.119:0/3288981296 learned_addr learned my addr 172.21.15.119:0/3288981296 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcdd7fff700 1 --2- 172.21.15.119:0/3288981296 >> v2:172.21.15.184:3300/0 conn(0x7fcdd8150550 0x7fcdd8157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcddd19c700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c7410 0x7fcdd8069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084756314 0 0) 0x7fcdd806e2a0 con 0x7fcdd8150180 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.624+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdc8003570 con 0x7fcdd8150180 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001892093 0 0) 0x7fcdc8003570 con 0x7fcdd8150180 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c7410 msgr2=0x7fcdd8069f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c7410 0x7fcdd8069f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 >> v2:172.21.15.184:3300/0 conn(0x7fcdd8150550 msgr2=0x7fcdd8157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:59.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 --2- 172.21.15.119:0/3288981296 >> v2:172.21.15.184:3300/0 conn(0x7fcdd8150550 0x7fcdd8157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd80c7bc0 con 0x7fcdd8150180 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fcdcc001470 con 0x7fcdd8150180 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 >> v1:172.21.15.119:6789/0 conn(0x7fcdd8150180 legacy=0x7fcdd80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc8003ae0 0x7fcdc8003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c7410 0x7fcdc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8004d50 0x7fcdc80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd806b2e0 con 0x7fcdc8003ae0 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80c78e0 con 0x7fcdc8004d50 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc800bb10 con 0x7fcdd80c7410 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcddc99b700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc8003ae0 0x7fcdc8003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcdd7fff700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c7410 0x7fcdc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.625+0000 7fcddd19c700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8004d50 0x7fcdc80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcddc99b700 1 -- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c7410 msgr2=0x7fcdc8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcddc99b700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd80c7410 0x7fcdc8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcddc99b700 1 -- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8004d50 msgr2=0x7fcdc80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcddc99b700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8004d50 0x7fcdc80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcddc99b700 1 -- 172.21.15.119:0/3288981296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc8003570 con 0x7fcdc8003ae0 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcddc99b700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc8003ae0 0x7fcdc8003ef0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00c300 tx=0x7fcdcc001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58ceb8597442c81 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc00cb90 con 0x7fcdc8003ae0 2023-10-08T02:31:59.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc00ccf0 con 0x7fcdc8003ae0 2023-10-08T02:31:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcdd77fe700 1 -- 172.21.15.119:0/3288981296 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc005b70 con 0x7fcdc8003ae0 2023-10-08T02:31:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.626+0000 7fcddd99d700 1 -- 172.21.15.119:0/3288981296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc8003ae0 msgr2=0x7fcdc8003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.627+0000 7fcddd99d700 1 --2- 172.21.15.119:0/3288981296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc8003ae0 0x7fcdc8003ef0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00c300 tx=0x7fcdcc001dc0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.627+0000 7fcddd19c700 1 -- 172.21.15.119:0/3288981296 reap_dead start 2023-10-08T02:31:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.627+0000 7fcddd99d700 1 -- 172.21.15.119:0/3288981296 shutdown_connections 2023-10-08T02:31:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.627+0000 7fcddd99d700 1 -- 172.21.15.119:0/3288981296 >> 172.21.15.119:0/3288981296 conn(0x7fcdd80cc1f0 msgr2=0x7fcdd80cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.627+0000 7fcddd99d700 1 -- 172.21.15.119:0/3288981296 shutdown_connections 2023-10-08T02:31:59.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.627+0000 7fcddd99d700 1 -- 172.21.15.119:0/3288981296 wait complete. 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.628+0000 7fcddd99d700 1 Processor -- start 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.628+0000 7fcddd99d700 1 -- start start 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.629+0000 7fcddd99d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8003ae0 0x7fcdd8157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.629+0000 7fcddd99d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd8159810 0x7fcdd80ae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.629+0000 7fcddd99d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd816a990 0x7fcdd80acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.629+0000 7fcddd99d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8159cb0 con 0x7fcdd816a990 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.629+0000 7fcddd99d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd8159f90 con 0x7fcdc8003ae0 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.629+0000 7fcddd99d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd815a270 con 0x7fcdd8159810 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd816a990 0x7fcdd80acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd816a990 0x7fcdd80acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50448/0 (socket says 172.21.15.119:50448) 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 -- 172.21.15.119:0/3271558452 learned_addr learned my addr 172.21.15.119:0/3271558452 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcdd7fff700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd8159810 0x7fcdd80ae1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddc99b700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8003ae0 0x7fcdd8157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 -- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd8159810 msgr2=0x7fcdd80ae1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd8159810 0x7fcdd80ae1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 -- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8003ae0 msgr2=0x7fcdd8157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8003ae0 0x7fcdd8157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 -- 172.21.15.119:0/3271558452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd815a550 con 0x7fcdd816a990 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddc99b700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8003ae0 0x7fcdd8157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.630+0000 7fcddd19c700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd816a990 0x7fcdd80acfd0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80c67e0 tx=0x7fcdd000d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:59.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.631+0000 7fcdd4ff9700 1 -- 172.21.15.119:0/3271558452 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0010040 con 0x7fcdd816a990 2023-10-08T02:31:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.631+0000 7fcdd4ff9700 1 -- 172.21.15.119:0/3271558452 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd0011040 con 0x7fcdd816a990 2023-10-08T02:31:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.631+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd8169440 con 0x7fcdd816a990 2023-10-08T02:31:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.631+0000 7fcdd4ff9700 1 -- 172.21.15.119:0/3271558452 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0016470 con 0x7fcdd816a990 2023-10-08T02:31:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.631+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd80ae700 con 0x7fcdd816a990 2023-10-08T02:31:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.632+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd81696b0 con 0x7fcdd816a990 2023-10-08T02:31:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.633+0000 7fcdd4ff9700 1 -- 172.21.15.119:0/3271558452 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcdd0016860 con 0x7fcdd816a990 2023-10-08T02:31:59.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.633+0000 7fcdd4ff9700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcdbc066f30 0x7fcdbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:31:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.633+0000 7fcdd4ff9700 1 -- 172.21.15.119:0/3271558452 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fcdd0085740 con 0x7fcdd816a990 2023-10-08T02:31:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.634+0000 7fcddc99b700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcdbc066f30 0x7fcdbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:31:59.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.634+0000 7fcddc99b700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcdbc066f30 0x7fcdbc0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc015040 tx=0x7fcdcc012000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:31:59.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.638+0000 7fcdd4ff9700 1 -- 172.21.15.119:0/3271558452 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcdd00527d0 con 0x7fcdd816a990 2023-10-08T02:31:59.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.830+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdd804c3c0 con 0x7fcdd816a990 2023-10-08T02:31:59.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.831+0000 7fcdd4ff9700 1 -- 172.21.15.119:0/3271558452 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+13939 (secure 0 0 0) 0x7fcdd0055df0 con 0x7fcdd816a990 2023-10-08T02:31:59.831 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:31:59.832 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":211,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:56.361060+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/3602020849":"2023-10-09T02:31:53.966925+0000","172.21.15.119:0/927931328":"2023-10-09T02:31:53.966925+0000","172.21.15.119:0/2124875896":"2023-10-09T02:31:53.966925+0000","172.21.15.119:0/2189804149":"2023-10-09T02:31:53.966925+0000","172.21.15.119:0/10097412":"2023-10-09T02:31:53.966925+0000","172.21.15.119:6835/1741909431":"2023-10-09T02:31:52.963764+0000","172.21.15.119:0/3003317145":"2023-10-09T02:31:53.966925+0000","172.21.15.119:6834/1741909431":"2023-10-09T02:31:52.963764+0000","172.21.15.119:0/842982126":"2023-10-09T02:31:43.569085+0000","172.21.15.119:6837/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/3912047418":"2023-10-09T02:31:53.966925+0000","172.21.15.119:6836/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/1887277487":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/4191951118":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/773312007":"2023-10-09T02:31:43.569085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:31:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcdbc066f30 msgr2=0x7fcdbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcdbc066f30 0x7fcdbc0693e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc015040 tx=0x7fcdcc012000 comp rx=0 tx=0).stop 2023-10-08T02:31:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd816a990 msgr2=0x7fcdd80acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:31:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd816a990 0x7fcdd80acfd0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fcdd80c67e0 tx=0x7fcdd000d040 comp rx=0 tx=0).stop 2023-10-08T02:31:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 shutdown_connections 2023-10-08T02:31:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcdbc066f30 0x7fcdbc0693e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdd816a990 0x7fcdd80acfd0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcdc8003ae0 0x7fcdd8157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 --2- 172.21.15.119:0/3271558452 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcdd8159810 0x7fcdd80ae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:31:59.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 >> 172.21.15.119:0/3271558452 conn(0x7fcdd80cc1f0 msgr2=0x7fcdd80696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:31:59.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 shutdown_connections 2023-10-08T02:31:59.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:31:59.833+0000 7fcddd99d700 1 -- 172.21.15.119:0/3271558452 wait complete. 2023-10-08T02:31:59.851 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-08T02:31:59.851 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:32:00.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.033+0000 7fc8733ff700 1 Processor -- start 2023-10-08T02:32:00.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.034+0000 7fc8733ff700 1 -- start start 2023-10-08T02:32:00.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc8733ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc86c0b1420 0x7fc86c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc8733ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0ccf90 0x7fc86c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc8733ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c0cc300 con 0x7fc86c0cbf30 2023-10-08T02:32:00.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc8733ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c156bf0 con 0x7fc86c0b1420 2023-10-08T02:32:00.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc8733ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0ae950 con 0x7fc86c0ccf90 2023-10-08T02:32:00.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc87199c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0ccf90 0x7fc86c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc87099a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc86c0b1420 0x7fc86c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc87199c700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0ccf90 0x7fc86c0c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:41588/0 (socket says 172.21.15.119:41588) 2023-10-08T02:32:00.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc87099a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fc86c0b1420 0x7fc86c0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:46756/0 (socket says 172.21.15.119:46756) 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc87199c700 1 -- 172.21.15.119:0/42932876 learned_addr learned my addr 172.21.15.119:0/42932876 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.035+0000 7fc87119b700 1 --1- 172.21.15.119:0/42932876 >> v1:172.21.15.184:6789/0 conn(0x7fc86c0cbf30 0x7fc86c0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:34682/0 (socket says 172.21.15.119:34682) 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.036+0000 7fc87199c700 1 -- 172.21.15.119:0/42932876 >> v2:172.21.15.184:3300/0 conn(0x7fc86c0b1420 msgr2=0x7fc86c0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.036+0000 7fc87199c700 1 --2- 172.21.15.119:0/42932876 >> v2:172.21.15.184:3300/0 conn(0x7fc86c0b1420 0x7fc86c0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.036+0000 7fc87199c700 1 -- 172.21.15.119:0/42932876 >> v1:172.21.15.184:6789/0 conn(0x7fc86c0cbf30 legacy=0x7fc86c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.036+0000 7fc87199c700 1 -- 172.21.15.119:0/42932876 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c0ad370 con 0x7fc86c0ccf90 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.036+0000 7fc87099a700 1 --2- 172.21.15.119:0/42932876 >> v2:172.21.15.184:3300/0 conn(0x7fc86c0b1420 0x7fc86c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.036+0000 7fc87199c700 1 --2- 172.21.15.119:0/42932876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0ccf90 0x7fc86c0c9740 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc864009f20 tx=0x7fc864009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7ebd9105a4400b6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.036+0000 7fc863fff700 1 -- 172.21.15.119:0/42932876 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400e070 con 0x7fc86c0ccf90 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc863fff700 1 -- 172.21.15.119:0/42932876 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400b9d0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc863fff700 1 -- 172.21.15.119:0/42932876 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640092e0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc8733ff700 1 -- 172.21.15.119:0/42932876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0ccf90 msgr2=0x7fc86c0c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc8733ff700 1 --2- 172.21.15.119:0/42932876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0ccf90 0x7fc86c0c9740 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc864009f20 tx=0x7fc864009840 comp rx=0 tx=0).stop 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc8733ff700 1 -- 172.21.15.119:0/42932876 shutdown_connections 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc8733ff700 1 --2- 172.21.15.119:0/42932876 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0ccf90 0x7fc86c0c9740 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc8733ff700 1 --2- 172.21.15.119:0/42932876 >> v2:172.21.15.184:3300/0 conn(0x7fc86c0b1420 0x7fc86c0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc8733ff700 1 -- 172.21.15.119:0/42932876 >> 172.21.15.119:0/42932876 conn(0x7fc86c06a320 msgr2=0x7fc86c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:00.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc8733ff700 1 -- 172.21.15.119:0/42932876 shutdown_connections 2023-10-08T02:32:00.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.037+0000 7fc8733ff700 1 -- 172.21.15.119:0/42932876 wait complete. 2023-10-08T02:32:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.038+0000 7fc8733ff700 1 Processor -- start 2023-10-08T02:32:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.039+0000 7fc8733ff700 1 -- start start 2023-10-08T02:32:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.039+0000 7fc8733ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc86c0b1420 0x7fc86c159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.039+0000 7fc8733ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0cbf30 0x7fc86c159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.039+0000 7fc8733ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c0ccf90 0x7fc86c15a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.039+0000 7fc8733ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1517a0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.039+0000 7fc8733ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1592e0 con 0x7fc86c0b1420 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc8733ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c1589f0 con 0x7fc86c0cbf30 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c0ccf90 0x7fc86c15a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87119b700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc86c0b1420 0x7fc86c159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c0ccf90 0x7fc86c15a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50460/0 (socket says 172.21.15.119:50460) 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 -- 172.21.15.119:0/196556540 learned_addr learned my addr 172.21.15.119:0/196556540 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:00.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87099a700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0cbf30 0x7fc86c159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 -- 172.21.15.119:0/196556540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0cbf30 msgr2=0x7fc86c159cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0cbf30 0x7fc86c159cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 -- 172.21.15.119:0/196556540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc86c0b1420 msgr2=0x7fc86c159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc86c0b1420 0x7fc86c159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 -- 172.21.15.119:0/196556540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c00b6d0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87119b700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc86c0b1420 0x7fc86c159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.040+0000 7fc87199c700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c0ccf90 0x7fc86c15a1f0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fc86400b610 tx=0x7fc864018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.041+0000 7fc8617fa700 1 -- 172.21.15.119:0/196556540 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864008b10 con 0x7fc86c0ccf90 2023-10-08T02:32:00.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.041+0000 7fc8617fa700 1 -- 172.21.15.119:0/196556540 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc864008cb0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.041+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc864009ae0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.041+0000 7fc8617fa700 1 -- 172.21.15.119:0/196556540 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640256b0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.041+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c158f70 con 0x7fc86c0ccf90 2023-10-08T02:32:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.042+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c04c3c0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.043+0000 7fc8617fa700 1 -- 172.21.15.119:0/196556540 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc864003040 con 0x7fc86c0ccf90 2023-10-08T02:32:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.044+0000 7fc8617fa700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc850066f30 0x7fc8500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.044+0000 7fc8617fa700 1 -- 172.21.15.119:0/196556540 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fc8640862d0 con 0x7fc86c0ccf90 2023-10-08T02:32:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.044+0000 7fc87119b700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc850066f30 0x7fc8500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.044+0000 7fc87119b700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc850066f30 0x7fc8500693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc868007fd0 tx=0x7fc868007f40 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.048+0000 7fc8617fa700 1 -- 172.21.15.119:0/196556540 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc864053360 con 0x7fc86c0ccf90 2023-10-08T02:32:00.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.237+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc86c0ad650 con 0x7fc850066f30 2023-10-08T02:32:00.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.240+0000 7fc8617fa700 1 -- 172.21.15.119:0/196556540 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50600 (secure 0 0 0) 0x7fc86c0ad650 con 0x7fc850066f30 2023-10-08T02:32:00.241 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:32:00.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc850066f30 msgr2=0x7fc8500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc850066f30 0x7fc8500693e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc868007fd0 tx=0x7fc868007f40 comp rx=0 tx=0).stop 2023-10-08T02:32:00.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c0ccf90 msgr2=0x7fc86c15a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c0ccf90 0x7fc86c15a1f0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fc86400b610 tx=0x7fc864018000 comp rx=0 tx=0).stop 2023-10-08T02:32:00.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 shutdown_connections 2023-10-08T02:32:00.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fc850066f30 0x7fc8500693e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c0ccf90 0x7fc86c15a1f0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fc86c0b1420 0x7fc86c159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 --2- 172.21.15.119:0/196556540 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fc86c0cbf30 0x7fc86c159cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 >> 172.21.15.119:0/196556540 conn(0x7fc86c06a320 msgr2=0x7fc86c0caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:00.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 shutdown_connections 2023-10-08T02:32:00.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.242+0000 7fc8733ff700 1 -- 172.21.15.119:0/196556540 wait complete. 2023-10-08T02:32:00.244 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:32:00.260 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-10-08T02:31:59.987627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2470344,"kb_used_data":22408,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752504376,"statfs":{"total":773094113280,"available":770564481024,"internally_reserved":0,"allocated":22945792,"data_stored":12492126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668755,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.996381"},"pg_stats":[{"pgid":"1.0","version":"209'370","reported_seq":1221,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-08T02:31:54.057694+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:31:54.057694+0000","last_peered":"2023-10-08T02:31:54.057694+0000","last_clean":"2023-10-08T02:31:54.057694+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:31:54.057694+0000","last_undegraded":"2023-10-08T02:31:54.057694+0000","last_fullsized":"2023-10-08T02:31:54.057694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317140,"kb_used_data":3468,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94054700,"statfs":{"total":96636764160,"available":96312012800,"internally_reserved":0,"allocated":3551232,"data_stored":2242400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93799999999999994,"5min":0.96899999999999997,"15min":1.0349999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85999999999999999,"5min":0.93899999999999995,"15min":0.99199999999999999},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":0.83499999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73299999999999998,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93400000000000005,"5min":0.98499999999999999,"15min":1.002},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.875,"5min":0.96699999999999997,"15min":1.0549999999999999},"last":0.442},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73699999999999999,"15min":0.748},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93000000000000005,"5min":0.96699999999999997,"15min":1.03},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.71699999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.78500000000000003},"min":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98099999999999998,"5min":1.0069999999999999,"15min":1.056},"last":0.63700000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":1.091},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.92200000000000004},"last":0.499}]},{"osd":5,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":1.012},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.47999999999999998,"15min":0.44500000000000001},"max":{"1min":0.873,"5min":0.92100000000000004,"15min":1.1240000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.877,"5min":1.0089999999999999,"15min":1.151},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70299999999999996,"15min":0.71399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.42599999999999999},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299848,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071992,"statfs":{"total":96636764160,"available":96329719808,"internally_reserved":0,"allocated":2621440,"data_stored":1323797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.55100000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92400000000000004,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.67600000000000005}]},{"osd":1,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":1.21},"last":0.749},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76300000000000001,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":3.851,"15min":3.851},"last":0.64000000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":0.86799999999999999,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.258},"last":0.622}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.52600000000000002},"max":{"1min":0.88500000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.378},"max":{"1min":0.95299999999999996,"5min":1.139,"15min":1.139},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.48499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":1.341,"15min":1.341},"last":0.44900000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.621,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.92800000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.443},"max":{"1min":0.91600000000000004,"5min":1.089,"15min":1.089},"last":0.68700000000000006}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.038,"15min":1.0980000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.95599999999999996,"5min":0.96999999999999997,"15min":1.054},"last":0.66400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575987,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312472,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059368,"statfs":{"total":96636764160,"available":96316792832,"internally_reserved":0,"allocated":2965504,"data_stored":1663348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.0780000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.29199999999999998,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.571},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":1.1100000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":1.1299999999999999},"last":0.54100000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.755,"5min":0.80200000000000005,"15min":1.2190000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":1.341,"15min":1.341},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.48199999999999998},"max":{"1min":1.171,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.70499999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.97499999999999998,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.127,"15min":1.27},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.81799999999999995},"min":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.915,"15min":1.915},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.50900000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.204},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.85799999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.306},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.66800000000000004,"5min":0.57999999999999996,"15min":0.55500000000000005},"max":{"1min":1.1399999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.77100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308648,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063192,"statfs":{"total":96636764160,"available":96320708608,"internally_reserved":0,"allocated":3244032,"data_stored":1927766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44700000000000001,"15min":0.46800000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23699999999999999},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.78400000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.76300000000000001,"15min":0.88100000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.48499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.56499999999999995,"5min":0.79400000000000004,"15min":0.83199999999999996},"last":0.48999999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.81799999999999995,"15min":0.93100000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.81399999999999995,"15min":0.89000000000000001},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.75700000000000001,"15min":0.78000000000000003},"min":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.85999999999999999,"5min":1.159,"15min":1.232},"last":0.629},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90500000000000003,"5min":1.306,"15min":1.306},"last":0.72299999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94099999999999995,"5min":1.1990000000000001,"15min":1.216},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98299999999999998,"5min":1.27,"15min":1.296},"last":0.748}]},{"osd":6,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79300000000000004,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.874,"5min":1.135,"15min":1.1579999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96899999999999997,"5min":1.262,"15min":1.262},"last":0.73699999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.83399999999999996},"min":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83199999999999996,"15min":0.86499999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.55700000000000005},"max":{"1min":0.89900000000000002,"5min":1.2250000000000001,"15min":1.3360000000000001},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":316260,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055580,"statfs":{"total":96636764160,"available":96312913920,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61699999999999999,"5min":0.755,"15min":0.871},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.39700000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.70599999999999996,"5min":0.876,"15min":0.876},"last":0.437},{"interface":"front","average":{"1min":0.495,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.80900000000000005},"last":0.49299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.73899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97799999999999998,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0229999999999999,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.751},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82399999999999995,"15min":0.84899999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.353,"15min":1.417},"last":0.80900000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.50900000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94799999999999995,"5min":1.153,"15min":1.333},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94599999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.79300000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.387,"15min":1.4059999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":1.252,"15min":1.5720000000000001},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.495,"15min":0.495},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59099999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.512},"last":0.65600000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.5,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.56000000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.0680000000000001,"5min":1.113,"15min":1.3180000000000001},"last":0.93500000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.57999999999999996,"5min":0.441,"15min":0.441},"max":{"1min":1.117,"5min":1.117,"15min":1.2789999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83499999999999996,"5min":1.1220000000000001,"15min":1.24},"last":0.98899999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89000000000000001,"5min":1.0880000000000001,"15min":1.161},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.95999999999999996,"5min":1.155,"15min":1.2849999999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.82699999999999996},"min":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91600000000000004,"5min":1.081,"15min":1.266},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.282},"last":0.81100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575986,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299868,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071972,"statfs":{"total":96636764160,"available":96329699328,"internally_reserved":0,"allocated":2641920,"data_stored":1336222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.028},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.497,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90800000000000003,"5min":0.998,"15min":1.0409999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.51400000000000001,"5min":0.496,"15min":0.49099999999999999},"max":{"1min":1.01,"5min":1.079,"15min":1.079},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.034},"last":0.621},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.65400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.022,"5min":1.097,"15min":1.097},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98399999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.1259999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.65600000000000003,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":1.004,"5min":1.0269999999999999,"15min":1.105},"last":0.70799999999999996}]},{"osd":6,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":0.98299999999999998,"15min":1.0389999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.52900000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92300000000000004,"5min":1.03,"15min":1.03},"last":0.72299999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.68899999999999995},"min":{"1min":0.55300000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98099999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.42699999999999999},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0669999999999999},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316232,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055608,"statfs":{"total":96636764160,"available":96312942592,"internally_reserved":0,"allocated":2621440,"data_stored":1323797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73599999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94399999999999995,"15min":0.99199999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73299999999999998,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.91900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.028,"15min":1.028},"last":0.751},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.75600000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90200000000000002,"5min":1.02,"15min":1.02},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.39600000000000002,"5min":0.38100000000000001,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":1.0449999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0029999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.92700000000000005,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.52600000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.873,"5min":1.0429999999999999,"15min":1.044},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:32:00.262 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:32:00.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.444+0000 7fbb9a1ff700 1 Processor -- start 2023-10-08T02:32:00.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.444+0000 7fbb9a1ff700 1 -- start start 2023-10-08T02:32:00.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.445+0000 7fbb9a1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb940ccf90 0x7fbb940ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.445+0000 7fbb9a1ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940cc300 con 0x7fbb940b1420 2023-10-08T02:32:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.445+0000 7fbb9a1ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb940ad370 con 0x7fbb940cbf30 2023-10-08T02:32:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.445+0000 7fbb9a1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94156bf0 con 0x7fbb940ccf90 2023-10-08T02:32:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb92ffd700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbb940b1420 0x7fbb940c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52126/0 (socket says 172.21.15.119:52126) 2023-10-08T02:32:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb92ffd700 1 -- 172.21.15.119:0/3502931468 learned_addr learned my addr 172.21.15.119:0/3502931468 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb93fff700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb940ccf90 0x7fbb940ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885977914 0 0) 0x7fbb940cc300 con 0x7fbb940b1420 2023-10-08T02:32:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb7c003570 con 0x7fbb940b1420 2023-10-08T02:32:00.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361677919 0 0) 0x7fbb940ad370 con 0x7fbb940cbf30 2023-10-08T02:32:00.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb940cc300 con 0x7fbb940cbf30 2023-10-08T02:32:00.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4085585259 0 0) 0x7fbb7c003570 con 0x7fbb940b1420 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb940ccf90 msgr2=0x7fbb940ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb940ccf90 0x7fbb940ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 >> v1:172.21.15.184:6789/0 conn(0x7fbb940cbf30 legacy=0x7fbb940b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb940ad650 con 0x7fbb940b1420 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.446+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7fbb88001470 con 0x7fbb940b1420 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 >> v1:172.21.15.119:6789/0 conn(0x7fbb940b1420 legacy=0x7fbb940c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb927fc700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb7c003c20 0x7fbb7c004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb927fc700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb7c004810 0x7fbb7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb927fc700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb940ccf90 0x7fbb7c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94156bf0 con 0x7fbb7c004810 2023-10-08T02:32:00.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c00bbf0 con 0x7fbb7c003c20 2023-10-08T02:32:00.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c00bdb0 con 0x7fbb940ccf90 2023-10-08T02:32:00.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb92ffd700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb7c004810 0x7fbb7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb93fff700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb940ccf90 0x7fbb7c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb937fe700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb7c003c20 0x7fbb7c004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb92ffd700 1 -- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb940ccf90 msgr2=0x7fbb7c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb92ffd700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb940ccf90 0x7fbb7c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb92ffd700 1 -- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb7c003c20 msgr2=0x7fbb7c004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb92ffd700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb7c003c20 0x7fbb7c004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.447+0000 7fbb92ffd700 1 -- 172.21.15.119:0/3502931468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c003570 con 0x7fbb7c004810 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb92ffd700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb7c004810 0x7fbb7c004c80 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fbb940b1820 tx=0x7fbb88001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0cfd85157d5c0a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8800cbb0 con 0x7fbb7c004810 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8800cd10 con 0x7fbb7c004810 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb927fc700 1 -- 172.21.15.119:0/3502931468 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb88005b70 con 0x7fbb7c004810 2023-10-08T02:32:00.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3502931468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb7c004810 msgr2=0x7fbb7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb9a1ff700 1 --2- 172.21.15.119:0/3502931468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb7c004810 0x7fbb7c004c80 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fbb940b1820 tx=0x7fbb88001d40 comp rx=0 tx=0).stop 2023-10-08T02:32:00.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb93fff700 1 -- 172.21.15.119:0/3502931468 reap_dead start 2023-10-08T02:32:00.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3502931468 shutdown_connections 2023-10-08T02:32:00.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3502931468 >> 172.21.15.119:0/3502931468 conn(0x7fbb9406a320 msgr2=0x7fbb940cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:00.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3502931468 shutdown_connections 2023-10-08T02:32:00.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.448+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3502931468 wait complete. 2023-10-08T02:32:00.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.450+0000 7fbb9a1ff700 1 Processor -- start 2023-10-08T02:32:00.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.450+0000 7fbb9a1ff700 1 -- start start 2023-10-08T02:32:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb9a1ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb7c004810 0x7fbb9416e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb9a1ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb9416a1f0 0x7fbb941511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb9a1ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb94159f50 0x7fbb9415a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb9a1ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9415a930 con 0x7fbb94159f50 2023-10-08T02:32:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb9a1ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb940aa7c0 con 0x7fbb9416a1f0 2023-10-08T02:32:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb9a1ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb9416e720 con 0x7fbb7c004810 2023-10-08T02:32:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb93fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb94159f50 0x7fbb9415a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb93fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb94159f50 0x7fbb9415a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50478/0 (socket says 172.21.15.119:50478) 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb93fff700 1 -- 172.21.15.119:0/3277572557 learned_addr learned my addr 172.21.15.119:0/3277572557 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb937fe700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb7c004810 0x7fbb9416e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb92ffd700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb9416a1f0 0x7fbb941511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb93fff700 1 -- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb7c004810 msgr2=0x7fbb9416e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb93fff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb7c004810 0x7fbb9416e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb93fff700 1 -- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb9416a1f0 msgr2=0x7fbb941511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb93fff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb9416a1f0 0x7fbb941511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb93fff700 1 -- 172.21.15.119:0/3277572557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb940a98d0 con 0x7fbb94159f50 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.451+0000 7fbb92ffd700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb9416a1f0 0x7fbb941511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.452+0000 7fbb93fff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb94159f50 0x7fbb9415a3f0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406d670 tx=0x7fbb8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.452+0000 7fbb7b7fe700 1 -- 172.21.15.119:0/3277572557 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00e040 con 0x7fbb94159f50 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.452+0000 7fbb7b7fe700 1 -- 172.21.15.119:0/3277572557 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8c00f040 con 0x7fbb94159f50 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.452+0000 7fbb7b7fe700 1 -- 172.21.15.119:0/3277572557 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8c00fe20 con 0x7fbb94159f50 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.452+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb940a9bb0 con 0x7fbb94159f50 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.452+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb940aa170 con 0x7fbb94159f50 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.453+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb940aa950 con 0x7fbb94159f50 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.454+0000 7fbb7b7fe700 1 -- 172.21.15.119:0/3277572557 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb8c015640 con 0x7fbb94159f50 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.455+0000 7fbb7b7fe700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb80066f30 0x7fbb800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.455+0000 7fbb7b7fe700 1 -- 172.21.15.119:0/3277572557 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fbb8c084ea0 con 0x7fbb94159f50 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.455+0000 7fbb937fe700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb80066f30 0x7fbb800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.456+0000 7fbb937fe700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb80066f30 0x7fbb800693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb84007a80 tx=0x7fbb84009040 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.459+0000 7fbb7b7fe700 1 -- 172.21.15.119:0/3277572557 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb8c051fd0 con 0x7fbb94159f50 2023-10-08T02:32:00.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.648+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb9404c3c0 con 0x7fbb80066f30 2023-10-08T02:32:00.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.650+0000 7fbb7b7fe700 1 -- 172.21.15.119:0/3277572557 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50600 (secure 0 0 0) 0x7fbb9404c3c0 con 0x7fbb80066f30 2023-10-08T02:32:00.652 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:32:00.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb80066f30 msgr2=0x7fbb800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb80066f30 0x7fbb800693e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbb84007a80 tx=0x7fbb84009040 comp rx=0 tx=0).stop 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb94159f50 msgr2=0x7fbb9415a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb94159f50 0x7fbb9415a3f0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7fbb9406d670 tx=0x7fbb8c00b040 comp rx=0 tx=0).stop 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 shutdown_connections 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fbb80066f30 0x7fbb800693e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fbb7c004810 0x7fbb9416e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb94159f50 0x7fbb9415a3f0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 --2- 172.21.15.119:0/3277572557 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fbb9416a1f0 0x7fbb941511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 >> 172.21.15.119:0/3277572557 conn(0x7fbb9406a320 msgr2=0x7fbb940c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 shutdown_connections 2023-10-08T02:32:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.653+0000 7fbb9a1ff700 1 -- 172.21.15.119:0/3277572557 wait complete. 2023-10-08T02:32:00.655 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:32:00.670 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-10-08T02:31:59.987627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2470344,"kb_used_data":22408,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752504376,"statfs":{"total":773094113280,"available":770564481024,"internally_reserved":0,"allocated":22945792,"data_stored":12492126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668755,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.996381"},"pg_stats":[{"pgid":"1.0","version":"209'370","reported_seq":1221,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-08T02:31:54.057694+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:31:54.057694+0000","last_peered":"2023-10-08T02:31:54.057694+0000","last_clean":"2023-10-08T02:31:54.057694+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:31:54.057694+0000","last_undegraded":"2023-10-08T02:31:54.057694+0000","last_fullsized":"2023-10-08T02:31:54.057694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317140,"kb_used_data":3468,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94054700,"statfs":{"total":96636764160,"available":96312012800,"internally_reserved":0,"allocated":3551232,"data_stored":2242400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93799999999999994,"5min":0.96899999999999997,"15min":1.0349999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85999999999999999,"5min":0.93899999999999995,"15min":0.99199999999999999},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":0.83499999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73299999999999998,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93400000000000005,"5min":0.98499999999999999,"15min":1.002},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.875,"5min":0.96699999999999997,"15min":1.0549999999999999},"last":0.442},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73699999999999999,"15min":0.748},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93000000000000005,"5min":0.96699999999999997,"15min":1.03},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.71699999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.78500000000000003},"min":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98099999999999998,"5min":1.0069999999999999,"15min":1.056},"last":0.63700000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":1.091},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.92200000000000004},"last":0.499}]},{"osd":5,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":1.012},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.47999999999999998,"15min":0.44500000000000001},"max":{"1min":0.873,"5min":0.92100000000000004,"15min":1.1240000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.877,"5min":1.0089999999999999,"15min":1.151},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70299999999999996,"15min":0.71399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.42599999999999999},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299848,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071992,"statfs":{"total":96636764160,"available":96329719808,"internally_reserved":0,"allocated":2621440,"data_stored":1323797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.55100000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92400000000000004,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.67600000000000005}]},{"osd":1,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":1.21},"last":0.749},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76300000000000001,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":3.851,"15min":3.851},"last":0.64000000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":0.86799999999999999,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.258},"last":0.622}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.52600000000000002},"max":{"1min":0.88500000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.378},"max":{"1min":0.95299999999999996,"5min":1.139,"15min":1.139},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.48499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":1.341,"15min":1.341},"last":0.44900000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.621,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.92800000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.443},"max":{"1min":0.91600000000000004,"5min":1.089,"15min":1.089},"last":0.68700000000000006}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.038,"15min":1.0980000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.95599999999999996,"5min":0.96999999999999997,"15min":1.054},"last":0.66400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575987,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312472,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059368,"statfs":{"total":96636764160,"available":96316792832,"internally_reserved":0,"allocated":2965504,"data_stored":1663348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.0780000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.29199999999999998,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.571},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":1.1100000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":1.1299999999999999},"last":0.54100000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.755,"5min":0.80200000000000005,"15min":1.2190000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":1.341,"15min":1.341},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.48199999999999998},"max":{"1min":1.171,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.70499999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.97499999999999998,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.127,"15min":1.27},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.81799999999999995},"min":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.915,"15min":1.915},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.50900000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.204},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.85799999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.306},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.66800000000000004,"5min":0.57999999999999996,"15min":0.55500000000000005},"max":{"1min":1.1399999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.77100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308648,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063192,"statfs":{"total":96636764160,"available":96320708608,"internally_reserved":0,"allocated":3244032,"data_stored":1927766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44700000000000001,"15min":0.46800000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23699999999999999},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.78400000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.76300000000000001,"15min":0.88100000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.48499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.56499999999999995,"5min":0.79400000000000004,"15min":0.83199999999999996},"last":0.48999999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.81799999999999995,"15min":0.93100000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.81399999999999995,"15min":0.89000000000000001},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.75700000000000001,"15min":0.78000000000000003},"min":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.85999999999999999,"5min":1.159,"15min":1.232},"last":0.629},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90500000000000003,"5min":1.306,"15min":1.306},"last":0.72299999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94099999999999995,"5min":1.1990000000000001,"15min":1.216},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98299999999999998,"5min":1.27,"15min":1.296},"last":0.748}]},{"osd":6,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79300000000000004,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.874,"5min":1.135,"15min":1.1579999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96899999999999997,"5min":1.262,"15min":1.262},"last":0.73699999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.83399999999999996},"min":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83199999999999996,"15min":0.86499999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.55700000000000005},"max":{"1min":0.89900000000000002,"5min":1.2250000000000001,"15min":1.3360000000000001},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":316260,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055580,"statfs":{"total":96636764160,"available":96312913920,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61699999999999999,"5min":0.755,"15min":0.871},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.39700000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.70599999999999996,"5min":0.876,"15min":0.876},"last":0.437},{"interface":"front","average":{"1min":0.495,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.80900000000000005},"last":0.49299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.73899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97799999999999998,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0229999999999999,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.751},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82399999999999995,"15min":0.84899999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.353,"15min":1.417},"last":0.80900000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.50900000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94799999999999995,"5min":1.153,"15min":1.333},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94599999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.79300000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.387,"15min":1.4059999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":1.252,"15min":1.5720000000000001},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.495,"15min":0.495},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59099999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.512},"last":0.65600000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.5,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.56000000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.0680000000000001,"5min":1.113,"15min":1.3180000000000001},"last":0.93500000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.57999999999999996,"5min":0.441,"15min":0.441},"max":{"1min":1.117,"5min":1.117,"15min":1.2789999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83499999999999996,"5min":1.1220000000000001,"15min":1.24},"last":0.98899999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89000000000000001,"5min":1.0880000000000001,"15min":1.161},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.95999999999999996,"5min":1.155,"15min":1.2849999999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.82699999999999996},"min":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91600000000000004,"5min":1.081,"15min":1.266},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.282},"last":0.81100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575986,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299868,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071972,"statfs":{"total":96636764160,"available":96329699328,"internally_reserved":0,"allocated":2641920,"data_stored":1336222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.028},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.497,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90800000000000003,"5min":0.998,"15min":1.0409999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.51400000000000001,"5min":0.496,"15min":0.49099999999999999},"max":{"1min":1.01,"5min":1.079,"15min":1.079},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.034},"last":0.621},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.65400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.022,"5min":1.097,"15min":1.097},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98399999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.1259999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.65600000000000003,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":1.004,"5min":1.0269999999999999,"15min":1.105},"last":0.70799999999999996}]},{"osd":6,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":0.98299999999999998,"15min":1.0389999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.52900000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92300000000000004,"5min":1.03,"15min":1.03},"last":0.72299999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.68899999999999995},"min":{"1min":0.55300000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98099999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.42699999999999999},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0669999999999999},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316232,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055608,"statfs":{"total":96636764160,"available":96312942592,"internally_reserved":0,"allocated":2621440,"data_stored":1323797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73599999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94399999999999995,"15min":0.99199999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73299999999999998,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.91900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.028,"15min":1.028},"last":0.751},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.75600000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90200000000000002,"5min":1.02,"15min":1.02},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.39600000000000002,"5min":0.38100000000000001,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":1.0449999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0029999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.92700000000000005,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.52600000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.873,"5min":1.0429999999999999,"15min":1.044},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:32:00.672 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-08T02:32:00.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:32:00.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.853+0000 7f730739d700 1 Processor -- start 2023-10-08T02:32:00.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.854+0000 7f730739d700 1 -- start start 2023-10-08T02:32:00.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f730739d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f7300150550 0x7f7300157770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f730739d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73000c7410 0x7f7300069f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f730739d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730006e2a0 con 0x7f7300150180 2023-10-08T02:32:00.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f730739d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730006b2e0 con 0x7f73000c7410 2023-10-08T02:32:00.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f730739d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73000c78e0 con 0x7f7300150550 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f730639b700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7300150180 0x7f73000c6d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52142/0 (socket says 172.21.15.119:52142) 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f730639b700 1 -- 172.21.15.119:0/3393666708 learned_addr learned my addr 172.21.15.119:0/3393666708 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f7306b9c700 1 --2- 172.21.15.119:0/3393666708 >> v2:172.21.15.119:3300/0 conn(0x7f73000c7410 0x7f7300069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.855+0000 7f7305b9a700 1 --2- 172.21.15.119:0/3393666708 >> v2:172.21.15.184:3300/0 conn(0x7f7300150550 0x7f7300157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7306b9c700 1 -- 172.21.15.119:0/3393666708 >> v2:172.21.15.184:3300/0 conn(0x7f7300150550 msgr2=0x7f7300157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7306b9c700 1 --2- 172.21.15.119:0/3393666708 >> v2:172.21.15.184:3300/0 conn(0x7f7300150550 0x7f7300157770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7306b9c700 1 -- 172.21.15.119:0/3393666708 >> v1:172.21.15.119:6789/0 conn(0x7f7300150180 legacy=0x7f73000c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7306b9c700 1 -- 172.21.15.119:0/3393666708 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000c7bc0 con 0x7f73000c7410 2023-10-08T02:32:00.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7305399700 1 -- 172.21.15.119:0/3393666708 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169717884 0 0) 0x7f730006e2a0 con 0x7f7300150180 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7306b9c700 1 --2- 172.21.15.119:0/3393666708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73000c7410 0x7f7300069f50 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f730006d1e0 tx=0x7f72f8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f211a92b05f36f server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7305399700 1 -- 172.21.15.119:0/3393666708 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f800e070 con 0x7f73000c7410 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7305399700 1 -- 172.21.15.119:0/3393666708 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f800bc30 con 0x7f73000c7410 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.856+0000 7f7305399700 1 -- 172.21.15.119:0/3393666708 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8012430 con 0x7f73000c7410 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.857+0000 7f730739d700 1 -- 172.21.15.119:0/3393666708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73000c7410 msgr2=0x7f7300069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.857+0000 7f730739d700 1 --2- 172.21.15.119:0/3393666708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73000c7410 0x7f7300069f50 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f730006d1e0 tx=0x7f72f8009780 comp rx=0 tx=0).stop 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.857+0000 7f730739d700 1 -- 172.21.15.119:0/3393666708 shutdown_connections 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.857+0000 7f730739d700 1 --2- 172.21.15.119:0/3393666708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73000c7410 0x7f7300069f50 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.857+0000 7f730739d700 1 --2- 172.21.15.119:0/3393666708 >> v2:172.21.15.184:3300/0 conn(0x7f7300150550 0x7f7300157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.857+0000 7f730739d700 1 -- 172.21.15.119:0/3393666708 >> 172.21.15.119:0/3393666708 conn(0x7f73000cc1f0 msgr2=0x7f73000cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:00.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.857+0000 7f730739d700 1 -- 172.21.15.119:0/3393666708 shutdown_connections 2023-10-08T02:32:00.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.857+0000 7f730739d700 1 -- 172.21.15.119:0/3393666708 wait complete. 2023-10-08T02:32:00.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.858+0000 7f730739d700 1 Processor -- start 2023-10-08T02:32:00.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.859+0000 7f730739d700 1 -- start start 2023-10-08T02:32:00.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.859+0000 7f730739d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73000c7410 0x7f730014ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.859+0000 7f730739d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7300150180 0x7f730014ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.859+0000 7f730739d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300150550 0x7f73000c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f730739d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73000af130 con 0x7f7300150180 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f730739d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73000af410 con 0x7f73000c7410 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f730739d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73000af6f0 con 0x7f7300150550 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7300150180 0x7f730014ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7300150180 0x7f730014ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50492/0 (socket says 172.21.15.119:50492) 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 -- 172.21.15.119:0/3409649585 learned_addr learned my addr 172.21.15.119:0/3409649585 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f730639b700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73000c7410 0x7f730014ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7306b9c700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300150550 0x7f73000c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 -- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300150550 msgr2=0x7f73000c1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300150550 0x7f73000c1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 -- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73000c7410 msgr2=0x7f730014ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:00.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73000c7410 0x7f730014ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 -- 172.21.15.119:0/3409649585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f8009a20 con 0x7f7300150180 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7306b9c700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300150550 0x7f73000c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f730639b700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73000c7410 0x7f730014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.860+0000 7f7305b9a700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7300150180 0x7f730014ae10 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f72f000d890 tx=0x7f72f0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.861+0000 7f72f67fc700 1 -- 172.21.15.119:0/3409649585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f000b600 con 0x7f7300150180 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.861+0000 7f72f67fc700 1 -- 172.21.15.119:0/3409649585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f000b760 con 0x7f7300150180 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.861+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73000af9d0 con 0x7f7300150180 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.861+0000 7f72f67fc700 1 -- 172.21.15.119:0/3409649585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f000bae0 con 0x7f7300150180 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.861+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730014b350 con 0x7f7300150180 2023-10-08T02:32:00.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.862+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730004c3c0 con 0x7f7300150180 2023-10-08T02:32:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.863+0000 7f72f67fc700 1 -- 172.21.15.119:0/3409649585 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f72f001e460 con 0x7f7300150180 2023-10-08T02:32:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.864+0000 7f72f67fc700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f72e4066f30 0x7f72e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.864+0000 7f72f67fc700 1 -- 172.21.15.119:0/3409649585 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f72f0084dc0 con 0x7f7300150180 2023-10-08T02:32:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.864+0000 7f730639b700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f72e4066f30 0x7f72e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:00.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.865+0000 7f730639b700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f72e4066f30 0x7f72e40693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f72fc001890 tx=0x7f72fc009040 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:00.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:00.868+0000 7f72f67fc700 1 -- 172.21.15.119:0/3409649585 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f72f0051e50 con 0x7f7300150180 2023-10-08T02:32:01.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.054+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f730006ada0 con 0x7f72e4066f30 2023-10-08T02:32:01.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.056+0000 7f72f67fc700 1 -- 172.21.15.119:0/3409649585 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50600 (secure 0 0 0) 0x7f730006ada0 con 0x7f72e4066f30 2023-10-08T02:32:01.058 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:32:01.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f72e4066f30 msgr2=0x7f72e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f72e4066f30 0x7f72e40693e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f72fc001890 tx=0x7f72fc009040 comp rx=0 tx=0).stop 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7300150180 msgr2=0x7f730014ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7300150180 0x7f730014ae10 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f72f000d890 tx=0x7f72f0008040 comp rx=0 tx=0).stop 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 shutdown_connections 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f72e4066f30 0x7f72e40693e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7300150550 0x7f73000c1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73000c7410 0x7f730014ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 --2- 172.21.15.119:0/3409649585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7300150180 0x7f730014ae10 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 >> 172.21.15.119:0/3409649585 conn(0x7f73000cc1f0 msgr2=0x7f7300157ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 shutdown_connections 2023-10-08T02:32:01.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.059+0000 7f730739d700 1 -- 172.21.15.119:0/3409649585 wait complete. 2023-10-08T02:32:01.061 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:32:01.077 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-10-08T02:31:59.987627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":2470344,"kb_used_data":22408,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752504376,"statfs":{"total":773094113280,"available":770564481024,"internally_reserved":0,"allocated":22945792,"data_stored":12492126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668755,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-317,"num_read_kb":-281,"num_write":-281,"num_write_kb":-2052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.996381"},"pg_stats":[{"pgid":"1.0","version":"209'370","reported_seq":1221,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-08T02:31:54.057694+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:31:54.057694+0000","last_peered":"2023-10-08T02:31:54.057694+0000","last_clean":"2023-10-08T02:31:54.057694+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:31:54.057694+0000","last_undegraded":"2023-10-08T02:31:54.057694+0000","last_fullsized":"2023-10-08T02:31:54.057694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575987,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317140,"kb_used_data":3468,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94054700,"statfs":{"total":96636764160,"available":96312012800,"internally_reserved":0,"allocated":3551232,"data_stored":2242400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93799999999999994,"5min":0.96899999999999997,"15min":1.0349999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85999999999999999,"5min":0.93899999999999995,"15min":0.99199999999999999},"last":0.47699999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":0.83499999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73299999999999998,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93400000000000005,"5min":0.98499999999999999,"15min":1.002},"last":0.53700000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.875,"5min":0.96699999999999997,"15min":1.0549999999999999},"last":0.442},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73699999999999999,"15min":0.748},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93000000000000005,"5min":0.96699999999999997,"15min":1.03},"last":0.52000000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.71699999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.78500000000000003},"min":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98099999999999998,"5min":1.0069999999999999,"15min":1.056},"last":0.63700000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":1.091},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.92200000000000004},"last":0.499}]},{"osd":5,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":1.012},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.47999999999999998,"15min":0.44500000000000001},"max":{"1min":0.873,"5min":0.92100000000000004,"15min":1.1240000000000001},"last":0.60299999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.877,"5min":1.0089999999999999,"15min":1.151},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70299999999999996,"15min":0.71399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.42599999999999999},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.59599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299848,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071992,"statfs":{"total":96636764160,"available":96329719808,"internally_reserved":0,"allocated":2621440,"data_stored":1323797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.55100000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92400000000000004,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.67600000000000005}]},{"osd":1,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":1.21},"last":0.749},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76300000000000001,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":3.851,"15min":3.851},"last":0.64000000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":0.86799999999999999,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.258},"last":0.622}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.52600000000000002},"max":{"1min":0.88500000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.57399999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.378},"max":{"1min":0.95299999999999996,"5min":1.139,"15min":1.139},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.48499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":1.341,"15min":1.341},"last":0.44900000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.621,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.92800000000000005},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.443},"max":{"1min":0.91600000000000004,"5min":1.089,"15min":1.089},"last":0.68700000000000006}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.038,"15min":1.0980000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.95599999999999996,"5min":0.96999999999999997,"15min":1.054},"last":0.66400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575987,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312472,"kb_used_data":2896,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059368,"statfs":{"total":96636764160,"available":96316792832,"internally_reserved":0,"allocated":2965504,"data_stored":1663348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.0780000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.29199999999999998,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.571},"last":0.40999999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":1.1100000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":1.1299999999999999},"last":0.54100000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.755,"5min":0.80200000000000005,"15min":1.2190000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":1.341,"15min":1.341},"last":0.60499999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.48199999999999998},"max":{"1min":1.171,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.70499999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.97499999999999998,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.127,"15min":1.27},"last":0.75600000000000001}]},{"osd":6,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.81799999999999995},"min":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.915,"15min":1.915},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.50900000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.204},"last":0.63700000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.85799999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.306},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.66800000000000004,"5min":0.57999999999999996,"15min":0.55500000000000005},"max":{"1min":1.1399999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.77100000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308648,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063192,"statfs":{"total":96636764160,"available":96320708608,"internally_reserved":0,"allocated":3244032,"data_stored":1927766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44700000000000001,"15min":0.46800000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23699999999999999},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.78400000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.76300000000000001,"15min":0.88100000000000001},"last":0.39600000000000002}]},{"osd":2,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.48499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.56499999999999995,"5min":0.79400000000000004,"15min":0.83199999999999996},"last":0.48999999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.81799999999999995,"15min":0.93100000000000005},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.81399999999999995,"15min":0.89000000000000001},"last":0.44800000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.75700000000000001,"15min":0.78000000000000003},"min":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.85999999999999999,"5min":1.159,"15min":1.232},"last":0.629},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90500000000000003,"5min":1.306,"15min":1.306},"last":0.72299999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94099999999999995,"5min":1.1990000000000001,"15min":1.216},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98299999999999998,"5min":1.27,"15min":1.296},"last":0.748}]},{"osd":6,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79300000000000004,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.874,"5min":1.135,"15min":1.1579999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96899999999999997,"5min":1.262,"15min":1.262},"last":0.73699999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.83399999999999996},"min":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83199999999999996,"15min":0.86499999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.55700000000000005},"max":{"1min":0.89900000000000002,"5min":1.2250000000000001,"15min":1.3360000000000001},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":316260,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055580,"statfs":{"total":96636764160,"available":96312913920,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61699999999999999,"5min":0.755,"15min":0.871},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.39700000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.70599999999999996,"5min":0.876,"15min":0.876},"last":0.437},{"interface":"front","average":{"1min":0.495,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.80900000000000005},"last":0.49299999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.73899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.41899999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97799999999999998,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68100000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0229999999999999,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.751},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82399999999999995,"15min":0.84899999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.353,"15min":1.417},"last":0.80900000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.50900000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94799999999999995,"5min":1.153,"15min":1.333},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94599999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.79300000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.387,"15min":1.4059999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":1.252,"15min":1.5720000000000001},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.495,"15min":0.495},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.59099999999999997}]},{"osd":1,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.61099999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.512},"last":0.65600000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.5,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.56000000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.0680000000000001,"5min":1.113,"15min":1.3180000000000001},"last":0.93500000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.57999999999999996,"5min":0.441,"15min":0.441},"max":{"1min":1.117,"5min":1.117,"15min":1.2789999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83499999999999996,"5min":1.1220000000000001,"15min":1.24},"last":0.98899999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89000000000000001,"5min":1.0880000000000001,"15min":1.161},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.95999999999999996,"5min":1.155,"15min":1.2849999999999999},"last":0.89900000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.82699999999999996},"min":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91600000000000004,"5min":1.081,"15min":1.266},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.282},"last":0.81100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575986,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299868,"kb_used_data":2580,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071972,"statfs":{"total":96636764160,"available":96329699328,"internally_reserved":0,"allocated":2641920,"data_stored":1336222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.028},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.497,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90800000000000003,"5min":0.998,"15min":1.0409999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.51400000000000001,"5min":0.496,"15min":0.49099999999999999},"max":{"1min":1.01,"5min":1.079,"15min":1.079},"last":0.68400000000000005}]},{"osd":2,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.034},"last":0.621},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.65400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.022,"5min":1.097,"15min":1.097},"last":0.60499999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98399999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.1259999999999999},"last":0.746},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.65600000000000003,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":1.004,"5min":1.0269999999999999,"15min":1.105},"last":0.70799999999999996}]},{"osd":6,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":0.98299999999999998,"15min":1.0389999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.52900000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92300000000000004,"5min":1.03,"15min":1.03},"last":0.72299999999999998}]},{"osd":7,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.68899999999999995},"min":{"1min":0.55300000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98099999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.42699999999999999},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0669999999999999},"last":0.67600000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575986,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316232,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055608,"statfs":{"total":96636764160,"available":96312942592,"internally_reserved":0,"allocated":2621440,"data_stored":1323797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.79100000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73599999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94399999999999995,"15min":0.99199999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73299999999999998,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.91900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.028,"15min":1.028},"last":0.751},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.75600000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.64700000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90200000000000002,"5min":1.02,"15min":1.02},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.76400000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.39600000000000002,"5min":0.38100000000000001,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":1.0449999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0029999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.92700000000000005,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.65700000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.52600000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.873,"5min":1.0429999999999999,"15min":1.044},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:32:01.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-08T02:32:01.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.258+0000 7f9ecbbff700 1 Processor -- start 2023-10-08T02:32:01.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.258+0000 7f9ecbbff700 1 -- start start 2023-10-08T02:32:01.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.259+0000 7f9ecbbff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9ec4159890 0x7f9ec415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:01.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.259+0000 7f9ecbbff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9ec4159c60 0x7f9ec415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:01.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.259+0000 7f9ecbbff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ec415a160 con 0x7f9ec415db10 2023-10-08T02:32:01.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.259+0000 7f9ecbbff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec415df50 con 0x7f9ec4159890 2023-10-08T02:32:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.259+0000 7f9ecbbff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ec40d13a0 con 0x7f9ec4159c60 2023-10-08T02:32:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec999b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9ec4159890 0x7f9ec415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec999b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9ec4159890 0x7f9ec415abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50506/0 (socket says 172.21.15.119:50506) 2023-10-08T02:32:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9eca19c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9ec415db10 0x7f9ec4069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52152/0 (socket says 172.21.15.119:52152) 2023-10-08T02:32:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec999b700 1 -- 172.21.15.119:0/2671357967 learned_addr learned my addr 172.21.15.119:0/2671357967 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:01.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec919a700 1 --2- 172.21.15.119:0/2671357967 >> v2:172.21.15.184:3300/0 conn(0x7f9ec4159c60 0x7f9ec415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:01.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec999b700 1 -- 172.21.15.119:0/2671357967 >> v2:172.21.15.184:3300/0 conn(0x7f9ec4159c60 msgr2=0x7f9ec415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:01.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec999b700 1 --2- 172.21.15.119:0/2671357967 >> v2:172.21.15.184:3300/0 conn(0x7f9ec4159c60 0x7f9ec415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec999b700 1 -- 172.21.15.119:0/2671357967 >> v1:172.21.15.119:6789/0 conn(0x7f9ec415db10 legacy=0x7f9ec4069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec999b700 1 -- 172.21.15.119:0/2671357967 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec415e140 con 0x7f9ec4159890 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.260+0000 7f9ec999b700 1 --2- 172.21.15.119:0/2671357967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec4159890 0x7f9ec415abd0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40b67f0 tx=0x7f9ec0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a7e8674c77e86d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ec8999700 1 -- 172.21.15.119:0/2671357967 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000b860 con 0x7f9ec4159890 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ec8999700 1 -- 172.21.15.119:0/2671357967 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec000b9c0 con 0x7f9ec4159890 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ec8999700 1 -- 172.21.15.119:0/2671357967 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000bd60 con 0x7f9ec4159890 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ecbbff700 1 -- 172.21.15.119:0/2671357967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec4159890 msgr2=0x7f9ec415abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/2671357967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec4159890 0x7f9ec415abd0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40b67f0 tx=0x7f9ec0009250 comp rx=0 tx=0).stop 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ecbbff700 1 -- 172.21.15.119:0/2671357967 shutdown_connections 2023-10-08T02:32:01.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/2671357967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec4159890 0x7f9ec415abd0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/2671357967 >> v2:172.21.15.184:3300/0 conn(0x7f9ec4159c60 0x7f9ec415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ecbbff700 1 -- 172.21.15.119:0/2671357967 >> 172.21.15.119:0/2671357967 conn(0x7f9ec4068e20 msgr2=0x7f9ec40d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:01.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ecbbff700 1 -- 172.21.15.119:0/2671357967 shutdown_connections 2023-10-08T02:32:01.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.261+0000 7f9ecbbff700 1 -- 172.21.15.119:0/2671357967 wait complete. 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.263+0000 7f9ecbbff700 1 Processor -- start 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.263+0000 7f9ecbbff700 1 -- start start 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9ecbbff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec4159890 0x7f9ec40c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9ecbbff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec4159c60 0x7f9ec4158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9ecbbff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec415db10 0x7f9ec40c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9ecbbff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40b41c0 con 0x7f9ec415db10 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9ecbbff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec40b4350 con 0x7f9ec4159890 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9ecbbff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec4158bb0 con 0x7f9ec4159c60 2023-10-08T02:32:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9eca19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec415db10 0x7f9ec40c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9eca19c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec415db10 0x7f9ec40c9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50514/0 (socket says 172.21.15.119:50514) 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9eca19c700 1 -- 172.21.15.119:0/1808381594 learned_addr learned my addr 172.21.15.119:0/1808381594 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9ec919a700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec4159c60 0x7f9ec4158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9ec999b700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec4159890 0x7f9ec40c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9eca19c700 1 -- 172.21.15.119:0/1808381594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec4159c60 msgr2=0x7f9ec4158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9eca19c700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec4159c60 0x7f9ec4158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9eca19c700 1 -- 172.21.15.119:0/1808381594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec4159890 msgr2=0x7f9ec40c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9eca19c700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec4159890 0x7f9ec40c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.264+0000 7f9eca19c700 1 -- 172.21.15.119:0/1808381594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec00094f0 con 0x7f9ec415db10 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.265+0000 7f9eca19c700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec415db10 0x7f9ec40c9fe0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00cca0 tx=0x7f9ebc001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.265+0000 7f9eb9ffb700 1 -- 172.21.15.119:0/1808381594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc005df0 con 0x7f9ec415db10 2023-10-08T02:32:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.265+0000 7f9eb9ffb700 1 -- 172.21.15.119:0/1808381594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ebc00a5f0 con 0x7f9ec415db10 2023-10-08T02:32:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.265+0000 7f9eb9ffb700 1 -- 172.21.15.119:0/1808381594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ebc00a970 con 0x7f9ec415db10 2023-10-08T02:32:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.265+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec40c8890 con 0x7f9ec415db10 2023-10-08T02:32:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.265+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec40c8e50 con 0x7f9ec415db10 2023-10-08T02:32:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.266+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ec404c3c0 con 0x7f9ec415db10 2023-10-08T02:32:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.267+0000 7f9eb9ffb700 1 -- 172.21.15.119:0/1808381594 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9ebc00ad20 con 0x7f9ec415db10 2023-10-08T02:32:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.268+0000 7f9eb9ffb700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ea8066f30 0x7f9ea80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.268+0000 7f9eb9ffb700 1 -- 172.21.15.119:0/1808381594 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f9ebc084660 con 0x7f9ec415db10 2023-10-08T02:32:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.268+0000 7f9ec999b700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ea8066f30 0x7f9ea80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.269+0000 7f9ec999b700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ea8066f30 0x7f9ea80693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40b67f0 tx=0x7f9ec0017000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:01.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.272+0000 7f9eb9ffb700 1 -- 172.21.15.119:0/1808381594 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ebc051770 con 0x7f9ec415db10 2023-10-08T02:32:01.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.464+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ec40b4490 con 0x7f9ec415db10 2023-10-08T02:32:01.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.465+0000 7f9eb9ffb700 1 -- 172.21.15.119:0/1808381594 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+13939 (secure 0 0 0) 0x7f9ebc054d90 con 0x7f9ec415db10 2023-10-08T02:32:01.466 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:32:01.466 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":211,"fsid":"260cffbb-ac40-4cab-945b-d7e76ccdf950","created":"2023-10-08T00:55:23.131125+0000","modified":"2023-10-08T02:31:56.361060+0000","last_up_change":"2023-10-08T00:55:35.225248+0000","last_in_change":"2023-10-08T00:55:26.441992+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-08T00:55:35.909253+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"a5097618-d43f-4d26-b009-6a2b9340a46c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6810","nonce":85787},{"type":"v1","addr":"172.21.15.119:6812","nonce":85787}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6815","nonce":85787},{"type":"v1","addr":"172.21.15.119:6818","nonce":85787}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6822","nonce":85787},{"type":"v1","addr":"172.21.15.119:6823","nonce":85787}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":85787},{"type":"v1","addr":"172.21.15.119:6821","nonce":85787}]},"public_addr":"172.21.15.119:6812/85787","cluster_addr":"172.21.15.119:6818/85787","heartbeat_back_addr":"172.21.15.119:6823/85787","heartbeat_front_addr":"172.21.15.119:6821/85787","state":["exists","up"]},{"osd":1,"uuid":"9d357ab1-5e58-4e2f-8c87-f5049b795080","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":85785},{"type":"v1","addr":"172.21.15.119:6801","nonce":85785}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":85785},{"type":"v1","addr":"172.21.15.119:6803","nonce":85785}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":85785},{"type":"v1","addr":"172.21.15.119:6807","nonce":85785}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":85785},{"type":"v1","addr":"172.21.15.119:6805","nonce":85785}]},"public_addr":"172.21.15.119:6801/85785","cluster_addr":"172.21.15.119:6803/85785","heartbeat_back_addr":"172.21.15.119:6807/85785","heartbeat_front_addr":"172.21.15.119:6805/85785","state":["exists","up"]},{"osd":2,"uuid":"e7ae1773-1b32-4fa9-9ccc-a984771510e5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":85796},{"type":"v1","addr":"172.21.15.119:6825","nonce":85796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":85796},{"type":"v1","addr":"172.21.15.119:6827","nonce":85796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":85796},{"type":"v1","addr":"172.21.15.119:6831","nonce":85796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":85796},{"type":"v1","addr":"172.21.15.119:6829","nonce":85796}]},"public_addr":"172.21.15.119:6825/85796","cluster_addr":"172.21.15.119:6827/85796","heartbeat_back_addr":"172.21.15.119:6831/85796","heartbeat_front_addr":"172.21.15.119:6829/85796","state":["exists","up"]},{"osd":3,"uuid":"08734d45-7b33-4bbc-8931-0e74277a0b6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":85788},{"type":"v1","addr":"172.21.15.119:6809","nonce":85788}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6811","nonce":85788},{"type":"v1","addr":"172.21.15.119:6813","nonce":85788}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":85788},{"type":"v1","addr":"172.21.15.119:6819","nonce":85788}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6814","nonce":85788},{"type":"v1","addr":"172.21.15.119:6816","nonce":85788}]},"public_addr":"172.21.15.119:6809/85788","cluster_addr":"172.21.15.119:6813/85788","heartbeat_back_addr":"172.21.15.119:6819/85788","heartbeat_front_addr":"172.21.15.119:6816/85788","state":["exists","up"]},{"osd":4,"uuid":"96b1be81-a6ef-46fc-b9db-7b666d515607","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6824","nonce":86003},{"type":"v1","addr":"172.21.15.184:6825","nonce":86003}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6826","nonce":86003},{"type":"v1","addr":"172.21.15.184:6827","nonce":86003}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6830","nonce":86003},{"type":"v1","addr":"172.21.15.184:6831","nonce":86003}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6828","nonce":86003},{"type":"v1","addr":"172.21.15.184:6829","nonce":86003}]},"public_addr":"172.21.15.184:6825/86003","cluster_addr":"172.21.15.184:6827/86003","heartbeat_back_addr":"172.21.15.184:6831/86003","heartbeat_front_addr":"172.21.15.184:6829/86003","state":["exists","up"]},{"osd":5,"uuid":"44d73378-7e35-4961-8f78-c27cd700d909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6816","nonce":86004},{"type":"v1","addr":"172.21.15.184:6817","nonce":86004}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6818","nonce":86004},{"type":"v1","addr":"172.21.15.184:6819","nonce":86004}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6822","nonce":86004},{"type":"v1","addr":"172.21.15.184:6823","nonce":86004}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6820","nonce":86004},{"type":"v1","addr":"172.21.15.184:6821","nonce":86004}]},"public_addr":"172.21.15.184:6817/86004","cluster_addr":"172.21.15.184:6819/86004","heartbeat_back_addr":"172.21.15.184:6823/86004","heartbeat_front_addr":"172.21.15.184:6821/86004","state":["exists","up"]},{"osd":6,"uuid":"138ed202-bc1d-4bd7-934a-784b5cb23b3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6808","nonce":86002},{"type":"v1","addr":"172.21.15.184:6809","nonce":86002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6810","nonce":86002},{"type":"v1","addr":"172.21.15.184:6811","nonce":86002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6814","nonce":86002},{"type":"v1","addr":"172.21.15.184:6815","nonce":86002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6812","nonce":86002},{"type":"v1","addr":"172.21.15.184:6813","nonce":86002}]},"public_addr":"172.21.15.184:6809/86002","cluster_addr":"172.21.15.184:6811/86002","heartbeat_back_addr":"172.21.15.184:6815/86002","heartbeat_front_addr":"172.21.15.184:6813/86002","state":["exists","up"]},{"osd":7,"uuid":"4b904447-2c07-4602-b89e-f3d50025a783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":204,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6800","nonce":86001},{"type":"v1","addr":"172.21.15.184:6801","nonce":86001}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6802","nonce":86001},{"type":"v1","addr":"172.21.15.184:6803","nonce":86001}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6806","nonce":86001},{"type":"v1","addr":"172.21.15.184:6807","nonce":86001}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6804","nonce":86001},{"type":"v1","addr":"172.21.15.184:6805","nonce":86001}]},"public_addr":"172.21.15.184:6801/86001","cluster_addr":"172.21.15.184:6803/86001","heartbeat_back_addr":"172.21.15.184:6807/86001","heartbeat_front_addr":"172.21.15.184:6805/86001","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.782298+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.749816+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.759558+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.669304+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.774405+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.674016+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.713580+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-08T00:55:31.781942+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/3602020849":"2023-10-09T02:31:53.966925+0000","172.21.15.119:0/927931328":"2023-10-09T02:31:53.966925+0000","172.21.15.119:0/2124875896":"2023-10-09T02:31:53.966925+0000","172.21.15.119:0/2189804149":"2023-10-09T02:31:53.966925+0000","172.21.15.119:0/10097412":"2023-10-09T02:31:53.966925+0000","172.21.15.119:6835/1741909431":"2023-10-09T02:31:52.963764+0000","172.21.15.119:0/3003317145":"2023-10-09T02:31:53.966925+0000","172.21.15.119:6834/1741909431":"2023-10-09T02:31:52.963764+0000","172.21.15.119:0/842982126":"2023-10-09T02:31:43.569085+0000","172.21.15.119:6837/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/3912047418":"2023-10-09T02:31:53.966925+0000","172.21.15.119:6836/134108":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/1887277487":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/4191951118":"2023-10-09T02:31:43.569085+0000","172.21.15.119:0/773312007":"2023-10-09T02:31:43.569085+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":9,"length":1},{"begin":14,"length":1},{"begin":19,"length":1},{"begin":24,"length":1},{"begin":31,"length":1},{"begin":36,"length":1},{"begin":42,"length":1},{"begin":47,"length":1},{"begin":53,"length":1},{"begin":58,"length":1},{"begin":63,"length":1},{"begin":68,"length":2},{"begin":74,"length":1},{"begin":80,"length":1},{"begin":85,"length":1},{"begin":91,"length":1},{"begin":96,"length":1},{"begin":102,"length":1},{"begin":107,"length":1},{"begin":112,"length":2},{"begin":117,"length":1},{"begin":213,"length":23}]},{"pool":7,"snaps":[{"begin":2,"length":4}]},{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-08T02:32:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ea8066f30 msgr2=0x7f9ea80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ea8066f30 0x7f9ea80693e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9ec40b67f0 tx=0x7f9ec0017000 comp rx=0 tx=0).stop 2023-10-08T02:32:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec415db10 msgr2=0x7f9ec40c9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec415db10 0x7f9ec40c9fe0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc00cca0 tx=0x7f9ebc001a70 comp rx=0 tx=0).stop 2023-10-08T02:32:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 shutdown_connections 2023-10-08T02:32:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9ea8066f30 0x7f9ea80693e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ec415db10 0x7f9ec40c9fe0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9ec4159890 0x7f9ec40c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 --2- 172.21.15.119:0/1808381594 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9ec4159c60 0x7f9ec4158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 >> 172.21.15.119:0/1808381594 conn(0x7f9ec4068e20 msgr2=0x7f9ec40c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 shutdown_connections 2023-10-08T02:32:01.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:01.468+0000 7f9ecbbff700 1 -- 172.21.15.119:0/1808381594 wait complete. 2023-10-08T02:32:02.487 INFO:tasks.ceph:Scrubbing osd.0 2023-10-08T02:32:02.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-08T02:32:02.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.674+0000 7fa634fff700 1 Processor -- start 2023-10-08T02:32:02.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.675+0000 7fa634fff700 1 -- start start 2023-10-08T02:32:02.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa634fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fa63015a440 0x7fa6300acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa634fff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa630068890 0x7fa6300c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa634fff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa63015a9b0 con 0x7fa6301563b0 2023-10-08T02:32:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa634fff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63015ac90 con 0x7fa630068890 2023-10-08T02:32:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa634fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63015af70 con 0x7fa63015a440 2023-10-08T02:32:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa62ed9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa630068890 0x7fa6300c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa62ed9d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa630068890 0x7fa6300c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50522/0 (socket says 172.21.15.119:50522) 2023-10-08T02:32:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa62ed9d700 1 -- 172.21.15.119:0/1653518685 learned_addr learned my addr 172.21.15.119:0/1653518685 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:02.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa62dd9b700 1 --2- 172.21.15.119:0/1653518685 >> v2:172.21.15.184:3300/0 conn(0x7fa63015a440 0x7fa6300acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa62ed9d700 1 -- 172.21.15.119:0/1653518685 >> v2:172.21.15.184:3300/0 conn(0x7fa63015a440 msgr2=0x7fa6300acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.676+0000 7fa62ed9d700 1 --2- 172.21.15.119:0/1653518685 >> v2:172.21.15.184:3300/0 conn(0x7fa63015a440 0x7fa6300acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa62ed9d700 1 -- 172.21.15.119:0/1653518685 >> v1:172.21.15.184:6789/0 conn(0x7fa6301563b0 legacy=0x7fa630156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa62ed9d700 1 -- 172.21.15.119:0/1653518685 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63015b250 con 0x7fa630068890 2023-10-08T02:32:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa62dd9b700 1 --2- 172.21.15.119:0/1653518685 >> v2:172.21.15.184:3300/0 conn(0x7fa63015a440 0x7fa6300acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa62ed9d700 1 --2- 172.21.15.119:0/1653518685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa630068890 0x7fa6300c8230 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fa63006fb10 tx=0x7fa628009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5516aca7357edc server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa62d59a700 1 -- 172.21.15.119:0/1653518685 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62800e070 con 0x7fa630068890 2023-10-08T02:32:02.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa62d59a700 1 -- 172.21.15.119:0/1653518685 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62800ba50 con 0x7fa630068890 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa62d59a700 1 -- 172.21.15.119:0/1653518685 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6280092e0 con 0x7fa630068890 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa634fff700 1 -- 172.21.15.119:0/1653518685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa630068890 msgr2=0x7fa6300c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.677+0000 7fa634fff700 1 --2- 172.21.15.119:0/1653518685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa630068890 0x7fa6300c8230 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fa63006fb10 tx=0x7fa628009840 comp rx=0 tx=0).stop 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.678+0000 7fa634fff700 1 -- 172.21.15.119:0/1653518685 shutdown_connections 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.678+0000 7fa634fff700 1 --2- 172.21.15.119:0/1653518685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa630068890 0x7fa6300c8230 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.678+0000 7fa634fff700 1 --2- 172.21.15.119:0/1653518685 >> v2:172.21.15.184:3300/0 conn(0x7fa63015a440 0x7fa6300acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.678+0000 7fa634fff700 1 -- 172.21.15.119:0/1653518685 >> 172.21.15.119:0/1653518685 conn(0x7fa6300c8f90 msgr2=0x7fa6300c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.678+0000 7fa634fff700 1 -- 172.21.15.119:0/1653518685 shutdown_connections 2023-10-08T02:32:02.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.678+0000 7fa634fff700 1 -- 172.21.15.119:0/1653518685 wait complete. 2023-10-08T02:32:02.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.679+0000 7fa634fff700 1 Processor -- start 2023-10-08T02:32:02.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.679+0000 7fa634fff700 1 -- start start 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.680+0000 7fa634fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630068890 0x7fa63016cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.680+0000 7fa634fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6301563b0 0x7fa630156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.680+0000 7fa634fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa63015a440 0x7fa6300ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.680+0000 7fa634fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa630158570 con 0x7fa6301563b0 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.680+0000 7fa634fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6301578f0 con 0x7fa63015a440 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.680+0000 7fa634fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa630157bd0 con 0x7fa630068890 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.680+0000 7fa62dd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6301563b0 0x7fa630156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.680+0000 7fa62dd9b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6301563b0 0x7fa630156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50528/0 (socket says 172.21.15.119:50528) 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62dd9b700 1 -- 172.21.15.119:0/2579773670 learned_addr learned my addr 172.21.15.119:0/2579773670 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62e59c700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630068890 0x7fa63016cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62ed9d700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa63015a440 0x7fa6300ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62dd9b700 1 -- 172.21.15.119:0/2579773670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630068890 msgr2=0x7fa63016cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62dd9b700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630068890 0x7fa63016cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62dd9b700 1 -- 172.21.15.119:0/2579773670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa63015a440 msgr2=0x7fa6300ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62dd9b700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fa63015a440 0x7fa6300ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62dd9b700 1 -- 172.21.15.119:0/2579773670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa628009ae0 con 0x7fa6301563b0 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62e59c700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fa630068890 0x7fa63016cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa62dd9b700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6301563b0 0x7fa630156f30 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d860 tx=0x7fa624008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6240125f0 con 0x7fa6301563b0 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa624012750 con 0x7fa6301563b0 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa624012ad0 con 0x7fa6301563b0 2023-10-08T02:32:02.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.681+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa630157eb0 con 0x7fa6301563b0 2023-10-08T02:32:02.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.682+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63016d0c0 con 0x7fa6301563b0 2023-10-08T02:32:02.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.682+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6300718b0 con 0x7fa6301563b0 2023-10-08T02:32:02.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.683+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa624014430 con 0x7fa6301563b0 2023-10-08T02:32:02.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.684+0000 7fa61e7fc700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa610066f60 0x7fa610069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.684+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fa624084b10 con 0x7fa6301563b0 2023-10-08T02:32:02.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.685+0000 7fa62e59c700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa610066f60 0x7fa610069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.685+0000 7fa61e7fc700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7fa61006d700 0x7fa61006fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.685+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 --> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6100701a0 con 0x7fa61006d700 2023-10-08T02:32:02.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.685+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=211) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa624089220 con 0x7fa6301563b0 2023-10-08T02:32:02.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.685+0000 7fa62ed9d700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7fa61006d700 0x7fa61006fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.685+0000 7fa62e59c700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa610066f60 0x7fa610069410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa620007fd0 tx=0x7fa620009040 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:02.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.686+0000 7fa62ed9d700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7fa61006d700 0x7fa61006fb70 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:02.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.686+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 <== osd.0 v2:172.21.15.119:6810/85787 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa6100701a0 con 0x7fa61006d700 2023-10-08T02:32:02.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.700+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 --> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa63006d6a0 con 0x7fa61006d700 2023-10-08T02:32:02.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa61e7fc700 1 -- 172.21.15.119:0/2579773670 <== osd.0 v2:172.21.15.119:6810/85787 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa63006d6a0 con 0x7fa61006d700 2023-10-08T02:32:02.701 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:32:02.702 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T02:32:02.702 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7fa61006d700 msgr2=0x7fa61006fb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6810/85787,v1:172.21.15.119:6812/85787] conn(0x7fa61006d700 0x7fa61006fb70 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa610066f60 msgr2=0x7fa610069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fa610066f60 0x7fa610069410 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fa620007fd0 tx=0x7fa620009040 comp rx=0 tx=0).stop 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6301563b0 msgr2=0x7fa630156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 --2- 172.21.15.119:0/2579773670 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6301563b0 0x7fa630156f30 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fa62400d860 tx=0x7fa624008040 comp rx=0 tx=0).stop 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa62ed9d700 1 -- 172.21.15.119:0/2579773670 reap_dead start 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 shutdown_connections 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 >> 172.21.15.119:0/2579773670 conn(0x7fa6300c8f90 msgr2=0x7fa630068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.701+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 shutdown_connections 2023-10-08T02:32:02.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.702+0000 7fa634fff700 1 -- 172.21.15.119:0/2579773670 wait complete. 2023-10-08T02:32:02.719 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-08T02:32:02.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.901+0000 7fb7fedff700 1 Processor -- start 2023-10-08T02:32:02.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.901+0000 7fb7fedff700 1 -- start start 2023-10-08T02:32:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7fedff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f8155c10 0x7fb7f8155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7fedff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7f80ac230 0x7fb7f80ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7fedff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f80ad930 con 0x7fb7f81568a0 2023-10-08T02:32:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7fedff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f80acd20 con 0x7fb7f80ac230 2023-10-08T02:32:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7fedff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f80ad000 con 0x7fb7f8155c10 2023-10-08T02:32:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7fcb9b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f8155c10 0x7fb7f8155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7effff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7f80ac230 0x7fb7f80ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7effff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb7f80ac230 0x7fb7f80ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:46866/0 (socket says 172.21.15.119:46866) 2023-10-08T02:32:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.903+0000 7fb7effff700 1 -- 172.21.15.119:0/2354170415 learned_addr learned my addr 172.21.15.119:0/2354170415 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7fcb9b700 1 -- 172.21.15.119:0/2354170415 >> v2:172.21.15.184:3300/0 conn(0x7fb7f80ac230 msgr2=0x7fb7f80ac6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7fcb9b700 1 --2- 172.21.15.119:0/2354170415 >> v2:172.21.15.184:3300/0 conn(0x7fb7f80ac230 0x7fb7f80ac6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7fcb9b700 1 -- 172.21.15.119:0/2354170415 >> v1:172.21.15.184:6789/0 conn(0x7fb7f81568a0 legacy=0x7fb7f8156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7fcb9b700 1 -- 172.21.15.119:0/2354170415 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f80ad2e0 con 0x7fb7f8155c10 2023-10-08T02:32:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7effff700 1 --2- 172.21.15.119:0/2354170415 >> v2:172.21.15.184:3300/0 conn(0x7fb7f80ac230 0x7fb7f80ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:32:02.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7fcb9b700 1 --2- 172.21.15.119:0/2354170415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f8155c10 0x7fb7f8155fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40098e0 tx=0x7fb7f40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eef80ef703607fa8 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:02.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7ef7fe700 1 -- 172.21.15.119:0/2354170415 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400b640 con 0x7fb7f8155c10 2023-10-08T02:32:02.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7ef7fe700 1 -- 172.21.15.119:0/2354170415 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f400b7a0 con 0x7fb7f8155c10 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.904+0000 7fb7ef7fe700 1 -- 172.21.15.119:0/2354170415 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400bb40 con 0x7fb7f8155c10 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.905+0000 7fb7fedff700 1 -- 172.21.15.119:0/2354170415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f8155c10 msgr2=0x7fb7f8155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.905+0000 7fb7fedff700 1 --2- 172.21.15.119:0/2354170415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f8155c10 0x7fb7f8155fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40098e0 tx=0x7fb7f40091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.905+0000 7fb7fedff700 1 -- 172.21.15.119:0/2354170415 shutdown_connections 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.905+0000 7fb7fedff700 1 --2- 172.21.15.119:0/2354170415 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f8155c10 0x7fb7f8155fe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.905+0000 7fb7fedff700 1 --2- 172.21.15.119:0/2354170415 >> v2:172.21.15.184:3300/0 conn(0x7fb7f80ac230 0x7fb7f80ac6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.905+0000 7fb7fedff700 1 -- 172.21.15.119:0/2354170415 >> 172.21.15.119:0/2354170415 conn(0x7fb7f806a320 msgr2=0x7fb7f80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.905+0000 7fb7fedff700 1 -- 172.21.15.119:0/2354170415 shutdown_connections 2023-10-08T02:32:02.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.905+0000 7fb7fedff700 1 -- 172.21.15.119:0/2354170415 wait complete. 2023-10-08T02:32:02.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.906+0000 7fb7fedff700 1 Processor -- start 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.907+0000 7fb7fedff700 1 -- start start 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.907+0000 7fb7fedff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f80ac230 0x7fb7f81587e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.907+0000 7fb7fedff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7f8155c10 0x7fb7f806e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.907+0000 7fb7fedff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f81568a0 0x7fb7f8156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7fedff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8158350 con 0x7fb7f8155c10 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7fedff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8158d20 con 0x7fb7f80ac230 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7fedff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f8157740 con 0x7fb7f81568a0 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7f8155c10 0x7fb7f806e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7f8155c10 0x7fb7f806e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50530/0 (socket says 172.21.15.119:50530) 2023-10-08T02:32:02.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 -- 172.21.15.119:0/3519409663 learned_addr learned my addr 172.21.15.119:0/3519409663 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7fcb9b700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f80ac230 0x7fb7f81587e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7fd39c700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f81568a0 0x7fb7f8156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 -- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f81568a0 msgr2=0x7fb7f8156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f81568a0 0x7fb7f8156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 -- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f80ac230 msgr2=0x7fb7f81587e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f80ac230 0x7fb7f81587e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 -- 172.21.15.119:0/3519409663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e800b780 con 0x7fb7f8155c10 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7fcb9b700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f80ac230 0x7fb7f81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.908+0000 7fb7effff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7f8155c10 0x7fb7f806e830 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800def0 tx=0x7fb7e800f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.909+0000 7fb7ecff9700 1 -- 172.21.15.119:0/3519409663 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8009cb0 con 0x7fb7f8155c10 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.909+0000 7fb7ecff9700 1 -- 172.21.15.119:0/3519409663 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e8009e10 con 0x7fb7f8155c10 2023-10-08T02:32:02.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.909+0000 7fb7ecff9700 1 -- 172.21.15.119:0/3519409663 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8017620 con 0x7fb7f8155c10 2023-10-08T02:32:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.909+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f4009420 con 0x7fb7f8155c10 2023-10-08T02:32:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.909+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f8157d00 con 0x7fb7f8155c10 2023-10-08T02:32:02.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.910+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f81584c0 con 0x7fb7f8155c10 2023-10-08T02:32:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.911+0000 7fb7ecff9700 1 -- 172.21.15.119:0/3519409663 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7e80179d0 con 0x7fb7f8155c10 2023-10-08T02:32:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.911+0000 7fb7ecff9700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb7e4066f30 0x7fb7e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.911+0000 7fb7ecff9700 1 -- 172.21.15.119:0/3519409663 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fb7e8084b60 con 0x7fb7f8155c10 2023-10-08T02:32:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.912+0000 7fb7fcb9b700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb7e4066f30 0x7fb7e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.912+0000 7fb7fcb9b700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb7e4066f30 0x7fb7e40693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40093f0 tx=0x7fb7f4017000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:02.916+0000 7fb7ecff9700 1 -- 172.21.15.119:0/3519409663 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7e8055720 con 0x7fb7f8155c10 2023-10-08T02:32:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.105+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7f804c3c0 con 0x7fb7e4066f30 2023-10-08T02:32:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.106+0000 7fb7ecff9700 1 -- 172.21.15.119:0/3519409663 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb7f804c3c0 con 0x7fb7e4066f30 2023-10-08T02:32:03.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb7e4066f30 msgr2=0x7fb7e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb7e4066f30 0x7fb7e40693e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb7f40093f0 tx=0x7fb7f4017000 comp rx=0 tx=0).stop 2023-10-08T02:32:03.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7f8155c10 msgr2=0x7fb7f806e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7f8155c10 0x7fb7f806e830 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fb7e800def0 tx=0x7fb7e800f040 comp rx=0 tx=0).stop 2023-10-08T02:32:03.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 shutdown_connections 2023-10-08T02:32:03.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb7e4066f30 0x7fb7e40693e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb7f81568a0 0x7fb7f8156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb7f80ac230 0x7fb7f81587e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 --2- 172.21.15.119:0/3519409663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7f8155c10 0x7fb7f806e830 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 >> 172.21.15.119:0/3519409663 conn(0x7fb7f806a320 msgr2=0x7fb7f8153120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 shutdown_connections 2023-10-08T02:32:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.108+0000 7fb7fedff700 1 -- 172.21.15.119:0/3519409663 wait complete. 2023-10-08T02:32:03.110 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 0 to deep-scrub 2023-10-08T02:32:03.126 INFO:tasks.ceph:Scrubbing osd.1 2023-10-08T02:32:03.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-08T02:32:03.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.310+0000 7f77a55ff700 1 Processor -- start 2023-10-08T02:32:03.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.310+0000 7f77a55ff700 1 -- start start 2023-10-08T02:32:03.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.311+0000 7f77a55ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77a015a440 0x7f77a00acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.311+0000 7f77a55ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a0068890 0x7f77a00c8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.311+0000 7f77a55ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a015a9b0 con 0x7f77a01563b0 2023-10-08T02:32:03.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.311+0000 7f77a55ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a015ac90 con 0x7f77a015a440 2023-10-08T02:32:03.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.311+0000 7f77a55ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a015af70 con 0x7f77a0068890 2023-10-08T02:32:03.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779e7fc700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77a015a440 0x7f77a00acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779f7fe700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a0068890 0x7f77a00c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779e7fc700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f77a015a440 0x7f77a00acf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:46886/0 (socket says 172.21.15.119:46886) 2023-10-08T02:32:03.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779e7fc700 1 -- 172.21.15.119:0/672713811 learned_addr learned my addr 172.21.15.119:0/672713811 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779dffb700 1 -- 172.21.15.119:0/672713811 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732249227 0 0) 0x7f77a015a9b0 con 0x7f77a01563b0 2023-10-08T02:32:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779dffb700 1 -- 172.21.15.119:0/672713811 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7788003570 con 0x7f77a01563b0 2023-10-08T02:32:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779f7fe700 1 -- 172.21.15.119:0/672713811 >> v2:172.21.15.184:3300/0 conn(0x7f77a015a440 msgr2=0x7f77a00acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779f7fe700 1 --2- 172.21.15.119:0/672713811 >> v2:172.21.15.184:3300/0 conn(0x7f77a015a440 0x7f77a00acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779f7fe700 1 -- 172.21.15.119:0/672713811 >> v1:172.21.15.184:6789/0 conn(0x7f77a01563b0 legacy=0x7f77a0156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779f7fe700 1 -- 172.21.15.119:0/672713811 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a015b250 con 0x7f77a0068890 2023-10-08T02:32:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.312+0000 7f779e7fc700 1 --2- 172.21.15.119:0/672713811 >> v2:172.21.15.184:3300/0 conn(0x7f77a015a440 0x7f77a00acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:32:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.313+0000 7f779f7fe700 1 --2- 172.21.15.119:0/672713811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a0068890 0x7f77a00c8230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f77980098e0 tx=0x7f77980091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=845b2d0c6a6a7cc6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.313+0000 7f779dffb700 1 -- 172.21.15.119:0/672713811 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798011630 con 0x7f77a0068890 2023-10-08T02:32:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.313+0000 7f779dffb700 1 -- 172.21.15.119:0/672713811 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798011790 con 0x7f77a0068890 2023-10-08T02:32:03.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.313+0000 7f779dffb700 1 -- 172.21.15.119:0/672713811 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798011b30 con 0x7f77a0068890 2023-10-08T02:32:03.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.314+0000 7f77a55ff700 1 -- 172.21.15.119:0/672713811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a0068890 msgr2=0x7f77a00c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.314+0000 7f77a55ff700 1 --2- 172.21.15.119:0/672713811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a0068890 0x7f77a00c8230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f77980098e0 tx=0x7f77980091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.314+0000 7f77a55ff700 1 -- 172.21.15.119:0/672713811 shutdown_connections 2023-10-08T02:32:03.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.314+0000 7f77a55ff700 1 --2- 172.21.15.119:0/672713811 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a0068890 0x7f77a00c8230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.314+0000 7f77a55ff700 1 --2- 172.21.15.119:0/672713811 >> v2:172.21.15.184:3300/0 conn(0x7f77a015a440 0x7f77a00acf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.314+0000 7f77a55ff700 1 -- 172.21.15.119:0/672713811 >> 172.21.15.119:0/672713811 conn(0x7f77a00c8f90 msgr2=0x7f77a00c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.314+0000 7f77a55ff700 1 -- 172.21.15.119:0/672713811 shutdown_connections 2023-10-08T02:32:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.314+0000 7f77a55ff700 1 -- 172.21.15.119:0/672713811 wait complete. 2023-10-08T02:32:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.315+0000 7f77a55ff700 1 Processor -- start 2023-10-08T02:32:03.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.315+0000 7f77a55ff700 1 -- start start 2023-10-08T02:32:03.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f77a55ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a0068890 0x7f77a00b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f77a55ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a01563b0 0x7f77a014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f77a55ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a015a440 0x7f77a014bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f77a55ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a00cb060 con 0x7f77a01563b0 2023-10-08T02:32:03.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f77a55ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a00ca4e0 con 0x7f77a0068890 2023-10-08T02:32:03.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f77a55ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a00ca7c0 con 0x7f77a015a440 2023-10-08T02:32:03.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f779e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a01563b0 0x7f77a014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f779e7fc700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a01563b0 0x7f77a014d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50542/0 (socket says 172.21.15.119:50542) 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f779e7fc700 1 -- 172.21.15.119:0/3414386601 learned_addr learned my addr 172.21.15.119:0/3414386601 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f779f7fe700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a015a440 0x7f77a014bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.316+0000 7f779effd700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a0068890 0x7f77a00b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f779e7fc700 1 -- 172.21.15.119:0/3414386601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a015a440 msgr2=0x7f77a014bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f779e7fc700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f77a015a440 0x7f77a014bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f779e7fc700 1 -- 172.21.15.119:0/3414386601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a0068890 msgr2=0x7f77a00b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f779e7fc700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f77a0068890 0x7f77a00b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f779e7fc700 1 -- 172.21.15.119:0/3414386601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940094f0 con 0x7f77a01563b0 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f779e7fc700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a01563b0 0x7f77a014d1c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f779400ec50 tx=0x7f7794009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779400c850 con 0x7f77a01563b0 2023-10-08T02:32:03.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779400c9b0 con 0x7f77a01563b0 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779400cd50 con 0x7f77a01563b0 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.317+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7798009420 con 0x7f77a01563b0 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.318+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a00cad80 con 0x7f77a01563b0 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.318+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f77a00718b0 con 0x7f77a01563b0 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.319+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7794013680 con 0x7f77a01563b0 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.320+0000 7f7786ffd700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f778c066f30 0x7f778c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.321+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f7794084780 con 0x7f77a01563b0 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.321+0000 7f779effd700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f778c066f30 0x7f778c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.321+0000 7f7786ffd700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f778c06d6d0 0x7f778c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.321+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 --> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f778c070170 con 0x7f778c06d6d0 2023-10-08T02:32:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.321+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=211) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7794086c80 con 0x7f77a01563b0 2023-10-08T02:32:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.321+0000 7f779f7fe700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f778c06d6d0 0x7f778c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.321+0000 7f779effd700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f778c066f30 0x7f778c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7790004140 tx=0x7f7790007f90 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.322+0000 7f779f7fe700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f778c06d6d0 0x7f778c06fb40 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.322+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 <== osd.1 v2:172.21.15.119:6800/85785 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f778c070170 con 0x7f778c06d6d0 2023-10-08T02:32:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.336+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 --> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f77a006d6a0 con 0x7f778c06d6d0 2023-10-08T02:32:03.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f7786ffd700 1 -- 172.21.15.119:0/3414386601 <== osd.1 v2:172.21.15.119:6800/85785 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f77a006d6a0 con 0x7f778c06d6d0 2023-10-08T02:32:03.338 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:32:03.338 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T02:32:03.338 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:32:03.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f778c06d6d0 msgr2=0x7f778c06fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6800/85785,v1:172.21.15.119:6801/85785] conn(0x7f778c06d6d0 0x7f778c06fb40 crc :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f778c066f30 msgr2=0x7f778c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f778c066f30 0x7f778c0693e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7790004140 tx=0x7f7790007f90 comp rx=0 tx=0).stop 2023-10-08T02:32:03.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a01563b0 msgr2=0x7f77a014d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 --2- 172.21.15.119:0/3414386601 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77a01563b0 0x7f77a014d1c0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f779400ec50 tx=0x7f7794009250 comp rx=0 tx=0).stop 2023-10-08T02:32:03.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f779f7fe700 1 -- 172.21.15.119:0/3414386601 reap_dead start 2023-10-08T02:32:03.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 shutdown_connections 2023-10-08T02:32:03.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 >> 172.21.15.119:0/3414386601 conn(0x7f77a00c8f90 msgr2=0x7f77a01572c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:03.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.337+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 shutdown_connections 2023-10-08T02:32:03.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.338+0000 7f77a55ff700 1 -- 172.21.15.119:0/3414386601 wait complete. 2023-10-08T02:32:03.355 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-08T02:32:03.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.537+0000 7fd1ef7ff700 1 Processor -- start 2023-10-08T02:32:03.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.537+0000 7fd1ef7ff700 1 -- start start 2023-10-08T02:32:03.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.538+0000 7fd1ef7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 0x7fd1e80ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.538+0000 7fd1ef7ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80cc300 con 0x7fd1e80b1420 2023-10-08T02:32:03.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.538+0000 7fd1ef7ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e80ad370 con 0x7fd1e80ccf90 2023-10-08T02:32:03.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.538+0000 7fd1ef7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8156bf0 con 0x7fd1e80cbf30 2023-10-08T02:32:03.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1edd9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd1e80b1420 0x7fd1e80c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52154/0 (socket says 172.21.15.119:52154) 2023-10-08T02:32:03.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1edd9c700 1 -- 172.21.15.119:0/683173920 learned_addr learned my addr 172.21.15.119:0/683173920 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:03.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1ed59b700 1 --2- 172.21.15.119:0/683173920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 0x7fd1e80ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1dffff700 1 -- 172.21.15.119:0/683173920 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184121477 0 0) 0x7fd1e80cc300 con 0x7fd1e80b1420 2023-10-08T02:32:03.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1dffff700 1 -- 172.21.15.119:0/683173920 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1d4003570 con 0x7fd1e80b1420 2023-10-08T02:32:03.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1dffff700 1 -- 172.21.15.119:0/683173920 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4028365246 0 0) 0x7fd1d4003570 con 0x7fd1e80b1420 2023-10-08T02:32:03.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1ed59b700 1 -- 172.21.15.119:0/683173920 >> v1:172.21.15.184:6789/0 conn(0x7fd1e80ccf90 legacy=0x7fd1e80b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1ed59b700 1 -- 172.21.15.119:0/683173920 >> v1:172.21.15.119:6789/0 conn(0x7fd1e80b1420 legacy=0x7fd1e80c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.539+0000 7fd1ed59b700 1 -- 172.21.15.119:0/683173920 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e80ad650 con 0x7fd1e80cbf30 2023-10-08T02:32:03.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.540+0000 7fd1ed59b700 1 --2- 172.21.15.119:0/683173920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 0x7fd1e80ae410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009910 tx=0x7fd1e40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=566bb72219dcc25 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.540+0000 7fd1dffff700 1 -- 172.21.15.119:0/683173920 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e00017f0 con 0x7fd1e80cbf30 2023-10-08T02:32:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.540+0000 7fd1dffff700 1 -- 172.21.15.119:0/683173920 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4011420 con 0x7fd1e80cbf30 2023-10-08T02:32:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.540+0000 7fd1dffff700 1 -- 172.21.15.119:0/683173920 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e40117c0 con 0x7fd1e80cbf30 2023-10-08T02:32:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.540+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/683173920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 msgr2=0x7fd1e80ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.540+0000 7fd1ef7ff700 1 --2- 172.21.15.119:0/683173920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 0x7fd1e80ae410 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4009910 tx=0x7fd1e40091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.541+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/683173920 shutdown_connections 2023-10-08T02:32:03.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.541+0000 7fd1ef7ff700 1 --2- 172.21.15.119:0/683173920 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 0x7fd1e80ae410 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.541+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/683173920 >> 172.21.15.119:0/683173920 conn(0x7fd1e806a320 msgr2=0x7fd1e80cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.541+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/683173920 shutdown_connections 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.541+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/683173920 wait complete. 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.542+0000 7fd1ef7ff700 1 Processor -- start 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.542+0000 7fd1ef7ff700 1 -- start start 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ef7ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e80b1420 0x7fd1e80c1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ef7ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 0x7fd1e81686e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ef7ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e80ccf90 0x7fd1e80c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ef7ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80c2ca0 con 0x7fd1e80b1420 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ef7ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e8168c20 con 0x7fd1e80ccf90 2023-10-08T02:32:03.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ef7ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e80c30c0 con 0x7fd1e80cbf30 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ed59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e80b1420 0x7fd1e80c1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ed59b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e80b1420 0x7fd1e80c1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50544/0 (socket says 172.21.15.119:50544) 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.543+0000 7fd1ed59b700 1 -- 172.21.15.119:0/2965225306 learned_addr learned my addr 172.21.15.119:0/2965225306 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1ed59b700 1 -- 172.21.15.119:0/2965225306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 msgr2=0x7fd1e81686e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1ed59b700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 0x7fd1e81686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1ed59b700 1 -- 172.21.15.119:0/2965225306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e80ccf90 msgr2=0x7fd1e80c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1ed59b700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e80ccf90 0x7fd1e80c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1ed59b700 1 -- 172.21.15.119:0/2965225306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e4009420 con 0x7fd1e80b1420 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1ed59b700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e80b1420 0x7fd1e80c1940 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40093f0 tx=0x7fd1e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1dd7fa700 1 -- 172.21.15.119:0/2965225306 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4010be0 con 0x7fd1e80b1420 2023-10-08T02:32:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1dd7fa700 1 -- 172.21.15.119:0/2965225306 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e4010d40 con 0x7fd1e80b1420 2023-10-08T02:32:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1dd7fa700 1 -- 172.21.15.119:0/2965225306 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e401d640 con 0x7fd1e80b1420 2023-10-08T02:32:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1e80c2020 con 0x7fd1e80b1420 2023-10-08T02:32:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.544+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e80c25e0 con 0x7fd1e80b1420 2023-10-08T02:32:03.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.545+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e80c2de0 con 0x7fd1e80b1420 2023-10-08T02:32:03.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.546+0000 7fd1dd7fa700 1 -- 172.21.15.119:0/2965225306 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd1e401d9b0 con 0x7fd1e80b1420 2023-10-08T02:32:03.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.547+0000 7fd1dd7fa700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd1cc066f30 0x7fd1cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.547+0000 7fd1dd7fa700 1 -- 172.21.15.119:0/2965225306 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fd1e4086280 con 0x7fd1e80b1420 2023-10-08T02:32:03.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.547+0000 7fd1ecd9a700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd1cc066f30 0x7fd1cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.548+0000 7fd1ecd9a700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd1cc066f30 0x7fd1cc0693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80cb780 tx=0x7fd1d8007b40 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.551+0000 7fd1dd7fa700 1 -- 172.21.15.119:0/2965225306 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd1e40533b0 con 0x7fd1e80b1420 2023-10-08T02:32:03.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.743+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1e804c3c0 con 0x7fd1cc066f30 2023-10-08T02:32:03.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.744+0000 7fd1dd7fa700 1 -- 172.21.15.119:0/2965225306 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd1e804c3c0 con 0x7fd1cc066f30 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd1cc066f30 msgr2=0x7fd1cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd1cc066f30 0x7fd1cc0693e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd1e80cb780 tx=0x7fd1d8007b40 comp rx=0 tx=0).stop 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e80b1420 msgr2=0x7fd1e80c1940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e80b1420 0x7fd1e80c1940 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7fd1e40093f0 tx=0x7fd1e4017000 comp rx=0 tx=0).stop 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 shutdown_connections 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fd1cc066f30 0x7fd1cc0693e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fd1e80ccf90 0x7fd1e80c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1e80b1420 0x7fd1e80c1940 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 --2- 172.21.15.119:0/2965225306 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fd1e80cbf30 0x7fd1e81686e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 >> 172.21.15.119:0/2965225306 conn(0x7fd1e806a320 msgr2=0x7fd1e8154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 shutdown_connections 2023-10-08T02:32:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.746+0000 7fd1ef7ff700 1 -- 172.21.15.119:0/2965225306 wait complete. 2023-10-08T02:32:03.748 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 1 to deep-scrub 2023-10-08T02:32:03.764 INFO:tasks.ceph:Scrubbing osd.2 2023-10-08T02:32:03.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-08T02:32:03.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.948+0000 7f0f5a3ff700 1 Processor -- start 2023-10-08T02:32:03.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.948+0000 7f0f5a3ff700 1 -- start start 2023-10-08T02:32:03.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f5a3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5415a440 0x7f0f540acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f5a3ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5415a9b0 con 0x7f0f541563b0 2023-10-08T02:32:03.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f5a3ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5415ac90 con 0x7f0f54068890 2023-10-08T02:32:03.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f5a3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5415af70 con 0x7f0f5415a440 2023-10-08T02:32:03.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f53fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0f541563b0 0x7f0f54156780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52164/0 (socket says 172.21.15.119:52164) 2023-10-08T02:32:03.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f53fff700 1 -- 172.21.15.119:0/1793606634 learned_addr learned my addr 172.21.15.119:0/1793606634 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:03.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032341154 0 0) 0x7f0f5415a9b0 con 0x7f0f541563b0 2023-10-08T02:32:03.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f4bfff700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5415a440 0x7f0f540acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.950+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f40003570 con 0x7f0f541563b0 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3695519801 0 0) 0x7f0f40003570 con 0x7f0f541563b0 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5415a440 msgr2=0x7f0f540acf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5415a440 0x7f0f540acf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 >> v1:172.21.15.184:6789/0 conn(0x7f0f54068890 legacy=0x7f0f540c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5415b250 con 0x7f0f541563b0 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3562518676 0 0) 0x7f0f44001470 con 0x7f0f541563b0 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 >> v1:172.21.15.119:6789/0 conn(0x7f0f541563b0 legacy=0x7f0f54156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f40003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5415a440 0x7f0f40004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f40004d50 0x7f0f400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5415af70 con 0x7f0f40003a90 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4000bb10 con 0x7f0f40004d50 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4000bcd0 con 0x7f0f5415a440 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f53fff700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f40003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f4bfff700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5415a440 0x7f0f40004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f5899c700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f40004d50 0x7f0f400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f53fff700 1 -- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5415a440 msgr2=0x7f0f40004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f53fff700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f5415a440 0x7f0f40004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f53fff700 1 -- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f40004d50 msgr2=0x7f0f400051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f53fff700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f40004d50 0x7f0f400051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.951+0000 7f0f53fff700 1 -- 172.21.15.119:0/1793606634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f40003570 con 0x7f0f40003a90 2023-10-08T02:32:03.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f53fff700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f40003e60 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c3c0 tx=0x7f0f44001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07ca1dc0addc3f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4400cc90 con 0x7f0f40003a90 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4400cdf0 con 0x7f0f40003a90 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f537fe700 1 -- 172.21.15.119:0/1793606634 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f44005b70 con 0x7f0f40003a90 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/1793606634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 msgr2=0x7f0f40003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f5a3ff700 1 --2- 172.21.15.119:0/1793606634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f40003e60 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c3c0 tx=0x7f0f44001e70 comp rx=0 tx=0).stop 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f5899c700 1 -- 172.21.15.119:0/1793606634 reap_dead start 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/1793606634 shutdown_connections 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.952+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/1793606634 >> 172.21.15.119:0/1793606634 conn(0x7f0f540c8f90 msgr2=0x7f0f540c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.953+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/1793606634 shutdown_connections 2023-10-08T02:32:03.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.953+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/1793606634 wait complete. 2023-10-08T02:32:03.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.954+0000 7f0f5a3ff700 1 Processor -- start 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.954+0000 7f0f5a3ff700 1 -- start start 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f5a3ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f5414f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f5a3ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5414f810 0x7f0f540acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f5a3ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540ad430 0x7f0f541590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f5a3ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f540ad8d0 con 0x7f0f40003a90 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f5a3ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54159d90 con 0x7f0f5414f810 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f5a3ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54157040 con 0x7f0f540ad430 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f53fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f5414f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f53fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f5414f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50568/0 (socket says 172.21.15.119:50568) 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f53fff700 1 -- 172.21.15.119:0/2235717544 learned_addr learned my addr 172.21.15.119:0/2235717544 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:03.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f4bfff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5414f810 0x7f0f540acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f5899c700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540ad430 0x7f0f541590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f53fff700 1 -- 172.21.15.119:0/2235717544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540ad430 msgr2=0x7f0f541590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f53fff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540ad430 0x7f0f541590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f53fff700 1 -- 172.21.15.119:0/2235717544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5414f810 msgr2=0x7f0f540acef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f53fff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f0f5414f810 0x7f0f540acef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f53fff700 1 -- 172.21.15.119:0/2235717544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54157320 con 0x7f0f40003a90 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.955+0000 7f0f5899c700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f0f540ad430 0x7f0f541590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.956+0000 7f0f53fff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f5414f2d0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c9b0 tx=0x7f0f44015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.956+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f440070d0 con 0x7f0f40003a90 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.956+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f44007230 con 0x7f0f40003a90 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.956+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f54157600 con 0x7f0f40003a90 2023-10-08T02:32:03.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.956+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f440075f0 con 0x7f0f40003a90 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.956+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f54169410 con 0x7f0f40003a90 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.957+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f540718b0 con 0x7f0f40003a90 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.958+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0f44019030 con 0x7f0f40003a90 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.959+0000 7f0f50ff9700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0f34066f30 0x7f0f340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.959+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f0f44085e60 con 0x7f0f40003a90 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.959+0000 7f0f4bfff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0f34066f30 0x7f0f340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.959+0000 7f0f50ff9700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f0f3406d6d0 0x7f0f3406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.959+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 --> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f34070170 con 0x7f0f3406d6d0 2023-10-08T02:32:03.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.959+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=211) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f44042ea0 con 0x7f0f40003a90 2023-10-08T02:32:03.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.959+0000 7f0f5899c700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f0f3406d6d0 0x7f0f3406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:03.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.960+0000 7f0f4bfff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0f34066f30 0x7f0f340693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c006dc0 tx=0x7f0f3c008040 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.960+0000 7f0f5899c700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f0f3406d6d0 0x7f0f3406fb40 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:03.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.960+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 <== osd.2 v2:172.21.15.119:6824/85796 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0f34070170 con 0x7f0f3406d6d0 2023-10-08T02:32:03.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.975+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 --> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0f540ad830 con 0x7f0f3406d6d0 2023-10-08T02:32:03.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.975+0000 7f0f50ff9700 1 -- 172.21.15.119:0/2235717544 <== osd.2 v2:172.21.15.119:6824/85796 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0f540ad830 con 0x7f0f3406d6d0 2023-10-08T02:32:03.976 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:32:03.976 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T02:32:03.976 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:32:03.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f0f3406d6d0 msgr2=0x7f0f3406fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6824/85796,v1:172.21.15.119:6825/85796] conn(0x7f0f3406d6d0 0x7f0f3406fb40 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:03.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0f34066f30 msgr2=0x7f0f340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0f34066f30 0x7f0f340693e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c006dc0 tx=0x7f0f3c008040 comp rx=0 tx=0).stop 2023-10-08T02:32:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 msgr2=0x7f0f5414f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 --2- 172.21.15.119:0/2235717544 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f40003a90 0x7f0f5414f2d0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c9b0 tx=0x7f0f44015000 comp rx=0 tx=0).stop 2023-10-08T02:32:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5899c700 1 -- 172.21.15.119:0/2235717544 reap_dead start 2023-10-08T02:32:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 shutdown_connections 2023-10-08T02:32:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 >> 172.21.15.119:0/2235717544 conn(0x7f0f540c8f90 msgr2=0x7f0f5406e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 shutdown_connections 2023-10-08T02:32:03.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:03.976+0000 7f0f5a3ff700 1 -- 172.21.15.119:0/2235717544 wait complete. 2023-10-08T02:32:03.993 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-08T02:32:04.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.174+0000 7fdc315ff700 1 Processor -- start 2023-10-08T02:32:04.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.174+0000 7fdc315ff700 1 -- start start 2023-10-08T02:32:04.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.175+0000 7fdc315ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc2c0cbf30 0x7fdc2c0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.175+0000 7fdc315ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fdc2c0b1420 0x7fdc2c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.175+0000 7fdc315ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc2c0cc300 con 0x7fdc2c0ccf90 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.175+0000 7fdc315ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c156bf0 con 0x7fdc2c0cbf30 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.175+0000 7fdc315ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc2c0ae950 con 0x7fdc2c0b1420 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc2c0cbf30 0x7fdc2c0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2affd700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc2c0cbf30 0x7fdc2c0ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50574/0 (socket says 172.21.15.119:50574) 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2affd700 1 -- 172.21.15.119:0/3869282831 learned_addr learned my addr 172.21.15.119:0/3869282831 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2a7fc700 1 --1- 172.21.15.119:0/3869282831 >> v1:172.21.15.119:6789/0 conn(0x7fdc2c0ccf90 0x7fdc2c0b0d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52176/0 (socket says 172.21.15.119:52176) 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2b7fe700 1 --2- 172.21.15.119:0/3869282831 >> v2:172.21.15.184:3300/0 conn(0x7fdc2c0b1420 0x7fdc2c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2affd700 1 -- 172.21.15.119:0/3869282831 >> v2:172.21.15.184:3300/0 conn(0x7fdc2c0b1420 msgr2=0x7fdc2c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2affd700 1 --2- 172.21.15.119:0/3869282831 >> v2:172.21.15.184:3300/0 conn(0x7fdc2c0b1420 0x7fdc2c0c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2affd700 1 -- 172.21.15.119:0/3869282831 >> v1:172.21.15.119:6789/0 conn(0x7fdc2c0ccf90 legacy=0x7fdc2c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc29ffb700 1 -- 172.21.15.119:0/3869282831 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839381028 0 0) 0x7fdc2c0cc300 con 0x7fdc2c0ccf90 2023-10-08T02:32:04.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2affd700 1 -- 172.21.15.119:0/3869282831 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0ad370 con 0x7fdc2c0cbf30 2023-10-08T02:32:04.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.176+0000 7fdc2affd700 1 --2- 172.21.15.119:0/3869282831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0cbf30 0x7fdc2c0ae410 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0b19e0 tx=0x7fdc1c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cf09edadefb6d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc29ffb700 1 -- 172.21.15.119:0/3869282831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00b820 con 0x7fdc2c0cbf30 2023-10-08T02:32:04.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc29ffb700 1 -- 172.21.15.119:0/3869282831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c00b980 con 0x7fdc2c0cbf30 2023-10-08T02:32:04.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc29ffb700 1 -- 172.21.15.119:0/3869282831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00bd20 con 0x7fdc2c0cbf30 2023-10-08T02:32:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc315ff700 1 -- 172.21.15.119:0/3869282831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0cbf30 msgr2=0x7fdc2c0ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc315ff700 1 --2- 172.21.15.119:0/3869282831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0cbf30 0x7fdc2c0ae410 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c0b19e0 tx=0x7fdc1c009250 comp rx=0 tx=0).stop 2023-10-08T02:32:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc315ff700 1 -- 172.21.15.119:0/3869282831 shutdown_connections 2023-10-08T02:32:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc315ff700 1 --2- 172.21.15.119:0/3869282831 >> v2:172.21.15.184:3300/0 conn(0x7fdc2c0b1420 0x7fdc2c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc315ff700 1 --2- 172.21.15.119:0/3869282831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0cbf30 0x7fdc2c0ae410 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.177+0000 7fdc315ff700 1 -- 172.21.15.119:0/3869282831 >> 172.21.15.119:0/3869282831 conn(0x7fdc2c06a320 msgr2=0x7fdc2c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:04.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.178+0000 7fdc315ff700 1 -- 172.21.15.119:0/3869282831 shutdown_connections 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.178+0000 7fdc315ff700 1 -- 172.21.15.119:0/3869282831 wait complete. 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.179+0000 7fdc315ff700 1 Processor -- start 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.179+0000 7fdc315ff700 1 -- start start 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc315ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0b1420 0x7fdc2c1426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc315ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc2c0cbf30 0x7fdc2c0a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc315ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc2c0ccf90 0x7fdc2c0aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc315ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0a8f40 con 0x7fdc2c0b1420 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc315ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c142fa0 con 0x7fdc2c0ccf90 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc315ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c0a8330 con 0x7fdc2c0cbf30 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0b1420 0x7fdc2c1426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc2affd700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0b1420 0x7fdc2c1426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50582/0 (socket says 172.21.15.119:50582) 2023-10-08T02:32:04.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc2affd700 1 -- 172.21.15.119:0/1391211563 learned_addr learned my addr 172.21.15.119:0/1391211563 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc2a7fc700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc2c0cbf30 0x7fdc2c0a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.180+0000 7fdc2b7fe700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc2c0ccf90 0x7fdc2c0aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc2affd700 1 -- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc2c0cbf30 msgr2=0x7fdc2c0a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc2affd700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc2c0cbf30 0x7fdc2c0a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc2affd700 1 -- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc2c0ccf90 msgr2=0x7fdc2c0aa6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc2affd700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc2c0ccf90 0x7fdc2c0aa6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc2affd700 1 -- 172.21.15.119:0/1391211563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0094f0 con 0x7fdc2c0b1420 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc2b7fe700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc2c0ccf90 0x7fdc2c0aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc2affd700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0b1420 0x7fdc2c1426e0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0094c0 tx=0x7fdc1c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc12ffd700 1 -- 172.21.15.119:0/1391211563 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c024440 con 0x7fdc2c0b1420 2023-10-08T02:32:04.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc12ffd700 1 -- 172.21.15.119:0/1391211563 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c0245c0 con 0x7fdc2c0b1420 2023-10-08T02:32:04.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc12ffd700 1 -- 172.21.15.119:0/1391211563 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c024980 con 0x7fdc2c0b1420 2023-10-08T02:32:04.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc2c0a8610 con 0x7fdc2c0b1420 2023-10-08T02:32:04.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.181+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc2c0a8bd0 con 0x7fdc2c0b1420 2023-10-08T02:32:04.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.182+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc2c04c3c0 con 0x7fdc2c0b1420 2023-10-08T02:32:04.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.183+0000 7fdc12ffd700 1 -- 172.21.15.119:0/1391211563 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdc1c01a420 con 0x7fdc2c0b1420 2023-10-08T02:32:04.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.184+0000 7fdc12ffd700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdc18066f30 0x7fdc180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.184+0000 7fdc12ffd700 1 -- 172.21.15.119:0/1391211563 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fdc1c086570 con 0x7fdc2c0b1420 2023-10-08T02:32:04.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.184+0000 7fdc2a7fc700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdc18066f30 0x7fdc180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.185+0000 7fdc2a7fc700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdc18066f30 0x7fdc180693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c155060 tx=0x7fdc20007d80 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.188+0000 7fdc12ffd700 1 -- 172.21.15.119:0/1391211563 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdc1c053600 con 0x7fdc2c0b1420 2023-10-08T02:32:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.380+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc2c0a9080 con 0x7fdc18066f30 2023-10-08T02:32:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.380+0000 7fdc12ffd700 1 -- 172.21.15.119:0/1391211563 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdc200048d0 con 0x7fdc18066f30 2023-10-08T02:32:04.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdc18066f30 msgr2=0x7fdc180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdc18066f30 0x7fdc180693e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c155060 tx=0x7fdc20007d80 comp rx=0 tx=0).stop 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0b1420 msgr2=0x7fdc2c1426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0b1420 0x7fdc2c1426e0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0094c0 tx=0x7fdc1c017000 comp rx=0 tx=0).stop 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 shutdown_connections 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fdc18066f30 0x7fdc180693e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fdc2c0ccf90 0x7fdc2c0aa6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc2c0b1420 0x7fdc2c1426e0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 --2- 172.21.15.119:0/1391211563 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fdc2c0cbf30 0x7fdc2c0a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.382+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 >> 172.21.15.119:0/1391211563 conn(0x7fdc2c06a320 msgr2=0x7fdc2c0c9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.383+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 shutdown_connections 2023-10-08T02:32:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.383+0000 7fdc315ff700 1 -- 172.21.15.119:0/1391211563 wait complete. 2023-10-08T02:32:04.385 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 2 to deep-scrub 2023-10-08T02:32:04.401 INFO:tasks.ceph:Scrubbing osd.3 2023-10-08T02:32:04.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-08T02:32:04.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.587+0000 7f0863dff700 1 Processor -- start 2023-10-08T02:32:04.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.587+0000 7f0863dff700 1 -- start start 2023-10-08T02:32:04.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.588+0000 7f0863dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c159ec0 0x7f085c0b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f0863dff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f085c0b53d0 0x7f085c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f0863dff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f085c0b5b00 con 0x7f085c0cce40 2023-10-08T02:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f0863dff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f085c0b5da0 con 0x7f085c0b53d0 2023-10-08T02:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f0863dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c0b6080 con 0x7f085c159ec0 2023-10-08T02:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f086139a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f085c0b53d0 0x7f085c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f086139a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f085c0b53d0 0x7f085c0cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50588/0 (socket says 172.21.15.119:50588) 2023-10-08T02:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f086139a700 1 -- 172.21.15.119:0/3302288446 learned_addr learned my addr 172.21.15.119:0/3302288446 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f0861b9b700 1 --2- 172.21.15.119:0/3302288446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c159ec0 0x7f085c0b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f086139a700 1 -- 172.21.15.119:0/3302288446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c159ec0 msgr2=0x7f085c0b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f086139a700 1 --2- 172.21.15.119:0/3302288446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c159ec0 0x7f085c0b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f086139a700 1 -- 172.21.15.119:0/3302288446 >> v1:172.21.15.184:6789/0 conn(0x7f085c0cce40 legacy=0x7f085c0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.589+0000 7f086139a700 1 -- 172.21.15.119:0/3302288446 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c0cc100 con 0x7f085c0b53d0 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.590+0000 7f086139a700 1 --2- 172.21.15.119:0/3302288446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0b53d0 0x7f085c0cc900 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f084c0099b0 tx=0x7f084c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27329b37ba0ef00c server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.590+0000 7f0860b99700 1 -- 172.21.15.119:0/3302288446 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c009ea0 con 0x7f085c0b53d0 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.590+0000 7f0860b99700 1 -- 172.21.15.119:0/3302288446 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c00b860 con 0x7f085c0b53d0 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.590+0000 7f0860b99700 1 -- 172.21.15.119:0/3302288446 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00bc00 con 0x7f085c0b53d0 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.590+0000 7f0863dff700 1 -- 172.21.15.119:0/3302288446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0b53d0 msgr2=0x7f085c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.590+0000 7f0863dff700 1 --2- 172.21.15.119:0/3302288446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0b53d0 0x7f085c0cc900 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f084c0099b0 tx=0x7f084c009250 comp rx=0 tx=0).stop 2023-10-08T02:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.591+0000 7f0863dff700 1 -- 172.21.15.119:0/3302288446 shutdown_connections 2023-10-08T02:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.591+0000 7f0863dff700 1 --2- 172.21.15.119:0/3302288446 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c159ec0 0x7f085c0b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.591+0000 7f0863dff700 1 --2- 172.21.15.119:0/3302288446 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0b53d0 0x7f085c0cc900 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.591+0000 7f0863dff700 1 -- 172.21.15.119:0/3302288446 >> 172.21.15.119:0/3302288446 conn(0x7f085c0999f0 msgr2=0x7f085c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.591+0000 7f0863dff700 1 -- 172.21.15.119:0/3302288446 shutdown_connections 2023-10-08T02:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.591+0000 7f0863dff700 1 -- 172.21.15.119:0/3302288446 wait complete. 2023-10-08T02:32:04.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.592+0000 7f0863dff700 1 Processor -- start 2023-10-08T02:32:04.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.592+0000 7f0863dff700 1 -- start start 2023-10-08T02:32:04.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f0863dff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c0b53d0 0x7f085c09ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f0863dff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0cce40 0x7f085c157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f0863dff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f085c159ec0 0x7f085c09d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f0863dff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c157c80 con 0x7f085c0cce40 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f0863dff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085c09bd00 con 0x7f085c159ec0 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f0863dff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085c09b0f0 con 0x7f085c0b53d0 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f086139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0cce40 0x7f085c157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f086139a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0cce40 0x7f085c157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50594/0 (socket says 172.21.15.119:50594) 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.593+0000 7f086139a700 1 -- 172.21.15.119:0/1379309580 learned_addr learned my addr 172.21.15.119:0/1379309580 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f0861b9b700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c0b53d0 0x7f085c09ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f086239c700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f085c159ec0 0x7f085c09d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f086139a700 1 -- 172.21.15.119:0/1379309580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c0b53d0 msgr2=0x7f085c09ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f086139a700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f085c0b53d0 0x7f085c09ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f086139a700 1 -- 172.21.15.119:0/1379309580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f085c159ec0 msgr2=0x7f085c09d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f086139a700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f085c159ec0 0x7f085c09d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f086139a700 1 -- 172.21.15.119:0/1379309580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c0094f0 con 0x7f085c0cce40 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f086139a700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0cce40 0x7f085c157740 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f085c089390 tx=0x7f084c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00bbe0 con 0x7f085c0cce40 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c011420 con 0x7f085c0cce40 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c0117c0 con 0x7f085c0cce40 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.594+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085c09b3d0 con 0x7f085c0cce40 2023-10-08T02:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.595+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c09b990 con 0x7f085c0cce40 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.595+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f085c0cabb0 con 0x7f085c0cce40 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.596+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f084c011bb0 con 0x7f085c0cce40 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.597+0000 7f0851ffb700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0840066f30 0x7f08400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.597+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f084c086b80 con 0x7f085c0cce40 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.597+0000 7f0861b9b700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0840066f30 0x7f08400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.598+0000 7f0851ffb700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7f084006d6d0 0x7f084006fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.598+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 --> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0840070170 con 0x7f084006d6d0 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.598+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=211) v2 ==== 24+0+0 (secure 0 0 0) 0x7f084c086e70 con 0x7f085c0cce40 2023-10-08T02:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.598+0000 7f086239c700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7f084006d6d0 0x7f084006fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.598+0000 7f0861b9b700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0840066f30 0x7f08400693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f085c063840 tx=0x7f0858009120 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.598+0000 7f086239c700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7f084006d6d0 0x7f084006fb40 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.599+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 <== osd.3 v2:172.21.15.119:6808/85788 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f0840070170 con 0x7f084006d6d0 2023-10-08T02:32:04.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.613+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 --> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f085c063c90 con 0x7f084006d6d0 2023-10-08T02:32:04.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0851ffb700 1 -- 172.21.15.119:0/1379309580 <== osd.3 v2:172.21.15.119:6808/85788 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f085c063c90 con 0x7f084006d6d0 2023-10-08T02:32:04.614 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:32:04.615 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T02:32:04.615 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7f084006d6d0 msgr2=0x7f084006fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6808/85788,v1:172.21.15.119:6809/85788] conn(0x7f084006d6d0 0x7f084006fb40 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0840066f30 msgr2=0x7f08400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f0840066f30 0x7f08400693e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f085c063840 tx=0x7f0858009120 comp rx=0 tx=0).stop 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0cce40 msgr2=0x7f085c157740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 --2- 172.21.15.119:0/1379309580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f085c0cce40 0x7f085c157740 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f085c089390 tx=0x7f084c017000 comp rx=0 tx=0).stop 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f086239c700 1 -- 172.21.15.119:0/1379309580 reap_dead start 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 shutdown_connections 2023-10-08T02:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 >> 172.21.15.119:0/1379309580 conn(0x7f085c0999f0 msgr2=0x7f085c0ce320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:04.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 shutdown_connections 2023-10-08T02:32:04.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.614+0000 7f0863dff700 1 -- 172.21.15.119:0/1379309580 wait complete. 2023-10-08T02:32:04.631 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-08T02:32:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.813+0000 7f695d19d700 1 Processor -- start 2023-10-08T02:32:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.813+0000 7f695d19d700 1 -- start start 2023-10-08T02:32:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.814+0000 7f695d19d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f6958159420 0x7f69580b5f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.814+0000 7f695d19d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f695815d230 0x7f695815d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.814+0000 7f695d19d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69580b2980 con 0x7f69581597f0 2023-10-08T02:32:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.814+0000 7f695d19d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695815db70 con 0x7f695815d230 2023-10-08T02:32:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.814+0000 7f695d19d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69580b11d0 con 0x7f6958159420 2023-10-08T02:32:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.814+0000 7f695c99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f695815d230 0x7f695815d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.814+0000 7f695c99c700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f695815d230 0x7f695815d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50606/0 (socket says 172.21.15.119:50606) 2023-10-08T02:32:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.814+0000 7f695c99c700 1 -- 172.21.15.119:0/3112599778 learned_addr learned my addr 172.21.15.119:0/3112599778 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f6957fff700 1 --2- 172.21.15.119:0/3112599778 >> v2:172.21.15.184:3300/0 conn(0x7f6958159420 0x7f69580b5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f695c99c700 1 -- 172.21.15.119:0/3112599778 >> v2:172.21.15.184:3300/0 conn(0x7f6958159420 msgr2=0x7f69580b5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f695c99c700 1 --2- 172.21.15.119:0/3112599778 >> v2:172.21.15.184:3300/0 conn(0x7f6958159420 0x7f69580b5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f695c99c700 1 -- 172.21.15.119:0/3112599778 >> v1:172.21.15.184:6789/0 conn(0x7f69581597f0 legacy=0x7f69580b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f695c99c700 1 -- 172.21.15.119:0/3112599778 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580b14b0 con 0x7f695815d230 2023-10-08T02:32:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f695c99c700 1 --2- 172.21.15.119:0/3112599778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f695815d630 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f69580b2f20 tx=0x7f6950009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703c3ceb4497d9f2 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f6956ffd700 1 -- 172.21.15.119:0/3112599778 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695000e070 con 0x7f695815d230 2023-10-08T02:32:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f6956ffd700 1 -- 172.21.15.119:0/3112599778 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f695000ba20 con 0x7f695815d230 2023-10-08T02:32:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.815+0000 7f6956ffd700 1 -- 172.21.15.119:0/3112599778 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69500092e0 con 0x7f695815d230 2023-10-08T02:32:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.816+0000 7f695d19d700 1 -- 172.21.15.119:0/3112599778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 msgr2=0x7f695815d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.816+0000 7f695d19d700 1 --2- 172.21.15.119:0/3112599778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f695815d630 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f69580b2f20 tx=0x7f6950009840 comp rx=0 tx=0).stop 2023-10-08T02:32:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.816+0000 7f695d19d700 1 -- 172.21.15.119:0/3112599778 shutdown_connections 2023-10-08T02:32:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.816+0000 7f695d19d700 1 --2- 172.21.15.119:0/3112599778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f695815d630 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.816+0000 7f695d19d700 1 --2- 172.21.15.119:0/3112599778 >> v2:172.21.15.184:3300/0 conn(0x7f6958159420 0x7f69580b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.816+0000 7f695d19d700 1 -- 172.21.15.119:0/3112599778 >> 172.21.15.119:0/3112599778 conn(0x7f69580d0660 msgr2=0x7f69580d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.816+0000 7f695d19d700 1 -- 172.21.15.119:0/3112599778 shutdown_connections 2023-10-08T02:32:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.816+0000 7f695d19d700 1 -- 172.21.15.119:0/3112599778 wait complete. 2023-10-08T02:32:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.817+0000 7f695d19d700 1 Processor -- start 2023-10-08T02:32:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.818+0000 7f695d19d700 1 -- start start 2023-10-08T02:32:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.818+0000 7f695d19d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6958159420 0x7f6958155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.818+0000 7f695d19d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581597f0 0x7f69580c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695d19d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f69580c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695d19d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69580c9740 con 0x7f695815d230 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695d19d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69580c5400 con 0x7f6958159420 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695d19d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69580c5650 con 0x7f69581597f0 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f69580c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f69580c7dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50618/0 (socket says 172.21.15.119:50618) 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 -- 172.21.15.119:0/1687490644 learned_addr learned my addr 172.21.15.119:0/1687490644 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f6957fff700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6958159420 0x7f6958155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f69577fe700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581597f0 0x7f69580c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 -- 172.21.15.119:0/1687490644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581597f0 msgr2=0x7f69580c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581597f0 0x7f69580c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 -- 172.21.15.119:0/1687490644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6958159420 msgr2=0x7f6958155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6958159420 0x7f6958155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 -- 172.21.15.119:0/1687490644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6950009ae0 con 0x7f695815d230 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.819+0000 7f695c99c700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f69580c7dc0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f6950009880 tx=0x7f6950018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.820+0000 7f693ffff700 1 -- 172.21.15.119:0/1687490644 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950008a80 con 0x7f695815d230 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.820+0000 7f693ffff700 1 -- 172.21.15.119:0/1687490644 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6950008c00 con 0x7f695815d230 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.820+0000 7f693ffff700 1 -- 172.21.15.119:0/1687490644 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f695001c410 con 0x7f695815d230 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.820+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69580c5930 con 0x7f695815d230 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.820+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69580c5ef0 con 0x7f695815d230 2023-10-08T02:32:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.821+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695804c3c0 con 0x7f695815d230 2023-10-08T02:32:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.822+0000 7f693ffff700 1 -- 172.21.15.119:0/1687490644 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6950003040 con 0x7f695815d230 2023-10-08T02:32:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.822+0000 7f693ffff700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6944066f30 0x7f69440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.823+0000 7f693ffff700 1 -- 172.21.15.119:0/1687490644 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f6950085e10 con 0x7f695815d230 2023-10-08T02:32:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.823+0000 7f6957fff700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6944066f30 0x7f69440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.823+0000 7f6957fff700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6944066f30 0x7f69440693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6948009db0 tx=0x7f69480091a0 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:04.827+0000 7f693ffff700 1 -- 172.21.15.119:0/1687490644 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6950089030 con 0x7f695815d230 2023-10-08T02:32:05.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.021+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f69580b1790 con 0x7f6944066f30 2023-10-08T02:32:05.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.022+0000 7f693ffff700 1 -- 172.21.15.119:0/1687490644 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f69580b1790 con 0x7f6944066f30 2023-10-08T02:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6944066f30 msgr2=0x7f69440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6944066f30 0x7f69440693e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6948009db0 tx=0x7f69480091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 msgr2=0x7f69580c7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f69580c7dc0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f6950009880 tx=0x7f6950018000 comp rx=0 tx=0).stop 2023-10-08T02:32:05.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 shutdown_connections 2023-10-08T02:32:05.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6944066f30 0x7f69440693e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f695815d230 0x7f69580c7dc0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6958159420 0x7f6958155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 --2- 172.21.15.119:0/1687490644 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f69581597f0 0x7f69580c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 >> 172.21.15.119:0/1687490644 conn(0x7f69580d0660 msgr2=0x7f69581586e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:05.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 shutdown_connections 2023-10-08T02:32:05.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.024+0000 7f695d19d700 1 -- 172.21.15.119:0/1687490644 wait complete. 2023-10-08T02:32:05.028 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 3 to deep-scrub 2023-10-08T02:32:05.042 INFO:tasks.ceph:Scrubbing osd.4 2023-10-08T02:32:05.043 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-08T02:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.227+0000 7fac9adff700 1 Processor -- start 2023-10-08T02:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.227+0000 7fac9adff700 1 -- start start 2023-10-08T02:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac9adff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fac94069a50 0x7fac940c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac9adff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fac940c8490 0x7fac9415a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac9adff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac940ac030 con 0x7fac94069e20 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac9adff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac94068aa0 con 0x7fac94069a50 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac9adff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac940ab820 con 0x7fac940c8490 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac98b9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fac94069a50 0x7fac940c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac9939c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fac94069e20 0x7fac94068390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46570/0 (socket says 172.21.15.119:46570) 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac9939c700 1 -- 172.21.15.119:0/2812380305 learned_addr learned my addr 172.21.15.119:0/2812380305 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac93fff700 1 --2- 172.21.15.119:0/2812380305 >> v2:172.21.15.184:3300/0 conn(0x7fac940c8490 0x7fac9415a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac98b9b700 1 -- 172.21.15.119:0/2812380305 >> v2:172.21.15.184:3300/0 conn(0x7fac940c8490 msgr2=0x7fac9415a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac98b9b700 1 --2- 172.21.15.119:0/2812380305 >> v2:172.21.15.184:3300/0 conn(0x7fac940c8490 0x7fac9415a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac98b9b700 1 -- 172.21.15.119:0/2812380305 >> v1:172.21.15.119:6789/0 conn(0x7fac94069e20 legacy=0x7fac94068390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac937fe700 1 -- 172.21.15.119:0/2812380305 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754097120 0 0) 0x7fac8c001490 con 0x7fac94069e20 2023-10-08T02:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.229+0000 7fac98b9b700 1 -- 172.21.15.119:0/2812380305 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9406dbb0 con 0x7fac94069a50 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.230+0000 7fac98b9b700 1 --2- 172.21.15.119:0/2812380305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac94069a50 0x7fac940c6ac0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fac9406b920 tx=0x7fac880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4a5b8b6659a51a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.230+0000 7fac937fe700 1 -- 172.21.15.119:0/2812380305 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800b6b0 con 0x7fac94069a50 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.230+0000 7fac937fe700 1 -- 172.21.15.119:0/2812380305 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8800b810 con 0x7fac94069a50 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.230+0000 7fac937fe700 1 -- 172.21.15.119:0/2812380305 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8800bbb0 con 0x7fac94069a50 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.230+0000 7fac9adff700 1 -- 172.21.15.119:0/2812380305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac94069a50 msgr2=0x7fac940c6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.230+0000 7fac9adff700 1 --2- 172.21.15.119:0/2812380305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac94069a50 0x7fac940c6ac0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7fac9406b920 tx=0x7fac880091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.231+0000 7fac9adff700 1 -- 172.21.15.119:0/2812380305 shutdown_connections 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.231+0000 7fac9adff700 1 --2- 172.21.15.119:0/2812380305 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac94069a50 0x7fac940c6ac0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.231+0000 7fac9adff700 1 --2- 172.21.15.119:0/2812380305 >> v2:172.21.15.184:3300/0 conn(0x7fac940c8490 0x7fac9415a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.231+0000 7fac9adff700 1 -- 172.21.15.119:0/2812380305 >> 172.21.15.119:0/2812380305 conn(0x7fac940c8f90 msgr2=0x7fac940c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:05.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.231+0000 7fac9adff700 1 -- 172.21.15.119:0/2812380305 shutdown_connections 2023-10-08T02:32:05.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.231+0000 7fac9adff700 1 -- 172.21.15.119:0/2812380305 wait complete. 2023-10-08T02:32:05.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.232+0000 7fac9adff700 1 Processor -- start 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.232+0000 7fac9adff700 1 -- start start 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.233+0000 7fac9adff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac94069a50 0x7fac941549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.233+0000 7fac9adff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac94069e20 0x7fac94167c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.233+0000 7fac9adff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac940c8490 0x7fac94168140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.233+0000 7fac9adff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac940c4a50 con 0x7fac940c8490 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.233+0000 7fac9adff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac940c5010 con 0x7fac94069a50 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.233+0000 7fac9adff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac94168760 con 0x7fac94069e20 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.233+0000 7fac9939c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac940c8490 0x7fac94168140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.233+0000 7fac9939c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac940c8490 0x7fac94168140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52232/0 (socket says 172.21.15.119:52232) 2023-10-08T02:32:05.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac9939c700 1 -- 172.21.15.119:0/3555664538 learned_addr learned my addr 172.21.15.119:0/3555664538 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac98b9b700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac94069a50 0x7fac941549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac93fff700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac94069e20 0x7fac94167c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac9939c700 1 -- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac94069e20 msgr2=0x7fac94167c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac9939c700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fac94069e20 0x7fac94167c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac9939c700 1 -- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac94069a50 msgr2=0x7fac941549c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac9939c700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac94069a50 0x7fac941549c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac9939c700 1 -- 172.21.15.119:0/3555664538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88009420 con 0x7fac940c8490 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac98b9b700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fac94069a50 0x7fac941549c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac9939c700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac940c8490 0x7fac94168140 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00ccc0 tx=0x7fac8c001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c004bb0 con 0x7fac940c8490 2023-10-08T02:32:05.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c004d10 con 0x7fac940c8490 2023-10-08T02:32:05.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c00ad80 con 0x7fac940c8490 2023-10-08T02:32:05.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.234+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac94168a40 con 0x7fac940c8490 2023-10-08T02:32:05.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.235+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac94154f00 con 0x7fac940c8490 2023-10-08T02:32:05.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.235+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac940718b0 con 0x7fac940c8490 2023-10-08T02:32:05.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.236+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fac8c007300 con 0x7fac940c8490 2023-10-08T02:32:05.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.237+0000 7fac90ff9700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fac78066f30 0x7fac780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.237+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fac8c084b40 con 0x7fac940c8490 2023-10-08T02:32:05.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.237+0000 7fac98b9b700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fac78066f30 0x7fac780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.237+0000 7fac90ff9700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7fac7806d6d0 0x7fac7806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.237+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 --> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac78070170 con 0x7fac7806d6d0 2023-10-08T02:32:05.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.237+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=211) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac8c084e30 con 0x7fac940c8490 2023-10-08T02:32:05.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.238+0000 7fac98b9b700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fac78066f30 0x7fac780693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac880093f0 tx=0x7fac88017000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.238+0000 7fac93fff700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7fac7806d6d0 0x7fac7806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.239+0000 7fac93fff700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7fac7806d6d0 0x7fac7806fb40 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.240+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 <== osd.4 v2:172.21.15.184:6824/86003 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fac78070170 con 0x7fac7806d6d0 2023-10-08T02:32:05.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.254+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 --> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fac940ac030 con 0x7fac7806d6d0 2023-10-08T02:32:05.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.254+0000 7fac90ff9700 1 -- 172.21.15.119:0/3555664538 <== osd.4 v2:172.21.15.184:6824/86003 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fac940ac030 con 0x7fac7806d6d0 2023-10-08T02:32:05.255 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:32:05.255 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T02:32:05.256 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:32:05.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7fac7806d6d0 msgr2=0x7fac7806fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.184:6824/86003,v1:172.21.15.184:6825/86003] conn(0x7fac7806d6d0 0x7fac7806fb40 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fac78066f30 msgr2=0x7fac780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fac78066f30 0x7fac780693e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fac880093f0 tx=0x7fac88017000 comp rx=0 tx=0).stop 2023-10-08T02:32:05.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac940c8490 msgr2=0x7fac94168140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 --2- 172.21.15.119:0/3555664538 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fac940c8490 0x7fac94168140 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00ccc0 tx=0x7fac8c001a50 comp rx=0 tx=0).stop 2023-10-08T02:32:05.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9939c700 1 -- 172.21.15.119:0/3555664538 reap_dead start 2023-10-08T02:32:05.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 shutdown_connections 2023-10-08T02:32:05.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 >> 172.21.15.119:0/3555664538 conn(0x7fac940c8f90 msgr2=0x7fac940ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:05.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 shutdown_connections 2023-10-08T02:32:05.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.255+0000 7fac9adff700 1 -- 172.21.15.119:0/3555664538 wait complete. 2023-10-08T02:32:05.272 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-08T02:32:05.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.454+0000 7f4f6359e700 1 Processor -- start 2023-10-08T02:32:05.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.454+0000 7f4f6359e700 1 -- start start 2023-10-08T02:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.455+0000 7f4f6359e700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4f640c6d00 0x7f4f640c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.455+0000 7f4f6359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f6406e730 0x7f4f6406ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.455+0000 7f4f6359e700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6406e2a0 con 0x7f4f64157770 2023-10-08T02:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.455+0000 7f4f6359e700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f64157b40 con 0x7f4f640c6d00 2023-10-08T02:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.455+0000 7f4f6359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640c7690 con 0x7f4f6406e730 2023-10-08T02:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.455+0000 7f4f61d9b700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f4f640c6d00 0x7f4f640c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.455+0000 7f4f62d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f6406e730 0x7f4f6406ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.456+0000 7f4f62d9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f6406e730 0x7f4f6406ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:40148/0 (socket says 172.21.15.119:40148) 2023-10-08T02:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.456+0000 7f4f62d9d700 1 -- 172.21.15.119:0/3660525417 learned_addr learned my addr 172.21.15.119:0/3660525417 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.456+0000 7f4f6259c700 1 --1- 172.21.15.119:0/3660525417 >> v1:172.21.15.184:6789/0 conn(0x7f4f64157770 0x7f4f64150180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:56644/0 (socket says 172.21.15.119:56644) 2023-10-08T02:32:05.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.456+0000 7f4f62d9d700 1 -- 172.21.15.119:0/3660525417 >> v2:172.21.15.184:3300/0 conn(0x7f4f640c6d00 msgr2=0x7f4f640c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:05.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.456+0000 7f4f62d9d700 1 --2- 172.21.15.119:0/3660525417 >> v2:172.21.15.184:3300/0 conn(0x7f4f640c6d00 0x7f4f640c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.456+0000 7f4f62d9d700 1 -- 172.21.15.119:0/3660525417 >> v1:172.21.15.184:6789/0 conn(0x7f4f64157770 legacy=0x7f4f64150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.456+0000 7f4f62d9d700 1 -- 172.21.15.119:0/3660525417 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6406b2e0 con 0x7f4f6406e730 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.456+0000 7f4f62d9d700 1 --2- 172.21.15.119:0/3660525417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f6406e730 0x7f4f6406ebb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009f20 tx=0x7f4f5c009840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafbcdf7a732e2cd server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6159a700 1 -- 172.21.15.119:0/3660525417 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0092e0 con 0x7f4f6406e730 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6159a700 1 -- 172.21.15.119:0/3660525417 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5c00b9d0 con 0x7f4f6406e730 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6159a700 1 -- 172.21.15.119:0/3660525417 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00bd90 con 0x7f4f6406e730 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6359e700 1 -- 172.21.15.119:0/3660525417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f6406e730 msgr2=0x7f4f6406ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3660525417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f6406e730 0x7f4f6406ebb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009f20 tx=0x7f4f5c009840 comp rx=0 tx=0).stop 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6359e700 1 -- 172.21.15.119:0/3660525417 shutdown_connections 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3660525417 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f6406e730 0x7f4f6406ebb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3660525417 >> v2:172.21.15.184:3300/0 conn(0x7f4f640c6d00 0x7f4f640c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6359e700 1 -- 172.21.15.119:0/3660525417 >> 172.21.15.119:0/3660525417 conn(0x7f4f640cc1f0 msgr2=0x7f4f640cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:05.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6359e700 1 -- 172.21.15.119:0/3660525417 shutdown_connections 2023-10-08T02:32:05.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.457+0000 7f4f6359e700 1 -- 172.21.15.119:0/3660525417 wait complete. 2023-10-08T02:32:05.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.459+0000 7f4f6359e700 1 Processor -- start 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.459+0000 7f4f6359e700 1 -- start start 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6359e700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f6406e730 0x7f4f6414e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6359e700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f640c6d00 0x7f4f640b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6359e700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f64157770 0x7f4f640b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6359e700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640aca80 con 0x7f4f6406e730 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6359e700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6414ea30 con 0x7f4f640c6d00 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6359e700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6414ed10 con 0x7f4f64157770 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f6406e730 0x7f4f6414e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6259c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f6406e730 0x7f4f6414e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52246/0 (socket says 172.21.15.119:52246) 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6259c700 1 -- 172.21.15.119:0/3873491022 learned_addr learned my addr 172.21.15.119:0/3873491022 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f61d9b700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f640c6d00 0x7f4f640b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f62d9d700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f64157770 0x7f4f640b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6259c700 1 -- 172.21.15.119:0/3873491022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f64157770 msgr2=0x7f4f640b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6259c700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f64157770 0x7f4f640b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6259c700 1 -- 172.21.15.119:0/3873491022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f640c6d00 msgr2=0x7f4f640b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6259c700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f640c6d00 0x7f4f640b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.460+0000 7f4f6259c700 1 -- 172.21.15.119:0/3873491022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c009ae0 con 0x7f4f6406e730 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.461+0000 7f4f6259c700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f6406e730 0x7f4f6414e190 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c7c0 tx=0x7f4f54007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.461+0000 7f4f527fc700 1 -- 172.21.15.119:0/3873491022 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54005a70 con 0x7f4f6406e730 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.461+0000 7f4f527fc700 1 -- 172.21.15.119:0/3873491022 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f54005bd0 con 0x7f4f6406e730 2023-10-08T02:32:05.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.461+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6414eff0 con 0x7f4f6406e730 2023-10-08T02:32:05.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.461+0000 7f4f527fc700 1 -- 172.21.15.119:0/3873491022 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f540070d0 con 0x7f4f6406e730 2023-10-08T02:32:05.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.461+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f640af7e0 con 0x7f4f6406e730 2023-10-08T02:32:05.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.462+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6404c3c0 con 0x7f4f6406e730 2023-10-08T02:32:05.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.463+0000 7f4f527fc700 1 -- 172.21.15.119:0/3873491022 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4f54007480 con 0x7f4f6406e730 2023-10-08T02:32:05.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.465+0000 7f4f527fc700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4f44066f30 0x7f4f440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.465+0000 7f4f527fc700 1 -- 172.21.15.119:0/3873491022 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f4f54083a60 con 0x7f4f6406e730 2023-10-08T02:32:05.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.465+0000 7f4f61d9b700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4f44066f30 0x7f4f440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.466+0000 7f4f61d9b700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4f44066f30 0x7f4f440693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f640a9cf0 tx=0x7f4f580091a0 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.469+0000 7f4f527fc700 1 -- 172.21.15.119:0/3873491022 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4f54050b90 con 0x7f4f6406e730 2023-10-08T02:32:05.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.662+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f6414ee80 con 0x7f4f44066f30 2023-10-08T02:32:05.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.662+0000 7f4f527fc700 1 -- 172.21.15.119:0/3873491022 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4f6414ee80 con 0x7f4f44066f30 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4f44066f30 msgr2=0x7f4f440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4f44066f30 0x7f4f440693e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f640a9cf0 tx=0x7f4f580091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f6406e730 msgr2=0x7f4f6414e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f6406e730 0x7f4f6414e190 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400c7c0 tx=0x7f4f54007f70 comp rx=0 tx=0).stop 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 shutdown_connections 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f4f44066f30 0x7f4f440693e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f4f64157770 0x7f4f640b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f6406e730 0x7f4f6414e190 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 --2- 172.21.15.119:0/3873491022 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f4f640c6d00 0x7f4f640b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 >> 172.21.15.119:0/3873491022 conn(0x7f4f640cc1f0 msgr2=0x7f4f640696c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:05.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 shutdown_connections 2023-10-08T02:32:05.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.664+0000 7f4f6359e700 1 -- 172.21.15.119:0/3873491022 wait complete. 2023-10-08T02:32:05.667 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 4 to deep-scrub 2023-10-08T02:32:05.682 INFO:tasks.ceph:Scrubbing osd.5 2023-10-08T02:32:05.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-08T02:32:05.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.866+0000 7f7e579ff700 1 Processor -- start 2023-10-08T02:32:05.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.866+0000 7f7e579ff700 1 -- start start 2023-10-08T02:32:05.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.867+0000 7f7e579ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 0x7f7e500b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e579ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e500b53d0 0x7f7e500cc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e579ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e500b5b00 con 0x7f7e500cce40 2023-10-08T02:32:05.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e579ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e500b5da0 con 0x7f7e500b53d0 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e579ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e500b6080 con 0x7f7e50159ec0 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e54f9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7e500b53d0 0x7f7e500cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e55f9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7e500cce40 0x7f7e500cd280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46582/0 (socket says 172.21.15.119:46582) 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e55f9c700 1 -- 172.21.15.119:0/2005902590 learned_addr learned my addr 172.21.15.119:0/2005902590 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e5579b700 1 --2- 172.21.15.119:0/2005902590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 0x7f7e500b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e54f9a700 1 -- 172.21.15.119:0/2005902590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 msgr2=0x7f7e500b4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e54f9a700 1 --2- 172.21.15.119:0/2005902590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 0x7f7e500b4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e54f9a700 1 -- 172.21.15.119:0/2005902590 >> v1:172.21.15.119:6789/0 conn(0x7f7e500cce40 legacy=0x7f7e500cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e54f9a700 1 -- 172.21.15.119:0/2005902590 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e500cc100 con 0x7f7e500b53d0 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.868+0000 7f7e47fff700 1 -- 172.21.15.119:0/2005902590 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271748811 0 0) 0x7f7e500b5b00 con 0x7f7e500cce40 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e54f9a700 1 --2- 172.21.15.119:0/2005902590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 0x7f7e500cc900 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f7e400098e0 tx=0x7f7e400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e590b0d36673650 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e47fff700 1 -- 172.21.15.119:0/2005902590 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000b640 con 0x7f7e500b53d0 2023-10-08T02:32:05.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e47fff700 1 -- 172.21.15.119:0/2005902590 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4000b7a0 con 0x7f7e500b53d0 2023-10-08T02:32:05.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e47fff700 1 -- 172.21.15.119:0/2005902590 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4000bb40 con 0x7f7e500b53d0 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e579ff700 1 -- 172.21.15.119:0/2005902590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 msgr2=0x7f7e500cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e579ff700 1 --2- 172.21.15.119:0/2005902590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 0x7f7e500cc900 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f7e400098e0 tx=0x7f7e400091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e579ff700 1 -- 172.21.15.119:0/2005902590 shutdown_connections 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e579ff700 1 --2- 172.21.15.119:0/2005902590 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 0x7f7e500b4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e579ff700 1 --2- 172.21.15.119:0/2005902590 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 0x7f7e500cc900 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.869+0000 7f7e579ff700 1 -- 172.21.15.119:0/2005902590 >> 172.21.15.119:0/2005902590 conn(0x7f7e500999f0 msgr2=0x7f7e500b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.870+0000 7f7e579ff700 1 -- 172.21.15.119:0/2005902590 shutdown_connections 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.870+0000 7f7e579ff700 1 -- 172.21.15.119:0/2005902590 wait complete. 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.871+0000 7f7e579ff700 1 Processor -- start 2023-10-08T02:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.871+0000 7f7e579ff700 1 -- start start 2023-10-08T02:32:05.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e579ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 0x7f7e5016b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e579ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e500cce40 0x7f7e5009d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e579ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 0x7f7e5009d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e579ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5009aca0 con 0x7f7e500b53d0 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e579ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5009af80 con 0x7f7e500cce40 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e579ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5009b260 con 0x7f7e50159ec0 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e5579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 0x7f7e5016b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e5579b700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 0x7f7e5016b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52264/0 (socket says 172.21.15.119:52264) 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e5579b700 1 -- 172.21.15.119:0/3166940965 learned_addr learned my addr 172.21.15.119:0/3166940965 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e54f9a700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e500cce40 0x7f7e5009d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.872+0000 7f7e55f9c700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 0x7f7e5009d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e5579b700 1 -- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 msgr2=0x7f7e5009d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e5579b700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 0x7f7e5009d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e5579b700 1 -- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e500cce40 msgr2=0x7f7e5009d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e5579b700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e500cce40 0x7f7e5009d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e5579b700 1 -- 172.21.15.119:0/3166940965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e40009420 con 0x7f7e500b53d0 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e55f9c700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f7e50159ec0 0x7f7e5009d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e54f9a700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f7e500cce40 0x7f7e5009d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e5579b700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 0x7f7e5016b080 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00ea00 tx=0x7f7e4c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00c850 con 0x7f7e500b53d0 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e4c00c9b0 con 0x7f7e500b53d0 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c00cd30 con 0x7f7e500b53d0 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5009b540 con 0x7f7e500b53d0 2023-10-08T02:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.873+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5016b5c0 con 0x7f7e500b53d0 2023-10-08T02:32:05.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.874+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e5009b0f0 con 0x7f7e500b53d0 2023-10-08T02:32:05.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.875+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7e4c013680 con 0x7f7e500b53d0 2023-10-08T02:32:05.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.876+0000 7f7e457fa700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7e34066f30 0x7f7e340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.876+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f7e4c084d50 con 0x7f7e500b53d0 2023-10-08T02:32:05.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.876+0000 7f7e54f9a700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7e34066f30 0x7f7e340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.876+0000 7f7e457fa700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f7e3406d6d0 0x7f7e3406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:05.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.876+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 --> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e34070170 con 0x7f7e3406d6d0 2023-10-08T02:32:05.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.876+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=211) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e4c0872b0 con 0x7f7e500b53d0 2023-10-08T02:32:05.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.877+0000 7f7e54f9a700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7e34066f30 0x7f7e340693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e400093f0 tx=0x7f7e40017000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.877+0000 7f7e55f9c700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f7e3406d6d0 0x7f7e3406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:05.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.877+0000 7f7e55f9c700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f7e3406d6d0 0x7f7e3406fb40 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:05.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.878+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 <== osd.5 v2:172.21.15.184:6816/86004 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f7e34070170 con 0x7f7e3406d6d0 2023-10-08T02:32:05.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.892+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 --> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7e500636f0 con 0x7f7e3406d6d0 2023-10-08T02:32:05.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.893+0000 7f7e457fa700 1 -- 172.21.15.119:0/3166940965 <== osd.5 v2:172.21.15.184:6816/86004 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7e500636f0 con 0x7f7e3406d6d0 2023-10-08T02:32:05.897 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:32:05.897 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T02:32:05.897 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:32:05.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f7e3406d6d0 msgr2=0x7f7e3406fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.184:6816/86004,v1:172.21.15.184:6817/86004] conn(0x7f7e3406d6d0 0x7f7e3406fb40 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:05.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7e34066f30 msgr2=0x7f7e340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f7e34066f30 0x7f7e340693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7e400093f0 tx=0x7f7e40017000 comp rx=0 tx=0).stop 2023-10-08T02:32:05.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 msgr2=0x7f7e5016b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:05.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 --2- 172.21.15.119:0/3166940965 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e500b53d0 0x7f7e5016b080 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00ea00 tx=0x7f7e4c009120 comp rx=0 tx=0).stop 2023-10-08T02:32:05.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e55f9c700 1 -- 172.21.15.119:0/3166940965 reap_dead start 2023-10-08T02:32:05.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 shutdown_connections 2023-10-08T02:32:05.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 >> 172.21.15.119:0/3166940965 conn(0x7f7e500999f0 msgr2=0x7f7e50159140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:05.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 shutdown_connections 2023-10-08T02:32:05.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:05.894+0000 7f7e579ff700 1 -- 172.21.15.119:0/3166940965 wait complete. 2023-10-08T02:32:05.911 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-08T02:32:06.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.091+0000 7fcc023ff700 1 Processor -- start 2023-10-08T02:32:06.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.091+0000 7fcc023ff700 1 -- start start 2023-10-08T02:32:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcc023ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fcbfc0cbf30 0x7fcbfc0ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcc023ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0ccf90 0x7fcbfc0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcc023ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbfc0cc300 con 0x7fcbfc15a270 2023-10-08T02:32:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcc023ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbfc156bf0 con 0x7fcbfc0cbf30 2023-10-08T02:32:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcc023ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0ae950 con 0x7fcbfc0ccf90 2023-10-08T02:32:06.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcc0099c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcbfc15a270 0x7fcbfc0c9740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46584/0 (socket says 172.21.15.119:46584) 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcc0099c700 1 -- 172.21.15.119:0/1880089336 learned_addr learned my addr 172.21.15.119:0/1880089336 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcbfb7fe700 1 --2- 172.21.15.119:0/1880089336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0ccf90 0x7fcbfc0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcbfbfff700 1 --2- 172.21.15.119:0/1880089336 >> v2:172.21.15.184:3300/0 conn(0x7fcbfc0cbf30 0x7fcbfc0ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcbfaffd700 1 -- 172.21.15.119:0/1880089336 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974727749 0 0) 0x7fcbfc0cc300 con 0x7fcbfc15a270 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.093+0000 7fcbfaffd700 1 -- 172.21.15.119:0/1880089336 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe4003570 con 0x7fcbfc15a270 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.094+0000 7fcbfb7fe700 1 -- 172.21.15.119:0/1880089336 >> v2:172.21.15.184:3300/0 conn(0x7fcbfc0cbf30 msgr2=0x7fcbfc0ae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.094+0000 7fcbfb7fe700 1 --2- 172.21.15.119:0/1880089336 >> v2:172.21.15.184:3300/0 conn(0x7fcbfc0cbf30 0x7fcbfc0ae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.094+0000 7fcbfb7fe700 1 -- 172.21.15.119:0/1880089336 >> v1:172.21.15.119:6789/0 conn(0x7fcbfc15a270 legacy=0x7fcbfc0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.094+0000 7fcbfb7fe700 1 -- 172.21.15.119:0/1880089336 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc0ad370 con 0x7fcbfc0ccf90 2023-10-08T02:32:06.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.094+0000 7fcbfb7fe700 1 --2- 172.21.15.119:0/1880089336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0ccf90 0x7fcbfc0b0d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000bb90 tx=0x7fcbf000b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c9a480f177add1a server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.094+0000 7fcbfaffd700 1 -- 172.21.15.119:0/1880089336 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf000da70 con 0x7fcbfc0ccf90 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.094+0000 7fcbfaffd700 1 -- 172.21.15.119:0/1880089336 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf000dbd0 con 0x7fcbfc0ccf90 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.094+0000 7fcbfaffd700 1 -- 172.21.15.119:0/1880089336 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0019480 con 0x7fcbfc0ccf90 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.095+0000 7fcc023ff700 1 -- 172.21.15.119:0/1880089336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0ccf90 msgr2=0x7fcbfc0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.095+0000 7fcc023ff700 1 --2- 172.21.15.119:0/1880089336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0ccf90 0x7fcbfc0b0d10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000bb90 tx=0x7fcbf000b430 comp rx=0 tx=0).stop 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.095+0000 7fcc023ff700 1 -- 172.21.15.119:0/1880089336 shutdown_connections 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.095+0000 7fcc023ff700 1 --2- 172.21.15.119:0/1880089336 >> v2:172.21.15.184:3300/0 conn(0x7fcbfc0cbf30 0x7fcbfc0ae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.095+0000 7fcc023ff700 1 --2- 172.21.15.119:0/1880089336 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0ccf90 0x7fcbfc0b0d10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.095+0000 7fcc023ff700 1 -- 172.21.15.119:0/1880089336 >> 172.21.15.119:0/1880089336 conn(0x7fcbfc06a320 msgr2=0x7fcbfc0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.095+0000 7fcc023ff700 1 -- 172.21.15.119:0/1880089336 shutdown_connections 2023-10-08T02:32:06.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.095+0000 7fcc023ff700 1 -- 172.21.15.119:0/1880089336 wait complete. 2023-10-08T02:32:06.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.096+0000 7fcc023ff700 1 Processor -- start 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.097+0000 7fcc023ff700 1 -- start start 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.097+0000 7fcc023ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0cbf30 0x7fcbfc0acbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.097+0000 7fcc023ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbfc0ccf90 0x7fcbfc16ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.097+0000 7fcc023ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbfc15a270 0x7fcbfc156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.097+0000 7fcc023ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0c70d0 con 0x7fcbfc15a270 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.097+0000 7fcc023ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc0ad120 con 0x7fcbfc0ccf90 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.097+0000 7fcc023ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc159fd0 con 0x7fcbfc0cbf30 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbfc15a270 0x7fcbfc156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbfc15a270 0x7fcbfc156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52274/0 (socket says 172.21.15.119:52274) 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 -- 172.21.15.119:0/321362884 learned_addr learned my addr 172.21.15.119:0/321362884 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:06.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcbfbfff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0cbf30 0x7fcbfc0acbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcbfb7fe700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbfc0ccf90 0x7fcbfc16ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 -- 172.21.15.119:0/321362884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0cbf30 msgr2=0x7fcbfc0acbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0cbf30 0x7fcbfc0acbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 -- 172.21.15.119:0/321362884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbfc0ccf90 msgr2=0x7fcbfc16ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbfc0ccf90 0x7fcbfc16ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 -- 172.21.15.119:0/321362884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf000b6d0 con 0x7fcbfc15a270 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.098+0000 7fcc0099c700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbfc15a270 0x7fcbfc156d30 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400cc40 tx=0x7fcbf4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.099+0000 7fcbe3fff700 1 -- 172.21.15.119:0/321362884 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400a830 con 0x7fcbfc15a270 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.099+0000 7fcbe3fff700 1 -- 172.21.15.119:0/321362884 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf400a990 con 0x7fcbfc15a270 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.099+0000 7fcbe3fff700 1 -- 172.21.15.119:0/321362884 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf400ad10 con 0x7fcbfc15a270 2023-10-08T02:32:06.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.099+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc0c67e0 con 0x7fcbfc15a270 2023-10-08T02:32:06.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.099+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc0c6d60 con 0x7fcbfc15a270 2023-10-08T02:32:06.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.100+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc0c7240 con 0x7fcbfc15a270 2023-10-08T02:32:06.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.101+0000 7fcbe3fff700 1 -- 172.21.15.119:0/321362884 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcbf4007300 con 0x7fcbfc15a270 2023-10-08T02:32:06.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.102+0000 7fcbe3fff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbe8066f30 0x7fcbe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.102+0000 7fcbe3fff700 1 -- 172.21.15.119:0/321362884 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fcbf40849a0 con 0x7fcbfc15a270 2023-10-08T02:32:06.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.102+0000 7fcbfbfff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbe8066f30 0x7fcbe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.103+0000 7fcbfbfff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbe8066f30 0x7fcbe80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc15dc00 tx=0x7fcbec0091a0 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.106+0000 7fcbe3fff700 1 -- 172.21.15.119:0/321362884 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcbf4051ab0 con 0x7fcbfc15a270 2023-10-08T02:32:06.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.295+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbfc04c3c0 con 0x7fcbe8066f30 2023-10-08T02:32:06.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.295+0000 7fcbe3fff700 1 -- 172.21.15.119:0/321362884 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcbfc04c3c0 con 0x7fcbe8066f30 2023-10-08T02:32:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbe8066f30 msgr2=0x7fcbe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbe8066f30 0x7fcbe80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc15dc00 tx=0x7fcbec0091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbfc15a270 msgr2=0x7fcbfc156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbfc15a270 0x7fcbfc156d30 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fcbf400cc40 tx=0x7fcbf4001a40 comp rx=0 tx=0).stop 2023-10-08T02:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 shutdown_connections 2023-10-08T02:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fcbe8066f30 0x7fcbe80693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fcbfc0cbf30 0x7fcbfc0acbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbfc15a270 0x7fcbfc156d30 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 --2- 172.21.15.119:0/321362884 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fcbfc0ccf90 0x7fcbfc16ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.297+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 >> 172.21.15.119:0/321362884 conn(0x7fcbfc06a320 msgr2=0x7fcbfc0c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.298+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 shutdown_connections 2023-10-08T02:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.298+0000 7fcc023ff700 1 -- 172.21.15.119:0/321362884 wait complete. 2023-10-08T02:32:06.300 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 5 to deep-scrub 2023-10-08T02:32:06.315 INFO:tasks.ceph:Scrubbing osd.6 2023-10-08T02:32:06.316 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-08T02:32:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.503+0000 7efc8b5ff700 1 Processor -- start 2023-10-08T02:32:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.503+0000 7efc8b5ff700 1 -- start start 2023-10-08T02:32:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.504+0000 7efc8b5ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7efc841563b0 0x7efc84156780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.504+0000 7efc8b5ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc8415a440 0x7efc840acf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.504+0000 7efc8b5ff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8415a9b0 con 0x7efc84068890 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.504+0000 7efc8b5ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8415ac90 con 0x7efc8415a440 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.504+0000 7efc8b5ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8415af70 con 0x7efc841563b0 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc88b9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efc8415a440 0x7efc840acf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc89b9c700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efc84068890 0x7efc840c8230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46588/0 (socket says 172.21.15.119:46588) 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc89b9c700 1 -- 172.21.15.119:0/2893091552 learned_addr learned my addr 172.21.15.119:0/2893091552 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc8939b700 1 --2- 172.21.15.119:0/2893091552 >> v2:172.21.15.184:3300/0 conn(0x7efc841563b0 0x7efc84156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc88b9a700 1 -- 172.21.15.119:0/2893091552 >> v2:172.21.15.184:3300/0 conn(0x7efc841563b0 msgr2=0x7efc84156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc88b9a700 1 --2- 172.21.15.119:0/2893091552 >> v2:172.21.15.184:3300/0 conn(0x7efc841563b0 0x7efc84156780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc88b9a700 1 -- 172.21.15.119:0/2893091552 >> v1:172.21.15.119:6789/0 conn(0x7efc84068890 legacy=0x7efc840c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc88b9a700 1 -- 172.21.15.119:0/2893091552 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc8415b250 con 0x7efc8415a440 2023-10-08T02:32:06.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc7bfff700 1 -- 172.21.15.119:0/2893091552 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890130174 0 0) 0x7efc8415a9b0 con 0x7efc84068890 2023-10-08T02:32:06.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.505+0000 7efc88b9a700 1 --2- 172.21.15.119:0/2893091552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc8415a440 0x7efc840acf30 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7efc8406a9f0 tx=0x7efc740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c0af0f714aef99 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc7bfff700 1 -- 172.21.15.119:0/2893091552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7400b6b0 con 0x7efc8415a440 2023-10-08T02:32:06.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc7bfff700 1 -- 172.21.15.119:0/2893091552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc7400b810 con 0x7efc8415a440 2023-10-08T02:32:06.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc7bfff700 1 -- 172.21.15.119:0/2893091552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc7400bbb0 con 0x7efc8415a440 2023-10-08T02:32:06.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc8b5ff700 1 -- 172.21.15.119:0/2893091552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc8415a440 msgr2=0x7efc840acf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc8b5ff700 1 --2- 172.21.15.119:0/2893091552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc8415a440 0x7efc840acf30 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7efc8406a9f0 tx=0x7efc740091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc8b5ff700 1 -- 172.21.15.119:0/2893091552 shutdown_connections 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc8b5ff700 1 --2- 172.21.15.119:0/2893091552 >> v2:172.21.15.184:3300/0 conn(0x7efc841563b0 0x7efc84156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc8b5ff700 1 --2- 172.21.15.119:0/2893091552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc8415a440 0x7efc840acf30 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc8b5ff700 1 -- 172.21.15.119:0/2893091552 >> 172.21.15.119:0/2893091552 conn(0x7efc840c8f90 msgr2=0x7efc840c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc8b5ff700 1 -- 172.21.15.119:0/2893091552 shutdown_connections 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.506+0000 7efc8b5ff700 1 -- 172.21.15.119:0/2893091552 wait complete. 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.508+0000 7efc8b5ff700 1 Processor -- start 2023-10-08T02:32:06.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.508+0000 7efc8b5ff700 1 -- start start 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc8b5ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc84068890 0x7efc84159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc8b5ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc841563b0 0x7efc8414dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc8b5ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc8415a440 0x7efc840c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc8b5ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c1040 con 0x7efc841563b0 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc8b5ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c0430 con 0x7efc84068890 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc8b5ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc840c0710 con 0x7efc8415a440 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc88b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc841563b0 0x7efc8414dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc88b9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc841563b0 0x7efc8414dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52294/0 (socket says 172.21.15.119:52294) 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc88b9a700 1 -- 172.21.15.119:0/4245639988 learned_addr learned my addr 172.21.15.119:0/4245639988 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc88b9a700 1 -- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc8415a440 msgr2=0x7efc840c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc88b9a700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7efc8415a440 0x7efc840c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc88b9a700 1 -- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc84068890 msgr2=0x7efc84159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc88b9a700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7efc84068890 0x7efc84159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.509+0000 7efc88b9a700 1 -- 172.21.15.119:0/4245639988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc74009420 con 0x7efc841563b0 2023-10-08T02:32:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.510+0000 7efc88b9a700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc841563b0 0x7efc8414dc90 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7efc8406bbb0 tx=0x7efc74017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.510+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74011420 con 0x7efc841563b0 2023-10-08T02:32:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.510+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc74011580 con 0x7efc841563b0 2023-10-08T02:32:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.510+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc840c09f0 con 0x7efc841563b0 2023-10-08T02:32:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.510+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc74011940 con 0x7efc841563b0 2023-10-08T02:32:06.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.510+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc84159c80 con 0x7efc841563b0 2023-10-08T02:32:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.511+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efc840718b0 con 0x7efc841563b0 2023-10-08T02:32:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.512+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efc74011d30 con 0x7efc841563b0 2023-10-08T02:32:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.512+0000 7efc797fa700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc68066f30 0x7efc680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.513+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7efc74085a20 con 0x7efc841563b0 2023-10-08T02:32:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.513+0000 7efc8939b700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc68066f30 0x7efc680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.513+0000 7efc797fa700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7efc6806d6d0 0x7efc6806fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.513+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 --> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efc68070170 con 0x7efc6806d6d0 2023-10-08T02:32:06.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.513+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=211) v2 ==== 24+0+0 (secure 0 0 0) 0x7efc7408a240 con 0x7efc841563b0 2023-10-08T02:32:06.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.513+0000 7efc8939b700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc68066f30 0x7efc680693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc80009a60 tx=0x7efc80008040 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.514+0000 7efc89b9c700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7efc6806d6d0 0x7efc6806fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.514+0000 7efc89b9c700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7efc6806d6d0 0x7efc6806fb40 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.515+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 <== osd.6 v2:172.21.15.184:6808/86002 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7efc68070170 con 0x7efc6806d6d0 2023-10-08T02:32:06.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.529+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 --> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efc8406d6a0 con 0x7efc6806d6d0 2023-10-08T02:32:06.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc797fa700 1 -- 172.21.15.119:0/4245639988 <== osd.6 v2:172.21.15.184:6808/86002 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efc8406d6a0 con 0x7efc6806d6d0 2023-10-08T02:32:06.530 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:32:06.531 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T02:32:06.531 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7efc6806d6d0 msgr2=0x7efc6806fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.184:6808/86002,v1:172.21.15.184:6809/86002] conn(0x7efc6806d6d0 0x7efc6806fb40 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc68066f30 msgr2=0x7efc680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7efc68066f30 0x7efc680693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efc80009a60 tx=0x7efc80008040 comp rx=0 tx=0).stop 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc841563b0 msgr2=0x7efc8414dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 --2- 172.21.15.119:0/4245639988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efc841563b0 0x7efc8414dc90 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7efc8406bbb0 tx=0x7efc74017000 comp rx=0 tx=0).stop 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc89b9c700 1 -- 172.21.15.119:0/4245639988 reap_dead start 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 shutdown_connections 2023-10-08T02:32:06.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 >> 172.21.15.119:0/4245639988 conn(0x7efc840c8f90 msgr2=0x7efc84150be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 shutdown_connections 2023-10-08T02:32:06.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.530+0000 7efc8b5ff700 1 -- 172.21.15.119:0/4245639988 wait complete. 2023-10-08T02:32:06.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-08T02:32:06.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.729+0000 7fe2e03ff700 1 Processor -- start 2023-10-08T02:32:06.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.729+0000 7fe2e03ff700 1 -- start start 2023-10-08T02:32:06.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.730+0000 7fe2e03ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.730+0000 7fe2e03ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2d8159c60 0x7fe2d815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.730+0000 7fe2e03ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d815a160 con 0x7fe2d815db10 2023-10-08T02:32:06.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.730+0000 7fe2e03ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d80d13a0 con 0x7fe2d8159c60 2023-10-08T02:32:06.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.730+0000 7fe2e03ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d815e140 con 0x7fe2d8159890 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2de19b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2dd99a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2d8159c60 0x7fe2d815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2dd99a700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fe2d8159c60 0x7fe2d815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3300/0 says I am v2:172.21.15.119:39262/0 (socket says 172.21.15.119:39262) 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2de19b700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.184:3301/0 says I am v2:172.21.15.119:40204/0 (socket says 172.21.15.119:40204) 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2dd99a700 1 -- 172.21.15.119:0/1310820780 learned_addr learned my addr 172.21.15.119:0/1310820780 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2de99c700 1 --1- 172.21.15.119:0/1310820780 >> v1:172.21.15.184:6789/0 conn(0x7fe2d815db10 0x7fe2d8069560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.184:6789/0 says I am v1:172.21.15.119:56652/0 (socket says 172.21.15.119:56652) 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2dd99a700 1 -- 172.21.15.119:0/1310820780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 msgr2=0x7fe2d815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2dd99a700 1 --2- 172.21.15.119:0/1310820780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2dd99a700 1 -- 172.21.15.119:0/1310820780 >> v1:172.21.15.184:6789/0 conn(0x7fe2d815db10 legacy=0x7fe2d8069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2dd99a700 1 -- 172.21.15.119:0/1310820780 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d815a3d0 con 0x7fe2d8159c60 2023-10-08T02:32:06.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2dd199700 1 -- 172.21.15.119:0/1310820780 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091504008 0 0) 0x7fe2d815a160 con 0x7fe2d815db10 2023-10-08T02:32:06.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.731+0000 7fe2de19b700 1 --2- 172.21.15.119:0/1310820780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:32:06.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.732+0000 7fe2dd99a700 1 --2- 172.21.15.119:0/1310820780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d8159c60 0x7fe2d815d5d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80099b0 tx=0x7fe2c8009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc70377be4d74c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.732+0000 7fe2dd199700 1 -- 172.21.15.119:0/1310820780 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8009ea0 con 0x7fe2d8159c60 2023-10-08T02:32:06.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.732+0000 7fe2dd199700 1 -- 172.21.15.119:0/1310820780 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c800b860 con 0x7fe2d8159c60 2023-10-08T02:32:06.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.732+0000 7fe2dd199700 1 -- 172.21.15.119:0/1310820780 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800bc00 con 0x7fe2d8159c60 2023-10-08T02:32:06.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.733+0000 7fe2e03ff700 1 -- 172.21.15.119:0/1310820780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d8159c60 msgr2=0x7fe2d815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.733+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/1310820780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d8159c60 0x7fe2d815d5d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80099b0 tx=0x7fe2c8009250 comp rx=0 tx=0).stop 2023-10-08T02:32:06.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.733+0000 7fe2e03ff700 1 -- 172.21.15.119:0/1310820780 shutdown_connections 2023-10-08T02:32:06.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.733+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/1310820780 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.733+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/1310820780 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d8159c60 0x7fe2d815d5d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.733+0000 7fe2e03ff700 1 -- 172.21.15.119:0/1310820780 >> 172.21.15.119:0/1310820780 conn(0x7fe2d8068e20 msgr2=0x7fe2d80d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:06.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.733+0000 7fe2e03ff700 1 -- 172.21.15.119:0/1310820780 shutdown_connections 2023-10-08T02:32:06.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.733+0000 7fe2e03ff700 1 -- 172.21.15.119:0/1310820780 wait complete. 2023-10-08T02:32:06.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.734+0000 7fe2e03ff700 1 Processor -- start 2023-10-08T02:32:06.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.734+0000 7fe2e03ff700 1 -- start start 2023-10-08T02:32:06.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.735+0000 7fe2e03ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d8151490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.735+0000 7fe2e03ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d8159c60 0x7fe2d80cf880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.735+0000 7fe2e03ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d815db10 0x7fe2d80cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.735+0000 7fe2e03ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8152e30 con 0x7fe2d8159c60 2023-10-08T02:32:06.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.735+0000 7fe2e03ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d81529a0 con 0x7fe2d815db10 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.735+0000 7fe2e03ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d8151d90 con 0x7fe2d8159890 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.735+0000 7fe2dd99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d8159c60 0x7fe2d80cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2dd99a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d8159c60 0x7fe2d80cf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52296/0 (socket says 172.21.15.119:52296) 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2dd99a700 1 -- 172.21.15.119:0/2648642471 learned_addr learned my addr 172.21.15.119:0/2648642471 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2de99c700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d815db10 0x7fe2d80cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2de19b700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d8151490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2dd99a700 1 -- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 msgr2=0x7fe2d8151490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2dd99a700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d8151490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2dd99a700 1 -- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d815db10 msgr2=0x7fe2d80cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2dd99a700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d815db10 0x7fe2d80cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2dd99a700 1 -- 172.21.15.119:0/2648642471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d4009420 con 0x7fe2d8159c60 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2de99c700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d815db10 0x7fe2d80cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2de19b700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d8151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2dd99a700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d8159c60 0x7fe2d80cf880 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8006fd0 tx=0x7fe2c8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2ce7fc700 1 -- 172.21.15.119:0/2648642471 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c800bbe0 con 0x7fe2d8159c60 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2ce7fc700 1 -- 172.21.15.119:0/2648642471 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c8011420 con 0x7fe2d8159c60 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c80094f0 con 0x7fe2d8159c60 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.736+0000 7fe2ce7fc700 1 -- 172.21.15.119:0/2648642471 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80117c0 con 0x7fe2d8159c60 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.737+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d8152350 con 0x7fe2d8159c60 2023-10-08T02:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.738+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d8152b30 con 0x7fe2d8159c60 2023-10-08T02:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.738+0000 7fe2ce7fc700 1 -- 172.21.15.119:0/2648642471 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe2c8011bb0 con 0x7fe2d8159c60 2023-10-08T02:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.739+0000 7fe2ce7fc700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe2bc066f30 0x7fe2bc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.739+0000 7fe2ce7fc700 1 -- 172.21.15.119:0/2648642471 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fe2c8085fe0 con 0x7fe2d8159c60 2023-10-08T02:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.739+0000 7fe2de19b700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe2bc066f30 0x7fe2bc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:06.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.740+0000 7fe2de19b700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe2bc066f30 0x7fe2bc0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40093f0 tx=0x7fe2d4008040 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:06.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.743+0000 7fe2ce7fc700 1 -- 172.21.15.119:0/2648642471 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe2c80530d0 con 0x7fe2d8159c60 2023-10-08T02:32:06.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.936+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2d804c3c0 con 0x7fe2bc066f30 2023-10-08T02:32:06.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.936+0000 7fe2ce7fc700 1 -- 172.21.15.119:0/2648642471 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe2d804c3c0 con 0x7fe2bc066f30 2023-10-08T02:32:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe2bc066f30 msgr2=0x7fe2bc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe2bc066f30 0x7fe2bc0693e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe2d40093f0 tx=0x7fe2d4008040 comp rx=0 tx=0).stop 2023-10-08T02:32:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d8159c60 msgr2=0x7fe2d80cf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d8159c60 0x7fe2d80cf880 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8006fd0 tx=0x7fe2c8017000 comp rx=0 tx=0).stop 2023-10-08T02:32:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 shutdown_connections 2023-10-08T02:32:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fe2bc066f30 0x7fe2bc0693e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fe2d8159890 0x7fe2d8151490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fe2d815db10 0x7fe2d80cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 --2- 172.21.15.119:0/2648642471 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2d8159c60 0x7fe2d80cf880 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:06.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 >> 172.21.15.119:0/2648642471 conn(0x7fe2d8068e20 msgr2=0x7fe2d8154280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:06.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 shutdown_connections 2023-10-08T02:32:06.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:06.939+0000 7fe2e03ff700 1 -- 172.21.15.119:0/2648642471 wait complete. 2023-10-08T02:32:06.941 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 6 to deep-scrub 2023-10-08T02:32:06.957 INFO:tasks.ceph:Scrubbing osd.7 2023-10-08T02:32:06.957 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-08T02:32:07.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.140+0000 7fabc9fff700 1 Processor -- start 2023-10-08T02:32:07.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.141+0000 7fabc9fff700 1 -- start start 2023-10-08T02:32:07.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc9fff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fabc4157740 0x7fabc40adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc9fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40b2100 0x7fabc4068390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc9fff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc406dc20 con 0x7fabc40688d0 2023-10-08T02:32:07.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc9fff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc40acc00 con 0x7fabc4157740 2023-10-08T02:32:07.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc9fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc406d6a0 con 0x7fabc40b2100 2023-10-08T02:32:07.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc3fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fabc40688d0 0x7fabc4069800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46594/0 (socket says 172.21.15.119:46594) 2023-10-08T02:32:07.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc3fff700 1 -- 172.21.15.119:0/310307280 learned_addr learned my addr 172.21.15.119:0/310307280 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:07.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabbbfff700 1 --2- 172.21.15.119:0/310307280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40b2100 0x7fabc4068390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc37fe700 1 --2- 172.21.15.119:0/310307280 >> v2:172.21.15.184:3300/0 conn(0x7fabc4157740 0x7fabc40adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.142+0000 7fabc2ffd700 1 -- 172.21.15.119:0/310307280 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041859502 0 0) 0x7fabc406dc20 con 0x7fabc40688d0 2023-10-08T02:32:07.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabc2ffd700 1 -- 172.21.15.119:0/310307280 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb0003570 con 0x7fabc40688d0 2023-10-08T02:32:07.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabbbfff700 1 -- 172.21.15.119:0/310307280 >> v2:172.21.15.184:3300/0 conn(0x7fabc4157740 msgr2=0x7fabc40adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:07.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabbbfff700 1 --2- 172.21.15.119:0/310307280 >> v2:172.21.15.184:3300/0 conn(0x7fabc4157740 0x7fabc40adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabbbfff700 1 -- 172.21.15.119:0/310307280 >> v1:172.21.15.119:6789/0 conn(0x7fabc40688d0 legacy=0x7fabc4069800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabbbfff700 1 -- 172.21.15.119:0/310307280 --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc40ae910 con 0x7fabc40b2100 2023-10-08T02:32:07.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabc37fe700 1 --2- 172.21.15.119:0/310307280 >> v2:172.21.15.184:3300/0 conn(0x7fabc4157740 0x7fabc40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-08T02:32:07.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabbbfff700 1 --2- 172.21.15.119:0/310307280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40b2100 0x7fabc4068390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fabac0098e0 tx=0x7fabac0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7377eb8fbe39588 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabc2ffd700 1 -- 172.21.15.119:0/310307280 <== mon.2 v2:172.21.15.184:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac011630 con 0x7fabc40b2100 2023-10-08T02:32:07.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabc2ffd700 1 -- 172.21.15.119:0/310307280 <== mon.2 v2:172.21.15.184:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac011790 con 0x7fabc40b2100 2023-10-08T02:32:07.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.143+0000 7fabc2ffd700 1 -- 172.21.15.119:0/310307280 <== mon.2 v2:172.21.15.184:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac011b30 con 0x7fabc40b2100 2023-10-08T02:32:07.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.144+0000 7fabc9fff700 1 -- 172.21.15.119:0/310307280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40b2100 msgr2=0x7fabc4068390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.144+0000 7fabc9fff700 1 --2- 172.21.15.119:0/310307280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40b2100 0x7fabc4068390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fabac0098e0 tx=0x7fabac0091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.144+0000 7fabc9fff700 1 -- 172.21.15.119:0/310307280 shutdown_connections 2023-10-08T02:32:07.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.144+0000 7fabc9fff700 1 --2- 172.21.15.119:0/310307280 >> v2:172.21.15.184:3300/0 conn(0x7fabc4157740 0x7fabc40adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.144+0000 7fabc9fff700 1 --2- 172.21.15.119:0/310307280 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40b2100 0x7fabc4068390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.144+0000 7fabc9fff700 1 -- 172.21.15.119:0/310307280 >> 172.21.15.119:0/310307280 conn(0x7fabc40c8f90 msgr2=0x7fabc40c9390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:07.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.144+0000 7fabc9fff700 1 -- 172.21.15.119:0/310307280 shutdown_connections 2023-10-08T02:32:07.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.144+0000 7fabc9fff700 1 -- 172.21.15.119:0/310307280 wait complete. 2023-10-08T02:32:07.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.145+0000 7fabc9fff700 1 Processor -- start 2023-10-08T02:32:07.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.145+0000 7fabc9fff700 1 -- start start 2023-10-08T02:32:07.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.146+0000 7fabc9fff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40688d0 0x7fabc40b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.146+0000 7fabc9fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabc40b2100 0x7fabc415c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.146+0000 7fabc9fff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabc4157740 0x7fabc415b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.146+0000 7fabc9fff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc414ff10 con 0x7fabc40b2100 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.146+0000 7fabc9fff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc414f4e0 con 0x7fabc4157740 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.146+0000 7fabc9fff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc415ca00 con 0x7fabc40688d0 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabc40b2100 0x7fabc415c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabc40b2100 0x7fabc415c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52298/0 (socket says 172.21.15.119:52298) 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 -- 172.21.15.119:0/989979662 learned_addr learned my addr 172.21.15.119:0/989979662 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabc3fff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabc4157740 0x7fabc415b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 -- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40688d0 msgr2=0x7fabc40b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fabc40688d0 0x7fabc40b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 -- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabc4157740 msgr2=0x7fabc415b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabc4157740 0x7fabc415b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 -- 172.21.15.119:0/989979662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb4009420 con 0x7fabc40b2100 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabc3fff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fabc4157740 0x7fabc415b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbbfff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabc40b2100 0x7fabc415c4c0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fabc406d7e0 tx=0x7fabac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.147+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac011b10 con 0x7fabc40b2100 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.148+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac0253f0 con 0x7fabc40b2100 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.148+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac0257b0 con 0x7fabc40b2100 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.148+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac009420 con 0x7fabc40b2100 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.148+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc415cc30 con 0x7fabc40b2100 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.148+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fabc40718b0 con 0x7fabc40b2100 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.149+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabac025ba0 con 0x7fabc40b2100 2023-10-08T02:32:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.150+0000 7fabbb7fe700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7faba4066f30 0x7faba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.150+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fabac086c60 con 0x7fabc40b2100 2023-10-08T02:32:07.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.151+0000 7fabc37fe700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7faba4066f30 0x7faba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.151+0000 7fabbb7fe700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7faba406d6d0 0x7faba406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.151+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 --> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faba4070170 con 0x7faba406d6d0 2023-10-08T02:32:07.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.151+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=211) v2 ==== 24+0+0 (secure 0 0 0) 0x7fabac086f50 con 0x7fabc40b2100 2023-10-08T02:32:07.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.151+0000 7fabc37fe700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7faba4066f30 0x7faba40693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009cd0 tx=0x7fabb40091a0 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.151+0000 7fabc3fff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7faba406d6d0 0x7faba406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.152+0000 7fabc3fff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7faba406d6d0 0x7faba406fb40 crc :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.152+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 <== osd.7 v2:172.21.15.184:6800/86001 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7faba4070170 con 0x7faba406d6d0 2023-10-08T02:32:07.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.167+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 --> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fabc40ca500 con 0x7faba406d6d0 2023-10-08T02:32:07.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.167+0000 7fabbb7fe700 1 -- 172.21.15.119:0/989979662 <== osd.7 v2:172.21.15.184:6800/86001 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fabc40ca500 con 0x7faba406d6d0 2023-10-08T02:32:07.169 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-10-08T02:32:07.169 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-08T02:32:07.169 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-10-08T02:32:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7faba406d6d0 msgr2=0x7faba406fb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.184:6800/86001,v1:172.21.15.184:6801/86001] conn(0x7faba406d6d0 0x7faba406fb40 crc :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7faba4066f30 msgr2=0x7faba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7faba4066f30 0x7faba40693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009cd0 tx=0x7fabb40091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabc40b2100 msgr2=0x7fabc415c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 --2- 172.21.15.119:0/989979662 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabc40b2100 0x7fabc415c4c0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fabc406d7e0 tx=0x7fabac017000 comp rx=0 tx=0).stop 2023-10-08T02:32:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc3fff700 1 -- 172.21.15.119:0/989979662 reap_dead start 2023-10-08T02:32:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 shutdown_connections 2023-10-08T02:32:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 >> 172.21.15.119:0/989979662 conn(0x7fabc40c8f90 msgr2=0x7fabc415d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 shutdown_connections 2023-10-08T02:32:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.168+0000 7fabc9fff700 1 -- 172.21.15.119:0/989979662 wait complete. 2023-10-08T02:32:07.185 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-08T02:32:07.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.367+0000 7f73e679d700 1 Processor -- start 2023-10-08T02:32:07.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.368+0000 7f73e679d700 1 -- start start 2023-10-08T02:32:07.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e679d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e0156310 0x7f73e00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e679d700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73e01566e0 0x7f73e00ae000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e679d700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e00ca860 con 0x7f73e00ca300 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e679d700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e00cab40 con 0x7f73e01566e0 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e679d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e00cae20 con 0x7f73e0156310 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e4f9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73e01566e0 0x7f73e00ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e4f9a700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73e01566e0 0x7f73e00ae000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52314/0 (socket says 172.21.15.119:52314) 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e4f9a700 1 -- 172.21.15.119:0/3047433480 learned_addr learned my addr 172.21.15.119:0/3047433480 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.369+0000 7f73e579b700 1 --2- 172.21.15.119:0/3047433480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e0156310 0x7f73e00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.370+0000 7f73e4f9a700 1 -- 172.21.15.119:0/3047433480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e0156310 msgr2=0x7f73e00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:07.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.370+0000 7f73e4f9a700 1 --2- 172.21.15.119:0/3047433480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e0156310 0x7f73e00c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.370+0000 7f73e4f9a700 1 -- 172.21.15.119:0/3047433480 >> v1:172.21.15.184:6789/0 conn(0x7f73e00ca300 legacy=0x7f73e00c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.370+0000 7f73e4f9a700 1 -- 172.21.15.119:0/3047433480 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e00cb100 con 0x7f73e01566e0 2023-10-08T02:32:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.370+0000 7f73e4f9a700 1 --2- 172.21.15.119:0/3047433480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e01566e0 0x7f73e00ae000 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bc50 tx=0x7f73d000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2795ac7b09dd0d server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.370+0000 7f73d7fff700 1 -- 172.21.15.119:0/3047433480 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d000dbf0 con 0x7f73e01566e0 2023-10-08T02:32:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.370+0000 7f73d7fff700 1 -- 172.21.15.119:0/3047433480 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d000dd50 con 0x7f73e01566e0 2023-10-08T02:32:07.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.370+0000 7f73d7fff700 1 -- 172.21.15.119:0/3047433480 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d00196b0 con 0x7f73e01566e0 2023-10-08T02:32:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.371+0000 7f73e679d700 1 -- 172.21.15.119:0/3047433480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e01566e0 msgr2=0x7f73e00ae000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.371+0000 7f73e679d700 1 --2- 172.21.15.119:0/3047433480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e01566e0 0x7f73e00ae000 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f73d000bc50 tx=0x7f73d000b4f0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.371+0000 7f73e679d700 1 -- 172.21.15.119:0/3047433480 shutdown_connections 2023-10-08T02:32:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.371+0000 7f73e679d700 1 --2- 172.21.15.119:0/3047433480 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e0156310 0x7f73e00c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.371+0000 7f73e679d700 1 --2- 172.21.15.119:0/3047433480 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e01566e0 0x7f73e00ae000 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.371+0000 7f73e679d700 1 -- 172.21.15.119:0/3047433480 >> 172.21.15.119:0/3047433480 conn(0x7f73e00cc1f0 msgr2=0x7f73e00cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.371+0000 7f73e679d700 1 -- 172.21.15.119:0/3047433480 shutdown_connections 2023-10-08T02:32:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.371+0000 7f73e679d700 1 -- 172.21.15.119:0/3047433480 wait complete. 2023-10-08T02:32:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.372+0000 7f73e679d700 1 Processor -- start 2023-10-08T02:32:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.372+0000 7f73e679d700 1 -- start start 2023-10-08T02:32:07.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.373+0000 7f73e679d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e00ca300 0x7f73e015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.373+0000 7f73e679d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e0156310 0x7f73e015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.373+0000 7f73e679d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e01566e0 0x7f73e0141c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.373+0000 7f73e679d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e0151390 con 0x7f73e0156310 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.373+0000 7f73e679d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e015afe0 con 0x7f73e01566e0 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.373+0000 7f73e679d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e0158fe0 con 0x7f73e00ca300 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e0156310 0x7f73e015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e0156310 0x7f73e015aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52318/0 (socket says 172.21.15.119:52318) 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 -- 172.21.15.119:0/550862596 learned_addr learned my addr 172.21.15.119:0/550862596 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e579b700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e00ca300 0x7f73e015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e5f9c700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e01566e0 0x7f73e0141c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 -- 172.21.15.119:0/550862596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e00ca300 msgr2=0x7f73e015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e00ca300 0x7f73e015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 -- 172.21.15.119:0/550862596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e01566e0 msgr2=0x7f73e0141c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e01566e0 0x7f73e0141c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 -- 172.21.15.119:0/550862596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d000b790 con 0x7f73e0156310 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73e4f9a700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e0156310 0x7f73e015aaa0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f73d000b530 tx=0x7f73d001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.374+0000 7f73d57fa700 1 -- 172.21.15.119:0/550862596 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0010470 con 0x7f73e0156310 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.375+0000 7f73d57fa700 1 -- 172.21.15.119:0/550862596 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d00105f0 con 0x7f73e0156310 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.375+0000 7f73d57fa700 1 -- 172.21.15.119:0/550862596 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d00109b0 con 0x7f73e0156310 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.375+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e01592c0 con 0x7f73e0156310 2023-10-08T02:32:07.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.375+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e0159880 con 0x7f73e0156310 2023-10-08T02:32:07.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.376+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e004c3c0 con 0x7f73e0156310 2023-10-08T02:32:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.376+0000 7f73d57fa700 1 -- 172.21.15.119:0/550862596 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f73d0022030 con 0x7f73e0156310 2023-10-08T02:32:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.377+0000 7f73d57fa700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f73c4066f30 0x7f73c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.377+0000 7f73d57fa700 1 -- 172.21.15.119:0/550862596 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f73d0014070 con 0x7f73e0156310 2023-10-08T02:32:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.378+0000 7f73e579b700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f73c4066f30 0x7f73c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.378+0000 7f73e579b700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f73c4066f30 0x7f73c40693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009d40 tx=0x7f73dc008040 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.381+0000 7f73d57fa700 1 -- 172.21.15.119:0/550862596 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f73d0055940 con 0x7f73e0156310 2023-10-08T02:32:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.573+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f73e00cd0b0 con 0x7f73c4066f30 2023-10-08T02:32:07.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.573+0000 7f73d57fa700 1 -- 172.21.15.119:0/550862596 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f73e00cd0b0 con 0x7f73c4066f30 2023-10-08T02:32:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f73c4066f30 msgr2=0x7f73c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f73c4066f30 0x7f73c40693e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f73dc009d40 tx=0x7f73dc008040 comp rx=0 tx=0).stop 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e0156310 msgr2=0x7f73e015aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e0156310 0x7f73e015aaa0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f73d000b530 tx=0x7f73d001b000 comp rx=0 tx=0).stop 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 shutdown_connections 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f73c4066f30 0x7f73c40693e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f73e00ca300 0x7f73e015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f73e01566e0 0x7f73e0141c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 --2- 172.21.15.119:0/550862596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e0156310 0x7f73e015aaa0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 >> 172.21.15.119:0/550862596 conn(0x7f73e00cc1f0 msgr2=0x7f73e006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.575+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 shutdown_connections 2023-10-08T02:32:07.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.576+0000 7f73e679d700 1 -- 172.21.15.119:0/550862596 wait complete. 2023-10-08T02:32:07.578 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 7 to deep-scrub 2023-10-08T02:32:07.594 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:32:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.775+0000 7fb3741ff700 1 Processor -- start 2023-10-08T02:32:07.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.775+0000 7fb3741ff700 1 -- start start 2023-10-08T02:32:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.776+0000 7fb3741ff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb36c0b1850 0x7fb36c0b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.776+0000 7fb3741ff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7fb36c0ae680 0x7fb36c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.776+0000 7fb3741ff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb36c0cb300 con 0x7fb36c0aeb00 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.776+0000 7fb3741ff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c06fc30 con 0x7fb36c0b1850 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.776+0000 7fb3741ff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36c0cc260 con 0x7fb36c0ae680 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb371f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb36c0b1850 0x7fb36c0b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb371f9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb36c0b1850 0x7fb36c0b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52330/0 (socket says 172.21.15.119:52330) 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb371f9b700 1 -- 172.21.15.119:0/1556705689 learned_addr learned my addr 172.21.15.119:0/1556705689 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb37179a700 1 --2- 172.21.15.119:0/1556705689 >> v2:172.21.15.184:3300/0 conn(0x7fb36c0ae680 0x7fb36c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb371f9b700 1 -- 172.21.15.119:0/1556705689 >> v2:172.21.15.184:3300/0 conn(0x7fb36c0ae680 msgr2=0x7fb36c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:07.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb371f9b700 1 --2- 172.21.15.119:0/1556705689 >> v2:172.21.15.184:3300/0 conn(0x7fb36c0ae680 0x7fb36c1572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb371f9b700 1 -- 172.21.15.119:0/1556705689 >> v1:172.21.15.184:6789/0 conn(0x7fb36c0aeb00 legacy=0x7fb36c15b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:07.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb371f9b700 1 -- 172.21.15.119:0/1556705689 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c0cb5e0 con 0x7fb36c0b1850 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb371f9b700 1 --2- 172.21.15.119:0/1556705689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0b1850 0x7fb36c0b1c20 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fb36c06d340 tx=0x7fb368009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d116a1b70556ffa server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.777+0000 7fb370f99700 1 -- 172.21.15.119:0/1556705689 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36800b820 con 0x7fb36c0b1850 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb370f99700 1 -- 172.21.15.119:0/1556705689 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb36800b980 con 0x7fb36c0b1850 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb370f99700 1 -- 172.21.15.119:0/1556705689 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb36800bd20 con 0x7fb36c0b1850 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb3741ff700 1 -- 172.21.15.119:0/1556705689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0b1850 msgr2=0x7fb36c0b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb3741ff700 1 --2- 172.21.15.119:0/1556705689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0b1850 0x7fb36c0b1c20 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7fb36c06d340 tx=0x7fb368009250 comp rx=0 tx=0).stop 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb3741ff700 1 -- 172.21.15.119:0/1556705689 shutdown_connections 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb3741ff700 1 --2- 172.21.15.119:0/1556705689 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0b1850 0x7fb36c0b1c20 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb3741ff700 1 --2- 172.21.15.119:0/1556705689 >> v2:172.21.15.184:3300/0 conn(0x7fb36c0ae680 0x7fb36c1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb3741ff700 1 -- 172.21.15.119:0/1556705689 >> 172.21.15.119:0/1556705689 conn(0x7fb36c06a320 msgr2=0x7fb36c0cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb3741ff700 1 -- 172.21.15.119:0/1556705689 shutdown_connections 2023-10-08T02:32:07.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.778+0000 7fb3741ff700 1 -- 172.21.15.119:0/1556705689 wait complete. 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.780+0000 7fb3741ff700 1 Processor -- start 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.780+0000 7fb3741ff700 1 -- start start 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb3741ff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb36c0ae680 0x7fb36c144f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb3741ff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0aeb00 0x7fb36c0c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb3741ff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb36c0b1850 0x7fb36c0c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb3741ff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0c12d0 con 0x7fb36c0aeb00 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb3741ff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0c1520 con 0x7fb36c0b1850 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb3741ff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c0c1800 con 0x7fb36c0ae680 2023-10-08T02:32:07.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37179a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0aeb00 0x7fb36c0c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37179a700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0aeb00 0x7fb36c0c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52332/0 (socket says 172.21.15.119:52332) 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37179a700 1 -- 172.21.15.119:0/260300113 learned_addr learned my addr 172.21.15.119:0/260300113 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb371f9b700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb36c0ae680 0x7fb36c144f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37279c700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb36c0b1850 0x7fb36c0c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37179a700 1 -- 172.21.15.119:0/260300113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb36c0ae680 msgr2=0x7fb36c144f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37179a700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb36c0ae680 0x7fb36c144f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37179a700 1 -- 172.21.15.119:0/260300113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb36c0b1850 msgr2=0x7fb36c0c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37179a700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb36c0b1850 0x7fb36c0c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.781+0000 7fb37179a700 1 -- 172.21.15.119:0/260300113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3680094f0 con 0x7fb36c0aeb00 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.782+0000 7fb37179a700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0aeb00 0x7fb36c0c3bc0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00d890 tx=0x7fb35c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.782+0000 7fb361ffb700 1 -- 172.21.15.119:0/260300113 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c0125f0 con 0x7fb36c0aeb00 2023-10-08T02:32:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.782+0000 7fb361ffb700 1 -- 172.21.15.119:0/260300113 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb35c012770 con 0x7fb36c0aeb00 2023-10-08T02:32:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.782+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c0c1ae0 con 0x7fb36c0aeb00 2023-10-08T02:32:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.782+0000 7fb361ffb700 1 -- 172.21.15.119:0/260300113 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb35c012b30 con 0x7fb36c0aeb00 2023-10-08T02:32:07.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.782+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c169ee0 con 0x7fb36c0aeb00 2023-10-08T02:32:07.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.783+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c0c2000 con 0x7fb36c0aeb00 2023-10-08T02:32:07.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.784+0000 7fb361ffb700 1 -- 172.21.15.119:0/260300113 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb35c01e460 con 0x7fb36c0aeb00 2023-10-08T02:32:07.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.785+0000 7fb361ffb700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:07.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.785+0000 7fb361ffb700 1 -- 172.21.15.119:0/260300113 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7fb35c084e30 con 0x7fb36c0aeb00 2023-10-08T02:32:07.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.785+0000 7fb371f9b700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:07.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.785+0000 7fb371f9b700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb36c06d340 tx=0x7fb368016000 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:07.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.789+0000 7fb361ffb700 1 -- 172.21.15.119:0/260300113 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb36c0c2000 con 0x7fb36c0aeb00 2023-10-08T02:32:07.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.976+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb36c0c1690 con 0x7fb350066f30 2023-10-08T02:32:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.978+0000 7fb361ffb700 1 -- 172.21.15.119:0/260300113 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50503 (secure 0 0 0) 0x7fb36c0c1690 con 0x7fb350066f30 2023-10-08T02:32:07.979 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 msgr2=0x7fb3500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb36c06d340 tx=0x7fb368016000 comp rx=0 tx=0).stop 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0aeb00 msgr2=0x7fb36c0c3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0aeb00 0x7fb36c0c3bc0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7fb35c00d890 tx=0x7fb35c008040 comp rx=0 tx=0).stop 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 shutdown_connections 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7fb350066f30 0x7fb3500693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7fb36c0ae680 0x7fb36c144f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7fb36c0b1850 0x7fb36c0c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 --2- 172.21.15.119:0/260300113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb36c0aeb00 0x7fb36c0c3bc0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:07.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 >> 172.21.15.119:0/260300113 conn(0x7fb36c06a320 msgr2=0x7fb36c06d890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:07.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 shutdown_connections 2023-10-08T02:32:07.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:07.981+0000 7fb3741ff700 1 -- 172.21.15.119:0/260300113 wait complete. 2023-10-08T02:32:07.983 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:32:07.999 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-10-08T02:32:05.988597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2469800,"kb_used_data":21864,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752504920,"statfs":{"total":773094113280,"available":770565038080,"internally_reserved":0,"allocated":22388736,"data_stored":11879920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001580"},"pg_stats":[{"pgid":"1.0","version":"209'370","reported_seq":1222,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-10-08T02:31:54.059304+0000","last_change":"2023-10-08T00:55:37.817903+0000","last_active":"2023-10-08T02:31:54.059304+0000","last_peered":"2023-10-08T02:31:54.059304+0000","last_clean":"2023-10-08T02:31:54.059304+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:31:54.059304+0000","last_undegraded":"2023-10-08T02:31:54.059304+0000","last_fullsized":"2023-10-08T02:31:54.059304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-08T00:55:36.227481+0000","last_clean_scrub_stamp":"2023-10-08T00:55:36.227481+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T03:20:11.068745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":370,"ondisk_log_size":370,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575989,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316840,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055000,"statfs":{"total":96636764160,"available":96312320000,"internally_reserved":0,"allocated":3244032,"data_stored":1927766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93799999999999994,"5min":0.96899999999999997,"15min":1.0349999999999999},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85999999999999999,"5min":0.93899999999999995,"15min":0.99199999999999999},"last":0.62}]},{"osd":1,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":0.83499999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73299999999999998,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93400000000000005,"5min":0.98499999999999999,"15min":1.002},"last":0.746}]},{"osd":2,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.875,"5min":0.96699999999999997,"15min":1.0549999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73699999999999999,"15min":0.748},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93000000000000005,"5min":0.96699999999999997,"15min":1.03},"last":0.66700000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.71699999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.78500000000000003},"min":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98099999999999998,"5min":1.0069999999999999,"15min":1.056},"last":0.56799999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":1.091},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.92200000000000004},"last":0.58999999999999997}]},{"osd":5,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":1.012},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.47999999999999998,"15min":0.44500000000000001},"max":{"1min":0.873,"5min":0.92100000000000004,"15min":1.1240000000000001},"last":0.71999999999999997}]},{"osd":6,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.877,"5min":1.0089999999999999,"15min":1.151},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70299999999999996,"15min":0.71399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.42599999999999999},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78500000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.55100000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92400000000000004,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.85699999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":1.21},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76300000000000001,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":3.851,"15min":3.851},"last":0.63600000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":0.86799999999999999,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.258},"last":0.66100000000000003}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.52600000000000002},"max":{"1min":0.88500000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.878}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.378},"max":{"1min":0.95299999999999996,"5min":1.139,"15min":1.139},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.48499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":1.341,"15min":1.341},"last":0.90900000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.621,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.92800000000000005},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.443},"max":{"1min":0.91600000000000004,"5min":1.089,"15min":1.089},"last":0.89500000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.038,"15min":1.0980000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.95599999999999996,"5min":0.96999999999999997,"15min":1.054},"last":0.92700000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312164,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059676,"statfs":{"total":96636764160,"available":96317108224,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.0780000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.29199999999999998,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.571},"last":0.49399999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":1.1100000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":1.1299999999999999},"last":0.51400000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.755,"5min":0.80200000000000005,"15min":1.2190000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":1.341,"15min":1.341},"last":0.52600000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.48199999999999998},"max":{"1min":1.171,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.82499999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.97499999999999998,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.127,"15min":1.27},"last":0.89800000000000002}]},{"osd":6,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.81799999999999995},"min":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.915,"15min":1.915},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.50900000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.204},"last":0.91600000000000004}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.85799999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.306},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.66800000000000004,"5min":0.57999999999999996,"15min":0.55500000000000005},"max":{"1min":1.1399999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.90600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308648,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063192,"statfs":{"total":96636764160,"available":96320708608,"internally_reserved":0,"allocated":3244032,"data_stored":1927766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44700000000000001,"15min":0.46800000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23699999999999999},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.78400000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.76300000000000001,"15min":0.88100000000000001},"last":0.42399999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.48499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.56499999999999995,"5min":0.79400000000000004,"15min":0.83199999999999996},"last":0.48599999999999999}]},{"osd":3,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.81799999999999995,"15min":0.93100000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.81399999999999995,"15min":0.89000000000000001},"last":0.51400000000000001}]},{"osd":4,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.75700000000000001,"15min":0.78000000000000003},"min":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.85999999999999999,"5min":1.159,"15min":1.232},"last":0.62},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90500000000000003,"5min":1.306,"15min":1.306},"last":0.67700000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94099999999999995,"5min":1.1990000000000001,"15min":1.216},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98299999999999998,"5min":1.27,"15min":1.296},"last":0.70199999999999996}]},{"osd":6,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79300000000000004,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.874,"5min":1.135,"15min":1.1579999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96899999999999997,"5min":1.262,"15min":1.262},"last":0.58699999999999997}]},{"osd":7,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.83399999999999996},"min":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83199999999999996,"15min":0.86499999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.55700000000000005},"max":{"1min":0.89900000000000002,"5min":1.2250000000000001,"15min":1.3360000000000001},"last":0.71599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":316260,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055580,"statfs":{"total":96636764160,"available":96312913920,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61699999999999999,"5min":0.755,"15min":0.871},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.437}]},{"osd":1,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.70599999999999996,"5min":0.876,"15min":0.876},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.495,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.80900000000000005},"last":0.52400000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.73899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97799999999999998,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.748}]},{"osd":5,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0229999999999999,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82399999999999995,"15min":0.84899999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.353,"15min":1.417},"last":0.629}]},{"osd":6,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.50900000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94799999999999995,"5min":1.153,"15min":1.333},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.745,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94599999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.387,"15min":1.4059999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":1.252,"15min":1.5720000000000001},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.495,"15min":0.495},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.33000000000000002}]},{"osd":1,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.44500000000000001}]},{"osd":2,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.512},"last":0.495}]},{"osd":4,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.5,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.56000000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.0680000000000001,"5min":1.113,"15min":1.3180000000000001},"last":0.60499999999999998}]},{"osd":5,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.57999999999999996,"5min":0.441,"15min":0.441},"max":{"1min":1.117,"5min":1.117,"15min":1.2789999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83499999999999996,"5min":1.1220000000000001,"15min":1.24},"last":0.61799999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89000000000000001,"5min":1.0880000000000001,"15min":1.161},"last":0.755},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.95999999999999996,"5min":1.155,"15min":1.2849999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.82699999999999996},"min":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91600000000000004,"5min":1.081,"15min":1.266},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.282},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.70499999999999996,"15min":0.71499999999999997},"min":{"1min":0.50600000000000001,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":1.028},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.70499999999999996,"15min":0.72899999999999998},"min":{"1min":0.502,"5min":0.497,"15min":0.40000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.56000000000000005}]},{"osd":1,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.70799999999999996,"15min":0.72299999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.90800000000000003,"5min":0.998,"15min":1.0409999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.76600000000000001,"5min":0.73299999999999998,"15min":0.73999999999999999},"min":{"1min":0.51400000000000001,"5min":0.496,"15min":0.49099999999999999},"max":{"1min":1.01,"5min":1.079,"15min":1.079},"last":0.54400000000000004}]},{"osd":2,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44600000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.034},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.78500000000000003,"15min":0.78900000000000003},"min":{"1min":0.65400000000000003,"5min":0.441,"15min":0.441},"max":{"1min":1.022,"5min":1.097,"15min":1.097},"last":0.5}]},{"osd":3,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.755,"15min":0.77400000000000002},"min":{"1min":0.52900000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.995,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79600000000000004,"15min":0.79900000000000004},"min":{"1min":0.63,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.98399999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.53600000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.62},"min":{"1min":0.42199999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":1.1259999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.754},"min":{"1min":0.65600000000000003,"5min":0.502,"15min":0.48199999999999998},"max":{"1min":1.004,"5min":1.0269999999999999,"15min":1.105},"last":0.59399999999999997}]},{"osd":6,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.65200000000000002},"min":{"1min":0.45900000000000002,"5min":0.45000000000000001,"15min":0.42399999999999999},"max":{"1min":0.82099999999999995,"5min":0.98299999999999998,"15min":1.0389999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.68200000000000005,"15min":0.67700000000000005},"min":{"1min":0.52900000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92300000000000004,"5min":1.03,"15min":1.03},"last":0.64200000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:31:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71799999999999997,"15min":0.68899999999999995},"min":{"1min":0.55300000000000005,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.98099999999999998,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75900000000000001,"15min":0.73399999999999999},"min":{"1min":0.51600000000000001,"5min":0.46999999999999997,"15min":0.42699999999999999},"max":{"1min":0.95199999999999996,"5min":1.0429999999999999,"15min":1.0669999999999999},"last":0.625}]}]},{"osd":5,"up_from":13,"seq":55834575988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":316260,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055580,"statfs":{"total":96636764160,"available":96312913920,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.70899999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73599999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94399999999999995,"15min":0.99199999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73299999999999998,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.91900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.68700000000000006}]},{"osd":2,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.028,"15min":1.028},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.75600000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90200000000000002,"5min":1.02,"15min":1.02},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.39600000000000002,"5min":0.38100000000000001,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":1.0449999999999999},"last":0.57399999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0029999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.92700000000000005,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.61799999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.52600000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.873,"5min":1.0429999999999999,"15min":1.044},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.753,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.74399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:32:08.003 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-08T00:55:36.227481+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=0, tm_min=55, tm_sec=36, tm_wday=6, tm_yday=281, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=8, tm_hour=2, tm_min=32, tm_sec=1, tm_wday=6, tm_yday=281, tm_isdst=0) 2023-10-08T02:32:08.003 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-08T02:32:14.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:32:14.233 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:32:28.004 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-08T02:32:28.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.195+0000 7f9524bff700 1 Processor -- start 2023-10-08T02:32:28.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.195+0000 7f9524bff700 1 -- start start 2023-10-08T02:32:28.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.196+0000 7f9524bff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f9520155c10 0x7f9520155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.196+0000 7f9524bff700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95200acac0 con 0x7f95200ac6d0 2023-10-08T02:32:28.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.196+0000 7f9524bff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95200ad930 con 0x7f95200ac230 2023-10-08T02:32:28.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.196+0000 7f9524bff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95200acd20 con 0x7f9520155c10 2023-10-08T02:32:28.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951ed9d700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f95200ac6d0 0x7f95200b0300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35768/0 (socket says 172.21.15.119:35768) 2023-10-08T02:32:28.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951ed9d700 1 -- 172.21.15.119:0/1471097186 learned_addr learned my addr 172.21.15.119:0/1471097186 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:28.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951e59c700 1 --2- 172.21.15.119:0/1471097186 >> v2:172.21.15.184:3300/0 conn(0x7f9520155c10 0x7f9520155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951d59a700 1 -- 172.21.15.119:0/1471097186 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443507069 0 0) 0x7f95200acac0 con 0x7f95200ac6d0 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951d59a700 1 -- 172.21.15.119:0/1471097186 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9508003570 con 0x7f95200ac6d0 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951d59a700 1 -- 172.21.15.119:0/1471097186 <== mon.1 v1:172.21.15.184:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263755314 0 0) 0x7f95200ad930 con 0x7f95200ac230 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951d59a700 1 -- 172.21.15.119:0/1471097186 --> v1:172.21.15.184:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95200acac0 con 0x7f95200ac230 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951e59c700 1 -- 172.21.15.119:0/1471097186 >> v1:172.21.15.184:6789/0 conn(0x7f95200ac230 legacy=0x7f95201568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951e59c700 1 -- 172.21.15.119:0/1471097186 >> v1:172.21.15.119:6789/0 conn(0x7f95200ac6d0 legacy=0x7f95200b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951d59a700 1 -- 172.21.15.119:0/1471097186 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734835254 0 0) 0x7f9508003570 con 0x7f95200ac6d0 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.197+0000 7f951e59c700 1 -- 172.21.15.119:0/1471097186 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95200ad000 con 0x7f9520155c10 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.198+0000 7f951e59c700 1 --2- 172.21.15.119:0/1471097186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9520155c10 0x7f9520155fe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f95100098e0 tx=0x7f95100091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f88e74eebf214e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.198+0000 7f951d59a700 1 -- 172.21.15.119:0/1471097186 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95180017f0 con 0x7f9520155c10 2023-10-08T02:32:28.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.198+0000 7f951d59a700 1 -- 172.21.15.119:0/1471097186 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9510011420 con 0x7f9520155c10 2023-10-08T02:32:28.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.198+0000 7f951d59a700 1 -- 172.21.15.119:0/1471097186 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95100117c0 con 0x7f9520155c10 2023-10-08T02:32:28.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.198+0000 7f9524bff700 1 -- 172.21.15.119:0/1471097186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9520155c10 msgr2=0x7f9520155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.198+0000 7f9524bff700 1 --2- 172.21.15.119:0/1471097186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9520155c10 0x7f9520155fe0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f95100098e0 tx=0x7f95100091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.199+0000 7f9524bff700 1 -- 172.21.15.119:0/1471097186 shutdown_connections 2023-10-08T02:32:28.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.199+0000 7f9524bff700 1 --2- 172.21.15.119:0/1471097186 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9520155c10 0x7f9520155fe0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.199+0000 7f9524bff700 1 -- 172.21.15.119:0/1471097186 >> 172.21.15.119:0/1471097186 conn(0x7f952006a320 msgr2=0x7f95200cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:28.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.199+0000 7f9524bff700 1 -- 172.21.15.119:0/1471097186 shutdown_connections 2023-10-08T02:32:28.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.199+0000 7f9524bff700 1 -- 172.21.15.119:0/1471097186 wait complete. 2023-10-08T02:32:28.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.200+0000 7f9524bff700 1 Processor -- start 2023-10-08T02:32:28.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.200+0000 7f9524bff700 1 -- start start 2023-10-08T02:32:28.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f9524bff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95200ac230 0x7f952013a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f9524bff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95200ac6d0 0x7f95200cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f9524bff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9520155c10 0x7f95200ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f9524bff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95201551c0 con 0x7f95200ac230 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f9524bff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95200ccb80 con 0x7f9520155c10 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f9524bff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95200ca760 con 0x7f95200ac6d0 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f951e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95200ac230 0x7f952013a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f951e59c700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95200ac230 0x7f952013a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41320/0 (socket says 172.21.15.119:41320) 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.201+0000 7f951e59c700 1 -- 172.21.15.119:0/1924313247 learned_addr learned my addr 172.21.15.119:0/1924313247 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f951e59c700 1 -- 172.21.15.119:0/1924313247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95200ac6d0 msgr2=0x7f95200cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f951e59c700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95200ac6d0 0x7f95200cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f951e59c700 1 -- 172.21.15.119:0/1924313247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9520155c10 msgr2=0x7f95200ca220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f951e59c700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9520155c10 0x7f95200ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f951e59c700 1 -- 172.21.15.119:0/1924313247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9510009420 con 0x7f95200ac230 2023-10-08T02:32:28.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f951e59c700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95200ac230 0x7f952013a8b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f95200acc50 tx=0x7f9510016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:28.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f950e7fc700 1 -- 172.21.15.119:0/1924313247 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951001cc90 con 0x7f95200ac230 2023-10-08T02:32:28.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f950e7fc700 1 -- 172.21.15.119:0/1924313247 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951001cdf0 con 0x7f95200ac230 2023-10-08T02:32:28.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952013d1f0 con 0x7f95200ac230 2023-10-08T02:32:28.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f950e7fc700 1 -- 172.21.15.119:0/1924313247 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951001a6a0 con 0x7f95200ac230 2023-10-08T02:32:28.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.202+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952013c8c0 con 0x7f95200ac230 2023-10-08T02:32:28.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.203+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952004c3c0 con 0x7f95200ac230 2023-10-08T02:32:28.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.204+0000 7f950e7fc700 1 -- 172.21.15.119:0/1924313247 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f951001aa50 con 0x7f95200ac230 2023-10-08T02:32:28.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.205+0000 7f950e7fc700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9500066f60 0x7f9500069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.205+0000 7f950e7fc700 1 -- 172.21.15.119:0/1924313247 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f9510085ff0 con 0x7f95200ac230 2023-10-08T02:32:28.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.205+0000 7f951dd9b700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9500066f60 0x7f9500069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.206+0000 7f951dd9b700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9500066f60 0x7f9500069410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95200b1970 tx=0x7f95140079d0 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:28.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.210+0000 7f950e7fc700 1 -- 172.21.15.119:0/1924313247 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9510053080 con 0x7f95200ac230 2023-10-08T02:32:28.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.396+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 --> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95200ad370 con 0x7f9500066f60 2023-10-08T02:32:28.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.399+0000 7f950e7fc700 1 -- 172.21.15.119:0/1924313247 <== mgr.10494 v2:172.21.15.119:6836/134108 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50627 (secure 0 0 0) 0x7f95200ad370 con 0x7f9500066f60 2023-10-08T02:32:28.400 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:32:28.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9500066f60 msgr2=0x7f9500069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9500066f60 0x7f9500069410 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f95200b1970 tx=0x7f95140079d0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95200ac230 msgr2=0x7f952013a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95200ac230 0x7f952013a8b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f95200acc50 tx=0x7f9510016000 comp rx=0 tx=0).stop 2023-10-08T02:32:28.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 shutdown_connections 2023-10-08T02:32:28.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9500066f60 0x7f9500069410 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9520155c10 0x7f95200ca220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f95200ac230 0x7f952013a8b0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 --2- 172.21.15.119:0/1924313247 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f95200ac6d0 0x7f95200cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 >> 172.21.15.119:0/1924313247 conn(0x7f952006a320 msgr2=0x7f95200af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:28.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 shutdown_connections 2023-10-08T02:32:28.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.401+0000 7f9524bff700 1 -- 172.21.15.119:0/1924313247 wait complete. 2023-10-08T02:32:28.404 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-10-08T02:32:28.419 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-10-08T02:32:27.993462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":2469800,"kb_used_data":21864,"kb_used_omap":0,"kb_used_meta":2447872,"kb_avail":752504920,"statfs":{"total":773094113280,"available":770565038080,"internally_reserved":0,"allocated":22388736,"data_stored":11879920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2506620928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002905"},"pg_stats":[{"pgid":"1.0","version":"211'371","reported_seq":1234,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-10-08T02:32:14.073968+0000","last_change":"2023-10-08T02:32:14.073931+0000","last_active":"2023-10-08T02:32:14.073968+0000","last_peered":"2023-10-08T02:32:14.073968+0000","last_clean":"2023-10-08T02:32:14.073968+0000","last_became_active":"2023-10-08T00:55:37.816760+0000","last_became_peered":"2023-10-08T00:55:37.816760+0000","last_unstale":"2023-10-08T02:32:14.073968+0000","last_undegraded":"2023-10-08T02:32:14.073968+0000","last_fullsized":"2023-10-08T02:32:14.073968+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"211'371","last_scrub_stamp":"2023-10-08T02:32:14.073834+0000","last_deep_scrub":"211'371","last_deep_scrub_stamp":"2023-10-08T02:32:14.073834+0000","last_clean_scrub_stamp":"2023-10-08T02:32:14.073834+0000","objects_scrubbed":2,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-09T13:43:28.428446+0000","scrub_duration":0.005610355,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2324,"num_read_kb":4838,"num_write":619,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":371,"ondisk_log_size":371,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575993,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316840,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055000,"statfs":{"total":96636764160,"available":96312320000,"internally_reserved":0,"allocated":3244032,"data_stored":1927766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.71499999999999997,"15min":0.73499999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.93799999999999994,"5min":0.96899999999999997,"15min":1.0349999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.85999999999999999,"5min":0.93899999999999995,"15min":0.99199999999999999},"last":0.80600000000000005}]},{"osd":1,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.72799999999999998,"15min":0.73699999999999999},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40600000000000003},"max":{"1min":0.83499999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.73299999999999998,"15min":0.73799999999999999},"min":{"1min":0.47399999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93400000000000005,"5min":0.98499999999999999,"15min":1.002},"last":0.753}]},{"osd":2,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.75,"15min":0.76000000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.875,"5min":0.96699999999999997,"15min":1.0549999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.73699999999999999,"15min":0.748},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.93000000000000005,"5min":0.96699999999999997,"15min":1.03},"last":0.76300000000000001}]},{"osd":3,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.70899999999999996,"15min":0.71699999999999997},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.89700000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.77400000000000002,"15min":0.78500000000000003},"min":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.98099999999999998,"5min":1.0069999999999999,"15min":1.056},"last":0.70399999999999996}]},{"osd":4,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.59899999999999998,"15min":0.60399999999999998},"min":{"1min":0.40200000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81899999999999995,"5min":0.89500000000000002,"15min":1.091},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.622,"15min":0.61499999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.38100000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.92200000000000004},"last":0.73499999999999999}]},{"osd":5,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64700000000000002,"15min":0.64300000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.39300000000000002},"max":{"1min":0.88700000000000001,"5min":0.92100000000000004,"15min":1.012},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.67100000000000004,"15min":0.67000000000000004},"min":{"1min":0.50600000000000001,"5min":0.47999999999999998,"15min":0.44500000000000001},"max":{"1min":0.873,"5min":0.92100000000000004,"15min":1.1240000000000001},"last":0.48999999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:53 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.76400000000000001,"15min":0.77000000000000002},"min":{"1min":0.58599999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.877,"5min":1.0089999999999999,"15min":1.151},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.70299999999999996,"15min":0.71399999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.42599999999999999},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.71699999999999997,"15min":0.72699999999999998},"min":{"1min":0.53300000000000003,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89700000000000002,"5min":0.97299999999999998,"15min":1.0449999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73899999999999999,"15min":0.72999999999999998},"min":{"1min":0.55100000000000005,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.92400000000000004,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.82799999999999996}]},{"osd":1,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.70799999999999996,"15min":0.73699999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.45300000000000001},"max":{"1min":0.89200000000000002,"5min":0.94599999999999995,"15min":1.21},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76300000000000001,"15min":0.751},"min":{"1min":0.59999999999999998,"5min":0.504,"15min":0.45700000000000002},"max":{"1min":0.92600000000000005,"5min":3.851,"15min":3.851},"last":0.74399999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68600000000000005,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44400000000000001},"max":{"1min":0.86799999999999999,"5min":0.92200000000000004,"15min":1.1000000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.74399999999999999,"15min":0.76400000000000001},"min":{"1min":0.57799999999999996,"5min":0.499,"15min":0.499},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":1.258},"last":0.90000000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.77300000000000002,"15min":0.78300000000000003},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.52600000000000002},"max":{"1min":0.88500000000000001,"5min":2.6970000000000001,"15min":2.6970000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.76200000000000001,"15min":0.77200000000000002},"min":{"1min":0.51100000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.91000000000000003,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.78200000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.46400000000000002,"5min":0.38800000000000001,"15min":0.378},"max":{"1min":0.95299999999999996,"5min":1.139,"15min":1.139},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63700000000000001,"15min":0.63500000000000001},"min":{"1min":0.48499999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82799999999999996,"5min":1.341,"15min":1.341},"last":0.53700000000000003}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.621,"15min":0.625},"min":{"1min":0.48499999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.92800000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68799999999999994,"15min":0.68400000000000005},"min":{"1min":0.45000000000000001,"5min":0.44600000000000001,"15min":0.443},"max":{"1min":0.91600000000000004,"5min":1.089,"15min":1.089},"last":0.64000000000000001}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.51100000000000001,"5min":0.47999999999999998,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.038,"15min":1.0980000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.753,"5min":0.73999999999999999,"15min":0.745},"min":{"1min":0.52600000000000002,"5min":0.437,"15min":0.437},"max":{"1min":0.95599999999999996,"5min":0.96999999999999997,"15min":1.054},"last":0.622}]}]},{"osd":1,"up_from":13,"seq":55834575993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":312164,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":309568,"kb_avail":94059676,"statfs":{"total":96636764160,"available":96317108224,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316997632},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47599999999999998,"15min":0.48599999999999999},"min":{"1min":0.28000000000000003,"5min":0.254,"15min":0.254},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":1.0780000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52200000000000002,"15min":0.53500000000000003},"min":{"1min":0.31900000000000001,"5min":0.29199999999999998,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0820000000000001,"15min":1.571},"last":0.59399999999999997}]},{"osd":2,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.33200000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":1.1100000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.76700000000000002,"15min":1.1299999999999999},"last":0.54600000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.755,"5min":0.80200000000000005,"15min":1.2190000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76900000000000002,"5min":1.341,"15min":1.341},"last":0.80700000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78800000000000003,"15min":0.78500000000000003},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.48199999999999998},"max":{"1min":1.171,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.84799999999999998,"15min":0.82099999999999995},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.40600000000000003},"max":{"1min":1.1020000000000001,"5min":4.2240000000000002,"15min":4.2240000000000002},"last":0.83199999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.79500000000000004,"15min":0.79000000000000004},"min":{"1min":0.56499999999999995,"5min":0.501,"15min":0.501},"max":{"1min":0.97499999999999998,"5min":1.4950000000000001,"15min":1.4950000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.79900000000000004,"15min":0.81200000000000006},"min":{"1min":0.60499999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.018,"5min":1.127,"15min":1.27},"last":0.74199999999999999}]},{"osd":6,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.82699999999999996,"15min":0.81799999999999995},"min":{"1min":0.55200000000000005,"5min":0.52900000000000003,"15min":0.52900000000000003},"max":{"1min":1.1060000000000001,"5min":1.915,"15min":1.915},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.83099999999999996,"15min":0.82599999999999996},"min":{"1min":0.59899999999999998,"5min":0.56899999999999995,"15min":0.50900000000000001},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.204},"last":0.71999999999999997}]},{"osd":7,"last update":"Sun Oct 8 02:31:30 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.84499999999999997,"15min":0.85799999999999998},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.202,"5min":1.202,"15min":1.306},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.872,"15min":0.86499999999999999},"min":{"1min":0.66800000000000004,"5min":0.57999999999999996,"15min":0.55500000000000005},"max":{"1min":1.1399999999999999,"5min":1.4630000000000001,"15min":1.4630000000000001},"last":0.91000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575992,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308648,"kb_used_data":3168,"kb_used_omap":0,"kb_used_meta":305472,"kb_avail":94063192,"statfs":{"total":96636764160,"available":96320708608,"internally_reserved":0,"allocated":3244032,"data_stored":1927766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312803328},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.44700000000000001,"15min":0.46800000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.23699999999999999},"max":{"1min":0.66100000000000003,"5min":0.77700000000000002,"15min":0.78400000000000003},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.49199999999999999,"15min":0.51600000000000001},"min":{"1min":0.33500000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63200000000000001,"5min":0.76300000000000001,"15min":0.88100000000000001},"last":0.47499999999999998}]},{"osd":2,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.46500000000000002,"15min":0.48499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.622,"5min":0.77700000000000002,"15min":0.84399999999999997},"last":0.442},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.52300000000000002,"15min":0.54300000000000004},"min":{"1min":0.36599999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.56499999999999995,"5min":0.79400000000000004,"15min":0.83199999999999996},"last":0.54400000000000004}]},{"osd":3,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54000000000000004,"15min":0.55500000000000005},"min":{"1min":0.40100000000000002,"5min":0.39400000000000002,"15min":0.36499999999999999},"max":{"1min":0.69099999999999995,"5min":0.81799999999999995,"15min":0.93100000000000005},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.48199999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76100000000000001,"5min":0.81399999999999995,"15min":0.89000000000000001},"last":0.69899999999999995}]},{"osd":4,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.75700000000000001,"15min":0.78000000000000003},"min":{"1min":0.51300000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.85999999999999999,"5min":1.159,"15min":1.232},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90500000000000003,"5min":1.306,"15min":1.306},"last":0.80000000000000004}]},{"osd":5,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.81599999999999995,"15min":0.83299999999999996},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.94099999999999995,"5min":1.1990000000000001,"15min":1.216},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.83599999999999997,"15min":0.86199999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98299999999999998,"5min":1.27,"15min":1.296},"last":0.81899999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.79300000000000004,"15min":0.82099999999999995},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.874,"5min":1.135,"15min":1.1579999999999999},"last":0.745},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.53200000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":0.96899999999999997,"5min":1.262,"15min":1.262},"last":0.77400000000000002}]},{"osd":7,"last update":"Sun Oct 8 02:31:50 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.81100000000000005,"15min":0.83399999999999996},"min":{"1min":0.54500000000000004,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.92000000000000004,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.83199999999999996,"15min":0.86499999999999999},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.55700000000000005},"max":{"1min":0.89900000000000002,"5min":1.2250000000000001,"15min":1.3360000000000001},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":316260,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055580,"statfs":{"total":96636764160,"available":96312913920,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.46300000000000002,"15min":0.47099999999999997},"min":{"1min":0.33900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.61699999999999999,"5min":0.755,"15min":0.871},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.51000000000000001,"15min":0.50900000000000001},"min":{"1min":0.33500000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65700000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54600000000000004}]},{"osd":1,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.70599999999999996,"5min":0.876,"15min":0.876},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.80900000000000005},"last":0.45500000000000002}]},{"osd":3,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60999999999999999,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.73899999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56399999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.67600000000000005,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.76100000000000001,"15min":0.78100000000000003},"min":{"1min":0.48899999999999999,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.98099999999999998,"5min":1.2090000000000001,"15min":1.3839999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.77200000000000002,"15min":0.79200000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97799999999999998,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.68300000000000005}]},{"osd":5,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.81399999999999995,"15min":0.83399999999999996},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0229999999999999,"5min":1.2270000000000001,"15min":1.3100000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.82399999999999995,"15min":0.84899999999999998},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.0429999999999999,"5min":1.353,"15min":1.417},"last":0.73299999999999998}]},{"osd":6,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.80500000000000005,"15min":0.81799999999999995},"min":{"1min":0.50900000000000001,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.94799999999999995,"5min":1.153,"15min":1.333},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.78700000000000003,"15min":0.81399999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.94599999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.86499999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:31:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.79900000000000004,"15min":0.81899999999999995},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.0620000000000001,"5min":1.387,"15min":1.4059999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81999999999999995,"15min":0.83199999999999996},"min":{"1min":0.54700000000000004,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.998,"5min":1.252,"15min":1.5720000000000001},"last":0.88400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.28199999999999997,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.495,"15min":0.495},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.311},"max":{"1min":0.74299999999999999,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.497}]},{"osd":1,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.496,"15min":0.49199999999999999},"min":{"1min":0.29899999999999999,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.34899999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.46800000000000003}]},{"osd":2,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.60499999999999998,"15min":0.61699999999999999},"min":{"1min":0.41399999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.4350000000000001,"5min":1.4350000000000001,"15min":1.4350000000000001},"last":0.621},{"interface":"front","average":{"1min":0.621,"5min":0.628,"15min":0.63500000000000001},"min":{"1min":0.48199999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.387,"5min":1.387,"15min":1.512},"last":0.65400000000000003}]},{"osd":4,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.752,"15min":0.76800000000000002},"min":{"1min":0.5,"5min":0.442,"15min":0.442},"max":{"1min":0.85099999999999998,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.56000000000000005,"5min":0.439,"15min":0.439},"max":{"1min":1.0680000000000001,"5min":1.113,"15min":1.3180000000000001},"last":0.70899999999999996}]},{"osd":5,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.76900000000000002,"15min":0.80300000000000005},"min":{"1min":0.57999999999999996,"5min":0.441,"15min":0.441},"max":{"1min":1.117,"5min":1.117,"15min":1.2789999999999999},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.79300000000000004,"15min":0.80400000000000005},"min":{"1min":0.52000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.83499999999999996,"5min":1.1220000000000001,"15min":1.24},"last":0.871}]},{"osd":6,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.77100000000000002,"15min":0.80600000000000005},"min":{"1min":0.54400000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":0.89000000000000001,"5min":1.0880000000000001,"15min":1.161},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.78800000000000003,"15min":0.81200000000000006},"min":{"1min":0.52700000000000002,"5min":0.5,"15min":0.5},"max":{"1min":0.95999999999999996,"5min":1.155,"15min":1.2849999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Sun Oct 8 02:31:46 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.80300000000000005,"15min":0.82699999999999996},"min":{"1min":0.58999999999999997,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":0.91600000000000004,"5min":1.081,"15min":1.266},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.80400000000000005,"15min":0.82399999999999995},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.161,"5min":1.161,"15min":1.282},"last":0.74099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":299876,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071964,"statfs":{"total":96636764160,"available":96329691136,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.69199999999999995,"15min":0.70999999999999996},"min":{"1min":0.48899999999999999,"5min":0.45600000000000002,"15min":0.40300000000000002},"max":{"1min":0.80100000000000005,"5min":0.98199999999999998,"15min":1.028},"last":1.0609999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.69199999999999995,"15min":0.72199999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40000000000000002},"max":{"1min":0.93700000000000006,"5min":0.97099999999999997,"15min":1.0589999999999999},"last":0.97699999999999998}]},{"osd":1,"last update":"Sun Oct 8 02:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68400000000000005,"15min":0.71699999999999997},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":1.0409999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.71999999999999997,"15min":0.73699999999999999},"min":{"1min":0.51600000000000001,"5min":0.496,"15min":0.49099999999999999},"max":{"1min":0.93400000000000005,"5min":1.01,"15min":1.079},"last":0.81899999999999995}]},{"osd":2,"last update":"Sun Oct 8 02:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.74099999999999999,"15min":0.77200000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.44600000000000001},"max":{"1min":0.84699999999999998,"5min":1.0309999999999999,"15min":1.034},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.76600000000000001,"15min":0.78300000000000003},"min":{"1min":0.5,"5min":0.441,"15min":0.441},"max":{"1min":0.86599999999999999,"5min":1.0629999999999999,"15min":1.097},"last":0.95399999999999996}]},{"osd":3,"last update":"Sun Oct 8 02:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.74099999999999999,"15min":0.76900000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.96199999999999997,"5min":1.0449999999999999,"15min":1.1259999999999999},"last":0.997},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77500000000000002,"15min":0.79500000000000004},"min":{"1min":0.53600000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.92800000000000005,"5min":1.0289999999999999,"15min":1.1539999999999999},"last":0.98399999999999999}]},{"osd":5,"last update":"Sun Oct 8 02:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.62,"15min":0.62},"min":{"1min":0.42999999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.90700000000000003,"5min":0.93600000000000005,"15min":1.1259999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76300000000000001,"15min":0.755},"min":{"1min":0.58899999999999997,"5min":0.502,"15min":0.48599999999999999},"max":{"1min":1.0009999999999999,"5min":1.004,"15min":1.105},"last":0.81100000000000005}]},{"osd":6,"last update":"Sun Oct 8 02:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.67100000000000004,"15min":0.65500000000000003},"min":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.437},"max":{"1min":0.94999999999999996,"5min":0.98299999999999998,"15min":1.0389999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68500000000000005,"15min":0.67900000000000005},"min":{"1min":0.55100000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":1.03},"last":0.73999999999999999}]},{"osd":7,"last update":"Sun Oct 8 02:32:13 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.71499999999999997,"15min":0.69399999999999995},"min":{"1min":0.52700000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.054,"5min":1.054,"15min":1.0900000000000001},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.75800000000000001,"15min":0.73999999999999999},"min":{"1min":0.51200000000000001,"5min":0.46999999999999997,"15min":0.42699999999999999},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.76400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":316260,"kb_used_data":2588,"kb_used_omap":0,"kb_used_meta":313664,"kb_avail":94055580,"statfs":{"total":96636764160,"available":96312913920,"internally_reserved":0,"allocated":2650112,"data_stored":1337398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":321191936},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.72499999999999998},"min":{"1min":0.46400000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1,"5min":1,"15min":1},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73399999999999999,"15min":0.746},"min":{"1min":0.44800000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98999999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.81999999999999995}]},{"osd":1,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71399999999999997,"15min":0.73599999999999999},"min":{"1min":0.435,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.91300000000000003,"5min":0.94399999999999995,"15min":0.99199999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.73299999999999998,"15min":0.74399999999999999},"min":{"1min":0.51200000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.91900000000000004,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.48599999999999999},"max":{"1min":0.94199999999999995,"5min":1.028,"15min":1.028},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.73099999999999998,"15min":0.75600000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.47099999999999997},"max":{"1min":0.93500000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.71899999999999997}]},{"osd":3,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.753,"15min":0.75700000000000001},"min":{"1min":0.503,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.90200000000000002,"5min":1.02,"15min":1.02},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.76700000000000002,"15min":0.77600000000000002},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.98299999999999998,"5min":1.0449999999999999,"15min":1.0880000000000001},"last":0.77000000000000002}]},{"osd":4,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.63100000000000001,"15min":0.61399999999999999},"min":{"1min":0.39600000000000002,"5min":0.38100000000000001,"15min":0.374},"max":{"1min":0.79100000000000004,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.65700000000000003,"15min":0.65800000000000003},"min":{"1min":0.42499999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":1.0449999999999999},"last":0.69899999999999995}]},{"osd":6,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.68100000000000005,"15min":0.67700000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0029999999999999,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.44800000000000001,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.92700000000000005,"5min":1.0149999999999999,"15min":1.1319999999999999},"last":0.78000000000000003}]},{"osd":7,"last update":"Sun Oct 8 02:31:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.70599999999999996,"15min":0.71299999999999997},"min":{"1min":0.52600000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.873,"5min":1.0429999999999999,"15min":1.044},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.74399999999999999,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.97099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.80200000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-08T02:32:28.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-08T02:32:28.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.595+0000 7f9658f9d700 1 Processor -- start 2023-10-08T02:32:28.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.595+0000 7f9658f9d700 1 -- start start 2023-10-08T02:32:28.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.596+0000 7f9658f9d700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f96541407e0 0x7f965406a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.596+0000 7f9658f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9654140bb0 0x7f965406a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.596+0000 7f9658f9d700 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96540c6820 con 0x7f965406d4f0 2023-10-08T02:32:28.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.596+0000 7f9658f9d700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96540c6de0 con 0x7f96541407e0 2023-10-08T02:32:28.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.596+0000 7f9658f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9654141270 con 0x7f9654140bb0 2023-10-08T02:32:28.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.596+0000 7f9653fff700 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f965406d4f0 0x7f965415b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:35774/0 (socket says 172.21.15.119:35774) 2023-10-08T02:32:28.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.596+0000 7f9653fff700 1 -- 172.21.15.119:0/824197245 learned_addr learned my addr 172.21.15.119:0/824197245 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-10-08T02:32:28.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f9652ffd700 1 --2- 172.21.15.119:0/824197245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9654140bb0 0x7f965406a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f96537fe700 1 --2- 172.21.15.119:0/824197245 >> v2:172.21.15.184:3300/0 conn(0x7f96541407e0 0x7f965406a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f96527fc700 1 -- 172.21.15.119:0/824197245 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926261307 0 0) 0x7f96540c6820 con 0x7f965406d4f0 2023-10-08T02:32:28.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f96527fc700 1 -- 172.21.15.119:0/824197245 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f963c003570 con 0x7f965406d4f0 2023-10-08T02:32:28.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f96537fe700 1 -- 172.21.15.119:0/824197245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9654140bb0 msgr2=0x7f965406a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:28.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f96537fe700 1 --2- 172.21.15.119:0/824197245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9654140bb0 0x7f965406a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f96537fe700 1 -- 172.21.15.119:0/824197245 >> v1:172.21.15.119:6789/0 conn(0x7f965406d4f0 legacy=0x7f965415b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f96537fe700 1 -- 172.21.15.119:0/824197245 --> v2:172.21.15.184:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f965406acf0 con 0x7f96541407e0 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.597+0000 7f96537fe700 1 --2- 172.21.15.119:0/824197245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96541407e0 0x7f965406a180 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f96440098e0 tx=0x7f96440091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da21f399ed7ba8c1 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f96527fc700 1 -- 172.21.15.119:0/824197245 <== mon.1 v2:172.21.15.184:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644011630 con 0x7f96541407e0 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f96527fc700 1 -- 172.21.15.119:0/824197245 <== mon.1 v2:172.21.15.184:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9644011790 con 0x7f96541407e0 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f96527fc700 1 -- 172.21.15.119:0/824197245 <== mon.1 v2:172.21.15.184:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644011b30 con 0x7f96541407e0 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f9658f9d700 1 -- 172.21.15.119:0/824197245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96541407e0 msgr2=0x7f965406a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f9658f9d700 1 --2- 172.21.15.119:0/824197245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96541407e0 0x7f965406a180 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f96440098e0 tx=0x7f96440091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f9658f9d700 1 -- 172.21.15.119:0/824197245 shutdown_connections 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f9658f9d700 1 --2- 172.21.15.119:0/824197245 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f96541407e0 0x7f965406a180 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f9658f9d700 1 --2- 172.21.15.119:0/824197245 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f9654140bb0 0x7f965406a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f9658f9d700 1 -- 172.21.15.119:0/824197245 >> 172.21.15.119:0/824197245 conn(0x7f96540cc1e0 msgr2=0x7f96540cc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f9658f9d700 1 -- 172.21.15.119:0/824197245 shutdown_connections 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.598+0000 7f9658f9d700 1 -- 172.21.15.119:0/824197245 wait complete. 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.600+0000 7f9658f9d700 1 Processor -- start 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.600+0000 7f9658f9d700 1 -- start start 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9658f9d700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965406d4f0 0x7f965413d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9658f9d700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96541407e0 0x7f965415e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9658f9d700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9654140bb0 0x7f96540b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9658f9d700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965416d6c0 con 0x7f965406d4f0 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9658f9d700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965416e520 con 0x7f9654140bb0 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9658f9d700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965416af50 con 0x7f96541407e0 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f96537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965406d4f0 0x7f965413d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f96537fe700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965406d4f0 0x7f965413d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41328/0 (socket says 172.21.15.119:41328) 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f96537fe700 1 -- 172.21.15.119:0/2228260289 learned_addr learned my addr 172.21.15.119:0/2228260289 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9652ffd700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96541407e0 0x7f965415e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9653fff700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9654140bb0 0x7f96540b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f96537fe700 1 -- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96541407e0 msgr2=0x7f965415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f96537fe700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96541407e0 0x7f965415e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f96537fe700 1 -- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9654140bb0 msgr2=0x7f96540b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f96537fe700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9654140bb0 0x7f96540b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f96537fe700 1 -- 172.21.15.119:0/2228260289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9644009420 con 0x7f965406d4f0 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.601+0000 7f9653fff700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9654140bb0 0x7f96540b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.602+0000 7f96537fe700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965406d4f0 0x7f965413d7a0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f9644009ed0 tx=0x7f9644017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.602+0000 7f963b7fe700 1 -- 172.21.15.119:0/2228260289 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644011900 con 0x7f965406d4f0 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.602+0000 7f963b7fe700 1 -- 172.21.15.119:0/2228260289 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9644011a60 con 0x7f965406d4f0 2023-10-08T02:32:28.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.602+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f965416d910 con 0x7f965406d4f0 2023-10-08T02:32:28.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.602+0000 7f963b7fe700 1 -- 172.21.15.119:0/2228260289 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9644023600 con 0x7f965406d4f0 2023-10-08T02:32:28.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.602+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f965416ded0 con 0x7f965406d4f0 2023-10-08T02:32:28.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.603+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f965406c450 con 0x7f965406d4f0 2023-10-08T02:32:28.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.604+0000 7f963b7fe700 1 -- 172.21.15.119:0/2228260289 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f96440239f0 con 0x7f965406d4f0 2023-10-08T02:32:28.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.605+0000 7f963b7fe700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9640066f30 0x7f96400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:28.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.605+0000 7f963b7fe700 1 -- 172.21.15.119:0/2228260289 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f9644085950 con 0x7f965406d4f0 2023-10-08T02:32:28.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.605+0000 7f9652ffd700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9640066f30 0x7f96400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:28.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.606+0000 7f9652ffd700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9640066f30 0x7f96400693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f965413f3f0 tx=0x7f96480091a0 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:28.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.609+0000 7f963b7fe700 1 -- 172.21.15.119:0/2228260289 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9644052bb0 con 0x7f965406d4f0 2023-10-08T02:32:28.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.798+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f965406afd0 con 0x7f965406d4f0 2023-10-08T02:32:28.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.803+0000 7f963b7fe700 1 -- 172.21.15.119:0/2228260289 <== mon.0 v2:172.21.15.119:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9644052bb0 con 0x7f965406d4f0 2023-10-08T02:32:28.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.803+0000 7f963b7fe700 1 -- 172.21.15.119:0/2228260289 <== mon.0 v2:172.21.15.119:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v76)=0 v76) v1 ==== 123+0+0 (secure 0 0 0) 0x7f96440150c0 con 0x7f965406d4f0 2023-10-08T02:32:28.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9640066f30 msgr2=0x7f96400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9640066f30 0x7f96400693e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f965413f3f0 tx=0x7f96480091a0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965406d4f0 msgr2=0x7f965413d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:28.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965406d4f0 0x7f965413d7a0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f9644009ed0 tx=0x7f9644017000 comp rx=0 tx=0).stop 2023-10-08T02:32:28.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 shutdown_connections 2023-10-08T02:32:28.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f9640066f30 0x7f96400693e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f9654140bb0 0x7f96540b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f965406d4f0 0x7f965413d7a0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 --2- 172.21.15.119:0/2228260289 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f96541407e0 0x7f965415e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:28.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 >> 172.21.15.119:0/2228260289 conn(0x7f96540cc1e0 msgr2=0x7f96540c1240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:28.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 shutdown_connections 2023-10-08T02:32:28.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:28.806+0000 7f9658f9d700 1 -- 172.21.15.119:0/2228260289 wait complete. 2023-10-08T02:32:28.826 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-08T02:32:28.827 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-08T02:32:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.007+0000 7f668afff700 1 Processor -- start 2023-10-08T02:32:29.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.007+0000 7f668afff700 1 -- start start 2023-10-08T02:32:29.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f668afff700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6684155c10 0x7f6684155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:29.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f668afff700 1 --2- >> v2:172.21.15.184:3300/0 conn(0x7f66840ac6d0 0x7f66840b0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:29.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f668afff700 1 -- --> v1:172.21.15.184:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66840acac0 con 0x7f66840ac230 2023-10-08T02:32:29.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f668afff700 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840ad930 con 0x7f6684155c10 2023-10-08T02:32:29.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f668afff700 1 -- --> v2:172.21.15.184:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66840acd20 con 0x7f66840ac6d0 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f6688d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6684155c10 0x7f6684155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f6688d9b700 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6684155c10 0x7f6684155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41340/0 (socket says 172.21.15.119:41340) 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f6688d9b700 1 -- 172.21.15.119:0/812638414 learned_addr learned my addr 172.21.15.119:0/812638414 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f668959c700 1 --2- 172.21.15.119:0/812638414 >> v2:172.21.15.184:3300/0 conn(0x7f66840ac6d0 0x7f66840b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f6688d9b700 1 -- 172.21.15.119:0/812638414 >> v2:172.21.15.184:3300/0 conn(0x7f66840ac6d0 msgr2=0x7f66840b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f6688d9b700 1 --2- 172.21.15.119:0/812638414 >> v2:172.21.15.184:3300/0 conn(0x7f66840ac6d0 0x7f66840b0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f6688d9b700 1 -- 172.21.15.119:0/812638414 >> v1:172.21.15.184:6789/0 conn(0x7f66840ac230 legacy=0x7f66841568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.009+0000 7f6688d9b700 1 -- 172.21.15.119:0/812638414 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66840ad000 con 0x7f6684155c10 2023-10-08T02:32:29.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.010+0000 7f6688d9b700 1 --2- 172.21.15.119:0/812638414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6684155c10 0x7f6684155fe0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f66840ae0c0 tx=0x7f6678009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec13562822b99a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.010+0000 7f66837fe700 1 -- 172.21.15.119:0/812638414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800b860 con 0x7f6684155c10 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.010+0000 7f66837fe700 1 -- 172.21.15.119:0/812638414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f667800b9c0 con 0x7f6684155c10 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.010+0000 7f66837fe700 1 -- 172.21.15.119:0/812638414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667800bd80 con 0x7f6684155c10 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.010+0000 7f668afff700 1 -- 172.21.15.119:0/812638414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6684155c10 msgr2=0x7f6684155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.010+0000 7f668afff700 1 --2- 172.21.15.119:0/812638414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6684155c10 0x7f6684155fe0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f66840ae0c0 tx=0x7f6678009250 comp rx=0 tx=0).stop 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.011+0000 7f668afff700 1 -- 172.21.15.119:0/812638414 shutdown_connections 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.011+0000 7f668afff700 1 --2- 172.21.15.119:0/812638414 >> v2:172.21.15.184:3300/0 conn(0x7f66840ac6d0 0x7f66840b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.011+0000 7f668afff700 1 --2- 172.21.15.119:0/812638414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6684155c10 0x7f6684155fe0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.011+0000 7f668afff700 1 -- 172.21.15.119:0/812638414 >> 172.21.15.119:0/812638414 conn(0x7f668406a320 msgr2=0x7f66840cc530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:29.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.011+0000 7f668afff700 1 -- 172.21.15.119:0/812638414 shutdown_connections 2023-10-08T02:32:29.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.011+0000 7f668afff700 1 -- 172.21.15.119:0/812638414 wait complete. 2023-10-08T02:32:29.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.012+0000 7f668afff700 1 Processor -- start 2023-10-08T02:32:29.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.012+0000 7f668afff700 1 -- start start 2023-10-08T02:32:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f668afff700 1 --2- >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66840ac230 0x7f668413a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f668afff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66840ac6d0 0x7f668414fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f668afff700 1 --2- >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6684155c10 0x7f6684150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f668afff700 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668414e6e0 con 0x7f66840ac6d0 2023-10-08T02:32:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f668afff700 1 -- --> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6684152fb0 con 0x7f6684155c10 2023-10-08T02:32:29.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f668afff700 1 -- --> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6684153290 con 0x7f66840ac230 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f6683fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66840ac6d0 0x7f668414fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f6683fff700 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66840ac6d0 0x7f668414fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41346/0 (socket says 172.21.15.119:41346) 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f6683fff700 1 -- 172.21.15.119:0/128969910 learned_addr learned my addr 172.21.15.119:0/128969910 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f6688d9b700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66840ac230 0x7f668413a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.013+0000 7f668959c700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6684155c10 0x7f6684150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6683fff700 1 -- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66840ac230 msgr2=0x7f668413a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6683fff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66840ac230 0x7f668413a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6683fff700 1 -- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6684155c10 msgr2=0x7f6684150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6683fff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6684155c10 0x7f6684150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6683fff700 1 -- 172.21.15.119:0/128969910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66780094f0 con 0x7f66840ac6d0 2023-10-08T02:32:29.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f668959c700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6684155c10 0x7f6684150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-08T02:32:29.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6683fff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66840ac6d0 0x7f668414fe40 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f667000c4e0 tx=0x7f667000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:29.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6680ff9700 1 -- 172.21.15.119:0/128969910 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670011590 con 0x7f66840ac6d0 2023-10-08T02:32:29.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6680ff9700 1 -- 172.21.15.119:0/128969910 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6670011710 con 0x7f66840ac6d0 2023-10-08T02:32:29.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6684153570 con 0x7f66840ac6d0 2023-10-08T02:32:29.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f6680ff9700 1 -- 172.21.15.119:0/128969910 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670011af0 con 0x7f66840ac6d0 2023-10-08T02:32:29.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.014+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6684153b30 con 0x7f66840ac6d0 2023-10-08T02:32:29.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.015+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668404c3c0 con 0x7f66840ac6d0 2023-10-08T02:32:29.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.016+0000 7f6680ff9700 1 -- 172.21.15.119:0/128969910 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 121) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f667001b3b0 con 0x7f66840ac6d0 2023-10-08T02:32:29.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.017+0000 7f6680ff9700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6668066f30 0x7f66680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-08T02:32:29.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.017+0000 7f6680ff9700 1 -- 172.21.15.119:0/128969910 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6018+0+0 (secure 0 0 0) 0x7f6670083bb0 con 0x7f66840ac6d0 2023-10-08T02:32:29.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.017+0000 7f6688d9b700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6668066f30 0x7f66680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-08T02:32:29.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.018+0000 7f6688d9b700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6668066f30 0x7f66680693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66840ae0c0 tx=0x7f6678009250 comp rx=0 tx=0).ready entity=mgr.10494 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-08T02:32:29.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.021+0000 7f6680ff9700 1 -- 172.21.15.119:0/128969910 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6670050cc0 con 0x7f66840ac6d0 2023-10-08T02:32:29.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.258+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66840ad2e0 con 0x7f66840ac6d0 2023-10-08T02:32:29.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.259+0000 7f6680ff9700 1 -- 172.21.15.119:0/128969910 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+3543 (secure 0 0 0) 0x7f6670017070 con 0x7f66840ac6d0 2023-10-08T02:32:29.259 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-10-08T02:32:29.260 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":97,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8135,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6835/1044912493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6833","nonce":1044912493},{"type":"v1","addr":"172.21.15.184:6835","nonce":1044912493}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":9712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/1989876473","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":1989876473},{"type":"v1","addr":"172.21.15.119:6835","nonce":1989876473}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":97},{"gid":10197,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6833/1564057059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":1564057059},{"type":"v1","addr":"172.21.15.119:6833","nonce":1564057059}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10203,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.184:6834/2545901849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.184:6832","nonce":2545901849},{"type":"v1","addr":"172.21.15.184:6834","nonce":2545901849}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[]} 2023-10-08T02:32:29.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6668066f30 msgr2=0x7f66680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:29.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6668066f30 0x7f66680693e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f66840ae0c0 tx=0x7f6678009250 comp rx=0 tx=0).stop 2023-10-08T02:32:29.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66840ac6d0 msgr2=0x7f668414fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66840ac6d0 0x7f668414fe40 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7f667000c4e0 tx=0x7f667000c7f0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 shutdown_connections 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:6836/134108,v1:172.21.15.119:6837/134108] conn(0x7f6668066f30 0x7f66680693e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3301/0,v1:172.21.15.184:6790/0] conn(0x7f66840ac230 0x7f668413a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.184:3300/0,v1:172.21.15.184:6789/0] conn(0x7f6684155c10 0x7f6684150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 --2- 172.21.15.119:0/128969910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66840ac6d0 0x7f668414fe40 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 >> 172.21.15.119:0/128969910 conn(0x7f668406a320 msgr2=0x7f668406ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.261+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 shutdown_connections 2023-10-08T02:32:29.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-10-08T02:32:29.262+0000 7f668afff700 1 -- 172.21.15.119:0/128969910 wait complete. 2023-10-08T02:32:29.264 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 97 2023-10-08T02:32:29.279 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-08T02:32:29.280 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-08T02:32:29.280 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-08T02:32:29.280 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.293 INFO:tasks.ceph.mds.a:Stopped 2023-10-08T02:32:29.293 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-08T02:32:29.293 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.346 INFO:tasks.ceph.mds.c:Stopped 2023-10-08T02:32:29.347 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-08T02:32:29.347 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.362 INFO:tasks.ceph.mds.b:Stopped 2023-10-08T02:32:29.362 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-08T02:32:29.362 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.418 INFO:tasks.ceph.mds.d:Stopped 2023-10-08T02:32:29.418 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-08T02:32:29.418 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-08T02:32:29.419 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.468 INFO:tasks.ceph.osd.0:Stopped 2023-10-08T02:32:29.468 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-08T02:32:29.468 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.528 INFO:tasks.ceph.osd.1:Stopped 2023-10-08T02:32:29.529 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-08T02:32:29.529 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.583 INFO:tasks.ceph.osd.2:Stopped 2023-10-08T02:32:29.584 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-08T02:32:29.584 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.640 INFO:tasks.ceph.osd.3:Stopped 2023-10-08T02:32:29.641 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-08T02:32:29.641 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.690 INFO:tasks.ceph.osd.4:Stopped 2023-10-08T02:32:29.690 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-08T02:32:29.690 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.744 INFO:tasks.ceph.osd.5:Stopped 2023-10-08T02:32:29.744 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-08T02:32:29.744 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.796 INFO:tasks.ceph.osd.6:Stopped 2023-10-08T02:32:29.797 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-08T02:32:29.797 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.846 INFO:tasks.ceph.osd.7:Stopped 2023-10-08T02:32:29.847 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-08T02:32:29.847 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-08T02:32:29.847 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.914 INFO:tasks.ceph.mgr.y:Stopped 2023-10-08T02:32:29.914 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-08T02:32:29.914 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:29.976 INFO:tasks.ceph.mgr.x:Stopped 2023-10-08T02:32:29.976 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-08T02:32:29.977 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-08T02:32:29.977 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:30.008 INFO:tasks.ceph.mon.a:Stopped 2023-10-08T02:32:30.009 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-08T02:32:30.009 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:30.033 INFO:tasks.ceph.mon.b:Stopped 2023-10-08T02:32:30.033 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-08T02:32:30.033 INFO:teuthology.orchestra.run:waiting for 300 2023-10-08T02:32:30.097 INFO:tasks.ceph.mon.c:Stopped 2023-10-08T02:32:30.097 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-08T02:32:30.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-08T02:32:30.169 INFO:teuthology.orchestra.run.smithi119.stdout:1696731227.3553238 mon.a (mon.0) 1800 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-10-08T02:32:30.170 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-08T02:32:30.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-08T02:32:30.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-08T02:32:30.315 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-08T02:32:30.385 INFO:teuthology.orchestra.run.smithi119.stdout:1696731227.3553238 mon.a (mon.0) 1800 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-10-08T02:32:30.387 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:32:30.387 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-08T02:32:30.790 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:32:30.792 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-08T02:32:31.001 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:32:31.002 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-08T02:32:31.215 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:32:31.216 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-08T02:32:31.420 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T02:32:31.421 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-08T02:32:31.892 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T02:32:31.892 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-08T02:32:32.089 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T02:32:32.089 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-08T02:32:32.316 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T02:32:32.316 DEBUG:teuthology.orchestra.run.smithi184:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-08T02:32:32.506 INFO:tasks.ceph:Archiving mon data... 2023-10-08T02:32:32.507 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/data/mon.a.tgz 2023-10-08T02:32:32.507 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-10-08T02:32:32.547 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.QEYEV4qxoD 2023-10-08T02:32:32.548 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.QEYEV4qxoD 2023-10-08T02:32:33.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.QEYEV4qxoD 2023-10-08T02:32:33.156 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.QEYEV4qxoD is 3MB 2023-10-08T02:32:33.283 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.QEYEV4qxoD 2023-10-08T02:32:33.320 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/data/mon.b.tgz 2023-10-08T02:32:33.320 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-10-08T02:32:33.361 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.K9bTbVtfvj 2023-10-08T02:32:33.361 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.K9bTbVtfvj 2023-10-08T02:32:33.948 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.K9bTbVtfvj 2023-10-08T02:32:34.045 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.K9bTbVtfvj is 3MB 2023-10-08T02:32:34.182 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.K9bTbVtfvj 2023-10-08T02:32:34.218 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/data/mon.c.tgz 2023-10-08T02:32:34.219 DEBUG:teuthology.orchestra.run.smithi184:> mktemp 2023-10-08T02:32:34.294 INFO:teuthology.orchestra.run.smithi184.stdout:/tmp/tmp.mTc7C5AoKd 2023-10-08T02:32:34.294 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.mTc7C5AoKd 2023-10-08T02:32:34.811 DEBUG:teuthology.orchestra.run.smithi184:> sudo chmod 0666 /tmp/tmp.mTc7C5AoKd 2023-10-08T02:32:34.912 DEBUG:teuthology.orchestra.remote:smithi184:/tmp/tmp.mTc7C5AoKd is 3MB 2023-10-08T02:32:35.041 DEBUG:teuthology.orchestra.run.smithi184:> rm -fr /tmp/tmp.mTc7C5AoKd 2023-10-08T02:32:35.077 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-08T02:32:35.077 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T02:32:35.082 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-08T02:32:35.177 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-08T02:32:35.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T02:32:35.182 DEBUG:teuthology.orchestra.run.smithi184:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-08T02:32:35.250 INFO:teuthology.orchestra.run.smithi184.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T02:32:35.261 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-08T02:32:35.277 INFO:tasks.ceph:Archiving crash dumps... 2023-10-08T02:32:35.279 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/remote/smithi119/crash 2023-10-08T02:32:35.280 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T02:32:35.341 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/remote/smithi184/crash 2023-10-08T02:32:35.342 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-08T02:32:35.401 INFO:tasks.ceph:Shutting down logrotate 2023-10-08T02:32:35.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:32:35.405 DEBUG:teuthology.orchestra.run.smithi184:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-08T02:32:35.501 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T02:32:35.561 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-08T02:32:35.620 INFO:tasks.ceph:Compressing logs... 2023-10-08T02:32:35.621 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T02:32:35.625 DEBUG:teuthology.orchestra.run.smithi184:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-08T02:32:35.681 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131446.log 2023-10-08T02:32:35.682 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136739.log 2023-10-08T02:32:35.683 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131446.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131446.log.gz 2023-10-08T02:32:35.683 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122111.log 2023-10-08T02:32:35.684 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136739.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.136739.log.gz 2023-10-08T02:32:35.684 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112828.log 2023-10-08T02:32:35.684 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85687.log 2023-10-08T02:32:35.685 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122111.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122111.log.gz 2023-10-08T02:32:35.685 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134672.log 2023-10-08T02:32:35.686 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85775.log 2023-10-08T02:32:35.686 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112828.log: gzip 88.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.112828.log.gz-- 2023-10-08T02:32:35.686 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93371.log 2023-10-08T02:32:35.688 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.85687.log: gzip 87.2%-5 -- replaced with /var/log/ceph/ceph-client.admin.85687.log.gz--verbose 2023-10-08T02:32:35.688 INFO:teuthology.orchestra.run.smithi184.stderr:-- /var/log/ceph/ceph-client.admin.85900.log 2023-10-08T02:32:35.689 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.85775.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85775.log.gz 2023-10-08T02:32:35.689 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85817.log 2023-10-08T02:32:35.689 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.85900.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.85900.log.gz 2023-10-08T02:32:35.689 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-08T02:32:35.689 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.134672.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.134672.log.gz 2023-10-08T02:32:35.690 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94800.log 2023-10-08T02:32:35.690 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93371.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93371.log.gz 2023-10-08T02:32:35.690 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138012.log 2023-10-08T02:32:35.690 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94800.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94800.log.gz 2023-10-08T02:32:35.690 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114414.log 2023-10-08T02:32:35.692 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.138012.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138012.log.gz 2023-10-08T02:32:35.692 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85786.log 2023-10-08T02:32:35.692 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114414.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114414.log.gz 2023-10-08T02:32:35.692 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115815.log 2023-10-08T02:32:35.693 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.85786.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.85786.log.gz 2023-10-08T02:32:35.693 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141703.log 2023-10-08T02:32:35.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.85817.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.85817.log.gz 2023-10-08T02:32:35.693 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.92031.log 2023-10-08T02:32:35.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-08T02:32:35.693 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.1.92031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.92031.log.gz 2023-10-08T02:32:35.694 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85644.log 2023-10-08T02:32:35.694 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T02:32:35.694 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.85644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85603.log 2023-10-08T02:32:35.694 INFO:teuthology.orchestra.run.smithi184.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.85644.log.gz 2023-10-08T02:32:35.695 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85730.log 2023-10-08T02:32:35.695 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.85603.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85603.log.gz 2023-10-08T02:32:35.697 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140210.log 2023-10-08T02:32:35.697 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115815.log.gz 2023-10-08T02:32:35.697 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128791.log 2023-10-08T02:32:35.697 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141703.log.gz 2023-10-08T02:32:35.697 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140210.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140210.log.gz 2023-10-08T02:32:35.697 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144421.log 2023-10-08T02:32:35.697 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128791.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128791.log.gz 2023-10-08T02:32:35.698 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92708.log 2023-10-08T02:32:35.698 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144421.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144421.log.gz 2023-10-08T02:32:35.698 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115510.log 2023-10-08T02:32:35.700 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-08T02:32:35.701 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.85730.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85730.log.gz 2023-10-08T02:32:35.701 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-08T02:32:35.701 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-08T02:32:35.701 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T02:32:35.701 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-08T02:32:35.702 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92708.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.92708.log.gz 2023-10-08T02:32:35.702 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111073.log 2023-10-08T02:32:35.702 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115510.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115510.log.gz 2023-10-08T02:32:35.702 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91850.log 2023-10-08T02:32:35.702 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111073.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111073.log.gz 2023-10-08T02:32:35.702 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129517.log 2023-10-08T02:32:35.702 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91850.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91850.log.gz 2023-10-08T02:32:35.702 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111994.log 2023-10-08T02:32:35.704 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.4.log: 93.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T02:32:35.704 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85859.log 2023-10-08T02:32:35.704 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-08T02:32:35.705 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141335.log 2023-10-08T02:32:35.705 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.129517.log.gz 2023-10-08T02:32:35.705 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111994.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111994.log.gz 2023-10-08T02:32:35.705 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132186.log 2023-10-08T02:32:35.705 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141335.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.141335.log.gz 2023-10-08T02:32:35.706 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113973.log 2023-10-08T02:32:35.706 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132186.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132186.log.gz 2023-10-08T02:32:35.706 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135737.log 2023-10-08T02:32:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113973.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.113973.log.gz 2023-10-08T02:32:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115944.log 2023-10-08T02:32:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135737.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135737.log.gz 2023-10-08T02:32:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114629.log 2023-10-08T02:32:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115944.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115944.log.gz 2023-10-08T02:32:35.709 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.95159.log 2023-10-08T02:32:35.710 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.admin.85859.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85859.log.gz 2023-10-08T02:32:35.710 INFO:teuthology.orchestra.run.smithi184.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-08T02:32:35.710 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-08T02:32:35.710 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.92417.log 2023-10-08T02:32:35.711 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114629.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114629.log.gz 2023-10-08T02:32:35.711 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128665.log 2023-10-08T02:32:35.711 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.2.95159.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.95159.log.gz 2023-10-08T02:32:35.712 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129173.log 2023-10-08T02:32:35.712 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128665.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128665.log.gz 2023-10-08T02:32:35.712 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100503.log 2023-10-08T02:32:35.712 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129173.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129173.log.gz 2023-10-08T02:32:35.712 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119917.log 2023-10-08T02:32:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100503.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100503.log.gz 2023-10-08T02:32:35.713 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111124.log 2023-10-08T02:32:35.714 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119917.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119917.log.gz 2023-10-08T02:32:35.714 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108338.log 2023-10-08T02:32:35.715 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111124.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111124.log.gz 2023-10-08T02:32:35.715 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140905.log 2023-10-08T02:32:35.716 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108338.log: gzip -5 --verbose 88.8%-- -- replaced with /var/log/ceph/ceph-client.admin.108338.log.gz/var/log/ceph/ceph-client.admin.104482.log 2023-10-08T02:32:35.716 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-10-08T02:32:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140905.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140905.log.gz 2023-10-08T02:32:35.717 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101783.log 2023-10-08T02:32:35.718 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104482.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104482.log.gz 2023-10-08T02:32:35.718 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133340.log 2023-10-08T02:32:35.719 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104104.log 2023-10-08T02:32:35.719 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.101783.log.gz 2023-10-08T02:32:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.133340.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133340.log.gz 2023-10-08T02:32:35.720 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103888.log 2023-10-08T02:32:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104104.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104104.log.gz 2023-10-08T02:32:35.721 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132882.log 2023-10-08T02:32:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103888.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.103888.log.gz 2023-10-08T02:32:35.722 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96231.log 2023-10-08T02:32:35.723 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-08T02:32:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132882.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.132882.log.gz 2023-10-08T02:32:35.723 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121202.log 2023-10-08T02:32:35.724 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-client.3.92417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.92417.log.gz 2023-10-08T02:32:35.724 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96231.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96231.log.gz 2023-10-08T02:32:35.725 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129391.log 2023-10-08T02:32:35.725 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121202.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121202.log.gz 2023-10-08T02:32:35.725 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144841.log 2023-10-08T02:32:35.726 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129391.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129391.log.gz 2023-10-08T02:32:35.726 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137292.log 2023-10-08T02:32:35.727 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144841.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144841.log.gz 2023-10-08T02:32:35.727 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95386.log 2023-10-08T02:32:35.728 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137292.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.137292.log.gz 2023-10-08T02:32:35.728 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119279.log 2023-10-08T02:32:35.729 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95386.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95386.log.gz 2023-10-08T02:32:35.729 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141565.log 2023-10-08T02:32:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119279.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119279.log.gz 2023-10-08T02:32:35.730 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122296.log 2023-10-08T02:32:35.731 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141565.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141565.log.gz 2023-10-08T02:32:35.731 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141609.log 2023-10-08T02:32:35.732 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122296.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122296.log.gz 2023-10-08T02:32:35.732 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131194.log 2023-10-08T02:32:35.733 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109239.log 2023-10-08T02:32:35.733 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.141609.log.gz 2023-10-08T02:32:35.734 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131194.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131194.log.gz 2023-10-08T02:32:35.734 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128879.log 2023-10-08T02:32:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109239.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109239.log.gz 2023-10-08T02:32:35.735 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131586.log 2023-10-08T02:32:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128879.log: gzip 88.7%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.128879.log.gz-- 2023-10-08T02:32:35.736 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137226.log 2023-10-08T02:32:35.737 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131586.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.131586.log.gz 2023-10-08T02:32:35.737 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135925.log 2023-10-08T02:32:35.738 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100545.log 2023-10-08T02:32:35.738 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.137226.log.gz 2023-10-08T02:32:35.739 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135925.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.135925.log.gz 2023-10-08T02:32:35.739 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100977.log 2023-10-08T02:32:35.740 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100545.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100545.log.gz 2023-10-08T02:32:35.740 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106487.log 2023-10-08T02:32:35.741 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100977.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100977.log.gz 2023-10-08T02:32:35.741 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115383.log 2023-10-08T02:32:35.742 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135967.log 2023-10-08T02:32:35.742 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106487.log.gz 2023-10-08T02:32:35.743 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93668.log 2023-10-08T02:32:35.743 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115383.log.gz 2023-10-08T02:32:35.744 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135967.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.135967.log.gz 2023-10-08T02:32:35.744 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116174.log 2023-10-08T02:32:35.745 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123802.log 2023-10-08T02:32:35.745 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93668.log.gz 2023-10-08T02:32:35.746 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119237.log 2023-10-08T02:32:35.746 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116174.log.gz 2023-10-08T02:32:35.747 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123802.log: 88.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.123802.log.gz--verbose 2023-10-08T02:32:35.747 INFO:teuthology.orchestra.run.smithi119.stderr:-- /var/log/ceph/ceph-client.admin.104793.log 2023-10-08T02:32:35.747 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119237.log: gzip 89.2%-5 -- replaced with /var/log/ceph/ceph-client.admin.119237.log.gz--verbose 2023-10-08T02:32:35.748 INFO:teuthology.orchestra.run.smithi119.stderr:-- /var/log/ceph/ceph-client.admin.145051.log 2023-10-08T02:32:35.749 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104793.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104793.log.gz 2023-10-08T02:32:35.749 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94748.log 2023-10-08T02:32:35.749 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.145051.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145051.log.gz 2023-10-08T02:32:35.750 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141915.log 2023-10-08T02:32:35.750 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.94748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.94748.log.gz 2023-10-08T02:32:35.751 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135883.log 2023-10-08T02:32:35.752 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141915.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.141915.log.gz 2023-10-08T02:32:35.752 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113210.log 2023-10-08T02:32:35.753 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135883.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135883.log.gz 2023-10-08T02:32:35.753 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122552.log 2023-10-08T02:32:35.754 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113210.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113210.log.gz 2023-10-08T02:32:35.754 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93378.log 2023-10-08T02:32:35.755 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122552.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122552.log.gz 2023-10-08T02:32:35.755 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130778.log 2023-10-08T02:32:35.756 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93378.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93378.log.gz 2023-10-08T02:32:35.756 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111185.log 2023-10-08T02:32:35.757 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105393.log 2023-10-08T02:32:35.757 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130778.log.gz 2023-10-08T02:32:35.758 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111185.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111185.log.gz 2023-10-08T02:32:35.758 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109323.log 2023-10-08T02:32:35.759 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105393.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105393.log.gz 2023-10-08T02:32:35.759 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143167.log 2023-10-08T02:32:35.760 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109323.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109323.log.gz 2023-10-08T02:32:35.760 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114190.log 2023-10-08T02:32:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143167.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143167.log.gz 2023-10-08T02:32:35.761 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-08T02:32:35.762 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114190.log: 88.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.114190.log.gz--verbose -- 2023-10-08T02:32:35.762 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136051.log 2023-10-08T02:32:35.763 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97147.log 2023-10-08T02:32:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136051.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136051.log.gz 2023-10-08T02:32:35.764 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116080.log 2023-10-08T02:32:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97147.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97147.log.gz 2023-10-08T02:32:35.765 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110290.log 2023-10-08T02:32:35.766 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116080.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116080.log.gz 2023-10-08T02:32:35.766 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104150.log 2023-10-08T02:32:35.767 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110290.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110290.log.gz 2023-10-08T02:32:35.767 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96567.log 2023-10-08T02:32:35.768 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104150.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104150.log.gz 2023-10-08T02:32:35.768 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127759.log 2023-10-08T02:32:35.769 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96567.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96567.log.gz 2023-10-08T02:32:35.769 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122426.log 2023-10-08T02:32:35.770 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97581.log 2023-10-08T02:32:35.770 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127759.log.gz 2023-10-08T02:32:35.771 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127715.log 2023-10-08T02:32:35.771 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122426.log.gz 2023-10-08T02:32:35.772 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97581.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97581.log.gz 2023-10-08T02:32:35.772 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129475.log 2023-10-08T02:32:35.773 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91482.log 2023-10-08T02:32:35.773 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127715.log.gz 2023-10-08T02:32:35.773 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129475.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129475.log.gz 2023-10-08T02:32:35.774 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131794.log 2023-10-08T02:32:35.775 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91482.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91482.log.gz 2023-10-08T02:32:35.775 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92324.log 2023-10-08T02:32:35.776 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131794.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131794.log.gz 2023-10-08T02:32:35.776 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95429.log 2023-10-08T02:32:35.777 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137969.log 2023-10-08T02:32:35.777 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92324.log.gz 2023-10-08T02:32:35.777 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95429.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95429.log.gz 2023-10-08T02:32:35.778 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121062.log 2023-10-08T02:32:35.779 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115858.log 2023-10-08T02:32:35.779 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137969.log.gz 2023-10-08T02:32:35.780 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121062.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121062.log.gz 2023-10-08T02:32:35.780 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141745.log 2023-10-08T02:32:35.781 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115858.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115858.log.gz 2023-10-08T02:32:35.781 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110542.log 2023-10-08T02:32:35.781 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110248.log 2023-10-08T02:32:35.782 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.141745.log.gz 2023-10-08T02:32:35.783 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110542.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.110542.log.gz 2023-10-08T02:32:35.783 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117350.log 2023-10-08T02:32:35.784 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110248.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110248.log.gz 2023-10-08T02:32:35.784 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119959.log 2023-10-08T02:32:35.785 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142083.log 2023-10-08T02:32:35.785 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117350.log.gz 2023-10-08T02:32:35.786 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119959.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119959.log.gz 2023-10-08T02:32:35.786 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97924.log 2023-10-08T02:32:35.787 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142083.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142083.log.gz 2023-10-08T02:32:35.787 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112998.log 2023-10-08T02:32:35.789 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97924.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97924.log.gz 2023-10-08T02:32:35.789 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95555.log 2023-10-08T02:32:35.789 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112998.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112998.log.gz 2023-10-08T02:32:35.789 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96894.log 2023-10-08T02:32:35.790 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95555.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95555.log.gz 2023-10-08T02:32:35.790 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145093.log 2023-10-08T02:32:35.791 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96894.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96894.log.gz 2023-10-08T02:32:35.792 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110458.log 2023-10-08T02:32:35.792 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.145093.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145093.log.gz 2023-10-08T02:32:35.792 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131320.log 2023-10-08T02:32:35.793 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112564.log 2023-10-08T02:32:35.793 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110458.log.gz 2023-10-08T02:32:35.794 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131320.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.131320.log.gz 2023-10-08T02:32:35.797 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92492.log 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112564.log: gzip 88.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.112564.log.gz 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:--verbose -- /var/log/ceph/ceph-client.admin.92240.log 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92492.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92492.log.gz 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143551.log 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92240.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92240.log.gz 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97495.log 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143551.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143551.log.gz 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114675.log 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97495.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.97495.log.gz 2023-10-08T02:32:35.800 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129922.log 2023-10-08T02:32:35.801 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114675.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114675.log.gz 2023-10-08T02:32:35.801 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111595.log 2023-10-08T02:32:35.802 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129922.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129922.log.gz 2023-10-08T02:32:35.802 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140441.log 2023-10-08T02:32:35.803 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111595.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.111595.log.gz-5 2023-10-08T02:32:35.803 INFO:teuthology.orchestra.run.smithi119.stderr:--verbose -- /var/log/ceph/ceph-client.admin.113672.log 2023-10-08T02:32:35.803 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140441.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140441.log.gz 2023-10-08T02:32:35.803 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122888.log 2023-10-08T02:32:35.804 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122340.log 2023-10-08T02:32:35.804 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.113672.log.gz 2023-10-08T02:32:35.805 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122888.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122888.log.gz 2023-10-08T02:32:35.805 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109984.log 2023-10-08T02:32:35.806 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122340.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122340.log.gz 2023-10-08T02:32:35.806 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95597.log 2023-10-08T02:32:35.807 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109984.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109984.log.gz 2023-10-08T02:32:35.807 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121300.log 2023-10-08T02:32:35.808 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95597.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95597.log.gz 2023-10-08T02:32:35.809 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92706.log 2023-10-08T02:32:35.809 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121300.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121300.log.gz 2023-10-08T02:32:35.810 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111742.log 2023-10-08T02:32:35.810 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118438.log 2023-10-08T02:32:35.811 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92706.log.gz 2023-10-08T02:32:35.811 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108706.log 2023-10-08T02:32:35.812 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111742.log.gz 2023-10-08T02:32:35.812 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118438.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118438.log.gz 2023-10-08T02:32:35.812 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104300.log 2023-10-08T02:32:35.813 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108706.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108706.log.gz 2023-10-08T02:32:35.814 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116786.log 2023-10-08T02:32:35.814 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104300.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104300.log.gz 2023-10-08T02:32:35.815 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114768.log 2023-10-08T02:32:35.816 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116786.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120025.log 2023-10-08T02:32:35.816 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116786.log.gz 2023-10-08T02:32:35.816 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114768.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114768.log.gz 2023-10-08T02:32:35.817 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133115.log 2023-10-08T02:32:35.817 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120025.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120025.log.gz 2023-10-08T02:32:35.818 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135357.log 2023-10-08T02:32:35.818 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.133115.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.133115.log.gz 2023-10-08T02:32:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93377.log 2023-10-08T02:32:35.819 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99253.log 2023-10-08T02:32:35.819 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135357.log.gz 2023-10-08T02:32:35.820 INFO:teuthology.orchestra.run.smithi184.stderr:/var/log/ceph/ceph-mon.b.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-08T02:32:35.821 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93377.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93377.log.gz 2023-10-08T02:32:35.821 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97766.log 2023-10-08T02:32:35.821 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99253.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99253.log.gzgzip 2023-10-08T02:32:35.822 INFO:teuthology.orchestra.run.smithi119.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.135841.log 2023-10-08T02:32:35.822 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97766.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97766.log.gz 2023-10-08T02:32:35.823 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127624.log 2023-10-08T02:32:35.823 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96189.log 2023-10-08T02:32:35.823 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135841.log.gz 2023-10-08T02:32:35.824 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127624.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127624.log.gz 2023-10-08T02:32:35.824 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97105.log 2023-10-08T02:32:35.825 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96189.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96189.log.gz 2023-10-08T02:32:35.825 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121155.log 2023-10-08T02:32:35.826 INFO:teuthology.orchestra.run.smithi184.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-08T02:32:35.827 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97105.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97105.log.gz 2023-10-08T02:32:35.827 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142970.log 2023-10-08T02:32:35.827 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121155.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121155.log.gz 2023-10-08T02:32:35.827 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122930.log 2023-10-08T02:32:35.828 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142970.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142970.log.gz 2023-10-08T02:32:35.828 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97411.log 2023-10-08T02:32:35.829 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121721.log 2023-10-08T02:32:35.829 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122930.log.gz 2023-10-08T02:32:35.830 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97411.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97411.log.gz 2023-10-08T02:32:35.830 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132974.log 2023-10-08T02:32:35.831 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121721.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121721.log.gz 2023-10-08T02:32:35.831 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114081.log 2023-10-08T02:32:35.832 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132974.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132974.log.gz 2023-10-08T02:32:35.832 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112312.log 2023-10-08T02:32:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140531.log 2023-10-08T02:32:35.833 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114081.log.gz 2023-10-08T02:32:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112312.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112312.log.gz 2023-10-08T02:32:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111548.log 2023-10-08T02:32:35.835 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117935.log 2023-10-08T02:32:35.835 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140531.log.gz 2023-10-08T02:32:35.836 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111548.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111548.log.gz 2023-10-08T02:32:35.836 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144014.log 2023-10-08T02:32:35.837 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132354.log 2023-10-08T02:32:35.837 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117935.log.gz 2023-10-08T02:32:35.838 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144014.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144014.log.gz 2023-10-08T02:32:35.838 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109097.log 2023-10-08T02:32:35.839 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116233.log 2023-10-08T02:32:35.839 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132354.log.gz 2023-10-08T02:32:35.840 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109097.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109097.log.gz 2023-10-08T02:32:35.840 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121016.log 2023-10-08T02:32:35.841 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116233.log: gzip -5 --verbose 89.3% -- replaced with /var/log/ceph/ceph-client.admin.116233.log.gz 2023-10-08T02:32:35.841 INFO:teuthology.orchestra.run.smithi119.stderr:-- /var/log/ceph/ceph-client.admin.144883.log 2023-10-08T02:32:35.842 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121016.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121016.log.gz 2023-10-08T02:32:35.842 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93962.log 2023-10-08T02:32:35.843 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144883.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144883.log.gz 2023-10-08T02:32:35.843 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105531.log 2023-10-08T02:32:35.844 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93962.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93962.log.gz 2023-10-08T02:32:35.844 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115650.log 2023-10-08T02:32:35.845 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119363.log 2023-10-08T02:32:35.845 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105531.log.gz 2023-10-08T02:32:35.846 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115650.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115650.log.gz 2023-10-08T02:32:35.846 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114913.log 2023-10-08T02:32:35.847 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119363.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119363.log.gz 2023-10-08T02:32:35.847 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109407.log 2023-10-08T02:32:35.848 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114913.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114913.log.gz 2023-10-08T02:32:35.848 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128963.log 2023-10-08T02:32:35.849 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109407.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109407.log.gz 2023-10-08T02:32:35.849 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117392.log 2023-10-08T02:32:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128963.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.128963.log.gz 2023-10-08T02:32:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91976.log 2023-10-08T02:32:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117392.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117392.log.gz 2023-10-08T02:32:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143496.log 2023-10-08T02:32:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91976.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.91976.log.gz 2023-10-08T02:32:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129005.log 2023-10-08T02:32:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143496.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143496.log.gz 2023-10-08T02:32:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94130.log 2023-10-08T02:32:35.854 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129005.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129005.log.gz 2023-10-08T02:32:35.854 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121637.log 2023-10-08T02:32:35.855 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94130.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94130.log.gz 2023-10-08T02:32:35.855 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141873.log 2023-10-08T02:32:35.856 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121637.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121637.log.gz 2023-10-08T02:32:35.856 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108754.log 2023-10-08T02:32:35.856 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141873.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141873.log.gz 2023-10-08T02:32:35.857 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97021.log 2023-10-08T02:32:35.857 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108754.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108754.log.gz 2023-10-08T02:32:35.858 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93752.log 2023-10-08T02:32:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97021.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97021.log.gz 2023-10-08T02:32:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136483.log 2023-10-08T02:32:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93752.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93752.log.gz 2023-10-08T02:32:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121951.log 2023-10-08T02:32:35.861 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136483.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136483.log.gz 2023-10-08T02:32:35.861 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119447.log 2023-10-08T02:32:35.862 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121951.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121951.log.gz 2023-10-08T02:32:35.862 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121679.log 2023-10-08T02:32:35.863 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119447.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119447.log.gz 2023-10-08T02:32:35.863 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119489.log 2023-10-08T02:32:35.864 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121679.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121679.log.gz 2023-10-08T02:32:35.864 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-08T02:32:35.865 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119489.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.119489.log.gz 2023-10-08T02:32:35.865 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115468.log 2023-10-08T02:32:35.866 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.123274.log 2023-10-08T02:32:35.867 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115468.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115468.log.gz 2023-10-08T02:32:35.867 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91667.log 2023-10-08T02:32:35.868 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.123274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.123274.log.gz 2023-10-08T02:32:35.868 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91892.log 2023-10-08T02:32:35.869 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91667.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91667.log.gz 2023-10-08T02:32:35.870 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110332.log 2023-10-08T02:32:35.871 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91892.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.91892.log.gz 2023-10-08T02:32:35.871 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119681.log 2023-10-08T02:32:35.872 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110332.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110332.log.gz 2023-10-08T02:32:35.873 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136093.log 2023-10-08T02:32:35.873 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-08T02:32:35.873 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119681.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119681.log.gz 2023-10-08T02:32:35.874 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97816.log 2023-10-08T02:32:35.874 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136093.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136093.log.gz 2023-10-08T02:32:35.875 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108846.log 2023-10-08T02:32:35.875 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97816.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97816.log.gz 2023-10-08T02:32:35.875 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136697.log 2023-10-08T02:32:35.876 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108846.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108846.log.gz 2023-10-08T02:32:35.876 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116130.log 2023-10-08T02:32:35.877 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136697.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.136697.log.gz 2023-10-08T02:32:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111425.log 2023-10-08T02:32:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116130.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116130.log.gz 2023-10-08T02:32:35.878 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126360.log 2023-10-08T02:32:35.879 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111425.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111425.log.gz 2023-10-08T02:32:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.119874.log 2023-10-08T02:32:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.126360.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.126360.log.gz 2023-10-08T02:32:35.880 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118761.log 2023-10-08T02:32:35.881 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.119874.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.119874.log.gz 2023-10-08T02:32:35.881 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98810.log 2023-10-08T02:32:35.882 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118761.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.118761.log.gz 2023-10-08T02:32:35.882 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136233.log 2023-10-08T02:32:35.883 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98810.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98810.log.gz 2023-10-08T02:32:35.884 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134572.log 2023-10-08T02:32:35.884 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136233.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136233.log.gz 2023-10-08T02:32:35.884 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97453.log 2023-10-08T02:32:35.885 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.134572.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.134572.log.gz 2023-10-08T02:32:35.886 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127315.log 2023-10-08T02:32:35.886 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97453.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97453.log.gz 2023-10-08T02:32:35.886 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122636.log 2023-10-08T02:32:35.887 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127315.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127315.log.gz 2023-10-08T02:32:35.888 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132016.log 2023-10-08T02:32:35.888 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122636.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122636.log.gz 2023-10-08T02:32:35.888 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122846.log 2023-10-08T02:32:35.889 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132016.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132016.log.gz 2023-10-08T02:32:35.889 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141236.log 2023-10-08T02:32:35.890 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122846.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122846.log.gz 2023-10-08T02:32:35.891 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140630.log 2023-10-08T02:32:35.891 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141236.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.141236.log.gz 2023-10-08T02:32:35.892 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116828.log 2023-10-08T02:32:35.892 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140630.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140630.log.gz 2023-10-08T02:32:35.893 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136399.log 2023-10-08T02:32:35.894 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116828.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116828.log.gz 2023-10-08T02:32:35.894 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105309.log 2023-10-08T02:32:35.894 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136399.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136399.log.gz 2023-10-08T02:32:35.895 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144757.log 2023-10-08T02:32:35.895 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105309.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105309.log.gz 2023-10-08T02:32:35.896 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93028.log 2023-10-08T02:32:35.896 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144757.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144757.log.gz 2023-10-08T02:32:35.897 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112626.log 2023-10-08T02:32:35.897 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93028.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93028.log.gz 2023-10-08T02:32:35.898 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120134.log 2023-10-08T02:32:35.898 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112626.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112626.log.gz 2023-10-08T02:32:35.899 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135501.log 2023-10-08T02:32:35.899 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120134.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.120134.log.gz 2023-10-08T02:32:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120924.log 2023-10-08T02:32:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135501.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135501.log.gzgzip 2023-10-08T02:32:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.100919.log 2023-10-08T02:32:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120924.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120924.log.gz 2023-10-08T02:32:35.902 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113738.log 2023-10-08T02:32:35.902 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100919.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100919.log.gz 2023-10-08T02:32:35.903 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113168.log 2023-10-08T02:32:35.903 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113738.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113738.log.gz 2023-10-08T02:32:35.904 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96483.log 2023-10-08T02:32:35.905 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113168.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113168.log.gz 2023-10-08T02:32:35.905 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137100.log 2023-10-08T02:32:35.906 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96483.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96483.log.gz 2023-10-08T02:32:35.906 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131924.log 2023-10-08T02:32:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137100.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137100.log.gz 2023-10-08T02:32:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101035.log 2023-10-08T02:32:35.908 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131924.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.131924.log.gz 2023-10-08T02:32:35.908 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141481.log 2023-10-08T02:32:35.909 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99554.log 2023-10-08T02:32:35.909 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101035.log.gz 2023-10-08T02:32:35.910 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141481.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141481.log.gz 2023-10-08T02:32:35.910 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130592.log 2023-10-08T02:32:35.911 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99554.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.99554.log.gz 2023-10-08T02:32:35.911 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132270.log 2023-10-08T02:32:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130592.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130592.log.gz 2023-10-08T02:32:35.912 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128132.log 2023-10-08T02:32:35.913 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132270.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.132270.log.gz 2023-10-08T02:32:35.913 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142191.log 2023-10-08T02:32:35.914 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128132.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128132.log.gz 2023-10-08T02:32:35.914 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105785.log 2023-10-08T02:32:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142191.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142191.log.gz 2023-10-08T02:32:35.915 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123534.log 2023-10-08T02:32:35.916 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105785.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105785.log.gz 2023-10-08T02:32:35.916 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.98117.log 2023-10-08T02:32:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123534.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.123534.log.gz 2023-10-08T02:32:35.917 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93878.log 2023-10-08T02:32:35.918 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.98117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.98117.log.gz 2023-10-08T02:32:35.918 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105615.log 2023-10-08T02:32:35.919 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93878.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93878.log.gz 2023-10-08T02:32:35.919 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141957.log 2023-10-08T02:32:35.920 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105615.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105615.log.gz 2023-10-08T02:32:35.920 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130821.log 2023-10-08T02:32:35.920 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141957.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141957.log.gz 2023-10-08T02:32:35.921 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136441.log 2023-10-08T02:32:35.922 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130821.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130821.log.gz 2023-10-08T02:32:35.922 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117212.log 2023-10-08T02:32:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136441.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136441.log.gz 2023-10-08T02:32:35.923 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137184.log 2023-10-08T02:32:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117212.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117212.log.gz 2023-10-08T02:32:35.924 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94172.log 2023-10-08T02:32:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137184.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137184.log.gz 2023-10-08T02:32:35.925 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132144.log 2023-10-08T02:32:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94172.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94172.log.gz 2023-10-08T02:32:35.926 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106756.log 2023-10-08T02:32:35.927 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132144.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132144.log.gz 2023-10-08T02:32:35.928 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118719.log 2023-10-08T02:32:35.928 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106756.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106756.log.gz 2023-10-08T02:32:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115257.log 2023-10-08T02:32:35.929 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118719.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118719.log.gz 2023-10-08T02:32:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109533.log 2023-10-08T02:32:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115257.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115257.log.gz 2023-10-08T02:32:35.930 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92703.log 2023-10-08T02:32:35.931 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109533.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109533.log.gz 2023-10-08T02:32:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144253.log 2023-10-08T02:32:35.932 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92703.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92703.log.gz 2023-10-08T02:32:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110078.log 2023-10-08T02:32:35.933 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144253.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144253.log.gz 2023-10-08T02:32:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115299.log 2023-10-08T02:32:35.934 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110078.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110078.log.gz 2023-10-08T02:32:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137619.log 2023-10-08T02:32:35.935 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115299.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115299.log.gz 2023-10-08T02:32:35.936 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96399.log 2023-10-08T02:32:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137619.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137619.log.gz 2023-10-08T02:32:35.937 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140721.log 2023-10-08T02:32:35.938 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96399.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.96399.log.gz 2023-10-08T02:32:35.938 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92709.log 2023-10-08T02:32:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140721.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.140721.log.gz 2023-10-08T02:32:35.939 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142125.log 2023-10-08T02:32:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92709.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92709.log.gz 2023-10-08T02:32:35.940 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144925.log 2023-10-08T02:32:35.941 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123409.log 2023-10-08T02:32:35.941 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.142125.log.gz 2023-10-08T02:32:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144925.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144925.log.gz 2023-10-08T02:32:35.942 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114145.log 2023-10-08T02:32:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123409.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123409.log.gz 2023-10-08T02:32:35.943 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115988.log 2023-10-08T02:32:35.944 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114145.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114145.log.gz 2023-10-08T02:32:35.944 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109595.log 2023-10-08T02:32:35.945 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115988.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115988.log.gz 2023-10-08T02:32:35.945 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128342.log 2023-10-08T02:32:35.946 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109595.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.109595.log.gz 2023-10-08T02:32:35.946 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132102.log 2023-10-08T02:32:35.947 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128342.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128342.log.gz 2023-10-08T02:32:35.947 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143267.log 2023-10-08T02:32:35.948 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132102.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.132102.log.gz 2023-10-08T02:32:35.948 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144715.log 2023-10-08T02:32:35.949 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143267.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143267.log.gz 2023-10-08T02:32:35.949 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129818.log 2023-10-08T02:32:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144715.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.144715.log.gz 2023-10-08T02:32:35.950 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105911.log 2023-10-08T02:32:35.951 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129818.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129818.log.gz 2023-10-08T02:32:35.951 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128174.log 2023-10-08T02:32:35.952 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105911.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105911.log.gz 2023-10-08T02:32:35.952 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135002.log 2023-10-08T02:32:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128174.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128174.log.gz 2023-10-08T02:32:35.953 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101151.log 2023-10-08T02:32:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135002.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135002.log.gz 2023-10-08T02:32:35.954 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95471.log 2023-10-08T02:32:35.955 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101151.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101151.log.gz 2023-10-08T02:32:35.955 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144589.log 2023-10-08T02:32:35.956 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95471.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95471.log.gz 2023-10-08T02:32:35.956 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140586.log 2023-10-08T02:32:35.957 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144589.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144589.log.gz 2023-10-08T02:32:35.957 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112522.log 2023-10-08T02:32:35.958 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140586.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140586.log.gz 2023-10-08T02:32:35.958 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-08T02:32:35.959 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112522.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112522.log.gz 2023-10-08T02:32:35.959 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135655.log 2023-10-08T02:32:35.960 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98987.log 2023-10-08T02:32:35.961 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135655.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135655.log.gz 2023-10-08T02:32:35.961 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95681.log 2023-10-08T02:32:35.963 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98987.log: gzip -5 88.2%--verbose -- replaced with /var/log/ceph/ceph-client.admin.98987.log.gz-- 2023-10-08T02:32:35.963 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103113.log 2023-10-08T02:32:35.963 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95681.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95681.log.gz 2023-10-08T02:32:35.964 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-08T02:32:35.964 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100419.log 2023-10-08T02:32:35.965 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103113.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103113.log.gz 2023-10-08T02:32:35.965 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135263.log 2023-10-08T02:32:35.965 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100419.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100419.log.gz 2023-10-08T02:32:35.965 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110374.log 2023-10-08T02:32:35.966 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135263.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135263.log.gz 2023-10-08T02:32:35.966 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104547.log 2023-10-08T02:32:35.967 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110374.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110374.log.gz 2023-10-08T02:32:35.968 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130483.log 2023-10-08T02:32:35.968 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104547.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104547.log.gz 2023-10-08T02:32:35.968 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108799.log 2023-10-08T02:32:35.969 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130483.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130483.log.gz 2023-10-08T02:32:35.969 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109898.log 2023-10-08T02:32:35.970 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108799.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108799.log.gz 2023-10-08T02:32:35.970 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116275.log 2023-10-08T02:32:35.971 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109898.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109898.log.gz 2023-10-08T02:32:35.971 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127949.log 2023-10-08T02:32:35.972 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116275.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116275.log.gz 2023-10-08T02:32:35.972 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96441.log 2023-10-08T02:32:35.973 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127949.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127949.log.gz 2023-10-08T02:32:35.973 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109365.log 2023-10-08T02:32:35.974 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96441.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96441.log.gz 2023-10-08T02:32:35.975 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95243.log 2023-10-08T02:32:35.975 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109365.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109365.log.gz 2023-10-08T02:32:35.976 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136882.log 2023-10-08T02:32:35.976 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95243.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.95243.log.gz 2023-10-08T02:32:35.977 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140302.log 2023-10-08T02:32:35.977 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136882.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.136882.log.gz 2023-10-08T02:32:35.978 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130359.log 2023-10-08T02:32:35.978 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140302.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140302.log.gz 2023-10-08T02:32:35.978 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108615.log 2023-10-08T02:32:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130359.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.130359.log.gz 2023-10-08T02:32:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144631.log 2023-10-08T02:32:35.980 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108615.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108615.log.gz 2023-10-08T02:32:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118542.log 2023-10-08T02:32:35.981 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144631.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144631.log.gz 2023-10-08T02:32:35.982 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93920.log 2023-10-08T02:32:35.982 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95201.log 2023-10-08T02:32:35.983 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118542.log.gz 2023-10-08T02:32:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93920.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93920.log.gz 2023-10-08T02:32:35.983 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93043.log 2023-10-08T02:32:35.984 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95201.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95201.log.gz 2023-10-08T02:32:35.984 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144211.log 2023-10-08T02:32:35.985 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93043.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93043.log.gz 2023-10-08T02:32:35.986 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98751.log 2023-10-08T02:32:35.986 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95513.log 2023-10-08T02:32:35.987 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144211.log.gz 2023-10-08T02:32:35.987 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98751.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98751.log.gz 2023-10-08T02:32:35.987 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103613.log 2023-10-08T02:32:35.988 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95513.log.gz 2023-10-08T02:32:35.988 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135545.log 2023-10-08T02:32:35.989 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103613.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103613.log.gz 2023-10-08T02:32:35.989 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105267.log 2023-10-08T02:32:35.990 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135545.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135545.log.gz 2023-10-08T02:32:35.990 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122384.log 2023-10-08T02:32:35.991 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105267.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105267.log.gz 2023-10-08T02:32:35.992 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116485.log 2023-10-08T02:32:35.992 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122384.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122384.log.gz 2023-10-08T02:32:35.992 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122468.log 2023-10-08T02:32:35.994 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144673.log 2023-10-08T02:32:35.994 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.116485.log.gz 2023-10-08T02:32:35.994 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122468.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122468.log.gz 2023-10-08T02:32:35.995 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131362.log 2023-10-08T02:32:35.996 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144673.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144673.log.gz 2023-10-08T02:32:35.996 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93042.log 2023-10-08T02:32:35.997 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131362.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131362.log.gz 2023-10-08T02:32:35.997 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108889.log 2023-10-08T02:32:35.998 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93042.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93042.log.gz 2023-10-08T02:32:35.998 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120711.log 2023-10-08T02:32:35.999 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108889.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.108889.log.gz 2023-10-08T02:32:35.999 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93710.log 2023-10-08T02:32:36.000 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145235.log 2023-10-08T02:32:36.002 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120711.log.gz 2023-10-08T02:32:36.002 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93710.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93710.log.gz 2023-10-08T02:32:36.002 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143972.log 2023-10-08T02:32:36.002 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.145235.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145235.log.gz 2023-10-08T02:32:36.002 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97361.log 2023-10-08T02:32:36.003 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143972.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143972.log.gz 2023-10-08T02:32:36.003 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110969.log 2023-10-08T02:32:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97361.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97361.log.gz 2023-10-08T02:32:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132924.log 2023-10-08T02:32:36.004 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110969.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110969.log.gz 2023-10-08T02:32:36.005 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100861.log 2023-10-08T02:32:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132924.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132924.log.gz 2023-10-08T02:32:36.006 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96315.log 2023-10-08T02:32:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100861.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100861.log.gz 2023-10-08T02:32:36.007 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127485.log 2023-10-08T02:32:36.008 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96315.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96315.log.gz 2023-10-08T02:32:36.008 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139997.log 2023-10-08T02:32:36.009 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127485.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127485.log.gz 2023-10-08T02:32:36.009 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136571.log 2023-10-08T02:32:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.139997.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139997.log.gz 2023-10-08T02:32:36.010 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121805.log 2023-10-08T02:32:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136571.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136571.log.gz 2023-10-08T02:32:36.011 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129860.log 2023-10-08T02:32:36.012 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121805.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121805.log.gz 2023-10-08T02:32:36.012 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123080.log 2023-10-08T02:32:36.013 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129860.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129860.log.gz 2023-10-08T02:32:36.013 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122195.log 2023-10-08T02:32:36.014 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123080.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123080.log.gz 2023-10-08T02:32:36.014 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140989.log 2023-10-08T02:32:36.015 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122195.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122195.log.gz 2023-10-08T02:32:36.015 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113336.log 2023-10-08T02:32:36.016 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140989.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140989.log.gz 2023-10-08T02:32:36.016 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102581.log 2023-10-08T02:32:36.017 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113336.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113336.log.gz 2023-10-08T02:32:36.017 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132630.log 2023-10-08T02:32:36.018 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102581.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102581.log.gz 2023-10-08T02:32:36.018 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134066.log 2023-10-08T02:32:36.019 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132630.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.132630.log.gz 2023-10-08T02:32:36.019 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104437.log 2023-10-08T02:32:36.020 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.134066.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.134066.log.gz 2023-10-08T02:32:36.020 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128300.log 2023-10-08T02:32:36.021 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104437.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104437.log.gz 2023-10-08T02:32:36.021 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-08T02:32:36.022 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128300.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.128300.log.gz 2023-10-08T02:32:36.022 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121245.log 2023-10-08T02:32:36.023 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140348.log 2023-10-08T02:32:36.024 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121245.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121245.log.gz 2023-10-08T02:32:36.024 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114584.log 2023-10-08T02:32:36.025 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140348.log: gzip -5 --verbose 89.4%-- -- replaced with /var/log/ceph/ceph-client.admin.140348.log.gz/var/log/ceph/ceph-client.admin.121889.log 2023-10-08T02:32:36.025 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-10-08T02:32:36.026 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114584.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114584.log.gz 2023-10-08T02:32:36.027 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105003.log 2023-10-08T02:32:36.028 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121889.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121889.log.gz 2023-10-08T02:32:36.028 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119069.log 2023-10-08T02:32:36.029 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105003.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105003.log.gz 2023-10-08T02:32:36.029 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115004.log 2023-10-08T02:32:36.030 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119069.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119069.log.gz 2023-10-08T02:32:36.030 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115215.log 2023-10-08T02:32:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115004.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115004.log.gz 2023-10-08T02:32:36.031 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112270.log 2023-10-08T02:32:36.032 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115215.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115215.log.gz 2023-10-08T02:32:36.032 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96852.log 2023-10-08T02:32:36.033 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112270.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112270.log.gz 2023-10-08T02:32:36.033 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100461.log 2023-10-08T02:32:36.034 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96852.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96852.log.gz 2023-10-08T02:32:36.035 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93295.log 2023-10-08T02:32:36.036 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100461.log: gzip 88.9%-5 -- replaced with /var/log/ceph/ceph-client.admin.100461.log.gz 2023-10-08T02:32:36.036 INFO:teuthology.orchestra.run.smithi119.stderr:--verbose -- /var/log/ceph/ceph-client.admin.104393.log 2023-10-08T02:32:36.037 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93295.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93295.log.gz 2023-10-08T02:32:36.037 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93836.log 2023-10-08T02:32:36.037 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104393.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104393.log.gz 2023-10-08T02:32:36.038 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104877.log 2023-10-08T02:32:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93836.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93836.log.gz 2023-10-08T02:32:36.039 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143361.log 2023-10-08T02:32:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104877.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104877.log.gz 2023-10-08T02:32:36.040 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119615.log 2023-10-08T02:32:36.041 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143361.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143361.log.gz 2023-10-08T02:32:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136315.log 2023-10-08T02:32:36.042 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119153.log 2023-10-08T02:32:36.043 INFO:teuthology.orchestra.run.smithi119.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.119615.log.gz 2023-10-08T02:32:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136315.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.136315.log.gz 2023-10-08T02:32:36.044 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128623.log 2023-10-08T02:32:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119153.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119153.log.gz 2023-10-08T02:32:36.045 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132840.log 2023-10-08T02:32:36.046 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128623.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128623.log.gz 2023-10-08T02:32:36.046 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99124.log 2023-10-08T02:32:36.047 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.132840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.132840.log.gz 2023-10-08T02:32:36.047 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112912.log 2023-10-08T02:32:36.048 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99124.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99124.log.gz 2023-10-08T02:32:36.048 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129131.log 2023-10-08T02:32:36.049 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112912.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112912.log.gz 2023-10-08T02:32:36.049 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116551.log 2023-10-08T02:32:36.050 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129131.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129131.log.gz 2023-10-08T02:32:36.051 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143704.log 2023-10-08T02:32:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116551.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.116551.log.gz 2023-10-08T02:32:36.052 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98267.log 2023-10-08T02:32:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143704.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.143704.log.gz 2023-10-08T02:32:36.053 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105143.log 2023-10-08T02:32:36.054 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98267.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98267.log.gz 2023-10-08T02:32:36.054 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113931.log 2023-10-08T02:32:36.055 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105143.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105143.log.gz 2023-10-08T02:32:36.055 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142885.log 2023-10-08T02:32:36.056 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.113931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113931.log.gz 2023-10-08T02:32:36.056 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96979.log 2023-10-08T02:32:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142885.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142885.log.gz 2023-10-08T02:32:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130638.log 2023-10-08T02:32:36.058 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96979.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96979.log.gz 2023-10-08T02:32:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144295.log 2023-10-08T02:32:36.059 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130638.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130638.log.gz 2023-10-08T02:32:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141523.log 2023-10-08T02:32:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144295.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144295.log.gz 2023-10-08T02:32:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94004.log 2023-10-08T02:32:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141523.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.141523.log.gz 2023-10-08T02:32:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97858.log 2023-10-08T02:32:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94004.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94004.log.gz 2023-10-08T02:32:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136840.log 2023-10-08T02:32:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.142384.log 2023-10-08T02:32:36.065 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.97858.log.gz 2023-10-08T02:32:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136840.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136840.log.gz 2023-10-08T02:32:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135403.log 2023-10-08T02:32:36.066 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.142384.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.142384.log.gz 2023-10-08T02:32:36.067 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132522.log 2023-10-08T02:32:36.068 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135403.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135403.log.gz 2023-10-08T02:32:36.068 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132228.log 2023-10-08T02:32:36.069 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141031.log 2023-10-08T02:32:36.069 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.132522.log.gz 2023-10-08T02:32:36.070 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132228.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.132228.log.gz 2023-10-08T02:32:36.070 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117793.log 2023-10-08T02:32:36.071 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141031.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141031.log.gz 2023-10-08T02:32:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115900.log 2023-10-08T02:32:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117793.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117793.log.gz 2023-10-08T02:32:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128707.log 2023-10-08T02:32:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115900.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115900.log.gz 2023-10-08T02:32:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100377.log 2023-10-08T02:32:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128707.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128707.log.gz 2023-10-08T02:32:36.075 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116879.log 2023-10-08T02:32:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100377.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100377.log.gz 2023-10-08T02:32:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113420.log 2023-10-08T02:32:36.077 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116879.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.116879.log.gz-5 2023-10-08T02:32:36.077 INFO:teuthology.orchestra.run.smithi119.stderr:--verbose -- /var/log/ceph/ceph-client.admin.92072.log 2023-10-08T02:32:36.078 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113420.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113420.log.gz 2023-10-08T02:32:36.078 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95807.log 2023-10-08T02:32:36.079 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92072.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92072.log.gz 2023-10-08T02:32:36.079 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144463.log 2023-10-08T02:32:36.080 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95807.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95807.log.gz 2023-10-08T02:32:36.081 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141397.log 2023-10-08T02:32:36.081 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144463.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144463.log.gz 2023-10-08T02:32:36.081 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131152.log 2023-10-08T02:32:36.083 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141397.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141397.log.gz 2023-10-08T02:32:36.083 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102049.log 2023-10-08T02:32:36.084 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131152.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.131152.log.gz 2023-10-08T02:32:36.084 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110206.log 2023-10-08T02:32:36.085 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102049.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102049.log.gz 2023-10-08T02:32:36.085 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144379.log 2023-10-08T02:32:36.086 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110206.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110206.log.gz 2023-10-08T02:32:36.086 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94214.log 2023-10-08T02:32:36.088 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144379.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144379.log.gz 2023-10-08T02:32:36.088 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112786.log 2023-10-08T02:32:36.090 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140488.log 2023-10-08T02:32:36.090 INFO:teuthology.orchestra.run.smithi119.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.94214.log.gz 2023-10-08T02:32:36.090 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112786.log: 89.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.112786.log.gz--verbose 2023-10-08T02:32:36.091 INFO:teuthology.orchestra.run.smithi119.stderr:-- /var/log/ceph/ceph-client.admin.118396.log 2023-10-08T02:32:36.092 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140488.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.140488.log.gz 2023-10-08T02:32:36.092 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136966.log 2023-10-08T02:32:36.092 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118396.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118396.log.gz 2023-10-08T02:32:36.093 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118845.log 2023-10-08T02:32:36.093 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136966.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136966.log.gz 2023-10-08T02:32:36.093 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97317.log 2023-10-08T02:32:36.094 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118845.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118845.log.gz 2023-10-08T02:32:36.094 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108568.log 2023-10-08T02:32:36.096 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97317.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.97317.log.gz 2023-10-08T02:32:36.096 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95933.log 2023-10-08T02:32:36.097 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108568.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108568.log.gz 2023-10-08T02:32:36.097 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129257.log 2023-10-08T02:32:36.098 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95933.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95933.log.gz 2023-10-08T02:32:36.098 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100251.log 2023-10-08T02:32:36.099 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129257.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129257.log.gz 2023-10-08T02:32:36.099 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96771.log 2023-10-08T02:32:36.100 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100251.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100251.log.gz 2023-10-08T02:32:36.100 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97231.log 2023-10-08T02:32:36.101 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96771.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.96771.log.gz 2023-10-08T02:32:36.101 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132312.log 2023-10-08T02:32:36.102 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97231.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97231.log.gz 2023-10-08T02:32:36.102 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119195.log 2023-10-08T02:32:36.103 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132312.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132312.log.gz 2023-10-08T02:32:36.103 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113588.log 2023-10-08T02:32:36.105 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111011.log 2023-10-08T02:32:36.105 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119195.log.gz 2023-10-08T02:32:36.105 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113588.log.gz 2023-10-08T02:32:36.106 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110584.log 2023-10-08T02:32:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111011.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111011.log.gz 2023-10-08T02:32:36.107 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119321.log 2023-10-08T02:32:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110584.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110584.log.gz 2023-10-08T02:32:36.108 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92694.log 2023-10-08T02:32:36.109 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119321.log: gzip 88.2%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.119321.log.gz-- 2023-10-08T02:32:36.109 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110626.log 2023-10-08T02:32:36.110 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92694.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.92694.log.gz 2023-10-08T02:32:36.110 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100335.log 2023-10-08T02:32:36.111 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110626.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110626.log.gz 2023-10-08T02:32:36.111 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100803.log 2023-10-08T02:32:36.112 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96147.log 2023-10-08T02:32:36.112 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100335.log.gz 2023-10-08T02:32:36.113 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104919.log 2023-10-08T02:32:36.114 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100803.log.gz 2023-10-08T02:32:36.114 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96147.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96147.log.gz 2023-10-08T02:32:36.115 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115049.log 2023-10-08T02:32:36.116 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104919.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104919.log.gz 2023-10-08T02:32:36.116 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98869.log 2023-10-08T02:32:36.117 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115049.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115049.log.gz 2023-10-08T02:32:36.117 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135591.log 2023-10-08T02:32:36.118 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98869.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98869.log.gz 2023-10-08T02:32:36.118 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137682.log 2023-10-08T02:32:36.119 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135591.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135591.log.gz 2023-10-08T02:32:36.119 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104961.log 2023-10-08T02:32:36.120 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137682.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.137682.log.gz 2023-10-08T02:32:36.120 INFO:teuthology.orchestra.run.smithi119.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.117836.log 2023-10-08T02:32:36.121 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104961.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104961.log.gz 2023-10-08T02:32:36.122 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137008.log 2023-10-08T02:32:36.123 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117836.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117836.log.gz 2023-10-08T02:32:36.123 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110120.log 2023-10-08T02:32:36.124 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137008.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137008.log.gz 2023-10-08T02:32:36.124 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92408.log 2023-10-08T02:32:36.125 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140947.log 2023-10-08T02:32:36.125 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110120.log.gz 2023-10-08T02:32:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92408.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92408.log.gz 2023-10-08T02:32:36.126 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92156.log 2023-10-08T02:32:36.127 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140947.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140947.log.gz 2023-10-08T02:32:36.127 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112438.log 2023-10-08T02:32:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92156.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92156.log.gz 2023-10-08T02:32:36.128 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117515.log 2023-10-08T02:32:36.130 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112438.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112438.log.gz 2023-10-08T02:32:36.130 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101251.log 2023-10-08T02:32:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117515.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117515.log.gz 2023-10-08T02:32:36.131 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143044.log 2023-10-08T02:32:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101251.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101251.log.gz 2023-10-08T02:32:36.132 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95639.log 2023-10-08T02:32:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143044.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.143044.log.gz 2023-10-08T02:32:36.133 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143453.log 2023-10-08T02:32:36.134 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95639.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95639.log.gz 2023-10-08T02:32:36.134 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129349.log 2023-10-08T02:32:36.135 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143453.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143453.log.gz 2023-10-08T02:32:36.135 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108944.log 2023-10-08T02:32:36.136 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129349.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129349.log.gz 2023-10-08T02:32:36.136 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141439.log 2023-10-08T02:32:36.138 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108944.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108944.log.gz 2023-10-08T02:32:36.138 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135447.log 2023-10-08T02:32:36.138 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141439.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141439.log.gz 2023-10-08T02:32:36.139 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112954.log 2023-10-08T02:32:36.140 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135447.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.135447.log.gz 2023-10-08T02:32:36.141 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142041.log 2023-10-08T02:32:36.141 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112954.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112954.log.gz 2023-10-08T02:32:36.142 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96937.log 2023-10-08T02:32:36.142 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142041.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142041.log.gz 2023-10-08T02:32:36.142 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106271.log 2023-10-08T02:32:36.143 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96937.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96937.log.gz 2023-10-08T02:32:36.143 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95302.log 2023-10-08T02:32:36.144 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106271.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106271.log.gz 2023-10-08T02:32:36.145 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136655.log 2023-10-08T02:32:36.145 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95302.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95302.log.gz 2023-10-08T02:32:36.146 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93626.log 2023-10-08T02:32:36.147 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136655.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.136655.log.gz 2023-10-08T02:32:36.147 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131836.log 2023-10-08T02:32:36.148 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93626.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93626.log.gz 2023-10-08T02:32:36.148 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118620.log 2023-10-08T02:32:36.149 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131836.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131836.log.gz 2023-10-08T02:32:36.149 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111851.log 2023-10-08T02:32:36.150 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118620.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118620.log.gz 2023-10-08T02:32:36.150 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143219.log 2023-10-08T02:32:36.151 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111851.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111851.log.gz 2023-10-08T02:32:36.151 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121595.log 2023-10-08T02:32:36.152 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143219.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143219.log.gz 2023-10-08T02:32:36.152 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110668.log 2023-10-08T02:32:36.154 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121595.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.121595.log.gz 2023-10-08T02:32:36.154 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127669.log 2023-10-08T02:32:36.155 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98928.log 2023-10-08T02:32:36.155 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.110668.log.gz 2023-10-08T02:32:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127669.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127669.log.gz 2023-10-08T02:32:36.156 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113042.log 2023-10-08T02:32:36.157 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98928.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98928.log.gz 2023-10-08T02:32:36.157 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137050.log 2023-10-08T02:32:36.158 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113042.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113042.log.gz 2023-10-08T02:32:36.158 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104835.log 2023-10-08T02:32:36.159 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137050.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137050.log.gz 2023-10-08T02:32:36.159 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100745.log 2023-10-08T02:32:36.160 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122720.log 2023-10-08T02:32:36.161 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104835.log.gz 2023-10-08T02:32:36.161 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100745.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100745.log.gz 2023-10-08T02:32:36.162 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109940.log 2023-10-08T02:32:36.163 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122720.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122720.log.gz 2023-10-08T02:32:36.163 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108444.log 2023-10-08T02:32:36.164 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109940.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.109940.log.gz 2023-10-08T02:32:36.164 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122762.log 2023-10-08T02:32:36.165 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108444.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108444.log.gz 2023-10-08T02:32:36.165 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104346.log 2023-10-08T02:32:36.166 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122762.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122762.log.gz 2023-10-08T02:32:36.166 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-08T02:32:36.167 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104346.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.104346.log.gz 2023-10-08T02:32:36.167 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137569.log 2023-10-08T02:32:36.168 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104709.log 2023-10-08T02:32:36.169 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137569.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137569.log.gz 2023-10-08T02:32:36.170 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111640.log 2023-10-08T02:32:36.171 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105827.log 2023-10-08T02:32:36.171 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104709.log.gz 2023-10-08T02:32:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111640.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111640.log.gz 2023-10-08T02:32:36.172 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100587.log 2023-10-08T02:32:36.174 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129307.log 2023-10-08T02:32:36.174 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105827.log.gz 2023-10-08T02:32:36.175 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100587.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100587.log.gz 2023-10-08T02:32:36.175 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105351.log 2023-10-08T02:32:36.176 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129307.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-08T02:32:36.176 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129307.log.gz 2023-10-08T02:32:36.177 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105351.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105351.log.gz 2023-10-08T02:32:36.178 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95344.log 2023-10-08T02:32:36.179 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101517.log 2023-10-08T02:32:36.180 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95344.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95344.log.gz 2023-10-08T02:32:36.180 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105225.log 2023-10-08T02:32:36.181 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100687.log 2023-10-08T02:32:36.182 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101517.log.gz 2023-10-08T02:32:36.183 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105225.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105225.log.gz 2023-10-08T02:32:36.183 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140087.log 2023-10-08T02:32:36.184 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100687.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100687.log.gz 2023-10-08T02:32:36.185 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136357.log 2023-10-08T02:32:36.185 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140087.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.140087.log.gz 2023-10-08T02:32:36.187 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111694.log 2023-10-08T02:32:36.191 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136357.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136357.log.gz 2023-10-08T02:32:36.191 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96693.log 2023-10-08T02:32:36.191 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111694.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111694.log.gz 2023-10-08T02:32:36.191 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114723.log 2023-10-08T02:32:36.191 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96693.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.96693.log.gz 2023-10-08T02:32:36.191 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112103.log 2023-10-08T02:32:36.192 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114723.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114723.log.gz 2023-10-08T02:32:36.192 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95891.log 2023-10-08T02:32:36.193 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112103.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112103.log.gz 2023-10-08T02:32:36.193 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118270.log 2023-10-08T02:32:36.195 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95891.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95891.log.gz 2023-10-08T02:32:36.195 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106363.log 2023-10-08T02:32:36.196 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118270.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118270.log.gz 2023-10-08T02:32:36.197 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143313.log 2023-10-08T02:32:36.198 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106363.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106363.log.gz 2023-10-08T02:32:36.198 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93033.log 2023-10-08T02:32:36.199 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143313.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.143313.log.gz 2023-10-08T02:32:36.200 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130302.log 2023-10-08T02:32:36.201 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93033.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93033.log.gz 2023-10-08T02:32:36.201 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110500.log 2023-10-08T02:32:36.202 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130302.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130302.log.gz 2023-10-08T02:32:36.203 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142468.log 2023-10-08T02:32:36.204 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110500.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110500.log.gz 2023-10-08T02:32:36.204 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122594.log 2023-10-08T02:32:36.205 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142468.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142468.log.gz 2023-10-08T02:32:36.206 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99339.log 2023-10-08T02:32:36.207 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145153.log 2023-10-08T02:32:36.208 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.122594.log.gz 2023-10-08T02:32:36.208 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99339.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99339.log.gz 2023-10-08T02:32:36.209 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104059.log 2023-10-08T02:32:36.209 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.145153.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.145153.log.gz 2023-10-08T02:32:36.210 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114867.log 2023-10-08T02:32:36.211 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104059.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104059.log.gz 2023-10-08T02:32:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135173.log 2023-10-08T02:32:36.213 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114867.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114867.log.gz 2023-10-08T02:32:36.214 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97189.log 2023-10-08T02:32:36.215 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135173.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135173.log.gz 2023-10-08T02:32:36.216 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105953.log 2023-10-08T02:32:36.216 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97189.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97189.log.gz 2023-10-08T02:32:36.217 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131278.log 2023-10-08T02:32:36.219 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109491.log 2023-10-08T02:32:36.219 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.105953.log.gz 2023-10-08T02:32:36.220 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131278.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131278.log.gz 2023-10-08T02:32:36.220 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129433.log 2023-10-08T02:32:36.221 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109491.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109491.log.gz 2023-10-08T02:32:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96017.log 2023-10-08T02:32:36.222 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129433.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129433.log.gz 2023-10-08T02:32:36.223 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117562.log 2023-10-08T02:32:36.224 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96017.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96017.log.gz 2023-10-08T02:32:36.225 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136613.log 2023-10-08T02:32:36.226 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117562.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117562.log.gz 2023-10-08T02:32:36.226 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122153.log 2023-10-08T02:32:36.227 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136613.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.136613.log.gz 2023-10-08T02:32:36.228 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93381.log 2023-10-08T02:32:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122153.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122153.log.gz 2023-10-08T02:32:36.229 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109814.log 2023-10-08T02:32:36.230 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93381.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93381.log.gz 2023-10-08T02:32:36.231 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105701.log 2023-10-08T02:32:36.232 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109814.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109814.log.gz 2023-10-08T02:32:36.232 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113294.log 2023-10-08T02:32:36.233 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105701.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105701.log.gz 2023-10-08T02:32:36.234 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117308.log 2023-10-08T02:32:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113294.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113294.log.gz 2023-10-08T02:32:36.235 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131974.log 2023-10-08T02:32:36.236 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117308.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117308.log.gz 2023-10-08T02:32:36.237 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118480.log 2023-10-08T02:32:36.238 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131974.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.131974.log.gz 2023-10-08T02:32:36.238 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92114.log 2023-10-08T02:32:36.239 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118480.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118480.log.gz 2023-10-08T02:32:36.240 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92028.log 2023-10-08T02:32:36.241 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92114.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.92114.log.gz 2023-10-08T02:32:36.242 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105659.log 2023-10-08T02:32:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92028.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.92028.log.gz 2023-10-08T02:32:36.243 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133037.log 2023-10-08T02:32:36.244 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105659.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105659.log.gz 2023-10-08T02:32:36.245 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112396.log 2023-10-08T02:32:36.246 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.133037.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.133037.log.gz 2023-10-08T02:32:36.247 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128090.log 2023-10-08T02:32:36.247 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112396.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112396.log.gz 2023-10-08T02:32:36.248 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92198.log 2023-10-08T02:32:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128090.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.128090.log.gz 2023-10-08T02:32:36.249 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109856.log 2023-10-08T02:32:36.250 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92198.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92198.log.gz 2023-10-08T02:32:36.251 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92702.log 2023-10-08T02:32:36.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109856.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109856.log.gz 2023-10-08T02:32:36.252 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95765.log 2023-10-08T02:32:36.253 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92702.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.92702.log.gz 2023-10-08T02:32:36.254 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-08T02:32:36.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95765.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95765.log.gz 2023-10-08T02:32:36.255 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118975.log 2023-10-08T02:32:36.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110734.log 2023-10-08T02:32:36.258 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118975.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.118975.log.gz 2023-10-08T02:32:36.259 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93030.log 2023-10-08T02:32:36.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110734.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.110734.log.gz 2023-10-08T02:32:36.260 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130920.log 2023-10-08T02:32:36.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93030.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145009.log 2023-10-08T02:32:36.262 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93030.log.gz 2023-10-08T02:32:36.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130920.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118312.log 2023-10-08T02:32:36.264 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130920.log.gz 2023-10-08T02:32:36.265 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.145009.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137527.log 2023-10-08T02:32:36.265 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.145009.log.gz 2023-10-08T02:32:36.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118312.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118312.log.gz 2023-10-08T02:32:36.267 INFO:teuthology.orchestra.run.smithi119.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.128524.log 2023-10-08T02:32:36.268 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137527.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.137527.log.gz 2023-10-08T02:32:36.268 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95849.log 2023-10-08T02:32:36.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128524.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128524.log.gz 2023-10-08T02:32:36.270 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128258.log 2023-10-08T02:32:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95849.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.95849.log.gz 2023-10-08T02:32:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119531.log 2023-10-08T02:32:36.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128258.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.128258.log.gz 2023-10-08T02:32:36.274 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115572.log 2023-10-08T02:32:36.275 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119531.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.119531.log.gz 2023-10-08T02:32:36.276 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100293.log 2023-10-08T02:32:36.277 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115572.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115572.log.gz 2023-10-08T02:32:36.278 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97682.log 2023-10-08T02:32:36.278 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100293.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100293.log.gz 2023-10-08T02:32:36.279 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108660.log 2023-10-08T02:32:36.280 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97682.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97682.log.gz 2023-10-08T02:32:36.281 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96525.log 2023-10-08T02:32:36.282 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108660.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.108660.log.gz 2023-10-08T02:32:36.283 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97640.log 2023-10-08T02:32:36.285 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96525.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96525.log.gz 2023-10-08T02:32:36.286 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143846.log 2023-10-08T02:32:36.286 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97640.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97640.log.gz 2023-10-08T02:32:36.287 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116941.log 2023-10-08T02:32:36.288 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143846.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.143846.log.gz 2023-10-08T02:32:36.289 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131236.log 2023-10-08T02:32:36.289 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116941.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116941.log.gz 2023-10-08T02:32:36.290 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92366.log 2023-10-08T02:32:36.291 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131236.log: 93.2% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-08T02:32:36.291 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.131236.log.gz 2023-10-08T02:32:36.293 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93794.log 2023-10-08T02:32:36.293 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92366.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92366.log.gz 2023-10-08T02:32:36.293 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99296.log 2023-10-08T02:32:36.294 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93794.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93794.log.gz 2023-10-08T02:32:36.295 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91934.log 2023-10-08T02:32:36.296 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99296.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99296.log.gz 2023-10-08T02:32:36.297 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92697.log 2023-10-08T02:32:36.298 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106426.log 2023-10-08T02:32:36.298 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.91934.log.gz 2023-10-08T02:32:36.299 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92697.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.92697.log.gz 2023-10-08T02:32:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141787.log 2023-10-08T02:32:36.300 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106426.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106426.log.gz 2023-10-08T02:32:36.301 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117746.log 2023-10-08T02:32:36.302 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141787.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141787.log.gz 2023-10-08T02:32:36.303 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113630.log 2023-10-08T02:32:36.304 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117746.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117746.log.gz 2023-10-08T02:32:36.304 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136009.log 2023-10-08T02:32:36.305 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113630.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113630.log.gz 2023-10-08T02:32:36.306 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116401.log 2023-10-08T02:32:36.307 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115773.log 2023-10-08T02:32:36.307 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.136009.log.gz 2023-10-08T02:32:36.308 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116401.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116401.log.gz 2023-10-08T02:32:36.308 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130731.log 2023-10-08T02:32:36.309 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115773.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115773.log.gz 2023-10-08T02:32:36.310 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140396.log 2023-10-08T02:32:36.311 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130731.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130731.log.gz 2023-10-08T02:32:36.312 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128384.log 2023-10-08T02:32:36.313 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140396.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.140396.log.gz 2023-10-08T02:32:36.313 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135312.log 2023-10-08T02:32:36.314 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128384.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128384.log.gz 2023-10-08T02:32:36.315 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120072.log 2023-10-08T02:32:36.316 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115731.log 2023-10-08T02:32:36.316 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135312.log.gz 2023-10-08T02:32:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120072.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120072.log.gz 2023-10-08T02:32:36.317 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106229.log 2023-10-08T02:32:36.318 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115731.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115731.log.gz 2023-10-08T02:32:36.319 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93045.log 2023-10-08T02:32:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.106229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106229.log.gz 2023-10-08T02:32:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131880.log 2023-10-08T02:32:36.321 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93045.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93045.log.gz 2023-10-08T02:32:36.322 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108989.log 2023-10-08T02:32:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131880.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131880.log.gz 2023-10-08T02:32:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116744.log 2023-10-08T02:32:36.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108989.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108989.log.gz 2023-10-08T02:32:36.325 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120359.log 2023-10-08T02:32:36.327 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.116744.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.116744.log.gz 2023-10-08T02:32:36.327 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114015.log 2023-10-08T02:32:36.328 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120359.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120359.log.gz 2023-10-08T02:32:36.328 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118044.log 2023-10-08T02:32:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114015.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114015.log.gz 2023-10-08T02:32:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92701.log 2023-10-08T02:32:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118044.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118044.log.gz 2023-10-08T02:32:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136790.log 2023-10-08T02:32:36.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92701.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92701.log.gz 2023-10-08T02:32:36.332 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130966.log 2023-10-08T02:32:36.333 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136790.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136790.log.gz 2023-10-08T02:32:36.334 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144337.log 2023-10-08T02:32:36.335 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122029.log 2023-10-08T02:32:36.335 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.130966.log.gz 2023-10-08T02:32:36.337 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144337.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144337.log.gz 2023-10-08T02:32:36.337 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101209.log 2023-10-08T02:32:36.338 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122029.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122029.log.gz 2023-10-08T02:32:36.338 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117891.log 2023-10-08T02:32:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118931.log 2023-10-08T02:32:36.340 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101209.log.gz 2023-10-08T02:32:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117891.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117891.log.gz 2023-10-08T02:32:36.341 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93379.log 2023-10-08T02:32:36.342 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118931.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118931.log.gz 2023-10-08T02:32:36.343 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96609.log 2023-10-08T02:32:36.344 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93379.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93379.log.gz 2023-10-08T02:32:36.344 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130035.log 2023-10-08T02:32:36.345 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96609.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96609.log.gz 2023-10-08T02:32:36.346 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127904.log 2023-10-08T02:32:36.347 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132058.log 2023-10-08T02:32:36.347 INFO:teuthology.orchestra.run.smithi119.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.130035.log.gz 2023-10-08T02:32:36.348 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127904.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.127904.log.gz 2023-10-08T02:32:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112480.log 2023-10-08T02:32:36.349 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132058.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132058.log.gz 2023-10-08T02:32:36.350 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142512.log 2023-10-08T02:32:36.351 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112480.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112480.log.gz 2023-10-08T02:32:36.352 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113252.log 2023-10-08T02:32:36.353 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142512.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142512.log.gz 2023-10-08T02:32:36.353 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118228.log 2023-10-08T02:32:36.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113252.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113252.log.gz 2023-10-08T02:32:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129973.log 2023-10-08T02:32:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118228.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118228.log.gz 2023-10-08T02:32:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136924.log 2023-10-08T02:32:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129973.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129973.log.gz 2023-10-08T02:32:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98692.log 2023-10-08T02:32:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136924.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.136924.log.gz 2023-10-08T02:32:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113126.log 2023-10-08T02:32:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98692.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98692.log.gz 2023-10-08T02:32:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101093.log 2023-10-08T02:32:36.362 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99382.log 2023-10-08T02:32:36.363 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113126.log.gz 2023-10-08T02:32:36.363 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101093.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101093.log.gz 2023-10-08T02:32:36.364 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105437.log 2023-10-08T02:32:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99382.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99382.log.gz 2023-10-08T02:32:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99511.log 2023-10-08T02:32:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105437.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105437.log.gz 2023-10-08T02:32:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.137485.log 2023-10-08T02:32:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99511.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99511.log.gz 2023-10-08T02:32:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131710.log 2023-10-08T02:32:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.137485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.137485.log.gz 2023-10-08T02:32:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144505.log 2023-10-08T02:32:36.371 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131710.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131710.log.gz 2023-10-08T02:32:36.371 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85918.log 2023-10-08T02:32:36.372 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144505.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144505.log.gz 2023-10-08T02:32:36.373 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141174.log 2023-10-08T02:32:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.85918.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85918.log.gz 2023-10-08T02:32:36.374 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143930.log 2023-10-08T02:32:36.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141174.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141174.log.gz 2023-10-08T02:32:36.376 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134630.log 2023-10-08T02:32:36.377 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143930.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143930.log.gz 2023-10-08T02:32:36.378 INFO:teuthology.orchestra.run.smithi119.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-10-08T02:32:36.378 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96357.log 2023-10-08T02:32:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.134630.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.134630.log.gz 2023-10-08T02:32:36.379 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122972.log 2023-10-08T02:32:36.380 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96357.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96357.log.gz 2023-10-08T02:32:36.382 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129583.log 2023-10-08T02:32:36.383 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122972.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.122972.log.gz 2023-10-08T02:32:36.383 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130876.log 2023-10-08T02:32:36.384 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129583.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129583.log.gz 2023-10-08T02:32:36.385 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141073.log 2023-10-08T02:32:36.386 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137743.log 2023-10-08T02:32:36.386 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130876.log.gz 2023-10-08T02:32:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141073.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141073.log.gz 2023-10-08T02:32:36.387 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100629.log 2023-10-08T02:32:36.388 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137743.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.137743.log.gz 2023-10-08T02:32:36.389 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131011.log 2023-10-08T02:32:36.390 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100629.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100629.log.gz 2023-10-08T02:32:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123014.log 2023-10-08T02:32:36.391 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131011.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131011.log.gz 2023-10-08T02:32:36.392 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142642.log 2023-10-08T02:32:36.393 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123359.log 2023-10-08T02:32:36.394 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142642.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.123014.log.gz 2023-10-08T02:32:36.394 INFO:teuthology.orchestra.run.smithi119.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.142642.log.gz 2023-10-08T02:32:36.395 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131752.log 2023-10-08T02:32:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123359.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.123359.log.gz 2023-10-08T02:32:36.396 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98392.log 2023-10-08T02:32:36.398 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142581.log 2023-10-08T02:32:36.398 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131752.log.gz 2023-10-08T02:32:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98392.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.98392.log.gz 2023-10-08T02:32:36.399 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93380.log 2023-10-08T02:32:36.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142581.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142581.log.gz 2023-10-08T02:32:36.401 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128921.log 2023-10-08T02:32:36.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127814.log 2023-10-08T02:32:36.402 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93380.log.gz 2023-10-08T02:32:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128921.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128921.log.gz 2023-10-08T02:32:36.403 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110416.log 2023-10-08T02:32:36.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127814.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127814.log.gz 2023-10-08T02:32:36.405 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109034.log 2023-10-08T02:32:36.406 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110416.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.110416.log.gz 2023-10-08T02:32:36.407 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95975.log 2023-10-08T02:32:36.408 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109034.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109034.log.gz 2023-10-08T02:32:36.409 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117607.log 2023-10-08T02:32:36.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95975.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95975.log.gz 2023-10-08T02:32:36.410 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102315.log 2023-10-08T02:32:36.411 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122254.log 2023-10-08T02:32:36.411 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117607.log.gz 2023-10-08T02:32:36.412 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102315.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.102315.log.gz 2023-10-08T02:32:36.412 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115341.log 2023-10-08T02:32:36.413 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122254.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122254.log.gz 2023-10-08T02:32:36.414 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143888.log 2023-10-08T02:32:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115341.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.115341.log.gz 2023-10-08T02:32:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105065.log 2023-10-08T02:32:36.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143888.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143888.log.gz 2023-10-08T02:32:36.417 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109673.log 2023-10-08T02:32:36.418 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130686.log 2023-10-08T02:32:36.419 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105065.log.gz 2023-10-08T02:32:36.419 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109673.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109673.log.gz 2023-10-08T02:32:36.420 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92282.log 2023-10-08T02:32:36.421 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130686.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130686.log.gz 2023-10-08T02:32:36.422 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105869.log 2023-10-08T02:32:36.422 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92282.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.92282.log.gz 2023-10-08T02:32:36.423 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142928.log 2023-10-08T02:32:36.424 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105869.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105869.log.gz 2023-10-08T02:32:36.425 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98633.log 2023-10-08T02:32:36.426 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129215.log 2023-10-08T02:32:36.426 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142928.log.gz 2023-10-08T02:32:36.427 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98633.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98633.log.gz 2023-10-08T02:32:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128446.log 2023-10-08T02:32:36.428 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129215.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129215.log.gz 2023-10-08T02:32:36.429 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121453.log 2023-10-08T02:32:36.430 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128446.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.128446.log.gz 2023-10-08T02:32:36.430 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111949.log 2023-10-08T02:32:36.431 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121453.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121453.log.gz 2023-10-08T02:32:36.432 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-08T02:32:36.433 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111949.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111949.log.gz 2023-10-08T02:32:36.433 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116317.log 2023-10-08T02:32:36.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104249.log 2023-10-08T02:32:36.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116317.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.116317.log.gz-5 2023-10-08T02:32:36.437 INFO:teuthology.orchestra.run.smithi119.stderr:--verbose -- /var/log/ceph/ceph-client.admin.141115.log 2023-10-08T02:32:36.438 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104249.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104249.log.gz 2023-10-08T02:32:36.438 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97539.log 2023-10-08T02:32:36.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141115.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.141115.log.gz 2023-10-08T02:32:36.440 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122804.log 2023-10-08T02:32:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97539.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97539.log.gz 2023-10-08T02:32:36.441 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105743.log 2023-10-08T02:32:36.443 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122804.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122804.log.gz 2023-10-08T02:32:36.443 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109281.log 2023-10-08T02:32:36.444 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105743.log: gzip 88.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.105743.log.gz-- 2023-10-08T02:32:36.444 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123317.log 2023-10-08T02:32:36.446 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109281.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109281.log.gz 2023-10-08T02:32:36.446 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131668.log 2023-10-08T02:32:36.447 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123317.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123317.log.gz 2023-10-08T02:32:36.448 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112354.log 2023-10-08T02:32:36.449 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131668.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131668.log.gz 2023-10-08T02:32:36.449 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115426.log 2023-10-08T02:32:36.451 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112354.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112354.log.gz 2023-10-08T02:32:36.451 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85509.log 2023-10-08T02:32:36.452 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115426.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115426.log.gz 2023-10-08T02:32:36.452 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142426.log 2023-10-08T02:32:36.454 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.85509.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.85509.log.gz 2023-10-08T02:32:36.454 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91798.log 2023-10-08T02:32:36.455 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.142426.log: gzip -5 --verbose 89.1%-- -- replaced with /var/log/ceph/ceph-client.admin.142426.log.gz/var/log/ceph/ceph-client.admin.128835.log 2023-10-08T02:32:36.455 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-10-08T02:32:36.457 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130260.log 2023-10-08T02:32:36.457 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91798.log.gz 2023-10-08T02:32:36.458 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128835.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128835.log.gz 2023-10-08T02:32:36.458 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104751.log 2023-10-08T02:32:36.460 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130260.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130260.log.gz 2023-10-08T02:32:36.460 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127858.log 2023-10-08T02:32:36.461 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104751.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104751.log.gz 2023-10-08T02:32:36.462 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145193.log 2023-10-08T02:32:36.463 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127858.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127858.log.gz 2023-10-08T02:32:36.463 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140862.log 2023-10-08T02:32:36.466 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.145193.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145193.log.gz 2023-10-08T02:32:36.466 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93584.log 2023-10-08T02:32:36.466 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140862.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.140862.log.gz 2023-10-08T02:32:36.467 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131508.log 2023-10-08T02:32:36.468 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93584.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93584.log.gz 2023-10-08T02:32:36.469 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121847.log 2023-10-08T02:32:36.470 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131508.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131508.log.gz 2023-10-08T02:32:36.470 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112870.log 2023-10-08T02:32:36.471 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121847.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.121847.log.gz 2023-10-08T02:32:36.472 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127530.log 2023-10-08T02:32:36.473 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112870.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112870.log.gz 2023-10-08T02:32:36.474 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127575.log 2023-10-08T02:32:36.475 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127530.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127530.log.gz 2023-10-08T02:32:36.475 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128216.log 2023-10-08T02:32:36.477 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127575.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127575.log.gz 2023-10-08T02:32:36.477 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135126.log 2023-10-08T02:32:36.478 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128216.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128216.log.gz 2023-10-08T02:32:36.479 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129047.log 2023-10-08T02:32:36.480 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135126.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135126.log.gz 2023-10-08T02:32:36.481 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132480.log 2023-10-08T02:32:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129047.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129047.log.gz 2023-10-08T02:32:36.482 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113462.log 2023-10-08T02:32:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132480.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132480.log.gz 2023-10-08T02:32:36.484 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129089.log 2023-10-08T02:32:36.485 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113462.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119573.log 2023-10-08T02:32:36.486 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113462.log.gz 2023-10-08T02:32:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.129089.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129089.log.gz 2023-10-08T02:32:36.487 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102847.log 2023-10-08T02:32:36.488 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119573.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119573.log.gz 2023-10-08T02:32:36.489 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144547.log 2023-10-08T02:32:36.490 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102847.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102847.log.gz 2023-10-08T02:32:36.490 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110028.log 2023-10-08T02:32:36.491 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144547.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.144547.log.gz 2023-10-08T02:32:36.492 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96651.log 2023-10-08T02:32:36.493 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110028.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110028.log.gz 2023-10-08T02:32:36.494 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132396.log 2023-10-08T02:32:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96651.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96651.log.gz 2023-10-08T02:32:36.495 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96273.log 2023-10-08T02:32:36.496 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132396.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132396.log.gz 2023-10-08T02:32:36.497 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136155.log 2023-10-08T02:32:36.498 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96273.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96273.log.gz 2023-10-08T02:32:36.499 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116986.log 2023-10-08T02:32:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136155.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136155.log.gz 2023-10-08T02:32:36.500 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97273.log 2023-10-08T02:32:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116986.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116986.log.gz 2023-10-08T02:32:36.502 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118887.log 2023-10-08T02:32:36.504 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97273.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97273.log.gz 2023-10-08T02:32:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93368.log 2023-10-08T02:32:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118887.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118887.log.gz 2023-10-08T02:32:36.506 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118803.log 2023-10-08T02:32:36.507 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93368.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93368.log.gz 2023-10-08T02:32:36.507 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122678.log 2023-10-08T02:32:36.509 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118803.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.118803.log.gz 2023-10-08T02:32:36.509 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113378.log 2023-10-08T02:32:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122678.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.122678.log.gz 2023-10-08T02:32:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-08T02:32:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113378.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113378.log.gz 2023-10-08T02:32:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134024.log 2023-10-08T02:32:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128749.log 2023-10-08T02:32:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.134024.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134024.log.gz 2023-10-08T02:32:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113504.log 2023-10-08T02:32:36.517 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.128749.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128749.log.gz 2023-10-08T02:32:36.518 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99062.log 2023-10-08T02:32:36.519 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113504.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113504.log.gz 2023-10-08T02:32:36.520 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144056.log 2023-10-08T02:32:36.521 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99062.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99062.log.gz 2023-10-08T02:32:36.522 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105481.log 2023-10-08T02:32:36.523 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144056.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144056.log.gz 2023-10-08T02:32:36.524 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.79530.log 2023-10-08T02:32:36.525 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105481.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105481.log.gz 2023-10-08T02:32:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123759.log 2023-10-08T02:32:36.526 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.tmp-client.admin.79530.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.79530.log.gz 2023-10-08T02:32:36.527 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119025.log 2023-10-08T02:32:36.528 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123759.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123759.log.gz 2023-10-08T02:32:36.529 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116359.log 2023-10-08T02:32:36.530 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119025.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119025.log.gz 2023-10-08T02:32:36.531 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118354.log 2023-10-08T02:32:36.532 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116359.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116359.log.gz 2023-10-08T02:32:36.533 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118186.log 2023-10-08T02:32:36.534 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118354.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118354.log.gz 2023-10-08T02:32:36.535 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137142.log 2023-10-08T02:32:36.536 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118186.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118186.log.gz 2023-10-08T02:32:36.536 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104198.log 2023-10-08T02:32:36.537 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.137142.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.137142.log.gz 2023-10-08T02:32:36.538 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99468.log 2023-10-08T02:32:36.539 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104198.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.104198.log.gz 2023-10-08T02:32:36.540 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143406.log 2023-10-08T02:32:36.541 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99468.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99468.log.gz 2023-10-08T02:32:36.542 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121763.log 2023-10-08T02:32:36.543 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143406.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143406.log.gz 2023-10-08T02:32:36.544 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98159.log 2023-10-08T02:32:36.545 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121763.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.121763.log.gz 2023-10-08T02:32:36.546 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91574.log 2023-10-08T02:32:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98159.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98159.log.gz 2023-10-08T02:32:36.547 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114537.log 2023-10-08T02:32:36.548 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91574.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.91574.log.gz 2023-10-08T02:32:36.549 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114815.log 2023-10-08T02:32:36.550 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114537.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114537.log.gz 2023-10-08T02:32:36.551 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136527.log 2023-10-08T02:32:36.552 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114815.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114815.log.gz 2023-10-08T02:32:36.553 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121110.log 2023-10-08T02:32:36.554 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.136527.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136527.log.gz 2023-10-08T02:32:36.555 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112039.log 2023-10-08T02:32:36.556 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121110.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121110.log.gz 2023-10-08T02:32:36.556 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113546.log 2023-10-08T02:32:36.558 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112039.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112039.log.gz 2023-10-08T02:32:36.558 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106713.log 2023-10-08T02:32:36.559 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113546.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113546.log.gz 2023-10-08T02:32:36.560 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143641.log 2023-10-08T02:32:36.562 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106713.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106713.log.gz 2023-10-08T02:32:36.562 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130547.log 2023-10-08T02:32:36.563 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143641.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143641.log.gz 2023-10-08T02:32:36.563 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132564.log 2023-10-08T02:32:36.565 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.130547.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.130547.log.gz 2023-10-08T02:32:36.565 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104012.log 2023-10-08T02:32:36.567 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132564.log: gzip 91.7% -- replaced with /var/log/ceph/ceph-client.admin.132564.log.gz 2023-10-08T02:32:36.567 INFO:teuthology.orchestra.run.smithi119.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.114960.log 2023-10-08T02:32:36.568 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104012.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104012.log.gz 2023-10-08T02:32:36.569 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103363.log 2023-10-08T02:32:36.570 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114960.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114960.log.gz 2023-10-08T02:32:36.571 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92450.log 2023-10-08T02:32:36.572 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103363.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103363.log.gz 2023-10-08T02:32:36.573 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97724.log 2023-10-08T02:32:36.573 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92450.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.92450.log.gz 2023-10-08T02:32:36.574 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117981.log 2023-10-08T02:32:36.576 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97724.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97724.log.gz 2023-10-08T02:32:36.576 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93037.log 2023-10-08T02:32:36.577 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117981.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117981.log.gz 2023-10-08T02:32:36.577 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121344.log 2023-10-08T02:32:36.578 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93037.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93037.log.gz 2023-10-08T02:32:36.579 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94046.log 2023-10-08T02:32:36.580 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121344.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.121344.log.gz 2023-10-08T02:32:36.581 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116443.log 2023-10-08T02:32:36.582 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94046.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94046.log.gz 2023-10-08T02:32:36.583 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117701.log 2023-10-08T02:32:36.584 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116443.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116443.log.gz 2023-10-08T02:32:36.585 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111903.log 2023-10-08T02:32:36.586 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117701.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117701.log.gz 2023-10-08T02:32:36.587 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106313.log 2023-10-08T02:32:36.587 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111903.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111903.log.gz 2023-10-08T02:32:36.588 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129776.log 2023-10-08T02:32:36.589 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106313.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106313.log.gz 2023-10-08T02:32:36.590 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97063.log 2023-10-08T02:32:36.591 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.129776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.129776.log.gz 2023-10-08T02:32:36.592 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112704.log 2023-10-08T02:32:36.593 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97063.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97063.log.gz 2023-10-08T02:32:36.594 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127438.log 2023-10-08T02:32:36.595 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112704.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112704.log.gz 2023-10-08T02:32:36.596 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93044.log 2023-10-08T02:32:36.598 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.127438.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127438.log.gz 2023-10-08T02:32:36.598 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126318.log 2023-10-08T02:32:36.599 INFO:teuthology.orchestra.run.smithi184.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-08T02:32:36.600 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93044.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93044.log.gz 2023-10-08T02:32:36.600 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98314.log 2023-10-08T02:32:36.601 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.126318.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.126318.log.gz 2023-10-08T02:32:36.601 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109449.log 2023-10-08T02:32:36.602 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98314.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98314.log.gz 2023-10-08T02:32:36.603 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106019.log 2023-10-08T02:32:36.605 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109449.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.109449.log.gz 2023-10-08T02:32:36.605 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120801.log 2023-10-08T02:32:36.606 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106019.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106019.log.gz 2023-10-08T02:32:36.607 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123472.log 2023-10-08T02:32:36.608 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120801.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.120801.log.gz 2023-10-08T02:32:36.608 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143595.log 2023-10-08T02:32:36.609 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.123472.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123472.log.gz 2023-10-08T02:32:36.610 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94348.log 2023-10-08T02:32:36.612 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.143595.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143595.log.gz 2023-10-08T02:32:36.612 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141999.log 2023-10-08T02:32:36.613 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94348.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94348.log.gz 2023-10-08T02:32:36.614 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99167.log 2023-10-08T02:32:36.615 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141999.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.141999.log.gz 2023-10-08T02:32:36.616 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121390.log 2023-10-08T02:32:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99167.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99167.log.gz 2023-10-08T02:32:36.617 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110927.log 2023-10-08T02:32:36.619 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121390.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121390.log.gz 2023-10-08T02:32:36.619 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119111.log 2023-10-08T02:32:36.620 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.110927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110927.log.gz 2023-10-08T02:32:36.621 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95723.log 2023-10-08T02:32:36.622 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119111.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119111.log.gz 2023-10-08T02:32:36.623 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91748.log 2023-10-08T02:32:36.624 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95723.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95723.log.gz 2023-10-08T02:32:36.625 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99210.log 2023-10-08T02:32:36.626 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.91748.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.91748.log.gz 2023-10-08T02:32:36.627 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109772.log 2023-10-08T02:32:36.628 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99210.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99210.log.gz 2023-10-08T02:32:36.629 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92534.log 2023-10-08T02:32:36.631 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109772.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109772.log.gz 2023-10-08T02:32:36.632 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120971.log 2023-10-08T02:32:36.632 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92534.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92534.log.gz 2023-10-08T02:32:36.632 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141831.log 2023-10-08T02:32:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120971.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120971.log.gz 2023-10-08T02:32:36.634 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119405.log 2023-10-08T02:32:36.635 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141831.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.141831.log.gz 2023-10-08T02:32:36.636 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144967.log 2023-10-08T02:32:36.638 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119405.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119405.log.gz 2023-10-08T02:32:36.638 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110162.log 2023-10-08T02:32:36.639 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144967.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.144967.log.gz 2023-10-08T02:32:36.639 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111787.log 2023-10-08T02:32:36.641 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110162.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110162.log.gz 2023-10-08T02:32:36.641 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140676.log 2023-10-08T02:32:36.642 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111787.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111787.log.gz 2023-10-08T02:32:36.642 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122510.log 2023-10-08T02:32:36.644 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140676.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.140676.log.gz 2023-10-08T02:32:36.645 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144799.log 2023-10-08T02:32:36.645 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.122510.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122510.log.gz 2023-10-08T02:32:36.646 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140257.log 2023-10-08T02:32:36.647 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.144799.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144799.log.gz 2023-10-08T02:32:36.648 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141653.log 2023-10-08T02:32:36.649 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.140257.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.140257.log.gz 2023-10-08T02:32:36.650 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132438.log 2023-10-08T02:32:36.651 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.141653.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141653.log.gz 2023-10-08T02:32:36.652 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-08T02:32:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.132438.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132438.log.gz 2023-10-08T02:32:36.653 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113084.log 2023-10-08T02:32:36.655 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135218.log 2023-10-08T02:32:36.657 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113084.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113084.log.gz 2023-10-08T02:32:36.657 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98201.log 2023-10-08T02:32:36.659 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.135218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117653.log 2023-10-08T02:32:36.660 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98201.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.135218.log.gz 2023-10-08T02:32:36.660 INFO:teuthology.orchestra.run.smithi119.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.98201.log.gz 2023-10-08T02:32:36.661 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116038.log 2023-10-08T02:32:36.662 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117653.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117653.log.gz 2023-10-08T02:32:36.663 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105573.log 2023-10-08T02:32:36.665 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99425.log 2023-10-08T02:32:36.666 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105573.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105573.log.gz 89.0% 2023-10-08T02:32:36.666 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116038.log.gz 2023-10-08T02:32:36.667 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131404.log 2023-10-08T02:32:36.669 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99425.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99425.log.gz 2023-10-08T02:32:36.669 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94088.log 2023-10-08T02:32:36.671 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.131404.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.131404.log.gz 2023-10-08T02:32:36.672 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94088.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94088.log.gz 2023-10-08T02:32:36.946 INFO:teuthology.orchestra.run.smithi184.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-08T02:32:37.404 INFO:teuthology.orchestra.run.smithi184.stderr: 92.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-08T02:32:38.443 INFO:teuthology.orchestra.run.smithi184.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-08T02:32:38.496 INFO:teuthology.orchestra.run.smithi184.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-08T02:32:38.593 INFO:teuthology.orchestra.run.smithi119.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-08T02:32:38.637 INFO:teuthology.orchestra.run.smithi184.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-08T02:32:38.744 INFO:teuthology.orchestra.run.smithi119.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-08T02:32:38.876 INFO:teuthology.orchestra.run.smithi184.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-08T02:32:38.879 INFO:teuthology.orchestra.run.smithi184.stderr: 2023-10-08T02:32:38.879 INFO:teuthology.orchestra.run.smithi184.stderr:real 0m3.221s 2023-10-08T02:32:38.879 INFO:teuthology.orchestra.run.smithi184.stderr:user 0m14.375s 2023-10-08T02:32:38.879 INFO:teuthology.orchestra.run.smithi184.stderr:sys 0m1.343s 2023-10-08T02:32:39.564 INFO:teuthology.orchestra.run.smithi119.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-08T02:32:39.568 INFO:teuthology.orchestra.run.smithi119.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-08T02:32:39.898 INFO:teuthology.orchestra.run.smithi119.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-08T02:32:39.990 INFO:teuthology.orchestra.run.smithi119.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-08T02:32:39.993 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-10-08T02:32:39.993 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m4.340s 2023-10-08T02:32:39.994 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m17.757s 2023-10-08T02:32:39.994 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m3.325s 2023-10-08T02:32:39.994 INFO:tasks.ceph:Archiving logs... 2023-10-08T02:32:39.995 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/remote/smithi119/log 2023-10-08T02:32:39.996 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T02:32:42.902 DEBUG:teuthology.misc:Transferring archived files from smithi184:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/remote/smithi184/log 2023-10-08T02:32:42.903 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-08T02:32:44.780 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-08T02:32:44.789 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-08T02:32:44.789 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T02:32:44.794 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-08T02:32:44.907 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-08T02:32:44.917 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-08T02:32:44.917 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T02:32:44.922 DEBUG:teuthology.orchestra.run.smithi184:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-08T02:32:44.953 INFO:teuthology.orchestra.run.smithi119.stderr:bash: ntpq: command not found 2023-10-08T02:32:44.956 INFO:teuthology.orchestra.run.smithi184.stderr:bash: ntpq: command not found 2023-10-08T02:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout:210 Number of sources = 4 2023-10-08T02:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T02:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-10-08T02:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 128 -335us[ -320us] +/- 82ms 2023-10-08T02:32:44.959 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv02.front.sepia.ceph.com 3 7 377 120 -283us[ -268us] +/- 61ms 2023-10-08T02:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 128 +618us[ +633us] +/- 93ms 2023-10-08T02:32:44.960 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T02:32:44.961 INFO:teuthology.orchestra.run.smithi184.stdout:210 Number of sources = 4 2023-10-08T02:32:44.962 INFO:teuthology.orchestra.run.smithi184.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-08T02:32:44.962 INFO:teuthology.orchestra.run.smithi184.stdout:=============================================================================== 2023-10-08T02:32:44.962 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 72 -279us[ -261us] +/- 83ms 2023-10-08T02:32:44.962 INFO:teuthology.orchestra.run.smithi184.stdout:^* hv02.front.sepia.ceph.com 3 8 377 1 -245us[ -227us] +/- 63ms 2023-10-08T02:32:44.962 INFO:teuthology.orchestra.run.smithi184.stdout:^+ hv03.front.sepia.ceph.com 3 8 377 1 +671us[ +671us] +/- 63ms 2023-10-08T02:32:44.962 INFO:teuthology.orchestra.run.smithi184.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-10-08T02:32:44.963 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-08T02:32:44.972 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-08T02:32:44.973 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-08T02:32:44.982 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T02:32:45.103 DEBUG:teuthology.orchestra.run.smithi184:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-08T02:32:45.266 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T02:32:45.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T02:32:45.329 DEBUG:teuthology.orchestra.run.smithi184:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-08T02:32:45.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T02:32:45.396 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-08T02:32:45.407 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-08T02:32:45.425 INFO:teuthology.task.internal:Duration was 6701.615927 seconds 2023-10-08T02:32:45.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-08T02:32:45.436 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-08T02:32:45.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T02:32:45.441 DEBUG:teuthology.orchestra.run.smithi184:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-08T02:32:45.545 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T02:32:45.550 INFO:teuthology.orchestra.run.smithi184.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-08T02:32:45.815 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-08T02:32:45.815 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2023-10-08T02:32:45.816 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T02:32:45.866 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi184.front.sepia.ceph.com 2023-10-08T02:32:45.866 DEBUG:teuthology.orchestra.run.smithi184:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-08T02:32:45.917 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-08T02:32:45.917 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T02:32:45.922 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-08T02:32:46.003 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-08T02:32:46.003 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T02:32:46.007 DEBUG:teuthology.orchestra.run.smithi184:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-08T02:32:46.145 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-08T02:32:46.155 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-08T02:32:46.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T02:32:46.217 DEBUG:teuthology.orchestra.run.smithi184:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-08T02:32:46.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-08T02:32:46.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T02:32:46.294 DEBUG:teuthology.orchestra.run.smithi184:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-08T02:32:46.350 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2023-10-08T02:32:46.354 INFO:teuthology.orchestra.run.smithi184.stdout:kernel.core_pattern = core 2023-10-08T02:32:46.389 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T02:32:46.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T02:32:46.424 DEBUG:teuthology.orchestra.run.smithi184:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-08T02:32:46.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-08T02:32:46.458 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-08T02:32:46.477 INFO:teuthology.task.internal:Transferring archived files... 2023-10-08T02:32:46.478 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/remote/smithi119 2023-10-08T02:32:46.478 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T02:32:46.550 DEBUG:teuthology.misc:Transferring archived files from smithi184:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151/remote/smithi184 2023-10-08T02:32:46.550 DEBUG:teuthology.orchestra.run.smithi184:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-08T02:32:46.621 INFO:teuthology.task.internal:Removing archive directory... 2023-10-08T02:32:46.622 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T02:32:46.626 DEBUG:teuthology.orchestra.run.smithi184:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-08T02:32:46.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-08T02:32:46.677 INFO:teuthology.task.internal:Not uploading archives. 2023-10-08T02:32:46.677 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-08T02:32:46.688 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-08T02:32:46.688 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T02:32:46.702 DEBUG:teuthology.orchestra.run.smithi184:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-08T02:32:46.733 INFO:teuthology.orchestra.run.smithi119.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 02:32 /home/ubuntu/cephtest 2023-10-08T02:32:46.738 INFO:teuthology.orchestra.run.smithi184.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 8 02:32 /home/ubuntu/cephtest 2023-10-08T02:32:46.741 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-08T02:32:46.752 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-08T02:32:46.773 INFO:teuthology.nuke:Checking targets against current locks 2023-10-08T02:32:46.800 DEBUG:teuthology.nuke:shortname: smithi119 2023-10-08T02:32:46.801 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T02:32:46.823 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 00:32:16.316855', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB0cPAV/RG2BkvrxzZHnNdQyZ/3qQ2le4eTUMsncY32AFChDBT4VBnZ39y8jKIMU159V4Mr8ZGHz43X4cptSBhw='} 2023-10-08T02:32:46.827 DEBUG:teuthology.nuke:shortname: smithi184 2023-10-08T02:32:46.828 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-08T02:32:46.848 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi184.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-07_22:15:20-fs-wip-yuri-testing-2023-10-06-0949-quincy-distro-default-smithi/7418151', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-08 00:32:16.318299', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHKxQN3H142f/b14dRMLiTxs+qYVY/Qjhf0KIIvtk8nU49zt4ybtEXjzkUV3+nkQpiaJ0GXj9y8cgd6QJbyB3X8='} 2023-10-08T02:32:46.877 INFO:teuthology.orchestra.console.smithi119:Power off 2023-10-08T02:32:46.877 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T02:32:46.903 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2023-10-08T02:32:46.903 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T02:32:46.918 INFO:teuthology.orchestra.console.smithi184:Power off 2023-10-08T02:32:46.918 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-08T02:32:46.929 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-10-08T02:32:46.951 DEBUG:teuthology.orchestra.console.smithi184:power off output: Chassis Power Control: Down/Off 2023-10-08T02:32:46.951 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T02:32:46.975 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-10-08T02:32:50.931 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T02:32:50.976 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T02:32:51.041 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-10-08T02:32:51.085 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is on 2023-10-08T02:32:55.042 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T02:32:55.086 DEBUG:teuthology.orchestra.console.smithi184:pexpect command: ipmitool -H smithi184.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-08T02:32:55.153 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2023-10-08T02:32:55.197 DEBUG:teuthology.orchestra.console.smithi184:check power output: Chassis Power is off 2023-10-08T02:32:55.254 INFO:teuthology.orchestra.console.smithi119:Power off completed 2023-10-08T02:32:55.298 INFO:teuthology.orchestra.console.smithi184:Power off completed 2023-10-08T02:32:55.402 INFO:teuthology.run:Summary data: client.0-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c client.1-kernel-sha1: fef64f93da2bbe59bfa57cc5e0530b4dcd2f9c7c description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6701.615926980972 failure_reason: '"1696731227.3553238 mon.a (mon.0) 1800 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-08T02:32:55.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-08T02:32:55.478 INFO:teuthology.run:FAIL